PowerShell: Copy/Move Files based on a regex value, retaining the folder structure, etc

Question

Here's the scenario: We have a production web server that has a few thousand files and folders that have an "_DATE" attached to them. We want to move them to a temporary folder (make sure they aren't in use), and at a later date delete the files.

I can use:

Get-ChildItem -Path W:\IIS -Recurse | Where-Object{$_.Name -match "_\d{8,10}$"}

To get a list of all the files/folders and their locations. But Manually deleting them all seems like a lot of work, particularly if this is needed in the future. I have found a few examples that almost do what I want:

cls

$source = "W:\IIS"
$destination = "C:\Temp\DevTest" 

foreach ($i in Get-ChildItem -Path $source -Recurse)
{
    if ($i.Name -match "_\d{8,10}$")
    {
        Copy-Item -Path $i.FullName -Destination $item.FullName.ToString().Replace($source,$destination).Trim($item.Name)
    }
}

And:

cls

$source = "W:\IIS"
$destination = "C:\Temp\DevTest" 

$bin = Get-ChildItem -Path $source -Recurse | Where-Object{$_.Name -match "_\d{8,10}$"}

foreach ($item in $bin) {
    Copy-Item -Path $item.FullName -Container -Destination $item.FullName.ToString().Replace($source,$destination).Trim($item.Name) -Recurse
}

The issue with these two are that when I test them with just copy-item I end up with a flat directory, and I need to preserve the directory structure so that if the move goes wrong I can revert (preferably by drag and dropping all the folders back into the IIS folder) or I get a copy of a lot of extra folders/files that do not appear when I run the first command.

Modifying my first command to:

Get-ChildItem -Path W:\IIS -Recurse | Where-Object{$_.Name -match "_\d{8,10}$"} | Copy-Item -Container -Destination C:\Temp\DevTest -Recurse

Will copy everything I need, but with a flat directory tree, rather than retaining the tree structure (but substituting the source directory with the destination).

Any suggestions/comments?

Solution 1

You can play around with the following, which seems to get the job done (though it can use some refactoring). The core is ProcessFolder(…) which gets called recursively.

function Log
{
    param($Error)
    if(!$script:log)
    {
        $script:log = join-path $dst "log$timestamp.txt"
    }
    $timestamp
    $timestamp >> $script:log
    $Error
    $Error >> $script:log
}

function CopyFile
{
    param($Path)
    $tmp = join-path $dst $Path.Substring($src.Length)
    write-host "File src: $Path"
    write-host "File dst: $tmp"
    Try
    {
        #copy the file
        copy $Path $tmp -Force
    }
    Catch
    {
        Log "ERROR copying file $Path to $tmp`:`
        $_"
    }
}

function ProcessFolder
{
    param($Path)
    $tmp = join-path $dst $Path.Substring($src.Length)
    write-host "Dir. src: $Path"
    write-host "Dir. dst: $tmp"
    Try
    {
        #create target directory
        New-Item $tmp -itemtype directory -force > $null
        #process files if they match
        dir $Path | ?{!$_.PsIsContainer -and $_.Name -match "_\d{8,10}$" } | sort | %{ CopyFile $_.FullName }
        #process subdirectories
        dir $Path | ?{$_.PsIsContainer} | sort | %{ ProcessFolder $_.FullName }
        #remove target directory if it contains no files on any level
        if( !(dir $tmp -recurse | ?{!$_.PsIsContainer}) ) { del $tmp -recurse }
    }
    Catch
    {
        Log "ERROR copying folder $Path to $tmp`:`
        $_"
    }
}

cls

$src = "W:\IIS\"
$dst = "C:\Temp\DevTest\"
$log = $null
$timestamp = '{0:yyyyMMddHHmmssfffffff}' -f (Get-Date)
ProcessFolder $src

''
'DONE!'
if( $log )
{
    echo "Check the log file: $log"
}
Read-Host

Solution 2

The first one should ideally work. It does preserve the directory structure. But you have to be careful about copying folders. Assuming you want just the files in the pattern that you want in and you don't care about the folders being there, you can do below:

$source = "W:\IIS"
$destination = "C:\Temp\DevTest" 

if(-not (Test-Path $destination)) { mkdir $destination | out-null}

foreach ($i in Get-ChildItem -Path $source -Recurse)
{
    if (($i.Name -notmatch "_\d{8,10}$") -and (-not $i.PsIsContainer))
    {
        continue;
    }
    Copy-Item -Path $i.FullName -Destination $i.FullName.Replace($source,$destination).Trim($i.Name)
}

Solution 3

For me Copy-Item is a mess you can read other StackOverflow answers this one or this other one . Here is a solution which is a kind of 'bricolage'.

$source = "W:\\IIS" # Really double \

Get-ChildItem -Path $source -Recurse | Where-Object{($_.Name -match ""_\d{8,10}$") -or ($_.psiscontainer)} | % {copy-item  $_.fullname ($_.fullname -replace $source,'C:\Temp\DevTest') }