How to properly -filter multiple strings in a PowerShell copy script

Question

I am using the PowerShell script from this answer to do a file copy. The problem arises when I want to include multiple file types using the filter.

Get-ChildItem $originalPath -filter "*.htm"  | `
   foreach{ $targetFile = $htmPath + $_.FullName.SubString($originalPath.Length); ` 
 New-Item -ItemType File -Path $targetFile -Force;  `
 Copy-Item $_.FullName -destination $targetFile }

works like a dream. However, The problem arises when I want to include multiple file types using the filter.

Get-ChildItem $originalPath ` 
  -filter "*.gif","*.jpg","*.xls*","*.doc*","*.pdf*","*.wav*",".ppt*")  | `
   foreach{ $targetFile = $htmPath + $_.FullName.SubString($originalPath.Length); ` 
 New-Item -ItemType File -Path $targetFile -Force;  `
 Copy-Item $_.FullName -destination $targetFile }

Gives me the following error:

Get-ChildItem : Cannot convert 'System.Object[]' to the type 'System.String' required by parameter 'Filter'. Specified method is not supported.
At F:\data\foo\CGM.ps1:121 char:36
+ Get-ChildItem $originalPath -filter <<<<  "*.gif","*.jpg","*.xls*","*.doc*","*.pdf*","*.wav*",".ppt*" | `
    + CategoryInfo          : InvalidArgument: (:) [Get-ChildItem], ParameterBindingException
    + FullyQualifiedErrorId : CannotConvertArgument,Microsoft.PowerShell.Commands.GetChildItemCommand

I have various iterations of parentheses, no parentheses, -filter, -include, defining the inclusions as variable (e.g., $fileFilter) and each time get the above error, and always pointing to whatever follows -filter.

The interesting exception to that is when I code -filter "*.gif,*.jpg,*.xls*,*.doc*,*.pdf*,*.wav*,*.ppt*". There are no errors, but I and get no results and nothing back to the console. I suspect I've inadvertently coded an impicit and with that statement?

So what am I doing wrong, and how can I correct it?

Solution 1

-Filter only accepts a single string. -Include accepts multiple values, but qualifies the -Path argument. The trick is to append \* to the end of the path, and then use -Include to select multiple extensions. BTW, quoting strings is unnecessary in cmdlet arguments unless they contain spaces or shell special characters.

Get-ChildItem $originalPath\* -Include *.gif, *.jpg, *.xls*, *.doc*, *.pdf*, *.wav*, .ppt*

Note that this will work regardless of whether $originalPath ends in a backslash, because multiple consecutive backslashes are interpreted as a single path separator. For example, try:

Get-ChildItem C:\\\\\Windows

Solution 2

Something like this should work (it did for me). The reason for wanting to use -Filter instead of -Include is that include takes a huge performance hit compared to -Filter.

Below just loops each file type and multiple servers/workstations specified in separate files.

##  
##  This script will pull from a list of workstations in a text file and search for the specified string


## Change the file path below to where your list of target workstations reside
## Change the file path below to where your list of filetypes reside

$filetypes = gc 'pathToListOffiletypes.txt'
$servers = gc 'pathToListOfWorkstations.txt'

##Set the scope of the variable so it has visibility
set-variable -Name searchString -Scope 0
$searchString = 'whatYouAreSearchingFor'

foreach ($server in $servers)
    {

    foreach ($filetype in $filetypes)
    {

    ## below creates the search path.  This could be further improved to exclude the windows directory
    $serverString = "\\"+$server+"\c$\Program Files"


    ## Display the server being queried
    write-host “Server:” $server "searching for " $filetype in $serverString

    Get-ChildItem -Path $serverString -Recurse -Filter $filetype |
    #-Include "*.xml","*.ps1","*.cnf","*.odf","*.conf","*.bat","*.cfg","*.ini","*.config","*.info","*.nfo","*.txt" |
    Select-String -pattern $searchstring | group path | select name | out-file f:\DataCentre\String_Results.txt

    $os = gwmi win32_operatingsystem -computer $server
    $sp = $os | % {$_.servicepackmajorversion}
    $a = $os | % {$_.caption}

    ##  Below will list again the server name as well as its OS and SP
    ##  Because the script may not be monitored, this helps confirm the machine has been successfully scanned
        write-host $server “has completed its " $filetype "scan:” “|” “OS:” $a “SP:” “|” $sp


    }

}
#end script

Solution 3

Let's go over the options:

• -Filter only takes one pattern, so it doesn't work for this problem.

• -Include works but is very slow (which is totally fine in many cases).

• Piping to Where-Object is much faster than -Include. It is also the most powerful option because it gives you access to regex pattern matching (instead of the normal wildcard matching) and any other logic you might need, such as in the example below:

  # checking extension with regex
  Get-ChildItem $dir |
      Where-Object { $_.Extension -match '\.(xlsx?|jpe?g)$' }
  
  # checking extension and creation time
  Get-ChildItem $dir | Where-Object {
      $_.Extension -in '.xls', '.xlsx', '.jpg', '.jpeg' -and
      $_.CreationTime -gt $yesterday
  }
  

• -Path is slightly faster still but takes full paths rather than filenames, which is a pain to work with (see examples below) and only works for simple cases because path patterns can't match a variable number of directory levels. This is in contrast to typical shells, in which * matches a single directory and ** matches any number of nested directories.

  # simpler
  $paths = $dir\*.xls, $dir\*.xlsx, $dir\*.jpg, $dir\*.jpeg
  Get-ChildItem $paths
  
  # less repetitive
  $paths = 'xls', 'xlsx', 'jpg', 'jpeg' | % { Join-Path $dir *.$_ }
  Get-ChildItem $paths
  

Comments

[dwwilson66]

That didn't work. :( -filter -include *.file, *.types -filter -include (*.file, *.types), -filter -include "*.file", "*.types", and -filter -include ("*.file", "*.types") all errored out as per my question above. Eliminating the -filter parameter and just including the -include (same iterations of quotes and parens) did not result in runtime errors, but there was no result set in the destination directory.

[dwwilson66]

WooHoo! That \* trick has just solved about six problems. Awesome & Thanks!

[The Guy with The Hat]

Welcome to Stack Overflow! This answer turned up in the low quality review queue, presumably because you didn't explain the content. If you do explain this (in your answer), you are far more likely to get more upvotes—and the questioner actually learns something!

[Adi Inbar]

Also, it repeats the code from the answer I posted earlier, except that it encloses the list of extensions in an array expression evaluation operator (@()), which is superfluous, because a comma-separated list is inherently evaluated as an array.

[Ohad Schneider]

Note that the wildcard (\*) is not needed when -Recurse is specified.

[Ross Presser]

for some reason this doesn't work when searching for directories?

[Derek Evermore]

Adding -Recurse allows this to reach out to sub folders

[papo]

this is very true and in 5 similar questions here no one pointed out that although we can't do -filter *.jpg, *.png it might still be faster to do -filter *.jpg in one go, than do -filter *.png and join the results, than do one -Include *.jpg, *.png". I have a folder containing 126k files and 18k folders. I am searching for one file and one folder in each folder recursively. using -Filter takes 5sec and using -Include 30secs. Doing -Filter two times in 10secs is three times faster than one -Include go.

[Mark Lopez]

-Recuse does not work with -Include (in regards of traversing subdirectories).

[MarredCheese]

@MarkLopez I would argue that if you use the param "recuse" and the command does nothing, it's actually working perfectly. Sorry, couldn't resist :)

[Mark Lopez]

Ha! Thanks for that, unfortunately I can't edit anymore.