Expire Files In A Folder: Delete Files After x Days

Question

I'm looking to make a "Drop Folder" in a windows shared drive that is accessible to everyone. I'd like files to be deleted automagically if they sit in the folder for more than X days.

However, it seems like all methods I've found to do this, use the last modified date, last access time, or creation date of a file.

I'm trying to make this a folder that a user can drop files in to share with somebody. If someone copies or moves files into here, I'd like the clock to start ticking at this point. However, the last modified date and creation date of a file will not be updated unless someone actually modifies the file. The last access time is updated too frequently... it seems that just opening a directory in windows explorer will update the last access time.

Anyone know of a solution to this? I'm thinking that cataloging the hash of files on a daily basis and then expiring files based on hashes older than a certain date might be a solution.... but taking hashes of files can be time consuming.

Any ideas would be greatly appreciated!

Note:
I've already looked at quite a lot of answers on here... looked into File Server Resource Monitor, powershell scripts, batch scripts, etc. They still use the last access time, last modified time or creation time... which, as described, do not fit the above needs.

Solution 1

We used a combination of a powershell script and a policy. The policy specifies that the user must create a folder inside the Drop_Zone share and then copy whatever files they want into that folder. When the folder gets to be 7 days old (using CreationTime) the powershell script will delete it.

I also added some logging to the powershell script so we could verify it's operation and turned on shadow copies just to save the completely inept from themselves.

Here is the script without all the logging stuff.

$location = Get-ChildItem \\foo.bar\Drop_Zone
$date = Get-Date
foreach ($item in $location) {
  # Check to see if this is the readme folder
  if($item.PsIsContainer -and $item.Name -ne '_ReadMe') {
    $itemAge = ((Get-Date) - $item.CreationTime).Days
    if($itemAge -gt 7) {
      Remove-Item $item.FullName -recurse -force
    }
  }
  else {
  # must be a file
  # you can check age and delete based on that or just delete regardless
  # because they didn't follow the policy
  }
}

Solution 2

If you can assume NTFS you could write a key (Guid) into an alternate stream of the file. Plus the date, so you could basically store the database in the files.

More information can be found at

http://blogs.technet.com/b/askcore/archive/2013/03/24/alternate-data-streams-in-ntfs.aspx

Basically you can store additional content in a separate stream that is coded by a special name.

Solution 3

You could use IO.FileSystemWatcher, which allows you to "watch" a folder for new files created. Here are the pieces you'd need to make this work.

These variables configure the path to watch and a filter to fine-tune which files to track:

$watchFolderPath = $env:USERPROFILE
$watchFolderFilter = "*.*"

This sets up the parameters for the folder to watch and the actions to perform when the event occurs. Basically this resets the LastWriteTime on each file as it's written:

$watcher = New-Object IO.FileSystemWatcher $watchFolderPath, $watchFolderFilter -Property @{
    IncludeSubdirectories = $true
    NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
    }
$onCreated = Register-ObjectEvent $watcher Created -SourceIdentifier FileCreated -Action {
    $FileName = $Event.SourceEventArgs.FullPath
    $file = Get-Item $FileName
    $file.LastWriteTime = Get-Date
    }

The event can be unregistered if needed using this:

Unregister-Event -SourceIdentifier FileCreated

Finally, you can run this once a day in order to clean up the old files:

Get-ChildItem $watchFolderPath -Recurse | Where-Object {((Get-Date)-$_.LastWriteTime).TotalDays -gt 6} | Remove-Item

That should be everything you need...

Solution 4

There is no way to rely on the dates for when a file was copied or moved into a folder. Windows manages to preserve it across filesystems, drives, network shares, etc. You might be able to work something out with a linux file server, or prevent people from directly copying files by using FTP or a web based upload system.

If you are ok with people not being able to modify the files after they upload, you could have separate upload and access folders, and a script that moves files between them and re-dates them. But it sounds like you want people to be able to modify the files directly.

So a simple, if somewhat hacky, solution would be to mess with the dates. I would write two scripts:

Hourly Date Changer script

Have a script run once an hour or so, in your preferred language, that:

• Looks for any file with a date modified within the last 20 years.
• When it finds such a file, change it's date modified to today minus 20 years.

In powershell, it would look something like this:

$path = "D:\test"

$today = Get-Date
$before = $today.AddDays(-7300) #356*20 days

Get-ChildItem -Recurse -Path $path | foreach {
    if ($_.LastWriteTime -gt $before) {
        Write-Host $_.Name
        $_.LastWriteTime = $before
    }
}

Running this script today (May 27), sets the modified date of all files to June 1st, 1994 - exactly 356*20 days ago. Because it is changing only files newer than the $before value, it won't touch files it has already set to the past.

Cleanup Script

The cleanup script would run every night, and:

• Search for files with date modified "20 years and X days ago"
• Delete them

I won't write the script for this part - there are plenty of utilities that can handle deleting files that are older than a specified date, choose whichever you like. The important part is to look for files that are 7300+X days old, where X is the number of days you want to keep them since they were last modified.

Advantages

This has a few advantages over the other answers here:

• The timer will reset if someone modifies the file.
• No need for NTFS alternative streams to mark the files (which are preserved when moving the file, so could cause premature deletion of a modified file)
• Should have minimal if any performance impact. No need to keep a database or list of filenames and/or hashes.
• Nothing breaks horribly if the scripts fail to run. There is no service or constantly running program needed to update the date. Just a couple scheduled tasks. Solutions that rely on watching for new files and updating their last modified time to right now could end up deleting new files if the service fails or runs into a race condition.

The only problem I can see is if people copy a file that was last modified 20 years ago to the drop folder. I think in most scenarios, that's unlikely to be much of an issue, but it could come up.

Solution 5

It's been a while but I set up a relatively straight forward method for addressing this.

I would touch any files added to the drop directory (monitored via a resource monitoring utility) and set the last modified date to the date added to the folder.

I could then use the last modified date to purge any files that need to be aged off. This also has the advantage that if someone really does update the file it'll reset the countdown.

View more solutions

Comments

[Get-HomeByFiveOClock]

One question, as mentioned by @Michael Kjorling, does the timer stop counting if the file is modified after being dropped in the box?

[Avery Payne]

What you're looking for is the Windows equivelent of tmpwatch.

[Brett G]

How would one do this?

[Brett G]

Sorry, I know what alternate data streams are, I was just trying to understand their usage in this context. So you're saying instead of using a hash or something, use a GUID (and/or date) in the alternate data stream in order to track the files.. aha.

[TomTom]

Yeah. If you can reliably MARK a file - you can even put the marking date in it - then you do not need to calculate a hash.

[Tim Brigham]

@BrettG honestly it was nearly 10 years ago. I can't remember. You're making me feel old. :) If I were to do it today I'd execute a job based on file system auditing events in the event viewer. The FileSystemWatcher .NET object is available via PowerShell I think. It would be another option.

[Brett G]

Ha, I didn't realize you meant that long when you said "a while". Yeah funny enough I was just looking at FileSystemWatcher. Although, I don't think it would work with moved/copied files. Thanks for the reply!

[JohnP]

@BrettG - Filesystemwatcher could be used in conjunction with a tracking table, but it has its own issues. See here: stackoverflow.com/questions/1764809/… stackoverflow.com/questions/6000856/filesystemwatcher-issues

[JohnP]

@BrettG - Also, this is a good extension to FSW: codeproject.com/Articles/58740/…

[Tim Ferrill]

Edited this to set the LastWriteTime attribute when the file is created and then use that to delete files later.

[BeowulfNode42]

This seems simplest, doesn't fudge with the file datetime stamp, alternate data streams, or require some list of files and their drop dates. I was going to create an awesome script that did all sorts of magic, but then I saw this.

[BeowulfNode42]

and doesn't require a file system watch event triggering the script all the time, as it can be run once per day, and it doesn't matter that much if a day is missed for whatever reason.

[Brett G]

Great simple idea, just like @BeowulfNode42 pointed out. To ensure users must create a folder, a simple "Deny" of "Create Files / Write Data" ACL to "This Folder Only" will ensure that users must create subfolders as well.