What files should be excluded from a complete Windows backup?

Question

I'm starting to use CrashPlan to backup my Win 7 PC. I've got it writing to my external HD (for quick local restores) and to CrashPlan Central (for offsite storage). I'd like to backup my entire C:\ drive (the only partition) in a way that:

1. Preserves all of my installed software and configuration, but
2. Avoids backing up log files and other ephemeral / temporary files that are regenerated during normal operation of the OS.

Which files and/or directories should I be excluding from backups? I'd like to make this a community wiki, so that we could all contribute towards a definitive list.

Here's a list of regular expressions identifying the directories and files that CrashPlan excludes on Windows by default listed at http://support.crashplan.com/doku.php/articles/admin_excludes:

.*/(?:42|\d{8,})/(?:cp|~).*
(?i).*/CrashPlan.*/(?:cache|log|conf|manifest|upgrade)/.*
.*\.part
.*/iPhoto Library/iPod Photo Cache/.*
.*\.cprestoretmp.*
*\.rbf
:/Config\\.Msi.*
.*/Google/Chrome/.*cache.*
.*/Mozilla/Firefox/.*cache.*

.*\$RECYCLE\.BIN/.*
.*/System Volume Information/.*
.*/RECYCLER/.*
.*/I386.*
.*/pagefile.sys
.*/MSOCache.*
.*UsrClass\.dat\.LOG
.*UsrClass\.dat
.*/Temporary Internet Files/.*
(?i).*/ntuser.dat.*
.*/Local Settings/Temp.*
.*/AppData/Local/Temp.*
.*/AppData/Temp.*
.*/Windows/Temp.*
(?i).*/Microsoft.*/Windows/.*\.log
.*/Microsoft.*/Windows/Cookies.*
.*/Microsoft.*/RecoveryStore.*
(?i).:/Config\\.Msi.*
(?i).*\\.rbf
.*/Windows/Installer.*

Other excludes:

.*\.(class|obj)
.*/hiberfil.sys
(?i).*\.tmp
(?i).*/temp/
(?i).*/tmp/
.*Thumbs\.db
.*/Local Settings/History/
.*/NetHood/
.*/PrintHood/
.*/Cookies/
.*/Recent/
.*/SendTo/

Solution

CrashPlan isn't really meant for the type of full-system backup you're describing. It's meant for backing up your data. Even if you were to come up with some sort of master list of non-ephemeral Windows files and back everything up completely, you still wouldn't actually be able to restore it on a fresh installation because Windows wouldn't allow you to replace system files and all the other files that are in use while Windows is running. It's also not going to play very well with the various DRM schemes that applications/games use.

If you really want a full-system backup like you're describing, you're going to need to use another product based on disk images that supports bare-metal restore.

Don't get me wrong. I love CrashPlan and use it for all the computers in our house. But you have to use it for what it was intended for which is preventing you from losing data, not applications and system configuration. If you have specific applications that you really don't want to have to re-configure on a fresh install, analyze them individually and figure out where their configuration settings are stored. Then back those files up explicitly. If you're lazy, backing up the AppData folder in your user profile will get you 99% of the way there.

Comments

[tro]

I should say that the main reason I want to do this is because Crash Plan is constantly recording the files that have been changed and then backs them up when it's triggered next (once an hour for me). I end up with 1 GB of files being written to disk every hour when I'm not even using my PC.

[Moab]

.*/hiberfil.sys

[bbodenmiller]

superuser.com/a/444277/143551 provides a through answer to this question.

[tro]

I was suspecting this, but needed someone to confirm it. :)

[bbodenmiller]

While I agree with the answer I still think that it would be a great idea to continue with building this list as many users would like to backup their whole User folder which contains many unnecessary files. I've added a few more files to the list and hope others will as well.