Invalidate all files in a folder in cloudfront console

Question

I know cloudfront provides a mechanism to invalidate a file, but what if I want to invalidate all files in a specific folder ? The documentation mentions that I can't use wildcards to do this.

Here's the instruction taken from the official documentation:

You must explicitly invalidate every object and every directory that you want CloudFront to stop serving. You cannot use wildcards to invalidate groups of objects, and you cannot invalidate all of the objects in a directory by specifying the directory path.

Solution 1

Back in 2013, in a previous version of this answer, I wrote:

You can't do this because "files" in cloudfront are not in "folders." Everything is an object and every object is independent.

At the time, that was entirely true. It's still true that everything is an object and every object is independent, but CloudFront has changed its invalidation logic. Keep reading.

At the time, this was also true, and again, to a certain extent, it still is:

The cloudfront documentation mentions "invalidating directories," but this refers to web sites that actually allow a directory listing [when] the listing is what you want to invalidate, so this won't help you either.

However, times have changed significantly.

Technically, each object is still independent, and CloudFront does not really store them in hierarchical folders, but the invalidation interface has been enhanced, to support a left-anchored wildcard match. You can invalidate the contents of a "folder" or any number of objects that you can match with a wildcard at the end of the string. Anything that matches will be evicted from the cache:

To invalidate objects, you can specify either the path for individual objects or a path that ends with the * wildcard, which might apply to one object or to many, as shown in the following examples:

/images/image1.jpg

/images/image*

/images/*

^{— http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html}

Nice enhancement. But is there a catch?

Other than the fact that an invalidation requires -- as always -- 10 to 15 minutes to complete under normal operations, the answer is no, there's not really a catch. The first 1,000 invalidation paths (formerly "requests," and a "request" was for a single object) you submit within a month are free; after that, there is a charge, but:

The price is the same whether you're invalidating individual objects or using the * wildcard to invalidate multiple objects.

^{— http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html#PayingForInvalidation}

Note that if you don't include the * at the end, then an invalidation for /images/ (for example) will only tell CloudFront to invalidate whatever single object your origin server returns for requests for /images/.

The leading slash is documented as optional.

Solution 2

As of 2015-05-25, you can invalidate using a wildcard. Ex: /* or /images/*

It is also far less costly to do it this way, as something like /images/* counts as one object for invalidation, rather than being charged for the thousands of images in the /images directory.

http://aws.amazon.com/about-aws/whats-new/2015/05/amazon-cloudfront-makes-it-easier-to-invalidate-multiple-objects/

Solution 3

As long as you want to invalidate a reasonable amount of objects, one of the easier ways I've found is to select the objects in Cyberduck, right click > select Info and click on Distribution tab and you can invalidate from there. Cyberduck will submit one invalidation request to your Cloudfront with the list of selected files.

Cyberduck is open source too.

_{ps: not affiliated with the product in any way. Just listing an alternative.}

Comments

[Scott]

This is a great way to do it, but only if you set up the Cloudfront distribution through Cyberduck itself. Cyberduck seems unable to recognize an already-existing Cloudfront distro.

[iamserious]

@Scottie - I've setup all my Cloudfront via the amazon web console.. My cyberduck doesn't actually recognize the CDN urls (and hence gives me empty urls etc) but it still lets me invalidate files.

[Scott]

Huh! Will have to give that a closer look then. Thanks!

[Michael - sqlbot]

Yep, sure enough. Curiously, my attention was drawn back to this question because I got an upvote yesterday on my then-obsolete answer from two years prior -- which I appreciate very much -- but my answer has now been revised to reflect the current state of affairs.