When to use and not to use ETags

Question

I was just looking at our site on WebPageTest.org and one of their recommendations for speeding up a website is:

ETag headers should generally not be used unless you have an explicit reason to need them

I was wondering what this means. Does it mean that static content you know will not be changing should not have them, or does it mean content you know will be changing regularly shouldn't have them, or does it mean that you shouldn't use them generally unless you have a specific need.

If it's the latter, when is the right time to use them? Thanks for any help.

Solution 1

ETags are an alternative to (but can be used in combination with) "Last-Modified-Time" in order to determine cache validation.

The client can send a pre-condition such as if-matches or if-none-matches based on the ETag. This is not just for GET requests (which is what webpagetest.org does) you can use "opportunistic updating" so that a PUT request has a pre-condition and will not perform the update operation if the resource has been updated since the ETag was last acquired.

Put simply: you hit edit on a page in your CMS, your friend hits edit on a page in your CMS, your friend performs their edit and hits save and finally you hit save - without an ETag or Content-MD5 HTTP header you would need to reinvent the wheel to prevent problems occuring (such as you wiping over your friends changes) the solution is already part of the HTTP protocol and therefore it makes sense to just use it.

Generally I agree with AOL (who run webpagetest.org) on their "one size fits all" advice - it is better not to clog up HTTP headers with cryptic strings (ETags are generally not pretty or human readable) when a second of difference (which Last-Modified-Time can detect) will do for the job in hand.

If a page is being updated several times a second and you absolutely need the latest most accurate version to be displayed, you may want to experiment with solutions other than HTTP GETs or just use ETags.

Be careful that your ETags do not include per filesystem, per server configuration change, etc information (such as inodes which is default on Apache) otherwise you'll have problems when there are two servers (the ETags from each one will not match).

Solution 2

To paraphrase Coding Horror's excellent evaluation of the YSlow Firebug plugin (which it appears WebPageTest.org is using as a basis for their evaluation):

"Yahoo is one of the busiest websites in the world - its problems are probably not your problems."

If you're not dealing in millions of uniques across your load-balanced network every day, there's a good chance that the suggestions provided are not all the right choices when optimizing your site.

Comments

[Django Reinhardt]

I appreciate the comment, but you're wrong. Steve Sounders explains here: stevesouders.com/blog/2010/09/07/webpagetest-org-and-page-sp‌​eed (He's a big proponent of useful improvements -- i.e. not just improvements that will work for Google.) Our site is definitely big enough to feel a speed increase from such improvements (and already has).

[John Conde]

The yslow firebug extension now includes a test for smaller sites which is much more realistic for most sites.

[Jonx]

@Django Reinhardt - I reworded my reply, it's true that many of the suggestions are valid for any site, but ETags in particular is contentious because, in Y!'s case, the feature conflicts with load-balancing

[Django Reinhardt]

The link you provided, had a link onto Yahoo's description of ETags (developer.yahoo.com/performance/rules.html#etags) which answers my questions nicely. (Yes, it would probably be better for us to not serve ETags.) Thanks.

[Lèse majesté]

@Django Reinhardt: are you using cloud hosting/VPS or some other form of load-balancing?

[DisgruntledGoat]

@Django: If you have multiple servers then ETags can be a problem. However, as long as the same file always returns the same ETag then it's perfectly fine. See also: webmasters.stackexchange.com/questions/1459/…

[Metalshark]

That's ok. There is one example that I am not sure of: if you have multiple versions of content all on one URI (such as a mobile or Internet Explorer versions) ETags may be used to search through EVERY version for a match (hence it's called if-none-matches not if-not-match) - depending on who you ask there are different answers (such as do not have one permanent URI for several representations, etc).