Why does Browser still sends request for cache-control public with max-age?

Question

I have Amazon S3 objects, and for each object, I have set

Cache-Control: public, max-age=3600000

That is roughly 41 days.

And I have Amazon CloudFront Distribution set with Minimum TTL also with 3600000.

This is the first request after clearing cache.

GET /1.0.8/web-atoms.js HTTP/1.1
Host: d3bhjcyci8s9i2.cloudfront.net
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8

And Response is

HTTP/1.1 200 OK
Content-Type: application/x-javascript
Content-Length: 226802
Connection: keep-alive
Date: Wed, 28 Aug 2013 10:37:38 GMT
Cache-Control: public, max-age=3600000
Last-Modified: Wed, 28 Aug 2013 10:36:42 GMT
ETag: "124752e0d85461a16e76fbdef2e84fb9"
Accept-Ranges: bytes
Server: AmazonS3
Age: 342557
Via: 1.0 6eb330235ca3971f6142a5f789cbc988.cloudfront.net (CloudFront)
X-Cache: Hit from cloudfront
X-Amz-Cf-Id: 92Q2uDA4KizhPk4TludKpwP6Q6uEaKRV0ls9P_TIr11c8GQpTuSfhw==

Even while Amazon clearly sends Cache-Control, Chrome still makes second request instead of reading it from Cache.

GET /1.0.8/web-atoms.js HTTP/1.1
Host: d3bhjcyci8s9i2.cloudfront.net
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
If-None-Match: "124752e0d85461a16e76fbdef2e84fb9"
If-Modified-Since: Wed, 28 Aug 2013 10:36:42 GMT

Question: Why does chrome makes second request?

Expires This behavior changes when I put an explicit Expires attribute in headers. Browser will not send subsequent request for Expires header, but for cache-control public, it does send it. My all S3 objects will never change, they are immutable, when we change file, we put them as new object with new URL.

In Page Script Reference Chrome makes subsequent requests only sometimes, I did this test by actually typing URL in browser. When script is referenced by HTML page, for few subsequent requests chrome loads cached scripts, but once again after sometime, once in a while it does send request to server. There is no Disk Size issue here, Chrome has sufficient cache space.

Problem is we get charged for every request, and I want S3 objects to be cached forever, and should be loaded from Cache and should never connect to server back.

Solution 1

If the HTTP Response contains the etag entry, the conditional request will always be made. ETag is a cache validator tag. The client will always send the etag to the server to see if the element has been modified.

Solution 2

When you press F5 in Chrome, it will always send requests to the server. These will be made with the Cache-Control:max-age=0 header. The server will usually respond with a 304 (Not Changed) status code.

When you press Ctrl+F5 or Shift+F5, the same requests are performed, but with the Cache-Control:no-cache header, thus forcing the server to send an uncached version, usually with a 200 (OK) status code.

If you want to make sure that you're utilizing the local browser cache, simply press Enter in the address bar.

Solution 3

If Chrome Developer Tools are open (F12), Chrome usually disables caching.

It is controllable in the Developer Tools settings - the Gear icon to the right of the dev-tools top bar.

Solution 4

Chrome adds Cache-control: max-age=0 header when you use self-signed certificate. Switching from HTTPS to HTTP will remove this header.

Firefox doesn't add this header.

Solution 5

If you are hitting the refresh button for loading the particular page or resource, the if-modified-since header request is sent everytime, if you instead request the page/resource as a separate request in a new tab or via a link in a script or html page, it will load the page/resource from the browser cache itself.

This is what has happened in my case, may be this is the general universal case. I am not completely sure, but this is what I gathered via my digging.

View more solutions

Comments

[Akash Kava]

Thanks, but that is wrong design, shouldn't it ignore ETag while Max Age & public cache is specified. S3 automatically adds ETag, do I have to remove it or can I change headers to support expected behaviour? I don't know if there is way to remove ETag in S3.

[woolagaroo]

On Chrome I have found that removing the entry is the only way to impede a conditional http request. IE does a much better job at this IMHO.

[Akash Kava]

So is it a bug in Chrome, or is it correct according to HTTP 1.1 protocol?

[woolagaroo]

When I had the issue I did not find any metions on the RFC of the http 1.1 Protocol on which one should take precedence, or if the two entries could be combined. I did find some other answers (not officials) vadmyst.blogspot.co.at/2005/09/… stackoverflow.com/questions/6350384/… Both answers indicate that it is a bug in Chrome

[Alex Jones]

If there's an http response then the request already took place and the max-age was ignored.

[Costa]

In my testing, Etags don't come into play (all my resources have an etag). I'm pretty sure this is how you're testing with Chrome. Pressing "enter" on the address bar is best way to test cache. F5 or clicking reload with send the "max-age=0" header. See also: stackoverflow.com/a/16510707/789658

[Akash Kava]

F12 only disables caching when you tick the box to disable it, when I am testing caching I am certain that caching is enabled.

[Alexis Wilke]

It worked correctly for me. I suspect that if it was an issue in Oct 2013, it was fixed since.

[DarkNeuron]

As of this date, Costa's reply is still relevant: Chrome sends a max-age=0 in the request header if you refresh the page. If you press enter in the URL, it doesn't.

[Soumya Kanti]

Great answer. I made the relevant changes in the server and was inspecting the results with Dev Tools (F12) open and could not see the impact. There is so much little things to know. +1 from me.

[Volksman]

OMG! I didn't realize that! Developer Tools "Disable cache" was checked in my browser - Doh!

[Piyush Beli]

Yes, that's correct. If you refresh the page then it gives 304 http code and goes to server while if you hit the url again it would be fetched from cache with 200 http status code.

[Gras Double]

Simple and effective answer. TIL F5 and Enter make different requests.

[Bradley]

In Chrome Version 57.0.2987.133 (64-bit), hitting enter on the URL still seems to be sending a conditional request and max-age=0 in the request even though the response from cloud front has Cache-Control:max-age=300. Basically, I can't seem to get it to ever fetch from the browser's cache regardless of how I issue the request. I've verified the the browser cache is not disabled in Dev Tools. Any ideas? This is an image. I'm wondering if the Content-Type has anything to do with it.

[Bradley]

2 things: 1. When going to URL for my image in CloudFront directly in Chrome tab, the only way I can get it to reliably pull from browser cache is to get to the URL via Back/Forward. If I F5 or hit Enter in URL, I always see conditional request with 304. 2. Only appears to be the case when going directly to image URL. When used in content of an HTML document, the caching works as described above where an F5 refresh and hitting Enter in the URL yield different results. Can only assume Chrome is treating navigating directly to URL as a sign it should do a conditional request. Expected?

[seabass]

@DarkNeuron saved me a lot of headache. I didn't know why Cache-Control: max-age=0 kept getting sent on reload. Opening a new tab lets the cache work. Thank you so much for saying that!

[Alfonso Nishikawa]

Three days hitting my head with this. THANKS! I was presing Ctrl + F5 and didn't know where Control:no-cache came from and why the browser was ignoring my Cache-Control, or if I was doing something wrong. God, thanks! And @GrasDouble, thx for the link. I know the comments should not be used to thanks, but finding you answer was a great relief.

[Victor Castro]

Great answer, I didn't check the request headers, you can run fetch to check if it's really getting from disk ex: fetch('YOUR_URL')

[Debajit Majumder]

Is there any way by which I can avoid sending the request to server in case of F5? As of now it sends the request to server, then server responds back with 304 and then it gets loaded from cache. I want browser to load it from cache directly when someone press F5, as my resources are available in disk cache.

[Oliver Salzburg]

@DebajitMajumder Then you need to allow the client to do that. Usually with an Expires header.

[maaartinus]

With Shift+F5, there's also no if-none-match header in the request.

[Ashwin Aggarwal]

@Bradley comment still holds true, saved me so much time! F5 or directly loading a resource from URL navigator, chrome sends Cache-Control: max-age=0. While when asking for the resource in the app, the resource is served from cache.

[Joy George Kunjikkuru]

This save the day. Any official links explaining this behavior?

[Oliver Salzburg]

@JoyGeorgeKunjikkuru Sorry, no. It's just what I observed when I researched it.

[Jeremy Caney]

I ran into a very similar issue when testing response caching on localhost with Chrome—though, instead of sending max-age=0, it was sending nocache. Same result, of course. Regardless, it was clear that it was sending this with each request, but it wasn't obvious why. This is useful to know when testing caching on a local development server, which will almost certainly be using a self-signed certificate.

[Steven]

@Bradley greatly helps when I'm digging the issue for hours!