Ignoring GET parameters in Varnish VCL
Solution 1
If you need to leave the key=SOMEALPHANUM
intact to the backend in case not delivered from cache, then it might be better to regsub
within the vcl_hash
function instead, as this won't really alter the url but instead it just alters the hash of the key.
sub vcl_hash {
if(req.http.host ~ "the-site-in-question.com" & req.url ~ "^/api/") {
set req.http.X-Sanitized-URL = req.url;
set req.http.X-Sanitized-URL = regsub(req.http.X-Sanitized-URL, "&key=[A-Za-z0-9]+", "");
set req.hash += req.http.X-Sanitized-URL;
} else {
set req.hash += req.url;
}
set req.hash += req.http.host;
hash;
}
Solution 2
in vcl_recv
set req.url = regsub(req.url, "&key=.*$", "");
Solution 3
Similar to the answer by cd34, but taking into account the possibility of different orders for the query parameters and relying on the fact that the problem defines the value as alphanumeric:
set req.url = regsub(req.url, "&key=[A-Za-z0-9]*", "");
(I can't comment yet, otherwise this would be a comment on cd34's answer)
Related videos on Youtube
JamesHarrison
Updated on September 17, 2022Comments
-
JamesHarrison almost 2 years
Okay: I've got a site set up which has some APIs we expose to developers, which are in the format
/api/item.xml?type_ids=34,35,37®ion_ids=1000002,1000003&key=SOMERANDOMALPHANUM
In this URI, type_ids is always set, region_ids and key are optional.
The important thing to note is that the key variable does not affect the content of the response. It is used for internal tracking of requests so we can identify people who make slow or otherwise unwanted requests.
In Varnish, we have a VCL like this:
if (req.http.host ~ "the-site-in-question.com") { if (req.url ~ "^/api/.+\.xml") { unset req.http.cookie; } }
We just strip cookies out and let the backend do the rest as far as times are concerned (this is a hackaround since Rails/authlogic sends session cookies with API responses).
At present though, any distinct developers are basically hitting different caches since
&key=SOMEALPHANUM
is considered as part of the Varnish hash for storage. This is obviously not a great solution and I'm trying to work out how to tell Varnish to ignore that part of the URI. -
Cooper Wainczak about 14 yearsset req.url = regsub(req.url, "&key=[A-Za-z0-9]+", ""); might be more accurate. Both of ours would do a greedy regexp to the end of the string.