Ignoring GET parameters in Varnish VCL

8,318

Solution 1

If you need to leave the key=SOMEALPHANUM intact to the backend in case not delivered from cache, then it might be better to regsub within the vcl_hash function instead, as this won't really alter the url but instead it just alters the hash of the key.

sub vcl_hash {
  if(req.http.host ~ "the-site-in-question.com" & req.url ~ "^/api/") {
    set req.http.X-Sanitized-URL = req.url;
    set req.http.X-Sanitized-URL = regsub(req.http.X-Sanitized-URL, "&key=[A-Za-z0-9]+", "");
    set req.hash += req.http.X-Sanitized-URL;
  } else {
    set req.hash += req.url;
  }
  set req.hash += req.http.host;
  hash;
}

Solution 2

in vcl_recv

set req.url = regsub(req.url, "&key=.*$", "");

Solution 3

Similar to the answer by cd34, but taking into account the possibility of different orders for the query parameters and relying on the fact that the problem defines the value as alphanumeric:

set req.url = regsub(req.url, "&key=[A-Za-z0-9]*", ""); 

(I can't comment yet, otherwise this would be a comment on cd34's answer)

Share:
8,318

Related videos on Youtube

JamesHarrison
Author by

JamesHarrison

Updated on September 17, 2022

Comments

  • JamesHarrison
    JamesHarrison almost 2 years

    Okay: I've got a site set up which has some APIs we expose to developers, which are in the format

    /api/item.xml?type_ids=34,35,37&region_ids=1000002,1000003&key=SOMERANDOMALPHANUM
    

    In this URI, type_ids is always set, region_ids and key are optional.

    The important thing to note is that the key variable does not affect the content of the response. It is used for internal tracking of requests so we can identify people who make slow or otherwise unwanted requests.

    In Varnish, we have a VCL like this:

    if (req.http.host ~ "the-site-in-question.com") {
      if (req.url ~ "^/api/.+\.xml") {
        unset req.http.cookie;
      }
    }
    

    We just strip cookies out and let the backend do the rest as far as times are concerned (this is a hackaround since Rails/authlogic sends session cookies with API responses).

    At present though, any distinct developers are basically hitting different caches since &key=SOMEALPHANUM is considered as part of the Varnish hash for storage. This is obviously not a great solution and I'm trying to work out how to tell Varnish to ignore that part of the URI.

  • Cooper Wainczak
    Cooper Wainczak about 14 years
    set req.url = regsub(req.url, "&key=[A-Za-z0-9]+", ""); might be more accurate. Both of ours would do a greedy regexp to the end of the string.