Preserve HTTP referer when using a 301 redirect

Question

I have htaccess detecting absent folders with the action of then sending the user to another page with a 301 re-redirect.

RewriteCond %{REQUEST_FILENAME} !-f 
RewriteCond %{REQUEST_FILENAME} !-d 


RewriteRule ^([^/]*)/?([^/]*)/?$ /page.html?a=$1&b=$2 [QSA,L,R=301]

The problem is that I need to log the the HTTP_REFERER on page.html but this is lost by the 301. If there is no 301 the value is also lost.

Apache can access HTTP_REFERER with %{HTTP_REFERER} but that only appears accessible to condition rules and not the rewrite. So I can't append the URL with the %{HTTP_REFERER} value as I'd like to.

From what I can tell there is no easy solution here, so I am wondering if there is another way to approach this.

Solution

but this is lost by the 301

An ordinary 3xx redirect will maintain the HTTP Referer. However, whether the HTTP Referer is present or not is entirely dependent on the client making the request. If the HTTP Referer is not present, it is because the client is not sending it. If the HTTP Referer is not present on the redirected request then it probably wasn't present on the initial request either.

Note that the HTTP Referer sent by the client can be influenced by a Referrer-Policy implemented on the source site.

So I can't append the URL with the %{HTTP_REFERER} value

Yes you can. For example:

RewriteRule ^([^/]*)/?([^/]*)/?$ /page.html?a=$1&b=$2&referer=%{HTTP_REFERER} [QSA,L,R=301]

If you are getting an empty referer URL parameter then it's because the HTTP Referer is empty (not present).

I have htaccess detecting absent folders

Note that your directives are "detecting absent files" as well.

But, as noted by @HBruijn in comments, why "externally redirect" the request? It would seem to be preferable to just "internally rewrite" the request instead and avoid the second request. In this case, simply remove the R flag. For example:

RewriteRule ^([^/]*)/?([^/]*)/?$ /page.html?a=$1&b=$2 [QSA,L]

However, why not implement a custom 404 instead and avoid mod_rewrite entirely?

Comments

[HBruijn]

HTTP 301 and 302 are instructions that the server sends to a browser that a resource has moved. Included in the 301/302 response is the new URL. After getting such a response the browser makes a new request to the target of the redirect. Unless you have a particular reason to send such redirect, simply let Apache rewrite the request and it will be processed directly and neither the referrer will be lost, nor is a second request by the browser needed.

[MrWhite]

I've updated my answer.

[Kline]

Well you are right it wasn't the 301 dropping the HTTP_REFERER, however switching incoming to http links to https is losing the HTTP_REFERER and that is the cause of the problem. However I need https to be forced so I am back to square one.

[Kline]

It looks like HTTP_REFERER is never preserved when switching to HTTPS, so that is that.

[MrWhite]

"when switching to HTTPS" - The Referer should still be preserved (by default) when navigating to HTTPS. However, when navigating from HTTPS to HTTP it is not (which I assume is what you are referring to in your previous comment), since the browser's default Referrer-Policy is no-referrer-when-downgrade. Any redirect (erg. back to HTTPS) after this will naturally be missing the Referer as well. This is to protect users privacy when navigating from a secure to non-secure network.

[MrWhite]

It is possible to override this behaviour on the source site (if you have control of this), as mentioned above, by setting the HTTP response header Referrer-Policy: unsafe-url (but this is not recommended - and user-agent's may not honour this anyway).

[Kline]

My tests were from HTTPS to HTTP so this was the cause of the problem.