redirect 204 with htaccess if url contain certain string

Question

I get tons of url with "undefined" added at the end of url, consisting about 20% of my pageview. I tried to check the troubling script source and can't find it. Tried some solutions but none of them are working. Lastly now i'm trying to send url with "undefined" string to redirect 204 using htaccess

is this correct?

RewriteRule ^undefined$ - [R=204,NC,L]

do i need to add "last L" flag?

where do i put it?

above wordpress?

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule ^undefined$ - [R=204,NC,L]
</IfModule>

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

inside wordpress?

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule ^undefined$ - [R=204,NC,L]
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

or after wordpress?

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule ^undefined$ - [R=204,NC,L]
</IfModule>

i've looking for this everywhere, and i want to make sure if this is correct by asking on this stack who dealt with this often. thanks a lot!

Solution 1

In order to match "undefined" at the end of the URL-path you need the regex pattern undefined$. The pattern ^undefined$ (which you've used in your question) matches the exact URL "undefined", which is never going to match, unless the request is for http://example.com/undefined.

This directive should go at the top of your .htaccess file (after the RewriteEngine On directive). You only need one RewriteEngine On directive at the top of your script.

So, if you are sure that these requests are coming from your site then you could serve a 204 No Content as you are doing (yes, the L flag is required).

RewriteRule undefined$ - [R=204,NC,L]

In compliant browsers, as far as the user is concerned, literally nothing happens if the user follows such a link (if this really is a link they are clicking on). However, this can obviously be confusing for the user. Also, if this is an external website that is linking to you, then again, nothing happens when the user clicks the link - they don't reach your website.

However, it would be preferable to redirect to the correct URL (if it's simply a case of removing the "undefined" portion from the end of the URL-path). This would catch all situations and inform search engines of the correct URL (reports in the linked articles suggest that Googlebot is also crawling these URLs).

RewriteRule (.*)/undefined$ /$1 [R=301,L]

Solution 2

If you're getting alot of requests with undefined added to the URLs and you know the things accessing those URLs are people and not robots, you're much better off using HTTP status code 301 and redirecting the URL to the correct one.

Using status code 204 will not help because it means "No content" and the user will then need to manually modify the URL in the address bar to the correct one to access the page. Depending on the browser, a request returning a status codes 204 will either produce a pop-up message indicating the document has no content or the screen will simply be blank.

Comments

[MrWhite]

"redirect 204"? Can you please give an example of a URL that contains this "undefined" string. If "undefined" is part of the URL-path (which is what you are testing with RewriteRule) then I'm surprised your server is responding with a valid response (which makes me think it's not in the URL-path)? Btw: ^undefined$ is an exact match. undefined$ will match at the end of the string.

[somuch72]

here's the example: example.com/test/undefined. the real page without "undefined". here's the problem i had, but more severe stackoverflow.com/questions/11017609/… the undefined page goes to 404. this rewrite, where do i have to put it? inside wordpress part of htaccess?

[somuch72]

Sorry, it is 204. i put wrong code on question, just edited it. Yes, it will trigger 404, but the undefined page is around 20% of my pageviews, so it's better if it can be stopped.

[MrWhite]

204? A 204 is a success code, not a redirection. (?) Is a 204 something you are seeing or is what you are trying to respond with? R=204 does not make sense.

[somuch72]

I see your point, why i choose 204 because i want when the visitor somehow click the undefined page url, nothing will happen. So the visitor will stay in the current page, going nowhere.

[somuch72]

response 204. sorry, i'm pretty new with this code :D

[MrWhite]

Do you really think these are internal links? Do your access logs (HTTP Referer) agree with this? The redirect to the "correct" URL might be the best thing to do (see my update) - since this would seem to be easy to do if it's just a case of removing the "undefined" string.

[somuch72]

The problem is the real page is the referer. Example.com/test/undefined refered by example.com/test/ I know this is weird, but this is real problem. It's been discussed several time but there's no real solution. If it's just 1%-2% i can ignore it, but on mine it's up to 20%. This caused by real visitor, not bot or spam, so i can't block the IP. another discussion about this productforums.google.com/forum/#!topic/chrome/G1snYHaHSOc I'll try both code as i don't know which one will suit better with this situation

[somuch72]

The referer for undefined pages are its real page, so with 204 it will stay at that page. There's no need to redirect to the correct one because it's already on the correct page. I know this is weird, but it's real. I checked on raw access log several time, all undefined page come from firefox on various version. Now i'm using this RedirectMatch 204 (.*)/undefined$

[Mike -- No longer here]

I apologize if you don't like my opinion but its a poor user experience to serve empty pages with an HTTP 204 status code.

[somuch72]

It's okay, every opinion is important, i will try 301 as well, i don't know which one will suit better. I'm new with this kind of thing, it should be me that have to apologize, sorry. Thanks!

[MrWhite]

Sorry, I see what you are saying now regarding the 204 - this might actually be OK in this situation. However, a 301 redirect would still be preferable IMO. I've updated my answer. The sites you reference all suggest that this is a client-side issue, possibly caused by a rogue browser plugin. In this situation the user is not actually clicking a link - in fact, they don't see any error at all. It's the script/plugin that is making the request in the background. The user is already on your site, so you may not be losing any traffic because of this!?

[somuch72]

Thank you for the update. My best guess this is not on my site's error. I think the users not noticing it, as it happen for serveral minutes long after they browse the site. I can conclude this is not bot because i see the pattern of human browsing a site, but the undefined seems not human, with a few seconds range to the same request of undefined. No, not losing any traffic at all, but this causing the server resource's usage to go up, and sometimes spiking if it happen at once.