Why is my 301 Redirect taking so long?

Question

In a long tiredsome quest to speed up my site, I have figured out something is wrong with the redirection: currently my index.php handles all the homepage redirections via PHP header location 301 Redirect Permanently: website.com >> website.com/en/home and website.de >> website.de/de/home etcettera etcettera (around 20 for this multilingual website) it takes anywhere from 200ms to 6000ms to do the redirecting. Check out the waterfall!

After that, the page loads in a thunderbolt's blink of an eye! What a waste of time wouldn't you say? What is the server doing all this time? After careful examination, my best guesse is: ITS DOING LAUNDRY!

I am almost giving up on PHP for this! Any and all clues to my puzzling prob are very welcomed +1

A. Given facts: Apache/2.0.54 Fedora, PHP 5.2.9. there is no database: just flat php files with around 15 php includes that completes my page with headers and footers). YSlow Grade: 92/100! Good page Speed: 93/100! javascript and css are as much as possible combined. Cache controlls seem well set too (as proven by the grades). Whats missing in those 7 points out of 100: not using Keep-Alive (beyong my controll in shared hosting and not using Content Delivery Network. I can live with those missing 7 points, but this is major hit on speed!

B. Furthermore: i recently was given great insights over here that i should use url rewriting via htacces. Point taken, BUT, perhaps there is sometin else wrong here that i should correct before moving on to the for me more difficult apache regex syntaxes.

C. Faster way: When I php include the intended homepage, instead of redirect, then all loads fast, but the url is not rewritten: it sits at website.com on the browser bar, whereas i wish after including it to become website.com/en/home. Is this possible with PHP? To include+change the current address of the url, too?

Conclusions: you can redirect using index.php, or using .htaccess. Sofar from my tests (coming from the genius answers below!THANKS EVERYONE!) the latter seems unmatched in speed: much faster redirecting than a php redirect! reducing the redirect to shorter than the first dns lookup.

see here how to do this correclty for multilingual site

Solution 1

Damn, I hate getting stuck with this kind of problem. You need to eliminate some variables.

First I should point out that PHP will not flush all of its own headers until you start outputting things (or, if the output_buffering(?) ini directive is set to x bytes, until you have output x bytes). So the following script will not finish "sending headers" until the very end:

<?php
header('Content-Type: text/pants');
sleep(6);
header('Ding-Ding: time to put the socks in the dryer');
echo "z"; // headers are sent here

What happens to the call to en/home if you put exit; or echo "wheeeee"; exit; at the very top of that PHP script? Then what happens when you substitute it with a plain, empty file? If the php script with exit is slow but the plain text file is fast, the PHP interpreter is probably playing funny buggers. If you still get the delay for both, you've eliminated the actual response generation as the cause (but I'm still trying to come up with some ideas if this is the case).

Also, can you ssh to the server? If so, can you try wgetting the same page from inside the server? If you can without the speed problem, I would be looking at the client side. If you can't SSH, you could try doing a request from PHP, though I'm really not sure if this will work:

<?php
$context = stream_context_create(array(
    'http'=>array(
        // send request headers if you need to
        'header'=>array(
            'Foo: Bar',
            'Bar: Baz',
        ),
    ),
));
$start = microtime(true);
$response = file_get_contents('http://yourserver.com/', null, context);
$end = microtime(true) - $start;
var_dump($end);

// for some bizarre reason, PHP emits this variable into the local scope.
var_dump($http_response_header);

Have you tried doing the same request from other machines, or other places in the world? This can confirm or deny if it's just your machine.

Another thing you can try if it is the response generation is to do a little bit of hack-profiling on the production server. I hate having to do this stuff, but sometimes your code just refuses to behave on the production server like it behaves in your development environment or on staging. Do this to the script that generates /en/home:

<?php
// put this at the very top
$rqid = uniqid('', true);
$h = fopen(__DIR__.'/crap.log', 'a');
fwrite($h, $rqid.' [START] '.microtime(true).PHP_EOL);
fclose($h);

// do all that other wonderful stuff, like laundry or making a cup of tea

// put this at the very end
$h = fopen(__DIR__.'/crap.log', 'a');
fwrite($h, $rqid.' [END]   '.microtime(true).PHP_EOL.PHP_EOL);
fclose($h);

Run a few requests against it, check to make sure 'crap.log' is getting stuff written to it (check permissions!!), and then you'll have some data that will show whether there is something in your script that needs to be investigated further as the cause of the slowness.

Oh, did I mention MySQL indexes? Are you doing any queries during the request? Have you added all of the proper indexes to the tables?

Steven Xu raises a good point in the comments for your question - are you sure the program you're using to generate the waterfall is giving you good info? Try installing Firebug if you haven't already, click the little firebug icon in the bottom right of firefox and make sure the "Net" panel is open, then re-run your request and see if the waterfall is consistent with the results you're seeing in the program you used.

Also, I know this is kind of a boneheaded suggestion and I apologise, but I think it needs to be said: your host doesn't allow ssh and only uses PHP 4? I would seriously consider another host. It may even solve this specific problem.

I will add more stuff as I think of it.

Solution 2

If it is indeed the headers taking ages, then your JS/CSS/HTML is irrelevant.

You can do the forwarding in .htaccess.

RewriteEngine On
RewriteRule ^$ en/home [R=301]

This will essentially send the same header, but it won't invoke the PHP engine first to do it :)

Update

On closer inspection, it would seem to me that your en/home page is taking the longer time to download.

Solution 3

I think Ignacio Vazquez-Abrams may have the answer: after you call header() to do the redirection you need to call exit() to cause the PHP script execution to stop. Without that the script will keep executing, sending output to the browser, until the end. Since the browser has to wait for the server side script to end before performing the redirection that could cause the problem.

Update

Just read Alex's update and he seems to be correct. The /en/home page is where the time is.

Comments

[ta.speot.is]

en/home page is taking a longer time to start downloading. The delay could be attributed to PHP or the database server spinning up for the first time.

[Sam]

thanks, i updated my question. you see that its a multilingual site, requiring each domain to go to its own home, so an All domains to one home will not suffice im afraid.

[user3167101]

@Sam Ah yeah, I didn't think of that. :)

[Sam]

Now i undertstand what Ignacio was saying, Jeremy: I HAVE to manually stop the script, since it doesnt need to continue running all the rest of the index.php file. Is this proper implementation?: case "website.bla": header('HTTP/1.1 301 Moved Permanently'); header('Location: /bla/home'); exit(); then next case etc etc.

[Jeremy]

Sam, that looks good. Give that a go and see if it works for you. BTW you don't need two calls to header. You can use the second and third arguments to the function to set the type of redirect in one hit, sort of like this: header('Location: /bla/home', true, 301);

[Steven]

@Sam: Is there any reason why you're using the 301 instead of the 302 here (Also, n.b. PHP automatically defaults to a 302 if you specify a location header unless you explicitly declare your HTTP status as you do here)? It seems as if your situation is semantically a 302, not a 301. You have a canonical base URL (the root) that 3xx redirects to your current handler. Today, your CMS is such that /en/home makes sense. Tomorrow, it might not. But / will always be your entrance path. Seems like a 302 to me. But that's a whole 'nother question.

[Sam]

Just read your updated answer Shabbyrobe. very insightfull. piculiar thing that output buffering. Just to be certain: currently in my htacces has this: <ifmodule mod_php4.c> php_value zlib.output_compression 16386 </ifmodule> i guesse thats something else right? My php.ini says: output_buffering = Off i think i can leave it to off right? I dont have access to commandline, is that ssh? Another question: if i do a exit(); right after a match with header location is found, then all the worrying about buffers are past or?

[Shabbyrobe]

You're only using PHP 4? That configuration directive makes PHP gzip the output if the client can handle it. That's a good thing and will be contributing significantly to the high YSlow scores you're getting. The manual for zlib.output_compression (php.net/manual/en/zlib.configuration.php) suggests that if an integer value is used, the output is buffered that many bytes. So what your setting means is that PHP will queue up the first 16kb of data you echo before flushing the headers. I don't think this is the cause of your problem. The buffer flushes when the script execution ends too.

[Jeremy]

Actually, it's 302 redirects that search engines don't like. After a while they begin to think that something fishy is going on (302 is only supposed to be a temporary redirect) and that can affect the page ranking. With a 301 redirect though the search engine will remove the old page from the index and replace it with the one being redirected too. You need to consider how either of these options will affect your ranking. How likely is it that a search engine spider will ever come across the 301 redirect?

[Sam]

Jeremy, please see update in question: do i place the exit(); before or after break; ?

[Jeremy]

It would have to go before the break. Since break terminates the current control statement (your switch) and makes program execution jump to after the end of it you couldn't place the break after the break without always terminating the script after the switch (at least not without some extra code...)