Linux PHP web server horribly slow when accessed from any windows browser

Question

I have a Linux server (Ubuntu 10.04) running apache2 and PHP. Everything runs fine when accessing a page from any browser from another Linux machine or Mac. But when I try to access a page from any combination of Windows machine and browser I get about a 30 second delay before the page comes back. Accessing a plain old HTML file from the Windows browser runs lickity split. So it seems to be just PHP. MySQL is installed but a simple test page that uses no MySQL is still slow.

I don't think it is DNS because if I hard code the IP address in the URL nothing changes. There doesn't seem to be anything in the log files that I can tell.

What could be causing this behavior on Windows clients?

Solution

Update: some "3rd base" debugging tips;

A fairly brutal way to get some debugging is to strace the running apache process, and this is actually easier because the processes are going to be hung for a while.

This command below will only work as stated if your apache is in prefork mode, but I would assume that it would work vaguely similar in worker mode. (but you would have to spend some time getting a ps and grep to find the thread ids...) anyway I think php requires prefork...

First check that the server is in prefork mode...

root@server-72839:/home/ubuntu# apachectl -V | grep MPM
Server MPM:     Prefork             <----------- prefork mode works with php
 -D APACHE_MPM_DIR="server/mpm/prefork"

install strace

root@server-72839:/home/ubuntu# apt-get install strace

make some request to the server, and then run the following command to trace the syscalls made by the hung process;

 root@server-72839:~# netstat -antp | grep "ESTABLISHED" | grep 80 | while read _ _ _ _ client _ proc; do strace -f -p ${proc#/*} &  done



root@server-72839:~# Process 21570 attached - interrupt to quit
restart_syscall(<... resuming interrupted call ...>) = 0
chdir("/")                              = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fabb16b0000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fabb16ae000

let the page request run through on a windoze box that is slow and then paste all that output back into a pastebin and put the link in a comment.

If you don't have any joy with this, then turn on debugging of php, and httpd Loglevel to debug and paste all that into a pastebin and provide the link....

---

Edit: (ok maybe not definitely) possibly a reverse DNS problem on the apache server. try the following....

make sure you have set HostnameLookups Off on your apache server.

http://httpd.apache.org/docs/2.2/mod/core.html#hostnamelookups

Also check that your resolv.conf is specifying working nameservers;

[root@workstation001 /root]# cat /etc/resolv.conf 
....
nameserver 192.168.1.254       <---- this must work. test it with dig

test nameserver with dig;

 [root@workstation001 /root]# dig @192.168.1.254 www.google.co.uk +short
 www-cctld.l.google.com.
 173.194.67.94

make the nameserver work, by changing it to 8.8.8.8 (google public dns server)

/etc/resolv.conf

 nameserver 8.8.8.8
 nameserver 8.8.8.4

---

Start by checking out the php.log and apache logs files for the connections which are slow, the problem is likely right there in logs.

However if you are sure that this is not a DNS problem (which you can check using the nslookup command line tool) and there is nothing obvious server side then I would then use google chrome in-built developer tools to see the timeline of the page load.

This will tell you which item in the page is taking so long, and whether the delay is during the connection, or during resource loading etc.

You can move on to tools like wget, curl in cygwin, or plain telnet to further investigate from the client side.

Comments

[thinice]

serverfault.com/questions/359515/…

[triawan]

Thanks. On the client nslookup worked fine and sees the server. In the Chrome timeline all I see are big time gaps on the Send Request and Receive Response !Chrome Timeline

[triawan]

A wget on a PHP page from Windows takes over 20 seconds for a file that simply says <?php echo 'hello'; ?>. It is always just waiting for a response from the server. If the file has no php in it then is is fast.

[gokva]

man, that is so to do with DNS. make sure you have HostnameLookups Off in your httpd.conf, put that in your virtualhost as well just to make sure.

[gokva]

i bet 20 quid this is something to do with DNS on the server side.

[triawan]

I'll give it a shot first thing in the morn. Thanks.

[triawan]

Based on the description above I tested DNS using google nameserver and it seems fine. My nameservers seems fine as well. I get the results above. HostnameLookups was already off (by default).

[triawan]

Sorry for ignorance, not sure what the Virtualhost comment means. Also not sure why DNS can be an issue when things work fine when 1) client is Linux or Mac or 2) Even on Windows client reading/response of html file is fast but reading/response of php file is slow.

[gokva]

Yes, it a confusing one. But just to clear up one thing... once the client connects httpd and php do a reverse lookup on the connecting ip address which means that its important that the server dns works correctly. Is there a public URL for this server?

[gokva]

basically, the differences can be the "user-agent" string, sometime the server treats clients differently based on that, the TCP stack (almost never the problem unless you are using authentication or old WinXP), or something that is common with your clients, i.e. that the win clients are also some particular client or network IP that is the problem instead.

[gokva]

if you have some other services running on port 8080, or your ip address is xxx.yyy.zzz.80, then that grep pattern is not specific enough so you might need to iterate this once more if it doesn't work first time

[triawan]

By the way I've seen folks refer to the php.log file. I can't seem to find one in the /var/log anywhere.

[triawan]

Wow. Thanks. Here's the strace during a slow windows access strace When I run the strace the big wait is when it says "Connection Timed Out" right at the beginning of the strace.

[triawan]

It was in prefork mode by the way. Also, when I am logged on to the web server I can dig/ping/nslookup that address 10.32.95.50 just fine.

[gokva]

could you do that again with the -t option set, i missed that one first time; netstat -antp | grep "ESTABLISHED" | grep 80 | while read _ _ _ _ client _ proc; do strace -t -f -p ${proc#/*} & done

[triawan]

Here it is pastebin_with_strace_t_option

[gokva]

basically the first 2 lines of the output indicate that httpd is attempting to make some connection to another server, which is timing out after 20 seconds. 11:54:28 connect(10, {sa_family=AF_INET, sin_port=htons(9000), sin_addr=inet_addroot@csweb:~# ) = -1 ETIMEDOUT (Connection timed out) 11:54:48 close(10) = 0 you can rerun the netstat/strace command again. this time passing the -s0 option to give full details of that request like so ; netstat -antp | grep "ESTABLISHED" | grep 80 | while read _ _ _ _ client _ proc; do strace -s0 -t -f -p ${proc#/*} & done

[gokva]

let us continue this discussion in chat