How to debug slow browsing speed?

12.04 networking internet

6,604

Use `curl` to find out which part of the process of loading a website is giving you trouble

In general, you can debug "browsing" (or http/s) by using curl with the -w option.

Open the terminal, and sudo apt-get install curl (if you don't have it already)

Create a file called curl-timing.cfg in say, your home directory. In it, paste:

\n
      DNS lookup                          :  %{time_namelookup}\n
      Connect to server (TCP)             :  %{time_connect}\n
      Connect to server (HTTP/S)          :  %{time_appconnect}\n
      Time from start until transfer began:  %{time_pretransfer}\n
      Time for redirection (if any)       :  %{time_redirect}\n
      Total time before transfer started  :  %{time_starttransfer}\n
\n
             Total time                   :  %{time_total}\n
             Size of download (bytes)     :  %{size_download}\n
             Average d/l speed (bytes/s)  :  %{speed_download}\n
\n

Then, try downloading any URL, suppose google, with:

curl -w "@curl-timing.cfg" -o /dev/null -s http://www.google.com/

and the output will be of the form:


  DNS lookup                          :  0.012
  Connect to server (TCP)             :  0.031
  Connect to server (HTTP/S)          :  0.000
  Time from start until transfer began:  0.031
  Time for redirection (if any)       :  0.000
  Total time before transfer started  :  0.098

         Total time                   :  0.117
         Size of download (bytes)     :  14527
         Average d/l speed (bytes/s)  :  124347.000

Compare to fetching a webpage from an Indian server to the US:

$ curl -w "@curl-timing.cfg" -o /dev/null -s http://india.gov.in

  DNS lookup                          :  0.377
  Connect to server (TCP)             :  0.716
  Connect to server (HTTP/S)          :  0.000
  Time from start until transfer began:  0.716
  Time for redirection (if any)       :  0.000
  Total time before transfer started  :  1.974

         Total time                   :  3.650
         Size of download (bytes)     :  34345
         Average d/l speed (bytes/s)  :  9408.000

This will tell which step(s) are the weak link. Try with multiple websites, and even file downloads. If curl gives you good results overall, the problem may lie with your browser/application, instead of TCP/HTTP and your network connection.
For more options, see the curl man page. Search for --writeout to quickly jump to the relevant section.

6,604

Binoy Babu

Updated on September 18, 2022

Comments

Binoy Babu almost 2 years

I'm using Ubuntu 12.04, Chromium 18.0.1025.151 (Developer Build 130497 Linux) Ubuntu 12.04, Firefox 12.0 but I'm getting poor browsing speed. The browser always shows "Waitng for aaa.bbb.com" at the bottom of the browser. How can I debug this and find out the reason? I'm connecting to 3G network via Bluetooth PAN network of an Android device. Some stats below.

$ dig asdsa.com

; <<>> DiG 9.8.1-P1 <<>> asdsa.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39878
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 13, ADDITIONAL: 13

;; QUESTION SECTION:
;asdsa.com.         IN  A

;; ANSWER SECTION:
asdsa.com.      307707  IN  A   200.31.91.194

;; AUTHORITY SECTION:
com.            34662   IN  NS  e.gtld-servers.net.
com.            34662   IN  NS  a.gtld-servers.net.
com.            34662   IN  NS  k.gtld-servers.net.
com.            34662   IN  NS  h.gtld-servers.net.
com.            34662   IN  NS  g.gtld-servers.net.
com.            34662   IN  NS  l.gtld-servers.net.
com.            34662   IN  NS  j.gtld-servers.net.
com.            34662   IN  NS  c.gtld-servers.net.
com.            34662   IN  NS  b.gtld-servers.net.
com.            34662   IN  NS  m.gtld-servers.net.
com.            34662   IN  NS  d.gtld-servers.net.
com.            34662   IN  NS  f.gtld-servers.net.
com.            34662   IN  NS  i.gtld-servers.net.

;; ADDITIONAL SECTION:
a.gtld-servers.net. 50538   IN  A   192.5.6.30
a.gtld-servers.net. 59333   IN  AAAA    2001:503:a83e::2:30
b.gtld-servers.net. 40208   IN  A   192.33.14.30
b.gtld-servers.net. 45531   IN  AAAA    2001:503:231d::2:30
c.gtld-servers.net. 45531   IN  A   192.26.92.30
d.gtld-servers.net. 45526   IN  A   192.31.80.30
e.gtld-servers.net. 45526   IN  A   192.12.94.30
f.gtld-servers.net. 40177   IN  A   192.35.51.30
g.gtld-servers.net. 35936   IN  A   192.42.93.30
h.gtld-servers.net. 40177   IN  A   192.54.112.30
i.gtld-servers.net. 45526   IN  A   192.43.172.30
j.gtld-servers.net. 45526   IN  A   192.48.79.30
k.gtld-servers.net. 35936   IN  A   192.52.178.30

;; Query time: 227 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Jun  6 22:40:19 2012
;; MSG SIZE  rcvd: 499


$ ping google.com
PING google.com (74.125.236.199) 56(84) bytes of data.
64 bytes from maa03s17-in-f7.1e100.net (74.125.236.199): icmp_req=1 ttl=54 time=173 ms
64 bytes from maa03s17-in-f7.1e100.net (74.125.236.199): icmp_req=2 ttl=54 time=162 ms
64 bytes from maa03s17-in-f7.1e100.net (74.125.236.199): icmp_req=3 ttl=54 time=228 ms
64 bytes from maa03s17-in-f7.1e100.net (74.125.236.199): icmp_req=4 ttl=54 time=236 ms
64 bytes from maa03s17-in-f7.1e100.net (74.125.236.199): icmp_req=5 ttl=54 time=162 ms
64 bytes from maa03s17-in-f7.1e100.net (74.125.236.199): icmp_req=6 ttl=54 time=215 ms
64 bytes from maa03s17-in-f7.1e100.net (74.125.236.199): icmp_req=7 ttl=54 time=218 ms
64 bytes from maa03s17-in-f7.1e100.net (74.125.236.199): icmp_req=8 ttl=54 time=232 ms
64 bytes from maa03s17-in-f7.1e100.net (74.125.236.199): icmp_req=9 ttl=54 time=212 ms
^C
--- google.com ping statistics ---
9 packets transmitted, 9 received, 0% packet loss, time 8007ms
rtt min/avg/max/mdev = 162.119/204.385/236.105/28.286 ms

Also is there a browser that provides more verbose while loading a page?

Edit : more info from the answer of @izx

$ curl -w "@curl-timing.cfg" -o /dev/null -s http://www.google.com/

      DNS lookup                          :  0.363
      Connect to server (TCP)             :  0.512
      Connect to server (HTTP/S)          :  0.000
      Time from start until transfer began:  0.512
      Time for redirection (if any)       :  0.000
      Total time before transfer started  :  0.749

             Total time                   :  0.749
             Size of download (bytes)     :  221
             Average d/l speed (bytes/s)  :  294.000

$ curl -w "@curl-timing.cfg" -o /dev/null -s http://india.gov.in

      DNS lookup                          :  0.351
      Connect to server (TCP)             :  0.526
      Connect to server (HTTP/S)          :  0.000
      Time from start until transfer began:  0.526
      Time for redirection (if any)       :  0.000
      Total time before transfer started  :  0.770

             Total time                   :  1.643
             Size of download (bytes)     :  34346
             Average d/l speed (bytes/s)  :  20905.000

Dr_Bunsen about 12 years

start it with the terminal?! and if that aint work add the parameter -debug or -dev, I have no idea if this works, but its worth a try.
jackweirdy about 12 years

The problem probably lies in the fact that you're connected through a PAN of a mobile device which is over 3G, which can be slow at times anyway. The slow pings from google kinda suggest that too.
fluffy about 12 years

Also look at the output of netstat -an to see what open, bandwidth-chewing connections you might have without being aware of it (such as Ubuntu's automatic update mechanism, a forgotten torrent client, etc.)
Binoy Babu about 12 years

@jackweirdy I'm not talking about just slow browsing. It loads forever sometimes and when I click go again loads instantly. About slow pings, I'm from India.
Binoy Babu about 12 years

@fluffy Nothing is using up bandwidth

fluffy about 12 years

Wow, curl really IS a swiss-army knife at this point. I didn't know about the -w option. Handy!
Binoy Babu about 12 years

@izx please see edited question. The output seems fine. Also elinks work fine. When using the affected browser, same page would either load fast (normal) or keep loading forever. What is the browser(Chromium) doing while it displays "Waiting for <host>..."?
ish about 12 years

Please try it with the sites that are giving you problems - I just used those two as an example. I don't (regularly) use chromium, so my guess would be it is waiting for the connection to be established...the Connect to Server (TCP/HTTP) part.
phil294 almost 8 years

Cool thing! But for me, it says: DNS lookup: 8.5 sec / Connect to server (TCP): 9.0 sec / Total time: 9.6 sec" This doesn't fit together. (?)