Can't access select https sites on Linux over PPPoE

Question

My internet connection used to be a direct LAN connection to my provider. Back then, everything would load fine on both Windows and Ubuntu (dual boot). However, a while ago they started needing me to dial (PPPoE) using a username and password. Gateway, subnet mask, IP, DNS servers all stayed the same. But since then, I haven't been able to browse certain websites on Ubuntu, even though there have been no such issues on Windows. Some example websites are - Ovi's sign in page (although share.ovi.com loads fine, and nokia.com loads fine), Live Mail (works on Chrome(ium) and Opera but not on Firefox (both 3.6 and 4)) Mozilla Addons website and other random websites.

Some of the websites that don't load show timeout messages and for some websites (like the moz addons one), the browser will keep trying to load without an end (I've left it like that even for hours but not noticed anything different happen).

I have tried changing the DNS servers to public ones. I have even tried booting from a Fedora LiveCD and then changing the DNS to those (and even to the ones of OpenDNS), but the exact same thing happens. What could be inherently wrong with some config within Linux itself that is causing this problem?

Does anyone know why this is happening and how it can be fixed?

Note: This question has been cross-posted on SU, but not gotten any responses.

Update: Just saw here that someone else was having similar problem and solved it by putting a NetworkManager.conf file in /etc/NetworkManager. What needs to be in that file?

Solution 1

You have the symptoms of an MTU problem: some TCP connections freeze, more or less reproducibly for a given command or URL but with no easily discernible overall pattern. A telltale symptom is that interactive ssh sessions work well but file transfers almost always fail. Furthermore pppoe is the number one bringer of MTU problem for home users. So I prescribe an MTU check.

What is it? The maximum transmission unit is the maximum size of a packet over a network link. The MTU varies from transport medium to transport medium, e.g. wired Ethernet and wifi (802.11) have different MTUs, and ATM links (which make up most of the long-distance infrastructure) each have their own MTU. PPPOE is an encapsulated protocol, which means that every packet consists of a few bytes of header followed by the underlying packet — so it lowers the maximum packet size by the size of the header. IP allows routers to fragment packets if they detect that they're too big for the next hop, but this doesn't always work. In theory the proper MTU should be discovered automatically, but this also doesn't always work either. In particular googling suggests that Network Manager doesn't always properly act on MTU information obtained from MTU discovery, but I don't know what versions are affected or what the problematic use cases are.

How to measure it. If you have tracepath from the Linux iputils, run tracepath 8.8.8.8 to see the MTU over the path to Google's DNS server. If your version of traceroute has a --mtu option, run traceroute -n --mtu 8.8.8.8. See Discover MTU between me and destination IP for more options.

Lacking automated tools, you can measure manually. Try sending ping packets of a given size to an outside hosts that responds to them, e.g. ping -c 1 -s 42 8.8.8.8 (on Linux; on other systems, look up the documentation of your ping command). Your packets should get through for small enough values of 42 (if 42 doesn't work, something is blocking pings.). For larger values, the packet won't get through. 1464 is a typical maximum value if the limiting piece of infrastructure is your local Ethernet network. If you're lucky, when you send a too large packet, you'll see a message like Frag needed and DF set (mtu = 1492). If you're not lucky, just keep experimenting with the value until you find what the maximum is, then add 28 (-s specifies the payload size, and there are 28 bytes of headers in addition to that). See also How to Optimize your Internet Connection using MTU and RWIN on the Ubuntu forums.

How to set it (replace 1454 by the MTU you have determined, and eth0 by the name of your network interface)

• As a once-off (Linux): run ifconfig eth0 mtu 1454
• Permanently (Debian and derivatives such as Ubuntu, if not using Network Manager): Edit /etc/network/interfaces. Just after the entry for your network interface (after the iface eth0 … directive), add a line with pre-up ifconfig $IFACE mtu 1454. Alternatively, if your IP address is static, you can add the mtu 1454 parameter to the iface eth0 inet static directive.

• Permanently (Debian and derivatives such as Ubuntu, with or without Network Manager): Create a script called /etc/network/if-pre-up.d/mtu with the following contents and make it world-executable (chmod a+rx):

  #!/bin/sh
  ifconfig $IFACE mtu 1454
  

Further resources

• How to diagnose a reliably unreliable connection? (particularly Mike Pennington's answer) may be of assistance if the simple measure-and-limit approach described here doesn't work.

Solution 2

It appears that the core problem is something to do with SSL. All of your problem URLs are https://.... ones.

I don't see why a change to PPPoE affects this, but perhaps your ISP changed more than one thing at once, and you're blaming the wrong change.

I would try adding a hardware router, one specifically recommended by model number by your ISP. Not only is that likely to negotiate the PPPoE connection exactly as your ISP wants, perhaps it will solve the issue with SSL connections, too.

If it doesn't help your immediate problem, you do still get a few side benefits from it.

First, a hardware firewall adds a layer of security. If you need to allow connections to the machine behind the firewall, see PortForward.com for guides for port forwarding guides every router you're likely to use.

Second, most home routers let you share your Internet connection with multiple PCs.

Solution 3

I had this exact same problem with chromium (and chrome). I assumed it was a webkit issue. I never found a permanent solution but if you google that error code (without the actual values) you'll see many people have the same issue. I could temporarily get it to work by closing the tab that was connected to the particular website and then cleared my cache and cookies and everything.

I never found a solution and have since gone back to firefox.

Comments

[Mussnoon]

You may have something there. But how about Gmail and Yahoo and AOL mail websites? And Twitter? I've specifically tried twitter.com and it works.

[Mussnoon]

In my case, however, it doesn't have anything to do with the browser, I'm sure. I've tried from IE and Safari from an XP virtual machine as well - same results.

[Mussnoon]

I've stopped connecting using network manager and started using pppoeconf. pppoeconf suggested setting mtu to 1452 and it's working fine. Thanks a lot.

[Mussnoon]

Also, if you're on superuser, can you please post the answer there as well so I can accept it there too. Here's link: superuser.com/questions/213264/…

[Santiago]

Hi, thanks for the post. But in your final solution, I cannot do anything with ifconfig, since in my distribution this is obsolete. I only have "ip". How can I do it using "ip"?

[Gilles 'SO- stop being evil']

@Santi I think it's ip link set eth0 mtu 1454 but I'm not an ip expert, check the manual.

[Santiago]

@Gilles thanks! That was it. I need sudo rights though, but that was no problem. Let's hope it helps, I am having problem accessing https repositories (it hangs most of the time) and ssh connection to them doesn't even work. With the same laptop on my institute's network, both protocols work fine.

[Santiago]

@Gilles, another question: With tracepath 8.8.8.8 I get: 31: 192.0.0.2 5.433ms pmtu 1474 Too many hops: pmtu 1474 Resume: pmtu 1474. While with ping -c 1 -s 65512 8.8.8.8, I get: Error: packet size 65512 is too large. Maximum is 65507. What should I set for the mtu?

[Gilles 'SO- stop being evil']

@Santi No idea. Ask a new question.