Can't connect to certain HTTPS sites

12.04 ssl https

43,253

Solution 1

Thanks for all your help , the problem is finally fixed!

I was trying to limit the MTU to see if it would help and ended up using pppoeconf to set up the PPPoE connection since it limits the MTU for me. I then disabled the DSL connection I previously used.

For anyone experiencing a similar problem you can try this solution out by typing sudo ppoeconf and following the instructions. Then you can connect with pon adsl-provider and disconnect with poff

Solution 2

This is an old question, but for those getting here via Google, this will help. The issue is fragmentation on SSL is bad, and breaks the protocol. If you are using PPPOE, the normal MTU in your router/DSL/Cable modem is 1492. That is too high, and will cause fragmentation. 1476 is the magic number that will work with the most sites. Some sites use different SSL implementations so 1480 may work, or even 1488. For the MOST compatibility, the MTU on the WAN side of your network device (router, modem, etc) should be 1476.

Solution 3

Here are a couple of things to try:

Check your network card settings. Neither of your eth interfaces are showing IPv4 addresses. Make sure you have IPv4 turned on (you may need to re-establish your connection with your router to renew the IP). If that doesn't work, try turning off IPv6 support and see if that makes a difference. Do this by right clicking the networking icon by your clock (when on an Ethernet connection, it's a pair of arrows, one pointing up, the other down) and selecting "Edit Connections...". In the "IPv4 Settings" tab, make sure it's set to "Automatic (DHCP)". If you want to turn off IPv6, go to its tab and set it to "Ignore".
Check to see if you can connect to the sites using other methods. What does ping respond with for the sites you can't connect to? How about a traceroute (you may have to install traceroute to use it, FYI)? Their responses might help you troubleshoot the issue. If they can't get to the URL's servers, then it might be a DNS issue (however, if they can get to the URL's servers, but are then dropped, it might just mean those commands are blocked).
Bypass the router. If your router and your modem are two different machines, try hooking up your computer directly to your modem and seeing if that changes anything.
Restart your modem and router. Sometimes, they just suck.
Restart your computer. Sometimes, they just suck.
Try a different computer. If you have one, does another computer work where this one fails? If not, then it might be something with your specific computer.
Clear your computer's cache, cookies, etc. Sometimes, bad sessions cookies, cache, etc. can interfere with connecting to a site (I had this issue with Google a while back). Clear them out and start fresh and see what you get.
Disconnect any VPN connections. Point-to-Point protocol is often used for VPN (the PPP interface), and VPNs can interfere with connecting to sites. Make sure you aren't connected by right clicking your network icon by your clock, finding the "VPN Connections" entry and making sure no listing are checked (if you don't have a "VPN Connections" menu item, then you don't have one set up). If there are any checked, then you're connected to it, disconnect from it.

Remember: Not everything you do will result in a simple "work or fail," any change in the reaction of the server to your request will tell us something. So, if you do any of the above and get a new message, don't forget to update your question.

Solution 4

I have seen this behaviour twice in practice for which I have found the following solutions.

Some computer in the local network was successfully attempting a man-in-the-middle attack. It was ARP-spoofing the gateway, thus redirecting all traffic to go through this machine, modifying requests and other nasty stuff. The machine was running Windows and found to be infected with some nasty malware. As soon as this machine was disconnected from the network physically, symptoms vanished.
An MTU problem on your or another gateway. In IPv4 gateways are responsible for fragmenting and re-assembling IP packets on the network if the frame size of the networks it's routing traffic for isn't the same. For DSL connections using PPPoE/PPPoA the MTU size is usually smaller than the 1500 bytes at the LAN side. Also routers in between fail and you need to enable TCP MSS Clamping on your router. I always needed to set this on the connection of my previous ISP, but it was resolving more than just SSL-related issues. Check if your modem/router has such an option. Consider this to be a workaround.
I was in a network probably running a transparent proxy to also pass SSL traffic, but failing at TLSv1 for some reason. The same request worked when using a VPN connection. scary
Try running curl with the option --sslv3. If that solves it, then it stinks.

General stuff to try:

Check if you're running the latest firmware on your modem/router. If not, try upgrading.
Capture the traffic using tcpdump or Whireshark and get it analysed (post it here for example).
```
  # 1. start the dump
$ sudo tcpdump -w httpstrafficdump.pcap -i eth0 -s 0 port 443
  # 2. open a new terminal window and do your HTTPS request there (curl/browser)
  # 3. end tcpdump (Ctrl+C)
  # 4. open the file in wireshark
$ wireshark httpstrafficdump.pcap
```
If you're getting Reassembling errors or Previous segment lost repeatedly, this is a clear sign about packet loss caused by a wrong MTU size.
However, HTTPS traffic is encrypted and hard to analyse from network traffic by itself.

Edit:

From your tcpdump the root of your SSL problem is clear: TCP Previous segment lost. General networking troubleshooting should apply here, but it may be outside the scope of your local network and a problem with your ISP.

View more solutions

43,253

Hassan Aftab

Updated on September 18, 2022

Comments

Hassan Aftab over 1 year

I've just moved to a new apartment and with internet connection via a router and I'm finding that I can't connect to quite a few sites that use SSL.

For example trying to connect to PayPal:

curl -v https://paypal.com
* About to connect() to paypal.com port 443 (#0)
*   Trying 66.211.169.3... connected
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* Unknown SSL protocol error in connection to paypal.com:443 
* Closing connection #0
curl: (35) Unknown SSL protocol error in connection to paypal.com:443

curl -v -ssl https://paypal.com gives the same output.

For some sites it works:

curl -v https://www.google.com
* About to connect() to www.google.com port 443 (#0)
*   Trying 74.125.235.112... connected
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-RC4-SHA
* Server certificate:
*    subject: C=US; ST=California; L=Mountain View; O=Google Inc; CN=www.google.com
*    start date: 2011-10-26 00:00:00 GMT
*    expire date: 2013-09-30 23:59:59 GMT
*    common name: www.google.com (matched)
*    issuer: C=ZA; O=Thawte Consulting (Pty) Ltd.; CN=Thawte SGC CA
*    SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Host: www.google.com
> Accept: */*
> 
< HTTP/1.1 302 Found
< Location: https://www.google.co.jp/
  .
  .
  .

I'm using Ubuntu 12.04, with Windows 7 installed as well. These sites work on Windows :(

Not sure if this information helps but I ran ifconfig and got the following:

eth0      Link encap:Ethernet  HWaddr 1c:c1:de:bc:e2:4f  
          inet6 addr: 2408:c3:7fff:991:686b:8d18:81b3:8dd1/64 Scope:Global
          inet6 addr: 2408:c3:7fff:991:1ec1:deff:febc:e24f/64 Scope:Global
          inet6 addr: fe80::1ec1:deff:febc:e24f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:87075 errors:0 dropped:0 overruns:0 frame:0
          TX packets:54522 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:78167937 (78.1 MB)  TX bytes:10016891 (10.0 MB)
          Interrupt:46 Base address:0x4000 

eth1      Link encap:Ethernet  HWaddr ac:81:12:0d:93:80  
          inet6 addr: fe80::ae81:12ff:fe0d:9380/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:498
          TX packets:0 errors:26 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:17 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:630 errors:0 dropped:0 overruns:0 frame:0
          TX packets:630 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:39592 (39.5 KB)  TX bytes:39592 (39.5 KB)

ppp0      Link encap:Point-to-Point Protocol  
          inet addr:180.57.228.200  P-t-P:118.23.8.175  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1492  Metric:1
          RX packets:39631 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22391 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3 
          RX bytes:43462054 (43.4 MB)  TX bytes:2834628 (2.8 MB)

I've run PING:

ping www.paypal.com
PING e6166.b.akamaiedge.net (184.31.66.234) 56(84) bytes of data.
64 bytes from a184-31-66-234.deploy.akamaitechnologies.com (184.31.66.234): icmp_req=1 ttl=54 time=15.3 ms
64 bytes from a184-31-66-234.deploy.akamaitechnologies.com (184.31.66.234): icmp_req=2 ttl=54 time=15.0 ms
64 bytes from a184-31-66-234.deploy.akamaitechnologies.com (184.31.66.234): icmp_req=3 ttl=54 time=15.2 ms
64 bytes from a184-31-66-234.deploy.akamaitechnologies.com (184.31.66.234): icmp_req=4 ttl=54 time=17.2 ms
64 bytes from a184-31-66-234.deploy.akamaitechnologies.com (184.31.66.234): icmp_req=5 ttl=54 time=16.6 ms
64 bytes from a184-31-66-234.deploy.akamaitechnologies.com (184.31.66.234): icmp_req=6 ttl=54 time=16.7 ms
64 bytes from a184-31-66-234.deploy.akamaitechnologies.com (184.31.66.234): icmp_req=7 ttl=54 time=14.8 ms
^C
--- e6166.b.akamaiedge.net ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 6009ms
rtt min/avg/max/mdev = 14.878/15.890/17.214/0.901 ms

And without www:

ping paypal.com
PING paypal.com (66.211.169.66) 56(84) bytes of data.
^C
--- paypal.com ping statistics ---
303 packets transmitted, 0 received, 100% packet loss, time 302265ms

TRACEROUTE:

traceroute www.paypal.com
traceroute to www.paypal.com (184.31.66.234), 30 hops max, 60 byte packets
 1  118.23.8.175 (118.23.8.175)  8.424 ms  8.404 ms  8.540 ms
 2  118.23.10.121 (118.23.10.121)  8.212 ms  8.189 ms  8.162 ms
 3  122.1.164.213 (122.1.164.213)  9.405 ms  11.359 ms  13.469 ms
 4  60.37.55.165 (60.37.55.165)  8.049 ms  8.072 ms  8.040 ms
 5  118.23.168.89 (118.23.168.89)  8.574 ms  8.549 ms  8.558 ms
 6  210.163.230.238 (210.163.230.238)  8.667 ms  7.605 ms  7.545 ms
 7  xe-4-0-0.a21.osakjp01.jp.ra.gin.ntt.net (61.213.169.218)  18.255 ms  18.232 ms xe-3-0-0.a21.osakjp01.jp.ra.gin.ntt.net (61.213.162.206)  19.042 ms
 8  * * *
 9  * * *
   .
   .
   .
29  * * *
30  * * *

without www:

traceroute paypal.com
traceroute to paypal.com (66.211.169.66), 30 hops max, 60 byte packets
 1  118.23.8.175 (118.23.8.175)  5.607 ms  5.674 ms  5.875 ms
 2  118.23.10.121 (118.23.10.121)  5.468 ms  5.453 ms  5.576 ms
 3  122.1.164.213 (122.1.164.213)  7.595 ms  10.062 ms  11.660 ms
 4  60.37.55.165 (60.37.55.165)  5.684 ms  5.660 ms  5.635 ms
 5  60.37.27.90 (60.37.27.90)  5.960 ms  5.924 ms  5.898 ms
 6  ae-11.r20.tokyjp01.jp.bb.gin.ntt.net (129.250.12.197)  86.468 ms  30.960 ms  30.899 ms
 7  as-1.r20.sttlwa01.us.bb.gin.ntt.net (129.250.4.189)  161.185 ms  144.343 ms  132.410 ms
 8  ae-1.r05.sttlwa01.us.bb.gin.ntt.net (129.250.5.47)  139.008 ms  127.377 ms  139.050 ms
 9  xe-0.sprint.sttlwa01.us.bb.gin.ntt.net (129.250.9.190)  116.006 ms  104.306 ms  115.954 ms
10  144.232.1.153 (144.232.1.153)  141.046 ms  129.870 ms  140.991 ms
11  sl-crs2-sj-0-5-2-0.sprintlink.net (144.232.18.204)  131.271 ms  131.248 ms  142.544 ms
12  sl-st31-sj-0-15-0-0.sprintlink.net (144.232.8.151)  129.543 ms  141.575 ms  141.066 ms
13  * * *
14  * * *
    .
    .
    .
29  * * *
30  * * *

The tcpdump:

1   0.000000    114.178.88.59   66.211.169.66   TCP 76  37374 > https [SYN] Seq=0 Win=14520 Len=0 MSS=1452 SACK_PERM=1 TSval=68855 TSecr=0 WS=64
2   0.136291    66.211.169.66   114.178.88.59   TCP 80  https > 37374 [SYN, ACK] Seq=0 Ack=1 Win=4356 Len=0 MSS=1460 WS=1 TSval=3608913175 TSecr=68855 SACK_PERM=1
3   0.136322    114.178.88.59   66.211.169.66   TCP 68  37374 > https [ACK] Seq=1 Ack=1 Win=14528 Len=0 TSval=68889 TSecr=3608913175
4   0.137409    114.178.88.59   66.211.169.66   SSL 309 Client Hello
5   0.274446    66.211.169.66   114.178.88.59   SSL 95  [TCP Previous segment lost] Continuation Data
6   0.274469    114.178.88.59   66.211.169.66   TCP 80  [TCP Dup ACK 4#1] 37374 > https [ACK] Seq=242 Ack=1 Win=14528 Len=0 TSval=68923 TSecr=3608913175 SLE=2881 SRE=2908
7   7.117833    91.189.89.76    114.178.88.59   TLSv1   142 Application Data, Application Data
8   7.118823    114.178.88.59   91.189.89.76    TLSv1   216 Application Data, Application Data, Application Data, Application Data
9   7.393725    91.189.89.76    114.178.88.59   TCP 68  https > 41264 [ACK] Seq=75 Ack=149 Win=146 Len=0 TSval=875420654 TSecr=70634
10  60.301444   66.211.169.66   114.178.88.59   TCP 56  https > 37374 [RST, ACK] Seq=2908 Ack=242 Win=4597 Len=0

This is a Japanese ISP and even though I'm connecting to with a cable to the modem / router I need to add a username and password, but with Ubuntu's "Wired" connection I couldn't add these. My housemate told me to create an OCN connection but I'm not sure if that's a name of a type of network or just the Japanese company... but after looking at her computer we found it to be a PPPoE connection. After some googling I learned that to create a PPPoE connection I would need to create a DSL connection and that I could add a password & username to it. I also changed the "Wired" connection to not connect automatically.

I get the same problem if I hook up to the modem directly.

I've tried changing the DSL MTU to 500, 1500, 1492 and 1482 but it didn't make a difference.

Also for some reason Ubuntu doesn't always pick up the connection, I sometimes have to restart for it to connect.

adempewolff over 11 years

are these sites only not opening with curl or are they not opening with other browsers as well?
Hassan Aftab over 11 years

They're not opening at all, I was just trying to find out where the problem is...
Shauna over 11 years

@mind.blank - What does the browser give you when you try to connect? If you don't get anything from the main window, try installing Firebug (if you use Firefox; if you use Chrome, it has the developer tools built in), open it, and activate the console and net tabs and see if there are any errors, then report them here.
adempewolff over 11 years

Were you using the router before or is it new? Also, you haven't done anything silly and upgraded your ssl packages or anything from a default install? I'm accessing all of those sites fine (well, through my proxy at least, China randomly blocks ssl protocol otherwise), so I'm guessing the problem lies either with the router or with an upgraded/installed package.
Hassan Aftab over 11 years

@adempewolff the router is new. At least new to this computer since I just moved in to the apartment - never had this problem on this laptop before... I haven't changed SSL although after all these problems I tried to update them but it said they were up to date already...
gertvdijk almost 11 years

Similar issue: Why can I not access imgur.com and gravatar.com from Ubuntu but can do so from Windows?

Hassan Aftab over 11 years

1) Sorry not sure how to do both these things. cyberciti.biz/faq/setting-up-an-network-interfaces-file - not sure what IP addresses to add. 2) I've added both of those in the original question. 3) I'll see if I have that option tomorrow (modem etc. is in roomate's room). 4) Same as above although I have restarted the router at least. 5) Tried a few times. 6) I don't have another comp with ubuntu, these connections work however when I switch to Windows. 7) Tried that.
Hassan Aftab over 11 years

I've tried running curl with --sslv3 and it still doesn't work. Also I tried to capture the dump but it doesn't seem to be working? tcpdump: WARNING: eth0: no IPv4 address assigned 0 packets captured 6 packets received by filter 0 packets dropped by kernel - I'm not sure how to assign the IPv4... I will have to try the rest tomorrow as it's getting late and my brain isn't working well. Thanks for your help everyone so far!
gertvdijk over 11 years

@mind.blank For you it is ppp0 interface instead of eth0 which makes me think: Why do you need PPP for a connection when using a router?
Shauna over 11 years

@mind.blank You don't need to do anything of the sort in the link. Simply right click the networking icon by the clock and select "Edit Connections...". Click on the connection and select "Edit" and select the "IPv4 Settings" tab and make sure it's set to "Automatic (DHCP)". Then go to the "IPv6 Settings" tab and make sure "Require IPv6 addressing for this connection to complete" is unchecked. Save and reconnect. If/when you want to turn off IPv6, then come back to the IPv6 Settings tab and change "Automatic" to "Ignore".
Hassan Aftab over 11 years

In Network Connections > DSL > Edit, IPv4 settings are on "Automatic (PPPoE)" and there is no IPv6 tab...
gertvdijk over 11 years

@mind.blank The dump is very useful, but doesn't point to a solution just. See my updated answer.
gertvdijk almost 11 years

I don't see how fragmentation will break SSL. Fragmented packets will be reassembled by routers in IPv4. SSL happens on top of TCP, on a level where you should not notice this. I do think this has to do with MTU values, but I don't think your explanation qualifies. I think it has to do with MTU settings not in sync on both ends, resulting in wrong reassembling of fragmented packets. The magic number may work in your case, but not for others.
Rookie almost 11 years

The DF (Dont Fragment) bit is always set on SSL traffic by design - fragmentation is a security hole. A fragmented SSL packet will get dropped 99.9% of the time. An MTU too low on PPoE traffic will cause fragmentation.
playful almost 9 years

This happened to me with the PayPal website. I tried to settle with MTU 1476 and did not work, but with 1480 worked. Thank you!
WayBehind about 8 years

I spent 6am-2pm trying to solve this until I found your response. Much appreciated!
jwd630 over 7 years

holy cow! This enabled me to sudo yum install docker-engine after adding the yum.dockerproject.org repo on a CentOS 7 box: sudo ip link set mtu 1476 dev enp6s0 to lower the MTU from its default of 1500 to 1476. Been scratching my head for a day trying to figure out why yum.dockerproject.org was accessible via https from other nodes on the same network.