How do I deal with a certificate based transparent proxy in Arch Linux?

Question

I have setup an Arch Linux VM at work, and my company uses a transparent (cut-through) proxy with a self-signed certificate and requires authentication. Whenever I try to install any packages through pacman I get the following error (e.g. trying to install lynx)

error: failed retrieving file 'lynx-2.8.7-6-x86_64.pkg.tar.xz' from mirrors.kernel.org : SSL certificate problem: self signed certificate in certificate chain

It will throw this error for every mirror I have defined in /etc/pacman.d/mirrorlist. I have similar problems trying to use any other HTTP/S clients such as curl, but I do not have any issues with FTP (primarily because my company only monitors ports 80 and 443).

What I've tried already...

• Placing my companies self signed CA certificate in /usr/share/ca-certificates, and /etc/ca-certificates
• Setting http_proxy and https_proxy to point to my company's proxy server with my credentials
• Talked to the person who manages our proxy server, but he doesn't know anything about Linux, so doesn't have a clue where to begin.

For now, when I need to install something or access the Internet, I'm opening an SSH tunnel to my home server to get around this.

Below is the output of openssl s_client -CApath /etc/ca-certificates -showcerts -connect mirrors.kernel.org:443

CONNECTED(00000003)
depth=0 C = US, ST = California, L = San Jose, O = The Linux Kernel Organization, CN = *.kernel.org
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = US, ST = California, L = San Jose, O = The Linux Kernel Organization, CN = *.kernel.org
verify error:num=27:certificate not trusted
verify return:1
depth=0 C = US, ST = California, L = San Jose, O = The Linux Kernel Organization, CN = *.kernel.org
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=San Jose/O=The Linux Kernel Organization/CN=*.kernel.org
   i:/C=US/ST=MyCompanyState/L=MyCompanyCity/O=MyCompanyName/OU=IT/CN=MyCompanyProxyServer/[email protected]
-----BEGIN CERTIFICATE-----
... blah blah blah
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=California/L=San Jose/O=The Linux Kernel Organization/CN=*.kernel.org
issuer=/C=US/ST=MyCompanyState/L=MyCompanyCity/O=MyCompanyName/OU=IT/CN=MyCompanyProxyServer/[email protected]
---
No client certificate CA names sent
---

Note I could probably try to simply use FTP mirrors instead, but that doesn't really solve the problem for other HTTP/S applications (e.g. curl).

Solution

From the info you provided, it sounds like you added the server certificate to your /etc/ca-certificates file. You need to add the CA certificate instead.

The openssl s_client -showcerts command you ran should have included one or more other certificates after the first one. These are the ones you care about.
If there is more than one certificate after the first, then you might need to add both to your ca-certificates file. It all depends on how the certificate authority is set up on the proxy. However if both are not required, and you add both anyway, it won't hurt.

However, if the openssl s_client -showcerts does not show any certificates other than the first one, you're up a creek. It means the proxy is not sending the signing cert. You could try to contact your network team and ask them for it. Security wise there's absolutely no reason for them not to give it to you (it's the key that must be protected).

---

For example I get the following (yes, since I'm not behind a SSL proxy, I get the real certs, but the concept remains):

# openssl s_client -showcerts -connect mirrors.kernel.org:443
CONNECTED(00000003)
depth=2 C = US, O = "thawte, Inc.", OU = Certification Services Division, OU = "(c) 2006 thawte, Inc. - For authorized use only", CN = thawte Primary Root CA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=US/ST=California/L=San Jose/O=The Linux Kernel Organization/CN=*.kernel.org
   i:/C=US/O=Thawte, Inc./CN=Thawte SSL CA
-----BEGIN CERTIFICATE-----
MIID2zCCAsOgAwIBAgIQZ7xPhWo7v0ZlPnI5mJWMazANBgkqhkiG9w0BAQUFADA8
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMVGhhd3RlLCBJbmMuMRYwFAYDVQQDEw1U
aGF3dGUgU1NMIENBMB4XDTEyMDEwMzAwMDAwMFoXDTE0MDQwMzIzNTk1OVowdDEL
MAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExETAPBgNVBAcUCFNhbiBK
b3NlMSYwJAYDVQQKFB1UaGUgTGludXggS2VybmVsIE9yZ2FuaXphdGlvbjEVMBMG
A1UEAxQMKi5rZXJuZWwub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
AQEApSbajGUzM+C1Oz6CzyPmek9HH03ToicZeHxOlBfS7yyvIyybxTCBrmkItyc+
sf6b5g2Zf8IPwloaj+ACQaLfm0OOaFrvHERIN1t0pitvmeSDI6BXVYq0eSLEPpSG
YC/3AMdKE21NO1jmL7mtsCB8CW4NAAvy97HkvsPsTPNOp6d/LtcIMYRS347RFgRj
O/J1PU05EEjcpXdy6eCyJLwi2AFLgfBzjS+0UK4tq01HFalULfO/akjC59WG+qrc
P/Yi/1TM8bE2mun1mcuCP7bmyZFDfrGnO2TYcZ/s7z4bTUmEixR6kTdj1foqi7Xk
svko9dXCiUZj4c692ZKZXE+gJwIDAQABo4GgMIGdMAwGA1UdEwEB/wQCMAAwOgYD
VR0fBDMwMTAvoC2gK4YpaHR0cDovL3N2ci1vdi1jcmwudGhhd3RlLmNvbS9UaGF3
dGVPVi5jcmwwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMDIGCCsGAQUF
BwEBBCYwJDAiBggrBgEFBQcwAYYWaHR0cDovL29jc3AudGhhd3RlLmNvbTANBgkq
hkiG9w0BAQUFAAOCAQEAd7mKHpW1mCB1fGuvR44iX83PmSy3BVEpvOlnH1zNSU/Y
2zMmKDbXqAJTY6t20Yq/GpjW3BA2G4xEA6i64WAGZcSmXfc/NDOSh0i5lTrCofCG
3tuts9HTmHJLfeAz9cZiT4rc4ROMV4K17/Uw81UIid952M/4b6OvhaCu5OSnvDUI
3Z6OUy+AuJHbwEyB5bGOJ/mqVKUztgUK17bJiwDhwZ4Q8PT6YKUj5NgPcG6cUKxY
HK6yxvoqH/s1DQJB3JHDSFVgY47ECmoHyQ5MvJN+naNrZJUIH0RTmmHNbQH772W7
m8I66jiDhXTd6+4v8DipvHJOSEv7ebG0Jf6gv7lc6w==
-----END CERTIFICATE-----
 1 s:/C=US/O=Thawte, Inc./CN=Thawte SSL CA
   i:/C=US/O=thawte, Inc./OU=Certification Services Division/OU=(c) 2006 thawte, Inc. - For authorized use only/CN=thawte Primary Root CA
-----BEGIN CERTIFICATE-----
MIIEbDCCA1SgAwIBAgIQTV8sNAiyTCDNbVB+JE3J7DANBgkqhkiG9w0BAQUFADCB
qTELMAkGA1UEBhMCVVMxFTATBgNVBAoTDHRoYXd0ZSwgSW5jLjEoMCYGA1UECxMf
Q2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjE4MDYGA1UECxMvKGMpIDIw
MDYgdGhhd3RlLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxHzAdBgNV
BAMTFnRoYXd0ZSBQcmltYXJ5IFJvb3QgQ0EwHhcNMTAwMjA4MDAwMDAwWhcNMjAw
MjA3MjM1OTU5WjA8MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMVGhhd3RlLCBJbmMu
MRYwFAYDVQQDEw1UaGF3dGUgU1NMIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
MIIBCgKCAQEAmeSFW3ZJfS8F2MWsyMip09yY5tc0pi8M8iIm2KPJFEyPBaRF6BQM
WJAFGrfFwQalgK+7HUlrUjSIw1nn72vEJ0GMK2Yd0OCjl5gZNEtB1ZjVxwWtouTX
7QytT8G1sCH9PlBTssSQ0NQwZ2ya8Q50xMLciuiX/8mSrgGKVgqYMrAAI+yQGmDD
7bs6yw9jnw1EyVLhJZa/7VCViX9WFLG3YR0cB4w6LPf/gN45RdWvGtF42MdxaqMZ
pzJQIenyDqHGEwNESNFmqFJX1xG0k4vlmZ9d53hR5U32t1m0drUJN00GOBN6HAiY
XMRISstSoKn4sZ2Oe3mwIC88lqgRYke7EQIDAQABo4H7MIH4MDIGCCsGAQUFBwEB
BCYwJDAiBggrBgEFBQcwAYYWaHR0cDovL29jc3AudGhhd3RlLmNvbTASBgNVHRMB
Af8ECDAGAQH/AgEAMDQGA1UdHwQtMCswKaAnoCWGI2h0dHA6Ly9jcmwudGhhd3Rl
LmNvbS9UaGF3dGVQQ0EuY3JsMA4GA1UdDwEB/wQEAwIBBjAoBgNVHREEITAfpB0w
GzEZMBcGA1UEAxMQVmVyaVNpZ25NUEtJLTItOTAdBgNVHQ4EFgQUp6KDuzRFQD38
1TBPErk+oQGf9tswHwYDVR0jBBgwFoAUe1tFz6/Oy3r9MZIaarbzRutXSFAwDQYJ
KoZIhvcNAQEFBQADggEBAIAigOBsyJUW11cmh/NyNNvGclYnPtOW9i4lkaU+M5en
S+Uv+yV9Lwdh+m+DdExMU3IgpHrPUVFWgYiwbR82LMgrsYiZwf5Eq0hRfNjyRGQq
2HGn+xov+RmNNLIjv8RMVR2OROiqXZrdn/0Dx7okQ40tR0Tb9tiYyLL52u/tKVxp
EvrRI5YPv5wN8nlFUzeaVi/oVxBw9u6JDEmJmsEj9cIqzEHPIqtlbreUgm0vQF9Y
3uuVK6ZyaFIZkSqudZ1OkubK3lTqGKslPOZkpnkfJn1h7X3S5XFV2JMXfBQ4MDzf
huNMrUnjl1nOG5srztxl1Asoa06ERlFE9zMILViXIa4=
-----END CERTIFICATE-----
 2 s:/C=US/O=thawte, Inc./OU=Certification Services Division/OU=(c) 2006 thawte, Inc. - For authorized use only/CN=thawte Primary Root CA
   i:/C=ZA/ST=Western Cape/L=Cape Town/O=Thawte Consulting cc/OU=Certification Services Division/CN=Thawte Premium Server CA/[email protected]
-----BEGIN CERTIFICATE-----
MIIERTCCA66gAwIBAgIQM2VQCHmtc+IwueAdDX+skTANBgkqhkiG9w0BAQUFADCB
zjELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJ
Q2FwZSBUb3duMR0wGwYDVQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UE
CxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UEAxMYVGhh
d3RlIFByZW1pdW0gU2VydmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNl
cnZlckB0aGF3dGUuY29tMB4XDTA2MTExNzAwMDAwMFoXDTIwMTIzMDIzNTk1OVow
gakxCzAJBgNVBAYTAlVTMRUwEwYDVQQKEwx0aGF3dGUsIEluYy4xKDAmBgNVBAsT
H0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2aXNpb24xODA2BgNVBAsTLyhjKSAy
MDA2IHRoYXd0ZSwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MR8wHQYD
VQQDExZ0aGF3dGUgUHJpbWFyeSBSb290IENBMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEArKDw+4BZ1JzHpM+doVlzCRBFDA0sbmjxbFtIaElZN/wLMxnC
d3/MEC2VNBzm600JpxzSuMmXNgK3idQkXwbAzESUlI0CYm/rWt0RjSiaXISQEHoN
vXRmL2o4oOLVVETrHQefB7pv7un9Tgsp9T6EoAHxnKv4HH6JpOih2HFlDaNRe+68
0iJgDblbnd+6/FFbC6+Ysuku6QToYofeK8jXTsFMZB7dz4dYukpPymgHHRydSsbV
L5HMfHFyHMXAZ+sy/cmSXJTahcCbv1N9Kwn0jJ2RH5dqUsveCTakd9h7h1BE1T5u
KWn7OUkmHgmlgHtALevoJ4XJ/mH9fuZ8lx3VnQIDAQABo4HCMIG/MA8GA1UdEwEB
/wQFMAMBAf8wOwYDVR0gBDQwMjAwBgRVHSAAMCgwJgYIKwYBBQUHAgEWGmh0dHBz
Oi8vd3d3LnRoYXd0ZS5jb20vY3BzMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQU
e1tFz6/Oy3r9MZIaarbzRutXSFAwQAYDVR0fBDkwNzA1oDOgMYYvaHR0cDovL2Ny
bC50aGF3dGUuY29tL1RoYXd0ZVByZW1pdW1TZXJ2ZXJDQS5jcmwwDQYJKoZIhvcN
AQEFBQADgYEAhKhMyT4qvJrizI8LsiV3xGGJiWNa1KMVQNT7Xj+0Q+pjFytrmXSe
Cajd1FYVLnp5MV9jllMbNNkV6k9tcMq+9oKp7dqFd8x2HGqBCiHYQZl/Xi6Cweiq
95OBBaqStB+3msAHF/XLxrRMDtdW3HEgdDjWdMbWj2uvi42gbCkLYeA=
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=California/L=San Jose/O=The Linux Kernel Organization/CN=*.kernel.org
issuer=/C=US/O=Thawte, Inc./CN=Thawte SSL CA
---
No client certificate CA names sent
---
SSL handshake has read 3566 bytes and written 636 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-SHA
    Session-ID: EAB5C93513AA8829036C7BB9E3F74D99076C8A339EB220832F76DF73D52D0B22
    Session-ID-ctx: 
    Master-Key: 0A599E2D1CCAA8249E50871FDF03A2137BA034BCB20FA691D1413822BE08E15303CB0F59CDEC0376D670E08632EF0D46
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - ac 28 d2 81 da 5e b9 7f-43 52 ab e8 2c b0 ac 1d   .(...^..CR..,...
    0010 - c0 27 92 cb 5e 0e 1e 36-e4 77 34 bf 34 f8 cb 2f   .'..^..6.w4.4../
    0020 - 04 af 1a 9b ea b3 9e 6f-32 44 4b ce d1 b4 2b 42   .......o2DK...+B
    0030 - 55 a8 e3 ec 9a 6a 76 5d-c0 84 e0 aa 20 29 ae ac   U....jv].... )..
    0040 - 7d 45 2b 3b 56 3c 2e 4b-d3 69 60 c8 fb 67 36 07   }E+;V...;.]p.......
    00a0 - de 09 7d bf 3f b9 2c 9c-af 5d b1 af b2 9b bc 7a   ..}.?.,..].....z
    00b0 - 5e b3 92 26 02 3a 0e 47-c9 4b 10 6c 5b f4 2f c3   ^..&.:.G.K.l[./.

    Start Time: 1380663897
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---

I would add certificate 2, the (last one). I would not need certificate 1 as it's a chain cert and is authorized to be a chain cert by certificate 2. However I could add certificate 1 as well and it wouldn't hurt.

Comments

[Paul]

@StephaneChazelas, I edited the question to include the output.

[Stéphane Chazelas]

That's the part in the "blah blah" that could have been interesting. Does the self-signed certificate in there match the one you added to /etc/ca-certificates?

[Paul]

@StephaneChazelas, no it does not. There are actually two certificates I added. It doesn't match either of them.

[Paul]

I have already added the CA, and the server certificate. The openssl... command only shows me one cert and it doesn't match either the server cert for my company, the CA cert, or any of the ones listed in your answer.

[phemmer]

Yes, the first cert is the server cert. It's not supposed to match anything. However that means you're going to have to ask your network team for the CA cert the proxy uses to generate the server certs.

[Paul]

thanks for trying to help, but as I've tried to explain, I've already obtained the CA cert and added it before I posted the question. It didn't make a difference.

[phemmer]

@druciferre you should A) put that info in your question (I just re-read it, and that's not mentioned anywhere), B) provide openssl x509 -noout -text -in /path/to/CA_cert