Can't get Nginx to serve correct certificate chain

nginx ssl-certificate certificate

9,760

Turns out, the Nginx documentation is somewhat misleading and goes at odds with SNI.

Running openssl s_client -connect lanzz.org:443:

CONNECTED(00000003)
depth=0 C = GB, CN = www.lanzz.org, emailAddress = [email protected]
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = GB, CN = www.lanzz.org, emailAddress = [email protected]
verify error:num=27:certificate not trusted
verify return:1
depth=0 C = GB, CN = www.lanzz.org, emailAddress = [email protected]
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/C=GB/CN=www.lanzz.org/[email protected]
   i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA
---

Running openssl s_client -connect lanzz.org:443 -servername lanzz.org:

CONNECTED(00000003)
depth=1 C = IL, O = StartCom Ltd., OU = Secure Digital Certificate Signing, CN = StartCom Class 1 Primary Intermediate Server CA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=GB/CN=www.lanzz.org/[email protected]
   i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA
 1 s:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA
   i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority
---

Apparently, on one hand I had a default virtual host configured with the wrong certificate file (one without the intermediate), and on the other openssl s_client does not automatically provide a SNI extension for the hostname passed to the -connect argument. The Nginx documentation naively provides a command that would only test the SSL setup of the default virtual host, which I would assume is rarely what people usually want.

TL;DR: Supply the `-servername` parameter when testing your SSL chains:

openssl s_client -connect <hostname>:443 -servername <hostname>

9,760

lanzz

Updated on September 18, 2022

Comments

lanzz almost 2 years

I'm trying to set up Nginx (1.4.6-1ubuntu3.1) with a StartSSL certificate. I've been following the documentation to get it working, but Nginx only serves the server certificate, not the intermediate.

My server config:

server {
    server_name lanzz.org www.lanzz.org;
    root /var/www/lanzz.org/public;
    index index.html;
    include listen.conf;
    ssl_certificate /etc/ssl/nginx/lanzz.org.chained.pem;
    ssl_certificate_key /etc/ssl/nginx/lanzz.org.key;
}

My certificate bundle (/etc/ssl/nginx/lanzz.org.chained.pem):

Output from openssl s_client -connect lanzz.org:443:

CONNECTED(00000003)
depth=0 C = GB, CN = www.lanzz.org, emailAddress = [email protected]
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = GB, CN = www.lanzz.org, emailAddress = [email protected]
verify error:num=27:certificate not trusted
verify return:1
depth=0 C = GB, CN = www.lanzz.org, emailAddress = [email protected]
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/C=GB/CN=www.lanzz.org/[email protected]
   i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGMzCCBRugAwIBAgIDE5WyMA0GCSqGSIb3DQEBCwUAMIGMMQswCQYDVQQGEwJJ
TDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0
YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xhc3Mg
MSBQcmltYXJ5IEludGVybWVkaWF0ZSBTZXJ2ZXIgQ0EwHhcNMTQxMDE0MTMzNjI1
WhcNMTUxMDE1MTUzODI3WjBKMQswCQYDVQQGEwJHQjEWMBQGA1UEAxMNd3d3Lmxh
bnp6Lm9yZzEjMCEGCSqGSIb3DQEJARYUcG9zdG1hc3RlckBsYW56ei5vcmcwggEi
MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCs8AMEaHFRZ4S3gGYhDJg05vj3
EadGIi29EJf5iW1YwKduuA62Zv9D2JGh2FCBJDBTswefxFbs2v/HHdP70gr0l669
Vz47RUJgn6xH13xEnVv5btQfPtioLQJNwLnBDR3ycw+9I/CGq+/BmStXBT2fTlDp
7FlDaemkc/mjd4TM6DBL0mfsAfqcSA4GHgQraJSwMyRGn3lon02mOWsDso6nTMEt
QYmCvYoM7wVtiBxKGP9Q6Nz3s5Ouc1U7mxKxuLNIO5ZeT+zocW7HQXk1sGal/Hxi
Y+Us/SsmcDAvqvI9f44Xe4StMfPDBphEDrOvJt9/zuDu8SNMnEA1cqZHGQFVAgMB
AAGjggLdMIIC2TAJBgNVHRMEAjAAMAsGA1UdDwQEAwIDqDATBgNVHSUEDDAKBggr
BgEFBQcDATAdBgNVHQ4EFgQUiw5c8ahxug4Ltshgeob3+CjaFSAwHwYDVR0jBBgw
FoAU60I00Jiwq5/0G2sI98xkLu8OLEUwIwYDVR0RBBwwGoINd3d3Lmxhbnp6Lm9y
Z4IJbGFuenoub3JnMIIBVgYDVR0gBIIBTTCCAUkwCAYGZ4EMAQIBMIIBOwYLKwYB
BAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNv
bS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp
Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNz
dWVkIGFjY29yZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVt
ZW50cyBvZiB0aGUgU3RhcnRDb20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZv
ciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBjb21wbGlhbmNlIG9mIHRoZSByZWx5
aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA1BgNVHR8ELjAsMCqgKKAmhiRodHRwOi8v
Y3JsLnN0YXJ0c3NsLmNvbS9jcnQxLWNybC5jcmwwgY4GCCsGAQUFBwEBBIGBMH8w
OQYIKwYBBQUHMAGGLWh0dHA6Ly9vY3NwLnN0YXJ0c3NsLmNvbS9zdWIvY2xhc3Mx
L3NlcnZlci9jYTBCBggrBgEFBQcwAoY2aHR0cDovL2FpYS5zdGFydHNzbC5jb20v
Y2VydHMvc3ViLmNsYXNzMS5zZXJ2ZXIuY2EuY3J0MCMGA1UdEgQcMBqGGGh0dHA6
Ly93d3cuc3RhcnRzc2wuY29tLzANBgkqhkiG9w0BAQsFAAOCAQEAEom5lVxCbfu9
3K+BuowfCgTyA4keiQcYmTUJYXRBV9OiFUc/V5tXyhmgdyYeJB3oKMaEQ3glClZm
ueXUkALhaIlEzXjoNZgOh/bdbBPwfOq2WMBaWJbXX3x4C77s52zPBbkqhsBq5nge
1YDho1Z7tVYe8iyqBPUIFq0//LfGVAMoR7ZSwVpUgeiWs3oVKQMyR2BzrNSq392L
f8kIdIQ8fTgomtCZ2F2qp8EBIxl6G4UVpeN3Nes9UODpdNeG6lzLoSvYsVX+eWXx
3m29Aem42vlMhtQL1dtahb71nTY26qWAG337fwQoRxAXht3vSs0HhdUEHyGQGsuu
k+fcnm/wiQ==
-----END CERTIFICATE-----
subject=/C=GB/CN=www.lanzz.org/[email protected]
issuer=/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA
---
No client certificate CA names sent
---
SSL handshake has read 2256 bytes and written 451 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : ECDHE-RSA-AES256-SHA
    Session-ID: 958F818087118CA5F3344796B13FD463097B08ACF4F5497A3AD7A40D550FA155
    Session-ID-ctx: 
    Master-Key: 3AB94A204BF8E30BEDC615832E7DD5B2347458A6324A85C8C02BD441AF9F39BA09191BEB85A5F40ED99049013A51F7AB
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 27 5f 93 a2 81 ed c3 e7-ba 41 27 0f 8f 95 be f5   '_.......A'.....
    0010 - 68 a6 7a 8b 1a 84 71 6a-00 cd ec 94 fc 3f 2c 64   h.z...qj.....?,d
    0020 - d8 b1 19 76 ba 8e ff e6-97 39 89 fb 1d 69 47 4a   ...v.....9...iGJ
    0030 - b3 40 96 a5 30 9e c6 b6-79 7f 93 36 02 31 4d 16   [email protected].
    0040 - 2b d9 d6 0b 3c a9 07 10-63 af 89 b2 19 0e ac b3   +...<...c.......
    0050 - 3f c2 41 df f5 23 62 5c-11 d8 f6 be 55 1e e2 74   ?.A..#b\....U..t
    0060 - 6c 4f cd ba 72 e1 a1 f7-b4 fb d1 19 6d 33 d5 63   lO..r.......m3.c
    0070 - d6 94 a0 e6 e5 0c 33 e4-67 e3 49 fe fc b1 d5 25   ......3.g.I....%
    0080 - 9d d7 d3 72 b4 fb 5a 1f-6e 7f 5e 4b 41 21 3a d5   ...r..Z.n.^KA!:.
    0090 - 37 ef 4a b4 35 2d 14 bd-29 77 6d c5 6c c5 3c a4   7.J.5-..)wm.l.<.

    Start Time: 1422525959
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)
---

As you can see, only one certificate is present in the chain, the intermediate is not served by Nginx.

I've verified that Nginx is seeing the correct bundle, because if I swap the server cert with the intermediate cert I do see the expected SSL_CTX_use_PrivateKey_file error mentioned in the docs. When I put them in the correct order I see no errors at startup, no errors at request time, but no intermediate certificate in the SSL session.

The chain is otherwise correct, as I have verified that a browser that already has the intermediate will accept the server certificate, and I've verified that the fingerprints for the intermediate and the server certificates I see in the browser match what I get for the certificates in my bundle on the server.

I'm at a complete loss here, and I don't know what else to try. I think everything is set up by the book, but I still don't get the expected result in s_client. I could not find anyone else even having such a problem, much less any existing solution to it. I've even tried including the root certificate in the bundle, which had no effect whatsoever. Any tips or pointers will be much appreciated.

zakjan over 9 years

It is weird. To request from openssl s_client your server sends indeed only one certificate: www.lanzz.org/[email protected] (1). However, to request from browser it sends three certs: 1, StartCom Class 1 Primary Intermediate Server CA (2) and AddTrust External CA Root (3). 2 is from your cert bundle, it is correct. But where does 3 come from? 2-3 path is not valid. Maybe it can help you find your bug. Browser still trusts it, because 3 is ignored.
lanzz over 9 years

How are you determining what is sent to a browser? The certificate details shown from the padlock in the URL don't show only the certificates sent by the server, but also other certificates already known to the browser; AddTrust External CA is a root certificate that is trusted by the browser (for Firefox) or by the OS (for other browsers) and the StartCom intermediate is very likely cached by your browser from other StartSSL chains that you've encountered elsewhere.
zakjan over 9 years

I saw it in Wireshark.