No subject alternative DNS name matching ex.ample.com found

Question

For my application running at ex.ample.com I have the following certificate from StartSSL/StartCom:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 163069 (0x27cfd)
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C=IL, O=StartCom Ltd., OU=Secure Digital Certificate Signing, CN=StartCom Class 1 Primary Intermediate Server CA
        Validity
            Not Before: Nov 27 21:26:01 2010 GMT
            Not After : Nov 29 15:32:05 2011 GMT
        Subject: description=303703-Sv1xMdnmzg6garMt, C=NL, O=Persona Not Validated, OU=StartCom Free Certificate Member, CN=ex.ample.com/[email protected]
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (2048 bit)
                Modulus (2048 bit):
                    ....
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            X509v3 Key Usage: 
                Digital Signature, Key Encipherment, Key Agreement
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Subject Key Identifier: 
                31:68:B2:7B:A2:7C:79:54:B7:3E:66:FD:12:04:18:FC:FB:9B:34:64
            X509v3 Authority Key Identifier: 
                keyid:EB:42:34:D0:98:B0:AB:9F:F4:1B:6B:08:F7:CC:64:2E:EF:0E:2C:45

            X509v3 Subject Alternative Name: 
                DNS:ex.ample.com, DNS:ample.com
            X509v3 Certificate Policies: 
                Policy: 1.3.6.1.4.1.23223.1.2.2
                  CPS: http://www.startssl.com/policy.pdf
                  CPS: http://www.startssl.com/intermediate.pdf
                  User Notice:
                    Organization: StartCom Ltd.
                    Number: 1
                    Explicit Text: Limited Liability, see section *Legal Limitations* of the StartCom Certification Authority Policy available at http://www.startssl.com/policy.pdf

            X509v3 CRL Distribution Points: 
                URI:http://www.startssl.com/crt1-crl.crl
                URI:http://crl.startssl.com/crt1-crl.crl

            Authority Information Access: 
                OCSP - URI:http://ocsp.startssl.com/sub/class1/server/ca
                CA Issuers - URI:http://www.startssl.com/certs/sub.class1.server.ca.crt

            X509v3 Issuer Alternative Name: 
                URI:http://www.startssl.com/
    Signature Algorithm: sha1WithRSAEncryption
        .....

I have this certificate installed correctly. It works when I access the app with Firefox. When I use Java's HttpURLConnection to retrieve a page from it though, I get the following error:

Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching ex.ample.com found.
    at sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:208)
    at sun.security.util.HostnameChecker.match(HostnameChecker.java:94)
    at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:285)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:271)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1012)
    ... 14 more

I don't understand why this happens. ex.ample.com is the Common Name (CN) in the cert and is also listed in the SANs. I've imported StarSLL's certificates in my truststore, so that's not the problem here.

Solution

We faced this issue recently, it was a nightmare because we were able to reproduce it only in Production servers, were access to debug is near to zero. The rest of environments were just working fine. Our stack was JDK 1.8.x+, JBoss EAP 7+, Java Spring Boot app and Okta as identity provider (the SSL handshake was failing when recovering the well-known configuration from Okta, where okta is available in AWS Cloud - virtual servers).

Finally, we discover that (no one knows why) the JBoss EAP application server that we were using it was having an additional JVM System Property:

jsse.enableSNIExtension = false

This was preventing to establish TLS connection and we were able to reproduce the issue by adding that same system property/value in other environments. So the solution was simple to remove that undesired property and value.

As per Java Security Doc, this property is set by default to true for Java 7+ (refer to https://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization)

• jsse.enableSNIExtension system property. Server Name Indication (SNI) is a TLS extension, defined in RFC 4366. It enables TLS connections to virtual servers, in which multiple servers for different network names are hosted at a single underlying network address. Some very old SSL/TLS vendors may not be able handle SSL/TLS extensions. In this case, set this property to false to disable the SNI extension.