Accept server's self-signed ssl certificate in Java client

Question

It looks like a standard question, but I couldn't find clear directions anywhere.

I have java code trying to connect to a server with probably self-signed (or expired) certificate. The code reports the following error :

[HttpMethodDirector] I/O exception (javax.net.ssl.SSLHandshakeException) caught 
when processing request: sun.security.validator.ValidatorException: PKIX path 
building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target

As I understand it, I have to use keytool and tell java that it's OK to allow this connection.

All instructions to fix this problem assume I'm fully proficient with keytool, such as

generate private key for server and import it into keystore

Is there anybody who could post detailed instructions?

I'm running unix, so bash script would be best.

Not sure if it's important, but code executed in jboss.

Solution 1

You have basically two options here: add the self-signed certificate to your JVM truststore or configure your client to

Option 1

Export the certificate from your browser and import it in your JVM truststore (to establish a chain of trust):

<JAVA_HOME>\bin\keytool -import -v -trustcacerts
-alias server-alias -file server.cer
-keystore cacerts.jks -keypass changeit
-storepass changeit 

Option 2

Disable Certificate Validation:

// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[] { 
    new X509TrustManager() {     
        public java.security.cert.X509Certificate[] getAcceptedIssuers() { 
            return new X509Certificate[0];
        } 
        public void checkClientTrusted( 
            java.security.cert.X509Certificate[] certs, String authType) {
            } 
        public void checkServerTrusted( 
            java.security.cert.X509Certificate[] certs, String authType) {
        }
    } 
}; 

// Install the all-trusting trust manager
try {
    SSLContext sc = SSLContext.getInstance("SSL"); 
    sc.init(null, trustAllCerts, new java.security.SecureRandom()); 
    HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
} catch (GeneralSecurityException e) {
} 
// Now you can access an https URL without having the certificate in the truststore
try { 
    URL url = new URL("https://hostname/index.html"); 
} catch (MalformedURLException e) {
} 

Note that I do not recommend the Option #2 at all. Disabling the trust manager defeats some parts of SSL and makes you vulnerable to man in the middle attacks. Prefer Option #1 or, even better, have the server use a "real" certificate signed by a well known CA.

Solution 2

There's a better alternative to trusting all certificates: Create a TrustStore that specifically trusts a given certificate and use this to create a SSLContext from which to get the SSLSocketFactory to set on the HttpsURLConnection. Here's the complete code:

File crtFile = new File("server.crt");
Certificate certificate = CertificateFactory.getInstance("X.509").generateCertificate(new FileInputStream(crtFile));
// Or if the crt-file is packaged into a jar file:
// CertificateFactory.getInstance("X.509").generateCertificate(this.class.getClassLoader().getResourceAsStream("server.crt"));


KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
keyStore.load(null, null);
keyStore.setCertificateEntry("server", certificate);

TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(keyStore);

SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, trustManagerFactory.getTrustManagers(), null);

HttpsURLConnection connection = (HttpsURLConnection) new URL(url).openConnection();
connection.setSSLSocketFactory(sslContext.getSocketFactory());

You can alternatively load the KeyStore directly from a file or retrieve the X.509 Certificate from any trusted source.

Note that with this code, the certificates in cacerts will not be used. This particular HttpsURLConnection will only trust this specific certificate.

Solution 3

Apache HttpClient 4.5 supports accepting self-signed certificates:

SSLContext sslContext = SSLContexts.custom()
    .loadTrustMaterial(new TrustSelfSignedStrategy())
    .build();
SSLConnectionSocketFactory socketFactory =
    new SSLConnectionSocketFactory(sslContext);
Registry<ConnectionSocketFactory> reg =
    RegistryBuilder.<ConnectionSocketFactory>create()
    .register("https", socketFactory)
    .build();
HttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(reg);        
CloseableHttpClient httpClient = HttpClients.custom()
    .setConnectionManager(cm)
    .build();
HttpGet httpGet = new HttpGet(url);
CloseableHttpResponse sslResponse = httpClient.execute(httpGet);

This builds an SSL socket factory which will use the TrustSelfSignedStrategy, registers it with a custom connection manager then does an HTTP GET using that connection manager.

I agree with those who chant "don't do this in production", however there are use-cases for accepting self-signed certificates outside production; we use them in automated integration tests, so that we're using SSL (like in production) even when not running on the production hardware.

Solution 4

I chased down this problem to a certificate provider that is not part of the default JVM trusted hosts as of JDK 8u74. The provider is www.identrust.com, but that was not the domain I was trying to connect to. That domain had gotten its certificate from this provider. See Will the cross root cover trust by the default list in the JDK/JRE? -- read down a couple entries. Also see Which browsers and operating systems support Let’s Encrypt.

So, in order to connect to the domain I was interested in, which had a certificate issued from identrust.com I did the following steps. Basically, I had to get the identrust.com (DST Root CA X3) certificate to be trusted by the JVM. I was able to do that using Apache HttpComponents 4.5 like so:

1: Obtain the certificate from indettrust at Certificate Chain Download Instructions. Click on the DST Root CA X3 link.

2: Save the string to a file named "DST Root CA X3.pem". Be sure to add the lines "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" in the file at the beginning and the end.

3: Create a java keystore file, cacerts.jks with the following command:

keytool -import -v -trustcacerts -alias IdenTrust -keypass yourpassword -file dst_root_ca_x3.pem -keystore cacerts.jks -storepass yourpassword

4: Copy the resulting cacerts.jks keystore into the resources directory of your java/(maven) application.

5: Use the following code to load this file and attach it to the Apache 4.5 HttpClient. This will solve the problem for all domains that have certificates issued from indetrust.com util oracle includes the certificate into the JRE default keystore.

SSLContext sslcontext = SSLContexts.custom()
        .loadTrustMaterial(new File(CalRestClient.class.getResource("/cacerts.jks").getFile()), "yourpasword".toCharArray(),
                new TrustSelfSignedStrategy())
        .build();
// Allow TLSv1 protocol only
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
        sslcontext,
        new String[] { "TLSv1" },
        null,
        SSLConnectionSocketFactory.getDefaultHostnameVerifier());
CloseableHttpClient httpclient = HttpClients.custom()
        .setSSLSocketFactory(sslsf)
        .build();

When the project builds then the cacerts.jks will be copied into the classpath and loaded from there. I didn't, at this point in time, test against other ssl sites, but if the above code "chains" in this certificate then they will work too, but again, I don't know.

Reference: Custom SSL context and How do I accept a self-signed certificate with a Java HttpsURLConnection?

Solution 5

Rather than setting the default socket factory (which IMO is a bad thing) - yhis will just affect the current connection rather than every SSL connection you try to open:

URLConnection connection = url.openConnection();
    // JMD - this is a better way to do it that doesn't override the default SSL factory.
    if (connection instanceof HttpsURLConnection)
    {
        HttpsURLConnection conHttps = (HttpsURLConnection) connection;
        // Set up a Trust all manager
        TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager()
        {

            public java.security.cert.X509Certificate[] getAcceptedIssuers()
            {
                return null;
            }

            public void checkClientTrusted(
                java.security.cert.X509Certificate[] certs, String authType)
            {
            }

            public void checkServerTrusted(
                java.security.cert.X509Certificate[] certs, String authType)
            {
            }
        } };

        // Get a new SSL context
        SSLContext sc = SSLContext.getInstance("TLSv1.2");
        sc.init(null, trustAllCerts, new java.security.SecureRandom());
        // Set our connection to use this SSL context, with the "Trust all" manager in place.
        conHttps.setSSLSocketFactory(sc.getSocketFactory());
        // Also force it to trust all hosts
        HostnameVerifier allHostsValid = new HostnameVerifier() {
            public boolean verify(String hostname, SSLSession session) {
                return true;
            }
        };
        // and set the hostname verifier.
        conHttps.setHostnameVerifier(allHostsValid);
    }
InputStream stream = connection.getInputStream();

View more solutions

Comments

[Matthew Flaschen]

See How do I accept a self-signed certificate with a Java HttpsURLConnection?. Obviously, it would be better if you can get the site to use a valid cert.

[Nikita Rybak]

Thanks for the link, I didn't see it while searching. But both solutions there involve special code to send a request and I'm using existing code (amazon ws client for java). Respectively, it's their site I'm connecting and I can't fix its certificate problems.

[jww]

@MatthewFlaschen - "Obviously, it would be better if you can get the site to use a valid cert..." - A self signed certificate is a valid certificate if the client trusts it. Many think conferring trust to the CA/Browser cartel is a security defect.

[jww]

Related, see The most dangerous code in the world: validating SSL certificates in non-browser software. (The link is provided since you seem to be getting those spammy answers that disable validation).

[Nikita Rybak]

I've tried 2 (for testing), but it doesn't seem to work. Maybe, because amazon ws client employs apache HttpClient and HttpMethod classes to make a call. I executed the code right before that call. Also, do you mean in the last sentence that our server having bad certificate can prevent connection? I thought it's only about checking target server's certificate. Thanks, and I'll see what I can do with keytool.

[user207421]

Not just the MIM attack. It renders you vulnerable to connecting to the wrong site. It is completely insecure. See RFC 2246. I am opposed to posting this TrustManager at all times. It's not even correct w.r.t. its own specification.

[Devanshu Mevada]

@EJP I'm really not recommending the second option (I've updated my answer to make it clear). However, not posting it won't solve anything (this is public information) and doesn't IMHO deserve a downvote.

[Nikita Rybak]

It turns out, the problem was in our ssl certificate (which can't be valid because VM is used for testing). I didn't expect it: browsers don't require you to have certificate, so why do I need one in jboss? Anyway, playing around with keytool helped. I guess, I'll have to find time and really learn all this stuff :/

[user207421]

Pascal, I disagree. If you're not recommending it don't post it. I tremble to think how many production systems this has been embedded in just because someone used it to 'get it working' and then moved on. The thing doesn't even confirm to its own specification let alone any conceivable security scenario, and it doesn't just 'defeat some parts of SSL', it defeats the point of using SSL at all.

[Devanshu Mevada]

@EJP It's by teaching people that you educate them, not by hiding things. So keeping things secret or in obscurity is not a solution at all. This code is public, the Java API is public, it's better to talk about it than to ignore it. But I can live with you not agreeing.

[user963601]

You should use the TrustManager to validate the self-signed cert. You can get the cert's fingerprint using a java.security.KeyStore.

[djangofan]

Can you integrate the ALLOW_ALL_HOSTNAME_VERIFIER into your answer for me?

[Donal Fellows]

The other option – the one you don't mention – is to get the server's certificate fixed either by fixing it yourself or by calling up the relevant support people. Single host certificates are really very cheap; futzing around with self-signed stuff is penny-wise pound-foolish (i.e., for those not familiar with that English idiom, a totally stupid set of priorities that costs lots to save almost nothing).

[Nemi]

I am not sure the -trustcacerts switch is actually needed in this case. The documentation on that switch is poorly written, but I think just importing a cert into the cacerts truststore is enough for it to be used as a CA cert.

[Rich]

Would you be able to update your answer with a pointer on how to obtain a suitable "server.cer" file, given only a specific HTTPS site I need to connect to which has a self-signed certificate? Can I derive a "cer" file just by connecting to the server, or do I need to contact the site admin?

[Rich]

In response to my previous comment, and in case anyone else has the same problem: you can get a suitable "server.cer" file by doing "openssl s_client -connect myserver:myport" and copy from "---BEGIN CERTIFICATE---" to "----END CERTIFICATE---" into a ".cer" file

[jocull]

Thanks so much for the tip about exporting from the browser. That's probably the easiest solution I have seen.

[user207421]

@Nemi You seem to be asserting that the -trustcacerts switch doesn't actually do anything. You are of course mistaken about that.

[user2360915]

Any way to restore the cert validation once the code has been executed for option #2?

[Vova Rozhkov]

btw, HttpsURLConnection and other stuff should be imported from javax.net.ssl/java.security, not from the com.sun... Just in case anyone will import from sun packages by mistake.

[Gus]

Just because some server decided to use https, doesn't mean that the person with the client gives a crap about security for their own purposes.

[Tom Close]

Where is cacerts.jks meant to be stored, in your home directory?

[cr1pto]

I'm surprised this answer got downvoted so much. I would love to understand a little more why. It seems like EJP is suggesting that you shouldn't be doing option 2 because it allows for a major security flaw. Could someone explain why (other than pre-deployment settings) why this answer is low quality?

[Igor Ronner]

the option 2 result in exception javax.net.ssl.SSLHandshakeException: Connection closed by peer

[user207421]

@Gus If the server decided to use HTTPS it is because they want some security, and they are entitled to expect the client not to undermine it.

[user207421]

The question is about a self-signed certificate. This answer isn't.

[Gus]

@EJP "they want security" and so they use a self signed cert, or let their cert expire? and it breaks their security that I should trust their self signed cert that they provided? I don't follow what you mean...

[K.Nicholas]

Read the question (and answer) a little closer.

[Piohen]

@EJP: Please note that I didn't downvote the answer and that I didn't make any arguments ad personam. I understand that seeing the answer rejected by a few people might not be pleasant (especially taking the reputation into account), but I really doubt people downvote it because of my comment, mainly because it was rejected before I wrote it. Everyone makes mistakes sometimes, you and I are no exceptions.

[user207421]

@Piohen I'm still waiting for you to tell me what mistake I have made here. All you've done is posted a lot of irrelevant rubbish that has nothing to do with what I wrote, either in my answer or in my reply to your comment. Specifically, you've claimed that I have said 'that self-signed certs are always bad'. Please show me where

[Piohen]

No problem @EPJ: "If 'they' are using a self-signed certificate it is up to them to take the steps required to make their server usable." ;-)

[jww]

This answer does not seem to be downvote-worthy to the level its has sunk (at least to me). Perhaps you can provide the code for the client to use the server's self-signed certificate. I'm not sure I understand the statement "Don't even think about the insecure TrustManager posted here". I thought a custom TrustManager overriding checkServerTrusted was one of the ways to do it. Maybe you could explain it.

[user207421]

@Piohen If you can't tell the difference between 'it is up to them ...' and 'always bad' you are beyond any help I could possibly give you. All I have done here is state a basic security requirement. Sorry you don't get it.

[user207421]

@jww What part of 'it is up to them to take the steps required to make their server usable' etc. don't you understand? And how on earth does this relate to me providing code?

[jww]

Well, I guess all of it. What does "it is up to them to take the steps required to make their server usable" mean? What does a server operator have to do to make it usable? What are the steps you have in mind? Can you provide a list of them? But stepping back to 1,000 feet, how does that even relate to the problem the OP asked? He wants to know how to make the client accept the self-signed certificate in Java. That's a simple matter of conferring trust in the code since the OP finds the certificate acceptable.

[user207421]

@jww It means that if they want clients to accept their self-signed certificate it is up to them to provide it in a form which can be imported into client truststores, via a secure offline procedure. Nothing else is secure. All of which I have already stated in the answer.

[jww]

@EJP - Correct me if I am wrong, but accepting a self-signed certificate is a client side policy decision. It has nothing to do with server side operations. The client must make the decision. The parities must work together to solve the key distribution problem, but that has nothing to do with making the server usable.

[user207421]

@jww It is a client-side policy decision that must be based on the actual certificate, which means the certifficate must be delivered to the client offline, for the client to identiify securely when it appears online. And that does not, cannot possibly, mean acquiring the certificate that is going to be used to authenticate the channel via which the presently unauthenticated channel is acquiring the certificate. This is nothing more than a contradiction in terms. Interpreting 'client-side decision' as meaning 'trust all certificates' is just playing at security.

[jww]

Your checkServerTrusted does not implement the necessary logic to actually trust the certificate and ensure non-trusted certificates are rejected. This might make some good reading: The most dangerous code in the world: validating SSL certificates in non-browser software.

[user207421]

Your getAcceptedIssuers() method does not conform to the specifcation, and this 'solution' remains radically insecure.

[jww]

@EJP - I certainly did not say, "trust all certificates". Perhaps I am missing something (or you are reading too much into things)... The OP has the certificate, and its acceptable to him. He wants to know how to trust it. I'm probably splitting hairs, but the cert does not need to be delivered offline. An out-of-band verification should be used, but that's not the same as "must be delivered to the client offline". He might even be the shop producing the server and the client, so he has the requisite a priori knowledge.

[user207421]

@jww You will have to explain the diffference between 'out-of-band' and 'offline'. You will also have to explain the difference between 'a custom TrustManager and 'trust all certificates'. If you're only going to trust one, or several, self-signed certicates, you don't need a custom TrustManager: you only need a custom truststore: and this solution is preferable, as it relies on configuration rather than custom code.

[Siddhartha]

Fantastic answer, thanks so much. Couple things to be careful of, when I first did the command in Option 1, it created a new cacerts.jks and the JVM didn't use it. I had to use just cacerts and then it appended the self-signed certificate to the cacerts truststore file. Also, if you open the file in textedit, you'd be able to see (with some binary gibberish) the signing authorities it trusts, e.g. Verisign. You should see your own cert there once you add it.

[Siddhartha]

@Rich, 6 years late, but you can also get hold of the server cert in firefox by clicking the lock icon -> more information -> View Certificate -> Details tab -> Export... It's well hidden.

[Lo-Tan]

Not entirely sure of why you are being down voted, unless the code doesn't work. Perhaps it's the assumption that Android is somehow involved when the original question didn't tag Android.

[Andika Ristian Nugraha]

i have try this but i get another error message org.apache.xmlrpc.XmlRpcException: Failed to read server's response: java.security.cert.CertificateException: Certificates do not conform to algorithm constraints

[Daniel]

The [android documentation] (developer.android.com/training/articles/security-ssl#SelfSi‌​gned) basically gives your explanation. It just gives a bit more of an explanation, code and warnings.

[Johannes Brodwall]

An option that is much safer than #2 and avoids having to deal with global config outside the program like #1 is to trust the specific certificate: stackoverflow.com/a/57046889/27658

[actunderdc]

In my opinion this should be the accepted answer! The advantage of this method is that you do everything programatically and still maintain security and you do not alter the JRE's trust store. Option #2 from the accepted answer is completely wrong from security purposes. Thank you for providing this solution!

[Martynas Jusevičius]

I don't get option #1. What does the browser got to do with the OP's question? He's mentioning some kind of Java client (not browser), and it is the server certificate that is self-signed, as I understand.

[Mikolasan]

If someone is going to pack the certificate into jar file, then replace InputStream in the method generateCertificate to this: this.class.getClassLoader().getResourceAsStream("server.crt"‌​)

[MMHossain]

Thanks a lot. It worked for me, was struggling to find an HTTPS GET client for my self-signing Spring Boot HTTPS server.