Why do I get a handshake failure (Java SSL)

Question

I'm connecting to a web service over HTTPS. I've done all that I think is required to make it work, but in the end I get a handshake failure.

I found out that as a new user I can't post more than 2 links due to "spam protection" - thanx a lot stackoverflow...anyway here's a link to a pastebin post with all the links spelled out...so when I write "link#1" here it's a reference to these links: http://pastebin.com/y4zGNRC7

• I verified the same behavior using HttpClient (GET on the service URL) and actually calling the web service via a CXF proxy
• I'm setting both the keystore and truststore - I tried both the "in code" way ( link#1 ) and setting the system properties - i.e. System.setProperty("javax.net.ssl.keyStore", "mykeystore.jks");
• SSL debug is on ( javax.net.debug=all )
• SSL debug blurts out the contents of both keystore and truststore (i.e. looks like java "knows about them") - link#2
• seems like there's some client-server communication going on, but then it crashes for some reason link#3
• I successfully connected to the server using the client and CA certificates both in a browser (Chrome) and using openssl s_client
• wireshark shows less client-server talk from java ( link#4 ) then for example from Chrome ( link#5 )

Another strange thing is, that I seem to be getting the same behavior when I set the keystore and when I don't (the only difference is that when I do the keystore contents get printed in the console, but that's it).

I tried googling the problem and I saw numerous similar posts here on stackoverflow, but nothing helped. I tried changing the protocol version ("TLSv1", "SSLv3", even the weird v2 Hello). Any help would be appreciated - maybe there's some fundamental thing I might have overlooked...I'm getting desperate here... Thanx

PS I'm running java 1.6 update 30 on Fedora Core 15 (64bit)

Solution 1

The problem was that even though the keystore and truststore was set, java decided not to send the client certificate to the server. The reason for this was the fact, that the server requested a certificate signed by the RootCA authority, but the client certificate is signed by a SubCA authority (which is issued by the RootCA).

Originally the keystore only contained the client cert and the truststore the SubCA cert. I then tried to add the SubCA cert to the keystore too, but java just ignored it.

So this solves the hanshake failure mystery, but not my problem.

I created a separate question for that...sigh :-( why doesn't java send the client certificate during SSL handshake?

Solution 2

I think the trust store not containing the CA is the most likely issue. You can use the Java keytool to import the certificate for the site into the cacerts file doing something like:

keytool -keystore pathtocacerts -import -trustcacerts -v -alias aliasName -file root.crt

The default cacerts keystore password is changeit. The cacerts file is usually under jre/lib/security directory.

Solution 3

You don't provide enough information, but I'm guessing your client truststore is not properly configured. The truststore contains the trusted certificates that are used to sign other certs, and must include the root certificate(s) for the server and client cert chains. The client keystore contains the client SSL certificate and private key.

Comments

[Jakub Hlavatý]

Thanx for your reply Greg. Please tell me what more information I should provide. I tried to be as thorough as possible, but I know I might have missed something. As I wrote I set both the truststore and the keystore. I did put Xs in the logs instead of actual certificate names and stuff as I didn't want to post those online. The keystore contains the client certificate and the private key. The truststore contains both the certificate of the CA which issued the certificate and the root CA certificate. The stacktrace doesn't seem to be too helpful here (I posted it in a comment above).

[Jakub Hlavatý]

I'm pretty sure the truststore does contain the CA. When that didn't work i also imported the root CA certificate into the truststore...but that didn't seem to help.

[Jakub Hlavatý]

Note that with SSL debug on, the contents of both the truststore and keystore get printed in the console. Also the client certificate and the CA cert (not the root one) suffice for this to work in a browser.

[Jakub Hlavatý]

I tried checking the contents of the stores with the keytool and everything seems to be OK. I even tried setting both stores via commandline options on startup (i.e. -Djavax.net.ssl.keyStore=) to no avail...

[Jakub Hlavatý]

It seems the client keystore gets loaded, but the client certificate is not being presented to the server and that's why the handshake fails. The log shows an empty client certificate chain. I have no idea why though :-(