How can the SSL client validate the server's certificate?

Question

I am building an application and I am planning on using OpenSSL for securing data transfers.

I am planning on only having the client validate the server's certificate. I am confused on how I should secure the server's certificate. I would like to encrypt the server's certificate containing the private key, but I do not want to use any hard coded keys for this encryption.

What are some of the common practices followed by applications employing SSL?

Solution

Just to make sure we have our terminology straight, an "SSL certificate" in common parlance is really composed of two components:

• A public certificate
• A private key

The public certificate component is signed by your chosen CA (certificate authority), after which it can be freely distributed. It does not need to be secured or encrypted, and indeed it will be sent to clients that connect to your server as part of the SSL negotiation.

The private key component should be protected. In the majority of cases, this is simply stored as an encrypted file on the server. Upscale solutions use dedicated "tamperproof" crypto hardware (HSMs -- hardware security modules) to store the private key. These range from smart-card based solutions to multi-key, network enabled appliances with m/n controls etc etc. There are risks (not to mention costs) associated with HSMs that I will not go into here.

Many applications simply retain the private key on disk. There are a couple of options to secure the key file:

• Rely on system and file permission security (ie don't encrypt private key). For example, most ssh daemons do this.
• Use whatever mechanism your server provides to encrypt the file - password-protected encryption is a standard feature across most web servers. (If you're rolling your own using the OpenSSL API, pick one of the obvious native key formats).

As always, there is a security trade-off. In particular, if you are using password-protected encryption on the private key file and you experience an unexpected application restart (eg power outage), then somebody will need to be available to provide the password to the app as it restarts. Storing the password in a file that is read by system initialization scripts (as encouraged by at least two web server vendors) adds little in terms of real security. It's hard to recommend leaving the private key file unencrypted but if you are the sole admin/techy in a small shop, you should definitely consider what might happen if the server reboots when you are not available, and what the costs might be to your business.

Comments

[msvcyc]

Thanks for the response. Could you please elaborate on password protected encryption? How do applications secure these passwords?

[Martin Carpenter]

"Badly", in general. Typically the passwords go in a file, only readable by the application user, so you're still relying on filesystem perms (to protect pw file rather than key file). Some apps will obfuscate the pw in the file for you. Again, that's weak, for hopefully obvious reasons.

[Bruno]

Not sure I'd recommend this terminology. The "SSL certificate" (meaning the X.509 certificate, I presume), is not "composed" of a Public Key Certificate (PKC) and a private key: it is the PKC, and it is associated with the private key of this key-pair. The private key isn't part of the certificate, ever.

[Bruno]

Quick point of terminology: the public key isn't used to decrypt a message encrypted with the private key: it's used to verify (the signature of) a message that has been signed with the private key. Decrypting is done with the private key, following encryption with the private key. (It doesn't make sense to encrypt something with the private key, so that anyone can decrypt it with the public key.)

[Martin Carpenter]

@Bruno: that was the exactly point that I was trying to make. I've added "air quotes" to the first sentence to try to make that clearer and amended the words in the 3rd para slightly.

[AlikElzin-kilaka]

Regarding the CA, it's used just to validate/get the public key. For example, when using a browser, because a user needs to get the public key somewhere. With desktop/mobile/non-browser apps, the public key is in the app/client itself, which means the CA is not part of the process.

[user207421]

The terminology here is not correct. An SSL certificate wraps a public key. It does not consist in any sense of components one of which is a private key. The private key is entirely distinct from the certificate, related only by the public key. I don't like the term 'public certificate component' either. All certificates are public.

[user207421]

You check its signature via whatever algorithm the signature was created with. You don't get to choose.

[Charlie Martin]

And when it's set up, don't use MD-5. In fact, now people should be moving away fro0m SHA-1.

[Martin Carpenter]

Now with air quotes and "in common parlance". If it's still not clear that I'm trying to make a distinction between what folks generally say ("TLS cert") and the technicalities that I and the educated commenters above are aware of (there is a key and a cert) then please suggest an edit.