How to securely handle AES “Key” and “IV” values

Question

If I use AES (System.Security.Cryptography) to simply encrypt and decrypt blob or memo fields in a SQL server, then where do I store the “Key” and “IV” values on the server? (File, Regkey, Dbase,...)

And what with the protection of those AES “Key” and “IV” values?

The background question is more : If “they” hack the server and get the dbase... then probably they can get to the program that do the encryption stuff also (It's on the same server, can't help it)... and if "they" are very good, then they will notice where the “Key” and “IV” values are stored...(.NET 4.5 ILSPY) and everything can be decrypted again.

Please advice? How do you all handle AES “Key” and “IV” value’s?

Ps: This is not about pwd fields... so, it's not about hashing... its pure data cryptography.

Solution 1

The IV has been thoroughly covered by the other answers, so I'll focus just on storing the key.

First...

I can't except it could not be done on a single server at software level.

Anything done in software can be undone in software. You can encrypt, hide, and lock it in as many safes as you want, but your application still needs to be able to get access to the key. If your application has access, then someone with the same level of access as your application is able to get to it as well.

Developers have been dealing with this problem for a very long time and there is no silver bullet.

This all is setup in a single server environment (application plus dbase), so I’m not able to send/retrieve the key to a second server. Also, in this “special” case I’m not able to encrypt the key by a machine-level or user-level RSA key container.

I can think of two possible solutions.

Option 1:

Store the key on disk and, at the OS level, configure file access so that only the account your application is running under can read the file the key is contained in. The file could be a flat file, or an encrypted container that's protected by a password which your application knows (up to you to decide, but an encrypted container is better).

Pros:

• Restarts without human intervention.

Cons:

• You have to do OS security right and there is no room for error.
• An attacker with administrator access can get to the key.

Another similar option to this would be to use DPAPI instead of files for storing the key (as long as you're able to do this given your "special case"). This is an API built in to windows that utilizes the password for whatever windows account you (or your application) is running under to securely store data. Only the windows account that stored the data is able to retrieve it.

One particularly nice feature of DPAPI is that, if an administrator resets a users password (via computer management), access to that users DPAPI data is lost. An attacker would need to compromise the actual account that was used to store the data in the first place without resetting the password.

Option 2:

Require that a pass phrase be entered by a person at application start up and derive an encryption key from that pass phrase. Once you have the key, discard the pass phrase and retain the key in memory only.

Pros:

• The key is never on disk.
• Even if the server is rooted, getting to the key is not a simple task.

Cons:

• Automated reboots are not possible.
• You'll likely have to share the pass phrase with anyone handling support.
• You need to keep in mind that data stored in memory may transparently be written to disk in certain situations.

Or you could do a compromise between these two systems where, a pass phrase is initially used to derive the encryption key which is held in memory, and the key is temporarily written to disk or encrypted container whenever the application is gracefully restarted. When the restart is complete the application loads the key and then deletes it from temporary storage (and if necessary, be sure to overwrite the disk location where the key was stored so that it can't be recovered).

Solution 2

Rules of thumb are:

• Key must be secret at all times (must not be anywhere near the database)
• IV must be different for each record.
• IV must be "indistinguishable from random" and unpredictable, preferably it must come from the same source as your AES keys; other option is to encrypt some value (different for each record) with a secret key.
• IV needs not to be secret

Hence, one scheme you can use is:

1. Create a table with fields ID (unique, int), IV (unique, 16 bytes), Encrypted(variable bytes, NULLable)
2. To write a new record into the database, create new unique IV and create a new record in the database with empty encrypted data (to prevent collisions)
3. Encrypt the data with your secret key and IV from step 2 (CBC or CTR mode - CTR is better) and update the record.

Step two may be performed by taking the IV from previous record and encrypting it with the same secret key - AES's properties will make this an effectively random IV.

This will be as secure as you can get with AES - meaning CCA/CPA secure. The only thing it does not prevent is tampering

Solution 3

The IV does not need to be kept as secret as the key, the only thing it serves to do is to make sure two of the exact same blobs encrypted with the same key produce two outputs that are totally different from each other (so you can't tell the same message was sent twice). Many encryption systems just make the IV the first bytes of the message.

Encryption keys are a harder thing to manage, the best thing you can do is keep the database itself and the application separate so "If “they” hack the server and get the dbase" (Say a SQL injection attack lets them do a dump of the database's tables) they still can't decrypt the fields themselves.

Solution 4

Segregating your web server and db server would be helpful here. You want to lock down access (permissions-wise) to your encryption keys, and keep them in memory as SecureString. Can't do much more than that. Choose strong passwords and follow up-to-date security practices.

Here's a good post as well Where to Store Encryption Keys MVC Application

View more solutions

Comments

[ntoskrnl]

Note that the IV should be different (unique, preferably random) every time the encryption function is called.

[ChiYoung]

Oké, the IV's are unique and are stored with the encrypted memo or blob to save space. But still, the actual key is not protected! How can I protect the most essential part of the algorithm : the Key! Still need to know this? Are you symmetric encrypting the Key? Are you using certificates for encrypting this Key? How do you handle the Key in memory? Still need help please...

[ChiYoung]

I really can't segregating the web and SQL server. They are one. And furthermore, if the webApplication can access the Key, then (in case there is a security hole in the webapp) a hacker probably can also... So it all comes down on how to securely store/handle the encryption key.

[Scott Chamberlain]

If you are working on a project where you are concerned about the key management the only "real" solution is using a Hardware Security Module, it is a piece of hardware that does the encryption/decryption so even if someone stole the database and the source code they still could not decrypt it without that piece of hardware. However if you do proper separation of your database server and your application server it should be fairly difficult to get both the data and source code.

[ChiYoung]

Hi Scott. I can't except it could not be done on a single server at software level. Microsoft is doing the same thing with the SharePoint Secure Store Service database. To setup the SSS DB, MS request a pass-phrase from the user to actually set it all up. The pass-phrase by itself is never stored on the server (or anywhere else). The SSS DB is used to securely encrypt/decrypt user accounts in real-time. The only time you re-enter the pass-phrase is for recovering the SSS DB or adding extra servers to the SharePoint farm. So, there must be a secure way/methodology… I’m sure about that…

[ChiYoung]

Note: About SharePoint SSS Encryption Keys : technet.microsoft.com/en-us/library/ee806866.aspx - Read the part: Work with encryption keys. Quote : Before using the Secure Store Service, you must generate an encryption key. The key is used to encrypt and decrypt the credentials that are stored in the Secure Store Service database. Other quote : The pass phrase that you enter is not stored. Make sure that you write this down and store it in a safe place. You must have it to refresh the key, such as when you add a new application server to the server farm.

[jbtule]

@ChiYoung "So, there must be a secure way/methodology", that is wishful thinking, in SharePoint's case likely there is a key that is randomly generated, and then encrypted separately with a machine key and that passphrase, so you need the passphrase later when adding another machine to get the master key without the original machine key. It's just an extra hoop, there is nothing you can do software wise that can protect the level of compromise that you are worried about. The best you can do is separate the key from the ciphertext and hope a partial compromise doesn't share both as scott says.

[ntoskrnl]

CTR IVs need not necessarily be unpredictable, and CBC IVs need not necessarily be unique, but it is critical that CTR IVs are unique and that CBC IVs are unpredictable. Neither extra property hurts though, so better safe than sorry.

[DarkWanderer]

Nope, CBC IV must be unique as well - otherwise, CPA attack is possible.

[ntoskrnl]

Ah, you are right. But IV reuse is not as critical with CBC as it is with CTR.

[DarkWanderer]

It is :) Having IV repeating allows for CPA by detecting the case when two ciphertexts are repeating - violating the "indistinguishable from random" ciphertext property. See materials from coursera.org/course/crypto

[ntoskrnl]

IV reuse with CTR allows plaintext recovery, so it must never happen. IV reuse with CBC allows the attacker to learn certain properties of it if he can choose the plaintext – bad, but not generally critical to security, as long as IV reuse is unpredictable and rare.

[DarkWanderer]

There's no "generally" or "rare" in cryptography. It's either secure, or not. IV reuse is not secure - and this fact is proven both in practice and in theory ;-) Read some theory about CPA/CCA attacks. The theory is actually pretty counterintuitive, but works.

[ntoskrnl]

I meant "rare" as in "no need to keep a database of used up IVs" as might be necessary in some cases with CTR. But yes, I agree, don't ever reuse IVs.

[Simon Mourier]

@ScottChamberlain - you should mention "the hardware security module" in your answer. IMHO, that's the only interesting thing I learned here, but it's worth a +1 - no offense :-)

[ChiYoung]

I’m close to the same architecture as advised by option 1. Never less, in this case I can’t use DPAPI. I hope I can use a certificate to symmetrically encrypt the KEY to add an extra level of protection (…never did it before…so this is new for me). Also, I can use the SecureString Class (System.Security) to protect the un-encrypted KEY in memory. Do you have any remarks on using certificates to protect/encrypt/decrypt KEY's? The IV’s are indeed clear :) Thnx a lot Syon!

[Syon]

Certificates are fine for an extra layer. An easy to use tool for creating certs is XCA. You'll want to export the cert with it's private key to a PKCS12 (p12) file, and then either load it in to the system keystore (and load it from the store in code), or load the p12 file directly to an X509Certificate2.

[Syon]

After that you can get the RSACryptoServiceProvider.

[Anthony Rutledge]

@Syon I was wondering if you had any thoughts about this question: stackoverflow.com/questions/37825261/…

[MsBao]

@ChiYoung I think using SecureString is giving you a false sense of security. If the secret was/is ever a normal String before or after becoming a SecureString, that plaintext secret may still be in memory. SecureString has the most utility when used in conjunction with API's designed for it. If you have created a SecureString from a plaintext string or derive a plaintext string from SecureString, you're better off not using SecureString at all, because it's just complicating your code without adding any of its benefits.