How do API Keys and Secret Keys work? Would it be secure if I have to pass my API and secret keys to another application?

Question

I am just starting to think about how api keys and secret keys work. Just 2 days ago I signed up for Amazon S3 and installed the S3Fox Plugin. They asked me for both my Access Key and Secret Access Key, both of which require me to login to access.

So I'm wondering, if they're asking me for my secret key, they must be storing it somewhere right? Isn't that basically the same thing as asking me for my credit card numbers or password and storing that in their own database?

How are secret keys and api keys supposed to work? How secret do they need to be? Are these applications that use the secret keys storing it somehow?

Solution 1

Basically elaborating on what's outlined here.

Here's how it works: let's say we have a function that takes a number from zero through nine, adds three and, if the result is greater than ten, subtracts ten. So f(2) = 5, f(8) = 1, etc. Now, we can make another function, call it f', that goes backwards, by adding seven instead of three. f'(5) = 2, f'(1) = 8, etc.

That's an example of a two-way function and its inverse. Theoretically, any mathematical functions that maps one thing to another can be reversed. In practice, though, you can make a function that scrambles its input so well that it's incredibly difficult to reverse.

Taking an input and applying a one-way function is called "hashing" the input, and what Amazon stores on their system is a "hash" of your secret key. SHA1 is an example of this kind of "one-way" function, it's also hardened against attacks.

The HMAC function builds on established hash functions to use a known key to authenticate a string of text. It works like this:

• You take the text of your request and your secret key and apply the HMAC function.
• You add that authentication header to your request and send it to Amazon.
• Amazon looks up their copy of the secret key, and the text you just sent and applies the HMAC function.
• If the result matches, they know that you have the same secret key.

The difference between this and PKI is that this method is RESTful, allowing a minimum number of exchanges between your system and Amazon's servers.

Isn't that basically the same thing as asking me for my credit card numbers or password and storing that in their own database?

Yes, though the damage someone can do with S3 seems to be limited to draining your account.

How secret do they need to be? Are these applications that use the secret keys storing it somehow?

At some point, you're going to have to load the secret key, and with most Unix based systems, if an attacker can get root access they can get the key. If you encrypt the key, you have to have code to decrypt it, and at some point the decryption code has to be plain text so it can be executed. This is the same problem DRM has, except that you own the computer.

In many cases, I just put secret keys in a file with limited permissions, and take the usual precautions to prevent my system from being rooted. There are a few tricks to make it work properly with a multiuser system, such as avoiding temporary files and such.

Solution 2

Public Key Cryptography is used to defend against very specific attacks, some of which are common. In short this is complex math that allows one to verify that at individual has both the Public and Private Key pair while only knowing the public key. This is very different from a credit card or static password. As an example if you are authenticating with an OpenSSH server then the server doesn't need the private key.

Ideally if Amazon's API database where to be compromised the attacker would have a list of public keys and would be unable to access the user's API using this information. However ideal systems are not always put into practice and i don't know for sure if Amazon is protecting against this attack vector, but they should be.

In public key authentication is statistically immune to brute force. Passwords are often dictionary words which can be broken relativity fast. However a private key is a massive number that isn't easy to guess. If the attacker had the public key then they could perform many guesses "offline" on a super computer, but even then it would take a lot of time and money to break the key.

Solution 3

AWS has designed their own custom authentication algorithm. v4 was released in 2014. Details are outlined here: Authenticating Requests (AWS Signature Version 4) . A major point is that the request is not signed with the secret itself, but with a signing key which is generated using the secret. It also uses HMAC-SHA256 for signing.

Using asymmetric keys would be safer since AWS would only store a public key instead of the secret, which is stored by both the user and AWS.

Comments

[Franklin]

"Taking an input and applying a one-way function is called "hashing" the input, and what Amazon stores on their system is a "hash" of your secret key" - If Amazon stores a HASH of your secret key, how is it possible for Amazon to HASH the text sent to them ?

[Sean]

First you say "what Amazon stores on their system is a "hash" of your secret key" and then later "Amazon looks up their copy of the secret key". These seem to contradict each other. I believe the first statement is wrong.

[landed]

foursquare for example have a client ID and a client secret you get when you sign up. And I need to send both to get a response. So isnt this just going to be able to be seen and intercepted on the 'wire'. They dont make mention of making or forcing https. There is also a common scenario of a serverless App these days so it was a) send the request to FS via the server (still able to see intercepted) b) send via client browser to FS (very easy to spot the client key and secret.) What am I missing...the secret is surely like a password you must never SEND it http always https.

[asyncwait]

This url tells more details of Amazon S3 Auth implementation - docs.aws.amazon.com/AmazonS3/latest/dev/S3_Authentication2.h‌​tml

[Leo]

"Theoretically, any mathematical functions that maps one thing to another can be reversed" - Thats not true, hash functions are the example. it is very easy to show. lets say we have a function that turns words into numbers, based on sum of values(a=1, b=2, c=3 etc). Eg "SO" would be 18 + 14 = 32. So we have changed SO into 32 but if i reveal this function to somebody, and give him number 32, there is no way he can know if our basic word was "SO" or "ZF"(26+6) or one of dozens other possibilities

[Matthias Kleine]

@Leo Correct, but you can try to manipulate the string until the hash matches the original one. I know that you can cause a collision in MD5.But in most scenarios, the "new" content with the same hash will make no sense.

[setzamora]

the doesn't need the private key link is broken now.

[oligofren]

@Joset updated the link pointing to a copy in the internet wayback machine from 2008

[cowlinator]

According to the document that @asyncwait linked, amazon definitely stores your secret key, not just a hash of it. In fact, it looks like the only hashing going on is whatever occurs inside the HMAC function

[aug]

@cowlinator yeah I also found that bit interesting. I somewhat believe it relates to the fact that they provide a Secrets manager service so they are simply trying to reuse their own products internally.

[JBaczuk]

The signing algorithm has changed somewhat as of 2014 (see v4 docs.aws.amazon.com/AmazonS3/latest/API/…) It uses sha256 now, for example.