Non-random salt for password hashes

Question

UPDATE: I recently learned from this question that in the entire discussion below, I (and I am sure others did too) was a bit confusing: What I keep calling a rainbow table, is in fact called a hash table. Rainbow tables are more complex creatures, and are actually a variant of Hellman Hash Chains. Though I believe the answer is still the same (since it doesn't come down to cryptanalysis), some of the discussion might be a bit skewed.
The question: "What are rainbow tables and how are they used?"

---

Typically, I always recommend using a cryptographically-strong random value as salt, to be used with hash functions (e.g. for passwords), such as to protect against Rainbow Table attacks.

But is it actually cryptographically necessary for the salt to be random? Would any unique value (unique per user, e.g. userId) suffice in this regard? It would in fact prevent using a single Rainbow Table to crack all (or most) passwords in the system...
But does lack of entropy really weaken the cryptographic strength of the hash functions?

---

Note, I am not asking about why to use salt, how to protect it (it doesn't need to be), using a single constant hash (don't), or what kind of hash function to use.
Just whether salt needs entropy or not.

---

Thanks all for the answers so far, but I'd like to focus on the areas I'm (a little) less familiar with. Mainly implications for cryptanalysis - I'd appreciate most if anyone has some input from the crypto-mathematical PoV.
Also, if there are additional vectors that hadn't been considered, that's great input too (see @Dave Sherohman point on multiple systems).
Beyond that, if you have any theory, idea or best practice - please back this up either with proof, attack scenario, or empirical evidence. Or even valid considerations for acceptable trade-offs... I'm familiar with Best Practice (capital B capital P) on the subject, I'd like to prove what value this actually provides.

---

EDIT: Some really good answers here, but I think as @Dave says, it comes down to Rainbow Tables for common user names... and possible less common names too. However, what if my usernames are globally unique? Not necessarily unique for my system, but per each user - e.g. email address.
There would be no incentive to build a RT for a single user (as @Dave emphasized, the salt is not kept secret), and this would still prevent clustering. Only issue would be that I might have the same email and password on a different site - but salt wouldnt prevent that anyway.
So, it comes back down to cryptanalysis - IS the entropy necessary, or not? (My current thinking is it's not necessary from a cryptanalysis point of view, but it is from other practical reasons.)

Solution 1

Salt is traditionally stored as a prefix to the hashed password. This already makes it known to any attacker with access to the password hash. Using the username as salt or not does not affect that knowledge and, therefore, it would have no effect on single-system security.

However, using the username or any other user-controlled value as salt would reduce cross-system security, as a user who had the same username and password on multiple systems which use the same password hashing algorithm would end up with the same password hash on each of those systems. I do not consider this a significant liability because I, as an attacker, would try passwords that a target account is known to have used on other systems first before attempting any other means of compromising the account. Identical hashes would only tell me in advance that the known password would work, they would not make the actual attack any easier. (Note, though, that a quick comparison of the account databases would provide a list of higher-priority targets, since it would tell me who is and who isn't reusing passwords.)

The greater danger from this idea is that usernames are commonly reused - just about any site you care to visit will have a user account named "Dave", for example, and "admin" or "root" are even more common - which would make construction of rainbow tables targeting users with those common names much easier and more effective.

Both of these flaws could be effectively addressed by adding a second salt value (either fixed and hidden or exposed like standard salt) to the password before hashing it, but, at that point, you may as well just be using standard entropic salt anyhow instead of working the username into it.

Edited to Add: A lot of people are talking about entropy and whether entropy in salt is important. It is, but not for the reason most of the comments on it seem to think.

The general thought seems to be that entropy is important so that the salt will be difficult for an attacker to guess. This is incorrect and, in fact, completely irrelevant. As has been pointed out a few times by various people, attacks which will be affected by salt can only be made by someone with the password database and someone with the password database can just look to see what each account's salt is. Whether it's guessable or not doesn't matter when you can trivially look it up.

The reason that entropy is important is to avoid clustering of salt values. If the salt is based on username and you know that most systems will have an account named either "root" or "admin", then you can make a rainbow table for those two salts and it will crack most systems. If, on the other hand, a random 16-bit salt is used and the random values have roughly even distribution, then you need a rainbow table for all 2^16 possible salts.

It's not about preventing the attacker from knowing what an individual account's salt is, it's about not giving them the big, fat target of a single salt that will be used on a substantial proportion of potential targets.

Solution 2

Using a high-entropy salt is absolutely necessary to store passwords securely.

Take my username 'gs' and add it to my password 'MyPassword' gives gsMyPassword. This is easily broken using a rainbow-table because if the username hasn't got enough entropy it could be that this value is already stored in the rainbow-table, especially if the username is short.

Another problem are attacks where you know that a user participates in two or more services. There are lots of common usernames, probably the most important ones are admin and root. If somebody created a rainbow-table that have salts with the most common usernames, he could use them to compromise accounts.

They used to have a 12-bit salt. 12 bit are 4096 different combinations. That was not secure enough because that much information can be easily stored nowadays. The same applies for the 4096 most used usernames. It's likely that a few of your users will be choosing a username that belongs to the most common usernames.

I've found this password checker which works out the entropy of your password. Having smaller entropy in passwords (like by using usernames) makes it much easier for rainbowtables as they try to cover at least all passwords with low entropy, because they are more likely to occur.

Solution 3

It is true that the username alone may be problematic since people may share usernames among different website. But it should be rather unproblematic if the users had a different name on each website. So why not just make it unique on each website. Hash the password somewhat like this

hashfunction("www.yourpage.com/"+username+"/"+password)

This should solve the problem. I'm not a master of cryptanalysis, but I sure doubt that the fact that we don't use high entropy would make the hash any weaker.

Solution 4

I like to use both: a high-entropy random per-record salt, plus the unique ID of the record itself.

Though this doesn't add much to security against dictionary attacks, etc., it does remove the fringe case where someone copies their salt and hash to another record with the intention of replacing the password with their own.

(Admittedly it's hard to think of a circumstance where this applies, but I can see no harm in belts and braces when it comes to security.)

Solution 5

The strength of a hash function is not determined by its input!

Using a salt that is known to the attacker obviously makes constructing a rainbow table (particularly for hard-coded usernames like root) more attractive, but it doesn't weaken the hash. Using a salt which is unknown to the attacker will make the system harder to attack.

The concatenation of a username and password might still provide an entry for an intelligent rainbow table, so using a salt of a series pseudo-random characters, stored with the hashed password is probably a better idea. As an illustration, if I had username "potato" and password "beer", the concatenated input for your hash is "potatobeer", which is a reasonable entry for a rainbow table.

Changing the salt each time the user changes their password might help to defeat prolonged attacks, as would the enforcement of a reasonable password policy, e.g. mixed case, punctuation, min length, change after n weeks.

However, I would say your choice of digest algorithm is more important. Use of SHA-512 is going to prove to be more of a pain for someone generating a rainbow table than MD5, for example.

View more solutions

Comments

[Martin Carpenter]

I don't understand this comment. You say "use entropy", and then: "use time"??

[Martin Carpenter]

The strength of the function does not change, but the output definitely changes. If the input can be influenced or known, then perhaps something can be deduced about the hash value. That's the risk.

[David Grant]

@Martin: The only thing you should be able to deduce from the hash value if whether or not you have a match! Putting "roota" or "rootb" (where "a" and "b" represent the password) into a hash function will give you radically different output.

[dmajkic]

if Username is used for salt, it' not entropic enough. But if you use username+current dateime - it is, since it's hard to guess when exactly salt was created.

[Martin Carpenter]

"entropic enough" is subjective; it's your standard. Basing this value on time may not be adequate.

[Stefan]

The salts are known to an attacker using rainbow-tables.

[Stefan]

A constant hash is worth nothing. All passwords can be cracked using a single rainbow-table. The purpose of the salt is that it's impossible to create a rainbow-table.

[dmajkic]

There is no such a thing as "absolute true randomness". It's on us to say when it's "good enough". If you think that in this case time is not good enough, use something else. stackoverflow.com/questions/84556/…

[David Grant]

+1 for a good point, however, you are talking about a potentially massive rainbow table. To cover off all values of gsMyPassword length (assuming mixed-case alphanumeric) requires 36^12 rows in the table!

[Steve Jessop]

"The point of the salt, is so that you can't use standard rainbow table to solve every password in the database". I disagree. The point is so that you can't use a standard rainbow table to solve any password. If the salt is the username, then the rainbow table for "root" could crack root's pw.

[David Grant]

As a follow up: the distributed rainbow table project has 63,970 cracked hashes, but 36^12 is 4,738,381,338,321,616,896!

[Steve Jessop]

Agreed. I'd put more emphasis on re-use of usernames, since I think that's the attack most likely to be successful against hypothetical systems using the username as a salt, that would have failed against a system using random salt.

[Kibbee]

That's probably about the only rule that really matters. Basically you want the password to be something that's unique on your system, but it doesn't matter what it is. It could still be something simple though. You don't need a whole lot of entropy.

[Guillaume]

@Potato: you assume a rainbow table containing all possible combinations of alphanumeric characters. This doesnt need to be the case, an effective rainbow table will contain hashes for common passwords/hashes. Salt is there to protect against dictionary attacks or combined rainbowtable/dictionary.

[AviD]

I appreciate your point on cross-system security. Also, I guess that in the future its not unlikely to see user-specific rainbow tables...

[David Grant]

@AviD: You think so? That's quite a specific attack vector.

[Stefan]

@ Mr Potate Head: My computer can generate 1000 hashes per second, 63,970 hashes doesn't say how many hashes the distributed rainbow table has stored.

[Stefan]

No, if for example you want to have compromised accounts to send spam you don't mind what the account name is.

[David Grant]

@gs: I meant that using the username as salt is quite a specific vector.

[AviD]

Actually, the point of salt is to cause each hash result to be unique - in order to prevent (a) rainbow table attacks, and (b) identical password identification. The question is, what else is entropy needed for, if at all?

[David Grant]

@gs: Since I would imagine the project has more than one minute's CPU time, I guess my figure is wrong. :P

[Stefan]

The server has to know the unencrypted salt (to check the password against the hash), thus one can take for granted that an attacker has access to the salt, or can get it very easily.

[Stefan]

@ Mr Potate Head: It says something about "10181 million chains completed", how many hashes might that be?

[AviD]

Right, right, thats a given - to be more specific I meant the strength of the overall cryptosystem (or -subsystem, wrt hashes).

[AviD]

Again, I'm not referring to using a single constant salt - rather a non-random, but user-unique value. E.g. userId.

[AviD]

This was my thinking too - but I got stuck at the "probably okay". I still don't see this theory being proved - or at least verified that there are no cryptanalysis implications.

[David Grant]

@AviD: Since you a username + password concatenation might still give you an entry from a intelligent rainbow table, I would say it was better to use a random series of characters as your salt, and store the salt in the same place. If you like, you could change the salt each time the user logs in.

[David Grant]

@gs: Not sure - I've seen something about chains, but more interesting are the databases - they go up to 8 mixed-alphanumeric characters!

[teedyay]

Depends on what the ID is: I use the primary key of the table, which you can't really change at will. TBH, if the hacker's writing to your database, you're in some pretty big trouble already...

[brady]

Nah, I like it. An attacker is often an "insider". I could dream up scenarios where what he's after isn't in the database, but if he can overwrite the credentials in the database, he can authenticate himself to do what he wants.

[brunocascio]

@gs - I don't see why you can't construct a rainbow table that "includes" the effects of a constant salt. The attack space (the password) won't have changed, so the table won't need to grow, just be recalculated.

[brunocascio]

@Potato - depends on the tradeoff. If your company's 20k logins were all hashed with the same constant salt, it may be cheaper to compute a fresh rainbow table than to dictionary attack them all individually. Hence my emphasis of "BULK".

[teedyay]

Good point. We do find it makes it increasingly difficult to add the first user to a new database: we've effectively made our applications so secure we can barely hack them ourselves. [Also, use an application-specific salt so you can't copy credentials from one database to another.]

[Martin Carpenter]

Imagine a system that creates batches of accounts (random passwords) on the first day of each month. Imagine it uses a low granularity time for salt (or is a really fast system that creates hundreds of accounts per time unit). Now we're vulnerable to both AviD's (a) and (b).

[Martin Carpenter]

@AviD: and if we turn your question around: When is it bad to use high-entropy salt? Extra CPU cycles? Predictable PRNG?

[Kibbee]

Along with nsayer. You could use a system-wide constant string, that there wouldn't be a pregenerated RT for, like #(D83d8, along with the user name as the salt, and that would probalby prevent any rainbow table attacks.

[AviD]

However, in most applications / business systems, there is no builtin "Administrator", "root", etc.

[AviD]

@Martin, the problem isnt actually one of high-entropy - rather the "complexity" of any additional information, seeing as how we already have a unique username its a hard to convince them of the necessity...

[Valeria]

In this case, is it worthwhile to invest in the additional time of a cryptographically secure random number generator? Any RNG will prevent user login collisions.

[Lawrence Hutton]

Not for generating salts, no. The only attacks which are affected by salt are those made directly against the hashed passwords. An attack can't make this sort of attack unless they have a copy of your password database, which will also contain the salts. Since the attacker will already have the salts in his possession, they only require sufficient entropy to avoid having multiple passwords hashed with the same salt, which any basic (P)RNG will provide.

[Lawrence Hutton]

Third sentence should start "An attacker can't make...".

[Stefan]

I believe you're right that this is sufficient. However, given that hashes are a complicated mathematical field and it's very well possible that low entropy salts make the hashes more easily to guess in the future, (especially as hashes get broken), I wouldn't bet on the security, when the proven more secure solution is not much more expensive.

[Admin]

So is salt prefixed to a password secure? for example if i store the password as passhash = md5("urhu2389udfcbdvalk" + UserPassword) would that be secure?

[Lawrence Hutton]

@acidzombie24: Using a single fixed salt (usually called a "nonce" in my experience) for all passwords is less secure than using a unique random salt for each password, as it still allows an attacker to trivially determine whether two or more accounts share the same password. On the other hand, the nonce would likely be stored in your code rather than in the database, so an attacker who only has your database won't have access to it; because of this, the most secure option is to use both a system-wide nonce (stored in the code) and a per-password salt (stored in the database).

[Yoshiyahu]

+1 for the last paragraph. It made me understand needing salt-per-user rather than single-use-salt.

[JMac]

@DaveSherohman very informative, based on this advice, I found a very nice Java library that does what you've suggested: jasypt.org/encrypting-passwords.html

[Sedat Kapanoglu]

Using usernames also creates an unnecessary coupling with the password so you cannot create a feature of "change username" without asking for password anymore.

[Sedat Kapanoglu]

"High entropy salt" isn't bigger search space because "the salt part" is always known. Let's say you have 4096-bit salt and 8-bit password. As long as you know the salt, your search space is just 256.

[Andrew]

Finally I found someone saying this: adding something user-specific to the salt. That avoids an intruder to copy some credentials to another user, and also prevents a hacker to guess any password if he manages to reach your hash and salt field in the table. One thing more I like to do: not using a round number of iterations in the hashing. Don't go for 10k or 20k, but rather something like 19835.