"Look, Ma. No Password!" - Secure Hashing in ColdFusion

Wednesday, June 24, 2009

I soaked up tons of great stuff at Frank Kim's recent Sansfire 2009 training in Baltimore. As a matter of fact, it was the single best training event I have ever attended and I highly recommend it. After the module on Java Cryptography Architecture (JCA) -- which was at the end of Day 3 and though my brain was fried -- I realized I had a big gap in my own knowledge in something that should be a fundamental skill for programmers today. Broadly and simply stated, that skill is to be able to protect data in your application. Now, if you're anything like me and you start hearing some terms like symmetric and asymmetric ciphers, MD5, SHA-1/2/3 message digest, cryptographically secure random number generation, etc., you might be inclined back away slowly (or quickly) and leave all that to someone else. Well, I'm here to say that implementing strong cryptography in ColdFusion is a lot easier than I expected. And knowing how do this and why will give you valuable skills that can only make your software safer and your programming stronger.

Let's start with a common web application task - storing credentials in a database. One appealing NIST recommendation is that you should not store the passwords at all. Rather, you should store a secure hash of the password instead. When authentication needs to take place, instead of reading the password from a database and comparing that to the password entered by the user, you retrieve the password hash from the database, hash the password entered in the form, then compare the hashes. If you understand why we do this and just want to see how, skip down to Secure Hashing. If you're mumbling to yourself "That sounds like a fucking waste of time," read the next two paragraphs. The driver here is that passwords are secret and nobody should have access to a user's password other than the user. System administrators and support staff should have the ability to reset passwords but not to read them. You might ask, "But my sites don't contain any sensitive information, why should I care?". Though true, how much do you want to bet that users registered on your site are using the same set of credentials to log into other sites, even banks? Does this possibly make your site a target for harvesting? Could this challenge customers' trust in you over time? Play that tape couple of times ...

The good news is that password hashing is so simple I can see no real reason not to do it. I'm not claiming to make your site secure; I'm simply saying that if you securely store password hashes instead of passwords, you can mitigate a number of potential attacks; for example, the Rainbow Table attack [7]. Note that password hashing differs from storing passwords encrypted. Why is this important? Because anything encrypted needs to be unencrypted to be read. To unencrypt, there must be a key and frequently it will be one key that can be used to read all passwords. We don't ever want passwords to be read by anyone. Period. So, we want to create a strong, secure, one-way hash. Secure Hashing The technical term is Cryptographic Hash Function, and there are several algorithmic variations : most notably, Message Digest (MD5), Secure Hash Algorithm (SHA), and others. Essentially, a hash function generates a unique and fixed-sized representation of some data; aka a Fingerprint. A correct hash will always produce the same result for a given input; e.g., f(x)=y. For example, the string foo, when hashed with MD5 should always produce ACBD18DB4CC2F85CEDEF654FCCC4A4D8 in ColdFusion. If another algorithm is used, the output will be different than this, yet it still should always produce consistent output.

Now, what if there are users with the same passwords? Won't the hashes be the same in the database? Right you are. What a Rainbow Table[7] is, is essentially a large list of passwords that have been hashed using a number of algorithms. The attacker compares the hashes in your database with the ones in his Rainbow Table and then finds the matching password. The solution to this is to create unique hashes using salt and to repeatedly hash the hash. Salting is the process of inserting random data into the original data. Then, we repeat the hashing about 1000 times. This creates a unique and, if done correctly, a more secure hash. Let's start with some usage. We use an example utility component ,Crypto , to perform the hashing. Full source code can be found here.

 <cfset user_id = "bill" />
 <cfset password = "p@ssW0rd" />
 <cfset crypto = createObject('component', 'Crypto') />
 <cfset salt = crypto.genSalt() />
 <cfset passwordHash = crypto.computeHash(password,salt) />
view raw This Gist brought to you by GitHub.

When run, this generates a password hash that looks something like 72C2CA24AD19EF7A8A9FC89184BA17BFA2D65CF3BE4FD2F0BBF3B460BD123550BC2D6BAB3AADF76EEAA4D8B5F6641A73365B21D17158779468049CACC32F0F2E and a base64 secure random salt value of 4OP5GFmp/z0YYgUUlprVoig4pkGWhNGgMl37Tn4sLPw=. When a user creates or changes their password, store only the hash and salt:

 <cfquery name="q" datasource="users">
    insert into users
    (user_id,password_hash,salt)
    values
    ('#user_id#','#passwordHash#','#salt#')
 </cfquery>
view raw This Gist brought to you by GitHub.

Now, to authenticate a user, you do something like this:

 <!-------------------------------------------------------------------------
Assume a login form with username and pasword:
form.user_id and form.password
-------------------------------------------------------------------------->
 
  <cfquery name="loginQuery" datasource="users" maxrows="1">
    SELECT password_hash, salt
    FROM USERS where user_id = <cfqueryparam cfsqltype="CF_SQL_VARCHAR" value="#form.user_id#" maxlength="12" />
  </cfquery>
 
  <cfset hashedFormPassword = crypto.computeHash(form.password, loginQuery.salt) />
 
  <cfif loginQuery.password_hash eq hashedFormPassword>
   Valid user.
  <cfelse>
    Invalid user.
  </cfif>
view raw This Gist brought to you by GitHub.
We're simply comparing the hashed value of the password entered by the user to the hashed value stored in the database. We do this by first retrieving the salt from the database and using that to generate the hashedFormPassword value. If you've read this far, I'm sure you want to look under the hood:
<cffunction name="computeHash" access="public" returntype="String">
  <cfargument name="password" type="string" />
  <cfargument name="salt" type="string" />
  <cfargument name="iterations" type="numeric" required="false" default="1024" />
  <cfargument name="algorithm" type="string" required="false" default="SHA512" />
  <cfscript>
    var hashed = '';
    var i = 1;
    hashed = hash( password & salt, arguments.algorithm, 'UTF-8' );
    for (i = 1; i <= iterations; i++) {
      hashed = hash( hashed & salt, arguments.algorithm, 'UTF-8' );
    }
    return hashed;
  </cfscript>
</cffunction>
view raw This Gist brought to you by GitHub.

It doesn't get much easier for a complex topic, does it? computeHash() accepts 4 arguments: password, salt, iterations, and algorithm. The required ones are password and salt - salt should be a secure random base64 string. iterations is the number of times the hash will be rehashed. PKCS 5 recommends doing this "a modest" 1000 times[9]. This would be a performance concern if this operation were done frequently, but since it is done only when a user logs in or creates an account, it's considered acceptable. The last parameter, algorithm, is a specific message digest algorithm. It's specified here are SHA512, which is the strongest FIPS-140 approved secure hashing algorithm.

Adobe has some very good documentation on available security providers and other related security best practices - this is required reading for ColdFusion developers! [5] Salt Generation: The core concept behind salt generation is to create a random set of bytes of N length. Since ColdFusion sits on Java, I chose the java.security.SecureRandom class.

<cffunction name="genSalt" access="public" returnType="string">
    <cfargument name="size" type="numeric" required="false" default="16" />
    <cfscript>
     var byteType = createObject('java', 'java.lang.Byte').TYPE;
     var bytes = createObject('java','java.lang.reflect.Array').newInstance( byteType , size);
     var rand = createObject('java', 'java.security.SecureRandom').nextBytes(bytes);
     return toBase64(bytes);
    </cfscript>
</cffunction>
view raw This Gist brought to you by GitHub.

Summary Using the example Crypto component or your own variation will give you a viable option to storing passwords directly in the database and contribute to better security in your application. However, do not take my word alone for this. Please do some thorough research and learn for yourself. When it comes to protecting your customer's data and privacy (and your job) make sure you know what you are doing and why.

Test and be Happy!

References (Many are PDFs):
[1] OWASP Java Hashing - http://www.owasp.org/index.php/Hashing_Java
[2] NIST Secure Hash Standard - http://csrc.nist.gov/publications/fips/fips180-3/fips180-3_final.pdf
[3] FIPS-140-2 Draft (June,18,2009) - http://csrc.nist.gov/publications/fips/fips140-2/fips1402annexa.pdf

[4] Guide to Enterprise Password Management (Draft) - http://csrc.nist.gov/publications/drafts/800-118/draft-sp800-118.pdf
[5] ColdFusion 8 developer security guidelines - http://www.adobe.com/devnet/coldfusion/articles/dev_security/coldfusion_security_cf8.pdf
[6] Java Cryptography Architecture (JCA) - http://java.sun.com/javase/6/docs/technotes/guides/security/crypto/CryptoSpec.html
[7] Rainbow table - http://en.wikipedia.org/wiki/Rainbow_table
[8] NIST Advanced Encryption Standard Algorithm Validation List - http://csrc.nist.gov/groups/STM/cavp/documents/aes/aesval.html [9] PKCS #5 v2.1: Password-Based Cryptography Standard - ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-5v2/pkcs5v2_1.pdf

22 comments:

Henry Ho said...

You may use GenerateSecretKey() to generate a salt.

http://www.cfquickdocs.com/cf8/#GenerateSecretKey

billy said...

@henry, thanks. you know, generateSecretKey() was the first route i took; but, in the end, i felt more comfortable using SecureRandom (
http://java.sun.com/javase/6/docs/api/java/security/SecureRandom.html), simply because the API was more articulate with respect to standards than what's available for generateSecretKey(). i would assume, however, ColdFusion probably uses SecureRandom under the hood. Can anyone confirm? --bill

John Whish said...

Hi Billy, really interesting post.
I was just looking at the computeHash method and the iterations don't seem to do anything. At the moment the code is:

var hashed = '';
hashed = hash( password & salt, arguments.algorithm, 'UTF-8' );
for (i = 1; i <= iterations; i++) {
hashed = hash( password & salt, arguments.algorithm, 'UTF-8' );
}
return hashed;

shouldn't that be:

var hashed = '';
hashed = hash( password & salt, arguments.algorithm, 'UTF-8' );
for (i = 1; i <= iterations; i++) {
hashed = hash( hashed, arguments.algorithm, 'UTF-8' );
}
return hashed;

Otherwise the loop is not re-hashing the string each time. I alos noticed that the "i" isn't var scoped :)

billy said...

@john, good eye! you're absolutely correct about the rehash. thanks for pointing that out! i originally used a different implementation, and introduced that while refactoring. (note to self: find missing test). the gist is updated, so, the post will reflect the new code. i kept the salt in the rehash. any comments on whether that will make the hash stronger or not? --bill

Peter Bell said...

Great post. One quick comment:

If your salt isn't secret (e.g. you're gonna store it with the password in the same db that the same person steals at the same time), you get the benefits of salting using almost anything unique as a salt (that you need to rerun the dictionary attack against each password - not just against the whole db).

Because of that, if I'm not using a secret hash (stored in a separate db with a separate access mechanism on a separate server), I just use a simple derivation of the email/username as the salt. It means you only have to run one query to validate a user instead of two.

Tony E said...

So according to Peter, if the salt and the hashed password are stored in the same table, it's not really securing the password?

What's the best practice for using salt then?

Anonymous said...

La principal ventaja de adicionar una "sal" (provisto que la sal sea lo suficientemente gande y randómica) es obligar al atacante a generar una tabla arcoiris especializada para atacar tu base de datos y no usar una preconstruida.

Desde ese punto de vista, tener una sal "fija" para toda la base de datos es un acierto.

Esto no implica no adicionar otra segunda sal variable para cada registro, de forma que obliguemos al atacante a recostruir la tabla arcoiris por cada registro.

NOTA: No pensar que por este método uno le adicona ENTROPIA a la password, solo la preserva

One of the main advantages of adding a salt, (provided that the salt is not only random but large enough and unique) is to make an attacker to build a new rainbow table, and to force him to not use a pre constructed one.

Form that point of view to have a "fixed" salt for the whole data base is a good prectice.

This method does not implies we cannot add another second salt for each register, and to force the attacker to rebuild the rainbow table for each register.

NOTE: You cannot add "ENTROPY" to the password(s) what is a different concept.


Too Jai

Comentarios /Comments:
karlinga*(at) fastmail(dot) fm


There are interesting tools in Javascript also, to make hashes on the browser side also: see Paj Cripto Home page for more info
http://pajhome.org.uk/

Anonymous said...

It's weird to append a base64 encoded string salt to a literal string password to do the hash.

Shouldn't both be in the same format?

Normal hash functions usually append literal string salt to the literal string password.

This is complex in ColdFusion, because it's not intuitive to get byte arrays into literal strings without encoding. That is, CharsetEncode changes the byte values, which is bad.

It seems that's the reason why there is the Encrypt and EncryptBinary functions.

Encrypt does what you did, including the iterations, but takes a byte array salt.

EncryptBinary does the same, but takes a byte array string and byte array salt.

Anonymous said...

Instead of:
return toBase64(bytes);

You could:
return createObject("java", "java.lang.String").init(bytes);

bill shelton said...

@Anonymous 1

>It's weird ...Shouldn't both be in the same format?
Why? The computeHash() method accepts any salt you want to give it. Consider secure random salt recommendations as per
PKCS #5 v2.1: ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-5v2/pkcs5v2_1.pdf
Maybe I'm missing something, but I don't see the weirdness of generating a hash from two strings one of which is random and one that is not.

>Normal hash functions usually append literal string salt to the literal string password.
"Normal" sounds too broad and general to be a valid point.

>It seems that's the reason why there is the Encrypt and EncryptBinary functions.
Probably ... But in CF7 and in Standard Edition ColdFusion Licenses, you're limitted to 5 basic encryption algorithms. I felt more confident using a hand rolled and recommended method rather than relying on a black box implementation.

@Anonymous 2
That works and makes the string prior to the hashing more readable wahoo ... ;-)

Anonymous said...

> Maybe I'm missing something, but I don't see the weirdness of generating a hash from two strings one of which is random and one that is not.

It's not about random v. not. It's about one string being Base64 and the other being literal.


> >Normal hash functions usually append literal string salt to the literal string password.
> "Normal" sounds too broad and general to be a valid point.

Normal is good.

What if you change programming languages?

Or want to allow other apps to test against the same hashes.

Your algorithm better be repeatable in other languages and implementations.

That's why you should do iterative hashes exactly the same as everyone else.

And concatenating a base64 data and string literal data is not normal.

Anonymous said...

Why not do:


<cffunction name="genSalt" access="public" returnType="string">
   <cfargument name="size" type="numeric" required="false" default="16" />
   <cfscript>
      var byteType = createObject('java', 'java.lang.Byte').TYPE;
      var bytes = createObject('java','java.lang.reflect.Array').newInstance( byteType , size);
      createObject('java', 'java.security.SecureRandom').nextBytes(bytes);
      return createObject("java", "java.lang.String").init(bytes);
   </cfscript>
</cffunction>

bill shelton said...

Maybe you could point me and other readers to the "normal" hashing algorithms to which you refer? I'll quickly correct a mistake, error, or omission, but a good citation would help with that rather than blindly agreeing with an anonymous comment, don't you agree?

bill

Anonymous said...

http://en.wikipedia.org/wiki/Salt_%28cryptography%29

http://www.aspheute.com/english/20040105.asp

http://www.java2s.com/Tutorial/Java/0490__Security/Setpasswordsalt.htm

http://www.jasypt.org/howtoencryptuserpasswords.html

Also, it's important to understand that hashes should ultimately be made from bytes. Not simply strings. In fact, the ColdFusion's Hash function converts the input string to bytes during it's process.

You need to make hashes work the same everywhere. You can provide hashes so others can check integrity on their own. For example, hashing files.

http://en.wikipedia.org/wiki/Examples_of_SHA_digests

bill shelton said...

Good links and points. Thanks!

bill

Smith said...

Hi, Programming an application is different from programming with security in mind. Ordinary programmers mostly do not know today's computer security scenario in the world, and how simple it can take a hacker to break the security of the application in minutes. For more information on hacking and related subject go through the following site:http://www.eccouncil.org/certification/certified_ethical_hacker.aspx

Jamie Krug said...

This is awesome, Bill--must read for any developer not already aware of this important information. I bookmarked this a while back and just came back as I had a good use case for it.

I did need to correct one error in your Crypto.cfc: each instance of "SHA512" should be "SHA-512" to work as expected (missing hyphen).

Thanks!

bill shelton said...

Thanks, Jamie, especially for the SHA-512 correction.

bill

Mark said...

Very nice blog.

Could you comment on the value of using your Crypto class vs. using GenerateSecretKey and Encrypt/Decrypt coldfusion encryption library functions to create the salt and hash.

Mark

bill shelton said...

Thanks Mark,

The Crypto I wrote and the built-in ColdFusion encryption functions "should" do the same things, for the most part. However, the built-in functions are black boxes in which you need to trust Adobe to implement it correctly *and* certify it according to standards. I didn't see any assurances in the docs; so, I felt more comfortable using Java to do the crypto stuff. (It was also a good exercise)

Additionally, I compared the hash() function in CF to one written by hand in Java, and (if I recall correctly) they computed comparable results.

http://github.com/virtix/cfcrypto/blob/master/Crypto.cfc

best,
bill

Mike Henke said...

I fixed some things up like lowercasing the hash to make more compatible with other languages. The cool thing I added was a secure ajax form. https://github.com/mhenke/cfcrypto

Icon Brickell said...

This looks great, I sooo need to try this soon! Thanks!
Icon brickell miami