Related
I came across many APIs that give the user both an API key and a secret. But my question is: what is the difference between both?
In my eyes, one key can be enough. Say I have a key and only I and the server know it. I create a HMAC hash with this key and do an API call. On the server, we create the HMAC hash again and compare it with the sent hash. If it's the same, the call is authenticated.
So why use two keys?
Edit: or is that API key used to lookup the API secret?
You need two separate keys, one that tells them who you are, and the other one that proves you are who you say you are.
The "key" is your user ID, and the "secret" is your password. They just use the "key" and "secret" terms because that's how they've implemented it.
Simple answer, if I understood it correctly...
If you use your API key for encryption, how will the service know who is contacting them? How will they decrypt that message?
You use API key to state who you are, this is what you are sending in plain text.
The SECRET key you do not send to anyone. You simply use it for encryption. Then you send the encrypted message. You do not send the key that was used for encryption, that would defeat the purpose.
One thing that I did not see mentioned here, although it is an extension of Marcus Adams's answer, is that you should not be using a single piece of information to both identify and authenticate a user if there is a possibility of timing attacks, which can use the differences in response times to guess how far a string comparison got.
If you are using a system which uses a "key" to look up the user or credential, that piece of information could be incrementally guessed over time by sending thousands of requests and examining the time that it takes for your database to find (or not find) a record. This is especially true if the "key" is stored in plaintext instead of a one-way hash of the key. You would want to store users's keys in a plaintext or symmetrically-encrypted for if you need to be able to display the key to the user again.
By having a second piece of information, or "secret", you can first look up the user or credential using the "key", which could be vulnerable to a timing attack, then use a timing-safe compare function to check the value of the "secret".
Here is Python's implementation of that function:
https://github.com/python/cpython/blob/cd8295ff758891f21084a6a5ad3403d35dda38f7/Modules/_operator.c#L727
And it is exposed in the hmac lib (and probably others):
https://docs.python.org/3/library/hmac.html#hmac.compare_digest
One thing to note here is that I don't think that this kind of attack will work on values that are hashed or encrypted before lookup, because the values that are being compared change randomly each time a character in the input string changes. I found a good explanation of this here.
Solutions for storing API keys would then be:
Use a separate key and secret, use the key to look up the record, and use a timing-safe compare to check the secret. This allows you to show the user the key and secret to a user again.
Use a separate key and secret, use symmetrical, deterministic encryption on the secret, and do a normal comparison of encrypted secrets. This allows you to show the user the key and secret again, and could save you from having to implement a timing-safe comparison.
Use a separate key and secret, display the secret, hash and store it, then do a normal comparison of the hashed secret. This removes the necessity to use two-way encryption, and has the added benefit of keeping your secret secure if the system is compromised. It has the downside that you cannot show the secret to the user again.
Use a single key, show it to the user once, hash it, then do a normal lookup of the hashed or encrypted key. This uses a single key, but it is not able to be shown to the user again. Has the benefit of keeping keys secure if the system is compromised.
Use a single key, show it to the user once, encrypt it, and do a normal lookup of the encrypted secret. Can be shown to the user again, but at the cost of having keys vulnerable if they system is compromised.
Of these, I think that 3 is the best balance of security and convenience. I have seen this implemented on many websites when getting keys issued.
Also, I invite any actual security experts to critique this answer. I just wanted to get this out there as another discussion point.
There are answers explaining what the secret and (public) key is. It's a public-private key pair that they give confusing names to. But nobody says why the APIs require both, and many APIs only give you one secret! I've also never seen any API's docs explain why they have two keys, so the best I can do is speculate...
It's best to put only your public key in your request and sign the request locally with your private key; sending anything more shouldn't be needed. But some get away with just having the secret in the request. Ok, any good API will use some transport security like TLS (usually over HTTPS). But you're still exposing your private key to the server that way, increasing the risk of them somehow mishandling it (see: GitHub and Twitter's password logging bug recently discovered). And HTTPS is theoretically just as secure, but there are always implementation flaws out there.
But many – actually most it seems – APIs have you send both keys in requests since that's easier than making people do their own signatures; can't have pure cURL examples otherwise! In that case, it's pointless to have them separate. I guess the separate keys are just for in case they change the API later to take advantage of them. Or some have a client library that might do it the more secure way.
I have been following a couple of articles regarding RESTful web services with WCF and more specifically, how to go about authentication in these. The main article I have been referencing is Aaron Skonnard's RESTful Web Services with WCF 3.5. Another one that specifically deals with HMAC authentication is Itai Goldstiens article which is based on Skonnards article.
I am confused about the "User Key" that is referenced to in both articles. I have a client application that is going to require a user to have both a user name and password.
Does this then mean that the key I use to initialise the
System.Security.Cryptography.HMACMD5 class is simply the users
password?
Given the method used to create the Mac in Itai's article
(shown below), am I right is thinking that key is the users
password and text is the string we are using confirm that the
details are in fact correct?
public static string EncodeText(byte[] key, string text, Encoding encoding)
{
HMACMD5 hmacMD5 = new HMACMD5(key);
byte[] textBytes = encoding.GetBytes(text);
byte[] encodedTextBytes =
hmacMD5.ComputeHash(textBytes);
string encodedText =
Convert.ToBase64String(encodedTextBytes);
return encodedText;
}
In my example, the text parameter would be a combination of request uri, a shared secret and timestamp (which will be available as a request header and used to prevent replay attacks).
Is this form of authentication decent? I've come across another thread here that suggests that the method defined in the articles above is "..a (sic) ugly hack." The author doesn't suggest why, but it is discouraging given that I've spent a few hours reading about this and getting it working. However, it's worth noting that the accepted answer on this question talks about a custom HMAC authorisation scheme so it is possible the ugly hack reference is simply the implementation of it rather than the use of HMAC algorithms themselves.
The diagram below if from the wikipedia article on Message Authentication Code. I feel like this should be a secure way to go, but I just want to make sure I understand it's use correctly and also make sure this isn't simply some dated mechanism that has been surpassed by something much better.
The key can be the user's password, but you absolutely should not do this.
First - the key has an optimal length equal to the size of the output hash, and a user's password will rarely be equal to that.
Second, there will never be enough randomness (entropy to use the technical term) in those bytes to be an adequate key.
Third, although you're preventing replay attacks, you're allowing anyone potentially to sign any kind of request, assuming they can also get hold of the shared secret (is that broadcast by the server at some point or is it derived only on the client and server? If broadcast, a man-in-the-middle attack can easily grab and store that - height of paranoia, yes, but I think you should think about it) unless the user changes their password.
Fourth - stop using HMACMD5 - use HMAC-SHA-256 as a minimum.
This key should at the very least be a series of bytes that are generated from the user's password - typically using something like PBKDF2 - however you should also include something transitory that is session-based and which, ideally, can't be known by an attacker.
That said, a lot of people might tell you that I'm being far too paranoid.
Personally I know I'm not an expert in authentication - it's a very delicate balancing act - so I rely on peer-reviewed and proven technologies. SSL (in this case authentication via client certificates), for example, might have it's weaknesses, but most people use it and if one of my systems gets exploited because of an SSL weakness, it's not going to be my fault. However if an exploit occurs because of some weakness that I wasn't clever enough to identify? I'd kick myself out of the front door.
Indidentally, for my rest services I now use SCRAM for authentication, using SHA512 and 512 bits of random salt for the stretching operation (many people will say that's excessive, but I won't have to change it for a while!), and then use a secure token (signed with an HMAC and encrypted with AES) derived from the authentication and other server-only-known information to persist an authenticated session. The token is stateless in the same way that Asp.Net forms authentication cookies are.
The password exchange works very well indeed, is secure even without SSL (in protecting the password) and has the added advantage of authenticating both client and server. The session persistence can be tuned based on the site and client - the token carries its own expiry and absolute expiry values within it, and these can be tuned easily. By encrypting client ID information into that token as well, it's possible to prevent duplication on to another machine by simply comparing the decrypted values from the client-supplied values. Only thing about that is watching out for IP address information, yes it can be spoofed but, primarily, you have to consider legitimate users on roaming networks.
Being unable to locate a working php/javascript implementation of blowfish, I'm now considering using SHA1 hashing to implement web-based authentication, but the lack of knowledge in this particular field makes me unsure of whether the chosen method is secure enough.
The planned roadmap:
User's password is stored on the server as an MD5 hash.
Server issues a public key (MD5 hash of current time in milliseconds)
Client javascript function takes user password as input, and calculates its MD5 hash
Client then concatenates public key and password hash from above, and calculates SHA1 of the resulting string
Client sends SHA1 hash to the server, where similar calculations are performed with public key and user's password MD5 hash
Server compares the hashes, a match indicates successful authentication.
A mismatch indicates authentication failure, and server issues a new public key, effectively expiring the one already used.
Now, the problematic part is about concatenating two keys before SHA1, could that be prone to some kind of statistical or other attacks?
Is there any specific order in which keys should be concatenated to improve the overall quality (i.e. higher bits being more important to reliability of encryption)?
Thank you in advance.
If you're only using the 'public key' (which isn't actually a public key, it's a nonce, and should really be random, unless you really want it to be usable over a certain timeframe, in which case make sure you use HMAC with a secret key to generate it so an adversary cannot predict the nonce) to prevent replay attacks, and it's a fixed size, then concatenation might not be a problem.
That said, I'm a bit concerned that you might not have a well-thought-out security model. What attack is this trying to prevent, anyway? The user's password hash is unsalted, so a break of your password database will reveal plaintext passwords easily enough anyway, and although having a time-limited nonce will mitigate replay attacks from a passive sniffer, such a passive sniffer could just steal the user's session key anyway. Speaking of which, why not just use the session key as the nonce instead of a timestamp-based system?
But really, why not just use SSL? Cryptography is really hard to get right, and people much smarter than you or I have spent decades reviewing SSL's security to get it right.
Edit: If you're worried about MITM attacks, then nothing short of SSL will save you. Period. Mallory can just replace your super-secure login form with one that sends the password in plaintext to him. Game over. And even a passive attacker can see everything going over the wire - including your session cookie. Once Eve has the session cookie, she just injects it into her browser and is already logged in. Game over.
If you say you can't use SSL, you need to take a very hard look at exactly what you're trying to protect, and what kinds of attacks you will mitigate. You're going to probably need to implement a desktop application of some sort to do the cryptography - if MITMs are going around, then you cannot trust ANY of your HTML or Javascript - Mallory can replace them at will. Of course, your desktop app will need to implement key exchange, encryption and authentication on the data stream, plus authentication of the remote host - which is exactly what SSL does. And you'll probably use pretty much the same algorithms as SSL to do it, if you do it right.
If you decide MITMs aren't in scope, but you want to protect against passive attacks, you'll probably need to implement some serious cryptography in Javascript - we're talking about a Diffie-Hellman exchange to generate a session key that is never sent across the wire (HTML5 Web storage, etc), AES in Javascript to protect the key, etc. And at this point you've basically implemented half of SSL in Javascript, only chances are there are more bugs in it - not least of which is the problem that it's quite hard to get secure random numbers in Javascript.
Basically, you have the choice between:
Not implementing any real cryptographic security (apparently not a choice, since you're implementing all these complex authentication protocols)
Implementing something that looks an awful lot like SSL, only probably not as good
Using SSL.
In short - if security matters, use SSL. If you don't have SSL, get it installed. Every platform that I know of that can run JS can also handle SSL, so there's really no excuse.
bdonlan is absolutely correct. As pointed out, an adversary only needs to replace your Javascript form with evil code, which will be trivial over HTTP. Then it's game over.
I would also suggest looking at moving your passwords to SHA-2 with salts, generated using a suitable cryptographic random number generator (i.e. NOT seeded using the server's clock). Also, perform the hash multiple times. See http://www.jasypt.org/howtoencryptuserpasswords.html sections 2 and 3.
MD5 is broken. Do not use MD5.
Your secure scheme needs to be similar to the following:
Everything happens on SSL. The authentication form, the server-side script that verifies the form, the images, etc. Nothing fancy needs to be done here, because SSL does all the hard work for you. Just a simple HTML form that submits the username/password in "plaintext" is all that is really needed, since SSL will encrypt everything.
User creates new password: you generate a random salt (NOT based off the server time, but from good crypto random source). Hash the salt + the new password many times, and store the salt & resulting hash in your database.
Verify password: your script looks up salt for the user, and hashes the salt + entered password many times. Check for match in database.
The only thing that should be stored in your database is the salt and the hash/digest.
Assuming you have a database of MD5 hashes that you need to support, then the solution might be to add database columns for new SHA-2 hashes & salts. When the user logs in, you check against the MD5 hash as you have been doing. If it works, then follow the steps in "user creates new password" to convert it to SHA-2 & salt, and then delete the old MD5 hash. User won't know what happened.
Anything that really deviates from this is probably going to have some security flaws.
Is it any safer to create a table holding user information and another one for their passwords than using the same table for everything?
No I would just do this:
id, username, password.
Where id is just autoincrement, username is a varchar of 20 (or so, depending on your needs) and password is an MD5 or SHA1 hashed password with a salt.
Using two tables for this just doesn't make sense. Then you need to work with joins to get the data. And that's just an unnecessary burden.
No, I cannot see how that can make it safer.
You should actually refrain from storing passwords at all. Just store their salted hash.
Further reading:
Stack Overflow: Preferred Method of Storing Passwords In Database
I disagree with other people - put the authentication information in a separate table and to the greatest extent possible pull authentication out of your application entirely. You shouldn't care. Think siteminder and the like - your web application doesn't have any information about how the user is authenticated. Password, smart card, etc. (Same thing with Kerberos or Active Directory on desktop applications.)
This approach even works if you use a framework like Spring Security. Just set up your interceptor so it looks at the authentication tables alone. You could even use separate DataSources so your interceptor can't see application data or vice versa.
Obviously your application will still need to manage information about the user's authorizations, something usually handled in a 'roles' table. But there's no need to for it to know how the user was authenticated.
i think having the username and password in the same table is ok ,
but also l have seen people doing silly stuff especially when working with the ORM , someone might end up exposing password hashes etc
for example entity framework C#
someone can just do
appcontext.Users.ToList();
no attributes kept ensuring that password is hidden nor DTOs (Data Transfer Object) ,
upon noticing this l just keep another authentication table and the other thing l there are a lot of fields for forgot password, last password change all those fields l will have them in another table with the password
No it is not safer. Just make sure your passwords are Salted+Hashed before you stored them in the DB.
No. Not unless each table required a different user account to access it - which would make querying it a complete pain - and even then, the hacker has worked out one login, so chances are they can get the other.
Just make sure that you are storing passwords in a hashed form, using a secure hash like SHA-2 and a salt. Do not store them in plain text and (IMHO) don't store them encrypted.
Btw, there is already a pretty simple (and powerful) c# salted hash class library (they've also included a small demonstration of how to use the library) out there - Link .
It also provides a verification mechanism (so you can verify the user's input as a valid password) and you can choose the Hash algorithm yourself (Sha1/MD5/etc.)
There is no security benefit, but using multiple tables can be useful for storing credentials to multiple systems tied to a single login.
As mentioned above, security should be provided by salted hash for passwords.
Here's the situation - its a bit different from the other database/password questions on StackOverflow.com
I've got two sets of users. One are the "primary" users. The others are the "secondary" users. Every one has a login/password to my site (say mysite.com - that isn't important).
Background: Primary users have access to a third site (say www.something.com/PrimaryUser1). Every secondary user "belongs" to a primary user and wants access to a subpart of that other site (say www.something.com/PrimaryUser1/SecondaryUser1).
At mysite.com, the primary users have to provide their credentials to me which they use to access www.something.com/PrimaryUser1, and they specify which "subparts" the secondary users of their choice get get access to.
Mysite.com helps manage the sub-access of the secondary users to the primary user's site. The secondary users can't "see" their primary user's password, but through my site, they can access the "subparts" of the other site - but ONLY to their restricted subpart.
In a crude way, I'm implementing OAuth (or something like that).
The question here is - how should I be storing the primary user's credentials to the other site? The key point here is that mysite.com uses these credentials to provide access to the secondary users, so it MUST be able to read it. However, I want to store it in such a way, that the primary users are reassured that I (as the site owner) cannot read their credentials.
I suppose this is more of a theoretical approach question. Is there anything in the world of cryptography that can help me with this?
Text added:
Since most ppl are completely missing the question, here's attempt #2 at explaining it.
PrimaryUser1 has a username/password to www.something.com/PrimaryUser1Site
He wishes to give sub-access to two people- SecondaryUser1 and SecondaryUser2 to the folders- www.something.com/PrimaryUser1Site/SecondaryUser1 and www.something.com/PrimaryUser1Site/SecondaryUser2
Mysite.com takes care of this sub-user management, so PrimaryUser1 goes there and provides his credentials to Mysite.com. MySite.com internally uses the credentials provided by PrimaryUser1 to give subusers limited access. Now, SecondaryUser1 and SecondaryUser2 can access their respective folders on www.something.com/PrimaryUser1Site through the MySite.com
NOW, the question arises, how should I store the credentials that PrimaryUser1 has provided?
First rule: Never, ever store passwords!
Second rule: Calculate a hash over password, with additional salt, and store this in your database.
Third rule: A username (uppercased) could be used as salt, but preferably add a little more as salt! (Some additional text, preferably something long.)
Fourth rule: It doesn't matter how secure a hashing algorithm is, they will all be hacked sooner or later. All it takes is time!
Fifth rule: The security of your site depends on the value of what's behind it. The more value the content has, the more likely that you'll be attacked!
Sixth rule: You will discover, sooner or later, that your site is hacked but not through a hacked password, but through a loophole somewhere else in your code. The biggest risk is expecting your site is secure now you've implemented some strong security.
Seventh rule: All security can be broken, all sites can get hacked, all your secrets can be discovered, if only people are willing to invest enough time to do so.
Security is an illusion but as long as no one breaks it, you can continue to dream on! Always be prepared for rough awakenings that will require you to rebuild your illusion again. (In other words, make regular backups! (Preferably daily.) Don't overwrite the backups of the last week and make sure you keep at least one backup of every week, just in case you discover your site was hacked months ago and all your backups ever since are infected!
Now, if you really need to store passwords, use a hash over username plus password. Then hash again with hash plus salt! Better yet, create a list of salts (just list of words) and whenever a new user account is created, pick a random salt word to use to hash his username plus password. Store the index of the salt with the user account so you know which one to use whenever he logs on again.
And:
Eight rule: Always use HTTPS! It's not as secure as most people thing but it does give a feeling of security to your users!Since you've added text, I'll add more answer.
Since you want user1 to grant temporary access to user 2, you'll need a secondary user table. (Or expand the user table with a parent user ID. Also add a timestamp to keep track of the account age. User 1 can create the credentials and this is done in the normal way. Just store a hash with combined username and salt. In this case, use the username of user 1 as additional salt! Just make sure you'll disable the user 2 account when user 1 logs off or when a certain amount of time has gone by. And allow user 1 to enable all accounts again that he created, so they can re-use an account instead of having to create new ones all the time.
Security isn't a matter that depend on primary or secondary users. In general, treat them the same way! Secondary users have an added bonus that you can use the primary account as additional salt. The rest of it has nothing to do with authentication any more. It's authorization that you're dealing with. And while authentication and authorization have a strong relationship, be aware that you should treat them as two different, stand-alone techniques.
When user 1 logs on, he's granted access to the primary site. When he grants access to user 2, user 2 gets a reduced set of roles. But this has nothing to do with storing user names or passwords. You just have an user-ID which happens to be member of certain roles, or groups. Or not, but those would be inaccessible.
They're both just users, one with more rights than the other.
It depends on the kind of authentication your primary site and the secondary site agree on. Is it forms authentication, HTTP Basic or HTTP Digest? If is forms or basic then you have no choice, you must store the password, so your only choice is to encrypt it. You cannot store a password hash as you must present the clear text during authentication for both forms and HTTP Basic. The problems that arise from storing the encrypted password are due to either incorrect use of cryptography (ie. you don't use an IV or salt or you don't use correctly a stream cipher), but more importantly you'll have key management problems (where to store the key used to encrypt the passwords and how to access it from a non-interactive service/demon).
If the 3rd party site accepts HTTP Digest then you're in better luck, you can store the HA1 hash part of the Digest hash (ie. MD5 of username:realm:password) because you can construct the Digest response starting straight from HA1.
I did not address how the user provision the secondary credentials (ie. how you get the secondary site username and password n the first place), I assume you have secured a protected channel (ie. HTTPS from client to your primary site).
BTW this assumes that the authentication occurs between your primary and secondary site and the secondary site content is tunneled through an HTTP request made to the primary site. If that's not the case and the secondary site is actually accessed straight from the browser, then the secondary site must support some sort of pre-authenticated token based authorization of third parties like OAuth. Relying on credential authentication and storing the credentials on the primary site when the credentials are actually needed by the browser has so many problems is not even worth talking about.
Have you thought about accepting OpenID like Stack Overflow does? That way you are not responsible for storing passwords at all.
There is only one way to do this, and it is probably too burdomesome for the users.
You could encrypt the users password with a public/private key, the user keeps their key so the password can be unencrypted only when the key is submitted back to your server. The only way to make this simple would to be to have some web browser plugins that auto submit the information.
And either way, you could always packet sniff the communication to/from the server so its still mostly pointless.
there has got be a better way to explain this :(
but if you just want to know how to store the passwords safely do this:
username:john, password:pass
key = '!!#ijs09789**&*';
md5(username.password.key);
when they login just check to see if md5(username.password.key) = is equal to the one in the db - you can also use sha1 and or any other encryption method.
http://us.php.net/md5 & http://us.php.net/sha1
Never store passwords in a database but store a salted and hashed version of every password.
Check this article if this is chinese for you.
If you want to store the password yourself the best apporach is to use a one-way hashing algorithm such as MD5 or SHA-1. The advantage of this approach is that you cannot derive the password from the hashed value.
Precisely which algorithm you choose depends the precise products you are using. Some front-end tools offer these functions, as do some database products. Otherwise you'll need a third-party library.
Edit
Secondary users ought to have their own passowrds. Why wouldn't they?
You're making it too complex. You need to stop trying to mix authentication and authorization.
What you want to do is establish credentials for everyone, not worrying at this point if they are "primary" or "secondary" users. Then on the main site, where you manage the users and the primary/secondary relationships, you can do the logic of which users are primary or secondary and store all that stuff in a table. You grant or deny whatever rights and sub-rights you wish to each secondary user whenever the primary users update their relationships with them. When they're done, you finally need to replicate the appropriate user credentials from the main site out to the secondary site(s).
Then when a secondary user wants to head to any site in your farm, they authenticate themselves only as themselves - they never impersonate the primary user! And they have only the rights you granted them when the primary users gave them "secondary" status.
--
OK, since you shot that solution down in the comment, consider this:
First, I doubt anything will be truly secure. You can always recover the secret if you monitor the users' activity.
Now, this is completely off the cuff, and I haven't cryptanalyzed it, but check into what is called a secret sharing scheme. Store the "effective" or "real" main-site primary user password as the shared secret. Use the salted hash of the password given by the primary user as one secret. Use the salted hash of the password given by the first secondary user as another secret, and so on for each additional secondary user. Don't store the salted hashes! Just store the salt and the protected shared secret.
When a user enters their password, you retrieve the protected shared secret, use the salt and hash of their password to produce the salted hash, decrypt the protected shared secret, and now you've got the original primary user password.