I have been working with APIs and I've always wondered why you have to use a key and a secret?
Why do you need two types of authentication?
When a server receives an API call, it needs to know two things: Who is making the call, and whether or not the call is legitimate.
If you just had one item ("key"), and included it with every call, it would answer both questions. Based on the "key" the server knows who you are, and because only you know the key it proves that the call is actually coming from you. But including the key with every call is bad security practice: if someone can read even one of your messages in transit, your key is compromised, and someone can pretend to be you. So unless you're using HTTPS, this approach doesn't work.
Instead, you can include a digital signature with every call, signed with some "secret" number. (The "secret" number itself is not sent). If an attacker manages to read your message, they won't be able to figure out this "secret" number from the signature. (This is how digital signatures work: they are one-way).
But this doesn't solve the identification question: In the latter case, how does the server know who is making the call? It could try to verify the signature against the "secret" of every single user, but of course this would be very time-consuming.
So, here's what we do: Send both a "key" (that identifies the user), and a signature created using the "secret" number (that proves that the message is legitimate). The server looks up the user based on the key, and then validates the signature using that user's "secret" number.
This is a bit like when you write a check: It has an account number on it (to identify you) and your signature (to prove that you're you). Having just the account number wouldn't prove that you actually wrote the check. Having just the signature without the account number would force the bank to compare your check against all of its signatures for all of its accounts, which would obviously be inefficient.
Related
There're plenty of sites, where you have to sign their 'sign in' message in order to get JWT from them. For example, https://www.cryptokitties.co uses such login system. It verifies the signature on the back-end and sends JWT back if address matches. It works good, but such approach disturbs me in the matter of security.
Assume, that someone has created absolutely identical to cryptokitties fake website. User hasn't noticed that domain is different, signs the same message ("To avoid digital cat burglars, sign below to authenticate with CryptoKitties") and at this point he already provided scammer with his signature and address, as message was the same, therefore it will work on original website. So basically you can loose your account by signing the same message on the completely different site. The saddest part, is that you cannot reset the private key, which means that your account has gone for good.
I'm not an expert, but it seems to me like a huge hole in security.
The solution I'm thinking about, is to encrypt the signature on the client before sending it on the back-end. With such approach, back-end will only send you a JWT if you've signed a message on our front-end. So, firstly back-end decrypts the signature and then verifies the message and address. It will skip signatures which were created on other sites as the decryption will fail.
So far we eliminated fake websites problem. But there is another one: attacker can intercept an already encrypted signature and use it on our site. And once again there is no way to reset the signature, it'll remain the same. So what I came up with is, signature must be disposable, it can be used only once. Before signing a message client requests from the back-end special random number linked with according wallet. Based on this number we build signature message like this: "To avoid digital cat burglars, sign below to authenticate with CryptoKitties #564324". Firstly, back-end decrypts the signature, verifies the address and then checks whether specified random number exists in database. Once login is succeeded, the random number is deleted from the database. Now, even if user looses his signature, it can't be used by attacker, because it's already expired.
What do you think? Does described approach make sense?
You have the right idea with "signature must be disposable". The concept is called a nonce (a value used to protect private communications by preventing replay attacks).
Your following logic is correct as well, except that you don't need to delete the nonce from the database, but rather rotate it. I.e. update the value to a new pseudo-random (or at least hard to guess) value.
I am working on a project where two parties can each make API requests to each other and are authenticating using a shared secret. The other party wants the shared secret to be the same both ways, but I don't understand how that would work. So I'm either looking for an explanation of how to do this securely, or to find out if this is not something that is possible and that the shared secrets should be different based on the direction of the request.
If the API request were just going in one direction, I would salt the password using a key defined function, and store the hash and salt, then I can authenticate the user by generating and matching the hash on each request. But if I need to use that same shared secret to make requests to the other API, then it seems like I would need to store the password in a way that it can be decrypted, which seems wrong/not possible.
Is there a way of doing this, or should the shared secret be different depending on which direction the request is going?
Your analysis is correct. If both side can be the caller, then both sides need to know the secret (not just be able to verify it).
Having two separate keys (one for each caller, you might as well call them "passwords" in this use case) seems to be a reasonable default setup.
It is always good to think of "parties" and give each their own credentials, especially if more parties will be involved later. Makes it much easier to revoke individual access rights later on, to enforce fine-grained access control, and minimizes the impact of leaked credentials (an attacker can only impersonate that one party, not anyone else in the system).
Having just one may appear slightly easier to manage, but then you become responsible for not leaking the other guy's key (in addition to your own). I would try to avoid that extra burden.
I came across many APIs that give the user both an API key and a secret. But my question is: what is the difference between both?
In my eyes, one key can be enough. Say I have a key and only I and the server know it. I create a HMAC hash with this key and do an API call. On the server, we create the HMAC hash again and compare it with the sent hash. If it's the same, the call is authenticated.
So why use two keys?
Edit: or is that API key used to lookup the API secret?
You need two separate keys, one that tells them who you are, and the other one that proves you are who you say you are.
The "key" is your user ID, and the "secret" is your password. They just use the "key" and "secret" terms because that's how they've implemented it.
Simple answer, if I understood it correctly...
If you use your API key for encryption, how will the service know who is contacting them? How will they decrypt that message?
You use API key to state who you are, this is what you are sending in plain text.
The SECRET key you do not send to anyone. You simply use it for encryption. Then you send the encrypted message. You do not send the key that was used for encryption, that would defeat the purpose.
One thing that I did not see mentioned here, although it is an extension of Marcus Adams's answer, is that you should not be using a single piece of information to both identify and authenticate a user if there is a possibility of timing attacks, which can use the differences in response times to guess how far a string comparison got.
If you are using a system which uses a "key" to look up the user or credential, that piece of information could be incrementally guessed over time by sending thousands of requests and examining the time that it takes for your database to find (or not find) a record. This is especially true if the "key" is stored in plaintext instead of a one-way hash of the key. You would want to store users's keys in a plaintext or symmetrically-encrypted for if you need to be able to display the key to the user again.
By having a second piece of information, or "secret", you can first look up the user or credential using the "key", which could be vulnerable to a timing attack, then use a timing-safe compare function to check the value of the "secret".
Here is Python's implementation of that function:
https://github.com/python/cpython/blob/cd8295ff758891f21084a6a5ad3403d35dda38f7/Modules/_operator.c#L727
And it is exposed in the hmac lib (and probably others):
https://docs.python.org/3/library/hmac.html#hmac.compare_digest
One thing to note here is that I don't think that this kind of attack will work on values that are hashed or encrypted before lookup, because the values that are being compared change randomly each time a character in the input string changes. I found a good explanation of this here.
Solutions for storing API keys would then be:
Use a separate key and secret, use the key to look up the record, and use a timing-safe compare to check the secret. This allows you to show the user the key and secret to a user again.
Use a separate key and secret, use symmetrical, deterministic encryption on the secret, and do a normal comparison of encrypted secrets. This allows you to show the user the key and secret again, and could save you from having to implement a timing-safe comparison.
Use a separate key and secret, display the secret, hash and store it, then do a normal comparison of the hashed secret. This removes the necessity to use two-way encryption, and has the added benefit of keeping your secret secure if the system is compromised. It has the downside that you cannot show the secret to the user again.
Use a single key, show it to the user once, hash it, then do a normal lookup of the hashed or encrypted key. This uses a single key, but it is not able to be shown to the user again. Has the benefit of keeping keys secure if the system is compromised.
Use a single key, show it to the user once, encrypt it, and do a normal lookup of the encrypted secret. Can be shown to the user again, but at the cost of having keys vulnerable if they system is compromised.
Of these, I think that 3 is the best balance of security and convenience. I have seen this implemented on many websites when getting keys issued.
Also, I invite any actual security experts to critique this answer. I just wanted to get this out there as another discussion point.
There are answers explaining what the secret and (public) key is. It's a public-private key pair that they give confusing names to. But nobody says why the APIs require both, and many APIs only give you one secret! I've also never seen any API's docs explain why they have two keys, so the best I can do is speculate...
It's best to put only your public key in your request and sign the request locally with your private key; sending anything more shouldn't be needed. But some get away with just having the secret in the request. Ok, any good API will use some transport security like TLS (usually over HTTPS). But you're still exposing your private key to the server that way, increasing the risk of them somehow mishandling it (see: GitHub and Twitter's password logging bug recently discovered). And HTTPS is theoretically just as secure, but there are always implementation flaws out there.
But many – actually most it seems – APIs have you send both keys in requests since that's easier than making people do their own signatures; can't have pure cURL examples otherwise! In that case, it's pointless to have them separate. I guess the separate keys are just for in case they change the API later to take advantage of them. Or some have a client library that might do it the more secure way.
I have been following a couple of articles regarding RESTful web services with WCF and more specifically, how to go about authentication in these. The main article I have been referencing is Aaron Skonnard's RESTful Web Services with WCF 3.5. Another one that specifically deals with HMAC authentication is Itai Goldstiens article which is based on Skonnards article.
I am confused about the "User Key" that is referenced to in both articles. I have a client application that is going to require a user to have both a user name and password.
Does this then mean that the key I use to initialise the
System.Security.Cryptography.HMACMD5 class is simply the users
password?
Given the method used to create the Mac in Itai's article
(shown below), am I right is thinking that key is the users
password and text is the string we are using confirm that the
details are in fact correct?
public static string EncodeText(byte[] key, string text, Encoding encoding)
{
HMACMD5 hmacMD5 = new HMACMD5(key);
byte[] textBytes = encoding.GetBytes(text);
byte[] encodedTextBytes =
hmacMD5.ComputeHash(textBytes);
string encodedText =
Convert.ToBase64String(encodedTextBytes);
return encodedText;
}
In my example, the text parameter would be a combination of request uri, a shared secret and timestamp (which will be available as a request header and used to prevent replay attacks).
Is this form of authentication decent? I've come across another thread here that suggests that the method defined in the articles above is "..a (sic) ugly hack." The author doesn't suggest why, but it is discouraging given that I've spent a few hours reading about this and getting it working. However, it's worth noting that the accepted answer on this question talks about a custom HMAC authorisation scheme so it is possible the ugly hack reference is simply the implementation of it rather than the use of HMAC algorithms themselves.
The diagram below if from the wikipedia article on Message Authentication Code. I feel like this should be a secure way to go, but I just want to make sure I understand it's use correctly and also make sure this isn't simply some dated mechanism that has been surpassed by something much better.
The key can be the user's password, but you absolutely should not do this.
First - the key has an optimal length equal to the size of the output hash, and a user's password will rarely be equal to that.
Second, there will never be enough randomness (entropy to use the technical term) in those bytes to be an adequate key.
Third, although you're preventing replay attacks, you're allowing anyone potentially to sign any kind of request, assuming they can also get hold of the shared secret (is that broadcast by the server at some point or is it derived only on the client and server? If broadcast, a man-in-the-middle attack can easily grab and store that - height of paranoia, yes, but I think you should think about it) unless the user changes their password.
Fourth - stop using HMACMD5 - use HMAC-SHA-256 as a minimum.
This key should at the very least be a series of bytes that are generated from the user's password - typically using something like PBKDF2 - however you should also include something transitory that is session-based and which, ideally, can't be known by an attacker.
That said, a lot of people might tell you that I'm being far too paranoid.
Personally I know I'm not an expert in authentication - it's a very delicate balancing act - so I rely on peer-reviewed and proven technologies. SSL (in this case authentication via client certificates), for example, might have it's weaknesses, but most people use it and if one of my systems gets exploited because of an SSL weakness, it's not going to be my fault. However if an exploit occurs because of some weakness that I wasn't clever enough to identify? I'd kick myself out of the front door.
Indidentally, for my rest services I now use SCRAM for authentication, using SHA512 and 512 bits of random salt for the stretching operation (many people will say that's excessive, but I won't have to change it for a while!), and then use a secure token (signed with an HMAC and encrypted with AES) derived from the authentication and other server-only-known information to persist an authenticated session. The token is stateless in the same way that Asp.Net forms authentication cookies are.
The password exchange works very well indeed, is secure even without SSL (in protecting the password) and has the added advantage of authenticating both client and server. The session persistence can be tuned based on the site and client - the token carries its own expiry and absolute expiry values within it, and these can be tuned easily. By encrypting client ID information into that token as well, it's possible to prevent duplication on to another machine by simply comparing the decrypted values from the client-supplied values. Only thing about that is watching out for IP address information, yes it can be spoofed but, primarily, you have to consider legitimate users on roaming networks.
Many sites with registration that ask email confirmation and sites use tokens in their url.
Why do they use it?
For example in case of email confirmation: why just not use registered user id instead of token?! In case of using it in web pages, i didn't get at all!!
Explanation with real applications would be appreciated!
Thanks in advance!
A token in this context is typically a disposable time-limited random string used for verification. A token of (say) 40 characters can be generated easily [such as sha1(microtime() . rand(1, 10000)))], which isn't guessable by the user and isn't brute-forceable (within reason).
For email verification, a token will be generated and linked with your account ID. When you visit the address containing the token, the account gets activated. Since we've established that a token can't be brute-forced or guessed (within reason), we've just established that a certain user does indeed have the email address they gave us.
If we just used their member number, they could do several things to just guess it, thus bypassing the email check entirely.
When logging in or submitting a form of some kind, the term "token" may be used in a slightly different context - it's still a disposable time-limited random string, but it's used to make sure that the person who submitted the form has just come from the form they tried to submit.
For example, say you log into your online banking. They might have a form to transfer money to another bank account. If you go to www.nastysite.com they might include an iframe that points to <iframe src="http://www.mybank.com/send_money.php?amount=9001&to=Joe">. If your bank don't verify that you were actually on the form, that payment will go through, and you won't be best happy. Even if you are on the form, the chances of the correct token on your form being used in the fake page-load are (almost) nil.
This is called "Cross-Site Request Forgery", or CSRF. For some more reading on CSRF, have a look at this Wikipedia article. Also, I've just got that link after writing this post and seen that they use a very similar example to mine - genuine coincidence haha.