I have been following a couple of articles regarding RESTful web services with WCF and more specifically, how to go about authentication in these. The main article I have been referencing is Aaron Skonnard's RESTful Web Services with WCF 3.5. Another one that specifically deals with HMAC authentication is Itai Goldstiens article which is based on Skonnards article.
I am confused about the "User Key" that is referenced to in both articles. I have a client application that is going to require a user to have both a user name and password.
Does this then mean that the key I use to initialise the
System.Security.Cryptography.HMACMD5 class is simply the users
password?
Given the method used to create the Mac in Itai's article
(shown below), am I right is thinking that key is the users
password and text is the string we are using confirm that the
details are in fact correct?
public static string EncodeText(byte[] key, string text, Encoding encoding)
{
HMACMD5 hmacMD5 = new HMACMD5(key);
byte[] textBytes = encoding.GetBytes(text);
byte[] encodedTextBytes =
hmacMD5.ComputeHash(textBytes);
string encodedText =
Convert.ToBase64String(encodedTextBytes);
return encodedText;
}
In my example, the text parameter would be a combination of request uri, a shared secret and timestamp (which will be available as a request header and used to prevent replay attacks).
Is this form of authentication decent? I've come across another thread here that suggests that the method defined in the articles above is "..a (sic) ugly hack." The author doesn't suggest why, but it is discouraging given that I've spent a few hours reading about this and getting it working. However, it's worth noting that the accepted answer on this question talks about a custom HMAC authorisation scheme so it is possible the ugly hack reference is simply the implementation of it rather than the use of HMAC algorithms themselves.
The diagram below if from the wikipedia article on Message Authentication Code. I feel like this should be a secure way to go, but I just want to make sure I understand it's use correctly and also make sure this isn't simply some dated mechanism that has been surpassed by something much better.
The key can be the user's password, but you absolutely should not do this.
First - the key has an optimal length equal to the size of the output hash, and a user's password will rarely be equal to that.
Second, there will never be enough randomness (entropy to use the technical term) in those bytes to be an adequate key.
Third, although you're preventing replay attacks, you're allowing anyone potentially to sign any kind of request, assuming they can also get hold of the shared secret (is that broadcast by the server at some point or is it derived only on the client and server? If broadcast, a man-in-the-middle attack can easily grab and store that - height of paranoia, yes, but I think you should think about it) unless the user changes their password.
Fourth - stop using HMACMD5 - use HMAC-SHA-256 as a minimum.
This key should at the very least be a series of bytes that are generated from the user's password - typically using something like PBKDF2 - however you should also include something transitory that is session-based and which, ideally, can't be known by an attacker.
That said, a lot of people might tell you that I'm being far too paranoid.
Personally I know I'm not an expert in authentication - it's a very delicate balancing act - so I rely on peer-reviewed and proven technologies. SSL (in this case authentication via client certificates), for example, might have it's weaknesses, but most people use it and if one of my systems gets exploited because of an SSL weakness, it's not going to be my fault. However if an exploit occurs because of some weakness that I wasn't clever enough to identify? I'd kick myself out of the front door.
Indidentally, for my rest services I now use SCRAM for authentication, using SHA512 and 512 bits of random salt for the stretching operation (many people will say that's excessive, but I won't have to change it for a while!), and then use a secure token (signed with an HMAC and encrypted with AES) derived from the authentication and other server-only-known information to persist an authenticated session. The token is stateless in the same way that Asp.Net forms authentication cookies are.
The password exchange works very well indeed, is secure even without SSL (in protecting the password) and has the added advantage of authenticating both client and server. The session persistence can be tuned based on the site and client - the token carries its own expiry and absolute expiry values within it, and these can be tuned easily. By encrypting client ID information into that token as well, it's possible to prevent duplication on to another machine by simply comparing the decrypted values from the client-supplied values. Only thing about that is watching out for IP address information, yes it can be spoofed but, primarily, you have to consider legitimate users on roaming networks.
Related
It might sound like a silly question, because passwords of course need to be hashed and never store the original.
However, for API secrets, generally I see them displayed in the clear when signing up for them.
For example, if I go to the google api console and look at my credentials page, I can view my client secret, same for twitter.
Surely api keys are just as sensitive as passwords?
Is it just because from the provider side, you can be confident that a sufficiently strong password is being generated?
If that's the case, then that doesn't provide any protection is your database is compromised.
Or is it perhaps because if you are using token based authentication, you're either doing password grant type, which requires you to send your credentials along with the client id and secret, or a refresh token, so a user would have already had to have been compromised?
I can imagine a few possible answers to this:
In some cases, it may be required for the server to have persistent storage of the plaintext API key in order to satisfy usability requirements (Google and Twitter being examples).
In some cases, the API key alone is not enough to do much at all -- additionally one needs to have an authenticated account -- and therefore the API key by itself is of limited value (hence less value than a password).
In a number of cases, the API key is hardcoded in a client application (especially mobile applications, which almost always do this) and therefore it does not make sense to add the extra protection on the server side when the same token can be trivially extracted from the client.
The security industry is just not that mature yet. Maybe once hackers start dumping API keys, ideas like this may be taken more seriously.
BTW, I am very serious about the last point. The truth is that a lot of good ideas don't become a reality until there is a critical mass of support behind them. As an example, I once blogged about a related topic -- protecting user confidential information by hashing it in the database but in a way that it could be recovered when the legitimate user logs in. I used Ashley Madison as an example -- in that case, the hackers were more after email addresses, phone numbers, and physical addresses than passwords. So when the hackers snatched the database, they immediately had what they wanted, and they could care less about the bcrypt encoded passwords (in fact, some older passwords were encoded with only MD5!) Unfortunately, concepts like this do not have enough of a push to make them a reality. Even zero-knowledge web designs are very few in the real world.
I'm working on a project which contains data belonging to multiple clients, and I'm trying to find a secure way to notify their servers of certain sensitive changes to their data.
The issue is that the most secure method I found for this is OAuth, but since my server will be pushing the updates to them, that would mean that each client would have to implement an OAuth provider solely to authenticate my server, and it feels like a bit of an overkill.
My question is: Keeping in mind that not all clients will use HTTPS, would it be enough to simply use a shared secret, a timestamp, and some form of encryption for their servers to safely receive and validate my updates or will that leave them vulnerable to attacks?
Yes, that would be secure. For simple messages I think JTW would be a very good choice. You could use it for just authentication or the actual notification itself. A few reasons you might want to use it:
It's signed, so you know the message hasn't been tampered with.
You can encrypt with public/private key pairs.
You can add any data you like.
It's very simple to implement and doesn't require back-and forth exchange between servers like OAuth often does.
I want my REST API server to be able to communicate only with my iOS app. The user base is going to be no more than 1000 people, and the market is pretty small and unpopular in general. That's why I think anything beyond a simple challenge-response authentication (HTTP, OAuth 2.0, SSL) would be an overkill. But I'm not sure exactly how this auth should flow. Here is what I have in mind:
Client app (user) sends a request: api.example.com/auth?username=john
Server responds with a randomly generated string: "somerandomlygeneratedstring"
Client takes the string, appends it to the username and then appends a secret string, hard coded in the app and uses MD5 to hash the entire string.
Client passes the string to the server: api.example.com/auth?username=john&response=thenewMD5hashstring
Server generates the same MD5 hash string and if they match, marks this user as authenticated in the database and all API requests from this user will be handled from now on.
Do I have the right idea? Or am I totally wrong? Please have in mind, I want basic security, anything too fancy would be an overkill for such a small project.
Also, I'm not keeping any sensitive data on my database like personal information.
You should simply use HTTP Basic auth for every request, through the Authorization header, and have all your interactions over SSL. If you want basic security, there's no need to go beyond that.
There are several problems with the scheme you have in mind.
Your last step is essentially a server-side session, which isn't acceptable in REST.
MD5 is effectively broken and shouldn't be used for anything but integrity checking.
In REST, you should use the standardized authentication method provided by the protocol if it fits your needs. Reinventing it to use URI parameters like you have in mind is unnecessary.
The hashing scheme you have in mind only makes sense when you want to sign the request, guaranteeing it wasn't tampered with.
Being unable to locate a working php/javascript implementation of blowfish, I'm now considering using SHA1 hashing to implement web-based authentication, but the lack of knowledge in this particular field makes me unsure of whether the chosen method is secure enough.
The planned roadmap:
User's password is stored on the server as an MD5 hash.
Server issues a public key (MD5 hash of current time in milliseconds)
Client javascript function takes user password as input, and calculates its MD5 hash
Client then concatenates public key and password hash from above, and calculates SHA1 of the resulting string
Client sends SHA1 hash to the server, where similar calculations are performed with public key and user's password MD5 hash
Server compares the hashes, a match indicates successful authentication.
A mismatch indicates authentication failure, and server issues a new public key, effectively expiring the one already used.
Now, the problematic part is about concatenating two keys before SHA1, could that be prone to some kind of statistical or other attacks?
Is there any specific order in which keys should be concatenated to improve the overall quality (i.e. higher bits being more important to reliability of encryption)?
Thank you in advance.
If you're only using the 'public key' (which isn't actually a public key, it's a nonce, and should really be random, unless you really want it to be usable over a certain timeframe, in which case make sure you use HMAC with a secret key to generate it so an adversary cannot predict the nonce) to prevent replay attacks, and it's a fixed size, then concatenation might not be a problem.
That said, I'm a bit concerned that you might not have a well-thought-out security model. What attack is this trying to prevent, anyway? The user's password hash is unsalted, so a break of your password database will reveal plaintext passwords easily enough anyway, and although having a time-limited nonce will mitigate replay attacks from a passive sniffer, such a passive sniffer could just steal the user's session key anyway. Speaking of which, why not just use the session key as the nonce instead of a timestamp-based system?
But really, why not just use SSL? Cryptography is really hard to get right, and people much smarter than you or I have spent decades reviewing SSL's security to get it right.
Edit: If you're worried about MITM attacks, then nothing short of SSL will save you. Period. Mallory can just replace your super-secure login form with one that sends the password in plaintext to him. Game over. And even a passive attacker can see everything going over the wire - including your session cookie. Once Eve has the session cookie, she just injects it into her browser and is already logged in. Game over.
If you say you can't use SSL, you need to take a very hard look at exactly what you're trying to protect, and what kinds of attacks you will mitigate. You're going to probably need to implement a desktop application of some sort to do the cryptography - if MITMs are going around, then you cannot trust ANY of your HTML or Javascript - Mallory can replace them at will. Of course, your desktop app will need to implement key exchange, encryption and authentication on the data stream, plus authentication of the remote host - which is exactly what SSL does. And you'll probably use pretty much the same algorithms as SSL to do it, if you do it right.
If you decide MITMs aren't in scope, but you want to protect against passive attacks, you'll probably need to implement some serious cryptography in Javascript - we're talking about a Diffie-Hellman exchange to generate a session key that is never sent across the wire (HTML5 Web storage, etc), AES in Javascript to protect the key, etc. And at this point you've basically implemented half of SSL in Javascript, only chances are there are more bugs in it - not least of which is the problem that it's quite hard to get secure random numbers in Javascript.
Basically, you have the choice between:
Not implementing any real cryptographic security (apparently not a choice, since you're implementing all these complex authentication protocols)
Implementing something that looks an awful lot like SSL, only probably not as good
Using SSL.
In short - if security matters, use SSL. If you don't have SSL, get it installed. Every platform that I know of that can run JS can also handle SSL, so there's really no excuse.
bdonlan is absolutely correct. As pointed out, an adversary only needs to replace your Javascript form with evil code, which will be trivial over HTTP. Then it's game over.
I would also suggest looking at moving your passwords to SHA-2 with salts, generated using a suitable cryptographic random number generator (i.e. NOT seeded using the server's clock). Also, perform the hash multiple times. See http://www.jasypt.org/howtoencryptuserpasswords.html sections 2 and 3.
MD5 is broken. Do not use MD5.
Your secure scheme needs to be similar to the following:
Everything happens on SSL. The authentication form, the server-side script that verifies the form, the images, etc. Nothing fancy needs to be done here, because SSL does all the hard work for you. Just a simple HTML form that submits the username/password in "plaintext" is all that is really needed, since SSL will encrypt everything.
User creates new password: you generate a random salt (NOT based off the server time, but from good crypto random source). Hash the salt + the new password many times, and store the salt & resulting hash in your database.
Verify password: your script looks up salt for the user, and hashes the salt + entered password many times. Check for match in database.
The only thing that should be stored in your database is the salt and the hash/digest.
Assuming you have a database of MD5 hashes that you need to support, then the solution might be to add database columns for new SHA-2 hashes & salts. When the user logs in, you check against the MD5 hash as you have been doing. If it works, then follow the steps in "user creates new password" to convert it to SHA-2 & salt, and then delete the old MD5 hash. User won't know what happened.
Anything that really deviates from this is probably going to have some security flaws.
Overview
I'm looking to create a (REST) API for my application. The initial/primary purpose will be for consumption by mobile apps (iPhone, Android, Symbian, etc). I've been looking into different mechanisms for authentication and authorization for web-based APIs (by studying other implementations). I've got my head wrapped around most of the fundamental concepts but am still looking for guidance in a few areas. The last thing I want to do is reinvent the wheel, but I'm not finding any standard solutions that fits my criteria (however my criteria my be misguided so feel free to critique that as well). Additionally, I want the API to be the same for all platforms/applications consuming it.
oAuth
I'll go ahead and throw out my objection to oAuth since I know that will likely be the first solution offered. For mobile applications (or more specifically non-web applications), it just seems wrong to leave the application (to go to a web-browser) for the authentication. Additionally, there is no way (I am aware of) for the browser to return the callback to the application (especially cross-platform). I know a couple of apps that do that, but it just feels wrong and gives a break in the application UX.
Requirements
User enters username/password into application.
Every API call is identified by the calling application.
Overhead is kept to a minimum and the auth aspect is intuitive for developers.
The mechanism is secure for both the end user (their login credentials are not exposed) as well as the developer (their application credentials are not exposed).
If possible, not require https (by no means a hard requirement).
My Current Thoughts on Implementation
An external developer will request an API account. They will receive an apikey and apisecret. Every request will require at minimum three parameters.
apikey - given to developer at regisration
timestamp - doubles as a unique identifier for each message for a given apikey
hash - a hash of the timestamp + the apisecret
The apikey is required to identify the application issuing the request. The timestamp acts similarly to the oauth_nonce and avoids/mitigates replay attacks. The hash ensures that request was actually issued from the owner of the given apikey.
For authenticated requests (ones done on the behalf of a user), I'm still undecided between going with an access_token route or a username and password hash combo. Either way, at some point a username/password combo will be required. So when it does, a hash of several pieces of information (apikey, apisecret, timestamp) + the password would be used. I'd love feedback on this aspect. FYI, they would have to hash the password first, since I don't store the passwords in my system without hashing.
Conclusion
FYI, this isn't a request for how to build/structure the API in general only how to handle the authentication and authorization from solely within an application.
Random Thoughts/Bonus Questions
For APIs that only require an apikey as part of the request, how do you prevent someone other than the apikey owner from being able to see the apikey (since sent in the clear) and make excessive requests to push them over usage limits? Maybe I'm just over thinking this, but shouldn't there be something to authenticate that a request was verified to the apikey owner? In my case, that was the purpose of the apisecret, it is never shown/transmitted without being hashed.
Speaking of hashes, what about md5 vs hmac-sha1? Does it really matter when all of the values are hashed with with sufficiently long data (ie. apisecret)?
I had been previously considering adding a per user/row salt to my users password hash. If I were to do that, how could the application be able to create a matching hash without knowing the salt used?
The way I'm thinking about doing the login part of this in my projects is:
before login the user requests a login_token from the server. These are generated and stored on the server on request, and probably have a limited lifetime.
to login the application calculates the hash of the users password, then hashes the password with the login_token to get a value, they then return both the login_token and the combined hash.
The server checks the login_token is one that it has generated, removing it from its list of valid login_tokens. The server then combines its stored hash of the user's password with the login_token and ensures that it matches the submitted combined token. If it matches you have authenticated your user.
Advantages of this are that you never store the user's password on the server, the password is never passed in the clear, the password hash is only passed in the clear on account creation (though there may be ways around this), and it should be safe from replay attacks as the login_token is removed from the DB on use.
That's a whole lot of questions in one, I guess quite a few people didn't manage to read all the way to the end :)
My experience of web service authentication is that people usually overengineer it, and the problems are only the same as you would encounter on a web page. Possible very simple options would include https for the login step, return a token, require it to be included with future requests. You could also use http basic authentication, and just pass stuff in the header. For added security, rotate/expire the tokens frequently, check the requests are coming from the same IP block (this could get messy though as mobile users move between cells), combine with API key or similar. Alternatively, do the "request key" step of oauth (someone suggested this in a previous answer already and it's a good idea) before authenticating the user, and use that as a required key to generate the access token.
An alternative which I haven't used yet but I've heard a lot about as a device-friendly alternative to oAuth is xAuth. Have a look at it and if you use it then I'd be really interested to hear what your impressions are.
For hashing, sha1 is a bit better but don't get hung up about it - whatever the devices can easily (and quickly in a performance sense) implement is probably fine.
Hope that helps, good luck :)
So what you're after is some kind of server side authentication mechanism that will handle the authentication and authorisation aspects of a mobile application?
Assuming this is the case, then I would approach it as follows (but only 'cos I'm a Java developer so a C# guy would do it differently):
The RESTful authentication and authorisation service
This will work only over HTTPS to prevent eavesdropping.
It will be based on a combination of RESTEasy, Spring Security and CAS (for single sign on across multiple applications).
It will work with both browsers and web-enabled client applications
There will be a web-based account management interface to allow users to edit their details, and admins (for particular applications) to change authorisation levels
The client side security library/application
For each supported platform (e.g.
Symbian, Android, iOS etc) create a
suitable implementation of the
security library in the native
language of the platform (e.g. Java,
ObjectiveC, C etc)
The library
should manage the HTTPS request
formation using the available APIs
for the given platform (e.g. Java
uses URLConnection etc)
Consumers of the general authentication and
authorisation library ('cos that's
all it is) will code to a specific
interface and won't be happy if it
ever changes so make sure it's very
flexible. Follow existing design
choices such as Spring Security.
So now that the view from 30,000ft is complete how do you go about doing it? Well, it's not that hard to create an authentication and authorisation system based on the listed technologies on the server side with a browser client. In combination with HTTPS, the frameworks will provide a secure process based on a shared token (usually presented as a cookie) generated by the authentication process and used whenever the user wishes to do something. This token is presented by the client to the server whenever any request takes place.
In the case of the local mobile application, it seems that you're after a solution that does the following:
Client application has a defined Access Control List (ACL) controlling runtime access to method calls. For example, a given user can read a collection from a method, but their ACL only permits access to objects that have a Q in their name so some data in the collection is quiety pulled by the security interceptor. In Java this is straightforward, you just use the Spring Security annotations on the calling code and implement a suitable ACL response process. In other languages, you're on your own and will probably need to provide boilerplate security code that calls into your security library. If the language supports AOP (Aspect Oriented Programming) then use it to the fullest for this situation.
The security library caches the complete list of authorisations into it's private memory for the current application so that it doesn't have to remain connected. Depending on the length of the login session, this could be a one-off operation that never gets repeated.
Whatever you do, don't try to invent your own security protocol, or use security by obscurity. You'll never be able to write a better algorithm for this than those that are currently available and free. Also, people trust well known algorithms. So if you say that your security library provides authorisation and authentication for local mobile applications using a combination of SSL, HTTPS, SpringSecurity and AES encrypted tokens then you'll immediately have creditibility in the marketplace.
Hope this helps, and good luck with your venture. If you would like more info, let me know - I've written quite a few web applications based on Spring Security, ACLs and the like.
Twitter addressed the external application issue in oAuth by supporting a variant they call xAuth. Unfortunately there's already a plethora of other schemes with this name so it can be confusing to sort out.
The protocol is oAuth, except it skips the request token phase and simply immediately issues an access token pair upon receipt of a username and password. (Starting at step E here.) This initial request and response must be secured - it's sending the username and password in plaintext and receiving back the access token and secret token. Once the access token pair has been configured, whether the initial token exchange was via the oAuth model or the xAuth model is irrelevant to both the client and server for the rest of the session. This has the advantage that you can leverage existing oAuth infrastructure and have very nearly the same implementation for mobile/web/desktop applications. The main disadvantage is that the application is granted access to the client's user name and password, but it appears like your requirements mandate this approach.
In any case, I'd like to agree with your intuition and that of several other answerers here: don't try to build something new from scratch. Security protocols can be easy to start but are always hard to do well, and the more convoluted they become the less likely your third-party developers are to be able to implement against them. Your hypothetical protocol is very similar to o(x)Auth - api_key/api_secret, nonce, sha1 hashing - but instead of being able to use one of the many existing libraries your developers are going to need to roll their own.
Super late to the party but I wanted to throw in some additional points to consider for anyone interested in this issue. I work for a company doing mobile API security solutions (approov) so this whole area is definitely relevant to my interests.
To start with, the most important thing to consider when trying to secure a mobile API is how much it is worth to you. The right solution for a bank is different to the right solution for someone just doing things for fun.
In the proposed solution you mention that a minimum of three parameters will be required:
apikey - given to developer at registration
timestamp - doubles as a unique identifier for each message for a given apikey
hash - a hash of the timestamp + the apisecret
The implication of this is that for some API calls no username/password is required. This can be useful for applications where you don't want to force a login (browsing in online shops for example).
This is a slightly different problem to the one of user authentication and is more like authentication or attestation of the software. There is no user, but you still want to ensure that there is no malicious access to your API. So you use your API secret to sign the traffic and identify the code accessing the API as genuine. The potential problem with this solution is that you then have to give away the secret inside every version of the app. If someone can extract the secret they can use your API, impersonating your software but doing whatever they like.
To counter that threat there are a bunch of things you can do depending on how valuable the data is. Obfuscation is a simple way to make it harder to extract the secret. There are tools that will do that for you, more so for Android, but you still have to have code that generates your hash and a sufficiently skilled individual can always just call the function that does the hashing directly.
Another way to mitigate against excessive use of an API that doesn't require a login is to throttle the traffic and potentially identify and block suspect IP addresses. The amount of effort you want to go to will largely depend upon how valuble your data is.
Beyond that you can easily start getting into the domain of my day job. Anyway, it's another aspect of securing APIs that I think is important and wanted to flag up.