I'm building a very basic REST API for my site. The only verb I'm using at the moment is GET which simply outputs a list of posts on my site.
For authentication, I have been reading about HMAC and in particular this article:
http://websec.io/2013/02/14/API-Authentication-Public-Private-Hashes.html
My question centres around what the 'hashed content' should be. As I am not posting any data to the API, I have just been hashing my public key (with a simple salt) using my private key.
Is this a secure method or should I use a different 'content hash'? The data is not sensitive in any way - this was just a learning exercise.
You will want to consider the "replay attacker". When the attacker captures a packet between your API client and the server, what damage can she do when she replays it later?
In your case, if you only use the API key of the user in the HMAC, then the attacker will be able to impersonate that user when she replay the requests. She can call any API request and just set the HMAC to what she captured, as it will validate.
If none of the parameters of the request are included, the attacker will be able to call the request and specify her own parameters. So it's better if the parameters are also included in the HMAC. It doesn't prevent replay of the request with these specific parameters though.
You can include a timestamp parameter to the request and in the HMAC. The server will recompute the HMAC including the timestamp passed in, and it will also verify that the timestamp is recent enough. As the attacker cannot forge new HMAC out of thin air, she will only be able to use ones with matching timestamps that you will reject based on age.
Related
I am new and need directions so I can ask the correct questions. Here's the deal:
I have developed a REST API under HTTPS.
The user must provide a valid token to use the API.
The token expires after not being used for more than 5 minutes.
To obtain the token, the client must call the authentication API passing his private primary or secondary key, along with his user number.
Each key is unique, and on the database I save it's hash.
The user passes his primary or secondary key through the header with key "pk" or "sk" and "usernumber".
The server will get those keys and send to the database, which will apply the hash and check if they are valid.
Once the keys are valid, a token itself is generated on the database, and returned to the user.
My concern regards passing the primary key or secondary key on the headers. I am not sure if someone can obtain those data from outside and neither if it is the best practice. I am trying to get some directions, and I have came upon basic auth, oauth, and others. But they all seem to be on HTTP.
I have not found much about API HTTPS, so I also need some directions here. Can I make my API accept only https requests? If so, does the same security rules apply?
Thanks in advance.
There are 4 security aspects to consider. Most of the frameworks define the flow for Authentication and Authorization. Some frameworks define Integrity as well via Signatures.
But almost all heavily rely on encrypted data for for Confidentiality. i.e they recommend HTTPS if the communication is based on HTTP
Authentication:
Identifying who is talking to your API.
Authorisation:
Once you have identified who is talking to your API, ensuring they have the permission to talk to. If authentication is like checking someone's Id and allowing them into the building. Then authorisation is like allowing them to go into room for which they have access code.
Integrity:
One you know who are you talking to and what they are allowed to do, you still need to make sure that data you are receiving is from them and not tampered data.
Confidentiality
May be they are not tampering the data but reading all the data over the wire so later they can use that data and pretend to be the person you trust. So except for the sender and receiver no one else to see the data.
Note:
The above 4 aspects are for security on the flow.
You also have to consider security at rest. You seems to have strong design here on the server side for this aspect.
Have you considered what would you do when the token expires after 5 minutes. You user won't be happy entering user number and primary key/secondary key every 5 minutes. And if you plan to store it on client side so you can automate it every 5 minute, then you have to think about where would you store it in the client side and security at rest aspect for that
First of all: regarding HTTPS VS HTTP.
HTTPS is HTTP over TLS, where TLS is another layer of protection meant to secure the communication channel. All HTTP rules regarding headers apply to HTTPS too. TLS will protect your data confidentiality and integrity. It will protect the whole HTTP request including headers and body.
Regarding passing secrets as headers.
It's ok to pass secrets as headers or body. It's not ok, to pass secrets in URL. It's not ok to log out secrets on servers and proxies along the way. It's not ok to implement your own authentication mechanisms unless really needed.
If you want to read more about what is required to protect the communication channel (and the rest of the application), look into the OWASP Application Security Verification Standard.
My service allow any HTML documents to be converted to PDF using a POST request.
It is mostly used on the backend of my client's server and thus, the API key used for the communication is kept private.
Now, I'm thinking of a way to let my client's visitors be able to call my service on behalf of my client API key, without exposing this secure API Key.
My main issue here is security. If my client add an XHR POST requests that contains the API key, someone can take that API key and use it for their own purpose and abusing my client's account.
I could filter by domain, but this is easily spoofed so it's not possible.
I was wondering if there was a way to call a private service and be identified without risking its identity to be stolen, from the client ('s client) side?
If you're providing this sublet for authenticated users, then it's fairly trivial to give them unique keys (something that hashes their user ID or session against the API key and an initial timestamp, and checks it / logs it / looks for brutes before accessing the API). If you're doing it on the open web, without any kind of user authentication, then rate limiting gets very tricky indeed. Generally you'd want to use a combination of session hashes, IP address, operating system and browser data to create an anonymous profile that gets a temporary key on the frontend. One fairly solid way to do this is to force users through a CAPTCHA before serving a temporary key that allows them a limited number of uses of the permanent key. Any user whose ip/browser/session matches the existing attributes of a known client key is shunted to that one (and gets to skip the CAPTCHA); anyone who doesn't match an existing profile gets the CAPTCHA. That makes you a less attractive target for spoofing. On top of that, you should always rate-limit the entire thing, within a reasonable number of hits per day based on what kind of traffic you expect (or can afford), just so you don't have any surprises. This is the minimal security you'd want if your client's money is on the line every time their API key is used. It will require a simple database to store these "profiles", track usage, check for brutes and maintain the currently valid client keys. Client keys should always be expired regularly - either with a time diff against when they were created, or a regular cron process, or a maximum number of uses, etc.
One other thing I frequently do is rate-limit based on a curve. If I think 5 uses per minute is reasonable, for example, then after 5 uses in a minute from a session, each usage adds a delay of a fraction of a second * the number of uses in the last minute, squared, before the data is served.
The best answer would be to put this all behind a login system and secure that.
Assuming that you are using OAuth kind of system, In that case, make use of Access Token Mechanism that provides access to private API/User's data on behalf of User(Client) without exposing his/her credentials or API Key(Authentication key), also the access token can be expired based on the time/usage.
Example: The access token is generated against a single endpoint that can be the Html Conversion endpoint and will be expired once the action completion.
https://auth0.com/docs/tokens/access-token
And following blog post would be helpful to architect your authentication system
https://templth.wordpress.com/2015/01/05/implementing-authentication-with-tokens-for-restful-applications/
there is no good way to do front-end secure storage but my recommendation is :
is an API that used HMAC signing of requests in combination with OAuth authentication. The API key is actually a signing key. they key does not get transferred. The API key can still get found on the front-end but it becomes useless because you still need the OAuth token to send a valid request.
i know users will have to login in, but you can see this as an advantage because atleast you can log who is using the app by getting information from oauth.
please consider back-end secure storage!
You can use JWT tokens in my opinion. On the basis of username, password or any other info you can generate unique jwt tokens for different users.
Anyone can decipher these jwt tokens but not he unique security token.
If you want to add more more security to tokens, use JWE, encrypted web tokens.
More about these schemes can be found at https://medium.facilelogin.com/jwt-jws-and-jwe-for-not-so-dummies-b63310d201a3
Hashing is a decent option and should be done anyway, but for a fully secure method that wouldn't add too much complexity, you could simply abstract away from the authorization/API key by building your own API to interface with the API. This way you could both limit the kinds of things that can be done with the API key and also completely obscure the API key from the user
I don't think you should always go for user auth or JWT, it just doesn't fit all use cases. The idea of using a Captcha is interesting but also somewhat complex.
If complexity is not an issue I would rather use an infrastructure approach, I'm most familiar with AWS so I'll focus on that. Assuming you can change the host of your front end you can have your site hosted on an S3 bucket, served through a CDN, and create a proxy Lambda function that will hold the logic to call your API and store the API key as an encrypted environment variable. This Lambda you call through an API Gateway that can only be called by a specific IAM role which the S3 bucket also uses. You can also use a Cognito User Pool without authentication.
Going back to a simpler alternative the Captcha approach can be implemented as an attestation provider. I know of two services that do this, Firebase and KOR Connect. Due to Firebase using this approach only for their own resources as of the time of this writing I much rather use KOR Connect as it’s a very simple middleware that basically solves this issue. I won't go into detail about these services as it’s not the main concern of this topic but you can check the documentation their respective links.
I have been reading about securing REST APIs and have read about oAuth and JWTs. Both are really great approaches, but from what I have understood, they both work after a user is authenticated or in other words "logged in". That is based on user credentials oAuth and JWTs are generated and once the oAuth token or JWT is obtained the user can perform all actions it is authorized for.
But my question is, what about the login and sign up apis? How does one secure them? If somebody reads my javascript files to see my ajax calls, they can easily find out the end points and the parameters passed, and they could hit it multiple times through some REST Client, more severely they could code a program that hits my sign up api say a thousand times, which would be create a thousand spam users, or they could even brute force the login api. So how does one secures them?
I am writing my API in yii2.
The Yii 2.0 framework has a buil-in filter called yii\filters\RateLimiter that implements a rate limiting algorithm based on the leaky bucket algorithm. It will allow you to limit the maximum number of accepted requests in a certain interval of time. As example you may limit both login and signup endpoints to accept at most 100 API calls within a 10 minutes interval of time. When that limit is exceeded a yii\web\TooManyRequestsHttpException exception (429 status code) will be thrown.
You can read more about it in the Yii2 RESTful API related documentation or within this SO post.
I didn't use it myself so far but from what I did read about it in official docs, and I mean this:
Note that RateLimiter requires
$user
to implement the
yii\filters\RateLimitInterface.
RateLimiter will do nothing if
$user
is not set or does not implement
yii\filters\RateLimitInterface.
I guess it was designed to work with logged in users only maybe by using the user related database table, the default one introduced within the advanced template. I'm not sure about it but I know it needs to store the number of allowed requests and the related timestamp to some persistent storage within the saveAllowance method that you'll need to define in the user class. So I think you will have to track your guest users by IP addresses as #LajosArpad did suggest then maybe redesigning your user class to hold their identities so you can enable it.
A quick google search let me to this extension:yii2-ip-ratelimiter to which you may also have a look.
Your URLs will easily be determined. You should have a black list of IP addresses and when an IP address acts suspiciously, just add it to the black list. You define what suspicious is, but if you are not sure, you can start with the following:
Create something like a database table with this schema:
ip_addresses(ip, is_suspicious, login_attempts, register_attempts)
Where is_suspicious means it is blacklisted. login_attemtps and register_attempts should be json values, showing the history of that ip address trying to log in/register. If the last 20 attempts were unsuccessful and were within a minute, then the ip address should be blacklisted. Blacklisted ip addresses should receive a response that they are blacklisted whatever their request was. So if they deny your services or try to hack things, then you deny your services from them.
Secure passwords using sha1, for example. That algorithm is secure-enough and it is quicker than sha256, for instance, which might be an overkill. If your API involves bank accounts or something extremely important like that, important-enough for the bad guys to use server parks to hack it, then force the users to create very long passwords, including numbers, special characters, big and small letters.
For javascript you should use OAuth 2.0 Implicit Grant flow like Google or Facebook.
Login and Signup use 2 basic web page. Don't forget add captcha for them.
For some special client such as mobile app or webServer:
If you sure that your binary file is secure, You can create a custom login API for it. In this API you must try to verify your client.
A simple solution, you can refer:
use an encryption algorithm such as AES or 3DES to encrypt password
from client use a secret key (only client and server knows about it)
use a hash algorithm such as sha256 to hash (username + client time + an other
secret key). Client will send both client time and hash string to
server. Server will reject request if client time is too different
from server or hash string is not correct.
Eg:
api/login?user=user1&password=AES('password',$secret_key1)&time=1449570208&hash=sha256('user1'+'|'+'1449570208'+'|'+$secret_key2)
Note: In any case, server should use captcha to avoid brute force attack, Do not believe in any other filter
About captcha for REST APIs, we can create captcha base on token.
Eg.
For sign up action: you must call 2 api
/getSignupToken : to get image captcha url and a signup token
respectively.
/signup : to post sign up data (include signup token and
captcha typed by user)
For login action: we can require captcha by count failed logins base on username
Folow my api module here for reference. I manager user authentication by access token. When login, i generate token, then access again, client need send token and server will check.
Yii2 Starter Kit lite
I want my REST API server to be able to communicate only with my iOS app. The user base is going to be no more than 1000 people, and the market is pretty small and unpopular in general. That's why I think anything beyond a simple challenge-response authentication (HTTP, OAuth 2.0, SSL) would be an overkill. But I'm not sure exactly how this auth should flow. Here is what I have in mind:
Client app (user) sends a request: api.example.com/auth?username=john
Server responds with a randomly generated string: "somerandomlygeneratedstring"
Client takes the string, appends it to the username and then appends a secret string, hard coded in the app and uses MD5 to hash the entire string.
Client passes the string to the server: api.example.com/auth?username=john&response=thenewMD5hashstring
Server generates the same MD5 hash string and if they match, marks this user as authenticated in the database and all API requests from this user will be handled from now on.
Do I have the right idea? Or am I totally wrong? Please have in mind, I want basic security, anything too fancy would be an overkill for such a small project.
Also, I'm not keeping any sensitive data on my database like personal information.
You should simply use HTTP Basic auth for every request, through the Authorization header, and have all your interactions over SSL. If you want basic security, there's no need to go beyond that.
There are several problems with the scheme you have in mind.
Your last step is essentially a server-side session, which isn't acceptable in REST.
MD5 is effectively broken and shouldn't be used for anything but integrity checking.
In REST, you should use the standardized authentication method provided by the protocol if it fits your needs. Reinventing it to use URI parameters like you have in mind is unnecessary.
The hashing scheme you have in mind only makes sense when you want to sign the request, guaranteeing it wasn't tampered with.
I have been following a couple of articles regarding RESTful web services with WCF and more specifically, how to go about authentication in these. The main article I have been referencing is Aaron Skonnard's RESTful Web Services with WCF 3.5. Another one that specifically deals with HMAC authentication is Itai Goldstiens article which is based on Skonnards article.
I am confused about the "User Key" that is referenced to in both articles. I have a client application that is going to require a user to have both a user name and password.
Does this then mean that the key I use to initialise the
System.Security.Cryptography.HMACMD5 class is simply the users
password?
Given the method used to create the Mac in Itai's article
(shown below), am I right is thinking that key is the users
password and text is the string we are using confirm that the
details are in fact correct?
public static string EncodeText(byte[] key, string text, Encoding encoding)
{
HMACMD5 hmacMD5 = new HMACMD5(key);
byte[] textBytes = encoding.GetBytes(text);
byte[] encodedTextBytes =
hmacMD5.ComputeHash(textBytes);
string encodedText =
Convert.ToBase64String(encodedTextBytes);
return encodedText;
}
In my example, the text parameter would be a combination of request uri, a shared secret and timestamp (which will be available as a request header and used to prevent replay attacks).
Is this form of authentication decent? I've come across another thread here that suggests that the method defined in the articles above is "..a (sic) ugly hack." The author doesn't suggest why, but it is discouraging given that I've spent a few hours reading about this and getting it working. However, it's worth noting that the accepted answer on this question talks about a custom HMAC authorisation scheme so it is possible the ugly hack reference is simply the implementation of it rather than the use of HMAC algorithms themselves.
The diagram below if from the wikipedia article on Message Authentication Code. I feel like this should be a secure way to go, but I just want to make sure I understand it's use correctly and also make sure this isn't simply some dated mechanism that has been surpassed by something much better.
The key can be the user's password, but you absolutely should not do this.
First - the key has an optimal length equal to the size of the output hash, and a user's password will rarely be equal to that.
Second, there will never be enough randomness (entropy to use the technical term) in those bytes to be an adequate key.
Third, although you're preventing replay attacks, you're allowing anyone potentially to sign any kind of request, assuming they can also get hold of the shared secret (is that broadcast by the server at some point or is it derived only on the client and server? If broadcast, a man-in-the-middle attack can easily grab and store that - height of paranoia, yes, but I think you should think about it) unless the user changes their password.
Fourth - stop using HMACMD5 - use HMAC-SHA-256 as a minimum.
This key should at the very least be a series of bytes that are generated from the user's password - typically using something like PBKDF2 - however you should also include something transitory that is session-based and which, ideally, can't be known by an attacker.
That said, a lot of people might tell you that I'm being far too paranoid.
Personally I know I'm not an expert in authentication - it's a very delicate balancing act - so I rely on peer-reviewed and proven technologies. SSL (in this case authentication via client certificates), for example, might have it's weaknesses, but most people use it and if one of my systems gets exploited because of an SSL weakness, it's not going to be my fault. However if an exploit occurs because of some weakness that I wasn't clever enough to identify? I'd kick myself out of the front door.
Indidentally, for my rest services I now use SCRAM for authentication, using SHA512 and 512 bits of random salt for the stretching operation (many people will say that's excessive, but I won't have to change it for a while!), and then use a secure token (signed with an HMAC and encrypted with AES) derived from the authentication and other server-only-known information to persist an authenticated session. The token is stateless in the same way that Asp.Net forms authentication cookies are.
The password exchange works very well indeed, is secure even without SSL (in protecting the password) and has the added advantage of authenticating both client and server. The session persistence can be tuned based on the site and client - the token carries its own expiry and absolute expiry values within it, and these can be tuned easily. By encrypting client ID information into that token as well, it's possible to prevent duplication on to another machine by simply comparing the decrypted values from the client-supplied values. Only thing about that is watching out for IP address information, yes it can be spoofed but, primarily, you have to consider legitimate users on roaming networks.