I am making a Social Networking website and I want to provide APIs for developers to use. So, I am thinking of using REST API for this.
Now this question is not about how to create a REST API. That is well explained in many websites and SO questions.
My Question is:
When I give a developer this API (lets say which can be used to get user info).
Say USER 1 does not want his date of birth and email to be visible
Say USER 2 does not want his mobile number to be visible
Say USER 3 does not want anyone apart from his friends to see his info
... and so on (each user have their own privacy setting which I have already stored in DB)
Now, how do I authenticate properly to make sure:
1) Developers will not be able to access User info when they have not authenticated the Application to use API to access their info
2) Restrict Developers from accessing fields which has been set by the user as private
Do I have to make separate DBs for this? Is there any easy, professional and secure way to do this?
If you look at the documentation for the various Google+ and FB API's, you'll see what they do. Here's an example:
https://developers.facebook.com/docs/marketing-api/authentication
Generally, you use https to connect, you supply a couple of different key codes (api key, etc.) to an authentication call of some sort, and you get back a session token, which you pass to all subsequent calls that require one (which potentially involve non-public data).
There isn't anything all that special about the authentication that is typically used for this kind of thing, any secure authentication method works as well as any other. All authentication does is establish the identity of the API caller and/or the identity of a user. It is then up to the API code to determine who can access what.
You could have a small group of trusted developers develop the API in a development environment, and then an even smaller group of trusted administrators deploy that API to production servers, and make sure no one else can access those servers except through the API. I believe that's essentially what FB and Twitter and so forth do.
But if you really want to prevent the developers who develop the API and the administrators who set up and maintain your production servers (and databases) from being able to get at user's private data, that data will have to be encrypted on the users' computers, before it is ever sent to the server, using keys that are never sent to the server. So neither the server nor anyone in the server environment will have any way at all to decrypt that data.
But the developers who develop the encryption program that runs on users' computers could make it send keys or decrypted versions of data to a computer somewhere, etc., so there isn't really any way to avoid having to trust at least some of the people involved.
By the way, Google+, Twitter, Facebook, etc., don't do that. They may store some private data in an encrypted form in their production databases, and they may restrict access to those databases to a small group of people, and even those people who have access to the databases may not be able to see the private information without decrypting it - but the data is sent to the servers using https; it is encrypted by a user's browser, but then decrypted on the web server, and it exists in an unencrypted form in the memory of the web server, at least for a short time, so it would certainly be theoretically possible for people at those companies who have the right expertise and the right access to get at users' private data. So our private data on Twitter / FB etc, isn't really 100% secure. (I doubt that any data anywhere is really 100% secure.)
While there are common practices that you can glean from reading their API docs, there aren't really any established standards or best practices that specifically relate to how to code authentication systems for social media APIs in PHP.
It's worth mentioning that many web sites that require user authentication, these days, allow users to authenticate using various external authentication systems, so that once they are signed into one place that uses that system, they're effectively signed into every place that uses it. One really popular external authentication system is GoogleID; for example, users of Stack Overflow can authenticate using their Google logins:
https://developers.google.com/identity/
https://developers.google.com/identity/choose-auth
GoogleID may have been inspired by OpenID. Here is a list of libraries for implementing an API that facilitates OpenID style authentication:
http://openid.net/developers/libraries/
That page includes a couple of php libraries which may be of interest.
By the way, speaking of data security in the context of a social media site, here's an introductory wiki page about the general legal requirements, in the US, for companies that process (potentially) personally identifying information:
https://termsfeed.com/blog/privacy-policies-are-mandatory-by-law/
Related
I wish to create a functionality that is very similar to facebook or pokerstars if you have used them before. Basically the apps require the user to login and their information can be accessed from both browsers and native and web apps.
How can I go about achieving this? Please advice on what services to research on to accomplish this. To my current understanding. I would be creating the website in html and php and creating a webservice using RESTful protocols and hosting them on amazon aws servers. I can then connect to these servers in the native apps? I am not very clear on how the native apps will interact with the servers
If you know of any particular protocol or a better server hosting service please let me know.
If I'm interpreting your question correctly, you are looking for something like this:
The user starts either your browser app or your native app (perhaps a mobile app)
Since the user does not have an account yet, you present them with the appropriate dialog to create said account.
You then ask the "Identity Service" to create a profile for that user
The identity service returns a token for access
This is something we do in the mobile network industry all the time. Technically, we have TAC/ACS or HSS profile services, but in either case, it's the same thing -- a dedicated service and network process that:
Accepts connections from various clients (web, mobile, desktop...)
Has various primitives along the database CRUD (Create, Read, Update, Delete) model
Answers requests the database
If you want a pre-configured solution, you could just use any networked database with a RESTstyle connector for example (MongoDB maybe?) But you could also just through this in a process that talks to a NoSQL or SQLLite database. The end result is the same.
For commercial solutions, I might like at OpenStack as you can run your code on it and they have identity brokers you might be able to CoOpt.
Personally, I'd just have a datastore running on a cloud somewhere like Amazon's EC2 which answers RESTful requests such as:
Create a user with a given profile set, return a unique token
Delete a user given a token
Update elements of the profile for a given token
I'm leaving out the necessary things like security here, but you get the idea.
This also has the advantage that you can have a single identity service for all of your applications/application services. The specifics for a given application element are just sub-fields in the profile. This gives you, not only a common identity broker for web, desktop and mobile, but a single-sign-on for all your applications. The user signs in once and is authenticated for everything you have. Moving from site to site, now just became seamless.
Lastly, you place your identity management, backup, security token management, etc OUTSIDE of your application. If you later want to add Google Authenticator for second-factor authentication, you don't have to add it to every application you have.
I should also add that you don't want to keep the identity database on the direct internet connection point. Someone could make your life difficult and get ahold it later on. Rather, you want your identity server to have a private link to it. Then do something like this:
When the account is created, don't store passwords, store hashes -- much safer
Have your application (web or otherwise) compute a key as the login
In this case, the user might enter a username and password, but the application or website would convert it into a token. THAT is what you send across.
Next, using that token (and suitable security magic), use THAT as the owner key
Send that key to the datastore and retrieve any needed values
Encrypt them back into a blob with the token
Send the block
THe application decrypts the blob to get at values
Why do we do this?
First, if someone were to try to get at your identity database, there's nothing useful. It contains only opaque tokens for logins, and blobs of encrypted data. Go ahead -- take the database. We don't care.
Second, sniffing the transport gets an attacker nothing -- again, it's all encrypted blobs.
This means later on, when you have five applications using the broker, and someone hacks the network and steals the database, you don't care, because your users never gave out logins and passwords in the first place, and even if they did, the data itself is garbage to anyone without the user key.
Does this help?
We have a large high traffic site with a lot of data on it (similar to Kickstarter), and we want to provide to our content/project creators a means of pulling their data from our site via an API.
Someone suggested I use OAuth, however my experience with OAuth is limited to setting up a twitter datasource.
What I want to do
Provide a user an Application ID and a 'secret'
Allow this user to connect to our application via an api endpoint, authorizing themselves using the api ID and secret
Once verified, allow this user to pull only their data from the application
The data that a user can pull: votes they have cast, pledges they have made, purchases they have made, projects/ideas they have launched, data about those projects/ideas (votes/purchases/orders/cancellations etc)
My question is:
Is OAuth overkill?
Is there a better way to handle a user/users website to connect to our API and pull/verify certain data by using the API we make available, while requiring each incoming request to be authorized for the user/site initiating that request.
Ideally, we will have an endpoint that is accessed as:
https://api.oursite.com/request/params
We want this to be as simple as possible for our users that wish to implement this interface. Thanks for your help!
Generally it's OAuth, in combination with SSL. That's the standard and is likely to stay. Before we saw also logins: username + password to access an API but that's becoming less and less.
So the suggested way is OAuth. There are no serious other solutions yet. To make it easier to adopt your API you could release some classes in some development languages so developers can have a quick start. You could start releasing those classes at for example GitHub to raise adoption of your API and get a quick access to developers. They might, if you do well, even start improving it.
Background: I am trying to create an SMS API service. The developers have a Dev ID, and an API secret key assigned to their developer account. The developers will be creating apps which will issue calls to my API. But the application issuing the call must be verified first.
Issue: The main issue i have is with authentication. I read up on OAuth and pretty much uderstood it. I read through this presentation (Slide 71-82). All OAuth articles talk about the OAuth 'dance' or the 'love triangle'. My problem seems to be, that i dont see a proper triangle in my case. Or, a better way to put it would be, the triangle doesn't seem to be complete.
What i mean by that is, in the case of lets say, LinkedIn, trying to make some app which helps users associate their LinkedIn acc with twitter, OAuth makes complete sense. Because LinkedIn needs to get resources from twitter ON THE USERS BEHALF (Cuz the user HAS A TWITTER ACCOUNT). In my case, only the consumer has a developer account registered with my service. The end-user doesn't have any credentials for the consumer to ask on behalf of. So how can i implement Oauth? So what will the consumer ask the provider? Will it only state that "watch out, here i come?". Cuz that seems pretty pointless unless its asking for a request token in exchange for an access token. But in this case since the end user doesnt even have an account, the steps seem useless.
So, i cant figure out how to go about this authentication issue. Ive tried thinking about using php sessions so it can help me associate a token with the particular client who is using the API. But the REST/OAUTH purists seem to disagree on the usage of sessions in authentication. They claim that OAuth is a standard which has proven itself and that is what I should use instead of coming up with my own obscure schemes.
From your description it seem that you're in a two party scenario only (developers write code which accesses your API on their own behalf, not on behalf of an end-user), so that means indeed that doing the full 3-legged oAuth scenario isn't needed.
You could use pretty much any authentication scheme and that would work (API Keys, other oAuth grant types [see below] or even ID/Secret combinations. In the oAuth world:
Look at the other oAuth 2.0 Grant types: especially resource owner PW grants - https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-26#section-4.3. It's slightly better than username-password because the PW isn't passed across the channel all the time (it passes once though) and it assumes the developer writing the code is the owner of the credentials.
Look at oAuth v1.0: this different in various ways to v2.0 but one feature it does have which some people like is the way tokens are used - which is rather than being passed across the wire they are used to generate a hash in the client and then the hash is verified on the server side. It's more expensive and complex than checking a key but it's less prone to attack.
In the non-oAuth world, if it's primarily a server resource used by developers directly, an ID/Secret or API-Key pattern is probably more than sufficient and it's much easier to implement for your developers.
Re: oAuth - if you're doing any type of user auth then definitely stick with the standard - the stuff is complex and having libraries out there really helps. If it's developer-api you likely don't need to go that far.
If you want the API to be secure in an ideal world anything which requires the security token to pass across the gaps should be secured using SSL, especially if that client code could be running on a mobile device or laptop which might communicate over wireless. If this isn't the case, someone could jump in an copy a token from one of the devs.
The only one of the protocols above that avoids this is the oAuth 1.0 variation since the secret never leaves the client but is used to hash instead. But it's complex. Last one to look at is the Amazon AWS pattern which does hashing similar to oAuth 1.0 http://docs.amazonwebservices.com/AmazonS3/latest/dev/RESTAuthentication.html and people emulate quite a bit.
Which are the steps must I follow to implement a token authentication in my web page?
Any summary or links will be appreciated.
I want to implement similar to Facebook or Google, first time client loggin and receive token and then use it in next actions.
I read also about OAuth but I don't want to give access to my application from 3rd party.
Thanks for the long response and it seems clear to me I need to read more about this.
What I want is to know the "steps" to implement a basic web application that uses token authentication. That is user logging once and then can make some actions: add content, edit, etc.
I know what I'm saying is similar to a session, where server adds a SESSION_ID on the HTML header and later request are identified and associated with that session. I read sessions way isn't good to scale so I want to implement a similar system like gmail or facebook before they go to OAuth. Probably I'm talking about something similar to oauth (i don't read in much depth) but witj two-legged instead three-legged.
You should think about your requirements, pick an appropriate protocol and some decent piece of software that implements it.
It's really hard to say more without more details:
are you talking about authentication for one or for multiple web applications? do you need single sign on between different web applications?
should all user data be stored on your server or should user be able to login e.g. with the google account?
should the token contain informations about the user?
on what platform are your applications developed?
what authentication method should be used?
do you want to realize a portal?
There is a really wide range of protocols and tools which might or might not fit to your requirements:
http://en.wikipedia.org/wiki/Category:Authentication_methods
http://en.wikipedia.org/wiki/Category:Identity_management_systems
I personally like CAS ( http://www.jasig.org/cas) for token-base SSO between multiple web applications. It's Java based but also has some support for PHP and .Net.
OpenID is fine, if you want to allow users to login with their Google, Yahoo, whatever account (configurable...) and don't want to store user information by yourself.
Kerberos/SPNEGO is the way to go if you want to haven integrated windows-sso for your corporate intranet applications.
For university applications SAML/Shibboleth probably is best. Outside universities it's somewhat less popular, probably cause it's a fairly complex protocol.
Oh and I almost forget: Most of the web frameworks/standards have there own version of plain-old "form based authentication". Where a user goes to a login form enters its username and password. Both are with or without SSL transported to the web/application server. The server validates it against some kind of database and gives a cookie to the user, which is transmitted and validated every time the user sends a request. But beside all this shiny protocols this seems to be pretty boring :-)
And before doing anything with web authentication, you might think for a moment about web security in general ( http://journal.paul.querna.org/articles/2010/04/11/internet-security-is-a-failure/ http://www.eff.org/files/DefconSSLiverse.pdf) and what you can do to not make it even worse on your site ( http://www.codinghorror.com/blog/2008/08/protecting-your-cookies-httponly.html http://owasptop10.googlecode.com/files/OWASP%20Top%2010%20-%202010.pdf).
see your point.
On the protocol level a very simplistic token approach is HTTP Basic Authentication. But this often doesn't fit, as there is no logout function etc.
A custom, simple cookie based approach can for example look like this:
The server generates some kind of secret (a value that is hard to guess)
When a user tries to access a protected resource, he is redirected to a login form
after successful authentication he gets a cookie. This cookie contains three values: username, timestamp and a hash of {username server-secret timestamp}.
with every user request the server recalculates the hash values and compares it to the value which the client sends in its cookie
(needs more consideration of: httponly and secure flag, transport layer security, replay attacks etc)
Amazon S3 stores its authentication token in an HTTP Header and uses HMAC for calculating it. It's described here: http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?S3_Authentication.html (Not necessarily recommended for using with a browser based web application)
If there is a book about REST anywhere near you, you may look if it has a chapter about authentication. Probably things are much nicer explained there than here :-)
There are some frameworks which are capable of doing this kind of authentication. For security reasons it would make sense to check them first before implementing your own stuff.
Overview
I'm looking to create a (REST) API for my application. The initial/primary purpose will be for consumption by mobile apps (iPhone, Android, Symbian, etc). I've been looking into different mechanisms for authentication and authorization for web-based APIs (by studying other implementations). I've got my head wrapped around most of the fundamental concepts but am still looking for guidance in a few areas. The last thing I want to do is reinvent the wheel, but I'm not finding any standard solutions that fits my criteria (however my criteria my be misguided so feel free to critique that as well). Additionally, I want the API to be the same for all platforms/applications consuming it.
oAuth
I'll go ahead and throw out my objection to oAuth since I know that will likely be the first solution offered. For mobile applications (or more specifically non-web applications), it just seems wrong to leave the application (to go to a web-browser) for the authentication. Additionally, there is no way (I am aware of) for the browser to return the callback to the application (especially cross-platform). I know a couple of apps that do that, but it just feels wrong and gives a break in the application UX.
Requirements
User enters username/password into application.
Every API call is identified by the calling application.
Overhead is kept to a minimum and the auth aspect is intuitive for developers.
The mechanism is secure for both the end user (their login credentials are not exposed) as well as the developer (their application credentials are not exposed).
If possible, not require https (by no means a hard requirement).
My Current Thoughts on Implementation
An external developer will request an API account. They will receive an apikey and apisecret. Every request will require at minimum three parameters.
apikey - given to developer at regisration
timestamp - doubles as a unique identifier for each message for a given apikey
hash - a hash of the timestamp + the apisecret
The apikey is required to identify the application issuing the request. The timestamp acts similarly to the oauth_nonce and avoids/mitigates replay attacks. The hash ensures that request was actually issued from the owner of the given apikey.
For authenticated requests (ones done on the behalf of a user), I'm still undecided between going with an access_token route or a username and password hash combo. Either way, at some point a username/password combo will be required. So when it does, a hash of several pieces of information (apikey, apisecret, timestamp) + the password would be used. I'd love feedback on this aspect. FYI, they would have to hash the password first, since I don't store the passwords in my system without hashing.
Conclusion
FYI, this isn't a request for how to build/structure the API in general only how to handle the authentication and authorization from solely within an application.
Random Thoughts/Bonus Questions
For APIs that only require an apikey as part of the request, how do you prevent someone other than the apikey owner from being able to see the apikey (since sent in the clear) and make excessive requests to push them over usage limits? Maybe I'm just over thinking this, but shouldn't there be something to authenticate that a request was verified to the apikey owner? In my case, that was the purpose of the apisecret, it is never shown/transmitted without being hashed.
Speaking of hashes, what about md5 vs hmac-sha1? Does it really matter when all of the values are hashed with with sufficiently long data (ie. apisecret)?
I had been previously considering adding a per user/row salt to my users password hash. If I were to do that, how could the application be able to create a matching hash without knowing the salt used?
The way I'm thinking about doing the login part of this in my projects is:
before login the user requests a login_token from the server. These are generated and stored on the server on request, and probably have a limited lifetime.
to login the application calculates the hash of the users password, then hashes the password with the login_token to get a value, they then return both the login_token and the combined hash.
The server checks the login_token is one that it has generated, removing it from its list of valid login_tokens. The server then combines its stored hash of the user's password with the login_token and ensures that it matches the submitted combined token. If it matches you have authenticated your user.
Advantages of this are that you never store the user's password on the server, the password is never passed in the clear, the password hash is only passed in the clear on account creation (though there may be ways around this), and it should be safe from replay attacks as the login_token is removed from the DB on use.
That's a whole lot of questions in one, I guess quite a few people didn't manage to read all the way to the end :)
My experience of web service authentication is that people usually overengineer it, and the problems are only the same as you would encounter on a web page. Possible very simple options would include https for the login step, return a token, require it to be included with future requests. You could also use http basic authentication, and just pass stuff in the header. For added security, rotate/expire the tokens frequently, check the requests are coming from the same IP block (this could get messy though as mobile users move between cells), combine with API key or similar. Alternatively, do the "request key" step of oauth (someone suggested this in a previous answer already and it's a good idea) before authenticating the user, and use that as a required key to generate the access token.
An alternative which I haven't used yet but I've heard a lot about as a device-friendly alternative to oAuth is xAuth. Have a look at it and if you use it then I'd be really interested to hear what your impressions are.
For hashing, sha1 is a bit better but don't get hung up about it - whatever the devices can easily (and quickly in a performance sense) implement is probably fine.
Hope that helps, good luck :)
So what you're after is some kind of server side authentication mechanism that will handle the authentication and authorisation aspects of a mobile application?
Assuming this is the case, then I would approach it as follows (but only 'cos I'm a Java developer so a C# guy would do it differently):
The RESTful authentication and authorisation service
This will work only over HTTPS to prevent eavesdropping.
It will be based on a combination of RESTEasy, Spring Security and CAS (for single sign on across multiple applications).
It will work with both browsers and web-enabled client applications
There will be a web-based account management interface to allow users to edit their details, and admins (for particular applications) to change authorisation levels
The client side security library/application
For each supported platform (e.g.
Symbian, Android, iOS etc) create a
suitable implementation of the
security library in the native
language of the platform (e.g. Java,
ObjectiveC, C etc)
The library
should manage the HTTPS request
formation using the available APIs
for the given platform (e.g. Java
uses URLConnection etc)
Consumers of the general authentication and
authorisation library ('cos that's
all it is) will code to a specific
interface and won't be happy if it
ever changes so make sure it's very
flexible. Follow existing design
choices such as Spring Security.
So now that the view from 30,000ft is complete how do you go about doing it? Well, it's not that hard to create an authentication and authorisation system based on the listed technologies on the server side with a browser client. In combination with HTTPS, the frameworks will provide a secure process based on a shared token (usually presented as a cookie) generated by the authentication process and used whenever the user wishes to do something. This token is presented by the client to the server whenever any request takes place.
In the case of the local mobile application, it seems that you're after a solution that does the following:
Client application has a defined Access Control List (ACL) controlling runtime access to method calls. For example, a given user can read a collection from a method, but their ACL only permits access to objects that have a Q in their name so some data in the collection is quiety pulled by the security interceptor. In Java this is straightforward, you just use the Spring Security annotations on the calling code and implement a suitable ACL response process. In other languages, you're on your own and will probably need to provide boilerplate security code that calls into your security library. If the language supports AOP (Aspect Oriented Programming) then use it to the fullest for this situation.
The security library caches the complete list of authorisations into it's private memory for the current application so that it doesn't have to remain connected. Depending on the length of the login session, this could be a one-off operation that never gets repeated.
Whatever you do, don't try to invent your own security protocol, or use security by obscurity. You'll never be able to write a better algorithm for this than those that are currently available and free. Also, people trust well known algorithms. So if you say that your security library provides authorisation and authentication for local mobile applications using a combination of SSL, HTTPS, SpringSecurity and AES encrypted tokens then you'll immediately have creditibility in the marketplace.
Hope this helps, and good luck with your venture. If you would like more info, let me know - I've written quite a few web applications based on Spring Security, ACLs and the like.
Twitter addressed the external application issue in oAuth by supporting a variant they call xAuth. Unfortunately there's already a plethora of other schemes with this name so it can be confusing to sort out.
The protocol is oAuth, except it skips the request token phase and simply immediately issues an access token pair upon receipt of a username and password. (Starting at step E here.) This initial request and response must be secured - it's sending the username and password in plaintext and receiving back the access token and secret token. Once the access token pair has been configured, whether the initial token exchange was via the oAuth model or the xAuth model is irrelevant to both the client and server for the rest of the session. This has the advantage that you can leverage existing oAuth infrastructure and have very nearly the same implementation for mobile/web/desktop applications. The main disadvantage is that the application is granted access to the client's user name and password, but it appears like your requirements mandate this approach.
In any case, I'd like to agree with your intuition and that of several other answerers here: don't try to build something new from scratch. Security protocols can be easy to start but are always hard to do well, and the more convoluted they become the less likely your third-party developers are to be able to implement against them. Your hypothetical protocol is very similar to o(x)Auth - api_key/api_secret, nonce, sha1 hashing - but instead of being able to use one of the many existing libraries your developers are going to need to roll their own.
Super late to the party but I wanted to throw in some additional points to consider for anyone interested in this issue. I work for a company doing mobile API security solutions (approov) so this whole area is definitely relevant to my interests.
To start with, the most important thing to consider when trying to secure a mobile API is how much it is worth to you. The right solution for a bank is different to the right solution for someone just doing things for fun.
In the proposed solution you mention that a minimum of three parameters will be required:
apikey - given to developer at registration
timestamp - doubles as a unique identifier for each message for a given apikey
hash - a hash of the timestamp + the apisecret
The implication of this is that for some API calls no username/password is required. This can be useful for applications where you don't want to force a login (browsing in online shops for example).
This is a slightly different problem to the one of user authentication and is more like authentication or attestation of the software. There is no user, but you still want to ensure that there is no malicious access to your API. So you use your API secret to sign the traffic and identify the code accessing the API as genuine. The potential problem with this solution is that you then have to give away the secret inside every version of the app. If someone can extract the secret they can use your API, impersonating your software but doing whatever they like.
To counter that threat there are a bunch of things you can do depending on how valuable the data is. Obfuscation is a simple way to make it harder to extract the secret. There are tools that will do that for you, more so for Android, but you still have to have code that generates your hash and a sufficiently skilled individual can always just call the function that does the hashing directly.
Another way to mitigate against excessive use of an API that doesn't require a login is to throttle the traffic and potentially identify and block suspect IP addresses. The amount of effort you want to go to will largely depend upon how valuble your data is.
Beyond that you can easily start getting into the domain of my day job. Anyway, it's another aspect of securing APIs that I think is important and wanted to flag up.