I'm developing an application that manipulates data in Google Cloud Storage
buckets owned by the user. I would like to set it up so the user can arrange to
grant the application access to only one of his or her buckets, for the sake of
compartmentalization of damage if the app somehow runs amok (or it is
impersonated by a bad actor or whatever).
But I'm more than a bit confused by the documentation around GCS authorization.
The docs on OAuth 2.0 authentication show that there are only three
choices for scopes: read-only, read-write, and full-control. Does this
mean that what I want is impossible, and if I grant access to read/write one
bucket I'm granting access to read/write all of my buckets?
What is extra confusing to me is that I don't understand how this all plays in
with GCS's notion of projects. It seems like I have to create a project to get
a client ID for my app, and the N users also have to create N projects for
their buckets. But then it doesn't seem to matter -- the client ID from project
A can access the buckets from project B. What are project IDs actually for?
So my questions, in summary:
Can I have my installed app request an access token that is good for only a
single bucket?
If not, are there any other ways that developers and/or careful users
typically limit access?
If I can't do this, it means the access token has serious security
implications. But I don't want to have to ask the user to go generate a new one
every time they run the app. What is the typical story for caching the token?
What exactly are project IDs for? Are they relevant to authorization in any
way?
I apologize for the scatter-brained question; it reflects what appears to be
scatter-brained documentation to me. (Or at least documentation that isn't
geared toward the installed application use case.)
I had the same problem as you.
Go to : https://console.developers.google.com
Go to Credentials and create new Client ID
You have to delete the email* in "permissions" of your projet.
And add it manually in the ACL of your bucket.
*= the email of the Service Account. xxxxxxxxxxxx-xxxxxxxxx#developer.gserviceaccount.com
if you are building an app. It's Server to server OAuth.
https://developers.google.com/accounts/docs/OAuth2ServiceAccount
"Can you be clearer about which project I create the client ID on (the developer's project that owns the installed application, or the user's project that own's the bucket)?"
the user's project that own's the bucket
It's the user taht own the bucket who grant access.
It turns out I'm using the wrong OAuth flow if I want to do this. Thanks to Euca
for the inspiration to figure this out.
At the time I asked the question, I was assuming there were multiple projects
involved in the Google Developers Console:
One project for me, the developer, that contained generated credentials for
an "installed application", with the client ID and (supposed) secret baked into
my source code.
One project for each of my users, owning and being billed for a bucket that
they were using the application to access.
Instead of using "installed application" credentials, what I did was switch to
"service account" credentials, generated by the user in the project that owns
their bucket. That allows them to create and download a JSON key file that they
can feed to my application, which then uses the JSON Web Tokens flow of OAuth
2.0 (aka "two-legged OAuth") to obtain authorization. The benefits of this are:
There is no longer a need for me to have my own project, which was a weird
wart in the process.
By default, the service account credentials allow my application to access
only the buckets owned by the project for which they were generated. If the
user has other projects with other buckets, the app cannot access them.
But, the service account has an "email address" just like any other user, and
can be added to the ACLs for any bucket regardless of project, granting
access to that bucket.
About your answer.
Glad you solved your problem.
You can also reduce the access to only ONE bucket of the projet. For example, if you have several buckets and the application does not need access to all.
By default, the service account has FULL access Read, write and ACL of all buckets. I usually limited to the needed bucket.
Related
Suppose I have a simple node backend application which when ran needs to connect to a specific GSuite instance, query some things (users, groups, etc.) and then close and not run again until needed, which can mean either a very long time or a few seconds. From what I gathered from Google's documentation there may be multiple ways of doing this, including having an OAuth client and follow the whole flow in setting it up, managing token lifecycle, etc.
However I do NOT want to go with this option for now for various reasons and I am wondering if there is any way of getting access by means of an API Key / secret, like many other 3rd party services allow nowadays. Simply put I would like to generate a key pair somewhere on GSuite, no idea where, and use those keys for auth instead of OAuth, something Google suggests is possible, both on the GSuite Admin app (with a broken link that leads nowhere - not surprising) and on GCloud API and Credentials subpage where you setup credentials (however there it says that API Keys can only be used for very limited resources, none of them having anything to do with GSuite).
I think your best option is to see if what you want to do can be done by a service account. You can create a service account, grant administrator privileges to it in GSuite, enable some APIs, and then that account can do a lot of things without using OAuth directly. The credentials for the service account can then be provided to your application as a json key file, which it can use to authenticate to GSuite. You can also grant service accounts permissions to specific objects like files in Drive, but it doesn't sound like that would be sufficient to your needs.
A guide that may be helpful in the details of how to do this is https://m.fin.com/2017/10/04/navigating-the-google-suite-directory-api/
Is there a way I can fetch user credentials to authenticate with ADLS from within a spark job. I am trying to write a library that be used by users in their spark job to read data and want to hide implementation details. Also, what would be the best way to get credentials for a user from within spark job?
You can't fetch user credentials. You'll either need to save the user login session (tokens) somewhere securely, or...
If you want to have your solution authenticate without any interaction from an end user, then using a service principal + a certificate or key is the right way to go. This means your application will log in as the service principal, rather than as a user, and that service principal will need permissions on the resources and data upon which your solution should operate.
Azure Active Directory is what you'll authenticate with in either case, and their documentation for their authentication libraries is located here. They have a REST API, SDKs in many languages, and I'm sure there are third-party libraries are also available, if your language of choice isn't covered yet.
I hope this helps!
Are there step by step instructions anywhere on how to generate a "ticket" for an iCloud user given their username/password. I'd like to build a service that access iCloud data (server to server) without having to store the iCloud username or password.
My understanding is that you use the username/password to generate a Kerberos ticket from iCloud. That's based on the answer to How does Sunrise for iOS use iCloud credentials to access our calendar? . But I haven't found instructions online on how to do that.
Does anyone know how to do that? Thanks!
Let me start by pointing out that by default iCloud app storage is "sandboxed" in containers. A signed application can only access its own container without having the API key to authenticate to other application containers. You can make multiple applications share the same container, or use multiple containers in the same application if needed, but essentially you have to be the developer of all applications or have explicit permission to do this. Check out Incorporating iCloud into your app and Enabling CloudKit for more details.
Other (non-appstore) applications and services can authenticate to use an application's data via CloudKit Web Services:
Authenticating to iCloud (redirect based, so credentials still are never revealed and are known only by the user and iCloud server itself);
Further authenticating with your application API key;
The process is described in detail here, as already kindly pointed out by Adam Taylor.
All the above being said, If I understand correctly, you want to have access to all of the user's iCloud data. I think, you won't be able to do so for multiple reasons:
Data is protected by application key, so you need to have this to access a container in addition to the basic credentials;
I'm sure that Apple has a design policy to never ask for user credentials in plain text. Asking the user explicitly for credentials will be against their policy and even if it turns out it is not, having the credentials won't help you much, because you have to enter/send them somewhere. But all iCloud authentication mechanisms are designed to ask for authentication only by the end-user.
This is why I don't believe it is possible to just use user credentials and get access to all of their iCloud data. Now, my 2 cents on why Sunrise works:
As far as I understand, the Sunrise application works, because the calendar data is designed to be shared via CalDav, that works on a concrete URL, so you can import and link your calendar in various calendar client applications. The URL can be found out with a bit of investigation. CalDAV is kind of similar to IMAP and POP3 for mailbox access.
Be so kind to elaborate a bit more on what kind of data you're trying to extract (apple application specific, developer application specific, documents, key-value pairs or something else) and me or other users might help you further.
I'm working on integrating a PaaS product with the Google Cloud Storage API v2 and am very happy that OAuth2 is available. What makes me wonder though is having to specify the "x-goog-project-id" header on each request since that implies that my product's users provide that number additionally to the nice OAuth2 process. I understand that I could ask the user of my product to press the "Make this my default project for interoperable storage access" button to enable the v1 access, but that's rather user unfriendly and doesn't sounds very future-proof, too.
Instead I wondered whether there is a way to use the user's OAuth2 access token to retrieve a list of Cloud Storage projects the user has access to. I could then let the user decide which project to use. Does such an API exist? How likely is it that such an API will be provided in the future? Is there any other way to handle this particular problem?
There is no API to list a user's Cloud Storage projects. The user will have to give you their project ID.
When using Windows Identity Foundation (WIF) with multiple Security Token Services (STS), is it possible to provision users before they first access the application?
For example, let's say I have a web site called BufferOverrun where users can login and ask/answer questions and I want to support authentication with external Google accounts. When a user first accesses the page, they have to authenticate with their Google account, then they can access the web application. In this scenario, there are two STSes, Google (for identity authentication) and a custom one for my application (for authorization).
How can I assign claims to a user before that users accesses the system?
Since the identity is owned externally to my application, I cannot assign claims directly to that identity (and I wouldn't want to anyway, as they would be application specific). But since the user has not accessed the system, I do not have an internal identity to assign claims to. I see two possible solutions:
Wait for a user to access the system (creating some default application-specific claims), then use some internal provisioning tool to modify those claims as desired.
Have the provisioning tool allow users to manually map a default identity claim (email address, for example) before that identity authenticates by manually typing it in, so that on first access if the identity asserts that claim, a specific set of application claims are granted.
I see a few issues with both 1 and 2. For 1, all users have some implicit access to the system, even if the default application claims allow no functionality. This seems to work great for something like stackoverflow where the initial account has a certain permission set, and as the user uses the systems, new claims are granted. However, this is likely not desirable for all applications. 2 is error prone, as it requires an admin to manually specify a claim.
In both cases above, how do I provision the identity which has access to actually use the provisioning tool (i.e., an admin account)?
For this, I envision that during application installation time, I require a user to authenticate and set the applicaton claims for that identity to be such that they have "administrative" privileges. Is this a good implementation?
Historically (I am now referring to an existing application), the application specifically interfaced with Active Directory only. The way this was handled was that there was a built-in admin account (not affiliated with AD) that allowed the admin user to first login. After authenticating with the admin application, that user could search AD for users/groups and provision them individually. Any user/group not provisioned by the admin would not have access to the system at all. I don't see this paradigm being applicable to using an external STS like Google, etc, so I am trying to conceive an architecture that would enable external STS systems. Retaining the ability to search the STS is desired, though not required. In practice, the two STSes involved would likely both be Active Directory using federated services.
Note: This is similar to this question and this question.
When using Windows Identity Foundation (WIF) with multiple Security Token Services (STS), is it possible to provision users before they first access the application?
The answer is yes, if you have a way of identitfying those users (e.g. their e-mail)
In this scenario, there are two STSes, Google (for identity authentication) and a custom one for my application (for authorization).
This is frequently used, but not necessarilly always the case. If you rely just on Google, then you could simply have the authorization code in the app itself (e.g. "AuthorizationManager" classes, etc). The value of another STS is that it can be a broker for multiple identities (e.g. Google, LiveID, Yahoo!, whatever) and you can do some authorization related transformations.
Since the identity is owned externally to my application, I cannot assign claims directly to that identity (and I wouldn't want to anyway, as they would be application specific).
Why not? You can define a rule that says:
"Anyone authenticated with Google is a 'reader' in App BufferOverrun". You can even say:
"someone#gmail.com is a 'reader' on BufferOverrun", before someone accesses the app.
You can still use the original approach (an out of band admin account for setup). Or you can also "bootstrap" config during provisioning defining which is the claim that will identify admin users.
Take a look at sample "Federation with Multiple Partners and ACS" (sample 7) in http://claimsid.codeplex.com
We do exactly that.