I have a pipeline deployed through gcp cloud based Kubeflow. Now I want to manage the pipeline from outside the google account i.e from different google account or from a local host. I want to manage the pipeline through Rest API calls. But I am getting error while trying. Do I need the details of google service account? if so, how to pass that information? Kindly help me solve these issue.
Once again I am stating the problem statement, need to access the pipeline through Rest API calls, which is in cloud based Kubeflow.
Related
Is there a way to call a Bigquery Rest API using service account? I need to push data from my powerapp to bigquery and I find the only way out is to create a custom connector. But due to org policies I am constrained to use a service account for authentication. I could not figure out a way to create a custom connector that authenticates through a service account. Following is my post on powerapps forum. Any help would be highly appreciated.
https://powerusers.microsoft.com/t5/Connector-Development/Big-query-connector-with-Service-Account-Authentication/m-p/1853380#M1739
Thanks,
Haripriya
Currently I am able to call BQ get api from a sample dataset with personal gmail credentials, but I get authentication error when I try with my org login. It is possibly because my org userId would not have permissions to query the API. I will have to use the service account that has the permissions. But I could not figure a way out to authenticate via a service account.
I have an API that deployed on Google Cloud Run. Then for security, I use Google Cloud Endpoints as the Gateway. As long as I know, Cloud Endpoints has a rate limiting feature by setting it in the OpenAPI config but it can be specified which user I want to assign the limit to. Is there any other solutions? Thank you.
Related to Security Cloud Run services for end-users and other services
I'm using:
Firebase Auth to generate id tokens for users with Google, Microsoft, GitHub ... identities
Cloud Endpoints on Cloud Run to invoke (Cloud Run) gRPC services
Firebase Auth users are auth'd by one of my services
Where I'm struggling....
My app provides 1 or more Cloud Run services that the app's users should be able to curl. But authenticating Cloud Run services require per-service id tokens; the id token's audience must use the Cloud Run service URL and the Cloud Run service URL is service-specific.
It seems as though I ought to be able to exchange the Firebase Auth id token for (Google Account) id tokens (with appropriate audiences) that can then be used to invoke the Cloud Run service. The proxy could also run on Cloud Run and it would use my app's auth service to verify whether the id token user should be issued with a Google id token.
Guillaume Blaquire's answer proposes either Coud Endpoints or a proxy similar to what I describe above. However, Cloud Endpoints requires that the backend services be known at deploy time (which these Cloud Run services won't be) and I want to provide the user with the id token so that they can use curl or some other tool to make the auth'd request.
Cloud Run has some compelling documentation for Authenticate (sic.) but I want something between:
Authenticating users -- I have the JWT but I want to receive a Google id token for the Cloud Run service
Authenticating service-to-service which Guillaume's alternative proposal in the answer.
Rather than place your Cloud Run behind Cloud Endpoints, where you have to know the Cloud Run instances ahead of time, you can handle the request and authentication inside the Cloud Run instance itself.
To be able to handle Firebase Authentication tokens inside the Cloud Run instance, they must be setup so that they can be invoked unauthenticated. Then, inside the Cloud Run, it should launch a web server, parse the incoming request (paying attention to the Authorization header - Firebase Auth sample) and then either action or terminate the request.
To achieve this, take a look at this thread for details on how you can handle both HTTP and service-service requests. Alternatively, you could just deploy the Functions Framework image from which that thread's code is based.
If you want cleaner URLs, host multiple endpoints within a single Cloud Run instance and then place that instance behind Cloud Endpoints or you can take a more manual approach via a custom domain using a service like Firebase Hosting.
I am working on a web application which comprises of ReactJs frontend and Java SpringBoot backend. This application would require users to upload data from their own Google Cloud storage into my Google Cloud Storage.
The application flow will be as follows -
The frontend requests the user for read access on their storage. For this I have used oauth 2.0 access tokens as described here
The generated Oauth token will be passed to the backend.
The backend will also have credentials for my service account to allow it to access my Google Cloud APIs. I have created the service account with required permissions and generated the key using the instructions from here
The backend will use the generated access token and my service account credentials to transfer the data.
In the final step, I want to create a transfer job using the google Storage-Transfer API. I am using the Java API client provided here for this.
I am having difficulty providing the authentication credentials to the transfer api.
In my understanding, there are two different authentications required - one for reading the user's bucket and another for starting the transfer job and writing the data in my cloud storage. I haven't found any relevant documentation or working examples for my use-case. In all the given samples, it is always assumed that the same service account credentials will have access to both the source and sink buckets.
tl;dr
Does the Google Storage Transfer API allow setting different source and target credentials for GCS to GCS transfers? If yes, how does one provide these credentials to the transfer job specification.
Any help is appreciated. Thanks!
This is not allowed for the the GCS Transfer API unfortunately, for this to work it would be required that the Service Account have access to both the source and the sink buckets, as you mentioned.
You can try opening a feature request in Google's Issue Tracker if you'd like so that Google's Product Team can consider such a functionality for newer versions of the API, also you could mention that this is subject is not touched in the documentation, so it can be improved.
We are building a small web-UI using React that will be served up by GCP App-Engine (standard). The UI will display a carousel of images along with some image metadata to our client's employees when they click on a link inside of their internal GIS system. We are looking to authenticate these calls since the App-Engine endpoint will be exposed publicly, and are hoping to use a GCP Service Account private key that will be used by the client to create a time-limited JSON web-token that will give temporary access to the GIS user when they open the web-UI. We are following this GCP documentation. In summary:
We create a new service-account with necessary IAM permissions in GCP along with a key
We share the private key with client which they then use to sign a Json Web Token which is passed in the call to our endpoint when user accesses our web-UI from their GIS system
Call is authenticated by GCP backend (ESP/OpenAPI)
Question: is this a recommended approach for external system accessing GCP resources or is there a better pattern more applicable to this type of situation (external system accessing GCP resource)?
I believe this is the recommended approach for your use case.
According to the official documentation: