I wanna ask you how I gonna hide the secrete keys in API like database connection username, password or services API keys
When I deploy the application on virtual machine instance should I move them outside code like environment variables or I need to move them outside virtual machine to another place ?
Because the URL of the instance is publically accessable to reach API so people already has the URL of the instance .. is it safe to keep them in environment variables? Or need to move them to another place
Thanks
There are already a number of discussions regarding storing secrets in environment variables (good to avoid committing them to version control - bad for a number of reasons). I'm not going to comment further on this but in any case, Google Cloud's Secret Manager is made exactly for that. It is made to store API keys, passwords, certificates, and other sensitive data.
You can create your secrets via a number of ways (Cloud Console, Cloud SDK, etc...) then add a secret version that will contain the actual contents of a secret.
Once done, you can retrieve them from your code using the specific client library for your programming language. For example, in Python:
def access_secret_version(project_id, secret_id, version_id):
"""
Access the payload for the given secret version if one exists. The version
can be a version number as a string (e.g. "5") or an alias (e.g. "latest").
"""
# Import the Secret Manager client library.
from google.cloud import secretmanager
# Create the Secret Manager client.
client = secretmanager.SecretManagerServiceClient()
# Build the resource name of the secret version.
name = f"projects/{project_id}/secrets/{secret_id}/versions/{version_id}"
# Access the secret version.
response = client.access_secret_version(request={"name": name})
# Verify payload checksum.
crc32c = google_crc32c.Checksum()
crc32c.update(response.payload.data)
if response.payload.data_crc32c != int(crc32c.hexdigest(), 16):
print("Data corruption detected.")
return response
# Print the secret payload.
#
# WARNING: Do not print the secret in a production environment - this
# snippet is showing how to access the secret material.
payload = response.payload.data.decode("UTF-8")
print("Plaintext: {}".format(payload))
Related
I'm trying to implement a "Sign in with Google" button, as documented here:
https://developers.google.com/identity/gsi/web/guides/display-button
I'm confused by what it expects for data-login_uri as illustrated below (taken from the documentation linked above):
<div id="g_id_onload"
data-client_id="YOUR_GOOGLE_CLIENT_ID"
data-login_uri="https://your.domain/your_login_endpoint"
data-auto_prompt="false">
</div>
I have my app's client ID properly configured, and I can make it through most of the sign-in/auth flow provided by Google's pop-up window. However, once the pop-up window closes, it tries to POST to whatever URI I specify as the data-login_uri.
This leads me to believe that we need a back-end endpoint to do... something... but I can't track down any documentation regarding how this endpoint is supposed to behave, and as such I'm not sure what requirements to communicate to my back-end devs.
What did I miss?
TL;DR
You need a backend process (scripted in PHP, Python, Node, etc.) on your server which can relay a token_id (received from the div you quoted) to Google for validation.
Why?
Google's documentation says:
Warning: Do not accept plain user IDs, such as those you can get with the GoogleUser.getId() method, on your backend server. A modified client application can send arbitrary user IDs to your server to impersonate users, so you must instead use verifiable ID tokens to securely get the user IDs of signed-in users on the server side.
Details
The value of the data-auto_prompt parameter should point to an endpoint of an API or an executable CGI process in the back end.
Let's say your domain name is 'example.com'. There needs to be an endpoint, or executable cgi script at that endpoint that is capable of capturing a POST request, with application/x-www-form-urlencoded encoding. It might be something like this: https://www.example.com/login.
At this endpoint, a script / route should be capable of extracting the 'tokenid'
Google's documentation describes what the back end must do in two places:
Verify the Google ID token on your server side:
Verify the Cross-Site Request Forgery (CSRF) token and
Verify the integrity of the ID token
Here's a python code fragment for a 'login' route, using the Flask framework:
(A virtual environment is recommended and a pip install of two google api's are required.)
At the command line: pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
# Required imports from google API
from google.oauth2 import id_token
from google.auth.transport import requests
#bp.route('/login', methods=['POST'])
def login():
# Supplied by g_id_onload
tokenid = request.form['credential']
# Hardcoded client ID. Substitute yours.
clientid = XXXXX
# Display the encrypted credential
current_app.logger.debug(f"Token = {tokenid}")
try:
idinfo = id_token.verify_oauth2_token(tokenid,
requests.Request(), clientid)
# Display the verified user information
current_app.logger.debug(f"idinfo = {idinfo}")
# jsonify returns a response object
user_data = jsonify({
'username': idinfo['email'],
'name': idinfo['name'],
'id': idinfo['sub']
})
return user_data
except:
return Response(status=404)
The POST contents are described here data-login_uri with the expected actions for you to take on the back end described in Verify the Google ID token on your server side.
Generating a nonce per request and confirming the expected value on the back end are also strongly recommended to avoid replay attacks (and distinct from a CSP nonce to prevent cross-site scripting).
According to the Vue docs, I can store environment variables in .env files and they are loaded automatically. This works fine but this page says to not store any sensitive ENV variables in these files since they are bundled with the build.
WARNING
Do not store any secrets (such as private API keys) in your app!
Environment variables are embedded into the build, meaning anyone can
view them by inspecting your app's files.
https://cli.vuejs.org/guide/mode-and-env.html#environment-variables
So, if I am developing a webapp that talks to an API that requires an ID and a secret to be sent in order to get a token, where am I supposed to store these ENV variables (CLIENT_ID, CLIENT_SECRET)?
Locally I have an .env.local file that contains these values and I'm also defining them in the Netlify env variables.
Anything secret should be stored only on your server. It should never be part of your Vue app source code no matter how well you think you've hidden it, or encrypted it.
Generally speaking, you should have the user send their username/password, create a token on the server (look into JSON Web Tokens for a lightweight approach) and then either (1) Set it as an HttpOnly cookie (one that javascript cannot touch) or store it in localStorage on the client and set it as a common header in axios (using headers: { Authorization: 'Bearer ' } is fairly common.
Now, the client should be sending back that JWT with every request. So on your server, you can check the JWT is valid, and if so, send back the data requested.
Goal
Using googleapis with firebase functions. Get a JWT token so firebase functions can use a service account with domain-wide delegation to authorize G Suite APIs like directory and drive.
Question
What goes in path.join();
What is __dirname
What is 'jwt.keys.json'?
From this example:
https://github.com/googleapis/google-api-nodejs-client/blob/master/samples/jwt.js
// Create a new JWT client using the key file downloaded from the Google Developer Console
const auth = new google.auth.GoogleAuth({
keyFile: path.join(__dirname, 'jwt.keys.json'), // <---- WHAT GOES IN path.join()
scopes: 'https://www.googleapis.com/auth/drive.readonly',
});
Error
When I run
const auth = new google.auth.GoogleAuth({
keyFile: path.join(__dirname, "TEST"), // <-- __dirname == /srv/ at runtime
scopes: 'https://www.googleapis.com/auth/drive.readonly',
});
From the GCP Logs I get this error:
Error: ENOENT: no such file or directory, open '/srv/TEST'
Obviously TEST isn't valid, but is '/srv/?
What is the keyFile, a file path? a credential?
Another Example
https://github.com/googleapis/google-api-nodejs-client#service-to-service-authentication
I found documentation here:
https://googleapis.dev/nodejs/google-auth-library/5.10.1/classes/JWT.html
If you do not want to include a file, you can use key, keyId, and email to submit credentials when requesting authorization.
You seem to have a lot of questions around how this works. I would strongly encourage you to read the basics of Google authentication.
JWT is short for JSON Web Token. It is a standard standard defining secure way to transmit information between parties in JSON format. In your code "jwt" is a class containing a keys property. There are a ton of JWT libraries. There are some popularly packages using Node/Express frameworks.
__dirname // In Node this is the absolute path of the directory containing the currently executing file.
path.join is a method that joins different path segments into one path.
Here you are taking the absolute path and concatenating some piece of information to the end of the path. I am not certain what is contained in jwt.keys.json but that is what is being appended to the end of the absolute path in this case.
Without knowing your project structure or what you are pointing to it's not really possible to say what is and is not a valid path in your project.
keyFile is a key in an object (as denoted by the {key: value} format) under google.auth. As seen in the sample code you referenced, the script is taking the google.auth library and calling a method to construct and object with the information to are providing so that it abstract away other elements of the authentication process for you. You are giving it two pieces of information: 1) The location of the keyFile which presumably are the credentials and 2) The scope or set of permissions you are allowing. In the example it is readonly access to Drive.
EDIT: The private key file that the calling service uses to sign the JWT.
I created a repository on hub.docker.com and now want to push my image to the Dockerhub using my credentials. I am wondering whether I have to use my username and password or whether I can create some kind of access token to push the docker image.
What I want to do is using the docker-image resource from Concourse to push an image to Dockerhub. Therefore I have to configure credentials like:
type: docker-image
source:
email: {{docker-hub-email}}
username: {{docker-hub-username}}
password: {{docker-hub-password}}
repository: {{docker-hub-image-dummy-resource}}
and I don't want to use my Dockerhub password for that.
In short, you can't. There are some solutions that may appeal to you, but it may ease your mind first to know there's a structural reason for this:
Resources are configured via their source and params, which are defined at the pipeline level (in your yml file). Any authentication information has to be defined there, because there's no way to get information from an earlier step in your build into the get step (it has no inputs).
Since bearer tokens usually time out after "not that long" (i.e. hours or days) which is also true of DockerHub tokens, the concourse instance needs to be able to fetch a new token from the authentication service every time the build runs if necessary. This requires some form of persistent auth to be stored in the concourse server anyway, and currently Dockerhub does not support CI access tokens a la github.
All that is to say, you will need to provide a username and password to Concourse one way or another.
If you're worried about security, there are some steps you can most likely take to reduce risk:
you can use --load-vars-from to protect your credentials from being saved in your pipeline, storing them elsewhere (LastPass, local file, etc).
you might be able to create a user on Dockerhub that only has access to the particular repo(s) you want to push, a "CI bot user" if you will.
Docker Hub supports Access Token
goto Account Settings > Security
its same as Github personal access token (PAT)
You can use this token instead of actual password
I have a locally running pyspark cluster and want to load data from big query.
I have generated the server-to-server json credentials file
Now how do I pass that to my code.
TIA
When I write code to talk to Google's APIs from a tool, I generate Service Account Credentials within the project I want to use, download them, and then initialize the client with those credentials.
From python, I would use oauth2client.client.SignedJwtAssertionCredentials to create the credentials, and then pass those credentials using the credentials keyword parameter to the apiclient constructor. Something like:
key_bytes = ... # read your downloaded key from somewhere
bigquery_service = discovery.build(
'bigquery', 'v2',
credentials=oauth2client.client.SignedJwtAssertionCredentials(
'1234567890#developer.gserviceaccount.com',
key_bytes,
['https://www.googleapis.com/auth/bigquery'],
private_key_password='notasecret',
user_agent=_CLIENT_USER_AGENT))
Then whenever you use bigquery-service, your credentials will be passed along.