Can Google Cloud Build use an NPM token during the install process?
I'm using a private NPM package across applications and I know we can place the token within an .npmrc file
//registry.npmjs.org/:_authToken=<YOUR_AUTH_TOKEN>
But I would like to keep the token out of the codebase(s) and use SM for what it's for.
But this doesn't seem possible and the docs do not seem to cover this.
Meanwhile, posting the solution proposed by sethvargo and guillaume blaquiere for visibility.
You can set the token value as an environment variable, NPM_TOKEN. In that case, you can use the native Cloud Build + Secret Manager integration to inject the value: cloud.google.com/build/docs/securing-builds/use-secrets
If the NPM_TOKEN solution doesn't work, you can still use the Secret Manager and Cloud Build to load your token, write your .npmrc file with a script and then run your build.
I had to get experienced help for Secret Manager for this and it still took over 20 hours.
Assuming you possess general knowledge of GCP and Secret Manager and have your NPM token on-hand (found at npmjs.com/settings/USERNAME/tokens)
Create a Secret Manager secret named NPM_TOKEN with its value
being the private NPM package's automation token
Within the cloudbuild.yaml build process create an .npmrc
steps:
- name: bash
args: ['-c', 'echo //registry.npmjs.org/:_authToken=$$NPM_TOKEN > .npmrc'] <-- main takeaway
secretEnv: ['NPM_TOKEN']
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
args: ['gcloud', 'app', 'deploy']
availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/NPM_TOKEN/versions/latest
env: 'NPM_TOKEN'
Now the cloud build process has an NPM_TOKEN value to pull in the private NPM package.
Related
In my Dockerfile I want to clone a repo from github. To achieve that I applied the following command to my Dockerfile:
RUN git clone -b docker https://<pat-token>:x-oauth-basic#github.com/<mygithubaccount>/<repository.git> /myfolder
Of course, the phrases in square brackets are placeholders.
When I build the image from my Dockerfile I can read authentication error in the output:
remote: Support for password authentication was removed on August 13, 2021.
remote: Please see https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication.
I assume that is something around my authentication string. Maybe I cannot use a personal access token?
How do I correctly authenticate against github in my Dockerfile?
Fixed it - I've had a type in the token 🙈
I'm trying to set up dependabot-standalone to run in a GitLab-CI pipeline in a private instance.
It's an npm package and I'm using a private npm registry to fetch my dependencies from. According to the docs, I should set up my registry access data within the .gitlab/dependabot.yml file.
The docs for settings in dependabot.yml clearly say how to set up the credentials to access a given registry:
registries:
npm-npmjs:
type: npm-registry
url: https://registry.npmjs.org
username: octocat
password: ${{secrets.MY_NPM_PASSWORD}} # Must be an unencoded password
registries:
npm-github:
type: npm-registry
url: https://npm.pkg.github.com
token: ${{secrets.MY_GITHUB_PERSONAL_TOKEN}}
But only for its use in GitHub with project secrets. Is there a way to use GitLab's custom defined CI/CD variables instead?
So far I have unsuccessfully tried to use ${{CI_PRIVATE_NPM_ACCESS_TOKEN}} and ${CI_PRIVATE_NPM_ACCESS_TOKEN}
I think npm-login-noninteractive could be helpful in this case.
job-name:
script:
- npm install -g npm-login-noninteractive
- NPM_USER=testUser NPM_PASS=testPass NPM_EMAIL=test#example.com NPM_REGISTRY=your.npm.com npm-login-noninteractive
Other possible useful answers, here: How to set npm credentials using `npm login` without reading from stdin?
According to this example, you need to set the following environment variables within GitLab:
SETTINGS__GITHUB_ACCESS_TOKEN
SETTINGS__GITLAB_ACCESS_TOKEN
To set these variables at the project level, within GitLab go to: Project -> Settings -> CI/CD -> Variables.
I'd make my first npm publish into my Gitlab project. The project is private so I generated a deploy token with read & write permission for package registry.
Locally I created an .npmrc file and used these lines:
#laszlo-bottlik:registry=https://gitlab.com/api/v4/projects/28303714/packages/npm/
//gitlab.com/api/v4/packages/npm/:_authToken='<deploy-name>:<deploy-password>'
//gitlab.com/api/v4/projects/28303714/packages/npm/:_authToken='<deploy-name>:<deploy-password>'
Name value in package json is #laszlo-bottlik/playground
Url for the project is https://gitlab.com/laszlo-bottlik/playground
When I run npm publish command in terminal then I get 401 error:
HttpErrorGeneral: 401 Unauthorized - PUT https://gitlab.com/api/v4/projects/28303714/packages/npm/#laszlo-bottlik%2fplayground
Any kind of help can lead me to the solution.
What should I check in my local environment?
What should I check in my project settings under Gitlab?
Is there a reason you're using a deploy token as opposed to a job token? Either way, it looks like you may have an extra variable in your authToken argument above - You have <deploy-name>:<deploy-password> in your example code, but GitLab's code only shows a single auth_token argument (reference):
npm config set -- '//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "<your_token>"
I am running version 0.8.4 as a container in my lab. CLI is also at version 0.8.4
I am trying to use a secret in a command one of my containers is trying to run.
Following the documentation has me needing to sign a repo to allow the job to consume the secret. The drone CLI does not seem to have a
drone sign command for me to run. So I create the secret with a --skip-verify=true flag. This creates the secret but when I run the job it errors out. The output in the UI shows a blank space where the secret should be injected.
Here is an excerpt of my .drone.yml where I am trying to inject secrets -s production -u ${cf_user} -p ${cf_password} --s
I have tried all the following ways to create a secret:
drone secret add <repo_name> --name <key> --value <value> --skip-verify=true
drone secret add <repo_name> --name <key> --value <value>
GUI Creation
I notice when I create an all capital name value the UI represents the value in all lowercase when the CLI shows it in capitals.
I also notice that if I include hyphens in the name and try to use that in my drone.yml the job errors out immediately with a bad substitution error.
Any help understanding what I am doing wrong would be much appreciated!
I got lost in the different documentation available. Should have been looking here rather than secret-guide.
In case I am not alone, I needed to add a secrects block in my pipeline.
I also needed to access them with $SECRET_KEY rather than ${SECRET_KEY}
pipeline:
publish:
image: governmentpaas/cf-cli
secrets: [ cf_user, cf_password ]
Just a little update on this one, I stumbled over it as well because the docs are inconsistent.
In the 0.8.5 version the only thing I had to do is:
add secrets via CLI or UI
add secrets array to utilise it
no need to pass variables to environment.
Attempt to deploy via serverless framework using Windows 10 fails:
C:\Users\xxxxxx>sls deploy --verbose Serverless: Packaging
service... Serverless: Excluding development dependencies...
Error --------------------------------------------------
EPERM: operation not permitted, scandir
'C:\Users\xxxxxx\AppData\Local\ElevatedDiagnostics' For debugging
logs, run again after setting the "SLS_DEBUG=*" environment variable.
Your Environment Information ----------------------------- OS: win32
Node Version: 6.11.2 Serverless Version: 1.19.0
Tried again with command prompt under elevated privileges:
EBUSY: resource busy or locked, scandir
'C:\Users\xxxxxx\AppData\Local\Microsoft\InputPersonalization\TextHarvester\WaitList.dat'
I assumed there was a permissions issue at first so I retried with the command prompt at full admin mode but just ran into the the second error. My research suggested an issue with windows search so I turned it off (and also all background apps). Trying again (and again) I just ran into more similar issues and am unable to deploy anything. Anyone had similar issues and found a way around them?
I worked it out finally, so in case anyone else encounters this issue here is a summary. There seem to be 2 issues:
Don't create functions in your root folder. Create a specific folder for your serverless function i.e. not in C:\Users\nnnnnn> but within your regular document storage. In Windows 10 it works nicely if you use a OneDrive folder, with the benefit that your function(s) are also then replicated to other dev machines that you might use (and are automatically backed up offsite).
More importantly, the serverless framework seems to have an issue if you attempt to deploy to a region other than the default region set in your aws CLI configuration. I've no idea why this should be since the credentials I use with the AWS CLI are authorised for all regions. I also have no idea why the issue should result in serverless attempting to access a whole series of windows files for which it has no authority but nevertheless...
In my case, I primarily use region ap-southeast-2. By default, SLS CREATE generates a serverless.yml using a default US region. If this is left as-is, there is then a mismatch between the deployment region and your AWS CLI region. Not good. To avoid the minor pain of having to specify a deployment region in the SLS deploy command, just update the deployment region in the serverless.yml file to match the CLI region.
Now works a treat...