I'm trying to set up dependabot-standalone to run in a GitLab-CI pipeline in a private instance.
It's an npm package and I'm using a private npm registry to fetch my dependencies from. According to the docs, I should set up my registry access data within the .gitlab/dependabot.yml file.
The docs for settings in dependabot.yml clearly say how to set up the credentials to access a given registry:
registries:
npm-npmjs:
type: npm-registry
url: https://registry.npmjs.org
username: octocat
password: ${{secrets.MY_NPM_PASSWORD}} # Must be an unencoded password
registries:
npm-github:
type: npm-registry
url: https://npm.pkg.github.com
token: ${{secrets.MY_GITHUB_PERSONAL_TOKEN}}
But only for its use in GitHub with project secrets. Is there a way to use GitLab's custom defined CI/CD variables instead?
So far I have unsuccessfully tried to use ${{CI_PRIVATE_NPM_ACCESS_TOKEN}} and ${CI_PRIVATE_NPM_ACCESS_TOKEN}
I think npm-login-noninteractive could be helpful in this case.
job-name:
script:
- npm install -g npm-login-noninteractive
- NPM_USER=testUser NPM_PASS=testPass NPM_EMAIL=test#example.com NPM_REGISTRY=your.npm.com npm-login-noninteractive
Other possible useful answers, here: How to set npm credentials using `npm login` without reading from stdin?
According to this example, you need to set the following environment variables within GitLab:
SETTINGS__GITHUB_ACCESS_TOKEN
SETTINGS__GITLAB_ACCESS_TOKEN
To set these variables at the project level, within GitLab go to: Project -> Settings -> CI/CD -> Variables.
Related
Can Google Cloud Build use an NPM token during the install process?
I'm using a private NPM package across applications and I know we can place the token within an .npmrc file
//registry.npmjs.org/:_authToken=<YOUR_AUTH_TOKEN>
But I would like to keep the token out of the codebase(s) and use SM for what it's for.
But this doesn't seem possible and the docs do not seem to cover this.
Meanwhile, posting the solution proposed by sethvargo and guillaume blaquiere for visibility.
You can set the token value as an environment variable, NPM_TOKEN. In that case, you can use the native Cloud Build + Secret Manager integration to inject the value: cloud.google.com/build/docs/securing-builds/use-secrets
If the NPM_TOKEN solution doesn't work, you can still use the Secret Manager and Cloud Build to load your token, write your .npmrc file with a script and then run your build.
I had to get experienced help for Secret Manager for this and it still took over 20 hours.
Assuming you possess general knowledge of GCP and Secret Manager and have your NPM token on-hand (found at npmjs.com/settings/USERNAME/tokens)
Create a Secret Manager secret named NPM_TOKEN with its value
being the private NPM package's automation token
Within the cloudbuild.yaml build process create an .npmrc
steps:
- name: bash
args: ['-c', 'echo //registry.npmjs.org/:_authToken=$$NPM_TOKEN > .npmrc'] <-- main takeaway
secretEnv: ['NPM_TOKEN']
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
args: ['gcloud', 'app', 'deploy']
availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/NPM_TOKEN/versions/latest
env: 'NPM_TOKEN'
Now the cloud build process has an NPM_TOKEN value to pull in the private NPM package.
I have tried to find out in what order the statements of the serverless file are evaluated (maybe it is more common to say that 'variables are resolved').
I haven't been able to find any information about this and to some extent it makes working with serverless feel like a guessing game for me.
As an example, the latest surprise I got was when I tried to run:
$ sls deploy
serverless.yaml
useDotenv: true
provider:
stage: ${env:stage}
region: ${env:region}
.env
region=us-west-1
stage=dev
I got an error message stating that env is not available at the time when stage is resolved. This was surprising to me since I have been able to use env to resolve other variables in the provider section, and there is nothing in the syntax to indicate that stage is resolved earlier.
In what order is the serverless file evaluated?
In effect you've created a circular dependency. Stage is special because it is needed to identify which .env file to load. ${env:stage} is being resolved from ${stage}.env, but Serverless needs to know what ${stage} is in order to find ${stage}.env etc.
This is why it's evaluated first.
Stage (and region, actually) are both optional CLI parameters. In your serverless.yml file what you're setting is a default, with the CLI parameter overriding it where different.
Example:
provider:
stage: staging
region: ca-central-1
Running serverless deploy --stage prod --region us-west-2 will result in prod and us-west-2 being used for stage and region (respectively) for that deployment.
I'd suggest removing any variable interpolation for stage and instead setting a default, and overriding via CLI when needed.
Then dotenv will know which environment file to use, and complete the rest of the template.
I'd make my first npm publish into my Gitlab project. The project is private so I generated a deploy token with read & write permission for package registry.
Locally I created an .npmrc file and used these lines:
#laszlo-bottlik:registry=https://gitlab.com/api/v4/projects/28303714/packages/npm/
//gitlab.com/api/v4/packages/npm/:_authToken='<deploy-name>:<deploy-password>'
//gitlab.com/api/v4/projects/28303714/packages/npm/:_authToken='<deploy-name>:<deploy-password>'
Name value in package json is #laszlo-bottlik/playground
Url for the project is https://gitlab.com/laszlo-bottlik/playground
When I run npm publish command in terminal then I get 401 error:
HttpErrorGeneral: 401 Unauthorized - PUT https://gitlab.com/api/v4/projects/28303714/packages/npm/#laszlo-bottlik%2fplayground
Any kind of help can lead me to the solution.
What should I check in my local environment?
What should I check in my project settings under Gitlab?
Is there a reason you're using a deploy token as opposed to a job token? Either way, it looks like you may have an extra variable in your authToken argument above - You have <deploy-name>:<deploy-password> in your example code, but GitLab's code only shows a single auth_token argument (reference):
npm config set -- '//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "<your_token>"
I am trying to reverse engineer the drone.io docker plugin and understand how to run the docker daemon in a pipeline step (DinD).
drone.io uses the library github.com/cncd/pipeline to compile and execute .drone.yml files.
The first thing the plugins/docker does is to start the docker daemon:
+ /usr/local/bin/dockerd -g /var/lib/docker
This works if fine in the official plugin, but I cannot get it to work with my own image where I do the same:
pipeline.yml
workspace:
base: /go
path: src/github.com/fnbk/hello
pipeline:
test:
image: fnbk/drone-daemon
fnbk/drone-daemon/run.sh
#!/bin/sh
/usr/local/bin/dockerd # <= ERROR: containerd: write /proc/17/oom_score_adj: permission denied
# ...
It will give me the following error:
containerd: write /proc/14/oom_score_adj: permission denied
The full example can be found on github: https://github.com/cncd/pipeline/pull/45
Any suggestions are highly appreciated.
You need to add your plugin to a whitelist via the DRONE_ESCALATE environment variable, which is passed to the server. This is the default value:
DRONE_ESCALATE=plugins/docker,plugins/gcr,plugins/ecr
So you would pass something like this:
-DRONE_ESCALATE=plugins/docker,plugins/gcr,plugins/ecr
+DRONE_ESCALATE=plugins/docker,plugins/gcr,plugins/ecr,fnbk/my-custom-plugin
Note that this should be the image name only. It must not include the tag.
I am working to set up a gitlab runner for multiple projects, and we want to be able to set up environment variables for all of the projects. I tried to set global variables in the .bashrc for both the gitlab-runner and root users but it did not recognize them during the CI script. What is the correct location to declare global environment variables?
You can also inject environment variables to your gitlab-runner directly in the commandline, as the gitlab-runner exec docker --help states:
OPTIONS: ..
--env value Custom environment variables injected to build environment [$RUNNER_ENV] ..
Here is a small example how I use it in a script:
Change the declarations as needed:
declare jobname="your_jobname"
declare runnerdir="/path/to/your/repository"
Get the env file into a bash array.
[ -f "$runnerdir/env" ] \
&& declare -a envlines=($(cat "$runnerdir/env"))
declare -a envs=()
for env in "${envlines[#]}"; do
envs+=(--env "$env")
done
And finally pass it to the gitlab-runner.
[ -d "$runnerdir" ] && cd "$runnerdir" \
&& gitlab-runner exec docker "${envs[#]}" $jobname \
&& cd -
You can define environment variables to inject in the runner's config.toml file. See the advanced runner configuration documentation in the [[runners]] section.
There doesn't seem to be a way to specify environment variables in the GitLab UI just for a specific runner.
With GitLab 13.1 (June 2020), you now have:
Instance-level CI/CD variables
GitLab now supports instance-level variables.
With this ability to set global variables, you no longer need to manually enter the same credentials repeatedly for all your projects.
This MVC introduces access to this feature by API, and the next iteration of this feature will provide the ability to configure instance-level variables directly in the UI.
See Documentation and issue.
Consider using an external persistent secret storage service like Vault or Keywhiz
Disclaimer: I am not associated nor used any of the above services
I have added export MY_VAR="FOO" to gitlab-runner's .bashrc, and it works.
echo export MY_VAR=\"FOO\" >> /home/gitlab-runner/.bashrc
Check which type of executor do you use? (shell, kubernetes, docker-ssh, parallels...) I use shell executor.
Check what type of shell does gitlab-runner use? (How to determine the current shell I'm working on) And edit the proper rc file for that.
Check the Gitlab CI Runner user.
I suggest dump all environment variables for further debugging, by add env to the .gitlab-ci.yml's script:
#.gitlab-ci.yml
job:
script: env
Making changes in ~/.bash_profile NOT ~/.bashrc.
See my answer
You can easily setup Variables in the GitLab Settings:
Project-level variables can be added by going to your project's Settings > CI/CD, then finding the section called Variables.
To make sure your variables are only used in
See:
https://docs.gitlab.com/ee/ci/variables/#variables