GitlabCI - Package Registry - NPM Package can't publish - npm

My goal is to publish a npm package (private if possible) on the Gitlab Registry.
Here is my files :
Package.json :
{
"name": "#sushislasher/sushislasher-package",
"version": "0.0.1",
"description": "Package for game",
"main": "index.ts",
"repository": "https://gitlab.com/sushislasher/sushislasher-package",
"author": "Gildraen",
"license": "MIT",
"private": false
}
Gitlab-CI.yml :
stages:
- deploy
image: $CI_REGISTRY_IMAGE/docker-in-docker
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay2
COMPOSE_FILE: "docker-compose.yml:docker-compose.gitlab-ci.yml"
.template-load-gitlab-image: &internal-image
before_script:
- docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY
- docker-compose pull js || true
deploy:
<<: *internal-image
stage: deploy
script:
- |
{
echo "#${CI_PROJECT_ROOT_NAMESPACE}:registry=${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/npm/"
echo "${CI_API_V4_URL#https?}/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}"
} | tee -a app/.npmrc
- make install
- make npm-publish
make install build container, up them and do a yarn install.
make npm publish do basically a npm publish.
All in container.
I'm using node:15.7-alpine
But for now, despite all my tries and research, can't make this work, i have this error :
npm ERR! need auth This command requires you to be logged in.
npm ERR! need auth You need to authorize this machine using `npm adduser`
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2021-02-03T11_51_48_233Z-debug.log
1
I try to follow gitlab doc.. but i'm kind of stuck right now.. What am i doing wrong ?

I managed to do it. With a lot of try and try, here are the key points.
Package name. It is very important to keep an eye on it. What the documentation does not say it's that the "package.json" need the same name as the registry.
If your registry is #foo, your registry need #foo/something as project name. With the code I provide in the script, it takes the root project, and this is the registry.
For me it's https://gitlab.com/gildraen/sushislasher-package
so my package.json name is #gildraen/sushislasher-package (though the name seems not important)
So I did it all over again from scratch, with paying more attention to this, and it worked.
I really think this was my real mistake: the registry name.

Related

How to publish a scoped npm package to gitlab's package registry in a group namespace via a CI/CD pipeline

I have a private dummy project in gitlab which I want to publish to gitlab's package registry. My dummy project contains four files:
package.json
{
"name": "#<my-group>/<my-project>",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
"type": "git",
"url": "git+https://gitlab.com/<my-group>/<my-project>.git"
},
"keywords": [],
"author": "",
"license": "ISC",
"bugs": {
"url": "https://gitlab.com/<my-group>/<my-project>/issues"
},
"homepage": "https://gitlab.com/<my-group>/<my-project>#readme"
}
.gitlab-ci.yml
image: node:latest
stages:
- deploy
deploy:
stage: deploy
script:
- echo "#<my-group>:registry=https://${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/">.npmrc
- echo "//${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">>.npmrc
- npm publish
index.js
console.log('success');
README.md
<my-project>
When I commit my project to gitlab, the job fails with the following output
npm notice package: #<my-group>/<my-project>#1.0.0
npm notice === Tarball Contents ===
npm notice 8B README.md
npm notice 20B index.js
npm notice 612B package.json
npm notice === Tarball Details ===
npm notice name: #<my-group>/<my-project>
npm notice version: 1.0.0
npm notice filename: <my-group>-<my-project>-1.0.0.tgz
npm notice package size: 475 B
npm notice unpacked size: 640 B
npm notice shasum: 7b3db...
npm notice integrity: sha512-xDv0dl9A86...
npm notice total files: 3
npm notice
npm notice Publishing to https://gitlab.com/api/v4/projects/<my-project-id>/packages/npm/ with tag latest and default access
npm ERR! code E403
npm ERR! 403 403 Forbidden - PUT https://gitlab.com/api/v4/projects/<my-project-id>/packages/npm/#<my-group>%2f<my-project> - insufficient_scope
npm ERR! 403 In most cases, you or one of your dependencies are requesting
npm ERR! 403 a package version that is forbidden by your security policy, or
npm ERR! 403 on a server you do not have access to.
NOTE:
I have replaced the actual group name, project name and project id with <my-group>, <my-project>, and <my-project-id> in the code sections above.
I have followed gitlab's official documentation on setting this up (see https://docs.gitlab.com/ee/user/packages/npm_registry/) and believe that I can safely rule out the following:
I have made sure that Package registryis enabled in the project setup
I followed naming convention as described in the documentation
I am using a CI_JOB_TOKEN which should always be valid and should have appropriate permissions.
I made sure that there is no other package with the same name or version within the given scope.
I have made sure that the scoped package's URL includes a trailing slash (see gitlab-ci.yml above)
I have confirmed the path of the <my-group> namespace querying https://gitlab.com/api/v4/groups (just to make sure that the root namespace is correct)
I have used npm init --scope=#<my-group> --yes for initialization
The url of the repository is indeed: https://gitlab.com/<my-group>/<my-project>/
Any help on getting this to work would be much appreciated.
After changing from a job token ${CI_JOB_TOKEN} to a deploy token ${CI_DEPLOY_PASSWORD}, I was able to publish to the registry.
See Predefined variables reference for more info on predefined variables.
Gitlab deploy tokens can be created for projects under Settings > Repository > Deploy tokens with the following scopes: read_repository, read_package_registry, write_package_registry

Get npm package latest version from a gitlab registry

I'm trying to put a package in a Gitlab registry using npm and .gitlab-ci.yml.
.gitlab-ci.yml
npm-package:
stage: build
image: node:buster
before_script:
- git config --global user.name "${GITLAB_USER_NAME}"
- git config --global user.email "${GITLAB_USER_EMAIL}"
script:
- npm config set #${CI_PROJECT_ROOT_NAMESPACE}:registry=https://${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/
- npm config set //${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}
- NPM_PACKAGE_NAME=$(node -p "require('.package.json').name")
- NPM_PACKAGE_VERSION=$(node -p "require('./package.json').version")
- echo $(npm view "${NPM_PACKAGE_NAME}" versions)
- echo ${NPM_PACKAGE_NAME}
- echo ${NPM_PACKAGE_VERSION}
- |
if [[ $(npm view "${NPM_PACKAGE_NAME}" versions) != *"'${NPM_PACKAGE_VERSION}'"* ]]; then
npm config set //${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}
npm publish
echo "Successfully published version ${NPM_PACKAGE_VERSION} of ${NPM_PACKAGE_NAME} to GitLab's NPM registry: ${CI_PROJECT_URL}/-/packages"
else
echo "Version ${NPM_PACKAGE_VERSION} of ${NPM_PACKAGE_NAME} has already been published, so no new version has been published."
fi
I tried my first time and the package was saved successfully in the repo. And now I'm trying to run it under the condition: if the version's package has changed then run an npm publish.
But the variable $(npm view "${NPM_PACKAGE_NAME}" versions) seems to be empty, and when I try to echo it I get the error:
npm ERR! code E401
npm ERR! 401 Unauthorized - GET https://gitlab.example.com/api/v4/projects/1/packages/npm/#my_scope/my_package
Any help?
Your CI does not have enough rights.
1- You need to generate an access token (automation type) from your npm registry via the npm UI (or via the command line).
The process with the UI:
https://docs.npmjs.com/creating-and-viewing-access-tokens
2- Assign the token to an environment variable (named NPM_TOKEN in this example) accessible to your CI
3- Then create (or update) a .npmrc file with this line at the top:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}

NPM install is crashing with error "ERR! cb() never called!"

We are periodically seeing builds fail within our cloud build process with the following error:
docker.io/library/node:14 npm ERR! cb() never called!
npm ERR! This is an error with npm itself. Please report this error
at: npm ERR! https://npm.community
Steps from build configuration:
steps:
- name: 'gcr.io/cloud-builders/git'
id: 'fetch'
entrypoint: 'bash'
args:
- '-c'
- |
# convert the shallow clone to regular one
git fetch --unshallow --no-tags
waitFor: ['-']
- name: 'node:14'
id: 'npm-install'
entrypoint: 'bash'
args:
- '-c'
- |
npm ci --unsafe-perm
env:
- 'CYPRESS_INSTALL_BINARY=0'
- 'CYPRESS_CACHE_FOLDER=/cypress_cache'
waitFor: ['fetch']
Any thoughts on things to try or to remediate this issue?
EDIT: All packages are installed from npm.
As suggested by #Milan Tenk a good approach would be to implement a retry logic in your build by wrapping your command in a bash script.
Here are some examples on how you can modify you yaml file to a run bash scripts in Cloud Build:
steps:
- name: 'bash'
args: ['./retryPolicy.bash']
If bash is not the default entrypoint:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args: ['tools/retryPolicy.sh','--foo']
And here is an example of a retry logic implementation in bash:
#!/bin/bash
set -euo pipefail
function myFunction() {
myCommand
return $?
}
retry=0
maxRetries=5
retryInterval=15
until [ ${retry} -ge ${maxRetries} ]
do
myFunction && break
retry=$[${retry}+1]
echo "Retrying [${retry}/${maxRetries}] in ${retryInterval}(s) "
sleep ${retryInterval}
done
if [ ${retry} -ge ${maxRetries} ]; then
echo "Failed after ${maxRetries} attempts!"
exit 1
fi
Please let me know if that approach worked for you.
See NPM CLI WiKi:
https://github.com/npm/cli/wiki/%22cb()-never-called%3F-Exit-handler-never-called%3F-I'm-having-the-same-problem!%22
Sadly, there's never a straightforward answer.
However, many people seem to suggest deleting the node_modules folder in your project and also running the command npm cache clean -force.
For me it did not work - so I copied a node_modules folder from a PC, which could run npm install without having the error.
This at least allowed to use npm install without errors and build the project again.

NPM not installing devDependencies on bitbucket pipeline?

I'm trying to setup my first Bitbucket pipeline which simply builds my application and deploys it to my FTP server using the following bitbucket-pipelines.yml
image: node:6.9.4
pipelines:
default:
- step:
caches:
- node
script:
- npm install
- npm test
- step:
script:
- npm run build
- node deploy.js
The issue lies in the npm install because when bitbucket tries to run the npm run build command it says that rimraf (a npm package) is not found. rimraf however is listed in my devDependencies, all regular dependencies in my package.json are downloaded correctly.
There is no global variable set by my so the NODE_ENV could not be it right?
I had the same issue with gulp.
Gulp was in devDependencies and also specified in package.json as script but still it said npm ERR! missing script: gulp
The documentation says to install it globally, so there might be a related issue with your package.
https://confluence.atlassian.com/bitbucket/javascript-node-js-with-bitbucket-pipelines-873891287.html
I had this same issue. For me, the problem was that the version of Node on my local development device was different from the version of Node in the bitbucket-pipelines.yml file.
To fix it, I went into bitbucket-pipelines.yml and changed this line:
image: node:10.15.3
to this:
image: node:14.15.0

Gitlab CI - npm install command terminates current step

Using Gitlab (Community Version 8.13) CI I want to be able to install all jspm dependencies before publishing site.
I've package.json which defines jspm as dev dependency:
{
"jspm": {
"directories": {
"baseURL": ...
},
"dependencies": {
...
},
"devDependencies": {
...
}
},
"devDependencies": {
"jspm": "^0.16.48"
}
}
Now in my .gitlab-ci.yml file I've defined a step:
jspm:
stage: jspm
script:
- echo "npm install"
- 'npm install'
- echo "Trigger jspm install"
- ./node_modules/.bin/jspm install
The problem which I have is that after npm install next script command is not being triggered.
Here you could see log from this step. Looks like npm is able to install all packages but then even next echo command is not being triggered. (To simplify the log I've removed all installed packages listed by npm).
"npm install"
$ npm install
C:\Multi-Runner\builds\c144e1e9\0\{path to website}
`-- jspm#0.16.48
{lots of dependend packages listed here}
Build succeeded
Do you have any suggestions ? I could provide more details if needed.
My gitlab runner was installed on Windows machine.
According to: https://github.com/npm/npm/issues/2938
I've changed the step:
jspm:
stage: jspm
script:
- echo "npm install"
- call npm install
- echo "Trigger jspm install"
- call ./node_modules/.bin/jspm install
And it works fine now.