How should I use Jfrog artifactory in Dockerfile for npm install - npm

We are using JFrog Artifactory for NPM Packages in our Jenkins CI Pipeline.. I would like to download npm packages from Artifactory while building docker images in our CI for npm ci/npm install step to decrease the duration of docker build process.
I tried as below via copy the .npmrc file that contains our private registry informations from local to docker container and when I checked the logs of npm install it able to download the dependencies from our JFrog Artifactory.. But this is not a secure approach cause I do not want to keep .npmrc in local repository and commit to the VCS.
What might be the best approach of doing this ?
Dockerfile
FROM node:12.21.0-alpine3.12 AS builder
WORKDIR /usr/src/app
ARG NPM_TOKEN
ARG NODE_ENVIRONMENT=development
ENV NODE_ENV=$NODE_ENVIRONMENT
COPY package.json /usr/src/app/package.json
COPY package-lock.json* .
COPY .npmrc /usr/src/app/.npmrc
RUN npm ci --loglevel verbose
RUN rm -f .npmrc
FROM node:12.21.0-alpine3.12
WORKDIR /usr/src/app
RUN apk update && apk add curl
COPY --from=builder /usr/src/app /usr/src/app
COPY . .
EXPOSE 50005 9183
CMD [ "npm", "run", "start:docker" ]
.npmrc
registry=https://artifacts.[company].com/artifactory/api/npm/team-npm-development-virtual
_auth = xxxxxxxxxx
always-auth = true
email = firstname.lastname#company.com

You can store your .npmrc on your VCS as long as it doesn't contain the "_auth" entry.
On azure devops we use service connections.
On "build containers" it wouldn't be possible, AFAIK.
So my approach would be using a protected build variable, to store the credential, and inject it in build time. Right before npm install you can set the "_auth" value on .npmrc.
You can achieve this in many different ways, but this is it.

Related

How to configure .npmrc file to be able to combine different NPM registries?

How to set an .npmrc file to tell npm install to download all the dependencies from https://registry.npmjs.org but only a specific one from a corporate Artifactory under https://artifactory.corporation.io/artifactory/npm/ that needs authentication using ?
We can configure https://registry.npmjs.org as a remote repository(test-npm-remote) in Artifactory. If you have any packages which were developed locally can be deployed to the local(test-npm-local) repository. We can create a virtual repository that consists of local and remote repositories and pull the npm artifacts using the virtual repository(test-npm). You can find more details in the below link.
https://jfrog.com/screencast/setting-up-an-npm-registry-with-jfrog-artifactory-in-less-than-one-minute/
npm config set registry http://art.local/artifactory/api/npm/test-npm/
npm login
<enter credentials>
Sample NPM client configuration for your reference.
$ cat ~/.npmrc
email=test#test.com
always-auth=true
registry=http://art.local/artifactory/api/npm/test-npm/
//art.local/artifactory/api/npm/test-npm/:username=testuser
//art.local/artifactory/api/npm/test-npm/:_password=XXXXXXX
To install a package: npm install <PACKAGE_NAME>

Get npm package latest version from a gitlab registry

I'm trying to put a package in a Gitlab registry using npm and .gitlab-ci.yml.
.gitlab-ci.yml
npm-package:
stage: build
image: node:buster
before_script:
- git config --global user.name "${GITLAB_USER_NAME}"
- git config --global user.email "${GITLAB_USER_EMAIL}"
script:
- npm config set #${CI_PROJECT_ROOT_NAMESPACE}:registry=https://${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/
- npm config set //${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}
- NPM_PACKAGE_NAME=$(node -p "require('.package.json').name")
- NPM_PACKAGE_VERSION=$(node -p "require('./package.json').version")
- echo $(npm view "${NPM_PACKAGE_NAME}" versions)
- echo ${NPM_PACKAGE_NAME}
- echo ${NPM_PACKAGE_VERSION}
- |
if [[ $(npm view "${NPM_PACKAGE_NAME}" versions) != *"'${NPM_PACKAGE_VERSION}'"* ]]; then
npm config set //${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}
npm publish
echo "Successfully published version ${NPM_PACKAGE_VERSION} of ${NPM_PACKAGE_NAME} to GitLab's NPM registry: ${CI_PROJECT_URL}/-/packages"
else
echo "Version ${NPM_PACKAGE_VERSION} of ${NPM_PACKAGE_NAME} has already been published, so no new version has been published."
fi
I tried my first time and the package was saved successfully in the repo. And now I'm trying to run it under the condition: if the version's package has changed then run an npm publish.
But the variable $(npm view "${NPM_PACKAGE_NAME}" versions) seems to be empty, and when I try to echo it I get the error:
npm ERR! code E401
npm ERR! 401 Unauthorized - GET https://gitlab.example.com/api/v4/projects/1/packages/npm/#my_scope/my_package
Any help?
Your CI does not have enough rights.
1- You need to generate an access token (automation type) from your npm registry via the npm UI (or via the command line).
The process with the UI:
https://docs.npmjs.com/creating-and-viewing-access-tokens
2- Assign the token to an environment variable (named NPM_TOKEN in this example) accessible to your CI
3- Then create (or update) a .npmrc file with this line at the top:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}

Can't remove file based variables

It looks like this error output from gitlab runner isn't well documented on the Internet so far so I'm starting a topic. Error output:
Step 11/13 : RUN chown -R node /usr/src/app/node_modules
---> Running in d9cb1a93d908
WARNING: Error while executing file based variables removal script error=context canceled job=1 project=0
ERROR: Failed to cleanup volumes
ERROR: Job failed: execution took longer than 30m0s seconds
FATAL: execution took longer than 30m0s seconds
this is coming from gitlab runner installed locally on Ubuntu machine.
Any suggestions as to how to troubleshoot this error ?
Dockerfile:
FROM node:16-alpine
WORKDIR /usr/src/app
ENV PATH /usr/src/app/node_modules/.bin:$PATH
COPY package.json /usr/src/app/package.json
COPY package-lock.json /usr/src/app/package-lock.json
RUN mkdir /usr/src/app/node_modules
RUN mkdir /usr/src/app/node_modules/.cache
RUN npm config set unsafe-perm true # to avoid permission problems when accessing .cache dir later on
RUN npm ci --loglevel=error
RUN npm i -g --silent --loglevel=error
RUN chown -R node /usr/src/app/node_modules
USER node
CMD ["npm", "start"]
I don't think this is related to gitlab, you are overwriting the owner of every dependency in your project, with nodejs those tend to grow exponentially.
You didn't post your Dockerfile, but this would be a matter of just rewriting in the following way:
FROM node:16-alpine
ENV PATH /home/node/app/node_modules/.bin:$PATH
# Create a directory for the project, change the owner and group
# to it, but do this before installing anything in the node_modules
# directory, to avoid timeouts.
RUN mkdir -p /home/node/app/node_modules \
&& mkdir -p /home/node/app/node_modules/.cache \
&& chown -R node:node /home/node/app
# Change to a workdir that can be owned by the 'node' user
WORKDIR /home/node/app
# Copying the lockfiles to make them cached by Docker,
# also sets the right permissions at the same time
COPY --chown=node:node package*.json ./
# to avoid permission problems when accessing .cache dir later on
RUN npm config set unsafe-perm true
RUN npm ci --loglevel=error \
&& npm i -g --silent --loglevel=error
# Now, change to the node user and install your deps
USER node
# Copy the remaining files and make sure to set the user/group
# right, to avoid permission issues
COPY --chown=node:node . .
RUN npm install
CMD ["npm", "start"]

Azure DevOps - commit to Bitbucket repository updated file on release pipeline?

I have an npm source code that needs to be built and push to npmjs repository. In details it looks like:
Build pipeline:
1) Get sources from Bitbucket repository
2) Get from package.json version number (ex. 0.0.3), increase it by 0.0.1 (0.0.4) and add this value to build variables $(version)
3) Make NPM install and build.
4) Take package-0.0.4.tgz and package.json to the artifacts folder and publish it.
Release pipeline:
1) Download artifacts
2) Extract package-0.0.4.tgz to npm-publish folder
3) Copy package.json to npm-publish folder
4) Publish npm folder to npmjs repository.
My question - is it possible to commit to Bitbucket repository updated package.json file with new version after publishing to npmjs repository?
It is possible to commit to Bitbucket repository. You only need to add a script task to execute the git commands.
For below example. I add a powershell task to run below commands in the pipeline to commit changes and push to the Bitbucket repo.
- powershell: |
git config --global user.email "you#example.com"
git config --global user.name "user.name"
#echo "some-text" > filename.txt
git add .
git commit -m "update package version"
git push https://username:password#bitbucket.org/name/repo.git HEAD:master -q
#if your password or username contain # replace it with %40
displayName: 'push to bitbucket'

Can't get drone.io CI to share files between pipelin steps

here is my .drone.yml:
pipeline:
test:
image: node:10
commands:
- npm install --production
- npm run build --production
publish:
image: plugins/docker
repo: myhub/image_name
when:
event: push
branch: master
the command npm run build creates a folder named build with static files inside. However, the publish step fails when building the docker image. Here is my Dockerfile:
FROM node:10-alpine
RUN mkdir -p /app
WORKDIR /app
COPY build build
The error being: COPY failed: stat /var/lib/docker/tmp/docker-builder090186817/build: no such file or directory time="2018-05-28T21:19:25Z" level=fatal msg="exit status 1"
So I don't quite understand how to build some files in one step, and copy them in the docker publish step...
Thanks for your help!
So anything in the workspace will get shared to the next step ;)
Are you able to build the docker image without drone with just docker build .?
ie. You might want to try to change COPY build build to be COPY ./build /app/build or the something like it.