npm version patch: clean working directory not clean - npm

I hae a problem running npm version patch on Azure Pipelines. Initially, I discovered that SonarCloud scanner drops unwanted files in the repository. I git-ignored the directory
Consider the following fragment
- script: git checkout -f $(Build.SourceBranchName)
displayName: Force checkout of current branch
- script: git status
displayName: Git status
- script: git reset --hard
displayName: Git reset
- task: Npm#1
displayName: Tag repository and set release version
inputs:
command: custom
customCommand: version patch
- script: git push origin $(Build.SourceBranchName) --tags
displayName: Git push with tags
Required in order to push a subsequent commit
Displays a summary of the repository
Unhappy with git-status ok, I force cleaning the repo
Makes a commit and a tag out of the repository
Pushes branch and tag
git-status output
Starting: Git status
==============================================================================
Task : Command line
Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version : 2.201.1
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
Script contents:
git status
========================== Starting Command Output ===========================
/usr/bin/bash --noprofile --norc /home/vsts/work/_temp/447a869a-56d7-4a29-a4c0-9a49b5ffce74.sh
On branch main
Your branch is up to date with 'origin/main'.
nothing to commit, working tree clean
Finishing: Git status
git-reset output
Starting: Git reset
==============================================================================
Task : Command line
Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version : 2.201.1
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
Script contents:
git reset --hard
========================== Starting Command Output ===========================
/usr/bin/bash --noprofile --norc /home/vsts/work/_temp/85fedcd4-9756-4d38-aa3d-d41c03032565.sh
HEAD is now at ad6aebf Pipelines
Finishing: Git reset
npm-version output
Starting: Tag repository and set release version
==============================================================================
Task : npm
Description : Install and publish npm packages, or run an npm command. Supports npmjs.com and authenticated registries like Azure Artifacts.
Version : 1.202.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/package/npm
==============================================================================
/opt/hostedtoolcache/node/16.15.0/x64/bin/npm --version
8.5.5
/opt/hostedtoolcache/node/16.15.0/x64/bin/npm config list
; "project" config from /home/vsts/work/1/s/.npmrc
message = ":[npm-autoversion]: %s"
; "env" config from environment
cache = "/home/vsts/work/1/.npm"
userconfig = "/home/vsts/work/1/npm/7212.npmrc"
; node bin location = /opt/hostedtoolcache/node/16.15.0/x64/bin/node
; cwd = /home/vsts/work/1/s
; HOME = /home/vsts
; Run `npm config ls -l` to show all defaults.
/opt/hostedtoolcache/node/16.15.0/x64/bin/npm version patch
npm ERR! Git working directory not clean.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/vsts/work/1/.npm/_logs/2022-05-19T13_01_16_005Z-debug-0.log
##[warning]Couldn't find a debug log in the cache or working directory
##[error]Error: Npm failed with return code: 1
Finishing: Tag repository and set release version
Question: how is it possible that npm fails to make a version bump if I am triple-sure that the repository is clean?

Related

NPX command not running in Azure pipeline

I need your help
I was trying to run my cypress test cases which I generally run using command
"npx cypress run"
so I tried to have the same command in tasks when I created Azure pipeline
after NPM Install
I even tried installing npx via
npm task and custom command "npm install nx"
and this causing the below issue so can anyone suggest to me how to proceed in this case
"##[warning]Couldn't find a debug log in the cache or working directory
##[error]Error: Npm failed with return code: 1
"
From the error screenshot, it shows that you are using the Npm Task and running the command: npm npm install nx.
The command is invalid.
To solve this issue, you need to remove the npm in the NPM task -> Command and arguments .
Refer to the following sample:
YAML Pipeline:
- task: Npm#1
displayName: 'npm custom'
inputs:
command: custom
verbose: false
customCommand: 'install nx'
Classic Pipeline:

Get npm package latest version from a gitlab registry

I'm trying to put a package in a Gitlab registry using npm and .gitlab-ci.yml.
.gitlab-ci.yml
npm-package:
stage: build
image: node:buster
before_script:
- git config --global user.name "${GITLAB_USER_NAME}"
- git config --global user.email "${GITLAB_USER_EMAIL}"
script:
- npm config set #${CI_PROJECT_ROOT_NAMESPACE}:registry=https://${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/
- npm config set //${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}
- NPM_PACKAGE_NAME=$(node -p "require('.package.json').name")
- NPM_PACKAGE_VERSION=$(node -p "require('./package.json').version")
- echo $(npm view "${NPM_PACKAGE_NAME}" versions)
- echo ${NPM_PACKAGE_NAME}
- echo ${NPM_PACKAGE_VERSION}
- |
if [[ $(npm view "${NPM_PACKAGE_NAME}" versions) != *"'${NPM_PACKAGE_VERSION}'"* ]]; then
npm config set //${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}
npm publish
echo "Successfully published version ${NPM_PACKAGE_VERSION} of ${NPM_PACKAGE_NAME} to GitLab's NPM registry: ${CI_PROJECT_URL}/-/packages"
else
echo "Version ${NPM_PACKAGE_VERSION} of ${NPM_PACKAGE_NAME} has already been published, so no new version has been published."
fi
I tried my first time and the package was saved successfully in the repo. And now I'm trying to run it under the condition: if the version's package has changed then run an npm publish.
But the variable $(npm view "${NPM_PACKAGE_NAME}" versions) seems to be empty, and when I try to echo it I get the error:
npm ERR! code E401
npm ERR! 401 Unauthorized - GET https://gitlab.example.com/api/v4/projects/1/packages/npm/#my_scope/my_package
Any help?
Your CI does not have enough rights.
1- You need to generate an access token (automation type) from your npm registry via the npm UI (or via the command line).
The process with the UI:
https://docs.npmjs.com/creating-and-viewing-access-tokens
2- Assign the token to an environment variable (named NPM_TOKEN in this example) accessible to your CI
3- Then create (or update) a .npmrc file with this line at the top:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}

How to solve npm error code: 254 in azure DevOps?

When I try to include my cypress tests (via nodejs & .yml) to azure DevOps, I get the following Error:
Npm failed with return code: 254
What does it mean and how can I solve this?
My complete Log is here:
Starting: Npm
==============================================================================
Task : npm
Description : Install and publish npm packages, or run an npm command. Supports npmjs.com and authenticated registries like Azure Artifacts.
Version : 1.174.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/package/npm
==============================================================================
SYSTEMVSSCONNECTION exists true
SYSTEMVSSCONNECTION exists true
/opt/hostedtoolcache/node/10.22.0/x64/bin/npm --version
6.14.6
/opt/hostedtoolcache/node/10.22.0/x64/bin/npm config list
; cli configs
metrics-registry = "https://registry.npmjs.org/"
scope = ""
user-agent = "npm/6.14.6 node/v10.22.0 linux x64"
; environment configs
userconfig = "/home/vsts/work/1/npm/2560.npmrc"
; node bin location = /opt/hostedtoolcache/node/10.22.0/x64/bin/node
; cwd = /home/vsts/work/1/s
; HOME = /home/vsts
; "npm config ls -l" to show all defaults.
/opt/hostedtoolcache/node/10.22.0/x64/bin/npm run test
npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /home/vsts/work/package.json
npm ERR! errno -2
I use the following YAML:
To create this yaml i used a tutorial for Continuous integration of Cypress into azure devops.
# Node.js
# Build a general Node.js project with npm.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
displayName: 'npm install'
- task: Npm#1
inputs:
command: 'custom'
customCommand: 'run test'
continueOnError: true
- task: PublishTestResults#2
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '*.xml'
searchFolder: '$(System.DefaultWorkingDirectory)/cypress/reports/junit'
mergeTestResults: true
testRunTitle: 'Publish Test Results'
According to your error message in your logs:ENOENT: no such file or directory, open ‘/home/vsts/work/package.json’, npm cannot find your package.json file. Please check which folder your package.json is in. For example:
My package.json is in the web/app folder. I need to set this folder as working folder in npm task.
Here is the configuration of my npm task:
- task: Npm#1
inputs:
command: 'custom'
workingDir: '$(Build.SourcesDirectory)/web/app'
customCommand: 'run test'
continueOnError: true

How to use AWS Codecommit repo as npm dependency

this is my buildspec file:
version: 0.2
env:
git-credential-helper: yes
phases:
install:
runtime-versions:
nodejs: 10
commands:
- npm install
build:
commands:
- npm run build
post_build:
commands:
- echo Build completed on `date`
and in package.json I added:-
dependencies : {
"sharedLib":"git+https://git-codecommit.us-east-1.amazonaws.com/v1/repos/sharedLib#branchname"
}
I am getting build error in npm install. Any help will be appritiated.
Through IAM services give a Code commit access to the Role which is assigned to code build.
Then Code build should Run.

Azure Pipelines "Cache#2" fails with "##[error]The system cannot find the file specified"

I'm using Azure Pipelines with hosted builds to build a web project. Our build times were hitting 10-15 minutes, with most (5-10 minutes) of the time spent doing npm install. To speed this up, I'm trying to use the Cache task (https://learn.microsoft.com/en-us/azure/devops/pipelines/caching/?view=azure-devops).
However, when the auto-added task Post-job: Cache runs, it always errors out with:
##[error]The system cannot find the file specified
The host server is Windows Server 2017.
Here is my entire build YAML
# Node.js with Vue
# Build a Node.js project that uses Vue.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- develop
pool:
name: Default
variables:
FONTAWESOME_NPM_AUTH_TOKEN: $(FONTAWESOME_NPM_AUTH_TOKEN_VARIABLE)
npm_config_cache: $(Pipeline.Workspace)/.npm
steps:
- task: DutchWorkzToolsAllVariables#1
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- task: Cache#2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
path: $(npm_config_cache)
cacheHitVar: NPM_CACHE_RESTORED
- task: Npm#1
displayName: 'npm install'
inputs:
command: 'install'
condition: ne(variables.NPM_CACHE_RESTORED, 'true')
- task: Npm#1
displayName: 'npm run build'
inputs:
command: 'custom'
customCommand: 'run build'
- task: CopyFiles#2
inputs:
SourceFolder: '$(Build.Repository.LocalPath)\dist'
Contents: '**'
TargetFolder: '$(Build.StagingDirectory)'
CleanTargetFolder: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
Cache task output:
Starting: Cache
==============================================================================
Task : Cache
Description : Cache files between runs
Version : 2.0.0
Author : Microsoft Corporation
Help : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key:
- npm [string]
- "Windows_NT" [string]
- package-lock.json [file] --> F93EFA0B87737CC825F422E1116A9E72DFB5A26F609ADA41CC7F80A039B17299
Resolved to: npm|"Windows_NT"|rbCoKv9PzjbAOWAsH9Pgr3Il2ZhErdZTzV08Qdl3Mz8=
Information, ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session zzzzz
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: `npm|"Windows_NT"|rbCoKv9PzjbAOWAsH9Pgr3Il2ZhErdZTzV08Qdl3Mz8=`
Information, There is a cache miss.
Information, ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session zzzzz
Finishing: Cache
Post-job: Cache output:
Starting: Cache
==============================================================================
Task : Cache
Description : Cache files between runs
Version : 2.0.0
Author : Microsoft Corporation
Help : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key:
- npm [string]
- "Windows_NT" [string]
- package-lock.json [file] --> 2F208E865E6510DE6EEAA6DB0CB7F87B323386881F42EB63E18ED1C0D88CA84E
Resolved to: npm|"Windows_NT"|OQo0ApWAY09wL/ZLr6fxlRIZ5qcoTrNLUv1k6i6GO9Q=
Information, ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session zzzzz
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: `npm|"Windows_NT"|OQo0ApWAY09wL/ZLr6fxlRIZ5qcoTrNLUv1k6i6GO9Q=`
Information, There is a cache miss.
Information, ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session zzzzz
##[error]The system cannot find the file specified
Finishing: Cache
How can I fix my build definition so the caching works?
#Levi Lu-MSFT was right in his comment but there's a gotcha.
#FLabranche has a working solution in his answer but I believe reasoning is not quite right.
The problem
npm install and #Cache task are looking for the npm cache at different locations. Consider the flow when pipeline runs for the first time:
#Cache task: does nothing since there's no cache yet.
npm i (or npm ci) task: installs packages in node_modules/ and updates the npm cache at default location. Default location is ~/.npm on Linux/Mac and %AppData%/npm-cache on Windows. On Linux hosted cloud agent the absolute path will be /home/vsts/.npm.
(... more tasks from your pipeline)
Post-job #Cache task (added implicitly): reads the npm cache found at user-provided location to store it for future reuse. User-provided location is set by the npm_config_cache: $(Pipeline.Workspace)/.npm variable. On Linux hosted cloud agent the absolute path will be /home/vsts/work/1/.npm.
As a result, #Cache task fails with tar: /home/vsts/work/1/.npm: Cannot open: No such file or directory.
Solution
Make npm install and #Cache task use the same npm cache location.
One option suggested by Levi Lu is to update the npm config with npm config set cache $(npm_config_cache) --global but it won't work in the pipeline (at least it didn't work for me in an Azure-hosted Linux agent): Error: EACCES: permission denied, open '/usr/local/etc/npmrc'
npm ci --cache $(npm_config_cache) updates the npm cache location for a single call and it does work in this case. It feels a bit hacky though since --cache option is not even documented on the npm website.
All in all this code worked for me:
variables:
NPM_CACHE_FOLDER: $(Pipeline.Workspace)/.npm
steps:
- task: Cache#2
displayName: Cache npm dependencies
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
npm
path: $(NPM_CACHE_FOLDER)
- script: npm ci --cache $(NPM_CACHE_FOLDER)
displayName: 'Install npm dependencies'
...
You can log into your Windows Server 2017 server and check if the folder $(Pipeline.Workspace)/.npm is created and the dependencies are stored inside.
I copied and tested your yaml. It worked both on local agent(win2019) and cloud agents. You can try to run your pipeline on the cloud agents or other agents with newer system to check if it is the agent that cause this error.
The keys generated with your package-lock.json differ between the two tasks.
It happens when the file is modified. Here, they're modified by your npm install task.
You can use the restoreKeys option when configuring the Cache task to fall back onto the latest cache entry.
And I think you don't need the 'npm install' task.
Could you try replacing this :
- task: Cache#2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
path: $(npm_config_cache)
cacheHitVar: NPM_CACHE_RESTORED
- task: Npm#1
displayName: 'npm install'
inputs:
command: 'install'
condition: ne(variables.NPM_CACHE_RESTORED, 'true')
By this definition :
- task: Cache#2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
npm
path: $(npm_config_cache)
displayName: Cache npm
- script: npm ci --cache $(npm_config_cache)
Yesterday, I was able to get it working with no issue at all on a self-hosted machine agent by using this:
- task: Cache#2
inputs:
key: '**/package-lock.json, !**/node_modules/**/package-lock.json, !**/.*/**/package-lock.json'
path: '$(System.DefaultWorkingDirectory)/node_modules'
displayName: 'Cache Node Modules'
Today, trying to work on a hosted agent today and this doesn't cut it at all. Aggh, Back to the grinding board. Anyhow, maybe could work for you on your self-hosted pipeline
This seems to be related to this open issue.
I have resolved the problem by switching the build agent pool to hosted and using windows-latest image.
pool:
vmImage: 'windows-latest'