npm ci can only install packages with an existing package-lock.json or npm-shrinkwrap.json with lockfileVersion >= 1 - npm

This is the issue that I am facing when running the command npm ci to install dependencies in my GitHub Action file.
I am working on an expo managed app and using GitHub Actions as a CI for triggering builds whenever I push my code to developmemt branch.
Here's my build script:
name: EAS PIPELINE
on:
push:
branches:
- development
workflow_dispatch:
jobs:
build:
name: Install and build
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
with:
persist-credentials: false
- name: Setup Node.js
uses: actions/setup-node#v1
with:
node-version: 14.x
- name: Setup Expo
uses: expo/expo-github-action#v6
with:
expo-version: 4.x
token: ${{ secrets.EXPO_TOKEN }}
expo-cache: true
- name: Install dependencies
run: npm ci
- name: Build on EAS
run: EAS_BUILD_AUTOCOMMIT=${{1}} npx eas-cli build --platform all --non-interactive
Here's the issue that I am facing Install dependencies step.
Run npm ci
npm ci
shell: /usr/bin/bash -e {0}
env:
EXPO_TOKEN: ***
npm ERR! cipm can only install packages with an existing package-lock.json or npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm#5 or later to generate it, then try again.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/runner/.npm/_logs/2021-10-28T15_16_06_934Z-debug.log
Error: Process completed with exit code 1.

After a lot of research, I was able to figure out that this happens when you are not using npm install for installing dependencies. In my case, I was only using yarn for the dependencies so I was only having yarn.lock file and no package-lock.json file.
One way to resolve this was using npm install to install the dependencies, then you'll have a package-lock.json file and CI won't throw any error.
And the other way if you only want to use yarn, then you need to update that step in your eas-pipeline.yml file for installing the dependencies.
*****************************************************************************************
- name: Install dependencies
run: |
if [ -e yarn.lock ]; then
yarn install --frozen-lockfile
elif [ -e package-lock.json ]; then
npm ci
else
npm i
fi
***************************************************************************************
As I wasn't able to find any solution on StackOverflow and it is our first go-to place to look for any issue. So, I decided to write this answer here.
Here's the original answer: https://github.com/facebook/docusaurus/issues/2846#issuecomment-691706184

I had a similar error.
`npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
with bunch of missing dependency name following this error.
I would run npm ci locally and it would run successfully. However, it would give me the error above when npm ci is run in the CI pipeline and in my case it was because of the version difference of the Node.js installed in the environment that Jenkins pipeline is running in.
My local Node version was 16.x and in Jenkins container it was 12.x.
Upgrading it fixed.

Old post, but I found this while searching for this same error. In my case I did have a package-lock.json file in my root directory. However, when opening it, I realized that there was a JSON syntax error that slipped in during a previous merge conflict. After running npm i again the file was fixed. The npm ERR! The 'npm ci' command can only install with an existing package-lock.json wasn't a super helpful error in that case.

This same thing happened to me, and I couldn't figure it out for the longest time. It turns out that I had legacy-peer-deps=true set globally, and I had no idea.
This caused my npm install command to alter the package-lock.json in a way that broke the build on our CI server. I reset package-lock.json with the version from master, removed that npm config, and reinstalled. Everything worked fine after that.

For the people who has this issue on AWS Amplify. You might have to run npm install and commit the package-lock.json file, then deploy again.

If you're using pnpm, you install node dependencies via this step:
- name: Install Node.js dependencies
run: |
npm i -g pnpm
pnpm i

After battling with this issue for about 2 days, It's finally deploying successfully to Firebase Functions after deleting package-lock.json from both the src and src/functions folders.

This occurred because the package lock file was not generated with npm, despite the fact that the npm ci requires npm to install the packages.
And because npm requires package-lock.json, we get this error. To fix this error for GitHub actions, this what I did:
- run: yarn install --frozen-lockfile
- run: yarn lint
- run: yarn test:ci
Commit diff:

deleting deploy.json helped me, since the token is updated when overwritten
rm ~/.config/configstore/#vkontakte/vk-miniapps-deploy.json
but I have other services

This happens sometimes because of the version difference of the Node.js installed in the environment that the pipeline is running in.
To fixe this, I ran:
$ firebase init hosting:github
then type Y to set up workflow when asked to.
finally, add "npm i" as one of the scripts to run before deploying like this:
npm i && npm ci && npm run build

I was using npm package manager and migrated to yarn package manager removing package-lock.json file.
I had this configuration in my .circleci/config.yml file
- node/install-packages
changed to
- node/install-packages:
pkg-manager: yarn

In my case the problem were some 'extraneous' packages, concretely local path dependencies. After removing them from package.json the problem was solved.
I got the error message while running npm install instead of npm ci.

On package.json I change this :
"overrides": {
"trim-newlines": "^3.0.1"
},
to : `
"overrides": {
"trim-newlines": "^1.0.0"
}
`
That Work for me successfully.

I had a similar problem deploying to heroku. I simply deleted the existing package-lock.json file and then ran
npm install
Merging the new lock file fixed the deploy.

In my case, I had this error with npm ci while using Yarn. I eventually figured out the version of Node I was using wasn't supported.
I did the following:
node -v to confirm my node version (18.0.1)
nvm use 16.13.0
Delete the node_modules directory
Delete yarn.lock
Run yarn
Run yard add + package names
After this, the error no longer occurred and the app deployed correctly.

Related

How can I set the npm version in gitlab CI?

I have a gitlab pipeline for an Angular project with the image in .gitlab-ci.yml
image: node:16
build:
stage: build
script:
- npm ci
- nodejs -v
- npm -v
- npm run build:prod
- npm doctor
When the pipeline reaches the npm doctor it fails with the following error on npm version:
$ npm doctor
Check Value Recommendation/Notes
npm ping ok
npm -v not ok Use npm v7.24.0
node -v ok current: v16.9.1, recommended: v16.9.1
npm config get registry ok using default registry (https://registry.npmjs.org/)
which git ok /usr/bin/git
Perms check on cached files ok
Perms check on local node_modules ok
Perms check on global node_modules ok
Perms check on local bin folder ok
Perms check on global bin folder ok
Verify cache contents ok verified 1361 tarballs
How can I solve this?
In my machine I have the npm version v7.24.0 and the command succeeds.
Install npm before you do anything else, specifying a version, i.e. npm install npm#version -g.
For completeness, if you can't update the global npm, you can do so locally, by replacing npm with npx npm#latest or npx npm#7.24.0. npx will install npm if it needs to, locally, and then run the local installation.
Lastly you can install locally with npm install npm#latest and then run it with $(npm bin)/npm, but this is what npx is for.
Note that I don't understand why you're running npm doctor after a build. Presumably if you care about npm thining it's set up properly you should run it before the build, so the pipeline fails early.
P.S. I stupidly read 'github' as 'gitlab'. If you were using github I would recommend using
setup-node if only for dependency caching, which can seriously speed up pipelines. I don't know if gitlab does anything similar.

How I can skip installing optional dependencies by 'npm ci'?

How I can skip installing optional dependencies from package-lock.json by npm ci?
You can use npm ci --no-optional .
If npm still installs the optional package. Then try after removing package.lock.json and run the command again.
There was an error in NPM's implementation of npm ci --no-optional. It has been fixed in versions > 6.13.3 - maybe earlier versions as well, but I can only vouch for 6.13.4 and up.
I was facing this issue with CI workflow script and even "--no-optional" was not working
npm ci --no-optional
The above command only worked when I added the optional package as
"optionalDependencies": {
"fsevents": "^2.3.2"
}
in the package.json file
It's not a proper solution, rather an ugly one, but it helped me out. It looks like npm ci --no-optional doesn't work and probably never worked. But at the same time flag --production works. And if we afford mutating package.json (e.g. in a docker container) then...
So I wrote a simple script that:
reads package.json content
Object.assign(cfg.dependencies, cfg.devDependencies)
delete cfg.devDependencies
overwrites the initial package.json
So finally we have:
dependencies contains both normal & dev dependencies
devDependencies section is empty
optionalDependencies are intact
And when we run npm ci --production we got what we want - no optional dependencies (in my case cypress). Due to the fact that all these steps are performed inside of a docker container we can mutate package.json.
But I'm not sure that it'll help you too.
In order to make npm ci --no-optional skip/ignore an optional pacakge, it's important to understand how npm intracts with package.json and pacakge-lock.json.
npm install --no-optional (is only effective if pacakge-lock.json doesn't exists otherwise it would ignore --no-optional)*
npm ci --no-optional is only effective if pakcage-lock.json was already created with npm install --no-optional**.
* This means if you want to make an already installed package an optional, you can would have to
Add it "optionalDependencies": either manulally or through npm install pacakge-name --save-optional
Delete the pacakge-lock.json.
then run rm -rf node_modules/
Lastly run npm install --no-optional
Add this point npm ci --no-optional isn't suppose to install it.
** TIP: you could debug if a certian package is assigned as optional by running npm ls package-name
Note: This one the reason why its recommended to keep trak pacakge-lock.json with git repo of the project.

npm - tarball data for material-design-icons seems to be corrupted

I'm having this error while running a npm install material-design-icons#3.0.1:
tarball data for material-design-icons#3.0.1 (sha1-mnHEh0chjrylHlGmbaaCA4zct78=) seems to be corrupted
npm ERR! path D:\speech-analytics\node_modules\.staging\material-design-icons-7d5a1f73\action\drawable-xxhdpi\ic_assignment_ind_white_48dp.png
npm ERR! code EPERM
npm ERR! errno -4048
npm ERR! syscall unlink
npm ERR! Error: EPERM: operation not permitted, unlink 'D:\\speech-analytics\node_modules\.staging\material-design-icons-7d5a1f73\action\drawable-xxhdpi\ic_assignment_ind_white_48dp.png'
npm ERR! { Error: EPERM: operation not permitted, unlink 'D:\\speech-analytics\node_modules\.staging\material-design-icons-7d5a1f73\action\drawable-xxhdpi\ic_assignment_ind_white_48dp.png'
Here it's documented as a bug, but still without an answer nor a fix.
I've tried to reinstall node, upgraded to latest npm version (currently running 6.4.1), did a cache clean --force, tried a npm install --no-optional, removed package-lock.json, removed npm & npm-cache folders from AppData directory, running everything as Administrator, but still no luck.
I even tried with material-design-icons#3.0.0 but the error remains.
If I navigate to the folder that appears in the log (node_modules\.staging\material-design-icons-7d5a1f73\action\drawable-xxhdpi), it's empty, and is the only folder that exists in the entire node_modules directory. I can delete that dir without any problems, so it does not seem to be a permissions/lock issue.
Any suggestions?
Finally, I got this fixed by:
Removing node_modules folder
Running npm update
Running npm install
As far I understand, the npm update should have updated the package.json file, but all dependencies kept the same versions as we had it before.
I resolved this with the command: npm cache verify which output:
Cache verified and compressed (C:\Programs\DCPS\npm-cache\_cacache):
Content verified: 1344 (164824963 bytes)
Content garbage-collected: 1 (3491551 bytes)
Index entries: 1522
Finished in 8.187s
The line that stands out to me is: Content garbage-collected: 1 (3491551 bytes)
Does this sort of thing happen because a new version of a package is published to npmjs without a version bump?
No need to run npm update (I didn't want to update any packages) or delete the entire node_modules folder. I solved this by
deleting package-lock.json
deleting node_modules\material-design-icons-xxxxxxx
running npm install again
If npm update is not a solution, and as deleting package-lock.json can bring issues of its own, I could solve it simply by:
deleting the node_modules/ folder
in package-lock.json, deleting the sections referencing to the corrupted package
running npm install again
If you are on windows env, I fixed it by running the cmd as administrator
You need to confirm whether the Typescript is installed and after installing typescript it worked for me
running the below comment will show the typescript version
tsc -v
If it shows some error install the typescript
npm install -g typescript
If then typescript is installed you can try checking the Angular Cli version
ng --version
If it shows some error then install Angular Cli Ref: https://cli.angular.io/
npm install -g #angular/cli
" If you are on windows env, I fixed it by running the cmd as administrator "
This worked for me. However, chromedriver was not installed. So, i installed it separately using the command 'npm install chromedriver'.
None of the answers solved my problem, because in my case was the git. Maybe someone can have the same problem.
I had some dependencies from git in the project and my git was not working on the terminal. So fixing the path for git fixed it!
I had this in Bitbucket Pipeline when using a private package.
I was missing to install git in pipeline:
script:
- apk update && apk upgrade && apk add --no-cache bash git openssh # <- THIS
- npm ci --prefer-offline
Dependency was
"some-private-package": "git+ssh://git#bitbucket.org/workspace/some-private-package#v1.0.12",`
Well I could not resolve this problem with a lot of tries so I made the download of the github ZIP, unzip and install and it worked !
download material-design-icons from github
unzip to the directory of your project (or c:\tmp)
npm install ./material-design-icons
or
npm install c:/tmp/material-design-icons

NPM not installing devDependencies on bitbucket pipeline?

I'm trying to setup my first Bitbucket pipeline which simply builds my application and deploys it to my FTP server using the following bitbucket-pipelines.yml
image: node:6.9.4
pipelines:
default:
- step:
caches:
- node
script:
- npm install
- npm test
- step:
script:
- npm run build
- node deploy.js
The issue lies in the npm install because when bitbucket tries to run the npm run build command it says that rimraf (a npm package) is not found. rimraf however is listed in my devDependencies, all regular dependencies in my package.json are downloaded correctly.
There is no global variable set by my so the NODE_ENV could not be it right?
I had the same issue with gulp.
Gulp was in devDependencies and also specified in package.json as script but still it said npm ERR! missing script: gulp
The documentation says to install it globally, so there might be a related issue with your package.
https://confluence.atlassian.com/bitbucket/javascript-node-js-with-bitbucket-pipelines-873891287.html
I had this same issue. For me, the problem was that the version of Node on my local development device was different from the version of Node in the bitbucket-pipelines.yml file.
To fix it, I went into bitbucket-pipelines.yml and changed this line:
image: node:10.15.3
to this:
image: node:14.15.0

NPM Cannot read property '0' of undefined

After updated Node (upto v8.6.0) and npm (upto v5.5.1) I cannot execute command npm install.
After npm install I've error message:
npm ERR! Cannot read property '0' of undefined
What's trouble or I need downgrade node/npm ?
I had the same problem.
I removed both node_modules and package-lock.json and then did:
npm install
And it worked.
Edit by #OwlyMoly
Due to new updates and the restriction to old dependencies in package-lock.json is causing this conflicts. By doing npm install won't fix this issue. Instead by ditching npm_modules and package-lock.json and doing npm install will load a new node_modules and that supposed to be required by package.json. You have to commit the new package-lock.json along with your latest changes of the project.
Do 2 steps bellow (Window):
rm -rf ./node_modules to remove node folder
rm package-lock.json to remove package-lock.json file
then npm install to re-install the node modules
Just download and install latest Yarn which is also a node package manager, developed by facebook, but has a much better dependency management. Also update your node cli (optional).
And then, install your dependencies using yarn:
yarn install
or
yarn // short version of yarn install
No errors!
You can continue to use npm after you have installed all dependencies with yarn or continue with yarn....it's your choice.
I've made some tests:
nodejs#8.6.0 npm#5.5.1 - I have trouble and the test fails
nvm use 8.5.0
nodejs#8.5.0 npm#5.5.1 - I have trouble and the test fails
nvm use 8.4.0
nodejs#8.4.0 npm#5.5.1 - I have trouble and the test fails
npm install npm#^5 -g
nodejs#8.4.0 npm#5.4.2 - I have trouble and the test fails
nvm use 8.6.0
npm install npm#^4 -g
nodejs#8.6.0 npm#4.6.1 - no trouble, this fixes it.
Seems to be an issue with a combination of factors.
Some workarounds here:
https://github.com/npm/npm/issues/18238
npm 5.3.0 is broken for windows 10 after upgrading the nodeJS.
You should downgrade the npm, it is a temporary solution but works fine.
npm install -g npm#5.2.0
For me (npm#6.9.0) solved the issue by deleting node_modules and performing npm install, but without removing package.json.lock file.
Just remove both node_modules and package-lock.json and run: npm install
or
Just run: npm install -g npm#latest to upgrade it to the latest version
Try with nvm(Node Version Manager).it help you to install any node version for any project without any Error.
I found same problem when using npm version 5.5.1 to install babel-preset-stage-0
Solution:
I downgraded npm to version 5.2.0 and try to install again then it can solve the issue.
npm i -g npm#5.2.0
npm i -D babel-preset-stage-0
I bumped into this issue using nvs (Node Version Switcher - https://github.com/jasongin/nvs) node#10.15.3 and npm#6.9.0. The reason was a local package I had linked with npm link. The solution was to remove that folder.
In my case reinstalling node_modules have not fixed this issue.
Problem was with one *.ts file which was missing in source codes.
Do not know why It was not displaying compilation error, but adding this missing file to repository solved this issue.
Upgrading npm to version 7.5.4 did the job for me.
npm install -g npm#latest
What worked for me:
npm ci
Install a project with a clean slate
docs: https://docs.npmjs.com/cli/v7/commands/npm-ci
Deletes node_modules and installs everything based on package-lock.json, so no need to regenerate that