Ocassionally, when installing an (unrelated) dependency, I lose the resolved values from each of my private nexus repository dependencies, meaning that when my build server runs npm ci it falls back to attempting to install these from the npm repository, which obviously fails.
I am using npm 8.5.5/node 16.15
I am using NPM's workspaces feature to construct a monorepo, meaning that I have several project package.json files as well as a root package.json
My .npmrc (at root level) looks like this:
engine-strict=true
#foo:registry=http://prod-nexus.foo.com/repository/bar/
always-auth=true
After an (unrelated, random) install my package-lock.json will have this change:
"#foo": {
"version": "1.2.3",
- "resolved": "http://prod-nexus.foo.com/repository/bar/#foo/-/lib-1.2.3.tgz,
- "integrity": "sha...",
+ "license": "MIT",
"dependencies": { ....
Note that the resolved and integrity fields have disappeared and the license has been added.
I have run into this problem several times, each time I have solved it by rolling back and some manual editing and eventually it goes away, but I really need to understand what is going on.
What is causing this, why is it random, what can I do to defend against it?
This could be related to the issue https://github.com/npm/cli/issues/4263
clean the npm cache npm cache clean -f
run npm install again
If that doesn't work, try it again while deleting more:
clean the npm cache npm cache clean -f
remove node_modules in project folder
remove package-lock.json file
run npm install again
Related
I want to update a dependency (packageX) without changing a locked dependency of that package, (packageY). In my package-lock.json, I have:
"packageX": {
"requires": {
"packageY": "1.0.0",
}
},
Each time I do "npm install packageX," I'd like to update packageX but have packageY stay on the defined version. How can I do that?
There is no way to do this, may be this link can explain better https://dev.to/saurabhdaware/but-what-the-hell-is-package-lock-json-b04
The story about package.json vs package-lock.json is tricky: npm install does not ignore package.json versions, nor does it ignore the package-lock.json. What it does is verify that the package.json and package-lock.json correspond to each other. That is, if the semver versions described in package.json fit with the locked versions in package-lock.json, npm install will use the latter completely, just like npm ci would.
Now, if you change package.json such that the versions in package-lock.json are no longer valid, your npm install will be treated as if you'd done npm install some-pkg#x.y.z, where x.y.z is the new version in the package.json for some-package.
We have a private JFrog artifactory (name anonymised below) that npm is configured in a project root .npmrc -file:
registry=https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/
The resolved-field in the package-lock.json file shared via Git between developers is constantly changing between runs of "npm install" without any changes to package.json.
Some times a dl query parameter (pointing to the exactly same URL) gets added to the resolved URL:
- "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/#sailshq/lodash/-/lodash-3.10.3.tgz",
+ "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/#sailshq/lodash/-/lodash-3.10.3.tgz?dl=https://artifactory.jfrog.private.com/#sailshq/lodash/-/lodash-3.10.3.tgz",
Some times the query parameter points to npmjs.org registry:
- "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/aproba/-/aproba-1.2.0.tgz",
- "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/aproba/-/aproba-1.2.0.tgz?dl=https://registry.npmjs.org/aproba/-/aproba-1.2.0.tgz",
And some times the field points directly to npmjs.org repository:
- "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/acorn/-/acorn-3.3.0.tgz",
+ "resolved": "https://registry.npmjs.org/acorn/-/acorn-3.3.0.tgz",
Any of these changes may also go to the inverse direction.
This is really irritating, since it means we constantly have meaningless changes in package-lock.json, which causes merge conflicts and often prevents npm ci from executing correctly. npm cache clean --force does not seem to help. I know that npm install can resolve package-lock.json merge conflicts automatically, but that does not help with npm ci (since the whole point is to not run npm install in the CI environment). And, anyway, what is the benefit of seeing how the virtual npm registry resolves the packages internally (as I suspect is happening here)?
Is there some kind of configuration option to prevent JFrog Artifactory from making these kinds of changes to the resolved package URLs in a virtual npm registry? Or is it maybe a bug in npm?
Environment:
npm 6.11.3
JFrog Artifactory 6.10.6
I don't know why those alternate URLs are appearing or how to make them stop. But you can reduce (or maybe even eliminate!) the merge conflict pain for your developers by using npm-merge-driver. It was written by one of the devs who was employed on the npm cli team for years, and its sole purpose is to automate away package-lock.json merge conflicts.
Our team has had success running npm ci first to ensure our locally pulled down and cached dependencies match the package-lock.json file.
Then, further npm installs should resolve as expected.
This sort of thing is normally caused by developers having slightly different versions of npm installed. Version 7 of npm just got released, so it is the perfect time to make sure the team all have exactly the same version installed.
If that doesn’t work try switching the team to yarn or pnpm.
In my package.json file I have:
"dependencies": {
"#aws/dynamodb-data-mapper": "^0.7.3",
"#aws/dynamodb-data-mapper-annotations": "^0.7.3",
"#my-company/common": "file:../common",
...
}
#my-company/common specifies a local module which is installed by yarn.
However, when I run npm prune, this module is pruned.
I thought that including it in the package.json dependencies object would stop it from being pruned? Did I miss something?
For those who come across this, I discovered that running npm ls showed that there was an error with the common module.
Installing with NPM instead of Yarn fixed the issue.
Installing with Yarn did create the package within node_modules, however there was obviously an error somewhere which I couldn't uncover.
I cloned a repository from github which has a package-lock.json (but no package.json). Then in a git bash terminal I go to the directory and run npm install but I just get a message saying there is no package.json and then everything in package-lock.json gets deleted so it's basically empty except for the project name and version.
I thought running npm install with a package-lock.json in the directory was enough to re-create node_modules, but am I seriously misunderstanding how this works? By the way I have node 8.12.0 and npm 6.4.1 and am running on Windows 10. Also, I think the package-lock.json was created on a unix system so could there be problems when using package-lock.json on a different OS?
I already tried running npm init just to get a package.json file and then running npm install but that still didn't get me a node_modules folder.
Starting from Mar 5, 2018, you can run npm ci to install packages from package-lock.json.
npm ci bypasses a package’s package.json to install modules from a
package’s lockfile.
https://blog.npmjs.org/post/171556855892/introducing-npm-ci-for-faster-more-reliable
package-lock.json records the exact version and url of packages need to install, thus you can use npm to install them accordingly:
npm can install from urls that point to tarballs
--no-package-lock option to tell npm to not touch package-lock.json file
For example, to install all packages in package-lock.json:
cat package-lock.json | jq '.dependencies[].resolved' | xargs npm i --no-package-lock
jq is a command line tool to pares jq, you can write a simple JavaScript script to parse it instead (if you do not want to install jq or learn jq's query syntax).
AFAIK, the package-lock.json file relies on the presence of a package.json file, so you'll not be able to recreate your node_modules folder from the package-lock.json file alone (happy to be proved wrong here).
Therefore, your best bet is to (mis)use a module like auto-install that is capable of generating the package.json file based on a project's dependencies, as they appear in the files.
Install it globally (npm install -g auto-install), then you'll need to generate an empty package.json file for it to run (use npm init -y in your project root). Kick things off with the command auto-install and it should add the dependencies to the package.json file.
HTH
Let's say that someone installed an invalid local dependency. (file does not exists locally)
package-lock.json
"mock-framework": {
"version": "file:../../../mock-framework",
package.json:
"dependencies": {
"mock-framework": "file:../../../mock-framework"
}
I need to reinstall the framework, but it's located differently on my machine and does not follow the structure that was provided in the package locks. Because running the npm install command is giving me the error:
Could not install from "../../../mock-framework" as it does not contain a package.json file.
Would it be possible to clean it up through the command line? I tried with npm uinstall and still no luck.
I recently faced similar issue with local dependency integrity in package-lock.json
Ideally npm uninstall should remove the entry in package-lock.json but since it is not and you only have one local framework as changed dependency, you can try following -
Fix the dependency path and run rm package-lock.json && npm i
Hope I'm inline to your problem statement.