Unable to install NPM packages due to path length constraint using NPM 5.X - npm

I'm running Windows 10, NPM v5.5.1, and node v8.9.1.
Edit: I have since updated to node v8.9.4 and npm v5.6.0. The problem remains unchanged.
I set up my development environment on this machine about 2 months ago. I'm using Angular CLI to build my client apps.
Everything was going fine. I added a few packages via NPM without a problem, such as Gulp and ngx-bootstrap. Then I started to run into errors on some of my package installs.
The errors are:
npm ERR! code PATH_LENGTH_EXCEEDED
npm ERR! errno PATH_LENGTH_EXCEEDED
npm ERR! request to https://registry/npmjs.org/{{package}} failed, reason: path length constraint exceeded
npm ERR! A complete log of this run can be found in:
npm ERR! {{npm_cache path}}\_logs\2018-02-08T14_43_40_856Z-debug.log
The contents of that log file are:
0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli 'C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli 'install',
1 verbose cli 'ngx-cookie-service',
1 verbose cli '--save' ]
2 info using npm#5.5.1
3 info using node#v8.9.1
4 verbose npm-session d55be12849534a0a
5 silly install loadCurrentTree
6 silly install readLocalPackageData
7 silly fetchPackageMetaData error for ngx-cookie-service#latest request to https://registry.npmjs.org/ngx-cookie-service failed, reason: path length constraint exceeded
8 verbose type system
9 verbose stack FetchError: request to https://registry.npmjs.org/ngx-cookie-service failed, reason: path length constraint exceeded
9 verbose stack at ClientRequest.req.on.err (C:\Program Files\nodejs\node_modules\npm\node_modules\pacote\node_modules\make-fetch-happen\node_modules\node-fetch-npm\src\index.js:68:14)
9 verbose stack at emitOne (events.js:116:13)
9 verbose stack at ClientRequest.emit (events.js:211:7)
9 verbose stack at TLSSocket.socketErrorListener (_http_client.js:387:9)
9 verbose stack at emitOne (events.js:116:13)
9 verbose stack at TLSSocket.emit (events.js:211:7)
9 verbose stack at emitErrorNT (internal/streams/destroy.js:64:8)
9 verbose stack at _combinedTickCallback (internal/process/next_tick.js:138:11)
9 verbose stack at process._tickCallback (internal/process/next_tick.js:180:9)
10 verbose cwd C:\Development\test
11 verbose Windows_NT 10.0.15063
12 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "ngx-cookie-service" "--save"
13 verbose node v8.9.1
14 verbose npm v5.5.1
15 error code PATH_LENGTH_EXCEEDED
16 error errno PATH_LENGTH_EXCEEDED
17 error request to https://registry.npmjs.org/ngx-cookie-service failed, reason: path length constraint exceeded
18 verbose exit [ 1, true ]
I researched this, and found the Windows 10 file path 260 character limitation. I thought that npm version 3.1.0 should have fixed this issue, but I decided to followed the instructions in this answer, because I'm still running into this issue despite using NPM v5.5.1.
I found that my version of Windows was prior to the roll out of the feature that lets you disable this limitation. I updated to Version 1703 (OS Build 15063.483), and to enable Win32 long paths.
Yet I'm still getting the error, and unable to install a variety of npm packages.
In testing to see if it was an issue with packages that have a lot of dependencies, I tried installing or reinstalling a couple of other packages.
Some fail with the errors above. Others succeed with the following warnings:
npm WARN Unexepcted warming for https://registry.npmjs.org/: Miscellaneous Warning PATH_LENGTH_EXCEEDED: request to https://registry.npmjs.org/{{package name}} failed, reason: path length constraint exceeded
npm WARN Using stale package data from https://registry.npmjs.org/ due to a request error during validation
Packages throwing the error include:
jslint
ngx-cookie-service
chart.js
angular-cli
Packages that throw the warnings include:
font-awesome
#angular/cli
moment
ngx-bootstrap
bootstrap
jasmine
I suspect that something has changed, either in a policy on my workstation, or in our network security, that is causing this. All of the packages that succeed seem to be packages that I had previously installed, either as a direct install, or as a dependency of #angular/cli (one of the first packages I installed). I'm assuming that the stale package data warning indicates that whatever problem I have with the path length is preventing updated data being retrieved, and therefore npm is falling back to cached data from previous installations of the selected package and its dependencies.
So now I'm stumped as to how to diagnose exactly where the issue is happening. If its a security policy that changed, then I need to be able to describe exactly what is being blocked so that I can work with our security department to fix this. But the only information I can find about the PATH_LENGTH_EXCEEDED errors lead me to old information about versions of npm prior to 3.1.0.
How can I identify the source of this problem?

After additional searching, I found references to path length exceeded with regards to SSL/TLS certificate authorities. I tested this issue by changing the npm registry to use http instead of https (note: this is a temporary step only; I have turned https back on once I verified the issue):
npm config set registry http://registry.npmjs.org/
This allowed me to install the problematic packages.
It seems the underlying cause is SSL Inspection performed by our network security team. The decryption, inspection, and re-encryption process, and subsequent changes in the Issuing Authority on the certificate, apparently triggers this error.

Related

npm install throws ECONNRESET error but only for certain packages

I'm working behind a corporate proxy. I have npm configured like this:
$ npm config get
...
https_proxy = "http://proxy.my-domain.com:8080"
https-proxy = "http://proxy.my-domain.com:8080"
proxy = "http://proxy.my-domain.com:8080"
...
With these settings, I can install some packages fine, but not others. For example $ npm i react works perfectly fine, whereas installing #babel/core throws an ECONNRESET error.
$ npm i #babel/core
npm ERR! code ECONNRESET
npm ERR! syscall read
npm ERR! errno -54
npm ERR! network read ECONNRESET
...
Strangely, I'm able to install the package with yarn (which is configured for the proxy the same way npm is), but it too tells me that I have network issues (even though it successfully installs the package 🤔)
$ yarn add #babel/core
...
✨ Done in 2.99s.
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
I can't tell why it works with yarn, but not with npm. Here's what the npm debug log looks like:
73 silly tarball no local data for #jridgewell/sourcemap-codec#https://registry.npmjs.org/#jridgewell/sourcemap-codec/-/sourcemap-codec-1.4.14.tgz. Extracting by manifest.
74 silly tarball no local data for #jridgewell/trace-mapping#https://registry.npmjs.org/#jridgewell/trace-mapping/-/trace-mapping-0.3.17.tgz. Extracting by manifest.
75 silly tarball no local data for #jridgewell/set-array#https://registry.npmjs.org/#jridgewell/set-array/-/set-array-1.1.2.tgz. Extracting by manifest.
76 verbose stack Error: read ECONNRESET
76 verbose stack at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)
77 verbose cwd /Users/justinsmith/Dev/test-npm
78 verbose Darwin 21.4.0
79 verbose node v18.9.0
80 verbose npm v8.19.1
81 error code ECONNRESET
82 error syscall read
83 error errno -54
Anyone have any ideas why this is happening?
tl;dr adding this to my ~./npmrc provided a temporary fix:
# This is what I already had set
proxy=http://proxy.my-domain.com:8080/
https-proxy=http://proxy.my-domain.com:8080/
https_proxy=http://proxy.my-domain.com:8080/
# This is what I added
strict-ssl=false
registry=http://registry.npmjs.org/
maxsockets=3
This forces npm to make requests with http instead of https and limits the number of parallel requests to three.
One reason this works is because evidently the corporate proxy was scanning files for malware before forwarding them to the client. Evidently npm making too many requests in parallel over TLS overwhelms the malware scanner which eventually refuses some connections.
Yarn on the other hand will retry a couple of times whereas npm just throws an error and gives up.
A longer-term solution would be to get your IT department to add npm to a malware scanning exclusion list.
Full credit goes to this blog post, which provided these helpful solutions.

npm install fails with 405 Method Not Allowed

I am currently working on upgrade some projects to use node 18 with npm 8, from node 12 with npm 6.
We're using github as npm repository for our private packages.
Now I am getting this error on npm install:
npm ERR! code E405
npm ERR! 405 Method Not Allowed - GET https://npm.pkg.github.com/<our-github-organisation>/#types/mime/-/mime-3.0.1.tgz
In the error log:
456 verbose stack HttpErrorGeneral: 405 Method Not Allowed - GET https://npm.pkg.github.com/<our-github-organisation>/#types/mime/-/mime-3.0.1.tgz
456 verbose stack at /home/cclausen/.nvm/versions/node/v18.7.0/lib/node_modules/npm/node_modules/npm-registry-fetch/lib/check-response.js:93:15
456 verbose stack at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
457 verbose statusCode 405
458 verbose pkgid #types/mime#https://npm.pkg.github.com/<our-github-organisation>/#types/mime/-/mime-3.0.1.tgz
This does not happen on every package, only on a few select ones, I can't see pattern.
None of the affected patterns our ours, just random packages.
Why would this happen?
It seems the content of the .npmrc file used in the repository to specifc from where to get packages has changed, updating that file to comply with the newest information from githubs documentation has fixed this issue.
So the new content for me is:
#OWNER:registry=https://npm.pkg.github.com
Where OWNER is my organisation.

Cannot run npm commands due to sh: 1: webpack: not found error; same exact codebase runs commands fine elsewhere

I'm having a very strange issue with nvm, npm and webpack.
We have a webpack file with a npm run publish command. I have 4 stacks setup locally on the same code repository. In 3 of the stacks, I can npm run publish without an issue. However, in the 4th stack I always get a sh: 1: webpack: not found error when I run the command. The worrisome bit, is the codebases are identical. I've ran diffs, altered permissions, ran npm rebuild, re-ran npm install and nvm use. I've removed the node_modules directory and ran an npm install. No matter what I do, I get the same error.
Here is the verbose error with a few local names blocked out:
21 timing command:run-script Completed in 25ms
22 verbose stack Error: command failed
22 verbose stack at ChildProcess.proc.on (/home/<usr>/.nvm/versions/node/v11.14.0/lib/node_modules/npm/node_modules/#npmcli/promise-spawn/index.js:64:27)
22 verbose stack at ChildProcess.emit (events.js:193:13)
22 verbose stack at maybeClose (internal/child_process.js:999:16)
22 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:266:5)
Any help is very much appreciated! Thanks!
I ended up resolving it by copying over the node_modules directory from one of the working codebases. Not entirely sure what the problem was though.

Vue create “projectName” ERR_SOCKET_TIMEOUT - fails all the time no matter what - taobao

I am getting a bit confused about what’s been happening to me recently.
Whenever I want to create a new vue project, it fails with:
Vue CLI v4.5.11
✨ Creating project in /home/admin/Documents/dev/my_projects/vThesaurus/thesaurus.
⚙️ Installing CLI plugins. This might take a while...
npm ERR! code ERR_SOCKET_TIMEOUT
npm ERR! errno ERR_SOCKET_TIMEOUT
npm ERR! request to https://registry.npm.taobao.org/jsonfile failed, reason: Socket timeout
npm ERR! A complete log of this run can be found in:
npm ERR! /home/admin/.npm/_logs/2021-03-17T06_40_10_793Z-debug.log
ERROR command failed: npm install --loglevel error --legacy-peer-deps
another packages from prev attempts:
request to https://registry.npm.taobao.org/core-js-compat failed, reason: Socket timeout
request to https://cdn.npm.taobao.org/micromatch/-/micromatch-4.0.2.tgz failed, reason: Socket timeout
request to https://registry.npm.taobao.org/querystring-es3 failed, reason: Socket timeout
request to https://registry.npm.taobao.org/file-loader failed, reason: Socket timeout
… and many more
Log: (last part)
4675 timing command:install Completed in 378261ms
4676 verbose type system
4677 verbose stack FetchError: request to https://registry.npm.taobao.org/jsonfile failed, reason: Socket timeout
4677 verbose stack at ClientRequest.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:97:14)
4677 verbose stack at ClientRequest.emit (events.js:315:20)
4677 verbose stack at TLSSocket.socketErrorListener (_http_client.js:463:9)
4677 verbose stack at TLSSocket.emit (events.js:327:22)
4677 verbose stack at emitErrorNT (internal/streams/destroy.js:100:8)
4677 verbose stack at emitErrorCloseNT (internal/streams/destroy.js:68:3)
4677 verbose stack at processTicksAndRejections (internal/process/task_queues.js:84:21)
4678 verbose cwd /home/admin/Documents/dev/my_projects/vThesaurus/thesaurus
4679 verbose Linux 5.4.0-67-generic
4680 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "--loglevel" "error" "--legacy-peer-deps"
4681 verbose node v14.2.0
4682 verbose npm v7.6.3
4683 error code ERR_SOCKET_TIMEOUT
4684 error errno ERR_SOCKET_TIMEOUT
4685 error request to https://registry.npm.taobao.org/jsonfile failed, reason: Socket timeout
4686 verbose exit 1
the weirdest part is that it always fails on a different package …
I tried:
to remove nodejs and vue, reinstall again but no luck …
to increase timeout in the ~/.npmrc but no luck again … (I don’t have slow internet connection - ping 3ms, d: 96Mbps, u: 9.6Mbps)
nothing I found on the internet seems to be working …
changing IPs (nordvpn) to reach to diff servers (CDN) but no luck
vue2 and vue3 ends up with the same failure (random package failure all the time)
manually accessing the failed package - guess what, it works great
tried to add { "useTaobaoRegistry": false } in the ~/.vimrc … no luck again
tried different network provider... no luck :(
nothing seems to be working …
npm -v
7.6.3
vue --version
#vue/cli 4.5.11
OS: Ubuntu 20.04.2 LTS 64bit
Gnome version: 3.36.8
what am I missing?
Looks like you do have issues with taobao in particular. From this github thread, we may try to not use it anymore.
For that, you can go to your ~/.vuerc and set "useTaobaoRegistry": false.
yarn config get registry is giving me https://registry.yarnpkg.com.

"Already up-to-date" and "Run 'yarn' to install missing dependencies"?

I've got a bit of a weird situation where "ember s" reports "Missing yarn packages" followed by a list of missing packages (two of them) and then "Run 'yarn' to install missing dependencies".
However when I run 'yarn' I get the message "success Already up-to-date" .
Any ideas how to get out of this circle ?
There's something I did immediately before this started which I guess is the source of my problem
Getting rid of an Add-on
I had installed an Ember Add-on which had generated problems so I decided to get rid of it . To do that I did the following :
git push -f origin 913c2dee12b0363df0664c9c63e0443abe0adb57:foo-bar
git reset --hard 913c2dee12b0363df0664c9c63e0443abe0adb57
yarn
yarn install
yarn clean
yarn install
yarn
Where 913c2dee12b0363df0664c9c63e0443abe0adb57 was the last commit before the unwanted add-on was installed. I don't think there was anything wrong in doing that but now I'm wondering.
Output of running Ember Server
When I run 'ember s' I get this output ...
DEPRECATION: ember-cli-babel 5.x has been deprecated. Please upgrade to at least ember-cli-babel 6.6. Version 5.2.8 located: foobar -> ember-cli-build-date -> ember-cli-babel
Missing yarn packages:
Package: #bower_components/jquery
* Specified: jquery/jquery-dist#
* Installed: (not installed)
Package: #bower_components/jstree
* Specified: vakata/jstree#^3.3.5
* Installed: (not installed)
Run `yarn` to install missing dependencies.
Stack Trace and Error Report: /tmp/error.dump.33c6e8c0abc9fc8abd355dc080268f9c.log
An error occurred in the constructor for ember-cli-dependency-checker at /media/mint/dev/foobar/node_modules/ember-cli-dependency-checker
Warning about ember-cli versions
I don't think this is relevant but just for completeness
Running the 'yarn' results in the following appearing in the output text ...
warning foobar#0.0.0: "dependencies" has dependency "ember-cli" with range "~3.8.1" that collides with a dependency in "devDependencies" of the same name with version "3.8"
EDIT: OK I was able to resolve this with the help of the comment by dwenzel ...
rm -rf node_modules && yarn
Although that actually led into a different issue which I'm almost certain was unrelated but I'll just mention it here in case it's useful to those who come after.
The execution of yarn initially failed with the following error ...
35238 verbose stack Error: node-sass#4.5.3 postinstall: `node scripts/build.js`
35238 verbose stack Exit status 1
35238 verbose stack at EventEmitter.<anonymous> (/home/rshea/.nvm/versions/node/v10.15.3/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
35238 verbose stack at EventEmitter.emit (events.js:189:13)
35238 verbose stack at ChildProcess.<anonymous> (/home/rshea/.nvm/versions/node/v10.15.3/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
35238 verbose stack at ChildProcess.emit (events.js:189:13)
35238 verbose stack at maybeClose (internal/child_process.js:970:16)
35238 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
35239 verbose pkgid node-sass#4.5.3
35240 verbose cwd /media/mintextra/dev/ddn-build-test-temp/foobar
35241 verbose Linux 3.13.0-169-generic
35242 verbose argv "/home/rshea/.nvm/versions/node/v10.15.3/bin/node" "/home/rshea/.nvm/versions/node/v10.15.3/bin/npm" "install"
35243 verbose node v10.15.3
35244 verbose npm v6.4.1
35245 error code ELIFECYCLE
35246 error errno 1
35247 error node-sass#4.5.3 postinstall: `node scripts/build.js`
35247 error Exit status 1
35248 error Failed at the node-sass#4.5.3 postinstall script.
35248 error This is probably not a problem with npm. There is likely additional logging output above.
35249 verbose exit [ 1, true ]
I was able to resolve that with ...
npm install node-sass#latest
npm install
... at which point ember s was able to run without errors.
It looks like you want to do yarn install --check-files to verify that the files have not been removed from node_modules. This avoids having to wipe out your node_modules file which can be time consuming.
https://classic.yarnpkg.com/en/docs/cli/install/#toc-yarn-install-check-files
https://github.com/yarnpkg/yarn/issues/2240
When yarn gets confused like this, you can usually fix things up by clearing out your node modules before installing:
rm -rf node_modules && yarn
The way I managed to get around this without deleting the entire node_modules directory was just to remove the module from topLevelPatterns within node_modules/.yarn-integrity.
Disclaimer: I don't know what the implications of this are but yarn's integrity was already questionable and it works so meh!?
You may also try to reinstall/download all packages with:
yarn install --force