RollupJS - JavaScript heap out of memory - rollup

I know I have a lot of imports of .js files that are data (400K lines) so I suspect that is the problem. Anything that I can do about this?
workbench#1.0.0 build D:\wwwroot\Workbench
rollup -c
scripts/Main.js → bundle.js...
<--- Last few GCs --->
[11428:000001AAB15A09D0] 34674 ms: Mark-sweep 1509.9 (1568.1) -> 1509.8 (1569.1) MB, 1151.5 / 0.0 ms allocation failure GC in old space requested
[11428:000001AAB15A09D0] 35753 ms: Mark-sweep 1509.8 (1569.1) -> 1509.8 (1538.1) MB, 1078.9 / 0.0 ms last resort GC in old space requested
[11428:000001AAB15A09D0] 36888 ms: Mark-sweep 1509.8 (1538.1) -> 1509.8 (1538.1) MB, 1135.4 / 0.0 ms last resort GC in old space requested
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0000018AA5225729 <JSObject>
1: split(this=000002F007782201 <Very long string[7147106]>)
2: guessIndent(aka guessIndent) [D:\wwwroot\Workbench\node_modules\rollup\dist\rollup.js:639] [bytecode=00000237F3924959 offset=11](this=000002F71B0022D1 <undefined>,code=000002F007782201 <Very long string[7147106]>)
3: new MagicString$1 [D:\wwwroot\Workbench\node_modules\rollup\dist\rollup.js:865] [bytecode=00000237F3923C...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
npm ERR! code ELIFECYCLE
npm ERR! errno 134
npm ERR! workbench#1.0.0 build: `rollup -c`
npm ERR! Exit status 134
npm ERR!
npm ERR! Failed at the workbench#1.0.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\ysg4206\AppData\Roaming\npm-cache\_logs\2017-11-22T21_53_17_670Z-debug.log

Basically, one needs to tell node to increase the heap. E.g.
node --max-old-space-size=4096 /path/to/rollup -c
I found this npm utility that adds heap for one or all of your node binary scripts.
https://www.npmjs.com/package/increase-memory-limit
And that solved the problem. Thank you.

Related

npm install throws ECONNRESET error but only for certain packages

I'm working behind a corporate proxy. I have npm configured like this:
$ npm config get
...
https_proxy = "http://proxy.my-domain.com:8080"
https-proxy = "http://proxy.my-domain.com:8080"
proxy = "http://proxy.my-domain.com:8080"
...
With these settings, I can install some packages fine, but not others. For example $ npm i react works perfectly fine, whereas installing #babel/core throws an ECONNRESET error.
$ npm i #babel/core
npm ERR! code ECONNRESET
npm ERR! syscall read
npm ERR! errno -54
npm ERR! network read ECONNRESET
...
Strangely, I'm able to install the package with yarn (which is configured for the proxy the same way npm is), but it too tells me that I have network issues (even though it successfully installs the package 🤔)
$ yarn add #babel/core
...
✨ Done in 2.99s.
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
I can't tell why it works with yarn, but not with npm. Here's what the npm debug log looks like:
73 silly tarball no local data for #jridgewell/sourcemap-codec#https://registry.npmjs.org/#jridgewell/sourcemap-codec/-/sourcemap-codec-1.4.14.tgz. Extracting by manifest.
74 silly tarball no local data for #jridgewell/trace-mapping#https://registry.npmjs.org/#jridgewell/trace-mapping/-/trace-mapping-0.3.17.tgz. Extracting by manifest.
75 silly tarball no local data for #jridgewell/set-array#https://registry.npmjs.org/#jridgewell/set-array/-/set-array-1.1.2.tgz. Extracting by manifest.
76 verbose stack Error: read ECONNRESET
76 verbose stack at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)
77 verbose cwd /Users/justinsmith/Dev/test-npm
78 verbose Darwin 21.4.0
79 verbose node v18.9.0
80 verbose npm v8.19.1
81 error code ECONNRESET
82 error syscall read
83 error errno -54
Anyone have any ideas why this is happening?
tl;dr adding this to my ~./npmrc provided a temporary fix:
# This is what I already had set
proxy=http://proxy.my-domain.com:8080/
https-proxy=http://proxy.my-domain.com:8080/
https_proxy=http://proxy.my-domain.com:8080/
# This is what I added
strict-ssl=false
registry=http://registry.npmjs.org/
maxsockets=3
This forces npm to make requests with http instead of https and limits the number of parallel requests to three.
One reason this works is because evidently the corporate proxy was scanning files for malware before forwarding them to the client. Evidently npm making too many requests in parallel over TLS overwhelms the malware scanner which eventually refuses some connections.
Yarn on the other hand will retry a couple of times whereas npm just throws an error and gives up.
A longer-term solution would be to get your IT department to add npm to a malware scanning exclusion list.
Full credit goes to this blog post, which provided these helpful solutions.

npm install fails with 405 Method Not Allowed

I am currently working on upgrade some projects to use node 18 with npm 8, from node 12 with npm 6.
We're using github as npm repository for our private packages.
Now I am getting this error on npm install:
npm ERR! code E405
npm ERR! 405 Method Not Allowed - GET https://npm.pkg.github.com/<our-github-organisation>/#types/mime/-/mime-3.0.1.tgz
In the error log:
456 verbose stack HttpErrorGeneral: 405 Method Not Allowed - GET https://npm.pkg.github.com/<our-github-organisation>/#types/mime/-/mime-3.0.1.tgz
456 verbose stack at /home/cclausen/.nvm/versions/node/v18.7.0/lib/node_modules/npm/node_modules/npm-registry-fetch/lib/check-response.js:93:15
456 verbose stack at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
457 verbose statusCode 405
458 verbose pkgid #types/mime#https://npm.pkg.github.com/<our-github-organisation>/#types/mime/-/mime-3.0.1.tgz
This does not happen on every package, only on a few select ones, I can't see pattern.
None of the affected patterns our ours, just random packages.
Why would this happen?
It seems the content of the .npmrc file used in the repository to specifc from where to get packages has changed, updating that file to comply with the newest information from githubs documentation has fixed this issue.
So the new content for me is:
#OWNER:registry=https://npm.pkg.github.com
Where OWNER is my organisation.

Vue create “projectName” ERR_SOCKET_TIMEOUT - fails all the time no matter what - taobao

I am getting a bit confused about what’s been happening to me recently.
Whenever I want to create a new vue project, it fails with:
Vue CLI v4.5.11
✨ Creating project in /home/admin/Documents/dev/my_projects/vThesaurus/thesaurus.
⚙️ Installing CLI plugins. This might take a while...
npm ERR! code ERR_SOCKET_TIMEOUT
npm ERR! errno ERR_SOCKET_TIMEOUT
npm ERR! request to https://registry.npm.taobao.org/jsonfile failed, reason: Socket timeout
npm ERR! A complete log of this run can be found in:
npm ERR! /home/admin/.npm/_logs/2021-03-17T06_40_10_793Z-debug.log
ERROR command failed: npm install --loglevel error --legacy-peer-deps
another packages from prev attempts:
request to https://registry.npm.taobao.org/core-js-compat failed, reason: Socket timeout
request to https://cdn.npm.taobao.org/micromatch/-/micromatch-4.0.2.tgz failed, reason: Socket timeout
request to https://registry.npm.taobao.org/querystring-es3 failed, reason: Socket timeout
request to https://registry.npm.taobao.org/file-loader failed, reason: Socket timeout
… and many more
Log: (last part)
4675 timing command:install Completed in 378261ms
4676 verbose type system
4677 verbose stack FetchError: request to https://registry.npm.taobao.org/jsonfile failed, reason: Socket timeout
4677 verbose stack at ClientRequest.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:97:14)
4677 verbose stack at ClientRequest.emit (events.js:315:20)
4677 verbose stack at TLSSocket.socketErrorListener (_http_client.js:463:9)
4677 verbose stack at TLSSocket.emit (events.js:327:22)
4677 verbose stack at emitErrorNT (internal/streams/destroy.js:100:8)
4677 verbose stack at emitErrorCloseNT (internal/streams/destroy.js:68:3)
4677 verbose stack at processTicksAndRejections (internal/process/task_queues.js:84:21)
4678 verbose cwd /home/admin/Documents/dev/my_projects/vThesaurus/thesaurus
4679 verbose Linux 5.4.0-67-generic
4680 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "--loglevel" "error" "--legacy-peer-deps"
4681 verbose node v14.2.0
4682 verbose npm v7.6.3
4683 error code ERR_SOCKET_TIMEOUT
4684 error errno ERR_SOCKET_TIMEOUT
4685 error request to https://registry.npm.taobao.org/jsonfile failed, reason: Socket timeout
4686 verbose exit 1
the weirdest part is that it always fails on a different package …
I tried:
to remove nodejs and vue, reinstall again but no luck …
to increase timeout in the ~/.npmrc but no luck again … (I don’t have slow internet connection - ping 3ms, d: 96Mbps, u: 9.6Mbps)
nothing I found on the internet seems to be working …
changing IPs (nordvpn) to reach to diff servers (CDN) but no luck
vue2 and vue3 ends up with the same failure (random package failure all the time)
manually accessing the failed package - guess what, it works great
tried to add { "useTaobaoRegistry": false } in the ~/.vimrc … no luck again
tried different network provider... no luck :(
nothing seems to be working …
npm -v
7.6.3
vue --version
#vue/cli 4.5.11
OS: Ubuntu 20.04.2 LTS 64bit
Gnome version: 3.36.8
what am I missing?
Looks like you do have issues with taobao in particular. From this github thread, we may try to not use it anymore.
For that, you can go to your ~/.vuerc and set "useTaobaoRegistry": false.
yarn config get registry is giving me https://registry.yarnpkg.com.

Unable to install NPM packages due to path length constraint using NPM 5.X

I'm running Windows 10, NPM v5.5.1, and node v8.9.1.
Edit: I have since updated to node v8.9.4 and npm v5.6.0. The problem remains unchanged.
I set up my development environment on this machine about 2 months ago. I'm using Angular CLI to build my client apps.
Everything was going fine. I added a few packages via NPM without a problem, such as Gulp and ngx-bootstrap. Then I started to run into errors on some of my package installs.
The errors are:
npm ERR! code PATH_LENGTH_EXCEEDED
npm ERR! errno PATH_LENGTH_EXCEEDED
npm ERR! request to https://registry/npmjs.org/{{package}} failed, reason: path length constraint exceeded
npm ERR! A complete log of this run can be found in:
npm ERR! {{npm_cache path}}\_logs\2018-02-08T14_43_40_856Z-debug.log
The contents of that log file are:
0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli 'C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli 'install',
1 verbose cli 'ngx-cookie-service',
1 verbose cli '--save' ]
2 info using npm#5.5.1
3 info using node#v8.9.1
4 verbose npm-session d55be12849534a0a
5 silly install loadCurrentTree
6 silly install readLocalPackageData
7 silly fetchPackageMetaData error for ngx-cookie-service#latest request to https://registry.npmjs.org/ngx-cookie-service failed, reason: path length constraint exceeded
8 verbose type system
9 verbose stack FetchError: request to https://registry.npmjs.org/ngx-cookie-service failed, reason: path length constraint exceeded
9 verbose stack at ClientRequest.req.on.err (C:\Program Files\nodejs\node_modules\npm\node_modules\pacote\node_modules\make-fetch-happen\node_modules\node-fetch-npm\src\index.js:68:14)
9 verbose stack at emitOne (events.js:116:13)
9 verbose stack at ClientRequest.emit (events.js:211:7)
9 verbose stack at TLSSocket.socketErrorListener (_http_client.js:387:9)
9 verbose stack at emitOne (events.js:116:13)
9 verbose stack at TLSSocket.emit (events.js:211:7)
9 verbose stack at emitErrorNT (internal/streams/destroy.js:64:8)
9 verbose stack at _combinedTickCallback (internal/process/next_tick.js:138:11)
9 verbose stack at process._tickCallback (internal/process/next_tick.js:180:9)
10 verbose cwd C:\Development\test
11 verbose Windows_NT 10.0.15063
12 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "ngx-cookie-service" "--save"
13 verbose node v8.9.1
14 verbose npm v5.5.1
15 error code PATH_LENGTH_EXCEEDED
16 error errno PATH_LENGTH_EXCEEDED
17 error request to https://registry.npmjs.org/ngx-cookie-service failed, reason: path length constraint exceeded
18 verbose exit [ 1, true ]
I researched this, and found the Windows 10 file path 260 character limitation. I thought that npm version 3.1.0 should have fixed this issue, but I decided to followed the instructions in this answer, because I'm still running into this issue despite using NPM v5.5.1.
I found that my version of Windows was prior to the roll out of the feature that lets you disable this limitation. I updated to Version 1703 (OS Build 15063.483), and to enable Win32 long paths.
Yet I'm still getting the error, and unable to install a variety of npm packages.
In testing to see if it was an issue with packages that have a lot of dependencies, I tried installing or reinstalling a couple of other packages.
Some fail with the errors above. Others succeed with the following warnings:
npm WARN Unexepcted warming for https://registry.npmjs.org/: Miscellaneous Warning PATH_LENGTH_EXCEEDED: request to https://registry.npmjs.org/{{package name}} failed, reason: path length constraint exceeded
npm WARN Using stale package data from https://registry.npmjs.org/ due to a request error during validation
Packages throwing the error include:
jslint
ngx-cookie-service
chart.js
angular-cli
Packages that throw the warnings include:
font-awesome
#angular/cli
moment
ngx-bootstrap
bootstrap
jasmine
I suspect that something has changed, either in a policy on my workstation, or in our network security, that is causing this. All of the packages that succeed seem to be packages that I had previously installed, either as a direct install, or as a dependency of #angular/cli (one of the first packages I installed). I'm assuming that the stale package data warning indicates that whatever problem I have with the path length is preventing updated data being retrieved, and therefore npm is falling back to cached data from previous installations of the selected package and its dependencies.
So now I'm stumped as to how to diagnose exactly where the issue is happening. If its a security policy that changed, then I need to be able to describe exactly what is being blocked so that I can work with our security department to fix this. But the only information I can find about the PATH_LENGTH_EXCEEDED errors lead me to old information about versions of npm prior to 3.1.0.
How can I identify the source of this problem?
After additional searching, I found references to path length exceeded with regards to SSL/TLS certificate authorities. I tested this issue by changing the npm registry to use http instead of https (note: this is a temporary step only; I have turned https back on once I verified the issue):
npm config set registry http://registry.npmjs.org/
This allowed me to install the problematic packages.
It seems the underlying cause is SSL Inspection performed by our network security team. The decryption, inspection, and re-encryption process, and subsequent changes in the Issuing Authority on the certificate, apparently triggers this error.

npm install fails to exec postinstall script bower install

I snooped around the net for more than 48 hours & failed to solve this.
When I run the command 'npm install', it fails to execute the postinstall script 'bower install'. So, I I did an explicit bower install by using:
$npm install -g bower
$bower install
After this, the 'app/bower_components' folder got populated alright. But, again when I try 'npm install', same issue. Unable to proceed further.
I performed the following & did not succeed in installing:
Git is installed & its path added in the PATH environment variable. And then run 'npm install'. Even restarted the command prompt in order for the PATH variable to reflect.
Run 'npm install restify', though I don't know why the installation has to be made a REST service.
I used this, as it was one of the suggestions I came across.
In some post, it was mentioned that it could be due to firewall settings. Disabled the same & tried as well.
Changed all the files under my project 'angular-phonecat' to have read-write permissions. Did this by unchecking 'Read-only' option for the entire folder's contents.
Run Git Bash as admin & tried 'npm install'.
Please help me run this app. :(
Find below my logs:
600 info install angular-phonecat#0.0.0
601 info postinstall angular-phonecat#0.0.0
602 verbose unsafe-perm in lifecycle true
603 info angular-phonecat#0.0.0 Failed to exec postinstall script
604 error angular-phonecat#0.0.0 postinstall: `bower install`
604 error spawn ENOENT
605 error Failed at the angular-phonecat#0.0.0 postinstall script.
605 error This is most likely a problem with the angular-phonecat package,
605 error not with npm itself.
605 error Tell the author that this fails on your system:
605 error bower install
605 error You can get their info via:
605 error npm owner ls angular-phonecat
605 error There is likely additional logging output above.
606 error System Windows_NT 6.2.9200
607 error command "g:\\Workspace\\AngularProjects\\angular-phonecat\\node.exe" "g:\\Workspace\\AngularProjects\\angular-phonecat\\node_modules\\npm\\bin\\npm-cli.js" "install"
608 error cwd g:\Workspace\AngularProjects\angular-phonecat
609 error node -v v0.10.32
610 error npm -v 1.4.28
611 error syscall spawn
612 error code ELIFECYCLE
613 error errno ENOENT
614 verbose exit [ 1, true ]
You will need to provide more information in order to get an answer to this question. It looks like angular-phonecat is your own module that you created -- can you please publish its package.json as a gist https://gist.github.com and attach the link here?
Also, your npm is rather old (and was even at the time you asked this question). I would recommend updating to the latest version of npm, which at this writing is 2.2.0.
To update npm on Windows, follow the instructions here: https://github.com/npm/npm/wiki/Troubleshooting#upgrading-on-windows
Thanks!
try npm-check module to test if any component is outdated