npm installing hundreds of packages - npm

npm now installs a package-lock.json every time which seems to be a part of npm now. However, this simple package.json file when run with npm install installs over 300 packages. what could be going wrong and why are there 300+? even without the package-lock file when run (it then creates the lock file) and still gives 300+ packages
{
"name": "Package",
"version": "0.0.1",
"dependencies": {},
"devDependencies": {
"grunt": "^1.0.2",
"grunt-contrib-clean": "^1.1.0",
"grunt-contrib-compress": "^1.4.3",
"grunt-contrib-concat": "^1.0.1",
"grunt-contrib-connect": "^1.0.2",
"grunt-contrib-copy": "^1.0.0",
"grunt-contrib-cssmin": "^2.2.1",
"grunt-contrib-uglify": "^3.3.0",
"grunt-contrib-watch": "^1.1.0",
"grunt-sass": "^2.1.0",
"grunt-usemin": "^3.1.1"
}
}

Those packages depend on other packages, which depend on other packages, which depend on yet other packages…. A package manager resolves that entire dependency graph, which can easily grow into the hundreds from an innocent-looking initial list.
The package-lock.json file you mention can help you understand how this happens for your particular dependencies. Look for "requires". For example, in your setup, grunt-sass requires node-sass:
"grunt-sass": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/grunt-sass/-/grunt-sass-2.1.0.tgz",
"integrity": "sha512-XkexnQt/9rhReNd+Y7T0n/2g5FqYOQKfi2iSlpwDqvgs7EgEaGTxNhnWzHnbW5oNRvzL9AHopBG3AgRxL0d+DA==",
"dev": true,
"requires": {
"each-async": "1.1.1",
"node-sass": "4.9.0",
"object-assign": "4.1.1"
}
},
…which has 19 separate dependencies:
"node-sass": {
"version": "4.9.0",
...
"requires": {
"async-foreach": "0.1.3",
"chalk": "1.1.3",
"cross-spawn": "3.0.1",
"gaze": "1.1.3",
"get-stdin": "4.0.1",
"glob": "7.0.6",
"in-publish": "2.0.0",
"lodash.assign": "4.2.0",
"lodash.clonedeep": "4.5.0",
"lodash.mergewith": "4.6.1",
"meow": "3.7.0",
"mkdirp": "0.5.1",
"nan": "2.10.0",
"node-gyp": "3.6.2",
"npmlog": "4.1.2",
"request": "2.79.0",
"sass-graph": "2.2.4",
"stdout-stream": "1.4.0",
"true-case-path": "1.0.2"
},
...
}
…and just one of those, request, itself introduces another 20:
"request": {
"version": "2.87.0",
...
"requires": {
"aws-sign2": "0.7.0",
"aws4": "1.7.0",
"caseless": "0.12.0",
"combined-stream": "1.0.6",
"extend": "3.0.1",
"forever-agent": "0.6.1",
"form-data": "2.3.2",
"har-validator": "5.0.3",
"http-signature": "1.2.0",
"is-typedarray": "1.0.0",
"isstream": "0.1.2",
"json-stringify-safe": "5.0.1",
"mime-types": "2.1.18",
"oauth-sign": "0.8.2",
"performance-now": "2.1.0",
"qs": "6.5.2",
"safe-buffer": "5.1.2",
"tough-cookie": "2.3.4",
"tunnel-agent": "0.6.0",
"uuid": "3.2.1"
}
},
You can see how it gets to 300 so quickly.

Related

quasar failed on RUN quasar build

I want to build quasar project with command quasar build. The compile progress goes to 100% but I get the error after build:
App · ⚠️ 1 error encountered:
./node_modules/htmlparser2/lib/esm/index.js 59:9
Module parse failed: Unexpected token (59:9)
File was processed with these loaders:
* ./node_modules/#quasar/app/lib/webpack/loader.transform-quasar-imports.js
You may need an additional loader to handle the result of these loaders.
| return getFeed(parseDOM(feed, options));
| }
> export * as DomUtils from "domutils";
| // Old name for DomHandler
| export { DomHandler as DefaultHandler };
# ./node_modules/sanitize-html/index.js 1:19-41
# ./node_modules/vue-sanitize/dist/vue-sanitize.js
# ./src/boot/sanitize.ts
# ./.quasar/client-entry.js
# multi ./.quasar/client-entry.js
App · ⚠️ [FAIL] Build failed with 1 error. Check log above.
This is my package.json:
{
"name": "rfasdef",
"version": "0.0.1",
"description": "",
"productName": "",
"cordovaId": "",
"capacitorId": "",
"author": "",
"private": true,
"scripts": {
"lint": "eslint --ext .js,.ts,.vue --ignore-path .gitignore ./"
},
"dependencies": {
"#quasar/extras": "^1.0.0",
"#types/file-saver": "^2.0.5",
"#types/jwt-decode": "^3.1.0",
"axios": "^0.18.1",
"crypto-js": "^4.0.0",
"echarts": "^4.6.0",
"file-saver": "^2.0.5",
"formik": "^2.2.9",
"jalaali-js": "^1.2.3",
"jwt-decode": "^3.1.2",
"lodash": "^4.17.21",
"moment": "^2.29.4",
"moment-duration-format": "^2.3.2",
"moment-jalaali": "^0.9.6",
"num2persian": "^3.2.2",
"persianjs": "^0.4.0",
"qjdatetime": "^1.1.0",
"quasar": "^1.9.5",
"sass": "^1.45.0",
"vue-echarts": "^5.0.0-beta.0",
"vue-i18n": "^8.0.0",
"vue-persian-datetime-picker": "^2.10.1",
"vue-sanitize": "^0.2.0"
},
"devDependencies": {
"#quasar/app": "^1.6.2",
"#quasar/quasar-app-extension-dotenv": "^1.0.1",
"#types/lodash": "^4.14.150",
"#types/moment-jalaali": "^0.7.5",
"#types/node": "^10.17.15",
"#typescript-eslint/eslint-plugin": "^2.17.0",
"#typescript-eslint/parser": "^2.17.0",
"babel-eslint": "^10.0.1",
"eslint": "^6.8.0",
"eslint-config-standard": "^14.1.0",
"eslint-loader": "^3.0.3",
"eslint-plugin-import": "^2.14.0",
"eslint-plugin-node": "^11.0.0",
"eslint-plugin-promise": "^4.0.1",
"eslint-plugin-standard": "^4.0.0",
"eslint-plugin-vue": "^6.1.2",
"vue-property-decorator": "^8.3.0",
"vuex-class": "^0.3.2",
"vuex-module-decorators": "^0.11.0"
},
"engines": {
"node": ">= 10.18.1",
"npm": ">= 6.13.4",
"yarn": ">= 1.21.1"
},
"browserslist": [
"last 1 version, not dead, ie >= 11"
],
"resolutions": {
"#babel/parser": "7.7.5"
}
}
It was working yesterday but today it's not working!
I tried installing packages with both yarn and npm. But its not working.
Try this:
Deleting package-lock.json and folder node_modules
then run npm install or yarn install
then run quasar build

Why does package.lock file requirements section change when the related package stays the same?

How is this package-lock.json change possible:
Before:
...
"chokidar": {
"version": "1.7.0",
"resolved": "https://registry.npmjs.org/chokidar/-/chokidar-1.7.0.tgz",
"integrity": "sha1-eY5ol3gVHIB2tLNg5e3SjNortGg=",
"requires": {
"anymatch": "1.3.2",
"async-each": "1.0.3",
"glob-parent": "2.0.0",
"inherits": "2.0.4",
"is-binary-path": "1.0.1",
"is-glob": "2.0.1",
"path-is-absolute": "1.0.1",
"readdirp": "2.2.1"
}
},
...
After:
...
"chokidar": {
"version": "1.7.0",
"resolved": "https://registry.npmjs.org/chokidar/-/chokidar-1.7.0.tgz",
"integrity": "sha1-eY5ol3gVHIB2tLNg5e3SjNortGg=",
"requires": {
"anymatch": "^1.3.0",
"async-each": "^1.0.0",
"fsevents": "^1.0.0",
"glob-parent": "^2.0.0",
"inherits": "^2.0.1",
"is-binary-path": "^1.0.0",
"is-glob": "^2.0.0",
"path-is-absolute": "^1.0.0",
"readdirp": "^2.0.0"
}
},
...
This happens inside a docker container -- so platform is not an issue. integrity field matches -- so WTF. Required versions change (that could be npm audit) but fsevents as a requirement is new while the package has not changed.
Can anybody give me a clue?

Using 'express' inside Electron main thread, cannot find module

I'm trying to use express library in main.js file. It works fine on my dev build, but when I package the app I get
Error: Cannot find module 'express'
I'm not quite sure how electron main thread works, is it packaged separately by some other build tool, and do I need to define(include) package manually? My app is packaged by webpack, and I have included libraries in package.json. Every sample I have found just includes express library and moves on, I can't find any additional steps for this.
package.json
{
"name": "basic-electron-react-boilerplate",
"version": "0.7.0",
"description": "Minimal and modern react+electron+webpack boilerplate",
"author": "Phillip Barbiero",
"homepage": "https://github.com/pbarbiero/basic-electron-react-boilerplate",
"repository": {
"type": "git",
"url": "https://github.com/pbarbiero/basic-electron-react-boilerplate.git"
},
"build": {
"appId": "your.id",
"mac": {
"category": "your.app.category.type"
}
},
"license": "MIT",
"main": "main.js",
"scripts": {
"prod": "webpack --config webpack.build.config.js && electron --noDevServer .",
"test": "node test.js",
"dev": "webpack-dev-server --hot --host 0.0.0.0 --config=./webpack.dev.config.js",
"build": "webpack --config webpack.build.config.js",
"package": "webpack --config webpack.build.config.js",
"postpackage": "electron-packager ./ --out=./builds",
"pack": "electron-builder --dir",
"dist": "electron-builder"
},
"devDependencies": {
"#babel/core": "^7.1.2",
"#babel/plugin-proposal-class-properties": "^7.1.0",
"#babel/plugin-proposal-export-default-from": "^7.0.0",
"#babel/plugin-proposal-object-rest-spread": "^7.0.0",
"#babel/preset-env": "^7.1.0",
"#babel/preset-es2017": "^7.0.0-beta.53",
"#babel/preset-react": "^7.0.0",
"babel-core": "^6.24.1",
"babel-loader": "^8.0.4",
"babel-plugin-lodash": "^3.3.4",
"babel-preset-es2015": "^6.24.1",
"babel-preset-es2016": "^6.24.1",
"babel-preset-react": "^6.24.1",
"babel-preset-stage-2": "^6.24.1",
"babili-webpack-plugin": "^0.1.2",
"body-parser": "^1.18.3",
"css-loader": "^0.28.1",
"electron": "^1.7.8",
"electron-builder": "^20.28.4",
"electron-packager": "^9.1.0",
"express": "^4.16.4",
"extract-text-webpack-plugin": "^3.0.1",
"file-loader": "^1.1.5",
"html-webpack-plugin": "^2.28.0",
"react": "^16.0.0",
"react-dom": "^16.0.0",
"request": "^2.88.0",
"style-loader": "^0.19.0",
"webpack": "^3.6.0",
"webpack-dev-server": "^2.4.5"
},
"dependencies": {
"electron-fetch": "^1.2.1",
"electron-require": "^0.3.0",
"express": "^4.16.4",
"faker": "^4.1.0",
"lodash": "^4.17.11",
"moment": "^2.22.2",
"rc-time-picker": "^3.4.0",
"react-async-script-loader": "^0.3.0",
"react-dropdown": "^1.6.2",
"react-places-autocomplete": "^7.2.0",
"request": "^2.88.0",
"resolve-url-loader": "^3.0.0",
"semantic-ui-css": "^2.4.1",
"semantic-ui-react": "^0.82.5",
"styled-components": "^4.0.2"
}
}
You can run Express server inside Electron. Here is a sample repo for running express inside Electron.
You can fork a child process to run express app as follows
app = require("electron").remote.app),
node = require("child_process").fork(
`${app.getAppPath()}/express-app/bin/www`,
[],
{
stdio: ["pipe", "pipe", "pipe", "ipc"]
});
The express app used here is a generated one using express-generator.
The problem with your approach is that you are maintaining a single package.json file for both Electron and Express. Checkout Electron-React-Boilerplate, here i have two separate npm installation locations one is for Electron alone and other is for React stuffs. Electron-packager and Electron-builder works using this pattern. This is the reason why your process on main thread throws
Error: Cannot find module 'express'

NPM does not install dependencies

There are a handful of other questions on here, but none that seem to be the same problem.
Running npm install on a preexisting package.json, pulled in from git, does all of its processing, creates a node_modules directory, then ends without installing the modules. node_modules is empty, and there were no errors (just a couple warnings about deprecated modules). The entire directory is owned by the active user.
After a bunch of messing around, I've found that if I remove all of the devDependencies, the normal dependencies will install as expected.
OS: Ubuntu 16.04
Node: 6.9.1
NPM: 3.10.8
package.json:
{
"name": "project",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"start": "nodemon --ignore src --ignore public --ignore views",
"test": "node ../app.js"
},
"author": "xxx",
"license": "ISC",
"devDependencies": {
"autoprefixer": "^6.5.1",
"babel-preset-es2015": "^6.18.0",
"babelify": "^7.3.0",
"browserify": "^13.1.1",
"chalk": "^1.1.3",
"event-stream": "^3.3.4",
"fs-extra": "^1.0.0",
"git-guppy": "^1.2.1",
"glob": "^7.1.1",
"gulp": "^3.9.1",
"gulp-cssnano": "^2.1.2",
"gulp-duration": "0.0.0",
"gulp-filter": "^4.0.0",
"gulp-function": "^2.2.0",
"gulp-git": "^1.12.0",
"gulp-livereload": "^3.8.1",
"gulp-notify": "^2.2.0",
"gulp-postcss": "^6.2.0",
"gulp-sourcemaps": "^2.2.0",
"gulp-uglify": "^2.0.0",
"gulp-util": "^3.0.7",
"guppy-pre-commit": "^0.4.0",
"postcss-cssnext": "^2.8.0",
"postcss-math": "0.0.5",
"precss": "^1.4.0",
"q": "^1.4.1",
"utils-merge": "^1.0.0",
"vinyl-buffer": "^1.0.0",
"vinyl-source-stream": "^1.1.0",
"watchify": "^3.7.0"
},
"dependencies": {
"axios": "^0.15.2",
"dotenv": "^2.0.0",
"express": "^4.14.0",
"fs": "0.0.1-security",
"git-rev-sync": "^1.8.0",
"pug": "^2.0.0-beta6",
"yamljs": "^0.2.8"
}
}
This is running on a basic DO server, the same distribution I've spun up for a dozen other projects, with no problem. The only thing I can think of is if there's something within one of the devDependencies that is clogging things up. How to debug this?
It appears that "fs" is not a valid npm package. I came to this conclusion after running this command:
npm bugs fs
This ended up displaying bugs filed against a npm/security-holder package. It appears that the "fs" on NPM may have been malicious.
If you want node's fs package, you don't need an entry in your package.json for that, fs is built-in.

npm downloading hundreds of packages?

I'm working on an Angular2 (currently on RC1) project in Visual Studio 2015; and straining my understanding of NPM.
Looking in the Dependencies/npm folder, and /node_modules there are hundreds of npm packages. Most of which are labeled as extraneous by Visual Studio (in Dependencies/npm)
I went through and removed all the extraneous packages manually (npm prune did nothing).
I apparently accidentally removed a non-extraneous package, which I attempted to restore. NPM went ahead and re-downloaded all 500+ extraneous packages again...
What am I missing?
My package.json:
{
"name": "MySite",
"version": "0.0.1",
"description": "My Site",
"repository": "./src",
"license": "UNLICENSED",
"dependencies": {
"#angular/common": "^2.0.0-rc.1",
"#angular/compiler": "2.0.0-rc.1",
"#angular/core": "^2.0.0-rc.1",
"#angular/http": "2.0.0-rc.1",
"#angular/platform-browser": "2.0.0-rc.1",
"#angular/router": "^2.0.0-rc.1",
"es6-promise": "3.1.2",
"es6-shim": "0.35.0",
"jquery": "2.2.2",
"jquery-validation": "1.15.0",
"reflect-metadata": "0.1.2",
"rxjs": "5.0.0-beta.6",
"systemjs": "0.19.27",
"zone.js": "^0.6.12"
},
"devDependencies": {
"gulp": "3.9.1",
"gulp-cssmin": "0.1.7",
"gulp-concat": "2.6.0",
"gulp-sass": "2.3.1",
"typings": "0.8.1",
"typescript": "^1.8.10",
"null-loader": "^0.1.1",
"raw-loader": "^0.5.1",
"style-loader": "^0.13.1",
"ts-loader": "^0.8.1",
"webpack": "1.13.0",
"webpack-dev-server": "1.14.1",
"webpack-merge": "0.12.0"
},
"scripts": {
"postinstall": "typings install"
}
}