All our server projects contain a git submodule folder (let's say modules), which contains our custom modules/components.
Such module dependencies are installed locally (see serverApp/package.json) so that we don't have to include the whole submodule folder to the final rpm. What I'm having trouble with is limiting the number of files included in node_modules.
The submodule structure looks like the following:
modules
|--loader
|--dist => compiled js files here that are created when installing the module
|--ts => contains typescript files that shouldn't be included in node_modules
|--package.json
|--tsconfig.json
|--more modules
|--.gitignore
Adding an .npmignore file inside modules/loader doesn't seem to help as the whole folder is copied.
modules/loader/tsconfig.json:
{
"compilerOptions": {
"target": "es6",
"module": "commonjs",
"declaration": true,
"outDir": "./dist",
"strict": true
}
}
modules/loader/package.json:
{
"name": "loader",
"version": "1.2.0",
"private": true,
"description": "",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"scripts": {
"preinstall": "npm run build",
"build": "../../node_modules/typescript/bin/tsc",
"test": "echo \"Error: no test specified\" && exit 1"
},
"dependencies": {
"#types/lodash": "^3.9.3",
"#types/nomnom": "0.0.28",
"#types/yamljs": "^0.2.30",
"lodash": "^3.9.3",
"nomnom": "^1.8.1",
"yamljs": "^0.2.1"
},
"devDependencies": {
"typescript": "~2.3.4"
}
}
serverApp/package.json:
{
"name": "my-server-app",
"version": "2.3.0",
"description": "",
"main": "myServerApp.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"license": "private",
"dependencies": {
"loader": "file:modules/loader"
},
"devDependencies": {
"grunt": "^0.4.5",
"grunt-cli": "^0.1.13"
}
}
I'm not sure if it has to do with the fact that we have a .gitignore file or because the module is not published and installed locally.
npm version => 5.3.0
EDIT
Doesn't work with specifying the "files" in modules/loader/package.json either
As after checkout the issue I found out below useful points which need to mention out :
We use a .npmignore file to keep stuff out of your package.
If there's no .npmignore file, but there is a .gitignore file, then npm will ignore the stuff matched by the .gitignore file.
If you want to include something that is excluded by your .gitignore file, you can create an empty .npmignore file to override it.
Like git, npm looks for .npmignore and .gitignore files in all subdirectories of your package, not only the root directory.
Similar to .gitignore file .npmignore also follow these rules
Blank lines or lines starting with # are ignored & Standard glob patterns work.
You can end patterns with a forward slash / to specify a directory.
You can negate a pattern by starting it with an exclamation point !.
By default, the following paths and files are ignored, so there's no need to add them to .npmignore explicitly
Additionally, everything in node_modules is ignored, except for bundled dependencies. npm automatically handles this for you, so don't bother adding node_modules to .npmignore.
Testing whether your .npmignore or files config works
If you want to double check that your package will include only the files you intend it to when published, you can run the npm pack command locally which will generate a tarball in the working directory, the same way it does for publishing.
And you can also checkout same issue here Consider methodologies for reducing the number of files within node_modules #14872
Thanks.
Did you checked with node 0.6.13 / npm 1.1.9? This issue is common in npm 1.1.4 .
have a look on this link
You mentioned that you do not want to include "whole submodule" into the "final rpm", by which I presume the package you will finally prepare. I reproduced similar setup and added a '.npmignore' to ignore "submodule" which I installed using npm install --save ./task_in where 'task_in' was my module kept along side of the main package's('task') 'package.json'.
And when the final package was prepared using npm pack while being in the 'task' folder, I got a package(a tar file) without the folder('task_in') as indicated in the '.npmignore'.
While working, though, I found that the module 'task_in' folder was copied to 'node_modules' which is automatically not included in the final package( refer here). Also, while the package is prepared, ".gitignore" is over-ridden by ".npmignore".
So, this is my "two cents" and I hope it helps you.
Related
My top package.json:
{
"name": "foo",
"version": "0.0.0",
"private": true,
"workspaces": [
"apps/*",
"packages/*"
],
"devDependencies": {
"#foo/eslint-config": "*"
},
"engines": {
"npm": ">=7.0.0",
"node": ">=14.0.0"
},
"dependencies": {},
"packageManager": "npm#8.18.0"
}
and I have a package in packages/#foo/eslint-config.
However, when I do npm install, I get an error saying that #foo/eslint-config is not in the registry.
I am assuming that I have either wrong directory structure.
Figured it out.
The package should have gone directly to packages/eslint-config directory.
The package name still needs to have the scope, i.e. #foo/eslint-config.
It appears that workspaces do not use the same convention of nesting scoped packages under a sub-directory as node_modules do.
It also appears that the folder name has no significance, as long as it is directly in the directory defined in workspaces configuration and has the correct package name.
Alternatively, you can also just update your workspaces configuration to read from packages/#foo/*.
For a library written in ES6/7, I want to compile (to ES5) the library to a dist/ folder. I also want to run the tests (written in ES6/7) for this lib.
My dev dependencies look like this (package.json):
"devDependencies": {
"#babel/cli": "^7.4.4",
"#babel/core": "^7.4.5",
"#babel/preset-env": "^7.4.5",
"#babel/register": "^7.4.4",
"chai": "^4.2.0",
"mocha": "^6.1.4",
"sinon": "^7.3.2"
},
My build and test scripts looks like this (package.json):
"scripts": {
"test": "mocha --require #babel/register",
"build": "babel src -d dist --presets=#babel/preset-env"
},
Running npm run build works well. The dist/ folder gets populated with transpiled files.
Running npm run test does not seem to work - this is my problem.
> mocha --require #babel/register
/Users/dro/Repos/lib/node_modules/yargs/yargs.js:1163
else throw err
^
ReferenceError: regeneratorRuntime is not defined
Initially I got an import error, which was resolved by adding .babelrc file.
Below is my .babelrc file content.
{
"presets": ["#babel/preset-env"]
}
I was reading about regeneratorRuntime and it got me to this link about babel-polyfill where they explain I shouldn't need that polyfill.
This will emulate a full ES2015+ environment (no < Stage 4 proposals) and is intended to be used in an application rather than a library/tool.
What is needed to set this up properly?
I am not using webpack.
Testing in ES6 with Mocha and Babel 7. Look here: https://dev.to/bnorbertjs/my-nodejs-setup-mocha--chai-babel7-es6-43ei or http://jamesknelson.com/testing-in-es6-with-mocha-and-babel-6/
npm install --save #babel/runtime
npm install --save-dev #babel/plugin-transform-runtime
And, in .babelrc, add:
{
"presets": ["#babel/preset-env"],
"plugins": [
["#babel/transform-runtime"]
]
}
Look at the project documentation:
npm install --save-dev babel-register
In your package.json file make the following changes:
{
"scripts": {
"test": "mocha --require babel-register"
}
}
Some features will require a polyfill:
npm install --save-dev babel-polyfill
{
"scripts": {
"test": "mocha --require babel-polyfill --require babel-register"
}
}
Below steps are for applying Babel transformations & core-js polyfills for your tests file:
💡 All transformations are only done per current environment, so only what is needed to be transpiled/polyfilled, will be. Target environments may be defined from a .browserslist file or as a property in package.json file. (read more here)
Step 1: Install packages:
#babel/core (read why)
#babel/preset-env (read why)
#babel/register (read why)
core-js (read why)
Note that #babel/polyfill exists and uses core-js under the hood. However, it was deprecated in favor of using core-js directly.
Step 2: Create a Babel configuration file babel.config.js
(used to be .babelrc.js or a .json file).
Create this file at the root-level of your code.
The most basic configuration (for just testing and not bundling) would look like this:
module.exports = {
presets: [
['#babel/preset-env', {
"corejs": "3.26",
"useBuiltIns": "usage"
}],
};
corejs - This is the polyfills library and should be specified with the minor version, otherwise x.0 will be used.
It is needed when testing code on rather "old" Node versions, which do not support all of the language methods. This ofc depends on your own usage of such javascript methods. (for example String.prototype.replaceAll).
useBuiltIns - must be set in order for the corejs polyfills to be applied. Read about it in the official docs.
By default, #babel/preset-env will compile your code for the current environment, but you can specify a different environment by setting the "targets" option in the configuration.
Ofc, you can add more presets like #babel/preset-react for example, if your code it written in React, or any other plugins which are specifically needed for your code.
Step 3: Connect mocha to the babel configuration:
In your package.json file
Under the scripts section, simply write something like this:
"test": "mocha \"src/**/*.test.js\""
Create a .mocharc.json file with this content:
{
"exit": true,
"color": true,
"require": ["#babel/register"],
"ignore": "node_modules"
}
This will apply Babel transformations to all of your test files.
If you need need to apply some special global javascript before/to all of your tests, you can add another file to the require setting, for example, fixtures.cjs:
"require": ["#babel/register", "fixtures.cjs"],
fixtures.cjs:
Below example applies a chai (popular alongside Mocha) plugin for testing DOM-related code:
var chai = require('chai'),
chaiDOM = require('chai-dom');
// https://stackoverflow.com/questions/62255953/chai-usechaihttp-once-or-in-every-test-file
// https://mochajs.org/#global-teardown-fixtures
exports.mochaGlobalSetup = function () {
chai.use(chaiDOM);
}
Interesting reads:
Babel vs babel-core vs babel-runtime
How does mocha / babel transpile my test code on the fly?
I'm about to write a yeoman generator where the whole template is hosted on a git repository. So the package.json of my yeoman generator looks like
{
"name": "generator-foo",
"version": "0.1.0",
"description": "",
"files": [
"generators"
],
"keywords": [
"yeoman-generator"
],
"dependencies": {
"foo-template": "git://somewhere-in-the-world/foo-template.git#0.1.0",
"chalk": "^1.1.3",
"yeoman-generator": "^1.1.1",
"yosay": "^2.0.0"
}
}
Is there any way to prevent npm install from installing the foo-template package, i.e. running any postinstall script just for this package? Instead, it should be just downloaded to node_modules.
As describe here, postinstall scripts can be disabled globally for npm using --ignore-scripts flag.
As a complete solution, I would move your explicit dependency to foo-template to your local postinstall section with ignore scripts enabled:
{
"name": "generator-foo",
...
"postinstall": "npm install --ignore-scripts git://somewhere-in-the-world/foo-template.git#0.1.0",
"peerDependencies": {
"foo-template": "git://somewhere-in-the-world/foo-template.git#0.1.0"
}
}
Note that to make sure the dependency is explicitly described, we should mark it as a peerDependency (e.g. prevents package removal on prune).
I am using npm precommit hook, but it is not stopping a file with issues to be committed, nor am I getting the message "Pre commit checks" when I try to commit a file.
Package Json:
{
"name": "myfolder",
"version": "1.0.0",
"description": "",
"main": "",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 0",
"precommit-msg": "echo 'Pre-commit checks...' && exit 0",
"lint": "csslint global/css"
},
"author": "SR",
"license": "ISC",
"dependencies": {
"csslint": "^1.0.4",
"jshint": "^2.9.4",
"pre-commit": "^1.2.2"
},
"pre-commit": [
"precommit-msg",
"lint"
],
"devDependencies": {
"pre-commit": "^1.2.2"
}
}
Please, make sure that your 'package.json' file is in the same folder, where '.git' folder is (where git repository was initialized). When you install 'pre-commit' package, 'pre-commit' file should appear under '.git/hooks/'.
Just FYI I had this issue because the pre-commit file was missing in the hooks folder.
Running npm i pre-commit --save-dev again created the file and solved it for me.
Have't managed to implement it with few "pre-commit" NPM modules (#fastify/pre-commit, monorepo-staged-precommit) so implemented it "manually" with adding tools/pre-commit.sh file into repo with content like:
#!/bin/sh
DIR='web'
echo "Pre-commit actions (NPM tests for $DIR)..."
cd $DIR && npm run test
and updating package.json with:
"scripts": {
"test",
"install-precommit": "cp ../tools/pre-commit.sh ../.git/hooks/pre-commit"
This solution has some limitations (like instead of automatic installation need to ask somewhere in "README" about npm run install-precommit and I'm not sure how it would work on Windows especially without Git Bash) but it worked for me.
Notes:
Other pre-commit NPM packages either didn't work as well or asked for NVM and other extra tools which I don't want devs to install for such small task.
pre-commit.sh may has any name and don't be executable - "install-precommit" task and git care about.
When I change my files while webpack-dev-server is running, the bundle's files are not updated.
Here are my webpack.config.js and package.json files, as you can see from my npm script, I've solved running webpack watch and webpack-dev-server in the same command (npm run watch & webpack-dev-server --content-base ./ --port 9966):
webpack.config.js:
'use strict';
var ReactStylePlugin = require('react-style-webpack-plugin');
var ExtractTextPlugin = require('extract-text-webpack-plugin');
var webpack = require('webpack');
module.exports = {
devtool: 'sourcemap',
entry: ['./js/main.js'],
output: {
filename: 'bundle.js',
path: __dirname + '/assets',
publicPath: __dirname + '/'
},
module: {
loaders: [
{
test: /\.js$/,
loaders: [
ReactStylePlugin.loader(),
'jsx-loader?harmony'
]
},
{
test: /\.css$/,
loader: ExtractTextPlugin.extract('css-loader')
}
]
},
plugins: [
new ReactStylePlugin('bundle.css'),
new webpack.DefinePlugin({
'process.env': {
// To enable production mode:
//NODE_ENV: JSON.stringify('production')
}
})
]
}
package.json:
{
"name": "reactTest",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"watch": "webpack --watch",
"build": "webpack",
"web": "npm run watch & webpack-dev-server --content-base ./ --port 9966"
},
"author": "",
"license": "ISC",
"devDependencies": {
"css-loader": "^0.10.1",
"extract-text-webpack-plugin": "^0.3.8",
"jsx-loader": "^0.13.1",
"react-style-webpack-plugin": "^0.4.0",
"style-loader": "^0.10.2",
"url-loader": "^0.5.5",
"webpack": "^1.8.5",
"webpack-dev-server": "^1.8.0"
},
"dependencies": {
"react": "^0.13.1",
"react-style": "^0.5.3"
}
}
my directory structure is:
assets
bundle.css
bundle.css.map
bundle.js
bundle.js.map
js
AppActions.js
Profile.css.js
ProfileList.js
main.js
AppConstants.js
AppStore.js
Profile.js
ResultPage.js
package.json
index.html
node_modules
webpack.config.js
every file inside assets directory is generated by webpack
In order to get webpack to watch my file changes (Ubuntu 14.04), I had to increase the number of watchers (I had increased the number before, but it was still too low):
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
Source in the official docs: https://webpack.github.io/docs/troubleshooting.html#not-enough-watchers
I first suspected the cause to be fsevents which doesn't work on Ubuntu, but this apparently wasn't the case.
Furthermore, because now the watching and re-compiling worked, but the automatic browser refresh part didn't work, I added the --inline param to the answer of #deowk which enables the "inline mode":
webpack-dev-server --content-base ./ --port 9966 --hot --inline
Quote from the official docs: "The easiest way to use Hot Module Replacement with the webpack-dev-server is to use the inline mode."
Source: https://webpack.github.io/docs/webpack-dev-server.html#hot-module-replacement
you need to run webpack-dev-server with the --hot flag:
webpack-dev-server --content-base ./ --port 9966 --hot
Then you can access the hot-loading version localhost:9966/webpack-dev-server/
You don't need to run watch as well.
update:
This entry in your webpack config must change:
entry: ['./js/main.js'], --> entry: ['webpack/hot/dev-server' , './js/main.js']
Change your publicPath entry:
publicPath: '/assets/'
#funkybunky identified the right problem but (IMHO) fixed it the wrong way. At least in my case, webpack was trying to watch every file it used, including a deep chain of thousands of files of dependencies pulled from npm. I added this to my config, per the docs:
devServer: {
watchOptions: {
ignored: /node_modules/
}
}
Of course you legitimately could have thousands of files that might need to be watched, in which case go ahead and raise the limit, but you're probably better off ignoring vendor libraries that aren't likely to change.
I'll put this here just in case it helps anyone. My problem was the same, but caused by inconsistent capitalization of directory names and webpack alias declaration.
I had a WebGL directory which i referenced in my aliases as webgl, and unfortunately this worked for the build, but didn't work for code watching.
In my case, the error was caused by an empty space in the directory name, by changing "Repository Name" by "RepositoryName", everything worked fine !
Figured I'd post my solution as well. I had the same problem getting Flutter apps to run on OS X due to my hard drive setup.
The gist is if your project folder is in a symlinked folder, detecting the file changes may not work on OS X. Previously, we were on Webpack 3.X, I believe, and live reload/refresh worked fine in the original folder. However, after upgrading to the latest Webpack (^5.75.0) and Webpack Dev Server (^4.11.1), the hot-reloading no longer worked for me.
My original project folder was:
/Users/blakemiller/h/somefolder/v2/my-widget
The "/h" there is a symlink to: /System/Volumes/Data/projects/home/web/
I'm not sure what happened when I upgraded OS X at some point, but the upgrade changed the folders in a way that I don't really understand.
Putting the folder here instead, fixed the issue for me (no symlink):
/Users/blakemiller/my-widget
I doubt this will work for many people, as my setup is probably pretty specific, but maybe it will help someone else save 5 hours down the road...