EDIT: Not sure what happened in the initial test, but .gitignore and .npmignore are now working to exclude the bin files as desired.
I have a simple NPM package which just installs a binary by downloading it from GitHub. My goal is to leave the binary out of the published package and just let the script install it. My thinking is that the install script may need to install different binaries depending on the host platform, and secondarily, there's no need to bloat the package size with the binaries.
The problem is that I can't seem to get NPM to ignore the 'bin' files. I've tried using .gitignore, .npmignore and the package.json 'files' attribute as well as placing the bin files in a folder other than bin, and it seems that NPM insists on including the 'bin' files. The relevant package.json is basically:
{
"bin": {
"my-bin": "./bin/my-bin"
},
"files": [ "bin-install.sh" ],
"scripts": {
"preinstall": "./bin-install.sh"
},
}
As a workaround, I can juggle the 'bin' directory existence like:
"scripts": {
"preinstall": "./bin-install.sh",
"prepublishOnly": "! [[ -d bin ]] || mv bin tmp",
"prepack": "npm run prepublishOnly",
"postpublish": "! [[ -d tmp ]] || mv tmp bin",
"postpack": "npm run postpublish"
}
This works, but looks ugly and makes me think I'm missing something that's leading me to go against the grain.
Is there a cleaner approach to achieve what I'm trying to do?
.npmignore should work, here are the docs. You haven't shown how exactly you've attempted to set up ignore, but it sounds like you has been trying to ignore all files inside bin/ so this is what you needed:
# inside .npmignore
bin/
Related
We have a self-hosted GitLab (15.5.4) and I've configured everything we needed for publishing npm packages.
A CI/CD pipeline that properly creates the entry in the Package Registry.
The problem is that when I pull the package [npm i #scope/lib] (It doesn't change if I cast the auth token in the package.json or I pass through an environment variable as suggested in the documentation) the unwanted result is that the #scope/lib doesn't have the dist/ folder in it!! [node_module/#scope/lib/].
If I browse to the Package Registry and manually download the .tgz file I can see that the dist/ folder is present.
I've played around a bit with the .npmignore and "prepublish" script but I had no success and literally have no glue why this is happening.
Any tips would be very appreciated
To clarify:
The proper way is to tell npm to keep the dist/ folder, bypassing the .gitignore file (instead of defining an .npmignore article here ) is to define a files entry in the package.json :
{
"files": [
"dist",
"build",
...
]
}
Another unproper way to do get the result I needed is to use a postinstall command. But it is clearly an anti-pattern. Given that I am writing a typescript library, that is tested and then compiled by the CI, there's no need to recompile it within the postinstall command. But it could be an hacky solution when needed.
{
"scripts": {
"postinstall": "tsc src/index.ts"
}
}
To sum up, I think it was only an npm cache issue or more probably a server-side cache issue, because I've run npm cache clean --force different times.
Hope this helps.
The npm publish
command creates a tarball (with src dir) and publish it to registry.
is there a way to exclude the src dir avoiding use of .npmignore ?
npm provides no other built-in feature to achieve that, so a custom solution is required.
If you really don't want to use .npmignore to keep the src directory out of your published package, then consider utilizing pre and post hooks in your npm scripts instead.
The pertinent hooks are:
prepublishOnly: Run BEFORE the package is prepared and packed, ONLY on npm publish ...
postpublish: Run AFTER the package is published.
For *nix platforms
Add a prepublishOnly script to the scripts section of your package.json that moves the src directory to another location outside of your project directory prior to publishing.
Also, add a postpublish script that moves the src directory back to the project directory when publishing has completed.
Run npm publish (as per normal) to publish your package.
For instance:
package.json
...
"scripts": {
"prepublishOnly": "mv src/ ../",
"postpublish": "mv ../src .",
...
},
...
Note: You'll need to ensure that no other src folder/directory exists at the path location you choose to temporarily move the src directory to via your prepublish script.
Cross platform:
For a cross-platform solution consider utilizing shx. This package includes a portable mv command. In which case, configure your prepublish and postpublish scripts something like the following instead:
package.json
...
"scripts": {
"prepublishOnly": "shx mv src/ ../",
"postpublish": "shx mv ../src .",
...
},
...
You can use the files property in your package.json to explicitly include the files you want to publish.
{
"files": [
"dist",
"index.js"
]
}
As #RobC answer, there is no other way then a custom solution for avoid using .npmignore.
Since Iām using the publish command in a Jenkins pipeline, a solution is to create a temporary .npmignore while the publish step directly in the Jenkinsfile:
echo "src/" >> .npmignore
echo "*.js" >> .npmignore
echo "*.json" >> .npmignore
echo "Jenkinsfile" >> .npmignore
curl --insecure -u ${USERPASS} 'https://my-repo/api/npm/auth' >> /home/jenkins/.npmrc
npm publish --registry https:// my-repo/api/npm/npm-local/
I am trying to build my first Angular Component package using lerna and it was working pretty well until I realized I had to add "ng-packagr" to get all of the HTML bundled inline with the rest of the code. After adding that support and getting it to work now all of a sudden my files are not getting published into the tarball.
Here is my package.json
{
"name": "#custom/core",
"version": "0.0.7",
"description": "Test",
"main": "./dist/bundles/custom-core.umd.min.js",
"module": "./dist/esm2015/custom-core.js",
"typings": "./dist/index.d.ts",
"$schema": "./node_modules/ng-packagr/package.schema.json",
"ngPackage": {
"lib": {
"entryFile": "./src/index.ts"
},
"whitelistedNonPeerDependencies": [
"."
]
},
"scripts": {
"build": "ng-packagr -p package.json"
},
"files": [
"dist"
],
...
My dist folder contains all kinds of folders like this:
But then when I run a lerna publish or npm pack this is what happens:
As you can see only 1 file gets added to the tarball...
Does anyone know why this is happening all of a sudden? I've tried playing around with my .gitignore thinking maybe it was forcing the packaging to ignore these other files but it wasn't that.
UPDATE
Ok so I found that the culprit is ng-packagr. When I run my npm run build which uses ng-packagr -p package.json to build the different module packages, that CLI is also generating a package.json that goes inside of my dist folder. When the npm pack or lerna publish attempt to package everything using a package.json they must be looking at the generated on in dist rather than the one in the folder above it.
I'm not sure how I should be fixing this.
My solution was to abandon my plan to use the files key in the package.json file and to instead use a .npmignore file. Here is a copy of mine:
# Node generated files
node_modules
npm-debug.log
assets
package-lock.json
# aot files
aot
# OS generated files
Thumbs.db
.DS_Store
# Ignored files
*.ts
!*.d.ts
tsconfig.json
tsconfig-aot.json
tslint.json
*.tgz
config
src
I've got continuous deployment setup with Netlify and a git repo but so far no matter what I do, Netlify isn't npm-installing anything. There's no node_modules folder when I download a zip of the deploy, my site can't access node_modules, it's just not there. I've started with a random npm package (lodash) to try to get it to install but I've got nothing so far.
Netlify says that it automatically runs npm install. I've tried having no build commands and I've tried adding npm install as a build command with no results from either.
package.json:
{
"name": "netlify-test",
"version": "1.0.0",
"description": "stuff",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
"type": "git",
"url": "git+https://github.com/electrovir/netlify-test.git"
},
"author": "electrovir",
"license": "MIT",
"bugs": {
"url": "https://github.com/electrovir/netlify-test/issues"
},
"homepage": "https://github.com/electrovir/netlify-test#readme",
"dependencies": {
"lodash": "^4.17.11"
}
}
HTML:
<!doctype html>
<html>
<head>
<title>
hi
</title>
<script src="node_modules/lodash/_apply.js"></script>
</head>
<body>
Why can't this find node_modules??
</body>
</html>
It took a while to figure this out but I discovered that Netlify does npm install into your repo root or Base directory when you have a package.json. This can be seen by changing the Build command to something like ls and reading the deploy console output (there will be a node_modules folder listed).
However, at some point after running the Build command and before deploying, this node_modules is deleted. Hence, if you use something similar to the following as a Build command, node_modules can be copied to your Publish directory:
rm -rf dist && mkdir dist && rsync -rv * dist --exclude ./dist
This copies the Base directory contents into dist, which I have set as my Publish directory, ignoring ./dist itself (it can't be copied into itself). It'd be even better if this were added as an npm script in package.json with npm run <script name> for your Build command.
Note that Base directory, Build command, and Publish directory are all under Deploys > Deploy settings or Settings > Build & deploy in Netlify.
Thanks to lastmjs for help with figuring this out.
It would be odd if node_modules WERE there. Netlify's continuous deployment does run npm install for you, if and only if there is a package.json in the root of your repository (or instead in the base directory if you set one in netlify.toml. However, it also uses a custom npm directory that is outside of the deployment directory (see how it is setup here: https://github.com/netlify/build-image/blob/master/run-build-functions.sh#L34), since deploying your node_modules shouldn't be needed for a static site at browse time - only at build time.
The intended path for a Netlify deploy is:
you have your dependency manager configuration checked into your git repo at the root of the repo (if no base directory is set in the UI or in the toml file) or in the base directory if set. This could be any/all of Gemfile.lock, package.json, yarn.lock, or requirements.txt as explained in this article about the build settings at Netlify
you do not store your node_modules in the repo. Many of those will be specific to the architecture of the build environment - and your local machine is almost certainly different than Netlify's. You'll want those generated at build time.
your build is intended to USE those modules DURING BUILD. So you use for instance dotenv to read your .env file while building, or you use gulp-cli to process your Gulpfile.js
your build completes, and you've used all the modules during build to generate standalone html/js/css/images, so you're done with them - your static html and other assets are generated in your publish directory, where the build process has intentionally not put your node modules, and thus they are not deployed. Your static website should not need the modules at runtime, so deploying the (thousands!) of files in there is not efficient or needed.
You could have a situation that requires some/one of them - e.g. a function or you could need a file from one of them even on your static site - and that's fine, feel free to copy it into the publish directory explicitly. But that is not the norm :)
Suppose that somewhere in my package.json I have:
"dependencies": {
"bower": "1.0.0",
// zillion other dependencies
}
Is there a way to make npm install only bower#1.0.0 from my package.json? Like so: npm install --only bower.
My goal is to make npm install and bower install run simultaneously.
As a workaround you may use something like:
$ node -pe "require('./package').dependencies.bower"
// ā 1.0.0
$ npm install bower#$(node -pe "require('./package').dependencies.bower")
// npm install bower#1.0.0
// or with jq
$ npm install bower#$(< package.json jq -r '.dependencies.bower')
Where -e/--eval flag evaluates passed string and -p/--print prints result of eval.
š” Please consider other answers as well since this one may be outdated.
As #atalantus noted in comment, the accepted answer doesn't work on newer version of NPM. Working solution for newer versions (verified on NPM 6.13.4) is:
npm install --no-package-lock --no-save bower#1.0.0
This will install bower and all its dependencies, but prevents installation of anything else you might have in package.json. It also doesn't create or modify existing package-lock.json.
From npm documentation:
The --no-package-lock argument will prevent npm from creating a package-lock.json file. When running with package-lock's disabled npm will not automatically prune your node modules when installing.
--no-save: Prevents saving to dependencies.
Combining this with Anton Rudeshko's approach to find out version in package.json, the final solution is:
VERSION_BOWER=`node -p -e "require('./package.json').dependencies.bower"`
npm install --no-package-lock --no-save bower#"$VERSION_BOWER"
Update 2023
Alternative solution I've just used is to temporarily modify package.json to only keep the dependency you need. That can be done quite easily with help of jq
Given package.json
{
"name": "sample",
"main": "src/index.js",
"dependencies": {
"bower": "1.0.0",
"foo": "^1.2.3",
"bar": "^4.5.6",
},
"devDependencies": {
"baz": "^10.20.30",
}
}
Let's first modify it with jq to only keep bower
cat package.json| jq 'del(.devDependencies) | .dependencies |= {bower:.bower}'
Gives us
{
"name": "sample",
"main": "src/index.js",
"dependencies": {
"bower": "1.0.0"
}
}
by deleting devDependencies completely and leaving only bower in dependencies.
Next up you would want to overwrite package.json with this new, version, but you can't do it directly like this
cat package.json | jq 'del(.devDependencies) | .dependencies |= {bower:.bower}' > package.json
because the file gets overwritten before it's fully read. Instead you need to buffer the output stream of jq command, simplest solution just storing it in variable, and then overwriting the original file
updated=`cat package.json | jq 'del(.devDependencies) | .dependencies |= {bower:.bower}'`
echo $updated > package.json
And package.json now reads:
cat package.json
{
"name": "sample",
"main": "src/index.js",
"dependencies": {
"bower": "1.0.0"
}
}
Run npm install now and you only install bower#1.0.0.
Obviously not always we can afford to modify the file, but you could also back-up the original file, etc. Main point is here how to easily modify json file thanks to jq rather than playing magic tricks with sed :-)
Seeing as with newer versions of NPM the previous solutions don't work anymore, I came up with this:
npm i -g --prefix=$(pwd) bower # $(pwd) = current directory, UNIX only
node bin/bower
Installing a package globally allows you to specify a "prefix" - a directory where the package will be installed. Here I just put the current directory as the folder I want the package to appear in. In CI, this will probably be your repo root.
Keep in mind this will ignore your package.json version and just download the latest one. If you need a specific version, just use bower#someversion instead.
An alternative solution is just rm package.json && npm i bower, or mv package.json _package.json && npm i bower if you want to restore it later