We have a monorepo with many subprojects.
root/
complex/dir/layout/subproj1
complex/dir/layout/subprojB
other/dir/subproj2
I'd like to have certain scripts available globally that were available to all the different subprojects. An example is a script we've been using to deploy AWS lambdas, like
"scripts": {
"1up:slsdeploy": "npx serverless deploy --aws-profile prod --region 'us-west-2' --description $((git branch --show-current; git rev-parse --short HEAD; id -un) | tr '\\n' ' ')"
},
Copying this and maintaining this in our 40 different lambdas isn't DRY. An answer specifically about serverless is not the main goal, but that is the immediate issue. But I would prefer more general techniques to make global scripts available to subprojects.
The most obvious answer -- just putting it in the root package.json does not appear to work (as dependencies do).
I also came across an interesting thing about the environment variables npm makes available (toward the end) at https://www.twilio.com/blog/npm-scripts that says you can reference settings in .npmrc like $npm_config_, so I though I might be able to put something in the top level .npmrc, like
MY_SPECIAL_CMD = "do something really special"
And then reference that in my individual package.json
"scripts": {
"1up:slsdeploy": "$npm_config_MY_SPECIAL_CMD"
},
which would be better, and maybe even MORE correct, since every predefined script may not be appropriate for every subproject. But apparently this only applies to the DEFINED configuration settings for .npmrc (and npm silently ignores settings it doesn't recognize, which seems like a bug). So this doesn't work.
Related
I'm trying to set up an npx script to create a template project.
In package.json I have:
"bin": {
"init": "bin/init"
}
In the init script I'm using tag='v'$(npm pkg get version | tr -d '"') to get the version of the package. I then use git clone --depth 1 --branch $tag https://github.com/matriarx/typescript.git to clone that specific repository for that specific tag.
When I do yarn link and try use it locally, from within that specific project, it works because it's able to correctly pick up the package.json version. So the above only works if it's run inside an existing project. However that's not what I want to do.
I want to enable someone to run it even if they have nothing locally, by simply doing npx #matriarx/typescript init and it should create the new project by cloning it. More than that I want them to be able to clone any specific version by using npx #matriarx/typescript#0.0.1 init in order to clone a specific version.
However it seems that anything I try is only able to get the version from a local package.json that already exists.
I could just clone the current existing repository without specifying any tag, but that would defeat the point of having releases, then it would just clone any current code completely disregarding the release. So it has to clone the tagged release.
How can I get the remote package version stored on npm from the bin script without having anything locally before hand?
Alternatively is there a better way to do what I'm trying to do?
EDIT: I ended up just hardcoding the version in the script, which works but it sucks because it's tedious to have to update it every time I bump the version. Though for now I still don't know a better way to do it.
After some more time messing around I figured out there is a standard way of doing it, at least since npm 7.
If you have a project like example then you can create a completely separate project called create-example with a bin script.
When you use npm init example, npm will automatically search for a package prefixed with "create-" and execute its main bin script. So when running npm init example it will search for that create-example package and execute the bin script, which will install the example package.
This is how most of the bigger packages like react and next do it.
This approach comes with some disadvantages that I really don't like, for example it will show the incorrect dependencies on npm and it will cause you to have to maintain multiple projects and semvers on different projects. However it will also allow you to create a clean separation between the actual project and the installation of that project.
For some projects it might also make a lot more sense. For example if you have a project that doesn't have a package.json at all and you want to create a setup for it, it wouldn't make sense to create an npm package inside that project just for that. Instead you can create a separate "create-project" package just to set it up and keep npm out of the actual project. In other words it gives you a neat way to create bin scripts for a completely separate project that doesn't have anything to do with npm.
You could also just have created a normal shell script and execute it using curl but I guess npm just gives you another way to do it.
You still have to hardcode the version in that "create-project" package, I still have not seen a way to automatically determine the version from a remote package. The only way I've managed to do that is to completely download the package, get the version, then delete it, but that would be terrible for people with a slower internet connection or limited data.
Situation
I'm working on a project "main" and in order to split the code and make parts of it reusable, I created 2 npm projects/packages: "A" and "B". "main" requires "A" and "B", "B" requires "A"
main
+-- A
`-- B
`-- A
While developing in project main, A and B might require some improvements, But in order to have changes inside A and B take effect inside main I would need to publish a new version of them for every little change
npm link, yalc, workspaces
The common approach is npm link or npm workspaces but it has a few downsides, symlinks tend to behave differently depending on the environment and setup, also the symlinks tend to get messed up by running other npm commands, there are also some issues with webpack etc. etc.
yalc attempts to solve this by copying files around and putting file: / link: dependencies into the package.json
All of these "local" approaches have a few problems:
If I push the code of "main" to git, I can't run CI/CD pipelines that rely on the newest A/B being present because in the pipeline, those packages are not symlinked or the file: / link: dependencies point to folders that do not exist
When working in a small team, the improvements only exist on my machine, even if I push them into the repositories of A and B, others in my team would need to constantly need to update their repositories
others in my team would need the same folder/link structure on their machines like me
git repositories
I added A and B as git dependencies inside main:
main/package.json
"devDependencies": {
"A": "git+https://<link of my git repo>/A.git",
"B": "git+https://<link of my git repo>/B.git",
B/package.json
"devDependencies": {
"A": "git+https://<link of my git repo>/A.git"
Thats great because I can simply push a change in A or B and everybody can receive the newest version of A and B via npm update
The problem here is, that npm sadly but justifiably does not recursively resolve git dependencies, A and B inside main get resolved but A inside B does not get resolved, this means that when running webpack for example, "A" does not exist inside "B"
what now?
There has to be a better solution than this. But using real packages, even if its a private repository like verdaccio, means every little change requires me to create and publish a new version of the package, spamming the repository with package versions.
I am getting this when I am trying to push my code into github actions or building dockerimage.
shell: /usr/bin/bash -e {0}
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion#1, but package-lock.json was generated for lockfileVersion#2. I'll try to do my best with it!
I tried to implement this Link it works but again after some commit I am getting the same error and I have to repeat the same procedure again and again.
Any fix for that?
Look in your .gitignore if you have the lines :
package-lock.json
node_modules/
if not,then add them,
after that look in your Github repository and delete the package-lock.json file and the node_modules directory (if any)
Important Edit :
My bad, Kevin Martin is right the official documentation tell us to add it to the repository for CI/CD.
This file is intended to be committed into source repositories, and
serves various purposes:
Describe a single representation of a dependency tree such that
teammates, deployments, and continuous integration are guaranteed to
install exactly the same dependencies.
Provide a facility for users to "time-travel" to previous states of
node_modules without having to commit the directory itself.
To facilitate greater visibility of tree changes through readable
source control diffs.
And optimize the installation process by allowing npm to skip repeated
metadata resolutions for previously-installed packages.
But for my case (Azure Devops) i had a lot of trouble with it.
Is there an easy way to provide usage information for npm scripts?
Ideally, when I run npm run, I would get output like this (note the description at bottom of each task):
Lifecycle scripts included in product-discovery-service:
start
node server.js
available via `npm run-script`:
watch
run-p watch:build watch:run
Run in development mode and rebuild/restart when changes are made
watch:build
npm run build:dev -- --watch
Probably don't need this (would be nice to be able to omit tasks like this)
watch:run
nodemon --watch build/ --inspect
Probably don't need this (would be nice to be able to omit tasks like this)
prewatch:run
wait-on --log build/server.js
Probably don't need this (would be nice to be able to omit tasks like this)
build
babel server.js --out-dir build/
Build the project
prebuild
rimraf build/
Probably don't need this (would be nice to be able to omit tasks like this)
build:dev
npm run build -- --source-maps
Probably don't need this (would be nice to be able to omit tasks like this)
It looks like npm doesn't support this, but maybe there's a third-party with a solution? I found npm-scripts-help, but it feels clunky.
Short Answer:
Yes you're correct, npm does not provide a built-in feature to include descriptions when running npm run. So, any solution you choose will have some level of "feels clunky" associated with it.
As you've mentioned
npm-scripts-help is a package which can achieve this. I'm not aware of other similar third-party solutions.
Alternative custom solution:
The following steps describe how to write a simple custom Nodejs utility script (without using another third-party package dependency). This script can then be invoked via npm-scripts.
Create a simple Nodejs utility script as follows. Lets name the file usage.js.
usage.js
const usage = `
Lifecycle scripts included in ${process.env.npm_package_name}:
start
node server.js
available via \`npm run-script\`:
watch
run-p watch:build watch:run
Run in development mode and rebuild/restart when changes are made
watch:build
npm run build:dev -- --watch
Probably don't need this (would be nice to be able to omit tasks like this)
watch:run
nodemon --watch build/ --inspect
...`
console.log('%s', usage);
Save usage.js in your projects root directory at the same level where package.json is stored.
Add the following usage script to the scripts section of your package.json:
...
"scripts": {
"usage": "node usage",
...
},
...
Run npm run usage to print the usage info to the console. A script name (i.e. usage) has to be provided with npm run. Unfortunately your ideal of just running npm run will only log npm's simple log - which doesn't include descriptions.
Notes:
On line two in usage.js we reference the package name variable via the part which reads: ${process.env.npm_package_name}
If you change the location of where usage.json is stored in your project directory you'll need to redifine the path to it in your npm-script as required. For example, if you choose to store it in a folder named scripts, which resides in your projects root directory, then your usage script should be defined as follows:
...
"scripts": {
"usage": "node scripts/usage",
...
},
...
Adding ANSI/VT100 Control sequences
You can utilize ANSI/VT100 Control sequences in usage.js to add colors and formatting to the usage log.
For example, in the following usage.js, the codes:
\x1b[1m
\x1b[0m
...are utilized to embolden code snippets and reset the formatting back to default respectively.
Tip: If cross-platform is a requirement I suggest using the ANSI 8/16 Colors (listed at the previous link) only. The formatting codes for bold (\x1b[1m) do not work in Windows cmd.exe using terminals such as Windows Command Prompt or PowerShell.
usage.js (with formatting)
const BOLD = '\x1b[1m';
const NORM = '\x1b[0m';
const formattedUsage = `
Lifecycle scripts included in ${BOLD}${process.env.npm_package_name}:${NORM}
${BOLD}start
node server.js${NORM}
available via ${BOLD}npm run-script${NORM}
${BOLD}watch
run-p watch:build watch:run${NORM}
Run in development mode and rebuild/restart when changes are made
${BOLD}watch:build
npm run build:dev -- --watch${NORM}
...`
console.log('%s', formattedUsage);
You could also consider combining ES6 Template Literals with process.env and package.json vars to reference the values of each npm-script. For example:
`${BOLD}${process.env.npm_package_scripts_watch}:${NORM}`
The postinstall stage for my package.json uses tsc. I'd like to make it so such a global dependency is never expected of the user, and the solution is to replace it with ./node_modules/typescript/bin/tsc.
Unfortunately, this only works if node_modules is a subfolder of the one I'm currently in. So what I would like to do is use some form of env variable which probably exists but I can not find to do this:
$ROOT_NODE_PACKAGE/node_modules/typescript/bin/tsc $CURRENTLY_INSTALLING_PACKAGE
How do I do this?