Situation
I'm working on a project "main" and in order to split the code and make parts of it reusable, I created 2 npm projects/packages: "A" and "B". "main" requires "A" and "B", "B" requires "A"
main
+-- A
`-- B
`-- A
While developing in project main, A and B might require some improvements, But in order to have changes inside A and B take effect inside main I would need to publish a new version of them for every little change
npm link, yalc, workspaces
The common approach is npm link or npm workspaces but it has a few downsides, symlinks tend to behave differently depending on the environment and setup, also the symlinks tend to get messed up by running other npm commands, there are also some issues with webpack etc. etc.
yalc attempts to solve this by copying files around and putting file: / link: dependencies into the package.json
All of these "local" approaches have a few problems:
If I push the code of "main" to git, I can't run CI/CD pipelines that rely on the newest A/B being present because in the pipeline, those packages are not symlinked or the file: / link: dependencies point to folders that do not exist
When working in a small team, the improvements only exist on my machine, even if I push them into the repositories of A and B, others in my team would need to constantly need to update their repositories
others in my team would need the same folder/link structure on their machines like me
git repositories
I added A and B as git dependencies inside main:
main/package.json
"devDependencies": {
"A": "git+https://<link of my git repo>/A.git",
"B": "git+https://<link of my git repo>/B.git",
B/package.json
"devDependencies": {
"A": "git+https://<link of my git repo>/A.git"
Thats great because I can simply push a change in A or B and everybody can receive the newest version of A and B via npm update
The problem here is, that npm sadly but justifiably does not recursively resolve git dependencies, A and B inside main get resolved but A inside B does not get resolved, this means that when running webpack for example, "A" does not exist inside "B"
what now?
There has to be a better solution than this. But using real packages, even if its a private repository like verdaccio, means every little change requires me to create and publish a new version of the package, spamming the repository with package versions.
Related
I'm trying to set up an npx script to create a template project.
In package.json I have:
"bin": {
"init": "bin/init"
}
In the init script I'm using tag='v'$(npm pkg get version | tr -d '"') to get the version of the package. I then use git clone --depth 1 --branch $tag https://github.com/matriarx/typescript.git to clone that specific repository for that specific tag.
When I do yarn link and try use it locally, from within that specific project, it works because it's able to correctly pick up the package.json version. So the above only works if it's run inside an existing project. However that's not what I want to do.
I want to enable someone to run it even if they have nothing locally, by simply doing npx #matriarx/typescript init and it should create the new project by cloning it. More than that I want them to be able to clone any specific version by using npx #matriarx/typescript#0.0.1 init in order to clone a specific version.
However it seems that anything I try is only able to get the version from a local package.json that already exists.
I could just clone the current existing repository without specifying any tag, but that would defeat the point of having releases, then it would just clone any current code completely disregarding the release. So it has to clone the tagged release.
How can I get the remote package version stored on npm from the bin script without having anything locally before hand?
Alternatively is there a better way to do what I'm trying to do?
EDIT: I ended up just hardcoding the version in the script, which works but it sucks because it's tedious to have to update it every time I bump the version. Though for now I still don't know a better way to do it.
After some more time messing around I figured out there is a standard way of doing it, at least since npm 7.
If you have a project like example then you can create a completely separate project called create-example with a bin script.
When you use npm init example, npm will automatically search for a package prefixed with "create-" and execute its main bin script. So when running npm init example it will search for that create-example package and execute the bin script, which will install the example package.
This is how most of the bigger packages like react and next do it.
This approach comes with some disadvantages that I really don't like, for example it will show the incorrect dependencies on npm and it will cause you to have to maintain multiple projects and semvers on different projects. However it will also allow you to create a clean separation between the actual project and the installation of that project.
For some projects it might also make a lot more sense. For example if you have a project that doesn't have a package.json at all and you want to create a setup for it, it wouldn't make sense to create an npm package inside that project just for that. Instead you can create a separate "create-project" package just to set it up and keep npm out of the actual project. In other words it gives you a neat way to create bin scripts for a completely separate project that doesn't have anything to do with npm.
You could also just have created a normal shell script and execute it using curl but I guess npm just gives you another way to do it.
You still have to hardcode the version in that "create-project" package, I still have not seen a way to automatically determine the version from a remote package. The only way I've managed to do that is to completely download the package, get the version, then delete it, but that would be terrible for people with a slower internet connection or limited data.
(not a native speaker, sorry if things don't seems clear)
We are devs for a clients and we work with others devs from another company. The other devs make us a package that we sometimes change this in it, then submit the changes in a pull request.
So we end up having a local version of the package and the official version. And so we want to use the local version in dev (where we might make little changes in the package to match what the client want), and the official in prod (where our changes and other devs changes are merges).
I will show an example that don't work but that can help understand the idea:
[...]
"dependencies"{
[...]
"package":"prod-package"
},
"devDependencies"{
[...]
"package":"local/version/of/package"
},
and so, we should have this:
npm build # use local/version/of/package
npm build --prod # use prod-package
I'm not really good with npm (in fact, beside npm install and npm remove, I basically know nothing), so I might ask something obvious, but I can't find the answer anywere.
Have you tried using npm-link to link to the local version of the package you are consuming? That way, you could still commit your changes to the package back to the vendor.
https://docs.npmjs.com/cli/v8/commands/npm-link
You would essentially run npm link in the checked-out-from-VCS (git?) package folder, then you switch back to your project where you are consuming that package and run npm link {name-of-npm-package}
This would allow you to develop on that version locally, make your changes (and they will show up on VCS for easy pull request management).
We have a monorepo with many subprojects.
root/
complex/dir/layout/subproj1
complex/dir/layout/subprojB
other/dir/subproj2
I'd like to have certain scripts available globally that were available to all the different subprojects. An example is a script we've been using to deploy AWS lambdas, like
"scripts": {
"1up:slsdeploy": "npx serverless deploy --aws-profile prod --region 'us-west-2' --description $((git branch --show-current; git rev-parse --short HEAD; id -un) | tr '\\n' ' ')"
},
Copying this and maintaining this in our 40 different lambdas isn't DRY. An answer specifically about serverless is not the main goal, but that is the immediate issue. But I would prefer more general techniques to make global scripts available to subprojects.
The most obvious answer -- just putting it in the root package.json does not appear to work (as dependencies do).
I also came across an interesting thing about the environment variables npm makes available (toward the end) at https://www.twilio.com/blog/npm-scripts that says you can reference settings in .npmrc like $npm_config_, so I though I might be able to put something in the top level .npmrc, like
MY_SPECIAL_CMD = "do something really special"
And then reference that in my individual package.json
"scripts": {
"1up:slsdeploy": "$npm_config_MY_SPECIAL_CMD"
},
which would be better, and maybe even MORE correct, since every predefined script may not be appropriate for every subproject. But apparently this only applies to the DEFINED configuration settings for .npmrc (and npm silently ignores settings it doesn't recognize, which seems like a bug). So this doesn't work.
I am working in a very restricted environment, which means I do not have any access to internet. I setup everything in order to import an Angular Template Project and install all the packages from an offline cache. I followed these steps:
On the online machine
configure yarn-offline-mirror with pruning to false (directory X)
create a new angular app with ng new foo --skipInstall=true
install the packages with yarn (install) in order to generate a yarn.lock and to store all the tgz-packages in directory X
delete node_modules
Whenever I need more packages, I use npm-package-downloader with the argument -d to download all the dependencies as tgz-files too and copy them to directory X
On my offline machine
I import the tgz-files from directory X and put them in the same location
I configure yarn-offline-mirror like on my online machine
I Import the angular template and install the needed packages with yarn clean cache followed by yarn --offline
As long all the tgz-packages are on my offline machine, this works like a charm (because of yarn.lock).
My Problem
I want to add further packages on my offline machine. The packages are present in the offline mirror, because I copied them onto the machine. When I run
yarn clean cache
yarn add <absolute-path-to-tgz-package> --offline
two possible scenarios arise:
The package has no dependencies, in which case it works as intended
The package has dependencies: in this case, follwing error gets thrown for each "missing" dependency (package-name and package-version are placeholders):
Couldn't find any versions for "package-name" that matches
"package-version" in our cache. This is usually caused by a missing
entry in the lockfile, running Yarn without the --offline flag may
help fix the issue.
As far as I understand, yarn needs somehow to know, which dependencies a package requires. But it has to know, otherwise it couldn't throw this specific error. So my question is, how can I force yarn to look for dependencies in directory X too? The packages are there, since I download them with the respective dependencies. And obviously I am not supposed to manually edit yarn.lock. The docs for yarn add do not list such an option (as a matter of fact, it does not even list the --offline flag)
In my app, I have these dependencies:
TypeORM
typeorm-linq-repository AS A LOCAL INSTALL ("typeorm-linq-repository": "file:../../../IRCraziestTaxi/typeorm-linq-repository"), who has a dev dependency AND a peer dependency of TypeORM
The reason I use a "file:" installation of typeorm-linq-repository is that I am the developer and test changes in this app prior to pushing releases to npm.
I was previously using node ~6.10 (npm ~4), so when I used the "file:" installation, it just copied the published files over, which is what I want.
However, after upgrading to node 8.11.3 (npm 5.6.0), it now links the folder rather than copying the published files.
Note, if it matters, that my environment is Windows.
The problem is this: since both my app and the linked typeorm-linq-repository have TypeORM in their own node_modules folders, TypeORM is being treated as a separate "instance" of the module in each app.
Therefore, after creating a connection in the main app, when the code that accesses the connection in typeorm-linq-repository is reached, it throws an error of Connection "default" was not found..
I have searched tirelessly for a solution to this. I have tried --preserve-symlinks, but that does not work.
The only way for me to make this work right now is to manually create the folder in my app's node_modules and copy applicable files over, which is a huge pain.
How can I either tell npm to NOT symlink the "file:" installation or get it to use the same instance of the TypeORM module?
I made it work pretty easily, although I feel like it's kind of a band-aid. I will post the answer here to help anybody else who may be having this issue, but if anybody has a more proper solution, feel free to answer and I will accept.
The trick was to link my app's installation of TypeORM to the TypeORM folder in my other linked dependency's node_modules folder.
...,
"typeorm": "file:../../../IRCraziestTaxi/typeorm-linq-repository/node_modules/typeorm",
"typeorm-linq-repository": "file:../../../IRCraziestTaxi/typeorm-linq-repository",
...