Serverless Framework - Share code between multiple services - serverless-framework

I have multiple services
root/
services/
subscriptions/
users/
shared/
httpUtils.js
database.js
node_modules/
package.json
When I'm running severless locally using serverless-offline my imports work without a problem:
const _ = import('lodash') // module_modules
const database = require('../../shared/database')
However, when I deploy the application won't start because of
error: cannot find module lodash
error: cannot find module ../../shared/database
Does each service need to have its own node_modules dependencies within the service? It would be great if I could just load dependencies from a single node_modules at the root and the shared folder.
What's the standard practice in order to accomplish this for a project with multiple services?
each services gets its own copy of the shared files and node_modules?
private npm package with shared libs?
another packaging tool?
Thanks.

It is recommended that you use a packages.json for each service rather than try to use a single one shared across multiple services. You can get additional details about the use of package.json and the serverless framework here.
I have tried various techniques to share a single node_modules directory, but the solutions are fragile and eventually something will go wrong as I introduce more dependencies and start to version the services.
I do try to share as much as possible across my services, but have determined it isn't worth the effort to share a single node_modules directory.

Related

Private github npm repositories on Netlify

I've been trawling documentation on Netlify's site for how to deploy a node application (specifically a Vue application) which has a private npm repository as a dependency.
I have an .npmrc file setup as follows:
//npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}
#my:registry=https://npm.pkg.github.com
I also set the GITHUB_TOKEN variable in my build process's environment variables to the correct value.
However, the build fails.
There is an outline of how to achieve a successful build with a private Github package repository here: https://docs.netlify.com/configure-builds/repo-permissions-linking/#npm-packages.
However, it's still unclear as to what I need to do specifically to both my package.json file and also how to configure the build process' environment variables ...
Could anyone show me a full working example which doesn't leave me scratching my head?
Do I add:
"package-name": "git+https://<github_token>:x-oauth-basic#github.com/<user>/<repo>.git"
to the dependencies section of the package.json?
If yes, how do I then hide my <github_token> and provide the build process with this in Netlify?
How do I also get the same process working locally?
Why can't Netlify just inject the GITHUB_TOKEN into my .npmrc file so I have parity with my local development environment?

NPM : Create an NPM package that adds files and folders to the project root directory

I've created a web app template that I use frequently for many different projects.
I would like to create an NPM package for it so that it's easier to install for new projects, separate the template from the project files, separate the template dependencies from the project dependencies, and allow easier updating of the template across all projects.
The issue I have is that I need some files/folders to be installed in the root directory (i.e. where package.json is saved). Most can go in the node_modules folder however I have some files that must be placed in the root directory.
For example, the template uses Next.js with a custom _app.js file. This must be in the root directory in a folder named pages. I also have various config files that must be in the root directory.
Can this be done with NPM, or does everything need to be installed in the node_modules folder? I'm having trouble finding anything on SO or Google that answers this, so if you happen to know a guide online on how to do this or can outline things I should search for it would be much appreciated.
With pure npm, everything has to go to the node_modules folder, so you can't solve your issue this way.
Maybe going with a templating tool such as grunt init or yeoman could be a solution here, although – unfortunately – you'll then lose some of the benefits of being able to install a package via npm.
Another option might be to use GitHub template repositories, which have just been introduced recently.
Last but not least one option might also be to just have the files' contents in the npm package, but create the pages/_app.js manually, but inside of it simply require the file contents from an npm module, and that's it. This at least helps to have the content portable, but of course it still asks you to setup the file and folder structure on your own.
Sorry that I don't have a better answer, but I hope it helps anyway.
PS: One "solution" might also be to use the postinstall step in an npm module's package.json file to create folder structure, copy files to where they should be and so on, but at least to me this feels more like a clumsy workaround than like a real solution.

Peer dependency that is also dev dependency of linked npm module is acting as a separate instance

In my app, I have these dependencies:
TypeORM
typeorm-linq-repository AS A LOCAL INSTALL ("typeorm-linq-repository": "file:../../../IRCraziestTaxi/typeorm-linq-repository"), who has a dev dependency AND a peer dependency of TypeORM
The reason I use a "file:" installation of typeorm-linq-repository is that I am the developer and test changes in this app prior to pushing releases to npm.
I was previously using node ~6.10 (npm ~4), so when I used the "file:" installation, it just copied the published files over, which is what I want.
However, after upgrading to node 8.11.3 (npm 5.6.0), it now links the folder rather than copying the published files.
Note, if it matters, that my environment is Windows.
The problem is this: since both my app and the linked typeorm-linq-repository have TypeORM in their own node_modules folders, TypeORM is being treated as a separate "instance" of the module in each app.
Therefore, after creating a connection in the main app, when the code that accesses the connection in typeorm-linq-repository is reached, it throws an error of Connection "default" was not found..
I have searched tirelessly for a solution to this. I have tried --preserve-symlinks, but that does not work.
The only way for me to make this work right now is to manually create the folder in my app's node_modules and copy applicable files over, which is a huge pain.
How can I either tell npm to NOT symlink the "file:" installation or get it to use the same instance of the TypeORM module?
I made it work pretty easily, although I feel like it's kind of a band-aid. I will post the answer here to help anybody else who may be having this issue, but if anybody has a more proper solution, feel free to answer and I will accept.
The trick was to link my app's installation of TypeORM to the TypeORM folder in my other linked dependency's node_modules folder.
...,
"typeorm": "file:../../../IRCraziestTaxi/typeorm-linq-repository/node_modules/typeorm",
"typeorm-linq-repository": "file:../../../IRCraziestTaxi/typeorm-linq-repository",
...

Overriding default npm registry for single packages served from local folder

I'm adding several dependencies to a project that currently uses the default npm registry. Obviously the dependencies cannot be resolved since the packages are not found there.
I'm wondering if I can provide the packages via a folder or zip file instead and tell npm to bypass the registry for certain dependencies and take the packages directly from the folder. I want to avoid to setup my own registry.
Sinopia seems to be a lightweight solution for the problem. It is a private repository server that allows to use private packages, cache the npmjs.org registry, and to override public packages.
Disclaimer: I haven't tried it because my problem was solved by another private registry I didn't know at the time of writing the question. However, maybe it helps someone else.

Dropbox selective syncing - pattern matching?

I'm using Dropbox on daily basis and put my programming projects in there.
It works great, but once I got many projects my /node_modules dir's are putting
a struggle on Dropbox. It's syncing process starts to be slow and it eats up CPU time.
Is there any way to do a selective sync based on directory name or a mask pattern?
Would be nice to have to a .gitignore equivalent to configure.
Any 3rd party software for that task?
There is a way to selectively sync but I don't believe it has any advanced rules like you're describing:
https://www.dropbox.com/help/175/en
2 way to resolve this problem:
You can put node_modules upper then project directory in files tree. For example:
Project dir: c:/prj/myProjWrapper/myProj
In the c:/prj/myProjWrapper put package.json and make npm install here, NodeJS recursively will find it.
Win and Linux only, not for Mac! In project dir create .ds_store folder (it is not sync by dropbox). Put package.json in to it and do npm install. You must set NODE_PATH=./.ds_store/node_modules;. when starting NodeJS