I'm working in an environment where artifactory does not have internet access. We would like to use artifactory as a npm registry and host. Is it possible to upload external dependencies and their transitive dependencies?
For example: I'm on a computer with internet access and downloaded webpack and all its dependencies using npm install. Now I go to a different network with artifactory access and want to upload my node_modules Folder. Does that work somehow?
In addition to Artifactory's proxy/caching features, it can also host multiple local repositories (such as npm repositories) in it. This basically means that you can create an npm local repository in Artifactory and deploy any npm *.tgz packages (your dependencies) into this repository and Artifactory will generate the relevant metadata for your client. All you'll need to do is to deploy the relevant packages and configur your npm clients to resolve from Artifactory.
I have recently made an node module that should help with this problem.
You give it a list of packages that you want downloaded and it will download the packages with all dependencies as a tar.gz. It will then save them in the original npm folder structure, and create a tar.gz with everything inside.
You can then take the tar.gz with all your dependencies and deploy it to Artifactory using the deploy wizard.
When you deploy, select the checkbox "Deploy as Bundle Artifact". This will extract the tar.gz of packages and load them into the npm repository. Artifactory will read the package.json of all packages, and will load the relevant information, allowing you to pull packages with npm.
The package is called package-bundle, and can be downloaded from npm using npm install -g package-bundle
To download packages you can run the command pb bluebird express#1.0.1, which will fetch the specified packages, and all the required dependencies.
Related
I have a situation where there was an update in one of npm packages I use in my project, but author didn't publish it on npmjs registry, so up to date code sits in gitlab only.
What would be the best solution to get updated version of code? I believe there is a way to add dependency to project which will be downloaded from gitlab or github public repository directly? Is it possible to compile it like in npmjs as well?
Yes, you could install a dependency from a git repository directly. As can be seen in the npm docs. You can straight install a Git Remote repository like this:
npm install <git remote url>
e.g.
npm install git://github.com/npm/cli.git
But beware that installing directly from the source git might have unintended side effects (missing build files, additional documentation files in general changes to the npmjs Version).
Also installing from the repository I would recommend you install from a specific commit/Tag.
I'm more familiar with the Python ecosystem at this point and have a question about how I can do something with npm that I'm used to doing with pip.
Let's say I have a wheel for a particular Python package, as well as a wheel file for each of the Python package's dependencies. And let's say I have all these wheel files in a folder called /path/to/wheel/files. To install this package and all of its dependencies, I could run something like pip install /path/to/wheel/files/*.whl --no-deps, where --no-deps keeps me from having to install the various dependencies in the proper order.
Does npm have an equivalent to this? I'm using npm-offline-packager to create a tarball that contains a Node package (as its own tarball) and all of its dependencies (as their own tarballs). I know I can tell npm install to install a particular tarball. However, when I do this, it tries pulling in the required dependencies from the online NPM registry instead of pulling in the dependencies from the tarballs I already have.
Ideally, I'd like npm install to use the tarballs to add the main package to my project's package.json while adding the package's dependencies to my project's package-lock.json. And of course, I'd also like the main package and all its dependencies to be installed to my project's node_modules directory as well.
TL;DR Does npm have something equivalent to pip install /path/to/wheel/files/*.whl --no-deps?
I'm responding to my own question here, but note that my answer is only applicable to my particular use case and may not be applicable in general.
For my use case, I have access to two computers: one that has access to the internet and one that doesn't. For the machine that doesn't have access to the internet, I was attempting to use Verdaccio as a way of creating a self-hosted NPM registry. However, publishing packages to Verdaccio wasn't working because it kept trying to pull in the package's dependencies from the public NPM repository. The solution was to remove all references to "npmjs" in Verdaccio's config file (which, for me, Verdaccio created at ~/.config/verdaccio/config.yaml).
So, in case anyone needs to do development on a machine that doesn't have access to the internet, the process for setting up Verdaccio looks something like this:
On the machine that has access to the internet, create an NPM project using npm init (I called my project "verdaccio_runner"). The reason I did this is because, without already having an NPM registry on the machine that doesn't have access to the internet, it was hard doing a global install of Verdaccio.
Run npm install verdaccio to install Verdaccio to the NPM project that was created in the previous step.
Transfer this project over to the machine that doesn't have access to the internet.
Once it's transferred over, run Verdaccio from the project like this: npx verdaccio.
Quit out of Verdaccio.
Remove all references to "npmjs" from the config file that Verdaccio created (again, mine was at ~/.config/verdaccio/config.yaml).
Run Verdaccio again to pull in those changes.
Tell NPM where your private registry is: npm config set registry http://localhost:4873/.
Add yourself as a user by running npm adduser and by then filling out the information you're prompted for.
And the process for publishing packages to Verdaccio on a machine that doesn't have access to the internet looks like this:
For the package you want to install, on the machine that has access to the internet, run npo fetch <package name> --no-cache (assuming you've already done a global install of npm-offline-packager on the machine that has internet access).
Bring the tarball that npo created for you over to the machine that doesn't have internet access.
Untar the tarball.
From the directory that's created, run for file in ./*.tgz; do npm publish $file; done.
The published packages can now be npm installed to projects on the machine that doesn't have internet access.
Note: in order for Verdaccio to be accessible to other machines on the private network, I also had to add the following to Verdaccio's config file:
listen:
0.0.0.0:4873
Is it possible to use gitlab's package repository to feed our npm packages as well as public packages online.
On proget it is possible to register common npm packages and my private npm packages under the same URL using a proxy. Is it possible to do the same with Gitlab so that pointing to gitlab's repository in the .npmrc would be enough to install all the dependencies ?
Yes, you can have a different registry for your personal packages and e.g. company packages. You can reference them by #my-gitlab-username/foo-package or #company/bar-package.
NPM packages hosted on npmjs.com which get installed by npm install <package> will always be resolved if the lookup on your provided Gitlab package registry fails. Usually you do not have to provide a separate proxy.
Multiple private/non-public registries can be targeted by using npm install #company/<package>. So there should be no issue in targeting multiple Proget and/or Gitlab npm registries at the same time.
Authentification is described here: https://stackoverflow.com/a/42648251/4236831
I'm working on creating a local repository that will contain all packages I use in my project, so I can have those packages installed on a machine that does not have access to the internet. I think of the repository that I could clone on the machine and run yarn install to have all the packages available in the project from the local repository. How can I do that? Similar question was asked here Using npm how can I download a package as a zip with all of its dependencies included in the package
There's not enough information in your question to fully understand your situation, but if you commit your node_modules directory to the repository, the modules will be there without the user having to run npm or yarn to install them. This assumes the user will run code from the repo workspace and that there aren't any modules that require a compilation step or other build step that may be platform-specific. But if they're all plain ol' JavaScript modules, you should be fine.
If you want to have all the modules as a separate repo rather than checking in node_modules, I can offhand think of two ways this might work.
Have the packages repo be a check-in of a fully installed node_modules directory. Then make that repo a Git submodule of the main repo that gets cloned as node_modules in the main repo.
Use npm pack to create .tgz files for each package you need. Store those files in the packages repo. Clone that repo into a known path on your target machine. Have the main repo install via path names. For example, if you run npm install /var/packages/foo-1.0.0.tgz, it will add a line to your package.json that might look something like this: "foo": "file:../../../var/packages/foo-1.0.0.tgz". In that case, npm install will install from that path rather than over the network.
I've setup an Nexus (2.10.0) NPM repository and administrate an user with full access to this repo.
By setting the "_auth" and "always-auth" param in my local .npmrc the publishing and reading of npm artifacts are working well. But if i try to unpublish or deprecate an npm artifact i get still an error "...This request requires auth credentials. Run npm login and repeat the request...".
What is the preferred way to remove artifacts from an nexus npm repository?
thx,
David
Unpublish is currently not supported. Details see https://issues.sonatype.org/browse/NEXUS-6892
Also keep in mind that is not considered good practice to use unpublish as you can see from the npm documentation itself linked in the issue.
Unfortunately, Nexus 2.11 doesn't support npm unpublish or npm deprecate.
If the goal is to make your previously published versions of npm-packages inaccessible:
You can delete the packages from Nexus by manually removing the tarballs from .../nexus/storage/<your_npm_repo>/<your_package>/-/ on your Nexus server and creating a scheduled task to 'rebuild hosted npm metadata':
The npm metadata for a hosted repository
can be rebuilt based on the components found in the storage of a
hosted repository. The task can serve as a recovery tool in cases
where the npm metadata database got corrupted or the component storage
was created manually or via some external process like e.g. an rsync
copying.
This also works great for injecting old versions of packages into your npm-repo. (To get these tarballs, use npm pack.)