npm install of private package fails when package has public dependencies - npm

Lets say I have a private npm repository, hosted within JFrog artifactory:
https://my-domain.com/artifactory/api/npm/my-repo.
In this repository I published one npm package: my-package, which builds fine. my-package has a dependency (or more) to public npm packages e.g. lodash.
However, when I create a new project and attempt to install my-package I get the following error:
$ npm install my-package --registry https://my-domain.com/artifactory/api/npm/my-repo
npm ERR! code E404
npm ERR! 404 Not Found - GET https://my-domain.com/artifactory/api/npm/my-repo/lodash - not_found
npm ERR! 404
npm ERR! 404 'lodash^4.17.11' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 It was specified as a dependency of 'my-package'
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\<username>\AppData\Roaming\npm-cache\_logs\2019-04-29T12_47_51_647Z-debug.log
It appears as though npm is searching within my private repository for all the dependencies my-package requires when I specify the --registry option when running an npm install. However, my-package is dependent upon public dependencies, which are not in my private registry.
My Question
How to install an npm package from a private registry that has public dependencies? Perhaps this is also just a JFrog issue?
Any help would be greatly appreciated!

By specifying a registry with: --registry https://my-domain.com/artifactory/api/npm/my-repo npm is attempting to resolve all necessary packages, by name and version, from your private repository location:
domain.com/artifactory/api/npm/my-repo.
To resolve these public dependencies which your private library depends upon you have two options:
Set up a Virtual Npm Registry. (Recommend this approach)
Package all necessary dependencies within your private repository.
A Virtual Repository defined in Artifactory aggregates packages from both local and remote repositories. This allows you to access both locally hosted npm packages and remote proxied npm registries from a single URL defined for the virtual repository.
By setting up a virtual repository which references both your private repository location and the default public npmjs location you will be able to download your private libraries as well as any public npm package by specifying your above mentioned registry.
Since you mentioned JFrog take a look at their confluence page which walks you through the process of creating a virtual repository.
However, if you decide to use option 2 you will have to package all the necessary dependencies within your private repository. Then your private library will be able to properly pull the dependencies it's dependent upon. I would advise against this approach since you will be duplicating work already provided by npmjs and you will additionally have to keep updating your private repository to include new libraries or newer versions of existing libraries.
Hopefully that helps!

My solution...
involved updating my .npmrc file:
I changed registry=https://npm.pkg.my-domain.com
to #my-private-scope:registry=https://npm.pkg.my-domain.com
Here's why
In it, I had specified that registry=https://npm.pkg.my-domain.com (followed by my auth token) so that I could import a private package from the #my-private-scope scope. However since i left at the #my-private-scope: part, I was changing the URL for all decencies, not just the ones that are part of my private organization. I made the change to specify the my-domain URL for only the dependencies at that scope.

Related

npm i failed because of dependency in private registry

I published a private npm package for internal usage (to npm.pkg.github.com).
then i created another private npm package in the same registry which uses the first private package as a dependency. all the other packages are from the default npm registry.
when I try to install the second npm package to a new project, the install fails, because npm i tries to install the dependency (my first private package) from the default npm registry (i assume the .npmrc of my package gets ignored)
how can i fix that ?
what i have tried so far:
install the dependency package before adding the second package (still 404 after the second step)
Use the information from the npmrc file (where you scope the package to a specific registry) in your global .npmrc file on the device you want to install the private dependency package

Force NPM to skip package already installed from another source

I'm trying to npm install a package from a heavily proxied Enterprise computer. One of the dependencies of this package tries to install via GitHub, which throws a 407 error.
npm ERR! /usr/bin/git ls-remote -h -t https://github.com/ethereumjs/ethereumjs-abi.git
npm ERR!
npm ERR! fatal: unable to access 'https://github.com/ethereumjs/ethereumjs-abi.git/': Received HTTP code 407 from proxy after CONNECT
I've discussed this with the team that manages these servers- even with proper proxy authentication, access to sites outside of internal registries are very uncommon as a security measure.
As an alternative, I found the package on the NPM registry, and successfully installed it using the internal mirror we have. However, it still tries to install the GitHub version as a dependency of the first package, and then the installation fails. Is there any way to prevent this? I think I've seen something about editing the package's package.json to remove the dependency, but I don't have any experience with this, so I'll only do it if it's the only option to prevent this. Thank you

Can't install a scoped package I published to a npm registry in GCP

I published several npm packages to a private npm registry hosted in GCP and I can see all versions in the registry. Yet I can't download the package since the install command throws an E404.
NPM throws a generic 404 Error when trying to download the page:
npm ERR! code E404
npm ERR! 404 Not Found - GET https://europe-north1-npm.pkg.dev/[Project]/#[scope]/[packagename]/-/#[scope]/[packagename]-0.1.0.tgz
npm ERR! 404
npm ERR! 404 '#[scope]/[packagename]#https://europe-north1-npm.pkg.dev/[Project]/#[scope]/[packagename]/-/#[scope]/[packagename]-0.1.0.tgz' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.
Opening the url that is tring to resolve returns this error:
{"error":"Permission \"artifactregistry.repositories.downloadArtifacts\" denied on resource \"projects/[Project]/locations/europe-north1/repositories/#[scope]\" (or it may not exist)"}
{"error":"Unmatched scope name: \"\" != \"#[scope]\""}
Additonally, I checked and my account does in fact have to downloadArtifacts role.
I can't figure out why it's unable to resolve the URL since with the same config I'm able to both publish and even run npm view the package.
I've tried creating brand new packages, specifying a version, resetting the auth token and yet for some reason I'm still unable to download packages.
If it helps, my current .npmrc file looks like this:
registry:https://registry.npmjs.org/=true
#[scope]:registry=https://europe-north1-npm.pkg.dev/[project]/[repo name]/
//europe-north1-npm.pkg.dev/[project]/[repo name]/:_authToken=[gcp auth token]
//europe-north1-npm.pkg.dev/[project]/[repo name]/:always-auth=true
//registry.npmjs.org/:_authToken=[npm auth token]
Turns out this was not a GCP specific issue, but a Yarn Berry issue when publishing to GCP.
The tarballURL that is being generated uses a truncated registry URL which is why i was getting a 404 when trying to install the package.
A fix for this was proposed here.
https://github.com/yarnpkg/berry/pull/3513
I had an error similar to this one.
I did the entire repository creation process and created a new project to test the package.
When I tried npm install my-package I got a 404 error.
After some tests I discovered that I need the .npmrc file also in the project that will consume the package. It may seem kind of obvious to some people but I didn't know.
Add the returned configuration settings to the .npmrc configuration file in your Node.js projects. This file is usually in the same directory as package.json.
Make sure that you include these settings in Node.js projects for packages that you publish as well as projects that will install dependencies from your npm repository.
Set up authentication for npm

Issues with publishing npm package on private gitlab registry

I have a (test) npm package named #myScope/test which I want to publish into a private gitlab registry within a gitlab project whose address is
https://gitlab.<something>/myId/test-npm-registry/
with ID 9630
First, I'm a bit confused with npm login and .npmrc. If I understand correctly, answers given to npm login end-up as a couple of lines into .npmrc, is that right?
So essentially, to instruct npm about authenticating to a registry, one can either:
manually edit .npmrc
use npm config set
use npm login
Based on what I read on the gitlab help, I've inserted the following lines into my .npmrc:
#myScope:registry=https://gitlab.<something>/api/v4/projects/9630/packages/npm/
'//gitlab.<something>/api/v4/projects/9630/packages/npm/:_authToken'="${GITLAB_AUTH_TOKEN}"
Note: I had to use the project ID, having issues with the url format.
I have also added the following bit into the package.json :
publishConfig":{
"#myScope:registry": "https://gitlab.<something>/api/v4/projects/9630/packages/npm/"
}
Finally, I've created a token in gitlab, with "api, read_api, read_registry, write_registry" rights and I have assigned the token to the GITLAB_AUTH_TOKEN variable:
GITLAB_AUTH_TOKEN="xk4L7xxvzHuykyKawxQZ"
When I do npm publish I'm getting the following message :
npm ERR! code E401
npm ERR! 401 Unauthorized - PUT https://gitlab.<something>/api/v4/projects/9630/packages/npm/#myScope%2ftest-npm-registry
npm ERR! A complete log of this run can be found in:
The logs doesn't help, nor does the verbose mode. Also, I'm not getting if I have to perform an explicit npm login or if the configuration in the .npmrc (with the token) should do the trick. I actually cannot do npm login because the username has an # character in it and npm refuses it.
Any ideas ?

How do you publish and install a private npm package to GitHub packages as an organisation?

We have a monorepo with a shared package called "#myorganization/common", that is used by both the app and Google cloud functions.
Google cloud functions requires that the package is published to a repo to use it, since we do not publish compiled code, that can include local packages, but directly the raw TypeScript.
I found serveral tutorials but not for the use case of combining private packages and organizations.
I cd into the packages/common directory.
I added the following lines to my packages.json:
"publishConfig": { "registry": "https://npm.pkg.github.com/" },
"repository": {
"url": "git#github.myorganization/common.git"
},
I created a personal Auth Token on GitHub with full repo, packages and org rights.
Then I login to my organization:
npm login --registry=https://npm.pkg.github.com --scope=#myorganization
Then I hit:
npm publish
I get the following error:
npm ERR! code EPRIVATE
npm ERR! This package has been marked as private
npm ERR! Remove the 'private' field from the package.json to publish it.
I want this package to be only usable by organization members. It shall be both used as a local private package within the lerna monorepo, when used by React Native, and used by Google Cloud Function by getting it from GitHub packages. Yet no other people shall be able to access it.
Is it something that is possible ?
Regards
Edit:
I tried to update packages.json, removing private=true and trying to ensure I publish it to my private organization:
"publishConfig": {
"registry": "https://npm.pkg.github.com/:_authToken=xxxx --scope=#myorganization"
},
I now get the error:
npm ERR! code E404
npm ERR! 404 Not Found - PUT https://npm.pkg.github.com/#myorganization%2fcommon - The expected resource was not found.
npm ERR! 404
npm ERR! 404 '#myorganization/common#0.0.7' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
I was facing the exact same issue. Here's what worked for me.
I had to create that organization (i.e. myorganization in your example) on GitHub and a repo with same name (i.e. common in your example) as repo under that organization and it worked.