Can't install a scoped package I published to a npm registry in GCP - npm

I published several npm packages to a private npm registry hosted in GCP and I can see all versions in the registry. Yet I can't download the package since the install command throws an E404.
NPM throws a generic 404 Error when trying to download the page:
npm ERR! code E404
npm ERR! 404 Not Found - GET https://europe-north1-npm.pkg.dev/[Project]/#[scope]/[packagename]/-/#[scope]/[packagename]-0.1.0.tgz
npm ERR! 404
npm ERR! 404 '#[scope]/[packagename]#https://europe-north1-npm.pkg.dev/[Project]/#[scope]/[packagename]/-/#[scope]/[packagename]-0.1.0.tgz' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.
Opening the url that is tring to resolve returns this error:
{"error":"Permission \"artifactregistry.repositories.downloadArtifacts\" denied on resource \"projects/[Project]/locations/europe-north1/repositories/#[scope]\" (or it may not exist)"}
{"error":"Unmatched scope name: \"\" != \"#[scope]\""}
Additonally, I checked and my account does in fact have to downloadArtifacts role.
I can't figure out why it's unable to resolve the URL since with the same config I'm able to both publish and even run npm view the package.
I've tried creating brand new packages, specifying a version, resetting the auth token and yet for some reason I'm still unable to download packages.
If it helps, my current .npmrc file looks like this:
registry:https://registry.npmjs.org/=true
#[scope]:registry=https://europe-north1-npm.pkg.dev/[project]/[repo name]/
//europe-north1-npm.pkg.dev/[project]/[repo name]/:_authToken=[gcp auth token]
//europe-north1-npm.pkg.dev/[project]/[repo name]/:always-auth=true
//registry.npmjs.org/:_authToken=[npm auth token]

Turns out this was not a GCP specific issue, but a Yarn Berry issue when publishing to GCP.
The tarballURL that is being generated uses a truncated registry URL which is why i was getting a 404 when trying to install the package.
A fix for this was proposed here.
https://github.com/yarnpkg/berry/pull/3513

I had an error similar to this one.
I did the entire repository creation process and created a new project to test the package.
When I tried npm install my-package I got a 404 error.
After some tests I discovered that I need the .npmrc file also in the project that will consume the package. It may seem kind of obvious to some people but I didn't know.
Add the returned configuration settings to the .npmrc configuration file in your Node.js projects. This file is usually in the same directory as package.json.
Make sure that you include these settings in Node.js projects for packages that you publish as well as projects that will install dependencies from your npm repository.
Set up authentication for npm

Related

DevOps local npm ajv version has been deleted

We have an issue with devops upstream npm feed. We built and deployed our angular app a few month ago with no issues. We are now trying to build the project but its complaining the below error. This seems to be because the locally stored version of the upstream npm package, ajv version 6.12.6 has been deleted; due to a retention policy. We are unable to get it back, does anyone know how we get it back?
npm ERR! code E404
npm ERR! 404 Not Found - GET https://tfs.pkgs.visualstudio.com/_packaging/myproject/npm/registry/ajv/-/ajv-6.12.6.tgz - Cannot find the package 'ajv' in feed 'npm_***'
npm ERR! 404
npm ERR! 404 'ajv#https://tfs.pkgs.visualstudio.com/_packaging/myproject/npm/registry/ajv/-/ajv-6.12.6.tgz' is not in this registry.
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.
The artifact view shows the below:
Update:
It would seem that there is no way to correct this! as mentioned here https://developercommunity.visualstudio.com/t/deleted-versions-of-upstream-packages-cannot-be-re/1620100
Fairly poor thought on DevOps part! So currently we are now pointing to the standard npm package url (https://registry.npmjs.org/) and not the dev ops artifact one; in order to get it to build again!
As you know, Azure DevOps feed has a immutability constraint. So there doesn't have any way to add the same version in the true sense at this moment.
Currently available methods are:
1, Create a new feed and download the package via this feed.
2, Download the package change the package info and publish to Artifact feed(As another package.)
3, Directly use the public source.
There is a suggestion ticket about this feature in official community:
Relax Azure Artifacts restriction on re-publishing versions
You can upvote this suggestion ticket and share your ideas in it.

Azure Devops: Sudden failure of NPM package download

Since 2 days a download/install (no changes to the env at all) fails in my azure devops pipeline:
npm ERR! code E404 npm ERR! 404 Not Found - GET
https://pkgs.dev.azure.com/.../NPM-Mirror/npm/registry/esprima-fb/-/esprima-fb-15001.1001.0-dev-harmony-fb.tgz
Cannot find the package 'esprima-fb' in feed '' npm ERR! 404 npm ERR! 404
'esprima-fb#https://pkgs.dev.azure.com/.../NPM-Mirror/npm/registry/esprima-fb/-/esprima-fb-15001.1001.0-dev-harmony-fb.tgz'
is not in this registry. npm ERR! 404 You should bug the author to
publish it (or use the name yourself!) npm ERR! 404 npm ERR! 404 Note
that you can also install from a npm ERR! 404 tarball, folder, http
url, or git URL.
which basically points to that package here:
https://www.npmjs.com/package/esprima-fb/v/15001.1001.0-dev-harmony-fb
The package was last changed 7 years ago and worked fine until Wednesday of this week (even the underlying source was not changed for a while).
I tried Node version 16 and 17, both have same effect.
I tried to install the package by
- task: Npm#1
displayName: 'NPM: install'
inputs:
command: 'ci'
workingDir: 'path/'
and
- task: Npm#1
displayName: 'NPM: install'
inputs:
command: 'install'
workingDir: 'path/'
none worked.
I've recognized that this version of the package is not downloaded to the Azure Artifacts Upstream Archive (only an older version is available).
Tried to recreate another Feed, also no effect. Tried to add the dependency directly into the package.json file - all showed no effect at all.
Do you have any idea how to fix that problem? I ran out of ideas.
Azure DevOps feed for npm package sometimes throws incorrect HTTP status code when encountering authentication issue (it should be 401 instead of 404).
Typically npm install issues with Azure DevOps relies on an expired PAT token into your project's .npmrc file.
To be sure npm is always hitting npmjs.org for public packages, I'd suggest scoping your private packages (the ones coming from Azure DevOps feed) and defining the npmjs.org registry for all the others.
#[your_private_package_scope]:registry=[your_feed_url]
registry=https://registry.npmjs.org/

Force NPM to skip package already installed from another source

I'm trying to npm install a package from a heavily proxied Enterprise computer. One of the dependencies of this package tries to install via GitHub, which throws a 407 error.
npm ERR! /usr/bin/git ls-remote -h -t https://github.com/ethereumjs/ethereumjs-abi.git
npm ERR!
npm ERR! fatal: unable to access 'https://github.com/ethereumjs/ethereumjs-abi.git/': Received HTTP code 407 from proxy after CONNECT
I've discussed this with the team that manages these servers- even with proper proxy authentication, access to sites outside of internal registries are very uncommon as a security measure.
As an alternative, I found the package on the NPM registry, and successfully installed it using the internal mirror we have. However, it still tries to install the GitHub version as a dependency of the first package, and then the installation fails. Is there any way to prevent this? I think I've seen something about editing the package's package.json to remove the dependency, but I don't have any experience with this, so I'll only do it if it's the only option to prevent this. Thank you

Issues with publishing npm package on private gitlab registry

I have a (test) npm package named #myScope/test which I want to publish into a private gitlab registry within a gitlab project whose address is
https://gitlab.<something>/myId/test-npm-registry/
with ID 9630
First, I'm a bit confused with npm login and .npmrc. If I understand correctly, answers given to npm login end-up as a couple of lines into .npmrc, is that right?
So essentially, to instruct npm about authenticating to a registry, one can either:
manually edit .npmrc
use npm config set
use npm login
Based on what I read on the gitlab help, I've inserted the following lines into my .npmrc:
#myScope:registry=https://gitlab.<something>/api/v4/projects/9630/packages/npm/
'//gitlab.<something>/api/v4/projects/9630/packages/npm/:_authToken'="${GITLAB_AUTH_TOKEN}"
Note: I had to use the project ID, having issues with the url format.
I have also added the following bit into the package.json :
publishConfig":{
"#myScope:registry": "https://gitlab.<something>/api/v4/projects/9630/packages/npm/"
}
Finally, I've created a token in gitlab, with "api, read_api, read_registry, write_registry" rights and I have assigned the token to the GITLAB_AUTH_TOKEN variable:
GITLAB_AUTH_TOKEN="xk4L7xxvzHuykyKawxQZ"
When I do npm publish I'm getting the following message :
npm ERR! code E401
npm ERR! 401 Unauthorized - PUT https://gitlab.<something>/api/v4/projects/9630/packages/npm/#myScope%2ftest-npm-registry
npm ERR! A complete log of this run can be found in:
The logs doesn't help, nor does the verbose mode. Also, I'm not getting if I have to perform an explicit npm login or if the configuration in the .npmrc (with the token) should do the trick. I actually cannot do npm login because the username has an # character in it and npm refuses it.
Any ideas ?

npm install of private npm registry is failing

I have created a private npm registry using gitlab
I have the following .npmrc file, which is the same .npmrc file that I used to publish with no issues. I've replaced all company values with placeholders.
#myorg:registry=https://gitlab.com/api/v4/packages/npm/
//gitlab.com/api/v4/packages/npm/:_authToken=[[MY_TOKEN_HERE]]
//gitlab.com/api/v4/projects/12345678/packages/npm/:_authToken=[[MY_TOKEN_HERE]]
When I go to install I get the following error message
npm install #myorg/my-package-name-here
npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/#myorg/my-package-name-here - Not found
npm ERR! 404
npm ERR! 404 '#myorg/my-package-name-here#latest' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.
npm ERR! A complete log of this run can be found in:
This to me looks like it's not picking up the .npmrc file as the registry is different.
I have checked and the file is definately .npmrc and not a different extension etc.
Anyone have any ideas?
EDIT
I've done some investigation and it does look like my .npmrc file is being used, however the registry
'https://gitlab.com/api/v4/packages/npm/#myorg/my-package-name-here'
is being redirected to
'https://registry.npmjs.org/#myorg/my-package-name-here'
which is why it will be showing up as this...
It does not explain to me why it can't be found though using the auth token etc that I used to publish?
I've figured it out...
Everything is fine other than #myorg does not match my gitlab org name.
e.g. the gitlab url is something like this:
https://gitlab.com/my-company/...
instead it needs to match the github url, so in the above example
#my-company
After switching this it all kicked into life and I am able to get the npm package.
It is not necessarily true (as stated in the accepted answer) that the scope name "needs to match the github [sic] url" (believe github is meant to be GitLab).
Using GitLab, it is possible to have a scope name that does not match the root of your GitLab URL.
The difference is how the registry is configured in the project's (or user's) .npmrc.
From GitLab:
When you use the instance-level endpoint, only the packages with names in the format of #scope/package-name are available. For example, if your project is https://gitlab.example.com/my-org/engineering-group/team-amazing/analytics, the root namespace is my-org. When you publish a package, it must have my-org as the scope.
The registry config for an instance-level endpoint looks like this:
#scope:registry=https://gitlab.com/api/v4/packages/npm/
...however, if your scope name does not match the root of GitLab URL, you will need a project-level endpoint configuration:
#scope:registry=https://gitlab.com/api/v4/projects/<your_project_id>/packages/npm/.
https://docs.gitlab.com/ee/user/packages/npm_registry/index.html
My cents to help someone.
When you create a repository to handle with Package Registry, then you've created in a group or in your own user. It's very important to observe where you created it. Probably your registry repository url is something like this:
https://your.gitlab.com/{name_of_group_or_name_of_user}/[{optional_subgroup}/]{repository_registry_name}
Now, let's say you've a package in the url:
https://your.gitlab.com/my-group/my-package
And the url registry is:
https://your.gitlab.com/main/registry
The #scope of your my-package needs being #main/my-package, and not #my-group/my-package. If you define wrong #scope here, it gives you 404.
my-package:
{
"name": "#main/my-package",
...
}
The #scope is the first segment of the url of your registry repository path.