Verdaccio: how to publish to custom server from Github Actions with proper credentials? - npm

I have a working verdaccio server hosted on a google cloud server. I am able manually publish to it, but am struggling to create a GitHub Action to publish to it when I push to master branch.
I have a script that works perfectly when publishing to npmjs public repo. Here is the relevant part that works for npmjs.org
- name: Publish to npm
if: steps.semantic.outputs.new_release_published == 'true'
run: |
yarn install
git checkout upm
npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
Now, for my own server, I have included the following addition in package.json:
"publishConfig": {
"registry": "http://my.ip.0.0:port"
},
And then in the repositories secrets, I have created an NPM_TOKEN secret with my user's token copied from my computer's .npmrc file after logging in.
I'm getting the following error from the Github Actions result:
npm ERR! code E401
npm ERR! Unable to authenticate, your authentication token seems to be invalid.
npm ERR! To correct this please trying logging in again with:
npm ERR! npm login
So I'm clearly not authenticating properly.
I tried (on the server's cli) using npm token create but it gave me an unauthorized error, and I tried the same on my computer locally after logging in too, and got the same error.
How can I authenticate my Github Actions publish to my custom Verdaccio server? I'm pretty new to this whole CI business, so I suspect I'm missing something quite basic. I suspect I'm doing it wrong using NPM_TOKEN, but it worked fine to publish to npmjs.org public repo.
Again, I can manually publish using npm publish from the terminal on my Mac (after logging into custom server with npm login), so I know that the server is set up properly.

After much googling, I found a solution from this tutorial https://remysharp.com/2015/10/26/using-travis-with-private-npm-deps
It's not written for GitHub Actions but the same procedure worked.
First, you need to login to your private server from your computer. In your home folder look at the .npmrc file (turn on show hidden files).
add this line to the yaml action file:
echo "//YOURREGISTRYADDRESS/:_authToken=\${NODE_AUTH_TOKEN}" > .npmrc
Note that it should actually be NODE_AUTH_TOKEN, NOT your actual token.
The part in the quotes should mostly match the entry in your .npmrc file (without the token).
So now it looks like this
- name: Publish to npm
if: steps.semantic.outputs.new_release_published == 'true'
run: |
yarn install
git checkout upm
echo "//YOURREGISTRYADDRESS/:_authToken=\${NODE_AUTH_TOKEN}" > .npmrc
npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
Then in the Settings -> Secrets part of your GitHub repo, add a secret called NPM_TOKEN and paste in the auth token value from the .npmrc. It's a long series of letters and numbers.
Now this script should properly log in. Apparently the issue is that the default Verdaccio authorization plugin expects it to be used interactively. This line basically creates an .npmrc file on the fly and populates it with the correct info, as if you've already logged in interactively. The file isn't actually created though, and disappears after running, which is a nice touch. It also is pretty secure since it stores the token in the secrets part of the repo. The link above does a better job explaining it, so check it out!

Related

Setting Nexus auth token from the commandline

I figured this would be a problem that has been solved a million times over, but I just can't find the solution. I wish to setup my Java Maven project to install Angular dependencies from my private Nexus server. I use the frontend-maven-plugin to install a new npm every time, so the configuration must be available for that npm for it to work.
I know I can add the following to my .npmrc file and it works:
registry = http://nexus.global.dns/repository/npm-all/
_authToken = NpmToken.xxx
always-auth = true
The problem I have with this solution is that the auth token gets checked into git and that I have to remove it every time I work outside of my network, where I do not have access to the nexus server. This happens for example when I am developing something for the frontend away from home as my nexus server is not on the cloud. So I wish for nexus to be used only by my jenkins pipelines which will use the frontend-maven-plugin.
I figured I would set the registry the commandline way but that is proving to be a challenge. In my frontend-maven-plugin I have set up executions that run the following lines:
npm set registry http://nexus.global.dns/repository/npm-all/
npm //nexus.global.dns/:_authToken=${NEXUS_NPM_AUTH_TOKEN}
npm install --no-package-lock
This returns a 401 error as it is not able to authenticate: Unable to authenticate, need: BASIC realm="Sonatype Nexus Repository Manager"
My next attempt was to remove the explicit mention of the repository I want to use from Nexus, because maybe it can figure out which one it should use:
npm set registry http://nexus.global.dns
npm //nexus.global.dns/:_authToken=${NEXUS_NPM_AUTH_TOKEN}
npm install --no-package-lock
Authentication now seemingly works, but it is trying to pull the dependencies from http://nexus.global.dns as expected. I get the following error: 404 Not Found - GET http://nexus.global.dns/#angular-builders%2fjest
If I navigate to http://nexus.global.dns/repository/npm-all/#angular-builders%2fjest instead I do get the dependency's package.json as a response.
Clearly the registry should be http://nexus.global.dns/repository/npm-all/ but I can't get authentication to work with the command line. I have also tried:
npm //nexus.global.dns/repository/npm-all/:_authToken=${NEXUS_NPM_AUTH_TOKEN}
but this does not work either. How do I set the auth token for my Nexus private server through the command line?

Why can't I publish this specific package to a private NPM registry hosted by Verdaccio?

I'm using Verdaccio to host a private NPM registry on a machine that doesn't have access to the internet. So far, I've successfully published thousands of packages to the registry. However, when trying to publish hoist-non-react-statics-3.3.2-latest.tgz, I get the following error: "This command requires you to be logged in. You need to authorize this machine using 'npm adduser'." The thing is, I have added myself as a user (which is why I've been able to publish so many packages already), and I've confirmed that I'm logged in via npm whoami. I've also done an npm logout and an npm login. I've also tried turning off user authentication in Verdaccio (https://verdaccio.org/docs/authentication/). Unfortunately, nothing I've tried is letting me publish this package to the registry. Any ideas?
After digging into node/lib/node_modules/npm/lib/publish.js, I realized the registry it's trying to use when checking that I'm authenticated was the public NPM registry, not my self-hosted registry. The solution was to remove these lines of code from hoist-non-react-statics's package.json file before publishing it: https://github.com/mridgway/hoist-non-react-statics/blob/master/package.json#L49-L51.

Azure DevOps pipeline - authentication failure when trying to connect to private npm registry

I am trying to use a npm package from a private repository hosted in a different Azure DevOps organisation. I am getting the following error on npm install step:
npm ERR! code E401
npm ERR! Unable to authenticate, need: Bearer authorization_uri=https://login.windows.net/2dfb2f0b-4d21-4268-9559-72926144c918, Basic realm="https://pkgsprodcus1.pkgs.visualstudio.com/", TFS-Federated
My project .npmrc file looks like this:
#{scope}:registry=https://pkgs.dev.azure.com/{ORG}/{PROJECT}/_packaging/{FEED}/npm/registry/
always-auth=true
strict-ssl=false
I have followed the documentation on setting up the PAT token with Packaging Read & Write permission
I have tried connecting by setting up a service connection which did not work.
I have also tried to by adding the credentials to the project .npmrc file and that does not work either.
.npmrc file with creds:
#{scope}:registry=https://pkgs.dev.azure.com/{ORG}/{PROJECT}/_packaging/{FEED}/npm/registry/
always-auth=true
strict-ssl=false
//pkgs.dev.azure.com/{ORG}/{PROJECT}/_packaging/{FEED}/npm/registry/:username=ANYTHING-BUT-EMPTY
//pkgs.dev.azure.com/{ORG}/{PROJECT}/_packaging/{FEED}/npm/registry/:_password=BASE64-ENCODED-PAT
//pkgs.dev.azure.com/{ORG}/{PROJECT}/_packaging/{FEED}/npm/registry/:email={EMAIL}
/pkgs.dev.azure.com/{ORG}/{PROJECT}/_packaging/{FEED}/npm/registry/:always-auth=true
I have regenerated the PAT numerous times with different scopes but none worked. (I am generating the PAT from User Settings -> Security in Azure DevOps)
I am able to connect to the feed from my local machine using the vsts-npm-auth package.
Azure DevOps pipeline - authentication failure when trying to connect to private npm registry
According to the document Set up your client's npmrc, we need set the .npmrc containing the credentials in $home for Linux or Mac systems or $env.HOME for win systems in development machine instead of setting it in your the project .npmrc file.
On your development machine, you will also have a .npmrc in $home for
Linux or Mac systems or $env.HOME for win systems. This .npmrc should
contain credentials for all of the registries that you need to connect
to. The NPM client will look at your project's .npmrc, discover the
registry, and fetch matching credentials from $home/.npmrc or
$env.HOME/.npmrc. Credential acquisition will be discussed in the next
section.
And:
You should have a project specific .npmrc containing only your feed's
registry information that you discovered from the "Connect to Feed"
dialog. There should be no credentials in this file and the file
itself is usually adjacent to your project's package.json.
Besides, if you want use the hosted agent, you could use the npm Authenticate task to certification.
Hope this helps.
In my case, I needed to add an explicit npmAuthenticate step to log-in to the private NPM registry using the pipeline's credentials before the step in which npm install / yarn install was called, like so:
- task: npmAuthenticate#0
inputs:
workingFile: "$(Build.SourcesDirectory)/path/to/my/.npmrc"

TeamCity and NPM login via OAuth token

I'm trying to integrate TeamCity Server (Windows) with my private NPM repo. I have create an OAuth token.
But I can't find any examples how to use the token in regards to NPM login alone or via TeamCity?
SOLVED!
In a PowerShell (run as ADMIN) run this:
npm login --registry=https://registry.npmjs.org/
--scope=#your_npm_handle
(NB. Replace the your_npm_handle in the command above with YOUR private NPM handle!)
Enter your username, password and NPM e-mail when prompted
You should now see a successful login to NPM
Now run:
npm config set always-auth true
If you now run:
npm config edit
you should see something like this / 3 lines of code in the file .npmrc
#your_npm_handle=https://registry.npmjs.org/
//registry.npmjs.org/:_authToken=xxxx-xxxx-xxxx-xxxx
always-auth=true
Please remember: To use a placeholder to your AUTH token ID and place the actual AUTH token in an .env file and only ref. to it in the .npmrc file). Replace this: authToken=xxxx-xxxx-xxxx-xxxx with this: authToken=${NPM_TOKEN} and in your .env file add this: NPM_TOKEN=xxxx-xxxx-xxxx-xxxx
Add this .npmrc file in the ROOT of your project.
Commit and publish the .npmrc file to source control (not the .env file!)
TeamCity will now download the .npmrc file together with the rest of your source code and run with success because it will use the NPM config file and thereby be able to access your private NPM repo and download all you private #packages.
Remember to activate F2A to your NPM account + to your NPM AUTH tokens.
Be aware this solution ONLY works on multiple machines as long as none of the machines don't invalidate this single AUTH token. In this case use a separate AUTH token setup for each machine or simply use NPM LOGIN instead each time.
I stumbled upon this while dealing with something similar and i thought sharing a bit, cause the accepted answer is pretty insecure:
a) You should not push your token/keys into a version control. Period. This is a security flaw.
b) You don't need to log in via npm login to retrieve your key. You should log in to npmjs.com where you will see a section called "Access Tokens". There you can create tokens to use for this situation and mark them as read-only as well. (or you can use command line to create tokens as well)
c) Teamcity offers "parameters". You can set up a secret there to use during the build. You can do this at root or project level. Create an "environment" parameter and mark it as read-only and as a password so it is secure.
d) For example if you have a docker build, pass it in as a --build-arg NPM_TOKEN=%env.yourTokenName% as additional parameter, then ${NPM_TOKEN} can be used in your .npmrc file if you set it up in your Dockerfile correctly (ARG NPM_TOKEN)

401 on npm whoami when copying _authToken in .npmrc

As stated to provide npm authentication for deployment, I tried copying my .npmrc file containing the //registry.npmjs.org/:_authToken=00000000-0000-0000-0000-000000000000 line.
However, when I try npm whoami I get a 401, and npm install on my private modules also doesn't work.
I tried running npm login after on the same machine and everything worked. Then I changed the _authToken back to the original one that was failing, rather than the newly created one. It still worked after doing this. So clearly there is something going on with auth more than just the token. Does NPM keep track of authorized IPs or something else I'm unaware of? I'm trying to setup my deployment to be able to access my private repos.