Yarn can't authenticate with Github Packages - but npm can - npm

What do I need to do to configure yarn to authenticate with Github Packages?
I've got a hello-world npm package that I'm trying to publish privately to github. I want to be able to use yarn instead of npm, but for some reason yarn can't authenticate with the Github Package service:
error Couldn't publish package: "https://npm.pkg.github.com/#myuser/test-models: Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured."
NPM doesn't have any trouble publishing though, so I'm assuming it's some difference in the way yarn processes the config info.
Verbose yarn publish:
$ yarn publish --verbose
yarn publish v1.10.1
verbose 0.257170891 Checking for configuration file "/Users/myuser/sites/test-models/.npmrc".
verbose 0.257665932 Found configuration file "/Users/myuser/sites/test-models/.npmrc".
verbose 0.258189892 Checking for configuration file "/Users/myuser/.npmrc".
verbose 0.258433341 Found configuration file "/Users/myuser/.npmrc".
verbose 0.258899667 Checking for configuration file "/usr/local/etc/npmrc".
verbose 0.259174747 Checking for configuration file "/Users/myuser/sites/test-models/.npmrc".
verbose 0.259487532 Found configuration file "/Users/myuser/sites/test-models/.npmrc".
verbose 0.260003161 Checking for configuration file "/Users/myuser/sites/.npmrc".
verbose 0.260246617 Checking for configuration file "/Users/myuser/.npmrc".
verbose 0.260455656 Found configuration file "/Users/myuser/.npmrc".
verbose 0.260965719 Checking for configuration file "/Users/.npmrc".
verbose 0.263449669 Checking for configuration file "/Users/myuser/sites/test-models/.yarnrc".
verbose 0.263757077 Checking for configuration file "/Users/myuser/.yarnrc".
verbose 0.264027657 Found configuration file "/Users/myuser/.yarnrc".
verbose 0.264629421 Checking for configuration file "/usr/local/etc/yarnrc".
verbose 0.264905752 Checking for configuration file "/Users/myuser/sites/test-models/.yarnrc".
verbose 0.265187834 Checking for configuration file "/Users/myuser/sites/.yarnrc".
verbose 0.265428211 Checking for configuration file "/Users/myuser/.yarnrc".
verbose 0.265682381 Found configuration file "/Users/myuser/.yarnrc".
verbose 0.26612276 Checking for configuration file "/Users/.yarnrc".
verbose 0.268430721 current time: 2020-07-15T16:04:57.727Z
[1/4] Bumping version...
info Current version: 0.0.3
question New version:
info Proceeding with current version: 0.0.3
[2/4] Logging in...
[3/4] Publishing...
verbose 3.449685972 Performing "PUT" request to "https://npm.pkg.github.com/#myuser/test-models".
verbose 3.757506429 Request "https://npm.pkg.github.com/#myuser/test-models" finished with status code 401.
verbose 3.760342835 Error: Couldn't publish package: "https://npm.pkg.github.com/#myuser/test-models: Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured."
at MessageError.ExtendableBuiltin (/usr/local/Cellar/yarn/1.10.1/libexec/lib/cli.js:243:66)
at new MessageError (/usr/local/Cellar/yarn/1.10.1/libexec/lib/cli.js:272:123)
at /usr/local/Cellar/yarn/1.10.1/libexec/lib/cli.js:80723:13
at Generator.throw (<anonymous>)
at step (/usr/local/Cellar/yarn/1.10.1/libexec/lib/cli.js:98:30)
at /usr/local/Cellar/yarn/1.10.1/libexec/lib/cli.js:111:13
at processTicksAndRejections (internal/process/task_queues.js:97:5)
error Couldn't publish package: "https://npm.pkg.github.com/#myuser/test-models: Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured."
info Visit https://yarnpkg.com/en/docs/cli/publish for documentation about this command.
Yarn config:
$ yarn config list
yarn config v1.10.1
info yarn config
{
'version-tag-prefix': 'v',
'version-git-tag': true,
'version-commit-hooks': true,
'version-git-sign': false,
'version-git-message': 'v%s',
'init-version': '1.0.0',
'init-license': 'MIT',
'save-prefix': '^',
'bin-links': true,
'ignore-scripts': false,
'ignore-optional': false,
registry: 'https://registry.yarnpkg.com',
'strict-ssl': true,
'user-agent': 'yarn/1.10.1 npm/? node/v13.11.0 darwin x64',
lastUpdateCheck: 1594760187916
}
info npm config
{
'//registry.npmjs.org/:_authToken': '<auth_token1_here>',
'#myuser:registry': 'https://npm.pkg.github.com',
'//npm.pkg.github.com/:_authToken': '<auth_token2_here>',
python: '/usr/bin/python'
}
✨ Done in 0.11s.
All my config files:
$ cat /Users/myuser/.npmrc
//registry.npmjs.org/:_authToken=[my-npm-token]
$ cat /Users/myuser/sites/test-models/.npmrc
#myuser:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=[my-github-token]
$ cat /Users/myuser/.npmrc
//registry.npmjs.org/:_authToken=[my-npm-token]
$ cat /Users/myuser/.yarnrc
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
lastUpdateCheck 1594760187916

For npm.pkg.github.com/:_authToken, you could try and use the third type of token which is just released yesterday (Oct. 2nd, 2020):
npm automation tokens
npm is introducing a new setting for access tokens to support publishing to the npm registry from CI/CD workflows.
Previously, you could create an access token with one of two settings: read-only, and publish.
A publish token allows you to publish packages, like the name implies, but if you have two-factor authentication (2FA) enabled on your account, you'll be prompted for your one-time passcode.
We recommend that people set up 2FA on their account for added security, but requiring a passcode means that all publishing must be done interactively. Many people want to automate their publish step with a CI/CD workflow.
Today, we've added a third option for access tokens: automation. You can create an automation token in your access token settings page.
Using an automation token will not prompt for a one-time passcode, meaning that you can use it as a secret in your publish workflow. Now you can publish a package directly to the npm registry when you cut a release.
If you're a package maintainer, and you want to require that publishers to your package continue to use two-factor authentication and publish interactively, you can do that in the package settings.
If you already required 2FA, there's no change to this behavior; automation tokens won't be accepted unless you allow them to be.
If you've been waiting to enable two-factor authentication on your npm account because it prevented you from publishing in an automated workflow, you can now set up an automation token and enable 2FA.

Related

Artifactory UI download OK, npm install fail with auth error

How do we get scoped repo using artifactory?
setup npm.fontawesome.com remote repo in artifactory according to the doc
used username TOKEN and password - the actual token
auth check in UI OK
listing of #fortawesome/angular-fontawesome visible in UI
download of artifact via uI OK
npm rc defines scoped repo
#fortawesome:registry=https://artifactory/artifactory/api/npm/npm-fontawesome.com-remote/
Anonymous access granted to 3rdparty repo for users in my env - build machines included
npm -i fails with auth where the system looks to be going directly do npm.fontawesome.com and not my artifactory repo
2999 verbose stack HttpErrorAuthUnknown: Unable to authenticate, need: Basic realm="https://npm.fontawesome.com/",service="npm.fontawesome.com"
2999 verbose stack at /usr/local/lib/node_modules/npm/node_modules/npm-registry-fetch/lib/check-response.js:80:17
2999 verbose stack at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Alas, use of token auth for npm repos is not yet supported by jfrog artifactory (see RTFACT-19164)

Verdaccio: how to publish to custom server from Github Actions with proper credentials?

I have a working verdaccio server hosted on a google cloud server. I am able manually publish to it, but am struggling to create a GitHub Action to publish to it when I push to master branch.
I have a script that works perfectly when publishing to npmjs public repo. Here is the relevant part that works for npmjs.org
- name: Publish to npm
if: steps.semantic.outputs.new_release_published == 'true'
run: |
yarn install
git checkout upm
npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
Now, for my own server, I have included the following addition in package.json:
"publishConfig": {
"registry": "http://my.ip.0.0:port"
},
And then in the repositories secrets, I have created an NPM_TOKEN secret with my user's token copied from my computer's .npmrc file after logging in.
I'm getting the following error from the Github Actions result:
npm ERR! code E401
npm ERR! Unable to authenticate, your authentication token seems to be invalid.
npm ERR! To correct this please trying logging in again with:
npm ERR! npm login
So I'm clearly not authenticating properly.
I tried (on the server's cli) using npm token create but it gave me an unauthorized error, and I tried the same on my computer locally after logging in too, and got the same error.
How can I authenticate my Github Actions publish to my custom Verdaccio server? I'm pretty new to this whole CI business, so I suspect I'm missing something quite basic. I suspect I'm doing it wrong using NPM_TOKEN, but it worked fine to publish to npmjs.org public repo.
Again, I can manually publish using npm publish from the terminal on my Mac (after logging into custom server with npm login), so I know that the server is set up properly.
After much googling, I found a solution from this tutorial https://remysharp.com/2015/10/26/using-travis-with-private-npm-deps
It's not written for GitHub Actions but the same procedure worked.
First, you need to login to your private server from your computer. In your home folder look at the .npmrc file (turn on show hidden files).
add this line to the yaml action file:
echo "//YOURREGISTRYADDRESS/:_authToken=\${NODE_AUTH_TOKEN}" > .npmrc
Note that it should actually be NODE_AUTH_TOKEN, NOT your actual token.
The part in the quotes should mostly match the entry in your .npmrc file (without the token).
So now it looks like this
- name: Publish to npm
if: steps.semantic.outputs.new_release_published == 'true'
run: |
yarn install
git checkout upm
echo "//YOURREGISTRYADDRESS/:_authToken=\${NODE_AUTH_TOKEN}" > .npmrc
npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
Then in the Settings -> Secrets part of your GitHub repo, add a secret called NPM_TOKEN and paste in the auth token value from the .npmrc. It's a long series of letters and numbers.
Now this script should properly log in. Apparently the issue is that the default Verdaccio authorization plugin expects it to be used interactively. This line basically creates an .npmrc file on the fly and populates it with the correct info, as if you've already logged in interactively. The file isn't actually created though, and disappears after running, which is a nice touch. It also is pretty secure since it stores the token in the secrets part of the repo. The link above does a better job explaining it, so check it out!

How do I publish a package to npm using an API key?

NPM gives the ability to create access tokens with the right to publish packages to the NPM registry, assuming that you've set the package settings to "Require two-factor authentication or automation tokens" in "Publishing access" (and so implicitly it requires 2FA to be enabled, I assume).
How do I use this token to publish? I've checked the output of npm publish -- it asks for an OTP code, either as a prompt or a flag, and before I enabled 2FA it asked me for my password again. Using a fixed token would be easier than having to pull up my authenticator app. I could disable 2FA, but getting the token working is preferable.
For verdaccio or similar you can use this:
> npm config set registry http://verdaccio/npm/
# then create and update local .npmrc file:
> npm config set _authToken=%YOUR_ACCES_TOKEN%
# and publish pointing to your registry
> npm publish --registry http://verdaccio/npm/
For proget:
[~]$ npm config set registry http://proget/npm/private-npm
[~]$ npm config set always-auth true
[~]$ npm config set _auth {ENCODEDAPIKEY}
[~]$ npm config set email {email address}
# then as usual
[~]$npm publish --registry {YOUR_REGISTRY}
Instructions are here proget and here verdaccio
The npm registry has 3 different kinds of token: Read-only, Automation, and Publish.
It sounds like you created a Publish token. You need to create an Automation token instead, since this is the only one that bypasses 2FA.

Error installing a package published to Azure DevOps NPM Artifacts in other organization project

I'm doing a proof-of-concept for my organization using Azure DevOps Pipelines to handle our front-end CI builds.
I've created two Angular projects: a library project and an application project that consumes that library. I've each to its own DevOps Project within my Organization, each of which has its own Repo. (e.g., the library Angular code is in My-Org/My-Library's Project's Repo, and the application that consumes that library is in My-Org/My-Application's Project's Repo.)
I've successfully gotten DevOps to publish that library's package to its Artifacts. I've successfully installed that package from Artifacts for my application from the CLI using npm install.
When I try to build the same application using an Azure Pipeline, things start out looking good but then I get warnings:
...
2020-09-25T01:40:22.9633584Z npm verb npm-session b9c6c5c07bc27d0f
2020-09-25T01:40:22.9634637Z npm info lifecycle #<myorganization>/<my-application-package-name>#0.0.0~preinstall: #<myorganization>/<my-application-package-name>#0.0.0
...
2020-09-25T01:40:22.9652940Z npm http fetch GET 200 https://registry.npmjs.org/ansi-colors/-/ansi-colors-4.1.1.tgz 888ms
2020-09-25T01:40:22.9653589Z npm http fetch GET 200 https://registry.npmjs.org/path-exists/-/path-exists-4.0.0.tgz 885ms
...
2020-09-25T01:40:22.9696448Z npm http fetch GET 200 https://registry.npmjs.org/tar/-/tar-6.0.5.tgz 256ms
2020-09-25T01:40:22.9697172Z npm WARN tar ENOENT: no such file or directory, open '/home/vsts/work/1/s/node_modules/.staging/source-map-655ef13e/dist/source-map.js'
2020-09-25T01:40:22.9697948Z npm http fetch GET 200 https://registry.npmjs.org/minizlib/-/minizlib-2.1.2.tgz 254ms
2020-09-25T01:40:22.9698728Z npm WARN tar ENOENT: no such file or directory, open '/home/vsts/work/1/s/node_modules/.staging/#angular/cli-095a8231/commands/build-impl.js'
2020-09-25T01:40:22.9699541Z npm WARN tar ENOENT: no such file or directory, open '/home/vsts/work/1/s/node_modules/.staging/engine.io-client-a00fe2c5/LICENSE'
...
culminating with the unhappy finale:
...
2020-09-25T01:40:23.9366311Z npm WARN tar ENOENT: no such file or directory, open '/home/vsts/work/1/s/node_modules/.staging/rxjs-77a83855/LICENSE.txt'
2020-09-25T01:40:23.9367111Z npm WARN tar ENOENT: no such file or directory, open '/home/vsts/work/1/s/node_modules/.staging/rxjs-77a83855/src/LICENSE.txt'
2020-09-25T01:40:23.9367879Z npm verb unlock done using /home/vsts/.npm/_locks/staging-b6ade8de5fa1f467.lock for /home/vsts/work/1/s/node_modules/.staging
2020-09-25T01:40:23.9369511Z npm verb stack Error: 404 Not Found - GET https://pkgs.dev.azure.com/<MyOrganization>/<My-Library-Project>/_packaging/<My-Library-Project-Artifact-Feed>/npm/registry/#<myorganization>/<my-library-package-name>/-/<my-library-package-name>-0.0.6.tgz
...
2020-09-25T01:40:23.9371625Z npm verb statusCode 404
2020-09-25T01:40:23.9372024Z npm verb pkgid #<myorganization>/<my-library-package-name>#0.0.6
2020-09-25T01:40:23.9372260Z npm verb cwd /home/vsts/work/1/s
2020-09-25T01:40:23.9372620Z npm verb Linux 5.4.0-1025-azure
2020-09-25T01:40:23.9373034Z npm verb argv "/opt/hostedtoolcache/node/12.18.4/x64/bin/node" "/opt/hostedtoolcache/node/12.18.4/x64/bin/npm" "install"
2020-09-25T01:40:23.9373392Z npm verb node v12.18.4
2020-09-25T01:40:23.9373573Z npm verb npm v6.14.6
2020-09-25T01:40:23.9373749Z npm ERR! code E404
2020-09-25T01:40:23.9374536Z npm ERR! 404 Not Found - GET https://pkgs.dev.azure.com/<MyOrganization>/<My-Library-Project>/_packaging/<My-Library-Project-Artifact-Feed>/npm/registry/#<myorganization>/<my-library-package-name>/-/<my-library-package-name>-0.0.6.tgz
2020-09-25T01:40:23.9375074Z npm ERR! 404
2020-09-25T01:40:23.9375516Z npm ERR! 404 '#<myorganization>/<my-library-package-name>#0.0.6' is not in the npm registry.
...
But here's the weird part: if I click on that url in the browser for which the Pipeline is reporting a 404---https://pkgs.dev.azure.com/<MyOrganization>/<My-Library-Project>/_packaging/<My-Library-Project-Artifact-Feed>/npm/registry/#<myorganization>/<my-library-package-name>/-/<my-library-package-name>-0.0.6.tgz---the browser downloads my package!
So to summarize:
When doing an npm install from the command-line for my application project, I'm able to install the package from the library's DevOps Artifacts, and
When clicking on the very URL that the application Pipeline log says is failing, the browser downloads the package from the library's Artifacts, but
The application's Pipeline that runs inside my DevOps Organization can't find it.
I suspect some kind of permissions or authorization issue, but I'm not sure where to go from here.
I've tried the npmAuthenticate#0 task, and indeed the logs say encouraging things like:
2020-09-25T01:40:04.2511306Z ##[debug]Got auth token
..
2020-09-25T01:40:04.2540281Z ##[debug]Created webApi client for https://dev.azure.com/<MyOrganization>/; options: {"proxy":null,"allowRetries":true,"maxRetries":5,"ignoreSslError":false}
2020-09-25T01:40:04.2581233Z ##[debug]Getting URI for area ID <some GUID> from https://dev.azure.com/<MyOrganization>/
2020-09-25T01:40:04.3973124Z ##[debug]Found resource area with locationUrl: https://pkgs.dev.azure.com/<MyOrganization>/
2020-09-25T01:40:04.3976465Z ##[debug]Found serviceUri: https://pkgs.dev.azure.com/<MyOrganization>/
2020-09-25T01:40:04.3978178Z ##[debug]Getting credentials for local feeds
2020-09-25T01:40:04.3978962Z SYSTEMVSSCONNECTION exists true
2020-09-25T01:40:04.3979926Z ##[debug]SYSTEMVSSCONNECTION exists true
2020-09-25T01:40:04.4003325Z ##[debug]Got auth token
2020-09-25T01:40:04.4004250Z ##[debug]Agent.ProxyUrl=undefined
2020-09-25T01:40:04.4005572Z ##[debug]Created webApi client for https://pkgs.dev.azure.com/<MyOrganization>/; options: {"proxy":null,"allowRetries":true,"maxRetries":5,"ignoreSslError":false}
2020-09-25T01:40:04.4007252Z ##[debug]Acquiring Packaging endpoints...
2020-09-25T01:40:04.6490830Z ##[debug]Successfully acquired the connection data
2020-09-25T01:40:04.6502681Z ##[debug]Acquired location
2020-09-25T01:40:04.6503915Z ##[debug]{"PackagingUris":["https://dev.azure.com/<MyOrganization>/","https://pkgs.dev.azure.com/<MyOrganization>/","https://pkgsprodcus1.pkgs.visualstudio.com/","https://pkgs.dev.azure.com/<MyOrganization>/","https://<myorganization>.pkgs.visualstudio.com/","https://pkgs.dev.azure.com/<MyOrganization>/"],"DefaultPackagingUri":"https://pkgs.dev.azure.com/<MyOrganization>/"}
Yet it still fails.
Any suggestions on what I can try?
Thanks!!
P.S. I should add that I have a lot more information I can share including my package.json, .npmrc, and the entire log but I wanted to keep this brief. If you need me to share additional details, please let me know and I'll add them. Thank you!
When connecting to a private project scoped feed from an Azure DevOps pipeline that is in the same organization but in a different project, the project that the feed is scoped to must allow access to the other project's build service. The build service must also be separately added to the feed permissions, regardless of the scope of the feed.
This is a problem with project scoped feed permissions. In short, to access a project scoped feed that is scoped to a project that is different than the project that the pipeline is running in, the project that the pipeline is running in must have access to BOTH the project that the feed is scoped to and the feed itself.
Here's how to set the proper permissions.
Check the project that the pipeline is running in. The build service permission that needs to be added to the feed permission and the feed's project permissions is going to look like something like [Project name] Build Service ([Organization name]).
In the project that the feed is scoped to, go to the permission settings to add the pipeline's project build service ([Project name] Build Service ([Organization name])) to a the Contributors group, or some other group your project may have that allows contributor access to its users.
In the feed permission page, add the [Project name] Build Service ([Organization name]) at least Collaborator access, so packages can be ingested from upstream sources. If you only give read permissions, packages cannot be ingested from upstream sources.

npm publish azure artifacts

I'm trying to publish a scoped package to a private azure devops artifact feed. I followed the instructions here. I have a project .npmrc with the following entries:
#my-scope:registry=https://pkgs.dev.azure.com/my-org/_packaging/my-feed/npm/registry/
#my-scope:always-auth=true
In my global user .npmrc I have the following entries:
prefix=/usr/local
strict-ssl=false
unsafe-perm=true
//registry.npmjs.org/:_authToken="my-real-token"
//pkgs.dev.azure.com/my-org/_packaging/my-feed/npm/registry/:username=${NPM_USERNAME}
//pkgs.dev.azure.com/my-org/_packaging/my-feed/npm registry/:_password="my-real-base64-token"
//pkgs.dev.azure.com/my-org/_packaging/my-feed/npm/registry/:email=${NPM_EMAIL}
//pkgs.dev.azure.com/my-org/_packaging/my-feed/npm/registry/:always-auth=true
#my-scope:registry=https://pkgs.dev.azure.com/my-org/_packaging/my-feed/npm/registry/
//pkgs.dev.azure.com/my-org/_packaging/my-feed/npm registry/:_authToken="my-real-base64-token"
cafile=${NPM_CERT_LOCATION}
When I try: npm publish I get the following error:
Unable to authenticate, need: Bearer authorization_uri=https://login.windows.net/some-guid, Basic realm="https://pkgsprodcus1.pkgs.visualstudio.com/", TFS-Federated
Based on previous posts I see that I might need to do npm login. Executing npm login gives me this error:
npm verb node v6.9.2
npm verb npm v6.8.0
npm ERR! code E400
npm ERR! 400 Bad Request - PUT https://pkgs.dev.azure.com/my-org/_packaging/my-feed/npm/registry/-/user/org.couchdb.user:my-username
I looked at this and this which seemed to be related. However, neither of them worked.
I've tried: curl and curl -u which gave me the following error:
{"$id":"1","innerException":null,"message":"TF400813: Resource not available for anonymous access. Client authentication required.","typeName":"Microsoft.TeamFoundation.Framework.Server.UnauthorizedRequestException, Microsoft.TeamFoundation.Framework.Server","typeKey":"UnauthorizedRequestException","errorCode":0,"eventId":3000}%
I'v also tried with a proxy and a cert. However, with the same results.
This is a bit old, not sure if you are still stuck, but for Windows you can use this npm package: https://www.npmjs.com/package/vsts-npm-auth:
npm install -g vsts-npm-auth
vsts-npm-auth -config path-to-your\.npmrc
Here is a more complete article from Azure DevOps that walks you through setup, .npmrc and auth and publishing: https://learn.microsoft.com/en-us/azure/devops/artifacts/npm/npmrc?view=azure-devops&tabs=windows:
If you are developing on Linux or Mac, vsts-npm-auth is not supported and we recommend generating a token in the following manner for your $HOME/.npmrc
The Connect to feed dialog box generates an appropriately formatted token that you can place into your .npmrc file with a lifespan of 90 days.
From Azure Artifacts, select Connect to feed.
Select npm.
Select Generate npm credentials. Copy the credentials to add them to your user .npmrc file manually. For Windows this is in %USERPROFILE%.npmrc and can be useful if the above method doesn't work. For Linux it is in $HOME/.npmrc.