Which settings do I need to change to install packages from another private repo in Github actions? - npm

We've got a monorepo (call it A) that has several packages published to npm.pkg.github.com. I just started another repo (B), and would like to pull in one of the packages from A, but I'm getting a 403 during the GH Actions build process. Both repos are under the same organization. Here's the relevant bit of my action:
- name: authorize github npm
run: echo "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}" >> .npmrc
- run: yarn install --frozen-lockfile
The .npmrc exists (and is checked in), and should be right:
#<org-name>:registry=https://npm.pkg.github.com
I can "fix" the problem by adding a PAT to the repository secrets and using that instead of GITHUB_TOKEN but I'd rather not.
Is there something I need to change in the settings for repo A to let its packages be accessible from repo B?

Related

AWS install github private package in codeBuild

Hi I have codepipeline to deploy my angular app, and in that app I am using my private github package. Everything is working locally etc. But on codeBuild I have no idea how to register into github package repository.
my buildspec looks like:
version: 0.2
env:
variables:
S3_BUCKET: "{{s3_bucket_url}}"
BUILD_ENV: "{{BUILD_ENV}}"
BUILD_FOLDER: "dist"
phases:
install:
runtime-versions:
nodejs: 14
pre_build:
commands:
- echo Installing source NPM dependencies...
- npm install
- npm install -g #angular/cli
build:
commands:
- echo Build started on `date` with $BUILD_ENV flag.
- ng build $BUILD_ENV
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- '**/*'
base-directory: 'dist*'
if fails on npm install because 404 Not Found - GET https://registry.npmjs.org. For example in github actions I just simply define registry-url: 'https://npm.pkg.github.com' and thats correct.
Thanks for help :)
It fails because, in the execution context of the CodeBuild process, access to the repo containing the GitHub package is restricted, so it can't find the package because it doesn't have access to the repo's packages. You will need to authenticate to the GitHub Package API.
One way to authenticate is to create a Personal Access Token, include it in your CodeBuild Environment by linking a secret in the SecretsManager, then accessing that token in your buildspec script in the env section:
Create a personal access token: In GitHub, create a Personal access token with the read:packages permission. Here's a link to a tutorial on how to do that.
Register token as a secret in Secrets Manager: In SecretsManager, create a secret with one entry. Name the key of the entry GH_PERSONAL_ACCESS_TOKEN, and in the value field, provide the token that you created in step 1. Pick a descriptive name for your secret (something like codebuild/gh_token). Take note of the secret's name.
Authenticate to GitHub Packages using the Personal Access Token: In your buildspec script, you will need to retrieve the secret containing your Personal Access Token, then use that to authenticate before you run the npm install command:
env:
secrets-manager:
GH_PERSONAL_ACCESS_TOKEN: {SECRET_ARN}:PERSONAL_ACCESS_TOKEN # <- replace {SECRET_ARN} with arn of secret
phases:
#...
pre_build:
commands:
- echo Installing source NPM dependencies...
# this is needed to set the url where the package is located
- npm config set #OWNER:registry https://npm.pkg.github.com # <- replace OWNER with the organization/owner name
# this is needed to set the personal access token that we created
- npm config set //npm.pkg.github.com/:_authToken $GH_PERSONAL_ACCESS_TOKEN
- npm install
- npm install -g #angular/cli

Verdaccio: how to publish to custom server from Github Actions with proper credentials?

I have a working verdaccio server hosted on a google cloud server. I am able manually publish to it, but am struggling to create a GitHub Action to publish to it when I push to master branch.
I have a script that works perfectly when publishing to npmjs public repo. Here is the relevant part that works for npmjs.org
- name: Publish to npm
if: steps.semantic.outputs.new_release_published == 'true'
run: |
yarn install
git checkout upm
npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
Now, for my own server, I have included the following addition in package.json:
"publishConfig": {
"registry": "http://my.ip.0.0:port"
},
And then in the repositories secrets, I have created an NPM_TOKEN secret with my user's token copied from my computer's .npmrc file after logging in.
I'm getting the following error from the Github Actions result:
npm ERR! code E401
npm ERR! Unable to authenticate, your authentication token seems to be invalid.
npm ERR! To correct this please trying logging in again with:
npm ERR! npm login
So I'm clearly not authenticating properly.
I tried (on the server's cli) using npm token create but it gave me an unauthorized error, and I tried the same on my computer locally after logging in too, and got the same error.
How can I authenticate my Github Actions publish to my custom Verdaccio server? I'm pretty new to this whole CI business, so I suspect I'm missing something quite basic. I suspect I'm doing it wrong using NPM_TOKEN, but it worked fine to publish to npmjs.org public repo.
Again, I can manually publish using npm publish from the terminal on my Mac (after logging into custom server with npm login), so I know that the server is set up properly.
After much googling, I found a solution from this tutorial https://remysharp.com/2015/10/26/using-travis-with-private-npm-deps
It's not written for GitHub Actions but the same procedure worked.
First, you need to login to your private server from your computer. In your home folder look at the .npmrc file (turn on show hidden files).
add this line to the yaml action file:
echo "//YOURREGISTRYADDRESS/:_authToken=\${NODE_AUTH_TOKEN}" > .npmrc
Note that it should actually be NODE_AUTH_TOKEN, NOT your actual token.
The part in the quotes should mostly match the entry in your .npmrc file (without the token).
So now it looks like this
- name: Publish to npm
if: steps.semantic.outputs.new_release_published == 'true'
run: |
yarn install
git checkout upm
echo "//YOURREGISTRYADDRESS/:_authToken=\${NODE_AUTH_TOKEN}" > .npmrc
npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
Then in the Settings -> Secrets part of your GitHub repo, add a secret called NPM_TOKEN and paste in the auth token value from the .npmrc. It's a long series of letters and numbers.
Now this script should properly log in. Apparently the issue is that the default Verdaccio authorization plugin expects it to be used interactively. This line basically creates an .npmrc file on the fly and populates it with the correct info, as if you've already logged in interactively. The file isn't actually created though, and disappears after running, which is a nice touch. It also is pretty secure since it stores the token in the secrets part of the repo. The link above does a better job explaining it, so check it out!

Retrieve a Github Registry npm package from a Github Action

I have successfully deployed #myorganization/my-super-lib as a GH npm package. Now I would like to use it in an other GH project.
This is a private lib, and a private project (important to remember, as the point is moot with public ones).
Locally, no problem, just add the correct authtoken config with my personal token in my npm config.
Now, when using Github Actions, I am not sure what the elegant solution would be for a token to access the library package. The documentation says to use a personal one, but what if I leave the org? I could also use a dedicated technical account just for that, but that doesn't seem like the "right" solution.
Has anyone got a better idea?
Edit after comments: At first I thought that I could simply use the GITHUB_TOKEN, but it is restricted to access only the current repository, and this is logical from a security perspective after thinking about it.
There is currently no better option than using a personal access token, i.e.:
Create a personal read:packages access token from an account that has read access
Insert that token as "Secret" to the repository (where you execute the Github Actions)
Access the token via in the Secrets in the Github Actions Workflow to authenticate and install the dependency stored in the Github Registry
to use Github actions to install private github package you need also at
actions/setup-node#v1: define the scope of the package registry
npm install: use personal access token with read, repo access. This token should be created by any github account that has access to the private github package that you use and stored as secret to the repo that uses that action.
Note: the --ignore-scripts is an optional flag that adds extra protection against malicious scripts that can steal your personal access token
example:
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1
with:
node-version: '12.x'
registry-url: 'https://npm.pkg.github.com'
scope: '#antecha'
- run: npm install --ignore-scripts
env:
NODE_AUTH_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
UPDATE:
example repo: https://github.com/antecha/survey
Personal Access token as I mentioned can be created by ANY account that has access to the private registry. For orgs best is to create an admin-profile/account at github with only purpose to manage such tokens or/and org's github authentication to 3rd party apps.
I personaly use that script (.github/workflows/my-super-workflow-file.yml)
name: CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Setup node
uses: actions/setup-node#v1
with:
node-version: '10.x'
registry-url: 'https://npm.pkg.github.com'
- run: npm install --ignore-scripts
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- run: npm test

How to publish/deploy a npm package to custom artifactory

I want to do something like this:
Create an npm package. Basically, a common code which I want to use for all of my projects. Which I created.
But now What I want is, Every time I commit something in git for this project, Jenkins should build it with updated alpha/beta version and should publish to my own artifactory.
Your Jenkins job can be configured to be triggered by a webhook, which would take care of the first part (every time I commit). Depending on which Git server you're using you can find a lot of tutorials how to do that:
For GitHub
For GitLab
For Gogs
please note this is just a random selection of tutorials how to set up the webhook triggers to work with Git servers and by no means an exhaustive list
To publish your package to JFrog Artifactory you can either use the Jenkins Artifactory Plugin, or use the NPM command line. If you want to use the npm command line, you'll need to authenticate first:
# setting the default registry to Artifactory
npm config set registry http://<ARTIFACTORY_SERVER_DOMAIN>:8081/artifactory/api/npm/npm-repo/
# log in
npm login
alternatively you can get a .npmrc file directly from Artifactory using:
curl -u admin:<CREDENTIAL> http://<ARTIFACTORY_SERVER_DOMAIN>:8081/artifactory/api/npm/auth
After that, there are two ways you can push your package to Artifactory:
Edit your package.json file and add a publishConfig section to a local repository: "publishConfig":{"registry":"http://localhost:8081/artifactory/api/npm/npm-repo/"}
Provide a local repository to the npm publish command: npm publish --registry http://localhost:8081/artifactory/api/npm/npm-repo/

How to use yarn with private npm registry in Sonatyoe Nexus OSS?

I ve setup the nexus oss 3 and it looks cool. All my projects are installed by using yarn because of the --pure-lockfile option.
Steps to reproduce the issue:
1. Setup nexus oss 3 with a private npm registry (as in documentation)
2. Disable anonymous access from nexus oss 3 admin panel
3. On a linux server with alpine try to yarn install --pure-lockfile (you must have a package that is hosted on the private repo in package.json)
4. Does not work, return 401 error
I tried everything but i could not manage to make yarn to login to get those packages.
If i use npm install, it works.
Can someone tell me how to make yarn work nexus oss3 using the setup from above?
If npm install is working, then you must have login credentials and repository correctly defined.
Open terminal and run npm login, give your username and password for nexus account. This will create a file ~/.npmrc. Open this file nano ~/.npmrc, output look like
//<repository>:_authToken=NpmToken.<token>
A dummy example:
//test.server.com/repository/npm-group/:_authToken=NpmToken.123456-12345-12345-tok-en0onum
Go to the project directory cd <project_dir>, create a new file .yarnrc, open it nano .yarnrc. Insert the following line, save and exit (Ctrl+O, Ctrl+X) it.
registry "<repository>"
Create another file .npmrc in the same directory <project_dir>. Open, add the following line, save and exsit.
registry=<repository>
always-auth=true
//<repository>:_authToken=NpmToken.<token>
Delete the .npmrc at home directory rm ~/.npmrc.
Now you can download node_modules with yarn or yarn install.
I had same issue with nexus 3 and use this configuration on my .npmrc file:
registry=https://your.nexus.com/repository/some-npm/
always-auth=true
/* basic-auth-token: your user:password in base64 */
_auth=<basic-auth-token>
Hope this help you!
The fact that your requests returns 401 (Unauthorized) means that you should supply credentials when connecting to Nexus.
It is far from being a nice solution but I got it working using
yarn set registry https://user:pwd#your.nexus.host/nexus3/repository/npmjs/
I use yarn 1.4.0 (release candidate). It should also work on 1.3.2, but I cannot test that because 1.3.2 has issues with HTTPS_PROXY env vars.