AWS install github private package in codeBuild - npm

Hi I have codepipeline to deploy my angular app, and in that app I am using my private github package. Everything is working locally etc. But on codeBuild I have no idea how to register into github package repository.
my buildspec looks like:
version: 0.2
env:
variables:
S3_BUCKET: "{{s3_bucket_url}}"
BUILD_ENV: "{{BUILD_ENV}}"
BUILD_FOLDER: "dist"
phases:
install:
runtime-versions:
nodejs: 14
pre_build:
commands:
- echo Installing source NPM dependencies...
- npm install
- npm install -g #angular/cli
build:
commands:
- echo Build started on `date` with $BUILD_ENV flag.
- ng build $BUILD_ENV
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- '**/*'
base-directory: 'dist*'
if fails on npm install because 404 Not Found - GET https://registry.npmjs.org. For example in github actions I just simply define registry-url: 'https://npm.pkg.github.com' and thats correct.
Thanks for help :)

It fails because, in the execution context of the CodeBuild process, access to the repo containing the GitHub package is restricted, so it can't find the package because it doesn't have access to the repo's packages. You will need to authenticate to the GitHub Package API.
One way to authenticate is to create a Personal Access Token, include it in your CodeBuild Environment by linking a secret in the SecretsManager, then accessing that token in your buildspec script in the env section:
Create a personal access token: In GitHub, create a Personal access token with the read:packages permission. Here's a link to a tutorial on how to do that.
Register token as a secret in Secrets Manager: In SecretsManager, create a secret with one entry. Name the key of the entry GH_PERSONAL_ACCESS_TOKEN, and in the value field, provide the token that you created in step 1. Pick a descriptive name for your secret (something like codebuild/gh_token). Take note of the secret's name.
Authenticate to GitHub Packages using the Personal Access Token: In your buildspec script, you will need to retrieve the secret containing your Personal Access Token, then use that to authenticate before you run the npm install command:
env:
secrets-manager:
GH_PERSONAL_ACCESS_TOKEN: {SECRET_ARN}:PERSONAL_ACCESS_TOKEN # <- replace {SECRET_ARN} with arn of secret
phases:
#...
pre_build:
commands:
- echo Installing source NPM dependencies...
# this is needed to set the url where the package is located
- npm config set #OWNER:registry https://npm.pkg.github.com # <- replace OWNER with the organization/owner name
# this is needed to set the personal access token that we created
- npm config set //npm.pkg.github.com/:_authToken $GH_PERSONAL_ACCESS_TOKEN
- npm install
- npm install -g #angular/cli

Related

Which settings do I need to change to install packages from another private repo in Github actions?

We've got a monorepo (call it A) that has several packages published to npm.pkg.github.com. I just started another repo (B), and would like to pull in one of the packages from A, but I'm getting a 403 during the GH Actions build process. Both repos are under the same organization. Here's the relevant bit of my action:
- name: authorize github npm
run: echo "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}" >> .npmrc
- run: yarn install --frozen-lockfile
The .npmrc exists (and is checked in), and should be right:
#<org-name>:registry=https://npm.pkg.github.com
I can "fix" the problem by adding a PAT to the repository secrets and using that instead of GITHUB_TOKEN but I'd rather not.
Is there something I need to change in the settings for repo A to let its packages be accessible from repo B?

How to fix 404 error when installing npm package from GCP artifact registry with yarn?

I'm having an issue with installing an NPM package from GCP.
I was able to upload the package to the artifact registry of GCP by doing the following steps:
Login to my google account (gcloud auth application-default login)
Run
gcloud artifacts print-settings npm \ --project=[my-project]\ --repository=[my-repo] \ --location=us-east1 \ --scope=#[my-scope]
Pasting the output of the previous step in the .npmrc file located in the root of the project.
Refreshing the access token to GCP (npx google-artifactregistry-auth ./.npmrc)
Run yarn publish
My .npmrc file looks like this:
#[my-scope]:registry=https://us-east1-npm.pkg.dev/[my-project]/[my-repo]/
//us-east1-npm.pkg.dev/[my-project]/[my-repo]/:_authToken="[auth-token]"
//us-east1-npm.pkg.dev/[my-project]/[my-repo]/:always-auth=true
However, when I try to install the package on another project by:
Executing steps 1-4 mentioned above
Run yarn add #[my-scope]/[my-package]
I get an 404 error.
Looks like yarn is looking for the package in the default registry:
error An unexpected error occurred: "https://registry.yarnpkg.com/#[my-scope]/#[my-pacakge]/-/#[my-scope]/[my-package]-0.0.1.tgz: Request failed \"404 Not Found\"".
I simply followed the steps mentioned in the installation instructions in GCP but somehow it's not working.
I encountered a similar issue in this post: Can't install a scoped package I published to a npm registry in GCP but this not the exact error I get.
I would appreciate any help regarding this issue.
Thanks in advance!
I just had this problem for a couple of days and the solution is simple, DO NOT USE YARN when publishing. That's it.
I don't know which part of yarn causes this but basically it ignores .npmrc resulting in the tarball to point to the wrong repository, you can check it if you run yarn info. So when publishing to GCP artifact registry one should use npm publish instead.
In both setting up authentication for npm and Managing Node.js packages, Obtaining an access token section the command used is
npx google-artifactregistry-auth
In the same section there is a note that explains how to add flags if you need to change the path of the .npmrc file.
Note: If you need to store your repository settings and credentials in .npmrc files other than the defaults, you can run the credential helper with additional flags.
--repo-config is the .npmrc file with your repository settings. If you don't specify this flag, the default location is the current directory.
--credential-config is the path to the .npmrc file where you want to write the access token. The default is your user .npmrc file.
Instead of:
npx google-artifactregistry-auth ./.npmrc
It could be written as
npx google-artifactregistry-auth --repo-config=pathto/.npmrc --credential-config=pathto/.npmrc
If you are not sure where your file is you can run npm config ls -l | grep config as explained here
Also check you are specifying the correct .npmrc path if it is different than the default registry as shown in Configuring npm and confirm you are trying to install a package from the Node.js package repository with the correct scope, package, tag or version to be completely explicit.

Verdaccio: how to publish to custom server from Github Actions with proper credentials?

I have a working verdaccio server hosted on a google cloud server. I am able manually publish to it, but am struggling to create a GitHub Action to publish to it when I push to master branch.
I have a script that works perfectly when publishing to npmjs public repo. Here is the relevant part that works for npmjs.org
- name: Publish to npm
if: steps.semantic.outputs.new_release_published == 'true'
run: |
yarn install
git checkout upm
npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
Now, for my own server, I have included the following addition in package.json:
"publishConfig": {
"registry": "http://my.ip.0.0:port"
},
And then in the repositories secrets, I have created an NPM_TOKEN secret with my user's token copied from my computer's .npmrc file after logging in.
I'm getting the following error from the Github Actions result:
npm ERR! code E401
npm ERR! Unable to authenticate, your authentication token seems to be invalid.
npm ERR! To correct this please trying logging in again with:
npm ERR! npm login
So I'm clearly not authenticating properly.
I tried (on the server's cli) using npm token create but it gave me an unauthorized error, and I tried the same on my computer locally after logging in too, and got the same error.
How can I authenticate my Github Actions publish to my custom Verdaccio server? I'm pretty new to this whole CI business, so I suspect I'm missing something quite basic. I suspect I'm doing it wrong using NPM_TOKEN, but it worked fine to publish to npmjs.org public repo.
Again, I can manually publish using npm publish from the terminal on my Mac (after logging into custom server with npm login), so I know that the server is set up properly.
After much googling, I found a solution from this tutorial https://remysharp.com/2015/10/26/using-travis-with-private-npm-deps
It's not written for GitHub Actions but the same procedure worked.
First, you need to login to your private server from your computer. In your home folder look at the .npmrc file (turn on show hidden files).
add this line to the yaml action file:
echo "//YOURREGISTRYADDRESS/:_authToken=\${NODE_AUTH_TOKEN}" > .npmrc
Note that it should actually be NODE_AUTH_TOKEN, NOT your actual token.
The part in the quotes should mostly match the entry in your .npmrc file (without the token).
So now it looks like this
- name: Publish to npm
if: steps.semantic.outputs.new_release_published == 'true'
run: |
yarn install
git checkout upm
echo "//YOURREGISTRYADDRESS/:_authToken=\${NODE_AUTH_TOKEN}" > .npmrc
npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
Then in the Settings -> Secrets part of your GitHub repo, add a secret called NPM_TOKEN and paste in the auth token value from the .npmrc. It's a long series of letters and numbers.
Now this script should properly log in. Apparently the issue is that the default Verdaccio authorization plugin expects it to be used interactively. This line basically creates an .npmrc file on the fly and populates it with the correct info, as if you've already logged in interactively. The file isn't actually created though, and disappears after running, which is a nice touch. It also is pretty secure since it stores the token in the secrets part of the repo. The link above does a better job explaining it, so check it out!

Override npm project auth token with user auth token?

I've got a project that includes a $PROJECT/.npmrc that has an auth token granting read-only access to the proviat repos required by the project:
$ cat .npmrc
//registry.npmjs.org/:_authToken={read-only-token}
How can I override that token with my user token so I can publish packages?
$ cat ~/.npmrc
//registry.npmjs.org/:_authToken={my-token}
The documentation states that config files will be loaded in "priority order", where the project configuration has the highest priority, and there doesn't seem to be any way to override this:
$ cd my-project/
$ npm whoami
project-readonly-user
$ cd ~
$ npm whoami
wolever
I know that it's possible to define an NPM_TOKEN environment variable:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
But this means that every user of the project will need to define the NPM_TOKEN environment variable, which is undesirable (ie, because it means that every user - including read-only users - will need to define an NPM_TOKEN environment variable before they can use the project).
Just found a solution.
Edit your .npmrc file:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
Every time you want to publish(powershell):
$env:NPM_TOKEN="the-token"
npm publish --access public --registry https://registry.npmjs.org
Obviously the docs have change and setting the auth Token via CLI is not possible anymore on npm publish, so I provide more solution for using NPM with Github Actions + Font Awesome PRO + Github Package Registry:
name: Node.js Package
on:
release:
types: [created]
workflow_dispatch:
jobs:
publish:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout 🛎
uses: actions/checkout#master
- name: Setup node env 🏗
uses: actions/setup-node#v2
with:
node-version: '14.x'
registry-url: 'https://npm.pkg.github.com'
scope: '#mindfuel'
- name: Install Packages 🦺
run: npm ci
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Build 👷🏼
run: npm run build
- name: Prepare NPM Config 👮🏽‍♂️
run: rm -f .npmrc
- name: Setup publishing Env 🏗
uses: actions/setup-node#v2
with:
node-version: '14.x'
registry-url: 'https://npm.pkg.github.com'
scope: '#mindfuel'
- name: Publish Package 🚀
run: npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
For me the trick is to recall setup-node after deleting the existing .npmrc.
We have a shared .npmrc that all developers use. It includes a READ ONLY token to our companies registry:
# Font Awesome Pro Config
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=<your-token>
# Private Packages
#<github-username-or-org>:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=<the-read-only-token>
CLI arguments have precedence over local .npmrc config, so you could overwrite it this way:
npm publish --registry=https://registry.npmjs.org/:_authToken={my-token}
You could add an npm script for publishing that depends on an environment variable with a private token:
{
"scripts": {
"publish": "npm publish --registry=https://registry.npmjs.org/:_authToken=${NPM_PUBLISH_TOKEN}"
}
}

How to publish/deploy a npm package to custom artifactory

I want to do something like this:
Create an npm package. Basically, a common code which I want to use for all of my projects. Which I created.
But now What I want is, Every time I commit something in git for this project, Jenkins should build it with updated alpha/beta version and should publish to my own artifactory.
Your Jenkins job can be configured to be triggered by a webhook, which would take care of the first part (every time I commit). Depending on which Git server you're using you can find a lot of tutorials how to do that:
For GitHub
For GitLab
For Gogs
please note this is just a random selection of tutorials how to set up the webhook triggers to work with Git servers and by no means an exhaustive list
To publish your package to JFrog Artifactory you can either use the Jenkins Artifactory Plugin, or use the NPM command line. If you want to use the npm command line, you'll need to authenticate first:
# setting the default registry to Artifactory
npm config set registry http://<ARTIFACTORY_SERVER_DOMAIN>:8081/artifactory/api/npm/npm-repo/
# log in
npm login
alternatively you can get a .npmrc file directly from Artifactory using:
curl -u admin:<CREDENTIAL> http://<ARTIFACTORY_SERVER_DOMAIN>:8081/artifactory/api/npm/auth
After that, there are two ways you can push your package to Artifactory:
Edit your package.json file and add a publishConfig section to a local repository: "publishConfig":{"registry":"http://localhost:8081/artifactory/api/npm/npm-repo/"}
Provide a local repository to the npm publish command: npm publish --registry http://localhost:8081/artifactory/api/npm/npm-repo/