I am trying to deploy Verdaccio to my kubernetes cluster to use as shared registry for my other components. My Problem is, that I cannot seem to get it to properly let my authenticate. I use the most recent Helm-Chart for deployment.
The configuration allows no registrations and expects authentication.
# This is the config file used for the docker images.
# It allows all users to do anything, so don't use it on production systems.
#
# Do not configure host and port under `listen` in this file
# as it will be ignored when using docker.
# see https://github.com/verdaccio/verdaccio/blob/master/docs/docker.md#docker-and-custom-port-configuration
#
# Look here for more config file examples:
# https://github.com/verdaccio/verdaccio/tree/master/conf
#
# path to a directory with all packages
storage: /verdaccio/storage/data
web:
# WebUI is enabled as default, if you want disable it, just uncomment this line
#enable: false
title: DiPlom NPM Registry - Verdaccio
auth:
htpasswd:
file: /verdaccio/storage/htpasswd
# Maximum amount of users allowed to register, defaults to +infinity.
# You can set this to -1 to disable registration.
max_users: -1
# a list of other known repositories we can talk to
uplinks:
npmjs:
url: https://registry.npmjs.org/
packages:
'#*/*':
# scoped packages
access: $authenticated
publish: $authenticated
# proxy: npmjs
'**':
# allow all users (including non-authenticated users) to read and
# publish all packages
#
# you can specify usernames/groupnames (depending on your auth plugin)
# and three keywords: $all, $anonymous, $authenticated
access: $authenticated
# allow all known users to publish packages
# (anyone can register by default, remember?)
publish: $authenticated
# if package is not available locally, proxy requests to 'npmjs' registry
# proxy: npmjs
# To use `npm audit` uncomment the following section
middlewares:
audit:
enabled: true
# log settings
logs:
- {type: stdout, format: pretty, level: http}
#- {type: file, path: verdaccio.log, level: info}
I need to get user-credentials to obtain the corresponding Token so I can push and pull from the registry. However, so far nothing I entered into the HTPasswd file worked. I've looked through the issues on GitHub and a few tips how to generate the HTPasswd entry, but so far no success. When I try to login via npm login, I get the following response:
npm ERR! code E409
npm ERR! 409 Conflict - PUT http://<IngressURL>/-/user/org.couchdb.user:admin - user registration disabled
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/<USR>/.npm/_logs/2021-08-13T10_23_30_382Z-debug.log
I then tried to configure the server to allow registrations by changing max_users to 1. When running npm adduser I get the very same response. How am I supposed to use Verdaccio if there is no way to authenticate with the service? Or am I missing something?
Thanks.
Update:
I've played around with various deployments (docker, compose, Minikube) and figured out, that the ingress, that is created by the helm chart, seems to be the problem here. Not sure what exactly is not working, but as soon as I manually expose the pod via node-port I can properly create a user.
Related
I'm building an apptainer/singularity multi-stage recipe in a gitlab CI environment.
The first step of the recipe is built from an image hosted in a private registry, whereas the second built from an image hosted on dockerhub. Something like this:
# First stage
BootStrap: docker
Registry: <my_private_registry>
From: <my_image>
Stage: base
%files
...
%post
...
# Second stage
BootStrap: docker
Registry: index.docker.io
From: continuumio/miniconda3
Stage: final
%files from base
...
%post
...
Since the first registry is private, in the gitlab CI instance I'm setting the variables APPTAINER_DOCKER_USERNAME and APPTAINER_DOCKER_PASSWORD, as suggested here for CI/CD workflow.
This allows to build the first stage of the recipe succesfully.
Unfortunately, when the build of the second stage starts, it fails with:
> FATAL: While performing build: conveyor failed to get: unable to retrieve auth token: invalid username/password: unauthorized: incorrect username or password
I think because the credentials for my private registry are passed to dockerhub in the second stage.
How can I login to different registries in multi-stage builds?
Any idea about how to deal with this problem?
I found a way to accomplish what I wanted. The fact was that environment variables overrides other login modes.
So I deleted the APPTAINER_DOCKER_USERNAME and APPTAINER_DOCKER_PASSWORD environment variables and, using this method, I added the following before_script field to my .gitlab-ci.yaml:
apptainer:
stage: deploy
image:
name: kaczmarj/apptainer:1.1.3
entrypoint: [""]
tags:
- privileged
before_script:
- echo "$DOCKER_REGISTRY_TOKEN" | apptainer remote login --username <my_username> --password-stdin docker://$CI_REGISTRY
This way, both the private registry (stored in $CI_REGISTRY) and the public
one (dockerhub) are available.
I am trying to build a resume pdf from a resume.tex in my resume repository file whenever a commit is made.
I am getting the following error.
Error: File '/home/runner/work/resume/resume/resume.tex' cannot be found from the directory '/github/workspace'.
How should I access the resume.tex file? If I just say root_file: resume.tex, the error is:
Error: File 'resume.tex' cannot be found from the directory '/github/workspace'.
The .github/workflows/build_resume.yml file looks like this. The resume.tex file is in the root of my repository.
# This is a basic workflow to help you get started with Actions
name: Build PDF on commit.
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ]
# Allows you to run this workflow manually from the Actions tab
# workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Github Action for LaTeX
uses: xu-cheng/latex-action#v2
with:
# The root LaTeX file to be compiled
root_file: ${{ github.workspace }}/resume.tex
# Interpret the root_file input as bash glob pattern
# glob_root_file: # optional
# The working directory for this action
# working_directory: # optional
# The LaTeX engine to be invoked
# compiler: # optional, default is latexmk
# Extra arguments to be passed to the LaTeX engine
# args: # optional, default is -pdf -file-line-error -halt-on-error -interaction=nonstopmode
# [Deprecated] Install extra packages by tlmgr
# extra_packages: # optional
# Install extra packages by apk
# extra_system_packages: # optional
# Install extra .ttf/.otf fonts.
# extra_fonts: ./fonts/*.ttf
# Arbitrary bash codes to be executed before compiling LaTeX documents
pre_compile: tlmgr update --self && tlmgr update --all
# Arbitrary bash codes to be executed after compiling LaTeX documents
post_compile: latexmk -c
# Instruct latexmk to enable --shell-escape
# latexmk_shell_escape: # optional
# Instruct latexmk to use LuaLaTeX
# latexmk_use_lualatex: # optional
# Instruct latexmk to use XeLaTeX
latexmk_use_xelatex: true
When you want to execute files from the current repository, you need to use the actions/checkout first (at the beginning of your job's steps).
This will allow you to access the repository $github_workspace (one of Github environment variables) in your workflow.
Note: All commands you'll run after using the action/checkout will be executed at the repository root.
For example, considering that your resume.tex file is at the root of the repository, you would use something like this:
name: Example
on:
push:
pull_request:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: checkout repo
uses: actions/checkout#v2.3.4
- name: show resume.tex file content
run: cat resume.tex
Here is another workflow example from a personal repository, following the same logic if you want to execute a specific script located in the repository in your workflow. to perform any operation.
If someone is looking to do the same, the following steps need to be done.
Go into your repository.
Compile latex document using xu-cheng/latex-action#v2
Check if PDF is generated.
Push PDF to repository.
The configuration file to do that will look like this.
name: Github Actions CI to build pdf from tex source.
on: push
jobs:
build:
if: "!contains(github.event.head_commit.message, '[skip ci]')"
runs-on: ubuntu-latest
steps:
- name: Set up Git repository
uses: actions/checkout#v2
- name: Compile LaTeX document
uses: xu-cheng/latex-action#v2
with:
# The root LaTeX file to be compiled
root_file: resume_aditya_wagh.tex
# Interpret the root_file input as bash glob pattern
# glob_root_file: # optional
# The working directory for this action
# working_directory: # optional
# The LaTeX engine to be invoked
# compiler: # optional, default is latexmk
# Extra arguments to be passed to the LaTeX engine
args: -pdf -file-line-error -halt-on-error -interaction=nonstopmode
# Install extra packages by apk
# extra_system_packages: # optional
# Install extra .ttf/.otf fonts.
extra_fonts: ./fonts/*.ttf
# Arbitrary bash codes to be executed before compiling LaTeX documents
pre_compile: tlmgr update --self && tlmgr update --all
# Arbitrary bash codes to be executed after compiling LaTeX documents
post_compile: latexmk -c
# Instruct latexmk to enable --shell-escape
# latexmk_shell_escape: # optional
# Instruct latexmk to use LuaLaTeX
# latexmk_use_lualatex: # optional
# Instruct latexmk to use XeLaTeX
latexmk_use_xelatex: true
- name: Check pdf file
run: |
file resume_aditya_wagh.pdf | grep -q ' PDF '
- name: Upload file to repository
run: |
git config --global user.name "adityamwagh"
git config --global user.email "adityamwagh#gmail.com"
git add resume_aditya_wagh.pdf
git commit -m "commit message"
git push
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
I have a very basic integration configured for Gitlab-CI but it fails almost at the beginning when it has to clone the code.
My integration is this:
image: node:latest
stages:
- build
- test
cache:
paths:
- node_modules/
- dist/
build-prod:
stage: build
script:
- npm install
- npm run build-prod
artifacts:
paths:
- node_modules/
- dist/
test_with_karma:
stage: test
script: ng test
And the error that I get is this:
Running with gitlab-runner 11.7.0 (8bb608ff)
on fakehost 2eaf11ea
Using Docker executor with image node:latest ...
Pulling docker image node:latest ...
Using docker image sha256:8c67bfd7b95bdc535edc4a4144f5392b0f73efd6385fbcb47747d028d7059359 for node:latest ...
Running on runner-2eaf11ea-project-56-concurrent-0 via fakehost...
Cloning repository...
Cloning into '/builds/redacted/frontend'...
remote: You are not allowed to download code from this project.
fatal: unable to access 'https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx#working-domain.com/redacted/frontend.git/': The requested URL returned error: 403
/bin/bash: line 65: cd: /builds/redacted/frontend: No such file or directory
ERROR: Job failed: exit code 1
What is the problem here?
Check if this is covered by gitlab-org/gitlab-ce issue 39469
YAY - it works for me. This problem seems to have multiple solutions.
The one that worked for me is #44855
To summarize. Being an Administrator on Gitlab does not mean you have the "access" to do whatever you want to do in Gitlab.
"Unable to access" permissions applies to the person who is logged into Gitlab and running the job.
To fix the problem - the person / account running the job must be a member (master) of the project.
This will apply to private projects.
It is not necessary to make a private project Public even though that appears to fix the problem. GITLAB suggests you must have https for the project to work you can use http.
SOLUTION - add your account to the project even if you are the Administrator
And:
Conrad has described it correctly.
You need to have rights to the project to run pipeline, however, as administrator, you can start any pipeline.
I've got the case when the user being Admin in Gitlab could push his commit from command line, although theoretically having no rights to project - and the pipeline has failed.
This inconsistency need to be fixed, either Admin user should not be able to push/start pipeline, having no rights for it, or he should authomatically be granted all rights to all projects. I'd prefer the first one, because it separates gitlab administration from project rights. Sometimes I prefer not having full rights, just like working as non-root under Linux.
I would like to use an access token to publish and retrieve from an artifactory npm repo from a CI environment. I have created a Bearer token using the artifactory API but when I try and use that for access in the .npmrc with the format:
//mydomain.jrog.io/:_authToken=myveryverylongaccesstoken
registry=https://mydomain.jfrog.io/mydomain/api/npm/npm
I always receive 401 errors back.
In addition, though perhaps a different issue, if I try to use npm login with my actual artifactory credentials I get the response:
adduser Incorrect username or password
npm WARN adduser You can reset your account by visiting:
npm WARN adduser
npm WARN adduser https://npmjs.org/forgot
npm WARN adduser
npm ERR! code E403
npm ERR! forbidden No oauth npm default login configuration found: org.couchdb.user:myusername
The artifactory docs around access tokens explicitly say this is the sort of use case to set up an access token, but the docs around setting up the npm repo alwyas seem to imply you need a real user account and make no mention of access tokens...
What is your artifactory version?
The "npm login" way is only supported since 5.4 (aka. _authToken), so if you are below that, your only chance is to authenticate with BASIC authentication (aka. _auth).
Basic authentication is fairly easy to setup, Artifactory provides an easy entry point to help you set up your .npmrc
Launch the following (and do not forget to replace user and encrypted_password with the Artifactory user account you want to authenticate with)
curl -u user:encrypted_password https://mydomain.jfrog.io/mydomain/api/npm/auth
It will output something like
_auth = YourLongBasicAuthToken
always-auth = true
email = user#server.com
Copy these 3 lines in your .npmrc, and this should work well...
To generate the contents for .npmrc, use an existing user like admin with its password
curl -uadmin:<PASSWORD> http://<ARTIFACTORY_SERVER_DOMAIN>/artifactory/api/npm/auth
afterwards you can replace/set the _auth key with any base64 encoded username:ACCESS_TOKEN
for scoped packages, use
curl -uadmin:<PASSWORD> http://<ARTIFACTORY_SERVER_DOMAIN>/artifactory/api/npm/npm-repo/auth/<SCOPE>
and set ..username and .._password (base64) with any user and access token
Using Basic Authentication
Using the next described steps you should change your local .nprmc file with the output of the command
To retrieve the next fields _auth, email and always-auth automatically generated -> Run the following command to retrieve these strings directly from Artifactory:
curl -u {username}:{API_key}
http://{artifactory_server_domain}/{domain}/artifactory/api/npm/auth
Notes:
{username} -> Name near Welcome when you are logged in - take a look at the attached screenshot
{Api_key} -> Copy and paste this key - take a look at the attached screenshot
{artifactory_server_domain} - domain of your artifactory accrodingly.
{domain} - additional filed of domain in you url (if present)
Screenshot describes {artifactory_server_domain} and {domain}
The _auth line in the other answers breaks some scoped package retrieval from npmjs. I found that I needed to use specific _auth for just the one registry I was using. See my original answer here https://stackoverflow.com/a/57014522/442837
in addition to the previously offered solutions:
curl -u username:encrypted_password http://<ARTIFACTORY_SERVER_DOMAIN>/artifactory/api/npm/auth >> ~/.npmrc
curl -u username:personal_api_key http://<ARTIFACTORY_SERVER_DOMAIN>/artifactory/api/npm/auth >> ~/.npmrc
here is another option without using curl (*does not work with API key):
npm config set _auth $(echo -n username:encrypted_password | base64)
My experience adds to the comments made by others.
I was working with jfrog and already placed the package in jfrog. When trying to resolve that using jfrog.
I got the error "npm ERR! Unable to authenticate, need: Basic realm="Artifactory Realm"" when i checked logs.
After spending hours trying to resolve.
I came across the documentation.
https://www.jfrog.com/confluence/display/JFROG/npm+Registry#npmRegistry-ConfiguringthenpmClientforaScopeRegistry
This is very useful..The issue is I'm just pasting the access token instead of base64 encoded one.
the helpful part of the page
Then I ran that command curl -u admin with a new credentials and token and got back the base64 encoded one.
Then it started working ..Hope it's helpful to anyone working with jfrog
For windows user :-
1/ Add https://registry.npmjs.org to .npmrc to user profile on top:
registry=https://registry.npmjs.org
or 2/ Run below command
npm config set registry https://registry.npmjs.org
I want to write a Dockerfile which exports a directory from a remote Subversion repository into the build context so I can work with these files in subsequent commands. The repository is secured with user/password authentication.
That Dockerfile could look like this:
# base image
FROM ubuntu
# install subversion client
RUN apt-get -y update && apt-get install -y subversion
# export my repository
RUN svn export --username=myUserName --password=myPassword http://subversion.myserver.com/path/to/directory
# further commands, e.g. on container start run a file just downloaded from the repository
CMD ["/bin/bash", "path/to/file.sh"]
However, this has the drawback of printing my username and password on the screen or any logfile where the stdout is directed, as in Step 2 : RUN svn export --username=myUserName --password=myPassword http://subversion.myserver.com/path/to/directory. In my case, this is a Jenkins build log which is also accessible by other people who are not supposed to see the credentials.
What would be the easiest way to hide the echo of username and password in the output?
Until now, I have not found any way how to execute RUN commands in a Dockerfile silently when building the image. Could the password maybe be imported from somewhere else and attached to the command beforehand so it does not have to be printed anymore? Or are there any methods for password-less authentication in Subversion that would work in the Dockerfile context (in terms of setting them up without interaction)?
The Subversion Server is running remotely in my company and not on my local machine or the Docker host. To my knowledge, I have no access to it except for accessing my repository via username/password authentication, so copying any key files as root to some server folders might be difficult.
The Dockerfile RUN command is always executed and cached when the docker image is build so the variables that svn needs to authenticate must be provided at build time. You can move the svn export call when the docker run is executed in order to avoid this kind of problems. In order to do that you can create a bash script and declare it as a docker entrypoint and pass environment variables for username and password. Example
# base image
FROM ubuntu
ENV REPOSITORY_URL http://subversion.myserver.com/path/to/directory
# install subversion client
RUN apt-get -y update && apt-get install -y subversion
# make it executable before you add it here otherwise docker will coplain
ADD docker-entrypoint.sh /enrypoint.sh
ENTRYPOINT /entrypoint.sh
docker-entrypoint.sh
#!/bin/bash
# maybe here some validation that variables $REPO_USER $REPO_PASSOWRD exists.
svn export --username="$REMOTE_USER" --password="$REMOTE_PASSWORD" "$REPOSITORY_URL"
# continue execution
path/to/file.sh
Run your image:
docker run -e REPO_USER=jane -e REPO_PASSWORD=secret your/image
Or you can put the variables in a file:
.svn-credentials
REPO_USER=jane
REPO_PASSWORD=secret
Then run:
docker run --env-file .svn-credentials your/image
Remove the .svn-credentials file when your done.
Maybe using SVN with SSH is a solution for you? You could generate a public/private key pair. The private key could be added to the image whereas the public key gets added to the server.
For more details you could have a look at this stackoverflow question.
One solution is to ADD the entire SVN directory you previously checked out on your builder file-system (or added as a svn:externals if your Dockerfile is itself in a SVN repository like this: svn propset svn:externals 'external_svn_directory http://subversion.myserver.com/path/to/directory' ., then do a svn up).
Then in your Dockerfile you can simply have this:
ADD external_svn_directory /tmp/external_svn_directory
RUN svn export /tmp/external_svn_directory /path/where/to/export/to
RUN rm -rf /tmp/external_svn_directory
Subversion stores authentication details (if it not disabled in configuration) at client side and use stored username|password on request for the subsequent operations on the same URL.
Thus - you have to run (successful) svn export in Dockerfile with username|password only once and allow SVN to use cached credentials (remove auth. options from command-line) later