This is my first time trying to generate a gitlab-ci job artifact. My ci script generates a csv file that I want to download as an artifact in the job. The yaml file is as below.
unittest:
script:
- cd unittest
- bash ci-test.sh
artifacts:
paths:
- /*.csv
when: always
expire_in: 1 day
I obtained an error msg like this.
Uploading artifacts for successful job
00:02
Uploading artifacts...
WARNING: /*.csv: no matching files
ERROR: No files to upload
I have confirmed that the generated csv report is there.
Paths are relative to the project directory. With that said, you cannot access files outside of the project directory, so /*.csv is incorrect.
If files are in the project, then just set path relative to the project, e.g. if you have it in reports folder, then make path equal reports/*.csv
You can also set your project directory path to the artifacts path:
artifacts:
paths:
- $CI_PROJECT_DIR
expire_in: 1 week
This way in your code if you are not specifying any particular path for any file then it will always be in Project Directory and GitLab runner will be able to find it.
You need to store your output file to $CI_PROJECT_DIR
An example:
test:
stage: quality
image: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
script:
- cd /app
- coverage run -m pytest
- coverage report
- coverage xml -o $CI_PROJECT_DIR/coverage.xml
coverage: '/TOTAL.*\s([.\d]+)%/'
artifacts:
paths:
- coverage.xml
reports:
cobertura:
- coverage.xml
Related
I really dont know if this is a simple (must be), common or complex task.
I have a buildspec.yml file in my codebuild project, and i am trying to append the version written in package.json file to the output artifact.
I have already seen a lot of tutorials that teach how to append the date (not really useful to me), and others that tell me to execute a version.sh file with this
echo $(sed -nr 's/^\s*"version": "([0-9]{1,}.[0-9]{1,}.*)",$/\1/p' package.json)
and set it in a variable (it doesn't work).
i'm ending up with a build folder called: "my-project-$(version.sh)"
codebuild environment uses ubuntu and nodejs
Update (solved):
my version.sh file:
#!/usr/bin/env bash
echo $(sed -nr 's/^\s*\"version": "([0-9]{1,}\.[0-9]{1,}.*)",$/\1/p' package.json)
Then, i just found out 2 things:
Allow access to your version.sh file:
git update-index --add --chmod=+x version.sh
Declare a variable in any phase in buildspec, i dit in in build phase (just to make sure repository is already copied in environment)
TAGG=$($CODEBUILD_SRC_DIR/version.sh)
then reference it in artifact versioned name:
artifacts:
files:
- '**/*'
name: workover-frontend-$TAG
As result, my build artifact's name: myproject-1.0.0
In my case this script do not want to fetch data from package.json. On my local machine it working great but on AWS doesn't. I had to use chmod in different way, because i got message that i don't have right permissions. My buildspec:
version: 0.2
env:
variables:
latestTag: ""
phases:
pre_build:
commands:
- "echo sed version"
- sed --version
build:
commands:
- chmod +x version.sh
- latestTag=$($CODEBUILD_SRC_DIR/version.sh)
- "echo $latestTag"
artifacts:
files:
- '**/*'
discard-paths: yes
And results in console:
CodeBuild
I also have to mark that when i paste only for example echo 222 into version.sh file i got right answer in CodeBuild console.
I'm learning GitLab CI/CD, I want to when finished build send files in artifacts, the idea is possible?
image: maven:3.8.1-jdk-11
stages:
- build
- deploy
build:
stage: build
script:
- mvn clean install
artifacts:
paths:
- "*/target/*.jar"
deploy:
stage: deploy
script:
- scp -r <artifacts_path> root#test.com:~/Deploy
Could I get artifacts real path in runner then send files with scp?
Generally speaking, no. You must rely on artifact restoration process. Keep in mind that (1) artifacts are generally not stored on the runner and (2) docker runners execute jobs inside of a docker container and typically would not have access to files on the runner host, even if artifacts were stored there.
When jobs start, artifacts from previous stages are restored into the workspace.
So, as an alternative solution, you can simply start with an empty workspace (don't checkout the repo), then upload all files in the workspace, which should be only the restored artifacts, assuming there are no file-based variables.
deploy:
variables: # prevent checkout of repository
GIT_STRATEGY: none
stage: deploy
script:
- ls -laht # list files, which should be just restored artifacts
- scp -r ./ root#test.com:~/Deploy
Another way might be to just use the same glob pattern used in the artifacts:paths: to find the files and upload them.
variables:
ARTIFACTS_PATTERN: "*/target/*.jar"
build:
# ...
artifacts:
paths:
- $ARTIFACTS_PATTERN
deploy:
script: # something like this. Not sure if scp supports glob patterns
- rsync -a -m --include="$ARTIFACTS_PATTERN" user#remote:~/Deploy
I have a code using Fortran modules. I can build it with no problems under normal circumstances. CMake takes care of the ordering of the module files.
However, using a gitlab runner, it SOMETIMES happens that cmake does NOT order the Fortran modules by dependencies, but alphabetically instead, which than leads to a build failure.
The problem seems to occur at random. I have a branch that built in the CI. After adding a commit, that modified a utility script not involved in any way in the build, I ran into this problem. There is no difference in the output of the cmake configure step.
I use the matrix configuration for the CI to test different configurations. I found, that I could trigger this by adding another mpi version (e.g. openmpi/4.1.6). Without that version, it built. With it added in the matrix, ALL configurations showed the problem.
stages:
- configure
- build
- test
.basic_config:
tags:
- hpc_runner
variables:
# load submodules
GIT_SUBMODULE_STRATEGY: recursive
.config_matrix:
extends: .basic_config
# define job matrix
parallel:
matrix:
- COMPILER: [gcc/9.4.0]
PARALLELIZATION: [serial, openmpi/3.1.6]
TYPE: [option1, option2]
BUILD_TYPE: [debug, release]
- COMPILER: [gcc/10.3.0, intel/19.0.5]
PARALLELIZATION: [serial]
TYPE: [option2]
BUILD_TYPE: [debug]
###############################################################################
# setup script
# These commands will run before each job.
before_script:
- set -e
- uname -a
- |
if [[ "$(uname)" = "Linux" ]]; then
export THREADS=$(nproc --all)
elif [[ "$(uname)" = "Darwin" ]]; then
export THREADS=$(sysctl -n hw.ncpu)
else
echo "Unknown platform. Setting THREADS to 1."
export THREADS=1
fi
# load environment
- source scripts/build/load_environment $COMPILER $BUILD_TYPE $TYPE $PARALLELIZATION
# set path for build folder
- build_path=build/$COMPILER/$PARALLELIZATION/$TYPE/$BUILD_TYPE
configure:
stage: configure
extends: .config_matrix
script:
- mkdir -p $build_path
- cd $build_path
- $CMAKE_COMMAND
artifacts:
paths:
- build
expire_in: 1 days
###############################################################################
# build script
build:
stage: build
extends: .config_matrix
script:
- cd $build_path
- make
artifacts:
paths:
- build
expire_in: 1 days
needs:
- configure
###############################################################################
# test
test:
stage: test
extends: .config_matrix
script:
- cd $build_path
- ctest --output-on-failure
needs:
- build
The runner runs on an HPC machine which a complex setup, and I am not to familiar with the exact configuration. I contacted the admin with this problem, but wanted to see if anybody else had run into this before and have solutions or hints on what is going on.
With the help from our admin I figured it out.
The problem comes from cmake using absolute paths. The runner has actually several runners for parallel jobs, with each using a different prefix path, e.g. /runner/001/ or /runner/012/. So when I run configure on a specific runner, cmake saves that prefix path to the configuration.
Now in the build stage, there is no guarantee to have the same configuration run on the same runner. However, since there are absolute paths in the make files, make tries to access the folders in the configure runner's prefix. Now, that can be anything from non-existing, over old files from previous pipelines to the correct files downloaded by another case.
The only fix I currently can see is to run everything on the same runner in one stage, to avoid the roulette of prefix paths. If anybody has a different idea, or if there is a way to fix a specific matrix case to a specific runner prefix, please comment.
Dears
My continuous integration + tests project is running with the gitlabci file on a dedicated server. Run with multiple environments is my goal to complete this project.
For example: I have two environments (desenv and homol) and my desire is build the project and after run in each one these tests changing only some variables (link, db user) in the test-automation-inscricao-vestib.dll.config.
I createad a yml file with 3 jobs:
build:test -> nuget restore and build project
test:desenv -> change file.dll.config to the respective configuration file of the environment and run testes (NUnit)
test:homol -> change file.dll.config to the respective configuration file of the environment and run testes (NUnit)
Is there a way correctly to do this? Because my gitlabci-runner doesnt work fine with this configuration. E.g.:
Here is my yml code:
stages:
- build
- test
build:test:
only:
- schedules
- web
stage: build
tags:
- windows
script:
#Restore Nuget
- '"C:\\Gitlab-Runner\\nuget.exe" restore "test-automation-inscricao-vestib.sln"'
#Build project
- '"C:\\Program Files (x86)\\MSBuild\\14.0\\Bin\\msbuild.exe" /t:Clean,Build /p:Configuration=Debug "test-automation-inscricao-vestib.sln"'
artifacts:
paths:
- test-automation-inscricao-vestib\bin\Debug
test:desenv:
only:
- schedules
- web
stage: test
tags:
- teste
script:
#Change the environment to DESENV
- powershell Remove-Item test-automation-inscricao-vestib\bin\Debug\test-automation-inscricao-vestib.dll.config
- powershell Rename-Item test-automation-inscricao-vestib\test-automation-inscricao-vestib_DESENV.dll.config test-automation-inscricao-vestib\bin\Debug\test-automation-inscricao-vestib.dll.config
#Run tests
- cd test-automation-inscricao-vestib/bin/Debug
- '"C:\\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "test-automation-inscricao-vestib.dll" --where "cat==Producao"'
dependencies:
- build:test
test:homol:
only:
- schedules
- web
stage: test
tags:
- teste
script:
#Change the environment to HOMOL
- powershell Remove-Item test-automation-inscricao-vestib\bin\Debug\test-automation-inscricao-vestib.dll.config
- powershell Rename-Item test-automation-inscricao-vestib\test-automation-inscricao-vestib_HOMOL.dll.config test-automation-inscricao-vestib\bin\Debug\test-automation-inscricao-vestib.dll.config
#Run tests
- cd test-automation-inscricao-vestib/bin/Debug
- '"C:\\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "test-automation-inscricao-vestib.dll" --where "cat==Producao"'
dependencies:
- build:test
I found a solution for this mistake. I add some tags in each NUnit running line and it works! Look here:
Before:
- '"C:\\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "test-automation-inscricao-vestib.dll" --where "cat==Producao"'
After:
- '"C:\\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "test-automation-inscricao-vestib.dll" --inprocess --labels=On --where "cat==Producao"'
Fixed only including these tags: --inprocess --labels=On
I have a Travis CI build that is configured to upload the build artifacts to S3. I've followed the Travis artifacts documentation but when the build completes I get the following error (and the S3 container is empty).
ERROR: failed to upload: /home/travis/build/jonburney/KingsgateMediaPlayer-Android/
app/build/outputs/apk/app-release-unsigned.apk
err: The bucket you are attempting to access must be addressed using the specified
endpoint. Please send all future requests to this endpoint.
I have tried to specify the "endpoint" option in the configuration but it was ignored. It appears to be attempting to upload the file to
https://s3.amazonaws.com/kmp-build-output/jonburney/KingsgateMediaPlayer-Android/30/30.1/app/build/outputs/apk/app-release-unsigned.apk.
Here is a copy of the relevant section from my .travis.yml file
addons:
artifacts: true
s3_region: "us-west-2"
artifacts:
paths:
- $(git ls-files -o app/build/outputs | tr "\n" ":")
Have I missed a configuration option for this scenario? Any help is appreciated!
This was fixed after an email to the Travis-CI support team and some investigation. The code in my .travis.yml file was modified to ensure that "artifacts" was only present once, like so:
addons:
artifacts:
s3_region: "us-west-2"
paths:
- $(git ls-files -o app/build/outputs | tr "\n" ":")