I have buildspec.yml with multiple commands in pre_build phase and more in build phase. When I specify commands as array items
pre_build:
commands:
- $Env:TEST_SUBFOLDER='dev\';
- Invoke-Expression('cmd /c set')
each command is executed in its own shell session. In above example TEST_SUBFOLDER environment variable is not set when invoking next command.
But if I specify commands as single item
pre_build:
commands:
$Env:TEST_SUBFOLDER='dev\'; `
Invoke-Expression('cmd /c set')
every command runs in same shell and TEST_SUBFOLDER environment variable is set.
Where is this feature documented? I do not find any reference.
SO question Variable mysteriously disappears? AWS CodeBuild sugests to use version 0.2 of buildspec. I am using version 0.2
You can do a YAML block scalar syntax:
pre_build:
commands:
- >-
$Env:TEST_SUBFOLDER='dev\';
Invoke-Expression('cmd /c set')
Related
Building a docker container
Using alpine as build image
The Containerfile receives VERSION as argument
It is set as image version
The same version is passed to dotnet publish command.
The passed version (which is a variable) in current context never gets substituted. Instead of substituted variable ${VARIABLE} is passed to dotnet publish command.
ARG VERSION
FROM mcr.microsoft.com/dotnet/sdk:6.0-alpine as build
ARG VERSION
RUN echo ${VERSION}
RUN echo ${VERSION} > image_version
WORKDIR /app
COPY . .
RUN dotnet restore
# TODO Pass the version as build command below.
RUN dotnet publish --version-suffix ${VERSION} --output /app/published-app
I tried different ways to do the variable substitution like below:
"$VERSION"
"$(VERSION)"
$VERSION ..... and many other ways I found in the internet. None of them worked. The argument passed is exactly what it is before the variable is substituted.
Thoughts on what may be causing this?
Please help me in converting below written GitHub Action to Gitlab CI script. I am new to scripting in Gitlab.
From Github documentation I could understand that the below written line is for setting the value of environment variable. But I couldn't find any resource for setting environment variable in Gitlab.
run: >
DISPLAY=:0 xvfb-run -s '-screen 0 1024x768x24' julia --project=monorepo -e 'using Pkg; Pkg.test("GLMakie", coverage=true)'
&& echo "TESTS_SUCCESSFUL=true" >> $GITHUB_ENV
There are multiple ways to set environment variables, and it depends on what you want to achieve:
use it within the same job
use it in another job
Use it within the same job
In Bash or other Shells you can set an environment variable via export - in your case it would look like:
job:
script:
- DISPLAY=:0 xvfb-run -s '-screen 0 1024x768x24' julia --project=monorepo -e 'using Pkg; Pkg.test("GLMakie", coverage=true)' && export TESTS_SUCCESSFUL=true
- echo $TESTS_SUCCESSFUL #verification that it is set and can be used within the same job
Use it within another job
To handover variables to another job you need to define an artifact:report:dotenv. It is a file which can contain a list of key-value-pairs which will be injected as Environment variable in the follow up jobs.
The structure of the file looks like:
KEY1=VALUE1
KEY2=VALUE2
and the definition in the .gitlab-ci.yml looks like
job:
# ...
artifacts:
reports:
dotenv: <path to file>
and in your case this would look like
job:
script:
- DISPLAY=:0 xvfb-run -s '-screen 0 1024x768x24' julia --project=monorepo -e 'using Pkg; Pkg.test("GLMakie", coverage=true)' && echo "TESTS_SUCCESSFUL=true" >> build.env
artifacts:
reports:
dotenv: build.env
job2:
needs: ["job"]
script:
- echo $TESTS_SUCCESSFUL
see https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-to-another-job for further information.
I just started using Gitlab's CI/CD feature. I need to create the zip file for the published items. The last two lines of the script should do the magic.
build:
stage: build
script:
- echo "Testing CI CD"
- git clone https://gitlab.io/common-dependencies.git
- git clone https://gitlab.io/sitecore-nuget.git
- nuget restore
- msbuild Common.SC.sln /property:Configuration=Release
/p:PublishProfile=C:\CICDArtifacts\Artifacts_Release.pubxml
- cd .\bin\Release
- '"C:\Program Files\7-Zip\7z.exe" a C:\CICDArtifacts\$CI_PROJECT_NAME-$CI_JOB_ID.zip'
But when the build triggers, i get the following error.
However when i executed below command in Gitlab runner it runs perfectly.
"C:\Program Files\7-Zip\7z.exe" a C:\CICDArtifacts\commonProject.zip'
It looks like i'm missing something here. Please help me to understand, how gitlab runner understands commands like 7z , echo, msbuild etc.
Please note that, 7zip is installed and it is located under
C:\Program Files\7-Zip\7z.exe
Your help much appreciated.
I have resolved the problem using below steps.
updated %PATH% environment variable pointing to 7z.exe path.
Updated the script as below.
'- 7z a C:\CICDArtifacts\$CI_PROJECT_NAME-$CI_JOB_ID.zip'
Restarted the Gitlab runner as, Gitlab was not able to recognize the 7z command after updating the environment variables.
Hope it is helpful for everyone.
how gitlab runner understands commands like echo, msbuild etc
It should understand them if their parent folder is in the %PATH%.
If running the command without quotes does not work, try the alternative script line:
- cmd /C "\""C:\Program Files\7-Zip\7z.exe" a C:\CICDArtifacts\commonProject.zip"
The without quotes form would be: no single quotes:
- "C:\Program Files\7-Zip\7z.exe" a C:\CICDArtifacts\commonProject.zip
Use
- powershell -Command executable.exe arguments
This command has worked for me to kickoff an exe in gitlab yaml.
For my case I did a choco install 7zip.install and somehow I was able to use 7z in gitlab-ci-yml
I have some stages defined in drone.yml file. Is there a way to specify which stage need to be run through command line parameter? For example: below is my drone.yml file. I want to build buildOnContainer1 and buildOnContainer2 stage separately. So I am looking for a command such as drone exec buildOnContainer1. It only runs the command under buildOnContainer1.
buildOnContainer1:
image: container1
pull: true
commands:
- npm test:uat
buildOnContainer2:
image: container2
pull: true
commands:
- npm test:dev
My first thought to implement that granularity level of control is through environment variables.
Drone provides the ability to substitute environment variables at runtime. This gives us the ability to use dynamic build or commit details in our pipeline configuration.
You can pass environment variables to the command line and also to your Drone server using secrets. Check the Drone docs on ENV interpolation and docker exec command
You should build customized images for container1 and container2 to run the commands or skip them based on the values of specific environment variables.
A dirty example would be something like the following .drone.yml:
buildOnContainer1:
image: container1
pull: true
environment:
- SKIP=${skip.buildOnContainer1}
commands:
- ./myscript.sh test:uat
buildOnContainer2:
image: container2
pull: true
environment:
- SKIP=${skip.buildOnContainer2}
commands:
- ./mysqcrypt.sh test:dev
Your custom image should contain a mysqcript.sh bash script in your working directory. The script could check whether the value of the ENVAR SKIP is true or false. If true, it would do nothing. If false is would execute npm command with whatever args you had passed to the script.
Then you could execute the build locally:
drone exec --secret skip.buildOnContainer1=true --secret skip.buildOnContainer2=false
I am trying to make use of the variables: keyword documented in the Gitlab CI Documentation here:
FROM: https://docs.gitlab.com/ce/ci/yaml/README.html
variables
This feature requires gitlab-runner with version equal or greater than
0.5.0.
GitLab CI allows you to add to .gitlab-ci.yml variables that are set
in build environment. The variables are stored in repository and are
meant to store non-sensitive project configuration, ie. RAILS_ENV or
DATABASE_URL.
variables:
DATABASE_URL: "postgres://postgres#postgres/my_database"
These variables can be later used in all executed commands and
scripts.
The YAML-defined variables are also set to all created service
containers, thus allowing to fine tune them.
When I attempt to use it, my builds do not run any stages and are marked successful anyway, a good sign of bad YAML. I pasted my gitlab-ci.yml contents into the LINT tool in the settings area and the output error is:
Status: syntax is incorrect
Error: variables job: unknown parameter PACKAGE_NAME
I'm using my YAML syntax the same as the docs, however it will not work. I'm unable to find any open bugs related to this. Below are my current versions and a sanitized version of my gitlab-ci.yml.
Gitlab Version: 7.13.2 Omnibus
Gitlab Runner Version: 0.5.2
gitlab-ci.yml (Sanitized)
types:
- test
- build
variables:
PACKAGE_NAME: "awesome-django-app"
PACKAGE_SUMMARY: "Awesome webapp backend."
MAJOR_RELEASE: "1"
MINOR_RELEASE: "0"
PATCH_LEVEL: "0dev"
DEV_DB_URL: "db"
DEV_SERVER: "pydev.example.com"
PROD_SERVER: "pyprod.example.com"
TEST_SERVER: "pytest.example.com"
envtest:
type: test
script:
- ". ./testbuild.sh"
tags:
- python2.7
- postgres
- linux
except:
- tags
buildrpm:
type: build
script:
- mkdir -p ~/rpmbuild/SOURCES
- mkdir -p ~/rpmbuild/SPECS
- mkdir -p ~/tarbuild/$PACKAGE_NAME-$MAJOR_RELEASE.$MINOR_RELEASE.$PATCH_LEVEL
- cp $PACKAGE_NAME.spec ~/rpmbuild/SPECS/.
- cp -r * ~/tarbuild/$PACKAGE_NAME-$MAJOR_RELEASE.$MINOR_RELEASE.$PATCH_LEVEL/.
- cd ~/tarbuild
- tar -zcf ~/rpmbuild/SOURCES/$PACKAGE_NAME-$MAJOR_RELEASE.$MINOR_RELEASE.$PATCH_LEVEL.tar.gz *
- cd ~
- rm -Rf ~/tarbuild
- rpmlint -i ~/rpmbuild/SPECS/$PACKAGE_NAME.spec
- echo $CI_BUILD_ID
- 'rpmbuild -ba ~/rpmbuild/SPECS/$PACKAGE_NAME.spec \
--define="_build_number $CI_BUILD_ID" \
--define="_python_version_min 2.7" \
--define="_version $MAJOR_RELEASE.$MINOR_RELEASE.$PATCH_LEVEL" \
--define="_package_name $PACKAGE_NAME" \
--define="_summary $SUMMARY"'
- scp rpmbuild/RPMS/noarch/$PACKAGE_NAME-$MAJOR_RELEASE.$MINOR_RELEASE.$PATCH_LEVEL-$CI_BUILD_ID.noarch.rpm $DEV_SERVER:~/.
tags:
- python2.7
- postgres
- linux
- rpm
except:
- tags
Question:
How do I use this value properly?
Additional Info:
Removing this section from the YAML file causes everything to work so the rest of the file is in working order. (Of course undefined variables lead to script errors...)
Even just reducing the variables for testing down to just PACKAGE_NAME causes the same break.
The original answer is no longer correct.
The original documentation now stands, Now there are more ways as well. Variables can be created from the GUI, API, or by being defined in the .gitlab-ci.yml as well.
https://docs.gitlab.com/ce/ci/variables/README.html
While it is in the documentation, I do not believe that variables were included in the latest version of gitlab (7.13). The functionality to read variables out of the yaml files was brought in by a commit by ayufan 9 days ago.
Looking at the parser on the 7.13 stable branch, you can see that his contribution did not make it in. So assuming you're on 7.13 or earlier, I'm afraid we are out of luck. Since it is on master, I am fairly certain that we'll see it in the next release. Until then, we could either monkey patch, do a git pull if you're using the source directly, or just rely on the project variables until the next release.