I'm trying to create a pipeline with a production and a development deployment. In both environments the application should be built with docker. But only when something changed in the according directory.
For example:
When something changed in the frontend directory the frontend should be build and deployed
When something changed in the backend directory the backend should be build and deployed
At first I didn't had the needs: keyword. The pipeline always executed the deploy_backend and deploy_frontend even when the build jobs were not executed.
Now I've added the needs: keyword, but Gitlab says yaml invalid when there was only a change in one directory. When there is a change in both directories the pipeline works fine. When there for exaple a change in the README.md outside the 2 directories the says yaml invalid as well.
Does anyone knows how I can create a pipeline that only runs when there is a change in a specified directory and only runs the according deploy job when the build job has ran?
gitlab-ci.yml:
stages:
- build
- deploy
build_frontend:
stage: build
only:
refs:
- master
- development
changes:
- frontend/*
script:
- cd frontend
- docker build -t frontend .
build_backend:
stage: build
only:
refs:
- master
- development
changes:
- backend/*
script:
- cd backend
- docker build -t backend .
deploy_frontend_dev:
stage: deploy
only:
refs:
- development
script:
- "echo deploy frontend"
needs: ["build_frontend"]
deploy_backend_dev:
stage: deploy
only:
refs:
- development
- pipeline
script:
- "echo deploy backend"
needs: ["build_backend"]
The problem here is that your deploy jobs require the previous build jobs to actually exist.
However, by using the only.changes-rule, they only exist if actually something changed within those directories.
So when only something in the frontend-folder changed, the build_backend-Job is not generated at all. But the deploy_backend_dev job still is and then misses it's dependency.
A quick fix would be to add the only.changes configuration also to the deployment-jobs like this:
deploy_frontend_dev:
stage: deploy
only:
refs:
- development
changes:
- frontend/*
script:
- "echo deploy frontend"
needs: ["build_frontend"]
deploy_backend_dev:
stage: deploy
only:
refs:
- development
- pipeline
changes:
- backend/*
script:
- "echo deploy backend"
needs: ["build_backend"]
This way, both jobs will only be created if the dependent build job is created as well and the yaml will not be invalid.
Related
I don't get it. I have two repos. One for infrastructure, and the other for project code. Inside project code, I have .gitlab-ci.yml file that will include one job for creating env variables, another include file that will include other stages inside the job. Every other stage is triggered, but the first include is not triggered no matter what I do. What am I doing wrong?
Project gitlab-ci
# Stages
stages:
- pre-build test
- build
- post-build test
- deploy
- environment
- e2e
# Main Variables
variables:
GIT_SUBMODULE_STRATEGY: normal
IMAGE_VERSION: "latest"
CARGO_HOME: $CI_PROJECT_DIR/.cargo
FF_USE_FASTZIP: "true"
ARTIFACT_COMPRESSION_LEVEL: "fast"
CACHE_COMPRESSION_LEVEL: "fastest"
STAGING_BRANCH: "master"
VARIABLES_FILE: variables.txt
# Include main CI files from infrastructure repository
include:
- project: 'project/repo-one'
ref: master
file: '/gitlab-ci/env/ci-app-env.yml'
- project: 'project/repo-one'
ref: master
file: '/gitlab-ci/app/ci-merge-request.yml'
env ci file
env mr:
stage: .pre
before_script:
- TEST_VAR="TEST"
- IMAGE_PATH="/var/www"
script:
- echo "export TEST_VAR=$TEST_VAR" > $VARIABLES_FILE
- echo "export IMAGE_PATH=$IMAGE_PATH" > $VARIABLES_FILE
- cat $VARIABLES_FILE
artifacts:
paths:
- $VARIABLES_FILE
I can't find any guide on how to deploy a Blazor web assembly app to GitLab Pages as a static site. Has anyone managed to do so for .NET 6?
I have created a sample web assembly Blazor client application:
https://gitlab.com/sunnyatticsoftware/sasw-community/sasw-editor
The steps to create this simple web assembly are:
Install .NET 6 SDK
Create repo and clone it (e.g: sasw-editor)
Create the solution with web assembly Blazor project
dotnet new gitignore
dotnet new blazorwasm --name Sasw.Editor.Web --output src/Sasw.Editor.Web --no-https
dotnet new sln
dotnet sln add src/Sasw.Editor.Web
Compile and run it
dotnet build
dotnet run --project src/Sasw.Editor.Web
That's a way to run the blazor app on the port defined at the launchsettings.json
Building...
info: Microsoft.Hosting.Lifetime[14]
Now listening on: http://localhost:5291
info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
Content root path: C:\src\sasw-editor\src\Sasw.Editor.Web
I stop it. It works fine when served with Kestrel.
Now, the process to publish a distribution folder would be like this
dotnet publish -c Release -o publish
All the artifacts and files are now under publish folder. So, in theory, I can serve those things with a simple web server. I install a basic web server tool called local-web-server (it requires NodeJs/npm but you can use any other web server)
npm install -g local-web-server
Now I navigate to the publish/wwwroot folder where my index.html is
And I start the web server there
ws
Listening on http://5CD013DP5L:8000, http://192.168.1.13:8000, http://127.0.0.1:8000, http://172.21.208.1:8000
If I open the browser on http://127.0.0.1:8000, or any other of the above url, I can see my Blazor wasm app working perfectly.
I want to host that very same publish folder in GitLab pages which, in theory, is capable of serving static files.
So I create a .gitlab-ci.yml to compile, publish and copy contents to the public folder of GitLab pages.
image: mcr.microsoft.com/dotnet/sdk:6.0
variables:
GIT_DEPTH: 1000
PUBLISH_OUTPUT_DIR: publish
stages:
- build
- test
- publish
- delivery
build:
stage: build
script:
- dotnet restore --no-cache --force
- dotnet build --configuration Release --no-restore
artifacts:
paths:
- test
expire_in: 8 hour
rules:
- if: $CI_COMMIT_TAG
when: never
- when: always
test:
stage: test
script: dotnet test --blame --configuration Release
allow_failure: false
rules:
- if: $CI_COMMIT_TAG
when: never
- exists:
- test/**/*Tests.csproj
publish:
stage: publish
script:
- dotnet publish -c Release -o $PUBLISH_OUTPUT_DIR
artifacts:
paths:
- $PUBLISH_OUTPUT_DIR/
expire_in: 8 hour
rules:
- if: $CI_COMMIT_TAG
when: never
- when: on_success
pages:
stage: delivery
script:
- cp -a $PUBLISH_OUTPUT_DIR/ public
artifacts:
paths:
- public
only:
- main
The pipeline completes successfully. I can see the exact same structure I had locally within publish folder, this time under public folder in GitLab
But it fails to render the app
https://sunnyatticsoftware.gitlab.io/-/sasw-community/sasw-editor/-/jobs/1846501612/artifacts/public/wwwroot/index.html
shows
Loading...
An unhandled error has occurred. Reload 🗙
I can see it's attempting to access https://sunnyatticsoftware.gitlab.io/ or https://sunnyatticsoftware.gitlab.io/favicon.ico and returning 404
The favicon.ico would exist on https://sunnyatticsoftware.gitlab.io/-/sasw-community/sasw-editor/-/jobs/1846501612/artifacts/public/wwwroot/favicon.ico
so it must be some kind of URL re-write problem, right?
Any help would be much appreciated.
Always keep it at base href="/" and then in your ci change it to whatever you need. E.g. on gitlab you can use the CI_PROJECT_NAME variable.
pages:
stage: deploy
variables:
SED_COMMAND: 's#<base\shref="\/"\s?\/>#<base href="\/$CI_PROJECT_NAME\/" \/>#g'
script:
- cp -a $PUBLISH_OUTPUT_DIR/wwwroot public
- sed -r -i "$SED_COMMAND" public/index.html
artifacts:
paths:
- public
only:
- main
The solution is simply to use the following
pages:
stage: delivery
script:
- cp -a $PUBLISH_OUTPUT_DIR/wwwroot public
artifacts:
paths:
- public
only:
- main
and to use the <base href="/sasw-community/sasw-editor/" /> in index.html with the relative path.
I've recorded a quick tutorial on my Odysee channel https://odysee.com/#sunnyAtticSoftware:a/blazor-wasm-gitlab-pages:e
See https://gitlab.com/sunnyatticsoftware/training/blazorwasm-pages with a full sample
I still don't know of a good way to mix local development's base / relative path with prod base /sasw-community/sasw-editor/ and change it dynamically (is it even possible?)
But the problem is solved.
I have a CI pipeline in Gitlab (relevant part only):
default:
image: docker:latest
variables:
DOCKER_APP_TAG: ${REGISTRY_URL}/${APP_NAME}:${CI_COMMIT_SHORT_SHA}
stages:
- build
.config:
only:
- branches
- merge_requests
- tags
except:
- triggers
tags:
- prod
build-app:
extends: .config
stage: build
script:
- docker build --target production -t ${DOCKER_APP_TAG} -f ${CI_PROJECT_DIR}/etc/node/Dockerfile .
When I build from a branch (i.e. push to main branch) everything works well. The docker build command is ran with the proper value available in S{DOCKER_APP_TAG}.
However after I create a TAG in GitLab (and a release), the build on this GitLab TAG fails at the docker build ... line complaining that the docker tag is not valid:
invalid argument "/:e5dc27fd" for "-t, --tag" flag: invalid reference format
The variables ${REGISTRY_URL} and ${APP_NAME} are not expanded. I have checked GitLab docs and the only limitations I see is if I was running in a service. But it is not the case.
What am I missing to expand properly the variables even with tag builds?
I have setup my CI so that I can manually create a release-tag when all tests succeeds for a new commit on master branch. For this I have created a manual step in the CI config like so:
.release-template:
stage:
releasing
dependencies:
- assemble
script:
- ./gradlew reckonTagPush -Preckon.scope=$scope -Preckon.stage=$stage -Dorg.ajoberstar.grgit.auth.username=$GIT_USER -Dorg.ajoberstar.grgit.auth.password=$GIT_PASSSWORD
only:
- master
when: manual #ONLY MANUAL RELEASES, ONLY FROM MASTER
release-major:
extends: .release-template
variables:
scope: major
stage: final
release-minor:
extends: .release-template
variables:
scope: minor
stage: final
release-patch:
extends: .release-template
variables:
scope: patch
stage: final
This setup fails with an authentication error.
Execution failed for task ':reckonTagPush'.
> org.eclipse.jgit.api.errors.TransportException: https://gitlab-ci-token#gitlab.com/<group>/<project>.git: not authorized
I am running this on gitlab.com on a shared runner.
The username and password are configured in gitlab ci variables for the project. When running this locally inside the same docker image that is used in the gitlab runner, it works fine. So there must be something special about the way the gitlab runner is executing the gradle tasks, or communicating with the gitlab git repo.
Solved the issue with access to pushing to the git repo by adding the following script :
script:
- url_host=`git remote get-url origin | sed -e "s/https:\/\/gitlab-ci-token:.*#//g"`
- git remote set-url origin "https://gitlab-ci-token:$GIT_TOKEN#$url_host"
- ./gradlew reckonTagPush -Preckon.scope=$scope -Preckon.stage=$stage -Dorg.ajoberstar.grgit.auth.username="$GIT_USER" -Dorg.ajoberstar.grgit.auth.password="$GIT_TOKEN"
The notably changes here are setting the git remote url, as well as surrounding the gitlab ci variables with " when passing them to the reckon plugin
Dears
My continuous integration + tests project is running with the gitlabci file on a dedicated server. Run with multiple environments is my goal to complete this project.
For example: I have two environments (desenv and homol) and my desire is build the project and after run in each one these tests changing only some variables (link, db user) in the test-automation-inscricao-vestib.dll.config.
I createad a yml file with 3 jobs:
build:test -> nuget restore and build project
test:desenv -> change file.dll.config to the respective configuration file of the environment and run testes (NUnit)
test:homol -> change file.dll.config to the respective configuration file of the environment and run testes (NUnit)
Is there a way correctly to do this? Because my gitlabci-runner doesnt work fine with this configuration. E.g.:
Here is my yml code:
stages:
- build
- test
build:test:
only:
- schedules
- web
stage: build
tags:
- windows
script:
#Restore Nuget
- '"C:\\Gitlab-Runner\\nuget.exe" restore "test-automation-inscricao-vestib.sln"'
#Build project
- '"C:\\Program Files (x86)\\MSBuild\\14.0\\Bin\\msbuild.exe" /t:Clean,Build /p:Configuration=Debug "test-automation-inscricao-vestib.sln"'
artifacts:
paths:
- test-automation-inscricao-vestib\bin\Debug
test:desenv:
only:
- schedules
- web
stage: test
tags:
- teste
script:
#Change the environment to DESENV
- powershell Remove-Item test-automation-inscricao-vestib\bin\Debug\test-automation-inscricao-vestib.dll.config
- powershell Rename-Item test-automation-inscricao-vestib\test-automation-inscricao-vestib_DESENV.dll.config test-automation-inscricao-vestib\bin\Debug\test-automation-inscricao-vestib.dll.config
#Run tests
- cd test-automation-inscricao-vestib/bin/Debug
- '"C:\\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "test-automation-inscricao-vestib.dll" --where "cat==Producao"'
dependencies:
- build:test
test:homol:
only:
- schedules
- web
stage: test
tags:
- teste
script:
#Change the environment to HOMOL
- powershell Remove-Item test-automation-inscricao-vestib\bin\Debug\test-automation-inscricao-vestib.dll.config
- powershell Rename-Item test-automation-inscricao-vestib\test-automation-inscricao-vestib_HOMOL.dll.config test-automation-inscricao-vestib\bin\Debug\test-automation-inscricao-vestib.dll.config
#Run tests
- cd test-automation-inscricao-vestib/bin/Debug
- '"C:\\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "test-automation-inscricao-vestib.dll" --where "cat==Producao"'
dependencies:
- build:test
I found a solution for this mistake. I add some tags in each NUnit running line and it works! Look here:
Before:
- '"C:\\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "test-automation-inscricao-vestib.dll" --where "cat==Producao"'
After:
- '"C:\\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "test-automation-inscricao-vestib.dll" --inprocess --labels=On --where "cat==Producao"'
Fixed only including these tags: --inprocess --labels=On