I am using gitlab-ci to run scripts defined in .gitlab-ci.yml whenever a PR is raised.
I want to get the list of modified files since the last commit.
The use case is to run file-specific integration tests in a large codebases.
If you do not need to know the paths, but you simply need to run a specific job only when a specific file is changed, then use only/changes .gitlab-ci.yml configuration, e.g.
docker build:
script: docker build -t my-image:$CI_COMMIT_REF_SLUG .
only:
changes:
- Dockerfile
- docker/scripts/*
Alternatively, if you need to get paths of the modified scripts, you can use gitlab-ci CI_COMMIT_BEFORE_SHA and CI_COMMIT_SHA environment variables, e.g.
> git diff --name-only $CI_COMMIT_BEFORE_SHA $CI_COMMIT_SHA
src/countries/gb/homemcr.js
src/countries/gb/kinodigital.js
A common use case that some people will find useful. Run the job when a merge request is raised. Specifically, run lint on all the files that are changed w.r.t. target branch.
As one of the answers suggest, we can get the target branch name through CI_MERGE_REQUEST_TARGET_BRANCH_NAME variable (list of predefined variables). We can use git diff --name-only origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME to get the list of files that were changed. Then pass them on to the linter via xargs. The example configuration may look like.
code_quality:
only:
- merge_requests
script:
- git diff --name-only origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME | xargs <LINT_COMMAND_FOR_FILE>
Related
How to add .gtilbic.yml to project with many branches which has to be triggered only on commit to master branch.Do i have to edit yml with rule only master branch in every git branch?
Just add your .gitlab-ci.yml file to the default (master) branch. The file can contain a workflow:rules: that makes sure that pipelines are only triggered for commits to the master branch.
workflow:
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: always
my_job:
script:
- make test # or whatever
Typically, you need this file present in all branches where you want pipelines to run. However, because you only want to run pipelines on the default branch, it's not super important that your branches contain the yaml.
Adding CI for all branches
If you did want to add CI to run on your other branches you could either merge/cherry-pick the change from the default branch into all your branches.
Alternatively, you can use the projects CI/CD settings to specify a custom configuration file that lives in another project. When you apply this setting, it will apply the configuration from the remote project to be triggered for all branches (or other pipeline events).
I have a GitHub repo myRepo that scans the contents of another repo theirRepo and converts them to JSON files. The details aren't really that important, just so much that myRepo uses nodeJS and holds theirRepo as a submodule. Licensewise this is not a problem.
What I'd like to achieve is that, when theirRepo merges into main, myRepo magically updates and builds the new files. I'd like to use existing infrastructures such as GitHub actions, Netlify build processes etc.
How would you approach this?
I don't expect a detailed solution for the magical part, but am rather looking for a few pointers, something that gets me started.
As GitHub Actions (AFAIK) does not currently allow to trigger events based on changes in other repositories (unless you control another repository’s workflows) one might have to hack a little bit.
File change in OtherRepo
I’m not familiar with Node, but, depending on the project culture following files might change during the new release/main branch update:
package-lock.json
CHANGELOG.md (for semantic versioning)
This is a rough approximation, you might also want to identify multiple files likely to change with each merged PR.
Cron based jobs
Run your job every N hours/minutes or another time interval to check for changes.
Use caching
Run your action only when files in another repo change, something along these lines:
steps:
- run: curl <path to file> -o output1
- run: curl <path to file2> -o output2
- name: Cache
uses: actions/cache#v3
id: cache
with:
key: ${{ hashFiles(”output1”, “output2”) }}
- name: Update repo
if: steps.cache.output.cache-hit != “true”
run: <do your stuff>
Does gitlab-ci have a way to download a script and store it on the local file system so it could be run? Looks like other's have asked similar questions (see below).
One way to do it would be to use curl (but curl has to exist in the CI runner):
curl -o ./myscript -k https://example.com/myscript.sh
This was from https://stackoverflow.com/a/22800194/3281336.
If I have a script that I would like to use in multiple CI-pipelines, I'd like to have a way to download the script to the local file system to use in the pipeline. NOTE: I don't have the ability to create a custom runner or docker image in my given situation.
If the script were available via git or an https website, what are my alternatives?
Some search results
https://docs.gitlab.com/ee/ci/yaml/includes.html - Gitlab supports a way to include files, even from GIT repos. This might work I just haven't read how.
How to run a script from file in another project using include in GitLab CI? - Similar but the answer uses a multi-project pipeline and a trigger which is really (I think) a different answer.
.gitlab-ci.yml to include multiple shell functions from multiple yml files - Similar but the question is dealing with scripts in YAML files and I'm dealing with a stand alone script.
How to include a PowerShell script file in a GitLab CI YAML file - So far this is the closest to my question and some might consider it the same even though the question is asking about a powershell script. The answer said this wasn't possible to include a script (so maybe this is not possible using the GitLab CI syntax).
If it is possible, please let me know how to do this.
I trying to build a conditional pipeline that is only triggered with a certain file type or extension is being pushed.
For example, when pushing my markdown files I would like to compile and generate html, txt,...
Looking at the documentation, I see that there is support for only or except but they are relying only of tags, commit messages,... but it's not possible to specify the files modified in the commit. An extension of the only/except will look like:
only:
files:
- *.md
That is not built-in. You may write your own implementation but do note that if you want to compare with previous commit and you push several commits at once the build may be skipped as by default the jobs are only run for the latest commit. It would be a lot easier to just build the static site always.
A simplest way to implement it would be just to grep the result of git diff and end the job if the files don't match.
before_script:
- if ! grep "\.md$" <(git diff --name-only HEAD~1); then exit; fi;
I am successfully creating a Git tag from bamboo but need to add the branch name to the tag and not just the build number and I cannot get it to work. This is the script that works for just the build number as the tag name:
git tag -f -a ${bamboo.buildNumber} -m "${bamboo.planName} build number ${bamboo.buildNumber} passed automated acceptance testing." ${bamboo.planRepository.revision}
git remote add central ${bamboo.planRepository.repositoryUrl}
git push central ${bamboo.buildNumber}
git ls-remote --exit-code --tags central ${bamboo.buildNumber}
I have tried defining a variable including the bamboo.buildNumber and bamboo.repository.branch.name and that did not work either in the plan variables or in the build and package task script. I also added the bamboo.repository.branch.name variable to the git tag line in the script and that didn't work either.
The problem with the tag just being the build number is it can overlap with other tag names if multiple branches are using the same Bamboo plan. Also a tag should reference the branch it was created from.
If you use newer version of Bamboo, it might has ${bamboo.repository.branch.name} variable deprecated. Try using this one: ${bamboo.planRepository.branchName}