New to Gitlab CI/CD.
What is the proper construct to use in my .gitlab-ci.yml file to ensure that my validation job runs only when a "real" checkin happens?
What I mean is, I observe that the moment I create a merge request, say—which of course creates a new branch—the CI/CD process runs. That is, the branch creation itself, despite the fact that no files have changed, causes the .gitlab-ci.yml file to be processed and pipelines to be kicked off.
Ideally I'd only want this sort of thing to happen when there is actually a change to a file, or a file addition, etc.—in common-sense terms, I don't want CI/CD running on silly operations that don't actually really change the state of the software under development.
I'm passably familiar with except and only, but these don't seem to be able to limit things the way I want. Am I missing a fundamental category or recipe?
I'm afraid what you ask is not possible within Gitlab CI.
There could be a way to use the CI_COMMIT_SHA predefined variable since that will be the same in your new branch compared to your source branch.
Still, the pipeline will run before it can determine or compare SHA's in a custom script or condition.
Gitlab runs pipelines for branches or tags, not commits. Pushing to a repo triggers a pipeline, branching is in fact pushing a change to the repo.
Related
I currently have a gitlab ci pipeline that pushes commits to the branch the pipeline is running on to update code versions (using python-semantic-release). As far as I can tell, the later stages in my pipeline do not use this newly pushed code and instead a new pipeline is triggered for this commit. I am currently skipping the triggered pipeline using [skip ci]. I would like to be able to use the original CI pipeline to finish packaging the code and publishing documentation using the new commit. Is there anything I can do to update the commit that the current CI pipeline is running on or something?
I am not aware of changing the ref mid-pipeline.
You might try and experiment with downstream pipelines, especially the multi-project ones (even though would remain in the same project).
Those (downstream "multi-project" pipelines) are the ones which does not have to run under the same project, ref, and commit SHA as the upstream pipeline (as oppose to parent-child pipeline).
I would also push the code (after the python-semantic-release step) to a different branch, in order for your second pipeline to operate on that second branch, directly with the right code.
I have a Pipeline job that needs to run only after an MR has been merged to a certain branch (let’s assume it’s master).
This job is supposed to make an API call to send the name of the merged source branch.
The problem I’m encountering is that CI_MERGE_REQUEST_SOURCE_BRANCH_NAME will not be available on the Pipeline that runs right after the merge (since it’s not a merge request pipeline).
Is there a way (env var) to tell what was the branch that was just merged into master?
Many thanks in advance y’all!
Better you work with hashes not with the names.
Anyway I see two options:
In the pipeline for the merge request save the hash/name in an artifact. The subsequent pipeline can access this artifact an read the hash/name.
Run a independent pipeline and read the hash/name of the previous pipeline. To do this more secure you can add tags and read the previous pipeline only if it has the correct tag.
Is it possible to use a variable as build trigger? I've tried and the build doesn't get triggered. If I remove the variable and insert a value, the build gets triggered as expected.
Aren't variables allowed here? $(Mapping.ServerPath) is set to MyRepo/Branches/MyBranch. $/MyRepo/Branches/MyBranch triggers the build correctly.
No. And why should they?
The specified path results in a poll action being performed on the static path.
You can use wildcards if needed.
The build should trigger on a change, hence CI trigger.
Making the path a variable, when would you provide it?
If it's just keeping a static value elsewhere, why not fill it in?
If you want to provide the path when calling the build.
Then you don't intend to use the CI option as intended?
No, it is not supported.
There is a user voice that you can follow: Allow Variables in Repository, variables and triggers Tab.
We have a microservice architecture with dozens of builds, it makes sense to be able to use a variable that we can update when we start our next iteration. With our branching strategy we have a new branch for each sprint and for each release. Changing the CI trigger in every build every couple weeks is inefficient.
We are using on premise TFS2018 and from everything I've seen this is not supported.
I'm trying to configure a set of build configurations in TeamCity 6 and am trying to model a specific requirement in the cleanest possible manner way enabled by TeamCity.
I have a set of acceptance tests (around 4-8 suites of tests grouped by the functional area of the system they pertain to) that I wish to run in parallel (I'll model them as build configurations so they can be distributed across a set of agents).
From my initial research, it seems that having a AcceptanceTests meta-build config that pulls in the set of individual Acceptance test configs via Snapshot dependencies should do the trick. Then all I have to do is say that my Commit build config should trigger AcceptanceTests and they'll all get pulled in. So, lets say I also have AcceptanceSuiteA, AcceptanceSuiteB and AcceptanceSuiteC
So far, so good (I know I could also turn it around the other way and cause the Commit config to trigger AcceptanceSuiteA, AcceptanceSuiteB and AcceptanceSuiteC - problem there is I need to manually aggregate the results to determine the overall success of the acceptance tests as a whole).
The complicating bit is that while AcceptanceSuiteC just needs some Commit artifacts and can then live on it's own, AcceptanceSuiteA and AcceptanceSuiteB need to:
DeploySite (lets say it takes 2 minutes and I cant afford to spin up a completely isolated one just for this run)
Run tests against the deployed site
The problem is that I need to be able to ensure that:
the website only gets configured once
The website does not get clobbered while the two suites are running
If I set up DeploySite as a build config and have AcceptanceSuiteA and AcceptanceSuiteB pull it in as a snapshot dependency, AFAICT:
a subsequent or parallel run of AcceptanceSuiteB could trigger another DeploySite which would clobber the deployment that AcceptanceSuiteA and/or AcceptanceSuiteB are in the middle of using.
While I can say Limit the number of simultaneously running builds to force only one to happen at a time, I need to have one at a time and not while the dependent pieces are still running.
Is there a way in TeamCity to model such a hierarchy?
EDIT: Ideas:-
A crap solution is that DeploySite could set a 'in use flag' marker and then have the AcceptanceTests config clear that flag [after AcceptanceSuiteA and AcceptanceSuiteB have completed]. The problem then becomes one of having the next DeploySite down the pipeline wait until said gate has been opened again (Doing a blocking wait within the build, doesnt feel right - I want it to be flagged as 'not yet started' rather than looking like it's taking a long time to do something). However this sort of stuff a flag over here and have this bit check it is the sort of mutable state / flakiness smell I'm trying to get away from.
EDIT 2: if I could programmatically alter the agent configuration, I could set Agent Requirements to require InUse=false and then set the flag when a deploy starts and clear it after the tests have run
Seems you go look on the Jetbrains Devnet and YouTrack tracker first and remember to use the magic word clobber in your search.
Then you install groovy-plug and use the StartBuildPrecondition facility
To use the feature, add system.locks.readLock. or system.locks.writeLock. property to the build configuration.
The build with writeLock will only start when there are no builds running with read or write locks of the same name.
The build with readLock will only start when there are no builds running with write lock of the same name.
therein to manage the fact that the dependent configs 'read' and the DeploySite config 'writes' the shared item.
(This is not a full productised solution hence the tracker item remains open)
EDIT: And I still dont know whether the lock should be under Build Parameters|System Properties and what the exact name format should be, is it locks.writeLock.MYLOCKNAME (i.e., show up in config with reference syntax %system.locks.writeLock.MYLOCKNAME%) ?
Other puzzlers are: how does one manage giving builds triggered by build completion of a writeLock task read access - does the lock get dropped until the next one picks up (which would allow another writer in) - or is it necessary to have something queue up the parent and child dependency at the same time ?
We use a dev-test-prod branching scheme with bzr 2. I'd like to setup a bzr hook on the prod branch that will reject a push from the test branch. Looking at the bzr docs, this looks doable, but I'm kinda surprised that my searches don't turn up any one having done it, at least not via any of the keywords I've thought to search by. I'm hoping someone has already gotten this working and can share their path to success.
My current thought is to use the pre_change_branch_tip hook to check for the presence of a file on the test branch. If it's present, fail the commit.
You may ask, why test for a file, why not just test the branch name? Because I actually need to handle the case where our developers have branched their devel branch, pulled in the shared test branch and are now (erroneously) pushing that test branch to production instead of pushing their feature branch to production. And it seems a billion times easier to look for a file in the new branch than to try to interrogate the sending branch's lineage.
So has someone done this? seen it done? or do I get to venture out into the uncharted wasteland that is hook development with bzr? :)
your approach should work and the plugin will be quite simple: just raise an exception if the file is present.
(For some sample code you can look at a plugin I wrote that can prevent commits on some conditions https://launchpad.net/bzr-text-checker)