I am trying to execute multiple run parallel but configured Jobs in one plan are being executed sequentially. I have configured it for parallel run according to the link https://confluence.atlassian.com/bamboo/configuring-concurrent-builds-289277193.html .But not worked.
If I am doing anything wrong, please give the solution.
Thanks.
More details are needed to effectively answer your question (# of agents, how many are capable of running the tasks, etc), but you can follow this advice:
Make sure that you have different agents available to execute each job and that the ones available have the necessary capabilities to run those jobs. For example, if you have two jobs that you expect to run in parallel, but only 1 agent is capable of building them, then they're going to run sequentially.
Related
We have a test that takes several hours to run and that we'd like to run on our codebase as often as possible in GitLab CI. The idea is for it to validate as many commits as possible by merging them from dev into main but we know it's too slow to run on every commit.
It could run on a schedule, e.g. 18:00 every evening, but then it would run unnecessarily if there have been no changes and it wouldn't run as often as it could, e.g. 2-3 times a day.
Limiting concurrent jobs as suggested here isn't enough because the jobs will pile up, one per commit, and there will never be time to run them all.
We'd like it to complete the test for one commit, and then restart on the latest commit available, skipping over any commits that came in earlier.
I've looked through the rules section of the docs but don't see any magic variables that would let me say "run this job if it's not already running". Perhaps some kind of semaphore as described here (requested but not implemented as far as I can see).
How can we tell GitLab CI to run this particular job only if it's not already running and skip the job otherwise?
We are using the jenkins build flow plugin(https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin) to run our test cases by dividing them into small sub test cases and test them in parallel.
The current problem is even one of the job fails, the other parallel jobs and the hosting flow job will continue running, which is a big waste of resources.
I checked the doc there is no place to control the jobs inside the parallel {}. Any ideas how to deal with that?
Looking at the code, I don't see a way to achieve that. I would ask the user mailing list for help.
I am thinking to use Guard / Rescue imbedded in Parallel to do this.
Adding failFast: true within parallel block would cause the build to fail as soon as one of the parallel nodes fails.
You can view this as an example.
In our environment we have quite a few long-running functional tests which currently tie up build agents and force other builds to queue. Since these agents are only waiting on test results they could theoretically just be handing off the tests to other machines (test agents) and then run queued builds until the test results are available.
For CI builds (including unit tests) this should remain inline as we want instant feedback on failures, but it would be great to get a better balance between the time taken to run functional tests, the lead time of their results, and the throughput of our collective builds.
As far as I can tell, TeamCity does not natively support this scenario so I'm thinking there are a few options:
Spin up more agents and assign them to a 'Test' pool. Trigger functional build configs to run on these agents (triggered by successful Ci builds). While this seems the cleanest it doesn't scale very well as we then have a lead time of purchasing licenses and will often have need to run tests in alternate environments which would temporarily double (or more) the required number of test agents.
Add builds or build steps to launch tests on external machines, then immediately mark the build as successful so queued builds can be processed then, when the tests are complete, we mark the build as succeeded/failed. This is reliant on being able to update the results of a previous build (REST API perhaps?). It also feels ugly to mark something as successful then update it as failed later but we could always be selective in what we monitor so we only see the final result.
Just keep spinning up agents until we no longer have builds queueing. The problem with this is that it's a moving target. If we knew where the plateau was (or whether it existed) this would be the way to go, but our usage pattern means this isn't viable.
Has anyone had success with a similar scenario, or knows pros/cons of any of the above I haven't thought of?
Your description of the available options seems to be pretty accurate.
If you want live update of the builds progress you will need to have one TeamCity agent "busy" for each running build.
The only downside here seems to be the agent licenses cost.
If the testing builds just launch processes on other machines, the TeamCity agent processes themselves can be run on a low-end machine and even many agents on a single computer.
An extension to your second scenario can be two build configurations instead of single one: one would start external process and another one can be triggered on external process completeness and then publish all the external process results as it's own. It can also have a snapshot dependency on the starting build to maintain the relation.
For anyone curious we ended up buying more agents and assigning them to a test pool. Investigations proved that it isn't possible to update build results (I can definitely understand why this ugliness wouldn't be supported out of the box).
I have a Bamboo plan configuration with several stages. Now I want to have these stages to execute in certain order, but independently of the result of the previous stage.
This seems to be impossible with Bamboo. Subsequent stages are executed only if the previous one is successful.
Any thoughts?
"Now I want to have these stages to execute in certain order, but
independently of the result of the previous stage."
The statement seems self contradicting. If you want the stages to execute in order, that implies some dependency/coupling between them. Yet you want them to execute independently of the result of the previous stage.
The they are truly independent you can configure them as independent jobs as part of one stage. These jobs(which may contain several ordered tasks) will execute independently and in parallel.
Please see the following for more details:
http://quickstart.atlassian.com/download/bamboo/get-started/bamboo-elements
Hope this helps other who stumble upon this question.
Might be off-topic.
But if your only purpose is to make sure that some activities are not executed in parallel and the order does not matter, then you may try to configure these activities using Jobs.
Bamboo documentation states the following:
Jobs within a given Stage will run in parallel as long as there are agents available to execute them. Otherwise, Jobs run one at a time in no particular order.
It means that if there is only 1 agent available, then there will be no parallel execution and jobs will be executed one by one.
Dependencies is the way to do this. Separate your stages into separate plans. Have each plan depend on (be triggered by) the prior plan.
This doesn't give you exactly the same results, but it is the only way to do it if you need ordered stages.
Final stages were added to Bamboo 6.8.0 in February 2019
As mentioned at: https://jira.atlassian.com/browse/BAM-9771?focusedCommentId=1918676
The following Atlassian Community questions also asked this:
https://community.atlassian.com/t5/Answers-Developer-Questions/Creating-a-final-stage-in-Bamboo-that-is-always-run/qaq-p/543318
https://community.atlassian.com/t5/Bamboo-questions/Is-there-a-way-to-make-a-stage-run-regardless-of-previous/qaq-p/139255
Use Dependencies tab to do this, else use ant script.
I am running Bamboo 3.4.2. I have 2 plans (among many), one scheduled and one commit triggered, that I need to prevent running at the same time as they conflict. They are in the same Project so if there is a way to only allow one Plan in a Project to run that would be entirely acceptable. I am entirely willing to commit an agent to just service that Project but I can't find a way of doing that.
Thanks,
Matthew
Here's one pretty easy way to do this.
On the 2 plans in question, set a requirement for a particular capability, such as "MachineName" equals "Bob".
Then set up the single agent you would like to run those plans and give it a capability such that it matches, like "MachineName" equals "Bob".
Obviously, "Bob" may not be the ideal choice there.