I'm using bamboo to automate performance tests that should be run every night. I implemented two tests: first that run big queries and second that checks performance results.
First test (running queries) should be executed and after two hours second one (checking performance results) should be run. Obviously I don't want compile these tests into one test that run queries, waits 2 hours and checks results.
My solution is to have two bamboo plans: first plan with running queries test scheduled for 1:00 AM and second plan with checking performance results test scheduled for 3:00 AM. That works.
Is it possible to execute those tests within one bamboo plan (for example by setting two stages (with one test each) and setting delay between stages execution)?
Edit:
I have working solution that doesn't block agent for delay time (two scheduled plans). It works. I'm just wondering if it's possible to achieve same effect within one plan - sounds like functionality that could be available in Bamboo.
If blocking the build agent for 2 hours is not an issue, you may add a script task at the end of the first stage, so that it waits for 2 hours until the next stage is started.
sleep 2h
You may also define your result plan as a child plan (in Dependencies tab) and then introduce a sleep time using script task at the end of the first plan.
This way, your first plan will finish executing after 2 hours followed by the child plan.
Update: If your plan A is connected to a repository and is triggered when there is a new commit, you may connect the same repository in plan B and introduce a quiet period that will wait for 2 hours before it executes. This way, your agent is not blocked for 2 hours.
Related
We have a test that takes several hours to run and that we'd like to run on our codebase as often as possible in GitLab CI. The idea is for it to validate as many commits as possible by merging them from dev into main but we know it's too slow to run on every commit.
It could run on a schedule, e.g. 18:00 every evening, but then it would run unnecessarily if there have been no changes and it wouldn't run as often as it could, e.g. 2-3 times a day.
Limiting concurrent jobs as suggested here isn't enough because the jobs will pile up, one per commit, and there will never be time to run them all.
We'd like it to complete the test for one commit, and then restart on the latest commit available, skipping over any commits that came in earlier.
I've looked through the rules section of the docs but don't see any magic variables that would let me say "run this job if it's not already running". Perhaps some kind of semaphore as described here (requested but not implemented as far as I can see).
How can we tell GitLab CI to run this particular job only if it's not already running and skip the job otherwise?
I have a number of multi-part tests that require me to run a setup phase first, then wait about 30 minutes before I can run the verification phase. I'm limited to a single agent, so including the delay in the test itself is very undesirable as it ties up the CI/CD system.
What I would like to do is have the setup plan, when complete, cause the corresponding verification plan to be scheduled 30 minutes later.
I know you can have one plan trigger another plan, but that's not quite what I want as that will happen immediately after the first plan, which won't work.
What's the best way to do this with Bamboo or is it even possible?
Not possible. Unless you delay the job programatically from a script task which, as you mention, is not ideal for your use case.
I am implementing a Cucumber - JVM based Selenium automation framework.
One of the workflow in the webapps i test, requires a long wait so that a batch job that is scheduled as frequently as in every 3 minutes, runs, and creates a login id, which the user can utilize, to continue with the workflow.
I am currently handling it in such a way that i execute the initial part test case first and continue with other test cases, so that the framework gets ample time to wait for the user id to be created.
After all other test cases are run the second part of the test case is run. But, before running the second part of the test case, i query the database and verify whether the id is created. If the id is created then the execution continues else, fails saying that the user id was not created.
Although this works for now, i am sure there are better ways to handle such scenarios. Have any one of you come across such a scenario? How did you handle it ?
I think I understand your problem. You actually would like to have an execution sequence like this probably:
Test 1
Test 2
Test 3
But if you implement Test 1 "correctly" it will take very long because it has to wait for the system under test to do several long running things, right?
Therefore you split Test 1 into to parts and run the tests like this:
Test 1 (Part A)
Test 2
Test 3
Test 1 (Part B)
So that your system under test has time to finish the tasks triggered by Test 1 (Part A).
As you acknowledged before, this is considered bad test design, as your test cases are no longer independent from each other. (Generally speaking no test case should rely on side effects created by another test case beforehand.)
My recommendation for that scenario would be to leave Test 1 atomic, i.e. avoid splitting it in two parts, but still run the rest of the tests in parallel. Of course whether or ot this is feasible depends on your environment and on how you trigger the tests, but that would allow you to have a better structure of your tests plus the benefit of fast execution. So you would end up with this:
Test 1 starts Test 2 starts
Test 1 waits Test 2 finishes
Test 1 waits Test 3 starts
Test 1 runs Test 3 finishes
Test 1 finishes
I am not sure about the start—>wait—>wait—>run approach. It might work for few tests and may not work well for hundreds of tests, as the wait time would be more. Even if we run it in parallel mode it would consume some time. What if we wait for more such time taking components in the same flow? More the components more the wait time, I guess. You may also need to consider timeout of the system, if you wait for longer time...
I feel even the first approach should be fine. No need to create multiple files for a test case. You can structure it in the same file in a way that you run first part and end it. And, after ensuring the batch processing you can start with second part of your test case(file). The first part can be run in parallel mode and after waiting time part 2 can also be executed in parallel.
Is there a plugin or can I somehow configure it, that a job (that is triggered by 3 other jobs) queues until a specified time and only then executes the whole queue?
Our case is this:
we have tests run for 3 branches
each of the 3 build jobs for those branches triggers the same smoke-test-job that runs immediately
each of the 3 build jobs for those branches triggers the same complete-test-job
points 1. and 2. work perfectly fine.
The complete-test-job should queue the tests all day long and just execute them in the evening or at night (starting from a defined time like 6 pm), so that the tests are run at night and during the day the job is silent.
It's no option to trigger the complete-test-job on a specified time with the newest version. we absolutely need the trigger of the upstream build-job (because of promotion plugin and we do not want to run already run versions again).
That seems a rather strange request. Why queue a build if you don't want it now... And if you want a build later, then you shouldn't be triggering it now.
You can use Jenkins Exclusion plugin. Have your test jobs use a certain resource. Make another job whose task is to "hold" the resource during the day. While the resource is in use, the test jobs won't run.
Problem with this: you are going to kill your executors by having queued non-executing jobs, and there won't be free executors for other jobs.
Haven't tried it myself, but this sounds like a solution to your problem.
can we run two thread groups parallel by creating a single test plan in Jmeter ??
Example:
I have to add 2 test cases in a test plan, which has to be executed in parallel and can we combine this test plan with any other test plan to be executed simultaneously
Jmeter supports running more than one scenario in parallel as part of the same test plan.
Each scenario is managed in its own Thread Group element.
So for your case, add a new Thread Group to the test plan, and set the steps for the second scenario there. When you have more than 1 Thread Group, you can configure the test plan to start them at the same time (or one after the other).
There is no guarantee that the requests will be in the exact same time, but both Thread Groups will start simultaneously.
Hope it helps :)