Is there way to check if a certain plan variable is used in any steps in Bamboo build plan? Plan has multiple stages and each stage has few dozen steps, each step is about 1k script, so checking all manually would be time consuming.
Alternatively, is there way to export all build plan steps and stages into one or multiple text files?
Related
we started to develop some jobs with Flink. Our current dev / deployment process looked like this:
1. develop code in local IDE and compile
2. upload jar-file to server (via UI)
3. register new job
However it turns out that the generate jar-file is ~70MB and upload process takes a few minutes. What is the best way to speed up development (e.g. using a on-server ide?)
One solution is to use a version control system and after committing and pushing your changes, you might build the jar on the server itself. You could write a simple script for this.
The other solution, which would take time and effort is to set up a CI CD Pipeline which would automate the entire process and much manual effort would be minimised.
Also, try not to use fat jar to minimise the jar size if you have to scp your jar to the cluster.
First off, uploading 70 mb shouldn't take minutes nowadays. You might want to check your network configurations. Of course, if your internet connection is not good, you can't help it.
In general, I'd try to avoid cluster test runs as much as possible. It's slow and hard to debug. It should only ever be used for performance tests or right before releasing into production.
All logic should be unit tested. The complete job should be integration tested and ideally you'd also have an end-to-end test. I recommend to use a docker-based approach for external systems and use things like test containers for Kafka, such that you are able to run all tests from your IDE.
Going onto the test cluster should then be a rare thing. If you find any issue that has not been covered by your automatic tests, you need to add a new test case and solve it locally, such that there is a high probability that it will be solved on the test cluster.
edit (addressing comment):
If it's much easier for you to write a Flink job to generate data, then it sounds like a viable option. I'm just fearing that you would also need to test that job...
It rather sounds like you want to have an end-2-end setup where you run several Flink jobs in succession and compare the final results. That's a common setup for complex pipelines consisting of several Flink jobs. However, it's rather hard to maintain and may have unclear ownership status if it involves components from several teams. I like to rather solve it by having tons of metrics (how many records are produced, how many invalid records are filtered in each pipeline...) and having specific validation jobs that just assess the quality of (intermediate) output (potentially involving humans).
So from my experience, either you can test the logic in some local job setup or it's so big that you spend much more time in setting and maintaining the test setup than actually writing production code. In the latter case, I'd rather trust and invest in the monitoring and quality assurance capabilities of (pre-)production that you need to have anyways.
If you really just want to test one Flink job with another, you can also run the Flink job in testcontainers, so technically it's not an alternative but an addition.
There is a built in mechanism for protractor to run multiple instances of chrome for a given number of test suites.
However two tests running in parallel can and will change common data causing one or both to fail.
My best bet at the moment is to use docker containers running the app with separate mongo dbs which I'm thinking is a pain to set up.
This probably won't be the answer you want, but... the trick to running parallel tests is to ALWAYS write your tests so they can be run in parallel. This means taking advantage of any and all strategies towards this goal, including using multiple users/accounts, and creating/deleting test data for each test. This also means tests cannot depend on other tests (coupling), which is a bad idea regardless of sharding.
The reason you need to do it up front is there are no situations where you wouldn't want your tests to run faster. And in addition to just sharding protractor tests, you may want to further increase test speed by also employing Docker containers in parallel.
Again... probably not what you want to hear, but...
Good luck!
I want to observe the dynamics of performance changes.
For example, I have method which generates random number. The first version of this method spends 350 ms to generate. The second version spends 450 ms.
Tests on the TFS build server have to throw error because now the method is running slower.
Can TFS storage and compare previous performance results?
How to write test methods to do that?
I would recommend that instead of doing something as complicated as comparing does why not just f run it once and decide.
You can easily compare the current execution time to a 350ms baseline and fail the test of it goes over.
If you go down the road of comparing to the last you will unnecessarily increase the test time. You will add the time to load the last from storage and danger the new value...
You could use the Stopwatch class to measure the time needed to perform the calculations. After the calculations are done you write the time needed in to a file. On the next testrun you compare the measured time to the value in the file. If It's the same or faster you replace the value in the file and let the test succeed.
If it is higher, you fail the test and write the time previously needed and the time your test needed this time as your fail message.
If the file does not exist, your test has to succeed since it can't compare the measured time.
Of course that's not using the TFS test environment directly, but this way you can fine tune your stuff the exact way you want it.
I have four child pipelines in my project and the output for these is ingested in the main pipeline. I want the output files for the child pipelines to be automatically backed up after baseline in some directory. This will help if i disable individual forges and want to restore previous index. Please help
It should be straight-forward to add copy or archive jobs to your baseline update script to execute this sort of thing before your main forge is run.
radimbe, as forge itself is single-threaded (except for the rarely implemented multi-threaded java manipulator) and monolithic, this sort of architecture is commonly used to take better advantage of multi-processor machines and multi-threaded CPU cores. In addition, if data is made available at different times or with different frequencies, you can decouple child forges from your main baseline process, improving its overall turnaround time. And from a strategy POV, this approach could decompose what might be a large, unwieldy job into perhaps simpler, more focused and more easily maintained components.
We run BDD tests (Cucumber/Selenium) with Jenkins in a Continuous Integration process. The number of tests is increasing day by day and the time to run these tests is getting higher, making the whole CI process not really responsive (if you commit in the afternoon you would risk to see your building results the day after). Is there a way/pattern to keep the CI process quick in spite of increasing number of tests?
You could choose one of the following schemes:
Seperate projects for unit tests and integration tests. The unit tests will return their results faster and the integration project will run once or just a couple of times per day and not after each commit. The drawback is obvious, if the integration tests suite break there is no correlation with the breaking change.
Google approach - sort your tests according to their size: small, medium, large and enormous. Use separate projects for each kind of test and according to the total time it takes to run the specific test suite. You can read more in this book. Also, read this blog to get more wise ideas.
Try to profile your current test suite to eliminate bottlenecks. This might bring it back to give feedback in a timely fasion.
Hope that helps.
#Ikaso gave some great answers there. One more option would be to set up some build slaves (if you haven't already) and split the integration tests into multiple jobs that can be run in parallel on the slaves.