I would like to achieve running one example test case for multiple devices in parallel. I see that in doc there is an information about maxworkers. I add in detox config new device like 3 times add type of device and increase the number of workers to 3. Unfortunately it works only on one device. Any ideas, tips what should i do to make running tests on multiple devices in parallel? I need to run test cases for 7 emulators/devices to make sure that everything works correctly.
Related
As far as I know TestCafe default behaviour is to run tests in parallel.
Indeed the browsers function accepts an array of browser (which is cool).
What I would like to do however is quite different. I have fixtures based on area of my portal (search, payment etc...) and so I'd like to know if it's possible to run these tests in CLI in parallel as they are orthogonal.
The scope is of course to improve the execution time as the number test
cases will grow.
On the other hand I'd like also to catch the failures meaning that if a test ran in parallel on a specific metadata filter fails possibly we would like to stop the others too.
I am not using TestCafe's docker but our custom one with just Firefox, Chrome installed and we launch of tests in headless mode.
As a last point a great thing would be if we could run these scenario/metadata in parallel but somehow at the end of the test suite gather the reports together.
I understand the question is not easy especially because it involves either TestCafe or GitlabCi but probably someone else faced this problem too.
Thank you
If I understand you correctly, the behavior you described can be achieved by dividing the test execution among multiple CI jobs. For example, each CI job can test a particular area of your portal. For that, run TestCafe with specified metadata of your fixture/test. Also, most of the CI systems allow you to cancel all other jobs in a pipeline if one of the jobs fails (unfortunately, Gitlab hasn't released this feature yet).
On the other hand, you can use TestCafe's programmatic API: create multiple TestCafe runners, each running the desired subset of tests. However, at the end of the test execution, you'll need to merge generated reports into one report manually. Check this answer to get an idea of how to create multiple runners.
Background->So i am using Zephyr for Test case management.I have a manual test suit with around 100 test cases .All test cases i have to execute on around 10 mobile devices.
I dont want to write same test cases for 10 devices as it will become huge(100X10),rather i was exploring if in any case i can use same zephyr test case for all 10 devices and get reports separate for 10 devices.The problem is execution will be done by multiple testers and keeping just 1 test case will also be a problem as all testers can not kep on executing the same test case . Any solution to overcome this situation?
With the caveat that my group is using the Zephyr for Jira plugin, the solution my group is experimenting with track release sign off across our platform matrix is:
Create test cases
Create a cycle to track release sign off
Create a folder under the cycle to represent one platform/device (think test run)
Add test cases to folder (think test results for this platform)
Clone the folder and it's contents, rename to next platform
Repeat the cloning until entire matrix covered
Testers go into each folder to run tests and log results
It looks like the reporting is broken out by folder in the Cycle Summary page in Jira. Which means it's possible to track progress by platform/device.
I am using Selenium grid to scrape thousands of pages since all the pages are heavily populated by Javascript.
I found this tutorial which gave me a pretty good idea of how to set up Selenium grid and run script in parallel. However, my situation is a little different.
(1) I only want one type of browser, like Chrome(or Firefox), but I want to run as many as possible.
(2) To make sure this solution scale, I probably will use some Cloud service where the code will be running in Linux environment.
So here is my question:
Do I have to use TestNG/Junit frame work to run the code in parallel? If I run the code in multiple processes, all making requests to the same hub, will the hub coordinate them out of box?
(1) I only want one type of browser, like Chrome(or Firefox), but I
want to run as many as possible.
You should be running not as many possible, but a heuristic number, that just works for you. The reason being is, running like 30 chrome browsers at a time can give you unpredictable results.
(2) To make sure this solution scale, I probably will use some Cloud
service where the code will be running in Linux environment.
You can look at BrowserStack
Do I have to use TestNG/Junit frame work to run the code in parallel?
Thats upto you. As far as your creating the driver in multiple threads your fine. If your using your own FW, then you can create Thread pool and start creating the driver from each thread pool.
If I run the code in multiple processes, all making requests to the
same hub, will the hub coordinate them out of box?
Yes Selenium Hub will be co-ordinating this for you, out of the box. You no need to worry about anything here.
I have developed robotium automation script and its running sequentially in single genymotion instance
I want execute Parallelly two different automation script in two diffrent geny motion instances simultaneously
After opened two genymotion instance , i did ...
adb devices
List of devices attached
192.168.56.103:5555 device
192.168.56.101:5555 device
I have opened one terminal and exported : export ANDROID_SERIAL=192.168.56.101:5555 , one instance of geny motion
I have opened other terminal and exported other instances ANDROID_SERIAL=192.168.56.103:5555
I have executed the different script in corresponding terminal window but genymotion is not running different script in different instance , its running same script in two instance and generated the report .
Please let me know I want to run two different robotim automation script paralley in genymotion ?
Expecting you idea and solution !
Thanks ,
Muthu Selvan SR
So there are couple parts to this question as far as i can see. It seems like you have the first part sorted which is to have two devices connected to your machine and it seems like you can connect to them via ADB.
The next part is to be able to tell the devices to run the tests, it seems like whatever command you are using (if you could post that it might help) is telling it to run the same tests. Sadly there is no way to auto paraallelise the tests across the two devices so you will have to manually determine the sets of tests to send to each device which can be done in a number of ways.
If you look at the documentation here you can see that there are many ways to filter what tests should be run. What you will want to do is find a way to halve them approximately equal between the two emulators so that you get the desired results. I give an example of doing something like this over at this question
You can create a BAT file (Windows) or shell script .sh (UNIX, MAC) which will contain something like the following:
adb -s <serial1> shell am instrument -w <package.test>/android.test.InstrumentationTestRunner&
adb -s <serial2> shell am instrument -w <package.test>/android.test.InstrumentationTestRunner
And run your tests. This will run your test on both the devices at the same time.
I'm trying to simulate a firefox load testing situation. I want my to test how 10 simultaneous logins would play out on my system. I already have a connected selenium grid hub and 10 open nodes.
So far, I know I can write the test case and run it 10 times which isn't what I need because it isn't automated. I also know that I can use invocation count on the test to make it run as many times as i want but this only works on the same browser node.
Does anyone have any ideas on how to automatically distribute the same test case to multiple instances of the same driver profile?
i.e. Run a login case test times on the same firefox profile open in 10 different nodes in parallel.
Gracias!
P.S. I built my tests using testNG if that matters.
Basically selenium and testNG is not for such requiurement. You should use some dedicated tool for that like jmeter.
However you can run n methods parrallel let say if you want to login with 10 dif user in 10 thread/browser you can create test data driven and configure to run method in parrallel. Make sure you are providing proper value of parrallel thread count.
How about combining threadpoolsize with invocationcount. - http://testng.org/doc/documentation-main.html#parallel-running
Grid would take care to distribute on the 10 nodes.
use headless browser like GHOST and then invoke multiple threads as ghost has no UI so it would work in your case