I am evaluating AWS device farm for running mobile web test. These are the steps I am anticipating:
Create sample tests (Java)
Package that as zip file
Go through the AWS device farm console and upload the test zip only
Manually select the configuration and other things
Manually execute the test and evaluate results
For thing I need help are:
a) What if the tests needs some changes, do I need to go through the JAR package creation for every run? Can I run test from my IDE and if everything works fine then only package and upload it on AWS device farm?
b) To do (a) I noticed they have API to simulate step 3-5 to achieve running tests but wondering if there is some easy way to do it?
The steps you've listed are the correct sequence of tasks that need to be performed to run tests on a device. With AWS Device Farm, you have to perform the extra step of uploading the tests and application to the service. As you stated, every time you change your tests, you will need to rebuild the JAR and upload it. Most customers set up a continuous build/integration pipeline using a tool like Jenkins to perform this task automatically.
If you are running in Android Studio, you can use the Device Farm Gradle plugin, which will do the work for you.
Related
I have big task in work. I need to run my tests in parallel to reduce time of execute. At the moment i have around 150 end to end tests which are testing windows desktop app and web app (~1,5h) but i need to run it with 4 different systems which we support, so execution time went up to around 6h for all.
Runner for my tests is nunit, code is written with C#, and everything is stored in TFS. I need to create new project in Jenkins (Pipeline or other within i can achive my goal). I tried a lot of methods, but i don't know how to run multiple test on few VMs or RDPs (to have isolated desktop where i can use mouse and keyboard for each test). Did someone of you challenging same problem as i mentioned here and have some examples to try? I downloaded Parallel tests executor plugin for jenkins but don't have knowledge of how to use it with TFS repository.
We are working with a client that uses VeriFIX to test their FIX message flow. Whilst they have built up lots of tests in many suites, it is a manual process to run them and to collate the results.
On the VeriFIX website it says
Incorporate tests into nightly builds using VeriFIX’s command-line script player.
but I cannot find any details on how to to it. Does anyone have any experience in running VeriFIX tests in a continuous integration server (ideally a Jenkins pipeline).
Many thanks.
You can run VeriFIX playlists in batch mode from the command line:
"%VERIFIX_HOME%\verifixbatch\verifixbatch.exe" -version "FIX (x.y)" -playlist "myplaylist" -disablelogging "false"
If you have received the user manual with your installation of veriFIX, the details of how to integrate with CI are in there.
To integrate veriFIX with Jenkins you will create batch files containing tests and run the batch files as jobs in Jenkins.
The placement of your veriFIX installation is important. If your veriFIX is on a users machine, as is often the case, separate from the environment machine Jenkins resides on, there can be difficulties getting the tests to run.
If you have a centralised install of veriFIX things are much easier.
I'm new to Jenkins, so please go easy!
I have a web application which we are developing, and we've started automating our release using Jenkins.
I also have a standalone Selenium WebDriver script which will perform a Smoke test on our web app. We usually run this manually each time there's a new deployment.
I heard Jenkins can automatically trigger Selenium tests. So what I did was to create a batch file, which will in turn call the Selenium script. I added a Build Step which will call this batch file.
What's happening now is Jenkins first Builds the WAR file, executes the batch (for selenium) and then deploys it to the target Tomcat.
But I was wondering if I could change the order to Build WAR --> Deploy to Tomcat --> Call the Batch file that executes Selenium Test. I want to do this as Jenkins Tests before deploying, which means my Selenium test still runs on the old build. This makes little sense, as I would rather run the Selenium test on the new build.
In short, is there a way I can execute the Batch file as part of a Post Build Step rather than a Build Step?
Thank you Würgspaß !!
I solved my problem by creating a separate Selenium Job which gets triggered automatically if my Build is successful. This way, I can create any number of downstream jobs to be triggered for a successful build.
Basically my requirement is :
I have developed a framework using selenium webdriver,maven and testng for a application under test.
Now i have to pass that on to the client without sharing the source code ..so that if he wants he can run the test in his machine without having everything else installed ..except java. and the output should be stored in one of the specific folders of the client machine.
Can i use any plugin in maven which will store the result or copy the result file to any folder of client machine.?
Please guide me how to achieve this.. how to pass on the above to client so that test run smoothly in his machine .
is there anything i need to take care while am using maven in the project?
I am using chrome browser to run the tests..
The application under test is available to be accessed from any type of browser.
kindly suggest.
We are building apps for iOS and Android. Currently we are experimenting with TeamCity for CI.
Currently we have various tools to help us in the Test automation.
1. Selenium: For testing the web interface
2. SOAPUI: For testing the web services
3. Calabash-iOS: For testing iOS app
This is the scenario that we have done successfully with TeamCity:
1. Developer commits code to BitBucket
2. TeamCity runs calabash
3. If calabash pass, TeamCity will build and give the artifacts
This is the scenario that I am trying to do so that I can achieve an end-to-end automated testing:
When developer commits his code to our Bitbucket git repository,
TeamCity will trigger Calabash to run and check for the UI and the output of the details. On end,
TeamCity will trigger Selenium to add data in the Back Office web. On end,
TeamCity will trigger SOAP UI to check newly added data. On end,
TeamCity will trigger Calabash to check the newly added data
Has anyone tried this before or can give me an idea on how I can approach this?
You can setup different builds for each step since you can trigger builds via simple GET requests triggered by a script. TeamCity Accessing Server by HTTP
We run a similar setup. Basically we have one build kick off the Selenium/SoapUI scripts. That build doesn't wait for the job to finish, it just kicks it off. Then in our Selenium/SoapUI tests as the last step we hit the URL for the next build that picks up reports etc.