I'm not running robotium test case using command prompt, instead i am navigating to Dev Tools and finding the suite on that and running it and it is working fine for me on the Emulator.
But in case if it is failure, it is exiting the app. I just wanted to know that how to collect the logs to verify that which test case is failed.
The same way i need to know that is there anyway, i can run the suite from devices without connecting to PC (like how i am running on emulator using Dev tools)
Thanks
If you change your instrumentationtest runner to something like:
http://code.google.com/p/the-missing-android-xml-junit-test-runner/
then an xml file will be created containing all your test runs. This will also be useful as it is in standard junit xml format so you can put it into any CI system and it should work.
Related
I am using the Karate framework to do the API testing. As part of CI efforts, we send an email at the end of test execution listing the summary of test results. There is a need to include the screeshot of the test execution counts from 'overview-feature.html' file.
I did so through the TestRunner.java file - launched Chrome using Chrome.start() and then using it to take screenshot. It all works well locally on Windows.
However when executing on CI server which is a Unix box, the chrome executable is not present in the default location (usr/bin/google-chrome) and hence the connection for the localhost fails.
Is there a way we can change the default location of the chrome executable?
PS: Apologies if this was too trivial to be asked.
Yes Chrome on CI is hard to get right, refer: https://stackoverflow.com/a/62325328/143475 - note that CI boxes typically are "headless" a browser may not be even installed.
I think the best thing for you is to ZIP the HTML and send it. But I really think you need to work with some CI experts, because the report generation and e-mailing business is normally done by things like Jenkins. What you are doing is certainly not normal or best-practice.
If you really want, there is a Karate Docker container that can give you a proper Chrome instance (see docs) but that is overkill for what you need.
EDIT: The Chrome Java API allows for customization of the executable path and this is in the docs: https://github.com/intuit/karate/tree/master/karate-core#chrome-java-api
It should be something like this:
Chrome.start("/opt/blah/chrome");
I really don't know how to ask question to Google about this, so I excuse me that it is naive.
Our team is developing SPA application in ReactJS. We also do back-end programming for NodeJS. Our project recently got more e2e tests. They are written using webdriver.io packages. Everything works as expected but circa 30 tests run about 50 minutes. It is too long to pause developer work and force him to run tests.
We came with the idea that now when we have so many tests, we need to run them on separate computer (other than a developer's laptop, further I call it e2e-laptop).
So I programmed a bash script and installed Ubuntu on a e2e-laptop. My idea is, that developer who wants to run e2e test logs in on e2e-laptop with ssh, runs specified script with arguments (eg: --rev= specific git revision the tests should run on, --email= where to send Allure report) and logs out. After tests are done he gets Allure report in his mailbox.
This all sounds to me OK, but not very well. It works - it is like a dirty MVP. But what I really would like to give my team is the web browser based UI that gives the features my script has. I can imagine this software is hosted on e2e-laptop, every developer can open its webpage address in his local browser. Then after authorization, there are options: run all specs, run chosen specs, send report and more. It would be the best if that software could also allow simultaneous running of tests commissioned by multiple developers.
What software I need?
You need a continuous integration tool. https://stackify.com/top-continuous-integration-tools/
I recommend Jenkins.
I would first try to run your selenium tests headless in a docker container on your laptop. Once you are able to do that, use that same configuration in your docker container running in Bitbucket pipelines. It could actually be the same container and the same scripts. Then, developers can just make a branch and work with the tests on that branch. If only a certain subset of tests need to run, then the developer can make the necessary changes on his or her local branch to run those tests and push it up to Bitbucket. This should help with the configuration https://github.com/SeleniumHQ/docker-selenium.
Should end to end tests be run at build time (running the application on the build server), or after deployment? I have not yet found a solid answer for which one is the standard.
Edit
I mean after deploying either to QA/SIT/UAT etc... vs. just running it on a build server without fully deploying it.
The whole point of having a build server is to create a single build of the current source code, of which you run tests and make sure that things work before you deploy them. I don't know why anyone would want to run tests after then have been deployed. What happens if you find a bug? You going to roll back the deployment? Always test before deployment.
Ideally, you would have a build environment that mimics your production environment that will allow you to run tests in a "deployed" environment. It's the reason that you have a development/staging/production servers.
I'm trying to run Selenium tests utilizing an IISExpress server. I'm using the VSTest plugin to run my tests. Everything works fine locally in Visual Studio but when I run the tests in Jenkins the IISExpress process never starts. I even tried adding a Windows Batch Command step in Jenkins specifically for loading up the server, but even that doesn't seem to do anything. When I run Jenkins as a service that can interact with the desktop and I watch what its doing, I can see that Selenium loads up the browser but IISExpress just never starts. Any thoughts on this?
You should be able to launch IIS Express through command prompt by either
Explicitly providing the /path:"#PATH" and /port:"#PORT" command line arguments, and then invoking an asynchronous call to the IIS Express executable.
Your command in Jenkins should be something along the lines of:
start "YOUR_IIS_EXPRESS_EXE" /path:"PATH_TO_YOUR_APPLICATION" /port:"YOUR_APPLICATION_PORT"
Or by
Providing a value for the /config command line argument.
Whilst using this technique however, ensure that your application is the first entry in the application host.config file.
The Jenkins command should then be something similar to:
start "YOUR_IIS_EXPRESS_EXE" /config:"PATH_TO_YOUR_APPLICATIONHOST.CONFIG_FILE"
I'm trying to set up a an automated build process and together with some coded ui tests. I think I've managed to set up pretty much everything up and working, the last missing piece of the puzzle being able to run the coded UI tests on the test agent machine.
So basically, I have a CI build that also runs unit tests, and if successful, deploys the binaries on a shared location. My goal is to then trigger the other process that runs the coded UI tests. I got the coded UI tests working on my dev computer by hard coding the location to start the application from. However, I am at a loss on how to configure this to work on the test agent. I used the LabDefaultTemplate11 build process template, and configured it to use the latest build completed by the CI build. But how do I specify what executable the test agent should use?
At first I thought it was enough to specify the build definition and build configuration, but then I realized there might be multiple executables, so the test agent would have to guess. Doesn't sound too good.
So in the end I guess my question is, how to (robustly) add the startup of the application to my coded UI tests in a manner that works both on my local dev machine, and the machine running the test agent?
Oh and I'm using TFS 2012 (with VS 2012 premium).
The lab template expects you to create Test Cases in MTM then associated coded ui tests to them in visual studio by opening the test case, selecting the associated automation tab and clicking the "..." button. You need to have the project with the coded ui tests open at the time.
Then in the lab build you select one or more Test Suite (from MTM) that contains the Test Cases for those coded uit tests.
When you make your tests in the first place make sure you're running your program/website in a way that the test agent will be able to also - eg use a standard installation directory or domain.
It is best practice to open the program being tested at the start of every test and close it at the end. However you could get around that by executing the program as part of the deploy instructions in the lab build.