I would like to run local Selenium test script written in Java, via Jenkins/Hudson. Is it possible to run scripts from my local windows machine? So far I have written some simple Selenium tests in Java, which run perfectly if I execute them via Eclipse IDE. I would be thankful for an in-depth explanation.
Selenium test in Java: assuming them to be laid out as unit tests (junit or testng), second assumption is that project is governed by either ant or maven, so there is some test (or rather integration-test) target or phase being present and be running smoothly when invoked from IDE.
When such tests are launched, they hit to a running selenium server (remote-control) which in turn launch a browser and runs its magic. Here are options: selenium server might be running in background (and be always available), or it might be started right before that testing and shut down afterwards. The latter is a common case for maven: pre-integration-test phase is configured to launch selenium rc, (then integration-test phase runs the tests against it), post-integration-test shuts selenium rc down.
So up to this moment we might want to learn what targets (ant) or phases(goals) your IDE invokes when it launches your tests successfully (also, what variables it sets or what profiles it enables).
If you invoke the same command from cmd (like 'mvn clean integration-test') and it runs successfully (same as IDE), then just instruct jenkins to run the same goals/targets (I assume that jenkins is running on the same machine, at the same user session).
If cmd doesn't do the trick (and you've looked quite well into what IDE does for you when it launches your tests), then I'd asked for more details.
So, involved participants are: 0. ant/maven that run your junit tests 1. selenium rc that should be running in bg or launched by ant/maven first 2. browser (path to browser executable) 3. jenkins (which was assumed to be running in the same environment).
If any of the assumptions are false, please come up with more details of your setup.
Related
I'm trying to improve the testing process where I work, but without adjusting the structure.
What we have: VSTS, Selenium IDE, Testers who write test cases, but not code.
What I'd like to do is manage a way to marry our TFS continuous integration with the Selenium tests we write. These are NOT the code-driven selenium tests, but rather the IDE version where users click through, and set assertions using the IDE (All are just UI tests). I know we can export those tests plans as a .SIDE file, but what I can't figure out, is how to have our TFS server execute those as part of a deployment or build pipeline.
Ideally, developers/devops would setup projects in TFS from the onset with whatever solution makes sense to execute these Selenium .SIDE files, but afterwards, the testers would manage adding/modifying those tests cases elsewhere.
The real goal here is to not have testers writing code, or checking in code. Only writing these UI Selenium tests, but having TFS execute those as part of CI.
Researching this on the internet drives me basically always to something that requires testers to write code.
I don't think it can automate testing without code, at lease, you need a test project containing your automated tests.
Generally, in Azure DevOps, we use Visual Studio Test task to run tests. This task supports using the following tests:
Test assembly: Use this option to specify one or more test assemblies that contain your tests. You can optionally specify a
filter criteria to select only specific tests.
Test plan: Use this option to run tests from your test plan that have an automated test method associated with it. To learn more about
how to associate tests with a test case work item, see Associate
automated tests with test cases.
Test run: Use this option when you are setting up an environment to run tests from test plans. This option should not be used when
running tests in a continuous integration/continuous deployment
(CI/CD) pipeline.
This was a question that I had as well, and I think I found an imperfect but better solution.
I wasn't able to get my Selenium IDE tests running with Jenkins, but I was able to get them to run with TeamCity, another CI.
I created a build step like the following :
Runner type: Command Line
Working Directory: where the selenium IDE .side file is located
Run: Custom Script
With the build script content that I usually use to run my Selenium IDE Tests, such as selenium-side-runner sidefile.side
I also added the following so I could output the results in Junitor another form: --output-directory=results --output-format=junit
You can also add the following so the tests are run headlessly, this only works in Chrome : -c "goog:chromeOptions.args=[--headless,--nogpu] browserName=chrome"
Finally, I also use --filter to run one test suite at a time, but that is optional too.
I then used another build step to export the results to our test manger, xray, but I think that is beyond the scope of this question.
The problem with this solution is that it runs directly from a users individual machine still, but this can be work around.
I have big task in work. I need to run my tests in parallel to reduce time of execute. At the moment i have around 150 end to end tests which are testing windows desktop app and web app (~1,5h) but i need to run it with 4 different systems which we support, so execution time went up to around 6h for all.
Runner for my tests is nunit, code is written with C#, and everything is stored in TFS. I need to create new project in Jenkins (Pipeline or other within i can achive my goal). I tried a lot of methods, but i don't know how to run multiple test on few VMs or RDPs (to have isolated desktop where i can use mouse and keyboard for each test). Did someone of you challenging same problem as i mentioned here and have some examples to try? I downloaded Parallel tests executor plugin for jenkins but don't have knowledge of how to use it with TFS repository.
I'm new to Jenkins, so please go easy!
I have a web application which we are developing, and we've started automating our release using Jenkins.
I also have a standalone Selenium WebDriver script which will perform a Smoke test on our web app. We usually run this manually each time there's a new deployment.
I heard Jenkins can automatically trigger Selenium tests. So what I did was to create a batch file, which will in turn call the Selenium script. I added a Build Step which will call this batch file.
What's happening now is Jenkins first Builds the WAR file, executes the batch (for selenium) and then deploys it to the target Tomcat.
But I was wondering if I could change the order to Build WAR --> Deploy to Tomcat --> Call the Batch file that executes Selenium Test. I want to do this as Jenkins Tests before deploying, which means my Selenium test still runs on the old build. This makes little sense, as I would rather run the Selenium test on the new build.
In short, is there a way I can execute the Batch file as part of a Post Build Step rather than a Build Step?
Thank you Würgspaß !!
I solved my problem by creating a separate Selenium Job which gets triggered automatically if my Build is successful. This way, I can create any number of downstream jobs to be triggered for a successful build.
I would like to run multiple Selenium Tests (on a Jenkins server) at the same time.
It currently runs only a single test at a time cause ChromeDriver seems to communicate over a special port. So somehow I guess I have to pass some kind of port settings via Selenium to the ChromeDriver to start up multiple tests.
The Selenium website unfortunately is empty for that topic:
http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp#parallelizing-your-test-runs
From my point of view it makes no difference if the Test runs locally or on Jenkins, the problem is the same. We need to somehow configure ChromeDriver. The question is just how.
Anybody has some ideas or pointers where to look at and what files are involved to get this done?
You can run multiple instances of chromedriver locally quite easily, just instantiate multiple driver objects, chromedriver will keep the profiles separate and find a port to run on all by itself.
Here a link to an example that can run multiple tests using TestNG and Maven:
https://github.com/Ardesco/Selenium-Maven-Template
Just clone the above project and run the following in the command line:
mvn verify -Pselenium-tests -Dbrowser=chrome -Dthreads=2
It takes advantage of TestNG's ability to manage the thread pool and will open up multiple instances if specified. You can do the same thing with jUnit but you'll need to write a custom test runner to fire the tests off into individual threads.
If you decide to use gradle it can deal with managing the thread pools for you with both TestNG and jUnit and a lot of people prefer it to maven.
This is an old question, but for anyone still reading along, it is very possible to run multiple Selenium WebDriver instances in parallel without using Grid. I have successfully tested this using Chrome, FireFox, and PhantomJs (up to 5). Each WebDriver instance uses an isolated context, so session conflict should not be an issue. Be wary of server side conflicts though, depending on the requirements of your website!
For NUnit users, NUnit 3.2.1 now has a 'TestContext.Current.WorkerId' property that will allow you to isolate one WebDriver instance per NUnit worker.
Running multiple browsers on the same machine will often hinder performance, so be careful not to use too many browsers instances, or you may actually increase your testing time!
What you are looking for is Selenium Grid 2.
Grid allows you to :
scale by distributing tests on several machines ( parallel execution )
manage multiple environments from a central point, making it easy to run the tests against a vast combination of browsers / OS.
minimize the maintenance time for the grid by allowing you to implement custom hooks to leverage virtual infrastructure for instance.
I agree using grid in combination with Maven parallelized class, you can run multiple instance in one PC. Jenkins is possible when you are using Ant for your build ,then you can specify which test can be run parallel.
Its quite easy to set it up though ;)
I use Selenium 2 (i.e. web driver) for unit testing in VS2010/TFS2010 environment. When the build (call it build # n) is triggered via checkin, the unit tests run (including Selenium ones) and after passing, the new site is published. As you can see, I just ran Selenium tests against an old web site (from build n-1).
So the question is, how do I make sure that Selenium tests run against the latest site. I kind of want the deploy/publish to happen before running Selenium tests. Is that possible? Also, what if Selenium tests fails - deploy would have to be reverted back.
Your build should deploy to localhost / other dev site first and run the tests, then if they pass, deploy to the production site.