I've been using Selenium IDE for regression tests, but as the number of test cases increases, I've moved it over to Jenkins to run.
However, some tests that run perfectly in Selenium IDE fail in Jenkins. Simple commands like verifying text presence.
Is it just me? Has someone else encountered the same problem?
Trying adding the command waitForElementPresent and maybe a small pause (like 1000 ms) before you verify the text is present.
We ran into similar issues; the tests would run fine locally, but break on Jenkins. Slowing it down seemed to do the trick for us. Add more details to what your failing tests look like if this doesn't help.
Related
I'm trying to improve the testing process where I work, but without adjusting the structure.
What we have: VSTS, Selenium IDE, Testers who write test cases, but not code.
What I'd like to do is manage a way to marry our TFS continuous integration with the Selenium tests we write. These are NOT the code-driven selenium tests, but rather the IDE version where users click through, and set assertions using the IDE (All are just UI tests). I know we can export those tests plans as a .SIDE file, but what I can't figure out, is how to have our TFS server execute those as part of a deployment or build pipeline.
Ideally, developers/devops would setup projects in TFS from the onset with whatever solution makes sense to execute these Selenium .SIDE files, but afterwards, the testers would manage adding/modifying those tests cases elsewhere.
The real goal here is to not have testers writing code, or checking in code. Only writing these UI Selenium tests, but having TFS execute those as part of CI.
Researching this on the internet drives me basically always to something that requires testers to write code.
I don't think it can automate testing without code, at lease, you need a test project containing your automated tests.
Generally, in Azure DevOps, we use Visual Studio Test task to run tests. This task supports using the following tests:
Test assembly: Use this option to specify one or more test assemblies that contain your tests. You can optionally specify a
filter criteria to select only specific tests.
Test plan: Use this option to run tests from your test plan that have an automated test method associated with it. To learn more about
how to associate tests with a test case work item, see Associate
automated tests with test cases.
Test run: Use this option when you are setting up an environment to run tests from test plans. This option should not be used when
running tests in a continuous integration/continuous deployment
(CI/CD) pipeline.
This was a question that I had as well, and I think I found an imperfect but better solution.
I wasn't able to get my Selenium IDE tests running with Jenkins, but I was able to get them to run with TeamCity, another CI.
I created a build step like the following :
Runner type: Command Line
Working Directory: where the selenium IDE .side file is located
Run: Custom Script
With the build script content that I usually use to run my Selenium IDE Tests, such as selenium-side-runner sidefile.side
I also added the following so I could output the results in Junitor another form: --output-directory=results --output-format=junit
You can also add the following so the tests are run headlessly, this only works in Chrome : -c "goog:chromeOptions.args=[--headless,--nogpu] browserName=chrome"
Finally, I also use --filter to run one test suite at a time, but that is optional too.
I then used another build step to export the results to our test manger, xray, but I think that is beyond the scope of this question.
The problem with this solution is that it runs directly from a users individual machine still, but this can be work around.
When I'm running my unit tests right now (they are very intensive, lots out console output and run for a long time)... I noticed that my disk space was disappearing.
Turns out there are a few files being created
I'm not really sure what is causing this: IntelliJ, the IntelliJ Junit plugin or the console somehow. I don't think it is the console because after I noticed this problem, I installed a different console and the problem still occurs. I googled but couldn't find information about these files.
Anyone know how I can disable this?
I have created a Jenkins maven task to run selenium tests on one project, and now I want to use these selenium test in a proper way for CI.
Actually, I have a Jenkins task which 1-builds the project, 2- uses sonar, 3 - deploys the project. I would like to add the selenium tests to this process. The question is: can I run the selenium tests before deployment? Is it necessary to do a previous deploy for the selenium tests before the real deployment? Is there anyway to simulate a deployment or something like that so I can run the selenium tests?I would appreciate If you could advise me on how to do or any plugin which could help me.
As per my understanding, To deal with the scenario you should run the selenium test on QA environment. If all test pass then deployment should start for staging etc.
Additionally, once the deployment is done on staging, then selenium script should run again, test staging and if something went wrong then rollback from staging should be happened.
I never tried it but you can use below github plug-in for deployment(in case you are using github) :-
https://wiki.jenkins.io/display/JENKINS/GitHub+plugin
If you are using SVN then use below :-
https://www.packtpub.com/mapt/book/application_development/9781783553471/3
Hope it will help you :)
I would like to run local Selenium test script written in Java, via Jenkins/Hudson. Is it possible to run scripts from my local windows machine? So far I have written some simple Selenium tests in Java, which run perfectly if I execute them via Eclipse IDE. I would be thankful for an in-depth explanation.
Selenium test in Java: assuming them to be laid out as unit tests (junit or testng), second assumption is that project is governed by either ant or maven, so there is some test (or rather integration-test) target or phase being present and be running smoothly when invoked from IDE.
When such tests are launched, they hit to a running selenium server (remote-control) which in turn launch a browser and runs its magic. Here are options: selenium server might be running in background (and be always available), or it might be started right before that testing and shut down afterwards. The latter is a common case for maven: pre-integration-test phase is configured to launch selenium rc, (then integration-test phase runs the tests against it), post-integration-test shuts selenium rc down.
So up to this moment we might want to learn what targets (ant) or phases(goals) your IDE invokes when it launches your tests successfully (also, what variables it sets or what profiles it enables).
If you invoke the same command from cmd (like 'mvn clean integration-test') and it runs successfully (same as IDE), then just instruct jenkins to run the same goals/targets (I assume that jenkins is running on the same machine, at the same user session).
If cmd doesn't do the trick (and you've looked quite well into what IDE does for you when it launches your tests), then I'd asked for more details.
So, involved participants are: 0. ant/maven that run your junit tests 1. selenium rc that should be running in bg or launched by ant/maven first 2. browser (path to browser executable) 3. jenkins (which was assumed to be running in the same environment).
If any of the assumptions are false, please come up with more details of your setup.
I have some Selenium IDE scripts for automating tasks (deploying to an internal server via it's web interface), and I want to have bookmarks to launch them.
Don't want go to Tools -> Selenium IDE -> search test -> launch test, just want to click a bookmark and let the script run.
How can I achieve this? Am I missing a point?
Don't know if it makes a difference, but I'm currently using Firefox (if I have to change browser for this, it wouldn't be a problem).
Thanks
Not sure whether you be able to run tests pressing on a bookmark as browser plugins are usually run with a key combo. And bookmark is URL. Well the main issue in your question how to associate bookmark
with browser addon launching.
Well, I've investigated a lil bit workaround. Speaking about getting Selenium HTML Test Suites Running Automatically (i mean an alternative to bookmarks) some solutions are possible:
solution 1:
Patch the Selenium server code to support HTML suites.
Build a script to execute your HTML test suites.
Build a script to read the suites’ output reports and consolidate
the information for inclusion into your CC.NET report.
Setup details you can get here
Solution 2:
Selenium IDE plugin for Firefox to create tests
Java to run Selenium Server
Selenium Server to run test suites
Nant to generate reports for CruiseControl.NET
CruiseControl.Net as a continuous integration engine
Setup details here
Hope this be useful for you.
Bookmarks are nothing else than URLs, which makes it impossible to "bookmark" a functionality of a plugin, such as the Selenium IDE.
However, there might be other plugins out there that do what you want. I haven't had a closer look at it, but one candidate could be the Favorites (Selenium IDE) Firefox Add-on.