In our CI pipeline we are running automated acceptance tests through selenium. We are now looking to run these through a recording proxy and then use the recording as a basis for a load test.
This strategy will help us automatically add new acceptance tests to the load test. We are currently running load tests with JMeter, hence it would be nice to be able to achieve this using the same tools. However we can't figure out a way to start recording from the JMeter command-line interface. Is this at all possible? If not isn't here other tools that has this feature set?
It doesn't seem to be possible as per JMeter 3.1 so you will have to use GUI or go for another tool.
Taurus can be considered as an option, the main features you might be interested in are:
it's free
it's open source
it is command-line application
it supports JMeter and Selenium tests (you can use Taurus to execute them in parallel)
and finally, it comes with Proxy2JMX Converter module which can be used for recording Selenium tests in command-line mode and converting them into a JMeter test plan. Moreover, you will even get automatic correlation of dynamic parameters. See How to Convert Selenium Scripts into the JMX Converter article for details.
Related
I'm trying to improve the testing process where I work, but without adjusting the structure.
What we have: VSTS, Selenium IDE, Testers who write test cases, but not code.
What I'd like to do is manage a way to marry our TFS continuous integration with the Selenium tests we write. These are NOT the code-driven selenium tests, but rather the IDE version where users click through, and set assertions using the IDE (All are just UI tests). I know we can export those tests plans as a .SIDE file, but what I can't figure out, is how to have our TFS server execute those as part of a deployment or build pipeline.
Ideally, developers/devops would setup projects in TFS from the onset with whatever solution makes sense to execute these Selenium .SIDE files, but afterwards, the testers would manage adding/modifying those tests cases elsewhere.
The real goal here is to not have testers writing code, or checking in code. Only writing these UI Selenium tests, but having TFS execute those as part of CI.
Researching this on the internet drives me basically always to something that requires testers to write code.
I don't think it can automate testing without code, at lease, you need a test project containing your automated tests.
Generally, in Azure DevOps, we use Visual Studio Test task to run tests. This task supports using the following tests:
Test assembly: Use this option to specify one or more test assemblies that contain your tests. You can optionally specify a
filter criteria to select only specific tests.
Test plan: Use this option to run tests from your test plan that have an automated test method associated with it. To learn more about
how to associate tests with a test case work item, see Associate
automated tests with test cases.
Test run: Use this option when you are setting up an environment to run tests from test plans. This option should not be used when
running tests in a continuous integration/continuous deployment
(CI/CD) pipeline.
This was a question that I had as well, and I think I found an imperfect but better solution.
I wasn't able to get my Selenium IDE tests running with Jenkins, but I was able to get them to run with TeamCity, another CI.
I created a build step like the following :
Runner type: Command Line
Working Directory: where the selenium IDE .side file is located
Run: Custom Script
With the build script content that I usually use to run my Selenium IDE Tests, such as selenium-side-runner sidefile.side
I also added the following so I could output the results in Junitor another form: --output-directory=results --output-format=junit
You can also add the following so the tests are run headlessly, this only works in Chrome : -c "goog:chromeOptions.args=[--headless,--nogpu] browserName=chrome"
Finally, I also use --filter to run one test suite at a time, but that is optional too.
I then used another build step to export the results to our test manger, xray, but I think that is beyond the scope of this question.
The problem with this solution is that it runs directly from a users individual machine still, but this can be work around.
I have a large number of automated tests written using webdriverio.
I am trying to integrate a security scanner into my build pipeline that is able to use a selenium file to automate the steps needed to get my SPA into a state that security checks can be run. Is there a way to re-use work I have done in webdriverio (pageobjects and such) and convert to a standard selenium file?
Using Webdriver, I have automated web based Application in eclipse and using QTP, I have automated Power Builder application(like desktop App).
Now both become depended module for me, from PB data's are flowing to web base application.
so, how can I achieve this. please someone provide me how to proceed with situation and give me some strategy to proceed
We have exactly the same situation where powerbuilder application is integrating with the web application, below is the solution we have followed
We are using jenkins for executions, but this can be done without jenkins also.
We have one jenkins job which executes a UFT test first and then executes the selenium test if UFT test is successfull.
At the end of test execution UFT test will generate a file with the information required by selenium test and then jenkins will trigger selenium test, selenium test will read this file to gather any required information.
Results are interpreted in the following way
If UFT test fails then jenkins will not trigger the selenium test. If the whole jenkins job is executed successfully then integration is working, if the UFT part failed then Powerbuilder is having issue and if selenium part failed then web application is having issue.
Presently we built a Automation framework which uses Selenium Webdriver+ specflow + Nunit, and we suing bamboo as our CI to run our Job against our every build.
we written a build.xml to handle our targets (like clean, init, install latest build, run Selenium scripts, uninstall build. etc)
ant command will read the tag name from the build.xml and runs the respective feature/scenarios based on Tags (like #smoke, #Regression)with Nunit in CI machine.
Now our requirement is to use Selenium Grid to divide scripts into different machine and execute with above set-up. Grid has to divide the scripts based on feature file or based on Tags.How to achieve this.
Is there any thing need to done under [BeforeFeature] and [BeforeScenario] ?
If you provide in details steps or any link which explains detail steps that would be a great help.
Please any one can help in this regards.
Thanks,
Ashok
You have misunderstood the role Grid plays in distributed parallel testing. It does not "divide the scripts", but simply provides a single hub resource through which multiple tests can open concurrent sessions.
It is the role of the test runner (in your case Specflow) to divide tests and start multiple threads.
I believe that you require SpecFlow+ (http://www.specflow.org/plus/), but this does have a license cost.
It should be possible to create your own multithread test runner for Specflow but will require programming and technical knowledge.
If you want a free open source approach to parallel test execution in DotNet, then there is MbUnit (http://code.google.com/p/mb-unit) but this would require you to rewrite your tests
Selenium: Is there any JS (JavaScript) code coverage tool which we can integrate with Selenium Server/RC
Hi,
I want to JS code coverage done by my Selenium Test Suite
Is there any code coverage tool which can be integrated with my existing selenium rc framework??
In theory, you can use any coverage tool that instruments JavaScript code to collect coverage data.
For example, you can use JSCoverage either in proxy mode (for real-time instrumentation) or for static instrumentation to run your Selenium tests off the instrumented code.
One technical hurdle could be to devise a way to accumulate coverage data across multiple pages. The coverage tool of your choice could already have support for accumulation of data from multiple page runs. If not, you might have to handle that bit yourself, but this is probably not something trivial.
I have found the solution for my need, there are multiple tools available but I want a tool which can be integrated with my existing automation easily
Tool is JSCover
http://tntim96.github.io/JSCover/
Just run the Server
Run Server
java -cp JSCover-all.jar jscover.server.SimpleWebServer . 8080
Run Proxy Server
java -jar target/dist/JSCover-all.jar -ws --proxy --port=3128 --report-dir=target
Add Proxy in your Webdriver for port specified, 3128 in my case
After execution of test Generate Reports using
((JavascriptExecutor) driver).executeScript("jscoverage_report();");
You can add this line where you exit WebDriver or Test Case
Not aware of a tool for Selenium, but JsTestDriver has a design very similar to Selenium RC (can launch tests from the command line and they are run on a server that drives browsers headlessly) and provides code coverage information.
IntelliJ integrates with JsTestDriver and provides a visual display of coverage information.
Take a look at https://github.com/yui/yuitest/wiki/Code-Coverage
I've integrated this tool in my Selenium tests a time ago. You need a bit of work to gather coverage info before page changes (in any case js trigger a page reload, link etc...)
Once you set up everything, it will fully coverage any js executed while Selenium load and test your website pages.
PS : Even if it was specially adapted for YUI test, you can use it with selenium.
I don't know what you are trying to achieve, but:
Selenium is testing the final output, as seen on the page itself.
So it really does not matter if its PHP, HTML, JSP, ASP or .NET - the Selenium is designed to mimick the end user and click the final application - the final HTML code generated by whatever is under the hood.
Selenium is also not that good for code coverage tests - one piece code can be on many pages - so better approach with selenium is to do the "user" coverage - try to cover all the possible actions which living human could possibly do with your page
There is no particular tool that can integrate with Selenium to do JS coverage. However there are lots of tools which test JS on every page which can tell if the JS that executing on your web page had any errors. This may not ideal solution but on each page you will have the measure of uptil which point JS executed properly on your webpage under test. There are two solutions for that:
1.) JSErrorCollector API: It will integrate directly with Selenium and let you know if there were any error on the page. Can be found at: http://mguillem.wordpress.com/2011/10/11/webdriver-capture-js-errors-while-running-tests/
2.) Full fledged JS coverage tools: There is an excellent list of tools here which will essentially help you in covering JS on on your web pages. Can be found at: JavaScript unit test tools for TDD