I faced a problem with the run. IntelliJ run one test two times. I use only Robot Framework Language Server plugin
That's my run configuration for one test. enter image description here
Related
I'm trying to improve the testing process where I work, but without adjusting the structure.
What we have: VSTS, Selenium IDE, Testers who write test cases, but not code.
What I'd like to do is manage a way to marry our TFS continuous integration with the Selenium tests we write. These are NOT the code-driven selenium tests, but rather the IDE version where users click through, and set assertions using the IDE (All are just UI tests). I know we can export those tests plans as a .SIDE file, but what I can't figure out, is how to have our TFS server execute those as part of a deployment or build pipeline.
Ideally, developers/devops would setup projects in TFS from the onset with whatever solution makes sense to execute these Selenium .SIDE files, but afterwards, the testers would manage adding/modifying those tests cases elsewhere.
The real goal here is to not have testers writing code, or checking in code. Only writing these UI Selenium tests, but having TFS execute those as part of CI.
Researching this on the internet drives me basically always to something that requires testers to write code.
I don't think it can automate testing without code, at lease, you need a test project containing your automated tests.
Generally, in Azure DevOps, we use Visual Studio Test task to run tests. This task supports using the following tests:
Test assembly: Use this option to specify one or more test assemblies that contain your tests. You can optionally specify a
filter criteria to select only specific tests.
Test plan: Use this option to run tests from your test plan that have an automated test method associated with it. To learn more about
how to associate tests with a test case work item, see Associate
automated tests with test cases.
Test run: Use this option when you are setting up an environment to run tests from test plans. This option should not be used when
running tests in a continuous integration/continuous deployment
(CI/CD) pipeline.
This was a question that I had as well, and I think I found an imperfect but better solution.
I wasn't able to get my Selenium IDE tests running with Jenkins, but I was able to get them to run with TeamCity, another CI.
I created a build step like the following :
Runner type: Command Line
Working Directory: where the selenium IDE .side file is located
Run: Custom Script
With the build script content that I usually use to run my Selenium IDE Tests, such as selenium-side-runner sidefile.side
I also added the following so I could output the results in Junitor another form: --output-directory=results --output-format=junit
You can also add the following so the tests are run headlessly, this only works in Chrome : -c "goog:chromeOptions.args=[--headless,--nogpu] browserName=chrome"
Finally, I also use --filter to run one test suite at a time, but that is optional too.
I then used another build step to export the results to our test manger, xray, but I think that is beyond the scope of this question.
The problem with this solution is that it runs directly from a users individual machine still, but this can be work around.
always before launching my app I'm running all the unit tests to make sure there is no regression. So I have two separate tasks and run all tests, and then I launch the app.
I'm curious if there is any option of setting the configuration this way that I click "Run" button, and it invokes build, tests and launch?
Thanks!
IntelliJ IDEA provides Compound Run/Debug configurations exactly for that:
Use this dialog box to create configurations containing multiple run/debug configurations that you can launch at once. This is useful, for example, if you want to launch various automated tests and get test results in one window.
I would like to run local Selenium test script written in Java, via Jenkins/Hudson. Is it possible to run scripts from my local windows machine? So far I have written some simple Selenium tests in Java, which run perfectly if I execute them via Eclipse IDE. I would be thankful for an in-depth explanation.
Selenium test in Java: assuming them to be laid out as unit tests (junit or testng), second assumption is that project is governed by either ant or maven, so there is some test (or rather integration-test) target or phase being present and be running smoothly when invoked from IDE.
When such tests are launched, they hit to a running selenium server (remote-control) which in turn launch a browser and runs its magic. Here are options: selenium server might be running in background (and be always available), or it might be started right before that testing and shut down afterwards. The latter is a common case for maven: pre-integration-test phase is configured to launch selenium rc, (then integration-test phase runs the tests against it), post-integration-test shuts selenium rc down.
So up to this moment we might want to learn what targets (ant) or phases(goals) your IDE invokes when it launches your tests successfully (also, what variables it sets or what profiles it enables).
If you invoke the same command from cmd (like 'mvn clean integration-test') and it runs successfully (same as IDE), then just instruct jenkins to run the same goals/targets (I assume that jenkins is running on the same machine, at the same user session).
If cmd doesn't do the trick (and you've looked quite well into what IDE does for you when it launches your tests), then I'd asked for more details.
So, involved participants are: 0. ant/maven that run your junit tests 1. selenium rc that should be running in bg or launched by ant/maven first 2. browser (path to browser executable) 3. jenkins (which was assumed to be running in the same environment).
If any of the assumptions are false, please come up with more details of your setup.
I'm using IntelliJ Ultimate 11.1 which includes the goodies that comes with the latest PhpStorm.
I'm trying to get some things working with the new Code Coverage feature, but having a bit of trouble getting the coverage to show up in my IDE.
The unit testing is working 100%, I can even run the unit tests in debug mode, and trigger xdebug breakpoints etc, which I think is the most awesome thing since toast.
When I set --coverage-html the HTML coverage report is being generated ok, so it's clear that xdebug and phpunit are working nicely together.
The project is a Symfony2 project, which has it's PhpUnit configuration done in a file app/phpunit.xml.dist.
When I select Analyze > Show Code Coverage Data from the menu, I get a popup looking like this:
I'm clicking the [+] but not sure what I have to select, as the OK button on that dialog box is disabled all the time.
I understand that my Test Suite is defined inside the phpunit.xml.dist file, but not sure how to get IntelliJ to use that to figure out how to display code coverage.
I'd be really thrilled to be able to use this neat feature.
Please check the documentation:
To measure code coverage
Prepare tests manually or have test classes generated.
Create a run configuration of the type PHPUnit.
On the main toolbar, select the PHPUnit run configuration in the Run/Debug Configurations drop-down list and click the Run with Coverage button.
Monitor the code coverage in the Coverage tool window.
Do you use the Run with Coverage button?
I just ran a recent update, and everything started working perfectly!
I've tried in each and every way to test a grails-app using hudson. I've tried testing with maven, I've tried testing with the grails plugin and I've tried testing with a shell builder it seems that building via shell is the only thing that works..
Every time I get the same error:
org.hibernate.HibernateException:
contains is not valid without active
transaction
But If i go to a shell and type
grails test-app
everything runs fine.
Does anyone have any idea on what's going on?
I'm using CentOS with Java 1.6, no slaves, just a simple hudson deploy over Tomcat6.
I've tried both with maven and grails builder, both fail.
Edit: it seems that if I run both unit and integration tests on the same command (either with grails or with mvn) the integration tests fail always.
Hudson/Jenkins usually just use the command line for executing grails plugins (You should be able to confirm that from the build output). You could probably add a pre build step to dump the environment, so you can see if anything there (or in your own shell) cause it to be fundamentally different.
Otherwise try to log in as the hudson user and find the hudson workspace and repeat the process manually. That has been the easiest way to debug hard problems like this..
regards