I try to clone a Test Suite in the Microsoft Test Manager.
It gives me an error that a Configuration is inactive and therefor the Test Suite cannnot be cloned.
How do I find out which Element still uses the no more needed and therefor inactive configuration? I went manually through all the Test Cases and Test Suites but haven't found a reference to the inactive configuration.
Related
First, important note: This is a project in which the Selenium/Cucumber test suite is working. It's working locally for the developers, and it's working in the Tekton environment.
Second: I have an identical IntelliJ environment as the developers. And other Selenium/Cucumber projects are running fine on my local machine.
When a try to run a fresh copy of the project in question locally, none of the tests start. And, there is no error message.
Normally, this will be cause by missing Glue parameters in the Run/Debug configuration, since IntelliJ for some reason often is unable to add these automatically. This is not the case here.
The test runner when I try to start a specific scenario:
Sorry for the language. The tests are all in Norwegian. But I don't think that matters for seeing the problem.
Normally, when there is an actual error regarding the test step - in the case "Gitt jeg er en vanlig søker" - there would be some info to the right of the steps overview. This is now blank, as can be seen. So, the test stes aren't even started.
The run/debug configuration:
e2e.cucumber.felles is the package where the Java file with the first step definition is located. So, that's not the reason.
Any ideas?
I check out the source code of Cassandra, and I want to run a unit test case in debug mode to understand how it works
Below is my JUnit run configuration set up. The code can compile correctly using ant. And I tried both targets build and build-test.
IntelliJ can pick up the class in the run configuration, but when I run this profile, I got.
Process finished with exit code 1
Class not found: "org.apache.cassandra.tools.SSTableExportTest"
Empty test suite.
What part do I need to change the configuration so that IntelliJ can run this unit test cases?
Since you're trying to debug Cassandra. You can reference https://wiki.apache.org/cassandra/RunningCassandraInIDEA. They have already set up a ant target to generate required configurations.
I'm new to robot framework. Could any one explain to me the differences between suite setup and test setup? When to use? Examples...
Thank you in advance.
In the simplest terms possible, a suite setup runs exactly once at the start of a suite. A test setup runs at the start of a test case.
If you define a test setup in the settings section of the suite, it will run at the start of each test in the suite (ie: if you have 5 tests, it will run five times). If you define it in a specific test, it will only run for that test where it is defined.
A suite setup is a good place to do some initialization, such as starting a server, opening a browser, making a database connection, etc.
A test setup is a good place to do some work unique to the test that isn't actually part of the verification. For example, you might use the test setup to log in and navigate to the page being tested.
Test, is a set of operations performed to verify a feature.
Suite, Set of tests defined in same test file, testing the same feature.
Testing a feature means, you might have certain preconditions which may involve setting up environment for testing the feature. So these preconditions could of 2 types, one which can be performed once for entire Suite (all tests), others have to be performed before every test.
Suite Setup, as name implies it is a function (Set of operations) which needs to be performed before running a Suite.
Test Setup, as name implies it is a function (Set of operations) which needs to be performed before every test.
E.g.
*** Settings ***
Suite Setup Setup feature environment
Test Setup my Test Setup
*** Keyworods ***
Setup feature environment
<set required environment variables>
<copy required files and folders>
<etc, etc .. >
my Test Setup
<clean up or release previous test resources>
<reset environment>
<etc, etc .. >
*** Test Cases ***
First test
< test steps>
Second test
< test steps>
Third test
< test steps>
So here, when you run the whole file (Suite), all the 3 tests will be executed.
The order in which they are executed is :
Setup feature environment --> my Test Setup --> First Test --> my Test Setup --> Second test --> my Test Setup --> Third test
Hope it helps!
The easiest way to specify a setup or a teardown for test cases in a test case file is using the :setting:Test Setup and :setting:Test Teardown settings in the Setting table.
Individual test cases can also have their own setup or teardown. They are defined with the :setting:[Setup] or :setting:[Teardown] settings in the test case table and they override possible :setting:Test Setup and :setting:Test Teardown settings. Having no keyword after a :setting:[Setup] or :setting:[Teardown] setting means having no setup or teardown. It is also possible to use value NONE to indicate that a test has no setup/teardown.
* Settings *
Test Setup Open Application App A
Test Teardown Close Application
* Test Cases *
Default values
[Documentation] Setup and teardown from setting table
Do Something
Overridden setup
[Documentation] Own setup, teardown from setting table
[Setup] Open Application App B
Do Something
No teardown
[Documentation] Default setup, no teardown at all
Do Something
[Teardown]
No teardown 2
[Documentation] Setup and teardown can be disabled also with special value NONE
Do Something
[Teardown] NONE
Using variables
[Documentation] Setup and teardown specified using variables
[Setup] ${SETUP}
Do Something
[Teardown] ${TEARDOWN}
Taking above example posted by Waman
*** Settings ***
Suite Setup Setup feature environment
Test Setup my Test Setup
*** Keyworods ***
Setup feature environment
<set required environment variables>
<copy required files and folders>
<etc, etc .. >
my Test Setup
<clean up or release previous test resources>
<reset environment>
<etc, etc .. >
*** Test Cases ***
First test
< test steps>
Second test
[setup] Testcase specific setup
< test steps>
Third test
< test steps>
So here, when you run the whole file (Suite), all the 3 tests will be executed. The order in which they are executed is :
Setup feature environment --> my Test Setup --> First Test --> Testcase specific setup --> Second test --> my Test Setup --> Third test
The [setup] in second test case overrides the [Test Setup]. So 'my Test Setup' is not executed in second test case
Allure always set BROKEN status for tests that failed before assertion.
But often tests fail because of existing bug before assertion (like login not working).
Is there a possibility to mark tests that have issue assigned and have failed - as FAILED, not BROKEN in Allure?
The best way is to fix it in Allure adaptor and replace default one with your version.
I have done the same with java adaptor, we were need to add additional execeptions as FAILED and some description to all tests.
I’m trying to integrate Bugzilla Testopia with Jenkins with the aid of the Testopia Plugin for Jenkins.
The general configuration is probably fine as the connection between Testopia and Jenkins is well established (Jenkins log says: ‘Connecting to Testopia to retrieve automated test cases’ and no error occurs then). However I’m unable to retrieve any information concerning Test Runs/Test Cases etc. from Testopia.
Moreover I cannot perform any of ‘Iterative Test Build Steps’. If I want to ‘Execute Shell’ in ‘Iterative Test Build Steps’ with Testopia Plugin no operation is carried out (even if I try: echo 12345 etc.). If I use ‘Single Test Build Steps‘ then shell command is executed.
My goal is to retrieve test class name from Testopia (it is stored in Testopia Test Case’s field - Automation/Scripts ) and then run maven build from Jenkins with this class name set as the parameter. Afterwards depending on the Jenkins build success or failure status I’d like to update the Test Case Status in Testopia.
How to fetch any information from Testopia into Jenkins?
Why any of ‘Iterative Test Build Steps’ is not executed?
Any clues? - Testopia Plugin site example wasn't too helpful for me.
Both Bugzilla and Jenkins are hosted on the same Ubuntu 14.04. I've got the latest stable versions of Jenkins, Bugzilla and Testopia.
Thanks in advance,
M.
EDIT:
Well this debugging does not work for me. I added a new log recorder with the 'ALL' level chosen and I cannot see any additional log neither in job's console output nor in the newly created logger output.
Maybe sth is wrong with my Testopia installation? Some more details concerning my configuration:
I've got Testopia installed on the same machine (as Jenkins) and usually I access it through: http://'ip_address'/bugzilla
- in Testopia plugin configuration my URL to Testopia installation is: http://'ip_address'/bugzilla/xmlrpc.cgi
- I've got only one bugzilla account - these admin's credentials I use in Jenkins
- sometimes in Jenkins I can see a warning concerning improper reverse proxy configuraiton - maybe it has sth to do with the problem
After job execution all Testopia's fields are 0 - Run Id, Build Id etc. - which obviously indicates that no information was successfully retrieved from Testopia.
Any ideas how to check why I cannot retrieve any information from Testopia?
EDIT 2:
In the meantime I think I've found a clue in the jenkins.log file in jenkins installation directory:
Exceptions like these occur:
INFO: TESTOPIA_TEST_SUITE_3 #13 main build action completed: SUCCESS
org.apache.xmlrpc.XmlRpcException: The requested method 'TestRun.get' was not found.
at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197)
...
org.apache.xmlrpc.XmlRpcException: The requested method 'TestRun.get_test_cases' was not found.
at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197)
Shall I insert the full stacktrace?
It looks like that plugin logs a fair amount of information, though not all of it to the build console output itself.
To debug further, you could try adding a new log recorder for the logger jenkins.plugins.testopia (with log level "all"), run a build, then refresh the web page for the newly-created log recorder to see the output.
You should at least see "Filtering for automated test cases" after connection, information about each test case found, and then log output for each iterative build step as it's run on each test case.
I got the similiar problem when I use python xmlrpc to commnicate with my bugzilla-testopia server.
I chekc the error code and find XMLRPC.pm under my bugzilla install location : "./WebService/Server/XMLRPC.pm".
I know the error was thrown by this sub function, which checks login status and forward to modules where the moduel.function must be in PUBLIC_METHODS:
sub handle_login {
...
if (none { $_ eq $method } $class->PUBLIC_METHODS) {
ThrowCodeError('unknown_method', { method => $full_method });
}
...
}
I don't know why but TestCase.get could not be found in PUBLIC_METHODS, so I just marked them as comment and then it works. You can do this quick way to make sure your client setting is correct. Then you should solve the "PUBLIC_METHODS" problem next.