Robot framework: Differences between suite setup and test setup? - automation

I'm new to robot framework. Could any one explain to me the differences between suite setup and test setup? When to use? Examples...
Thank you in advance.

In the simplest terms possible, a suite setup runs exactly once at the start of a suite. A test setup runs at the start of a test case.
If you define a test setup in the settings section of the suite, it will run at the start of each test in the suite (ie: if you have 5 tests, it will run five times). If you define it in a specific test, it will only run for that test where it is defined.
A suite setup is a good place to do some initialization, such as starting a server, opening a browser, making a database connection, etc.
A test setup is a good place to do some work unique to the test that isn't actually part of the verification. For example, you might use the test setup to log in and navigate to the page being tested.

Test, is a set of operations performed to verify a feature.
Suite, Set of tests defined in same test file, testing the same feature.
Testing a feature means, you might have certain preconditions which may involve setting up environment for testing the feature. So these preconditions could of 2 types, one which can be performed once for entire Suite (all tests), others have to be performed before every test.
Suite Setup, as name implies it is a function (Set of operations) which needs to be performed before running a Suite.
Test Setup, as name implies it is a function (Set of operations) which needs to be performed before every test.
E.g.
*** Settings ***
Suite Setup Setup feature environment
Test Setup my Test Setup
*** Keyworods ***
Setup feature environment
<set required environment variables>
<copy required files and folders>
<etc, etc .. >
my Test Setup
<clean up or release previous test resources>
<reset environment>
<etc, etc .. >
*** Test Cases ***
First test
< test steps>
Second test
< test steps>
Third test
< test steps>
So here, when you run the whole file (Suite), all the 3 tests will be executed.
The order in which they are executed is :
Setup feature environment --> my Test Setup --> First Test --> my Test Setup --> Second test --> my Test Setup --> Third test
Hope it helps!

The easiest way to specify a setup or a teardown for test cases in a test case file is using the :setting:Test Setup and :setting:Test Teardown settings in the Setting table.
Individual test cases can also have their own setup or teardown. They are defined with the :setting:[Setup] or :setting:[Teardown] settings in the test case table and they override possible :setting:Test Setup and :setting:Test Teardown settings. Having no keyword after a :setting:[Setup] or :setting:[Teardown] setting means having no setup or teardown. It is also possible to use value NONE to indicate that a test has no setup/teardown.
* Settings *
Test Setup Open Application App A
Test Teardown Close Application
* Test Cases *
Default values
[Documentation] Setup and teardown from setting table
Do Something
Overridden setup
[Documentation] Own setup, teardown from setting table
[Setup] Open Application App B
Do Something
No teardown
[Documentation] Default setup, no teardown at all
Do Something
[Teardown]
No teardown 2
[Documentation] Setup and teardown can be disabled also with special value NONE
Do Something
[Teardown] NONE
Using variables
[Documentation] Setup and teardown specified using variables
[Setup] ${SETUP}
Do Something
[Teardown] ${TEARDOWN}
Taking above example posted by Waman
*** Settings ***
Suite Setup Setup feature environment
Test Setup my Test Setup
*** Keyworods ***
Setup feature environment
<set required environment variables>
<copy required files and folders>
<etc, etc .. >
my Test Setup
<clean up or release previous test resources>
<reset environment>
<etc, etc .. >
*** Test Cases ***
First test
< test steps>
Second test
[setup] Testcase specific setup
< test steps>
Third test
< test steps>
So here, when you run the whole file (Suite), all the 3 tests will be executed. The order in which they are executed is :
Setup feature environment --> my Test Setup --> First Test --> Testcase specific setup --> Second test --> my Test Setup --> Third test
The [setup] in second test case overrides the [Test Setup]. So 'my Test Setup' is not executed in second test case

Related

How to find test cases/tests suites referencing a configuration

I try to clone a Test Suite in the Microsoft Test Manager.
It gives me an error that a Configuration is inactive and therefor the Test Suite cannnot be cloned.
How do I find out which Element still uses the no more needed and therefor inactive configuration? I went manually through all the Test Cases and Test Suites but haven't found a reference to the inactive configuration.

Execute TestNG.xml in dry run mode via eclipse

Is there a way to execute TestNG.xml in dry run mode so that I can figure out what methods gets qualified for the test run. I am using Eclipse and intend to run the tests via testng.xml. How to configure Run Configurations for this.
Newbie to Selenium-TestNG and Eclipse
I tried to provide -Dtestng.mode.dryrun=true in Run Configurations -> Arguments tab under both Program argument and VM arguments
The run configurations had no effect on the execution. The tests were executed in normal fashion. I expected the configurations would just list test methods in the console
You are going to see all tests with no failures. That what you expect when you run testNg with the argument.
To check the dryrun argument works, make your test to fail. Then run your test with "-Dtestng.mode.dryrun=true". Add the argument in "VM arguments"
Also, check your version of testNG is 6.14 or higher

Run specific tests within test suite - Selenium Side Runner (IDE)

Is it possible to run a specific test within my selenium side runner test suite? For example, within a test suite, my first test logs me into a website, then the other tests, test specific areas of the website. Each of these tests first inherit the login test to auth the "user" when running the tests. But when I run the suite, it runs the tests in order, so it will first run the login test, then rerun the login test within my other tests. Hope this makes sense. So essentially i want to be able to specify which tests to run within my test suite. Thanks in advance
You may use the filter to run tests that have a common name:
Filter tests
You also have the option to run a targeted subset of your tests with the --filter
target command flag (where target is a regular expression value). Test names that
contain the given search criteria will be the only ones run.
[example] selenium-side-runner --filter smoke

Empty test suite error when running unit test case in intellij

I check out the source code of Cassandra, and I want to run a unit test case in debug mode to understand how it works
Below is my JUnit run configuration set up. The code can compile correctly using ant. And I tried both targets build and build-test.
IntelliJ can pick up the class in the run configuration, but when I run this profile, I got.
Process finished with exit code 1
Class not found: "org.apache.cassandra.tools.SSTableExportTest"
Empty test suite.
What part do I need to change the configuration so that IntelliJ can run this unit test cases?
Since you're trying to debug Cassandra. You can reference https://wiki.apache.org/cassandra/RunningCassandraInIDEA. They have already set up a ant target to generate required configurations.

Running NUnit/SpecFlow tests in TFS2017

I'm not sure whether the issue is with NUnit or SpecFlow but whenever I add the 'Run Functional Tests' task to my build, I get the following error when running a build in TFS2017...
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: Test discovery started.
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: Test Run Discovery Aborted . Test run id : 5
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: UnExpected error occured during test execution. Try again.
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: Error : NUnit Adapter 3.7.0.0: Test discovery complete
2017-05-06T00:11:00.4676774Z
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: Test run aborted. Test run id: 5
2017-05-06T00:11:00.4676774Z ##[error]System.Exception: The test run was aborted, failing the task.
2017-05-06T00:11:00.5175379Z ##[error]PowerShell script completed with 1 errors.
Does anyone know what the problem is?
Many thanks,
Run Functional Tests task are used for below scenarios.
Typical scenarios include:
Tests that require additional installations on the test machines, such as different browsers for Selenium tests
Coded UI tests
Tests that require a specific operating system configuration
To execute a large number of unit tests more quickly by using multiple test machines
If you are using specflow you need to make sure that you implement using MS Test.
Regardless of the fact which unit test provider you use for SpecFlow, because all the major unit test frameworks provide the necessary adapter for it (this is the same adapter that you need to run the tests from the Visual Studio Test Explorer Window)
So you need to use Visual Studio Test task in this scenario(Also for handling standard unit tests). For details of the settings please refer this blog: SpecFlow Tips--Run only specific scenarios in TFS/VSTS build