The old NUnit 2 console runner had an /exclude parameter to exclude tests with a specific Category attribute. Does anybody know if the console runner of NUnit3 supports this feature?
.\nunit3-console.exe --help doesn't state an exclude parameter. I only can see a Where parameter, can it be used for this purpose?
The description of the selection language can be found under this link. Based on that it should be possible to exclude a category via --where "cat != YourCategoryToExclude".
If you are using MSBuildTasks you can specify the same Where statement. For example:
<NUnit3 Assemblies="#(NUnitIntegrationAssemblies)"
ToolPath="$(NUnitRepositoryDir)"
OutputXmlFile="$(MSBuildStartupDirectory)\Tools\Log\NUnit.xml"
Where="cat != DatabaseContentDependent"
Framework="v4.0.30319" />
Related
This question already has an answer here:
Can you use wildcard characters with tags to get all matching tags
(1 answer)
Closed 1 year ago.
We are using tags to be able to group features and senarios. For example, we have something like:
#jira=123
Scenario: test scenario 1
...
#jira=456
Scenario: test scenario 2, known failure
...
Scenario: test scenario 3, new feature
Now, we are hoping to run test that are not tagged with #jira=123 or #jira=456. Because we have many features and scenarios tagged with the #jira=somevalue, it is impractical to add them all. So I am looking for a way to be able to exclude anything tagged with #jira. I tried ~#jira and "~#jira=" but no luck.
Looking at the following junit case:
TagTest.java#testToString()
Which is using "#foo=" as a tag, but was not able to find an example. Is there a way to exclude a group of scenarios tagged by #jira, regardless of the tag value ?
The tag value is the whole string, even if it contains a = and you may assume there is key and a value.
But you could consider to use multiple tags, they are allowed.
So, in your case, I would use something like:
#jira=123
#jira
Scenario: test scenario 1
...
#jira=456
#jira
Scenario: test scenario 2, known failure
And the you can use the ~#jira to exclude all the #jira scenarios.
This will allow you to still reference the single #jira=123 when needed.
Yes, we haven't documented this well but this question here can be a start. Karate actually supports a mini expression language for tags.
Have a look at this test for some options: TagsTest.java
And this should work for your requirement, do confirm in the comments ! Yes just use the string below where you would normally put #jira etc.
!valuesFor('#jira').isPresent
One more important point. When you use the special expression language, any AND or OR complexity has to be managed within the single expression that you pass into the tags option. Only one expression is needed and the use of comma-separated values or multiple values for the tag parameter is not applicable.
For example:
To select scenarios that have values for either the #fail tag or the #bad tag (note the use of the JS || (OR) operator):
valuesFor('#fail').isPresent || valuesFor('#bad').isPresent
And to select any scenario that has values for the #fail tag and where the #smoke tag is present (without values, just the plain tag and no = part):
valuesFor('#fail').isPresent && anyOf('#smoke')
And yes, you can use the "expression language" on the command-line i.e. within the karate.options or as the --tags or -t option to the stand-alone JAR: https://stackoverflow.com/a/72054253/143475
So I currently have an automation pack that I have created using Selenium/Specflow.
I wanted to know whether it is possible to have multiple BeforeTestRun hooks?
I've already tried: [BeforeTestRun("example1")] but I receive an error stating BeforeTestRunAttribute does not contain a constructor that takes 1 arguments
I tried the following but that also failed:
[BeforeTestRun]
[Scope(Tag = "example1")]
And referenced the above in the .feature file like this:
#example1
Scenario: This is an example
Given...
When...
Then...
Is there a way to implement this correctly such that in one .feature file I can have two scenarios that can use different [BeforeTestRun]?
If you cannot use [BeforeScenario] like suggested, you can try to manually check for tags using if statements. To get the current tags and compare them to the ones you need, try this:
var tags = ScenarioContext.ScenarioInfo.Tags;
if (tags.Any(x => x.Equals("MyTag")))
{
DoWork();
}
More info here: https://stackoverflow.com/a/42417623/9742876
I'd like to enable a test if a certain tag is "included", i.e. passed with option --include-tag of the ConsoleLauncher or useJUnitPlatform.includeTags property in Gradle. Is there any API to retrieve the value of this option in the context of test class or method?
I tried the script-based condition #EnabledIf like this:
#EnabledIf("'true' == systemProperty.get('itest.backendSystemPresent') || junitTags.contains('BackendSystemIT') == true")
But junitTags contains the #Tag annotations of the element in question, not the tags included at runtime.
Reading your question again, my answer is "No". You can't use junitTags to achieve your goals. And no, there's no such API at the moment. You would need something like:
#EnabledIf("'true' == evaluateTagExpression('BackendSystemIT') || ...)
Because you need to take care of tag expression here as well: https://junit.org/junit5/docs/current/user-guide/#running-tests-tag-expressions
But, tags are evaluated earlier in the process. Your condition will not get a chance to be executed when the test was already excluded by tag evaluation. So, I guess, you'll have to stick with the single system property switch to control the enabled state of the test method.
Btw. we are improving the tag expression language with any() and none() tokens, soon. https://github.com/junit-team/junit5/issues/1679
Possible solution:
Annotate your test with #Tag("BackendSystemIT")
Before running your tests, check for itest.backendSystemPresent system property and if it is set, pass a --include-tag "BackendSystemIT" to the test run.
Let Jupiter do the job of evaluating tag expressions
Is there any API to retrieve the value (of this option) of all tags that are attached directly or inherited in the context of test class or method?
Yes. Declare and use a org.junit.jupiter.api.TestInfo parameter in your test method.
#Test
#DisplayName("TEST 1")
#Tag("my-tag")
void test1(TestInfo testInfo) {
assertEquals("TEST 1", testInfo.getDisplayName());
assertTrue(testInfo.getTags().contains("my-tag"));
}
For details see https://junit.org/junit5/docs/current/user-guide/#writing-tests-dependency-injection
But junitTags contains the #Tag annotations of the element in question, not the tags included at runtime.
This is the expected behaviour -- the platform (here: console launcher) already applied the filter passed via --include-tag and other configuration parameters. In short: there's no need to manually check for tags in standard Jupiter tests. If there's problem with the built-in filtering, please create an issue here: https://github.com/junit-team/junit5/issues/new/choose
I have three-four Cucumber tags (say #smoke, #testing, #test) which are randomly put on scenarios and feature files. Now, I need to list all the scenarios which fall only under smoke. Can someone please suggest a way to do this
You can use the dryRun=true option in CucumberOptions with tag filters in your runner to get the list of scenarios in the report. This option will not execute your features but will list them out plus check if the steps have the appropriate glue code.
#CucumberOptions(plugin={"pretty", "html:report"}, tags={"#smoke"},
snippets=SnippetType.CAMELCASE,
glue="....", features="src/test/resources/features", dryRun=true)
Make sure you add the correct path of your glue code. Also the features should point to the top directory containing the feature files.
The above should list out the any scenario containing the #smoke tag in the html report.
But if you are looking for scenario list with only #smoke tag and not the others use this filter (tags="#smoke","~#testing","~#test").
Point of caution, if you have Scenario Outlines they will be repeated by the number of scenarios in the examples table.
You can write your own filter to print what you would run but run nothing.
require 'cucumber/core/filter'
# put inside features/support/
class PrintOnlyCukeFilter < Cucumber::Core::Filter.new
def test_case(test_case)
puts test_case.location.to_s
end
end
AfterConfiguration do |config|
config.filters << PrintOnlyCukeFilter.new
end
Put the file in features/support.
To print only file names you can see this gist.
I am using dynamic parameter to get value from user when he force build.User provides me publish directory path where i have to publish the solution if build is successful, so can any one please tell me how can i get this value in publishDir section.
Thanks in Advance.
u can refer your parameter by it's name, for example if your parameter name is pubdir then u can call it with following code.
$[pubdir] inside your publishdir section.