I'd like to enable a test if a certain tag is "included", i.e. passed with option --include-tag of the ConsoleLauncher or useJUnitPlatform.includeTags property in Gradle. Is there any API to retrieve the value of this option in the context of test class or method?
I tried the script-based condition #EnabledIf like this:
#EnabledIf("'true' == systemProperty.get('itest.backendSystemPresent') || junitTags.contains('BackendSystemIT') == true")
But junitTags contains the #Tag annotations of the element in question, not the tags included at runtime.
Reading your question again, my answer is "No". You can't use junitTags to achieve your goals. And no, there's no such API at the moment. You would need something like:
#EnabledIf("'true' == evaluateTagExpression('BackendSystemIT') || ...)
Because you need to take care of tag expression here as well: https://junit.org/junit5/docs/current/user-guide/#running-tests-tag-expressions
But, tags are evaluated earlier in the process. Your condition will not get a chance to be executed when the test was already excluded by tag evaluation. So, I guess, you'll have to stick with the single system property switch to control the enabled state of the test method.
Btw. we are improving the tag expression language with any() and none() tokens, soon. https://github.com/junit-team/junit5/issues/1679
Possible solution:
Annotate your test with #Tag("BackendSystemIT")
Before running your tests, check for itest.backendSystemPresent system property and if it is set, pass a --include-tag "BackendSystemIT" to the test run.
Let Jupiter do the job of evaluating tag expressions
Is there any API to retrieve the value (of this option) of all tags that are attached directly or inherited in the context of test class or method?
Yes. Declare and use a org.junit.jupiter.api.TestInfo parameter in your test method.
#Test
#DisplayName("TEST 1")
#Tag("my-tag")
void test1(TestInfo testInfo) {
assertEquals("TEST 1", testInfo.getDisplayName());
assertTrue(testInfo.getTags().contains("my-tag"));
}
For details see https://junit.org/junit5/docs/current/user-guide/#writing-tests-dependency-injection
But junitTags contains the #Tag annotations of the element in question, not the tags included at runtime.
This is the expected behaviour -- the platform (here: console launcher) already applied the filter passed via --include-tag and other configuration parameters. In short: there's no need to manually check for tags in standard Jupiter tests. If there's problem with the built-in filtering, please create an issue here: https://github.com/junit-team/junit5/issues/new/choose
Related
This question already has an answer here:
Can you use wildcard characters with tags to get all matching tags
(1 answer)
Closed 1 year ago.
We are using tags to be able to group features and senarios. For example, we have something like:
#jira=123
Scenario: test scenario 1
...
#jira=456
Scenario: test scenario 2, known failure
...
Scenario: test scenario 3, new feature
Now, we are hoping to run test that are not tagged with #jira=123 or #jira=456. Because we have many features and scenarios tagged with the #jira=somevalue, it is impractical to add them all. So I am looking for a way to be able to exclude anything tagged with #jira. I tried ~#jira and "~#jira=" but no luck.
Looking at the following junit case:
TagTest.java#testToString()
Which is using "#foo=" as a tag, but was not able to find an example. Is there a way to exclude a group of scenarios tagged by #jira, regardless of the tag value ?
The tag value is the whole string, even if it contains a = and you may assume there is key and a value.
But you could consider to use multiple tags, they are allowed.
So, in your case, I would use something like:
#jira=123
#jira
Scenario: test scenario 1
...
#jira=456
#jira
Scenario: test scenario 2, known failure
And the you can use the ~#jira to exclude all the #jira scenarios.
This will allow you to still reference the single #jira=123 when needed.
Yes, we haven't documented this well but this question here can be a start. Karate actually supports a mini expression language for tags.
Have a look at this test for some options: TagsTest.java
And this should work for your requirement, do confirm in the comments ! Yes just use the string below where you would normally put #jira etc.
!valuesFor('#jira').isPresent
One more important point. When you use the special expression language, any AND or OR complexity has to be managed within the single expression that you pass into the tags option. Only one expression is needed and the use of comma-separated values or multiple values for the tag parameter is not applicable.
For example:
To select scenarios that have values for either the #fail tag or the #bad tag (note the use of the JS || (OR) operator):
valuesFor('#fail').isPresent || valuesFor('#bad').isPresent
And to select any scenario that has values for the #fail tag and where the #smoke tag is present (without values, just the plain tag and no = part):
valuesFor('#fail').isPresent && anyOf('#smoke')
And yes, you can use the "expression language" on the command-line i.e. within the karate.options or as the --tags or -t option to the stand-alone JAR: https://stackoverflow.com/a/72054253/143475
#Peter - As per your suggestion from my previous queries, I have used ExecutionHooks to implement ReportPortal. I am finding difficulties in passing all the required values from my Runner to Base Runner. Below is my configuration-
BaseRunner.java
Results results = Runner.parallel(tags,path,ScenarioName,Collections.singletonList(new
ScenarioReporter()),threads,karateOutputPath);
Runner.java
#KarateOptions(tags = { "#Shakedown" },
features = "classpath:tests/Shakedown"
)
I want to understand how can I pass the attributes like Scenario Name, path and tags. ScenarioReporter() is my class where I have implemented Execution Hook. I have a base runner that will have all the details and a normal runner that will have minimal information. I have just given snippets, please don't mind if there are some syntactical errors.
You don't need the annotations any more, and you can set all parameters including tags using the new "builder" (fluent interface) on the Runner. Refer the docs: https://github.com/intuit/karate#parallel-execution
Results results = Runner.path("classpath:some/package").tags("~#ignore").parallel(5);
So it should be easier to inherit from base classes etc. just figure out a way to pass a List<String> of tags and use it.
Just watch out for this bug, fixed in 0.9.6.RC1: https://github.com/intuit/karate/issues/1061
So I currently have an automation pack that I have created using Selenium/Specflow.
I wanted to know whether it is possible to have multiple BeforeTestRun hooks?
I've already tried: [BeforeTestRun("example1")] but I receive an error stating BeforeTestRunAttribute does not contain a constructor that takes 1 arguments
I tried the following but that also failed:
[BeforeTestRun]
[Scope(Tag = "example1")]
And referenced the above in the .feature file like this:
#example1
Scenario: This is an example
Given...
When...
Then...
Is there a way to implement this correctly such that in one .feature file I can have two scenarios that can use different [BeforeTestRun]?
If you cannot use [BeforeScenario] like suggested, you can try to manually check for tags using if statements. To get the current tags and compare them to the ones you need, try this:
var tags = ScenarioContext.ScenarioInfo.Tags;
if (tags.Any(x => x.Equals("MyTag")))
{
DoWork();
}
More info here: https://stackoverflow.com/a/42417623/9742876
In my project Mockito.times(1) is often used when verifying mocks:
verify(mock, times(1)).call();
This is redundant since Mockito uses implicit times(1) for verify(Object), thus the following code does exactly what the code above does:
verify(mock).call();
So I'm going to write an a structural search drive inspection to report such cases (let's say, named something like Mockito.times(1) is redundant). As I'm not an expert in IntelliJ IDEA structural search, my first attempt was:
Mockito.times(1)
Obviously, this is not a good seach template because it ignores the call-site. Let's say, I find it useful for the following code and I would not like the inspection to trigger:
VerificationMode times = Mockito.times(1);
// ^ unwanted "Mockito.times(1) is redundant"
So now I would like to define the context where I would like the inspection to trigger. Now the inspection search template becomes:
Mockito.verify($mock$, Mockito.times(1))
Great! Now code like verify(mock, times(1)).call() is reported fine (if times was statically imported from org.mockito.Mockito). But there is also one thing. Mockito.times actually comes from its VerificationModeFactory class where such verification modes are grouped, so the following line is ignored by the inspection:
verify(mockSupplier, VerificationModeFactory.times(1)).get();
My another attempt to fix this one was something like:
Mockito.verify($mock$, $times$(1))
where:
$mock$ is still a default template variable;
$times$ is a variable with Text/regexp set to times, Whole words only and Value is read are set to true, and Expression type (regexp) is set to (Times|VerificationMode) -- at least this is the way I believed it should work.
Can't make it work. Why is Times also included to the regexp? This is the real implementation of *.times(int), so, ideally, the following line should be reported too:
verify(mockSupplier, new Times(1)).get();
Of course, I could create all three inspection templates, but is it possible to create such a template using single search template and what am I missing when configuring the $times$ variable?
(I'm using IntelliJ IDEA Community Edition 2016.1.1)
Try the following search query:
Mockito.verify($mock$, $Qualifier$.times(1))
With $Qualifier$ text/regexp VerificationModeFactory|Mockito and occurrences count 0,1 (to find it when statically imported also).
To also match new Times(1) you can use the following query:
Mockito.verify($mock$, $times$)
With $times$ text/regexp .*times\s*\(\s*1\s*\) and uncheck the Case sensitive checkbox.
I have a SoapUI project which contains around 60 plus services. Each service requires some input which will be changed for every execution. So I have created certain Global Properties and assign some values to that properties.
I have to use these properties values in my SoapUI request ( i.e. JSON Format request ).
If it is groovy script means, I will use like this.
String HTiC_Username = com.eviware.soapui.model.propertyexpansion.PropertyExpansionUtils.globalProperties['HTiC_Username'].value;
But, how to get the value of the Global Property in the request?
Hope you understand my question. Please provide proper guidance.
Thanks
To dynamically "expand" (i.e. substitute) the value of a property into a test step, the following syntax is used: ${#scope#propertyName}
...where 'scope' refers to the level at which the property has been defined (e.g. Global, Project, TestSuite, TestCase).
So to expand a property named username defined as a Global property, for example, the following code can be used directly within a Request Test Step (e.g within a JSON body, or header value, etc):
${#Global#username}
To access the same property value within a Groovy Test Step, you can use the following syntax:
context.expand('${#scope#propertyName}')
...as in the following example:
context.expand('${#Global#username}')
What we did was the following:
created a test data file to store all the specific input data for the different services (testdata.properties)
Example content of testdata.properties:
Billing_customerID=1234567
OtherService_paymentid=12121212
....
create a SoupUi global parameter (File/Preferences/Global properties): testdata_filepath=C:\...
For specific services we added a Properties test step. You can specify the "Load from" field to our new global parameter: ${#Global#testdata_filepath} Now you can use the Load button to load parameters.
Finally you can reference the parameter in your xml in the following format: ${Properties#Billing_customerID}
Example content of a service with parameter:
...
<BillingCustomerIdentification>
<BillingCustomerID>${#Properties#Billing_customerID}</BillingCustomerID>
</BillingCustomerIdentification>
...
To set up your projects in this manner also helps to automate service tests eg. using Hudson (see my previous SO answer).
If it is too heavy and automation is not a target, you can simply use ${#Global#someinputvariable} format in your xml ;-)