This question already has an answer here:
Can you use wildcard characters with tags to get all matching tags
(1 answer)
Closed 1 year ago.
We are using tags to be able to group features and senarios. For example, we have something like:
#jira=123
Scenario: test scenario 1
...
#jira=456
Scenario: test scenario 2, known failure
...
Scenario: test scenario 3, new feature
Now, we are hoping to run test that are not tagged with #jira=123 or #jira=456. Because we have many features and scenarios tagged with the #jira=somevalue, it is impractical to add them all. So I am looking for a way to be able to exclude anything tagged with #jira. I tried ~#jira and "~#jira=" but no luck.
Looking at the following junit case:
TagTest.java#testToString()
Which is using "#foo=" as a tag, but was not able to find an example. Is there a way to exclude a group of scenarios tagged by #jira, regardless of the tag value ?
The tag value is the whole string, even if it contains a = and you may assume there is key and a value.
But you could consider to use multiple tags, they are allowed.
So, in your case, I would use something like:
#jira=123
#jira
Scenario: test scenario 1
...
#jira=456
#jira
Scenario: test scenario 2, known failure
And the you can use the ~#jira to exclude all the #jira scenarios.
This will allow you to still reference the single #jira=123 when needed.
Yes, we haven't documented this well but this question here can be a start. Karate actually supports a mini expression language for tags.
Have a look at this test for some options: TagsTest.java
And this should work for your requirement, do confirm in the comments ! Yes just use the string below where you would normally put #jira etc.
!valuesFor('#jira').isPresent
One more important point. When you use the special expression language, any AND or OR complexity has to be managed within the single expression that you pass into the tags option. Only one expression is needed and the use of comma-separated values or multiple values for the tag parameter is not applicable.
For example:
To select scenarios that have values for either the #fail tag or the #bad tag (note the use of the JS || (OR) operator):
valuesFor('#fail').isPresent || valuesFor('#bad').isPresent
And to select any scenario that has values for the #fail tag and where the #smoke tag is present (without values, just the plain tag and no = part):
valuesFor('#fail').isPresent && anyOf('#smoke')
And yes, you can use the "expression language" on the command-line i.e. within the karate.options or as the --tags or -t option to the stand-alone JAR: https://stackoverflow.com/a/72054253/143475
Related
I'm wondering if you can use wildcard characters with tags to get all tagged scenarios/features that match a certain pattern.
For example, I've used 17 unique tags on many scenarios throughout many of my feature files. The pattern is "#jira=CIS-" followed by 4 numbers, like #jira=CIS-1234 and #jira=CIS-5678.
I'm hoping I can use a wildcard character or something that will find all of the matches for me.
I want to be able to exclude them from being run, when I run all of my features/scenarios.
I've tried the follow:
--tags ~#jira
--tags ~#jira*
--tags ~#jira=*
--tags ~#jira=
Unfortunately none have given my the results I wanted. I was only able to exclude them when I used the exact tag, ex. ~#jira=CIS-1234. It's not a good solution to have to add each single one (of the 17 different tags) to the command line. These tags can change frequently, with new ones being added and old ones being removed, plus it would make for one real long command.
Yes. First read this - there is this un-documented expression-language (based on JS) for advanced tag selction based on the #key=val1,val2 form: https://stackoverflow.com/a/67219165/143475
So you should be able to do this:
valuesFor('#jira').isPresent
And even (here s will be a string, on which you can even do JS regex if you know how):
valuesFor('#jira').isEach(s => s.startsWith('CIS-'))
Would be great to get your confirmation and then this thread itself can help others and we can add it to the docs at some point.
I'm wondering if you can use wildcard characters with tags to get all tagged scenarios/features that match a certain pattern.
For example, I've used 17 unique tags on many scenarios throughout many of my feature files. The pattern is "#jira=CIS-" followed by 4 numbers, like #jira=CIS-1234 and #jira=CIS-5678.
I'm hoping I can use a wildcard character or something that will find all of the matches for me.
I want to be able to exclude them from being run, when I run all of my features/scenarios.
I've tried the follow:
--tags ~#jira
--tags ~#jira*
--tags ~#jira=*
--tags ~#jira=
Unfortunately none have given my the results I wanted. I was only able to exclude them when I used the exact tag, ex. ~#jira=CIS-1234. It's not a good solution to have to add each single one (of the 17 different tags) to the command line. These tags can change frequently, with new ones being added and old ones being removed, plus it would make for one real long command.
Yes. First read this - there is this un-documented expression-language (based on JS) for advanced tag selction based on the #key=val1,val2 form: https://stackoverflow.com/a/67219165/143475
So you should be able to do this:
valuesFor('#jira').isPresent
And even (here s will be a string, on which you can even do JS regex if you know how):
valuesFor('#jira').isEach(s => s.startsWith('CIS-'))
Would be great to get your confirmation and then this thread itself can help others and we can add it to the docs at some point.
I would like to know if it is possible to assign "ID Keys" to each feature or scenarios in Karate DSL in order to keep all of them uniquelly referenced.
When you have hundreds of features/scenarios seems very difficult to keep all this list tidy . The name of the scenarios does not seem to be a valid unique reference as I am including variable data in its name.
It is probably best that you use a combination of the feature file name and the "scenario ref id" which looks like this: [1.4] or [2.1:10].
In the upcoming 1.0 / RC version, have a look at the values of karate.scenario and karate.feature when you are within a test execution: https://github.com/intuit/karate/wiki/1.0-upgrade-guide#karateinfo-deprecated
The other thing you can consider is add tags to every scenario: https://github.com/intuit/karate#tags
#id=001
Scenario: some name
* print karate.scenario.name
That's all we have right now, it may need you / the community to contribute if you have some ideas for the future.
I'd like to enable a test if a certain tag is "included", i.e. passed with option --include-tag of the ConsoleLauncher or useJUnitPlatform.includeTags property in Gradle. Is there any API to retrieve the value of this option in the context of test class or method?
I tried the script-based condition #EnabledIf like this:
#EnabledIf("'true' == systemProperty.get('itest.backendSystemPresent') || junitTags.contains('BackendSystemIT') == true")
But junitTags contains the #Tag annotations of the element in question, not the tags included at runtime.
Reading your question again, my answer is "No". You can't use junitTags to achieve your goals. And no, there's no such API at the moment. You would need something like:
#EnabledIf("'true' == evaluateTagExpression('BackendSystemIT') || ...)
Because you need to take care of tag expression here as well: https://junit.org/junit5/docs/current/user-guide/#running-tests-tag-expressions
But, tags are evaluated earlier in the process. Your condition will not get a chance to be executed when the test was already excluded by tag evaluation. So, I guess, you'll have to stick with the single system property switch to control the enabled state of the test method.
Btw. we are improving the tag expression language with any() and none() tokens, soon. https://github.com/junit-team/junit5/issues/1679
Possible solution:
Annotate your test with #Tag("BackendSystemIT")
Before running your tests, check for itest.backendSystemPresent system property and if it is set, pass a --include-tag "BackendSystemIT" to the test run.
Let Jupiter do the job of evaluating tag expressions
Is there any API to retrieve the value (of this option) of all tags that are attached directly or inherited in the context of test class or method?
Yes. Declare and use a org.junit.jupiter.api.TestInfo parameter in your test method.
#Test
#DisplayName("TEST 1")
#Tag("my-tag")
void test1(TestInfo testInfo) {
assertEquals("TEST 1", testInfo.getDisplayName());
assertTrue(testInfo.getTags().contains("my-tag"));
}
For details see https://junit.org/junit5/docs/current/user-guide/#writing-tests-dependency-injection
But junitTags contains the #Tag annotations of the element in question, not the tags included at runtime.
This is the expected behaviour -- the platform (here: console launcher) already applied the filter passed via --include-tag and other configuration parameters. In short: there's no need to manually check for tags in standard Jupiter tests. If there's problem with the built-in filtering, please create an issue here: https://github.com/junit-team/junit5/issues/new/choose
This question already has answers here:
Can we parameterize the request file name to the Read method in Karate?
(2 answers)
Closed 1 year ago.
I am using a JSON file which act as a test case document for my API testing. The JSON contain Test Case ID, Test case Description, Header and Request body details, which should be the driving factor of Automation
Currently i am looping a feature over this json file to set different header and body validations. However it will be helpful if i can set the Scenario name from JSON file while its iterating
Something like
serverpost.feature
Feature:re-usable feature to publish data
Scenario: TC_NAME # TC_NAME is avaliable in the JSON data passed to this feature. However, CURRENTLY ITS NOT TAKING THIS DATA FROM JSON FILE.
Given path TC_ID # TC ID is taken from JSON
Given url 'http://myappurl.com:8080/mytestapp/Servers/Data/uploadServer/'
And request { some: '#(BODY)' } # Request Body Details is taken from JSON
Please suggest
In my honest opinion, you are asking for a very un-necessary feature. Please refer to the demo examples, look for it in the documentation.
Specifically, look at this one: dynamic-params.feature. There are multiple ways to create / use a data table. Instead of trying to maintain 2 files - think of Karate as being both - your data table AND the test execution. There is no need to complicate things further.
If you really really want to re-use some JSON lying around, it is up to you but you won't be able to update the scenario name, sorry. What I suggest is just use the print statement to dump the name to the log and it will appear in the HTML report (refer to the doc). Note that when calling a feature in a loop using a JSON array, the call argument is ALREADY included the report, so you may not need to do anything.
Just an observation - your questions seem to be very basic, do you mind reading the doc and the examples a bit more thoroughly, thanks.