I have created a custom step in which I'm doing some calculations. I need to pass or fail the step according to the outcome of the calculations. Presently step always shows pass in the report even when the calculation fails. Also, I would like to know how to pass the fail note to the report as I can see it is implemented in common steps.
I'm using QAF version 3.0.1
Below is a sample example:
#QAFTestStep(description = "I check quantity")
public static void iCheckQuantity() {
String product = getBundle().getString("prj.product");
Int availableStock = getBundle().getString("prj.aStock");
Int minStock = getBundle().getString("prj.minStock");
if (availableStock < minStock) {
// I want to fail the step here telling - minimum stock required to run the test for "product" is "minStock" but presenlty available is "availableStock"
}
}
I was able to figure out the answer.
import com.qmetry.qaf.automation.util.Validator.*;
assertFalse(true, "FAIL message here", "SUCCESS message here");
The below links were useful to understand more on verifications and also on different types of verifications/assertions available in QAF.
Refer:
https://qmetry.github.io/qaf/latest/assertion_verification.html
https://qmetry.github.io/qaf/latest/javadoc/com/qmetry/qaf/automation/util/Validator.html
Related
We are using unit / integration tests during Shopware 6 development.
One technique we use is to disable database transaction behaviour to see the results for example of fixtures in the admin panel, for an easier debugging / understanding:
trait IntegrationTestBehaviour
{
use KernelTestBehaviour;
// use DatabaseTransactionBehaviour;
use FilesystemBehaviour;
use CacheTestBehaviour;
use BasicTestDataBehaviour;
use SessionTestBehaviour;
use RequestStackTestBehaviour;
}
Similar to this it would be helpful to send out actual emails during some tests (only for development, not in the CI and so on).
It is already possible to automatically test emails like this:
$eventDidRun = false;
$listenerClosure = function (MailSentEvent $event) use (&$eventDidRun): void {
$eventDidRun = true;
};
$this->addEventListener($dispatcher, MailSentEvent::class, $listenerClosure);
// do something that sends an email
static::assertTrue($eventDidRun, 'The mail.sent Event did not run');
But sometimes we want to manually see the actual email.
The .env.test already contains a valid mailer URL:
MAILER_URL=smtp://x:y#smtp.mailtrap.io:2525?encryption=tls&auth_mode=login
But still no mails get send during the test.
While I guess that this is fully intentional, is there some method to workaround the blockage of getting mails sent during testing?
The reason is the MAILER_URL variable is pre-set to null://localhost in the phpunit.xml.dist of the platform repository:
<server name="MAILER_URL" value="null://localhost"/>
You could set the MAILER_URL environment variable yourself before the tests of the class are executed:
/**
* #beforeClass
*/
public static function setMailerUrl(): void
{
$_SERVER['MAILER_URL'] = 'smtp://x:y#smtp.mailtrap.io:2525?encryption=tls&auth_mode=login';
}
I have below java class which runs with cucumberOptions
#CucumberOptions(tags = {"#userManagement"})
public class IC_API_Tests_Runner {
runner code here
}
From jenkins I am passing below command ti run the tests
clean test "-Dkarate.env=$WhereToRun" "-Dbvt.tags=#userManagement"
I am able to fetch the value of 'bvt.tags' using below command
bvtTags = karate.properties['bvt.tags'];
Now I need to pass the 'bvtTags' value to the CucumberOptions.
I tried
#CucumberOptions(tags = {"bvtTags"})
public class IC_API_Tests_Runner {
runner code here
}
But 'bvtTags' value is not substituted in the CucumberOptions. But I am able to print the value of 'bvtTags' with print statement in karate code.
Any help will be great help
No you can't do dynamic changing of the #CucumberOptions like that.
Use the API for dynamically choosing tests, see this example: DemoTestSelected.java.
Then do something like this (please change for your environment):
String tags = System.getProperty("bvt.tags");
List<String> tags = Arrays.asList(tags);
EDIT: actually you don't need to do any of this. (I guess that you will never read the docs :)
Please refer: https://github.com/intuit/karate#command-line
-Dkarate.options="--tags #userManagement"
I am using QAF as my Test Automation Framework.
I want to skip specific teststep in the production environment. How can I skip execution of BDD teststep using TestStepListener?
Here is an example use case:
For shopping cart application I have developed 200+ scenarios. I was executing all scenarios on the test environment. Now I want to execute all scenarios on production environment. Now I want to skip last steps of payment and order review on production environment. How can I do that?
Will you please provide details of use case? If my understanding is correct you don't want to execute specific step in the production environment. You can use step listener to jump to specific step index but not to skip current step. One of the way is group steps to high-level step. For example instead of writing detailed steps in bdd
Given some situation
When performing some action
Then step-1
And step-2 not for production
and step-3
You can have high level step
Given some situation
When performing some action
Then generic step for all environments
Here your generic step for all environments step can have implementation for different environments in different package. configure step provider package at runtime.
Another trick is set and reset dry-run mode in step listener. For example, in your step definition you can provide additional meta-data. In the step listener depends on meta-data if require set dry-run mode in before method and reset it after in method.
Step definition:
#MetaData("{'skip_prod':true}")
#QAFTestStep(description = "do payment")
public static void doPayment() {
//TODO: write your code here
}
Step listener code may look like:
public void beforExecute(StepExecutionTracker stepExecutionTracker) {
Map<String, Object> metadata = stepExecutionTracker.getStep().getMetaData();
if (null != metadata && metadata.containsKey("skip_prod") && "prod".equalsIgnoreCase(getBundle().getString("env"))) {
//do not run this step
getBundle().setProperty(ApplicationProperties.DRY_RUN_MODE.key,true);
}
}
public void afterExecute(StepExecutionTracker stepExecutionTracker) {
Map<String, Object> metadata = stepExecutionTracker.getStep().getMetaData();
if (null != metadata && metadata.containsKey("skip_prod") && "prod".equalsIgnoreCase(getBundle().getString("env"))) {
// this is not dry run so reset
getBundle().setProperty(ApplicationProperties.DRY_RUN_MODE.key,false);
}
}
I try to write a custom step that's generate step
my code looks like :
/**
* #Then /^Check_raoul$/
*/
public function checkRaoul()
{
// grab the content ...
// get players ...
$to_return = array();
foreach ($players as $player) {
$player = $player->textContent;
if (preg_match('/^.*video=([^&]*)&.*$/', $player, $matches))
{
array_push($to_return, new Step\Then('I check the video of id "'.$matches[1].'"'));
}
}
return $to_return;
}
/**
* #Then /^I check the video of id "([^"]*)"$/
*/
public function iCheckTheVideoOfId($id)
{
// ...
}
works fine but when integrating to jenkins or un cli, if many executions of iCheckTheVideoOfId fail, I see just one error. I wish generate a number of steps equal to the number of iCheckTheVideoOfId calls
what I a doing wrong ?
We abandoned using Jenkins to do BDD checks due to the differences in how test feedback is presented and what Jenkins is capable of. We found that just running our suites locally and then a full check before pushing code to the repo produced better results and helped everyone get better at using the framework.
To answer your question directly I would suggest configuring your jenkins job to not fail when a test fails.
This can be accomplished by not outputting results at all. You can modify your command line options to not output failures at all and just log results to an output file. You could then run a script at the end to check for failures.
I am playing a bit with scala, using maven and scala plugin.
I can't find a way to have
mvn test
report failure details - in particular, whenever some function returns wrong reply, I am getting information about the failure, but I have no way to see WHICH wrong reply was reported.
For example, with test like:
object MyTestSpec extends Specification {
"my_function" should {
"return proper value for 3" {
val cnt = MyCode.my_function(3)
cnt must be_==(3)
}
}
}
in case my function returns something different than 3, I get only
Failed tests:
my_function should return proper value for 3
but there is no information what value was actually returned.
Is it possible to get this info somehow (apart from injecting manual println's)?
The Maven JUnit plugin only shows the failed tests in the console and not error messages. If you want to read the failure message you need to open the corresponding target/site/surefire-reports/MyTestSpecs.txt file.
You can also swap Maven for sbt-0.6.9 (Simple Build Tool) which can run specs specifications.
Eric.
cnt aka "my_function return value" must be_==(3)