I'm trying to migrate an Ant build script to a Gradle one and i was wondering: Is there anyway to have a test task run several times?
This can be easikly done by subclassing the Test task class.
class StressTest extends Test {
// can be overwritten from within the task call
int times = 5
public FileTree getCandidateClassFiles() {
FileTree candidates = super.getCandidateClassFiles()
for (int i = 1; i < times; i++) {
candidates = candidates + super.getCandidateClassFiles()
}
return candidates
}
}
task stressTest(type: StressTest) {
// run test 10 times
times = 10
}
Inspired by Rene Groeschke, https://gist.github.com/breskeby/836316
Related
I am running cucumber tests in the TestNG framework using the maven command. Daily I am executing the test cases from Jenkins and generating the cucumber report in Jenkins. (using cucumber report plugin)
I am looking for a solution to re-run the failed test cases in Jenkins and it should give the final report.
Please provide me the approach to achieve this.
On of the simple way is, use IRetryAnalyzer in TestNG. It will re-run the failed test case.
In final report if re-run passed then i will display as passed (initially failed one display as skipped)
if re-run also failed, then marked as failure.
example:
public class Retry implements IRetryAnalyzer {
private int count = 0;
private static int maxTry = 3;
#Override
public boolean retry(ITestResult iTestResult) {
if (!iTestResult.isSuccess()) { //Check if test not succeed
if (count < maxTry) { //Check if maxtry count is reached
count++; //Increase the maxTry count by 1
iTestResult.setStatus(ITestResult.FAILURE); //Mark test as failed
return true; //Tells TestNG to re-run the test
} else {
iTestResult.setStatus(ITestResult.FAILURE); //If maxCount reached,test marked as failed
}
} else {
iTestResult.setStatus(ITestResult.SUCCESS); //If test passes, TestNG marks it as passed
}
return false;
}
}
Add in Testng.xml file
You can add test to also
#Test(retryAnalyzer = Retry.class)
This question already has an answer here:
Karate framework retry until not working as expected
(1 answer)
Closed 2 years ago.
Retry mechanism in karate testing framework How to retry tests on failure in karate testing framework like Junit and TestNG.
something like
public class Retry implements IRetryAnalyzer {
private int count = 0;
private static int maxTry = 3;
#Override
public boolean retry(ITestResult iTestResult) {
if (!iTestResult.isSuccess()) { //Check if test not succeed
if (count < maxTry) { //Check if maxtry count is reached
count++; //Increase the maxTry count by 1
iTestResult.setStatus(ITestResult.FAILURE); //Mark test as failed
return true; //Tells TestNG to re-run the test
} else {
iTestResult.setStatus(ITestResult.FAILURE); //If maxCount reached,test marked as failed
}
} else {
iTestResult.setStatus(ITestResult.SUCCESS); //If test passes, TestNG marks it as passed
}
return false;
}
}
It works for me on version 0.9.5.RC5 . But, maybe this is one of the before-mentioned "workarounds"?
All you do is something like this, which defaults to 3 attempts:
* retry until responseStatus == 404
When method get
As of now this is an open feature request: https://github.com/intuit/karate/issues/247
But there are multiple work arounds. You may get some ideas if you look at the polling example: https://github.com/intuit/karate/blob/master/karate-demo/src/test/java/demo/polling/polling.feature
Gradle allows me to start multiple jvms for testing like so:
test {
maxParallelForks = 10
}
Some of the tests for an application I have requires a fake ftp server which needs a port. This is quite easy to do with one jvm:
test {
systemProperty 'ftpPort', 10000
}
However, when running in parallel I would need to start 10 fake ftp servers. How do I add a custom system property for each jvm spawned by gradle?
Something like:
test {
maxParallelForks 10
customizeForks { index ->
systemProperty 'ftpPort', 10000 + index
}
}
There's no setup task to do before an after the fork. However, you can override the test task in your project to achieve the behavior (here in Java), as the Test task is simply a class:
public class ForkTest extends org.gradle.api.tasks.testing.Test {
private final AtomicInteger nextPort = new AtomicInteger(10000);
public Test copyTo(JavaForkOptions target) {
super.copyTo(target);
target.systemProperty("ftpPort", nextPort.getAndAIncrement());
return this;
}
}
Then in your build.gradle :
task testFork(Type: ForkTest){
forkEvery = 1
maxParallelForks = 10
...
}
I have a Windows Phone project. I want to run a background task three times, and automatically unregister after it has run three times.
My code:
var taskRegisted = false;
var exampleTaskName = "TimerRegister";
foreach (var task in BackgroundTaskRegistration.AllTasks)
{
if (task.Value.Name == exampleTaskName)
{
taskRegisted = true;
break;
}
}
if (taskRegisted == false)
{
var builder = new BackgroundTaskBuilder();
//var trigger = new SystemTrigger(SystemTriggerType.TimeZoneChange, false);
var trigger = new TimeTrigger(15, false);
builder.Name = exampleTaskName;
builder.TaskEntryPoint = typeof(ToastTask).FullName;
builder.SetTrigger(trigger);
var taskRegistion = builder.Register();
taskRegistion.Completed += taskRegistion_Completed;
}
My code runs every fifteen minutes, and I want it to run only three times.
It's unfortunately not possible to limit the number of times a background task will run in the manner you describe. The TimeTrigger does have the 'OneShot' property allowing you to have the task run only once, but it can't be configured to run X times, either 1 or infinite.
My suggestion would be to track the number of times your task has been run. Just write to a file in Isolated Storage with the number of times your task has run, and then check that value when your task starts up. After doing this if you've run 3 times return out of your Run() method without doing any real 'work'.
Nothing stands on your way to unregister your BackgroundTask inside the Run method (BackgroundTask's). For example the below code should run three times and then unregister itself:
// code in Run method:
int nrPreviousRuns = 0;
var settings = ApplicationData.Current.LocalSettings;
if (settings.Values.ContainsKey("numberOfRuns")) nrPreviousRuns = (int)settings.Values["numberOfRuns"];
if (nrPreviousRuns >= 3) // maksimum number of runs
{
// unregister the task
foreach (var cur in BackgroundTaskRegistration.AllTasks)
if (cur.Value.Name == "yourBGTaskName") cur.Value.Unregister(true);
settings.Values.Remove("numberOfRuns");
}
else settings.Values["numberOfRuns"] = ++nrPreviousRuns;
I'm currently working on a Petrel plug-in in which I need to run a simulation case (through a "For Loop"), I create my case runner, export it and the run it...but after finishing the simulation and closing the console, I check the CaseRunner.IsRunning property and it shows true! This cause that the results have not been loaded to the petrel system.
I tried to load the results manually after finishing the Run of my case (using caserunner and also using a batch file in my code) and I can't see any results in the programming environment.
Does anybody have a solution for this situation?
This is the related part of my code:
Case theCase = arguments.TheCase;
Case Test2 = simroots.CreateCase(theCase, "FinalCase");
CaseRunner cRunners = SimulationSystem.GetCaseRunner(Test2);
cRunners.Export();
cRunners.Run();
bool b = cRunners.IsRunning;
actually I checked when the process finishes; after "cRunners.Run" the code waits for exit the process using:
System.Diagnostics.Process[] parray = System.Diagnostics.Process.GetProcesses();
foreach (System.Diagnostics.Process pr in parray)
{
if (pr.ProcessName == "cmd")
{
pr.WaitForExit();//just wait
}
}
and when the console closes itself, i checked the cRunners.IsRunning term.
However, I'm not so expert... can you show me an example of using CaseRunnerMonitor? both definition of the derived class and its implementation.
All I need is running a simulation case n times via a for loop and
after each Run access to its provided summary results.
I tried some different scenarios to get my desired results, I put here some of them
First I create my CaseRunnerMonitor class:
public class MyMonitor : CaseRunnerMonitor
{
//…
public override void RunCompleted()
{
// define arguments
foreach (Slb.Ocean.Petrel.DomainObject.Simulation.SummaryResult sr in simroot.SummaryResults)
{
IEnumerable ….
List ….
// some codes to change the input arguments according to the current step simulation summary results
}
PetrelLogger.InfoOutputWindow("MyMonitor is completed!");
}
//…
}
And then use it:
private void button1_Click(object sender, EventArgs e)
{
// Some codes that define some arguments…
for (int j = 0; j < 8; j++)
{
// some changes in the arguments
Case MyTest;
MyMonitor monit4 = new MyMonitor();
SimulationRoot simroot = SimulationRoot.Get(PetrelProject.PrimaryProject);
using (ITransaction trans = DataManager.NewTransaction())
{
trans.Lock(simroot);
MyTest = simroot.CreateCase(OriginalCase, MycaseNameFunc());
trans.Commit();
}
CaseRunner cRun = SimulationSystem.GetCaseRunner(MyTest);
cRun.Export();
cRun.Run(monit4);
//Wait(); //waits for current process to close
}
}
But the thing is that MyTest case results part are empty after my run is completed. in this case all the results loaded to the petrel when the 8th (last) simulation completes. If I don’t activate the Wait() function, all 8 runs are almost calling simultaneously…
I changed my scenario, my callback after each run is read the simulation results, change something and call next run so
I create my CaseRunnerMonitor class:
public class MyMonitor2 : CaseRunnerMonitor
{
//…
public override void RunCompleted()
{
// define arguments
index++;
if (index <=8)
{
foreach (Slb.Ocean.Petrel.DomainObject.Simulation.SummaryResult sr in simroot.SummaryResults)
{
IEnumerable ….
List ….
// some codes to change the input arguments according to the current step simulation summary results
}
Case MyTest;
MyMonitor monit4 = new MyMonitor();
SimulationRoot simroot = SimulationRoot.Get(PetrelProject.PrimaryProject);
using (ITransaction trans = DataManager.NewTransaction())
{
trans.Lock(simroot);
MyTest = simroot.CreateCase(OriginalCase, MycaseNameFunc());
trans.Commit();
}
CaseRunner cRun = SimulationSystem.GetCaseRunner(MyTest);
cRun.Export();
cRun.Run(monit4);
}
PetrelLogger.InfoOutputWindow("MyMonitor2 is completed!");
}
//…
}
And then use it:
private void button1_Click(object sender, EventArgs e)
{
Index=0;
// Some codes that define some arguments…
// some changes in the arguments
Case MyTest;
MyMonitor monit5 = new MyMonitor();
SimulationRoot simroot = SimulationRoot.Get(PetrelProject.PrimaryProject);
using (ITransaction trans = DataManager.NewTransaction())
{
trans.Lock(simroot);
MyTest = simroot.CreateCase(OriginalCase, MycaseNameFunc());
trans.Commit();
}
CaseRunner cRun = SimulationSystem.GetCaseRunner(MyTest);
cRun.Export();
cRun.Run(monit5);
}
in this situation no need to wait() function is required. But the problem is that I access to MyTest case results in one level before the current run completes. i.e, I can view the step 5 results via MyTest.Results when the run 6 is completed while step 6 results are empty despite of completion of its run.
I check the CaseRunner.IsRunning property and it shows true
This is because Caserunner.Run() is non-blocking; that is, it starts another thread to launch the run. Control flow then passes immediately to your cRunners.IsRunning check which is true as simulation is in progress.
cRunners.Run(); //non-blocking
bool b = cRunners.IsRunning;
You should look at CaseRunnerMonitor if you want a call-back when the simulation is complete.
Edit:
can you show me an example of using CaseRunnerMonitor? both definition of the derived class and its implementation.
Create your monitor class:
public class CustomCaseRunnerMonitor : CaseRunnerMonitor
{
//...
public override void RunCompleted()
{
//This is probably the callback you want
}
}
Use it:
Case myCase = WellKnownSimulators.ECLIPSE100.CreateSimulationCase(...);
CaseRunner runner = SimulationSystem.GetCaseRunner(myCase);
var myMonitor = new CustomCaseRunnerMonitor(...);
runner.Run(myMonitor);
//Your callbacks defined in your CustomCaseRunnerMonitor will now be called
See also "Running and monitoring a Simulation" in SimulationSystem API documentation.
Ah, OK. I didn't realise you were trying to load results with the CaseMonitor.
I'm afraid the short answer is "No, you can't know when Petrel has loaded results".
The long answer is Petrel will automatically load results if the option is set in the Case arguments. (Define Simulation Case -> Advance -> Automatically load results).
In API:
EclipseFormatSimulator.Arguments args = EclipseFormatSimulator.GetEclipseFormatSimulatorArguments(myCase);
EclipseFormatSimulator.Arguments.RuntimeArguments runtimeArgs = args.Runtime;
runtimeArgs.AutoLoadResults = true;
runtimeArgs.AutoLoadResultsInterval = 120; //How frequently in seconds Petrel polls sim dir.
You will have to poll SimulationRoot.SummaryResults (using the same API you are already using) after case has finished.
You should use the CaseRunnerMonitor we discussed to determine when to start doing this, rather than the System.Diagnostics.Process[] parray = System.Diagnostics.Process.GetProcesses(); code you currently have.