We are trying to generate reports for our tests using serenity BDD. But we couldn't find anything to help with report generation.If anyone is familiar with this,Please suggest any simple way to achieve this.
To generate a report you need to use apply plugin: 'net.serenity-bdd.aggregator' plugin in your build.gradle file. Also while executing your project use gradlew clean test aggregate command from your command line.
After execution you will find index.html report under \target\site\serenity
You can use the Groovy MarkUpBuilder and create customized reports for your use case. Basically, you'll need to create a markup builder instance in Groovy as below :
def xmlWriter = new FileWriter(file("${project.buildDir}/index.html"))
def xmlMarkup = new MarkupBuilder(xmlWriter)
Create custom tags using below sytax :
xmlMarkup.myCustomTag("Lorem Ipsum")
which will produce :
<myCustomTag>Lorem Ipsum</myCustomTag>
So, for a syntax like xmlMarkup.h1("Lorem Ipsum") you'll get the output as <h1>Lorem Ipsum</h1>
Then you can just create a gradle task which parses all the test outputs (xml or json) into HTML.
I had written an article about that in the past which you can find here
Related
I couldn't find it online after a little bit of searching, so i'm asking it here. Is there a 'reference' bnf grammar file for yaml file of concourse's pipeline ? As a side project, I'm trying to create an IntelliJ plugin that could do syntax highlight and auto completion for CI/CD Concourse pipelines, and would try to avoid manually retyping all that grammar to minimize error risk and time.
I don't believe there's a "grammar file" - the types are defined in code. For example, the top level pipeline is defined here as:
type Config struct {
Groups GroupConfigs `yaml:"groups" json:"groups" mapstructure:"groups"`
Resources ResourceConfigs `yaml:"resources" json:"resources" mapstructure:"resources"`
ResourceTypes ResourceTypes `yaml:"resource_types" json:"resource_types" mapstructure:"resource_types"`
Jobs JobConfigs `yaml:"jobs" json:"jobs" mapstructure:"jobs"`
}
The atc.Config.Validate() command is also based on code - not an external grammar.
You could probably reason through those source files to determine the structure. It might be possible to generate a jsonschema from the go types and then use that.
I'm using Jmeter to design a test that requires data to be randomly read from text files. To save memory, I have set up a "setUp Thread Group" with a BeanShell PreProcessor with the following:
//Imports
import org.apache.commons.io.FileUtils;
//Read data files
List items = FileUtils.readLines(new File(vars.get("DataFolder") + "/items.txt"));
//Store for future use
props.put("items", items);
I then attempt to read this in my other thread groups and am trying to access a random line in my text files with something like this:
(props.get("items")).get(new Random().nextInt((props.get("items")).size()))
However, this throws a "Typed variable declaration" error and I think it's because the get() method returns an object and I'm trying to invoke size() on it, since it's really a List. I'm not sure what to do here. My ultimate goal is to define some lists of data once to be used globally in my test so my tests don't have to store this data themselves.
Does anyone have any thoughts as to what might be wrong?
EDIT
I've also tried defining the variables in the setUp thread group as follows:
bsh.shared.items = items;
And then using them as this:
(bsh.shared.items).get(new Random().nextInt((bsh.shared.items).size()))
But that fails with the error "Method size() not found in class'bsh.Primitive'".
You were very close, just add casting to List so the interpreter will know what's the expected object:
log.info(((List)props.get("items")).get(new Random().nextInt((props.get("items")).size())));
Be aware that since JMeter 3.1 it is recommended to use Groovy for any form of scripting as:
Groovy performance is much better
Groovy supports more modern Java features while with Beanshell you're stuck at Java 5 level
Groovy has a plenty of JDK enhancements, i.e. File.readLines() function
So kindly find Groovy solution below:
In the first Thread Group:
props.put('items', new File(vars.get('DataFolder') + '/items.txt').readLines()
In the second Thread Group:
def items = props.get('items')
def randomLine = items.get(new Random().nextInt(items.size))
Is there any way to generate code coverage report when using SimpleTest similar to PHPUnit.
I have read the documentation of SimpleTest on their website but can not find a clear way on how to do it!
I came across this website that says
we can add require_once (dirname(__FILE__).'/coverage.php')
to the intended file and it should generate the report, but it did not work!
If there is a helpful website on how to generate code coverage, please share it here.
Thanks alot.
I could not get it to work in the officially supported way either, but here is something I got working that I was able to hack together by examining their code. This works for v1.1.7 of SimpleTest, not their master code. At the time of this writing v1.1.7 is the latest release, and works with new versions of PHP 7, even though it is an old release.
First off you have to make sure you have Xdebug installed, configured, and working. On my system there is both a CLI and Apache version of the php.ini file that have to be configured properly depending on if I am trying to use PHP through Apache or just directly from the terminal. There are alternatives to Xdebug, but most people us Xdebug.
Then, you have to make the PHP_CodeCoverage library accessible from your code. I recommend adding it to your project as a composer package.
Now you just have to manually use that library to capture code coverage and generate a report. How exactly you do that will depend on how you run your tests. Personally, I run my tests on the terminal, and I have a bootstrap file that php runs before it starts the script. At the end of the bootstrap file, I include the SimpleTest autorun file so it will automatically run the tests in any test classes that get included like so:
require_once __DIR__.'/vendor/simpletest/simpletest/autorun.php';
Somewhere inside your bootstrap file you will need to create a filter, whitelist the directories and files you want to get reported, create a coverage object and pass in the filter to the constructor, start coverage, and create and register a shutdown function that will change the way SimpleTest executes the tests to make sure it also stops the coverage and generates the coverage report. Your bootstrap file might look something like this:
<?php
require __DIR__.'/vendor/autoload.php';
$filter = new \SebastianBergmann\CodeCoverage\Filter();
$filter->addDirectoryToWhitelist(__DIR__."/src/");
$coverage = new \SebastianBergmann\CodeCoverage\CodeCoverage(null, $filter);
$coverage->start('<name of test>');
function shutdownWithCoverage($coverage)
{
$autorun = function_exists('\run_local_tests'); // provided by simpletest
if ($autorun) {
$result = \run_local_tests(); // this actually runs the tests
}
$coverage->stop();
$writer = new \SebastianBergmann\CodeCoverage\Report\Html\Facade;
$writer->process($coverage, __DIR__.'/tmp/code-coverage-report');
if ($autorun) {
// prevent tests from running twice:
exit($result ? 0 : 1);
}
}
register_shutdown_function('\shutdownWithCoverage', $coverage);
require_once __DIR__.'/vendor/simpletest/simpletest/autorun.php';
It took me some time to figure out, as - to put it mildly - the documentation for this feature is not really complete.
Once you have your test suite up and running, just include these lines before the lines that are actually running it:
require_once ('simpletest/extensions/coverage/coverage.php');
require_once ('simpletest/extensions/coverage/coverage_reporter.php');
$coverage = new CodeCoverage();
$coverage->log = 'coverage/log.sqlite'; // This folder should exist
$coverage->includes = ['.*\.php$']; // Modify these as you wish
$coverage->excludes = ['simpletest.*']; // Or it is even better to use a setting file
$coverage->maxDirectoryDepth = '1';
$coverage->resetLog();
$coverage->startCoverage();
Then run your tests, for instance:
$test = new ProjectTests(); //It is an extension of the class TestSuite
$test->run(new HtmlReporter());
Finally generate your reports
$coverage->stopCoverage();
$coverage->writeUntouched();
$handler = new CoverageDataHandler($coverage->log);
$report = new CoverageReporter();
$report->reportDir = 'coverage/report'; // This folder should exist
$report->title = 'Code Coverage Report';
$report->coverage = $handler->read();
$report->untouched = $handler->readUntouchedFiles();
$report->summaryFile = $report->reportDir . '/index.html';
And that's it. Based on your setup, you might need to make some small adjustment to make it work. For instance, if you are using the autorun.php from simpletest, that might be a bit more tricky.
I want to write custom failure message in my Cucumber ExtentReports.
Tool using :
Cucumber
Java
Selenium
JUnit
ExtentReports
What's happening now:
I have a cucumber scenario.
Given something
When I do something
Then this step fails
The failed step Fails with:
Assert.assertTrue("CUSTOM_FAIL_MSG", some_condition);
In the ExtentReport, I see the
What I want to achieve:
What I have researched so far:
There is a scenario.write("") function but this creates a new info log into the report(But I am looking for CustomFailure message rather than a new log entry)
scenario.stepResults has the String which is displayed in the report. However, I could not find a way to set some value in the same.
Any ideas on this?
Have you tried using the create label markup?
Here is how to do it for the FAILED test:
test.log(Status.FAIL, MarkupHelper.createLabel(result.getName()+" Your MSG here!", ExtentColor.RED));
and the PASSED test:
test.log(Status.PASS, MarkupHelper.createLabel(result.getName()+" Test Case PASSED", ExtentColor.GREEN));
You can easily manipulate the string part (var interpolation?) according to your need.
Does this help?
try to replace the JUnit assertion library with the testNG library...also using cucumber you can see the failed step...why do you want to change this "narrative" log?...or if you want a better report try to use a 3rd party library
I develop Eclipse plugin which should programmatically create Ant Builder for specified project.
The same way as using UI: Properties->Builders->New->Ant Builder.
I tried to find solution several weeks but without success.
Could anybody help me?
Following might not be the cleanest way of doing what you are asking, but works. This adds a builder called "MyBuilderName" to the project's builders. You then need to create a file called "MyBuilderName.launch" under the folder .externalToolBuilders under the project folder.
Make an Ant builder and check the file in .externalToolBuilders folder to see what it should look like.
...
IProjectDescription desc = project.getDescription();
ICommand[] commands = desc.getBuildSpec();
ICommand[] newcommands = new ICommand[commands.length + 1];
System.arraycopy(commands, 0, newcommands, 1, commands.length);
ICommand command = desc.newCommand();
command.setBuilderName(
"org.eclipse.ui.externaltools.ExternalToolBuilder");
command.setArguments(
Collections.singletonMap("LaunchConfigHandle",
"<project>/.externalToolBuilders/MyBuilderName"));
newcommands[0] = command;
desc.setBuildSpec(newcommands);
project.setDescription(desc, monitor);
...