Here is my test code:
test('should set correct constant', (){
expect(Stores.CurrentContext, 'currentContext');
});
but the picture above shows that the static constant code not tested. and why?
version infos:
Flutter 1.2.2-pre.3 • channel master • https://github.com/flutter/flutter.git
Framework • revision 67cf21577f (4 days ago) • 2019-02-14 23:17:16 -0800
Engine • revision 3757390fa4
Tools • Dart 2.1.2 (build 2.1.2-dev.0.0 0a7dcf17eb)
A coverage tool registers which code instructions was accessed by the running code.
Think of it as a recording of the memory addresses of "code sections" visited by the Program Counter register
of the processor stepping through program functions.
A static variable is reached through a data memory access, there are no code instructions involved:
a variable should be on the stack, on the heap or in a data section if it is a constant.
Consider this code:
import 'package:rxdart/rxdart.dart';
class Stores {
static const String Login = 'login';
static const String CurrentContext = 'currentContext';
}
class Store {
final name;
static var eMap = Map();
Store._internal(this.name); // DA:13
factory Store(String name) { // DA:15
if (eMap.containsKey(name)) { // DA:16
return eMap[name]; // DA:17
} else {
final store = Store._internal(name); // DA:19
eMap[name] = store; // DA:20
return store;
}
}
}
and this code run:
test('should set correct constant', (){
Store('currentContext');
Store('currentContext');
expect(Stores.CurrentContext, 'currentContext');
});
If you look at the raw output of icov you will notice that lines number of static variable is never reached, giving meaning to the model described above:
SF:lib/stores.dart
DA:13,1
DA:15,1
DA:16,2
DA:17,2
DA:19,1
DA:20,2
LF:6
LH:6
The visual reporting tool shows a 100% coverage:
If your reporting tool shows red lines over static variables it has to be considered a "false positive": survive with it or change the reporting tool.
Related
my scenario: I have an existing unit test framework with ~3000 individual test cases. They are made from TEST, TEST_F and TEST_P macros.
Internally the tested modules make use of a logger library and now my goal is to create individual log files for each test case. To do so I would like to call a function as a SetUp for each test case.
Is there a way to register such function at the framework and get it called automatically?
The obvious solution for me would look like: do the work in a test fixture constructor or SetUp() but then I'd have to touch every single test case.
I do like the idea of registering a global setup at the framework with AddGlobalTestEnvironment() but as I understand this is handled only once per executable.
By the way: I have acceptance tests implemented in robot test and guess what? I want to repeat the task there...
Thanks for any inspiration!
Christoph
You mentioned:
The obvious solution for me would look like: do the work in a test fixture constructor or SetUp() but then I'd have to touch every single test case.
If the reason that you think you would need to touch every single test case is to set the filename differently, you can use the combination of SetUp() function and the current_test_info provided by GTest to get the test name for each test, and then use that to create a separate file for each test.
Here is an example:
// Class for test fixture
class MyTestFixture : public ::testing::Test {
protected:
void SetUp() override {
test_name_ = std::string(
::testing::UnitTest::GetInstance()->current_test_info()->name());
std::cout << "test_name_: " << test_name_ << std::endl;
// CreateYourLogFileUsingTestName(test_name_);
}
std::string test_name_;
};
TEST_F(MyTestFixture, Test1) {
EXPECT_EQ(this->test_name_, std::string("Test1"));
}
TEST_F(MyTestFixture, Test2) {
EXPECT_EQ(this->test_name_, std::string("Test2"));
}
Live example here: https://godbolt.org/z/YjzEG3G77
The solution I found in the gtest docs:
class TraceHandler : public testing::EmptyTestEventListener
{
// Called before a test starts.
void OnTestStart( const testing::TestInfo& test_info ) override
{
// set the logfilename here
}
// Called after a test ends.
void OnTestEnd( const testing::TestInfo& test_info ) override
{
// close the log here
}
};
int main( int argc, char** argv )
{
testing::InitGoogleTest( &argc, argv );
testing::TestEventListeners& listeners =
testing::UnitTest::GetInstance()->listeners();
// Adds a listener to the end. googletest takes the ownership.
listeners.Append(new TraceHandler);
return RUN_ALL_TESTS();
}
This way it automatically applies to all tests linked to this main-function.
Maybe I have to mention: my logger is a collection of static functions that send udp-packets to a receiver that cares for actual logging. I can control the filename by one of that functions. That's the reason why I don't need to insert code in every single TEST, TEST_F or TEST_P.
I run API tests using Groovy and Spock.
Request/response data, produced by third-party libraries, appear in the system out (I see it in the Jenkins log).
Question:
what is the proper way to start-stop system out recording for each test iteration to some list of strings?
TestNG has Reporter.getOutput(result) which returns all log entries, appeared while test iteration run.
Is it something similar in Spock?
Am I right assuming it should be some implementation of Run listener where I start recording in beforeIteration() and attach it to report in afterIteration()?
Solved by using the OutputCapture from
spring-boot-starter-test
The sample of RunListener:
class RunListener extends AbstractRunListener {
OutputCapture outputCapture
void beforeSpec(SpecInfo spec) {
Helper.log "[BEFORE SPEC]: ${spec.name}"
outputCapture = new OutputCapture()
outputCapture.captureOutput() //register a copy off system.out, system.err streams
}
void beforeFeature(FeatureInfo feature) {
Helper.log "[BEFORE FEATURE]: ${feature.name}", 2
}
void beforeIteration(IterationInfo iteration) {
outputCapture.reset() //clear the stream copy before each test iteration
}
void afterIteration(IterationInfo iteration) {
}
void error(ErrorInfo error) {
//attach the content of copy stream object to the report if test iteration failed
Allure.addAttachment("${error.method.iteration.name}_console_out", "text/html", outputCapture.toString(), "txt")
}
void afterFeature(FeatureInfo feature) {
}
void afterSpec(SpecInfo spec) {
outputCapture.releaseOutput()
}
}
Shortly:
CaptureOutput is an implementation of Output Stream which logs in both initial out and copy stream object. The copy gets cleared in beforeIteration() and is attached to the report in afterIteration() in RunListener, so each test receives its own part of output.
I wish to access many classes and variables, I would like to do this by dynamically setting the class name and variable name. Currently I am using
MyClass["myVariable1"]
to dynamically access the variable name
MyClass.myVariable1
I want to also dynanmically acces the class name, something like
["MyClass"]["myVariable1"]
But this does not work.
The purpose is that I have shared object with many user settings, I want to iterate through the shared object and set all the user settings across all the classes. I think if I cant dynamically access the class I must have a statement for each and every class name/variable.
I advise against such a practice. Although technically possible, it is like welcoming a disaster into the app architecture:
You rely on something you have no apparent control of: on the way Flash names the classes.
You walk out of future possibility to protect your code with identifier renaming obfuscation because it will render your code invalid.
Compile time error checks is better than runtime, and you are leaving it to runtime. If it happens to fail in non-debug environment, you will never know.
The next developer to work with your code (might be you in a couple of years) will have hard time finding where the initial data coming from.
So, having all of above, I encourage you to switch to another model:
package
{
import flash.net.SharedObject;
public class SharedData
{
static private var SO:SharedObject;
static public function init():void
{
SO = SharedObject.getLocal("my_precious_shared_data", "/");
}
static public function read(key:String):*
{
// if (!SO) init();
return SO.data[key];
}
static public function write(key:String, value:*):void
{
// if (!SO) init();
SO.data[key] = value;
SO.flush();
}
// Returns stored data if any, or default value otherwise.
// A good practice of default application values that might
// change upon user activity, e.g. sound volume or level progress.
static public function readSafe(key:String, defaultValue:*):*
{
// if (!SO) init();
return SO.data.hasOwnProperty(key)? read(key): defaultValue;
}
}
}
In the main class you call
SharedData.init();
// So now your shared data are available.
// If you are not sure you can call it before other classes will read
// the shared data, just uncomment // if (!SO) init(); lines in SharedData methods.
Then each class that feeds on these data should have an initialization block:
// It's a good idea to keep keys as constants
// so you won't occasionally mistype them.
// Compile time > runtime again.
static private const SOMAXMANA:String = "maxmana";
static private const SOMAXHP:String = "maxhp";
private var firstTime:Boolean = true;
private var maxmana:int;
private var maxhp:int;
// ...
if (firstTime)
{
// Make sure it does not read them second time.
firstTime = false;
maxhp = SharedData.readSafe(SOMAXHP, 100);
maxmana = SharedData.readSafe(SOMAXMANA, 50);
}
Well, again. The code above:
does not employ weird practices and easy to understand
in each class anyone can clearly see where the data come from
will be checked for errors at compile time
can be obfuscated and protected
You can try getting the class into a variable and going from there:
var myClass:Class = getDefinitionByName("MyClass") as Class;
myClass["myVariable1"] = x;
I want to run multiple test cases against the content of a whole set of files. I could use a data provider to load my files and use the same provider for all the tests like this:
class mytest extends PHPUnit_Framework_TestCase {
public function contentProvider() {
return glob(__DIR__ . '/files/*');
}
/**
* #dataProvider contentProvider
*/
public function test1($file) {
$content = file_get_contents($file);
// assert something here
}
...
/**
* #dataProvider contentProvider
*/
public function test10($file) {
$content = file_get_contents($file);
// assert something here
}
}
Obviously that means if I have 10 test cases, each file is loaded 10 times.
I could adjust the data provider to load all files and return one big structure with all the contents. But since the provider is called separately for each test it would still mean each file is loaded 10 times and in addition it would load all files into memory at the same time.
I could of course condense the 10 tests into one test with 10 assertions, but then it would abort right after the first assertion fails and I really want a report of all things that are wrong with the file.
I know that data providers can also return an iterator. But phpunit seems to rerun the iterator separately for each test, still resulting in loading each file 10 times.
Is there a clever way to make phpunit run an iterator only once and pass the result to each test, before continuing?
Test dependencies
If some tests are dependents, you should use the #depends annotation to declare Test dependencies. The data returned by the dependency is used by the test declaring this dependency.
But, if a test declared as dependency failed, the dependent test is not executed.
Statically stored data
To share data between tests, it's common to setup fixtures statically.
You can use the same method with data providers:
<?php
use PHPUnit\Framework\TestCase;
class MyTest extends TestCase
{
private static $filesContent = NULL;
public function filesContentProvider()
{
if (self::$filesContent === NULL) {
$paths = glob(__DIR__ . '/files/*');
self::$filesContent = array_map(function($path) {
return [file_get_contents($path)];
}, $paths);
}
}
/**
* #dataProvider filesContentProvider
*/
public function test1($content)
{
$this->assertNotEmpty($content, 'File must not be empty.');
}
/**
* #dataProvider filesContentProvider
*/
public function test2($content)
{
$this->assertStringStartsWith('<?php', $content,
'File must start with the PHP start tag.');
}
}
As you can see, it's not supported out of the box. As the test class instance is destroyed after each test method execution, you have to store the initialized data in a class variable.
Since getting started in Dart I've been watching for a way to execute Dart (Text) Source (that the same program may well be generating dynamically) as Code. Like the infamous "eval()" function.
Recently I have caught a few hints that the communication port between Isolates support some sort of "Spawn" that seems like it could allow this "trick". In Ruby there is also the possibility to load a module dynamically as a language feature, perhaps there is some way to do this in Dart?
Any clues or a simple example will be greatly appreciated.
Thanks in advance!
Ladislav Thon provided this answer on the Dart forum:
I believe it's very safe to say that Dart will never have eval. But it will have other, more structured ways of dynamically generating code (code name mirror builders). There is nothing like that right now, though.
There are two ways of spawning an isolate: spawnFunction, which runs an existing function from the existing code in a new isolate, so nothing you are looking for, and spawnUri, which downloads code from given URI and runs it in new isolate. That is essentially dynamic code loading -- but the dynamically loaded code is isolated from the existing code. It runs in a new isolate, so the only means of communicating with it is via message passing (through ports).
You can run a string as Dart code by first constructing a data URI from it and then passing it into Isolate.spawnUri.
import 'dart:isolate';
void main() async {
final uri = Uri.dataFromString(
'''
void main() {
print("Hellooooooo from the other side!");
}
''',
mimeType: 'application/dart',
);
await Isolate.spawnUri(uri, [], null);
}
Note that you can only do this in JIT mode, which means that the only place you might benefit from it is Dart VM command line apps / package:build scripts. It will not work in Flutter release builds.
To get a result back from it, you can use ports:
import 'dart:isolate';
void main() async {
final name = 'Eval Knievel';
final uri = Uri.dataFromString(
'''
import "dart:isolate";
void main(_, SendPort port) {
port.send("Nice to meet you, $name!");
}
''',
mimeType: 'application/dart',
);
final port = ReceivePort();
await Isolate.spawnUri(uri, [], port.sendPort);
final String response = await port.first;
print(response);
}
I wrote about it on my blog.
Eval(), in Ruby at least, can execute anything from a single statement (like an assignment) to complete involved programs. There is a substantial time penalty for executing many small snippets over most any other form of execution that is possible.
Looking at the problem closer, there are at least three different functions that were at the base of the various schemes where eval might be used. Dart handles at least 2 of these in at least minimal ways.
Dart does not, nor does it look like there is any plan to support "general" script execution.
However, the NoSuchMethod method can be used to effectively implement the dynamic "injection" of variables into your local class environment. It replaces an eval() with a string that would look like this: eval( "String text = 'your first name here';" );
The second function that Dart readily supports now is the invocation of a method, that would look like this: eval( "Map map = SomeClass.some_method()" );
After messing about with this it finally dawned on me that a single simple class can be used to store the information needed to invoke a method, for a class, as a string which seems to have general utility. I can replace a big maintenance prone switch statement that might otherwise be necessary to invoke a series of methods. In Ruby this was almost trivial, however in Dart there are some fairly less than intuitive calls so I wanted to get this "trick" in one place, which fits will with doing ordering and filtering on the strings such as you may need.
Here's the code to "accumulate" as many classes (a whole library?) into a map using reflection such that the class.methodName() can be called with nothing more than a key (as a string).
Note: I used a few "helper methods" to do Map & List functions, you will probably want to replace them with straight Dart. However this code is used and tested only using the functions..
Here's the code:
//The used "Helpers" here..
MAP_add(var map, var key, var value){ if(key != null){map[key] = value;}return(map);}
Object MAP_fetch(var map, var key, [var dflt = null]) {var value = map[key];if (value==null) {value = dflt;}return( value );}
class ClassMethodMapper {
Map _helperMirrorsMap, _methodMap;
void accum_class_map(Object myClass){
InstanceMirror helperMirror = reflect(myClass);
List methodsAr = helperMirror.type.methods.values;
String classNm = myClass.toString().split("'")[1]; ///#FRAGILE
MAP_add(_helperMirrorsMap, classNm, helperMirror);
methodsAr.forEach(( method) {
String key = method.simpleName;
if (key.charCodeAt(0) != 95) { //Ignore private methods
MAP_add(_methodMap, "${classNm}.${key}()", method);
}
});
}
Map invoker( String methodNm ) {
var method = MAP_fetch(_methodMap, methodNm, null);
if (method != null) {
String classNm = methodNm.split('.')[0];
InstanceMirror helperMirror = MAP_fetch(_helperMirrorsMap, classNm);
helperMirror.invoke(method.simpleName, []);
}
}
ClassMethodMapper() {
_methodMap = {};
_helperMirrorsMap = {};
}
}//END_OF_CLASS( ClassMethodMapper );
============
main() {
ClassMethodMapper cMM = new ClassMethodMapper();
cMM.accum_class_map(MyFirstExampleClass);
cMM.accum_class_map(MySecondExampleClass);
//Now you're ready to execute any method (not private as per a special line of code above)
//by simply doing this:
cMM.invoker( MyFirstExampleClass.my_example_method() );
}
Actually there some libraries in pub.dev/packages but has some limitations because are young versions, so that I can recommend you this library expressions to dart and flutter.
A library to parse and evaluate simple expressions.
This library can handle simple expressions, but no blocks of code, control flow statements and so on. It supports a syntax that is common to most programming languages.
There I create an example of code to evaluate arithmetic operations and comparations of data.
import 'package:expressions/expressions.dart';
import 'dart:math';
#override
Widget build(BuildContext context) {
final parsing = FormulaMath();
// Expression example
String condition = "(cos(x)*cos(x)+sin(x)*sin(x)==1) && respuesta_texto == 'si'";
Expression expression = Expression.parse(condition);
var context = {
"x": pi / 5,
"cos": cos,
"sin": sin,
"respuesta_texto" : 'si'
};
// Evaluate expression
final evaluator = const ExpressionEvaluator();
var r = evaluator.eval(expression, context);
print(r);
return Scaffold(
body: Container(
margin: EdgeInsets.only(top: 50.0),
child: Column(
children: [
Text(condition),
Text(r.toString())
],
),
),
);
}
I/flutter (27188): true