I use value-parameterized tests in gtest. For example, if I write
INSTANTIATE_TEST_CASE_P(InstantiationName,
FooTest,
::testing::Values("meeny", "miny", "moe"));
then in the output I see test names such as
InstantiationName/FooTest.DoesBlah/0 for "meeny"
InstantiationName/FooTest.DoesBlah/1 for "miny"
InstantiationName/FooTest.DoesBlah/2 for "moe"
Is there any way to make these names more meaningful? I'd like to see
InstantiationName/FooTest.DoesBlah/meeny
InstantiationName/FooTest.DoesBlah/miny
InstantiationName/FooTest.DoesBlah/moe
INSTANTIATE_TEST_CASE_P accepts an optional 4th argument which can be used for this purpose. See https://github.com/google/googletest/blob/fbef0711cfce7b8f149aac773d30ae48ce3e166c/googletest/include/gtest/gtest-param-test.h#L444.
This is now available in INSTANTIATE_TEST_SUITE_P.
The optional last argument to INSTANTIATE_TEST_SUITE_P() allows the
user to specify a function or functor that generates custom test name
suffixes based on the test parameters.
Of interest is also this section in the source:
// A user can teach this function how to print a class type T by
// defining either operator<<() or PrintTo() in the namespace that
// defines T. More specifically, the FIRST defined function in the
// following list will be used (assuming T is defined in namespace
// foo):
//
// 1. foo::PrintTo(const T&, ostream*)
// 2. operator<<(ostream&, const T&) defined in either foo or the
// global namespace.
Two ways: (http://osdir.com/ml/googletestframework/2011-09/msg00005.html)
1) Patch the existing PrettyUnitTestPrinter to print test names; something like:
--- a/gtest-1.7.0/src/gtest.cc
+++ b/gtest-1.7.0/src/gtest.cc
## -2774,6 +2774,7 ## void PrettyUnitTestResultPrinter::OnTestCaseStart(const TestCase& test_case) {
void PrettyUnitTestResultPrinter::OnTestStart(const TestInfo& test_info) {
ColoredPrintf(COLOR_GREEN, "[ RUN ] ");
PrintTestName(test_info.test_case_name(), test_info.name());
+ PrintFullTestCommentIfPresent(test_info);
printf("\n");
fflush(stdout);
}
2) Write a new TestListener to print test results however you like. (https://code.google.com/p/googletest/source/browse/trunk/samples/sample9_unittest.cc) GTest allows registering a new test listener (and un-registering the builtin default), allowing pretty flexible customization of test output. See the link for example code.
Related
my scenario: I have an existing unit test framework with ~3000 individual test cases. They are made from TEST, TEST_F and TEST_P macros.
Internally the tested modules make use of a logger library and now my goal is to create individual log files for each test case. To do so I would like to call a function as a SetUp for each test case.
Is there a way to register such function at the framework and get it called automatically?
The obvious solution for me would look like: do the work in a test fixture constructor or SetUp() but then I'd have to touch every single test case.
I do like the idea of registering a global setup at the framework with AddGlobalTestEnvironment() but as I understand this is handled only once per executable.
By the way: I have acceptance tests implemented in robot test and guess what? I want to repeat the task there...
Thanks for any inspiration!
Christoph
You mentioned:
The obvious solution for me would look like: do the work in a test fixture constructor or SetUp() but then I'd have to touch every single test case.
If the reason that you think you would need to touch every single test case is to set the filename differently, you can use the combination of SetUp() function and the current_test_info provided by GTest to get the test name for each test, and then use that to create a separate file for each test.
Here is an example:
// Class for test fixture
class MyTestFixture : public ::testing::Test {
protected:
void SetUp() override {
test_name_ = std::string(
::testing::UnitTest::GetInstance()->current_test_info()->name());
std::cout << "test_name_: " << test_name_ << std::endl;
// CreateYourLogFileUsingTestName(test_name_);
}
std::string test_name_;
};
TEST_F(MyTestFixture, Test1) {
EXPECT_EQ(this->test_name_, std::string("Test1"));
}
TEST_F(MyTestFixture, Test2) {
EXPECT_EQ(this->test_name_, std::string("Test2"));
}
Live example here: https://godbolt.org/z/YjzEG3G77
The solution I found in the gtest docs:
class TraceHandler : public testing::EmptyTestEventListener
{
// Called before a test starts.
void OnTestStart( const testing::TestInfo& test_info ) override
{
// set the logfilename here
}
// Called after a test ends.
void OnTestEnd( const testing::TestInfo& test_info ) override
{
// close the log here
}
};
int main( int argc, char** argv )
{
testing::InitGoogleTest( &argc, argv );
testing::TestEventListeners& listeners =
testing::UnitTest::GetInstance()->listeners();
// Adds a listener to the end. googletest takes the ownership.
listeners.Append(new TraceHandler);
return RUN_ALL_TESTS();
}
This way it automatically applies to all tests linked to this main-function.
Maybe I have to mention: my logger is a collection of static functions that send udp-packets to a receiver that cares for actual logging. I can control the filename by one of that functions. That's the reason why I don't need to insert code in every single TEST, TEST_F or TEST_P.
Various aspects of OptaPlanner configuration can be specified in the XML config then overridden via JavaAPI. For example you can set a default TerminationConfig in your XML then at runtime switch it for a user specified value.
The docs seem to indicate that it should be possible, but I haven't found any examples. The context is that I'm hoping to dynamically alter the UnionMove section of the config and omit Swap moves if the instance to be solved doesn't have at least 2 planning entities. Currently this results in lots of Bailing from infinite logging when users are experimenting with small problems.
Taken from https://www.optaplanner.org/docs/optaplanner/latest/planner-configuration/planner-configuration.html#solverConfigurationByJavaAPI :
private SolverConfig template;
public void init() {
template = SolverConfig.createFromXmlResource(
"org/optaplanner/examples/nqueens/solver/nqueensSolverConfig.xml");
template.setTerminationConfig(new TerminationConfig());
}
// Called concurrently from different threads
public void userRequest(..., long userInput) {
SolverConfig solverConfig = new SolverConfig(template); // Copy it
solverConfig.getTerminationConfig().setMinutesSpentLimit(userInput);
SolverFactory<NQueens> solverFactory = SolverFactory.create(solverConfig);
Solver<NQueens> solver = solverFactory.buildSolver();
...
}
In your case, you would need to modify the Phase List of the SolverConfig (see the SolverConfig Javadocs ); the move selectors can be gotten and set via LocalSearchPhaseConfig and ConstructionHeuristicPhaseConfig (you need to use instanceof when iterating phase list to determine which one to cast to).
The bailing from infinite logging sound like a bug.
I've a bunch of test cases in which each case would output a file. I'd like each files would be named after the corresponding case name so the result files won't get in the way with each other. Is there any way to do that by programming?
What I have now (hate to repeat the case name since it could be changed:
TEST_F(Foo, Bar)
{
...
std::ofstream("Bar.bat");
...
}
What I want:
TEST_F(Foo, Bar)
{
...
std::ofstream(magic_method_to_get_case_name() + ".dat");
...
}
The current test's name can be accessed via the UnitTest singleton class.
::testing::UnitTest::GetInstance()->current_test_info()->name();
Because UnitTest is a singleton, this can be called from static/global test utility methods as well.
Just find out how to do that. I'll share it here in case anyone would like to know it.
Looks like GTEST_TEST_ macro would keep the name information in a private static variable:
#define GTEST_TEST_(test_case_name, test_name, parent_class, parent_id)\
class GTEST_TEST_CLASS_NAME_(test_case_name, test_name) : public parent_class {\
public:\
GTEST_TEST_CLASS_NAME_(test_case_name, test_name)() {}\
private:\
virtual void TestBody();\
static ::testing::TestInfo* const test_info_;\
GTEST_DISALLOW_COPY_AND_ASSIGN_(\
GTEST_TEST_CLASS_NAME_(test_case_name, test_name));\
};\
So, the code in any case could get the case name through it:
test_info_->name()
Case name is not the only information that could be fetched -- please check the TestInfo class in gtest.h.
I have a class Configuration that reads in environment variables:
class Configuration {
has $.config_string_a;
has $.config_string_b;
has Bool $.config_flag_c;
method new() {
sub assertHasEnv(Str $envVar) {
die "environment variable $envVar must exist" unless %*ENV{$envVar}:exists;
}
assertHasEnv('CONFIG_STRING_A');
assertHasEnv('CONFIG_STRING_B');
assertHasEnv('CONFIG_FLAG_C');
return self.bless(
config_string_a => %*ENV{'CONFIG_STRING_A'},
config_string_b => %*ENV{'CONFIG_STRING_B'},
config_flag_c => Bool(%*ENV{'CONFIG_FLAG_C'}),
);
}
}
my $config = Configuration.new;
say $config.config_string_a;
say $config.config_string_b;
say $config.config_flag_c;
Is there a more concise way to express this? For example, I am repeating the environment variable name in the check and the return value of the constructor.
I could easily see writing another, more generic class that encapsulates the necessary info for a config parameter:
class ConfigurationParameter {
has $.name;
has $.envVarName;
has Bool $.required;
method new (:$name, :$envVarName, :$required = True) {
return self.bless(:$name, :$envVarName, :$required);
}
}
Then rolling these into a List in the Configuration class. However, I don't know how to refactor the constructor in Configuration to accommodate this.
The most immediate change that comes to mind is to change new to be:
method new() {
sub env(Str $envVar) {
%*ENV{$envVar} // die "environment variable $envVar must exist"
}
return self.bless(
config_string_a => env('CONFIG_STRING_A'),
config_string_b => env('CONFIG_STRING_B'),
config_flag_c => Bool(env('CONFIG_FLAG_C')),
);
}
While // is a definedness check rather than an existence one, the only way an environment variable will be undefined is if it isn't set. That gets down to one mention of %*ENV and also of each environment variable.
If there's only a few, then I'd likely stop there, but the next bit of repetition that strikes me is the names of the attributes are just lowercase of the names of the environment variables, so we could eliminate that duplication too, at the cost of a little more complexity:
method new() {
multi env(Str $envVar) {
$envVar.lc => %*ENV{$envVar} // die "environment variable $envVar must exist"
}
multi env(Str $envVar, $type) {
.key => $type(.value) given env($envVar)
}
return self.bless(
|env('CONFIG_STRING_A'),
|env('CONFIG_STRING_B'),
|env('CONFIG_FLAG_C', Bool),
);
}
Now env returns a Pair, and | flattens it in to the argument list as if it's a named argument.
Finally, the "power tool" approach is to write a trait like this outside of the class:
multi trait_mod:<is>(Attribute $attr, :$from-env!) {
my $env-name = $attr.name.substr(2).uc;
$attr.set_build(-> | {
with %*ENV{$env-name} -> $value {
Any ~~ $attr.type ?? $value !! $attr.type()($value)
}
else {
die "environment variable $env-name must exist"
}
});
}
And then write the class as:
class Configuration {
has $.config_string_a is from-env;
has $.config_string_b is from-env;
has Bool $.config_flag_c is from-env;
}
Traits run at compile time, and can manipulate a declaration in various ways. This trait calculates the name of the environment variable based on the attribute name (attribute names are always like $!config_string_a, thus the substr). The set_build sets the code that will be run to initialize the attribute when the class is created. That gets passed various things that in our situation aren't important, so we ignore the arguments with |. The with is just like if defined, so this is the same approach as the // earlier. Finally, the Any ~~ $attr.type check asks if the parameter is constrained in some way, and if it is, performs a coercion (done by invoking the type with the value).
So I mentioned this in a comment but I figured it would be good as an actual answer. I figured this would be useful functionality for anyone building a Docker based system so took Jonanthan's example code, added some functionality for exporting Traits Elizabeth showed me and made Trait::Env
Usage is :
use Trait::Env;
class Configuration {
has $.config_string_a is env;
has $.config-string-b is env(:required);
has Bool $.config-flag-c is env is default(True);
}
The :required flag turns on die if not found. And it plays nicely with the is default trait. Attribute names are upper cased and - is replaced with _ before checking %*ENV.
I have a couple of planned changes, make it throw a named Exception rather than just die and handle Boolean's a bit better. As %*ENV is Strings having a Boolean False is a bit of a pain.
I want to create a Developer Variable to the workspace in Blockly, but I cannot find the necessary function/method.
I do not want to create the variable over a button. The variable should be included even if there is no block in the workspace.
With these two functions I can get the already created variables:
var variables = workspace.getAllVariables();
var dev_var = Blockly.Variables.allDeveloperVariables(workspace);
But what is the setting function?
Developer variables are variables that will never be visible to the user, but will exist in the generated code. If that's what you're looking for: there's no API for it, but here are some things you can do.
If you want to reserve the name so that users can't accidentally override your variable, call yourGenerator.addReservedWords('var1,var2,...'). You can initialize the variable in your wrapper code.
If you really want Blockly to both reserve and declare the variable for you, you could override the init function on your generator.
On the other hand, if what you want is a user-visible variable that always shows up in the toolbox, without the user creating it, you should call yourWorkspace.createVariable('variable_name').
The unit test blocks all assume that the variable unittestResults exists and can be written to. To indicate this, the block definition includes the function getDeveloperVars, which returns an array of strings. Each string is a variable name.Follow this issue in gtihub
Blockly.Blocks['unittest_fail'] = {
// Always assert an error.
init: function() {
this.setColour(65);
this.setPreviousStatement(true);
this.setNextStatement(true);
this.appendDummyInput()
.appendField(new Blockly.FieldTextInput('test name'), 'MESSAGE')
.appendField('fail');
this.setTooltip('Records an error.');
},
getDeveloperVars: function() {
return ['unittestResults'];
}
};
LINK : https://github.com/google/blockly/issues/1535