take a look at the code below
class ExampleInterface {
public:
virtual void exMethod(int param)=0;
};
class MyMock : public ExampleInterface {
MOCK_METHOD1(exMethod, void(int));
};
TEST_F(TestCls, test1){
MyMock mock;
EXPECT_CALL(mock, exMethod(4)).Times(1);
mock.exMethod(4);
mock.exMethod(5);
}
this test fails with message
...Expected: to be called once
Actual: called once - saturated and active
I want this test to pass because I am not interested in other exMethod calls if they don't match the ecpect_call. how to tell gmock to ignore any calls that don't match the expectation?
You could tell it to also expect to be called with any parameter any number of times:
EXPECT_CALL(mock, exMethod(_)).Times(AnyNumber());
EXPECT_CALL(mock, exMethod(4)).Times(1);
Note that the order of the expectations is important, as the most recent expectations take priority. If you put them the other way around the "_" matcher would match everything and the "4" matcher would never be satisfied.
Related
my scenario: I have an existing unit test framework with ~3000 individual test cases. They are made from TEST, TEST_F and TEST_P macros.
Internally the tested modules make use of a logger library and now my goal is to create individual log files for each test case. To do so I would like to call a function as a SetUp for each test case.
Is there a way to register such function at the framework and get it called automatically?
The obvious solution for me would look like: do the work in a test fixture constructor or SetUp() but then I'd have to touch every single test case.
I do like the idea of registering a global setup at the framework with AddGlobalTestEnvironment() but as I understand this is handled only once per executable.
By the way: I have acceptance tests implemented in robot test and guess what? I want to repeat the task there...
Thanks for any inspiration!
Christoph
You mentioned:
The obvious solution for me would look like: do the work in a test fixture constructor or SetUp() but then I'd have to touch every single test case.
If the reason that you think you would need to touch every single test case is to set the filename differently, you can use the combination of SetUp() function and the current_test_info provided by GTest to get the test name for each test, and then use that to create a separate file for each test.
Here is an example:
// Class for test fixture
class MyTestFixture : public ::testing::Test {
protected:
void SetUp() override {
test_name_ = std::string(
::testing::UnitTest::GetInstance()->current_test_info()->name());
std::cout << "test_name_: " << test_name_ << std::endl;
// CreateYourLogFileUsingTestName(test_name_);
}
std::string test_name_;
};
TEST_F(MyTestFixture, Test1) {
EXPECT_EQ(this->test_name_, std::string("Test1"));
}
TEST_F(MyTestFixture, Test2) {
EXPECT_EQ(this->test_name_, std::string("Test2"));
}
Live example here: https://godbolt.org/z/YjzEG3G77
The solution I found in the gtest docs:
class TraceHandler : public testing::EmptyTestEventListener
{
// Called before a test starts.
void OnTestStart( const testing::TestInfo& test_info ) override
{
// set the logfilename here
}
// Called after a test ends.
void OnTestEnd( const testing::TestInfo& test_info ) override
{
// close the log here
}
};
int main( int argc, char** argv )
{
testing::InitGoogleTest( &argc, argv );
testing::TestEventListeners& listeners =
testing::UnitTest::GetInstance()->listeners();
// Adds a listener to the end. googletest takes the ownership.
listeners.Append(new TraceHandler);
return RUN_ALL_TESTS();
}
This way it automatically applies to all tests linked to this main-function.
Maybe I have to mention: my logger is a collection of static functions that send udp-packets to a receiver that cares for actual logging. I can control the filename by one of that functions. That's the reason why I don't need to insert code in every single TEST, TEST_F or TEST_P.
I am trying to test the code and get a result as String to send it to API later.
class FirstClass{
fun main(){
print("Hello world!")
}
}
Test:
ort org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import java.io.ByteArrayOutputStream
import java.io.PrintStream
import kotlin.test.assertEquals
internal class FirstClassTest {
private val outContent = ByteArrayOutputStream()
private val errContent = ByteArrayOutputStream()
private val originalOut = System.out
private val originalErr = System.err
#BeforeEach
fun setUp() {
System.setOut(PrintStream(outContent))
System.setErr(PrintStream(errContent))
}
#AfterEach
fun tearDown() {
System.setOut(originalOut)
System.setErr(originalErr)
}
#Test
fun main() {
val SUT = FirstClass()
SUT.main()
val testResult = assertEquals("Hello world!", outContent.toString())
print("Test result: $testResult")
val api = Api()
val apiResult = api.sendResult(testResult.toString())
print("Api result: $apiResult")
}
}
The test is passing, however, I do not see printed messages. How to get a test result as String?
There are several issues here. The main one is:
The redirection affects your test method too.
Because you've redirected System.out, the print() in your test method goes to outContent, along with the output from FirstClass.main() that you want to test, instead of to the screen or wherever you want it.
I can see two fixes for this.
The quick one is for your test method to output to originalOut:
originalOut.print("Test result: $testResult")
Your test method is in the class which does the redirection, so there's no problem with it knowing about the redirection, and it already has access to originalOut.
However, if you can, I think a better solution is to refactor FirstClass so that it doesn't hard-code the stream it writes to. For example, the stream could be passed as a parameter; or it could return the string directly (and the caller, in a thin non-tested wrapper, could write it to System.out).
That would be more work, but would make your code more flexible as well as easier to test.
Other issues include:
You're using print() instead of println().
Many streams are line-buffered, writing their output only after a newline, and so you might not see any results if there isn't one. (And even if you do, all the results would be jammed on a single line!)
You assign the result of assertEquals().
assertEquals() doesn't have a useful return value. (It return Unit.) So your code will simply show:
Test result: kotlin.Unit
Instead, like all the assert functions, it throws an exception if the assertion fails. So there's no point in storing or processing the return value; simply calling the assertion is enough.
— This means that there's usually no need to call print()/println() from your test method anyway! If there's a failure, it'll be obvious: running from the command line will stop with an exception message and stack trace; IntelliJ shows a big red mark next to that test; Maven and Gradle will stop the build (after all tests have run), showing the number of failures. So if everything continues smoothly, you know the tests have passed.
Api is not defined.
The code you posted above won't compile, because it doesn't include a definition or import for Api. (Those last lines can be removed, though, without affecting the question.)
main() is a confusing name for a test.
The unit testing framework will find and run all test methods annotated with #Test. A test class will often contain many different test methods, and it's usual to name them after the aspect they're testing. (That makes any failures clearer.) Calling it main() not only fails to describe what's being tested, but also suggests that the method will be run from outside the testing framework, which would probably not behave properly.
I want to use something like Cucumber JVM to drive performance tests written for Gatling.
Ideally the Cucumber features would somehow build a scenario dynamically - probably reusing predefined chain objects similar to the method described in the "Advanced Tutorial", e.g.
val scn = scenario("Scenario Name").exec(Search.search("foo"), Browse.browse, Edit.edit("foo", "bar")
I've looked at how the Maven plugin executes the scripts, and I've also seen mention of using an App trait but I can't find any documentation for the later and it strikes me that somebody else will have wanted to do this before...
Can anybody point (a Gatling noob) in the direction of some documentation or example code of how to achieve this?
EDIT 20150515
So to explain a little more:
I have created a trait which is intended to build up a sequence of, I think, ChainBuilders that are triggered by Cucumber steps:
trait GatlingDsl extends ScalaDsl with EN {
private val gatlingActions = new ArrayBuffer[GatlingBehaviour]
def withGatling(action: GatlingBehaviour): Unit = {
gatlingActions += action
}
}
A GatlingBehaviour would look something like:
object Google {
class Home extends GatlingBehaviour {
def execute: ChainBuilder =
exec(http("Google Home")
.get("/")
)
}
class Search extends GatlingBehaviour {...}
class FindResult extends GatlingBehaviour {...}
}
And inside the StepDef class:
class GoogleStepDefinitions extends GatlingDsl {
Given( """^the Google search page is displayed$""") { () =>
println("Loading www.google.com")
withGatling(Home())
}
When( """^I search for the term "(.*)"$""") { (searchTerm: String) =>
println("Searching for '" + searchTerm + "'...")
withGatling(Search(searchTerm))
}
Then( """^"(.*)" appears in the search results$""") { (expectedResult: String) =>
println("Found " + expectedResult)
withGatling(FindResult(expectedResult))
}
}
The idea being that I can then execute the whole sequence of actions via something like:
val scn = Scenario(cucumberScenario).exec(gatlingActions)
setup(scn.inject(atOnceUsers(1)).protocols(httpConf))
and then check the reports or catch an exception if the test fails, e.g. response time too long.
It seems that no matter how I use the 'exec' method it tries to instantly execute it there and then, not waiting for the scenario.
Also I don't know if this is the best approach to take, we'd like to build some reusable blocks for our Gatling tests that can be constructed via Cucumber's Given/When/Then style. Is there a better or already existing approach?
Sadly, it's not currently feasible to have Gatling directly start a Simulation instance.
Not that's it's not technically feasible, but you're just the first person to try to do this.
Currently, Gatling is usually in charge of compiling and can only be passed the name of the class to load, not an instance itself.
You can maybe start by forking io.gatling.app.Gatling and io.gatling.core.runner.Runner, and then provide a PR to support this new behavior. The former is the main entry point, and the latter the one can instanciate and run the simulation.
I recently ran into a similar situation, and did not want to fork gatling. And while this solved my immediate problem, it only partially solves what you are trying to do, but hopefully someone else will find this useful.
There is an alternative. Gatling is written in Java and Scala so you can call Gatling.main directly and pass it the arguments you need to run the Gatling Simulation you want. The problem is, the main explicitly calls System.exit so you have to also use a custom security manager to prevent it from actually exiting.
You need to know two things:
the class (with the full package) of the Simulation you want to run
example: com.package.your.Simulation1
the path where the binaries are compiled.
The code to run a Simulation:
protected void fire(String gatlingGun, String binaries){
SecurityManager sm = System.getSecurityManager();
System.setSecurityManager(new GatlingSecurityManager());
String[] args = {"--simulation", gatlingGun,
"--results-folder", "gatling-results",
"--binaries-folder", binaries};
try {
io.gatling.app.Gatling.main(args);
}catch(SecurityException se){
LOG.debug("gatling test finished.");
}
System.setSecurityManager(sm);
}
The simple security manager i used:
public class GatlingSecurityManager extends SecurityManager {
#Override
public void checkExit(int status){
throw new SecurityException("Tried to exit.");
}
#Override
public void checkPermission(Permission perm) {
return;
}
}
The problem is then getting the information you want out of the simulation after it has been run.
I've just tried to write a simple test for Auth:
use Mockery as m;
...
public function testHomeWhenUserIsNotAuthenticatedThenRedirectToWelcome() {
$auth = m::mock('Illuminate\Auth\AuthManager');
$auth->shouldReceive('guest')->once()->andReturn(true);
$this->call('GET', '/');
$this->assertRedirectedToRoute('general.welcome');
}
public function testHomeWhenUserIsAuthenticatedThenRedirectToDashboard() {
$auth = m::mock('Illuminate\Auth\AuthManager');
$auth->shouldReceive('guest')->once()->andReturn(false);
$this->call('GET', '/');
$this->assertRedirectedToRoute('dashboard.overview');
}
This is the code:
public function getHome() {
if(Auth::guest()) {
return Redirect::route('general.welcome');
}
return Redirect::route('dashboard.overview');
}
When I run, I've got the following error:
EF.....
Time: 265 ms, Memory: 13.00Mb
There was 1 error:
1) PagesControllerTest::testHomeWhenUserIsNotAuthenticatedThenRedirectToWelcome
Mockery\Exception\InvalidCountException: Method guest() from Mockery_0_Illuminate_Auth_AuthManager should be called
exactly 1 times but called 0 times.
—
There was 1 failure:
1) PagesControllerTest::testHomeWhenUserIsAuthenticatedThenRedirectToDashboard
Failed asserting that two strings are equal.
--- Expected
+++ Actual
## ##
-'http://localhost/dashboard/overview'
+'http://localhost/welcome'
My questions are:
Two similar test cases but why the error output differs? First one the mock Auth::guest() is not called while the second one seems to be called.
On the second test case, why does it fail?
Is there any way to write better tests for my code above? Or even better code to test.
Above test cases, I use Mockery to mock the AuthManager, but if I use the facade Auth::shoudReceive()->once()->andReturn(), then it works eventually. Is there any different between Mockery and Auth::mock facade here?
Thanks.
You're actually mocking a new instance of the Illuminate\Auth\AuthManager and not accessing the Auth facade that is being utilized by your function getHome(). Ergo, your mock instance will never get called. (Standard disclaimer that none of the following code is tested.)
Try this:
public function testHomeWhenUserIsNotAuthenticatedThenRedirectToWelcome() {
Auth::shouldReceive('guest')->once()->andReturn(true);
$this->call('GET', '/');
$this->assertRedirectedToRoute('general.welcome');
}
public function testHomeWhenUserIsAuthenticatedThenRedirectToDashboard() {
Auth::shouldReceive('guest')->once()->andReturn(false);
$this->call('GET', '/');
$this->assertRedirectedToRoute('dashboard.overview');
}
If you check out Illuminate\Support\Facades\Facade, you'll see that it takes care of mocking for you. If you really wanted to do it the way that you were doing it (creating an instance of mock instance of Auth), you'd have to somehow inject it into the code under test. I believe that it could be done with something like this assuming that you extend from the TestCase class provided by laravel:
public function testHomeWhenUserIsNotAuthenticatedThenRedirectToWelcome() {
$this->app['auth'] = $auth = m::mock('Illuminate\Auth\AuthManager');
// above line will swap out the 'auth' facade with your facade.
$auth->shouldReceive('guest')->once()->andReturn(true);
$this->call('GET', '/');
$this->assertRedirectedToRoute('general.welcome');
}
I'm attempting to test a typical controller flow for user login. There are extended relations, as with most login systems, and the Grails documentation is completely useless. It doesn't have a single example that is actually real-world relevant for typical usage and is a feature complete example.
my test looks like this:
#TestFor(UserController)
class UserControllerTests extends GroovyTestCase {
void testLogin() {
params.login = [email: "test1#example.com", password: "123"]
controller.login()
assert "/user/main" == response.redirectUrl
}
}
The controller does:
def login() {
if (!params.login) {
return
}
println("Email " + params.login.email)
Person p = Person.findByEmail(params?.login?.email)
...
}
which fails with:
groovy.lang.MissingMethodException: No signature of method: immigration.Person.methodMissing() is applicable for argument types: () values: []
The correct data is shown in the println, but the method fails to be called.
The test suite cannot use mocks overall because there are database triggers that get called, the result of which will need to be tested.
The bizarre thing is that if I call the method directly in the test, it works, but calling it in the controller doesn't.
For an update: I regenerated the test directly from the grails command, and it added the #Test annotation:
#TestFor(UserController)
class UserControllerTests extends GroovyTestCase {
#Test
void testLogin() {
params.login = [email: "test1#example.com", password: "123"]
Person.findByEmail(params.login.email)
controller.login()
}
}
This works if I run it with
grail test-app -integration UserController
though the result isn't populated correctly - the response is empty, flash.message is null even though it should have a value, redirectedUrl is null, content body is empty, and so is view.
If I remove the #TestFor annotation, it doesn't work even in the slightest. It fails telling me that 'params' doesn't exist.
In another test, I have two methods. The first method runs, finds Person.findAllByEmail(), then the second method runs and can't find Person.findAllByEmail and crashes with a similar error - method missing.
In another weird update - it looks like the response object is sending back a redirect, but to the application baseUrl, not to the user controller at all.
Integration tests shouldn't use #TestFor. You need to create an instance of the controller in your test, and set params in that:
class UserControllerTests extends GroovyTestCase {
void testLogin() {
def controller = new UserController()
controller.params.login = [email:'test1#example.com', password:'123']
controller.login()
assert "/user/main" == controller.response.redirectedUrl
}
}
Details are in the user guide.
The TestFor annotation is used only in unit tests, since this mocks the structure. In the integration tests you have access of the full Grails environment, so there's no need for this mocks. So just remove the annotation and should work.
class UserControllerTests extends GroovyTestCase {
...
}