I'm mocking a method with a rather complex parameter list in jmockit, and I'm suspecting that some things are not sufficiently 'equals'. Where in jmockit might one place a breakpoint to observe the comparison process and step through it?
Related
(I'm reasonably sure the answer is "no", but I want to make sure.)
In JUnit 5 you can write an extension that is an implementation of ParameterResolver. Before your test runs, if the method in question has parameters, then an extension that implements ParameterResolver can return the object suitable as an argument for that parameter.
You can also write an extension that is an implementation of InvocationInterceptor, that is in charge of intercepting a test method's execution. You can get any arguments (such as those resolved by ParameterResolvers), but it appears you cannot change them.
In terms of execution order, if there are relevant parameters, then a ParameterResolver will "fire" first, and then any InvocationInterceptors will "fire" next.
(Lastly, if your test method declares parameters, but there are no ParameterResolvers to resolve them, everything craps out.)
Putting this all together:
Consider the case when a parameter can't really be properly resolved until the stuff that an interceptor sets up prior to execution is complete:
What is the best way, if there is one, to have all of the following:
A parameter that conceivably only the interceptor could resolve
Deferred resolution of that parameter (i.e. the actual parameter value is not sought by the JUnit internals until interception time so that the interceptor could resolve it just-in-time before calling proceed()
…?
(In my very concrete case, I got lucky: the parameter I'm interested in is an interface, so I "resolve" it to a dummy implementation, and then, at interception time, "fill" the dummy implementation with a delegate that does the real work. I can't think of a better way with the existing JUnit 5 toolkit.)
(I can almost get there if ReflectiveInvocationContext would allow me to set its arguments: my resolveParameter implementation could return null and my interceptor could replace the null reference it found in the arguments with an appropriate non-null argument just-in-time.)
(I also am at least aware of the ExecutableInvoker interface that is reachable from the ExtensionContext, but I'm unclear how that would help me in this scenario, since parameter resolution happens before interception.)
JUnit 5's InvocationInterceptor.Invocation#skip() method's documentation says:
Explicitly skip this invocation.
I find this a bit unclear.
Suppose the invocation represents a test method. Suppose further there are two interceptors registered. Suppose interceptor #1 calls skip(). Of the following two options, which describes what is documented to happen (I'm not interested in current behavior that happens to be the case as of this writing, which is option 1)?
The test method will not be executed, and interceptor #2 will not be called either. ("this invocation" means the execution of the test method and any interceptor processing that might occur "around" it.)
The test method will not be executed, but interception will continue with interceptor #2, which might end up calling proceed() or otherwise carrying on normally. ("this invocation" means only the execution of the test method, i.e. not also any interceptor processing that might occur "around" it.)
Is it a good practice to write a EXPECT(something) inside a test double (e.g. spy or mock) method? To ensure the test double is used in a specific way for testing?
If not, what would be a preferred solution?
If you would write a true Mock (as per definition from xUnit Test Patterns) this is exactly what defines this kind of test double. It is set up with the expectations how it will be called and therefore also includes the assertions. That's also how mocking frameworks produce mock objects under the hood. See also the definition from xUnit Test Patterns:
How do we implement Behavior Verification for indirect outputs of the SUT?
How can we verify logic independently when it depends on indirect inputs from other software components?
Replace an object the system under test (SUT) depends on with a test-specific object that verifies it is being used correctly by the SUT.
Here, indirect outputs means that you don't want to verify that the method under test returns some value but that there is something happening inside the method being tested that is behaviour relevant to callers of the method. For instance, that while executing some method the correct behaviour lead to an expected important action. Like sending an email or sending a message somewhere. The mock would be the doubled dependency that also verifies itself that this really happened, i.e. that the method under test really called the method of the dependency with the expected parameter(s).
A spy on the other hand shall just record things of interest that happened to the doubled dependency. Interrogating the spy about what happened (and sometimes also how often) and then judging if that was correct by asserting on the expected events is the responsibility of the test itself. So a mock is always also a spy with the addition of the assertion (expectation) logic. See also Uncle Bobs blog The Little Mocker for a great explanation of the different types of test doubles.
TL;DR
Yes, the mock includes the expectations (assertion) itself, the spy just records what happened and lets the test itself asks the spy and asserts on the expected events.
Mocking frameworks also implement mocks like explained above as they all follow the specified xunit framework.
mock.Verify(p => p.Send(It.IsAny<string>()));
If you look at the above Moq example (C#), you see that the mock object itself is configured to in the end perform the expected verification. The framework makes sure that the mock's verification methods are executed. A hand-written would be setup and than you would call the verification method on the mock object yourself.
Generally, you want to put all EXPECT statements inside individual tests to make your code readable.
If you want to enforce certain things on your test stub/spy, it is probably better to use exceptions or static asserts because your test is usually using them as a black box, and it uses them in an unintended way, your code will either not get compiled, or it will throw and give you the full stack trace which also will cause your test to fail (so you can catch the misuse).
For mocks, however, you have full control over the use and you can be very specific about how they are called and used inside each test. For example in Google test, using GMock matchers, you can say something like:
EXPECT_CALL(turtle, Forward(Ge(100)));
which means expect Forward to be called on the mock object turtle with a parameter equal or greater than 100. Any other value will cause the test to fail.
See this video for more examples on GMock matchers.
It is also very common to check general things in a test fixture (e.g. in Setup or TearDown). For example, this sample from google test enforces each test to finish in a certain amount of time, and the EXPECT statement is in teardown rather than each individual test.
Currently I have a gtest which has a gtest object with some member variables and functions.
I have a simple test, as well as more complex tests later on. If I comment out the complex tests, my simple test runs perfectly fine. However, when I include the other tests (even though I'm using gtest_filter to only run the first test), I start getting segfaults. I know it's impossible to debug without posting my code, but I guess I wanted to know more at a high level how this could occur. My understanding is that TEST_F constructs/destructs a new object every time it is run, so how could it be possible that the existence of a test affects another? Especially if I'm filtering, shouldn't the behavior be exactly the same?
TEST_F does not construct/destruct a new "object" ( at this point I assume that object here is to be interpreted as instance of the feature test class) for each test
What is done before each test of the test feature is to call the SetUp method and after each test the TearDown method is called.
Test feature constructor and destructor are called only once.
But because you did not provide a mvce , we can not assume further
There are quite a few written about stub vs mocks, but I can't see the real difference between fake and stub. Can anyone put some light on it?
I assume you are referring to the terminology as introduced by Meszaros. Martin Fowler does also mentions them regularly. I think he explains the difference pretty well in that article.
Nevertheless, I'll try again in my own words :)
A Fake is closer to a real-world implementation than a stub. Stubs contain basically hard-coded responses to an expected request; they are commonly used in unit tests, but they are incapable of handling input other than what was pre-programmed.
Fakes have a more real implementation, like some kind of state that may be kept for example. They can be useful for system tests as well as for unit testing purposes, but they aren't intended for production use because of some limitation or quality requirement.
A fake has the same behavior as the thing that it replaces.
A stub has a "fixed" set of "canned" responses that are specific to your test(s).
A mock has a set of expectations about calls that are made. If these expectations are not met, the test fails.
All of these are similar in that they replace production collaborators that code under test uses.
To paraphrase Roy Osherove in his book The Art of Unit Testing (second edition):
A Fake is any object made to imitate another object. Fakes can be used either as stubs or mocks.
A Stub is a fake that is provided to the class you are testing to satisfy its requirements, but is otherwise ignored in the unit test.
A Mock is a fake that is provided to the class you are testing, and will be inspected as part of the unit test to verify functionality.
For example, the MyClass class you are testing may utilize both a local logger and a third-party web service as part of its operation. You would create a FakeLogger and a FakeWebService, but how they are used determines whether they are stubs or mocks.
The FakeLogger might be used as a stub: it is provided to MyClass and pretends to be a logger, but actually ignores all input and is otherwise just there to get MyClass to operate normally. You don't actually check FakeLogger in your unit tests, and as far as you're concerned it's there to make the compiler shut up.
The FakeWebService might be used as a mock: you provide it to MyClass, and in one of your units tests you call MyClass.Foo() which is supposed to call the third party web service. To verify that this happened, you now check your FakeWebService to see if it recorded the call that it was supposed to receive.
Note that either of these could be reversed and depend on what it is you're testing in a particular unit test. If your unit test is testing the content of what is being logged then you could make a FakeLogger that dutifully records everything it's told so you can interrogate it during the unit test; this is now a mock. In the same test you might not care about when the third-party web service is called; your FakeWebService is now a stub. How you fill in the functions of your fake thus depends on whether it needs to be used as a stub or a mock or both.
In summary (direct quote from the book):
A fake is a generic term that can be used to describe either a stub or a mock object because they both look like the real object. . . . The basic difference is that stubs can't fail tests. Mocks can.
All the rest is implementation details.
These might help
http://xunitpatterns.com/Mocks,%20Fakes,%20Stubs%20and%20Dummies.html
http://hamletdarcy.blogspot.com/2007/10/mocks-and-stubs-arent-spies.html