Run specific Spock Tests without setup? - testing

In the documentation Spock mentions these function in the section 'Fixture Methods':
def setup() {} // run before every feature method
def cleanup() {} // run after every feature method
def setupSpec() {} // run before the first feature method
def cleanupSpec() {} // run after the last feature method
I am using the setup function to initialize new objects for all tests. However some tests are supposed to test what happens when there are no other objects yet.
Because of the setup these tests fail now.
Is there any way to suspend the setup for certain functions? An annotation perhaps?
Or do I have to create a separate "setup" function and call them in each test I do. The majority of tests uses it!

You can always override value inside your test method only for that specific method. Take a look at following example:
import spock.lang.Shared
import spock.lang.Specification
class SpockDemoSpec extends Specification {
String a
String b
#Shared
String c
def setup() {
println "This method is executed before each specification"
a = "A value"
b = "B value"
}
def setupSpec() {
println "This method is executed only one time before all other specifications"
c = "C value"
}
def "A empty"() {
setup:
a = ""
when:
def result = doSomething(a)
then:
result == ""
}
def "A"() {
when:
def result = doSomething(a)
then:
result == "A VALUE"
}
def "A NullPointerException"() {
setup:
a = null
when:
def result = doSomething(a)
then:
thrown NullPointerException
}
def "C"() {
when:
def result = doSomething(c)
then:
result == 'C VALUE'
}
private String doSomething(String str) {
return str.toUpperCase()
}
}
In this example a and b are set up before each test and c is set up only once before any specification is executed (that's why it requires #Shared annotation to keep it's value between executions).
When we run this test we will see following output in the console:
This method is executed only one time before all other specifications
This method is executed before each specification
This method is executed before each specification
This method is executed before each specification
This method is executed before each specification
The general rule of thumb is that you shouldn't change c value after it is set up in setupSpec method - mostly because you don't know if which order tests are going to be executed (in case you need to guarantee the order you can annotate your class with #Stepwise annotation - Spock will run all cases in order they are defined in this case). In case of a and b you can override then at the single method level - they will get reinitialized before another test case is executed. Hope it helps.

Related

Junit - how to get test result as string?

I am trying to test the code and get a result as String to send it to API later.
class FirstClass{
fun main(){
print("Hello world!")
}
}
Test:
ort org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import java.io.ByteArrayOutputStream
import java.io.PrintStream
import kotlin.test.assertEquals
internal class FirstClassTest {
private val outContent = ByteArrayOutputStream()
private val errContent = ByteArrayOutputStream()
private val originalOut = System.out
private val originalErr = System.err
#BeforeEach
fun setUp() {
System.setOut(PrintStream(outContent))
System.setErr(PrintStream(errContent))
}
#AfterEach
fun tearDown() {
System.setOut(originalOut)
System.setErr(originalErr)
}
#Test
fun main() {
val SUT = FirstClass()
SUT.main()
val testResult = assertEquals("Hello world!", outContent.toString())
print("Test result: $testResult")
val api = Api()
val apiResult = api.sendResult(testResult.toString())
print("Api result: $apiResult")
}
}
The test is passing, however, I do not see printed messages. How to get a test result as String?
There are several issues here.  The main one is:
The redirection affects your test method too.
Because you've redirected System.out, the print() in your test method goes to outContent, along with the output from FirstClass.main() that you want to test, instead of to the screen or wherever you want it.
I can see two fixes for this.
The quick one is for your test method to output to originalOut:
originalOut.print("Test result: $testResult")
Your test method is in the class which does the redirection, so there's no problem with it knowing about the redirection, and it already has access to originalOut.
However, if you can, I think a better solution is to refactor FirstClass so that it doesn't hard-code the stream it writes to.  For example, the stream could be passed as a parameter; or it could return the string directly (and the caller, in a thin non-tested wrapper, could write it to System.out).
That would be more work, but would make your code more flexible as well as easier to test.
Other issues include:
You're using print() instead of println().
Many streams are line-buffered, writing their output only after a newline, and so you might not see any results if there isn't one.  (And even if you do, all the results would be jammed on a single line!)
You assign the result of assertEquals().
assertEquals() doesn't have a useful return value.  (It return Unit.)  So your code will simply show:
Test result: kotlin.Unit
Instead, like all the assert functions, it throws an exception if the assertion fails.  So there's no point in storing or processing the return value; simply calling the assertion is enough.
— This means that there's usually no need to call print()/println() from your test method anyway!  If there's a failure, it'll be obvious: running from the command line will stop with an exception message and stack trace; IntelliJ shows a big red mark next to that test; Maven and Gradle will stop the build (after all tests have run), showing the number of failures.  So if everything continues smoothly, you know the tests have passed.
Api is not defined.
The code you posted above won't compile, because it doesn't include a definition or import for Api.  (Those last lines can be removed, though, without affecting the question.)
main() is a confusing name for a test.
The unit testing framework will find and run all test methods annotated with #Test.  A test class will often contain many different test methods, and it's usual to name them after the aspect they're testing.  (That makes any failures clearer.)  Calling it main() not only fails to describe what's being tested, but also suggests that the method will be run from outside the testing framework, which would probably not behave properly.

Difference between Mock / Stub / Spy in Spock test framework

I don't understand the difference between Mock, Stub, and Spy in Spock testing and the tutorials I have been looking at online don't explain them in detail.
Attention: I am going to oversimplify and maybe even slightly falsify in the upcoming paragraphs. For more detailed info see Martin Fowler's website.
A mock is a dummy class replacing a real one, returning something like null or 0 for each method call. You use a mock if you need a dummy instance of a complex class which would otherwise use external resources like network connections, files or databases or maybe use dozens of other objects. The advantage of mocks is that you can isolate the class under test from the rest of the system.
A stub is also a dummy class providing some more specific, prepared or pre-recorded, replayed results to certain requests under test. You could say a stub is a fancy mock. In Spock you will often read about stub methods.
A spy is kind of a hybrid between real object and stub, i.e. it is basically the real object with some (not all) methods shadowed by stub methods. Non-stubbed methods are just routed through to the original object. This way you can have original behaviour for "cheap" or trivial methods and fake behaviour for "expensive" or complex methods.
Update 2017-02-06: Actually user mikhail's answer is more specific to Spock than my original one above. So within the scope of Spock, what he describes is correct, but that does not falsify my general answer:
A stub is concerned with simulating specific behaviour. In Spock this is all a stub can do, so it is kind of the simplest thing.
A mock is concerned with standing in for a (possibly expensive) real object, providing no-op answers for all method calls. In this regard, a mock is simpler than a stub. But in Spock, a mock can also stub method results, i.e. be both a mock and a stub. Furthermore, in Spock we can count how often specific mock methods with certain parameters have been called during a test.
A spy always wraps a real object and by default routes all method calls to the original object, also passing through the original results. Method call counting also works for spies. In Spock, a spy can also modify the behaviour of the original object, manipulating method call parameters and/or results or blocking the original methods from being called at all.
Now here is an executable example test, demonstrating what is possible and what is not. It is a bit more instructive than mikhail's snippets. Many thanks to him for inspiring me to improve my own answer! :-)
package de.scrum_master.stackoverflow
import org.spockframework.mock.TooFewInvocationsError
import org.spockframework.runtime.InvalidSpecException
import spock.lang.FailsWith
import spock.lang.Specification
class MockStubSpyTest extends Specification {
static class Publisher {
List<Subscriber> subscribers = new ArrayList<>()
void addSubscriber(Subscriber subscriber) {
subscribers.add(subscriber)
}
void send(String message) {
for (Subscriber subscriber : subscribers)
subscriber.receive(message);
}
}
static interface Subscriber {
String receive(String message)
}
static class MySubscriber implements Subscriber {
#Override
String receive(String message) {
if (message ==~ /[A-Za-z ]+/)
return "ok"
return "uh-oh"
}
}
Subscriber realSubscriber1 = new MySubscriber()
Subscriber realSubscriber2 = new MySubscriber()
Publisher publisher = new Publisher(subscribers: [realSubscriber1, realSubscriber2])
def "Real objects can be tested normally"() {
expect:
realSubscriber1.receive("Hello subscribers") == "ok"
realSubscriber1.receive("Anyone there?") == "uh-oh"
}
#FailsWith(TooFewInvocationsError)
def "Real objects cannot have interactions"() {
when:
publisher.send("Hello subscribers")
publisher.send("Anyone there?")
then:
2 * realSubscriber1.receive(_)
}
def "Stubs can simulate behaviour"() {
given:
def stubSubscriber = Stub(Subscriber) {
receive(_) >>> ["hey", "ho"]
}
expect:
stubSubscriber.receive("Hello subscribers") == "hey"
stubSubscriber.receive("Anyone there?") == "ho"
stubSubscriber.receive("What else?") == "ho"
}
#FailsWith(InvalidSpecException)
def "Stubs cannot have interactions"() {
given: "stubbed subscriber registered with publisher"
def stubSubscriber = Stub(Subscriber) {
receive(_) >> "hey"
}
publisher.addSubscriber(stubSubscriber)
when:
publisher.send("Hello subscribers")
publisher.send("Anyone there?")
then:
2 * stubSubscriber.receive(_)
}
def "Mocks can simulate behaviour and have interactions"() {
given:
def mockSubscriber = Mock(Subscriber) {
3 * receive(_) >>> ["hey", "ho"]
}
publisher.addSubscriber(mockSubscriber)
when:
publisher.send("Hello subscribers")
publisher.send("Anyone there?")
then: "check interactions"
1 * mockSubscriber.receive("Hello subscribers")
1 * mockSubscriber.receive("Anyone there?")
and: "check behaviour exactly 3 times"
mockSubscriber.receive("foo") == "hey"
mockSubscriber.receive("bar") == "ho"
mockSubscriber.receive("zot") == "ho"
}
def "Spies can have interactions"() {
given:
def spySubscriber = Spy(MySubscriber)
publisher.addSubscriber(spySubscriber)
when:
publisher.send("Hello subscribers")
publisher.send("Anyone there?")
then: "check interactions"
1 * spySubscriber.receive("Hello subscribers")
1 * spySubscriber.receive("Anyone there?")
and: "check behaviour for real object (a spy is not a mock!)"
spySubscriber.receive("Hello subscribers") == "ok"
spySubscriber.receive("Anyone there?") == "uh-oh"
}
def "Spies can modify behaviour and have interactions"() {
given:
def spyPublisher = Spy(Publisher) {
send(_) >> { String message -> callRealMethodWithArgs("#" + message) }
}
def mockSubscriber = Mock(MySubscriber)
spyPublisher.addSubscriber(mockSubscriber)
when:
spyPublisher.send("Hello subscribers")
spyPublisher.send("Anyone there?")
then: "check interactions"
1 * mockSubscriber.receive("#Hello subscribers")
1 * mockSubscriber.receive("#Anyone there?")
}
}
Try it in the Groovy Web Console.
The question was in the context of the Spock framework and I don't believe the current answers take this into account.
Based on Spock docs (examples customized, my own wording added):
Stub: Used to make collaborators respond to method calls in a certain way. When stubbing a method, you don’t care if and how many times the method is going to be called; you just want it to return some value, or perform some side effect, whenever it gets called.
subscriber.receive(_) >> "ok" // subscriber is a Stub()
Mock: Used to describe interactions between the object under specification and its collaborators.
def "should send message to subscriber"() {
when:
publisher.send("hello")
then:
1 * subscriber.receive("hello") // subscriber is a Mock()
}
A Mock can act as a Mock and a Stub:
1 * subscriber.receive("message1") >> "ok" // subscriber is a Mock()
Spy: Is always based on a real object with original methods that do real things. Can be used like a Stub to change return values of select methods. Can be used like a Mock to describe interactions.
def subscriber = Spy(SubscriberImpl, constructorArgs: ["Fred"])
def "should send message to subscriber"() {
when:
publisher.send("hello")
then:
1 * subscriber.receive("message1") >> "ok" // subscriber is a Spy(), used as a Mock an Stub
}
def "should send message to subscriber (actually handle 'receive')"() {
when:
publisher.send("hello")
then:
1 * subscriber.receive("message1") // subscriber is a Spy(), used as a Mock, uses real 'receive' function
}
Summary:
A Stub() is a Stub.
A Mock() is a Stub and Mock.
A Spy() is a Stub, Mock and Spy.
Avoid using Mock() if Stub() is sufficient.
Avoid using Spy() if you can, having to do so could be a smell and hints at incorrect test or incorrect design of object under test.
In simple terms:
Mock: You mock a type and on the fly you get an object created. Methods in this mock object returns the default values of return type.
Stub: You create a stub class where methods are redefined with definition as per your requirement. Ex: In real object method you call and external api and return the username against and id. In stubbed object method you return some dummy name.
Spy: You create one real object and then you spy it. Now you can mock some methods and chose not to do so for some.
One usage difference is you can not mock method level objects. whereas you can create a default object in method and then spy on it to get the desired behavior of methods in spied object.
Stubs are really only to facilitate the unit test, they are not part of the test. Mocks, are part of the test, part of the verification, part of the pass / fail.
So, say you have a method that takes in a object as a parameter. You never do anything which changes this parameter in the test. You simply read a value from it. That's a stub.
If you change anything, or need to verify some sort of interaction with the object, then it it is a mock.

Spock MissingMethodException

I have something that looks like the similar specification:
def "my spec"(Record record) {
given:
Something something = getSomething()
and:
otherThing = getOtherThing()
doFlow(something, record)
if (record.someType = Types.SOME_SPECIFIC_TYPE) {
doFlow(something, record)
}
}
def doFlow(Something something, Record record) {
when:
//code
then:
//asserts
when:
//code
and:
//mode code
then:
//code
}
However, at runtime, I get: groovy.lang.MissingMethodException: No signature of method doFlow() is applicable for arguments Something, Record values: [given values].
Both "my flow" and "doFlow" are feature methods as they have blocks such as given, when, and then. It's Spock's responsibility to invoke feature methods, and one feature method cannot call another one. If doFlow is meant to be a helper method, it should use explicit assert statements, and shouldn't have any blocks.
PS: Feature methods can't declare method parameters unless they are data-driven (i.e. have a where block).
PPS: A feature method can't just have a given/and block. (You'll get a compile error for this.)

Execute some action when Spock test fails

I'd like to execute some action when Spock test fails. Specifically, take a screenshot. Is it possible? How to do it?
Create a listener class
class ExampleListener extends AbstractRunListener {
def void error(ErrorInfo error) {
println "Actual on error logic"
}
}
then add it to each specification using implementation of IGlobalExtension that is executed for each Spec
class GlobalSpecExtension implements IGlobalExtension {
#Override
void visitSpec(SpecInfo specInfo) {
specInfo.addListener(new ExampleListener())
}
}
and finally create file named org.spockframework.runtime.extension.IGlobalExtension in your META-INF/services directory (typically it will be under src/test/resources if you are using Maven) with the full name of your IGlobalExtension implementation e.g.
com.example.tests.GlobalSpecExtension
The best way to achieve this is to write a (global or annotation-driven) Spock extension that implements and registers an AbstractRunListener. For an example, see OptimizeRunOrderExtension. For how to register a global extension, see the IGlobalExtension descriptor.
There isn't much documentation on extensions because the APIs are still subject to change. If you want to play it safe (and can live with some restrictions), you can implement a JUnit Rule instead.
One problem that you may encounter in both cases is that they don't provide access to the current spec instance. If you need this, you may have to use both an AbstractRunListener (to be notified of the failure) and an IMethodInterceptor (to get hold of the spec instance), both registered by the same extension. (Shouldn't be this hard, but that's what's currently there.)
I've managed to do it this way:
class ExampleTest extends GebSpec{
static boolean success = false
def setup(){
success = false
}
def cleanup(){
assert success == true, someAction()
}
def someAction(){
}
def "TestCase"(){
expect:
/*What you expect here*/
(success = true) != null
}
}
Before each test case "success" is set to false by the setup() method.
At the end of each test case you add the "(success = true) != null" statement. Therefore "success" will only be true if the test case has passed.
After each test case the cleanup() method will verify if "success" is true. If it isn't the method someAction() will be called.
I can't upvote or comment on user3074543's answer, but it's simpler than creating an extension. I want easy. So I shortened user*'s a little (I don't mean the 1-line methods). You can make the logic simpler by recording failure instead of success, and reduce typing with a done() helper.
class Test extends spock.lang.Specification {
def fail
def setup(){ fail = true }
def done(){ !(fail = false) }
def cleanup(){ fail && doStuffWhenFail() }
def 'test things'(){
expect:
stuff
done()
}
}

Grails integration test failing with MissingMethodException

I'm attempting to test a typical controller flow for user login. There are extended relations, as with most login systems, and the Grails documentation is completely useless. It doesn't have a single example that is actually real-world relevant for typical usage and is a feature complete example.
my test looks like this:
#TestFor(UserController)
class UserControllerTests extends GroovyTestCase {
void testLogin() {
params.login = [email: "test1#example.com", password: "123"]
controller.login()
assert "/user/main" == response.redirectUrl
}
}
The controller does:
def login() {
if (!params.login) {
return
}
println("Email " + params.login.email)
Person p = Person.findByEmail(params?.login?.email)
...
}
which fails with:
groovy.lang.MissingMethodException: No signature of method: immigration.Person.methodMissing() is applicable for argument types: () values: []
The correct data is shown in the println, but the method fails to be called.
The test suite cannot use mocks overall because there are database triggers that get called, the result of which will need to be tested.
The bizarre thing is that if I call the method directly in the test, it works, but calling it in the controller doesn't.
For an update: I regenerated the test directly from the grails command, and it added the #Test annotation:
#TestFor(UserController)
class UserControllerTests extends GroovyTestCase {
#Test
void testLogin() {
params.login = [email: "test1#example.com", password: "123"]
Person.findByEmail(params.login.email)
controller.login()
}
}
This works if I run it with
grail test-app -integration UserController
though the result isn't populated correctly - the response is empty, flash.message is null even though it should have a value, redirectedUrl is null, content body is empty, and so is view.
If I remove the #TestFor annotation, it doesn't work even in the slightest. It fails telling me that 'params' doesn't exist.
In another test, I have two methods. The first method runs, finds Person.findAllByEmail(), then the second method runs and can't find Person.findAllByEmail and crashes with a similar error - method missing.
In another weird update - it looks like the response object is sending back a redirect, but to the application baseUrl, not to the user controller at all.
Integration tests shouldn't use #TestFor. You need to create an instance of the controller in your test, and set params in that:
class UserControllerTests extends GroovyTestCase {
void testLogin() {
def controller = new UserController()
controller.params.login = [email:'test1#example.com', password:'123']
controller.login()
assert "/user/main" == controller.response.redirectedUrl
}
}
Details are in the user guide.
The TestFor annotation is used only in unit tests, since this mocks the structure. In the integration tests you have access of the full Grails environment, so there's no need for this mocks. So just remove the annotation and should work.
class UserControllerTests extends GroovyTestCase {
...
}