Continuing another but similar question about testing (see here). I'll use a similare example (pseudocode)
class LinkDisplayer
method constructor(LinkStorage)
method displayLatestLinksByCategory(number_of_them)
class LinkStorage
method saveLink(Link)
method retrieveLatestLinksByCategory(category, number_of_them)
class Link
method getUrl()
method getDescription()
method getCategory()
So the linkDisplayer uses the LinkStorage to get Link(s). The behavior I want to test is 'shouldDisplayLatestLinks'. In my test, do I need to mock LinkStorage, and let it return mocked Link objects with mocked getUrl() etc behavior?
Testing 'leaf' classes is easy enough but I still find it very difficult to find my way in testing the other ones.
Short Answer:
You should be mocking/stubbing anything that is not under the direct control of your SUT. Only test the behavior of your SUT, and never write a test that attempts to confirm behavior that is outside that scope (ie. testing mocks/stubs).
Long Answer:
it can be difficult to see the forest for the trees. When you are the one who wrote all the code you can find it difficult sometimes to steer clear of testing too granular of an implementation detail.
You have the right idea with wanting to test behavior, and rightly so red flags went off when you started thinking about your test. This is exactly what TDD is all about, as it helps to expose design flaws. Just remember though, only test the behavior of the SUT. Everything else should be under the control of your unit test (mocks), otherwise it would not be a unit test.
Put yourself in the position of the consumer of the LinkDisplayer class, and ask yourself "How would I use this in production code?" You would probably just call the method and expect that it worked. You most certainly would not make a call to the database to ensure that it did in fact retrieve the correct number of elements, or that they were sorted by category would you? So why attempt to write a test for that.
What should the end result be of calling displayLatestLinksByCategory?
I see two possible answers to that question based on your sample, and what actions you should take based on those answers:
Get some stuff from the database using a specific method, and display it somewhere.
If this is what that method does, then those are the only two things you should be testing for. Mainly, that the right method was called on the data access component with the right arguments; and that the data returned got to where it needed to go.
What you should not be testing is the shape*** of the data returned as this would only be testing behavior that your test is already in complete control over. It would be like testing that a variable you just set to true in your test is in fact true.
Get some stuff from the database using a specific method and return it
At this point red flags should be going off all over the place. If this is the only thing that this method is good for, then you have to ask yourself why you would not just call the data access method directly? As the consumer of this class you already have control over which data access component is being used (you provided it) so why have a middle man.
* It is ok to test how your SUT responds to specifically shaped data. You might want to throw an exeption if null is returned, or the collection is empty. You might want to clip extra values from the list if more than N were returned, but you never want to test something that is outside the behavior of your SUT. Because you are in direct control over how the data being returned from you mocks is shaped, testing that would lead to the worst kind of test possible; a test that doesn't test anything, but still passes.
Related
I am trying to learn Propery-Based Testing(PBT)I think I know how to implement it but when should I apply PBT?
For example in this case I am trying to compare if the function getCurrentName() returns the expected name. Should I randomize this test?
#Test
public void getNameTest() {
assertEquals(nameProxy, proxyFoto.getCurrentName());
}
Your question is so generic that it cannot have a specific answer. I suggest you look at some of the stuff that has been written about how to come up with good properties, e.g. https://johanneslink.net/how-to-specify-it/
As for your concrete example, the answer if writing a property for the current name makes sense depends on a few things:
How does the name get into the proxy object? Is there a reasonable chance that depending on the shape/length/encoding etc of the name the behaviour is different?
What is the name being used for? Should it be normalised, formatted, shortened or processed in any specific way?
Properties and PBT are about finding and falsifying assumptions about the behaviour of your code under test. If there is nothing that you might get wrong, any form of automated testing can be considered unnecessary. If there are quite a few edge cases and paths that could show unexpected behaviour, then PBT looks like a worthwhile approach.
As a pragmatic recommendation: Start to translate some of your example tests into properties and see which ones are pulling their weight. Then try to add additional properties, eg by using ideas from the article I linked to.
Suppose I was writing a clone of the game 2048 (http://gabrielecirulli.github.io/2048/) and I want to write a test to verify that "the right thing" happens when the game is "won". Suppose that my game state is encapsulated in a class and that the state itself is private.
I suppose that I could write code to play the game, evaluate through the public interface when I'm about to win and then make the winning move; however, this seems like overkill. I would instead like to set a game state, make the winning move and verify that the object behaves as expected.
What is the recommended way of designing such a test? My current thought is that the test should either be a public member function of the class or that the test infrastructure should be friended by the class. Both of these seem distasteful.
Edit: In response to the first question: I'm assuming in this example that I don't have a method to set the game state and that there's no reason to write one; therefore it would be adding additional functionality just to write a test... ...one that then requires another member function to test, a get game state function. So then I'm writing at least two more public methods and test just to write this one test. Worse, these are methods that essentially break encapsulation such that if the internal details change I have to change these two methods and their tests for no other reason than to have a test. This seems more distasteful than friending a test function.
First, remember that Test-Driven Development is a design-oriented methodology. The primary goal of the tests is to influence the design of the SUT and its collaborators; everything else is just along for the ride.
Second, TDD emphasizes small steps. In his book, Test-Driven Development: By Example, Kent Beck says:
If you have to spend a hundred lines creating the objects for one single assertion, then something is wrong. Your objects are too big and need to be split. (p. 194)
This means you should listen to your intuition about writing the code necessary to win the game being overkill.
You also said:
I would instead like to set a game state, make the winning move and verify that the object behaves as expected.
Which is exactly what you should do.
Why? Because you're testing end-game scenarios. Most/all of the details that led to the end-game are irrelevant - you just want to make sure the program does "the right thing... when the game is won." As such, these are the only details that are relevant to your tests.
So what are these details that are relevant to your tests? To figure them out, it helps to discuss things with a colleague.
Q: How does the test configure the system to indicate the game has been won - without actually playing the game?
A: Tell something that the game has been won.
Q: What object would the test tell that the game has been won?
A: I don't know. But to keep things simple, let's say it's some object serving the role of "Referee".
By asking these questions, we've teased out some details of the design. Specifically, we've identified a role which can be represented in OOP by an interface.
What might this "Referee" role look like? Perhaps:
(pseudocode)
begin interface Referee
method GameHasBeenWon returns boolean
end interface
The presence of an interface establishes a seam in the design, which allows tests to use test-doubles in place of production objects. Not only that, it allows the implementation of this functionality to change (e.g., a rule change affecting how a game is determined to be "won") without having to modify any of the surrounding code.
This ties in directly with something else you mentioned:
I'm assuming in this example that I don't have a method to set the game state and that there's no reason to write one; therefore it would be adding additional functionality just to write a test...
A test is a consumer of your code. If it is difficult for a test to interact with your code, then it will be even more difficult for production code (having many more constraints) to interact with it. This is what is meant by "Listening to your tests".
Note that there are a lot of possible designs that can fall out of TDD. Every developer is going to have their own preferences which will influence the look and feel of the architecture. The main takeaway is that TDD helps break your program up into many small pieces, which is one of the core tenets of object oriented design.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 11 months ago.
Improve this question
Still a novice when it comes to writing unit-tests I often come across cases where I'm left scratching my head as to what is the right way to do things. Writing tests for a planned design I came across one of these dandruff-inducing instances. My design:
One ViewController sending the message to a dataFetcherClass based on the user's input. (The below code has been changed to protect the innocent).
-(void) userPushedLocalBusinessButtons{
[_businessDataFetcher fetchLocalData];
}
-(void) userPushedWorldwideBusinessButtons{
[_businessDataFetcher fetchWorldwideData];
}
The data format is identical for these actions, it's the location the dataFetcher should collect the data from that changes. So, in the BusinessDataFetcherClass I have these methods:
-(void) fetchLocalData{
_dataAddress = #"localData.json";
[self fetchData];
}
-(void) fetchWorldwideData{
_dataAddress = #"worldwideData.json";
[self fetchData];
}
The fetchData method fetch the data asynchronously and send a notification with the collected data when done. Now, I would like to write unit tests checking that the ivar _dataAddress has changed when fetchLocalData or fetchWorldwideData has been executed.
This is clearly not possible without altering the code. Some would say that this could easily be remedied by making _dataAddress into a public property, and that's one solution. Another would be to create a method returning the value of the _dataAddress ivar. I am not entirely happy with either alternative as they in both cases force me to change the code just for the tests, rather than improve the overall quality of the actual code-base itself.
I landed on the second alternative and included a method -(NSString *) dataAddress; My question (as stated in the headline) is whether this is OK? Is my design the problem? Obviously the number one goal of TDD is to avoid regression, but I believe improving the overall code-quality is also an important goal. Is adding the occasional fluff to be expected?
I would like to write unit tests checking that the ivar _dataAddress
has changed when fetchLocalData or fetchWorldwideData has been
executed.
When you write a unit test, it should test the external behavior of a class. This is an implementation detail of the class. If you want to change the way fetching data works, the class might still work while the unit tests fail. This makes your unit test annoying, not helpful.
fetch the data asynchronously and send a notification with the
collected data when done.
It sounds like this is the external behavior of those methods in that class. This is what you should write your test to check. I don't know objective-c, so here's a psuedo-code example:
setup expected local data (preferably with a mock)
call fetchLocalData on BusinessDataFetcherClass
wait a little bit
check that local data is populated on ViewController
Is my design the problem?
Your design here does make writing tests a little harder, though it isn't a big problem. In particular, that "wait" which needs to happen in the test. The design problem that your tests are pointing out is that your class has at least two responsibilities: fetching data and managing asynchrony. If you split those responsibilities apart, they would each be easier to test.
Obviously the number one goal of TDD is to
avoid regression, but I believe improving the overall code-quality is
also an important goal. Is adding the occasional fluff to be expected?
I don't think you probably need more fluff in this case, but it does happen with unit tests sometimes. When you write code with tests, you end up with two clients of your code: the test code and the production code. It's the need to satisfy two client in different contexts that forces some of this "fluff" in, or forces some design changes. The good news is that when you have a design that can easily satisfy two client, you will probably be able to satisfy a third and a fourth fairly easily, if the need should arise. To me, this effect is one of the most important benefits to TDD.
You don't want to test the internal state of your class -- this makes no sense. The only thing you care about is what your class is doing in its interactions with the outside world (whether info is going inwards or outwards or both ways).
To put it another way: once you've written a test for your class, rewriting the implementation (internals) of your class while maintaining its visible behaviour should not break your tests. If it does, your tests are broken IMO.
A good way of testing the behaviour of your class is to use Mock objects -- for example, see OCMock for iOS.
Mock objects allow you to test the behaviour of your target class. In order to do this, you need to write your target class in a certain way: in your example, you need to be able to pass in a network provider class, rather than have your class go off and use a certain provider which is hardcoded (re-usable components should never configure themselves, but be configured). Once you've set things up this way, your unit test class can pass in a mock network service provider which checks that the correct URL is being hit.
Mock objects might seem convoluted at first glance, but you're testing the correct thing -- the behaviour of your target class -- without polluting it with any special testing method etc.
Note also that making your code amenable to testing is also making it more amenable to re-use: your test cases become a second 'user' of your code.
I also am not an ObjectiveC developer, but I think that the reason you're posting this is because you're listening to your code, and your code's telling you that something isn't quite right.
I would ask what you're doing with the results of the fetchData call? I suspect you're rendering the data somewhere. If iOS is rendering it, then there's probably a callback somewhere that you can assert rather than asserting the instance variable. If you're updating the UI from within the class, it will make it easier to test if you introduce an Observer to decouple your UI and your code that's fetching the data. You can then have your test register as the receiver and assert your state change there.
Hope that helps!
Brandon
This is most certainly a language agnostic question and one that has bothered me for quite some time now. An example will probably help me explain the dilemma I am facing:
Let us say we have a method which is responsible for reading a file, populating a collection with some objects (which store information from the file), and then returning the collection...something like the following:
public List<SomeObject> loadConfiguration(String filename);
Let us also say that at the time of implementing this method, it would seem infeasible for the application to continue if the collection returned was empty (a size of 0). Now, the question is, should this validation (checking for an empty collection and perhaps the subsequent throwing of an exception) be done within the method? Or, should this methods sole responsibility be to perform the load of the file and ignore the task of validation, allowing validation to be done at some later stage outside of the method?
I guess the general question is: is it better to decouple the validation from the actual task being performed by a method? Will this make things, in general, easier at a later stage to change or build upon - in the case of my example above, it may be the case at a later stage where a different strategy is added to recover from the event of an empty collection being return from the 'loadConfiguration' method..... this would be difficult if the validation (and resulting exception) was being done in the method.
Perhaps I am being overly pedantic in the quest for some dogmatic answer, where instead it simply just relies on the context in which a method is being used. Anyhow, I would be very interested in seeing what others have to say regarding this.
Thanks all!
My recommendation is to stick to the single responsibility principle which says, in a nutshell, that each object should have 1 purpose. In this instance, your method has 3 purposes and then 4 if you count the validation aspect.
Here's my recommendation on how to handle this and how to provide a large amount of flexibility for future updates.
Keep your LoadConfig method
Have it call the a new method for reading the file.
Pass the previous method's return value to another method for loading the file into the collection.
Pass the object collection into some validation method.
Return the collection.
That's taking 1 method initially and breaking it into 4 with one calling 3 others. This should allow you to change pieces w/o having any impact on others.
Hope this helps
I guess the general question is: is it
better to decouple the validation from
the actual task being performed by a
method?
Yes. (At least if you really insist on answering such a general question – it’s always quite easy to find a counter-example.) If you keep both the parts of the solution separate, you can exchange, drop or reuse any of them. That’s a clear plus. Of course you must be careful not to jeopardize your object’s invariants by exposing the non-validating API, but I think you are aware of that. You’ll have to do some little extra typing, but that won’t hurt you.
I will answer your question by a question: do you want various validation methods for the product of your method ?
This is the same as the 'constructor' issue: is it better to raise an exception during the construction or initialize a void object and then call an 'init' method... you are sure to raise a debate here!
In general, I would recommend performing the validation as soon as possible: this is known as the Fail Fast which advocates that finding problems as soon as possible is better than delaying the detection since diagnosis is immediate while later you would have to revert the whole flow....
If you're not convinced, think of it this way: do you really want to write 3 lines every time you load a file ? (load, parse, validate) Well, that violates the DRY principle.
So, go agile there:
write your method with validation: it is responsible for loading a valid configuration (1)
if you ever need some parametrization, add it then (like a 'check' parameter, with a default value which preserves the old behavior of course)
(1) Of course, I don't advocate a single method to do all this at once... it's an organization matter: under the covers this method should call dedicated methods to organize the code :)
To deflect the question to a more basic one, each method should do as little as possible. So in your example, there should be a method that reads in the file, a method that extracts the necessary data from the file, another method to write that data to the collection, and another method that calls these methods. The validation can go in a separate method, or in one of the others, depending on where it makes the most sense.
private byte[] ReadFile(string fileSpec)
{
// code to read in file, and return contents
}
private FileData GetFileData(string fileContents)
{
// code to create FileData struct from file contents
}
private void FileDataCollection: Collection<FileData> { }
public void DoItAll (string fileSpec, FileDataCollection filDtaCol)
{
filDtaCol.Add(GetFileData(ReadFile(fileSpec)));
}
Add validation, verification to each of the methods as appropriate
You are designing an API and should not make any unnecessary assumptions about your client. A method should take only the information that it needs, return only the information requested, and only fail when it is unable to return a meaningful value.
So, with that in mind, if the configuration is loadable but empty, then returning an empty list seems correct to me. If your client has an application specific requirement to fail when provided an empty list, then it may do so, but future clients may not have that requirement. The loadConfiguration method itself should fail when it really fails, such as when it is unable to read or parse the file.
But you can continue to decouple your interface. For example, why must the configuration be stored in a file? Why can't I provide a URL, a row in a database, or a raw string containing the configuration data? Very few methods should take a file path as an argument since it binds them tightly to the local file system and makes them responsible for opening, reading, and closing files in addition to their core logic. Consider accepting an input stream as an alternative. Or if you want to allow for elaborate alternatives -- like data from a database -- consider accepting a ConfigurationReader interface or similar.
Methods should be highly cohesive ... that is single minded. So my opinion would be to separate the responsibilities as you have described. I sometimes feel tempted to say...it is just a short method so it does not matter...then I regret it 1.5 weeks later.
I think this depends on the case: If you could think of a scenario where you would use this method and it returned an empty list, and this would be okay, then I would not put the validation inside the method. But for e.g. a method which inserts data into a database which have to be validated (is the email address correct, has a name been specified, ... ) then it should be ok to put validation code inside the function and throw an exception.
Another alternative, not mentioned above, is to support Dependency Injection and have the method client inject a validator. This would allow the preservation of the "strong" Resource Acquisition Is Initialization principle, that is to say Any Object which Loads Successfully is Ready For Business (Matthieu's mention of Fail Fast is much the same notion).
It also allows a resource implementation class to create its own low-level validators which rely on the structure of the resource without exposing clients to implementation details unnecessarily, which can be useful when dealing with multiple disparate resource providers such as Ryan listed.
I just read the Wikipedia article on mock objects, but I'm still not entirely clear on their purpose. It appears they are objects that are created by a test framework when the actual object would be too complex or unpredictable (you know 100% sure what the values of the mock object are because you fully control them).
However, I was under the impression that all testing is done with objects of known values, so I must be missing something. For example, in a course project, we were tasked with a calendar application. Our test suite consisted of event objects that we knew exactly what they were so we could test the interactions between multiple event objects, various subsystems, and the user interface. I'm guessing these are mock objects, but I don't know why you wouldn't do this because without the objects of known values, you can't test a system.
A mock object is not just an object with known values. It is an object that has the same interface as a complex object that you cannot use in test (like a database connection and result sets), but with an implementation that you can control in your test.
There are mocking frameworks that allow you to create these objects on the fly and in essence allow you to say something like: Make me an object with a method foo that takes an int and returns a bool. When I pass 0, it should return true. Then you can test the code that uses foo(), to make sure it reacts appropriately.
Martin Fowler has a great article on mocking:
http://martinfowler.com/articles/mocksArentStubs.html
Think of the classic case of having client and server software. To test the client, you need the server; to test the server, you need the client. This makes unit testing pretty much impossible - without using mocks. If you mock the server, you can test the client in isolation and vice versa.
The point of the mock is not to duplicate the behaviour of the things its mocking though. It is more to act as a simple state machine whose state changes can be analysed by the test framework. So a client mock might generate test data, send it to the server and then analyse the response. You expect a certain response to a specific request, and so you can test if you get it.
I agree with everything #Lou Franco says and you should definitely read the excellent Martin Fowler article on test doubles that #Lou Franco points you to.
The main purpose of any test double (fake, stub or mock) is to isolate the object under test so that your unit test is only testing that object (not its dependencies and the other types it collaborates or interacts with).
An object that provides the interface that your object is dependent on can be used in place of the actual dependency so that expectations can be placed that certain interactions will occur. This can be useful but there is some controversy around state-based vs. interaction-based testing. Overuse of mock expectation will lead to brittle tests.
A further reason for test doubles is to remove dependencies on databases or file systems or other types that are expensive to set up or perform time consuming operations. This means you can keep the time required to unit test the object you're interested in to a minimum.
Here's an example: if you're writing code that populates a database you may want to check if a particular method has added data to the database.
Setting up a copy of the database for testing has the problem that if you assume there are no records before the call to the tested method and one record after, then you need to roll back the database to a previous state, thus adding to the overhead for running the test.
If you assume there is only one more record than before, it may clash with a second tester (or even a second test in the same code) connecting to the same database, thus causing dependencies and making the tests fragile.
The mock allows you to keep the tests independent of each other and easy to set up.
This is just one example - I'm sure others can supply more.
I agree 100% with the other contributors on this topic, especially with the recommendation for the Martin Fowler article.
You might be interested in our book, see http://www.growing-object-oriented-software.com/. It's in Java, but the ideas still apply.