Related
I've started to use the #DisplayName annotation from Jupiter API (Junit 5) to describe what's in going on my tests. This feature is very usefull to help other developers to better understand what the tests are intended to do (because your texts can contain spaces, special characters, and even emojis).
For now, I'm using the following strategy to create my descriptions:
Class level: "Check [the general feature being tested]"
Method level: "When [condition to be meet]"
However, I'm wondering if there are betters ways to describe the tests. So my question is, what are the good patterns to be followed when using #DisplayName annotation to improve my tests descriptions?
I'm looking for things like:
Keywords that help to categorize tests by their testing objectives
Emojis to indicate features, importance, etc...
I have a task to write tests for future Django Channels+DRF project, don't ask why (we only have swagger documentation for now). So the tests have to test the user use cases (like scenario that may be complex). I have researched about that and found BDD. Here is the question, considering that our project later may have simple unit tests too what should I use, i.e. BDD seems decent but I think it may be excessive for use and may be there is a way of just writing unittests for user use case scenarious and I can get by with that. Does anyone have experience with that? It would be great if you provide articles and code examples.
Scenarios are a bit different to use-cases. A use-case often covers several capabilities. For instance, in the simple laundry use-case shown here, a housekeeper does several things when performing a wash:
washes each load
dries each load.
folds certain items
irons some items
All of these go into the "weekly laundry" use-case.
A scenario in BDD is much more fine-grained. It describes one capability taking place in a particular context or set of contexts. So for instance you might have:
Given the weekly laundry has been washed and dried
And it contains several sheets
And some underpants
When the housekeeper does the folding
Then the sheets should be folded
But the underpants should not.
You can see that we've skipped a couple of the capabilities. This scenario is focused on the capability of folding, and shows how a well-behaved housekeeper would do it. Washing and drying would have to be covered in separate scenarios.
So that's the difference between a use-case and a scenario. Now let's look at a unit test.
When we write code, we don't write it all in one big class or function. We split it up into small pieces. In the same way that a scenario describes an example of the behaviour of the system from the perspective of the users, a unit test describes the behaviour of a class or other small piece of code from the perspective of its users - usually other classes!
So let's imagine that we're on a car purchasing site. We have several capabilities:
Authentication
Searching for cars
Purchasing a car
Listing a car
Removing a car from the list
Each of these will have lots of different classes making it up. Even searching for a car could involve a front-end, a search component, a database of cars, a persistence layer, a webserver, etc.. For each piece of code, we describe the behaviour of that code.
(BDD actually started out at this level; with examples of how classes behave - JBehave was intended to replace JUnit. But JUnit got better and we didn't need this bit any more. I still find it helpful to think of these as examples rather than tests.)
Typically I'll have both scenarios and unit tests in my codebase; one set of them looking from a user / stakeholder perspective at the whole system, and the other set describing my classes in finer detail.
The scenarios help me show how the system behaves and why it's valuable. The unit tests help me drive out good design and separate responsibilities. Both of them provide living documentation which helps to keep the system maintainable and make it easier for newcomers to come on board.
Generally this is how I program:
I have a rough idea of what I want to achieve
I talk to someone about it and write down some scenarios
If we don't quite know what we're looking for, I'll get something working (a spike)
Once we understand better what we're looking for, I automate the scenario first
I take the simplest case and start writing the UI
When the UI needs another class to work, I write some examples of how that code should work (unit tests) first
Then I write the code (or refactor it, because spikes are messy)
When that code needs another class to work, I write some examples of it
If I don't have code that's needed at any point in my unit tests, I use mocks.
Generally we keep the scenarios and the unit tests in different places.
You can see some examples of how I've done this here. It's a tetris game with scenarios which automate the whole game through the UI, and lower-level unit tests which describe the behaviour of particular pieces like the heartbeat which drops the shapes.
Having said that - if your codebase is very simple, you can probably get away with just the scenarios or just the unit tests; you might not need both. But if it starts getting more complex, consider refactoring and adding whatever you need. It's OK to be pragmatic about it, as long as it's easy to change.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
In the context of Agile software development, what's the difference between User Acceptance Test (UAT), Test Case Scenario and Functional Test?
The members of the team I am part of, they consider the three things as different, but I see them as exactly the same thing.
In fact, all of them are designed having the end user in mind
There's a lot of different sorts of testing. Many of them overlap. Many use the same tools. Many are specializations of other more general terms. Often they blur together. People argue about the terminology all the time.
You're correct that they all have the end user in mind, but they are different.
User Acceptance Test
This is a specific form of an acceptance test where a subject-matter expert, ideally the client or their representative, tests the software. This is in addition to functional and acceptance testing done by QA. It's designed to simulate, as closely as possible, an actual end-user using the software; the tester is asked to perform a bunch of common tasks with the new system, but not given specific instructions nor coaching on how to do it.
For example, if you were creating a site for an airline, they might be asked to register, login, book a flight, make a payment, check in, check their flight status, and so on.
Functional Test
This is blackbox testing done by the QA role. It verifies the thing does what it's supposed to do; you give it inputs, you check the outputs. Typically this is testing against the specification and/or requirements document.
"Functional" here doesn't refer to code functions, but that the system functions as expected. Testing specific functions is unit testing.
They can be purely functional, "when I do X I get Y". They can be about resource use, "when I do X it uses no more than Y memory/time". Or about error checking, "when I give it garbage I get a well formed error". Anything that validates it meets the requirements.
Test Case Scenario
Sounds like Scenario Testing: this uses stories, similar to user stories, that help a tester work through a complicated testing scenario. Scenario testing tests complicated combinations of things which might arise during actual use and often cut across multiple systems.
An example of a test scenario might be: "in the middle of processing the system runs out of disk space; verify an admin is notified, that processing resumes once space is cleared, and that no data is lost".
A User Acceptance Test might use Scenario Testing.
These are my rules of thumb:
Unit testing: does this one function work?
Integration testing: do the functions work together?
Functional testing: does it function as required?
Acceptance testing: is it acceptable to the client?
Regression testing: does it still work like it used to?
User Acceptance Testing is having business users trying out your app.
There is also an Acceptance Testing done by QA when they check the new functionality - you can call it a Story Acceptance Testing to distinguish between these. These are not necessarily Functional Tests (could be Security, Performance testing, etc.).
Test Case is a number of steps to check a small piece of functionality. It has Prerequisites, Steps, Expected Result, Actual Result. This is one of the ways of carrying out Functional Testing. Others could be: exploratory testing, checklists.
Test Scenario - steps that cover a bigger picture. Often they cover cases of how real users would use the app. But these are carried out by QA team.
Functional Test - a test that checks the functionality as opposed to e.g. Performance. This can be a unit test as well, but since this terminology is mostly used by QA - when people talk about them they usually mean Functional System Test.
Note, that different authorities may use different definitions of the same terms. Check out Holes in testing terminology: Test Types and Test Levels. Since it's impossible to find the one true terminology it's more important that you use terms consistently within your team even if they are used differently in other companies and teams.
User acceptance testing is a process that obtains confirmation that a system meets agreed customer/product manager requirements.
Functional Testing is actual functionality test of the software there can be many different types of testing but in simple word testing the functionality what is should be expected.
The test scenario is the high level of testing cases when the first classification of module testing then module dividing into a scenario and at last small and specifical test steps with expected result says test cases, so the test scenario is group test cases with limited to specific functionality and module.
So I'm looking into introducing a more extensive testing process and I was reading about Functional Testing. I've worked with Scenario Testing before, but I'm fairly new to the term Functional Testing.
It seems that the two are synonymous, however I've not been able to find any information on whether they are synonymous or whether one is a subcategory of the other or whether they are two separate things. I've searched on stackoverflow, I've read the wiki pages for both, I've read some blogs on both and some university pages on both and can't seem to find the answer.
So as the title says; is there a difference between Scenario Testing and Functional Testing or are they synonymous?
Scenario testing got introduced with BDD (Behavorial driven development). So, basically while testing a single scenario, you will most of the times have multiple functional testing scenarios. The aim of Scenario testing is to test the scenario as a whole and BDD promotes it majorly.
Let me try to elaborate it with an example. So, say that a project is following BDD and the scenario to be tested is usually specified in the form of Acceptance Criteria as below:
As an authenticated user, in order to withdraw cash, I should be able to use the ATM.
So, in this case to test this Scenario, you would need multiple functional tests to run. Listing few of them below:
Authenticate user
Enter valid cash to withdraw
Cash withdrawn
This just one flow and there will be multiple positive and negative functional tests involved in this scenario which need to be designed.
So, we can say that a scenario consists of multiple functional tests. However, they are not synonyms since Scenario testing can also include scenarios related to performance, usability testing etc. and is not just restricted to functional testing.
Hope I could resolve your query to certain extent.
I honestly don't see the difference between BDD and TDD. I mean, both are just tests if what is expected happens. I've seen BDD Tests that are so fleshed out they practically count as TDD tests, and I've seen TDD tests that are so vague that they black box a lot of code. Let's just say I'm pretty convinced that having both is better.
Here's a fun question though. Where do I start? Do I start out with high level BDD tests? Do I start out with low level TDD tests?
I honestly don't see the difference between BDD and TDD.
That's because there isn't any.
I mean, both are just tests if what is expected happens.
That's wrong. BDD and TDD have absolutely nothing whatsoever to do with testing. None. Nada. Zilch. Zip. Nix. Not in the slightest.
Unfortunately, TDD has the word "test" in pretty much everything (not only in its name, but also in test framework, unit test, TestCase (the class you tpyically inherit from), FooTest (the class which typically holds your tests), testBar (the typical naming pattern for a test method), plus a lot test-related terminology such as "assertion" and "verification") which leads some people to believe that it actually does have something to do with tests. So, some smart people said: "Hey, let's just change the name" to remove any potential for confusion.
And that's what BDD is. It's just TDD with any test-related terminology replaced by examples-of-behavior-related terminology:
Test → Example
Assertion → Expectation
assert → should
Unit → Behavior
Verification → Specification
… and so on
BDD is just TDD with different words. If you do TDD right, you are doing BDD. The difference is that – provided you believe at least in the weak form of the Sapir-Whorf Hypothesis – the different words make it easier to do it right.
BDD is from customers point of view and focuses on excpected behavior of the whole system.
TDD is from developpers point of view and focuses on the implementation of one unit/class/feature. It benefits among others from better architecture (Design for testability, less coupling between modules).
From technical point of view (how to write the "test") they are similar.
I would (from an agile point of view) start with one bdd-userstory and implement it using TDD.
From what I've gathered on Wikipedia, BDD includes acceptance and QA test that can't be done without stakeholders/user input. Also BDD uses a natural language to specify its test while TDD usually uses programming language. There might be some overlap between the two but I think it's not the vagueness but BDD's language that is the main difference.
As for where you are to start, well that really depends on your development process, doesn't it? I assume if you are doing bottom-up that you're going to write TDD first and once you reach higher level you'll use BDD to test if those features work as expected.
As k3b noted: main difference would be that BDD is problem-domain oriented while TDD is more oriented solution-domain.
Just copying the answer from Matthew Flynn which I agree more than "TDD and BDD have nothing to do with tests":
Behavior Driven Development is an extension/revision of Test Driven Development. Its purpose is to help the folks devising the system (i.e., the developers) identify appropriate tests to write -- that is, tests that reflect the behavior desired by the stakeholders. The effect ends up being the same -- develop the test and then develop the code/system that passes the test. The hope in BDD is that the tests are actually useful in showing that the system meets the requirements.
UPDATE
Units of code (individual methods) may be too granular to represent the behavior represented by the behavioral tests, but you should still test them with unit tests to guarantee they function appropriately. If this is what you mean by "TDD" tests, then yes, you still need them.
BDD is about getting your TDD right. It provides "structure and diciplene" to your TDD. It guides you in testing the right thing and doing the right amount of test. Here is a fantastic small post on BDD and TDD,
http://codingcraft.wordpress.com/2011/11/12/bdd-get-your-tdd-right/
I think the biggest contribution of BDD over TDD or any other approaches, is making non-technical people(product owners/customers) part of the software development process at all levels.
Writing executable scenarios in natural languages have almost bridged the gap between the requirement and the delivery.
Product owners can himself run the scenarios he had written and test with different data sets if he wants to play around the behavior of the code written by the development team.
That's amazing! Customer is sitting right at the center and precisely not just asking what he really wants but verifying and experiencing the deliverables as well.
A fantastic article on the differences between TDD and BDD:
http://www.lostechies.com/blogs/sean_chambers/archive/2008/12/07/starting-with-bdd-vs-starting-with-tdd.aspx
Should give you everything you need to know, including problems with both, and examples.
Terminology are different, but in my work, i use TDD to dev detail, mainly for unit test, and the BDD is more high level, for customer, QA or no-tech man .
Overall
BDD is really Design-by-Contract using different terms. Generally speaking, BDD is in the form of Given-When-Then, which is roughly analogous to Preconditions (Given), Check-conditions/Loop-invariants (When), and Post-conditions/Invariants (Then).
Notice
Note that BDD is very much Hoare-logic (i.e. {P}C{Q} or {P}recondition-[C]ommand-{Q}Post-condition). Therefore:
Preconditions (Given) must hold true for the command (method/function) to compute correctly. Any violation of the Given (precondition) signals a fault in the calling Client code.
Command(s) (When) are what happens after the precondition(s) are met. In Eiffel, they can be punctuated within the method or function code with other contracts. Think about these as though they are QA/QC checks along a process assembly line.
Post-conditions (Then) must hold true once the Command (When) is finished.
Moral of the Story
Because BDD is just DbC (Hoare-logic) repackaged in different words, this means it is not TDD. Why? Because TDD is not about preconditions/checks/post-condition contracts tied directly to methods, functions, properties, and class-state. TDD is the next step up the ladder in testing methods, functions, properties, and classes with their discrete states. Once you see this and fully appreciate that TDD is not BDD and BDD is not TDD, but that they are separate and complementary technologies for software correctness proofs—THEN—you will finally understand these topics correctly. You will also use and apply them correctly.
Conclusion
Eiffel is the only language I am aware of where BDD (Design-by-Contract) is baked raw into both the language specification and compiler. It is not a Frankenstein bolt-on monster with limitations. In Eiffel, BDD (aka DbC) is an elegant, helpful, useful, and direct participant in the software correctness toolbox.
See Also
Wikipedia helps defined Hoare-logic. See: https://en.wikipedia.org/wiki/Hoare_logic
I have created an example in Eiffel that you can look at. See:
Primary class: https://github.com/ljr1981/stack_overflow_answers/blob/main/src/so_73347395/so_73347395.e
Test class: https://github.com/ljr1981/stack_overflow_answers/blob/main/testing/so_73347395/so_73347395_test_set.e
The main difference is just the wording. BDD uses a more verbose style so that it can be read almost like a sentence.