So I have learned that branch coverage differs from decision coverage as branch coverage typically includes also unconditional branches, e.g. methods calls, using of throw, break and other keywords in C#.
But I wonder, is this kind of branch coverage actually used in code analyzers? I suspect they use decision coverage, making sure that all decision outcomes (i.e. resulting branches) are covered.
I mean, the following code has 2 conditional, but 5 unconditional branches:
if(A)
B();
C();
D();
E();
else
X();
And I believe that if I write a test to evaluate A to just false, the code analyzers will tell me that the branch coverage is 50%. But from the unconditional branches perspective, more will nto be executed.
Is that correct?
Branch coverage doesn't tell you if a decision has been tested as both true and false.
Example:
if (c) {
x=...
}
y=...
If c evaluates to TRUE, the block containing x=... is executed, and
branch coverage will detect that. It will also detect that the code starting at y has been executed. So you'll get 100% coverage if C is true, without having any idea what happens if C is false.
With decision coverage, you would know that C has been evaluated and produces both TRUE and FALSE, if you had 100% coverage.
If your conditional if has a then block and an else block, then branch coverage and decision coverage will give you the same information.
Related
First, I am well aware of the "hidden branch" problem caused by throws/exceptions. This is not that.
What I am observing is:
My test framework (googletest) has testing macros (EXPECT_TRUE for example).
I write passing tests using the macros
Measuring branch coverage now asymptotes at 50% because I have not evaluated that test in both a passing and a failing condition...
Consider the following:
TEST (MyTests, ContrivedTest)
{
EXPECT_TRUE(function_that_always_returns_true());
}
Now assuming that I have every line and every branch perfectly covered in function_that_always_returns_true(), this branch coverage report will asymptote at 50% (because gcov does not observe line 3 evaluating in a failing condition, intentionally)
The only idea that I've had around this issue is that I could exclude the evaluation macros with something like LCOV_EXCL_BR_LINE, but this feels both un-ergonomic and hacky.
TEST (MyTests, ContrivedTest)
{
bool my_value = function_that_always_returns_true();
EXPECT_TRUE(my_value); //LCOV_EXCL_BR_LINE
}
This cannot be a niche problem, and I have to believe that people successfully use googletest with lcov/gcov. What do people do to get around this limitation?
After looking for far too long, I realized that all the testing calls I want to filter out are of the pattern EXPECT_*. So simply adding:
lcov_excl_br_line=LCOV_EXCL_BR_LINE|EXPECT_*
to my lcovrc solved my problem
When speaking of coverage criteria such as MCDC (Modified Condition/Decision Criteria)...
It is stated that "Every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken all possible outcomes at least once, and each condition has been shown to affect that decision outcome independently. A condition is shown to affect a decision's outcome independently by varying just that condition while holding fixed all other possible conditions. [...]"
- https://en.wikipedia.org/wiki/Modified_condition/decision_coverage
This description is rather vague of what constitutes an independent criteria... So, what are they? Examples are helpful in any language (C-family/python/haskell preferred).
The wikipedia definition is an informal statement, a more precise definition of MDCD is:
For each condition c, in each decision d, there is a test such that:
There is a test such that c == true
There is a test such that c == false
If the outcome of d when c == true is x, then the result of d when c == false must be !x.
All other conditions in d evaluate identically in both test cases.
If it is possible to create test set which meet these criteria, then this shows that each condition is not redundant: each condition at least influences the control of the program in some situation (as there is a test case that demonstrates this). This is what is meant by "independently influences the outcome".
Does multiple condition coverage imply branch coverage ?
I have been reading online for some comparisons between different coverage criteria, and I found some resources claim that multiple condition coverage doesn't necessarily imply branch coverage and others claim it does since it tests all possible combinations.
I understand that multiple condition coverage is difficult to achieve with short circuit logical operators, but I am not sure whether this is related to the comparison.
No, it does not.
Logically it should, but you can have a defect in the code causing the condition evaluate always to, for example, false, despite the combinations of sub-conditions.
E.g. (from Bullseye.com):
bool f(bool e) { return false; }
bool a[2] = { false, false };
if (f(a && b)) ...
if (a[int(a && b)]) ...
if ((a && b) ? false : false)
Yes. Multiple condition coverage (MCC) implies branch coverage.
Unlike plain condition coverage the MCC metric deals with all combinations of conditions that influence a decision. And decisions control branching.
In that sense multiple condition coverage is very similar to MC/DC.
While solving MCQs for a practice test I came across this statement - "In any program 100% statement coverage implies 100 % branch coverage" and it is termed as incorrect. I think its a correct statement because if we cover all the statements then it means we also cover all the paths and hence all the branches. Could someone please shed more light on this one?
Consider this code:
...
if (SomeCondition) DoSomething();
...
If SomeCondition is always true, you can have 100% statement coverage (SomeCondition and DoSomething() will be covered), but you never exercise the case when the condition is false, when you skip DoSomething().
Below example, a = true will cover 100% of statements, but fails to test the branch where a division by zero fault is possible.
int fun(bool a){
int x = 0;
if (a) x =1;
return 100/x;
}
For a test set to achieve 100% branch coverage, every branching point in the code must have been taken in each direction, at least once.
The archetypical example, showing that 100% statement coverage does not imply 100% branch coverage, was already given by Alexey Frunze. It is a consequence of the fact that (at least in the majority of programming languages) it is possible to have branches that do not involve statements (such a branch basically skips the statements in the other branch).
The reason for wanting 100% branch coverage, rather than just 100% statement coverage, is that your tests must also show that skipping some statements works as expected.
My main reason for providing this answer is to point out that the converse, viz. "100% branch coverage implies 100% statement coverage" is correct.
Just because you cover every statement doesnt mean that you covered every branch the program could have taken.
you have to look at every possible branch, not just the statements inside every branch.
if(condition1)
dosomething1();
if(condition2)
dosomething2();
if(condition3)
dosomething3();
Is it full branch testing if I have two test cases in this example
condition1 = condition2 = condition3 = true;
condition1 = condition2 = condition3 = false;
Or have I misunderstood it?
Trying to figure out the difference between branch and path testing. I get path testing so hope this is correct.
Branch Testing:
Testing in which all branches in the program source code are tested at least once.
Yes; you are performing correct branch testing, since all your branches are hit. In fact you can remove your second test case, since case 1 executes all the branches.
Obviously branch testing is less encompassing than path testing, since it's likelyhood of hitting dependies is low and as such, ought not to be your only form of testing.
As per my understanding, Branch coverage is also known as Decision coverage and it covers both the true and false conditions
unlike the statement coverage. With an IF statement, the exit can either be TRUE or FALSE, depending on the value
of the logical condition that comes after IF.
Let us take one example to explain Branch coverage:
IF "A > B"
PRINT A is greater than B
ENDIF
So the Test Set for 100% branch coverage will be:
Test Case 1: A=5, B=2 which will return true.
Test Case 2: A=2, B=5 which will return false.
So in your case, both the test cases 1 and 2 are required for Branch coverage.
With only Test cases1, it will be statement coverage.
I disagree with the chosen answer that you can remove the second test line!
Wikipedia's definition of Branch testing states:
"Branch coverage – Has each branch (also called DD-path) of each control structure (such as in if and case statements) been executed? For example, given an if statement, have both the true and false branches been executed? Another way of saying this is, has every edge in the program been executed?"Link here: https://en.wikipedia.org/wiki/Code_coverage
Also checkout this video lecture from Georgia Tech's Computer Science program on branch testing where this requirement is demonstrated in action.
Link here: https://www.youtube.com/watch?v=JkJFxPy08rk
To achieve 100% basis path coverage, you need to define your basis set. The cyclomatic complexity of this method is four (one plus the number of decisions), so you need to define four linearly independent paths. To do this, you pick an arbitrary first path as a baseline, and then flip decisions one at a time until you have your basis set.
Path 1: Any path will do for your baseline, so pick true for the decisions' outcomes (represented as TTT). This is the first path in your basis set.
Path 2: To find the next basis path, flip the first decision (only) in your baseline, giving you FTT for your desired decision outcomes.
Path 3: You flip the second decision in your baseline path, giving you TFT for your third basis path. In this case, the first baseline decision remains fixed with the true outcome.
Path 4 : Finally, you flip the third decision in your baseline path, giving you TTF for your fourth basis path. In this case, the first baseline decision remains fixed with the true outcome.
So, your four basis paths are TTT, FTT, TFT, and TTF. Now, make up your tests and see what happens.
Remember, the goal of basis path testing is to test all decision outcomes independently of one another
(Extract from http://www.codign.com/pathbranchcode.html)
If I understand what you are asking, then you may need eight test cases to completely cover the alternatives in the given code. For example, what if dosomething2() relies on some other state set up by dosomething1()? Your test cases would not catch that requirement.
Yes, you understand correctly. Branch testing is just "all branches are executed."