When speaking of coverage criteria such as MCDC (Modified Condition/Decision Criteria)...
It is stated that "Every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken all possible outcomes at least once, and each condition has been shown to affect that decision outcome independently. A condition is shown to affect a decision's outcome independently by varying just that condition while holding fixed all other possible conditions. [...]"
- https://en.wikipedia.org/wiki/Modified_condition/decision_coverage
This description is rather vague of what constitutes an independent criteria... So, what are they? Examples are helpful in any language (C-family/python/haskell preferred).
The wikipedia definition is an informal statement, a more precise definition of MDCD is:
For each condition c, in each decision d, there is a test such that:
There is a test such that c == true
There is a test such that c == false
If the outcome of d when c == true is x, then the result of d when c == false must be !x.
All other conditions in d evaluate identically in both test cases.
If it is possible to create test set which meet these criteria, then this shows that each condition is not redundant: each condition at least influences the control of the program in some situation (as there is a test case that demonstrates this). This is what is meant by "independently influences the outcome".
Related
I am searching some SWI-Prolog function which is able to make some set union with variables as parameters inside. My aim is to make the union first and define the parameters at further on in source code.
Means eg. I have some function union and the call union(A, B, A_UNION_B) makes sense. Means further more the call:
union(A, [1,2], C), A=[3].
would give me as result
C = [3, 1, 2].
(What you call union/3 is most probably just concatenation, so I will use append/3 for keeping this answer short.)
What you expect is impossible without delayed goals or constraints. To see this, consider the following failure-slice
?- append(A, [1,2], C), false, A=[3].
loops, unexpected. % observed, but for us unexpected
false. % expected, but not the case
This query must terminate, in order to make the entire question useful. But there are infinitely many lists of different length for A. So in order to describe all possible solutions, we would need infinitely many answer substitutions, like
?- append(A, [1,2], C).
A = [], C = [1,2]
; A = [_A], C = [_A,1,2]
; A = [_A,_B], C = [_A,_B,1,2]
; A = [_A,_B,_C], C = [_A,_B,_C,1,2]
; ... .
The only way around is to describe that set of solutions with finitely many answers. One possibility could be:
?- when((ground(A);ground(C)), append(A,B,C)).
when((ground(A);ground(C)),append(A,B,C)).
Essentially it reads: Yes, the query is true, provided the query is true.
While this solves your exact problem, it will now delay many otherwise succeeding goals, think of A = [X], B = [].
A more elaborate version could provide more complex tests. But it would require a somehow different definition than append/3 is. Some systems like sicstus-prolog provide block declarations to make this more smoothly (SWI has a coarse emulation for that).
So it is possible to make this even better, but the question remains whether or not this makes much sense. After all, debugging delayed goals becomes more and more difficult with larger programs.
In many situations it is preferable to prevent this and produce an instantiation error in its stead as iwhen/2 does:
?- iwhen((ground(A);ground(C)),append(A,B,C)).
error(instantiation_error,iwhen/2).
That error is not the nicest answer possible, but at least it is not incorrect. It says: You need to provide more instantiations.
If you really want to solve this problem for the general case you have to delve into E-unification. That is an area with most trivial problem statements and extremely evolved answers. Often, just decidability is non-trivial let alone an effective algorithm. For your particular question, either ACI (for sets) or ANlr (for concatenation) are of interest. Where ACI requires solving Diophantine Equations and associative unification alone is even more complex than that. I am unaware of any such implementation for a Prolog system that solves the general problem.
Prolog IV offered an associative infix operator for concatenation but simply delayed more complex cases. So debugging these remains non-trivial.
Does multiple condition coverage imply branch coverage ?
I have been reading online for some comparisons between different coverage criteria, and I found some resources claim that multiple condition coverage doesn't necessarily imply branch coverage and others claim it does since it tests all possible combinations.
I understand that multiple condition coverage is difficult to achieve with short circuit logical operators, but I am not sure whether this is related to the comparison.
No, it does not.
Logically it should, but you can have a defect in the code causing the condition evaluate always to, for example, false, despite the combinations of sub-conditions.
E.g. (from Bullseye.com):
bool f(bool e) { return false; }
bool a[2] = { false, false };
if (f(a && b)) ...
if (a[int(a && b)]) ...
if ((a && b) ? false : false)
Yes. Multiple condition coverage (MCC) implies branch coverage.
Unlike plain condition coverage the MCC metric deals with all combinations of conditions that influence a decision. And decisions control branching.
In that sense multiple condition coverage is very similar to MC/DC.
There's a certain over-verbosity that I have to engage in when writing certain Boolean expressions, at least with all the languages I've used, and I was wondering if there were any languages that let you write more concisely?
The way it goes is like this:
I want to find out if I have a Thing that can be either A, B, C, or D.
And I'd like to see if Thing is an A or a B.
The logical way for me to express this is
//1: true if Thing is an A or a B
Thing == (A || B)
Yet all the languages I know expect it to be written as
//2: true if Thing is an A or a B
Thing == A || Thing == B
Are there any languages that support 1? It doesn't seem problematic to me, unless Thing is a Boolean.
Yes. Icon does.
As a simple example, here is how to get the sum of all numbers less than 1000 that are divisble by three or five (the first problem of Project Euler).
procedure main ()
local result
local n
result := 0
every n := 1 to 999 do
if n % (3 | 5) == 0 then
result +:= n
write (result)
end
Note the n % (3 | 5) == 0 expression. I'm a bit fuzzy on the precise semantics, but in Icon, the concept of booleans is not like other languages. Every expression is a generator and it may pass (generating a value) or fail. When used in an if expression, a generator will continue to iterate until it passes or exhausts itself. In this case, n % (3 | 5) == 0 is a generator which uses another generator (3 | 5) to test if n is divisible by 3 or 5. (To be entirely technical, this isn't even syntactic sugar.)
Likewise, in Python (which was influenced by Icon) you can use the in statement to test for equality on multiple elements. It's a little weaker than Icon though (as in, you could not translate the modulo comparison above directly). In your case, you would write Thing in (A, B), which translates exactly to what you want.
There are other ways to express that condition without trying to add any magic to the conditional operators.
In Ruby, for example:
$> thing = "A"
=> "A"
$> ["A","B"].include? thing
=> true
I know you are looking for answers that have the functionality built into the language, but here are two other means that I find work better as they solve more problems and have been in use for many decades.
Have you considered using a preprocessor?
Also languages like Lisp have macros which is part of the language.
If I have a Boolean expression to check
(A && B)
If A is found to be false will the language bother to check B? Does this vary from language to language?
The reason I ask is that I'm wondering if it's the case that B is checked even if A is false then wouldn't
if (A) {
if(B) {
} else {
// code x
}
} else {
// code x
}
be marginally quicker than
if (A && B) {
} else {
// code x
}
This depends on the language. Most languages will implement A && B as a short-circuit operator, meaning that if A evaluates to false, B will never be evaluated. There's a detailed list on Wikipedia.
Almost every language implements something called short-circuit evaluation, which means that yes, (A && B) will not evaluate B if A is false. This also takes effect if you write:
if (A || B) {
...
}
and A is true. This is worth remembering if B may take a long time to load, but generally it's not something to worry about.
As a bit of history, in my mind this is a bit of a sore part of LISP because code like this:
(if (and (= x 5) (my-expensive-query y)) "Yes" "No")
is not made of functions, but rather so-called "special forms" (that is, "and" could not be defun'd here).
This would depend 100% on how the language compiles said code. Anything is possible :)
Is there a specific language you're wondering about?
In short, no. A double branch involves various forms of branch prediction. If A and B are simple to evaluate, it may be faster to do if (A && B) in a non-short-circuit way than if (A) if (B). In addition, you've duplicated code in the second form. This is virtually always (exception to every rule .. I guess) bad and far worse than any gain.
Secondly, this is the kind of micro-optimization that you give to the language interpreter, JIT or compiler.
Many languages (including almost all curly-brace languages, like C/C++/Java/C#) offer short-circuit boolean evaluation. In those languages, if A is false then B won't be evaluated. You'll need to see (or ask) whether this is true for your specific language, or whether there's a way to do it (VB has AndAlso, for example).
If you find your language doesn't support it, you'll also need to consider whether the cost of evaluating B is worth having to maintain two identical pieces of code -- and the potential doubling in cache footprint (not to mention all the extra branching) that'd come from doing that duplication every time.
As others have said, it depends on the language and/or compiler. For me, I don't care how fast or slow it might be to short-circuit or not, the need to duplicate code is a deal-killer.
If A and B are actually calls that have side-effects (i.e. they do more than simply return a value suitable for comparison), then I would argue that those calls should be made into variable assignments that are then used in your comparison. It doesn't matter whether or not you always require those side-effects or only require them conditionally, the code will be more readable if you don't depend on whether or not short-circuit exists.
That last bit about readability is based on my feeling that reducing the need to refer to external documentation improves readability. Reading a book with a bunch of new words that require dictionary look-ups is much more challenging than reading that same book when you already have the necessary vocabulary. In this case, short-circuit is invisible, so anybody that needs to look it up won't even know that they need to look it up.
if(condition1)
dosomething1();
if(condition2)
dosomething2();
if(condition3)
dosomething3();
Is it full branch testing if I have two test cases in this example
condition1 = condition2 = condition3 = true;
condition1 = condition2 = condition3 = false;
Or have I misunderstood it?
Trying to figure out the difference between branch and path testing. I get path testing so hope this is correct.
Branch Testing:
Testing in which all branches in the program source code are tested at least once.
Yes; you are performing correct branch testing, since all your branches are hit. In fact you can remove your second test case, since case 1 executes all the branches.
Obviously branch testing is less encompassing than path testing, since it's likelyhood of hitting dependies is low and as such, ought not to be your only form of testing.
As per my understanding, Branch coverage is also known as Decision coverage and it covers both the true and false conditions
unlike the statement coverage. With an IF statement, the exit can either be TRUE or FALSE, depending on the value
of the logical condition that comes after IF.
Let us take one example to explain Branch coverage:
IF "A > B"
PRINT A is greater than B
ENDIF
So the Test Set for 100% branch coverage will be:
Test Case 1: A=5, B=2 which will return true.
Test Case 2: A=2, B=5 which will return false.
So in your case, both the test cases 1 and 2 are required for Branch coverage.
With only Test cases1, it will be statement coverage.
I disagree with the chosen answer that you can remove the second test line!
Wikipedia's definition of Branch testing states:
"Branch coverage – Has each branch (also called DD-path) of each control structure (such as in if and case statements) been executed? For example, given an if statement, have both the true and false branches been executed? Another way of saying this is, has every edge in the program been executed?"Link here: https://en.wikipedia.org/wiki/Code_coverage
Also checkout this video lecture from Georgia Tech's Computer Science program on branch testing where this requirement is demonstrated in action.
Link here: https://www.youtube.com/watch?v=JkJFxPy08rk
To achieve 100% basis path coverage, you need to define your basis set. The cyclomatic complexity of this method is four (one plus the number of decisions), so you need to define four linearly independent paths. To do this, you pick an arbitrary first path as a baseline, and then flip decisions one at a time until you have your basis set.
Path 1: Any path will do for your baseline, so pick true for the decisions' outcomes (represented as TTT). This is the first path in your basis set.
Path 2: To find the next basis path, flip the first decision (only) in your baseline, giving you FTT for your desired decision outcomes.
Path 3: You flip the second decision in your baseline path, giving you TFT for your third basis path. In this case, the first baseline decision remains fixed with the true outcome.
Path 4 : Finally, you flip the third decision in your baseline path, giving you TTF for your fourth basis path. In this case, the first baseline decision remains fixed with the true outcome.
So, your four basis paths are TTT, FTT, TFT, and TTF. Now, make up your tests and see what happens.
Remember, the goal of basis path testing is to test all decision outcomes independently of one another
(Extract from http://www.codign.com/pathbranchcode.html)
If I understand what you are asking, then you may need eight test cases to completely cover the alternatives in the given code. For example, what if dosomething2() relies on some other state set up by dosomething1()? Your test cases would not catch that requirement.
Yes, you understand correctly. Branch testing is just "all branches are executed."