what's the difference between
L1 = {m ∈ (a + b)*} L1 = {m ∈ (a , b)*}
i feel like there is no difference but i cant really understand the difference between this + and the plus in this expression for example (a,b)+
You are correct, there is no difference. a+b translates to a union b and (a , b) is another (albeit much less common) way that some researchers refer to a union b. If you so choose to continue your study of automata you will find that there is an incredible variety of ways that different researchers refer to the same concepts. In this instance alone, off the top of my head I know this can be written as:
L1 = {m ∈ (a + b)*}
L1 = {m ∈ (a , b)*}
L1 = {m ∈ (a | b)*}
L1 = {m ∈ (a U b)*}
Related
Does that statement follow Big O transitivity?
I am new to Big O notation and Time Complexity so I am struggling with the basics.
Any help would be greatly appreciated!
Think of the counterexample:
f(n) = n3
g(n) = n2
h(n) = n.
Indeed, g = O(f) and h = O(f). But is g = O(h)?
A good way to attack this problem is to leverage the definition of Big O. I say "good" because you'll be better off in the long run by developing a deeper understanding of the theory behind this question.
First, note that f(n) = O(g(n)) if and only if f(n) ≤ c∙g(n) for all n ≥ n0 and some c > 0.
Next, apply that definition to the statements in the question.
If g(n) = O(f(n)) then g(n) ≤ c0∙f(n) for all n ≥ n0 and some c0 > 0...
Similarly, if h(n) = O(f(n)), then h(n) ≤ c1∙f(n) ... etc.
So, given these two facts, are you able to prove that g(n) = O(h(n))? Well, what does that actually mean?
It means that (again!):
g(n) ≤ c2∙h(n) for all n ≥ n2 and some c2 > 0
So what you have is that:
g(n) is "less" than f(n)
h(n) is also "less" than f(n)
Can you conclude that g(n) is "less" than h(n)? No, you cannot. Now, after breaking this down and figuring out what you think the right answer is, you can try to find a counterexample (which has already been provided).
I generally take this approach when I'm trying to answer one of those tricky "True or False? If true, prove it. If false, give a counterexample" problems since I find that it enhances my understanding of the concepts that I'm studying!
An old man trying to learn more and got stuck on this exercise in some old exam:
Specify the complexity, in Θ(.) Notation, of the Test(n) function, detailed below, in each of the following three cases:
1/ n is even.
2/ n is a perfect square, that is, there exists an integer i such that i² = n.
3/ n is a prime number.
Function Test( n : Integer) : Integer
Variable
i : Integer
Start
for i := 2 to n do
if n mod i = 0 Return( i ) End-if
End-for
Return(n)
End
I think the comments have answered your general question, but a note about proving Big Theta time complexity:
To show f(n) ∈ Θ(g(n)), you don't necessarily have to prove f(n) ∈ O(g(n)) and f(n) ∈ Ω(g(n)) through the method you alluded to of showing there exists a constant c and an integer n0 for which for all n > n0 f(n) < cg(n), and then finding another c and n0 for which for all n > n0 f(n) > cg(n). Although this is perfectly valid and widely taught, there is an alternative, equally mathematically rigorous but generally much cleaner and more practical approach, which is just to show that:
0 < lim n-> \infty f(n) / g(n) < \infty
That is, f(n) ∈ Θ(g(n)) iff the limit as n goes to infinity of f(n) over g(n) is some constant.
If you showed only that
lim n-> \infty f(n) / g(n) < \infty
you would have shown that f(n) grows no faster than g(n): that is, that f(n) ∈ O(g(n)).
Similarly:
0 < lim n-> \infty f(n) / g(n)
implies that f(n) grows at least as fast as g(n): that f(n) ∈ Ω(g(n)). So together, they imply f(n) ∈ Θ(g(n))
Generally I think this is a lot less tedious than proofs of the form you mentioned, which involve actually finding c values and n0 values for the big O case, proving some needlessly particular statement involving those values, and then repeating the whole process for the Ω case, but that is just my opinion: whatever style works for you is good.
What is the definition of f(n) = O(n^2)?
This means the following:
There exist c > 0 and n0 such that f(n) <= c*n^2 for all n >= n0.
What is the precise definition of f(E, V) = O(E + V)?
It really works in the same way in your case. One valid definition might be:
There exist c > 0 and n0 such that f(E, V) <= c*(E+V) for all E, V >= n0.
However, you can also define it differently, e.g. by introducing two variables c, d > 0 and requiring f(E,V) <= cE + dE. Both are valid definitions.
However, most likely you have encountered this definition in the context of graph algorithms, where E is the number of edges and V the number of vertices. The time complexity O(E + V) occurs a lot in this field as it really is the same thing as saying O(max(E, V)). It is still linear time complexity.
Consider a dependent type
Inductive dep (n: nat) :=
mkDep : dep n.
Now, consider a simple theorem I wish to prove:
Theorem equalTypes (n n': nat): n = n' -> dep n = dep n'.
Proof.
intros.
Abort.
How do I show that two dependent types are equal? What is a notion of type equality?
Worse, consider this "theorem" (which does not compile)
Theorem equalInhabitants (n n' : nat): n = n' -> mkDep n = mkDep n'.
Abort.
This very statement is wrong, because the types mkDep n and mkDep n' don't match. However, in some sense, this statement is true, because they are the same value under the assumption n = n'.
I wish to understand how to formalize and prove statements about dependent types (specifically, their equality and notions thereof)
How do I show that two dependent types are equal?
In this case, you can prove it with apply f_equal; assumption or subst; reflexivity (or destruct H; reflexivity or case H; reflexivity or induction H; reflexivity or exact (eq_rect n (fun n' => dep n = dep n') eq_refl n' H)).
What is a notion of type equality?
The same as any other equality; Print eq. gives:
Inductive eq (A : Type) (x : A) : A -> Prop := eq_refl : x = x
which says that the only special fact you have to construct a proof of equality is that x = x for any x. The way to use a proof of equality, eq_rect, is that, if you have x = y, to prove a property P of y, it suffices to prove P of x. In this case, since we have n = n', to prove dep n = dep n', it suffices to prove dep n = dep n (where P := fun n' => dep n = dep n').
There is a deeper sense in which this question can be asked, because it turns out that equality of types in Coq is under-constrained. Given
Inductive unit1 := tt1.
Inductive unit2 := tt2.
you can not prove unit1 = unit2, nor can you prove unit1 <> unit2. In fact, it turns out that the only type inequalities T1 <> T2 that you can prove are cases where you can prove that T1 and T2 are not isomorphic. The Univalence axiom is a way of "filling in the details" of type equality to say that any isomorphic types are equal. There are other consistent interpretations, though (for example, I believe that it's consistent to assume A * B = C * D -> A = C /\ B = D, though this contradicts univalence).
Worse, consider this "theorem" (which does not compile)
Theorem equalInhabitants (n n' : nat): n = n' -> mkDep n = mkDep n'.
Right. This is because we do not have an equality reflection rule in Coq, and judgmental/definitional equality is not the same as propositional equality. The way to state this is to "cast" the term mkDep n across the proof of equality.
Import EqNotations.
Theorem equalInhabitants (n n' : nat): forall H : n = n', rew H in mkDep n = mkDep n'.
intros.
subst; reflexivity.
Qed.
Note that rew binds more tightly than =, and is a notation for eq_rect. This says that for any proof H of n = n', the term mkDep n, when transported across H to become a term of type dep n', is equal to mkDep n'. (Note also that we could just as well have used destruct H or induction H or case H (but not apply f_equal) instead of subst.)
Still working on my SHA1 implementation in Haskell. I've now got a working implementation and this is the inner loop:
iterateBlock' :: Int -> [Word32] -> Word32 -> Word32 -> Word32 -> Word32 -> Word32 -> [Word32]
iterateBlock' 80 ws a b c d e = [a, b, c, d, e]
iterateBlock' t (w:ws) a b c d e = iterateBlock' (t+1) ws a' b' c' d' e'
where
a' = rotate a 5 + f t b c d + e + w + k t
b' = a
c' = rotate b 30
d' = c
e' = d
The profiler tells me that this function takes 1/3 of the runtime of my implementation. I can think of no way to further optimize it other than maybe inlining the temp variables but I believe -O2 will do that for me anyway.
Can anyone see a significant optimization that can be further applied?
FYI the k and f calls are below. They are so simple I don't think there is a way to optimize these other. Unless the Data.Bits module is slow?
f :: Int -> Word32 -> Word32 -> Word32 -> Word32
f t b c d
| t <= 19 = (b .&. c) .|. ((complement b) .&. d)
| t <= 39 = b `xor` c `xor` d
| t <= 59 = (b .&. c) .|. (b .&. d) .|. (c .&. d)
| otherwise = b `xor` c `xor` d
k :: Int -> Word32
k t
| t <= 19 = 0x5A827999
| t <= 39 = 0x6ED9EBA1
| t <= 59 = 0x8F1BBCDC
| otherwise = 0xCA62C1D6
Looking at the core produced by ghc-7.2.2, the inlining works out well. What doesn't work so well is that in each iteration a couple of Word32 values are first unboxed, to perform the work, and then reboxed for the next iteration. Unboxing and re-boxing can cost a surprisingly large amount of time (and allocation).
You can probably avoid that by using Word instead of Word32. You couldn't use rotate from Data.Bits then, but would have to implement it yourself (not hard) to have it work also on 64-bit systems. For a' you would have to manually mask out the high bits.
Another point that looks suboptimal is that in each iteration t is compared to 19, 39 and 59 (if it's large enough), so that the loop body contains four branches. It will probably be faster if you split iterateBlock' into four loops (0-19, 20-39, 40-59, 60-79) and use constants k1, ..., k4, and four functions f1, ..., f4 (without the t parameter) to avoid branches and have smaller code-size for each loop.
And, as Thomas said, using a list for the block data isn't optimal, an unboxed Word array/vector would probably help too.
With the bang patterns, the core looks much better. Two or three less-than-ideal points remain.
(GHC.Prim.narrow32Word#
(GHC.Prim.plusWord#
(GHC.Prim.narrow32Word#
(GHC.Prim.plusWord#
(GHC.Prim.narrow32Word#
(GHC.Prim.plusWord#
(GHC.Prim.narrow32Word#
(GHC.Prim.plusWord#
(GHC.Prim.narrow32Word#
(GHC.Prim.or#
(GHC.Prim.uncheckedShiftL# sc2_sEn 5)
(GHC.Prim.uncheckedShiftRL# sc2_sEn 27)))
y#_aBw))
sc6_sEr))
y#1_XCZ))
y#2_XD6))
See all these narrow32Word#? They're cheap, but not free. Only the outermost is needed, there may be a bit to harvest by hand-coding the steps and using Word.
Then the comparisons of t with 19, ..., they appear twice, once to determine the k constant, and once for the f transform. The comparisons alone are cheap, but they cause branches and without them, further inlining may be possible. I expect a bit could be gained here too.
And still, the list. That means w can't be unboxed, the core could be simpler if w were unboxable.