Typically scan, both left and right variants, are O(n) in both space and time. However it seems that APL's \ operator is like scanl but seems to behave differently in that it is right associative and runs over the array each time, making it O(n^2).
For example,
nums ← 10?10 ⍝ 1 7 4 5 10 3 9 6 2 8
⌈\nums ⍝ 1 7 7 7 10 10 10 10 10 10
gives me the correct behavior but by right associativity is equivalent to
(1 f (7 f (4 f (5 f (10 f (3 f (9 f (6 f (2 f 8))))))))) ⍝ where f ← (⊣,⌈)
so the last operation is 1 f (7 7 7 10 10 10 10 10 10)
Isn't this inefficient, what is the actual big O complexity here and/or is there some idiomatic optimization?
You are right in your algorithmic description Scan, and in the general case, it is indeed O(N²). However, by far the most common uses of it is with a known set of scalar primitives (including +, ∨, ∧, ⌈, ⌊, <, ≤, ≠). These are recognised by the interpreter, which then uses special code O(n) (or less, as they might leave early) code.
We can easily demonstrate this by comparing the performance of + with the functionally identical +∘⊢ (plus, where the right argument is pre-processed by the identity function):
'cmpx'⎕CY'dfns'
a←?2000⍴127
cmpx'+∘⊢\a'
1.8E¯1
a←?4000⍴127
cmpx'+∘⊢\a'
7.6E¯1
We can see that +∘⊢\ took about 4 times as long (0.2 s → 0.8 s) when we doubled the number of small integers from 2000 to 4000. Whereas:
a←?2000⍴127
cmpx'+\a'
1.5E¯6
a←?4000⍴127
cmpx'+\a'
2.9E¯6
+\ only doubles (15 ms → 29 ms) when going from 2000 to 4000 small integers. Also note the extreme performance difference between the optimised case and the non-optimised case.
Scan gets its order of evaluation from Reduce. Iverson explains in Conventions Governing Order of Evaluation:
In the definition F/x ≡ x1 F x2 F x3 F ... F x⍴x
the right-to-left convention leads to a more useful definition for nonassociative functions F than does the left-to-right convention. For example, -/x denotes the alternating sum of the components of x , whereas in a left-to-right convention it would denote the first component minus the sum of the remaining components. Thus if d is the vector of decimal digits representing the number n , then the value of the expression 0=9|+/d determines the divisibility of n by 9 ; in the right-to-left convention, the similar expression 0=11|-/d determines divisibility by 11 .
Related
I'm trying to implement Algorithm D from Knuth's "The Art of Computer Programming, Vol 2" in Rust although I'm having trouble understating how to implement the very last step of unnormalizing. My natural numbers are a class where each number is a vector of u64, in base u64::MAX. Addition, subtraction, and multiplication have been implemented.
Knuth's Algorithm D is a euclidean division algorithm which takes two natural numbers x and y and returns (q,r) where q = x / y (integer division) and r = x % y, the remainder. The algorithm depends on an approximation method which only works if the first digit of y is greater than b/2, where b is the base you're representing the numbers in. Since not all numbers are of this form, it uses a "normalizing trick", for example (if we were in base 10) instead of doing 200 / 23, we calculate a normalizer d and do (200 * d) / (23 * d) so that 23 * d has a first digit greater than b/2.
So when we use the approximation method, we end up with the desired q but the remainder is multiplied by a factor of d. So the last step is to divide r by d so that we can get the q and r we want. My problem is, I'm a bit confused at how we're suppose to do this last step as it requires division and the method it's part of is trying to implement division.
(Maybe helpful?):
The way that d is calculated is just by taking the integer floor of b-1 divided by the first digit of y. However, Knuth suggests that it's possible to make d a power of 2, as long as d * the first digit of y is greater than b / 2. I think he makes this suggestion so that instead of dividing, we can just do a binary shift for this last step. Although I don't think I can do that given that my numbers are represented as vectors of u64 values, instead of binary.
Any suggestions?
Suppose we have an algorithm that is of order O(2^n). Furthermore, suppose we multiplied the input size n by 2 so now we have an input of size 2n. How is the time affected? Do we look at the problem as if the original time was 2^n and now it became 2^(2n) so the answer would be that the new time is the power of 2 of the previous time?
Big 0 is not for telling you the actual running time, just how the running time is affected by the size of input. If you double the size of input the complexity is still O(2^n), n is just bigger.
number of elements(n) units of work
1 1
2 4
3 8
4 16
5 32
... ...
10 1024
20 1048576
There's a misunderstanding here about how Big-O relates to execution time.
Consider the following formulas which define execution time:
f1(n) = 2^n + 5000n^2 + 12300
f2(n) = (500 * 2^n) + 6
f3(n) = 500n^2 + 25000n + 456000
f4(n) = 400000000
Each of these functions are O(2^n); that is, they can each be shown to be less than M * 2^n for an arbitrary M and starting n0 value. But obviously, the change in execution time you notice for doubling the size from n1 to 2 * n1 will vary wildly between them (not at all in the case of f4(n)). You cannot use Big-O analysis to determine effects on execution time. It only defines an upper boundary on the execution time (which is not even guaranteed to be the minimum form of the upper bound).
Some related academia below:
There are three notable bounding functions in this category:
O(f(n)): Big-O - This defines a upper-bound.
Ω(f(n)): Big-Omega - This defines a lower-bound.
Θ(f(n)): Big-Theta - This defines a tight-bound.
A given time function f(n) is Θ(g(n)) only if it is also Ω(g(n)) and O(g(n)) (that is, both upper and lower bounded).
You are dealing with Big-O, which is the usual "entry point" to the discussion; we will neglect the other two entirely.
Consider the definition from Wikipedia:
Let f and g be two functions defined on some subset of the real numbers. One writes:
f(x)=O(g(x)) as x tends to infinity
if and only if there is a positive constant M such that for all sufficiently large values of x, the absolute value of f(x) is at most M multiplied by the absolute value of g(x). That is, f(x) = O(g(x)) if and only if there exists a positive real number M and a real number x0 such that
|f(x)| <= M|g(x)| for all x > x0
Going from here, assume we have f1(n) = 2^n. If we were to compare that to f2(n) = 2^(2n) = 4^n, how would f1(n) and f2(n) relate to each other in Big-O terms?
Is 2^n <= M * 4^n for some arbitrary M and n0 value? Of course! Using M = 1 and n0 = 1, it is true. Thus, 2^n is upper-bounded by O(4^n).
Is 4^n <= M * 2^n for some arbitrary M and n0 value? This is where you run into problems... for no constant value of M can you make 2^n grow faster than 4^n as n gets arbitrarily large. Thus, 4^n is not upper-bounded by O(2^n).
See comments for further explanations, but indeed, this is just an example I came up with to help you grasp Big-O concept. That is not the actual algorithmic meaning.
Suppose you have an array, arr = [1, 2, 3, 4, 5].
An example of a O(1) operation would be directly access an index, such as arr[0] or arr[2].
An example of a O(n) operation would be a loop that could iterate through all your array, such as for elem in arr:.
n would be the size of your array. If your array is twice as big as the original array, n would also be twice as big. That's how variables work.
See Big-O Cheat Sheet for complementary informations.
I found the time complexity of Prims algorithm everywhere as O((V + E) log V) = E log V. But as we can see the algorithm:
It seems like the time complexity is O(V(log V + E log V)). But if its time complexity is O((V + E) log V). Then the nesting must have to be like this:
But the above nesting is seems to be wrong.
MST-PRIM(G, w, r)
1 for each u ∈ G.V
2 u.key ← ∞
3 u.π ← NIL
4 r.key ← 0
5 Q ← G.V
6 while Q ≠ Ø
7 u ← EXTRACT-MIN(Q)
8 for each v ∈ G.Adjacent[u]
9 if v ∈ Q and w(u, v) < v.key
10 v.π ← u
11 v.key ← w(u, v)
Using a Binary Heap
The time complexity required for one call to EXTRACT-MIN(Q) is O(log V) using a min priority queue. The while loop at line 6 is executing total V times.so EXTRACT-MIN(Q) is called V times. So the complexity of EXTRACT-MIN(Q) is O(V logV).
The for loop at line 8 is executing total 2E times as length of each adjacency lists is 2E for an undirected graph. The time required to execute line 11 is O(log v) by using the DECREASE_KEY operation on the min heap. Line 11 also executes total 2E times. So the total time required to execute line 11 is O(2E logV) = O(E logV).
The for loop at line 1 will be executed V times. Using the procedure to perform lines 1 to 5 will require a complexity of O(V).
Total time complexity of MST-PRIM is the sum of the time complexity required to execute steps 1 through 3 for a total of O((VlogV) + (E logV) + (V)) = O(E logV) since |E| >= |V|.
Using a Fibonacci Heap
Same as above.
Executing line 11 requires O(1) amortized time. Line 11 executes a total of 2E times. So the total time complexity is O(E).
Same as above
So the total time complexity of MST-PRIM is the sum of executing steps 1 through 3 for a total complexity of O(V logV + E + V)=O(E + V logV).
Your idea seems correct. Let's take the complexity as
V(lg(v) + E(lg(v)))
Then notice that in the inner for loop, we are actually going through all the vertices, and not the edge, so let's modify a little to
V(lg(v) + V(lg(v)))
which means
V(lg(v)) + V*V(lg(v))
But for worst case analysis(dense graphs), V*V is roughly equal to number of edges, E
V(lg(v)) + E(lg(v))
(V+E((lg(v))
but since V << E, hence
E(lg(v))
The time complexity of Prim's algorithm is O(VlogV + ElogV). It seems like you understand how the VlogV came to be, so let's skip over that. So where does ElogV come from? Let's start by looking at Prim's algorithm's source code:
| MST-PRIM(Graph, weights, r)
1 | for each u ∈ Graph.V
2 | u.key ← ∞
3 | u.π ← NIL
4 | r.key ← 0
5 | Q ← Graph.V
6 | while Q ≠ Ø
7 | u ← EXTRACT-MIN(Q)
8 | for each v ∈ Graph.Adj[u]
9 | if v ∈ Q and weights(u, v) < v.key
10| v.π ← u
11| v.key ← weights(u, v)
Lines 8-11 are executed for every element in Q, and we know that there are V elements in Q (representing the set of all vertices). Line 8's loop is iterating through all the neighbors of the currently extracted vertex; we will do the same for the next extracted vertex, and for the one after that. Djistkra's Algorithm does not repeat vertices (because it is a greedy, optimal algorithm), and will have us go through each of the connected vertices eventually, exploring all of their neighbors. In other words, this loop will end up going through all the edges of the graph twice at some point (2E).
Why twice? Because at some point we come back to a previously explored edge from the other direction, and we can't rule it out until we've actually checked it. Fortunately, that constant 2 is dropped during our time complexity analysis, so the loop is really just doing E amounts of work.
Why wasn't it V*V? You might reach that term if you just consider that we have to check each Vertex and its neighbors, and in the worst case graph the number of neighbors approaches V. Indeed, in a dense graph V*V = E. But the more accurate description of the work of these two loops is "going through all the edges twice", so we refer to E instead. It's up to the reader to connect how sparse their graph is with this term's time complexity.
Let's look at a small example graph with 4 vertices:
1--2
|\ |
| \|
3--4
Assume that Q will give us the nodes in the order 1, 2, 3, and then 4.
In the first iteration of the outer loop, the inner loop will run 3 times (for 2, 3, and 4).
In the second iteration of the outer loop, the inner loop runs 2 times (for 1 and 4).
In the third iteration of the outer loop, the inner loop runs 2 times (for 1 and 4).
In the last iteration of the outer loop, the inner loop runs 3 times (for 1, 2, and 3).
The total iterations was 10, which is twice the number of edges (2*5).
Extracting the minimum and tracking the updated minimum edges (usually done with a Fibonacci Heap, resulting in log(V) time complexity) occurs inside the loop iterations - the exact mechanisms involve end up needing to occur inside the inner loop enough times that they are controlled by the time complexity of both loops. Therefore, the complete time complexity for this phase of the algorithm is O(2*E*log(V)). Dropping the constant yields O(E*log(V)).
Given that the total time complexity of the algorithm is O(VlogV + ElogV), we can simplify to O((V+E)logV). In a dense graph E > V, so we can conclude O(ElogV).
actually as you are saying as for is nested inside while time complexity should be v.E lg V is correct in case of asymptotic analysis. But in cormen they have done amortized analysis thats why it comes out to be (Elogv)
As title says, I don't understand why f^:proposition^:_ y is a while loop. I have actually used it a couple times, but I don't understand how it works. I get that ^: repeats functions, but I'm confused by its double use in that statement.
I also can't understand why f^:proposition^:a: y works. This is the same as the previous one but returns the values from all the iterations, instead of only the last one as did the one above.
a: is an empty box and I get that has a special meaning used with ^: but even after having looked into the dictionary I couldn't understand it.
Thanks.
Excerpted and adapted from a longer writeup I posted to the J forums in 2009:
while =: ^:break_clause^:_
Here's an adverb you can apply to any code (which would equivalent of the
loop body) to create a while loop. In case you haven't seen it before, ^: is the power conjunction. More specifically, the phrase f^:n y applies the function f to the argument y exactly n times. The count n maybe be an integer or a function which applied to y produces an integer¹.
In the adverb above, we see the power conjunction twice, once in ^:break_clause and again in ^:_ . Let's first discuss the latter. That _ is J's notation for infinity. So, read literally, ^:_ is "apply the function an infinite number of times" or "keep reapplying forever". This is related to a while-loop's function, but it's not very useful if applied literally.
So, instead, ^:_ and its kin were defined to mean "apply a function to its limit", that is, "keep applying the function until its output matches its input". In that case, applying the function again would have no effect, because the next iteration would have the same input as the previous (remember that J is a functional language). So there's
no point in applying the function even once more: it has reached its limit.
For example:
cos=: 2&o. NB. Cosine function
pi =: 1p1 NB. J's notation for 1*pi^1 analogous to scientific notation 1e1
cos pi
_1
cos cos cos pi
0.857553
cos^:3 pi
0.857553
cos^:10 pi
0.731404
cos^:_ pi NB. Fixed point of cosine
0.739085
Here, we keep applying cosine until the answer stops changing: cosine has reached its fixed point, and more applications are superfluous. We can visualize this by showing the
intermediate steps:
cos^:a: pi
3.1415926535897 _1 0.54030230586813 ...73 more... 0.73908513321512 0.73908513321
So ^:_ applies a function to its limit. OK, what about ^:break_condition? Again, it's the same concept: apply the function on the left the number of times specified by the function on the right. In the case of _ (or its function-equivalent, _: ) the output is "infinity", in the case of break_condition the output will be 0 or 1 depending on the input (a break condition is boolean).
So if the input is "right" (i.e. processing is done), then the break_condition will be 0, whence loop_body^:break_condition^:_ will become loop_body^:0^:_ . Obviously, loop_body^:0 applies the loop_body zero times, which has no effect.
To "have no effect" is to leave the input untouched; put another way, it copies the input to the output ... but if the input matches the output, then the function has reached its limit! Obviously ^:_: detects this fact and terminates. Voila, a while loop!
¹ Yes, including zero and negative integers, and "an integer" should be more properly read as "an arbitrary array of integers" (so the function can be applied at more than one power simultaneously).
f^:proposition^:_ is not a while loop. It's (almost) a while loop when proposition returns 1 or 0. It's some strange kind of while loop when proposition returns other results.
Let's take a simple monadic case.
f =: +: NB. Double
v =: 20 > ] NB. y less than 20
(f^:v^:_) 0 NB. steady case
0
(f^:v^:_) 1 NB. (f^:1) y, until (v y) = 0
32
(f^:v^:_) 2
32
(f^:v^:_) 5
20
(f^:v^:_) 21 NB. (f^:0) y
21
This is what's happening: every time that v y is 1, (f^:1) y is executed. The result of (f^:1) y is the new y and so on.
If y stays the same for two times in a row → output y and stop.
If v y is 0→ output y and stop.
So f^:v^:_ here, works like double while less than 20 (or until the result doesn't change)
Let's see what happens when v returns 2/0 instead of 1/0.
v =: 2 * 20 > ]
(f^:v^:_) 0 NB. steady state
0
(f^:v^:_) 1 NB. (f^:2) 1 = 4 -> (f^:2) 4 = 16 -> (f^:2) 16 = 64 [ -> (f^:0) 64 ]
64
(f^:v^:_) 2 NB. (f^:2) 2 = 8 -> (f^:2) 8 = 32 [ -> (f^:0) 32 ]
32
(f^:v^:_) 5 NB. (f^:2) 5 = 20 [ -> (f^:0) 20 ]
20
(f^:v^:_) 21 NB. [ (f^:0) 21 ]
21
You can have many kinds of "strange" loops by playing with v. (It can even return negative integers, to use the inverse of f).
I understand Big-O notation, but I don't know how to calculate it for many functions. In particular, I've been trying to figure out the computational complexity of the naive version of the Fibonacci sequence:
int Fibonacci(int n)
{
if (n <= 1)
return n;
else
return Fibonacci(n - 1) + Fibonacci(n - 2);
}
What is the computational complexity of the Fibonacci sequence and how is it calculated?
You model the time function to calculate Fib(n) as sum of time to calculate Fib(n-1) plus the time to calculate Fib(n-2) plus the time to add them together (O(1)). This is assuming that repeated evaluations of the same Fib(n) take the same time - i.e. no memoization is used.
T(n<=1) = O(1)
T(n) = T(n-1) + T(n-2) + O(1)
You solve this recurrence relation (using generating functions, for instance) and you'll end up with the answer.
Alternatively, you can draw the recursion tree, which will have depth n and intuitively figure out that this function is asymptotically O(2n). You can then prove your conjecture by induction.
Base: n = 1 is obvious
Assume T(n-1) = O(2n-1), therefore
T(n) = T(n-1) + T(n-2) + O(1) which is equal to
T(n) = O(2n-1) + O(2n-2) + O(1) = O(2n)
However, as noted in a comment, this is not the tight bound. An interesting fact about this function is that the T(n) is asymptotically the same as the value of Fib(n) since both are defined as
f(n) = f(n-1) + f(n-2).
The leaves of the recursion tree will always return 1. The value of Fib(n) is sum of all values returned by the leaves in the recursion tree which is equal to the count of leaves. Since each leaf will take O(1) to compute, T(n) is equal to Fib(n) x O(1). Consequently, the tight bound for this function is the Fibonacci sequence itself (~θ(1.6n)). You can find out this tight bound by using generating functions as I'd mentioned above.
Just ask yourself how many statements need to execute for F(n) to complete.
For F(1), the answer is 1 (the first part of the conditional).
For F(n), the answer is F(n-1) + F(n-2).
So what function satisfies these rules? Try an (a > 1):
an == a(n-1) + a(n-2)
Divide through by a(n-2):
a2 == a + 1
Solve for a and you get (1+sqrt(5))/2 = 1.6180339887, otherwise known as the golden ratio.
So it takes exponential time.
I agree with pgaur and rickerbh, recursive-fibonacci's complexity is O(2^n).
I came to the same conclusion by a rather simplistic but I believe still valid reasoning.
First, it's all about figuring out how many times recursive fibonacci function ( F() from now on ) gets called when calculating the Nth fibonacci number. If it gets called once per number in the sequence 0 to n, then we have O(n), if it gets called n times for each number, then we get O(n*n), or O(n^2), and so on.
So, when F() is called for a number n, the number of times F() is called for a given number between 0 and n-1 grows as we approach 0.
As a first impression, it seems to me that if we put it in a visual way, drawing a unit per time F() is called for a given number, wet get a sort of pyramid shape (that is, if we center units horizontally). Something like this:
n *
n-1 **
n-2 ****
...
2 ***********
1 ******************
0 ***************************
Now, the question is, how fast is the base of this pyramid enlarging as n grows?
Let's take a real case, for instance F(6)
F(6) * <-- only once
F(5) * <-- only once too
F(4) **
F(3) ****
F(2) ********
F(1) **************** <-- 16
F(0) ******************************** <-- 32
We see F(0) gets called 32 times, which is 2^5, which for this sample case is 2^(n-1).
Now, we want to know how many times F(x) gets called at all, and we can see the number of times F(0) is called is only a part of that.
If we mentally move all the *'s from F(6) to F(2) lines into F(1) line, we see that F(1) and F(0) lines are now equal in length. Which means, total times F() gets called when n=6 is 2x32=64=2^6.
Now, in terms of complexity:
O( F(6) ) = O(2^6)
O( F(n) ) = O(2^n)
There's a very nice discussion of this specific problem over at MIT. On page 5, they make the point that, if you assume that an addition takes one computational unit, the time required to compute Fib(N) is very closely related to the result of Fib(N).
As a result, you can skip directly to the very close approximation of the Fibonacci series:
Fib(N) = (1/sqrt(5)) * 1.618^(N+1) (approximately)
and say, therefore, that the worst case performance of the naive algorithm is
O((1/sqrt(5)) * 1.618^(N+1)) = O(1.618^(N+1))
PS: There is a discussion of the closed form expression of the Nth Fibonacci number over at Wikipedia if you'd like more information.
You can expand it and have a visulization
T(n) = T(n-1) + T(n-2) <
T(n-1) + T(n-1)
= 2*T(n-1)
= 2*2*T(n-2)
= 2*2*2*T(n-3)
....
= 2^i*T(n-i)
...
==> O(2^n)
Recursive algorithm's time complexity can be better estimated by drawing recursion tree, In this case the recurrence relation for drawing recursion tree would be T(n)=T(n-1)+T(n-2)+O(1)
note that each step takes O(1) meaning constant time,since it does only one comparison to check value of n in if block.Recursion tree would look like
n
(n-1) (n-2)
(n-2)(n-3) (n-3)(n-4) ...so on
Here lets say each level of above tree is denoted by i
hence,
i
0 n
1 (n-1) (n-2)
2 (n-2) (n-3) (n-3) (n-4)
3 (n-3)(n-4) (n-4)(n-5) (n-4)(n-5) (n-5)(n-6)
lets say at particular value of i, the tree ends, that case would be when n-i=1, hence i=n-1, meaning that the height of the tree is n-1.
Now lets see how much work is done for each of n layers in tree.Note that each step takes O(1) time as stated in recurrence relation.
2^0=1 n
2^1=2 (n-1) (n-2)
2^2=4 (n-2) (n-3) (n-3) (n-4)
2^3=8 (n-3)(n-4) (n-4)(n-5) (n-4)(n-5) (n-5)(n-6) ..so on
2^i for ith level
since i=n-1 is height of the tree work done at each level will be
i work
1 2^1
2 2^2
3 2^3..so on
Hence total work done will sum of work done at each level, hence it will be 2^0+2^1+2^2+2^3...+2^(n-1) since i=n-1.
By geometric series this sum is 2^n, Hence total time complexity here is O(2^n)
The proof answers are good, but I always have to do a few iterations by hand to really convince myself. So I drew out a small calling tree on my whiteboard, and started counting the nodes. I split my counts out into total nodes, leaf nodes, and interior nodes. Here's what I got:
IN | OUT | TOT | LEAF | INT
1 | 1 | 1 | 1 | 0
2 | 1 | 1 | 1 | 0
3 | 2 | 3 | 2 | 1
4 | 3 | 5 | 3 | 2
5 | 5 | 9 | 5 | 4
6 | 8 | 15 | 8 | 7
7 | 13 | 25 | 13 | 12
8 | 21 | 41 | 21 | 20
9 | 34 | 67 | 34 | 33
10 | 55 | 109 | 55 | 54
What immediately leaps out is that the number of leaf nodes is fib(n). What took a few more iterations to notice is that the number of interior nodes is fib(n) - 1. Therefore the total number of nodes is 2 * fib(n) - 1.
Since you drop the coefficients when classifying computational complexity, the final answer is θ(fib(n)).
It is bounded on the lower end by 2^(n/2) and on the upper end by 2^n (as noted in other comments). And an interesting fact of that recursive implementation is that it has a tight asymptotic bound of Fib(n) itself. These facts can be summarized:
T(n) = Ω(2^(n/2)) (lower bound)
T(n) = O(2^n) (upper bound)
T(n) = Θ(Fib(n)) (tight bound)
The tight bound can be reduced further using its closed form if you like.
It is simple to calculate by diagramming function calls. Simply add the function calls for each value of n and look at how the number grows.
The Big O is O(Z^n) where Z is the golden ratio or about 1.62.
Both the Leonardo numbers and the Fibonacci numbers approach this ratio as we increase n.
Unlike other Big O questions there is no variability in the input and both the algorithm and implementation of the algorithm are clearly defined.
There is no need for a bunch of complex math. Simply diagram out the function calls below and fit a function to the numbers.
Or if you are familiar with the golden ratio you will recognize it as such.
This answer is more correct than the accepted answer which claims that it will approach f(n) = 2^n. It never will. It will approach f(n) = golden_ratio^n.
2 (2 -> 1, 0)
4 (3 -> 2, 1) (2 -> 1, 0)
8 (4 -> 3, 2) (3 -> 2, 1) (2 -> 1, 0)
(2 -> 1, 0)
14 (5 -> 4, 3) (4 -> 3, 2) (3 -> 2, 1) (2 -> 1, 0)
(2 -> 1, 0)
(3 -> 2, 1) (2 -> 1, 0)
22 (6 -> 5, 4)
(5 -> 4, 3) (4 -> 3, 2) (3 -> 2, 1) (2 -> 1, 0)
(2 -> 1, 0)
(3 -> 2, 1) (2 -> 1, 0)
(4 -> 3, 2) (3 -> 2, 1) (2 -> 1, 0)
(2 -> 1, 0)
The naive recursion version of Fibonacci is exponential by design due to repetition in the computation:
At the root you are computing:
F(n) depends on F(n-1) and F(n-2)
F(n-1) depends on F(n-2) again and F(n-3)
F(n-2) depends on F(n-3) again and F(n-4)
then you are having at each level 2 recursive calls that are wasting a lot of data in the calculation, the time function will look like this:
T(n) = T(n-1) + T(n-2) + C, with C constant
T(n-1) = T(n-2) + T(n-3) > T(n-2) then
T(n) > 2*T(n-2)
...
T(n) > 2^(n/2) * T(1) = O(2^(n/2))
This is just a lower bound that for the purpose of your analysis should be enough but the real time function is a factor of a constant by the same Fibonacci formula and the closed form is known to be exponential of the golden ratio.
In addition, you can find optimized versions of Fibonacci using dynamic programming like this:
static int fib(int n)
{
/* memory */
int f[] = new int[n+1];
int i;
/* Init */
f[0] = 0;
f[1] = 1;
/* Fill */
for (i = 2; i <= n; i++)
{
f[i] = f[i-1] + f[i-2];
}
return f[n];
}
That is optimized and do only n steps but is also exponential.
Cost functions are defined from Input size to the number of steps to solve the problem. When you see the dynamic version of Fibonacci (n steps to compute the table) or the easiest algorithm to know if a number is prime (sqrt(n) to analyze the valid divisors of the number). you may think that these algorithms are O(n) or O(sqrt(n)) but this is simply not true for the following reason:
The input to your algorithm is a number: n, using the binary notation the input size for an integer n is log2(n) then doing a variable change of
m = log2(n) // your real input size
let find out the number of steps as a function of the input size
m = log2(n)
2^m = 2^log2(n) = n
then the cost of your algorithm as a function of the input size is:
T(m) = n steps = 2^m steps
and this is why the cost is an exponential.
Well, according to me to it is O(2^n) as in this function only recursion is taking the considerable time (divide and conquer). We see that, the above function will continue in a tree until the leaves are approaches when we reach to the level F(n-(n-1)) i.e. F(1). So, here when we jot down the time complexity encountered at each depth of tree, the summation series is:
1+2+4+.......(n-1)
= 1((2^n)-1)/(2-1)
=2^n -1
that is order of 2^n [ O(2^n) ].
No answer emphasizes probably the fastest and most memory efficient way to calculate the sequence. There is a closed form exact expression for the Fibonacci sequence. It can be found by using generating functions or by using linear algebra as I will now do.
Let f_1,f_2, ... be the Fibonacci sequence with f_1 = f_2 = 1. Now consider a sequence of two dimensional vectors
f_1 , f_2 , f_3 , ...
f_2 , f_3 , f_4 , ...
Observe that the next element v_{n+1} in the vector sequence is M.v_{n} where M is a 2x2 matrix given by
M = [0 1]
[1 1]
due to f_{n+1} = f_{n+1} and f_{n+2} = f_{n} + f_{n+1}
M is diagonalizable over complex numbers (in fact diagonalizable over the reals as well, but this is not usually the case). There are two distinct eigenvectors of M given by
1 1
x_1 x_2
where x_1 = (1+sqrt(5))/2 and x_2 = (1-sqrt(5))/2 are the distinct solutions to the polynomial equation x*x-x-1 = 0. The corresponding eigenvalues are x_1 and x_2. Think of M as a linear transformation and change your basis to see that it is equivalent to
D = [x_1 0]
[0 x_2]
In order to find f_n find v_n and look at the first coordinate. To find v_n apply M n-1 times to v_1. But applying M n-1 times is easy, just think of it as D. Then using linearity one can find
f_n = 1/sqrt(5)*(x_1^n-x_2^n)
Since the norm of x_2 is smaller than 1, the corresponding term vanishes as n tends to infinity; therefore, obtaining the greatest integer smaller than (x_1^n)/sqrt(5) is enough to find the answer exactly. By making use of the trick of repeatedly squaring, this can be done using only O(log_2(n)) multiplication (and addition) operations. Memory complexity is even more impressive because it can be implemented in a way that you always need to hold at most 1 number in memory whose value is smaller than the answer. However, since this number is not a natural number, memory complexity here changes depending on whether if you use fixed bits to represent each number (hence do calculations with error)(O(1) memory complexity this case) or use a better model like Turing machines, in which case some more analysis is needed.