I was solving a time-complexity question on Interview Bit as given in the below image.
The answer given is Θ(theta)(logn) and I am not able to grasp how the logn term arrive here in the time complexity of this program.
Can someone please explain how the answer is theta of logn?
Theorem given any x, gcd(n, m) where n < fib(x) is recursive called equal or less than x times.
Note: fib(x) is fibonacci(x), where fib(x) = fib(x-1) + fib(x-2)
Prove
Basis
every n <= fib(1), gcd(n, m) is gcd(1,m) only recursive once
Inductive step
assume the theorem is hold for every number less than x, which means:
calls(gcd(n, m)) <= x for every n <= fib(x)
consider n where n <= fib(x+1)
if m > fib(x)
calls(gcd(n, m))
= calls(gcd(m, (n-m))) + 1
= calls(gcd(n-m, m%(n-m))) + 2 because n - m <= fib(x-1)
<= x - 1 + 2
= x + 1
if m <= fib(x)
calls(gcd(n, m))
= calls(gcd(m, (n%m))) + 1 because m <= fib(x)
<= x + 1
So the theorem also holds for x + 1, as mathematical induction, the theorem holds for every x.
Conclusion
gcd(n,m) is Θ(reverse fib) which is Θ(logn)
This algorithm generates a decreasing sequence of integer (m, n) pairs. We can try to prove that such sequence decays fast enough.
Let's say we start with m_1 and n_1, with m_1 < n_1.
At each step we take n_1 % m_1, which is < m_1, and repeat recursively on the pair m_2 = n_1 % m_1 and n_2 = m_1.
Now, let's say n_1 % m_1 = m_1 - p for some p where 0 < p < m_1.
We have max(m_2, n_2) = m_1 - p.
Let's take another step (m_2, n_2) -> (m_3, n_3), we can easily see that max(m_3, n_3) < p, but clearly it is also true that max(m_3, n_3) < m_1 - p as the sequence is strictly decreasing.
So we can write max(m_3, n_3) < min(m_1 - p, p), where min(m_1 - p, p) = m_1 / 2. This result expresses the fact that the sequence decreases geometrically, therefore the algorithm has to terminate in at most log_2(m_1) steps.
Related
I have a loop that runs n-2 times, what would be the time complexity in this case.
for(int m=1; m<arr.length-1; m++) {
}
I am not convinced for it to be O(n) because it will never run n times, not even in worst case scenarios.
O(n) just means "on the order of n". Specifically, the definition is that some function f(n) is O(g(n)) if there exist some k and c such that for all n > k, f(n) < c * g(n). In this case, set f(n) = n - 2 and g(n) = n, you can see that for k = 10 and c = 2, n < 2 * (n - 2) for all n > 10, so n - 2 is indeed O(n).
So if I have a loop like this?
int x, y, z;
for(int i = 0; i < n - 1; i++) {
for(int j = 0; j < n - 1 - i; j++){
x = 1;
y = 2;
z = 3;
}
}
so we start with the x, y, z definition so we have 4 operations there,
int i = 0 occurs once, i < n - 1 and i++ iterate n - 1 times, int j = 0, iterates n - 1 times and j < n - 1 - i and j++ iterates (n - 1) * (n - 1 - i) and xyz = 1 would iterate (n - 1) * (n - 1 - i) as well. So if I were to simplify this, would the above code run at O(n^2)?
so we start with the x, y, z definition so we have 4 operations there
This is not necessary, we need only count critical operations (i.e. in this case how often the loop body executes).
So if I were to simplify this, would the above code run at O(n²)?
A function T(n) is in O(g(n)) if T(n) <= c*g(n) (under the assumption n >= n0) for some constants c > 0, n0 > 0.
So for your code, the loop body is executed n - i times for every i, of which there are n. So we have:
Which is indeed true for c = 1/2, n0 = 1. Therefore T(n) ∈ O(n²).
You are correct that the complexity is O(n^2). There is more than one way to approach the question of why.
The formal way is to count the number of iterations of the inner loop, which will be n-1 the first time, then n-2, then n-3, ... all the way down to 1, giving a total of n*(n-1)/2 iterations, which is O(n^2).
An informal way is to say the outer loop runs O(n) times, and "on average", i is roughly n/2, so the inner loop runs on average about (n - n/2) = n/2 times, which is also O(n). So the total number of iterations is O(n) * O(n) = O(n^2).
With both of these techniques, it's not enough to just say that the loop body iterates O(n^2) times - we also need to check the complexity of the inner loop body. In this code, the body of the inner loop just does a few assignments, so it has a complexity of O(1). This means the overall complexity of the code is O(n^2) * O(1) = O(n^2). If instead the inner loop body did e.g. a binary search over an array of length n, then that would be O(log n) and the overall complexity of the code would be O(n^2 log n), for example.
Yes, you are right. Time complexity of this program will be O(n^2) at its worst case.
Problem 6: Find the complexity of the below program:
void function(int n)
{
int i = 1, s =1;
while (s <= n)
{
i++;
s += i;
printf("*");
}
}
Solution: We can define the terms ‘s’ according to relation si = si-1 + i. The value of ‘i’ increases by one for each iteration. The value contained in ‘s’ at the ith iteration is the sum of the first ‘i’ positive integers. If k is total number of iterations taken by the program, then while loop terminates if: 1 + 2 + 3 ….+ k = [k(k+1)/2] > n So k = O(√n).
Time Complexity of the above function O(√n).
FROM: https://www.geeksforgeeks.org/analysis-algorithms-set-5-practice-problems/
Looking it over and over.
Apparently they are saying the Time Complexity is O(√n). I don't understand how they are getting to this result, and i've tried looking at this problem over and over. Can anyone break it down into detail?
At the start of the while-loop, we have s = 1; i = 1, and n is some (big) number. In each step of the loop, the following is done,
Take the current i, and increment it by one;
Add this new value for i to the sum s.
It is not difficult to see that successive updates of i forms the sequence 1, 2, 3, ..., and s the sequence 1, 1 + 2, 1 + 2 + 3, .... By a result attributed to the young Gauss, the sum of the first k natural numbers 1 + 2 + 3 + ... + k is k(k + 1) / 2. You should recognise that the sequence s fits this description, where k indicates the number of iterations!
The while-loop terminates when s > n, which is now equivalent to finding the lowest iteration number k such that (k(k + 1) / 2) > n. Simplifying for the asymptotic case, this gives a result such that k^2 > n, which we can simplify for k as k > sqrt(n). It follows that this algorithm runs in a time proportional to sqrt(n).
It is clear that k is the first integer such that k(k+1)/2 > n (otherwise the loop would have stopped earlier).
Then k-1 cannot have this same property, which means that (k-1)((k-1)+1)/2 <= n or (k-1)k/2 <= n. And we have the following sequence of implications:
(k-1)k/2 <= n → (k-1)k <= 2n
→ (k-1)^2 < 2n ; k-1 < k
→ k <= sqrt(2n) + 1 ; solve for k
<= sqrt(2n) + sqrt(2n) ; 1 < sqrt(2n)
= 2sqrt(2)sqrt(n)
= O(sqrt(n))
I'm trying to prove that this formula (n2+1)/(n+1) is O(n)
As you know, we need to come up with n0 and C.
So I'm confused a little bit about how to choose an appropriate C since the equation here is division.
So with C=1, (n2+1) / (n+1) / n
(n2+n) / (n+n) / n >= (n2+1) /(n+1)
but I'm stuck here in how to simplify the division here.
As n tends to infinity your original equation becomes n^2/n which is equivalent to O(n)
Choosing c = 1:
(n^2 + 1)/(n + 1) <= 1*n definition of Big-Oh with c = 1
n^2 + 1 <= n^2 + n multiplying both sides by n + 1
1 <= n subtracting n^2 from both sides
n >= 1 rearranging
Therefore, the choice n0 = 1 works for c = 1.
for
f = n(log(n))^5
g = n^1.01
is
f = O(g)
f = 0(g)
f = Omega(g)?
I tried dividing both by n and i got
f = log(n)^5
g = n^0.01
But I am still clueless to which one grows faster. Can someone help me with this and explain the reasoning to the answer? I really want to know how (without calculator) one can determine which one grows faster.
Probably easiest to compare their logarithmic profiles:
If (for some C1, C2, a>0)
f < C1 n log(n)^a
g < C2 n^(1+k)
Then (for large enough n)
log(f) < log(n) + a log(log(n)) + log(C1)
log(g) < log(n) + k log(n) + log(C2)
Both are dominated by log(n) growth, so the question is which residual is bigger. The log(n) residual grows faster than log(log(n)), regardless of how small k or how large a is, so g would grow faster than f.
So in terms of big-O notation: g grows faster than f, so f can (asymptotically) be bounded from above by a function like g:
f(n) < C3 g(n)
So f = O(g). Similarly, g can be bounded from below by f, so g = Omega(f). But f cannot be bounded from below by a function like g, since g will eventually outgrow it. So f != Omega(g) and f != Theta(g).
But aaa makes a very good point: g does not begin to dominate over f until n becomes obscenely large.
I don't have a lot of experience with algorithm scaling, so corrections are welcome.
I would break this up into several easy, reusable lemmas:
Lemma 1: For a positive constant k, f = O(g) if and only if f = O(k g).
Proof: Suppose f = O(g). Then there exist constants c and N such that |f(n)| < c |g(n)| for n > N.
Thus |f(n)| < (c/k) (k |g(n)| ) for n > N and constant (c/k), so f = O (k g). The converse is trivially similar.
Lemma 2: If h is a positive monotonically increasing function and f and g are positive for sufficiently large n, then f = O(g) if and only if h(f) = O( h(g) ).
Proof: Suppose f = O(g). Then there exist constants c and N such that |f(n)| < c |g(n)| for n > N. Since f and g are positive for n > M, f(n) < c g(n) for n > max(N, M). Since h is monotonically increasing, h(f(n)) < c h(g(n)) for n > max(N, M), and lastly |h(f(n))| < c |h(g(n))| for n > max(N, M) since h is positive. Thus h(f) = O(h(g)).
The converse follows similarly; the key fact being that if h is monotonically increasing, then h(a) < h(b) => a < b.
Lemma 3: If h is an invertible monotonically increasing function, then f = O(g) if and only if f(h) + O(g(h)).
Proof: Suppose f = O(g). Then there exist constants c, N such that |f(n)| < c |g(n)| for n > N. Thus |f(h(n))| < c |g(h(n))| for h(n) > N. Since h(n) is invertible and monotonically increasing, h(n) > N whenever n > h^-1(N). Thus h^-1(N) is the new constant we need, and f(h(n)) = O(g(h(n)).
The converse follows similarly, using g's inverse.
Lemma 4: If h(n) is nonzero for n > M, f = O(g) if and only if f(n)h(n) = O(g(n)h(n)).
Proof: If f = O(g), then for constants c, N, |f(n)| < c |g(n)| for n > N. Since |h(n)| is positive for n > M, |f(n)h(n)| < c |g(n)h(n)| for n > max(N, M) and so f(n)h(n) = O(g(n)h(n)).
The converse follows similarly by using 1/h(n).
Lemma 5a: log n = O(n).
Proof: Let f = log n, g = n. Then f' = 1/n and g' = 1, so for n > 1, g increases more quickly than f. Moreover g(1) = 1 > 0 = f(1), so |f(n)| < |g(n)| for n > 1 and f = O(g).
Lemma 5b: n != O(log n).
Proof: Suppose otherwise for contradiction, and let f = n and g = log n. Then for some constants c, N, |n| < c |log n| for n > N.
Let d = max(2, 2c, sqrt(N+1) ). By the calculation in lemma 5a, since d > 2 > 1, log d < d. Thus
|f(2d^2)| = 2d^2 > 2d(log d) >= d log d + d log 2 = d (log 2d) > 2c log 2d > c log (2d^2) = c g(2d^2) = c |g(2d^2)| for 2d^2 > N, a contradiction. Thus f != O(g).
So now we can assemble the answer to the question you originally asked.
Step 1:
log n = O(n^a)
n^a != O(log n)
For any positive constant a.
Proof: log n = O(n) by Lemma 5a. Thus log n = 1/a log n^a = O(1/a n^a) = O(n^a) by Lemmas 3 (for h(n) = n^a), 4, and 1. The second fact follows similarly by using Lemma 5b.
Step 2:
log^5 n = O(n^0.01)
n^0.01 != O(log^5 n)
Proof: log n = O(n^0.002) by step 1. Then by Lemma 2 (with h(n) = n^5), log^5 n = O( (n^0.002)^5 ) = O(n^0.01). The second fact follows similarly.
Final answer:
n log^5 n = O(n^1.01)
n^1.01 != O(n log^5 n)
In other words,
f = O(g)
f != 0(g)
f != Omega(g)
Proof: Apply Lemma 4 (using h(n) = n) to step 2.
With practice these rules become "obvious" and second nature. and unless your test requires that you prove your answer you'll find yourself whipping through these kinds of big-O problems.
how about checking their intersections?
Solve[Log[n] == n^(0.01/5), n]
1809
{{n -> 2.72374}, {n -> 8.70811861815 10 }}
I cheated with Mathematica
you can also reason with derivatives,
In[71]:= D[Log[n], n]
1
-
n
In[72]:= D[n^(0.01/5), n]
0.002
------
0.998
n
consider what happens as n gets really large, change in first tends to zero, later function doesnt lose its derivative (exponent is greater than 0).
this tells you which is more complex theoretically.
however in the practical region, first function is going to grow faster.
This is not 100% mathematically kosher without proving something about logs, but here he goes:
f = log(n)^5
g = n^0.01
We take logs of both:
log(f) = log(log(n)^5)) = 5*log(log(n)) = O(log(log(n)))
log(g) = log(n^0.01) = 0.01*log(n) = O(log(n))
From this we see that the first one grows asymptotically slower, because it has a double log in it and logs grow slowly. An non-formal argument why this reasoning by taking logs is valid is this: log(n) tells you roughly how many digits there are in the number n. So if the number of digits of g is growing asymptotically faster than the number of digits of f, then surely the actual number g is growing faster than the number f!