Why because of the i=i*2 is the runtime of the loop below considered O(logN)?
for (int i = 1; i <= N;) {
code with O(1);
i = i * 2;
}
Look at 1024 = 210. How many times do you have to double the number 1 to get 1024?
Times 1 2 3 4 5 6 7 8 9 10
Result 2 4 8 16 32 64 128 256 512 1024
So you would have to run your doubling loop ten times to get 210 And in general, you have to run your doubling loop n times to get 2n. But what is n? It's the log2 2n, so in general if n is some power of 2, the loop has to run log2n times to reach it.
To have the algorithm in O(logN), for any N it would need (around) log N steps. In the example of N=32 (where log 32 = 5) this can be seen:
i = 1 (start)
i = 2
i = 4
i = 8
i = 16
i = 32 (5 iterations)
i = 64 (abort after 6 iterations)
In general, after x iterations, i=2^x holds. To reach i>N you need x = log N + 1.
PS: When talking about complexities, the log base (2, 10, e, ...) is not relevant. Furthermore, it is not relevant if you have i <= N or i < N as this only changes the number of iterations by one.
You can prove it pretty simply.
Claim:
For the tth (0 base) iteration, i=2^t
Proof, by induction.
Base: 2^0 = 1, and indeed in the first iteration, i=1.
Step: For some t+1, the value of i is 2*i(t) (where i(t) is the value of i in the t iteration). From Induction Hypothesis we know that i(t)=2^t, and thus i(t+1) = 2*i(t) = 2*2^t = 2^(t+1), and the claim holds.
Now, let's examine our stop criteria. We iterate the loop while i <= N, and from the above claim, that means we iterate while 2^t <= N. By Doing log_2 on both sides, we get log_2(2^t) <= log_2(N), and since log_2(2^t) = t, we get that we iterate while t <= log_2(N) - so we iterate Theta(log_2(N)) times. (And that concludes the proof).
i starts in 1. In each iteration you multiply i by 2, so in the K-th iteration, i will be 2K-1.
After a number K of iterations, 2K-1 will be bigger than (or to) N.
this means N ā¤ 2K-1
this means log2(N) ā¤ K-1
K-1 will be the number of iterations your loop will run, and since K-1 is greater or equal to log(N), your algorithm is logarithmic.
Related
f(n):
s = 1
m = n
while m >= 1:
for j in 1,2,3, .., m:
s = s + 1
m = m / 2
My attempt
How many times does the statement s = s + 1 run? Well, let's try a number or two for n and see what we get.
n = 32
m = n = 32
m = 32 => inner for loop, loops 32 times
m = 16 => -.- 16 times
m = 8 => -.- 8 times
m = 4 => -.- 4 times
m = 2 => -.- 2 times
m = 1 => -.- 1 time
In total 16+8+4+2+1 = 31 times for n = 32
Doing the same for n = 8 gives 15 times
Doing the same for n = 4 gives 7 times
Doing the same for n = 64 gives 127 times
Notice how it always seemingly is 2 * n - 1, I believe this to be no coincidence because what we're really observing is that the total amount of times the statement gets executed is equal to the geometric sum sum of 2^k from k=0 to k=log(n) which has a closed form solution 2n + 1 given that we're using base 2 logarithm.
As such, my guess is that the complexity is O(n).
Is this correct?
Well what you just observed is true indeed. We can prove it mathematically as well.
Inner loop execution for first iteration -> n
Inner loop execution for second iteration -> n/2
Inner loop execution for third iteration -> n/(2^2)
and so on....
Therfore total time is (n + (n/2) + (n/(2^2)) + ... + (n/(2^k))) where n/(2^k) should be equal to 1 which implies k = log(n). Taking n common from all terms will give us a GP And now using GP formulae in above series , total time will come out to be 1(1 - ((1/2)^k)) / (1 - 1/2). putting value of k = log(n), we will get total time as 2*(n-1).
Note -:
This gives us time complexity of O(n).
log is of base 2.
I have some difficulties in understanding the time complexity analysis for one solution for the Happy Number Question from Leet code, for my doubts on complexity analysis, I marked them in bold and really appreciate your advice
Here is the question:
Link: https://leetcode.com/problems/happy-number/
Question:
Write an algorithm to determine if a number is "happy".
A happy number is a number defined by the following process: Starting with any positive integer, replace the number by the sum of the squares of its digits, and repeat the process until the number equals 1 (where it will stay), or it loops endlessly in a cycle which does not include 1. Those numbers for which this process ends in 1 are happy numbers.
Example:
Input: 19
Output: true
Explanation:
1^2(square of 1) + 9^2 = 82
8^2 + 2^2 = 68
6^2 + 8^2 = 100
1^2 + 0^2 + 0^2 = 1
Here is the code:
class Solution(object):
def isHappy(self, n):
#getnext function will compute the sum of square of each digit of n
def getnext(n):
totalsum = 0
while n>0:
n,v = divmod(n,10)
totalsum+=v**2
return totalsum
#we declare seen as a set to track the number we already visited
seen = set()
#we stop checking if: either the number reaches one or the number was visited #already(ex.a cycle)
while n!=1 and (n not in seen):
seen.add(n)
n = getnext(n)
return n==1
Note: feel free to let me know if I need to explain how the code works
Time Complexity Analysis:
Time complexity : O(243 * 3 + logN + loglogN + log loglog N)...=O(logN).
Finding the next value for a given number has a cost of O(log n)because we are processing each digit in the number, and the number of digits in a number is given by logN.
My doubt: why the number of digits in a number is given by logN? what is N here? the value of a specific number or something else?
To work out the total time complexity, we'll need to think carefully about how many numbers are in the chain, and how big they are.
We determined above that once a number is below 243, it is impossible for it to go back up above 243.Therefore, based on our very shallow analysis we know for sure that once a number is below 243, it is impossible for it to take more than another 243 steps to terminate.
Each of these numbers has at most 3 digits. With a little more analysis, we could replace the 243 with the length of the longest number chain below 243, however because the constant doesn't matter anyway, we won't worry about it.
My doubt: I think the above paragraph is related to the time complexity component of 243*3, but I cannot understand why we multiply 243 by 3
For an n above 243, we need to consider the cost of each number in the chain that is above 243. With a little math, we can show that in the worst case, these costs will be O(log n) + O(log log n) + O(log log log N)... Luckily for us, the O(logN) is the dominating part, and the others are all tiny in comparison (collectively, they add up to less than logN), so we can ignore them. My doubt: what is the reasoning behind O(log log n) O(log log log N) for an n above 243?
Well, my guess for the first doubt is that the number of digits of a base 10 number is given by it's value (N) taken to the logarithm at base 10, rounded down. So for example, 1023 would have floor(log10(1023)) digits, which is 3. So yes, the N is the value of the number. the log in time complexity indicates a logarithm, not specifically that of base 2 or base e.
As for the second doubt, it probably has to do with the work required to reduce a number to below 243, but I am not sure. I'll edit this answer once I work that bit out.
Let's say N has M digits. Than getnext(N) <= 81*M. The equality happens when N only has 9's.
When N < 1000, i.e. at most 3 digits, getnext(N) <= 3*81 = 243. Now, you will have to call getnext(.) at most O(243) times to figure out if N is indeed happy.
If M > 3, number of digits of getnext(N) must be less than M. Try getnext(9999), getnext(99999), and so on [1].
Notes:
[1] Adding a digit to N can make it at most 10*N + 9, i.e. adding a 9 at the end. But the number of digits increases to M+1 only. It's a logarithmic relationship between N and M. Hence, the same relationship holds between N and 81*M.
Using the Leetcode solution
class Solution {
private int getNext(int n) {
int totalSum = 0;
while (n > 0) {
int d = n % 10;
n = n / 10;
totalSum += d * d;
}
return totalSum;
}
public boolean isHappy(int n) {
Set<Integer> seen = new HashSet<>();
while (n != 1 && !seen.contains(n)) {
seen.add(n);
n = getNext(n);
}
return n == 1;
}
}
}
O(243*3) for n < 243
3 is the max number of digits in n
e.g. For n = 243
getNext() will take a maximum of 3 iterations because there are 3 digits for us to loop over.
isHappy() can take a maximum of 243 iterations to find a cycle or terminate, because we can store a max of 243 numbers in our hash set.
O(log n) + O(log log n) + O(log log log N)... for n > 243
1st iteration + 2nd iteration + 3rd iteration ...
getNext() will be called a maximum of O(log n) times. Because log10 n is the number of digits.
isHappy() will be called a maximum of 9^2 per digit. This is the max we can store in the hash set before we find a cycle or terminate.
First Iteration
9^2 * number of digits
O(81*(log n)) drop the constant
O(log n)
+
Second Iteration
O(log (81*(log n))) drop the constant
O(log log n)
+
Third Iteration
O(log log log N)
+
ect ...
Function f(n)
s = 0
i = 1
while i < 7n^1/2 do
j = i
while j > 5 do
s = s + i -j
j = j -2
end
i = 5i
end
return s
end f
I am trying to solve the running time for big theta with the code above. I have been looking all over the place for something to help me with an example, but everything is for loops or only one while loop. How would you go about this problem with nested while loops?
Let's break this down into two key points:
i starts from 1, and is self-multiplied by 5, until it is greater than or equal to 7 sqrt(n). This is an exponential increase with logarithmic number of steps. Thus we can change the code to the following equivalent:
m = floor(log(5, 7n^(1/2)))
k = 0
while k < m do
j = 5^k
// ... inner loop ...
end
For each iteration of the outer loop, j starts from i, and decreases in steps of 2, until it is less than or equal to 5. Note that in the first execution of the outer loop i = 1, and in the second i = 5, so the inner loop is not executed until the third iteration. The loop limit means that the final value of j is 7 if k is odd, and 6 if even (you can check this with pen and paper).
Combining the above steps, we arrive at:
First loop will do 7 * sqrt(n) iterations. Exponent 1/2 is the same as sqrt() of a number.
Second loop will run m - 2 times since first two values of i are 1 and 5 respectively, not passing the comparison.
i is getting an increment of 5i.
Take an example where n = 16:
i = 1, n = 16;
while( i < 7 * 4; i *= 5 )
//Do something
First value of i = 1. It runs 1 time. Inside loop will run 0 times.
Second value of i = 5. It runs 2 times. Inside loop will run 0 times.
Third value of i = 25. It runs 3 times. Inside loop will run 10 times.
Fourth value of i = 125. It stops.
Outer iterations are n iterations while inner iterations are m iterations, which gives O( 7sqrt(n) * (m - 2) )
IMO, is complex.
The outer loop executes n times while the inner loop executes ? So the total time is n*something.
Do i need to learn summation,if yes then any book to refer?
for(int i=1;i<=n;i++)
for(int j=1;j<=n;j+=i)
printf("*");
This question can be approached by inspection:
n = 16
i | j values | # terms
1 | 1, 2, ..., 16 | n
2 | 1, 3, 5, ..., 16 | n / 2
.. | .. | n / 3
16 | 16 | n / n
In the above table, i is the outer loop value, and j values show the iterations of the inner loop. By inspection, we can see that the loops will take n * (1 + 1/2 + 1/3 + ... + 1/n) steps. This is a bounded harmonic series. As this Math Stack Exchange article shows, there is no closed form for the above expression in terms of n. However, as this SO article shows, there is an upper bound of O(n*ln(n)).
So, the running time for your two loops is O(n*ln(n)).
I believe the time complexity of that is O(n*log(n)). Here is why:
Let us pick some arbitrary natural number i and see how many steps the inner loop takes for this given i. Well for this i, you are going from j=1 to j<=n with a jump of i in between. So basically you are doing this summation many steps:
summation = 1 + (1+i) + (1+2i) + ... (1+ki)
where k is the largest integer such that 1+ki <= n. That is, k is the number of steps and this is what we want to solve for. Well we can solve for k in the equality resulting in k <= (n-1)/i and thus k = ā(n-1)/iā. That is, k is the floor function/integer division of (n-1)/i. Since we are dealing with time complexities, this floor function doesn't matter so we will just say k = n/i for simplicity. This is the number of steps that the inner loop will take for a given i. So we basically need to add all these for i = 1 to i <= n.
So numsteps will be this addition:
numsteps = n/1 + n/2 + n/3 + ... n/n
= n(1 + 1/2 + 1/3 + ... 1+n)
So we need to find the sum of 1 + 1/2 + ... 1/n to finish this. There is actually no good closed form for this sum but it is on the order of ln(n). You can read more about this here. You can also guess this since the integral from 1 to n of 1/x is ln(n). Again, since we are dealing with time complexity, we can just use ln(n) to represent its complexity. Thus we have:
numsteps = n(ln(n))
And so the time complexity is O(n*log(n)).
Edit: My bad, i was calculating the sum :P
I have to find the time complexity of the following program:
function(int n)
{
for(int i=0;i<n;i++) //O(n) times
for(int j=i;j<i*i;j++) //O(n^2) times
if(j%i==0)
{ //O(n) times
for(int k=0;k<j;k++) //O(n^2) times
printf("8");
}
}
I analysed this function as follows:
i : O(n) : 1 2 3 4 5
j : : 1 2..3 3..8 4..15 5..24 (values taken by j)
O(n^2): 1 2 6 12 20 (Number of times executed)
j%i==0 : 1 2 3,6 4,8,12 5,10,15,20 (Values for which the condition is true)
O(n) : 1 1 2 3 4
k : 1 2 3,6 4,8,12 5,10,15,20 (Number of times printf is executed)
Total : 1 2 9 24 50 (Total)
However I am unable to bring about any conclusions since I don't find any correlation between $i$ which is essentially O(n) and Total of k (last line). In fact I don't understand if we should be looking at the time complexity in terms of number of times printf is executed since that will neglect O(n^2) execution of j-for loop. The answer given was O(n^5) which I presume is wrong but then whats correct? To be more specific about my confusion I am not able to figure out how that if(j%i==0) condition have effect on the overall runtime complexity of the function.
The answer is definitely not O(n^5). It can be seen very easily. Suppose your second inner loop always runs n^2 times and your innermost loop always runs n times, even then total time complexity would be O(n^4).
Now let us see what is actual time complexity.
1.The outermost loop always runs O(n) times.
2.Now let us see how many times second inner loop runs for a single iteration of outer loop:
The loop will run
0 time for i = 0
0 time for i = 1
2 times for i = 2
....
i*i - i times for j = i.
i*i - i is O(i^2)
3. Coming to the innermost loop, it runs only when j is divisble by i and j varies from i to i*i-1.
This means j goes through i*1, i*2 , i*3 ..... till last multiple of i less than i*i. Which is clearly O(i), Hence for a single iteration of second inner loop innermost loop runs O(i) times, this means total iterations of two inner loops is O(i^3).
Summing up O(i^3) for i = 0 to n-1 will definitely give a term that is O(n^4).
Therefore, the correct time complexity is O(n^4).