I am having a little bit of difficulty analyzing the time complexity of this algorithm:
for i = 1 to n do
for j = 1 to n do
k = j
while k <= n do
k = k*3
end while
end for
end for
I know that the outer for loop will run n times, and the inner for loop will run n^2 times. However, for the while loop, I've narrowed the cost down to n^2 + (some factor)*n, but I don't know where to go from here. I'd appreciate the help.
We can immediately factor out the outer for-loop, since none of the inner logic depends on its iteration variable i.
Now let's look at the while loop. You have correctly deduced that it is logarithmic - but what variables does it depend on? After the m-th execution of this loop, the value of k is:
The loop breaks when this is greater than n, so the number of times max(m) it executes is:
The total complexity, equal to n times the combined complexity of the inner for-loop and this while-loop is therefore the following summation:
What do we do about the rounding brackets? If you round a number up to the nearest integer, the result always differs from the original number by less than 1. Therefore the difference can just be written as a constant O(1):
Where in:
(1), (2), (4) we used three of the logarithm rules,
(3) we used the definition of the factorial n! = n(n-1)(n-2)...1,
(4) we used Stirling's approximation.
Thus the total time complexity is:
And as I mentioned in the comments, there is no logarithmic term or factor.
EDIT numerical tests to confirm this result. JavaScript code:
function T(n) {
var m = 0;
for (var i = 1; i <= n; i++) {
for (var j = 1; j <= n; j++) {
var k = j;
while (k <= n) {
k *= 3;
m++;
}
}
}
return m;
}
Test results:
n T(n)
-----------------
1000 1498000
1500 3370500
2000 5992000
2500 9365000
3000 13494000
3500 18357500
4000 23984000
4500 30361500
5000 37475000
5500 45347500
6000 53976000
6500 63336000
7000 73472000
7500 84345000
8000 95952000
8500 108324000
9000 121482000
9500 135337000
10000 149960000
Plot of T(n) against n^2:
A very tidy straight line, which proves that T(n) is directly proportional to n^2 for large inputs.
Related
What is the running time complexity of fun()?
int fun(int n)
{
int count = 0;
for (int i = n; i > 0; i =i-2)
for (int j = 2; j < i; j=j*j)
for (int k=j; k>0; k=k/2)
count += 1;
return count;
}
is it O(n * lglgn * lglglgn)?
----
Edit:
j = loglog(i) times, big iterative value of j can be almost n(for example n=17,max(j)=16)
k= log(j), since the max value of j is at most n. then the max iterative times can be log(n)
so we can say that the big O of this question is O(n* lglgn * lgn)
Since the value of j and k depends on the previous iterative value (i, j), maybe there is better tight answer to this question.
We need to count this carefully, because the number of iterations of each inner loop depends in a non-trivial way on the outer loop's variable. Simply averaging for each loop and multiplying the results together will not give the right answer.
The outer loop runs O(n) times, because i counts from n down to 0 in constant steps.
The middle loop's values of j are 2, then 2*2 = 4, then 4*4 = 16, and on the m'th iteration, j = 2^2^m. The last iteration will be when 2^2^m >= i, in which case m >= log log i. So this runs O(log log i) times.
The innermost loop runs O(log j) times, because on the m'th iteration, k = j / 2^m. The last iteration will be when k <= 1, in which case m >= log j. So this runs O(log j) times.
However, it is not correct to multiply these together to get O(n * log log n * log log log n) - because i is not n on every iteration, and j is not log log n on every iteration. This gives an upper bound, but not a tight one. To calculate the true time complexity, you will need to write it as a double-summation, and simplify it algebraically.
As a simpler example to think about, consider the following code:
for(i = 1; i < n; i *= 2) {
for(j = 0; j < i; j += 1) {
// do something
}
}
The outer loop runs O(log n) times, and the inner loop runs O(i) times, but the overall complexity is actually O(n). To see this, count how many times // do something is reached; the first time the outer loop iterates it'll be 1, then it'll be 2, then 4, then 8, and so on up to n. This is a geometric progression with a sum <= 2n, giving a total number of steps which is O(n).
Note that if we naively multiply the two loops' complexities we get O(n log n) instead, which is an upper bound, but not a tight one.
Using Big O notation:
With the first outer loop we get O(N/2). You have a loop of N items and you are reducing it by 2 everytime, though you get a total of N/2 loops.
With the outter loop we get O(Log(I)).
With the most inner loop we have O(Log(J)), because you are dividing by 2 your iterator on every loop.
If we multiply the three complexities because they are nested:
O(N/2)*O(Log(I)) + O(Log(j)) ~ O(N/2*Log(I)*Log(J)) ~ O(N/2*Log^2(N)) ~ O(N*Log^2(N)).
We get a linearithmic complexity: O(N*Log^2(N))
So if I have a loop like this?
int x, y, z;
for(int i = 0; i < n - 1; i++) {
for(int j = 0; j < n - 1 - i; j++){
x = 1;
y = 2;
z = 3;
}
}
so we start with the x, y, z definition so we have 4 operations there,
int i = 0 occurs once, i < n - 1 and i++ iterate n - 1 times, int j = 0, iterates n - 1 times and j < n - 1 - i and j++ iterates (n - 1) * (n - 1 - i) and xyz = 1 would iterate (n - 1) * (n - 1 - i) as well. So if I were to simplify this, would the above code run at O(n^2)?
so we start with the x, y, z definition so we have 4 operations there
This is not necessary, we need only count critical operations (i.e. in this case how often the loop body executes).
So if I were to simplify this, would the above code run at O(n²)?
A function T(n) is in O(g(n)) if T(n) <= c*g(n) (under the assumption n >= n0) for some constants c > 0, n0 > 0.
So for your code, the loop body is executed n - i times for every i, of which there are n. So we have:
Which is indeed true for c = 1/2, n0 = 1. Therefore T(n) ∈ O(n²).
You are correct that the complexity is O(n^2). There is more than one way to approach the question of why.
The formal way is to count the number of iterations of the inner loop, which will be n-1 the first time, then n-2, then n-3, ... all the way down to 1, giving a total of n*(n-1)/2 iterations, which is O(n^2).
An informal way is to say the outer loop runs O(n) times, and "on average", i is roughly n/2, so the inner loop runs on average about (n - n/2) = n/2 times, which is also O(n). So the total number of iterations is O(n) * O(n) = O(n^2).
With both of these techniques, it's not enough to just say that the loop body iterates O(n^2) times - we also need to check the complexity of the inner loop body. In this code, the body of the inner loop just does a few assignments, so it has a complexity of O(1). This means the overall complexity of the code is O(n^2) * O(1) = O(n^2). If instead the inner loop body did e.g. a binary search over an array of length n, then that would be O(log n) and the overall complexity of the code would be O(n^2 log n), for example.
Yes, you are right. Time complexity of this program will be O(n^2) at its worst case.
So these are the for loops that I have to find the time complexity, but I am not really clearly understood how to calculate.
for (int i = n; i > 1; i /= 3) {
for (int j = 0; j < n; j += 2) {
... ...
}
for (int k = 2; k < n; k = (k * k) {
...
}
For the first line, (int i = n; i > 1; i /= 3), keeps diving i by 3 and if i is less than 1 then the loop stops there, right?
But what is the time complexity of that? I think it is n, but I am not really sure.
The reason why I am thinking it is n is, if I assume that n is 30 then i will be like 30, 10, 3, 1 then the loop stops. It runs n times, doesn't it?
And for the last for loop, I think its time complexity is also n because what it does is
k starts as 2 and keeps multiplying itself to itself until k is greater than n.
So if n is 20, k will be like 2, 4, 16 then stop. It runs n times too.
I don't really think I am understanding this kind of questions because time complexity can be log(n) or n^2 or etc but all I see is n.
I don't really know when it comes to log or square. Or anything else.
Every for loop runs n times, I think. How can log or square be involved?
Can anyone help me understanding this? Please.
Since all three loops are independent of each other, we can analyse them separately and multiply the results at the end.
1. i loop
A classic logarithmic loop. There are countless examples on SO, this being a similar one. Using the result given on that page and replacing the division constant:
The exact number of times that this loop will execute is ceil(log3(n)).
2. j loop
As you correctly figured, this runs O(n / 2) times;
The exact number is floor(n / 2).
3. k loop
Another classic known result - the log-log loop. The code just happens to be an exact replicate of this SO post;
The exact number is ceil(log2(log2(n)))
Combining the above steps, the total time complexity is given by
Note that the j-loop overshadows the k-loop.
Numerical tests for confirmation
JavaScript code:
T = function(n) {
var m = 0;
for (var i = n; i > 1; i /= 3) {
for (var j = 0; j < n; j += 2)
m++;
for (var k = 2; k < n; k = k * k)
m++;
}
return m;
}
M = function(n) {
return ceil(log(n)/log(3)) * (floor(n/2) + ceil(log2(log2(n))));
}
M(n) is what the math predicts that T(n) will exactly be (the number of inner loop executions):
n T(n) M(n)
-----------------------
100000 550055 550055
105000 577555 577555
110000 605055 605055
115000 632555 632555
120000 660055 660055
125000 687555 687555
130000 715055 715055
135000 742555 742555
140000 770055 770055
145000 797555 797555
150000 825055 825055
M(n) matches T(n) perfectly as expected. A plot of T(n) against n log n (the predicted time complexity):
I'd say that is a convincing straight line.
tl;dr; I describe a couple of examples first, I analyze the complexity of the stated problem of OP at the bottom of this post
In short, the big O notation tells you something about how a program is going to perform if you scale the input.
Imagine a program (P0) that counts to 100. No matter how often you run the program, it's going to count to 100 as fast each time (give or take). Obviously right?
Now imagine a program (P1) that counts to a number that is variable, i.e. it takes a number as an input to which it counts. We call this variable n. Now each time P1 runs, the performance of P1 is dependent on the size of n. If we make n a 100, P1 will run very quickly. If we make n equal to a googleplex, it's going to take a little longer.
Basically, the performance of P1 is dependent on how big n is, and this is what we mean when we say that P1 has time-complexity O(n).
Now imagine a program (P2) where we count to the square root of n, rather than to itself. Clearly the performance of P2 is going to be worse than P1, because the number to which they count differs immensely (especially for larger n's (= scaling)). You'll know by intuition that P2's time-complexity is equal to O(n^2) if P1's complexity is equal to O(n).
Now consider a program (P3) that looks like this:
var length= input.length;
for(var i = 0; i < length; i++) {
for (var j = 0; j < length; j++) {
Console.WriteLine($"Product is {input[i] * input[j]}");
}
}
There's no n to be found here, but as you might realise, this program still depends on an input called input here. Simply because the program depends on some kind of input, we declare this input as n if we talk about time-complexity. If a program takes multiple inputs, we simply call those different names so that a time-complexity could be expressed as O(n * n2 + m * n3) where this hypothetical program would take 4 inputs.
For P3, we can discover it's time-complexity by first analyzing the number of different inputs, and then by analyzing in what way it's performance depends on the input.
P3 has 3 variables that it's using, called length, i and j. The first line of code does a simple assignment, which' performance is not dependent on any input, meaning the time-complexity of that line of code is equal to O(1) meaning constant time.
The second line of code is a for loop, implying we're going to do something that might depend on the length of something. And indeed we can tell that this first for loop (and everything in it) will be executed length times. If we increase the size of length, this line of code will do linearly more, thus this line of code's time complexity is O(length) (called linear time).
The next line of code will take O(length) time again, following the same logic as before, however since we are executing this every time execute the for loop around it, the time complexity will be multiplied by it: which results in O(length) * O(length) = O(length^2).
The insides of the second for loop do not depend on the size of the input (even though the input is necessary) because indexing on the input (for arrays!!) will not become slower if we increase the size of the input. This means that the insides will be constant time = O(1). Since this runs in side of the other for loop, we again have to multiply it to obtain the total time complexity of the nested lines of code: `outside for-loops * current block of code = O(length^2) * O(1) = O(length^2).
The total time-complexity of the program is just the sum of everything we've calculated: O(1) + O(length^2) = O(length^2) = O(n^2). The first line of code was O(1) and the for loops were analyzed to be O(length^2). You will notice 2 things:
We rename length to n: We do this because we express
time-complexity based on generic parameters and not on the ones that
happen to live within the program.
We removed O(1) from the equation. We do this because we're only
interested in the biggest terms (= fastest growing). Since O(n^2)
is way 'bigger' than O(1), the time-complexity is defined equal to
it (this only works like that for terms (e.g. split by +), not for
factors (e.g. split by *).
OP's problem
Now we can consider your program (P4) which is a little trickier because the variables within the program are defined a little cloudier than the ones in my examples.
for (int i = n; i > 1; i /= 3) {
for (int j = 0; j < n; j += 2) {
... ...
}
for (int k = 2; k < n; k = (k * k) {
...
}
}
If we analyze we can say this:
The first line of code is executed O(cbrt(3)) times where cbrt is the cubic root of it's input. Since i is divided by 3 every loop, the cubic root of n is the number of times the loop needs to be executed before i is smaller or equal to 1.
The second for loop is linear in time because j is executed
O(n / 2) times because it is increased by 2 rather than 1 which
would be 'normal'. Since we know that O(n/2) = O(n), we can say
that this for loop is executed O(cbrt(3)) * O(n) = O(n * cbrt(n)) times (first for * the nested for).
The third for is also nested in the first for, but since it is not nested in the second for, we're not going to multiply it by the second one (obviously because it is only executed each time the first for is executed). Here, k is bound by n, however since it is increased by a factor of itself each time, we cannot say it is linear, i.e. it's increase is defined by a variable rather than by a constant. Since we increase k by a factor of itself (we square it), it will reach n in 2log(n) steps. Deducing this is easy if you understand how log works, if you don't get this you need to understand that first. In any case, since we analyze that this for loop will be run O(2log(n)) time, the total complexity of the third for is O(cbrt(3)) * O(2log(n)) = O(cbrt(n) *2log(n))
The total time-complexity of the program is now calculated by the sum of the different sub-timecomplexities: O(n * cbrt(n)) + O(cbrt(n) *2log(n))
As we saw before, we only care about the fastest growing term if we talk about big O notation, so we say that the time-complexity of your program is equal to O(n * cbrt(n)).
Assuming do_something is O(1), how do I calculate the time complexity of this function?
function run(n) {
for(var i = 1; i <= n; i++) {
var counter = 1;
while(counter <= n) {
for (var j = counter; j >= 1; j--) {
do_something();
}
counter = counter * 2;
}
}
}
I'm assuming the initial for loop means the that the complexity will be n and the inner while loop means log(n). Is this true?
How do I calculate the complexity of everything?
Thanks.
It's not trivial to calculate the complexity of everything because there are functions (e. g. collatz conjecture) where no exact analysis is known. In such a case you can do an empirical analysis where you guess which run time class would match best.
Related to your sample:
The outer for loop is called n times
The while loop is called log(n) times
The inner for loop is called counter times where counter grows 2^m. But counter is recalculated log(n) times
That means the inner exponential for loop (2^m) is called m=log(n) times.
So you have 2^(log n) = n log(2) = n
You can also do it vice versa. The inner exponential function is called log times. Means log(2^m) -> (2^n = 2^m) -> m = n.
You can also simply create a sum of 2^m from m=0 to m=log(n) which is 2n-1 -> O(n).
Then you have to multiply linear time of outer for loop with linear time of while loop (incl. inner for loop). So you have O(n*n) = O(n²) time.
The exact amount of steps can be calculated.
The outer loop runs n times
The while and inner-most loop is run counter times for each of the values of counter, these are 1, 2, 4, 8, ..., 2^(bitLength(n)-1)
which sums to 2^bitLength(n) - 1
where bitLength(n) = trunc(ln(n)/ln(2))+1
Multiplying these two, yields the exact amount of steps:
n * (2^bitLength(n) - 1)
The approximate complexity is O(n2), because O(2^bitLength(n) - 1) = O(n)
I am getting confused about how to analysis the time complexity within a nested while loop which divide into odd and even situation. could anyone help to explain how to deal with the situation?
i = 1
while (i < n) {
k = i
while ( k < n ) {
if ( k % 2 == 1 )
k ++
else
k = k + 0.01*n
}
i = i + 0.1*n
}
So in a problem like this, the factors 0.01 and 0.1 play a huge role.
First let's consider the inner while loop, if k is odd, we increment k by 1. If k is even, we increment k by one-hundredths of n. How man iterations can this inner while loop run?
Clearly if all iterations were of type-1(odd case), the inner while loop would run n-k times, and similarly if all the iterations were of type-2(even case), the inner while loop would run atmost a 100 times(as we increment the value of k by one-hundredths of n each time).
Given value of k, the number of iterations of the inner while loop is:
max(n-k,100). From now on, we will assume the value of n-k to be greater than 100 always, without loss of generality.
Okay, how does the outer loop iterate? In each iteration of the outer loop, the value of i increases by one-tenths of n each time, so the outer while will run at most 10 times.
Making the running times explicit and calculating the overall running time:
Running time for first iteration of outer loop : n-k
Running time for second iteration of outer loop : + n-(k+0.1*n)
+ n-(k+0.2*n)
...
+ n-(k+0.9*n)
-----------
= 10n-10k-(4.5)n
Plugging in k=1(as this is the start value of k),
10n-10-4.5n = 5.5 n -10 = O(n)
Hence complexity is O(n) time.