When using "Big-Oh Notation", I understand that I should ignore constants and recognize that certain terms "dominate" others. That is, O(1) < O(logn) < O(n) < O(nlogn) < O(n^2) < O(2^n) < O(n!). I having a little trouble using this knowledge to prove the statement above, as a still seems to stick around even though its a constant?
Related
i get confused after i ready many articles about complexity specially with complexity of condition in many website i been looking to know my solution for my answer for 1 hour but none of website provide it so i just want to know what is the best when i use conditions like
int x = 10;
if(x<1)
cout<<"done";
else
for(int i=0;i<n;i++){
cout<<"done";
}
i know that in this case we will choice the higher complexity of both of them and it will be big(O) which is O(N)
but what about if i use else if is it will be the same as i think of above code ?
int x = 10;
if(x<1)
cout<<"done";
else if(x> 20)
for(int i=0;i<n;i++){
cout<<"done";
}
else
for(int i=0;i<n;i++){
cout<<"done";
}
does it make this example O(N) too like the above example
please if you have any resources or links that provide good content of algorithm complexity write it after you answer my question , thank you
It remains O(n). Think of big O as "as n changes, how does the runtime length of the algorithm change?" Since n changes the number of iterations of a regular loop, you always add one loop iteration for each addition to n, so it's linear (O(n)).
I was trying the time complexity mcq questions given in codechef under practice for Data Structures and Algorithms. One of the questions had a line a*< a[i]. What does that line mean?
I know that if there wasn't an and statement the complexity would have been O(n^2). But the a*< is completely alien to me. I searched for it in the internet but all I got was about the a star algorithm and asterisks! I tried running the program in python with a print statement but it says that * is invalid. Does that mean something like a pointer to the 1st element in the array or something?
Find the time complexity of the following function
n = len(a)
j = 0
for i =0 to n-1:
while (j < n and a* < a[j]):
j += 1
The answer is given as O(n). But there are nested loops so it is supposed to be O(n^2).Help required! Thanks
It doesn't actually matter what a* means. The question is to determine the time complexity of the algorithm. Notice that although there are two nested loops, the inner while loop isn't a full independent loop. Its index is j, which starts at 0 and is only ever incremented, with an upper bound of n. So the inner loop can only run a maximum of n times in total. This means that the overall complexity is only O(n).
I was recently asked an interview question about testing the validity of a Sudoku board. A basic answer involves for loops. Essentially:
for(int x = 0; x != 9; ++x)
for(int y = 0; y != 9; ++y)
// ...
Do this nested for loops to check the rows. Do it again to check the columns. Do one more for the sub-squares but that one is more funky because we're dividing the suoku board into sub-boards so we end end up more than two nested loops, maybe three or four.
I was later asked the complexity of this code. Frankly, as far as I'm concerned, all the cells of the board are visited exactly three times so O(3n). To me, the fact that we have nested loops doesn't mean this code is automatically O(n^2) or even O(n^highest-nesting-level-of-loops). But I have suspicion that that's the answer the interviewer expected...
Posed another way, what is the complexity of these two pieces of code:
for(int i = 0; i != n; ++i)
// ...
and:
for(int i = 0; i != sqrt(n); ++i)
for(int j = 0; j != sqrt(n); ++j)
// ...
Your general intuition is correct. Let's clarify a bit about Big-O notation:
Big-O gives you an upper bound for the worst-case (time) complexity for your algorithm, in relation to n - the size of your input. In essence, it is a measurement of how the amount of work changes in relation to the size of the input.
When you say something like
all the cells of the board are visited exactly three times so O(3n).
you are implying that n (the size of your input) is the the number of cells in the board and therefore visiting all cells three times would indeed be an O(3n) (which is O(n)) operation. If this is the case you would be correct.
However usually when referring to Sudoku problems (or problems involving a grid in general), n is taken to be the number of cells in each row/column (an n x n board). In this case, the runtime complexity would be O(3n²) (which is indeed equal to O(n²)).
In the future, it is perfectly valid to ask your interviewer what n is.
As for the question in the title (Is a nested for loop automatically O(n^2)?) the short answer is no.
Consider this example:
for(int i = 0 ; i < n ; i++) {
for(int j = 0 ; j < n ; j * 2) {
... // some constant time operation
}
}
The outer loops makes n iterations while the inner loop makes log2(n) iterations - therefore the time complexity will be O(nlogn).
In your examples, in the first one you have a single for-loop making n iterations, therefore a complexity of (at least) O(n) (the operation is performed an order of n times).
In the second one you two nested for loops, each making sqrt(n) iterations, therefore a total runtime complexity of (at least) O(n) as well. The second function isn't automatically O(n^2) simply because it contains a nested loop. The amount of operations being made is still of the same order (n) therefore these two examples have the same complexity - since we assume n is the same for both examples.
This is the most crucial point to sail home. To compare between the performance of two algorithms, you must be using the same input to make the comparison. In your sudoku problem you could have defined n in a few different ways, and the way you did would directly affect the complexity calculation of the problem - even if the amount of work is all the same.
*NOTE - this is unrelated to your question, but in the future avoid using != in loop conditions. In your second example, if log(n) is not a whole number, the loop could run forever, depending on the language and how it is defined. It is therefore recommended to use < instead.
It depends on how you define the so-called N.
If the size of the board is N-by-N, then yes, the complexity is O(N^2).
But if you say, the total number of grids is N (i.e., the board id sqrt(N)-by-sqrt(N)), then the complexity is O(N), or 3O(N) if you mind the constant.
I've been asked a question that seems strange to me. Given the following two equalities, which is true, and which is false (or are they both either true or false)?
O(n^2) = O(n^3)
O(n^3) = O(n^2)
To me, this question seems absurd, since O(f(n)) just means that for some time function T(n), lim as n -> infty of T(n) <= c * f(n).
O(f(n)) can be thought of as the class of all functions whose growth is bounded above by f(n). Taking into account this, the two options are false.
Martinus gives a good example of a code where compiler optimizes the code at run-time by calculating out multiplication:
Martinus code
int x = 0;
for (int i = 0; i < 100 * 1000 * 1000 * 1000; ++i) {
x += x + x + x + x + x;
}
System.out.println(x);
His code after Constant Folding -compiler's optimization at compile-time (Thanks to Abelenky for pointing that out)
int x = 0;
for (int i = 0; i < 100000000000; ++i) {
x += x + x + x + x + x;
}
System.out.println(x);
This optimization technique seems to be a trivial in my opinion.
I guess that it may one of the techniques Sun started to keep trivial recently.
I am interested in two types of optimizations made by compilers:
optimizations which were omitted in today's compilers as trivial such as in Java's compiler at run-time
optimizations which are used by the majority of today's compilers
Please, put each optimization technique to a separate answer.
Which techniques have compilers used in 90s (1) and today (2)?
Just buy the latest edition of the Dragon Book.
How about loop unrolling?:
for (i = 0; i < 100; i++)
g ();
To:
for (i = 0; i < 100; i += 2)
{
g ();
g ();
}
From http://www.compileroptimizations.com/. They have many more - too many for an answer per technique.
Check out Trace Trees for a cool interpreter/just-in-time optimization.
The optimization shown in your example, of collapsing 100*1000*1000*1000 => 100000000000 is NOT a run-time optimization. It happens at compile-time. (and I wouldn't even call it an optimization)
I don't know of any optimizations that happen at run-time, unless you count VM engines that have JIT (just-in-time) compiling.
Optimizations that happen at compile-time are wide ranging, and frequently not simple to explain. But they can include in-lining small functions, re-arranging instructions for cache-locality, re-arranging instructions for better pipelining or hyperthreading, and many, many other techniques.
EDIT: Some F*ER edited my post... and then down-voted it. My original post clearly indicated that collapsing multiplication happens at COMPILE TIME, not RUN TIME, as the poster suggested. Then I mentioned I don't really consider collapsing constants to be much of an optimization. The pre-processor even does it.
Masi: if you want to answer the question, then answer the question. Do NOT edit other people's answers to put in words they never wrote.
Compiler books should provide a pretty good resource.
If this is obvious, please ignore it, but you're asking about low-level optimizations, the only ones that compilers can do. In non-toy programs, high-level optimizations are far more productive, but only the programmer can do them.