Why is the complexity of n^2 logn + n(logn)^2 = O(n^2 (logn)^2) ?
I saw this as a solution to Cornell's Final exam paper (found online) but I'm not too sure if it is correct.
The statement n² log n + n (log n)² ∈ O(n² (log n)²) is valid. It's just not the best upper bound one could give on this function. But it's still true. Note that f(n) ∈ O(n² log n) ⇒ f(n) ∈ O(n² (log n)²). That's why people never (or shouldn't) write f(n) = O(g(n)), but should instead write f(n) ∈ O(g(n)).
On the other hand, the provided bound is a bit conservative, it could very well be because n² log n is smaller than n (log n)² for small values of n (n < 1). Or maybe, because it's part of a test, it could also be a way to test wether students fully understood what f(x) ∈ O(g(n)) means (a trick question, but an interesting trick question). Even though I would have added a question right before asking wether the following is a valid statement: is "n² log n + n (log n)² ∈ O(n² (log n))".
The more conservative bound could hypothetically make sense depending on the context (if n is very small and 1 could already be considered a "large value"), but usually if that's the case it would be specified explicitly.
If nothing specifies otherwise, people would assume that the best upper bound for this function is O(n² log n).
Let us consider the theory as explained in Wikipedia.
There it is written that if f1 is O(g1) and f2 is O(g2), f1 + f2 is O(max(g1, g2)).
Let's consider f1 = n² log n and f2 = n log² n.
Then f1 is O(n² log n) and f2 is O(n log² n).
Now, for n > 1, since n > log n, we can multiply both sides of the inequality by n log n to obtain n² log n > n log² n. This proves that O(n² log n) > O(n log² n) and, by the definition from above we have: O(n² log n).
Related
In big O notation of time complexity in algorithmic analysis, is O(n + k log n) the same as O(n log n) if k is larger than n? I am not entirely sure about this.
I am not 100% sure what you mean by N+KlogN. I'm used to seeing K used as a subset of N, for example "the top Kth set of items in N" which for large N it is common to simply return the top K items in N because then the Big-O time is NlogK which is much faster than NlogN (because K is a smaller number).
If you literally mean N+KlogN, then that would be more complex than simply NlogN as K adds to the number. For example, as K goes to zero you simply end up with NlogN, otherwise you get a greater than NlogN, which I hope would be obvious is more complex.
I hope that does something to answer the question. I confess I feel like I might be missing the point here and if so I apologize.
No, in the specific case you’re mentioning these are not the same. For example, consider this algorithm: given an array of length N and a number K ≥ N, do a linear scan over the array, then do K binary searches on the array. How much work is done here? Well, the linear search takes time O(N), and the K binary searches collectively take time O(K log N), so the total work done is O(N + K log N).
However, the work here is not O(N log N). Since K can be arbitrarily large, the value of K log N can exceed the value of N log N by an arbitrary amount. A different way of seeing this: a bound of O(N log N) means that the runtime depends purely on N and not on K. But that can’t be the case here, since cranking K way, way up definitely increases the runtime, independently of what N is.
Hope this helps!
I assume it as N + (K log N) where N is total count and K is the subset count. Now assuming K is very small compared to N (possibly a constant to get top K numbers from varying N) it reduces to linear time.
For example, to get top 100 items from array of 10000 elements
10000 + (100 * log (10000) base 2) = 10000 + 1300
Now when N is 20000, k log n changes to 1400
So as N increases linearly, the k log n increases in logarithmic manner reducing the overall complexity to linear.
O(n + (k log n)) is approximately O(n)
The obvious one is a constant on a linear term for example 2n, 4n and 8n are all just n or O(n).
But what about the exponential constant 1.6^n and 2^n. In this case the constant seems to have a greater effect on the time complexity.
Also there is not really a convenient way to write a catch all for exponential time complexity.
O(K^n) perhaps.
In this cheat sheet, they seem to use O(2^n) does that mean that all exponential complexities should be written that way?
Probably not.
You're right that 2n, 4n and 8n are all just O(n), and you're also right that O(1.6n) is not the same as O(2n). To understand why, we need to refer to the definition of big O notation.
The notation O(...) means a set of functions. A function f(n) is in the set O(g(n)) if and only if, for some constants c and n0, we have f(n) ≤ c * g(n) whenever n ≥ n0. Given this definition:
The function f(n) = 8n is in the set O(n) because if we choose c = 8 and n0 = 1, we have 8n ≤ 8 * n for all n ≥ 1.
The function f(n) = 2n is not in the set O(1.6n), because whichever c and n0 we choose, 2n > c * 1.6n for some sufficiently large n. We can choose n > log2 c + n0 log2 1.6 for a concrete counterexample.
Note however that f(n) = 1.6n is in the set O(2n), because 1.6n ≤ 1 * 2n for all n ≥ 1.
For a "catch-all" way of writing exponential complexity, you can write 2O(n). This includes exponential functions with arbitrary bases, e.g. the function f(n) = 16n since this equals 24n, and 4n is in the set O(n). It's an abuse of notation, since raising the number 2 to the power of a set of functions doesn't really make sense in this context, but it is common enough to be understood.
That is correct. The cheat sheet you linked to here can not show all the different complexities so it picks the most common ones.
Simply put, if you have a function growing at 3 ^ n. It can not be classified as 2 ^ n because it will break the definition of Big O.
The some what complex looking math that describes Big O is simply saying that it can't ever be bigger. And also ignore linear growth constants.
f(n) ≤ c * g(n) whenever n ≥ n0
What I am trying to do is to sort the following functions:
n, n^3, nlogn, n/logn, n/log^2n, sqrt(n), sqrt(n^3)
in increasing order of asymptotic growth.
What I did is,
n/logn, n/log^2n, sqrt(n), n, sqrt(n^3), nlogn, n^3.
1) Is my answer correct?
2) I know about the time complexity of the basic functions such as n, nlogn, n^2, but I am really confused on the functions like, n/nlogn, sqrt(n^3).
How should I figure out which one is faster or slower? Is there any way to do this with mathematical calculations?
3) Are the big O time complexity and asymptotic growth different thing?
I would be really appreciated if anyone blows up my confusion... Thanks!
An important result we need here is:
log n grows more slowly than n^a for any strictly positive number a > 0.
For a proof of the above, see here.
If we re-write sqrt(n^3) as n^1.5, we can see than n log n grows more slowly (divide both by n and use the result above).
Similarly, n / log n grows more quickly than any n^b where b < 1; again this is directly from the result above. Note that it is however slower than n by a factor of log n; same for n / log^2 n.
Combining the above, we find the increasing order to be:
sqrt(n)
n / log^2 n
n / log n
n
n log n
sqrt(n^3)
n^3
So I'm afraid to say you got only a few of the orderings right.
EDIT: to answer your other questions:
If you take the limit of f(n) / g(n) as n -> infinity, then it can be said that f(n) is asymptotically greater than g(n) if this limit is infinite, and lesser if the limit is zero. This comes directly from the definition of big-O.
big-O is a method of classifying asymptotic growth, typically as the parameter approaches infinity.
If an algorithm has time complexity O(n+m), and we know that m >= n (e.g., we're traversing a connected graph with n nodes and m edges). Then I think the following is true:
O(n+m) = O(m)
O(n log n + m) cannot by simplified
Is this correct?
Yes. To eliminate one of those terms you'd have to know more about n / m.
I'm sitting here with this assignment in a course on algorithms with massive data sets and the use of Little-Oh notation has got me confused, although I'm perfectly confident with Big-Oh.
I do not want a solution to the assignment, and as such I will not present it. Instead my question is how I interpret the time complexity o(log n)?
I know from the definition, that an algorithm A must grow asymptotically slower than o(log n), but I'm uncertain as to whether this means that the algorithm must be running in constant time or if it is still allowed to be log n under certain conditions, such that c = 1 or if it is really log (n-1).
Say an algorithm has a running time of O(log n) but in fact does two iterations and as such c = 2, but 2*log n is still O(log n), am I right when I say that this does not hold for Little-Oh?
Any help is greatly appreciated and if strictly needed for clarification, I will provide the assignment
To say the f is 'little-oh-of g' f = o(g), means that the quotient
|f(x)/g(x)|
approaches 0 as x approaches infinity. Referring to your example of o(log n), that class contains functions like log x / log (log x), sqrt(log x) and many more, so o(log x) definitely doesn't imply O(1). On the other hand, log (x/2) = log x - log 2, so
log (x/2) / log x = 1 - log 2 / log x -> 1
and log (x/2) is not in the class o(log x).
For Little-Oh, f(x) does not have to be smaller than g(x) for all x. It has to be smaller only after a certain value of x. (For your question, it is still allowed to be log n under certain conditions.)
For example:
let f(x) = 5000x and g(x) = x^2
f(x) / g(x) as x approaches infinity is 0, so f(x) is litte-o of g(x). However, at x = 1, f(x) is greater than g(x). Only after x becomes greater than 5000 will g(x) be bigger than f(x).
What little-o really tells us is that g(x) always grows at a faster rate than f(x). For example, look how much f(x) grew between x = 1 and x = 2:
f(1) = 5000
f(2) = 10000 - f(x) became twice as big
Now look at g(x) on the same interval:
g(1) = 1
g(2) = 4 - g(x) became four times bigger
This rate will increase even more at bigger values of x. Now, since g(x) increases at a faster rate and because we take x to the infinity, at some point it will become larger than f(x). But this is not what little-o is concerned with, it's all about the rate of change.