Do in place and space complexity O(1) mean different things? If yes, can someone explain the difference?
Space complexity of O(1) is a stronger requirement than in-place, because O(1) implies that the changes are done in place, but not the other way around.
You can make an in-place algorithm that has a space complexity above O(1). For example, the recursive re-heapifying algorithm of Heapsort is in-place, but its recursive implementation without tail call optimization has an O(log N) space complexity.
I think, Sergey's explanation is a bit confusing.
The terminology "in-place" is used for algorithms which doesn't require additional Data Structures to transform the input. However, to be considered as in-place algorithm, it must have maximum of O(log(N)) space complexity for additional pointers, pivots, stacks pointers. Any algorithm which requires more than O(log(N)) is not considered in-place algorithm
So, O(1) doesn't always mean it is in-place algorithm. For example, think of a method that returns random element from an array. You generate a random number with the index boundary of the input array, which requires O(1) space. And just return the matching value under the random index. But you are not transforming the input array. So, this is not in-place algorithm.
Related
I know, we should drop the non-dominant terms when calculating time complexity of an algorithm. I am wondering if we should drop them when calculating space complexity. For example, if I have a string of N letters, I'd like to:
construct a list of letters from this string -> Space: O(N);
sort this list -> Worst-case space complexity for Timsort (I use Python): O(N).
In this case, would the entire solution take O(N) + O(N) space or just O(N)?
Thank you.
Welcome to SO!
First of all, I think you do misunderstand complexity: Complexity is defined independently of constant factors. It depends only on the large scale behavior of the data set size N. Thus, O(N) + O(N) is the same complexity as O(N).
Thus, your question might have been:
If I construct a list of letters using an algorithm with O(N) space complexity, followed by a sort algorithm with O(N) space complexity, would the entire solution use twice as much space?
But this question cannot be answered, since a complexity does not give you any measure how much space is actually used.
A well-known example: A brute force sorting algorithm, BubbleSort, with time complexity O(N^2) is faster for small data sets than a very good sorting algorithm, QuickSort, with average time complexity O(Nlog(N)).
EDIT:
It is no contradiction, that one can compute a space complexity, and that it does not say how much space is actually used.
A simple example:
Say, for a certain problem algorithm 1 has linear space complexity O(n), and algorithm 2 space complexity O(n^2).
One could thus assume (but this is wrong) that algorithm 1 would always use less space than algorithm 2.
First, it is clear that for large enough n algorithm 2 will use more space than algorithm 1, because n^2 grows faster than n.
However, consider the case where n is small enough, say n = 1, and algorithm 1 is implemented on a computer that uses storage in doubles (64 bits), whereas algorithm 2 is implemented on a computer that uses bytes (8 bits). Then, obviously, the O(n^2) algorithm uses less space than the O(n) algorithm.
Is it possible to get some optimization on any algorithm used for getting the gcd of numbers in an array if the array is sorted?
Thanks!
So, let's see. The general method of finding the GCD of an array of numbers is:
result = a[0]
for i = 1 to length(a)-1
result = gcd(result, a[i])
So what's the complexity of the gcd algorithm? Well, that's a rather involved question. See, for example, Time complexity of Euclid's Algorithm
If we pretend, as posited in the accepted answer, that the GCD algorithm is constant time (i.e. O(1)), then the complexity of the loop above is O(n). That's a reasonable assumption for numbers that fit into computer registers. And if that's the case then spending O(n log n) time to sort the array would almost certainly be a loser.
But in reality the GCD calculation is linear in the number of digits in the two numbers. If your input data consists of lots of large numbers, it's possible that sorting the array first will give you an advantage. The reasoning is that the result of gcd(a, b) will by definition give you a number that's no larger than min(a,b). So by getting the GCD of the two smallest numbers first, you limit the number of digits you have to deal with. Whether that limiting will overcome the cost of sorting the array is unclear.
If the numbers are larger than will fit into a computer register (hundreds of digits), then the GCD calculation is more expensive. But then again, so is sorting.
So the answer to your question is that sorting will almost certainly increase the speed of calculating the GCD of an array of numbers, but whether the performance improvement will offset the cost of sorting is unclear.
I think the only way you'll know for sure is to test it with representative data.
Imagine T1(n) and T2(n) are running times of P1 and P2 programs, and
T1(n) ∈ O(f(n))
T2(n) ∈ O(g(n))
What is the amount of T1(n)+T2(n), when P1 is running along side P2?
The Answer is O(max{f(n), g(n)}) but why?
When we think about Big-O notation, we generally think about what the algorithm does as the size of the input n gets really big. A lot of times, we can fall back on some sort of intuition with math. Consider two functions, one that is O(n^2) and one that is O(n). As n gets really large, both algorithms increases without bound. The difference is, the O(n^2) algorithm grows much, MUCH faster than O(n). So much, in fact, that if you combine the algorithms into something that would be O(n^2+n), the factor of n by itself is so small that it can be ignored, and the algorithm is still in the class O(n^2).
That's why when you add together two algorithms, the combined running time is in O(max{f(n), g(n)}). There's always one that 'dominates' the runtime, making the affect of the other negligible.
The Answer is O(max{f(n), g(n)})
This is only correct if the programms run independently of each other. Anyhow, let's assume, this is the case.
In order to answer the why, we need to take a closer look at what the BIG-O-notation represents. Contrary to the way you stated it, it does not represent time but an upperbound on the complexity.
So while running both programms might take more time, the upperbound on the complexity won't increase.
Lets considder an example: P_1 calculates the the product of all pairs of n numbers in a vector, it is implemented using nested loops, and therefore has a complexity of O(n*n). P_2 just prints the numbers in a single loop and therefore has a complexity of O(n).
Now if we run both programms at the same time, the nested loops of P_1 are the most 'complex' part, leaving the combination with a complexity of O(n*n)
To complete task: find gcd(a,b) for integers a>b>0
Consider an algorithm that checks all of the numbers up to b and keeps track of the max number that divides a and b. It would use the % operator twice per check (for a and b). What would the complexity of this algorithm be?
I have not yet taken any formal CS courses in complexity theory (I will soon) so I am just looking for a quick answer.
The modulo operation is implemented in hardware, and it's pseudo O(1). Strictly speaking, it is not constant, but it depends on the number of bits of a and b. However, even then the number of bits is the same for all input sizes, so we usually ignore this factor.
The worst-case complexity of brute force GCD is just O(n) (also O(a), O(b), or O(min(a,b)); they're all the same), and it happens when when the GCD is either 1, a, or b.
Suppose for various input strings an algorithm generates binary string with same number of 0's and 1's. The output for two different input strings may or may not be the same. Can we say anything about the space complexity of the algorithm?
The question isn't quite right.
Kolmogorov complexity K(x) doesn't apply to programs, it applies to a string x.
More specifically, the Kolmogorov complexity of a string x is the minimum program length needed to compute a particular string x.
It has been formally proven that one can't compute the Kolmogorov complexity of a string. In practice, you can approximate via an upper bound.
The following paper by Ferbus-Zanda and Griorieff gives you the theory http://arxiv.org/abs/1010.3201
An intuitive way of thinking about such an approximate upper bound is to consider the length of a compression program that can decompress to a particular string.
Applying this to your problem, the string you describe is a random binary one, doubled. The input string acts a seed for the random number generator.
Ignoring the kolmogorov complexity part of your question, and just looking at space complexity (ie. memory footprint) aspect as #templatetypedef did, the criteria you mention are so loose that all you can say is that the lower space bound for the algorithm is O(1) and the upper bound O(n), where n is the output.
No, I don't believe so. Consider the algorithm "print 01," which requires space Θ(1), and the algorithm "double the length of the input string, then print 01," which requires space Θ(n). Both algorithms meet the criteria you've provided, so just given those criteria you can't say anything about the space complexity of the algorithm.
Hope this helps!