In Algorithms, 4th edition by Robert Sedgewick, the time complexity table for different algorithms is given as:
Based on this table, the searching time complexity of a BST is N, and of binary search in and of itself is logN.
What is the difference between the two? I have seen explanations about these separately and they made sense, however, I can't seem to understand why the searching time complexity of a BST isn't logN, as we are searching by continually breaking the tree in half and ignoring the other parts.
From binary-search-trees-bst-explained-with-examples
...on average, each comparison allows the operations to skip about half of the tree, so that each lookup, insertion or deletion takes time proportional to the logarithm of the number of items stored in the tree, O(log n) . However, some times the worst case can happen, when the tree isn't balanced and the time complexity is O(n) for all three of these functions.
So, you kind of expect log(N) but it's not absolutely guaranteed.
the searching time complexity of a BST is N, and of binary search in and of itself is logN. What is the difference between the two?
The difference is that a binary search on a sorted array always starts at the middle element (i.e. the median when n is odd). This cannot be guaranteed in a BST. The root might be the middle element, but it doesn't have to be.
For instance, this is a valid BST:
10
/
8
/
5
/
2
/
1
...but it is not a balanced one, and so the process of finding the value 1 given the root of that tree, will include visiting all its nodes. If however the same values were presented in a sorted list (1,2,5,8,10), a binary search would start at 5 and never visit 8 or 10.
Adding self-balancing trees to the table
We can extend the given table with self-balancing search trees, like AVL, and then we get this:
implementation
search
insert
delete
sequential search (unordered list)
π
π
π
binary search (ordered array)
lgπ
π
π
BST
π
π
π
AVL
lgN
lgN
lgN
Related
I am right now working with a Binary Search tree. I wonder what is the Tim complexity for a Binary Search tree. More Specific what is the worst case Time complexity for the operation height, leaves and toString for a Binary Search tree and why?
All three operations have a O(n) worst-case time complexity.
For height: all nodes will be visited when the tree is degenerate, and all nodes except one have exactly one child.
For leaves: each node will have to be visited in order to check whether they are a leave.
For toString: obviously all nodes need to be visited.
What would be the worst case time complexity for finding a key that appears twice in the sorted array using binary search? I know that the worst case time complexity for binary search on a sorted array is O(log n). So, in the case that the key appears more than once the time complexity should be lesser than O(log n). However, I am not sure how to calculate this.
In the worst case the binary search needs to perform βlog_2(n) + 1β iterations to find the element or to conclude that the element is not in the array.
By having a duplicate you might just need one step less.
For instance, suppose your duplicate elements appear in the first and second indices of the array (same if they are in the last and one before the last).
In such a case you would have βlog_2(n)β comparisons, thus, still O(log(n)) as a worst case time complexity.
ternery search tree requires O(log(n)+k) comparisons where n is number of strings and k is the length of the string to be searched and binary search tree requires log(n) comparisons then why is TST faster than BST ?
Because in the ternary case it is log3(n) where in the binary case it is log2(n).
Ternary search trees are specifically designed to store strings, so our analysis is going to need to take into account that each item stored is a string with some length. Let's say that the length of the longest string in the data structure is L and that there are n total strings.
You're correct that a binary search tree makes only O(log n) comparisons when doing a lookup. However, since the items stored in the tree are all strings, each one of those comparisons takes time O(L) to complete. Consequently, the actual runtime of doing a search with a binary search tree in this case would be O(L log n), since there are O(log n) comparisons costing O(L) time each.
Now, let's consider a ternary search tree. With a standard TST implementation, for each character of the input string to look up, we do a BST lookup to find the tree to descend into. This takes time O(log n) and we do it L times for a total runtime of O(L log n), matching the BST lookup time. However, you can actually improve upon this by replacing the standard BSTs in the ternary search tree with weight-balanced trees whose weight is given by the number of strings in each subtree, a more careful analysis can be used to show that the total runtime for a lookup will be O(L + log n), which is significantly faster than a standard BST lookup.
Hope this helps!
I understand the basics of minimax and alpha-beta pruning. In all the literature, they talk about the time complexity for the best case is O(b^(d/2)) where b = branching factor and d = depth of the tree, and the base case is when all the preferred nodes are expanded first.
In my example of the "best case", I have a binary tree of 4 levels, so out of the 16 terminal nodes, I need to expand at most 7 nodes. How does this relate to O(b^(d/2))?
I don't understand how they come to O(b^(d/2)).
O(b^(d/2)) correspond to the best case time complexity of alpha-beta pruning. Explanation:
With an (average or constant) branching factor of b, and a search
depth of d plies, the maximum number of leaf node positions evaluated
(when the move ordering is pessimal) is O(bb...*b) = O(b^d) β the
same as a simple minimax search. If the move ordering for the search
is optimal (meaning the best moves are always searched first), the
number of leaf node positions evaluated is about O(b*1*b*1*...*b) for
odd depth and O(b*1*b*1*...*1) for even depth, or O(b^(d/2)). In the
latter case, where the ply of a search is even, the effective
branching factor is reduced to its square root, or, equivalently, the
search can go twice as deep with the same amount of computation.
The explanation of b*1*b*1*... is that all the first player's moves
must be studied to find the best one, but for each, only the best
second player's move is needed to refute all but the first (and best)
first player move β alphaβbeta ensures no other second player moves
need be considered.
Put simply, you "skip" every two level:
O describes the limiting behavior of a function when the argument tends towards a particular value or infinity, so in your case comparing precisely O(b^(d/2)) with small values of b and d doesn't really make sense.
I'm not entirely sure what amortized complexity means. Take a balanced binary search tree data structure (e.g. a red-black tree). The cost of a normal search is naturally log(N) where N is the number of nodes. But what is the amortized complexity of a sequence of m searches in, let's say, ascending order. Is it just log(N)/m?
Well you can consider asymptotic analysis as a strict method to set a upper bound for the running time of algorithms, where as amortized analysis is a some what liberal method.
For example consider an algorithm A with two statements S1 and S2. The cost of executing S1 is 10 and S2 is 100. Both the statements are placed inside a loop as follows.
n=0;
while(n<100)
{
if(n % 10 != 0)
{
S1;
}
else
{
s2;
}
n++;
}
Here the number of times S1 executed is 10 times the count of S2. But asymptotic analysis will only consider the facts that S2 takes a time of 10 units and it is inside a loop executing 100 times. So the upper limit for execution time is of the order of 10 * 100 = 1000. Where as amortized analysis averages out the number of times the statements S1 and S2 are executed. So the upper time limit for execution is of the order of 200. Thus amortized analysis gives a better estimate of the upper limit for executing an algorithm.
I think it is mlog(N) because you have to do m search operations (each time from root node downto target node), while the complexity of one single operation is log(N).
EDIT: #user1377000 you are right, I have mistaken amortized complexity from asymptotic complexity. But I don't think it is log(N)/m... because it is not guaranteed that you can finished all m search operations in O(logN) time.
What is amortized analysis of algorithms?
I think this might help.
In case of a balanced search tree the amortized complexity is equal to asymptotic one. Each search operation takes O(logn) time, both asymptotic and average. Therefore for m searches the average complexity will be O(mlogn).
Pass in the items to be found all at once.
You can think of it in terms of divide-and-conquer.
Take the item x in the root node.
Binary-search for x into your array of m items.
Partition the array into things less than x and greater than x. (Ignore things equal to x, since you already found it.)
Recursively search for the former partition in your left child, and for the latter in your right child.
One worst case: your array of items is just the list of things in the leaf nodes. (n is roughly 2m.) You'd have to visit every node. Your search would cost lg(n) + 2*lg(n/2) + 4*lg(n/4) + .... That's linear. Think of it as doing smaller and smaller binary searches until you hit every element in the array once or twice.
I think there's also a way to do it by keeping track of where you are in the tree after a search. C++'s std::map and std::set return iterators which can move left and right within the tree, and they might have methods which can take advantage of an existing iterator into the tree.