The Problem:
Given a BST with N nodes, with a domain of cardinality D (domain being the possible values for the node keys).
Given a key that is in the domain but may or may not be a member of the BST.
At the start, our confidence that the node is in the tree should be 1/D, but as we go deeper into the tree both D and N are split approximately in half. That would suggest that our confidence that our key is a member of the tree should remain constant until we hit the bottom or discover the key. However, I'm not sure if that reasoning is complete, since it seems more like we are choosing N nodes from D.
I was thinking something along the lines of this, but the reasoning here still doesn't seem complete. Can somebody point me in the right direction?
Apriori, the probability that your key in is the tree is N/D.
Without loss of generality, let assume that the node's value range is [1..D].
When you walk down the tree, either:
The current node matches your key, hence P = 1
The current node has value C which is larger than your key, you go left, but you don't know how many items are in the left sub-tree. Now you can make one of these assumptions:
The tree is balanced. The range in the subtree is [1..C-1], and there are (D-1)/2 nodes in the subtree. Hence, P = ((D-1)/2)/(C-1)
The tree is not balanced. The range in the subtree is [1..C-1], and the maximum likelihood estimation for the number of nodes in the subtree is N * (C-1)/D. Hence, P = (N*(C-1)/D)/(C-1) = N/D. (no change)
If you know more about how the tree was constructed - you can make a better MLE for the number of nodes in the subtree.
The current node has value C which is smaller than your key, you go right, but you don't know how many items are in the right sub-tree.
...
Related
I'm implementing an AVL tree, and I'm trying to wrap my head around the time complexity of the adding process. It's my understanding that in order to achieve O(log n) you need to keep either balance or height state in tree nodes so that you don't have to recalculate them every time you need them (which may require a lot of additional tree traversal).
To solve this, I have a protocol that recursively "walks back up" a trail of parent pointers to the root, balancing if needed and setting heights along the way. This way, the addition algorithm kind of has a "capture" and "bubble" phase down and then back up the tree - like DOM events.
My question is: is this still technically O(log n) time? Technically, you only deal with divisions of half at every level in the tree, but you also need to travel down and then back up every time. What is the exact time complexity of this operation?
Assuming the height of the tree is H and the structure stays balanced during all operation.
Then, as you mentioned, inserting a node will take O(H).
However, every time a node is added to the AVL tree, you need to update the height of the parents all the way up to the root node.
Since the tree is balanced, updating height will traverse only the linked-list like structure with the newly inserted node in the tail.
The height updating can be viewed equivalent to traversing a linked-list with length equals to H.
Therefore, updating height will take another O(H) and the total update time is 2 * O(H), which is still O(log N) if we get rid of the constant factor.
Hope this makes sense to you.
"Technically, you only deal with divisions of half at every level in the tree, but you also need to travel down and then back up every time. What is the exact time complexity of this operation?"
You've stated that you have to travel down and up every time.
So, we can say that your function is upper bounded by a runtime of 2 * logn.
It's clear that this is O(logn).
More specifically, we could assign the constant 3 and a starting value of 1, such that
2 * logn <= 3 * logn for all values of n >= 1.
This reduces to 2 <= 3, which is of course true.
The idea behind big-O is to understand the basic shape of the function that upper-bounds your function's runtime as the input size moves towards infinity - thus, we can drop the constant factor of 2.
Is it possible to convert a non balanced BST (the size of the tree is n and the height is h) to RBT in time complexirty of O(n) and space complexity of O(h)?
If you know the number of nodes before hand this is doable, knowing the number of nodes tells you the height of the target RB tree (regardless of what the original tree height).
Therefore you can simply 'peel' nodes off the original tree one-by-one starting from the minimum and place them in the correct tree slot. The easiest way to do this will end up with every row except for a potentially empty bottom row black. (That is, if you have a tree with 7 nodes they will all be black but if you have a tree with 6 the first 2 rows will be black and the bottom row will have 3 red nodes).
This will take O(n) time - to visit each node in the original tree - and O(h) space because you will need to keep track of some bookkeeping depending on where you are in the process.
And note this will only work if you know the number of nodes in the original tree, as it depends on knowing which nodes will be in the bottom row of the produced tree.
Consider a tree of depth B (i.e.: all the paths have length B) whose nodes represent system states and edges represent actions.
Each action a in ActionSet has a gain and makes the system move from a state to another.
Performing the sequence of actions A-B-C or C-B-A (or any other permutation of these actions) brings to the same gain. Moreover:
the higher the number of actions performed before a, the lower the increase of total gain when a is asked
the gain achieved by each path cannot be greater than a quantity H, i.e.: some paths may achieve a gain that is lower than H, but whenever performing an action makes the total gain equal to H, all the other actions performed from that point on will gain 0
what is gained by the sequence of actions #b,h,j, ..., a# is g(a) (0 <= g(a) <= H)
once an action has been performed on a path from the root to a leaf, it cannot be performed a second time on the same path
Application of Algorithm1. I apply the following algorithm (A*-like):
Start from the root.
Expand the first level of the tree, which will contain all the actions in ActionSet. Each expanded action a has gain f(a) = g(a) + h(a), where g(a) is defined as stated before and h(a) is an estimate of what will be earned by performing other B-1 actions
Select the action a* that maximizes f(a)
Expand the children of a*
Iterate 2-3 until an entire path of B actions from the root to a leaf that guarantees the highest f(n) is visited. Notice that the new selected action can be selected also from the nodes which were abandoned at previous levels. E.g., if after expanding a* the node maximizing f(a) is a children of the root, it is selected as the new best node
Application of Algorithm2. Now, suppose I have a greedy algorithm that looks only to the g(n) component of the knowledge-plus-heuristic function f(n), i.e., this algorithm chooses actions according to the gain that has been already earned:
at the first step I choose the action a maximizing the gain g(a)
at the second step I choose the action b maximizing the gain g(b)
Claim. Experimental proofs showed me that the two algorithms bring to the same result, which might be mixed (e.g., the first one suggests the sequence A-B-C and the second one suggests B-C-A).
However, I didn't succeed in understanding why.
My question is: is there a formal way of proving that the two algorithms return the same result, although mixed in some cases?
Thank you.
A* search will return the optimal path. From what I understand of the problem, your greedy search is simply performing bayes calculations and wlll continue to do so until it finds an optimal set of nodes to take. Since the order of the nodes do not matter, the two should return the same set of nodes, albiet in different orders.
I think this is correct assuming you have the same set of actions you can perform from every node.
I understand the basics of minimax and alpha-beta pruning. In all the literature, they talk about the time complexity for the best case is O(b^(d/2)) where b = branching factor and d = depth of the tree, and the base case is when all the preferred nodes are expanded first.
In my example of the "best case", I have a binary tree of 4 levels, so out of the 16 terminal nodes, I need to expand at most 7 nodes. How does this relate to O(b^(d/2))?
I don't understand how they come to O(b^(d/2)).
O(b^(d/2)) correspond to the best case time complexity of alpha-beta pruning. Explanation:
With an (average or constant) branching factor of b, and a search
depth of d plies, the maximum number of leaf node positions evaluated
(when the move ordering is pessimal) is O(bb...*b) = O(b^d) – the
same as a simple minimax search. If the move ordering for the search
is optimal (meaning the best moves are always searched first), the
number of leaf node positions evaluated is about O(b*1*b*1*...*b) for
odd depth and O(b*1*b*1*...*1) for even depth, or O(b^(d/2)). In the
latter case, where the ply of a search is even, the effective
branching factor is reduced to its square root, or, equivalently, the
search can go twice as deep with the same amount of computation.
The explanation of b*1*b*1*... is that all the first player's moves
must be studied to find the best one, but for each, only the best
second player's move is needed to refute all but the first (and best)
first player move – alpha–beta ensures no other second player moves
need be considered.
Put simply, you "skip" every two level:
O describes the limiting behavior of a function when the argument tends towards a particular value or infinity, so in your case comparing precisely O(b^(d/2)) with small values of b and d doesn't really make sense.
I am trying to write a formula to find:
"The number of structurally different binary trees that can exist with nodes that have either 0 or 1 children".
How would I go about doing this?
Seems to me that a "binary tree" that has nodes with only 0 or 1 children is a chain. If by "structurally different" you mean that you treat differently whether a given non-terminal node has a left child or a right child, then observe that you can describe that tree with a binary number that is N-1 bits long. So the number of different trees for a given N would be 2**N-1.
(And, obviously, if you mean how many different "shapes" of the "tree" can exist for a given N, the answer is 1.)