I have a question about Entropy and Information Flow. Suppose that X = {-1, 1}; meaning that it can be either -1 or 1, and the following assignment for Y:
Y := X * X
My question is that the value of Y, after the assignment, will always be 1. If X = -1, then Y=1 and if X = 1, then Y= 1. Knowing this, can I still assume that the conditional entropy H(X/Y) = 0, because knowing X will always tell you the Value of Y. On the other hand, the conditional entropy H(Y/X) = 1.0 because knowing Y will not give me the value of X.
Am I thinking in the right direction? Please help
You are partially correct, though it seems like you are rather "swapped" in your notation and your definition.
H(X|Y) is entropy of X given Y rather than entropy of Y given X.
Also, you should try to look at the condition here more carefully. Since you have a very clear relationship between X and Y, that means Y = f(X). And in that situation, just as you say, the conditional entropy is always 0 (yet you are swapped in your notation). Thus it should be
H(Y|X) = 0
On the other hand, if you have Y, you completely have no clue of what is X and both -1 and 1 have equal probability. So in this case
H(X|Y) = 1
Related
I'm running functions to create cyclical datetime features, so I have converted timestamps to sine and cosine representations for ML model training.
In one sample, x = 305.2116709309027, giving np.sin(x) = -0.459279 and np.cos(x) = -0.888292, my question is how to retrieve x from these sin and cos features later?
I assumed np.arcsin(-0.459279) == 305.2116709309027 and I could then decode the timestamp used from there but I'm not having any luck.
You should be aware that mathematically, sin(x) and cos(x) are periodic functions, meaning multiple different values as input can yield the same output.
For example, x=0, x=2pi, and x=4pi can all yield the same value. So you can't decode the x from y, except you know that the input is restricted within a period, such as between [0, 2pi].
HOWEVER, for arcsin(x), since the domain of x is limited, and each y corresponds to a unique x, you can get the x from y.
I have curve that initially Y increases linearly with X, then reach a plateau at point C.
In other words, the curve can be defined as:
if X < C:
Y = k * X + b
else:
Y = k * C + b
The training data is a list of X ~ Y values. I need to determine k, b and C through a machine learning approach (or similar), since the data is noisy and refection point C changes over time. I want something more robust than get C through observing the current sample data.
How can I do it using sklearn or maybe scipy?
WLOG you can say the second equation is
Y = C
looks like you have a linear regression to fit the line and then a detection point to find the constant.
You know that in the high values of X, as in X > C you are already at the constant. So just check how far back down the values of X you get the same constant.
Then do a linear regression to find the line with value of X, X <= C
Your model is nonlinear
I think the smartest way to solve this is to do these steps:
find the maximum value of Y which is equal to k*C+b
M=max(Y)
drop this maximum value from your dataset
df1 = df[df.Y != M]
and then you have simple dataset to fit your X to Y and you can use sklearn for that
Given variables y and z, both of which depend on a tensor x. By product rule, if I do tf.gradients(yz,x), it would give me y'(x)z(x) + z'(x)y(x). Is there a way I can specify y as a constant with respect to x such that tf.gradients(yz,x) only gives me z'(x)y(x)?
I know y_=tf.constant(sess.run(y)) will give me y as a constant, but I cannot use that solution in my code.
You can use tf.stop_gradient() to block backpropagation. To block gradients in your example:
y = function1(x)
z = function2(x)
blocked_y = tf.stop_gradient(y)
product = blocked_y * z
After you backpropagate through product, the backpropagation will continue to z and not y.
I am using the following python code to find two binary numbers that:
sum to a certain number
their highest bits cast to integers must sum up to 2
The second constraint is more important to me; and in my case, it will scale: let's say it might become that highest bits of [N] number must sum up to [M].
I am not sure why z3 does not give the correct result. Any hints? Thanks a lot.
def BV2Int(var):
return ArithRef(Z3_mk_bv2int(ctx.ref(), var.as_ast(), 0), var.ctx)
def main():
s = Solver()
s.set(':models', True)
s.set(':auto-cfgig', False)
s.set(':smt.bv.enable_int2bv',True)
x = BitVec('x',4)
y = BitVec('y',4)
s = Solver()
s.add(x+y == 16, Extract(3,3,x) + Extract(3,3,y) == 2)
s.check()
print s.model()
# result: [y = 0, x = 0], fail both constraint
s = Solver()
s.add(x+y == 16, BV2Int(Extract(3,3,x)) + BV2Int(Extract(3,3,y)) == 2)
s.check()
print s.model()
# result: [y = 15, x = 1], fail the second constraint
Update: Thanks the answer from Christoph. Here is a quick fix:
Extract(3,3,x) -> ZeroExt(SZ, Extract(3,3,x)) where SZ is the bit width of RHS minus 1.
(Aside: auto-cfgig should be auto-config.)
Note that bv2int and int2bv are essentially treated as uninterpreted, so if this part is crucial to your problem, then don't use them (see documentation and previous questions).
The problem with this example are the widths of the bit-vectors. Both x and y are 4-bit variables, and the numeral 16 as a 4-bit vector is 0 (modulo 2^4), so, indeed x + y is equal to 16 when x=0 and y=0.
Further, the Extract(...) terms extract 1-bit vectors, which means that the sum Ex.. + Ex.. is again a 1-bit value and the numeral 2 as a 1-bit vector is 0 (modulo 2^1), i.e., it is indeed the case that Ex... + Ex... = 2.
I am trying to graph two functions, but i want to graph one function for a condition but graph using another function if another condition is met.
A simple example would be:
if x > 0
then sin(x)
else cos(x)
It would then graph cos and sin depending on the x value, there being an obvious gap at x = 0, as cos(0) = 1 and sin(0) = 0.
EDIT: There is a built-in way. I'll leave my original answer below for posterity, but try using the piecewise() function:
plot(piecewise(((cos(x),x<0), (sin(x), 0<x))))
See it here.
I would guess that there's a built-in way to do this, but I don't know it. You can multiply your functions by the Heaviside Step Function to accomplish this task. The step function is 1 if x > 0 and 0 if x < 0, so multiplying this into your functions and then summing them together will select only one of them based on the sign of x, that is to say:
f(x) := heaviside(x) * sin(x) + heaviside(-x) * cos(x)
If x > 0, heaviside(x) = 1 and heaviside(-x) = 0, so f(x) = sin(x).
If x < 0, heaviside(x) = 0 and heaviside(-x) = 1, so f(x) = cos(x).
See it in action here. In general, note that if you want the transition to be at x = a, then you could do heaviside(x-a) and heaviside(-x+a), respectively. If you want N functions, you'll have to have (N-1) multiplied step functions on each term, each with their own (x-a_i) argument. I hope someone else can contribute a cleaner solution.