I want to add noise to a column of values in the range of 0-1.
But the noise shouldn't exceed these ranges, so my thought process was to check if adding the noise would be outside of the range, if it did, don't add the noise.
I tried:
df['val_x'].apply(lambda x: (x + np.random.normal(0, 0.2)) if (0 < x + np.random.normal(0, 0.2) < 1) else x)
at first, but I'm assuming it creates two separate random values, so some of the values pass the check with one and apply to the data frame with the other.
I feel like I need something like:
df['val_x'].apply(lambda x, withNoise = x + np.random.normal(0, 0.2): withNoise if (0 < withNoise < 1) else x)
defining the argument beforehand, but lambda doesn't support defining arguments with other arguments.
I want to do this without creating another function, but if it is the only way, I can.
Thanks in advance.
What about clipping?
df['val_x'] = df['val_x'].add(np.random.normal(0, 0.2, size=len(df))).clip(0, 1)
Or, adding your noise and only update the valid values:
s = df['val_x'].add(np.random.normal(0, 0.2, size=len(df)))
df['val_x'] = s.where(s.between(0, 1), df['val_x'])
# or
df.loc[s.between(0, 1), 'val_x'] = s
Related
I'm building a model using or-tools CP tools. The values I want to find are placed in a vector X, and I want to add a constraint that says up to each position of X, the next position cannot have as a value something bigger than the maximum found until X[:i] + 1
It would be something like this:
X[i] <= (max(X[:i]) + 1)
Of course, I cannot add this as a linear constraint with a max(), and creating one extra feature for each value of X upper bound seems excessive and also I would need to minimize each one to make it the "max", otherwise those are just upper bounds that could be huge and not prune my search space (and I already have an objective function).
I already have an objective function.
I know that one trick to add for instance a min-max (min(max(x[i])) problem is to create another variable that is an upper bound of each x and minimize that one. It would be sth like this:
model = cp_model.CpModel()
lb =0; ub=0
model.NewIntVar(z, lb, ub)
for i in domain(X):
model.NewIntVar(X[i], lb, up)
model.Add(X[i] <= z)
model.Minimize(z)
In case you don't want to program this you can use the method in or-tools:
model.AddMaxEquality(z, X)
Now I want to add a constraint that at each value of X sets an upper limit which is the maximum value found until the previous x. It would be something like this:
X[i] <= max(X[:i]) + 1
I was thinking of replicating the previous idea but that would require creating a "z" for each x... not sure if that is the best approach and how much it will reduce my space of solutions. At the same time couldn't find a method in or-tools to do this.
Any suggestions, please?
PS: I already have as an objective function min(z) like it is in the example presented.
Example:
For instance, you can have as a result of the model:
[0, 1, 2, 0, 2, 3]
But you shouldn't have:
[0, 1, 1, 2, 4]
Since the max until X[:3] is 2, so the ub of X[4] should be 2 + 1.
Thanks!
I have no specific hints except:
you need to experiment. One modeling trick may work on one kind of model and not on the other
make sure to use reuse the max variable at index i - 1. With X the array of variables and M the array of max, i.e. M[i] = max(X[0], .., X[i - 1])
M[i] = max(M[i - 1], X[i - 1])
X[i] <= M[i] + 1
I am trying to add a constraint which specifies that, in the optimization, the solver must pick a value of u for a set duration of time and can only switch after that set amount of time. For instance, say I have a mechanical device which can only switch its input value every 10 seconds. Then, I want the optimizer to account for that. I'll just attach the code here:
for it_i in range(0, N-1, equivalence_samples):
print("N: {}".format(N))
for it_j in range(0, equivalence_samples - 1):
if (it_i + it_j + 1) > N-1:
print("Breaking")
break
else:
constraint_u0 = prog.AddConstraint(u[0, it_i + it_j] == u[0, it_i + it_j + 1]) # add equivalence constraints
constraint_u1 = prog.AddConstraint(u[1, it_i + it_j] == u[1, it_i + it_j + 1]) # add equivalence constraints
print('Constraint_u_PE: {}'.format(constraint_u0))
print('Constraint_u_NI: {}'.format(constraint_u1))
I have implemented this in, what I expect to be a working solution. Sometimes it seems like it is working and other times, it does not.
I will show some photos of the output constraints from this and then a not working example.
Then, here are the plots that come out which clearly show there is some delineation between the switching times, but the values are not equivalent. I am attaching the code which generates this plot as well.
u_sol = result.GetSolution(u)
u_time = np.linspace(0, N-1, num = N)
# u_sol_trajectory = PiecewisePolynomial.ZeroOrderHold(u_time, u_sol)
plt.figure()
plt.plot(u_time, u_sol[0, :], 'o')
plt.plot(u_time, u_sol[1, :], 'o')
plt.xlabel('time steps')
plt.ylabel('u [mcg/min]')
plt.legend(['u_PE', 'u_NI'])
The particular solver that was being used in this case is the OSQP solver. Although I, in an ideal solver world, specified the correct constraints in the above code (that input 1 == input 2, input 2 == input 3, etc.), I did not account for the fact that solvers have an accuracy with which they try to uphold the constraints.
I can fix this problem by either updating the accuracy of the solver (as recommended by https://osqp.discourse.group/t/constraints-violated/143) or inputing more explicit constraints. I solved this with the second option. Now, I am specifying not just constraints like the following pattern:
input 1 == input 2, input 2 == input 3, etc.
but I am also including constraints like the following pattern:
input 1 == input 3, input 1 == input 4, input 1 == input 5
input 2 == input 4, input 2 == input 5 etc.
By being more explicit, my solver is now doing what I asked with small deviations from the constraint. The small deviations are acceptable for my application, however! It is a bit slower, but this isn't a problem for what I am using this for at the moment. Here is my updated code:
for it_i in range(0, N-1, equivalence_samples):
for it_j in range(0, equivalence_samples - 1):
for it_f in range(1, equivalence_samples - it_j):
if (it_i + it_j + it_f) > N-1:
print("Breaking")
break
else:
con_0 = eq(u[:, it_i + it_j], u[:, it_i + it_j + it_f])
constraint_u = prog.AddConstraint(con_0) # add equivalence constraints
print('Constraint_u: {}'.format(constraint_u))
Not the prettiest code in the world, but it works.
I am trying to write the code for solving the extremely difficult differential equation:
x' = 1
with the finite element method.
As far as I understood, I can obtain the solution u as
with the basis functions phi_i(x), while I can obtain the u_i as the solution of the system of linear equations:
with the differential operator D (here only the first derivative). As a basis I am using the tent function:
def tent(l, r, x):
m = (l + r) / 2
if x >= l and x <= m:
return (x - l) / (m - l)
elif x < r and x > m:
return (r - x) / (r - m)
else:
return 0
def tent_half_down(l,r,x):
if x >= l and x <= r:
return (r - x) / (r - l)
else:
return 0
def tent_half_up(l,r,x):
if x >= l and x <= r:
return (x - l) / (r - l)
else:
return 0
def tent_prime(l, r, x):
m = (l + r) / 2
if x >= l and x <= m:
return 1 / (m - l)
elif x < r and x > m:
return 1 / (m - r)
else:
return 0
def tent_half_prime_down(l,r,x):
if x >= l and x <= r:
return - 1 / (r - l)
else:
return 0
def tent_half_prime_up(l, r, x):
if x >= l and x <= r:
return 1 / (r - l)
else:
return 0
def sources(x):
return 1
Discretizing my space:
n_vertex = 30
n_points = (n_vertex-1) * 40
space = (0,5)
x_space = np.linspace(space[0],space[1],n_points)
vertx_list = np.linspace(space[0],space[1], n_vertex)
tent_list = np.zeros((n_vertex, n_points))
tent_prime_list = np.zeros((n_vertex, n_points))
tent_list[0,:] = [tent_half_down(vertx_list[0],vertx_list[1],x) for x in x_space]
tent_list[-1,:] = [tent_half_up(vertx_list[-2],vertx_list[-1],x) for x in x_space]
tent_prime_list[0,:] = [tent_half_prime_down(vertx_list[0],vertx_list[1],x) for x in x_space]
tent_prime_list[-1,:] = [tent_half_prime_up(vertx_list[-2],vertx_list[-1],x) for x in x_space]
for i in range(1,n_vertex-1):
tent_list[i, :] = [tent(vertx_list[i-1],vertx_list[i+1],x) for x in x_space]
tent_prime_list[i, :] = [tent_prime(vertx_list[i-1],vertx_list[i+1],x) for x in x_space]
Calculating the system of linear equations:
b = np.zeros((n_vertex))
A = np.zeros((n_vertex,n_vertex))
for i in range(n_vertex):
b[i] = np.trapz(tent_list[i,:]*sources(x_space))
for j in range(n_vertex):
A[j, i] = np.trapz(tent_prime_list[j] * tent_list[i])
And then solving and reconstructing it
u = np.linalg.solve(A,b)
sol = tent_list.T.dot(u)
But it does not work, I am only getting some up and down pattern. What am I doing wrong?
First, a couple of comments on terminology and notation:
1) You are using the weak formulation, though you've done this implicitly. A formulation being "weak" has nothing to do with the order of derivatives involved. It is weak because you are not satisfying the differential equation exactly at every location. FE minimizes the weighted residual of the solution, integrated over the domain. The functions phi_j actually discretize the weighting function. The difference when you only have first-order derivatives is that you don't have to apply the Gauss divergence theorem (which simplifies to integration by parts for one dimension) to eliminate second-order derivatives. You can tell this wasn't done because phi_j is not differentiated in the LHS.
2) I would suggest not using "A" as the differential operator. You also use this symbol for the global system matrix, so your notation is inconsistent. People often use "D", since this fits better to the idea that it is used for differentiation.
Secondly, about your implementation:
3) You are using way more integration points than necessary. Your elements use linear interpolation functions, which means you only need one integration point located at the center of the element to evaluate the integral exactly. Look into the details of Gauss quadrature to see why. Also, you've specified the number of integration points as a multiple of the number of nodes. This should be done as a multiple of the number of elements instead (in your case, n_vertex-1), because the elements are the domains on which you're integrating.
4) You have built your system by simply removing the two end nodes from the formulation. This isn't the correct way to specify boundary conditions. I would suggesting building the full system first and using one of the typical methods for applying Dirichlet boundary conditions. Also, think about what constraining two nodes would imply for the differential equation you're trying to solve. What function exists that satisfies x' = 1, x(0) = 0, x(5) = 0? You have overconstrained the system by trying to apply 2 boundary conditions to a first-order differential equation.
Unfortunately, there isn't a small tweak that can be made to get the code to work, but I hope the comments above help you rethink your approach.
EDIT to address your changes:
1) Assuming the matrix A is addressed with A[row,col], then your indices are backwards. You should be integrating with A[i,j] = ...
2) A simple way to apply a constraint is to replace one row with the constraint desired. If you want x(0) = 0, for example, set A[0,j] = 0 for all j, then set A[0,0] = 1 and set b[0] = 0. This substitutes one of the equations with u_0 = 0. Do this after integrating.
I am using the following python code to find two binary numbers that:
sum to a certain number
their highest bits cast to integers must sum up to 2
The second constraint is more important to me; and in my case, it will scale: let's say it might become that highest bits of [N] number must sum up to [M].
I am not sure why z3 does not give the correct result. Any hints? Thanks a lot.
def BV2Int(var):
return ArithRef(Z3_mk_bv2int(ctx.ref(), var.as_ast(), 0), var.ctx)
def main():
s = Solver()
s.set(':models', True)
s.set(':auto-cfgig', False)
s.set(':smt.bv.enable_int2bv',True)
x = BitVec('x',4)
y = BitVec('y',4)
s = Solver()
s.add(x+y == 16, Extract(3,3,x) + Extract(3,3,y) == 2)
s.check()
print s.model()
# result: [y = 0, x = 0], fail both constraint
s = Solver()
s.add(x+y == 16, BV2Int(Extract(3,3,x)) + BV2Int(Extract(3,3,y)) == 2)
s.check()
print s.model()
# result: [y = 15, x = 1], fail the second constraint
Update: Thanks the answer from Christoph. Here is a quick fix:
Extract(3,3,x) -> ZeroExt(SZ, Extract(3,3,x)) where SZ is the bit width of RHS minus 1.
(Aside: auto-cfgig should be auto-config.)
Note that bv2int and int2bv are essentially treated as uninterpreted, so if this part is crucial to your problem, then don't use them (see documentation and previous questions).
The problem with this example are the widths of the bit-vectors. Both x and y are 4-bit variables, and the numeral 16 as a 4-bit vector is 0 (modulo 2^4), so, indeed x + y is equal to 16 when x=0 and y=0.
Further, the Extract(...) terms extract 1-bit vectors, which means that the sum Ex.. + Ex.. is again a 1-bit value and the numeral 2 as a 1-bit vector is 0 (modulo 2^1), i.e., it is indeed the case that Ex... + Ex... = 2.
I am trying to graph two functions, but i want to graph one function for a condition but graph using another function if another condition is met.
A simple example would be:
if x > 0
then sin(x)
else cos(x)
It would then graph cos and sin depending on the x value, there being an obvious gap at x = 0, as cos(0) = 1 and sin(0) = 0.
EDIT: There is a built-in way. I'll leave my original answer below for posterity, but try using the piecewise() function:
plot(piecewise(((cos(x),x<0), (sin(x), 0<x))))
See it here.
I would guess that there's a built-in way to do this, but I don't know it. You can multiply your functions by the Heaviside Step Function to accomplish this task. The step function is 1 if x > 0 and 0 if x < 0, so multiplying this into your functions and then summing them together will select only one of them based on the sign of x, that is to say:
f(x) := heaviside(x) * sin(x) + heaviside(-x) * cos(x)
If x > 0, heaviside(x) = 1 and heaviside(-x) = 0, so f(x) = sin(x).
If x < 0, heaviside(x) = 0 and heaviside(-x) = 1, so f(x) = cos(x).
See it in action here. In general, note that if you want the transition to be at x = a, then you could do heaviside(x-a) and heaviside(-x+a), respectively. If you want N functions, you'll have to have (N-1) multiplied step functions on each term, each with their own (x-a_i) argument. I hope someone else can contribute a cleaner solution.