I want to solve a non-linear optimization problem. I was trying to solve it with KKT, but after all, I realized that it is hard to write code to solve this.
I want to know the optimal payment allocation that minimizes the repayment period until I pay off all the debts. Below is the function of my objective equation - My logic was to get the payoff period for each debt and minimize the sum:
My_Loans = {
'Name' : ['A', 'B','C','D'],
'Principal' : [350, 2000, 600, 750],
'APR' : [6, 4, 4, 5]}
My_Loans = pd.DataFrame(My_Loans)
PV = My_Loans['Principal']
APR = My_Loans['APR']
def repayment_period(PV, APR, PMT):
'''inputs are in lists format. PV is the principal amount, APR is Annual Percentage Rate, and PMT will be my variable which represents the monthly payment.'''
num_loans = len(PV)
times = []
for j in range(num_loans):
i = (APR[j]/12)/100
accrued_interest = PV[j] * i
N = round(-(log(1-((PV[j]*i)/PMT[j])))/log(1+i))
times.append(N)
total_period = sum(times)
return total_period
and it has one inequality constraint which is the sum of PMT (variable) < 2000 (the sum of monthly allocation must be less than 2000).
def ieq_constraint(x):
return np.atleast_1d(np.sum(x)-2000)
from scipy import optimize as pf
op.fmin_slsqp(how_long, np.array([0]), ieqcons=[ieq_constraint])
But I can't get this to work. What can I try next? I am open to better approaches if necessary.
Related
I am new in Python in optimization with scipy and very enthousiastic about it!
I am trying to optimize inventory allocation for spare parts (very low demand) using SciPy.
The goal is to minimize the inventory value while obtaining a target service rate (otif for on-time in full).
I defined a function otif to return a scalar for the objective.
There is also a function stock_val to evaluate the value of the stock (taking the allocation and the price)
Finally I have minimize_stock in which I define an initial guess, provide bounds and constraint.
The constraint is simply to reach the targeted otif.
My issue is that minimize fails and just return the initial guess.
As entry:
- part_demand is a dataframe with a column with material part number and columns of ordered quantity per months.
- part_price is a dataframe with a column with material part number and price.
from scipy.optimize import minimize
def otif(part_demand, stocks):
"""
Returns the OnTimeInFull covered with stock using parts demand and stocks
"""
df_occ = part_demand[(part_demand !=0)].count(axis=1)
df_cov = part_demand[(part_demand !=0) & (part_demand.le(stocks, axis=0))].count(axis = 1)
return df_cov.sum() / df_occ.sum()
def stock_val(stocks, part_price):
"""
Returns the stock value of stock using parts prices
"""
return np.dot(stocks,part_price)
def minimize_stock(target_otif, part_demand, part_price):
"""
Returns the optimal stock value and distribution for a part demand and targeted OnTimeInFull
"""
n = part_demand.shape[0]
init_guess = np.repeat(10,n)
bounds = ((0,10),) * n # an N-tuple of 2-tuples
# construct the constraints
return_is_target = {'type': 'eq',
'fun': lambda stocks: target_otif - otif(part_demand, stocks)}
stocks = minimize(stock_val, init_guess,
args=(part_price,), method='SLSQP',
options={'disp': True},
constraints=(return_is_target),
bounds=bounds)
return stocks
# Define entry Variables
material_items = pnMonthly[dataMthColMask]
#display(material_items)
df_groupby = PNtable.groupby("Material").last()
material_price = df_groupby['UnitPrice']
n = material_items.shape[0]
init_guess = np.repeat(10000,n)
print(stock_val(init_guess, material_price))
sol = minimize_stock(target_otif=0.89, part_demand=material_items, part_price=material_price)
display(sol)
print('OTIF is ', format(otif(material_items, sol.x)))
print('Stock value is ', format(stock_val(sol.x, material_price)))
Thank you in advance for your help!
I tried different algorithms apart from SLSQP and still get failure.
I am not sure about which algorithm is best for this kind of optimization problem.
I am also not sure about the syntax for minimize or what are the conditions for the algorithm to work.
I have tried to do optimal EV charging scheduling using the GEKKO packages. However, my code is stuck on some variable boundary condition when it is set to be lower than or equal to zero, i.e., x=m.Array(m.Var,n_var,value=0,lb=0,ub=1.0). The error message is 'Unsuccessful with error code 0'.
Below is my python script. If you have any advice on this problem, please don't hesitate to let me know.
Thanks,
Chitchai
#------------------------------
import numpy as np
import pandas as pd
import math
import os
from gekko import GEKKO
print('...... Preparing data for optimal EV charging ........')
#---------- Read data from csv.file -----------)
Main_path = os.path.dirname(os.path.realpath(__file__))
Baseload_path = Main_path + '/baseload_data.csv'
TOU_path = Main_path + '/TOUprices.csv'
EV_path = Main_path + '/EVtravel.csv'
df_Baseload = pd.read_csv(Baseload_path, index_col= 'Time')
df_TOU = pd.read_csv(TOU_path, index_col= 'Time')
df_EVtravel = pd.read_csv(EV_path, index_col= 'EV_no')
#Create and change run directory
rd= r'.\RunDir'
if not os.path.isdir(os.path.abspath(rd)):
os.mkdir(os.path.abspath(rd))
#--------------------EV charging optimization function----------------#
def OptEV(tou_rate, EV_data, P_baseload):
"""This function is to run EV charging optimization for each houshold"""
#Initialize EV model and traveling data in 96 intervals(interval = 15min)
s_time= 4*(EV_data[0]//1) + math.ceil(100*(EV_data[0]%1)/15) #starting time
d_time= 4*(EV_data[1]//1) + math.floor(100*(EV_data[1]%1)/15) #departure time
Pch_rating= EV_data[2] #charing rated power(kW)
kWh_bat= EV_data[3] #Battery rated capacity(kWh)
int_SoC= EV_data[4] #Initial battery's SoC(p.u.)
#Calculation charging period
if d_time<= s_time:
ch_period = 96+d_time-s_time
else:
ch_period = d_time-s_time
Np= int(ch_period)
print('charging period = %d intervals'%(Np))
#Construct revelant data list based on charging period
ch_time = [0]*Np #charging time step list
price_rate = [0]*Np #electricity price list
kW_baseload = [0]*Np #kW house baseload power list
#Re-arrange charging time, electricity price rate and baseload
for i in range(Np):
t_step = int(s_time)+i
if t_step <= 95: #Before midnight
ch_time[i] = t_step
price_rate[i] = tou_rate[t_step]
kW_baseload[i] = P_baseload[t_step]/1000 #active house baseload
else: #After midnight
ch_time[i] = t_step-96
price_rate[i] = tou_rate[t_step-96]
kW_baseload[i] = P_baseload[t_step-96]/1000
#Initialize Model
m = GEKKO(remote=False) # or m = GEKKO() for solve locally
m.path = os.path.abspath(rd) # change run directory
#define parameter
ch_eff= m.Const(value=0.90) #charging/discharging efficiency
alpha= m.Const(value= 0.00005) #regularization constant battery profile
net_load= [None]*Np #net metering houshold load power array
elec_price= [None]*Np #purchased electricity price array
SoC= [None]*(Np+1) #SoC of batteries array
#initialize variables
n_var= Np #number of dicision variables
x = m.Array(m.Var,n_var,value=0,lb=0,ub=1.0) #dicision charging variables
#Calculation relevant evaluated parameters
#x[0] = m.Intermediate(-1.025)
SoC[0]= m.Intermediate(int_SoC) #initial battery SoC
for i in range(Np):
#Netload metering evaluation
net_load[i]= m.Intermediate(kW_baseload[i]+x[i]*Pch_rating)
#electricity cost price evaluation(cent/kWh)
Neg_pr= (1/4)*net_load[i]*price_rate[i] # Reverse power cost
Pos_pr= (1/4)*net_load[i]*price_rate[i] # Purchased power cost
elec_price[i]= m.Intermediate(m.if3(net_load[i], Neg_pr, Pos_pr))
#current battery's SoC evaluation
j=i+1
SoC_dch= (1/4)*(x[i]*Pch_rating/ch_eff)/kWh_bat #Discharging(V2G)
SoC_ch= (1/4)*ch_eff*x[i]*Pch_rating/kWh_bat #Discharging
SoC[j]= m.Intermediate(m.if3(x[i], SoC[j-1]+SoC_dch, SoC[j-1]+SoC_ch))
#m.solve(disp=False)
#-------Constraint functions--------#
#EV battery constraint
m.Equation(SoC[-1] >= 0.80) #required departure SoC (minimum=80%)
for i in range(Np):
j=i+1
m.Equation(SoC[j] >= 0.20) #lower SoC limit = 20%
for i in range(Np):
j=i+1
m.Equation(SoC[j] <= 0.95) #upper SoC limit = 95%
#household Net-power constraint
for i in range(Np):
m.Equation(net_load[i] >= -10.0) #Lower netload power limit
for i in range(Np):
m.Equation(net_load[i] <= 10.0) #Upper netload power limit
#Objective functions
elec_cost = m.Intermediate(m.sum(elec_price)) #electricity cost
#battery degradation cost
bat_cost = m.Intermediate(m.sum([alpha*xi**2 for xi in x]))
#bat_cost = 0 #Not consider battery degradation cost
m.Minimize(elec_cost + bat_cost) # objective
#Set global options
m.options.IMODE = 3 #steady state optimization
#Solve simulation
try:
m.solve(disp=True) # solve
print('--------------Results---------------')
print('Objective Function= ' + str(m.options.objfcnval))
print('x= ', x)
print('price_rate= ', price_rate)
print('net_load= ', net_load)
print('elec_price= ', elec_price)
print('SoC= ', SoC)
print('Charging time= ', ch_time)
except:
print('*******Not successful*******')
print('--------------No convergence---------------')
# from gekko.apm import get_file
# print(m._server)
# print(m._model_name)
# f = get_file(m._server,m._model_name,'infeasibilities.txt')
# f = f.decode().replace('\r','')
# with open('infeasibilities.txt', 'w') as fl:
# fl.write(str(f))
Pcharge = x
return ch_time, Pcharge
pass
#---------------------- Run scripts ---------------------#
TOU= df_TOU['Prices'] #electricity TOU prices rate (c/kWh)
Load1= df_Baseload['Load1']
EV_data = [17.15, 8.15, 3.3, 24, 0.50] #[start,final,kW_rate,kWh_bat,int_SoC]
OptEV(TOU, EV_data, Load1)
#--------------------- End of a script --------------------#
When the solver fails to find a solution and reports "Solution Not Found", there is a troubleshooting method to diagnose the problem. The first thing to do is to look at the solver output with m.solve(disp=True). The solver may have identified either an infeasible problem or it reached the maximum number of iterations without converging to a solution. In your case, it identified the problem as infeasible.
Infeasible Problem
If the solver failed because of infeasible equations then it found that the combination of variables and equations is not solvable. You can try to relax the variable bounds or identify which equation is infeasible with the infeasibilities.txt file in the run directory. Retrieve the infeasibilities.txt file from the local run directory that you can view with m.open_folder() when m=GEKKO(remote=False).
Maximum Iteration Limit
If the solver reached the default iteration limit (m.options.MAX_ITER=250) then you can either try to increase this limit or else try the strategies below.
Try a different solver by setting m.options.SOLVER=1 for APOPT, m.options.SOLVER=2 for BPOPT, m.options.SOLVER=3 for IPOPT, or m.options.SOLVER=0 to try all the available solvers.
Find a feasible solution first by solving a square problem where the number of variables is equal to the number of equations. Gekko a couple options to help with this including m.options.COLDSTART=1 (sets STATUS=0 for all FVs and MVs) or m.options.COLDSTART=2 (sets STATUS=0 and performs block diagonal triangular decomposition to find possible infeasible equations).
Once a feasible solution is found, try optimizing with this solution as the initial guess.
To explain the question it's best to start with this
picture
I am modeling an optimization decision problem and a feature that I'm trying to implement is heat transfer between the process stages (a = 1, 2) taking into account which equipment type is chosen (j = 1, 2, 3) by the binary decision variable y.
The temperatures for the equipment are fixed values and my goal is to find (in the case of the picture) dT = 120 - 70 = 50 while keeping the temperature difference as a parameter (I want to keep the problem linear and need to multiply the temperature difference with a variable later on).
Things I have tried:
dT = T[a,j] - T[a-1,j]
(this obviously gives T = 80 for T[a-1,j] which is incorrect)
T[a-1] = sum(T[a-1,j] * y[a-1,j] for j in (1,2,3)
This will make the problem non-linear when I multiply with another variable.
I am using pyomo and the linear "glpk" solver. Thank you for reading my post and if someone could help me with this it is greatly appreciated!
If you only have 2 stages and 3 pieces of equipment at each stage, you could reformulate and let a binary decision variable Y[i] represent each of the 9 possible connections and delta_T[i] be a parameter that represents the temp difference associated with the same 9 connections which could easily be calculated and put into a model parameter.
If you want to keep in double-indexed, and assuming that there will only be 1 piece of equipment selected at each stage, you could take the sum-product of the selection variable and temps at each stage and subtract them.
dT[a] = sum(T[a, j]*y[a, j] for j in J) - sum(T[a-1, j]*y[a-1, j] for j in J)
for a ∈ {2, 3, ..., N}
I have the following problem and I can't figure out if cvxpy can do what I need.
Context: I optimize portfolios. When buying bonds and optimizing the quantity of each bond to buy, it's only possible to buy each bond only in multiples of 1,000 units.
However, the minimum piece required to be bought is most of the time 10,000.
This means we either don't buy a bond at all or if we buy it, the quantity bought has to be either 10,000, 11,000, 12,000 and so on.
Is there a way (it seems it doesn't) to restrict certain values from the possible solutions an integer variable can have?
So let's assume we have an integer variable x that is non negative.
We basically want to buy 1000x but we know that x can be x = {0, 10, 11, 12, ...}
Is it possible to skip values 1.. 9 without adding other variables?
For example:
import numpy as np
import pandas as pd
import cvxpy as cvx
np.random.seed(1)
# np.random.rand(3)
p = pd.DataFrame({'bond_id': ['s1','s2', 's3', 's4', 's5', 's6', 's7','s8','s9', 's10'],
'er': np.random.rand(10),
'units': [10000,2000,3000,4000,27000,4000,0,0,0,0] })
final_units = cvx.Variable( 10, integer=True)
constraints = list()
constraints.append( final_units >= 0)
constraints.append(sum(final_units*1000) <= 50000)
constraints.append(sum(final_units*1000) >= 50000)
constraints.append(final_units <= 15)
obj = cvx.Maximize( final_units # np.array(list(p['er'])) )
prob = cvx.Problem(obj, constraints)
solve_val = prob.solve()
print("\n* solve_val = {}".format(solve_val))
solution_value = prob.value
solution = str(prob.status).lower()
print("\n** SOLUTION 3: {} Value: {} ".format(solution, solution_value))
print("\n* final_units -> \n{}\n".format(final_units.value))
p['FINAL_SOL'] = final_units.value * 1000
print("\n* Final Portfolio: \n{}\n".format(p))
This solution is a very simplified version of the problem I face. The final vector final_units can suggest values like in this example where we have to buy 5,000 units of bond s9, however I can't since the min I can buy is 10,000.
I know I could add an additional integer vector to express an OR condition, but in reality my real problem is way bigger than this, I have thousand of integer variables already. Hence, I wonder if there's a way to exclude values from 1 to 9 without adding additional variables to the problem.
Thank you
No, not with CVXPY. You can model it with an integer variable x[i] plus a binary variable y[i], and using the constraints (in math notation):
y[i] * 10 <= x[i] <= y[i] * 15
This results in x[i] ∈ {0, 10..15}.
Some solvers have a variable type for this: semi-integer variables. Using this you don't need to have an extra binary variable and these 2 constraints. CVXPY does not support this variable type AFAIK.
A simple optimization problem: Find the optimal control sequence for a refrigerator based on the cost of energy. The only constraint is to stay below a temperature threshold, and the objective function tries to minimize the cost of energy used. This problem is simplified so the control is simply a binary array, ie. [0, 1, 0, 1, 0], where 1 means using electricity to cool the fridge, and 0 means to turn of the cooling mechanism (which means there is no cost for this period, but the temperature will increase). We can assume each period is fixed period of time, and has a constant temperature change based on it's on/off status.
Here are the example values:
Cost of energy (for our example 5 periods): [466, 426, 423, 442, 494]
Minimum cooling periods (just as a test): 3
Starting temperature: 0
Temperature threshold(must be less than or equal): 1
Temperature change per period of cooling: -1
Temperature change per period of warming (when control input is 0): 2
And here is the code in PuLP
from pulp import LpProblem, LpMinimize, LpVariable, lpSum, LpStatus, value
from itertools import accumulate
l = list(range(5))
costy = [466, 426, 423, 442, 494]
cost = dict(zip(l, costy))
min_cooling_periods = 3
prob = LpProblem("Fridge", LpMinimize)
si = LpVariable.dicts("time_step", l, lowBound=0, upBound=1, cat='Integer')
prob += lpSum([cost[i]*si[i] for i in l]) # cost function to minimize
prob += lpSum([si[i] for i in l]) >= min_cooling_periods # how many values must be positive
prob.solve()
The optimization seems to work before I try to account for the temperature threshold. With just the cost function, it returns an array of 0s, which does indeed minimize the cost (duh). With the first constraint (how many values must be positive) it picks the cheapest 3 cooling periods, and calculates the total cost correctly.
obj = value(prob.objective)
print(f'Solution is {LpStatus[prob.status]}\nThe total cost of this regime is: {obj}\n')
for v in prob.variables():
print(f'{v.name} = {v.varValue}')
output:
Solution is Optimal
The total cost of this regime is: 1291.0
time_step_0 = 0.0
time_step_1 = 1.0
time_step_2 = 1.0
time_step_3 = 1.0
time_step_4 = 0.0
So, if our control sequence is [0, 1, 1, 1, 0], the temperature will look like this at the end of each cooling/warming period: [2, 1, 0, -1, 1]. The temperature goes up 2 whenever the control input is 1, and down 1 whenever the control input is 1. This example sequence is a valid answer, but will have to change if we add a max temperature threshold of 1, which would mean the first value must be a 1, or else the fridge will warm to a temperature of 2.
However I get incorrect results when trying to specify the sequential constraint of staying within the temperature thresholds with the condition:
up_temp_thresh = 1
down = -1
up = 2
# here is where I try to ensure that the control sequence would never cause the temperature to
# surpass the threshold. In practice I would like a lower and upper threshold but for now
# let us focus only on the upper threshold.
prob += lpSum([e <= up_temp_thresh for e in accumulate([down if si[i] == 1. else up for i in l])]) >= len(l)
In this case the answer comes out the same as before, I am clearly not formulating it correctly as the sequence [0, 1, 1, 1, 0] would surpass the threshold.
I am trying to encode "the temperature at the end of each control sequence must be less than the threshold". I do this by turning the control sequence into an array of the temperature changes, so control sequence [0, 1, 1, 1, 0] gives us temperature changes [2, -1, -1, -1, 2]. Then using the accumulate function, it computes a cumulative sum, equal to the fridge temp after each step, which is [2, 1, 0, -1, 1]. I would like to just check if the max of this array is less than the threshold, but using lpSum I check that the sum of values in the array less than the threshold is equal to the length of the array, which should be the same thing.
However I'm clearly formulating this step incorrectly. As written this last constraint has no effect on the output, and small changes give other wrong answers. It seems the answer should be [1, 1, 1, 0, 0], which gives an acceptable temperature series of [-1, -2, -3, -1, 1]. How can I specify the sequential nature of the control input using PuLP, or another free python optimization library?
The easiest and least error-prone approach would be to create a new set of auxillary variables of your problem which track the temperature of the fridge in each interval. These are not 'primary decision variables' because you cannot directly choose them - rather the value of them is constrained by the on/off decision variables for the fridge.
You would then add constraints on these temperature state variables to represent the dynamics. So in untested code:
l_plus_1 = list(range(6))
fridge_temp = LpVariable.dicts("fridge_temp", l_plus_1, cat='Continuos')
fridge_temp[0] = init_temp # initial temperature of fridge - a known value
for i in l:
prob += fridge_temp[i+1] == fridge_temp[i] + 2 - 3*s[i]
You can then sent the min/max temperature constraints on these new fridge_temp variables.
Note that in the above I've assumed that the fridge temperature variables are defined at one more intervals than the on/off decisions for the fridge. The fridge temperature variables represent the temperature at the start of an interval - and having one extra one means we can ensure the final temperature of the fridge is acceptable.