My program is optimizing the charging and decharging of a home battery to minimize the cost of electricity at the end of the year. In this case there also is a PV, which means that sometimes you're injecting electricity into the grid and receive money. The net offtake is the result of the usage of the home and the PV installation. So these are the possible situations:
Net offtake > 0 => usage home > PV => discharge from battery or take from grid
Net offtake < 0 => usage home < PV => charge battery or injection into grid
The electricity usage of homes is measured each 15 minutes, so I have 96 measurement point in 1 day. I want to optimilize the charging and decharging of the battery for 2 days, so that day 1 takes the usage of day 2 into account.
I wrote a controller that reads the data and gives each time the input values for 2 days to the optimization. With a rolling principle, it goes to the next 2 days and so on. Below you can see the code from my controller.
from gekko import GEKKO
from simulationModel2_2d_1 import getSimulation2
from exportModel2 import exportToExcelModel2
import numpy as np
#import matplotlib.pyplot as plt
import pandas as pd
import time
import math
# ------------------------ Import and read input data ------------------------
file = r'Data Sim 2.xlsx'
data = pd.read_excel(file, sheet_name='Input', na_values='NaN')
dataRead = pd.DataFrame(data, columns= ['Timestep','Verbruik woning (kWh)','Netto afname (kWh)','Prijs afname (€/kWh)',
'Prijs injectie (€/kWh)','Capaciteit batterij (kW)',
'Capaciteit batterij (kWh)','Rendement (%)',
'Verbruikersprofiel','Capaciteit PV (kWp)','Aantal dagen'])
timestep = dataRead['Timestep'].to_numpy()
usage_home = dataRead['Verbruik woning (kWh)'].to_numpy()
net_offtake = dataRead['Netto afname (kWh)'].to_numpy()
price_offtake = dataRead['Prijs afname (€/kWh)'].to_numpy()
price_injection = dataRead['Prijs injectie (€/kWh)'].to_numpy()
cap_batt_kW = dataRead['Capaciteit batterij (kW)'].iloc[0]
cap_batt_kWh = dataRead['Capaciteit batterij (kWh)'].iloc[0]
efficiency = dataRead['Rendement (%)'].iloc[0]
usersprofile = dataRead['Verbruikersprofiel'].iloc[0]
days = dataRead['Aantal dagen'].iloc[0]
pv = dataRead['Capaciteit PV (kWp)'].iloc[0]
# ------------- Optimization model & Rolling principle (2 days) --------------
# Initialise model
m = GEKKO()
# Output data
ts = []
charging = [] # Amount to charge/decharge batterij
e_batt = [] # Amount of energy in the battery
usage_net = [] # Usage after home, battery and pv
p_paid = [] # Price paid for energy of 15min
# Energy in battery to pass
energy = 0
# Iterate each day for one year
for d in range(int(days)-1):
d1_timestep = []
d1_net_offtake = []
d1_price_offtake = []
d1_price_injection = []
d2_timestep = []
d2_net_offtake = []
d2_price_offtake = []
d2_price_injection = []
# Iterate timesteps
for i in range(96):
d1_timestep.append(timestep[d*96+i])
d2_timestep.append(timestep[d*96+i+96])
d1_net_offtake.append(net_offtake[d*96+i])
d2_net_offtake.append(net_offtake[d*96+i+96])
d1_price_offtake.append(price_offtake[d*96+i])
d2_price_offtake.append(price_offtake[d*96+i+96])
d1_price_injection.append(price_injection[d*96+i])
d2_price_injection.append(price_injection[d*96+i+96])
# Input data simulation of 2 days
ts_temp = np.concatenate((d1_timestep, d2_timestep))
net_offtake_temp = np.concatenate((d1_net_offtake, d2_net_offtake))
price_offtake_temp = np.concatenate((d1_price_offtake, d2_price_offtake))
price_injection_temp = np.concatenate((d1_price_injection, d2_price_injection))
if(d == 7):
print(ts_temp)
print(energy)
# Simulatie uitvoeren
charging_temp, e_batt_temp, usage_net_temp, p_paid_temp, energy_temp = getSimulation2(ts_temp, net_offtake_temp, price_offtake_temp, price_injection_temp, cap_batt_kW, cap_batt_kWh, efficiency, energy, pv)
# Take over output first day, unless last 2 days
energy = energy_temp
if(d == (days-2)):
for t in range(1,len(ts_temp)):
ts.append(ts_temp[t])
charging.append(charging_temp[t])
e_batt.append(e_batt_temp[t])
usage_net.append(usage_net_temp[t])
p_paid.append(p_paid_temp[t])
elif(d == 0):
for t in range(int(len(ts_temp)/2)+1):
ts.append(ts_temp[t])
charging.append(charging_temp[t])
e_batt.append(e_batt_temp[t])
usage_net.append(usage_net_temp[t])
p_paid.append(p_paid_temp[t])
else:
for t in range(1,int(len(ts_temp)/2)+1):
ts.append(ts_temp[t])
charging.append(charging_temp[t])
e_batt.append(e_batt_temp[t])
usage_net.append(usage_net_temp[t])
p_paid.append(p_paid_temp[t])
print('Simulation day '+str(d+1)+' complete.')
# ------------------------ Export output data to Excel -----------------------
a = exportToExcelModel2(ts, usage_home, net_offtake, price_offtake, price_injection, charging, e_batt, usage_net, p_paid, cap_batt_kW, cap_batt_kWh, efficiency, usersprofile, pv)
print(a)
The optimization with Gekko happens in the following code:
from gekko import GEKKO
def getSimulation2(timestep, net_offtake, price_offtake, price_injection,
cap_batt_kW, cap_batt_kWh, efficiency, start_energy, pv):
# ---------------------------- Optimization model ----------------------------
# Initialise model
m = GEKKO(remote = False)
# Global options
m.options.SOLVER = 1
m.options.IMODE = 6
# Constants
speed_charging = cap_batt_kW/4
m.time = timestep
max_cap_batt = m.Const(value = cap_batt_kWh)
min_cap_batt = m.Const(value = 0)
max_charge = m.Const(value = speed_charging) # max battery can charge in 15min
max_decharge = m.Const(value = -speed_charging) # max battery can decharge in 15min
# Parameters
usage_home = m.Param(net_offtake)
price_offtake = m.Param(price_offtake)
price_injection = m.Param(price_injection)
# Variables
e_batt = m.Var(value=start_energy, lb = min_cap_batt, ub = max_cap_batt) # energy in battery
price_paid = m.Var() # price paid each 15min
charging = m.Var(lb = max_decharge, ub = max_charge) # amount charge/decharge each 15min
usage_net = m.Var(lb=min_cap_batt)
# Equations
m.Equation(e_batt==(m.integral(charging)+start_energy)*efficiency)
m.Equation(-charging <= e_batt)
m.Equation(usage_net==usage_home + charging)
price = m.Intermediate(m.if2(usage_net*1e6, price_injection, price_offtake))
price_paid = m.Intermediate(usage_net * price / 100)
# Objective
m.Minimize(price_paid)
# Solve problem
m.options.COLDSTART=2
m.solve()
m.options.TIME_SHIFT=0
m.options.COLDSTART=0
m.solve()
# Energy to pass
energy_left = e_batt[95]
#m.cleanup()
return charging, e_batt, usage_net, price_paid, energy_left
The data you need for input can be found in this Excel document:
https://docs.google.com/spreadsheets/d/1S40Ut9-eN_PrftPCNPoWl8WDDQtu54f0/edit?usp=sharing&ouid=104786612700360067470&rtpof=true&sd=true
With this code, it always ends at day 17 with the "Solution Not Found" Error.
I already tried extending the default iteration limit to 500 but it didn't work.
I also tried with other solvers but also no improvement.
By presolving with COLDSTART it already reached day 17, without this it ends at day 8.
I solved the days were my optimization ends individually and then the solution was always found immediately with the same code.
Someone who can explain this to me and maybe find a solution? Thanks in advance!
This is kind of big to troubleshoot, but here are some general ideas that might help. This assumes, as you said, that the model solves fine for day 1-2, and day 3-4, and day 5-6, etc. And that those results pass inspection (aka the basic model is working as you say).
Then something is (obviously) amiss around day 17. Some things to look at and try:
Start the model at day 16-17, see if it works in isolation
gather your results as you go and do a time series plot of the key variables, maybe one of them is on an obvious bad trend towards a limit causing an infeasibility... Perhaps the e_batt variable is slowly declining because not enough PV energy is available and hits minimum on Day 17
Radically change the upper/lower bounds on your variables to test them to see if they might be involved in the infeasibility (assuming there is one)
Make a different (fake) dataset where the solution is assured and kind of obvious... all constants or a pattern that you know will produce some known result and test the model outputs against it.
It might also be useful to pursue the tips in this excellent post on troubleshooting gekko, and edit your question with the results of that effort.
edit: couple ideas from your comment...
You didn't say what the result was from troubleshooting and whether the error is infeasible or max iterations, or ???. But...
If the model seems to crunch after about 15 days, I'm betting that it is becoming infeasible. Did you plot the battery level over course of days?
Also, I'm suspicious of your equation for the e_batt level... Why are you multiplying the prior battery state by the efficiency factor? That seems incorrect. That is charge that is already in the battery. Odds are you are (incorrectly) hitting the battery charge level every day with the efficiency tax and that the max charge level isn't sufficient to keep up with demand.
In addition to tips above try:
fix your efficiency formula to not multiply the efficiency times the previous state
change the efficiency to 100%
make the upper limit on charge huge
As an aside: I don't really see the connection to PV energy available here. What you are basically modeling is some "mystery battery" that you can charge and discharge anytime you want. I would get this debugged first and then look at the energy available by time of day...you aren't going to be generating charge at midnight. :).
I want to fetche the mid price of BTC from Binance, but not to sure on how.
That's my code:
import requests
symbol = "BTCUSDT"
res = requests.get(f"https://api.binance.com/api/v3/ticker/bookTicker?symbol={symbol}").json()
mid = (float(res["bidPrice"]) + float(res["askPrice"])) / 2
print(mid)
I am a beginner... so please let me know if i have a bug :)
I am trying to analyze trips in a long time period in r5r that requires more than one GTFS files. I am using a for loop since I want to study trips in various depature dates in the Excel file. Right now, I have placed all three GTFS.zip files with different names in the data path together, but I could only receive mode information by public transportation within one date range, while trips in the other two dates produced walk time only. Is there a way to let r5r include all of them?
options(java.parameters = "-Xmx16G")
library(r5r)
library(sf)
library(data.table)
File_Path = file.path("C:","Research", "Data Sep", fsep = .Platform$file.sep)
list.files(File_Path)
poi <- fread(file.path(File_Path, "OriginsDestinationsPugetSound.csv"))
r5r_core <- setup_r5(data_path = File_Path, verbose = TRUE)
mode <- c("WALK","TRANSIT")
max_walk_dist <- 1000 # in meters
max_trip_duration <- 300 # in minutes
LengthOfFile = length(poi[[1]])
ListOfDetailedItineries = (matrix(ncol = 15,nrow = 0))
start = 1
end = 25
for (i in start:end) {
OriginPoint = poi[i,2:4]
DestinationPoint = poi[i,5:7]
Time_of_Trip = poi[i,9]
departure_datetime = as.POSIXct(Time_of_Trip[[1]], format = "%m/%d/%Y %H:%M")
dit <- detailed_itineraries(r5r_core = r5r_core,
origins = OriginPoint,
destinations = DestinationPoint,
mode = mode,
departure_datetime = departure_datetime,
max_walk_dist = max_walk_dist,
max_trip_duration = max_trip_duration,
shortest_path = TRUE,
verbose = TRUE)
ListOfDetailedItineries = rbind(ListOfDetailedItineries, as.matrix(dit))
cat('On iteration ',i,'\n',dit[[9]],"\n")
flush.console()
}
dit1 = as.data.frame(ListOfDetailedItineries)
As far as I understood, you have 3 feeds inside your data directory and you tried to generate travel time estimates for 3 different departure times, but only one of them return transit public transport trips. Is that correct?
If that's the case, you have to make sure that the public transit services listed in your GTFS feeds run on your specified departure times. This information is usually listed in the calendar table, but can also be listed in the calendar_dates table in some feeds.
The best practice here would be to choose dates that fall inside the service intervals of all 3 of your feeds. Alternatively, you can edit the start_date/end_date columns of their calendar table to include the days you have already chosen.
I have posted invoice in xero & passing data in 4 decimal points of unit price but xero is converting it into 2 decimal points, so i am getting total amount issue due to rounding...
Through API I know unitdp=4 have to pass but how to do same thing with sdks?
LineItem qboln = new LineItem();
qboln.Description = "Desc"
qboln.Quantity = 3385;
qboln.UnitAmount = 0.7550;
qboln.AccountCode = code;
qboln.TaxType = "INPUT";
qboln.TaxAmount = 0.678;
inv.LineItems.Add(qboln);
so here xero is rounding unit price into 0.76 so I am getting total amount issue that is different with my original invoice.
Please tell me how to solve it ?
I need to calculate messages count in telegram or get messages history.
Tell me, is it possible to do this?
I know, that i can see messages count in chat from some member in telegram desktop. Maybe i can do this in any conversation?
Thank you!
You can count messages in telegram, get messages history and do plenty of other things with Telegram API. Here is a brilliant article that describes the process step by step: https://towardsdatascience.com/introduction-to-the-telegram-api-b0cd220dbed2
It worked like a charm for me both for retrieving messages and counting them. Let's look at the code from the article:
counts = {}
# create dictionary of ids to users and chats
users = {}
chats = {}
for u in dialogs.users:
users[u.id] = u
for c in dialogs.chats:
chats[c.id] = c
for d in dialogs.dialogs:
peer = d.peer
if isinstance(peer, PeerChannel):
id = peer.channel_id
channel = chats[id]
access_hash = channel.access_hash
name = channel.title
input_peer = InputPeerChannel(id, access_hash)
elif isinstance(peer, PeerChat):
id = peer.chat_id
group = chats[id]
name = group.title
input_peer = InputPeerChat(id)
elif isinstance(peer, PeerUser):
id = peer.user_id
user = users[id]
access_hash = user.access_hash
name = user.first_name
input_peer = InputPeerUser(id, access_hash)
else:
continue
get_history = GetHistoryRequest(
peer=input_peer,
offset_id=0,
offset_date=None,
add_offset=0,
limit=1,
max_id=0,
min_id=0,
)
history = client(get_history)
if isinstance(history, Messages):
count = len(history.messages)
else:
count = history.count
counts[name] = count
print(counts)
Let's add sorting:
sorted_counts = sorted(counts.items(), key=lambda x: x[1], reverse=True)
for name, count in sorted_counts:
print('{}: {}'.format(name, count))
We got messages counter result:
Group chat 1: 10000
Group chat 2: 3003
Channel 1: 2000
Chat 1: 1500
Chat 2: 300
P.S. Here is a simple (user's) way to get some chats from official wiki, but from my perspective it's very limited and unsuitable for programmatic purposes:
https://telegram.wiki/general/exporting-chats