What is wrong with my while loop (Python)? - while-loop

I am quite new to programming and I face a strange issue with the below while-loop in Python. When I manually run the code below the loop (i.e. starting with "time now..." to "t += 1"), I get the output I want:
The time is now: 13:31:01
Your current position is -1
This is iteration: 1
The time is now: 13:32:01
Your current position is -1
This is iteration: 2
However, when I run the full loop, I get double, triple or multiple prints of the same minute (see output below the while-loop code).
Do you see the issue with this loop? Its as if the "t += 1" increment is running several times per loop (i.e. for each minute several prints). I dont get it.
Thanks for your help!
Blockquote
t=1
while t < 2000:
time_now = (time.strftime('%H:%M:%S', time.localtime(int(time.time()))))
if time_now[6:8] == str('00'):
sleep(1)
elif time_now[6:8] == str('01'):
sleep(0)
else:
x = 61 - int(time_now[6:8])
sleep(x)
time_now = (time.strftime('%H:%M:%S', time.localtime(int(time.time()))))
print("The time is now: " + time_now)
#+1 is the middle bar
totalBars = leftBars + rightBars + 1
swing_candles = client.Trade.Trade_getBucketed(symbol=symbol, binSize="1m", count=totalBars, reverse=True).result()[0]
last_highs = []
last_lows = []
i=0
while i <= (len(swing_candles)-1):
last_highs.append(swing_candles[i]["high"])
last_lows.append(swing_candles[i]["low"])
i += 1
#get the highest high and the lowest low
highest_high = max(last_highs)
lowest_low = min(last_lows)
#check if there are existing positions & orders
positions_quantity = client.Position.Position_get().result()[0][0]["currentQty"]
#check existing orders
buy_orders_quantity = []
sell_orders_quantity = []
orders_quantity = client.Order.Order_getOrders(filter=json.dumps({"open": True})).result()[0]
h=0
while h <= len(orders_quantity)-1:
if orders_quantity[h]["side"] == "Sell":
sell_orders_quantity.append(orders_quantity[h])
elif orders_quantity[h]["side"] == "Buy":
buy_orders_quantity.append(orders_quantity[h])
h += 1
if highest_high == last_highs[rightBars] and positions_quantity == 0:
if buy_orders_quantity == []:
client.Order.Order_new(symbol = symbol, orderQty = orderQty*1, side = "Buy", ordType = 'Stop', stopPx = highest_high, execInst ='LastPrice' ).result()
elif buy_orders_quantity != []:
orderID = buy_orders_quantity[0]["orderID"]
client.Order.Order_amend(orderID=orderID, orderQty=orderQty*1, stopPx = highest_high).result()
else:
pass
elif highest_high == last_highs[rightBars] and positions_quantity > 0:
#dont place any additional long
pass
elif highest_high == last_highs[rightBars] and positions_quantity < 0:
if buy_orders_quantity != []:
orderID = buy_orders_quantity[0]["orderID"]
client.Order.Order_amend(orderID=orderID, orderQty=orderQty*2, stopPx = highest_high).result()
else:
client.Order.Order_new(symbol = symbol, orderQty = (orderQty)*2, side = "Buy", ordType = 'Stop', stopPx = highest_high, execInst ='LastPrice' ).result()
elif lowest_low == last_lows[rightBars] and positions_quantity == 0:
if sell_orders_quantity == []:
client.Order.Order_new(symbol = symbol, orderQty = (orderQty)*-1, side = "Sell", ordType = 'Stop', stopPx = lowest_low, execInst ='LastPrice' ).result()
elif sell_orders_quantity != []:
orderID = sell_orders_quantity[0]["orderID"]
client.Order.Order_amend(orderID=orderID, orderQty=orderQty*-1, stopPx = lowest_low ).result()
else:
pass
elif lowest_low == last_lows[rightBars] and positions_quantity < 0:
#dont place any additional shorts
pass
elif lowest_low == last_lows[rightBars] and positions_quantity > 0:
if sell_orders_quantity != []:
orderID = sell_orders_quantity[0]["orderID"]
client.Order.Order_amend(orderID=orderID, orderQty=orderQty*-2, stopPx = lowest_low).result()
else:
#if there is no order, place new order with double amount
client.Order.Order_new(symbol = symbol, orderQty = (orderQty)*-2, side = "Sell", ordType = 'Stop', stopPx = lowest_low, execInst ='LastPrice' ).result()
positions_quantity = client.Position.Position_get().result()[0][0]["currentQty"]
print("Your current position is " + str(positions_quantity))
print("This is iteration: " + str(t))
t += 1
This is the output when I run the while-loop above (however each minute there should be just one print, but starting with iteration two there are several prints for the same minute (loop)):
The time is now: 13:39:01
Your current position is -1
This is iteration: 1
The time is now: 13:39:01
Your current position is -1
This is iteration: 2
The time is now: 13:40:01
Your current position is -1
This is iteration: 3
The time is now: 13:40:01
Your current position is -1
This is iteration: 4
The time is now: 13:40:01
Your current position is -1
This is iteration: 5
The time is now: 13:40:01
Your current position is -1
This is iteration: 6
The time is now: 13:40:01
Your current position is -1
This is iteration: 7
The time is now: 13:41:01
Your current position is -1
This is iteration: 8
The time is now: 13:41:01
Your current position is -1
This is iteration: 9
The time is now: 13:41:01
Your current position is -1
This is iteration: 10
The time is now: 13:41:01
Your current position is -1
This is iteration: 11
The time is now: 13:41:01
Your current position is -1
This is iteration: 12

Assuming that "running the full loop" (scenario A) is faster than when you "manually runtime code" (scenario B), what appears to be happening is that, when the # of seconds in time_now is "01", the scenario A is fast enough to complete a few iterations with the # of seconds in time_now staying at "01", while scenario B is not.
If you slept for at least 1 second when the # of seconds in time_now was "01", that should prevent this, as it would change the # of seconds in time_now.

Related

SMOTE adds many rows with 0 values to dataframe

Please help me, i cannot understand why X_synthetic_df returns hundreds of rows with 0 values. All of the rows have normal values fine until row 1745. From that row, all the other row values contain nothing but zeros
def nearest_neighbors(nominal_columns, numeric_columns, df, row, k):
def distance(row1, row2):
distance = 0
for col in nominal_columns:
if row1[col] != row2[col]:
distance += 1
for col in numeric_columns:
distance += (row1[col] - row2[col])**2
return distance**0.5
distances = []
for i in range(len(df)):
r = df.iloc[i]
if r.equals(row):
continue
d = distance(row, r)
if(d!=0):
distances.append((d, i))
distances.sort()
nearest = [i for d, i in distances[:k]]
return nearest
def smotenc(X, y, nominal_cols, numeric_cols, k=5, seed=None):
minority_class = y[y==1]
majority_class = y[y==0]
minority_samples = X[y == 1]
minority_target = y[y == 1]
n_synthetic_samples = len(majority_class)-len(minority_class)
synthetic_samples = np.zeros((n_synthetic_samples, X.shape[1]))
if seed is not None:
np.random.seed(seed)
for i in range(len(minority_samples)):
nn = nearest_neighbors(nominal_cols, numeric_cols, minority_samples, minority_samples.iloc[i], k=k)
for j in range(min(k, n_synthetic_samples - i*k)):
nn_idx = int(np.random.choice(a=nn))
diff = minority_samples.iloc[(nn_idx)] - minority_samples.iloc[i]
print(diff)
if (diff == 0).all():
continue
synthetic_sample = minority_samples.iloc[i] + np.random.rand() * diff
synthetic_samples[(i*k)+j, :] = synthetic_sample
X_resampled = pd.concat([X[y == 1], pd.DataFrame(synthetic_samples,columns=X.columns)], axis=0)
y_resampled = np.concatenate((y[y == 1], [1] * n_synthetic_samples))
return X_resampled, y_resampled
minority_features = df_nominal.columns.get_indexer(df_nominal.columns)
synthetic = smotenc(check_x.head(3000),check_y.head(3000),nominal_cols,numeric_cols,seed = None)
X_synthetic_df = synthetic[0]
X_synthetic_df = pd.DataFrame(X_synthetic_df,columns = X.columns)
I was a dataframe with n synthetic samples, where n is the difference between the majority samples and minority class samples

Pi Pico Micropython Loop seems to get stuck

I am using a Pico as part of my dissertation along with an Arduino Due for a low cost fully autonomous UAV.
Anyway to the point, the code was working before I added the GPS part but now even if I take it out the ultrasonics won't work correctly. only shows the left sensor and doesn't continue on to the GPS.
I was also trying to use the multithreader with no joy :(
Any help or suggestions would be great.
Its a bit messy I know not quite got round to polishing it.
from machine import Pin
import utime, _thread, machine
import os
from rp2 import PIO, StateMachine, asm_pio
#print sys info
print(os.uname())
led_onboard = machine.Pin(25, machine.Pin.OUT)
led_onboard.value(0) # onboard LED OFF for 0.5 sec
utime.sleep(0.5)
led_onboard.value(1)
uart = machine.UART(0, baudrate=9600)
ser = machine.UART(1, baudrate=9600)
print(uart)
print(ser)
baton = _thread.allocate_lock()
rcvChar = b""
trigger = Pin(3, Pin.OUT) #left
echo = Pin(2, Pin.IN)
trigger2 = Pin(6, Pin.OUT) #right
echo2 = Pin(7, Pin.IN)
trigger4 = Pin(9, Pin.OUT) #backward
echo4 = Pin(8, Pin.IN)
def decode(coord):
#Converts DDDMM.MMMMM > DD deg MM.MMMMM min
x = coord.split(".")
head = x[0]
tail = x[1]
deg = head[0:-2]
min = head[-2:]
return deg + " deg " + min + "." + tail + " min"
def ultraleft():
trigger.low()
utime.sleep_us(2)
trigger.high()
utime.sleep_us(5)
trigger.low()
while echo.value() == 0:
signaloff = utime.ticks_us()
while echo.value() == 1:
signalon = utime.ticks_us()
timepassed = signalon - signaloff
Ldistance = (timepassed * 0.0343) / 2
utime.sleep(0.1)
trigger.low()
utime.sleep_us(2)
trigger.high()
utime.sleep_us(5)
trigger.low()
while echo.value() == 0:
signaloff = utime.ticks_us()
while echo.value() == 1:
signalon = utime.ticks_us()
timepassed = signalon - signaloff
Ldistance2 = (timepassed * 0.0343) / 2
newLdist = (Ldistance + Ldistance2) / 2
if newLdist > 120:
newLdist = 120
elif newLdist <= 100:
print("Distance Left less than 100")
return True
print("The distance Left from object is ",newLdist,"cm")
def ultraright():
trigger2.low()
utime.sleep_us(2)
trigger2.high()
utime.sleep_us(5)
trigger2.low()
while echo2.value() == 0:
signaloff2 = utime.ticks_us()
while echo2.value() == 1:
signalon2 = utime.ticks_us()
timepassed2 = signalon2 - signaloff2
Rdistance = (timepassed2 * 0.0343) / 2
utime.sleep(0.1)
trigger2.low()
utime.sleep_us(2)
trigger2.high()
utime.sleep_us(5)
trigger2.low()
while echo2.value() == 0:
signaloff2 = utime.ticks_us()
while echo2.value() == 1:
signalon2 = utime.ticks_us()
timepassed2 = signalon2 - signaloff2
Rdistance2 = (timepassed2 * 0.0343) / 2
newRdist = (Rdistance + Rdistance2) / 2
if newRdist > 120:
newRdist = 120
elif newRdist <= 100:
print("Distance Right less than 100")
return True
print("The distance Right from object is ",newRdist,"cm")
def ultradwn():
trigger4.low()
utime.sleep_us(2)
trigger4.high()
utime.sleep_us(5)
trigger4.low()
while echo4.value() == 0:
signaloff4 = utime.ticks_us()
while echo4.value() == 1:
signalon4 = utime.ticks_us()
timepassed4 = signalon4 - signaloff4
Ddistance = (timepassed4 * 0.0343) / 2
utime.sleep(0.1)
trigger4.low()
utime.sleep_us(2)
trigger4.high()
utime.sleep_us(5)
trigger4.low()
while echo4.value() == 0:
signaloff4 = utime.ticks_us()
while echo4.value() == 1:
signalon4 = utime.ticks_us()
timepassed4 = signalon4 - signaloff4
Ddistance2 = (timepassed4 * 0.0343) / 2
newDdist = (Ddistance + Ddistance2) / 2
if newDdist > 120:
newDdist = 120
elif newDdist >20 :
print("Distance Down is greater than 20")
x = 1
#uart.write("D20")
#uart.write("\n")
#print("Sent TO Height")
return True
elif newDdist <12 :
print("Distance Down is less than 12")
x = 2
#uart.write("D12")
#uart.write("\n")
#print("Sent Landed")
return True
print("The distance Down from object is ",newDdist,"cm")
def gps():
while True:
#baton.acquire()
rcvChar = ser.readline()
gps_data =rcvChar.decode("ASCII")
data = gps_data
if (data[0:6] == "$GPRMC"):
sdata = data.split(",")
if (sdata[2] == 'V'):
print("no satellite data available")
print ("---Parsing GPRMC---")
time = sdata[1][0:2] + ":" + sdata[1][2:4] + ":" + sdata[1][4:6]
lat = decode(sdata[3]) #latitude
dirLat = sdata[4] #latitude direction N/S
lon = decode(sdata[5]) #longitute
dirLon = sdata[6] #longitude direction E/W
speed = sdata[7] #Speed in knots
trCourse = sdata[8] #True course
date = sdata[9][0:2] + "/" + sdata[9][2:4] + "/" + sdata[9][4:6]#date
print ("time : %s, latitude : %s(%s), longitude : %s(%s), speed : %s, True Course : %s, Date : %s" % (time,lat,dirLat,lon,dirLon,speed,trCourse,date))
#baton.acquire()
#_thread.start_new_thread(gps(), ())
while True:
x = 0
#baton.acquire()
ultraleft()
utime.sleep(0.1)
ultraright()
utime.sleep(0.1)
ultradwn()
utime.sleep(0.1)
if ultraleft():
uart.write("LO")
uart.write("\n")
print("Sent Left")
utime.sleep(1)
if ultraright():
uart.write("RO")
uart.write("\n")
print("Sent Right")
uart.sendbreak()
utime.sleep(1)
if ultradwn():
if x == 1:
uart.write("D20")
uart.write("\n")
print("Sent TO Height")
utime.sleep(1)
if x == 2:
uart.write("D12")
uart.write("\n")
print("Sent Landed")
utime.sleep(1)
utime.sleep(1)
gps()
#baton.release()

Why doesn't this return the tuple and instead returns nonetype?

Why doesn't this return the tuple and instead returns nonetype? I've tried removing everything I could think of to get it to return the tuple. I either have mistyped something which i have spent an hour or so checking, or i am missing something bigger..
Thanks!!
import time
def BONGO_BONGO(item, num=1, wait_time=4):
time.sleep(.99)
list_of_positions = []
if item in list_of_positions:
result = (1, 333)
return result
elif num >= wait_time:
print(str(wait_time) + " seconds have passed.. ")
print("test")
result = (2, 9)
print(result)
return result
else:
print(str(item) + " execution checked: " + str(num))
num += 1
BONGO_BONGO(item=item, num=num)
y = BONGO_BONGO("ITEM444")
if __name__ == "__main__":
print(y)
Okay the answer is that the nested bongo_bongo function under the "else" is the function that in the end will return the desired tuple from the "elif". So you need to return that last bongo_bongo function under the "else".
import time
def BONGO_BONGO(item, num=1, wait_time=4):
time.sleep(.99)
list_of_positions = []
if item in list_of_positions:
result = (1, 333)
return result
elif num >= wait_time:
print(str(wait_time) + " seconds have passed.. ")
print("test")
result = (2, 9)
print(result)
return result
else:
print(str(item) + " execution checked: " + str(num))
num += 1
result = BONGO_BONGO(item=item, num=num)
return result
y = BONGO_BONGO("ITEM444")
if __name__ == "__main__":
print("Y=", y)

Why is setting the expiry time ineffective in Redis

Now i use redispipeline to set key to redis ,and set key timeout.After running codes and timeout,and the keys are still in redis?It seemed doesn't work.Is there sth wrong with my code.
X[1] is dict,such as dict["a"]=b dict["c"]=d
redis_pool = redis.ConnectionPool(host = redis_ip, port = redis_port, decode_responses = False)
redis_connection = redis.StrictRedis(connection_pool = redis_pool)
redis_pipeline = redis_connection.pipeline(transaction = False)
sync_count = 0
for x in iterator:
key = suffix + x[0]
#value = str(x['tagid'])+"\t"+x['tag_name']
value = x[1]
redis_pipeline.hmset(key, value)
redis_pipeline.expire(key,60)
sync_count += 1
if (sync_count % 100 == 0):
result = redis_pipeline.execute()
print (result)
time.sleep(0.001)
break
if sync_count == 10000:
break
redis_pipeline.execute()

How to declare constraints with variable as array index in Z3Py?

Suppose x,y,z are int variables and A is a matrix, I want to express a constraint like:
z == A[x][y]
However this leads to an error:
TypeError: object cannot be interpreted as an index
What would be the correct way to do this?
=======================
A specific example:
I want to select 2 items with the best combination score,
where the score is given by the value of each item and a bonus on the selection pair.
For example,
for 3 items: a, b, c with related value [1,2,1], and the bonus on pairs (a,b) = 2, (a,c)=5, (b,c) = 3, the best selection is (a,c), because it has the highest score: 1 + 1 + 5 = 7.
My question is how to represent the constraint of selection bonus.
Suppose CHOICE[0] and CHOICE[1] are the selection variables and B is the bonus variable.
The ideal constraint should be:
B = bonus[CHOICE[0]][CHOICE[1]]
but it results in TypeError: object cannot be interpreted as an index
I know another way is to use a nested for to instantiate first the CHOICE, then represent B, but this is really inefficient for large quantity of data.
Could any expert suggest me a better solution please?
If someone wants to play a toy example, here's the code:
from z3 import *
items = [0,1,2]
value = [1,2,1]
bonus = [[1,2,5],
[2,1,3],
[5,3,1]]
choices = [0,1]
# selection score
SCORE = [ Int('SCORE_%s' % i) for i in choices ]
# bonus
B = Int('B')
# final score
metric = Int('metric')
# selection variable
CHOICE = [ Int('CHOICE_%s' % i) for i in choices ]
# variable domain
domain_choice = [ And(0 <= CHOICE[i], CHOICE[i] < len(items)) for i in choices ]
# selection implication
constraint_sel = []
for c in choices:
for i in items:
constraint_sel += [Implies(CHOICE[c] == i, SCORE[c] == value[i])]
# choice not the same
constraint_neq = [CHOICE[0] != CHOICE[1]]
# bonus constraint. uncomment it to see the issue
# constraint_b = [B == bonus[val(CHOICE[0])][val(CHOICE[1])]]
# metric definition
constraint_sumscore = [metric == sum([SCORE[i] for i in choices ]) + B]
constraints = constraint_sumscore + constraint_sel + domain_choice + constraint_neq + constraint_b
opt = Optimize()
opt.add(constraints)
opt.maximize(metric)
s = []
if opt.check() == sat:
m = opt.model()
print [ m.evaluate(CHOICE[i]) for i in choices ]
print m.evaluate(metric)
else:
print "failed to solve"
Turns out the best way to deal with this problem is to actually not use arrays at all, but simply create integer variables. With this method, the 317x317 item problem originally posted actually gets solved in about 40 seconds on my relatively old computer:
[ 0.01s] Data loaded
[ 2.06s] Variables defined
[37.90s] Constraints added
[38.95s] Solved:
c0 = 19
c1 = 99
maxVal = 27
Note that the actual "solution" is found in about a second! But adding all the required constraints takes the bulk of the 40 seconds spent. Here's the encoding:
from z3 import *
import sys
import json
import sys
import time
start = time.time()
def tprint(s):
global start
now = time.time()
etime = now - start
print "[%ss] %s" % ('{0:5.2f}'.format(etime), s)
# load data
with open('data.json') as data_file:
dic = json.load(data_file)
tprint("Data loaded")
items = dic['items']
valueVals = dic['value']
bonusVals = dic['bonusVals']
vals = [[Int("val_%d_%d" % (i, j)) for j in items if j > i] for i in items]
tprint("Variables defined")
opt = Optimize()
for i in items:
for j in items:
if j > i:
opt.add(vals[i][j-i-1] == valueVals[i] + valueVals[j] + bonusVals[i][j])
c0, c1 = Ints('c0 c1')
maxVal = Int('maxVal')
opt.add(Or([Or([And(c0 == i, c1 == j, maxVal == vals[i][j-i-1]) for j in items if j > i]) for i in items]))
tprint("Constraints added")
opt.maximize(maxVal)
r = opt.check ()
if r == unsat or r == unknown:
raise Z3Exception("Failed")
tprint("Solved:")
m = opt.model()
print " c0 = %s" % m[c0]
print " c1 = %s" % m[c1]
print " maxVal = %s" % m[maxVal]
I think this is as fast as it'll get with Z3 for this problem. Of course, if you want to maximize multiple metrics, then you can probably structure the code so that you can reuse most of the constraints, thus amortizing the cost of constructing the model just once, and incrementally optimizing afterwards for optimal performance.