Conditional Replacement in Keras Backend - tensorflow

Target:
Hey i want to build a custom metrics method, where different specific classes are allowed do be confused with specific (because their idea in some conditions in nearly the same).
How it could be reached:
I got 25 classes, where the following confusions should be allowed
(1 can be 2,3,4) ; (2 can be 3,4) ; (3 can be 4) ;(15 can be 16,17,18);(16 can be 17) ; (17 can be 18) ; (6 can be 7,8 ) ; (7 can be 8 ); (19 can be 20,21 ) ; (20 can be 21 );(10 can be 11 );(13 can be 14 );(22 can be 23 ); (24 can be 14 );
quite much huh... therefore i put it into a mask:
mask = mask =(y_true!=y_pred)&(((y_true==1)&((y_pred == 2)| (y_pred == 3)| (y_pred == 4))) | ((y_true==2) & ((y_pred == 3)|(y_pred == 4))) | ((y_true==3)&((y_pred == 4)))| ((y_true==15)&((y_pred == 16) | (y_pred == 17)| (y_pred == 18))) | ((y_true==16) & ((y_pred == 17)|(y_pred == 18))) | ((y_true==17)&((y_pred == 18))) | ((y_true==6) & ((y_pred == 7)|(y_pred == 8)))|((y_true==7)&((y_pred == 8)))| ((y_true==19) & ((y_pred == 20)|(y_pred == 21)))|((y_true==20)&((y_pred == 21)))|((y_true==10)&((y_pred == 11)))|((y_true==13)&((y_pred == 14)))|((y_true==22)&((y_pred == 23)))|((y_true==24)&((y_pred == 14))) )
I was hoping to find a function (equivalent to np.where) to conditional replace the values in the "y_pred" which fit the condition, by the "y_true" tensors values at the same index.
I thought tf.where_v2() could work (sadly, i wasnt able to check if it works because tf.equal() and tf.reduce_all() only have limited abilities to work in a metrics function (or am I wrong?)
the following creates no debug errors
def profit_metrics(y_true, y_pred):
mask =(y_true!=y_pred)&(((y_true==1)&((y_pred == 2)| (y_pred == 3)| (y_pred == 4))) | ((y_true==2) & ((y_pred == 3)|(y_pred == 4))) | ((y_true==3)&((y_pred == 4)))| ((y_true==15)&((y_pred == 16) | (y_pred == 17)| (y_pred == 18))) | ((y_true==16) & ((y_pred == 17)|(y_pred == 18))) | ((y_true==17)&((y_pred == 18))) | ((y_true==6) & ((y_pred == 7)|(y_pred == 8)))|((y_true==7)&((y_pred == 8)))| ((y_true==19) & ((y_pred == 20)|(y_pred == 21)))|((y_true==20)&((y_pred == 21)))|((y_true==10)&((y_pred == 11)))|((y_true==13)&((y_pred == 14)))|((y_true==22)&((y_pred == 23)))|((y_true==24)&((y_pred == 14))) )
new_pred = tf.where_v2(mask,y_true,y_pred)
return sparse_categorical_accuracy(y_true, new_pred)
(...)
model.compile(...,metrics=['accuracy',profit_metrics,])
but the thing is... it returns exactly the same accuracy value as the standard accuracy method!
I tried to check if both tensors are still the same, but sadly all i tried didnt work out, maybe because i need to use only Keras backend functions?
def profit_metrics(y_true, y_pred):
mask =(y_true!=y_pred)&(((y_true==1)&((y_pred == 2)| (y_pred == 3)| (y_pred == 4))) | ((y_true==2) & ((y_pred == 3)|(y_pred == 4))) | ((y_true==3)&((y_pred == 4)))| ((y_true==15)&((y_pred == 16) | (y_pred == 17)| (y_pred == 18))) | ((y_true==16) & ((y_pred == 17)|(y_pred == 18))) | ((y_true==17)&((y_pred == 18))) | ((y_true==6) & ((y_pred == 7)|(y_pred == 8)))|((y_true==7)&((y_pred == 8)))| ((y_true==19) & ((y_pred == 20)|(y_pred == 21)))|((y_true==20)&((y_pred == 21)))|((y_true==10)&((y_pred == 11)))|((y_true==13)&((y_pred == 14)))|((y_true==22)&((y_pred == 23)))|((y_true==24)&((y_pred == 14))) )
new_pred = tf.where_v2(mask,y_true,y_pred)
bool_tensor = tf.equal(new_pred,y_pred)
same = tf.math.reduce_all(bool_tensor)
session = tf.keras.backend.get_session(op_input_list=())
K.set_session(same)
if session.run(same)==True:
return sparse_categorical_accuracy(y_true, new_pred)#K.mean(K.equal(K.argmax(y_true, axis=-1), K.argmax(new_pred, axis=-1)))
else:
return tf.zeros([16, 25], tf.int32)
(both tensors have shape [16, 25])
something similar to where_v2 from keras.backend
or at least a method to show if my code works somehow(?) would be great
But in the end:
All i want is to make the conditional confusion for the mentioned conditions possible in my custom accuracy metrics and i would be extremely glad for anyones helpful idea (thank you) :'/

Related

local Variable 'beta' referenced before assignment

My program uses the minimax-algorithm to choose the most otimal move against the human player in tic-tac-toe. (this is working)
After that I tried to implement the alpha-beta-pruning to optimize the time the algorithm needs in order to get the optimal move. (this doesnt work)
I know why the Error is raised, but I dont get why the Error is raised by 'alpha' and 'beta' and not by 'board' or 'minX' or 'minY' too.
I think the Variables I declared should all be global variables, should they?
Error:
UnboundLocalError: local Variable 'beta' referenced before assignment
#Global Variables
values = [0,1,2]
minX = 0
minY = 0
alpha = -999
beta = 999
global alpha
global beta
board = [["-" for x in range(3)] for y in range(3)]
#ausgabe
def ausgabe():
for zeile in range(3):
for spalte in range(3):
print("|"+str(board[zeile][spalte]),end="")
print("|")
#auswertung
#liefert 0, wenn O gewonnen hat
#liefert 2, wenn X gewonnen hat
#liefert 1, wenn es unentschieden ausgeht
#liefert -1 wenn noch nicht klar
def auswertung():
for i in ("X", "O"):
for x in range(3):
if board[x][0] == i and board[x][1] == i and board[x][2] == i:
return i
for y in range(3):
if board[0][y] == i and board[1][y] == i and board[2][y] == i:
return i
if board[0][0] == i and board[1][1] == i and board[2][2] == i:
return i
if board[0][2] == i and board[1][1] == i and board[2][0] == i:
return i
for zeile in range(3):
for spalte in range(3):
if board[zeile][spalte] == "-":
return -1
return 1
#max
def max():
temp = auswertung()
if temp != -1:
if temp == "X":
return 2
elif temp == "O":
return 0
else:
return temp
maximalwert = -999
for x in range(3):
for y in range(3):
if board[x][y] == "-":
board[x][y] = "X"
temp = alpha
alpha = beta
beta = temp
if alpha < beta:
break
wert = min()
board[x][y] = "-"
if wert > maximalwert:
maximalwert = wert
return maximalwert
#min
def min():
temp = auswertung()
if temp != -1:
if temp == "X":
return 2
elif temp == "O":
return 0
else:
return temp
minimalwert = 999
for x in range(3):
for y in range(3):
if board[x][y] == "-":
board[x][y] = "O"
temp = beta
beta = alpha
alpha = temp
if alpha > beta:
break
wert = max()
board[x][y] = "-"
if wert < minimalwert:
minimalwert = wert
return minimalwert
#wo befindet sich das min
def minWo():
temp = auswertung()
if temp != -1:
if temp == "X":
return 2
elif temp == "O":
return 0
else:
return temp
global minX
global minY
minimalwert = 999
for x in range(3):
for y in range(3):
if board[x][y] == "-":
board[x][y] = "O"
temp = beta
beta = alpha
alpha = temp
if alpha > beta:
break
wert = max()
board[x][y] = "-"
if wert < minimalwert:
minX = x
minY = y
minimalwert = wert
return minimalwert
def user_input():
while True:
try:
number1 = int(input("number1: "))
number2 = int(input("number2: "))
if number1 in values and number2 in values:
if board[number1][number2] == "-":
return number1, number2
else:
print("Invalid Input!\ttry again")
except ValueError:
print("Invalid Input!\ttry again")
def game_still_going():
winner = auswertung()
if winner == "X" or winner == "O":
print("Player "+winner+" won the game")
return False
elif winner == 1:
print("It's a draw")
return False
return True
def play_game():
while game_still_going():
number1,number2 = user_input()
board[number1][number2] = "X"
minWo()
board[minX][minY] = "O"
ausgabe()
play_game()
You should set alpha respective beta to -999 and 999 instead of maximalwert, that should work. Look at the pseudo code for minimax.

Pandas .loc[] method is too slow, how can I speed it up

I have a dataframe with 40 million rows,and I want to change some colums by
age = data[data['device_name'] == 12]['age'].apply(lambda x : x if x != -1 else max_age)
data.loc[data['device_name'] == 12,'age'] = age
but this method is too slow, how can I speed it up.
Thanks for all reply!
you might wanna change the first part to :
age = data[data['device_name'] == 12]['age']
age[age == -1] = max_age
data.loc[data['device_name'] == 12,'age'] = age
you could use, to me more concise(this could gain you a little speed)
cond = data['device_name'] == 12
age = data.loc[cond, age]
data.loc[cond,'age'] = age.where(age != -1, max_age)

My .loc with multiple conditions keeps running...help me land the plane

When I try to run the code below, it just keeps running. Is it something obvious?
df.loc[(df['Target_Group'] == 'x') & (df['Period'].dt.year == df['Year_Performed'].dt.year), ['Target_P']] = df.loc[(df['Target_Group'] == 'x') & (df['Period'].dt.year == df['Year_Performed'].dt.year), ['y']]
I think you need assign condition to variable and the reuse:
m = (df['Target_Group'] == 'x') & (df['Period'].dt.year == df['Year_Performed'].dt.year)
df.loc[m, 'Target_P'] = df.loc[m, 'y']
For improve performance is possible use numpy.where:
df['Target_P'] = np.where(m, df['y'], df['Target_P'])
pandas is index sensitive , so you do not need repeat the condition for assignment
cond=(df['Target_Group'] == 'x') & (df['Period'].dt.year == df['Year_Performed'].dt.year)
df.loc[cond, 'Target_P'] = df.y
More info, example
df=pd.DataFrame({'cond':[1,2],'v1':[-110,-11],'v2':[9999,999999]})
df.loc[df.cond==1,'v1']=df.v2
df
Out[200]:
cond v1 v2
0 1 9999 9999
1 2 -11 999999
If index contain duplicate
df.loc[cond, 'Target_P'] = df.loc[cond,'y'].values

No module named 'Models', python 3.5.2

Got this Arkanoid clone code from internet however even after installing pygame binary still doesn' t want to work ("ImportError: No module named 'Models'"). I am using Spyder Python 3.5 and i cloned all repo from there https://github.com/Killavus/PyGame-Arkanoid
import pygame
import sys
import math
import random
from pygame.locals import *
import GameModule
from GameModule.Models import *
from GameModule.Constants import *
class Game:
def playSound( self, fileName ):
sound = pygame.mixer.Sound(fileName)
sound.play()
def __init__(self):
pygame.init()
pygame.mixer.init(11025)
self.actualState = states["START"]
self.display = pygame.display.set_mode( gfx["screen"] )
pygame.display.set_caption( "Arkanoid v" + version )
# 4 basic objects, which define the game
## Painter - drawing actual situation on screen
## Mapper - loading the map
## Status - holding stance of the game: points, lifes, actual powerups etc
## Objects - holding actual game's objects: balls, bricks, pad etc
self.objects = GameModule.Objects()
self.mapper = GameModule.Mapper( self.objects )
self.status = Status()
self.painter = GameModule.Painter( self.display, self.objects, self.status )
self.lastState = None
self.fps = pygame.time.Clock()
self.newLevel()
def loop(self):
self.fps.tick( gfx["framerate"] )
for event in pygame.event.get():
self.handleGlobalEvents( event )
self.handleEvents( event )
self.handleKeyboard()
self.updateState()
if self.actualState != states["GAMEOVER"]:
self.painter.draw()
else:
self.display.fill( (0,0,0) )
bigFont = pygame.font.SysFont( "sans-serif", 32 )
gameOver = bigFont.render( "GAME OVER!", True, (255,255,255) )
smallFont = pygame.font.SysFont( "sans-serif", 18 )
points = smallFont.render( "Punktow: " + str(self.status.points), True, (255,255,255) )
self.display.blit( gameOver, (gfx["screen"][0]/2 - gameOver.get_size()[0]/2, gfx["screen"][1]/2 - gameOver.get_size()[1]/2) )
self.display.blit( points, (gfx["screen"][0]/2 - points.get_size()[0]/2, gfx["screen"][1]/2 - gameOver.get_size()[1]/2 - 40) )
pygame.display.update()
def handleEvents( self, event ):
if self.actualState != states["PAUSE"]:
if event.type == KEYDOWN:
if event.key == K_p:
self.lastState = self.actualState
self.actualState = states["PAUSE"]
self.painter.paused = True
else:
if event.type == KEYDOWN:
if event.key == K_p:
self.actualState = self.lastState
self.painter.paused = False
if self.actualState == states["START"]:
if event.type == KEYDOWN:
if event.key == K_SPACE:
self.actualState = states["PROGRESS"]
(self.objects.balls())[0].speedChange( int( math.copysign( 5., float(self.objects.pad().lastMove) ) ), -5 )
def handleGlobalEvents( self, event ):
if event.type == QUIT:
sys.exit()
def handleKeyboard(self):
pressed = pygame.key.get_pressed()
if self.actualState == states["PROGRESS"] or self.actualState == states["START"]:
if pressed[K_LEFT]:
if self.objects.pad().move(-10) and self.actualState == states["START"]:
(self.objects.balls())[0].move(-10,0)
if pressed[K_RIGHT]:
if self.objects.pad().move(10) and self.actualState == states["START"]:
(self.objects.balls())[0].move(10,0)
def updateState(self):
if self.actualState == states["PAUSE"]:
return
if self.actualState == states["PROGRESS"]:
self.objects.updatePowerups()
for ball in self.objects.balls():
speed = ball.speed()
ball.move( speed[0], speed[1] )
if ball.collideWithWall():
pos = ball.pos()
if pos[0] == 0 or pos[0] >= gfx["screen"][0] - ball.image.get_size()[0] - 1:
ball.xInvert()
if pos[1] == 0:
ball.yInvert()
if pos[1] >= gfx["screen"][1] - ball.image.get_size()[1] - 1:
self.status.lives -= 1
self.actualState = states["START"]
self.playSound( "Resources/snd/chord.wav" )
while len(self.objects.balls()) != 1:
self.objects.removeBall((self.objects.balls())[0])
for powerup in self.objects.powerups():
self.objects.deletePowerup(powerup)
self.objects.balls()[0].reset()
self.objects.pad().reset()
for obj in self.objects.grid():
if ball.collision(obj):
self.status.points += 100
if obj.realx <= ball.position[0] and obj.realx + gfx["grid"][0] >= ball.position[0]:
ball.yInvert()
if obj.realy <= ball.position[1] and obj.realy + gfx["grid"][1] >= ball.position[1]:
ball.xInvert()
self.handleBrickCollision(obj)
if ball.collision(self.objects.pad()):
ball.yInvert()
if self.objects.pad().position[1] <= ball.position[1] and self.objects.pad().position[1] + gfx["grid"][1] >= ball.position[1]:
ball.xInvert()
for powerup in self.objects.powerups():
pad = self.objects.pad()
if powerup.collision(pad):
self.objects.deletePowerup(powerup)
self.generatePowerup()
if powerup.y == gfx["screen"][1] - 1 - powerup.image.get_size()[1]:
self.objects.deletePowerup(powerup)
if self.status.lives == 0:
self.actualState = states["GAMEOVER"]
if self.objects.grid() == []:
self.status.level += 1
if len(levels) == self.status.level:
self.actualStatus = states["GAMEOVER"]
else:
self.newLevel()
for modifier in self.status.modifiers:
modifier[1] -= 1
if modifier[1] == 0:
if modifier[0] == "big_pad":
self.objects.pad().setWidth(3)
del modifier
def generatePowerup(self):
r = random.randint(1,3)
# 3 possible powerups:
# 1 - +1 life
# 2 - for 30 sec the pad is larger
# 3 - additional ball
print (r)
if r == 1:
self.status.lives += 1
elif r == 2:
self.status.modifiers.append( [ "big_pad", gfx["framerate"] * 30 ] )
self.objects.pad().setWidth(5)
elif r == 3:
ball = Ball()
ball.position[0] = self.objects.pad().position[0] + (gfx["grid"][0]*self.objects.pad().gridWidth/2)
ball.speedChange( int( math.copysign( 5., float(self.objects.pad().lastMove) ) ), -5 )
self.objects.addBall(ball)
def handleBrickCollision(self,obj):
brickType = obj.getType()
if brickType == "simple":
self.objects.setGrid( obj.x, obj.y, None )
rand = random.randint( 0, 9 )
if rand == 9: # 1/10 chance for powerup
self.objects.spawnPowerup( obj.x, obj.y )
elif brickType == "solid":
self.objects.setGrid( obj.x, obj.y, SimpleBrick(obj.x,obj.y) )
elif brickType == "ghost":
newType = random.randint( 0, 1 )
if newType == 0:
self.objects.setGrid( obj.x, obj.y, SimpleBrick(obj.x,obj.y) )
elif newType == 1:
self.objects.setGrid( obj.x, obj.y, SolidBrick(obj.x,obj.y) )
self.playSound( "Resources/snd/ding.wav" )
def newLevel(self):
self.playSound( "Resources/snd/tada.wav" )
self.mapper.load( self.status.level )
self.objects.pad().reset()
while len(self.objects.balls()) != 1:
self.objects.removeBall((self.objects.balls())[0])
self.objects.balls()[0].reset()
self.actualState = states["START"]
class Status:
def __init__(self):
self.level = 0
self.points = 0
self.lives = startState["lives"]
self.modifiers = []
game = Game()
while True:
game.loop()

How to write a loop with an "or" condition

What would be the best way to write this statement in Java.
continue loop while either x or y equals true
I have tried these and they fail. I am not sure of the proper way to construct this statement.
while (x || y == true)
while (x | y == true)
while (y == true) || (x == true)
while (y == true) | (x == true)
Thanks in advance.
Alright, so in Java, the best way would be
while(x || y)
If x and y are Booleans. If you are testing conditions,
while(x == 5 || y == 3)
while( x == true || y == true ){
//do loopy stuff
}
This may be different for some languages, but for C based syntax languages, this should work