I would like to convert the ActorDistributionModel from a trained PPOClipAgent into a Tensorflow Lite model for deployment. How should I accomplish this?
I have tried following this tutorial (see section at bottom converting policy to TFLite), but the network outputs a single action (the policy) rather than the density function over actions that I desire.
I think perhaps something like this could work:
tf.compat.v2.saved_model.save(actor_net, saved_model_path, signature=?)
... if I knew how to set the signature parameter. That line of code executes without error when I omit the signature parameter, but I get the following error on load (I assume because the signature is not set up correctly):
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_path)
File "/home/ais/salesmentor.ai/MDPSolver/src/solver/ppo_budget.py", line 336, in train_eval
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_path)
File "/home/ais/.local/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 1275, in from_saved_model
raise ValueError("Only support a single signature key.")
ValueError: Only support a single signature key.
This appears to work. I won't accept the answer until I have completed an end-to-end test, though.
def export_model(actor_net, observation_spec, saved_model_path):
predict_signature = {
'action_pred':
tf.function(func=lambda x: actor_net(x, None, None)[0].logits,
input_signature=(tf.TensorSpec(shape=observation_spec.shape),)
)
}
tf.saved_model.save(actor_net, saved_model_path, signatures=predict_signature)
# Convert to TensorFlow Lite model.
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_path,
signature_keys=["action_pred"])
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]
tflite_policy = converter.convert()
with open(os.path.join(saved_model_path, 'policy.tflite'), 'wb') as f:
f.write(tflite_policy)
The solution wraps the actor_net in a lambda because I was unable to figure out how to specify the signature with all three expected arguments. Through the lambda, I convert the function into using a single argument (a tensor). I expect to pass None to the other two arguments in my use case, so there is nothing lost in this approach.
I see you using CartPole as the model simulation, Agent DQN, and Model learning and Evaluation from links provided TF-Agent Checkpointer. For simple understanding, you need to understand about the distributions and your model limits ( less than 6 actions determining at a time ).
Discretes Distribution, answer the question to the points but the links is how they implement AgentDQN on TF- Agent.
temp = tf.random.normal([10], 1, 0.2, tf.float32), mean is one and the standard deviation is 0.2. Overall of result summation product is nearby one and its variance is 0.2, when they have 10 actions to determine the possibility of the result is the same action is 1 from 5 or 0.5. random normal
Coefficient is ladder steps or you understand as IF and ELSE conditions or SWITCH conditions such as at the gap of 0 to 5, 5 to 10, 10 to 15, and continue.
The matrixes product from the Matrix coefficients and randoms is selected 4 - 5 actions sorted by priority, significant and select the most effects in rows.
The ArgMax is 0 to 9 which is actions 0 - 9 that respond to the environment input co-variances.
Sample: To the points, random distributions and selective agents ( we call selective agent maybe the questioner has confused with NN DQN )
temp = tf.random.normal([10], 1, 0.2, tf.float32)
temp = np.asarray(temp) * np.asarray([ coefficient_0, coefficient_1, coefficient_2, coefficient_3, coefficient_4, coefficient_5, coefficient_6, coefficient_7, coefficient_8, coefficient_9 ])
temp = tf.nn.softmax(temp)
action = int(np.argmax(temp))
I am trying to compute features and define feature matrix on a dataset.
The code is as follows:
def compute_features(features, cutoff_time):
np.random.shuffle(features)
feature_matrix = ft.calculate_feature_matrix(features,
cutoff_time=cutoff_time,
approximate='36d',
verbose=True,entities=entities, relationships=relationships)
print("Finishing computing...")
feature_matrix, features = ft.encode_features(feature_matrix, features,
to_encode=["pickup_neighborhood", "dropoff_neighborhood"],
include_unknown=False)
return feature_matrix
Up until this point, The code is working fine. However, the next line after
feature_matrix1 = compute_features(features, cutoff_time)
I am getting the following error:
KeyError: "['time'] not in index"
I am sorting a data frame and then inserting it into the statsmodel OLS function. Before I pass my X and Y into the function X looks like this:
and Y looks like this:
I then pass it to the mentioned function in the following way:
model = sm.OLS(Y,X).fit()
After running this I get the following error:
ValueError: zero-size array to reduction operation maximum which has no identity
In addition, the X and Y matrices are empty when I check them. My questions are: Why am getting this error and why are my matrices being altered?
I am trying to set a simple KNN problem implementation with a three class dataset but whenever I try to execute the train function I keep the said (-215:Assertion failed) new_samples.type() == CV_32F in function 'cv::ml::Impl::train error.
I have tried reshaping the responses array into many different things since most of the errors came from that part of the code, that goes from 1 x n matrix to a single list. I am following this tutorial. I can get it done with two classes by defining my own data just like I do with three classes but I can't manage to train with three classes.
import numpy as np
import cv2 as cv
classA=([(10,1,1),(9,2,2),(11,1,2),(8,3,2),(7,2,3),(8,5,4),(9,3,4),(6,6,5),(8,6,6),(9,7,7)])
classB=([(5,1,20),(5,2,19),(5,1,21),(4,2,18),(4,1,19),(6,3,20),(6,2,19),(4,4,18),(4,5,21),(6,4,19)])
classC=([(5,14,10),(6,13,9),(4,12,11),(6,11,9),(6,7,12),(7,6,13),(7,7,10), (7,8,11),(8,8,12),(7,6,11)])
points = classA + classB + classC
responses = ([0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2])
# Using numpy's array?
points_np = np.asarray(points)
responses_np = np.asarray(responses).reshape((30,1))
#print(points_np)
#print(responses_np)
knn = cv.ml.KNearest_create()
knn.train(points_np, cv.ml.ROW_SAMPLE, responses_np)
I know both sample and response data should follow a similar structure so the function can associate each point to a class but I think my issue is on the type of structure I am using for the responses variable. How should I shape or set the responses variable in order to be readable for the train function?
As indicated in the assertion, the data type for samples must be CV_32F, which stands for 32 bit float.
points_np = np.asarray(points).astype(np.float32)
responses_np = np.asarray(responses).reshape((30,1)).astype(np.float32)
I'm trying to make a grid in ggplot to plot 4 graphs, as if it were a basic pair (mfrow = c (2,2)). However, I can not execute the code. I have already tried with gridExtra and cowplot with the functions plot_grid, grid.arrange, ggplot2.multiplot and also tried with the multiplot function. The error that appears is as follows:
Error: Aesthetics must be either length 1 or the same as the data (8598): alpha, x, y, group
gridExtra::grid.arrange(ggplot(),ggplot(),ggplot(),ggplot(), nrow=2)
produces
you may want to debug your code for each individual plot first.