I am trying to use the scikit-learn in Keras to fine tune the model that has one input(images) and 2 outputs(rotational vector and translation vector). The code snippet is as below,
img_input =Input(shape=(img_rows, img_cols, img_channels))
model = KerasRegressor(build_fn = toy_model, verbose = 1)
loss_weights = [[1.0, 250.0], [1.0, 500.0], [1.0, 750.0]]
epochs =[10, 20]
batches = [5, 10]
param_grid = dict(loss_weight= loss_weights, epochs = epochs,
batch_size = batches)
grid = GridSearchCV(estimator = model, param_grid=param_grid)
grid_result = grid.fit(train_imgs, [train_pose_tx, train_pose_rt])
I want to fine tune the "loss_weights" parameter for this model. However, I get the following error
ValueError: Found input variables with inconsistent numbers of samples:[895, 2]
As I understand since this model has single input, this functionality must be supported.
Link to Github gist :
https://gist.github.com/sushant4788/1f84cd2781f96fb752ee1f16a56d1bcb
Related
I am a high school student trying to learn the basics of TensorFlow. I am currently building a model with TFRecords input files, the default dataset file type from TensorFlow, that have been compressed from the original raw data. I am currently using a convoluted way of parsing the data into numpy arrays for Keras to interpret it. While Keras is a part of TF, it should be easily able to read TFRecord datasets. Is there any other way for Keras to understand TFRecord files?
I use the _decodeExampleHelper method to prepare the data for training.
def _decodeExampleHelper(example) :
dataDictionary = {
'xValues' : tf.io.FixedLenFeature([7], tf.float32),
'yValues' : tf.io.FixedLenFeature([3], tf.float32)
}
# Parse the input tf.Example proto using the data dictionary
example = tf.io.parse_single_example(example, dataDictionary)
xValues = example['xValues']
yValues = example['yValues']
# The Keras Sequential network will have "dense" as the name of the first layer; dense_input is the input to this layer
return dict(zip(['dense_input'], [xValues])), yValues
data = tf.data.TFRecordDataset(workingDirectory + 'training.tfrecords')
parsedData = data.map(_decodeExampleHelper)
We can see that the parsedData has the correct dimensions in the following code block.
tmp = next(iter(parsedData))
print(tmp)
This outputs the first set of data in the correct dimensions that Keras should be able to interpret.
({'dense_input': <tf.Tensor: id=273, shape=(7,), dtype=float32, numpy=
array([-0.6065675 , -0.610906 , -0.65771157, -0.41417238, 0.89691925,
0.7122903 , 0.27881026], dtype=float32)>}, <tf.Tensor: id=274, shape=(3,), dtype=float32, numpy=array([ 0. , -0.65868723, -0.27960175], dtype=float32)>)
Here is a very simple model with only two layers and train it with the data I just parsed.
model = tf.keras.models.Sequential(
[
tf.keras.layers.Dense(20, activation = 'relu', input_shape = (7,)),
tf.keras.layers.Dense(3, activation = 'linear'),
]
)
model.compile(optimizer = 'adam', loss = 'mean_absolute_error', metrics = ['accuracy'])
model.fit(parsedData, epochs = 1)
The line model.fit(parsedData, epochs = 1) gives an error of ValueError: Error when checking input: expected dense_input to have shape (7,) but got array with shape (1,) despite the dense_input being 7.
What problem could there be in this case? Why can Keras no interpret tensors from the file correctly?
You need to be batching your data before passing it to Keras and using an Input layer. The following works for me just fine:
import tensorflow as tf
ds = tf.data.Dataset.from_tensors((
{'dense_input': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7]}, [ 0.0, 0.1, -0.1]))
ds = ds.repeat(32).batch(32)
model = tf.keras.models.Sequential(
[
tf.keras.Input(shape=(7,), name='dense_input'),
tf.keras.layers.Dense(20, activation = 'relu'),
tf.keras.layers.Dense(3, activation = 'linear'),
]
)
model.compile(optimizer = 'adam', loss = 'mean_absolute_error', metrics = ['accuracy'])
model.fit(ds, epochs = 1)
I created a char-level language generation with Tensorflow here. I used tf.placeholder API, which according to the google docs:
Feeding is least efficient way to feed data into a TensorFlow program.
I decided to change my code and replace it with new TensroFlow Dataset API.
I used from_generator to create Dataset:
dataset = tf.data.Dataset.from_generator(gen, (tf.int32, tf.int32),
(tf.TensorShape([None, None]),
tf.TensorShape([None, None])))
self.iterator = dataset.make_initializable_iterator()
self.inp, self.target = self.iterator.get_next()
As can be seen in above code, I used [None, None] for Tensorshape to give the model more generality. During the training everything is perfectly fine. But at inference some problem arise. In tf.placeholder API I used following code to generate characters:
def inference(self):
converter = utils.TextReader(filename=FLAGS.CONVERTER_PATH)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
samples = []
new_state = sess.run(self.init_state)
c = 12 # random starting token
samples.append(c)
for i in range(1000):
x = np.zeros((1, 1))
x[0, 0] = c
feed_dict = {
self.inp: x,
self.init_state: new_state
}
preds, new_state = sess.run([self.prediction, self.final_state], feed_dict=feed_dict)
c = utils.pick_top_n(preds, converter.vocab_size)
samples.append(c)
samples = np.array(samples)
print(converter.arr_to_text(samples))
In Dataset API, I dont have tf.placeholder to feed my previous character. And when I use the above code, as expected, following error happened:
InvalidArgumentError (see above for traceback): ConcatOp : Dimensions of inputs should match: shape[0] = [1,50] vs. shape[1] = [32,50]
At inference, the model use the same input shape ([32,50]) that I used for training. Which is not what I want (Actually, I define TensorShape([None,None]) to handle this but not works).
How can I fix the issue with new Dataset API?
Complete code.
I am currently trying to get a simple tensorflow model to train by data provided by a custom input pipeline. It should work as efficient as possible. Although I've read lots of tutorials, I can't get it to work.
THE DATA
I have my training data split over several csv files. File 'a.csv' has 20 samples and 'b.csv' has 30 samples in it, respectively. They have the same structure with the same header:
feature1; feature2; feature3; feature4
0.1; 0.2; 0.3; 0.4
...
(No labels, as it is for an autoencoder.)
THE CODE
I have written an input pipeline and would like to feed the data from it to the model. My code looks like this:
import tensorflow as tf
def input_pipeline(filenames, batch_size):
dataset = tf.data.Dataset.from_tensor_slices(filenames)
dataset = dataset.flat_map(
lambda filename: (
tf.data.TextLineDataset(filename)
.skip(1)
.shuffle(10)
.map(lambda csv_row: tf.decode_csv(
csv_row,
record_defaults=[[-1.0]]*4,
field_delim=';'))
.batch(batch_size)
)
)
return dataset.make_initializable_iterator()
iterator = input_pipeline(['/home/sku/data/a.csv',
'/home/sku/data/b.csv'],
batch_size=5)
next_element = iterator.get_next()
# Build the autoencoder
x = tf.placeholder(tf.float32, shape=[None, 4], name='in')
z = tf.contrib.layers.fully_connected(x, 2, activation_fn=tf.nn.relu)
x_hat = tf.contrib.layers.fully_connected(z, 4)
# loss function with epsilon for numeric stability
epsilon = 1e-10
loss = -tf.reduce_sum(
x * tf.log(epsilon + x_hat) + (1 - x) * tf.log(epsilon + 1 - x_hat))
train_op = tf.train.AdamOptimizer(learning_rate=1e-3).minimize(loss)
with tf.Session() as sess:
sess.run(iterator.initializer)
sess.run(tf.global_variables_initializer())
for i in range(50):
batch = sess.run(next_element)
sess.run(train_op, feed_dict={x: batch, x_hat: batch})
THE PROBLEM
When trying to feed the data to the model, I get an error:
ValueError: Cannot feed value of shape (4, 5) for Tensor 'in:0', which has shape '(?, 4)'
When printing out the shapes of the batched data, I get this for example:
(array([ 4.1, 5.9, 5.5, 6.7, 10. ], dtype=float32), array([0.4, 7.7, 0. , 3.4, 8.7], dtype=float32), array([3.5, 4.9, 8.3, 7.2, 6.4], dtype=float32), array([-1. , -1. , 9.6, -1. , -1. ], dtype=float32))
It makes sense, but where and how do I have to reshape this? Also, this additional info dtype only appears with batching.
I also considered that I did the feeding wrong. Do I need input_fn or something like that? I remember that feeding dicts is way to slow. If somebody could give me an efficient way to prepare and feed the data, I would be really grateful.
Regards,
I've figured out a solution, that requires a second mapping function. You have to add the following line to the input function:
def input_pipeline(filenames, batch_size):
dataset = tf.data.Dataset.from_tensor_slices(filenames)
dataset = dataset.flat_map(
lambda filename: (
tf.data.TextLineDataset(filename)
.skip(1)
.shuffle(10)
.map(lambda csv_row: tf.decode_csv(
csv_row,
record_defaults=[[-1.0]]*4,
field_delim=';'))
.map(lambda *inputs: tf.stack(inputs)) # <-- mapping required
.batch(batch_size)
)
)
return dataset.make_initializable_iterator()
This seems to convert the array-like output to a matrix, that can be fed to the network.
However, I'm still not sure if feeding it via feed_dict is the most efficient way. I'd still appreciate support here!
I have a basic classification code for Irish dataset.
import tensorflow as tf
import pandas as pd
COLUMN_NAMES = [
'SepalLength',
'SepalWidth',
'PetalLength',
'PetalWidth',
'Species'
]
# Import training dataset
training_dataset = pd.read_csv('iris_training.csv', names=COLUMN_NAMES, header=0)
train_x = training_dataset.iloc[:, 0:4]
train_y = training_dataset.iloc[:, 4]
# Import testing dataset
test_dataset = pd.read_csv('iris_test.csv', names=COLUMN_NAMES, header=0)
test_x = test_dataset.iloc[:, 0:4]
test_y = test_dataset.iloc[:, 4]
columns_feat = [
tf.feature_column.numeric_column(key='SepalLength'),
tf.feature_column.numeric_column(key='SepalWidth'),
tf.feature_column.numeric_column(key='PetalLength'),
tf.feature_column.numeric_column(key='PetalWidth')
]
classifier = tf.estimator.DNNClassifier(
feature_columns=columns_feat,
# Two hidden layers of 10 nodes each.
hidden_units=[10, 10],
# The model is classifying 3 classes
n_classes=3)
def train_function(inputs, outputs, batch_size):
dataset = tf.data.Dataset.from_tensor_slices((dict(inputs), outputs))
dataset = dataset.shuffle(1000).repeat().batch(batch_size)
return dataset.make_one_shot_iterator().get_next()
# Train the Model.
classifier.train(
input_fn=lambda:train_function(train_x, train_y, 100),
steps=1000)
def evaluation_function(attributes, classes, batch_size):
attributes=dict(attributes)
if classes is None:
inputs = attributes
else:
inputs = (attributes, classes)
dataset = tf.data.Dataset.from_tensor_slices(inputs)
assert batch_size is not None, "batch_size must not be None"
dataset = dataset.batch(batch_size)
return dataset.make_one_shot_iterator().get_next()
# Evaluate the model.
eval_result = classifier.evaluate(
input_fn=lambda:evaluation_function(test_x, test_y, 100))
I evaluate the result but how can i make a prediction on my data because now i get only console info of loss and epochs, accuracy. For example if i have everything except species. I want to give my own sepal length and etc so i can get prediction of the species and it will be another variable. Do i have to create variables like pred_x or pred_y(pandas dataframe) and then put them into eval_result?
Is that what you mean? for example:new_samples = np.array([[6.4, 3.2, 4.5, 1.5], [5.8, 3.1, 5.0, 1.7]], dtype=np.float32) If you want new data like this to make predictions, then you can refer to this code.TensorFlow-Iris-Classification
Like all estimator classes, the DNNClassifier class has a predict method that makes real-world predictions. The documentation is here.
Trying to implement a minimal toy RNN example in tensorflow.
The goal is to learn a mapping from the input data to the target data, similar to this wonderful concise example in theanets.
Update: We're getting there. The only part remaining is to make it converge (and less convoluted). Could someone help to turn the following into running code or provide a simple example?
import tensorflow as tf
from tensorflow.python.ops import rnn_cell
init_scale = 0.1
num_steps = 7
num_units = 7
input_data = [1, 2, 3, 4, 5, 6, 7]
target = [2, 3, 4, 5, 6, 7, 7]
#target = [1,1,1,1,1,1,1] #converges, but not what we want
batch_size = 1
with tf.Graph().as_default(), tf.Session() as session:
# Placeholder for the inputs and target of the net
# inputs = tf.placeholder(tf.int32, [batch_size, num_steps])
input1 = tf.placeholder(tf.float32, [batch_size, 1])
inputs = [input1 for _ in range(num_steps)]
outputs = tf.placeholder(tf.float32, [batch_size, num_steps])
gru = rnn_cell.GRUCell(num_units)
initial_state = state = tf.zeros([batch_size, num_units])
loss = tf.constant(0.0)
# setup model: unroll
for time_step in range(num_steps):
if time_step > 0: tf.get_variable_scope().reuse_variables()
step_ = inputs[time_step]
output, state = gru(step_, state)
loss += tf.reduce_sum(abs(output - target)) # all norms work equally well? NO!
final_state = state
optimizer = tf.train.AdamOptimizer(0.1) # CONVERGEs sooo much better
train = optimizer.minimize(loss) # let the optimizer train
numpy_state = initial_state.eval()
session.run(tf.initialize_all_variables())
for epoch in range(10): # now
for i in range(7): # feed fake 2D matrix of 1 byte at a time ;)
feed_dict = {initial_state: numpy_state, input1: [[input_data[i]]]} # no
numpy_state, current_loss,_ = session.run([final_state, loss,train], feed_dict=feed_dict)
print(current_loss) # hopefully going down, always stuck at 189, why!?
I think there are a few problems with your code, but the idea is right.
The main issue is that you're using a single tensor for inputs and outputs, as in:
inputs = tf.placeholder(tf.int32, [batch_size, num_steps]).
In TensorFlow the RNN functions take a list of tensors (because num_steps can vary in some models). So you should construct inputs like this:
inputs = [tf.placeholder(tf.int32, [batch_size, 1]) for _ in xrange(num_steps)]
Then you need to take care of the fact that your inputs are int32s, but a RNN cell works on float vectors - that's what embedding_lookup is for.
And finally you'll need to adapt your feed to put in the input list.
I think the ptb tutorial is a reasonable place to look, but if you want an even more minimal example of an out-of-the-box RNN you can take a look at some of the rnn unit tests, e.g., here.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/kernel_tests/rnn_test.py#L164