Related
I'm trying to build an audio super-resolution model. The inputs to the model are audio clips of length 256, so the actual shape of one input sample is (256,1), where 256 is the length of the audio clip and 1 is the number of channels. If we also take in consideration the size of a batch (in my case it's 8), the overall shape of the input should be (8, 256, 1).
My issue is that I don't understand what happens when the input and target data are passed as parameters to the model.fit(...) function. If I specify a batch size, does this mean that model.fit() will create the batches automatically? If so, why is there any incompatibility between my input data and the input shape of the model?
This is the error that I get:
WARNING:tensorflow:Model was constructed with shape (8, 256, 1) for input KerasTensor(type_spec=TensorSpec(shape=(8, 256, 1), dtype=tf.float32, name='input_1'), name='input_1', description="created by layer 'input_1'"), but it was called on an input with incompatible shape (None,).
Traceback (most recent call last):
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/training.py", line 31, in <module>
model.fit(input_data, target_data,
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 1129, in autograph_handler
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1366, in test_function *
return step_function(self, iterator)
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1356, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1349, in run_step **
outputs = model.test_step(data)
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1303, in test_step
y_pred = self(x, training=False)
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/input_spec.py", line 227, in assert_input_compatibility
raise ValueError(f'Input {input_index} of layer "{layer_name}" '
ValueError: Exception encountered when calling layer "model" (type Functional).
Input 0 of layer "conv1d" is incompatible with the layer: expected min_ndim=3, found ndim=1. Full shape received: (None,)
Call arguments received:
• inputs=tf.Tensor(shape=(None,), dtype=string)
• training=False
• mask=None
I do understand that the model needs inputs with 3 dimensions, but why does the error mention that my inputs have 1 dimension when in fact they have 2 dimensions (i.e (256,1)).
Here is the code from the training script:
...
for index in range(0, BATCH_SIZE*NUMBER_OF_BATCHES):
input_data.append(np.load("preprocessed_dataset/low_res/" + input_data_files[index]))
target_data.append(np.load("preprocessed_dataset/high_res/" + target_data_files[index]))
print("Loaded sample {}".format(index))
input_data = np.array(input_data)
target_data = np.array(target_data)
print("Some input tensor shape: {}".format(input_data[0].shape))
print("Some target tensor shape: {}".format(target_data[0].shape))
print("Input data: {}".format(input_data.shape))
print("Target data: {}".format(target_data.shape))
print("Training started...")
model.fit(input_data, target_data,
batch_size=BATCH_SIZE,
epochs=1,
validation_data=(input_validation_files, target_validation_files),
verbose=True)
...and here is the model input shape
...
def create_model(number_of_blocks, batch_size=BATCH_SIZE, input_size=SAMPLE_SIZE):
x = Input((input_size, 1), batch_size=batch_size)
...
Do I actually have to arrange the input samples in batches? Or is that job done by model.fit(...) if I pass a value to the batch_size parameter?
Here is the Github repo, if that helps: https://github.com/Calandrinon/Audio-Super-Resolution/tree/master/Project
I was looking throw the tensorflow contrib API and I wanted to use the RNNClassifier available with Tensorflow 1.13. Contrary to non sequence estimators, this one needs sequence feature columns only. However I was not able to make it work on a toy dataset. I keep getting an error while using sequence_numeric_column.
Here is the structure of my toy dataset:
idSeq,kind,label,size
0,0,dwarf,117.6
0,0,dwarf,134.4
0,0,dwarf,119.0
0,1,human,168.0
0,1,human,145.25
0,2,elve,153.9
0,2,elve,218.49999999999997
0,2,elve,210.9
1,0,dwarf,166.6
1,0,dwarf,168.0
1,0,dwarf,131.6
1,1,human,150.5
1,1,human,208.25
1,1,human,210.0
1,2,elve,199.5
1,2,elve,161.5
1,2,elve,197.6
where idSeq allow us to see which rows belong to which sequence.
I want to predict the "kind" column thanks to the "size" column.
Below there is my code about make my RNN training on my dataset.
import numpy as np
import pandas as pd
import tensorflow as tf
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
tf.logging.set_verbosity(tf.logging.INFO)
dataframe = pd.read_csv("data_rnn.csv")
dataframe_test = pd.read_csv("data_rnn_test.csv")
train_x = dataframe
train_y = dataframe.loc[:,(["kind"])]
size_feature_col = tf.contrib.feature_column.sequence_numeric_column('size ')
estimator = tf.contrib.estimator.RNNClassifier(
sequence_feature_columns=[size_feature_col ],
num_units=[32, 16],
cell_type='lstm',
model_dir=None,
n_classes=3,
optimizer='Adagrad'
)
def make_dataset(
batch_size,
x,
y=None,
shuffle=False,
shuffle_buffer_size=1000,
shuffle_seed=1):
"""
An input function for training, evaluation or prediction.
Parameters
----------------------
batch_size: integer
the size of the batch to use for the training of the neural network
x: pandas dataframe
dataframe that contains the features of the samples to study
y: pandas dataframe or array (Default: None)
dataframe or array that contains the values to predict of the samples
to study. If none, we want a dataset for evaluation or prediction.
shuffle: boolean (Default: False)
if True, we shuffle the elements of the dataset
shuffle_buffer_size: integer (Default: 1000)
if we shuffle the elements of the dataset, it is the size of the buffer
used for it.
shuffle_seed : integer
the random seed for the shuffling
Returns
---------------------
dataset.make_one_shot_iterator().get_next(): Tensor
a nested structure of tf.Tensors containing the next element of the
dataset to study
"""
def input_fn():
if y is not None:
dataset = tf.data.Dataset.from_tensor_slices((dict(x), y))
else:
dataset = tf.data.Dataset.from_tensor_slices(dict(x))
if shuffle:
dataset = dataset.shuffle(
buffer_size=shuffle_buffer_size,
seed=shuffle_seed).batch(batch_size).repeat()
else:
dataset = dataset.batch(batch_size)
return dataset.make_one_shot_iterator().get_next()
return input_fn
batch_size = 50
random_seed = 1
input_fn_train = make_dataset(
batch_size=batch_size,
x=train_x,
y=train_y,
shuffle=True,
shuffle_buffer_size=len(train_x),
shuffle_seed=random_seed)
estimator.train(input_fn=input_fn_train, steps=5000)
But I only got the following error :
INFO:tensorflow:Calling model_fn.
Traceback (most recent call last):
File "main.py", line 125, in <module>
estimator.train(input_fn=input_fn_train, steps=5000)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 358, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 1124, in _train_model
return self._train_model_default(input_fn, hooks, saving_listeners)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 1154, in _train_model_default
features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 1112, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_estimator/contrib/estimator/python/estimator/rnn.py", line 512, in _model_fn
config=config)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_estimator/contrib/estimator/python/estimator/rnn.py", line 332, in _rnn_model_fn
logits, sequence_length_mask = logit_fn(features=features, mode=mode)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_estimator/contrib/estimator/python/estimator/rnn.py", line 226, in rnn_logit_fn
features=features, feature_columns=sequence_feature_columns)
File "/root/.local/lib/python3.5/site-packages/tensorflow/contrib/feature_column/python/feature_column/sequence_feature_column.py", line 120, in sequence_input_layer
trainable=trainable)
File "/root/.local/lib/python3.5/site-packages/tensorflow/contrib/feature_column/python/feature_column/sequence_feature_column.py", line 496, in _get_sequence_dense_tensor
sp_tensor, default_value=self.default_value)
File "/root/.local/lib/python3.5/site-packages/tensorflow/python/ops/sparse_ops.py", line 1432, in sparse_tensor_to_dense
sp_input = _convert_to_sparse_tensor(sp_input)
File "/root/.local/lib/python3.5/site-packages/tensorflow/python/ops/sparse_ops.py", line 68, in _convert_to_sparse_tensor
raise TypeError("Input must be a SparseTensor.")
TypeError: Input must be a SparseTensor.
So I don't understand what I've done wrong because on the documentation, it is written that we have to give a sequence column to the RNNEstimator. They do not say anything about giving sparse tensor.
Thanks in advance for your help and advices.
I am trying to build a CNN + RNN model and I am getting the following error.
Any help will be appreciated.
fc2 has shape (?,4096)
cell = tf.contrib.rnn.BasicLSTMCell(self.rnn_hidden_units)
stack = tf.contrib.rnn.MultiRNNCell([cell]*self.rnn_layers)
initial_state = cell.zero_state(self.batch_size, tf.float32)
initial_state = tf.identity(initial_state, name='initial_state')
outputs, _ = tf.nn.dynamic_rnn(stack, fc2,dtype=tf.float32)
File "rcnn.py", line 182, in model
outputs, _ = tf.nn.dynamic_rnn(stack, [fc2],dtype=tf.float32)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 574, in dynamic_rnn
dtype=dtype)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 637, in _dynamic_rnn_loop
for input_ in flat_input)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 637, in
for input_ in flat_input)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 649, in with_rank_at_least
raise ValueError("Shape %s must have rank at least %d" % (self, rank))
ValueError: Shape (4096, ?) must have rank at least 3
Copying the answer of #jdehesa from his comment for better visibility:
The error seems fairly clear, tf.nn.dynamic_rnn expects a 3-dimensional tensor as input (i.e. rank 3), but fc2 has only two dimensions. The shape of fc2 should be something like (<batch_size>, <max_time>, <num_features>) (or (<max_time>, <batch_size>, <num_features>) if you pass time_major=True)
The default input of tf.nn.dynamic_rnn has a dimension of 3 (Batchsize, sequence_length, num_features). Since your num_features is 1 you can expand your fc_seq with
fc2 = tf.expand_dims(fc2, axis = 2)
and then use the code you have above.
I have the following code:
def dense_layers(pool3):
with tf.variable_scope('local1') as scope:
# Move everything into depth so we can perform a single matrix multiply.
shape_d = pool3.get_shape()
shape = shape_d[1] * shape_d[2] * shape_d[3]
# tf_shape = tf.stack(shape)
tf_shape = 1024
print("shape:", shape, shape_d[1], shape_d[2], shape_d[3])
# So note that tf_shape = 1024, this means that we have 1024 features are fed into the network. And
# the batch size = 1024. Therefore, the aim is to divide the batch_size into num_steps so that
reshape = tf.reshape(pool3, [-1, tf_shape])
# Now we need to reshape/divide the batch_size into num_steps so that we would be feeding a sequence
# And note that most importantly is to have batch_partition_length followed by step_size in the parameter list.
lstm_inputs = tf.reshape(reshape, [batch_partition_length, step_size, tf_shape])
# print('RNN inputs shape: ', lstm_inputs.get_shape()) # -> (128, 8, 1024).
# Note that the state_size is the number of neurons.
lstm = tf.contrib.rnn.BasicLSTMCell(state_size)
lstm_outputs, final_state = tf.nn.dynamic_rnn(cell=lstm, inputs=lstm_inputs, initial_state=init_state)
tf.assign(init_state, final_state)
So, I am taking the output of the pool layer and try to feed it into the LSTM in the network.
Initially I have declared the following:
state_size = 16
step_size = 8
batch_partition_length = int(batch_size / step_size)
init_state = tf.Variable(tf.zeros([batch_partition_length, state_size])) # -> [128, 16].
Therefore, I am getting an error on:
lstm_outputs, final_state = tf.nn.dynamic_rnn(cell=lstm, inputs=lstm_inputs, initial_state=init_state)
As follows:
Traceback (most recent call last):
File "C:/Users/user/PycharmProjects/AffectiveComputing/Brady_with_LSTM.py", line 197, in <module>
predictions = dense_layers(conv_nets_output)
File "C:/Users/user/PycharmProjects/AffectiveComputing/Brady_with_LSTM.py", line 162, in dense_layers
lstm_outputs, final_state = tf.nn.dynamic_rnn(cell=lstm, inputs=lstm_inputs, initial_state=init_state)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\rnn.py", line 553, in dynamic_rnn
dtype=dtype)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\rnn.py", line 720, in _dynamic_rnn_loop
swap_memory=swap_memory)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2623, in while_loop
result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2456, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2406, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\rnn.py", line 705, in _time_step
(output, new_state) = call_cell()
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\rnn.py", line 691, in <lambda>
call_cell = lambda: cell(input_t, state)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\contrib\rnn\python\ops\core_rnn_cell_impl.py", line 238, in __call__
c, h = state
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 504, in __iter__
raise TypeError("'Tensor' object is not iterable.")
TypeError: 'Tensor' object is not iterable.
Any help is much appreciated!!
The state for LSTMs really consists of two parts
State for the cell(s)
Previous outputs
This is alluded to in the docs for BasicLSTMCell. This paper has a good explanation of how LSTMs work which will help you understand why you need to keep two sets of states in an LSTM implementation. The reason an error is being thrown is because you need to supply a tuple of tensors for the initial state.
That said you have two options:
Supply an initial state that consists of two tensors.
Let the RNN cell generate its own initial state.
You would usually only do 1. if you wanted to override default behavior. In this case you are using the default (zero) initial state so you can do 2.
lstm_outputs, final_state = tf.nn.dynamic_rnn(cell=lstm, inputs=lstm_inputs, dtype=tf.float32)
I'm using the Dataset API for input pipelines in TensorFlow (version: r1.2). I built my dataset and batched it with a batch size of 128. The dataset fed into the RNN.
Unfortunately, the dataset.output_shape returns dimension(none) in the first dimension, so the RNN raises an error:
Traceback (most recent call last):
File "untitled1.py", line 188, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/home/harold/anaconda2/envs/tensorflow_py2.7/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "untitled1.py", line 121, in main
run_training()
File "untitled1.py", line 57, in run_training
is_training=True)
File "/home/harold/huawei/ConvLSTM/ConvLSTM.py", line 216, in inference
initial_state=initial_state)
File "/home/harold/anaconda2/envs/tensorflow_py2.7/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 566, in dynamic_rnn
dtype=dtype)
File "/home/harold/anaconda2/envs/tensorflow_py2.7/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 636, in _dynamic_rnn_loop
"Input size (depth of inputs) must be accessible via shape inference,"
ValueError: Input size (depth of inputs) must be accessible via shape inference, but saw value None.
I think this error is caused by the shape of input, the first dimension should be batch size but not none.
here is the code:
origin_dataset = Dataset.BetweenS_Dataset(FLAGS.data_path)
train_dataset = origin_dataset.train_dataset
test_dataset = origin_dataset.test_dataset
shuffle_train_dataset = train_dataset.shuffle(buffer_size=10000)
shuffle_batch_train_dataset = shuffle_train_dataset.batch(128)
batch_test_dataset = test_dataset.batch(FLAGS.batch_size)
iterator = tf.contrib.data.Iterator.from_structure(
shuffle_batch_train_dataset.output_types,
shuffle_batch_train_dataset.output_shapes)
(images, labels) = iterator.get_next()
training_init_op = iterator.make_initializer(shuffle_batch_train_dataset)
test_init_op = iterator.make_initializer(batch_test_dataset)
print(shuffle_batch_train_dataset.output_shapes)
I print output_shapes and it gives:
(TensorShape([Dimension(None), Dimension(36), Dimension(100)]), TensorShape([Dimension(None)]))
I suppose that it should be 128, because I have batched dataset:
(TensorShape([Dimension(128), Dimension(36), Dimension(100)]), TensorShape([Dimension(128)]))
This feature has been added with the drop_remainder parameter used like the following:
batch_test_dataset = test_dataset.batch(FLAGS.batch_size, drop_remainder=True)
From the docs:
drop_remainder: (Optional.) A tf.bool scalar tf.Tensor, representing whether the last batch should be dropped in the case its has fewer than batch_size elements; the default behavior is not to drop the smaller batch.
They hardcoded batch size in implementation and it always will return None (tf 1.3).
def _padded_shape_to_batch_shape(s):
return tensor_shape.vector(None).concatenate(
tensor_util.constant_value_as_shape(s))
In this way, they can batch all elements (e.g. dataset_size=14, batch_size=5, last_batch_size=4).
You can use dataset.filter and dataset.map to fix this issue
d = contrib.data.Dataset.from_tensor_slices([[5] for x in range(14)])
batch_size = 5
d = d.batch(batch_size)
d = d.filter(lambda e: tf.equal(tf.shape(e)[0], batch_size))
def batch_reshape(e):
return tf.reshape(e, [args.batch_size] + [s if s is not None else -1 for s in e.shape[1:].as_list()])
d = d.map(batch_reshape)
r = d.make_one_shot_iterator().get_next()
print('dataset_output_shape = %s' % r.shape)
with tf.Session() as sess:
while True:
print(sess.run(r))
Output
dataset_output_shape = (5, 1)
[[5][5][5][5][5]]
[[5][5][5][5][5]]
OutOfRangeError