Related
I'm trying to build an audio super-resolution model. The inputs to the model are audio clips of length 256, so the actual shape of one input sample is (256,1), where 256 is the length of the audio clip and 1 is the number of channels. If we also take in consideration the size of a batch (in my case it's 8), the overall shape of the input should be (8, 256, 1).
My issue is that I don't understand what happens when the input and target data are passed as parameters to the model.fit(...) function. If I specify a batch size, does this mean that model.fit() will create the batches automatically? If so, why is there any incompatibility between my input data and the input shape of the model?
This is the error that I get:
WARNING:tensorflow:Model was constructed with shape (8, 256, 1) for input KerasTensor(type_spec=TensorSpec(shape=(8, 256, 1), dtype=tf.float32, name='input_1'), name='input_1', description="created by layer 'input_1'"), but it was called on an input with incompatible shape (None,).
Traceback (most recent call last):
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/training.py", line 31, in <module>
model.fit(input_data, target_data,
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 1129, in autograph_handler
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1366, in test_function *
return step_function(self, iterator)
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1356, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1349, in run_step **
outputs = model.test_step(data)
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1303, in test_step
y_pred = self(x, training=False)
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/input_spec.py", line 227, in assert_input_compatibility
raise ValueError(f'Input {input_index} of layer "{layer_name}" '
ValueError: Exception encountered when calling layer "model" (type Functional).
Input 0 of layer "conv1d" is incompatible with the layer: expected min_ndim=3, found ndim=1. Full shape received: (None,)
Call arguments received:
• inputs=tf.Tensor(shape=(None,), dtype=string)
• training=False
• mask=None
I do understand that the model needs inputs with 3 dimensions, but why does the error mention that my inputs have 1 dimension when in fact they have 2 dimensions (i.e (256,1)).
Here is the code from the training script:
...
for index in range(0, BATCH_SIZE*NUMBER_OF_BATCHES):
input_data.append(np.load("preprocessed_dataset/low_res/" + input_data_files[index]))
target_data.append(np.load("preprocessed_dataset/high_res/" + target_data_files[index]))
print("Loaded sample {}".format(index))
input_data = np.array(input_data)
target_data = np.array(target_data)
print("Some input tensor shape: {}".format(input_data[0].shape))
print("Some target tensor shape: {}".format(target_data[0].shape))
print("Input data: {}".format(input_data.shape))
print("Target data: {}".format(target_data.shape))
print("Training started...")
model.fit(input_data, target_data,
batch_size=BATCH_SIZE,
epochs=1,
validation_data=(input_validation_files, target_validation_files),
verbose=True)
...and here is the model input shape
...
def create_model(number_of_blocks, batch_size=BATCH_SIZE, input_size=SAMPLE_SIZE):
x = Input((input_size, 1), batch_size=batch_size)
...
Do I actually have to arrange the input samples in batches? Or is that job done by model.fit(...) if I pass a value to the batch_size parameter?
Here is the Github repo, if that helps: https://github.com/Calandrinon/Audio-Super-Resolution/tree/master/Project
I am using finetune AlexNet architecture written by #kratzert on my own dataset which, works properly (I got the code from here: https://github.com/kratzert/finetune_alexnet_with_tensorflow) and I want to figure out how to build confusion matrix from his code. I have tried to use tf.confusion_matrix(labels, predictions, num_classes) to build confusion matrix but I can't. I am confused what should be the values for labels and predictions, I mean, I know what should be but each time I feed these value got an error. Can anyone help me on this or have a look at the code (above link) and guide me?
I added these two lines in finetune.py exactly after calculating accuracy to make the labels and the predictions as the number of the class.
with tf.name_scope("accuracy"):
correct_pred = tf.equal(tf.argmax(score, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
**true_class = tf.argmax(y, 1)
predicted_class = tf.argmax(score, 1)**
and I have added tf.confusion_matrix() inside my session at the very bottom before saving checkpoint of the model
for _ in range(val_batches_per_epoch):
img_batch, label_batch = sess.run(next_batch)
acc, cost = sess.run([accuracy, loss], feed_dict={x: img_batch,
y: label_batch,
keep_prob: 1.})
test_acc += acc
test_count += 1
test_acc /= test_count
print("{} Validation Accuracy = {:.4f} -- Validation Loss = {:.4f}".format(datetime.now(),test_acc, cost))
print("{} Saving checkpoint of model...".format(datetime.now()))
**print(sess.run(tf.confusion_matrix(true_class, predicted_class, num_classes)))**
# save checkpoint of the model
checkpoint_name = os.path.join(checkpoint_path,
'model_epoch'+str(epoch+1)+'.ckpt')
save_path = saver.save(sess, checkpoint_name)
print("{} Model checkpoint saved at {}".format(datetime.now(),
checkpoint_name))
I have tried other places as well but each time I will get an error:
Caused by op 'Placeholder_1', defined at:
File "/home/armin/Desktop/Alexnet_DataPipeline/finetune.py", line 85, in <module>
y = tf.placeholder(tf.float32, [batch_size, num_classes])
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py", line 1777, in placeholder
return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 4521, in placeholder
"Placeholder", dtype=dtype, shape=shape, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3290, in create_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1654, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder_1' with dtype float and shape [128,3]
any help will be appreciated, Thanks.
It's a fairly long piece of code you're referring to, and you did not specify where you put your confusion matrix line.
Just by experience, the most frequent problem with confusion matrices is that tf.confusion_matrix() requires both the labels and the predictions as the number of the class, not as one-hot vectors. In other words, the label and the prediction should be in the form of the number 5 instead of [ 0, 0, 0, 0, 0, 1, 0, 0, 0, 0 ].
In the code you refer to, y is in the one-hot format. The output of the network, score is a vector, giving the probability of each class. That is also not the required format. You need to do something like
true_class = tf.argmax( y, 1 )
predicted_class = tf.argmax( score, 1 )
and use those with the confusion matrix like
tf.confusion_matrix( true_class, predicted_class, num_classes )
(Basically, if you take a look at line 123 of finetune.py, that has both of those elements for determining accuracy, but they are not saved in separate tensors.)
If you want to keep a running total of confusion matrices of all batches, you just have to add them up - since each cell of the matrix counts the number of examples falling into that category, an element-wise addition creates the confusion matrix for the whole set:
cm_running_total = None
cm_nupmy_array = sess.run(tf.confusion_matrix(true_class, predicted_class, num_classes), feed_dict={x: img_batch, y: label_batch, keep_prob: 1.} )
if cm_running_total is None:
cm_running_total = cm_numpy_array
else:
cm_running_total += cm_numpy_array
I am trying to build a CNN+LSTM+CTC model for word recognition.
Initially I have an image, I am converting the features extracted using CNN on word image and building a sequence of features which I am passing as sequential data to RNN.
Follwing is how I am converting the features to sequential data:
[[a1,b1,c1],[a2,b2,c2],[a3,b3,c3]] -> [[a1,a2,a3],[b1,b2,b3],[c1,c2,c3]]
Where a,b,c are 3 features extracted using CNN.
Presently I can pass a constant batch_size to the model common.BATCH_SIZE, but what I want is to be able to pass variable batch_size to the model.
How can this be done ?
inputs = tf.placeholder(tf.float32, [common.BATCH_SIZE, common.OUTPUT_SHAPE[1], common.OUTPUT_SHAPE[0], 1])
# Here we use sparse_placeholder that will generate a
# SparseTensor required by ctc_loss op.
targets = tf.sparse_placeholder(tf.int32)
# 1d array of size [batch_size]
seq_len = tf.placeholder(tf.int32, [common.BATCH_SIZE])
model = tf.layers.conv2d(inputs, 64, (3,3),strides=(1, 1), padding='same', name='c1')
model = tf.layers.max_pooling2d(model, (3,3), strides=(2,2), padding='same', name='m1')
model = tf.layers.conv2d(model, 128,(3,3), strides=(1, 1), padding='same', name='c2')
model = tf.layers.max_pooling2d(model, (3,3),strides=(2,2), padding='same', name='m2')
model = tf.transpose(model, [3,0,1,2])
shape = model.get_shape().as_list()
model = tf.reshape(model, [shape[0],-1,shape[2]*shape[3]])
cell = tf.nn.rnn_cell.LSTMCell(common.num_hidden, state_is_tuple=True)
cell = tf.nn.rnn_cell.DropoutWrapper(cell, input_keep_prob=0.5, output_keep_prob=0.5)
stack = tf.nn.rnn_cell.MultiRNNCell([cell]*common.num_layers, state_is_tuple=True)
outputs, _ = tf.nn.dynamic_rnn(cell, model, seq_len, dtype=tf.float32,time_major=True)
UPDATE:
batch_size = tf.placeholder(tf.int32, None, name='batch_size')
inputs = tf.placeholder(tf.float32, [batch_size, common.OUTPUT_SHAPE[1], common.OUTPUT_SHAPE[0], 1])
# Here we use sparse_placeholder that will generate a
# SparseTensor required by ctc_loss op.
targets = tf.sparse_placeholder(tf.int32)
# 1d array of size [batch_size]
seq_len = tf.placeholder(tf.int32, [batch_size])
model = tf.layers.conv2d(inputs, 64, (3,3),strides=(1, 1), padding='same', name='c1')
model = tf.layers.max_pooling2d(model, (3,3), strides=(2,2), padding='same', name='m1')
model = tf.layers.conv2d(model, 128,(3,3), strides=(1, 1), padding='same', name='c2')
model = tf.layers.max_pooling2d(model, (3,3),strides=(2,2), padding='same', name='m2')
model = tf.transpose(model, [3,0,1,2])
shape = model.get_shape().as_list()
model = tf.reshape(model, [shape[0],-1,shape[2]*shape[3]])
cell = tf.nn.rnn_cell.LSTMCell(common.num_hidden, state_is_tuple=True)
cell = tf.nn.rnn_cell.DropoutWrapper(cell, input_keep_prob=0.5, output_keep_prob=0.5)
stack = tf.nn.rnn_cell.MultiRNNCell([cell]*common.num_layers, state_is_tuple=True)
outputs, _ = tf.nn.dynamic_rnn(cell, model, seq_len, dtype=tf.float32,time_major=True)
I am getting error as below:
Traceback (most recent call last):
File "lstm_and_ctc_ocr_train.py", line 203, in <module>
train()
File "lstm_and_ctc_ocr_train.py", line 77, in train
logits, inputs, targets, seq_len, batch_size = model.get_train_model()
File "/home/himanshu/learning-tf/tf/code/tensorflow_lstm_ctc_ocr/model.py", line 20, in get_train_model
inputs = tf.placeholder(tf.float32, [batch_size, common.OUTPUT_SHAPE[1], common.OUTPUT_SHAPE[0], 1])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1530, in placeholder
return gen_array_ops._placeholder(dtype=dtype, shape=shape, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 1954, in _placeholder
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 705, in apply_op
attr_value.shape.CopyFrom(_MakeShape(value, key))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 198, in _MakeShape
return tensor_shape.as_shape(v).as_proto()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 798, in as_shape
return TensorShape(shape)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 434, in __init__
self._dims = [as_dimension(d) for d in dims_iter]
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 376, in as_dimension
return Dimension(value)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 32, in __init__
self._value = int(value)
TypeError: int() argument must be a string or a number, not 'Tensor'
You should be able to pass batch_size to dynamic RNN as a placeholder. In my experience, the only headache you might experience is if you don't specify its shape in advance, so you should pass [] to make things work, something like this:
batchsize = tf.placeholder(tf.int32, [], name='batchsize')
and then feed its value during sess.run() in the usual way. This works well for me while training on a large batch size, but then generating with batch of 1.
But strictly speaking, you don't even need to specify batch size for dynamic_rnn specifically, do you? You need it if you use MultiRNNCell to get zero state, but I don't see you doing that in your code...
*** UPDATE:
As discussed in the comment, your problem seems to have nothing to do with dynamic_rnn and more to do with the fact that you are using a placeholder inputs to specify the shape of another placeholder seq_len. Here's the code that reproduces the same error:
import tensorflow as tf
a = tf.placeholder(tf.int32, None, name='a')
b = tf.placeholder(tf.int32, [a, 5], name='b')
c = b * 5
with tf.Session() as sess:
C = sess.run(c, feed_dict={a:1, b:[[1,2,3,4,5]]})
And here's the error:
TypeError: int() argument must be a string, a bytes-like object or a number, not 'Tensor'
Before getting to the headache of dynamic_rnn, I would recommend finding a way around this, either by changing your code or by asking a separate question on how to fudge it with placeholders.
I have the following code:
def dense_layers(pool3):
with tf.variable_scope('local1') as scope:
# Move everything into depth so we can perform a single matrix multiply.
shape_d = pool3.get_shape()
shape = shape_d[1] * shape_d[2] * shape_d[3]
# tf_shape = tf.stack(shape)
tf_shape = 1024
print("shape:", shape, shape_d[1], shape_d[2], shape_d[3])
# So note that tf_shape = 1024, this means that we have 1024 features are fed into the network. And
# the batch size = 1024. Therefore, the aim is to divide the batch_size into num_steps so that
reshape = tf.reshape(pool3, [-1, tf_shape])
# Now we need to reshape/divide the batch_size into num_steps so that we would be feeding a sequence
# And note that most importantly is to have batch_partition_length followed by step_size in the parameter list.
lstm_inputs = tf.reshape(reshape, [batch_partition_length, step_size, tf_shape])
# print('RNN inputs shape: ', lstm_inputs.get_shape()) # -> (128, 8, 1024).
# Note that the state_size is the number of neurons.
lstm = tf.contrib.rnn.BasicLSTMCell(state_size)
lstm_outputs, final_state = tf.nn.dynamic_rnn(cell=lstm, inputs=lstm_inputs, initial_state=init_state)
tf.assign(init_state, final_state)
So, I am taking the output of the pool layer and try to feed it into the LSTM in the network.
Initially I have declared the following:
state_size = 16
step_size = 8
batch_partition_length = int(batch_size / step_size)
init_state = tf.Variable(tf.zeros([batch_partition_length, state_size])) # -> [128, 16].
Therefore, I am getting an error on:
lstm_outputs, final_state = tf.nn.dynamic_rnn(cell=lstm, inputs=lstm_inputs, initial_state=init_state)
As follows:
Traceback (most recent call last):
File "C:/Users/user/PycharmProjects/AffectiveComputing/Brady_with_LSTM.py", line 197, in <module>
predictions = dense_layers(conv_nets_output)
File "C:/Users/user/PycharmProjects/AffectiveComputing/Brady_with_LSTM.py", line 162, in dense_layers
lstm_outputs, final_state = tf.nn.dynamic_rnn(cell=lstm, inputs=lstm_inputs, initial_state=init_state)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\rnn.py", line 553, in dynamic_rnn
dtype=dtype)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\rnn.py", line 720, in _dynamic_rnn_loop
swap_memory=swap_memory)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2623, in while_loop
result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2456, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2406, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\rnn.py", line 705, in _time_step
(output, new_state) = call_cell()
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\rnn.py", line 691, in <lambda>
call_cell = lambda: cell(input_t, state)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\contrib\rnn\python\ops\core_rnn_cell_impl.py", line 238, in __call__
c, h = state
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 504, in __iter__
raise TypeError("'Tensor' object is not iterable.")
TypeError: 'Tensor' object is not iterable.
Any help is much appreciated!!
The state for LSTMs really consists of two parts
State for the cell(s)
Previous outputs
This is alluded to in the docs for BasicLSTMCell. This paper has a good explanation of how LSTMs work which will help you understand why you need to keep two sets of states in an LSTM implementation. The reason an error is being thrown is because you need to supply a tuple of tensors for the initial state.
That said you have two options:
Supply an initial state that consists of two tensors.
Let the RNN cell generate its own initial state.
You would usually only do 1. if you wanted to override default behavior. In this case you are using the default (zero) initial state so you can do 2.
lstm_outputs, final_state = tf.nn.dynamic_rnn(cell=lstm, inputs=lstm_inputs, dtype=tf.float32)
I'm a beginner to TensorFlow and still trying to figure out how it works, so I'm not sure if the error is a problem with my architecture or something more basic -- here I'm trying to train a siamese neural network (we feed a left and right input into left and right NN with identical weights, and try to map it to feature vectors that have small distance if the inputs are similar and large distance if the inputs are different).
The error I get occurs at the regression step:
File "siamese.py", line 59, in <module>
network = regression(y_pred, optimizer='adam',
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tflearn/models/dnn.py", line 63, in __init__
best_val_accuracy=best_val_accuracy)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tflearn/helpers/trainer.py", line 120, in __init__
clip_gradients)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tflearn/helpers/trainer.py", line 646, in initialize_training_ops
ema_num_updates=self.training_steps)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tflearn/summaries.py", line 236, in add_loss_summaries
loss_averages_op = loss_averages.apply([loss] + other_losses)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/training/moving_averages.py", line 292, in apply
colocate_with_primary=(var.op.type == "Variable"))
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/training/slot_creator.py", line 106, in create_zeros_slot
val = array_ops.zeros(primary.get_shape().as_list(), dtype=dtype)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1071, in zeros
shape = ops.convert_to_tensor(shape, dtype=dtypes.int32, name="shape")
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 628, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 198, in _tensor_shape_tensor_conversion_function
"Cannot convert a partially known TensorShape to a Tensor: %s" % s)
ValueError: Cannot convert a partially known TensorShape to a Tensor: (?,)
I don't know how to resolve this problem if the first dimension needs to be None for the batch size (correct me if I'm wrong).
Relevant parts of the code are below:
BATCH_SIZE=100
def contrastive_loss(y_pred, y_true, margin=1.0):
return tf.mul(1-y_true, tf.square(y_pred)) + tf.mul(y_true, tf.square(tf.maximum((margin-y_pred),0)))
## Load dataset
f = h5py.File('./data/paired_training_data.hdf','r')
X1 = f["train_X1"]
X2 = f["train_X2"]
Y = f["train_Y_paired"]
## Inputs: 1 example (phoneme pair), dropout probability
inp_sound1 = input_data(shape=[None, 1, N_MFCC_CHANNELS, N_IN_CHANNELS])
networkL = conv_1d(inp_sound1, reuse=None, scope="conv1d")
networkL = max_pool_1x6(networkL)
networkL = fully_connected(networkL, n_units=N_FULLY_CONN, activation='relu', scope="fc1")
networkL = dropout(networkL, .5) # unshared?
networkL = fully_connected(networkL, n_units=N_FULLY_CONN, activation='relu', scope="fc2")
inp_sound2 = input_data(shape=[None, 1, N_MFCC_CHANNELS, N_IN_CHANNELS])
networkR = conv_1d(inp_sound2, reuse=True, scope="conv1d")
networkR = max_pool_1x6(networkR)
networkR = fully_connected(networkR, n_units=N_FULLY_CONN, activation='relu', reuse=True, scope="fc1")
networkR = dropout(networkR, .5)
networkR = fully_connected(networkR, n_units=N_FULLY_CONN, activation='relu', reuse=True, scope="fc2")
l2_loss = tf.reduce_sum(tf.square(tf.sub(networkL, networkR)), 1)
y_pred = tf.sqrt(l2_loss)
#y_true = input_data(shape=[None])
## Training
network = regression(y_pred, optimizer='adam',
loss=contrastive_loss, learning_rate=0.0001, to_one_hot=False)
model = tflearn.DNN(network, tensorboard_verbose=0)
model.fit([X1, X2], Y, n_epoch=10, batch_size=BATCH_SIZE, show_metric=True, validation_set=0.1)
Any help -- especially with understanding how to debug these issues on my own in the future -- would be greatly appreciated!
It looks like TensorFlow cannot infer the shape of your contrastive_loss. Try to call set_shape in your contrastive_loss function if you know its output shape in advance:
def contrastive_loss(y_pred, y_true, margin=1.0):
loss = tf.mul(1-y_true, tf.square(y_pred)) + tf.mul(y_true, tf.square(tf.maximum((margin-y_pred),0)))
loss.set_shape([...])
return loss