I have two data which type is ndarray.
I want to merge X_train and Y_train, but they are have different shape.
I try to use concatenate, stack but it show me same error.
print(X_train.shape)
print(type(X_train[1]))
print(Y_train.shape)
print(type(Y_train[1]))
(17731, 30, 14, 1)
<class 'numpy.ndarray'>
(17731, 1)
<class 'numpy.ndarray'>
x_train_tt = np.stack([X_train , Y_train])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-43-74b8e20eca7c> in <module>
----> 3 x_train_tt = np.stack([X_train , Y_train])
<__array_function__ internals> in stack(*args, **kwargs)
~\anaconda3\envs\newenvironment2.3.6\lib\site-packages\numpy\core\shape_base.py in stack(arrays, axis, out)
424 shapes = {arr.shape for arr in arrays}
425 if len(shapes) != 1:
--> 426 raise ValueError('all input arrays must have the same shape')
427
428 result_ndim = arrays[0].ndim + 1
ValueError: all input arrays must have the same shape
Is there any solution?
I want to get shape like that
x_train_tt.shape
(17731,30,15,1)
Related
class CropLayer(layers.Layer):
def __init__(self, crop_t, **kwargs):
super().__init__(**kwargs)
self.crop_t = crop_t
def get_config(self):
config = super().get_config()
config.update({'crop_t': self.crop_t})
return config
def call(self, inputs):
outputs = []
for i, x in enumerate(inputs):
t = tf.shape(x[0])[0]
start = tf.experimental.numpy.random.randint(0, t-self.crop_t, dtype='int32')
end = start + self.crop_t
outputs.append(inputs[i, :, start:end].to_list())
return tf.constant(outputs)
def get_cropper(crop_t):
return keras.Sequential(
[
keras.Input(shape=(N, None), ragged=True),
CropLayer(crop_t)
]
)
cropper = get_cropper(crop_t)
I want to make a custom layer that the ragged tensor as input and tensor as output.
The layer crop ragged tensors to fit the size, so it can convert to tensor format. But when I run this code, the following error occurs.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-96-6430b3383331> in <module>()
----> 1 cropperr = get_cropper(crop_t)
3 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py in wrapper(*args, **kwargs)
690 except Exception as e: # pylint:disable=broad-except
691 if hasattr(e, 'ag_error_metadata'):
--> 692 raise e.ag_error_metadata.to_exception(e)
693 else:
694 raise
ValueError: Exception encountered when calling layer "crop_layer_7" (type CropLayer).
in user code:
File "<ipython-input-93-419907fac9d0>", line 31, in call *
outputs.append(inputs[i, :, start:end].to_list())
ValueError: to_list can only be used in eager mode.
I would like to write tensorflow example records to a TFRecordWriter from inside an AutoGraph generated graph.
The documentation for tensorflow 2.0 states the following:
The simplest way to handle non-scalar features is to use tf.serialize_tensor to convert tensors to binary-strings. Strings are scalars in tensorflow.
However, tf.io.serialize_tensor returns a tensor of byte-string. Creating an Example proto requires a bytes list, not a tensor.
How do I write a tf.train.Example to a tf record from inside a graph?
Code to reproduce:
%tensorflow_version 2.x
import tensorflow as tf
#tf.function
def example_write():
writer = tf.io.TFRecordWriter("test.tfr")
x = tf.constant([[0, 1], [2, 3]])
x = tf.io.serialize_tensor(x)
feature = {
"data": tf.train.Features(
bytes_list=tf.train.BytesList(value=[x]))
}
ex = tf.train.Example(features=tf.train.Features(
feature=feature))
writer.write(ex.SerializeToString())
example_write()
and the error
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-df8a97eb17c9> in <module>()
12 writer.write(ex.SerializeToString())
13
---> 14 example_write()
8 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
966 except Exception as e: # pylint:disable=broad-except
967 if hasattr(e, "ag_error_metadata"):
--> 968 raise e.ag_error_metadata.to_exception(e)
969 else:
970 raise
TypeError: in user code:
<ipython-input-6-df8a97eb17c9>:6 example_write *
feature = {
TypeError: <tf.Tensor 'SerializeTensor:0' shape=() dtype=string> has type Tensor, but expected one of: bytes
It's pretty straightforward:
use x = tf.io.serialize_tensor(x).numpy()
I am trying to configure a model that I previously trained to classify images in a such a way that it accepts images as base64-strings (instead of a NumPy array), converts them to a NumPy array and then performs the prediction. How do I add a layer on top of my regular input layer that accepts strings and outputs a NumPy array?
So I've already pre-trained a model that predicts images based on the ResNet architecture. Having looked at this and this answer, I am trying to create a Lambda layer that converts strings to RGB jpeg images. I have done this as shown in the sample code below:
image = tf.placeholder(shape=[], dtype=tf.string)
input_tensor = keras.layers.Input(shape = (1,), tensor = image, dtype=tf.string)
x = keras.layers.Lambda(lambda image: tf.image.decode_jpeg(image))(input_tensor)
output_tensor = model(x)
new_model = Model(input_tensor, output_tensor)
Where model() is the Keras keras.models.Model model that I have pre-trained.
I am expecting new_model() to be the new Keras model that has 1 extra layer on top of my previous model, which accepts base64-string and outputs a NumPy array into the next layer.
However, the third line of my code raises the following error:
TypeError: Input 'contents' of 'DecodeJpeg' Op has type float32 that does not match expected type of string.
My understanding of this is that the 'image' in the Lambda layer that uses the decode_jpeg() is a float32 instead of a string, which seems odd to me as I have set the dtype of both the placeholder as well as the Input layer to tf.string.
I have searched all over stackoverflow for this but can't find a solution for this error. It appears this question also has not been able to find a solution for this specific issue.
EDIT 1: corrected typo and added full error message
The full error message is show below:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
509 as_ref=input_arg.is_ref,
--> 510 preferred_dtype=default_dtype)
511 except TypeError as err:
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in internal_convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, ctx)
1103 if ret is None:
-> 1104 ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
1105
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in _TensorTensorConversionFunction(t, dtype, name, as_ref)
946 "Tensor conversion requested dtype %s for Tensor with dtype %s: %r" %
--> 947 (dtype.name, t.dtype.name, str(t)))
948 return t
ValueError: Tensor conversion requested dtype string for Tensor with dtype float32: 'Tensor("lambda_28/Placeholder:0", shape=(?, 1), dtype=float32)'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-47-5793b0703860> in <module>
1 image = tf.placeholder(shape=[], dtype=tf.string)
2 input_tensor = Input(shape = (1,), tensor = image, dtype=tf.string)
----> 3 x = Lambda(lambda image: tf.image.decode_jpeg(image))(input_tensor)
4 output_tensor = model(x)
5
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)
472 if all([s is not None
473 for s in to_list(input_shape)]):
--> 474 output_shape = self.compute_output_shape(input_shape)
475 else:
476 if isinstance(input_shape, list):
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/layers/core.py in compute_output_shape(self, input_shape)
650 else:
651 x = K.placeholder(shape=input_shape)
--> 652 x = self.call(x)
653 if isinstance(x, list):
654 return [K.int_shape(x_elem) for x_elem in x]
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/layers/core.py in call(self, inputs, mask)
685 if has_arg(self.function, 'mask'):
686 arguments['mask'] = mask
--> 687 return self.function(inputs, **arguments)
688
689 def compute_mask(self, inputs, mask=None):
<ipython-input-47-5793b0703860> in <lambda>(image)
1 image = tf.placeholder(shape=[], dtype=tf.string)
2 input_tensor = Input(shape = (1,), tensor = image, dtype=tf.string)
----> 3 x = Lambda(lambda image: tf.image.decode_jpeg(image))(input_tensor)
4 output_tensor = model(x)
5
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_image_ops.py in decode_jpeg(contents, channels, ratio, fancy_upscaling, try_recover_truncated, acceptable_fraction, dct_method, name)
946 try_recover_truncated=try_recover_truncated,
947 acceptable_fraction=acceptable_fraction, dct_method=dct_method,
--> 948 name=name)
949 _result = _op.outputs[:]
950 _inputs_flat = _op.inputs
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
531 if input_arg.type != types_pb2.DT_INVALID:
532 raise TypeError("%s expected type of %s." %
--> 533 (prefix, dtypes.as_dtype(input_arg.type).name))
534 else:
535 # Update the maps with the default, if needed.
TypeError: Input 'contents' of 'DecodeJpeg' Op has type float32 that does not match expected type of string.
When trying to use tf.maximum as one would expect:
loss = tf.maximum(0, basic_loss)
this error is obtained
--------------------------------------------------------------------------- ValueError Traceback (most recent call
last)
/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py
in apply_op(self, op_type_name, name, **keywords)
489 as_ref=input_arg.is_ref,
--> 490 preferred_dtype=default_dtype)
491 except TypeError as err:
/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/ops.py
in internal_convert_to_tensor(value, dtype, name, as_ref,
preferred_dtype)
740 if ret is None:
--> 741 ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
742
/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/ops.py
in _TensorTensorConversionFunction(t, dtype, name, as_ref)
613 "Tensor conversion requested dtype %s for Tensor with dtype %s: %r"
--> 614 % (dtype.name, t.dtype.name, str(t)))
615 return t
ValueError: Tensor conversion requested dtype int32 for Tensor with
dtype float32: 'Tensor("add_13:0", shape=(), dtype=float32)'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call
last) in ()
5 tf.random_normal([3, 128], mean=1, stddev=1, seed = 1),
6 tf.random_normal([3, 128], mean=3, stddev=4, seed = 1))
----> 7 loss = triplet_loss(y_true, y_pred)
8
9 print("loss = " + str(loss.eval()))
in triplet_loss(y_true, y_pred, alpha)
26 basic_loss = pos_dist - neg_dist + alpha
27 # Step 4: Take the maximum of basic_loss and 0.0. Sum over the training examples.
---> 28 loss = tf.maximum(0, basic_loss)
29 ### END CODE HERE ###
30
/opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py
in maximum(x, y, name) 1261 A Tensor. Has the same type as
x. 1262 """
-> 1263 result = _op_def_lib.apply_op("Maximum", x=x, y=y, name=name) 1264 return result 1265
/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py
in apply_op(self, op_type_name, name, **keywords)
524 "%s type %s of argument '%s'." %
525 (prefix, dtypes.as_dtype(attrs[input_arg.type_attr]).name,
--> 526 inferred_from[input_arg.type_attr]))
527
528 types = [values.dtype]
TypeError: Input 'y' of 'Maximum' Op has type float32 that does not
match type int32 of argument 'x'.
What seems to be the problem?
The tensor flow doc does not state that the maximum function is non-commutative.
It works only if the type of the 1st argument is a Tensor, but not if its type is int.
Need to call this function with replaced positions of the arguments for constants:
tf.maximum(basic_loss, 0)
instead of
tf.maximum(0, basic_loss)
If I use tf.train.batch( allow_smaller_final_batch=True, ) the shape of the tensor is unknown:
allow_smaller_final_batch=True: Tensor shape= (?, 224, 224, 3)
allow_smaller_final_batch=False: Tensor shape= (16, 224, 224, 3)
This is giving me an error downstream.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/anaconda/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py in make_tensor_proto(values, dtype, shape, verify_shape)
467 try:
--> 468 str_values = [compat.as_bytes(x) for x in proto_values]
469 except TypeError:
/anaconda/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py in <listcomp>(.0)
467 try:
--> 468 str_values = [compat.as_bytes(x) for x in proto_values]
469 except TypeError:
/anaconda/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/compat.py in as_bytes(bytes_or_text, encoding)
64 raise TypeError('Expected binary or unicode string, got %r' %
---> 65 (bytes_or_text,))
66
TypeError: Expected binary or unicode string, got None
...
TypeError: Failed to convert object of type <class 'list'> to Tensor. Contents: [None, 1]. Consider casting elements to a supported type.
How do I get the batch_size before I evaluate in session?
batch_size is a hyper-parameter that you should assign to it, rather than a value to be evaluated in TensorFlow session, while there's not enough information to determine what trouble you encounter.