I am currently trying to perform a TensorFlow slice assignment similar to this PyTorch code.
input_seq[1:, :] = torch.from_numpy(stroke[:-1, :])
A plain item assignment like the above does not work for TensorFlow and gives the following error.
TypeError: 'tensorflow.python.framework.ops.EagerTensor' object does not support item assignment.
Previous solutions for the same problem are quite dated using older versions of TensorFlow. I would greatly appreciate any help regarding how to tackle the same.
Here's an example. You can't do numpy like slice assignements. But can do the following (Tested tensorflow==2.9).
tensorflow as tf
import numpy as np
a = tf.constant([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
b = np.array([[6, 5, 4], [9, 8, 7]])
c = tf.tensor_scatter_nd_update(a, [[1], [2]], b)
Related
I would like to add skip connections for my inner layers of a fully convolutional network in keras, there is a keras.layers.Add option and there is a keras.layers.concatenate option.
What is the difference? and which one I should use?
What is the difference?
Add layer adds two input tensor while concatenate appends two tensors. You can refer this documentation for more info.
Example:
import keras
import tensorflow as tf
import keras.backend as K
a = tf.constant([1,2,3])
b = tf.constant([4,5,6])
add = keras.layers.Add()
print(K.eval(add([a,b])))
#output: [5 7 9]
concat = keras.layers.Concatenate()
print(K.eval(concat([a,b])))
#output: array([1, 2, 3, 4, 5, 6], dtype=int32)
which one I should use?
You can use Add for skip connections.
I was using Tensorflow 2.0 to build a super resolution model. During pre-processing, I wanted to crop both the low and high resolution images by a given patch size. In order to do so, I wanted to get the height and width of the low and high resolution images. But tf.shape(image) is returning None.
Is there a better approach?
Currently I am just resizing every image to some size before using tf.shape, but since not all images have equal size, it is affecting the quality of the imaged. Looking forward to your suggestions.
Edited part:
Here is some parts of the code
low_r = tf.io.decode_jpeg(lr_filename, channels=3)
low_r = tf.cast(low_r, dtype=tf.float32)
print(low_r.shape)
The print statement prints (None, None, 3)
What I wanted was to get the height and weight, like (240,360,3)
I'm not sure if this is also your case, but in my TensorFlow (v2.4.0rc2), my_tensor.shape() also returns TensorShape([None, None, None, None]). It is connected to the fact that the TensorShape tensor is generated during the build and not during the execution.
Using tf.shape() (mentioned in your question, but not used in your code snippet actually) solves it for me.
> my_tensor.shape()
TensorShape([None, None, None, None])
> tf.shape(my_tensor)
[10 512 512 8]
I'm unable to repeat your issue, but this should give you a way to test out your Tensorflow 2.0 install and compare with the results you're currently getting.
Create a tensor and check it's shape:
import tensorflow as tf
t = tf.constant([[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]])
tf.shape(t) # [2, 2, 3]
Out[1]: <tf.Tensor: id=1, shape=(3,), dtype=int32, numpy=array([2, 2, 3])>
Next, checking what the function return when called:
tf_shape_var = tf.shape(t)
print(tf_shape_var)
Output:
tf.Tensor([2 2 3], shape=(3,), dtype=int32)
Finally, calling it on an int and string to get back a valid return:
tf.shape(1)
Out[10]: <tf.Tensor: id=12, shape=(0,), dtype=int32, numpy=array([], dtype=int32)>
tf.shape('asd')
Out[11]: <tf.Tensor: id=15, shape=(0,), dtype=int32, numpy=array([], dtype=int32)>
And the print statements:
print(tf.shape(1))
print(tf.shape('asd'))
Output:
tf.Tensor([], shape=(0,), dtype=int32)
tf.Tensor([], shape=(0,), dtype=int32)
Link for tf.shape() https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/shape
There's an argument in that function seed=something. Even when I set its value, the shuffle give random results. I want the same results.
tf.random.suffle(tf.range(5), seed=5)
If you want to reproduce shuffle results, use (on TF 2.0 beta) the following
tf.random.set_seed(5)
tf.random.shuffle(tf.range(5))
<tf.Tensor: id=35, shape=(5,), dtype=int32, numpy=array([0, 4, 1, 3, 2], dtype=int32)>
tf.random.set_seed(5)
tf.random.shuffle(tf.range(5))
<tf.Tensor: id=41, shape=(5,), dtype=int32, numpy=array([0, 4, 1, 3, 2], dtype=int32)>
tf.random.set_seed(5)
tf.random.shuffle(tf.range(5))
<tf.Tensor: id=47, shape=(5,), dtype=int32, numpy=array([0, 4, 1, 3, 2], dtype=int32)>
About the seed you have used, it indeed fails to reproduce results, tested in TF 2.0 beta
In TF 1.x I believe the right functions is tf.random.set_random_seed
From the docs, I can see there are op level seed, and graph level seed. You are setting the op level, which is not enough - setting the graph level seed with the function in the code above solves this behavior.
Came across this question. It seems like there is now a tf.random.experimental.stateless_shuffle that should give reproducible results.
To get random seeds, you can use tf.random.Generator.
i have a model that extract 512 features from an image (numbers between -1,1).
i converted this model to tflite float format using the instruction here
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite
i run an inference on the same image with the original model and the tflite model.
i am getting different results for the vector, i was expecting to get very similar results as i didn't use quantized format. and from what i understand tf-lite should only improve the inference performance time and not effect the features calculation.
my question is this normal ? anyone else encountered this ?
i didn't find any topics regarding this at any place.
Updated with code.
i have this network i trained (removed many items as i can't share full network)
placeholder = tf.placeholder(name='input', dtype=tf.float32,shape=[None, 128,128, 1])
with slim.arg_scope([slim.conv2d, slim.separable_conv2d],
activation_fn=tf.nn.relu, normalizer_fn=slim.batch_norm):
net = tf.identity(placeholder)
net = slim.conv2d(net, 32, [3, 3], scope='conv11')
net = slim.separable_conv2d(net, 64, [3, 3], scope='conv12')
net = slim.max_pool2d(net, [2, 2], scope='pool1') # 64x64
net = slim.separable_conv2d(net, 128, [3, 3], scope='conv21')
net = slim.max_pool2d(net, [2, 2], scope='pool2') # 32x32
net = slim.separable_conv2d(net, 256, [3, 3], scope='conv31')
net = slim.max_pool2d(net, [2, 2], scope='pool3') # 16x16
net = slim.separable_conv2d(net, 512, [3, 3], scope='conv41')
net = slim.max_pool2d(net, [2, 2], scope='pool4') # 8x8
net = slim.separable_conv2d(net, 1024, [3, 3], scope='conv51')
net = slim.avg_pool2d(net, [8, 8], scope='pool5') # 1x1
net = slim.dropout(net)
net = slim.conv2d(net, feature_vector_size, [1, 1], activation_fn=None, normalizer_fn=None, scope='features')
embeddings = tf.nn.l2_normalize(net, 3, 1e-10, name='embeddings')
bazel-bin/tensorflow/contrib/lite/toco/toco --input_file=/tmp/network_512.pb
--input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE --output_file=/tmp/tffiles/network_512.tflite
--inference_type=FLOAT --input_type=FLOAT --input_arrays=input --output_arrays=embeddings --input_shapes=1,128,128,1
i run network_512.pb using tensorflow in python and network_512.tflite using the code from https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/java/demo
where i modified the code to load my network with and run it.
Update that i have found. the test i did was using the Demo app tensorflow provide and change it to use my costume model and extracting features, and there i noticed the difference in the features values.
once i compiled the tf-lite c++ lib manually on latest android, and run the flow with the same flow i use (which is TF-C API until now) i got almost same results for the features.
didn't have time to investigate from where come the difference. but i am happy now.
I am looking for a way to unpack bits in TF in the same way I can do this with np.unpackbits. So revert the operation like:
import numpy as np
import tensorflow as tf
original = np.random.choice(a=[1, 0], size=(100))
data = np.packbits(original.astype(np.bool), axis=None)
X = tf.constant(data)
Assuming I have access only to X, how to convert it to original in TF. Of course I can use numpy, but this will move data from TF to python and then back to TF.
Few thoughts I had in mind (have not implemented any of them):
use tf.map_fn
use tf.contrib.lookup
For both of them the ideas is to map each number to a vector, concat all the vectors, reshape, remove unneeded elements.
Both of the approaches seems more complicated that they should be. Does anyone has an efficient way (in terms of speed) how to achieve numpy's unpackbits in tensorflow?
Perhaps something like this:
import tensorflow as tf
x = tf.constant((1, 2, 7, 0, 255), dtype=tf.uint8)
b = tf.constant((128, 64, 32, 16, 8, 4, 2, 1), dtype=tf.uint8)
unpacked = tf.reshape(tf.mod(tf.to_int32(x[:,None] // b), 2), [-1])
unpacked is in int32 due to tf.mod not accepting bytes, you may want to cast it to uint8 again.
Tensorflow 1.3 will have bitwise operations, so this last line could be replaced with
unpacked = tf.reshape(tf.bitwise.bitwise_and(x, b), [-1])
which will hopefully be faster (and the result in uint8).