I have a tensor with the following properties. I want to save the numpy = 1 but I don't know how to access this value. How do I do this?
test_labels[1]
<tf.Tensor: shape=(), dtype=int32, numpy=1>
You can use tf.print() to get the values of the tensor.
For Example:
a=tf.constant(1) #Output:<tf.Tensor: shape=(), dtype=int32, numpy=1>
tf.print(a) #Output:1
Related
I am exploring tensorflow internals, and sometimes when I print the value of a tensor, I will see data like the following: Tensor("x/PlaceholderWithDefault:0", shape=(), dtype=int32)
and other times I will see tf.Tensor(0, shape=(), dtype=int32).
What is the difference between these two expressions? Are Tensor and tf.Tensor different? And if not, why are they displayed differently (and seem to have different behavior)?
I am trying to create a tf.data.Dataset to be able to train an object detection model. Besides images, the dataset will contain bounding-box and label information. For each image the number of bounding boxes/labels can differ.
Let's assume I have the following two tensors:
<tf.Tensor: shape=(1, 2, 4), dtype=float32, numpy=
array([[[0.12723215, 0.43973213, 0.46205357, 0.39285713],
[0.5089286 , 0.19196428, 0.25223213, 0.25892857]]], dtype=float32)>,
<tf.Tensor: shape=(1, 3, 4), dtype=float32, numpy=
array([[[0.14285715, 0.45982143, 0.4174107 , 0.2857143 ],
[0.546875 , 0.18973215, 0.24776785, 0.25892857],
[0.66741073, 0.10044643, 0.01785714, 0.02455357]]], dtype=float32)>
With similar size tensors I can concatenate them tf.concat([tensor1, tensor2], axis=0) and create a dataset with tf.data.Dataset.from_tensor_slices(concatenated_tensor), but how do I get these tensors with different sizes into a dataset?
I am trying to port the implementation of concrete dropout in keras in https://github.com/yaringal/ConcreteDropout/blob/master/concrete-dropout-keras.ipynb to tensorflow 2. This is mostly straightforward, as tf 2 has most of the keras API built into it. However, the custom losses are being cleared before fitting.
After the model is defined, and before compiling it, I can see that the losses for each concrete dropout layer have been added to the model losses by the line self.layer.add_loss(regularizer) run when the layers are built:
>>> print(model.losses)
[<tf.Tensor: id=64, shape=(), dtype=float32, numpy=-8.4521576e-05>, <tf.Tensor: id=168, shape=(), dtype=float32, numpy=-0.000650166>, <tf.Tensor: id=272, shape=(), dtype=float32, numpy=-0.000650166>, <tf.Tensor: id=376, shape=(), dtype=float32, numpy=-0.000650166>, <tf.Tensor: id=479, shape=(), dtype=float32, numpy=-0.000650166>]
After the compilation, however, model.losses becomes an empty list, and the assertion assert len(model.losses) == 5 fails. If I choose to ignore the assertion, the fact that the layer losses are being neglected shows up in the warning WARNING:tensorflow:Gradients do not exist for variables ['concrete_dropout/p_logit:0', 'concrete_dropout_1/p_logit:0', 'concrete_dropout_2/p_logit:0', 'concrete_dropout_3/p_logit:0', 'concrete_dropout_4/p_logit:0'] when minimizing the loss. when training the model.
After digging into the compilation code in https://github.com/tensorflow/tensorflow/blob/r2.0/tensorflow/python/keras/engine/training.py#L184 I believe the problematic lines are
# Clear any `_eager_losses` that was added.
self._clear_losses()
Why is this being done at model compilation? And how can I add a loss per layer in tensorflow 2, if this is not the way to do it?
Since that custom loss is not dependent on the model's inputs, you should add it using a zero-argument callable like this:
self.layer.add_loss(lambda: regularizer)
My code has similar pattern with tensorflow 2.0 tutorial.
I want my dataset object to reshuffle in every epochs.
dataset = tf.data.Dataset.from_tensor_slices(['a','b','c','d'])
dataset = dataset.shuffle(100)
for epoch in range(10):
for d in dataset:
print(d)
Result:
tf.Tensor(b'c', shape=(), dtype=string)
tf.Tensor(b'a', shape=(), dtype=string)
tf.Tensor(b'b', shape=(), dtype=string)
tf.Tensor(b'd', shape=(), dtype=string)
tf.Tensor(b'c', shape=(), dtype=string)
tf.Tensor(b'a', shape=(), dtype=string)
tf.Tensor(b'b', shape=(), dtype=string)
tf.Tensor(b'd', shape=(), dtype=string)
...
It seems the dataset doesn't shuffle for each epoch.
Should I call .shuffle() for each epoch?
Yes, you should call .shuffle during the inner loop. Moreover, it is better to do not mix python code and TensorFlow code when pure tf.* method equivalent to the Python statements are available.
import tensorflow as tf
dataset = tf.data.Dataset.from_tensor_slices(["a", "b", "c", "d"])
# dataset = dataset.shuffle(2)
#tf.function
def loop():
for epoch in tf.range(10):
for d in dataset.shuffle(2):
tf.print(d)
loop()
The loop call produces the different values every time (and tf.print prints the content of the tf.Tensor, differently from print that prints the object).
I am trying to use tensorflow-probability and started off with something really simple:
import tensorflow as tf
import tensorflow_probability as tfp
tf.enable_eager_execution()
tfd = tfp.distributions
poiss = tfd.Poisson(0.8)
poiss.sample(2, seed=1)
#> Out: <tf.Tensor: id=3569, shape=(2,), dtype=float32, numpy=array([0., 0.], dtype=float32)>
poiss.sample(2, seed=1)
#> Out: <tf.Tensor: id=3695, shape=(2,), dtype=float32, numpy=array([1., 0.], dtype=float32)>
poiss.sample(2, seed=1)
#> Out: <tf.Tensor: id=3824, shape=(2,), dtype=float32, numpy=array([2., 2.], dtype=float32)>
poiss.sample(2, seed=1)
#> Out: <tf.Tensor: id=3956, shape=(2,), dtype=float32, numpy=array([0., 1.], dtype=float32)>
I was thinking I would get the same results when re-using the same seed, but somehow that's not true.
I also tried without eager execution, but the results still weren't reproducible. Same story if I add something like tf.set_random_seed(12).
I suppose there is something basic I am missing?
For those interested, I am running Python 3.5.2 on Ubuntu 16.04 with
tensorflow-probability==0.5.0
tensorflow==1.12.0
For deterministic output in graph mode, you need to set both the graph random seed (tf.set_random_seed) and the op random seed (seed= in your sample call).
The workings of random samplers in TFv2 are still being sorted out. For now, my best understanding is that you can call tf.set_random_seed prior to each call to a sampler, and pass the sampler a seed=, if you want deterministic output in eager.
This is now cleaner, we support fully deterministic randomness in TFP. You can pass a tuple of two ints for seed, or a Tensor of shape (2,) to trigger the deterministic behavior. tfp.random.split_seed is also relevant here.
Besides setting the seed for sample or sample_chain in mcmc, you might need to set the followings as well:
seed = 24
os.environ['TF_DETERMINISTIC_OPS'] = 'true'
os.environ['PYTHONHASHSEED'] = f'{seed}'
np.random.seed(seed)
random.seed(seed)
tf.random.set_seed(seed)