I have two lines and I want to understand whether they will produce the same output or not?
In tensorflow: tf.norm(my_tensor, ord=2, axis=1)
In pytorch: torch.norm(my_tensor, p=2, dim=1)
Say the shape of my_tensor is [100,2]
Will the above two lines give the same result? Or is the axis attribute different from dim?
Yes, they are the same!
import tensorflow as tf
tensor = [[1., 2.], [4., 5.], [3., 6.], [7., 8.], [5., 2.]]
tensor = tf.convert_to_tensor(tensor, dtype=tf.float32)
t_norm = tf.norm(tensor, ord=2, axis=1)
print(t_norm)
Output
tf.Tensor([ 2.236068 6.4031243 6.708204
10.630146 5.3851647], shape=(5,), dtype=float32)
import torch
tensor = [[1., 2.], [4., 5.], [3., 6.], [7., 8.], [5., 2.]]
tensor = torch.tensor(tensor, dtype=torch.float32)
t_norm = torch.norm(tensor, p=2, dim=1)
print(t_norm)
Output
tensor([ 2.2361, 6.4031, 6.7082, 10.6301, 5.3852])
Related
If there is a Torch tensor of 3 x 3 how can I use Torch only to convert it to a 3 x 1 x 3 tensor?
Starting with this 3x3 tensor...
tensor([[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.]])
... That I want to convert to this 3x1x3 tensor
tensor([[[1., 1., 1.]],
[[1., 1., 1.]],
[[1., 1., 1.]]])
How can I do this in Torch?
I know it can by done by indexing like so...
torch.ones(3, 3)[:, None, :] but I need to do this in Torch and Torch alone.
I tried with this line...
torch.ones(3, 3).unsqueeze(-1)
But it gave me
tensor([[[1.],
[1.],
[1.]],
[[1.],
[1.],
[1.]],
[[1.],
[1.],
[1.]]])
Which doesn't do the trick.
(The reason I can't do torch.ones(3, 3)[:, None, :] is that I'm using LibTorch in C++, and this type of indexing is not possible, but we can leave LibTorch and C++ out of it and just focus on if Torch can do this.)
Consider the (3,3) tensor x = torch.ones((3,3)). To reshape this to a (3,1,3) tensor, use x.unsqueeze(1).
I have two matrices. Matrix A is contains some values and matrix B contains indices. The shape of matrix A and B is (batch, values) and (batch, indices), respectively.
My goal is to select values from matrix A based on indices of matrix B along the batch dimension.
For example:
# Matrix A
<tf.Tensor: shape=(2, 5), dtype=float32, numpy=
array([[0., 1., 2., 3., 4.],
[5., 6., 7., 8., 9.]], dtype=float32)>
# Matrix B
<tf.Tensor: shape=(2, 2), dtype=int32, numpy=
array([[0, 1],
[1, 2]], dtype=int32)>
# Expected Result
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[0., 1.],
[6., 7.]], dtype=int32)>
How can I achieve this in Tensorflow?
Many thanks in advance!
You can achieve this with the tf.gather function.
mat_a = tf.constant([[0., 1., 2., 3., 4.],
[5., 6., 7., 8., 9.]])
mat_b = tf.constant([[0, 1], [1, 2]])
out = tf.gather(mat_a, mat_b, batch_dims=1)
out.numpy()
array([[0., 1.],
[6., 7.]], dtype=float32)
I want to shift a tensor in a given axis. It's easy to do this in pandas or numpy. Like this:
import numpy as np
import pandas as pd
data = np.arange(0, 6).reshape(-1, 2)
pd.DataFrame(data).shift(1).fillna(0).values
Output is:
array([[0., 0.],
[0., 1.],
[2., 3.]])
But in tensorflow, the closest solution I found is tf.roll. But it shift the last row to the first row. (I don't want that). So I have to use something like
tf.roll + tf.slice(remove the last row) + tf.concat(add tf.zeros to the first row).
It's really ugly.
Is there a better way to handle shift in tensorflow or keras?
Thanks.
I think I find a better way for this problem.
We could use tf.roll, then apply tf.math.multiply to set the first row to zeros.
Sample code is as follows:
Original tensor:
A = tf.cast(tf.reshape(tf.range(27), (-1, 3, 3)), dtype=tf.float32)
A
Output:
<tf.Tensor: id=117, shape=(3, 3, 3), dtype=float32, numpy=
array([[[ 0., 1., 2.],
[ 3., 4., 5.],
[ 6., 7., 8.]],
[[ 9., 10., 11.],
[12., 13., 14.],
[15., 16., 17.]],
[[18., 19., 20.],
[21., 22., 23.],
[24., 25., 26.]]], dtype=float32)>
Shift (like pd.shift):
B = tf.concat((tf.zeros((1, 3)), tf.ones((2, 3))), axis=0)
C = tf.expand_dims(B, axis=0)
tf.math.multiply(tf.roll(A, 1, axis=1), C)
Output:
<tf.Tensor: id=128, shape=(3, 3, 3), dtype=float32, numpy=
array([[[ 0., 0., 0.],
[ 0., 1., 2.],
[ 3., 4., 5.]],
[[ 0., 0., 0.],
[ 9., 10., 11.],
[12., 13., 14.]],
[[ 0., 0., 0.],
[18., 19., 20.],
[21., 22., 23.]]], dtype=float32)>
Try this:
import tensorflow as tf
input = tf.constant([[0, 1, 3], [4, 5, 6], [7, 8, 9]])
shifted_0dim = input[1:]
shifted_1dim = input[:, 1:]
shifted2 = input[2:]
Generalizing the accepted answer to arbitrary tensor shapes, desired shift, and axis to shift:
import tensorflow as tf
def tf_shift(tensor, shift=1, axis=0):
dim = len(tensor.shape)
if axis > dim:
raise ValueError(
f'Value of axis ({axis}) must be <= number of tensor axes ({dim})'
)
mask_dim = dim - axis
mask_shape = tensor.shape[-mask_dim:]
zero_dim = min(shift, mask_shape[0])
mask = tf.concat(
[tf.zeros(tf.TensorShape(zero_dim) + mask_shape[1:]),
tf.ones(tf.TensorShape(mask_shape[0] - zero_dim) + mask_shape[1:])],
axis=0
)
for i in range(dim - mask_dim):
mask = tf.expand_dims(mask, axis=0)
return tf.multiply(
tf.roll(tensor, shift, axis),
mask
)
EDIT:
This code above doesn't allow for negative shift values, and is pretty slow. Here is a more efficient version utilizing tf.roll and tf.concat without creating a mask and multiplying the tensor of interest by it.
import tensorflow as tf
def tf_shift(values: tf.Tensor, shift: int = 1, axis: int = 0):
pad = tf.zeros([val if i != axis else abs(shift) for i, val in enumerate(values.shape)],
dtype=values.dtype)
size = [-1 if i != axis else val - abs(shift) for i, val in enumerate(values.shape)]
if shift > 0:
shifted = tf.concat(
[pad, tf.slice(values, [0] * len(values.shape), size)],
axis=axis
)
elif shift < 0:
shifted = tf.concat(
[tf.slice(values, [0 if i != axis else abs(shift) for i, _ in enumerate(values.shape)], size), pad],
axis=axis
)
else:
shifted = values
return shifted
Assuming a 2d tensor, this function should mimic a Dataframe shift:
def shift_tensor(tensor, periods, fill_value):
num_row = len(tensor)
num_col = len(tensor[0])
pad = tf.fill([periods, num_col], fill_value)
if periods > 0:
shifted_tensor = tf.concat((pad, tensor[:(num_row - periods), :]), axis=0)
else:
shifted_tensor = tf.concat((tensor[:(num_row - periods), :], pad), axis=0)
return shifted_tensor
I have a 2D tensor. I would like to take each vector in that 2D tensor and tf.tensordot(vector, matrix, axes=1) to the matrix in a 3D tensor that has the same index in the 3D tensor as the vector does in the 2D tensor.
Essentially, I'd like the same result as I'd get with this for loop, but by doing tensorflow matrix operations rather than numpy and looping:
tensor2d = np.array([[1.,1.,1.,0.,0.],
[1.,1.,0.,0.,0.]],
np.float32)
tensor3d = np.array([
[
[1., 2., 3.],
[2., 2., 3.],
[3., 2., 3.],
[4., 2., 3.],
[5., 2., 3.],
],
[
[1., 2., 3.],
[2., 2., 3.],
[3., 2., 3.],
[4., 2., 3.],
[5., 2., 3.],
]
], np.float32)
results = []
for i in range(len(tensor2d)):
results.append(np.tensordot(tensor2d[i], tensor3d[i], axes=1))
Output of this should be a matrix that looks like this (though types would be different):
[array([6., 6., 9.], dtype=float32), array([3., 4., 6.], dtype=float32)]
Ok, the self-found answer boils down to use tf.math.multiply and mess around with transposes until the result is the desired shape. Would be great if someone could come up with a more principled answer at some point, but for now, this worked:
result = tf.transpose(tf.math.multiply(tensor2d, tensor3d.transpose([2,0,1])), [1,2,0])
Would it be possible to use numpy/scipy to multiply matrices composed of polynomials?
Specifically I wish to multiply a 120 by 120 sparse matrix who's entries can look like a+7*b+c by itself.
Honestly, I haven't tried very hard to do this. I see that there is a polynomial module in numpy but I have no experience with it. I am just hoping that someone sees this and says "obviously it's possible, do this".
There is one relevant question asked before from what I've seen: Matrices whose entries are polynomials
I don't know about sparse, but numpy object arrays work fine.
In [1]: from numpy.polynomial import Polynomial as P
In [2]: a = np.array([[P([1,2]), P([3,4])]]*2)
In [3]: a
Out[3]:
array([[Polynomial([ 1., 2.], [-1, 1], [-1, 1]),
Polynomial([ 3., 4.], [-1, 1], [-1, 1])],
[Polynomial([ 1., 2.], [-1, 1], [-1, 1]),
Polynomial([ 3., 4.], [-1, 1], [-1, 1])]], dtype=object)
In [4]: np.dot(a, a)
Out[4]:
array([[Polynomial([ 4., 14., 12.], [-1., 1.], [-1., 1.]),
Polynomial([ 12., 34., 24.], [-1., 1.], [-1., 1.])],
[Polynomial([ 4., 14., 12.], [-1., 1.], [-1., 1.]),
Polynomial([ 12., 34., 24.], [-1., 1.], [-1., 1.])]], dtype=object)