I defined the following slice layer, where I want to split Nx7 into Nx3 and Nx4 two blobs
layers {
name: "slice_label"
type: SLICE
bottom: "label"
top: "label_pos"
top: "label_rotation"
slice_param {
slice_dim: 1
slice_point: 3
}
How can I do the same thing in Tensorflow?
I believe that you can do this with a hand-crafted layer, using the split method. You can also do it with separate calls to slice. Something like
slice3, slice4 = tf.split(input_tensor, [3, 4], 1)
Correction per OP comment:
slice3, slice4 = tf.split_v(labels, [3, 4], 1)
BTW, the spacing follows PEP8 standards. It's not required, of course, but there is checking software that prefers the spacing this way.
Related
I am trying to apply style transfer to a webcam capture. I am reading a frozen model I've previously trained in python and converted for TFjs. The output tensor's shape and rank is as follows:
I am having issues in the last line of this function, when I try to apply tf.browser.toPixels
function predictWebcam() {
tf.tidy(() => {
loadmodel().then(model=>{
//let tensor= model.predict(tf.expandDims(tf.browser.fromPixels(video)));
let tensor= model.predict(tf.browser.fromPixels(video, 3).toFloat().div(tf.scalar(255)).expandDims());
console.log('shape', tensor.shape);
console.log('rank', tensor.rank);
tf.browser.toPixels(tensor, resultImage);
});
});
}
I get this error. I cannot figure out how to reshape or modify the tensor to get an image out of it:
Uncaught (in promise) Error: toPixels only supports rank 2 or 3 tensors, got rank 4.
Maybe I have to replicate tensor_to_image function from python to javascript as in the example in the website.
Thanks in advance!
given your tensor is [1, 15, 20, 512]
you can remove any dims with value of 1 (same dim you've added by running expandDims) by running
const squeezed = tf.squeeze(tensor)
that will give you shape of [15, 20, 512]
but that still doesn't make sense - what is width, height and channels (e.g. rgb) here?
i think that model result needs additional post-processing, that is not an image.
I'm trying to understand the sparse tensor with tensorflow lite schema, which is very hard to do for me though.
Luckily, there is only one json example that be made from this schema(tensorflow/lite/testdata/sparse_tensor.json).
"sparsity": {
"traversal_order": [0, 1, 2, 3],
"block_map": [0, 1],
"dim_metadata": [
{
"format": "DENSE",
"dense_size": 2
},
{
"format": "SPARSE_CSR",
"array_segments_type": "Uint8Vector",
"array_segments": {"values": [0, 2, 3]},
"array_indices_type": "Uint8Vector",
"array_indices": {"values": [0, 1, 1]}
},
{
"format": "DENSE",
"dense_size": 2
},
{
"format": "DENSE",
"dense_size": 2
}
]
}
"buffers": [
{
},
{
"data": [
1, 0, 0, 4,
2, 3, 0, 0,
5, 0, 0, 6
]
}
]
And, this is the schema I refer to(tensorflow/lite/schema/schema.fbs).
table DimensionMetadata {
// Whether a dimension is dense or sparse.
format:DimensionType;
// Index metadata used for a dimension.
// - If format is DimensionType.DENSE then we use the dense_size field to
// store the size of that dimension. Each index in that dimension is
// stored implicitly.
// - If format is DimensionType.SPARSE_CSR then we use array_segments and
// array_indices to encode that dimension. array_segments represents how
// to segment the indices array, each segment corresponds to one element
// in the previous dimension. array_indices represents the index of the
// non-zero elements within this dimension (as those in the CSR matrix
// format, where the first array is row pointers and the second array is
// column indices).
dense_size:int;
array_segments:SparseIndexVector;
array_indices:SparseIndexVector;
}
// Parameters to encode a sparse TfLite tensor.
table SparsityParameters {
// The traversal order of the dimensions defined in the `shape` field of the
// conceptual dense tensor. For a n-dimensional tensors with dims (d0, d1,
// ..., dn-1),
// - if not block sparse, the traversal_order is just a permutation of (d0,
// ..., dn-1). For example, a 2-D matrix stored in row-major order would
// have traversal_order = (d0, d1).
// - if block sparse with a k-dimensional block (0 <= k <= n), the
// traversal_order has n + k elements. The first n elements are still a
// permutation of (d0, ..., dn-1). The lask k elements are a permutation
// of (dn, ..., dn+k-1), defining how to traverse a block internally. For
// example, a 2-D matrix with 2-D blocks, both stored in row-major order
// would have traversal_order = (d0, d1, d2, d3).
traversal_order:[int];
// For an n-dimensional tensor with a k-dimensional block (0 <= k <= n),
// stores how a block dimension in (dn, ..., dn+k-1) maps to the original
// tensor dimension in (d0, ..., dn).
// It's stored in the order of (dn, ..., dn+k-1).
// If not block-sparse, this field is NULL.
block_map:[int];
// In the traversal order defined above, the metadata needed for
// each dimension to locate the non-zero values in the original dense tensor.
// The size of the dim_metadata array = the size of the traversal_order array
// = n + k.
dim_metadata:[DimensionMetadata];
}
As you can see above, there is a buffer that contains sparse tensor's contents.
"buffers": [
{
},
{
"data": [
1, 0, 0, 4,
2, 3, 0, 0,
5, 0, 0, 6
]
}
]
AFAIK, if I want to generate a sparse tensor like above, I have to write a code like below.
st1 = tf.compat.v1.sparse.SparseTensor(
indices=[[0, 0], [0, 3], [1, 0], [1, 1], [2, 0], [2, 2]], values=buffers, dense_shape=[4, 4])
But, above json example don't match with my understanding at all.
I think array_indices should be [0, 3, 0, 1, 0, 3] rather than [0, 1, 1] and array_segements be [0, 2, 2, 4, 4, 6] rather than [0, 2, 3].
Moreover, actually, none of that comments in schema understand at all..
What does the metadata with DENSE format stand for?
{
"format": "DENSE",
"dense_size": 2
},
As shcema's comment said, it is a field to store the size of that dimension.
But, which dimension has got value "2"? The shape is [4, 4]. I can't even infer where the number 2 come from.
What is the "block"?
AFAIK, block is a box that contains non-zero value. But then, there are lots of blocks in above buffer I think.
If there is this block like,
1 2 0
3 4 0
0 0 0
I would say, it has one 2x2 block.
But there are six 1x1 blocks in above buffer.
Then, how can I make this things a block map..?
Actually, I don't know about traversal_order as well but if I knew the above things, I would understand it as well.
Please somebody help me..
Currently TFLite uses a different sparse tensor representation from Tensorflow. The format it uses is called TACO. Please refer to this paper for more details: http://tensor-compiler.org/kjolstad-oopsla17-tensor-compiler.pdf, section 3.
The tf.SparseTensor is not meant to work with this, since it uses the COO format.
A block is an inner sub-unit of a tensor that needs to be stored together. It can contain 0-valued element. The example flatbuffer shows a 2-D 4x4 tensor with a 2-D 2x2 inner block. TFLite uses blocked sparse tensor to take advantage of the NEON SIMD instructions.
I would like to simply place a text object under an image object I created using jsxgraph.
Tried setting for the text object layer:2 and the image object layer:7.
Tried setting board.options.layer.text=2, then created the text object, then set board.options.layer.text=9 again.
The object does give the correct layer value when investigating, however it does not visually do this. This works well for non text objects. Would this a text related bug?
JSXGraph distinguishes two types of texts: display:'html' and display:'internal'. The former type of text element uses an HTML tag and is always strictly "above" any JSXGraph construction (and thus ignores the layer structur). The latter type of text element obeys the layer structure. The default is display:'html'.
Here is an example (https://jsfiddle.net/8xms49pu/2/):
var circ = board.create('circle', [[0, 0], 3],
{fillColor: 'yellow', fillOpacity: 0.8});
var txt1 = board.create('text', [1, 1, 'HTML'], {layer: 1});
var txt1 = board.create('text', [-3.2, -1, 'internal'],
{layer: 1, display: 'internal'});
Images can be put into layers. This means, one also has to use display:'internal'. Here is an example (https://jsfiddle.net/jdw7z1nq/1/):
var im = board.create('image', ['https://jsxgraph.org/wp/img/logo_tw.png', [-3, -3], [6, 6]],
{layer: 10});
var circ = board.create('circle', [[0,0], 3],
{fillColor: 'yellow', layer: 6});
var txt = board.create('text', [1, 1, 'Hello'],
{layer: 5, display: 'internal'});
The advantage of HTML texts is that they can contain any kind HTML content, for example MathJax or form tags or images, ... So, you might also consider putting the image into a text element.
I want to construct an L2-norm layer in Caffe style (well, I actually want to use Tensorflow in a pycaffe layer, since using CUDA to write .cu files in Caffe is an onerous task.)
Forward pass:
- input(x): n-D array
- output(y): n-D array that has the same shape of input
- operation:
y = x / sqrt(sum(x^2,axis=(0,1))) # channel wise L2 normalization
class L2NormLayer:
def __init__(self):
self.eps = 1e-12
self.sess = tf.Session()
def forward(self, in_x):
self.x = tf.constant(in_x)
self.xp2 = tf.pow(self.x, 2)
self.sum_xp2 = tf.reduce_sum(self.xp2, axis=(0, 1))
self.sqrt_sum_xp2 = tf.sqrt(self.sum_xp2 + self.eps)
self.hat = tf.div(self.x, self.sqrt_sum_xp2)
return self.sess.run(self.hat)
def backward(self, dl):
# 'dl' is loss calculated at upper layer (chain rule)
# how do I calculate this gradient automatically using Tensorflow
# hand-craft backward version
loss = tf.constant(dl)
d_x1 = tf.div(loss, self.sqrt_sum_xp2)
d_sqrt_sum_xp2 = tf.div(-tf.reduce_sum(self.x * dl, axis=(0, 1)), (self.eps + tf.pow(self.sqrt_sum_xp2, 2)))
d_sum_xp2 = tf.div(d_sqrt_sum_xp2, (self.eps + 2 * tf.sqrt(self.sum_xp2)))
d_xp2 = tf.ones_like(self.xp2) * d_sum_xp2
d_x2 = 2 * self.x * d_xp2
d_x = d_x1 + d_x2
return self.sess.run(d_x)
As commented in the code, how can I calcualte the gradient of the forward pass function by using Tensorflow automatically?
I think your best strategy would be to use existing caffe layers to achieve your goal.
First, use "Reduction" layer to compute the sq. L2 norm of x:
layer {
name: "norm_x_sq"
type: "Reduction"
bottom: "x"
top: "norm_x_sq"
reduction_param { operation: SUMSQ axis: 1 }
}
Use "Power" layer to take the square root of the norm and compute its reciprocal:
layer {
name: "norm_x-1"
type: "Power"
bottom: "norm_x_sq"
top: "norm_x-1"
power_param { power: -0.5 }
}
Once you have the denominator, you need to "Tile" it back to the same shape as x:
layer {
name: "denom"
type: "Tile"
bottom: "norm_x-1"
top: "denom"
tile_param { axis:1 tiles: N } # here you'll have to manually put the target dimension N
}
Finally, use "Eltwise" layer to normalize x:
layer {
name: "x_norm"
type: "Eltwise"
bottom: "x"
bottom: "denom"
top: "x_norm"
eltwise_param { operation: PROD }
}
Some additional notes:
1. Dividing by the norm might be numerically unstable if the norm is very little. You might want to consider adding a tiny constant to "norm_x_sq" before taking the reciprocal of the square root. You can do that using existing layers as well.
2. This example showed how to normalize according to axis=1 dimension. Depending how your vectors are arranged in the blob, you might be able to use "Scale" layer for the division instead of tile+eltwise.
3. You might also find this thread useful.
I'm writing a custom Tensorflow op using the tutorial and I'm having trouble understanding how to read and write to/from Tensors.
let's say I have a Tensor in my OpKernel that I get from
const Tensor& values_tensor = context->input(0); (where context = OpKernelConstruction*)
if that Tensor has shape, say, [2, 10, 20], how can I index into it (e.g. auto x = values_tensor[1, 4, 12], etc.)?
equivalently, if I have
Tensor *output_tensor = NULL;
OP_REQUIRES_OK(context, context->allocate_output(
0,
{batch_size, value_len - window_size, window_size},
&output_tensor
));
how can I assign to output_tensor, like output_tensor[1, 2, 3] = 11, etc.?
sorry for the dumb question, but the docs are really tripping me up here and the examples in the Tensorflow kernel code for built-in ops somehow obfuscate this to the point that I get very confused :)
thank you!
The easiest way to read from and write to tensorflow::Tensor objects is to convert them to an Eigen tensor, using the tensorflow::Tensor::tensor<T, NDIMS>() method. Note that you have to specify the (C++) type of elements in tensor as template parameter T.
For example, to read a particular value from a DT_FLOAT32 tensor:
const Tensor& values_tensor = context->input(0);
auto x = value_tensor.tensor<float, 3>()(1, 4, 12);
To write a particular value to a DT_FLOAT32 tensor:
Tensor* output_tensor = ...;
output_tensor->tensor<float, 3>()(1, 2, 3) = 11.0;
There are also convenience methods for accessing a scalar, vector, or matrix.