I want to create a Tensor with as uppertriangular part the values from a vector. I have found in MATLAB this can be done with
a = [1 2 3 4 5 6 7 8 9 10];
b = triu(ones(5),1);
b = b'
b(b==1) = a
b = b'
My tensorflow implementation so far
b = tf.matrix_band_part(tf.ones([dim,dim]), 0, -1) # make upper triangular part 1
b = tf.transpose(b)
...
b = tf.transpose(b)
Who can help me?
I haven't seen a fantastic way to do it, but it's certainly possible. Here's one way (expanding the last dimension of a tensor into a matrix; the preceding dimensions may be batch dimensions):
import tensorflow as tf
def matrix_with_upper_values(upper_values):
# Check that the input is at least a vector
upper_values = tf.convert_to_tensor(upper_values)
upper_values.get_shape().with_rank_at_least(1)
# Put the batch dimensions last
upper_values = tf.transpose(
upper_values,
tf.concat(0, [[tf.rank(upper_values) - 1],
tf.range(tf.rank(upper_values) - 1)]))
input_shape = tf.shape(upper_values)[0]
# Compute the size of the matrix that would have this upper triangle
matrix_size = (1 + tf.cast(tf.sqrt(tf.cast(input_shape * 8 + 1, tf.float32)),
tf.int32)) // 2
# Check that the upper triangle size is valid
check_size_op = tf.Assert(
tf.equal(matrix_size ** 2, input_shape * 2 + matrix_size),
["Not a valid upper triangle size: ", input_shape])
with tf.control_dependencies([check_size_op]):
matrix_size = tf.identity(matrix_size)
# Compute indices for the whole matrix and the upper diagonal
index_matrix = tf.reshape(tf.range(matrix_size ** 2),
[matrix_size, matrix_size])
diagonal_indicies = (matrix_size * tf.range(matrix_size)
+ tf.range(matrix_size))
upper_triangular_indices, _ = tf.unique(tf.reshape(
tf.matrix_band_part(
index_matrix, 0, -1) # upper triangular part
- tf.diag(diagonal_indicies), # remove diagonal
[-1]))
batch_dimensions = tf.shape(upper_values)[1:]
return_shape_transposed = tf.concat(0, [[matrix_size, matrix_size],
batch_dimensions])
# Fill everything else with zeros; later entries get priority
# in dynamic_stitch
result_transposed = tf.reshape(
tf.dynamic_stitch(
[index_matrix,
upper_triangular_indices[1:]], # discard 0
[tf.zeros(return_shape_transposed, dtype=upper_values.dtype),
upper_values]),
return_shape_transposed)
# Transpose the batch dimensions to be first again
return tf.transpose(
result_transposed,
tf.concat(0, [tf.range(2, tf.rank(upper_values) + 1), [0, 1]]))
with tf.Session():
print(matrix_with_upper_values([1]).eval())
print(matrix_with_upper_values([2,7,1]).eval())
print(matrix_with_upper_values([3,1,4,1,5,9]).eval())
print(matrix_with_upper_values([]).eval())
print(matrix_with_upper_values([[2,7,1],[4,3,5]]).eval())
print(matrix_with_upper_values(tf.zeros([0, 3])).eval())
Prints:
[[0 1]
[0 0]]
[[0 2 7]
[0 0 1]
[0 0 0]]
[[0 3 1 4]
[0 0 1 5]
[0 0 0 9]
[0 0 0 0]]
[[ 0.]]
[[[0 2 7]
[0 0 1]
[0 0 0]]
[[0 4 3]
[0 0 5]
[0 0 0]]]
[]
Related
In Gurobi and JuMP 0.21, it is well documented here on how you would access a variable with a callback:
using JuMP, Gurobi, Test
model = direct_model(Gurobi.Optimizer())
#variable(model, 0 <= x <= 2.5, Int)
#variable(model, 0 <= y <= 2.5, Int)
#objective(model, Max, y)
cb_calls = Cint[]
function my_callback_function(cb_data, cb_where::Cint)
# You can reference variables outside the function as normal
push!(cb_calls, cb_where)
# You can select where the callback is run
if cb_where != GRB_CB_MIPSOL && cb_where != GRB_CB_MIPNODE
return
end
# You can query a callback attribute using GRBcbget
if cb_where == GRB_CB_MIPNODE
resultP = Ref{Cint}()
GRBcbget(cb_data, cb_where, GRB_CB_MIPNODE_STATUS, resultP)
if resultP[] != GRB_OPTIMAL
return # Solution is something other than optimal.
end
end
# Before querying `callback_value`, you must call:
Gurobi.load_callback_variable_primal(cb_data, cb_where)
x_val = callback_value(cb_data, x)
y_val = callback_value(cb_data, y)
# You can submit solver-independent MathOptInterface attributes such as
# lazy constraints, user-cuts, and heuristic solutions.
if y_val - x_val > 1 + 1e-6
con = #build_constraint(y - x <= 1)
MOI.submit(model, MOI.LazyConstraint(cb_data), con)
elseif y_val + x_val > 3 + 1e-6
con = #build_constraint(y + x <= 3)
MOI.submit(model, MOI.LazyConstraint(cb_data), con)
end
if rand() < 0.1
# You can terminate the callback as follows:
GRBterminate(backend(model))
end
return
end
# You _must_ set this parameter if using lazy constraints.
MOI.set(model, MOI.RawParameter("LazyConstraints"), 1)
MOI.set(model, Gurobi.CallbackFunction(), my_callback_function)
optimize!(model)
#test termination_status(model) == MOI.OPTIMAL
#test primal_status(model) == MOI.FEASIBLE_POINT
#test value(x) == 1
#test value(y) == 2
i.e., you would use x_val = callback_value(cb_data, x). However, how should you do when you have an array of variables with specific indexes not starting at 1, i.e. my variables are not in a vector but declared thanks to:
#variable(m, x[i=1:n, j=i+1:n], Bin)
Should I access x with double for loops on its two dimensions and call multiple times callback_value? If so, the indexes for j will not be the same, won't they?
Use broadcasting:
x_val = callback_value.(Ref(cb_data), x)
Or just call callback_value(cb_data, x[i, j]) when you need the value.
For example:
using JuMP, Gurobi
model = Model(Gurobi.Optimizer)
#variable(model, 0 <= x[i=1:3, j=i+1:3] <= 2.5, Int)
function my_callback_function(cb_data)
x_val = callback_value.(Ref(cb_data), x)
display(x_val)
for i=1:3, j=i+1:3
con = #build_constraint(x[i, j] <= floor(Int, x_val[i, j]))
MOI.submit(model, MOI.LazyConstraint(cb_data), con)
end
end
MOI.set(model, MOI.LazyConstraintCallback(), my_callback_function)
optimize!(model)
yields
julia> optimize!(model)
Gurobi Optimizer version 9.1.0 build v9.1.0rc0 (mac64)
Thread count: 4 physical cores, 8 logical processors, using up to 8 threads
Optimize a model with 0 rows, 3 columns and 0 nonzeros
Model fingerprint: 0x5d543c3a
Variable types: 0 continuous, 3 integer (0 binary)
Coefficient statistics:
Matrix range [0e+00, 0e+00]
Objective range [0e+00, 0e+00]
Bounds range [2e+00, 2e+00]
RHS range [0e+00, 0e+00]
JuMP.Containers.SparseAxisArray{Float64,2,Tuple{Int64,Int64}} with 3 entries:
[1, 2] = -0.0
[2, 3] = -0.0
[1, 3] = -0.0
JuMP.Containers.SparseAxisArray{Float64,2,Tuple{Int64,Int64}} with 3 entries:
[1, 2] = 2.0
[2, 3] = 2.0
[1, 3] = 2.0
JuMP.Containers.SparseAxisArray{Float64,2,Tuple{Int64,Int64}} with 3 entries:
[1, 2] = 2.0
[2, 3] = 2.0
[1, 3] = 2.0
JuMP.Containers.SparseAxisArray{Float64,2,Tuple{Int64,Int64}} with 3 entries:
[1, 2] = 2.0
[2, 3] = -0.0
[1, 3] = -0.0
Presolve time: 0.00s
Presolved: 0 rows, 3 columns, 0 nonzeros
Variable types: 0 continuous, 3 integer (0 binary)
JuMP.Containers.SparseAxisArray{Float64,2,Tuple{Int64,Int64}} with 3 entries:
[1, 2] = -0.0
[2, 3] = -0.0
[1, 3] = -0.0
Found heuristic solution: objective 0.0000000
Explored 0 nodes (0 simplex iterations) in 0.14 seconds
Thread count was 8 (of 8 available processors)
Solution count 1: 0
Optimal solution found (tolerance 1.00e-04)
Best objective 0.000000000000e+00, best bound 0.000000000000e+00, gap 0.0000%
User-callback calls 31, time in user-callback 0.14 sec
My PyTorch model outputs a segmented image with values (0,1,2) for each one of the three classes. During the preparation of the set, I mapped black to 0, red to 1 and white to 2. I have two questions:
How can I show what each class represents? for example take a look at the image:
I am currently using the following method to show each class:
output = net(input)
input = input.cpu().squeeze()
input = transforms.ToPILImage()(input)
probs = F.softmax(output, dim=1)
probs = probs.squeeze(0)
full_mask = probs.squeeze().cpu().numpy()
fig, (ax0, ax1, ax2, ax3, ax4) = plt.subplots(1, 5, figsize=(20,10), sharey=True)
ax0.set_title('Input Image')
ax1.set_title('Background Class')
ax2.set_title('Neuron Class')
ax3.set_title('Dendrite Class')
ax4.set_title('Predicted Mask')
ax0.imshow(input)
ax1.imshow(full_mask[0, :, :].squeeze())
ax2.imshow(full_mask[1, :, :].squeeze())
ax3.imshow(full_mask[2, :, :].squeeze())
full_mask = np.argmax(full_mask, 0)
img = mask_to_image(full_mask)
But there appears to be shared pixels between the classes, is there a better way to show this (I want the first image to only of the background class, the the second only of the neuron class and the third only of the dendrite class)?
2.My second question is about generating a black, red and white image from the mask, currently the mask is of shape (512,512) and has the following values:
[[0 0 0 ... 0 0 0]
[0 0 0 ... 2 0 0]
[0 0 0 ... 2 2 0]
...
[2 1 2 ... 2 2 2]
[2 1 2 ... 2 2 2]
[0 2 0 ... 2 2 2]]
And the results look like this:
Since I am using this code to convert to image:
def mask_to_image(mask):
return Image.fromarray((mask).astype(np.uint8))
But there appears to be shared pixels between the classes, is there a
better way to show this (I want the first image to only of the
background class, the the second only of the neuron class and the
third only of the dendrite class)?
Yes, you can take argmax along 0th dimension so the one with highest logit (unnormalized probability) will be 1, rest will be zero:
output = net(input)
binary_mask = torch.argmax(output, dim=0).cpu().numpy()
ax.set_title('Neuron Class')
ax.imshow(binary_mask == 0)
My second question is about generating a black, red and white image
from the mask, currently the mask is of shape (512,512) and has the
following values
You can spread [0, 1, 2] values into the zero-th axis making it channel-wise. Now [0, 0, 0] values across all channels for single pixel will be black, [255, 255, 255] would be white and [255, 0, 0] would be red (as PIL is RGB format):
import torch
tensor = torch.randint(high=3, size=(512, 512))
red = tensor == 0
white = tensor == 2
zero_channel = red & white
image = torch.stack([zero_channel, white, white]).int().numpy() * 255
Image.fromarray((image).astype(np.uint8))
For the dense tensor, we can use tf.nn.topk to find values and indices of the k largest entries for the last dimension.
For the sparse tensor, I would like to efficiently get the top n items of each row, without converting the sparse tensor to dense.
This was kind of tricky but here is something that works (assumes 2D sparse tensor, although I think should work the same for more outer dimensions). The idea is to first sort the whole sparse tensor (without making it dense) and then slice the first columns. To do that, I needed something like np.lexsort, which as far as I know is not provided in TensorFlow as such - however, tf.sparse.reorder actually does something like a lexsort, so I made another intermediate sparse tensor to take advantage of that.
import tensorflow as tf
import numpy as np
np.random.seed(0)
# Input data
k = 3
r = np.random.randint(10, size=(6, 8))
r[np.random.rand(*r.shape) < .5] = 0
sp = tf.sparse.from_dense(r)
print(tf.sparse.to_dense(sp).numpy())
# [[0 0 0 0 0 0 3 0]
# [2 4 0 6 8 0 0 6]
# [7 0 0 1 5 9 8 9]
# [4 0 0 3 0 0 0 3]
# [8 1 0 3 3 7 0 1]
# [0 0 0 0 7 0 0 7]]
# List of value indices
n = tf.size(sp.values, out_type=sp.indices.dtype)
r = tf.range(n)
# Sort values
s = tf.dtypes.cast(tf.argsort(sp.values, direction='DESCENDING'), sp.indices.dtype)
# Find destination index of each sorted value
si = tf.scatter_nd(tf.expand_dims(s, 1), r, [n])
# Abuse sparse tensor functionality to do lexsort with column and destination index
sp2 = tf.sparse.SparseTensor(indices=tf.stack([sp.indices[:, 0], si], axis=1),
values=r,
dense_shape=[sp.dense_shape[0], n])
sp2 = tf.sparse.reorder(sp2)
# Build top-k result
row = sp.indices[:, 0]
# Make column indices
d = tf.dtypes.cast(row[1:] - row[:-1] > 0, r.dtype)
m = tf.pad(r[1:] * d, [[1, 0]])
col = r - tf.scan(tf.math.maximum, m)
# Get only up to k elements per row
m = col < k
row_m = tf.boolean_mask(row, m)
col_m = tf.boolean_mask(col, m)
idx_m = tf.boolean_mask(sp2.values, m)
# Make result
scatter_idx = tf.stack([row_m, col_m], axis=-1)
scatter_shape = [sp.dense_shape[0], k]
# Use -1 for rows with less than k values
# (0 is ambiguous)
values = tf.tensor_scatter_nd_update(-tf.ones(scatter_shape, sp.values.dtype),
scatter_idx, tf.gather(sp.values, idx_m))
indices = tf.tensor_scatter_nd_update(-tf.ones(scatter_shape, sp.indices.dtype),
scatter_idx, tf.gather(sp.indices[:, 1], idx_m))
print(values.numpy())
# [[ 3 -1 -1]
# [ 8 6 6]
# [ 9 9 8]
# [ 4 3 3]
# [ 8 7 3]
# [ 7 7 -1]]
print(indices.numpy())
# [[ 6 -1 -1]
# [ 4 3 7]
# [ 5 7 6]
# [ 0 3 7]
# [ 0 5 3]
# [ 4 7 -1]]
EDIT: Here is an alternative possibility, which may work well if your tensor is very sparse in all rows. The idea is to "condense" all the sparse tensor values into the first columns (like the previous snippet already did for sp3) and then make that into a dense tensor and apply top-k as usual. The caveat is that the indices would be referred to the condensed tensor, so you have to take yet another step if you want to get the right indices with respect to initial sparse tensor.
import tensorflow as tf
import numpy as np
np.random.seed(0)
# Input data
k = 3
r = np.random.randint(10, size=(6, 8))
r[np.random.rand(*r.shape) < .8] = 0
sp = tf.sparse.from_dense(r)
print(tf.sparse.to_dense(sp).numpy())
# [[0 0 0 0 0 0 3 0]
# [0 4 0 6 0 0 0 0]
# [0 0 0 0 5 0 0 9]
# [0 0 0 0 0 0 0 0]
# [8 0 0 0 0 7 0 0]
# [0 0 0 0 7 0 0 0]]
# Build "condensed" sparse tensor
n = tf.size(sp.values, out_type=sp.indices.dtype)
r = tf.range(n)
# Make indices
row = sp.indices[:, 0]
d = tf.dtypes.cast(row[1:] - row[:-1] > 0, r.dtype)
m = tf.pad(r[1:] * d, [[1, 0]])
col = r - tf.scan(tf.math.maximum, m)
# At least as many columns as k
ncols = tf.maximum(tf.math.reduce_max(col) + 1, k)
sp2 = tf.sparse.SparseTensor(indices=tf.stack([row, col], axis=1),
values=sp.values,
dense_shape=[sp.dense_shape[0], ncols])
# Get in dense form
condensed = tf.sparse.to_dense(sp2)
# Top-k (indices do not correspond to initial sparse matrix)
values, indices = tf.math.top_k(condensed, k)
print(values.numpy())
# [[3 0 0]
# [6 4 0]
# [9 5 0]
# [0 0 0]
# [8 7 0]
# [7 0 0]]
# Now get the right indices
sp3 = tf.sparse.SparseTensor(indices=tf.stack([row, col], axis=1),
values=sp.indices[:, 1],
dense_shape=[sp.dense_shape[0], ncols])
condensed_idx = tf.sparse.to_dense(sp3)
actual_indices = tf.gather_nd(condensed_idx, tf.expand_dims(indices, axis=-1),
batch_dims=1)
print(actual_indices.numpy())
# [[6 0 0]
# [3 1 0]
# [7 4 0]
# [0 0 0]
# [0 5 0]
# [4 0 0]]
Not sure whether this would be faster or not though.
I have been experimenting with tensorflow Datasets but I cannot figure out how to efficiently create RLE-masks.
FYI, I am using data from the Airbus Ship Detection Challenge in Kaggle: https://www.kaggle.com/c/airbus-ship-detection/data
I know my RLE-decoding function works (borrowed) from one of the kernels:
def rle_decode(mask_rle, shape=(768, 768)):
'''
mask_rle: run-length as string formated (start length)
shape: (height,width) of array to return
Returns numpy array, 1 - mask, 0 - background
'''
if not isinstance(mask_rle, str):
img = np.zeros(shape[0]*shape[1], dtype=np.uint8)
return img.reshape(shape).T
s = mask_rle.split()
starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]
starts -= 1
ends = starts + lengths
img = np.zeros(shape[0]*shape[1], dtype=np.uint8)
for lo, hi in zip(starts, ends):
img[lo:hi] = 1
return img.reshape(shape).T
.... BUT it does not seem to play nicely with the pipeline:
list_ds = tf.data.Dataset.list_files(train_paths_abs)
ds = list_ds.map(parse_img)
With the following parse function, everything works fine:
def parse_img(file_path,new_size=[128,128]):
img_content = tf.io.read_file(file_path)
img = tf.image.decode_jpeg(img_content)
img = tf.image.convert_image_dtype(img, tf.float32)
img = tf.image.resize(img,new_size)
return img
But things go rogue if I include the mask:
def parse_img(file_path,new_size=[128,128]):
# Image
img_content = tf.io.read_file(file_path)
img = tf.image.decode_jpeg(img_content)
img = tf.image.convert_image_dtype(img, tf.float32)
img = tf.image.resize(img,new_size)
# Mask
file_id = tf.strings.split(file_path,'/')[-1]
objects = [rle_decode(m) for m in df2[df.ImageId==file_id]]
mask = np.sum(objects,axis=0)
mask = np.expand_dims(mask,3) # Force mask to have 3 channels, necessary for resize step
mask = tf.image.convert_image_dtype(mask, tf.int8)
mask = tf.clip_by_value(mask,0,1)
mask = tf.image.resize(mask,new_size)
mask = tf.squeeze(mask) # squeeze back
mask = tf.image.convert_image_dtype(mask, tf.int8)
return img, mask
Although my parse_img function works fine (I have checked it on a sample, it takes 271 µs ± 67.9 µs per run); the list_ds.map step takes forever (>5 minutes) before hanging.
I can't figure out what's wrong and it drives me crazy!
Any idea?
You can rewrite the function rle_decode with tensorflow like this (here I do not do the final transposition to keep it more general, but you can do it later):
import tensorflow as tf
def rle_decode_tf(mask_rle, shape):
shape = tf.convert_to_tensor(shape, tf.int64)
size = tf.math.reduce_prod(shape)
# Split string
s = tf.strings.split(mask_rle)
s = tf.strings.to_number(s, tf.int64)
# Get starts and lengths
starts = s[::2] - 1
lens = s[1::2]
# Make ones to be scattered
total_ones = tf.reduce_sum(lens)
ones = tf.ones([total_ones], tf.uint8)
# Make scattering indices
r = tf.range(total_ones)
lens_cum = tf.math.cumsum(lens)
s = tf.searchsorted(lens_cum, r, 'right')
idx = r + tf.gather(starts - tf.pad(lens_cum[:-1], [(1, 0)]), s)
# Scatter ones into flattened mask
mask_flat = tf.scatter_nd(tf.expand_dims(idx, 1), ones, [size])
# Reshape into mask
return tf.reshape(mask_flat, shape)
A small test (TensorFlow 2.0):
mask_rle = '1 2 4 3 9 4 15 5'
shape = [4, 6]
# Original NumPy function
print(rle_decode(mask_rle, shape))
# [[1 0 0 1]
# [1 0 0 0]
# [0 1 1 0]
# [1 1 1 0]
# [1 1 1 0]
# [1 1 1 0]]
# TensorFlow function (transposing is done out of the function)
tf.print(tf.transpose(rle_decode_tf(mask_rle, shape)))
# [[1 0 0 1]
# [1 0 0 0]
# [0 1 1 0]
# [1 1 1 0]
# [1 1 1 0]
# [1 1 1 0]]
Given a tensor of rank 1 eg. p = [x y z w], how can I "min-max clamp" within the provided boundaries: max = [1 10 5 3] and min = [-1 -10 -5 -3] such that the i-th element in p is always within the boundaries defined by mini and maxi
Extra: Would it be possible to do this for ranks > 1?
I found the following solution adequate. See the documentation for tf.minimum and tf.maximum. Solution:
import tensorflow as tf
p = tf.Variable([-1, 1, 3, 7])
clamp_min = tf.Variable([1, 1, 1, 1])
clamp_max = tf.Variable([5, 5, 5, 5])
p = tf.minimum(p, clamp_max)
p = tf.maximum(p, clamp_min)
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)
print(sess.run(p))
Produces:
[1 1 3 5]