I'm trying to visualize a tensor summary in tensorboard. However I can't see the tensor summary at all in the board. Here is my code:
out = tf.strided_slice(logits, begin=[self.args.uttWindowSize-1, 0], end=[-self.args.uttWindowSize+1, self.args.numClasses],
strides=[1, 1], name='softmax_truncated')
tf.summary.tensor_summary('softmax_input', out)
where out is a multi-dimensional tensor. I guess there must be something wrong with my code. Probably I used the tensor_summary function incorrectly.
What you do is you create a summary op, but you don't invoke it and don't write the summary (see documentation).
To actually create a summary you need to do the following:
# Create a summary operation
summary_op = tf.summary.tensor_summary('softmax_input', out)
# Create the summary
summary_str = sess.run(summary_op)
# Create a summary writer
writer = tf.train.SummaryWriter(...)
# Write the summary
writer.add_summary(summary_str)
Explicitly writing a summary (last two lines) is only necessary if you don't have a higher level helper like a Supervisor. Otherwise you invoke
sv.summary_computed(sess, summary_str)
and the Supervisor will handle it.
More info, also see:
How to manually create a tf.Summary()
Hopefully a workaround which achieves what you want. ..
If you wish to view the tensor values, you can convert them using as_string, then use summary.text. The values will appear in the tensorboard text tab.
Not tried with 3D tensors, but feel free to slice according to needs.
code snippet, which includes use of inserting a print statement to get console output as well.
predictions = tf.argmax(reshaped_logits, 1)
txtPredictions = tf.Print(tf.as_string(predictions),[tf.as_string(predictions)], message='predictions', name='txtPredictions')
txtPredictions_op = tf.summary.text('predictions', txtPredictions)
Not sure whether this is kinda obvious, but you could use something like
def make_tensor_summary(tensor, name='defaultTensorName'):
for i in range(tensor.get_shape()[0]:
for j in range(tensor.get_shape()[1]:
tf.summary.scalar(Name + str(i) + '_' + str(j), tensor[i, j])
in case you know it is a 'matrix-shaped' Tensor in advance.
Related
I have split my data into training and test, followed by the split of training into another train set and a validation set. To this new train set and validation set I have applied the below transformation. I am implementing a Random forest regression, so at the next step I apply the transformations to these set and try to combine it into one. The issue is np.vstack isn't returning me the correct shape:
Output:
(2, 1) <- should have been (25455, 2394)
(21636, 2394)
(3819, 2394)
Could someone one tell me what am I doing wrong?
Xtrain_rf, Xtest_rf = train_test_split(insurance_data_prep, test_size=0.15, random_state=42)
Xtrain2_rf, Xval_rf = train_test_split(Xtrain_rf, test_size=0.15, random_state=42)
full_transform_rf = ColumnTransformer([
("num", StandardScaler(), attributes_num),
("cat", OneHotEncoder(handle_unknown='ignore'), attributes_cat),
])
## fit transform in the train set
Xtrain2_rf_att_prepared = full_transform_rf.fit_transform(Xtrain2_rf_att)
## transform in the validation set
Xval_rf_att_prepared = full_transform_rf.transform(Xval_rf_att)
whole_train_set_attributes_rf = np.vstack((Xtrain2_rf_att_prepared, Xval_rf_att_prepared))
print(whole_train_set_attributes_rf.shape)
print(Xtrain2_rf_att_prepared.shape)
print(Xval_rf_att_prepared.shape)
I tried to replicate your code with a dataset of my own, and one problem I have ran into is that Xtrain2_rf_att and Xval_rf_att variables have not been defined in your code snippet. Those are the variables that you passed to your ColumnTransformer.
I guess that in your case the variables are defined but what you really want to pass to the ColumnTransformer are Xtrain2_rf and Xval_rf.
If this does not solve your problem, could you edit your question and add the following information?:
insurance_data_prep shape and columns (or some of them)
attributes_num and attributes_dat
clearly provide the outputs from the three prints statements at the end of your code and not just one of them
You might need to use a MRE to replace your dataframe, we will see
I am trying to build the BiseNet shown in the figure at "https://github.com/Blaizzy/BiSeNet-Implementation".
When I want to use the GlobalAveragePooling2D() in Keras(tf-backend) to finish the Attention Refined Module in Figure(b), I find the output shape of the GlobalAveragePooling2D() is not suitable for the next convolution.
I checked out many implementation of BiSeNet code in github, however, most of them use AveragePooling2D(size=(1,1)) instead. But AveragePooling2D(size=(1,1)) is completely non-sense.
So I define a lambda layer to do what I want (The selected code is shown as below). The lambda layer works but seems very ugly:
def samesize_globalAveragePooling2D(inputtensor):
# inputtensor shape:(?, 28,28,32)
x = GlobalAveragePooling2D()(inputtensor) # x shape:(?, 32)
divide = tf.divide(inputtensor, inputtensor) # divide shape:(?, 28,28,32)
x2 = x * divide # x2 shape:(?, 28,28,32)
global_pool = Lambda(function=samesize_globalAveragePooling2D)(conv_0)
Hope to get suggestion to make this lambda to be more graceful.
Thanks!
This could be done using a lambda layer on tf.reduce_mean.
tf.keras.layers.Lambda(lambda x: tf.reduce_mean(x, axis=[1, 2], keep_dims=True))
I have the following code to average embeddings for list of item-ids.
(Embedding is trained on review_meta_id_input, and used as look up for pirors_input and for getting average embedding)
review_meta_id_input = tf.keras.layers.Input(shape=(1,), dtype='int32', name='review_meta_id')
priors_input = tf.keras.layers.Input(shape=(None,), dtype='int32', name='priors') # array of ids
item_embedding_layer = tf.keras.layers.Embedding(
input_dim=100, # max number
output_dim=self.item_embedding_size,
name='item')
review_meta_id_embedding = item_embedding_layer(review_meta_id_input)
selected = tf.nn.embedding_lookup(review_meta_id_embedding, priors_input)
non_zero_count = tf.cast(tf.math.count_nonzero(priors_input, axis=1), tf.float32)
embedding_sum = tf.reduce_sum(selected, axis=1)
item_average = tf.math.divide(embedding_sum, non_zero_count)
I also have some feature columns such as..
(I just thought feature_column looked cool, but not many documents to look for..)
kid_youngest_month = feature_column.numeric_column("kid_youngest_month")
kid_age_youngest_buckets = feature_column.bucketized_column(kid_youngest_month, boundaries=[12, 24, 36, 72, 96])
I'd like to define [review_meta_id_iput, priors_input, (tensors from feature_columns)] as an input to keras Model.
something like:
inputs = [review_meta_id_input, priors_input] + feature_layer
model = tf.keras.models.Model(inputs=inputs, outputs=o)
In order to get tensors from feature columns, the closest lead I have now is
fc_to_tensor = {fc: input_layer(features, [fc]) for fc in feature_columns}
from https://github.com/tensorflow/tensorflow/issues/17170
However I'm not sure what the features are in the code.
There's no clear example on https://www.tensorflow.org/api_docs/python/tf/feature_column/input_layer either.
How should I construct the features variable for fc_to_tensor ?
Or is there a way to use keras.layers.Input and feature_column at the same time?
Or is there an alternative than tf.feature_column to do the bucketing as above? then I'll just drop the feature_column for now;
The behavior you desire could be achieved through following steps.
This works in TF 2.0.0-beta1, but may being changed or even simplified in further reseases.
Please check out issue in TensorFlow github repository Unable to use FeatureColumn with Keras Functional API #27416. There you will find the more general example and useful comments about tf.feature_column and Keras Functional API.
Meanwhile, based on the code in your question the input tensor for feature_column could be get like this:
# This you have defined feauture column
kid_youngest_month = feature_column.numeric_column("kid_youngest_month")
kid_age_youngest_buckets = feature_column.bucketized_column(kid_youngest_month, boundaries=[12, 24, 36, 72, 96])
# Then define layer
feature_layer = tf.keras.layers.DenseFeatures(kid_age_youngest_buckets)
# The inputs for DenseFeature layer should be define for each original feature column as dictionary, where
# keys - names of feature columns
# values - tf.keras.Input with shape =(1,), name='name_of_feature_column', dtype - actual type of original column
feature_layer_inputs = {}
feature_layer_inputs['kid_youngest_month'] = tf.keras.Input(shape=(1,), name='kid_youngest_month', dtype=tf.int8)
# Then you can collect inputs of other layers and feature_layer_inputs into one list
inputs=[review_meta_id_input, priors_input, [v for v in feature_layer_inputs.values()]]
# Then define outputs of this DenseFeature layer
feature_layer_outputs = feature_layer(feature_layer_inputs)
# And pass them into other layer like any other
x = tf.keras.layers.Dense(256, activation='relu')(feature_layer_outputs)
# Or maybe concatenate them with outputs from your others layers
combined = tf.keras.layers.concatenate([x, feature_layer_outputs])
#And probably you will finish with last output layer, maybe like this for calssification
o=tf.keras.layers.Dense(classes_number, activation='softmax', name='sequential_output')(combined)
#So you pass to the model:
model_combined = tf.keras.models.Model(inputs=[s_inputs, [v for v in feature_layer_inputs.values()]], outputs=o)
Also note. In model fit() method you should pass info which data sould be used for each input.
One way, if you use tf.data.Dataset, take care that you have used the same names for features in Dataset and for keys in feature_layer_inputs dictionary
Other way use explicite notation like:
model.fit({'review_meta_id_input': review_meta_id_data, 'priors_input': priors_data, 'kid_youngest_month': kid_youngest_month_data},
{'outputs': o},
...
)
I've captured some interesting operations, say "Conv2D"s, in a TF network. The goal is to experiment some data manipulation on the weight input "weights". Let's call the data manipulation function "white_box_func".
I've done some reading of the graph editor module. It seems that I can wrap my "white_box_func" in a "tf.py_func". But then, the original "weights" tensor needs to be routed through the "py_func(white_box_func, ...)", whose output needs to be properly connected to the "Conv2D".
<weights> --
\
===> Conv2D
/
<features> --
becomes
<weights> --> <py_func(white_box_func,...)--
\
===> Conv2D
/
<features> --------------------------------
Having a hard time figuring out a clean way of doing that. Any suggestion is appreciated!
I've figured this one out. The code below shows how to edit a quantized node:
#Assuming <p> is the node we want to edit
for i in list(p.inputs):
if tf.quint8 == i.dtype and 'weights' in i.name:
orig_rtype = i.op.outputs[0].dtype
new_rtype = tf.uint8 if tf.quint8 == orig_rtype else orig_rtype
with p.graph.as_default():
my_i = tf.py_func(my_shiny_func, i.op.outputs,
[new_rtype, tf.float32, tf.float32],
stateful=False, name=p.name + 'shiny')
my_i_cast = tf.bitcast(my_i[0], orig_rtype)
ge.connect(ge.sgv(my_i_cast.op, my_i[0].op),
ge.sgv(p).remap_inputs([1, 4, 5]))
Suppose now I have a training input pipeline which finally generate train_x and train_y using tf.train.shuffle_batch. I export meta graph and re-import the graph in another code file. Now I want to detach the input pipeline, i.e., the train_x and train_y, and connect a new test_x and test_y. How can I make accomplish this using tf.contrib.graph_editor?
EDIT: As suggested by #iga, I change my input directory using input_map
filenames = tf.train.match_filenames_once(FLAGS.data_dir + '*', name='matching_filenames')
if FLAGS.ckpt != '':
latest = FLAGS.log_dir + FLAGS.ckpt
else:
latest = tf.train.latest_checkpoint(FLAGS.log_dir)
if not latest or not os.path.exists(latest+'.meta'):
print("checkpoint " + latest + " does not exist")
sys.exit(1)
saver = tf.train.import_meta_graph(latest+'.meta',
input_map={'matching_filenames:0':filenames},
import_scope='import')
g = tf.get_default_graph()
but I get the following error:
ValueError: graph_def is invalid at node u'matching_filenames/Assign':
Input tensor 'matching_filenames:0' Cannot convert a tensor of type
string to an input of type string_ref.
Are there any elegant way to resolve this?
For this task, you should be able to just use input_map argument to https://www.tensorflow.org/api_docs/python/tf/import_graph_def. If you are using import_meta_graph, you can pass the input_map into its kwargs and it will get passed down to import_graph_def.
RESPONSE TO EDIT: I am assuming that your original graph (the one you are deserializing) had the same matching_filenames variable. Quite confusingly, the tensor name "matching_filenames:0" actually refers to the tensor going from the VariableV2 op to the Assign op. The type of this edge is string_ref and you don't really want to break that edge.
The output from a variable typically goes through an identity op called matching_filenames/read. This is what you want to use as the key in your input_map. For the value, you want the same tensor in your new filenames. So, your call should probably look like:
tf.train.import_meta_graph(latest+'.meta',
input_map={'matching_filenames/read': filenames.read_value()},
import_scope='import')
In general, variables are fairly complicated. If this does not work, you can use some placeholder op and feed the names into it manually.