I've trained two models. Each of them can produce useful representative features. And I want to use the features to train the third model. That means the first two models can produce the inputs to the third model. And I want to use the tf.contrib.learn.train() to train the third model. The question is how should I import the two checkpoint files to the first two models to init the weights. I know there's init_fn and init_op parameters in the function. However, I don't know how to use more than one checkpoint files. Can you help me out? Thank you.
you can use tf.contrib.framework.init_from_checkpoint.
Related
I am trying to enforce a structural prior on the two outputs of my network.
The network has two heads and each predicts a different quantity: f_1(x) and f_2(x) (and each branch has its own separate loss function).
I am basically trying to add a third loss that is a function of only the outputs: L(f_1(x),f_2(x)).
I know this can easily be done if I had a custom TF loop, but any idea how to accomplish this in Keras? (I am using tf.dataset to feed the input data).
Thank you
Corresponding Tensorflow documentation I trained 3 objects and get result (It can recognize these objects). When I show other objects (not the 3 ones) it doesn't work correctly.
I want to train only one object (example: a cup) and recognize only this object. Is it possible to do via Tensorflow ?
Your question doesn't provide enough details, but as I can guess your trained the network with softmax activation and Categorical or SparseCategorical cross entropy loss. If my guess is right, such network always generates prediction to one of three classess, regardless to actual data, i.e. there is no option of "no-one".
In order to train network to recognize only one class of objects, make the only one output with only one channel and sigmoid activation. Use BinaryCrossEntropy loss to train your model for the specific object. Provide dataset that includes examples with this object and without it.
Since all vars/graph are loaded anyways, why am I required to provide inputs, outputs to tf.saved_model.simple_save()?
I tried loading a variable with get_tensor_by_name() that I didn't specify in inputs/outputs dictionaries and it worked! So why won't it let me have blank/None inputs/outputs and I grab my variables by their names?
When you specify the input and output tensors of your model, the inference graph is fully specified. Imagine a model that has a single input, but two outputs . For instance, the model predicts the temperature for tomorrow and whether it will rain or not. Maybe I want to save an inference graph for a model that only gives me the temperature.
When you specify the ins and outs, TensorFlow knows which layers connect them. The reason why get_tensor_by_name() worked in your case, is probably because you fetched a layer that connects your inputs to your outputs.
I am using tensorflow serving to serve a pre-trained model.
The strange thing is when I input same data for this model, I got different outcome each time.
I thought it might be my problem at variable initialize, I am wondering is there any clue I debug my model, or how can I find the cause, thanks.
Two common problems:
There's a known issue with main_op in which variables are re-initialized to random.
You left dropout layers in your prediction graph.
To address (1), use this instead:
def main_op():
init_local = variables.local_variables_initializer()
init_tables = lookup_ops.tables_initializer()
return control_flow_ops.group(init_local, init_tables)
To address (2), be sure that you aren't directly exporting your training graph. You need to build a new graph for prediction/serving. If you are using the tf.estimator framework, then you will only conditionally add dropout layers when mode is tf.estimator.ModeKeys.TRAIN.
I have some questions on the use of the embedding columns implemented in tensorflow in tf.contrib.layers.embedding_column.
I'm training a binary classifier for a two player game (tennis, in this case). My features that I pass to the DNNClassifier as feature_columns look as follows:
deep_columns = [
tf.contrib.layers.embedding_column(player1, dimension=9),
tf.contrib.layers.embedding_column(player2, dimension=9),
tf.contrib.layers.embedding_column(court, dimension=1),
tf.contrib.layers.embedding_column(surface, dimension=1),
p1Rank, p2Rank]
What I'm wondering about is this: am I now learning two different embeddings for the same set of players? And if so, is there a way to use one embedding layer for both players? Or is there nothing wrong with doing it the way I'm currently doing it?
A second question regarding the embedding_column: the docs mention this, as possible arguments for embedding_column:
ckpt_to_load_from: (Optional). String representing checkpoint name/pattern to restore the column weights. Required if tensor_name_in_ckpt is not None.
tensor_name_in_ckpt: (Optional). Name of the Tensor in the provided checkpoint from which to restore the column weights. Required if ckpt_to_load_from is not None.
Does this imply that if none of these is provided, the embedding layers are initialized randomly again when restoring my model from a checkpoint?
And then one final question: The two embedding columns for court and surface have a dimension of 1, as they only have very few options. Is this a bad use of an embedding column? Or is it okay to use it like that?
Thanks in advance!