I have 3 models: model1, model2, and model3.
Now depending on the input x we will either use the output of model1 or model2 and pass it to model3.
I want to serialize this functionality in tensorflow but unable to find the right way to do it.
Is there a way to create a single pipeline for this use-case?
Can someone please help me here?
Related
I am trying to train NN with 3 heads sharing some initial layers. However each of my training targets has only output for 2 of them.
I would like to create separate batches with samples that contains output only for the same heads and use them to update only respective heads.
Is there any way how to achieve this in any DL framework?
As your question is somewhat general, I will answer assuming you are using PyTorchLightning.
I suggest you use a model that looks like this:
class MyModel(LightningModule):
def training_step(self, batch: MyMultiTaskBatch):
backbone_output = self.backbone(batch.x)
head = self.heads[batch.task_name]
head_output = head(backbone_output)
loss = self.losses[batch.task_name]
return loss(head_output, batch.y)
Where your batch tells the model which head it should run, and which loss it should use out of dictionaries that map task names to heads and losses. You will also need to implement a dataloader that returns a MyMultiTaskBatch as its batches.
I'm currently asking myself how to build a model with a couple of extra functions.
I got an entity of custom functions, and I want to embed them as layers into my model (NN).
For that I'm using TF 2.0. but I'm currently struggling to do that.
All I find is answers about activation functions, but that's not what I'm looking for.
A custom function returns something like a+b or any other algorithm (matrix multiplication etc.)
What we can say is, I have one layer to another one, and want to embed my custom function in between those two layers like so:
I'm going to say that the activation function from one layer to another is the custom function. But what if my custom function takes two inputs? Or I have two functions I want to process my input in before I pass it to the next function?
Another way to solve that problem:
Let's say I got my custom functions cm*, and my layers l*;
what I do is build a model for each layer I want to put in between two custom functions
cm1 -> model(l1) -> cm2 -> model(l2,l3) -> cm3 -> cm4 -> model(l4) -> ....
but wouldn't it be stupid to build a model for each of those trajectories?
And what about the loss? The back propagation of residual connected layers is something else than having a lot of models and functions layered together.
Or am I wrong?
I'm not sure about TF 2.0, but in Keras you can build your own custom layers that can receive multiple inputs by overriding the Layer class. See https://keras.io/guides/making_new_layers_and_models_via_subclassing/ for more details. The link doesn't explain how to pass in multiple inputs to a layer, but all you have to do is to call the layer with a list of inputs and unpack them inside the call function, something like this:
class MyCustomLayer(tf.keras.Layer):
def __init__(self):
# your code here
pass
def call(self, inputs): # example call: MyCustomLayer()([1, 2])
x, y = inputs
# your code here
output = x + y # placeholder
return output
I'm constructing several complex neural networks now using tensorflow. They share some variables and come neural network's output is another neural network's input. Sometimes I just get confused of which variables I should pass in to tf.sess.run()? Cause the structure is not very easy to find.
Lets say you have two models (model1 and model2) and model1 output is input for model2 then model2 will come in sess.run().
In sess.run() that model will come which is giving you last prediction.
So I frequently run models with different architectures, but have code intended to apply to all of them which runs inference off the saved models. Thus, I will be calling eval() on the last layer of this model, like this:
yhat = graph.get_tensor_by_name("name_of_my_last_layer:0")
decoded_image = yhat.eval(session=sess, feed_dict={x : X})
However, without arduous log parsing, I don't know exactly what the last layer is named, and I'm currently hand-coding it. I've considered creating a generic 'output' tensor in my graph but that seems wasteful/brittle. What is the better way?
The best way is to either making the layer you want to analyse a model output or to fix its name (by passing the name= keyword argument to the layer function when creating the layer) to be a known string.
I've trained two models. Each of them can produce useful representative features. And I want to use the features to train the third model. That means the first two models can produce the inputs to the third model. And I want to use the tf.contrib.learn.train() to train the third model. The question is how should I import the two checkpoint files to the first two models to init the weights. I know there's init_fn and init_op parameters in the function. However, I don't know how to use more than one checkpoint files. Can you help me out? Thank you.
you can use tf.contrib.framework.init_from_checkpoint.