I want to use the VGG converted tensorflow model from Ryan.
https://github.com/ry/tensorflow-vgg16
Now I want to adjust the layers and add another layer or change the fully connected layers. But I don't know how to get the single layers/weights out of the graphDef or how to adjust the graph.
Short answer: you can't adjust a graph, but there are probably ways to get what you want accomplished.
Long answer: TensorFlow Graph objects are structurally immutable. You can modify some aspects of them (e.g., the shape of a tensor flowing into a node), but you can't remove a node or add a node between two existing nodes. However, there are a couple ways to get the same effect:
If your changes are limited to additions only, then there's no problem with doing this. For instance, if you wanted to add a layer on the end of a network, go for it. Likewise, you can "replace" the last layer by simply adding a new layer which takes the second-to-last layer as input and just ignoring the existing last layer. When you run the graph, if you never ask for the output of the original last layer, TensorFlow will never compute it.
If you need to do modifications, one way is to slowly build up a copy of the graph node by node. So read in the original graph definition, then build your own new graph by iterating over the original and adding similar nodes to your new copy. This is somewhat tedious and can be error-prone. Moreover...
...You might not need to "adjust" the graph at all. If you want something similar to that VGG-16 implementation, you can just work off the python code directly. Don't like the width of fc6? Just edit the code that generates it.
This brings us to the real issue, though. If your goal is to modify the network and be able to re-use the weights, then 2. and 3. aren't going to work. Realistically, this isn't possible in a lot of cases. For instance, if I wanted to add or remove a layer in the middle of VGG-16 (say, adding another convolutional layer), the pre-trained weights are no longer valid. You might be able to salvage any pre-trained weights which are upstream of your changes, but everything downstream will basically be wrong. You'll need to retrain the network anyways. (Maybe you can use the pre-trained networks as initialization, but you'll still need to retrain.) Even if you're just adding to the network (as in 1.), you'll still need to train the network.
Thanks! I have recreated the graph and then loaded every single weight by getting the value of the graph definition.
This was done by graph.get_tensor_by_name('import/...') where ... is the name of the weight
https://www.tensorflow.org/versions/r0.9/how_tos/tool_developers/index.html
Related
I'm training a neural network using keras but I'm not sure how to feed the training data into the model in the way that I want.
My training data set is effectively infinite, I have some code to generate training examples as needed, so I just want to pipe a continuous stream of novel data into the network. keras seems to want me to specify my entire dataset in advance by creating a numpy array with everything in it, but this obviously wont work with my approach.
I've experimented with creating a generator class based on keras.utils.Sequence which seems like a better fit, but it still requires me to specify a length via the __len__ method which makes me think it will only create that many examples before recycling them. Can someone suggest a better approach?
Im currenty working on a project at University, where we are using python + tensorflow and keras to train an image object detector, to detect different parts of the root system of Arabidopsis.
Our current ressults are pretty bad, as we do only have about 100 images to train the model with at this moment, but we are currently working on cultuvating more plants in order to get more images(more data) to train the tensorflow model.
We have implemented the following Mask_RCNN model:Github- Mask_RCNN tensorflow
We are looking to detect three object clases: stem, main root and secondary root.
But the model detects main roots incorrectly where the secondary roots are located.
It should be able to detect something like this:Root detection example
Training root data set that we are using right now:training images
What is the usual sample size that is used to train a neural network accurate results?
First off: I think there is no simple rule to estimate the sample size but at least it depends on:
1. Quality of your images
I downloaded the images and I think you need to preprocess them before you can use it to reduce the "problem complexity". In some projects, in which I worked with biological data, a background removal (image - low pass filter) was the key to get better results. But you should definitely remove/crop the area outside the region of your interest (like the tape and the ruler). I would try to get the cleanest data set as possible (including manually adjustments cv2/ gimp/ etc.) to focus the network to solve "the right problem".. After that you could apply some random distortion to make it also work on fuzzy/bad/realistic images as well.
2. The way you work with your data
There are a few tricks that enables you to "expand" your dataset.
Sometimes it's very helpful to let a generator method crop random small patches from your input data. This allows you to work with more batches (on small gpus) and gives your network more "variety", (just think about the conv2d task: if you don't use random cropping your filters will slide over the same areas over and over again (at the same image)). Because of the same reason: apply random distortion, flip and rotate your images.
3. Network architecture
In your case I would prefer a U-Net architecture with a last conv2d output of 3 (your classes) feature maps, a final softmax activation and an categorical_crossentropy, this enables you to play with the depth, because sometimes you need sophisticated architectures to solve a problem (close to 100%) but in your case you just want to see a first working result. So fewer layers and a simple architecture could also help you to get things work. Maybe there are some trained network weights for a U-Net which meets your requirements (search on kaggle for example). Because it is also helpful (to reduce the data you need) to use "transfer learning" -> use the first layers of an network (weights) which is already trained. Using a semantic segmentation the first filters will become something like an edge detection for the most given problems/images.
4. Your mental model of "accurate results"
This is the hardest part.. because it evolves during your project. Eg. in the same moment your networks starts to perform well on preprocessed input images you will start to think about architecture/data changes to make it work on fuzzy images as well. This is why you should start with a feasible problem but always improve your dataset (including rare kinds of roots) and tune your network architecture step by step.
I have trained a faster rcnn model with a custom dataset using Tensorflow's Object Detection Api. Over time I would like to continue to update the model with additional images (collected weekly). The goal is to optimize for accuracy and to weight newer images over time.
Here are a few alternatives:
Add images to previous dataset and train a completely new model
Add images to previous dataset and continue training previous model
New dataset with just new images and continue training previous model
Here are my thoughts:
option 1: would be more time consuming, but all images would be treated "equally".
Option 2: would like take less additional training time, but one concern is that the algorithm might be weighting the earlier images more.
Option 3: This seems like the best option. Take original model and simply focus on training the new stuff.
Is one of these clearly better? What would be the pros/cons of each?
In addition, I'd like to know if it's better to keep one test set as a control for accuracy or to create a new one each time that includes newer images. Perhaps adding some portion of new images to model and another to the test set, and then feeding older test set images back into model (or throwing them out)?
Consider the case where your dataset is nearly perfect. If you ran the model on new images (collected weekly), then the results (i.e. boxes with scores) would be exactly what you want from the model and it would be pointless adding these to the dataset because the model would not be learning anything new.
For the imperfect dataset, results from new images will show (some) errors and these are appropriate for further training. But there may be "bad" images already in the dataset and it is desirable to remove these. This indicates that Option 1 must occur, on some schedule, to remove entirely the effect of "bad" images.
On a shorter schedule, Option 3 is appropriate if the new images are reasonably balanced across the domain categories (in some sense a representative subset of the previous dataset).
Option 2 seems pretty safe and is easier to understand. When you say "the algorithm might be weighting the earlier images more", I don't see why this is a problem if the earlier images are "good". However, I can see that the domain may change over time (evolution) in which case you may well wish to counter-weight older images. I understand that you can modify the training data to do just that as discussed in this question:
Class weights for balancing data in TensorFlow Object Detection API
Is it possible to load part of a trained model into part of a newly built model in TensorFlow?
I mean, for instance, some formerly trained model of no use. But part of it is still useful. And that part could be used in a newly built model. Except this part, others of the newly built model should be trained, but this part need not be retrained again. And the newly built model is quite different from the old model except that part is the same.
If this could be done, how to write such code?
One of the possibilities:
Get the code for the model you are importing
Find the last operation of the part of the graph that you want to keep
Split the code into two parts (classes or functions) at this op
Build the shared part of the graph and load the weights
Proceed adding the rest of the new graph
Run the terminal op in the newly added part
Alternatively, you may build the entire old model, have a "fork" in the middle and, when running, ignore the terminal op in the old, unused part, and use only the terminal op in the newly added part. This way, only the new branch will be trained.
If you don't have the code but only have the saved graph definition, it is a bit more complicated: you will need to find the op you want to fork in the loaded graph by name.
I have a deep fully connected network.
I want to be able to change the structure of middle layers of the network dynamically.
What is the best way of doing that?
What I did right now is to create an output placeholder for my network. I thought I will create a network dynamically by using feed_dict. However, when I run it it says.
`ValueError: No gradients provided for any variable, check your graph for ops that do not support gradients, between variables ... `
Tensorflow won't make this easy for you. Once you define the graph and open a session it's fixed. I believe you need to define a new graph, copy over your variables, and move on from there every time you want to alter the architecture. Kinda annoying for experimenting with this kind of stuff.
I have a friend/fellow researcher who's been experimenting with dynamic neural network architectures and is tackling this in pytorch, which has specific support for dynamically altering network architectures.