Seems, that there is no way to convert from the box Mobilenet (and other models from TF OD API) to uff format and then to TensorRT format, because of much unsupported layers.
Is there some way to remove \ replace that layers? For example with graph_transform tool maybe? I understand the purpose of not all these layers.
Here is the default model, if someone wants to try.
The UFF conversion tools seems to choke on some unrecognized layers, perhaps this will be improved in the GA release. For now, you will need to remove those layers (And keep only the smallest subset needed for inference), then implement those using the nvinfer plugin API.
From 2017 year there is still no significant progress.
Many new networks has been developed, but UFF and TensorRT converter still can not work with many models even from 2017, what can I say about 2019.
There is also some indirect information in the internet that, in sense, with UFF and TRT convertion in NVIDIA works very small department with just a few guys. So seems that way is dead end.
Better way and the way they are counting on is conversion tools inside framework itself. Like Tensorflow do with their TRT bindings.
With such approach new layers are added by developer of framework and not Nvidia, because guys there don't keep up with marketing. So more chances eventually to get working tool.
Related
I am currently trying to train a custom model for use in Unity (Barracuda) for object detection and I am struggling near what I believe to be the last part of the pipeline. Following various tutorials and git-repos I have done the following...
Using Darknet, I have trained a custom-model using the Tiny-Yolov2 model. (model tested successfully on a webcam python script)
I have taken the final weights from that training and converted them
to a Keras (h5) file. (model tested successfully on a webcam python
script)
From Keras, I then use tf.save_model to turn it into a
save_model.pd.
From save_model.pd I then convert it using tf2onnx.convert to change
it to an onnx file.
Supposedly from there it can then work in one of a few Unity sample
projects...
...however, this project fails to read in the Unity Sample projects I've tried to use. From various posts it seems that I may need to use a 'frozen' save_model.pd before converting it to ONNX. However all the guides and python functions that seem to be used for freezing save_models require a lot more arguments than I have awareness of or data for after going through so many systems. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py - for example, after converting into Keras, I only get left with a h5 file, with no knowledge of what an input_graph_def, or output_node_names might refer to.
Additionally, for whatever reason, I cannot find any TF version (1 or 2) that can successfully run this python script using 'from tensorflow.python.checkpoint import checkpoint_management' it genuinely seems like it not longer exists.
I am not sure why I am going through all of these conversions and steps but every attempt to find a cleaner process between training and unity seemed to lead only to dead ends.
Any help or guidance on this topic would be sincerely appreciated, thank you.
I am new to machine learning models and data science libraries. I wanted to use the Hidden Markov model for statistical data prediction on the fly which read the data from kafka and builds the model which is used to predict the data during the run-time and do the same for continous stream always.
Currently i can see only Tensorflow hidden markov model implementation in tensorflow python (tensorflow_probability distribution). Is their any other library available which can help me acheive the above scenario
Suggestions can involve the libraries of JAVA and python
Please feel free to add any resource links that can help me to understand the usage of tensorflow for hidden markov model
this might be a nice place to start: https://hmmlearn.readthedocs.io/en/latest/tutorial.html
Other alternatives, I found, are
Java:
Mallet library and it's extention GRMM in particular.
Python:
Pommegranate with it's HMM support.
Having said that, TensorFlow is much better known active and supported library, in my impression. I'd try that first.
I'm searching a library that would support Hierarchical HMMs (HHMM). That would probably require some tweaking into one of the listed ones.
I want to know if CNTK dead? Release notes on GitHub dated 03/31/2019: "Today’s 2.7 release will be the last main release of CNTK." I've spent months developing software using CNTK and now it appears to be a waste of time and money. I've search for an answer on numerous sites and still no answer. stackoverflow is one of the sites recommend by Microsoft.
From KedengMS, one of the maintainers for CNTK. Reposted from github.
Thanks for all the CNTK supporters, and I am privileged to have worked
on it, and learned a lot in the process. You can continue to use CNTK
for training and inference in the way it currently is, as other
Microsoft internal teams that still runs old models even in
BrainScript or NDL. Stopping adding new features does not mean CNTK is
no longer open source, it just means that going forward, there will be
no new GPU support (say, CUDA 11+), and no major new features added.
For different user scenarios, I think you may have different choices:
Deep learning newcomers: IMO CNTK is still a good entry to understand basics of deep learning, if you found CNTK
documents/tutorials/examples useful. Once you learnt the basic, it
won't be too hard to switch between frameworks. However, the DL field
is changing rapidly and CNTK has already lagged behind in a lot of
ways, so if you need more advanced features like dynamic graph,
PyTorch would be a better choice.
Model maintainers: If you already have CNTK models working, and to maintain it just means training with new data, you can continue to use
CNTK the way you currently use it. Actually, teams inside Microsoft
are doing this too. If there are serious bugs preventing productivity,
they still will be fixed. For inference, you can continue to use CNTK
C/C++/Python/C#/Java APIs, or you may export CNTK models in ONNX
format, and use ONNX Runtime or ORT as a slimmer and faster inference
engine. You'll be surprised to find how much faster it is comparing to
CNTK, and how slimmer the setup is (forget about OpenMPI when you just
need inference!). ORT currently provides C/C++/Python/C# interfaces.
Model builders: If you have CNTK model, and want to use features that are not currently supported in CNTK, please consider switch to
other frameworks like TensorFlow/PyTorch/etc. Our team has done lots
of data reader work inside PyTorch to ensure teams in Microsoft can
switch from CNTK to PyTorch. Besides, we are also in the process of
migrating CNTK specific distributed trainer like BMUF to PyTorch.
Hopefully you'll find that useful too when migrating your model.
The good thing about open source is that the community can continue to
fork/evolve if needed, unlike other Microsoft products that only ship
binaries (Win7 I am looking at you).
Is there any equivalent/alternate library to fastai in tensorfow for easier training and debugging deep learning models including analysis on results of trained model in Tensorflow.
Fastai is built on top of pytorch looking for similar one in tensorflow.
The obvious choice would be to use tf.keras.
It is bundled with tensorflow and is becoming its official "high-level" API -- to the point where in TF 2 you would probably need to go out of your way not using it at all.
It is clearly the source of inspiration for fastai to easy the use of pytorch as Keras does for tensorflow, as mentionned by the authors time and again:
Unfortunately, Pytorch was a long way from being a good option for part one of the course, which is designed to be accessible to people with no machine learning background. It did not have anything like the clear simple API of Keras for training models. Every project required dozens of lines of code just to implement the basics of training a neural network. Unlike Keras, where the defaults are thoughtfully chosen to be as useful as possible, Pytorch required everything to be specified in detail. However, we also realised that Keras could be even better. We noticed that we kept on making the same mistakes in Keras, such as failing to shuffle our data when we needed to, or vice versa. Also, many recent best practices were not being incorporated into Keras, particularly in the rapidly developing field of natural language processing. We wondered if we could build something that could be even better than Keras for rapidly training world-class deep learning models.
I want to study on the research of deep learning, but I don't know which framwork should I choice between TensorFlow and PaddlePaddle. who can make a contrast between the two frameworks? which one is better? especially in the running efficiency of CPU
It really depends what you are shooting for...
If you plan on training, CPU is not going to work well for you. Use colab or kaggle.
Assuming you do get a GPU, it depends if you want to focus on classification or object detection.
If you focus on classification, Keras is probably the easiest to work with or pytorch if you want some advanced stuff and to be able to change things.
If you plan on object detection, things are getting complicated... Inference is reasonably easy but training is complicated. There are actually 4 platforms you should consider:
Tensorflow - powerful but very difficult to work with. If you do not use Keras (and for OD you usually can't), you need to preprocess the dataset into tfrecords and it is a pain. The OD Api has very cryptic messages and it is very sensitive to the combination of tf version and api version. On the other hand, cool models like efficientdet are more or less easy to use.
MMdetection - very powerful framework, has lots of advanced models and once you understand how to work with it, you can easily work with and of the models it supports. Downside is that some models are slow to arrive (efficientdet, for example)
paddlepaddle - if you know Chinese, this should work ok, maybe. The documentation is a bit behind and usually requires lots of improvisation. Basically it is similar to mmdetection just with a few unique models and a few missing models.
detectron2 - I didn't work with this one, but it seems to support only a few models.
You probably need first to define for yourself what do you want to do and then choose.
Good luck!
It is not that trivial. Some models run faster with one kind of framework others with another. Furthermore, it depends on the hardware as well. See this blog. If inference is your only concern, then you can develop your model in any of the popular frameworks like TensorFlow, PyTorch, etc. In the end convert your model to ONNX format and benchmark its performance with DNN-Bench to choose the best inference engine for your application.