I am trying this for a lot of time so I will appreciate some help. Qgis graphical modeler allow us to put some advanced layers, but if we use them directly in the process the modeler will crash.
I am trying to tackle this problem with the conditional branche, so I would like to put a Condition in the conditional branche for it to work only if the Advanced layer is there. And if we put none layer there the rest of the model would work but not this branche.
Do someone know what condition I could put for it to see if the layer is there or no?
I have tried several ones that have not worked:
layer_property( #layer ,'extent')= NULL
Graphical modeler with conditional branch
Conditional branch
Related
I am writing an optimization code for a finite-difference radiation solver model. I started to use "src_indices" for connecting parameters rather promoting all the variables. But when I changed the connection, optimization does not calculate derivatives, gives "no impact to objective" error, and successfully terminates optimization after first iteration. Could not find any clue for finding the error in the logs (Bug may be in a completely different reason).
Is there any suggestion where I can start?
I uploaded the code to GitHub https://github.com/TufanAkba/opt_question
The first thing that comes to mind when you mention "design variables have no impact on objective" is that there may be a missing connection. Since this behavior only started after you changed the connection style, I think this is even more likely.
There are a couple of tools you can use to diagnose this. The first is the n2 viewer, which you can launch by typing the following at your command prompt:
openmdao n2 receiver_opt.py
This will launch a browser window that contains a graphical model viewer which is described in detail here. You can use this to explore the structure of your model. To find unconnected inputs in your model, look for any input blocks that are colored orange. These are technically connected to a hidden IndepVarComp called _auto_ivc, and will include design variables, which are set by the optimizer. You will want to look for any that should be connected to other component outputs.
OpenMDAO also has a connection viewer that just shows connections.
openmdao view_connections receiver_opt.py
You can use this tool to just focus on the connections. It is described here. If you choose to use this, just filter to see any connection to _auto_ivc in the source output string to see the unconnected inputs.
If you reach this point, and are satisfied that all the connections are correct, then there are a couple of other possibilities:
Are all of your src_indices correct? Maybe some of them are an empty set, or maybe some create a "degenerate" case. For example, if you have a set of cascading components that each multiply an incoming vector by a diagonal matrix, and if your indices are [0] in one connection, and [4] in another connection, then you've effectively severed the entire model. None of our visualization tools can pick that up, and you will need to inspect the indices manually.
It could also be a derivative problem, though what you describe sounds like connections. In that case, I recommend using check_partials to look for any missing or incorrect derivatives.
Are you computing any derivatives using complex step? It is possible that you are losing the complex part of the calculation through a complex-unsafe operation. Checking your derivatives against 'fd' can help to find these.
Actually, in ML pipeline components we are specifying inputs and outputs clearly .
For example in TFX statisticgen take input from examplegen and outputs some statistics.so input and output is clear which is same in all components .so why we need orchestrators .if anyone knows please help me?
In real-life projects, everything can be much more complicated:
the input data can be from the different sources: database, file system, third-party services. So we need to do classical ETL before we can start working with data.
you can use different technologies in the one pipeline. For instance, Spark as a preprocessing tool, after you can need to use an instance with GPU for the model training.
last, but not least - in production you need to care much more things. For instance data validation, model evaluation, etc. I wrote a separate article about how to organize this part using Apache Airflow.
I'm doing a data analysis task in SPSS Modeler and I have finally arrived to the point of the stream where I'm trying to fit some models on the data.
However when I tried to run the mentioned c5.0 modeling node on my data, the node generated a modeling nugget containing only a single leaf, so there are no decision rules in the model. I partitioned the data before to train and test subsets (70-30). I did not use misclassification cost, used the properly predefined attribute roles. In the model's model page I checked the use partitioned data, build model for each split, Group symbolics, Use global pruning options in, I also tried to use expert mode, but it fails on simple mode too. I have tried to use different options but it gives the same output without a single split.
How can I make the model give back a more complex decision tree, I suppose that this is not the expected outcome.
Any suggestions are welcomed.
Please, check your distribution of the target variable and share it.
If the balances differs greatly from 50%-50%, you may need to balance your inputs first.
Missclassification cost is another technique to give you an output, but again it should be based on your empirical distributions.
The simplest example of what I'm trying to recognize:
I use DOCUMENT_TEXT_DETECTION, but in the answer I get the hieroglyphics.
If I use Eng in the ImageContext parameter for the addAllLanguageHints method, then I have 111 in result. Better, but still bad.
Is there any way to indicate that the numbers are recognised or somehow improve the results?
Also, how is the setRepeatedField option in ImageContext is used? I could not find any examples of its use.
Thanks in advance.
Even if it doesn't work out of the box ... you'd need is to classify images using custom labels, when the default labels won't suffice. Cloud Auto ML Vision (select Vision from that blue drop-down menu) let's you train custom models, which can be used to recognize that font. And since the possible amount of shapes is quite limited with that 7-segment display, it shouldn't be too difficult to train it. If you'd get a calculator with a better display, it might also work better. The LCD above looks a little cheap, with those huge spaces and cut-off endings - but nevertheless, one can train it to read that.
This year Google produced 5 different packages for seq2seq:
seq2seq (claimed to be general purpose but
inactive)
nmt (active but supposed to be just
about NMT probably)
legacy_seq2seq
(clearly legacy)
contrib/seq2seq
(not complete probably)
tensor2tensor (similar purpose, also
active development)
Which package is actually worth to use for the implementation? It seems they are all different approaches but none of them stable enough.
I've had too a headache about some issue, which framework to choose? I want to implement OCR using Encoder-Decoder with attention. I've been trying to implement it using legacy_seq2seq (it was main library that time), but it was hard to understand all that process, for sure it should not be used any more.
https://github.com/google/seq2seq: for me it looks like trying to making a command line training script with not writing own code. If you want to learn Translation model, this should work but in other case it may not (like for my OCR), because there is not enough of documentation and too little number of users
https://github.com/tensorflow/tensor2tensor: this is very similar to above implementation but it is maintained and you can add more of own code for ex. reading own dataset. The basic usage is again Translation. But it also enable such task like Image Caption, which is nice. So if you want to try ready to use library and your problem is txt->txt or image->txt then you could try this. It should also work for OCR. I'm just not sure it there is enough documentation for each case (like using CNN at feature extractor)
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/seq2seq: apart from above, this is just pure library, which can be useful when you want to create a seq2seq by yourself using TF. It have a function to add Attention, Sequence Loss etc. In my case I chose that option as then I have much more freedom of choosing the each step of framework. I can choose CNN architecture, RNN cell type, Bi or Uni RNN, type of decoder etc. But then you will need to spend some time to get familiar with all the idea behind it.
https://github.com/tensorflow/nmt : another translation framework, based on tf.contrib.seq2seq library
From my perspective you have two option:
If you want to check the idea very fast and be sure that you are using very efficient code, use tensor2tensor library. It should help you to get early results or even very good final model.
If you want to make a research, not being sure how exactly the pipeline should look like or want to learn about idea of seq2seq, use library from tf.contrib.seq2seq.