I have a keras classification model saved remotely, and I use RESTful API to call the model on my site and make predictions. The JSON request is done as follows
URL:
POST https://my.site/models/v1:predict
JSON:
{"instances": [["...something useful to predict a label..."]]}
and what I obtain is of the form (one value for each label)
{
"predictions": [
[
0.001
0.832
...
0.104
]
]
}
Is there a way to obtain also the label corresponding to each prediction probability? That is, to get something in the form
{
"predictions": [
[
"label_1": 0.001
"label_2": 0.832
...
"label_n": 0.104
]
]
}
or
{
"predictions": [
[
0.001
0.832
...
0.104
]
[
"label_1"
"label_2"
...
"label_n"
]
]
}
I read documentation about specifying signatures when exporting a model, which seems to be related to my question, but it is not very clear about what has to be done. Basically it consists of adding the argument signatures when saving the model:
#tf.function()
def my_predict(my_prediction_inputs):
...
my_signatures = my_predict.get_concrete_function(...)
tf.keras.models.save_model(model, path, signatures=my_signatures, ...)
Related
I trained a Logistic Regression model on Bigquery and downloaded it locally.
Afterwards I wanted to load it and make a prediction but it gives me an error which I cannot solve.
This is the simple code I've written. In particular, query contains the predictors (I've read to submit them in JSON, that's why I've encoded them like this) and I added .signatures['serving_default'] since otherwise it gives me an error, i.e. 'AutoTrackable' object is not callable.
import tensorflow as tf
model = tf.saved_model.load('./log_reg')
query = [{
"pcoordinate_x": "11.191853",
"pcoordinate_y": "45.892605",
"mvalue": "0",
"pcode": "IT*TNK*ETN046",
"porigin": "route220",
"scode": "IT*TNK*ETN046-IT*TNK*ETN046",
"pmetadata_provider": "Route220",
"pmetadata_accessType": "PRIVATE_WITHPUBLICACCESS",
"pmetadata_capacity": "1",
"pmetadata_categories": "['EAT&CHARGE']",
"smetadata_outlets_outletTypeCode": "Schuko",
"smetadata_outlets_maxPower": "3.7",
"smetadata_outlets_maxCurrent": "0.0",
"smetadata_outlets_minCurrent": "0.0",
"mvalue_p": "0.0",
"mvalue_t": "13.3",
"season": "2",
"altitude": "1440.0",
"hour": "09",
"day": "12",
"month": "09"
}]
model.signatures['serving_default'](query)
Running the code now it gives me this error: enter image description here
Has someone been able to make a prediction with their model and can help me? Thanks in advance!
Python is expecting this be called as:
model.signatures['serving_default'](pcoordinate_x = 11.191853, pcoordinate_y = 45.892605, mvalue=0, ...)
I am using Karate API framework for the API automation and came across with one scenario, the scenario is when I am hitting a post call it gives me some json response and few of the items are having tags whereas few of them are showing tags as blank to get all the tags below is the feature file scenario line
* def getTags = get response.items[*].resource.tags
It is giving me response as
[
[
],
[
],
[
{
"tags" : "Entertainment"
}
],
[
],
[
{
"tags" : "Family"
}
],
As you can see out of 5 or 6 tags only 2 tags are having the value, so I want to capture if any tags value is showing or not. What would be the logic for the assertion considering these tags can all come as empty and sometimes with come with a string value. In above case "Family" & "Entertainment"
Thanks in advance !
* match each response.items[*].resource.tags == "##string"
This will validate that tags either doesn't exist or is a string.
I think you can use a second variable to strip out the empties, or maybe your original JsonPath should use .., you can experiment:
* def allowed = ['Music', 'Entertainment', 'Documentaries', 'Family']
* def response =
"""
[
[
],
[
],
[
{
"tags":"Entertainment"
}
],
[
],
[
{
"tags":"Family"
}
]
]
"""
* def temp = get response..tags
* print temp
* match each temp == "#? allowed.contains(_)"
I have successfully got my code to detect the correct intent but no entities appear even though I provided some entities in my training data.
data.json:
{ “common_examples”: [
{ “text”:“Hello”,
“intent”:“greeting”,
“entities”:[] },
{ “text”:“Hi”,
“intent”:“greeting”,
“entities”:[] },
{ “text”:“I want a recipe for my lunch”,
“intent”:“get_recipe”,
“entities”:[
{ “start”:22,
“end”: 28,
“value”: “lunch”,
“entity”: “mealtime” }
]
},
{ “text”:“Can you give me a recipe for dinner tonight?”,
“intent”:“get_recipe”,
“entities”:[
{ “start”:29,
“end”:35,
“value”: “dinner”,
“entity”: “mealtime” }
]
},
{ “text”:“I don’t know what to have for lunch”,
“intent”:“get_recipe”,
“entities”:[
{ “start”:31,
“end”: 35,
“value”: “lunch”,
“entity”: “mealtime” }
]
}
},
}
],
"regex_features": [],
"entity_synonyms":[]
}
}
This is just a snippet. I have created 15 examples in total for the get_recipe intent. I just need it to pick out the entity of ‘mealtime’ from the message put to the bot.
My config.yml is as follows:
language: “en”
pipeline:
-name: “nlp_spacy”
-name: “tokenizer_spacy”
-name: “intent_entity_featurizer_regex”
-name: “intent_featurizer_spacy”
-name: “ner_crf”
-name: “ner_synonyms”
-name: “intent_featurizer_count_vectors”
-name: “intent_classifier_tensorflow_embedding”
and this is the code I run to train the bot:
from rasa_nlu.training_data import load_data
from rasa_nlu.model import Trainer
from rasa_nlu import config
from rasa_nlu.model import Interpreter
def train_bot(data_json,config_file,model_dir):
training_data = load_data(data_json)
trainer = Trainer(config.load(config_file))
trainer.train(training_data)
model_directory=trainer.persist(model_dir,fixed_model_name=‘vegabot’)
This runs fine.
And the code I run to predict the intent:
def predict_intent(text):
interpreter = Interpreter.load(‘models/nlu/default/vegabot’)
print(interpreter.parse(text))
Which produces the result:
{‘intent’: {‘name’: ‘get_recipe’, ‘confidence’: 0.9701309204101562}, ‘entities’: [], ‘intent_ranking’: [{‘name’: ‘get_recipe’, ‘confidence’: 0.9701309204101562}, {‘name’: ‘greeting’, ‘confidence’: 0.03588612377643585}], ‘text’: ‘can you find me a recipe for dinner’}
As you can see the intent is correct but entities is blank [] and I can’t figure out why. I don't seem to be getting any errors. Everything runs okay apart from this!
I also ran an evaluation and got:
- intent examples: 12 (2 distinct intents)
- Found intents: ‘greeting’, ‘get_recipe’
- entity examples: 10 (1 distinct entities)
- found entities: ‘mealtime’ which all looks fine.
So obviously it knows to look out for the mealtime entity but why isn't it picking it up from my test messages?
e.g. I need a recipe for lunch, Can you give me a dinner time recipe? etc
I’m using RASA NLU version 0.14.
Any help would be greatly appreciated. Thank you.
The machine learning models in Rasa need a bit of data to train. As correctly suggested in the comments, you have to give the conditional random field a couple of examples, so that it is actually able to generalize. Also make sure to vary the sentences around it, otherwise crf will not generalize to other contexts.
In the training output for a hyperparameter tuning job on Google Cloud ML Engine, I do not see the values of the objective calculated for each trial. The training output is the following:
{
"completedTrialCount": "4",
"trials": [
{
"trialId": "2",
"hyperparameters": {
"learning-rate": "0.0010000350944297609"
}
},
{
"trialId": "3",
"hyperparameters": {
"learning-rate": "0.0053937227881987841"
}
},
{
"trialId": "4",
"hyperparameters": {
"learning-rate": "0.099948384760813816"
}
},
{
"trialId": "1",
"hyperparameters": {
"learning-rate": "0.02917661111653325"
}
}
],
"consumedMLUnits": 0.38,
"isHyperparameterTuningJob": true
}
The hyperparameter tuning job appears to run correctly and displays a green check mark next to the job. However, I expected that I would see the value of the objective function for each trial in the training output. Without this, I don't know which trial is best. I have attempted to add the value of the objective into the summary graph as follows:
with tf.Session() as sess:
...
final_cost = sess.run(tf.reduce_sum(tf.square(Y-y_model)), feed_dict={X: trX, Y:trY})
summary = Summary(value=[Summary.Value(tag='hyperparameterMetricTag', simple_value=final_cost)])
summary_writer.add_summary(summary)
summary_writer.flush()
I believe I have followed all the steps discussed in the documentation to set up a hyperparameter tuning job. What else is required to ensure that I get an output that lets me compare different trials?
Could you please check if you can find the value of hyperparameterMetricTag on tensorboard to make sure you report the metric correctly? And please make sure you specify the same hyperparameterMetricTag name(it's hyperparameterMetricTag in your case) in your job request(HyperparameterSpec) and your code.
I'm trying to create a simple program with Rasa which extracts a (French) street address from a text input.
Following the advice in Rasa-NLU doc (http://rasa-nlu.readthedocs.io/en/latest/entities.html), I want to use spaCy to do the address detection.
I saw (https://spacy.io/usage/training) that the corresponding spaCy prebuilt entity would be LOC.
However, I don't understand how to create a training dataset with this entity.
Here is an excerpt from my current JSON training dataset :
{
"text" : "je vis au 2 Rue des Platanes",
"intent" : "donner_adresse",
"entities" : [
{
"start" : 10,
"end" : 28,
"value" : 2 Rue des Platanes",
"entity" : "adresse"
}
]
}
If I train the program and run it with the text input "je vis au 2 Rue des Hetres", I get this output :
{
"entities": [
"end": 26,
"entity": "adresse",
"extractor": "ner_crf",
"start": 10,
"value": "2 rue des hetres"
],
"intent": null,
"intent_ranking": [],
"text": "je vis au 2 Rue des Hetres"
}
Which is fine given my training dataset. But I would like to use spaCy's LOC entity.
How can I achieve that ? (What am I doing wrong ?)
Here is a relevant summary of my config file, if needed :
{
"pipeline" : "spacy_sklearn",
"language" : "fr",
"spacy_model_name" : "fr_core_news_md"
}
If you want to use spaCy's pre-trained NER, you just need to add it to your pipeline, e.g.
pipeline = ["nlp_spacy", "tokenizer_spacy", "ner_spacy"]
But depending on what you need, you might want to just copy one of the preconfigured pipelines and add "ner_spacy" at the end