Dropped event message OpenNLP. Training data is dropped in OpenNLP - training-data

I have labeled data (label and text), like this:
category1, "train message 1"
category1, "train message 2"
category1, "train message 3"
category2, "train message 4"
category2, "train messsage 5"
I try to train my categorize model with Java OpenNLP library.
DoccatModel model = DocumentCategorizerME.train("pt", sampleStream, params, customFactory);
When i training model, i get strange messages:
**Indexing events using cutoff of 5**
**Computing event counts... done. 5441 events**
Dropped event animals*:[bow=live, bow=animals, ng=:live:animals]
Dropped event animals*:[bow=aquariums]
Dropped event animals*:[bow=aquatic, bow=plant, bow=fertilizers, ng=:aquatic:plant,ng=:aquatic:plant:fertilizers, ng=:plant:fertilizers]
Dropped event apparel*:[bow=activewear]
Dropped event apparel*:[bow=one, bow=pieces, ng=:one:pieces]
Why does it mean Dropped event "category": [....]?**

I added custom factory, it work
int minNgramSize = 2;
int maxNgramSize = 3;
DoccatFactory customFactory = new DoccatFactory(new FeatureGenerator[]{
new BagOfWordsFeatureGenerator(),
new NGramFeatureGenerator(minNgramSize, maxNgramSize)
});
DoccatModel model = DocumentCategorizerME.train("pt", sampleStream, params, customFactory);

Related

Using telebot register_next_step_handler structure I get error with mixed users' data

I want to get user's nickname and after that to get user's screenshot. Than all the data will be send to Google Sheet. But I have the problem. When multiple users(at least 2) are using bot at the same time, their data are mixed up. For example first user's nickname is second user's nickname or first user's id is second user's id. Can I make a unique user session to store data in unique user class. It means for each user it will be created their own user class. Here is the code:
#bot.message_handler(commands=['start'])
def send_hello(message):
bot.send_message(message.chat.id, "Hi! Let`s start verification.")
msg = bot.send_message(message.chat.id, "Enter your nickname:")
bot.register_next_step_handler(msg, process_nickname)
def process_nickname(message):
user.name=message.text
user.id=message.from_user.id
msg = bot.send_message(message.chat.id, 'Super! Now send screenshot:')
bot.register_next_step_handler(msg, process_screenshot)
def process_screenshot(message):
fileID = message.photo[-1].file_id
file = bot.get_file(fileID)
file_path=file.file_path
metadata = {
'name': user.name,
'parents':[folder_id]
}
url=(f"https://api.telegram.org/file/bot{token}/{file_path}")
response = requests.get(url)
image_data = BytesIO(response.content)
media=MediaIoBaseUpload(image_data, 'image/jpeg')
serviceDrive.files().create(body=metadata,media_body=media,fields='id').execute()

Adding a Retokenize pipe while training NER model

I am currenly attempting to train a NER model centered around Property Descriptions. I could get a fully trained model to function to my liking however, I now want to add a retokenize pipe to the model so that I can set up the model to train other things.
From here, I am having issues getting the retokenize pipe to actually work. Here is the definition:
def retok(doc):
ents = [(ent.start, ent.end, ent.label) for ent in doc.ents]
with doc.retokenize() as retok:
string_store = doc.vocab.strings
for start, end, label in ents:
retok.merge(
doc[start: end],
attrs=intify_attrs({'ent_type':label},string_store))
return doc
i am adding it into my training like this:
nlp.add_pipe(retok, after="ner")
and I am adding it into the Language Factories like this:
Language.factories['retok'] = lambda nlp, **cfg: retok(nlp)
The issue I keep getting is "AttributeError: 'English' object has no attribute 'ents'". Now I am assuming I am getting this error because the parameter that is being passed through this function is not a doc but actually the NLP model itself. I am not really sure to get a doc to flow into this pipe during training. At this point I don't really know where to go from here to get the pipe to function the way I want.
Any help is appreciated, thanks.
You can potentially use the built-in merge_entities pipeline component: https://spacy.io/api/pipeline-functions#merge_entities
The example copied from the docs:
texts = [t.text for t in nlp("I like David Bowie")]
assert texts == ["I", "like", "David", "Bowie"]
merge_ents = nlp.create_pipe("merge_entities")
nlp.add_pipe(merge_ents)
texts = [t.text for t in nlp("I like David Bowie")]
assert texts == ["I", "like", "David Bowie"]
If you need to customize it further, the current implementation of merge_entities (v2.2) is a good starting point:
def merge_entities(doc):
"""Merge entities into a single token.
doc (Doc): The Doc object.
RETURNS (Doc): The Doc object with merged entities.
DOCS: https://spacy.io/api/pipeline-functions#merge_entities
"""
with doc.retokenize() as retokenizer:
for ent in doc.ents:
attrs = {"tag": ent.root.tag, "dep": ent.root.dep, "ent_type": ent.l
abel}
retokenizer.merge(ent, attrs=attrs)
return doc
P.S. You are passing nlp to retok() below, which is where the error is coming from:
Language.factories['retok'] = lambda nlp, **cfg: retok(nlp)
See a related question: Spacy - Save custom pipeline

Telegram bot: How to get chosen inline result

I'm sending InlineQueryResultArticle to clients and i'm wondering how to get chosen result and it's data (like result_id,...).
here is the code to send results:
token = 'Bot token'
bot = telegram.Bot(token)
updater = Updater(token)
dispatcher = updater.dispatcher
def get_inline_results(bot, update):
query = update.inline_query.query
results = list()
results.append(InlineQueryResultArticle(id='1000',
title="Book 1",
description='Description of this book, author ...',
thumb_url='https://fakeimg.pl/100/?text=book%201',
input_message_content=InputTextMessageContent(
'chosen book:')))
results.append(InlineQueryResultArticle(id='1001',
title="Book 2",
description='Description of the book, author...',
thumb_url='https://fakeimg.pl/300/?text=book%202',
input_message_content=InputTextMessageContent(
'chosen book:')
))
update.inline_query.answer(results)
inline_query_handler = InlineQueryHandler(get_inline_results)
dispatcher.add_handler(inline_query_handler)
I'm looking for a method like on_inline_chosen(data) to get id of the chosen item. (1000 or 1001 for snippet above) and then send the appropriate response to user.
You should set /setinlinefeedback in #BotFather, then you will get this update
OK, i got my answer from here
Handling user chosen result:
from telegram.ext import ChosenInlineResultHandler
def on_result_chosen(bot, update):
print(update.to_dict())
result = update.chosen_inline_result
result_id = result.result_id
query = result.query
user = result.from_user.id
print(result_id)
print(user)
print(query)
print(result.inline_message_id)
bot.send_message(user, text='fetching book data with id:' + result_id)
result_chosen_handler = ChosenInlineResultHandler(on_result_chosen)
dispatcher.add_handler(result_chosen_handler)

How to get delivery report from SMSC using Jamaa-Smpp and C#?

How to get delivery report from SMSC using Jamaa-Smpp and C#?
Thanks in advance,
We can get delivery report in Jamaa-Smpp by using client_MessageDelivered event
First ,we need to use this code:
SmppClient(); client = new SmppClient();
client.MessageDelivered += new EventHandler<MessageEventArgs>(client_MessageDelivered);
Second , we need to use this code:
void client_MessageDelivered(object sender, MessageEventArgs e)
{
TextMessage msg = e.ShortMessage as TextMessage;
string msgtext=msg.text;
}
and do not forget to set value to SubmitSm.RegisteredDelivery as your work requirements:
//SMSC delivery receipt requested where final delivery outcome is deliver success or failure
SubmitSm.RegisteredDelivery = RegisteredDelivery.DeliveryReceipt;
or
//SMSC delivery receipt requested where the final delivery outcome is delivery failure
SubmitSm.RegisteredDelivery = RegisteredDelivery.DeliveryReceiptFailure;

how to set a different message for an email template in odoo?

I created a custom module and had used the calendar object to create an event and the code is as follows
def create_calender_event(self,cr,uid,ids,context=None):
calendar_obj = self.pool.get('calendar.event')
for rec in self.browse(cr,uid,ids,context=context):
if rec.action:
for rec_res in rec.action:
calendar_obj.create(cr,uid,{'name' : rec_res.act_ion,
'user_id' : rec_res.asgnd_to.id,
'start_date' : rec_res.due_date,
'stop_date' : rec_res.due_date,
'allday' : True,
'partner_ids' : [(6,0, [rec_res.asgnd_to.partner_id.id])]
},context=context)
This will create a event in respective user's calendar, but it uses default template message.
How can i replace the calendar invitation template message by custom message ?
you can do like this from py file
1) get the template_id and browse the object
2) the template body will be stored in 'body_html' field
3) store the body_html field in one variable, lets say: old_body
4) then add your customized code to template's 'body_html' field and write the values into template using the above temlate_id
5) send the mail, using send method
6) then write back the old_body value back to the template.
just for idea, refer this....
template_id = template_pool.search(cr,uid,[('name','=ilike',template_name)])
if template_id:
template_obj = template_pool.browse(cr, uid, template_id)
body = template_obj.body_html
body_old = body
count = 0
body += " For %s Study Notes PDF Click here "%(url['subject'],url['url'])
template_pool.write(cr, uid, template_id, {'body_html':body})
template_pool.send_mail(cr, uid, template_id[0], record.id)
template_pool.write(cr, uid, template_id, {'body_html':
body_old})
It's email template with xml id "calendar_template_meeting_invitation" is used to send invitation. So find this template and change to whatever you want. Under template its called "Meeting Invitation" email template.
OLD===================
UPDATES================================
on calendar.event object create and write method calls the method create_attendees which create all attendee and send email invitation by calling method _send_mail_to_attendees code ref, so you need to overload that function and remvoe that statement so it doesn't send email when attendee is created rather send invitation when you want.
Bests