In my BQ database table I have a column called payload which contains raw facebook webhooks JSON payloads as string. One of them contains a text with an emoji like Sample 🏦. In big query it look like
{"object":"page","entry":[{"id":"xxxx","time":1602757469275,"messaging":[{"sender":{"id":"xxxx"},"recipient":{"id":"xxxx"},"timestamp":1602757469062,"message":{"mid":"m_xxxx","text":"Sample \ud83c\udfe6","quick_reply":{"payload":"{\"key\": \"value\"}"},"tags":{"source":"source"}}}]}]}
I would like to create a view with a column text with extracted text field value from the raw json. I created an sql like
SELECT
JSON_EXTRACT_SCALAR(payload, '$.entry[0].messaging[0].message.text') as text,
FROM `my_table.facebook.webhook_received`
Sadly the result I get looks like that Sample ��
Does anyone know how to make big query decode the emoji properly or at least just not change it to those � signs ?
Those characters you have embedded are not for a bank icon which is your issue I believe.
Run the following in BQ and it returns the desired emoji:
select " Sample \U0001f3e6"
Ref:https://emojipedia.org/bank/
The two you have provided seem to default to the '?', invalid character
http://unicode.scarfboy.com/?s=U%2Bdfe6
edit: what ever is handling the message maybe throwing the encodings you're seeing in your message which may be the actual problem.
If you are using BigQuery Python client and its load_table_from_json method, there is a Unicode bug (especially its byte is over 0xFFFF, like 🏦) in the previous version, and I have submitted this bug fix which is already included in the latest release include it, https://github.com/googleapis/python-bigquery/releases/tag/v2.24.0. By the way, you should use \U0001F3E6, not \ud83c\udfe6 (UTF-16 hex type) to present 🏦 in your Python code with BigQuery.
Unicode Character 'BANK': https://www.fileformat.info/info/unicode/char/1f3e6/index.htm,
https://charbase.com/1f3e6-unicode-bank
I'm working with ThemeFuse I found this data format. But it not looks like anything I saw before.
a:66:{s:24:"autotrader_slider_images";a:0:{}s:26:"autotrader_thumbnail_image";s:0:"";s:19:"seek_property_price";s:0:"";s:23:"seek_property_vat_price";s:0:"";s:21:"seek_property_mileage";s:0:"";s:25:"seek_property_engine_size";s:0:"";s:30:"seek_property_engine_power_bhp";s:0:"";s:29:"seek_property_engine_power_kw";s:0:"";s:23:"seek_property_reduction";s:1:"0";s:25:"seek_property_consumption";s:1:"0";s:20:"seek_property_origin";s:0:"";s:22:"seek_property_emission";s:13:"super emitivo";s:23:"autotrader_vehicle_type";s:3:"SUV";s:20:"autotrader_fuel_type";s:6:"Diesel";s:23:"autotrader_gearbox_type";s:9:"Automatic";s:17:"autotrader_status";s:6:"Intact";s:16:"autotrader_color";s:5:"White";s:18:"seek_property_year";s:0:"";s:26:"autotrader_enable_comments";s:5:"false";s:29:"autotrader_enable_breadcrumbs";s:4:"true";s:25:"autotrader_header_element";s:4:"none";s:23:"autotrader_header_image";s:0:"";s:23:"autotrader_header_title";s:0:"";s:24:"autotrader_select_slider";s:2:"-1";s:19:"autotrader_page_map";s:0:"";s:19:"autotrader_map_text";s:11:"We are here";s:19:"autotrader_map_zoom";s:2:"13";s:25:"autotrader_search_element";s:4:"none";s:22:"autotrader_content_top";s:0:"";s:26:"autotrader_content_bottom1";s:0:"";s:25:"autotrader_footer_element";s:4:"none";s:31:"autotrader_select_slider_footer";s:2:"-1";s:25:"autotrader_content_bottom";s:0:"";s:26:"autotrader_content_bottom2";s:0:"";s:23:"autotrader_top_ad_space";s:5:"false";s:23:"autotrader_top_ad_image";s:0:"";s:21:"autotrader_top_ad_url";s:0:"";s:25:"autotrader_top_ad_adsense";s:0:"";s:30:"autotrader_bfcontent_ads_space";s:5:"false";s:25:"autotrader_bfcontent_type";s:5:"image";s:27:"autotrader_bfcontent_number";s:3:"one";s:31:"autotrader_bfcontent_ads_image1";s:0:"";s:29:"autotrader_bfcontent_ads_url1";s:0:"";s:33:"autotrader_bfcontent_ads_adsense1";s:0:"";s:31:"autotrader_bfcontent_ads_image2";s:0:"";s:29:"autotrader_bfcontent_ads_url2";s:0:"";s:33:"autotrader_bfcontent_ads_adsense2";s:0:"";s:31:"autotrader_bfcontent_ads_image3";s:0:"";s:29:"autotrader_bfcontent_ads_url3";s:0:"";s:33:"autotrader_bfcontent_ads_adsense3";s:0:"";s:31:"autotrader_bfcontent_ads_image4";s:0:"";s:29:"autotrader_bfcontent_ads_url4";s:0:"";s:33:"autotrader_bfcontent_ads_adsense4";s:0:"";s:31:"autotrader_bfcontent_ads_image5";s:0:"";s:29:"autotrader_bfcontent_ads_url5";s:0:"";s:33:"autotrader_bfcontent_ads_adsense5";s:0:"";s:31:"autotrader_bfcontent_ads_image6";s:0:"";s:29:"autotrader_bfcontent_ads_url6";s:0:"";s:33:"autotrader_bfcontent_ads_adsense6";s:0:"";s:31:"autotrader_bfcontent_ads_image7";s:0:"";s:29:"autotrader_bfcontent_ads_url7";s:0:"";s:33:"autotrader_bfcontent_ads_adsense7";s:0:"";s:21:"autotrader_hook_space";s:5:"false";s:21:"autotrader_hook_image";s:0:"";s:19:"autotrader_hook_url";s:0:"";s:23:"autotrader_hook_adsense";s:0:"";}
What should I use to parse and unparse this format?
This is serialized PHP array.
All you need to do is to unserialize it.
http://php.net/manual/en/function.unserialize.php
I created a cTAKES custom dictionary from UMLS database with ICD10 codes.
Right now I able to analyze the text by for example disease name, like Asthma and annotation index will contain the ICD10 code for this matching code = "J45.90".
Is it possible to configure cTAKES in order to reverse this process in order to look for ICD10 code appearance in the text instead?
The XML output contains the start and ends of a matched concept in the original corpus. I personally find it easier to convert the XML to a simple JSON format and then loop through it as needed.
I have been working on an open source solution for parsing out the data and displaying the corpus with the matches it in HTML: https://github.com/GoTeamEpsilon/ctakes-friendly-web-ui#demonstration - let me know if you'd like to contribute.
I have a parameter Like the pictures Below :
How to Pull The XRLabel(Parameters) From GroupHeader1 to GroupFooter1 Like XRCrossbandBox did ?
unfortunately there is no such built in functionality
The API doc and support article are a bit unclear about the exact usage of the regionText parameter.
Is it supposed to be a simple string or an XPath? For example, look at http://www.circlecount.com/community/114481059214254340537 - I would like to extract the table in the middle-right. My current API request looks like this:
https://api.import.io/store/data/_magic?url=http://www.circlecount.com/community/114481059214254340537®ionText=//*[#id=follower_table_114481059214254340537]&_apikey=XXX&_user=YYY
Region text should be simple string, not an xpath