Querying an IoT Hub Object with number key - azure-iot-hub

I am trying to retrieve information from a twin device of a third party IoT Hub instance.
The data I'm trying to access has the next format:
{
"properties": {
"reported": {
"softwareLoad": {
"0": {
"systemSWVer": "1.2.0",
"picSWVer": "0.0.42",
"bootStatus": "inactive",
"partitionId": 0
},
"1": {
"systemSWVer": "1.2.0",
"picSWVer": "0.0.42",
"bootStatus": "active",
"partitionId": 1
},
"beamTableCRC": "0x5454"
}
}
}
}
I was trying to reach the variable systemSWVer to use it as a where clause, but each time I try to access I had an error retrieving the information in 0 and IoT Hub returns an error of a Bad Request.
I tried with this query
SELECT properties.reported.softwareLoad FROM devices WHERE properties.reported.softwareLoad.0.systemSWVer in ["1.2.0", "2.1.0"]
Is there a way to use it as a where clause in my query?
Note: I don't have access to the resource to change the format of the information.

I tried to recreate this case and found the same result. Always an error when you add a number in the expression. I also found a workaround; instead of using 0 or 1, you can add them as an escaped Unicode character. It's important to add them as an ASCII Unicode character, though. 0 becomes \u0030 and 1 becomes \u0031.
Try this query:
SELECT properties.reported.softwareLoad FROM devices
WHERE properties.reported.softwareLoad.\u0030.systemSWVer in ['1.2.0', '2.1.0']
Please note: in any case, you need to use single quotes.
Edit: in my excitement, I forgot to test if just escaping the integer would also work. It does.
SELECT properties.reported.softwareLoad FROM devices
WHERE properties.reported.softwareLoad.\0.systemSWVer in ['1.2.0', '2.1.0']

Related

Writing the correct SQL statement in AWS IoT rule

I am working on AWS IoT to develop my custom solution based on a set of sensors, and I have a problem regarding how to write the SQL statement related to the kind of data I receive from a Zigbee sensor.
An example of what I receive from my sensor is reported here:
{
"type": "reportAttribute",
"from": "WIFI",
"deviceCode": "aws_device_code",
"to": "CLOUD",
"mac": "30:ae:7b:e2:e1:e6",
"time": 1668506014,
"data": {...}
}
What I would like to do is to select messages that have the from field equal to GREENPOWER, something along the lines of SELECT * FROM 'test' WHERE from = 'GREENPOWER', but from is also a keyword in SQL hence my problem. I am no expert whatsoever in SQL, so I am not sure how this can be done. I am also looking for a way to modify the received data, but solving this problem on AWS would be much easier.
Thank you very much for your help!
There are quite a lot of SQL functions that exist in AWS IoT Rule. You can find them here: https://docs.aws.amazon.com/iot/latest/developerguide/iot-sql-functions.html
In your case, something like this should work:
SELECT * FROM 'test' WHERE get(*, "from") = "GREENPOWER"

Left join did not working properly in Azure Stream Analytics

I'm trying to create a simple left join between two inputs (event hubs), the source of inputs is an app function that process a rabbitmq queue and send to a event hub.
In my eventhub1 I have this data:
[{
"user": "user_aa_1"
}, {
"user": "user_aa_2"
}, {
"user": "user_aa_3"
}, {
"user": "user_cc_1"
}]
In my eventhub2 I have this data:
[{
"user": "user_bb_1"
}, {
"user": "user_bb_2"
}, {
"user": "user_bb_3
}, {
"user": "user_cc_1"
}]
I use that sql to create my left join
select hub1.[user] h1,hub2.[user] h2
into thirdTestDataset
from hub1
left join hub2
on hub2.[user] = hub1.[user]
and datediff(hour,hub1,hub2) between 0 and 5
and test result looks ok...
the problem is when I try it on job running... I got this result in power bi dataset...
Any idea why my left isn't working like any sql query?
I tested your query sql and it works well for me too.So when you can't get expected output after executing ASA job,i suggest you following troubleshoot solutions in this document.
Based on your output,it seems that the HUB2 becomes the left table.You could use diagnostic log in ASA to locate the truly output of job execution.
I tested the end-to-end using blob storage for input 1 and 2 and your sample and a PowerBI dataset as output and observed the expected result.
I think there are few things that can go wrong with your query:
First, your join has a 5-hours windows: basically that means it looks at EH1 and EH2 for matches during that large window, so live results will be different from sample input for which you have only 1 row. Can you validate that you had no match during this 5-hour window?
Additionally by default PBI streaming datasets are "hybrid datasets" so it will accumulate results without a good way to know when the result was emitted since there is no timestamp in your output schema. So you can also view previous data here. I'd suggest few things here:
In Power BI, change the option of your dataset: disable "Historic data analysis" to remove caching of data
Add a timestamp column to make sure to identify when the data is generated (the first line of you query will become: select System.timestamp() as time, hub1.[user] h1,hub2.[user] h2 )
Let me know if it works for you.
Thanks,
JS (Azure Stream Analytics)

Azure Logic Apps: Set condition to False when SQL query returns no rows of data

How can i conditionally test the output from an Execute SQL Query to make sure it returns some rows of data.
In my example below if the query returns no rows I don't want it to send an email, I want to do something else. What is the test?
Thanks for your time
I test, if it queries result is no rows, the query body will be like this:
{
"OutputParameters": {},
"ResultSets": {}
}
So you could add a Condition with #{body('Execute_a_SQL_query')['OutputParameters']} is equal to {}. If true, do the things you want. Yo could set this in the Code view mode.
The below is the test result, hope this is what you want.
This will work in Query SQL V2.
What is does is takes the ResultSet and converts to string. This prevent s a null error on the length function. As an empty result set is {}, the length is 2. So if the length is 2 then the the result is empty.
"expression": {
"and": [
{
"equals": [
"#length(string(body('Execute_a_SQL_query_(V2)')?['ResultSets']))",
2
]
}
]
}
I am using similar to this in an until condition which runs until the length is zero. I guess you could do the same?
#equals(length(body('Execute_a_SQL_query')?['value']), 0)

How to extract this json into a table?

I've a sql column filled with json document, one for row:
[{
"ID":"TOT",
"type":"ABS",
"value":"32.0"
},
{
"ID":"T1",
"type":"ABS",
"value":"9.0"
},
{
"ID":"T2",
"type":"ABS",
"value":"8.0"
},
{
"ID":"T3",
"type":"ABS",
"value":"15.0"
}]
How is it possible to trasform it into tabular form? I tried with redshift json_extract_path_text and JSON_EXTRACT_ARRAY_ELEMENT_TEXT function, also I tried with json_each and json_each_text (on postgres) but didn't get what expected... any suggestions?
desired results should appear like this:
T1 T2 T3 TOT
9.0 8.0 15.0 32.0
I assume you printed 4 rows. In postgresql
SELECT this_column->'ID'
FROM that_table;
will return column with JSON strings. Use ->> if you want text column. More info here: https://www.postgresql.org/docs/current/static/functions-json.html
In case you were using some old Postgresql (before 9.3), this gets harder : )
Your best option is to use COPY from JSON Format. This will load the JSON directly into a normal table format. You then query it as normal data.
However, I suspect that you will need to slightly modify the format of the file by removing the outer [...] square brackets and also the commas between records, eg:
{
"ID": "TOT",
"type": "ABS",
"value": "32.0"
}
{
"ID": "T1",
"type": "ABS",
"value": "9.0"
}
If, however, your data is already loaded and you cannot re-load the data, you could either extract the data into a new table, or add additional columns to the existing table and use an UPDATE command to extract each field into a new column.
Or, very worst case, you can use one of the JSON Functions to access the information in a JSON field, but this is very inefficient for large requests (eg in a WHERE clause).

Azure HBASE REST - simple get failing for colfam:col

This should be a very simple one (been searching for a solution all day - read a thousand and a half posts).
I put a test row in my HBASE table in hbase shell:
put 'iEngine','testrow','SVA:SourceName','Journal of Fun'
I can get the value for a column family using the REST API in DHC Chrome:
https://ienginemaster.azurehdinsight.net/hbaserest/iEngine/testrow/SVA
I can't seem to get it for the specific cell: https://ienginemaster.azurehdinsight.net/hbaserest/iEngine/testrow/SVA:SourceName
{
"Row": [{
"key": "dGVzdHJvdw==",
"Cell": [{
"column": "U1ZBOlNvdXJjZU5hbWU=",
"timestamp": 1440602453975,
"$": "Sm91cm5hbCBvZiBGdW4="
}]
}]
}
I get back a 400 error.
When successfully asking for just the family, I get back:
I tried replacing the encoded value for SVA:SourceName, and a thousand other things. I'm assuming I'm missing something simple.
Also, the following works:
hbase(main):012:0> get 'iEngine', 'testrow', 'SVA:SourceName'
COLUMN CELL
SVA:SourceName timestamp=1440602453975, value=Journal of Fun
1 row(s) in 0.0120 seconds
hbase(main):013:0>
I opened a case with Microsoft support. I received confirmation that it is a bug (IIS and the colon separator not working). They are working on a fix - they are slightly delayed as the decide on the "best" way to fix it.