How can find all devices (in an Azure IoT-Hub) where reported and desired properties differ? - azure-iot-hub

Question

I'm attempting to query an IoT-hub for devices where reported and desired properties differ. The purpose of this is to be able to notify users when devices fail to update for a given period of time. Running this query
SELECT * FROM c WHERE properties.desired != properties.reported
generates alot of false positives since both desired and reported contains a $metadata property with timestamps that always differ.
So to be clear, I want to list all devices where any of the "real" values (not metadata) differ from desired to reported.

Answer 1

The simply workaround to avoid the '$' properties for comparing between the desired and reported properties is to create a separate complex object within the desired and reported properties. This complex object will represent a state between the real and shadow device.
Example:
"Config": {
"abc": 123,
"status": "inprocess",
"battery": {
"level": 90
}
}
In this case, the query string for query all devices where their Config is different from desired to reported properties looks the following:
SELECT deviceId FROM devices WHERE is_defined(properties.desired.Config) and is_defined(properties.reported.Config) and properties.desired.Config != properties.reported.Config
UPDATE:
Another option (workaround) is using an Azure IoT Hub eventing for changes in the device twin. These notification changes can be routed to the custom endpoint e.g. Event Hub and consumed by EventHubTrigger function. The routing query:
is_object($body.properties.reported) OR is_object($body.properties.desired)
The function can easy obtained a device twin and comparing its properties such as desired and reported after their cleanup metadata parts.
The result of the comparing properties can be stored in the device twin tags, e.g.:
"tags": {
"Watchdog": {
"timestamp": "2019-08-12T14:24:36.1805155Z",
"status": "inprocess"
}
}
Note, that the tags property is not visible by device.
Once we have a watchdog status in the device tags, we can query devices for its status, e.g.:
"query": "SELECT deviceId FROM devices WHERE is_defined(devices.tags.Watchdog) and devices.tags.Watchdog.status='inprocess' "
The following code snippet shows an example of the function:
using Microsoft.Azure.Devices;
using Microsoft.Azure.EventHubs;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System;
using System.Text;
using System.Threading.Tasks;
namespace FunctionApp14
{
public static class Function2
{
static RegistryManager registryManager = RegistryManager.CreateFromConnectionString(Environment.GetEnvironmentVariable("AzureIoTHubShariedAccessPolicy"));
[FunctionName("Function2")]
public static async Task Run([EventHubTrigger("%myTwinChanges%", Connection = "myTwinChangesEventHubConnectionString", ConsumerGroup = "local")]EventData message, ILogger log)
{
var msg = Encoding.UTF8.GetString(message.Body.Array);
log.LogInformation($"C# IoT Hub trigger function processed a message: {msg}");
if (message.SystemProperties["iothub-message-source"]?.ToString() == "twinChangeEvents")
{
var twinChnages = JsonConvert.DeserializeAnonymousType(msg, new { properties = new JObject() });
if (twinChnages?.properties != null)
{
// deviceId
var connectionDeviceId = message.SystemProperties["iothub-connection-device-id"].ToString();
// device twin
var twin = await registryManager.GetTwinAsync(connectionDeviceId);
// cleanup and compare the twin properties
twin.Properties.Desired.ClearMetadata();
twin.Properties.Reported.ClearMetadata();
var desired = JObject.Parse(twin.Properties.Desired.ToJson());
var reported = JObject.Parse(twin.Properties.Reported.ToJson());
var status = JToken.DeepEquals(desired, reported) ? "ok" : "inprocess";
log.LogWarning($"desired-reported status = {status}");
// put the state on the tags
var twinPatch = JsonConvert.SerializeObject(new { tags = new { Watchdog = new { timestamp = DateTime.UtcNow, status = status } } });
await registryManager.UpdateTwinAsync(connectionDeviceId, twinPatch, twin.ETag);
}
}
await Task.CompletedTask;
}
}
}

Answer 2

You might need to do this in code. I had a piece of code that I used for something similar, I put it in a GitHub repo for this question.
The code is pretty naive, it compares the string values of the properties (excluding the metadata). It should be pretty easy to change it to check the property keys/values if needed.