iotedge: How to requeue message that could not be processed - azure-iot-hub

There are publisher and consumer custom modules that are running on an Edge IoT device. The publisher module keeps producing messages at constant rate no matter consumer module processes it or not. The consumer module POSTs the message to external service and given that there is no Internet connection, the consumer module would like to requeue the messsage so that is not lost and tried again.
I do not prefer to write an infinite loop to keep retrying; also if the module is restarted the message would be lost. So i prefer to requeue the message to edgeHub/RocksDB.
Where do I find documentation on available responses that can be provided for IoTHubMessageDispositionResult? what is the response to be sent if message needs to be requeued?
if message.processed():
return IoTHubMessageDispositionResult.ACCEPTED
else:
return IoTHubMessageDispositionResult.??

You don't have to implement your own requeuing of messages. IotEdge provides offline functionality as described in this blog post and on this documentation page.
The edgeHub will locally store messages on the edgeDevice if there is no connection to the IotHub. It will automatically start sending those messages (in the correct order) once the connection is established again.
You can configure how long edgeHub will buffer messages like this:
"$edgeHub": {
"properties.desired": {
"schemaVersion": "1.0",
"routes": {},
"storeAndForwardConfiguration": {
"timeToLiveSecs": 7200
}
}
}
The 7200 seconds (2 hours) is also the default if you don't configure anything.
By default, the messages will be written to a folder within the edgeHub docker container. If you want to store them somewhere else you can do so with this configuration:
"edgeHub": {
"type": "docker",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-hub:1.0",
"createOptions": {
"HostConfig": {
"Binds": ["<HostStoragePath>:<ModuleStoragePath>"],
"PortBindings": {
"8883/tcp": [{"HostPort":"8883"}],
"443/tcp": [{"HostPort":"443"}],
"5671/tcp": [{"HostPort":"5671"}]
}
}
}
},
"env": {
"storageFolder": {
"value": "<ModuleStoragePath>"
}
},
"status": "running",
"restartPolicy": "always"
}
Replace HostStoragePath and ModuleStoragePath with the wanted values. Example:
"createOptions": {
"HostConfig": {
"Binds": [
"/etc/iotedge/storage/:/iotedge/storage/"
],
...
}
}
},
"env": {
"storageFolder": {
"value": "/iotedge/storage/"
},
...
Please note that you probably have to manually give the iotEdge user (or all users) access to that folder (using chmod).
Update:
If you are just looking for the available values of IoTHubMessageDispositionResult you will find the answer here:
class IoTHubMessageDispositionResult(Enum):
ACCEPTED = 0
REJECTED = 1
ABANDONED = 2
Update 2:
Messages that have been ACCEPTED are removed from the message queue because they have been successfully delivered.
Messages that are ABANDONED are added to the message queue again and the module will try to send it again as defined in the retryPolicy. For more insight on the retryPolicy you can read this thread.
Messages that are REJECTED are not added to the message queue again.

Related

kucoin websocket api, how to "subscribe" to their public channel, they say no authorization required, but they ask for a token :(

The question is about kucoin websocket public channel (not trades) just last trades
I just want a live feed of trades like other crypto exchanges...
but when I want to connect to "wss://ws-api-futures.kucoin.com/endpoint" I get WebSocketError: Received unexpected status code (401 Unauthorized)
the documentation https://docs.kucoin.com/futures/#create-connection lack explications :(
normally with other exchanges I can just do this in javascript
bybit_market_ws = new WebSocket("wss://stream.bybit.com/spot/quote/ws/v2");
bybit_market_ws.onmessage = event => bybit_trades(event.data);
bybit_market_ws.onopen = event => bybit_market_ws.send(JSON.stringify({"topic":"trade","params":{"symbol":"BTCUSDT","binary":false},"event":"sub"}));
function bybit_trades (jsonx) { console.log(JSON.parse(jsonx)); }
so how can I do that with kucoin websocket ?
according to the documentation i would need a "public token"...
but there is no explication on how to get that token :(
does someone knows how I would retrieve the last trades via websocket (public) channel ?
Note that the following steps may be changed when the API is updated.
All information can be found at https://docs.kucoin.com/#apply-connect-token
Get the public token
Send a empty http POST (GET will not work) message to https://api.kucoin.com/api/v1/bullet-public.
Response:
{
"code": "200000",
"data": {
"token": "2neAiuYvAU61ZD...",
"instanceServers": [
{
"endpoint": "wss://ws-api.kucoin.com/endpoint",
"encrypt": true,
"protocol": "websocket",
"pingInterval": 18000,
"pingTimeout": 10000
}
]
}
}
Connect to the Websocket
With the data of the repsonse above:
websocket: endpoint + "?token=" + token
Example: wss://ws-api.kucoin.com/endpoint?token=2neAiu....
Get all supported trading pairs
send a http GET message to https://api.kucoin.com/api/v1/symbols
{
"code": "200000",
"data": [
{
"symbol": "REQ-ETH",
"name": "REQ-ETH",
"baseCurrency": "REQ",
"quoteCurrency": "ETH",
...
},
{
"symbol": "BTC-USDC",
"name": "BTC-USDC",
"baseCurrency": "BTC",
"quoteCurrency": "USDC",
...
},
...
Get trading data
When the websocket connection is established send a http POST message:
{
"type": "subscribe", //subscribe or unsubscribe
"topic": "/market/ticker:BTC-USDT,BTC-USDC"
}
maybe this answer will not please you at all, but i will try, most of the people who work from the API in KuCoin do it with python, in fact the SDK for Nodejs is out of date, your best bet is to ask in the telegram channel https://t.me/KuCoin_API, there are KuCoin engineers who always help, although most of them use python, there is also the academy channel https://t.me/kucoin_learning, where there are examples, in short I can only mention references because I was also where you are, and the best I could do was that and review the SDk code and from there intuit and create my own adjustments
PD: the datafeed.js file is your best option, check it out https://github.com/Kucoin/kucoin-futures-node-sdk/blob/master/src/lib/datafeed.js

Firebase Cloud Messaging Reports - not displaying sent count in Reports

We have set "analytics_label" in the message as stated in the documentation and the message is getting delivered as well. But we do not see any entry in the report. Please check our message string and let us know what might be wrong. Appreciate your help.
REST API being called
https://fcm.googleapis.com/fcm/send
Message being sent
{"topic":"81xxxxx42","android":{"priority":"high"},"priority":"high","fcm_options":{"analytics_label":"nwy81xxxxx42"},"data":{"MID":-1,"frm":"99xxxxx32","MTP":9,"msg":""}}
I'm not certain what library you're using or if you're just POSTing directly to the REST API, but looking at code that I know works I think you just need to make fcm_options and analytics_label camel case.
{
"topic": "81xxxxx42",
"android": {
"priority": "high"
},
"priority": "high",
"fcmOptions": {
"analyticsLabel": "nwy81xxxxx42"
},
"data": {
"MID": -1,
"frm": "99xxxxx32",
"MTP": 9,
"msg": ""
}
}

Fiware Orion context broker subscription notification issue

I'm using orion context broker GE image orion-psb-image-R5.4 version 1.7.0 and I registered a device entity in it , then i implemented in my raspberry pi simple python server script that listens to any incoming message and print it on the Pi's logs . then i sent a subscription message to the context broker to let my raspberry pi subscribe to its corresponding entity in the context broker. The issue is that whenever i update the condition attributes in the entity in the context broker , they're supposed to trigger a notification to the raspberry pi and then the server script in the PI print the notification in the Pi's logs . But what really happens is that the context broker may trigger the notification for several times and then suddenly stops sending any notification when any additional change is applied to the condition attribute , and on every attempt i make i retrieve the subscription status in the context broker and i find that there was a failure stated by the lastfailure attribute giving me the time of my last failed attempt.
I thought the problem could be the connection to my Pi or even in the server script itself but when i launched direct requests from my terminal to the raspberry pi , it prints the all messages immediately even when the update is made from a remote place . So i concluded that the problem is definitely with the context broker and the notification process of the subscription itself .
Here's the subscription request i made :
curl -v contextbrokeraddress:1026/v2/subscriptions -s -S --header "Fiware-Service: XYZ" --header "Fiware-ServicePath: /XYZ" --header 'Content-Type: application/json' \
-d #- <<EOF
{
"description": " Try",
"subject": {
"entities": [
{
"id": "Controller1",
"type": "Controller"
}
],
"condition": {
"attrs": [
"switch",
"datashow"
]
}
},
"notification": {
"http": {
"url": "http://raspberryPiaddress:8080"
},
"attrs": [
"switch",
"datashow"
]
},
"expires": "2040-01-01T14:00:00.00Z",
"throttling": 5
}
EOF
now when the switch attribute is updated with a different value , it may trigger the notification to the raspberry pi for the first time only but then fails on any following attempts.
this is the simple python script that listens to the incoming notifications and print it in its logs:
import socket
HOST, PORT = '', 8080
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listen_socket.bind((HOST, PORT))
listen_socket.listen(1)
while True:
print "listening on port 8080"
client_connection, client_address = listen_socket.accept()
print "notified"
request = client_connection.recv(1024)
print request
client_connection.close()
And this is how the notification from the context broker is printed on the Pi's logs in its successful times:
listening on port 8080
notified
POST / HTTP/1.1
User-Agent: orion/1.7.0 libcurl/7.19.7
Host: raspberryPiaddress:8080
fiware-service: XYZ
Fiware-ServicePath: /XYZ
X-Auth-Token: token
Accept: application/json
Content-length: 208
Content-type: application/json; charset=utf-8
Fiware-Correlator: f48ced60-1069-11e7-b743-fa163e7c4daf
Ngsiv2-AttrsFormat: normalized
{"subscriptionId":"58cd49191e9c000de6ea89c7","data":[{"id":"Controller1","type":"Controller","switch":{"type":"command","value":"OFF","metadata":{}},"datashow":{"type":"string","value":"OFF","metadata":{}}}]}
And when the notification is not received on any following updates on the entity , i check that it was failure by retrieving the subscription status which states that a failure happened in the context broker on sending the notification at the exact time of my attempt. This the status i retrieve to check that:
[
{
"description": " Try",
"expires": "2040-01-01T14:00:00.00Z",
"id": "58cd49191e9c000de6ea89c7",
"notification": {
"attrs": [
"switch",
"datashow"
],
"attrsFormat": "normalized",
"http": {
"url": "http://ahmadpi.ddns.net:8080"
},
**"lastFailure": "2017-03-24T08:22:23.00Z",**
"lastNotification": "2017-03-24T08:22:18.00Z",
"lastSuccess": "2017-03-23T22:09:33.00Z",
"timesSent": 66
},
"status": "failed",
"subject": {
"condition": {
"attrs": [
"switch",
"datashow"
]
},
"entities": [
{
"id": "Controller1",
"type": "Controller"
}
]
},
"throttling": 5
}
]
the problem now seems to be relating to the context broker and the way the subscription/notification processes are handled inside it . Now, i want to know whether the problem regards the context broker image version that i used or whether it regards something else . I just want to know where's the problem and how it can be handled please and thanks so much.
Although I'm not fully sure as don't have all the inputs (specially, CB log traces), by "it worked well as I said but sometimes stops for some reasons" (see comments thread in the question post) I tend to think is some networking/connectivity problem, not directly related with Orion Context Broker.

Can we receive step data in the pub sub payload?

I can see in the payload JSON that the pubsub notification includes the events but is it also capable of sending the activity or sleep amounts (e.g steps or sleep minutes)?
No, the pubsub notification will only include the user and the list of events that took place. You can then use this information to query the APIs that pertain to those specific types of events.
For example, suppose you receive the following pubsub notification:
{
"notification_timestamp": "1372787949",
"events": [
{
"user_xid": "RGaCBFg9CsB83FsEcMY44A",
"event_xid": "EJpCkyAtwoO0XTdkYyuTNw",
"type": "move",
"action": "creation" ,
"timestamp": "1372787849"
},
{
"user_xid": "RGaCBFg9CsB83FsEcMY44A",
"event_xid": "blaHyAtwoO0XTdkYyuTNw",
"type": "sleep",
"action": "updation" ,
"timestamp": "1372787859"
}
],
"secret_hash": "e570b3071a0964f9e2e69d13nd9ba19535392aaa",
}
Then you know that a move event was created and that a sleep event was updated for the user specified by user_xid: RGaCBFg9CsB83FsEcMY44A
Use this user's xid to determine which previously saved access_token to send with your next API requests.
You can get the details for the move event by sending a request to the moves endpoint with the event_xid: EJpCkyAtwoO0XTdkYyuTNw:
GET https://jawbone.com/nudge/api/v.1.1/moves/EJpCkyAtwoO0XTdkYyuTNw
And yo ucan get the details for the sleep event by querying the sleeps endpoint with the event_xid: blaHyAtwoO0XTdkYyuTN:
GET https://jawbone.com/nudge/api/v.1.1/sleeps/blaHyAtwoO0XTdkYyuTN

Pushwoosh can not delete message created by API

EDIT: as noted in the answer below, this was a problem on the pushwoosh side, it has been fixed!
When I create a push message through the pushwoosh API (using /createTargetedMessage) I'm not able to delete the message through the API. Messages made with the pushwoosh interface can be deleted through the API, no prob...
These are the steps I took to produce this error:
1. Create push message with the following params
{
"request":{
"auth":"AUTH TOKEN",
"send_date":"2015-09-22 15:07",
"content":{
"nl":"teststsdfgh",
"en":"teststsdfgh"
},
"devices_filter":"A(\"8A1EB-4E875\") * T(\"inholidaypark\", BETWEEN, [\"2015-09-22 00:00\",\"2015-09-22 23:59\"]) * T(\"Language\", IN, [\"nl\", \"en\"])"
}
}
2. This returns the following response; the messageCode is stored in our local DB for later use
{
"status":200,
"response":{
"status_code":200,
"status_message":"OK",
"response":
"messageCode":"D3F6-60769243-68B30EA8"
}
}
}
3. Call /deleteMessage with following data
{
"request":{
"auth":"AUTH TOKEN",
"message": "D3F6-60769243-68B30EA8"
}
}
4. API keeps returning:
{
"status_code": 210,
"status_message": "Message not found",
"response": null
}
But when I look at the push history the message is there (with the same messageCode and all). And it can be deleted through the pushwoosh interface, but not through the API.
On a side note: when the message is sent, we can obviously no longer delete it, then the API returns a more or less correct error:
{
"status_code": 210,
"status_message": "Forbidden",
"response": null
}
Just FYI for the rest of the readers, this issue has been identified and fixed on Pushwoosh side.
Move Along, Nothing to See Here. :)