I’m currently working on bidirectional communication between some IoT sensors and a mobile app using Pusher channels.
I have been able to get Pusher working on the IoT nodes using the Arduino library and also in my React Native app.
However, I’ve hit a bit of a roadblock. My IoT sensors aren’t capable of producing JSON payloads and arrays (they broadcast batches of 30 readings at 30 second intervals) due to memory constraints. Each reading can be up to 60 characters in length unprocessed and a full payload being sent every 30s would be at least 1800 bytes, then there’s the header data (Auth token and session data for the sensor’s context).
I do not want to parse this hex data on my React Native app (as some comes from proprietary sensors where the protocol cannot be risked being divulged) so need it to happen on my ExpressJS API (currently handles the authentication and historic data retrieval) before it enters Pusher and pings the React Native app.
My questions:
If the sensor made a POST request to my Express API in which the route performed the processing from hex into nice JSON (with full text values!) - can I use the Pusher client to get this data into the correct channel? Instead of having the sensor talk straight to Pusher?
Is there a way to bridge the Pusher service through my API so that the React Native app “points” to my API for the updates instead of directly to Pusher?
Here is the architecture that I’m trying to achieve - hoping those of you who have experience with Pusher can tell me if this is possible:
I’ve seen the “pusher-http-node” server library but there’s no tangible description of why this would be used.
Really hoping I don’t need to go down the MQTT route and have my own micro service (something I’ve wanted to avoid given the costs and scalability concerns).
DevRel at Pusher here.
To answer your questions:
That setup makes sense. Your Express API can preprocess each sensor event before sending them to the Channels service - you'd use the Node SDK on the server. That preprocessing can include putting it in the correct format, and sending to the correct channel.
For the difference between SDKs:
The pusher-http-node SDK is designed to run on servers, and can send messages to any channels you want.
It also holds your private keys - something that your React
Native client SDK doesn't.
Server SDK doesn't implement subscriptions.
The pusher-js client SDK that you use in the React Native apps on the other hand can only subscribe or send client events.
Your client applications should connect to Pusher Channels service so they can receive realtime updates. Channels serves as the transport here. Your Express API should use pusher-http-node to send these events to Pusher Channels.
So, to summarise:
Have your IOT sensors communicate raw data to your Express API
In your Express API do the preprocessing of these raw events, turning them into something your React Native apps can understand and use
In your Express API use the pusher-http-node library (server SDK) to send these processed events to Pusher Channels service
In your React Native apps use the pusher-js library (client SDK) to connect to Channels service and subscribe to the events your Express API is sending.
I hope this clarifies it a bit!
Related
Currently I am using SignalR for push notification, but due to cost constrain looking for the new alternative. When I read about gRPC its mostly used for microservice communication. Will this gRPC is suitable for web push notification? is there any example I can find. Thanks.
Using agora.io I'd like to do something like realtime subtitles for video meetings.
Is there a way to have a server get at the data of the video/audio streams?
The server REST api seems very basic just for project management.
I want to get at the streaming audio data, not a local client capture like this demo
Perhaps I could use something like the cloud proxy although I would need to "T Pipe" the data, and I assume the data would be encrypted, not sure if the keys used are something I can get access to.
It also seems like the web client has browser dependencies; perhaps I could run that on a headless/chrome node server, but that seems like a hack and very resource intensive.
Any other suggestions on places I can look for APIs?
Also interested in other webrtc APIs and provider alternatives. Maybe Twilio has some APIs for this.
We provide Linux SDK on a case by case basis which you can deploy headless in your server. This SDK itself will act as a client that joins the channel and receives the video streams in the channel.
The SDK is written in c++ and is not public. you can request access by opening a support ticket at https://agora-ticket.agora.io/
I would like to use Agora Signalling SDK to create a chat app.
I am aware that I can use the onMessageChannelReceive and onMessageInstantReceive callbacks to receive messages that were sent.
Is it possible to get messages that were sent previously? If a user logouts and logs in back, is there a way to get the messages that were sent previously?
Currently the Agora Signaling SDK does not support persistent messages. There is a new Realtime Messaging SDK that is in beta that will have more robust features. I would recommend you reach out to the Agora.io team to get more information because I think it will help with your use case.
Problem description
i am working on a Xamarin application that consumes a REST API written in Python flask.
The Xamarin application offers virtual shopping lists where user can collaborate on buying stuff they have on a shared list.
To improve the user experience, i want to be able to actively notify the user about finished items on the list.
Possible solutions:
Synchronous API polling from client side
Notifications are stored by the API in a relational database and have a flag indicating if the user received the notification already.
The API has an endpoint GET /users/:user_id/notifications/ that queries the database for notifications and returns a JSON response with those.
Advantages
fairly simple to implement
Problems
synchronous polling creates a huge amount of http requests
API service remains stateless, making a horizontal scaling with a loadbalancer easier
Websocket endpoint on the API
The API has an endpoint POST /users/:user_id/notifications/register which creates a websocket connection between client and API.
The connection is stored to a global array in which each entry maps a client id to a websocket connection.
When a new notification is created, the endpoint makes a lookup in the connection dictionary by comparing the owner id of the notification with the dictionary entries. The notification is sent to appropriate user through the websocket.
Notifications are stored in the database like in the first approach.
When a user calls the endpoint, a new websocket connection will be established first and upon success the API sends all unseen notifications from the database to the user.
Advantages
API can push notifications to clients asynchronously
Problems
When a user terminates the websocket connection his dictionary entry will persis
Retaining one websocket connection per user permanently adds additional overhead to the API
Horizontal scalability of the API is more difficult because the service is not stateless anymore (Websocket connection information saved in
RabbitMQ
The API uses a RabbitMQ service to send notifications to the client. Every client uses subscribes to his own notification queue to prevent the broadcasting of messages.
Advantages
API remains stateless
Problems
Notifications needs to be resend to the exchange when a user is offline
Amount of queues grows drastically
Additional costs for RabbitMQ service
High temporary load on the RabbitMQ service when many users come online in the same time
Final words
It would be interesting to hear the opinion of others.
I believe the active distribution of notifications from backen services to clients i a very common use case.
best,
D
I would use RabbitMQ and consume events forwarding them as push notifications. This will work while the user is not actively connected to the website and enhance the engagement with each user experience that will return to the website when notified for more information see How to setup basic web push notification functionality using a Flask backend or How to send push notifications to a browser in ASP.NET Core or Sending Notifications with Spring Boot, Angular, and Firebase Cloud Messaging this way the RabbitMQ will not wait until the user is back online. If the user is online you can forward the notification directly to the Xamarin application via WebSockets and a load balancer like NGINX that can handle many WebSockets in an optimized way.
Synchronous API polling from the client-side is the less preferred way since it overloads the webserver with requests while nothing was changed.
I don't think the scalability of WebSocket is a problem. You can scale up easily with pub/sub. The hotspot of long connections is a kind of serious problem.
For one-way communication, I would suggest Server sent event. In the end, it usually depends on what your team is confident with.
I can recommend on a different approach for API that provides JSON which is called GraphQL
It supports subscriptions capabilities that are pushed by the GraphQL API Server (using web sockets)
GraphQL is considered today to be better than RESTful API since its very flexible and you can get exactly the data you need with one query.
We (me and my team) are building a food delivery real-time Web and Mobile app (using react-native) which also includes payment integration and admin dashboard for a client.
The tech stack we've chosen for the app:
View
React (Web app)
React-native (Mobile app)
Back-end
Express
Firebase
We thought of sharing the app data using a common back-end for the web and mobile app. Basically, we would've created an API that provides the end-points using Express and then Express would Save/Retrieve data to/from Firebase. Express would be our middle-ware.
We created 2 project folders keeping first of all only web app in mind:
react-webapp
express-webapp
And then, we start the respective server for the packages.
Unfortunately, API's aren't real-time and we may have to implement our mechanism to make the flow real-time.
So, we switched to merging firebase with react. We decided that'll use express just for sending emails. So, the folder structure for the web app is something like:
react-webapp
node_modules
public
src
firebase
With this approach, we created a demo and we do get real-time updates and we can also use ReactFireMixin. Later we can use the same folder and add it to react-native as well for Saving/Retrieving data from the database.
My question is, as we don't have any prior experience of building a Web and Mobile app with a common database/back-end and React/React-native, is this approach apt? Is there anyway in which we can segregate the front-end code from the back-end and utilize the real-time feature of firebase?
The reason for segregating the backend from front-end is to keep a common real-time backend for react and react-native without having to keep 2 separate firebase folders for the web and mobile app.
Note: If you are wondering why real-time then the client has asked for a real-time order placement mechanism.
It may work to use socket.io with express to allow for realtime back end updates that come from firebase.