I create an IotHub and create an eventprocesshost to receive the events from the hub in a console application, then I create an ASA job and make the Iothub as the input with the default consumer group, but when I start the job, I get the exception:
There is more than one receiver with different epoch value connected to the same Event Hub. Please ensure Service Bus Explorer or an EventProcessorHost application is not connected while Stream Analytics job is running.
So the job input and the event processor host use the same consumer group make this happen???
The input and host cannot use the same consumer group???
You can't have EPH and ASA job running side by side due to Epoch conflict. Please close your EPH application and then restart ASA job.
Related
We are trying to use the Continuous Query feature of Ignite. But we are facing an issue on handling that event. Below is our problem statement
We have defined a Continuous Query with remote filter for a cache and shared the filter definition with Thick Client.
We are running multiple replica of the "Thin Client" in Kubernetes cluster.
Now the problem is each instance of the "Thin Client" running in k8s cluster have registered the remote filter and each instance receiving the event and trying to process the data in parallel. This resulting in duplicating data process or even overriding the data in my store.
Is there any way to form a consumer group and ensure that only one instance of the "Thin Client" is receiving the notification and its processing the data ?
My Thick client and Thin Clients are in .NET
Couldn't found any details on Ignite document
https://ignite.apache.org/docs/latest/key-value-api/continuous-queries
Here each thin client is starting its own continuous query and thereby, by design, each thin client is getting its own event to consume. If you want to route an event to a specific client then you would need to start only one continuous query, and distribute that event to your app as you see fit.
Take a look at ignite messaging to see whether it fits your use case.
Also check out the distributed Queue/Set which have unique delivery guarantees.
I recently started a project with Azure IoT Edge with the IoT Hub Free Tier so I'm a total beginner. I set up a device sensor, a module and I am sucessfully sending data to my module and IoT Hub. I can
monitor the messages sent from the sensor with Azure IoT Hub expension from Visual Studio Code.
I can see the messages I'm sending but I am having an issue with the number of messages being sent.
I use Azure portal metrics to monitor the number of messages sent and very often Azure would show me different numbers as I refresh. For example "1000" messages and after a refresh "800" messages etc...
Another issue I'm having is also that when using the Metrics functionality, it shows me that some messages are being sent during a time where my sensors weren't sending any messages.
Is there a way to get a detailled report with a history on the messages that the Hub receives?
Any help or advice would be highly appreciated! Thank you
As far as I know there is no "nice and simple" report which will show you what you need. However, if you want to get historical events which IoT hub processed it can be done. Note, though, that history is limited and can be no longer than 7 days. Current retention period can be seen on the azure portal in "Built-in endpoints" section - there is a "Retain for" setting with the value from 1 day (default) to 7 days.
If events are within this range, you can use Powershell command "az iot hub monitor-events" with the --enqueued-time parameter to "look back" in history. The time is specified in milliseconds since unix epoch. Example command:
az iot hub monitor-events --hub-name 'your_iot_hub_name' --device-id 'your_registered_device_id' --properties all --enqueued-time 1593734400000
I'm having APi Service that handle some information from my device. After that i use default Iot-HUB SDK that allows me to send data into Iot-Hub. In my event hub I usually get data that I send more than once. Is there any options to see what it happens ? Maybe some extra settings that to retransmission or something that send messages more than once from my web api service ?
I had the same problem with my code. I was sending messages to an iothub using an Azure function. I had the message sending time trigger as a variable. When sending messages in shorter frequencies (i.e. messages that are sent each second) some of these messages are duplicated.
By changing the message protocol it solved. Given below is the corrected code,
DeviceClient deviceClient = DeviceClient.CreateFromConnectionString(connectionString, TransportType.Http1);
I followed a IOT Hub tutorial and got it working. I then created a Stream Analytics job and used the above as an Input (which upon test connection works).
However I do not see any inputs being received. When running a sample test I get the following error:
Description Error code: ServiceUnavailable Error message: Unable to
connect to input source at the moment. Please check if the input
source is available and if it has not hit connection limits.
I can see telemetry messages being received in the IOT Hub. Any help would be appreciated
Is the stream analytics job running?
I had a similar problem where i wasn't getting any events from stream analytics and i had forgotten to turn it on.
Click on the stream analytics > overview > start
I had the same problem (using Event Hubs in my case). The root cause was that I had too many queries within my job running against the same input. I solved it by splitting my input into several inputs across multiple consumer groups.
From the documentation (emphasis added):
Each Stream Analytics IoT Hub input should be configured to have its own consumer group. When a job contains a self-join or multiple inputs, some input may be read by more than one reader downstream, which impacts the number of readers in a single consumer group. To avoid exceeding IoT Hub limit of 5 readers per consumer group per partition, it is a best practice to designate a consumer group for each Stream Analytics job.
I have exactly the same problem, though my modules on my raspberry pi are running without failure.
SA says: "Make sure the input has recently received data and the correct format of those events has been selected.
All ,
We have a requirement to develop a azure based platform, in which the user can configure multiple pharmaceutical instruments, start measurements on them and analyze the measured data. The typical components in the azure based platform will be following
1 - A .NET based 4 client application running on the computer connected to each instrument. This client application should receive the start measurement command from the azure platform , perform the measurement and update the result back to the azure*
2 - A set of services[probably REST based] which will get the results from the client application and update the database on the cloud
3 - A set of services and business logic which which can be used to perform analysis on the data
4 - A asp.net web application where the user can view instrument details , start measurement etc
There is a two way communication between the Azure platform and the client application i.e. the client needs to update results to the azure and the azure needs to initiate measurement on the instrument via the client application
In such a scenario , what is the recommended approach for the azure platform to communicate to the clients. Is it any of the following
1 - Create a duplex service between the client and server and provide a call back interface to start the measurement
2 - Create a command queue using Azure message queue for each client. when a measurement needs to be started , a message will the put on the queue. The client app will always read from the queue and execute the command
or do we have any other ways to do this , any help is appreciated
We do not fully understand your scenario and constraints around it, but as pointers, we have seen lot of customers use Azure storage queues to implement master-worker scenario (some component adds message to appropriate queue to get work done (take measurements in your case) and workers polling the queue to process this work (client computer connected to your instrument in this case)).
In terms of storing the results back, your master component could provide SAS access to client to write results back to specific blob in an Azure storage account and either have your service and business logic monitor existence of that blob to start your analysis.
Above approach will decouple your client from server and make communication asynchronous via storage. Again, these are just pointers and you would be the best person to pick the right approach that suits your requirement
For communication between the server and the client, you could use SignalR http://signalr.net/ there are two forms of messaging systems supported "as a service" on Azure, these are Service Bus and Message Queues - see this link http://msdn.microsoft.com/en-us/library/hh767287.aspx