I have a general question concerning serial port communication and storage of the data. When communicating with a serial port (strictly reading from the port in this case) how would one go about storing and manipulating the data in vb.net? For my project I'm doing, I need to take strings from the serial port and then pull numbers from those strings and sort them (numerically, i.e. highest number found at the top and lowest number at the bottom) For some reason in my code I get inner exception errors when I try to move the data from strings to string arrays but I'm determined to figure it out.
As a general question in terms of vb.net programming in relation to serial port communication, is it intelligent to use backgroundworkers? For example, could/should I use a backgroundworker to handle reading from a serial port and then do arithmetic on my data outside of the backgroundworker?
I'm basically just trying to store my data from my serial port into an array, but I don't know how big the array will be that holds the data (i.e. I don't know how many times I'll have data sent to my serial port)
Any tips/info would be appreciated! Thanks
As a general rule if there is going to be any long running task, you should run it in a seperate thread. You do this so that the user experiance is not affected and the GUI stays responsive.
In the case of serial communications there is usually a poll respond architecture which requires constant event handling.
In my experiance I have had great success handling the Interaction with the serial port in a seperate thread that bubbles up events to the GUI. This way I can then process the data to be displayed or stored in another seperate thread and keep the polling running in almost real time.
When I was consuming registers I would store them many different ways but from what you describe, it sounds like the data you are consuming would be best stored in a List(of String).
This structure can be added to almost infinitly and throught the use of predicates can be sorted. The List structure in .net also has a method to convert itself to an array if necessary.
So here is how I can imagine your interaction:
The GUI thread is started and you initate a connection to your device.
You then set up a thread that will be receiveing the incoming communications from the device
In this thread when data is captured it triggers an event in the GUI.
In the GUI event handler the data is stored in a list and if manipulations need to be preformed on it, they are done in another processing thread that will have a call back handler.
In the call back you can then display or push the data to its final destination.
The key points are that if you are using a GUI you should absolutly have the communication in a seperate thread to maintian stability of the GUI and create a good user experiance.
Related
Question
I want to pass data between applications, in a publish-subscribe manner. Data may be produced at a much higher rate than consumed and messages get lost, which is not a problem. Imagine a fast sensor and a slow sensor data processor. For that, I use redis pub/sub and wrote a class which acts as a subscriber, receives every message and puts that into a buffer. The buffer is overwritten when a new message comes in or nullified when the message is requested by the "real" function. So when I ask this class, I immediately get a response (hint that my function is slower than data comes in) or I have to wait (hint that my function is faster than the data).
This works pretty good for the case that data comes in fast. But for data which comes in relatively seldom, let's say every five seconds, this does not work: imagine my consumer gets launched slightly after the producer, the first message is lost and my consumer needs to wait nearly five seconds, until it can start working.
I think I have to solve this with Redis tools. Instead of a pub/sub, I could simply use the get/set methods, thus putting the cache functionality into Redis directly. But then, my consumer would have to poll the database instead of the event magic I have at the moment. Keys could look like "key:timestamp", and my consumer now has to get key:* and compare the timestamps permamently, which I think would cause a lot of load. There is no natural possibility to sleep, since although I don't care about dropped messages (there is nothing I can do about), I do care about delay.
Does someone use Redis for a similar thing and could give me a hint about clever use of Redis tools and data structures?
edit
Ideally, my program flow would look like this:
start the program
retrieve key from Redis
tell Redis, "hey, notify me on changes of key".
launch something asynchronously, with a callback for new messages.
By writing this, an idea came up: The publisher not only publishes message on topic key, but also set key message. This way, an application could initially get and then subscribe.
Good idea or not really?
What I did after I got the answer below (the accepted one)
Keyspace notifications are really what I need here. Redis acts as the primary source for information, my client subscribes to keyspace notifications, which notify the subscribers about events affecting specific keys. Now, in the asynchronous part of my client, I subscribe to notifications about my key of interest. Those notifications set a key_has_updates flag. When I need the value, I get it from Redis and unset the flag. With an unset flag, I know that there is no new value for that key on the server. Without keyspace notifications, this would have been the part where I needed to poll the server. The advantage is that I can use all sorts of data structures, not only the pub/sub mechanism, and a slow joiner which misses the first event is always able to get the initial value, which with pub/sib would have been lost.
When I need the value, I obtain the value from Redis and set the flag to false.
One idea is to push the data to a list (LPUSH) and trim it (LTRIM), so it doesn't grow forever if there are no consumers. On the other end, the consumer would grab items from that list and process them. You can also use keyspace notifications, and be alerted each time an item is added to that queue.
I pass data between application using two native redis command:
rpush and blpop .
"blpop blocks the connection when there are no elements to pop from any of the given lists".
Data are passed in json format, between application using list as queue.
Application that want send data (act as publisher) make a rpush on a list
Application that want receive data (act as subscriber) make a blpop on the same list
The code shuold be (in perl language)
Sender (we assume an hash pass)
#Encode hash in json format
my $json_text = encode_json \%$hash_ref;
#Connect to redis and send to list
my $r = Redis->new(server => "127.0.0.1:6379");
$r->rpush("shared_queue","$json_text");
$r->quit;
Receiver (into a infinite loop)
while (1) {
my $r = Redis->new(server => "127.0.0.1:6379");
my #elem =$r->blpop("shared_queue",0);
#Decode hash element
my $hash_ref=decode_json($elem\[1]);
#make some stuff
}
I find this way very usefull for many reasons:
The element are stored into list, so temporary disabling of receiver has no information loss. When recevier restart, can process all items into the list.
High rate of sender can be handled with multiple instance of receiver.
Multiple sender can send data on unique list. In ths case should be easily implmented a data collector
Receiver process that act as daemon can be monitored with specific tools (e.g. pm2)
From Redis 5, there is new data type called "Streams" which is append-only datastructure. The Redis streams can be used as reliable message queue with both point to point and multicast communication using consumer group concept Redis_Streams_MQ
I am working on a hardware/software application where, connected via usb, is a device that does some off board processing on some data. The application is meant to be open multiple times and which device needs which data is identified by an in-stream parameter. My question is, can more than one application claim an interface? my first implementation used WinUSB but I quickly realized that that limits me to only one instance. The libusb documentation claims that this limitation is removed in their driver.
My concern is, because I intend to have far more than 8 instances running, having only the 8 interfaces allotted will not be sufficient. If I cannot, in fact, claim an interface more than once, is there a method where I could have the applications call a shared library that claims the interface and manages and routes traffic between the applications?
As far as I know you can only have one handle open to a device in either implementation.
I think you are on track in terms of how to handle this problem. The way I have done something like this in the past is to create a service that is to run in the background. This service should be launched by the first instance of the application, and can keep a reference count of it's clients. On your next instance of the application increment your reference count, and whenever a client application closes decrement the reference count. When the last application closes the service can close too.
The service would have the job of opening the device and reading all data in to a buffer. From there you can either put smarts in to the service to process the data and load it in to different shared buffers that are each individually accessible by your other client application instances, or you could simply make one huge buffer available to everyone (but this is a riskier solution).
I've been trying to intercept UDP packets before they reach an applications logic. More precisely, that application is using a DirectPlay Server and there is no source.
So I found out that DirectPlay uses async IO by posting multiple WSARecvFrom, then having some workerthreads waiting with WaitForSingleObject and finally retrieving IO status with WSAGetOverlappedResult.
When WSARecvFrom returns, lpBuffers is not filled with data yet of course, because the operation is still pending and will complete later.
So my idea to get to the data was to save the lpOverlapped/lpBuffers pair in a std::map for every WSARecvFrom call and then, if an IO operation completes (in WSAGetOverlappedResult), I would get to the corresponding (now filled) lpBuffers by looking up the lpOverlapped in the map.
However, there seems to be a big problem: DirectPlay calls WSARecvFrom multiple times with the same lpOverlapped address sometimes, and even with the same lpOverlapped->hEvent or lpBuffers addresses, also for the same socket (none of these operations complete at this time, so they are all pending). I cannot understand why this happens, the doc clearly says: "If multiple I/O operations are simultaneously outstanding, each must reference a separate WSAOVERLAPPED structure."
Because of this I cannot correctly retrieve the lpBuffers, because when WSAGetOverlappedResult is called, I don't know to which WSARecvFrom the lpOverlapped corresponds because there were several WSARecvFroms called, each with the same lpOverlapped! How can this be? Does anyone know how DirectPlay handles this? Could there be another way intercepting (and eventually dropping) UDP Packets? (I don't want to use drivers)
(There is a good reason why I'm trying to do this: Someone is sending exploited UDP packets to a gameserver using DirectPlay, and it "confuses" the DirectPlay logic, basically shutting down the server. So I have to filter out specific UDP packets before they even reach DirectPlay)
Happy for any hint!
Thanks a lot!
hi guys i'm making a client-server software and this is my first question
i'd like to ask: how to distinguish data that sended by tcp Connection?
Well, my points are:
-we can determine data that sended by tcpconnection.
for example, we have 3 Listviews in our form
the point of the first listview is for Biodata of client.
the point of second listview is for *The value obtained from the clients
n the point of third listview is for The picture obtained from the clients
in this case we have 3 main points that must be processed.
in fact, we only have 1 connection in our system.
Well, here I'm confused..
how to determine that data we received is for the first listview or second listview or third listview?
remember, the data of third listview is a picture that we received from tcpconnection
How do we do that with 1 connection in our system?
do i have to make 3 connection to control third listviews?
With socket communication, both the client and the server must use the same agreed-upon protocol so that they can understand each other. There are many standard protocols that have already been created, so for most tasks, creating your own protocol is unnecessary. However, if you must, you can always define your own protocol. The nature of your protocol will obviously depend completely on your specific needs, so it would be impossible to tell you what the protocol should be. However, generally speaking, the first thing your protocol must define is how to know where each complete message begins and ends. This is often accomplished by separating each message with a delimiter (e.g. new line, EOF, null). As Francois recommended, you could alternatively specify the length of the message at the beginning of each message. Within each message, you then will need a header section which, among other things, would specify the type (the format) of the data stored in the body of the message.
A simple implementation might be to send each message as a JSON or XML document. Doing so makes it very easy to define and modify the format of the message.
However, unless you really need to, I would recommend using one of the built-in communication frameworks that are provided with .NET. For simple tasks, often a simple asmx web service is sufficient. For more complex tasks, often WCF is a good choice. An asmx web service uses SOAP via HTTP via TCP/IP. WCF uses SOAP, but the lower level connection is configurable so it doesn't have to use TCP/IP, but it can easily do so.
I have an issue with a 3rd party DLL, which is NOT thread-safe, but which I need to call within an orchestration.
I'm making the DLL call within an Expression shape. The same DLL is called in a number of different orchestrations.
The problem I have is that for a series of incoming messages, BizTalk will run multiple orchestrations (or multiple instances of an orchestration) in parallel - which leads to exceptions within the DLL.
Is there any way around this, given that refactoring the DLL isn't an option. Or, is there a way to throttle BizTalk to only run one orchestration at any one time. (I've seen some hacks restricting the working pool to the number of processors, but this doesn't seem to help. We can't downgrade to a single-core machine!)
I would rather find a way of keeping the DLL happy (though I can't think how) than throttle BizTalk - but if there is a way to throttle that would be an acceptable short-term solution whilst we discuss with the 3rd party. (who are a large organisation and really should know better!)
Even on a single core machine, BizTalk will run concurrent orchestrations.
You could throttle the orchestration by implementing the singleton pattern in the orchestration.
You do this by creating a loop in the orchestration and having two receive shapes, one before the start of the loop and one inside the loop.
Both these receive are bound to the same inbound logical port.
You create a correlation set which specifies something like BTS.MessageType and set the first receive shape to initiate the correlation and the second receive to follow the correlation.
As long as the loop does not end you can guarantee that any message of a certain type will always be processed by the same instance of the orchestration.
However, using singletons is a design decision which comes with drawbacks. For example, throughput suffers, and you have to ensure that your singleton cannot suspend, else it will create a block for all subsequent messages.
Hope this helps.