I'm coding a simple TCP client/server in VB.NET to transfer files of all sizes. I decided to use System.Net.Sockets's command SendFile to transfer the bytes through the socket.
On the receiving side, my code to retrieve the bytes works fairly well, but occasionally the transfer randomly stops.
I figured out that by putting a small sleep delay between retrieving the next block of data makes the transfers 100% stable.
My code to retrieve the data (until there is no data available) is simplified as this:
While newSocket.Available > 0
Threading.Thread.Sleep(100)
newSocket.ReceiveFrom(data, Remote)
End While
I really hate using that sleep delay and figure there must be a proper method/function to retrieve data from SendFile?
Socket.Available returns the total number of bytes that have been received so far that have not yet been read. Therefore, if you read the data faster than its coming in (which is quite possible on a slow network), there will be no more data to read even though the client is still in the middle of sending the data.
If the client makes a new connection to the server for each file it sends, you could simply change it to something like this:
While newSocket.Connected
If newSocket.Available > 0 Then
newSocket.ReceiveFrom(data, Remote)
End If
End While
However, I would suggest using the asynchronous calls, instead, such as BeginReceive. Then, your delegate will be called as soon as there is data to be processed, rather than waiting in a constant loop. See this link for an example:
http://msdn.microsoft.com/en-us/library/dxkwh6zw.aspx
Related
I'm writing a program in LabVIEW 2014 in order to control a linear actuator. The program is very simple, it sets a speed and then runs the subVIs to move the actuator back and forth.
There is a case structure inside a while loop so it would stop when a desired number or iterations is reached. The problem is that the iteration count of the while loop occurs faster than the execution of the program inside the case structure, and therefore the program stops before all the cycles of movement have been completed.
send pulses subVI:
activate subVI:
I tried different time delays in different parts of the code, but none of that worked. I think that the issue is that the while loop iterations run faster than the code of the case structure and somehow I need to slow it down. Or maybe I'm wrong and it is a complete different thing.
Here is the link of the actuator documentation:
https://jp.optosigma.com/html/en_jp/software/motorize/manual_en/SRC-101_InstructionManual_Ver1_1_EN.pdf
Welcome to the fun and infuriating world of interfacing to serial instruments.
Each iteration of a LabVIEW loop can only complete once all the code inside the loop structure has completed, so it's not possible that 'the while loop iterations run faster than the code of the case structure'. There's nothing explicitly wrong with any of your code, but evidently it isn't doing what you expected it to. The way to approach developing an instrument driver is always to start with the simplest case (e.g. one single movement of your actuator), get that working, and build up from there.
The documentation for an instrument's serial interface is rarely perfect and yours is no exception, but it does tell us that
every command is acknowledged by a response, and
you should not send a new command until you have received the response from the previous command.
Your code to send commands and receive the response looks OK. A VISA Read operation will read bytes from the computer's serial buffer until either the number of bytes to read is reached, or a byte matching the termination char is read, or the timeout expires. The manual implies that the instrument's responses are followed by the CR and LF characters, and the default configuration of the serial port in LabVIEW is to terminate each read when an LF is received, so you shouldn't need a time delay between each write and the following read; the instrument's response will be received into the buffer by the OS, then your code will read it out and return it as soon as it hits the LF.
What isn't completely clear from the manual is how the instrument responds to the activation command, G: - does it
Return the acknowledgement immediately, then execute the movement: you can check whether the movement is finished using the status command !:, or
Execute the movement, then return the acknowledgement to show that it's finished.
I think it's 1. but that's the first thing I would check. Unless all your movements are completed in less than 500 ms then I think this is what is wrong here: your program receives the acknowledgement then moves straight on to send the next command, but the actuator is still moving and not ready. In this case you have two options:
add a time delay after the read, calculated to be long enough for the actuator move to finish - this would be easiest, but potentially unreliable
in a While loop after you have got the acknowledgement of the G: command, send the !: command and check the response until you get R for 'ready'. (Remember that the acknowledgement string you receive will also have the CRLF on the end.) Use a time delay in this loop so you don't bombard the instrument with status checks - maybe something like 200 to 1000 ms would be suitable.
If it's case 2. then you would also have two options:
configure your serial port with a read timeout long enough to cover the longest move operation, then the read operation will just block until the acknowledgement is received - again this is the quick and dirty way, or
configure a short timeout, say 1000 ms, and place the read in a While loop that repeats until the acknowledgement is received or too many timeouts have occurred. Note that a timeout is considered an error, so you will have to turn off automatic error handling for the VI and instead test the error wire out of the VISA Read, discard the timeout error and handle any other error yourself.
Just as a general tip, whenever you pass an error wire into a loop and out again, I would use a shift register. That way if one iteration generates an error, the next iteration will see that error and fail immediately, so (for example) if communication fails you don't have to wait for the read timeouts to expire multiple times before your code can exit.
You'll probably have to do some experimenting and referring to LabVIEW help to get this fully working but hopefully this is enough to get you going.
I'm trying to wrap my head around the logic behind hasSpaceAvailable on NSOutputStream.
In my app, I'm sending large amounts of data (100MB) broken up into 4080byte chunks (hard limit) over a CFSocket managed by NSInput/output streams.
When I start writing the data, about a quarter way through hasSpaceAvailable suddenly becomes NO, and so I add the data to a queue. However, if I ignore that and try to write the data anyways, the write seems to work as the return value of write:maxLength: matches the maxLength parameter (4080).
What does the output stream have space for? As far as I can tell, when using UNIX/Berkley sockets there is no logic available to determine if the socket can be written to, you just write and determine if all of the data was written.
The documentation for the property states:
A boolean value that indicates whether the receiver can be written to. (read-only)
YES if the receiver can be written to or if a write must be attempted in order to determine if space is available, NO otherwise.
In my example where I'm seeing a NO, what factor is causing this result when I can still write to that socket.
I think the hasSpaceAvailable property just returns YES if the stream has sent a "space available" stream event since the last time you called the write method. You shouldn't poll that property, and it arguably shouldn't even exist. Instead, you should wait for a stream event on the output stream that says that there's space available for writing instead.
When that stream event occurs, it means that the outgoing packet queue has at least one byte fewer than the maximum number of bytes that the socket is configured to allow you to queue up. In other words, a send() or write() system call on the socket is guaranteed to write at least one byte without blocking, and the socket is guaranteed to be in a nonblocking mode.
Note that after you write data, the stream will send another space available event immediately if the stream's buffer can take more data (or after it has sent some data if the buffer is full).
From user view, the property of "may not transmit all of the data" is a trouble thing. That will cause handler calls more than one time(may be).
The free function async_write ensure handler call only once, but it requires caller must call it in sequence or the data written will be interleaving. For network application usage, this is more bad than handler be called more than once.
If user want to handler called only once and data written is correct, user need to to do something.
I want to ask is: why asio not just make socket::async_write_some transmit all data?
I want to ask is: why asio not just make socket::async_write_some
transmit all data?
Opposed to async_write, socket::async_write_some is lower-level method.
The OS network stack is designed with send buffers and receive buffers. This buffers are required to be limited with some amount of memory. When you send many data over socket, receiving side can be more slow than sending and/or there can be network speed issues.
This is the reason why socket send buffers are limited and as a result system's syscalls like write or writev should be able to notify user program that system cannot accept chunk of data right now. With socket in async mode its even more critical. So, socket syscalls cannot work in async manner without signaling program to hold on.
So, the async_write_some as a mid-level wrapper to writev is required to support partial writes. In other hand async_write is composed operation and can call async_write_some many times in order to send buffers until operation is complete or possibly failed. It calls completion handler only once, not for each chunk of data passed to network stack.
If user want to handler called only once and data written is correct,
user need to to do something.
Nothing special, just to use async_write, not socket::async_write_some.
I've been trying to intercept UDP packets before they reach an applications logic. More precisely, that application is using a DirectPlay Server and there is no source.
So I found out that DirectPlay uses async IO by posting multiple WSARecvFrom, then having some workerthreads waiting with WaitForSingleObject and finally retrieving IO status with WSAGetOverlappedResult.
When WSARecvFrom returns, lpBuffers is not filled with data yet of course, because the operation is still pending and will complete later.
So my idea to get to the data was to save the lpOverlapped/lpBuffers pair in a std::map for every WSARecvFrom call and then, if an IO operation completes (in WSAGetOverlappedResult), I would get to the corresponding (now filled) lpBuffers by looking up the lpOverlapped in the map.
However, there seems to be a big problem: DirectPlay calls WSARecvFrom multiple times with the same lpOverlapped address sometimes, and even with the same lpOverlapped->hEvent or lpBuffers addresses, also for the same socket (none of these operations complete at this time, so they are all pending). I cannot understand why this happens, the doc clearly says: "If multiple I/O operations are simultaneously outstanding, each must reference a separate WSAOVERLAPPED structure."
Because of this I cannot correctly retrieve the lpBuffers, because when WSAGetOverlappedResult is called, I don't know to which WSARecvFrom the lpOverlapped corresponds because there were several WSARecvFroms called, each with the same lpOverlapped! How can this be? Does anyone know how DirectPlay handles this? Could there be another way intercepting (and eventually dropping) UDP Packets? (I don't want to use drivers)
(There is a good reason why I'm trying to do this: Someone is sending exploited UDP packets to a gameserver using DirectPlay, and it "confuses" the DirectPlay logic, basically shutting down the server. So I have to filter out specific UDP packets before they even reach DirectPlay)
Happy for any hint!
Thanks a lot!
I am trying to write an app that exchanges data with other iPhones running the app through the Game Kit framework. The iPhones discover each other and connect fine, but the problems happens when I send the data. I know the iPhones are connected properly because when I serialize an NSString and send it through the connection it comes out on the other end fine. But when I try to archive a larger object (using NSKeyedArchiver) I get the error message "AGPSessionBroadcast failed (801c0001)".
I am assuming this is because the data I am sending is too large (my files are about 500k in size, Apple seems to recommend a max of 95k). I have tried splitting up the data into several transfers, but I can never get it to unarchive properly at the other end. I'm wondering if anyone else has come up against this problem, and how you solved it.
I had the same problem w/ files around 300K. The trouble is the sender needs to know when the receiver has emptied the pipe before sending the next chunk.
I ended up with a simple state engine that ran on both sides. The sender transmits a header with how many total bytes will be sent and the packet size, then waits for acknowledgement from the other side. Once it gets the handshake it proceeds to send fixed size packets each stamped with a sequence number.
The receiver gets each one, reads it and appends it to a buffer, then writes back to the pipe that it got packet with the sequence #. Sender reads the packet #, slices out another buffer's worth, and so on and so forth. Each side keeps track of the state they're in (idle, sending header, receiving header, sending data, receiving data, error, done etc.) The two sides have to keep track of when to read/write the last fragment since it's likely to be smaller than the full buffer size.
This works fine (albeit a bit slow) and it can scale to any size. I started with 5K packet sizes but it ran pretty slow. Pushed it to 10K but it started causing problems so I backed off and held it at 8096. It works fine for both binary and text data.
Bear in mind that the GameKit isn't a general file-transfer API; it's more meant for updates of where the player is, what the current location or other objects are etc. So sending 300k for a game doesn't seem that sensible, though I can understand hijacking the API for general sharing mechanisms.
The problem is that it isn't a TCP connection; it's more a UDP (datagram) connection. In these cases, the data isn't a stream (which gets packeted by TCP) but rather a giant chunk of data. (Technically, UDP can be fragmented into multiple IP packets - but lose one of those, and the entire UDP is lost, as opposed to TCP, which will re-try).
The MTU for most wired networks is ~1.5k; for bluetooth, it's around ~0.5k. So any UDP packet that you sent (a) may get lost, (b) may be split into multiple MTU-sized IP packets, and (c) if one of those packets is lost, then you will automatically lose the entire set.
Your best strategy is to emulate TCP - it sends out packets with a sequence number. The receiving end can then request dupe transmissions of packets which went missing afterwards. If you're using the equivalent of an NSKeyedArchiver, then one suggestion is to iterate through the keys and write those out as individual keys (assuming each keyed value isn't that big on its own). You'll need to have some kind of ACK for each packet that gets sent back, and a total ACK when you're done, so the sender knows it's OK to drop the data from memory.