Ok so I have been at this bug all day, and I think I've got it narrowed down to the fundamental problem.
Background:
I am working on an app that has required me to write my own versions of NSNetService and NSNetServiceBrowser to allow for Bonjour over Bluetooth in iOS 5. It has been a great adventure, as I knew nothing of network programming before I started this project. I have learned a lot from various example projects and from the classic Unix Network Programming textbook. My implementation is based largely on Apple's DNSSDObjects sample project. I have added code to actually make the connection between devices once a service has been resolved. An NSInputStream and an NSOutputStream are attained with CFStreamCreatePairWithSocketToHost( ... ).
Problem:
I am trying to send some data over this connection. The data consists of an integer, a few NSStrings and an NSData object archived with NSKeyedArchiver. The size of the NSData is around 150kb so the size of the whole message is around 160kb. After sending the data over the connection I am getting the following exception when I try to unarchive...
Terminating app due to uncaught exception 'NSInvalidArgumentException',
reason: '*** -[NSKeyedUnarchiver initForReadingWithData:]: incomprehensible archive
After further exploration I have noticed that the received data is only about 2kb.. The message is being truncated, thus rendering the archive "incomprehensible."
Potentially relevant code:
The method that sends the data to all connected devices
- (void) sendMessageToPeers:(MyMessage *)msg
{
NSEnumerator *e = [self.peers objectEnumerator];
//MyMessage conforms to NSCoding, messageAsData getter calls encodeWithCoder:
NSData *data = msg.messageAsData;
Peer *peer;
while (peer = [e nextObject]) {
if (![peer sendData:data]) {
NSLog(#"Could not send data to peer..");
}
}
}
The method in the Peer class that actually writes data to the NSOutputStream
- (BOOL) sendData:(NSData *)data
{
if (self.outputStream.hasSpaceAvailable) {
[self.outputStream write:data.bytes maxLength:data.length];
return YES;
}
else {
NSLog(#"PEER DIDN'T HAVE SPACE!!!");
return NO;
}
}
NSStreamDelegate method for handling stream events ("receiving" the data)
The buffer size in this code is 32768 b/c that's what was in whatever example code I learned from.. Is it arbitrary? I tried changing it to 200000, thinking that the problem was just that the buffer was too small, but it didn't change anything.. I don't think I fully understand what's happening.
- (void)stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode
{
switch (eventCode) {
case NSStreamEventHasBytesAvailable: {
NSInteger bytesRead;
uint8_t buffer[32768]; // is this the issue?
// uint8_t buffer[200000]; //this didn't change anything
bytesRead = [self.inputStream read:buffer maxLength:sizeof(buffer)];
if (bytesRead == -1) ...;
else if (bytesRead == 0) ...;
else {
NSData *data = [NSData dataWithBytes:buffer length:bytesRead];
[self didReceiveData:data];
}
} break;
/*omitted code for other events*/
}
}
NSStream over a network like that will be using a TCP connection. It can vary, but the maximum packet size is often around 2k. As the message you’re sending is actually 160k, it will be split up into multiple packets.
TCP abstracts this away to just be a stream of data, so you can be sure all these packets will receive in the correct order.
However, the stream:handleEvent: delegate method is probably being called when only the first 2k of data has arrived – there’s no way for it to know that there’s more coming until it does.
Note the method read:maxLength: doesn’t gauruntee you’ll always get that max length – in this case it seems to be only giving you up to 2k.
You should count up the actual bytesReceived, and concatenate all the data together until you receive the total amount you’re waiting for.
How does the receiver know how much data it wants? – you might want to design your protocol so before sending data, you send an integer of defined size indicating the length of the coming data. Alternatively, if you’re only ever sending one message over the socket, you could simply close it when finished, and have the receiver only unarchive after the socket is closed.
You seem to be reading from self.inputStream but the stream passed into your stream:handleEvent: method is called aStream. Are they referencing the same object somehow? Otherwise I'm not sure you're reading the stream that actually has bytes available
Related
So my application works along these lines:
An iPod continuously sends NSDictionaries that contain: an image encoded in JPEG and some image properties as NSStrings.
The NSDictionary is encoded using NSPropertyListSerialization with the format BinaryFormat_v1_0 and sent in packets of 1024 bytes via NSStream to the central computer running an app on OSX.
The OSX app receives the data packets, continuously appending to a single NSMutableData object, until it sees the first packet of the next NSData object (which in binary format I've found starts as 'bplist').
The NSData is converted back to an NSDictionary to be used by the OSX app, by calling NSPropertyListSerialization.
Once the NSData was successfully converted (or not),the NSData object is set back to zero to start reading the next round of packets.
A few more notes: both the NSInputStream and NSOutput streams are running on their respective device's currentRunLoop in NSDefaultRunLoopMode.
When running this process, sometimes the conversion back to NSDictionary works fine with no errors (about 1/3 of the attempts), but the other times the conversion returns this error:
Error: Failed to convert NSData to NSDict : Error Domain=NSCocoaErrorDomain Code=3840 "Unexpected character b at line 1" UserInfo={NSDebugDescription=Unexpected character b at line 1, kCFPropertyListOldStyleParsingError=Error Domain=NSCocoaErrorDomain Code=3840 "Conversion of string failed." UserInfo={NSDebugDescription=Conversion of string failed.}}
Following are the parts of the program that parse the data from the stream:
... method to handle stream events:
-(void)stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode {
switch(eventCode) {
case NSStreamEventHasBytesAvailable: {
uint8_t buf[1024];
unsigned int len = (unsigned)[(NSInputStream *)aStream read:buf maxLength:1024];
if(len) {
[self handleEventBuffer:buf WithLength:len];
}
...
... and the method that takes care of the data:
-(void)handleEventBuffer:(uint8_t*)buf WithLength:(unsigned int)len {
...
NSString *bufStr = [NSString stringWithFormat:#"%s",(const char*)buf];
if ([bufStr containsString:#"bplist00"] && [self.cameraData length] > 0) {
// Detected new file, enter in all the old data and reset for new data
NSError *error;
NSDictionary *tempDict = [[NSDictionary alloc] init];
tempDict = [NSPropertyListSerialization propertyListWithData:self.cameraData
options:0
format:NULL
error:&error];
if (error != nil) {
// Expected good file but no good file, erase and restart
NSLog(#"Error: Failed to convert NSData to NSDict : %#", [error description]);
[self.cameraData setLength:0];
}
...
[self.cameraData setLength:0];
[self.cameraData appendBytes:buf length:len];
} else {
// Still recieving data
[self.cameraData appendBytes:buf length:len];
}
So, the question that I'm getting at is:
How can I fix my parsing method to give me reliable results that don't randomly fail to convert?
OR is there a better way than this to parse buffer streams for this purpose?
OR am I just doing something stupid or missing something obvious?
You appear to be relying on each write to the stream resulting in a matching read of the same size, do you know this is guaranteed by NSStream? If not then any read could contain parts of two (or more) of your encoded dictionaries, and you would get the parsing errors you see.
Alternative approach:
For each encoded dictionary to send:
Write end:
Send a message containing the size in bytes of the encoded dictionary that will follow.
Write the encoded dictionary in chunks, the last chunk may be short
Repeat
Read end:
Read the size message specifying its exact length in bytes.
Read the encoded dictionary in chunks, making sure you read only the number of bytes reported by (1).
Repeat.
Provided you are using a reliable communication stream this should enable you to read each encoded dictionary reliably. It avoids you trying to figure out where the boundary between each encoded dictionary is, as that information is part of your protocol.
HTH
I want to use NSURLSession to receive a xml stream from server and display each xml immediately on the screen.
Here is my delegate code:
- (void)URLSession:(NSURLSession *)session dataTask:(NSURLSessionDataTask *)dataTask didReceiveData:(NSData *)data {
// Enumerate each message-body.
[data enumerateByteRangesUsingBlock:^(const void * _Nonnull bytes, NSRange byteRange, BOOL * _Nonnull stop) {
// Convert message-body to xml string.
NSString *string = [[NSString alloc] initWithBytes:bytes
length:byteRange.length
encoding:NSUTF8StringEncoding];
dispatch_async(dispatch_get_main_queue(), ^{
// Some code to display string.
// ...
});
}];
}
This code works fine except one problem.
The problem is, sometimes when receiving stream, the didReceiveData doesn't be called immediately after didReceiveResponse, it sometimes receive more than one HTTP messages, and then call didReceiveData once to pass all messages which it just receive for me.
It can sometimes take a while for receiving multiple messages, and makes my application not able to display the xml in realtime.
Is there any configuration or property can make it call didReceiveData immediately? I read the document but find nothing useful.
Thanks a bunch.
Update:
I tried to use NSURLConnection to do the same things, it runs perfectly without this problem.
Each didReceiveData is called behind didReceiveResponse immediately.
How can I make the didReceiveData of NSURLSession work just like NSURLConnection?
IIRC, NSURLSession should send data as it receives it, but only after it receives a certain about of data, or after a period of time.
If you're trying to get individual chunks of data, you might instead consider sending them back from the server as a multipart response. Each "part" would contain one of your messages, and you would get a new didReceiveResponse: callback between each one.
With that said, I'm not sure why NSURLConnection would behave differently. They use a lot of the same code under the hood. You might try filing a bug with Apple.
I'm working on porting a VB program to the PC. It uses serial communication to interact with a physical device. I have a version up and running on the Mac using ORSSerialPort. However, once piece of the VB library that is great is the SerialPort.ReadExisting() function. This essentially reads any messages and discards them.
Has anyone built something similar on the Mac side? I've tried pulling out the ORSSerialPort into a function to directly read values (see below). However, unless I send a message I receive a null response. The readExisting function would be great for a scenario when things get a little out of alignment such as:
I send a message "Message1" to the device and nothing happens (was expecting Response1).
I send a message "Message2" to the device and receive: "Response1" instead of "Response2"
I'd like to detect this, call an equivalent to SerialPort.readExisting() since Response2 is the next to be found if I continue.
My read function:
-(NSString *) directRead
{
// Read Directly
int localPortFD = self.fileDescriptor;
struct timeval timeout;
int result=0;
fd_set localReadFDSet;
FD_ZERO(&localReadFDSet);
FD_SET(localPortFD, &localReadFDSet);
timeout.tv_sec = 0;
timeout.tv_usec = 100000; // Check to see if port closed every 100ms
result = select(localPortFD+1, &localReadFDSet, NULL, NULL, &timeout);
if (!self.isOpen) return nil; // Port closed while select call was waiting
if (result < 0)
{
NSLog(#"No Data To Read");
}
if (result == 0 || !FD_ISSET(localPortFD, &localReadFDSet)) return nil;
// Data is available
char buf[1024];
long lengthRead = read(localPortFD, buf, sizeof(buf));
if (lengthRead>0)
{
NSData *readData = [NSData dataWithBytes:buf length:lengthRead];
if (readData != nil)
return [[NSString alloc] initWithData:readData encoding:NSUTF8StringEncoding];
}
return nil;
}
You would think just doing:
NSString *result = nil;
do
{
result = [serialPort directRead];
NSLog(#"Past Message is: %#", result);
} while(result != nil);
Would flush out the messages. However, it acts as if there aren't any messages. However, if I call sendData:Message1 again I'd still see Response2 show up (in the above scenario).
Thanks for any and all help.
I'm having a little trouble following exactly what you're trying to do here, but in any case, it's something you'll have to implement at a higher level than ORSSerialPort or the POSIX serial read functions. ORSSerialPort will tell you any time bytes are available, as reported by the underlying standard POSIX file APIs it uses. So, there's nothing to "flush" from its perspective. If you get a response you were not expecting, e.g. a response to an "old" request, there's no way at all for the serial port to know that, it's up to your code to figure it out.
Something like this:
- (void)serialPort:(ORSSerialPort *)serialPort didReceiveData:(NSData *)data
{
if (![self dataIsResponseToLatestRequest:data]) return; // Try again next time around
// Process a valid (expected response)
}
This raises a couple other issues. You can't be guaranteed that data will come in "whole packets". The serial hardware, low level serial APIs, and ORSSerialPort have no way of knowing what a whole packet looks like, and therefore just deliver data a bit at a time as it arrives. See this answer for more on that. So, it's up to you to buffer incoming data and assemble it into packets.
If you need to match requests you sent with responses coming in, you have to do that yourself. (sounds like you're really already doing this).
Lastly, if you're only receiving "Response1" after sending "Message2", this leads me to think there's a problem with the device on the other side of the connection not responding to requests properly. It would probably be worth fixing that, if you can (assuming it's your hardware/software).
I'm sending a file between two devices using iOS 7's Multipeer Connectivity Framework. I'm using NSStreams to tranfer the file, as my previous attempt using MCSession's sendData:toPeers:withMode was really unreliable. Unfortunately the transfer speeds I'm getting are really slow, around 100kb/s, which won't work for the application I'm working on. Here are my Input and Output stream delegate methods, which is where the file transfer happens.
Output Stream (in the delegate for the stream)
...//previous code
case NSStreamEventHasSpaceAvailable: {
//we will only open the stream when we want to send the file.
NSLog(#"Stream has space available");
//[self updateStatus:#"Sending"];
// If we don't have any data buffered, go read the next chunk of data.
if (totalBytesWritten< outgoingDataBuffer.length) {
//more stuff to read
int towrite;
int diff = outgoingDataBuffer.length-packetSize;
if (diff <= totalBytesWritten)
{
towrite = outgoingDataBuffer.length - totalBytesWritten;
} else
towrite = packetSize;
NSRange byteRange = {totalBytesWritten, towrite};
uint8_t buffer[towrite];
[outgoingDataBuffer getBytes:buffer range:byteRange];
NSInteger bytesWritten = [outputStream write:buffer maxLength:towrite];
totalBytesWritten += bytesWritten;
NSLog(#"Written %d out of %d bytes",totalBytesWritten, outgoingDataBuffer.length);
} else {
//we've written all we can write about the topic?
NSLog(#"Written %d out of %d bytes",totalBytesWritten, outgoingDataBuffer.length);
[self endStream];
}
// If we're not out of data completely, send the next chunk.
} break;
Input Stream
- (void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)eventCode {
switch(eventCode) {
case NSStreamEventHasBytesAvailable:
{
NSLog(#"Bytes Available");
//Sent when the input stream has bytes to read, we need to read bytes or else this wont be called again
//when this happens... we want to read as many bytes as we can
uint8_t buffer[1024];
int bytesRead;
bytesRead = [inputStream read:buffer maxLength:sizeof(buffer)];
[incomingDataBuffer appendBytes:&buffer length:bytesRead];
totalBytesRead += bytesRead;
NSLog(#"Read %d bytes, total read bytes: %d",bytesRead, totalBytesRead);
}break;
case NSStreamEventEndEncountered:
{
UIImage *newImage = [[UIImage alloc]initWithData:incomingDataBuffer];
[[self.detailViewController imageView] setImage:newImage];
NSLog(#"End Encountered");
[self closeStream];
//this should get called when there aren't any more bytes being sent down the stream
}
}
}
Is there a way to speed up this file transfer through either multithreading or using a slightly modified NSStream subclass that makes use of asynchronous sockets?
There are a couple of things I have noticed that cause slow file transfers:
Having a non-airdrop compatible device in the session (http://en.wikipedia.org/wiki/AirDrop)
WiFi issues (saturated network, router problems, the channel, etc.)
I have been getting decent speeds (and by decent I mean 500K / second) between an iPhone 5S, iPad Air and iPhone Simulator on the MAC.
I had some pretty decent transfer rates going for a while and suddenly it seemed that I was receiving one packet every 15 to 30 seconds. All looked fine on the sender side, but the packets trickle into the receiver. I use sendData:toPeers:withMode in my app.
One of my devices had WiFi turned off. Turning it back on restored my performance even though neither device is connected to a wifi network.
So I'm only ever doing peer-to-peer ad-hoc connections with MPC, but still, with no WiFi hub involved, it appears that iOS is making use of the WiFi signal to carry my data.) I did see in a presentation from Apple a while back that MPC over straight BlueTooth is a dog and I can tell you for a fact that it is.
MPC work on WiFi or bluetooth so it depends on what you are using. If you are using wifi to send files than you will get maximum speed as per the network strength.
I have two functions in my software that cause important latency problems. The software is written in Objective-C. I receive serial data from an usb device and my goal is to encapsulate them and then to send them to another object which will process the data.
This section of the program causes large cpu and latency issues and I don't know how to solve this at all. The device only sends data when its state changes, thus when lots of changes occur well everything becomes laggy..
- (void)getSerialData {
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
[self getSerialDataLoop];
});
}
- (void)getSerialDataLoop {
readThreadRunning = YES;
char byte_buffer[2]; // buffer for holding incoming data
int numBytes=0; // number of bytes read during read
NSString *text;
// this will loop untilthe serial port closes
while(TRUE) {
// read() blocks until some data is available or the port is closed
numBytes = (int)read(serialFileDescriptor, byte_buffer, 1); // read up to the size of the buffer
if(numBytes>0) {
///text = [NSString stringWithCString:byte_buffer encoding:NSSymbolStringEncoding];
if(![text isEqualToString:#""]){
text = [NSString stringWithUTF8String:byte_buffer];
[self performSelectorOnMainThread:#selector(processNSStringData:) withObject:text waitUntilDone:YES];
}
} else {
break; // Stop the thread if there is an error
}
}
// make sure the serial port is closed
if (serialFileDescriptor != -1) {
close(serialFileDescriptor);
serialFileDescriptor = -1;
}
// mark that the thread has quit
readThreadRunning = FALSE;
}
Do you have any ideas or pointers?
You've basically reinvented NSStream here. I would first recommend that you investigate this already available solution that ties into the run loop.
You also could easily be overwhelming yourself with calls to getSerialData. Nothing in your system prevents multiple calls to this routine, and if you make multiple calls, you'll get dueling concurrent operations. Using NSStream would address that. In any case, though, you shouldn't keep creating new read blocks if one is already running.
You're also reading one byte at time and processing it. This is likely your biggest impact. Calling back to the main thread for every byte is likely quite expensive. If nothing else you're creating a new NSString object for every byte.
Note that your code is very dangerous and could crash. You never initialize byte_buffer, and you only read one byte into it. When you call stringWithUTF8String:, you're assuming that the second byte is \0, which depends on the current state of the stack.