iOS writing to server using CFWriteStrem not writing? - objective-c

I'm basically sending a post to my server. The string I'm writing is #"POST /api?f=### HTTP/1.1\r\nHost: http://server.loc.a.tion\r\nContent-Type: application/x-www-form-urlencoded\r\nContent-Length: ##\r\n\r\ndata=somedataandstuff". After this the server is supposed to send me back a message. (not that I've got that far yet.)
I'm following the basic polling code as lined out by apple, but I've made some changes.
Note the following is after I've opened the streams), data_to_send.buffer is a (char *), and data_to_send.buffer_size is 1500.
//Preform Write operations
bool done = false;
int bufLen = data_to_send.buffer_size;
const uint8_t * rawstring =(const uint8_t *)data_to_send.buffer;
CFIndex bytesWritten;
while (!done) {
CFStreamStatus stat = CFWriteStreamGetStatus(writeStream); //for me while debugging
if (CFWriteStreamCanAcceptBytes(writeStream)) {
//ADDED:
if(bufLen > 0) bytesWritten = CFWriteStreamWrite(writeStream, rawstring, (CFIndex)bufLen);
//ADDED: to check if it ever writes
if( CFWriteStreamGetStatus(writeStream) == kCFStreamStatusWriting){
NSLog(#"lalallaa");
}
if (bytesWritten < 0) {
//error stuff
} else if (bytesWritten == 0) {
if (CFWriteStreamGetStatus(writeStream) == kCFStreamStatusAtEnd) {
done = TRUE;
rawstring = nil;
}
} else if (bytesWritten != bufLen) {
// Determine how much has been written and adjust the buffer
bufLen = bufLen - bytesWritten;
memmove(rawstring, rawstring + bytesWritten, bufLen);
//error stuff
} else { //ADDED
bufLen = 0;
bytesWritten = 0;
//loop and wait until finished
}
}
}
CFWriteStreamClose(writeStream);
CFRelease(writeStream);
writeStream = NULL;
I modified the code because CFWriteStreamWrite returns the full 1500 (this is a flag in my mind: if bufLen is 1500 and it writes it all shouldn't it return 0?), which wasn't accounted for in apples code (ie: bytesWritten == bufLen)(another flag as I assume apple wouldn't miss anything unless this was the intended functionality). In turn the bufLen or the buffer (rawstring) don't get modified and we just repeatedly return 1500, and commence a never ending cycle. So to account for this I added the else which sets bufLen to 0 and bytes written to 0 so that the code will loop to check when it reaches KCFStreamStatusAtEnd. Now when I step through this I noticed that once it does CFWriteStreamWrite the status is till 2 (CFStreamStatusOpen) and hence another never ending cycle. The check I added to see if it ever writes never gets hit so as far as I can tell it doesn't.
But, now if we check the apache access log on the server we have:
... "-" 408 -
where a working call (made from working java code) would look like
... "POST /api?f=blahblah" 200 ##
so I feel like something is at least trying to be written. Although a 408 status is a Request times out. This could be because the stream sits there while I debug with the stream open? Or because when I debug and stop I never close the stream?
Any ideas or suggestions would be greatly appreciated. (I'm new to iOS/objective-c)

If anyone is interested in this I basically gave up and went with a NSMutableURLrequest.
NSString *urlString = #"http://url.ca/api?f=##";
NSMutableURLrequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:url]];
[request setHTTPMethod:#"POST"];
[request setValue:#"##" for HTTPHeaderField#"Contect-Length"];
[request setValue:#"application/x-www-form-urlencoded" for HTTPHeaderField#"Content-Type"];
[request setHTTPBody:#"data=lalalalala" dataUsingEncoding:NSASCIIStringEncoding]];
NSURLResponse *response;
NSError *err;
NSData *responseData = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&err];
//error check & handle data
Keep in mind if you want to do something similar that this is synchronous so it will delay your UI (so run in it's own thread or do asynchronously). This basically does the same thing that I was trying to code.

Related

JSON data has "bad" characters that causes NSJSONSerialization to die

I am using the ATV version of TVH Client - if you haven't looked at this it's worth looking at TVH to glimpse madness in the face. It has a JSON API that sends back data, including the electronic program guide. Sometimes the channels put accented characters in their data. Here is an example, this is the result from Postman, note the ? char in the description:
{
"eventId": 14277,
"episodeId": 14278,
"channelName": "49.3 CometTV",
"channelUuid": "02fe96403d58d53d71fde60649bf2b9a",
"channelNumber": "49.3",
"start": 1480266000,
"stop": 1480273200,
"title": "The Brain That Wouldn't Die",
"description": "Dr. Bill Cortner and his fianc�e, Jan Compton , are driving to his lab when they get into a horrible car accident. Compton is decapitated. But Cortner is not fazed by this seemingly insurmountable hurdle. His expertise is in transplants, and he is excited to perform the first head transplant. Keeping Compton's head alive in his lab, Cortner plans the groundbreaking yet unorthodox surgery. First, however, he needs a body."
},
If this data is fed into NSJSONSerialization, it returns an error. So to avoid this, the data is first fed into this function:
+ (NSDictionary*)convertFromJsonToObjectFixUtf8:(NSData*)responseData error:(__autoreleasing NSError**)error {
NSMutableData *FileData = [NSMutableData dataWithLength:[responseData length]];
for (int i = 0; i < [responseData length]; ++i) {
char *a = &((char*)[responseData bytes])[i];
if ( (int)*a >0 && (int)*a < 0x20 ) {
((char*)[FileData mutableBytes])[i] = 0x20;
} else {
((char*)[FileData mutableBytes])[i] = ((char*)[responseData bytes])[i];
}
}
NSDictionary* json = [NSJSONSerialization JSONObjectWithData:FileData //1
options:kNilOptions
error:error];
if( *error ) {
NSLog(#"[JSON Error (2nd)] output - %#", [[NSString alloc] initWithData:responseData encoding:NSUTF8StringEncoding]);
NSDictionary *userInfo = #{ NSLocalizedDescriptionKey:[NSString stringWithFormat:NSLocalizedString(#"Tvheadend returned malformed JSON - check your Tvheadend's Character Set for each mux and choose the correct one!", nil)] };
*error = [[NSError alloc] initWithDomain:#"Not ready" code:NSURLErrorBadServerResponse userInfo:userInfo];
return nil;
}
return json;
}
This cleans up the case when there is a control character in the data, but not an accent like the case above. When I feed in that data I get the "Tvheadend returned malformed JSON" error.
One problem is that the user can change the character set among a limited number of selections, and the server does not tell the client what it is. So one channel might use UTF8 and another ISO-8891-1, and there is no way to know which to use on the client side.
So: can anyone offer a suggestion on how to process this data so we feed clean strings into NSJSONSerialization?
I still do not know the root cause of the problem I am seeing - the server is sending not only high-bit characters like the ones I noted above, but I also found that it contained control characters too! Looking over other threads it appears I am not the only one seeing this problem, so hopefully others will find this useful...
The basic trick is to convert the original data from the server to a string using UTF8. If there are any of these "bad" chars in it, the conversion will fail. So you check if the resulting string is empty, and try another charset. Eventually you'll get data back. Now you take that string and strip out any control chars. Now you take that result, which is now UTF8 "clean", and convert it back to UTF8 NSData. That will pass through the JSON conversion without error. Phew!
Here is the solution I finally used:
// ... the original data from the URL is in responseData
NSString *str = [[NSString alloc] initWithData:responseData encoding:NSUTF8StringEncoding];
if ( str == nil ) {
str = [[NSString alloc] initWithData:responseData encoding:NSISOLatin1StringEncoding];
}
if ( str == nil ) {
str = [[NSString alloc] initWithData:responseData encoding:NSASCIIStringEncoding];
}
NSCharacterSet *controls = [NSCharacterSet controlCharacterSet];
NSString *stripped = [[str componentsSeparatedByCharactersInSet:controls] componentsJoinedByString:#""];
NSData *data = [stripped dataUsingEncoding:NSUTF8StringEncoding];
NSDictionary* json = [NSJSONSerialization JSONObjectWithData:data options:kNilOptions error:&error];
I hope someone finds this useful!

Persisting bookmark in core-data

I have an OSX application that is supposed to have a list of files from anywhere in the user's disk.
The first version of the app saves the path to these files in a core-data model.
However, if the file is moved or renamed, the tool loses its purpose and the app can crash.
So I decided to use bookmarks. It seems to be working, but every time I try to recover the data, I get the old path of the files. Why is that? What am I missing?
My core-data entity uses a binary data field to persist the bookmark.
The bookmark itself is done like this:
NSData * bookmark = [filePath bookmarkDataWithOptions:NSURLBookmarkCreationMinimalBookmark
includingResourceValuesForKeys:NULL
relativeToURL:NULL
error:NULL];
And on loading the application, I have a loop to iterate all the table and recover the bookmark like this:
while (object = [rowEnumerator nextObject]) {
NSError * error = noErr;
NSURL * bookmark = [NSURL URLByResolvingBookmarkData:[object fileBookmark]
options:NSURLBookmarkResolutionWithoutUI
relativeToURL:NULL
bookmarkDataIsStale:NO
error:&error];
if (error != noErr)
DDLogCError(#"%#", [error description]);
DDLogCInfo(#"File Path: %#", [bookmark fileReferenceURL]);
}
If I rename the file, the path is null. I see no difference between storing this NSData object and a string with the path. So I am obviously missing something.
Edit:
I also often get an error like this: CFURLSetTemporaryResourcePropertyForKey failed because it was passed this URL which has no scheme.
I appreciate any help, thanks!
I can't find any issues in my code, so I changed it.
After looking for the reason of the "no scheme" message, I came to the conclusion some third-party application is required for this code to work, and that's undesirable.
I am now using aliases. This is how I create them:
FSRef fsFile, fsOriginal;
AliasHandle aliasHandle;
NSString * fileOriginalPath = [[filePath absoluteString] stringByReplacingPercentEscapesUsingEncoding:NSUTF8StringEncoding];
OSStatus status = FSPathMakeRef((unsigned char*)[fileOriginalPath cStringUsingEncoding: NSUTF8StringEncoding], &fsOriginal, NULL);
status = FSPathMakeRef((unsigned char*)[fileOriginalPath cStringUsingEncoding: NSUTF8StringEncoding], &fsFile, NULL);
OSErr err = FSNewAlias(&fsOriginal, &fsFile, &aliasHandle);
NSData * aliasData = [NSData dataWithBytes: *aliasHandle length: GetAliasSize(aliasHandle)];
And now I recover the path like this:
while (object = [rowEnumerator nextObject]) {
NSData * aliasData = [object fileBookmark];
NSUInteger aliasLen = [aliasData length];
if (aliasLen > 0) {
FSRef fsFile, fsOriginal;
AliasHandle aliasHandle;
OSErr err = PtrToHand([aliasData bytes], (Handle*)&aliasHandle, aliasLen);
Boolean changed;
err = FSResolveAlias(&fsOriginal, aliasHandle, &fsFile, &changed);
if (err == noErr) {
char pathC[2*1024];
OSStatus status = FSRefMakePath(&fsFile, (UInt8*) &pathC, sizeof(pathC));
NSAssert(status == 0, #"FSRefMakePath failed");
NSLog(#"%#", [NSString stringWithCString: pathC encoding: NSUTF8StringEncoding]);
} else {
NSLog(#"The file disappeared!");
}
} else {
NSLog(#"CardCollectionUserDefault was zero length");
}
}
However, I am still curious on why my previous code failed. I appreciate any thoughts on that. Thanks!

Data corruption when reading realtime H.264 output from AVAssetWriter

I'm using some tricks to try to read the raw output of an AVAssetWriter while it is being written to disk. When I reassemble the individual files by concatenating them, the resulting file is the same exact number of bytes as the AVAssetWriter's output file. However, the reassembled file will not play in QuickTime or be parsed by FFmpeg because there is data corruption. A few bytes here and there have been changed, rendering the resulting file unusable. I assume this is occurring on the EOF boundary of each read, but it isn't consistent corruption.
I plan to eventually use code similar to this to parse out individual H.264 NAL units from the encoder to packetize them and send them over RTP, however if I can't trust the data being read from disk I might have to use another solution.
Is there an explanation/fix for this data corruption? And are there any other resources/links you have found on how to parse the NAL units to packetize over RTP?
Full code here: AVAppleEncoder.m
// Modified from
// http://www.davidhamrick.com/2011/10/13/Monitoring-Files-With-GCD-Being-Edited-With-A-Text-Editor.html
- (void)watchOutputFileHandle
{
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
int fildes = open([[movieURL path] UTF8String], O_EVTONLY);
source = dispatch_source_create(DISPATCH_SOURCE_TYPE_VNODE,fildes,
DISPATCH_VNODE_DELETE | DISPATCH_VNODE_WRITE | DISPATCH_VNODE_EXTEND | DISPATCH_VNODE_ATTRIB | DISPATCH_VNODE_LINK | DISPATCH_VNODE_RENAME | DISPATCH_VNODE_REVOKE,
queue);
dispatch_source_set_event_handler(source, ^
{
unsigned long flags = dispatch_source_get_data(source);
if(flags & DISPATCH_VNODE_DELETE)
{
dispatch_source_cancel(source);
//[blockSelf watchStyleSheet:path];
}
if(flags & DISPATCH_VNODE_EXTEND)
{
//NSLog(#"File size changed");
NSError *error = nil;
NSFileHandle *fileHandle = [NSFileHandle fileHandleForReadingFromURL:movieURL error:&error];
if (error) {
[self showError:error];
}
[fileHandle seekToFileOffset:fileOffset];
NSData *newData = [fileHandle readDataToEndOfFile];
if ([newData length] > 0) {
NSLog(#"newData (%lld): %d bytes", fileOffset, [newData length]);
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *basePath = ([paths count] > 0) ? [paths objectAtIndex:0] : nil;
NSString *movieName = [NSString stringWithFormat:#"%d.%lld.%d.mp4", fileNumber, fileOffset, [newData length]];
NSString *path = [NSString stringWithFormat:#"%#/%#", basePath, movieName];
[newData writeToFile:path atomically:NO];
fileNumber++;
fileOffset = [fileHandle offsetInFile];
}
}
});
dispatch_source_set_cancel_handler(source, ^(void)
{
close(fildes);
});
dispatch_resume(source);
}
Here are some similar questions I have found, but don't exactly answer my question:
Get PTS from raw H264 mdat generated by iOS AVAssetWriter
streaming video FROM an iPhone
Parsing h.264 NAL units from a quicktime MOV file
Realtime Audio/Video Streaming FROM iPhone to another device (Browser, or iPhone)
When I eventually figure this out, I will release an open source library to assist people who try to do this in the future.
Thank you!
Update: The corruption doesn't happen at the EOF boundary. It seems like parts of the file are re-written after finishWriting is called. This first file was chunked at 4KB, so the area changed isn't anywhere near an EOF boundary. It seems to be corrupted near new "moov" elements as well when movieFragmentInterval is enabled.
Correct file on the left, broken file on the right.
I ended up abandoning the "read while it's written" approach in favor of a manual chunking approach where I call finishWriting every 5 seconds on a background thread. I was able to drop a negligible number of frames using a method originally described here:
- (void) segmentRecording:(NSTimer*)timer {
AVAssetWriter *tempAssetWriter = self.assetWriter;
AVAssetWriterInput *tempAudioEncoder = self.audioEncoder;
AVAssetWriterInput *tempVideoEncoder = self.videoEncoder;
self.assetWriter = queuedAssetWriter;
self.audioEncoder = queuedAudioEncoder;
self.videoEncoder = queuedVideoEncoder;
//NSLog(#"Switching encoders");
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
[tempAudioEncoder markAsFinished];
[tempVideoEncoder markAsFinished];
if (tempAssetWriter.status == AVAssetWriterStatusWriting) {
if(![tempAssetWriter finishWriting]) {
[self showError:[tempAssetWriter error]];
}
}
if (self.readyToRecordAudio && self.readyToRecordVideo) {
NSError *error = nil;
self.queuedAssetWriter = [[AVAssetWriter alloc] initWithURL:[self newMovieURL] fileType:(NSString *)kUTTypeMPEG4 error:&error];
if (error) {
[self showError:error];
}
self.queuedVideoEncoder = [self setupVideoEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:videoFormatDescription bitsPerSecond:videoBPS];
self.queuedAudioEncoder = [self setupAudioEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:audioFormatDescription bitsPerSecond:audioBPS];
//NSLog(#"Encoder switch finished");
}
});
}
Full source code: https://github.com/chrisballinger/FFmpeg-iOS-Encoder/blob/master/AVSegmentingAppleEncoder.m
When reading a MOV file that is ACTIVELY recording on iOS, you MUST check the 4 bytes mentioned for changes, and re-write this four bytes, then check for additional data in file, and send additional data. Then when done, truncate the file to the file size written.
Obviously this depends on where you are sending the file. I use a send (offset,number of bytes) to receiver. So I send "additional data", "more additional data", ... , new data at (24,4), "more additional data".
Typically iOS only writes the 4 byte (size of data section) record when file is about to be closed (aka after last media write). (see info on "Quicktime atoms"). Unfortunately, this also means the MOV file is not PLAYABLE until recording is completed (and movie descriptors written at END of file).

dataWithContentsOfURL - What is expected from the server?

I am trying to create NSData with the contents of an URL:
NSString *theUrl = [NSString stringWithString:#"http://127.0.0.1:8090"];
NSError *connectionError = nil;
NSData *inData = [NSData dataWithContentsOfURL:[NSURL URLWithString:theUrl] options:NSDataReadingUncached error:&connectionError];
NSInteger code = [connectionError code];
if (code != 0)
{
NSString *locDesc = [NSString stringWithString:[connectionError localizedDescription]];
NSString *locFail = [NSString stringWithString:[connectionError localizedFailureReason]];
NSLog(#"Error: %d %# %#", code, locDesc, locFail);
}
else if ([inData length] == 0)
{
NSLog(#"No data");
}
I have a super simple Java http server running on the local host that returns Hello World to a client:
DataOutputStream os = new DataOutputStream(s.getOutputStream()); // s is the socket
os.writeBytes(new String("Hello World\0"));
os.flush();
os.close();
s.close();
When pointing Google Chrome to http://127.0.0.1:8090 it displays Hello World as expected so data is sent back. When I run the objective-c code the inData is empty (0x0, data length is 0), and the error code is 0 so I don't have an error to inspect. If I change theUrl to "http://www.google.com" it seems works fine as the data length becomes > 0.
So my question is why inData is empty when I go the to local http-server. Does the stream have to be terminated with a specific data sequence?
Is the server outputting an HTTP status code like it's supposed to? If the response doesn't contain a 200 status indicating that the request was completed successfully, that might be causing dataWithContentsOfURL:options:error: to fail.
A little more context would be useful, but a wild guess is that your "super simple" HTTP server does not send any headers or not the ones expected by NSURL.
Have you tried curl -i http://127.0.0.1:8090 to see what the output actually looks like?

Cocoa Touch Bonjour how to deal with NSNetService addresses and uint8_t

I'm attempting to make an iOS app communicate with a server that uses Bonjour and uses HTTP commands. So far I have been able to find the local domain and locate the particular service I'm looking for. I am able to resolve the address of the service, but I don't know how to get something useful out of the address. The address from the NSNetService is a NSData object and I have no idea what to do with it. I need to send commands like GET and PUT. What cocoa classes handle things like this?
I also tried getting input and output streams from the Service, but they seem to be extremely low level streams and I don't know how to properly deal with buffers and all that.
[service getInputStream:&inputStream outputStream:&outputStream]
the NSOutputStream write method takes in a uint8_t buffer which I have no idea how to create.
the NSInputStream read method returns a uint8_t buffer and I don't know how to interpret it.
I am able to communicate with this server using terminal commands. For instance, sending it the command LIST causes it to print out the list of files I am looking for. How do I send and get information like this in Cocoa?
To write data to the output stream, therefore sending it to the server:
NSString * stringToSend = #"Hello World!\n"; //The "\n" lets the receiving method described below function correctly. I don't know if you need it or not.
NSData * dataToSend = [stringToSend dataUsingEncoding:NSUTF8StringEncoding];
if (outputStream) {
int remainingToWrite = [dataToSend length];
void * marker = (void *)[dataToSend bytes];
while (0 < remainingToWrite) {
int actuallyWritten = 0;
actuallyWritten = [outputStream write:marker maxLength:remainingToWrite];
remainingToWrite -= actuallyWritten;
marker += actuallyWritten;
}
}
You can send any data like this, just put it in a NSData object.
To receive data from the server use this code in the input stream's NSStreamDelegate:
- (void)stream:(NSStream *)aStream handleEvent:(NSStreamEvent)streamEvent {
NSInputStream * istream;
NSOutputStream * ostream;
switch(streamEvent) {
case NSStreamEventHasBytesAvailable:;
istream = (NSInputStream *)aStream;
ostream = (NSOutputStream *)CFDictionaryGetValue(connections, istream);
uint8_t buffer[2048];
int actuallyRead = [istream read:(uint8_t *)buffer maxLength:2048];
if (actuallyRead > 0) {
NSData *data;
data = [NSData dataWithBytes:buffer length:actuallyRead];
NSString *string = [[[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding]autorelease];
string = [string stringByReplacingOccurrencesOfString:#"\n" withString:#""];
//Do something with the string...
}
break;
case NSStreamEventEndEncountered:;
istream = (NSInputStream *)aStream;
ostream = nil;
if (CFDictionaryGetValueIfPresent(connections, istream, (const void **)&ostream)) {
[self shutdownInputStream:istream outputStream:ostream];
}
break;
case NSStreamEventHasSpaceAvailable:
case NSStreamEventErrorOccurred:
case NSStreamEventOpenCompleted:
case NSStreamEventNone:
default:
break;
}
}
Take a look at Apple's CocoaEcho Sample Code. It should help you.