I'm currently working on an application for uploading large video files from the iPhone to a webservice through simple http post. As of right now, I build an NSURLRequest and preload all of the video file data before loading the request. This naturally eats a ton of ram if the file is considerably big, in some cases it's not even possible.
So basically my question is: Is there a correct way of streaming the data or loading it in chunks without applying any modifications to the webserver?
Thanks.
EDIT for clarification: I am searching for a way to stream large multipart/form data FROM the iPhone TO a webserver. Not the other way arround.
EDIT after accepting answer: I just found out that apple has some nifty source code written for this exact purpose and it shows appending additional data to the post not just the big file itself. Incase anyone ever needs it: SimpleURLConnections - PostController.m
Yet another EDIT: While using that piece of source code from apple I encountered a very stupid and ugly problem that even wireshark couldn't help me debug. Some webservers don't understand the boundary string when it's declared in between quotes (like in apples example). I had problems with it on Apache Tomcat and removing the quotes worked just wonderful.
You can use NSInputStream on NSMutableURLRequest. For example:
NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:uploadURL];
NSInputStream *stream = [[NSInputStream alloc] initWithFileAtPath:filePath];
[request setHTTPBodyStream:stream];
[request setHTTPMethod:#"POST"];
[NSURLConnection sendAsynchronousRequest:request queue:[NSOperationQueue mainQueue]
completionHandler:^(NSURLResponse *response, NSData *data, NSError *error) {
NSLog(#"Finished with status code: %i", [(NSHTTPURLResponse *)response statusCode]);
}];
You can use a NSInputStream to provide the data to post via -[NSMutableURLRequest setHTTPBodyStream:]. This could be an input stream that reads from a file. You might need to implement the connection:needNewBodyStream: method in your URL connection delegate to provide a new, unopened stream in case the system needs to retransmit the data.
One way to do this is to use an asynchronous NSInputStream in concert with a file. When the asynchronous connection asks you to provide more data, you read in the data from a file. You have a few ways to do this:
UNIX/BSD interface. use open (or fopen), malloc, read, and create a NSData object from the malloced data
use the above with mmap() if you know it
use the Foundation class NSFileHandle APIs to do more or less the same using ObjectiveC
You can read up on streams in the 'Stream Programming Guide'. If this doesn't work for you there are lots of open source projects that can upload files, for instance MKNetworkKit
Related
I am working on a very simple application to discover a device using SSDP and I am trying to find the easiest way to parse the response from this command. I am trying to avoid having to do a bunch of NSString or regular expressions manipulations.
I have tried the following two approaches:
Approach 1:
Using GCDAsyncUdpSocket, I am able to successfully send the discovery command and get the following response:
HTTP/1.1 200 OK
Cache-Control: max-age=300
ST: roku:ecp
USN: uuid:roku:ecp:1234567890
Ext:
Server: Roku UPnP/1.0 MiniUPnPd/1.4
Location: http://192.168.XX.XX:8060/
This looks like a regular HTTP response, but using GCDAsyncUdpSocket, I am getting the response as an NSData object, which I can easily convert to an NSString. However, what would be ideal is to somehow cast this to an NSHTTPURLResponse and then use its methods to get the field values. Any idea if this can be done?
Approach 2:
I have tried using a regular NSURLRequest to try to send this command and then I would be able to get an NSHTTPURLResponse back. However, I keep on getting an error because the SSDP discovery command requires me to send this request to port 1900.
I use the following code to send the "HTTP" request, I know it is not strictly HTTP, but I thought it may be an easier way to send this UDP command as the requirements look very similar to HTTP.
NSURL *url = [NSURL URLWithString:#"http://239.255.255.250:1900"];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url
cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:60.0];
[request setHTTPMethod:#"M-SEARCH *"];
[request setValue:#"239.255.255.250:1900" forHTTPHeaderField:#"Host"];
[request setValue:#"\"ssdp:discover\"" forHTTPHeaderField:#"Man"];
[request setValue:#"roku:ecp" forHTTPHeaderField:#"ST"];
NSURLConnection *connection = [[NSURLConnection alloc]initWithRequest:request delegate:self startImmediately:YES];
if (connection)
{
NSLog(#"Connection success!");
}
else
{
NSLog(#"Connection failed!");
}
When I do this, the connection is successful, but I get the following error in the didFailWithError delegate for NSURLConnection:
failed Error Domain=NSPOSIXErrorDomain Code=47 "The operation couldn’t be completed. Address family not supported by protocol family" UserInfo=0x8875940 {NSErrorFailingURLKey=http://239.255.255.250:1900/, NSErrorFailingURLStringKey=http://239.255.255.250:1900/}
This error only happens if I use port 1900, if I leave this out or use another more HTTP friendly port such as 8080, then this works, but obviously the device I am trying to discover does not respond correctly unless it gets the request in port 1900.
Thanks for any help you can provide.
Port 1900 is fixed for SSDP. All UPnP devices are fixed to listen on this port and the connecting nodes expect that. You can't change it. It's a part of the UPnP design goal to "work out of the box". Furthermore, my Cocoa expertise is very limited, but i think that NSHTTPURLResponse won't make things simpler for you. [NSHTTPURLResponse allHeaderFields] returns NSDictionary, which means that you don't know anymore what was the original order of the header fields. And you need to know what was coming after Ext: which is a meta-header.
I suggest either parsing the response yourself, which shouldn't be much more complicated than one cycle, separating the response by lines and then splitting by :. Or instead of trying to roll your own SSDP handshake, use ready made Cocoish library like Upnpx, Cyberlink or Platinum. It might feel like inappropriately heavy artillery just for the discovery phase, but i wonder what else you would do with the device after that, other than actually trying to invoke some actions on the device.
I know that conventionally for an app to interact with the internet, it must use a web service to exchange information. However, how would one upload data(photos, text, audio recordings etc.etc.) from app to server(which holds data for all user accounts)? I know some people use an email-to-server tactic from research but even then it sounds ineffective and slow. How do apps such as Instagram upload so fast? I am trying to replicate that sort of uploading. Please guide me in the right direction.
Thanks for the help!
You should definitely look into AFNetworking. Here is an example of my uploading an image to a php web service:
NSData *imageData = UIImagePNGRepresentation(pageImage);
AFHTTPClient *client= [AFHTTPClient clientWithBaseURL:[NSURL URLWithString:#"http://www.SERVER.com"]];
//You can add POST parameteres here
NSDictionary *params = [NSDictionary dictionaryWithObjectsAndKeys:
author, #"author",
title, #"title",
nil];
NSMutableURLRequest *request = [client multipartFormRequestWithMethod:#"POST" path:#"/PATH/TO/WEBSERVICE.php" parameters:params constructingBodyWithBlock: ^(id <AFMultipartFormData>formData) {
//This is the image
[formData appendPartWithFileData: imageData name:#"cover_image" fileName:#"temp.png" mimeType:#"image/png"];
}];
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
//Setup Upload block to return progress of file upload
[operation setUploadProgressBlock:^(NSInteger bytesWritten, long long totalBytesWritten, long long totalBytesExpectedToWrite) {
float progress = totalBytesWritten / (float)totalBytesExpectedToWrite;
NSLog(#"Upload Percentage: %f %%", progress*100);
}];
//Setup Completeion block to return successful or failure
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
NSString *response = [operation responseString];
NSLog(#"response: [%#]",response);
//Code to run after webservice returns success response code
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"error: %#", [operation error]);
//Code to Run if Failed
}];
[operation start];
Edit- Also I use MBProgressHUD to display to the user the uploading on longer uploads.
As you might know, upload speed is always bound to the speed of the connection type you're using. Even the best upload technique will be slow when the connection is slow (GPRS for example, or EDGE, even 3G can be slow if network coverage is not good).
To upload large sets of data faster/better one thing you could do is compressing the data you're sending using ZIP or any other file compression format you wish or even develop you own compression algorithm (you might not want to do that ;-)).
If you want to reduce the overhead of HTTP/HTTPS connections for example, you can write your very own protocol for data exchange, implement it on the client/server side and upload faster. This will be a lot of work as you have to do all the implementation work not only for the protocol itself as you need to add security etc. But even if you choose to create a protocol, as said in the beginning, it will be slow if the connection is slow.
Update: A presenatation by Mike Krieger (Co-Founder of Instagram) where he covers your question just crossed my way https://speakerdeck.com/u/mikeyk/p/secrets-to-lightning-fast-mobile-design?slide=1.
The reason why you think it's so fast is, that they're updating the UI before the request (the Upload in this case) even completes. This is what Mike describes as "being optimistic". If the request fails you can still notify the user, but in the meantime make him feel productive and act like the request completed successfully.
This is a pretty open ended question but here are a few things to look at:
"Uploading fast" depends on the user's connection and server bandwidth so I won't get into that.
You can upload photos (and other files) by creating NSData objects and attaching them to a POST request. There is already a ton of sample code for uploading NSData but to convert a UIImage you will do the following:
NSData *imageData = UIImagePNGRepresentation(image);
You can do this using the built in Cocoa classes (NSMutableURLRequest) and with 3rd party networking classes (such as AFNetworking - just scroll down to file uploads).
When I send simple data to my webserver, I use the following approach: Use the ASIHttpRequest framework for connecting to your sever. Send the data in HTTP Post body, which is easy to do in the ASIHttpRequest framework. You will want to convert your data to either XML or JSON(use the SBJson framework for this) before sending it. I then write php scripts that parse the json or xml and then input this data into my database with custom SQL scripts. I can give you code snippets if you need them for any of these procedures...
It seems to me that, with your first sentence, you've basically answered your own question.
You need something on your server to receive the files and then you write client code to match. It could be as simple as ftp or as complex as a custom protocol depending on the security and control that you need.
Ok, I'm trying to get a file from my webserver. But I'm getting kinda confused about some stuff. When I use NSURL I can get my xml-file with an url like this: "localhost...../bla.xml". But I'm also trying to test some things... Like... What will happen to my app if I have an open connection to the webserver, and I lose connection to internet? The above method with the NSURL, I haven't really established any connection where it always is connected right? or should I use be using NSURLConnection?
Maybe it's a little confusing, because I'm confused. I hope someone can give me some info I can research about.
Thanks.
Take a look at NSURLConnection Class.
http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/Foundation/Classes/NSURLConnection_Class/Reference/Reference.html
Create connection object and set a timeout value, if you lose the connection or the connection times out NSURLConnection delegate method: - (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error gets called and you would be notified of that event.
You might also use NSURLConnection sendSynchronousRequest method, but its strongly discouraged to use that method as it would block the thread its running.
Are you trying to access the content of a file? If so, you would use the following.
NSError *error;
NSURL *url = [NSURL URLwithString:#"localhost/file.html"];
NSString *filecontents = [NSString stringWithContentsOfURL:url encoding:NSUTF8StringEncoding error:&error];
The NSString object filecontents would contain your string.
You wouldn't be able to loose connection in such a short time. If there is no connection, an error would be applied to error.
EDIT:
If you wanted to constantly stay connected to a server, that is a different story.
You have have to use C's send() and recv() functions, which you can read about here.
I don't know much about it, and I'm learning it myself, so you should ask someone else on how to set up a server. But you will need to have another program running simultaniously.
I have a url that is a link to an audio file and will be played using AVFoundation.framework. But for some reason, when the app reaches setting the NSData it crashes. Please help.
NSURL *url = [NSURL URLWithString:songPathForm];
NSData *soundData = [NSData dataWithContentsOfURL:url];
EDIT::::
This is what I did to make it stop crashing, but the data contains nothing
NSString *url = [songPathForm stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding];
//NSURL *url = [NSURL URLWithString:songPathForm];
NSURLRequest *songRequest = [NSURLRequest requestWithURL:[NSURL URLWithString:url]
cachePolicy:NSURLRequestReloadIgnoringLocalCacheData timeoutInterval:60.0];
NSURLConnection *songConnection = [[NSURLConnection alloc] initWithRequest:songRequest delegate:self];
if(songConnection)
{
songData = [[NSMutableData data] retain];
}
It comes out as <>
There is absolutely no way to answer this question as it is worded, but here are some clues:
if there is a crash, there is a backtrace. Post it.
if there is a crash, there is some kind of an error. Post it.
Given that code, there are two failure modes that I can think of:
songPathForm is nil or corrupt and/or not an URL.
the data at the URL is too large to download and causes the app to attempt to allocate a HUGE amount of memory (there are cases where an allocation can be large enough to crash an app w/o the system jetsam mechanism kicking in).
There is no crash message, it just freezes and stops responding. url
is not nil it is...
Then why did you say your app crashed?!
dataWithContentsOfURL: is synchronously downloading the contents of whatever is at the URL.
Thus, you are blocking the main event loop during the download and that is why your app is not responsive.
You need to asynchronously download the data; i.e. not block the main event loop.
However that probably won't entirely fix your problem as it looks like the contents of that URL is really large and, thus, you are likely going to run out of memory if you try to download in memory.
You either need to download the file to the disk or you need to download only parts of it that you need right now or you need to stream it (if it really is a large audio file as the URL implies).
The code in your updated question doesn't make any sense. How do you expect songData to be filled with data from the connection?
When doing stuff asynchronously, you are basically saying "go do this stuff and let me know every now and then how it is going". In this case, that'd be a notification that more data is available or that the connection is done reading (or in error).
You can't ask for the data immediately because the data isn't immediately available.
You'll want to read through this guide.
Is it possible to use an NSURLConnection/NSURLRequest combination to send a PUT request to a server with a Content-Range header? By that I mean I want to resume an upload to the server which can accept a byte range in the request to resume the upload.
I see you can set an NSInputStream as the request body so I figured that I could subclass that and override the the open/seek functions and set the request header but it seems to call on undocumented selectors and break the implementation.
I'm sure I could do this with CFNetwork but it just seems like there must be a way to do it with the higher level APIs.
Any ideas on where to start?
EDIT:
To answer my own question, this is indeed possible after reading a blog [http://bjhomer.blogspot.com/2011/04/subclassing-nsinputstream.html] which details the undocumented callbacks which relate to CFStream. Once those are implemented I can call the following in the open callback to skip ahead:
CFReadStreamSetProperty((CFReadStreamRef)parentStream, kCFStreamPropertyFileCurrentOffset, (CFNumberRef)[NSNumber numberWithUnsignedLongLong:streamOffset]);
Thanks,
J
I think the server needs to supports put method combines with range but this will be the way to do it with high level Objective-C API
NSURL *URL = [NSURL URLWithString:strURL];
NSMutableURLRequest *urlRequest = [NSMutableURLRequest requestWithURL:URL];
NSString *range = [NSString stringWithFormat:#"bytes=%lld-%lld",bytesUploaded,uploadSize];
[urlRequest addValue:range forHTTPHeaderField:#"Range"];
[urlRequest setHTTPMethod:#"PUT"];
self.connection = [NSURLConnection connectionWithRequest:urlRequest delegate:self];
Cheers
First, if you want to do fancy work with HTTP, I typically recommend ASIHTTPRequest. It's solid stuff that simplifies a lot of more complicated HTTP problems. It's not really needed for setting a simple header, but if you're starting to build something more complex, it can be nice to move over to ASI sooner rather than later.
With an NSMutableURLRequest, you can set any header you want using addValue:forHTTPHeaderField:. You can use that to set your Content-Range.
Like I posted in my comment, you can facilitate what you want without dropping down to the CoreFoundation level:
As NSInputStream inherits NSStream, it is possible to prepare the stream as follows:
NSNumber *streamOffset = [NSNumber numberWithUnsignedInteger:lastOffset];
[inputStream setProperty:streamOffset forKey: NSStreamFileCurrentOffsetKey];
(Assuming lastOffset is an NSUInteger representation of your last file offset in bytes and inputStream is the stream you want to set as the request's HTTPBodyStream.)