I'm looking for ways of writing to file the results of a web request. In languages based on the JVM or the CLR there are appropriate Stream-based techniques I'm familiar with, however I'm clueless on how could that be done in Objective-C.
What I need is essentially a way to send an HTTP request (with a custom header set) and write the HTTP response content as I receive it (due to memory constraints I can't afford to get the whole file or even a large portion of it before persisting the contents).
Ideas/suggestions/snippets?
Thanks in advance!
P.S.: I'm developing for Mac OS and I'm already using ASIHTTPRequest, if that can be of help.
Edit: I should specify that I don't want to write all of the contents returned by the server to disk unless I can write them directly at a certain offset of a file (which I'll then be able to manipulate), so anything that dumps straight to a new file or to the beginning of a file won't work for me.
There a few ways of doing it, depends on how you want to handle the responds
ASIHTTPRequest *request = [ASIHTTPRequest requestWithURL:url];
[request setDownloadDestinationPath:#"/Users/Test/Desktop/cool.html"];
with setDownloadDestinationPath: set, it'll save into temporary path, and when it finished, it'll move it to your downloadDestinationPath you set.
Or you can implement request:didReceiveData: delegate (see ASIHTTPRequestDelegate.h), and handle it yourself. This is similar to stream.
PS. I only ever use ASIHTTPRequest on iOS, not Mac OS, so I'm not entirely sure if it will work for you.
Related
I need to invoke a remote REST interface handler and submit it a file in request body. Please note that I don't control the server. I cannot change the request to be multipart, the client has to work in accordance to external specification.
So far I managed to make it work like this (omitting headers etc. for brevity):
byte[] data = readFileCompletely ();
client.target (url).request ().post (Entity.entity (data, "file/mimetype"));
This works, but will fail with huge files that don't fit into memory. And since I have no restriction on filesize, this is a concern.
Question: is it somehow possible to use streams or something similar to avoid reading the whole file into memory?
If possible, I'd prefer to avoid implementation-specific extensions. If not, a solution that works with RESTEasy (on Wildfly) is also acceptable.
ReastEasy as well as Jersey support InputStream out of the box so simply use Entity.entity(inputStream, "application/octet-stream"); or whatever Content-Type header you want to set.
You can go low-level and construct the HTTP request using a library such as the plain java.net.URLConnection.
I have not tried it myself but there is example code which reads a local file and writes it to the request stream without loading it into a byte array.
Upload files from Java client to a HTTP server
Of course this solution requires more manual coding but it should work (unless java.net.URLConnection loads the whole file into memory)
Having followed the Ray Wenderlich Parse + Core Data + AFNetworking tutorial, it seems that he pulls JSON from Parse by creating an AFHTTPRequestOperation object using an NSMutableURLRequest and then adds that operation to a queue via enqueueBatchOfHTTPRequestOperations:(NSArray *)operations progressBlock:(void (^__strong)(NSUInteger, NSUInteger))progressBlock completionBlock:(void (^__strong)(NSArray *__strong))completionBlock.
My question is: Is it possible to process an AFHTTPRequestOperation immediately (rather than adding it to queue) so that the subsequent line of code processes once the request has been fully processed? Is this bad form?
What I am actually trying to do: If a requested object does not exist locally, then attempt to download it from Parse. If that request fails, assume (for now) that the object does not exist on Parse. If that request succeeds, then the user can now start using that object locally.
Advanced apologies if this is a dumb question as I am new to AFNetworking and any kind of data synchronization...also the few iOS developer friends I have have never used AFNetworking / Parse / any kind of data synchronization.
Most people consider it bad form, although I understand wanting to try it when you are getting started. I would advise never doing it in code you are submitting to the App Store. Some discussion about the same thing here: Synchronous AFNetworking calls
Which NSURLConnection wrapper handles GET and POST equally well ?
For GET method, I prefer google's GTMHTTPFetcher over ASIHTTPRequest. ASIHTTPRequest uses delegate, which probably the idea you will come up with normally. But that's the exactly reason I chose not to use it because when you have several connections(many connections in my case), then each connection has its own delegate and you end up with too many object. Or you can have just 1 delegate but you have find a way to find out which response is for which connection.
GTMHTTPFetcher handle this way much better in my opinion. It uses 1 SEL for 1 connection, sorta like target-action model. The code is much cleaner than delegate model.
But for POST method, ASIHTTPRequest has ASIFormDataRequest. I did not find an easy way to do POST with GTMHTTPFetcher. It does have setPostData method to set post data. But you have to set post body and those mime parameters by yourselves(from what I have see) And that's the headache. I find it has another class called GTMHTTPUploadFetcher. But I can't really figure out how to use it (I keep getting the NSAssert "need upload location hdr").
So for POST, I guess ASIHTTPRequest is easier.
I did not get a chance to use facebook-ios-sdk. And would like to hear other opinion about it.
So is there NSURLConnection wrapper handle both GET and POST well ? And any idea how to use GTMHTTPUploadFetcher?
The GTMMIMEDocument class is used to create a stream for uploading via GTMHTTPFetcher (or via anything else that takes an NSInputStream.) An example is here.
GTMMIMEDocument keeps a sparse list of the data parts to be uploaded, avoiding duplicating the data in memory.
I'm building a program, and I'm quite confident using Objective-C, but I don't know how to programmatically download a file from the web and copy it on the hard drive.
I started with :
NSString url = #"http://spiritofpolo.com/images/logo.png";
NSData* data = [NSData dataWithContentsOfURL:[NSURL URLWithString:url]];
But then I don't know what to do with the data... that sucks, no ;)
Can somebody help?
You're close; the last thing you need is a call to -[NSData writeToFile:atomically:].
While that approach, with the final step provided by fbrereto, will work, it does not handle failure gracefully (indeed, it does not handle any sort of failure at all) and will block your application for the duration of the download.
Use NSURLDownload instead. It requires more code, but broken network connections, cut-off downloads, and inaccessible destination paths will not (necessarily) silently break your app.
I've just started development on Macs and have found Cocoa to be a useful and thoughtful framework, but its HTTP functionality has me puzzled.
I have an NSURLConnection object to download a file from my webserver using the HTTP GET method. NSURLConnect's asynchronous connection is great, I get plenty of feedback, I get each chunk received as a new NSData object that I can use to atomically rebuild the file on the client end and, importantly, provide the user with a progress report: [myData length].
Uploads, however, are nowhere near as neat. You can either stick a synchronous request in its own thread or call an asynchronous request (which I believe spawns its own thread), but neither provide you with any useful feedback. There's no delegates to request data or even let me know when data is being sent. Presumably this limits me to files smaller than available memory.
My question is, therefore, is there a simple and elegant solution to HTTP POST file uploads using Cocoa that provides a good deal of feedback and the ability to read files part-by-part, rather than all at once? Or should I write my own class from low-level networking functionality?
Thanks!
You may want to look at the ASIHTTPRequest framework. I haven't used it for uploading but it looks like it has more feedback and the usage is pretty straightforward.
I decided to go with CFNetwork functions instead of NSURLConnection. There appears to be a bit more flexibility in async notifications and in specific features (authentication for instance). Unfortunately it's a bit more complicated (run loops for instance blow my mind) so I recommend you read the CFNetwork reference guide if you go this route:
http://developer.apple.com/documentation/Networking/Conceptual/CFNetwork/Introduction/Introduction.html
Here's a snippet of code from my POST routine, FWIW:
// Create our URL
CFStringRef url = CFSTR("Submit");
CFURLRef myURL = CFURLCreateWithString(kCFAllocatorDefault, url, baseUrl);
// Create the message request (POST)
CFStringRef requestMethod = CFSTR("POST");
CFHTTPMessageRef myRequest = CFHTTPMessageCreateRequest(kCFAllocatorDefault, requestMethod, myURL, kCFHTTPVersion1_1);
// Connect the read socket to the HTTP request stream
CFReadStreamRef myReadStream = CFReadStreamCreateForStreamedHTTPRequest(kCFAllocatorDefault, myRequest, readStream);
// TODO: why does this have to be done?
succ &= CFReadStreamSetClient(myReadStream,
kCFStreamEventOpenCompleted | kCFStreamEventCanAcceptBytes | kCFStreamEventErrorOccurred | kCFStreamEventEndEncountered,
(CFReadStreamClientCallBack) &MyReadCallBack, &myClientContext);
CFReadStreamScheduleWithRunLoop(myReadStream, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode);
succ &= CFReadStreamOpen(myReadStream);
ASIHTTPRequest was originally designed just for this purpose (tracking POST progress), since in the 2.x API, this isn't possible with NSURLConnection. It will definitely be easier to integrate than rolling your own with CFNetwork, and you get lots of other stuff for free (e.g. progress tracking across multiple requests, resuming downloads etc). :)
If the files you are uploading are large, be sure to look at the options for streaming directly from disk, so you don't have to hold the data in memory.
Unfortunately, you're quite correct that NSURLConnection is weak here. The most flexible approach that I would recommend is CocoaAsyncSocket. It means rolling your own HTTP, which is unfortunate, but in most cases not that difficult. CocoaHTTPServer demonstrates how to build a full HTTP server on top of CocoaAsyncSocket, and may have a lot of useful code for your problem. I've found both of these very useful.
Another approach that may be worth investigating is WebKit. Create an invisible WebView, and loadRequest: a POST. I haven't dug into whether the estimatedChange notification system includes the time to upload or only the time to download, but it's worth a try.
You can take a look at the HTTPMessage section of my toolkit repository on github for a simple ObjC wrapper around CFHTTPMessageRef; among other things it'll hand you an NSInputStream object, which saves you thinking about plain-C callback functions.
Depending on what you're reading, you may want to take a look at the StreamingXMLParser section of the same repository for an XML (and HTML) parser which will parse data directly from said NSInputStream on your behalf.