Why does NSURLConnection timeout when sending many requests? - objective-c

Question

I am writing a network class for an iOS app. This class will take care of all logging and network traffic. I have a problem where I have to send possibly thousands of requests at one time, but NSURLConnections are timing out because the delegate methods will not be called until all the NSURLConnections are started, by which time the timeout period has expired. I am using a rest API for Drupal and, unfortunately, I do not know of a way to create multiple instances with one request. How can I receive responses while simultaneously sending them? If I use GCD to pass off the creation of the NSURLConnections, will that solve the problem? I think I would have to pass the entire operation of iterating over the objects to send and sending to GCD to free up the main thread to answer to responses.
-(BOOL)sendOperation:(NetworkOperation)op
NetworkDataType:(NetworkDataType)dataType
JsonToSend:(NSArray *)json
BackupData:(NSArray *)data
{
if(loggingMode)
{
return YES;
}
NSURLConnection *networkConnection;
NSData *send;
NSString *uuid = [self generateUUID];
NSMutableArray *connections = [[NSMutableArray alloc] init];
NSMutableURLRequest *networkRequest;
for (int i=0; i<[json count] && (data ? i<[data count] : YES); i++)
{
if(op == Login)
{
/*Grab all cookies from the server domain and delete them, this prevents login failure
because user was already logged in. Probably find a better solution like recovering
from the error*/
NSArray *cookies = [[NSHTTPCookieStorage sharedHTTPCookieStorage] cookiesForURL:
[[NSURL alloc] initWithString:networkServerAddress]];
for (NSHTTPCookie *cookie in cookies)
{
[[NSHTTPCookieStorage sharedHTTPCookieStorage] deleteCookie:cookie];
}
networkRequest = [[NSMutableURLRequest alloc] initWithURL:
[NSURL URLWithString:[networkServerAddress stringByAppendingString:#"/user/login"]]];
}
else if(op == StartExperiment)
{
networkRequest = [[NSMutableURLRequest alloc] initWithURL:
[NSURL URLWithString:[networkServerAddress stringByAppendingString:#"/node"]]];
}
else if(op == Event || op == EndExperiment || op == SendAll)
{
networkRequest = [[NSMutableURLRequest alloc] initWithURL:
[NSURL URLWithString:[networkServerAddress stringByAppendingString:#"/node"]]];
}
else if(op == Logout)
{
networkRequest = [[NSMutableURLRequest alloc] initWithURL:
[NSURL URLWithString:[networkServerAddress stringByAppendingString:#"/user/logout"]]];
}
send = [[json objectAtIndex:i] dataUsingEncoding:NSUTF8StringEncoding];
//Set the headers appropriately
[networkRequest setHTTPMethod:#"POST"];
[networkRequest setValue:#"application/json"
forHTTPHeaderField: #"Content-type"];
[networkRequest setValue:[NSString stringWithFormat:#"%d", [send length]]
forHTTPHeaderField:#"Content-length"];
[networkRequest setValue:#"application/json"
forHTTPHeaderField:#"Accept"];
//Set the body to the json encoded string
[networkRequest setHTTPBody:send];
//Starts async request
networkConnection = [[NSURLConnection alloc] initWithRequest:networkRequest delegate:self];
//Successfully created, we are off
if(networkConnection)
{
[networkConnectionsAndData setValue:[[NSMutableArray alloc] initWithObjects:uuid,
[[NSNumber alloc] initWithInt:op], [[NSNumber alloc] initWithInt:dataType], [[NSMutableData alloc] init], (data ? [data objectAtIndex:i] : [NSNull null]), nil]
forKey:[networkConnection description]];
}
else //Failed to conn ect
{
NSLog(#"Failed to create NSURLConnection");
return NO;
}
}
[[self networkOperationAndConnections] setObject:[[NSMutableDictionary alloc] initWithObjectsAndKeys:[[NSMutableArray alloc] initWithObjects:connections, nil], #"connections", [[NSMutableArray alloc] init], #"errors", nil]
forKey:uuid];
return YES;
}
The dictionaries are used to keep track of the correlating data with each NSURLConnection and also to group the NSURLConnections together into one group to determine ultimate success or failure of an entire operation.
Update
AFNetworking was key in finishing this project. It not only cleaned up the code substantially, but dealt with all the threading issues inherit in sending so many requests. Not to mention with AFNetworking I could batch all the requests together into a single operation. Using blocks, like AFNetworking uses, was a much cleaner and better solution than the standard delegates for NSURLConnections.

Answer 1

You definitely need to allow the NSURLRequest / Connection to be operating on another thread. (Not the main thread!)
Edited for clarity**:
I noticed your comment of "//Starts async request" and I wanted to be sure you realized that your call there is not what you would expect out of a typical "asynch" function. Really its just firing off the request synchronously, but since its a web request it inherently behaves asynchronously. You want to actually place these requests on a another thread for full asynch behavior.
Everything else aside, I really suggest digging into Apple's networking example project here: MVCNetworking
As for specifics on your question, there's a couple ways to do this.
One is to keep your connection from starting immediately using initWithRequest:<blah> delegate:<blah> startImmediately:FALSE and then schedule your NSURLConnection instances on another thread's run-loop using: scheduleInRunLoop:forMode:
(Note: You then have to kick off the connection by calling start-- it's best to do this via an NSOperation + NSOperationQueue.)
Or use this static method on NSURLConnection to create/launch the connection instead of doing an alloc/init: sendAsynchronousRequest:queue:completionHandler:
(Note: this approach accomplishes pretty much same as above but obfuscates the details and takes some of the control out of your hands.)
To be honest my quick answers above won't be sufficient to finish this kind of project, and you'll need to do a bit of research to fill in the blanks, especially for the NSOperationQueue, and that's where the MVCNetworking project will help you.
Network connections are a fickle beast -- You can time-out and kill your connections even if they're running on a background thread simply by trying to perform too much work simultaneously! I would seriously reconsider opening up several thousand NSURLConnections at once, and using an NSOperationQueue would help work around this.
ADDITIONAL RESOURCES:
Here's a 3rd party library that may make your networking adventures less painful:
https://github.com/AFNetworking/AFNetworking
http://engineering.gowalla.com/2011/10/24/afnetworking/