NSURLCache: inconsistent behaviour - objective-c

Question

I was observirng some strange behaviour of my app sometime caching responses and sometime not caching them (all the responses have Cache-Control: max-age=600).
The test is simple: I did a test.php script that was just setting the headers and returning a simple JSON:
<?php
header('Content-Type: application/json');
header('Cache-Control: max-age=600');
?>
{
"result": {
"employeeId": "<?php echo $_GET['eId']; ?>",
"dateTime": "<?php echo date('Y-m-d H:i:s'); ?>'" }
}
This is the response I get from the PHP page:
HTTP/1.1 200 OK
Date: Thu, 28 Nov 2013 11:41:55 GMT
Server: Apache
X-Powered-By: PHP/5.3.17
Cache-Control: max-age=600
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/json
{
"result": {
"employeeId": "",
"dateTime": "2013-11-28 11:41:55'"
}
}
Then I've created a simple app and added AFNetworking library.
When I call the script with few parameters, the cache works properly:
AFHTTPRequestOperationManager *manager = [AFHTTPRequestOperationManager manager];
NSDictionary *params = #{
#"oId": #"4011",
#"eId": self.firstTest ? #"1" : #"0",
#"status": #"2031",
};
[manager GET:#"http://www.mydomain.co.uk/test.php" parameters:params success:^(AFHTTPRequestOperation *operation, id responseObject) {
NSLog(#"JSON: %#", responseObject);
NSLog(#"Cache current memory usage (after call): %d", [cache currentMemoryUsage]);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"Error: %#", error);
}];
But when I increase the number of parameters, like:
NSDictionary *params = #{
#"organizationId": #"4011",
#"organizationId2": #"4012",
#"organizationId3": #"4013",
#"organizationId4": #"4014",
#"organizationId5": #"4015",
#"organizationId6": #"4016",
#"eId": self.firstTest ? #"1" : #"0",
#"status": #"2031",
};
it doesn't work anymore and it execute a new request each time it is called.
I've done many tests and it seems to me that it is related to the length of the URL, because if I includes this set of params:
NSDictionary *params = #{
#"oId": #"4011",
#"oId2": #"4012",
#"oId3": #"4013",
#"oId4": #"4014",
#"oId5": #"4015",
#"oId6": #"4016",
#"eId": self.firstTest ? #"1" : #"0",
#"status": #"2031",
};
It works!!
I've done many tests and that's the only pattern I've found...
To exclude AFNetworking from the equation, I've created another test program that uses NSURLConnection only and I can see the same behaviour so it's not AFNetworking and definitely NSURLCache. This is the other test:
NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:#"http://www.mydomain.co.uk/test.php?eId=%#&organizationId=4011&organizationId2=4012&organizationId3=4013&organizationId4=4014&organizationId5=4015&organizationId6=4016", self.firstTest ? #"1" : #"0"]]; // doesn't work
//NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:#"http://www.mydomain.co.uk/test.php?eId=%#&oId=4011&oId2=4012&oId3=4013&oId4=4014&oId5=4015&oId6=4016", self.firstTest ? #"1" : #"0"]]; // work
//NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:#"http://www.mydomain.co.uk/test.php?eId=%#", self.firstTest ? #"1" : #"0"]]; // work
NSURLRequest *request = [NSURLRequest requestWithURL:url];
NSURLResponse *response = nil;
NSError *error = nil;
NSData *data = [NSURLConnection sendSynchronousRequest:request
returningResponse:&response
error:&error];
if (error == nil) {
// Parse data here
NSString *responseDataStr = [NSString stringWithUTF8String:[data bytes]];
NSLog(#"Response data: %#", responseDataStr);
}
I've also tried to establish how many characters in the URL will trigger the problem but even in this case I've got strange results:
This one is 112 characters long and it doesn't work:
http://www.mydomain.co.uk/test.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&orgaId4=4
This one is 111 characters long and it works:
http://www.mydomain.co.uk/test.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&orgId4=4
Ive renamed the PHP script to see if the first part of the URL would matter and I've got a strange behaviour again:
This one is 106 characters long and it doesn't work:
http://www.mydomain.co.uk/t.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&org=40
This one is 105 characters long and it works:
http://www.mydomain.co.uk/t.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&org=4
So I've removed 3 characters from the page name and I've got a working threshold 6 characters lower.
Any suggestion?
Thanks,
Dem

Answer 1

I am witnessing something similar with certain responses not being cached by NSURLCache and I have come up with another possible reason:
In my case I have been able to ascertain that the responses not being cached are the ones that are returned using Chunked transfer-encoding. I've read elsewhere that NSURLCache should cache those after iOS 6 but for some reason it doesn't in my case (iOS 7.1 and 8.1).
I see that your example response shown here, also has the Transfer-Encoding: chunked header.
Could it be that some of your responses are returned with chunked encoding (those that are not cached) and some are not (those that are cached)?
My back-end is also running PHP on Apache and I still can't figure out why it does that...
Probably some Apache extension...
Anyway, I think it sounds more plausible than the request URL length scenario.
EDIT:
It's been a while, but I can finally confirm that in our case, it is the chunked transfer encoding that causes the response not to be cached. I have tested that with iOS 7.1, 8.1, 8.3 and 8.4.
Since I understand that it is not always easy to change that setting on your server, I have a solution to suggest, for people who are using AFNetworking 2 and subclassing AFHTTPSessionManager.
You could add your sub-class as an observer for AFNetworking's AFNetworkingTaskDidCompleteNotification, which contains all the things you will need to cache the responses yourself. That means: the session data task, the response object and the response data before it has been processed by the response serializer.
If your server uses chunked encoding for only a few of its responses, you could add code in -(void)didCompleteTask: to only cache responses selectively. So for example you could check for the transfer-encoding response header, or cache the response based on other criteria.
The example HTTPSessionManager sub-class below caches all responses that return any data:
MyHTTPSessionManager.h
#interface MyHTTPSessionManager : AFHTTPSessionManager
#end
MyHTTPSessionManager.m
#import "MyHTTPSessionManager.h"
#implementation MyHTTPSessionManager
+ (instancetype)sharedClient {
static MyHTTPClient *_sharedClient = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
[[NSNotificationCenter defaultCenter] addObserver:_sharedClient selector:#selector(didCompleteTask:) name:AFNetworkingTaskDidCompleteNotification object:nil];
});
return _sharedClient;
}
- (void)didCompleteTask:(NSNotification *)notification {
NSURLSessionDataTask *task = notification.object;
NSHTTPURLResponse *response = (NSHTTPURLResponse *)task.response;
NSData *responseData = notification.userInfo[AFNetworkingTaskDidCompleteResponseDataKey];
if (!responseData.length) {
// Do not cache empty responses.
// You could place additional checks above to cache responses selectively.
return;
}
NSCachedURLResponse *cachedResponse = [[NSCachedURLResponse alloc] initWithResponse:response data:responseData];
[[NSURLCache sharedURLCache] storeCachedResponse:cachedResponse forRequest:task.currentRequest];
}
I tried to come up with some sort of cleaner solution, but it seems that AFNetworking does not provide a callback or a delegate method that returns everything we need early enough - that is, before it has been serialized by the response serializer.
Hope people will find this helpful :)

Answer 2

Did you try to configure
NSURLRequestCachePolicy
for NSURLRequest
+ (id)requestWithURL:(NSURL *)theURL cachePolicy:(NSURLRequestCachePolicy)cachePolicy timeoutInterval:(NSTimeInterval)timeoutInterval
These constants are used to specify interaction with the cached responses.
enum
{
NSURLRequestUseProtocolCachePolicy = 0,
NSURLRequestReloadIgnoringLocalCacheData = 1,
NSURLRequestReloadIgnoringLocalAndRemoteCacheData =4,
NSURLRequestReloadIgnoringCacheData = NSURLRequestReloadIgnoringLocalCacheData,
NSURLRequestReturnCacheDataElseLoad = 2,
NSURLRequestReturnCacheDataDontLoad = 3,
NSURLRequestReloadRevalidatingCacheData = 5
};
typedef NSUInteger NSURLRequestCachePolicy;

Answer 3

You could investigate what your cached response is from the sharedURLCache by subclassing NSURLProtocol and overriding startLoading:
add in AppDelegate application:didFinishLaunchingWithOptions:
[NSURLProtocol registerClass:[CustomURLProtocol class]];
Then create a subclass of NSURLProtocol (CustomURLProtol) and override startLoading
- (void)startLoading
{
self.cachedResponse = [[NSURLCache sharedURLCache] cachedResponseForRequest:self.request];
if (self.cachedResponse) {
[self.client URLProtocol:self
didReceiveResponse:[self.cachedResponse response]
cacheStoragePolicy:[self.cachedResponse storagePolicy]];
[self.client URLProtocol:self didLoadData:[self.cachedResponse data]];
}
[self.client URLProtocolDidFinishLoading:self];
}
self.cachedResponse is a property NSCachedURLResponse i've added. You can see if anything is wrong with any cachedResponse here.