Here is what I want to achieve. 1. Searching all the files 2. find all .jpg files during the searching 3. save all .jpg file paths into NSMutableArray
Here are the codes:
Created the NSMutableArray:
NSMutableArray *jpgFiles = [[[NSMutableArray alloc]init]autorelease];
Searching all the parent folders under (/Users/) path (Start NSThread in here):
NSString* filePath = [url path];
NSArray *dirFiles = [[NSFileManager defaultManager]contentsOfDirectoryAtPath:filePath error:nil];
if([dirFiles count]!=0)
{
for (int j=0; j<[dirFiles count]; j++) {
NSString* pathExtension = [[dirFiles objectAtIndex:j] pathExtension];
//if extension is null, we forwards to next level.
if ([pathExtension isEqualTo:#""])
{
#autoreleasepool {
[NSThread detachNewThreadSelector:#selector(searchingPicture:) toTarget:self withObject:[filePath stringByAppendingPathComponent:[dirFiles objectAtIndex:j]]];
}
}
else
{
//if find jpg in this level, save into array
if([pathExtension isEqualTo:#"JPG"])
{
[jpgFiles addObject:[filePath stringByAppendingPathComponent:[dirFiles objectAtIndex:j]]];
}
}
}
}
Keep searching the rest of sub folders and save proper file path into array:
-(void)searchingPicture:(NSString*)path
{
NSFileManager *fileManager = [[[NSFileManager alloc] init] autorelease];
NSURL *directoryURL = [NSURL URLWithString:[path stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]];
NSArray *keys = [NSArray arrayWithObject:NSURLIsDirectoryKey];
NSDirectoryEnumerator *enumerator = [fileManager
enumeratorAtURL:directoryURL
includingPropertiesForKeys:keys
options:0
errorHandler:^(NSURL *url, NSError *error) {
// Handle the error.
// Return YES if the enumeration should continue after the error.
return YES;
}];
for (NSURL *url in enumerator) {
NSError *error;
NSNumber *isDirectory = nil;
if (! [url getResourceValue:&isDirectory forKey:NSURLIsDirectoryKey error:&error]) {
// handle error
}
else if (! [isDirectory boolValue]) {
// No error and it’s not a directory; do something with the file
if([[[url path] pathExtension]isEqualTo:#"JPG"])
{
//This line gives me error !!!
[jpgFiles addObject:[url path]];
}
}
}
}
Error: (At beginning, it works fine and save many different files into array, but after saved around 50 files it starts to give me error and crash at the end).
Here is the correct element adds into array:
/Users/NAME/Documents/Graduate Books/IMG_2517.JPG
Here is the error message:
-[NSPathStore2 addObject:]: unrecognized selector sent to instance 0x10011d4d0
However, even this error occurs, it still keeps saving some of paths into array and then it will throw another error:
An uncaught exception was raised
Could you guys tell me how to fix it?? Thanks !!
First, trying to increase performance by randomly spawning threads is guaranteed failure. Concurrency must be considered and controlled.
Secondly, trying to decrease execution time of code that is accessing a slow resource (like the filesystem) by concurrently accessing said resource without constraint will be slower than serialized access. I/O to filesystems is relatively slow and linear I/O is always faster than concurrent, conflicted, random I/O.
Finally, NSMutableDictionary is not thread safe. Nor are the other mutable collection classes. If you are shoving stuff into collections from multiple threads, you'll see undefined behavior (typically crashes).
NSMutableArray is not thread safe. It is thread safe when you guard it correctly -- so that no more than one thread is able to use it any time.
To illustrate:
- (void)addPath:(NSString *)pPath
{
[self.lock lock];
[self.files addObject:pPath];
[self.lock unlock];
}
- (NSUInteger)countPaths
{
[self.lock lock];
const NSUInteger count = self.files.count;
[self.lock unlock];
return count;
}
- (NSArray *)copyPaths
{
[self.lock lock];
NSArray * paths = [self.files copy];
[self.lock unlock];
return paths;
}
And as bbum pointed out, directory enumeration as seen in your example is not a problem which lends itself well to parallelization -- Parallelization hurts in this scenario. A more practical approach would be to enumerate from just one thread. If you want to immediately load some of the images, just load them from the "I/O thread" as well.
Related
I would like to obtain the following: I have two NSOperations in a NSOperationQueue. The firs is a download from a website (gets some json data) the next is parsing that data. This are dependent operations.
I don't understand how to link them together. If they are both allocated and in the queue, how do I transfer the json string to the operation that parses it? Is it a problem if this queue is inside another NSOperationQueue that executes an NSOperation that consists of the two mentioned previously?
All I could find is transfers of data to a delegate on the main thread (performSelectorOnMainThread), but I need all this operations to execute in the background.
Thanks.
Code:
NSDownload : NSOperation
- (instancetype)initWithURLString:(NSString *)urlString andDelegate:(id<JSONDataDelegate>)delegate
{
self = [super init];
if (self) {
_urlStr = urlString;
_delegate = delegate; /// this needs to be a NSOPeration
_receivedData = [NSMutableData dataWithCapacity:256];
}
return self;
}
#pragma mark - OVERRIDE
- (void)main
{
#autoreleasepool {
if (self.isCancelled) {
return;
}
NSURL *url = [NSURL URLWithString:self.urlStr];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
self.urlConnection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:YES];
}
}
#pragma mark - NSURLConnectionDataDelegate
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
if (self.isCancelled) {
[connection cancel];
self.receivedData = nil;
return;
}
[self.receivedData appendData:data];
}
- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
if (self.isCancelled) {
self.receivedData = nil;
return;
}
// return data to the delegate
NSDictionary *responseDict = #{JSON_REQUESTED_URL : self.urlStr,
JSON_RECEIVED_RESPONSE : self.receivedData};
[(NSObject *)self.delegate performSelectorOnMainThread:#selector(didReceiveJSONResponse:) withObject:responseDict waitUntilDone:NO]; // ok to uses performSelector as this data is not for use on the main thread ???
}
- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error
{
// return error to the delegate
[(NSObject *)self.delegate performSelectorOnMainThread:#selector(didFailToReceiveDataWithError:) withObject:error waitUntilDone:NO];
}
#user1028028:
Use the following approach.
1) Maintain the operation queue reference that you are using to add DownloadOperation.
2) In connectionDidFinishLoading method, create ParseOperation instance, set the json data and add it to operation queue. Maintain ParseOperation strong reference variable in DownloadOperation and handling of cancelling of parsing operation through DownloadOperation interface.
3) After completed parsing call the UI functionality in main thread.
I hope this helps.
As lucianomarisi notes, it would usually be best to just have the first operation generate the second operation. This is usually simpler to manage. Operation dependencies aren't really that common in my experience.
That said, it's of course possible to pass data between operations. For instance, you could create a datasource property on the second operation. That would be the object to ask for its data; that object would be the first operation. This approach may require locking, though.
You can also create a nextOp property on the first operation. When it completes, it would call setData: on the second operation before exiting. You probably wouldn't need locking for this, but you might. In most cases it would be better for the first operation to just schedule the nextOp at this point (which again looks like lucianomarisi's answer).
The point is that an operation is just an object. It can have any methods and properties you want on it. And you can pass one operation to another.
Keep in mind that since an operation runs in the background, there's no reason you need to use the asynchronous interface to NSURLConnection. The synchronous API (sendSynchronousRequest:returningResponse:error: is fine for this, and much simpler to code. You could even use a trivial NSBlockOperation. Alternately, you can use the asynchronous NSURLConnection interface, but then you really don't need an NSOperation.
I also notice:
_receivedData = [NSMutableData dataWithCapacity:256];
Is it really such a small piece of JSON data? It's hard to believe that this complexity is worth it to move such a small parsing operation to the background.
(As a side note, unless you know precisely the size of the memory, there's not usually much benefit to specifying a capacity manually. Even then it's not always clear that it's a benefit. I believe NSURLConnection is using dispatch data under the covers now, so you're actually requesting a memory block that will never be used. Of course Cocoa also won't allocate it because it optimizes that out... the point is that you might as well just use [NSMutableData data]. Cocoa is quite smart about these kinds of things; you generally can only get in the way of its optimizations.)
As Rob said, unless you have any particular reason to use operations use the synchronized call. Then perform the selector on MainThread or on any other thread you need. Unless you want to separate the retrieval and parsing in separate operations or thread (explicitly).
Here is the code I was using for json retrieval and parsing:
-(BOOL) loadWithURL:(NSString*) url params: (NSDictionary*) params andOutElements:(NSDictionary*) jElements
{
NSError *reqError = nil;
NSString* urlStr = #"";//#"http://";
urlStr = [urlStr stringByAppendingString:url];
NSURL* nsURL = [NSURL URLWithString:urlStr];
//Private API to bypass certificate ERROR Use only for DEBUG
//[NSURLRequest setAllowsAnyHTTPSCertificate:YES forHost:[nsURL host]];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL: nsURL
cachePolicy:NSURLRequestUseProtocolCachePolicy
timeoutInterval:60.0];
[request setHTTPMethod:#"POST"];
NSString *postString = #"";
if(params!=nil) {
NSEnumerator* enumerator = params.keyEnumerator;
NSString* aKey = nil;
while ( (aKey = [enumerator nextObject]) != nil) {
NSString* value = [params objectForKey:aKey];
//Use our own encoded implementation instead of above Apple one due to failing to encode '&'
NSString* escapedUrlString =[value stringByAddingPercentEscapesUsingEncoding:NSASCIIStringEncoding];
//Required to Fix Apple bug with not encoding the '&' to %26
escapedUrlString = [escapedUrlString stringByReplacingOccurrencesOfString: #"&" withString:#"%26"];
//this is custom append method. Please implement it for you -> the result should be 'key=value' or '&keyNotFirst=value'
postString = [self appendCGIPairs:postString key:aKey value:escapedUrlString isFirst:false];
}
}
//************** Use custom enconding instead !!!! Error !!!!! **************
[request setHTTPBody:[postString dataUsingEncoding:NSUTF8StringEncoding]];
NSData *response = [NSURLConnection sendSynchronousRequest:request returningResponse:nil error:&reqError];
if(reqError!=nil) {
NSLog(#"SP Error %#", reqError);
return NO;
}
NSString *json_string = [[NSString alloc] initWithData:response encoding:NSUTF8StringEncoding];
//Handles Server Errors during execution of the web service that handles the call.
if([json_string hasPrefix:#"ERROR"] == YES){
NSLog(#"SP Error %#", lastError);
return NO;
}
//Very Careful!!!!!! Will stop for any reason.!!!!!!
//Handles errors from IIS Server that serves teh request.
NSRange range = [json_string rangeOfString:#"Runtime Error"];
if(range.location != NSNotFound) {
NSLog(#"SP Error %#", lastError);
return NO;
}
//Do the parsing
jElements = [[parser objectWithString:json_string error:nil] copy];
if([parser error] == nil) {
NSLog(#"Parsing completed");
} else {
jElements = nil;
NSLog(#"Json Parser error: %#", parser.error);
NSLog(#"Json string: %#", json_string);
return NO;
}
//Parsed JSON will be on jElements
return YES;
}
Ok, im a bit lost with this one, i am currently trying to run a background core data operation using a second ManagedObjectContext with its type set to NSPrivateQueueConcurrencyType and failing miserably with the above error.
I have a custom subclass of NSOperation, which is being passed an NSArray of strings, and the PersistentStoreCoordinator from the main thread, it then creates its own ManagedObjectContext, runs a query and performs and operation.
Here is the code from the class:
//
// ProcessProfanity.m
// Hashtag Live Desktop
//
// Created by Gareth Jeanne on 24/03/2014.
// Copyright (c) 2014 Gareth Jeanne. All rights reserved.
//
#import "ProcessProfanity.h"
#import "Tweet.h"
static const int ImportBatchSize = 250;
#interface ProcessProfanity ()
#property (nonatomic, copy) NSArray* badWords;
#property (nonatomic, strong) NSManagedObjectContext* backgroundContext;
#property (nonatomic, strong) NSPersistentStoreCoordinator* persistentStoreCoordinator;
#end
#implementation ProcessProfanity
{
}
- (id)initWithStore:(NSPersistentStoreCoordinator*)store badWords:(NSArray*)words
{
self = [super init];
if(self) {
self.persistentStoreCoordinator = store;
self.badWords = words;
}
return self;
}
- (void)main
{
_backgroundContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
_backgroundContext.persistentStoreCoordinator = [self persistentStoreCoordinator];
_backgroundContext.undoManager = nil;
[_backgroundContext performBlockAndWait:^
{
[self import];
}];
}
- (void)import
{
//Create new fetch request
NSFetchRequest *request = [[NSFetchRequest alloc] init];
//Setup the Request
[request setEntity:[NSEntityDescription entityForName:#"Tweet" inManagedObjectContext:self.backgroundContext]];
NSError *error = nil;
//Create an array from the returned objects
NSArray* tweetsToProcess = [self.backgroundContext executeFetchRequest:request error:&error];
NSAssert2(tweetsToProcess != nil && error == nil, #"Error fetching events: %#\n%#", [error localizedDescription], [error userInfo]);
for (Tweet* tweetToCheck in tweetsToProcess){
__block NSString *result = nil;
[self.badWords indexOfObjectWithOptions:NSEnumerationConcurrent
passingTest:^(NSString *obj, NSUInteger idx, BOOL *stop)
{
if (tweetToCheck){
if ([tweetToCheck.text rangeOfString:obj].location != NSNotFound)
{
result = obj;
*stop = YES;
//return YES;
}
}
return NO;
}];
if (!result){
//DDLogVerbose(#"The post does not contain any of the words from the naughty list");
if(tweetToCheck){
tweetToCheck.profanity = [NSNumber numberWithBool:false];
}
}
else{
if(tweetToCheck){
//DDLogVerbose(#"The string contains '%#' from the the naughty list", result);
tweetToCheck.profanity = [NSNumber numberWithBool:true];
}
}
}
[self.backgroundContext save:NULL];
}
#end
And this is how i am calling it:
-(void)checkForProfanity{
if(!self.operationQueue){
self.operationQueue = [[NSOperationQueue alloc] init];
}
NSArray* termsToPass = [self.filterTerms copy];
ProcessProfanity* operation = [[ProcessProfanity alloc] initWithStore:self.persistentStoreCoordinator badWords:termsToPass];
[self.operationQueue addOperation:operation];
}
Edit 1
The specific line i seem to be getting the error on, or at least where Xcode is breaking is:
if ([tweetToCheck.text rangeOfString:obj].location != NSNotFound)
I have managed to narrow this down a bit, the NSArray that contains the list of terms to search the strings for is potentially quite large, possibly over a 1,000 NSStrings. If i test with an array of that size, i get the issue. However if i reduce the array to around 15 NSStrings, i do not get the error, so i don't think this is necessarily a thread related issue, i'm wondering if the array is getting released in the main thread. I have modified the code to make a deep copy, and then a __block copy as follows, but it doesn't seem to have helped.
self.badWords = [[NSArray alloc] initWithArray:words copyItems:YES];
and
for (Tweet* tweetToCheck in tweetsToProcess){
__block NSArray *array = [[NSArray alloc] initWithArray:self.badWords copyItems:YES];
__block NSString *result = nil;
[array indexOfObjectWithOptions:NSEnumerationConcurrent
In fact, at the point where Xcode breaks, if i PO array, i get an object not found message, but if i po result, i correct get an object returned that is nil.
Edit 2
So i have made the following changes, with no change:
Made the NSArray strong rather than copy:
#property (nonatomic, strong) NSArray* badWords;
And made it a copy when allocated:
self.badWords = [[NSArray alloc] initWithArray:words copyItems:YES];
And created a local copy of the NSArray with the ___block declaration inside the actual method processing the objects:
__block NSArray *array = [[NSArray alloc] initWithArray:self.badWords copyItems:YES];
Which should surely mean it sticks around for the life of the ProcessProfanity object?
Am i wrong in expecting to be able to PO the array from the breakpoint within the block?
In this instance the error message "error: NULL _cd_rawData but the object is not being turned into a fault" indicates that you are accessing a managed object outside of its context. Basically your fetch returns all the Tweets from your persistent store as faults. Once you try and access a property on the Managed Object, Core Data will fire a fault and fetch the full object from the store.
By calling the NSArray method indexOfObjectWithOptions:passingTest: with an option of NSEnumerationConcurrent you are implying that you want to perform asynchronous execution on the elements in your array. The keyword concurrent indicates that multiple threads can be used to operate on the array elements.
In your context this means that accessing a managed object inside this block might result in accessing it on a different thread from the managed object context that owns the object. So when you access tweetToCheck.text in your conditional check - if ([tweetToCheck.text rangeOfString:obj].location != NSNotFound), under the hood Core Data is fetching that managed object from the persistent store and returning it to a thread that is not part of the managed object contexts thread.
Furthermore, it is not necessary to use the method indexOfObjectWithOptions:passingTest: since you are not actually interested in the result of this operation.
It seems to me that it might be more convenient for you to use an NSSet as you are only testing to see whether or not a given tweet word exists in your profane words. Quoting the documentation for NSSet: "You can use sets as an alternative to arrays when the order of elements isn’t important and performance in testing whether an object is contained in the set is a consideration". Clearly this seems to meet your criteria.
So your init would look like:
-(id)initWithStore:(NSPersistentStoreCoordinator*)store
badWords:(NSSet*)badWords
{
self = [super init];
if(self) {
self.persistentStoreCoordinator = store;
self.badWords = [words copy];
}
return self;
}
Since you are only interested in updating tweets that have not yet been tagged for profanity you would probably only want to fetch tweets that haven't been flagged profane:
//Create new fetch request
NSFetchRequest *request = [[NSFetchRequest alloc] init];
//Setup the Request
[request setEntity:[NSEntityDescription entityForName:#"Tweet" inManagedObjectContext:self.backgroundContext]];
[request setPredicate:[NSPredicate predicateWithFormat:#"profanity = NO"]];
Now that you have an array of tweets that are not profane you could iterate through your tweets and check each word if it contains a profane word. The only thing you will need to deal with is how to separate your tweet into words (ignoring commas and exclamation marks etc). Then for each word you are going to need to strip it of diacritics and probably ignore the case. So you would end up with someone along the lines of:
if([self.badWords containsObject:badWordString]) {
currentTweet.profanity = [NSNumber numberWithBOOL:YES];
}
Remember, you can run predicates on an NSSet so you could actually perform a case and diacritic insensitive query:
NSPredicate *searchPredicate = [NSPredicate predicateWithFormat:#"SELF = %#[cd]",wordToCheck];
BOOL foundABadWord = ([[[self.badWords filteredSetUsingPredicate:searchPredicate] allObjects] count] > 0);
Another thing you might want to consider is removing duplicate words in your tweets, you don't really want to perform the same check multiple times. So depending on how you find the performance you could place each word of your tweet into an NSSet and simply run the query on the unique words in your tweet:
if([[self.badWords intersectsSet:tweetDividedIntoWordsSet]) {
//we have a profane tweet here!
}
Which implementation you choose is up to you but assuming you are only using english in your app you are definitely going to want to run a case and diacritic insensitive search.
EDIT
One final thing to note is that no matter how much you try, people will always be the best means of detecting profane or abusive language. I encourage you to read this SO's post on detecting profanity - How do you implement a good profanity filter?
Ok, so still not quite sure what was going on, but i followed Daniels advice and re-wrote the indexOfObjectWithOptions method and now it's working. For completeness, and so it hopefully helps someone else, this is what i ended up doing.
DDLogInfo(#"Processing posts to check for bad language");
for (Tweet* tweetToCheck in tweetsToProcess){
__block NSArray *array = [[NSArray alloc] initWithArray:self.badWords copyItems:YES];
__block NSString *result = nil;
NSRange tmprange;
for(NSString *string in array) {
tmprange = [tweetToCheck.text rangeOfString:[NSString stringWithFormat:#" %# ", string]];
if (tmprange.location != NSNotFound) {
result = string;
DDLogVerbose(#"Naughty Word Found: %#", string);
break;
}
}
if (!result){
//DDLogVerbose(#"The post does not contain any of the words from the naughty list");
if(tweetToCheck){
tweetToCheck.profanity = [NSNumber numberWithBool:false];
}
}
else{
if(tweetToCheck){
//DDLogVerbose(#"The string contains '%#' from the the naughty list", result);
tweetToCheck.profanity = [NSNumber numberWithBool:true];
}
}
I've been storing images (JPEG) in CoreData (SQLite store) for a while without problem. I've recently had to import some data from an external source and it appeared to be working fine with the new data. I've just come across an instance of data where the following applies:
1. The entities load fine and have the relevant information
2. The entity relationships are restored
3. The entity that contains the NSData property returns nil for that property
4. When I query the backing store for that object it has binary data showing in the equivalent column
When I've looked into the store further it appears that the problem rows in question only have 38 bytes of information stored in the binary data column. Consistently 30% of all of the images that are processed have this 38 bytes of data stored while the remainder have completely accurate and correct data stored.
The import process runs in a background thread. In order to manage the memory requirements of 4.5Gb of image data I run the import in batches of 100-1000 (initially 1000 but have tried 100 to see if any difference). I create an nsmanagedobjectcontext at the beginning of the batch, process the batch then save the context. During the processing I NSLog the length of data being assigned to the NSData property of the objects in question and they are accurate 100% of the time. However when the data is saved and I look in the store 30% have only 38 bytes of data. I have also tried saving the context after every object is created but have the same 30% dud result.
I have a strong feeling there is an element of multi-threading screwing up my import process but I can't find anything online about this. Would love some input from a CoreData expert as to what the hell is going on with my data!!!
Edit: A subset of the code (there is a lot of other stuff going on in the import process but the image import is reasonably linear)
NSArray *importLines = [[importFileContents componentsSeparatedByString:#"\r\n"] retain];
int batch = 100;
int currentBatch = 0;
int totalLines = importLines.count;
while(currentBatch*batch<totalLines)
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc]init];
// Create a live context
NSManagedObjectContext *liveContext = [[PRModel sharedInstance] newLiveManagedObjectContext];
NSMutableArray *errorMessages = [[NSMutableArray alloc]init];
for(int i=currentBatch*batch; i<(currentBatch+1)*batch && i<totalLines; i++)
{
NSString *importLine = [importLines objectAtIndex:i];
NSArray *importLineData = [importLine componentsSeparatedByString:#"~"];
if([importLineData count]>0)
{
NSString *lineType = [importLineData objectAtIndex:0];
int lineTypeID = [lineType intValue];
NSString *errorMessage = nil;
switch(lineTypeID)
{
// other cases handle other object types and all work fine
case 5:
errorMessage = [DataImporter processPhoto:importLineData withRefDate:refDate intoContext:liveContext];
// try saving every object to see if it makes an effect
[Utils saveManagedObjects:liveContext];
break;
default:
break;
}
if(errorMessage!=nil)
{
[errorMessages addObject:errorMessage];
}
}
}
[Utils saveManagedObjects:liveContext];
[liveContext release];
// Update the errors/missing areas/missing items files in case we get a memory crash
[DataImporter writeErrors:errorMessages];
[errorMessages release];
currentBatch++;
NSLog(#"Batch %d complete",currentBatch);
[pool release];
}
PRModel newLiveManagedObjectContext
-(NSManagedObjectContext*)newLiveManagedObjectContext
{
NSPersistentStoreCoordinator *coordinator = [self livePSC];
NSManagedObjectContext *newContext = nil;
if (coordinator != nil) {
newContext = [[NSManagedObjectContext alloc] init];
[newContext setPersistentStoreCoordinator: coordinator];
}
return newContext;
}
- (NSPersistentStoreCoordinator *)livePSC {
if (livePSC != nil) {
return livePSC;
}
LiveDatabaseUpgradeController *upgrader = [[LiveDatabaseUpgradeController alloc]initWithDelegate:nil];
#try{
livePSC = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:self.currentModel];
NSError *error = nil;
if (![persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:[NSURL fileURLWithPath:[self currentModelStorePath]] options:nil error:&error])
{
NSLog(#"Failed To Open Data Store. Error:%#, %#", error, [error userInfo]);
#throw [NSException exceptionWithName:#"Upgrade Required!" reason:#"attempted to open your database but failed. To quit press the home button." userInfo:nil];
}
}
#catch(NSException *e){
[Utils showAlert:[e name] withMessage:[e reason] withDelegate:nil];
}
#finally{
[upgrader release];
}
return livePSC;
}
The processPhoto method
+(NSString*)processPhoto:(NSArray*)data withRefDate:(NSDate*)refDate intoContext:(NSManagedObjectContext*)liveContext// withData:(bool)saveData
{
if(data.count!=10)
return [NSString stringWithFormat:#"Expected 10 items, found %d",[data count]];
NSString *partialPath = [data objectAtIndex:8];
if([partialPath isEqualToString:#"NULL"])
return nil;
NSString *filePath = [[[PRModel sharedInstance] applicationDocumentsDirectory] stringByAppendingPathComponent:partialPath];
if(![[NSFileManager defaultManager] fileExistsAtPath:filePath])
{
return [NSString stringWithFormat:#"File not found for %#",filePath];
}
// Got everything we need so create a photo
PhotoEntity *photo = [NSEntityDescription insertNewObjectForEntityForName:#"Photo" inManagedObjectContext:liveContext];
photo.photoID = [data objectAtIndex:9];
photo.imageType = #"PHOTO";
photo.photoDescription = #"";
UIImage *photoImage = [[UIImage alloc]initWithContentsOfFile:filePath];
[photo setImage:photoImage inContext:liveContext];
[photoImage release];
return nil;
}
The setImage method is where the data is actually set
-(void)setImage:(UIImage *)image inContext:(NSManagedObjectContext *)context
{
if(self.photoData==nil)
{
// Create a new photo data entity
self.photoData = [NSEntityDescription insertNewObjectForEntityForName:#"PhotoData" inManagedObjectContext:context];
}
UIImage *resizedImage = [image copyImageResizedWithLength:MAX_STORE_RESOLUTION interpolationQuality:kCGInterpolationHigh];
NSData *data = UIImageJPEGRepresentation(resizedImage, 0.6);
NSLog(#"ImageLength=%d",[data length]);
self.photoData.imageData = data;
[resizedImage release];
}
So this code works in 70% of cases but fails in 30%... There's a whole bunch of other stuff going on around this but the code for handling images has been rock solid for at least 6 months in the field with 500 users so it's definitely related to the fact that I'm using it in a mass object background thread process.
Sigh... It turns out that CoreData has decided to store files > 130Kb as external data now. The 38 bytes I am seeing in the store is the Guid relating to the external file. Now I need to work out why valid data is returning as nil in the main body of my app when there is clearly data present.
Edit:
Ok, worked out the missing data issue as well. I need to run the import on the simulator to maximise the available RAM. I then transferred the database (minus external data folder because I didn't realise it was there) to an iPad for testing... When the app loaded the data it attempted to retrieve the external data file but couldn't find it and so loaded the data property as nil. There weren't any exceptions or errors indicating this was the case which is frustrating to say the least.
I retrieve data back from our server and I need to process it.
For each key, I create a NSManagedObject. Each object is created in the same context. I am using Magical Record.
-(id)init {
if (self = [super init]){
self.context = [NSManagedObjectContext MR_contextForCurrentThread];
}
return self;
}
Threading:
dispatch_queue_t queue = dispatch_get_global_queue(0, 0);
dispatch_group_t group = dispatch_group_create();
for (id key in boundariesDictionary) {
dispatch_group_async(group, queue, ^{
DLog(#"nsthread: %#", [NSThread currentThread]);
NSString *boundaryIDString;
if ([key isKindOfClass:[NSString class]]) {
boundaryIDString = key;
}
else if ([key isKindOfClass:[NSNumber class]]) {
boundaryIDString = [key stringValue];
}
if (boundaryIDString) {
DLog(#"boundaryIDString: %#", boundaryIDString)
NSDictionary *boundaryDictionary = [boundariesDictionary objectForKey:key];
Boundary *boundary = [Boundary MR_findFirstWithPredicate:[NSPredicate predicateWithFormat:#"boundaryID == %# AND api == %#", [NSNumber numberWithInteger:[boundaryIDString integerValue]], self.serverCall.API] inContext:self.context];
if ([boundaryDictionary objectForKey:AVI_NAME]) {
if (boundary == nil) {
DLog(#"creating boundary %#", boundaryIDString);
boundary = [Boundary MR_createInContext:self.context];
boundary.boundaryID = [NSNumber numberWithInteger:[boundaryIDString integerValue]];
}
}
boundary = [self processBoundary:boundary fromBoundaryDictionary:boundaryDictionary];
}
}
}
[self processBoundary] just takes the dictionary and sets it to the managed object's attributes.
if ([boundaryDictionary objectForKey:#"name"]) {
boundary.name = [boundaryDictionary objectForKey:#"name"];
}
//more data processing
This is causing an error though:
*** Terminating app due to uncaught exception 'NSGenericException', reason: '*** Collection <__NSCFSet: 0x1776f7f0> was mutated while being enumerated.'
It runs fine if I don't use the same context for each thread.
I don't understand what set other then the NSDictionary boundariesDictionary that i'm enumerating through. I am not mutating boundariesDictionary at all, only copying the data into core data.
When I PO the object (0x1776f7f0 in this case), I get a list of Boundary objects in a set. Those Boundary objects would only exist in the NSManagedObjectContext "set", I don't add them to an NSArray, NSDictionary, or NSSet. But I don't believe I enumerate over that set. I do mutate it by creating new boundary objects to it.
I think there is something going on that I don't understand or quite grasp yet.
Any ideas?
UPDATE:
for (id key in boundariesDictionary) {
NSString *boundaryIDString;
if ([key isKindOfClass:[NSString class]]) {
boundaryIDString = key;
}
else if ([key isKindOfClass:[NSNumber class]]) {
boundaryIDString = [key stringValue];
}
if (boundaryIDString) {
[MagicalRecord saveWithBlock:^(NSManagedObjectContext *localContext) {
DLog(#"saveWithBlock thread: %#", [NSThread currentThread]);
NSDictionary *boundaryDictionary = [boundariesDictionary objectForKey:key];
Boundary *boundary = [Boundary MR_findFirstWithPredicate:[NSPredicate predicateWithFormat:#"boundaryID == %# AND api == %#", [NSNumber numberWithInteger:[boundaryIDString integerValue]], self.serverCall.API] inContext:localContext];
if ([boundaryDictionary objectForKey:AVI_NAME]) {
if (boundary == nil) {
boundary = [Boundary MR_createInContext:localContext];
boundary.boundaryID = [NSNumber numberWithInteger:[boundaryIDString integerValue]];
}
boundary = [self processBoundary:boundary fromBoundaryDictionary:boundaryDictionary];
if (boundary == nil) {
//Prompt Error
}
else {
for (NSNumber *groupID in groupIDs) {
if ([groupID isKindOfClass:[NSNumber class]]) {
Group *group = [Group MR_findFirstWithPredicate:[NSPredicate predicateWithFormat:#"groupID == %# OR groupID == 0 AND api == %#", groupID, self.serverCall.API]];
if (group != nil) {
group.lastUpdated = [NSDate date];
[group addBoundariesObject:boundary];
}
else {
DLog(#"group %# DNE", groupID);
}
}
}
}
}
} completion:^(BOOL success, NSError *error) {
DLog(#"saveWithBlock completion Block | time: %f", [[NSDate date] timeIntervalSinceDate:startTime]);
}];
}
}
So for my Group 29 should see all the boundaries I'm creating, but its not. Its inconsistent. Sometimes sees all, sometimes some, and sometimes none.
Also, I often see
NO CHANGES IN ** BACKGROUND SAVING (ROOT) ** CONTEXT - NOT SAVING
in the log. Its also inconsistent on how many of these messages I see, while the context that do save will insert more then 1 object.
Not sure if that is how it should be behaving, seems like it should be a 1-to-1 ratio if each block has its own context and each block only creates 1 object.
I log each thread ID, and it is creating a new thread for each block. No thread ID is being logged twice, so the threads shouldn't be being reused.
Managed object contexts are not thread safe, you must not use them or any managed objects/sets/data structures/whatever they might return from different queues or threads.
I don't know if magical record supports it, but you should definitely be using queue containment for your contexts, and then you have to do everything within the context of the MOC's private queue.
If you use thread containment, you must guarantee that the context and any managed objects created by the context are always serially accessed, which you're absolutely not doing in the code above.
You're issue is that gcd queues a threads are not a one to one mapping. GCD reuses threads, and thus your are likely crossing thread boundaries unknowingly here. My suggestion is to simply create a new context and stop using contextForCurrentThread. I wrote more details about the issues on my blog. ContextForCurrentThread will be deleted in an upcoming release.
I am new to ObC and have a problem that i just cant fix. There may be other issues as well but the main issue is this:
Starting the app
Press button = load new view
In the new viewDidLoad i call another object/function and send a NSMutableArray
Process data and send back a NSMutableArray
App crash, see comment where. Most often when i go back and back again but sometimes the first time
As i am new to this i guess i do a lot of this wrong but could someone nice take a look at the code and give me some advice. I would assume i have problem with releasing something.
- (void)viewDidLoad {
[super viewDidLoad];
NSLog(#" ");
NSLog(#"viewDidLoad ");
NSLog(#" ");
NSLog(#">>Processing prepareGame<<");
NSMutableArray *propArray1 = [[NSMutableArray alloc] initWithObjects:#"9999", nil]; //Init with dummy numbers
AccessPropertiesFile *readMyProperties = [AccessPropertiesFile new]; //Init function call to read file
NSLog(#"Prepare to call readProperties");
propArray1 = [readMyProperties readPropertiesFile:propArray1];
NSLog(#"Back from readProperties:error after this");
/*
for (NSString *element in propArray1) {
NSLog(#"Elements in prop2Array; %#", element);
}
*/
[readMyProperties release];
[propArray1 release];
}
-(NSMutableArray *)readPropertiesFile:(NSMutableArray *)readDataArray {
NSLog(#"Processing readProperties");
// For error information
NSError *error;
//Prepare File Manager
NSString *filePath = [self dataFilePath];
NSFileManager *fileMgr;
fileMgr = [NSFileManager defaultManager];
NSArray *propertiesArray = [NSArray alloc]; //Alloc array
//Check from what module the call is coming from to ecide what to do
if ([fileMgr fileExistsAtPath: filePath] == NO) {
NSLog (#"File not found");
//File does not exists, this is the first time the game starts
//Set up default parameters
NSString *fileString =#"0\n30\n30\n10\n1\n1\n1\n2\n1\n1\n1\n1\n1\n1\n1\n1\n1\n1\n1\n1\n1\n1\n1\n1\n1\n";
// Write default parameters to file
[fileString writeToFile:filePath atomically:YES encoding:NSUTF8StringEncoding error:&error];
propertiesArray = [fileString componentsSeparatedByString:#"\n"]; // each line, adjust character for line endings
}
else { //File exists
NSLog (#"File exists");
NSString *fileString = [NSString stringWithContentsOfFile:filePath
encoding:NSUTF8StringEncoding error:nil]; // reads file into memory as an NSString
propertiesArray = [fileString componentsSeparatedByString:#"\n"]; // each line, adjust character for line endings
}
//Clean readDataArray
[readDataArray removeAllObjects];
//Populate return array
for (NSString *element in propertiesArray) {
//NSLog(#"Elements in propertiesArray; %#", element);
[readDataArray addObject:element];
}
NSLog(#"readDataArray: %#", readDataArray);
[propertiesArray release];
[readDataArray autorelease];
NSLog(#"returning from readProperties");
return readDataArray;
}
#end
You are over-releasing readDataArray (known as propArray1 in the method that didn't create it). You create it and autorelease it in your second method, then you release it again at the end of your first method (where it wasn't created).
I suggest you use Analyze feature that comes with latest XCode. It is a good feature that I always use to track if I forget to release or release too much.
I also spotted that you also over-release the propertiesArray because it contains the result from [fileString componentsSeparatedByString:], which will be autorelease according to Cocoa convention.