Sqlite3 database in iPhone gets locked - how to avoid? - objective-c

I have a query that performs a search on an Sqlite3 DB. It does nothing but read using a reader.
For each found match it calls a callback to the UI which updates a result view.
While this search is running, I hit a button in the UI which will perform some other action in a new thread. In the end it is supposed to remove the search controller's view and show a new controller.
However, at some point the the triggered action wants to write to the databse. And there it just hangs and eventually I will see an exception that the DB is locked.
Interesting is also, that the search reader does not continue either, it is a deadlock.
Do I have to open the database in some special way to support multithreaded usage? What would the constructor for the connection be?

MonoTouch 5.1+ provides an API to let you select the threading model to be used with SQLite.
SqliteConnection.SetConfig (SQLiteConfig.MultiThread);
This maps to some of the connection options of SQLite library.
UPDATE: If you're using an earlier version of MonoTouch (e.g. between 4.2 and 5.0.x) you can use the binary attached to the bug report #652 (follow the instructions) or copy-paste the patch (p/invoke and enum) inside your own application.

I'm not sure I interpret your description correctly, but the way you describe it, it sounds to me that your "reader" steps through the database row-by-row and every time it finds a result it does a callback to a callback function? Is this correct?
If that is the case, you might repeatedly lock your DB, and your search will be slow.
The right way is to extract all matches into a result set in one single query - once that query is completed the lock will be released and you have a result set from SQL that contains only the matching rows.
You let SQLite create a result set like this by using a query of the type "SELECT * FROM tablename WHERE columnX LIKE '%searchstring%'"
(or similar, depending on your search criteria)
This will create a result set with all matches in the database and then release the database lock. Then you can step through the result and create objects and put into and NSArray that is connected to your UI view.
NSArray retval = [NSMutableArray array];
//Create a query
NSString *query = [NSString stringWithFormat:#"SELECT * FROM %# WHERE %# LIKE %#",
tableName, columnName, searchString];
sqlite3_stmt *statement;
//Database locks here
if (sqlite3_prepare_v2(_database, [query UTF8String], -1, &statement, nil)
== SQLITE_OK) {
//Database should unlock here, the query is finished
while (sqlite3_step(statement) == SQLITE_ROW) {
char *nameChars = (char *) sqlite3_column_text(statement, 0);
NSString *name = [NSString stringWithUTF8String:nameChars];
SomeClass *info = [[SomeClass alloc] initWithName:name];
/* Extract other columns and put in your object */
[retval addObject:info];
[info release];
}
sqlite3_finalize(statement);
} else {
NSLog(#"SQL-statement failed");
}
Doing this way there shouldn't be a problem to write to the DB when it is necessary. Only perform new queries to the DB when it's absolutely necessary, for example when your search criteria changed or the content in the DB has been updated.
Do not run repeated queries to a DB that has not changed, or with unchanged search criteria.

Related

NSPredicate with "NOT IN" condition fails

Ok, here's my problem. I am synchronizing data from a server via a REST-api. The returned data is in JSON, I loop through it and takes appropriate actions depending on the data. That is, I either store it as a new object, updates the object if it already exists or deletes it if only exists locally.
To achieve this, I collect the IDs from the returned objects when I loop through the JSON. This gives me a index of all the returned objects. I then query my locally stored data to see if it contains any objects that should be deleted (in other words, if the local ID does exists or not in the JSON response).
And here's my issue (sorry for a somewhat lengthy prologue); the NSPredicate that I use only works for certain scenarios and which ones work or fails seems to be random.
[MagicalRecord saveWithBlock:^(NSManagedObjectContext *localContext) {
// Array which populates with the IDs from the server
NSMutableArray *arrayOfLogIDS = [[NSMutableArray alloc] init];
/*
Fetching and parsing JSON ... collecting IDs and adding them to the array. See example below;
*/
NSArray *logs = [[json valueForKey:#"Logs"] valueForKey:#"Object"];
// Looping through the logs array
for (NSArray *log in logs) {
[arrayOfLogIDS addObject:[log valueForKey:#"serverID"]];
}
// The NSPredicate
NSPredicate *serverIDS = [NSPredicate predicateWithFormat:#"NOT (serverID IN %#)", arrayOfLogIDS];
// The array which holds the objects that should be deleted
NSArray *logs = [Logs MR_findAllWithPredicate:serverIDS inContext:localContext];
}];
The problem is just that the NSPredicate won't work for this specific circumstance. It returns no results even though I know I have objects locally that should be deleted.
I use this approach in other places in the application, and it works as expected. As you can see I am using Magical Record for Core Data management in this app.
I feel that I have completely run out of things to try next, so any help would be much appreciated! :)
Ok, as it turns out, the array of IDs sometimes had the values stored as string and sometimes as integers. Integers worked well with NSPredicate, strings not so much :) Solved! Thanks all for your time.

Objective-C - FMDB - Large SQLite Dump Import

I have a large SQLite file filled up with queries to create my database tables and insert all records. The file is rather large and running the SQL file seems to take much longer than I would have expected.
I am using FMDB for an iPad app I am working on, and I really want to just replace the current DB file with a new one, but I am not sure that an sql file is the same as a DB file. It doens't contain any of the same header info, etc...
What's the best way to go about doing this?
If doing a lot of separate UPDATE or INSERT calls with FMDatabase, consider doing beginTransaction at the start and commit at the end:
[db beginTransaction];
// do all of your updates
[db commit];
Or, if using FMDatabaseQueue, use inTransaction:
[databaseQueue inTransaction:^(FMDatabase *db , BOOL *rollback) {
// do all of your updates
}];
If you don't use one of those, it will commit after each insert, which makes it much slower. The difference can be dramatic if adding lots of rows (e.g. I've seen performance differences of two orders of magnitude when adding/updating lots of little records).
The above assumes that you are trying to perform a series of separate SQL commands. If it's all in one file (such as .dump output), FMDB hasn't historically had an interface to do that (even though there is a SQLite function, sqlite3_exec, that does precisely this). There was a recent addition to the extra folder called FMDatabaseSplitter, which attempts to splits a long string of SQL into separate calls which you can then invoke separately.
Personally, it makes me nervous to use a third-party SQL parsing routine, so I'd just be inclined to call the SQLite function sqlite3_exec directly. To do that, you can access the sqlite3 pointer from your FMDatabase object using the FMDB sqliteHandle method, and then use that in conjunction with the sqlite3_exec function directly:
NSError *error = nil;
NSString *dumpSQL = [NSString stringWithContentsOfFile:dumpFilePath encoding:NSUTF8StringEncoding error:&error];
NSAssert(dumpSQL, #"Loading of SQL failed: %#", error);
int rc = sqlite3_exec(db.sqliteHandle, [dumpSQL UTF8String], NULL, NULL, NULL);
if (rc != SQLITE_OK) {
NSLog(#"sqlite3_exec error: %#", [db lastErrorMessage]);
}
I must confess, it makes me a little nervous to just take bulk SQL an import it into an app's database. An innocent mistake in the SQL could brick the app of your entire install-base if you're not extremely careful. I'd rather see the app request JSON or XML feed from the server, and then do the updates itself, but if you want to use the .dump output to update the app's database with FMDB, this is one way to do it.
FMDB v2.3 has introduced a wrapper for sqlite3_exec called executeStatements:
BOOL success;
NSString *sql = #"create table bulktest1 (id integer primary key autoincrement, x text);"
"create table bulktest2 (id integer primary key autoincrement, y text);"
"create table bulktest3 (id integer primary key autoincrement, z text);"
"insert into bulktest1 (x) values ('XXX');"
"insert into bulktest2 (y) values ('YYY');"
"insert into bulktest3 (z) values ('ZZZ');";
success = [db executeStatements:sql];

SQLite will not prepare query when accessing database in Xcode

I'm a beginner iPhone developer trying to take information out of a sqlite database in Xcode 4.3. I have my database (which is named DB_Info.sqlite) in the same directory as my .h and .m files, and I also dragged the database into the folders section on the left bar in Xcode.
Could you please take a quick look at my code and let me know where my mistake is? I have used NSLogs to identify where the problem occurs, at the very last if statement, and it's written in comments. Thank you so much in advance!
#import <sqlite3.h>
#implementation Player
{
sqlite3 *DB_Info;
NSString *databasePath;
NSString *docsDir;
NSArray *dirPaths;
}
-(Player*)createPlayer:(NSString*)playerName
{
dirPaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
docsDir = [dirPaths objectAtIndex:0];
databasePath = [[NSString alloc] initWithString: [docsDir stringByAppendingPathComponent: #"DB_Info.sqlite"]];
const char *dbpath = [databasePath UTF8String];
sqlite3_stmt *statement;
if (sqlite3_open(dbpath, &DB_Info) == SQLITE_OK) { //works fine
NSString *querySQL = [NSString stringWithFormat: #"SELECT * FROM playerlist WHERE fullName=\"%#\"", playerName];
const char *query_stmt = [querySQL UTF8String];
if (sqlite3_prepare_v2(DB_Info, query_stmt, -1, &statement, NULL) == SQLITE_OK) { //PROBLEM: This is where the problem is, and the if statement never goes through
//....rest of code here
} else {
NSLog(#"Error");
}
}
First, rather than just saying "Error", log the SQL error message
NSLog(#"%s SQL error '%s' (%1d)", __FUNCTION__, sqlite3_errmsg(database), sqlite3_errcode(database));`
It will tell you precisely what's going wrong. A common error on people's first time SQL projects is that the table is not found. If so, read on. (If not, feel free to ignore the rest of this.)
Second, you're looking for your database in your Documents folder. Did you explicitly copy it from your bundle to your Documents folder at some point? Or did you create it programmatically? But if you prepared it in advance, it won't be in the Documents folder until you copy it there.
Third, I'd also suggest that you consider using sqlite3_open_v2 instead of sqlite3_open. You are checking to see if that's SQLITE_OK, but that may be giving a false sense of security. The sqlite3_open will create a database if it's not there, which is clearly not your intent. Your app should presumably being copying the db from the bundle or creating the database and the tables for you before you get to this method (and you're not showing that, so I'm not sure if you're doing that). Anyway, the sqlite3_open_v2 function will not create the database if it's not there unless you explicitly request it does so by including SQLITE_OPEN_CREATE. So, something like the following will not create the database if it's not found:
if (sqlite3_open_v2(dbpath, &DB_Info, SQLITE_OPEN_READWRITE, NULL) == SQLITE_OK) {
On the off chance that a blank database has been created for you, I'd suggest you reset your simulator via "Reset Content and Settings" on the simulator's menu, or explicitly delete the app, so any blank databases that might have been created can be removed. If you're running this on a device, delete the app and reinstall it.
Fourth, if the database has been created in advance, have you confirmed that the database has been included in the "Copy Bundle Resources" setting? E.g.:
Fifth, if you're running the app on the simulator, you can always browse the simulator's file structure and make sure your files are where you think they are. You can also run the Mac OS sqlite program to inspect the database that the simulator is using to make sure everything is ok. You can even test your SQL right in the db that the simulator uses, which can be useful for diagnostic purposes. (Or, if you don't like the Mac text based sqlite program, you can also buy/download graphical tools, such as Base.) Before you can do this, you might first want to configure your Mac so you can easily browse the Simulator's files requires that you fire up the Terminal program, and issue the command:
chflags nohidden ~/Library
Then you can browse to "~/Library/Application\ Support/iPhone\ Simulator/5.1/Applications/" and then browse the various apps that you have and make sure you db file is there.

Why can't i RemoveAllObjects of NSMutableArray in iOS?

I'm developing a app in Searching with sqlite database using ARC.
I have to retrieve data from database into NSMutableArray to show retrieved data when textDidChange Event of UISearchBar.
It's has alot of data retrieve from database when textDidChange of UISearchBar.
After retrieved alot of data from database,i have always clear from NSMutableArray with following codes.
[self.myArrayData removeAllObjects];
However i remove all of data from NSMutableArray , memory allocations are still occurring.
Not reduced.
Next time when i type something in UISearchBar, memory allocations are still increasing.
After i typed 10 times or above,I receive memory warning from XCodes and out of memory.
I want to know why data from NSMutableArray isn't clear when i called removeAllObjects method.
I am developing my app with ARC.
Please help me.
You don't have a lot of information here but it may not be NSMutableArray holding on to the data. SQLite also keeps a cache of recently retrieved records. You can reduce/remove the SQLCache by executing this after you've opened the database. Set the cache size to the number of pages you want SQLite to cache. The default is 2000. Each page is 1024 bytes in size.
const char *pragmaSql = "PRAGMA cache_size = 0";
if (sqlite3_exec(database, pragmaSql, NULL, NULL, NULL) != SQLITE_OK) {
NSAssert1(0, #"Error: failed to execute pragma statement with message '%s'.", sqlite3_errmsg(database));
}
EDIT: Technically you can't reduce the cache below 10. If you try, the program will simply set the cache to 10.
if after [self.myArrayData removeAllObjects]; [self.myArrayData count] == 0 you can be sure that your array has removed its objects and decremented their retain counts... but if other objects still own them they they wont actually be dealloc'ed... so the objects may be over-retained and therefor leaked or they may just be owned by something else.

problem with saving data at coredata?

In my application there is searchBar. when we input a text, it will do functionGrab (grab data from internet and save it to coredata), example :
if we input "Hallo"
if([[dict objectForKey:#"Category"] isNotEmpty] && [[[dict objectForKey:#"Category"] objectAtIndex:0] class]!=[NSNull class]){
NSMutableArray * DownloadedTags =[dict objectForKey:#"Category"];
NSMutableSet * TagsReturn=[NSMutableSet set];
for(int i=0;i<[DownloadedTags count];i++){
NSString * Value=[DownloadedTags objectAtIndex:i];
Tag * thisTag= (Tag*)[GrabClass getObjectWithStringOfValue:Value fromTable:#"Tag" withAttribut:#"Name"];
[TagsReturn addObject:thisTag];
}
NSMutableSet * manyManagedObjects = [BusinessToSave mutableSetValueForKey:#"Tags"];
[self removeDifferenceBetween2MutableManagedObjectSets:manyManagedObjects withDownloadedVersion:TagsReturn];
}
So each biz has many categories. WHat happen in multi threaded application is one thread put category. The other thread also put the same category before committing.
So, [GrabClass getObjectWithStringOfValue:Value fromTable:#"Tag" withAttribut:#"Name"]; gives a new object even though some other thread already created the same object without knowing it.
If I synchronized the whole thing that the code would run serially and that'll be slow.
functionGrab:"H"
functionGrab:"Ha"
functionGrab:"Hal"
functionGrab:"Hall"
functionGrab:"Hallo"
something like,it do that functionGrab 5 times
I want to make functionGrab at background, but the problem is when I do that function without synchronized it will save more than one of data, so the result is there are 5 hallo words in my coredata, but if I do that with synchronized, it spent so much time, so slow..
is there any way to help my problem?
I do not recommended having more than one thread "creating" the same types of data for the exact reason you are running into.
I would suggest you queue all of your "creates" into a single thread and a single NSManagedObjectContext to avoid merge or duplication issues.
The other option would be to make the app Lion only and use the parent/child NSManagedObjectContext design and then your children will be more "aware" of each other.