I'm working on a iOS 6 app, and am using a SQLite database to store data. At startup the app does a select on the database and displays the results on the first screen.
However I've started to get an "Database disk image is malformed" error when trying to run the select.
The strange thing is that I can use SQLite browser, http://sqlitebrowser.sourceforge.net/, to do the select on the database in the project folder. But if I try to open the database after it has been copied to the simulator folder, /Users//Library/ApplicationSupport/iPhone simulator/6.0/Applications/... then I get a disk image is malformed.
The database is not being accessed by a background thread, nor am I using breakpoints to halt execution as is being suggested as a reason here sqlite database disk image malformed on iPhone SDK.
No more than one execution being done on the database at once.
All hints, tips and possible solutions are appreciated.
I've found out that this had something to do with me bundling a rather large database file with the app (larger than 3 GB). I've fixed the problem by doing a in-app download of the data over wifi. This caused the database error to fade away.
Related
Using react-native-sqlite-storage for the first time, I was able to create an sqlite db file in the Android emulator, created a table in it and inserted a row successfully. After that on a button click, I also verified by a select that the row was inserted.
Then I used Android Studio's "device file explorer" to locate the test.db file in the data\data<apppkg>\databases folder and used "save as" to copy it to the PC's folder. Now when I open the test.db file in the browser utility, it's empty! There is no table in it.
Suspecting a "flush" problem, I also tried a db.close() in the app. But still I'm not able to get the populated database copy from the Android emulator. What am I doing wrong?
Update: I noticed that on the android emulator, there are 2 more files with the database file--an shm and a wal file. Reading some other threads, it seems they are keeping the journal so it seems the data is not committed yet. Those threads mention closing the cursor. But there is no such mention in the storage library docs. How do I ensure a cursor close from the App other than calling the close() that does not really close it and commit the data.
Finally, for my simple, single table requirement, I ended up calling executeSql on the db object instead of doing it through a transaction. That solved the problem and on close, it's always a single file. Somehow, when you use a transaction, the other files linger around even if you close the db.
We have recently planned to switch from SQLite to Realm in macOS and iOS app due to db file corruption issue with SQLite so we first started with macOS app. All coding changes were smooth and app started working fine.
Background about app and DB usage - app really uses DB very heavily and performs too many read and writes to DB in each minute and saves big xml's to it. In each minute it writes / updates around 10-12 records (at max) with xml and reads 25-30records too. After each read it deletes data along with xml from database and my expectation is once data is deleted it should free up space and reduce file size but looks like it is growing continuously.
To test the new DB changes we kept app running app 3-4 days and DB file size went to 64.42GB and app started being slow. Please refer the attached screen shot.
To further debug, I started app with new DB file and size was 4KB but within 5min it goes to 295KB and never reduced in size even records were continuously added and deleted.
To further clarify, app uses NSThreads to perform various operations and those threads writes and reads data to DB but with proper begin\commit transactions. I also read at 'Large File Size' at https://realm.io/docs/java/latest/#faq and tried to find compactRealm but can't find it in objective c.
Can anybody please advise.
Update - I Give up on Realm
After 15days of efforts, Finally I have stopped usage of Realm and starting to fix/workaround db file corruption issue with SQLite. Realm Huge DB file issue was fixed by making changes to code for threads but then I started getting Too many open files error after running the app for 7-8 hours.
Debugged for whole week and made all possible changes and at some point looks like all was good as xcode were not showing any open files. But again I started getting Too many open files crash and then debugged with instrument and found there were so many open files to realm database, lock, commit and cv files.
I am sure there are no leaks in app and xcode also does not shows those open files in Disk usage as well. I decided to invoke lsof command in code before and after Realm calls and most of the it doesn't increase open file count but sometime n between it increases. In My app it started from 120 files to 550 in around 6 hours. Xcode looks all fine via Disk usage but instrument shows open files.
No good support from Realm team, sent email to them, just got one response. Made many changes to code following their suggestions and doesn't work at all so gave up on it. I think it's good for small apps only.
I am using phonegap for IOS app development. Now, phonegap tries to find the db by default in 'NSCachesDirectory', but whenever IOS runs into memory problems, it tries to delete data from 'NSCachesDirectory', so, the data is not secured. If i am not mistaken, this problem was solved in cordova 2.1.0, where the back-up of data is taken and then restored afterwards. So, just wanted to confirm if i am heading in the right direction or data itself can be stored in 'NSDocumentDirectory' so that data is secured and somehow phonegap looks for db in 'NSDocumentDirectory' and not Caches direcory. Thanks.
Only documents and other data that is user-generated, or that cannot otherwise be recreated by your application, should be stored in the /Documents directory(You can store DB in Documents directory)
Data that can be downloaded again or regenerated should be stored in the /Library/Caches directory.
I'm using sqlite3 in an iOS application and I've encountered a very strange issue multiple
times.
I'm using WAL and all of my writes happen on a managed thread that only allows 1 operation at a time and my reads use a different database handle and everything works fine. The issue I'm seeing is sometimes my read handle gets into this weird state where it won't read committed data. It's like it has an uncommitted read transaction...
I can write successfully to the database and I've exported my results to my computer where I see the newly written results just fine. However, my reads seem to access the database at an older point in time...it's like they're stuck. If I close the application and reopen it, they're fine and they read the newly committed data, but I'm wondering how my application is getting stuck in this state.
ANY help would be appreciated. Thanks in advance.
I ran into the exact same issue but on Debian Lenny (kernel 2.6.33) w/ SQLite3 3.7.7.1.
It turned out that I got some processes hanging on to some stale DB file descriptors after I deleted and recreated a database file.
I fixed it by making sure all processes that used the DB were killed before recreating the DB again.
By getting rid of the old processes, the file handles were deleted as well.
My application is keep crashing first or second time with the error "abc.sqlite is corrupted. SQLite error code:11, 'database disk image is malformed', NSSQLiteErrorDomain=11"
I am unable to track it . anyone Plz help
thanks
(Taken from one of the comments above)
The app was crashing because it was loading on a different thread, app tried to retrieve data before the database was even installed.
This is not entirely true, I ran into same issue today on my iphone, hookup to macbook and use xcode to bring db from iphone to macbook. I used SQLite DB browser, ran PRAGMA integrity_check, it shows an error on one of the pages with code 11. Luckily my table only has 10 records. Wierd thing was when I run "select * from tableA", only 3 records come back. I was able to fix the database accidentally by re-number of some record ID, when saving changes, those missing records shows up mysteriously.. while the corrupted record disappeared.