I grab data from the web and then I save that in coredata. Not all data must be created new. If the data is the same with previously stored data, then we use old data.
To speed up the saving process, I prefetched old data.
Turns out it's still slow. The saving process is still slow.
I suspected that core data makes IO trips again after the prefetch.
How do we know whether it does again? Can we break on "IO" trip or something?
To monitor Core Data/SQLite activity, you can
Goto "Edit Scheme ..." and add "-com.apple.CoreData.SQLDebug 1" to the "Arguments Passed on Launch". Then you will get console output for all SQLite actions.
Profile your app with Instruments and the "Core Data" tool.
Related
I am using saveEventually in Parse iOS SDK instead of SaveInBackground, just in case there is no network available. Most of the time the save happens immediately. However, sometimes (presumably if there's a bad network connection) I'm having multiple items just getting stuck and nothing is being saved even though some time has passed and a network has clearly been available. The only way to push up all stored items and save them is by closing the app completely and then reopening it, and usually this will immediately save all!
I have found this mentioned from Parse:
"Objects saved with saveEventually will be stored locally in an on-disk
cache until they can be saved. They will be sent
immediately if possible. Otherwise, they will be sent the next time a
network connection is available."
That is clearly not the case in my experience.
I am wondering why the SDK is not realizing the network is available and if I can do anything to force it to try more often. I am also wondering how many times SaveInBackground would try to find a network and save, as I believe it will try a few times at least and if so I could possibly switch to trying that method.
Any ideas?
Over many different Objective-C iOS coding projects I have frequently come across the issue of having data be accessible after I initially got it.
For example, currently I am reading from the stackoverflow API. I do this with a session and get a dictionary back (my JSON response).
But outside the scope of the session, the dictionary is unavailable! I can't copy the contents to a different dictionary that I've defined globally, or anything. It's like it disappears outside of the session.
So I am wondering, what's the best way to save this data that I want to use? From what I've been reading it seems like NSUserDefaults or maybe creating a plist file, although admittedly I've been having trouble with both options. If there is a method that is best for this then I can concentrate on that.
Thank you!
It depends on how persistent you want to be.
If you save this dictionary into a global variable, it is stored in the part of device's RAM that is reserved for the running app. When the app stops running (gets killed by OS or removed by the user) or if your device reboots - this memory is lost.
If you save this dictionary onto the device's flash memory drive (and its file system) - it will live past restarts/reboots.
Usually people combine the approaches: when you get the data from the network, you keep it in a global variable, and save it to the file system. After the app restart you try to load the data from the file system. The reason for not using the FS all the time is that it is much slower than RAM access. I guess I'm describing caching.
Note that you can implement manual caching (using plain data or text files, NSUserDefaults, Core Data or other libraries), but also you can utilize a builtin HTTP cache - NSURLCache. If you create a session with NSURLSession.sharedSession it will use the default NSURLCache and respect a caching policy dictated by the server side.
For more control and full offline support I'd recommend to implement caching manually. See this about reading and writing plists and writeToFile:atomically:.
I have a quiz app which uses Parse.com for some data storage, app has a local sqlite database to store quiz related data. Now I want to store all the quiz database on Parse and download and store it into sqlite when the app runs first time. How is it possible and what is the best way to do that.
I think in your case it may be possible to rely largely on the Parse cache. As Alexandre pointed out in his answer, accessing the cache is a matter of setting the cachePolicy of your PFQuery correctly. However, keep in mind that you don't have control over this cache. According to the Parse docs:
Parse takes care of automatically flushing the cache when it takes up
too much space.
You will have to do some testing with your actual data and see if you run into the limitations of the Parse cache. You may also run into limitations with your pdf files. I imagine you would store these on Parse as PFFiles. PFFiles are caches on your local device, but again, you don't have full control over when those caches are deleted and you probably have to test the limits of this system. You can always fall back to re-downloading missing data.
Why would you use sqlite instead of cache/
You can use for example:
PFQuery *query = [PFQuery queryWithClassName:#"TestObject"];
query.cachePolicy = kPFCachePolicyCacheElseNetwork;
I need to take UIImages that are being fed in a video stream, all of this is on the iPad with limited memory, save them to the file system quickly while the stream is still feeding, then process them after a "recording" session. I need to save the UIImages coming in quickly to avoid interrupting the feed which will still be viewing on the iPad. I'm thinking of saving each frame to a separate file then afterward reading these files sequentially and combining them into a .mov file.
The tricks are: how to save the UIImages quickly, maybe raw data, then when processing the movie, append each UIImage file to it to make a seamless movie file? I will need to do some processing of each frame like scaling and transforms before appending.
Any advice would be greatly appreciated.
Depending on how big your images are, you could let the new coredata "use external storage" attribute do this for you.
Here is the explanation what it does copied from another answer of mine:
Since we are on IO5 now, you no longer need to write images to disk neccessarily.
You are now able to set "allow external storage" on an coredata binary attribute. According to apples release notes it means the following:
Small data values like image thumbnails may be efficiently stored in a
database, but large photos or other media are best handled directly by
the file system. You can now specify that the value of a managed
object attribute may be stored as an external record - see
setAllowsExternalBinaryDataStorage: When enabled, Core Data
heuristically decides on a per-value basis if it should save the data
directly in the database or store a URI to a separate file which it
manages for you. You cannot query based on the contents of a binary
data property if you use this option.
There are several advantages using this approach.
First coredate is saving the files at least as fast as you could when writing to the file system. But if there are any small images which apply to the conditions described above, it'll be much faster because they will be saved directly in the coredata sqlite file.
Further with iOS 5 it is very easy possible to work on separate managed contexts and perform changes on a child context in background. If finished successfully you can merge this child context into your main managed object context and do the processing you need.
[child performBlock:^{
[childsave:&parentError]; //do this in background on child context
}];
There is a NSPrivateQueueConcurrentType for creating "child-moc" - see [apple documentation][1]
And at least you can work with coredata objects which enables you to cache, limit and optimize further processing after your download completed
[1]: https://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/CoreData/Articles/cdConcurrency.html#//apple_ref/doc/uid/TP40003385 for more info
Background
I have an SQL CE database, that is constantly updated (every second).
I have a (web) application that allows a user to look at the data in real-time. At some point a user can click "take a snapshot" button, and it will open the snapshot in a different window.
And then on that form, there is "print" and "download" buttons that will either generate a page for printing, or will stream the data as CSV file - but same data snapshot has to be used, i.e. I can't go to the DB to get latest data for that.
Details
SQL CE dabatase is exposed through WCF web service.
Snapshot consists of up to 500 records, 10 columns each. Expiration time on the snapshot of 2 hours is sufficient.
It is a low-traffic application, so I don't expect more than few (5) connections at the same time.
Loosing snapshot is not a big deal, user can simply generate new one.
database is accessed by self-hosted WCF web service using Linq-to-SQL.
Web site is ASP.NET MVC hosted on UltiDev Cassini.
database, and web site are most likely be on the same box, when deployed. The entire app is intranet bound.
Problem
I need to cache the snapshot of the data at the moment user pressed "take a snapshot" button, so that I can use same data to generate print page, or generate a file for download.
Solution 1:
Each time there is a need to generate a snapshot, I will create a table in the database. Since there are no temp tables in SQL CE, I will need to clean it up myself.
Solution 2:
Cache the snapshot in-memory on either DB server, or web server.
Question:
Is there anything wrong with proposed solutions? Any different solution suggestions?
A consideration is the typical usage pattern. Do most snapshots eventually result in either being printed or exported or both?
If such is the case, we might as well "get it in memory" (temporarily) in the form of a non blocking (asynchronous) select statement from the device to the server. In this fashion the data will "be there" or well on its way when user decides to use it.
If on the other hand many snapshot end up not being effectively used, Solution #1 seems quite ok (maybe the table could be named after the account/user, hence guaranteeing "self clean up" based on the number of snapshot a user can maintain at a given time (though it seems to be just one, with even the tolerance of loosing it sometimes).
500 rows by 10 columns isn't really very large at all. For the sake of simplicity in this case, I might just generate the CSV data at the same time I generate the initial snapshot page, and then place the CSV data in a hidden field in the snapshot page. The "Print" and "Download CSV" buttons would then POST the form that contains the CSV data to a Print page that generates the printable version from the posted CSV data, or a page that streams the CSV directly back to the client's browser, respectively. This way, at least, you wouldn't have any clean-up issues to deal with, and you avoid having to cache something on the server (either in the cache proper or in the database) that might well end up never being used at all.
If you cached the CSV data in a hidden field client-side, you could even handle both the printing and the CSV display completely client-side with javascript, although I don't know if that's worth the trouble or not.