Could someone please explain the key differences between JSONStore and CloudantSync (CDTStore) (and as a bonus PouchDB) on the MobileFirst Platform?
I am having difficulty deciding which one would be better to use.
I see that the documentation here states that JSONStore is better if you need FIPS 140-2 compliance, need to synchronize, or are building a hybrid app. But, it looks like you can do all three of those with CDTStore as well. Also, I've seen PouchDB mentioned in some tutorials. I am trying to understand the key differences between what looks like different methods of doing the same thing. Any insight would be greatly appreciated.
The answer to this question is a bit murky as functionality does overlap a bunch, so sorry in advance. I work on the Cloudant team, so am probably unintentionally biased. I'll try to elaborate on the similarities and differences below (as they stand right now), which will hopefully allow you (and others) to see better which libraries aligns with your app's needs.
To set the stage, the similarities are broadly:
Both store JSON data and expose a NoSQL rather than relational interface.
Both use SQLite as an underlying durable datastore.
However, neither expose SQLite to the client -- it's an implementation detail. Sync in particular has a somewhat complicated data model in SQLite to support synchronisation metadata.
Both are designed to securely store data from remote services locally on a device from remote services.
Both offer querying JSON data on device. Both use SQLite's indexing engine under-the-hood.
Sync offers a more flexible and powerful system based on Cloudant Query. Sync offers full text search via SQLite's FTS engine.
Sync also allows creating and dropping indexes at any point, whereas I believe JSONStore requires defining indexes when creating a collection.
Sync
Both Cloudant Sync and JSONStore were designed around the idea of storing data from remote services locally on the device for offline use, or just to make manipulating the data quicker for users.
We originally designed Cloudant Sync (CDTDatastore on iOS, sync-android on Android) with the purpose of synchronising data with a remote Cloudant or CouchDB server. When used with Cloudant/CouchDB, Cloudant Sync supports automatic, incremental synchronisation as local and remote data changes. Cloudant Sync also has a powerful and flexible model, taken from CouchDB, for resolving changes concurrently made on different servers and devices asynchronously. When used together, the Cloudant Sync + Cloudant/CouchDB are a powerful combination, and uses a reliable and long-in-production sync protocol.
As you found, JSONStore, on the other hand, is more agnostic as to what its remote database is. This, however, comes at the cost of sync being a fairly manual process -- importantly, though, JSONStore's data model provides capabilities to significantly help with the process.
The key piece here is that JSONStore can tell you the documents that have changed locally, so you can incrementally upload only changed data. But unless your remote database supports a similar capability of streaming changed data to the client, you have to either send full data snapshots or include the necessary data in your data model to allow for only changed data to be sent (and implement the server-side piece to allow for that to be queried and calculated). JSONStore doesn't support conflict resolution out of the box.
We provide similar client-accessible hooks for working out locally changed data in Cloudant Sync -- these are used by our own sync engine -- but they are in a rawer and less documented form than JSONStore's, so we generally recommend sticking with the better documented JSONStore. If you want to try out Sync's hooks, the docs for the iOS version are here.
In summary, if you're working with Cloudant/CouchDB, I suggest Cloudant Sync. If you're working with another datastore, JSONStore may be easier to get started with.
Security
Both JSONStore and Cloudant Sync support encrypting data on the client. Both use SQLCipher for encrypting JSON data. Additionally, Cloudant Sync can store "attachments" -- small binary blobs associated with JSON data -- which are also encrypted. JSONStore cannot store binary data.
When used alongside MobileFirst's client-side SDKs, JSONStore can be used in a FIPS 140-2 solution on MF's supported platforms. MF ships a self-contained OpenSSL binary for this.
Cloudant Sync uses SQLCipher and CommonCrypto on iOS. SQLCipher can be bought commercially in a FIPS 140-2 version and used with Cloudant Sync. Certain versions of CommonCrypto are FIPS 140-2 validated (see Apple's docs for more). Cloudant Sync for iOS uses exclusively FIPS 140-2 mandated crypto suites where it chooses them for itself.
On Android, Cloudant Sync again uses SQLCipher, with the same FIPS 140-2 version available for developers to integrate. Android Sync uses javax.crypto to encrypt attachments, however, which is usually not provided in a FIPS 140-2 validated version by vendors. Again, the Sync code uses FIPS 140-2 mandated suites (at the time of writing!).
What this means is that the situation is more complicated for Cloudant Sync and we've not yet been able to fully confirm our status w.r.t. FIPS 140-2. However, the actual security of the on-device encryption used in JSONStore and Cloudant Sync is identical.
The summary here is basically that if you don't need FIPS 140-2, go for whichever library is more suitable for your needs or has a function you require. JSONStore is currently recommended for FIPS 140-2 requirements as it's been fully vetted.
Platform support & Hybrid
JSONStore supports several platforms. Cloudant Sync is exclusively iOS and Android.
JSONStore can be used in hybrid applications targeting its supported platforms. Cloudant Sync cannot, yet, without manual wrapping of the library.
JSONStore's actual implementations are native per platform, so again if you are using iOS or Android the choice is mostly a choice based on other factors.
PouchDB
PouchDB can, similarly to Cloudant Sync, sync with Cloudant and CouchDB. As a JS library, it can be easily used in hybrid applications. Unfortunately I'm not that familiar with it, so can't say much more than that with any authority :-/
A couple of notes:
CDTDatastore is used for native app development on iOS (it is an Objective-C library). There's also an Android version available.
JSONStore is available for iOS, Android, JavaScript, and Cordova environments.
PouchDB is a JavaScript library that can run in multiple environments: Node.js, web, hybrid mobile, desktop (e.g. Electron), and even on some IoT devices that can run JavaScript.
The functionality between the three is more or less the same. The biggest differences will be in the APIs, the ability to encrypt data, and in which environments the libraries will run.
I see CDTStore (iOS) and CloudantSync (Android) referring only to native implementation, not to Hybrid (Cordova), so that's one key differentiation that is valid.
When using Azure Web Sites (WAWS) general opinion seems to be that uploaded content such as photo's or files should be stored in Azure Storage Blobs and not in the WAWS File System.
Clearly using Azure Storage is a great idea if you have a lot of data and need scale and redundancy however for small or simple sites it seems to add another layer of complexity and also means you can't easily use things like ImageResizer without purchasing the Azure compatible licence etc.
So given that products like WordPress from the Azure Gallery uses "/site/wwwroot/wp-content/uploads/" to store all uploaded files on WAWS is there anything wrong with using the WAWS file system for storage or are there other considerations to take into account when using Azure WAWS?
The major drawback to using the WAWS storage is that your data is now intermingled with the application. By saving all of your plugins/images/blobs externally in a database or blob storage, you retain the flexibility to redeploy your application to a new region/datacenter by just pushing your code to the new website and changing connection strings.
If your plugins/images are stored on disk in WAWS, then you need to make sure that you are backing it up appropriately. If anything happens, you need to restore the site along with all of the data that had been uploaded.
Azure Web Sites is using Azure storage as a file storage so essentially the level of complexity you're talking about is abstracted.
Another great benefit that comes with this approach is if you scale your web site to multiple instances all of them will work with exact same file content.
Of course if you want to use pure Azure Storage features like snapshots or sharing specific content to specific users this is not available as is. But for the web site purposes is quite good.
Hope that helps
I wrote a Windows 8 / JavaScript application that uses indexeddb as its storage mechanism based on this guidance by MS (I have large amount of data).
http://msdn.microsoft.com/en-us/library/windows/apps/hh781225.aspx
I would like now to provide a way for users to synchronize across their Win 8 devices.
What options are available to synchronize the indexeddb data across devices?
IndexedDB is designed for client-side storage scenarios. The API is not designed to handle synchronization to a server-side database. You are going to have to write some code to do that. There are a few of ways that you can accomplish the task; from least fidelity least work to most fidelity most work:
Use the built in Windows 8 app data roaming capabilities. The
pros of this approach are that it is built in functionality. The
cons are that there are limits to how much data you can synchronize
across devices. See Roaming Your App Data for a further
discussion.
You can use a back-end service that already exists like Skydrive
or Azure Mobile Services. The pros here are that the
server-side work is easier and the APIs already exist. The con (I
guess) is that there may be less fidelity than the last option
(below).
You can build your own back end that handles the sync. The pros here
are that this option offers you the most fidelity and control of how
the system works. The con is pretty obvious -- this is the most
work.
Which option you choose depends on your particular needs.
I would like to create an app where the user can add and view data. Either adding at a desktop/tablet or phone and reading from either source. I would like the data store to be synced between any of the user's devices.
I'm starting to play with the Trial of Azure, and it looks promising. Probably a solid way to sync data through to cloud between users' devices. Other than syncing between a users devices, I have no need for cloud services currently.
I've seen some apps that do a 'Backup/Restore' model with the user's SkyDrive account. But this seems to be a manual process. I'd like to see it done seamlessly.
I've looked into Sync services, but that would be more like a hub/spoke solution. Again, I don't need a central database.
What are some options? At this point, I would be fine using just Windows 8 patterns/practices.
Because they are separate devices, you will need to have some service layer to do the store/forward for you. With that you have two basic choices, use the end user's own storage (aka SkyDrive) or use your own storage (aka Windows Azure).
SkyDrive is fully supported through the Live SDKs and provides a secure way to allow a user to share store their data, and synchronize it across multiple devices. You get security, and there is no cost for the server side storage on your part. The user owns their storage, not you. The limitation is that you may have issues sharing that same data across other devices or users where SkyDrive (or whatever file sync service you use) is not available.
With a service layer, like Azure, you have a lot more flexibility, but you also will be responsible for maintaining (and paying for) that server side storage / services. Have you looked at "Windows Azure Mobile Services". With your Azure account you get 10 free Azure Mobile Services. You will pay for the SQL data storage on the backend, and that cost will depend on the amount of data you store on the server side. You also need to make sure to architect your application in a way to protect an individual users' data, but it is actually pretty easy to do, well documented, and gives you a lot of options.
Lastly, you may consider what type of data you want to share. SkyDrive is great for "Files". Pics, Songs, Videos, etc. Windows Azure Mobile Services (WAMS) is great for "Data".
Neither model is right or wrong. It just depends on your goals.
Hope that helps you go through the thought process
I was wondering if there are memory limits for metro style apps. I am not talking about the RAM. That I already found out has a limit of 150 mb, right?
I want to know, if there is a restriction of using the local memory (the hard discstorage). I am creating a database to save alot of data. Can i do so until the device runs out of storage? (I am not actually planning to do so. But occupying like 80mb of the memory would be nice)
Thanks in advance.
There is no limit on local data.
Local application data should be used for any information that needs to be preserved between application sessions and is not suitable type or size wise, for roaming application data. Data that is not applicable on other devices should be stored here as well. There are no general size restriction on local data stored. Location is available via the localFolder property. Use the local app data store for data that it does not make sense to roam and for large data sets.
From here. There is a limit on roaming data. Same document has that.