I am creating a cocoa base core data application. I would like to protect the sqlite database, prevent to read it out of the application. How?
You could use cipher algorithms to encrypt the database and decrypt when you use it in your app. CommonCrypto or SecurityTransform may be your choice. Take a look at the Cryptographic Services Guide Apple Dev-Docs.
The needed credentials could be stored securely in the OS X keychain.
So the user could per app start/login decrypt the database and on leave or something, encrypt it.
Another way could be to hardcode the credentials (maybe not a good idea, depends on the security standard you want to use by your app) and do the en-/decrypt on the fly per read/write into the database, so that the database itself is not encrypted but the records in it are. That could be more fault tolerant if your app crashes.
So there is no "right" way to do the task, it depends on what you want to archive and how secure the data has to be.
But what ever you do, don't save any credentials in the NSUserDefaults, that is absolutely insecure.
That would be like to have a secured chest and the key for it lays right on the chest.
For the iOS side there is is the Project iMas-encrypted-core-data on github. It might help you on Cocoa, too.
Aim of the project is to:
Provides a Core Data store that encrypts all data that is persisted. Besides the initial setup, the usage is exactly the same as Core Data and can be used in existing projects that use Core Data.
Under the hood they use SQLCipher and wrap the coreData methods into sql. So you get an encrypted storage but can use coreData syntax for accessing. No need to know about SQL syntax.
The project looks rather promising. It is definitely worth a look.
Related
I am looking for advice and best practice in encrypting parts/whole logs in various formats (say JSON or CSV). I have 0 experience in encrypting anything.
(Disclaimer - I am using Nlog so if you have experience in that library, it would be great to get insight but I am also asking for general ideas/practices/processes.)
I want to encrypt and then decrypt parts of my logs, for example stack trace, user name and other minor things, while keeping other parts of logs untouched (they could be encrypted but there is no need). How to do that? Should I encrypt part of my logs, whole logs or whole files?
How common is it to encrypt logs? How should it be done? Using specific parts of a library to do the encryption or simply encrypting a string before passing it to the logger? How do you manage decrypting the logs after? Without encryption I can simply import my logs from JSON or CSV to Excel and do everything, but when parts of them are encrypted it complicates the process.
How do you organize the decryption process? Where/how do you decrypt? Is encrypting/decrypting resource intensive process, which might affect performance of a logging library?
Questions are rather basic but I am a novice programmer and I have not found answers on SO or other sites. Encrypting logs seems like an "odd thing".
Many thanks for all help.
Could someone please explain the key differences between JSONStore and CloudantSync (CDTStore) (and as a bonus PouchDB) on the MobileFirst Platform?
I am having difficulty deciding which one would be better to use.
I see that the documentation here states that JSONStore is better if you need FIPS 140-2 compliance, need to synchronize, or are building a hybrid app. But, it looks like you can do all three of those with CDTStore as well. Also, I've seen PouchDB mentioned in some tutorials. I am trying to understand the key differences between what looks like different methods of doing the same thing. Any insight would be greatly appreciated.
The answer to this question is a bit murky as functionality does overlap a bunch, so sorry in advance. I work on the Cloudant team, so am probably unintentionally biased. I'll try to elaborate on the similarities and differences below (as they stand right now), which will hopefully allow you (and others) to see better which libraries aligns with your app's needs.
To set the stage, the similarities are broadly:
Both store JSON data and expose a NoSQL rather than relational interface.
Both use SQLite as an underlying durable datastore.
However, neither expose SQLite to the client -- it's an implementation detail. Sync in particular has a somewhat complicated data model in SQLite to support synchronisation metadata.
Both are designed to securely store data from remote services locally on a device from remote services.
Both offer querying JSON data on device. Both use SQLite's indexing engine under-the-hood.
Sync offers a more flexible and powerful system based on Cloudant Query. Sync offers full text search via SQLite's FTS engine.
Sync also allows creating and dropping indexes at any point, whereas I believe JSONStore requires defining indexes when creating a collection.
Sync
Both Cloudant Sync and JSONStore were designed around the idea of storing data from remote services locally on the device for offline use, or just to make manipulating the data quicker for users.
We originally designed Cloudant Sync (CDTDatastore on iOS, sync-android on Android) with the purpose of synchronising data with a remote Cloudant or CouchDB server. When used with Cloudant/CouchDB, Cloudant Sync supports automatic, incremental synchronisation as local and remote data changes. Cloudant Sync also has a powerful and flexible model, taken from CouchDB, for resolving changes concurrently made on different servers and devices asynchronously. When used together, the Cloudant Sync + Cloudant/CouchDB are a powerful combination, and uses a reliable and long-in-production sync protocol.
As you found, JSONStore, on the other hand, is more agnostic as to what its remote database is. This, however, comes at the cost of sync being a fairly manual process -- importantly, though, JSONStore's data model provides capabilities to significantly help with the process.
The key piece here is that JSONStore can tell you the documents that have changed locally, so you can incrementally upload only changed data. But unless your remote database supports a similar capability of streaming changed data to the client, you have to either send full data snapshots or include the necessary data in your data model to allow for only changed data to be sent (and implement the server-side piece to allow for that to be queried and calculated). JSONStore doesn't support conflict resolution out of the box.
We provide similar client-accessible hooks for working out locally changed data in Cloudant Sync -- these are used by our own sync engine -- but they are in a rawer and less documented form than JSONStore's, so we generally recommend sticking with the better documented JSONStore. If you want to try out Sync's hooks, the docs for the iOS version are here.
In summary, if you're working with Cloudant/CouchDB, I suggest Cloudant Sync. If you're working with another datastore, JSONStore may be easier to get started with.
Security
Both JSONStore and Cloudant Sync support encrypting data on the client. Both use SQLCipher for encrypting JSON data. Additionally, Cloudant Sync can store "attachments" -- small binary blobs associated with JSON data -- which are also encrypted. JSONStore cannot store binary data.
When used alongside MobileFirst's client-side SDKs, JSONStore can be used in a FIPS 140-2 solution on MF's supported platforms. MF ships a self-contained OpenSSL binary for this.
Cloudant Sync uses SQLCipher and CommonCrypto on iOS. SQLCipher can be bought commercially in a FIPS 140-2 version and used with Cloudant Sync. Certain versions of CommonCrypto are FIPS 140-2 validated (see Apple's docs for more). Cloudant Sync for iOS uses exclusively FIPS 140-2 mandated crypto suites where it chooses them for itself.
On Android, Cloudant Sync again uses SQLCipher, with the same FIPS 140-2 version available for developers to integrate. Android Sync uses javax.crypto to encrypt attachments, however, which is usually not provided in a FIPS 140-2 validated version by vendors. Again, the Sync code uses FIPS 140-2 mandated suites (at the time of writing!).
What this means is that the situation is more complicated for Cloudant Sync and we've not yet been able to fully confirm our status w.r.t. FIPS 140-2. However, the actual security of the on-device encryption used in JSONStore and Cloudant Sync is identical.
The summary here is basically that if you don't need FIPS 140-2, go for whichever library is more suitable for your needs or has a function you require. JSONStore is currently recommended for FIPS 140-2 requirements as it's been fully vetted.
Platform support & Hybrid
JSONStore supports several platforms. Cloudant Sync is exclusively iOS and Android.
JSONStore can be used in hybrid applications targeting its supported platforms. Cloudant Sync cannot, yet, without manual wrapping of the library.
JSONStore's actual implementations are native per platform, so again if you are using iOS or Android the choice is mostly a choice based on other factors.
PouchDB
PouchDB can, similarly to Cloudant Sync, sync with Cloudant and CouchDB. As a JS library, it can be easily used in hybrid applications. Unfortunately I'm not that familiar with it, so can't say much more than that with any authority :-/
A couple of notes:
CDTDatastore is used for native app development on iOS (it is an Objective-C library). There's also an Android version available.
JSONStore is available for iOS, Android, JavaScript, and Cordova environments.
PouchDB is a JavaScript library that can run in multiple environments: Node.js, web, hybrid mobile, desktop (e.g. Electron), and even on some IoT devices that can run JavaScript.
The functionality between the three is more or less the same. The biggest differences will be in the APIs, the ability to encrypt data, and in which environments the libraries will run.
I see CDTStore (iOS) and CloudantSync (Android) referring only to native implementation, not to Hybrid (Cordova), so that's one key differentiation that is valid.
I am hoping to use the Diff-Match-Patch algorithms available from google as apart of the Google-Mobwrite real time collaborative text editor protocol in order to embed a real time collaborative text editor in my program.
Anyways I was wondering what exactly might be the most efficient way of storing "global" copies of each document that users are editing. I would like to have each document stored on a server that is not local to any user and each time a user performs an "operation" ( delete insert paste cut ) that the diff is computed between their copy and the server and its patched etc... if you know the Google mobwrite protocol you probably understand what I am saying.
Should the servers text files be stored as a file that is changed or inside an sql database as a long string or what? Should I be using websockets to communicate with the server? I am honestly kind of an amateur when it comes to this but am generally a fast learner. Does anyone have any tips or resources I could follow perhaps? Thanks lot
This would be a big project to tackle from scratch, so I suggest you use one of the many open source projects in this area. For example, etherPad:
https://code.google.com/p/etherpad/
Mobwrite is using Differential Synchronization technique and its totally different from Operational Transformation technique.
Differential Synchronization suppose to have a communication circle that always starts from the client(the browser), which means you cant use web-sockets to send diffs from the server directly. The browser needs to request the server frequently to get the updates (lets say every 2 seconds), otherwise your shadow-copies will be out of sync.
For storing your shadow-copies when the user is active, you can use whatever you want, but its better to to use in-memory DB (Redis) since you need fast access to do the diffs and patches. And when the user leaves the session you don't need his copy anymore. But, If you need persistence in you app, you should persist only the server-copy not the shadow-copy (shadow-copies are used to find-out the diffs), then you can use MySQL or whatever you like.
But for Operational Transformation technique there are some nice libs out there
NodeJS:
ShareJS (sharejs.org): supports all operations for JSON.
RacerJS: synchronization model built on top of ShareJS
DerbyJS: Complete framework that uses RacerJS as its model.
OpenCoweb (opencoweb.org):
The server is either Java or Python, the client is built with Dojo
I am building a full featured web application. Naturally, you can save when you are in 'offline' mode to the local datastore. I want to be able to sync across devices, so people can work on one machine, save, then get on another machine and load their stuff.
The questions are:
1) Is it a bad idea to store json on the server? Why parse the json on the server into model objects when it is just going to be passed back to the (other) client(s) as json?
2) Im not sure if I would want to try a NoSql technology for this. I am not breaking the json down, for now the only relationships in the db would be from a user account to their entries. Other than the user data, the domain model would be a String, which is the json. Advice welcome.
In theory, in the future I might want to do some processing on the server or set up more complicated relationships. In other words, right now I would just be saving the json, but in the future I might want a more traditional relational system. Would NoSQL approach get in the way of this?
3) Are there any security concerns with this? JS injection for example? In theory, for this use case, the user doesn't get to enter anything, at least right now.
Thank you in advance.
EDIT - Thanx for the answers. I chose the answer I did because it went into the most detail on the advantages and disadvantages of NoSql.
JSON on the SERVER
It's not a bad idea at all to store JSON on the server, especially if you go with a noSQL solution like MongoDB or CouchDB. Both use JSON as their native format(MongoDB actually uses BSON but it's quite similar).
noSQL Approach: Assuming CouchDB as the storage engine
Baked in replication and concurrency handling
Very simple Rest API, talk to the data base with HTTP.
Store data as JSON natively and not in blobs or text fields
Powerful View/Query engine that will allow you to continue to grow the complexity of your documents
Offline Mode. You can talk to CouchDb directly using javascript and have the entire app continue to run on the client if the internet isn't available.
Security
Make sure you're parsing the JSON documents with the browers JSON.parse or a Javascript library that is safe(json2.js).
Conclusion
I think the reason I'd suggest going with noSQL here, CouchDB in particular, is that it's going to handle all of the hard stuff for you. Replication is going to be a snap to setup. You won't have to worry about concurrency, etc.
That said, I don't know what kind of App you're building. I don't know what your relationship is going to be to the clients and how easy it'll be to get them to put CouchDB on their machines.
Links
CouchDB # Apache
CouchOne
CouchDB the definitive guide
MongoDB
Update:
After looking at the app I don't think CouchDB will be a good client side option as you're not going to require folks to install a database engine to play soduku. That said, I still think it'd be a great server side option. If you wanted to sync the server CouchDb instance with the client you could use something like BrowserCouch which is a JavaScript implementation of CouchDB for local-storage.
If most of your processing is going to be done on the client side using JavaScript, I don't see any problem in storing JSON directly on the server.
If you just want to play around with new technologies, you're most welcome to try something different, but for most applications, there isn't a real reason to depart from traditional databases, and SQL makes life simple.
You're safe as long as you use the standard JSON.parse function to parse JSON strings - some browsers (Firefox 3.5 and above, for example) already have a native version, while Crockford's json2.js can replicate this functionality in others.
Just read your post and I have to say I quite like your approach, it heralds the way many web applications will probably work in the future, with both an element of local storage (for disconnected state) and online storage (the master database - to save all customers records in one place and synch to other client devices).
Here are my answers:
1) Storing JSON on server: I'm not sure I would store the objects as JSON, its possible to do so if your application is quite simple, however this will hamper efforts to use the data (running reports and emailing them on a batch job for example). I would prefer to use JSON for TRANSFERRING the information myself and a SQL database for storing it.
2) NoSQL Approach: I think you've answered your own question there. My preferred approach would be to setup a SQL database now (if the extra resource needed is not a problem), that way you'll save yourself a bit of work setting up the data access layer for NoSQL since you will probably have to remove it in the future. SQLite is a good choice if you dont want a fully-featured RDBMS.
If writing a schema is too much hassle and you still want to save JSON on the server, then you can hash up a JSON object management system with a single table and some parsing on the server side to return relevant records. Doing this will be easier and require less permissioning than saving/deleting files.
3) Security: You mentioned there is no user input at the moment:
"for this use case, the user doesn't
get to enter anything"
However at the begining of the question you also mentioned that the user can
"work on one machine, save, then get
on another machine and load their
stuff"
If this is the case then your application will be storing user data, it doesn't matter that you havent provided a nice GUI for them to do so, you will have to worry about security from more than one standpoint and JSON.parse or similar tools only solve half the the problem (client-side).
Basically, you will also have to check the contents of your POST request on the server to determine if the data being sent is valid and realistic. The integrity of the JSON object (or any data you are tying to save) will need to be validated on the server (using php or another similar language) BEFORE saving to your data store, this is because someone can easily bypass your javascript-layer "security" and tamper with the POST request even if you didnt intend them to do so and then your application will be sending the evil input out the client anyway.
If you have the server side of things tidied up then JSON.parse becomes a bit obsolete in terms of preventing JS injection. Still its not bad to have the extra layer, specially if you are relying on remote website APIs to get some of your data.
Hope this is useful to you.
I've got an issue with WCF, streaming, and security that isn't the biggest deal but I wanted to get people's thoughts on how I could get around it.
I need to allow clients to upload files to a server, and I'm allowing this by using the transferMode="StreamedRequest" feature of the BasicHttpBinding. When they upload a file, I'd like to transactionally place this file in the file system and update the database with the metadata for the file (I'm actually using Sql Server 2008's FILESTREAM data type, that natively supports this). I'm using WCF Windows Authentication and delegating the Kerberos credentials to SQL Server for all my database authentication.
The problem is that, as the exception I get helpfully notes, "HTTP request streaming cannot be used in conjunction with HTTP authentication." So, for my upload file service, I can't pass the Windows authentication token along with my message call. Even if I weren't using SQL Server logins, I wouldn't even be able to identify my calling client by their Windows credentials.
I've worked around this temporarily by leaving the upload method unsecured, and having it dump the file to a temporary store and return a locator GUID. The client then makes a second call to a secure, non-streaming service, passing the GUID, which uploads the file from the temporary store to the database using Windows authentication.
Obviously, this isn't ideal. From a performance point of view, I'm doing an extra read/write to the disk. From a scalability point of view, there's (in principle, with a load balancer) no guarantee that I hit the same server with the two subsequent calls, meaning that the temporary file store needs to be on a shared location, meaning not a scalable design.
Can anybody think of a better way to deal with this situation? Like I said, it's not the biggest deal, since a) I really don't need to scale this thing out much, there aren't too many users, and b) it's not like these uploads/downloads are getting called a lot. But still, I'd like to know if I'm missing an obvious solution here.
Thanks,
Daniel