GunDB is supposed to support peer-to-peer data access, so I'm trying to better understand how this would work. If I were to build a twitter clone, what would the high-level architecture look like if I wanted each user to store their own tweets on their own server?
Answer by the author of GunDB:
Every tweet is cryptographically signed by the user (in fact, with SEA, all data by that user is signed automatically, so you don't have to worry/do anything!) which means that no matter where it is stored (by the user, by a server, by another user), it cannot be tampered with.
By default with GUN (and you could modify this if you wanted, but you'd be adding a lot of unnecessary complexity, which I don't recommend), the tweets are stored by whoever subscribes to that data. That would mean: (A) the user who made the tweet would store it (B) a server peer, which is subscribed to everything, stores it, (C) and friends/viewers/followers/audience who reads the tweet also stores it.
Realistically, most users probably are using a browser to access their app, so you wouldn't want to assume this is reliable - but you can Electron-ify (or something similar) your app so that users can install it on their desktop, in which case, they would be their own server. Then you (or other users) can also deploy the app to AWS/Heroku/DigitalOcean/whatever and ALSO store data as a backup there (like in the case of B, if you add your S3 credentials, it backs data up to S3 - ideally this would be IPFS instead, or similar), etc.
Related
Suppose we have a client application(mobile app or web app etc) working with a our back-end services.T he client wants to store some object(like profile photo or a PDF etc). Now suppose that we want to store this data on the servers of a third party storage provider(like aws S3 etc). What is the best practice to do so?
To achieve this goal I have two ideas:
Our client sends the object directly to the back-end (maybe in a base64 format).
The back-end software takes care of storing this data on the cloud and saving the path of that data on to its own database for later usage, it then informs the client that the operation has succeeded or not.
The client informs the back-end that it wants to store some file. Our software generates a unique Secure URL with limited expiration time and sends it to the client. The client can now save that specific file to the cloud and then inform the back-end. Then the back-end makes sure the file is saved on updates its own database for later usage.
Recently I have encounter such a situation and don't know which method to take. Personally I think the second method is more elegant and idiomatic but it comes at the cost of more complexity and network calls. What method should I choose and If there is other patterns for implementing this kind of task what are they?
I'm building an app and I'm being very cautious about the user privacy on the app. I do not want to collect any data. Not even email IDs. Is there any way to do user authentication without email/phone number?
One idea for now is using biometrics on the phone itself. FaceID/TouchID etc. But that brings up challenges wrt syncing their data in case the user wants to use from a different phone. Any workarounds?
You can safely collect as much data as you like inside the app, so long as it stays there.
The problem is that you want to reliably identify a user account (though not any specific user data) without letting any of that data out. It's become quite common to use hashes (usually SHA256) of email addresses as identifiers, however, if everyone uses the same approach it's like a distributed rainbow table. You can however deploy the usual defence against such attacks by salting your hashes so that they are unique to your service.
If you encrypt the user's data on-device using a key that only they know, and only ever transmit and store encrypted data (i.e. to which you have no access), then they would be able to use the same local identifiers and their key to read the data from a different device. Because data is only ever encrypted and decrypted with the same key you can use symmetric encryption - look at using libsodium to do this.
You might want to consider some kind of 2FA to go with this as otherwise intercepting this identifier could allow unauthorised access.
Depending on the sensitivity of the traffic, you may want to try to hide meta-information such as connection dates, times, and volumes, so you could get the app to generate random data to hide the real data within.
Disclaimer - I Am Not A Cryptographer! I recommend you ask for more qualified responses on https://security.stackexchange.com.
I know API keys need to be stored securely and should not be accessible client side. That being said, I also know that a lot of Wordpress plugins/ custom sites/ and such allow users to copy paste the API key into a text input on the admin panel.
My question is how do you do this securely? Do they hash it and save it to their database?
Say for example I made a react app or wordpress plugin that allowed users to do something with the Google Maps API. I know I can go get their API key and just hard code it in... but if I wanted to let the user update the key on their own - What would be the reccomended steps?
Thanks!
If I understand you correctly, you want your application to process secrets of third party APIs. A bit scary, but if you get the user consent - why not? First thing first - make sure the user understands what he is doing. Point out exactly what you will do with the API keys, what you will not do with the API keys and how will they be protected.
Personally I would never want to store such secrets in my own database, as this would be a single point of failure. When you are hacked, everyone is hacked. Why not put such secrets in - say - local storage so it never touches one of your servers?
Ok, in case it is your server that needs to do something, you could get the API key passed in a request, do something, but never log or persistently store the secret anywhere.
In case it is enough for the Java Script to do the job, local storage is even better solution.
One could think about encrypting the keys in the local storage, but I don't believe this would improve security a lot. I mean this would be security through obscurity and could by bypassed by someone with physical access to the machine/browser/user agent. But if someone would have such access, then probably some of the API keys would be one of the smaller problems.
I have been following your project for quite some time now and am intrigued by the functionality of gunDB where it doesn't require a database in between and keeps security in check.
However, I've got some questions about GunDB which I've been thinking about for quite some time now before I can give Gun a go with a project I'm currently working on. In this project it is necessary that data is safe but should also be shareable once a group has been setup. The project is a mobile app project and ata is mostly stored on the device in a SQLite database.
I have been looking into Gun as it allows for better usage of the app in sense of collaboration. The questions I have, however, are:
User authentication
How is user authentication handled through private keys? So how can a user "register" with, for example, a username and password to login to the service.
For authentication I am currently using Firebase where it is possible to use username/password authentication and I would like to know how Gun approaches this case and how it's implemented.
Data storage
In the documentation and on the website it's stated that data is stored locally with every client and can be stored on a "node" or server using either a local hard drive or the Amazon S3 storage option.
What I am curious about is what data is actually stored at the client? Is this only the data he/she has access to or is this a copy of the whole dataset where the client can only access whatever he/she is granted to have access to?
Maintaining your data
When I've got a production system running with a lot of data, how will I be able to manage my data flows and/or help out my clients with issues they have in the system?
In other words, how can I make sure I can keep up with the system if I want to throw in an update and/or service my clients with data issues.
My main concern is the ability to synchronize their local storage correctly.
Those are all my questions for now.
Thank you very much in advance for providing some clarity on these subjects.
Best regards,
(Answered by Mark Nadal on Github: https://github.com/amark/gun/issues/398#issuecomment-320418285)
#sleever great to hear from you! Thanks for finally jumping into the discussion! :D
User Authentication,
this is currently in alpha. If you haven't already seen these links, check them out:
https://github.com/amark/gun/wiki/auth
http://gun.js.org/explainers/data/security.html
https://github.com/amark/gun/blob/master/sea.js#L23-L43
https://github.com/BrockAtkinson/login-riot-gun
If you have already, would love to either (A) get you to alpha test and help push things forward or (B) hear any specific questions you have about it. This thread is also a more at length discussion about alternative security API ideas: #321 .
Data storage.
Browser peers by default store the data that they subscribe to, not the full data set. You could ask it to store everything, but the browser wouldn't like that. Meanwhile NodeJS peers, especially if hooked up to S3 or others, would store all data and act as a backup.
Does this make data insecure? No, encryption should keep it secure, even if anybody/everybody stores it, the encryption makes it safe. (See [insert link to (1)] for more information).
Maintenance.
You would service your customers by deploying an update to your app code. It would not be ideal for your customers if you could meddle with their data directly. If they wanted you to do that, my recommendation would be that they change their password, give the new password to you, and you login and make any necessary changes. Why? Because if you have admin access to their data, their privacy is fundamentally violated.
I wish to create a functionality that is very similar to facebook or pokerstars if you have used them before. Basically the apps require the user to login and their information can be accessed from both browsers and native and web apps.
How can I go about achieving this? Please advice on what services to research on to accomplish this. To my current understanding. I would be creating the website in html and php and creating a webservice using RESTful protocols and hosting them on amazon aws servers. I can then connect to these servers in the native apps? I am not very clear on how the native apps will interact with the servers
If you know of any particular protocol or a better server hosting service please let me know.
If I'm interpreting your question correctly, you are looking for something like this:
The user starts either your browser app or your native app (perhaps a mobile app)
Since the user does not have an account yet, you present them with the appropriate dialog to create said account.
You then ask the "Identity Service" to create a profile for that user
The identity service returns a token for access
This is something we do in the mobile network industry all the time. Technically, we have TAC/ACS or HSS profile services, but in either case, it's the same thing -- a dedicated service and network process that:
Accepts connections from various clients (web, mobile, desktop...)
Has various primitives along the database CRUD (Create, Read, Update, Delete) model
Answers requests the database
If you want a pre-configured solution, you could just use any networked database with a RESTstyle connector for example (MongoDB maybe?) But you could also just through this in a process that talks to a NoSQL or SQLLite database. The end result is the same.
For commercial solutions, I might like at OpenStack as you can run your code on it and they have identity brokers you might be able to CoOpt.
Personally, I'd just have a datastore running on a cloud somewhere like Amazon's EC2 which answers RESTful requests such as:
Create a user with a given profile set, return a unique token
Delete a user given a token
Update elements of the profile for a given token
I'm leaving out the necessary things like security here, but you get the idea.
This also has the advantage that you can have a single identity service for all of your applications/application services. The specifics for a given application element are just sub-fields in the profile. This gives you, not only a common identity broker for web, desktop and mobile, but a single-sign-on for all your applications. The user signs in once and is authenticated for everything you have. Moving from site to site, now just became seamless.
Lastly, you place your identity management, backup, security token management, etc OUTSIDE of your application. If you later want to add Google Authenticator for second-factor authentication, you don't have to add it to every application you have.
I should also add that you don't want to keep the identity database on the direct internet connection point. Someone could make your life difficult and get ahold it later on. Rather, you want your identity server to have a private link to it. Then do something like this:
When the account is created, don't store passwords, store hashes -- much safer
Have your application (web or otherwise) compute a key as the login
In this case, the user might enter a username and password, but the application or website would convert it into a token. THAT is what you send across.
Next, using that token (and suitable security magic), use THAT as the owner key
Send that key to the datastore and retrieve any needed values
Encrypt them back into a blob with the token
Send the block
THe application decrypts the blob to get at values
Why do we do this?
First, if someone were to try to get at your identity database, there's nothing useful. It contains only opaque tokens for logins, and blobs of encrypted data. Go ahead -- take the database. We don't care.
Second, sniffing the transport gets an attacker nothing -- again, it's all encrypted blobs.
This means later on, when you have five applications using the broker, and someone hacks the network and steals the database, you don't care, because your users never gave out logins and passwords in the first place, and even if they did, the data itself is garbage to anyone without the user key.
Does this help?