I look out many password managers like keeper, 1password, secret-in and I am following secret-in password manager to create my own project and trying to add same features, but got stuck at storing the data of users like his/her secrets, payment secrets in encrypted form. I read encryption model of keeper here but still didn't understand. Where to store a server side encryption key?
I have some data that is symmetrically encrypted with a single key in my database. Rather than hard coding it into my code, I am looking for a safer way to store the encryption key. Where can I safely store it?
The approach here is quite simple.
You only send encrypted data to the server for storage/backup.
The encrypted data received doesn't come with a key.
You need to ensure all encryption and decryption occurs locally on the users device. Thus the user needs to supply the key.
Users aren't good at providing high quality key material, so instead, require the user to provide a password, take that password and pass it through a hash-based key derivation function with parameters that make the function slow (high ops, high mem requirements). An algorithm like pbkdf2 with a strong PRF like HMAC-SHA-2 should be sufficient.
Update:
To answer your specific questions, you need to perform the following steps, you will need to use a cryptographic library that supports key derivation from password and symmetric encryption, like libsodium.
request password from user on first use
run this password through key derivation to derive a key from it: https://libsodium.gitbook.io/doc/key_derivation
execute encryption of user data with key: https://libsodium.gitbook.io/doc/secret-key_cryptography
destroy the key and send data to server for backup
Related
I'm working on a distributed application, and we need a way to authorize clients. Every client has a PGP keypair associated with them, and we require them to enter their private key (which is not stored) through which their pubkey is derived and used to identify clients and modify their fields on the distributed database.
Now, considering user experience, entering a private key on a website whenever you need to do something is troublesome. But we also cannot maintain a central database for username/password based authentication as that creates a single failure point (not what the distributed application is meant to do)
I am hoping there is some way to get an easier to remember human readable descriptor of the private key which can be used to authenticate if something like that exists. Any other ideas are appreciated too.
I'll throw a bunch of ideas to clarify the question:
Are you certain that PGP is suited for your needs?
I feel like you should provide some more details to be sure:
- How are the private and public keys used exactly? I understand the following : user inputs its private key, information is decrypted and modified, public key is deduced from private and used to encrypt the updated information.
- How is this different from using a regular password and being able to authenticate the user and encrypt data?
- You seems to send private keys on the network, and they should remain private.
How you could use a regular password:
User has a password, your application uses a function (e.g sha256, KDF,...) to make it usable by classical encryption algorithms (e.g AES). With the same (not stored) key, you can then encrypt and decrypt datas. You just have to make sure that decryption is valid, by adding a known value at the beginning of the data (e.g a part of the key).
Quick example:
password: "WeakPassword"
key: sha256("WeakPassword"+"MySalt") = 493e3ae...b2eec8ef
Decrypt(data,key)
clearData = "493e3ae{123;456}" //valid, because you can check the beginning
Encrypt(clearData,key)
Advantages: usable passwords, faster symmetric encryption
Use a QR code
If you stick to PGP and have a camera available it is pretty handy.
You could also use hexadecimal, base64, ... but for a 2048 bits RSA key, that still gets you hundreds of characters.
My application connects to many other application and so i need to store user names/passwords of other applications in my database.
I do not want to store them as plain text, but my application will need to send a plain text password for authentication to other.
Please suggest the best way to securely store the passwords.
Thanks.
Use a reversible encryption (AES, probably) on the sensitive data fields in the database, and have your app decrypt it every time it needs the value. You'll need to have the encryption key accessible to your application somehow, but that should probably be in a deployment-specific config file.
This will protect you if someone gets a dump of your database but not a dump of the filesystem where your app resides.
It's rarely necessary, if ever, to use a reversible encryption strategy to store passwords. The reason for storing a password is so the user can prove their identity to you (for some value of "prove") at some point in the future, and that can generally be done with a one-way hash like MD5 or SHA1.
Using a one-way hash to store your passwords does protect your users against an intruder getting access to your system, since there is no key that will allow them to obtain the plaintext from the encrypted passwords.
so basically I am trying to log user in with a cookie and do not query DB to improve performance
here is a brief idea:
transmit everything via SSL
set a Global secret key A and secret key B
generate a random verification string on registration and password change
encrypt the verification string with A, store it in cookie
encrypt the verification string with B, store it in cookie
when user tries to login, I decrypt each string with A and B, compare if they match
I am wondering if it is a good idea
if it is:
how can I actually do the encryption in Java, using bouncycastle ASE-256, Digest or whatever?
how much does this encryption/decryption process affect the performance, when compared with authentication by storing a session variable in a super fast DB like Redis
if it is not:
what should I do..
You can simply encrypt a known value together with the authentication data, when you decrypt verify that the data is present in the authentication token (the cookie). No need to use two keys.
The speed difference with a database depends on the database configuration as well as the cryptography that is performed. I would rather opt for a proven scheme first and only invent your ownif performance leaves you no other choice.
Schemes as better verified on http://security.stackexchange.com.
I came across many APIs that give the user both an API key and a secret. But my question is: what is the difference between both?
In my eyes, one key can be enough. Say I have a key and only I and the server know it. I create a HMAC hash with this key and do an API call. On the server, we create the HMAC hash again and compare it with the sent hash. If it's the same, the call is authenticated.
So why use two keys?
Edit: or is that API key used to lookup the API secret?
You need two separate keys, one that tells them who you are, and the other one that proves you are who you say you are.
The "key" is your user ID, and the "secret" is your password. They just use the "key" and "secret" terms because that's how they've implemented it.
Simple answer, if I understood it correctly...
If you use your API key for encryption, how will the service know who is contacting them? How will they decrypt that message?
You use API key to state who you are, this is what you are sending in plain text.
The SECRET key you do not send to anyone. You simply use it for encryption. Then you send the encrypted message. You do not send the key that was used for encryption, that would defeat the purpose.
One thing that I did not see mentioned here, although it is an extension of Marcus Adams's answer, is that you should not be using a single piece of information to both identify and authenticate a user if there is a possibility of timing attacks, which can use the differences in response times to guess how far a string comparison got.
If you are using a system which uses a "key" to look up the user or credential, that piece of information could be incrementally guessed over time by sending thousands of requests and examining the time that it takes for your database to find (or not find) a record. This is especially true if the "key" is stored in plaintext instead of a one-way hash of the key. You would want to store users's keys in a plaintext or symmetrically-encrypted for if you need to be able to display the key to the user again.
By having a second piece of information, or "secret", you can first look up the user or credential using the "key", which could be vulnerable to a timing attack, then use a timing-safe compare function to check the value of the "secret".
Here is Python's implementation of that function:
https://github.com/python/cpython/blob/cd8295ff758891f21084a6a5ad3403d35dda38f7/Modules/_operator.c#L727
And it is exposed in the hmac lib (and probably others):
https://docs.python.org/3/library/hmac.html#hmac.compare_digest
One thing to note here is that I don't think that this kind of attack will work on values that are hashed or encrypted before lookup, because the values that are being compared change randomly each time a character in the input string changes. I found a good explanation of this here.
Solutions for storing API keys would then be:
Use a separate key and secret, use the key to look up the record, and use a timing-safe compare to check the secret. This allows you to show the user the key and secret to a user again.
Use a separate key and secret, use symmetrical, deterministic encryption on the secret, and do a normal comparison of encrypted secrets. This allows you to show the user the key and secret again, and could save you from having to implement a timing-safe comparison.
Use a separate key and secret, display the secret, hash and store it, then do a normal comparison of the hashed secret. This removes the necessity to use two-way encryption, and has the added benefit of keeping your secret secure if the system is compromised. It has the downside that you cannot show the secret to the user again.
Use a single key, show it to the user once, hash it, then do a normal lookup of the hashed or encrypted key. This uses a single key, but it is not able to be shown to the user again. Has the benefit of keeping keys secure if the system is compromised.
Use a single key, show it to the user once, encrypt it, and do a normal lookup of the encrypted secret. Can be shown to the user again, but at the cost of having keys vulnerable if they system is compromised.
Of these, I think that 3 is the best balance of security and convenience. I have seen this implemented on many websites when getting keys issued.
Also, I invite any actual security experts to critique this answer. I just wanted to get this out there as another discussion point.
There are answers explaining what the secret and (public) key is. It's a public-private key pair that they give confusing names to. But nobody says why the APIs require both, and many APIs only give you one secret! I've also never seen any API's docs explain why they have two keys, so the best I can do is speculate...
It's best to put only your public key in your request and sign the request locally with your private key; sending anything more shouldn't be needed. But some get away with just having the secret in the request. Ok, any good API will use some transport security like TLS (usually over HTTPS). But you're still exposing your private key to the server that way, increasing the risk of them somehow mishandling it (see: GitHub and Twitter's password logging bug recently discovered). And HTTPS is theoretically just as secure, but there are always implementation flaws out there.
But many – actually most it seems – APIs have you send both keys in requests since that's easier than making people do their own signatures; can't have pure cURL examples otherwise! In that case, it's pointless to have them separate. I guess the separate keys are just for in case they change the API later to take advantage of them. Or some have a client library that might do it the more secure way.
Using hashing algorithms like md5 we suffer from a limited entropy which means that very long passwords might result in hashes that can be regenerated by a shorter password.
I was thus wondering if it would be a good idea to store a password encrypted with the public key of public/private key encryption on the server. As decrypting is not needed, one could just throw away the secret key to avoid losing passwords when the server is compromised.
However, this method does not seem to be used widely. So are there drawbacks? If yes, then which?
There are several drawbacks. Among them:
You now have a token which must be protected. If someone gets your secret key, they have every password that was encrypted with that key. Asymmetric encryption is less of an issue if you "lose" the private key, but you better pray all copies of it are gone. Hashes can't be decrypted, period.
An encrypted password can be pretty much any length, and thus would require a pretty large field in the database (or length limits for the plaintext) in order to be stored. Hashes have a known length.
If you can decrypt the password, you know it. If ever there was a problem with someone using that password to break into something else, everyone who knew that password belonged to that user is a suspect. That now means you. Even if you use one-way encryption as a hash, you'd better be able to prove you can't decrypt it -- and then, if you don't want to decrypt it, why encrypt?
Generally, you'd only choose encryption over hashing when you have to know the password for something else -- like, say, when you're using it to log in to another system on behalf of the user. And ideally, you'd exhaust all other possibilities first.
Well there is always the drawback that computing a cryptographic hash is much less costly than encrypting a tiny password with a public-key encryption algorithm.
Secondly, you still suffer from limited entropy with public-key encryption, your bit string will still be limited. If you need more bits, use a hash with a bigger internal state (SHA-512, Whirlpool, etc...)
Third, you would need to store the public key along with the password, which results in a pretty hefty storage cost (considering how big public keys are), and if you're thinking of using the same public key for all passwords, don't - if that key is compromised it's over.
Also another consideration: without a password size limit there's a chance even padded block encryption will result in different storage sizes for different passwords, which can make database indexing a hell of a lot more difficult (probably not a huge concern but to keep in mind).
And finally, the biggie - hashes are meant to unequivocally destroy all structure in the input data, which is exactly what you want when storing passwords for verification. Encryption algorithms don't do that - they transform the data to make it unintelligible without the proper key, which sort of goes against what you're trying to use it for.
So, no, this method should not be used because it is self-defeating. Use a hash with a bigger internal state.