CryptoAPI wrapped keys - cryptography

With CryptoAPI, is there a way to decrypt (using CryptDecrypt) a key written into a SYMMETRICWRAPKEYBLOB?
In my c++ program, i wrap a symmetric key k1 with another symmetric key k2 into a symmetric key blob. I have a third key, k3, equal to k2 but with a different handle. My goal is to decrypt the blob with this key. I have already did it using a SIMPLEBLOB and a public key.
Thanks in advance for your attention.
Documentation here

As the documentation you link to says, the format used for SYMMETRICWRAPKEYBLOB follows RFC 3217. It is a weird format in which the data is encrypted, then reversed (last byte becomes first, and so on), and then encrypted again. Both encryptions use CBC. If you want to do it by hand, instead of using CryptImportKey(), then you will have to follow RFC 3217, with two calls to CryptDecrypt(), and your code will also have to do the byte reversal and the rest of the packaging.
Alternatively, import the key blob with CryptImportKey(), then export it again by encrypting with an asymmetric (RSA) key of your own, which you can then decrypt. At some point, Microsoft themselves were documenting that, in order to export a symmetric key "as is", the best way was to call CryptExportKey() with a handcrafted RSA public key with a public exponent equal to 1, i.e. not really a correct RSA key -- with such a public exponent, RSA encryption is mostly a no-operation.

Related

How can I verify with mbedtls, that a cert validates a key?

Mbedtls can validate an x509 cert with its mbedtls_x509_crt_verify(...) function (link).
However, what I have:
A public/private key pair (saved in an mbedtls_pk_context).
A certificate I've got from a different source (thus, there is no guarantee that it does not contain any, possible intelligent modifications).
There is no problem with the validation of the certificate.
However, what if that certificate validates a different key? (What can be the result of a software problem and also a crack attempt.) Of course, such a key/cert pair will be unable for a tls handshake, but I don't think that I would need to build up a tcp connection for that.
This source (although it is for openssl scripting) makes likely, that certificate-key matching validation can happen with simply a modulus match.
There is also an mbedtls_pk_verify(...) function (ref), but it seems to me that it plays mostly with signatures. But I have no signatures, I have a cert (got in a pem format), and my key (I have also that in a pem format). Processing them into internal mbedtls data structures (mbedtls_x509_crt and mbedtls_pk_context) is not a problem, but how could I verify that they match?
As this security.SE answer says, for the validation is it enough, if the pubkey in the certificate and in the private key file are the same. It is because it is the only shared information common in them.
Thus, we need to dig out the public key from the mbedtls_pk_content and from the mbedtls_x509_cert and compare them.
The mbedtls has no general API call for this task, but it can be done by algorithm-specific solutions. The steps are the following. Assume that we have
mbedtls_x509_cert crt;
mbedtls_pk_context pk;
Where crt has the certificate and pk is our public-private key pair.
We get the keypair from both. In the case of elliptic curve ciphers, it is being done by the mbedtls_pk_ec(...) macro. In the case of rsa, mbedtls_rsa_context(...) is needed:
mbedtls_ecp_keypair* crt_pair = mbedtls_pk_ec(crt->pk);
mbedtls_ecp_keypair* pk_pair = mbedtls_pk_ec(*pk);
Note, although crt_pair is now a keypair, only its public part will be non-zero, because the certificate has obviously no private key part. mbedtls_pk_ec(...) looks to be some macro-like thing to me, because it doesn't use a pointer to the structs, instead it uses directly a struct.
Then, we compare the public keys in the keypairs:
mbedtls_mpi *pk_pub_X = &pk_pair->Q.X;
mbedtls_mpi *pk_pub_Y = &pk_pair->Q.Y;
mbedtls_mpi *crt_pub_X = &crt_pair->Q.X;
mbedtls_mpi *crt_pub_Y = &crt_pair->Q.Y;
In the case of other algorithms (RSA), these parts might differ, however we always need to have a set of big numbers (mbedtls_mpi), and compare these big numbers.
Then, we use the mbedtls big number functionality to compare them:
bool does_it_differ = mbedtls_mpi_cmp_mpi(pk_pub_X, crt_pub_X) || mbedtls_mpi_cmp_mpi(pk_pub_Y, crt_pub_Y);
Note: verifying the cert match is not enough to verify the cert validity, it is only required for that. The verification of the certificate can be done with the already much more pleasant and simpler mbedtls_x509_crt_verify(...) function.
I know this is an older question, but perhaps mbedtls_pk_check_pair is what you are looking for. Pass it your private/public key pair and the certificates public key.
/**
* \brief Check if a public-private pair of keys matches.
*
* \param pub Context holding a public key.
* \param prv Context holding a private (and public) key.
*
* \return \c 0 on success (keys were checked and match each other).
* \return #MBEDTLS_ERR_PK_FEATURE_UNAVAILABLE if the keys could not
* be checked - in that case they may or may not match.
* \return #MBEDTLS_ERR_PK_BAD_INPUT_DATA if a context is invalid.
* \return Another non-zero value if the keys do not match.
*/
int mbedtls_pk_check_pair( const mbedtls_pk_context *pub, const mbedtls_pk_context *prv );

What prevents CA private key from being calculated based on CA public key?

In RSA you basically have two primes for decryption and product for encryption. Normally you make decryption key private and and encryption public, however for CA signature verification the roles are reversed - CA encrypts the signature and browser decrypts it, so the decryption key is public. This means that the two primes are public, and once they are known everybody can multiply them together and get their dirty hands on the super-secret CA private key. What am I missing here?
Normally you make decryption key private and and encryption public, however for CA signature verification the roles are reversed - CA encrypts the signature and browser decrypts it, so the decryption key is public.
The signature is done on the server side by using the private key only known to the server. The signature is validated by the client using the public key. This means only the public key is public and the private key stays secret at the server.
This means that your assumption that both primes are public is wrong.
CA encrypts the signature and browser decrypts it, so the decryption key is public
No, the CA signs the message with the private key; and others can verify the message using the public key.
What am I missing here?
The confusion probably comes from the way that many people learn how signing works, specifically because they learn about RSA as "encryption" is m^e % n and "decryption" is m^d % n. Then you learn that "signing" is a proof-of-private-key, done by m^d % n and "verification" is done by doing m^e % n and comparing the expected result to the digest of the message. Conclusion: signing == decryption.
The reason you get taught that is because RSA is a hard algorithm to work out on paper (and even hard to write correctly for the computer) if you are using "sensible" payload sizes (that is, any size big enough to hold even an MD5 hash (128 bits), which would require a minimum key size of 216-bit, resulting in doing ModExp with 5.26e64 < d < 1.06e65)
For RSA encryption (PKCS#1 v1.5 padding) you take your original message bytes and prepend them with
0x00
0x02
(n.Length - m.Length - 3) random non-zero values (minimum 8 of these)
0x00
So encryption is actually (00 02 padding 00 m)^e % n; or more generically pad(m)^e % n (another encryption padding option, OAEP, works very differently). Decryption now reverses that, and becomes depad(m^d % n).
Signing, on the other hand, uses different padding:
Compute T = DER-Encode(SEQUENCE(hashAlgorithmIdentifier, hash(m)))
Construct
0x00
0x01
(n.Length - T.Length - 3) zero-valued padding bytes
0x00
T
Again, the more generic form is just pad(m)^d % n. (RSA signatures have a second padding mode, called PSS, which is quite different)
Now signature verification deviates. The formula is verify(m^e % n). The simplest, most correct, form of verify for PKCS#1 signature padding (RSASSA-PKCS1-v1_5) is:
Run the signing padding formula again.
Verify that all the bytes are the same as what was produced as the output of the public key operation.
PSS verification is very different. PSS padding a) adds randomness that PKCS#1-signature padding doesn't have, and b) has a verify formula that only reveals "correct" or "not correct" without revealing what the expected message hash should be.
So the private key was always (n, d) and the public key was always (n, e). And signing and decrypting aren't really the same thing (but they both involve "the private key operation"). (The private key can also be considered the triplet (p, q, e), or the quintuple (p, q, dp, dq, qInv), but let's keep things simple :))
For more information see RFC 8017, the most recent version of the RSA specification (which includes OAEP and PSS, as well as PKCS#1 encryption and PKCS#1 signature).
Normally you make decryption key private and and encryption public, however for CA signature verification the roles are reversed - CA encrypts the signature and browser decrypts it, so the decryption key is public.
No. The signature is signed with the private key and verified with the public key. There is no role reversal of the keys as far as privacy is concerned. If there was, the digital signature would be worthless, instead of legally binding.
This means that the two primes are public
No it doesn't.
What am I missing here?
Most of it.
CA works as a trusted authority to handle digital certificates. In RSA digital signature, you have the private key to sign and the public key to verify the signature. Your browsers have the public keys of all the major CAs.The browser uses this public key to verify the digital certificate of the web server signed by a trusted CA. So the private key is not public and you can't compromise it. You can do a simple google search to get a clear understanding of CA and digital certificates.

Where Bitcoin using ECDSA and ECDH

1.var hash = crypto.randomBytes(32);
2.var publickey = ecdh.setPrivateKey(hash,'hex').getPublicKey('hex');
then sha256 => ripemd160 => encode... => address
I know how the bitcoin address generated,but seems it just using ECDH to generate bitcoin address,but I saw lots of talk says it using ECDSA,I want to know where bitcoin using ECDSA and how bitcoin using cryptography to verify transaction and which crypto function using for signing transaction.
Thanks.
Bitcoin doesn't use ECDSA for its address generation, save from the fact that you can use the public key of the ECDSA to generate a Bitcoin address. Where the Elliptic curve is really used is in signing the transaction.
First you create a raw transaction with all the correct fields in, including the ScriptPubKey and then convert that to list of bytes. You then take the SHA256 of the transaction bytes and then SHA256 the result of the SHA256. This is the message digest and this is what you need to sign with the ECDSA private key to generate the SigScript. The SigScript however contains more than just the signed digest. First comes a digest length + 1 for the SIGHASH_CODE, then comes the signature itself, followed by sig hash code, then a length for the public key and finally the public key itself. Once you've concatenated all of these, that's your SigScript that needs to be inserted into the transaction. Obviously the entire SigScript is prefixed with a size, just like the SigPubKey. Lastly insert the size and the result and now you have a signed transaction you can broadcast.

Message Encryption/Decryption Type detection is possibele?

If my Encrypted message = "81dc9bdb52d04dc20036dbd8313ed055"
and Normal text of this is= "1234"
then its possible to detect encryption method or encryption key
You can't with certainty.
AES is an encryption algorithm. Encryption is reversible if you have the key. AES is an example of a symmetric key encryption: that you need the same key for encryption and decryption of the message. Key is by definition secret, so in theory you should never find it out.
There is a set of "standard" symmetric encryption algorithms:
Examples of popular symmetric algorithms include Twofish, Serpent, AES (Rijndael), Blowfish, CAST5, RC4, 3DES, Skipjack, Safer+/++ (Bluetooth), and IDEA
Source: http://en.wikipedia.org/wiki/Symmetric-key_algorithm
If you know the key you can try each one of them and see if you get lucky. Same thing would apply to asymmetric keys such as: RSA.
MD5 is a hashing function. Hashing function by default is a one-way. A MD5-string is valid when:
regex = [0-9a-f]{32}
There also a pretty awesome tool called "Hash identifier": https://code.google.com/p/hash-identifier/
Or you can try writing your own: http://www.insidepro.com/hashes.php
For further reading try:
https://security.stackexchange.com/questions/3989/how-to-determine-what-type-of-encoding-encryption-has-been-used
http://en.wikipedia.org/wiki/Public-key_cryptography
http://en.wikipedia.org/wiki/Symmetric-key_algorithm

Why shouldn't we crypt with the private key in an asymmetrical cryptographic system?

In standard use of asymmetrical cryptographic system, encryption is done with public key, decryption with private key.
Inversing the process, "encryption with private key" is called "signing".
Standard tools, despite terminology and lack of direct tools, allows to implement encryption system that would use the private key for encryption.
Could anyone explain clearly why such solution is vulnerable?
User Case:
Consider that Alice wants to send to Bob some stuff in a non-traditional way:
Alice and Bob once met and Alice gave Bob a "public key" generated from a private key she created BUT she warned Bob to keep it secret. AND she kept secret the private key, and didn't ever give to anyone else the public key.
Could Bob be sure that messages he receives from Alice (provided these are encrypted by Alice private key) are only readable by him (provided he really kept his copy of Alice's public key secret)?
And how compares this encryption solidity to the traditional way, which would, in our case, be Bob sending messages (encrypted by the public key of Alice) to Alice?
What the question is about
The fact that asymmetrical keys are named "private" and "public" doesn't help understanding my question. Keys have underlying properties, and it's me broadcasting the "public key" that gives it its "public" property. Please make this distinction clear before answering: I'm not considering the "public" and "private" properties of these keys but the solidity of the "private key" encryption versus "public key" encryption.
I cannot use another terminology even if it is misleading in this special case.
I know that this case is non-traditional, and could lead to several inconsistency, or is not the point of the asymmetrical crypto systems as Bob and Alice here share some sort of a common secret and that's not the point of asymmetrical crypto.
I saw several Stackoverflow answers which suggest that "private key" and "public key" are exchangeable (just read below answers). This is not true for RSA as it is trivial to generate the public key from the secret key and this is guaranteed not to be computationally feasible in the other way round. For non-believers, the process of key generation in openssl with RSA is:
generate a secret key
extract the public key from the secret key.
If there are so big differences between "private key" and "public key", is there a solidity difference between "private key" encryption versus traditional "public key" encryption?
Short answer from long selected answer
Misunderstanding on what exactly is the "private key" wasn't helping me.
There's two different definition of "private key". The "practical private key", and the "theoretical private key".
Theoretical private key from RSA theory shares mathematical symmetricity with public key:
You cannot deduce one from the other
Encryption is equally solid in either way
Practical private key from RSA tools (like openssl) contains additional information for efficiency reason, and often, a part of the public key is even set by convention. These assumptions breaks the symmetricity:
It is trivial to get public key from "pratical private key"
But encryption remains equally solid
For more detail, see the selected answer ! Please comment if misconceptions remains...
Edit note:
Asymmetrical crypto system key pairs are frequently advertised as swappable (even in current stackoverflow answers), I try to bring reflexion around the fact that it could be dangerous misunderstanding as it isn't the case in REAL life tools.
Added the user case, I hope this will clarify my question
Added final 'short answer'
In standard use of asymmetrical cryptographic system, encryption is done with public key, decryption with private key.
It depends on who is doing what. Suppose Alice wants to send a message to Bob that only Bob can decode. Alice encrypts the message using Bob's public key (under the standard definition of 'public key', meaning the one that is known to people other than its owner). Now only someone who knows Bob's private key (presumably, the only person who knows Bob's private key is in fact Bob) can decrypt Alice's message to Bob.
If Alice wants Bob to know that only she could have sent it, she can encrypt the message with her own private key, assuming Bob knows her public key, either before or after encrypting the message with Bob's public key. Let's assume she encrypts the message with her private key, then the result with Bob's public key. To read the message, Bob has to decrypt the message with his (Bob's) private key, and then decrypt the result again with Alice's public key. If what he reads is now sensible text, he knows that someone who knows both Alice's private key (presumably Alice) and his public key (could be anyone at all) sent the message.
In practice, the asymmetric algorithms are expensive to compute, so what you really do is choose a random session key of an appropriate length and an agreed upon standard symmetric encryption algorithm such as AES. Then the main message is encrypted with the (relatively fast) symmetric algorithm and sent as one part of the message. The other part of the message is the encrypted - or doubly encrypted - random session key. Bob can decrypt the session key section of the message to obtain the session key; he then uses that to decrypt the main part of the message.
Note that if you are sending a message to many people, you can use one encryption of the message proper, and then encrypt the session key once for each recipient, using the recipient's public key. Each recipient can only decrypt the session key information using the key that belongs to them, but all can actually decrypt it. If the message is substantial (say 2 MB of PDF), then this is much more economical than separately encrypting the message with each recipients public key.
Inversing the process, "encryption with private key" is called "signing".
No; signing is a separate operation. If you read Schneier's "Practical Cryptography", you'll see that the authors suggest using one public/private key pair for encryption, and a second pair for signature work. For example, a signature encrypts a fixed length hash of the original message using the private key from the signing key. Anybody who knows the public key part of the signing key can then decrypt the signature to obtain the hash of the original message. Presumably, the same recipient can also decrypt the message (using the public key of the signature key pair), and then can check that the hash of the message received matches the hash derived from the signature. Any mismatch indicates a problem and the message should be discarded.
There are many ways to do these things - depending on the security requirements.
But the basic point is that one person knows the private key of an asymmetric key, and potentially many people know the public part of the asymmetric key (and this is perfectly safe). Data can be encrypted by the sender using the recipients public key; it may also be encrypted by the sender using their own private key. The recipient can decrypt the received message using their own private key and, if necessary, using the sender's public key.
The question, even as amended at about 2009-09-05T13:00-07:00, is not completely coherent, IMNSHO.
You should read chapter 13 "RSA" in "Practical Cryptography" (probably after reading some of the earlier chapters too - most notably section 3.3 Public-Key Encryption).
Notation for Encryption and Decryption
Let's define a bit of notation for discussing orthodox public key cryptography. Let's start with basic symmetric encryption:
C = E(K,m) is the encrypted message (cipher text, C) generated by encryption algorithm E using key K on (plain text) message m.
P = D(K,C) is the plain text message (plain text, P) discovered by decryption algorith D using key K on (encrypted) message c.
To be a working system, m = P, so D(K,E(K,m)) = m.
So far, this notation applies to symmetric encryption because the same value K is used in both encryption and decryption. Anyone who knows K (and the algorithm, but Kerckhoff's Principle that 'secrecy is in the keys' means that you assume the attackers know the algorithm - any contrary assumption is cryptographic 'snake oil') can decrypt the message.
With an asymmetric encryption system, Ea and Da are the encryption and decryption methods for algorithm A. The key distinguishing feature of an asymmetric cryptographic cipher is that the key Kencrypt used by Ea is different from the key Kdecrypt used by Da. Further, to be practical, it must be computationally infeasible to deduce Kdecrypt even if you know Kencrypt and vice versa.
With asymmetric encryption, Alice creates a pair of keys, (Salice, Palice). Conventionally, Salice is the secret key and Palice is the public key. Note that Alice knows both keys. All that matters is:
Salice and Palice are different.
Alice does not let anyone else know about one of the keys (Salice); it is crucial that this information is not known to anyone else.
Alice can let other people know about the other key (Palice) without compromising the security of the system.
Similarly, Bob will create a pair of keys, (Sbob, Pbob). Note that:
Bob knows the keys Sbob, Pbob, and Palice.
Alice knows the keys Salice, Palice, and Pbob.
Alice sends a message to Bob
Now, when Alice wants to send a message, Malice-bob, to Bob so that Bob can read it (but no-one else can), she has to encrypt it with Bob's key Pbob. So, she creates a message:
Calice-bob = Ea(Pbob, Malice-bob)
Bob knows (from external evidence) that the message was encrypted with Pbob, so he knows that he must decrypt it with Sbob:
Malice-bob = Da(Sbob, Calice-bob)
However, at this point, all he knows about the message is that it came from someone who knew his Pbob key. He does not know that it came from Alice except via extrinsic evidence.
If Bob and Alice agree that their messages must be encrypted such that they are both confident that the message received came from the other, then both must be confident that no-one other than Alice knows Salice and that no-one other than Bob knows Sbob. They must also be confident that Palice is known to Bob and Bob must be confident that Palice really does belong to Alice, and that Pbob is known to Alice and Alice must be confident that Pbob really does belong to Bob. Establishing these trust relationships is a lot of what PKI (public key infrastructure) is about.
Assuming that these criteria are met, then Alice can send her message to Bob in such a way that Bob is confident that only Alice could have sent it. As outlined previously, the mechanism is a double encryption:
C1alice-bob = Ea(Salice,Malice-bob)
C2alice-bob = Ea(Pbob,C1alice-bob)
Alice sends C2alice-bob to Bob (along with some signature or MAC to confirm that it was not corrupted in transit), and then Bob computes:
D1alice-bob = Da(Sbob,C2alice-bob)
D2alice-bob = Da(Palice,D1alice-bob)
If everything has gone according to plan, D2alice-bob = Malice-bob.
Mechanics of RSA Key Pairs
The RSA encryption algorithm is based on the fact that if you have two publicly known numbers (which are two parts of one public key), the exponent e and the modulus n, then given a message m, it is easy to compute c = me mod n. However, it is computationally infeasible to deduce m given just c (and e and n). If, however, you know another exponent, d, then you can magically calculate r = cd mod n, and r = m if you have computed e, d and n appropriately. It is not feasible to compute d from e and n without knowing some other information.
Under the RSA encryption scheme, you start work with two (large) randomly determined prime numbers, p and q, and their product is n. The RSA algorithm is predicated on the fact that it is extremely difficult to factor n (determine p and q given just n); if anyone ever finds an easy way of factoring large numbers, then the RSA algorithm is instantly broken.
Once you have n, you need to determine exponents e and d such that:
ed = 1 mod t where t = LCM(p-1, q-1), and LCM is the least common multiple.
You can choose one of the two values as a small odd number - Schneier and Ferguson suggest e = 3, for example. You then calculate d using some computations that they cover in about 6 pages of their book. Typically, d will be a rather large number. You can then publish the pair (e, n) as the composite public key, keeping the values (p, q, t, d) secret as the private key. Given e and n, it is not computationally feasible to deduce d without first factoring n. "Practical Cryptography" suggests using two different pairs (e1, d1) and (e2, d2), derived from the same value n, where you use e1 to encrypt messages, and e2 for digital signatures; they even suggest using the values 3 and 5 for these.
OpenSSL and Key Generation
Your description of how the RSA keys are generated by OpenSSL is confused, I believe.
The generation process first has to generate to large random prime numbers, p and q in the notation above. There are stochastic methods for determining whether a given large number is (probably) prime; it takes a little while to compute two such prime numbers. Taken together, these are used to compute first n, and then d (assuming e is established by some convention). The two stages you see in OpenSSL are determining n, and then determining d.
Dissection of User Case
The question says:
Consider that Alice wants to send to Bob some stuff in a non-traditional way:
Alice and Bob once met and Alice gave Bob a "public key" generated from a private key she created BUT she warned Bob to keep it secret. AND she kept secret the private key, and didn't ever give to anyone else the public key.
So far, so good. The 'public key' isn't very public, but there's no harm in that.
Could Bob be sure that messages he receives from Alice (provided these are encrypted by Alice private key) are only readable by him (provided he really kept his copy of Alice's public key secret)?
If the encryption technology is of any use, then yes; only Alice and Bob can read the message that Alice encrypted with her secret key because only Alice and Bob know the public key that goes with her secret key.
And how compares this encryption solidity to the traditional way, which would, in our case, be Bob sending messages (encrypted by the public key of Alice) to Alice?
Confusion: the section started by discussing Alice sending messages to Bob; now you've switched to Bob sending messages to Alice.
When Bob and Alice met, Alice gave Bob her Palice public key. Presumably, Bob also gave Alice his Pbob public key. And both public keys have very limited public circulation - that's good, but not crucial to the security of the system.
Now, when Bob wants to send a message to Alice, he can encrypt it with her Palice public key, and Alice (and only Alice) can decrypt the message using her Salice secret key. Alternatively, Bob could encrypt the message with his Sbob secret key, and Alice could decrypt it with Bob's Pbob public key. Both sets of encryption and decryption would work.
What the question is about
The fact that asymmetrical keys are named "private" and "public" doesn't help understanding my question. Keys have underlying properties, and it's me broadcasting the "public key" that gives it its "public" property. Please make this distinction clear before answering: I'm not considering the "public" and "private" properties of these keys but the solidity of the "private key" encryption versus "public key" encryption.
It is equally reliable to encrypt with the correct private key and decrypt with the correct public key as it is to encrypt with the correct public key and decrypt with the correct private key. The difference is in who can do which operation. If you understand clearly who is doing the encrypting and who is doing the decrypting, and who knows which keys, then the secrecy of the methods become fairly clear.
I cannot use another terminology even if it is misleading in this special case.
Well, the 'public keys' in your case are not all that widely known, but that's all that's unusual about it.
I know that this case is non-traditional, and could lead to several inconsistency, or is not the point of the asymmetrical crypto systems as Bob and Alice here share some sort of a common secret and that's not the point of asymmetrical crypto.
The whole point of asymmetric encryption schemes is that it does not matter whether the attackers (classically called Eve, the eavesdropper) knows the public key. As long as the private keys are kept private by Alice and Bob, the messages can be sent securely. However, you must understand that if Alice sends a message to Bob that is encrypted only by Alice's secret key, then anyone (such as Eve) who knows Alice's public key can read the message. Eve can't create a fake message that purports to come from Alice unless she also knows the secret key - if Eve discovers Alice's secret key, Eve can pretend to be Alice at any time she likes. But she can read it. If Alice sends a message to Bob that is encrypted only by Bob's public key, then only Bob can read the message (using his secret key), but Bob has no way of knowing whether it actually came from Alice or whether Eve sent it pretending to be Alice. That's why you have to work hard to ensure that Bob knows that only Alice could have sent the message, and Alice knows that only Bob can read the message.
Simply because when you encrypt something, you are masking it so that only one person can read it (the person with the private key). You do not possess that person's private key, all you have is their public key.
If you are encrypting it with your private key, anyone can decrypt it with your public key - this is the principle of signing - they can tell that it was encrypted by your private key!
To put it a little more explicitly, 'encryption with a private key' means that to decrypt you need to use the public key. This isn't an issue, except that anyone can then decrypt your [insert item here], since the public key is just that: public. It isn't useful to protect data, this system is used to verify data.
For instance, Alice wants to send a file toBob (yea, yea, shoot me). Alice doesn't care if anyone else can read her file, it's not confidential, but she wants Bob to be sure that what she sent is what he recieved. She can then encrypt her file with her private key, and Bob can decrypt the file on his end with her public key, ensuring that the file hasn't been tampered with. But if someone else is listening in to the transaction, they can also decrypt and read the file. They just can't change it.
For the case you provide, a better way would be exchanging keys when they meet so that there are actually two keypairs. You yourself mentioned that RSA in particular doesn't actually workr if you try to encrypt with the public key because of optimisations made in the algorithm. I wouldn't be entirely surprised if this is a common case with other algorithms. They are designed to be run one way (private/encrypt, public/decrypt) and are a known "expensive" operation, therefore they likely to be heavily optimised in reality.
Other than that, I don't see any security concerns with your plan... As long as the keys are truely kept private. Private/public are just common names based on typical usage. There's nothing forcing you to make a public key fully public. In your case you may like to term them 'encryption key' and 'decryption key', but I wouldn't use each key for both. Infact, I'd recommend you did term them such inside your program, for the reasons given by Jonathan Leffler in his comments:
A 'public key' is something that can be shared by multiple people. That's the definition of 'public key'. Anything else is very confusing
I think that you are missing the point of public/private key encryption (at least as I understand it).
In the situation you have, symmetric encryption would work just as well. The reason to use non symmetric encryption is a matter of scale.
Say you have, not just Bob and Alice, but imaginary people for every letter of the alphabet. These people want to be able to send messages to anyone, ensuring sure that only the recipient can read it. Using a normal, symmetric encryption, this would require a shared key between every person, so if we have the 26 people from the alphabet town, that is 26x25 keys, with every person having to remember and secure 25 secret keys.
Enter symmetric (aka public/private key) encryption. Now every person has a private key, and a public key, with the normal rules. To send a message to Fred, you look up his (and there is only one) public key. Then you send him the message. Only Fred can read this message. In this scheme, you have 26x2 keys, and each person only needs to remember and secure 1 secret key. There also needs to be a source of public keys, but this is easy.
Using asymmetric encryption the way you describe, with a pair of keys for every set of people, would then require 26x25x2 keys.
So again, it is about scalability. The number of keys needed for symmetric schemes is N^2-N, where in asymmetric schemes, it is only 2*N.
I don't know if there are some copyright concerns but I'll quote "Valery Pryamikov"
from this forum.
Signature and Encryption are two different prototypes with different
security requirements that among other require different padding
modes. Use phrase "decrypt with public" key was the biggest obuse of
terminology in history of cryptography that was widespread by Bruce
Schneier's book "Applied Cryptography". The phrase it self were
supposed to be used to describe signature schemes with message
recovery (such as RSA). This phrase was also used to adjust asymmetric
encryption and signature to old protocol verification models such as
BAN. However, by it self this is just a missnomer - public key is
known to everybody and decrypt operation has meaning of providing
privacy to the content - which is impossible if decryption key is
known to everyone.
Even so raw RSA allows interchange of public and private key, but in
reality they can't be interchanged. Private key decryption is
implemented with using CRT (chinese remainder theorem) to provide 4x
better performance of private key operation. For that - you need not
only exponent, but also factorization of modulus and multiplicative
inverses of some product these factors. Public key has only modulus
and exponent and can't be used with such calculation.
You're misusing the terms here.
If the keys are truly private and public, then yes, anything encrypted with the private key can only be decrypted by the public key, but if the key is truly public, anyone can decrypt that.
Let's disregard that.
The problem here is what Bob knows. Does Bob actually know if Alice sent her public key to anyone else? If not, he can not ensure that only he can decrypt the message. There is nothing in the technology that ensures this. Anything encrypted by Alices private key can be decrypted by her public key, and thus by anyone in possession of that key. By the very nature of public keys, that should be anyone.
The only way to ensure that a message for Bob is only decryptable by Bob is for Bob to give Alice his public key, and make Alice encrypt everything she wants to send to Bob by his public key, which will make the data un-decryptable by anyone except Bob. Whether she also encrypts the same data by her private key (ie. signs the data) is besides the point.
Of course, again, Bob, cannot know that Alice did not send the exact same message to anyone else, encrypting it for others public keys.