Why shouldn't we crypt with the private key in an asymmetrical cryptographic system? - signing

In standard use of asymmetrical cryptographic system, encryption is done with public key, decryption with private key.
Inversing the process, "encryption with private key" is called "signing".
Standard tools, despite terminology and lack of direct tools, allows to implement encryption system that would use the private key for encryption.
Could anyone explain clearly why such solution is vulnerable?
User Case:
Consider that Alice wants to send to Bob some stuff in a non-traditional way:
Alice and Bob once met and Alice gave Bob a "public key" generated from a private key she created BUT she warned Bob to keep it secret. AND she kept secret the private key, and didn't ever give to anyone else the public key.
Could Bob be sure that messages he receives from Alice (provided these are encrypted by Alice private key) are only readable by him (provided he really kept his copy of Alice's public key secret)?
And how compares this encryption solidity to the traditional way, which would, in our case, be Bob sending messages (encrypted by the public key of Alice) to Alice?
What the question is about
The fact that asymmetrical keys are named "private" and "public" doesn't help understanding my question. Keys have underlying properties, and it's me broadcasting the "public key" that gives it its "public" property. Please make this distinction clear before answering: I'm not considering the "public" and "private" properties of these keys but the solidity of the "private key" encryption versus "public key" encryption.
I cannot use another terminology even if it is misleading in this special case.
I know that this case is non-traditional, and could lead to several inconsistency, or is not the point of the asymmetrical crypto systems as Bob and Alice here share some sort of a common secret and that's not the point of asymmetrical crypto.
I saw several Stackoverflow answers which suggest that "private key" and "public key" are exchangeable (just read below answers). This is not true for RSA as it is trivial to generate the public key from the secret key and this is guaranteed not to be computationally feasible in the other way round. For non-believers, the process of key generation in openssl with RSA is:
generate a secret key
extract the public key from the secret key.
If there are so big differences between "private key" and "public key", is there a solidity difference between "private key" encryption versus traditional "public key" encryption?
Short answer from long selected answer
Misunderstanding on what exactly is the "private key" wasn't helping me.
There's two different definition of "private key". The "practical private key", and the "theoretical private key".
Theoretical private key from RSA theory shares mathematical symmetricity with public key:
You cannot deduce one from the other
Encryption is equally solid in either way
Practical private key from RSA tools (like openssl) contains additional information for efficiency reason, and often, a part of the public key is even set by convention. These assumptions breaks the symmetricity:
It is trivial to get public key from "pratical private key"
But encryption remains equally solid
For more detail, see the selected answer ! Please comment if misconceptions remains...
Edit note:
Asymmetrical crypto system key pairs are frequently advertised as swappable (even in current stackoverflow answers), I try to bring reflexion around the fact that it could be dangerous misunderstanding as it isn't the case in REAL life tools.
Added the user case, I hope this will clarify my question
Added final 'short answer'

In standard use of asymmetrical cryptographic system, encryption is done with public key, decryption with private key.
It depends on who is doing what. Suppose Alice wants to send a message to Bob that only Bob can decode. Alice encrypts the message using Bob's public key (under the standard definition of 'public key', meaning the one that is known to people other than its owner). Now only someone who knows Bob's private key (presumably, the only person who knows Bob's private key is in fact Bob) can decrypt Alice's message to Bob.
If Alice wants Bob to know that only she could have sent it, she can encrypt the message with her own private key, assuming Bob knows her public key, either before or after encrypting the message with Bob's public key. Let's assume she encrypts the message with her private key, then the result with Bob's public key. To read the message, Bob has to decrypt the message with his (Bob's) private key, and then decrypt the result again with Alice's public key. If what he reads is now sensible text, he knows that someone who knows both Alice's private key (presumably Alice) and his public key (could be anyone at all) sent the message.
In practice, the asymmetric algorithms are expensive to compute, so what you really do is choose a random session key of an appropriate length and an agreed upon standard symmetric encryption algorithm such as AES. Then the main message is encrypted with the (relatively fast) symmetric algorithm and sent as one part of the message. The other part of the message is the encrypted - or doubly encrypted - random session key. Bob can decrypt the session key section of the message to obtain the session key; he then uses that to decrypt the main part of the message.
Note that if you are sending a message to many people, you can use one encryption of the message proper, and then encrypt the session key once for each recipient, using the recipient's public key. Each recipient can only decrypt the session key information using the key that belongs to them, but all can actually decrypt it. If the message is substantial (say 2 MB of PDF), then this is much more economical than separately encrypting the message with each recipients public key.
Inversing the process, "encryption with private key" is called "signing".
No; signing is a separate operation. If you read Schneier's "Practical Cryptography", you'll see that the authors suggest using one public/private key pair for encryption, and a second pair for signature work. For example, a signature encrypts a fixed length hash of the original message using the private key from the signing key. Anybody who knows the public key part of the signing key can then decrypt the signature to obtain the hash of the original message. Presumably, the same recipient can also decrypt the message (using the public key of the signature key pair), and then can check that the hash of the message received matches the hash derived from the signature. Any mismatch indicates a problem and the message should be discarded.
There are many ways to do these things - depending on the security requirements.
But the basic point is that one person knows the private key of an asymmetric key, and potentially many people know the public part of the asymmetric key (and this is perfectly safe). Data can be encrypted by the sender using the recipients public key; it may also be encrypted by the sender using their own private key. The recipient can decrypt the received message using their own private key and, if necessary, using the sender's public key.
The question, even as amended at about 2009-09-05T13:00-07:00, is not completely coherent, IMNSHO.
You should read chapter 13 "RSA" in "Practical Cryptography" (probably after reading some of the earlier chapters too - most notably section 3.3 Public-Key Encryption).
Notation for Encryption and Decryption
Let's define a bit of notation for discussing orthodox public key cryptography. Let's start with basic symmetric encryption:
C = E(K,m) is the encrypted message (cipher text, C) generated by encryption algorithm E using key K on (plain text) message m.
P = D(K,C) is the plain text message (plain text, P) discovered by decryption algorith D using key K on (encrypted) message c.
To be a working system, m = P, so D(K,E(K,m)) = m.
So far, this notation applies to symmetric encryption because the same value K is used in both encryption and decryption. Anyone who knows K (and the algorithm, but Kerckhoff's Principle that 'secrecy is in the keys' means that you assume the attackers know the algorithm - any contrary assumption is cryptographic 'snake oil') can decrypt the message.
With an asymmetric encryption system, Ea and Da are the encryption and decryption methods for algorithm A. The key distinguishing feature of an asymmetric cryptographic cipher is that the key Kencrypt used by Ea is different from the key Kdecrypt used by Da. Further, to be practical, it must be computationally infeasible to deduce Kdecrypt even if you know Kencrypt and vice versa.
With asymmetric encryption, Alice creates a pair of keys, (Salice, Palice). Conventionally, Salice is the secret key and Palice is the public key. Note that Alice knows both keys. All that matters is:
Salice and Palice are different.
Alice does not let anyone else know about one of the keys (Salice); it is crucial that this information is not known to anyone else.
Alice can let other people know about the other key (Palice) without compromising the security of the system.
Similarly, Bob will create a pair of keys, (Sbob, Pbob). Note that:
Bob knows the keys Sbob, Pbob, and Palice.
Alice knows the keys Salice, Palice, and Pbob.
Alice sends a message to Bob
Now, when Alice wants to send a message, Malice-bob, to Bob so that Bob can read it (but no-one else can), she has to encrypt it with Bob's key Pbob. So, she creates a message:
Calice-bob = Ea(Pbob, Malice-bob)
Bob knows (from external evidence) that the message was encrypted with Pbob, so he knows that he must decrypt it with Sbob:
Malice-bob = Da(Sbob, Calice-bob)
However, at this point, all he knows about the message is that it came from someone who knew his Pbob key. He does not know that it came from Alice except via extrinsic evidence.
If Bob and Alice agree that their messages must be encrypted such that they are both confident that the message received came from the other, then both must be confident that no-one other than Alice knows Salice and that no-one other than Bob knows Sbob. They must also be confident that Palice is known to Bob and Bob must be confident that Palice really does belong to Alice, and that Pbob is known to Alice and Alice must be confident that Pbob really does belong to Bob. Establishing these trust relationships is a lot of what PKI (public key infrastructure) is about.
Assuming that these criteria are met, then Alice can send her message to Bob in such a way that Bob is confident that only Alice could have sent it. As outlined previously, the mechanism is a double encryption:
C1alice-bob = Ea(Salice,Malice-bob)
C2alice-bob = Ea(Pbob,C1alice-bob)
Alice sends C2alice-bob to Bob (along with some signature or MAC to confirm that it was not corrupted in transit), and then Bob computes:
D1alice-bob = Da(Sbob,C2alice-bob)
D2alice-bob = Da(Palice,D1alice-bob)
If everything has gone according to plan, D2alice-bob = Malice-bob.
Mechanics of RSA Key Pairs
The RSA encryption algorithm is based on the fact that if you have two publicly known numbers (which are two parts of one public key), the exponent e and the modulus n, then given a message m, it is easy to compute c = me mod n. However, it is computationally infeasible to deduce m given just c (and e and n). If, however, you know another exponent, d, then you can magically calculate r = cd mod n, and r = m if you have computed e, d and n appropriately. It is not feasible to compute d from e and n without knowing some other information.
Under the RSA encryption scheme, you start work with two (large) randomly determined prime numbers, p and q, and their product is n. The RSA algorithm is predicated on the fact that it is extremely difficult to factor n (determine p and q given just n); if anyone ever finds an easy way of factoring large numbers, then the RSA algorithm is instantly broken.
Once you have n, you need to determine exponents e and d such that:
ed = 1 mod t where t = LCM(p-1, q-1), and LCM is the least common multiple.
You can choose one of the two values as a small odd number - Schneier and Ferguson suggest e = 3, for example. You then calculate d using some computations that they cover in about 6 pages of their book. Typically, d will be a rather large number. You can then publish the pair (e, n) as the composite public key, keeping the values (p, q, t, d) secret as the private key. Given e and n, it is not computationally feasible to deduce d without first factoring n. "Practical Cryptography" suggests using two different pairs (e1, d1) and (e2, d2), derived from the same value n, where you use e1 to encrypt messages, and e2 for digital signatures; they even suggest using the values 3 and 5 for these.
OpenSSL and Key Generation
Your description of how the RSA keys are generated by OpenSSL is confused, I believe.
The generation process first has to generate to large random prime numbers, p and q in the notation above. There are stochastic methods for determining whether a given large number is (probably) prime; it takes a little while to compute two such prime numbers. Taken together, these are used to compute first n, and then d (assuming e is established by some convention). The two stages you see in OpenSSL are determining n, and then determining d.
Dissection of User Case
The question says:
Consider that Alice wants to send to Bob some stuff in a non-traditional way:
Alice and Bob once met and Alice gave Bob a "public key" generated from a private key she created BUT she warned Bob to keep it secret. AND she kept secret the private key, and didn't ever give to anyone else the public key.
So far, so good. The 'public key' isn't very public, but there's no harm in that.
Could Bob be sure that messages he receives from Alice (provided these are encrypted by Alice private key) are only readable by him (provided he really kept his copy of Alice's public key secret)?
If the encryption technology is of any use, then yes; only Alice and Bob can read the message that Alice encrypted with her secret key because only Alice and Bob know the public key that goes with her secret key.
And how compares this encryption solidity to the traditional way, which would, in our case, be Bob sending messages (encrypted by the public key of Alice) to Alice?
Confusion: the section started by discussing Alice sending messages to Bob; now you've switched to Bob sending messages to Alice.
When Bob and Alice met, Alice gave Bob her Palice public key. Presumably, Bob also gave Alice his Pbob public key. And both public keys have very limited public circulation - that's good, but not crucial to the security of the system.
Now, when Bob wants to send a message to Alice, he can encrypt it with her Palice public key, and Alice (and only Alice) can decrypt the message using her Salice secret key. Alternatively, Bob could encrypt the message with his Sbob secret key, and Alice could decrypt it with Bob's Pbob public key. Both sets of encryption and decryption would work.
What the question is about
The fact that asymmetrical keys are named "private" and "public" doesn't help understanding my question. Keys have underlying properties, and it's me broadcasting the "public key" that gives it its "public" property. Please make this distinction clear before answering: I'm not considering the "public" and "private" properties of these keys but the solidity of the "private key" encryption versus "public key" encryption.
It is equally reliable to encrypt with the correct private key and decrypt with the correct public key as it is to encrypt with the correct public key and decrypt with the correct private key. The difference is in who can do which operation. If you understand clearly who is doing the encrypting and who is doing the decrypting, and who knows which keys, then the secrecy of the methods become fairly clear.
I cannot use another terminology even if it is misleading in this special case.
Well, the 'public keys' in your case are not all that widely known, but that's all that's unusual about it.
I know that this case is non-traditional, and could lead to several inconsistency, or is not the point of the asymmetrical crypto systems as Bob and Alice here share some sort of a common secret and that's not the point of asymmetrical crypto.
The whole point of asymmetric encryption schemes is that it does not matter whether the attackers (classically called Eve, the eavesdropper) knows the public key. As long as the private keys are kept private by Alice and Bob, the messages can be sent securely. However, you must understand that if Alice sends a message to Bob that is encrypted only by Alice's secret key, then anyone (such as Eve) who knows Alice's public key can read the message. Eve can't create a fake message that purports to come from Alice unless she also knows the secret key - if Eve discovers Alice's secret key, Eve can pretend to be Alice at any time she likes. But she can read it. If Alice sends a message to Bob that is encrypted only by Bob's public key, then only Bob can read the message (using his secret key), but Bob has no way of knowing whether it actually came from Alice or whether Eve sent it pretending to be Alice. That's why you have to work hard to ensure that Bob knows that only Alice could have sent the message, and Alice knows that only Bob can read the message.

Simply because when you encrypt something, you are masking it so that only one person can read it (the person with the private key). You do not possess that person's private key, all you have is their public key.
If you are encrypting it with your private key, anyone can decrypt it with your public key - this is the principle of signing - they can tell that it was encrypted by your private key!

To put it a little more explicitly, 'encryption with a private key' means that to decrypt you need to use the public key. This isn't an issue, except that anyone can then decrypt your [insert item here], since the public key is just that: public. It isn't useful to protect data, this system is used to verify data.
For instance, Alice wants to send a file toBob (yea, yea, shoot me). Alice doesn't care if anyone else can read her file, it's not confidential, but she wants Bob to be sure that what she sent is what he recieved. She can then encrypt her file with her private key, and Bob can decrypt the file on his end with her public key, ensuring that the file hasn't been tampered with. But if someone else is listening in to the transaction, they can also decrypt and read the file. They just can't change it.
For the case you provide, a better way would be exchanging keys when they meet so that there are actually two keypairs. You yourself mentioned that RSA in particular doesn't actually workr if you try to encrypt with the public key because of optimisations made in the algorithm. I wouldn't be entirely surprised if this is a common case with other algorithms. They are designed to be run one way (private/encrypt, public/decrypt) and are a known "expensive" operation, therefore they likely to be heavily optimised in reality.
Other than that, I don't see any security concerns with your plan... As long as the keys are truely kept private. Private/public are just common names based on typical usage. There's nothing forcing you to make a public key fully public. In your case you may like to term them 'encryption key' and 'decryption key', but I wouldn't use each key for both. Infact, I'd recommend you did term them such inside your program, for the reasons given by Jonathan Leffler in his comments:
A 'public key' is something that can be shared by multiple people. That's the definition of 'public key'. Anything else is very confusing

I think that you are missing the point of public/private key encryption (at least as I understand it).
In the situation you have, symmetric encryption would work just as well. The reason to use non symmetric encryption is a matter of scale.
Say you have, not just Bob and Alice, but imaginary people for every letter of the alphabet. These people want to be able to send messages to anyone, ensuring sure that only the recipient can read it. Using a normal, symmetric encryption, this would require a shared key between every person, so if we have the 26 people from the alphabet town, that is 26x25 keys, with every person having to remember and secure 25 secret keys.
Enter symmetric (aka public/private key) encryption. Now every person has a private key, and a public key, with the normal rules. To send a message to Fred, you look up his (and there is only one) public key. Then you send him the message. Only Fred can read this message. In this scheme, you have 26x2 keys, and each person only needs to remember and secure 1 secret key. There also needs to be a source of public keys, but this is easy.
Using asymmetric encryption the way you describe, with a pair of keys for every set of people, would then require 26x25x2 keys.
So again, it is about scalability. The number of keys needed for symmetric schemes is N^2-N, where in asymmetric schemes, it is only 2*N.

I don't know if there are some copyright concerns but I'll quote "Valery Pryamikov"
from this forum.
Signature and Encryption are two different prototypes with different
security requirements that among other require different padding
modes. Use phrase "decrypt with public" key was the biggest obuse of
terminology in history of cryptography that was widespread by Bruce
Schneier's book "Applied Cryptography". The phrase it self were
supposed to be used to describe signature schemes with message
recovery (such as RSA). This phrase was also used to adjust asymmetric
encryption and signature to old protocol verification models such as
BAN. However, by it self this is just a missnomer - public key is
known to everybody and decrypt operation has meaning of providing
privacy to the content - which is impossible if decryption key is
known to everyone.
Even so raw RSA allows interchange of public and private key, but in
reality they can't be interchanged. Private key decryption is
implemented with using CRT (chinese remainder theorem) to provide 4x
better performance of private key operation. For that - you need not
only exponent, but also factorization of modulus and multiplicative
inverses of some product these factors. Public key has only modulus
and exponent and can't be used with such calculation.

You're misusing the terms here.
If the keys are truly private and public, then yes, anything encrypted with the private key can only be decrypted by the public key, but if the key is truly public, anyone can decrypt that.
Let's disregard that.
The problem here is what Bob knows. Does Bob actually know if Alice sent her public key to anyone else? If not, he can not ensure that only he can decrypt the message. There is nothing in the technology that ensures this. Anything encrypted by Alices private key can be decrypted by her public key, and thus by anyone in possession of that key. By the very nature of public keys, that should be anyone.
The only way to ensure that a message for Bob is only decryptable by Bob is for Bob to give Alice his public key, and make Alice encrypt everything she wants to send to Bob by his public key, which will make the data un-decryptable by anyone except Bob. Whether she also encrypts the same data by her private key (ie. signs the data) is besides the point.
Of course, again, Bob, cannot know that Alice did not send the exact same message to anyone else, encrypting it for others public keys.

Related

How to ensure someone did not generate the signature using public key

I am following these instructions https://www.onebigfluke.com/2013/11/public-key-crypto-math-explained.html. My question is - how do I know if some malicious agent did not write a malevolent message, then generated a signature using my public key? For example:
Bob has his private key.
Eve sends a message to Alice, together with a "confirmation" calculated using Bob's public key.
Alice receives the message and Bob's public key, calculates confirmation == signature => Message was sent by Bob (?)
How is it different from Bob sending his message and a signature calculated using his private key?
EDIT
privatekey = (95,49) # Bob's private key
publickey = (95,25) #Bob's public key
message = 122 #Eve writes a mallicious message
hash = message**65537 % 2**8 #Eve computes the hash
sig = hash**publickey[1]%publickey[0] #Eve computes the signature using bob's public key
Letter = (message, sig) #Eve sends her message and the signature to Alice
Alice_hash = Letter[0]**65537 % 2**8 #Alice computes the hash on her own
Alice_confirmation = Alice_hash**publickey[1]%publickey[0] #Alice computes the confirmation
if Alice_confirmation == Letter[1]:
print("The message was sent by Bob")
It works. What part am I getting wrong? (The only thing computed with the private key is the signature, but it can be also computed using the public key [unless the tutorial I follow (link above) is wrong])
The question is not silly at all. Asymmetric cryptography is a bit confusing. I always need to refresh my memory on this topic since I don't work with it daily.
At the core of asymmetric cryptography is the notion that you have a public and a private key. You can convert a message into a cipher using one of them, but you can only ever convert the cipher back to the message using the other one.
So if Alice wants to encrypt a message to Bob, she applies Bob's public key, and not even Alice herself would be able to decrypt that message, because she needs to apply Bob's private key, that only Bob has. Applying the public key twice will not work.
It's like a lock that has two keys, one for locking and a second one for unlocking.
If Alice wants to sign a message, it works the other way around. She uses her private key, that only she has. Bob, or anyone with access to her public key will be able to verify Alice's identity, provided he can safely verify that the public key is in fact Alice's. If Eve could make Bob believe that the public key belongs to Alice, when in fact it's Eve's key and she signed the message with her private key, then that would be a successful attack.
As with the encryption example, signing with a public key and checking with the same public key will not work, it has to be one for signing and the other for checking. That's why it is called asymmetric.
The attack you describes fails because Eve tries to sign a message using Bob's public key. If Alice were to verify this message using Bob's public key a second time, the verification would fail.
Code Review
After you gave your code example and a link to your sources, I was able to look a bit further into it.
Before we get started: If everything is done correctly, then applying the public key twice will not get you the same result as first applying the private key and then applying the public key. This is the reason why your suggested attack will fail. Let's get into why the attack seems to work in your code:
Problem 1: You have a bug in your code when you transferred it from the example. This line
hash = message**65537 % 2**8 # incorrect
should use multiplication instead of the power function
hash = message*65537 % 2**8 # correct
This bug was on you, but from here on out nothing is your fault, since the source you linked is faulty itself. So let's go ahead and fix your code bit by bit.
I will switch to the regular case of signing with a private key and checking with a public key to make sure the algorithm works and then we'll run your attack again.
Problem 2: Alice_confirmation is not computed correctly. The idea is that Bob computes the hash, then encrypts the hash to get the signature. Now Alice decrypts the signature and compares it to the hash, which she also computes. This last step was switched around in the example.
So this
Alice_confirmation = Alice_hash**publickey[1] % publickey[0]
if Alice_confirmation == sig:
print("The message was sent by Bob")
should actually be switched around to look like this:
Alice_confirmation = sig**publickey[1] % publickey[0]
if Alice_confirmation == Alice_hash:
print("The message was sent by Bob")
This is a crucial difference and the reason why your attack seemed to work. If I'm not mistaken, the side effect is that a correctly signed message would fail the test (This did not happen in your original code due to problem 1, though).
Problem 3: I can't reconstruct how you got your private and public keys, so I will use the ones provided by the example website. The problem here is that as a rule, the hash must be smaller than n. In your case, the hash can grow to be 255 (due to the % 2**8) which is greater than n = 91. This is even worse on the website since they use % 2**32 in their hash function which creates even greater numbers.
So let's instead take the values provided by the Wikipedia page at https://en.wikipedia.org/wiki/RSA_(cryptosystem). You are encouraged to try and roll your own, just make sure they are large enough for your hash function range.
public key: n = 3233, e = 17
private key: n = 3233, d = 413
Here your hashing function works, since a applying % 2**8 guarantees the result to be smaller than 256 and therefore smaller than n = 3233. In general it is probably a good idea to use a stronger hash, like the one provided in your example link (message * 2654435761 mod 2**32), but then of course you would have to choose your n adequately, so that it exceeds 2**32).
So, applying the fixes and cleaning up your code a little would give us this:
private_key = (3233, 17) # Bob's private key
public_key = (3233, 413) # Bob's public key
bob_message = 122
bob_hash = bob_message * 65537 % 2**8
bob_sig = bob_hash**private_key[1] % private_key[0]
# Alice receives Bob's message and his signature
alice_hash = bob_message * 65537 % 2**8 # same as bob_hash
alice_confirmation_hash = bob_sig**public_key[1] % public_key[0]
if alice_hash == alice_confirmation_hash:
print("The message was sent by Bob")
else:
print("The message was not sent by Bob")
print()
print(f"message: {bob_message}, signature: {bob_sig}")
print(f"hash: {alice_hash}, confirmation: {alice_confirmation_hash}")
If we run this code we get the following output:
The message was sent by Bob
message: 122, signature: 1830
hash: 122, confirmation: 122
In case you're wondering why the message is the same as the hash, this is a result of your message being smaller than 2**8. A message greater than that will provide a hash that is different from the message.
Now let's run the attack you proposed: Eve uses Bob's public key when computing the signature. This results in Alice using the public key a second time when she tries to verify the signature, with the following result:
The message was not sent by Bob
message: 122, signature: 1159
hash: 122, confirmation: 1891
So here the hash and the confirmation hash do not match. Eve's attack has failed.

How can I verify with mbedtls, that a cert validates a key?

Mbedtls can validate an x509 cert with its mbedtls_x509_crt_verify(...) function (link).
However, what I have:
A public/private key pair (saved in an mbedtls_pk_context).
A certificate I've got from a different source (thus, there is no guarantee that it does not contain any, possible intelligent modifications).
There is no problem with the validation of the certificate.
However, what if that certificate validates a different key? (What can be the result of a software problem and also a crack attempt.) Of course, such a key/cert pair will be unable for a tls handshake, but I don't think that I would need to build up a tcp connection for that.
This source (although it is for openssl scripting) makes likely, that certificate-key matching validation can happen with simply a modulus match.
There is also an mbedtls_pk_verify(...) function (ref), but it seems to me that it plays mostly with signatures. But I have no signatures, I have a cert (got in a pem format), and my key (I have also that in a pem format). Processing them into internal mbedtls data structures (mbedtls_x509_crt and mbedtls_pk_context) is not a problem, but how could I verify that they match?
As this security.SE answer says, for the validation is it enough, if the pubkey in the certificate and in the private key file are the same. It is because it is the only shared information common in them.
Thus, we need to dig out the public key from the mbedtls_pk_content and from the mbedtls_x509_cert and compare them.
The mbedtls has no general API call for this task, but it can be done by algorithm-specific solutions. The steps are the following. Assume that we have
mbedtls_x509_cert crt;
mbedtls_pk_context pk;
Where crt has the certificate and pk is our public-private key pair.
We get the keypair from both. In the case of elliptic curve ciphers, it is being done by the mbedtls_pk_ec(...) macro. In the case of rsa, mbedtls_rsa_context(...) is needed:
mbedtls_ecp_keypair* crt_pair = mbedtls_pk_ec(crt->pk);
mbedtls_ecp_keypair* pk_pair = mbedtls_pk_ec(*pk);
Note, although crt_pair is now a keypair, only its public part will be non-zero, because the certificate has obviously no private key part. mbedtls_pk_ec(...) looks to be some macro-like thing to me, because it doesn't use a pointer to the structs, instead it uses directly a struct.
Then, we compare the public keys in the keypairs:
mbedtls_mpi *pk_pub_X = &pk_pair->Q.X;
mbedtls_mpi *pk_pub_Y = &pk_pair->Q.Y;
mbedtls_mpi *crt_pub_X = &crt_pair->Q.X;
mbedtls_mpi *crt_pub_Y = &crt_pair->Q.Y;
In the case of other algorithms (RSA), these parts might differ, however we always need to have a set of big numbers (mbedtls_mpi), and compare these big numbers.
Then, we use the mbedtls big number functionality to compare them:
bool does_it_differ = mbedtls_mpi_cmp_mpi(pk_pub_X, crt_pub_X) || mbedtls_mpi_cmp_mpi(pk_pub_Y, crt_pub_Y);
Note: verifying the cert match is not enough to verify the cert validity, it is only required for that. The verification of the certificate can be done with the already much more pleasant and simpler mbedtls_x509_crt_verify(...) function.
I know this is an older question, but perhaps mbedtls_pk_check_pair is what you are looking for. Pass it your private/public key pair and the certificates public key.
/**
* \brief Check if a public-private pair of keys matches.
*
* \param pub Context holding a public key.
* \param prv Context holding a private (and public) key.
*
* \return \c 0 on success (keys were checked and match each other).
* \return #MBEDTLS_ERR_PK_FEATURE_UNAVAILABLE if the keys could not
* be checked - in that case they may or may not match.
* \return #MBEDTLS_ERR_PK_BAD_INPUT_DATA if a context is invalid.
* \return Another non-zero value if the keys do not match.
*/
int mbedtls_pk_check_pair( const mbedtls_pk_context *pub, const mbedtls_pk_context *prv );

What prevents CA private key from being calculated based on CA public key?

In RSA you basically have two primes for decryption and product for encryption. Normally you make decryption key private and and encryption public, however for CA signature verification the roles are reversed - CA encrypts the signature and browser decrypts it, so the decryption key is public. This means that the two primes are public, and once they are known everybody can multiply them together and get their dirty hands on the super-secret CA private key. What am I missing here?
Normally you make decryption key private and and encryption public, however for CA signature verification the roles are reversed - CA encrypts the signature and browser decrypts it, so the decryption key is public.
The signature is done on the server side by using the private key only known to the server. The signature is validated by the client using the public key. This means only the public key is public and the private key stays secret at the server.
This means that your assumption that both primes are public is wrong.
CA encrypts the signature and browser decrypts it, so the decryption key is public
No, the CA signs the message with the private key; and others can verify the message using the public key.
What am I missing here?
The confusion probably comes from the way that many people learn how signing works, specifically because they learn about RSA as "encryption" is m^e % n and "decryption" is m^d % n. Then you learn that "signing" is a proof-of-private-key, done by m^d % n and "verification" is done by doing m^e % n and comparing the expected result to the digest of the message. Conclusion: signing == decryption.
The reason you get taught that is because RSA is a hard algorithm to work out on paper (and even hard to write correctly for the computer) if you are using "sensible" payload sizes (that is, any size big enough to hold even an MD5 hash (128 bits), which would require a minimum key size of 216-bit, resulting in doing ModExp with 5.26e64 < d < 1.06e65)
For RSA encryption (PKCS#1 v1.5 padding) you take your original message bytes and prepend them with
0x00
0x02
(n.Length - m.Length - 3) random non-zero values (minimum 8 of these)
0x00
So encryption is actually (00 02 padding 00 m)^e % n; or more generically pad(m)^e % n (another encryption padding option, OAEP, works very differently). Decryption now reverses that, and becomes depad(m^d % n).
Signing, on the other hand, uses different padding:
Compute T = DER-Encode(SEQUENCE(hashAlgorithmIdentifier, hash(m)))
Construct
0x00
0x01
(n.Length - T.Length - 3) zero-valued padding bytes
0x00
T
Again, the more generic form is just pad(m)^d % n. (RSA signatures have a second padding mode, called PSS, which is quite different)
Now signature verification deviates. The formula is verify(m^e % n). The simplest, most correct, form of verify for PKCS#1 signature padding (RSASSA-PKCS1-v1_5) is:
Run the signing padding formula again.
Verify that all the bytes are the same as what was produced as the output of the public key operation.
PSS verification is very different. PSS padding a) adds randomness that PKCS#1-signature padding doesn't have, and b) has a verify formula that only reveals "correct" or "not correct" without revealing what the expected message hash should be.
So the private key was always (n, d) and the public key was always (n, e). And signing and decrypting aren't really the same thing (but they both involve "the private key operation"). (The private key can also be considered the triplet (p, q, e), or the quintuple (p, q, dp, dq, qInv), but let's keep things simple :))
For more information see RFC 8017, the most recent version of the RSA specification (which includes OAEP and PSS, as well as PKCS#1 encryption and PKCS#1 signature).
Normally you make decryption key private and and encryption public, however for CA signature verification the roles are reversed - CA encrypts the signature and browser decrypts it, so the decryption key is public.
No. The signature is signed with the private key and verified with the public key. There is no role reversal of the keys as far as privacy is concerned. If there was, the digital signature would be worthless, instead of legally binding.
This means that the two primes are public
No it doesn't.
What am I missing here?
Most of it.
CA works as a trusted authority to handle digital certificates. In RSA digital signature, you have the private key to sign and the public key to verify the signature. Your browsers have the public keys of all the major CAs.The browser uses this public key to verify the digital certificate of the web server signed by a trusted CA. So the private key is not public and you can't compromise it. You can do a simple google search to get a clear understanding of CA and digital certificates.

CryptoAPI wrapped keys

With CryptoAPI, is there a way to decrypt (using CryptDecrypt) a key written into a SYMMETRICWRAPKEYBLOB?
In my c++ program, i wrap a symmetric key k1 with another symmetric key k2 into a symmetric key blob. I have a third key, k3, equal to k2 but with a different handle. My goal is to decrypt the blob with this key. I have already did it using a SIMPLEBLOB and a public key.
Thanks in advance for your attention.
Documentation here
As the documentation you link to says, the format used for SYMMETRICWRAPKEYBLOB follows RFC 3217. It is a weird format in which the data is encrypted, then reversed (last byte becomes first, and so on), and then encrypted again. Both encryptions use CBC. If you want to do it by hand, instead of using CryptImportKey(), then you will have to follow RFC 3217, with two calls to CryptDecrypt(), and your code will also have to do the byte reversal and the rest of the packaging.
Alternatively, import the key blob with CryptImportKey(), then export it again by encrypting with an asymmetric (RSA) key of your own, which you can then decrypt. At some point, Microsoft themselves were documenting that, in order to export a symmetric key "as is", the best way was to call CryptExportKey() with a handcrafted RSA public key with a public exponent equal to 1, i.e. not really a correct RSA key -- with such a public exponent, RSA encryption is mostly a no-operation.

A commutative cipher?

I am looking for a commutative cipher - that is
E(K₁,E(K₂,P)) = E(K₂,E(K₁,P))
but is not associative - that is
E(K,P) ≠ E(P,K)
That rules out XOR, which otherwise would have been ok.
A symmetric cipher would be preferable, but an asymmetric cipher would work too.
The basic protocol I want to implement is:
Alice has a list of tokens (32-bit integers) and she encrypts each token with the same key (K0)
Alice sends the list of encrypted tokens to Bob
Bob randomises the list, encrypts each token with a separate key (K1 - Kn), labels each token and returns the list to Alice.
Alice decrypts each token with K0, leaving her a list of tokens, each encrypted with a separate key (K1 - Kn)
Sometime later, Bob sends Alice a key for a specific label (Kx)
Alice decrypts the token with Kx giving her the plaintext for the token labelled x
Bob may see the plaintext, so he must not be able to derive K0 from it given the information he was previously given.
Can someone suggest a cipher I can use and also point me to an implementation of that cipher?
I have an understanding of cryptographic protocols and applications but I don't really grok the mathematics of most of the ciphers out there. Step-by-step mathematical guides would be ok though.
I plan to implement this in Clojure so any Java libraries are also good. However, any code is good because I understand code.
It sounds like you're trying to implement "Mental Poker" (or if not, you should look up the research into it, since it's analagous to your problem).
The SRA algorithm has the properties you desire. It's a bit hard to find information on, but it is essentially just RSA except that both the e and d exponents are kept secret. Trivially:
(Pe1)e2 == (Pe2)e1
below refers to my hobbiest solution using my own cipher program in C#. program and source are free and available.
Remember the 'locked box puzzle' recounted on an SECURITY NOW podcast?
here's the episode...
Episode #33 | 30 Mar 2006 | 43 min.
Symmetric Block Ciphers
https://www.grc.com/sn/sn-033.txt
Steve says...
...Leo and I answer last week's Puzzler/BrainTeaser which explored the idea
of using two private
one-time pad "keys," like two padlocks, to securely convey a message between
two parties, neither of
whom would have the other's key. Then we continue our ongoing tour of
fundamental crypto technology by
describing the operation of Symmetric Block Ciphers...
Steve and Leo agreed that an eavesdropper seeing ALICE's cipher text before
and after encryption could
XOR both together and derive her secret key.
However, if a complex, commutative cipher which doesn't use simple XORing
to encrypt is used then I
think the key exchange would be secure and the key exchange would work.
for example...
BOB encrypts msg with his key.
ALICE encrypts BOB's encrypted above msg with her key.
ALICE sends above encrypted msg back to BOB.
BOB decrypts ALICE's above msg with his key.
BOB sends above to ALICE.
ALICE decrypts above with her key.
ALICE can now read BOB'S original decrypted cipher text and they didn't need
to exchange keys.
An eavesdropper attack will not work if the algorithm is not a simple
'xor'ing of plain text and key.
this cipher is a commutative , complex algorithm.
starting with notepad text file containing one character, an 'm'.
m is hex 6d 01101101.
 is hex c2 11000010 is 'm' encrypted by bob and then sent to alice.
ø is hex d8 11011000 is alice's encryption of 'Â' which bob decrypts to '£'
and sends to alice.
£ is hex a3 10100011 which alice decrypts to 'm' with her key.
m is alice decrypt result
an eavesdropper sees  alice's msg before her encryption.
the eavesdropper sees ø alice's msg after her encryption.
the eavesdropper xors  and ø.
11000010 'Â'
11011000 'ø'
00011010 the eavesdropper's xor result = 1a in hex.
if an eavesdropper attack worked he would have found 'E' hex 45 01001001
which is first letter of
alice's key.
this seems a simpler key exchange than PGP etc. All that's needed is that
both parties use the same
crypto program and agree on an authenticator.
I confess to being a hobbiest. If anyone wants the WINDOWS C# .NET program
and/or the source code for the cipher they
may have it/them.
below is example with longer, random keys.
PLAIN TEXT
this is a test.
BOB'S KEY
kZtOfS0kKqcRLjTNPh7OjcJKZZFLjmm5OVm02YlrBQN0zI9SxOD1zJjQcpetUbX
BOB'S CIPHER TEXT TO ALICE.
1IÎ.8Ío#"ëìAùJ'
ALICE'S KEY
O1yfuV7MpX3n4wtefUhr6YctRaeCcrrzH7LqLNRUQCMVZuL5Mr0Bw3qMeIT92hg
ALICE'S CIPHER TEXT TO BOB
µRÖ³#ïÓO,fzkÆaå
BOB DECODES ALICE'S ABOVE WHICH = BELOW.
øqqøð<ª>P¸&#<,
AND SENDS ABOVE BACK TO ALICE WHICH ALICE DECODES YIELDING...
this is a test.
I'm doing something similar and I'm using AES in OFB (output feedback) mode. In this mode, an IV (publicly known random value) is encrypted with AES using some key. The output of this is then XORed with your data. The output (before being XORed with the data) is then encrypted again to get another output to XOR with more data. This is not only commutative but reciprocal in the sense that the encryption and decryption algorithms are identical.
http://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Output_Feedback_.28OFB.29
I know the question was asked long ago, but nobody seems to have suggested Massey-Omura, so here it goes. It satisfies the original requirements exactly. The original description (also adopted in Wikipedia) used the multiplicative group of GF(2^m), but any secure group (like elliptic curves) will do.
Here is a page with commutative encryption algorithms :
https://xianmu.github.io/posts/2018-09-19-commutative-encryption.html