We are storing Public Key, Private Key and CA Ceritificate on the client. There is no way we can pass it to the server for signing. We are creating a signature on the client and want to send it to the server for signature embedding. The purpose is to keep the client as light as possible also the same server code we can use for all clients. So is this possible?
PDFBox supports external signature container creation. For usage details cf. the pdfbox-example classes CreateSignatureBase, CreateSignature, CreateVisibleSignature, and CreateVisibleSignature. In particular check the comments of the isExternalSigning and isLateExternalSigning property definitions and usages.
Thus, the server can
can prepare the PDF for signing,
extract the bytes-to-sign, and
forward them or their hash to the client.
Then the client can
sign the given data and
push the resulting signature container to the server.
Eventually the server can
inject this signature container into the previously prepared PDF.
Related
I'm designing my first RESTful API and am trying to figure out how I'm going to authenticate API calls. I've worked with the Gengo API (dev docs) in the past and had great luck with it, so admittedly, am basing a lot of my auth design on their algorithm described in that link.
To sum their process up, to create a valid/authenticated API call:
Register for an account with them and generate a public/private key set. Then for each API call:
Obtain the UNIX epoch timestamp that the call is being made at.
Calculate the SHA-1 hash of your timestamp "against" your private key.
Make sure that your public key, private key and the calculated hash (above) is present as 3 separate HTTP parameters with every single API call.
At first this was a little confusing to me, but I was able to get authentication working pretty quickly with their API. But I never fully understood why I had to generate this SHA-1 hash, and I had no clue what they were doing on the server-side to actually authenticate my API calls.
Now that I'm writing my own authenticated API, I need to understand these things. So I ask:
What purpose does the timestamp and its derived SHA-1 hash serve? Why is it less secure to just require users send me their public/private keys with each API call?
Is this pubkey + privkey + hashed_timestamp method that Gengo is using a standardized practice for API auth? If so, does it have a name/algorithm? Are there other, equally-secure competitors to it?
I'm confused by the whole HMAC/SHA-1 stuff (see the link above for concrete example). I always thought SHA-1 was a one-way function that turned a string into a unqiue, encoded strinig similar to what MD5 offers. But in that example (see link), it looks like it's passing SHA-1 and the string to some HMAC algorithm. What purpose does this HMAC serve and why does it require 3 arguments (SHA-1, the timestamp and the private key)?
Finally, what do I do with the 3 parameters (pub key, priv key, hashed timestamp) on the server-side to perform authentication? If I was designing a system that only used the pub/priv keys, then I would treat them like a username/password combo and would check the database to see if that combo existed or not. But the hashed timestamp is really throwing me off here.
What purpose does the timestamp and its derived SHA-1 hash serve? Why is it less secure to just require users send me their public/private keys with each API call?
To clear any misunderstanding you seem to have up front, the user should never send the private key over the network. The private key is to stay private. It is a secret shared between you and the user. Reread the Gengo link, you'll see that it is only used as a parameter to the HMAC function. It is up to the user to find a way to secure it, but your API does not need it to verify calls.
The timestamp serves two purposes. First it is a piece of data for which you will get both the plaintext and the HMAC. You will be recomputing the HMAC on your side with the private key of the user. If the HMAC checks, it means that not only the timestamp was not tampered with, but also that only someone knowing the private key could have sent it. It provides integrity and authenticity for that piece of data.
If it was a simple SHA1, a attacker could have intercepted the message, changed the timestamp, and recomputed the hash. By using a keyed hash, you ensure that the sender is who you think he is.
The second purpose for the timestamp is to prevent replay attacks. Even if using a keyed hash, the attacker could have captured an old request and send it again, possibly triggering unwanted actions. If your users hash the time and you test it and reject requests that are unreasonably old, you can prevent such replay attacks.
Is this pubkey + privkey + hashed_timestamp method that Gengo is using a standardized practice for API auth? If so, does it have a name/algorithm? Are there other, equally-secure competitors to it?
Again the privkey is not sent through the pipe. Using HMAC for API authentication is quite common. It is used for Amazon Web Services for example. When used in the Gengo way, the fact that there is seemingly a public/private key pair can be confusing, it is really still symmetric cryptography, and the private key is used as a shared secret.
However I think it is better to include more than just the timestamp in the data that is HMAC'ed. Otherwise an attacker could tamper with other parts of the request. The headers, the HTTP verb, and a hash of the content of the request should be included as well.
Another scheme is to use the private key on client side to sign (encrypt with the private key) a piece of data, so the server only needs to verify it with the public key of the client and needs not know the private key of the client. Embedding a time information is still needed to prevent replays. I do not know much about this scheme, it might be hard to reliably link clients with a given public key in the first place.
What purpose does this
HMAC serve and why does it require 3 arguments (SHA-1, the timestamp
and the private key)?
An HMAC is a keyed hash. Consider the simplest form of message authentication: hash(key + message). It was found that this was not secure (see length extension attack) and a nested structure fixes the vulnerability.
HMAC is a generic name of that structure: hash(k1 + hash(k2 + message)), where k1 and k2 are derived from the actual secret key. So when we do an HMAC we need to pass the name of the actual hash algorithm that will be used (here SHA-1), the message (here, the timestamp), and the secret key.
Finally, what do I do with the 3 parameters (pub key, priv key, hashed
timestamp) on the server-side to perform authentication? If I was
designing a system that only used the pub/priv keys, then I would
treat them like a username/password combo and would check the database
to see if that combo existed or not. But the hashed timestamp is
really throwing me off here.
Hopefully clearer by now. You use the public key as an identifier to retrieve the private key. You take the ts header and recompute the HMAC of it with the private key. If it matches with the hmac header sent, the request is authentic. You check the actual timestamp to see if it's not an old request replayed by some attacker. If all checks, the call can go through. I think it's better to embed all the important information in the HMAC, not just a timestamp though.
You need either public key cryptography or an HMAC, not both.
Let's come back to the timestamp later, and you're confusing authentication with integrity, which we'll also come back to later.
Authentication: in your case this is where the client proves knowledge of some secret to the server. Two common ways to do this are via public key cryptography and using an HMAC.
PKC: before using the service a public/private key pair is generated. The client has the private key; the server has the public key. Important: the private key never leaves the client. In particular, the server does not have access to the private key. To authenticate, the client encrypts some random value N (called a nonce), and sends N and its encrypted form to the server. The server uses the public key to decrypt the encrypted nonce and confirms that it equals the supplied nonce. This proves to the server that the client has the private key.
HMAC: client and server agree a shared secret K beforehand. To authenticate, the client creates a nonce N, computes HMAC(K, N), and sends N and HMAC(K, N) to the server. The server also computes HMAC(K, N) since it knows the shared secret and has received N from the client. If the computed and received HMAC(K, N) values are the same then the server knows that the client has the shared secret K.
The HMAC approach has one significant weakness compared with PKC: if the server is compromised then the attacked gains knowledge of K and can then use that to masquerade as the client.
If using PKC, ideally generate the keypair on the client and send the public key to the server. That way the server never has the private key.
However, unless the communication channel is confidential (e.g. uses SSL/TLS), both approaches have a problem: replay attacks. A passive observer can record the N+encrypted form, or N+HMAC(K,N) and replay them to the server. The server will then think that the observer is a valid client.
Two standard defences are:
Use a time-based nonce.
The server remembers previously-seen nonces, and rejects new requests that use a previously-seen nonce.
That's where the timestamp comes in, and is discussed in more detail here: Should I sign (with a public/private key) a form based authentication (no SSL)?
Integrity: we've proved to the server that we're a valid client, but we haven't provided any protection of the request itself. An attacker could modify our request in flight: we'd authenticate correctly but then execute the attacker's request rather than the client's original request.
To resolve this we need to protect the integrity of the request. We can do this with the same mechanism as above. Rather than just using a nonce (N) or nonce+timestamp, include the entire request in the data been encrypted or hashed. An important consideration here is that encrypted and hashing operate on bytes, not REST requests. You therefore need a reliable way to convert the REST request (i.e. HTTP method, URL, request parameters) into bytes. This is often called "canonicalisation": the client and server must both canonicalise the request in exactly the same way, so that they are encrypting/hashing the same bytes given the request.
This whole process is standardised in things like OAuth, e.g. https://dev.twitter.com/docs/auth/authorizing-request
To answer your specific questions:
The timestamp defends against replay attacks: passive observers can't reply a client's session. The SHA-1 hash is used as a component of the HMAC.
Yes, to a point. But I'd use a fully-fledged implementation of it rather than rolling your own, such as something OAuth-based.
An HMAC is a keyed hash: it's like a standard cryptographic hash (such as SHA-1, except that you also include a shared secret key in the hash. Simply concatenating the key to the data being hashed has cryptographic weaknesses that the HMAC construct avoids. (https://en.wikipedia.org/wiki/HMAC.)
If you're using PKC then you look up the client's public key on the server (based on some client ID, which ist not the client's private key), use that to decrypt the encrypted request, and verify that that request matches the received request. If you're using HMAC then you look up the client's shared secret, canonicalise the request, compute HMAC(K, R) and verify that it matches the received HMAC(K, R). In both cases you must also verify timestamps/nonces to protected against replays.
BUT: rule #1 of crypto: don't roll your own. Use an established mechanism, such as OAuth. You probably also want to use SSL/TLS, which would then also let you use client certificates as a third authentication option. If you used those then you could also rely on SSL/TLS to give you integrity and replay protection. However, implementing SSL/TLS certificate validation correctly seems to fox many developers...
I'm developing an API system which requires this type of authentication. The user will send an HTTP request with its id, the data, and a hash (the data hashed with the private key) and then in the server I check if the hash matches with the same procees here.
Is it like the implementation must be?
And I'm going to use as public key the id of the client but what is the best method to generate the private key, sha256 of the id, of a randon number?
This is not Private/Public-key cryptography. In such a system, the client would encrypt the whole data and you would then decrypt it using your private key. You would never give your private key to the client.
What you are doing is request signature. You give a "secret" (what you are calling "private key"), then they use this to generate a hash. You can then check the hash on your server since you also know the secret.
There would not be a public key in this kind of system. You normally identify the client using a session key or token, so the client needs to authenticate first (which means you need some authentication method). Essentially, what you are trying to implement is OAuth. Perhaps have a look at the library and see if you can use it directly - http://oauth.net/documentation/getting-started/
checking out what amazon s3 is doing for the authentication might be a good start
basically they are doing the same thing as you proposed, and they use RFC 2104HMAC-SHA1 for the hashing. The implementation depends on your choice of languages.
When a server sends a Certificate message to a client, the public key in the server's certificate will be used to verify server’s identity(decryption with the public key).
The server follows its Certificate message with a ServerKeyExchange message, the session key information is signed using the same public key contained in the server’s certificate (encryption with the public key).
So I feel a Public key can be used to encrypt and decrypt data as well, am I right?
If yes, I wonder why text book just states one key(e.g. public key) is used to encrypt , and the other one(private key) is used to decrypt, rather than mention that a key can be used to both encrypt and decrypt?
[UPDATE2]
Really thanks for Bruno's help.
After reading Bruno's replies and RFC4346(section7.4.2 and 7.4.3) again and again, I suddenly felt I grasp the main points. :)
However, I am not sure I am right, and hope someone can confirm my following understanding. Thanks.
1.Server Certificate
SSL and TLS Essential section 3.6.1:
(SSL and TLS Essential: Securing the Web Author: Stephen A. Thomas)
...the public key in the server's certificate will only be used to verify its(server) identity.
Bruno wrote,
The public key in the server certificate isn't used to verify the server's identity itself.
I agree Bruno's viewpoint now, because a certificate is only a private-key-signed(encrypted) message that also contains someone else's(e.g., a client ) public key, so a client should use its trusted copy of the server's public key (usually, web browsers include dozens of these certificates in advance), instead of the public key in the server certificate, to verify the server's identity.
Is it right?
2.Server Key Exchange
SSL and TLS Essential section 3.6.2:
...the key information is signed using the public key contained in the server's certificate.
Bruno wrote,
Similarly, you don't really sign something with a public key. You only need one of the keys to sign, and that's the private key. You verify the signature with the matching public key.
...
"signing with a public key" is an unusual and misleading expression.
I think Bruno is right. The reasons are as follows,
RFC4346 section 7.4.2. Server Certificate
It MUST contain a key that matches the key exchange method, as follows.
Key Exchange Algorithm Certificate Key Type
RSA RSA public key; the certificate MUST
allow the key to be used for encryption.
DHE_DSS DSS public key.
DHE_RSA RSA public key that can be used for signing.
DH_DSS Diffie-Hellman key. The algorithm used
to sign the certificate MUST be DSS.
DH_RSA Diffie-Hellman key. The algorithm used
to sign the certificate MUST be RSA.
So the server sends one of 6 public key types in a certificate first.
RFC4346 section [7.4.3 Server Key Exchange Message][2]
>The server key exchange message is sent by the server.
>...
>This is true for the following key exchange methods:
>
DHE_DSS
DHE_RSA
DH_anon
>...
>This message conveys cryptographic information to allow the client to communicate the premaster secret.
The server chooses one of 3 key exchange methods, and uses its private key to sign(encrypt) cryptographic information.
When the client receives the encrypted cryptographic information, it will use the public key in ServerCertificate message to verify(decrypt) and get the plain-text cryptographic information.
Is it right?
In public key cryptography:
The private key is used for signing and deciphering/decrypting.
The public key is used for verifying signatures and enciphering/encrypting.
See the glossary of the TLS specification:
public key cryptography:
A class of cryptographic techniques employing two-key ciphers.
Messages encrypted with the public key can only be decrypted with
the associated private key. Conversely, messages signed with the
private key can be verified with the public key.
You cannot encrypt with a private key or decrypt with a public key, not for mathematical reasons, but because it doesn't make sense w.r.t. the definition of encrypt:
To convert ordinary language or other data into code; to hide
the meaning of a message by converting it into a form that
cannot be interpreted without knowing the secret method for
interpretation, called the key; to encode.
In a situation where you "encrypt with the private key", you effectively "scramble" the data indeed, but what's required to turn back the message into its original form is not a secret. Hence, it doesn't make sense to talk about encrypting in this context. Whether the mathematical operations behind it may work one way or the other doesn't matter at this stage.
Similarly, you don't really sign something with a public key. You only need one of the keys to sign, and that's the private key. You verify the signature with the matching public key.
It's quite common (even in the TLS specification) to say "signing with a certificate", when what's really implied is computing the signature with the private key matching the certificate. In many cases, not specifically TLS, the certificate itself is communicated along with the signature (whether-or-not one chooses to trust that certificate is another matter).
Expressions such as "using your certificate to authenticate" or "using your certificate to sign" are generally acceptable, so long as you understand that "certificate" is used there to shorten "certificate and its private key", and that it's in fact the private key that's necessary for those operations.
I don't have the book you're quoting, but this quote sounds misleading or incorrect (perhaps taken out of context here):
...the public key in the server's certificate will only be used to
verify its(server) identity.
The public key in the server certificate isn't used to verify the server's identity itself. What it does is ensuring that only someone/something with the corresponding private key will be able to decipher what you've encrypted with this public key: in this case (authenticated key exchange), the pre-master-secret which the server will prove it knows to the client by producing the correct Finished message, based on the pre-master-master it has managed to decipher.
The identity binding is done by the certificate itself, with is the signed combination of the public key, some identifiers (e.g. Subject DN and Subject Alternative Names) and possibly various other attributes (e.g. key usage, ...).
This side of the identity verification (i.e. checking who this certificate belongs to) is established by verifying the integrity of the certificate and that you trust what it says (usually PKI), and by verifying that the identity it belongs to is indeed the one you wanted to connect to (host name verification). This is done by verifying the certificate signature itself using a Certification Authority (CA) or by an outside mechanism, for example if you have explicitly granted an exception for a given certificate (possibly self-signed) using knowledge you have outside the scope of the PKI to which the certificate belongs. This step is rather independent of the TLS specification, although you'll need all these pieces together to make the communication secure.
There's a similar problem with this quote (again, possibly taken out of context):
...the key information is signed using the public key contained in the
server's certificate.
Although saying "signed with a certificate" is a common expression (as explained above), I'd say "signing using the public key" is definitely confusing, since "public key" is normally used in contrast to "private key", and it's really the private key that's used for signing. While even the TLS specification (Section F.1.1.2) talks about "signing with a certificate" in a few places, "signing with a public key" is an unusual and misleading expression.
I'm not sure whether "(decrypt)" and "(encrypt)" are in the book or your additions:
the public key in the server's certificate can be used to
verify(decrypt) the server's identity and sign(encrypt) the key
information(,then the client will use the key information to encrypt
pre_master_secret)
You actually verify you're talking to the actual server identified by that certificate because it's the only one capable of deciphering what you've encrypted with its public key (in the client key exchange message).
As it's put in the TLS specification Section F.1.1.2:
After verifying the server's certificate, the client encrypts a
pre_master_secret with the server's public key. By successfully
decoding the pre_master_secret and producing a correct finished
message, the server demonstrates that it knows the private key
corresponding to the server certificate.
What you're asking at the end doesn't completely make sense:
I know the public key can be used to verify server's
identity(Certificate message), but I can't understand the public key
why can be used to sign the key information, because the client
doesn't have the corresponding private key, how does the client verify
the key information?
The public key isn't used to verify the server's identity. You verify that you're talking to the server who has the private key matching the certificate it presented earlier by the fact it was able to decipher the pre-master-key and produce the correct finished message.
EDIT 2:
Following your edit, it seems that you're still using "sign(encrypt)" and "verify(decrypt)" as if encrypting was the same as signing and verifying was the same as decrypting. I would suggest once again that you stop making these associations: these are 4 distinct operations. While the maths may be the same when using RSA, this doesn't work for DSA, which is only a signature algorithm (so signing/verifying only).
When the client receives the encrypted cryptographic information, it
will use the public key in ServerCertificate message to
verify(decrypt) and get the plain-text cryptographic information.
The client doesn't receive any encrypted data during the handshake (only signed data).
For a better general understanding, you should start by trying to understand how Diffie-Hellman and its ephemeral variant (for DHE cipher suites) work.
(In practice, I wouldn't focus too much on non-ephemeral DH_RSA/DH_DSS cipher suites. To be honest, I'm not sure whether they're much used. I haven't seen any example of certificate with the necessary DH attributes, and these cipher suites are not in supported lists by OpenSSL or Java 7. DHE/EDH are much more common, and don't require special attributes in the certificate.)
If you use an RSA key exchange algorithm, the client will encrypt the pre-master-key in the client key exchange message; if it's using one of the DH key exchange algorithms, it will send its DH parameters so that client and server can agree on a pre-master-key (in this case, the client will have checked that the server's DH parameters come from the right server by verifying the signature of the server key exchange message sent beforehand). See description of the Client Key Exchange Message:
With this message, the premaster secret is set, either though
direct transmission of the RSA-encrypted secret or by the
transmission of Diffie-Hellman parameters that will allow each
side to agree upon the same premaster secret.
Regarding the other points:
a certificate is only a private-key-signed(encrypted) message that
also contains someone else's(e.g., a client ) public key, so a client
should use its trusted copy of the server's public key (usually, web
browsers include dozens of these certificates in advance), instead of
the public key in the server certificate, to verify the server's
identity.
Three things happen to verify you're talking to the right server:
The handshake itself, if successful, guarantees that you're talking to the server that has the private key for the certificate it has presented in the server certificate message. If using RSA key exchange, this is guaranteed by the fact it's the only one that can decipher what the client sent in the client key exchange message (since it's encrypted with the public key); if using an EDH key exchange, this is guaranteed because the server signed its DH parameters in the server key exchange message, verifiable with this public key.
The fact that you can verify the certificate itself. This is rather independent of how TLS works, but it's usually done using a PKI: the client has a pre-set list of trusted CA certificates, the public keys of which can be used to verify the signature in new certificates it doesn't already know about (such as the server certificate). Verifying that signature allows the client to bind that public key to an identifier (Subject DN and/or alt. name). This gives you the identity of the server to which the client is talking.
The host name verification: it's not good enough to know that you're talking to someone who's presented you a genuine ID that's valid for them, you also need to check that the name matches the server you intended to connect to.
When I said "The public key in the server certificate isn't used to verify the server's identity itself", I meant that the public key wasn't used to verify points 2 and 3. Point 1 guarantees you that you're talking to the server that has the private key matching the certificate it presented, but it doesn't tell you who this is. Verifying the identity of the server is is up to point 2, so as to be able to bind an identifier to that key/cert.
In a PPK algorithm like RSA, you have two different communication channels. Information encrypted using the public key is only readable to the possessor of the private key, and information encrypted using the private key is only readable to the possessor of the public key.
In fact, the selection of which half of the pair is "public" is completely arbitrary.
Now, in practice this doesn't much matter; the whole world has access to the public key, so encrypting something with the private part wouldn't do anything to secure it. But you can use this for authentication: since only one holder has the private key, if a message is validly encrypted using it, then the private-key holder must have been the author.
That's why your book doesn't say that the private key is used for encryption: because it's used for integrity, not for confidentiality, as any message sealed using it would be readable to anyone possessing the non-secret public half. While the integrity validation mechanism is technically encryption (it is encipherment using modular exponentiation), it would be confusing to mention this in a foundations-of-cryptography context as it's not what people think when they hear "encryption" - they think "privacy".
I had the same restlessness, since its difficult to ask without falling in technical holes. Quick, easy answer: regardless of the method details, yes, you could reverse engineer and decrypt from one key (is just math at the end) but because of the complexity of the operation, using a long value (you may use ie. a 4096-bit value) would make the reverse operation timeless, so you had to use either a machine so big that still doesnt exists or wait indefinitely, making it inviable... until someone comes up with a quick method to do it.
may want to take a look:
http://www.usna.edu/CS/si110arch/si110AY12F/lec/l29/lec.html
I have created two SecKeyRef items via SecKeyGeneratePair, but now I would like to turn the public key into a x509 Digital Certificate – and/or both the public and private keys into a PKCS #12 (.p12) certificate – and save it to disk as a file. This way I can do whatever I need to with it, including sending the certificates to other services or computers.
I would prefer to not use the keychain, but even with that I am having some trouble finding good documentation on exactly how to create a certificate out of a pair of SecKeyRefs, and writing them out as a certificate file.
You want the Security Transforms Programming Guide. It covers most of what you need for this. Some parts (like SecItemExport) are not documented in the reference documentation, only in the public header files. But SecItemExport is the new 10.7 way to handle this stuff.
I'm using VB.Net's RijndaelManaged (RM) to encrypt files, using the RM.GenerateKey and RM.GenerateIV methods to generate the Key and IV and encrypting the file using the CryptoStream class. I'm planning on saving this Key and IV to a file and want to make sure I'm doing it the right way. I am combining the IV+Key, and encrypting that with my RSA Public key and writing it out to a file. Then, to decrypt I use the RSA Private key on this file to get the IV+Key, split them up and set RM.Key and RM.IV to these values and run the decryptor.
Is this the best method to accomplish this, or is there a preferred method for saving the IV & Key?
That's the general idea. The IV does not need to be kept secret, however. Only the "secret key" is secret.
You might consider using a standard like S/MIME (or its underlying Cryptographic Message Syntax) for your file format. I'm not sure if Bouncycastle has ported their S/MIME (or PGP) libraries to .NET, or if .NET has S/MIME support built-in, but there is likely to be a free library available for your platform. A widely used open source library like this has probably been tested and scrutinized more closely than you'll be able to do yourself, and they implement protocols designed by experts in cryptography so your don't have to wonder what pitfalls you've overlooked.
The model you've described is similar to S/MIME's "KeyTransRecipientInfo" method for key exchange. A symmetric key is generated to encrypt the data, then that key is encrypted with the public key of each intended recipient. The encrypted key and the IV (which is not encrypted in this standard) are bundled with information about the algorithms used in a CMS structure. The encrypted data can be included in the same bundle, or be in an separate file.
PGP has mode that is an essentially equivalent.