I'm have tool which manually dissect ssl/http traffic using openssl library. It's work fine in most cases, but failed wheen client/server usese Extended Master Secret extension.
Fail occured on last stage of handshake, where veryfying of test encripted data performed .
As I understand for proper work of this extension I need properly fill handshake buffer with specific TLS fields (client hello, server hello, key exchange) and then master key should be generated baasing on hashing of handshake buffer.
Unfortunatelly it's not work for me.
So my question-which exactly tls packeds should be hashed for generating proper master key ?
Will be gratefull for any comments.
Thanks.
PS: I downloaded experemental version of openssl from github, where this functionality implemented.
openssl s_server/s_client uses corespond extension, Ican see it in Wireshark, and able to decode traffic with server pem file.
You need to do the following to generate the session hash
1) Append all the Handshake messages, other than encrypted handshakes in order of arrival, please don't include the record header to the handshakes. only messages and message headers
2) Hash them based , simple to say,
Hash algorithm varies based on Cipher Suite and Protocal Version
TLS1 and TLS1.1 its mixture of SHA1 and MD5 (16 bit each)
TLS1.2 its mostly SHA256 or SHA384 based on cipher suite.
Related
I want to set up and run SSL client and server with TLS 1.2 using openSSL 1.1.1 for my project.
I have a few doubts and/or requirements with openSSL:
what is the right way to select TLS 1.2 as version? Currently I am using TLS_client_method(). the other methods seem to be deprecated. Is there a specific API for choosing particular version?
I need to run an SSL clinet with following handshake extensions.
ec_point_formats with "uncompressed" as value
supported_groups with list of "secp521r","secp384r1","secp256r1","secp224r1","secp192r1","secp160r1","ffdhe2048"
encrypt_then_mac with value 0
server_name
extended_master_secret without any value and length set to 0
I need to run an SSL server with following handshake extensions
encrypt_then_mac with value 0
extended_master_secret without any value and length set to 0
Apart from above header extension fields I want to disable everything else. How to accomplish that?
From client I want to provide only "TLS_PSK_WITH_AES_128_CBC_SHA256" cipher suite as option.
Is client certificate and key files (pem) necessary for successful connection establishment and communication when PSK cipher is used?
Is there any example or open source client-server implementation with PSK authentication?
what is the right way to select TLS 1.2 as version? Currently I am using TLS_client_method(). the other methods seem to be deprecated. Is there a specific API for choosing particular version?
TLS_client_method() is the correct method to use. To specify that no version below TLSv1.2 is used you should do this:
SSL_CTX_set_min_proto_version(ctx, TLS1_2_VERSION);
Where ctx is your SSL_CTX object.
If you also want to prevent anything higher than TLSv1.2 from being used then do this:
SSL_CTX_set_max_proto_version(ctx, TLS1_2_VERSION);
See this page for information on these calls:
https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_min_proto_version.html
I need to run an SSL clinet with following handshake extensions.
ec_point_formats with "uncompressed" as value
supported_groups with list of "secp521r","secp384r1","secp256r1","secp224r1","secp192r1","secp160r1","ffdhe2048"
These extensions aren't used if you specify the PSK ciphersuite that you've selected so OpenSSL won't send them (assuming you've restricted the max TLS protocol version as described above, and have configured only that ciphersuite). Doing so would be pointless. Even if it did, OpenSSL 1.1.1 does not support "ffdhe2048". It does support all the others.
encrypt_then_mac with value 0
Not sure what you mean by "with value 0" since this extension is always empty and has no value. I assume you mean with length 0. This extension is sent by default so you don't need to do anything.
server_name
You should call:
SSL_set_tlsext_host_name(ssl, "hostname of the server");
Where ssl is your SSL object, and replacing "hostname of the server" with the server's real hostname.
See this page for information on this call:
https://www.openssl.org/docs/man1.1.1/man3/SSL_set_tlsext_host_name.html
extended_master_secret without any value and length set to 0
This extension is sent by default so you don't need to do anything.
I need to run an SSL server with following handshake extensions
encrypt_then_mac with value 0
extended_master_secret without any value and length set to 0
If the client sent them, then the server will echo them back by default. You don't need to do anything.
Apart from above header extension fields I want to disable everything else. How to accomplish that?
An OpenSSL client will additionally send the session_ticket extension. Its harmless, but if you really want to disable it you can do this:
SSL_CTX_set_options(ctx, SSL_OP_NO_TICKET);
See this page for further information:
https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_options.html
An OpenSSL server will additionally send the "renegotiate" extension. You must not disable this (in fact you cannot). Ignore it. Its harmless.
From client I want to provide only "TLS_PSK_WITH_AES_128_CBC_SHA256" cipher suite as option.
That is the official IANA name for the ciphersuite. OpenSSL knows it as "PSK-AES128-CBC-SHA256".
Configure it on both the client and the server like this:
SSL_CTX_set_cipher_list(ctx, "PSK-AES128-CBC-SHA256");
See this page for further information:
https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_cipher_list.html
In addition you will need to provide PSK callbacks to tell OpenSSL what the PSK is that you want to use.
On the client you need to call SSL_CTX_set_psk_client_callback(). On the server you must call SSL_CTX_set_psk_server_callback().
See these pages for further information:
https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_psk_client_callback.html
https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_psk_server_callback.html
TLSv1.3 ciphersuites will still automatically be sent unless you have additionally restricted the max TLS protocol version to TLSv1.2 as described above. Finally you will also see a ciphersuite called "TLS_EMPTY_RENEGOTIATION_INFO_SCSV" being sent. This isn't actually a real ciphersuite at all. It is always sent and cannot be suppressed. It will never be negotiated and is harmless. Ignore it.
Is client certificate and key files (pem) necessary for successful connection establishment and communication when PSK cipher is used?
No.
Is there any example or open source client-server implementation with PSK authentication?
You can look at how s_client and s_server do it:
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/s_client.c
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/s_server.c
I have 100 HTTPS services running on a single server using SNI. (Actually, I don't have access to them. It's an assignment. All I know are their domain names N.xxx.yy where N is in range from 00 to 99.) The goal of the assignment is to evaluate security of every single connection to each of these servers. So some of the servers contain expired certificates, certificates with wrong CN, etc.
My problem is that I cannot get past the handshake on some of the servers. I have written my own application in C++ using OpenSSL, but I've also tried it with openssl s_client. This is how I connect to the server:
openssl s_client -host N.xxx.yy -port 443 -verify 1 -servername N.xxx.yy -CAfile assignment-ca.pem
And this is what I get:
139625941858168:error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:s3_pkt.c:1493:SSL alert number 80
139625941858168:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
In Wireshark, I see that client sent ClientHello, server responded with ServerHello (choosing TLSv1.2 and ECDHE-RSA-AES256-GCM-SHA384) followed by Certificate and then it sent me Alert message containing Internal Error (80).
After trying different thing I have found out that if I run s_client with -tls1 or -tls1_1 I can successfully get past the handshake. -tls1_2 does not work. What is even stranger is that connection through Chrome/Firefox/any other browser succeeds even if TLSv1.2 is negotiated. From what I see, Chrome is sending a different cipher list than me or s_client but even after modifying the cipher list to match the one in Chrome (and making sure that server chooses ECDHE-RSA-AES128-GCM-SHA256), it does not work either. Chrome is sending these TLS extensions, which I don't but most of them seem empty:
Unknown 47802
renegotiation_info
Extended Master Secret
signed_certificate_timestamp
status_request
Application Layer Protocol Negotiation
channel_id
Unknown 6682
Can anybody explain me what is happening here? Unfortunately, I have no way to debug it on the server side so this is all I know.
UPDATE:
After playing around with forged ClientHello messages I managed to track it down to signature_algorithms extension. My app and s_client provide SHA384 + {RSA,DSA,ECDSA} but if I remove these and keep just SHA256 + {RSA,DSA,ECDSA}, as Chrome does, it works and I receive Server Key Exchange message successfully. Could it be that server somehow does not support it, but instead of providing meaningful error message, it just ends unexpectedly and gives me this internal error?
UPDATE 2:
I found answer to why it works with TLS versions prior to 1.2 in RFC5246. Question from the previous UPDATE still holds.
Note: this extension is not meaningful for TLS versions prior to 1.2.
Clients MUST NOT offer it if they are offering prior versions.
However, even if clients do offer it, the rules specified in [TLSEXT]
require servers to ignore extensions they do not understand.
Since you wrote that -tls1_2 does not work I assume either you and/or the server uses an older openssl library. The current version while writing this is 1.1.0e
There were quite some fixes since 0.9.8, which could often be seen on older systems.
For Version 1.0.1 there was this fix, which sounds like your problem:
`Some servers which support TLS 1.0 can choke if we initially indicate
support for TLS 1.2 and later renegotiate using TLS 1.0 in the RSA
encrypted premaster secret. As a workaround use the maximum permitted
client version in client hello, this should keep such servers happy
and still work with previous versions of OpenSSL.`
Maybe also notable:
Don't allow TLS 1.2 SHA-256 ciphersuites in TLS 1.0, 1.1 connections.
So I would suggest to update your openssl-Version and in case of the servers out of your control I would stick to the settings you already found.
I am trying to decrypt a pcap file. This pcap file contains a capture of an HLS encrypted video stream. The pcap contains TLSv1.2 packets.
Below are some information from the pcap file
Server Hello message Cipher Suite:
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384.
EC Diffie-Hellman server Params: pubkey (1)
The Certificate Status message:
Signature Hash Algorithm Hash: SHA256
Signature Hash Algorithm Signature: ECDSA
Client Key Exchange Message
EC Diffie-Hellman server Params: pubkey (2)
I tried to follow [this Wireshark SSL decryption tutorial][1]. But it seems that it works only for RSA encryptions.
I have been researching for a while and found [this discussion][2]. I am quoting an extract from this discussion:
There is an important parameter to mind: decryption of a passively
recorded session (with a copy of the server private key) works only if
the key exchange was of type RSA or static DH; with "DHE" and "ECDHE"
cipher suites, you won't be able to decrypt such a session, even with
knowledge of the server private key. In that case, you will need
either the negotiated "master secret", or to use the server private
key to actively intercept the connection
It's note worthy that I have the client private key. In my case, the client is FFmpeg video streamer (FFplay). I had a look also on the [TLS v1.2 RFC][3].
My question:
Is it possible to do a decryption in this scenario ? If yes, what do I need to have to do so?
Is the decryption done using the client's private key or using the pre_shared_master (i.e. Diffie-Hellman) ?
[1]: https://wiki.wireshark.org/SSL
[2]: https://security.stackexchange.com/questions/117778/how-to-decrypt-ssl-traffic-with-a-packet-sniffer-when-i-have-the-private-key
[3]: https://www.rfc-editor.org/rfc/rfc5246
First, the clients private or public key are not involved in the key exchange in any way but only used to authenticate the client (if requested by the server). What is used in the key exchange are the servers private and public key, but only if RSA key exchange is used. For DHE/ECDHE they are not used so private/public key are not sufficient. See it is possible to decrypt HTTPS with the (private, public) pair if it uses DHE? for the details why this is the case.
What you would need instead of the private key is actually the exchanged key which is unique for each TLS session even if the private key is the same. Some clients can store this key for later use and if your client can do it see Decrypting TLS Browser Traffic With Wireshark – The Easy Way! how to proceed then to decrypt the traffic.
No, it is not possible to decrypt in this scenario. That would involve breaking EC Diffie-Hellman.
Decryption is not directly performed using the pre_master_secret but it is performed by keys directly derived from the pre-master secret. That is: the client and server decryption keys that are derived from it by first deriving the master_secret and then performing the PRF and dividing the output to the session keys and IV's.
I am experiencing a strange issue with assigning a new SHA256 certificate to an IIS web server.
The server has SSL 3.0, TLS 1.0, 1.1 and 1.2 enabled and when using a server certificate on a site signed with RSA (rather than SHA256RSA) the client connects and negotiates TLS_RSA_WITH_AES_256_CBC_SHA for the TLS encryption.
The second the SHA256 certificate is used on the site it then tries to use TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA.
What is odd is that when checking using www.ssllabs.com server test I can see that completely different ciphers are presented from the server when the SHA256 certificate is used or is not.
Ciphers Suites presented when site using SHA1 certificate.
TLS_RSA_WITH_AES_256_CBC_SHA256
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLS_ECHDE_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WTH_RC4_128_MD5
Ciphers Suites presented when site using SHA256 certificate.
TLS_ECHDE_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
I can find no information to show why this happens but I was aware TLS_RSA_WITH_AES ciphers were TLS 1.2 compatible and there seems to be no literature to state if the server certificate is SHA256 that it enforces Elliptical Curve Diffie-Hellman Exchange for the key encryption.
Can anyone explain why this might be happening?
Kind Regards
James Tighe
I have now managed to resolve this issue in case anyone else is getting it.
It seems that it is to do with the Cryptography/Key Store Provider used to request the CSR.
If the CSR is generated on the Windows Server 2008 R2 machine using a Crypto API providers (such as Microsoft Strong Cryptographic Provider) and the KeySpec is not specifically mentioned when generated the CSR it sets the KeySpec = 2 AT_SIGNATURE. The KeySpec determines whether the key can be used for Signatures, Key Exchange (encryption) or both.
When setting the Cryptography provider to a Crypto API provider seems to cause the Windows Server 2008 R2 machine to complete the request but default to KeySpec = 2 In our case it was because we had generated a .INF file to be used with CERTREQ as IIS was not able to enforce the use of SHA256 as the signing algorithm.
If the KeySpec is set in the .INF file to KeySpec = 1 AT_KEYEXCHANGE then this should work although we resolved this issue a different way.
In order to resolve the issue we changed the .INF file to set the ProviderName = "Microsoft Software Key Storage Provider" (a CNG provider). When the certificate was completed it showed the KeySpec = 0 AT_NONE which is when the key store provider is set to CNG.
This issue was raised to Microsoft to help us get a better understanding and it appears that we were right in the KeySpec behaviour and the usage of the Cryptography Provider.
They confirmed that in the .INF file (or the customer template enrolment in Cert MMC) you need to ensure the KeySpec property is set to 1 AT_KEYEXCHANGE or else it will default to 2 AT_SIGNATURE. If you use a CNG provider it will instead default to use 0 AT_NONE and will work correctly.
I hope this can help other people that might be getting this annoying issue as it was quite hard to find a solution.
Kind Regards
James Tighe
I'm trying to figure out the SSL handshake process. After reading up on TLS in Wikipedia I've seen that
The server sends its Certificate message (depending on the selected cipher suite, this may be omitted by the server)
I've also seen such behavior in real-life sniffs, but only in cases where the user eventually received an "Invalid certificate" warning.
I was wondering in which cases can a server omit the certificate? How can the client verify the server's identity in this case then? Or is it only reserved to cases where the server have no certificate and gives up on sending a fake one, knowing that the user will see a browser warning anyway?
Thanks!
Some cipher suites don't rely on certificates:
The anonymous cipher suites, defined in the main TLS RFC (the names containing DH_anon). Some of them can provide encryption, but without authentication, which is insecure. Section A.5 says the following about them:
The following cipher suites are used for completely anonymous
Diffie-Hellman communications in which neither party is
authenticated. Note that this mode is vulnerable to man-in-the-
middle attacks. Using this mode therefore is of limited use: These
cipher suites MUST NOT be used by TLS 1.2 implementations unless the
application layer has specifically requested to allow anonymous key
exchange. (Anonymous key exchange may sometimes be acceptable, for
example, to support opportunistic encryption when no set-up for
authentication is in place, or when TLS is used as part of more
complex security protocols that have other means to ensure
authentication.)
Kerberos cipher suites, in which case the identification is done via the Kerberos ticket, and the name is verified against the Kerberos principal name (host/MachineName#Realm).
Pre-Shared Keys cipher suites (see section on PSK Identity Encoding).
There is one valid case for anonymous ciphers: the opportunistic encryption of e-mail over SMTP with STARTTLS. As an MITM could easily prevent the use of TLS, there is no use in protecting agains MITM at the TLS level.
If the client knows that he will proceed anyway, even if the certificate was invalid, the can request an anonymous ciphersuite saving the server the generation of a signature and himself the verification the certificate and the signature.