Problem Background:
As part of the Computer Networking course assignment, I have been given task of implementing a Proxy Server ( using python socket and ssl module ) that handles https communications between the browser and the origin server (The real server that my browser wants to talk to).
What I have done so far:
I have implemented the above requirement using ssl sockets and also generated self-signed 'cert.pem' 'key.pem' files.
What I need to do:
Now I just need to tell my browser (chrome 89 on kubuntu 20.04) to accept this self-signed certificate and then test the working of my proxy server.
Reading from this stackoverflow question, I can see that I have to:
(1) become my own CA (2) then sign my SSL certificate as a CA. (3) Then import the CA certificate (not the SSL certificate, which goes onto my server) into Chrome.
My confusion/question:
So if I do this, when eventually I am done with this assignment, how do I reverse all these steps to get my browser in the previous state before I had made all these changes. Also, how to reverse the "become your own CA" and also delete the SSL certificates signed by my CA.
Basically, I want my system to return to the previous state it was before I would have made all these changes.
UPDATE:
I have done the previously outlined steps but now I get an error.
Here is a snippet of my code:
serv_socket = socket(AF_INET, SOCK_STREAM)
serv_socket.bind(('', serv_port))
serv_socket.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context = context.load_cert_chain('cert.pem', 'key.pem')
context.set_ciphers('EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH')
serv_socket.listen(10)
socket_to_browser, addr = serv_socket.accept()
conn_socket_to_browser = context.wrap_socket(socket_to_browser, server_side=True)
At the last line conn_socket_to_browser = context.wrap_socket(socket_to_browser, server_side=True) an exception is thrown: [SSL: HTTPS_PROXY_REQUEST] https proxy request (_ssl.c:1123)
What am I doing wrong ?
As glamorous as "becoming your own CA" sounds, with openssl it basically comes down to creating a self-signed certificate, and then creating a directory where some CA-specific configuration will be stored (I don't fully remember the specifics, but I think it was just some files related to CNs and serial numbers) so basically reversing the "become your own CA" step is something as mundane as deleting this directory along with the private key and self-signed certificate you were using for the CA. That's it, the CA is no more.
And for chrome returning to the previous state, you would just go the the CA list where you added the CA certificate, select it and delete it. Chrome will stop accepting certificates signed by your CA.
Regarding your new problem... In my opinion, you have developed some kind of reverse proxy (meaning that you expect normal HTTPS requests that you then redirect to the real server) but you have configured Chrome to use it as a forward proxy. In this case, Chrome does not send it a normal HTTPS request, it sends a special non-encrypted CONNECT command and only after receiving the non-encrypted response, it negotiates the TLS connection. That's why openssl says "https proxy request" because it has detected a "https proxy request" (a CONNECT command) instead of the normal TLS negotiation.
You can take a look at How can a Python proxy server (using SSL socket) pretend to be an HTTPS server and specify my own keys to get decrypted data?
It's python, but I think that you'll get the idea
Related
Is it possible to directly expose a server behind a L4 load balancer, with a public certificate?
This server is inside a Kubernetes pod. There is a TCP loadbalancer service in front of it which creates the external L4 LB.
My problem is that the TLS traffic does not reach the container inside the pod. So if you succeeded with a similar configuration, I would be interested into knowing.
Update
I did not mention that the traffic is GRPC.
Here is what I did: I have a domain and a corresponding official certificate. I want to secure the grpc connection.
I tried two approches:
with google ESP container, I put the cert as an nginx secret, pass it to the container, set an ssl-port. Behind the ESP in the same pod, I have my grpc server
In this case I get a message like this on the client side:
D0610 14:38:46.246248584 32401 security_handshaker.cc:176] Security
handshake failed:
{"created":"#1591792726.246234613","description":"Handshake
failed","file":"../deps/grpc/src/core/lib/security/transport/security_handshaker.cc","file_line":291,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}
I see some TLS exchanges with wireshark but no log in esp.
without ESP, I put the cert in my GRPC server. There the GRPC server fails with something like this:
error:1408F10B:SSL routines:ssl3_get_record:wrong version number
In the google ESP documentation I see that I have to prove the domain belongs to me and upload the cert (but where)?
Update 2
As of today, I see no evidence that it is feasible.
IMO, the main issue is that the L4 has the IP corresponding to the domain name of the certificate. Hence the pods don't have the correct IP to prove that they can use the cert so their request towards roots are denied (I have no proof of that as I can't get debug info from nginx in the ESP. I have seen a request with the pure GRPC server solution though).
The issue was in TLS exchange.
By installing the cert in the ESP, it works fine with a web browser and the certificate is indicated valid, whereas with a GRPC client, the TLS handshake fails. Adding additional trace info helped.
By checking my certificate (not self signed but attached to my domain), I found that there is an intermediate certificate provided with it. I installed it along with the domain certificate (in the same crt file) and then it worked.
I don't know exactly why it is behaving like this but maybe it's due to the root_cert file in grpc client lib being too old.
By the way for a domain cert, there is no specific requirement regarding CN and subjectAltName for the certificate. It works without it. So it must only apply to self signed certs as I have seen elsewhere.
I had another issue that disturbed my task: be careful not to name the service port of the L4 load balancer with 'http2' inside. I had some side effect, making another deployment fail due to this. In fact when you do https, don't put http2 in the name.
Anyway it is now working and answers the request for the bounty. Thanks to all who tried to help :)
Sim800 supports SSL protocol. AT command "AT+CIPSSL" sets TCP to use SSL function.
In the "sim800_series_ssl_application_note_v1.01.pdf" is noted that: "Module will automatic begin SSL certificate after TCP connected."
My Problem: What is the exact meaning of the begin SSL certificate? what does sim800 do exactly? Does sim800 get SSL certificate from website? where does sim800 save SSL certificate?
As far as I know, SIM800 has some certificates in it and when you use a TCP+SSL or HTTP+SSL connection it will automatically use those certificates.
If those certificates are not ok for you, you will need to use an SD card, save there the certificates you want and use the command AT+SSLSETCERT to set the certificate you saved on your SD card. Here you can find how to use the File System.
Usually the certificates that come with the module are enough and you won't need this. But for example they didn't work for me when I tried to communicate with Azure via MQTT. I had to encrypt the data myself using wolfSSL library and send it using TCP without SSL.
Note: Not all SIM800 modules have SD card support.
There are a very few information about sim800 and ssl certificate on the web, and like you i got a lot of questions about it.
About your questions on how does sim800 get certificate and where does it save it, it seems, according to sim800_series_ssl_application_note_v1.01.pdf, that you can create (defining your own path), write and import a ssl certificate on your own with the AT+FSCREATE, AT+FSWRITE and AT+SSLSETCERT commands. An example is provided at the paragraph 3.10.
I'm sorry, i can't answer your other questions.
Anyway, if you get further informations about sim800 and ssl, i would be grateful if you share it with me.
When you use AT+CIPSSL you tell the SIM-module to use the SSL connection with TCP. When you use +CIPSTART command->
SIM module requests the TCP connection with the server through SSL.
Server sends the Server SSL certificate.
The authenticity of that certificate is checked with internal certificate authority certificate (The one that resides inside SIM-module) which is cryptographically connected with server certificate.
If the authenticity of certificate can not be confirmed SIM-module will close the connection unless you use the command AT+SSLOPT=0,0 (which forces the SIM-module to ignore invalid certificate authentication) prior to AT+CIPSSL command.
//Key exchange
SIM-module then encrypts it's master key (already inside SIM-module cannot be changed or read) with the public key (Which is part of the already sent server certificate) and sends it back to server.
Server then encrypts it's master key with SIM-module's master-key and sends it back to SIM-module. Key exchange is now complete as both (server and SIM-module) recieved master keys.
SIM-module currently doesn't support Client authentication which means that server cannot authenticate the client. That means there must be some other option of authentication (For example in MQTT that can be username and password that only client knows)
If you want your module to be able to authenticate server you will need to create the self-signed certificate for server and certificate authority certificate (for SIM-module) which is cryptographically connected to self-signed certificate and upload them to server and SIM-module (through AT+SSLSETCERT command from SD card).
If you only want to encrypt the data traffic you can ignore invalid certificate (AT+SSLOPT=0,0) as you will recieve publickey nevertheless. But if you want to be sure about server authenticity you will need to upload right certificate to module.
I am trying to create an HTTPS-tunnel on my machine. My intention is having all requests to https://localhost:8888/<something> (the port where Fiddler is listening to) be directed to https://myserver.net/<something>. I am using the following script as per Fiddler doc:
static function OnBeforeRequest(oSession: Session) {
// <Fiddler 2 preexisting code>
// HTTPS redirect -----------------------
if (oSession.HTTPMethodIs("CONNECT") &&
(oSession.PathAndQuery == "localhost:8888"))
{
oSession.PathAndQuery = "myserver.net:443";
}
if (oSession.HostnameIs("localhost"))
oSession.hostname = "myserver.net";
// --------------------------------------
// <Fiddler 2 preexisting code>
}
Also in Fiddler settings I checked the decryption check and installed certificates as you can see in the image below:
I restart Fiddler, it prompts me to install its fake certificates, I agree. I can see the certificate in my Windows Certificate System Repository when using certmgr. It is a self-signed certificate.
So What I do is opening a browser and type: https://localhost:8888/mypage.html, and what I get is an error. Internet Explorer reports this:
Error: Mismatched Address. The security certificate presented by this
website was issued for a different website's address. This problem
might indicate an attempt to fool you or intercept any data...
When I get certificate info (basically the certificate presented by the contacted host is being rejected, the same certificate can be displayed), I can see that the rejected certificate was issued by Fiddler and the subject is myserver.net.
So the certificate is ok because it is certifying myserver.net, I see that the problem is that probably my browser was expecting a certificate whose subject is localhost. Is it true?
How to handle this situation?
Assumption
I can understand that the problem is a certificate being issued for a website which I did not ask for. So the solution would be using a certificate certifying localhost:8888?
A certificate is valid if it is directly or indirectly (via intermediate certificates) signed by a trusted CA and if the hostname matches the certificate. If the last condition would not be enforced anybody with a valid certificate from a trusted CA could incorporate any other site.
To make use of fiddler and not run into this problem you should configure your browser to use fiddler as a web proxy and then use the real URL inside the browser instead of ip:port of fiddler.
I am testing a web service with an external partner using 2 way SSL under IIS 7.5. I am requiring SSL, requiring a client cert, and using one to one mapping to authenticate to a domain account. I have configured everything and it works fine on our network (I am able to provide a client cert, get authenticated and invoke the service from browser and test harness).
From outside of our network (in most cases, see below), I am getting a 403.7 error. I have gone through the machine level certificate store and made sure the certificates and CAs are trusted.
Here's the weird thing. I obtained a Type I cert to test from home (and got 403.7 like our intended partner is). So I setup Fiddler to debug SSL and send my certificate, and this works for some reason. I setup a test harness to pass the exact same certificate, and got 403.7. I test in my browser (IE 9), don't get a prompt for a client cert, and get 403.7.
Any help appreciated.
Bill
Last time I checked, IIS was using re-negotiation (by default) to get the client certificate: there is a first handshake where the server doesn't request a client certificate, followed by another handshake (encrypted this time) where the server requests the certificate (via a TLS CertificateRequest message). This will prevent you from seeing anything from Wireshark, unless you configure it to use the server's private key and decipher the traffic (note that this only works with some cipher suites).
One way to see the client-certificate negotiation is to configure IIS to use initial client certificate negotiation, using netsh and clientcertnegotiation=true (which is about initial negotiation). At least the CertificateRequest and the certificate will be sent in clear during the handshake, so you should be able to see this with Wireshark.
If the client isn't sending a certificate to the server as a response to the CertificateRequest, you'll still see an empty Certificate message from the client.
If you don't export the private key with the certificate to use with Fiddler or whichever other client, there is no chance that it will be able to use the certificate. It may at best try to send the certificate, but the handshake will fail (since the CertificateVerify message needs to be signed by the client's private key).
I guess you may encounter a problem whereby:
not presenting a certificate is accepted by the server (it's effectively optional),
presenting an invalid certificate makes it fail and causes this 403.7 status code (many servers and SSL/TLS stacks would implement this as a fatal error, but TLS specification doesn't say that unsupported_certificate, certificate_revoked, certificate_expired, certificate_unknown should be fatal, so this is at the server's discretion).
Are you using the same physical machine to test both the in-network and external-network connections? If not, are you sure that the external-network client has the private key accessible?
I have not configured Fiddler client authentication before. Does it read the client certificate and key from the standard certificate stores? Does it read directly from a PKCS12?
One other thing that may be helpful is inspecting the TLS handshake in WireShark. Specifically, check out the Server's "Certificate Request" message, as the data here clues the client (IE9) which client certificates it should display in the prompt. Compare this for the internal and external connections.
I have open ssl installed on the server, all the key ,csr and crt on the server. Configured apache conf to the correct path for key and cert but i don't see a lock in the url(firefox 3.6.2).In chrome it shows https crossed out with red.Does this mean the certificate is not working properly? I have apache2 as the web server.
tls provides both encryption and authentication.
Encryption means that outsiders are unable to read your traffic.
Authentication means that you are confident of the identity of the host your are communicating with.
If chrome crosses out the https, it means that you are using tls, and you have probably set up encryption properly, but chrome is not confident in the authentication of the server. Typically, this is caused by an untrusted certificate; either the subject does not match, or the CA is not trusted.
If you are using a self-signed cert, then it's probably an untrusted CA. Installing the CA into chrome should fix the problem.
I face same problem some time ago that I have installed the SSL certificate successfully but still it show cross on browser address bar, I found the issue was caused due to a image and a javascript file which was included as absolute HTTP url. I change absolute URL to relative and now both files were loading over HTTPS and browser show green bar.