I am trying to create an HTTPS-tunnel on my machine. My intention is having all requests to https://localhost:8888/<something> (the port where Fiddler is listening to) be directed to https://myserver.net/<something>. I am using the following script as per Fiddler doc:
static function OnBeforeRequest(oSession: Session) {
// <Fiddler 2 preexisting code>
// HTTPS redirect -----------------------
if (oSession.HTTPMethodIs("CONNECT") &&
(oSession.PathAndQuery == "localhost:8888"))
{
oSession.PathAndQuery = "myserver.net:443";
}
if (oSession.HostnameIs("localhost"))
oSession.hostname = "myserver.net";
// --------------------------------------
// <Fiddler 2 preexisting code>
}
Also in Fiddler settings I checked the decryption check and installed certificates as you can see in the image below:
I restart Fiddler, it prompts me to install its fake certificates, I agree. I can see the certificate in my Windows Certificate System Repository when using certmgr. It is a self-signed certificate.
So What I do is opening a browser and type: https://localhost:8888/mypage.html, and what I get is an error. Internet Explorer reports this:
Error: Mismatched Address. The security certificate presented by this
website was issued for a different website's address. This problem
might indicate an attempt to fool you or intercept any data...
When I get certificate info (basically the certificate presented by the contacted host is being rejected, the same certificate can be displayed), I can see that the rejected certificate was issued by Fiddler and the subject is myserver.net.
So the certificate is ok because it is certifying myserver.net, I see that the problem is that probably my browser was expecting a certificate whose subject is localhost. Is it true?
How to handle this situation?
Assumption
I can understand that the problem is a certificate being issued for a website which I did not ask for. So the solution would be using a certificate certifying localhost:8888?
A certificate is valid if it is directly or indirectly (via intermediate certificates) signed by a trusted CA and if the hostname matches the certificate. If the last condition would not be enforced anybody with a valid certificate from a trusted CA could incorporate any other site.
To make use of fiddler and not run into this problem you should configure your browser to use fiddler as a web proxy and then use the real URL inside the browser instead of ip:port of fiddler.
Related
Problem Background:
As part of the Computer Networking course assignment, I have been given task of implementing a Proxy Server ( using python socket and ssl module ) that handles https communications between the browser and the origin server (The real server that my browser wants to talk to).
What I have done so far:
I have implemented the above requirement using ssl sockets and also generated self-signed 'cert.pem' 'key.pem' files.
What I need to do:
Now I just need to tell my browser (chrome 89 on kubuntu 20.04) to accept this self-signed certificate and then test the working of my proxy server.
Reading from this stackoverflow question, I can see that I have to:
(1) become my own CA (2) then sign my SSL certificate as a CA. (3) Then import the CA certificate (not the SSL certificate, which goes onto my server) into Chrome.
My confusion/question:
So if I do this, when eventually I am done with this assignment, how do I reverse all these steps to get my browser in the previous state before I had made all these changes. Also, how to reverse the "become your own CA" and also delete the SSL certificates signed by my CA.
Basically, I want my system to return to the previous state it was before I would have made all these changes.
UPDATE:
I have done the previously outlined steps but now I get an error.
Here is a snippet of my code:
serv_socket = socket(AF_INET, SOCK_STREAM)
serv_socket.bind(('', serv_port))
serv_socket.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context = context.load_cert_chain('cert.pem', 'key.pem')
context.set_ciphers('EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH')
serv_socket.listen(10)
socket_to_browser, addr = serv_socket.accept()
conn_socket_to_browser = context.wrap_socket(socket_to_browser, server_side=True)
At the last line conn_socket_to_browser = context.wrap_socket(socket_to_browser, server_side=True) an exception is thrown: [SSL: HTTPS_PROXY_REQUEST] https proxy request (_ssl.c:1123)
What am I doing wrong ?
As glamorous as "becoming your own CA" sounds, with openssl it basically comes down to creating a self-signed certificate, and then creating a directory where some CA-specific configuration will be stored (I don't fully remember the specifics, but I think it was just some files related to CNs and serial numbers) so basically reversing the "become your own CA" step is something as mundane as deleting this directory along with the private key and self-signed certificate you were using for the CA. That's it, the CA is no more.
And for chrome returning to the previous state, you would just go the the CA list where you added the CA certificate, select it and delete it. Chrome will stop accepting certificates signed by your CA.
Regarding your new problem... In my opinion, you have developed some kind of reverse proxy (meaning that you expect normal HTTPS requests that you then redirect to the real server) but you have configured Chrome to use it as a forward proxy. In this case, Chrome does not send it a normal HTTPS request, it sends a special non-encrypted CONNECT command and only after receiving the non-encrypted response, it negotiates the TLS connection. That's why openssl says "https proxy request" because it has detected a "https proxy request" (a CONNECT command) instead of the normal TLS negotiation.
You can take a look at How can a Python proxy server (using SSL socket) pretend to be an HTTPS server and specify my own keys to get decrypted data?
It's python, but I think that you'll get the idea
I've always been an end consumer of HTTPS and have never really understood it that well but am looking to change that.
I am calling a RESTful web service over HTTPS. For example...
curl -X GET \
https://myCompanydns/rest/connect/v1.4/myEndpoint
With all my requests I send a basic authentication header i.e a username and password.
When I make these calls via my application I was expecting to have to add a certificate into like a jks (which I've had to do in the past) but on this occasion I've found that I can call the HTTPS web service without that.
For HTTPS to work I believe there is an SSL handshake? How is that happening successfully is this scenario without a jks?
Again, sorry for this beginner type question.
When doing a https://... request the client needs to verify that the servers certificate is the expected one - and not some man in the middle. This is done (among other things) by making sure that the servers certificate was issued by a trusted certificate authority (CA). Which CA is trusted is setup in the local trust store (i.e. local to the client). In the above call where no explicit trust store is given curl is using its default trust store. In the case where you've explicitly gave a jks you've provided the application with a specific trust store it should use.
For more on how the server certificates gets validated see SSL Certificate framework 101: How does the browser actually verify the validity of a given server certificate?.
I've read quite a few articles on the client certificate authentication in IIS but they mostly talk about the Required mode for the client certificate. In this case, the authentication is fully done on the web server side and the user is redirected to the error page in case the certificate is missing or invalid. What I need is to configure IIS to check the client certificate if it exists and pass the results as http headers to my backend. It can be easily done with nginx or apache web servers. The client authentication can be set up as optional. Then, the web server checks the certificate and pass the results of the check as SSL_CLIENT_CERT (PEM representation of the certificate) and SSL_CLIENT_VERIFY (the result of the check - SUCCESS, NONE,...) headers to the application backend. So, on the backend part, I can read the headers and use the values to do the actual authentication -- find a user in the database, issue an auth token. Is it possible in IIS? If yes, is there any documentation on how to bypass the certificate and the check state as http headers?
After a while, I'm writing an answer to my question.
IIS has to be set up with ARR extension to act as a reverse proxy. Then, the client certificate authentication can be enabled for the default web site. Here are several links to instructions how to achieve it:
https://blogs.msdn.microsoft.com/benjaminperkins/2014/06/02/configure-application-request-routing-arr-with-client-certificates/
https://blogs.msdn.microsoft.com/asiatech/2014/01/27/configuring-arr-with-client-certificate/
and couple of links to very solid explanations of the SSL/TLS handshake and certificates in general:
https://blogs.msdn.microsoft.com/kaushal/2013/08/02/ssl-handshake-and-https-bindings-on-iis/
https://blogs.msdn.microsoft.com/kaushal/2013/01/09/self-signed-root-ca-and-intermediate-ca-certificates/
https://blogs.msdn.microsoft.com/kaushal/2015/05/27/client-certificate-authentication-part-1/
Basically, the Application Request Routing (ARR) extension should be enabled for the IIS to act as a proxy. Then, you set up proxy bypass routes. Next, you need to set up a valid server certificate for the IIS server and use it in the Default Web Site https bindings. The certificate should be issued by a CA which certificate should be placed into the Trusted Root Certification Authorities and Intermediate Certification Authorities of the Local Computer. Then, you should require SSL in the SSL settings of the Default Web Site with the client certificates setting equal to Accept. In this case, any client that connects to the web server will be asked for a valid client certificate that has been issued by the same CA as the server certificate.
Actually, IIS sends a list of distinguished names of root issuers that are trusted by the web server to the client browser. The browser finds an intersection of this names with client trusted certificate issuers and looks for valid certificates that have been issued by the issuers in the intersection. After that, the user selects one of them (or none) and the selected certificate is checked against the CA certificate. If the certificate passes the check the request is "redirected" to the backend application with the certificate in the X-ARR-ClientCert header. The name of the header can be changed in the Server -> Configuration Editor -> system.webServer/proxy -> clientCertHeaderName IIS parameter. In case the user selects (or has) none of the required certificates, the request is "redirected" to the app backend without the header.
Seems, there is no need in the SSL_CLIENT_VERIFY header with the state of the check at all. If the certificate is valid it is passed in the header. If the certificate is missing the header is empty. If the certificate is provided by the client but is invalid then the request fails and is not "redirected" to the backend app server at all. Seems, it is a rare case but I have an example.
Imagine, a server certificate is issued by a CA with a distinguished name XXX, and there is a client certificate YYY (on the client computer) that is issued by a CA with the distinguished name XXX but those CAs are not the same (one or both of them are self-signed). In this case, the YYY certificate is present in the certificate selection dialog that is shown by a browser but the certificate doesn't pass the further validation against the real web server CA.
That's it. Also, seems IIS has no way to require (or accept) a client certificate for some app endpoints (addresses) only. I haven't found any other option apart from enabling in for the whole web site.
IIS ARR can also be configured with a client certificate for the backend or upstream server. In this case users connecting to this IIS do not need to provide the certificate as it will be attached by IIS automatically.
This configuration is available on IIS server level only.
I have my website https://www.MyWebSite.com running on port 433. But I also have a admin login that only are available from the office local network http://MyServer:9999/Login.aspx. Both addresses points to the same site but different bindings.
Is it possible to get the one on port 9999 to use https? I tried creating a self signed certificate in IIS but my browser still complained, even though I exported the certificate and stored it in my CA Trusted root.
So just to sum everything:
My regular site: https://MyWebSite.com <-- working fine
My admin login, only accessible via local network: http://MyServer:9999/Login.aspx works fine.
When adding a selfsigned certificate issued to "MyServer" (not MyWebSite) and add the new binding on port 9999 I though to the website but Chrome is giving me a warning NET::ERR_CERT_COMMON_NAME_INVALID, even though the cert is Issued To MyServer and are trusted
Is it possible to get the one on port 9999 to use https?
yes it is possible to setup another port with selfsigned
certificate.
Normally Selfsigned certificate will have fully qualified machine name
e.g. machinename.subdomain.domain so you have to browse using https://machinename.subdomain.domain:9999/
Please double check what error you are running into ,In chrome
Your connection is not private
Attackers might be trying to steal your information from in08706523d (for example, passwords, messages, or credit cards). NET::ERR_CERT_COMMON_NAME_INVALID
in IE,you may get
There is a problem with this website’s security certificate.
The security certificate presented by this website was issued for a different website's address.
Security certificate problems may indicate an attempt to fool you or intercept any data you send to the server.
In that case,assuming you have given hostname as * in IIS binding, and also installed the selfsigned certificate installed your "Root Certification Authorities " You should be able to browse to
https://machinename.subdomain.domain:9999/ without any issues
I have a unique situation where I need to implement client certificate authentication over HTTPS between IE browser and IIS 6. The browser and IIS are separated by a firewall that only allows the browser to connect to IIS on the SSL port.
We have an internal certificate server on the same network as IIS. I've generated an SSL server cert for IIS and that is installed. I configured IIS to only allow SSL, require client certificates.
The limitation here is the browser machine is on a disconnected network, so I can't go to the CA's http://caserver/CertSrv URL and request a client cert like you normally would.
I figured if there were a way that I could generate a CSR against the Root CA's public key, I can copy it to the CA server to generate the client cert. But, there appears to be no provision in IE or the Certificates MMC to do this. The Certificates MMC seems to require a direct connection to the CA.
Has anyone solved this before?
FYI, All servers referenced run Windows Server 2003.
Update: Thanks to Jonas Oberschweiber and Mark Sutton for pointing out the CertReq.exe command line tool. Using this, I've generated a CSR, and consequently a client certificate that installs successfully. However, IE is apparently not sending this client cert when accessing the IIS server in question; it still generates a 403.7 "Forbidden: SSL client certificate is required." I suspect that the reason is that the Subject field of the client cert does not match the user id of the account running IE, thus perhaps not sending a mismatching client cert. The Subject matches that of the user I used to submit the CSR and generate the client cert on the other end of the firewall.
Does the Subject field matter? Is there something else I need to do to enable IE to send this cert?
Use the certreq command on your client as follows
certreq -new -f filein c:\certrequest.req
Here is and example of the filein
[Version]
Signature="$Windows NT$"
[NewRequest]
Subject="CN=dc1.extranet.frbrikam.com"
EncipherOnly = False
Exportable = False
KeyLength = 1024
KeySpec = 1
KeyUsage = 0xA0
MachineKeySet = True
ProviderName = "Microsoft RSA SChannel Cryptographic Provider"
ProviderType = 12
RequestType = CMC
[RequestAttributes]
CertificateTemplate=TLSServer
Replace the CertificateTemplate with the name of your certificate template
Once you have your request file you need to take it to the certificate authority on a usb stick and use the web enrolment interface as usual to process the request file.
Take the output certificate back to the client open it and click install.
You sound like you have already tried a couple of things so my guess is that you are already aware of these, but I'm going to post them anyway, just in case: Certificate Command Line Tools. I am not sure, however, if they do what you want.
Go the http://caserver/CertSrv site that you mentioned using a 3rd computer that can see the CA server. Select the 3rd option, download a CA cert, cert chai, or CRL. On the next page select 'Download CA Certificate Chain', which will download the p7b file. Using a flash drive (or email, etc) transfer this to the other computer which will allow you to import it into the trusted root servers in IE.
http://technet.microsoft.com/en-us/library/cc787796.aspx
Suggestiong for the update, just in case - what is the trusted cert list of in the server?
Subject DN being the same as Windows username has never been a problem for me - although I don't use IIS much. However, somewhere in IIS there is sure to be a trusted certificate list. This error sounds to me like the server's trusted certs list does not include the CA or Root CA that issued the client certificate.
This is particularly true if you never get a certificate selection popup window in IE when you hit the IIS server - even though you have a certificate configured in your IE cert store. That means that the client hit the server, the server gave a list of trusted certs and the client didn't have a cert that fit the list. So the SSL session went to the Forbidden error state.
If the certificate selection window popped up, and you selected and sent the cert, there may be other configuration problems on the server side..