Kestrel's web server is timing out, saying Connection Closed, after loading a publicly-signed SSL Certificate.
Background - we have a docker container that hosts a dotnet 3.1 webapi/react app, where the user can upload a custom SSL certificate. The PKCS#12 certificate is stored in our database and bound at startup using .ConfigureKestrel((context,options)) and options.ConfigureHttpsDefaults(listenOptions=>{listenOptions.ServerCertificate = certFromDatabase; }). This has been working flawless.
However, the problem now is that a user is attempting to run this app in a restrictive firewalled environment and is receiving HTTP connection closed errors when attempting to access Kestrel immediately after loading a new certificate and restarting the app.
Whenever Kestrel receives an incoming request, it begins attempting to download the intermediate certificate from the certificate's CA's public CDN repository via http on port 80. It appears to be using the URL from the Authority Information Access portion of the certificate. Since the firewall is blocking this, it retries repeatedly for about 20 seconds, during which time the client's TLS handshake sits waiting on a server response. When the server eventually fails to fetch the intermediate certificate, it cancels the TLS handshake and closes the connection.
I can't figure out why it's attempting to download this certificate, considering the same certificate is embedded in the PKCS#12 PFX bundle that is bound to Kestrel. Am I supposed to load either the root CA or intermediate into the CA trust folder in file system? (Ex. /usr/local/share/ca-certificates/ - I can't load the intermediate there, only the CA?)
public static IWebHost BuildFullWebHost(string[] args)
{
var webHostBuilder = GetBaseWebHostBuilder(args);
return webHostBuilder
.ConfigureAppConfiguration((context, builder) => { [...] })
.ConfigureLogging((hostingContext, logging) => { [...] })
.UseStartup<Startup>()
.ConfigureKestrel((context, options) =>
{
var sp = options.ApplicationServices;
using (var scope = sp.CreateScope())
{
var dbContext = scope.ServiceProvider.GetService<DbContext>();
var cert = Example.Services.HttpsCertificateService.GetHttpsCert(dbContext);
//this returns a new X509Certificate2(certificate.HttpsCertificate, certificate.Password);
options.ConfigureHttpsDefaults(listenOptions =>
{
listenOptions.ServerCertificate = cert;
listenOptions.CheckCertificateRevocation = false;
});
}
})
.Build();
}
Not a great solution, but upgrading to .NET 5.0 resolved the issue. It seems that in .NET 5.0, Kestrel attempts to fetch the certificate chain during initial application startup only (and fails). Subsequent incoming HTTP requests don't trigger the fetch process and requests are served as expected.
Related
I am currently working on a prototype for a WCF service that will make use of client-certificate authentication. We would like to be able to directly publish our application to IIS, but also allow SSL offloading using IIS ARR (Application Request Routing).
After digging through the documentation, I have been able to test both configurations successfully. I am able to retrieve the client certificate used to authenticate from:
X-Arr-ClientCert - the header that contains the certificate when using ARR.
X509CertificateClaimSet - when published directly to IIS, this is how to retrieve the Client Certificate
To verify that the request is allowed, I match the thumbprint of the certificate to the expected thumbprint that is configured somewhere. To my surprise, when getting the certificate through different methods, the same certificate has different thumbprints.
To verify what is going on, I have converted the "RawData" property on both certificates to Base64 and found that it's the same, except that in the case of the X509CertificateClaimSet, there are spaces in the certificate data, while in the case of ARR, there are not. Otherwise, both strings are the same:
My question:
Has anyone else run into this, and can I actually rely on thumbprints? If not, my backup plan is to implement a check on Subject and Issuer, but I am open to other suggestions.
I have included some (simplified) sample code below:
string expectedThumbprint = "...";
if (OperationContext.Current.ServiceSecurityContext == null ||
OperationContext.Current.ServiceSecurityContext.AuthorizationContext.ClaimSets == null ||
OperationContext.Current.ServiceSecurityContext.AuthorizationContext.ClaimSets.Count <= 0)
{
// Claimsets not found, assume that we are reverse proxied by ARR (Application Request Routing). If this is the case, we expect the certificate to be in the X-ARR-CLIENTCERT header
IncomingWebRequestContext request = WebOperationContext.Current.IncomingRequest;
string certBase64 = request.Headers["X-Arr-ClientCert"];
if (certBase64 == null) return false;
byte[] bytes = Convert.FromBase64String(certBase64);
var cert = new System.Security.Cryptography.X509Certificates.X509Certificate2(bytes);
return cert.Thumbprint == expectedThumbprint;
}
// In this case, we are directly published to IIS with Certificate authentication.
else
{
bool correctCertificateFound = false;
foreach (var claimSet in OperationContext.Current.ServiceSecurityContext.AuthorizationContext.ClaimSets)
{
if (!(claimSet is X509CertificateClaimSet)) continue;
var cert = ((X509CertificateClaimSet)claimSet).X509Certificate;
// Match certificate thumbprint to expected value
if (cert.Thumbprint == expectedThumbprint)
{
correctCertificateFound = true;
break;
}
}
return correctCertificateFound;
}
Not sure what your exact scenario is, but I've always liked the Octopus Deploy approach to secure server <-> tentacle (client) communication. It is described in their Octopus Tentacle communication article. They essentially use the SslStream class, self-signed X.509 certificates and trusted thumbprints configured on both server and tentacle.
-Marco-
When setting up the test again for a peer review by colleagues, it appears that my issue has gone away. I'm not sure if I made a mistake (probably) or if rebooting my machine helped, but in any case, the Thumbprint now is reliable over both methods of authentication.
I have a Java application that connects to webMethods IS via SSL.
public static QueueConnection createSSLEnabledQueueConnectionToWebmethods(Context context, String username, String password, Properties props, String factoryName) throws Exception
{
String pathToKeyStore = props.getProperty("keystore.path");
String pathToTrustStore = props.getProperty("truststore.path");
WmConnectionFactoryImpl factory = (WmConnectionFactoryImpl)
context.lookup(factoryName);
((WmConnectionFactoryImpl)factory).setSSLKeystore(pathToKeyStore);
((WmConnectionFactoryImpl)factory).setSSLTruststore(pathToTrustStore);
((WmConnectionFactoryImpl)factory).setSSLEncrypted(true);
return ((WmConnectionFactoryImpl)factory).createQueueConnection(username, password);
}
I have a keyStore.p12 file and a TrustStore.jks file that have been working for years in our Production environment (and still work with previous application builds).
In our Test environment, with the new application build, I can use the above keyStore and TrustStore files to establish SSL connection with webMethods. however, in the Production environment, the exact application installation results in the below error:
javax.jms.JMSSecurityException: [BRM.10.5061] JMS: SSL certificate "keystore.p12": bad certificate.
at com.webmethods.jms.protocol.link.LinkSsl.createSslContext(LinkSsl.java:377)
at com.webmethods.jms.protocol.link.LinkSsl.connect(LinkSsl.java:112)
at com.webmethods.jms.protocol.ProtocolHandler.connect(ProtocolHandler.java:218)
at com.webmethods.jms.protocol.BinaryProtocolHandler.connect(BinaryProtocolHandler.java:1950)
at com.webmethods.jms.impl.WmConnectionImpl.connect(WmConnectionImpl.java:302)
at com.webmethods.jms.impl.WmConnectionImpl.initConnection(WmConnectionImpl.java:280)
at com.webmethods.jms.impl.WmConnectionImpl.(WmConnectionImpl.java:219)
at com.webmethods.jms.impl.WmConnectionImpl.(WmConnectionImpl.java:193)
at com.webmethods.jms.impl.WmQueueConnectionImpl.(WmQueueConnectionImpl.java:44)
at com.webmethods.jms.impl.WmConnectionFactoryImpl.createQueueConnection(WmConnectionFactoryImpl.java:328)
Given the same application build and same key/truststore files (both configured to connect to PROD webMethods), I have the following test results:
In the TEST environment:
Full application start-up with SSL connection => SUCCESSFUL
Test harness performing SSL connection only => SUCCESSFUL
In the PRODUCTION environment:
Full application start-up with SSL connection => FAILURE >>> bad certificate
Test harness performing SSL connection only => SUCCESSFUL
I am wondering if there is anything in the JDK that could be causing the inconsistent behaviour in the PRODUCTION environment?
Or is there any useful knowledge out there around the very vague "bad certificate" error?
I have a WebApi controller action that I decorated with my [x509Authorize] attribute. I'm debugging this endpoint locally - and at the same time running a console application that tries to call this endpoint.
Client side
Here's the client code - slightly simplified:
X509Certificate Cert = X509Certificate.CreateFromCertFile("C:\\Temp\\ht-android-client.pfx");
HttpWebRequest Request = (HttpWebRequest)WebRequest.Create("https://localhost:44300/api/mobile/predict");
Request.ClientCertificates.Add(Cert);
HttpWebResponse Response = (HttpWebResponse)Request.GetResponse();
....
I've asserted that the Cert is the correct certificate. I've installed the .pfx in my CurrentUser\Personal store and in the LocalMachine\Personal store - and modified to take the Cert from that store, as suggested here but that doesn't seem to make a difference:
var store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
store.Open(OpenFlags.OpenExistingOnly | OpenFlags.ReadOnly);
var Cert = store.Certificates.Find(X509FindType.FindBySubjectName, "Android", true)[0];
Server side
And I'm listening on the WebAPI endpoint like with the following code:
public class x509AuthorizeAttribute : AuthorizeAttribute
{
public override Task OnAuthorizationAsync(HttpActionContext actionContext, CancellationToken cancellationToken)
{
var cert = actionContext.Request.GetClientCertificate();
// value of 'cert' is null
I hit a breakpoint in the console app first - see that the correct certificate is selected. Then I hit the breakpoint on the server and see that the value of .GetClientCertificate() is null. What am I doing wrong? The other SO questions 1 and 2 didn't help me any further.
Additional information on the certificates
I've created a self-signed CA certificate which is installed on the LocalMachine\Trusted root CA store. I've created the android client cert - and signed it with my self-signed CA certificate. Then I converted that into a pkcs12 file. This is the certificate that the client is using - which is also installed in my personal stores ( both machine and currentUser ) and is valid ( you can see the chain go back to the ROOT CA cert ).
Also - the certificate's purpose is set to clientAuth:
So the problem is indeed that the server needs to have the following set in the web.config in order to force IIS to start the SSL cert negotiation:
<security>
<access sslFlags="SslNegotiateCert" />
</security>
If this is not present - the certificate will be ignored and you will get null on the GetClientCertificate() call.
This implies however that all clients for my WebAPI are now forced to present a valid certificate - so my original idea of having just one controller method requiring a certificate does not seem possible.
Then there's the challenge of setting this config paramter in web.config, because of the restrictions for Azure Cloud Services. However - this answer provides a solution for that.
EDIT
On a side note this is not supported yet in ASP.NET vNext ( v rc-01-final )
Hi I am having a Restful service (DotNet 4.0 WCF VS 2012) in HTTPS. My client will attach a certificate to it (certificate is given by me (.cer file)) I need to get the certificate back from the request and read its information to authenticate it, The serial Number, Thumprint are stored in DB when I need to check the same.
I did the SSL and Share the cer file to the client.
I used the following code to read back my certificate
C# code start
if (OperationContext.Current.ServiceSecurityContext.AuthorizationContext.ClaimSets == null)
throw new Exception ("No claimset service configured wrong");
if (OperationContext.Current.ServiceSecurityContext.AuthorizationContext.ClaimSets.Count <= 0)
throw new Exception ("No claimset service configured wrong");
var cert = ((X509CertificateClaimSet)OperationContext.Current.ServiceSecurityContext.
AuthorizationContext.ClaimSets[0]).X509Certificate;
C# code end
in the above code i always gets claimSets.Count = 0.
Is any setting I need to do in my server web.config, I did the following setting in my Server Side web.config
'security mode="Transport"'
'transport clientCredentialType="None"'
'security'
Please let me know Is I am missing any settings in the client side or the server side.
In the client side I am using following code the add the cer to the request
C# Code Start
X509Certificate2 cert = new X509Certificate2 ("C:\\xxxxxx.cer");
System.Net.ServicePointManager.ServerCertificateValidationCallback =
delegate(Object obj, X509Certificate X509certificate, X509Chain chain, System.Net.Security.SslPolicyErrors errors)
{
return true;
};
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(xxxxx.Text.Trim());
webRequest.ClientCertificates.Add(cert);
C# Code End
I did not have any special setting in my client web.config file.
Why you use clientCredentialType="None" and not clientCredentialType="Certificate"?
It is also possible your client does not send any certificate. Try to enable Network Tracing in App.config on the client - instructions here. That should create network.log with more debug info. Look for SecureChannel entries in log.
Tried my luck at research, but so far no joy.
I would like to connect a SignalR javascript client to a self-hosted SignalR Windows Service binding to a self-signed SSL certificate.
My application works quite well over http, but the client repetitively disconnects when the Owin WebApplication starts using https.
Here is what I've done to configure SignalR with SSL.
Created a Self-Signed certificate using IIS
Imported the certificate into the Trusted Root Certification Authorities in the mmc (not sure if that helped)
Ran NETSH command to bind SSL to port 8080
netsh http add sslcert ipport=0.0.0.0:8080 certhash=123456f6790a35f4b017b55d09e28f7ebe001bd appid={12345678-db90-4b66-8b01-88f7af2e36bf}
Added code in self-hosted HubConnection instances to add exported SSL like this (though this shouldn't matter because it's the client that cannot connect):
if (File.Exists("MyCert.cer")
&& Settings.GetSetting(Settings.Setting.SrProtocol).Equals("https", StringComparison.InvariantCultureIgnoreCase))
connection.AddClientCertificate(X509Certificate.CreateFromCertFile("MyCert.cer"));
Starting Owin WebApplication using https (this should create the binding in http.sys)
string registerUrl = string.Format("{0}://SOME.WHERE.COM:{1}", Service.Server.SrProtocol, Service.Server.SrPort);
WebApp.Start<StartUp>(registerUrl);
In the SignalR 2.0 documentation, it says:
To start the web server, call WebApplication.Start(endpoint). You should now be able to navigate to endpoint/signalr/hubs in your browser.
When I browse to the URL http://SOME.WHERE.COM:8080/signalr/hubs I am successful receiving the javascript that drives SignalR.
When I browse to the URL https://SOME.WHERE.COM:8080/signalr/hubs I am unsuccessful and I receive "The connection to the server was reset" using FF.
Some additional points I've considered:
NETSH SHOW indicates the url is registered
URL group ID: E300000240000022
State: Active
Request queue name: Request queue is unnamed.
Properties:
Max bandwidth: inherited
Max connections: inherited
Timeouts:
Timeout values inherited
Number of registered URLs: 1
Registered URLs: HTTPS://SOME.WHERE.COM:8080/
NETSH SHOW indicates the SSL certificate is bound to 8080:
IP:port : 0.0.0.0:8080
Certificate Hash : 123456f6790a35f4b017b55d09e28f7ebe001bd
Application ID : {12345678-db90-4b66-8b01-88f7af2e36bf}
Certificate Store Name : (null)
Verify Client Certificate Revocation : Enabled
Verify Revocation Using Cached Client Certificate Only : Disabled
Usage Check : Enabled
Revocation Freshness Time : 0
URL Retrieval Timeout : 0
Ctl Identifier : (null)
Ctl Store Name : (null)
DS Mapper Usage : Disabled
Negotiate Client Certificate : Disabled
Any help is greatly appreciated!
I believe its all working for me now. Here is a run down of the steps I took to get things flowing:
SSL NOTES
SSL & SignalR (Owin WebApplication) requires binding a certificate to a port.
Use IIS to generate an self-signed cert, this should place the certificate into the LOCAL COMPUTER > Personal > Certificates folder in CERTMGR
In CERTMGR shift+drag certificate to LOCAL COMPUTER > Trusted Root Certification Authorities > Certificates folder, which should make a copy of it there
Run the following command to bind the SSL certificate to 0.0.0.0:8080
netsh http add sslcert ipport=0.0.0.0:8080 certhash=123456f6790a35f4b017b55d09e28f7ebe001bd appid={12345678-db90-4b66-8b01-88f7af2e36bf}
netsh http show urlacl > D:\urlacl.txt
Output:
Reserved URL : https://*:8080/
User: SOMEWHERE\Administrator
Listen: Yes
Delegate: No
SDDL: D:(A;;GX;;;S-1-5-21-138209071-46972887-2260295844-1106)
Run the following NETSH command to reserve all IP addresses for port 8080 to the My Service application ID and service account
netsh http add urlacl url=https://*:8080/ user=SOMEWHERE\Administrator listen=yes
netsh http show sslcert > D:\sslcert.txt
Output:
IP:port : 0.0.0.0:8080
Certificate Hash : 123456f6790a35f4b017b55d09e28f7ebe001bd
Application ID : {12345678-db90-4b66-8b01-88f7af2e36bf}
Certificate Store Name : (null)
Verify Client Certificate Revocation : Enabled
Verify Revocation Using Cached Client Certificate Only : Disabled
Usage Check : Enabled
Revocation Freshness Time : 0
URL Retrieval Timeout : 0
Ctl Identifier : (null)
Ctl Store Name : (null)
DS Mapper Usage : Disabled
Negotiate Client Certificate : Disabled
Update the MyServices.exe.config file to use https protocol (These are appSetting keys used to dynamically set the protocol and port of SignalR when My Service starts)
<add key="SrProtocol" value="https" />
<add key="SrPort" value="8080" />
Start the My Service using the NETSTAT START command
Run the following NETSH command to show the service state is occupying the registered url
netsh http show servicestate > D:\servicestate.txt
Output:
Server session ID: C300000320000039
Version: 2.0
State: Active
Properties:
Max bandwidth: 4294967295
Timeouts:
Entity body timeout (secs): 120
Drain entity body timeout (secs): 120
Request queue timeout (secs): 120
Idle connection timeout (secs): 120
Header wait timeout (secs): 120
Minimum send rate (bytes/sec): 150
URL groups:
URL group ID: C600000340000138
State: Active
Request queue name: Request queue is unnamed.
Properties:
Max bandwidth: inherited
Max connections: inherited
Timeouts:
Timeout values inherited
Number of registered URLs: 1
Registered URLs:
HTTPS://*:8080/
My application does NOT depend on IIS, but once I used IIS to temporarily create a port binding to my SSL certificate, my application started to work, and I was able to inspect the NETSH servicestate to see how IIS does it. I have since dropped the IIS binding and ran through the setup notes, and still have success.
My Owing startup looks somethign like this:
private void configureMessaging()
{
string registerUrl = string.Format("{0}://*:{1}", Service.Server.SrProtocol, Service.Server.SrPort);
try
{
#if DEBUG
//System.Diagnostics.Debugger.Launch();
#endif
// Starts an owin web application to host SignalR, using the protocol and port defined.
WebApp.Start<StartUp>(registerUrl);
}
catch (Exception ex)
{
Logger.Logs.Log(string.Format("Failed to configure messaging. Exception: {0}", ex.RecurseInnerException()), LogType.Error);
if (ex is HttpListenerException || ex.InnerException is HttpListenerException)
{
try
{
Process p = new Process();
p.StartInfo.UseShellExecute = false;
p.StartInfo.RedirectStandardOutput = true;
p.StartInfo.FileName = "netsh.exe";
p.StartInfo.Arguments = string.Format("netsh http delete urlacl url={0}"
, registerUrl
);
p.Start();
p.StandardOutput.ReadToEnd();
p.WaitForExit();
}
catch (Exception exP)
{
Logger.Logs.Log(string.Format("Failed to delete urlacl {0}. Exception: {1}"
, registerUrl
, exP.RecurseInnerException()
)
, LogType.Error
)
;
retries = 5;
}
}
if (retries < 5)
{
retries++;
Logger.Logs.Log(string.Format("Attempting to configure messaging again. Attempt No. {0}", retries), LogType.Warn);
Thread.Sleep(1000);
configureMessaging();
}
else
Logger.Logs.Log(string.Format("Exceeded total number of retries to configure messaging.", retries), LogType.Error);
}
}
And self-hosted HubConnetion instances look like this:
public IHubProxy MyHubProxy
{
get
{
if (this._MyHubProxy == null)
{
var connection = new HubConnection(string.Format("{0}://{1}:{2}/"
, Settings.GetSetting(Settings.Setting.SrProtocol)
, MyHub.GetLocalhostFqdn(null)
, Settings.GetSetting(Settings.Setting.SrPort)
)
)
;
this._MyHubProxy = connection.CreateHubProxy("MyHub");
if (File.Exists("My.cer")
&& Settings.GetSetting(Settings.Setting.SrProtocol).Equals("https", StringComparison.InvariantCultureIgnoreCase))
connection.AddClientCertificate(X509Certificate.CreateFromCertFile("My.cer"));
connection.Start().Wait();
}
return this._MyHubProxy;
}
}
There is a little more code here than relevant, but hopefully it may be of help!