Service Fabric certificate swap. Apps failing to activate - ssl

We have 5 Service Fabric nodes running 2 applications in 4 environments in Azure. Our Network team wanted to switch out our cluster certificate so they generated a new one (I believe in the key vault) and swapped it to primary. We updated our project to use the new cert's thumbprint. We successfully authenticated to the cluster and deployed (via Visual Studio) using the new cert, but now the application will not activate. The error we are seeing is:
System.Hosting' reported Error for property 'Activation:1.0:1.0:131965678558388988'.
There was an error during activation.There is already a certificate with thumbprint 123oldCertNumber bound to port 200appPort. New certificate thumbprint specified: 321newCertNumber
Additionally, we tried deleting the old cert which is now in the secondary slot, but it just processes for hours saying "Cluster is updating user certificate." and eventually fails to delete the cert.
Any help would be greatly appreciated!

Here's what worked for me:
I deleted all applications, but did not unprovision them. I then reset all nodes to clear them out. Then I deleted the old cert - this time it worked. I redeployed and viola, it activated no problem. Well, almost, I have one node that is stuck with the same error message. I've tried deleting data and resetting it, but haven't been able to clear it yet.

If both of you applications were using the old certificate then you may have encountered a problem described in documentation describing Upgrading multiple applications with HTTPS endpoints.
When the first application goes to update it will fail to configure the HTTPS port with the new certificate, since the second application is still running and has already configured the HTTPS port with the old certificate. The only path forward is to remove both applications that are sharing the port and then upgrade.
For this reason you may want to consider approaches to prevent this problem in the future. You could:
Combine services from the two applications into a single application, or
Run each application on a different port.

I just finished upgrading my applications to use a new cert and here is what you must do.
I have 3 applications using the one cert.
Instructions
In the Application Manifest, I removed the binding
from 2 of my 3 applications. It is important that you do not remove the binding from all of your apps at once.
I then redeployed the 2 apps with the binding removed.
I updated the 3rd Application with the new cert and redeployed
i then added back the binding in the other 2 applications and updated them to use the new cert and redeployed.
That was all that was needed.
Here is a link to the solution
Renew endpoint certificate

Related

Lucee SSL Certificate being dropped

An installed SSL certificate (via Lucee server admin > Services > SSL Certificates) keeps getting lost. We enter the target host name click LIST, see the cert and click INSTALL.
It's all fine for a few weeks the suddenly it just disappears (and our 2 way SSL automation dies). We have no idea why it suddenly just goes though, the cert is valid for another year and there doesn't appear to be a way of manually removing the certs so we are at a loss.
We also don't know where that install process actually stores the cert (in a file, Database, memory) in order to try and monitor it a bit better.
Whats confusing me even more is this is happening across 3 servers (Win 2012).
Each time this happens all we've done is reinstall the cert to Lucee (its already on the cert store on the windows box & the java store) and that seems to resolve it.
Any ideas on why it's happening and how to stop it from happening again?

RavenDB "Connection Was Reset" - After Enabling SSL

I am having some trouble with RavenDB after enabling SSL. I have followed the instructions on the RavenDB documentation for enabling SSL which can be found here.
Namely, I updated the Raver.Server.exe.config file to include the new setting and I also installed the cert using /installSSL on Raven.Server.exe.
If I disable SSL - everything works fine, but once I enable SSL I get a connection was reset message in the browser.
I have enabled the Raven logs with the Nlog.config file and also checked the event viewer for logs. Neither of these two have any information. The RavenDB service is running, but it just won't connect when I go to the studio. The problem is beyond the studio because I can't get our APIs to connect to it either.
I have been searching around for a couple of days, but I can't find anything, anywhere.
Any ideas on how to fix this?
It appears that this error occurs when the SSL cert has not been installed correctly into the Windows Cert Store. After inspecting the cert store, I found that the cert was not installed correctly into the Personal collection.
It might be important to note that I disabled SSL on RavenDB and then fixed the Cert Store.
After installing the cert, I re-enabled SSL on RavenDB and everything was working fine again.

Using ssl with localhost with asp.net mvc 5 on VS2015

I'm trying to test my website locally using SSL with IIS Express. It has the following properties set:
SSL Enabled set to 'true'
SSL URL is set https://localhost:44354/
But whenever I open the https address, I get the following error:
In "Microsoft Edge":
In Google Chrome:
I've read article after articles, including some on SO but to no avail. I've tried the following:
I've deleted my IIS Express Development Certificate
I've repaired IIS Express 10 via the Control Panel
I've removed the localhost certificate I had created manually.
I've added <binding protocol="https" bindingInformation="*:44354:localhost" /> to the bindings section the applicationhost.config
I've stopped and restarted IIS Express.
Some suggest to change the port to 443 but my SSL URL is read-only in .NET IDE.
Some articles I've read:
Running IIS Express without Administrative Privileges
How do I fix a missing IIS Express SSL Certificate?
IIS Express — Getting SSL to Work
and many more...
Any help would be greatly appreciated.
Thanks.
I finally figured it out by following this article How to trust the IIS Express Self-Signed Certificate but a few additional steps:
Export IIS Express Development certificate to a local from. This certificate can be found in the Server Certificates section in IIS 10.0.
Open the Certificate console by calling mmc.exe certmgr.msc from File|Run.
Delete all instances of localhost under the Trusted Root Certification Authorities|Certificates.
Import the newly created certificate. You will get prompted with the following:
Once imported, I went back to Asp.Net MVC 5 project, recompile it and ran it. When I ran it, I got prompted with the following:
This is when I knew I was on the right track as this was the first time I had ever seen this prompt! Click Yes, and now this prompt appears:
And click Yes on this prompt as well. Your project will then launch the relevant browser.
Go to the https address defined in your .net project, in my instance, https://localhost:44354/, and you will now see the padlock displayed in the address bar to indicate that it is a secure site:
Most of these answers were already available in different answers provided on SO but the points that were missing or that I missed were that I had to export my IIS Express Development certificate, delete all localhost entries (which I had done) and then re-import this certificate. Once done, .NET detects the change and you get prompted accordingly.
Anyway, I hope this will help others.
Self signed certificates need to be trusted or browsers won't accept them. You can easily use Jexus Manager to configure that,
https://www.jexusmanager.com/en/latest/tutorials/self-signed.html#to-trust-self-signed-certificate
While if you prefer manually, you can import the certificates to the Trusted Root Certificate Authority store in Windows.
Learn more about SSL, certificates, stores and so on (Google each of them and learn them thoroughly), so that next time you really understand what is the culprit before trying so many irrelevant things.
Jexus Manager also has an SSLDiag feature to identify potential issues,
https://www.jexusmanager.com/en/latest/tutorials/ssl-diagnostics.html
But you need to know enough so as to interpret its output correctly.

SSL with WCF not working! How to Debug

I am dabbling with WCF and SSL and have hit a bit of a problem.
I have a self-hosted WCF service and a basic client which connects to request a token.
I have it set up on my development machine it appears to communicating over SSL just fine.
But when I set it up on a different machine to test the deployment the client keeps getting nothing back from the service.
I assume it has something to do with the certificates I have set up as this is the only thing I think could be different.
I have used the netsh command to associate my port with an existing certificate that was already on the server. And it appears the same as my dev machine (where I created the certificate with makecert manually.
I initially tried to follow the same steps on the deployment server but failed as there was already a certificate with a common name of the server, so eve though the certificate generated when ever I tried to use the cert hash thumbprint of the newly generated certificate I was being given the :
SSL Certificate add failed, Error: 1312
A specified logon session does not exist...
The only way I could get the netsh http add to work was by specifying the cert hash of the existing certificate.
Is this my problem or a red herring? How do you debug issues like this?
Personally I just configure WCF tracing. It usually provides more user friendly info about a problem. Here's a how to link Configuring Tracing
Hope it helps!

Cannot get Azure WCF service to work with Client Certificates

I have a WCF service that I want to secure with Client Certificates but I cannot get it to work on Azure.
I removed Azure from the equation by hosting the service on a standard Windows Server on Amazon. I installed both the service and client certificates (none are self-signed) into the Local Machine 'Personal' store on this server including the chained certificates and it all worked as expected, called from my local PC, with the client cert set against the binding/behavior. It did not work without the certificate being specified so it definitely worked correctly this way.
I then deployed the service to Azure. The client and server certificates are uploaded to the portal and set in the config against "Local Machine/My" and the CA and root certificates are uploaded and I tried them in various stores including "My", "Trusted" and "CA". Every variation I try, I continue to get "The HTTP request was forbidden with client authentication scheme 'Anonymous'" called from exactly the same program locally with the only change being the client endpoint address.
As another detail, I can get it to work without certificates so there is no problem with the web service but I am unsure how to work out what is actually happening with the certificate handshake between client and service.
I have finally got it to work, and have written a guide here: Blogspot.co.uk
I'm not sure what I had got wrong before since I have not done anything too weird to make it work. I think perhaps I had a small defect somewhere in configuration that I eventually fixed by starting again. Anyway, it DOES work and provides some useful security on Azure.
See my answer to this SO post - bottom line, cert in LocalMachine/My and run with elevated privileges in csdef file add:
<Runtime executionContext="elevated" />