Vaultsharp is not able to authenticate with vault for TLS AUTH method
C# code on windows 10, cert and key in personal store
environment windows
X509Certificate2 clientCertificate = null;
X509Store store = new X509Store(StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
X509Certificate2Collection certificateList =
store.Certificates.Find(X509FindType.FindBySubjectName, "subject name", false);
if (certificateList.Count > 0)
{
clientCertificate = certificateList[0];
};
store.Close();
// got clientCertificate here, it has private key as well
try
{
IAuthMethodInfo authMethod = new CertAuthMethodInfo(clientCertificate);
var vaultClientSettings = new VaultClientSettings("endpoint:8200", authMethod);
IVaultClient vaultClient = new VaultClient(vaultClientSettings);
Secret<Dictionary<string, object>> secret = null;
Task.Run(async () =>
{
secret = await vaultClient.V1.Secrets.KeyValue.V1.ReadSecretAsync("dummy_app/dev/connection_strings");
}).GetAwaiter().GetResult();
Above code is throwing error
{"errors":["client certificate must be supplied"]}
It should return the secret instead of throwing exception
Please check the following.
That the certificate really has a private key. (HasPrivateKey check on the object) Typically you read a private key from a store using a passphrase. I don't see that above, so it maybe that what you have is a public key.
Please ensure that the certificate is a valid cert with the full chain. The Vault API (not VaultSharp) throws an error if it cannot find the parent chain.
Please inspect the http or tcp connection to see if the cert is truly attached.
Related
I am trying to implement the mTLS in cluster across micro service for secured communication. I know that there are service meshes are available for this purpose. But we would like to stay away from service mesh and implement the mTLS in cluster.
So, after going through several posts, then I am able to create the tls secret and mount the volume as part of the service deployment. This certificate i can retrieve from X509Store:
using var certificateStore = new X509Store(StoreName.Root, StoreLocation.LocalMachine, OpenFlags.ReadOnly);
if (certificateStore.Certificates.Any())
{
var certs = certificateStore.Certificates.Find(X509FindType.FindByIssuerName, issuerName, true);
if (certs.Any())
{
return certs.First();
}
}
return null;
But, now, when i am trying to assign this certificate as part of the
kestrelServerOptions.ConfigureHttpsDefaults(listenOptions =>
{
Log.Information($"Configuring the https defaults.");
if (serverCertificate == null)
{
return;
}
// self signed certificate
Log.Information($"Before : Private key: {serverCertificate?.HasPrivateKey}");
Log.Information($"After : Server certificate: {listenOptions.ServerCertificate?.Issuer}");
listenOptions.ServerCertificate = serverCertificate; // throws exception saying that the serer certificate should have the private key.
....
my secret volume has both .crt(pem) and .key files stored as part of the tls secret. But service is not able to attach this private .key to it.
I am really lost here... and not able to proceed further.
I really appreciate if someone help me to work with this certificate and mTLS.
Thanks in advance.
I was not able to make http client code in .net 5 to send both intermediate and leaf certificates (in 3 certificate hierarchy) to the server. However I was able to send the leaf certificate from client to the server successfully. Here is my setup:
I have 3 certificates on my windows box:
TestRoot.pem
TestIntermediate.pem
TestLeaf.pem (without private key for server - windows box)
TestLeaf.pfx (with private key for client - windows box)
The none of the above certificates were NOT added to windows certificate manager as I would like to be able to run the same code on non-windows machines eventually. For my testing, I am running following client and server code on the same windows box.
On my windows box, I have following simple client side code using .net 5:
HttpClientHandler handler = new HttpClientHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.SslProtocols = System.Security.Authentication.SslProtocols.Tls12;
X509Certificate2 leafCert = new X509Certificate2(File.ReadAllBytes(#"C:\Temp\TestLeaf.pfx"), "<password>");
handler.ClientCertificates.Add(leafCert);
HttpClient httpClient = new HttpClient(handler);
StringContent content = new StringContent("{}"); //Test json string
content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(MediaTypeNames.Application.Json);
//With local.TestServer.com resolving to localhost in the host file
HttpResponseMessage response = httpClient.PostAsync("https://local.TestServer.com/...", content).Result;
if (response.IsSuccessStatusCode)
{
var responseString = response.Content.ReadAsStringAsync().Result;
Console.WriteLine(responseString);
}
else
{
Console.WriteLine(x.StatusCode);
Console.WriteLine(x.ReasonPhrase);
}
On same window box, I have following example snippet of server side code using kestrel in .net 5:
services.Configure<KestrelServerOptions>(options =>
{
// Keep track of what certs belong to each port
var certsGroupedByPort = ...;
var certsPerDistinctSslPortMap = ...;
// Listen to each distinct ssl port a cert specifies
foreach (var certsPerDistinctSslPort in certsPerDistinctSslPortMap)
{
options.Listen(IPAddress.Any, certsPerDistinctSslPort.Key, listenOptions =>
{
var httpsConnectionAdapterOptions = new HttpsConnectionAdapterOptions();
httpsConnectionAdapterOptions.ClientCertificateValidation = (clientCertificate, chain, sslPolicyErrors) =>
{
bool trusted = false;
if (sslPolicyErrors == System.Net.Security.SslPolicyErrors.RemoteCertificateChainErrors)
{
chain.ChainPolicy.TrustMode = X509ChainTrustMode.CustomRootTrust;
X509Certificate2 certRoot = new X509Certificate2(#"C:\Temp\TestRoot.pem");
X509Certificate2 certIntermdiate = new X509Certificate2(#"C:\Temp\TestIntermediate.pem");
chain.ChainPolicy.CustomTrustStore.Add(certRoot);
chain.ChainPolicy.ExtraStore.Add(certIntermdiate);
trusted = chain.Build(clientCertificate);
}
return trusted;
};
httpsConnectionAdapterOptions.ServerCertificateSelector = (connectionContext, sniName) =>
{
var defaultCert = //Get default cert
return defaultCert;
};
httpsConnectionAdapterOptions.ClientCertificateMode = ClientCertificateMode.RequireCertificate;
httpsConnectionAdapterOptions.SslProtocols = SslProtocols.Tls12;
listenOptions.UseHttps(httpsConnectionAdapterOptions);
});
}
options.Listen(IPAddress.Any, listeningPort);
});
The above code works as expected because the client code sends the leaf certificate to the server and the server code has access to both intermediate as well as root certificates. The server code can successfully rebuild the certificate hierarchy with received leaf certificate and its configured intermediate and root certs for the leaf certificate.
My following attempt to send the intermediate certificate (along with leaf certificate) to the server (so that it can only use the root certificate and incoming leaf and intermediate certificates in the request to build the certificate hierarchy) failed.
Tried to add the intermediate certificate by doing following in my client code:
X509Certificate2 leafCert = new X509Certificate2(File.ReadAllBytes(#"C:\Temp\TestLeaf.pfx"), "");
X509Certificate2(Convert.FromBase64String(File.ReadAllText(#"C:\Temp\TestIntermediate.pem"));
handler.ClientCertificates.Add(leafCert);
handler.ClientCertificates.Add(intermediateCert);
This did not send the intermediate certificate to the server. I verified this with the code block for httpsConnectionAdapterOptions.ClientCertificateValidation on the server side.
Question:
Is there a way to ensure that intermediate certificate is sent by the client (in addition to the leaf cert) to the server?
I have a grpc-js server using self signed ssl certificates.
var credentials = grpc.ServerCredentials.createSsl(
fs.readFileSync('./node/grpc/ssl/ca.crt'),
[{
cert_chain: fs.readFileSync('./node/grpc/ssl/server.crt'),
private_key: fs.readFileSync('./node/grpc/ssl/server.key')
}],
true
);
I then tested this setup with a grpc-js client with the following credential setup and this works.
var credentials = grpc.credentials.createSsl(
fs.readFileSync('./node/grpc/ssl/ca.crt'),
fs.readFileSync('./node/grpc/ssl/client.key'),
fs.readFileSync('./node/grpc/ssl/client.crt')
);
I want to replicate this in Android using OkHttpChannelBuilder but it is a bit more complicated. This is what I have so far.
private val mChannel : ManagedChannel
init {
/**
* Server certificate to make it trusted.
*/
val serverCrtFile = applicationContext.resources.openRawResource(R.raw.server)
val serverCertificate: X509Certificate =
CertificateFactory.getInstance("X.509").generateCertificate(serverCrtFile) as X509Certificate
val caKeyStore: KeyStore = KeyStore.getInstance(KeyStore.getDefaultType()).apply {
load(null, null)
setCertificateEntry("server", serverCertificate)
}
val trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm()).apply {
init(caKeyStore)
}
val sslContext = SSLContext.getInstance("TLS").apply {
init(null, trustManagerFactory.trustManagers, null)
}
mChannel = OkHttpChannelBuilder
.forAddress(BuildConfig.GRPC_HOST_ADDRESS, BuildConfig.GRPC_HOST_PORT)
.sslSocketFactory(sslContext.socketFactory)
.keepAliveTime(10, TimeUnit.SECONDS)
.useTransportSecurity()
.keepAliveWithoutCalls(true)
.build()
}
Everything worked before implementing ssl (so using plaintext() on the channel builder).
The error I get now is io.grpc.StatusRuntimeException: UNAVAILABLE: End of stream or IOException.
Can someone please tell me if I am doing something wrong and how I can get a successful connection like between the js server and client.
Looks like the SSL handshake failed on the server side so it will be helpful to get server side detailed logs to see what went wrong.
One possibility is using KeyStore.getInstance. Can you try using "PKCS12"?
KeyStore.getInstance("PKCS12")
Extra information at the end after comments from Crypt32 (thank you Crypt32!)
I have to send data to a server. I need mutual authentication: the server needs to be certain it is me, and I need to be certain that the server really is the server that I trust. This needs to be dons in a windows program.
To identify itself, the server will send me a certificate that is issued by certificate authorities that I trust: a root certificate and an intermediate certificate:
Root CA-G2.PEM
Intermediate CA-G2.PEM
To identify me, the organization gave me a certificate and a private key
Root CA-G3.PEM
Intermediate CA-G3.PEM
MyCertificate.CRT (= pem) and MyCertificate.Private.Key (=RSA)
I have imported all root certificates and intermediate certificates into the windows keystore.
To sent the message:
const string url = "https://...//Deliver";
HttpWebRequest webRequest = WebRequest.CreateHttp(url);
// Security:
webRequest.AuthenticationLevel=AuthenticationLevel.MutualAuthRequired;
webRequest.Credentials = CredentialCache.DefaultCredentials;
// Should I add my certificate?
X509Certificate myCertificate = new X509Certificate("MyCertificate.CRT");
// Should I add Certificate authorities?
// only the CA-G2 authorities, so my WebRequest can trust the certificate
// that will be sent by the Server?
// or Should I also add the CA-G3 who issued MyCertificate
// and what about MyCertificate.Private.Key, the RSA file?
// Fill the rest of the WebRequest:
webRequest.Method = "Post";
webRequest.Accept = "text/xml";
webRequest.Headers.Add("SOAP:Action");
webRequest.ContentType = "text/xml;charset=\"utf-8\"";
... etc
// do the call and display the result
using (WebResponse response = webRequest.GetResponse())
{
using (var reader = new StreamReader(response.GetResponseStream()))
{
string soapResult = reader.ReadToEnd();
Console.WriteLine(soapResult);
}
}
The WebResponse doesn't indicate any error. The returned data is an empty (non-null) string. Yet:
response.StatusCode == NoContent (204)
soapResult == String.Empty
response.IsMutuallyAuthenticated == false
The NoContent and the empty data result are expected. Does the false IsMutuallyAuthenticated indicate that something is wrong with my authentication?
Added information
Crypt32 suggested I should convert MyCertificate.CRT and MyCertificate.Private.Key into one PFX (or P12) file.
For this I use openssl.
I concatenated the CA-G3 files into one TrustG3.Pem and created the P12 file:
openssl.exe pkcs12 -export -name "<some friendly name>"
-certfile TrustG3.Pem
-in MyCertificate.CRT
-inkey MyCertificate.Private.Key
-out MyCertificate.P12
After providing a password a proper Pkcs12 file (PFX) was created. The source code changes slightly:
HttpWebRequest webRequest = WebRequest.CreateHttp(url);
// Security:
webRequest.AuthenticationLevel=AuthenticationLevel.MutualAuthRequired;
webRequest.Credentials = CredentialCache.DefaultCredentials;
var p12Certificate = new X509Certificate("MyCertificate.P12", "my password");
webRequest.ClientCertificates.Add(p12Certificate);
Alas, this didn't help. The webResponse still says:
response.IsMutuallyAuthenticated == false
Does the false IsMutuallyAuthenticated indicate that something is wrong with my authentication?
yes, it does. Because you add only public part of client certificate. There is no associated private key specified. Either, use certificate from certificate store (assuming, cert store contains private key) or import certificate from PFX.
Update:
now your client authentication code looks correct. Next step is to check if your client certificate is trusted by server.
I have a specific problem that I can't solve. Let me explain in detail. I'm new to this technology so I might be using some wrong terms. Please correct and explain or ask for explanation if you don't understand.
I am creating a self hosted WCF REST server, hosted in WPF application. It uses https, SLL with WebHttpSecurityMode.Transport. I am using my own generated certificate.
I would like to create a WinForms client that would use this service. The format of the response form the server is JSON.
I would like to validate the certificate on the client with my custom validator inherited from X509CertificateValidator.
This is my server side code. I'm using a custom username validator that works fine. I have configured the certificate in the IIS Manager on my machine for the Default Website > Bindings, where I have generated the certificate (Windows 7).
WebServiceHost sh = new WebServiceHost(typeof(ReachService));
string uri = "https://localhost:9000/Service";
WebHttpBinding wb = new WebHttpBinding();
wb.Security.Mode = WebHttpSecurityMode.Transport;
wb.Security.Transport.ClientCredentialType = HttpClientCredentialType.Basic;
sh.AddServiceEndpoint(typeof(IReachService), wb, uri);
sh.Credentials.UserNameAuthentication.CustomUserNamePasswordValidator = new CustomUserNameValidator();
sh.Credentials.UserNameAuthentication.UserNamePasswordValidationMode = UserNamePasswordValidationMode.Custom;
sh.Open();
and this is my client code
Uri uri = new Uri("https://localhost:9000/Service");
WebChannelFactory<ReachService> cf = new WebChannelFactory<IReachService>(uri);
WebHttpBinding wb = cf.Endpoint.Binding as WebHttpBinding;
wb.Security.Transport.ClientCredentialType = HttpClientCredentialType.Basic;
wb.Security.Mode = WebHttpSecurityMode.Transport;
cf.Credentials.ServiceCertificate.Authentication.CertificateValidationMode = X509CertificateValidationMode.Custom;
cf.Credentials.ServiceCertificate.Authentication.CustomCertificateValidator = new CustomCertificateValidator("PL2"); // this is the name that issued the certificate
cf.Credentials.UserName.UserName = "user1";
cf.Credentials.UserName.Password = "user1";
IReachService service = cf.CreateChannel();
try
{
CustomersList auth = service.GetCustomers();
}
catch (Exception ex)
{
throw new Exception(ex.Message);
}
on calling service.GetCustomers() I get:
Could not establish trust relationship for the SSL/TLS secure channel with authority
'localhost:9000'.
InnerException Message:
The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
InnerException Message:
The remote certificate is invalid according to the validation procedure.
The server is working fine when I test in the browser.
But the client code is wrong cause it doesn't go to the custom cert validator class. And this class is the same as in the MSDN example on http://msdn.microsoft.com/en-us/library/system.identitymodel.selectors.x509certificatevalidator.aspx.
Can anyone please tell me where am I going wrong with this approach?
If you need more info please ask.
Thank you
It looks like the issue occurs because certificate was issued for some other hostname. You can check this (and customize if necessary) by providing custom ServicePointManager.ServerCertificateValidationCallback.
//don't use HttpWebRequest --you lose all of the strongly-typed method and data contracts!
//the code to create the channel and call a method:
SetCertPolicy();
var cf1 = new WebChannelFactory<TService>(new Uri(remoteServiceAddressSecure));
var service = cf1.CreateChannel();
sevice.DoMethod();
protected static void SetCertPolicy()
{
ServicePointManager.ServerCertificateValidationCallback += RemoteCertValidate;
}
private static bool RemoteCertValidate(object sender, X509Certificate cert, X509Chain chain,
SslPolicyErrors error)
{
// trust any cert!!!
return true;
}
If you want to use WCF on the client, then don't use WebHttpBinding, stick with the SOAP stuff it will work much better.
However, if you want to use a standard HTTP client like, WebClient or HttpWebRequest or HttpClient V.prototype or HttpClient V.Next then stick with the webHttpBinding.
Sorry for not addressing your direct question but you are likely to run into more problems because you are using a binding that was intended to make WCF services accessible to non-WCF platforms but then using WCF to try and access it.