I'm trying unsuccessfully to get a basic GRPC server and client working with SSL/TLS, with a node client and Java server. Starting with no security:
// client.js
const creds = grpc.credentials.createInsecure()
const stub = new hello_proto.Greeter('localhost:50051', creds)
stub.sayHello(...)
// server.java
Server server = ServerBuilder.forPort(50051)
.addService(serviceImplementation)
.build();
server.start();
All works as expected here. I then tried to add SSL credentials, generating a certificate and private key like this (following a Python example):
$ openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 365 -out server.crt
This generates a certificate (server.crt) and private key (server.key). I then add those credentials to client and server (private key on server only), following the guidance from the grpc.io Auth Guide and grpc-java respectively:
// client.js
const rootCert = fs.readFileSync("path/to/server.crt");
const channelCreds = grpc.credentials.createSsl(rootCert);
const stub = new hello_proto.Greeter('localhost:50051', channelCreds);
stub.sayHello(...)
// server.java
File certChainFile = File("path/to/server.crt")
File privateKeyFile = File("path/to/server.key")
Server server = ServerBuilder.forPort(50051)
.useTransportSecurity(certChainFile, privateKeyFile)
.addService(serviceImplementation)
.build();
server.start();
Now I get an error UNAVAILABLE: No connection established on the client side:
Error: 14 UNAVAILABLE: No connection established
at Object.callErrorFromStatus (path/to/node_modules/#grpc/grpc-js/build/src/call.js:31:26)
at Object.onReceiveStatus (path/to/node_modules/#grpc/grpc-js/build/src/client.js:176:52)
at Object.onReceiveStatus (path/to/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:336:141)
at Object.onReceiveStatus (path/to/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:299:181)
at path/to/node_modules/#grpc/grpc-js/build/src/call-stream.js:130:78
at processTicksAndRejections (node:internal/process/task_queues:76:11) {
code: 14,
details: 'No connection established',
metadata: Metadata { internalRepr: Map(0) {}, options: {} }
}
No error on the server side. The client-side error is, unhelpfully, identical to the one I get when the server is down.
How do I implement basic TLS authentication between a Java server and node client?
Maybe you can reference my code in helloworlde/grpc-java-sample, feel free to translate Chinese;
For the both side, it need SslContext
Server
File keyCertChainFile = new File("server.pem");
File keyFile = new File("server.key");
SslContext sslContext = GrpcSslContexts.forServer(keyCertChainFile, keyFile)
.clientAuth(ClientAuth.OPTIONAL)
.build();
Server server = NettyServerBuilder.forAddress(new InetSocketAddress(9090))
.addService(new HelloServiceImpl())
.sslContext(sslContext)
.build();
Client
File trustCertCollectionFile = new File("server.pem");
SslContext sslContext = GrpcSslContexts.forClient()
.trustManager(trustCertCollectionFile)
.build();
ManagedChannel channel = NettyChannelBuilder.forAddress("127.0.0.1", 9090)
.overrideAuthority("localhost")
.sslContext(sslContext)
.build();
Related
I was not able to make http client code in .net 5 to send both intermediate and leaf certificates (in 3 certificate hierarchy) to the server. However I was able to send the leaf certificate from client to the server successfully. Here is my setup:
I have 3 certificates on my windows box:
TestRoot.pem
TestIntermediate.pem
TestLeaf.pem (without private key for server - windows box)
TestLeaf.pfx (with private key for client - windows box)
The none of the above certificates were NOT added to windows certificate manager as I would like to be able to run the same code on non-windows machines eventually. For my testing, I am running following client and server code on the same windows box.
On my windows box, I have following simple client side code using .net 5:
HttpClientHandler handler = new HttpClientHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.SslProtocols = System.Security.Authentication.SslProtocols.Tls12;
X509Certificate2 leafCert = new X509Certificate2(File.ReadAllBytes(#"C:\Temp\TestLeaf.pfx"), "<password>");
handler.ClientCertificates.Add(leafCert);
HttpClient httpClient = new HttpClient(handler);
StringContent content = new StringContent("{}"); //Test json string
content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(MediaTypeNames.Application.Json);
//With local.TestServer.com resolving to localhost in the host file
HttpResponseMessage response = httpClient.PostAsync("https://local.TestServer.com/...", content).Result;
if (response.IsSuccessStatusCode)
{
var responseString = response.Content.ReadAsStringAsync().Result;
Console.WriteLine(responseString);
}
else
{
Console.WriteLine(x.StatusCode);
Console.WriteLine(x.ReasonPhrase);
}
On same window box, I have following example snippet of server side code using kestrel in .net 5:
services.Configure<KestrelServerOptions>(options =>
{
// Keep track of what certs belong to each port
var certsGroupedByPort = ...;
var certsPerDistinctSslPortMap = ...;
// Listen to each distinct ssl port a cert specifies
foreach (var certsPerDistinctSslPort in certsPerDistinctSslPortMap)
{
options.Listen(IPAddress.Any, certsPerDistinctSslPort.Key, listenOptions =>
{
var httpsConnectionAdapterOptions = new HttpsConnectionAdapterOptions();
httpsConnectionAdapterOptions.ClientCertificateValidation = (clientCertificate, chain, sslPolicyErrors) =>
{
bool trusted = false;
if (sslPolicyErrors == System.Net.Security.SslPolicyErrors.RemoteCertificateChainErrors)
{
chain.ChainPolicy.TrustMode = X509ChainTrustMode.CustomRootTrust;
X509Certificate2 certRoot = new X509Certificate2(#"C:\Temp\TestRoot.pem");
X509Certificate2 certIntermdiate = new X509Certificate2(#"C:\Temp\TestIntermediate.pem");
chain.ChainPolicy.CustomTrustStore.Add(certRoot);
chain.ChainPolicy.ExtraStore.Add(certIntermdiate);
trusted = chain.Build(clientCertificate);
}
return trusted;
};
httpsConnectionAdapterOptions.ServerCertificateSelector = (connectionContext, sniName) =>
{
var defaultCert = //Get default cert
return defaultCert;
};
httpsConnectionAdapterOptions.ClientCertificateMode = ClientCertificateMode.RequireCertificate;
httpsConnectionAdapterOptions.SslProtocols = SslProtocols.Tls12;
listenOptions.UseHttps(httpsConnectionAdapterOptions);
});
}
options.Listen(IPAddress.Any, listeningPort);
});
The above code works as expected because the client code sends the leaf certificate to the server and the server code has access to both intermediate as well as root certificates. The server code can successfully rebuild the certificate hierarchy with received leaf certificate and its configured intermediate and root certs for the leaf certificate.
My following attempt to send the intermediate certificate (along with leaf certificate) to the server (so that it can only use the root certificate and incoming leaf and intermediate certificates in the request to build the certificate hierarchy) failed.
Tried to add the intermediate certificate by doing following in my client code:
X509Certificate2 leafCert = new X509Certificate2(File.ReadAllBytes(#"C:\Temp\TestLeaf.pfx"), "");
X509Certificate2(Convert.FromBase64String(File.ReadAllText(#"C:\Temp\TestIntermediate.pem"));
handler.ClientCertificates.Add(leafCert);
handler.ClientCertificates.Add(intermediateCert);
This did not send the intermediate certificate to the server. I verified this with the code block for httpsConnectionAdapterOptions.ClientCertificateValidation on the server side.
Question:
Is there a way to ensure that intermediate certificate is sent by the client (in addition to the leaf cert) to the server?
I am working on a code that connects to slack through a proxy which act as a MITM and replaces slack cert with its own self signed cert. I added proxy's cert into a trust store and configured my RestTemplate to use the trust store:
def sslContext = new SslContextBuilder().withTrustStore(trustStoreResource, trustStorePassword).build()
def proxy = proxyEnabled ? new HttpHost(proxyHost, proxyPort) : null
def httpClient = HttpClients.custom().setProxy(proxy).setSSLContext(sslContext).build()
def result = new RestTemplate(new HttpComponentsClientHttpRequestFactory(httpClient))
That works fine. However, on my local I don't go through the proxy and connect to slack directly. In other words, the httpClient in the above code would be configured with SSLContext but not proxy. I was expecting this to be fine since Slack's cert is signed with a valid root CA but my code fails to verify Slack's cert.
I am assuming this is because my trustore but I am confused as why this is happening. Is it happening because root CAs are not imported in my trustsore? If so, how would I do that without having to maintain the root CAs?
I understand that locally I can refrain from setting up a trust store but I would like to avoid adding branches in the code if possible.
What I finally ended up doing was to use the implementation in https://gist.github.com/JensRantil/9b7fecb3647ecf1e3076 to combine system's default trust store with mine and then used the following class to build my SSL context. It's a shame HttpClient doesn't offer this but there might be a good reason for it.
import org.springframework.core.io.Resource
import javax.net.ssl.KeyManager
import javax.net.ssl.KeyManagerFactory
import javax.net.ssl.SSLContext
import javax.net.ssl.TrustManager
import java.security.KeyStore
class SslContextBuilder {
private KeyManager[] keyManagers = []
private TrustManager[] trustManagers = []
SslContextBuilder withKeyStore(Resource resource, String password) {
def keyStore = KeyStore.getInstance('JKS')
keyStore.load(resource.getInputStream(), password.chars)
KeyManagerFactory kmfactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm())
kmfactory.init(keyStore, password.chars)
KeyManager[] kms = kmfactory.getKeyManagers()
keyManagers += kms ? kms : []
this
}
SslContextBuilder withTrustStore(Resource resource, String password) {
def trustStore = KeyStore.getInstance('JKS')
trustStore.load(resource.getInputStream(), password.chars)
def tss = CompositeX509TrustManager.getTrustManagers(trustStore)
trustManagers += tss ? tss : []
this
}
SSLContext build() {
def sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManagers, trustManagers, null)
sslContext
}
}
I have logstash-6.5.4 (with ssl), web and scheduler in my local (hostname: webbox) and kafka-2.0 (with ssl) on another (hostname: kafkabox).
I am not able to receive message in kafka topic when message is sent from logstash.
Neither error message is displayed not message is sent to kafka topic. I tried to import logstash.crt into kafka's truststore but it also didn't worked.
Created logstash.crt and logstash.key with below command.
sudo openssl req -x509 -batch -nodes -days 3650 -newkey rsa:2048 -keyout /etc/logstash/logstash.key -out /etc/logstash/logstash.crt
Imported the logstash.crt into kafka's truststore file also and tried.
keytool -import -alias logstash -file logstash.crt -keystore cacerts
Logstash conf file is given below...
input {
tcp {
host=>"0.0.0.0"
port=>5514
type=>"syslogType"
ssl_enable=>true
ssl_cert=>"/etc/logstash/logstash.crt"
ssl_key=>"/etc/logstash/logstash.key"
ssl_verify=>false
}
}
filter {
}
output {
kafka {
bootstrap_servers=>"kafkabox:9093"
codec=>"json_lines"
topic_id=>"a_test"
ssl_keystore_location=>"keystore file"
ssl_keystore_password=>"changeit"
ssl_key_password=>"changeit"
ssl_truststore_location=>"truststore file"
ssl_truststore_password=>"changeit"
security_protocol=>"SSL"
}
}
Expecting message is sent from logstash (with SSL) to kafka (with SSL).
Java Code to connect to logstash which internally failing to send message to kafka topics (in ssl mode).
public class LogstashClient {
private static String message = "<86>Jun 25 14:32:25 webbox sshd[7517]: Failed password for root from 196.165.132.192 port 45691 ssh2";
public static void main(String[] args) throws Exception {
nonSSL();
//SSL();
}
private static void SSL() throws Exception {
// logstash.crt is directly imported into kafka's truststore
// Below <<Client Truststore>> will also have logstash.crt imported for handshaking while connecting
System.setProperty("javax.net.ssl.trustStore", "<<Client Truststore>>");
System.setProperty("javax.net.ssl.trustStorePassword", "test1234");
SSLSocketFactory factory = (SSLSocketFactory)SSLSocketFactory.getDefault();
SSLSocket socket = (SSLSocket) factory.createSocket("localhost", 5514);
System.out.println("Handshaking...");
socket.startHandshake();
PrintWriter printWriter = new PrintWriter(socket.getOutputStream(), true);
boolean checkError = printWriter.checkError();
printWriter.println(message);
}
private static void nonSSL() throws Exception {
Socket socket = new Socket("localhost", 5514);
PrintWriter printWriter = new PrintWriter(socket.getOutputStream(), true);
printWriter.println(message);
}
}
Thanks,
RK,
Vaultsharp is not able to authenticate with vault for TLS AUTH method
C# code on windows 10, cert and key in personal store
environment windows
X509Certificate2 clientCertificate = null;
X509Store store = new X509Store(StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
X509Certificate2Collection certificateList =
store.Certificates.Find(X509FindType.FindBySubjectName, "subject name", false);
if (certificateList.Count > 0)
{
clientCertificate = certificateList[0];
};
store.Close();
// got clientCertificate here, it has private key as well
try
{
IAuthMethodInfo authMethod = new CertAuthMethodInfo(clientCertificate);
var vaultClientSettings = new VaultClientSettings("endpoint:8200", authMethod);
IVaultClient vaultClient = new VaultClient(vaultClientSettings);
Secret<Dictionary<string, object>> secret = null;
Task.Run(async () =>
{
secret = await vaultClient.V1.Secrets.KeyValue.V1.ReadSecretAsync("dummy_app/dev/connection_strings");
}).GetAwaiter().GetResult();
Above code is throwing error
{"errors":["client certificate must be supplied"]}
It should return the secret instead of throwing exception
Please check the following.
That the certificate really has a private key. (HasPrivateKey check on the object) Typically you read a private key from a store using a passphrase. I don't see that above, so it maybe that what you have is a public key.
Please ensure that the certificate is a valid cert with the full chain. The Vault API (not VaultSharp) throws an error if it cannot find the parent chain.
Please inspect the http or tcp connection to see if the cert is truly attached.
I have some misunderstanding with running SF on local cluster with SSL, on localhost.
Microsoft created greate article about configuring HTTPS on your endpoints But it works well only if you use their certificate generator CertSetup.ps1 . If you try install your own pfx, it will not work.
First I created localhost self-signed cert by OpenSSL:
set OPENSSL_CONF=W:\OpenSSL-Win32\bin\openssl.cfg
openssl genrsa -out W:\CERTS\wepapissl.key -passout pass:1234567890 -aes256 2048
openssl req -x509 -new -key W:\CERTS\wepapissl.key -days 10000 -out W:\CERTS\wepapissl.crt -passin pass:1234567890 -subj /CN="localhost"
openssl pkcs12 -export -inkey W:\CERTS\wepapissl.key -in W:\CERTS\wepapissl.crt -out W:\CERTS\wepapissl.pfx -passout pass:0987654321 -passin pass:1234567890`
Second I have created default ASP.NET Core Web Application (Core 2.0, API template). And added code for configure Kestrel to use HTTPS:
public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseKestrel(opt =>
{
opt.Listen(IPAddress.Any, port, listenOptions =>
{
listenOptions.UseHttps(GetCertificateFromStore());
});
})
.UseStartup<Startup>()
.Build();
private static X509Certificate2 GetCertificateFromStore()
{
var store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
try
{
store.Open(OpenFlags.ReadOnly);
var certCollection = store.Certificates;
var currentCerts = certCollection.Find(X509FindType.FindBySubjectDistinguishedName, "CN=localhost", false);
return currentCerts.Count == 0 ? null : currentCerts[0];
}
finally
{
store.Close();
}
}
I have got expected result. Page with warning about website’s security certificate:
Result from ValueController with warning
Third I have created Service Fabric Application (Stateless ASP.NET Core template). Change my ServiceManifest.xml by editing Endpoint section:
<Endpoint Protocol="https" Name="ServiceEndpoint" Type="Input" Port="8256" />
And added code for configure Kestrel to use HTTPS (class Web1 : StatelessService):
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
return new ServiceInstanceListener[]
{
new ServiceInstanceListener(serviceContext =>
new KestrelCommunicationListener(serviceContext, "ServiceEndpoint", (url, listener) =>
{
ServiceEventSource.Current.ServiceMessage(serviceContext, $"Starting Kestrel on {url}");
return new WebHostBuilder()
.UseKestrel(opt =>
{
int port = serviceContext.CodePackageActivationContext.GetEndpoint("ServiceEndpoint").Port;
opt.Listen(IPAddress.IPv6Any, port, listenOptions =>
{
listenOptions.UseHttps(this.GetCertificateFromStore());
});
})
.ConfigureServices(
services => services
.AddSingleton<StatelessServiceContext>(serviceContext))
.UseContentRoot(Directory.GetCurrentDirectory())
.UseStartup<Startup>()
.UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
.UseUrls(url)
.Build();
}))
};
}
private X509Certificate2 GetCertificateFromStore()
{
var store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
try
{
store.Open(OpenFlags.ReadOnly);
var certCollection = store.Certificates;
var currentCerts = certCollection.Find(X509FindType.FindBySubjectDistinguishedName, "CN=localhost", false);
return currentCerts.Count == 0 ? null : currentCerts[0];
}
finally
{
store.Close();
}
}
Result: Successful build and deploy code on local SF cluster. But my resource can't be reached
P.S. I will repeat again, if you install new cert by using PowerShell provided by Mircosoft - CertSetup.ps1, it works well for SF application. I was trying to dig in PS script, but I can not understand what I missed.
P.P.S I am new in creating certificates, but it seems strange.
I have installed pfx by CertSetup.ps1. All works well (resource is reachable).
Then I have exported cert to pfx with private key and all extended properties
Delete from LocalMachine (MY and Root), CurrentUser (MY) stores
Install exported pfx to LocalMachine (My and Root), CurrentUser (My) stores
Rebuild & Redeploy code
Resoucre can not be reached
Is it magic? Or I miss something?
Couple details was not enough clear for me, any way. Answer:
If you tried to use your own generated certificate (openssl, makecert or etc), you shoud set privileges for NETWORK SERVICE.
To manually do this on your dev box, open up certlm.msc, expand Personal->Certificates, and right-click your cert. Select All Tasks->Manage private keys and then add NETWORK SERVICE.
More here: https://github.com/Azure/service-fabric-issues/issues/714#issuecomment-381281701