I have some misunderstanding with running SF on local cluster with SSL, on localhost.
Microsoft created greate article about configuring HTTPS on your endpoints But it works well only if you use their certificate generator CertSetup.ps1 . If you try install your own pfx, it will not work.
First I created localhost self-signed cert by OpenSSL:
set OPENSSL_CONF=W:\OpenSSL-Win32\bin\openssl.cfg
openssl genrsa -out W:\CERTS\wepapissl.key -passout pass:1234567890 -aes256 2048
openssl req -x509 -new -key W:\CERTS\wepapissl.key -days 10000 -out W:\CERTS\wepapissl.crt -passin pass:1234567890 -subj /CN="localhost"
openssl pkcs12 -export -inkey W:\CERTS\wepapissl.key -in W:\CERTS\wepapissl.crt -out W:\CERTS\wepapissl.pfx -passout pass:0987654321 -passin pass:1234567890`
Second I have created default ASP.NET Core Web Application (Core 2.0, API template). And added code for configure Kestrel to use HTTPS:
public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseKestrel(opt =>
{
opt.Listen(IPAddress.Any, port, listenOptions =>
{
listenOptions.UseHttps(GetCertificateFromStore());
});
})
.UseStartup<Startup>()
.Build();
private static X509Certificate2 GetCertificateFromStore()
{
var store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
try
{
store.Open(OpenFlags.ReadOnly);
var certCollection = store.Certificates;
var currentCerts = certCollection.Find(X509FindType.FindBySubjectDistinguishedName, "CN=localhost", false);
return currentCerts.Count == 0 ? null : currentCerts[0];
}
finally
{
store.Close();
}
}
I have got expected result. Page with warning about website’s security certificate:
Result from ValueController with warning
Third I have created Service Fabric Application (Stateless ASP.NET Core template). Change my ServiceManifest.xml by editing Endpoint section:
<Endpoint Protocol="https" Name="ServiceEndpoint" Type="Input" Port="8256" />
And added code for configure Kestrel to use HTTPS (class Web1 : StatelessService):
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
return new ServiceInstanceListener[]
{
new ServiceInstanceListener(serviceContext =>
new KestrelCommunicationListener(serviceContext, "ServiceEndpoint", (url, listener) =>
{
ServiceEventSource.Current.ServiceMessage(serviceContext, $"Starting Kestrel on {url}");
return new WebHostBuilder()
.UseKestrel(opt =>
{
int port = serviceContext.CodePackageActivationContext.GetEndpoint("ServiceEndpoint").Port;
opt.Listen(IPAddress.IPv6Any, port, listenOptions =>
{
listenOptions.UseHttps(this.GetCertificateFromStore());
});
})
.ConfigureServices(
services => services
.AddSingleton<StatelessServiceContext>(serviceContext))
.UseContentRoot(Directory.GetCurrentDirectory())
.UseStartup<Startup>()
.UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
.UseUrls(url)
.Build();
}))
};
}
private X509Certificate2 GetCertificateFromStore()
{
var store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
try
{
store.Open(OpenFlags.ReadOnly);
var certCollection = store.Certificates;
var currentCerts = certCollection.Find(X509FindType.FindBySubjectDistinguishedName, "CN=localhost", false);
return currentCerts.Count == 0 ? null : currentCerts[0];
}
finally
{
store.Close();
}
}
Result: Successful build and deploy code on local SF cluster. But my resource can't be reached
P.S. I will repeat again, if you install new cert by using PowerShell provided by Mircosoft - CertSetup.ps1, it works well for SF application. I was trying to dig in PS script, but I can not understand what I missed.
P.P.S I am new in creating certificates, but it seems strange.
I have installed pfx by CertSetup.ps1. All works well (resource is reachable).
Then I have exported cert to pfx with private key and all extended properties
Delete from LocalMachine (MY and Root), CurrentUser (MY) stores
Install exported pfx to LocalMachine (My and Root), CurrentUser (My) stores
Rebuild & Redeploy code
Resoucre can not be reached
Is it magic? Or I miss something?
Couple details was not enough clear for me, any way. Answer:
If you tried to use your own generated certificate (openssl, makecert or etc), you shoud set privileges for NETWORK SERVICE.
To manually do this on your dev box, open up certlm.msc, expand Personal->Certificates, and right-click your cert. Select All Tasks->Manage private keys and then add NETWORK SERVICE.
More here: https://github.com/Azure/service-fabric-issues/issues/714#issuecomment-381281701
Related
I am trying to run the application using IP address instead of localhost on https like https://192.1638.2.1:5001/index.html.
What I have done:
I have created a self-signed certificate using below power shell code.
New-SelfSignedCertificate -CertStoreLocation "cert:\LocalMachine\My" -dnsname "192.168.2.1" -NotAfter (Get-Date).AddYears(10) -FriendlyName "SS_192_168_2_1" -KeyUsageProperty All -KeyUsage CertSign, CRLSign, DigitalSignature
Thumbprint Subject
---------- -------
ABE394F15852C9389655F3EBC111FCE624D43479 CN=192.168.2.1
$mypwd = ConvertTo-SecureString -String "SS123" -Force -AsPlainText
Get-ChildItem -Path cert:\localMachine\my\ABE394F15852C9389655F3EBC111FCE624D43479 | Export-PfxCertificate -FilePath "D:\Certificates\SS_192_168_2_1.pfx" -Password $mypwd
Added SS_192_168_2_1.pfx to certificate store in Trusted Root Certificate Authority of Current User. While adding this a popup came asking for password setup and have entered a new password and got saved warning popup.
Configured .NET core web API app to use this file as server certificate.
public class Program
{
public static void Main(string[] args)
{
CreateHostBuilder(args).Build().Run();
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
webBuilder.ConfigureKestrel(options =>
{
options.ConfigureEndpointDefaults(listenOptions =>
{
// Loads the certificate (a must-have)
listenOptions.UseHttps(#"D:\Certificates\SS_192_168_2_1.pfx", "SS123");
});
options.ConfigureHttpsDefaults(o =>
{
o.ClientCertificateMode = ClientCertificateMode.NoCertificate;
});
});
});
}
Issue:
Still the browser shows Not Secure and post man shows as Self signed certificate whereas localhost is showing Secure. What is that I am missing here??
I'm trying unsuccessfully to get a basic GRPC server and client working with SSL/TLS, with a node client and Java server. Starting with no security:
// client.js
const creds = grpc.credentials.createInsecure()
const stub = new hello_proto.Greeter('localhost:50051', creds)
stub.sayHello(...)
// server.java
Server server = ServerBuilder.forPort(50051)
.addService(serviceImplementation)
.build();
server.start();
All works as expected here. I then tried to add SSL credentials, generating a certificate and private key like this (following a Python example):
$ openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 365 -out server.crt
This generates a certificate (server.crt) and private key (server.key). I then add those credentials to client and server (private key on server only), following the guidance from the grpc.io Auth Guide and grpc-java respectively:
// client.js
const rootCert = fs.readFileSync("path/to/server.crt");
const channelCreds = grpc.credentials.createSsl(rootCert);
const stub = new hello_proto.Greeter('localhost:50051', channelCreds);
stub.sayHello(...)
// server.java
File certChainFile = File("path/to/server.crt")
File privateKeyFile = File("path/to/server.key")
Server server = ServerBuilder.forPort(50051)
.useTransportSecurity(certChainFile, privateKeyFile)
.addService(serviceImplementation)
.build();
server.start();
Now I get an error UNAVAILABLE: No connection established on the client side:
Error: 14 UNAVAILABLE: No connection established
at Object.callErrorFromStatus (path/to/node_modules/#grpc/grpc-js/build/src/call.js:31:26)
at Object.onReceiveStatus (path/to/node_modules/#grpc/grpc-js/build/src/client.js:176:52)
at Object.onReceiveStatus (path/to/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:336:141)
at Object.onReceiveStatus (path/to/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:299:181)
at path/to/node_modules/#grpc/grpc-js/build/src/call-stream.js:130:78
at processTicksAndRejections (node:internal/process/task_queues:76:11) {
code: 14,
details: 'No connection established',
metadata: Metadata { internalRepr: Map(0) {}, options: {} }
}
No error on the server side. The client-side error is, unhelpfully, identical to the one I get when the server is down.
How do I implement basic TLS authentication between a Java server and node client?
Maybe you can reference my code in helloworlde/grpc-java-sample, feel free to translate Chinese;
For the both side, it need SslContext
Server
File keyCertChainFile = new File("server.pem");
File keyFile = new File("server.key");
SslContext sslContext = GrpcSslContexts.forServer(keyCertChainFile, keyFile)
.clientAuth(ClientAuth.OPTIONAL)
.build();
Server server = NettyServerBuilder.forAddress(new InetSocketAddress(9090))
.addService(new HelloServiceImpl())
.sslContext(sslContext)
.build();
Client
File trustCertCollectionFile = new File("server.pem");
SslContext sslContext = GrpcSslContexts.forClient()
.trustManager(trustCertCollectionFile)
.build();
ManagedChannel channel = NettyChannelBuilder.forAddress("127.0.0.1", 9090)
.overrideAuthority("localhost")
.sslContext(sslContext)
.build();
In this link :
https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/linux-nginx?view=aspnetcore-5.0#data-protection
it says "If data protection isn't configured, the keys are held in memory and discarded when the app restarts.", and I don't want that to happen so I configured the data protection in a startup.cs :
services.AddDataProtection()
.PersistKeysToFileSystem(new DirectoryInfo(#"PATH-HERE"))
and when I started the app to test it, a warning shows up in the logs saying: No XML encryptor configured. Key {GUID} may be persisted to storage in unencrypted form..
I have found out that I need to use ProtectKeysWith* to encrypt the Key. but because I'm trying to publish the app to a Linux server, I cant use ProtectKeysWithDpapi or ProtectKeysWithDpapiNG ( because they can only be used on Windows servers ), so the only option left was X.509.
basically, I did some searching, and I found out I can use these commands to create a self-signed X.509 certificate :
"C:\Program Files\Git\usr\bin\openssl.exe" genrsa -out private.key 2048
"C:\Program Files\Git\usr\bin\openssl.exe" req -new -x509 -key private.key -out publickey.cer -days 2000
"C:\Program Files\Git\usr\bin\openssl.exe" pkcs12 -export -out idp.pfx -inkey private.key -in publickey.cer
and I can add this certificate in the startup like this :
services
.AddDataProtection()
.PersistKeysToFileSystem(new DirectoryInfo(#"PATH-TO-SAVE-KEYS"))
.SetDefaultKeyLifetime(new TimeSpan(90, 0, 0, 0, 0))
.SetApplicationName("APPNAME-HERE")
.ProtectKeysWithCertificate(new X509Certificate2(#"CERTIFICATE-PATH", "CERTIFICATE-PASSWORD"));
So my question is do I even need to encrypt the keys? and if I should, is my solution valid? can I use this solution in production without any problem? ( keep in mind that I'm going to use a Linux server for my app )
Update 1:
I did more digging in the StackOverflow questions and I have found this :
https://stackoverflow.com/a/48867984/14951696.
apparently using a self-signed certificate ( like what I was doing ) will be fine as long as you are using it internally. I will update again after I have published my app in case anyone has the same question.
Update 2:
I have decided to use Windows servers, and I have found no problem using the self-signed certificate to encrypt the keys. if anything happens I will update again.
My two cents on this, don't know what the problem is with what you found, if not you can us bellow solution.
In order to have a shared Data Protection key is needed to explicitly enforce it.
With one note, in at startup is key doesn't exist in the source is created with an expiration associated. The Ideea is to save XElement and the name of it on a storage that can be used to retrieve at startup that value.
At startup:
services.Configure<KeyManagementOptions>(options =>
{
IDataProtectionRepo dataProtection = services.BuildServiceProvider().GetRequiredService<IDataProtectionRepo>();
options.NewKeyLifetime = DateTime.Now.AddYears(10) - DateTime.Now; // new one is created
options.XmlRepository = new DataProtectionKeyRepository(dataProtection);
});
where DataProtectionKeyRepository is the implementation of IXmlRepository
public class DataProtectionKeyRepository : IXmlRepository
{
private IDataProtectionRepo dataProtectionRepo;
public DataProtectionKeyRepository(IDataProtectionRepo dataProtectionRepo)
{
this.dataProtectionRepo = dataProtectionRepo;
}
public IReadOnlyCollection<XElement> GetAllElements()
{
return new ReadOnlyCollection<XElement>(dataProtectionRepo.GetAll().Select(k => XElement.Parse(k.XmlData)).ToList());
}
public void StoreElement(XElement element, string friendlyName)
{
dataProtectionRepo.AddOrUpdate(new ProtectionKeyModel { Name = friendlyName, XmlData = element.ToString() });
}
}
Communication class
public class ProtectionKeyModel
{
public string Name { get; set; }
public string XmlData { get; set; }
}
And the storage repo, can be a DB, file system, cloud storage, whatever is fine for you, implement bellow interface how you like to
public interface IDataProtectionRepo
{
IEnumerable<ProtectionKeyModel> GetAll();
void AddOrUpdate(ProtectionKeyModel protectionKeyModel);
}
I have a Flutter app that communicates with a server using gRPC. The server is using a self-signed certificate for TLS. I have added the certificate to my Flutter app, and this works on Android. However on iOS I get CERTIFICATE_VERIFY_FAILED error. Does iOS just not allow self-signed certificates?
I am setting up my gRPC client as follows:
var cert = await rootBundle.load('assets/cert.crt');
var creds = ChannelCredentials.secure(
certificates: cert.buffer.asUint8List().toList()
);
var channel = ClientChannel(
host,
port: port,
options: new ChannelOptions(credentials: creds));
return GrpcClient(channel);
There doesn't seem to be an obvious solution on iOS for adding a trusted, self-signed root CA. Since production will likely have a publically trusted CA, you can work around by disabling TLS verification for development only.
Here's the relevant snippet of my full example repo:
Future<ClientChannel> makeChannel() async {
final caCert = await rootBundle.loadString('assets/pki/ca/ca.crt');
return ClientChannel(
'localhost',
port: 13100,
options: ChannelOptions(
credentials: ChannelCredentials.secure(
certificates: utf8.encode(caCert),
// --- WORKAROUND FOR SELF-SIGNED DEVELOPMENT CA ---
onBadCertificate: (certificate, host) => host == 'localhost:13100',
),
),
);
}
In this case, my server is listening on localhost:13100.
The following was adapted from:
https://github.com/grpc/grpc-dart/issues/134
It allows for specifying a custom (or self-signed) CA cert, client certificates, and/or a custom domain:
import 'dart:convert';
import 'dart:io';
import 'package:grpc/grpc.dart';
class CustomChannelCredentials extends ChannelCredentials {
final String caCert;
final String? clientCert;
final String? clientKey;
CustomChannelCredentials({
required this.caCert,
this.clientCert,
this.clientKey,
String? authority, // Custom domain used by server cert
}) : super.secure(
authority: authority,
onBadCertificate: (cert, host) {
// This is a work-around for iOS, it seems self-signed certs are not being properly verified;
return host == '<the common name used self-signed CA>';
},
);
#override
SecurityContext get securityContext {
final context = SecurityContext(
withTrustedRoots: false, // We want to specify a custom CA cert
);
context.setTrustedCertificatesBytes(utf8.encode(caCert));
context.setAlpnProtocols(supportedAlpnProtocols, false);
if (clientCert != null) {
context.useCertificateChainBytes(utf8.encode(clientCert!));
}
if (clientKey != null) {
context.usePrivateKeyBytes(utf8.encode(clientKey!));
}
return context;
}
}
Example usage:
final channel = ClientChannel(
serverAddress,
port: serverPort,
options: ChannelOptions(
credentials: CustomChannelCredentials(
caCert: selfSignedCaCertPem,
// clientCert: clientCertPem,
// clientKey: clientKeyPem,
authority: 'localhost',
),
),
);
I have logstash-6.5.4 (with ssl), web and scheduler in my local (hostname: webbox) and kafka-2.0 (with ssl) on another (hostname: kafkabox).
I am not able to receive message in kafka topic when message is sent from logstash.
Neither error message is displayed not message is sent to kafka topic. I tried to import logstash.crt into kafka's truststore but it also didn't worked.
Created logstash.crt and logstash.key with below command.
sudo openssl req -x509 -batch -nodes -days 3650 -newkey rsa:2048 -keyout /etc/logstash/logstash.key -out /etc/logstash/logstash.crt
Imported the logstash.crt into kafka's truststore file also and tried.
keytool -import -alias logstash -file logstash.crt -keystore cacerts
Logstash conf file is given below...
input {
tcp {
host=>"0.0.0.0"
port=>5514
type=>"syslogType"
ssl_enable=>true
ssl_cert=>"/etc/logstash/logstash.crt"
ssl_key=>"/etc/logstash/logstash.key"
ssl_verify=>false
}
}
filter {
}
output {
kafka {
bootstrap_servers=>"kafkabox:9093"
codec=>"json_lines"
topic_id=>"a_test"
ssl_keystore_location=>"keystore file"
ssl_keystore_password=>"changeit"
ssl_key_password=>"changeit"
ssl_truststore_location=>"truststore file"
ssl_truststore_password=>"changeit"
security_protocol=>"SSL"
}
}
Expecting message is sent from logstash (with SSL) to kafka (with SSL).
Java Code to connect to logstash which internally failing to send message to kafka topics (in ssl mode).
public class LogstashClient {
private static String message = "<86>Jun 25 14:32:25 webbox sshd[7517]: Failed password for root from 196.165.132.192 port 45691 ssh2";
public static void main(String[] args) throws Exception {
nonSSL();
//SSL();
}
private static void SSL() throws Exception {
// logstash.crt is directly imported into kafka's truststore
// Below <<Client Truststore>> will also have logstash.crt imported for handshaking while connecting
System.setProperty("javax.net.ssl.trustStore", "<<Client Truststore>>");
System.setProperty("javax.net.ssl.trustStorePassword", "test1234");
SSLSocketFactory factory = (SSLSocketFactory)SSLSocketFactory.getDefault();
SSLSocket socket = (SSLSocket) factory.createSocket("localhost", 5514);
System.out.println("Handshaking...");
socket.startHandshake();
PrintWriter printWriter = new PrintWriter(socket.getOutputStream(), true);
boolean checkError = printWriter.checkError();
printWriter.println(message);
}
private static void nonSSL() throws Exception {
Socket socket = new Socket("localhost", 5514);
PrintWriter printWriter = new PrintWriter(socket.getOutputStream(), true);
printWriter.println(message);
}
}
Thanks,
RK,