converting .pem keys to .der compiles but results in errors - ssl

Calling my hyper based API now ported to HTTPS, with Python's requests I'm getting
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)' on every request.
As per the docs for tokio_rustls:
Certificate has to be DER-encoded X.509
PrivateKey has to be DER-encoded ASN.1 in either PKCS#8 or PKCS#1 format.
The keys I used in my PKEY and CERT variables are my certbot generated .pem keys converted to .der format using those commands:
openssl x509 -outform der -in /etc/letsencrypt/live/mysite.com/cert.pem -out /etc/letsencrypt/live/mysite.com/cert.der
openssl pkcs8 -topk8 -inform PEM -outform DER -in /etc/letsencrypt/live/mysite.com/privkey.pem -out /etc/letsencrypt/live/mysite.com/privkey.der -nocrypt
And loaded up with include_bytes!() macro.
Well it compiles, polls... and just throws this error on every request Bad connection: cannot decrypt peer's message whilst the caller gets the SSLError mentioned in the beginning.
Here is the script used for the API:
fn tls_acceptor_impl(cert_der: &[u8], key_der: &[u8]) -> tokio_rustls::TlsAcceptor {
let key = PrivateKey(cert_der.into());
let cert = Certificate(key_der.into());
Arc::new(
ServerConfig::builder()
.with_safe_defaults()
.with_no_client_auth()
.with_single_cert(vec![cert], key)
.unwrap(),
)
.into()
}
fn tls_acceptor() -> tokio_rustls::TlsAcceptor {
tls_acceptor_impl(PKEY, CERT)
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let addr = SocketAddr::from(...);
let mut listener = tls_listener::builder(tls_acceptor())
.max_handshakes(10)
.listen(AddrIncoming::bind(&addr).unwrap());
let (tx, mut rx) = mpsc::channel::<tokio_rustls::TlsAcceptor>(1);
let http = Http::new();
loop {
tokio::select! {
conn = listener.accept() => {
match conn.expect("Tls listener stream should be infinite") {
Ok(conn) => {
let http = http.clone();
// let tx = tx.clone();
// let counter = counter.clone();
tokio::spawn(async move {
// let svc = service_fn(move |request| handle_request(tx.clone(), counter.clone(), request));
if let Err(err) = http.serve_connection(conn, service_fn(my_query_handler)).await {
eprintln!("Application error: {}", err);
}
});
},
Err(e) => {
eprintln!("Bad connection: {}", e);
}
}
},
message = rx.recv() => {
// Certificate is loaded on another task; we don't want to block the listener loop
let acceptor = message.expect("Channel should not be closed");
}
}
}
How can I make any sense of the errors, when the certificate keys work on Web (as they are the apache2 server's keys)? I've tried various other encodings, that are against the docs, and all fail in the same way.

I'm not familiar enough with rust, but I know that proper configuration of a TLS server (no matter which language) requires the server certificate, the key matching the certificate and all intermediate CA needed to build the trust chain from server certificate to the root CA on the clients machine. These intermediate CA are not provided in your code. That's why the Python code fails to validate the certificate.
What you need is probably something like this:
ServerConfig::builder()
....
.with_single_cert(vec![cert, intermediate_cert], key)
Where intermediate_cert is the internal representation of the Let’s Encrypt R3 CA, which you can find here.

Related

jose-jwt can generate JWT token via private key, but failed to valid token via RSA public key

I was trying to implement JWT authentication based on jose-jwt for .NET, I can generate JWT token by RSA private key successfully, and pass checking on https://jwt.io debugger, but I'm getting stuck on validating the token with RSA public key by the following error message:
RsaUsingSha alg expects key to be of AsymmetricAlgorithm type.
Here is my steps and code snippets:
create PKCS#12 and certificate file
openssl x509 -signkey jwt-auth-private.key -in jwt-auth-csr.csr -req -days 365 -out jwt-auth-certificate.crt
openssl pkcs12 -inkey jwt-auth-private.key -in jwt-auth-certificate.crt -export -CSP "Microsoft Enhanced RSA and AES Cryptographic Provider" -out jwt-auth-pkcs12.pfx
Create JWT token with RSA private key (It works)
var pfxFilePath = string.Format("{0}\\{1}", baseDir, "jwt-auth-pkcs12.pfx");
var cipher = "private-key-cipher";
var privateKey = new X509Certificate2(pfxFilePath, cipher, X509KeyStorageFlags.Exportable | X509KeyStorageFlags.MachineKeySet).PrivateKey as RSACryptoServiceProvider;
var payload = new Dictionary<string, object>()
{
{ "sub", "mr.x#contoso.com" },
{ "exp", 1300819380 }
};
token = Jose.JWT.Encode(payload, privateKey, JwsAlgorithm.RS256);
Valid token with public key (Failed)
var crtFilePath = string.Format("{0}\\{1}", baseDir, "jwt-auth-certificate.crt");
var cipher = "private-key-cipher";
var publicKey = new X509Certificate2(crtFilePath, cipher).GetPublicKey();
var data = Jose.JWT.Decode(token, publicKey);
When I tried to decode token, I got an error message:
RsaUsingSha alg expects key to be of AsymmetricAlgorithm type.
I had no idea on how to find what's wrong, please help. thanks.
Remark: It looks like that the .crt and .pfx file had no issues, cause if I use JWT.NET instead of jose-jwt libs, I can decode the token successfully in the same way.

SSL implementation with Tungstenite: SSL alert number 42

I created a working WebSocket server with async_tungstenite and async_std.
I now want to add SSL using async_native_tls.
If I understood correctly, this crates provides a function accept which takes a TcpStream, handles the TLS handshake and provides a TlsStream<TcpStream> which should behave like a TcpStream but handles the encryption and decryption behind the scene.
To test the server, I created a self-signed certificate.
Based on that, here is how the code handling new TCP connections evolved:
async fn accept_connection(stream: TcpStream, addr: SocketAddr) {
//Websocket stream
let accept_resut = async_tungstenite::accept_async(stream).await;
if let Err(err) = accept_resut {
println!(
"Error while trying to accept websocket: {}",
err.to_string()
);
panic!(err);
}
println!("New web socket: {}", addr);
}
async fn accept_connection(stream: TcpStream, addr: SocketAddr) {
//Open tls certificate !should be done one time and not for each connection!
let file = File::open("identity.pfx").await.unwrap();
let acceptor_result = TlsAcceptor::new(file, "glacon").await;
if let Err(err) = acceptor_result {
println!("Error while opening certificate: {}", err.to_string());
panic!(err);
}
let acceptor = acceptor_result.unwrap();
//Get a stream where tls is handled
let tls_stream_result = acceptor.accept(stream).await;
if let Err(err) = tls_stream_result {
println!("Error during tls handshake: {}", err.to_string());
panic!(err);
}
let tls_stream = tls_stream_result.unwrap();
//Websocket stream
let accept_resut = async_tungstenite::accept_async(tls_stream).await;
if let Err(err) = accept_resut {
println!(
"Error while trying to accept websocket: {}",
err.to_string()
);
panic!(err);
}
println!("New web socket: {}", addr);
}
With this implementation, I now call from a webpage
const sock = new WebSocket('wss://localhost:8020');
This results in the error:
Error while trying to accept websocket:
IO error: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:../ssl/record/rec_layer_s3.c:1543:SSL alert number 42
thread 'async-std/runtime' panicked at 'Box<Any>', src/main.rs:57:9
It seems like the handshake was successful as the error does not occur during the acceptor.accept. The error states that the certificate is not valid so here is how I created my self-signed certificate.
The openssl version is 1.1.1f
# Create a key
openssl req -nodes -new -key server.key -out server.csr
# Create the self-signed certificate
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
# Convert the certificate to pfx format
openssl pkcs12 -export -out identity.pfx -inkey server.key -in server.crt
I thought that this problem had to do with security feature from the browser as the "SSL alert number 42" seems to come from the client. I tried to disable this option in Firefox settings
Query OCSP responder servers to confirm the current validity of certificates
I also tried to add my server.crt to the Authorities of the certificate manager.
Neither of these worked.
The problem came from the security features of Firefox.
Firefox detects that the certificate is not signed by an authority and sends back an error.
It seems like adding the certificate to the known authorities does not work.
To avoid this issue, I found this thread which indicates that an exception should be added for the address and port of your development Websocket server.
Go to Settings > Certificates > View Certificates > Servers > Add Exception...
Type in your local server (for me localhost:8020).
Add exception.

Adding extension in CSR for generating an intermediate certificate

I am generating a Certificate Signing Request for an intermediate certificate. I want to make the certificate a certificate authority (CA), so I want to add the basic constraints extension in CSR. I am currently using the following code
exts = sk_X509_EXTENSION_new_null();
add_ext(exts, x509_req, NID_basic_constraints, "critical,CA:TRUE");
X509_REQ_add_extensions(x509_req, exts);
sk_X509_EXTENSION_pop_free(exts, X509_EXTENSION_free);
The add extension function looks like this
int add_ext(STACK_OF(X509_EXTENSION) *sk, X509_REQ* req, int nid, char *value)
{
X509_EXTENSION *ex;
X509V3_CTX ctx;
X509V3_set_ctx_nodb(&ctx);
X509V3_set_ctx(&ctx, NULL, NULL, req, NULL, 0);
ex = X509V3_EXT_conf_nid(NULL, &ctx, nid, value);
if (!ex)
{
log("X509V3_EXT_conf_nid generated error", cspsdk::Logging::LOG_LEVEL_INFO);
return 0;
}
sk_X509_EXTENSION_push(sk, ex);
return 1;
}
The problem is that after getting signed, the certificate has the CA value of basic constraints extension set to false. I am at a loss here. Can anybody point out the issue.
Your issuer can choose to override the constraints like CA: False even though you requested for CA: True. You need to contact them unless you are self-signing your certs.
openssl x509 -in your-signed-cert.pem -text -noout
Please check if the output contains "CA:True".

How do I use frisby.js with SSL client certificates?

I have a question about the use of Frisby.js with a REST service that uses SSL client certificates.
If I use the Request package to talk to the service, I can do something like this:
request.get(URL, {
agentOptions: {
// Or use `pfx` property replacing `cert` and `key` when using private key, certificate and CA certs in PFX or PKCS12 format:
pfx: fs.readFileSync(pfxFilePath),
passphrase: 'password',
ca: fs.readFileSync(caFilePath)
}
});
Since Frisby uses request, I assume I can do something similar in a Frisby test, but I can’t seem to get the syntax correct.
Can you suggest something that might work? Thanks...
The .get() and .post() methods can take an additional struct with contents similar to the options parameter of https.request(). You will need to supply values for (some of): pfx, key, cert, ca and passphrase.
Something like this:
var fs = require('fs');
var frisby = require('frisby');
frisbee.create( 'Test name' )
.get(
'https://localhost/rest/endpoint',
{
key: fs.readFileSync( "/path/to/certificates/certificate.key" ),
cert: fs.readFileSync( "/path/to/certificates/certificate.crt" ),
ca: fs.readFileSync( "/path/to/certificates/ca.crt" ),
passphrase: "your pass phrase"
}
)
.expectStatus( 200 )
.toss();

Issues with TLS connection in Golang

I have the following certificate hierarchy:
Root-->CA-->3 leaf certificates
The entire chain has both serverAuth and clientAuth as extended key usages explicitly defined.
In my go code, I create a tls.Config object like so:
func parseCert(certFile, keyFile string) (cert tls.Certificate, err error) {
certPEMBlock , err := ioutil.ReadFile(certFile)
if err != nil {
return
}
var certDERBlock *pem.Block
for {
certDERBlock, certPEMBlock = pem.Decode(certPEMBlock)
if certDERBlock == nil {
break
}
if certDERBlock.Type == "CERTIFICATE" {
cert.Certificate = append(cert.Certificate, certDERBlock.Bytes)
}
}
// Need to flip the array because openssl gives it to us in the opposite format than golang tls expects.
cpy := make([][]byte, len(cert.Certificate))
copy(cpy, cert.Certificate)
var j = 0
for i := len(cpy)-1; i >=0; i-- {
cert.Certificate[j] = cert.Certificate[i]
j++
}
keyData, err := ioutil.ReadFile(keyFile)
if err != nil {
return
}
block, _ := pem.Decode(keyData)
if err != nil {
return
}
ecdsaKey, err := x509.ParseECPrivateKey(block.Bytes)
if err != nil {
return
}
cert.PrivateKey = ecdsaKey
return
}
// configure and create a tls.Config instance using the provided cert, key, and ca cert files.
func configureTLS(certFile, keyFile, caCertFile string) (tlsConfig *tls.Config, err error) {
c, err := parseCert(certFile, keyFile)
if err != nil {
return
}
ciphers := []uint16 {
tls.TLS_RSA_WITH_AES_256_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
}
certPool := x509.NewCertPool()
buf, err := ioutil.ReadFile(caCertFile)
if nil != err {
log.Println("failed to load ca cert")
log.Fatal(seelog.Errorf("failed to load ca cert.\n%s", err))
}
if !certPool.AppendCertsFromPEM(buf) {
log.Fatalln("Failed to parse truststore")
}
tlsConfig = &tls.Config {
CipherSuites: ciphers,
ClientAuth: tls.RequireAndVerifyClientCert,
PreferServerCipherSuites: true,
RootCAs: certPool,
ClientCAs: certPool,
Certificates: []tls.Certificate{c},
}
return
}
certFile is the certificate chain file and keyFile is the private key file. caCertFile is the truststore and consists of just the root certificate
So basically, here is what I expect to have inside of my tls.Config object that comes out of this function:
RootCAs: Just the root certificate from caCertFile
ClientCAs: Again, just the root certificate from caCertFile, same as RootCAs
Certificates: A single certificate chain, containing all of the certificates in certFile, ordered to be leaf first.
Now, I have 3 pieces here. A server, a relay, and a client. The client connects directly to the relay, which in turn forwards the request to the server. All three pieces use the same configuration code, of course using different certs/keys. The caCertFile is the same between all 3 pieces.
Now, if I stand up the server and the relay and connect to the relay from my browser, all goes well, so I can assume that the connection between relay and server is fine. The issue comes about when I try to connect my client to the relay. When I do so, the TLS handshake fails and the following error is returned:
x509: certificate signed by unknown authority
On the relay side of things, I get the following error:
http: TLS handshake error from : remote error: bad certificate
I am really at a loss here. I obviously have something setup incorrectly, but I am not sure what. It's really weird that it works from the browser (meaning that the config is correct from relay to server), but it doesn't work with the same config from my client.
Update:
So if I add InsecureSkipVerify: true to my tls.Config object on both the relay and the client, the errors change to:
on the client: remote error: bad certificate
and on the relay: http: TLS handshake error from : tls: client didn't provide a certificate
So it looks like the client is rejecting the certificate on from the server (the relay) due to it being invalid for some reason and thus never sending its certificate to the server (the relay).
I really wish go had better logging. I can't even hook into this process to see what, exactly, is going on.
When you say
Need to flip the array because openssl gives it to us in the opposite format than golang tls expects.
I have used certificates generated by openssl and had no problem opening them with:
tls.LoadX509KeyPair(cert, key)
Anyway, the error message bad certificate is due to the server not managing to match the client-provided certificate against its RootCAs. I have also had this problem in Go using self-signed certificats and the only work-around I've found is to install the caCertFile into the machines system certs, and use x509.SystemCertPool() instead of x509.NewCertPool().
Maybe someone else will have another solution?
Beside what beldin0 suggested.
I have tried another way to do this.
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(crt)
client := &http.Client{
//some config
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: caCertPool,
},
},
}
Here, the variable "crt" is the content in your certificate.
Basically, you just add it into your code(or read as a config file).
Then everything would be fine.