How to verify a Microsoft certificate - ssl

This is the certificate https://gist.github.com/larytet/2fb447e875831577584592cd99980fd1 (x5t VjWIUjS5JS3eAFdm2dnydlZfY-I)
I am doing
openssl verify -verbose -x509_strict certificate.pen
I am getting
CN = estsclient.coreauth.outlook.com
error 20 at 0 depth lookup: unable to get local issuer certificate
error certificate.pem: verification failed
Where do I find the certificate, the whole chain or the root which should be installed in my system?

You need to provide the CA certificate, and most real CAs will list locations where its certificate can be found in an extension in each certificate they issue.
With OpenSSL, you can view this extension:
openssl x509 -text -noout < certificate.pem
Look for the "Authority Information Access extension", and its "CA Issuers" field to find the URL and download the certificate from Microsoft.
Because this file is encoded with DER, it needs to be transcoded to PEM for use with openssl verify:
openssl x509 -inform der < Microsoft\ IT\ TLS\ CA\ 2.crt > Microsoft\ IT\ TLS\ CA\ 2.pem
Because you just downloaded this file from who knows where over HTTP, you need some way to verify its authenticity.
You'll notice that it too lists an issuer, so you can perform this process recursively to obtain the entire certificate chain back to a root certificate that you already trust. Usually, we trust the certificates that come pre-installed on our systems. But, in theory, an attacker could have compromised that set, so people sometimes do find out-of-band means to verify their root CA certificates. What's appropriate for you depends on your application.
The chain of certificates that you download on your way to the trust anchor are "intermediate" certificates; you don't have to trust them directly, because you'll be verifying a chain starting with one of the anchors on your system.
Concatenate the PEM-encoded certificates, including headers and footers, together in a single file of untrusted certificates. In my case, the "Baltimore CyberTrust Root" certificate that issued the "Microsoft IT TLS CA 2" intermediate certificate is pre-installed as a root CA on my system, so I only have to download the Microsoft certificate, and it's the only one in my file of "untrusted" certificates.
Now you have the necessary information to attempt your original command:
openssl verify --verbose -untrusted Microsoft\ IT\ TLS\ CA\ 2.pem -x509_strict certificate.pem

In case anyone finds this question I ended up with something like this + lot of comment
RUN curl --silent https://outlook.com/autodiscover/metadata/json/1 > ./outlook.com.autodiscover.metadata.json.1
RUN pem_file=certificate.pem \
&& echo "-----BEGIN CERTIFICATE-----" > $pem_file \
&& cat ./outlook.com.autodiscover.metadata.json.1 | jq --raw-output '.keys[0].keyvalue.value' >> $pem_file \
&& echo "-----END CERTIFICATE-----" >> $pem_file \
&& cat $pem_file \
&& openssl x509 -text -noout < $pem_file | grep "CA Issuers" \
&& curl https://cacerts.digicert.com/BaltimoreCyberTrustRoot.crt.pem > rootCA.pem \
&& curl http://www.microsoft.com/pki/mscorp/Microsoft%20IT%20TLS%20CA%202.crt | openssl x509 -inform der >> rootCA.pem \
&& curl https://cacerts.digicert.com/DigiCertCloudServicesCA-1.crt | openssl x509 -inform der >> rootCA.pem \
&& cat rootCA.pem \
&& cat certificate.pem
RUN pem_file=certificate.pem && openssl verify -verbose -x509_strict -untrusted rootCA.pem $pem_file

Related

python confluent kafka client - unable to access Kafka on GKE using SSL

I have a simple python Kafka producer, and i'm trying to access the Strimzi Kafka Cluster on GKE, and i'm getting following error :
cimpl.KafkaException: KafkaError{code=_INVALID_ARG,val=-186,str="Failed to create producer: ssl.key.location failed: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch"}
Here is the Kafka producer code:
from confluent_kafka import Producer
kafkaBrokers='<host>:<port>'
caRootLocation='/Users/karanalang/Documents/Technology/strimzi/gcp_certs_nov28/pem-user2/cacerts.pem'
certLocation='/Users/karanalang/Documents/Technology/strimzi/gcp_certs_nov28/pem-user2/cert.pem'
keyLocation='/Users/karanalang/Documents/Technology/strimzi/gcp_certs_nov28/pem-user2/key.pem'
password='<password>'
conf = {'bootstrap.servers': kafkaBrokers,
'security.protocol': 'SSL',
'ssl.ca.location':caRootLocation,
'ssl.certificate.location': certLocation,
'ssl.key.location':keyLocation,
'ssl.key.password' : password
}
topic = 'my-topic1'
producer = Producer(conf)
for n in range(100):
producer.produce(topic, key=str(n), value="val -> "+str(n))
producer.flush()
To get the pem files (from the secrets - PKCS files), here are the commands used
kubectl get secret my-cluster-lb-ssl-certs-cluster-ca-cert -n kafka -o jsonpath='{.data.ca\.p12}' | base64 -d > ca.p12
kubectl get secret my-cluster-lb-ssl-certs-cluster-ca-cert -n kafka -o jsonpath='{.data.ca\.password}' | base64 -d > ca.password
kubectl get secret my-bridge1 -n kafka -o jsonpath='{.data.user\.p12}' | base64 -d > user2.p12
kubectl get secret my-bridge1 -n kafka -o jsonpath='{.data.user\.password}' | base64 -d > user2.password
- to get the user private key i.e. key.pem
openssl pkcs12 -in user2.p12 -nodes -nocerts -out key.pem -passin pass:<passwd>
# CARoot - extract cacerts.cer
openssl pkcs12 -in ca.p12 -cacerts -nokeys -chain | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > cacerts.cer
# convert to pem format
openssl x509 -in cacerts.cer -out cacerts.pem
# get the ca.crt from the secret
kubectl get secret my-cluster-lb-ssl-certs-cluster-ca-cert -n kafka -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
# convert to pem
openssl x509 -in ca.crt -out cert.pem
Any ideas how to fix this issue ?
Pls note -
I'm able to access Kafka Cluster using commandline Kafka producer/consumer on SSL
This is fixed, pls see below configuration that is expected:
'ssl.ca.location' -> CARoot (certifying authority, used to sign all the user certs)
'ssl.certificate.location' -> User Cert (used by Kubernetes to authenticate to API server)
'ssl.key.location' -> User private key
The above error was due to incorrect User Cert being used, it should match the User Private Key

Trouble getting a self-signed cert working in Postman

I'm finally getting into using SSL on my personal sites, so I started by trying to make a multi-domain self signed cert for my local development (to handle api.mydomain.local, www.mydomain.local, and mydomain.local). I don't know if this was my first mistake, but...
As I couldn't find a single encompassing guide, I started by using two tutorials (from EasyEngine and DeveloperSide) to create my cert and install it on my host (Win10). I then used a DigitalOcean guide to figure out how to setup my Apache on my dev server (a Ubuntu VM); up to there, no big trouble, other than some minor issues caused by working from multiple guides at the same time.
I go ahead and try to hit my api in Chrome, it gives me an untrusted certificate value as expected, I pass and it works. As far as I know, this means the cert worked? However, when I try to hit my api in Postman, I get an error that indicates it can't accept an untrusted cert, which is fine, as it has a tutorial on how to fix that. However, it still doesn't work. I can't figure out what else to do to fix this, am I on the right track? Is my cert completely borked? Did I make a core mistake in trying to do a multi-domain cert?
One thing I did notice is that in the Dev Tools security tab, it says
Subject Alternative Name missing
So I'm not sure if that means my alt names aren't working, but if they weren't, it wouldn't try to load the certificate when I hit it in Chrome, right?
I had a similar issue while writing an article for my website on SSL certificates. So i wrote shell script for the same
#!/bin/bash
CERT_COMPANY_NAME=${CERT_COMPANY_NAME:=Tarun Lalwani}
CERT_COUNTRY=${CERT_COUNTRY:=IN}
CERT_STATE=${CERT_STATE:=DELHI}
CERT_CITY=${CERT_CITY:=DELHI}
CERT_DIR=${CERT_DIR:=certs}
ROOT_CERT=${ROOT_CERT:=rootCA.pem}
ROOT_CERT_KEY=${ROOT_CERT_KEY:=rootCA.key.pem}
# make directories to work from
mkdir -p $CERT_DIR
create_root_cert(){
# Create your very own Root Certificate Authority
openssl genrsa \
-out $CERT_DIR/$ROOT_CERT_KEY \
2048
# Self-sign your Root Certificate Authority
# Since this is private, the details can be as bogus as you like
openssl req \
-x509 \
-new \
-nodes \
-key ${CERT_DIR}/$ROOT_CERT_KEY \
-days 1024 \
-out ${CERT_DIR}/$ROOT_CERT \
-subj "/C=$CERT_COUNTRY/ST=$CERT_STATE/L=$CERT_CITY/O=$CERT_COMPANY_NAME Signing Authority/CN=$CERT_COMPANY_NAME Signing Authority"
}
create_domain_cert()
{
local FQDN=$1
local FILENAME=${FQDN/\*/wild}
# Create a Device Certificate for each domain,
# such as example.com, *.example.com, awesome.example.com
# NOTE: You MUST match CN to the domain name or ip address you want to use
openssl genrsa \
-out $CERT_DIR/${FILENAME}.key \
2048
# Create a request from your Device, which your Root CA will sign
if [[ ! -z "${SAN}" ]]; then
openssl req -new \
-key ${CERT_DIR}/${FILENAME}.key \
-out ${CERT_DIR}/${FILENAME}.csr \
-subj "/C=${CERT_COUNTRY}/ST=${CERT_STATE}/L=${CERT_CITY}/O=$CERT_COMPANY_NAME/CN=${FQDN}" \
-reqexts san_env -config <(cat /etc/ssl/openssl.cnf <(cat ./openssl-san.cnf))
else
openssl req -new \
-key ${CERT_DIR}/${FILENAME}.key \
-out ${CERT_DIR}/${FILENAME}.csr \
-subj "/C=${CERT_COUNTRY}/ST=${CERT_STATE}/L=${CERT_CITY}/O=$CERT_COMPANY_NAME/CN=${FQDN}"
fi
# Sign the request from Device with your Root CA
if [[ ! -z "${SAN}" ]]; then
openssl x509 \
-sha256 \
-req -in $CERT_DIR/${FILENAME}.csr \
-CA $CERT_DIR/$ROOT_CERT \
-CAkey $CERT_DIR/$ROOT_CERT_KEY \
-CAcreateserial \
-out $CERT_DIR/${FILENAME}.crt \
-days 500 \
-extensions san_env \
-extfile openssl-san.cnf
else
openssl x509 \
-sha256 \
-req -in $CERT_DIR/${FILENAME}.csr \
-CA $CERT_DIR/$ROOT_CERT \
-CAkey $CERT_DIR/$ROOT_CERT_KEY \
-CAcreateserial \
-out $CERT_DIR/${FILENAME}.crt \
-days 500
fi
}
METHOD=$1
ARGS=${*:2}
echo "Called with $METHOD and $ARGS"
if [ -z "${METHOD}" ]; then
echo "Usage ./sslcerts.sh [create_root_cert|create_domain_cert] <args>"
echo "Below are the environment variabls you can use:"
echo "CERT_COMPANY_NAME=Company Name"
echo "CERT_COUNTRY=Country"
echo "CERT_STATE=State"
echo "CERT_CITY=City"
echo "CERT_DIR=Directory where certificate needs to be genereated"
echo "ROOT_CERT=Name of the root cert"
echo "ROOT_CERT_KEY=Name of root certificate key"
else
${METHOD} ${ARGS}
fi
You can change the environment variables on the TOP and generate a self-signed certificate using below
$ SAN=DNS.1:*.tarunlalwani.com,DNS.2:tarunlalwani.com ./sslcerts.sh create_domain_cert '*.tarunlalwani.com'
Edit 1
Earlier browsers use to rely on FQDN, but now some of them have started using SAN which is "Subject Alternative Name". Generally openssl doesn't come up v3 extensions configured. SAN is a part of the v3 extensions. So when you generated a self signed certificated it has the correct FQDN (Full qualified domain name) but not SAN. Chrome will show error for these certificates but you will see firefox working fine.
PS: Taken from article http://tarunlalwani.com/post/self-signed-certificates-trusting-them/

Use self signed certificate with cURL?

I have a flask application running using a self signed certificate. I'm able to send in a curl request using:
curl -v -k -H "Content-Type: application/json" -d '{"data":"value1","key":"value2"}' https://<server_ip>:<port>
The verbose logs show that everything went alright.
I wanted to avoid using the -k (--insecure) option and instead specify a .pem file that curl could use. Looking at the curl man page I found that you could do this using the --cert option.
So I created a .pem file using this:
openssl rsa -in server.key -text > private.pem
CURL throws me this error when using the private.pem file:
curl: (58) unable to use client certificate (no key found or wrong pass phrase?)
Any suggestions? - or is this only possible with a properly signed certificate?
Tnx
This is just another version of this question: Using openssl to get the certificate from a server
Or put more bluntly:
Using curl --cert is wrong, it is for client certificates.
First, get the the certs your server is using:
$ echo quit | openssl s_client -showcerts -servername server -connect server:443 > cacert.pem
(-servername is necessary for SNI so that you get the right virtual server's certificate back)
Then make your curl command line use that set to verify the server in subsequent operations:
$ curl --cacert cacert.pem https://server/ [and the rest]
special teaser
Starting with curl 7.88.0 (to be shipped in February 2023), curl can save the certificates itself with the new %{certs} variable for the -w option. Blogged about here.
To make request from https server through curl. I make use of below steps
Step1: Generate self signed certificate with below code at root of the project you want to make use of it.openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -nodes
Step2: Fill the prompt with required details but when you get to Common name input localhost e.g Common Name (eg, fully qualified host name) []:localhost
step3: When your openssl cert.pem & key.pem has being generated startup your server then in another terminal or command line run curl --cacert cert.pem https://localhost:443
Note: I use port 443 which is the default https port, you can make use of another port then make sure cert.pem file path is well referenced.

How to setup Dart to use a CA SSL certificate?

I recently deployed a Dart server application that serves HTTP requests. I wanted to add support for HTTPS so I have been trying to add SSL to the Dart server application.
This answer gives a clear explanation of how to add a self-signing SSL certificate to Dart. However, I want to add an SSL certificate I bought from an SSL provider.
The SSL provider e-mailed my 4 files:
Root CA Certificate - AddTrustExternalCARoot.crt
Intermediate CA Certificate - COMODORSAAddTrustCA.crt
Intermediate CA Certificate - COMODORSADomainValidationSecureServerCA.crt
Your PositiveSSL Certificate - my_domain.crt
I have been trying to figure out how certutil works and how to add these certificates to the certificate database, but I just can't figure it all out.
Anyone with experience enabling a CA SSL certificate in Dart?
SOLVED: Thanks to suggestion in the comments, I solved the issue. This is the gist of my complete setup: https://gist.github.com/stevenroose/e6abde14258971eae982
First of all, you probably have three files generated with openssl for your private key, server certificate and CA certificate. To convert all those into a PKCS12 file, you can use openssl:
openssl pkcs12 -export -out server.p12 -inkey server.key -in server.crt -certfile CAcert.crt
Then, you can adapt the certutil commands as shown to load you PKCS12 instead of generating new certificates:
certutil -N -d sql:certdb
certutil -A -n mycertnick -i server.crt -t "TCu,Cu,Tuw" -d sql:certdb
certutil -A -n myCA -i CAcert.crt -t "TCu,Cu,Tuw" -d sql:certdb
pk12util -i server.p12 -d sql:certdb
It seems to work with the sample code in the referenced question.
Unfortunately the SSL management in Dart is known to be very lacking.
I reported this many times, with no serious answer from the Dart team.
Star this issue if you want something done about it:
https://code.google.com/p/dart/issues/detail?id=20967

How does the browser deal with missing intermediate certs

I've come across the site https://alpower.com, this site is only providing its own site certificate. Because of this I can't access the site properly with cURL as the cacerts used are only root certsificates.
The site is accessible in Firefox however. How exactly is Firefox able to verify the site's identity where as cURL isn't?
Browsers will cache intermediate certificates. So if the missing certificate was already provided by another site the browser will have it already and will use it. But, if you use a fresh browser profile you might get the same problems as you get with curl, because the intermediate certificate is not cached.
This is at least how it works with Firefox. Other browsers might look into the Authority Information Access section of the certificate and if they find the URL issuer certificate they will download the certificate to continue with the chain verification.
Most browsers are using the AIA information embedded in the certificate (see comment on browsers exceptions).
To expose the URL of the CA Issuer with openssl:
openssl x509 -in "YOUR_CERT.pem" -noout -text
There is a section Authority Information Access with CA Issuers - URI which would be the "parent" certificate (intermediate or root certificate).
This can be reproduced up to the root CA.
In a gist:
ssl_endpoint=<ENDPOINT:443>
# first, get the endpoint cert
echo | openssl s_client -showcerts -connect $ssl_endpoint 2>/dev/null | openssl x509 -outform PEM > endpoint.cert.pem
# then extract the intermediate cert URI
intermediate_cert_uri=$(openssl x509 -in endpoint.cert.pem -noout -text | (grep 'CA Issuers - URI:' | cut -d':' -f2-))
# and get the intermediate cert (convert it from DER to PEM)
curl -s "${intermediate_cert_uri}" | openssl x509 -outform PEM -inform DER > intermediate.cert.pem