I am trying to authenticate against an API using certs.
I have a pfx file that I need to convert to required certs.
In order to generate the Public cert, i used the command :
openssl pkcs12 -in ~/Downloads/file.pfx -nodes -clcerts -nokeys | openssl x509 -out public.crt
This public cert was uploaded on the API side.
Now from the client side I need to connect to the api using curl
So first I generated the Private Key using the command :
openssl pkcs12 -in ~/Downloads/file.pfx -nodes -nocerts | openssl rsa -out private_new.key
And now Im trying to connect to the API using the command :
curl -I -k --key ./private_new.key --cert ./public.crt https://<API-END-POINT>/foo/bar
But in the response I get :
HTTP/1.1 401 Unauthorized
Date: Thu, 31 Jan 2019 00:24:43 GMT
Server: Foo
X-IDS-ID: 4E178F65-78F2-4CB9-B31A-8D6288F854C5
WWW-Authenticate: Basic realm=CPS Rest Services
X-message-code: MISSING_AUTHORIZATION_HEADER
Content-Type: text/html
Vary: X-CSP-STRIP
X-IDS-Node: idp15
X-IDS-Pool: green
X-IDS-Project: prod
X-IDS-Landscape: eu-nl-1
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=31536000;includeSubDomains;preload
Cache-Control: private,no-cache,no-store
Is there something I have missed ? Im new to this so Im not sure what I might have missed.
HTTP/1.1 401 Unauthorized
...
WWW-Authenticate: Basic realm=CPS Rest Services
X-message-code: MISSING_AUTHORIZATION_HEADER
The API requires a proper Authentication header within your request but you don't send one. It is unclear what the contents should be but usually there is some username and password encoded together.
There is nothing known about the API you are trying to access so no help is possible on how to use it properly:
Specifically it is not known if the site has authentication using a client certificate at all. If it doesn't then you can't enforce it.
If it supports client certificates then it might find your self-generated certificate not sufficient since there is no way to verify who is behind this certificate. In this case it might try with Basic Authentication as a fallback to the failed client certificate.
It might also be that the site needs Basic Authentication in addition to a client certificate.
Related
I have an Apache server that was recently upgraded from Debian 9 to Debian 10:
Apache : 2.4.25-3+deb9u13 => 2.4.52
Openssl: 1.1.0l-1~deb9u5 => 1.1.1n-0+deb10u1
Since this upgrade, connections from clients where client is authenticated (SSLVerifyClient require in Apache configuration) fails. Error in Apache debug logs is:
[Tue Apr 19 17:41:57.686913 2022] [ssl:error] [pid 19335] SSL Library Error: error:1414D17A:SSL routines:tls12_check_peer_sigalg:wrong curve
TLS_v1.2 was forced on server (Apache configuration: SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 +TLSv1.2 -TLSv1.3).
I made a wireshark capture and I am able to decode it. When authentication works (with Debian 9 version), messages exchanged are the following:
messagesExchange
When error occurs, The last "Finished" message sent by server is replaced by a TLS Alert (Level Fatal, Description: Illegal parameter).
This happens just after the client certificate is received by server, my conclusion is that the client certificate is not accepted by server.
Client certificate was generated by our PKI from a private key and a certificate signing request created on client using these commands:
openssl ecparam -genkey -name secp256k1 -noout -out ${PRIVATE_KEY}
openssl req -new -sha256 -key ${PRIVATE_KEY} -out ${CSR} -subj "/C=FR/O=XXX/OU=YYY/CN=ZZZ"
So curve used for this client certificate is secp256k1. This curve is not supported by TLS_v1.3.
If I create a private key using prime256v1 curve instead if using secp256k1, recreate CSR and certificate using the same PKI, I am able to connect to server without errors. Commands become:
openssl ecparam -genkey -name prime256v1 -noout -out /user/private_prime256v1.key
openssl req -new -sha256 -key /user/private_prime256v1.key -out /user/prime256v1.csr -subj "/C=FR/O=XXX/OU=YYY/CN=ZZZ"
So I assume my problem is linked to usage of secp256k1 curve in client certificates (and this new versions of Apache and Openssl).
I have tons of clients with certificates created from a private key using secp256k1 curve, so changing client certificates is not an option. I am looking for an Apache / mod_ssl / openssl configuration to accept client certificates using secp256k1 certificates even on this new version. I tried:
SSLOpenSSLConfCmd Groups "secp256k1:secp384r1:secp521r1:prime256v1:sect283k1:sect283r1:sect409k1:sect409r1:sect571k1:sect571r1"
without success.
There are many other parameters I can modify using SSLOpenSSLConfCmd but I didn't find any other configurations precisely linked to client authentication or to curves used in certificates.
There is an application (Java) deployed in OpenShift. The istio-ingress-gateway is also configured. My task is to call my service using the https protocol with mTLS. At the moment, I have the following settings in the ingress-gateway in the tls block:
caCertificates: somePath/ca.pem
mode: MUTUAL
privateKey: somePath/tls.key
serverCertificate: somePath/tls.crt
I want to call my service through Google Chrome / Postman / curl, as I understand it, for this I need to generate a client certificate based on the certificates specified in openshift and put it in the calling client.
My question is how can I generate a client certificate from this and how can I put it on the caller?
it's work for browser - openssl pkcs12 -export -out cert.pfx -inkey tls.key -in tls.crt -certfile ca.pem - don't forget import cert.pfx to windows
For Curl - 'c u r l https://mTLSHost -k --key ./tls.key --cert ./tls.crt'
I have verified ssl certificate (got it when bought hosting). It consist of four parts: 1. private key, 2. certificate, 3. root sertificate, 4. intermediate certificate. I made two files .key(private), .crt(certificate, intermediate, root) and confugure nginx. Everything good, my domain shows https, and https://www.sslshopper.com says that it works.
But when I set telegram bot webhook
def start_request():
url = 'https://api.telegram.org/bot{token}/{method}'.format(
token = 'myToken',
method = 'setWebhook'
)
data = {
'url' : 'MyDomain',
'certificate' : open('myCertificate', 'r')
}
r = requests.post(url, data = data)
webhook status always
result
url "myDomain"
has_custom_certificate false
pending_update_count 5
last_error_date 1515041749
last_error_message "Wrong response from the webhook: 403 Forbidden"
and
nginx log says
149.154.167.214 - - [04/Jan/2018:07:07:00 +0300] "POST myDomain" 403 997 "-" "-"
Is certificate problem ?
What certificate or part of the certificate and in what format I should send to telegram ?
I think setWebhook is seccessd. You can use this Android application to determine.
BTW, you can refer to this guide, use curl to debug yourself.
In my case I take root certificate (only root) and convert it to .der and to .pem ,
openssl x509 -in root.crt -outform der -out root.der
openssl x509 -in root.der -inform der -outform pem -out root.pem
after that I set webhook with "Awesome Telegram Bot" android application using root.pem certificate.
And using getWebhookInfo method I get
url "https://myDomain"
has_custom_certificate true
We're looking into an AS2 implementation and would like to be able to build meaningful test-cases to be used with SoapUI or Postman.
In order to do this we have two ways:
just try to tcp-dump/trace calls from an existing client
manually build some simple calls starting from the plain EDI documents
or various reasons we've discarded (1) so we must go with (2) and we need some documentation.
This doc from Oracle is a good starting point: https://docs.oracle.com/cd/E19398-01/820-1228/agfat/index.html
but we cannot really find a step-by-step guide for building the AS2 request starting from the ORDERS edi document (which we already have).
Ideally I'd like a step-by-step guide which says something like:
generate a private cert: +commandline
encrypt the EDI document: +commandline +sampleoutput
create signature: +commandline +sampleoutput
build the S/MIME package: +commandline +sampleoutput
send over HTTP/S: +commandline +samplehttptrace
Sending Message
To send a message we need a file with message to send, two pairs of keys for both sides (each partner has one pair of keys for signing and another pair of keys for encrypting, each pair of keys consists of public and private keys). For this tutorial we will use the same pair of keys for signing and encryption per partner.
Generate Keys
Using OpenSSL create two pairs of keys:
openssl req -x509 -newkey rsa:2048 -keyout P1_private.pem -out P1_public.pem -days 365
The same for second pair:
openssl req -x509 -newkey rsa:2048 -keyout P2_private.pem -out P2_public.pem -days 365
Extend Private Key Files
P1_private.pem and P2_private.pem contain only private keys. OpenSSL supports extended PEM format where both, private and public keys, can be in one file. This simplifies some OpenSSL commands (one file can be given instead of two). This tutorial assumes that this has been done:
cat P1_public.pem >> P1_private.pem
cat P2_public.pem >> P2_private.pem
Otherwise, check OpenSSL commands manual to see how to specify second file where needed.
Sign File
Let's assume we want to send GETMSG.edi file via AS2 protocol from P1 partner => to P2, with signing and encryption. First we add appropriate MIME headers to it:
Content-Type: application/edi-consent
Content-Disposition: attachment; filename="GETMSG.edi"
UNA:+./*'
UIB+UNOA:0++2289+++77777777:C:PASSWORDA+111111:M+19971001:074620'
UIH+SCRIPT:010:006:GETMSG+111'
UIT+111+2'
UIZ++1'
Save new file as GETMSG.msg
Then sign the file with private key of sending partner P1:
openssl smime -sign -in GETMSG.msg -out GETMSG_SIGNED.msg -signer P1_private.pem
Note: OpenSSL puts old MIME types that start with 'x-pkcs7-', some applications (pyAS2) can handle only new MIME types without eks 'pkcs7-'. Just remove 'x-' everywhere from resulting file.
Encrypt
Now we have multipart attachment with first part being the file content and the second part is the signature. Now we encrypt it with public key of receiving partner P2:
openssl smime -encrypt -in GETMSG_SIGNED.msg -out GETMSG_ENC.msg -des3 P2_public.pem
Resulting file GETMSG_ENC.msg can be send now to recipient as request with additional AS2 headers.
Note: OpenSSL puts old MIME types that start with 'x-pkcs7-', some applications (pyAS2) can handle only new MIME types without eks 'pkcs7-'. Just remove 'x-' everywhere from resulting file.
Send via cURL
To send with cURL we have to separate headers and body of POST request. Remove from file GETMSG_ENC.msg all headers (it shoud start with 'MI..').
Use this command to send it from P1 (AS2 ID: p1as2) to P2 (AS2 ID: p2as2), assuming that P2 URL is "http://localhost:8080/pyas2/as2receive":
set NOW=%DATE:~10,4%%DATE:~4,2%%DATE:~7,2%%TIME:~0,2%%TIME:~3,2%%TIME:~6,2%
curl -i -X POST-H "Content-Disposition: attachment; filename=\"smime.p7m\""-H "Content-Type: application/pkcs7-mime; smime-type=enveloped-data; name=\"smime.p7m\""-H "Content-Transfer-Encoding: base64"-H "AS2-TO: p1as2"-H "AS2-FROM: p2as2"-H "AS2-VERSION: 1.2"-H "MESSAGE-ID: <openssl%NOW%#LOCALHOST>"-H "Disposition-Notification-To: response#localhost"-H "DISPOSITION-NOTIFICATION-OPTIONS: signed-receipt-protocol=required, pkcs7-signature; signed-receipt-micalg=optional, sha1"--data-binary #GETMSG_ENC.msg http://localhost:8080/pyas2/as2receive
Note: this command would also request MDN with signature (because of presence of Disposition-Notification-To and value of DISPOSITION-NOTIFICATION-OPTIONS).
Receiving Message
The following will already happen on receiving side. That would include decrypting message, verifying signature, extracting payload and preparing acknowledgment or message disposition notification (MDN). It would also require to calculate hash of received message (MIC) to be send in acknowledgment.
Decrypt Request
Using OpenSSL command decode request content saved as smime.p7m:
openssl smime -decrypt -in smime.p7m -recip P2_public.pem -inkey P2_private.pem -out request.txt
Verify Signature
openssl smime -verify -in request.txt -nosigs -noverify -signer P2_public.pem -out original.txt
Calculate Received-content-MIC
Digest of received content has to be calculated over original file WITH MIME headers, in our case that is content of GETMSG.msg.
openssl dgst -sha1 -binary GETMSG.msg | openssl enc -e -base64
Note: in real life receiver has to extract that from decrypted content of type multipart/signed. First part is the received content.
Message Disposition Notification
Not signed message disposition notification or acknowledgement would look like multipart report:
Content-Type: multipart/report; report-type="disposition-notification"; boundary="===============1785295974=="
--===============1785295974==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
The AS2 message has been processed. Thank you for exchanging AS2 messages with Pyas2.
--===============1785295974==
Content-Type: message/disposition-notification; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Reporting-UA: Bots Opensource EDI Translator
Original-Recipient: rfc822; p1as2
Final-Recipient: rfc822; p1as2
Original-Message-ID: <openssl20170706165018#LOCALHOST>
Disposition: automatic-action/MDN-sent-automatically; processed
Received-content-MIC: 1GZ1SDk5vvGz5YFGYP6lfhk4MXE=, sha1
--===============1785295974==--
Signed Message Disposition Notification
If signed MDN is requested (depends on 'Disposition-Notification-Options' HTTP header) then multipart report shown above is wrapped in multipart/signed. And seconds part is signature of the first part shown above. If we assume that mdn.txt contains not signed MDN as shown above, then to sign it:
openssl cms -sign -signer P2_private.pem -in mdn.txt -out mdn_signed.txt
Now we need only to add some AS2 specific headers to the top of mdn.txt or mdn_signed.txt, depending on what kind of MDN was requested, before transmitting it over HTTP (taken from actual response of pyAS2):
ediint-features: CEM
as2-from: p1as2
user-agent: PYAS2, A pythonic AS2 server
AS2-Version: 1.2
as2-to: p2as2
date: Thu, 06 Jul 2017 16:50:18 +0200
X-Frame-Options: SAMEORIGIN
Message-ID: <149935261885.25752.7388914440262498594#HOSTNAME>
Transfer-Encoding: chunked
Server: pyas2-webserver
Based on sources: pyAS2 documentation, OpenSSL documentation, NCPDP SCRIPT message example.
I have recently been attempting to test a new AS2 implementation using the approach described by Andrew Haritonkin in his answer to this question.
I found a few quirks that are worth mentioning in their own right.
openssl smime -encrypt by default emits - logically enough - S/MIME formatted content. This is not accepted by my AS2 decoder - for reference, this is the Azure Logic Apps AS2 decode action. I found that the correct format is specified by adding the -outform DER option to the encrypt step, which causes openssl to emit the encrypted message in DER format
I was starting with a .pfx certificate generated by Azure Key Vault. I couldn't find a way to get openssl to accept this for signing, so I had to convert it to PEM first. The command line openssl pkcs12 -in mycert.pfx -out mycert.pem -passin pass: -nodes seemed to do the trick. -passin pass: specifies the private key password for the input certificate as being the empty string, which it is by default when generated by Azure. Not doing this causes openssl to prompt for the password, which failed the whole process in the context I was using it (a Docker container). -nodes means "no DES" not "nodes" and indicates that the private key should not be encrypted or protected by a passphrase in the output certificate. Needless to say this sounds like quite a high-risk piece of data to be handing around because it has no protection whatsoever against compromise of the private key.
I'm using cUrl to request data from a corporate website site using a .cer certificate that they sent me.
This is the command:
cUrl --header "Content-Type: text/xml;charset=UTF-8" \
--data #bustaRequestISEE2015ConsultazioneAttestazione.xml \
-o bustaResponseISEE2015ConsultazioneAttestazione.xml \
--cert ./caaffabisrl.cer \
https://istitutonazionaleprevidenzasociale.spcoop.gov.it/PD
When I run it, I get this error message:
curl: (58) could not load PEM client certificate, OpenSSL error error:0906D06C:PEM routines:PEM_read_bio:no start line, (no key found, wrong pass phrase, or wro
ng file format?)
Is there anybody who can help me?
Tks, Cristiano.
It is not possible to connect to a TLS server with curl using only a client certificate, without the client private key. Either they forgot to send you the private key file, or, what they sent you was not the client certificate but the server certificate for verification.
The first thing I would try is using --cacert instead of --cert. That is, tell curl that this is the server's certificate that curl can use to verify that the server is who you think it is.
You can also try removing --cert and not using --cacert, and you will probably get an error that the server is not trusted. Then add the --insecure argument and see if that works. I would not keep that argument, as then you have no proof of who you are talking to.
My guess is that it is the server cert, and that using --cacert instead of --cert will solve the problem.
My guess is that your certificate file is a DER encoded binary certificate instead of base-64 encoded certificate. To covert the from binary to base-64, you can use OpenSSL.
openssl x509 -inform der -in certificate.cer -out certificate.pem
I always forget all the arguments and have the following site bookmarked, as it gives examples of how to convert pretty much any certificate format. https://www.sslshopper.com/ssl-converter.html
First, you need to specify whether you're expected to perform two-way TLS/SSL or MTLS (mutual TLS). This would typically be the reason for sending a certificate. If they sent the server certificate, but you can connect to the server with a browser, you can down load the certificate. If their server is configured to send the server certificate and CA chain, then you can get the entire chain in a single request using "openssl s_client -connect [hostname:port] -showcerts". Save the certs in the console to a file, copying the cert blob(s) to individual cert files (cert1.crt, cert2.crt). However, if they are expecting MTLS and attempting to send a client certificate to you, either you've already generated a private key and CSR (certificate signing request) and send them the CSR. They would have then signed a certificate with their CA certificate using the CSR. The cert they returned would then need to be paired with the private key used to generate the CSR. They should not be generating the public/private key pair and sending them over mail. The private key should be stored security on the one system used to establish the connection. If it's one-way (server ssl only), then your client system (assuming it's not the browser), needs a truststore file, with the CA certificate chain installed and set to trusted. If the platform is Java, read Java's keytool documentation. Note, a keystore is for your systems public/private keypair. A truststore is for the CA certificates that you trust to sign public certificates that your system should trust as being authentic. You need to read any of the PKI x509 overviews by DigiCert, SSLABS, Sectigo, etc.