Error when setting Root Certificate on SIM800L - ssl

I desperately need some help on 2 questions:
I'm trying to connect a LILYGO TTGO T-Call SIM800L (IP5306 20190610) to AWS IoT, which requires an SSL authentication through a Root Certificate, a Client Certificate and a Private Key.
I'm able to successfully create the .crt files, write on them and set the Client Certificate through the AT Command:
modem.sendAT(GF("+SSLSETCERT=C:\User\clientcert.crt"));
But when I try to set the RootCA like this:
modem.sendAT(GF("+SSLSETROOT=C:\User\rootca.crt,1188"));
the GSM module returns "ERROR".
This is the documentation I'm using as a reference (Page 13):
https://microchip.ua/simcom/2G/Application%20Notes/SIM800%20Series_SSL_Application%20Note_V1.05.pdf
In the "Reference" section of the command "AT+SSLSETROOT", it's written "The files to be imported must be binary encoded". This confused me a little, so at first I simply wrote on the file the string format of the certificate, but I've also tried to change the extension to .der (which technically is the binary encoded format for certificates). I've tried to write on the file the hexdump version of the string format, and tried all the other avaiable extensions (.crt, .cer, .pem, .p12).
I've also tried to update the firmware following the procedure here: https://github.com/Xinyuan-LilyGO/LilyGo-T-Call-SIM800/blob/master/doc/How%20to%20update%20firmware.md
The only difference is that I used the Download Tool v1.10, because the other versions of the tool hanged on "Waiting" whenever I started the update. I retrieved it from here: https://simcom.ee/documents/?dir=SIM800x
I saw on other discussions that the SIM800L does not support TLS 1.2, but on page 6 of the SIM800 documentation regarding SSL, it's reported that "SIM800 series support SSL2.0, SSL3.0, TLS1.0 and TLS1.2."
On top of that, the command "AT+CIPSSL=1" works fine since it returns "OK".
I also read this: https://github.com/vshymanskyy/TinyGSM/issues/29#issuecomment-328802556
I'm attaching the function that sets up the certificates.
void setCertificates() {
modem.getModemInfo();
modem.sendAT(GF("+FSCREATE=C:\\User\\rootca.crt"));
modem.waitResponse();
modem.sendAT(GF("+FSCREATE=C:\\User\\clientcert.crt"));
modem.waitResponse();
char rootcertific[1188];
strcpy(rootcertific,rootCA);
modem.sendAT(GF("+FSWRITE=C:\\User\\rootca.crt,0,1188,1"));
modem.waitResponse(">");
SerialAT.print(rootcertific);
modem.waitResponse();
delay(1000 / portTICK_PERIOD_MS);
char clientcertific[2903];
strcpy(clientcertific,certificate_pem_crt);
modem.sendAT(GF("+FSWRITE=C:\\User\\clientcert.crt,0,2900,2"));
modem.waitResponse(">");
Serial1.print(strcat(clientcertific,private_pem_key));
modem.waitResponse();
delay(1000 / portTICK_PERIOD_MS);
modem.sendAT(GF("+FSREAD=C:\\User\\rootca.crt,0,1188,1"));
modem.waitResponse();
modem.waitResponse();
modem.sendAT(GF("+FSREAD=C:\\User\\clientcert.crt,0,2900,1"));
modem.waitResponse();
modem.waitResponse();
modem.sendAT(GF("+SSLSETROOT=C:\\User\\rootca.crt,1188"));
modem.waitResponse();
modem.sendAT(GF("+SSLSETCERT=C:\\User\\clientcert.crt"));
modem.waitResponse();
}
On page 11 of the SIM800 SSL documentation, the command AT+SSLSETCERT is described as "Client Client Certificate File with Private Key". Does this mean that I somehow have to write both the Client Cert and the Private Key on the same file?
As you might have noticed in the function, I used "strcat(clientcertific,private_pem_key)" and it returns no error, but I'm not sure whether this is the correct way to do this.
If anyone can help me out on this one I'd be infinitely grateful. It's been keeping me stuck for almost a month now.
Thank you!

Related

WebSocketpp handshake issue with TLS

I have been learning with WebSocket++ and built some of the server examples (Windows 10 Visual Studio 2019). The non-TLS examples work without issues, however, the TLS-enabled examples (echo_server_both.cpp and echo_server_tls.cpp) can't do the handshake. I am very new to web development in general so I know I must be doing something wrong with regards to the certificate and keys.
I am testing the servers with WebSocket King client, an extension of Google Chrome that connects correctly to other websocket servers like wss://echo.websocket.org and to my own localhost when I don't use TLS.
The echo_server_both example comes with a server.pem file, and the echo_server_tls example comes with server.pem and dh.pem. I have used the same files that come with the samples, and I have also tried generating and registering my own .pem files using openSSL. In both cases I get this when the client tries to connect:
[2021-06-29 20:51:21] [error] handle_transport_init received error: sslv3 alert certificate unknown
[2021-06-29 20:51:21] [fail] WebSocket Connection [::1]:63346 - "" - 0 asio.ssl:336151574 sslv3 alert certificate unknown
[2021-06-29 20:51:21] [info] asio async_shutdown error: asio.ssl:336462231 (shutdown while in init)
I discovered these errors after I edited handle_init() in tls.hpp, following a suggestion in another site, to look like this:
void handle_init(init_handler callback,lib::asio::error_code const & ec) {
if (ec) {
//m_ec = socket::make_error_code(socket::error::tls_handshake_failed);
m_ec = ec;
} else {
m_ec = lib::error_code();
}
callback(m_ec);
}
This change let the actual openSSL error to show in the console, otherwise it would show a generic "handshake failed" error.
I know I'm not doing what I should with the certificates, but I have no idea where else to look or what to do next. Can anyone here help please? Should I use the .pem files that come with the examples, or should I generate my own? in case I should generate my own, what would be the openSSL command to do that correctly and how do I tell my PC to recognize these as valid so that the server works?
Found the problem: WebSocket++ will not accept a self-signed certificate (the ones you can create directly in your own PC using OpenSSL or the Windows utilities). There is no way around it. You must have a valid, authority-validated and endorsed certificate. You can get such a certificate for free (valid only for 90 days) from https://zerossl.com/. The site has detailed instructions on how to request, obtain and install a certificate. After getting a valid certificate and installing it on my server, everything worked as it should.

go-swagger TLS Config

I generated a golang server with go-swagger. I set the scheme to https (and only https). When I startup my server I get a TLS error.
the required flags `--tls-certificate` and `--tls-key` were not specified
It is clear that I haven't properly set my TLS flags but I really don't know the best way to to do this in go with go-swagger.
Anyone have any experience setting up TLS with go-swagger as I couldn't find any good links?
Thank you.
When you get certificates you get a private key file and a public key (certificate) file.
Here's an example of how they are used: https://github.com/go-swagger/go-swagger/tree/master/examples/todo-list#run-full-server
./todo-list-server --tls-certificate mycert1.crt --tls-key mycert1.key

JMeter: "javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake" while recording in JMeter [duplicate]

I am getting javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake exception when I try to do HTTPS Post of a web service through internet. But same code works for other internet hosted web services. I tried many things, nothing is helping me. I posted my sample code here. Can anyone please help me to resolve this problem?
public static void main(String[] args) throws Exception {
String xmlServerURL = "https://example.com/soap/WsRouter";
URL urlXMLServer = new URL(xmlServerURL);
// URLConnection supports HTTPS protocol only with JDK 1.4+
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(
"xxxx.example.com", 8083));
HttpURLConnection httpsURLConnection = (HttpURLConnection) urlXMLServer
.openConnection(proxy);
httpsURLConnection.setRequestProperty("Content-Type","text/xml; charset=utf-8");
//httpsURLConnection.setDoInput(true);
httpsURLConnection.setDoOutput(true);
httpsURLConnection.setConnectTimeout(300000);
//httpsURLConnection.setIgnoreProxy(false);
httpsURLConnection.setRequestMethod("POST");
//httpsURLConnection.setHostnameVerifier(DO_NOT_VERIFY);
// send request
PrintWriter out = new PrintWriter(
httpsURLConnection.getOutputStream());
StringBuffer requestXML = new StringBuffer();
requestXML.append(getProcessWorkOrderSOAPXML());
// get list of user
out.println(requestXML.toString());
out.close();
out.flush();
System.out.println("XML Request POSTed to " + xmlServerURL + "\n");
System.out.println(requestXML.toString() + "\n");
//Thread.sleep(60000);
// read response
BufferedReader in = new BufferedReader(new InputStreamReader(
httpsURLConnection.getInputStream()));
String line;
String respXML = "";
while ((line = in.readLine()) != null) {
respXML += line;
}
in.close();
// output response
respXML = URLDecoder.decode(respXML, "UTF-8");
System.out.println("\nXML Response\n");
System.out.println(respXML);
}
Full stacktrace:
Exception in thread "main" javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:946)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1091)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
at com.labcorp.efone.vendor.TestATTConnectivity.main(TestATTConnectivity.java:43)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.read(InputRecord.java:482)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
... 8 more
Actually, there are two scenarios here. When I work as a standalone Java program I am getting the above exception. But when I try to execute in weblogic application server, I am getting the below exception: Any clue what could be the reason?
java.io.IOException: Connection closed, EOF detected
at weblogic.socket.JSSEFilterImpl.handleUnwrapResults(JSSEFilterImpl.java:637)
at weblogic.socket.JSSEFilterImpl.unwrapAndHandleResults(JSSEFilterImpl.java:515)
at weblogic.socket.JSSEFilterImpl.doHandshake(JSSEFilterImpl.java:96)
at weblogic.socket.JSSEFilterImpl.doHandshake(JSSEFilterImpl.java:75)
at weblogic.socket.JSSEFilterImpl.write(JSSEFilterImpl.java:448)
at weblogic.socket.JSSESocket$JSSEOutputStream.write(JSSESocket.java:93)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
at weblogic.net.http.HttpURLConnection.writeRequests(HttpURLConnection.java:192)
at weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:433)
at weblogic.net.http.SOAPHttpsURLConnection.getInputStream(SOAPHttpsURLConnection.java:37)
at com.labcorp.efone.service.impl.WorkOrderServiceImpl.processATTWorkOrder(ATTWorkOrderServiceImpl.java:86)
at com.labcorp.efone.bds.WorkOrderBusinessDelegateImpl.processATTWorkOrder(WorkOrderBusinessDelegateImpl.java:59)
at com.labcorp.efone.actions.ATTWorkOrderAction.efonePerformForward(ATTWorkOrderAction.java:41)
at com.labcorp.efone.actions.EfoneAction.efonePerformActionForward(EfoneAction.java:149)
at com.labcorp.efone.actions.EfoneAction.execute(EfoneAction.java:225)
at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:484)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1482)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:525)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:751)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:844)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:280)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:254)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:136)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:341)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:25)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at com.labcorp.efone.security.EfoneAuthenticationFilter.doFilter(EfoneAuthenticationFilter.java:115)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3367)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3333)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)
at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2220)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2146)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2124)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1564)
at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:254)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:295)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:254)
Exception: java.io.IOException: Connection closed, EOF detected
Java 7 defaults to TLS 1.0, which can cause this error when that protocol is not accepted. I ran into this problem with a Tomcat application and a server that would not accept TLS 1.0 connections any longer. I added
-Dhttps.protocols=TLSv1.1,TLSv1.2
to the Java options and that fixed it. (Tomcat was running Java 7.)
I faced the same problem and solved it by adding:
System.setProperty("https.protocols", "TLSv1,TLSv1.1,TLSv1.2");
before openConnection method.
Not an answer yet, but too much for a comment. This is clearly not a server cert problem; the symptoms of that are quite different. From your system's POV, the server appears to be closing during the handshake. There are two possibilities:
The server really is closing, which is a SSL/TLS protocol violation though a fairly minor one; there are quite a few reasons a server might fail to handshake with you but it should send a fatal alert first, which your JSSE or the weblogic equivalent should indicate. In this case there may well be some useful information in the server log, if you are able (and permitted) to communicate with knowledgeable server admin(s). Or you can try putting a network monitor on your client machine, or one close enough it sees all your traffic; personally I like www.wireshark.org. But this usually shows only that the close came immediately after the ClientHello, which doesn't narrow it down much. You don't say if you are supposed to and have configured a "client cert" (actually key&cert, in the form of a Java privateKeyEntry) for this server; if that is required by the server and not correct, some servers may perceive that as an attack and knowingly violate protocol by closing even though officially they should send an alert.
Or, some middlebox in the network, most often a firewall or purportedly-transparent proxy, is deciding it doesn't like your connection and forcing a close. The Proxy you use is an obvious suspect; when you say the "same code" works to other hosts, confirm if you mean through the same proxy (not just a proxy) and using HTTPS (not clear HTTP). If that isn't so, try testing to other hosts with HTTPS through the proxy (you needn't send a full SOAP request, just a GET / if enough). If you can, try connecting without the proxy, or possibly a different proxy, and connecting HTTP (not S) through the proxy to the host (if both support clear) and see if those work.
If you don't mind publishing the actual host (but definitely not any authentication credentials) others can try it. Or you can go to www.ssllabs.com and request they test the server (without publishing the results); this will try several common variations on SSL/TLS connection and report any errors it sees, as well as any security weaknesses.
A first step to diagnose the issue is by starting the client - and if you are running the server yourself, a private test instance of the server - by starting Java with the VM option:
-Djavax.net.debug=all
See also https://blogs.oracle.com/java-platform-group/entry/diagnosing_tls_ssl_and_https
I encountered a similar problem with glassfish application server and Oracle JDK/JRE but not in Open JDK/JRE.
When connecting to a SSL domain I always ran into:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
...
Caused by: java.io.EOFException: SSL peer shut down incorrectly
The solution for me was to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files because the server only understood certificates that are not included in Oracle JDK by default, only OpenJDK includes them.
After installing everything worked like charme.
JCE 7: http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html
JCE 8: http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
I think you are missing your certificates.
You can try generating them by using InstallCerts app. Here you can see how to use it:
https://github.com/escline/InstallCert
Once you get your certificate, you need to put it under your security directory within your jdk home, for example:
C:\Program Files\Java\jdk1.6.0_45\jre\lib\security
Let me know if it works.
I ran into a similar issue and found I was hitting the wrong port. After fixing the port things worked great.
In my case, I got this problem because I had given the server a non-existent certificate, due to a typo in the config file. Instead of throwing an exception, the server proceeded like normal and sent an empty certificate to the client. So it might be worth checking to make sure that the server is providing the correct response.
I experienced this error while using the Jersey Client to connect to a server. The way I resolved it was by debugging the library and seeing that it actually did receive an EOF the moment it tried to read. I also tried connecting using a web browser and got the same results.
Just writing this here in case it ends up helping anyone.
You May Write this below code insdie your current java programme
System.setProperty("https.protocols", "TLSv1.1");
or
System.setProperty("http.proxyHost", "proxy.com");
System.setProperty("http.proxyPort", "911");
Thanks to all for sharing your answers and examples. The same standalone program worked for me by small changes and adding the lines of code below.
In this case, keystore file was given by webservice provider.
// Small changes during connection initiation..
// Please add this static block
static {
HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier()
{ #Override
public boolean verify(String hostname, SSLSession arg1) {
// TODO Auto-generated method stub
if (hostname.equals("X.X.X.X")) {
System.out.println("Return TRUE"+hostname);
return true;
}
System.out.println("Return FALSE");
return false;
}
});
}
String xmlServerURL = "https://X.X.X.X:8080/services/EndpointPort";
URL urlXMLServer = new URL(null,xmlServerURL,new sun.net.www.protocol.https.Handler());
HttpsURLConnection httpsURLConnection = (HttpsURLConnection) urlXMLServer .openConnection();
// Below extra lines are added to the same program
//Keystore file
System.setProperty("javax.net.ssl.keyStore", "Drive:/FullPath/keystorefile.store");
System.setProperty("javax.net.ssl.keyStorePassword", "Password"); // Password given by vendor
//TrustStore file
System.setProperty("javax.net.ssl.trustStore"Drive:/FullPath/keystorefile.store");
System.setProperty("javax.net.ssl.trustStorePassword", "Password");
I encountered this problem with Java 1.6. Running under Java 1.7 fixed my particular rendition of the problem. I think the underlying cause was that the server I was connecting to must have required stronger encryption than was available under 1.6.
I had the same error, but in my case it was caused by the DEBUG mode in Intellij IDE. The debug slowed down the library and then server ended communication at handshake phase. The standard "RUN" worked perfectly.
I run my application with Java 8 and Java 8 brought security certificate onto its trust store. Then I switched to Java 7 and added the following into VM options:
-Djavax.net.ssl.trustStore=C:\<....>\java8\jre\lib\security\cacerts
Simply I pointed to the location where a certificate is.
I was using the p12 which I exported with Keychain in my MacBook, however, it didn't work on my java-apns server code. What I had to do was to create a new p12 key as stated here, using my already generated pem keys:
openssl pkcs12 -export -in your_app.pem -inkey your_key.pem -out your_app_key.p12
Then updated the path to that new p12 file and everything worked perfectly.
How you would solve it is by going to
Settings
Search"Network"
Choose "Use IDEA general proxy settings as default Subversion"
As per https://kb.informatica.com/solution/23/Pages/69/570664.aspx adding this property works
CryptoProtocolVersion=TLSv1.2
With base at TLSv1.2 ALERT: fatal, handshake_failure I obtained after debug with this thread previos answer
-Djavax.net.debug=all
I went to https://www.ssllabs.com/and observed that the web server required a SSLv3 connection deprecate at june 2015, and deprecated at JDKu31 Release notes
I edited the ${java_home}/jre/lib/security/java.security at the line
jdk.tls.disabledAlgorithms=SSLv3, RC4, DES, MD5withRSA, DH keySize < 1024,
EC keySize < 224, 3DES_EDE_CBC, anon, NULL
to
jdk.tls.disabledAlgorithms= RC4, DES, MD5withRSA, DH keySize < 1024,
EC keySize < 224, 3DES_EDE_CBC, anon, NULL
As a final step I got this error
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target [javax.net.ssl.SSLHandshakeException]
I fixed this intalling the cert with the java keytool, following this answer PKIX path building failed” and “unable to find valid certification path to requested target”
I get this error when specifying a https url and in the same url explicitly specifying an http port (instead of an https port). Removing the explicit port :8080 solved the issue for me.
Adding certificates to Java\jdk\jre\lib\security folder worked for me. If you are using Chrome click on the green bulb [https://support.google.com/chrome/answer/95617?p=ui_security_indicator&rd=1] and save the certificate in security folder.
I faced the same issue once. I think its because of the URL
String xmlServerURL = "https://example.com/soap/WsRouter";
Check whether its a proper one or not ??
javax.net.ssl.SSLHandshakeException is because the server not able to connect to the specified URL because of following reason-
Either the identity of the website is not verified.
Server's certificate does not match the URL.
Or, Server's certificate is not trusted.
This is what solve my problem.
If you are trying to use debugger make sure you breakpoint is not on URL or URLConnection just put your breakpoint on BufferReader or inside while loop.
If nothing works try using apache library http://hc.apache.org/index.html.
no SSL, no JDK update needed, no need to set properties even, just simple trick :)

SSL certificate pinning with libcurl

I'd like to know if this example is enough to provide certificate pinning with libcurl:
http://curl.haxx.se/libcurl/c/cacertinmem.html
because I have found that curl also allows http://curl.haxx.se/libcurl/c/CURLOPT_PINNEDPUBLICKEY.html
Since I'll be using a self-signed certificate and only trust on it I don't know if it's truly necessary to pinn it too.
resume: Can the connection be compromised if I only add my certificate (self-signed) to the x509 certificate store like the example? do I need to add extra checks? do I need to use the CURLOPT_PINNEDPUBLICKEY option?
Thanks.
You can find another example in the implementation of the new curl option in git 2.8 (March 2016):
See commit aeff8a6 (15 Feb 2016) by Christoph Egger (siccegge).
(Merged by Junio C Hamano -- gitster -- in commit e79112d, 24 Feb 2016)
http: implement public key pinning
Add the http.pinnedpubkey configuration option for public key pinning. It allows any string supported by libcurl -- base64(sha256(pubkey)) or filename of the full public key.
If cURL does not support pinning (is too old) output a warning to the user.
The git config man page mentions:
http.pinnedpubkey:
Public key of the https service.
It may either be the filename of a PEM or DER encoded public key file or a string starting with 'sha256//' followed by the base64 encoded sha256 hash of the public key.
See also libcurl 'CURLOPT_PINNEDPUBLICKEY'.
git will exit with an error if this option is set but not supported by cURL.
With Git 2.34 (Q4 2021), HTTPS error handling is updated when it comes to SSL certificate pinning:
See commit 3e8084f (24 Sep 2021) by Ævar Arnfjörð Bjarmason (avar).
(Merged by Junio C Hamano -- gitster -- in commit 97492aa, 11 Oct 2021)
http: check CURLE_SSL_PINNEDPUBKEYNOTMATCH when emitting errors
Signed-off-by: Ævar Arnfjörð Bjarmason
Change the error shown when a http.pinnedPubKey doesn't match to point the http.pinnedPubKey variable added in aeff8a6 ("http: implement public key pinning", 2016-02-15, Git v2.8.0-rc0 -- merge listed in batch #8), e.g.:
git -c http.pinnedPubKey=sha256/someNonMatchingKey ls-remote https://github.com/git/git.git
fatal: unable to access 'https://github.com/git/git.git/' with http.pinnedPubkey configuration: SSL: public key does not match pinned public key!
Before this we'd emit the exact same thing without the " with http.pinnedPubkey configuration".
The advantage of doing this is that we're going to get a translated message (everything after the ":" is hardcoded in English in libcurl), and we've got a reference to the git-specific configuration variable that is causing the error.
Unfortunately we can't test this easily, as there are no tests that require https:// in the test suite, and t/lib-httpd.sh doesn't know how to set up such tests.
See this thread for the start of a discussion about what it would take to have divergent "t/lib-httpd/apache.conf" test setups.

ColdFusion CFHTTP I/O Exception: peer not authenticated - even after adding certs to Keystore

I'm currently working with a payment processor. I can browse to the payment URL from our server, so it's not a firewall issue, but when I try to use CFHTTP I get a I/O Exception: peer not authenticated. I've downloaded and installed their latest security cert into cacerts keystore and restarted CF and am still getting the same error. Not only have I installed the providers cert, but also the 2 other Verisign certificate authority certs in the certificate chain. The cert is one of the newer Class 3 Extended Validation certs.
Has anybody come across this before and found a solution?
A colleague of mine found the following after experiencing the same issue when connecting to a 3rd party.
http://www.coldfusionjedi.com/index.cfm/2011/1/12/Diagnosing-a-CFHTTP-issue--peer-not-authenticated
https://www.raymondcamden.com/2011/01/12/Diagnosing-a-CFHTTP-issue-peer-not-authenticated/
We used the solution provided in the comment by Pete Freitag further down the page. It works, but I think should be used with caution, as it involves dynamically removing and adding back in a particular property of the JsafeJCE provider.
For the sake of archiving, here is the original content of Pete Freitag's comment:
I've narrowed this down a bit further, and removing the
KeyAgreement.DiffieHellman from the RSA JsafeJCE provider (which
causes the default sun implementation to be used instead) seams to
work, and probably has less of an effect on your server than removing
the entire provider would. Here's how you do it:
<cfset objSecurity = createObject("java", "java.security.Security") />
<cfset storeProvider = objSecurity.getProvider("JsafeJCE") />
<cfset dhKeyAgreement = storeProvider.getProperty("KeyAgreement.DiffieHellman")>
<!--- dhKeyAgreement=com.rsa.jsafe.provider.JSA_DHKeyAgree --->
<cfset storeProvider.remove("KeyAgreement.DiffieHellman")>
Do your http call, but pack the key agreement if you want:
<cfset storeProvider.put("KeyAgreement.DiffieHellman", dhKeyAgreement)>
I figured this out by using the SSLSocketFactory to create a https
connection, which provided a bit more details in the stack trace, than
when using cfhttp:
yadayadayada Caused by: java.security.InvalidKeyException: Cannot
build a secret key of algorithm TlsPremasterSecret at
com.rsa.jsafe.provider.JS_KeyAgree.engineGenerateSecret(Unknown
Source) at javax.crypto.KeyAgreement.generateSecret(DashoA13*..) at
com.sun.net.ssl.internal.ssl.DHCrypt.getAgreedSecret(DHCrypt.java:166)
Would be great if the exception thrown from ColdFusion was a bit less
generic.
specific to coldfusion 8 with an webserver with modern ssl ciphers:
I use coldfusion 8 on JDK 1.6.45 and had problems with <cfdocument ...> giving me just red crosses instead of images, and also with cfhttp not able to connect to the local webserver with ssl.
my test script to reproduce with coldfusion 8 was
<CFHTTP URL="https://www.onlineumfragen.com" METHOD="get" ></CFHTTP>
<CFDUMP VAR="#CFHTTP#">
this gave me the quite generic error of " I/O Exception: peer not authenticated."
I then tried to add certificates of the server including root and intermediate certificates to the java keystore and also the coldfusion keystore, but nothing helped.
then I debugged the problem with
java SSLPoke www.onlineumfragen.com 443
and got
javax.net.ssl.SSLException: java.lang.RuntimeException: Could not generate DH keypair
and
Caused by: java.security.InvalidAlgorithmParameterException: Prime size must be
multiple of 64, and can only range from 512 to 1024 (inclusive)
at com.sun.crypto.provider.DHKeyPairGenerator.initialize(DashoA13*..)
at java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:627)
at com.sun.net.ssl.internal.ssl.DHCrypt.<init>(DHCrypt.java:107)
... 10 more
I then had the idea that the webserver (apache in my case) had very modern ciphers for ssl and is quite restrictive (qualys score a+) and uses strong Diffie-Hellman groups with more than 1024 bits. obviously, coldfusion and java jdk 1.6.45 can not manage this.
Next step in the odyssee was to think of installing an alternative security provider for java, and I decided for bouncy castle.
see also http://www.itcsolutions.eu/2011/08/22/how-to-use-bouncy-castle-cryptographic-api-in-netbeans-or-eclipse-for-java-jse-projects/
I then downloaded the
bcprov-ext-jdk15on-156.jar
from http://www.bouncycastle.org/latest_releases.html and installed it under
C:\jdk6_45\jre\lib\ext or where ever your jdk is, in original install of coldfusion 8 it would be under C:\JRun4\jre\lib\ext but I use a newer jdk (1.6.45) located outside the coldfusion directory. it is very important to put the bcprov-ext-jdk15on-156.jar in the \ext directory (this cost me about two hours and some hair ;-)
then I edited the file C:\jdk6_45\jre\lib\security\java.security (with wordpad not with editor.exe!) and put in one line for the new provider. afterwards the list looked like
#
# List of providers and their preference orders (see above):
#
security.provider.1=org.bouncycastle.jce.provider.BouncyCastleProvider
security.provider.2=sun.security.provider.Sun
security.provider.3=sun.security.rsa.SunRsaSign
security.provider.4=com.sun.net.ssl.internal.ssl.Provider
security.provider.5=com.sun.crypto.provider.SunJCE
security.provider.6=sun.security.jgss.SunProvider
security.provider.7=com.sun.security.sasl.Provider
security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI
security.provider.9=sun.security.smartcardio.SunPCSC
security.provider.10=sun.security.mscapi.SunMSCAPI
(see the new one in position 1)
then restart coldfusion service completely.
you can then
java SSLPoke www.onlineumfragen.com 443 (or of course your url!)
and enjoy the feeling...
and of course
<cfhttp and <cfdocument worked like a charm and like before we "hardened" our ssl ciphers in apache!
what a night and what a day. Hopefully this will help (partially or fully) to someone out there. if you have questions, just mail me at info ... (domain above).
Did you add it to the correct keystore? Remember that ColdFusion uses it's own Java instance. I spent several hours on this once before remembering that fact. The one you want is at somewhere like /ColdFusion8/runtime/jre/lib/security/
Try with this in CMD
C:\ColdFusion9\runtime\jre\bin>
keytool -import -keystore ../lib/security/cacerts
-alias uniquename -file certificatename.cer
Note: We must choose the correct keystore present inside the security folder,as there are other keystore file present inside bin.If we will import the certificate to those key stores it will not work.
What I just found out was referenced at this article: http://kb2.adobe.com/cps/400/kb400977.html and a few other places after a lot of digging.
If you are looking at this article you have most likely inserted your "server.crt" certificate in the proper root locations and you have probably inserted it into the cacerts file in /ColdFusion9/runtime/jre/lib/security using the command
\ColdFusion9\runtime\jre\bin\keytool -import -v -alias someServer-cert -file someServerCertFile.crt -keystore cacerts -storepass changeit
(if you haven't done this, do it now).
The thing I was running into was that I am setting up ssl on my localhost so after doing these steps I was still getting the same error.
As it turns out, you need to also insert your "server.crt" into the "trustStore" file commonly located in /ColdFusion9/runtime/jre/lib using the command
\ColdFusion9\runtime\jre\bin\keytool -import -v -alias someServer-cert -file someServerCertFile.cer -keystore trustStore -storepass changeit
Hopefully this will save someone time.
I am using JRun. After trying a lot of different things I came across a snippet of information that was applicable in my setup. I had configured an (1)HTTPS SSLService with my own truststore file. This caused the piece of information in the following link to become important.
http://helpx.adobe.com/coldfusion/kb/import-certificates-certificate-stores-coldfusion.html
Note: If you are using JRun as the underlying J2EE server (either the
Server Configuration or the Multiserver/J2EE with JRun Configuration)
and have enabled SSL for the internal JRun Web server (JWS), you will
need to import the certificate to the truststore defined in the
jrun.xml file for the Secure JWS rather than the JRE key store. By
default, the file is called "trustStore" and is typically located
under jrun_root/lib for the Multiserver/J2EE with JRun configuration
or cf_root/runtime/lib for the ColdFusion Server configuration. You
use the same Java keytool to manage the trustStore.
Here is the excerpt from my jrun.xml file:
<service class="jrun.servlet.http.SSLService" name="SSLService">
<attribute name="port">8301</attribute>
<attribute name="keyStore">/app/jrun4/cert/cfusion.jks</attribute>
<attribute name="trustStore">/app/jrun4/cert/truststore.jks</attribute>
<attribute name="name">SSLService</attribute>
<attribute name="bindAddress">*</attribute>
<attribute name="socketFactoryName">jrun.servlet.http.JRunSSLServerSocketFactory</attribute>
<attribute name="interface">*</attribute>
<attribute name="keyStorePassword">cfadmin</attribute>
<attribute name="deactivated">false</attribute>
</service>
Once I imported the certificate into this truststore (/app/jrun4/cert/truststore.jks) it worked after restarting ColdFusion.
(1) http://helpx.adobe.com/legacy/kb/ssl-jrun-web-server-connector.html
Adding the cert to the keystore did not work for me on CF9 Enterprise.
Ended up using the CFX tag, CFX_HTTP5.
I realize this is a very old discussion, but since it still comes up near the top of a search for the "peer not authenticated" error in CF, I wanted to share that for most people, the simple solution is to update the JVM that CF uses. (More in a moment on how to do that.)
The cause of the problem is generally that the service BEING CALLED has made a change that requires a later version of TLS or SSL (and perhaps a change to supported algorithms). Later JVMs offer that, while earlier ones did not. Since CF runs atop the JVM, it's the calls out of CF (va cfhttp, cfldap, cfmail, etc) that "suddenly" start to fail.
And sure, sometimes a cert update is the answer (and even then, you have to do it carefully), but it's not always needed. And updating the JVM also gives other benefits, in terms of bug fixes, etc.
The only challenge is knowing what JVM your version of CF will support. (But even people still running on an old CF version have found that updating the JVM CF uses has solved this problem and not caused any others.)
I discuss all this in a 2019 post:
https://coldfusion.adobe.com/2019/06/error-calling-cf-via-https-solved-updating-jvm/
Hope that may help someone.