Opening an SSL web-browser connection in HtmlUnit library - ssl

I've searched through web for couple hours on this issue, and none of the answers I found didn't really fit into my problem, so here's me, asking my first-ever question in SOF.
So, I'm trying to open a web-browser from a java program using the htmlunit library. The web site I need to connect requires SSL connection, and the certificate is stored in a USB key. Its iKey2023 product.
The system used to work(I did not write it), but one of the certificates in the USB key expired, so it automatically moved on to the next one (there were 4 certificates in total), and it suddenly stopped working.
It is giving me javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated error.
I'm back home now and I forgot the exact name of the method, but I remember the following.
Browser instance is created, using IE8
browser.setWebConnection method was called. This method, according to the API, is an internal API.
Make connection to the website by passing the URL as parameter
It's throwing the exception at step 3.
Some more details. The little details might be incorrect but I'm trying to describe a big picture.
At step 2, the method requites WebConnection object as a parameter, and there is a implementation of that interface. Within this implementation, a keystore is created using sun.security.pkcs11.SunPKCS11(configFileInputStream) (did I spell that correctly?)
It was sth like this.
Provider p = new sun.security.pkcs11.SunPKCS11(configFileInputStream);
Security.addProvider(p);
And create a keystore from this provider.
Using this keystore, within the WebConnection implementation, it creates a SSLSocket.
So, after the certificate has been switched to a new one, it's not picking up the certificate correctly.
Here's what I've tried.
I've tried to use different methods in the htmlunit library, something like setSecurityProvider, and I tried to put the Provider object created in above code snippet. I got class cast exception.
I tried to manually set the system properties(trustStore, trustStorePassword, keyStore, etc). In order to do this, I wanted to export the certificate out of the USB key, but it did not let me export the private key out from it, so I could not really create a valid PKCS12 file out of it (openSSL wanted a private key file along with .pem file for conversion, and I did not have that key file).
They did not work, and I'm so stuck right now.

I have a similar issue. In my case, an admin changed the certificate and I began encountering the same SSLPeerUnverifiedException.
I found that I can set the WebClient to use insecureSSL (prior to calling getPage())and I will no longer get the exception.
webClient.setUseInsecureSSL(true);
This however, doesn't resolve the issue as the server basically doesn't authenticate the client.
Its as if the WebClient is storing something that doesn't work with the new certificate.

Related

IBM MQ: Establishing an SSL connection

We're struggling to get IBM MQ to work across SSL.
We've been provided with the certificate chain for the remote host and installed into the Windows Certificate Store (Local Machine). These all look valid.
We're using the following connection properties:
connectionProperties.Add(MQC.SSL_PEER_NAME_PROPERTY, "other-server.com");
connectionProperties.Add(MQC.SSL_CIPHER_SUITE_PROPERTY, "TLS_RSA_WITH_AES_256_CBC_SHA256"); connectionProperties.Add(MQC.SSL_CIPHER_SPEC_PROPERTY, "TLS_RSA_WITH_AES_256_CBC_SHA256");
connectionProperties.Add(MQC.SSL_CERT_STORE_PROPERTY, "*SYSTEM");
connectionProperties.Add("CertificateLabel", "ibmwebspheremqmywindowsusernamewithoutdomain");
MQEnvironment.SSLCertRevocationCheck = true;
We've established that the "CertificateLabel" is the "Friendly name" in Windows parlance.
We've proven unencrypted communication and network-level configuration.
We're using 8.0.0.7 client.
These are the issues we've come across:
All secure communications fail with a 2538 error. (MQRC_HOST_NOT_AVAILABLE, https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_7.5.0/com.ibm.mq.tro.doc/q045380_.htm)
No success setting the Friendly Name to ibmwebspheremq and ibmwebspheremqmywindowsusername#domain and ibmwebspheremqmywindowsusernamewithoutdomain
General questions:
Are we correct in assuming that we can install generated certificates exclusively in the Windows Certificate Store?
Is the 2538 error even related to SSL communications? It feels like a network error, though there is that final point in the referenced error documentation.
Is there anywhere we can look for more informative error information? eg. relating to the SSL trust chain to see if there is an issue there?
The issue was the following line:
connectionProperties.Add(MQC.SSL_PEER_NAME_PROPERTY, "otherserver.com");
Turns out that:
It needs it in a canonical format, so DN=, etc.
You don't even need that line
Though we did learn a few things along the way:
The line:
connectionProperties.Add("CertificateLabel", "ibmwebspheremqmyusername");
Is the string ibmwebspheremq plus your Windows username (without your domain) and the label should be set on the Friendly name of your client machine's outgoing certificate NOT including the username.
The various folders inside your Windows certificate store are significant. The intermediate CAs should be correctly filed.

SAP SSL handshake failed

I'm trying to retrieve data from an open data api. I have downloaded the certificate from the site and imported it into STRUST (SSL Client Anonymous).
Then I created a HTTP connection to external server in SM59. In the beginning it worked fine, until last week when the api changed its URL and so its DNS.
Of course it could no longer be reached by the current host. So I did above steps again for the new URL (changed everything accordingly like hostname etc. in SM59), but this time I receive following error:
SSL handshake with 'hostname:port' failed: SSSLERR_CONN_CLOSED (-10)#Remote
Peer has closed the network connection##SapSSLSessionStartNB()==SSSLERR_CONN_CLOSED##
Anyone has an idea on how to solve this?
On another forum someone helped me solve the problem. He pointed me out that the problem lies with SNI see: https://security.stackexchange.com/questions/101965/ssl3-error-when-requesting-connection-using-tls-1-2/102018#102018
https://en.wikipedia.org/wiki/Server_Name_Indication
To solve this problem you need to add following parameter: icm/HTTPS/client_sni_enabled and set it to TRUE on the DEFAULT profile. Afterwards you need to restart the application server in order to activate the effects of the parameter.
Link to the full question on SCN: https://answers.sap.com/questions/473015/sap-ssl-handshake-failed.html
EDIT:
I came across this error again later on, but this time it seemed that the error was caused because we used a certificate with TLS 1.2 which was not supported by our system. You can check this link: https://launchpad.support.sap.com/#/notes/510007 we implemented number 7 to fix this.

EppLib.net - Which SSL Certificate to use

When using EppLib.net (and as per the Nominet instructions - http://registrars.nominet.uk/namespace/uk/registration-and-domain-management/registrar-systems/epp/testbed)
In order to verify the identity of the secure server you will need the 'Verisign Class 3 Public Primary Certification Authority' root certificate available free from www.verisign.com (the certificate is also distributed with most web browsers).
I am supplying the Verisign Class 3 PPCA Certs in .crt format (I can't export as a pfx) within the code (and have tried every cert installed on the server).
However each one fails with the error (when trying to negotiate the connection):
An existing connection was forcibly closed by the remote host.
I've confirmed that I can connect from the server to Nominet by running cURL
curl -v https://testbed-epp.nominet.org.uk:700
and that responds as I would expect - or at least without an error!
I have also tried to use the same cert that curl appears to be using without success.
Can anyone help with pointing me in the right direction with the SSL that I should be using or how to get hold of the correct certificate?
thanks in advance
For anyone having a similar issue, it wasn't a question of which SSL certificate to use, but to ensure the correct protocol was being used. Incidentally, I got it working initially using the curl-ca-bundle.crt provided with cURL.
This certificate, in combination with a change to the target .net framework for the EppLib.net project resolved my issue.
On cloning from Github the target was .net 4.0 and Nominet needs at least TLS1.1, which wasn't available in v4.
Therefore I also updated the target to 4.5.1 for the EppLib.net project and changed the SslProtocol from
SslProtocol.Tls to
SslProtocol.Tls12

Fiddler https error: "because they do not possess a common algorithm"

I am trying to monitor https traffic with Fiddler, using current newest version:2.4.4.5
I've successfully set up https, certificates and I can see the full https encrypted traffic for example browsing my bank's web site.
...however...
When I trying to monitor an other server I got this error message in the response window:
"Failed to secure existing connection for 77.87.178.160. A call to SSPI failed, see inner exception. InnerException: System.ComponentModel.Win32Exception: The client and server cannot communicate, because they do not possess a common algorithm"
For full Fiddler window see:
The client is not a in this case browser, but a custom client program, which communicates with its own server.
My question: Is this exception misleading and in reality some other error prevents the secure channel to set up?
...or...
We have still chance to monitor this https communication?
Thx in advance
What is the client program?
This error typically indicates that that client application is only offering certain HTTPS ciphers, and those ciphers are not supported by Fiddler.
However, in this case, the specific problem here is almost certainly this: http://blogs.msdn.com/b/ieinternals/archive/2009/12/08/aes-is-not-a-valid-cipher-for-sslv3.aspx
The client is trying to use AES with SSLv3, but that isn't one of the valid ciphers for SSL3. As a consequence, the connection fails.
You might be able to workaround this by clicking Rules > Customize Rules. Scroll down to the Main() function and add the following line within the function:
CONFIG.oAcceptedServerHTTPSProtocols =
System.Security.Authentication.SslProtocols.Ssl3;
Please let me know if this works.
NOTE Current versions of Fiddler offer a UI link for this: Look at the lis of enabled protocols on the HTTPS tab.
Unbelievably this issue is still present some 6 years later.
Just installed the latest version of Fiddle (v5.0.20194.41348), and sure enough on Win7 using Chrome or IE it keeps failing with the dreaded error:
"fiddler.network.https> HTTPS handshake to google.com (for #1) failed. System.ComponentModel.Win32Exception The client and server cannot communicate, because they do not possess a common algorithm"
After some hours of testing, I found a middle ground solution which seems to work with virtually all websites. The aim was to get the highest possible security with no errors in the log. Without needing to add any code, simply changing this line under Tools > Options > HTTPS > Protocols is what worked for me (just copy and paste it):
<client>;ssl3;tls1.1;tls1.2
Basically removed the ssl2 and tls1.0 protocols which leaves us with some pretty decent security and no errors so far. Having spent hours of frustration with this error, hope someone out there might find this useful, and a big thanks to EricLaw who discovered the root of the problem.
Yes I too have seen this error when working outside of fiddler and it was connected with AuthenticateAsServer but only went wrong when using IE10 and not Chrome as the browser.
Odd thing is that it did not break all the time for IE10 using SslProtocols.Tls for the protocol so I will add a bit of code to switch the protocol if one fails
The protocol that can be used also seems to change on if you are using a proxy server like Fiddler or using an invisible server by hijacking the DNS via the hosts file to divert traffic to the server

Sporadic invalid_request 400 errors connecting to Shopify /admin/oauth/access_token

I am using a java raw HTTP client to connect to Shopify API (specifically, using Play Framework with the non-defualt sync driver which is actually the JDK's default driver).
My application usually manages to connect successfully and convert the temporary access token into a permanent one by calling the /admin/oauth/access_token endpoint.
However, sometimes I get this error result from the API:
Generic Error(400)
{"error":"invalid_request"}
I haven't been able to reproduce the issue with my test stores - I've tried installing a fresh store, reinstalling existing stores after uninstalling, I'm not sure why this call sometimes fail and how to debug it. The API call still continues to succeed for some stores using our application.
Some things that I am doing:
Even if the URL of the store is on a custom domain, I'm always using the https://foo.myshopfiy.com/admin/oauth/access_token URL and not the URL of the custom domain, to prevent a redirect.
I am always using an https URL and never an http one, again to prevent a redirect (we noticed a few issues with redirect with the Java HTTP client, so we aim to have zero redirects)
A thread I found about this error suggest possible problems with our SSL certificates, however I don't think this is my problem because some requests work for us, and the result of running openssl on our machine does't show any issues.
How should I proceed? Open a support ticket with Shopify?
FYI, I see that this specific problem only started yesterday on Feb 19 2013, so it might be a temporary issue.
FYI, the problem was caused by reusing a temporary access code.
Our fault - Shopify could have been more clear in their error message though.