Azure Devops Self-Hosted Agent fail connecting behind corporation network - SSL Error / An existing connection was forcibly closed by the remote host - ssl

Got this problem with me for 2 weeks so far and still not figure it out yet, been trying to test on multiple computer and here is the symptom.
When setup Agent on any Computer that is using Corporation Network got SSL exception:
System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception
---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
....
Attempt 1 of GET request to https://dev.azure.com/{org}/_apis/connectionData?connectOptions=1&lastChangeId=366981152&lastChangeId64=366981152 failed (Socket Error: ConnectionReset)
Below is what i had tried:
1). Pass the GET request directly to Browser / Postman > working fine and receive the connectionData from Azure Devops:
*Note: no proxy and no web filter applied to the site
https://dev.azure.com/{org}/_apis/connectionData?connectOptions=1&lastChangeId=366981152&lastChangeId64=366981152
2). Telnet from cmd > working fine as well
> Telnet test.dev.azure.com 443
3). Follow the instruction here https://stackoverflow.com/a/71667810/9391928 and run Debugging Script and Mitigation Script (Modify Registry) to Enforce TLS1.2 and so far all check is green but still Unknown ISSUE FOUND:
Probing: status.dev.azure.com
ISSUE FOUND: This may be TLS compatibility issue!
Probe failed when TLS-negotiating to 13.107.6.183:443. Error: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
***
TLS 1.2 client usage enabled.
***
Cipher Suite check passed: at least one of the TLS 1.2 cipher suites supported by Azure DevOps is enabled.
***
Key Exchange check passed.
***
Elliptic curve check passed: at least one elliptic curve supported by Azure DevOps is enabled
***
.NET Framework release is 4.7+ (release 528372)
TLS 1.2 enforced for applications targetting .NET Framework 4.0/4.5.x
TLS 1.2 enforced for applications targetting .NET Framework 4.0/4.5.x (32bit app on 64bit OS)
TLS 1.2 enforced for applications targetting .NET Framework 3.5
TLS 1.2 enforced for applications targetting .NET Framework 3.5 (32bit app on 64bit OS)
All mitigations required to ensure TLS 1.2-compatibility of legacy .NET applications are in place.
4). Prepare a Laptop and test register/run agent with vs without corporation network, fail whenever connect to corporation network (using non-corporation network work fine)
5). Running Client SSL check from postman / browser for both case with vs without corporation network and got return the same TLS1.3 with all similar cipher suites supported (included "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384" & "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" which is used by Azure Devops)
request : https://www.howsmyssl.com/a/check
response: "given_cipher_suites": [
"TLS_AES_128_GCM_SHA256",
"TLS_AES_256_GCM_SHA384",
"TLS_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA",
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA",
"TLS_RSA_WITH_AES_128_GCM_SHA256",
"TLS_RSA_WITH_AES_256_GCM_SHA384",
"TLS_RSA_WITH_AES_128_CBC_SHA",
"TLS_RSA_WITH_AES_256_CBC_SHA",
"TLS_RSA_WITH_3DES_EDE_CBC_SHA"
],
"ephemeral_keys_supported": true,
"session_ticket_supported": true,
"tls_compression_supported": false,
"unknown_cipher_suite_supported": false,
"beast_vuln": false,
"able_to_detect_n_minus_one_splitting": false,
"insecure_cipher_suites": {},
"tls_version": "TLS 1.3",
"rating": "Probably Okay"
6). Change the Probing Script to probe www.google.com and fail as well (of-courses success in normal network ^_V)
7). Double check DNS resolver to IP Address > resolver work fine, azure devops ip address is and same between both networks.
8). Write a .net console app to run a test (similar to the Probing Script) and always got Exception "An existing connection was forcibly closed by the remote host" when in Corporation Network. When out of Corporation Network the run is fine and Stream return NegotiatedCipherSuite TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 for test.dev.azure.com (and TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 for www.google.com)
client.Connect(connectHost, 443); // Success
...
var askedProtocols = System.Security.Authentication.SslProtocols.Tls12;
stream.AuthenticateAsClient(connectHost, null, askedProtocols, false); // Always throw exception "An existing connection was forcibly closed by the remote host."
return new(connectHost, true, remoteEndpoint, null);
...
Any suggestion will be much appreciate guys !

Related

Cannot acces to localhost:8443/ejbca

I'm new in ejbca and i have to install it on a virtual machine for job
Ubuntu 20.04
ejbca_7_4_3_2
wildfly-18.0.0.Final
mariadb-server version: 10.3.32-MariaDB-0ubuntu0.20.04.1 Ubuntu 20.04
openjdk version "1.8.0_312"
Apache Ant(TM) version 1.10.7 compiled on October 24 2019
After a few try's(and a lot of virtual machines cloned and deleted), i finally get the "build successfully" message with the commands ant runinstall and ant deploy-keystore
But when i try to use the URL https://localhost:8443/ejbca/ (the certificate SuperAdmin.p12 is installed) my browser(firefox 96.0 64bits) give the message
An error occurred during a connection to localhost:8443. Cannot communicate securely with peer: no common encryption algorithm(s).
Error code: SSL_ERROR_NO_CYPHER_OVERLAP
i have this errors on my log file, the first one related with ant -q clean deployear
and the last, appear every time i try to access via URL https://localhost:8443/ejbca/
ERROR [org.jboss.as.jsf] (MSC service thread 1-1) WFLYJSF0002: Could not load JSF managed bean class: org.ejbca.ui.web.admin.peerconnector.PeerConnectorMBean
ERROR [io.undertow.request] (default I/O-2) Closing SSLConduit after exception on handshake: javax.net.ssl.SSLHandshakeException: no cipher suites in common
at sun.security.ssl.Alert.createSSLException(Alert.java:131)
at sun.security.ssl.Alert.createSSLException(Alert.java:117)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:311)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
at sun.security.ssl.ServerHello$T12ServerHelloProducer.chooseCipherSuite(ServerHello.java:461)
at sun.security.ssl.ServerHello$T12ServerHelloProducer.produce(ServerHello.java:296)
at sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:421)
at sun.security.ssl.ClientHello$T12ClientHelloConsumer.consume(ClientHello.java:1020)
at sun.security.ssl.ClientHello$ClientHelloConsumer.onClientHello(ClientHello.java:727)
at sun.security.ssl.ClientHello$ClientHelloConsumer.consume(ClientHello.java:693)
at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:981)
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:968)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:915)
at io.undertow.protocols.ssl.SslConduit$5.run(SslConduit.java:1072)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.lang.Thread.run(Thread.java:748)
ERROR [io.undertow.request] (default I/O-2) Closing SSLConduit after exception
Sounds like a TLS configuration issue. You will find the TLS configuration you did when configuring WildFly in the commands you ran like:
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/server-ssl-context=httpspriv:add(key-manager=httpsKM,protocols=["TLSv1.2"],use-cipher-suites-order=false,cipher-suite-filter="TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",trust-manager=httpsTM,need-client-auth=true)'
The result is somewhere in standalone.xml in WildFly, and you can modify it directly in WildFly. For example if you have EC keys in the server certificate while using the above RSA algorithm selection.
In server.log you should also see when WildFly starts up if there are any error in parsing the values, or keystores.
Make sure that you server and client certificates have keys and algorithms that match the TLS algorithm settings, otherwise WildFly will remove those algortihms.

MQTT Error BR_ERR_BAD_VERSION on Shelly 1PM with Tasmota

I try to connect a Shelly 1 PM smart power relay to a managed MQTT broker.
The firmware on the device is a custom-built Tasmota 8.3.1 from the dev branch with USE_MQTT_TLS enabled. The port is set correctly to 8883 for TLS and the broker service is running at mqtt.bosch-iot-hub.com
When the device boots up, I can see the log messages on the serial port as follows:
23:03:03 MQT: Connect failed to mqtt.bosch-iot-hub.com:8883, rc 4. Retry in 10 sec
23:03:14 MQT: Attempting connection...
23:03:14 MQT: TLS connection error: 0
Return Code 4 is, according to the Tasmota documentation (https://tasmota.github.io/docs/TLS/), the code for BR_ERR_BAD_VERSION
And this error constant seems to be from BearSSL and means "Incoming record version does not match the expected version." (according to http://sources.freebsd.org/HEAD/src/contrib/bearssl/tools/errors.c)
Using an online TLS testing tool and checking mqtt.bosch-iot-hub, it supports only TLS 1.2 (1.3, 1.1 and 1.0 being disabled as well as SSLv2 and SSLv3). BearSSL website states that it supports TLS 1.2
I tried setting the log level of Tasmota in my_user_config.h , but it does not log any more verbose or detailed information.
#define SERIAL_LOG_LEVEL LOG_LEVEL_DEBUG_MORE // [SerialLog] (LOG_LEVEL_NONE, LOG_LEVEL_ERROR, LOG_LEVEL_INFO, LOG_LEVEL_DEBUG, LOG_LEVEL_DEBUG_MORE)
What is the error message supposed to mean? Is it a TLS incompatibility of the BearSSL stack or on the service side?
How can I enable verbose logging on Tasmota to see detailed TLS handshake information?
Anything else I am missing?
I appreciate after 6 months the question may have been a little expired, however the error code is not the TLS one as you describe, but rather the return code for the MQTT connection, as described in
https://tasmota.github.io/docs/MQTT/#return-codes-rc
which means your error code corresponds to
4 MQTT_CONNECT_BAD_CREDENTIALS the username/password were rejected

VB.NET Web Service connection impossible : "Could not create SSL/TLS secure channel"

I am trying to connect to a web service which is written in Java language on the server side. My program is written in VB.NET 4.0 on the client side.
When i try to call any methods from the web service I receive the following error message :
WS-Metadata Exchange Error URI: [http://...] Metadata contains a reference that cannot be resolved. The request was aborted: Could not create SSL/TLS secure channel
I am using a certificate to connect to the web service, however it seems that the problem is on the communication level between my host machine (client) and the server. The certificate i am using is never been checked as the client and the server can even not communicate.
I have tried to set the security protocol to "SSLv3" or "TLS12" but it didn't help. Thank you in advance for any suggestions.
Also i should mention that i managed to connect to this web service using a Java class. It seems that when the client side is written in Java it works but when it's in VB.NET it doesn't work.
P. HAB
EDITED :
I used "Wireshark" to discover the HTTPS packets which are sent and received by my program, here what I found in the transmitted packets :
Request by the client :
Secure Sockets Layer
TLSv1 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 140
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 136
Version: TLS 1.0 (0x0301)
Random
Session ID Length: 0
Cipher Suites Length: 24
Cipher Suites (12 suites)
Compression Methods Length: 1
Compression Methods (1 method)
Extensions Length: 71
Extension: server_name
Extension: elliptic_curves
Extension: ec_point_formats
Extension: Extended Master Secret
Extension: renegotiation_info
Response from the server :
Secure Sockets Layer
TLSv1 Record Layer: Alert (Level: Fatal, Description: Handshake Failure)
Content Type: Alert (21)
Version: TLS 1.0 (0x0301)
Length: 2
Alert Message
Level: Fatal (2)
Description: Handshake Failure (40)
I have tried to change the security protocol to different versions of SSL/TLS but it changed nothing. My client tries to send a "Client hello" with standard values (no user-customized parameters were specified) but the server is not accepting it. Any suggestions ?
Solution found :
The problem is in fact related to the Cipher suites and specially the RC4 algorithme. Actually my client is written in VB.NET 4.0 and in this version of .NET the deprecated RC4 algorithme is still supported and proposed by the .NET framework to any Web server during the "Hello Client" (Handshake) process.
The only presence of the RC4 cipher suites in the supported list of the cipher suites by my client was making the server to abort the connection with the following error message : "The request was aborted: Could not create SSL/TLS secure channel"
So I had to deactivate the RC4 cipher suites on my client machine by going in the system registry using regedit.exe and add the following key to these two specified paths :
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft.NETFramework\v4.0.30319
"SchUseStrongCrypto"=dword:00000001
And
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft.NETFramework\v4.0.30319
"SchUseStrongCrypto"=dword:00000001
After restarting the client machine I was able to connect to the web service succesfully.
For more information about the deprecated RC4 algorithmes :
https://technet.microsoft.com/en-us/library/security/2960358.aspx
=> Also you should notice that this problem exists only in .NET 4.0-4.5. From the version 4.6 and above its working fine.

WAS 8.5.5 + Worklight 6.1 and TLS 1.2 in hybrid application

We are migrating to WAS 8.5.5 and TLS 1.2 and are observing some unexpected problems.
The inputs are:
Worklight 6.1.0.1
WAS 8.5.5.9 + SDK Java 8
WAS is switched to TLS 1.2 (following this guide
https://developer.ibm.com/answers/questions/206952/how-do-i-configure-websphere-application-server-ss.html)
Application is Hybrid
When we use application via web emulator - it works fine.
When we use it from hardware device (Android or iOS) via IMC - we get exception that says
"client" uses TLS 1.1
server uses TLS 1.2
error of HTTPS handshake
It's not clear what is "client" in that case and why it uses TLS v1.1.
iOS device (iPhone) web browser is TLS 1.2 enabled - can open HTTPS links with TLS 1.2 protocol.
Here is full stack trace from SystemOut.log
[6/14/16 11:16:32:197 EDT] 000000b2 SSLHandshakeE E SSLC0008E: Unable to initialize SSL connection. Unauthorized access was denied or security settings have expired. Exception is javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1.1 not enabled or not supported
at com.ibm.jsse2.C.z(C.java:532)
at com.ibm.jsse2.ap.b(ap.java:476)
at com.ibm.jsse2.ap.c(ap.java:112)
at com.ibm.jsse2.ap.wrap(ap.java:277)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:21)
at com.ibm.ws.ssl.channel.impl.SSLUtils.handleHandshake(SSLUtils.java:748)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.readyInbound(SSLConnectionLink.java:567)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.ready(SSLConnectionLink.java:296)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture$1.run(AsyncChannelFuture.java:205)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1881)
Caused by: javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1.1 not enabled or not supported
at com.ibm.jsse2.j.a(j.java:31)
at com.ibm.jsse2.ap.a(ap.java:11)
at com.ibm.jsse2.C.a(C.java:342)
at com.ibm.jsse2.C.a(C.java:100)
at com.ibm.jsse2.E.a(E.java:140)
at com.ibm.jsse2.E.a(E.java:813)
at com.ibm.jsse2.C.r(C.java:44)
at com.ibm.jsse2.C$b.a(C$b.java:2)
at com.ibm.jsse2.C$b.run(C$b.java:3)
at java.security.AccessController.doPrivileged(AccessController.java:686)
at com.ibm.jsse2.C$c.run(C$c.java:11)
at com.ibm.ws.ssl.channel.impl.SSLUtils.handleHandshake(SSLUtils.java:835)
... 8 more
I have no idea what our next steps should be.
Any help will be much appreciated.
Seems pretty clear that TLSv1.2-only is too aggressive for your clients. In terms of WAS config, "ssl_tlsv2" is probably the best you can do.
IMC was causing it. Had to configure it to start using TLS v1.2

IIS - WCF 4.0 Calling Java (Could not establish secure channel for SSL/TLS)

I'm running into a bit of a problem with my WCF service that is trying to talk to a Java Web Service.
I have a ASP.Net MVC front end that is talking to a WCF service over HTTP. The WCF service then talks to a JAVA web service over HTTPS utilising mutual authentication of certificates. The problem currently is that I am getting the following error when the WCF service trys to call the JAVA backend:
Could not establish secure channel for SSL/TLS
On the Java side, I am running JBOSS, with SSL3 and TLS ciphers for the SSL only. The error I have on that end is:
21:49:48,701 INFO [STDOUT] http-0.0.0.0-8543-2, WRITE: TLSv1 Handshake, length = 1514
21:49:49,499 INFO [STDOUT] http-0.0.0.0-8543-2, received EOFException: error
21:49:49,499 INFO [STDOUT] http-0.0.0.0-8543-2, handling exception: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
21:49:49,507 INFO [STDOUT] http-0.0.0.0-8543-2, SEND TLSv1 ALERT: fatal, description = handshake_failure
21:49:49,507 INFO [STDOUT] http-0.0.0.0-8543-2, WRITE: TLSv1 Alert, length = 2
21:49:49,507 INFO [STDOUT] http-0.0.0.0-8543-2, called closeSocket()
21:49:49,508 INFO [STDOUT] http-0.0.0.0-8543-2, called close()
As I'm using mutual authentication of certs, my first port of call was bad certificate. So I have opened up the service wsdl page (also requires authentication of cert), and everything is ok. My Certificates are fine and trusted.
I then started thinking that maybe the cert is not getting put on the service call. So I created a console app that calls the Java service with the same certifcate (looked up via an endpoint behaviour in the config file). Lo and behold, this work fine, and the service response data is shown on the screen.
So that leaves me thinking that there is something in IIS that is stopping the SSL channel being opened, and this is where I need a bit of help really.
My IIS is version 7.0 running on Windows Server 2008 R2. The service is running on .Net 4.
(One thing I should point out was that my consle app was running .Net v3.5 not v4.)
I have dabbled a bit in the SCHANNEL settings, but don't really know which settings should be enabled, and which should be disabled.
Currently I have:
TLS 1.0/Server/Enabled = 1
SSL 3.0/Server/Enabled = 1
SSL 2.0/Server/Enabled = 1
PCT 1.0/Server/Enabled = 1
I also have:
SSL 2.0/Client/DisabledByDefault = 0
Does anyone have any ideas on where to start?
Thanks in advance,
Nick
[UPDATE]
Am now getting the following error in the Windows Error Log:
A fatal error occurred when attempting to access the SSL client credential private key.
The error code returned from the cryptographic module is 0x8009030d.
The internal error state is 10003.
...but I'm not sure to resolve it. The certifcate is being picked up ok by the looks of this. it just can't get the password.
[Answered]
It turns out that the priviledges on the Certificate hadn't been set for my IIS_IUSRS account.
When I set those up, everything worked fine.