Using awk on Nmap ssl-enum-cipher results - ssl

I'm creating a bash script that loops through a collection of IP-addresses and scans them with the Nmap ssl-enum-ciphers script. Everything works just fine except for one feature.
I need to find a way how I can link the scanned port with the TLS versions & ciphers found.
The Nmap data looks like this ->
PORT STATE SERVICE VERSION
21/tcp open ftp vsftpd 3.0.2
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA (dh 1024) - A
| TLS_DH_anon_WITH_AES_128_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_AES_256_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA (dh 1024) - F
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_ECDH_anon_WITH_AES_128_CBC_SHA (secp256r1) - F
| TLS_ECDH_anon_WITH_AES_256_CBC_SHA (secp256r1) - F
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (rsa 4096) - A
| compressors:
| NULL
| cipher preference: client
| warnings:
| Anonymous key exchange, score capped at F
| Key exchange (dh 1024) of lower strength than certificate key
| Key exchange (secp256r1) of lower strength than certificate key
| TLSv1.2:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (dh 1024) - A
| TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA (dh 1024) - A
| TLS_DH_anon_WITH_AES_128_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_AES_128_CBC_SHA256 (dh 1024) - F
| TLS_DH_anon_WITH_AES_128_GCM_SHA256 (dh 1024) - F
| TLS_DH_anon_WITH_AES_256_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_AES_256_CBC_SHA256 (dh 1024) - F
| TLS_DH_anon_WITH_AES_256_GCM_SHA384 (dh 1024) - F
| TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA (dh 1024) - F
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
| TLS_ECDH_anon_WITH_AES_128_CBC_SHA (secp256r1) - F
| TLS_ECDH_anon_WITH_AES_256_CBC_SHA (secp256r1) - F
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 4096) - A
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 4096) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 4096) - A
| TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 4096) - A
| TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (rsa 4096) - A
| compressors:
| NULL
| cipher preference: client
| warnings:
| Anonymous key exchange, score capped at F
| Key exchange (dh 1024) of lower strength than certificate key
| Key exchange (secp256r1) of lower strength than certificate key
|_ least strength: F
22/tcp open ssh OpenSSH 7.4 (protocol 2.0)
80/tcp open http nginx
443/tcp open ssl/http nginx
| ssl-enum-ciphers:
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| compressors:
| NULL
| cipher preference: server
|_ least strength: A
I need to format this data in something like this ->
21/tcp open ftp vsftpd 3.0.2
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA (dh 1024) - A
| TLS_DH_anon_WITH_AES_128_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_AES_256_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA (dh 1024) - F
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_ECDH_anon_WITH_AES_128_CBC_SHA (secp256r1) - F
| TLS_ECDH_anon_WITH_AES_256_CBC_SHA (secp256r1) - F
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (rsa 4096) - A
| TLSv1.2:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (dh 1024) - A
| TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA (dh 1024) - A
| TLS_DH_anon_WITH_AES_128_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_AES_128_CBC_SHA256 (dh 1024) - F
| TLS_DH_anon_WITH_AES_128_GCM_SHA256 (dh 1024) - F
| TLS_DH_anon_WITH_AES_256_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_AES_256_CBC_SHA256 (dh 1024) - F
| TLS_DH_anon_WITH_AES_256_GCM_SHA384 (dh 1024) - F
| TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA (dh 1024) - F
| TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA (dh 1024) - F
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
| TLS_ECDH_anon_WITH_AES_128_CBC_SHA (secp256r1) - F
| TLS_ECDH_anon_WITH_AES_256_CBC_SHA (secp256r1) - F
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 4096) - A
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 4096) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 4096) - A
| TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 4096) - A
| TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (rsa 4096) - A
| TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (rsa 4096) - A
443/tcp open ssl/http nginx
| ssl-enum-ciphers:
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
I've managed to create a awk script that collects all the open ports with status and versions running ->
#!/usr/bin/awk -f
/open/ {
port = $1
state = $2
service = $3
}
/ssl-enum-ciphers/ {
print port, state, service
}
outputs ->
21/tcp open ftp
443/tcp open ssl/http
As you can see the only things that are missing are the found TLS versions + ciphers.
Can someone give me a push in the right direction?
Thanks in advance,
Sleek

Using a reduced dataset:
$ cat nmap.dat
PORT STATE SERVICE VERSION
21/tcp open ftp vsftpd 3.0.2
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 1024) - A
| compressors:
| NULL
| cipher preference: client
| warnings:
| Anonymous key exchange, score capped at F
| Key exchange (dh 1024) of lower strength than certificate key
| TLSv1.2:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (dh 1024) - A
| compressors:
| NULL
| cipher preference: client
| warnings:
| Anonymous key exchange, score capped at F
| Key exchange (dh 1024) of lower strength than certificate key
|_ least strength: F
22/tcp open ssh OpenSSH 7.4 (protocol 2.0)
80/tcp open http nginx
443/tcp open ssl/http nginx
| ssl-enum-ciphers:
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| compressors:
| NULL
| cipher preference: server
|_ least strength: A
General approach is to use a flag (printme) to determine when to print a line; comparisons will determine when to turn on (1) or off (0) the flag.
One awk idea:
awk '
/open/ { port = $1
state = $2
service = $3
}
/ssl-enum-ciphers/ { print port, state, service
printme=1
}
/compressors:/ { printme=0 }
/cipher preference:/ { printme=0 }
/warnings:/ { printme=0 }
/TLS.*:/ { printme=1 }
printme
' nmap.dat
NOTES:
assumes all cipher subsections (under ssl-enum-ciphers) of interest are of the form TLS*:; otherwise OP can add more patterns for enabling the print flag (printme=1)
based solely on the provided sample input a single test for /compressors:/ should be sufficient to clear the print flag (printme=0); I'm not familiar with all possible nmap output formats so I've gone ahead and added the /cipher preference:/ and /warnings:/ tests, too; these last two tests can be removed if OP determines /compressors:/ is sufficient for clearing the print flag
This generates:
21/tcp open ftp
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 1024) - A
| TLSv1.2:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (dh 1024) - A
443/tcp open ssl/http
| ssl-enum-ciphers:
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A

Related

Multiple waiting approaches in #Appium

What is the best possible Explicit Wait implementation in Appium?
There are 3 possibilities I am aware of:
Use wait FluentWait provided by Selenium
Use WebDriverWait from Selenium-Support which internally extends FluentWait only
Use Appium's native AppiumFluentWait which also extends FluentWait from Selenium.
Our framework is going through some major changes and this is the perfect time to make this future ready.
Current implementation is using, WebDriverWait but for this we have to explicitly add import for selenium-support library as Appium's implementation imports it in runtime only scopy.
Is there some issue using explicitly importing selenium-support library because of which Appium team thought of adding library import in this scope, could AppiumFluentWait be a better approach?
But Again appium has removed some common functions which are there in selenium ex: MobileElement with its new version, is this method prone to be removed in future?
Output for: mvn dependency:tree
+- io.appium:java-client:jar:8.3.0:compile
[INFO] | +- org.seleniumhq.selenium:selenium-api:jar:4.7.1:compile (version selected from constraint [4.7.0,5.0))
[INFO] | +- org.seleniumhq.selenium:selenium-remote-driver:jar:4.7.1:compile (version selected from constraint [4.7.0,5.0))
[INFO] | | +- com.google.auto.service:auto-service-annotations:jar:1.0.1:compile
[INFO] | | +- com.google.auto.service:auto-service:jar:1.0.1:compile
[INFO] | | | \- com.google.auto:auto-common:jar:1.2:compile
[INFO] | | +- io.netty:netty-buffer:jar:4.1.84.Final:compile
[INFO] | | +- io.netty:netty-codec-http:jar:4.1.84.Final:compile
[INFO] | | | \- io.netty:netty-codec:jar:4.1.84.Final:compile
[INFO] | | +- io.netty:netty-common:jar:4.1.84.Final:compile
[INFO] | | +- io.netty:netty-transport-classes-epoll:jar:4.1.84.Final:compile
[INFO] | | +- io.netty:netty-transport-classes-kqueue:jar:4.1.84.Final:compile
[INFO] | | +- io.netty:netty-transport-native-epoll:jar:4.1.84.Final:linux-x86_64
[INFO] | | +- io.netty:netty-transport-native-kqueue:jar:4.1.84.Final:osx-x86_64
[INFO] | | +- io.netty:netty-transport-native-unix-common:jar:4.1.84.Final:compile
[INFO] | | +- io.netty:netty-transport:jar:4.1.84.Final:compile
[INFO] | | +- io.opentelemetry:opentelemetry-api:jar:1.19.0:compile
[INFO] | | +- io.opentelemetry:opentelemetry-context:jar:1.19.0:compile
[INFO] | | +- io.opentelemetry:opentelemetry-exporter-logging:jar:1.19.0:compile
[INFO] | | | +- io.opentelemetry:opentelemetry-sdk-metrics:jar:1.19.0:compile
[INFO] | | | \- io.opentelemetry:opentelemetry-sdk-logs:jar:1.19.0-alpha:compile
[INFO] | | | \- io.opentelemetry:opentelemetry-api-logs:jar:1.19.0-alpha:compile
[INFO] | | +- io.opentelemetry:opentelemetry-sdk-common:jar:1.19.0:compile
[INFO] | | +- io.opentelemetry:opentelemetry-sdk-extension-autoconfigure-spi:jar:1.19.0:compile
[INFO] | | +- io.opentelemetry:opentelemetry-sdk-extension-autoconfigure:jar:1.19.0-alpha:compile
[INFO] | | | \- io.opentelemetry:opentelemetry-exporter-common:jar:1.19.0:runtime
[INFO] | | +- io.opentelemetry:opentelemetry-sdk-trace:jar:1.19.0:compile
[INFO] | | +- io.opentelemetry:opentelemetry-sdk:jar:1.19.0:compile
[INFO] | | +- io.opentelemetry:opentelemetry-semconv:jar:1.19.0-alpha:compile
[INFO] | | +- io.ous:jtoml:jar:2.0.0:compile
[INFO] | | +- net.bytebuddy:byte-buddy:jar:1.12.18:compile
[INFO] | | +- org.apache.commons:commons-exec:jar:1.3:compile
[INFO] | | +- org.asynchttpclient:async-http-client:jar:2.12.3:compile
[INFO] | | | +- org.asynchttpclient:async-http-client-netty-utils:jar:2.12.3:compile
[INFO] | | | +- io.netty:netty-codec-socks:jar:4.1.60.Final:compile
[INFO] | | | +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.60.Final:compile
[INFO] | | | +- io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.60.Final:compile
[INFO] | | | +- org.reactivestreams:reactive-streams:jar:1.0.3:compile
[INFO] | | | +- com.typesafe.netty:netty-reactive-streams:jar:2.0.4:compile
[INFO] | | | \- com.sun.activation:jakarta.activation:jar:1.2.2:compile
[INFO] | | +- org.seleniumhq.selenium:selenium-http:jar:4.7.1:compile
[INFO] | | | \- dev.failsafe:failsafe:jar:3.3.0:compile
[INFO] | | +- org.seleniumhq.selenium:selenium-json:jar:4.7.1:compile
[INFO] | | \- org.seleniumhq.selenium:selenium-manager:jar:4.7.1:compile
[INFO] | +- org.seleniumhq.selenium:selenium-support:jar:4.7.1:runtime (version selected from constraint [4.7.0,5.0))
[INFO] | +- com.google.code.gson:gson:jar:2.10:compile
[INFO] | +- cglib:cglib:jar:3.3.0:runtime
[INFO] | | \- org.ow2.asm:asm:jar:7.1:runtime
[INFO] | +- commons-validator:commons-validator:jar:1.7:runtime
[INFO] | | +- commons-beanutils:commons-beanutils:jar:1.9.4:runtime
[INFO] | | +- commons-digester:commons-digester:jar:2.1:runtime
[INFO] | | +- commons-logging:commons-logging:jar:1.2:compile
[INFO] | | \- commons-collections:commons-collections:jar:3.2.2:runtime
[INFO] | +- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO] | +- commons-io:commons-io:jar:2.11.0:compile
[INFO] | \- org.slf4j:slf4j-api:jar:2.0.5:compile
So first of all it depends on the architecture.
Second, Do you really need FluentWait? The WebDriverWait class has optional arguments: timeout, poll_frequency, and ignored_exceptions and you could supply them there.
:Args:
- driver - Instance of WebDriver (Ie, Firefox, Chrome or Remote)
- timeout - Number of seconds before timing out
- poll_frequency - sleep interval between calls
By default, it is 0.5 second.
- ignored_exceptions - iterable structure of exception classes ignored during calls.
By default, it contains NoSuchElementException only.
How I see it:
wait = WebDriverWait(driver, 15)
s_wait = WebDriverWait(driver, 5)
l_wait = WebDriverWait(driver, 30)
fl_wait = WebDriverWait(driver, 15, poll_frequency=1,
ignored_exceptions=[ElementNotVisibleException])
Then implement methods that return: waits
def wait(self, locator, wait: WebDriverWait = None):
if wait is None:
wait = self._wait
return wait.until(ec.presence_of_element_located(locator))
Then call the necessary method when it is needed, in general, it can be implemented within base class
An example in python, but you can reuse the logic

Why would BIO_do_connect() from OpenSSL not work right with GDAX (a.k.a. cloudflare) sandbox?

I wrote some software in C++ and I'm trying to get the GDAX /products list for now (mainly as a test at this time.)
UPDATE: I wanted to add that the connection is actually to cloudflare and not directly to GDAX. So it is likely a problem with cloudflare and not directly GDAX servers.
Only, the BIO_do_connect() function returns -1 each time. It does not give me much to go on with it. I write the following in my log. So the main info is the error occurs on line 794 of s23_clnt.c...
OpenSSL: [336031996/20|119|252]:[]:[]:[]:[s23_clnt.c]:[794]:[(no details)]
I can tell that this means the TCP connection itself happens, but somehow it's not able to get an acceptable secure connection. I've see similar behaviors before when a machine would only some old encryption methods. But I checked with nmap and the connection definitely supports TLS 1.2. I ran the following command and got:
nmap --script ssl-enum-ciphers api-public.sandbox.gdax.com
And I get the following output which proves that port 443 is open and has all the necessary encryption schemes necessary.
Starting Nmap 7.01 ( https://nmap.org ) at 2018-03-24 21:57 PDT
Nmap scan report for api-public.sandbox.gdax.com (104.28.30.142)
Host is up (0.016s latency).
Other addresses for api-public.sandbox.gdax.com (not scanned): 104.28.31.142
Not shown: 996 filtered ports
PORT STATE SERVICE
80/tcp open http
443/tcp open https
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C
| compressors:
| NULL
| cipher preference: server
| TLSv1.1:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
| TLSv1.2:
| ciphers:
| TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
|_ least strength: C
8080/tcp open http-proxy
8443/tcp open https-alt
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C
| compressors:
| NULL
| cipher preference: server
| TLSv1.1:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
| TLSv1.2:
| ciphers:
| TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
|_ least strength: C
Nmap done: 1 IP address (1 host up) scanned in 8.07 seconds
Now, I tested my code against the normal REST API address (api.gdax.com) and my own website (www.m2osw.com) and the encryption part works just fine. I really don't see what I would be doing wrong that it would fail like that with the sandbox URL (api-public.sandbox.gdax.com) unless its SSL setup is weird.
Note that when I try to connect to port 80 (which is wrong, I know), it works as expected. That is, I get a 301 with a Location to the same address with protocol HTTPS.
Anyone has had some problem with connecting to the sandbox?
There are all the functions that get called. The full implementation is available on github in the libsnapwebsites around line 1111 at this point (bio_client constructor).
// called once on initialization
SSL_library_init();
ERR_load_crypto_strings();
ERR_load_SSL_strings();
SSL_load_error_strings();
OpenSSL_add_all_algorithms();
crypto_thread_setup();
// call each time we connect
SSL_CTX * ssl_ctx = SSL_CTX_new(SSLv23_client_method();
SSL_CTX_set_verify_depth(ssl_ctx, 4);
SSL_CTX_set_options(ssl_ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_TLSv1 | SSL_OP_NO_COMPRESSION);
// just in case I tried with "ALL", but no difference
//SSL_CTX_set_cipher_list(ssl_ctx, "ALL");
SSL_CTX_set_cipher_list(ssl_ctx, "HIGH:!aNULL:!kRSA:!PSK:!SRP:!MD5:!RC4");
SSL_CTX_load_verify_locations(ssl_ctx, NULL, "/etc/ssl/certs");
BIO * bio = BIO_new_ssl_connect(ssl_ctx);
SSL * ssl(nullptr);
BIO_get_ssl(bio, &ssl);
SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);
BIO_set_conn_hostname(bio, const_cast<char *>(addr.c_str()));
BIO_set_conn_int_port(bio, &port);
int const cr(BIO_do_connect(bio));
// here cr == -1 when I use api-public.sandbox.gdax.com
Again, this code works find and cr > 0 if I use api.gdax.com so I'm really at a loss at this point!? And I know that the TCP connection itself happens since it gets in that s23_clnt.c which is after that part happens.
Okay, I spent the whole day (okay about half a day) working on this one comparing my code with libcurl's code which also uses the SSL_CTX and SSL structures of OpenSSL. The code looks very much the same... except that the libcurl version includes this:
[...]
switch(data->set.ssl.version) {
case CURL_SSLVERSION_DEFAULT:
[...]
use_sni(TRUE);
break;
[...]
if((0 == Curl_inet_pton(AF_INET, conn->host.name, &addr)) &&
(0 == Curl_inet_pton(AF_INET6, conn->host.name, &addr)) &&
sni &&
!SSL_set_tlsext_host_name(connssl->handle, conn->host.name))
infof(data, "WARNING: failed to configure server name indication (SNI) "
"TLS extension\n");
[...]
As we can see, they have something called SNI and if true they set the TLS extension called Hostname. If that Hostname parameter is not included in the SSL HELLO message, then the GDAX server (or most probably the cloudflare one) refuses the connection immediately.
So, on my end I will be forcing the SNI (Server Name Identification) and that way it is likely to work on way more servers. libcurl allows to not include it, but it looks like you should always have it. It shouldn't hurt, at least.
BIO_get_ssl(bio, &ssl);
SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);
SSL_set_tlsext_host_name(ssl, const_cast<char *>(addr.c_str()));
BIO_set_conn_hostname(bio, const_cast<char *>(addr.c_str()));
Note that the SSL_set_tlsext_host_name() function must be given the correct hostname, not an IPv4 or IPv6 address.

What is the relationship between the TLS version (1.x) and the cipher suite

I'm writing a client server application that requires a secured connection.
I want to understand better the difference between the TLS version (1.x) and the cipher suites that is in use.
this is the result of an nmap scan on my server (scanned for supported TLS options):
PORT STATE SERVICE
443/tcp open https
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256k1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (dh 1024) - D
| TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (secp256k1) - C
| TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C
| compressors:
| NULL
| cipher preference: server
| warnings:
| 64-bit block cipher 3DES vulnerable to SWEET32 attack
| Key exchange (dh 1024) of lower strength than certificate key
| TLSv1.1:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256k1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (dh 1024) - D
| TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (secp256k1) - C
| TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C
| compressors:
| NULL
| cipher preference: server
| warnings:
| 64-bit block cipher 3DES vulnerable to SWEET32 attack
| Key exchange (dh 1024) of lower strength than certificate key
| TLSv1.2:
| ciphers:
| TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (dh 1024) - D
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 1024) - A
| TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (secp256k1) - C
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256k1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256k1) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256k1) - A
| TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
| compressors:
| NULL
| cipher preference: client
| warnings:
| 64-bit block cipher 3DES vulnerable to SWEET32 attack
| Key exchange (dh 1024) of lower strength than certificate key
|_ least strength: D
What does it mean that a cipher suite is supported in more than one version of TLS?

CAS server SAML 1.1 authentication got empty response

CAS server SAML 1.1 authentication got empty response
We have been suffering pain for the CAS SAML 1.1 protocol for a long time.
We are using phpCAS which is a CAS client library.
We are doing with the protocol SAML 1.1 in order to get user attributes, such as user email.
However, it failed with empty reponse error.
Here's the client log: (it's the part of interaction logs, a little bit long, but easy to understand)
3CEE .| | | | => CAS_Client::_readURL('https://portail.cigref.fr/cas/samlValidate?TARGET=https%3A%2F%2Frfly-99.cvtest.com%2Fztest%2FphpCAS%2Fdocs%2Fexamples%2Fexample_simple.php', NULL, NULL, NULL) [Client.php:2062]
3CEE .| | | | | => CAS_Client::_buildSAMLPayload() [Client.php:2806]
3CEE .| | | | | <= '<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header/><SOAP-ENV:Body><samlp:Request xmlns:samlp="urn:oasis:names:tc:SAML:1.0:protocol" MajorVersion="1" MinorVersion="1" RequestID="_192.168.16.51.1024506224022" IssueInstant="2002-06-19T17:03:44.022Z"><samlp:AssertionArtifact>ST-5376-cjRtmnPYQBxIMFWOQY1x-cas01.example.org</samlp:AssertionArtifact></samlp:Request></SOAP-ENV:Body></SOAP-ENV:Envelope>'
3CEE .| | | | | => CAS_Request_CurlRequest::sendRequest() [AbstractRequest.php:242]
3CEE .| | | | | | CURL: Set CURLOPT_CAINFO ../../ssl/portail.cigref.fr.pem [CurlRequest.php:132]
3CEE .| | | | | | Response Body:
3CEE .| | | | | |
3CEE .| | | | | | [CurlRequest.php:84]
3CEE .| | | | | <= true
3CEE .| | | | <= true
3CEE .| | | | server version: S1 [Client.php:2071]
3CEE .| | | | dom->loadXML() failed [Client.php:2082]
3CEE .| | | | => CAS_AuthenticationException::__construct(CAS_Client, 'SA not validated', 'https://portail.cigref.fr/cas/samlValidate?TARGET=https%3A%2F%2Frfly-99.cvtest.com%2Fztest%2FphpCAS%2Fdocs%2Fexamples%2Fexample_simple.php', false, true, '') [Client.php:2087]
3CEE .| | | | | => CAS_Client::getURL() [AuthenticationException.php:76]
3CEE .| | | | | <= 'https://rfly-99.cvtest.com/ztest/phpCAS/docs/examples/example_simple.php'
3CEE .| | | | | CAS URL: https://portail.cigref.fr/cas/samlValidate?TARGET=https%3A%2F%2Frfly-99.cvtest.com%2Fztest%2FphpCAS%2Fdocs%2Fexamples%2Fexample_simple.php [AuthenticationException.php:79]
3CEE .| | | | | Authentication failure: SA not validated [AuthenticationException.php:80]
3CEE .| | | | | Reason: bad response from the CAS server [AuthenticationException.php:85]
3CEE .| | | | | CAS response: [AuthenticationException.php:101]
3CEE .| | | | | exit()
3CEE .| | | | | -
The ticket is valid and payload has been sent but Response Body is empty.
The logs above is for our customers' CAS server.
We did test on our local CAS server and it's ok.
We also checked the customers' CAS server log which is tomcat log and found this:
2016-03-22 03:31:25,647 INFO [com.github.inspektr.audit.support.Slf4jLoggingAuditTrailManager] - Audit trail record BEGIN
=============================================================
WHO: audit:unknown
WHAT: ST-5376-cjRtmnPYQBxIMFWOQY1x-cas01.example.org
ACTION: SERVICE_TICKET_VALIDATED
APPLICATION: CAS
WHEN: Tue Mar 22 03:31:25 CET 2016
CLIENT IP ADDRESS: 124.127.186.129
SERVER IP ADDRESS: 178.22.130.235
=============================================================
2016-03-22 03:31:25,647 DEBUG [org.jasig.cas.web.ServiceValidateController] - Successfully validated service ticket ST-5376-cjRtmnPYQBxIMFWOQY1x-cas01.example.org for service [https://rfly-99.cvtest.com/ztest/phpCAS/docs/examples/example_simple.php]
Ticket is validated, no errors in the log except some logout error which should be not related.
What is the reason of this issue? Anyone ever encoutered this issue? Please help us out.
Shouldnt you be passing the ticket to samlValidate as well? I only see TARGET.

compute engine load balancer UDP/DNS responses dropped

Have been testing out GCE and the load balancing capabilities - however have been seeing some unexpected results.
The trial configuration involves 2 instances acting as DNS resolvers in a target pool with a 3rd test instance. There is also a http server running on the hosts. No health check scripts have been added.
DNS request to individual instance public IP (from ANY) - OK
HTTP request to individual instance public IP (from ANY) - OK
HTTP request to load balance IP (from ANY) - OK
DNS request to load balance IP (from an instance in the target pool) - OK
DNS request to load balance IP (from an instance in the same network - but not in the target pool) - NOK
DNS request to load balance IP (other) - NOK
I can see in the instance logs that the DNS request arrive for all cases and are distributed evenly - though the replies don't seem to get back to the originator.
The behavior seems unexpected. I've played with the session affinity with similar results - though the default behavior is the most desired option.
Have hit a wall. Are there some ideas to try?
Information on the setup:
$ gcutil listhttphealthchecks
+------+------+------+
| name | host | port |
+------+------+------+
$ gcutil listtargetpools
+----------+-------------+
| name | region |
+----------+-------------+
| dns-pool | us-central1 |
+----------+-------------+
$ gcutil listforwardingrules
+---------+-------------+-------------+
| name | region | ip |
+---------+-------------+-------------+
| dns-tcp | us-central1 | 8.34.215.45 |
+---------+-------------+-------------+
| dns-udp | us-central1 | 8.34.215.45 |
+---------+-------------+-------------+
| http | us-central1 | 8.34.215.45 |
+---------+-------------+-------------+
$ gcutil getforwardingrule dns-udp
+---------------+----------------------------------+
| name | dns-udp |
| description | |
| creation-time | 2013-12-28T12:28:05.816-08:00 |
| region | us-central1 |
| ip | 8.34.215.45 |
| protocol | UDP |
| port-range | 53-53 |
| target | us-central1/targetPools/dns-pool |
+---------------+----------------------------------+
$ gcutil gettargetpool dns-pool
+------------------+-------------------------------+
| name | dns-pool |
| description | |
| creation-time | 2013-12-28T11:48:08.896-08:00 |
| health-checks | |
| session-affinity | NONE |
| failover-ratio | |
| backup-pool | |
| instances | us-central1-a/instances/dns-1 |
| | us-central1-b/instances/dns-2 |
+------------------+-------------------------------+
[#dns-1 ~]$ curl "http://metadata/computeMetadata/v1/instance/network-interfaces/?recursive=true" -H "X-Google-Metadata-Request: True"
[{"accessConfigs":[{"externalIp":"162.222.178.116","type":"ONE_TO_ONE_NAT"}],"forwardedIps":["8.34.215.45"],"ip":"10.240.157.97","network":"projects/763472520840/networks/default"}]
[#dns-2 ~]$ curl "http://metadata/computeMetadata/v1/instance/network-interfaces/?recursive=true" -H "X-Google-Metadata-Request: True"
[{"accessConfigs":[{"externalIp":"8.34.215.162","type":"ONE_TO_ONE_NAT"}],"forwardedIps":["8.34.215.45"],"ip":"10.240.200.109","network":"projects/763472520840/networks/default"}]
$ gcutil getfirewall dns2
+---------------+------------------------------------+
| name | dns2 |
| description | Allow the incoming service traffic |
| creation-time | 2013-12-28T10:35:18.185-08:00 |
| network | default |
| source-ips | 0.0.0.0/0 |
| source-tags | |
| target-tags | |
| allowed | tcp: 53 |
| allowed | udp: 53 |
| allowed | tcp: 80 |
| allowed | tcp: 443 |
+---------------+------------------------------------+
The instances are CentOS and have their iptables firewalls disabled.
Reply from instance in target pool
#dns-1 ~]$ nslookup test 8.34.215.45 | grep answer
Non-authoritative answer:
#dns-1 ~]$
Reply from other instance in target pool
#dns-2 ~]$ nslookup test 8.34.215.45 | grep answer
Non-authoritative answer:
#dns-2 ~]$
No reply from instance not in the target pool on the load balanced IP. However it gets a reply from all other interfaces
#dns-3 ~]$ nslookup test 8.34.215.45 | grep answer
#dns-3 ~]$
#dns-3 ~]$ nslookup test 8.34.215.162 | grep answer
Non-authoritative answer:
#dns-3 ~]$ nslookup test 10.240.200.109 | grep answer
Non-authoritative answer:
#dns-3 ~]$ nslookup test 10.240.157.97 | grep answer
Non-authoritative answer:
#dns-3 ~]$ nslookup test 162.222.178.116 | grep answer
Non-authoritative answer:
-- Update --
Added a health check so that the instances wouldn't be marked as UNHEALTHY. However got the same result.
$ gcutil gettargetpoolhealth dns-pool
+-------------------------------+-------------+--------------+
| instance | ip | health-state |
+-------------------------------+-------------+--------------+
| us-central1-a/instances/dns-1 | 8.34.215.45 | HEALTHY |
+-------------------------------+-------------+--------------+
| us-central1-b/instances/dns-2 | 8.34.215.45 | HEALTHY |
+-------------------------------+-------------+--------------+
-- Update --
Looks like the DNS service is not responding with the same IP that the request came in on. This is for sure be the reason it doens't appear to be responding.
0.000000 162.222.178.130 -> 8.34.215.45 DNS 82 Standard query 0x5323 A test.internal
2.081868 10.240.157.97 -> 162.222.178.130 DNS 98 Standard query response 0x5323 A 54.122.122.227
Looks like the DNS service is not responding with the same IP that the request came in on. This is for sure be the reason it doens't appear to be responding.
0.000000 162.222.178.130 -> 8.34.215.45 DNS 82 Standard query 0x5323 A test.internal
2.081868 10.240.157.97 -> 162.222.178.130 DNS 98 Standard query response 0x5323 A 54.122.122.227