I tried posting this on ask.openstack but it has been stuck in the moderator for 5 days now. I thought I'd try here.
I was trying to debug a Nova issue and wanted to decode the SSL / TLS packets being exchanged using Wireshark. Part of the changes I was making was setting Nova up to use SSL / TLS and I wanted to be sure that part of it I had set correctly. I eventually figure out my issues from the various log files but I'm somewhat assuming that being able to watch the network traffic may help in some very difficult cases.
The exchange uses TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 at one point. According to this security stackexchannge question, there is a "pre-master secret" or various other terms. I've wrestled with this before in a previous life doing IPSec. Usually you can set debug in the application and it will spew out the secret into the log file. I tried "debug = true" under Default in nova.conf and got lots of debug but no secret. There was two items that looked interesting that were reported as **** in the log: keystone_authtoken.memcache_secret_key and neutron.metadata_proxy_shared_secret. I wasn't sure if those were the secrets I was looking for or not. In this case, I'm looking at the nova-api traffic going to port 8774.
Also, since all of openstack is Python and uses the same "request" and "certifi" packages, it may be possible to generalize this to all of the openstack components.
nova --version report 9.1.1
Related
After following Mike Newswanger's writeup on Building a Secure PKI for Kubernetes (https://www.mikenewswanger.com/posts/2018/kubernetes-pki/), I run last step in the guide to request the certificate from my client machine:
cfssl gencert -config=request-profile.json -hostname=myhost.example.com -tls-remote-ca ca.pem -profile=default csr.json | cfssljson -bare myhost
The error is
{"code":7300,"message":"read tcp 192.168.122.106:37618-\u003e192.168.122.1:8888: read: connection reset by peer"}
Using tcpdump on the multirootca host, I see that cfssl is sending an HTTP request when multirootca is expecting HTTPS.
Nothing in the cfssl documentation that I've been able to find indicates how to force cfssl to use HTTPS on cert requests, and Mike's post indicates that it should just "work" at this point.
Has anyone had success using the latest release of cfssl or am I missing something trivial?
Note: I did have to modify the request-profile.json to remove the <:port> from the ca-server before I was able to get to this point
So after several trials/tribulations, the solution was pretty straightforward. In request-profile.json the URI needs to be https://<ca_server>:<port>.
This github issue https://github.com/cloudflare/cfssl/issues/898 helped point me in the right direction.
Also, if anyone else happens across this and are stuck, don't download the binaries from https://pkg.cfssl.org/ as they are severely outdated. Installing using Go is the best way to get the latest versions that actually work in the write-up mentioned in my Q (as well as the CloudFlare Blog).
I'm having problems enabling TLS in Elasticsearch 7.1.1 running on Windows 7.
I have a single node with certificates created as
elasticsearch-certutil ca
elasticsearch-certutil cert --ca elastic-stack-ca.p12
The elasticsearch.yml file has the following settings
node.name: node1
discovery.type: single-node
xpack.security.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: 'C:\elasticsearch-7.1.1\config\certs\elastic-certificates.p12'
xpack.security.transport.ssl.truststore.path: 'C:\elasticsearch-7.1.1\config\certs\elastic-certificates.p12'
This works fine but when I add the below
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: 'C:\elasticsearch-7.1.1\config\certs\elastic-certificates.p12'
xpack.security.http.ssl.truststore.path: 'C:\elasticsearch-7.1.1\config\certs\elastic-certificates.p12'
and start up elasticsearch I see the following error
[2019-06-25T07:34:19,659][WARN ][o.e.h.AbstractHttpServerTransport]
[node1] caught exception while handling client http traffic, closing
connection Netty4HttpChannel{localAddress=0.0.0.0/0.0.0.0:9200,
remoteAddress=/127.0.0.1:6757}
io.netty.handler.codec.DecoderException:
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
This is repeated every 10-15 seconds.
https is enabled though and I can access the node using https://localhost:9200
I don't know why I receive the above error though as nothing else is running and accessing elasticsearch.
Any help would be much appreciated.
Thanks heaps
It was pointed out to me, on the elastic forum, that the above is a warning and not an error. I still couldn't understand what was causing it as I wasn't running any service or anything else that could be causing it but eventually found something called heartbeat that was running. This was obviously setup in an earlier version/previous installation of Kibana and this was still running, making a call using http and thus causing the above error (this is used for creating dummy data to use for/by to demo Kibana).
I came across this problem too. And IF you have previously installed Elastic Search then high chance you got some residual indices with "red" status, which makes the process of enabling TLS unsuccessful.
Try this command to verify your indices and their statuses
curl -XGET https://localhost:9200/_cat/indices
then delete those with red status.
In our system, virtual hosts configuration is stored in redis. During connection setup, when the SNI is received, we would like to query redis for the correct certificate and key pair to use for the TLS connection and create a new Context instance with that attached.
The bulk of the code is similar to the accepted answer here: Twisted listenSSL virtualhosts
The issue we are facing is that, since accessing the certificates involves an additional network operation, we would like to make the set_tlsext_servername_callback function return a deferred.
Is there a way to tell Twisted/pyOpenSSL to wait until the deferred fires?
Edit: I found this link which seems promising, but falls short of providing a solution: https://mta.openssl.org/pipermail/openssl-dev/2015-January/000480.html
You can find an example of Twisted and SNI here: https://pypi.python.org/pypi/txsni. I would really, really like that callback to be able to take a Deferred. I think that the way to do this would be to pause the underlying transport from delivering any further bytes either in or out (stopReading/stopWriting) and then resume when the Deferred fires, after doing the rest of the SNI dance. However, I'm not even sure if this is possible with OpenSSL, because the SNI is received with the rest of ClientHello and you may need to be able to react immediately to serve the correct certificate. In this worst-possible-case scenario, you could feed the first chunk of bytes you receive into a dummy memory-BIO, wait for the TLS handshake, throw it away and never deliver any generated responses, and then don't initialize your "real" sub-transport until you've decided on which context object to use.
Hope this helps - and if you figure it out, please contribute a patch to TxSNI or Twisted!
I am trying to monitor https traffic with Fiddler, using current newest version:2.4.4.5
I've successfully set up https, certificates and I can see the full https encrypted traffic for example browsing my bank's web site.
...however...
When I trying to monitor an other server I got this error message in the response window:
"Failed to secure existing connection for 77.87.178.160. A call to SSPI failed, see inner exception. InnerException: System.ComponentModel.Win32Exception: The client and server cannot communicate, because they do not possess a common algorithm"
For full Fiddler window see:
The client is not a in this case browser, but a custom client program, which communicates with its own server.
My question: Is this exception misleading and in reality some other error prevents the secure channel to set up?
...or...
We have still chance to monitor this https communication?
Thx in advance
What is the client program?
This error typically indicates that that client application is only offering certain HTTPS ciphers, and those ciphers are not supported by Fiddler.
However, in this case, the specific problem here is almost certainly this: http://blogs.msdn.com/b/ieinternals/archive/2009/12/08/aes-is-not-a-valid-cipher-for-sslv3.aspx
The client is trying to use AES with SSLv3, but that isn't one of the valid ciphers for SSL3. As a consequence, the connection fails.
You might be able to workaround this by clicking Rules > Customize Rules. Scroll down to the Main() function and add the following line within the function:
CONFIG.oAcceptedServerHTTPSProtocols =
System.Security.Authentication.SslProtocols.Ssl3;
Please let me know if this works.
NOTE Current versions of Fiddler offer a UI link for this: Look at the lis of enabled protocols on the HTTPS tab.
Unbelievably this issue is still present some 6 years later.
Just installed the latest version of Fiddle (v5.0.20194.41348), and sure enough on Win7 using Chrome or IE it keeps failing with the dreaded error:
"fiddler.network.https> HTTPS handshake to google.com (for #1) failed. System.ComponentModel.Win32Exception The client and server cannot communicate, because they do not possess a common algorithm"
After some hours of testing, I found a middle ground solution which seems to work with virtually all websites. The aim was to get the highest possible security with no errors in the log. Without needing to add any code, simply changing this line under Tools > Options > HTTPS > Protocols is what worked for me (just copy and paste it):
<client>;ssl3;tls1.1;tls1.2
Basically removed the ssl2 and tls1.0 protocols which leaves us with some pretty decent security and no errors so far. Having spent hours of frustration with this error, hope someone out there might find this useful, and a big thanks to EricLaw who discovered the root of the problem.
Yes I too have seen this error when working outside of fiddler and it was connected with AuthenticateAsServer but only went wrong when using IE10 and not Chrome as the browser.
Odd thing is that it did not break all the time for IE10 using SslProtocols.Tls for the protocol so I will add a bit of code to switch the protocol if one fails
The protocol that can be used also seems to change on if you are using a proxy server like Fiddler or using an invisible server by hijacking the DNS via the hosts file to divert traffic to the server
I am trying to fetch a file over HTTPS in Io language:
url := URL with("https://api.example.com")
url fetch println
And I get this:
Error_0x7f97e1509a80:
location = "/opt/local/lib/io/addons/Socket/io/URL.io:232"
message = "Protocol 'https' unsupported"
I was trying to find something on the net, but, as everybody knows, it's not easy because of the name. I only found this thread http://tech.groups.yahoo.com/group/iolanguage/message/10898 but that's quite old.
How can I get the HTTPS support in Io?
EDIT
I've found that there is a SecureSocket addon, a wrapper over OpenSSL, in Io's source. It wasn't installed when I did sudo port io install on my MacBook with Mountain Lion, though. I tried building it from source, but no luck. It didn't build for me on a Linux machine, either.
EDIT2
I just tried to build Io from source (git clone https://github.com/stevedekorte/io.git) again (using the included script build.sh) and it turned out that cmake did detect OpenSSL:
-- Found OpenSSL: /usr/lib/libssl.dylib;/usr/lib/libcrypto.dylib
But then the SecureSocket addon is not built. Its readme file: https://github.com/stevedekorte/io/tree/master/addons/SecureSocket says:
The DTLS1 bindings are not usable unless the patches in this file are
applied to OpenSSL 0.9.8e. However, this patch includes a
deactivation of the handshake retransmission code in d1_both.c,
making it unsuitable for production environments. I take no
responsibility, etc, etc. If you want to use it anyway, apply the
patches(gathered from various newsgroups and my own experimentation)
and uncomment the commented-out block of build.io. For what it's
worth, DTLS support in OpenSSL is new as of 0.9.8 and is pretty buggy
to begin with. It's a nice idea, but it doesn't seem to be
production ready at all yet. These bindings are no exception.
If you can't get io to do it your best option would be calling an external tool like wget or curl which can and then loading the file/result locally or returning it via a pipe.
For anybody else interested in another workaround, it should be possible to put stud in front of an Io program which will do the SSL stuff. I have not tested that myself yet.
stud - The Scalable TLS Unwrapping Daemon stud is a network proxy that
terminates TLS/SSL connections and forwards the unencrypted traffic to
some backend. It's designed to handle 10s of thousands of connections
efficiently on multicore machines.