I've recently been passed an embedded project where an MCU uses libwebsockets (version 3.1.0) to setup a websockets client. The MCU is connected to a SIMCOM modem for 4g connectivity.
Up until now the communication was non-secured: SSL context not set and modem configured to provide just a TCP link to the server. The server URI was a wss one, but security was not enforced.
I've now been asked to set the communication to use SSL/TLS with server and client authentication.
Having never used libwebsockets before and being short on time, my idea was to:
a) Leave libwebsockets set as it was, so with no SSL context set up.
b) Configure the modem to creat an SSL/TLS context and connect to the server through that one.
I did this and I can see my device sending the request to switch from http to websockets protocol, as well as the server's successful reply (code 101). But my problem is that the server's reply never gets processed by libwebsockets and the connection is dropped after a few seconds.
I can se3 that the bytes that make up the server's reply are received by the modem and passed into the MCU's buffer used to pass data into libwebsockets, but then the library never calls its net_recv method to actually read from such buffer.
Also, libwebsockets is currently built with TLS support and uses mbedTLS as SSL/TLS library. But, as said about, SSL context is currently left disabled (ssl_connection in the connection info struct is left set to 0).
So, I was wondering:
Is my approach something that can work at all? Or should I setup the SSL context from libwebsockets and let the modem setup just a TCP connection?
If I were to setup the SSL context from libwebsockets, is there a way to pass certificates and keys tot the library as just C arrays? Or do I have to have them stored as files on a filesystem and then pass their paths to the lib?
Also, I should add that the MCU has a second connection to the server, an HTTPS one, that one too setup with client and server authentication and that works with no problems at all. Therfore, I'm sure that the modem is correctly configured.
If needed, I will be able to provide more info on library configuration and used from Monday, because I'm currently travelling and don't have access to the code. But I wanted to get the ball rolling.
Thanks in advance for your help.
Yesterday, finally I had time to look at the code again. It turns out that the problem was in my code (this was always a strong possibility).
The local implementation of the net_send had a bug and returned always negative values. Thus, libwebsockets thought that the HTTP request to switch protocol had failed and hence was closing down the connection.
So, to answer the two questions above:
Yes, it's possible to setup libwebsockets to not handle SSL/TLS and then provide a SSL/TLS connection from a lower layer (the modem in my case).
It is possible to store certificates and keys in C arrays of bytes and pass them to libwebsockets if you create a custom platform implementation for your platform and create a custom implementation of POSIX like methods (open, close, read, etc).
By default the library expects to work on a POSIX like filesystem, though. So, realising a custom implementation can be a bit of a job.
The above is true for libwebsockets version 3.1.0. I haven't used any other version of the library, so things might have changed since then.
All in all, I think that configuring libwebsockets to handle SSL and the TLS handshake and then provide it with only a TCP connection is the best way of using the library.
I chose a different strategy for the wrong reasons (tight deadline + not being familiar with the codebase), but I'm definitely planning on reviewing my approach at a later date.
Related
Suppose I am sending "hello" to an api over ssl. My understanding is there would be a symmetric key exchange established over ssl and then the message "hello" will be encrypted using that symmetric key and sent over to the other server.
Now my question is, the next time I send a "hello 2", does the symmetric key exchange happen again? My guess would be that if it's a persistent connection, there would be no need for the key exchange again. Can someone confirm?
Meta: this doesn't appear to me to be programming, although it might be development and is mostly dupe How long does SSL connection between a client and a server persist? .
It depends on the application protocol used on top of SSL (which since 1999 is really TLS, although many things e.g. implementation classes still use the old name) and usually the implementations at both ends. For example, HTTP/1.1 defaults to connection persistence (which was often done in 1.0 as an extension called keep-alive), but either endpoint can change this by specifying connection: close, and even if the connection is kept open can choose to close it anytime later, perhaps after a minute or two, perhaps after a day or a week. The HTTPS implementation in browsers usually keeps connections open for a little while but has limits on the total connections open so those that haven't been used recently may need to be closed when others are opened. Other applications, libraries, and platforms vary. Other protocols also vary; for example an email agent using SMTPS would normally make a connection, transmit one or more emails, and then disconnect.
In addition, SSL-now-TLS through 1.2 supports session resumption, which allows the keyexchange (and other handshake results) performed on one connection to be saved (at both endpoints, or with the 'ticket' option, at client only) and reused on a new connection, for as long as the endpoint(s) agree; implementations usually call this session caching. See e.g. RFC 5246 section 7.3 specifically the part starting in the middle of page 36, and for one fairly common server (Apache) see SessionCache and SessionCacheTimeout directives. Resumption uses a new handshake but not a full keyexchange on that handshake.
However, this creates a security vulnerability if an endpoint's sesssion cache is compromised, so TLS 1.3 replaces it with a different method using dynamically created PSKs; see RFC 8446 section 2.2. This allows either a partial handshake (doing the actual keyexchange with [EC]DHE but authentication tied to the previous session by the PSK rather than full certificate-based authentication) which provides forward secrecy, or a minimal handshake (using the PSK both as the new initial secret and for authentication) which does not.
If you want an answer for specific software, and specific server(s), you need to look at the capabilities and configuration, and often also the current status, of that software and those server(s).
We are using the SIM868E module for connection via GSM, with internal communication over UART.
We need the SIM868E module to communicate with an HTTPS server, (using SSL), however after sending the commands AT+HTTPSSL=?, AT+HTTPSSL=1 or AT+CIPSSL=? the SIM868E module responds:
+CME Error: unknown
(with CMEE level 2).
The specs note that SSL/TLS are supported by this module: (https://simcom.ee/modules/gsm-gprs-gnss/sim868e/).
The firmware on the chip (requested using AT+CGMR) is Revision:1418B02SIM868E32_BLE_EAT.
How can we fix this problem?
According to 2019 SIMCom products catalogue it seems that for SIM868E device there's not support for TLS feature (TLS is the standard name for the SSL protocol).
Just in case the link becomes unreachable, I attach a screenshot of the relevant page of the document:
as you can see, TLS dot is "empty".
Nevertheless, I have to say that SIMCom documents are sometimes contradictory and confusing: in fact in the SIM868E flier claims that the SSL SW feature is supported, but in the SIM800 series AT command manual there's no mention of SSL AT commands.
I will update in case I find any new piece of information about it.
I would honestly avoid trying strange things like flashing the SW of another module. It would be like searching for trouble. And in your case you would lose your positioning capabilities (GPS/GNSS).
Talking about your issue in particular, you'll probably have to compile an SSL library on your host processor (for SSL handshake and data encryption) using the TCP/HTTP commands of your device to transport data to the server. It's not a simple job.
I'm testing SSL/TLS stream proxying within NGINX that will connect to a web server using gnutls as the underlying TLS API. Using the command line test tool in gnutls (gnutls-serv) the entire process works, but I can't understand the logic:
the NGINX client (proxying HTTP requests from an actual client to the gnutls server) seems to want to handshake the connection multiple times. In fact in most tests it seems to handshake 3 times without error before the server will respond with a test webpage. Using wireshark, or just debugging messages, it looks like the socket on the client side (in the perspective of the gnutls server) is being closed and reopened on different ports. Finally on the successful connection, gnutls uses a resumed sessions, which I imagine is one of the previously mentioned successful handshakes.
I am failing to find any documentation about this sort of behaviour, and am wondering if this is just an 'NGINX thing.'
Though the handshake eventually works with the test programs, it seems kind of wasteful (to have multiple expensive handshakes) and implementing handshake logic in a non-test environment will be tricky without actually understanding what the client is trying to do.
I don't think there are any timeouts or problems happening on the transport, the test environment is a few different VMs on the same subnet connected between 1 switch.
NGINX version is the latest mainline: 1.11.7. I was originally using 1.10.something, and the behaviour was similar though there were more transport errors. Those errors seemed to get cleaned up nicely with upgrading.
Any info or experience from other people is greatly appreciated!
Use either RSA key exchange between NGINX and the backend server or use SSLKEYLOGFILE LD_PRELOAD for NGINX to have the necessary data for Wireshark to decrypt the data.
While a single incoming connection should generate just one outgoing connection, there may be some optimisations in NGINX to fetch common files (favicon.ico, robots.txt).
I'm working at a server that needs to support wss://. The server needs to process the websocket header, to identify the request, and then may decide to pass the SSL context to a worker process. For now, the server uses OpenSSL for SSL comunications, but from my understanding sharing a secure socket between processes is not possible with OpenSSL (tried with SSL_SESSION in parent process and d2i_SSL_SESSION/SSL_CTX_add_session in child process) - reference: http://openssl.6102.n7.nabble.com/How-to-share-SSL-sessions-between-parent-and-child-process-when-doing-fork-exec-td11077.html.
I'm looking to other SSL libraries that may allow this, currently looking at NSS.
Is this possible with any mature open source SSL library?
After a few months of trying to find a way to achieve this with libssl, I decided to make my own TLS implementation. I found no way of implementing this functionality without understanding and modifying libssl (or libressl). I still think is possible, I just didn't find a way. I've implemented a TLS library from scratch and put it on github. Now I have the two needed functions tls_export_context and tls_import_context.
Is this possible with any mature open source SSL library?
I don't think it is possible with any SSL library which is implemented in user-space because then you would continuously need to share the state of a single SSL connection among multiple processes. Contrary to this the state of the underlying TCP connection is managed inside the kernel and there is only a single state even if the same connection is open by parent and child process.
And I don't know of any SSL library which is not implemented in user-space.
WCF supports several different protocols for communicating. My WCF services are deployed on the same machine only. I just wanted to know whether TCP is more effecient than HTTP or should i go with HTTP?
If your WCF services are on the same machine, use named pipes. I've found this flow chart helpful.
.
If your service will only run on the same machine, then try using NetNamedPipeBinding.
Of course, in any case, you should measure the performance you receive using realistic test data.
The advantage of HTTP - Application layer (7 in the OSI model) - is
close to user (human) usage, via text commands (and many responses)
one can use telnet (to a port where an application dialogs via http protocol) for instance and issue some simple commands to dialog with the remote server
the http protocol deals for you with otherwise complex actions
HTTP is (usually) based on TCP (transport) / IP (Network). Thus all the advantages described above bear a performance penalty. You could define yourself an application with a more flexible protocol (at the user/application level) but it usually requires more programming, like dealing with issues that were already included in HTTP. Also, as the name protocol implies, nobody will understand your own protocol if you define one, unlike http. You'll have to design, program and build not only the server side, but also the client side. Clients will have to install your program and use it.
HTTP is built on top of TCP, therefore TCP will be definitely faster. Also HTTP has to parse the text headers which is another bunch of time spent. If your use case allows that, go with TCP.
HTTP is a protocol on top of TCP, so it's most likely faster NOT to add an additional protocol on top. See also the OSI Model.
You can clearly see that HTTP is on top of TCP here:
http://en.wikipedia.org/wiki/OSI_model
OR
even better here:
http://en.wikipedia.org/wiki/TCP/IP_model
As has been said before, TCP is the transmission control protocol, HTTP is a protocol on top. You can create your own custom protocol that could be more efficient as it would not have some of the http baggage. I had to do this to grab frame numbers from a video stream being recorded on a remote computer.