In OpenSSL doc, said:
If necessary, a write function will negotiate a TLS/SSL session, if not already explicitly performed by SSL_connect(3) or SSL_accept(3). If the peer requests a re-negotiation, it will be performed transparently during the write function operation. The behaviour of the write functions depends on the underlying BIO.
And I have called SSL_set_accept_state().
For the transparent negotiation to succeed, the ssl must have been initialized to client or server mode. This is being done by calling SSL_set_connect_state(3) or SSL_set_accept_state() before the first call to a write function.
So can I write less code？
So can I write less code？
Yes. In some cases it might simplify the code if SSL_accept is not explicitly called. In other cases one might want to clearly separate the initial TLS handshake from the rest of the code for better control, like which errors are expected at which stage of communication and how these are handled.
Related
I've been reading a lot about OpenSSL, specifically the TLS and DTLS APIs. Most of it makes sense, it's a pretty intuitive API once you understand it. One thing has really got me scratching my head though...
When/why would I use BIOs?
For example, this wiki page demonstrates setting up a barebones TLS server. There isn't even a mention of BIOs anywhere in the example.
Now this page Uses BIOs exclusively, not ever using the read and write functions of the SSL struct. Granted it's from 2013, but it's not the only one that uses BIOs.
To make it even more confusing this man page suggests that the SSL struct has an "underlying BIO" without ever needing to set it explicitly.
So why would I use BIOs if I can get away with using SSL_read() and SSL_write()? What are the advantages? Why do some examples use BIOs and others don't? What Is the Airspeed Velocity of an Unladen Swallow?
BIO's are always there, but they might be hidden by the simpler interface. Directly using the BIO interface is useful if you want more control - with more effort. If you just want to use TLS on a TCP socket then the simple interface is usually sufficient. If you instead want to use TLS on your own underlying transport layer or if you want have more control on how it interacts with the transport layer then you need BIO.
An example for such a use case is this proposal where TLS is tunneled as JSON inside HTTPS, i.e. the TLS frames are encoded in JSON and which is then transferred using POST requests and responses. This can be achieved by handling the TLS with memory BIO's which are then encoded to and decoded from JSON.
First, your Q is not very clear. SSL is (a typedef for) a C struct type, and you can't use the dot operator on a struct type in C, only an instance. Even assuming you meant 'an instance of SSL', as people sometimes do, in older versions (through 1.0.2) it did not have members read and write, and in 1.1.0 up it is opaque -- you don't even know what its members are.
Second, there are two different levels of BIO usage applicable to the SSL library. The SSL/TLS connection (represented by the SSL object, plus some related things linked to it like the session) always uses two BIOs to respectively send and receive protocol data -- including both protocol data that contains the application data you send with SSL_write and receive with SSL_read, and the SSL/TLS handshake that is handled within the library. Much as Steffen describes, these normally are both set to a socket-BIO that sends to and receives from the appropriate remote host process, but they can instead be set to BIOs that do something else in-between, or even instead. (This normal case is automatically created by SSL_set_{,r,w}fd which it should be noted on Windows actually takes a socket handle -- but not any other file handle; only on Unix are socket descriptors semi-interchangeable with file descriptors.)
Separately, the SSL/TLS connection itself can be 'wrapped' in an ssl-BIO. This allows an application to handle an SSL/TLS connection using mostly the same API calls as a plain TCP connection (using a socket-BIO) or a local file, as well as the provided 'filter' BIOs like a digest (md) BIO or a base64 encoding/decoding BIO, and any additional BIOs you add. This is the case for the IBM webpage you linked (which is for a client not a server BTW). This is similar to the Unix 'everything is (mostly) a file' philosophy, where for example the utility program grep, by simply calling read on fd 0, can search data from a file, the terminal, a pipe from another program, or (if run under inetd or similar) from a remote system using TCP (but not SSL/TLS, because that isn't in the OS). I haven't encountered many cases where it is particularly beneficial to be able to easily interchange SSL/TLS data with some other type of source/sink, but OpenSSL does provide the ability.
Does OpenSSL and/or the SSL/TLS protocol provide some kind of built in protection against infinite renegotiation?
In particular, is it possible for SSL_read() to continue executing forever because the remote side (possibly maliciously) keeps requesting renegotiations without sending payload data?
I am worried about this because I want to service a number of SSL connections from a single thread using a polling mechanism and also ensure a form of fairness where the processing of I/O on one connection does not lead to starvation of I/O on the other connections.
When I call regular read() on a socket in nonblocking mode, I know it cannot keep executing forever, because the buffer will fill up eventually.
However, since SSL_read() can handle renegotiations transparently, it seems to me that if the remote side (possibly maliciously) keeps requesting renegotiations without sending payload data, and the underlying transport layer is fast enough to make the underlying reads and writes never fail with EWOULDBLOCK, then SSL_read() could end up executing forever, and thereby starving the other connections.
Therefore my question: Does OpenSSL or the protocols have mechanisms for avoiding that? The question applies equally to SSL_write() by the way.
EDIT: For example, can I be sure that SSL_read() will return with an SSL_ERROR_WANT_READ/SSL_ERROR_WANT_WRITE indication before engaging in multiple renegotiations, even if the underlying read/write operations never fail with EWOULDBLOCK?
EDIT: For the purpose of this question, assume that I am using a regular socket BIO (BIO_s_socket()) and that the underlying socket is in nonblocking mode.
There is no built-in protection in OpenSSL. But you can use SSL_CTX_set_info_callback or similar to set a function which gets called on each negotiation. This way you can cut the connection if too much renegotiations happen inside the same connection. See Protect against client-initiated renegotiation DoS in OpenSSL/Python for more information.
I'm still trying to master Twisted while in the midst of finishing an application that uses it.
My question is:
My application uses LineReceiver.sendLine to send messages from a Twisted TCP server.
I would like to know if the sendLine succeeded.
I gather that I need to somehow add a success (and error?) callback to sendLine but I don't know how to do this.
Thanks for any pointers / examples
You need to define "succeeded" in order to come up with an answer to this.
All sendLine does immediately (probably) is add some bytes to a send buffer. In some sense, as long as it doesn't raise an exception (eg, MemoryError because your line is too long or TypeError because your line was the number 3 instead of an actual line) it has succeeded.
That's not a very useful kind of success, though. Unfortunately, the useful kind of success is more like "the bytes were added to the send buffer, the send buffer was flushed to the socket, the peer received the bytes, and the receiving application acted on the data in a persistent way".
Nothing in LineReceiver can tell you that all those things happened. The standard solution is to add some kind of acknowledgement to your protocol: when the receiving application has acted on the data, it sends back some bytes that tell the original sender the message has been handled.
You won't get LineReceiver.sendLine to help you much here because all it really knows how to do is send some bytes in a particular format. You need a more complex protocol to handle acknowledgements.
Fortunately, Twisted comes with a few. twisted.protocols.amp is one: it offers remote method calls (complete with responses) as a basic feature. I find that AMP is suitable for a wide range of applications so it's often safe to recommend for new development. It largely supersedes the older twisted.spread (aka "PB") which also provides both remote method calls and remote object references (and is therefore more complex - in my experience, more complex than most applications need). There are also some options that are a bit more standard: for example, Twisted Web includes an HTTP implementation (HTTP, as you may know, is good at request/response style interaction).
I am building an API and I was wondering is it worth having a method in an API that returns the status of the API whether its alive or not?
Or is this pointless, and its the API users job to be able to just make a call to the method that they need and if it doesn't return anything due to network issues they handle it as needed?
I think it's quite useful to have a status returned. On the one hand, you can provide more statuses than 'alive' or not and make your API more poweful, and on the other hand, it's more useful for the user, since you can tell him exactly what's going on (e.g. 'maintainance').
But if your WebService isn't available at all due to network issues, then, of course, it's up to the user to catch that exception. But that's not the point, I guess, and it's not something you could control with your API.
It's useless.
The information it returns is completely out of date the moment it is returned to you because the service may fail right after the status return call is dispatched.
Also, if you are load balancing the incoming requests and your status request gets routed to a failing node, the reply (or lack thereof) would look to the client like a problem with the whole API service. In the meantime, all the other nodes could be happily servicing requests. Now your client will think that the whole API service is down but subsequent requests would work just fine (assuming your load balancer would remove the failed node or restart it).
HTTP status codes returned from your application's requests are the correct way of indicating availability. Your clients of course have to be coded to tolerate and handle them.
What is wrong with standard HTTP response status codes? 503 Service Unavailable comes to mind. HTTP clients should already be able to handle that without writing any code special to your API.
Now, if the service is likely to be unavailable frequently and it is expensive for the client to discover that but cheap for the server, then it might be appropriate to have a separate 'health check' URL that can quickly let the client know that the service is available (at the time of the GET on the health check URL).
It is not necessary most of the time. At least when it returns simple true or false. It just makes client code more complicated because it has to call one more method. Even if your client received active=true from service, next useful call may still fail. Let you client make the calls that they need during normal execution and have them handle network, timeout and HTTP errors correctly. Very useful pattern for such cases is called Circuit Breaker.
The reasons where status check may be useful:
If all the normal calls are considered to be expensive there may be an advantage in first calling lightweight status-check method (just to avoid expensive call).
Service can have different statuses and client can change its behavior depending on these statuses.
It might also be worth looking into stateful protocols like XMPP.
Few methods in my WCF service are quite time taking - Generating Reports and Sending E-mails.
According to current requirement, it is required so that Client application just submits the request and then do not wait for the whole process to complete. It will allow user to continue doing other operations in client applications instead of waiting for the whole process to finish.
I am in a doubt over which way to go:
AsyncPattern = true OR
IsOneWay=true
Please guide.
It can be both.
Generally I see no reason for WCF operation to not be asynchronous, other than developer being lazy.
You should not compare them, because they are not comparable.
In short, AsyncPattern=True performs asynchronous invocation, regardless of whether you're returning a value or not.
OneWay works only with void methods, and puts a lock on your thread waiting for the receiver to ack it received the message.
I know this is an old post, but IMO in your scenario you should be using IsOneWay on the basis that you don't care what the server result is. Depending on whether you need to eventually notify the client (e.g. of completion or failure of the server job) then you might also need to look at changing the interface to use SessionMode=required and then using a Duplex binding.
Even if you did want to use asynchronous 2-way communication because your client DID care about the result, there are different concepts:
AsyncPattern=true on the Server - you would do this in order to free up server resources, e.g. if the underlying resource (?SSRS for reporting, Mail API etc) supported asynchronous operations. But this would benefit the server, not the client.
On the client, you can always generate your service reference proxy with "Generate Asynchronous Operations" ticked - in which case your client won't block and the callback will be used when the operation is complete.