WCF does not support request streaming (aka streaming upload of large data) over HTTP with HTTP authentication. My first guess was it is because of authentication handshake causing the streaming request to be send twice to the server. But that is also the case for large request in buffered mode so it doesn't make sense.
You can easily implement request streaming with HTTP authentication in custom ASP.NET http handler. If you have control over the client you can even avoid "multiple requests problem" by doing explicit HTTP HEAD to pre-authenticate to server and then reuse persistent connection to do the actual streaming request with HTTP POST.
So can anybody think of the reason(s) WCF not supporting this? (other than no time to do that)
Thanks
The reason is that you must first send the whole request (even streamed) to get HTTP 401 and follow security handshake and finally send the whole request again. Because streaming is supposed to be used with very large messages this process can be very slow and add unwanted traffic over the network so MS probably did design decision to not allow it at all.
The trick with HEAD request is not implemented in WCF.
Related
TL;DR: How to authenticate against NGINX if auth headers are not supported on client side?
I am building an IoT-related project using NGINX as a reverse proxy for the server side services and 1NCE as the LTE carrier for the mobile devices. All traffic is authenticated based on HTTPBasicAuth over SSL-encrypted connections and handling "normal" requests works as desired.
As mobile service might be interrupted and the Internet connection might be lost, I want to send SMS for critical status reports and alarm notifications. 1NCE supports SMS mobile originated SMS (MO SMS) which are handled by the 1NCE's internal infrastructure and forwarded to a configurable API endpoint. So, MO SMS are not delivered to a specified phone, but forwarded via an API request which I need to process on my side.
According to 1NCE's SMS documentation and in consultation with their customer support, SMS forwarding does not support any authentication headers. SMS forwarding can only be done by specifying an HTTPS URL (including the desired API endpoint) and a port. The incoming SMS is then wrapped in a request to the given URL and sent in the request body.
I want to add authentication to the SMS forwarding endpoint (receiving forwarded SMS on my side) as well and am currently wondering about how I could achieve this. NGINX supports authentication on subrequest which could be used to evaluate incoming requests by an internal service. So my first idea was to add some credentials to each SMS (as I am also responsible for the SMS sending part of the code on the mobile devices, I could implement whatever is needed) and check those credentials with an internal service called by NGINX's subrequest. However, this does not seem to be doable. According to this SO question GET requests are used for the internal subrequests hence any body of the incoming POST request is discarded. Therefore, the credentials of the forwarded SMS would also be not available to my internal auth service. Extending NGINX's auth capabilities by writing an custom Lua-based plugin was my second idea, but this does not only seem to be not feasible but is also not supported by the NGINX instance I am using (Lua modules are disabled, switching to openresty seems to be a big thing).
My last idea would be to forward all incoming requests to a Python web service (written in Flask, other services I am using are also written in Flask) and parsing the forwarded SMS in Python. Based on the result of the credential evaluation I could return an 401/Unauthorized status code if credentials provided in the SMS (which is part of the request body) are invalid and process the request otherwise. However, I think that this approach is quite ugly as all incoming requests need to be passed to Flask and invalid requests are not rejected at the level of my Reverse Proxy.
Do you have any ideas about how to approach this issue? What would be a considerable approach with regards to "best practises"? Can I extend NGINX in a way to solve this or should I completely drop NGINX in favor of a "better" proxy?
Is there any added value in encrypted data we send in http request and response between client and server if we already have SSL / TLS ?
I get SSL/TLS already encrypts traffic across the transport layer for SSL/TLS connections but if we wanted to prevent browser users from reading request and response data, would encrypting it before it is sent add any value in preventing users being able to read it?
For instance, I could go to Network -> XHR requests -> and see what data is being transferred between client and server.
... but if we wanted to prevent browser users from reading request and response data, ...
It looks like you are trying to prevent the simple reverse engineering possible with using the developer console and looking at the requests and responses. While encrypting/decrypting the data in Javascript would make this simple reverse engineering harder it will not prevent reverse engineering in general: ultimately the browser needs to have access to the information in plain so some reverse engineer could tap into the relevant parts of your Javascript based crypto to get access to the data before encryption and after decryption.
In general: if the security of your applications requires protecting against the user of the application itself, then the security design is likely wrong.
I am using puppeteer to create screenshots of websites and I want to eleminate all unneccesary traffic. Besides blocking analytics sites and the like I want to block websocket traffic as well.
I was not able to find something in the puppeteer API. Is there maybe a startup argument for this?
You should be looking for Upgrade headers in http requests. Puppeteer has an API for intercepting requests here, however it's not well documented on what gets passed into that function, so you might have to inspect/debug that out a bit.
In short, all websocket requests start with an HTTP request with an Upgrade header as a handshake of sorts. If you can reject those requests then the following websocket request shouldn't ever happen.
I'm trying to build a complete web caching proxy using Boost Asio and LibCURL, I've already built the server and everything works fine. It receives http requests (GET, POST, upload using POST ...) correctly and also it sends back the responses to the browser for e.g correctly.
Now, I want to extend it, so it can handles https requests. I read about it in LibCURL web site http://curl.haxx.se/libcurl/c/libcurl-tutorial.html (proxy section), I understood how it works and I have a clear idea how it should be done. But I didn't find a good documentation about how proxies handle https requests. and:
what are the possible messages (information, format, length ...) exchanged by the source application and the proxy ?
things to consider.
...
Thanks in advance :-) .
You will receive the CONNECT command in plain text, and respond to it ditto, then the communications after that will be encrypted. If your proxy is to be an SSL endpoint, which is highly problematic given that HTTPS requires a certificate that matches the target host-address, you will then need to enter SSL mode on both connections. More probably you should just start copying bytes in both directions without attempting to process the contents.
I've read a lot of things about HTTP 2 (which is still in development), so I also heard about the server push feature, but I my head, this is not clear.
Does this server push feature mean that the server will be able to send a response to the client without the latter making a request? Just like a vanilla TCP connection? Or I'm missing the point?
The HTTP2 push mechanism is not a generic server push mechanism like websocket or server sent events.
It is designed for a specific optimisation of HTTP conversations. Specifically when a client asks for a resource (eg index.html) the server can guess that it is going to next ask for a bunch of associated resources (eg theme.css, jquery.js, logo.png, etc. etc.) Typically a webpage can have 10s of such associated requests.
With HTTP/1.1, the server had to wait until the client actually sends request for these associated resources, and then the client is limited by connections to only ask for approx 6 at a time. Thus it can take many round trips before all the associated resources that are needed by a webpage are actually sent.
With HTTP/2, the server can send in the response to the index.html GET push promises to tell the client that it is going to also send theme.css, jquery.js, logo.png, etc. as if the client had requested them. The client can then cancel those pushes or just wait for them to be sent without incurring the extra latency of multiple round trips.
Here is a demo of push with SPDY (the basis for HTTP2) with Jetty https://www.youtube.com/watch?v=4Ai_rrhM8gA . Here is a blog about the push API for HTTP2 and SPDY in jetty: https://webtide.com/http2-push-with-experimental-servlet-api/
Essentially your understanding is correct, however, there is a lot more to it.
The server will only be able to send a resource to the client after a request for an HTTP page has been made and the resources required by that page for it to render properly, i.e. images, JavaScript files, CSS etc, have been identified. The mechanism responsible for this is the server side framework. In Java, this will be Servlet 4 and possibly JSF.
A server can not just send any resource to the client when it feels like it. Only under the above circumstance will it occur and a client will always be able to reject the server request to push a resource.
The mechanism of HTTP/2 server push has been really well designed and to get to grips with it I recommend this overview of HTTP/2 and this in depth article diving into the internals of the HTTP/2 protocol.