I was able to setup nginx as a message server for building a real-time javascript application with Dojo. For the setup I used the nginx http_push_module which can be configured to handle publish/subscribe requests on different “channels”. Channels is “A resource representing an isolated pathway for message transmission. Each channel has a single unique message queue”.
Channels are identified by a id parameter in the url used within the XHR requests.
I'm in the need of implementing some sort of private channel, which the application can use to push messages to the users, but I've no idea of how to implement channel authentication.
Does anybody ever used http_push_module to create private channels or have suggestions about implementing them?
Thanks in advance for your support.
Maybe you can use my fork of the http_push_module module I've been working on, that one implements fine grain security access to channels in it. I just updated the README for you to know how to use it, but it basically uses md5 hashes, it provides expiration times to channels and per-client-IP/per-channel security (it additionally adds jsonp support if you need it):
https://github.com/Kronuz/nginx_http_push_module
Related
I want to be able to link various requests that are coming from same client (browser). I came with adding header based on cookie:
backend servers
description My backend
http-request set-header Request-Id %[req.cook(AspNet.Session),sha1,hex]
server srv_01 127.0.0.1:5000
This is going to be used only for debug purposes when I want to find what user was doing. Should I be worried about performance? My cookie is around 300 bytes. There are also other hash functions (like xxh64 or wt6). Does it makes sense to use it?
Debian Buster, haproxy 2.2, ASP.Net Core as backend server.
Actually, If it related to your debug environment, I think is not bad for performance, but consider this, every sidecar added, has its own overhead to the response time, You can follow one of the following scenarios:
write a action filter to log request/response/userInfo ActionFilter
use some event-based patterns. this approach has less overhead on the response time because the logging will be processed separately and independent of the current request threat.
but I'm quite sure, there are other patterns that can be used to logging requests and user info.
I have an API that limit 10 calls/second from an IP - Lets call this API-1
I have a webapp that consumes API-1. Lets call this WebApp-1
If I my Web App have more traffic and needs to make more calls per second than allowed, how do I design WebApp-1 call to API-1?
Some ideas of how to approach a rate limited API that come to mind:
Raise the API limits for your client key. Probably not your case but may be an option in some cases.
Create/purchase more client accounts (access keys) to the API to raise the overall rate limit. Split traffic among the keys evenly.
Cache results on the querying side (WebApp in your case). It depends on the application, but if WebApp is a browser-based application caching may not be effective as there's no shared cache between clients.
Introduce caching proxy. WebApp makes requests to the proxy which forwards them to the rate limited API. This will help with maintaining the shared cache. Some options to implement the proxy: Nginx, Varnish, AWS API Gateway, etc.
Introduce a query queue (synchronous). Again if WebApp is a browser application, you may need to put a backend service as a proxy between the WebApp and the API. Proxy would keep a steady flow of requests to the API. If there's a burst in incoming requests it would delay processing to respect API's rate limit (WebApp may have to wait longer to get an answer from the proxy). Not really scalable.
Introduce a query queue (asynchronous). WebApp sends requests to the proxy. Proxy acknowledges the receipt and returns a receipt ID. Then either the proxy makes a callback request to the WebApp when response from the API is ready or WebApp is polling the proxy to know if there's any data for a given receipt ID.
Another (obviously shady) solution is making requests from different machines and IPs. Probably not something API owner wants to see!
I have mobile app which heavily depends on apis response, I was using charles proxy and fiddler to see the api calls made by my app and I have noticed for one of get api call I am able to see full url with all request parameters(which is fine) and request headers(which include secure keys).
So using those info anyone can execute that api outside of mobile app. my app has millions of user and if someone run script to increase traffic it also increase load on server. so is there any way I can secure or hide those keys ?
I am able to think only one way of doing it is
encryption on both app and api side
is there any better way of doing it ?
You can implement certificate or public-key pinning in your app (for the leaf or the root-CA-certificate). This makes it harder for an attacker to use a proxy and intercept HTTPS traffic. However with XPosed and SSL-Unpinning module this will still work.
Also keep in mind that APK files can be decompiled easily, therefore you don't have to attack the network traffic.
Therefore the next step is to harden your app to make it resistent against manipulation via XPosed or Frida. Note that good harding frameworks cost a lot of money. Usually the protection offered is raising with the cost.
See also this related question.
I am looking for ways to authorize each individual client request made through the rest proxy. Is there a mechanism to integrate the proxy with existing Kafka ACL's?
I already configured the HTTPS authentication with client certificates so I have a unique client token I can include with every request for authorization purpose. My preferred approach would be to introduce a custom servlet filter that integrates with Kafka ACL system using something like SimpleAclAuthorizer. Unfortunately, the Rest Proxy is not a standard Web application but runs the embedded Jetty, so configuration is a bit more convoluted.
My question is, what is the least intrusive way to to accomplish this?
Thank you in advance.
You can configure a single kafka client credentials for the REST Proxy to use when connecting to Kafka, but today you cannot pass through the credentials of each HTTP(s) client separately. That feature is being worked on and will likely come out in a future release.
RBAC is available now in Confluent Kafka but it is still in preview here is the link
I've read a lot of things about HTTP 2 (which is still in development), so I also heard about the server push feature, but I my head, this is not clear.
Does this server push feature mean that the server will be able to send a response to the client without the latter making a request? Just like a vanilla TCP connection? Or I'm missing the point?
The HTTP2 push mechanism is not a generic server push mechanism like websocket or server sent events.
It is designed for a specific optimisation of HTTP conversations. Specifically when a client asks for a resource (eg index.html) the server can guess that it is going to next ask for a bunch of associated resources (eg theme.css, jquery.js, logo.png, etc. etc.) Typically a webpage can have 10s of such associated requests.
With HTTP/1.1, the server had to wait until the client actually sends request for these associated resources, and then the client is limited by connections to only ask for approx 6 at a time. Thus it can take many round trips before all the associated resources that are needed by a webpage are actually sent.
With HTTP/2, the server can send in the response to the index.html GET push promises to tell the client that it is going to also send theme.css, jquery.js, logo.png, etc. as if the client had requested them. The client can then cancel those pushes or just wait for them to be sent without incurring the extra latency of multiple round trips.
Here is a demo of push with SPDY (the basis for HTTP2) with Jetty https://www.youtube.com/watch?v=4Ai_rrhM8gA . Here is a blog about the push API for HTTP2 and SPDY in jetty: https://webtide.com/http2-push-with-experimental-servlet-api/
Essentially your understanding is correct, however, there is a lot more to it.
The server will only be able to send a resource to the client after a request for an HTTP page has been made and the resources required by that page for it to render properly, i.e. images, JavaScript files, CSS etc, have been identified. The mechanism responsible for this is the server side framework. In Java, this will be Servlet 4 and possibly JSF.
A server can not just send any resource to the client when it feels like it. Only under the above circumstance will it occur and a client will always be able to reject the server request to push a resource.
The mechanism of HTTP/2 server push has been really well designed and to get to grips with it I recommend this overview of HTTP/2 and this in depth article diving into the internals of the HTTP/2 protocol.