I have an NGINX config in an API gateway repo that handles requests and routes them to back-end microservices.
Currently I have it so users can use REST to easily make requests. The requests get handled and routed to the appropriate microservice with NGINX.
However, the back-end services also have gRPC integrated. Ideally, I want it so that users can pass in REST requests to the API gateway (easier for external clients), and then I can translate the REST (JSON) to gRPC (protobuf) to route the requests with gRPC (faster for internal communication) to the back-ends.
I believe it would require some sort of middleware in the API gateway that intercepts requests, translates them to protobuf, and then routes them with gRPC pass using NGINX. Is there a way to intercept or manipulate incoming requests on the fly with NGINX?
I found this http://web.archive.org/web/20180322022735/https://tarunlalwani.com/post/request-capturing-nginx-lua/
Which essentially logs the requests that come into NGINX. I feel like it may be on the right path, but I would need to manipulate the requests before they are routed. Maybe a separate script called before the proxy path so instead of logging after the request is routed, the request can be changed before?
Related
I want an API-gateway/reverse-proxy to intercept HTTP requests from a third party (which I have no control over aside from a single URL that gets invoked). I need it to intercept requests to do the following;
Extract a value from the body of the HTTP request from the third party
Invoke a synchronous HTTP request to an existing back-end route using the value from #1 as a param
Route the request to one of two different locations based on the response value of #2
Is it possible to do something like this using NGINX or some other open source reverse proxy (e.g. Envoy, HA Proxy, etc)?
I know I can do this using a bespoke back-end application e.g. .NET Core, Java, etc., but I'm hoping I can avoid having to do this. I need the API gw regardless, so hoping to handle all the routing in this layer if possible...
i am taking a course in web data so i understand that when we want to retrive a webpage on a browser we do a request response cycle using a communication protocol like http or https and a web service is a piece of software which i dont know where it is stored or how it is accessed so we can make two applications from different architectures communicate using a serialization language like XML or JSON i dont know what is the difference between a web service and http they are both a way to connect 2 different computers together and what confused me the more is api which according to the research i did is something used to access web services.
Let's begin with defining all the terms in your question since it's a bit all over the place.
HTTP (Hypertext-Transport Protocol): Allows you to transfer data over the web. Your browser will perform a request using HTTP to your web service.
Service: Any software that performs a specific task. We are interested in a web service, which is typically invoked via HTTP, however this can be anything else such as a Linux signal.
For now, let's assume it listens on HTTP.
API (Application Programming Interface): An interface by which all clients of your software have to abide by to use it. For example, in our web service, we can dictate an API so requests follow some convention.
Let's put it all together now.
You're making a website that wants to calculate the sum of two numbers. First, users will go to http://yoursite.com, and then the browser will always do an HTTP request to the domain yoursite.com on port 80. This will hit either your hosting site or some backend server.
Here you have the option if you're using something like GitHub pages to serve static content or you have some server (i.e., serverd) that will load a file and serve it.
So now the web-browser did an HTTP request and your webpage should load with an index.html. The user can now click on buttons, and everything looks good until they press Calculate -- what happens now?
We want to offload the computation to our backend. We perform an HTTP request to our backend server. We can define an API, that is in our case an endpoint, so that the HTTP request can hit it and it'll return the sum of the two numbers.
How do we return the result? We need to represent the data somehow, and this can be done through a body payload that is encoded as either JSON or XML. Again, this is a serialization format and can encode it in various different ways. JSON is nice because you can parse it easily with JavaScript on the client side.
Great -- so now we got an entire site and it works! Now we can do an HTTP request from our browser straight to the backend according to our setup endpoint and it should fulfill our request. Notice how now we're using the API from the backend server from within our site.
Other keywords you can may run into: CORS, AJAX, Apache Server; good luck!
For my deployment I have a number of 3rd party systems that can only send HTTP POST requests with metrics (I need in the queue) and they cannot be re-configured. My goal is to have specific endpoints (or vhosts) that when POST'd to will automatically route to the correct queue, without needing the necessary routing key and other standard rabbitmq JSON data. As this modification is not possible in the 3rd party systems.
I can't find any way to do this natively as of now, but I believe it may be possible to configure a HTTP reverse proxy in the front, whereby any data sent to the specific endpoint, will be re-directed to the correct rabbitMQ HTTP endpoint, where I could then bolt in the nessary JSON data so it can be parsed by rabbitmq and placed in the realvent queue. I wanted to check if this is the only logical solution to this, or am I missing something obvious that can be done within rabbitmq's administration page or via config files.
I have an application that sends a request to a web service. Unfortunately the application is compiled and the link to the web service is embedded in the application as http. (Yes I know how dumb that is, I didn't write it)
Recently, the 3rd party company is no longer allowing http requests, everything must be https.
The application runs as a webapp on Tomcat. The server is a windows server.
Is there a way to intercept the call to this web service and force it to use https?
As you can't change the application's source code (as it is compiled), and you can't change the web service (as it is 3rd party) either, the only way to solve this problem is making a proxy between the application and web service. To do that, you need to (assume the proxy is running in localhost):
As the web service URL is embedded into the compiled application, in order to let application send HTTP request to our proxy, hosts mapping need to change (e.g. /etc/hosts) to override DNS. For example, if the HTTP request in application is GET http://example.com/api/sample, in /etc/hosts, example.com need to be mapped to 127.0.0.1.
Make a proxy web server in localhost and open the same port as the web service. This proxy is a very simple web server (any backend programming tech can do it), it is only responsible for request-forwarding. In this way, when application send HTTP request to example.com, it sends the request to the proxy server.
After receiving HTTP request from application, the proxy server extract the request URL/header/body and send HTTPS request to example.com's real IP address. Please note: in this HTTPS request, a header host whose value is example.com should be added. 3rd party web service may check this header.
After the real response is returned from example.com, proxy will return it to the application.
Of course, you can also use reverse engineering (Java Decompiler) to get the application's "source code", change the web service URL and then compile again to a webapp. However, as the application may need to update/upgrade and it is may not under your control, this reverse engineering method is not recommended.
You could use a proxy script. Write it in any server-side language you want, for example PHP, set the API URL to this script, the script does the https request for you and pass the results back to your app.
You could also use Apache itself as the proxy and use something like: Apache config: how to proxypass http requests to https
I need to capture and run realtime analysis on messages being exchanged between various web services implemented in Java and client apps. The server code and config can not be modified and is hosted on various servers.
Is it possible to build a proxy layer that will take all calls from the client app and route them to actual web services.
So it needs to do the following:
Accept a config file containing endpoints for various web services that need to be proxied
For each end point, generate a proxy URL
The client apps will point to these proxy URLs
The proxy layer will listen to traffic on these proxy URLs, and route them to real end points.
Track all SOAP traffic in between the client and services and run the necessary analysis.
I considered SoapUI but it does not seem to provide enough control that I need for realtime analysis.
You should start with WCF Routing service. Once you have working communication you can add some custom message processing through custom behaviours or channels to grab SOAP messages and do your analysis.