I want to request some api and set the response as a nginx variable. But it says "set_by_lua_block" directive is not allowed here. How can I achieve this?
http {
set_by_lua_block $hostIp {
local http = require 'resty.http'
local httpc = http.new()
local res, err = httpc:request_uri('http://some-pai')
local body, err = res:read_body()
ngx.log(ngx.INFO, "Using ngx.INFO")
ngx.log(ngx.INFO, body)
return body
}
...
}
set_by_lua_block is not allowed in http context
https://github.com/openresty/lua-nginx-module#set_by_lua
set_by_lua_* may be used within server context.
But your code will not work anyway because resty.http uses cosocket API.
At least the following API functions are currently disabled within the
context of set_by_lua:
Output API functions (e.g., ngx.say and ngx.send_headers)
Control API functions (e.g., ngx.exit)
Subrequest API functions (e.g., ngx.location.capture and ngx.location.capture_multi)
Cosocket API functions (e.g., ngx.socket.tcp and ngx.req.socket).
Sleeping API function ngx.sleep.
If you really need to request something once before nginx start - write script and set environment variable. Then
set_by_lua $my_var 'return os.getenv("MY_VAR")';
Related
Anyone can suggest a correct place to inject a code in ocelot (a handler or similar) to get a request, look at it and if there is a header "Accept" with version specs add it to the path.
For a header value application/vnd.myapp.v2+json it would signal that we want to call API v2 and adjust request accordingly.
this logic needs to be executed before route rules are applied because downstream routes will have version in the path:
a call GET /teams/ (with accept header application/vnd.myapp.v2+json) becomes
a call GET /v2/teams/ which, using redirect rule will be send to a service that handles teams calls V2.
example logic (need adjustment to add version in the beginnig)
private static Microsoft.AspNetCore.Http.PathString AppendVersionInPath(DownstreamContext ctx)
{
if (ctx.HttpContext.Request.Headers.TryGetValue("accept", out var acceptHeaderValue))
{
var resultString = Regex.Match(acceptHeaderValue, #"\d+").Value;
if (resultString.Length > 0)
{
var versionPath = $"/v{resultString}";
ctx.HttpContext.Request.Path = ctx.HttpContext.Request.Path.Add(new Microsoft.AspNetCore.Http.PathString(versionPath));
}
}
return ctx.HttpContext.Request.Path;
}
After some considerations we ended up using .net rewrite functionality by implementing pipeline handler that takes request and rewrites URL path based on the header value (if present)
I do not expect this to be the best answer but almost certainly ocelot does not allow such transformation.
Objective:
We want few API calls should go to mock-server(https://192.x.x.x:8001) and others should go to an actual downstream application server(https://dev.api.acme.com).
Setup :
On local, mock server is up with standalone jar on port 8001. e.g https://192.x.x.x:8001
In application config file (config.property)downstream system(which need to mock) defined with mockserver IP i.e https://192.x.x.x:8001
Testing scenario and problem:
1.
Scenario: pathMatches('/profile/v1/users/{id}/user')
* karate.proceed('https://dev.api.acme.com')
* def response = read ('findScope.json')
* def responseStatus = 200ˀˀ
* print 'created response is: ' + response
Now, when we hit API request via postman or feature file then it does karate.proceed properly to https://dev.api.acme.com/profile/v1/users/123/user instead of 192.x.x.x. However, in this request, host is referring to https://192.x.x.x:8001 instead of https://dev.api.acme.com which create a problem for us.
How can we override request header in this case? I did try with karate.set and also with header host=https://192.x.x.x:8001 but no luck.
Thanks!
Please see if the 1.0 version works: https://github.com/intuit/karate/wiki/1.0-upgrade-guide
Unfortunately https proxying may not work as mentioned. If you are depending on this, we may need your help (code contribution) to get this working
If the Host header is still not mutable, that also can be considered a feature request, and here also I'd request you to consider contributing code
I use Ktor for a backend service and I'd like to log incoming requests. I have the feature installed and everything is great but how I can obtain the remote IP address?
call.request.origin.remoteHost
I use this line but I get a hostname, not the IP. I use the standard getByName method from the InetAddress class to get the IP.
Is there a better way?
You can't do this with Ktor as it is not Ktor's job to resolve IP addresses from domains.
What you can use however is Java's InetAddress:
val url = "http://google.com";
val ip = Inet4Address.getByName(url);
Depends on how accurate you want this information to be.
If you read through Ktor code, you'll see that the remoteHost is actually set from the HTTP X-Forwarded-Host header.
From API documentation:
* NEVER use it for user authentication as it can be easily falsified (user can simply set some HTTP headers
* such as X-Forwarded-Host so you should NEVER rely on it in any security checks.
* If you are going to use it to create a back-connection please do it with care as an offender can easily
* use it to force you to connect to some host that is not intended to be connected to so that may cause
* serious consequences.
A better way might be to get the IP from the application engine itself. Unfortunately, the ApplicationCall itself is private, so you'll have to resort to reflection, which isn't optimal:
class RoutingApplicationCall(private val call: ApplicationCall,
Still, this is possible:
// Getting the private field through reflection
val f = context::class.memberProperties.find { it.name == "call" }
f?.let {
// Making it accessible
it.isAccessible = true
val w = it.getter.call(context) as NettyApplicationCall
// Getting the remote address
val ip: SocketAddress? = w.request.context.pipeline().channel().remoteAddress()
println("IP: $ip")
}
I am developing a backend system for my application on Google App Engine.
My application and backend server communicating with json. Like http://server.example.com/api/check_status/3838373.json or only http://server.example.com/api/check_status/3838373/
And I am planning to use CloudFlare for caching JSON pages.
Which one I should use on header? :
Content-type: application/json
Content-type: text/html
Is CloudFlare cache my server's responses to reduce my costs? Because I'll not use CSS, image, etc.
The standard Cloudflare cache level (under your domain's Performance Settings) is set to Standard/Aggressive, meaning it caches only certain types by default scripts, stylesheets, images. Aggressive caching won't cache normal web pages (ie at a directory location or *.html) and won't cache JSON. All of this is based on the URL pattern (e.g. does it end in .jpg?) and regardless of the Content-Type header.
The global setting can only be made less aggressive, not more, so you'll need to setup one or more Page Rules to match those URLs, using Cache Everything as the custom cache rule.
http://blog.cloudflare.com/introducing-pagerules-advanced-caching
BTW I wouldn't recommend using an HTML Content-Type for a JSON response.
By default, Cloudflare does not cache JSON file. I've ended up with config a new page rule:
https://example.com/sub-directiory/*.json*
Cache level: Cache Everything
Browser Cache TTL: set a timeout
Edge Cache TTL: set a timeout
Hope it saves someone's day.
The new workers feature ($5 extra) can facilitate this:
Important point:
Cloudflare normally treats normal static files as pretty much never expiring (or maybe it was a month - I forget exactly).
So at first you might think "I just want to add .json to the list of static extensions". This is likely NOT want you want with JSON - unless it really rarely changed - or is versioned by filename. You probably want something like 60 seconds or 5 minutes so that if you update a file it'll update within that time but your server won't get bombarded with individual requests from every end user.
Here's how I did this with a worker to intercept all .json extension files:
// Note: there could be tiny cut and paste bugs in here - please fix if you find!
addEventListener('fetch', event => {
event.respondWith(handleRequest(event));
});
async function handleRequest(event)
{
let request = event.request;
let ttl = undefined;
let cache = caches.default;
let url = new URL(event.request.url);
let shouldCache = false;
// cache JSON files with custom max age
if (url.pathname.endsWith('.json'))
{
shouldCache = true;
ttl = 60;
}
// look in cache for existing item
let response = await cache.match(request);
if (!response)
{
// fetch URL
response = await fetch(request);
// if the resource should be cached then put it in cache using the cache key
if (shouldCache)
{
// clone response to be able to edit headers
response = new Response(response.body, response);
if (ttl)
{
// https://developers.cloudflare.com/workers/recipes/vcl-conversion/controlling-the-cache/
response.headers.append('Cache-Control', 'max-age=' + ttl);
}
// put into cache (need to clone again)
event.waitUntil(cache.put(request, response.clone()));
}
return response;
}
else {
return response;
}
}
You could do this with mime-type instead of extension - but it'd be very dangerous because you'd probably end up over-caching API responses.
Also if you're versioning by filename - eg. products-1.json / products-2.json then you don't need to set the header for max-age expiration.
You can cache your JSON responses on Cloudflare similar to how you'd cache any other page - by setting the Cache-Control headers. So if you want to cache your JSON for 60 seconds on the edge (s-maxage) and the browser (max-age), just set the following header in your response:
Cache-Control: max-age=60, s-maxage=60
You can read more about different cache control header options here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
Please note that different Cloudflare plans have different value for minimum edge cache TTL they allow (Enterprise plan allows as low as 1 second). If your headers have a value lower than that, then I guess they might be ignored. You can see the limits here:
https://support.cloudflare.com/hc/en-us/articles/218411427-What-does-edge-cache-expire-TTL-mean-#summary-of-page-rules-settings
My application is generating some absolute links via $this->get('request')->getHost().
Problem is: when I try to run testcases, I get following error message:
[exception] 500 | Internal Server Error | Twig_Error_Runtime
[message] An exception has been thrown during the rendering of a template ("Undefined index: HTTP_HOST") in "::base.html.twig" at line 69.
Somehow it's clear to me that there is no host when calling my app via CLI, but I think there must be a way to prevent Symfony2 from throwing that error.
Anyone knows how to get rid of it?
You could create the request like this:
$request = Request::create('http://example.com/path');
That will make the HTTP host be set.
Maybe what you could do is to inject the host you need directly in the request headers before calling the getter. The host is retrieved by looking at various parameter values. First, the headers parameter X_FORWARDED_HOST is checked to see if it is set. If it is set, it is returned otherwise the method getHost checks if the headers parameter HOST is set then the if the server parameter SERVER_NAME is set and finally if the server parameter SERVER_ADDR is set.
What you could try is to set the header parameter HOST like this before calling the getHost method:
$request = $this->get('request');
$request->headers->set('HOST', 'yourhosthere');
$request->getHost(); // Should return yourhosthere
That being said, I'm not sure this will solve the problem because the error you mentioning tells us that the template tries to retrieve the value of the index HTTP_HOST but it is not defined. Looking at the methods $request->getHost and $request->getHttpHost, I don't see anything trying to retrieve a value having HTTP_HOST as the index but I could have missed it. Could you post the file ::base.html.twig to see if the problem could be lying there.
Regards,
Matt
Thanks guys- your answers lead me into the right direction.
This is no Symfony2 issue, as i figured out:
It's just the facebook API PHP wrapper which directly accesses the SERVER parameters. This code solved my issue:
$facebook = $this->container->get('facebook');
$returnUrl = 'http://'.$request->getHost();
$returnUrl .= $this->container->get('router')->generate('_validate_facebook');
$_SERVER['HTTP_HOST'] = $request->getHost();
$_SERVER['REQUEST_URI'] = $request->getRequestUri();
$loginUrl = $facebook->getLoginUrl(array(
'req_perms' => 'publish_stream',
'next' => $returnUrl,
));
return $loginUrl;
now my app runs from web and CLI again