When I use openresty to monitor IP through Lua monitor, why does access_by_lua_file call twice when accessing the root directory
Here's how I use it:
http {
access_by_lua_file lua/test.lua;
server{
location / {
default_type text/html;
}
}
}
https://nginx.org/en/docs/http/ngx_http_index_module.html
It should be noted that using an index file causes an internal redirect
That is, the request to the root (/) is internally redirected to the /index.html.
Here is a demo:
http {
access_log /dev/stdout;
access_by_lua_block {
ngx.log(ngx.INFO, ngx.var.uri, ' ', ngx.req.is_internal())
}
server {
listen 8888;
location / {
default_type text/html;
}
}
}
curl localhost:8888/index.html:
2020/08/17 15:14:22 [info] 22411#22411: *5 [lua] access_by_lua(nginx.conf:15):2: /index.html false, client: 127.0.0.1, server: , request: "GET /index.html HTTP/1.1", host: "localhost:8888"
127.0.0.1 - - [17/Aug/2020:15:14:22 +0300] "GET /index.html HTTP/1.1" 200 14 "-" "curl/7.68.0"
curl localhost:8888/:
2020/08/17 15:15:31 [info] 22411#22411: *6 [lua] access_by_lua(nginx.conf:15):2: / false, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", host: "localhost:8888"
2020/08/17 15:15:31 [info] 22411#22411: *6 [lua] access_by_lua(nginx.conf:15):2: /index.html true, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", host: "localhost:8888"
127.0.0.1 - - [17/Aug/2020:15:15:31 +0300] "GET / HTTP/1.1" 200 14 "-" "curl/7.68.0"
I'm using Traefik to implement a reverse proxy an Apache docker container with Letsencrypt. Everything works well, except when the backend returns a 301/302 Redirect for some resources.
For example:
curl https://example.com/js/conf.js
> GET /js/conf.js HTTP/2
> Host: example.com
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/2 302
< cache-control: no-store, no-cache, must-revalidate
< content-type: text/html; charset=iso-8859-1
< date: Wed, 27 Mar 2019 16:50:05 GMT
< expires: Thu, 01 Jan 1970 00:00:00 GMT
< location: http://example.com/it/js/conf.js
< server: Apache/2.4.34 (Unix) OpenSSL/1.0.2l
< content-length: 224
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved here.</p>
</body></html>
I expect traefik to set the location header to https://example.com/it/js/conf.js instead.
This is the config I'm using:
debug = true
logLevel = "info"
defaultEntryPoints = ["https","http"]
#[accessLog]
[api]
dashboard = true
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[retry]
[docker]
endpoint = "unix:///var/run/docker.sock"
watch = true
exposedByDefault = false
[acme]
email = "test#example.com"
storage = "/etc/traefik/acme/acme.json"
entryPoint = "https"
onHostRule = true
[acme.httpChallenge]
entryPoint = "http"
I'm running the container with the following labels:
--label traefik.enable=true --label traefik.website.frontend.rule=Host:example.com --label traefik.frontend.redirect.entryPoint=https
I had a somewhat similar issue with traefik and gunicorn here. In my case I had to pass the X-FORWARDED_PROTO header to let the webserver know that the initial connection was done over HTTPS. That might be why your Apache is redirecting to HTTP.
I have been struggling to implement an RDP probe to check multiple ports in Windows machines using Prometheus Blackbox.
So far I manage to check DNS, ping, ports 80,8080 but I cannot manage to test 3389!
As a rule of thumb I would like to be able to ping/probe any ports that have services running on this hosts
My blackbox.yml is:
modules:
http_2xx:
prober: http
http:
http_get_2xx:
prober: http
http:
method: GET
http_post_2xx:
prober: http
timeout: 5s
http:
method: POST
headers:
Content-Type: application/json
body: '{}'
tcp_connect:
prober: tcp
pop3s_banner:
prober: tcp
tcp:
query_response:
- expect: "^+OK"
tls: true
tls_config:
insecure_skip_verify: false
ssh_banner:
prober: tcp
tcp:
query_response:
- expect: "^SSH-2.0-"
irc_banner:
prober: tcp
tcp:
query_response:
- send: "NICK prober"
- send: "USER prober prober prober :prober"
- expect: "PING :([^ ]+)"
send: "PONG ${1}"
- expect: "^:[^ ]+ 001"
icmp:
prober: icmp
dns_test:
prober: dns
timeout: 5s
dns:
query_name: google.com
preferred_ip_protocol: ip4
And my prometheus.yml 3389 port probe entry is:
- job_name: "rdp-dev-status"
metrics_path: /probe
params:
module: [dns_test]
static_configs:
- targets:
- nostradata-dvmh-prodweb-01
# file_sd_configs:
# - files:
# - /opt/prometheus/tools/targets/rdp-dev-targets.yml
relabel_configs:
# Ensure port is 22, pass as URL parameter
- source_labels: [__address__]
regex: (.*)(:.*)?
replacement: ${1}:3389
target_label: __param_target
# Make instance label the target
- source_labels: [__param_target]
target_label: instance
# Actually talk to the blackbox exporter though
- target_label: __address__
replacement: PROD-NIFI:9115
module: [dns_test]
Using a DNS probe is probably not going to work with RDP. Try the tcp_connect module.
I want to unconditionally forward all requests to a backend, but when I try to visit it, I keep getting 404. Here are my configs.
docker-compose.yml
version: '3.3'
services:
load-balancer:
image: traefik:1.5-alpine
ports:
- 8080:80
volumes:
- ./traefik:/etc/traefik
server1:
build: server1
ports:
- 8081:80
traefik.toml
defaultEntryPoints = ["http"]
debug = true
logLevel = "DEBUG"
[accessLog]
[entryPoints.http]
address = ":80"
[frontends.dummy]
backend = "dummy"
entrypoints = ["http"]
[frontends.dummy.routes.all]
rule = "PathPrefix:/"
[backends.dummy.servers.server1]
url = "http://server1:80"
I can access server1 with no problem from localhost:8081 and http://server1 from inside the load-balancer container. But localhost:8080 keeps returning 404
The debug log
load-balancer_1 | time="2017-12-15T12:56:09Z" level=info msg="Using TOML configuration file /etc/traefik/traefik.toml"
load-balancer_1 | time="2017-12-15T12:56:09Z" level=info msg="Traefik version v1.5.0-rc2 built on 2017-12-06_03:07:42PM"
load-balancer_1 | time="2017-12-15T12:56:09Z" level=info msg="
load-balancer_1 | Stats collection is disabled.
load-balancer_1 | Help us improve Traefik by turning this feature on :)
load-balancer_1 | More details on: https://docs.traefik.io/basic/#collected-data
load-balancer_1 | "
load-balancer_1 | time="2017-12-15T12:56:09Z" level=debug msg="Global configuration loaded {"LifeCycle":{"RequestAcceptGraceTimeout":0,"GraceTimeOut":0},"GraceTimeOut":0,"Debug":true,"CheckNewVersion":true,"SendAnonymousUsage":false,"AccessLogsFile":"","AccessLog":{"format":"common"},"TraefikLogsFile":"","TraefikLog":null,"LogLevel":"DEBUG","EntryPoints":{"http":{"Network":"","Address":":80","TLS":null,"Redirect":null,"Auth":null,"WhitelistSourceRange":null,"Compress":false,"ProxyProtocol":null,"ForwardedHeaders":{"Insecure":true,"TrustedIPs":null}}},"Cluster":null,"Constraints":[],"ACME":null,"DefaultEntryPoints":["http"],"ProvidersThrottleDuration":2000000000,"MaxIdleConnsPerHost":200,"IdleTimeout":0,"InsecureSkipVerify":false,"RootCAs":null,"Retry":null,"HealthCheck":{"Interval":30000000000},"RespondingTimeouts":null,"ForwardingTimeouts":null,"Web":null,"Docker":null,"File":null,"Marathon":null,"Consul":null,"ConsulCatalog":null,"Etcd":null,"Zookeeper":null,"Boltdb":null,"Kubernetes":null,"Mesos":null,"Eureka":null,"ECS":null,"Rancher":null,"DynamoDB":null,"ServiceFabric":null,"Rest":null,"API":null,"Metrics":null,"Ping":null}"
load-balancer_1 | time="2017-12-15T12:56:09Z" level=info msg="Preparing server http &{Network: Address::80 TLS:<nil> Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc42060ea60} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
load-balancer_1 | time="2017-12-15T12:56:09Z" level=info msg="Starting server on :80"
load-balancer_1 | 172.21.0.1 - - [15/Dec/2017:12:56:17 +0000] "GET / HTTP/1.1" - - - "curl/7.54.0" 1 - - 0ms
The last access log is from the command
curl localhost:8080
and it returned
404 page not found
I don't want to use the docker backend, just want to let traefik do the conventional round-robin from the container. I wonder if it's possible?
okay, after joining slack, Machael Matul pointed out you will need
[file] declared somewhere after entry points
[file]
and for backend to work, it has to be before routes declaration on frontend.
example:
[frontends]
[frontends.frontend1]
backend = "buffalo"
[frontends.frontend1.routes.web]
rule = "Host:xyz.com"
entrypoints = ["http","https"]
example of complete:
debug = true
logLevel = "DEBUG"
defaultEntryPoints = ["https","http"]
[accessLog]
[web]
address = ":8080"
[web.auth.basic]
users = ["admin:md5hashedpassword"]
[acme]
email = "info#xyz.com"
storage = "/acme.json"
entryPoint = "https"
OnHostRule = true
[acme.httpChallenge]
entryPoint = "http"
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[file]
watch = true
[backends]
[backends.buffalo]
[backends.buffalo.loadbalancer.stickiness]
[backends.buffalo.servers.server1]
url = "http://node1:3000"
[backends.buffalo.servers.server2]
url = "http://node2:3000"
[backends.backendNginx]
[backends.backendNginx.loadbalancer.stickiness]
[backends.backendNginx.servers.server1]
url = "http://nginx1:8080"
[backends.backendNginx.servers.server2]
url = "http://nginx2:8080"
[backends.ai6forum]
[backends.ai6forum.loadbalancer.stickiness]
[backends.ai6forum.servers.server1]
url = "http://10.140.0.9:80"
[frontends]
[frontends.frontend1]
backend = "buffalo"
[frontends.frontend1.routes.web]
rule = "Host:xyz.com"
entrypoints = ["http","https"]
passHostHeader = true
passTLSCert = true
priority = 10
[frontends.nginx]
backend = "backendNginx"
[frontends.nginx.routes.web]
i have setup varnish 4 to run on port 8081 while apache is configured to run on port 80.
The problem with my setup is that when i browse my domain
http//:mydomain.com:8180/.
i get a (301) permanent redirect to http//:mydomain.com/.
Due to this redirect am unable to get the difference between calling the cached domain http//:mydomain.com:8180/ visa v the uncached domain http//:mydomain.com/.
my varnish config
DAEMON_OPTS="-a :8180\
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,1G"
.......
also the vcl
backend mydomain {
.host = "x.x.x.x";
.port = "80";
.connect_timeout = 60s;
.first_byte_timeout = 60s;
.between_bytes_timeout = 60s;
.max_connections = 800;
}
.......
the responce header shows that apache is the one redirecting.
HTTP/1.1 301 Moved Permanently
Date: Fri, 04 Sep 2015 11:58:04 GMT
Server: Apache
X-Powered-By: PHP/5.3.3
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
X-Pingback: http//:mydomain.com/xmlrpc.php
Location: http//:mydomain.com/
Vary: Accept-Encoding
Content-Length: 0
Content-Type: text/html; charset=UTF-8
X-Varnish: 32795
Age: 0
Via: 1.1 varnish-v4
Connection: keep-alive
my question is How do i stop the redirect?
fixed this by adding
set req.http.host = "http//:mydomain.com";
in vlc as shown below.
if (req.http.host ~ "mydomain.com:8180") {
set req.http.host = "mydomain.com";
set req.backend_hint = mydomain;
}
by doing this we ensure that the request host is recognized by apache hence apache will not redirect