How to use Envoy as a Reverse Proxy for a specific URL on localhost - reverse-proxy

I want to use Envoy as a reverse proxy in which I want to redirect the request from
http://example.com:3443/node-exporter/metrics
to
http://localhost:9100/metrics
I want to redirect to a specific URL /metrics on the port 9100.
This is my current envoy_conf.yaml file
listeners:
- name: prom_listener
address:
socket_address : {address: 0.0.0.0, port_value: 3443}
filter_chains:
- name: prom_filter_chain
filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: http_connection_manager
route_config:
virtual_hosts:
- name: prom_local_host
domains: ["*"]
routes:
- name: node-exporter-route
match: {prefix: "/node-exporter/"}
route:
cluster: node-exporter-cluster-server
timeout: 0s
idle_timeout: 0s
http_filters:
- name: envoy.filters.http.router
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: node-exporter-cluster-server
type: static
connect_timeout: 2s
load_assignment:
cluster_name: node-exporter-cluster-server
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 9100
What changes (additions/deletions) should I make in this in order to achieve the reverse proxying to a specific url on the localhost and port mentioned?

With your configuration, when you are requesting example.com:3443/node-exporter/metrics, you are actually trying to access 127.0.0.1:9100/node-exporter/metrics, which does not exist.
To access 127.0.0.1:9100/metrics (notice the missing /node-exporter part of the URL), you only have to configure your route to tell Envoy to rewrite the prefix. You should use the prefix_rewrite option to strip the /node-exporter part:
routes:
- name: node-exporter-route
match: {prefix: "/node-exporter/"}
route:
cluster: node-exporter-cluster-server
prefix_rewrite: "/"
timeout: 0s
idle_timeout: 0s

Related

Envoy proxy returning 'no healthy upstream' and/or ERR_EMPTY_RESPONSE for ASP.NET application

I'm trying to setup an example ASP.NET project that uses envoy to route between the secure and non-secure versions of the app. The application works fine in Docker on both http and https, but when I try to route to it through envoy I get 'no healthy upstream' on the http site, and ERR_EMPTY_RESPONSE on the https site.
My envoy.yaml:
static_resources:
listeners:
- name: listener_http
address:
socket_address:
address: 0.0.0.0
port_value: 80
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
http_filters:
- name: envoy.filters.http.router
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/admin"
direct_response:
status: 403
body:
inline_string: "Forbidden, yo"
- match:
prefix: "/"
route:
cluster: exampleagg-http
- name: listener_https
address:
socket_address:
address: 0.0.0.0
port_value: 443
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
http_filters:
- name: envoy.filters.http.router
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/admin"
direct_response:
status: 403
body:
inline_string: "Forbidden, yo"
- match:
prefix: "/"
route:
cluster: exampleagg-https
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: /etc/https/aspnetapp.crt
private_key:
filename: /etc/https/aspnetapp.key
clusters:
- name: exampleagg-http
type: LOGICAL_DNS
# Comment out the following line to test on v6 networks
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: exampleagg-http
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: http://example-api/WeatherForecast
port_value: 80
- name: exampleagg-https
type: LOGICAL_DNS
# Comment out the following line to test on v6 networks
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: exampleagg-https
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: https://example-api/WeatherForecast
port_value: 443
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: /etc/https/aspnetapp.crt
private_key:
filename: /etc/https/aspnetapp.key
My docker-compose.yaml:
networks:
envoy:
name: envoy
services:
api-gateway:
image: envoyproxy/envoy:v1.23-latest
container_name: api-gateway
volumes:
- ./ApiGateways/Envoy/config:/etc/envoy
- ${USERPROFILE}/.aspnet/https:/etc/https/
networks:
- envoy
ports:
- "8080:80"
- "8081:443"
depends_on:
- example-api
example-api:
image: ${REGISTRY:-hexsorcerer}/example-proxy-envoy:${PLATFORM:-linux}-${TAG:-latest}
container_name: example-api
volumes:
- ${USERPROFILE}/.aspnet/https:/https/
environment:
ASPNETCORE_ENVIRONMENT: Development
ASPNETCORE_URLS: "https://+;http://+"
ASPNETCORE_HTTPS_PORT: 443
ASPNETCORE_Kestrel__Certificates__Default__Password: "password"
ASPNETCORE_Kestrel__Certificates__Default__Path: /https/aspnetapp.pfx
networks:
- envoy
expose:
- "80"
- "443"
ports:
- "5000:80"
- "5001:443"
build:
context: .
dockerfile: Services/Example/Example.API/Dockerfile
The Dockerfile for the example application:
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
# It's important to keep lines from here down to "COPY . ." identical in all Dockerfiles
# to take advantage of Docker's build cache, to speed up local container builds
COPY "ExampleEnvoyProxy.sln" "ExampleEnvoyProxy.sln"
COPY "Services/Example/Example.API/Example.API.csproj" "Services/Example/Example.API/Example.API.csproj"
#RUN dotnet restore "ExampleEnvoyProxy.sln"
COPY . .
WORKDIR /src/Services/Example/Example.API
RUN dotnet publish -c Release -o /app
EXPOSE 80 443
FROM build AS publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "Example.API.dll"]
I've been hacking away at this for days and made some good progress, but just can't quite get there. Any help would be greatly appreciated.
After not getting any responses for a couple weeks, I was forced to keep hacking around on this and I finally figured it out.
I think the biggest problem with what I was doing was a misunderstanding about how proxy servers route traffic. I was attempting to route the incoming path '/' to a backed service path of '/WeatherForecast', but that's not how it works. Your incoming endpoint will be "passed along" to the backend service, it's simply the cluster that you choose to pass it to. This is probably clear to veterans but as someone who doesn't work on these often it wasn't obvious to me at first.
There was also an issue of certificates, and I ended up generating one for the app and one for envoy, each of which were different formats, that also had to be trusted on my machine. This took a little extra effort for the envoy cert, but it worked much better than trying to use a single cert for both.
I documented what I learned with some instructions and a fully working example here.

How to terminate TLS in Envoy proxy

I am scouting through a lot of Envoy documentation but have not found a satisfactory answer yet. Our requirement is simple to terminate the TLS connection at Envoy proxy and send the upstream connection (upstream means the backend traffic) over the HTTP/unencrypted channel.
My use case is really simple:
The clients want to talk to Envoy over HTTPS
Envoy terminates the TLS connection and connects to the backend using HTTP (Our backend pool exposes both HTTP and HTTPS ports but we specifically want to connect to HTTP port)
We are using Dynamic Forward Proxy and a few basic envoy HTTP filters which do the host rewriting, there is no other fancy logic in Envoy
We would need something like this but I don't see it out of the box anywhere - https://github.com/envoyproxy/envoy/pull/14634
Current envoy.config
admin:
access_log_path: "/etc/logs/envoy/envoy.log"
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 20000
static_resources:
listeners:
- name: host_manipulation
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 443
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: gateway
domains:
- "*"
require_tls: EXTERNAL_ONLY
routes:
- match:
prefix: "/"
route:
cluster: dynamic_forward_proxy_cluster
host_rewrite_path_regex:
pattern:
google_re2: { }
regex: "^/(.+)/(.+)/.+$"
substitution: \2-\1.mesh
http_filters:
- name: envoy.filters.http.dynamic_forward_proxy
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.dynamic_forward_proxy.v3.FilterConfig
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: V4_ONLY
- name: envoy.filters.http.router
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: "/ca/tls.crt"
private_key:
filename: "/ca/tls.key"
clusters:
- name: dynamic_forward_proxy_cluster
connect_timeout: 1s
lb_policy: CLUSTER_PROVIDED
cluster_type:
name: envoy.clusters.dynamic_forward_proxy
typed_config:
"#type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: V4_ONLY
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
validation_context:
trust_chain_verification: ACCEPT_UNTRUSTED

omnidb behind envoy proxy

I'm trying to setup an omnidb server behind a envoy proxy
It was working fine with Nginx but I had to change to envoy for some reason...
I'm using omnidb v2.17
The issue is with the websocket omnidb is using. I can connect fine to omnidb, I can loggin but when I run SQL query, I get the following error:
cannot connect to websocket server with ports 443 (external) and 26000 (internal)
When I inspect in the browser I see the following error in the console:
WebSocket connection to 'wss://my-domain.com/wss' failed: Error during WebSocket handshake: Unexpected response code: 404
After few second I have this error in the console:
WebSocket connection to 'wss://my-domain.com:26000/wss' failed: Error in connection establishment: net::ERR_CONNECTION_TIMED_OUT
EDIT: In envoy log I have this:
[2021-02-16T18:52:19.016Z] "GET /wss HTTP/1.1" 404 - 0 77 63 - "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36" "0d9be0f1-9517-43e0-8a66-355804dd23c7" "my-domain.com" "10.0.0.1:8080"
So it seems it try to forward to "10.0.0.1:8080" instead of port 26000. Is it that the prefix "/" match before "/wss" so everything goes to port 8080 ?
Here is my envoy.yaml file:
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 443
filter_chains:
- filter_chain_match:
server_names:
- my-domain.com
filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http_and_wss
upgrade_configs:
- upgrade_type: websocket
access_log:
- name: envoy.access_loggers.file
typed_config:
"#type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
path: /dev/stdout
http_filters:
- name: envoy.filters.http.router
route_config:
name: omnidb
virtual_hosts:
- name: local_service
domains:
- "*"
routes:
- match:
prefix: "/wss/"
route:
prefix_rewrite: "/"
cluster: omnidb_ws
- match:
prefix: "/ws/"
route:
prefix_rewrite: "/"
cluster: omnidb_ws
- match:
prefix: "/"
route:
cluster: omnidb
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
certificate_chain:
filename: /etc/letsencrypt/live/my-domain.com/cert.pem
private_key:
filename: /etc/letsencrypt/live/my-domain.com/privkey.pem
clusters:
- name: omnidb
connect_timeout: 30s
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: omnidb
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.1
port_value: 8080
- name: omnidb_ws
connect_timeout: 0.25s
dns_lookup_family: V4_ONLY
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
load_assignment:
cluster_name: omnidb_ws
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.1
port_value: 26000
PS: I can't create the tag omnidb so I put SQL instead, would be nice to have a tag omnidb
After the edit above to get the / route as the last, otherwise it will match everything, now you need to fix how you send your request or how you treat the two routes in regard to trailing slashes.
The main point here:
You have route matches for /wss/ and /ws/, both with trailing slashes
You send a request with /wss with NO trailing slash.
This request matches neither of the two first routes, so it gets to the / route again.
You can send your request with /wss/ (note the trailing slash) or you can add/modify your routes. This can be done a number of ways, the simplest would probably be to just match on /wss and /ws. though if the trailing slash is important to the end application (which it can be in UIs), you can have /wss redirect to /wss/
I tested this just slight modifications to your config. Ignore the changes in the filter chains, the only thing that matters is the routes.
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 8443
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http_and_wss
access_log:
- name: envoy.access_loggers.file
typed_config:
"#type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
path: /dev/stdout
http_filters:
- name: envoy.filters.http.router
route_config:
name: omnidb
virtual_hosts:
- name: local_service
domains:
- "*"
routes:
- match:
prefix: "/wss"
route:
prefix_rewrite: "/"
cluster: omnidb_ws
- match:
prefix: "/ws"
route:
prefix_rewrite: "/"
cluster: omnidb_ws
- match:
prefix: "/"
route:
cluster: omnidb
clusters:
- name: omnidb
connect_timeout: 30s
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: omnidb
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.1
port_value: 8080
- name: omnidb_ws
connect_timeout: 0.25s
dns_lookup_family: V4_ONLY
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
load_assignment:
cluster_name: omnidb_ws
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.1
port_value: 26000
Then curl localhost:8443/wss or curl localhost:8443/wss/ show they get to your 26000 address.
[2021-02-22T15:57:22.818Z] "GET /wss/ HTTP/1.1" 503 UF 0 91 3 - "-" "curl/7.68.0" "806c2c28-4ab4-4069-acf1-15b75405d390" "localhost:8443" "10.0.0.1:26000"
[2021-02-22T15:57:27.287Z] "GET /wss HTTP/1.1" 503 UF 0 91 3 - "-" "curl/7.68.0" "55d32a64-c9f7-46cc-8f5e-4a024c0de00d" "localhost:8443" "10.0.0.1:26000"

Envoy Proxy (v3 api) with http and https both

We are running envoy server v1.15 on vm which serve the traffic for http and https both.
We have two listener one for http and one for https.
1.We are able to get all the route for application and http://api.example.com/stats/prometheus for envoy working on non-tls port for http listener.
2.For https listener we have provided the path of letsencrypt self-signed certificated and configured according the envoy documentation
Please find the config for same..
admin:
access_log_path: "/tmp/admin_access.log"
address:
socket_address:
address: 127.0.0.1
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 80
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
http_protocol_options:
accept_http_10: true
route_config:
name: local_route
virtual_hosts:
- name: local_envoy_admin_service
domains:
- "*"
routes:
- match:
path: "/stats/prometheus"
route:
cluster: envoy_admin_service
- match:
prefix: "/"
route:
cluster: local_service
timeout: 15s
http_filters:
- name: envoy.filters.http.router
- name: listener_https
address:
socket_address:
address: 0.0.0.0
port_value: 443
listener_filters:
- name: envoy.filters.listener.tls_inspector
typed_config: {}
filter_chains:
- filter_chain_match:
server_names:
- api.example.com
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: "/etc/letsencrypt/live/api.example.com/fullchain.pem"
private_key:
filename: "/etc/letsencrypt/live/api.example.com/privkey.pem"
filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_https
use_remote_address: true
http2_protocol_options:
max_concurrent_streams: 100
access_log:
- name: envoy.access_loggers.file
typed_config:
"#type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
path: "/var/log/envoy/access.log"
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains:
- api.example.com
routes:
- match:
path: "/stats/prometheus"
route:
cluster: envoy_admin_service
- match:
path: "/"
route:
cluster: local_service
http_filters:
- name: some.customer.filter
- name: envoy.filters.http.router
clusters:
- name: envoy_admin_service
connect_timeout: 0.25s
type: STATIC
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: envoy_admin
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 9901
- name: local_service
connect_timeout: 15s
type: STATIC
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 8081
If https request comes, it should do the downstream TLS termination and forward non-tls request to desired upstream cluster.
Please help us to figure it out what we are missing for HTTPS configuration

How to use Traefik and Envoy in the same project to handle grpc-web?

I've got a service up and running on traefik with LetsEncrypt at grpc.mydomain.com. However, traefik doesn't support routing grpc-web request due to some issue with CORS (https://github.com/containous/traefik/issues/4210). Envoy appears to be an alternative to traefik which works with grpc-web, but I don't want to go about reconfiguring everything.
If I put envoy at envoy.mydomain.com then it actually hits traefik first and traefik can't route the grpc-web requests to envoy. So this doesn't work.
If I put envoy outside of traefik (mydomain.com:9091) then envoy doesn't have the TLS support that traefik has.
Do I need to switch everything to envoy? Is there an alternative I haven't considered? Any guidance welcome :)
Current Traefik Setup:
traefik:
image: traefik:v2.0.0
container_name: traefik
command:
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --entrypoints.grpc.address=:8090
- --providers.docker
- --api
- --serversTransport.rootCAs=/certs/grpc.cert
# Lets Encrypt Resolvers
- --certificatesresolvers.leresolver.acme.email=${EMAIL}
- --certificatesresolvers.leresolver.acme.storage=/etc/acme/cert.json
- --certificatesresolvers.leresolver.acme.tlschallenge=${TLS_CHALLENGE}
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /etc/acme/:/etc/acme/
- ./secrets/grpc.cert:/certs/grpc.cert
# Dynamic Configuration
labels:
# Dashboard
- "traefik.http.routers.traefik.rule=Host(`traefik.${DOMAIN}`)"
- "traefik.http.routers.traefik.service=api#internal"
- "traefik.http.routers.traefik.tls.certresolver=leresolver"
- "traefik.http.routers.traefik.entrypoints=websecure"
- "traefik.http.routers.traefik.middlewares=authtraefik"
# https://docs.traefik.io/middlewares/basicauth/
# password generated from `echo $(htpasswd -nb admin $PASSWORD) | sed -e s/\\$/\\$\\$/g`
- "traefik.http.middlewares.authtraefik.basicauth.users=admin:$$apr1$$6VzI3S0N$$29FC82dYEbjFN9tPSfWLX1"
# global redirect to https
- "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
- "traefik.http.routers.http-catchall.entrypoints=web"
- "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
# middleware redirect
- "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
ports:
- 80:80
- 443:443
- 8090:8090
networks:
- internal
- proxied
grpc_server:
image: ${GRPC_IMAGE}
container_name: grpc_server
volumes:
- /tmp/keyset.json:/tmp/keyset.json
- ./secrets/:/secrets/
working_dir: /app/__main__/
labels:
- "traefik.http.routers.combined_server.rule=Host(`grpc.${DOMAIN}`)"
- "traefik.http.routers.combined_server.entrypoints=grpc"
- "traefik.http.routers.combined_server.tls=true"
- "traefik.http.routers.combined_server.tls.certresolver=leresolver"
# http
- "traefik.http.services.grpc-svc.loadbalancer.server.scheme=h2c"
- "traefik.http.services.grpc-svc.loadbalancer.server.port=8090"
expose:
- 8090
networks:
- internal
- proxied
I also tried setting these to fix the CORS error but got nowhere.
- "traefik.http.middlewares.testheader.headers.accesscontrolallowmethods=GET,PUT,DELETE,POST,OPTIONS"
- "traefik.http.middlewares.testheader.headers.accesscontrolallowheaders=keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout"
- "traefik.http.middlewares.testheader.headers.accesscontrolmaxage=100"
- "traefik.http.middlewares.testheader.headers.addvaryheader=true"
- "traefik.http.middlewares.testheader.headers.alloworigin=*"
A way to approach this issue is by using 2 different URLs that are both being handled by traefik at first. One URL is being used for "direct grpc" (grpc.mydomain.com), the other one for grpc-web (let's call it grpc-web.mydomain.com). Traefik does TLS termination for both.
The grpc.mydomain.com traffic is directly passed to the container running the grpc_server. The grpc-web.mydomain.com traffic is passed to envoy which acts as a grpc-web-proxy and then passes the traffic to the grpc_server.
So as you are using docker-compose, you would need to add an envoy service to your docker-compose.yml:
---
version: '3'
services:
traefik:
# traefik configuration from your question
# ...
grpc-server:
# grpc_server configuration from your question
# ...
envoy:
image: envoyproxy/envoy:v1.14.1
restart: unless-stopped
volumes:
- ./envoy.yaml:/etc/envoy/envoy.yaml
labels:
- traefik.enable=true
- traefik.http.routers.envoy.rule=Host(`grpc-web.mydomain.com`)
- traefik.http.services.envoy.loadbalancer.server.port=8080
- traefik.http.routers.envoy.tls=true
- traefik.http.routers.envoy.tls.certresolver=leresolver
The envoy.yaml configuration (mounted in the volumes section above) looks like this:
admin:
access_log_path: /tmp/admin_access.log
address:
socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 8080 }
filter_chains:
- filters:
- name: envoy.http_connection_manager
config:
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match: { prefix: "/" }
route:
cluster: grpc_service
max_grpc_timeout: 0s
cors:
allow_origin_string_match:
- prefix: "*"
allow_methods: GET, PUT, DELETE, POST, OPTIONS
allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
max_age: "1728000"
expose_headers: custom-header-1,grpc-status,grpc-message
http_filters:
- name: envoy.grpc_web
- name: envoy.cors
- name: envoy.router
clusters:
- name: grpc_service
connect_timeout: 0.25s
type: logical_dns
http2_protocol_options: {}
lb_policy: round_robin
hosts: [{ socket_address: { address: grpc-server, port_value: 8090 }}]
This is a pretty basic grpc-web config for envoy. The important part to notice is that we set address: grpc-server, port_value: 8090 in the configuration of the "grpc_service" cluster configuration to the service name from the docker-compose.yml and to the port your grpc-server is listening on. Please note I renamed your service from grpc_server to grpc-server as the underscore is not a valid charater in hostnames.
On the client side, use:
"grpc-web.mydomain.com" in your javascript (grpc-web) code.
"grpc.mydomain.com" when writing a client in another language (like golang for example).
I created a working example, which can be found under: https://github.com/rbicker/greeter
If you want to get rid of deprecated warnings in envoy, you can update envoy.yaml from this answer with those three changes:
replace:
- name: envoy.http_connection_manager
config:
with:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
replace
- name: envoy.grpc_web
- name: envoy.cors
- name: envoy.router
with
- name: envoy.filters.http.grpc_web
- name: envoy.filters.http.cors
- name: envoy.filters.http.router
replace
hosts: [{ socket_address: { address: grpc-server, port_value: 8090 }}]
with
load_assignment:
cluster_name: cluster_0
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: grpc-server
port_value: 8090