I want to use Envoy as an L7 Load balancer implementation running alongside kubernetes, but I cannot give it admin access, as that is a security problem I have to worry about. Some research tells me that Envoy needs admin access to modify some IP tables, but do the other containers in the pod also need admin access? If so, why? And is that something for which a workaround exists?
Thank you
According to the instructions, you should create iptables rules with Envoy user UID (like in Istio).
sudo iptables -t nat -A PREROUTING -p tcp -j REDIRECT --to-port 5001
sudo iptables -t nat -A OUTPUT -p tcp -j REDIRECT --to-port 5001 -m owner '!' --uid-owner ${Envoy_User_UID}
In order to run this rule, a new user with ${Envoy_User_UID} needs to be created before launching Envoy. Envoy then needs to be run under this user id, so that packets from Envoy don't get redirected to themselves.
On the other hand, you can use Ambassador - it is an open source distribution of Envoy designed for Kubernetes.
How to deploy and configure Ambassador you can find in Envoy as an API Gateway in Kubernetes instruction.
You can find more details on Setup IP tables rules to redirect inbound/outbound traffic to Envoy sidecar page.
Also, Use Envoy as tcp/http proxy for all outbound traffic could be useful.
Related
I have a VM running OpenVPN with client-to-client disabled and I need some specific forwarding rules. IP forwarding on the VM is turned on.
The OpenVPN base network is 172.30.0.0/16 and that is further subdivided into /24 subnets with their own rules.
172.30.0.0/24 should have access to all the clients. The rest should not. I have 2 subnets defined at the moment; 172.30.0.0/24 and 172.30.10.0/24.
Following the suggestion at the bottom here;
https://openvpn.net/community-resources/configuring-client-specific-rules-and-access-policies/ - I set up my rules as follows;
iptables -A FORWARD -i tun1 -s 172.30.0.0/24 -j ACCEPT
iptables -P FORWARD DROP
This does not work. If I add a -j LOG at the top, I can see that traffic from my client at 172.30.0.1 connects fine to the client at 172.30.10.3, but all traffic it sends BACK is blocked.
If I set the policy to ACCEPT everything works and I can connect to the client just fine, so this is not a routing problem.
How can I set this up? And why doesn't the suggestion in that OpenVPN guide work?
I solved this by adding
iptables -A FORWARD -i tun1 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
which allows established connections to return. Everything works as desired now.
I have a container setup with LXD running several wordpress webpages (apache2)
All is working fine.
I added portforwarding by:
lxc config device add CONTAINER lxd_proxy_port80 proxy listen=tcp:0.0.0.0:80 connect=tcp:INTERNALIP:80
…and same for port 443. Thats all working correctly.
Unfortunately I cannot see originating IP’s in my apache2 logs (var/log/apache2/access.log) but only see the local IP.
By using iptables I wanted to change this. I did:
iptables -A FORWARD -p tcp -d LOCALIP --dport 443 -j ACCEPT
iptables -A FORWARD -p tcp -d LOCALIP --dport 80 -j ACCEPT
and deleting my proxy device with
lxc config device remove CONTAINER lxd_proxy_port80
lxc config device remove CONTAINER lxd_proxy_port443
I can actually access files on my server correctly and I also see now my external IP in the apache2 access logs.
However, wordpress does not reach update-servers anymore (external) and seems to have problems reaching the outside world and one of my wordpress pages cannot access the index.php file anymore (it hangs loading). I suppose the latter effect is due to some external content not being loaded correctly.
Could you help me understand what is going on?
This conversation answers the questions:
https://discuss.linuxcontainers.org/t/iptables-apache-in-lxd-container/6143
A good video on this:
https://www.youtube.com/watch?v=1p-fbS_OYTg
My solution did end up working by adding a -d MYIP/32 in the iptables rule to only make it apply for incoming traffic.
I'm trying to setup a haproxy gateway between server and client for full transparent proxy like below diagram. My main aim is to provide load balancing.
There is a simple application that listens port 25 in the server side. The client tries to connect port 25 on the gateway machine, and haproxy on the gateway chooses an avaliable server then redirects the connection to the server.
Network analysis of this approach produces tcp flow like diagram: The client resets the connection at the end since it doesn't send a syn packet to the server.
Is this haproxy usage true and my problem related configuation? Or should the client connect to the server directly (This doesn't make much sense to me but I'm not sure actually. If this is true then how haproxy will intervene the connection and make load balancing)?
EDIT:
I've started to think this problem is related to routing and NAT on the gateway. All of these three machines are in same subnetwork but I've added routes to the gateway for both client and server. Also rules on the gateway are:
iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 0x01/0x01
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A PREROUTING -p tcp --dport 25 -j TPROXY \
--tproxy-mark 0x1/0x1 --on-port 10025
ip route flush table 100
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
Now the question is what should I do in the gateway to change "syn-ack (src: S, dst: C)" to "syn-ack (src: GW, dst: C)"?
Here is the definition of my situation.
Here comes the transparent proxy mode: HAProxy can be configured to spoof the client IP address when establishing the TCP connection to the server. That way, the server thinks the connection comes from the client directly (of course, the server must answer back to HAProxy and not to the client, otherwise it can’t work: the client will get an acknowledge from the server IP while it has established the connection on HAProxy‘s IP).
And the answer is to set ip_nonlocal_bind system control.
I want to make sure that the only network traffic on my linux CentOS server is my own.
All my server runs is a Tomcat instance with one servlet. This servlet takes a parameter, which is a URL to download. It will download from that url, and pass the contents back to the calling program through the usual http response.
I want to block all network traffic to this server except
1) Ability to ssh
2) Ability to download from host abc.xyz.com
3) Ability for server with IP 111.222.333.444 to download from me
Can someone please tell me the commands to do this in iptables? I tried finding this out myself but I was a bit out my depth with the lingo.
thanks
Configuring a firewall is simple, first of all select what ports you want to be open.
For example Webserver ports:
iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
For example SSH port:
iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
In any way your server is able to download files from other server/hosts.
3) Ability for server with IP 111.222.333.444 to download from me
I suppose that must be port 80, (or any port where the server is downloading from) if your uploading files to your website.
After these steps you need to look if the firewall is configured right:
iptables -L -n
If it's looking good then you're able to save your iptables, and restart the service.
If there is any problem configureren your firewall, please let me know.
Here is the situation.
We have multiple server on our intranet 192.168.1.0/24
One of them is the default gateway for all of them and have two interfaces ($GATEWAY_INTERNAL_IP and $GATEWAY_EXTERNAL_IP).
We have also another server PUBLICHOST2 which has two IP as well $PUBLICHOST_EXTERNAL_IP and $PUBLICHOST_INTERNAL_IP.
We have a third server SERVER which have only one IP $PRIVIP and bind on port $PORT.
What we want is to be able to forward port $PORT on $PUBLICHOST_EXTERNAL_IP to host SERVER on $PRIVIP.
But when we do the port forwarding using iptables on PUBLICHOST2, SERVER receive the request but the response goes through the gateway and the connection is not successfull.
How can we properly do the setup so that the response can go back through PUBLICHOST2 ?
Thanks
You may need to set forwarding on for the interface. Try tne command.
sysctl -w net.ipv4.conf.eth0.forwarding=1
If you need additional help look for documentation on routeback or the Shorewall FAQ.
Well here what happens:
Client1 sends a request to PublicHost
The requests arrives and the iptables rules redirects the traffic (PAT) to the Server on the correct AppPort
Server sends back a reply to Client1 which will be routed by Gateway
Gateway is doing NAT and replaces the source IP with it's own
Client1 or Client1sGateway receives the IP packet with Gateway as the source but it expected PublicHost's IP in the source field of the IP packet.
Eventually Client1 resends the SYN/ACK (except if you're using a synproxy) to PublicHost and then drops the connection when whatever network related timer expires.
Now if you want to fix this, you should route all TCP traffic going OUT of Server and with a source port of AppPort to PublicHost.
If this doesn't work, PublicHost is not properly configured. Be sure to test the configuration with tcpdump.
I've been trying to do something similar. After running through a bunch of tutorials that never seemed to work until I Wiresharked the connection to discover that the destination address was still set to the external IP address, (exactly like you've described), I tried using the POSTROUTING chain to change the source IP address to that of the server:
iptables -t nat -A POSTROUTING -p <tcp/udp> --dport <destination_port> -j SNAT --to <$PUBLICHOST_INTERNAL_IP>
After I added that rule, the connection was forwarded into the private network and the response packets retraced the same path back to the client, rather than through the network gateway. I'm not positive what allowed the response packets back out through the firewall server, but I think it was because of the rule I already had on the INPUT chain to allow established connections:
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
The thing to be sure to keep in mind with this solution is: if you ever change the firewall server's internal IP address, then you will need to update the above POSTROUTING rule. (Needless to say, it's probably best if the firewall server has a statically assigned internal IP address).