I have been trying for some time to configure my Proxmox with a PFSense VM filtering internet traffic to my other VMs.
So far I have managed to install PFSense and configure the Proxmox interfaces. I have also managed to go to the PFSense web interface. However, my VMs do not always have access to the internet, so I try to modify my iptables to manage to redirect all the traffic on PFSense.
Here are my interfaces:
interfaces
On the shell I did this operation :
cat > /root/pfsense-route.sh << EOF
#!/bin/sh
## IP forwarding activation echo 1 > /proc/sys/net/ipv4/ip_forward
## Rediriger les paquets destinés au LAN pour l'interface WAN de la PFSense ip route change 192.168.9.0/24 via 10.0.0.2 dev vmbr1 EOF
And I modified the file /etc/hosts :
[...]
auto vmbr2
iface vmbr2 inet static
address 192.168.9.1/24
bridge-ports none
bridge-stp off
bridge-fd 0
post-up /root/pfsense-route.sh
#LAN
And now the heart of the problem, the iptables. Here is my current file :
#!/bin/sh
# ---------
# VARIABLES
# ---------
## Proxmox bridge holding Public IP
PrxPubVBR="vmbr0"
## Proxmox bridge on VmWanNET (PFSense WAN side)
PrxVmWanVBR="vmbr1"
## Proxmox bridge on PrivNET (PFSense LAN side)
PrxVmPrivVBR="vmbr2"
## Network/Mask of VmWanNET
VmWanNET="10.0.0.0/30"
## Network/Mmask of PrivNET
PrivNET="192.168.9.0/24"
## Network/Mmask of VpnNET
VpnNET="10.2.2.0/24"
## Public IP => Your own public IP address
PublicIP="91.121.134.145"
## Proxmox IP on the same network than PFSense WAN (VmWanNET)
ProxVmWanIP="10.0.0.1"
## Proxmox IP on the same network than VMs
ProxVmPrivIP="192.168.9.1"
## PFSense IP used by the firewall (inside VM)
PfsVmWanIP="10.0.0.2"
# ---------------------
# CLEAN ALL & DROP IPV6
# ---------------------
### Delete all existing rules.
iptables -F
iptables -t nat -F
iptables -t mangle -F
iptables -X
### This policy does not handle IPv6 traffic except to drop it.
ip6tables -P INPUT DROP
ip6tables -P OUTPUT DROP
ip6tables -P FORWARD DROP
# --------------
# DEFAULT POLICY
# --------------
### Block ALL !
iptables -P OUTPUT DROP
iptables -P INPUT DROP
iptables -P FORWARD DROP
# ------
# CHAINS
# ------
### Creating chains
iptables -N TCP
iptables -N UDP
# UDP = ACCEPT / SEND TO THIS CHAIN
iptables -A INPUT -p udp -m conntrack --ctstate NEW -j UDP
# TCP = ACCEPT / SEND TO THIS CHAIN
iptables -A INPUT -p tcp --syn -m conntrack --ctstate NEW -j TCP
# ------------
# GLOBAL RULES
# ------------
# Allow localhost
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Don't break the current/active connections
iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Allow Ping - Comment this to return timeout to ping request
iptables -A INPUT -p icmp --icmp-type 8 -m conntrack --ctstate NEW -j ACCEPT
# --------------------
# RULES FOR PrxPubVBR
# --------------------
### INPUT RULES
# ---------------
# Allow SSH server
iptables -A TCP -i \$PrxPubVBR -d \$PublicIP -p tcp --dport 22 -j ACCEPT
# Allow Proxmox WebUI
iptables -A TCP -i \$PrxPubVBR -d \$PublicIP -p tcp --dport 8006 -j ACCEPT
### OUTPUT RULES
# ---------------
# Allow ping out
iptables -A OUTPUT -p icmp -j ACCEPT
### Proxmox Host as CLIENT
# Allow HTTP/HTTPS
iptables -A OUTPUT -o \$PrxPubVBR -s \$PublicIP -p tcp --dport 80 -j ACCEPT
iptables -A OUTPUT -o \$PrxPubVBR -s \$PublicIP -p tcp --dport 443 -j ACCEPT
# Allow DNS
iptables -A OUTPUT -o \$PrxPubVBR -s \$PublicIP -p udp --dport 53 -j ACCEPT
### Proxmox Host as SERVER
# Allow SSH
iptables -A OUTPUT -o \$PrxPubVBR -s \$PublicIP -p tcp --sport 22 -j ACCEPT
# Allow PROXMOX WebUI
iptables -A OUTPUT -o \$PrxPubVBR -s \$PublicIP -p tcp --sport 8006 -j ACCEPT
### FORWARD RULES
# ----------------
### Redirect (NAT) traffic from internet
# All tcp to PFSense WAN except 22, 8006
iptables -A PREROUTING -t nat -i \$PrxPubVBR -p tcp --match multiport ! --dports 22,8006 -j DNAT --to \$PfsVmWanIP
# All udp to PFSense WAN
iptables -A PREROUTING -t nat -i \$PrxPubVBR -p udp -j DNAT --to \$PfsVmWanIP
# Allow request forwarding to PFSense WAN interface
iptables -A FORWARD -i \$PrxPubVBR -d \$PfsVmWanIP -o \$PrxVmWanVBR -p tcp -j ACCEPT
iptables -A FORWARD -i \$PrxPubVBR -d \$PfsVmWanIP -o \$PrxVmWanVBR -p udp -j ACCEPT
# Allow request forwarding from LAN
iptables -A FORWARD -i \$PrxVmWanVBR -s \$VmWanNET -j ACCEPT
### MASQUERADE MANDATORY
# Allow WAN network (PFSense) to use vmbr0 public adress to go out
iptables -t nat -A POSTROUTING -s \$VmWanNET -o \$PrxPubVBR -j MASQUERADE
# --------------------
# RULES FOR PrxVmWanVBR
# --------------------
### Allow being a client for the VMs
iptables -A OUTPUT -o \$PrxVmWanVBR -s \$ProxVmWanIP -p tcp -j ACCEPT
For now with this I still manage to go on my VMs in proxmox, but I’m not internet access on it. Moreover, the shell of my server is no longer accessible on proxmox and SSH connections are no longer accessible.
Some details:
I use port 22 as ssh port
My server ip is 91.121.134.145
My version of linux is Debian 10 (Buster)
Honestly I don’t know where the problem comes from, I’m a beginner and I find the majority of this configuration on the internet. If you see what is wrong I would be very happy to have the answer! In the meantime I thank you in advance for your reading and your answers!
Edit :
I tried to pass the iptables in legacy mode using these commands :
update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
update-alternatives --set ebtables /usr/sbin/ebtables-legacy
Only this command to refuse to work :
update-alternatives --set arptables /usr/sbin/arptables-legacy
Moreover I don’t know why but my VMs have good access to the internet, the problem is therefore centered on the SSH port that no longer works (I can no longer go on the shell since proxmox)
Without iptables rules I am able to mount my NFSSERVER:/PATH but with it(firewall/iptables) enabled I am not able to mount.
[.e.g., after iptables --flush/ firewaalld stop ; mount NFSSERVER:/Path works ]
I am not supposed to disable/clear the firewall/iptables but I am allowed to open a port. What is the rule that I need to add to open up the port/mount?
Current default policy is DROP all INCOMING/OUTGOING/FORWARD and there are couple of rules to allow wget from external 80 port etc.,
adding the NFS Server port didnt help.
iptables -A OUTPUT -p tcp --dport 2049 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --sport 2049 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p udp --dport 2049 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p udp --sport 2049 -m state --state ESTABLISHED -j ACCEPT
Thanks.
PS: This is for nfs client not NFS server machine.
If all you need is NFS version 4 (which is already over 10 years old), you don't need to go to all of the effort described in #Sathish's answer. Just make sure TCP port 2049 is open the server's firewall, and that the client's firewall allows outbound traffic to port 2049 on the server.
CentOS 5 (also old) has a nice explanation of why NFSv4 is more firewall friendly than v3 and v2.
NFS SERVER:
Configure Ports for rquotd(875/udp; 875/tcp), lockd(32803/tcp; 32769/udp), mountd(892/udp; 892/tcp), statd(10053/udp; 10053/tcp), statd_outgoing(10054/udp; 10054/tcp)
vim /etc/sysconfig/nfs
If desired, disable NFS v3 and NFS v2 suport by editing lines 5 & 6 of /etc/sysconfig/nfs
MOUNTD_NFS_V2="no"
MOUNTD_NFS_V3="no"
Save current Iptables rules for later use. (if iptables-save is absent in your distribution, you may try iptables -S filename )
iptables-save > pre-nfs-firewall-rules-server
Flush and check Iptables rules
iptables -F
iptables -L
Stop and Start NFS and related Services in the following sequence
service rpcbind stop
service nfslock stop
service nfs stop
service rpcbind start
service nfslock start
service nfs start
Make sure the configured NFS and its associated ports shows as set before and notedown the port numbers and the OSI layer 4 protcols. The standard port numbers for rpcbind (or portmapper) are 111/udp, 111/tcp and nfs are 2049/udp, 2049/tcp.
rpcinfo -p | sort -k 3
Restore the pre-nfs-firewall-rules now
iptables-restore < pre-nfs-firewall-rules-server
Write iptables rules for NFS server (Note: Loopback adapter has to allowed, else you will see packets dropped and also when you restart nfs service, it will spit ERROR {Starting NFS quotas: Cannot register service: RPC: Timed out rpc.rquotad: unable to register (RQUOTAPROG, RQUOTAVERS, udp). [FAILED]} for rquotad daemon. You can check this by adding a rule with LOG jump target at the bottom of INPUT or OUTPUT chains of filter table)
iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -A INPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -p udp -m multiport --dports 10053,111,2049,32769,875,892 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -p tcp -m multiport --dports 10053,111,2049,32803,875,892 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -p udp -m multiport --sports 10053,111,2049,32769,875,892 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -p tcp -m multiport --sports 10053,111,2049,32803,875,892 -m state --state ESTABLISHED -j ACCEPT
iptables -I INPUT -i lo -d 127.0.0.1 -j ACCEPT
iptables -I OUTPUT -o lo -s 127.0.0.1 -j ACCEPT
iptables -L -n --line-numbers
Configure NFS exports directory
vim /etc/exports
exportfs -av
showmount -e
rpcinfo -p
Stop and Start NFS and related Services in the following sequence
service rpcbind stop
service nfslock stop
service nfs stop
service rpcbind start
service nfslock start
service nfs start
NFS CLIENT:
Save current Iptables rules for later use. (if iptables-save is absent in your distribution, you may try iptables -S filename )
iptables-save > pre-nfs-firewall-rules-client
Flush and check Iptables rules
iptables -F
iptables -L
Obtain the firewalled NFS Server ports from the client machine and notedown the port numbers and the OSI layer 4 protcols.
rpcinfo -p 'ip-addr-nfs-server' | sort -k 3
Restore the pre-nfs-firewall-rules now
iptables-restore < pre-nfs-firewall-rules-client
Write iptables rules for NFS client (Note: Loopback adapter has to allowed, else you will see packets dropped and also when you restart nfs service, it will spit ERROR {Starting NFS quotas: Cannot register service: RPC: Timed out rpc.rquotad: unable to register (RQUOTAPROG, RQUOTAVERS, udp). [FAILED]} for rquotad daemon. You can check this by adding a rule with LOG jump target at the bottom of INPUT or OUTPUT chains of filter table)
iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -A INPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -p udp -m multiport --sports 10053,111,2049,32769,875,892 -m state --state ESTABLISHED -j ACCEPT
iptables -A INPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -p tcp -m multiport --sports 10053,111,2049,32803,875,892 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -p udp -m multiport --dports 10053,111,2049,32769,875,892 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -p tcp -m multiport --dports 10053,111,2049,32803,875,892 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -I INPUT -i lo -d 127.0.0.1 -j ACCEPT
iptables -I OUTPUT -o lo -s 127.0.0.1 -j ACCEPT
iptables -L -n --line-numbers
Stop and Start NFS and related Services in the following sequence
service rpcbind stop
service nfslock stop
service nfs stop
service rpcbind start
service nfslock start
service nfs start
List NFS Server exports
showmount -e 'ip-addr-nfs-server'
Mount NFS Exports manually (persistent mounts can be configured using /etc/fstab)
mount -t nfs ip-addr-nfs-server:/exported-directory /mount-point -o rw,nfsvers=3
mount -t nfs ip-addr-nfs-server:/exported-directory /mount-point -o rw --> For NFS4 version
Configure autofs, if automounting is preferred for nfs exports and with ldap user home directories (Direct and Indirect Maps can be set)
vim /etc/auto.master -> specify the mount point and map-name (Eg: auto.nfs)
vim /etc/map-name
service autofs stop
service autofs start
Check mounted NFS Exports
df -h -F nfs
mount | grep nfs
List all pseudo root NFS-V4 export directories (NFS Lazy mount)
ls /net/ip-addr-nfs-server
My centos server has an iptables rule.
iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 50 -j REJECT --reject-with tcp-reset
this code is doing the work like firewall but I don't want to block of my server ips.
my server ips:
"127.0.0.1", "my server ip1", "my server ip2", etc.
How do I get them out of this ip tables rule?
Thank you very much!
Just use :
# Loopback
iptables -I INPUT -s 127.0.0.1 -i lo -j ACCEPT
# Repeat for each SERVER_IP
iptables -I INPUT -s SERVER_IP -j ACCEPT
Note that this will open everything for SERVER_IPs. YMMV depending on want you want to allow.
For instance, if you just want to open HTTP port for those IPs :
# Loopback
iptables -I INPUT -s 127.0.0.1 -i lo -j ACCEPT
# Repeat for each SERVER_IP
iptables -I INPUT -s SERVER_IP -p tcp --dport 80 -j ACCEPT
Trying to understand iptables (I have cPanel installed on VPS) and having a little play so may sound like a silly question what I am doing.
I have copied the default iptables config to backup (in case goes wrong to restore) and created custom iptables config (/etc/sysconfig/iptables) were I DROP INPUT/OUTPUT/FORWARDING (so everything).
I then managed to get all the ports I want access to required working (incoming/outgoing HTTP/s/SSH/FTP etc) apart from emails (:25). I am using Roundcube and using the below config for emails but emails can not be sent/received on my server (works if I restore default config (ACCEPT everything) so apart from port :25 is there any other ports I need to allow access to for mail to be sent knowing everything has been dropped?). I am using below config for email in my custom (/etc/sysconfig/iptables):-
-A INPUT -p tcp --dport 25 -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --sport 25 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --dport 25 -j ACCEPT
-A INPUT -p tcp --sport 25 -m state --state ESTABLISHED -j ACCEPT
IP is a bidirectional communication, when you receive a mail, packets are sent on your server on port 25, and you will send response packet on a arbitrary allocated port number (determined during connection establishment).
So, common rule on iptables are :
Accepting packet on input from a specified port (25 for mail) whatever the state of connection
-A INPUT -p tcp --dport 25 -j ACCEPT
Accepting to send back packets for all established connection whatever the destination port.-A OUTPUT -p tcp -m state --state ESTABLISHED -j ACCEPT
Now, if you want to send mail to a server, you have to allow packet to go out to port 25 and allow incoming all established connection.
-A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --dport 25 -j ACCEPT
Another idea , will be to log packets that should be dropped.
put log line a the end of all chain's rules.
iptables -A INPUT -p tcp -j LOG --log-prefix "INPUT PACKET DROPPED "
iptables -A OUTPUT -p tcp -j LOG --log-prefix "OUTPUT PACKET DROPPED "
With that, you will see in /var/log/message (or with dmesg) a line for each packet reaching the end of chain's rule and beeing dropped.
I'm securing my server (with iptables) so that only http and ssh ports are open and that is fine, although I use the mail command (server: CentOS 6.2) in some applications and it does not get through now thanks to iptables blocking everything.
What ports do I allow it access to?
Mail usage: echo "{{message}}" | mail -s "{{subject}}" me#mail.com
I've tried the standard mail port 25, but I have had no success with that.
Here is the current setup:
iptables --flush
iptables -P INPUT DROP
iptables -P OUTPUT DROP
# incoming ssh
iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 22 -m state --state ESTABLISHED -j ACCEPT
# outgoing ssh
iptables -A OUTPUT -o eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --sport 22 -m state --state ESTABLISHED -j ACCEPT
#HTTP
iptables -A INPUT -i eth0 -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 80 -m state --state ESTABLISHED -j ACCEPT
# mail (does not work)
iptables -A INPUT -i eth0 -p tcp --dport 25 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 25 -m state --state ESTABLISHED -j ACCEPT
(EDIT) ANSWER: The working iptables rule:
iptables -A OUTPUT -o eth0 -p tcp --sport 25 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --dport 25 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --sport 25 -m state --state ESTABLISHED -j ACCEPT
The OUTPUT commands should also refer to --dport, not --sport. You'll also want to allow NEW outgoing packets in order to initiate the connection to the SMTP server.
In general, however, since OUTPUT controls only those packets that your own system generates, you can set the OUTPUT policy to ACCEPT unless you need to prevent the generation of outgoing packets.
Two more comments:
1. Jay D's suggestion to "allow everything and then start blocking specific traffic" is insecure. Never configure iptables this way because you'd have to know in advance which ports an attacker might use and block them all individually. Always use a whitelist instead of a blacklist if you can.
2. A hint from the trenches: when you're debugging iptables, it's often helpful to -Insert and -Append log messages at the beginning and end of each chain, then clear the counters, and run an experiment. (In your case, issue the mail command.) Then check the counters and logs to understand how the packet(s) migrated through the chains and where they may have been dropped.