Networking

proxmox 和 pfsense VM 之間的 IPTables 問題

  • December 9, 2021

我對 IPTables 問題感到絕望。

我開始設置自己的專用伺服器，即 proxmox 伺服器。我有一個所有流量都路由到的 pfSense 虛擬機。我通過建構 VPN 來保護 LAN 訪問。一切正常，每台機器都互相 ping 通，但我的主機有一個問題：proxmox 伺服器**。**我可以 ping 它，但我無法訪問 Proxmox Web UI（埠 8006）或任何其他埠（我嘗試使用埠 8000 的 http.server）。

這是我設置 iptables 的腳本：

#!/bin/sh

   # ---------
   # VARIABLES
   # ---------

## Proxmox bridge holding Public IP
PrxPubVBR="vmbr0"
## Proxmox bridge on VmWanNET (PFSense WAN side)
PrxVmWanVBR="vmbr1"

## Network/Mask of VmWanNET
VmWanNET="192.168.0.0/30"

## Public IP => Your own public IP address
PublicIP="1.2.3.4"
## Proxmox IP on the same network than PFSense WAN (VmWanNET)
ProxVmWanIP="192.168.0.1"
## PFSense IP used by the firewall (inside VM)
PfsVmWanIP="192.168.0.2"

   # ---------------------
   # CLEAN ALL & DROP IPV6
   # ---------------------

### Delete all existing rules.
iptables -F
iptables -t nat -F
iptables -t mangle -F
iptables -X
### This policy does not handle IPv6 traffic except to drop it.
ip6tables -P INPUT DROP
ip6tables -P OUTPUT DROP
ip6tables -P FORWARD DROP

   # --------------
   # DEFAULT POLICY
   # --------------

### Block ALL !
iptables -P OUTPUT DROP
iptables -P INPUT DROP
iptables -P FORWARD DROP

   # ------
   # CHAINS
   # ------

### Creating chains
iptables -N TCP
iptables -N UDP

# UDP = ACCEPT / SEND TO THIS CHAIN
iptables -A INPUT -p udp -m conntrack --ctstate NEW -j UDP
# TCP = ACCEPT / SEND TO THIS CHAIN
iptables -A INPUT -p tcp --syn -m conntrack --ctstate NEW -j TCP

   # ------------
   # GLOBAL RULES
   # ------------

# Allow localhost
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Don't break the current/active connections
iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Allow Ping - Comment this to return timeout to ping request
iptables -A INPUT -p icmp --icmp-type 8 -m conntrack --ctstate NEW -j ACCEPT

   # --------------------
   # RULES FOR PrxPubVBR
   # --------------------

### INPUT RULES
# ---------------

# Allow SSH server
iptables -A TCP -i $PrxPubVBR -d $PublicIP -p tcp --dport 56361 -j ACCEPT
# Allow Proxmox WebUI
#iptables -A TCP -i $PrxPubVBR -d $PublicIP -p tcp --dport 8006 -j ACCEPT

### OUTPUT RULES
# ---------------

# Allow ping out
iptables -A OUTPUT -p icmp -j ACCEPT

### Proxmox Host as CLIENT
# Allow HTTP/HTTPS
iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --dport 80 -j ACCEPT
iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --dport 443 -j ACCEPT
# Allow DNS
iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p udp --dport 53 -j ACCEPT

### Proxmox Host as SERVER
# Allow SSH
iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --sport 56361 -j ACCEPT
# Allow PROXMOX WebUI
#iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --sport 8006 -j ACCEPT

### FORWARD RULES
# ----------------

### Redirect (NAT) traffic from internet
# All tcp to PFSense WAN except 56361, 8006
iptables -A PREROUTING -t nat -i $PrxPubVBR -p tcp --match multiport ! --dport 56361 -j DNAT --to $PfsVmWanIP
# All udp to PFSense WAN
iptables -A PREROUTING -t nat -i $PrxPubVBR -p udp -j DNAT --to $PfsVmWanIP

# Allow request forwarding to PFSense WAN interface
iptables -A FORWARD -i $PrxPubVBR -d $PfsVmWanIP -o $PrxVmWanVBR -p tcp -j ACCEPT
iptables -A FORWARD -i $PrxPubVBR -d $PfsVmWanIP -o $PrxVmWanVBR -p udp -j ACCEPT

# Allow request forwarding from LAN
iptables -A FORWARD -i $PrxVmWanVBR -s $VmWanNET -j ACCEPT

### MASQUERADE MANDATORY
# Allow WAN network (PFSense) to use vmbr0 public adress to go out
iptables -t nat -A POSTROUTING -s $VmWanNET -o $PrxPubVBR -j MASQUERADE

#Allow WAN network (PFSense) to use vmbr1
iptables -A TCP -i $PrxVmWanVBR -d $ProxVmWanIP -p tcp --dport 8006 -j ACCEPT

   # --------------------
   # RULES FOR PrxVmWanVBR
   # --------------------

### Allow being a client for the VMs
#iptables -A OUTPUT -o $PrxVmWanVBR -s $ProxVmWanIP -p tcp -j ACCEPT

#Debug
iptables -A OUTPUT -o vmbr1 -s 192.168.0.1 -p tcp -j LOG

proxmox 伺服器上的iptables -L (192.168.0.1)

Chain INPUT (policy DROP)
target     prot opt source               destination
UDP        udp  --  anywhere             anywhere             ctstate NEW
TCP        tcp  --  anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN ctstate NEW
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     icmp --  anywhere             anywhere             icmp echo-request ctstate NEW

Chain FORWARD (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             192.168.0.2
ACCEPT     udp  --  anywhere             192.168.0.2
ACCEPT     all  --  192.168.0.0/30       anywhere

Chain OUTPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere
ACCEPT     tcp  --  proxhost             anywhere             tcp dpt:http
ACCEPT     tcp  --  proxhost             anywhere             tcp dpt:https
ACCEPT     udp  --  proxhost             anywhere             udp dpt:domain
ACCEPT     tcp  --  proxhost             anywhere             tcp spt:56361
LOG        tcp  --  192.168.0.1          anywhere             LOG level warning

Chain TCP (1 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             proxhost             tcp dpt:56361
ACCEPT     tcp  --  anywhere             192.168.0.1          tcp dpt:8006

Chain UDP (1 references)
target     prot opt source               destination

tcpdump -i vmbr1 -p tcp port 8006PROXMOX 伺服器 (192.168.0.1) 上的範例

21:55:20.469413 IP 192.168.0.2.19382 > 192.168.0.1.8006: Flags [S], seq 2361462968, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:51.271929 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:51.525784 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:52.279653 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:52.536924 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:54.293975 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:54.550895 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:58.308703 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:58.551754 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:57:06.313913 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:57:06.552666 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0

我一定錯過了什麼，但我真的看不出在哪裡。謝謝 ：）

編輯：

iptables-save -c

# Generated by iptables-save v1.8.2 on Thu Dec  9 20:23:23 2021
*mangle
:PREROUTING ACCEPT [38261:1392282]
:INPUT ACCEPT [24532:758223]
:FORWARD ACCEPT [10902:503215]
:OUTPUT ACCEPT [21470:682727]
:POSTROUTING ACCEPT [29151:1040069]
COMMIT
# Completed on Thu Dec  9 20:23:23 2021
# Generated by iptables-save v1.8.2 on Thu Dec  9 20:23:23 2021
*nat
:PREROUTING ACCEPT [6492:234198]
:INPUT ACCEPT [2:104]
:OUTPUT ACCEPT [227:14700]
:POSTROUTING ACCEPT [2486:123267]
[2311:104588] -A PREROUTING -i vmbr0 -p tcp -m multiport ! --dports 56361 -j DNAT --to-destination 192.168.0.2
[2967:147367] -A PREROUTING -i vmbr0 -p udp -j DNAT --to-destination 192.168.0.2
[307:23332] -A POSTROUTING -s 192.168.0.0/30 -o vmbr0 -j MASQUERADE
COMMIT
# Completed on Thu Dec  9 20:23:23 2021
# Generated by iptables-save v1.8.2 on Thu Dec  9 20:23:23 2021
*raw
:PREROUTING ACCEPT [7393057:7076961567]
:OUTPUT ACCEPT [135130:41526439]
COMMIT
# Completed on Thu Dec  9 20:23:23 2021
# Generated by iptables-save v1.8.2 on Thu Dec  9 20:23:23 2021
*filter
:INPUT DROP [3256:115951]
:FORWARD DROP [2988:97686]
:OUTPUT DROP [233:48187]
:TCP - [0:0]
:UDP - [0:0]
[0:0] -A INPUT -p udp -m conntrack --ctstate NEW -j UDP
[2:104] -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j TCP
[3:156] -A INPUT -i lo -j ACCEPT
[21271:642012] -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
[1189:95194] -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
[6277:270428] -A FORWARD -d 192.168.0.2/32 -i vmbr0 -o vmbr1 -p tcp -j ACCEPT
[140:16523] -A FORWARD -d 192.168.0.2/32 -i vmbr0 -o vmbr1 -p udp -j ACCEPT
[308:23384] -A FORWARD -s 192.168.0.0/30 -i vmbr1 -j ACCEPT
[3:156] -A OUTPUT -o lo -j ACCEPT
[21055:611502] -A OUTPUT -p icmp -j ACCEPT
[0:0] -A OUTPUT -s 1.2.3.4/32 -o vmbr0 -p tcp -m tcp --dport 80 -j ACCEPT
[0:0] -A OUTPUT -s 1.2.3.4/32 -o vmbr0 -p tcp -m tcp --dport 443 -j ACCEPT
[70:4312] -A OUTPUT -s 1.2.3.4/32 -o vmbr0 -p udp -m udp --dport 53 -j ACCEPT
[115:19514] -A OUTPUT -s 1.2.3.4/32 -o vmbr0 -p tcp -m tcp --sport 56361 -j ACCEPT
[2:104] -A TCP -d 1.2.3.4/32 -i vmbr0 -p tcp -m tcp --dport 56361 -j ACCEPT
[0:0] -A TCP -d 192.168.0.1/32 -i vmbr1 -p tcp -m tcp --dport 8006 -j ACCEPT
COMMIT
# Completed on Thu Dec  9 20:23:23 2021

和

ip -br link:

lo               UNKNOWN        00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP>
eno1             UP             ac:1f:6b:71:2c:f6 <BROADCAST,MULTICAST,UP,LOWER_UP>
eno2             DOWN           ac:1f:6b:71:2c:f7 <BROADCAST,MULTICAST>
vmbr0            UP             ac:1f:6b:71:2c:f6 <BROADCAST,MULTICAST,UP,LOWER_UP>
vmbr1            UP             fe:b6:c7:0f:cb:04 <BROADCAST,MULTICAST,UP,LOWER_UP>
vmbr2            UP             82:6f:f1:04:65:39 <BROADCAST,MULTICAST,UP,LOWER_UP>

ip -br address:

lo               UNKNOWN        127.0.0.1/8 ::1/128
eno1             UP
eno2             DOWN
vmbr0            UP             1.2.3.4/32 fe80::ae1f:6bff:fe71:2cf6/64
vmbr1            UP             192.168.0.1/30 fe80::2c70:1ff:fe15:c679/64
vmbr2            UP             192.168.1.1/24 fe80::be:fcff:fea0:2bca/64

ip route:

default via 1.2.3.4 dev vmbr0 proto kernel onlink
10.2.2.0/24 via 192.168.0.2 dev vmbr1
192.168.0.0/30 dev vmbr1 proto kernel scope link src 192.168.0.1
192.168.1.0/24 via 192.168.0.2 dev vmbr1

您的規則集在轉發/輸對外連結中缺少允許回复流量的有狀態規則。

添加這個應該通過允許有狀態的回復來解決問題：

iptables -I OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

請注意，在iptables腳本中，允許回复（使用無狀態規則）的此命令無效，並且不會重新出現在規則集中，因為它已被註釋掉：

# Allow PROXMOX WebUI
#iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --sport 8006 -j ACCEPT

引用自：https://unix.stackexchange.com/questions/680774

相關問答

Networking

如何使用 tcpdump 之類的工具獲取聚合數據包資訊

  • November 8, 2022
檢視問答
Shell

tcpdump：總是顯示數字埠號

  • August 25, 2022
檢視問答
Linux

跨網路命名空間的 Linux 數據包標記

  • May 31, 2022
檢視問答
Networking

GRE IP 到虛擬機 (Proxmox) - Traceroute 顯示完整路由

  • April 27, 2022
檢視問答
Ubuntu

如何從 netdata 禁用 tcp_resets_received

  • April 25, 2022
檢視問答
Networking

本地透明代理 SSLSplit 導致轉發循環

  • February 27, 2022
檢視問答