Linux

使用本地介面作為下一跳為 virbr0 配置靜態路由

  • July 14, 2020

具有以下拓撲:

hostA(virbr0: 192.168.122.1) – TAP 介面 – (eth0: 192.168.122.85) gw (eth1: 192.168.3.51) — (LAN: 192.168.3.0/24)

gw:QEMU VM Guest,作為其他 QEMU VM Guest 的網關,通過eth1介面連接到它。gw通過 Linux TAP 介面,使用 interface 連接到 QEMU 主機eth0。TAB 介面正在動態分配 IP eth0

hostA: QEMU 虛擬機主機

root@hostA:~$ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
       inet 172.16.254.134  netmask 255.255.255.0  broadcast 172.16.254.255
       inet6 fe80::20c:29ff:fe25:1670  prefixlen 64  scopeid 0x20<link>
       ether 00:0c:29:25:16:70  txqueuelen 1000  (Ethernet)
       RX packets 18116  bytes 1361094 (1.3 MB)
       RX errors 0  dropped 0  overruns 0  frame 0
       TX packets 14579  bytes 5334119 (5.3 MB)
       TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

gns3tap0-0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
       inet6 fe80::50bd:d3ff:fe1a:e55  prefixlen 64  scopeid 0x20<link>
       ether 52:bd:d3:1a:0e:55  txqueuelen 1000  (Ethernet)
       RX packets 353  bytes 28543 (28.5 KB)
       RX errors 0  dropped 0  overruns 0  frame 0
       TX packets 3348  bytes 183399 (183.3 KB)
       TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
       inet 127.0.0.1  netmask 255.0.0.0
       inet6 ::1  prefixlen 128  scopeid 0x10<host>
       loop  txqueuelen 1000  (Local Loopback)
       RX packets 662311  bytes 99203163 (99.2 MB)
       RX errors 0  dropped 0  overruns 0  frame 0
       TX packets 662311  bytes 99203163 (99.2 MB)
       TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
       inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
       ether 52:54:00:62:0a:c5  txqueuelen 1000  (Ethernet)
       RX packets 353  bytes 23601 (23.6 KB)
       RX errors 0  dropped 0  overruns 0  frame 0
       TX packets 379  bytes 28629 (28.6 KB)
       TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
gw:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 0C:FE:27:0C:E4:00
         inet addr:192.168.122.85  Bcast:192.168.122.255  Mask:255.255.255.0
         inet6 addr: fe80::efe:27ff:fe0c:e400/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:379 errors:3150 dropped:0 overruns:0 frame:3150
         TX packets:353 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:31851 (31.1 KiB)  TX bytes:28543 (27.8 KiB)

eth1      Link encap:Ethernet  HWaddr 0C:FE:27:0C:E4:01
         inet addr:192.168.3.51  Bcast:0.0.0.0  Mask:255.255.255.0
         inet6 addr: fe80::efe:27ff:fe0c:e401/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:302 errors:0 dropped:0 overruns:0 frame:0
         TX packets:300 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:25796 (25.1 KiB)  TX bytes:24782 (24.2 KiB)
gw:~# netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.122.1   0.0.0.0         UG        0 0          0 eth0
192.168.3.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 eth0

目標是從 hostA 到內部 LAN 的 VM 192.168.3.0/24 的可達性

以下路由配置正在執行:

root@hostA:~$ sudo route add -net 192.168.3.0 netmask 255.255.255.0 gw 192.168.122.85
root@hostA:~$ netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         172.16.254.2    0.0.0.0         UG        0 0          0 eth0
172.16.254.0    0.0.0.0         255.255.255.0   U         0 0          0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U         0 0          0 docker0
192.168.3.0     192.168.122.85  255.255.255.0   UG        0 0          0 virbr0
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 virbr0
root@hostA:~$ ping 192.168.3.102
PING 192.168.3.102 (192.168.3.102) 56(84) bytes of data.
64 bytes from 192.168.3.102: icmp_seq=1 ttl=63 time=3.07 ms
^C
--- 192.168.3.102 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.071/3.071/3.071/0.000 ms
root@hostA:~$ ping 192.168.3.103
PING 192.168.3.103 (192.168.3.103) 56(84) bytes of data.
64 bytes from 192.168.3.103: icmp_seq=1 ttl=63 time=3.03 ms
64 bytes from 192.168.3.103: icmp_seq=2 ttl=63 time=2.99 ms
^C
--- 192.168.3.103 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 2.997/3.014/3.032/0.057 ms

但是,由於Linux TAP是eth0動態分配IP的,所以需要有路由,不會使用下一個路由器的IP地址作為下一跳,而是出口介面的本地IP,類似到以下:


root@hostA:~$ route add -net 192.168.0.0 netmask 255.255.0.0 gw 192.168.122.1 dev virbr0
root@hostA:~$ netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         172.16.254.2    0.0.0.0         UG        0 0          0 eth0
172.16.254.0    0.0.0.0         255.255.255.0   U         0 0          0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U         0 0          0 docker0
192.168.0.0     192.168.122.1   255.255.0.0     UG        0 0          0 virbr0
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 virbr

root@hostA:~$ ping -c 1 192.168.122.85
PING 192.168.122.85 (192.168.122.85) 56(84) bytes of data.
64 bytes from 192.168.122.85: icmp_seq=1 ttl=64 time=0.585 ms

--- 192.168.122.85 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.585/0.585/0.585/0.000 ms
root@hostA:~$ ping -c 1 192.168.3.51
PING 192.168.3.51 (192.168.3.51) 56(84) bytes of data.
64 bytes from 192.168.3.51: icmp_seq=1 ttl=64 time=0.956 ms

--- 192.168.3.51 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.956/0.956/0.956/0.000 ms

root@hostA:~$ ping -c 1 192.168.3.102
PING 192.168.3.102 (192.168.3.102) 56(84) bytes of data.
From 192.168.122.1 icmp_seq=1 Destination Host Unreachable

--- 192.168.3.102 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

root@hostA:~$ ping -c 1 192.168.3.103
PING 192.168.3.103 (192.168.3.103) 56(84) bytes of data.
From 192.168.122.1 icmp_seq=1 Destination Host Unreachable

--- 192.168.3.103 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

可以看出,當使用上述路由配置時,hostA能夠到達eth1介面,但無法到達內部 LAN (192.168.3.0/24) 中的任何其他虛擬機。但是,內部 LAN 主機本身能夠訪問 hostA。

這是iptables配置:

root@hostA:~$ sudo iptables -nvL FORWARD
Chain FORWARD (policy ACCEPT 21 packets, 1624 bytes)
pkts bytes target     prot opt in     out     source               destination
 197 15616 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
 197 15616 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
   0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
   0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
   0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
   0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
  88  7346 ACCEPT     all  --  *      virbr0  0.0.0.0/0            192.168.122.0/24     ctstate RELATED,ESTABLISHED
  88  6646 ACCEPT     all  --  virbr0 *       192.168.122.0/24     0.0.0.0/0
   0     0 ACCEPT     all  --  virbr0 virbr0  0.0.0.0/0            0.0.0.0/0

gw:~# iptables -nvL FORWARD
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
   0     0 ACCEPT     all  --  eth1   eth0    192.168.122.0/24     0.0.0.0/0            ctstate NEW
   0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
gw:~# sysctl -a | grep ipv4.ip_forward
net.ipv4.ip_forward = 1

dns:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 0C:FE:27:0C:76:00
         inet addr:192.168.3.103  Bcast:0.0.0.0  Mask:255.255.255.0
         inet6 addr: fe80::efe:27ff:fe0c:7600/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:18638 errors:0 dropped:0 overruns:0 frame:0
         TX packets:18634 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:1543312 (1.4 MiB)  TX bytes:1705846 (1.6 MiB)

dns:~# netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.3.51    0.0.0.0         UG        0 0          0 eth0
192.168.3.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0

問題:

  1. 配置是否缺少某些內容?
  2. 有沒有辦法配置工作路由條目,它將使用出口介面的本地 IP 作為下一跳 IP?

有兩三個不同的問題。

動態網關 IP

為此使用路由協議將是矯枉過正(除非您已經出於其他原因使用了一個)。我會嘗試將 gw/eth0 的配置更改為靜態。如果這不可能,那麼您可以創建一個使用者,在該使用者gwhostA獲得sudo設置到該網路的路由的權限。每次gw獲得新 IP 時,它都可以通過 SSH 連接hostA並執行該命令(可能通過ForcedCommand)。

gw上的路由

在 上未(正確)配置路由gw。您必須通過net.ipv4.ip_forward = 0( man sysctl) 完全啟用它並允許防火牆中的相應連接,請參閱iptables -nvL FORWARD

在 192.168.3.0 中的虛擬機上路由

連接的虛擬機gw/eth1必須gw配置為預設網關。如果不是這種情況,那麼gw必須SNAT對這個網路做(iptables)。

引用自:https://unix.stackexchange.com/questions/598477