Linux
Kubernetes 節點錯誤同步 pod
我在 Ubuntu 伺服器 20.04 上安裝了 kubernetes。
大師配置flannel網路沒有任何問題。
初始化命令:
kubeadm init --pod-network-cidr=10.10.10.0/24 --apiserver-advertise-address=172.16.200.10
主機上的網路介面:
cni0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 [4/651] inet 10.10.10.1 netmask 255.255.255.0 broadcast 10.10.10.255 inet6 fe80::8e1:9ff:fe5e:56fb prefixlen 64 scopeid 0x20<link> ether be:cc:5a:4b:f0:c0 txqueuelen 1000 (Ethernet) RX packets 6565 bytes 400726 (400.7 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 3121 bytes 291909 (291.9 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255 ether 02:42:9e:d3:e7:73 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.16.200.10 netmask 255.255.255.0 broadcast 172.16.200.255 inet6 fe80::20c:29ff:fe6b:dbf8 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:6b:db:f8 txqueuelen 1000 (Ethernet) RX packets 7134 bytes 713894 (713.8 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 9077 bytes 2726670 (2.7 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 10.10.10.0 netmask 255.255.255.255 broadcast 10.10.10.0 inet6 fe80::ec9f:dfff:fe19:b2bd prefixlen 64 scopeid 0x20<link> ether ee:9f:df:19:b2:bd txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 14 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 222414 bytes 41222254 (41.2 MB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 222414 bytes 41222254 (41.2 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 veth92c2b55e: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet6 fe80::bccc:5aff:fe4b:f0c0 prefixlen 64 scopeid 0x20<link> ether be:cc:5a:4b:f0:c0 txqueuelen 0 (Ethernet) RX packets 3252 bytes 244283 (244.2 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1602 bytes 148781 (148.7 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vethdc37182b: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet6 fe80::d84e:c0ff:feda:9875 prefixlen 64 scopeid 0x20<link> ether da:4e:c0:da:98:75 txqueuelen 0 (Ethernet) RX packets 3313 bytes 248353 (248.3 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1540 bytes 144798 (144.7 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
但是節點
/var/log/syslog
在加入集群後會導致此錯誤。Jan 27 08:08:33 k8s-node1 kubelet[5393]: E0127 08:08:33.534423 5393 pod_workers.go:191] Error syncing pod 90b895af-b046-4149-ac18-bbec859c37a2 ("kube-flannel-ds-vsgx8_kube-system(90b895af-b046-4149-ac18-bbec859c37a2)"), skipping: failed to "StartContainer" for "kube-flannel" with CrashLoopBackOff: "back-off 2m40s restarting failed container=kube-flannel pod=kube-flannel-ds-vsgx8_kube-system(90b895af-b046-4149-ac18-bbec859c37a2)"
這是節點狀態:
root@k8s-master:~# kubectl get node -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-master Ready control-plane,master 42m v1.20.2 172.16.200.10 <none> Ubuntu 20.04.1 LTS 5.4.0-62-generic docker://19.3.8 k8s-node1 Ready <none> 35m v1.20.2 172.16.200.20 <none> Ubuntu 20.04.1 LTS 5.4.0-62-generic docker://19.3.8
這是什麼錯誤?
如何解決?
更新
這是 Node1 中存在的容器:
root@k8s-node1:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 358f8cdc6141 f03a23d55e57 "/opt/bin/flanneld -…" 15 seconds ago Exited (1) 12 seconds ago k8s_kube-flannel_kube-flannel-ds-vsgx8_kube-system_90b895af-b046-4149-ac18-bbec859c37a2_7 e3a40bcc9db3 f03a23d55e57 "cp -f /etc/kube-fla…" 11 minutes ago Exited (0) 11 minutes ago k8s_install-cni_kube-flannel-ds-vsgx8_kube-system_90b895af-b046-4149-ac18-bbec859c37a2_0 7ccb8514bed7 43154ddb57a8 "/usr/local/bin/kube…" 11 minutes ago Up 11 minutes k8s_kube-proxy_kube-proxy-mvbw2_kube-system_5090f36a-4178-4318-aa15-19c61239045f_0 27ae4b2a1157 k8s.gcr.io/pause:3.2 "/pause" 11 minutes ago Up 11 minutes k8s_POD_kube-flannel-ds-vsgx8_kube-system_90b895af-b046-4149-ac18-bbec859c37a2_0 507f2d4ab72c k8s.gcr.io/pause:3.2 "/pause" 11 minutes ago Up 11 minutes k8s_POD_kube-proxy-mvbw2_kube-system_5090f36a-4178-4318-aa15-19c61239045f_0
容器錯誤:
root@k8s-node1:~# docker logs k8s_kube-flannel_kube-flannel-ds-vsgx8_kube-system_90b895af-b046-4149-ac18-bbec859c37a2_7 ERROR: logging before flag.Parse: I0127 08:16:06.400204 1 main.go:519] Determining IP address of default interface ERROR: logging before flag.Parse: I0127 08:16:06.402530 1 main.go:532] Using interface with name eth0 and address 172.16.200.20 ERROR: logging before flag.Parse: I0127 08:16:06.402721 1 main.go:549] Defaulting external address to interface address (172.16.200.20) W0127 08:16:06.402840 1 client_config.go:608] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. ERROR: logging before flag.Parse: I0127 08:16:07.100353 1 kube.go:116] Waiting 10m0s for node controller to sync ERROR: logging before flag.Parse: I0127 08:16:07.100618 1 kube.go:299] Starting kube subnet manager ERROR: logging before flag.Parse: I0127 08:16:08.101341 1 kube.go:123] Node controller sync successful ERROR: logging before flag.Parse: I0127 08:16:08.101408 1 main.go:253] Created subnet manager: Kubernetes Subnet Manager - k8s-node1 ERROR: logging before flag.Parse: I0127 08:16:08.101423 1 main.go:256] Installing signal handlers ERROR: logging before flag.Parse: I0127 08:16:08.102395 1 main.go:391] Found network config - Backend type: vxlan ERROR: logging before flag.Parse: I0127 08:16:08.103389 1 vxlan.go:123] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false ERROR: logging before flag.Parse: E0127 08:16:08.104363 1 main.go:292] Error registering network: failed to acquire lease: node "k8s-node1" pod cidr not assigned ERROR: logging before flag.Parse: I0127 08:16:08.105069 1 main.go:371] Stopping shutdownHandler...
為什麼在節點中,法蘭絨容器退出?!!!
這
--pod-network-cidr 10.10.10.0/24
聽起來不對。如果你做 a
kubectl describe node <your-master-node-name>
,我敢打賭會有類似 a 的東西PodCIDR: 10.10.10.0/24
。您的第一個節點已經佔用了整個 pod 網路 CIDR,因為您計劃的一個太小了。一旦 flannel 第一次在 Kubernetes 節點上啟動,它就會為自己分配一個 /24 子網,該子網位於 pod 網路範圍內。我假設您的 IP 已經用完,您的第二個節點將加入 SDN。