Linux

openstack 實時遷移問題

  • July 26, 2018

我有兩個帶有Ceph共享儲存 (RBD) 的計算節點 1 和節點 2，我正在嘗試設置實時遷移，但它失敗並出現以下錯誤，不確定出了什麼問題。

我正在使用 Openstack pike 16.0.16

[root@compute-01 instances]# cat /etc/libvirt/libvirtd.conf
# Ansible managed

listen_tls = 0
listen_tcp = 1
unix_sock_group = "libvirt"
unix_sock_ro_perms = "0777"
unix_sock_rw_perms = "0770"
auth_unix_ro = "none"
auth_unix_rw = "none"
auth_tcp = "none"

nova.log 中出現以下錯誤

如果第一次進行虛擬機實時遷移，那麼它可以工作，但會引發以下錯誤並且虛擬機進入Error狀態

C1 ----> C2 （第一次工作但有錯誤）

lt] [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Received unexpected event network-vif-unplugged-251b70a9-2118-4f95-8b35-e9e52f4392e7 for instance
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [req-e0cd3865-151e-4d07-8b94-3a8943dafb57 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Live migration failed.: AttributeError: 'Guest' object has no attribute 'migrate_configure_max_speed'
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Traceback (most recent call last):
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 5580, in _do_live_migration
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]     block_migration, migrate_data)
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6436, in live_migration
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]     migrate_data)
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6944, in _live_migration
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]     guest.migrate_configure_max_speed(
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] AttributeError: 'Guest' object has no attribute 'migrate_configure_max_speed'
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]
2018-07-25 17:32:41.646 2833 WARNING nova.compute.manager [req-eb9e883f-08c3-427d-89c3-cdcf012e7c8b 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Received unexpected event network-vif-plugged-251b70a9-2118-4f95-8b35-e9e52f4392e7 for instance
2018-07-25 17:32:49.516 2833 WARNING nova.compute.manager [req-d70c12f9-42fc-43be-ae8e-6dd6b21b1b1f 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Received unexpected event network-vif-plugged-251b70a9-2118-4f95-8b35-e9e52f4392e7 for instance

要修復 VM 的錯誤狀態，我必須做

[root@ostack-infra-01-utility-container-a8dbff46 ~]# nova list
+--------------------------------------+------+--------+------------+-------------+-----------------------+
| ID                                   | Name | Status | Task State | Power State | Networks              |
+--------------------------------------+------+--------+------------+-------------+-----------------------+
| 4f4009ee-902d-4ee9-ae99-e9bc55267b32 | d1   | ERROR  | -          | NOSTATE     | net-vlan31=10.31.1.10 |
+--------------------------------------+------+--------+------------+-------------+-----------------------+

nova reset-state --active 4f4009ee-902d-4ee9-ae99-e9bc55267b32

即使在成功遷移後，我的 Horizo​​n 儀表板仍然顯示C1

現在 VM 在 C2 上執行，我必須將其移回 C1 我收到以下錯誤，看起來 nova 在遷移 VM 後沒有清理文件，或者可能是因為它處於錯誤狀態並且它沒有清理以前的文件..

2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [req-285a7f42-99a1-47a2-86c1-79140fcfca31 040960f6067d42c2b52c3fcac9ebde6d 2349c3efbf8a4c6ba6dc3b961160c81b - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Pre live migration failed at ostack-compute-02.v1v0x.net: DestinationDiskExists_Remote: The supplied disk path (/var/lib/nova/instances/aa58095d-7027-488e-901e-f3259353de0d) already exists, it is expected not to exist.
Traceback (most recent call last):

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
   res = self.dispatcher.dispatch(message)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
   return self._do_dispatch(endpoint, method, ctxt, args)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
   result = func(ctxt, **new_args)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped
   function_name, call_dict, binary)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
   self.force_reraise()

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
   six.reraise(self.type_, self.value, self.tb)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped
   return f(self, context, *args, **kw)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/utils.py", line 880, in decorated_function
   return function(self, context, *args, **kwargs)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 218, in decorated_function
   kwargs['instance'], e, sys.exc_info())

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
   self.force_reraise()

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
   six.reraise(self.type_, self.value, self.tb)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 206, in decorated_function
   return function(self, context, *args, **kwargs)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 5507, in pre_live_migration
   migrate_data)

 File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7072, in pre_live_migration
   raise exception.DestinationDiskExists(path=instance_dir)

DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/aa58095d-7027-488e-901e-f3259353de0d) already exists, it is expected not to exist.
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d] Traceback (most recent call last):
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 5562, in _do_live_migration
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     block_migration, disk, dest, migrate_data)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/rpcapi.py", line 745, in pre_live_migration
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     disk=disk, migrate_data=migrate_data)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in call
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     retry=self.retry)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/transport.py", line 123, in _send
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     timeout=timeout, retry=retry)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 566, in send
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     retry=retry)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 557, in _send
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     raise result
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d] DestinationDiskExists_Remote: The supplied disk path (/var/lib/nova/instances/aa58095d-7027-488e-901e-f3259353de0d) already exists, it is expected not to exist.
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d] Traceback (most recent call last):
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     res = self.dispatcher.dispatch(message)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     return self._do_dispatch(endpoint, method, ctxt, args)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     result = func(ctxt, **new_args)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     function_name, call_dict, binary)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     self.force_reraise()
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     six.reraise(self.type_, self.value, self.tb)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     return f(self, context, *args, **kw)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/utils.py", line 880, in decorated_function
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     return function(self, context, *args, **kwargs)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 218, in decorated_function
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     kwargs['instance'], e, sys.exc_info())
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     self.force_reraise()
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     six.reraise(self.type_, self.value, self.tb)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 206, in decorated_function
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     return function(self, context, *args, **kwargs)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 5507, in pre_live_migration
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     migrate_data)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7072, in pre_live_migration
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     raise exception.DestinationDiskExists(path=instance_dir)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d] DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/aa58095d-7027-488e-901e-f3259353de0d) already exists, it is expected not to exist.
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:06.908 8785 WARNING nova.compute.manager [req-cda02056-75c7-463c-ac7e-2925ba2cd29c 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Received unexpected event network-vif-unplugged-7a356ab1-e0d3-4a69-9aa9-f71329caa17f for instance
2018-07-25 19:42:07.821 8785 ERROR nova.virt.libvirt.driver [req-cda02056-75c7-463c-ac7e-2925ba2cd29c 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Live Migration failure: Domain not found: no domain with matching name 'instance-00000056': libvirtError: Domain not found: no domain with matching name 'instance-00000056'
2018-07-25 19:42:08.294 8785 WARNING nova.compute.manager [req-5d3abd38-c8a9-499d-b97f-1d9e748a19d3 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Received unexpected event network-vif-plugged-7a356ab1-e0d3-4a69-9aa9-f71329caa17f for instance
2018-07-25 19:42:14.934 8785 WARNING nova.compute.manager [req-e9442ba9-1700-4221-a495-31d76a2b5bf0 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Received unexpected event network-vif-plugged-7a356ab1-e0d3-4a69-9aa9-f71329caa17f for instance

原始錯誤來自 libvirt 驅動程序呼叫guest.migrate_configure_max_speed這裡：

https://github.com/openstack/nova/blob/stable/pike/nova/virt/libvirt/driver.py#L6944

直到 Pike 之後（ https://github.com/openstack/nova/commit/23446a9552b5be3b040278646149a0f481d0a005 ），該方法才添加到來賓模組中，並且它似乎沒有被反向移植到 Pike，因為它在這裡失去：

https://github.com/openstack/nova/blob/stable/pike/nova/virt/libvirt/guest.py

在此送出中將呼叫guest.migrate_configure_max_speed添加到驅動程序：https ://github.com/openstack/nova/commit/ff747792b8f5aefe1bebb01bdf49dacc01353348

您可以使用 no-op 手動修補 guest.py 以解決該錯誤：

def migrate_configure_max_speed(self, bandwidth):
   pass

引用自：https://unix.stackexchange.com/questions/458441

相關問答

Linux

Qemu 配置文件錯誤“無效的參數主機”

  • July 2, 2021
檢視問答
Linux

如何查看在 VM 上執行的作業系統版本

  • December 14, 2020
檢視問答
Linux

如何使用 QEMU 將 USB 驅動器傳遞到 VM，而不會失去對主機作業系統的訪問權限？

  • January 20, 2020
檢視問答
Linux

如何從一個地方管理多個 kvm？

  • February 9, 2018
檢視問答
Windows

Virt Manager：Windows vm 看不到任何網路

  • August 9, 2022
檢視問答
Routing

為什麼儘管防火牆規則阻止了所有發往它的出站流量，但流量仍繼續流經我的網關？

  • May 6, 2022
檢視問答