From nobody Tue May 13 12:58:22 2025 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675354255; cv=none; d=zohomail.com; s=zohoarc; b=A6RengGbc7VsxMA3W8PQ0BnM9yGuTKo/5pTfp+8q2S7Qb8VQ1ALbnN8ER1OFDYmRldJjcodDYVna96Fw2pbaVsrRTIB2t2pp74TE2hdBMsB0NGvhaEWn1I3BU0YkN36BIcLRewBGBsEiMY6DrTFFYmLDOKjIm1ZfzUxQBLYBl5c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675354255; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7arfj3QdjtnFGkIbZuw6cq2c+PvzWVH+SNgQFzzJt5I=; b=KLSz5ogM5ra1CzHi4YmtRh6KZ54Gp09N8zKa10H33O6nywKA/8zlcON3mxcEix/+FNGkjVDtMRx31mwmv2vRi9Uc8jKyLzU82vR3L331dlTMmYQ7nof/iUlnBbTwsOMz76Qk9uqWhcSPqgIS1VKmjga3AnyNAFHcvInfXUg05eo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675354255775975.3738455435075; Thu, 2 Feb 2023 08:10:55 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pNc8j-0002LD-Po; Thu, 02 Feb 2023 11:08:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pNc8h-000241-QW for qemu-devel@nongnu.org; Thu, 02 Feb 2023 11:08:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pNc8f-0005Kk-UA for qemu-devel@nongnu.org; Thu, 02 Feb 2023 11:08:43 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-250-mJXB7BG2MVuc20zjskvFKQ-1; Thu, 02 Feb 2023 11:08:37 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2B0B880D0F9; Thu, 2 Feb 2023 16:08:17 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.57]) by smtp.corp.redhat.com (Postfix) with ESMTP id 46ADC51E5; Thu, 2 Feb 2023 16:08:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675354120; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7arfj3QdjtnFGkIbZuw6cq2c+PvzWVH+SNgQFzzJt5I=; b=Q7N9nNyOYanO05Sb2Wp+7f1Kc8kPmcN5/X1qbQnLIdWrbhillEmyEA+6gMz7dGINYg91Ad yEGNNf1isdleJMTTCDAdUQG1ayrd1ylyM3IHdYEGnEBzcoBZBdSQCm5kezM0Chq/5N4/wX bHTqS5lIoB4xocOEfh40UWZh5gG0ND4= X-MC-Unique: mJXB7BG2MVuc20zjskvFKQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Richard Henderson , "Michael S. Tsirkin" , Laurent Vivier , Ilya Leoshkevich , Halil Pasic , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Coiby Xu , Eric Farman , Alex Williamson , Christian Borntraeger , Stefan Hajnoczi , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Berger , Eric Blake , Eduardo Habkost , "Dr. David Alan Gilbert" , Thomas Huth , David Hildenbrand , Marcel Apfelbaum , John Snow , Yanan Wang , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org, Paolo Bonzini , Juan Quintela , Fam Zheng , qemu-s390x@nongnu.org, Peter Xu Subject: [PULL 18/26] virtio-mem: Migrate immutable properties early Date: Thu, 2 Feb 2023 17:06:32 +0100 Message-Id: <20230202160640.2300-19-quintela@redhat.com> In-Reply-To: <20230202160640.2300-1-quintela@redhat.com> References: <20230202160640.2300-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675354256492100005 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand The bitmap and the size are immutable while migration is active: see virtio_mem_is_busy(). We can migrate this information early, before migrating any actual RAM content. Further, all information we need for sanity checks is immutable as well. Having this information in place early will, for example, allow for properly preallocating memory before touching these memory locations during RAM migration: this way, we can make sure that all memory was actually preallocated and that any user errors (e.g., insufficient hugetlb pages) can be handled gracefully. In contrast, usable_region_size and requested_size can theoretically still be modified on the source while the VM is running. Keep migrating these properties the usual, late, way. Use a new device property to keep behavior of compat machines unmodified. Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela S Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- include/hw/virtio/virtio-mem.h | 8 ++++++ hw/core/machine.c | 4 ++- hw/virtio/virtio-mem.c | 51 ++++++++++++++++++++++++++++++++-- 3 files changed, 60 insertions(+), 3 deletions(-) diff --git a/include/hw/virtio/virtio-mem.h b/include/hw/virtio/virtio-mem.h index 7745cfc1a3..f15e561785 100644 --- a/include/hw/virtio/virtio-mem.h +++ b/include/hw/virtio/virtio-mem.h @@ -31,6 +31,7 @@ OBJECT_DECLARE_TYPE(VirtIOMEM, VirtIOMEMClass, #define VIRTIO_MEM_BLOCK_SIZE_PROP "block-size" #define VIRTIO_MEM_ADDR_PROP "memaddr" #define VIRTIO_MEM_UNPLUGGED_INACCESSIBLE_PROP "unplugged-inaccessible" +#define VIRTIO_MEM_EARLY_MIGRATION_PROP "x-early-migration" #define VIRTIO_MEM_PREALLOC_PROP "prealloc" =20 struct VirtIOMEM { @@ -74,6 +75,13 @@ struct VirtIOMEM { /* whether to prealloc memory when plugging new blocks */ bool prealloc; =20 + /* + * Whether we migrate properties that are immutable while migration is + * active early, before state of other devices and especially, before + * migrating any RAM content. + */ + bool early_migration; + /* notifiers to notify when "size" changes */ NotifierList size_change_notifiers; =20 diff --git a/hw/core/machine.c b/hw/core/machine.c index f7761baab5..b5cd42cd8c 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -41,7 +41,9 @@ #include "hw/virtio/virtio-pci.h" #include "qom/object_interfaces.h" =20 -GlobalProperty hw_compat_7_2[] =3D {}; +GlobalProperty hw_compat_7_2[] =3D { + { "virtio-mem", "x-early-migration", "false" }, +}; const size_t hw_compat_7_2_len =3D G_N_ELEMENTS(hw_compat_7_2); =20 GlobalProperty hw_compat_7_1[] =3D { diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c index 02f7b5469a..ca37949df8 100644 --- a/hw/virtio/virtio-mem.c +++ b/hw/virtio/virtio-mem.c @@ -31,6 +31,8 @@ #include CONFIG_DEVICES #include "trace.h" =20 +static const VMStateDescription vmstate_virtio_mem_device_early; + /* * We only had legacy x86 guests that did not support * VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE. Other targets don't have legacy gu= ests. @@ -878,6 +880,10 @@ static void virtio_mem_device_realize(DeviceState *dev= , Error **errp) =20 host_memory_backend_set_mapped(vmem->memdev, true); vmstate_register_ram(&vmem->memdev->mr, DEVICE(vmem)); + if (vmem->early_migration) { + vmstate_register(VMSTATE_IF(vmem), VMSTATE_INSTANCE_ID_ANY, + &vmstate_virtio_mem_device_early, vmem); + } qemu_register_reset(virtio_mem_system_reset, vmem); =20 /* @@ -899,6 +905,10 @@ static void virtio_mem_device_unrealize(DeviceState *d= ev) */ memory_region_set_ram_discard_manager(&vmem->memdev->mr, NULL); qemu_unregister_reset(virtio_mem_system_reset, vmem); + if (vmem->early_migration) { + vmstate_unregister(VMSTATE_IF(vmem), &vmstate_virtio_mem_device_ea= rly, + vmem); + } vmstate_unregister_ram(&vmem->memdev->mr, DEVICE(vmem)); host_memory_backend_set_mapped(vmem->memdev, false); virtio_del_queue(vdev, 0); @@ -1015,18 +1025,53 @@ static const VMStateDescription vmstate_virtio_mem_= sanity_checks =3D { }, }; =20 +static bool virtio_mem_vmstate_field_exists(void *opaque, int version_id) +{ + const VirtIOMEM *vmem =3D VIRTIO_MEM(opaque); + + /* With early migration, these fields were already migrated. */ + return !vmem->early_migration; +} + static const VMStateDescription vmstate_virtio_mem_device =3D { .name =3D "virtio-mem-device", .minimum_version_id =3D 1, .version_id =3D 1, .priority =3D MIG_PRI_VIRTIO_MEM, .post_load =3D virtio_mem_post_load, + .fields =3D (VMStateField[]) { + VMSTATE_WITH_TMP_TEST(VirtIOMEM, virtio_mem_vmstate_field_exists, + VirtIOMEMMigSanityChecks, + vmstate_virtio_mem_sanity_checks), + VMSTATE_UINT64(usable_region_size, VirtIOMEM), + VMSTATE_UINT64_TEST(size, VirtIOMEM, virtio_mem_vmstate_field_exis= ts), + VMSTATE_UINT64(requested_size, VirtIOMEM), + VMSTATE_BITMAP_TEST(bitmap, VirtIOMEM, virtio_mem_vmstate_field_ex= ists, + 0, bitmap_size), + VMSTATE_END_OF_LIST() + }, +}; + +/* + * Transfer properties that are immutable while migration is active early, + * such that we have have this information around before migrating any RAM + * content. + * + * Note that virtio_mem_is_busy() makes sure these properties can no longer + * change on the migration source until migration completed. + * + * With QEMU compat machines, we transmit these properties later, via + * vmstate_virtio_mem_device instead -- see virtio_mem_vmstate_field_exist= s(). + */ +static const VMStateDescription vmstate_virtio_mem_device_early =3D { + .name =3D "virtio-mem-device-early", + .minimum_version_id =3D 1, + .version_id =3D 1, + .early_setup =3D true, .fields =3D (VMStateField[]) { VMSTATE_WITH_TMP(VirtIOMEM, VirtIOMEMMigSanityChecks, vmstate_virtio_mem_sanity_checks), - VMSTATE_UINT64(usable_region_size, VirtIOMEM), VMSTATE_UINT64(size, VirtIOMEM), - VMSTATE_UINT64(requested_size, VirtIOMEM), VMSTATE_BITMAP(bitmap, VirtIOMEM, 0, bitmap_size), VMSTATE_END_OF_LIST() }, @@ -1211,6 +1256,8 @@ static Property virtio_mem_properties[] =3D { DEFINE_PROP_ON_OFF_AUTO(VIRTIO_MEM_UNPLUGGED_INACCESSIBLE_PROP, VirtIO= MEM, unplugged_inaccessible, ON_OFF_AUTO_AUTO), #endif + DEFINE_PROP_BOOL(VIRTIO_MEM_EARLY_MIGRATION_PROP, VirtIOMEM, + early_migration, true), DEFINE_PROP_END_OF_LIST(), }; =20 --=20 2.39.1