From nobody Tue May 13 12:50:34 2025 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675354538; cv=none; d=zohomail.com; s=zohoarc; b=ZBz6MolCwK5Bgu8D1Yz5wjXUWG4v4p69S8hw0oAjTNVrnNnVqDrgBawIR9mVCLY/Mp96OvP3iLauvnz5VqBGINujN/DLGcibLTk6EHTLcTKNoayCYb9Z2HiKPY/+QqEHT0U3aVOSduZzU+wmQ/q7VuooZWsjrCEv55BBj75Mj0Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675354538; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0e4q/mah2t6LaHc8IYmZiEvJhspLzN0NjJkVfWmRqy4=; b=cCKs8IqxfzkvsNKjA7SiA32KmgK2mNw0CiIl4GHbDJXlQJKpOyPkd0LCWVHEM9l60PnSZov3jTumnGYf0FElHj0PhEKQznHhk0Zo9oDBNAihOYXvxV7BiKViQ9hV6AsBxlPrinkryFagYA32kZWBpBh/FiuYfGEHsoqHLMKZCuU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675354538457646.9438196552587; Thu, 2 Feb 2023 08:15:38 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pNc9Q-0005Fe-Fx; Thu, 02 Feb 2023 11:09:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pNc9B-0004Sz-2P for qemu-devel@nongnu.org; Thu, 02 Feb 2023 11:09:13 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pNc97-0005VT-V8 for qemu-devel@nongnu.org; Thu, 02 Feb 2023 11:09:12 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-288-G0wIRsXsOdSjPuzhaF1z2A-1; Thu, 02 Feb 2023 11:09:04 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0CFDF3C14864; Thu, 2 Feb 2023 16:09:03 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.57]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0DD3451E5; Thu, 2 Feb 2023 16:08:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675354149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0e4q/mah2t6LaHc8IYmZiEvJhspLzN0NjJkVfWmRqy4=; b=HgD/YIs2ryPCkR1MPxn899llP+y4mg/oOuPsW8idXo2gNiuqIgLDsc9PrMz8Iwgg7kRhBC glmRuewkWIbVpzc0zRTIsv5A0G5k86hmY7UwKYyHh8cwGoZ1raLOXIqzqaNoHRyMfY8qGE wLzq9YhqpFWMWQ57rkfMSgEZY9QaRZE= X-MC-Unique: G0wIRsXsOdSjPuzhaF1z2A-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Richard Henderson , "Michael S. Tsirkin" , Laurent Vivier , Ilya Leoshkevich , Halil Pasic , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Coiby Xu , Eric Farman , Alex Williamson , Christian Borntraeger , Stefan Hajnoczi , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Berger , Eric Blake , Eduardo Habkost , "Dr. David Alan Gilbert" , Thomas Huth , David Hildenbrand , Marcel Apfelbaum , John Snow , Yanan Wang , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org, Paolo Bonzini , Juan Quintela , Fam Zheng , qemu-s390x@nongnu.org, "manish.mishra" , Peter Xu Subject: [PULL 26/26] migration: check magic value for deciding the mapping of channels Date: Thu, 2 Feb 2023 17:06:40 +0100 Message-Id: <20230202160640.2300-27-quintela@redhat.com> In-Reply-To: <20230202160640.2300-1-quintela@redhat.com> References: <20230202160640.2300-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675354540009100003 Content-Type: text/plain; charset="utf-8" From: "manish.mishra" Current logic assumes that channel connections on the destination side are always established in the same order as the source and the first one will always be the main channel followed by the multifid or post-copy preemption channel. This may not be always true, as even if a channel has a connection established on the source side it can be in the pending state on the destination side and a newer connection can be established first. Basically causing out of order mapping of channels on the destination side. Currently, all channels except post-copy preempt send a magic number, this patch uses that magic number to decide the type of channel. This logic is applicable only for precopy(multifd) live migration, as mentioned, the post-copy preempt channel does not send any magic number. Also, tls live migrations already does tls handshake before creating other channels, so this issue is not possible with tls, hence this logic is avoided for tls live migrations. This patch uses read peek to check the magic number of channels so that current data/control stream management remains un-effected. Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrange Reviewed-by: Juan Quintela Suggested-by: Daniel P. Berrange Signed-off-by: manish.mishra Signed-off-by: Juan Quintela --- migration/channel.h | 5 ++++ migration/multifd.h | 2 +- migration/postcopy-ram.h | 2 +- migration/channel.c | 45 +++++++++++++++++++++++++++++++++ migration/migration.c | 54 ++++++++++++++++++++++++++++------------ migration/multifd.c | 19 +++++++------- migration/postcopy-ram.c | 5 +--- 7 files changed, 101 insertions(+), 31 deletions(-) diff --git a/migration/channel.h b/migration/channel.h index 67a461c28a..5bdb8208a7 100644 --- a/migration/channel.h +++ b/migration/channel.h @@ -24,4 +24,9 @@ void migration_channel_connect(MigrationState *s, QIOChannel *ioc, const char *hostname, Error *error_in); + +int migration_channel_read_peek(QIOChannel *ioc, + const char *buf, + const size_t buflen, + Error **errp); #endif diff --git a/migration/multifd.h b/migration/multifd.h index e2802a9ce2..ff3aa2e2e9 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -18,7 +18,7 @@ void multifd_save_cleanup(void); int multifd_load_setup(Error **errp); int multifd_load_cleanup(Error **errp); bool multifd_recv_all_channels_created(void); -bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp); +void multifd_recv_new_channel(QIOChannel *ioc, Error **errp); void multifd_recv_sync_main(void); int multifd_send_sync_main(QEMUFile *f); int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset); diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 6147bf7d1d..25881c4127 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -190,7 +190,7 @@ enum PostcopyChannels { RAM_CHANNEL_MAX, }; =20 -bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); +void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); int postcopy_preempt_setup(MigrationState *s, Error **errp); int postcopy_preempt_wait_channel(MigrationState *s); =20 diff --git a/migration/channel.c b/migration/channel.c index 1b0815039f..ca3319a309 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -92,3 +92,48 @@ void migration_channel_connect(MigrationState *s, migrate_fd_connect(s, error); error_free(error); } + + +/** + * @migration_channel_read_peek - Peek at migration channel, without + * actually removing it from channel buffer. + * + * @ioc: the channel object + * @buf: the memory region to read data into + * @buflen: the number of bytes to read in @buf + * @errp: pointer to a NULL-initialized error object + * + * Returns 0 if successful, returns -1 and sets @errp if fails. + */ +int migration_channel_read_peek(QIOChannel *ioc, + const char *buf, + const size_t buflen, + Error **errp) +{ + ssize_t len =3D 0; + struct iovec iov =3D { .iov_base =3D (char *)buf, .iov_len =3D buflen = }; + + while (true) { + len =3D qio_channel_readv_full(ioc, &iov, 1, NULL, NULL, + QIO_CHANNEL_READ_FLAG_MSG_PEEK, errp); + + if (len <=3D 0 && len !=3D QIO_CHANNEL_ERR_BLOCK) { + error_setg(errp, + "Failed to peek at channel"); + return -1; + } + + if (len =3D=3D buflen) { + break; + } + + /* 1ms sleep. */ + if (qemu_in_coroutine()) { + qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 1000000); + } else { + g_usleep(1000); + } + } + + return 0; +} diff --git a/migration/migration.c b/migration/migration.c index 6509203080..f4f7d207f0 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -31,6 +31,7 @@ #include "migration.h" #include "savevm.h" #include "qemu-file.h" +#include "channel.h" #include "migration/vmstate.h" #include "block/block.h" #include "qapi/error.h" @@ -663,10 +664,6 @@ static bool migration_incoming_setup(QEMUFile *f, Erro= r **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 - if (multifd_load_setup(errp) !=3D 0) { - return false; - } - if (!mis->from_src_file) { mis->from_src_file =3D f; } @@ -733,31 +730,56 @@ void migration_ioc_process_incoming(QIOChannel *ioc, = Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); Error *local_err =3D NULL; - bool start_migration; QEMUFile *f; + bool default_channel =3D true; + uint32_t channel_magic =3D 0; + int ret =3D 0; =20 - if (!mis->from_src_file) { - /* The first connection (multifd may have multiple) */ + if (migrate_use_multifd() && !migrate_postcopy_ram() && + qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_READ_MSG_PEEK)) { + /* + * With multiple channels, it is possible that we receive channels + * out of order on destination side, causing incorrect mapping of + * source channels on destination side. Check channel MAGIC to + * decide type of channel. Please note this is best effort, postco= py + * preempt channel does not send any magic number so avoid it for + * postcopy live migration. Also tls live migration already does + * tls handshake while initializing main channel so with tls this + * issue is not possible. + */ + ret =3D migration_channel_read_peek(ioc, (void *)&channel_magic, + sizeof(channel_magic), &local_er= r); + + if (ret !=3D 0) { + error_propagate(errp, local_err); + return; + } + + default_channel =3D (channel_magic =3D=3D cpu_to_be32(QEMU_VM_FILE= _MAGIC)); + } else { + default_channel =3D !mis->from_src_file; + } + + if (multifd_load_setup(errp) !=3D 0) { + error_setg(errp, "Failed to setup multifd channels"); + return; + } + + if (default_channel) { f =3D qemu_file_new_input(ioc); =20 if (!migration_incoming_setup(f, errp)) { return; } - - /* - * Common migration only needs one channel, so we can start - * right now. Some features need more than one channel, we wait. - */ - start_migration =3D !migration_needs_multiple_sockets(); } else { /* Multiple connections */ assert(migration_needs_multiple_sockets()); if (migrate_use_multifd()) { - start_migration =3D multifd_recv_new_channel(ioc, &local_err); + multifd_recv_new_channel(ioc, &local_err); } else { assert(migrate_postcopy_preempt()); f =3D qemu_file_new_input(ioc); - start_migration =3D postcopy_preempt_new_channel(mis, f); + postcopy_preempt_new_channel(mis, f); } if (local_err) { error_propagate(errp, local_err); @@ -765,7 +787,7 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Er= ror **errp) } } =20 - if (start_migration) { + if (migration_has_all_channels()) { /* If it's a recovery, we're done */ if (postcopy_try_recover()) { return; diff --git a/migration/multifd.c b/migration/multifd.c index 000ca4d4ec..eeb4fb87ee 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -1164,9 +1164,14 @@ int multifd_load_setup(Error **errp) uint32_t page_count =3D MULTIFD_PACKET_SIZE / qemu_target_page_size(); uint8_t i; =20 - if (!migrate_use_multifd()) { + /* + * Return successfully if multiFD recv state is already initialised + * or multiFD is not enabled. + */ + if (multifd_recv_state || !migrate_use_multifd()) { return 0; } + if (!migrate_multi_channels_is_allowed()) { error_setg(errp, "multifd is not supported by current protocol"); return -1; @@ -1227,11 +1232,9 @@ bool multifd_recv_all_channels_created(void) =20 /* * Try to receive all multifd channels to get ready for the migration. - * - Return true and do not set @errp when correctly receiving all channel= s; - * - Return false and do not set @errp when correctly receiving the curren= t one; - * - Return false and set @errp when failing to receive the current channe= l. + * Sets @errp when failing to receive the current channel. */ -bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp) +void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) { MultiFDRecvParams *p; Error *local_err =3D NULL; @@ -1244,7 +1247,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error = **errp) "failed to receive packet" " via multifd channel %d: ", qatomic_read(&multifd_recv_state->count)); - return false; + return; } trace_multifd_recv_new_channel(id); =20 @@ -1254,7 +1257,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error = **errp) id); multifd_recv_terminate_threads(local_err); error_propagate(errp, local_err); - return false; + return; } p->c =3D ioc; object_ref(OBJECT(ioc)); @@ -1265,6 +1268,4 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error = **errp) qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, QEMU_THREAD_JOINABLE); qatomic_inc(&multifd_recv_state->count); - return qatomic_read(&multifd_recv_state->count) =3D=3D - migrate_multifd_channels(); } diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 0c55df0e52..b98e95dab0 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1538,7 +1538,7 @@ void postcopy_unregister_shared_ufd(struct PostCopyFD= *pcfd) } } =20 -bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile) +void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile) { /* * The new loading channel has its own threads, so it needs to be @@ -1547,9 +1547,6 @@ bool postcopy_preempt_new_channel(MigrationIncomingSt= ate *mis, QEMUFile *file) qemu_file_set_blocking(file, true); mis->postcopy_qemufile_dst =3D file; trace_postcopy_preempt_new_channel(); - - /* Start the migration immediately */ - return true; } =20 /* --=20 2.39.1