From nobody Fri Apr 26 02:59:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675888180; cv=none; d=zohomail.com; s=zohoarc; b=gAhLCwnMrBi+u4lzWYOJxIyvKEMU03yU65w1Sml0cOKDJ/YUmlN1ZxOqO15S6MFX1B7RztbJuBcvdmgFWWj3a66PhaSdZSaQ8c+JTt5ZJug3b0qViTnOn4W2tFRgTOSW1A06yx61vQGAQYWUoJqGVl/T6vvRimdoOItuEw/Z9fE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675888180; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XMhNYhoIW4dKVouk/RH4lce42AUOJCI51X2hRP83pA8=; b=ECp5L9kxocU2MV97xfNryGgCiHn6+DDx5FkuIdheDwTlcCcp1PWsJ7sI1CW0121dBMk5FUu1i1MyxgffB3uS3GN+0CyWNwZmyTOY+HtK2vJyzAopd5vjJ+M2iZVvN1N1KYzfzLqQEiSy8hXd1h5xyespVTVJjri9GrMDRpDhrVU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675888180965162.79067646695523; Wed, 8 Feb 2023 12:29:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPr3L-0003SQ-Eo; Wed, 08 Feb 2023 15:28:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPr3J-0003QP-7u for qemu-devel@nongnu.org; Wed, 08 Feb 2023 15:28:25 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.145.221.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPr3G-000495-FI for qemu-devel@nongnu.org; Wed, 08 Feb 2023 15:28:24 -0500 Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-2-gDtjvFojMdCBcwHV2UAvHw-1; Wed, 08 Feb 2023 15:28:19 -0500 Received: by mail-qv1-f72.google.com with SMTP id j19-20020a056214033300b0056c11dfb0ddso2934920qvu.19 for ; Wed, 08 Feb 2023 12:28:19 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-56-70-30-145-63.dsl.bell.ca. [70.30.145.63]) by smtp.gmail.com with ESMTPSA id g4-20020a37e204000000b0070d11191e91sm12287144qki.44.2023.02.08.12.28.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Feb 2023 12:28:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675888100; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XMhNYhoIW4dKVouk/RH4lce42AUOJCI51X2hRP83pA8=; b=aonFWssPD7QxOhn0SmV+0Ax1KrZ3uduJCIEIgwoWPbNIQXTp0DrDLm1mPu/VMET7NQF/VF IJlkUVUp7h0OxKZ5zup8ggH784W/jbgzcApeIQfXsNyhy6wbFfsRiUfoBDjMrp1hw4/Mbo oIEZ18uRbIMU1zZNikNAmifOYJ2gJcQ= X-MC-Unique: gDtjvFojMdCBcwHV2UAvHw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XMhNYhoIW4dKVouk/RH4lce42AUOJCI51X2hRP83pA8=; b=WCSEKLCNs7ZWXHLWqbSAA1dPmIn8eyj3KJxzBjlivsG25D8pdPT6ovNb6o+MLagDpv qi1C7+TpGqkJQPcohaAF90JxbARHxO/QgaDJisBnYrPjSOwZaYqy7wCP0zEy1tCW1pRi 0ijoAoGQM4XsSuamiRLJ9vv+qJ5yZu/3ZsDSZk120fL5NxmrH3O7T5FkRjBYqwosD/sz gqL8gW5TObEOhIksdOV4kIteToX4RueB+CriBnZiOoZ5zYKwun5lGVdNIoBKCGD0MfRL aQA5qomO4zCs+Z631IqgkpS37mMgvmV4OLOCXKGSP35mGJwvl5+46MPns6XFh2nmFZat 2u+g== X-Gm-Message-State: AO0yUKXVGBL6Ph1DQ7Z6Q779cautTYwNcejDUxObTh0KIJ8XQpk6pU0x H3YSPibFaoCSRzDpIWiA8O2s70QuajQM0jD+KFh7dmItnyq2iCcnoKzWH82luHiQ7/IzEeIBEOc uAPAZJ/TTiLA8y209ZWvh6dTRuConNaOfvf2AHKfwFWNTagSZAPFtZcOmKomkjZaSQJgTlg== X-Received: by 2002:a05:622a:1744:b0:3b8:5dfe:c3dc with SMTP id l4-20020a05622a174400b003b85dfec3dcmr16848297qtk.3.1675888098256; Wed, 08 Feb 2023 12:28:18 -0800 (PST) X-Google-Smtp-Source: AK7set+BEQBwN3s63/RBDsyi71y9bzny/uR8YQXte+Gyp/0MSDLFfbeTc5mGWnaJT9+yl3ZlVNLwYA== X-Received: by 2002:a05:622a:1744:b0:3b8:5dfe:c3dc with SMTP id l4-20020a05622a174400b003b85dfec3dcmr16848229qtk.3.1675888097732; Wed, 08 Feb 2023 12:28:17 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Cc: Leonardo Bras Soares Passos , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , "Dr . David Alan Gilbert" , Manish Mishra Subject: [PATCH v3 1/4] migration: Rework multi-channel checks on URI Date: Wed, 8 Feb 2023 15:28:10 -0500 Message-Id: <20230208202813.1363225-2-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230208202813.1363225-1-peterx@redhat.com> References: <20230208202813.1363225-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: permerror client-ip=216.145.221.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_FAIL=0.001, SPF_HELO_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675888182446100003 Content-Type: text/plain; charset="utf-8" The whole idea of multi-channel checks was not properly done, IMHO. Currently we check multi-channel in a lot of places, but actually that's not needed because we only need to check it right after we get the URI and that should be it. If the URI check succeeded, we should never need to check it again because we must have it. If it check fails, we should fail immediately on either the qmp_migrate or qmp_migrate_incoming, instead of failingg it later after the connection established. Neither should we fail any set capabiliities like what we used to do here: 5ad15e8614 ("migration: allow enabling mutilfd for specific protocol only",= 2021-10-19) Because logically the URI will only be set later after the capability is set, so it doesn't make a lot of sense to check the URI type when setting the capability, because we're checking the cap with an old URI passed in, and that may not even be the URI we're going to use later. This patch mostly reverted all such checks for before, dropping the variable migrate_allow_multi_channels and helpers. Instead, add a common helper to check URI for multi-channels for either qmp_migrate and qmp_migrate_incoming and that should do all the proper checks. The failure will only trigger with the "migrate" or "migrate_incoming" command, or when user specified "-incoming xxx" where "xxx" is not "defer". Signed-off-by: Peter Xu --- migration/migration.c | 49 +++++++++++++++++++++------------------- migration/migration.h | 3 --- migration/multifd.c | 12 ++-------- migration/postcopy-ram.c | 6 ----- 4 files changed, 28 insertions(+), 42 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 7a14aa98d8..f242d657e8 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -184,16 +184,27 @@ static int migration_maybe_pause(MigrationState *s, int new_state); static void migrate_fd_cancel(MigrationState *s); =20 -static bool migrate_allow_multi_channels =3D true; +static bool migration_needs_multiple_sockets(void) +{ + return migrate_use_multifd() || migrate_postcopy_preempt(); +} =20 -void migrate_protocol_allow_multi_channels(bool allow) +static bool uri_supports_multi_channels(const char *uri) { - migrate_allow_multi_channels =3D allow; + return strstart(uri, "tcp:", NULL) || strstart(uri, "unix:", NULL) || + strstart(uri, "vsock:", NULL); } =20 -bool migrate_multi_channels_is_allowed(void) +static bool +migration_channels_and_uri_compatible(const char *uri, Error **errp) { - return migrate_allow_multi_channels; + if (migration_needs_multiple_sockets() && + !uri_supports_multi_channels(uri)) { + error_setg(errp, "Migration requires multi-channel URIs (e.g. tcp)= "); + return false; + } + + return true; } =20 static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) @@ -493,12 +504,15 @@ static void qemu_start_incoming_migration(const char = *uri, Error **errp) { const char *p =3D NULL; =20 - migrate_protocol_allow_multi_channels(false); /* reset it anyway */ + /* URI is not suitable for migration? */ + if (!migration_channels_and_uri_compatible(uri, errp)) { + return; + } + qapi_event_send_migration(MIGRATION_STATUS_SETUP); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { - migrate_protocol_allow_multi_channels(true); socket_start_incoming_migration(p ? p : uri, errp); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { @@ -723,11 +737,6 @@ void migration_fd_process_incoming(QEMUFile *f, Error = **errp) migration_incoming_process(); } =20 -static bool migration_needs_multiple_sockets(void) -{ - return migrate_use_multifd() || migrate_postcopy_preempt(); -} - void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); @@ -1378,15 +1387,6 @@ static bool migrate_caps_check(bool *cap_list, } #endif =20 - - /* incoming side only */ - if (runstate_check(RUN_STATE_INMIGRATE) && - !migrate_multi_channels_is_allowed() && - cap_list[MIGRATION_CAPABILITY_MULTIFD]) { - error_setg(errp, "multifd is not supported by current protocol"); - return false; - } - if (cap_list[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT]) { if (!cap_list[MIGRATION_CAPABILITY_POSTCOPY_RAM]) { error_setg(errp, "Postcopy preempt requires postcopy-ram"); @@ -2471,6 +2471,11 @@ void qmp_migrate(const char *uri, bool has_blk, bool= blk, MigrationState *s =3D migrate_get_current(); const char *p =3D NULL; =20 + /* URI is not suitable for migration? */ + if (!migration_channels_and_uri_compatible(uri, errp)) { + return; + } + if (!migrate_prepare(s, has_blk && blk, has_inc && inc, has_resume && resume, errp)) { /* Error detected, put into errp */ @@ -2483,11 +2488,9 @@ void qmp_migrate(const char *uri, bool has_blk, bool= blk, } } =20 - migrate_protocol_allow_multi_channels(false); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { - migrate_protocol_allow_multi_channels(true); socket_start_outgoing_migration(s, p ? p : uri, &local_err); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { diff --git a/migration/migration.h b/migration/migration.h index 66511ce532..c351872360 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -474,7 +474,4 @@ void migration_cancel(const Error *error); void populate_vfio_info(MigrationInfo *info); void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page); =20 -bool migrate_multi_channels_is_allowed(void); -void migrate_protocol_allow_multi_channels(bool allow); - #endif diff --git a/migration/multifd.c b/migration/multifd.c index b7ad7002e0..a1141bf5dd 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -516,7 +516,7 @@ void multifd_save_cleanup(void) { int i; =20 - if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { + if (!migrate_use_multifd()) { return; } multifd_send_terminate_threads(NULL); @@ -918,10 +918,6 @@ int multifd_save_setup(Error **errp) if (!migrate_use_multifd()) { return 0; } - if (!migrate_multi_channels_is_allowed()) { - error_setg(errp, "multifd is not supported by current protocol"); - return -1; - } =20 thread_count =3D migrate_multifd_channels(); multifd_send_state =3D g_malloc0(sizeof(*multifd_send_state)); @@ -1026,7 +1022,7 @@ int multifd_load_cleanup(Error **errp) { int i; =20 - if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { + if (!migrate_use_multifd()) { return 0; } multifd_recv_terminate_threads(NULL); @@ -1180,10 +1176,6 @@ int multifd_load_setup(Error **errp) return 0; } =20 - if (!migrate_multi_channels_is_allowed()) { - error_setg(errp, "multifd is not supported by current protocol"); - return -1; - } thread_count =3D migrate_multifd_channels(); multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 53299b7a5e..9a9d0ecf49 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1635,12 +1635,6 @@ int postcopy_preempt_setup(MigrationState *s, Error = **errp) return 0; } =20 - if (!migrate_multi_channels_is_allowed()) { - error_setg(errp, "Postcopy preempt is not supported as current " - "migration stream does not support multi-channels."); - return -1; - } - /* Kick an async task to connect */ socket_send_channel_create(postcopy_preempt_send_channel_new, s); =20 --=20 2.37.3 From nobody Fri Apr 26 02:59:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675888190; cv=none; d=zohomail.com; s=zohoarc; b=Bj2POLfoHqpRPbN/Vx3WtjdNrCo7SwchgaMd5tfvIIAwEsqN7+2HOTsF/Ucx2knLjQvND+XMOjDGIWk83uuW+xHc88o7/iK0yTl0YIfsspA3+iV5FjbcP6hUZj20GC3/u5TBB73QdfM5pz9wygxrBpxQ/inLaCSSmfj31CACHhM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675888190; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kL5OJw/kTWW9OVoIT/FKhECSOQQPWNF0MruWa3VEjgY=; b=X5dMgLeTOf60fIGdonwyzyetnIj56QVRZUaHEB2ukQlC2JiaDQjLpRDiGFI3UKc/mRLheiAjq2HxyNmsgAWE1ny06DLUfg2XdpnOvwjAmhZ6hwwTMPQKUYsB5kb81ovJt1qluxahQqE00o/Rcjm61HixIFVq2Xwn76PcK6u/N2U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 167588819058724.99280699327676; Wed, 8 Feb 2023 12:29:50 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPr3O-0003TG-48; Wed, 08 Feb 2023 15:28:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPr3M-0003Sa-Sz for qemu-devel@nongnu.org; Wed, 08 Feb 2023 15:28:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPr3L-0004AG-7H for qemu-devel@nongnu.org; Wed, 08 Feb 2023 15:28:28 -0500 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-421-ePcC67iVNPKG0QfzNkabYQ-1; Wed, 08 Feb 2023 15:28:21 -0500 Received: by mail-qt1-f199.google.com with SMTP id s4-20020ac85284000000b003b849aa2cd6so11372287qtn.15 for ; Wed, 08 Feb 2023 12:28:21 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-56-70-30-145-63.dsl.bell.ca. [70.30.145.63]) by smtp.gmail.com with ESMTPSA id g4-20020a37e204000000b0070d11191e91sm12287144qki.44.2023.02.08.12.28.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Feb 2023 12:28:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675888106; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kL5OJw/kTWW9OVoIT/FKhECSOQQPWNF0MruWa3VEjgY=; b=I65AZgCEE0VpMWv459eipMpSjCbT6Gygbr2o0qk7STzPo1937ltBriWQqlm5rMqgv0oLGL swU+ofCVnfqq1S8Kp+HV3sTSO8rhyghC+dAjjIDkr6Ip0xVzYGQ+zHYfdCluvD58tWy9mv kz3Rh8y/x1QlXuSA2GpymySQ+4mMXM0= X-MC-Unique: ePcC67iVNPKG0QfzNkabYQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kL5OJw/kTWW9OVoIT/FKhECSOQQPWNF0MruWa3VEjgY=; b=uz0XURSKQxYGQSAi+/Ez7Zh707xJ5qwjnAdQKfH87qHTrmT0+PF29YrKHGOy7Wpbmr xxTJBmYAWnIDmnIWYtB9IyItneB++DYssSAQJpAvLWiTNcstS7oNPVvg6nWgyppCD1s+ 11pDuKeLiTRIwj3mN+Yr5JHCkCXV6aO6IIbEP3pEn7C9FrhXen+XYcFy661vkUx+ZBr8 XHtZJ4cqp5dNyFLXoyBP+T395uuUDWp4VXljUCqi7XUpTYKvSBEL66t5qiHQQe4+o3+B CAGshbQUIa4uK1maGXRxxu7GvfAyHGiBWpWLU/PU2LCJNbC3BTAEc1IN+JDpC6OiqrJy qG+w== X-Gm-Message-State: AO0yUKXcVNxrtoXQtf2qEYVjR6SRSuyQvfvhRo5fhzv2XNNrOjYh0vM1 6PO4hHnwY0Is1XC+kUwXDiTgh8tS9DPQR174qyedfT+keYfHdK/RCs4aRsdJW0SpyfkbUyVD98v SUwGg89l007Xomzl53nehvFzPvTI2Iy+23VbANMOLf6G4QSGyACkB79RoRZ73ErvLUNeECw== X-Received: by 2002:a05:622a:ce:b0:39c:da22:47b8 with SMTP id p14-20020a05622a00ce00b0039cda2247b8mr17587799qtw.1.1675888100242; Wed, 08 Feb 2023 12:28:20 -0800 (PST) X-Google-Smtp-Source: AK7set+ycU6P1l1oOcgueKik20t1yaZF5TGGU4tr6vUcLh+mzow2FBIWtR8XAkZ2Vy7dqwttigxLjg== X-Received: by 2002:a05:622a:ce:b0:39c:da22:47b8 with SMTP id p14-20020a05622a00ce00b0039cda2247b8mr17587751qtw.1.1675888099866; Wed, 08 Feb 2023 12:28:19 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Cc: Leonardo Bras Soares Passos , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , "Dr . David Alan Gilbert" , Manish Mishra Subject: [PATCH v3 2/4] migration: Cleanup postcopy_preempt_setup() Date: Wed, 8 Feb 2023 15:28:11 -0500 Message-Id: <20230208202813.1363225-3-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230208202813.1363225-1-peterx@redhat.com> References: <20230208202813.1363225-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675888192333100007 Content-Type: text/plain; charset="utf-8" Since we just dropped the only case where postcopy_preempt_setup() can return an error, it doesn't need a retval anymore because it never fails. Move the preempt check to the caller, preparing it to be used elsewhere to do nothing but as simple as kicking the async connection. Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/migration.c | 8 ++------ migration/postcopy-ram.c | 8 +------- migration/postcopy-ram.h | 2 +- 3 files changed, 4 insertions(+), 14 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index f242d657e8..fb0ecf5649 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -4347,12 +4347,8 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) } =20 /* This needs to be done before resuming a postcopy */ - if (postcopy_preempt_setup(s, &local_err)) { - error_report_err(local_err); - migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, - MIGRATION_STATUS_FAILED); - migrate_fd_cleanup(s); - return; + if (migrate_postcopy_preempt()) { + postcopy_preempt_setup(s); } =20 if (resume) { diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 9a9d0ecf49..de6d4a3fd4 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1629,16 +1629,10 @@ int postcopy_preempt_wait_channel(MigrationState *s) return s->postcopy_qemufile_src ? 0 : -1; } =20 -int postcopy_preempt_setup(MigrationState *s, Error **errp) +void postcopy_preempt_setup(MigrationState *s) { - if (!migrate_postcopy_preempt()) { - return 0; - } - /* Kick an async task to connect */ socket_send_channel_create(postcopy_preempt_send_channel_new, s); - - return 0; } =20 static void postcopy_pause_ram_fast_load(MigrationIncomingState *mis) diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 25881c4127..d5604cbcf1 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -191,7 +191,7 @@ enum PostcopyChannels { }; =20 void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); -int postcopy_preempt_setup(MigrationState *s, Error **errp); +void postcopy_preempt_setup(MigrationState *s); int postcopy_preempt_wait_channel(MigrationState *s); =20 #endif --=20 2.37.3 From nobody Fri Apr 26 02:59:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675888121; cv=none; d=zohomail.com; s=zohoarc; b=DzFDfXaYMIeBXTIBMvMyCjnn2TATfCMl+OualkHb4VPy2Xd8UUBtZW+KwMppteJDEix/Cb65UgBnZUaiLkdovTbQJLAjHb2lU/kbqx56zt44jppXgJqNq7KLXworJSbhYsCNhwSSTFvIJcVgfRio5wC2uTQb2MljxIFLe5bcUrs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675888121; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Kwk67BsCUneg2OIixwsflKflQC4BLP4eo50c6bA+NIE=; b=HueqITxdMoBSjUDx88VafJOJmTfEYJqtE7Pu/gXmyXbf27Z96RTIrH5C/5p5cpOVm45fPlKaiuZ4HyuQ9tBGP3nC1S9OrAZYsdz6k4xaPPyapatwzB+jlm0he2/Y7TUqybCPJONarUKFqiTCOouButSUiJ6yz9kO5gpJmU1Wbjk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675888121002621.5401698587164; Wed, 8 Feb 2023 12:28:41 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPr3M-0003Sc-WF; Wed, 08 Feb 2023 15:28:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPr3L-0003SS-IP for qemu-devel@nongnu.org; Wed, 08 Feb 2023 15:28:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPr3K-0004A2-46 for qemu-devel@nongnu.org; Wed, 08 Feb 2023 15:28:27 -0500 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-508-EMkrM-zwPlevhmMwEIEcYw-1; Wed, 08 Feb 2023 15:28:23 -0500 Received: by mail-qt1-f197.google.com with SMTP id hf20-20020a05622a609400b003abcad051d2so11320556qtb.12 for ; Wed, 08 Feb 2023 12:28:23 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-56-70-30-145-63.dsl.bell.ca. [70.30.145.63]) by smtp.gmail.com with ESMTPSA id g4-20020a37e204000000b0070d11191e91sm12287144qki.44.2023.02.08.12.28.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Feb 2023 12:28:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675888105; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Kwk67BsCUneg2OIixwsflKflQC4BLP4eo50c6bA+NIE=; b=ZicHir1bUYSI8LggqtP3JImJAKrZw9tJbDI3WQEtJ6LdlxMzy8fHJ3Ee+gLEfWpIv+58+o /qTq/pjkHaDNEgF9JbNAXxEkZ1x79/s15CWz/lXfcQ1GuJTeBeOIH5BcmYhGEfJu4Tb+dQ eUwJD+7xZ49uLMazPH3gCXifvN+gVIg= X-MC-Unique: EMkrM-zwPlevhmMwEIEcYw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Kwk67BsCUneg2OIixwsflKflQC4BLP4eo50c6bA+NIE=; b=pjiOSRYkvnBJXJSAJaJRzCYcPRHycByxfzd1rZc+fNwr596eAEmyvVc/yYb1UkuL4S Mq1xwUBgkkSeEpDUnpOslEXWJ1IRLVT0RokZLRfm5HNtmJK5xhhPwm6/8HqOX2JRx3cg wwWuWLNzy9Bl6Fx8ijAIeAjECxkpgRpmqyS4gmbiGWfp+E1lZUWwKlXNNacjkFbdTt03 mrMa0FL3EtE9X30PZdzIg13GFA9peuKmYoTfiG+5V6ql9Vm8NoHG8dbUI9nU+dGnfqQi mR9QRzbzekRJUdiFp9SJIISBRBA2imO9afNSfSwt1pJ3RBihE+S/pWjmVj8ma8jg8C24 26yA== X-Gm-Message-State: AO0yUKXsnGJ8WBqNBtHeJ89MxEic7B1i5vF2v29+h7E5CZcqt21p8lQr 8Fi5ccQcJeP1nyACKrfPYNmnJGBD97diHvC+QO0X7oENt5v8kuTo/kEUJbOfE/OglaHPPREAv9L vzq4QeUZSJe8IxLnQ0Ai+xbJ9xRPkRfV+9i1op9rmHys4ia+zXfI738+ZTnuNJZnbjwMwSg== X-Received: by 2002:a0c:dd0f:0:b0:539:aae2:8dc4 with SMTP id u15-20020a0cdd0f000000b00539aae28dc4mr11854645qvk.4.1675888102394; Wed, 08 Feb 2023 12:28:22 -0800 (PST) X-Google-Smtp-Source: AK7set9bnJ6cO+1DZqZy5qVrvxU+5CIr7ACY1Pm9SO0tme7S0m+QLIj3s8Z3phQM63ZmIw7AUt9WxA== X-Received: by 2002:a0c:dd0f:0:b0:539:aae2:8dc4 with SMTP id u15-20020a0cdd0f000000b00539aae28dc4mr11854595qvk.4.1675888101934; Wed, 08 Feb 2023 12:28:21 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Cc: Leonardo Bras Soares Passos , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , "Dr . David Alan Gilbert" , Manish Mishra Subject: [PATCH v3 3/4] migration: Add a semaphore to count PONGs Date: Wed, 8 Feb 2023 15:28:12 -0500 Message-Id: <20230208202813.1363225-4-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230208202813.1363225-1-peterx@redhat.com> References: <20230208202813.1363225-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675888121957100007 Content-Type: text/plain; charset="utf-8" This is mostly useless, but useful for us to know whether the main channel is correctly established without changing the migration protocol. Signed-off-by: Peter Xu --- migration/migration.c | 3 +++ migration/migration.h | 6 ++++++ 2 files changed, 9 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index fb0ecf5649..a2e362541d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3025,6 +3025,7 @@ retry: case MIG_RP_MSG_PONG: tmp32 =3D ldl_be_p(buf); trace_source_return_path_thread_pong(tmp32); + qemu_sem_post(&ms->rp_state.rp_pong_acks); break; =20 case MIG_RP_MSG_REQ_PAGES: @@ -4524,6 +4525,7 @@ static void migration_instance_finalize(Object *obj) qemu_sem_destroy(&ms->postcopy_pause_sem); qemu_sem_destroy(&ms->postcopy_pause_rp_sem); qemu_sem_destroy(&ms->rp_state.rp_sem); + qemu_sem_destroy(&ms->rp_state.rp_pong_acks); qemu_sem_destroy(&ms->postcopy_qemufile_src_sem); error_free(ms->error); } @@ -4570,6 +4572,7 @@ static void migration_instance_init(Object *obj) qemu_sem_init(&ms->postcopy_pause_sem, 0); qemu_sem_init(&ms->postcopy_pause_rp_sem, 0); qemu_sem_init(&ms->rp_state.rp_sem, 0); + qemu_sem_init(&ms->rp_state.rp_pong_acks, 0); qemu_sem_init(&ms->rate_limit_sem, 0); qemu_sem_init(&ms->wait_unplug_sem, 0); qemu_sem_init(&ms->postcopy_qemufile_src_sem, 0); diff --git a/migration/migration.h b/migration/migration.h index c351872360..4cb1cb6fa8 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -276,6 +276,12 @@ struct MigrationState { */ bool rp_thread_created; QemuSemaphore rp_sem; + /* + * We post to this when we got one PONG from dest. So far it's an + * easy way to know the main channel has successfully established + * on dest QEMU. + */ + QemuSemaphore rp_pong_acks; } rp_state; =20 double mbps; --=20 2.37.3 From nobody Fri Apr 26 02:59:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675888189; cv=none; d=zohomail.com; s=zohoarc; b=VvBtwCYD0f5PXsDdFv9zfGNInR3NK4p4OsjB4YYS6HPl2a6y/312iZkdXQqVaZt6ZXuBW4inY1AFcHYukwQNxf8fdLCnghgUZuzjA0/AUcQc3R1y8QJ9SHmUBOBBOV6cjv4JYJ9XBHHDW6RbJXT0mywf6xr1PatonG3JFtwOmSI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675888189; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=4njJNPoarE0YqEnQov+lp+qUh6ZdxhUtp12zzDlQ/CE=; b=cXfCYcOlAolS7/jCpjcaEfRUWAi2JtDhKmFBHKyTBYgPrMQ2Q48XLoG14fzVPuC6/9s6/QS0F8RW3xLg3gQOnuojFH7wX+LhMs6A68HZOvPDkaVkv6/6+1L+UL6H3HIz5ky1npHmX8M6JqNYBQ4jj68RJuP6BVmMT5mGukObeBg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675888189742366.43790609763266; Wed, 8 Feb 2023 12:29:49 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPr3Q-0003U5-Lt; Wed, 08 Feb 2023 15:28:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPr3O-0003TN-63 for qemu-devel@nongnu.org; Wed, 08 Feb 2023 15:28:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPr3M-0004Aa-B4 for qemu-devel@nongnu.org; Wed, 08 Feb 2023 15:28:29 -0500 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-526-VacBtljGNnW8wKsdtoHuAA-1; Wed, 08 Feb 2023 15:28:26 -0500 Received: by mail-qt1-f197.google.com with SMTP id hf20-20020a05622a609400b003abcad051d2so11320627qtb.12 for ; Wed, 08 Feb 2023 12:28:26 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-56-70-30-145-63.dsl.bell.ca. [70.30.145.63]) by smtp.gmail.com with ESMTPSA id g4-20020a37e204000000b0070d11191e91sm12287144qki.44.2023.02.08.12.28.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Feb 2023 12:28:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675888107; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4njJNPoarE0YqEnQov+lp+qUh6ZdxhUtp12zzDlQ/CE=; b=F++iADMN3KiLWvMbJYD1pR2qL5NAFs34aA7XR8+TQLQyZaopUlde1SOZ/4s576zSsQpUMf CoFYfNTg42IDTcRNaOwd9x4S9t3cpc6+qDS/JjBFaXCCb19kYbnzzBqoOs+MwcV5p0HfqA 6cRflrzQoZAXEwuxvO3QvmTor6ibwHY= X-MC-Unique: VacBtljGNnW8wKsdtoHuAA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4njJNPoarE0YqEnQov+lp+qUh6ZdxhUtp12zzDlQ/CE=; b=KNzNnCF9uGL3ErS0vV8Kue04gzFUa9PeD996kRj6p43r4/Pem5UZKWZmi+T48S7LDK 8Mrb6ogM/u3dywkO6WJTMmFBpYa+qW2VY6LOQyPxgJOP7QyVFJ4/nqhH7pyfkZC+q+F+ ZqlZMpw1ZKoaioSw2nVx454LjvLdX5emjsAw0SAxsaW1180OD/AtRzNXvTim5pwnTmGX o78QqhUUK4PLFLy0kZvovVr8xpNUoJTfOVQUZ2grUI83WAOyAy9KzPqfvy9lXaeTvTU+ ZdTJLlTaPxE/iWTHPVbUpJbDba9zklhLJE+wx4Tqcyys0n6fuKxgoYWtIPrtrf5cvvao p3nA== X-Gm-Message-State: AO0yUKVemkDknKHWmmFD9AzflEhUwYj/xm+xERIpDmXw6dE0i/jBI6rC IfkG3hmwn8ee/J0PTlJsSePd7/8XgjSdhsTtBZyI2HCi2W2/zOEepdZUN5bX4NOKRvMnFpWOMwR YcQbsPcBbEJzA8MnTYoQSCCBCobPBq7CH064dhWXgXRxCUYqRxhh3uvcdNWiks5LHk0zHZA== X-Received: by 2002:a0c:e014:0:b0:56c:1e50:d68f with SMTP id j20-20020a0ce014000000b0056c1e50d68fmr4443726qvk.3.1675888105044; Wed, 08 Feb 2023 12:28:25 -0800 (PST) X-Google-Smtp-Source: AK7set/BJ0iujN1yFbWXk2LrwFZTTqbuVkJ5imgaAn6nc/tSRW6LyMYO+6R/oSxkUI3l1KrPtUZQAA== X-Received: by 2002:a0c:e014:0:b0:56c:1e50:d68f with SMTP id j20-20020a0ce014000000b0056c1e50d68fmr4443686qvk.3.1675888104531; Wed, 08 Feb 2023 12:28:24 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Cc: Leonardo Bras Soares Passos , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , "Dr . David Alan Gilbert" , Manish Mishra Subject: [PATCH v3 4/4] migration: Postpone postcopy preempt channel to be after main Date: Wed, 8 Feb 2023 15:28:13 -0500 Message-Id: <20230208202813.1363225-5-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230208202813.1363225-1-peterx@redhat.com> References: <20230208202813.1363225-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675888190362100003 Content-Type: text/plain; charset="utf-8" Postcopy with preempt-mode enabled needs two channels to communicate. The order of channel establishment is not guaranteed. It can happen that the dest QEMU got the preempt channel connection request before the main channel is established, then the migration may make no progress even during precopy due to the wrong order. To fix it, create the preempt channel only if we know the main channel is established. For a general postcopy migration, we delay it until postcopy_start(), that's where we already went through some part of precopy on the main channel. To make sure dest QEMU has already established the channel, we wait until we got the first PONG received. That's something we do at the start of precopy when postcopy enabled so it's guaranteed to happen sooner or later. For a postcopy recovery, we delay it to qemu_savevm_state_resume_prepare() where we'll have round trips of data on bitmap synchronizations, which means the main channel must have been established. Signed-off-by: Peter Xu --- migration/migration.c | 72 ++++++++++++++++++++++++++++++---------- migration/migration.h | 6 ++++ migration/postcopy-ram.c | 17 ++++++++-- migration/postcopy-ram.h | 2 +- migration/savevm.c | 6 +++- 5 files changed, 82 insertions(+), 21 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index a2e362541d..a5c22e327d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -235,6 +235,8 @@ void migration_object_init(void) qemu_sem_init(¤t_incoming->postcopy_pause_sem_dst, 0); qemu_sem_init(¤t_incoming->postcopy_pause_sem_fault, 0); qemu_sem_init(¤t_incoming->postcopy_pause_sem_fast_load, 0); + qemu_sem_init(¤t_incoming->postcopy_qemufile_dst_done, 0); + qemu_mutex_init(¤t_incoming->page_request_mutex); current_incoming->page_requested =3D g_tree_new(page_request_addr_cmp); =20 @@ -737,6 +739,31 @@ void migration_fd_process_incoming(QEMUFile *f, Error = **errp) migration_incoming_process(); } =20 +/* + * Returns true when we want to start a new incoming migration process, + * false otherwise. + */ +static bool migration_should_start_incoming(bool main_channel) +{ + /* Multifd doesn't start unless all channels are established */ + if (migrate_use_multifd()) { + return migration_has_all_channels(); + } + + /* Preempt channel only starts when the main channel is created */ + if (migrate_postcopy_preempt()) { + return main_channel; + } + + /* + * For all the rest types of migration, we should only reach here when + * it's the main channel that's being created, and we should always + * proceed with this channel. + */ + assert(main_channel); + return true; +} + void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); @@ -798,7 +825,7 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Er= ror **errp) } } =20 - if (migration_has_all_channels()) { + if (migration_should_start_incoming(default_channel)) { /* If it's a recovery, we're done */ if (postcopy_try_recover()) { return; @@ -3159,6 +3186,13 @@ static int await_return_path_close_on_source(Migrati= onState *ms) return ms->rp_state.error; } =20 +static inline void +migration_wait_main_channel(MigrationState *ms) +{ + /* Wait until one PONG message received */ + qemu_sem_wait(&ms->rp_state.rp_pong_acks); +} + /* * Switch from normal iteration to postcopy * Returns non-0 on error @@ -3173,9 +3207,12 @@ static int postcopy_start(MigrationState *ms) bool restart_block =3D false; int cur_state =3D MIGRATION_STATUS_ACTIVE; =20 - if (postcopy_preempt_wait_channel(ms)) { - migrate_set_state(&ms->state, ms->state, MIGRATION_STATUS_FAILED); - return -1; + if (migrate_postcopy_preempt()) { + migration_wait_main_channel(ms); + if (postcopy_preempt_establish_channel(ms)) { + migrate_set_state(&ms->state, ms->state, MIGRATION_STATUS_FAIL= ED); + return -1; + } } =20 if (!migrate_pause_before_switchover()) { @@ -3586,6 +3623,20 @@ static int postcopy_do_resume(MigrationState *s) return ret; } =20 + /* + * If preempt is enabled, re-establish the preempt channel. Note that + * we do it after resume prepare to make sure the main channel will be + * created before the preempt channel. E.g. with weak network, the + * dest QEMU may get messed up with the preempt and main channels on + * the order of connection setup. This guarantees the correct order. + */ + ret =3D postcopy_preempt_establish_channel(s); + if (ret) { + error_report("%s: postcopy_preempt_establish_channel(): %d", + __func__, ret); + return ret; + } + /* * Last handshake with destination on the resume (destination will * switch to postcopy-active afterwards) @@ -3647,14 +3698,6 @@ static MigThrError postcopy_pause(MigrationState *s) if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { /* Woken up by a recover procedure. Give it a shot */ =20 - if (postcopy_preempt_wait_channel(s)) { - /* - * Preempt enabled, and new channel create failed; loop - * back to wait for another recovery. - */ - continue; - } - /* * Firstly, let's wake up the return path now, with a new * return path channel. @@ -4347,11 +4390,6 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) } } =20 - /* This needs to be done before resuming a postcopy */ - if (migrate_postcopy_preempt()) { - postcopy_preempt_setup(s); - } - if (resume) { /* Wakeup the main migration thread to do the recovery */ migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_PAUSED, diff --git a/migration/migration.h b/migration/migration.h index 4cb1cb6fa8..2da2f8a164 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -116,6 +116,12 @@ struct MigrationIncomingState { unsigned int postcopy_channels; /* QEMUFile for postcopy only; it'll be handled by a separate thread */ QEMUFile *postcopy_qemufile_dst; + /* + * When postcopy_qemufile_dst is properly setup, this sem is posted. + * One can wait on this semaphore to wait until the preempt channel is + * properly setup. + */ + QemuSemaphore postcopy_qemufile_dst_done; /* Postcopy priority thread is used to receive postcopy requested page= s */ QemuThread postcopy_prio_thread; bool postcopy_prio_thread_created; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index de6d4a3fd4..f54f44d899 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1197,6 +1197,11 @@ int postcopy_ram_incoming_setup(MigrationIncomingSta= te *mis) } =20 if (migrate_postcopy_preempt()) { + /* + * The preempt channel is established in asynchronous way. Wait + * for its completion. + */ + qemu_sem_wait(&mis->postcopy_qemufile_dst_done); /* * This thread needs to be created after the temp pages because * it'll fetch RAM_CHANNEL_POSTCOPY PostcopyTmpPage immediately. @@ -1544,6 +1549,7 @@ void postcopy_preempt_new_channel(MigrationIncomingSt= ate *mis, QEMUFile *file) */ qemu_file_set_blocking(file, true); mis->postcopy_qemufile_dst =3D file; + qemu_sem_post(&mis->postcopy_qemufile_dst_done); trace_postcopy_preempt_new_channel(); } =20 @@ -1612,14 +1618,21 @@ out: postcopy_preempt_send_channel_done(s, ioc, local_err); } =20 -/* Returns 0 if channel established, -1 for error. */ -int postcopy_preempt_wait_channel(MigrationState *s) +/* + * This function will kick off an async task to establish the preempt + * channel, and wait until the connection setup completed. Returns 0 if + * channel established, -1 for error. + */ +int postcopy_preempt_establish_channel(MigrationState *s) { /* If preempt not enabled, no need to wait */ if (!migrate_postcopy_preempt()) { return 0; } =20 + /* Kick off async task to establish preempt channel */ + postcopy_preempt_setup(s); + /* * We need the postcopy preempt channel to be established before * starting doing anything. diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index d5604cbcf1..b4867a32d5 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -192,6 +192,6 @@ enum PostcopyChannels { =20 void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); void postcopy_preempt_setup(MigrationState *s); -int postcopy_preempt_wait_channel(MigrationState *s); +int postcopy_preempt_establish_channel(MigrationState *s); =20 #endif diff --git a/migration/savevm.c b/migration/savevm.c index e9cf4999ad..4ca45702c2 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2200,7 +2200,11 @@ static int loadvm_postcopy_handle_resume(MigrationIn= comingState *mis) qemu_sem_post(&mis->postcopy_pause_sem_fault); =20 if (migrate_postcopy_preempt()) { - /* The channel should already be setup again; make sure of it */ + /* + * The preempt channel will be created in async manner, now let's + * wait for it and make sure it's created. + */ + qemu_sem_wait(&mis->postcopy_qemufile_dst_done); assert(mis->postcopy_qemufile_dst); /* Kick the fast ram load thread too */ qemu_sem_post(&mis->postcopy_pause_sem_fast_load); --=20 2.37.3