From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849498; cv=none; d=zohomail.com; s=zohoarc; b=lm802s4ri0X7xv/V0iLUY+NS/vKFRGt1wjEizrZCnk6xo8QnOY4BgC5YV2sqtOPi3P+AMGF1crMJYOBdXrjwqdNfv2l46B6FY16gAML3Ya2gR7AedHjQNpO/K9PNceYL6j7SST59flDlgcUjJS/KOAlbY5YWMZVhvmlORTw6mkU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849498; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XIlHBRUuqaWxdpypKS8k9nPXvNlZJx/L50KFWlh9jH8=; b=H9LrmVDfr24qme+/LXAjR7z2zgRhjU26p0/ysmX5AsxXWMg74toiU4bXPr+aWsntb8HNlz4q5H5LumgD+r0fwGD4vCog3xNQ0dyz/iepRMjmW6n6aGuCUqsvUsixcTmXQbTCTdHAgIOwOxAkobkvnerCfWETPECpj4NF6wH1YtU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 167584949841037.02435710467421; Wed, 8 Feb 2023 01:44:58 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgyw-00050t-P0; Wed, 08 Feb 2023 04:43:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgyu-0004zz-VX for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:13 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgyq-0005Md-8y for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:11 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-395-wKCKScPAOFq-02_YWO0DNA-1; Wed, 08 Feb 2023 04:43:03 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BEB87802314; Wed, 8 Feb 2023 09:43:02 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id AB0DF2166B2B; Wed, 8 Feb 2023 09:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849387; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XIlHBRUuqaWxdpypKS8k9nPXvNlZJx/L50KFWlh9jH8=; b=RQjU3J0PpMQOJG9nBJ+U2lFH8IMFaPXOXveFMxCz5Oji+xEvhTS148WE1vflSpR3ThaL/l KKvdOPWQw5NkLY9AFfYVuXUpl67X3FI2ASdXbRU+SVw9UoD49ZPJDW6w87R50L3hn6quKD wWftHW/IvZJsZ+phiu+uRTaBAcrArj4= X-MC-Unique: wKCKScPAOFq-02_YWO0DNA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 01/13] vdpa net: move iova tree creation from init to start Date: Wed, 8 Feb 2023 10:42:41 +0100 Message-Id: <20230208094253.702672-2-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849498926100001 Only create iova_tree if and when it is needed. The cleanup keeps being responsible of last VQ but this change allows it to merge both cleanup functions. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- net/vhost-vdpa.c | 99 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 71 insertions(+), 28 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index de5ed8ff22..a9e6c8f28e 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -178,13 +178,9 @@ err_init: static void vhost_vdpa_cleanup(NetClientState *nc) { VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, nc); - struct vhost_dev *dev =3D &s->vhost_net->dev; =20 qemu_vfree(s->cvq_cmd_out_buffer); qemu_vfree(s->status); - if (dev->vq_index + dev->nvqs =3D=3D dev->vq_index_end) { - g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_delete); - } if (s->vhost_net) { vhost_net_cleanup(s->vhost_net); g_free(s->vhost_net); @@ -234,10 +230,64 @@ static ssize_t vhost_vdpa_receive(NetClientState *nc,= const uint8_t *buf, return size; } =20 +/** From any vdpa net client, get the netclient of first queue pair */ +static VhostVDPAState *vhost_vdpa_net_first_nc_vdpa(VhostVDPAState *s) +{ + NICState *nic =3D qemu_get_nic(s->nc.peer); + NetClientState *nc0 =3D qemu_get_peer(nic->ncs, 0); + + return DO_UPCAST(VhostVDPAState, nc, nc0); +} + +static void vhost_vdpa_net_data_start_first(VhostVDPAState *s) +{ + struct vhost_vdpa *v =3D &s->vhost_vdpa; + + if (v->shadow_vqs_enabled) { + v->iova_tree =3D vhost_iova_tree_new(v->iova_range.first, + v->iova_range.last); + } +} + +static int vhost_vdpa_net_data_start(NetClientState *nc) +{ + VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, nc); + struct vhost_vdpa *v =3D &s->vhost_vdpa; + + assert(nc->info->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); + + if (v->index =3D=3D 0) { + vhost_vdpa_net_data_start_first(s); + return 0; + } + + if (v->shadow_vqs_enabled) { + VhostVDPAState *s0 =3D vhost_vdpa_net_first_nc_vdpa(s); + v->iova_tree =3D s0->vhost_vdpa.iova_tree; + } + + return 0; +} + +static void vhost_vdpa_net_client_stop(NetClientState *nc) +{ + VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, nc); + struct vhost_dev *dev; + + assert(nc->info->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); + + dev =3D s->vhost_vdpa.dev; + if (dev->vq_index + dev->nvqs =3D=3D dev->vq_index_end) { + g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_delete); + } +} + static NetClientInfo net_vhost_vdpa_info =3D { .type =3D NET_CLIENT_DRIVER_VHOST_VDPA, .size =3D sizeof(VhostVDPAState), .receive =3D vhost_vdpa_receive, + .start =3D vhost_vdpa_net_data_start, + .stop =3D vhost_vdpa_net_client_stop, .cleanup =3D vhost_vdpa_cleanup, .has_vnet_hdr =3D vhost_vdpa_has_vnet_hdr, .has_ufo =3D vhost_vdpa_has_ufo, @@ -351,7 +401,7 @@ dma_map_err: =20 static int vhost_vdpa_net_cvq_start(NetClientState *nc) { - VhostVDPAState *s; + VhostVDPAState *s, *s0; struct vhost_vdpa *v; uint64_t backend_features; int64_t cvq_group; @@ -425,6 +475,15 @@ out: return 0; } =20 + s0 =3D vhost_vdpa_net_first_nc_vdpa(s); + if (s0->vhost_vdpa.iova_tree) { + /* SVQ is already configured for all virtqueues */ + v->iova_tree =3D s0->vhost_vdpa.iova_tree; + } else { + v->iova_tree =3D vhost_iova_tree_new(v->iova_range.first, + v->iova_range.last); + } + r =3D vhost_vdpa_cvq_map_buf(&s->vhost_vdpa, s->cvq_cmd_out_buffer, vhost_vdpa_net_cvq_cmd_page_len(), false); if (unlikely(r < 0)) { @@ -449,15 +508,9 @@ static void vhost_vdpa_net_cvq_stop(NetClientState *nc) if (s->vhost_vdpa.shadow_vqs_enabled) { vhost_vdpa_cvq_unmap_buf(&s->vhost_vdpa, s->cvq_cmd_out_buffer); vhost_vdpa_cvq_unmap_buf(&s->vhost_vdpa, s->status); - if (!s->always_svq) { - /* - * If only the CVQ is shadowed we can delete this safely. - * If all the VQs are shadows this will be needed by the time = the - * device is started again to register SVQ vrings and similar. - */ - g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_dele= te); - } } + + vhost_vdpa_net_client_stop(nc); } =20 static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, size_t out_len, @@ -667,8 +720,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, int nvqs, bool is_datapath, bool svq, - struct vhost_vdpa_iova_range iova_r= ange, - VhostIOVATree *iova_tree) + struct vhost_vdpa_iova_range iova_r= ange) { NetClientState *nc =3D NULL; VhostVDPAState *s; @@ -690,7 +742,6 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, s->vhost_vdpa.shadow_vqs_enabled =3D svq; s->vhost_vdpa.iova_range =3D iova_range; s->vhost_vdpa.shadow_data =3D svq; - s->vhost_vdpa.iova_tree =3D iova_tree; if (!is_datapath) { s->cvq_cmd_out_buffer =3D qemu_memalign(qemu_real_host_page_size(), vhost_vdpa_net_cvq_cmd_page_le= n()); @@ -760,7 +811,6 @@ int net_init_vhost_vdpa(const Netdev *netdev, const cha= r *name, uint64_t features; int vdpa_device_fd; g_autofree NetClientState **ncs =3D NULL; - g_autoptr(VhostIOVATree) iova_tree =3D NULL; struct vhost_vdpa_iova_range iova_range; NetClientState *nc; int queue_pairs, r, i =3D 0, has_cvq =3D 0; @@ -812,12 +862,8 @@ int net_init_vhost_vdpa(const Netdev *netdev, const ch= ar *name, goto err; } =20 - if (opts->x_svq) { - if (!vhost_vdpa_net_valid_svq_features(features, errp)) { - goto err_svq; - } - - iova_tree =3D vhost_iova_tree_new(iova_range.first, iova_range.las= t); + if (opts->x_svq && !vhost_vdpa_net_valid_svq_features(features, errp))= { + goto err; } =20 ncs =3D g_malloc0(sizeof(*ncs) * queue_pairs); @@ -825,7 +871,7 @@ int net_init_vhost_vdpa(const Netdev *netdev, const cha= r *name, for (i =3D 0; i < queue_pairs; i++) { ncs[i] =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, vdpa_device_fd, i, 2, true, opts->x_s= vq, - iova_range, iova_tree); + iova_range); if (!ncs[i]) goto err; } @@ -833,13 +879,11 @@ int net_init_vhost_vdpa(const Netdev *netdev, const c= har *name, if (has_cvq) { nc =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, vdpa_device_fd, i, 1, false, - opts->x_svq, iova_range, iova_tree); + opts->x_svq, iova_range); if (!nc) goto err; } =20 - /* iova_tree ownership belongs to last NetClientState */ - g_steal_pointer(&iova_tree); return 0; =20 err: @@ -849,7 +893,6 @@ err: } } =20 -err_svq: qemu_close(vdpa_device_fd); =20 return -1; --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849425; cv=none; d=zohomail.com; s=zohoarc; b=S4UraQItqak1u8EcWgel3Bxw50kLLDrgM/+tDdmKt1BXWqCQFuKUcDK412DzNx6QK342LaE8LPooJ+ba8CWnvGHLp+DNRVc0qph5/kD8qWb7ZrXNd4ATQJrHmqez1jbKorIknWvAPndCgsObyHCXdCMG9VTE8I19bXdj4gPkAIE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849425; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=B0P82Kk1stpb6zZU8JsMQ6xnrrkGr7H7LKuOD9O/E50=; b=TNmFKn42WvC7KOBwQVgGfbsT2TbKXIOIUlad4XAj2iYkXHtNJXTOWspJZ0cwTr7EGj5MLdbAqBQwsd1NzhUP3G9KYLcu8X7ZSzXVHTooLiEGN0+wR/r9C+7rjCdg8m5+LbVeKKNNiJ9JPqezjbsHkJe6W1pBY1kreF7Lj53s+eo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849425111137.92747161834905; Wed, 8 Feb 2023 01:43:45 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgyw-000515-SD; Wed, 08 Feb 2023 04:43:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgyv-000500-0a for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:13 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgys-0005Qa-KQ for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:11 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-561-lbqtdhxHOgKiC8TxvGWuyA-1; Wed, 08 Feb 2023 04:43:06 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 09D8F885620; Wed, 8 Feb 2023 09:43:06 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0932A2166B29; Wed, 8 Feb 2023 09:43:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B0P82Kk1stpb6zZU8JsMQ6xnrrkGr7H7LKuOD9O/E50=; b=Q7Mh3zSNX2CLruFOKX3/SLGbvxLpqXv04UeZ/KTTo1v6otJyfaLEHFUE9JDvPoCa9EJSZM U7INoS2wP67qUJq/QznZDN5LaoTyjv7JMSi4M5+PfL7F/dWV07qloWqbPUHHqpoFvR6TqM 4r6/KCS7UkRLb8MTA2kba4zVbx7Zx4I= X-MC-Unique: lbqtdhxHOgKiC8TxvGWuyA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 02/13] vdpa: Negotiate _F_SUSPEND feature Date: Wed, 8 Feb 2023 10:42:42 +0100 Message-Id: <20230208094253.702672-3-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849426488100003 This is needed for qemu to know it can suspend the device to retrieve its status and enable SVQ with it, so all the process is transparent to the guest. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- hw/virtio/vhost-vdpa.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 542e003101..2e79fbe4b2 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -659,7 +659,8 @@ static int vhost_vdpa_set_backend_cap(struct vhost_dev = *dev) uint64_t features; uint64_t f =3D 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | - 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID; + 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID | + 0x1ULL << VHOST_BACKEND_F_SUSPEND; int r; =20 if (vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, &features)) { --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849672; cv=none; d=zohomail.com; s=zohoarc; b=XTRBbhecp+Kbf4j+linAfTENp6y49fZ0ufE32EU2nnoH5wYo/NgbtT9agu/E18VAmIavhigaKxKRnYnKNINjHmBF3Tq8/fVB9q7Ksx0ldRja+jijV4cutJjULhVSQN1c2PDAgYvLTCeOPeF4VbI3KO5iOahC7kWdZ7mk06WaW3s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849672; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GAt7GrxAIzZhsST+eZ9QJJlPIQRoMtJpdb4YqG1t62k=; b=PW6hZ1iVQPNAbQ0xfnLSZPE5TbCYz6mLSZuKeQbEdWR2qN5pW56hDbWcpArwGPmMbViOmHpuShZclapiH27yJGi7DXuPBCu7G6zH0/VEjizfYU1/fXZlj5as+C2zUr6zv0psW60hH6Lmpj1rIPCafOI05+1T2f4WFum7LL0Upx0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849672053168.25613060605951; Wed, 8 Feb 2023 01:47:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgz7-0005Di-NP; Wed, 08 Feb 2023 04:43:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgz0-00053M-IY for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:18 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgyy-0005W2-77 for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:17 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-12-4Iw3BpCfMSaqrXzIZMbGRg-1; Wed, 08 Feb 2023 04:43:10 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 45E9E802314; Wed, 8 Feb 2023 09:43:09 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id 486712166B2A; Wed, 8 Feb 2023 09:43:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GAt7GrxAIzZhsST+eZ9QJJlPIQRoMtJpdb4YqG1t62k=; b=LUHy1uEYo2shSvn2J4t/r3iQfjkk2pYJeksTovAov5E3mlFgIz93oDs89o9VnZacNQjLEU ePcPOYUeSEws7K8/sXvl5KG2jjtqVZ1IpD7GXOJyCcY/4ATsoAf+R8JfCBFosjRAinhQjQ bVJrRt+lVZzE3SvOGNO72zT//5TYWx0= X-MC-Unique: 4Iw3BpCfMSaqrXzIZMbGRg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 03/13] vdpa: add vhost_vdpa_suspend Date: Wed, 8 Feb 2023 10:42:43 +0100 Message-Id: <20230208094253.702672-4-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849672493100001 The function vhost.c:vhost_dev_stop fetches the vring base so the vq state can be migrated to other devices. However, this is unreliable in vdpa, since we didn't signal the device to suspend the queues, making the value fetched useless. Suspend the device if possible before fetching first and subsequent vring bases. Moreover, vdpa totally reset and wipes the device at the last device before fetch its vrings base, making that operation useless in the last device. This will be fixed in later patches of this series. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- hw/virtio/vhost-vdpa.c | 19 +++++++++++++++++++ hw/virtio/trace-events | 1 + 2 files changed, 20 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 2e79fbe4b2..cbbe92ffe8 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1108,6 +1108,24 @@ static void vhost_vdpa_svqs_stop(struct vhost_dev *d= ev) } } =20 +static void vhost_vdpa_suspend(struct vhost_dev *dev) +{ + struct vhost_vdpa *v =3D dev->opaque; + int r; + + if (!vhost_vdpa_first_dev(dev) || + !(dev->backend_cap & BIT_ULL(VHOST_BACKEND_F_SUSPEND))) { + return; + } + + trace_vhost_vdpa_suspend(dev); + r =3D ioctl(v->device_fd, VHOST_VDPA_SUSPEND); + if (unlikely(r)) { + error_report("Cannot suspend: %s(%d)", g_strerror(errno), errno); + /* Not aborting since we're called from stop context */ + } +} + static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) { struct vhost_vdpa *v =3D dev->opaque; @@ -1122,6 +1140,7 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev= , bool started) } vhost_vdpa_set_vring_ready(dev); } else { + vhost_vdpa_suspend(dev); vhost_vdpa_svqs_stop(dev); vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); } diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index a87c5f39a2..8f8d05cf9b 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -50,6 +50,7 @@ vhost_vdpa_set_vring_ready(void *dev) "dev: %p" vhost_vdpa_dump_config(void *dev, const char *line) "dev: %p %s" vhost_vdpa_set_config(void *dev, uint32_t offset, uint32_t size, uint32_t = flags) "dev: %p offset: %"PRIu32" size: %"PRIu32" flags: 0x%"PRIx32 vhost_vdpa_get_config(void *dev, void *config, uint32_t config_len) "dev: = %p config: %p config_len: %"PRIu32 +vhost_vdpa_suspend(void *dev) "dev: %p" vhost_vdpa_dev_start(void *dev, bool started) "dev: %p started: %d" vhost_vdpa_set_log_base(void *dev, uint64_t base, unsigned long long size,= int refcnt, int fd, void *log) "dev: %p base: 0x%"PRIx64" size: %llu refcn= t: %d fd: %d log: %p" vhost_vdpa_set_vring_addr(void *dev, unsigned int index, unsigned int flag= s, uint64_t desc_user_addr, uint64_t used_user_addr, uint64_t avail_user_ad= dr, uint64_t log_guest_addr) "dev: %p index: %u flags: 0x%x desc_user_addr:= 0x%"PRIx64" used_user_addr: 0x%"PRIx64" avail_user_addr: 0x%"PRIx64" log_g= uest_addr: 0x%"PRIx64 --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849600; cv=none; d=zohomail.com; s=zohoarc; b=SY91YBJ7Fyrr389874LJzlftlt9nNXvV7jiLBj3l5nCWQG6p+IX299DafBSpDwO0VVfgvmPkxqPEdci2IorWWv5cakDjkYI1JUWXbIuLutR0LucGE6+mgwORXSymLopWc9bhePSVyaEhrZjpWDzpo1k9+YTpQGCBxYHrqYgiJSw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849600; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=NevxBDK/JYdlvXhbSXgzQErdEIElTbJBGsHTJTN+zX0=; b=LUubb6pl6jzAvDHhIaJqx3RULf4PuAYE5tt/NBV/+8QNx6gJ7fA2wIMdbaMJ82MTYVo3+WORdOPiIvybvugMVhBM7GQPwORwWZpuZPKlYVdRLPcYok+LoiBeAOXTGyWZSgBKEo2HZL86l/BFEu+ULmjfjTuYggvudofAOYDCsog= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849600924401.4990025443851; Wed, 8 Feb 2023 01:46:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgz6-0005BV-Sa; Wed, 08 Feb 2023 04:43:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgz4-00054f-Eu for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:22 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgz2-0005ZU-86 for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:21 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-515-kjZG_jbwMiy2eSkjBs6opQ-1; Wed, 08 Feb 2023 04:43:13 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 84EDC885626; Wed, 8 Feb 2023 09:43:12 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id 841182166B29; Wed, 8 Feb 2023 09:43:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849398; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NevxBDK/JYdlvXhbSXgzQErdEIElTbJBGsHTJTN+zX0=; b=DVx0vgOuO2bMCg1ZNQ5Prb7v6Lg+YhpXCJPhXqlwKaWj67TniXYRiyIgHUvEpjvvX4ssrv TZaaDbnbNtfLv+LZJabOk6E3Kz0TD+ZtN4WX2n74y1wbIPYE/CEAaxU3ZMrBVTS2AXZiXG Y/SirRIGK/bUhKy1anL2b2jqwL829rI= X-MC-Unique: kjZG_jbwMiy2eSkjBs6opQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 04/13] vdpa: move vhost reset after get vring base Date: Wed, 8 Feb 2023 10:42:44 +0100 Message-Id: <20230208094253.702672-5-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849601490100003 The function vhost.c:vhost_dev_stop calls vhost operation vhost_dev_start(false). In the case of vdpa it totally reset and wipes the device, making the fetching of the vring base (virtqueue state) totally useless. The kernel backend does not use vhost_dev_start vhost op callback, but vhost-user do. A patch to make vhost_user_dev_start more similar to vdpa is desirable, but it can be added on top. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- include/hw/virtio/vhost-backend.h | 4 ++++ hw/virtio/vhost-vdpa.c | 22 ++++++++++++++++------ hw/virtio/vhost.c | 3 +++ 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-ba= ckend.h index c5ab49051e..ec3fbae58d 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -130,6 +130,9 @@ typedef bool (*vhost_force_iommu_op)(struct vhost_dev *= dev); =20 typedef int (*vhost_set_config_call_op)(struct vhost_dev *dev, int fd); + +typedef void (*vhost_reset_status_op)(struct vhost_dev *dev); + typedef struct VhostOps { VhostBackendType backend_type; vhost_backend_init vhost_backend_init; @@ -177,6 +180,7 @@ typedef struct VhostOps { vhost_get_device_id_op vhost_get_device_id; vhost_force_iommu_op vhost_force_iommu; vhost_set_config_call_op vhost_set_config_call; + vhost_reset_status_op vhost_reset_status; } VhostOps; =20 int vhost_backend_update_device_iotlb(struct vhost_dev *dev, diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index cbbe92ffe8..26e38a6aab 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1152,14 +1152,23 @@ static int vhost_vdpa_dev_start(struct vhost_dev *d= ev, bool started) if (started) { memory_listener_register(&v->listener, &address_space_memory); return vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); - } else { - vhost_vdpa_reset_device(dev); - vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | - VIRTIO_CONFIG_S_DRIVER); - memory_listener_unregister(&v->listener); + } =20 - return 0; + return 0; +} + +static void vhost_vdpa_reset_status(struct vhost_dev *dev) +{ + struct vhost_vdpa *v =3D dev->opaque; + + if (dev->vq_index + dev->nvqs !=3D dev->vq_index_end) { + return; } + + vhost_vdpa_reset_device(dev); + vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | + VIRTIO_CONFIG_S_DRIVER); + memory_listener_unregister(&v->listener); } =20 static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base, @@ -1346,4 +1355,5 @@ const VhostOps vdpa_ops =3D { .vhost_vq_get_addr =3D vhost_vdpa_vq_get_addr, .vhost_force_iommu =3D vhost_vdpa_force_iommu, .vhost_set_config_call =3D vhost_vdpa_set_config_call, + .vhost_reset_status =3D vhost_vdpa_reset_status, }; diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index eb8c4c378c..a266396576 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -2049,6 +2049,9 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODev= ice *vdev, bool vrings) hdev->vqs + i, hdev->vq_index + i); } + if (hdev->vhost_ops->vhost_reset_status) { + hdev->vhost_ops->vhost_reset_status(hdev); + } =20 if (vhost_dev_has_iommu(hdev)) { if (hdev->vhost_ops->vhost_set_iotlb_callback) { --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849625; cv=none; d=zohomail.com; s=zohoarc; b=j/s+Z9XyQFOcUkeCYzA5wySS5nPipablN3Zcgr/Kl5QdcdqouwvGBMrzF/fGJKteKUj+HESqfLU50jIK5vnRNjErjegZ3iEp+QPjklcMi/fcSOzcxn9am/pdmF7E+2PSYK56Lin346HHDRODySYoorehjkvIc0pZY0GgWrlXSl8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849625; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Sc6f0oTgOqC7WhrLqgiGyfEdfKfEHQtA1UndKmWU7uo=; b=D/xajWD/0DP3e2vcELhv0DyyN5J1uHlNZThjodJFW6NbAI2ewl6O9a4sArp6Nkr/tAoaexEEDimNNp3IoRe4yDsGcyBFvOd3lJvK+GrfpcS35IjhMdoiG7PtKnwIw7U7tcFioB4bQcnQyxi07qrOP/K1nAagRK5FmE/b8cd8sss= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 167584962551837.92234400131474; Wed, 8 Feb 2023 01:47:05 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgz8-0005Fd-L3; Wed, 08 Feb 2023 04:43:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgz6-00059c-B9 for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:24 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgz4-0005c5-4n for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:23 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-25-YyQkoIczM72GeOT5A9sa7Q-1; Wed, 08 Feb 2023 04:43:16 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CA7E3811E6E; Wed, 8 Feb 2023 09:43:15 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id C37D92166B29; Wed, 8 Feb 2023 09:43:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849401; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sc6f0oTgOqC7WhrLqgiGyfEdfKfEHQtA1UndKmWU7uo=; b=GsZ4ivL1owlm6YeUv4HpBFHpGdsDeR2oQ96JMclzHUv/NffxV6mml1B2JjsWZdiq7MEHaF bdTt/MI56WR+iMhJqnTbTirbdNgWxSvCJXiJytLoi30WfEhcwon/C+YCsVQkAAw3HTufcv Byk/d/k5OB9Qj3JNkU10OefI3YHT5Zc= X-MC-Unique: YyQkoIczM72GeOT5A9sa7Q-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 05/13] vdpa: rewind at get_base, not set_base Date: Wed, 8 Feb 2023 10:42:45 +0100 Message-Id: <20230208094253.702672-6-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849626375100001 At this moment it is only possible to migrate to a vdpa device running with x-svq=3Don. As a protective measure, the rewind of the inflight descriptors was done at the destination. That way if the source sent a virtqueue with inuse descriptors they are always discarded. Since this series allows to migrate also to passthrough devices with no SVQ, the right thing to do is to rewind at the source so the base of vrings are correct. Support for inflight descriptors may be added in the future. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- hw/virtio/vhost-vdpa.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 26e38a6aab..d99db0bd03 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1211,18 +1211,7 @@ static int vhost_vdpa_set_vring_base(struct vhost_de= v *dev, struct vhost_vring_state *ring) { struct vhost_vdpa *v =3D dev->opaque; - VirtQueue *vq =3D virtio_get_queue(dev->vdev, ring->index); =20 - /* - * vhost-vdpa devices does not support in-flight requests. Set all of = them - * as available. - * - * TODO: This is ok for networking, but other kinds of devices might - * have problems with these retransmissions. - */ - while (virtqueue_rewind(vq, 1)) { - continue; - } if (v->shadow_vqs_enabled) { /* * Device vring base was set at device start. SVQ base is handled = by @@ -1241,6 +1230,19 @@ static int vhost_vdpa_get_vring_base(struct vhost_de= v *dev, int ret; =20 if (v->shadow_vqs_enabled) { + VirtQueue *vq =3D virtio_get_queue(dev->vdev, ring->index); + + /* + * vhost-vdpa devices does not support in-flight requests. Set all= of + * them as available. + * + * TODO: This is ok for networking, but other kinds of devices mig= ht + * have problems with these retransmissions. + */ + while (virtqueue_rewind(vq, 1)) { + continue; + } + ring->num =3D virtio_queue_get_last_avail_idx(dev->vdev, ring->ind= ex); return 0; } --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849440; cv=none; d=zohomail.com; s=zohoarc; b=mLW/YqKtz5FFwEsCjMVhKsb6lbYZ31rpdxiEdDgOicL2b9t2omT8VAkBqTEC83WVFYEN0XEW0WaMtfqQNBKRkBuqCOXAg/ew5ITQlRiMifVAfMJoXVMO6CKPrKWAuc6Ap26JJlbp493byEmuN+9X18ZrUgKhrnfy3cTxKpZT2PQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849440; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KjV3DPcBfm8K8qFiu1yK2BdhCAwgI+FicV5cNL33ulc=; b=EPv8BAHbR+CIKH8TYdirtX1VBXg4dV1+sGlKfusgJCXjoC/MgcQ3r2qWzpkmrwcssAXQXAC5Sc5CWfJdKM8G4h6ZwJvswOiR0vKTVVp3BXnpzM4OsNojE6ZUtjLSS7GaZcCYIo3hivJEmLN/HyF60C1urH78giHcFAb9oW6DUZo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849440553177.91368437747224; Wed, 8 Feb 2023 01:44:00 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgz9-0005Gj-G7; Wed, 08 Feb 2023 04:43:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgz7-0005Da-JK for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:25 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgz6-0005e8-2N for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:25 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-312-akInSrbQNJegqt8ceVGtiA-1; Wed, 08 Feb 2023 04:43:19 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2A8F4802314; Wed, 8 Feb 2023 09:43:19 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2D4C32166B29; Wed, 8 Feb 2023 09:43:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849403; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KjV3DPcBfm8K8qFiu1yK2BdhCAwgI+FicV5cNL33ulc=; b=EF3vB9FpH4Wu5Dch9OkSwwB+2D/4GqEy49sGUWQPnC92/Zk1Y0xgBiJrPGm8buZ5J3ogff SkTFmWDkqHSq/0OzDHBA43nItssabcOUhXK0mTrDv1wjPy2HWmebou3eOvUhlx1gG8wVqO Y2tMq7x9DKMfgZA/amUF696TWIcN4BU= X-MC-Unique: akInSrbQNJegqt8ceVGtiA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 06/13] vdpa net: allow VHOST_F_LOG_ALL Date: Wed, 8 Feb 2023 10:42:46 +0100 Message-Id: <20230208094253.702672-7-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849442489100003 Since some actions move to the start function instead of init, the device features may not be the parent vdpa device's, but the one returned by vhost backend. If transition to SVQ is supported, the vhost backend will return _F_LOG_ALL to signal the device is migratable. Add VHOST_F_LOG_ALL. HW dirty page tracking can be added on top of this change if the device supports it in the future. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- net/vhost-vdpa.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index a9e6c8f28e..dd686b4514 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -98,6 +98,8 @@ static const uint64_t vdpa_svq_device_features =3D BIT_ULL(VIRTIO_NET_F_MQ) | BIT_ULL(VIRTIO_F_ANY_LAYOUT) | BIT_ULL(VIRTIO_NET_F_CTRL_MAC_ADDR) | + /* VHOST_F_LOG_ALL is exposed by SVQ */ + BIT_ULL(VHOST_F_LOG_ALL) | BIT_ULL(VIRTIO_NET_F_RSC_EXT) | BIT_ULL(VIRTIO_NET_F_STANDBY); =20 --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849506; cv=none; d=zohomail.com; s=zohoarc; b=PTMypSYfOKC+qYyXR14eWjTHXd97I9LUE4VNTuPxEw4VNJcl+P+IX2A42R44Hmdl0I8ahupYX/6t8lMVZK7J2MVtC0gLr/UoF7+yok1I6or02JWsqZdwRz0KcOzxOlDsD5sEKBgxDH0MEgDrk/S9ztGYemdU88G2uuZ55r2WFvs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849506; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cjF4XgacC8zKnOGK0TvXOE59ICgnYZ9SEDqHvfr3++I=; b=MY2Zy95n1cuZD4RDeqn2KxIDPp4Z1XYm+qJY+tfZQjRRo/zgUqsBzxoG5o/2ruwSfdzeU//Y4o/s8nmQTz2N+p00NFyNnwyKd8lyG4OjDto0PM5ITEC4/sbJsSDcGH3vr4vL8uc+WRxouZ3U4YUiQFa2B/I+cHfTXoPOXxZee7k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849506841777.4739499270263; Wed, 8 Feb 2023 01:45:06 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgzC-0005HK-EA; Wed, 08 Feb 2023 04:43:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzB-0005H8-Ga for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgz9-0005iH-SL for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:29 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-262-mL-av6vCNXye-fuAhFNjcA-1; Wed, 08 Feb 2023 04:43:23 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 665CA100F901; Wed, 8 Feb 2023 09:43:22 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id 699432166B2A; Wed, 8 Feb 2023 09:43:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849407; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cjF4XgacC8zKnOGK0TvXOE59ICgnYZ9SEDqHvfr3++I=; b=KD/LZL1Q6G2l0gFOfwK1tlihpujmvek2N0cBD1nLuDqE3ycIIAHyjJqUkq3wP8aWLfVXne WLr20yRnfqDChQALZI/3sCZ9GqrMrmPG9871N56NfC5sbCXvnCdZLN29e5juj36cWy63Va K8vRN/K5IJpiFjw//v/55NjedNroZhQ= X-MC-Unique: mL-av6vCNXye-fuAhFNjcA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 07/13] vdpa: add vdpa net migration state notifier Date: Wed, 8 Feb 2023 10:42:47 +0100 Message-Id: <20230208094253.702672-8-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849509028100003 This allows net to restart the device backend to configure SVQ on it. Ideally, these changes should not be net specific. However, the vdpa net backend is the one with enough knowledge to configure everything because of some reasons: * Queues might need to be shadowed or not depending on its kind (control vs data). * Queues need to share the same map translations (iova tree). Because of that it is cleaner to restart the whole net backend and configure again as expected, similar to how vhost-kernel moves between userspace and passthrough. If more kinds of devices need dynamic switching to SVQ we can create a callback struct like VhostOps and move most of the code there. VhostOps cannot be reused since all vdpa backend share them, and to personalize just for networking would be too heavy. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- v3: * Add TODO to use the resume operation in the future. * Use migration_in_setup and migration_has_failed instead of a complicated switch case. --- net/vhost-vdpa.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index dd686b4514..bca13f97fd 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -26,12 +26,14 @@ #include #include "standard-headers/linux/virtio_net.h" #include "monitor/monitor.h" +#include "migration/misc.h" #include "hw/virtio/vhost.h" =20 /* Todo:need to add the multiqueue support here */ typedef struct VhostVDPAState { NetClientState nc; struct vhost_vdpa vhost_vdpa; + Notifier migration_state; VHostNetState *vhost_net; =20 /* Control commands shadow buffers */ @@ -241,10 +243,79 @@ static VhostVDPAState *vhost_vdpa_net_first_nc_vdpa(V= hostVDPAState *s) return DO_UPCAST(VhostVDPAState, nc, nc0); } =20 +static void vhost_vdpa_net_log_global_enable(VhostVDPAState *s, bool enabl= e) +{ + struct vhost_vdpa *v =3D &s->vhost_vdpa; + VirtIONet *n; + VirtIODevice *vdev; + int data_queue_pairs, cvq, r; + NetClientState *peer; + + /* We are only called on the first data vqs and only if x-svq is not s= et */ + if (s->vhost_vdpa.shadow_vqs_enabled =3D=3D enable) { + return; + } + + vdev =3D v->dev->vdev; + n =3D VIRTIO_NET(vdev); + if (!n->vhost_started) { + return; + } + + data_queue_pairs =3D n->multiqueue ? n->max_queue_pairs : 1; + cvq =3D virtio_vdev_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ) ? + n->max_ncs - n->max_queue_pairs : 0; + /* + * TODO: vhost_net_stop does suspend, get_base and reset. We can be sm= arter + * in the future and resume the device if read-only operations between + * suspend and reset goes wrong. + */ + vhost_net_stop(vdev, n->nic->ncs, data_queue_pairs, cvq); + + peer =3D s->nc.peer; + for (int i =3D 0; i < data_queue_pairs + cvq; i++) { + VhostVDPAState *vdpa_state; + NetClientState *nc; + + if (i < data_queue_pairs) { + nc =3D qemu_get_peer(peer, i); + } else { + nc =3D qemu_get_peer(peer, n->max_queue_pairs); + } + + vdpa_state =3D DO_UPCAST(VhostVDPAState, nc, nc); + vdpa_state->vhost_vdpa.shadow_data =3D enable; + + if (i < data_queue_pairs) { + /* Do not override CVQ shadow_vqs_enabled */ + vdpa_state->vhost_vdpa.shadow_vqs_enabled =3D enable; + } + } + + r =3D vhost_net_start(vdev, n->nic->ncs, data_queue_pairs, cvq); + if (unlikely(r < 0)) { + error_report("unable to start vhost net: %s(%d)", g_strerror(-r), = -r); + } +} + +static void vdpa_net_migration_state_notifier(Notifier *notifier, void *da= ta) +{ + MigrationState *migration =3D data; + VhostVDPAState *s =3D container_of(notifier, VhostVDPAState, + migration_state); + + if (migration_in_setup(migration)) { + vhost_vdpa_net_log_global_enable(s, true); + } else if (migration_has_failed(migration)) { + vhost_vdpa_net_log_global_enable(s, false); + } +} + static void vhost_vdpa_net_data_start_first(VhostVDPAState *s) { struct vhost_vdpa *v =3D &s->vhost_vdpa; =20 + add_migration_state_change_notifier(&s->migration_state); if (v->shadow_vqs_enabled) { v->iova_tree =3D vhost_iova_tree_new(v->iova_range.first, v->iova_range.last); @@ -278,6 +349,10 @@ static void vhost_vdpa_net_client_stop(NetClientState = *nc) =20 assert(nc->info->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); =20 + if (s->vhost_vdpa.index =3D=3D 0) { + remove_migration_state_change_notifier(&s->migration_state); + } + dev =3D s->vhost_vdpa.dev; if (dev->vq_index + dev->nvqs =3D=3D dev->vq_index_end) { g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_delete); @@ -741,6 +816,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, s->vhost_vdpa.device_fd =3D vdpa_device_fd; s->vhost_vdpa.index =3D queue_pair_index; s->always_svq =3D svq; + s->migration_state.notify =3D vdpa_net_migration_state_notifier; s->vhost_vdpa.shadow_vqs_enabled =3D svq; s->vhost_vdpa.iova_range =3D iova_range; s->vhost_vdpa.shadow_data =3D svq; --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849605; cv=none; d=zohomail.com; s=zohoarc; b=HH3ZV4dBM7dbh1WZSHzq4EKxvQmap+fyWLkcJpyaT+QifoEXzIHNrg6u/vNfpo+WQE9LYkYJJ5HFZUes/Du/BELJYdBUnpe5S1sv0wSvML1G8q2/mBBbH8qxPKBQ6cSZ0yjmtB/9695KTh12ko/XkQS2efP0c0gD2BpHbe7l2Ko= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849605; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wkku9aH4T9JQtO9QmdQbLIfMKt6EPaHeWgWAoXaYqTk=; b=JM5kpVxS3cTQovnxQgvDZRnyrdt9KFcaN/unmdoXlp9RW/rHR43EZCBXG6m59f82MUtXIZ7Jkot2g5lUEfUAhfhk93zdit+NA/ccP1O+QpE++UNimULF77fy0uc80jhDu6bG4dFTYAYk3cVimaf4ts9EH2UwKLEIoeunqiecwFs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849605360237.4138572794144; Wed, 8 Feb 2023 01:46:45 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgzG-0005IW-Ci; Wed, 08 Feb 2023 04:43:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzD-0005I6-Pk for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:31 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzC-0005kn-Al for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:31 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-131-NDZ2oySOMveDVwJosrRhog-1; Wed, 08 Feb 2023 04:43:26 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AAB218027FD; Wed, 8 Feb 2023 09:43:25 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id A60F02166B29; Wed, 8 Feb 2023 09:43:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wkku9aH4T9JQtO9QmdQbLIfMKt6EPaHeWgWAoXaYqTk=; b=PtQwsc7NgaG2m05FYXLM7YdksSVoUPL373VM4JDDgEMXQ5b7SGGQx/mjnUxpu2V+likTXg TCrVPf0TRqxjdqSL1oIwhhbE4uUAHPZe0HM/elL+fzWEQ97m+2gfZa2CG0k4Mvl2jyWrsv pYmm0FdY/cPFUit+npQWYMNCgObkUpg= X-MC-Unique: NDZ2oySOMveDVwJosrRhog-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 08/13] vdpa: disable RAM block discard only for the first device Date: Wed, 8 Feb 2023 10:42:48 +0100 Message-Id: <20230208094253.702672-9-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849605620100001 Although it does not make a big difference, its more correct and simplifies the cleanup path in subsequent patches. Move ram_block_discard_disable(false) call to the top of vhost_vdpa_cleanup because: * We cannot use vhost_vdpa_first_dev after dev->opaque =3D NULL assignment. * Improve the stack order in cleanup: since it is the last action taken in init, it should be the first at cleanup. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- hw/virtio/vhost-vdpa.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index d99db0bd03..84a6b9690b 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -431,16 +431,6 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void= *opaque, Error **errp) trace_vhost_vdpa_init(dev, opaque); int ret; =20 - /* - * Similar to VFIO, we end up pinning all guest memory and have to - * disable discarding of RAM. - */ - ret =3D ram_block_discard_disable(true); - if (ret) { - error_report("Cannot set discarding of RAM broken"); - return ret; - } - v =3D opaque; v->dev =3D dev; dev->opaque =3D opaque ; @@ -452,6 +442,16 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void= *opaque, Error **errp) return 0; } =20 + /* + * Similar to VFIO, we end up pinning all guest memory and have to + * disable discarding of RAM. + */ + ret =3D ram_block_discard_disable(true); + if (ret) { + error_report("Cannot set discarding of RAM broken"); + return ret; + } + vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); =20 @@ -577,12 +577,15 @@ static int vhost_vdpa_cleanup(struct vhost_dev *dev) assert(dev->vhost_ops->backend_type =3D=3D VHOST_BACKEND_TYPE_VDPA); v =3D dev->opaque; trace_vhost_vdpa_cleanup(dev, v); + if (vhost_vdpa_first_dev(dev)) { + ram_block_discard_disable(false); + } + vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); memory_listener_unregister(&v->listener); vhost_vdpa_svq_cleanup(dev); =20 dev->opaque =3D NULL; - ram_block_discard_disable(false); =20 return 0; } --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849476; cv=none; d=zohomail.com; s=zohoarc; b=RfwgShhGQx1j1vhu/EO8swLNQuZlgUnT2k+ItvR17iWmdgxTRZ+Tth8nScsdsXr6dGZ3lCYGIlRJhPMX6cgSJf3VXuRkj1FeJIQk42rldEB3kK6ACNdgPC4Ipb8H5UwP8ZJMR5X8tUOZ4JA+9at/9Cg3Fq1wdMcrPjT0/hW/fwU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849476; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Cqdlab7l8nHdwiCXObkZmhi8oBObwKS48XyRXQYZ434=; b=LJVHO+S6d89nqgGvdSMnRKwCMEmB4Sk8n5q53hvl3Qq7U2sN3EqQIUHueNgoAocaWy6n2gEbhAB4ujDeMF8LvOqXKJVqTVypIZYSX/OYXAn+K/zhveC+dtvW85YzrpYTQMIhAOZNsJBqiA47GObUuUdVSMxFBamMQu8lEvBJfbY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16758494763317.839137682067758; Wed, 8 Feb 2023 01:44:36 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgzJ-0005J0-5Z; Wed, 08 Feb 2023 04:43:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzH-0005Io-3z for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:35 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzF-0005nw-J9 for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:34 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-447-kvLAOZfENSSAqxAecCBmLA-1; Wed, 08 Feb 2023 04:43:29 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 011342800495; Wed, 8 Feb 2023 09:43:29 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id E92BE2166B29; Wed, 8 Feb 2023 09:43:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849412; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cqdlab7l8nHdwiCXObkZmhi8oBObwKS48XyRXQYZ434=; b=XQgB3gSTAVgY5VMudRRN7/wrxPHch7IkNdZgtcT7g8mLQafDTu/flmL4T3b1aFWnGyFTVP bRfPiMQp32FjhmV5m/ejhIonNay67J0ZtgiYBGV+eqLuCj9/FI6tX+7jsaeaNNiO+PwLS+ HmbglY63IMUwKTyWPxJ3Vgxig/nZZ8g= X-MC-Unique: kvLAOZfENSSAqxAecCBmLA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 09/13] vdpa net: block migration if the device has CVQ Date: Wed, 8 Feb 2023 10:42:49 +0100 Message-Id: <20230208094253.702672-10-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849477254100001 Devices with CVQ needs to migrate state beyond vq state. Leaving this to future series. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- net/vhost-vdpa.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index bca13f97fd..309861e56c 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -955,11 +955,17 @@ int net_init_vhost_vdpa(const Netdev *netdev, const c= har *name, } =20 if (has_cvq) { + VhostVDPAState *s; + nc =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, vdpa_device_fd, i, 1, false, opts->x_svq, iova_range); if (!nc) goto err; + + s =3D DO_UPCAST(VhostVDPAState, nc, nc); + error_setg(&s->vhost_vdpa.dev->migration_blocker, + "net vdpa cannot migrate with MQ feature"); } =20 return 0; --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849666; cv=none; d=zohomail.com; s=zohoarc; b=lAgZml/DqtHjk9H8HQyoX/l7AMxB+jjfYkXk16Vfcwf0M1v07QB9v5LwUxPRiqhfdX5mtaOHiDariMGM2dSadf4CtqJnctWw3esDA4MzSTqxuBjg30rfhvKUfH+2WYi4Ah3Kfr9+Bq/RMKYlra/hbBYc+zC2TyD8pEEVHkwjiH4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849666; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PQJuvt6i7H5KtDKMtl36daFtmGk30qd1bEA7/xzw5GU=; b=bpbZDJvaUA//M24OdGUd9807wGG7gCxe9otmyvgEA37385iqEeydgFSuC0iah02IKm3HuFBxV7vVsLph4uKGDdVMOaDzwOueS2FSLp6hgXTCYmna5re9D/oXLjoymSkxXk0hF1bN/rDMiv1Oi7rf10PtTxY0rQKGnzP/8gxV4W0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849666107809.7825379057947; Wed, 8 Feb 2023 01:47:46 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgzP-0005NJ-60; Wed, 08 Feb 2023 04:43:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzM-0005Jz-TH for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:41 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzL-0005tV-2d for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:40 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-660-navIAAeOON6LlW1Mk9JWwQ-1; Wed, 08 Feb 2023 04:43:32 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 50D248027FD; Wed, 8 Feb 2023 09:43:32 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id 42B172166B29; Wed, 8 Feb 2023 09:43:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849418; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PQJuvt6i7H5KtDKMtl36daFtmGk30qd1bEA7/xzw5GU=; b=dXeG14RTnGEIYL44HlQzhjS4i0st0kaPr3lw1pO28w3TiwSIZFytC2SXrvGX51euSsmif0 rDGVxr37BeGNHVfMnbfQNp+nxwnRkglfC0MdDZqr65/hX7PxPKhFU8d6CO2+SrAVnk7Q7V i+JtbDKjKm5/rTN9xztQx0EM+mLti40= X-MC-Unique: navIAAeOON6LlW1Mk9JWwQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 10/13] vdpa: block migration if device has unsupported features Date: Wed, 8 Feb 2023 10:42:50 +0100 Message-Id: <20230208094253.702672-11-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849666584100001 A vdpa net device must initialize with SVQ in order to be migratable at this moment, and initialization code verifies some conditions. If the device is not initialized with the x-svq parameter, it will not expose _F_LOG so the vhost subsystem will block VM migration from its initialization. Next patches change this, so we need to verify migration conditions differently. QEMU only supports a subset of net features in SVQ, and it cannot migrate state that cannot track or restore in the destination. Add a migration blocker if the device offer an unsupported feature. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- net/vhost-vdpa.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 309861e56c..a0c4d5de2c 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -952,6 +952,15 @@ int net_init_vhost_vdpa(const Netdev *netdev, const ch= ar *name, iova_range); if (!ncs[i]) goto err; + + if (i =3D=3D 0) { + VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, ncs[0]); + + if (!s->vhost_vdpa.dev->migration_blocker) { + vhost_vdpa_net_valid_svq_features(features, + &s->vhost_vdpa.dev->migration_bloc= ker); + } + } } =20 if (has_cvq) { --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849652; cv=none; d=zohomail.com; s=zohoarc; b=aTNSLS7jzL79+pwlGI5QxXqxlOgmEdP8vl3jEmP59RISrNJpXIQZB4zzxMBpZgDLpiSWx5mE8opnHLAVhCabvh0Ci+v+FORotQ2N0u3B13LO2Tx7Mra5/HczaElSDDsFq+/aqufcNk7qN8H6y/3HhVRJU2o2b4tUDHsyAOXjVko= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849652; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=caL9v9vyGcoER+hJRi66QJIxFA1GbRW9+sCrWU1yP8M=; b=OgTqPDv/la3+dSettaSRF9vpMncfdYVx51HYbgUkGJshEAYyL+3ja9aYKrX1wx+m8yytuTCFt+UFQB6s8XjNKjHjiGBjiIdgviFGwwPVzlML/KJShslIPsgEmqT9/mqaTIZEae3tyVxWgsOoN2IdgR32mMXb3VlYcMGMd01D72w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849652158979.5656085444047; Wed, 8 Feb 2023 01:47:32 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgzR-0005ft-Ok; Wed, 08 Feb 2023 04:43:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzQ-0005RG-00 for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzO-0005xH-DG for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:43 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-418-2uUgg8UtNI6qpBwqBHeDgA-1; Wed, 08 Feb 2023 04:43:36 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8E94D1C068CA; Wed, 8 Feb 2023 09:43:35 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id 900992166B29; Wed, 8 Feb 2023 09:43:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849421; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=caL9v9vyGcoER+hJRi66QJIxFA1GbRW9+sCrWU1yP8M=; b=dRZhS7RHHHquB2rdJkCMoD0jPzVpg9760WInpu1rOv+XAhXT91FYDNbxO/BBVSF8/ZjVBz do5CO5qirS2Yzqkusy1NsYRvD6SjGH30m9Xch4WL+O3JSgeu7FA9kxSAUPxci5jo2vdgVk 4SfKipAPN0wp2QemyR0gB9kIGS+9Fik= X-MC-Unique: 2uUgg8UtNI6qpBwqBHeDgA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 11/13] vdpa: block migration if dev does not have _F_SUSPEND Date: Wed, 8 Feb 2023 10:42:51 +0100 Message-Id: <20230208094253.702672-12-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849652656100001 Next patches enable devices to be migrated even if vdpa netdev has not been started with x-svq. However, not all devices are migratable, so we need to block migration if we detect that. Block vhost-vdpa device migration if it does not offer _F_SUSPEND and it has not been started with x-svq. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- hw/virtio/vhost-vdpa.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 84a6b9690b..9d30cf9b3c 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -442,6 +442,27 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void= *opaque, Error **errp) return 0; } =20 + /* + * If dev->shadow_vqs_enabled at initialization that means the device = has + * been started with x-svq=3Don, so don't block migration + */ + if (dev->migration_blocker =3D=3D NULL && !v->shadow_vqs_enabled) { + uint64_t backend_features; + + /* We don't have dev->backend_features yet */ + ret =3D vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, + &backend_features); + if (unlikely(ret)) { + error_setg_errno(errp, -ret, "Could not get backend features"); + return ret; + } + + if (!(backend_features & BIT_ULL(VHOST_BACKEND_F_SUSPEND))) { + error_setg(&dev->migration_blocker, + "vhost-vdpa backend lacks VHOST_BACKEND_F_SUSPEND feature.= "); + } + } + /* * Similar to VFIO, we end up pinning all guest memory and have to * disable discarding of RAM. --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849590; cv=none; d=zohomail.com; s=zohoarc; b=jjEeCqZYYD4zqHJOU7PtMw1kCXP7PhF9af5HTtNjENBtZLX3uKxyhCtzWGRD+/Qbz3LBHXLJNsvYZpCAeA/aSdLnunT//GcIZg48F3G2c1wmVktvHKs6NAbPJACOTNnwXg39NgG8DXDQqwpX4ZDKmJvflaCLE/1Zy9Cg7URO50U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849590; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xOEoMcM0eI54zFdouENFARTLrzVZ2tNl4VTCOZs1kkc=; b=aZsA0Ix8KOnJx8zBR6FH2ndd/WNSeJsuexAgVr/zCTxWfTJY44EAL/IZ/8rbAbmvj9O4Svt4ETj5IccpsmRFdW1gedMjnA6k1InfwbVHj3mzmhdFOLBvCTo87kQmPnoxpUmvUr3IuEPs+UgxpJ3EQ6DbJywDuqjddTI7ZoAqEqU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849590924914.4459168876593; Wed, 8 Feb 2023 01:46:30 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgzT-0005tK-Tb; Wed, 08 Feb 2023 04:43:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzS-0005ic-4w for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:46 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzQ-0005zI-JD for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:45 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-380-YGjkU-okOpyJkiE4NJqgeQ-1; Wed, 08 Feb 2023 04:43:39 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D28A0885621; Wed, 8 Feb 2023 09:43:38 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD7FB2166B29; Wed, 8 Feb 2023 09:43:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849423; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xOEoMcM0eI54zFdouENFARTLrzVZ2tNl4VTCOZs1kkc=; b=b8FlaihK+dMjtbvpWl4Z8Y4ZljHf+/8+Isk+ZSwcCvCW50epJ+UgPp7YxSppWjqrK17rA1 GXOHGo+TvcFMqIJDuJ6EOen4Q0pa2+oRJMMXFV/huHGmf/hCf8OofNockXa1Gu7guahTWl 4NzDqJS2NmwXinzafvWnRj5D5TRaYCc= X-MC-Unique: YGjkU-okOpyJkiE4NJqgeQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 12/13] vdpa: block migration if SVQ does not admit a feature Date: Wed, 8 Feb 2023 10:42:52 +0100 Message-Id: <20230208094253.702672-13-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849591501100001 Next patches enable devices to be migrated even if vdpa netdev has not been started with x-svq. However, not all devices are migratable, so we need to block migration if we detect that. Block migration if we detect the device expose a feature SVQ does not know how to work with. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- hw/virtio/vhost-vdpa.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 9d30cf9b3c..13a86a2bb1 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -460,6 +460,15 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void= *opaque, Error **errp) if (!(backend_features & BIT_ULL(VHOST_BACKEND_F_SUSPEND))) { error_setg(&dev->migration_blocker, "vhost-vdpa backend lacks VHOST_BACKEND_F_SUSPEND feature.= "); + } else { + /* We don't have dev->features yet */ + uint64_t features; + ret =3D vhost_vdpa_get_dev_features(dev, &features); + if (unlikely(ret)) { + error_setg_errno(errp, -ret, "Could not get device feature= s"); + return ret; + } + vhost_svq_valid_features(features, &dev->migration_blocker); } } =20 --=20 2.31.1 From nobody Tue May 7 08:22:55 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675849498; cv=none; d=zohomail.com; s=zohoarc; b=Ei28LHI2B3DbMa4+hjSxZHWpDKZ2FYQPABwzXPwb/kSAwvzSWBHVCcwV8bCET5hSydfNcBh2is1HtC/A7CQMch20eJUhMoCJ60Gjdfhdgp6O4zDutqmgOnEUU/gdLtjcPsqn0V871RftE1jUgyApASXTQL9D5BTItaGqoOfMSMs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675849498; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HoqHsWjR9/Smc5gmRokaBkpQTq5XubU+ikZILEovKuE=; b=D+8IpXia3nF925xwZBqP5rsw9sDZik2JfqN4XV4gmRgvOj5Fvba2ba2YUBCcDycRBgnWoqj5cjS0AknQdGRsZjvspTAKD2k3efLe246ZLBno9V0R3Sqw1zZwfDw5zaOkvocGg2f888FHZrzLlVMsONqLyNS4FnEvvIsWMHGDQv4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675849498892764.0525723653666; Wed, 8 Feb 2023 01:44:58 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPgzb-0006FW-3e; Wed, 08 Feb 2023 04:43:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzZ-00068v-9L for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:53 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPgzX-00064H-3k for qemu-devel@nongnu.org; Wed, 08 Feb 2023 04:43:52 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-639-ByU_2lPXOmOLnPyLBMyp9Q-1; Wed, 08 Feb 2023 04:43:43 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 20C3C2A5954D; Wed, 8 Feb 2023 09:43:42 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.190]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1E5682166B29; Wed, 8 Feb 2023 09:43:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675849429; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HoqHsWjR9/Smc5gmRokaBkpQTq5XubU+ikZILEovKuE=; b=OW9UVGJrhJ6R6N69C/OYhNs6/TzeseDY1xK8pWbT9drznbOeFT82iIUSZLlAamJC/xT/HC h0EHe8MgxsMY8hbCXgdUHNsekhBEz6om1DBtQ+6UFrDRqLQwuCE1/xbA/BWptiutE0lFmv D8zplaBFnu6VpnfnDhWnHCq5I2Dwtq4= X-MC-Unique: ByU_2lPXOmOLnPyLBMyp9Q-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Harpreet Singh Anand , "Gonglei (Arei)" , "Michael S. Tsirkin" , Jason Wang , Cindy Lu , alvaro.karsz@solid-run.com, Zhu Lingshan , Lei Yang , Liuxiangdong , Shannon Nelson , Parav Pandit , Gautam Dawar , Eli Cohen , Stefan Hajnoczi , Laurent Vivier , longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefano Garzarella , si-wei.liu@oracle.com Subject: [PATCH v2 13/13] vdpa: return VHOST_F_LOG_ALL in vhost-vdpa devices Date: Wed, 8 Feb 2023 10:42:53 +0100 Message-Id: <20230208094253.702672-14-eperezma@redhat.com> In-Reply-To: <20230208094253.702672-1-eperezma@redhat.com> References: <20230208094253.702672-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675849500926100007 vhost-vdpa devices can return this features now that blockers have been set in case some features are not met. Expose VHOST_F_LOG_ALL only in that case. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz Tested-by: Gautam Dawar Tested-by: Lei Yang --- hw/virtio/vhost-vdpa.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 13a86a2bb1..5fddc77c5c 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1319,10 +1319,9 @@ static int vhost_vdpa_set_vring_call(struct vhost_de= v *dev, static int vhost_vdpa_get_features(struct vhost_dev *dev, uint64_t *features) { - struct vhost_vdpa *v =3D dev->opaque; int ret =3D vhost_vdpa_get_dev_features(dev, features); =20 - if (ret =3D=3D 0 && v->shadow_vqs_enabled) { + if (ret =3D=3D 0) { /* Add SVQ logging capabilities */ *features |=3D BIT_ULL(VHOST_F_LOG_ALL); } --=20 2.31.1