From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652987791; cv=none; d=zohomail.com; s=zohoarc; b=Q3EWdYlQUE6QVaYPC8yerFBMh4ZpwQG+FMCAbucrHmDY6N5uQfUij3WYRc5isWkmGMcTbhCf5H7fHEq5ooEpk5lyZf7Gaba2NbGz+MrckWanaRnbldL8mtfGLbmi1I2w6Ed+PcxQwUsOlzEbzrdeMcOU3Gqea/IRU4pRh1KhH1o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652987791; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=oUJfB3B7a+Bk7RvXtVLt2MSI8vTY3JUQ2f3VfP7NMqc=; b=ZMrxnrZgaw7ZQbFY3jN8+oI2w0Z5m8JeF+qUQqgOroKI8E7UMkFob2H1kttFginNYi2EzVjsHtQlUcNiAXi//a31StU3MqIhAwqfPAxOEzgznISs6SW4dBx+x8VXmEdweIc7RT30DsSZVdejJX4KoZ79By1x7wM+q5J0hmAP25o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652987791510802.8368797925411; Thu, 19 May 2022 12:16:31 -0700 (PDT) Received: from localhost ([::1]:51814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrldN-0007S4-Tt for importer2@patchew.org; Thu, 19 May 2022 15:16:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47912) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaN-0004Q7-Uk for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:50001) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaK-0006Lj-CE for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:22 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-397-lVb3cG3jNxeX3MFjNDtE7Q-1; Thu, 19 May 2022 15:13:15 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7FF0A800B21; Thu, 19 May 2022 19:13:14 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id EC90A2024CD5; Thu, 19 May 2022 19:13:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987599; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oUJfB3B7a+Bk7RvXtVLt2MSI8vTY3JUQ2f3VfP7NMqc=; b=ElZXkskIq3R5/eLgzuY3wh9sEElNC5qvcsSQPThw1fPshgehfeqvLpAeR1F9WX6LFY2QFj h5RNo7lCamntIKCUu14gqkZ0q90QoFgJqK5n8nucyDtgEhGnF4ffolGyacWNLWl8rGnWHS p9n1/nAIES9eL7NHjgGT14gwPA1WiLg= X-MC-Unique: lVb3cG3jNxeX3MFjNDtE7Q-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 01/21] virtio-net: Expose ctrl virtqueue logic Date: Thu, 19 May 2022 21:12:46 +0200 Message-Id: <20220519191306.821774-2-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652987792292100001 This allows external vhost-net devices to modify the state of the VirtIO device model once vhost-vdpa device has acknowledge the control commands. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/virtio-net.h | 4 ++ hw/net/virtio-net.c | 84 ++++++++++++++++++++-------------- 2 files changed, 53 insertions(+), 35 deletions(-) diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h index eb87032627..cd31b7f67d 100644 --- a/include/hw/virtio/virtio-net.h +++ b/include/hw/virtio/virtio-net.h @@ -218,6 +218,10 @@ struct VirtIONet { struct EBPFRSSContext ebpf_rss; }; =20 +unsigned virtio_net_handle_ctrl_iov(VirtIODevice *vdev, + const struct iovec *in_sg, size_t in_n= um, + const struct iovec *out_sg, + unsigned out_num); void virtio_net_set_netclient_name(VirtIONet *n, const char *name, const char *type); =20 diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index 7ad948ee7c..0e350154ec 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -1434,57 +1434,71 @@ static int virtio_net_handle_mq(VirtIONet *n, uint8= _t cmd, return VIRTIO_NET_OK; } =20 -static void virtio_net_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq) +unsigned virtio_net_handle_ctrl_iov(VirtIODevice *vdev, + const struct iovec *in_sg, size_t in_n= um, + const struct iovec *out_sg, + unsigned out_num) { VirtIONet *n =3D VIRTIO_NET(vdev); struct virtio_net_ctrl_hdr ctrl; virtio_net_ctrl_ack status =3D VIRTIO_NET_ERR; - VirtQueueElement *elem; size_t s; struct iovec *iov, *iov2; - unsigned int iov_cnt; + + if (iov_size(in_sg, in_num) < sizeof(status) || + iov_size(out_sg, out_num) < sizeof(ctrl)) { + virtio_error(vdev, "virtio-net ctrl missing headers"); + return 0; + } + + iov2 =3D iov =3D g_memdup2(out_sg, sizeof(struct iovec) * out_num); + s =3D iov_to_buf(iov, out_num, 0, &ctrl, sizeof(ctrl)); + iov_discard_front(&iov, &out_num, sizeof(ctrl)); + if (s !=3D sizeof(ctrl)) { + status =3D VIRTIO_NET_ERR; + } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_RX) { + status =3D virtio_net_handle_rx_mode(n, ctrl.cmd, iov, out_num); + } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_MAC) { + status =3D virtio_net_handle_mac(n, ctrl.cmd, iov, out_num); + } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_VLAN) { + status =3D virtio_net_handle_vlan_table(n, ctrl.cmd, iov, out_num); + } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_ANNOUNCE) { + status =3D virtio_net_handle_announce(n, ctrl.cmd, iov, out_num); + } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_MQ) { + status =3D virtio_net_handle_mq(n, ctrl.cmd, iov, out_num); + } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_GUEST_OFFLOADS) { + status =3D virtio_net_handle_offloads(n, ctrl.cmd, iov, out_num); + } + + s =3D iov_from_buf(in_sg, in_num, 0, &status, sizeof(status)); + assert(s =3D=3D sizeof(status)); + + g_free(iov2); + return sizeof(status); +} + +static void virtio_net_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq) +{ + VirtQueueElement *elem; =20 for (;;) { + unsigned written; elem =3D virtqueue_pop(vq, sizeof(VirtQueueElement)); if (!elem) { break; } - if (iov_size(elem->in_sg, elem->in_num) < sizeof(status) || - iov_size(elem->out_sg, elem->out_num) < sizeof(ctrl)) { - virtio_error(vdev, "virtio-net ctrl missing headers"); + + written =3D virtio_net_handle_ctrl_iov(vdev, elem->in_sg, elem->in= _num, + elem->out_sg, elem->out_num); + if (written > 0) { + virtqueue_push(vq, elem, written); + virtio_notify(vdev, vq); + g_free(elem); + } else { virtqueue_detach_element(vq, elem, 0); g_free(elem); break; } - - iov_cnt =3D elem->out_num; - iov2 =3D iov =3D g_memdup2(elem->out_sg, - sizeof(struct iovec) * elem->out_num); - s =3D iov_to_buf(iov, iov_cnt, 0, &ctrl, sizeof(ctrl)); - iov_discard_front(&iov, &iov_cnt, sizeof(ctrl)); - if (s !=3D sizeof(ctrl)) { - status =3D VIRTIO_NET_ERR; - } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_RX) { - status =3D virtio_net_handle_rx_mode(n, ctrl.cmd, iov, iov_cnt= ); - } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_MAC) { - status =3D virtio_net_handle_mac(n, ctrl.cmd, iov, iov_cnt); - } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_VLAN) { - status =3D virtio_net_handle_vlan_table(n, ctrl.cmd, iov, iov_= cnt); - } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_ANNOUNCE) { - status =3D virtio_net_handle_announce(n, ctrl.cmd, iov, iov_cn= t); - } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_MQ) { - status =3D virtio_net_handle_mq(n, ctrl.cmd, iov, iov_cnt); - } else if (ctrl.class =3D=3D VIRTIO_NET_CTRL_GUEST_OFFLOADS) { - status =3D virtio_net_handle_offloads(n, ctrl.cmd, iov, iov_cn= t); - } - - s =3D iov_from_buf(elem->in_sg, elem->in_num, 0, &status, sizeof(s= tatus)); - assert(s =3D=3D sizeof(status)); - - virtqueue_push(vq, elem, sizeof(status)); - virtio_notify(vdev, vq); - g_free(iov2); - g_free(elem); } } =20 --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988102; cv=none; d=zohomail.com; s=zohoarc; b=jkzsloCxV7Cz3TEnGD01SRnaSOmF1yW+FKiu0udwWHhnfITEPxudJ4e5mRiyrkovwgqYydbsMgGXQS7gNhHlWhOzrvcW8wL5PX6dchHE5K8mx8f0i0zjrHbMD35zAlXzceCRj5SDyDntEPcybTLDZyYSXuXXx+B3CJRAJRMcJNg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988102; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HDw786wGMRmT2D/MM8Edkyg33TAde44KyAfMWbCvmAY=; b=GbTzm80pSDBncpihw2MDYCAycHPY2PpZKiwVhtlAgWXpX2l1pJU4JD/0hcprs+qh6v6McwqmIBX+rgpRu2BAze5vL4VcuA1OY9vibwjZWJ8yHrklCejmajFsPalgpwd8IlQ7lWHTdoKqItEXzIFljClee3DK3dai5o9j2eWt91g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988102761186.93504729181893; Thu, 19 May 2022 12:21:42 -0700 (PDT) Received: from localhost ([::1]:60244 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrliP-00066y-Cu for importer2@patchew.org; Thu, 19 May 2022 15:21:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47906) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaM-0004Q6-MF for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:41668) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaK-0006Lk-9y for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:22 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-81-iIpC_JSSMNaJf6C-ooghvQ-1; Thu, 19 May 2022 15:13:18 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5F53A80A0AD; Thu, 19 May 2022 19:13:17 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id C3D122026D6A; Thu, 19 May 2022 19:13:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987599; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HDw786wGMRmT2D/MM8Edkyg33TAde44KyAfMWbCvmAY=; b=LEFtPQ69QNdXpQf9DkMH4v0r9re+zi9jfv6BlVEfkE9GnScOIzJK3Eff/tnWqqWqkmx0mQ TGoQnpvLr5CEaOoBpKWYVLTIv69rLcXYemdBWzS1nT0UclSM5/Os0umDO8AVNvIEdcL4z0 dj3098IdjHpoKE5tU+HTIOSxblKdwNE= X-MC-Unique: iIpC_JSSMNaJf6C-ooghvQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 02/21] vhost: Add custom used buffer callback Date: Thu, 19 May 2022 21:12:47 +0200 Message-Id: <20220519191306.821774-3-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988105092100001 The callback allows SVQ users to know the VirtQueue requests and responses. QEMU can use this to synchronize virtio device model state, allowing to migrate it with minimum changes to the migration code. In the case of networking, this will be used to inspect control virtqueue messages. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 16 +++++++++++++++- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-shadow-virtqueue.c | 9 ++++++++- hw/virtio/vhost-vdpa.c | 3 ++- 4 files changed, 27 insertions(+), 3 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index c132c994e9..6593f07db3 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -15,6 +15,13 @@ #include "standard-headers/linux/vhost_types.h" #include "hw/virtio/vhost-iova-tree.h" =20 +typedef void (*VirtQueueElementCallback)(VirtIODevice *vdev, + const VirtQueueElement *elem); + +typedef struct VhostShadowVirtqueueOps { + VirtQueueElementCallback used_elem_handler; +} VhostShadowVirtqueueOps; + /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { /* Shadow vring */ @@ -59,6 +66,12 @@ typedef struct VhostShadowVirtqueue { */ uint16_t *desc_next; =20 + /* Optional callbacks */ + const VhostShadowVirtqueueOps *ops; + + /* Optional custom used virtqueue element handler */ + VirtQueueElementCallback used_elem_cb; + /* Next head to expose to the device */ uint16_t shadow_avail_idx; =20 @@ -85,7 +98,8 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODev= ice *vdev, VirtQueue *vq); void vhost_svq_stop(VhostShadowVirtqueue *svq); =20 -VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree); +VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, + const VhostShadowVirtqueueOps *ops); =20 void vhost_svq_free(gpointer vq); G_DEFINE_AUTOPTR_CLEANUP_FUNC(VhostShadowVirtqueue, vhost_svq_free); diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index a29dbb3f53..f1ba46a860 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -17,6 +17,7 @@ #include "hw/virtio/vhost-iova-tree.h" #include "hw/virtio/virtio.h" #include "standard-headers/linux/vhost_types.h" +#include "hw/virtio/vhost-shadow-virtqueue.h" =20 typedef struct VhostVDPAHostNotifier { MemoryRegion mr; @@ -35,6 +36,7 @@ typedef struct vhost_vdpa { /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; GPtrArray *shadow_vqs; + const VhostShadowVirtqueueOps *shadow_vq_ops; struct vhost_dev *dev; VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX]; } VhostVDPA; diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 56c96ebd13..167db8be45 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -410,6 +410,10 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, break; } =20 + if (svq->ops && svq->ops->used_elem_handler) { + svq->ops->used_elem_handler(svq->vdev, elem); + } + if (unlikely(i >=3D svq->vring.num)) { qemu_log_mask(LOG_GUEST_ERROR, "More than %u used buffers obtained in a %u size = SVQ", @@ -607,12 +611,14 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) * shadow methods and file descriptors. * * @iova_tree: Tree to perform descriptors translations + * @ops: SVQ operations hooks * * Returns the new virtqueue or NULL. * * In case of error, reason is reported through error_report. */ -VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree) +VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, + const VhostShadowVirtqueueOps *ops) { g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); int r; @@ -634,6 +640,7 @@ VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova= _tree) event_notifier_init_fd(&svq->svq_kick, VHOST_FILE_UNBIND); event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); svq->iova_tree =3D iova_tree; + svq->ops =3D ops; return g_steal_pointer(&svq); =20 err_init_hdev_call: diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 66f054a12c..7677b337e6 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -418,7 +418,8 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, = struct vhost_vdpa *v, =20 shadow_vqs =3D g_ptr_array_new_full(hdev->nvqs, vhost_svq_free); for (unsigned n =3D 0; n < hdev->nvqs; ++n) { - g_autoptr(VhostShadowVirtqueue) svq =3D vhost_svq_new(v->iova_tree= ); + g_autoptr(VhostShadowVirtqueue) svq =3D vhost_svq_new(v->iova_tree, + v->shadow_vq_o= ps); =20 if (unlikely(!svq)) { error_setg(errp, "Cannot create svq %u", n); --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652987822; cv=none; d=zohomail.com; s=zohoarc; b=P8qn+q+uJUdy+PvxzBpzVB7arrgygmyNeSvSfU3YlgSXf5gx+hJyXbnUpNRRpktkmHmFNPxgjvwP3BnDuYB6LiqvADpjk7PxT/C37gukfG42rqAbCdQkutL/+AiovfG7cfOONxtbXkKKRHn3qRzFQpZI6/JsORcefI1SN57q6WI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652987822; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vyHZhb6zx7ZuHZ4lGtFyTqLlMaDmn0PajVBkN5g/Ha8=; b=KtYOcl+dT+S5Syrh+GoF8aUYCpsK8270qeM23w0Jok+U32ooaB1FlXVzLvOWOp+UsJkbx79NRAzNh5Bs5I9SmeRZkldDaM11qWXl+YYvtwm59a5EHNYP1h8FOLQTHu5p+kP7zpGUxix9tPF8JohQne+aPCTWmin2AdnmqCrv9cs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652987822648139.52161835225627; Thu, 19 May 2022 12:17:02 -0700 (PDT) Received: from localhost ([::1]:52186 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrldt-0007hI-CV for importer2@patchew.org; Thu, 19 May 2022 15:17:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47960) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaS-0004W2-2E for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42799) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaP-0006Mf-E5 for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:27 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-22-BBd_GkHqNWeKvtw9HC-1AQ-1; Thu, 19 May 2022 15:13:20 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3D688101AA44; Thu, 19 May 2022 19:13:20 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0E9C2026D6A; Thu, 19 May 2022 19:13:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987603; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vyHZhb6zx7ZuHZ4lGtFyTqLlMaDmn0PajVBkN5g/Ha8=; b=iiB9NB0iPjNrTIv1XOp3vUm6boxZYuQTeYoLeoL2e20Gtmr09a6LInkCT/fJ+MaV1fB1kl bMPQUmoq0g8y23HOW921HW2OCy3uiJKpEo+6VbiQX8Bi39IjjXEKyn06OTkqylgd/RFHGk R68AXRAJ1ahGOq4Olqb4ftI8YFAeql0= X-MC-Unique: BBd_GkHqNWeKvtw9HC-1AQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 03/21] vdpa: control virtqueue support on shadow virtqueue Date: Thu, 19 May 2022 21:12:48 +0200 Message-Id: <20220519191306.821774-4-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652987824408100002 Introduce the control virtqueue support for vDPA shadow virtqueue. This is needed for advanced networking features like multiqueue. To demonstrate command handling, VIRTIO_NET_F_CTRL_MACADDR and VIRTIO_NET_CTRL_MQ are implemented. If vDPA device is started with SVQ support and virtio-net driver changes MAC or the number of queues virtio-net device model will be updated with the new one. Others cvq commands could be added here straightforwardly but they have been not tested. Signed-off-by: Eugenio P=C3=A9rez --- net/vhost-vdpa.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index df1e69ee72..ef12fc284c 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -11,6 +11,7 @@ =20 #include "qemu/osdep.h" #include "clients.h" +#include "hw/virtio/virtio-net.h" #include "net/vhost_net.h" #include "net/vhost-vdpa.h" #include "hw/virtio/vhost-vdpa.h" @@ -187,6 +188,46 @@ static NetClientInfo net_vhost_vdpa_info =3D { .check_peer_type =3D vhost_vdpa_check_peer_type, }; =20 +static void vhost_vdpa_net_handle_ctrl(VirtIODevice *vdev, + const VirtQueueElement *elem) +{ + struct virtio_net_ctrl_hdr ctrl; + virtio_net_ctrl_ack status =3D VIRTIO_NET_ERR; + size_t s; + struct iovec in =3D { + .iov_base =3D &status, + .iov_len =3D sizeof(status), + }; + + s =3D iov_to_buf(elem->out_sg, elem->out_num, 0, &ctrl, sizeof(ctrl.cl= ass)); + if (s !=3D sizeof(ctrl.class)) { + return; + } + + switch (ctrl.class) { + case VIRTIO_NET_CTRL_MAC_ADDR_SET: + case VIRTIO_NET_CTRL_MQ: + break; + default: + return; + }; + + s =3D iov_to_buf(elem->in_sg, elem->in_num, 0, &status, sizeof(status)= ); + if (s !=3D sizeof(status) || status !=3D VIRTIO_NET_OK) { + return; + } + + status =3D VIRTIO_NET_ERR; + virtio_net_handle_ctrl_iov(vdev, &in, 1, elem->out_sg, elem->out_num); + if (status !=3D VIRTIO_NET_OK) { + error_report("Bad CVQ processing in model"); + } +} + +static const VhostShadowVirtqueueOps vhost_vdpa_net_svq_ops =3D { + .used_elem_handler =3D vhost_vdpa_net_handle_ctrl, +}; + static NetClientState *net_vhost_vdpa_init(NetClientState *peer, const char *device, const char *name, @@ -211,6 +252,9 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, =20 s->vhost_vdpa.device_fd =3D vdpa_device_fd; s->vhost_vdpa.index =3D queue_pair_index; + if (!is_datapath) { + s->vhost_vdpa.shadow_vq_ops =3D &vhost_vdpa_net_svq_ops; + } ret =3D vhost_vdpa_add(nc, (void *)&s->vhost_vdpa, queue_pair_index, n= vqs); if (ret) { qemu_del_net_client(nc); --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652987823; cv=none; d=zohomail.com; s=zohoarc; b=RkwdDRt2FCKKHC03VqAfSvCL07H2nP8aCPimABb6IbzvgoSW3Wr0qeAuSjzRzxBRw4fXgKOLJ4atCyr4v8DWU1CfqpNILtR4kZotZfXTnBuoq2NstJ5KIwf4jZ5HAsFIGKkMRHxuiyrs2DH+qXchp/STkE8fncohlI2fnzLIcTg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652987823; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BqG4qekA76S75JbTkpvVcYBVSGbE2v88e/0ldIgXpkg=; b=nxRbV1Rr1Qf2g3dE6FHtNcspmROkXZQRY8uLaLML06foSRCvb12ccXKmPCz1wCfkzL1TfS+z+sbX9S8RNzCWwBc7LWc9h/Vri8xyfCXdWr5DTJTuPt31sYG8cFE/2Cxznyaw/koJLvKzgbx4vtTOwvbMp6apWGadDD08FpRAIO0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 165298782352858.06458996708136; Thu, 19 May 2022 12:17:03 -0700 (PDT) Received: from localhost ([::1]:52200 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrldu-0007i4-CU for importer2@patchew.org; Thu, 19 May 2022 15:17:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47982) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaU-0004bT-9J for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:51101) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaS-0006NB-LB for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:30 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-553-5JFdUoZsPtGfHBWCbGYEHw-1; Thu, 19 May 2022 15:13:23 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 13F851C0515A; Thu, 19 May 2022 19:13:23 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F0462026D6A; Thu, 19 May 2022 19:13:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987607; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BqG4qekA76S75JbTkpvVcYBVSGbE2v88e/0ldIgXpkg=; b=GwywN/WdBc9SrxJyKhCXwGFfQXcseiKfr12C9rRRwZQFOvdOeCBtptW3KzA8BWI/EN9X1L R5wfUUJwnCLQGOzFJLEmyg4LjZrvyt+Mp1gGKber+390zEMl5meJh9XkJgIkO+6veN1d/k RgOYSMEVX73PAUJmZap/9jbqTGVuPxo= X-MC-Unique: 5JFdUoZsPtGfHBWCbGYEHw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 04/21] virtio: Make virtqueue_alloc_element non-static Date: Thu, 19 May 2022 21:12:49 +0200 Message-Id: <20220519191306.821774-5-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652987824396100001 So SVQ can allocate elements by calling it. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/virtio.h | 1 + hw/virtio/virtio.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index db1c0ddf6b..5ca29e8757 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -198,6 +198,7 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueEleme= nt *elem, unsigned int len, unsigned int idx); =20 void virtqueue_map(VirtIODevice *vdev, VirtQueueElement *elem); +void *virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned in_num= ); void *virtqueue_pop(VirtQueue *vq, size_t sz); unsigned int virtqueue_drop_all(VirtQueue *vq); void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t s= z); diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 5d607aeaa0..b0929ba86c 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -1426,7 +1426,7 @@ void virtqueue_map(VirtIODevice *vdev, VirtQueueEleme= nt *elem) fa= lse); } =20 -static void *virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned= in_num) +void *virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned in_num) { VirtQueueElement *elem; size_t in_addr_ofs =3D QEMU_ALIGN_UP(sz, __alignof__(elem->in_addr[0])= ); --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988116; cv=none; d=zohomail.com; s=zohoarc; b=ZKmsiG8VDO2Zok8Tv6UUpcOhe5jfwXMH7YYHcidM4dVXbiuWEMnrcrcwbJE4K1gMo/vJzLJqIwDtExp1l/3PLfBk5+A7EkK2SVbAapQwfr145Yj32SCdjpuKsU2vgeZVJsRS3Pk8WzqBpGBIsI8/IG9pWaHX6NlIHj/YqVoC0PQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988116; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HGLUuyN8p/qjIgHrhxRr4KqiZKh85xcQjvgst259VZI=; b=cLvGB1hUiqlybokfKjRjmWJjrhRnnSt4PC4u8quwn4zklm0z3kDQn0mZEPY0Ww4wzufYEFzIX3bGXmXMd07XYCILNBhNBCLcSTMu4dU6v81293OAddRvTHnrXLkShBg1huS7JYd2XEPkE1HovGLgZWkXEC6lCKIXmtBAsXz6+6I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988116980197.2032101473751; Thu, 19 May 2022 12:21:56 -0700 (PDT) Received: from localhost ([::1]:60690 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrlid-0006P2-Bv for importer2@patchew.org; Thu, 19 May 2022 15:21:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48034) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaW-0004hQ-S1 for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24086) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaU-0006NQ-8d for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:32 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-312-nCrZntL7MlKET406acRF6A-1; Thu, 19 May 2022 15:13:26 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E2884101AA45; Thu, 19 May 2022 19:13:25 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 564A22026D6A; Thu, 19 May 2022 19:13:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987609; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HGLUuyN8p/qjIgHrhxRr4KqiZKh85xcQjvgst259VZI=; b=Tz9h0rslrdQlyuvK3VERhWY3sUlozVShcbazzhMN/wIKnrHXx3qWqkstpsSZtdU4QcJFW9 82P0ALYEOUwfm2AmDVtgs+E3Kb+XlYqIzsYicHA1CW/uvbqzzHvcJBDpQhU9xKGFv/cgXT 6N010ICENF5p0TBKJAkja1ovs12EVhE= X-MC-Unique: nCrZntL7MlKET406acRF6A-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 05/21] vhost: Add vhost_iova_tree_find Date: Thu, 19 May 2022 21:12:50 +0200 Message-Id: <20220519191306.821774-6-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988118964100001 Just a simple wrapper so we can find DMAMap entries based on iova Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-iova-tree.h | 2 ++ hw/virtio/vhost-iova-tree.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/hw/virtio/vhost-iova-tree.h b/hw/virtio/vhost-iova-tree.h index 6a4f24e0f9..1ffcdc5b57 100644 --- a/hw/virtio/vhost-iova-tree.h +++ b/hw/virtio/vhost-iova-tree.h @@ -19,6 +19,8 @@ VhostIOVATree *vhost_iova_tree_new(uint64_t iova_first, u= int64_t iova_last); void vhost_iova_tree_delete(VhostIOVATree *iova_tree); G_DEFINE_AUTOPTR_CLEANUP_FUNC(VhostIOVATree, vhost_iova_tree_delete); =20 +const DMAMap *vhost_iova_tree_find(const VhostIOVATree *iova_tree, + const DMAMap *map); const DMAMap *vhost_iova_tree_find_iova(const VhostIOVATree *iova_tree, const DMAMap *map); int vhost_iova_tree_map_alloc(VhostIOVATree *iova_tree, DMAMap *map); diff --git a/hw/virtio/vhost-iova-tree.c b/hw/virtio/vhost-iova-tree.c index 67bf6d57ab..1a59894385 100644 --- a/hw/virtio/vhost-iova-tree.c +++ b/hw/virtio/vhost-iova-tree.c @@ -56,6 +56,20 @@ void vhost_iova_tree_delete(VhostIOVATree *iova_tree) g_free(iova_tree); } =20 +/** + * Find a mapping in the tree that matches map + * + * @iova_tree The iova tree + * @map The map + * + * Return a matching map that contains argument map or NULL + */ +const DMAMap *vhost_iova_tree_find(const VhostIOVATree *iova_tree, + const DMAMap *map) +{ + return iova_tree_find(iova_tree->iova_taddr_map, map); +} + /** * Find the IOVA address stored from a memory address * --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988112; cv=none; d=zohomail.com; s=zohoarc; b=RZ0SJJVzpfcIRVpfiTdRqrhsdcwF5mqx4t29vF7GXhndrjBacUKbRLSHZzNDQHXt7yUmkcO7NiU7MpB8HWLuljp/ngbCt8GOxC+9JOSgwKtQvkN+H4aN3Ibm3caM5NMLXuO4soJS95Y3lq5X79nIpwiN4sYTlvXAs2LMmz+65Aw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988112; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BI4Gmpc2csDpVC3zORXeXJGHONcHX8Q4bedb+BbXAdw=; b=Nlex0EMNEfgyID8BXLGsPmFulJTqf7aeNJYNU8jKJgvadScdaz4ZKFUVBmOc/qjspvyFO/1bHKt65wtvG/JyydP53mA3L7RRRJnjAeR3RjAUBy8slp/YscgVUdZI8SlaAGzhw4ynnjEb/rV5RShxq3mNFnFlsKl72Vgw2ci4ZkA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16529881119971018.6876680679793; Thu, 19 May 2022 12:21:51 -0700 (PDT) Received: from localhost ([::1]:60506 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrliY-0006HE-R3 for importer2@patchew.org; Thu, 19 May 2022 15:21:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48050) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaZ-0004oH-0j for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:46222) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaX-0006Nv-8B for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:34 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-655-ZjuOUyNrO7SQ1VnvyHFHYQ-1; Thu, 19 May 2022 15:13:29 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BB030100B8CC; Thu, 19 May 2022 19:13:28 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 318722026D6A; Thu, 19 May 2022 19:13:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BI4Gmpc2csDpVC3zORXeXJGHONcHX8Q4bedb+BbXAdw=; b=hFrzrNOWG4ALNyLeW35vbwJM0DgU5EDbUyhhnNj06H+VGdTzrN2pRqKKy89aAtaAhN5T/I f2cJhmObtVaKHG0HQnZzERCJoVdxBgDb4n9J1XcXhvmeByHAigEPvQm4pj3TtWnGUHjNFK AFxQW387lljcM/bR+jLg+HFD3gUQAik= X-MC-Unique: ZjuOUyNrO7SQ1VnvyHFHYQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 06/21] vdpa: Add map/unmap operation callback to SVQ Date: Thu, 19 May 2022 21:12:51 +0200 Message-Id: <20220519191306.821774-7-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988113150100001 Net Shadow Control VirtQueue will use them to map buffers outside of the guest's address space. These are needed for other features like indirect descriptors. They can be = used to map SVQ vrings: It is currently done outside of vhost-shadow-virtqueue.c and that is a duplication. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 21 +++++++++++++++++++-- hw/virtio/vhost-shadow-virtqueue.c | 8 +++++++- hw/virtio/vhost-vdpa.c | 20 +++++++++++++++++++- 3 files changed, 45 insertions(+), 4 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 6593f07db3..50f45153c0 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -22,6 +22,15 @@ typedef struct VhostShadowVirtqueueOps { VirtQueueElementCallback used_elem_handler; } VhostShadowVirtqueueOps; =20 +typedef int (*vhost_svq_map_op)(hwaddr iova, hwaddr size, void *vaddr, + bool readonly, void *opaque); +typedef int (*vhost_svq_unmap_op)(hwaddr iova, hwaddr size, void *opaque); + +typedef struct VhostShadowVirtqueueMapOps { + vhost_svq_map_op map; + vhost_svq_unmap_op unmap; +} VhostShadowVirtqueueMapOps; + /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { /* Shadow vring */ @@ -69,6 +78,12 @@ typedef struct VhostShadowVirtqueue { /* Optional callbacks */ const VhostShadowVirtqueueOps *ops; =20 + /* Device memory mapping callbacks */ + const VhostShadowVirtqueueMapOps *map_ops; + + /* Device memory mapping callbacks opaque */ + void *map_ops_opaque; + /* Optional custom used virtqueue element handler */ VirtQueueElementCallback used_elem_cb; =20 @@ -98,8 +113,10 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIOD= evice *vdev, VirtQueue *vq); void vhost_svq_stop(VhostShadowVirtqueue *svq); =20 -VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, - const VhostShadowVirtqueueOps *ops); +VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_map, + const VhostShadowVirtqueueOps *ops, + const VhostShadowVirtqueueMapOps *map_= ops, + void *map_ops_opaque); =20 void vhost_svq_free(gpointer vq); G_DEFINE_AUTOPTR_CLEANUP_FUNC(VhostShadowVirtqueue, vhost_svq_free); diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 167db8be45..a6a8e403ea 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -612,13 +612,17 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) * * @iova_tree: Tree to perform descriptors translations * @ops: SVQ operations hooks + * @map_ops: SVQ mapping operation hooks + * @map_ops_opaque: Opaque data to pass to mapping operations * * Returns the new virtqueue or NULL. * * In case of error, reason is reported through error_report. */ VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, - const VhostShadowVirtqueueOps *ops) + const VhostShadowVirtqueueOps *ops, + const VhostShadowVirtqueueMapOps *map_= ops, + void *map_ops_opaque) { g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); int r; @@ -641,6 +645,8 @@ VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova= _tree, event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); svq->iova_tree =3D iova_tree; svq->ops =3D ops; + svq->map_ops =3D map_ops; + svq->map_ops_opaque =3D map_ops_opaque; return g_steal_pointer(&svq); =20 err_init_hdev_call: diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 7677b337e6..e6ef944e23 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -392,6 +392,22 @@ static int vhost_vdpa_get_dev_features(struct vhost_de= v *dev, return ret; } =20 +static int vhost_vdpa_svq_map(hwaddr iova, hwaddr size, void *vaddr, + bool readonly, void *opaque) +{ + return vhost_vdpa_dma_map(opaque, iova, size, vaddr, readonly); +} + +static int vhost_vdpa_svq_unmap(hwaddr iova, hwaddr size, void *opaque) +{ + return vhost_vdpa_dma_unmap(opaque, iova, size); +} + +static const VhostShadowVirtqueueMapOps vhost_vdpa_svq_map_ops =3D { + .map =3D vhost_vdpa_svq_map, + .unmap =3D vhost_vdpa_svq_unmap, +}; + static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *= v, Error **errp) { @@ -419,7 +435,9 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, = struct vhost_vdpa *v, shadow_vqs =3D g_ptr_array_new_full(hdev->nvqs, vhost_svq_free); for (unsigned n =3D 0; n < hdev->nvqs; ++n) { g_autoptr(VhostShadowVirtqueue) svq =3D vhost_svq_new(v->iova_tree, - v->shadow_vq_o= ps); + v->shadow_vq_ops, + &vhost_vdpa_svq_map= _ops, + v); =20 if (unlikely(!svq)) { error_setg(errp, "Cannot create svq %u", n); --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988264; cv=none; d=zohomail.com; s=zohoarc; b=lhdG6gV9/6Glx0d60qhz9uXNFuqjoaQKC2gTYmR/CLynMX6ptxH6LsUyq8yz8T9G5A3EomYQMRn1yNR6bnCIduuKwNRWX/ptbdMo1sqkLIjMujic/guc8zy09Y8tItjKCN5lg8BoAM9VOZ9L7fy24cGJNQPfJsFzXSCC30bytwo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988264; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lDoKExE0vkghFNApG6+2oWwkf5wQ4E9yWA1Lvr5tAxY=; b=FiClwFR+u+3QI4JM8dXTB5ZkD8fFVJ933cKFyjwqjMO7V9KjAFOM5x3TpQo+lmDnsrtUDwCy1Pn7zp2N9MGiJmPXToRVjS01lIjy75kD5Lm6ymxp2+ogPb1Kt/FJRqAMSnB4ioipM1kkSj6p65gpi72HrFy59mCrUT7Op62uZWM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988264184232.7025597657015; Thu, 19 May 2022 12:24:24 -0700 (PDT) Received: from localhost ([::1]:40366 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrll1-0003ZS-0Z for importer2@patchew.org; Thu, 19 May 2022 15:24:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48064) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlab-0004vF-Ff for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:41889) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaZ-0006OF-Q1 for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:37 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-262-e1JYlKPyPhKA3-sGyGINNw-1; Thu, 19 May 2022 15:13:32 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9091B1C0515A; Thu, 19 May 2022 19:13:31 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 085FB2026D6A; Thu, 19 May 2022 19:13:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987615; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lDoKExE0vkghFNApG6+2oWwkf5wQ4E9yWA1Lvr5tAxY=; b=DD/TwAwkmWsOF/VLFUKLolBf3dxyMNWn8+wvF+sc55Yupgy3Yb+6b3xzyaGh8DEAxxUELO wdUnMA63FvSmUx1kSREPltca4NIXN5EUjkj1FhJd9lBrHIxpFg8FJGeTvLIUIIxS0wxv/m b0Ic0lJRY1aOM5HUofFbG7amQW7ourE= X-MC-Unique: e1JYlKPyPhKA3-sGyGINNw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 07/21] vhost: move descriptor translation to vhost_svq_vring_write_descs Date: Thu, 19 May 2022 21:12:52 +0200 Message-Id: <20220519191306.821774-8-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988265656100001 It's done for both in and out descriptors so it's better placed here. Acked-by: Jason Wang Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 38 +++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index a6a8e403ea..2d5d27d29c 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -122,17 +122,35 @@ static bool vhost_svq_translate_addr(const VhostShado= wVirtqueue *svq, return true; } =20 -static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, hwaddr *sg, - const struct iovec *iovec, size_t num, - bool more_descs, bool write) +/** + * Write descriptors to SVQ vring + * + * @svq: The shadow virtqueue + * @sg: Cache for hwaddr + * @iovec: The iovec from the guest + * @num: iovec length + * @more_descs: True if more descriptors come in the chain + * @write: True if they are in descriptors + * + * Return true if success, false otherwise and print error. + */ +static bool vhost_svq_vring_write_descs(VhostShadowVirtqueue *svq, hwaddr = *sg, + const struct iovec *iovec, size_t = num, + bool more_descs, bool write) { uint16_t i =3D svq->free_head, last =3D svq->free_head; unsigned n; uint16_t flags =3D write ? cpu_to_le16(VRING_DESC_F_WRITE) : 0; vring_desc_t *descs =3D svq->vring.desc; + bool ok; =20 if (num =3D=3D 0) { - return; + return true; + } + + ok =3D vhost_svq_translate_addr(svq, sg, iovec, num); + if (unlikely(!ok)) { + return false; } =20 for (n =3D 0; n < num; n++) { @@ -150,6 +168,7 @@ static void vhost_vring_write_descs(VhostShadowVirtqueu= e *svq, hwaddr *sg, } =20 svq->free_head =3D le16_to_cpu(svq->desc_next[last]); + return true; } =20 static bool vhost_svq_add_split(VhostShadowVirtqueue *svq, @@ -169,21 +188,18 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue = *svq, return false; } =20 - ok =3D vhost_svq_translate_addr(svq, sgs, elem->out_sg, elem->out_num); + ok =3D vhost_svq_vring_write_descs(svq, sgs, elem->out_sg, elem->out_n= um, + elem->in_num > 0, false); if (unlikely(!ok)) { return false; } - vhost_vring_write_descs(svq, sgs, elem->out_sg, elem->out_num, - elem->in_num > 0, false); - =20 - ok =3D vhost_svq_translate_addr(svq, sgs, elem->in_sg, elem->in_num); + ok =3D vhost_svq_vring_write_descs(svq, sgs, elem->in_sg, elem->in_num= , false, + true); if (unlikely(!ok)) { return false; } =20 - vhost_vring_write_descs(svq, sgs, elem->in_sg, elem->in_num, false, tr= ue); - /* * Put the entry in the available array (but don't update avail->idx u= ntil * they do sync). --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988471; cv=none; d=zohomail.com; s=zohoarc; b=BMSeLvK/G86wRbaVcM0guo255UUePc3Pldw+bB1zZEH+GgXgR4UdYJJS/ass+x3YncsvJu4rFMZc0nZKYiojsD8C2ZrV4T4AZ+ItdqeBosA2wGgwvhOjE5hzjo5I7mk5LCZ6JZUKi+3FQ9ad1TGgbfUMqMdT4bbt9N4v7WPcyTg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988471; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=NPZ407xv/ruQf+ghZ96dkzUoKc935N+ZrkmCxbdz84w=; b=HX5sSiq6LDBDo280XsVLhE6jQUI4SY1dzGoTgw7NEALA1asaAcPw3o2L+93FS6hSbfLlmkBp0aPz9UVt66pGNtUI3ooC/9fsCXxyM4Fy2flu6BssYAhm9ZpBEs+9C3uCGdtDrrRWsd8xbEawHiRUOz1JaZqN3rTPDmf6dMk3HmI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988471694971.8297488204593; Thu, 19 May 2022 12:27:51 -0700 (PDT) Received: from localhost ([::1]:48494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrloM-0000sI-MJ for importer2@patchew.org; Thu, 19 May 2022 15:27:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48112) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlag-00058t-P5 for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:26673) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlad-0006Oa-NQ for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:42 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-187-u72ts2eIO-qHZA6QAbGExQ-1; Thu, 19 May 2022 15:13:35 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 693AF801210; Thu, 19 May 2022 19:13:34 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id D3E5A2026D6A; Thu, 19 May 2022 19:13:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987619; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NPZ407xv/ruQf+ghZ96dkzUoKc935N+ZrkmCxbdz84w=; b=ErI1u7jxBn0yNpn6luBpa2R4/qaJcptiognXJ4KNUxggdIzqaZogERZI8ozaCSFFh+BXzu LQ2SEJT7lgO8UFZlbUVYy/NCGOaIJlj7k6q1RJGkybvT/v5LMw6tJ69vYsIvcgqAfZxBEx nAT0wbQUwg1z8yyrOmaTy+Cn/ASE6rY= X-MC-Unique: u72ts2eIO-qHZA6QAbGExQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 08/21] vhost: Add SVQElement Date: Thu, 19 May 2022 21:12:53 +0200 Message-Id: <20220519191306.821774-9-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988472689100001 This allows SVQ to add metadata to the different queue elements. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 8 ++++-- hw/virtio/vhost-shadow-virtqueue.c | 46 ++++++++++++++++-------------- 2 files changed, 31 insertions(+), 23 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 50f45153c0..e06ac52158 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -15,6 +15,10 @@ #include "standard-headers/linux/vhost_types.h" #include "hw/virtio/vhost-iova-tree.h" =20 +typedef struct SVQElement { + VirtQueueElement elem; +} SVQElement; + typedef void (*VirtQueueElementCallback)(VirtIODevice *vdev, const VirtQueueElement *elem); =20 @@ -64,10 +68,10 @@ typedef struct VhostShadowVirtqueue { VhostIOVATree *iova_tree; =20 /* Map for use the guest's descriptors */ - VirtQueueElement **ring_id_maps; + SVQElement **ring_id_maps; =20 /* Next VirtQueue element that guest made available */ - VirtQueueElement *next_guest_avail_elem; + SVQElement *next_guest_avail_elem; =20 /* * Backup next field for each descriptor so we can recover securely, n= ot diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 2d5d27d29c..044005ba89 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -171,9 +171,10 @@ static bool vhost_svq_vring_write_descs(VhostShadowVir= tqueue *svq, hwaddr *sg, return true; } =20 -static bool vhost_svq_add_split(VhostShadowVirtqueue *svq, - VirtQueueElement *elem, unsigned *head) +static bool vhost_svq_add_split(VhostShadowVirtqueue *svq, SVQElement *svq= _elem, + unsigned *head) { + const VirtQueueElement *elem =3D &svq_elem->elem; unsigned avail_idx; vring_avail_t *avail =3D svq->vring.avail; bool ok; @@ -222,7 +223,7 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *s= vq, * takes ownership of the element: In case of failure, it is free and the = SVQ * is considered broken. */ -static bool vhost_svq_add(VhostShadowVirtqueue *svq, VirtQueueElement *ele= m) +static bool vhost_svq_add(VhostShadowVirtqueue *svq, SVQElement *elem) { unsigned qemu_head; bool ok =3D vhost_svq_add_split(svq, elem, &qemu_head); @@ -272,19 +273,21 @@ static void vhost_handle_guest_kick(VhostShadowVirtqu= eue *svq) virtio_queue_set_notification(svq->vq, false); =20 while (true) { + SVQElement *svq_elem; VirtQueueElement *elem; bool ok; =20 if (svq->next_guest_avail_elem) { - elem =3D g_steal_pointer(&svq->next_guest_avail_elem); + svq_elem =3D g_steal_pointer(&svq->next_guest_avail_elem); } else { - elem =3D virtqueue_pop(svq->vq, sizeof(*elem)); + svq_elem =3D virtqueue_pop(svq->vq, sizeof(*svq_elem)); } =20 - if (!elem) { + if (!svq_elem) { break; } =20 + elem =3D &svq_elem->elem; if (elem->out_num + elem->in_num > vhost_svq_available_slots(s= vq)) { /* * This condition is possible since a contiguous buffer in= GPA @@ -297,11 +300,11 @@ static void vhost_handle_guest_kick(VhostShadowVirtqu= eue *svq) * queue the current guest descriptor and ignore further k= icks * until some elements are used. */ - svq->next_guest_avail_elem =3D elem; + svq->next_guest_avail_elem =3D svq_elem; return; } =20 - ok =3D vhost_svq_add(svq, elem); + ok =3D vhost_svq_add(svq, svq_elem); if (unlikely(!ok)) { /* VQ is broken, just return and ignore any other kicks */ return; @@ -368,8 +371,7 @@ static uint16_t vhost_svq_last_desc_of_chain(const Vhos= tShadowVirtqueue *svq, return i; } =20 -static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq, - uint32_t *len) +static SVQElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq, uint32_t *= len) { const vring_used_t *used =3D svq->vring.used; vring_used_elem_t used_elem; @@ -399,8 +401,8 @@ static VirtQueueElement *vhost_svq_get_buf(VhostShadowV= irtqueue *svq, return NULL; } =20 - num =3D svq->ring_id_maps[used_elem.id]->in_num + - svq->ring_id_maps[used_elem.id]->out_num; + num =3D svq->ring_id_maps[used_elem.id]->elem.in_num + + svq->ring_id_maps[used_elem.id]->elem.out_num; last_used_chain =3D vhost_svq_last_desc_of_chain(svq, num, used_elem.i= d); svq->desc_next[last_used_chain] =3D svq->free_head; svq->free_head =3D used_elem.id; @@ -421,11 +423,13 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, vhost_svq_disable_notification(svq); while (true) { uint32_t len; - g_autofree VirtQueueElement *elem =3D vhost_svq_get_buf(svq, &= len); - if (!elem) { + g_autofree SVQElement *svq_elem =3D vhost_svq_get_buf(svq, &le= n); + VirtQueueElement *elem; + if (!svq_elem) { break; } =20 + elem =3D &svq_elem->elem; if (svq->ops && svq->ops->used_elem_handler) { svq->ops->used_elem_handler(svq->vdev, elem); } @@ -580,7 +584,7 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIOD= evice *vdev, memset(svq->vring.desc, 0, driver_size); svq->vring.used =3D qemu_memalign(qemu_real_host_page_size(), device_s= ize); memset(svq->vring.used, 0, device_size); - svq->ring_id_maps =3D g_new0(VirtQueueElement *, svq->vring.num); + svq->ring_id_maps =3D g_new0(SVQElement *, svq->vring.num); svq->desc_next =3D g_new0(uint16_t, svq->vring.num); for (unsigned i =3D 0; i < svq->vring.num - 1; i++) { svq->desc_next[i] =3D cpu_to_le16(i + 1); @@ -594,7 +598,7 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIOD= evice *vdev, void vhost_svq_stop(VhostShadowVirtqueue *svq) { event_notifier_set_handler(&svq->svq_kick, NULL); - g_autofree VirtQueueElement *next_avail_elem =3D NULL; + g_autofree SVQElement *next_avail_elem =3D NULL; =20 if (!svq->vq) { return; @@ -604,16 +608,16 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) vhost_svq_flush(svq, false); =20 for (unsigned i =3D 0; i < svq->vring.num; ++i) { - g_autofree VirtQueueElement *elem =3D NULL; - elem =3D g_steal_pointer(&svq->ring_id_maps[i]); - if (elem) { - virtqueue_detach_element(svq->vq, elem, 0); + g_autofree SVQElement *svq_elem =3D NULL; + svq_elem =3D g_steal_pointer(&svq->ring_id_maps[i]); + if (svq_elem) { + virtqueue_detach_element(svq->vq, &svq_elem->elem, 0); } } =20 next_avail_elem =3D g_steal_pointer(&svq->next_guest_avail_elem); if (next_avail_elem) { - virtqueue_detach_element(svq->vq, next_avail_elem, 0); + virtqueue_detach_element(svq->vq, &next_avail_elem->elem, 0); } svq->vq =3D NULL; g_free(svq->desc_next); --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988088; cv=none; d=zohomail.com; s=zohoarc; b=XQKyjo/PV4BrX4YB5vf8dt44PPR9XYTk5yG526AbdMDtWtiH2601rR8842tAw4M7mZZ3vImotiqTpsLVVkxPrIeOrnfxhYEsWtQJuXUJRTyDHO6tOdtF/EMjUUXTRPCwhK6nAD9j2ivqp3Bmw4gxJPzORW+KHGZs8q894ksv/GU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988088; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BN5qT8006jrUJpSdXRMJdAu+K0oZUNtCZ5Uz1NjXk+0=; b=ePqzrC+w3jQ2QsLIyBcE3/rV2OCEeSeVH44e5RP/59JwYbQa2tuOPrYOJ++MdfT0Q2ozWdJ/LEXDDDqjNSKaCes/RBbY42VcUh44JyxdlSGcu1mq9EqOUyp8iuhHYou8U2fCBWPZyRW1gmfqqwFhQbkbQuKJHmYolegfKxw77PY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988088803459.16929007844203; Thu, 19 May 2022 12:21:28 -0700 (PDT) Received: from localhost ([::1]:60238 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrliA-00066j-S2 for importer2@patchew.org; Thu, 19 May 2022 15:21:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48140) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaj-0005FT-Su for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:50842) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaf-0006Om-F5 for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:44 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-660-Rrue_lnAOy6iKwMxH22ZZw-1; Thu, 19 May 2022 15:13:37 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5CA5585A5BC; Thu, 19 May 2022 19:13:37 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id AAB072026D6A; Thu, 19 May 2022 19:13:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987620; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BN5qT8006jrUJpSdXRMJdAu+K0oZUNtCZ5Uz1NjXk+0=; b=ZgmHf0ZDKhuoIdCcJU413t89yL9Hk6DsB2592f74mA5geVomBo/cKhUYYqNUuigdyDL4yg PjjvoF5WfjQT3FmEn+W52OlpwUPYfX3+vlGpzMpR2wGfR8c19AvENnu9y//3z9PcMjc1rv klmiornuZNbeKUDq9EvT5TZmHKHYdMk= X-MC-Unique: Rrue_lnAOy6iKwMxH22ZZw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 09/21] vhost: Add svq copy desc mode Date: Thu, 19 May 2022 21:12:54 +0200 Message-Id: <20220519191306.821774-10-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988090902100001 Enable SVQ to not to forward the descriptor translating its address to qemu's IOVA but copying to a region outside of the guest. Virtio-net control VQ will use this mode, so we don't need to send all the guest's memory every time there is a change, but only on messages. Reversely, CVQ will only have access to control messages. This lead to less messing with memory listeners. We could also try to send only the required translation by message, but this presents a problem when many control messages occupy the same guest's memory region. Lastly, this allows us to inject messages from QEMU to the device in a simple manner. CVQ should be used rarely and with small messages, so all the drawbacks should be assumible. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 10 ++ include/hw/virtio/vhost-vdpa.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 174 +++++++++++++++++++++++++++-- hw/virtio/vhost-vdpa.c | 1 + net/vhost-vdpa.c | 1 + 5 files changed, 175 insertions(+), 12 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index e06ac52158..79cb2d301f 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -17,6 +17,12 @@ =20 typedef struct SVQElement { VirtQueueElement elem; + + /* SVQ IOVA address of in buffer and out buffer if cloned */ + hwaddr in_iova, out_iova; + + /* Length of in buffer */ + size_t in_len; } SVQElement; =20 typedef void (*VirtQueueElementCallback)(VirtIODevice *vdev, @@ -102,6 +108,9 @@ typedef struct VhostShadowVirtqueue { =20 /* Next head to consume from the device */ uint16_t last_used_idx; + + /* Copy each descriptor to QEMU iova */ + bool copy_descs; } VhostShadowVirtqueue; =20 bool vhost_svq_valid_features(uint64_t features, Error **errp); @@ -119,6 +128,7 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq); =20 VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_map, const VhostShadowVirtqueueOps *ops, + bool copy_descs, const VhostShadowVirtqueueMapOps *map_= ops, void *map_ops_opaque); =20 diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index f1ba46a860..dc2884eea4 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -33,6 +33,7 @@ typedef struct vhost_vdpa { struct vhost_vdpa_iova_range iova_range; uint64_t acked_features; bool shadow_vqs_enabled; + bool svq_copy_descs; /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; GPtrArray *shadow_vqs; diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 044005ba89..5a8feb1cbc 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -16,6 +16,7 @@ #include "qemu/log.h" #include "qemu/memalign.h" #include "linux-headers/linux/vhost.h" +#include "qemu/iov.h" =20 /** * Validate the transport device features that both guests can use with th= e SVQ @@ -70,6 +71,30 @@ static uint16_t vhost_svq_available_slots(const VhostSha= dowVirtqueue *svq) return svq->vring.num - (svq->shadow_avail_idx - svq->shadow_used_idx); } =20 +static void vhost_svq_alloc_buffer(void **base, size_t *len, + const struct iovec *iov, size_t num, + bool write) +{ + *len =3D iov_size(iov, num); + size_t buf_size =3D ROUND_UP(*len, 4096); + + if (!num) { + return; + } + + /* + * Linearize element. If guest had a descriptor chain, we expose the d= evice + * a single buffer. + */ + *base =3D qemu_memalign(4096, buf_size); + if (!write) { + iov_to_buf(iov, num, 0, *base, *len); + memset(*base + *len, 0, buf_size - *len); + } else { + memset(*base, 0, *len); + } +} + /** * Translate addresses between the qemu's virtual address and the SVQ IOVA * @@ -126,7 +151,9 @@ static bool vhost_svq_translate_addr(const VhostShadowV= irtqueue *svq, * Write descriptors to SVQ vring * * @svq: The shadow virtqueue + * @svq_elem: The shadow virtqueue element * @sg: Cache for hwaddr + * @descs_len: Total written buffer if svq->copy_descs. * @iovec: The iovec from the guest * @num: iovec length * @more_descs: True if more descriptors come in the chain @@ -134,7 +161,9 @@ static bool vhost_svq_translate_addr(const VhostShadowV= irtqueue *svq, * * Return true if success, false otherwise and print error. */ -static bool vhost_svq_vring_write_descs(VhostShadowVirtqueue *svq, hwaddr = *sg, +static bool vhost_svq_vring_write_descs(VhostShadowVirtqueue *svq, + SVQElement *svq_elem, hwaddr *sg, + size_t *descs_len, const struct iovec *iovec, size_t = num, bool more_descs, bool write) { @@ -142,18 +171,41 @@ static bool vhost_svq_vring_write_descs(VhostShadowVi= rtqueue *svq, hwaddr *sg, unsigned n; uint16_t flags =3D write ? cpu_to_le16(VRING_DESC_F_WRITE) : 0; vring_desc_t *descs =3D svq->vring.desc; - bool ok; - if (num =3D=3D 0) { return true; } =20 - ok =3D vhost_svq_translate_addr(svq, sg, iovec, num); - if (unlikely(!ok)) { - return false; + if (svq->copy_descs) { + void *buf; + DMAMap map =3D {}; + int r; + + vhost_svq_alloc_buffer(&buf, descs_len, iovec, num, write); + map.translated_addr =3D (hwaddr)(uintptr_t)buf; + map.size =3D ROUND_UP(*descs_len, 4096) - 1; + map.perm =3D write ? IOMMU_RW : IOMMU_RO, + r =3D vhost_iova_tree_map_alloc(svq->iova_tree, &map); + if (unlikely(r !=3D IOVA_OK)) { + error_report("Cannot map injected element"); + return false; + } + + r =3D svq->map_ops->map(map.iova, map.size + 1, + (void *)map.translated_addr, !write, + svq->map_ops_opaque); + /* TODO: Handle error */ + assert(r =3D=3D 0); + num =3D 1; + sg[0] =3D map.iova; + } else { + bool ok =3D vhost_svq_translate_addr(svq, sg, iovec, num); + if (unlikely(!ok)) { + return false; + } } =20 for (n =3D 0; n < num; n++) { + uint32_t len =3D svq->copy_descs ? *descs_len : iovec[n].iov_len; if (more_descs || (n + 1 < num)) { descs[i].flags =3D flags | cpu_to_le16(VRING_DESC_F_NEXT); descs[i].next =3D cpu_to_le16(svq->desc_next[i]); @@ -161,7 +213,7 @@ static bool vhost_svq_vring_write_descs(VhostShadowVirt= queue *svq, hwaddr *sg, descs[i].flags =3D flags; } descs[i].addr =3D cpu_to_le64(sg[n]); - descs[i].len =3D cpu_to_le32(iovec[n].iov_len); + descs[i].len =3D cpu_to_le32(len); =20 last =3D i; i =3D cpu_to_le16(svq->desc_next[i]); @@ -178,7 +230,8 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *s= vq, SVQElement *svq_elem, unsigned avail_idx; vring_avail_t *avail =3D svq->vring.avail; bool ok; - g_autofree hwaddr *sgs =3D g_new(hwaddr, MAX(elem->out_num, elem->in_n= um)); + g_autofree hwaddr *sgs =3D NULL; + hwaddr *in_sgs, *out_sgs; =20 *head =3D svq->free_head; =20 @@ -189,15 +242,24 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue = *svq, SVQElement *svq_elem, return false; } =20 - ok =3D vhost_svq_vring_write_descs(svq, sgs, elem->out_sg, elem->out_n= um, + if (!svq->copy_descs) { + sgs =3D g_new(hwaddr, MAX(elem->out_num, elem->in_num)); + in_sgs =3D out_sgs =3D sgs; + } else { + in_sgs =3D &svq_elem->in_iova; + out_sgs =3D &svq_elem->out_iova; + } + ok =3D vhost_svq_vring_write_descs(svq, svq_elem, out_sgs, (size_t[]){= }, + elem->out_sg, elem->out_num, elem->in_num > 0, false); if (unlikely(!ok)) { return false; } =20 - ok =3D vhost_svq_vring_write_descs(svq, sgs, elem->in_sg, elem->in_num= , false, - true); + ok =3D vhost_svq_vring_write_descs(svq, svq_elem, in_sgs, &svq_elem->i= n_len, + elem->in_sg, elem->in_num, false, tru= e); if (unlikely(!ok)) { + /* TODO unwind out_sg */ return false; } =20 @@ -276,6 +338,7 @@ static void vhost_handle_guest_kick(VhostShadowVirtqueu= e *svq) SVQElement *svq_elem; VirtQueueElement *elem; bool ok; + uint32_t needed_slots; =20 if (svq->next_guest_avail_elem) { svq_elem =3D g_steal_pointer(&svq->next_guest_avail_elem); @@ -288,7 +351,8 @@ static void vhost_handle_guest_kick(VhostShadowVirtqueu= e *svq) } =20 elem =3D &svq_elem->elem; - if (elem->out_num + elem->in_num > vhost_svq_available_slots(s= vq)) { + needed_slots =3D svq->copy_descs ? 1 : elem->out_num + elem->i= n_num; + if (needed_slots > vhost_svq_available_slots(svq)) { /* * This condition is possible since a contiguous buffer in= GPA * does not imply a contiguous buffer in qemu's VA @@ -411,6 +475,76 @@ static SVQElement *vhost_svq_get_buf(VhostShadowVirtqu= eue *svq, uint32_t *len) return g_steal_pointer(&svq->ring_id_maps[used_elem.id]); } =20 +/** + * Unmap a descriptor chain of a SVQ element, optionally copying its in bu= ffers + * + * @svq: Shadow VirtQueue + * @iova: SVQ IO Virtual address of descriptor + * @iov: Optional iovec to store device writable buffer + * @iov_cnt: iov length + * @buf_len: Length written by the device + * + * Print error message in case of error + */ +static bool vhost_svq_unmap_iov(VhostShadowVirtqueue *svq, hwaddr iova, + const struct iovec *iov, size_t iov_cnt, + size_t buf_len) +{ + DMAMap needle =3D { + /* + * No need to specify size since contiguous iova chunk was allocat= ed + * by SVQ. + */ + .iova =3D iova, + }; + const DMAMap *map =3D vhost_iova_tree_find(svq->iova_tree, &needle); + int r; + + if (!map) { + error_report("Cannot locate expected map"); + return false; + } + + r =3D svq->map_ops->unmap(map->iova, map->size + 1, svq->map_ops_opaqu= e); + if (unlikely(r !=3D 0)) { + error_report("Device cannot unmap: %s(%d)", g_strerror(r), r); + return false; + } + + if (iov) { + iov_from_buf(iov, iov_cnt, 0, (const void *)map->translated_addr, = buf_len); + } + qemu_vfree((void *)map->translated_addr); + vhost_iova_tree_remove(svq->iova_tree, &needle); + return true; +} + +/** + * Unmap shadow virtqueue element + * + * @svq_elem: Shadow VirtQueue Element + * @copy_in: Copy in buffer to the element at unmapping + */ +static bool vhost_svq_unmap_elem(VhostShadowVirtqueue *svq, SVQElement *sv= q_elem, uint32_t len, bool copy_in) +{ + VirtQueueElement *elem =3D &svq_elem->elem; + const struct iovec *in_iov =3D copy_in ? elem->in_sg : NULL; + size_t in_count =3D copy_in ? elem->in_num : 0; + if (elem->out_num) { + bool ok =3D vhost_svq_unmap_iov(svq, svq_elem->out_iova, NULL, 0, = 0); + if (unlikely(!ok)) { + return false; + } + } + + if (elem->in_num) { + return vhost_svq_unmap_iov(svq, svq_elem->in_iova, in_iov, in_coun= t, + len); + } + + return true; +} + static void vhost_svq_flush(VhostShadowVirtqueue *svq, bool check_for_avail_queue) { @@ -429,6 +563,13 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, break; } =20 + if (svq->copy_descs) { + bool ok =3D vhost_svq_unmap_elem(svq, svq_elem, len, true); + if (unlikely(!ok)) { + return; + } + } + elem =3D &svq_elem->elem; if (svq->ops && svq->ops->used_elem_handler) { svq->ops->used_elem_handler(svq->vdev, elem); @@ -611,12 +752,18 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) g_autofree SVQElement *svq_elem =3D NULL; svq_elem =3D g_steal_pointer(&svq->ring_id_maps[i]); if (svq_elem) { + if (svq->copy_descs) { + vhost_svq_unmap_elem(svq, svq_elem, 0, false); + } virtqueue_detach_element(svq->vq, &svq_elem->elem, 0); } } =20 next_avail_elem =3D g_steal_pointer(&svq->next_guest_avail_elem); if (next_avail_elem) { + if (svq->copy_descs) { + vhost_svq_unmap_elem(svq, next_avail_elem, 0, false); + } virtqueue_detach_element(svq->vq, &next_avail_elem->elem, 0); } svq->vq =3D NULL; @@ -632,6 +779,7 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) * * @iova_tree: Tree to perform descriptors translations * @ops: SVQ operations hooks + * @copy_descs: Copy each descriptor to QEMU iova * @map_ops: SVQ mapping operation hooks * @map_ops_opaque: Opaque data to pass to mapping operations * @@ -641,6 +789,7 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) */ VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, const VhostShadowVirtqueueOps *ops, + bool copy_descs, const VhostShadowVirtqueueMapOps *map_= ops, void *map_ops_opaque) { @@ -665,6 +814,7 @@ VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova= _tree, event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); svq->iova_tree =3D iova_tree; svq->ops =3D ops; + svq->copy_descs =3D copy_descs; svq->map_ops =3D map_ops; svq->map_ops_opaque =3D map_ops_opaque; return g_steal_pointer(&svq); diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index e6ef944e23..31b3d4d013 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -436,6 +436,7 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, = struct vhost_vdpa *v, for (unsigned n =3D 0; n < hdev->nvqs; ++n) { g_autoptr(VhostShadowVirtqueue) svq =3D vhost_svq_new(v->iova_tree, v->shadow_vq_ops, + v->svq_copy_descs, &vhost_vdpa_svq_map= _ops, v); =20 diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index ef12fc284c..174fec5e77 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -254,6 +254,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, s->vhost_vdpa.index =3D queue_pair_index; if (!is_datapath) { s->vhost_vdpa.shadow_vq_ops =3D &vhost_vdpa_net_svq_ops; + s->vhost_vdpa.svq_copy_descs =3D true; } ret =3D vhost_vdpa_add(nc, (void *)&s->vhost_vdpa, queue_pair_index, n= vqs); if (ret) { --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988311; cv=none; d=zohomail.com; s=zohoarc; b=SiT3uxArLXee3RHoXPx1S0ptdPe8U22dKDJmTfewmMJ5XxklV/Tz7s0vZkAXaVIYO/Gmyc+764Q9iu+WsPNmePWmyylsV8oYELPT0c0YVUqf1enCENwAO+FNcL8fmaW4V4fmMQWY1vy6V8OJLl8DZoEodlVOXLr/5twOIMRTXN4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988311; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=4Rk5vuJjlEY7VO/1pCIBCwcJaBWNUxkAO+ircCyr+84=; b=OvAWEtJjT/50ZyxjP9u9neAi/lVsTyp8ATOOOWotlbFO+xGn0bDQhqIieqYB8DbFCp9z9gIm/MKmWehrU6dVSjX1WW1dAcnpfdT4apqtMJMRFA8CHhFJmNV3/D3DzsjEoLPZCSzUN0zp4pl9vA4+FHePIgKvKywV5A8wueZk9W8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988311844531.796617440865; Thu, 19 May 2022 12:25:11 -0700 (PDT) Received: from localhost ([::1]:43450 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrllm-0005hd-B9 for importer2@patchew.org; Thu, 19 May 2022 15:25:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48168) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlam-0005OB-A1 for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:56816) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlak-0006PL-G7 for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:48 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-401-LTRMZHbePYyNPZUxoVGy8w-1; Thu, 19 May 2022 15:13:40 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3956A29AB3E1; Thu, 19 May 2022 19:13:40 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9F4962026D6A; Thu, 19 May 2022 19:13:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987623; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Rk5vuJjlEY7VO/1pCIBCwcJaBWNUxkAO+ircCyr+84=; b=f8m+oPNaG4+er28xWKz8GpdLbdmlsKoUNJ/59iOuhd5SAfqtYyrvewvZdBvEeYQVaXfWAf Zbc3giRTGNEsgxo9OSd7VsBnNil7AyKad2rPzAY29lr8Bw39RQ4cdnerxIrIqoB2Llh20i J31Od8pm3wftbgeJd3ybwPMOnAHx5BU= X-MC-Unique: LTRMZHbePYyNPZUxoVGy8w-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 10/21] vhost: Add vhost_svq_inject Date: Thu, 19 May 2022 21:12:55 +0200 Message-Id: <20220519191306.821774-11-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988313910100001 This allows qemu to inject buffers to the device without guest's notice. This will be use to inject net CVQ messages to restore status in the destination. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 5 +++ hw/virtio/vhost-shadow-virtqueue.c | 72 +++++++++++++++++++++++++----- 2 files changed, 65 insertions(+), 12 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 79cb2d301f..8fe0367944 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -23,6 +23,9 @@ typedef struct SVQElement { =20 /* Length of in buffer */ size_t in_len; + + /* Buffer has been injected by QEMU, not by the guest */ + bool not_from_guest; } SVQElement; =20 typedef void (*VirtQueueElementCallback)(VirtIODevice *vdev, @@ -115,6 +118,8 @@ typedef struct VhostShadowVirtqueue { =20 bool vhost_svq_valid_features(uint64_t features, Error **errp); =20 +int vhost_svq_inject(VhostShadowVirtqueue *svq, const struct iovec *iov, + size_t out_num, size_t in_num); void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); void vhost_svq_set_svq_call_fd(VhostShadowVirtqueue *svq, int call_fd); void vhost_svq_get_vring_addr(const VhostShadowVirtqueue *svq, diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 5a8feb1cbc..c535c99905 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -312,6 +312,43 @@ static void vhost_svq_kick(VhostShadowVirtqueue *svq) event_notifier_set(&svq->hdev_kick); } =20 +/** + * Inject a chain of buffers to the device + * + * @svq: Shadow VirtQueue + * @iov: Descriptors buffer + * @out_num: Number of out elements + * @in_num: Number of in elements + */ +int vhost_svq_inject(VhostShadowVirtqueue *svq, const struct iovec *iov, + size_t out_num, size_t in_num) +{ + SVQElement *svq_elem; + uint16_t num_slots =3D (in_num ? 1 : 0) + (out_num ? 1 : 0); + + /* + * To inject buffers in a SVQ that does not copy descriptors is not + * supported. All vhost_svq_inject calls are controlled by qemu so we = won't + * hit these assertions. + */ + assert(svq->copy_descs); + assert(num_slots > 0); + + if (unlikely(svq->next_guest_avail_elem)) { + error_report("Injecting in a full queue"); + return -ENOMEM; + } + + svq_elem =3D virtqueue_alloc_element(sizeof(*svq_elem), out_num, in_nu= m); + iov_copy(svq_elem->elem.in_sg, in_num, iov + out_num, in_num, 0, SIZE_= MAX); + iov_copy(svq_elem->elem.out_sg, out_num, iov, out_num, 0, SIZE_MAX); + svq_elem->not_from_guest =3D true; + vhost_svq_add(svq, svq_elem); + vhost_svq_kick(svq); + + return 0; +} + /** * Forward available buffers. * @@ -350,6 +387,7 @@ static void vhost_handle_guest_kick(VhostShadowVirtqueu= e *svq) break; } =20 + svq_elem->not_from_guest =3D false; elem =3D &svq_elem->elem; needed_slots =3D svq->copy_descs ? 1 : elem->out_num + elem->i= n_num; if (needed_slots > vhost_svq_available_slots(svq)) { @@ -575,19 +613,24 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, svq->ops->used_elem_handler(svq->vdev, elem); } =20 - if (unlikely(i >=3D svq->vring.num)) { - qemu_log_mask(LOG_GUEST_ERROR, - "More than %u used buffers obtained in a %u size = SVQ", - i, svq->vring.num); - virtqueue_fill(vq, elem, len, i); - virtqueue_flush(vq, i); - return; + if (!svq_elem->not_from_guest) { + if (unlikely(i >=3D svq->vring.num)) { + qemu_log_mask( + LOG_GUEST_ERROR, + "More than %u used buffers obtained in a %u size S= VQ", + i, svq->vring.num); + virtqueue_fill(vq, elem, len, i); + virtqueue_flush(vq, i); + return; + } + virtqueue_fill(vq, elem, len, i++); } - virtqueue_fill(vq, elem, len, i++); } =20 - virtqueue_flush(vq, i); - event_notifier_set(&svq->svq_call); + if (i > 0) { + virtqueue_flush(vq, i); + event_notifier_set(&svq->svq_call); + } =20 if (check_for_avail_queue && svq->next_guest_avail_elem) { /* @@ -755,7 +798,10 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) if (svq->copy_descs) { vhost_svq_unmap_elem(svq, svq_elem, 0, false); } - virtqueue_detach_element(svq->vq, &svq_elem->elem, 0); + + if (!svq_elem->not_from_guest) { + virtqueue_detach_element(svq->vq, &svq_elem->elem, 0); + } } } =20 @@ -764,7 +810,9 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) if (svq->copy_descs) { vhost_svq_unmap_elem(svq, next_avail_elem, 0, false); } - virtqueue_detach_element(svq->vq, &next_avail_elem->elem, 0); + if (!next_avail_elem->not_from_guest) { + virtqueue_detach_element(svq->vq, &next_avail_elem->elem, 0); + } } svq->vq =3D NULL; g_free(svq->desc_next); --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988595; cv=none; d=zohomail.com; s=zohoarc; b=BxxEAbxa2gcla4SwGUNmQhxapCwsZmOZl6yvND0O9/gBYocLiNQFzTbThwZy3EYg681EbW/Rf7zhtXpLbCTWruPm0D+KbImBj3zYv5RjvS0NeM3LnP2Sif/fhjMgg63TZ78ol3OHdBfNRZniZ/71NeQdtLXjOD/oNRg1dzSHjPw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988595; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GIZw997gWmNVLbvZCzFiBphYewoCNa31+YNmGOgndBI=; b=lFQdXS2wsJEmbv2LdKdQ+FwYdFeYYRHjKdUSQWjoedUvjMJB+9zZlYbOGCfwERSUAc6g8V3PiHYqGzNGLwIB40ZYNLGd+pdgUMkqOPzrn1Jah2D0sd7nzy/cVj2Z6n3+dH/LJNnjDnqL50Abi0vWfmICI8weKvQYSeTcxy6IpFw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988595696690.686195370169; Thu, 19 May 2022 12:29:55 -0700 (PDT) Received: from localhost ([::1]:55866 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrlqM-00063O-F8 for importer2@patchew.org; Thu, 19 May 2022 15:29:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlam-0005Nh-5C for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:52848) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlak-0006PM-Hb for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:47 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-647-G2Qa_7usPI2ALSKW4cbPgg-1; Thu, 19 May 2022 15:13:43 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1352685A5BE; Thu, 19 May 2022 19:13:43 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7CB382026D6A; Thu, 19 May 2022 19:13:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987624; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GIZw997gWmNVLbvZCzFiBphYewoCNa31+YNmGOgndBI=; b=cuIX37i1tZ47yiZhEIRIEBsIzY/l65GXfMD1CX+6YbXcVpzvWoV4GLaAHWHB3INVuj+MAd nLObiLDekNAOiiD3KXX9MsXOJG88vc3mDDwgr0Nj6d8RrRKNulM5LOduMryZPNj9C62yAd FzdDwuX4V95UKUzB1K86vEPta6FyjbQ= X-MC-Unique: G2Qa_7usPI2ALSKW4cbPgg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 11/21] vhost: Update kernel headers Date: Thu, 19 May 2022 21:12:56 +0200 Message-Id: <20220519191306.821774-12-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988597221100002 Signed-off-by: Eugenio P=C3=A9rez --- include/standard-headers/linux/vhost_types.h | 11 ++++++++- linux-headers/linux/vhost.h | 25 ++++++++++++++++---- 2 files changed, 30 insertions(+), 6 deletions(-) diff --git a/include/standard-headers/linux/vhost_types.h b/include/standar= d-headers/linux/vhost_types.h index 0bd2684a2a..ce78551b0f 100644 --- a/include/standard-headers/linux/vhost_types.h +++ b/include/standard-headers/linux/vhost_types.h @@ -87,7 +87,7 @@ struct vhost_msg { =20 struct vhost_msg_v2 { uint32_t type; - uint32_t reserved; + uint32_t asid; union { struct vhost_iotlb_msg iotlb; uint8_t padding[64]; @@ -153,4 +153,13 @@ struct vhost_vdpa_iova_range { /* vhost-net should add virtio_net_hdr for RX, and strip for TX packets. */ #define VHOST_NET_F_VIRTIO_NET_HDR 27 =20 +/* Use message type V2 */ +#define VHOST_BACKEND_F_IOTLB_MSG_V2 0x1 +/* IOTLB can accept batching hints */ +#define VHOST_BACKEND_F_IOTLB_BATCH 0x2 +/* IOTLB can accept address space identifier through V2 type of IOTLB + * message + */ +#define VHOST_BACKEND_F_IOTLB_ASID 0x3 + #endif diff --git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h index 5d99e7c242..d42eb46efd 100644 --- a/linux-headers/linux/vhost.h +++ b/linux-headers/linux/vhost.h @@ -89,11 +89,6 @@ =20 /* Set or get vhost backend capability */ =20 -/* Use message type V2 */ -#define VHOST_BACKEND_F_IOTLB_MSG_V2 0x1 -/* IOTLB can accept batching hints */ -#define VHOST_BACKEND_F_IOTLB_BATCH 0x2 - #define VHOST_SET_BACKEND_FEATURES _IOW(VHOST_VIRTIO, 0x25, __u64) #define VHOST_GET_BACKEND_FEATURES _IOR(VHOST_VIRTIO, 0x26, __u64) =20 @@ -154,6 +149,26 @@ /* Get the config size */ #define VHOST_VDPA_GET_CONFIG_SIZE _IOR(VHOST_VIRTIO, 0x79, __u32) =20 +/* Get the number of virtqueue groups. */ +#define VHOST_VDPA_GET_GROUP_NUM _IOR(VHOST_VIRTIO, 0x7A, unsigned int) + +/* Get the number of address spaces. */ +#define VHOST_VDPA_GET_AS_NUM _IOR(VHOST_VIRTIO, 0x7B, unsigned int) + +/* Get the group for a virtqueue: read index, write group in num, + * The virtqueue index is stored in the index field of + * vhost_vring_state. The group for this specific virtqueue is + * returned via num field of vhost_vring_state. + */ +#define VHOST_VDPA_GET_VRING_GROUP _IOWR(VHOST_VIRTIO, 0x7C, \ + struct vhost_vring_state) +/* Set the ASID for a virtqueue group. The group index is stored in + * the index field of vhost_vring_state, the ASID associated with this + * group is stored at num field of vhost_vring_state. + */ +#define VHOST_VDPA_SET_GROUP_ASID _IOW(VHOST_VIRTIO, 0x7D, \ + struct vhost_vring_state) + /* Get the count of all virtqueues */ #define VHOST_VDPA_GET_VQS_COUNT _IOR(VHOST_VIRTIO, 0x80, __u32) =20 --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988807; cv=none; d=zohomail.com; s=zohoarc; b=N8XY1OOCFE3RTf3AD9SniPayPoQ26PqCkbcB8izipOo8RI5gWmqMHqpAhGzF/HuGhtvjLA3GKNFvhX++lPVxgWc+XThRLG81cdNxFYrkp0MPev8vak0mBWxcsRe78qlOxBMUU3S9i4DpmgXRkaBfI+TDDMOiTUHHygFZ96Ni1R0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988807; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZuS07kiIQEXSFOQblRFIUzZ7Vg9qIIPzvSO/iExh9s4=; b=WAK1n9xag6rctrS5jbpVVRXNjbpGw7rKnWskiriYVV+L9JbcFLd63C04+VKEDEeVYNSEUN2/h8JiKT0/ivHPyM/3F4yUwMLPpqP+UGlbxQuexI2s45trGsj0kE6ptCmyznrIAUouPWVbliR1MIiAPJ5SRMWeKTiI9l/36vgwtO8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988807673722.3763612352334; Thu, 19 May 2022 12:33:27 -0700 (PDT) Received: from localhost ([::1]:33742 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrltm-0002Cy-K4 for importer2@patchew.org; Thu, 19 May 2022 15:33:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48192) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaq-0005Uh-0J for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:53530) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlao-0006Pv-GA for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:51 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-653--Wz40Z1KNPmUfbndZPXj9w-1; Thu, 19 May 2022 15:13:46 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DECBB800B21; Thu, 19 May 2022 19:13:45 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5571A2026D6A; Thu, 19 May 2022 19:13:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987629; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZuS07kiIQEXSFOQblRFIUzZ7Vg9qIIPzvSO/iExh9s4=; b=H310Lw+IfqQ2Jm1XnQ18sudkGEqsK26fF3ZmD6LM5r1fxxNeQKe3WfbBeReJ9AsAecVpG0 LK9TzMZ/MSae6EQi7r/m2aSo9ND9o2tlll0iVHH/GigvHPcwS+oDTyP+bw4pDAXFX18Sy7 S7peESyOVzhwfNfMUdcJsqoGDShj6J8= X-MC-Unique: -Wz40Z1KNPmUfbndZPXj9w-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 12/21] vdpa: delay set_vring_ready after DRIVER_OK Date: Thu, 19 May 2022 21:12:57 +0200 Message-Id: <20220519191306.821774-13-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988808381100001 To restore the device in the destination of a live migration we send the commands through control virtqueue. For a device to read CVQ it must have received DRIVER_OK status bit. However this open a window where the device could start receiving packets in rx queue 0 before it receive the RSS configuration. To avoid that, we will not send vring_enable until all configuration is used by the device. As a first step, reverse the DRIVER_OK and SET_VRING_ENABLE steps. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 31b3d4d013..13e5e2a061 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -748,13 +748,18 @@ static int vhost_vdpa_get_vq_index(struct vhost_dev *= dev, int idx) return idx; } =20 +/** + * Set ready all vring of the device + * + * @dev: Vhost device + */ static int vhost_vdpa_set_vring_ready(struct vhost_dev *dev) { int i; trace_vhost_vdpa_set_vring_ready(dev); - for (i =3D 0; i < dev->nvqs; ++i) { + for (i =3D 0; i < dev->vq_index_end; ++i) { struct vhost_vring_state state =3D { - .index =3D dev->vq_index + i, + .index =3D i, .num =3D 1, }; vhost_vdpa_call(dev, VHOST_VDPA_SET_VRING_ENABLE, &state); @@ -1117,7 +1122,6 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev= , bool started) if (unlikely(!ok)) { return -1; } - vhost_vdpa_set_vring_ready(dev); } else { ok =3D vhost_vdpa_svqs_stop(dev); if (unlikely(!ok)) { @@ -1131,16 +1135,22 @@ static int vhost_vdpa_dev_start(struct vhost_dev *d= ev, bool started) } =20 if (started) { + int r; memory_listener_register(&v->listener, &address_space_memory); - return vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); + r =3D vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); + if (unlikely(r)) { + return r; + } + vhost_vdpa_set_vring_ready(dev); } else { vhost_vdpa_reset_device(dev); vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); memory_listener_unregister(&v->listener); =20 - return 0; } + + return 0; } =20 static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base, --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988273; cv=none; d=zohomail.com; s=zohoarc; b=gXnL6Jn3vROSsM+giFrGlHfaqJe2n9Hv2ePcqdknCUUnEe+USPyfBPEjhLAq4yJxXz+Xkgacu68HqEyugPkhGKORrljMPfVwP9PGq5NVhu954yn4ZAZLWrTCREvWWNQti4L6CkNquoFvVqDyIOKd+39QKIpiTmzAvb/l8fg79jU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988273; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fU/RsuJkVNZGtkaJmkVTz8JDO/UhA/dz7iWVC1PvyFY=; b=SfTcjhlQ9px7+USMPYTe3WTWW/vF98BoNICO07juF11/UUeFtXGN5DpIHA9X3wnyqcais8NELH/x1V7wbgpOuVuh/cQcxe1on2rdPDPOanY+3pBXsXHipFSomcKAJ1qCX50AypXiUeYAm4WEitkgrvVPrrxxNztdO/5bZhXfaz4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988273920208.57589514582935; Thu, 19 May 2022 12:24:33 -0700 (PDT) Received: from localhost ([::1]:40634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrllA-0003ls-Uh for importer2@patchew.org; Thu, 19 May 2022 15:24:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48206) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlat-0005Wd-DD for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:56533) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaq-0006Q8-Vb for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:55 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-189-8NLZhAyjN8i85wnYpIvVPw-1; Thu, 19 May 2022 15:13:49 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B67F4811E78; Thu, 19 May 2022 19:13:48 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2CEBF200AE6D; Thu, 19 May 2022 19:13:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fU/RsuJkVNZGtkaJmkVTz8JDO/UhA/dz7iWVC1PvyFY=; b=Y9VVuSPf6PC7vaQvHYflXOsc0ATKkhCGFCgaQdbvkFqtLwUrnO7TcymlGoJMNHpewXPSAf IL3hYhFyUS8n0tH1Q18V318xJFzDyDxk3XUnhGixQFu97hvuUMGx+/sXBtyKa8/Jxf+APk MIs0HIrJn64yNLeIijI3+1AjELhubnI= X-MC-Unique: 8NLZhAyjN8i85wnYpIvVPw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 13/21] vhost: Add ShadowVirtQueueStart operation Date: Thu, 19 May 2022 21:12:58 +0200 Message-Id: <20220519191306.821774-14-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988275770100001 It allows to run commands at SVQ start. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 4 ++++ hw/virtio/vhost-vdpa.c | 14 ++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 8fe0367944..3c55fe2641 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -28,10 +28,14 @@ typedef struct SVQElement { bool not_from_guest; } SVQElement; =20 +typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; +typedef int (*ShadowVirtQueueStart)(VhostShadowVirtqueue *svq, + struct vhost_dev *dev); typedef void (*VirtQueueElementCallback)(VirtIODevice *vdev, const VirtQueueElement *elem); =20 typedef struct VhostShadowVirtqueueOps { + ShadowVirtQueueStart start; VirtQueueElementCallback used_elem_handler; } VhostShadowVirtqueueOps; =20 diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 13e5e2a061..eec6d544e9 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1141,6 +1141,20 @@ static int vhost_vdpa_dev_start(struct vhost_dev *de= v, bool started) if (unlikely(r)) { return r; } + + if (v->shadow_vqs_enabled) { + for (unsigned i =3D 0; i < v->shadow_vqs->len; ++i) { + VhostShadowVirtqueue *svq =3D g_ptr_array_index(v->shadow_= vqs, + i); + if (svq->ops && svq->ops->start) { + r =3D svq->ops->start(svq, dev); + if (unlikely(r)) { + return r; + } + } + } + } + vhost_vdpa_set_vring_ready(dev); } else { vhost_vdpa_reset_device(dev); --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988265; cv=none; d=zohomail.com; s=zohoarc; b=WhemPDTl+y6Hzx4NFLlRPrWl1bXs1gLVPqDO3t+AT6iFkq1shyFeG8P/jtfcjvxTT7i+EgyGU4iQNlE19TX1rUQAjOQSIb29uao67b3DfEfv2654CIrTBHHyO9QtNBiMZMZ9FNulu/YIc8YtwCdmwMiM9j4LwpOyZrFbpwU4Npo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988265; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=p0kmGxyJJl3JsSW5GyXv7DZGDrdPocHKltB5Md+G1dA=; b=CZn1ZUUe60dZafBRIO+tcsdkZYVdP1/RVSXchOQ4hbgFQxdoisldd7KAe8b52Yun4V+lZCNo7PN3DmyByyTQHX5eXKZW0fNVMDFrq7snLb6kg8Z3WLVBpKlrXEFH/SgjLD2fRFHVtZvfX1U53+RIE6MOiJIEE4Dt4lK2EDh2FM4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988265801288.78564640802733; Thu, 19 May 2022 12:24:25 -0700 (PDT) Received: from localhost ([::1]:40428 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrll2-0003c5-N4 for importer2@patchew.org; Thu, 19 May 2022 15:24:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48222) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlay-0005dR-5J for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:31061) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaw-0006RT-Bj for qemu-devel@nongnu.org; Thu, 19 May 2022 15:13:59 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-443-fSub-vEpMYyNiCzVzfecpA-1; Thu, 19 May 2022 15:13:52 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8CCF11C0515A; Thu, 19 May 2022 19:13:51 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0417F2026D6A; Thu, 19 May 2022 19:13:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987637; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p0kmGxyJJl3JsSW5GyXv7DZGDrdPocHKltB5Md+G1dA=; b=Y6Dx/cHsUcPp0Rg6xmKh2pMXi3fi+avQRYDRn+WoQVyXvtMBZYucVrbp4JT24PMi6wSlbh h4x7nzTfHWaP3NI/vbf+oZ3B2R5cJDjj/UkKlej5Ilx6gwh6tz/ZAGubyOo+8lTak1oaoD Pqk0E9VTW1Olrre/45pCGquaaYUfyVA= X-MC-Unique: fSub-vEpMYyNiCzVzfecpA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 14/21] vhost: Make possible to check for device exclusive vq group Date: Thu, 19 May 2022 21:12:59 +0200 Message-Id: <20220519191306.821774-15-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988267887100001 CVQ needs to be in its own group, not shared with any data vq. Enable the checking of it here, before introducing address space id concepts. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost.h | 2 + hw/net/vhost_net.c | 4 +- hw/virtio/vhost-vdpa.c | 79 ++++++++++++++++++++++++++++++++++++++- hw/virtio/trace-events | 1 + 4 files changed, 84 insertions(+), 2 deletions(-) diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index b291fe4e24..cebec1d817 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -84,6 +84,8 @@ struct vhost_dev { int vq_index_end; /* if non-zero, minimum required value for max_queues */ int num_queues; + /* Must be a vq group different than any other vhost dev */ + bool independent_vq_group; uint64_t features; uint64_t acked_features; uint64_t backend_features; diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index ccac5b7a64..1c2386c01c 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -339,14 +339,16 @@ int vhost_net_start(VirtIODevice *dev, NetClientState= *ncs, } =20 for (i =3D 0; i < nvhosts; i++) { + bool cvq_idx =3D i >=3D data_queue_pairs; =20 - if (i < data_queue_pairs) { + if (!cvq_idx) { peer =3D qemu_get_peer(ncs, i); } else { /* Control Virtqueue */ peer =3D qemu_get_peer(ncs, n->max_queue_pairs); } =20 net =3D get_vhost_net(peer); + net->dev.independent_vq_group =3D !!cvq_idx; vhost_net_set_vq_index(net, i * 2, index_end); =20 /* Suppress the masking guest notifiers on vhost user diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index eec6d544e9..52dd8baa8d 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -685,7 +685,8 @@ static int vhost_vdpa_set_backend_cap(struct vhost_dev = *dev) { uint64_t features; uint64_t f =3D 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | - 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH; + 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | + 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID; int r; =20 if (vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, &features)) { @@ -1110,6 +1111,78 @@ static bool vhost_vdpa_svqs_stop(struct vhost_dev *d= ev) return true; } =20 +static int vhost_vdpa_get_vring_group(struct vhost_dev *dev, + struct vhost_vring_state *state) +{ + int ret =3D vhost_vdpa_call(dev, VHOST_VDPA_GET_VRING_GROUP, state); + trace_vhost_vdpa_get_vring_group(dev, state->index, state->num); + return ret; +} + +static bool vhost_dev_is_independent_group(struct vhost_dev *dev) +{ + struct vhost_vdpa *v =3D dev->opaque; + struct vhost_vring_state this_vq_group =3D { + .index =3D dev->vq_index, + }; + int ret; + + if (!(dev->backend_cap & VHOST_BACKEND_F_IOTLB_ASID)) { + return true; + } + + if (!v->shadow_vqs_enabled) { + return true; + } + + ret =3D vhost_vdpa_get_vring_group(dev, &this_vq_group); + if (unlikely(ret)) { + goto call_err; + } + + for (int i =3D 1; i < dev->nvqs; ++i) { + struct vhost_vring_state vq_group =3D { + .index =3D dev->vq_index + i, + }; + + ret =3D vhost_vdpa_get_vring_group(dev, &vq_group); + if (unlikely(ret)) { + goto call_err; + } + if (unlikely(vq_group.num !=3D this_vq_group.num)) { + error_report("VQ %d group is different than VQ %d one", + this_vq_group.index, vq_group.index); + return false; + } + } + + for (int i =3D 0; i < dev->vq_index_end; ++i) { + struct vhost_vring_state vq_group =3D { + .index =3D i, + }; + + if (dev->vq_index <=3D i && i < dev->vq_index + dev->nvqs) { + continue; + } + + ret =3D vhost_vdpa_get_vring_group(dev, &vq_group); + if (unlikely(ret)) { + goto call_err; + } + if (unlikely(vq_group.num =3D=3D this_vq_group.num)) { + error_report("VQ %d group is the same as VQ %d one", + this_vq_group.index, vq_group.index); + return false; + } + } + + return true; + +call_err: + error_report("Can't read vq group, errno=3D%d (%s)", ret, g_strerror(-= ret)); + return false; +} + static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) { struct vhost_vdpa *v =3D dev->opaque; @@ -1118,6 +1191,10 @@ static int vhost_vdpa_dev_start(struct vhost_dev *de= v, bool started) =20 if (started) { vhost_vdpa_host_notifiers_init(dev); + if (dev->independent_vq_group && + !vhost_dev_is_independent_group(dev)) { + return -1; + } ok =3D vhost_vdpa_svqs_start(dev); if (unlikely(!ok)) { return -1; diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index ab8e095b73..ffb8eb26e7 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -46,6 +46,7 @@ vhost_vdpa_set_vring_ready(void *dev) "dev: %p" vhost_vdpa_dump_config(void *dev, const char *line) "dev: %p %s" vhost_vdpa_set_config(void *dev, uint32_t offset, uint32_t size, uint32_t = flags) "dev: %p offset: %"PRIu32" size: %"PRIu32" flags: 0x%"PRIx32 vhost_vdpa_get_config(void *dev, void *config, uint32_t config_len) "dev: = %p config: %p config_len: %"PRIu32 +vhost_vdpa_get_vring_group(void *dev, unsigned int index, unsigned int num= ) "dev: %p index: %u num: %u" vhost_vdpa_dev_start(void *dev, bool started) "dev: %p started: %d" vhost_vdpa_set_log_base(void *dev, uint64_t base, unsigned long long size,= int refcnt, int fd, void *log) "dev: %p base: 0x%"PRIx64" size: %llu refcn= t: %d fd: %d log: %p" vhost_vdpa_set_vring_addr(void *dev, unsigned int index, unsigned int flag= s, uint64_t desc_user_addr, uint64_t used_user_addr, uint64_t avail_user_ad= dr, uint64_t log_guest_addr) "dev: %p index: %u flags: 0x%x desc_user_addr:= 0x%"PRIx64" used_user_addr: 0x%"PRIx64" avail_user_addr: 0x%"PRIx64" log_g= uest_addr: 0x%"PRIx64 --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988596; cv=none; d=zohomail.com; s=zohoarc; b=A0ZvvyD4FwP2aJlAa7ZW1DFyYNSD4+7OOyyKR/eD6RM5ptAG6bOA4v23fj4PCR8uDbOErDEQXR9G7wG5FAYNHDQf8NI0yJ8Q7JSbGK6XJHNUs+6iOztGNt6cwCSZd6zPl4yFjLPsOBYJ264QbeP4vyY03J6izzqZyHB3c0gpoO0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988596; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CWiB2UPpPFNLsK1gITyT4zta7OXMHp9dwTDbjarxjKA=; b=cGuXovKBJwpivkqOToCr0Ld4OQcn5S1rB+4bhR3PPpJ7MdTDYef/z2BFFaDuVC7Xb+aQaWdgiAw1EmplCG1qn22q5uREKn4GKQWiCUg/HYrrFcb4cmjikrgGyQEmqjQhu5F2uS3PjrfIv65h9MzabaZ52YVsrGRR/wChDlhTq/o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16529885964971010.7139046858636; Thu, 19 May 2022 12:29:56 -0700 (PDT) Received: from localhost ([::1]:55900 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrlqN-00064V-Fi for importer2@patchew.org; Thu, 19 May 2022 15:29:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48262) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlb5-0005fc-7Z for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:53809) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlb2-0006Rp-Ii for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:06 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-550-xuHyKZcAP3K8SBC49GAQ7g-1; Thu, 19 May 2022 15:14:01 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 650D4800B21; Thu, 19 May 2022 19:13:54 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id CF2C02024CAE; Thu, 19 May 2022 19:13:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CWiB2UPpPFNLsK1gITyT4zta7OXMHp9dwTDbjarxjKA=; b=MxSu1Gdcc7cTUtob4iGVqz/ZMNhwbkFUud4q+eIM+MeemvwUh1FVHAac1A/wEQj5U2GQh8 yCd3QCuS5rt5DMBYhl5qoE5ZpX12jQB3HJZsP7OyFFOeEnavcHKTkK5ADqOSgKsh9X31Wx Ga0fgpWKd8YvQZXGv4+CQ6pL2mzhoUc= X-MC-Unique: xuHyKZcAP3K8SBC49GAQ7g-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 15/21] vhost: add vhost_svq_poll Date: Thu, 19 May 2022 21:13:00 +0200 Message-Id: <20220519191306.821774-16-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988597210100001 It allows the Shadow Control VirtQueue to wait the device to use the comman= ds that restore the net device state after a live migration. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 57 +++++++++++++++++++++++++++--- 2 files changed, 54 insertions(+), 4 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 3c55fe2641..20ca59e9a7 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -124,6 +124,7 @@ bool vhost_svq_valid_features(uint64_t features, Error = **errp); =20 int vhost_svq_inject(VhostShadowVirtqueue *svq, const struct iovec *iov, size_t out_num, size_t in_num); +ssize_t vhost_svq_poll(VhostShadowVirtqueue *svq); void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); void vhost_svq_set_svq_call_fd(VhostShadowVirtqueue *svq, int call_fd); void vhost_svq_get_vring_addr(const VhostShadowVirtqueue *svq, diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index c535c99905..831ffb71e5 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -10,6 +10,8 @@ #include "qemu/osdep.h" #include "hw/virtio/vhost-shadow-virtqueue.h" =20 +#include + #include "qemu/error-report.h" #include "qapi/error.h" #include "qemu/main-loop.h" @@ -583,10 +585,11 @@ static bool vhost_svq_unmap_elem(VhostShadowVirtqueue= *svq, SVQElement *svq_elem return true; } =20 -static void vhost_svq_flush(VhostShadowVirtqueue *svq, - bool check_for_avail_queue) +static size_t vhost_svq_flush(VhostShadowVirtqueue *svq, + bool check_for_avail_queue) { VirtQueue *vq =3D svq->vq; + size_t ret =3D 0; =20 /* Forward as many used buffers as possible. */ do { @@ -604,7 +607,7 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, if (svq->copy_descs) { bool ok =3D vhost_svq_unmap_elem(svq, svq_elem, len, true); if (unlikely(!ok)) { - return; + return ret; } } =20 @@ -621,10 +624,12 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, i, svq->vring.num); virtqueue_fill(vq, elem, len, i); virtqueue_flush(vq, i); - return; + return ret + 1; } virtqueue_fill(vq, elem, len, i++); } + + ret++; } =20 if (i > 0) { @@ -640,6 +645,50 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, vhost_handle_guest_kick(svq); } } while (!vhost_svq_enable_notification(svq)); + + return ret; +} + +/** + * Poll the SVQ for device used buffers. + * + * This function race with main event loop SVQ polling, so extra + * syncthronization is needed. + * + * Return the number of descriptors read from the device. + */ +ssize_t vhost_svq_poll(VhostShadowVirtqueue *svq) +{ + int fd =3D event_notifier_get_fd(&svq->hdev_call); + GPollFD poll_fd =3D { + .fd =3D fd, + .events =3D G_IO_IN, + }; + assert(fd >=3D 0); + int r =3D g_poll(&poll_fd, 1, -1); + + if (unlikely(r < 0)) { + error_report("Cannot poll device call fd "G_POLLFD_FORMAT": (%d) %= s", + poll_fd.fd, errno, g_strerror(errno)); + return -errno; + } + + if (r =3D=3D 0) { + return 0; + } + + if (unlikely(poll_fd.revents & ~(G_IO_IN))) { + error_report( + "Error polling device call fd "G_POLLFD_FORMAT": revents=3D%d", + poll_fd.fd, poll_fd.revents); + return -1; + } + + /* + * Max return value of vhost_svq_flush is (uint16_t)-1, so it's safe to + * convert to ssize_t. + */ + return vhost_svq_flush(svq, false); } =20 /** --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988473; cv=none; d=zohomail.com; s=zohoarc; b=YfI0hp1qcn3EXmKmz5IKQu443gSdnNFXZ1lw1ptg5LV77NDFLfISYvA5V1Da/epvNbVS9FyY58phZlfHES+CNWs7Gx+D2/u1wIDUlGQJ89xfXrIufcjitKgVZQduX8GwZP6/f+oJTuAJz1sZkr/ZrDHIxJrApQI8/TzrlIXbves= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988473; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PAexm7b2Cs4jUq28oY1VpULrfRFl6S2JEkKMgFXH7f4=; b=m2AdwLdF7kB2/AcTlnsmT2ZJQU0oGLEhS4RFNKSNKI0nbCbiZJPImJOMMo9rrq5/Y0tgj11dk4nPl8TVYTEk6Pms4zrJY1QPTorJt2L2QwoGi2Jsp+0/lcQLMU2QvJuW94IakUx1imwWjFIj1cnBdxEfixeCtisooizj2BBnR3E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988473961749.7953974204949; Thu, 19 May 2022 12:27:53 -0700 (PDT) Received: from localhost ([::1]:48586 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrloO-0000w1-VU for importer2@patchew.org; Thu, 19 May 2022 15:27:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48236) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlb2-0005fD-2f for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:22752) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlaz-0006Rd-Re for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:03 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-403-nOaSx4UgPzW9YcdemDnJrQ-1; Thu, 19 May 2022 15:13:57 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3BD84185A794; Thu, 19 May 2022 19:13:57 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id A78D72026D6A; Thu, 19 May 2022 19:13:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PAexm7b2Cs4jUq28oY1VpULrfRFl6S2JEkKMgFXH7f4=; b=REV5fKlrGlzo1Xc+9OELgMdGbkFKo97WY2zcYgSALBnCtbedV56WHicp14PPOvhegTNo0d jA/PDI44YxEvn93l4i4cBilcKEY4KNFbRv1hkx5aas0OEMaueRZ6hO/fm8YcOl4o0bA1bF yH+3J9E68AnylaUjmNAjgvyCSfU8XJg= X-MC-Unique: nOaSx4UgPzW9YcdemDnJrQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 16/21] vdpa: Add vhost_vdpa_start_control_svq Date: Thu, 19 May 2022 21:13:01 +0200 Message-Id: <20220519191306.821774-17-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988474657100003 As a first step we only enable CVQ first than others. Future patches add state restore. Signed-off-by: Eugenio P=C3=A9rez --- net/vhost-vdpa.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 174fec5e77..a66f73ff63 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -188,6 +188,66 @@ static NetClientInfo net_vhost_vdpa_info =3D { .check_peer_type =3D vhost_vdpa_check_peer_type, }; =20 +static int vhost_vdpa_start_control_svq(VhostShadowVirtqueue *svq, + struct vhost_dev *dev) +{ + struct vhost_vring_state state =3D { + .index =3D virtio_get_queue_index(svq->vq), + .num =3D 1, + }; + struct vhost_vdpa *v =3D dev->opaque; + VirtIONet *n =3D VIRTIO_NET(dev->vdev); + uint64_t features =3D dev->vdev->host_features; + int r; + size_t num =3D 0; + + assert(dev->vhost_ops->backend_type =3D=3D VHOST_BACKEND_TYPE_VDPA); + + r =3D ioctl(v->device_fd, VHOST_VDPA_SET_VRING_ENABLE, &state); + if (r < 0) { + return -errno; + } + + if (features & BIT_ULL(VIRTIO_NET_F_CTRL_MAC_ADDR)) { + const struct virtio_net_ctrl_hdr ctrl =3D { + .class =3D VIRTIO_NET_CTRL_MAC, + .cmd =3D VIRTIO_NET_CTRL_MAC_ADDR_SET, + }; + uint8_t mac[6]; + virtio_net_ctrl_ack ack; + const struct iovec data[] =3D { + { + .iov_base =3D (void *)&ctrl, + .iov_len =3D sizeof(ctrl), + },{ + .iov_base =3D mac, + .iov_len =3D sizeof(mac), + },{ + .iov_base =3D &ack, + .iov_len =3D sizeof(ack), + } + }; + + memcpy(mac, n->mac, sizeof(mac)); + r =3D vhost_svq_inject(svq, data, 2, 1); + if (unlikely(r)) { + return r; + } + num++; + } + + while (num) { + /* + * We can call vhost_svq_poll here because BQL protects calls to r= un. + */ + size_t used =3D vhost_svq_poll(svq); + assert(used <=3D num); + num -=3D used; + } + + return 0; +} + static void vhost_vdpa_net_handle_ctrl(VirtIODevice *vdev, const VirtQueueElement *elem) { @@ -226,6 +286,7 @@ static void vhost_vdpa_net_handle_ctrl(VirtIODevice *vd= ev, =20 static const VhostShadowVirtqueueOps vhost_vdpa_net_svq_ops =3D { .used_elem_handler =3D vhost_vdpa_net_handle_ctrl, + .start =3D vhost_vdpa_start_control_svq, }; =20 static NetClientState *net_vhost_vdpa_init(NetClientState *peer, --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988515; cv=none; d=zohomail.com; s=zohoarc; b=Da0jym0WHPevjLSp4rf45jVg1QAY79CuWwcSaY/NMBX/HCDQ0GWlCRPm798paTR1OqkzzOFI+kUoR1vlH4Mnt3wQdljshyoMWFkn/DtoLPiAfbKPr3hkqGdzGmg9/lCCld5BBfa0CwjpfES/jiaFq6OkHVIjVxZQtIyD334JJvI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988515; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=aiTcdjC2t6j5hiouoPQWrXyl2OP5znNxEQhzZTT0NpM=; b=cGEB7hwnjnFtRW1J/EIqEsd0m9XWbTBoRnI0jbXXqRNKDHWvUq9xA9GC4WsoUpScXPBfnBVi7HBnp9cPZRM9YSCK1v1EwKywh4mHsEqpM3DOhOBlEvIKfzVXG5o5l4kNRfFJb7pcbbM4mLZih/u9l8w3D5Rx2RwP0ONl5W08K6Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988515327399.3549492301926; Thu, 19 May 2022 12:28:35 -0700 (PDT) Received: from localhost ([::1]:51738 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrlp4-0003CE-9A for importer2@patchew.org; Thu, 19 May 2022 15:28:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48264) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlb5-0005fd-BM for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:50356) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlb2-0006Rj-CH for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:07 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-65-vgTngYMQMR28pc7DQH5wWA-1; Thu, 19 May 2022 15:14:00 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 153B11857F02; Thu, 19 May 2022 19:14:00 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7DD062026D6A; Thu, 19 May 2022 19:13:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987643; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aiTcdjC2t6j5hiouoPQWrXyl2OP5znNxEQhzZTT0NpM=; b=OxV3G7IRB61NTlt/jrt3cPzv6kC5HHutz8I68nXYGdTB12VNG7O3Ms4qMyL1me0ZvUXbrs Bg0dbdbZ8Qu9tVE8ZJn3va5BEO1MRAuXHSOzQHpu4RBDmH1t4JxjNuJWBpl4rhnMnCru/I 1axVcL36quKpcf0gKTy/eN+bzZBVZxU= X-MC-Unique: vgTngYMQMR28pc7DQH5wWA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 17/21] vdpa: Add asid attribute to vdpa device Date: Thu, 19 May 2022 21:13:02 +0200 Message-Id: <20220519191306.821774-18-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988516875100001 We can configure ASID per group, but we still use asid 0 for every vdpa device. Multiple asid support for cvq will be introduced in next patches Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost.h | 1 + hw/net/vhost_net.c | 1 + hw/virtio/vhost-vdpa.c | 71 +++++++++++++++++++++++++++++++++++---- hw/virtio/trace-events | 9 ++--- 4 files changed, 72 insertions(+), 10 deletions(-) diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index cebec1d817..eadaf055f0 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -84,6 +84,7 @@ struct vhost_dev { int vq_index_end; /* if non-zero, minimum required value for max_queues */ int num_queues; + uint32_t address_space_id; /* Must be a vq group different than any other vhost dev */ bool independent_vq_group; uint64_t features; diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index 1c2386c01c..4d79d622f7 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -348,6 +348,7 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *= ncs, } =20 net =3D get_vhost_net(peer); + net->dev.address_space_id =3D !!cvq_idx; net->dev.independent_vq_group =3D !!cvq_idx; vhost_net_set_vq_index(net, i * 2, index_end); =20 diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 52dd8baa8d..0208e36589 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -79,14 +79,18 @@ static int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwa= ddr iova, hwaddr size, int ret =3D 0; =20 msg.type =3D v->msg_type; + if (v->dev->backend_cap & BIT_ULL(VHOST_BACKEND_F_IOTLB_ASID)) { + msg.asid =3D v->dev->address_space_id; + } msg.iotlb.iova =3D iova; msg.iotlb.size =3D size; msg.iotlb.uaddr =3D (uint64_t)(uintptr_t)vaddr; msg.iotlb.perm =3D readonly ? VHOST_ACCESS_RO : VHOST_ACCESS_RW; msg.iotlb.type =3D VHOST_IOTLB_UPDATE; =20 - trace_vhost_vdpa_dma_map(v, fd, msg.type, msg.iotlb.iova, msg.iotlb.siz= e, - msg.iotlb.uaddr, msg.iotlb.perm, msg.iotlb.typ= e); + trace_vhost_vdpa_dma_map(v, fd, msg.type, msg.asid, msg.iotlb.iova, + msg.iotlb.size, msg.iotlb.uaddr, msg.iotlb.pe= rm, + msg.iotlb.type); =20 if (write(fd, &msg, sizeof(msg)) !=3D sizeof(msg)) { error_report("failed to write, fd=3D%d, errno=3D%d (%s)", @@ -104,12 +108,15 @@ static int vhost_vdpa_dma_unmap(struct vhost_vdpa *v,= hwaddr iova, int fd =3D v->device_fd; int ret =3D 0; =20 + if (v->dev->backend_cap & BIT_ULL(VHOST_BACKEND_F_IOTLB_ASID)) { + msg.asid =3D v->dev->address_space_id; + } msg.type =3D v->msg_type; msg.iotlb.iova =3D iova; msg.iotlb.size =3D size; msg.iotlb.type =3D VHOST_IOTLB_INVALIDATE; =20 - trace_vhost_vdpa_dma_unmap(v, fd, msg.type, msg.iotlb.iova, + trace_vhost_vdpa_dma_unmap(v, fd, msg.type, msg.asid, msg.iotlb.iova, msg.iotlb.size, msg.iotlb.type); =20 if (write(fd, &msg, sizeof(msg)) !=3D sizeof(msg)) { @@ -123,13 +130,19 @@ static int vhost_vdpa_dma_unmap(struct vhost_vdpa *v,= hwaddr iova, =20 static void vhost_vdpa_listener_begin_batch(struct vhost_vdpa *v) { + struct vhost_dev *dev =3D v->dev; int fd =3D v->device_fd; struct vhost_msg_v2 msg =3D { .type =3D v->msg_type, .iotlb.type =3D VHOST_IOTLB_BATCH_BEGIN, }; =20 - trace_vhost_vdpa_listener_begin_batch(v, fd, msg.type, msg.iotlb.type); + if (dev->backend_cap & BIT_ULL(VHOST_BACKEND_F_IOTLB_ASID)) { + msg.asid =3D v->dev->address_space_id; + } + + trace_vhost_vdpa_listener_begin_batch(v, fd, msg.type, msg.asid, + msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) !=3D sizeof(msg)) { error_report("failed to write, fd=3D%d, errno=3D%d (%s)", fd, errno, strerror(errno)); @@ -161,10 +174,14 @@ static void vhost_vdpa_listener_commit(MemoryListener= *listener) return; } =20 + if (dev->backend_cap & BIT_ULL(VHOST_BACKEND_F_IOTLB_ASID)) { + msg.asid =3D v->dev->address_space_id; + } + msg.type =3D v->msg_type; msg.iotlb.type =3D VHOST_IOTLB_BATCH_END; - - trace_vhost_vdpa_listener_commit(v, fd, msg.type, msg.iotlb.type); + trace_vhost_vdpa_listener_commit(v, fd, msg.type, msg.asid, + msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) !=3D sizeof(msg)) { error_report("failed to write, fd=3D%d, errno=3D%d (%s)", fd, errno, strerror(errno)); @@ -1183,10 +1200,48 @@ call_err: return false; } =20 +static int vhost_vdpa_set_vq_group_address_space_id(struct vhost_dev *dev, + struct vhost_vring_state *= asid) +{ + trace_vhost_vdpa_set_vq_group_address_space_id(dev, asid->index, asid-= >num); + return vhost_vdpa_call(dev, VHOST_VDPA_SET_GROUP_ASID, asid); +} + +static int vhost_vdpa_set_address_space_id(struct vhost_dev *dev) +{ + struct vhost_vring_state vq_group =3D { + .index =3D dev->vq_index, + }; + struct vhost_vring_state asid; + int ret; + + if (!dev->address_space_id) { + return 0; + } + + ret =3D vhost_vdpa_get_vring_group(dev, &vq_group); + if (unlikely(ret)) { + error_report("Can't read vq group, errno=3D%d (%s)", ret, + g_strerror(-ret)); + return ret; + } + + asid.index =3D vq_group.num; + asid.num =3D dev->address_space_id; + ret =3D vhost_vdpa_set_vq_group_address_space_id(dev, &asid); + if (unlikely(ret)) { + error_report("Can't set vq group %u asid %u, errno=3D%d (%s)", + asid.index, asid.num, ret, g_strerror(-ret)); + } + return ret; +} + static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) { struct vhost_vdpa *v =3D dev->opaque; bool ok; + int r =3D 0; + trace_vhost_vdpa_dev_start(dev, started); =20 if (started) { @@ -1195,6 +1250,10 @@ static int vhost_vdpa_dev_start(struct vhost_dev *de= v, bool started) !vhost_dev_is_independent_group(dev)) { return -1; } + r =3D vhost_vdpa_set_address_space_id(dev); + if (unlikely(r)) { + return r; + } ok =3D vhost_vdpa_svqs_start(dev); if (unlikely(!ok)) { return -1; diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index ffb8eb26e7..67adad8610 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -26,10 +26,10 @@ vhost_user_write(uint32_t req, uint32_t flags) "req:%d = flags:0x%"PRIx32"" vhost_user_create_notifier(int idx, void *n) "idx:%d n:%p" =20 # vhost-vdpa.c -vhost_vdpa_dma_map(void *vdpa, int fd, uint32_t msg_type, uint64_t iova, u= int64_t size, uint64_t uaddr, uint8_t perm, uint8_t type) "vdpa:%p fd: %d m= sg_type: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" uaddr: 0x%"PRIx64" p= erm: 0x%"PRIx8" type: %"PRIu8 -vhost_vdpa_dma_unmap(void *vdpa, int fd, uint32_t msg_type, uint64_t iova,= uint64_t size, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" iova: 0x%= "PRIx64" size: 0x%"PRIx64" type: %"PRIu8 -vhost_vdpa_listener_begin_batch(void *v, int fd, uint32_t msg_type, uint8_= t type) "vdpa:%p fd: %d msg_type: %"PRIu32" type: %"PRIu8 -vhost_vdpa_listener_commit(void *v, int fd, uint32_t msg_type, uint8_t typ= e) "vdpa:%p fd: %d msg_type: %"PRIu32" type: %"PRIu8 +vhost_vdpa_dma_map(void *vdpa, int fd, uint32_t msg_type, uint32_t asid, u= int64_t iova, uint64_t size, uint64_t uaddr, uint8_t perm, uint8_t type) "v= dpa:%p fd: %d msg_type: %"PRIu32" asid: %"PRIu32" iova: 0x%"PRIx64" size: 0= x%"PRIx64" uaddr: 0x%"PRIx64" perm: 0x%"PRIx8" type: %"PRIu8 +vhost_vdpa_dma_unmap(void *vdpa, int fd, uint32_t msg_type, uint32_t asid,= uint64_t iova, uint64_t size, uint8_t type) "vdpa:%p fd: %d msg_type: %"PR= Iu32" asid: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" type: %"PRIu8 +vhost_vdpa_listener_begin_batch(void *v, int fd, uint32_t msg_type, uint32= _t asid, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" asid: %"PRIu32"= type: %"PRIu8 +vhost_vdpa_listener_commit(void *v, int fd, uint32_t msg_type, uint32_t as= id, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" asid: %"PRIu32" type= : %"PRIu8 vhost_vdpa_listener_region_add(void *vdpa, uint64_t iova, uint64_t llend, = void *vaddr, bool readonly) "vdpa: %p iova 0x%"PRIx64" llend 0x%"PRIx64" va= ddr: %p read-only: %d" vhost_vdpa_listener_region_del(void *vdpa, uint64_t iova, uint64_t llend) = "vdpa: %p iova 0x%"PRIx64" llend 0x%"PRIx64 vhost_vdpa_add_status(void *dev, uint8_t status) "dev: %p status: 0x%"PRIx8 @@ -47,6 +47,7 @@ vhost_vdpa_dump_config(void *dev, const char *line) "dev:= %p %s" vhost_vdpa_set_config(void *dev, uint32_t offset, uint32_t size, uint32_t = flags) "dev: %p offset: %"PRIu32" size: %"PRIu32" flags: 0x%"PRIx32 vhost_vdpa_get_config(void *dev, void *config, uint32_t config_len) "dev: = %p config: %p config_len: %"PRIu32 vhost_vdpa_get_vring_group(void *dev, unsigned int index, unsigned int num= ) "dev: %p index: %u num: %u" +vhost_vdpa_set_vq_group_address_space_id(void *dev, unsigned int index, un= signed int num) "dev: %p index: %u num: %u" vhost_vdpa_dev_start(void *dev, bool started) "dev: %p started: %d" vhost_vdpa_set_log_base(void *dev, uint64_t base, unsigned long long size,= int refcnt, int fd, void *log) "dev: %p base: 0x%"PRIx64" size: %llu refcn= t: %d fd: %d log: %p" vhost_vdpa_set_vring_addr(void *dev, unsigned int index, unsigned int flag= s, uint64_t desc_user_addr, uint64_t used_user_addr, uint64_t avail_user_ad= dr, uint64_t log_guest_addr) "dev: %p index: %u flags: 0x%x desc_user_addr:= 0x%"PRIx64" used_user_addr: 0x%"PRIx64" avail_user_addr: 0x%"PRIx64" log_g= uest_addr: 0x%"PRIx64 --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988941; cv=none; d=zohomail.com; s=zohoarc; b=C3WmUy9YbkyLh2OXpju1IEKFf/1eXOuumJZNA21G5beeYdDlRHUSLDPGMW8QkLuYzU6fBh6/NgmJkT3Gpa7tYxk0q0AMWrSvvn7iGfxmrGjyPrTZnaEoY2JgYx0FXhzjTyz1pw3W3gTOk59cahfbIFX9z3nMtV0VJunY0XjeXoE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988941; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nlQj9ven0tTwwma+r2eSKJXu7pWJULNx+q8yt8E1T1w=; b=VwaTremmG9+LFi07mBDYkdHoLg5fZIFMerBR0fKTNzvserNG1hXlmI8Ec+NtIkDBXipB9C3H+20O/TVU0mFy6wfekCyKkaF61xGKPxPZ5hrRfQQR50/5OILJ7MBoslQ7OKFWBf0//+mmF+Zxwc3xz+YS2psWezvI3UWyai4W5PQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988941912247.68493020610026; Thu, 19 May 2022 12:35:41 -0700 (PDT) Received: from localhost ([::1]:38920 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrlvw-0006JM-N1 for importer2@patchew.org; Thu, 19 May 2022 15:35:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48278) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlb7-0005hJ-2T for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37437) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlb5-0006Rw-GH for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:08 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-668-708pikefPPavigekIvV0xQ-1; Thu, 19 May 2022 15:14:03 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E8C873C138BC; Thu, 19 May 2022 19:14:02 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5873A2026D6A; Thu, 19 May 2022 19:14:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987646; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nlQj9ven0tTwwma+r2eSKJXu7pWJULNx+q8yt8E1T1w=; b=Uek1AdhuNEbeNhyiEIMtHdR0eoUMoqWlpL9UlVA600Sve4O4lTCE+8Y7NkuXg5/CJnu5Kg XCPVWogk7FPpqGDyw+mrmV3H9N6WqGeblrRUf7mn+5DjLFUE+Sh7eF0tREblqcz5MoIiRF O8/OW/elTaQWP/vPqs5SQAmmG6MyLEk= X-MC-Unique: 708pikefPPavigekIvV0xQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 18/21] vdpa: Extract get features part from vhost_vdpa_get_max_queue_pairs Date: Thu, 19 May 2022 21:13:03 +0200 Message-Id: <20220519191306.821774-19-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988943000100001 To know the device features is needed for CVQ SVQ, so SVQ knows if it can handle all commands or not. Extract from vhost_vdpa_get_max_queue_pairs so we can reuse it. Signed-off-by: Eugenio P=C3=A9rez --- net/vhost-vdpa.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index a66f73ff63..8960b8db74 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -325,20 +325,24 @@ static NetClientState *net_vhost_vdpa_init(NetClientS= tate *peer, return nc; } =20 -static int vhost_vdpa_get_max_queue_pairs(int fd, int *has_cvq, Error **er= rp) +static int vhost_vdpa_get_features(int fd, uint64_t *features, Error **err= p) +{ + int ret =3D ioctl(fd, VHOST_GET_FEATURES, features); + if (ret) { + error_setg_errno(errp, errno, + "Fail to query features from vhost-vDPA device"); + } + return ret; +} + +static int vhost_vdpa_get_max_queue_pairs(int fd, uint64_t features, + int *has_cvq, Error **errp) { unsigned long config_size =3D offsetof(struct vhost_vdpa_config, buf); g_autofree struct vhost_vdpa_config *config =3D NULL; __virtio16 *max_queue_pairs; - uint64_t features; int ret; =20 - ret =3D ioctl(fd, VHOST_GET_FEATURES, &features); - if (ret) { - error_setg(errp, "Fail to query features from vhost-vDPA device"); - return ret; - } - if (features & (1 << VIRTIO_NET_F_CTRL_VQ)) { *has_cvq =3D 1; } else { @@ -368,10 +372,11 @@ int net_init_vhost_vdpa(const Netdev *netdev, const c= har *name, NetClientState *peer, Error **errp) { const NetdevVhostVDPAOptions *opts; + uint64_t features; int vdpa_device_fd; g_autofree NetClientState **ncs =3D NULL; NetClientState *nc; - int queue_pairs, i, has_cvq =3D 0; + int queue_pairs, r, i, has_cvq =3D 0; =20 assert(netdev->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); opts =3D &netdev->u.vhost_vdpa; @@ -385,7 +390,12 @@ int net_init_vhost_vdpa(const Netdev *netdev, const ch= ar *name, return -errno; } =20 - queue_pairs =3D vhost_vdpa_get_max_queue_pairs(vdpa_device_fd, + r =3D vhost_vdpa_get_features(vdpa_device_fd, &features, errp); + if (r) { + return r; + } + + queue_pairs =3D vhost_vdpa_get_max_queue_pairs(vdpa_device_fd, feature= s, &has_cvq, errp); if (queue_pairs < 0) { qemu_close(vdpa_device_fd); --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988700; cv=none; d=zohomail.com; s=zohoarc; b=CkZHtnRWyBROUsx0kaJCnk/aKF4x7Gt7Ymj757+8BcyUse/hONPjFf5wkTzzsmyj/2SjHGF6k4OMKyjopAlveHgkSe4fCZ5/OXm3d2yWa9hXSkf2Gs0y7NOUDSUTpBDDhc734uUzVtdG7b2bcCAdGnThsyXRuE32UTEeEA67+o4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988700; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2Jgq2TsrWzUgTQ5q2pts3SpmrlwBh6lTORD9NNR6VG4=; b=GDZ+1iCr4YqXHzc/CrcgBXhhnswFdIdeCftx02InpyBbL7NIHyLNzDY9jQ3x25ddijeaG0zmJcbcBx7l6NLTVc9GXA6O2zvkLr7o31WwPovtfDxWVDwlSHwWVlqL+myFOULS7uXMV75/Nlh4Cz8Avsi3M8PDNjl8m2qFokz1dnY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988700441733.7530451288609; Thu, 19 May 2022 12:31:40 -0700 (PDT) Received: from localhost ([::1]:58370 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrls3-0007qP-9B for importer2@patchew.org; Thu, 19 May 2022 15:31:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48316) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlbC-0005iF-26 for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:21076) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlbA-0006SS-7Q for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:13 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-247-loSv90j7OK-FZzp0sQZGCA-1; Thu, 19 May 2022 15:14:06 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DBABA3C025C3; Thu, 19 May 2022 19:14:05 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 37C0B2026D6A; Thu, 19 May 2022 19:14:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987651; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Jgq2TsrWzUgTQ5q2pts3SpmrlwBh6lTORD9NNR6VG4=; b=jIDpzA+uU2Trfxa3QCpTum5ekCYLDuez26ztr5Dko7NTm7r5pR8fqphU7S9bqnhwNUj9YE 2cjnuM80/4r3WwArKBdOCc2+KUEB/2T5qnjzaBSq8ojaSkWapLSbs6QAu9R3KAlVPYnHwD WEhpzxqmavLRoOWa5E2DIgvZi85todI= X-MC-Unique: loSv90j7OK-FZzp0sQZGCA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 19/21] vhost: Add reference counting to vhost_iova_tree Date: Thu, 19 May 2022 21:13:04 +0200 Message-Id: <20220519191306.821774-20-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988701978100001 Now that different vqs can have different ASIDs its easier to track them using reference counters. QEMU's glib version still does not have them so we've copied g_rc_box, so the implementation can be converted to glib's one when the minimum version is raised. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-iova-tree.h | 5 +++-- hw/virtio/vhost-iova-tree.c | 21 +++++++++++++++++++-- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/hw/virtio/vhost-iova-tree.h b/hw/virtio/vhost-iova-tree.h index 1ffcdc5b57..bacd17d99c 100644 --- a/hw/virtio/vhost-iova-tree.h +++ b/hw/virtio/vhost-iova-tree.h @@ -16,8 +16,9 @@ typedef struct VhostIOVATree VhostIOVATree; =20 VhostIOVATree *vhost_iova_tree_new(uint64_t iova_first, uint64_t iova_last= ); -void vhost_iova_tree_delete(VhostIOVATree *iova_tree); -G_DEFINE_AUTOPTR_CLEANUP_FUNC(VhostIOVATree, vhost_iova_tree_delete); +VhostIOVATree *vhost_iova_tree_acquire(VhostIOVATree *iova_tree); +void vhost_iova_tree_release(VhostIOVATree *iova_tree); +G_DEFINE_AUTOPTR_CLEANUP_FUNC(VhostIOVATree, vhost_iova_tree_release); =20 const DMAMap *vhost_iova_tree_find(const VhostIOVATree *iova_tree, const DMAMap *map); diff --git a/hw/virtio/vhost-iova-tree.c b/hw/virtio/vhost-iova-tree.c index 1a59894385..208476b3db 100644 --- a/hw/virtio/vhost-iova-tree.c +++ b/hw/virtio/vhost-iova-tree.c @@ -28,6 +28,9 @@ struct VhostIOVATree { =20 /* IOVA address to qemu memory maps. */ IOVATree *iova_taddr_map; + + /* Reference count */ + size_t refcnt; }; =20 /** @@ -44,14 +47,28 @@ VhostIOVATree *vhost_iova_tree_new(hwaddr iova_first, h= waddr iova_last) tree->iova_last =3D iova_last; =20 tree->iova_taddr_map =3D iova_tree_new(); + tree->refcnt =3D 1; return tree; } =20 /** - * Delete an iova tree + * Increases the reference count of the iova tree + */ +VhostIOVATree *vhost_iova_tree_acquire(VhostIOVATree *iova_tree) +{ + ++iova_tree->refcnt; + return iova_tree; +} + +/** + * Decrease reference counter of iova tree, freeing if it reaches 0 */ -void vhost_iova_tree_delete(VhostIOVATree *iova_tree) +void vhost_iova_tree_release(VhostIOVATree *iova_tree) { + if (--iova_tree->refcnt) { + return; + } + iova_tree_destroy(iova_tree->iova_taddr_map); g_free(iova_tree); } --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988809; cv=none; d=zohomail.com; s=zohoarc; b=PEtaSIwOp/PQ66A3a5VpBT2jGpjXCADcsA1Yo9Lr1YpILmAGS2/AXEqzjmp0ySfJcYpq0sP2ajFg/aoOZlh6ZtQMQaTEag0G83YrVBAoI7eZG9lnZNRILhFOhLczAgTaaK9IdXUiAUlnRSGW8+VGLxHvb3UAPAxXAlcV76QDF4A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988809; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=m6ytqvW59xz3BQQs0Gr86XfzqwarNYk1/FXs1BFeWbE=; b=gypBEbB5La+wahYZUQ468MD7JSZxVJmoc1msOlLa5Mmm5k82914hGqVuuon9IOBVN/FlMuZ2CZfW7f/+mHn6C3sebN64gmpuea0IydfhlutzUgRT6NP8Voh+IuOOtYNdOnsy6ucj0Y5oKeB7mS39Ri2XCxx0cEB9qf0Alc/KeUk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988809842257.73524756778966; Thu, 19 May 2022 12:33:29 -0700 (PDT) Received: from localhost ([::1]:33818 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrlto-0002MM-Dd for importer2@patchew.org; Thu, 19 May 2022 15:33:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48314) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlbB-0005iD-Kb for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42425) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlb8-0006SK-St for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:13 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-166-5nBkBX7PM9mlKNJGzolGuQ-1; Thu, 19 May 2022 15:14:09 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B7C7D101AA45; Thu, 19 May 2022 19:14:08 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2A3A12026D6A; Thu, 19 May 2022 19:14:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987650; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m6ytqvW59xz3BQQs0Gr86XfzqwarNYk1/FXs1BFeWbE=; b=OY7Qm4po//ch+Gm2D8iCwaHBOQReUktoKpzrLG7Zvt14y/ilM4Loj7rQX3El1t/zGOG+h5 iKcjP0dSZOgj0MOHS09Egj4o+xBVGr+A5RIGn8YOPjVlLgyPvZbppFwP4T/bPnDCk7kQ89 tt4yrRLhjd5DbCVVKPcKwZU3Yqe4xyU= X-MC-Unique: 5nBkBX7PM9mlKNJGzolGuQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 20/21] vdpa: Add x-svq to NetdevVhostVDPAOptions Date: Thu, 19 May 2022 21:13:05 +0200 Message-Id: <20220519191306.821774-21-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988810365100003 Finally offering the possibility to enable SVQ from the command line. Signed-off-by: Eugenio P=C3=A9rez --- qapi/net.json | 9 ++++++++- net/vhost-vdpa.c | 38 +++++++++++++++++++++++++++++++++++--- 2 files changed, 43 insertions(+), 4 deletions(-) diff --git a/qapi/net.json b/qapi/net.json index d6f7cfd4d6..cd7a1b32fe 100644 --- a/qapi/net.json +++ b/qapi/net.json @@ -445,12 +445,19 @@ # @queues: number of queues to be created for multiqueue vhost-vdpa # (default: 1) # +# @x-svq: Start device with (experimental) shadow virtqueue. (Since 7.1) +# (default: false) +# +# Features: +# @unstable: Member @x-svq is experimental. +# # Since: 5.1 ## { 'struct': 'NetdevVhostVDPAOptions', 'data': { '*vhostdev': 'str', - '*queues': 'int' } } + '*queues': 'int', + '*x-svq': {'type': 'bool', 'features' : [ 'unstable'] } } } =20 ## # @NetdevVmnetHostOptions: diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 8960b8db74..ef8c82f92e 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -129,6 +129,7 @@ static void vhost_vdpa_cleanup(NetClientState *nc) { VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, nc); =20 + g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_release); if (s->vhost_net) { vhost_net_cleanup(s->vhost_net); g_free(s->vhost_net); @@ -188,6 +189,14 @@ static NetClientInfo net_vhost_vdpa_info =3D { .check_peer_type =3D vhost_vdpa_check_peer_type, }; =20 +static int vhost_vdpa_get_iova_range(int fd, + struct vhost_vdpa_iova_range *iova_ra= nge) +{ + int ret =3D ioctl(fd, VHOST_VDPA_GET_IOVA_RANGE, iova_range); + + return ret < 0 ? -errno : 0; +} + static int vhost_vdpa_start_control_svq(VhostShadowVirtqueue *svq, struct vhost_dev *dev) { @@ -295,7 +304,9 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, int vdpa_device_fd, int queue_pair_index, int nvqs, - bool is_datapath) + bool is_datapath, + bool svq, + VhostIOVATree *iova_tree) { NetClientState *nc =3D NULL; VhostVDPAState *s; @@ -313,12 +324,18 @@ static NetClientState *net_vhost_vdpa_init(NetClientS= tate *peer, =20 s->vhost_vdpa.device_fd =3D vdpa_device_fd; s->vhost_vdpa.index =3D queue_pair_index; + s->vhost_vdpa.shadow_vqs_enabled =3D svq; + s->vhost_vdpa.iova_tree =3D iova_tree ? vhost_iova_tree_acquire(iova_t= ree) : + NULL; if (!is_datapath) { s->vhost_vdpa.shadow_vq_ops =3D &vhost_vdpa_net_svq_ops; s->vhost_vdpa.svq_copy_descs =3D true; } ret =3D vhost_vdpa_add(nc, (void *)&s->vhost_vdpa, queue_pair_index, n= vqs); if (ret) { + if (iova_tree) { + vhost_iova_tree_release(iova_tree); + } qemu_del_net_client(nc); return NULL; } @@ -377,6 +394,7 @@ int net_init_vhost_vdpa(const Netdev *netdev, const cha= r *name, g_autofree NetClientState **ncs =3D NULL; NetClientState *nc; int queue_pairs, r, i, has_cvq =3D 0; + g_autoptr(VhostIOVATree) iova_tree =3D NULL; =20 assert(netdev->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); opts =3D &netdev->u.vhost_vdpa; @@ -401,19 +419,31 @@ int net_init_vhost_vdpa(const Netdev *netdev, const c= har *name, qemu_close(vdpa_device_fd); return queue_pairs; } + if (opts->x_svq) { + struct vhost_vdpa_iova_range iova_range; + + if (has_cvq) { + error_setg(errp, "vdpa svq does not work with cvq"); + goto err_svq; + } + vhost_vdpa_get_iova_range(vdpa_device_fd, &iova_range); + iova_tree =3D vhost_iova_tree_new(iova_range.first, iova_range.las= t); + } =20 ncs =3D g_malloc0(sizeof(*ncs) * queue_pairs); =20 for (i =3D 0; i < queue_pairs; i++) { ncs[i] =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, - vdpa_device_fd, i, 2, true); + vdpa_device_fd, i, 2, true, opts->x_s= vq, + iova_tree); if (!ncs[i]) goto err; } =20 if (has_cvq) { nc =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, - vdpa_device_fd, i, 1, false); + vdpa_device_fd, i, 1, false, opts->x_svq, + iova_tree); if (!nc) goto err; } @@ -426,6 +456,8 @@ err: qemu_del_net_client(ncs[i]); } } + +err_svq: qemu_close(vdpa_device_fd); =20 return -1; --=20 2.27.0 From nobody Thu May 2 01:06:10 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652988943; cv=none; d=zohomail.com; s=zohoarc; b=Zx7OKJ8vhSRDzFC0iwBwSCqftBSLnBPheyJTipyYfeMfIlfV3Da2M0zaB07jbPA3iAcEtJXdao+p8z/qOXMG9i4GxhQQqowAgD1PmDKn0fdzXUeRgKCjBRCzQ0HcRd1z1m5OKw8axn3wo2cssYgw26VM8B3DnB0h30qW29JS1uU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652988943; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jHl10Grav3jvcUIoLdASsUU4Pdelp1R4NU9Pmy7E2yc=; b=SEiU7+OdxtpYGNTjgrV7VI+qw33wbtuPkYL45aYOZ5CThFjInyN7PbztFLCUG0C9ltg+GBOL36rv3T1l2Alu5O7vL++H3a3SUbjtPLpSTZNlJPifzwWW8cX5qAGmNLI1nCGIpuONTZlcKeca0GxSzRLtNKgF4H4OcYnH6YOFwOw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652988943542339.4502660866664; Thu, 19 May 2022 12:35:43 -0700 (PDT) Received: from localhost ([::1]:38986 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrlvy-0006Mm-IS for importer2@patchew.org; Thu, 19 May 2022 15:35:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48330) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlbE-0005mD-Ah for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39609) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrlbC-0006Sf-0B for qemu-devel@nongnu.org; Thu, 19 May 2022 15:14:15 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-536-sYJ01E54OiiFRMKW6Fd3jw-1; Thu, 19 May 2022 15:14:12 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8F9353C138B9; Thu, 19 May 2022 19:14:11 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 046A22026D6A; Thu, 19 May 2022 19:14:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652987653; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jHl10Grav3jvcUIoLdASsUU4Pdelp1R4NU9Pmy7E2yc=; b=Kdy2aUyI9k8c4ZFXPVYP7jm0SqtKTsGyFhs8njirW5ZkUGWzvjVuulzFiJSozTqjE7QTLY Sd0YyWj6pvKySyMKYM06csUV41KLRyRTewNyz/AMThUtgxgc4rs0MXI8xRUbGlIjvVLzjg wEUP6aS7RVvtMZEzzK94EqTlLAHvYu8= X-MC-Unique: sYJ01E54OiiFRMKW6Fd3jw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , "Michael S. Tsirkin" , Markus Armbruster , "Gonglei (Arei)" , Harpreet Singh Anand , Cornelia Huck , Zhu Lingshan , Laurent Vivier , Eli Cohen , Paolo Bonzini , Liuxiangdong , Eric Blake , Cindy Lu , Jason Wang , Parav Pandit Subject: [RFC PATCH v8 21/21] vdpa: Add x-cvq-svq Date: Thu, 19 May 2022 21:13:06 +0200 Message-Id: <20220519191306.821774-22-eperezma@redhat.com> In-Reply-To: <20220519191306.821774-1-eperezma@redhat.com> References: <20220519191306.821774-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652988945112100001 This isolates shadow cvq in its own group. Signed-off-by: Eugenio P=C3=A9rez --- qapi/net.json | 8 ++- net/vhost-vdpa.c | 134 ++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 133 insertions(+), 9 deletions(-) diff --git a/qapi/net.json b/qapi/net.json index cd7a1b32fe..f5b047ae15 100644 --- a/qapi/net.json +++ b/qapi/net.json @@ -447,9 +447,12 @@ # # @x-svq: Start device with (experimental) shadow virtqueue. (Since 7.1) # (default: false) +# @x-cvq-svq: Start device with (experimental) shadow virtqueue in its own +# virtqueue group. (Since 7.1) +# (default: false) # # Features: -# @unstable: Member @x-svq is experimental. +# @unstable: Members @x-svq and x-cvq-svq are experimental. # # Since: 5.1 ## @@ -457,7 +460,8 @@ 'data': { '*vhostdev': 'str', '*queues': 'int', - '*x-svq': {'type': 'bool', 'features' : [ 'unstable'] } } } + '*x-svq': {'type': 'bool', 'features' : [ 'unstable'] }, + '*x-cvq-svq': {'type': 'bool', 'features' : [ 'unstable'] } } } =20 ## # @NetdevVmnetHostOptions: diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index ef8c82f92e..ad006a2bf3 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -70,6 +70,30 @@ const int vdpa_feature_bits[] =3D { VHOST_INVALID_FEATURE_BIT }; =20 +/** Supported device specific feature bits with SVQ */ +static const uint64_t vdpa_svq_device_features =3D + BIT_ULL(VIRTIO_NET_F_CSUM) | + BIT_ULL(VIRTIO_NET_F_GUEST_CSUM) | + BIT_ULL(VIRTIO_NET_F_CTRL_GUEST_OFFLOADS) | + BIT_ULL(VIRTIO_NET_F_MTU) | + BIT_ULL(VIRTIO_NET_F_MAC) | + BIT_ULL(VIRTIO_NET_F_GUEST_TSO4) | + BIT_ULL(VIRTIO_NET_F_GUEST_TSO6) | + BIT_ULL(VIRTIO_NET_F_GUEST_ECN) | + BIT_ULL(VIRTIO_NET_F_GUEST_UFO) | + BIT_ULL(VIRTIO_NET_F_HOST_TSO4) | + BIT_ULL(VIRTIO_NET_F_HOST_TSO6) | + BIT_ULL(VIRTIO_NET_F_HOST_ECN) | + BIT_ULL(VIRTIO_NET_F_HOST_UFO) | + BIT_ULL(VIRTIO_NET_F_MRG_RXBUF) | + BIT_ULL(VIRTIO_NET_F_STATUS) | + BIT_ULL(VIRTIO_NET_F_CTRL_VQ) | + BIT_ULL(VIRTIO_NET_F_MQ) | + BIT_ULL(VIRTIO_F_ANY_LAYOUT) | + BIT_ULL(VIRTIO_NET_F_CTRL_MAC_ADDR) | + BIT_ULL(VIRTIO_NET_F_RSC_EXT) | + BIT_ULL(VIRTIO_NET_F_STANDBY); + VHostNetState *vhost_vdpa_get_vhost_net(NetClientState *nc) { VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, nc); @@ -352,6 +376,17 @@ static int vhost_vdpa_get_features(int fd, uint64_t *f= eatures, Error **errp) return ret; } =20 +static int vhost_vdpa_get_backend_features(int fd, uint64_t *features, + Error **errp) +{ + int ret =3D ioctl(fd, VHOST_GET_BACKEND_FEATURES, features); + if (ret) { + error_setg_errno(errp, errno, + "Fail to query backend features from vhost-vDPA device"); + } + return ret; +} + static int vhost_vdpa_get_max_queue_pairs(int fd, uint64_t features, int *has_cvq, Error **errp) { @@ -385,16 +420,56 @@ static int vhost_vdpa_get_max_queue_pairs(int fd, uin= t64_t features, return 1; } =20 +/** + * Check vdpa device to support CVQ group asid 1 + * + * @vdpa_device_fd: Vdpa device fd + * @queue_pairs: Queue pairs + * @errp: Error + */ +static int vhost_vdpa_check_cvq_svq(int vdpa_device_fd, int queue_pairs, + Error **errp) +{ + uint64_t backend_features; + unsigned num_as; + int r; + + r =3D vhost_vdpa_get_backend_features(vdpa_device_fd, &backend_feature= s, + errp); + if (unlikely(r)) { + return -1; + } + + if (unlikely(!(backend_features & VHOST_BACKEND_F_IOTLB_ASID))) { + error_setg(errp, "Device without IOTLB_ASID feature"); + return -1; + } + + r =3D ioctl(vdpa_device_fd, VHOST_VDPA_GET_AS_NUM, &num_as); + if (unlikely(r)) { + error_setg_errno(errp, errno, + "Cannot retrieve number of supported ASs"); + return -1; + } + if (unlikely(num_as < 2)) { + error_setg(errp, "Insufficient number of ASs (%u, min: 2)", num_as= ); + } + + return 0; +} + int net_init_vhost_vdpa(const Netdev *netdev, const char *name, NetClientState *peer, Error **errp) { const NetdevVhostVDPAOptions *opts; + struct vhost_vdpa_iova_range iova_range; uint64_t features; int vdpa_device_fd; g_autofree NetClientState **ncs =3D NULL; NetClientState *nc; int queue_pairs, r, i, has_cvq =3D 0; g_autoptr(VhostIOVATree) iova_tree =3D NULL; + ERRP_GUARD(); =20 assert(netdev->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); opts =3D &netdev->u.vhost_vdpa; @@ -419,14 +494,35 @@ int net_init_vhost_vdpa(const Netdev *netdev, const c= har *name, qemu_close(vdpa_device_fd); return queue_pairs; } - if (opts->x_svq) { - struct vhost_vdpa_iova_range iova_range; + if (opts->x_cvq_svq || opts->x_svq) { + vhost_vdpa_get_iova_range(vdpa_device_fd, &iova_range); + + uint64_t invalid_dev_features =3D + features & ~vdpa_svq_device_features & + /* Transport are all accepted at this point */ + ~MAKE_64BIT_MASK(VIRTIO_TRANSPORT_F_START, + VIRTIO_TRANSPORT_F_END - VIRTIO_TRANSPORT_F_S= TART); =20 - if (has_cvq) { - error_setg(errp, "vdpa svq does not work with cvq"); + if (invalid_dev_features) { + error_setg(errp, "vdpa svq does not work with features 0x%" PR= Ix64, + invalid_dev_features); goto err_svq; } - vhost_vdpa_get_iova_range(vdpa_device_fd, &iova_range); + } + + if (opts->x_cvq_svq) { + if (!has_cvq) { + error_setg(errp, "Cannot use x-cvq-svq with a device without c= vq"); + goto err_svq; + } + + r =3D vhost_vdpa_check_cvq_svq(vdpa_device_fd, queue_pairs, errp); + if (unlikely(r)) { + error_prepend(errp, "Cannot configure CVQ SVQ: "); + goto err_svq; + } + } + if (opts->x_svq) { iova_tree =3D vhost_iova_tree_new(iova_range.first, iova_range.las= t); } =20 @@ -441,11 +537,35 @@ int net_init_vhost_vdpa(const Netdev *netdev, const c= har *name, } =20 if (has_cvq) { + g_autoptr(VhostIOVATree) cvq_iova_tree =3D NULL; + + if (opts->x_cvq_svq) { + cvq_iova_tree =3D vhost_iova_tree_new(iova_range.first, + iova_range.last); + } else if (opts->x_svq) { + cvq_iova_tree =3D vhost_iova_tree_acquire(iova_tree); + } + nc =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, - vdpa_device_fd, i, 1, false, opts->x_svq, - iova_tree); + vdpa_device_fd, i, 1, false, + opts->x_cvq_svq || opts->x_svq, + cvq_iova_tree); if (!nc) goto err; + + if (opts->x_cvq_svq) { + struct vhost_vring_state asid =3D { + .index =3D 1, + .num =3D 1, + }; + + r =3D ioctl(vdpa_device_fd, VHOST_VDPA_SET_GROUP_ASID, &asid); + if (unlikely(r)) { + error_setg_errno(errp, errno, + "Cannot set cvq group independent asid"); + goto err; + } + } } =20 return 0; --=20 2.27.0