From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193852; cv=none; d=zohomail.com; s=zohoarc; b=gQSGs48FhveXLjjEvKhcXVvna+VfyrSKGmuWLXEkNWBbB0a9XrTLsR37fBcg0edLSvz4Y4Y3K8fyf0XfRcpGCZFOY1aUx28SO8KZCV03EG1L+hJJ838NmRqEmD0lA7sX0hX9FTqsLUdyJQNpfjACO1npfEsz1fe6UOrSQfJJnLw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193852; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ph7NhPNJqQFVQZvkebi126vtqIT7j0fmOZoCxa+ciPc=; b=hqW3RGDILT+rB1VxXlM39z0lvb6kp6vlKRBWJyDzye4tiF4W+aIPJp+B0eAL0agXcvvks9eQCA3i6rWX9Ila08F7cOGQE/5hdnz0CxjzUmUPThj+kKivH0lVhcdlFmIxDPuCvjAxIQggHnWwiMi5zfmPJXct9w/Xb5acumxhd/Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193852694771.6775147593692; Tue, 7 Mar 2023 04:57:32 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWqy-0005Mt-SA; Tue, 07 Mar 2023 07:55:40 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWqx-0005M2-7o for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:39 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWqv-00053B-Nu for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:38 -0500 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32783sst023573; Tue, 7 Mar 2023 12:55:36 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p416wnd2d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:35 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327BvUiX029145; Tue, 7 Mar 2023 12:55:34 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyekq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:34 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV51004358; Tue, 7 Mar 2023 12:55:34 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-2; Tue, 07 Mar 2023 12:55:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=ph7NhPNJqQFVQZvkebi126vtqIT7j0fmOZoCxa+ciPc=; b=ZYtFk1yCue7gAdqys/GJbU9RF16O86frhlMgJ+XeuFUT0MAxpBTfuz+ul5Jpon+odGNY iAtbYqbg/2nsWKUcGhglkSTTzcrLrobJPk9ATa3N9Ajd/c0pEZF/WpMgqgDUg7aHrDWl SrGW12AKpDcrIJb7taIyoqDOG1o86s7iXNs0ixPghM1nUEneZvTLIyQYWG5smzTM2QpK 6rRB1qxwd5HnpEQ0IDeIpZw/lIf7Re1kCi13DwiGYZSNKOKsAnhljaWnEcl8djmLFMbe DG0LibrBzufqFloddnq0CkuRPCoNkdCfxGCLMFXBPDK3lP4Anz3t/Tc2wUNyzOH5+rI8 gA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 01/15] vfio/common: Fix error reporting in vfio_get_dirty_bitmap() Date: Tue, 7 Mar 2023 12:54:36 +0000 Message-Id: <20230307125450.62409-2-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=852 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: DgA7aiAH0GfbLiVkgYcnwENjbLVUwBfe X-Proofpoint-ORIG-GUID: DgA7aiAH0GfbLiVkgYcnwENjbLVUwBfe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193854524100005 From: Avihai Horon Return -errno instead of -1 if VFIO_IOMMU_DIRTY_PAGES ioctl fails in vfio_get_dirty_bitmap(). Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index bab83c0e55cb..9fc305448fa2 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1335,6 +1335,7 @@ static int vfio_get_dirty_bitmap(VFIOContainer *conta= iner, uint64_t iova, =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { + ret =3D -errno; error_report("Failed to get dirty bitmap for iova: 0x%"PRIx64 " size: 0x%"PRIx64" err: %d", (uint64_t)range->iova, (uint64_t)range->size, errno); --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193825; cv=none; d=zohomail.com; s=zohoarc; b=if/Sq86zTunjixxO/waje57jI46RlsM/LG7P+2djrjPxFpRB7XpMeaATBVfsA82r+F1bXyOUo9jEJcaCY6UB5Z+9rKb0auNiMP8PSUxfFn9owD1dGaTzeK971eTqZYlcdImfq879VgJKBvlAdrhZKRnV359X99otmiX9L4UCDHA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193825; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xk8wvTPv6dJhnVr411G6qxujVLuqM/QjT1VPuR/+yew=; b=XBfOfqhaftKaylpDT1txD4NJusRduCSVKZO8WcNhCCUNBiKyc8GDKOj/KuIi8kvabnwAsUh7MVrqmtb6zI7VBmx6QWARErUah5Eqf1h8xRrm/n3wwMoLpFusqf+7rLHdg6az47qLKjQBzDw+lu5sOeR+ymxM46CjQlGlTiOyH2Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193825087878.5505745455274; Tue, 7 Mar 2023 04:57:05 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWr1-0005O1-L1; Tue, 07 Mar 2023 07:55:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr0-0005Nj-AJ for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:42 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWqy-00053h-K2 for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:42 -0500 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32784CJN032039; Tue, 7 Mar 2023 12:55:39 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p418xwdj2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:38 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CQi0G026663; Tue, 7 Mar 2023 12:55:37 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyemw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:37 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV53004358; Tue, 7 Mar 2023 12:55:36 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-3; Tue, 07 Mar 2023 12:55:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=xk8wvTPv6dJhnVr411G6qxujVLuqM/QjT1VPuR/+yew=; b=JB8hW5XFPl12mStEtYtKzTnZthbzaS35vRnqGFDtxAQ2G56G9x6w5TUJkN6uGBkUzVLv /XnAmm8R5m7VmRX1SWhlLhrBpWD0aU4ThL5ANQCxAevylaWmcMUJvMM0DPwpLyVac/Ek 73cgHGTOWCxRqhB5XhcV861VbbmA6GoYXHDRR4wVUnzA4qkt/li69R6VFGJZ2Ou2/5V5 nxk8tuV0+yVDcEyafYnhQ2kYs4kMI2t/zCdhKIt6d6xzm2c2hsDihSXo1zMpddDUV5dy u9/q0vQ6OPrW6joj+/OxEU7qDawj/bG2zbgmnTq8DWaDq/zOi10Btv1TlZ5ichjQtUwc oQ== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 02/15] vfio/common: Fix wrong %m usages Date: Tue, 7 Mar 2023 12:54:37 +0000 Message-Id: <20230307125450.62409-3-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=908 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: HNQ7ZkTTCLvr3oy9adfpYJwBM_bpuBPN X-Proofpoint-ORIG-GUID: HNQ7ZkTTCLvr3oy9adfpYJwBM_bpuBPN Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193826349100007 From: Avihai Horon There are several places where the %m conversion is used if one of vfio_dma_map(), vfio_dma_unmap() or vfio_get_dirty_bitmap() fail. The %m usage in these places is wrong since %m relies on errno value while the above functions don't report errors via errno. Fix it by using strerror() with the returned value instead. Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 9fc305448fa2..4d26e9cccf91 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -703,17 +703,17 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, I= OMMUTLBEntry *iotlb) read_only); if (ret) { error_report("vfio_dma_map(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx", %p) =3D %d (%m)", + "0x%"HWADDR_PRIx", %p) =3D %d (%s)", container, iova, - iotlb->addr_mask + 1, vaddr, ret); + iotlb->addr_mask + 1, vaddr, ret, strerror(-ret)); } } else { ret =3D vfio_dma_unmap(container, iova, iotlb->addr_mask + 1, iotl= b); if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") =3D %d (%m)", + "0x%"HWADDR_PRIx") =3D %d (%s)", container, iova, - iotlb->addr_mask + 1, ret); + iotlb->addr_mask + 1, ret, strerror(-ret)); } } out: @@ -1095,8 +1095,9 @@ static void vfio_listener_region_add(MemoryListener *= listener, vaddr, section->readonly); if (ret) { error_setg(&err, "vfio_dma_map(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx", %p) =3D %d (%m)", - container, iova, int128_get64(llsize), vaddr, ret); + "0x%"HWADDR_PRIx", %p) =3D %d (%s)", + container, iova, int128_get64(llsize), vaddr, ret, + strerror(-ret)); if (memory_region_is_ram_device(section->mr)) { /* Allow unexpected mappings not to be fatal for RAM devices */ error_report_err(err); @@ -1228,16 +1229,18 @@ static void vfio_listener_region_del(MemoryListener= *listener, ret =3D vfio_dma_unmap(container, iova, int128_get64(llsize), = NULL); if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") =3D %d (%m)", - container, iova, int128_get64(llsize), ret); + "0x%"HWADDR_PRIx") =3D %d (%s)", + container, iova, int128_get64(llsize), ret, + strerror(-ret)); } iova +=3D int128_get64(llsize); } ret =3D vfio_dma_unmap(container, iova, int128_get64(llsize), NULL= ); if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") =3D %d (%m)", - container, iova, int128_get64(llsize), ret); + "0x%"HWADDR_PRIx") =3D %d (%s)", + container, iova, int128_get64(llsize), ret, + strerror(-ret)); } } =20 @@ -1384,9 +1387,9 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifier= *n, IOMMUTLBEntry *iotlb) translated_addr); if (ret) { error_report("vfio_iommu_map_dirty_notify(%p, 0x%"HWADDR_PRIx"= , " - "0x%"HWADDR_PRIx") =3D %d (%m)", - container, iova, - iotlb->addr_mask + 1, ret); + "0x%"HWADDR_PRIx") =3D %d (%s)", + container, iova, iotlb->addr_mask + 1, ret, + strerror(-ret)); } } rcu_read_unlock(); --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193829; cv=none; d=zohomail.com; s=zohoarc; b=VqROoomZsmQGhwPfJaeeXMcQLhcPbYRg8pZScDgMAv5DiAZN5NBy+72kvDWbjCE8tLz0DBWCtXVQ8HmKtC2uibCUVCWnx+sApk5kIT5MXGTOC8cYUSQSxExwr54zDamDBklhZZcVYO6lnLyM4cl05Z+wZBdW3+Sb8TCx12yx1Co= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193829; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GiBCAPHsWsCaxFjIYhTKruwe4Yu7rgg+z8yv2dJvMcY=; b=fkNNEonQHUNFufGQGnwynVlQY+AeMyttX7rDncjBAXcoARHDDvbQhsica3HCTwOKJh6DMa9kIhFSeGk98B+Vldxd5c871n5u2vlxydlI1MApHrl+hW0cB8hcRZtA6h/CY2EK0HuTb9st5RwKWZyCRlKRTjwDGfljcH21ED7VsN4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193829900108.40289320876639; Tue, 7 Mar 2023 04:57:09 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWr3-0005Ol-Im; Tue, 07 Mar 2023 07:55:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr1-0005O9-UG for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:43 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr0-00053v-0b for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:43 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32784C8w001853; Tue, 7 Mar 2023 12:55:40 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p415hwe62-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:40 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327C0mPg027227; Tue, 7 Mar 2023 12:55:40 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyep5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:40 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV55004358; Tue, 7 Mar 2023 12:55:39 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-4; Tue, 07 Mar 2023 12:55:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=GiBCAPHsWsCaxFjIYhTKruwe4Yu7rgg+z8yv2dJvMcY=; b=nr+Ibn7vc3l4nMEbUKI45tsgvPEhmRzDjly3InI52LvyEfY0JpN8XeEB2nUDiFjTolR+ fwH2rHCw9zrEWiv1SHExtZljrE8nsI6v3mtjWdyMTTwO8nFwB/3GHmkES3sA1MudgpwS rdmPqFCpGW2oLy/CkPjJlQKlxeszxC5LHIQFfiXlqHnLeIRK7OU++AonRJggvEVZliuX UgV4lHZo8WD+/ox3qeI/PWhdaw6GLMWihJRTEV8THH/AaT/jqtPSBVfSPHzL+KoFAu8j btlxeJJuufkShF+lYrrJLUYfofE0K2/vbrOJF6kT8heCwTUakGrxYrUSbjACuZ2F/uju vA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 03/15] vfio/common: Abort migration if dirty log start/stop/sync fails Date: Tue, 7 Mar 2023 12:54:38 +0000 Message-Id: <20230307125450.62409-4-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: 9hz_l14gIcP9JT8SVbkwLzBYIWMZqian X-Proofpoint-ORIG-GUID: 9hz_l14gIcP9JT8SVbkwLzBYIWMZqian Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193830390100001 From: Avihai Horon If VFIO dirty pages log start/stop/sync fails during migration, migration should be aborted as pages dirtied by VFIO devices might not be reported properly. This is not the case today, where in such scenario only an error is printed. Fix it by aborting migration in the above scenario. Fixes: 758b96b61d5c ("vfio/migrate: Move switch of dirty tracking into vfio= _memory_listener") Fixes: b6dd6504e303 ("vfio: Add vfio_listener_log_sync to mark dirty pages") Fixes: 9e7b0442f23a ("vfio: Add ioctl to get dirty pages bitmap during dma = unmap") Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 53 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 45 insertions(+), 8 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 4d26e9cccf91..4c801513136a 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -42,6 +42,7 @@ #include "migration/migration.h" #include "migration/misc.h" #include "migration/blocker.h" +#include "migration/qemu-file.h" #include "sysemu/tpm.h" =20 VFIOGroupList vfio_group_list =3D @@ -390,6 +391,19 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker =3D NULL; } =20 +static void vfio_set_migration_error(int err) +{ + MigrationState *ms =3D migrate_get_current(); + + if (migration_is_setup_or_active(ms->state)) { + WITH_QEMU_LOCK_GUARD(&ms->qemu_file_lock) { + if (ms->to_dst_file) { + qemu_file_set_error(ms->to_dst_file, err); + } + } + } +} + static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) { VFIOGroup *group; @@ -680,6 +694,7 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOM= MUTLBEntry *iotlb) if (iotlb->target_as !=3D &address_space_memory) { error_report("Wrong target AS \"%s\", only system memory is allowe= d", iotlb->target_as->name ? iotlb->target_as->name : "no= ne"); + vfio_set_migration_error(-EINVAL); return; } =20 @@ -714,6 +729,7 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOM= MUTLBEntry *iotlb) "0x%"HWADDR_PRIx") =3D %d (%s)", container, iova, iotlb->addr_mask + 1, ret, strerror(-ret)); + vfio_set_migration_error(ret); } } out: @@ -1259,7 +1275,7 @@ static void vfio_listener_region_del(MemoryListener *= listener, } } =20 -static void vfio_set_dirty_page_tracking(VFIOContainer *container, bool st= art) +static int vfio_set_dirty_page_tracking(VFIOContainer *container, bool sta= rt) { int ret; struct vfio_iommu_type1_dirty_bitmap dirty =3D { @@ -1267,7 +1283,7 @@ static void vfio_set_dirty_page_tracking(VFIOContaine= r *container, bool start) }; =20 if (!container->dirty_pages_supported) { - return; + return 0; } =20 if (start) { @@ -1278,23 +1294,34 @@ static void vfio_set_dirty_page_tracking(VFIOContai= ner *container, bool start) =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, &dirty); if (ret) { + ret =3D -errno; error_report("Failed to set dirty tracking flag 0x%x errno: %d", dirty.flags, errno); } + + return ret; } =20 static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + int ret; =20 - vfio_set_dirty_page_tracking(container, true); + ret =3D vfio_set_dirty_page_tracking(container, true); + if (ret) { + vfio_set_migration_error(ret); + } } =20 static void vfio_listener_log_global_stop(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + int ret; =20 - vfio_set_dirty_page_tracking(container, false); + ret =3D vfio_set_dirty_page_tracking(container, false); + if (ret) { + vfio_set_migration_error(ret); + } } =20 static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, @@ -1370,19 +1397,18 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifi= er *n, IOMMUTLBEntry *iotlb) VFIOContainer *container =3D giommu->container; hwaddr iova =3D iotlb->iova + giommu->iommu_offset; ram_addr_t translated_addr; + int ret =3D -EINVAL; =20 trace_vfio_iommu_map_dirty_notify(iova, iova + iotlb->addr_mask); =20 if (iotlb->target_as !=3D &address_space_memory) { error_report("Wrong target AS \"%s\", only system memory is allowe= d", iotlb->target_as->name ? iotlb->target_as->name : "no= ne"); - return; + goto out; } =20 rcu_read_lock(); if (vfio_get_xlat_addr(iotlb, NULL, &translated_addr, NULL)) { - int ret; - ret =3D vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + = 1, translated_addr); if (ret) { @@ -1393,6 +1419,11 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifie= r *n, IOMMUTLBEntry *iotlb) } } rcu_read_unlock(); + +out: + if (ret) { + vfio_set_migration_error(ret); + } } =20 static int vfio_ram_discard_get_dirty_bitmap(MemoryRegionSection *section, @@ -1485,13 +1516,19 @@ static void vfio_listener_log_sync(MemoryListener *= listener, MemoryRegionSection *section) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + int ret; =20 if (vfio_listener_skipped_section(section)) { return; } =20 if (vfio_devices_all_dirty_tracking(container)) { - vfio_sync_dirty_bitmap(container, section); + ret =3D vfio_sync_dirty_bitmap(container, section); + if (ret) { + error_report("vfio: Failed to sync dirty bitmap, err: %d (%s)"= , ret, + strerror(-ret)); + vfio_set_migration_error(ret); + } } } =20 --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193810; cv=none; d=zohomail.com; s=zohoarc; b=YjcKu+3ksmVlc5dHeOjYmM9p29DSWL+w7JhZQtCyC/2hDn8Qq67Q53yF6ay9mGwTbZ8tE3Oi3St5hsMF0l/+8HYtluzFZ+HWBugzYH29m7RELgh7NUhrJ7Z7SLzZWPGUhX5Pp/PXbXsyNHU2Ygtuf9WjN5LWpB/HupKtILUsFM0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193810; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XOiVz9M/WFngl5j8R3/dZ/LeCA/WyHhtvPJBNq+AI8A=; b=OJmXr0APwn1QXvvPrpd8o3X4hDZEOjSb/dlOzk3r+JSV6VPbxc9pzoCpB+FvysG3O2yDIjl7E0B+mk9G2yty9Nc7vqwho0KGXgntTlp5+bdAObcDUw6Gwag3TNPXrCkW0dxicAOrwD3l9P7MzHUd8pSWUYe1p+15R+EcTWCu4k0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193810660905.7818063130907; Tue, 7 Mar 2023 04:56:50 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWr5-0005Ov-8s; Tue, 07 Mar 2023 07:55:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr4-0005Om-5I for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:46 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr2-00054D-BK for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:45 -0500 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 327846pE017250; Tue, 7 Mar 2023 12:55:43 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p417cdcsx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:43 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CUN5G026608; Tue, 7 Mar 2023 12:55:42 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyeq2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:42 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV57004358; Tue, 7 Mar 2023 12:55:42 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-5; Tue, 07 Mar 2023 12:55:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=XOiVz9M/WFngl5j8R3/dZ/LeCA/WyHhtvPJBNq+AI8A=; b=H/WJdmpX54a5V9qEJNG9ijzCeq5PKeqSew9i6562yoZh5vTK9LApAXpmbG1rxcW7rkE2 RLJ4SKXa/JxAUjhLuch+SO/OcEyz76a5Jhx1MAbkfiYd0LKRdhkoUgvnYTWxp5cqpras eYp5at5C+5Xi7KsDI2Moci4LOrq0HKHTXGiVWR1fJfypGZ+H2s71rdnNsrkhD9Qdb/FC yZ9d+KFcXO0vD0g4U/tWVUhKfrwrwOVftmAHEHjj4sHu97B2RWIZnVRcbq1VDRBHVWlM S94vRsu/oJFsYta9AUPaB4oopcP4YbSztGXCvj3/aV/QSzGthFi4Rp9gUkWg+8JIJHiq Lg== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 04/15] vfio/common: Add VFIOBitmap and alloc function Date: Tue, 7 Mar 2023 12:54:39 +0000 Message-Id: <20230307125450.62409-5-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=939 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: VB3w6WePYf-otBitPcI5ko9zihSZ5EsI X-Proofpoint-ORIG-GUID: VB3w6WePYf-otBitPcI5ko9zihSZ5EsI Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193812258100011 From: Avihai Horon There are already two places where dirty page bitmap allocation and calculations are done in open code. To avoid code duplication, introduce VFIOBitmap struct and corresponding alloc function and use them where applicable. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 73 +++++++++++++++++++++++++++++------------------- 1 file changed, 44 insertions(+), 29 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 4c801513136a..cec3de08d2b4 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -320,6 +320,25 @@ const MemoryRegionOps vfio_region_ops =3D { * Device state interfaces */ =20 +typedef struct { + unsigned long *bitmap; + hwaddr size; + hwaddr pages; +} VFIOBitmap; + +static int vfio_bitmap_alloc(VFIOBitmap *vbmap, hwaddr size) +{ + vbmap->pages =3D REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_size= (); + vbmap->size =3D ROUND_UP(vbmap->pages, sizeof(__u64) * BITS_PER_BYTE) / + BITS_PER_BYTE; + vbmap->bitmap =3D g_try_malloc0(vbmap->size); + if (!vbmap->bitmap) { + return -ENOMEM; + } + + return 0; +} + bool vfio_mig_active(void) { VFIOGroup *group; @@ -468,9 +487,14 @@ static int vfio_dma_unmap_bitmap(VFIOContainer *contai= ner, { struct vfio_iommu_type1_dma_unmap *unmap; struct vfio_bitmap *bitmap; - uint64_t pages =3D REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_si= ze(); + VFIOBitmap vbmap; int ret; =20 + ret =3D vfio_bitmap_alloc(&vbmap, size); + if (ret) { + return ret; + } + unmap =3D g_malloc0(sizeof(*unmap) + sizeof(*bitmap)); =20 unmap->argsz =3D sizeof(*unmap) + sizeof(*bitmap); @@ -484,35 +508,28 @@ static int vfio_dma_unmap_bitmap(VFIOContainer *conta= iner, * qemu_real_host_page_size to mark those dirty. Hence set bitmap_pgsi= ze * to qemu_real_host_page_size. */ - bitmap->pgsize =3D qemu_real_host_page_size(); - bitmap->size =3D ROUND_UP(pages, sizeof(__u64) * BITS_PER_BYTE) / - BITS_PER_BYTE; + bitmap->size =3D vbmap.size; + bitmap->data =3D (__u64 *)vbmap.bitmap; =20 - if (bitmap->size > container->max_dirty_bitmap_size) { - error_report("UNMAP: Size of bitmap too big 0x%"PRIx64, - (uint64_t)bitmap->size); + if (vbmap.size > container->max_dirty_bitmap_size) { + error_report("UNMAP: Size of bitmap too big 0x%"PRIx64, vbmap.size= ); ret =3D -E2BIG; goto unmap_exit; } =20 - bitmap->data =3D g_try_malloc0(bitmap->size); - if (!bitmap->data) { - ret =3D -ENOMEM; - goto unmap_exit; - } - ret =3D ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, unmap); if (!ret) { - cpu_physical_memory_set_dirty_lebitmap((unsigned long *)bitmap->da= ta, - iotlb->translated_addr, pages); + cpu_physical_memory_set_dirty_lebitmap(vbmap.bitmap, + iotlb->translated_addr, vbmap.pages); } else { error_report("VFIO_UNMAP_DMA with DIRTY_BITMAP : %m"); } =20 - g_free(bitmap->data); unmap_exit: g_free(unmap); + g_free(vbmap.bitmap); + return ret; } =20 @@ -1329,7 +1346,7 @@ static int vfio_get_dirty_bitmap(VFIOContainer *conta= iner, uint64_t iova, { struct vfio_iommu_type1_dirty_bitmap *dbitmap; struct vfio_iommu_type1_dirty_bitmap_get *range; - uint64_t pages; + VFIOBitmap vbmap; int ret; =20 if (!container->dirty_pages_supported) { @@ -1339,6 +1356,11 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, return 0; } =20 + ret =3D vfio_bitmap_alloc(&vbmap, size); + if (ret) { + return ret; + } + dbitmap =3D g_malloc0(sizeof(*dbitmap) + sizeof(*range)); =20 dbitmap->argsz =3D sizeof(*dbitmap) + sizeof(*range); @@ -1353,15 +1375,8 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, * to qemu_real_host_page_size. */ range->bitmap.pgsize =3D qemu_real_host_page_size(); - - pages =3D REAL_HOST_PAGE_ALIGN(range->size) / qemu_real_host_page_size= (); - range->bitmap.size =3D ROUND_UP(pages, sizeof(__u64) * BITS_PER_BYTE) / - BITS_PER_BYTE; - range->bitmap.data =3D g_try_malloc0(range->bitmap.size); - if (!range->bitmap.data) { - ret =3D -ENOMEM; - goto err_out; - } + range->bitmap.size =3D vbmap.size; + range->bitmap.data =3D (__u64 *)vbmap.bitmap; =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { @@ -1372,14 +1387,14 @@ static int vfio_get_dirty_bitmap(VFIOContainer *con= tainer, uint64_t iova, goto err_out; } =20 - cpu_physical_memory_set_dirty_lebitmap((unsigned long *)range->bitmap.= data, - ram_addr, pages); + cpu_physical_memory_set_dirty_lebitmap(vbmap.bitmap, ram_addr, + vbmap.pages); =20 trace_vfio_get_dirty_bitmap(container->fd, range->iova, range->size, range->bitmap.size, ram_addr); err_out: - g_free(range->bitmap.data); g_free(dbitmap); + g_free(vbmap.bitmap); =20 return ret; } --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678194056; cv=none; d=zohomail.com; s=zohoarc; b=gktBZrBjkc3oF3eKAEcleWTOG6xjV6fbqTonMKFzRs19i0wD0fHyXM54yiyooXOmPuAfLMGHK4Eyr+nvo5pfaI/COBtQQ70rtJfC+bzHiHTr2MVgggpWJ0Fl0bYL0XLrXyWkJP2xZJqd9dcDIKKm+dv+JZR4U1kr27JeWRNyNCE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678194056; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CtqlGVtMx510pMC0NN3oX37+wpQ2ZhL2IJt/65UoMNo=; b=LiOBYfBc+7zcuQmMH5q29EZGFJwOpJg4aQ3+Gifmy9feMU6bmeXSklclM4dMlLqf6ieJAU8Xo+zoI24rUfgob5KzagdAOUtmt+in+nrerYN1B/NM3GWpwBMeJXCzTpLOORJzT2MOEWfbsGMS/m5FzoL9+NJyvRdVULtJ6i9bebs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678194056027377.4960911990804; Tue, 7 Mar 2023 05:00:56 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWr8-0005PT-FY; Tue, 07 Mar 2023 07:55:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr6-0005PC-Bh for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:48 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr4-00054e-Ny for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:48 -0500 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32783owq016269; Tue, 7 Mar 2023 12:55:45 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p417cdct3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:45 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CSl2e028489; Tue, 7 Mar 2023 12:55:45 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyer6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:45 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV59004358; Tue, 7 Mar 2023 12:55:44 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-6; Tue, 07 Mar 2023 12:55:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=CtqlGVtMx510pMC0NN3oX37+wpQ2ZhL2IJt/65UoMNo=; b=dSDDCjugfGC9mTiSY+N24eBVxJe41i5rbprxlruIRWKMyjHkMP9BUuNdKz3WdpqKoYGo sl7xtVWBcnPgs7P4wnYk7eCVkLkspECUTak3ZpDhOGiExp2JgHyFTG+5sp5qtySZ3Aee olbNXokOW3pHwdVaYz7l/WMfykK9ABQTKzyIc5uUNBoIkMFjdITaTgrmsN607az0/WsN DP04mKLM5jhzDHPv0QzYjp2ACTLfQbw/j8HpoX6dME66nuU/i12WPrsL8onI6SSfO2LJ owha1bJcW0dVbP1xu4gxVjpELNi8eQQ83FBls2f8S2z87kvWOMw27GvgnJKl8E8FksHO vg== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 05/15] vfio/common: Add helper to validate iova/end against hostwin Date: Tue, 7 Mar 2023 12:54:40 +0000 Message-Id: <20230307125450.62409-6-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=774 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: Ga7Yt4DaHLEBv4wETyF7YJq1ZiBqnfQc X-Proofpoint-ORIG-GUID: Ga7Yt4DaHLEBv4wETyF7YJq1ZiBqnfQc Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678194058569100003 Move the code that finds the container host DMA window against a iova range. This avoids duplication on the common checks across listener callbacks. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Avihai Horon --- hw/vfio/common.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index cec3de08d2b4..99acb998eb14 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -901,6 +901,22 @@ static void vfio_unregister_ram_discard_listener(VFIOC= ontainer *container, g_free(vrdl); } =20 +static VFIOHostDMAWindow *vfio_find_hostwin(VFIOContainer *container, + hwaddr iova, hwaddr end) +{ + VFIOHostDMAWindow *hostwin; + bool hostwin_found =3D false; + + QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { + if (hostwin->min_iova <=3D iova && end <=3D hostwin->max_iova) { + hostwin_found =3D true; + break; + } + } + + return hostwin_found ? hostwin : NULL; +} + static bool vfio_known_safe_misalignment(MemoryRegionSection *section) { MemoryRegion *mr =3D section->mr; @@ -926,7 +942,6 @@ static void vfio_listener_region_add(MemoryListener *li= stener, void *vaddr; int ret; VFIOHostDMAWindow *hostwin; - bool hostwin_found; Error *err =3D NULL; =20 if (vfio_listener_skipped_section(section)) { @@ -1027,15 +1042,8 @@ static void vfio_listener_region_add(MemoryListener = *listener, #endif } =20 - hostwin_found =3D false; - QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { - if (hostwin->min_iova <=3D iova && end <=3D hostwin->max_iova) { - hostwin_found =3D true; - break; - } - } - - if (!hostwin_found) { + hostwin =3D vfio_find_hostwin(container, iova, end); + if (!hostwin) { error_setg(&err, "Container %p can't map guest IOVA region" " 0x%"HWADDR_PRIx"..0x%"HWADDR_PRIx, container, iova, e= nd); goto fail; @@ -1237,15 +1245,9 @@ static void vfio_listener_region_del(MemoryListener = *listener, if (memory_region_is_ram_device(section->mr)) { hwaddr pgmask; VFIOHostDMAWindow *hostwin; - bool hostwin_found =3D false; =20 - QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { - if (hostwin->min_iova <=3D iova && end <=3D hostwin->max_iova)= { - hostwin_found =3D true; - break; - } - } - assert(hostwin_found); /* or region_add() would have failed */ + hostwin =3D vfio_find_hostwin(container, iova, end); + assert(hostwin); /* or region_add() would have failed */ =20 pgmask =3D (1ULL << ctz64(hostwin->iova_pgsizes)) - 1; try_unmap =3D !((iova & pgmask) || (int128_get64(llsize) & pgmask)= ); --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193784; cv=none; d=zohomail.com; s=zohoarc; b=AS9W8pERyYppYxkur8xr1YljDTc88S8sS82zckj650G7wYFaQvJFjw/mIWTAgrLBZ45EOi8WbQZOuHBlffWN0YGHLHw/JZ4xhIjS6Pe+bc/d7eF0GPzqEnOOHGjFarGQSgHb77TzGOaZAcFmef8jH1waPMt40l7HtS3utcF9rJg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193784; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=X5el7/OT+rGDSa6e3+a+a+a518kxVGEBeqWOmUXCNOE=; b=ZZjqrEfWVwgOEzo4NYTpK9fJlS4p1kseprgDxXrOF9Cri8kLckvGxXMM41T0m+5vCOKrEiL1HmOn0XjRVebjMdGnM28KKuwGEk1p2ZqdeB7MgdaIyTaUowDzfbwQ2f/5votwPwvpawSL2hkUBK2FE6P8Jh6g0uoi5B80C0isX+M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193784075361.46607352162266; Tue, 7 Mar 2023 04:56:24 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrB-0005Y5-BA; Tue, 07 Mar 2023 07:55:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr9-0005QI-Au for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:51 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr7-000555-Mm for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:51 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3278422R001757; Tue, 7 Mar 2023 12:55:48 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p415hwe67-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:48 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327BuBse026621; Tue, 7 Mar 2023 12:55:47 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyesc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:47 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5B004358; Tue, 7 Mar 2023 12:55:47 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-7; Tue, 07 Mar 2023 12:55:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=X5el7/OT+rGDSa6e3+a+a+a518kxVGEBeqWOmUXCNOE=; b=HM81d2ZY1oUT4PYe/RtADJBy6ifBJNBihK2aJgFEGWm1cPSiUE8ddtPNu8pByfpWqlzT +epCVi1CLjryCmlRu4cUAWJroHGuxcRjEbwZlK+neoJUzNeE7g3WaAC2nb4qpwwK+CT/ cG3tNShqN4vXbKveBgifRsc41kYGVXN9Pm4OjemeZ77wG8+rrt9myCIglv4/rfZlFN72 L3adq60RzgBJdkBG+cNE3kOQ1paIQxBhK9EEXbOdKobZBg3UFY1++gUgphY+r3b2jjjA Z+lqJTcbD5bVDBpABtPN4E0FP03NhLPZ0+7HAKvaS82+xKk0p7MbIRHIlpCfLo346tR9 lg== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 06/15] vfio/common: Use a single tracepoint for skipped sections Date: Tue, 7 Mar 2023 12:54:41 +0000 Message-Id: <20230307125450.62409-7-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=834 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: BASHEQpoIkmy0Al17-GU1-bLOBNrkWz8 X-Proofpoint-ORIG-GUID: BASHEQpoIkmy0Al17-GU1-bLOBNrkWz8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193786212100003 Content-Type: text/plain; charset="utf-8" In preparation to turn more of the memory listener checks into common functions, one of the affected places is how we trace when sections are skipped. Right now there is one for each. Change it into one single tracepoint `vfio_listener_region_skip` which receives a name which refers to the callback i.e. region_add and region_del. Suggested-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 4 ++-- hw/vfio/trace-events | 3 +-- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 99acb998eb14..1cb62efa9743 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -945,7 +945,7 @@ static void vfio_listener_region_add(MemoryListener *li= stener, Error *err =3D NULL; =20 if (vfio_listener_skipped_section(section)) { - trace_vfio_listener_region_add_skip( + trace_vfio_listener_region_skip("region_add", section->offset_within_address_space, section->offset_within_address_space + int128_get64(int128_sub(section->size, int128_one()))); @@ -1183,7 +1183,7 @@ static void vfio_listener_region_del(MemoryListener *= listener, bool try_unmap =3D true; =20 if (vfio_listener_skipped_section(section)) { - trace_vfio_listener_region_del_skip( + trace_vfio_listener_region_skip("region_del", section->offset_within_address_space, section->offset_within_address_space + int128_get64(int128_sub(section->size, int128_one()))); diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 669d9fe07cd9..7173e6a5c721 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -96,13 +96,12 @@ vfio_pci_igd_lpc_bridge_enabled(const char *name) "%s" vfio_region_write(const char *name, int index, uint64_t addr, uint64_t dat= a, unsigned size) " (%s:region%d+0x%"PRIx64", 0x%"PRIx64 ", %d)" vfio_region_read(char *name, int index, uint64_t addr, unsigned size, uint= 64_t data) " (%s:region%d+0x%"PRIx64", %d) =3D 0x%"PRIx64 vfio_iommu_map_notify(const char *op, uint64_t iova_start, uint64_t iova_e= nd) "iommu %s @ 0x%"PRIx64" - 0x%"PRIx64 -vfio_listener_region_add_skip(uint64_t start, uint64_t end) "SKIPPING regi= on_add 0x%"PRIx64" - 0x%"PRIx64 +vfio_listener_region_skip(const char *name, uint64_t start, uint64_t end) = "SKIPPING %s 0x%"PRIx64" - 0x%"PRIx64 vfio_spapr_group_attach(int groupfd, int tablefd) "Attached groupfd %d to = liobn fd %d" vfio_listener_region_add_iommu(uint64_t start, uint64_t end) "region_add [= iommu] 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_add_ram(uint64_t iova_start, uint64_t iova_end, void = *vaddr) "region_add [ram] 0x%"PRIx64" - 0x%"PRIx64" [%p]" vfio_known_safe_misalignment(const char *name, uint64_t iova, uint64_t off= set_within_region, uintptr_t page_size) "Region \"%s\" iova=3D0x%"PRIx64" o= ffset_within_region=3D0x%"PRIx64" qemu_real_host_page_size=3D0x%"PRIxPTR vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint6= 4_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=3D0x%"PRIx64"= is not aligned to 0x%"PRIx64" and cannot be mapped for DMA" -vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING regi= on_del 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRI= x64" - 0x%"PRIx64 vfio_disconnect_container(int fd) "close container->fd=3D%d" vfio_put_group(int fd) "close group->fd=3D%d" --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193972; cv=none; d=zohomail.com; s=zohoarc; b=G3anl57EWXXcbUiuCVd0ScKVbe3O+7aUkJAJaP38QpqpjmOhtnZIXo3drghYh+eqCPL3NNRPlEMSimYyz4NZL1hIIqo3OS2jn3O58wFcmIHaiJo8GA3Yz0zCGTjIq7sWT+Aa9BqIoQtEA/XDOr5N3LlNdCEKwEkLbrEzbHHZiVU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193972; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FJh6jNJWa7Qb5orBtnaLlaUFUNsYDGsuxIcR1IIPg54=; b=YPLXjprDpQ9ZgAcxL+IV2m6dO6zYofc2a2WEKMGFEZ+62gqgVcHJVNrJ1Z0L7NdfF7hMDfqpijJID1nt7WVhYmOldVD9elwFJhsNF9rzLm4rk7rbksjS+NK1pBI3j4YDRA3/PbHsN3MNca1wiPui3PDXr4/HxhfO2ciozOzfa/I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193972595831.2773112188509; Tue, 7 Mar 2023 04:59:32 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrD-0005aW-27; Tue, 07 Mar 2023 07:55:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrB-0005YU-D6 for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:53 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWr9-00055e-Q9 for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:53 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32783n3T001667; Tue, 7 Mar 2023 12:55:51 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p415hwe6a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:50 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CEbFY026636; Tue, 7 Mar 2023 12:55:50 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyetf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:50 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5D004358; Tue, 7 Mar 2023 12:55:49 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-8; Tue, 07 Mar 2023 12:55:49 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=FJh6jNJWa7Qb5orBtnaLlaUFUNsYDGsuxIcR1IIPg54=; b=cs5ZBr64MOppP+01k2GG8MGnH1hufhn2LTRFEnhuY3QFzg0dsvymDXCgZb9MU7peyAHE CzINNV7f2n7MTJNJZbLoPHcl/jINJP8oP4m07bboKn35NPOAKyXYZTsxdo5anj5lsb3h t0A7sTmsGKXYUwhrfROSl1Xdj4zdxrCrOPJKVaDE/SuDBkKBFKTFc77w3rmriYepMeab rNFWRbYQO30Cu7zGCk1mQvV6RfkIRV7KjO4EQgnmOd9gZlNRepihyKqZHU+wtn3wbpoi O6qJf56e0csQ63936oevyJayUjwp/zZjvmoB2XBEEeaDq5fEPKoCutLb9TaEosFGf9lg Xw== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 07/15] vfio/common: Consolidate skip/invalid section into helper Date: Tue, 7 Mar 2023 12:54:42 +0000 Message-Id: <20230307125450.62409-8-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=761 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: 3GG2lNlQa8w13A5s19nkXH9zWcbGcdy1 X-Proofpoint-ORIG-GUID: 3GG2lNlQa8w13A5s19nkXH9zWcbGcdy1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193974201100001 The checks are replicated against region_add and region_del and will be soon added in another memory listener dedicated for dirty tracking. Move these into a new helper for avoid duplication. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Avihai Horon --- hw/vfio/common.c | 55 ++++++++++++++++++++---------------------------- 1 file changed, 23 insertions(+), 32 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 1cb62efa9743..d38b7c1969b9 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -933,23 +933,15 @@ static bool vfio_known_safe_misalignment(MemoryRegion= Section *section) return true; } =20 -static void vfio_listener_region_add(MemoryListener *listener, - MemoryRegionSection *section) +static bool vfio_listener_valid_section(MemoryRegionSection *section, + const char *name) { - VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); - hwaddr iova, end; - Int128 llend, llsize; - void *vaddr; - int ret; - VFIOHostDMAWindow *hostwin; - Error *err =3D NULL; - if (vfio_listener_skipped_section(section)) { - trace_vfio_listener_region_skip("region_add", + trace_vfio_listener_region_skip(name, section->offset_within_address_space, section->offset_within_address_space + int128_get64(int128_sub(section->size, int128_one()))); - return; + return false; } =20 if (unlikely((section->offset_within_address_space & @@ -964,6 +956,24 @@ static void vfio_listener_region_add(MemoryListener *l= istener, section->offset_within_region, qemu_real_host_page_size()); } + return false; + } + + return true; +} + +static void vfio_listener_region_add(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + hwaddr iova, end; + Int128 llend, llsize; + void *vaddr; + int ret; + VFIOHostDMAWindow *hostwin; + Error *err =3D NULL; + + if (!vfio_listener_valid_section(section, "region_add")) { return; } =20 @@ -1182,26 +1192,7 @@ static void vfio_listener_region_del(MemoryListener = *listener, int ret; bool try_unmap =3D true; =20 - if (vfio_listener_skipped_section(section)) { - trace_vfio_listener_region_skip("region_del", - section->offset_within_address_space, - section->offset_within_address_space + - int128_get64(int128_sub(section->size, int128_one()))); - return; - } - - if (unlikely((section->offset_within_address_space & - ~qemu_real_host_page_mask()) !=3D - (section->offset_within_region & ~qemu_real_host_page_mas= k()))) { - if (!vfio_known_safe_misalignment(section)) { - error_report("%s received unaligned region %s iova=3D0x%"PRIx64 - " offset_within_region=3D0x%"PRIx64 - " qemu_real_host_page_size=3D0x%"PRIxPTR, - __func__, memory_region_name(section->mr), - section->offset_within_address_space, - section->offset_within_region, - qemu_real_host_page_size()); - } + if (!vfio_listener_valid_section(section, "region_del")) { return; } =20 --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193809; cv=none; d=zohomail.com; s=zohoarc; b=TRyfVYLMX8o6XCn3M+qINc7Likmvk0ZDOZkVSlpLrGNVJsNDX35BmfBsDcgYM3myXKxsouxW5Xzbsfnl9gDBSDjO6s4IBh3FbgltrGSIQMOQ2igpS4okitFlj2xbrjOD7EqkH/e4ks3OO8Ru+7iyTGFORwRoSxKxUlEuk4PxO0k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193809; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=pUY4inpih/zRR+cMVCfCJujZdxgKM9m3KCfKyXD/GO4=; b=Jl4EwSwbaEta3HbZThQj6U2lKRvAXIBzS7QfnYQfGZQIf+EsZqSNHNUkWAVl4WLL5+SksFymQUOMHINKDecC77V/HPWj4udmr66HAXcBUFYTuChOBuwlrAzZIzA1EavnAN2xDR50bazd+fpgQ8dgtineo2eHqxLzlYy9FmurE0U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193809042273.1041042136362; Tue, 7 Mar 2023 04:56:49 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrF-0005bV-3E; Tue, 07 Mar 2023 07:55:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrE-0005an-4H for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:56 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrC-000565-ID for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:55 -0500 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32783eLP003670; Tue, 7 Mar 2023 12:55:53 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p4168nd0s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:53 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CFJT7026620; Tue, 7 Mar 2023 12:55:53 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyeus-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:53 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5F004358; Tue, 7 Mar 2023 12:55:52 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-9; Tue, 07 Mar 2023 12:55:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=pUY4inpih/zRR+cMVCfCJujZdxgKM9m3KCfKyXD/GO4=; b=DenzauD8IPH+U82xD1w5/pnwSXptk2PaldCWq84+irvVwEOGYuotxrWD51aBfJ3qE9V4 vtsd44AfTwI0ZvFlkqFHTvzh3U8mTbMsjLTCjkSUa+aCw6un72bZJndOb0TdX0/P+D5k /LKoQdK8cew7d0K1FrM+hf/Sfmq3ClH6/RCyxDX6vDgpd+WGs1qs8+y0dopNehaF+moG XtVdtxTj65R6oCDlYJE8DG5jhgO9vm+aVnj2Bz4jbRnLpV7L3O/RMuzEiHv1QMqcoWs9 ISEcwEDDax4mfCK0ami6L205NWMoOxUI16MNlyJbyORVlWuItjSnHhffoLeGjl7eSGvO Ag== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 08/15] vfio/common: Add helper to consolidate iova/end calculation Date: Tue, 7 Mar 2023 12:54:43 +0000 Message-Id: <20230307125450.62409-9-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=796 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: Oa5tN6nPEee12GHEKFNuBHnkYx-mZ8xc X-Proofpoint-ORIG-GUID: Oa5tN6nPEee12GHEKFNuBHnkYx-mZ8xc Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193810250100007 Content-Type: text/plain; charset="utf-8" In preparation to be used in device dirty tracking, move the code that calculate a iova/end range from the container/section. This avoids duplication on the common checks across listener callbacks. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 41 +++++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 14 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index d38b7c1969b9..63831eab78a1 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -962,6 +962,31 @@ static bool vfio_listener_valid_section(MemoryRegionSe= ction *section, return true; } =20 +static bool vfio_get_section_iova_range(VFIOContainer *container, + MemoryRegionSection *section, + hwaddr *out_iova, hwaddr *out_end, + Int128 *out_llend) +{ + Int128 llend; + hwaddr iova; + + iova =3D REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); + llend =3D int128_make64(section->offset_within_address_space); + llend =3D int128_add(llend, section->size); + llend =3D int128_and(llend, int128_exts64(qemu_real_host_page_mask())); + + if (int128_ge(int128_make64(iova), llend)) { + return false; + } + + *out_iova =3D iova; + *out_end =3D int128_get64(int128_sub(llend, int128_one())); + if (out_llend) { + *out_llend =3D llend; + } + return true; +} + static void vfio_listener_region_add(MemoryListener *listener, MemoryRegionSection *section) { @@ -977,12 +1002,7 @@ static void vfio_listener_region_add(MemoryListener *= listener, return; } =20 - iova =3D REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); - llend =3D int128_make64(section->offset_within_address_space); - llend =3D int128_add(llend, section->size); - llend =3D int128_and(llend, int128_exts64(qemu_real_host_page_mask())); - - if (int128_ge(int128_make64(iova), llend)) { + if (!vfio_get_section_iova_range(container, section, &iova, &end, &lle= nd)) { if (memory_region_is_ram_device(section->mr)) { trace_vfio_listener_region_add_no_dma_map( memory_region_name(section->mr), @@ -992,7 +1012,6 @@ static void vfio_listener_region_add(MemoryListener *l= istener, } return; } - end =3D int128_get64(int128_sub(llend, int128_one())); =20 if (container->iommu_type =3D=3D VFIO_SPAPR_TCE_v2_IOMMU) { hwaddr pgsize =3D 0; @@ -1219,15 +1238,9 @@ static void vfio_listener_region_del(MemoryListener = *listener, */ } =20 - iova =3D REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); - llend =3D int128_make64(section->offset_within_address_space); - llend =3D int128_add(llend, section->size); - llend =3D int128_and(llend, int128_exts64(qemu_real_host_page_mask())); - - if (int128_ge(int128_make64(iova), llend)) { + if (!vfio_get_section_iova_range(container, section, &iova, &end, &lle= nd)) { return; } - end =3D int128_get64(int128_sub(llend, int128_one())); =20 llsize =3D int128_sub(llend, int128_make64(iova)); =20 --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193852; cv=none; d=zohomail.com; s=zohoarc; b=j14s8rQ17ntNKea4AC+Ns0vEHIg/2rWySqc04g5o5e+WmFt5FYKdKs0HwXMJ0Cl9o08iSR0BstumhTkPwTkfif4f+fx9jHO2IwWDWoocLq705WiK8whDGZQmnYAvZg/BHh9ihILw0plw0/3tnHWvoVYF8S9rNt/p7fOQXNnZaj0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193852; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lqN+cTyBJoQc9qOSubsecppyC3ndUXvrVoAak4ksuik=; b=bwAaA7S6fQS74SbXa/q8MqTr9m7bHVOgcKV0YMmdLDTtIQm+m8mm0Y5KJ3+ygeXMbGHfkb4E+1Vdyf6KpIyyz+QMl28Lhck2C54fZPlXkMPPLacUepK84XBmE0cz5t0L3DH6qWcS7IxqlYaLGGmHMWZxM0BMrrZhLLPYowqr5Tw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193852993120.52977590105172; Tue, 7 Mar 2023 04:57:32 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrI-0005cL-OD; Tue, 07 Mar 2023 07:56:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrG-0005bj-Sg for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:58 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrF-00056P-4E for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:55:58 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32783kni010510; Tue, 7 Mar 2023 12:55:56 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p4180wbkx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:56 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CHHod029152; Tue, 7 Mar 2023 12:55:55 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyevw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:55 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5H004358; Tue, 7 Mar 2023 12:55:55 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-10; Tue, 07 Mar 2023 12:55:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=lqN+cTyBJoQc9qOSubsecppyC3ndUXvrVoAak4ksuik=; b=Pd6z3OP+sLePZHqnHn//hFFZ+GnnsGMoaKlVFZs9+D1+tFAO9HqbWbR9MbXfYIzCcMZE 2a1xABTE/7zncbvxd3gDH2/Cc/1emZ7PWGQccyhH/ssLE671GGT5nXxbOg+RybK0F0NE ZFq9L/0dkpZoY5IrmWpUnYAzo7r/bo7FUoK2So15P+qmfxODN7wVqhvT10XB9uSPhm8z 8G7QL1du8PKR9PuyJa40uptks89OBlXPDJqVRJiEUh2IIhHU3nZOBSa3Sd+4tCqn+I4M 861Yngr16kwSjnAMUdjVLo6jDUmMlEjwlQ1U65d9uKMRFiGP2yPcsCLmcPERZ2+eeqDM aA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 09/15] vfio/common: Record DMA mapped IOVA ranges Date: Tue, 7 Mar 2023 12:54:44 +0000 Message-Id: <20230307125450.62409-10-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=832 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-ORIG-GUID: aEekxBcub_gwuuocutQCB9uPnjtisS4s X-Proofpoint-GUID: aEekxBcub_gwuuocutQCB9uPnjtisS4s Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193854547100006 Content-Type: text/plain; charset="utf-8" According to the device DMA logging uAPI, IOVA ranges to be logged by the device must be provided all at once upon DMA logging start. As preparation for the following patches which will add device dirty page tracking, keep a record of all DMA mapped IOVA ranges so later they can be used for DMA logging start. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 85 ++++++++++++++++++++++++++++++++++++++++++++ hw/vfio/trace-events | 1 + 2 files changed, 86 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 63831eab78a1..811502dbc97c 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1325,11 +1325,96 @@ static int vfio_set_dirty_page_tracking(VFIOContain= er *container, bool start) return ret; } =20 +typedef struct VFIODirtyRanges { + hwaddr min32; + hwaddr max32; + hwaddr min64; + hwaddr max64; +} VFIODirtyRanges; + +typedef struct VFIODirtyRangesListener { + VFIOContainer *container; + VFIODirtyRanges ranges; + MemoryListener listener; +} VFIODirtyRangesListener; + +static void vfio_dirty_tracking_update(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIODirtyRangesListener *dirty =3D container_of(listener, + VFIODirtyRangesListener, + listener); + VFIODirtyRanges *range =3D &dirty->ranges; + hwaddr iova, end, *min, *max; + + if (!vfio_listener_valid_section(section, "tracking_update") || + !vfio_get_section_iova_range(dirty->container, section, + &iova, &end, NULL)) { + return; + } + + /* + * The address space passed to the dirty tracker is reduced to two ran= ges: + * one for 32-bit DMA ranges, and another one for 64-bit DMA ranges. + * The underlying reports of dirty will query a sub-interval of each of + * these ranges. + * + * The purpose of the dual range handling is to handle known cases of = big + * holes in the address space, like the x86 AMD 1T hole. The alternati= ve + * would be an IOVATree but that has a much bigger runtime overhead and + * unnecessary complexity. + */ + min =3D (end <=3D UINT32_MAX) ? &range->min32 : &range->min64; + max =3D (end <=3D UINT32_MAX) ? &range->max32 : &range->max64; + + if (*min > iova) { + *min =3D iova; + } + if (*max < end) { + *max =3D end; + } + + trace_vfio_device_dirty_tracking_update(iova, end, *min, *max); + return; +} + +static const MemoryListener vfio_dirty_tracking_listener =3D { + .name =3D "vfio-tracking", + .region_add =3D vfio_dirty_tracking_update, +}; + +static void vfio_dirty_tracking_init(VFIOContainer *container, + VFIODirtyRanges *ranges) +{ + VFIODirtyRangesListener dirty; + + memset(&dirty, 0, sizeof(dirty)); + dirty.ranges.min32 =3D UINT32_MAX; + dirty.ranges.min64 =3D UINT64_MAX; + dirty.listener =3D vfio_dirty_tracking_listener; + dirty.container =3D container; + + memory_listener_register(&dirty.listener, + container->space->as); + + *ranges =3D dirty.ranges; + + /* + * The memory listener is synchronous, and used to calculate the range + * to dirty tracking. Unregister it after we are done as we are not + * interested in any follow-up updates. + */ + memory_listener_unregister(&dirty.listener); +} + static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + VFIODirtyRanges ranges; int ret; =20 + vfio_dirty_tracking_init(container, &ranges); + ret =3D vfio_set_dirty_page_tracking(container, true); if (ret) { vfio_set_migration_error(ret); diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 7173e6a5c721..dd9fd7b9bddb 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -103,6 +103,7 @@ vfio_listener_region_add_ram(uint64_t iova_start, uint6= 4_t iova_end, void *vaddr vfio_known_safe_misalignment(const char *name, uint64_t iova, uint64_t off= set_within_region, uintptr_t page_size) "Region \"%s\" iova=3D0x%"PRIx64" o= ffset_within_region=3D0x%"PRIx64" qemu_real_host_page_size=3D0x%"PRIxPTR vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint6= 4_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=3D0x%"PRIx64"= is not aligned to 0x%"PRIx64" and cannot be mapped for DMA" vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRI= x64" - 0x%"PRIx64 +vfio_device_dirty_tracking_update(uint64_t start, uint64_t end, uint64_t m= in, uint64_t max) "section 0x%"PRIx64" - 0x%"PRIx64" -> update [0x%"PRIx64"= - 0x%"PRIx64"]" vfio_disconnect_container(int fd) "close container->fd=3D%d" vfio_put_group(int fd) "close group->fd=3D%d" vfio_get_device(const char * name, unsigned int flags, unsigned int num_re= gions, unsigned int num_irqs) "Device %s flags: %u, regions: %u, irqs: %u" --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193884; cv=none; d=zohomail.com; s=zohoarc; b=QVfpGHA7+oVd17H+6yNdEGlhRv2pxvzF2CgsU3agDUq08O9ocBowbkBY01urSkQLwmKvJNogkyYWtrw4jsJDjKExdnVXaIcTe161fOS4VIzhRmmPiz1b9YRzgSO8LEMc5R17D4MV2vlq1TgWpswaH/d61RzVKhsP+39dGwvqcJA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193884; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0lWNdWx6MxNPIE2Qu6RGMbbkkiDNDX1OB7YbQlTHUNg=; b=aRtnHNeY2hBFNS8gDy8Darwmt0hpsegvS5LEWsoYa9sZl9E+M7TODG1ktAw+QbpVfy1Z9O7XL3BgIA5jALFFGCsRlN0KbF6aVtIq854YnArufSPReUUv9Jz413x3j4NSmI6gGDBWCdLKOFQ5a5gaOrLbiACCyBocvj8DXrzu16E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193884946965.7964912877025; Tue, 7 Mar 2023 04:58:04 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrL-0005ca-GM; Tue, 07 Mar 2023 07:56:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrK-0005cQ-BX for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:02 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrI-00056n-9Z for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:01 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 327849NZ001821; Tue, 7 Mar 2023 12:55:59 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p415hwe6h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:59 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327C7XHu026640; Tue, 7 Mar 2023 12:55:58 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyexj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:55:58 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5J004358; Tue, 7 Mar 2023 12:55:57 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-11; Tue, 07 Mar 2023 12:55:57 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=0lWNdWx6MxNPIE2Qu6RGMbbkkiDNDX1OB7YbQlTHUNg=; b=jsZgYt2bW/wr8Wu2Yp/Wrc+fCHBsGo0Mh6jWj82LNzqKq+y46B+/ZaxNrRxGabtfh5kZ 72oOK3gPIYzucurMg24H6sXQjpqHeWezFtYfo93XzzAwS/2WTIid07KREg3CiKL0z8hi XV8TvkF5oqxX9dAMstjrCEOxLow9xxg5Esq/hZ4aoPmwYAscCBmV+L6S8bQtNxmsyuT/ dQU1DpgoraWNJlI31cAngk1xeccmck+1C6OeLtg1zz2qwarozXpOV1JivZEgDeog8h66 MAlU6xLt+qVVB+Nnug0u5aAmsHQqf83wju11OhjaFpTwRa49nAOF0Hq6MaXL/ONJfHmb UQ== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 10/15] vfio/common: Add device dirty page tracking start/stop Date: Tue, 7 Mar 2023 12:54:45 +0000 Message-Id: <20230307125450.62409-11-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=751 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: -l3ak8hDJyrgO_lKaW6r4vyCjjkQQSSM X-Proofpoint-ORIG-GUID: -l3ak8hDJyrgO_lKaW6r4vyCjjkQQSSM Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193886802100001 Content-Type: text/plain; charset="utf-8" Add device dirty page tracking start/stop functionality. This uses the device DMA logging uAPI to start and stop dirty page tracking by device. Device dirty page tracking is used only if all devices within a container support device dirty page tracking. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 173 +++++++++++++++++++++++++++++++++- hw/vfio/trace-events | 1 + include/hw/vfio/vfio-common.h | 2 + 3 files changed, 171 insertions(+), 5 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 811502dbc97c..80f2d287bab5 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -450,6 +450,22 @@ static bool vfio_devices_all_dirty_tracking(VFIOContai= ner *container) return true; } =20 +static bool vfio_devices_all_device_dirty_tracking(VFIOContainer *containe= r) +{ + VFIOGroup *group; + VFIODevice *vbasedev; + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (!vbasedev->dirty_pages_supported) { + return false; + } + } + } + + return true; +} + /* * Check if all VFIO devices are running and migration is active, which is * essentially equivalent to the migration being in pre-copy phase. @@ -1407,16 +1423,156 @@ static void vfio_dirty_tracking_init(VFIOContainer= *container, memory_listener_unregister(&dirty.listener); } =20 +static void vfio_devices_dma_logging_stop(VFIOContainer *container) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + VFIODevice *vbasedev; + VFIOGroup *group; + + feature->argsz =3D sizeof(buf); + feature->flags =3D VFIO_DEVICE_FEATURE_SET | + VFIO_DEVICE_FEATURE_DMA_LOGGING_STOP; + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (!vbasedev->dirty_tracking) { + continue; + } + + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + warn_report("%s: Failed to stop DMA logging, err %d (%s)", + vbasedev->name, -errno, strerror(errno)); + } + vbasedev->dirty_tracking =3D false; + } + } +} + +static struct vfio_device_feature * +vfio_device_feature_dma_logging_start_create(VFIOContainer *container, + VFIODirtyRanges *tracking) +{ + struct vfio_device_feature *feature; + size_t feature_size; + struct vfio_device_feature_dma_logging_control *control; + struct vfio_device_feature_dma_logging_range *ranges; + + feature_size =3D sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_dma_logging_control); + feature =3D g_try_malloc0(feature_size); + if (!feature) { + errno =3D ENOMEM; + return NULL; + } + feature->argsz =3D feature_size; + feature->flags =3D VFIO_DEVICE_FEATURE_SET | + VFIO_DEVICE_FEATURE_DMA_LOGGING_START; + + control =3D (struct vfio_device_feature_dma_logging_control *)feature-= >data; + control->page_size =3D qemu_real_host_page_size(); + + /* + * DMA logging uAPI guarantees to support at least a number of ranges = that + * fits into a single host kernel base page. + */ + control->num_ranges =3D !!tracking->max32 + !!tracking->max64; + ranges =3D g_try_new0(struct vfio_device_feature_dma_logging_range, + control->num_ranges); + if (!ranges) { + g_free(feature); + errno =3D ENOMEM; + + return NULL; + } + + control->ranges =3D (__u64)(uintptr_t)ranges; + if (tracking->max32) { + ranges->iova =3D tracking->min32; + ranges->length =3D (tracking->max32 - tracking->min32) + 1; + ranges++; + } + if (tracking->max64) { + ranges->iova =3D tracking->min64; + ranges->length =3D (tracking->max64 - tracking->min64) + 1; + } + + trace_vfio_device_dirty_tracking_start(control->num_ranges, + tracking->min32, tracking->max3= 2, + tracking->min64, tracking->max6= 4); + + return feature; +} + +static void vfio_device_feature_dma_logging_start_destroy( + struct vfio_device_feature *feature) +{ + struct vfio_device_feature_dma_logging_control *control =3D + (struct vfio_device_feature_dma_logging_control *)feature->data; + struct vfio_device_feature_dma_logging_range *ranges =3D + (struct vfio_device_feature_dma_logging_range *)(uintptr_t) contro= l->ranges; + + g_free(ranges); + g_free(feature); +} + +static int vfio_devices_dma_logging_start(VFIOContainer *container) +{ + struct vfio_device_feature *feature; + VFIODirtyRanges ranges; + VFIODevice *vbasedev; + VFIOGroup *group; + int ret =3D 0; + + vfio_dirty_tracking_init(container, &ranges); + feature =3D vfio_device_feature_dma_logging_start_create(container, + &ranges); + if (!feature) { + return -errno; + } + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (vbasedev->dirty_tracking) { + continue; + } + + ret =3D ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); + if (ret) { + ret =3D -errno; + error_report("%s: Failed to start DMA logging, err %d (%s)= ", + vbasedev->name, ret, strerror(errno)); + goto out; + } + vbasedev->dirty_tracking =3D true; + } + } + +out: + if (ret) { + vfio_devices_dma_logging_stop(container); + } + + vfio_device_feature_dma_logging_start_destroy(feature); + + return ret; +} + static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); - VFIODirtyRanges ranges; int ret; =20 - vfio_dirty_tracking_init(container, &ranges); + if (vfio_devices_all_device_dirty_tracking(container)) { + ret =3D vfio_devices_dma_logging_start(container); + } else { + ret =3D vfio_set_dirty_page_tracking(container, true); + } =20 - ret =3D vfio_set_dirty_page_tracking(container, true); if (ret) { + error_report("vfio: Could not start dirty page tracking, err: %d (= %s)", + ret, strerror(-ret)); vfio_set_migration_error(ret); } } @@ -1424,10 +1580,17 @@ static void vfio_listener_log_global_start(MemoryLi= stener *listener) static void vfio_listener_log_global_stop(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); - int ret; + int ret =3D 0; + + if (vfio_devices_all_device_dirty_tracking(container)) { + vfio_devices_dma_logging_stop(container); + } else { + ret =3D vfio_set_dirty_page_tracking(container, false); + } =20 - ret =3D vfio_set_dirty_page_tracking(container, false); if (ret) { + error_report("vfio: Could not stop dirty page tracking, err: %d (%= s)", + ret, strerror(-ret)); vfio_set_migration_error(ret); } } diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index dd9fd7b9bddb..bee95dbd977a 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -104,6 +104,7 @@ vfio_known_safe_misalignment(const char *name, uint64_t= iova, uint64_t offset_wi vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint6= 4_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=3D0x%"PRIx64"= is not aligned to 0x%"PRIx64" and cannot be mapped for DMA" vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRI= x64" - 0x%"PRIx64 vfio_device_dirty_tracking_update(uint64_t start, uint64_t end, uint64_t m= in, uint64_t max) "section 0x%"PRIx64" - 0x%"PRIx64" -> update [0x%"PRIx64"= - 0x%"PRIx64"]" +vfio_device_dirty_tracking_start(int nr_ranges, uint64_t min32, uint64_t m= ax32, uint64_t min64, uint64_t max64) "nr_ranges %d 32:[0x%"PRIx64" - 0x%"P= RIx64"], 64:[0x%"PRIx64" - 0x%"PRIx64"]" vfio_disconnect_container(int fd) "close container->fd=3D%d" vfio_put_group(int fd) "close group->fd=3D%d" vfio_get_device(const char * name, unsigned int flags, unsigned int num_re= gions, unsigned int num_irqs) "Device %s flags: %u, regions: %u, irqs: %u" diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 87524c64a443..9551d2d43025 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -143,6 +143,8 @@ typedef struct VFIODevice { VFIOMigration *migration; Error *migration_blocker; OnOffAuto pre_copy_dirty_page_tracking; + bool dirty_pages_supported; + bool dirty_tracking; } VFIODevice; =20 struct VFIODeviceOps { --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193908; cv=none; d=zohomail.com; s=zohoarc; b=J6lNbvuQd8C/Z9ogXhaVPwsf7ijRKESbtnZJwA2Kn95ZQsnoYpgXBUO+fDz4MiPfF0mpvnLepWQPqVk3jNRm8b6oDh2zqxpJUTB4rg/cwV8WRTsdew/x2VrrWd79N9LzFq6n9EMupisKaKXo4gR3IvaB2wt1qCcRsivhx1kh4lk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193908; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Rt2b32vQO/ssvEuuBTSH7ieidjSb2j+Cd5NM/HugcX4=; b=I6Ehjh55jKKqOXK+P6zxHxkjNyDPYsNIxxWqgjjIlDrD55JF8OBuizAcKluUdaOBt2fAUfbBVDhdsgrwwjt9vxaBhsPR5enHRm6Qmlz+kxRL83EiiFdtWlhrTSzWmkHN13UzUzwhjxbHXnDk7wHV+xUPxDP9wQ6wLY45nrOWcmA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTP id 1678193908641236.98456623261745; Tue, 7 Mar 2023 04:58:28 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrN-0005pg-Ct; Tue, 07 Mar 2023 07:56:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrM-0005fw-4c for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:04 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrK-00056t-HX for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:03 -0500 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32783h3x003689; Tue, 7 Mar 2023 12:56:01 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p4168nd0y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:01 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CQ3vu028608; Tue, 7 Mar 2023 12:56:01 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyf0b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:00 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5L004358; Tue, 7 Mar 2023 12:56:00 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-12; Tue, 07 Mar 2023 12:56:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=Rt2b32vQO/ssvEuuBTSH7ieidjSb2j+Cd5NM/HugcX4=; b=bp1gXwUHOPxDH1Kn6nCZ3UT8M73RA+rtaU7zNVKN/c5u4BEhC11aI4u4e/IYAtnUJUit ErspKixJdipL991G5/Cya1+gFqgu3+HSJe2t3ttLSDwXnnZX9PIvubfLUKNbl27qlWgb 2V/yqbi/136LBcAydEZMUf/ZML4wu0RLCQvcCWCq3HCZsmm8I65y4G3Q/sOFLDb6k9dB z9S2KH5PvYWdDxTU++W1Ffm2kbtzFUh6dKHEvMV6OaLvl4IxgLUBUO1RuA1xxMETWg1U bu+IGq3xJuz4JSAEHftp2vxjQEMKhOmC9Vkqy1IZ2DBCbGDrED3xTtKoVdqec7e41FfT hw== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 11/15] vfio/common: Extract code from vfio_get_dirty_bitmap() to new function Date: Tue, 7 Mar 2023 12:54:46 +0000 Message-Id: <20230307125450.62409-12-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=710 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: R0IzaefAltAl3cL1aF7mMjJdhS4cmNdt X-Proofpoint-ORIG-GUID: R0IzaefAltAl3cL1aF7mMjJdhS4cmNdt Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193909590100001 From: Avihai Horon Extract the VFIO_IOMMU_DIRTY_PAGES ioctl code in vfio_get_dirty_bitmap() to its own function. This will help the code to be more readable after next patch will add device dirty page bitmap sync functionality. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 57 +++++++++++++++++++++++++++++------------------- 1 file changed, 35 insertions(+), 22 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 80f2d287bab5..1f97a8a3db18 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1595,26 +1595,13 @@ static void vfio_listener_log_global_stop(MemoryLis= tener *listener) } } =20 -static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, - uint64_t size, ram_addr_t ram_addr) +static int vfio_query_dirty_bitmap(VFIOContainer *container, VFIOBitmap *v= bmap, + hwaddr iova, hwaddr size) { struct vfio_iommu_type1_dirty_bitmap *dbitmap; struct vfio_iommu_type1_dirty_bitmap_get *range; - VFIOBitmap vbmap; int ret; =20 - if (!container->dirty_pages_supported) { - cpu_physical_memory_set_dirty_range(ram_addr, size, - tcg_enabled() ? DIRTY_CLIENTS_= ALL : - DIRTY_CLIENTS_NOCODE); - return 0; - } - - ret =3D vfio_bitmap_alloc(&vbmap, size); - if (ret) { - return ret; - } - dbitmap =3D g_malloc0(sizeof(*dbitmap) + sizeof(*range)); =20 dbitmap->argsz =3D sizeof(*dbitmap) + sizeof(*range); @@ -1629,8 +1616,8 @@ static int vfio_get_dirty_bitmap(VFIOContainer *conta= iner, uint64_t iova, * to qemu_real_host_page_size. */ range->bitmap.pgsize =3D qemu_real_host_page_size(); - range->bitmap.size =3D vbmap.size; - range->bitmap.data =3D (__u64 *)vbmap.bitmap; + range->bitmap.size =3D vbmap->size; + range->bitmap.data =3D (__u64 *)vbmap->bitmap; =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { @@ -1638,16 +1625,42 @@ static int vfio_get_dirty_bitmap(VFIOContainer *con= tainer, uint64_t iova, error_report("Failed to get dirty bitmap for iova: 0x%"PRIx64 " size: 0x%"PRIx64" err: %d", (uint64_t)range->iova, (uint64_t)range->size, errno); - goto err_out; + } + + g_free(dbitmap); + + return ret; +} + +static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, + uint64_t size, ram_addr_t ram_addr) +{ + VFIOBitmap vbmap; + int ret; + + if (!container->dirty_pages_supported) { + cpu_physical_memory_set_dirty_range(ram_addr, size, + tcg_enabled() ? DIRTY_CLIENTS_= ALL : + DIRTY_CLIENTS_NOCODE); + return 0; + } + + ret =3D vfio_bitmap_alloc(&vbmap, size); + if (ret) { + return ret; + } + + ret =3D vfio_query_dirty_bitmap(container, &vbmap, iova, size); + if (ret) { + goto out; } =20 cpu_physical_memory_set_dirty_lebitmap(vbmap.bitmap, ram_addr, vbmap.pages); =20 - trace_vfio_get_dirty_bitmap(container->fd, range->iova, range->size, - range->bitmap.size, ram_addr); -err_out: - g_free(dbitmap); + trace_vfio_get_dirty_bitmap(container->fd, iova, size, vbmap.size, + ram_addr); +out: g_free(vbmap.bitmap); =20 return ret; --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193834; cv=none; d=zohomail.com; s=zohoarc; b=Eaq3B34m4GotwfpNBZ6z/faNFkAVTu+Bz2V4qW6LX3LcftEvH1W6N+ID6db7JTid/MJwmE/hlXwysEhoZ5oCmg0tBzGbHOeNvpDySB5CH3P2RH2ymqK84sj+TyV99OEEP42vtvBKj8kBwa8her5rt4PJCYrz7AJGkPW2IiP2qe4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193834; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yTQFnRhuMnhAdRFDlEuktm6GXenp18YNPnq3q4EXUgg=; b=A3MJUdaqT4Y4tYHPngwvNCcKkMyVS6F6DaK8PPTBDyX7sfjUEVEaK3ihaebXYCXp0CVyao5ZV5rEIp9+zYtotwnF8nnR1bws95mn9x2Vs5bwHg3YHooaDqc11gE2L5u7lvgcOGm2QvVThsEJ9Jz+oryLZ4h4qJvFTECQs4BWkU4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193834676737.1800890386334; Tue, 7 Mar 2023 04:57:14 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrS-0006IE-C0; Tue, 07 Mar 2023 07:56:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrP-00068n-SP for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:07 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrN-00057G-6P for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:07 -0500 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 327842Rf017163; Tue, 7 Mar 2023 12:56:04 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p417cdctj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:04 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CHHoh029152; Tue, 7 Mar 2023 12:56:03 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyf1t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:03 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5N004358; Tue, 7 Mar 2023 12:56:03 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-13; Tue, 07 Mar 2023 12:56:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=yTQFnRhuMnhAdRFDlEuktm6GXenp18YNPnq3q4EXUgg=; b=ubU3aibawlw8tH9A96DF7fa8PBleQUERDaGV7aJVRToXSXU6otnZM/NYNx/lvaryhe0R skA5lXFlKjziNSobxKqhDGl8NN03jwxG6TseWsT4HU0jEjkv0dNc6B8LkKQsDBiqGL4p UN2eIelRQPCH636WjVBm5zdETzhzVswj0M6BeEnV1MHrQWwt3DAN9/2fC2RchtT/3pcc Y0d125Gr7SJy6hkvImPaiRz/kBaMhjSL1ijMmsPR1ij15NqGSOHOYETq9N+ghNQkK7dj pYB5WHfu5O3G2FP4qEY9nNO/S/6KRAIb3p5tQtxyN6tOgt/MrTvRVKyf67mi8RksFvLZ xA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 12/15] vfio/common: Add device dirty page bitmap sync Date: Tue, 7 Mar 2023 12:54:47 +0000 Message-Id: <20230307125450.62409-13-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=947 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: MTvietXVkPS50XFv4RPfxfMGKvlWoE6x X-Proofpoint-ORIG-GUID: MTvietXVkPS50XFv4RPfxfMGKvlWoE6x Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193836396100001 Content-Type: text/plain; charset="utf-8" Add device dirty page bitmap sync functionality. This uses the device DMA logging uAPI to sync dirty page bitmap from the device. Device dirty page bitmap sync is used only if all devices within a container support device dirty page tracking. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 88 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 79 insertions(+), 9 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 1f97a8a3db18..2639b393a781 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -339,6 +339,9 @@ static int vfio_bitmap_alloc(VFIOBitmap *vbmap, hwaddr = size) return 0; } =20 +static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, + uint64_t size, ram_addr_t ram_addr); + bool vfio_mig_active(void) { VFIOGroup *group; @@ -562,10 +565,16 @@ static int vfio_dma_unmap(VFIOContainer *container, .iova =3D iova, .size =3D size, }; + bool need_dirty_sync =3D false; + int ret; + + if (iotlb && vfio_devices_all_running_and_mig_active(container)) { + if (!vfio_devices_all_device_dirty_tracking(container) && + container->dirty_pages_supported) { + return vfio_dma_unmap_bitmap(container, iova, size, iotlb); + } =20 - if (iotlb && container->dirty_pages_supported && - vfio_devices_all_running_and_mig_active(container)) { - return vfio_dma_unmap_bitmap(container, iova, size, iotlb); + need_dirty_sync =3D true; } =20 while (ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, &unmap)) { @@ -591,10 +600,12 @@ static int vfio_dma_unmap(VFIOContainer *container, return -errno; } =20 - if (iotlb && vfio_devices_all_running_and_mig_active(container)) { - cpu_physical_memory_set_dirty_range(iotlb->translated_addr, size, - tcg_enabled() ? DIRTY_CLIENTS_= ALL : - DIRTY_CLIENTS_NOCODE); + if (need_dirty_sync) { + ret =3D vfio_get_dirty_bitmap(container, iova, size, + iotlb->translated_addr); + if (ret) { + return ret; + } } =20 return 0; @@ -1595,6 +1606,58 @@ static void vfio_listener_log_global_stop(MemoryList= ener *listener) } } =20 +static int vfio_device_dma_logging_report(VFIODevice *vbasedev, hwaddr iov= a, + hwaddr size, void *bitmap) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_dma_logging_repo= rt), + sizeof(__u64))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + struct vfio_device_feature_dma_logging_report *report =3D + (struct vfio_device_feature_dma_logging_report *)feature->data; + + report->iova =3D iova; + report->length =3D size; + report->page_size =3D qemu_real_host_page_size(); + report->bitmap =3D (__u64)(uintptr_t)bitmap; + + feature->argsz =3D sizeof(buf); + feature->flags =3D VFIO_DEVICE_FEATURE_GET | + VFIO_DEVICE_FEATURE_DMA_LOGGING_REPORT; + + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + return -errno; + } + + return 0; +} + +static int vfio_devices_query_dirty_bitmap(VFIOContainer *container, + VFIOBitmap *vbmap, hwaddr iova, + hwaddr size) +{ + VFIODevice *vbasedev; + VFIOGroup *group; + int ret; + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + ret =3D vfio_device_dma_logging_report(vbasedev, iova, size, + vbmap->bitmap); + if (ret) { + error_report("%s: Failed to get DMA logging report, iova: " + "0x%" HWADDR_PRIx ", size: 0x%" HWADDR_PRIx + ", err: %d (%s)", + vbasedev->name, iova, size, ret, strerror(-re= t)); + + return ret; + } + } + } + + return 0; +} + static int vfio_query_dirty_bitmap(VFIOContainer *container, VFIOBitmap *v= bmap, hwaddr iova, hwaddr size) { @@ -1635,10 +1698,12 @@ static int vfio_query_dirty_bitmap(VFIOContainer *c= ontainer, VFIOBitmap *vbmap, static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, uint64_t size, ram_addr_t ram_addr) { + bool all_device_dirty_tracking =3D + vfio_devices_all_device_dirty_tracking(container); VFIOBitmap vbmap; int ret; =20 - if (!container->dirty_pages_supported) { + if (!container->dirty_pages_supported && !all_device_dirty_tracking) { cpu_physical_memory_set_dirty_range(ram_addr, size, tcg_enabled() ? DIRTY_CLIENTS_= ALL : DIRTY_CLIENTS_NOCODE); @@ -1650,7 +1715,12 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, return ret; } =20 - ret =3D vfio_query_dirty_bitmap(container, &vbmap, iova, size); + if (all_device_dirty_tracking) { + ret =3D vfio_devices_query_dirty_bitmap(container, &vbmap, iova, s= ize); + } else { + ret =3D vfio_query_dirty_bitmap(container, &vbmap, iova, size); + } + if (ret) { goto out; } --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678194057; cv=none; d=zohomail.com; s=zohoarc; b=iNAWIhwaGAnZ8an9s27VRwGzToGGeiu317C9AWlNKpmYZSmZ6fwaAd79LlrZfH6uW67lYggVGPo1UlDdO+pC+582Dii5gthjQpZEmHNfk7yatk6cqHieg8ju1uLoeJy4dBqKlIRUmdyOiLfqb0s+MjLkXNr6ZpMJ55g+9ifhKsU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678194057; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KN/Dww0mMuy9Ttz9jfjlfG5UfB60DZEob8ekqMpxEvU=; b=mxWDq9aQmDzkIFZrRQXdGs/ssLRwt37Zmb9Oh3aIBMiBMF02v8rZ4E5tp95gotAklQx0vKE8QMJ1iyphGRcP3lpWN4yBJp0ArPExM7HILq6FQli6EdROMhFIpFRszQ0INqXKN30uxKXbytkVoP1hLS6KmZqa6pO0hT410wqdMNI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678194057802879.4612595428224; Tue, 7 Mar 2023 05:00:57 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrV-0006JQ-HK; Tue, 07 Mar 2023 07:56:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrR-0006Ht-Q9 for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:10 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrQ-00057a-3S for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:09 -0500 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 327846pG017250; Tue, 7 Mar 2023 12:56:07 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p417cdctr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:07 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CTx97026623; Tue, 7 Mar 2023 12:56:06 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyf3r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:06 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5P004358; Tue, 7 Mar 2023 12:56:05 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-14; Tue, 07 Mar 2023 12:56:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=KN/Dww0mMuy9Ttz9jfjlfG5UfB60DZEob8ekqMpxEvU=; b=ql7xbu3HBNjVHNAT/+zBjfKgQgHTS1NfoQCWFD7upEUZiBaL31S2p0QXzssboGfBeXZE M29vuND7RZY9UnuDY8b8EHWrj+sdBp9zaxEQTAwFTilJPYMuBW3lBQAWKswLYx2nNBBl iZM8tuhFxPkoT87QaJoxvm2Gx0Ok7gmUjk29MZYrwSXvSNupVRMbcIXunIx9L+v3pV2o ZoeBfycNd22KzwkrAx7DZ+BSdtaMCrZjctKAY1zFZ95RO/g4mdpJ9UnWWn5g+wSF5qkH U57wp4VkRFsguRXPUDlTPfScxLO/ZQThSq0da251saA6hmqx+j9mmuKQW8UwIIFZzB53 jA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 13/15] vfio/migration: Block migration with vIOMMU Date: Tue, 7 Mar 2023 12:54:48 +0000 Message-Id: <20230307125450.62409-14-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=703 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: H4uRhhZyuCUfj0a4NeivcBYndsA3lODc X-Proofpoint-ORIG-GUID: H4uRhhZyuCUfj0a4NeivcBYndsA3lODc Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678194060691100007 Migrating with vIOMMU will require either tracking maximum IOMMU supported address space (e.g. 39/48 address width on Intel) or range-track current mappings and dirty track the new ones post starting dirty tracking. This will be done as a separate series, so add a live migration blocker until that is fixed. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 46 +++++++++++++++++++++++++++++++++++ hw/vfio/migration.c | 5 ++++ hw/vfio/pci.c | 1 + include/hw/vfio/vfio-common.h | 2 ++ 4 files changed, 54 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 2639b393a781..2b9bcf70aa36 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -362,6 +362,7 @@ bool vfio_mig_active(void) } =20 static Error *multiple_devices_migration_blocker; +static Error *giommu_migration_blocker; =20 static unsigned int vfio_migratable_device_num(void) { @@ -413,6 +414,51 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker =3D NULL; } =20 +static bool vfio_viommu_preset(void) +{ + VFIOAddressSpace *space; + + QLIST_FOREACH(space, &vfio_address_spaces, list) { + if (space->as !=3D &address_space_memory) { + return true; + } + } + + return false; +} + +int vfio_block_giommu_migration(Error **errp) +{ + int ret; + + if (giommu_migration_blocker || + !vfio_viommu_preset()) { + return 0; + } + + error_setg(&giommu_migration_blocker, + "Migration is currently not supported with vIOMMU enabled"); + ret =3D migrate_add_blocker(giommu_migration_blocker, errp); + if (ret < 0) { + error_free(giommu_migration_blocker); + giommu_migration_blocker =3D NULL; + } + + return ret; +} + +void vfio_unblock_giommu_migration(void) +{ + if (!giommu_migration_blocker || + vfio_viommu_preset()) { + return; + } + + migrate_del_blocker(giommu_migration_blocker); + error_free(giommu_migration_blocker); + giommu_migration_blocker =3D NULL; +} + static void vfio_set_migration_error(int err) { MigrationState *ms =3D migrate_get_current(); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index a2c3d9bade7f..776fd2d7cdf3 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -634,6 +634,11 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error *= *errp) return ret; } =20 + ret =3D vfio_block_giommu_migration(errp); + if (ret) { + return ret; + } + trace_vfio_migration_probe(vbasedev->name); return 0; =20 diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 939dcc3d4a9e..30a271eab38c 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3185,6 +3185,7 @@ static void vfio_instance_finalize(Object *obj) */ vfio_put_device(vdev); vfio_put_group(group); + vfio_unblock_giommu_migration(); } =20 static void vfio_exitfn(PCIDevice *pdev) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 9551d2d43025..009bec34c4bc 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -222,6 +222,8 @@ extern VFIOGroupList vfio_group_list; bool vfio_mig_active(void); int vfio_block_multiple_devices_migration(Error **errp); void vfio_unblock_multiple_devices_migration(void); +int vfio_block_giommu_migration(Error **errp); +void vfio_unblock_giommu_migration(void); int64_t vfio_mig_bytes_transferred(void); =20 #ifdef CONFIG_LINUX --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193806; cv=none; d=zohomail.com; s=zohoarc; b=XSnzMmZHqzuvgPQiK1BSLrMrK5CdvvVouhq/TjuUix0KXrUu55fBtcQ3M+AaK3X6TW0ruimFMme7BGOqyZiikGb/sarh1qYcreFihkhKAZiKsocDCLLRYVNPx6a15snEmcMZ3LLjIuMzZAgCTRNWxWra+2ujrBdW8+k2K08+1s8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193806; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RRUiwfGBnydyzEIfXtmJEXKi5yt86OH4+6XVYCR01nQ=; b=SrFDSh6H6ySmGJa90LxirXQWRvx47OvFmKuqlK/pNZKt5ucC1dJ0jpA0g0sIjcgz4fDUzhtXgEmecBfO87wyNwAEsQN9J+Y4euIPg5dFu0+hzbe4wpecNhfluzSOuVwwU3Tjdufq7GSHRjNEE7fL0H0H0G5veeso54oITDVB3zA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193806601791.5530237152756; Tue, 7 Mar 2023 04:56:46 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrd-0006XG-Vy; Tue, 07 Mar 2023 07:56:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrU-0006JU-A6 for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:13 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrS-000586-V8 for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:12 -0500 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32784DWe017315; Tue, 7 Mar 2023 12:56:10 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p417cdctt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:10 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327CBaw1029219; Tue, 7 Mar 2023 12:56:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyf5x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:09 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5R004358; Tue, 7 Mar 2023 12:56:08 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-15; Tue, 07 Mar 2023 12:56:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=RRUiwfGBnydyzEIfXtmJEXKi5yt86OH4+6XVYCR01nQ=; b=ztZlR50x7QRfYUZ9RWcbBx2e1DhpdsnIwBQg7yz9R8sqwsAc+/8EjyA1FvKKqLfVArs8 qIQ6qH9dlKqhGMg+B74Vzcdc5i4x+9b9zXxCtCWkUD7+jzK8UiFJp7Q9RaI9TNksWbRw IoDJHF6ZFW/raNA2WHsoq31vsVLhnLg/Kz2gtCNj+tQQVDQR0yTDpPQbd6Ab9nw9W3a2 s28zoTNgJwPZ4aKiP2jKrfSQXh4NaZhlgFVC8sVPXs53buu1EDNUvHkpxypJYRydBmsD RhJ6pXFfNa+mT77TV7wvAQEsetjkCdAoNY755etU3zTUjEzy/aFE1k0RV+0BIn7XSY0w DA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 14/15] vfio/migration: Query device dirty page tracking support Date: Tue, 7 Mar 2023 12:54:49 +0000 Message-Id: <20230307125450.62409-15-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: 9cZI1zqAdKzMhnDacaeXE5MgN0u0GgW0 X-Proofpoint-ORIG-GUID: 9cZI1zqAdKzMhnDacaeXE5MgN0u0GgW0 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193808288100003 Now that everything has been set up for device dirty page tracking, query the device for device dirty page tracking support. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/migration.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 776fd2d7cdf3..5d1e66f69194 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -555,6 +555,19 @@ static int vfio_migration_query_flags(VFIODevice *vbas= edev, uint64_t *mig_flags) return 0; } =20 +static bool vfio_dma_logging_supported(VFIODevice *vbasedev) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + + feature->argsz =3D sizeof(buf); + feature->flags =3D VFIO_DEVICE_FEATURE_PROBE | + VFIO_DEVICE_FEATURE_DMA_LOGGING_START; + + return !ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); +} + static int vfio_migration_init(VFIODevice *vbasedev) { int ret; @@ -589,6 +602,8 @@ static int vfio_migration_init(VFIODevice *vbasedev) migration->device_state =3D VFIO_DEVICE_STATE_RUNNING; migration->data_fd =3D -1; =20 + vbasedev->dirty_pages_supported =3D vfio_dma_logging_supported(vbasede= v); + oid =3D vmstate_if_get_id(VMSTATE_IF(DEVICE(obj))); if (oid) { path =3D g_strdup_printf("%s/vfio", oid); --=20 2.17.2 From nobody Mon May 6 22:21:08 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678193822; cv=none; d=zohomail.com; s=zohoarc; b=cm7okEkcVB639TBasrbi8g+P2z0eR2sakkdXVHJ4JZbmCdqL329ss90mReL1CZLaCQycP5VuZp5E2lhRd0K97wnmHSGGArbrP+jJmSU4HLM3XpN+pdbtWf9hG3rrYFZUSeT3Xkxh+sOhR/4CEhdM+5tep5NcHp1lBc8OBr3Cyjk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678193822; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Xel4ecbEPawZ38P8TKB/h9sUQWvBI6R7FLZBmhXQ7Bw=; b=ex5uru3RBsP0t1tnNXxY4HrTCJVjuuhmuDuilpglDyHVm9nyp13czkx6RRlkI0rgH3P0j2nLOhU5wbMAXfm/wv1NpXeoDcc0jF596sF8huZLbBS6nUoGo0WXLFXCDtEtiWrpsnqgKkKljPAvs67njvOaQQQTJSwmQNTManIiX0E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678193822283525.4938939485207; Tue, 7 Mar 2023 04:57:02 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZWrt-0006wv-V6; Tue, 07 Mar 2023 07:56:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrY-0006XR-IG for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:21 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZWrW-00058j-Qu for qemu-devel@nongnu.org; Tue, 07 Mar 2023 07:56:16 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32783uVZ007683; Tue, 7 Mar 2023 12:56:13 GMT Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p5nn91pvq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:13 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 327C8dnB029280; Tue, 7 Mar 2023 12:56:12 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p4u1eyf7k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 12:56:12 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 327CtV5T004358; Tue, 7 Mar 2023 12:56:11 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-196-64.vpn.oracle.com [10.175.196.64]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p4u1eyefv-16; Tue, 07 Mar 2023 12:56:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=Xel4ecbEPawZ38P8TKB/h9sUQWvBI6R7FLZBmhXQ7Bw=; b=kWFTIOxlYmwyIJKMQRKlvKrqLRNM+gNAgnbLFcCllO+kQKpu1wJcgxSWQ7R1im1eOehW 4STyuDOLMNYE+0UIPW6Ws99loEoQntvKfqmdT0LES0dLcq0ehQPTS5o7b+5/U8+vQ6O6 V1dLIsyJcJijuPXSswflfMyIOh3o4tZbjL9bDEcDf2oYJhTW0SzjK6x/JKeRYioIwqeX eJFwzP2LkOt8Rc97NZbqcFv9v5c2nico9wMZa2MOO/ugfKLDzFXJ2FUBKxOZMnjulPzg kCiW7FrJugxN1v6+yycKKg31E7XxPILefkvCb0P6HJ0rcNiM2enNV5QPFBtCAUAbDQVN cQ== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v5 15/15] docs/devel: Document VFIO device dirty page tracking Date: Tue, 7 Mar 2023 12:54:50 +0000 Message-Id: <20230307125450.62409-16-joao.m.martins@oracle.com> In-Reply-To: <20230307125450.62409-1-joao.m.martins@oracle.com> References: <20230307125450.62409-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-07_06,2023-03-07_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=990 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070117 X-Proofpoint-GUID: FIGetVEgc3P-rAaqPRilCMbO_Ubuj6L3 X-Proofpoint-ORIG-GUID: FIGetVEgc3P-rAaqPRilCMbO_Ubuj6L3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678193824375100003 From: Avihai Horon Adjust the VFIO dirty page tracking documentation and add a section to describe device dirty page tracking. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- docs/devel/vfio-migration.rst | 46 +++++++++++++++++++++++------------ 1 file changed, 31 insertions(+), 15 deletions(-) diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst index c214c73e2818..1b68ccf11529 100644 --- a/docs/devel/vfio-migration.rst +++ b/docs/devel/vfio-migration.rst @@ -59,22 +59,37 @@ System memory dirty pages tracking ---------------------------------- =20 A ``log_global_start`` and ``log_global_stop`` memory listener callback in= forms -the VFIO IOMMU module to start and stop dirty page tracking. A ``log_sync`` -memory listener callback marks those system memory pages as dirty which are -used for DMA by the VFIO device. The dirty pages bitmap is queried per -container. All pages pinned by the vendor driver through external APIs hav= e to -be marked as dirty during migration. When there are CPU writes, CPU dirty = page -tracking can identify dirtied pages, but any page pinned by the vendor dri= ver -can also be written by the device. There is currently no device or IOMMU -support for dirty page tracking in hardware. +the VFIO dirty tracking module to start and stop dirty page tracking. A +``log_sync`` memory listener callback queries the dirty page bitmap from t= he +dirty tracking module and marks system memory pages which were DMA-ed by t= he +VFIO device as dirty. The dirty page bitmap is queried per container. + +Currently there are two ways dirty page tracking can be done: +(1) Device dirty tracking: +In this method the device is responsible to log and report its DMAs. This +method can be used only if the device is capable of tracking its DMAs. +Discovering device capability, starting and stopping dirty tracking, and +syncing the dirty bitmaps from the device are done using the DMA logging u= API. +More info about the uAPI can be found in the comments of the +``vfio_device_feature_dma_logging_control`` and +``vfio_device_feature_dma_logging_report`` structures in the header file +linux-headers/linux/vfio.h. + +(2) VFIO IOMMU module: +In this method dirty tracking is done by IOMMU. However, there is currentl= y no +IOMMU support for dirty page tracking. For this reason, all pages are +perpetually marked dirty, unless the device driver pins pages through exte= rnal +APIs in which case only those pinned pages are perpetually marked dirty. + +If the above two methods are not supported, all pages are perpetually mark= ed +dirty by QEMU. =20 By default, dirty pages are tracked during pre-copy as well as stop-and-co= py -phase. So, a page pinned by the vendor driver will be copied to the destin= ation -in both phases. Copying dirty pages in pre-copy phase helps QEMU to predic= t if -it can achieve its downtime tolerances. If QEMU during pre-copy phase keeps -finding dirty pages continuously, then it understands that even in stop-an= d-copy -phase, it is likely to find dirty pages and can predict the downtime -accordingly. +phase. So, a page marked as dirty will be copied to the destination in both +phases. Copying dirty pages in pre-copy phase helps QEMU to predict if it = can +achieve its downtime tolerances. If QEMU during pre-copy phase keeps findi= ng +dirty pages continuously, then it understands that even in stop-and-copy p= hase, +it is likely to find dirty pages and can predict the downtime accordingly. =20 QEMU also provides a per device opt-out option ``pre-copy-dirty-page-track= ing`` which disables querying the dirty bitmap during pre-copy phase. If it is s= et to @@ -89,7 +104,8 @@ phase of migration. In that case, the unmap ioctl return= s any dirty pages in that range and QEMU reports corresponding guest physical pages dirty. Duri= ng stop-and-copy phase, an IOMMU notifier is used to get a callback for mapped pages and then dirty pages bitmap is fetched from VFIO IOMMU modules for t= hose -mapped ranges. +mapped ranges. If device dirty tracking is enabled with vIOMMU, live migra= tion +will be blocked. =20 Flow of state changes during Live migration =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --=20 2.17.2