From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154673; cv=none; d=zohomail.com; s=zohoarc; b=dbNeW1AOO4QsJLveV8cErV6wYScVn9mGR+B2xaPcTZJf12MGfEzXRH0eA/8ZWt06PkzBIpU96+aiAi8w8m24eJTJVNjsD1nI/xOCMUQN4F6aXvFcnZMP2DkJxg4rxnWc9jfpVSAF6rS2enbz4mB4qvHvvevQ4mgoIYen0pQOzn4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154673; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ph7NhPNJqQFVQZvkebi126vtqIT7j0fmOZoCxa+ciPc=; b=dTKA3PhFe1i5BOXgV6rnQsMVY6+dv7aHXmbPIZ+BrOnOrHqM0VTpVL45SdCLrztFtGdA2DsPyBT+NS6ihwbdAg2UJgLzT5yJx3aGsVmKPojqe52i9ZGfpYmhxVXRL2atWwQXIIOswZdCqF0O6fc2D6dKOQF5wQOvsLzjJj6xPZ0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154673038888.2177028515589; Mon, 6 Mar 2023 18:04:33 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMff-0006nQ-4Q; Mon, 06 Mar 2023 21:03:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfd-0006mb-N3 for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:17 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfY-0004uE-Bf for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:17 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NwhNY026366; Tue, 7 Mar 2023 02:03:11 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p4180vddy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:10 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 326Na29X036999; Tue, 7 Mar 2023 02:03:10 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txdubhw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:10 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 32723560016763; Tue, 7 Mar 2023 02:03:09 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-2; Tue, 07 Mar 2023 02:03:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=ph7NhPNJqQFVQZvkebi126vtqIT7j0fmOZoCxa+ciPc=; b=wqdrwab61/9w1NH7MD5A2A4vDQNc4x9nUgQHgzmrGWvdhM3gL789KNBY+b8ldAYoIdmn YFnom4QS4SVO9LANueHzPVR2nYkArEXZ5YaXAypLPNgTVBVjhx8ptlJ2ck01suG9QrZQ pEvbkyr8N75geWUAIANYs36mjP6ao6k6Bi4M9PB10F2+O21jmGWZNROV8ajDBjxKRKYO 491DMzbVoSqIq/zWokgf5TfvKEpVOyVg3kr4/1bh8NFw003DZ9kqTSo1tXaU0aJE4v3A xuf5GC6g/28+x7KHD7i9dHcWWI/pPE6vuUQrnfKlia1ZOq3KNph7v+8fsjndiUGjibTI GQ== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 01/14] vfio/common: Fix error reporting in vfio_get_dirty_bitmap() Date: Tue, 7 Mar 2023 02:02:45 +0000 Message-Id: <20230307020258.58215-2-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=852 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070016 X-Proofpoint-ORIG-GUID: jueCNsHgba3koWvQ4femhtEPRQU4efdg X-Proofpoint-GUID: jueCNsHgba3koWvQ4femhtEPRQU4efdg Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154674112100003 From: Avihai Horon Return -errno instead of -1 if VFIO_IOMMU_DIRTY_PAGES ioctl fails in vfio_get_dirty_bitmap(). Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index bab83c0e55cb..9fc305448fa2 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1335,6 +1335,7 @@ static int vfio_get_dirty_bitmap(VFIOContainer *conta= iner, uint64_t iova, =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { + ret =3D -errno; error_report("Failed to get dirty bitmap for iova: 0x%"PRIx64 " size: 0x%"PRIx64" err: %d", (uint64_t)range->iova, (uint64_t)range->size, errno); --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154635; cv=none; d=zohomail.com; s=zohoarc; b=PT2m+eaEzgJgpSnBV66jN0/6yWnH2zO0XVmjwWldAeLU6mEzRGp8Bd+CgqkTuntMFEpw4lDeG16ecXgCFh/J8wz1mmUMgup7HeDfbBbvVmruYzi9x71/AVDfAFjbFQgRIhnlISDm86EyhxTbxSFEAvdalebv22XBNmjwcyNzwW0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154635; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xk8wvTPv6dJhnVr411G6qxujVLuqM/QjT1VPuR/+yew=; b=lGqXyRK+T9WgpoXeHWD67OUAz1E7np6nPknM8s6a4C1mI3CtXV0STeGfoima7Smdje528BbYLpZStDLxiLfs12y0EMPTGpwPQDIzH8RiqArDlYe/OpWJIiCAKRjLjFKmN38KVm6i09CP12hUcqncMkhHagpckKrbbauQvbrWsqQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154634903746.9291029958522; Mon, 6 Mar 2023 18:03:54 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMff-0006nw-TM; Mon, 06 Mar 2023 21:03:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfd-0006mf-Td for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:17 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfc-0004uN-A7 for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:17 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NwnJw026409; Tue, 7 Mar 2023 02:03:14 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p4180vde0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:14 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3271P09C037278; Tue, 7 Mar 2023 02:03:13 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txdubkh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:13 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 32723562016763; Tue, 7 Mar 2023 02:03:12 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-3; Tue, 07 Mar 2023 02:03:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=xk8wvTPv6dJhnVr411G6qxujVLuqM/QjT1VPuR/+yew=; b=ObOvdxOR0aVRCxTaRnu8G+O5RmT/M/MmHsgFKx6SlKFgIQIiWbNH9AQQaN96fguI1EnD nipVEX/rsgVbj1J/GBd+a1ug8VA7ZUsCaQQC886SE9Fh38DJLCPJKDudfPgZFrPWwwUq Ch8MnWIHAoLNSYronA+gdid7/LofbHuk22Yc2VsfoVWzBtXdmcj+NRnxlzEP8ix0Q5/A mY5wU7ZKwdA3Vnnin6LY2yqxF+Vu0CXDgqMtPdgXq+EAZxdDt3KOaJ6HHoaLDkuOepPK UoS7ioobdafLpRwPHriU6cUTROi9ao4fD6N8coAyIJcrf47VZ0oJkMlK8FIK6F+rBMOf TQ== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 02/14] vfio/common: Fix wrong %m usages Date: Tue, 7 Mar 2023 02:02:46 +0000 Message-Id: <20230307020258.58215-3-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=908 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070016 X-Proofpoint-ORIG-GUID: 5TahqlcYH525isYbFRRWkbgFCzTl980E X-Proofpoint-GUID: 5TahqlcYH525isYbFRRWkbgFCzTl980E Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154636083100003 From: Avihai Horon There are several places where the %m conversion is used if one of vfio_dma_map(), vfio_dma_unmap() or vfio_get_dirty_bitmap() fail. The %m usage in these places is wrong since %m relies on errno value while the above functions don't report errors via errno. Fix it by using strerror() with the returned value instead. Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 9fc305448fa2..4d26e9cccf91 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -703,17 +703,17 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, I= OMMUTLBEntry *iotlb) read_only); if (ret) { error_report("vfio_dma_map(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx", %p) =3D %d (%m)", + "0x%"HWADDR_PRIx", %p) =3D %d (%s)", container, iova, - iotlb->addr_mask + 1, vaddr, ret); + iotlb->addr_mask + 1, vaddr, ret, strerror(-ret)); } } else { ret =3D vfio_dma_unmap(container, iova, iotlb->addr_mask + 1, iotl= b); if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") =3D %d (%m)", + "0x%"HWADDR_PRIx") =3D %d (%s)", container, iova, - iotlb->addr_mask + 1, ret); + iotlb->addr_mask + 1, ret, strerror(-ret)); } } out: @@ -1095,8 +1095,9 @@ static void vfio_listener_region_add(MemoryListener *= listener, vaddr, section->readonly); if (ret) { error_setg(&err, "vfio_dma_map(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx", %p) =3D %d (%m)", - container, iova, int128_get64(llsize), vaddr, ret); + "0x%"HWADDR_PRIx", %p) =3D %d (%s)", + container, iova, int128_get64(llsize), vaddr, ret, + strerror(-ret)); if (memory_region_is_ram_device(section->mr)) { /* Allow unexpected mappings not to be fatal for RAM devices */ error_report_err(err); @@ -1228,16 +1229,18 @@ static void vfio_listener_region_del(MemoryListener= *listener, ret =3D vfio_dma_unmap(container, iova, int128_get64(llsize), = NULL); if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") =3D %d (%m)", - container, iova, int128_get64(llsize), ret); + "0x%"HWADDR_PRIx") =3D %d (%s)", + container, iova, int128_get64(llsize), ret, + strerror(-ret)); } iova +=3D int128_get64(llsize); } ret =3D vfio_dma_unmap(container, iova, int128_get64(llsize), NULL= ); if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") =3D %d (%m)", - container, iova, int128_get64(llsize), ret); + "0x%"HWADDR_PRIx") =3D %d (%s)", + container, iova, int128_get64(llsize), ret, + strerror(-ret)); } } =20 @@ -1384,9 +1387,9 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifier= *n, IOMMUTLBEntry *iotlb) translated_addr); if (ret) { error_report("vfio_iommu_map_dirty_notify(%p, 0x%"HWADDR_PRIx"= , " - "0x%"HWADDR_PRIx") =3D %d (%m)", - container, iova, - iotlb->addr_mask + 1, ret); + "0x%"HWADDR_PRIx") =3D %d (%s)", + container, iova, iotlb->addr_mask + 1, ret, + strerror(-ret)); } } rcu_read_unlock(); --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154635; cv=none; d=zohomail.com; s=zohoarc; b=I2rrEkPrhHA5etd6TFq8K9p8arFGwfRWTnASujQ7pbcMHJriCRLWmOIp4sNgdQRE2eVttbXyowMH+2PFOJ3ueNrTyEYmTKQqcJJqVQdLWCuc+MHAH65DB6GOAGh4zHBwdyeL+2BqnXgiyLxNsqnTebaoDe81ZMkeK9tlCYLRD9g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154635; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GiBCAPHsWsCaxFjIYhTKruwe4Yu7rgg+z8yv2dJvMcY=; b=ZMWGshuQ3QnttJmVr5dblXn1Bb2lOeilHn1m+ebnmx+EVXMkzb3v7ODypnnDoJACemBd6RqcJR6Fy3QtGANp6Oci+U2sXoSUJ8UaZlBBFy/TiWFzYqLi6GsvM4+z/O4dhxR0HFbqpMlZ8UxVXLYBBDFamnJbLVKkpFE74wA4EOM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154635979568.1435797841975; Mon, 6 Mar 2023 18:03:55 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMfj-0006ox-3K; Mon, 06 Mar 2023 21:03:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfi-0006om-58 for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:22 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfg-0004va-8C for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:21 -0500 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NxYWS029931; Tue, 7 Mar 2023 02:03:18 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p418xvfdx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:18 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 32720Oo8036910; Tue, 7 Mar 2023 02:03:16 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txdubn3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:16 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 32723564016763; Tue, 7 Mar 2023 02:03:16 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-4; Tue, 07 Mar 2023 02:03:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=GiBCAPHsWsCaxFjIYhTKruwe4Yu7rgg+z8yv2dJvMcY=; b=CGjtzdBMaVjzTbfIk8c3QG/jeMampZF2JU5SFshK1VDVNmMpfts8v1Ef1itluFVdcXd8 B1Jkb0t4Y2wfrSGV0edi9vnznjXebDv/kQZ9USkfNQloImz1ZJEyDXgFGfYuao7qhhXw SRy1kJj8oUavfZcZEV6Y9SgoK+5u7l6kTPkXnZYTT7W6srb71V2aMQmFaaxNg5cisxr5 RKMTZpo7FqMo27eMCDwG7I16eyKfCKqLFlVmKvc6nMyB8DW43B2cAiul72of+s54bZE5 n2Suaiuidvq6ntglf48B4bI5WQJD3M0aR1/WNL+kpNE7jaZSZGAbV5u+hvneDlq3zlC8 2g== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 03/14] vfio/common: Abort migration if dirty log start/stop/sync fails Date: Tue, 7 Mar 2023 02:02:47 +0000 Message-Id: <20230307020258.58215-4-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070016 X-Proofpoint-GUID: -AK5jGLT1qYDwaMq-VrVu5avUbFwuFH_ X-Proofpoint-ORIG-GUID: -AK5jGLT1qYDwaMq-VrVu5avUbFwuFH_ Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154637844100007 From: Avihai Horon If VFIO dirty pages log start/stop/sync fails during migration, migration should be aborted as pages dirtied by VFIO devices might not be reported properly. This is not the case today, where in such scenario only an error is printed. Fix it by aborting migration in the above scenario. Fixes: 758b96b61d5c ("vfio/migrate: Move switch of dirty tracking into vfio= _memory_listener") Fixes: b6dd6504e303 ("vfio: Add vfio_listener_log_sync to mark dirty pages") Fixes: 9e7b0442f23a ("vfio: Add ioctl to get dirty pages bitmap during dma = unmap") Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 53 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 45 insertions(+), 8 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 4d26e9cccf91..4c801513136a 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -42,6 +42,7 @@ #include "migration/migration.h" #include "migration/misc.h" #include "migration/blocker.h" +#include "migration/qemu-file.h" #include "sysemu/tpm.h" =20 VFIOGroupList vfio_group_list =3D @@ -390,6 +391,19 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker =3D NULL; } =20 +static void vfio_set_migration_error(int err) +{ + MigrationState *ms =3D migrate_get_current(); + + if (migration_is_setup_or_active(ms->state)) { + WITH_QEMU_LOCK_GUARD(&ms->qemu_file_lock) { + if (ms->to_dst_file) { + qemu_file_set_error(ms->to_dst_file, err); + } + } + } +} + static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) { VFIOGroup *group; @@ -680,6 +694,7 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOM= MUTLBEntry *iotlb) if (iotlb->target_as !=3D &address_space_memory) { error_report("Wrong target AS \"%s\", only system memory is allowe= d", iotlb->target_as->name ? iotlb->target_as->name : "no= ne"); + vfio_set_migration_error(-EINVAL); return; } =20 @@ -714,6 +729,7 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOM= MUTLBEntry *iotlb) "0x%"HWADDR_PRIx") =3D %d (%s)", container, iova, iotlb->addr_mask + 1, ret, strerror(-ret)); + vfio_set_migration_error(ret); } } out: @@ -1259,7 +1275,7 @@ static void vfio_listener_region_del(MemoryListener *= listener, } } =20 -static void vfio_set_dirty_page_tracking(VFIOContainer *container, bool st= art) +static int vfio_set_dirty_page_tracking(VFIOContainer *container, bool sta= rt) { int ret; struct vfio_iommu_type1_dirty_bitmap dirty =3D { @@ -1267,7 +1283,7 @@ static void vfio_set_dirty_page_tracking(VFIOContaine= r *container, bool start) }; =20 if (!container->dirty_pages_supported) { - return; + return 0; } =20 if (start) { @@ -1278,23 +1294,34 @@ static void vfio_set_dirty_page_tracking(VFIOContai= ner *container, bool start) =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, &dirty); if (ret) { + ret =3D -errno; error_report("Failed to set dirty tracking flag 0x%x errno: %d", dirty.flags, errno); } + + return ret; } =20 static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + int ret; =20 - vfio_set_dirty_page_tracking(container, true); + ret =3D vfio_set_dirty_page_tracking(container, true); + if (ret) { + vfio_set_migration_error(ret); + } } =20 static void vfio_listener_log_global_stop(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + int ret; =20 - vfio_set_dirty_page_tracking(container, false); + ret =3D vfio_set_dirty_page_tracking(container, false); + if (ret) { + vfio_set_migration_error(ret); + } } =20 static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, @@ -1370,19 +1397,18 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifi= er *n, IOMMUTLBEntry *iotlb) VFIOContainer *container =3D giommu->container; hwaddr iova =3D iotlb->iova + giommu->iommu_offset; ram_addr_t translated_addr; + int ret =3D -EINVAL; =20 trace_vfio_iommu_map_dirty_notify(iova, iova + iotlb->addr_mask); =20 if (iotlb->target_as !=3D &address_space_memory) { error_report("Wrong target AS \"%s\", only system memory is allowe= d", iotlb->target_as->name ? iotlb->target_as->name : "no= ne"); - return; + goto out; } =20 rcu_read_lock(); if (vfio_get_xlat_addr(iotlb, NULL, &translated_addr, NULL)) { - int ret; - ret =3D vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + = 1, translated_addr); if (ret) { @@ -1393,6 +1419,11 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifie= r *n, IOMMUTLBEntry *iotlb) } } rcu_read_unlock(); + +out: + if (ret) { + vfio_set_migration_error(ret); + } } =20 static int vfio_ram_discard_get_dirty_bitmap(MemoryRegionSection *section, @@ -1485,13 +1516,19 @@ static void vfio_listener_log_sync(MemoryListener *= listener, MemoryRegionSection *section) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + int ret; =20 if (vfio_listener_skipped_section(section)) { return; } =20 if (vfio_devices_all_dirty_tracking(container)) { - vfio_sync_dirty_bitmap(container, section); + ret =3D vfio_sync_dirty_bitmap(container, section); + if (ret) { + error_report("vfio: Failed to sync dirty bitmap, err: %d (%s)"= , ret, + strerror(-ret)); + vfio_set_migration_error(ret); + } } } =20 --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154709; cv=none; d=zohomail.com; s=zohoarc; b=lKTX1fLp+1P6QOt5ejZ1/sh6wKbbl5vY/SFOAooYLqS75Rjfz2s2JnBjC75dauMbEL4b0q8DUu3kCxvUE7HcVeoYNI3EjycBqS4fM6rpAilZN/Mv7yzs8WKTO45baggQ5Ss7dQyXJf8XEnfU7m4+p2XpRwCZB7QgcqVSkVQ7fT0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154709; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5w+UzqUbMx72jclmr5DPxhul0r8RPY8u0pJnKNlmPBc=; b=LTCqOfwITOhKhFg4sFk3Fj6w64+dcbrH4fDj76RW6KJvMDhhC7g6o0lT8y2+yFgE16tWS7Z7Dpea6DR7aggg6nisOotge4ZhPs3p+3wtkib2p9azpy2BQtZdJ38SShCG1qCO4GCx1/91H3hhFPK/ZMPcgLltxODDPasRhJxr4R0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154709988412.7872745920396; Mon, 6 Mar 2023 18:05:09 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMfl-0006qB-OT; Mon, 06 Mar 2023 21:03:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfj-0006pB-S5 for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:23 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfi-0004vq-2O for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:23 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NwvMu022239; Tue, 7 Mar 2023 02:03:21 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p415hvfny-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:21 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3271svPk037115; Tue, 7 Mar 2023 02:03:20 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txdubpk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:20 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 32723566016763; Tue, 7 Mar 2023 02:03:19 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-5; Tue, 07 Mar 2023 02:03:19 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=5w+UzqUbMx72jclmr5DPxhul0r8RPY8u0pJnKNlmPBc=; b=uBxyy+l7NGEGFziJdI4NF1Qq+Kjs9F2Y5zhoLFBSWSBhyjQbSIFvXxjhvveJlexksArn cH5HJxMrS0MWCqNpWPpCvusXbmfr92PJ3Y81oyfUACJonjU1d96bokSO7GZWJSpF/tZf gi5HgYYU0ZaQ4LarSE4lezQNwKjyOfLUVUPnHMmRj9AOhUcE9rfq8wO0UW2Fj9/OWrtV 0jcTs7IqoPddCyF5M2tMdH32ugv4PBo6XUVT/WyEK+urIGiDS3lFnUovL0jeN3fao3v/ tYMMS4eeqtCXrQ93Qxct0nJ8oQISqFAIV2plQ+DCUg5BUv9Fivca1TXlRKyqE653ZRlF 1A== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 04/14] vfio/common: Add VFIOBitmap and alloc function Date: Tue, 7 Mar 2023 02:02:48 +0000 Message-Id: <20230307020258.58215-5-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070016 X-Proofpoint-GUID: 05cykTDuLbYrQSE6x7rxvbo_hbLotKHY X-Proofpoint-ORIG-GUID: 05cykTDuLbYrQSE6x7rxvbo_hbLotKHY Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154710298100005 From: Avihai Horon There are already two places where dirty page bitmap allocation and calculations are done in open code. With device dirty page tracking being added in next patches, there are going to be even more places. To avoid code duplication, introduce VFIOBitmap struct and corresponding alloc function and use them where applicable. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 73 +++++++++++++++++++++++++++++------------------- 1 file changed, 44 insertions(+), 29 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 4c801513136a..cec3de08d2b4 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -320,6 +320,25 @@ const MemoryRegionOps vfio_region_ops =3D { * Device state interfaces */ =20 +typedef struct { + unsigned long *bitmap; + hwaddr size; + hwaddr pages; +} VFIOBitmap; + +static int vfio_bitmap_alloc(VFIOBitmap *vbmap, hwaddr size) +{ + vbmap->pages =3D REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_size= (); + vbmap->size =3D ROUND_UP(vbmap->pages, sizeof(__u64) * BITS_PER_BYTE) / + BITS_PER_BYTE; + vbmap->bitmap =3D g_try_malloc0(vbmap->size); + if (!vbmap->bitmap) { + return -ENOMEM; + } + + return 0; +} + bool vfio_mig_active(void) { VFIOGroup *group; @@ -468,9 +487,14 @@ static int vfio_dma_unmap_bitmap(VFIOContainer *contai= ner, { struct vfio_iommu_type1_dma_unmap *unmap; struct vfio_bitmap *bitmap; - uint64_t pages =3D REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_si= ze(); + VFIOBitmap vbmap; int ret; =20 + ret =3D vfio_bitmap_alloc(&vbmap, size); + if (ret) { + return ret; + } + unmap =3D g_malloc0(sizeof(*unmap) + sizeof(*bitmap)); =20 unmap->argsz =3D sizeof(*unmap) + sizeof(*bitmap); @@ -484,35 +508,28 @@ static int vfio_dma_unmap_bitmap(VFIOContainer *conta= iner, * qemu_real_host_page_size to mark those dirty. Hence set bitmap_pgsi= ze * to qemu_real_host_page_size. */ - bitmap->pgsize =3D qemu_real_host_page_size(); - bitmap->size =3D ROUND_UP(pages, sizeof(__u64) * BITS_PER_BYTE) / - BITS_PER_BYTE; + bitmap->size =3D vbmap.size; + bitmap->data =3D (__u64 *)vbmap.bitmap; =20 - if (bitmap->size > container->max_dirty_bitmap_size) { - error_report("UNMAP: Size of bitmap too big 0x%"PRIx64, - (uint64_t)bitmap->size); + if (vbmap.size > container->max_dirty_bitmap_size) { + error_report("UNMAP: Size of bitmap too big 0x%"PRIx64, vbmap.size= ); ret =3D -E2BIG; goto unmap_exit; } =20 - bitmap->data =3D g_try_malloc0(bitmap->size); - if (!bitmap->data) { - ret =3D -ENOMEM; - goto unmap_exit; - } - ret =3D ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, unmap); if (!ret) { - cpu_physical_memory_set_dirty_lebitmap((unsigned long *)bitmap->da= ta, - iotlb->translated_addr, pages); + cpu_physical_memory_set_dirty_lebitmap(vbmap.bitmap, + iotlb->translated_addr, vbmap.pages); } else { error_report("VFIO_UNMAP_DMA with DIRTY_BITMAP : %m"); } =20 - g_free(bitmap->data); unmap_exit: g_free(unmap); + g_free(vbmap.bitmap); + return ret; } =20 @@ -1329,7 +1346,7 @@ static int vfio_get_dirty_bitmap(VFIOContainer *conta= iner, uint64_t iova, { struct vfio_iommu_type1_dirty_bitmap *dbitmap; struct vfio_iommu_type1_dirty_bitmap_get *range; - uint64_t pages; + VFIOBitmap vbmap; int ret; =20 if (!container->dirty_pages_supported) { @@ -1339,6 +1356,11 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, return 0; } =20 + ret =3D vfio_bitmap_alloc(&vbmap, size); + if (ret) { + return ret; + } + dbitmap =3D g_malloc0(sizeof(*dbitmap) + sizeof(*range)); =20 dbitmap->argsz =3D sizeof(*dbitmap) + sizeof(*range); @@ -1353,15 +1375,8 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, * to qemu_real_host_page_size. */ range->bitmap.pgsize =3D qemu_real_host_page_size(); - - pages =3D REAL_HOST_PAGE_ALIGN(range->size) / qemu_real_host_page_size= (); - range->bitmap.size =3D ROUND_UP(pages, sizeof(__u64) * BITS_PER_BYTE) / - BITS_PER_BYTE; - range->bitmap.data =3D g_try_malloc0(range->bitmap.size); - if (!range->bitmap.data) { - ret =3D -ENOMEM; - goto err_out; - } + range->bitmap.size =3D vbmap.size; + range->bitmap.data =3D (__u64 *)vbmap.bitmap; =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { @@ -1372,14 +1387,14 @@ static int vfio_get_dirty_bitmap(VFIOContainer *con= tainer, uint64_t iova, goto err_out; } =20 - cpu_physical_memory_set_dirty_lebitmap((unsigned long *)range->bitmap.= data, - ram_addr, pages); + cpu_physical_memory_set_dirty_lebitmap(vbmap.bitmap, ram_addr, + vbmap.pages); =20 trace_vfio_get_dirty_bitmap(container->fd, range->iova, range->size, range->bitmap.size, ram_addr); err_out: - g_free(range->bitmap.data); g_free(dbitmap); + g_free(vbmap.bitmap); =20 return ret; } --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154738; cv=none; d=zohomail.com; s=zohoarc; b=jQVqO5zmvpt25i72HoJRjiUmG/Qv0YKXNTk5Z2uR4nMhfck7mXFwYNQtasHIohv30FDQ0uSUXpCpSRkIJf/DFx8HBgf+VZBuzgecKCASZWt3RFPncxAHuQHaR4nOzzjjrZUP8+aQ4PTeE3kk+ZjvZlmOgAfeEoVIjGjAtaij+v8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154738; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=j8Gk6mLjlQ2gE15Bhh0tmCiIiPoQ6uZ8kpS/YiwQP18=; b=AqTbIu4HOLH2ue7LBMMEjrpVPefEPVhbvAHPdWioM/Y+8uzlhbL7gqFuyoGNN5vPPrVZo5Ka49b+/Ya8Lh+iXXeBwO00vu0eNtXAH20bR98VqndfOaO9Lmd+oMNLTyzXCXLfDh//lQkHQCDXKj9b+dSnZ+Gmn/Qk2UCbfYMYi58= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154738896347.68813231159265; Mon, 6 Mar 2023 18:05:38 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMfr-0006qo-BQ; Mon, 06 Mar 2023 21:03:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfm-0006qP-GZ for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:26 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfk-0004w7-TE for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:26 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326Nwk6i021487; Tue, 7 Mar 2023 02:03:24 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p415hvfp0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:23 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3271UTZu038062; Tue, 7 Mar 2023 02:03:23 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txdubqu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:23 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 32723568016763; Tue, 7 Mar 2023 02:03:22 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-6; Tue, 07 Mar 2023 02:03:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=j8Gk6mLjlQ2gE15Bhh0tmCiIiPoQ6uZ8kpS/YiwQP18=; b=OCwb6HhYI8CMo4rB03Mr9bhDj/twYRpW/5JZ8TfydLEPhqGKiWN9Ryf5cgwppxnRW7cy yt/7N/mWeQbvZjAdbTj+sYOg471Yfoz5XgY8YBYZaTQ6qeVU9FsggjyG/Crh+Pb2TiwI m+3xpMTQ7TyLFOsugCMzlgR68HTJvsq/lhDWkfJQc9hNWjbQUfAxLkVKhezT+StPzkth 4T7/9zXNXGqfeVddTFRtFN7O1vk4FUpEhFCxoSFaCZFv0apLfXngU6joC7VzQNJNkhMJ QLJ3drP0TikqhOMY1Bd38EjQH950dYpCsYjhykrsxfvZJqqtg/21IRMBzZwHONeL/td7 4g== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 05/14] vfio/common: Add helper to validate iova/end against hostwin Date: Tue, 7 Mar 2023 02:02:49 +0000 Message-Id: <20230307020258.58215-6-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=807 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070016 X-Proofpoint-GUID: 3WSAd1SCr42PhhB-FcgIePdpULg-geo_ X-Proofpoint-ORIG-GUID: 3WSAd1SCr42PhhB-FcgIePdpULg-geo_ Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154740386100007 In preparation to be used in device dirty tracking, move the code that finds the container host DMA window against a iova range. This avoids duplication on the common checks across listener callbacks. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Avihai Horon --- hw/vfio/common.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index cec3de08d2b4..99acb998eb14 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -901,6 +901,22 @@ static void vfio_unregister_ram_discard_listener(VFIOC= ontainer *container, g_free(vrdl); } =20 +static VFIOHostDMAWindow *vfio_find_hostwin(VFIOContainer *container, + hwaddr iova, hwaddr end) +{ + VFIOHostDMAWindow *hostwin; + bool hostwin_found =3D false; + + QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { + if (hostwin->min_iova <=3D iova && end <=3D hostwin->max_iova) { + hostwin_found =3D true; + break; + } + } + + return hostwin_found ? hostwin : NULL; +} + static bool vfio_known_safe_misalignment(MemoryRegionSection *section) { MemoryRegion *mr =3D section->mr; @@ -926,7 +942,6 @@ static void vfio_listener_region_add(MemoryListener *li= stener, void *vaddr; int ret; VFIOHostDMAWindow *hostwin; - bool hostwin_found; Error *err =3D NULL; =20 if (vfio_listener_skipped_section(section)) { @@ -1027,15 +1042,8 @@ static void vfio_listener_region_add(MemoryListener = *listener, #endif } =20 - hostwin_found =3D false; - QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { - if (hostwin->min_iova <=3D iova && end <=3D hostwin->max_iova) { - hostwin_found =3D true; - break; - } - } - - if (!hostwin_found) { + hostwin =3D vfio_find_hostwin(container, iova, end); + if (!hostwin) { error_setg(&err, "Container %p can't map guest IOVA region" " 0x%"HWADDR_PRIx"..0x%"HWADDR_PRIx, container, iova, e= nd); goto fail; @@ -1237,15 +1245,9 @@ static void vfio_listener_region_del(MemoryListener = *listener, if (memory_region_is_ram_device(section->mr)) { hwaddr pgmask; VFIOHostDMAWindow *hostwin; - bool hostwin_found =3D false; =20 - QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { - if (hostwin->min_iova <=3D iova && end <=3D hostwin->max_iova)= { - hostwin_found =3D true; - break; - } - } - assert(hostwin_found); /* or region_add() would have failed */ + hostwin =3D vfio_find_hostwin(container, iova, end); + assert(hostwin); /* or region_add() would have failed */ =20 pgmask =3D (1ULL << ctz64(hostwin->iova_pgsizes)) - 1; try_unmap =3D !((iova & pgmask) || (int128_get64(llsize) & pgmask)= ); --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154737; cv=none; d=zohomail.com; s=zohoarc; b=UYBMCN1C5FoPvh67qxwUq6pxWQ6GrDfptKYLb0NfLEPMn5CuFmsmxUC+9+Wtj/VZMYcc0dz20+avBQegTVtZiARK+KWPZmNnxskbEBLP1BU4e+gsUUCCMuBj1KiBQeaAAeAKVXusQaP8gKJ4h59oAppTS1NTg02NstcX6pcMdDg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154737; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=U424iuqvq3TR9AV41TeI7ouo9ejdeHYzTj2iwoghSbo=; b=A9nXoG+7//8YVgR1GNWPFESsgMije2vtvkfztdbp0ou2PrUHXKSu6U6rPYGRrjwjVCZEx9yCe0QnSt8VmaCkLXYdLFLkrK+q5YRv53Bfu749v4KDYQOawtqyfd/LAkwnlWjXXA8GHIvJZXkr5Low8QvjX8OleTOQofNCbUA1F68= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154737378936.639618092435; Mon, 6 Mar 2023 18:05:37 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMfu-0006s1-Ia; Mon, 06 Mar 2023 21:03:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfs-0006rC-A1 for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:32 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfo-0004wJ-KV for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:31 -0500 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NxYsM030050; Tue, 7 Mar 2023 02:03:27 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p418xvfe3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:27 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3271UTZv038062; Tue, 7 Mar 2023 02:03:25 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txdubsf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:25 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3272356A016763; Tue, 7 Mar 2023 02:03:25 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-7; Tue, 07 Mar 2023 02:03:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=U424iuqvq3TR9AV41TeI7ouo9ejdeHYzTj2iwoghSbo=; b=UlIaSMpX8gHQsTMitBCVmC8c5KGNc7h26sOFZYuxNb+LCO12UgMrO+gNu+sdVKlMW0Iq iDunWdJVVLv9DmL/96kfdx35Jy+zrAzebFW/WMgSLBg0jCRtr2F7G0JSY9qAjW1Xa7KK 9MSpfNuZ99cpkW1ETfHU6M1V9NRUMhD9p1C2LcYKBv8VYCsXW6uahQoxknSIr3VdIXz+ gSaMyZp7oEG0sRIrb6YMC6XQ5p3RrzVCwGFtGFBCG9PTubNN8YYEtkjUAOy4dLXNow5e rJHZNPlUduTZFmRj1107pKVHyXtBiz9uxE26ngRMLz7hLGVYKb1aZ5SxTq4L7sx/7Czn Tg== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 06/14] vfio/common: Consolidate skip/invalid section into helper Date: Tue, 7 Mar 2023 02:02:50 +0000 Message-Id: <20230307020258.58215-7-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=641 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070016 X-Proofpoint-GUID: PV3J8wcV388D5fSPhlNIQMdn5BLiLfZ- X-Proofpoint-ORIG-GUID: PV3J8wcV388D5fSPhlNIQMdn5BLiLfZ- Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154738386100001 The checks are replicated against region_add and region_del and will be soon added in another memory listener dedicated for dirty tracking. Move these into a new helper for avoid duplication. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Avihai Horon --- hw/vfio/common.c | 52 +++++++++++++++++++----------------------------- 1 file changed, 21 insertions(+), 31 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 99acb998eb14..54b4a4fc7daf 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -933,23 +933,14 @@ static bool vfio_known_safe_misalignment(MemoryRegion= Section *section) return true; } =20 -static void vfio_listener_region_add(MemoryListener *listener, - MemoryRegionSection *section) +static bool vfio_listener_valid_section(MemoryRegionSection *section) { - VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); - hwaddr iova, end; - Int128 llend, llsize; - void *vaddr; - int ret; - VFIOHostDMAWindow *hostwin; - Error *err =3D NULL; - if (vfio_listener_skipped_section(section)) { trace_vfio_listener_region_add_skip( section->offset_within_address_space, section->offset_within_address_space + int128_get64(int128_sub(section->size, int128_one()))); - return; + return false; } =20 if (unlikely((section->offset_within_address_space & @@ -964,6 +955,24 @@ static void vfio_listener_region_add(MemoryListener *l= istener, section->offset_within_region, qemu_real_host_page_size()); } + return false; + } + + return true; +} + +static void vfio_listener_region_add(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + hwaddr iova, end; + Int128 llend, llsize; + void *vaddr; + int ret; + VFIOHostDMAWindow *hostwin; + Error *err =3D NULL; + + if (!vfio_listener_valid_section(section)) { return; } =20 @@ -1182,26 +1191,7 @@ static void vfio_listener_region_del(MemoryListener = *listener, int ret; bool try_unmap =3D true; =20 - if (vfio_listener_skipped_section(section)) { - trace_vfio_listener_region_del_skip( - section->offset_within_address_space, - section->offset_within_address_space + - int128_get64(int128_sub(section->size, int128_one()))); - return; - } - - if (unlikely((section->offset_within_address_space & - ~qemu_real_host_page_mask()) !=3D - (section->offset_within_region & ~qemu_real_host_page_mas= k()))) { - if (!vfio_known_safe_misalignment(section)) { - error_report("%s received unaligned region %s iova=3D0x%"PRIx64 - " offset_within_region=3D0x%"PRIx64 - " qemu_real_host_page_size=3D0x%"PRIxPTR, - __func__, memory_region_name(section->mr), - section->offset_within_address_space, - section->offset_within_region, - qemu_real_host_page_size()); - } + if (!vfio_listener_valid_section(section)) { return; } =20 --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154660; cv=none; d=zohomail.com; s=zohoarc; b=iZUDhdcpjInilwznr8QQbdkxCTzS0olb+OD/7TLuegwNhpDfkmxDjalSUhryJgQHkl0Y7vVKbgdOoLTGWlGPmk1gNeXaMwJWiJgnkG6VepQewNQQClCO4VUOn+EFBP7G7XWr3TvjQa09jabzltdMztEztsoIFySIMZ5pAgITY7E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154660; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2o8V37OeLoQqX5tU00a0f9oNCHZwmWI6Rq26xHMurKw=; b=RHT5DJ2noZ97StrJzOzGQ18wURi0gb2ydIGxfXoCLEmThzP9K+3kNqU8KdyKJHnhXKEweEFoTOaNQEF5K2YxqFd4I6usoCI4Xqhha4s+CrgG3scNEg+9XGRN/dHd42+q65ipJ+KEc3m8tFiU46hDiVpR1DYmgLD0Y2SqA7i0QqM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154660807946.0263741509102; Mon, 6 Mar 2023 18:04:20 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMft-0006rk-Vu; Mon, 06 Mar 2023 21:03:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfs-0006rE-8S for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:32 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMfq-0004wN-MB for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:31 -0500 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NwpWr006043; Tue, 7 Mar 2023 02:03:29 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p4168mene-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:29 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3271kr00037245; Tue, 7 Mar 2023 02:03:28 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txdubu7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:28 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3272356C016763; Tue, 7 Mar 2023 02:03:28 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-8; Tue, 07 Mar 2023 02:03:28 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=2o8V37OeLoQqX5tU00a0f9oNCHZwmWI6Rq26xHMurKw=; b=UBBIiEarQ9VoH5sHwk15obiujPv3HjS7FT2oE6lVq8M6FrgYf1vS86YfZuwv36b0POPu NUu1puniQmzzqkadEMkWEaCa9TZmlGcepjzG7dt5ReuEPRj+JOHthYTMyk+UDbvjj+sq GGsQXTXDWSc8GwF5RhbQ3f3xQyzDpJDjRLBvMgiLO/ami1DgGp2yJUPNthu7fBuJHdjt BBFkydfG1kv6ilDb7Ncoj3NaJDKQODDakqmcyTtpG9N9auVB3UW61x5vUN/B8KYPU8R6 eBCMUV0NcLGzvHBosos/d1vLfkMpxTAK5xGhbUD7jAXS3SsB9nMNdAYo657xmWjL8sDI WA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 07/14] vfio/common: Add helper to consolidate iova/end calculation Date: Tue, 7 Mar 2023 02:02:51 +0000 Message-Id: <20230307020258.58215-8-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=735 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070016 X-Proofpoint-GUID: ZhB-dNGpxu8JAhZhemkl9TMu7wXH8hDY X-Proofpoint-ORIG-GUID: ZhB-dNGpxu8JAhZhemkl9TMu7wXH8hDY Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154661965100003 Content-Type: text/plain; charset="utf-8" In preparation to be used in device dirty tracking, move the code that calculates a iova/end range from the container/section. This avoids duplication on the common checks across listener callbacks. Signed-off-by: Joao Martins --- hw/vfio/common.c | 37 ++++++++++++++++++++++++++++++------- 1 file changed, 30 insertions(+), 7 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 54b4a4fc7daf..3a6491dbc523 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -961,6 +961,35 @@ static bool vfio_listener_valid_section(MemoryRegionSe= ction *section) return true; } =20 +/* + * Called for the dirty tracking memory listener to calculate the iova/end + * for a given memory region section. + */ +static bool vfio_get_section_iova_range(VFIOContainer *container, + MemoryRegionSection *section, + hwaddr *out_iova, hwaddr *out_end, + Int128 *out_llend) +{ + Int128 llend; + hwaddr iova; + + iova =3D REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); + llend =3D int128_make64(section->offset_within_address_space); + llend =3D int128_add(llend, section->size); + llend =3D int128_and(llend, int128_exts64(qemu_real_host_page_mask())); + + if (int128_ge(int128_make64(iova), llend)) { + return false; + } + + *out_iova =3D iova; + *out_end =3D int128_get64(int128_sub(llend, int128_one())); + if (out_llend) { + *out_llend =3D llend; + } + return true; +} + static void vfio_listener_region_add(MemoryListener *listener, MemoryRegionSection *section) { @@ -976,12 +1005,7 @@ static void vfio_listener_region_add(MemoryListener *= listener, return; } =20 - iova =3D REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); - llend =3D int128_make64(section->offset_within_address_space); - llend =3D int128_add(llend, section->size); - llend =3D int128_and(llend, int128_exts64(qemu_real_host_page_mask())); - - if (int128_ge(int128_make64(iova), llend)) { + if (!vfio_get_section_iova_range(container, section, &iova, &end, &lle= nd)) { if (memory_region_is_ram_device(section->mr)) { trace_vfio_listener_region_add_no_dma_map( memory_region_name(section->mr), @@ -991,7 +1015,6 @@ static void vfio_listener_region_add(MemoryListener *l= istener, } return; } - end =3D int128_get64(int128_sub(llend, int128_one())); =20 if (container->iommu_type =3D=3D VFIO_SPAPR_TCE_v2_IOMMU) { hwaddr pgsize =3D 0; --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154712; cv=none; d=zohomail.com; s=zohoarc; b=JiIXK/hiZhcRklo+EMIwfu3sS2vJK7f84o/ImbHG3TY2SCXLsvAk22q08QbES9YwF3uUKKADJ8NIWlRA/gYd6r293J0RFPw3jVf9jyaADBhIO1p2kgo3OEbPJbxbAo++ZYoLW+rBdDaSx7Fdrg+hcoSxTuR4rgB0WVLB0om9pu4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154712; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PxHDPVIQjHL8XFixusSmQZEvlQhds06ZnieTZQGjNKU=; b=NPisHLqCjqGMXEgxAtUMcXsCebPIeL20NdWhN/UdTy0I0poNQbCYlh9MkQf19DzS5p0kQBIA0Ex+Kl6oeP/h1+8EFHYVwQxjCoUjTuZQpQ+67Ro05Pc17pqjYp/072QGdbBYMPy4dUQORgq9wSZ50IamkwM5skaTZB1ZTiVg6v0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154712336876.6689269254873; Mon, 6 Mar 2023 18:05:12 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMg5-0006t3-Ch; Mon, 06 Mar 2023 21:03:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMg4-0006sq-LW for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:44 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMg0-0004wr-8F for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:44 -0500 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NxVEv007719; Tue, 7 Mar 2023 02:03:38 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p4161vdxp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:38 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3271fLw9037432; Tue, 7 Mar 2023 02:03:37 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txdubxc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:37 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3272356E016763; Tue, 7 Mar 2023 02:03:36 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-9; Tue, 07 Mar 2023 02:03:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=PxHDPVIQjHL8XFixusSmQZEvlQhds06ZnieTZQGjNKU=; b=Qkke71A8Z+8/S5iK9QJ3AYEXf8XKl7PW0FLOAJT/QgtUWazZ11vzz2dZHVktExg+UeVa Fn30bRnREr9ueo2/euZGSf/xWfmF0zIoon5i9Thw9jKNuCzEa9xZlzvVOH60/aSwoKiu avVayWXevWGsilRy00QNEmct+ioUsltTai995kFYSzA1iOFcN973NbUURmUFDT4rSf9U yLuI4rFYSerfX0uerpXKYsaqxnUOEv80GKb8eBFYEsKT8yHDkg+BFDSMJ6MYWrQB/mQx t/E3vg6+tY4MYP003tNZcEYhzLKrsQUB7/vAwQdYgJwt19xTdCQSgfLElbC98eBWtQFn 8Q== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 08/14] vfio/common: Record DMA mapped IOVA ranges Date: Tue, 7 Mar 2023 02:02:52 +0000 Message-Id: <20230307020258.58215-9-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=943 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070017 X-Proofpoint-GUID: RqnppRBWl2LonxXyStQHPixO3X7pXBG0 X-Proofpoint-ORIG-GUID: RqnppRBWl2LonxXyStQHPixO3X7pXBG0 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154714350100011 Content-Type: text/plain; charset="utf-8" According to the device DMA logging uAPI, IOVA ranges to be logged by the device must be provided all at once upon DMA logging start. As preparation for the following patches which will add device dirty page tracking, keep a record of all DMA mapped IOVA ranges so later they can be used for DMA logging start. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 76 +++++++++++++++++++++++++++++++++++ hw/vfio/trace-events | 1 + include/hw/vfio/vfio-common.h | 13 ++++++ 3 files changed, 90 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 3a6491dbc523..a9b1fc999121 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1334,11 +1334,87 @@ static int vfio_set_dirty_page_tracking(VFIOContain= er *container, bool start) return ret; } =20 +static void vfio_dirty_tracking_update(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIODirtyRanges *dirty =3D container_of(listener, VFIODirtyRanges, lis= tener); + VFIODirtyTrackingRange *range =3D &dirty->ranges; + hwaddr max32 =3D UINT32_MAX - 1ULL; + hwaddr iova, end; + + if (!vfio_listener_valid_section(section) || + !vfio_get_section_iova_range(dirty->container, section, + &iova, &end, NULL)) { + return; + } + + /* + * The address space passed to the dirty tracker is reduced to two ran= ges: + * one for 32-bit DMA ranges, and another one for 64-bit DMA ranges. + * The underlying reports of dirty will query a sub-interval of each of + * these ranges. + * + * The purpose of the dual range handling is to handle known cases of = big + * holes in the address space, like the x86 AMD 1T hole. The alternati= ve + * would be an IOVATree but that has a much bigger runtime overhead and + * unnecessary complexity. + */ + if (iova < max32 && end <=3D max32) { + if (range->min32 > iova) { + range->min32 =3D iova; + } + if (range->max32 < end) { + range->max32 =3D end; + } + trace_vfio_device_dirty_tracking_update(iova, end, + range->min32, range->max32); + } else { + if (!range->min64 || range->min64 > iova) { + range->min64 =3D iova; + } + if (range->max64 < end) { + range->max64 =3D end; + } + trace_vfio_device_dirty_tracking_update(iova, end, + range->min64, range->max64); + } + + return; +} + +static const MemoryListener vfio_dirty_tracking_listener =3D { + .name =3D "vfio-tracking", + .region_add =3D vfio_dirty_tracking_update, +}; + +static void vfio_dirty_tracking_init(VFIOContainer *container, + VFIODirtyRanges *dirty) +{ + memset(dirty, 0, sizeof(*dirty)); + dirty->ranges.min32 =3D UINT32_MAX; + dirty->ranges.min64 =3D UINT64_MAX; + dirty->listener =3D vfio_dirty_tracking_listener; + dirty->container =3D container; + + memory_listener_register(&dirty->listener, + container->space->as); + + /* + * The memory listener is synchronous, and used to calculate the range + * to dirty tracking. Unregister it after we are done as we are not + * interested in any follow-up updates. + */ + memory_listener_unregister(&dirty->listener); +} + static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + VFIODirtyRanges dirty; int ret; =20 + vfio_dirty_tracking_init(container, &dirty); + ret =3D vfio_set_dirty_page_tracking(container, true); if (ret) { vfio_set_migration_error(ret); diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 669d9fe07cd9..d97a6de17921 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -104,6 +104,7 @@ vfio_known_safe_misalignment(const char *name, uint64_t= iova, uint64_t offset_wi vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint6= 4_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=3D0x%"PRIx64"= is not aligned to 0x%"PRIx64" and cannot be mapped for DMA" vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING regi= on_del 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRI= x64" - 0x%"PRIx64 +vfio_device_dirty_tracking_update(uint64_t start, uint64_t end, uint64_t m= in, uint64_t max) "section 0x%"PRIx64" - 0x%"PRIx64" -> update [0x%"PRIx64"= - 0x%"PRIx64"]" vfio_disconnect_container(int fd) "close container->fd=3D%d" vfio_put_group(int fd) "close group->fd=3D%d" vfio_get_device(const char * name, unsigned int flags, unsigned int num_re= gions, unsigned int num_irqs) "Device %s flags: %u, regions: %u, irqs: %u" diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 87524c64a443..0f84136cceb5 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -96,6 +96,19 @@ typedef struct VFIOContainer { QLIST_ENTRY(VFIOContainer) next; } VFIOContainer; =20 +typedef struct VFIODirtyTrackingRange { + hwaddr min32; + hwaddr max32; + hwaddr min64; + hwaddr max64; +} VFIODirtyTrackingRange; + +typedef struct VFIODirtyRanges { + VFIOContainer *container; + VFIODirtyTrackingRange ranges; + MemoryListener listener; +} VFIODirtyRanges; + typedef struct VFIOGuestIOMMU { VFIOContainer *container; IOMMUMemoryRegion *iommu_mr; --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154708; cv=none; d=zohomail.com; s=zohoarc; b=dUW19duOikSina2Yb9upjT4C0DJlmCC//C+t+Tq+KxLNdS57iVBEom7+MgYKmABbmanFFfElEJ9ymG6eA9wHhHkbrEQytjHUSx8XfuCMcJs71yBxQE1HWQ1UqPaGDppRPM+08IooBo+BZr30Qbc9x8rxwpx8bDxz/UPey9ykA40= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154708; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vKfbrZSlP3Ezo9kwPcK88VsrHB2i63r60ouvVb4cxac=; b=e/PF08m5lb9Y3cY5jU+1Qor8bt4C/vQpcaXhSdI3rDtViOQawD9DksR+qNW6CLd2KXsr5LippYtWGDBh1TAGSoBIjH49GRsSyEdSov52YLOH0K+m73W6WUueb77Xx2ZpNrtykRgO3u3UZNgQIQIIsJmTj+Tvy+1DOyp5ORRF+ZI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154708017908.2866335686471; Mon, 6 Mar 2023 18:05:08 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMg6-0006tm-W4; Mon, 06 Mar 2023 21:03:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMg5-0006sz-5G for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:45 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMg3-0004ww-4O for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:44 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326Nx6CO025153; Tue, 7 Mar 2023 02:03:41 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p5nn90s1d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:41 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 326NT6Kr037009; Tue, 7 Mar 2023 02:03:40 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txdubyp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:40 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3272356G016763; Tue, 7 Mar 2023 02:03:39 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-10; Tue, 07 Mar 2023 02:03:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=vKfbrZSlP3Ezo9kwPcK88VsrHB2i63r60ouvVb4cxac=; b=gew/UQ+4CjCZY3KcMVReagpdneXoOE9tSaSIMURblG6gzGvSZsMAAvxJ3zUV7kWm82WA RQerPLwh0NxoVLS/LapOg7hSWdaaXK6KTHkAr14LsetGfBw0gcWlre40pAZ7q71gEwEM FbzZ+UmRpcFd3RZFrUsw9crrVdLYF3R1H20tV9uC0EAxS1WpimU2He8EVpgrMxvaSZ3W 9IkbPdIzxyxS+tipa2NGN7UlBiEw2vIPtrGFQzMXef8ekKI98r8AXist/pzwPwUxtny7 ZRHxDXtkn9xzBWINEBcf0U3gNJJfIm4MPAWb8Va9iFOEk8ZGVxhWYw70gJ25WoUJaIqt Zw== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 09/14] vfio/common: Add device dirty page tracking start/stop Date: Tue, 7 Mar 2023 02:02:53 +0000 Message-Id: <20230307020258.58215-10-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=771 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070017 X-Proofpoint-GUID: 3EbEo4SK8dvwJw8LKRAIoTEz4niRxaCl X-Proofpoint-ORIG-GUID: 3EbEo4SK8dvwJw8LKRAIoTEz4niRxaCl Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154708340100001 Content-Type: text/plain; charset="utf-8" Add device dirty page tracking start/stop functionality. This uses the device DMA logging uAPI to start and stop dirty page tracking by device. Device dirty page tracking is used only if all devices within a container support device dirty page tracking. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 175 +++++++++++++++++++++++++++++++++- hw/vfio/trace-events | 1 + include/hw/vfio/vfio-common.h | 2 + 3 files changed, 173 insertions(+), 5 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index a9b1fc999121..a42f5f1e7ffe 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -450,6 +450,22 @@ static bool vfio_devices_all_dirty_tracking(VFIOContai= ner *container) return true; } =20 +static bool vfio_devices_all_device_dirty_tracking(VFIOContainer *containe= r) +{ + VFIOGroup *group; + VFIODevice *vbasedev; + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (!vbasedev->dirty_pages_supported) { + return false; + } + } + } + + return true; +} + /* * Check if all VFIO devices are running and migration is active, which is * essentially equivalent to the migration being in pre-copy phase. @@ -1407,16 +1423,158 @@ static void vfio_dirty_tracking_init(VFIOContainer= *container, memory_listener_unregister(&dirty->listener); } =20 +static void vfio_devices_dma_logging_stop(VFIOContainer *container) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + VFIODevice *vbasedev; + VFIOGroup *group; + int ret =3D 0; + + feature->argsz =3D sizeof(buf); + feature->flags =3D VFIO_DEVICE_FEATURE_SET | + VFIO_DEVICE_FEATURE_DMA_LOGGING_STOP; + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (!vbasedev->dirty_tracking) { + continue; + } + + ret =3D ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); + if (ret) { + warn_report("%s: Failed to stop DMA logging, err %d (%s)", + vbasedev->name, ret, strerror(errno)); + } + vbasedev->dirty_tracking =3D false; + } + } +} + +static struct vfio_device_feature * +vfio_device_feature_dma_logging_start_create(VFIOContainer *container, + VFIODirtyTrackingRange *track= ing) +{ + struct vfio_device_feature *feature; + size_t feature_size; + struct vfio_device_feature_dma_logging_control *control; + struct vfio_device_feature_dma_logging_range *ranges; + + feature_size =3D sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_dma_logging_control); + feature =3D g_try_malloc0(feature_size); + if (!feature) { + errno =3D ENOMEM; + return NULL; + } + feature->argsz =3D feature_size; + feature->flags =3D VFIO_DEVICE_FEATURE_SET | + VFIO_DEVICE_FEATURE_DMA_LOGGING_START; + + control =3D (struct vfio_device_feature_dma_logging_control *)feature-= >data; + control->page_size =3D qemu_real_host_page_size(); + + /* + * DMA logging uAPI guarantees to support at least a number of ranges = that + * fits into a single host kernel base page. + */ + control->num_ranges =3D !!tracking->max32 + !!tracking->max64; + ranges =3D g_try_new0(struct vfio_device_feature_dma_logging_range, + control->num_ranges); + if (!ranges) { + g_free(feature); + errno =3D ENOMEM; + + return NULL; + } + + control->ranges =3D (__u64)(uintptr_t)ranges; + if (tracking->max32) { + ranges->iova =3D tracking->min32; + ranges->length =3D (tracking->max32 - tracking->min32) + 1; + ranges++; + } + if (tracking->max64) { + ranges->iova =3D tracking->min64; + ranges->length =3D (tracking->max64 - tracking->min64) + 1; + } + + trace_vfio_device_dirty_tracking_start(control->num_ranges, + tracking->min32, tracking->max3= 2, + tracking->min64, tracking->max6= 4); + + return feature; +} + +static void vfio_device_feature_dma_logging_start_destroy( + struct vfio_device_feature *feature) +{ + struct vfio_device_feature_dma_logging_control *control =3D + (struct vfio_device_feature_dma_logging_control *)feature->data; + struct vfio_device_feature_dma_logging_range *ranges =3D + (struct vfio_device_feature_dma_logging_range *)(uintptr_t) contro= l->ranges; + + g_free(ranges); + g_free(feature); +} + +static int vfio_devices_dma_logging_start(VFIOContainer *container) +{ + struct vfio_device_feature *feature; + VFIODirtyRanges dirty; + VFIODevice *vbasedev; + VFIOGroup *group; + int ret =3D 0; + + vfio_dirty_tracking_init(container, &dirty); + feature =3D vfio_device_feature_dma_logging_start_create(container, + &dirty.ranges); + if (!feature) { + return -errno; + } + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (vbasedev->dirty_tracking) { + continue; + } + + ret =3D ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); + if (ret) { + ret =3D -errno; + error_report("%s: Failed to start DMA logging, err %d (%s)= ", + vbasedev->name, ret, strerror(errno)); + goto out; + } + vbasedev->dirty_tracking =3D true; + } + } + +out: + if (ret) { + vfio_devices_dma_logging_stop(container); + } + + vfio_device_feature_dma_logging_start_destroy(feature); + + return ret; +} + static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); - VFIODirtyRanges dirty; int ret; =20 - vfio_dirty_tracking_init(container, &dirty); + if (vfio_devices_all_device_dirty_tracking(container)) { + ret =3D vfio_devices_dma_logging_start(container); + } else { + ret =3D vfio_set_dirty_page_tracking(container, true); + } =20 - ret =3D vfio_set_dirty_page_tracking(container, true); if (ret) { + error_report("vfio: Could not start dirty page tracking, err: %d (= %s)", + ret, strerror(-ret)); vfio_set_migration_error(ret); } } @@ -1424,10 +1582,17 @@ static void vfio_listener_log_global_start(MemoryLi= stener *listener) static void vfio_listener_log_global_stop(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); - int ret; + int ret =3D 0; + + if (vfio_devices_all_device_dirty_tracking(container)) { + vfio_devices_dma_logging_stop(container); + } else { + ret =3D vfio_set_dirty_page_tracking(container, false); + } =20 - ret =3D vfio_set_dirty_page_tracking(container, false); if (ret) { + error_report("vfio: Could not stop dirty page tracking, err: %d (%= s)", + ret, strerror(-ret)); vfio_set_migration_error(ret); } } diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index d97a6de17921..7a7e0cfe5b23 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -105,6 +105,7 @@ vfio_listener_region_add_no_dma_map(const char *name, u= int64_t iova, uint64_t si vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING regi= on_del 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRI= x64" - 0x%"PRIx64 vfio_device_dirty_tracking_update(uint64_t start, uint64_t end, uint64_t m= in, uint64_t max) "section 0x%"PRIx64" - 0x%"PRIx64" -> update [0x%"PRIx64"= - 0x%"PRIx64"]" +vfio_device_dirty_tracking_start(int nr_ranges, uint64_t min32, uint64_t m= ax32, uint64_t min64, uint64_t max64) "nr_ranges %d 32:[0x%"PRIx64" - 0x%"P= RIx64"], 64:[0x%"PRIx64" - 0x%"PRIx64"]" vfio_disconnect_container(int fd) "close container->fd=3D%d" vfio_put_group(int fd) "close group->fd=3D%d" vfio_get_device(const char * name, unsigned int flags, unsigned int num_re= gions, unsigned int num_irqs) "Device %s flags: %u, regions: %u, irqs: %u" diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 0f84136cceb5..7817ca7d8706 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -156,6 +156,8 @@ typedef struct VFIODevice { VFIOMigration *migration; Error *migration_blocker; OnOffAuto pre_copy_dirty_page_tracking; + bool dirty_pages_supported; + bool dirty_tracking; } VFIODevice; =20 struct VFIODeviceOps { --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154717; cv=none; d=zohomail.com; s=zohoarc; b=hSVr6ivpPamg9QoNyShMaa/l3qRDfmpJ1mE8A710BD5H6XxLKg4WpN8cPIyrDwZlV3r32Nv9LKAQS88oXpnzVT8O1/ccMT6SpVgoYQ8uqtS6xfpoHDjV/k/qpkTzX7fu2Nu38ffc34SQ2s5PW744cvCBfI6bLKMioyFXqQjfuKU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154717; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=+hLAnqQDZ8k5x6NfAr4Ghuxu6cwVnOjdcp6s8DtGNGI=; b=hDmUb46sl6SOIX18oCqbo4uXhZbAdlAu9eM9tgrgjP9qaASdW1frXB9r3OxXU2+0l1J9Xpe38HjydPDT5V0pMnyWvbXXD9smNHY1UZupkcU9k62dmqkxRVsmwdRcIV5OwC6a6mKWIpH5e6IA8CtEd4xWgPNLjjZ1rUyIBu7fyvk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154717243180.44619635726076; Mon, 6 Mar 2023 18:05:17 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMgA-0006u4-2A; Mon, 06 Mar 2023 21:03:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMg8-0006tu-Bh for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:48 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMg6-0004xJ-Js for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:48 -0500 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NxHdR025813; Tue, 7 Mar 2023 02:03:45 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p416wmf2u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:44 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 326NSB3x038193; Tue, 7 Mar 2023 02:03:43 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txduc17-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:43 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3272356I016763; Tue, 7 Mar 2023 02:03:43 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-11; Tue, 07 Mar 2023 02:03:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=+hLAnqQDZ8k5x6NfAr4Ghuxu6cwVnOjdcp6s8DtGNGI=; b=LUFK2a18HNHlA7hhvAyrwJPrqhP0dDnf6emApckRt6XlqsDBhiEkC+vEeKWd31C5ybFM kMmQMoL+MNPx1xYvDGakBBoHFWow9SELiyhWJvrpE593AlQqMutGDxFPXsu52AzxfJFU rLLDSKvHRKsBMFeDpxfGc+hZasC+4EjwoARAqCXA+pBDEOEQW3KbocN7KF4a+XgJaT5c im34ouPOpGgcw8ckUomRPA3ZeJPb1bphnfC0tqjxgqeAr/ECiF/Srt68sCeuvjkyP5or GmtKpDC2Il5b+8oj78yO+XGU+S1ls8dbkQ99zi1duogWuVLOOIaRgpFVcxPjPh874+LL Ug== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 10/14] vfio/common: Extract code from vfio_get_dirty_bitmap() to new function Date: Tue, 7 Mar 2023 02:02:54 +0000 Message-Id: <20230307020258.58215-11-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=710 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070017 X-Proofpoint-GUID: GdA4HZD1xja-jBOykaNbG9sR6mmMFYk7 X-Proofpoint-ORIG-GUID: GdA4HZD1xja-jBOykaNbG9sR6mmMFYk7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154718320100003 From: Avihai Horon Extract the VFIO_IOMMU_DIRTY_PAGES ioctl code in vfio_get_dirty_bitmap() to its own function. This will help the code to be more readable after next patch will add device dirty page bitmap sync functionality. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 57 +++++++++++++++++++++++++++++------------------- 1 file changed, 35 insertions(+), 22 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index a42f5f1e7ffe..136665ca2c4e 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1597,26 +1597,13 @@ static void vfio_listener_log_global_stop(MemoryLis= tener *listener) } } =20 -static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, - uint64_t size, ram_addr_t ram_addr) +static int vfio_query_dirty_bitmap(VFIOContainer *container, VFIOBitmap *v= bmap, + hwaddr iova, hwaddr size) { struct vfio_iommu_type1_dirty_bitmap *dbitmap; struct vfio_iommu_type1_dirty_bitmap_get *range; - VFIOBitmap vbmap; int ret; =20 - if (!container->dirty_pages_supported) { - cpu_physical_memory_set_dirty_range(ram_addr, size, - tcg_enabled() ? DIRTY_CLIENTS_= ALL : - DIRTY_CLIENTS_NOCODE); - return 0; - } - - ret =3D vfio_bitmap_alloc(&vbmap, size); - if (ret) { - return ret; - } - dbitmap =3D g_malloc0(sizeof(*dbitmap) + sizeof(*range)); =20 dbitmap->argsz =3D sizeof(*dbitmap) + sizeof(*range); @@ -1631,8 +1618,8 @@ static int vfio_get_dirty_bitmap(VFIOContainer *conta= iner, uint64_t iova, * to qemu_real_host_page_size. */ range->bitmap.pgsize =3D qemu_real_host_page_size(); - range->bitmap.size =3D vbmap.size; - range->bitmap.data =3D (__u64 *)vbmap.bitmap; + range->bitmap.size =3D vbmap->size; + range->bitmap.data =3D (__u64 *)vbmap->bitmap; =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { @@ -1640,16 +1627,42 @@ static int vfio_get_dirty_bitmap(VFIOContainer *con= tainer, uint64_t iova, error_report("Failed to get dirty bitmap for iova: 0x%"PRIx64 " size: 0x%"PRIx64" err: %d", (uint64_t)range->iova, (uint64_t)range->size, errno); - goto err_out; + } + + g_free(dbitmap); + + return ret; +} + +static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, + uint64_t size, ram_addr_t ram_addr) +{ + VFIOBitmap vbmap; + int ret; + + if (!container->dirty_pages_supported) { + cpu_physical_memory_set_dirty_range(ram_addr, size, + tcg_enabled() ? DIRTY_CLIENTS_= ALL : + DIRTY_CLIENTS_NOCODE); + return 0; + } + + ret =3D vfio_bitmap_alloc(&vbmap, size); + if (ret) { + return ret; + } + + ret =3D vfio_query_dirty_bitmap(container, &vbmap, iova, size); + if (ret) { + goto out; } =20 cpu_physical_memory_set_dirty_lebitmap(vbmap.bitmap, ram_addr, vbmap.pages); =20 - trace_vfio_get_dirty_bitmap(container->fd, range->iova, range->size, - range->bitmap.size, ram_addr); -err_out: - g_free(dbitmap); + trace_vfio_get_dirty_bitmap(container->fd, iova, size, vbmap.size, + ram_addr); +out: g_free(vbmap.bitmap); =20 return ret; --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154643; cv=none; d=zohomail.com; s=zohoarc; b=Qa5aDu4ryc+L7pTh4GW4HkX88/7nGrpUIu21fkAIt0iw1+ZkRFVJ17Ii7EBxXAynxsHChntAgAE3/BKNfSvTFWwaAWRyeUxMzkERYAyTZ7RC4W9n9RrCCPEu44KIYLUONuyZ/u49LjIbpACKp8sBr+2a4STeMvEumIAefZunzik= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154643; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HOPMxMJKNR21GNecMXMpNUhj9OLewVm6I3NQuDX2/ko=; b=bTYkEr4UampFftpfRep6Q+AcxidczsiCzJgELb3zWbZGIzzT86iBxF0gdq45V3E1b04BBN63ilTsKffxVNjpNeutmfKoSjDQNgj2fB8MSo7qElu8VoiWGuLUT7dvaiscsvgC9OtLInL3Y299sHQUxQHJpIsueTAKmsPYkynWk/g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154643865320.1906999008107; Mon, 6 Mar 2023 18:04:03 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMgG-00072b-8l; Mon, 06 Mar 2023 21:03:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMgE-000701-IG for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:54 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMg9-0004xY-9d for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:50 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NxZ2h026520; Tue, 7 Mar 2023 02:03:48 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p5nn90s1j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:47 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3271svPq037115; Tue, 7 Mar 2023 02:03:46 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txduc2k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:46 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3272356K016763; Tue, 7 Mar 2023 02:03:46 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-12; Tue, 07 Mar 2023 02:03:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=HOPMxMJKNR21GNecMXMpNUhj9OLewVm6I3NQuDX2/ko=; b=sxOXWjaUJSFP336ChxlStBAO75wcb/MhLN+Uxp+ZENYR7uU8afw3b6v01f8wtohTOTbN Po6iO8dG+RDXHLf5KgKOk3PlrWpdX9TK0UchsLiMg0WDIKZs/M9wbP/LUz3LIiluo9oc vERo6Kt4Qdaf/4rbMlBAtVwNJ9fljKpKr7M4K/Ui++81cWw0BmGb0nQvHQvmkwahky5m i/lC5IV+qKnzeArkOCIy/5uDMe1/COpZjotQUHIHzFs9EJow+qjDE1ZAN+UaX/ueuGNf E/5no9oWnjHXEWpFBzNmjGdq9sblYYzd44+VLJj3b/fj++5hhHEG8QmatxlPuevf5wqB MQ== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 11/14] vfio/common: Add device dirty page bitmap sync Date: Tue, 7 Mar 2023 02:02:55 +0000 Message-Id: <20230307020258.58215-12-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=898 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070017 X-Proofpoint-GUID: 4Dss-UYPmermDLpEEkUd7-7DHpSA1_iD X-Proofpoint-ORIG-GUID: 4Dss-UYPmermDLpEEkUd7-7DHpSA1_iD Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154645851100003 Content-Type: text/plain; charset="utf-8" Add device dirty page bitmap sync functionality. This uses the device DMA logging uAPI to sync dirty page bitmap from the device. Device dirty page bitmap sync is used only if all devices within a container support device dirty page tracking. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 88 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 79 insertions(+), 9 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 136665ca2c4e..75b4902bbcc9 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -339,6 +339,9 @@ static int vfio_bitmap_alloc(VFIOBitmap *vbmap, hwaddr = size) return 0; } =20 +static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, + uint64_t size, ram_addr_t ram_addr); + bool vfio_mig_active(void) { VFIOGroup *group; @@ -562,10 +565,16 @@ static int vfio_dma_unmap(VFIOContainer *container, .iova =3D iova, .size =3D size, }; + bool need_dirty_sync =3D false; + int ret; + + if (iotlb && vfio_devices_all_running_and_mig_active(container)) { + if (!vfio_devices_all_device_dirty_tracking(container) && + container->dirty_pages_supported) { + return vfio_dma_unmap_bitmap(container, iova, size, iotlb); + } =20 - if (iotlb && container->dirty_pages_supported && - vfio_devices_all_running_and_mig_active(container)) { - return vfio_dma_unmap_bitmap(container, iova, size, iotlb); + need_dirty_sync =3D true; } =20 while (ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, &unmap)) { @@ -591,10 +600,12 @@ static int vfio_dma_unmap(VFIOContainer *container, return -errno; } =20 - if (iotlb && vfio_devices_all_running_and_mig_active(container)) { - cpu_physical_memory_set_dirty_range(iotlb->translated_addr, size, - tcg_enabled() ? DIRTY_CLIENTS_= ALL : - DIRTY_CLIENTS_NOCODE); + if (need_dirty_sync) { + ret =3D vfio_get_dirty_bitmap(container, iova, size, + iotlb->translated_addr); + if (ret) { + return ret; + } } =20 return 0; @@ -1597,6 +1608,58 @@ static void vfio_listener_log_global_stop(MemoryList= ener *listener) } } =20 +static int vfio_device_dma_logging_report(VFIODevice *vbasedev, hwaddr iov= a, + hwaddr size, void *bitmap) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_dma_logging_repo= rt), + sizeof(__u64))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + struct vfio_device_feature_dma_logging_report *report =3D + (struct vfio_device_feature_dma_logging_report *)feature->data; + + report->iova =3D iova; + report->length =3D size; + report->page_size =3D qemu_real_host_page_size(); + report->bitmap =3D (__u64)(uintptr_t)bitmap; + + feature->argsz =3D sizeof(buf); + feature->flags =3D VFIO_DEVICE_FEATURE_GET | + VFIO_DEVICE_FEATURE_DMA_LOGGING_REPORT; + + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + return -errno; + } + + return 0; +} + +static int vfio_devices_query_dirty_bitmap(VFIOContainer *container, + VFIOBitmap *vbmap, hwaddr iova, + hwaddr size) +{ + VFIODevice *vbasedev; + VFIOGroup *group; + int ret; + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + ret =3D vfio_device_dma_logging_report(vbasedev, iova, size, + vbmap->bitmap); + if (ret) { + error_report("%s: Failed to get DMA logging report, iova: " + "0x%" HWADDR_PRIx ", size: 0x%" HWADDR_PRIx + ", err: %d (%s)", + vbasedev->name, iova, size, ret, strerror(-re= t)); + + return ret; + } + } + } + + return 0; +} + static int vfio_query_dirty_bitmap(VFIOContainer *container, VFIOBitmap *v= bmap, hwaddr iova, hwaddr size) { @@ -1637,10 +1700,12 @@ static int vfio_query_dirty_bitmap(VFIOContainer *c= ontainer, VFIOBitmap *vbmap, static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, uint64_t size, ram_addr_t ram_addr) { + bool all_device_dirty_tracking =3D + vfio_devices_all_device_dirty_tracking(container); VFIOBitmap vbmap; int ret; =20 - if (!container->dirty_pages_supported) { + if (!container->dirty_pages_supported && !all_device_dirty_tracking) { cpu_physical_memory_set_dirty_range(ram_addr, size, tcg_enabled() ? DIRTY_CLIENTS_= ALL : DIRTY_CLIENTS_NOCODE); @@ -1652,7 +1717,12 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, return ret; } =20 - ret =3D vfio_query_dirty_bitmap(container, &vbmap, iova, size); + if (all_device_dirty_tracking) { + ret =3D vfio_devices_query_dirty_bitmap(container, &vbmap, iova, s= ize); + } else { + ret =3D vfio_query_dirty_bitmap(container, &vbmap, iova, size); + } + if (ret) { goto out; } --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154693; cv=none; d=zohomail.com; s=zohoarc; b=IESm0jr0CgygDZ3sKNGQXDR0abFty4qSR0VqLhu9V07pLF1N81Esp26Wko/F1TpxnPxQEn4TwsF6U6sNDrig/HtY9BPvp0UVa/4EIqt3aGR5pGtXy0L5Ahxrr3QhtlIurJGMbjIz48GFrdKWUTs+0FD6nxc3vTVViJhP2q6dtpw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154693; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3TlfzMmU33d/mrhyVXNtB875Mp25T+mC21R+9zGUXoU=; b=g7qy6tA1kxq4QoS62ZrfdEXRjXNBfV0otITIhvPndb9A5GMhxLBXGy8VAdzeD5OJwT+7bZ4xbHnv08OSdiaNb5GnjwtUDhE/g3YGerZVSUq/rVFefxmKHUaHhYphaGHmCwkUsqtqzrbbMfbPlWJ5egg89LdmE7POaxZyTJ9g/PI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154693392997.5730542637095; Mon, 6 Mar 2023 18:04:53 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMgH-00073W-UK; Mon, 06 Mar 2023 21:03:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMgG-00072U-2t for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:56 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMgE-0004xf-9i for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:55 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326Nxj4e027798; Tue, 7 Mar 2023 02:03:51 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p5nn90s1n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:50 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3271P09Q037278; Tue, 7 Mar 2023 02:03:49 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txduc43-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:49 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3272356M016763; Tue, 7 Mar 2023 02:03:49 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-13; Tue, 07 Mar 2023 02:03:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=3TlfzMmU33d/mrhyVXNtB875Mp25T+mC21R+9zGUXoU=; b=rDFPqGgEGIm0E1qjjKqhBNWYHXkej6ajOy1O/9uJUAEr7qlUyaTyq8EcXPDJQdbK/0rg 4tgvuPhbX3FCPIh4AosUdZTewieiQuhFO81SBgBLKZaIPOw7+nu/4HzbpVNNEaAqYQ+Y z4Uv7jwNIj0oppw6O+MlPoFg0CJpnKQSYV82Oj5K5TGvQlVfgOQk5cVW0CO2b/aAfKOw +9usG/nuxMlY3x4T5axU3KaTLxV4O/v6MHfrDWOHwfWzeahqYhV0WPwmOzJQfUsJTofa 4DzHaSej9+8ctei9tusslcEoE85vAkGzr3U6jTi21lBFOMIe0CHdVex7ETm0whcXqVry Gw== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 12/14] vfio/migration: Block migration with vIOMMU Date: Tue, 7 Mar 2023 02:02:56 +0000 Message-Id: <20230307020258.58215-13-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=703 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070017 X-Proofpoint-GUID: toorQ8hM9mOlmWC2h7143Pj-5oHVW3oN X-Proofpoint-ORIG-GUID: toorQ8hM9mOlmWC2h7143Pj-5oHVW3oN Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154694256100003 Content-Type: text/plain; charset="utf-8" Migrating with vIOMMU will require either tracking maximum IOMMU supported address space (e.g. 39/48 address width on Intel) or range-track current mappings and dirty track the new ones post starting dirty tracking. This will be done as a separate series, so add a live migration blocker until that is fixed. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 46 +++++++++++++++++++++++++++++++++++ hw/vfio/migration.c | 5 ++++ hw/vfio/pci.c | 1 + include/hw/vfio/vfio-common.h | 2 ++ 4 files changed, 54 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 75b4902bbcc9..7278baa82f7d 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -362,6 +362,7 @@ bool vfio_mig_active(void) } =20 static Error *multiple_devices_migration_blocker; +static Error *giommu_migration_blocker; =20 static unsigned int vfio_migratable_device_num(void) { @@ -413,6 +414,51 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker =3D NULL; } =20 +static bool vfio_viommu_preset(void) +{ + VFIOAddressSpace *space; + + QLIST_FOREACH(space, &vfio_address_spaces, list) { + if (space->as !=3D &address_space_memory) { + return true; + } + } + + return false; +} + +int vfio_block_giommu_migration(Error **errp) +{ + int ret; + + if (giommu_migration_blocker || + !vfio_viommu_preset()) { + return 0; + } + + error_setg(&giommu_migration_blocker, + "Migration is currently not supported with vIOMMU enabled"); + ret =3D migrate_add_blocker(giommu_migration_blocker, errp); + if (ret < 0) { + error_free(giommu_migration_blocker); + giommu_migration_blocker =3D NULL; + } + + return ret; +} + +void vfio_unblock_giommu_migration(void) +{ + if (!giommu_migration_blocker || + vfio_viommu_preset()) { + return; + } + + migrate_del_blocker(giommu_migration_blocker); + error_free(giommu_migration_blocker); + giommu_migration_blocker =3D NULL; +} + static void vfio_set_migration_error(int err) { MigrationState *ms =3D migrate_get_current(); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index a2c3d9bade7f..776fd2d7cdf3 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -634,6 +634,11 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error *= *errp) return ret; } =20 + ret =3D vfio_block_giommu_migration(errp); + if (ret) { + return ret; + } + trace_vfio_migration_probe(vbasedev->name); return 0; =20 diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 939dcc3d4a9e..30a271eab38c 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3185,6 +3185,7 @@ static void vfio_instance_finalize(Object *obj) */ vfio_put_device(vdev); vfio_put_group(group); + vfio_unblock_giommu_migration(); } =20 static void vfio_exitfn(PCIDevice *pdev) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 7817ca7d8706..63f93ab54811 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -235,6 +235,8 @@ extern VFIOGroupList vfio_group_list; bool vfio_mig_active(void); int vfio_block_multiple_devices_migration(Error **errp); void vfio_unblock_multiple_devices_migration(void); +int vfio_block_giommu_migration(Error **errp); +void vfio_unblock_giommu_migration(void); int64_t vfio_mig_bytes_transferred(void); =20 #ifdef CONFIG_LINUX --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154740; cv=none; d=zohomail.com; s=zohoarc; b=dHp+ozhtqfWUIsBFRe15ExmE+ReTuGa0lvRB0xV2Pb1U4VHtOL5dT7Vt5/pfWtNy/k0wUFu2gisBOmY2XxyV7IU/2fDewe0hf43o5QKcjm6Eh9geoLspGKykgMBuMWNJxp8go0nPNBZ4s6aumVDezmcfcwERnpZK08HJ4B+R6PA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154740; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xuofWDdTaohhml0uhXqEE2pCYISUUN5doa0iJ2TR+A4=; b=GSBqPmLpk8RR9GuMgUOiX1Zvio0uOJCFxgPO8odSYVLdTbUobEgSAK6OtEIKES5K/qR6gY3/+pVO+Tc55jUVGNswdSsEnM2KeECcrfbte7JhA/wEyGsrTwPCWsEF/ycuQAvyYh6/bQybubKRLAE11ncC1bAu5mvJvcT7GSBSljg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154740939669.5977906084961; Mon, 6 Mar 2023 18:05:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMgI-000744-K4; Mon, 06 Mar 2023 21:03:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMgH-00073M-9s for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:57 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMgF-0004zL-Lp for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:03:57 -0500 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NxYsO030050; Tue, 7 Mar 2023 02:03:54 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p418xvfev-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:53 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 32721G9J037277; Tue, 7 Mar 2023 02:03:52 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txduc5s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:52 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3272356O016763; Tue, 7 Mar 2023 02:03:51 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-14; Tue, 07 Mar 2023 02:03:51 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=xuofWDdTaohhml0uhXqEE2pCYISUUN5doa0iJ2TR+A4=; b=3bPA832MBknOsHxs9b5W6FcLHbPOx7+wDJrCjVNCleotkxcMyxyDG0jL8XsCDtHN/sao FbPi3Da62A6cIKW92SuOLl+Y5PakmSkviVoEQlTWNdTvAvs70z77a/YVkgamWvqNKpVS l0ml21tSUDtP//SYVcCROB7GXyuhgJE15EDD0fxmLoJoHlNFyv5CJgnpt6DJPuo2euOQ 7tozlK1cxUsko/oo50IQHivjHTwkDu/AfjPrkzx9N4cp0t6KOcmWlW5jlAt9Y62UZK3a lM8GjL9BirkbbD5UNT1dAMYq4DOaa9eV6Xws+87/fwoNmf2ww6MMeM+GfZ6SRKEdO3xM 7g== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 13/14] vfio/migration: Query device dirty page tracking support Date: Tue, 7 Mar 2023 02:02:57 +0000 Message-Id: <20230307020258.58215-14-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070017 X-Proofpoint-GUID: U3Nld0kBtYK4Fvs_4zoe6AYttJ5CSIY3 X-Proofpoint-ORIG-GUID: U3Nld0kBtYK4Fvs_4zoe6AYttJ5CSIY3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154742382100011 Now that everything has been set up for device dirty page tracking, query the device for device dirty page tracking support. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/migration.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 776fd2d7cdf3..127a44ccaf19 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -555,6 +555,19 @@ static int vfio_migration_query_flags(VFIODevice *vbas= edev, uint64_t *mig_flags) return 0; } =20 +static bool vfio_dma_logging_supported(VFIODevice *vbasedev) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + + feature->argsz =3D sizeof(buf); + feature->flags =3D + VFIO_DEVICE_FEATURE_PROBE | VFIO_DEVICE_FEATURE_DMA_LOGGING_START; + + return !ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); +} + static int vfio_migration_init(VFIODevice *vbasedev) { int ret; @@ -589,6 +602,8 @@ static int vfio_migration_init(VFIODevice *vbasedev) migration->device_state =3D VFIO_DEVICE_STATE_RUNNING; migration->data_fd =3D -1; =20 + vbasedev->dirty_pages_supported =3D vfio_dma_logging_supported(vbasede= v); + oid =3D vmstate_if_get_id(VMSTATE_IF(DEVICE(obj))); if (oid) { path =3D g_strdup_printf("%s/vfio", oid); --=20 2.17.2 From nobody Wed May 1 23:17:20 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1678154683; cv=none; d=zohomail.com; s=zohoarc; b=ZwFLqUfg92ZYRxCffWXEe8J3dSM4rvmo3LRDe+pdFh0ZjiWi4wuiOkvNh+QcIQ1aefgl8X3xye55g8sra7p8wXDKNXj5s2ep9qRlDl4LMf5obvKPX110dG+Qe0WUGeYFHY7ZqEqKHaSjIYp4lIdXlDLbKaGgzmq2lX+Z0ZoWmpo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678154683; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Xel4ecbEPawZ38P8TKB/h9sUQWvBI6R7FLZBmhXQ7Bw=; b=bli63poEvsbo+KcfNneVvYK0Il+P/UCAWJ2ZuhcrTwHskmEBC8iFcqsmi8+ddoqdn+pJd7tcD4RDLlCWkFZr6cyGIxvWY5y8syLrZlrWbAZ+A2Zxig02+v4uHqrqCZhg0yONUibQwxKM1e4nVk6pRteXKUbQ+EUUHrSKCQAQH54= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1678154683344867.9216060902697; Mon, 6 Mar 2023 18:04:43 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pZMgR-0007Eb-JY; Mon, 06 Mar 2023 21:04:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMgK-00074W-Pv for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:04:00 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZMgI-0004zj-Ub for qemu-devel@nongnu.org; Mon, 06 Mar 2023 21:04:00 -0500 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 326NxnVi030307; Tue, 7 Mar 2023 02:03:57 GMT Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p418xvfex-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:57 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3271eQQu037536; Tue, 7 Mar 2023 02:03:56 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3p4txduc7q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 07 Mar 2023 02:03:56 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3272356Q016763; Tue, 7 Mar 2023 02:03:55 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-187-101.vpn.oracle.com [10.175.187.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3p4txdubf4-15; Tue, 07 Mar 2023 02:03:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=Xel4ecbEPawZ38P8TKB/h9sUQWvBI6R7FLZBmhXQ7Bw=; b=hPRgIJaLIwcfnDBw8ueurXO6SILpIY5pBbLPFAm3PBc6K+jdN3SE89eveuqGreTnU0VJ A2JEYBL6oMHGKDCmCfiYvrBgVaq/R3ff7e9i33WVd2L8gX57JQpwTkLjc6Bn6dEDdV6R SdQwdUSZ/Hr+SZXjd/5dF2pqkGQaW8E/PwTyj8FZaGQxxfbQgQLOeA5gCSla+7B4WOyY w96rUBM6ZwwqaRWNcIgWe//rszFvowiPiouiZKI7FouM6Qa8pbiyYmI6ME08fnEHR39d 1xkxbcXoN2kewjaevBf/PmPczriLCKZkUKd99v2BDbLoJIybfkgl4ZxkfUHYF503Q7h1 vw== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v4 14/14] docs/devel: Document VFIO device dirty page tracking Date: Tue, 7 Mar 2023 02:02:58 +0000 Message-Id: <20230307020258.58215-15-joao.m.martins@oracle.com> In-Reply-To: <20230307020258.58215-1-joao.m.martins@oracle.com> References: <20230307020258.58215-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-06_14,2023-03-06_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 mlxlogscore=990 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303070017 X-Proofpoint-GUID: tT8tCkpw35mj0MITeMgMeQZ7X9aga_YS X-Proofpoint-ORIG-GUID: tT8tCkpw35mj0MITeMgMeQZ7X9aga_YS Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1678154684188100003 From: Avihai Horon Adjust the VFIO dirty page tracking documentation and add a section to describe device dirty page tracking. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- docs/devel/vfio-migration.rst | 46 +++++++++++++++++++++++------------ 1 file changed, 31 insertions(+), 15 deletions(-) diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst index c214c73e2818..1b68ccf11529 100644 --- a/docs/devel/vfio-migration.rst +++ b/docs/devel/vfio-migration.rst @@ -59,22 +59,37 @@ System memory dirty pages tracking ---------------------------------- =20 A ``log_global_start`` and ``log_global_stop`` memory listener callback in= forms -the VFIO IOMMU module to start and stop dirty page tracking. A ``log_sync`` -memory listener callback marks those system memory pages as dirty which are -used for DMA by the VFIO device. The dirty pages bitmap is queried per -container. All pages pinned by the vendor driver through external APIs hav= e to -be marked as dirty during migration. When there are CPU writes, CPU dirty = page -tracking can identify dirtied pages, but any page pinned by the vendor dri= ver -can also be written by the device. There is currently no device or IOMMU -support for dirty page tracking in hardware. +the VFIO dirty tracking module to start and stop dirty page tracking. A +``log_sync`` memory listener callback queries the dirty page bitmap from t= he +dirty tracking module and marks system memory pages which were DMA-ed by t= he +VFIO device as dirty. The dirty page bitmap is queried per container. + +Currently there are two ways dirty page tracking can be done: +(1) Device dirty tracking: +In this method the device is responsible to log and report its DMAs. This +method can be used only if the device is capable of tracking its DMAs. +Discovering device capability, starting and stopping dirty tracking, and +syncing the dirty bitmaps from the device are done using the DMA logging u= API. +More info about the uAPI can be found in the comments of the +``vfio_device_feature_dma_logging_control`` and +``vfio_device_feature_dma_logging_report`` structures in the header file +linux-headers/linux/vfio.h. + +(2) VFIO IOMMU module: +In this method dirty tracking is done by IOMMU. However, there is currentl= y no +IOMMU support for dirty page tracking. For this reason, all pages are +perpetually marked dirty, unless the device driver pins pages through exte= rnal +APIs in which case only those pinned pages are perpetually marked dirty. + +If the above two methods are not supported, all pages are perpetually mark= ed +dirty by QEMU. =20 By default, dirty pages are tracked during pre-copy as well as stop-and-co= py -phase. So, a page pinned by the vendor driver will be copied to the destin= ation -in both phases. Copying dirty pages in pre-copy phase helps QEMU to predic= t if -it can achieve its downtime tolerances. If QEMU during pre-copy phase keeps -finding dirty pages continuously, then it understands that even in stop-an= d-copy -phase, it is likely to find dirty pages and can predict the downtime -accordingly. +phase. So, a page marked as dirty will be copied to the destination in both +phases. Copying dirty pages in pre-copy phase helps QEMU to predict if it = can +achieve its downtime tolerances. If QEMU during pre-copy phase keeps findi= ng +dirty pages continuously, then it understands that even in stop-and-copy p= hase, +it is likely to find dirty pages and can predict the downtime accordingly. =20 QEMU also provides a per device opt-out option ``pre-copy-dirty-page-track= ing`` which disables querying the dirty bitmap during pre-copy phase. If it is s= et to @@ -89,7 +104,8 @@ phase of migration. In that case, the unmap ioctl return= s any dirty pages in that range and QEMU reports corresponding guest physical pages dirty. Duri= ng stop-and-copy phase, an IOMMU notifier is used to get a callback for mapped pages and then dirty pages bitmap is fetched from VFIO IOMMU modules for t= hose -mapped ranges. +mapped ranges. If device dirty tracking is enabled with vIOMMU, live migra= tion +will be blocked. =20 Flow of state changes during Live migration =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --=20 2.17.2