From nobody Mon May 6 20:00:40 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677494932; cv=none; d=zohomail.com; s=zohoarc; b=WoNpMfU30Dg5KV/Ify1z4LMEVEWgYbmqwr+dmy9JB8WwG7N9WGayB3zPDTWePDWH+upU32EeM3+rc9oWwOy7iIogXRmFkd//UmmV63D6VAJ0XqGlr0ohT0oSYNsZoNPFj888ZuFRm5uvOue83oAyisk6TwpLO+BSsnnvlsM6lsI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677494932; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eyWqpE9+1CkDMMOO7j3zRjCLoSOlzdVHaovFbYf9L2E=; b=F2AmTtabE1dsVtYEpKobkRayYltt2e4vvDKTGNwLNpWkRy8Fw58aLC/vcaMfj2ILRuUaUk38QQjgbifSDLH8tqIOyJpzNWwxpNNQJA9T8cDT75YAxle1Tx74wWa4RU+MDAbI6Ti/baBjwiI377cpCeCkPISPjMVc//SiW9XIOnU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677494932252637.497306515882; Mon, 27 Feb 2023 02:48:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pWb2o-0001if-2F; Mon, 27 Feb 2023 05:47:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pWb2d-0001dl-Ok for qemu-devel@nongnu.org; Mon, 27 Feb 2023 05:47:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pWb2c-0005Oy-2F for qemu-devel@nongnu.org; Mon, 27 Feb 2023 05:47:35 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-81-7UC7FokRONq6XDblWxcO2w-1; Mon, 27 Feb 2023 05:47:29 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2AC901C05AB6; Mon, 27 Feb 2023 10:47:29 +0000 (UTC) Received: from localhost (unknown [10.45.228.208]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D67BD140EBF4; Mon, 27 Feb 2023 10:47:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677494853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eyWqpE9+1CkDMMOO7j3zRjCLoSOlzdVHaovFbYf9L2E=; b=YN12QvnKsrtrgeGINPAgdnyANJSN3n1BM93cdCMaP2yylHI4MjuDpjQ7078jsN+sS6BPOy ha9lMfmjPdMlto/vr1nA49h50en9XOZcG7LJE4orI/dJj3oh770UVxXHAuKrYCBO3IBYyX uJFNQuZmyEC6LCCSZiWcmvCvkKNZk7Y= X-MC-Unique: 7UC7FokRONq6XDblWxcO2w-1 From: Hanna Czenczek To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Hanna Czenczek , Kevin Wolf Subject: [PATCH 1/2] block/fuse: Let PUNCH_HOLE write zeroes Date: Mon, 27 Feb 2023 11:47:24 +0100 Message-Id: <20230227104725.33511-2-hreitz@redhat.com> In-Reply-To: <20230227104725.33511-1-hreitz@redhat.com> References: <20230227104725.33511-1-hreitz@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677494933211100001 Content-Type: text/plain; charset="utf-8" fallocate(2) says about PUNCH_HOLE: "After a successful call, subsequent reads from this range will return zeros." As it is, PUNCH_HOLE is implemented as a call to blk_pdiscard(), which does not guarantee this. We must call blk_pwrite_zeroes() instead. The difference to ZERO_RANGE is that we pass the `BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK` flags to the call -- the storage is supposed to be unmapped, and a slow fallback by actually writing zeroes as data is not allowed. Closes: https://gitlab.com/qemu-project/qemu/-/issues/1507 Signed-off-by: Hanna Czenczek --- block/export/fuse.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/block/export/fuse.c b/block/export/fuse.c index e5fc4af165..06fa41079e 100644 --- a/block/export/fuse.c +++ b/block/export/fuse.c @@ -673,7 +673,16 @@ static void fuse_fallocate(fuse_req_t req, fuse_ino_t = inode, int mode, do { int size =3D MIN(length, BDRV_REQUEST_MAX_BYTES); =20 - ret =3D blk_pdiscard(exp->common.blk, offset, size); + ret =3D blk_pwrite_zeroes(exp->common.blk, offset, size, + BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLB= ACK); + if (ret =3D=3D -ENOTSUP) { + /* + * fallocate() specifies to return EOPNOTSUPP for unsuppor= ted + * operations + */ + ret =3D -EOPNOTSUPP; + } + offset +=3D size; length -=3D size; } while (ret =3D=3D 0 && length > 0); --=20 2.39.1 From nobody Mon May 6 20:00:40 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677494932; cv=none; d=zohomail.com; s=zohoarc; b=kKtOwLXm/GlvkCq3a3llZTRdBm1wTRo6gsuAmP9HASl8Fs4yiEBa2RUNfhKeOl9EdNuuV11CtvgF1u1sdpiduAm2IJAt7ZAx1zC0rw7C4dPdynzkMSvaYxIDvUOZj62wO5TrSGqavKWsDb6i+Fq4NcAKj6cqYY6FnVOvp/S7B+4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677494932; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hr12Jwjpz7bj0h98Ck0zTrqbBUXhzdLhMSMZsxkDv+c=; b=TY7f8CXX+OnffhFHtiKn1l71/rnhXcQ0Kg2TWJ07JCFDBtGHjdfAk6LYVwDpT2O+wMdejpw+9m5yZaOneAaypGMJeJ5NGW8IMDBaBuHYAvQXCi1hpDWDHttkUVGg08Mw9m0lAbQvvHDWC9/FeOQoL1nWdZOsRHOYmb60DOpzFoA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677494932293879.6545267465473; Mon, 27 Feb 2023 02:48:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pWb2o-0001j8-95; Mon, 27 Feb 2023 05:47:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pWb2f-0001do-6S for qemu-devel@nongnu.org; Mon, 27 Feb 2023 05:47:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pWb2d-0005PP-Gj for qemu-devel@nongnu.org; Mon, 27 Feb 2023 05:47:36 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-640-m0hfHNHpNIWVE4aysN5BgA-1; Mon, 27 Feb 2023 05:47:31 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DD7E52A5955E; Mon, 27 Feb 2023 10:47:30 +0000 (UTC) Received: from localhost (unknown [10.45.228.208]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 94D0040C6EC4; Mon, 27 Feb 2023 10:47:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677494855; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hr12Jwjpz7bj0h98Ck0zTrqbBUXhzdLhMSMZsxkDv+c=; b=RlXwO8Qbp0CedroSQevDg8exSp117eNMU9O+ScZXKQIQLAR/ceTChsasbp4dIsU5J4J3OL uOuzQpXPc075TQ5gRNsepObTAeKAWTIHjyx6PC1ZbbtRNvWSMZiyVZXDaAm7k8TY4gLT0S pSDW5chud0+jxom81CccGBBF6ucrJNI= X-MC-Unique: m0hfHNHpNIWVE4aysN5BgA-1 From: Hanna Czenczek To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Hanna Czenczek , Kevin Wolf Subject: [PATCH 2/2] iotests/308: Add test for 'write -zu' Date: Mon, 27 Feb 2023 11:47:25 +0100 Message-Id: <20230227104725.33511-3-hreitz@redhat.com> In-Reply-To: <20230227104725.33511-1-hreitz@redhat.com> References: <20230227104725.33511-1-hreitz@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677494933223100002 Content-Type: text/plain; charset="utf-8" Try writing zeroes to a FUSE export while allowing the area to be unmapped; block/file-posix.c generally implements writing zeroes with BDRV_REQ_MAY_UNMAP ('write -zu') by calling fallocate(PUNCH_HOLE). This used to lead to a blk_pdiscard() in the FUSE export, which may or may not lead to the area being zeroed. HEAD^ fixed this to use blk_pwrite_zeroes() instead (again with BDRV_REQ_MAY_UNMAP), so verify that running `qemu-io 'write -zu'` on a FUSE exports always results in zeroes being written. Signed-off-by: Hanna Czenczek --- tests/qemu-iotests/308 | 43 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/308.out | 35 +++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 index 09275e9a10..de12b2b1b9 100755 --- a/tests/qemu-iotests/308 +++ b/tests/qemu-iotests/308 @@ -370,6 +370,49 @@ echo echo '=3D=3D=3D Compare copy with original =3D=3D=3D' =20 $QEMU_IMG compare -f raw -F $IMGFMT "$COPIED_IMG" "$TEST_IMG" +_cleanup_test_img + +echo +echo '=3D=3D=3D Writing zeroes while unmapping =3D=3D=3D' +# Regression test for https://gitlab.com/qemu-project/qemu/-/issues/1507 +_make_test_img 64M +$QEMU_IO -c 'write -s /dev/urandom 0 64M' "$TEST_IMG" | _filter_qemu_io + +_launch_qemu +_send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'qmp_capabilities'}" \ + 'return' + +_send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'blockdev-add', + 'arguments': { + 'driver': '$IMGFMT', + 'node-name': 'node-format', + 'file': { + 'driver': 'file', + 'filename': '$TEST_IMG' + } + } }" \ + 'return' + +fuse_export_add 'export' "'mountpoint': '$EXT_MP', 'writable': true" + +# Try writing zeroes by unmapping +$QEMU_IO -f raw -c 'write -zu 0 64M' "$EXT_MP" | _filter_qemu_io + +# Check the result +$QEMU_IO -f raw -c 'read -P 0 0 64M' "$EXT_MP" | _filter_qemu_io + +_send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'quit'}" \ + 'return' + +wait=3Dyes _cleanup_qemu + +# Check the original image +$QEMU_IO -c 'read -P 0 0 64M' "$TEST_IMG" | _filter_qemu_io + +_cleanup_test_img =20 # success, all done echo "*** done" diff --git a/tests/qemu-iotests/308.out b/tests/qemu-iotests/308.out index e4467a10cf..d5767133b1 100644 --- a/tests/qemu-iotests/308.out +++ b/tests/qemu-iotests/308.out @@ -171,4 +171,39 @@ OK: Post-truncate image size is as expected =20 =3D=3D=3D Compare copy with original =3D=3D=3D Images are identical. + +=3D=3D=3D Writing zeroes while unmapping =3D=3D=3D +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +wrote 67108864/67108864 bytes at offset 0 +64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +{'execute': 'qmp_capabilities'} +{"return": {}} +{'execute': 'blockdev-add', + 'arguments': { + 'driver': 'IMGFMT', + 'node-name': 'node-format', + 'file': { + 'driver': 'file', + 'filename': 'TEST_DIR/t.IMGFMT' + } + } } +{"return": {}} +{'execute': 'block-export-add', + 'arguments': { + 'type': 'fuse', + 'id': 'export', + 'node-name': 'node-format', + 'mountpoint': 'TEST_DIR/t.IMGFMT.fuse', 'writable': true + } } +{"return": {}} +wrote 67108864/67108864 bytes at offset 0 +64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 67108864/67108864 bytes at offset 0 +64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +{'execute': 'quit'} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_EXPORT_DELETED", "data": {"id": "export"}} +read 67108864/67108864 bytes at offset 0 +64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done --=20 2.39.1