The following changes since commit 191710c221f65b1542f6ea7fa4d30dde6e134fd7: Merge tag 'pull-request-2023-12-20' of https://gitlab.com/thuth/qemu into staging (2023-12-20 09:40:16 -0500) are available in the Git repository at: https://repo.or.cz/qemu/kevin.git tags/for-upstream for you to fetch changes up to b6948ab01df068bef591868c22d1f873d2d05cde: virtio-blk: add iothread-vq-mapping parameter (2023-12-21 22:49:28 +0100) ---------------------------------------------------------------- Block layer patches - virtio-blk: Multiqueue support (configurable iothread per queue) - Made NBD export and hw/scsi thread-safe without AioContext lock - Fix crash when loading snapshot on inactive node ---------------------------------------------------------------- Kevin Wolf (3): block: Fix crash when loading snapshot on inactive node vl: Improve error message for conflicting -incoming and -loadvm iotests: Basic tests for internal snapshots Stefan Hajnoczi (30): nbd/server: avoid per-NBDRequest nbd_client_get/put() nbd/server: only traverse NBDExport->clients from main loop thread nbd/server: introduce NBDClient->lock to protect fields block/file-posix: set up Linux AIO and io_uring in the current thread virtio-blk: add lock to protect s->rq virtio-blk: don't lock AioContext in the completion code path virtio-blk: don't lock AioContext in the submission code path scsi: only access SCSIDevice->requests from one thread virtio-scsi: don't lock AioContext around virtio_queue_aio_attach_host_notifier() scsi: don't lock AioContext in I/O code path dma-helpers: don't lock AioContext in dma_blk_cb() virtio-scsi: replace AioContext lock with tmf_bh_lock scsi: assert that callbacks run in the correct AioContext tests: remove aio_context_acquire() tests aio: make aio_context_acquire()/aio_context_release() a no-op graph-lock: remove AioContext locking block: remove AioContext locking block: remove bdrv_co_lock() scsi: remove AioContext locking aio-wait: draw equivalence between AIO_WAIT_WHILE() and AIO_WAIT_WHILE_UNLOCKED() aio: remove aio_context_acquire()/aio_context_release() API docs: remove AioContext lock from IOThread docs scsi: remove outdated AioContext lock comment job: remove outdated AioContext locking comments block: remove outdated AioContext locking comments block-coroutine-wrapper: use qemu_get_current_aio_context() string-output-visitor: show structs as "<omitted>" qdev-properties: alias all object class properties qdev: add IOThreadVirtQueueMappingList property type virtio-blk: add iothread-vq-mapping parameter qapi/virtio.json | 29 ++ docs/devel/multiple-iothreads.txt | 47 +-- hw/block/dataplane/virtio-blk.h | 3 + include/block/aio-wait.h | 16 +- include/block/aio.h | 17 - include/block/block-common.h | 3 - include/block/block-global-state.h | 23 +- include/block/block-io.h | 12 +- include/block/block_int-common.h | 2 - include/block/graph-lock.h | 21 +- include/block/snapshot.h | 2 - include/hw/qdev-properties-system.h | 5 + include/hw/qdev-properties.h | 4 +- include/hw/scsi/scsi.h | 7 +- include/hw/virtio/virtio-blk.h | 5 +- include/hw/virtio/virtio-scsi.h | 17 +- include/qapi/string-output-visitor.h | 6 +- include/qemu/job.h | 20 -- block.c | 363 +++------------------ block/backup.c | 4 +- block/blklogwrites.c | 8 +- block/blkverify.c | 4 +- block/block-backend.c | 33 +- block/commit.c | 16 +- block/copy-before-write.c | 22 +- block/export/export.c | 22 +- block/export/vhost-user-blk-server.c | 4 - block/file-posix.c | 103 +++--- block/graph-lock.c | 44 +-- block/io.c | 45 +-- block/mirror.c | 41 +-- block/monitor/bitmap-qmp-cmds.c | 20 +- block/monitor/block-hmp-cmds.c | 29 -- block/qapi-sysemu.c | 27 +- block/qapi.c | 18 +- block/qcow2.c | 4 +- block/quorum.c | 8 +- block/raw-format.c | 5 - block/replication.c | 72 +--- block/snapshot.c | 30 +- block/stream.c | 12 +- block/vmdk.c | 20 +- block/write-threshold.c | 6 - blockdev.c | 320 ++++-------------- blockjob.c | 30 +- hw/block/dataplane/virtio-blk.c | 165 +++++++--- hw/block/dataplane/xen-block.c | 17 +- hw/block/virtio-blk.c | 209 +++++++----- hw/core/qdev-properties-system.c | 55 +++- hw/core/qdev-properties.c | 18 +- hw/scsi/scsi-bus.c | 183 +++++++---- hw/scsi/scsi-disk.c | 67 +--- hw/scsi/scsi-generic.c | 20 +- hw/scsi/virtio-scsi-dataplane.c | 8 +- hw/scsi/virtio-scsi.c | 80 ++--- job.c | 16 - migration/block.c | 34 +- migration/migration-hmp-cmds.c | 3 - migration/savevm.c | 22 -- nbd/server.c | 208 +++++++++--- net/colo-compare.c | 2 - qapi/string-output-visitor.c | 16 + qemu-img.c | 4 - qemu-io.c | 10 +- qemu-nbd.c | 2 - replay/replay-debugging.c | 4 - system/dma-helpers.c | 10 +- system/vl.c | 4 + tests/unit/test-aio.c | 67 +--- tests/unit/test-bdrv-drain.c | 91 ++---- tests/unit/test-bdrv-graph-mod.c | 26 +- tests/unit/test-block-iothread.c | 31 -- tests/unit/test-blockjob.c | 137 -------- tests/unit/test-replication.c | 11 - util/async.c | 14 - util/vhost-user-server.c | 3 - scripts/block-coroutine-wrapper.py | 13 +- tests/qemu-iotests/202 | 2 +- tests/qemu-iotests/203 | 3 +- tests/qemu-iotests/tests/qcow2-internal-snapshots | 170 ++++++++++ .../tests/qcow2-internal-snapshots.out | 107 ++++++ tests/tsan/suppressions.tsan | 1 - 82 files changed, 1341 insertions(+), 2041 deletions(-) create mode 100755 tests/qemu-iotests/tests/qcow2-internal-snapshots create mode 100644 tests/qemu-iotests/tests/qcow2-internal-snapshots.out The following changes since commit 13356edb87506c148b163b8c7eb0695647d00c2a: Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging (2023-01-24 09:45:33 +0000) are available in the Git repository at: https://repo.or.cz/qemu/kevin.git tags/for-upstream for you to fetch changes up to d570177b50c389f379f93183155a27d44856ab46: qemu-img: Change info key names for protocol nodes (2023-02-01 16:52:33 +0100) v4: - Fixed the 'qemu-img-close-errors' test case to run only on Linux and only with the file protocol, use qemu-io instead of truncate v3: - Make the compiler happier on BSD and CentOS Stream 8 v2: - Rebased to resolve merge conflicts in coroutine.h ---------------------------------------------------------------- Block layer patches - qemu-img info: Show protocol-level information - Move more functions to coroutines - Make coroutine annotations ready for static analysis - qemu-img: Fix exit code for errors closing the image - qcow2 bitmaps: Fix theoretical corruption in error path - pflash: Only load non-zero parts of backend image to save memory - Code cleanup and test case improvements ---------------------------------------------------------------- Alberto Faria (2): coroutine: annotate coroutine_fn for libclang block: Add no_coroutine_fn and coroutine_mixed_fn marker Emanuele Giuseppe Esposito (14): block-coroutine-wrapper: support void functions block: Convert bdrv_io_plug() to co_wrapper block: Convert bdrv_io_unplug() to co_wrapper block: Convert bdrv_is_inserted() to co_wrapper block: Rename refresh_total_sectors to bdrv_refresh_total_sectors block: Convert bdrv_refresh_total_sectors() to co_wrapper_mixed block-backend: use bdrv_getlength instead of blk_getlength block: use bdrv_co_refresh_total_sectors when possible block: Convert bdrv_get_allocated_file_size() to co_wrapper block: Convert bdrv_get_info() to co_wrapper_mixed block: Convert bdrv_eject() to co_wrapper block: Convert bdrv_lock_medium() to co_wrapper block: Convert bdrv_debug_event() to co_wrapper_mixed block: Rename bdrv_load/save_vmstate() to bdrv_co_load/save_vmstate() Hanna Reitz (12): block: Improve empty format-specific info dump block/file: Add file-specific image info block/vmdk: Change extent info type block: Split BlockNodeInfo off of ImageInfo qemu-img: Use BlockNodeInfo block/qapi: Let bdrv_query_image_info() recurse block/qapi: Introduce BlockGraphInfo block/qapi: Add indentation to bdrv_node_info_dump() iotests: Filter child node information iotests/106, 214, 308: Read only one size line qemu-img: Let info print block graph qemu-img: Change info key names for protocol nodes Kevin Wolf (4): qcow2: Fix theoretical corruption in store_bitmap() error path qemu-img commit: Report errors while closing the image qemu-img bitmap: Report errors while closing the image qemu-iotests: Test qemu-img bitmap/commit exit code on error Paolo Bonzini (2): qemu-io: do not reinvent the blk_pwrite_zeroes wheel block: remove bdrv_coroutine_enter Philippe Mathieu-Daudé (1): block/nbd: Add missing <qemu/bswap.h> include Thomas Huth (2): tests/qemu-iotests/312: Mark "quorum" as required driver tests/qemu-iotests/262: Check for availability of "blkverify" first Xiang Zheng (1): pflash: Only read non-zero parts of backend image qapi/block-core.json | 123 +++++++- include/block/block-common.h | 11 +- include/block/block-io.h | 41 ++- include/block/block_int-common.h | 26 +- include/block/block_int-io.h | 5 +- include/block/nbd.h | 1 + include/block/qapi.h | 14 +- include/qemu/osdep.h | 44 +++ include/sysemu/block-backend-io.h | 31 +- block.c | 88 +++--- block/blkdebug.c | 11 +- block/blkio.c | 15 +- block/blklogwrites.c | 6 +- block/blkreplay.c | 6 +- block/blkverify.c | 6 +- block/block-backend.c | 38 +-- block/commit.c | 4 +- block/copy-on-read.c | 18 +- block/crypto.c | 14 +- block/curl.c | 10 +- block/file-posix.c | 137 +++++---- block/file-win32.c | 18 +- block/filter-compress.c | 20 +- block/gluster.c | 23 +- block/io.c | 76 ++--- block/iscsi.c | 17 +- block/mirror.c | 6 +- block/monitor/block-hmp-cmds.c | 2 +- block/nbd.c | 8 +- block/nfs.c | 4 +- block/null.c | 13 +- block/nvme.c | 14 +- block/preallocate.c | 16 +- block/qapi.c | 317 ++++++++++++++++----- block/qcow.c | 5 +- block/qcow2-bitmap.c | 5 +- block/qcow2-refcount.c | 2 +- block/qcow2.c | 17 +- block/qed.c | 11 +- block/quorum.c | 8 +- block/raw-format.c | 25 +- block/rbd.c | 9 +- block/replication.c | 6 +- block/ssh.c | 4 +- block/throttle.c | 6 +- block/vdi.c | 7 +- block/vhdx.c | 5 +- block/vmdk.c | 22 +- block/vpc.c | 5 +- blockdev.c | 8 +- hw/block/block.c | 36 ++- hw/scsi/scsi-disk.c | 5 + qemu-img.c | 100 +++++-- qemu-io-cmds.c | 62 +--- tests/unit/test-block-iothread.c | 3 + scripts/block-coroutine-wrapper.py | 20 +- tests/qemu-iotests/iotests.py | 18 +- block/meson.build | 1 + tests/qemu-iotests/065 | 2 +- tests/qemu-iotests/106 | 4 +- tests/qemu-iotests/214 | 6 +- tests/qemu-iotests/262 | 3 +- tests/qemu-iotests/302.out | 5 + tests/qemu-iotests/308 | 4 +- tests/qemu-iotests/312 | 1 + tests/qemu-iotests/common.filter | 22 +- tests/qemu-iotests/common.rc | 22 +- tests/qemu-iotests/tests/qemu-img-close-errors | 96 +++++++ tests/qemu-iotests/tests/qemu-img-close-errors.out | 23 ++ 69 files changed, 1209 insertions(+), 552 deletions(-) create mode 100755 tests/qemu-iotests/tests/qemu-img-close-errors create mode 100644 tests/qemu-iotests/tests/qemu-img-close-errors.out