[PATCH v2 00/33] block: publish backup-top filter

Vladimir Sementsov-Ogievskiy posted 33 patches 2 years, 11 months ago
Only 32 patches received!
There is a newer version of this series
qapi/block-core.json                        |  30 ++-
block/{backup-top.h => copy-before-write.h} |  26 +-
include/block/block-copy.h                  |   4 +-
include/block/block.h                       |   2 +
include/hw/qdev-properties.h                |   1 +
include/sysemu/block-backend.h              |   1 +
block.c                                     |  52 +++-
block/backup-top.c                          | 253 ------------------
block/backup.c                              | 115 ++-------
block/block-backend.c                       |   8 +
block/block-copy.c                          |  98 ++++++-
block/copy-before-write.c                   | 268 ++++++++++++++++++++
hw/core/qdev-properties-system.c            |  43 +++-
hw/core/qdev-properties.c                   |   6 +-
MAINTAINERS                                 |   4 +-
block/meson.build                           |   2 +-
python/qemu/machine.py                      |  30 ++-
tests/qemu-iotests/222                      | 159 ------------
tests/qemu-iotests/222.out                  |  67 -----
tests/qemu-iotests/283                      |  35 ++-
tests/qemu-iotests/283.out                  |   4 +-
tests/qemu-iotests/297                      |   2 +-
tests/qemu-iotests/iotests.py               |   9 +-
tests/qemu-iotests/tests/image-fleecing     | 192 ++++++++++++++
tests/qemu-iotests/tests/image-fleecing.out | 139 ++++++++++
25 files changed, 890 insertions(+), 660 deletions(-)
rename block/{backup-top.h => copy-before-write.h} (56%)
delete mode 100644 block/backup-top.c
create mode 100644 block/copy-before-write.c
delete mode 100755 tests/qemu-iotests/222
delete mode 100644 tests/qemu-iotests/222.out
create mode 100755 tests/qemu-iotests/tests/image-fleecing
create mode 100644 tests/qemu-iotests/tests/image-fleecing.out
[PATCH v2 00/33] block: publish backup-top filter
Posted by Vladimir Sementsov-Ogievskiy 2 years, 11 months ago
Hi all!

v2:
01-02: new
03: don't bother with supporting empty child: we should never have such
    at this point
05: add comment
06: keep checking conflict with global
    add realized_set_allowed to qdev_prop_drive_iothread
07: improve cbw_cbw() name
    improve commit message
10: rebased on unchanged backup_calculate_cluster_size(). keep r-b  CHECK ME
12: new
13: drop extra bdrv_unref()
18: add compress local variable
    add comment about x-deprecated-compress
19: new, replacement for "[PATCH 17/21] block/block-copy: switch to fully set bitmap by default"
22: improve qapi documentation
23-33: test: a lot of refactoring

We have image fleecing scheme to export point-in-time state of active
disk (iotest 222):


                                      backup(sync=none)
                     ┌───────────────────────────────────────┐
                     ▼                                       │
┌────────────┐     ┌────────────────┐  backing             ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘     └────────────────┘                      └─────────────┘
                                                             ▲
┌────────────┐                                               │
│ guest blk  │ ──────────────────────────────────────────────┘
└────────────┘                        


Actually, backup job inserts a backup-top filter, so in detail it looks
like:

                                      backup(sync=none)
                     ┌───────────────────────────────────────┐
                     ▼                                       │
┌────────────┐     ┌────────────────┐  backing             ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘     └────────────────┘                      └─────────────┘
                     ▲                                       ▲
                     │ target                                │
                     │                                       │
┌────────────┐     ┌────────────────┐  backing               │
│ guest blk  │ ──▶ │   backup-top   │ ───────────────────────┘
└────────────┘     └────────────────┘

And job does nothing here. In a new blockdev world user is intended to
operate on node level, and insert/remove filters by hand. Let's get rid
of job in the scheme:

┌────────────┐     ┌────────────────┐  backing             ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘     └────────────────┘                      └─────────────┘
                     ▲                                       ▲
                     │ target                                │
                     │                                       │
┌────────────┐     ┌────────────────┐  backing               │
│ guest blk  │ ──▶ │   backup-top   │ ───────────────────────┘
└────────────┘     └────────────────┘


The series prepares qom-set to make possible inserting filters above
root node (patches 03-06), rename backup-top to copy-before-write, do
other preparations for publishing the filter, and finally publish it,
add qapi interface and test new fleecing scheme in 222 (first, some
good test refactoring).

Vladimir Sementsov-Ogievskiy (33):
  block: rename bdrv_replace_child to bdrv_replace_child_tran
  block: comment graph-modifying function not updating permissions
  block: introduce bdrv_replace_child_bs()
  block: introduce blk_replace_bs
  qdev-properties: PropertyInfo: add realized_set_allowed field
  qdev: allow setting drive property for realized device
  block: rename backup-top to copy-before-write
  block/backup: drop support for copy_range
  block-copy: always set BDRV_REQ_SERIALISING flag
  block/backup: move cluster size calculation to block-copy
  block/copy-before-write: relax permission requirements when no parents
  block/copy-before-write: drop extra bdrv_unref on failure path
  block/copy-before-write: use file child instead of backing
  block/copy-before-write: bdrv_cbw_append(): replace child at last
  block/copy-before-write: introduce cbw_init()
  block/copy-before-write: cbw_init(): rename variables
  block/copy-before-write: cbw_init(): use file child after attaching
  block/copy-before-write: cbw_init(): use options
  block/copy-before-write: initialize block-copy bitmap
  block/block-copy: make setting progress optional
  block/copy-before-write: make public block driver
  qapi: publish copy-before-write filter
  python/qemu/machine.py: refactor _qemu_args()
  python/qemu/machine: QEMUMachine: improve qmp() method
  iotests.py: VM: add own __enter__ method
  iotests/222: fix pylint and mypy complains
  iotests/222: constantly use single quotes for strings
  iotests: move 222 to tests/image-fleecing
  iotests.py: hmp_qemu_io: support qdev
  iotests/image-fleecing: proper source device
  iotests/image-fleecing: rename tgt_node
  iotests/image-fleecing: prepare for adding new test-case
  iotests/image-fleecing: add test-case for copy-before-write filter

 qapi/block-core.json                        |  30 ++-
 block/{backup-top.h => copy-before-write.h} |  26 +-
 include/block/block-copy.h                  |   4 +-
 include/block/block.h                       |   2 +
 include/hw/qdev-properties.h                |   1 +
 include/sysemu/block-backend.h              |   1 +
 block.c                                     |  52 +++-
 block/backup-top.c                          | 253 ------------------
 block/backup.c                              | 115 ++-------
 block/block-backend.c                       |   8 +
 block/block-copy.c                          |  98 ++++++-
 block/copy-before-write.c                   | 268 ++++++++++++++++++++
 hw/core/qdev-properties-system.c            |  43 +++-
 hw/core/qdev-properties.c                   |   6 +-
 MAINTAINERS                                 |   4 +-
 block/meson.build                           |   2 +-
 python/qemu/machine.py                      |  30 ++-
 tests/qemu-iotests/222                      | 159 ------------
 tests/qemu-iotests/222.out                  |  67 -----
 tests/qemu-iotests/283                      |  35 ++-
 tests/qemu-iotests/283.out                  |   4 +-
 tests/qemu-iotests/297                      |   2 +-
 tests/qemu-iotests/iotests.py               |   9 +-
 tests/qemu-iotests/tests/image-fleecing     | 192 ++++++++++++++
 tests/qemu-iotests/tests/image-fleecing.out | 139 ++++++++++
 25 files changed, 890 insertions(+), 660 deletions(-)
 rename block/{backup-top.h => copy-before-write.h} (56%)
 delete mode 100644 block/backup-top.c
 create mode 100644 block/copy-before-write.c
 delete mode 100755 tests/qemu-iotests/222
 delete mode 100644 tests/qemu-iotests/222.out
 create mode 100755 tests/qemu-iotests/tests/image-fleecing
 create mode 100644 tests/qemu-iotests/tests/image-fleecing.out

-- 
2.29.2


Re: [PATCH v2 00/33] block: publish backup-top filter
Posted by Max Reitz 2 years, 10 months ago
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
>
> v2:
> 01-02: new
> 03: don't bother with supporting empty child: we should never have such
>      at this point
> 05: add comment
> 06: keep checking conflict with global
>      add realized_set_allowed to qdev_prop_drive_iothread
> 07: improve cbw_cbw() name
>      improve commit message
> 10: rebased on unchanged backup_calculate_cluster_size(). keep r-b  CHECK ME

I checked you! It’s ok. :)

(and btw, I’ll look at the rest tomorrow)

> 12: new
> 13: drop extra bdrv_unref()
> 18: add compress local variable
>      add comment about x-deprecated-compress
> 19: new, replacement for "[PATCH 17/21] block/block-copy: switch to fully set bitmap by default"
> 22: improve qapi documentation
> 23-33: test: a lot of refactoring


Re: [PATCH v2 00/33] block: publish backup-top filter
Posted by Vladimir Sementsov-Ogievskiy 2 years, 10 months ago
31.05.2021 20:11, Max Reitz wrote:
> On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote:
>> Hi all!
>>
>> v2:
>> 01-02: new
>> 03: don't bother with supporting empty child: we should never have such
>>      at this point
>> 05: add comment
>> 06: keep checking conflict with global
>>      add realized_set_allowed to qdev_prop_drive_iothread
>> 07: improve cbw_cbw() name
>>      improve commit message
>> 10: rebased on unchanged backup_calculate_cluster_size(). keep r-b  CHECK ME
> 
> I checked you! It’s ok. :)
> 
> (and btw, I’ll look at the rest tomorrow)

Thanks)

Actually, "CHECK ME" was for myself, and "ME" is a patch, not me) I forget to drop it.. Interesting, did I forget to check

> 
>> 12: new
>> 13: drop extra bdrv_unref()
>> 18: add compress local variable
>>      add comment about x-deprecated-compress
>> 19: new, replacement for "[PATCH 17/21] block/block-copy: switch to fully set bitmap by default"
>> 22: improve qapi documentation
>> 23-33: test: a lot of refactoring
> 


-- 
Best regards,
Vladimir

[PATCH v2 28/33] iotests: move 222 to tests/image-fleecing
Posted by Vladimir Sementsov-Ogievskiy 2 years, 11 months ago
Give a good name to test file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/{222 => tests/image-fleecing}         | 0
 tests/qemu-iotests/{222.out => tests/image-fleecing.out} | 0
 2 files changed, 0 insertions(+), 0 deletions(-)
 rename tests/qemu-iotests/{222 => tests/image-fleecing} (100%)
 rename tests/qemu-iotests/{222.out => tests/image-fleecing.out} (100%)

diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/tests/image-fleecing
similarity index 100%
rename from tests/qemu-iotests/222
rename to tests/qemu-iotests/tests/image-fleecing
diff --git a/tests/qemu-iotests/222.out b/tests/qemu-iotests/tests/image-fleecing.out
similarity index 100%
rename from tests/qemu-iotests/222.out
rename to tests/qemu-iotests/tests/image-fleecing.out
-- 
2.29.2


Re: [PATCH v2 28/33] iotests: move 222 to tests/image-fleecing
Posted by Max Reitz 2 years, 10 months ago
On 20.05.21 16:22, Vladimir Sementsov-Ogievskiy wrote:
> Give a good name to test file.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>   tests/qemu-iotests/{222 => tests/image-fleecing}         | 0
>   tests/qemu-iotests/{222.out => tests/image-fleecing.out} | 0
>   2 files changed, 0 insertions(+), 0 deletions(-)
>   rename tests/qemu-iotests/{222 => tests/image-fleecing} (100%)
>   rename tests/qemu-iotests/{222.out => tests/image-fleecing.out} (100%)

Reviewed-by: Max Reitz <mreitz@redhat.com>