Qemu is adding qemu:joint-allocation as a single context combining the
two bits of base:allocation and a compression of qemu:allocation-depth
into two bits [1].  Decoding the bits makes it easier for humans to
see the result of that context.

[1] https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg02446.html
---

Obviously, this libnbd patch should only go in if the qemu RFC is
accepted favorably.  With this patch applied, the example listed in my
qemu patch 2/2 commit message [2] becomes

$ ~/libnbd/run nbdinfo --map=qemu:joint-allocation nbd://localhost
         0       65536    3  hole,zero,unallocated
     65536       65536    4  allocated,local
    131072       65536    7  hole,zero,local
    196608       65536    3  hole,zero,unallocated

[2] https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg02448.html

For what it's worth, you can also play with the qemu+libnbd patches at:
https://repo.or.cz/qemu/ericb.git/ master
https://repo.or.cz/libnbd/ericb.git/ master

(I sometimes rewind those branches, but they'll be stable for at least
a few days after this email)

 info/map.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/info/map.c b/info/map.c
index ae6d4fe..21e8657 100644
--- a/info/map.c
+++ b/info/map.c
@@ -226,6 +226,27 @@ extent_description (const char *metacontext, uint32_t type)
       return ret;
     }
   }
+  else if (strcmp (metacontext, "qemu:joint-allocation") == 0) {
+    /* Combo of base:allocation and stripped-down qemu:allocation-depth */
+    const char *base, *depth;
+    switch (type & 3) {
+    case 0: base = "allocated"; break;
+    case 1: base = "hole"; break;
+    case 2: base = "zero"; break;
+    case 3: base = "hole,zero"; break;
+    }
+    switch (type & 0xc) {
+    case 0: depth = "unallocated"; break;
+    case 4: depth = "local"; break;
+    case 8: depth = "backing"; break;
+    case 12: depth = "<unexpected depth>"; break;
+    }
+    if (asprintf (&ret, "%s,%s", base, depth) == -1) {
+      perror ("asprintf");
+      exit (EXIT_FAILURE);
+    }
+    return ret;
+  }

   return NULL;   /* Don't know - description field will be omitted. */
 }
-- 
2.31.1

On Thu, Jun 10, 2021 at 12:32 AM Eric Blake <eblake@redhat.com> wrote:
>
> Qemu is adding qemu:joint-allocation as a single context combining the
> two bits of base:allocation and a compression of qemu:allocation-depth
> into two bits [1].  Decoding the bits makes it easier for humans to
> see the result of that context.
>
> [1] https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg02446.html
> ---
>
> Obviously, this libnbd patch should only go in if the qemu RFC is
> accepted favorably.  With this patch applied, the example listed in my
> qemu patch 2/2 commit message [2] becomes
>
> $ ~/libnbd/run nbdinfo --map=qemu:joint-allocation nbd://localhost
>          0       65536    3  hole,zero,unallocated
>      65536       65536    4  allocated,local
>     131072       65536    7  hole,zero,local
>     196608       65536    3  hole,zero,unallocated
>
> [2] https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg02448.html
>
> For what it's worth, you can also play with the qemu+libnbd patches at:
> https://repo.or.cz/qemu/ericb.git/ master
> https://repo.or.cz/libnbd/ericb.git/ master
>
> (I sometimes rewind those branches, but they'll be stable for at least
> a few days after this email)
>
>  info/map.c | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
>
> diff --git a/info/map.c b/info/map.c
> index ae6d4fe..21e8657 100644
> --- a/info/map.c
> +++ b/info/map.c
> @@ -226,6 +226,27 @@ extent_description (const char *metacontext, uint32_t type)
>        return ret;
>      }
>    }
> +  else if (strcmp (metacontext, "qemu:joint-allocation") == 0) {
> +    /* Combo of base:allocation and stripped-down qemu:allocation-depth */
> +    const char *base, *depth;
> +    switch (type & 3) {
> +    case 0: base = "allocated"; break;
> +    case 1: base = "hole"; break;
> +    case 2: base = "zero"; break;
> +    case 3: base = "hole,zero"; break;
> +    }
> +    switch (type & 0xc) {
> +    case 0: depth = "unallocated"; break;

Is this possible? qemu reports BDRV_BLOCK_DATA but not BDRV_BLOCK_ALLOCATED?

Anyway this seems like a valid way to present qemu response.

> +    case 4: depth = "local"; break;
> +    case 8: depth = "backing"; break;
> +    case 12: depth = "<unexpected depth>"; break;

This should not be possible based on the qemu patch, but printing this
seems like a good solution, and can help to debug such an issue.

Thinking about client code trying to copy extents based on the flags,
the client should abort the operation since qemu response is invalid.

> +    }
> +    if (asprintf (&ret, "%s,%s", base, depth) == -1) {
> +      perror ("asprintf");
> +      exit (EXIT_FAILURE);
> +    }
> +    return ret;
> +  }
>
>    return NULL;   /* Don't know - description field will be omitted. */
>  }
> --
> 2.31.1
>

On Thu, Jun 10, 2021 at 01:20:13AM +0300, Nir Soffer wrote:
> > +  else if (strcmp (metacontext, "qemu:joint-allocation") == 0) {
> > +    /* Combo of base:allocation and stripped-down qemu:allocation-depth */
> > +    const char *base, *depth;
> > +    switch (type & 3) {
> > +    case 0: base = "allocated"; break;
> > +    case 1: base = "hole"; break;
> > +    case 2: base = "zero"; break;
> > +    case 3: base = "hole,zero"; break;
> > +    }
> > +    switch (type & 0xc) {
> > +    case 0: depth = "unallocated"; break;
> 
> Is this possible? qemu reports BDRV_BLOCK_DATA but not BDRV_BLOCK_ALLOCATED?

No, qemu should never report a status of 0 (which in this code would
produce the string "allocated,unallocated", although a v2 may change
to print "<unexpected value>").

Remember, BDRV_BLOCK_ALLOCATED is a bit of a misnomer - it has nothing
to do with whether a cluster occupies allocated space, but rather
whether the local image in the backing chain provides the contents of
the cluster (rather than deferring to the backing chain).  The code in
block/io.c guarantees that if a block device reports BDRV_BLOCK_DATA,
then the block layer also reports BDRV_BLOCK_ALLOCATED (that is, any
cluster that provides guest-visible data by necessity implies that the
current layer of the backing chain is important).

However, it DOES point out that "allocated" might not be the best name
in libnbd; perhaps "data" or "normal" would be better for the NBD
base:allocation status of 0.

> 
> Anyway this seems like a valid way to present qemu response.
> 
> > +    case 4: depth = "local"; break;
> > +    case 8: depth = "backing"; break;
> > +    case 12: depth = "<unexpected depth>"; break;
> 
> This should not be possible based on the qemu patch, but printing this
> seems like a good solution, and can help to debug such an issue.
> 
> Thinking about client code trying to copy extents based on the flags,
> the client should abort the operation since qemu response is invalid.
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

On Thu, Jun 10, 2021 at 4:06 PM Eric Blake <eblake@redhat.com> wrote:
>
> On Thu, Jun 10, 2021 at 01:20:13AM +0300, Nir Soffer wrote:
> > > +  else if (strcmp (metacontext, "qemu:joint-allocation") == 0) {
> > > +    /* Combo of base:allocation and stripped-down qemu:allocation-depth */
> > > +    const char *base, *depth;
> > > +    switch (type & 3) {
> > > +    case 0: base = "allocated"; break;
> > > +    case 1: base = "hole"; break;
> > > +    case 2: base = "zero"; break;
> > > +    case 3: base = "hole,zero"; break;
> > > +    }
> > > +    switch (type & 0xc) {
> > > +    case 0: depth = "unallocated"; break;
> >
> > Is this possible? qemu reports BDRV_BLOCK_DATA but not BDRV_BLOCK_ALLOCATED?
>
> No, qemu should never report a status of 0 (which in this code would
> produce the string "allocated,unallocated", although a v2 may change
> to print "<unexpected value>").
>
> Remember, BDRV_BLOCK_ALLOCATED is a bit of a misnomer - it has nothing
> to do with whether a cluster occupies allocated space, but rather
> whether the local image in the backing chain provides the contents of
> the cluster (rather than deferring to the backing chain).  The code in
> block/io.c guarantees that if a block device reports BDRV_BLOCK_DATA,
> then the block layer also reports BDRV_BLOCK_ALLOCATED (that is, any
> cluster that provides guest-visible data by necessity implies that the
> current layer of the backing chain is important).
>
> However, it DOES point out that "allocated" might not be the best name
> in libnbd; perhaps "data" or "normal" would be better for the NBD
> base:allocation status of 0.

Yes! it also aligns better with zero, and the output is similar to qemu-img map.
Hopefully the semantics of "data" in qemu-img map and libnbd is the same.