[PULL v2 for 8.0 00/35] various fixes (testing, plugins, gitdm)

Alex Bennée posted 35 patches 1 year, 1 month ago
Only 0 patches received!
include/hw/core/cpu.h                              |   2 +-
include/qemu/plugin-event.h                        |  26 +++++
include/qemu/plugin.h                              |  27 ++---
include/qemu/qemu-plugin.h                         |  47 ++++++++-
include/user/syscall-trace.h                       |   1 +
accel/accel-softmmu.c                              |   2 +-
accel/tcg/cpu-exec-common.c                        |   3 +
accel/tcg/cpu-exec.c                               |   5 +-
block/monitor/block-hmp-cmds.c                     |   1 +
cpu.c                                              |   1 +
dump/dump.c                                        |   1 +
dump/win_dump.c                                    |   1 +
gdbstub/gdbstub.c                                  |   1 +
hw/arm/collie.c                                    |   2 +
hw/arm/cubieboard.c                                |   1 +
hw/arm/musicpal.c                                  |   2 +
hw/arm/npcm7xx_boards.c                            |   2 +
hw/arm/nseries.c                                   |   2 +
hw/arm/omap_sx1.c                                  |   2 +
hw/arm/orangepi.c                                  |   1 +
hw/arm/palm.c                                      |   2 +
hw/core/loader.c                                   |   1 +
hw/core/machine-smp.c                              |   2 +
hw/i386/kvm/xen_evtchn.c                           |   1 +
hw/i386/kvm/xen_xenstore.c                         |   1 +
hw/i386/sgx.c                                      |   1 +
hw/intc/apic.c                                     |   1 +
hw/loongarch/acpi-build.c                          |   1 +
hw/loongarch/virt.c                                |   2 +
hw/m68k/next-cube.c                                |   1 +
hw/m68k/q800.c                                     |   1 +
hw/m68k/virt.c                                     |   1 +
hw/mem/memory-device.c                             |   1 +
hw/mem/sparse-mem.c                                |   1 +
hw/openrisc/boot.c                                 |   1 +
hw/ppc/ppc4xx_sdram.c                              |   1 +
hw/ppc/spapr_softmmu.c                             |   2 +
hw/riscv/opentitan.c                               |   1 +
hw/riscv/shakti_c.c                                |   1 +
hw/riscv/virt-acpi-build.c                         |   1 +
hw/vfio/display.c                                  |   1 +
hw/vfio/igd.c                                      |   1 +
hw/vfio/migration.c                                |   1 +
linux-user/elfload.c                               |   1 +
linux-user/exit.c                                  |   1 +
linux-user/syscall.c                               |   1 +
migration/dirtyrate.c                              |   1 +
migration/exec.c                                   |   1 +
plugins/core.c                                     |  11 --
target/i386/cpu.c                                  |   1 +
target/i386/host-cpu.c                             |   1 +
target/i386/kvm/xen-emu.c                          |   1 +
target/i386/sev.c                                  |   1 +
target/i386/whpx/whpx-apic.c                       |   1 +
target/mips/cpu.c                                  |   1 +
target/s390x/cpu-sysemu.c                          |   1 +
target/s390x/cpu_models.c                          |   1 +
target/s390x/diag.c                                |   2 +
tcg/tcg-op.c                                       |   1 -
tests/qtest/migration-test.c                       |  10 +-
.../custom-runners/centos-stream-8-x86_64.yml      |  18 ++--
contrib/gitdm/domain-map                           |   8 +-
contrib/gitdm/group-map-alibaba                    |   7 ++
contrib/gitdm/group-map-amd                        |   8 ++
contrib/gitdm/group-map-individuals                |   4 +
gitdm.config                                       |   2 +
.../ci/org.centos/stream/8/build-environment.yml   |   1 +
scripts/ci/setup/gitlab-runner.yml                 |  20 +++-
tests/avocado/avocado_qemu/__init__.py             |  10 ++
tests/avocado/igb.py                               |  38 -------
tests/avocado/machine_aarch64_virt.py              |   8 +-
tests/avocado/multiprocess.py                      |   1 +
tests/avocado/netdev-ethtool.py                    | 116 +++++++++++++++++++++
tests/avocado/tuxrun_baselines.py                  |  60 +++++------
tests/check-block.sh                               |  43 --------
tests/docker/Makefile.include                      |   2 +-
tests/qemu-iotests/check                           |  30 +++++-
tests/qemu-iotests/meson.build                     |  35 +++++--
tests/qemu-iotests/testenv.py                      |  20 ++--
tests/qemu-iotests/testrunner.py                   |  43 +++-----
tests/tcg/Makefile.target                          |   7 ++
tests/vm/openbsd                                   |  30 +++++-
ui/cocoa.m                                         |   1 +
83 files changed, 470 insertions(+), 236 deletions(-)
create mode 100644 include/qemu/plugin-event.h
create mode 100644 contrib/gitdm/group-map-alibaba
create mode 100644 contrib/gitdm/group-map-amd
delete mode 100644 tests/avocado/igb.py
create mode 100644 tests/avocado/netdev-ethtool.py
delete mode 100755 tests/check-block.sh
[PULL v2 for 8.0 00/35] various fixes (testing, plugins, gitdm)
Posted by Alex Bennée 1 year, 1 month ago
The following changes since commit c283ff89d11ff123efc9af49128ef58511f73012:

  Update version for v8.0.0-rc1 release (2023-03-21 17:15:43 +0000)

are available in the Git repository at:

  https://gitlab.com/stsquad/qemu.git tags/pull-for-8.0-220323-1

for you to fetch changes up to e35b9a2e81ccce86db6f1417b1d73bb97d7cbc17:

  qtests: avoid printing comments before g_test_init() (2023-03-22
  15:08:26 +0000)

Note you will need to remove the old openbsd disk image to trigger a
rebuild that avoids the issues with -ENOSPC. My pipeline can be seen
here:

  https://gitlab.com/stsquad/qemu/-/pipelines/814624909

----------------------------------------------------------------
Misc fixes for 8.0 (testing, plugins, gitdm)

  - update Alpine image used for testing images
  - include libslirp in custom runner build env
  - update gitlab-runner recipe for CentOS
  - update docker calls for better caching behaviour
  - document some plugin callbacks
  - don't use tags to define drives for lkft baseline tests
  - fix missing clear of plugin_mem_cbs
  - fix iotests to report individual results
  - update the gitdm metadata for contributors
  - avoid printing comments before g_test_init()
  - probe for multiprocess support before running avocado test
  - refactor igb.py into netdev-ethtool.py avocado test
  - rebuild openbsd to have more space space for iotests

----------------------------------------------------------------
Alex Bennée (16):
      tests/avocado: probe for multi-process support before running test
      tests/avocado: re-factor igb test to avoid timeouts
      tests/docker: all add DOCKER_BUILDKIT to RUNC environment
      scripts/ci: add libslirp-devel to build-environment
      scripts/ci: update gitlab-runner playbook to handle CentOS
      gitlab: update centos-8-stream job
      include/qemu: add documentation for memory callbacks
      tests/tcg: add some help output for running individual tests
      tests/avocado: don't use tags to define drive
      contrib/gitdm: Add ASPEED Technology to the domain map
      contrib/gitdm: Add SYRMIA to the domain map
      contrib/gitdm: add Amazon to the domain map
      contrib/gitdm: add Alibaba to the domain-map
      contrib/gitdm: add revng to domain map
      contrib/gitdm: add more individual contributors
      contrib/gitdm: add group map for AMD

Daniel P. Berrangé (11):
      tests/vm: skip X11 in openbsd installation
      tests/vm: custom openbsd partitioning to increase /home space
      iotests: explicitly pass source/build dir to 'check' command
      iotests: allow test discovery before building
      iotests: strip subdir path when listing tests
      iotests: print TAP protocol version when reporting tests
      iotests: connect stdin to /dev/null when running tests
      iotests: always use a unique sub-directory per test
      iotests: register each I/O test separately with meson
      iotests: remove the check-block.sh script
      qtests: avoid printing comments before g_test_init()

Marcin Juszkiewicz (1):
      tests/avocado: update AArch64 tests to Alpine 3.17.2

Richard Henderson (7):
      tcg: Clear plugin_mem_cbs on TB exit
      tcg: Drop plugin_gen_disable_mem_helpers from tcg_gen_exit_tb
      include/qemu/plugin: Remove QEMU_PLUGIN_ASSERT
      *: Add missing includes of qemu/error-report.h
      *: Add missing includes of qemu/plugin.h
      include/qemu: Split out plugin-event.h
      include/qemu/plugin: Inline qemu_plugin_disable_mem_helpers

 include/hw/core/cpu.h                              |   2 +-
 include/qemu/plugin-event.h                        |  26 +++++
 include/qemu/plugin.h                              |  27 ++---
 include/qemu/qemu-plugin.h                         |  47 ++++++++-
 include/user/syscall-trace.h                       |   1 +
 accel/accel-softmmu.c                              |   2 +-
 accel/tcg/cpu-exec-common.c                        |   3 +
 accel/tcg/cpu-exec.c                               |   5 +-
 block/monitor/block-hmp-cmds.c                     |   1 +
 cpu.c                                              |   1 +
 dump/dump.c                                        |   1 +
 dump/win_dump.c                                    |   1 +
 gdbstub/gdbstub.c                                  |   1 +
 hw/arm/collie.c                                    |   2 +
 hw/arm/cubieboard.c                                |   1 +
 hw/arm/musicpal.c                                  |   2 +
 hw/arm/npcm7xx_boards.c                            |   2 +
 hw/arm/nseries.c                                   |   2 +
 hw/arm/omap_sx1.c                                  |   2 +
 hw/arm/orangepi.c                                  |   1 +
 hw/arm/palm.c                                      |   2 +
 hw/core/loader.c                                   |   1 +
 hw/core/machine-smp.c                              |   2 +
 hw/i386/kvm/xen_evtchn.c                           |   1 +
 hw/i386/kvm/xen_xenstore.c                         |   1 +
 hw/i386/sgx.c                                      |   1 +
 hw/intc/apic.c                                     |   1 +
 hw/loongarch/acpi-build.c                          |   1 +
 hw/loongarch/virt.c                                |   2 +
 hw/m68k/next-cube.c                                |   1 +
 hw/m68k/q800.c                                     |   1 +
 hw/m68k/virt.c                                     |   1 +
 hw/mem/memory-device.c                             |   1 +
 hw/mem/sparse-mem.c                                |   1 +
 hw/openrisc/boot.c                                 |   1 +
 hw/ppc/ppc4xx_sdram.c                              |   1 +
 hw/ppc/spapr_softmmu.c                             |   2 +
 hw/riscv/opentitan.c                               |   1 +
 hw/riscv/shakti_c.c                                |   1 +
 hw/riscv/virt-acpi-build.c                         |   1 +
 hw/vfio/display.c                                  |   1 +
 hw/vfio/igd.c                                      |   1 +
 hw/vfio/migration.c                                |   1 +
 linux-user/elfload.c                               |   1 +
 linux-user/exit.c                                  |   1 +
 linux-user/syscall.c                               |   1 +
 migration/dirtyrate.c                              |   1 +
 migration/exec.c                                   |   1 +
 plugins/core.c                                     |  11 --
 target/i386/cpu.c                                  |   1 +
 target/i386/host-cpu.c                             |   1 +
 target/i386/kvm/xen-emu.c                          |   1 +
 target/i386/sev.c                                  |   1 +
 target/i386/whpx/whpx-apic.c                       |   1 +
 target/mips/cpu.c                                  |   1 +
 target/s390x/cpu-sysemu.c                          |   1 +
 target/s390x/cpu_models.c                          |   1 +
 target/s390x/diag.c                                |   2 +
 tcg/tcg-op.c                                       |   1 -
 tests/qtest/migration-test.c                       |  10 +-
 .../custom-runners/centos-stream-8-x86_64.yml      |  18 ++--
 contrib/gitdm/domain-map                           |   8 +-
 contrib/gitdm/group-map-alibaba                    |   7 ++
 contrib/gitdm/group-map-amd                        |   8 ++
 contrib/gitdm/group-map-individuals                |   4 +
 gitdm.config                                       |   2 +
 .../ci/org.centos/stream/8/build-environment.yml   |   1 +
 scripts/ci/setup/gitlab-runner.yml                 |  20 +++-
 tests/avocado/avocado_qemu/__init__.py             |  10 ++
 tests/avocado/igb.py                               |  38 -------
 tests/avocado/machine_aarch64_virt.py              |   8 +-
 tests/avocado/multiprocess.py                      |   1 +
 tests/avocado/netdev-ethtool.py                    | 116 +++++++++++++++++++++
 tests/avocado/tuxrun_baselines.py                  |  60 +++++------
 tests/check-block.sh                               |  43 --------
 tests/docker/Makefile.include                      |   2 +-
 tests/qemu-iotests/check                           |  30 +++++-
 tests/qemu-iotests/meson.build                     |  35 +++++--
 tests/qemu-iotests/testenv.py                      |  20 ++--
 tests/qemu-iotests/testrunner.py                   |  43 +++-----
 tests/tcg/Makefile.target                          |   7 ++
 tests/vm/openbsd                                   |  30 +++++-
 ui/cocoa.m                                         |   1 +
 83 files changed, 470 insertions(+), 236 deletions(-)
 create mode 100644 include/qemu/plugin-event.h
 create mode 100644 contrib/gitdm/group-map-alibaba
 create mode 100644 contrib/gitdm/group-map-amd
 delete mode 100644 tests/avocado/igb.py
 create mode 100644 tests/avocado/netdev-ethtool.py
 delete mode 100755 tests/check-block.sh

-- 
2.39.2


Re: [PULL v2 for 8.0 00/35] various fixes (testing, plugins, gitdm)
Posted by Peter Maydell 1 year, 1 month ago
On Wed, 22 Mar 2023 at 16:33, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit c283ff89d11ff123efc9af49128ef58511f73012:
>
>   Update version for v8.0.0-rc1 release (2023-03-21 17:15:43 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/stsquad/qemu.git tags/pull-for-8.0-220323-1
>
> for you to fetch changes up to e35b9a2e81ccce86db6f1417b1d73bb97d7cbc17:
>
>   qtests: avoid printing comments before g_test_init() (2023-03-22
>   15:08:26 +0000)
>
> Note you will need to remove the old openbsd disk image to trigger a
> rebuild that avoids the issues with -ENOSPC. My pipeline can be seen
> here:
>
>   https://gitlab.com/stsquad/qemu/-/pipelines/814624909
>
> ----------------------------------------------------------------
> Misc fixes for 8.0 (testing, plugins, gitdm)
>
>   - update Alpine image used for testing images
>   - include libslirp in custom runner build env
>   - update gitlab-runner recipe for CentOS
>   - update docker calls for better caching behaviour
>   - document some plugin callbacks
>   - don't use tags to define drives for lkft baseline tests
>   - fix missing clear of plugin_mem_cbs
>   - fix iotests to report individual results
>   - update the gitdm metadata for contributors
>   - avoid printing comments before g_test_init()
>   - probe for multiprocess support before running avocado test
>   - refactor igb.py into netdev-ethtool.py avocado test
>   - rebuild openbsd to have more space space for iotests

I saw this on ppc64. I suspect it of being a pre-existing
intermittent -- I'm retrying it.

▶ 737/761 qcow2 copy-before-write
           FAIL
737/761 qemu:block / io-qcow2-copy-before-write
           ERROR           6.80s   exit status 1
――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――
stderr:
--- /home/pm215/qemu/tests/qemu-iotests/tests/copy-before-write.out
+++ /home/pm215/qemu/build/all/scratch/qcow2-file-copy-before-write/copy-before-write.out.bad
@@ -1,5 +1,21 @@
-....
+..F.
+======================================================================
+FAIL: test_timeout_break_guest (__main__.TestCbwError.test_timeout_break_guest)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+  File "/home/pm215/qemu/tests/qemu-iotests/tests/copy-before-write",
line 200, in test_timeout_break_guest
+    self.assertEqual(log, """\
+AssertionError: 'wrot[90 chars])\nwrote 524288/524288 bytes at offset
524288\[151 chars]c)\n' != 'wrot[90 chars])\nwrite failed: Connection
timed out\nread 10[85 chars]c)\n'
+  wrote 524288/524288 bytes at offset 0
+  512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
++ write failed: Connection timed out
+- wrote 524288/524288 bytes at offset 524288
+- 512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+  read 1048576/1048576 bytes at offset 0
+  1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+
 ----------------------------------------------------------------------
 Ran 4 tests

-OK
+FAILED (failures=1)

-- PMM
Re: [PULL v2 for 8.0 00/35] various fixes (testing, plugins, gitdm)
Posted by Alex Bennée 1 year, 1 month ago
Peter Maydell <peter.maydell@linaro.org> writes:

> On Wed, 22 Mar 2023 at 16:33, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit c283ff89d11ff123efc9af49128ef58511f73012:
>>
>>   Update version for v8.0.0-rc1 release (2023-03-21 17:15:43 +0000)
>>
>> are available in the Git repository at:
>>
>>   https://gitlab.com/stsquad/qemu.git tags/pull-for-8.0-220323-1
>>
>> for you to fetch changes up to e35b9a2e81ccce86db6f1417b1d73bb97d7cbc17:
>>
>>   qtests: avoid printing comments before g_test_init() (2023-03-22
>>   15:08:26 +0000)
>>
>> Note you will need to remove the old openbsd disk image to trigger a
>> rebuild that avoids the issues with -ENOSPC. My pipeline can be seen
>> here:
>>
>>   https://gitlab.com/stsquad/qemu/-/pipelines/814624909
>>
>> ----------------------------------------------------------------
>> Misc fixes for 8.0 (testing, plugins, gitdm)
>>
>>   - update Alpine image used for testing images
>>   - include libslirp in custom runner build env
>>   - update gitlab-runner recipe for CentOS
>>   - update docker calls for better caching behaviour
>>   - document some plugin callbacks
>>   - don't use tags to define drives for lkft baseline tests
>>   - fix missing clear of plugin_mem_cbs
>>   - fix iotests to report individual results
>>   - update the gitdm metadata for contributors
>>   - avoid printing comments before g_test_init()
>>   - probe for multiprocess support before running avocado test
>>   - refactor igb.py into netdev-ethtool.py avocado test
>>   - rebuild openbsd to have more space space for iotests
>
> I saw this on ppc64. I suspect it of being a pre-existing
> intermittent -- I'm retrying it.

On what platform is that?

>
> ▶ 737/761 qcow2 copy-before-write
>            FAIL
> 737/761 qemu:block / io-qcow2-copy-before-write
>            ERROR           6.80s   exit status 1
> ――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――
> stderr:
> --- /home/pm215/qemu/tests/qemu-iotests/tests/copy-before-write.out
> +++ /home/pm215/qemu/build/all/scratch/qcow2-file-copy-before-write/copy-before-write.out.bad
> @@ -1,5 +1,21 @@
> -....
> +..F.
> +======================================================================
> +FAIL: test_timeout_break_guest (__main__.TestCbwError.test_timeout_break_guest)
> +----------------------------------------------------------------------
> +Traceback (most recent call last):
> +  File "/home/pm215/qemu/tests/qemu-iotests/tests/copy-before-write",
> line 200, in test_timeout_break_guest
> +    self.assertEqual(log, """\
> +AssertionError: 'wrot[90 chars])\nwrote 524288/524288 bytes at offset
> 524288\[151 chars]c)\n' != 'wrot[90 chars])\nwrite failed: Connection
> timed out\nread 10[85 chars]c)\n'
> +  wrote 524288/524288 bytes at offset 0
> +  512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> ++ write failed: Connection timed out
> +- wrote 524288/524288 bytes at offset 524288
> +- 512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +  read 1048576/1048576 bytes at offset 0
> +  1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +
> +
>  ----------------------------------------------------------------------
>  Ran 4 tests
>
> -OK
> +FAILED (failures=1)
>
> -- PMM


-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro
Re: [PULL v2 for 8.0 00/35] various fixes (testing, plugins, gitdm)
Posted by Peter Maydell 1 year, 1 month ago
On Wed, 22 Mar 2023 at 21:54, Alex Bennée <alex.bennee@linaro.org> wrote:
>
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > On Wed, 22 Mar 2023 at 16:33, Alex Bennée <alex.bennee@linaro.org> wrote:
> >>
> >> The following changes since commit c283ff89d11ff123efc9af49128ef58511f73012:
> >>
> >>   Update version for v8.0.0-rc1 release (2023-03-21 17:15:43 +0000)
> >>
> >> are available in the Git repository at:
> >>
> >>   https://gitlab.com/stsquad/qemu.git tags/pull-for-8.0-220323-1
> >>
> >> for you to fetch changes up to e35b9a2e81ccce86db6f1417b1d73bb97d7cbc17:
> >>
> >>   qtests: avoid printing comments before g_test_init() (2023-03-22
> >>   15:08:26 +0000)
> >>
> >> Note you will need to remove the old openbsd disk image to trigger a
> >> rebuild that avoids the issues with -ENOSPC. My pipeline can be seen
> >> here:
> >>
> >>   https://gitlab.com/stsquad/qemu/-/pipelines/814624909
> >>
> >> ----------------------------------------------------------------
> >> Misc fixes for 8.0 (testing, plugins, gitdm)
> >>
> >>   - update Alpine image used for testing images
> >>   - include libslirp in custom runner build env
> >>   - update gitlab-runner recipe for CentOS
> >>   - update docker calls for better caching behaviour
> >>   - document some plugin callbacks
> >>   - don't use tags to define drives for lkft baseline tests
> >>   - fix missing clear of plugin_mem_cbs
> >>   - fix iotests to report individual results
> >>   - update the gitdm metadata for contributors
> >>   - avoid printing comments before g_test_init()
> >>   - probe for multiprocess support before running avocado test
> >>   - refactor igb.py into netdev-ethtool.py avocado test
> >>   - rebuild openbsd to have more space space for iotests
> >
> > I saw this on ppc64. I suspect it of being a pre-existing
> > intermittent -- I'm retrying it.
>
> On what platform is that?

ppc64be Linux (it's one of the gcc compile farm machines).
It was indeed intermittent, in that it didn't happen on retry. So:

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/8.0
for any user-visible changes.

-- PMM