The --without-default-devices configure option removes the
'default=y' from Kconfig files. It is useful to test missing
Kconfig dependencies for users wanting to have QEMU (system)
binaries with a particular subset of machines builtin.

If a machine can be built standalone, it can certainly be
built as part of a set. So the best way to test for regressions
is to test each machine individually.

As this is painful to test manually, add CI jobs to do it [*].
Since all jobs follow the same template, to ease maintenance
we generate the jobs using the jsonnet tool, which emit a YAML
file filled with all our jobs.

Since there is no "--enable-my-config" option, we have to write
the standalone config manually, overwritting each target .mak
file in default-configs/devices/.

The job is set to manual (there is no change in the current
set of jobs), but can be started automatically by setting the
QEMU_CI_STANDALONE_MACHINE_TESTING variable.

[*] https://docs.gitlab.com/ee/ci/yaml/#trigger-child-pipeline-with-generated-configuration-file

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
Example: https://gitlab.com/philmd/qemu/-/pipelines/309272883

Based-on: <20210525082556.4011380-1-f4bug@amsat.org>
---
 .gitlab-ci.d/qemu-project.yml             |  1 +
 .gitlab-ci.d/standalone-jobs-template.yml | 32 ++++++++++
 .gitlab-ci.d/standalone-jobs.jsonnet      | 74 +++++++++++++++++++++++
 .gitlab-ci.d/standalone-jobs.yml          | 27 +++++++++
 .gitlab-ci.yml                            |  3 +
 5 files changed, 137 insertions(+)
 create mode 100644 .gitlab-ci.d/standalone-jobs-template.yml
 create mode 100644 .gitlab-ci.d/standalone-jobs.jsonnet
 create mode 100644 .gitlab-ci.d/standalone-jobs.yml

diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
index 64cb2ba1da5..4018c48761c 100644
--- a/.gitlab-ci.d/qemu-project.yml
+++ b/.gitlab-ci.d/qemu-project.yml
@@ -9,3 +9,4 @@ include:
   - local: '/.gitlab-ci.d/crossbuilds.yml'
   - local: '/.gitlab-ci.d/buildtest.yml'
   - local: '/.gitlab-ci.d/static_checks.yml'
+  - local: '/.gitlab-ci.d/standalone-jobs.yml'
diff --git a/.gitlab-ci.d/standalone-jobs-template.yml b/.gitlab-ci.d/standalone-jobs-template.yml
new file mode 100644
index 00000000000..23e60cb4069
--- /dev/null
+++ b/.gitlab-ci.d/standalone-jobs-template.yml
@@ -0,0 +1,32 @@
+.native_build_standalone_job_template:
+  stage: build
+  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+  variables:
+    IMAGE: fedora
+    CONFIGURE_ARGS:
+      --without-default-devices
+      --disable-user
+      --disable-tools
+      --disable-docs
+  before_script:
+    - JOBS=$(expr $(nproc) + 1)
+  script:
+    - for target in $TARGETS ; do
+        echo > default-configs/devices/${target}-softmmu.mak ;
+        for line in $DEVICE_CONFIGS ; do
+          echo "$line"
+          echo "$line" > default-configs/devices/${target}-softmmu.mak ;
+        done ;
+      done
+    - mkdir build
+    - cd build
+    - ../configure --enable-werror $CONFIGURE_ARGS --target-list="$TARGETS"
+      || { cat config.log meson-logs/meson-log.txt && exit 1; }
+    - QEMU_BINS=$(echo $(for target in $TARGETS; do echo qemu-system-${target//-softmmu/}; done))
+    - make -j"$JOBS" $QEMU_BINS
+    - for binary in $QEMU_BINS ; do
+        for machine in $(./$binary -M help | egrep -v "^(Supported|none)" | awk '{print $1}') ; do
+            echo "=== testing $machine with $binary ===" ;
+            echo q | ./$binary -M $machine -nodefaults -display none $TESTING_ARGS -monitor stdio -S || exit 1 ;
+        done ;
+      done
diff --git a/.gitlab-ci.d/standalone-jobs.jsonnet b/.gitlab-ci.d/standalone-jobs.jsonnet
new file mode 100644
index 00000000000..6f403a51091
--- /dev/null
+++ b/.gitlab-ci.d/standalone-jobs.jsonnet
@@ -0,0 +1,74 @@
+local param_job(targets, configs, cmdline="") =
+  {
+    extends: ".native_build_standalone_job_template",
+
+    variables: {
+      TARGETS: targets,
+      DEVICE_CONFIGS: configs,
+      TESTING_ARGS: cmdline,
+    }
+  };
+
+{
+  include: { "local": "/.gitlab-ci.d/standalone-jobs-template.yml" },
+
+  // alpha
+    "alpha dp264": param_job("alpha-softmmu", "CONFIG_DP264=y"),
+
+  // arm
+
+  // avr
+    "avr arduino": param_job("avr-softmmu", "CONFIG_ARDUINO=y"),
+
+  // cris
+
+  // hexagon
+
+  // hppa
+    "hppa dino": param_job("hppa-softmmu", "CONFIG_DINO=y"),
+
+  // i386
+    "x86 isapc": param_job("i386-softmmu", "CONFIG_ISAPC=y"),
+    "x86 microvm": param_job("x86_64-softmmu", "CONFIG_MICROVM=y"),
+
+  // m68k
+
+  // microblaze
+    "mb zynqmp-pmu": param_job("microblaze-softmmu", "CONFIG_XLNX_ZYNQMP_PMU=y"),
+    "mb ml605": param_job("microblaze-softmmu", "CONFIG_PETALOGIX_ML605=y"),
+    "mb s3adsp1800": param_job("microblaze-softmmu", "CONFIG_PETALOGIX_S3ADSP1800=y"),
+
+  // mips
+    "mips jazz": param_job("mips64-softmmu", "CONFIG_FULOONG=y"),
+    "mips loongson3v": param_job("mips64-softmmu", "CONFIG_LOONGSON3V=y"),
+    "mips malta": param_job("mips-softmmu mipsel-softmmu", "CONFIG_MALTA=y", "-bios /dev/null"),
+
+  // nios2
+    "nios2 10m50": param_job("nios2-softmmu", "CONFIG_NIOS2_10M50=y"),
+    "nios2 nommu": param_job("nios2-softmmu", "CONFIG_NIOS2_GENERIC_NOMMU=y"),
+
+  // openrisc
+    "or1k sim": param_job("or1k-softmmu", "CONFIG_OR1K_SIM=y"),
+
+  // ppc
+
+  // riscv
+    "riscv opentitan": param_job("riscv32-softmmu", "CONFIG_OPENTITAN=y"),
+
+  // rx
+    "rx gdbsim": param_job("rx-softmmu", "CONFIG_RX_GDBSIM=y", "-bios /dev/null"),
+
+  // s390x
+    "s390 ccw": param_job("s390x-softmmu", "CONFIG_S390_CCW_VIRTIO=y"),
+
+  // sh4
+
+  // sparc
+
+  // tricore
+    "triboard": param_job("tricore-softmmu", "CONFIG_TRIBOARD=y"),
+
+  // xtensa
+    "xtensa sim": param_job("xtensaeb-softmmu", "CONFIG_XTENSA_SIM=y CONFIG_SEMIHOSTING=y"),
+    "xtensa virt": param_job("xtensa-softmmu", "CONFIG_XTENSA_VIRT=y CONFIG_SEMIHOSTING=y"),
+}
diff --git a/.gitlab-ci.d/standalone-jobs.yml b/.gitlab-ci.d/standalone-jobs.yml
new file mode 100644
index 00000000000..b094173ff12
--- /dev/null
+++ b/.gitlab-ci.d/standalone-jobs.yml
@@ -0,0 +1,27 @@
+build-standalone-machines:
+  stage: containers
+  image: alpine:latest
+  script:
+    - apk add -U jsonnet
+    - jsonnet .gitlab-ci.d/standalone-jobs.jsonnet > standalone-jobs-generated.yml
+  artifacts:
+    paths:
+      - standalone-jobs-generated.yml
+  rules:
+    # Automatically add the jobs if QEMU_CI_STANDALONE_MACHINE_TESTING is set
+    - if: '$QEMU_CI_STANDALONE_MACHINE_TESTING'
+      when: on_success
+    # Otherwise, set to manual (the jobs are created but not run).
+    - when: manual
+      allow_failure: true
+
+trigger-standalone-machines:
+  stage: containers-layer2
+  needs:
+    - job: build-standalone-machines
+    - job: amd64-fedora-container
+  trigger:
+    include:
+      - artifact: standalone-jobs-generated.yml
+        job: build-standalone-machines
+    strategy: depend
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 830c5b7af97..8122b0fab3a 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -35,6 +35,9 @@
 #
 # - QEMU_CI_AVOCADO_TESTING
 #   If set, tests using the Avocado framework will be run
+#
+# - QEMU_CI_STANDALONE_MACHINE_TESTING
+#   If set, jobs that build QEMU with a single machine will be added
 
 include:
   - local: '/.gitlab-ci.d/qemu-project.yml'
-- 
2.26.3

ping?

On 5/25/21 5:29 PM, Philippe Mathieu-Daudé wrote:
> The --without-default-devices configure option removes the
> 'default=y' from Kconfig files. It is useful to test missing
> Kconfig dependencies for users wanting to have QEMU (system)
> binaries with a particular subset of machines builtin.
> 
> If a machine can be built standalone, it can certainly be
> built as part of a set. So the best way to test for regressions
> is to test each machine individually.
> 
> As this is painful to test manually, add CI jobs to do it [*].
> Since all jobs follow the same template, to ease maintenance
> we generate the jobs using the jsonnet tool, which emit a YAML
> file filled with all our jobs.
> 
> Since there is no "--enable-my-config" option, we have to write
> the standalone config manually, overwritting each target .mak
> file in default-configs/devices/.
> 
> The job is set to manual (there is no change in the current
> set of jobs), but can be started automatically by setting the
> QEMU_CI_STANDALONE_MACHINE_TESTING variable.
> 
> [*] https://docs.gitlab.com/ee/ci/yaml/#trigger-child-pipeline-with-generated-configuration-file
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> Example: https://gitlab.com/philmd/qemu/-/pipelines/309272883
> 
> Based-on: <20210525082556.4011380-1-f4bug@amsat.org>
> ---
>  .gitlab-ci.d/qemu-project.yml             |  1 +
>  .gitlab-ci.d/standalone-jobs-template.yml | 32 ++++++++++
>  .gitlab-ci.d/standalone-jobs.jsonnet      | 74 +++++++++++++++++++++++
>  .gitlab-ci.d/standalone-jobs.yml          | 27 +++++++++
>  .gitlab-ci.yml                            |  3 +
>  5 files changed, 137 insertions(+)
>  create mode 100644 .gitlab-ci.d/standalone-jobs-template.yml
>  create mode 100644 .gitlab-ci.d/standalone-jobs.jsonnet
>  create mode 100644 .gitlab-ci.d/standalone-jobs.yml
> 
> diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
> index 64cb2ba1da5..4018c48761c 100644
> --- a/.gitlab-ci.d/qemu-project.yml
> +++ b/.gitlab-ci.d/qemu-project.yml
> @@ -9,3 +9,4 @@ include:
>    - local: '/.gitlab-ci.d/crossbuilds.yml'
>    - local: '/.gitlab-ci.d/buildtest.yml'
>    - local: '/.gitlab-ci.d/static_checks.yml'
> +  - local: '/.gitlab-ci.d/standalone-jobs.yml'
> diff --git a/.gitlab-ci.d/standalone-jobs-template.yml b/.gitlab-ci.d/standalone-jobs-template.yml
> new file mode 100644
> index 00000000000..23e60cb4069
> --- /dev/null
> +++ b/.gitlab-ci.d/standalone-jobs-template.yml
> @@ -0,0 +1,32 @@
> +.native_build_standalone_job_template:
> +  stage: build
> +  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> +  variables:
> +    IMAGE: fedora
> +    CONFIGURE_ARGS:
> +      --without-default-devices
> +      --disable-user
> +      --disable-tools
> +      --disable-docs
> +  before_script:
> +    - JOBS=$(expr $(nproc) + 1)
> +  script:
> +    - for target in $TARGETS ; do
> +        echo > default-configs/devices/${target}-softmmu.mak ;
> +        for line in $DEVICE_CONFIGS ; do
> +          echo "$line"
> +          echo "$line" > default-configs/devices/${target}-softmmu.mak ;
> +        done ;
> +      done
> +    - mkdir build
> +    - cd build
> +    - ../configure --enable-werror $CONFIGURE_ARGS --target-list="$TARGETS"
> +      || { cat config.log meson-logs/meson-log.txt && exit 1; }
> +    - QEMU_BINS=$(echo $(for target in $TARGETS; do echo qemu-system-${target//-softmmu/}; done))
> +    - make -j"$JOBS" $QEMU_BINS
> +    - for binary in $QEMU_BINS ; do
> +        for machine in $(./$binary -M help | egrep -v "^(Supported|none)" | awk '{print $1}') ; do
> +            echo "=== testing $machine with $binary ===" ;
> +            echo q | ./$binary -M $machine -nodefaults -display none $TESTING_ARGS -monitor stdio -S || exit 1 ;
> +        done ;
> +      done
> diff --git a/.gitlab-ci.d/standalone-jobs.jsonnet b/.gitlab-ci.d/standalone-jobs.jsonnet
> new file mode 100644
> index 00000000000..6f403a51091
> --- /dev/null
> +++ b/.gitlab-ci.d/standalone-jobs.jsonnet
> @@ -0,0 +1,74 @@
> +local param_job(targets, configs, cmdline="") =
> +  {
> +    extends: ".native_build_standalone_job_template",
> +
> +    variables: {
> +      TARGETS: targets,
> +      DEVICE_CONFIGS: configs,
> +      TESTING_ARGS: cmdline,
> +    }
> +  };
> +
> +{
> +  include: { "local": "/.gitlab-ci.d/standalone-jobs-template.yml" },
> +
> +  // alpha
> +    "alpha dp264": param_job("alpha-softmmu", "CONFIG_DP264=y"),
> +
> +  // arm
> +
> +  // avr
> +    "avr arduino": param_job("avr-softmmu", "CONFIG_ARDUINO=y"),
> +
> +  // cris
> +
> +  // hexagon
> +
> +  // hppa
> +    "hppa dino": param_job("hppa-softmmu", "CONFIG_DINO=y"),
> +
> +  // i386
> +    "x86 isapc": param_job("i386-softmmu", "CONFIG_ISAPC=y"),
> +    "x86 microvm": param_job("x86_64-softmmu", "CONFIG_MICROVM=y"),
> +
> +  // m68k
> +
> +  // microblaze
> +    "mb zynqmp-pmu": param_job("microblaze-softmmu", "CONFIG_XLNX_ZYNQMP_PMU=y"),
> +    "mb ml605": param_job("microblaze-softmmu", "CONFIG_PETALOGIX_ML605=y"),
> +    "mb s3adsp1800": param_job("microblaze-softmmu", "CONFIG_PETALOGIX_S3ADSP1800=y"),
> +
> +  // mips
> +    "mips jazz": param_job("mips64-softmmu", "CONFIG_FULOONG=y"),
> +    "mips loongson3v": param_job("mips64-softmmu", "CONFIG_LOONGSON3V=y"),
> +    "mips malta": param_job("mips-softmmu mipsel-softmmu", "CONFIG_MALTA=y", "-bios /dev/null"),
> +
> +  // nios2
> +    "nios2 10m50": param_job("nios2-softmmu", "CONFIG_NIOS2_10M50=y"),
> +    "nios2 nommu": param_job("nios2-softmmu", "CONFIG_NIOS2_GENERIC_NOMMU=y"),
> +
> +  // openrisc
> +    "or1k sim": param_job("or1k-softmmu", "CONFIG_OR1K_SIM=y"),
> +
> +  // ppc
> +
> +  // riscv
> +    "riscv opentitan": param_job("riscv32-softmmu", "CONFIG_OPENTITAN=y"),
> +
> +  // rx
> +    "rx gdbsim": param_job("rx-softmmu", "CONFIG_RX_GDBSIM=y", "-bios /dev/null"),
> +
> +  // s390x
> +    "s390 ccw": param_job("s390x-softmmu", "CONFIG_S390_CCW_VIRTIO=y"),
> +
> +  // sh4
> +
> +  // sparc
> +
> +  // tricore
> +    "triboard": param_job("tricore-softmmu", "CONFIG_TRIBOARD=y"),
> +
> +  // xtensa
> +    "xtensa sim": param_job("xtensaeb-softmmu", "CONFIG_XTENSA_SIM=y CONFIG_SEMIHOSTING=y"),
> +    "xtensa virt": param_job("xtensa-softmmu", "CONFIG_XTENSA_VIRT=y CONFIG_SEMIHOSTING=y"),
> +}
> diff --git a/.gitlab-ci.d/standalone-jobs.yml b/.gitlab-ci.d/standalone-jobs.yml
> new file mode 100644
> index 00000000000..b094173ff12
> --- /dev/null
> +++ b/.gitlab-ci.d/standalone-jobs.yml
> @@ -0,0 +1,27 @@
> +build-standalone-machines:
> +  stage: containers
> +  image: alpine:latest
> +  script:
> +    - apk add -U jsonnet
> +    - jsonnet .gitlab-ci.d/standalone-jobs.jsonnet > standalone-jobs-generated.yml
> +  artifacts:
> +    paths:
> +      - standalone-jobs-generated.yml
> +  rules:
> +    # Automatically add the jobs if QEMU_CI_STANDALONE_MACHINE_TESTING is set
> +    - if: '$QEMU_CI_STANDALONE_MACHINE_TESTING'
> +      when: on_success
> +    # Otherwise, set to manual (the jobs are created but not run).
> +    - when: manual
> +      allow_failure: true
> +
> +trigger-standalone-machines:
> +  stage: containers-layer2
> +  needs:
> +    - job: build-standalone-machines
> +    - job: amd64-fedora-container
> +  trigger:
> +    include:
> +      - artifact: standalone-jobs-generated.yml
> +        job: build-standalone-machines
> +    strategy: depend
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 830c5b7af97..8122b0fab3a 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -35,6 +35,9 @@
>  #
>  # - QEMU_CI_AVOCADO_TESTING
>  #   If set, tests using the Avocado framework will be run
> +#
> +# - QEMU_CI_STANDALONE_MACHINE_TESTING
> +#   If set, jobs that build QEMU with a single machine will be added
>  
>  include:
>    - local: '/.gitlab-ci.d/qemu-project.yml'
> 

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> The --without-default-devices configure option removes the
> 'default=y' from Kconfig files. It is useful to test missing
> Kconfig dependencies for users wanting to have QEMU (system)
> binaries with a particular subset of machines builtin.
>
> If a machine can be built standalone, it can certainly be
> built as part of a set. So the best way to test for regressions
> is to test each machine individually.
>
> As this is painful to test manually, add CI jobs to do it [*].
> Since all jobs follow the same template, to ease maintenance
> we generate the jobs using the jsonnet tool, which emit a YAML
> file filled with all our jobs.
>
> Since there is no "--enable-my-config" option, we have to write
> the standalone config manually, overwritting each target .mak
> file in default-configs/devices/.

Yet....

> The job is set to manual (there is no change in the current
> set of jobs), but can be started automatically by setting the
> QEMU_CI_STANDALONE_MACHINE_TESTING variable.

I'm wary of tests that don't get run automatically because it generally
means they won't be. However it also explodes the matrix with even more
builds so I can see the desire to avoid that.

>
> [*] https://docs.gitlab.com/ee/ci/yaml/#trigger-child-pipeline-with-generated-configuration-file
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> Example: https://gitlab.com/philmd/qemu/-/pipelines/309272883
>
> Based-on: <20210525082556.4011380-1-f4bug@amsat.org>
> ---
>  .gitlab-ci.d/qemu-project.yml             |  1 +
>  .gitlab-ci.d/standalone-jobs-template.yml | 32 ++++++++++
>  .gitlab-ci.d/standalone-jobs.jsonnet      | 74 +++++++++++++++++++++++
>  .gitlab-ci.d/standalone-jobs.yml          | 27 +++++++++
>  .gitlab-ci.yml                            |  3 +
>  5 files changed, 137 insertions(+)
>  create mode 100644 .gitlab-ci.d/standalone-jobs-template.yml
>  create mode 100644 .gitlab-ci.d/standalone-jobs.jsonnet
>  create mode 100644 .gitlab-ci.d/standalone-jobs.yml
>
> diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
> index 64cb2ba1da5..4018c48761c 100644
> --- a/.gitlab-ci.d/qemu-project.yml
> +++ b/.gitlab-ci.d/qemu-project.yml
> @@ -9,3 +9,4 @@ include:
>    - local: '/.gitlab-ci.d/crossbuilds.yml'
>    - local: '/.gitlab-ci.d/buildtest.yml'
>    - local: '/.gitlab-ci.d/static_checks.yml'
> +  - local: '/.gitlab-ci.d/standalone-jobs.yml'
> diff --git a/.gitlab-ci.d/standalone-jobs-template.yml b/.gitlab-ci.d/standalone-jobs-template.yml
> new file mode 100644
> index 00000000000..23e60cb4069
> --- /dev/null
> +++ b/.gitlab-ci.d/standalone-jobs-template.yml
> @@ -0,0 +1,32 @@
> +.native_build_standalone_job_template:
> +  stage: build
> +  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> +  variables:
> +    IMAGE: fedora
> +    CONFIGURE_ARGS:
> +      --without-default-devices
> +      --disable-user
> +      --disable-tools
> +      --disable-docs
> +  before_script:
> +    - JOBS=$(expr $(nproc) + 1)
> +  script:
> +    - for target in $TARGETS ; do
> +        echo > default-configs/devices/${target}-softmmu.mak ;
> +        for line in $DEVICE_CONFIGS ; do
> +          echo "$line"
> +          echo "$line" > default-configs/devices/${target}-softmmu.mak ;
> +        done ;
> +      done
> +    - mkdir build
> +    - cd build
> +    - ../configure --enable-werror $CONFIGURE_ARGS --target-list="$TARGETS"
> +      || { cat config.log meson-logs/meson-log.txt && exit 1; }
> +    - QEMU_BINS=$(echo $(for target in $TARGETS; do echo qemu-system-${target//-softmmu/}; done))
> +    - make -j"$JOBS" $QEMU_BINS
> +    - for binary in $QEMU_BINS ; do
> +        for machine in $(./$binary -M help | egrep -v "^(Supported|none)" | awk '{print $1}') ; do
> +            echo "=== testing $machine with $binary ===" ;
> +            echo q | ./$binary -M $machine -nodefaults -display none $TESTING_ARGS -monitor stdio -S || exit 1 ;
> +        done ;
> +      done
> diff --git a/.gitlab-ci.d/standalone-jobs.jsonnet b/.gitlab-ci.d/standalone-jobs.jsonnet
> new file mode 100644
> index 00000000000..6f403a51091
> --- /dev/null
> +++ b/.gitlab-ci.d/standalone-jobs.jsonnet
> @@ -0,0 +1,74 @@
> +local param_job(targets, configs, cmdline="") =
> +  {
> +    extends: ".native_build_standalone_job_template",
> +
> +    variables: {
> +      TARGETS: targets,
> +      DEVICE_CONFIGS: configs,
> +      TESTING_ARGS: cmdline,
> +    }
> +  };

I'm reluctant to have a parallel configuration architecture here. If we
are going to test these then let's get the configure support in first.

> +
> +{
> +  include: { "local": "/.gitlab-ci.d/standalone-jobs-template.yml" },
> +
> +    "alpha dp264": param_job("alpha-softmmu", "CONFIG_DP264=y"),
> +    "avr arduino": param_job("avr-softmmu", "CONFIG_ARDUINO=y"),
> +    "hppa dino": param_job("hppa-softmmu", "CONFIG_DINO=y"),
> +    "nios2 10m50": param_job("nios2-softmmu", "CONFIG_NIOS2_10M50=y"),
> +    "nios2 nommu": param_job("nios2-softmmu", "CONFIG_NIOS2_GENERIC_NOMMU=y"),
> +    "or1k sim": param_job("or1k-softmmu", "CONFIG_OR1K_SIM=y"),
> +    "rx gdbsim": param_job("rx-softmmu", "CONFIG_RX_GDBSIM=y", "-bios /dev/null"),
> +    "triboard": param_job("tricore-softmmu", "CONFIG_TRIBOARD=y"),
> +    "xtensa sim": param_job("xtensaeb-softmmu", "CONFIG_XTENSA_SIM=y CONFIG_SEMIHOSTING=y"),
> +    "xtensa virt": param_job("xtensa-softmmu", "CONFIG_XTENSA_VIRT=y CONFIG_SEMIHOSTING=y"),

Do we really have a plethora of users running trimmed down custom
configurations that we need to defend each of these exotic build
combinations in the CI?

> +
> +  // i386
> +    "x86 isapc": param_job("i386-softmmu", "CONFIG_ISAPC=y"),
> +    "x86 microvm": param_job("x86_64-softmmu", "CONFIG_MICROVM=y"),

> +  // s390x
> +    "s390 ccw": param_job("s390x-softmmu", "CONFIG_S390_CCW_VIRTIO=y"),

I think is possibly a valid use case. 

<snip>
> +  // microblaze
> +    "mb zynqmp-pmu": param_job("microblaze-softmmu", "CONFIG_XLNX_ZYNQMP_PMU=y"),
> +    "mb ml605": param_job("microblaze-softmmu", "CONFIG_PETALOGIX_ML605=y"),
> +    "mb s3adsp1800": param_job("microblaze-softmmu", "CONFIG_PETALOGIX_S3ADSP1800=y"),
> +
> +  // mips
> +    "mips jazz": param_job("mips64-softmmu", "CONFIG_FULOONG=y"),
> +    "mips loongson3v": param_job("mips64-softmmu", "CONFIG_LOONGSON3V=y"),
> +    "mips malta": param_job("mips-softmmu mipsel-softmmu", "CONFIG_MALTA=y", "-bios /dev/null"),
> +
> +  // riscv
> +    "riscv opentitan": param_job("riscv32-softmmu", "CONFIG_OPENTITAN=y"),

Again I'm not sure how many emulation targets really want to cut down to
a minimal configuration.

-- 
Alex Bennée

On 17/06/2021 12.42, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
>> The --without-default-devices configure option removes the
>> 'default=y' from Kconfig files. It is useful to test missing
>> Kconfig dependencies for users wanting to have QEMU (system)
>> binaries with a particular subset of machines builtin.
>>
>> If a machine can be built standalone, it can certainly be
>> built as part of a set. So the best way to test for regressions
>> is to test each machine individually.
>>
>> As this is painful to test manually, add CI jobs to do it [*].
>> Since all jobs follow the same template, to ease maintenance
>> we generate the jobs using the jsonnet tool, which emit a YAML
>> file filled with all our jobs.
>>
>> Since there is no "--enable-my-config" option, we have to write
>> the standalone config manually, overwritting each target .mak
>> file in default-configs/devices/.

I'd appreciate if we could get such a --enable-config option first - that 
would also be very helpful for downstream RHEL where we also modify the 
default-configs with downstream-only patches.

>> +
>> +{
>> +  include: { "local": "/.gitlab-ci.d/standalone-jobs-template.yml" },
>> +
>> +    "alpha dp264": param_job("alpha-softmmu", "CONFIG_DP264=y"),
>> +    "avr arduino": param_job("avr-softmmu", "CONFIG_ARDUINO=y"),
>> +    "hppa dino": param_job("hppa-softmmu", "CONFIG_DINO=y"),
>> +    "nios2 10m50": param_job("nios2-softmmu", "CONFIG_NIOS2_10M50=y"),
>> +    "nios2 nommu": param_job("nios2-softmmu", "CONFIG_NIOS2_GENERIC_NOMMU=y"),
>> +    "or1k sim": param_job("or1k-softmmu", "CONFIG_OR1K_SIM=y"),
>> +    "rx gdbsim": param_job("rx-softmmu", "CONFIG_RX_GDBSIM=y", "-bios /dev/null"),
>> +    "triboard": param_job("tricore-softmmu", "CONFIG_TRIBOARD=y"),
>> +    "xtensa sim": param_job("xtensaeb-softmmu", "CONFIG_XTENSA_SIM=y CONFIG_SEMIHOSTING=y"),
>> +    "xtensa virt": param_job("xtensa-softmmu", "CONFIG_XTENSA_VIRT=y CONFIG_SEMIHOSTING=y"),
> 
> Do we really have a plethora of users running trimmed down custom
> configurations that we need to defend each of these exotic build
> combinations in the CI?

I think I agree with Alex - in our CI, we should test what users really 
need, and not each and every distantly possible combination.

So what I'd really would like to see:

1) Introduce a "--with-build-configs" switch (feel free to bikeshed on the 
name) to the configure script that allows us to use a folder with a 
different set of config files.

2) rename default-configs/ to configs/default/

3) Introduce some useful alternate config sets, e.g. configs/rhel or 
configs/lean-kvm or whatever people want to use, and change some of the CI 
jobs to work with those configs.

  Thomas