1. Patchew
  2. Libvirt
  3. [libvirt] [PATCH v3 00/12] qemu: Validate PCI controller options
  4. View patch

[libvirt] [PATCH v3 01/12] tests: Add some tests for PCI controller options

Andrea Bolognani posted 12 patches 7 years, 2 months ago
There is a newer version of this series
[libvirt] [PATCH v3 01/12] tests: Add some tests for PCI controller options
Posted by Andrea Bolognani 7 years, 2 months ago 
The input configurations set all existing options for all PCI
controllers, even those that are not valid for the controller.
As we implement validation for PCI controller options, we expect
these test to start failing.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
 .../i440fx-controllers-pciopts.args                | 24 +++++++
 .../i440fx-controllers-pciopts.xml                 | 36 ++++++++++
 .../pseries-controllers-pciopts.args               | 22 +++++++
 .../pseries-controllers-pciopts.xml                | 35 ++++++++++
 .../qemuxml2argvdata/q35-controllers-pciopts.args  | 28 ++++++++
 tests/qemuxml2argvdata/q35-controllers-pciopts.xml | 60 +++++++++++++++++
 tests/qemuxml2argvtest.c                           | 17 +++++
 .../i440fx-controllers-pciopts.xml                 | 45 +++++++++++++
 .../pseries-controllers-pciopts.xml                | 43 ++++++++++++
 .../qemuxml2xmloutdata/q35-controllers-pciopts.xml | 76 ++++++++++++++++++++++
 tests/qemuxml2xmltest.c                            | 17 +++++
 11 files changed, 403 insertions(+)
 create mode 100644 tests/qemuxml2argvdata/i440fx-controllers-pciopts.args
 create mode 100644 tests/qemuxml2argvdata/i440fx-controllers-pciopts.xml
 create mode 100644 tests/qemuxml2argvdata/pseries-controllers-pciopts.args
 create mode 100644 tests/qemuxml2argvdata/pseries-controllers-pciopts.xml
 create mode 100644 tests/qemuxml2argvdata/q35-controllers-pciopts.args
 create mode 100644 tests/qemuxml2argvdata/q35-controllers-pciopts.xml
 create mode 100644 tests/qemuxml2xmloutdata/i440fx-controllers-pciopts.xml
 create mode 100644 tests/qemuxml2xmloutdata/pseries-controllers-pciopts.xml
 create mode 100644 tests/qemuxml2xmloutdata/q35-controllers-pciopts.xml

diff --git a/tests/qemuxml2argvdata/i440fx-controllers-pciopts.args b/tests/qemuxml2argvdata/i440fx-controllers-pciopts.args
new file mode 100644
index 0000000000..d85fae5c96
--- /dev/null
+++ b/tests/qemuxml2argvdata/i440fx-controllers-pciopts.args
@@ -0,0 +1,24 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/home/test \
+USER=test \
+LOGNAME=test \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-x86_64 \
+-name guest \
+-S \
+-M pc \
+-m 1024 \
+-smp 1,sockets=1,cores=1,threads=1 \
+-numa node,nodeid=0,cpus=0,mem=1024 \
+-uuid 496d7ea8-9739-544b-4ebd-ef08be936e8b \
+-nographic \
+-nodefaults \
+-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-guest/monitor.sock,\
+server,nowait \
+-mon chardev=charmonitor,id=monitor,mode=readline \
+-no-acpi \
+-boot c \
+-global i440FX-pcihost.pci-hole64-size=1024K \
+-device pci-bridge,chassis_nr=2,id=pci.1,bus=pci.0,addr=0x3 \
+-device pxb,bus_nr=3,id=pci.2,numa_node=0,bus=pci.0,addr=0x4
diff --git a/tests/qemuxml2argvdata/i440fx-controllers-pciopts.xml b/tests/qemuxml2argvdata/i440fx-controllers-pciopts.xml
new file mode 100644
index 0000000000..06008b7338
--- /dev/null
+++ b/tests/qemuxml2argvdata/i440fx-controllers-pciopts.xml
@@ -0,0 +1,36 @@
+<domain type='qemu'>
+  <name>guest</name>
+  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
+  <memory unit='KiB'>1048576</memory>
+  <vcpu placement='static'>1</vcpu>
+  <os>
+    <type arch='x86_64' machine='pc'>hvm</type>
+  </os>
+  <cpu>
+    <numa>
+      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
+    </numa>
+  </cpu>
+  <devices>
+    <emulator>/usr/bin/qemu-system-x86_64</emulator>
+    <controller type='pci' index='0' model='pci-root'>
+      <!-- Only pcihole64 is valid -->
+      <target busNr='1' chassis='1' chassisNr='1' index='0' port='1'/>
+      <pcihole64 unit='KiB'>1024</pcihole64>
+    </controller>
+    <controller type='pci' index='1' model='pci-bridge'>
+      <!-- Only chassisNr is valid -->
+      <target busNr='2' chassis='2' chassisNr='2' index='2' port='2'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='pci' index='2' model='pci-expander-bus'>
+      <!-- Only busNr and numaNode are valid -->
+      <target busNr='3' chassis='3' chassisNr='3' index='3' port='3'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='usb' index='0' model='none'/>
+    <memballoon model='none'/>
+  </devices>
+</domain>
diff --git a/tests/qemuxml2argvdata/pseries-controllers-pciopts.args b/tests/qemuxml2argvdata/pseries-controllers-pciopts.args
new file mode 100644
index 0000000000..5f1edfc833
--- /dev/null
+++ b/tests/qemuxml2argvdata/pseries-controllers-pciopts.args
@@ -0,0 +1,22 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/home/test \
+USER=test \
+LOGNAME=test \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-ppc64 \
+-name guest \
+-S \
+-M pseries \
+-m 1024 \
+-smp 1,sockets=1,cores=1,threads=1 \
+-numa node,nodeid=0,cpus=0,mem=1024 \
+-uuid 496d7ea8-9739-544b-4ebd-ef08be936e8b \
+-nographic \
+-nodefaults \
+-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-guest/monitor.sock,\
+server,nowait \
+-mon chardev=charmonitor,id=monitor,mode=readline \
+-boot c \
+-device spapr-pci-host-bridge,index=1,id=pci.1,numa_node=0 \
+-device pci-bridge,chassis_nr=3,id=pci.2,bus=pci.0,addr=0x1
diff --git a/tests/qemuxml2argvdata/pseries-controllers-pciopts.xml b/tests/qemuxml2argvdata/pseries-controllers-pciopts.xml
new file mode 100644
index 0000000000..088c7159c3
--- /dev/null
+++ b/tests/qemuxml2argvdata/pseries-controllers-pciopts.xml
@@ -0,0 +1,35 @@
+<domain type='qemu'>
+  <name>guest</name>
+  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
+  <memory unit='KiB'>1048576</memory>
+  <vcpu placement='static'>1</vcpu>
+  <os>
+    <type arch='ppc64' machine='pseries'>hvm</type>
+  </os>
+  <cpu>
+    <numa>
+      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
+    </numa>
+  </cpu>
+  <devices>
+    <emulator>/usr/bin/qemu-system-ppc64</emulator>
+    <controller type='pci' index='0' model='pci-root'>
+      <!-- Only targetIndex is valid -->
+      <target busNr='1' chassis='1' chassisNr='1' index='0' port='1'/>
+    </controller>
+    <controller type='pci' index='1' model='pci-root'>
+      <!-- Only numaNode and targetIndex are valid -->
+      <target busNr='2' chassis='2' chassisNr='2' index='1' port='2'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='pci' index='2' model='pci-bridge'>
+      <!-- Only chassisNr is valid -->
+      <target busNr='3' chassis='3' chassisNr='3' index='3' port='3'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='usb' index='0' model='none'/>
+    <memballoon model='none'/>
+  </devices>
+</domain>
diff --git a/tests/qemuxml2argvdata/q35-controllers-pciopts.args b/tests/qemuxml2argvdata/q35-controllers-pciopts.args
new file mode 100644
index 0000000000..44259f5027
--- /dev/null
+++ b/tests/qemuxml2argvdata/q35-controllers-pciopts.args
@@ -0,0 +1,28 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/home/test \
+USER=test \
+LOGNAME=test \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-x86_64 \
+-name guest \
+-S \
+-M q35 \
+-m 1024 \
+-smp 1,sockets=1,cores=1,threads=1 \
+-numa node,nodeid=0,cpus=0,mem=1024 \
+-uuid 496d7ea8-9739-544b-4ebd-ef08be936e8b \
+-nographic \
+-nodefaults \
+-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-guest/monitor.sock,\
+server,nowait \
+-mon chardev=charmonitor,id=monitor,mode=readline \
+-no-acpi \
+-boot c \
+-global q35-pcihost.pci-hole64-size=1024K \
+-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \
+-device pci-bridge,chassis_nr=3,id=pci.2,bus=pci.1,addr=0x0 \
+-device pxb-pcie,bus_nr=4,id=pci.3,numa_node=0,bus=pcie.0,addr=0x2 \
+-device pcie-root-port,port=0x5,chassis=5,id=pci.4,bus=pcie.0,addr=0x3 \
+-device x3130-upstream,id=pci.5,bus=pci.4,addr=0x0 \
+-device xio3130-downstream,port=0x7,chassis=7,id=pci.6,bus=pci.5,addr=0x0
diff --git a/tests/qemuxml2argvdata/q35-controllers-pciopts.xml b/tests/qemuxml2argvdata/q35-controllers-pciopts.xml
new file mode 100644
index 0000000000..f5b8d1e355
--- /dev/null
+++ b/tests/qemuxml2argvdata/q35-controllers-pciopts.xml
@@ -0,0 +1,60 @@
+<domain type='qemu'>
+  <name>guest</name>
+  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
+  <memory unit='KiB'>1048576</memory>
+  <vcpu placement='static'>1</vcpu>
+  <os>
+    <type arch='x86_64' machine='q35'>hvm</type>
+  </os>
+  <cpu>
+    <numa>
+      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
+    </numa>
+  </cpu>
+  <devices>
+    <emulator>/usr/bin/qemu-system-x86_64</emulator>
+    <controller type='pci' index='0' model='pcie-root'>
+      <!-- Only pcihole64 is valid -->
+      <target busNr='1' chassis='1' chassisNr='1' index='0' port='1'/>
+      <pcihole64 unit='KiB'>1024</pcihole64>
+    </controller>
+    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
+      <!-- No option is valid -->
+      <target busNr='2' chassis='2' chassisNr='2' index='2' port='2'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='pci' index='2' model='pci-bridge'>
+      <!-- Only chassisNr is valid -->
+      <target busNr='3' chassis='3' chassisNr='3' index='3' port='3'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='pci' index='3' model='pcie-expander-bus'>
+      <!-- Only busNr and numaNode are valid -->
+      <target busNr='4' chassis='4' chassisNr='4' index='4' port='4'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='pci' index='4' model='pcie-root-port'>
+      <!-- Only chassis and port are valid -->
+      <target busNr='5' chassis='5' chassisNr='5' index='5' port='5'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='pci' index='5' model='pcie-switch-upstream-port'>
+      <!-- No option is valid -->
+      <target busNr='6' chassis='6' chassisNr='6' index='6' port='6'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='pci' index='6' model='pcie-switch-downstream-port'>
+      <!-- Only chassis and port are valid -->
+      <target busNr='7' chassis='7' chassisNr='7' index='7' port='7'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='usb' index='0' model='none'/>
+    <memballoon model='none'/>
+  </devices>
+</domain>
diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c
index 688846b9b4..9015eb5fb3 100644
--- a/tests/qemuxml2argvtest.c
+++ b/tests/qemuxml2argvtest.c
@@ -2436,6 +2436,23 @@ mymain(void)
                         QEMU_CAPS_DEVICE_IOH3420,
                         QEMU_CAPS_DEVICE_PXB_PCIE);
 
+    DO_TEST("i440fx-controllers-pciopts",
+            QEMU_CAPS_I440FX_PCI_HOLE64_SIZE,
+            QEMU_CAPS_DEVICE_PCI_BRIDGE,
+            QEMU_CAPS_DEVICE_PXB);
+    DO_TEST("q35-controllers-pciopts",
+            QEMU_CAPS_Q35_PCI_HOLE64_SIZE,
+            QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE,
+            QEMU_CAPS_DEVICE_PCI_BRIDGE,
+            QEMU_CAPS_DEVICE_PXB_PCIE,
+            QEMU_CAPS_DEVICE_PCIE_ROOT_PORT,
+            QEMU_CAPS_DEVICE_X3130_UPSTREAM,
+            QEMU_CAPS_DEVICE_XIO3130_DOWNSTREAM);
+    DO_TEST("pseries-controllers-pciopts",
+            QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE,
+            QEMU_CAPS_SPAPR_PCI_HOST_BRIDGE_NUMA_NODE,
+            QEMU_CAPS_DEVICE_PCI_BRIDGE);
+
     DO_TEST("hostdev-scsi-lsi",
             QEMU_CAPS_VIRTIO_SCSI, QEMU_CAPS_SCSI_LSI,
             QEMU_CAPS_DEVICE_SCSI_GENERIC);
diff --git a/tests/qemuxml2xmloutdata/i440fx-controllers-pciopts.xml b/tests/qemuxml2xmloutdata/i440fx-controllers-pciopts.xml
new file mode 100644
index 0000000000..d171d13705
--- /dev/null
+++ b/tests/qemuxml2xmloutdata/i440fx-controllers-pciopts.xml
@@ -0,0 +1,45 @@
+<domain type='qemu'>
+  <name>guest</name>
+  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
+  <memory unit='KiB'>1048576</memory>
+  <currentMemory unit='KiB'>1048576</currentMemory>
+  <vcpu placement='static'>1</vcpu>
+  <os>
+    <type arch='x86_64' machine='pc'>hvm</type>
+    <boot dev='hd'/>
+  </os>
+  <cpu>
+    <numa>
+      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
+    </numa>
+  </cpu>
+  <clock offset='utc'/>
+  <on_poweroff>destroy</on_poweroff>
+  <on_reboot>restart</on_reboot>
+  <on_crash>destroy</on_crash>
+  <devices>
+    <emulator>/usr/bin/qemu-system-x86_64</emulator>
+    <controller type='pci' index='0' model='pci-root'>
+      <target chassisNr='1' chassis='1' port='0x1' busNr='1' index='0'/>
+      <pcihole64 unit='KiB'>1024</pcihole64>
+    </controller>
+    <controller type='pci' index='1' model='pci-bridge'>
+      <model name='pci-bridge'/>
+      <target chassisNr='2' chassis='2' port='0x2' busNr='2' index='2'>
+        <node>0</node>
+      </target>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pci-expander-bus'>
+      <model name='pxb'/>
+      <target chassisNr='3' chassis='3' port='0x3' busNr='3' index='3'>
+        <node>0</node>
+      </target>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
+    </controller>
+    <controller type='usb' index='0' model='none'/>
+    <input type='mouse' bus='ps2'/>
+    <input type='keyboard' bus='ps2'/>
+    <memballoon model='none'/>
+  </devices>
+</domain>
diff --git a/tests/qemuxml2xmloutdata/pseries-controllers-pciopts.xml b/tests/qemuxml2xmloutdata/pseries-controllers-pciopts.xml
new file mode 100644
index 0000000000..bbe360e25d
--- /dev/null
+++ b/tests/qemuxml2xmloutdata/pseries-controllers-pciopts.xml
@@ -0,0 +1,43 @@
+<domain type='qemu'>
+  <name>guest</name>
+  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
+  <memory unit='KiB'>1048576</memory>
+  <currentMemory unit='KiB'>1048576</currentMemory>
+  <vcpu placement='static'>1</vcpu>
+  <os>
+    <type arch='ppc64' machine='pseries'>hvm</type>
+    <boot dev='hd'/>
+  </os>
+  <cpu>
+    <numa>
+      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
+    </numa>
+  </cpu>
+  <clock offset='utc'/>
+  <on_poweroff>destroy</on_poweroff>
+  <on_reboot>restart</on_reboot>
+  <on_crash>destroy</on_crash>
+  <devices>
+    <emulator>/usr/bin/qemu-system-ppc64</emulator>
+    <controller type='pci' index='0' model='pci-root'>
+      <model name='spapr-pci-host-bridge'/>
+      <target chassisNr='1' chassis='1' port='0x1' busNr='1' index='0'/>
+    </controller>
+    <controller type='pci' index='1' model='pci-root'>
+      <model name='spapr-pci-host-bridge'/>
+      <target chassisNr='2' chassis='2' port='0x2' busNr='2' index='1'>
+        <node>0</node>
+      </target>
+    </controller>
+    <controller type='pci' index='2' model='pci-bridge'>
+      <model name='pci-bridge'/>
+      <target chassisNr='3' chassis='3' port='0x3' busNr='3' index='3'>
+        <node>0</node>
+      </target>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='usb' index='0' model='none'/>
+    <memballoon model='none'/>
+    <panic model='pseries'/>
+  </devices>
+</domain>
diff --git a/tests/qemuxml2xmloutdata/q35-controllers-pciopts.xml b/tests/qemuxml2xmloutdata/q35-controllers-pciopts.xml
new file mode 100644
index 0000000000..5ef7aa564c
--- /dev/null
+++ b/tests/qemuxml2xmloutdata/q35-controllers-pciopts.xml
@@ -0,0 +1,76 @@
+<domain type='qemu'>
+  <name>guest</name>
+  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
+  <memory unit='KiB'>1048576</memory>
+  <currentMemory unit='KiB'>1048576</currentMemory>
+  <vcpu placement='static'>1</vcpu>
+  <os>
+    <type arch='x86_64' machine='q35'>hvm</type>
+    <boot dev='hd'/>
+  </os>
+  <cpu>
+    <numa>
+      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
+    </numa>
+  </cpu>
+  <clock offset='utc'/>
+  <on_poweroff>destroy</on_poweroff>
+  <on_reboot>restart</on_reboot>
+  <on_crash>destroy</on_crash>
+  <devices>
+    <emulator>/usr/bin/qemu-system-x86_64</emulator>
+    <controller type='pci' index='0' model='pcie-root'>
+      <target chassisNr='1' chassis='1' port='0x1' busNr='1' index='0'/>
+      <pcihole64 unit='KiB'>1024</pcihole64>
+    </controller>
+    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
+      <model name='i82801b11-bridge'/>
+      <target chassisNr='2' chassis='2' port='0x2' busNr='2' index='2'>
+        <node>0</node>
+      </target>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pci-bridge'>
+      <model name='pci-bridge'/>
+      <target chassisNr='3' chassis='3' port='0x3' busNr='3' index='3'>
+        <node>0</node>
+      </target>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
+    <controller type='pci' index='3' model='pcie-expander-bus'>
+      <model name='pxb-pcie'/>
+      <target chassisNr='4' chassis='4' port='0x4' busNr='4' index='4'>
+        <node>0</node>
+      </target>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
+    </controller>
+    <controller type='pci' index='4' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassisNr='5' chassis='5' port='0x5' busNr='5' index='5'>
+        <node>0</node>
+      </target>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
+    </controller>
+    <controller type='pci' index='5' model='pcie-switch-upstream-port'>
+      <model name='x3130-upstream'/>
+      <target chassisNr='6' chassis='6' port='0x6' busNr='6' index='6'>
+        <node>0</node>
+      </target>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
+    </controller>
+    <controller type='pci' index='6' model='pcie-switch-downstream-port'>
+      <model name='xio3130-downstream'/>
+      <target chassisNr='7' chassis='7' port='0x7' busNr='7' index='7'>
+        <node>0</node>
+      </target>
+      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
+    </controller>
+    <controller type='usb' index='0' model='none'/>
+    <controller type='sata' index='0'>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
+    </controller>
+    <input type='mouse' bus='ps2'/>
+    <input type='keyboard' bus='ps2'/>
+    <memballoon model='none'/>
+  </devices>
+</domain>
diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c
index 0eb9e6c77a..7b8a160788 100644
--- a/tests/qemuxml2xmltest.c
+++ b/tests/qemuxml2xmltest.c
@@ -1102,6 +1102,23 @@ mymain(void)
             QEMU_CAPS_DEVICE_IOH3420,
             QEMU_CAPS_HDA_DUPLEX);
 
+    DO_TEST("i440fx-controllers-pciopts",
+            QEMU_CAPS_I440FX_PCI_HOLE64_SIZE,
+            QEMU_CAPS_DEVICE_PCI_BRIDGE,
+            QEMU_CAPS_DEVICE_PXB);
+    DO_TEST("q35-controllers-pciopts",
+            QEMU_CAPS_Q35_PCI_HOLE64_SIZE,
+            QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE,
+            QEMU_CAPS_DEVICE_PCI_BRIDGE,
+            QEMU_CAPS_DEVICE_PXB_PCIE,
+            QEMU_CAPS_DEVICE_PCIE_ROOT_PORT,
+            QEMU_CAPS_DEVICE_X3130_UPSTREAM,
+            QEMU_CAPS_DEVICE_XIO3130_DOWNSTREAM);
+    DO_TEST("pseries-controllers-pciopts",
+            QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE,
+            QEMU_CAPS_SPAPR_PCI_HOST_BRIDGE_NUMA_NODE,
+            QEMU_CAPS_DEVICE_PCI_BRIDGE);
+
     DO_TEST("hostdev-scsi-vhost-scsi-ccw",
             QEMU_CAPS_VIRTIO_SCSI, QEMU_CAPS_DEVICE_VHOST_SCSI,
             QEMU_CAPS_DEVICE_SCSI_GENERIC, QEMU_CAPS_VIRTIO_CCW);
-- 
2.14.3

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3 01/12] tests: Add some tests for PCI controller options
Posted by Laine Stump 7 years, 2 months ago 
On 02/21/2018 09:14 AM, Andrea Bolognani wrote:
> The input configurations set all existing options for all PCI
> controllers, even those that are not valid for the controller.
> As we implement validation for PCI controller options, we expect
> these test to start failing.

A noble cause, but since multiple options are being tested for multiple
controllers in the same file, once you have all the proper checks in
place the tests won't actually be verifying all of the negative tests -
only the first failure will be noticed - if one of the others is missed,
it won't "fail extra hard" or anything.

Although I hate to explode the number of tests, I think if you want to
have proper negative testing for every options that doesn't belong on a
particular controller, then you'll need a separate test case for each
combination of option and controller model. And since that would make
for a *lot* of test cases if we tried to extrapolate to all other
options for all other elements, I don't know that it's worth going down
that rabbit hole.

(What we really need is a more intelligent test rig that would allow
specifying a single test with a list of variations/diffs and a note of
whether the test should succeed or fail with that difference. It would
end up taking the same amount of time to run the tests, but at least the
amount of xml would be drastically reduced.)


>
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
>  .../i440fx-controllers-pciopts.args                | 24 +++++++
>  .../i440fx-controllers-pciopts.xml                 | 36 ++++++++++
>  .../pseries-controllers-pciopts.args               | 22 +++++++
>  .../pseries-controllers-pciopts.xml                | 35 ++++++++++
>  .../qemuxml2argvdata/q35-controllers-pciopts.args  | 28 ++++++++
>  tests/qemuxml2argvdata/q35-controllers-pciopts.xml | 60 +++++++++++++++++
>  tests/qemuxml2argvtest.c                           | 17 +++++
>  .../i440fx-controllers-pciopts.xml                 | 45 +++++++++++++
>  .../pseries-controllers-pciopts.xml                | 43 ++++++++++++
>  .../qemuxml2xmloutdata/q35-controllers-pciopts.xml | 76 ++++++++++++++++++++++
>  tests/qemuxml2xmltest.c                            | 17 +++++
>  11 files changed, 403 insertions(+)
>  create mode 100644 tests/qemuxml2argvdata/i440fx-controllers-pciopts.args
>  create mode 100644 tests/qemuxml2argvdata/i440fx-controllers-pciopts.xml
>  create mode 100644 tests/qemuxml2argvdata/pseries-controllers-pciopts.args
>  create mode 100644 tests/qemuxml2argvdata/pseries-controllers-pciopts.xml
>  create mode 100644 tests/qemuxml2argvdata/q35-controllers-pciopts.args
>  create mode 100644 tests/qemuxml2argvdata/q35-controllers-pciopts.xml
>  create mode 100644 tests/qemuxml2xmloutdata/i440fx-controllers-pciopts.xml
>  create mode 100644 tests/qemuxml2xmloutdata/pseries-controllers-pciopts.xml
>  create mode 100644 tests/qemuxml2xmloutdata/q35-controllers-pciopts.xml
>
> diff --git a/tests/qemuxml2argvdata/i440fx-controllers-pciopts.args b/tests/qemuxml2argvdata/i440fx-controllers-pciopts.args
> new file mode 100644
> index 0000000000..d85fae5c96
> --- /dev/null
> +++ b/tests/qemuxml2argvdata/i440fx-controllers-pciopts.args
> @@ -0,0 +1,24 @@
> +LC_ALL=C \
> +PATH=/bin \
> +HOME=/home/test \
> +USER=test \
> +LOGNAME=test \
> +QEMU_AUDIO_DRV=none \
> +/usr/bin/qemu-system-x86_64 \
> +-name guest \
> +-S \
> +-M pc \
> +-m 1024 \
> +-smp 1,sockets=1,cores=1,threads=1 \
> +-numa node,nodeid=0,cpus=0,mem=1024 \
> +-uuid 496d7ea8-9739-544b-4ebd-ef08be936e8b \
> +-nographic \
> +-nodefaults \
> +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-guest/monitor.sock,\
> +server,nowait \
> +-mon chardev=charmonitor,id=monitor,mode=readline \
> +-no-acpi \
> +-boot c \
> +-global i440FX-pcihost.pci-hole64-size=1024K \
> +-device pci-bridge,chassis_nr=2,id=pci.1,bus=pci.0,addr=0x3 \
> +-device pxb,bus_nr=3,id=pci.2,numa_node=0,bus=pci.0,addr=0x4
> diff --git a/tests/qemuxml2argvdata/i440fx-controllers-pciopts.xml b/tests/qemuxml2argvdata/i440fx-controllers-pciopts.xml
> new file mode 100644
> index 0000000000..06008b7338
> --- /dev/null
> +++ b/tests/qemuxml2argvdata/i440fx-controllers-pciopts.xml
> @@ -0,0 +1,36 @@
> +<domain type='qemu'>
> +  <name>guest</name>
> +  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
> +  <memory unit='KiB'>1048576</memory>
> +  <vcpu placement='static'>1</vcpu>
> +  <os>
> +    <type arch='x86_64' machine='pc'>hvm</type>
> +  </os>
> +  <cpu>
> +    <numa>
> +      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
> +    </numa>
> +  </cpu>
> +  <devices>
> +    <emulator>/usr/bin/qemu-system-x86_64</emulator>
> +    <controller type='pci' index='0' model='pci-root'>
> +      <!-- Only pcihole64 is valid -->
> +      <target busNr='1' chassis='1' chassisNr='1' index='0' port='1'/>
> +      <pcihole64 unit='KiB'>1024</pcihole64>
> +    </controller>
> +    <controller type='pci' index='1' model='pci-bridge'>
> +      <!-- Only chassisNr is valid -->
> +      <target busNr='2' chassis='2' chassisNr='2' index='2' port='2'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='pci' index='2' model='pci-expander-bus'>
> +      <!-- Only busNr and numaNode are valid -->
> +      <target busNr='3' chassis='3' chassisNr='3' index='3' port='3'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='usb' index='0' model='none'/>
> +    <memballoon model='none'/>
> +  </devices>
> +</domain>
> diff --git a/tests/qemuxml2argvdata/pseries-controllers-pciopts.args b/tests/qemuxml2argvdata/pseries-controllers-pciopts.args
> new file mode 100644
> index 0000000000..5f1edfc833
> --- /dev/null
> +++ b/tests/qemuxml2argvdata/pseries-controllers-pciopts.args
> @@ -0,0 +1,22 @@
> +LC_ALL=C \
> +PATH=/bin \
> +HOME=/home/test \
> +USER=test \
> +LOGNAME=test \
> +QEMU_AUDIO_DRV=none \
> +/usr/bin/qemu-system-ppc64 \
> +-name guest \
> +-S \
> +-M pseries \
> +-m 1024 \
> +-smp 1,sockets=1,cores=1,threads=1 \
> +-numa node,nodeid=0,cpus=0,mem=1024 \
> +-uuid 496d7ea8-9739-544b-4ebd-ef08be936e8b \
> +-nographic \
> +-nodefaults \
> +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-guest/monitor.sock,\
> +server,nowait \
> +-mon chardev=charmonitor,id=monitor,mode=readline \
> +-boot c \
> +-device spapr-pci-host-bridge,index=1,id=pci.1,numa_node=0 \
> +-device pci-bridge,chassis_nr=3,id=pci.2,bus=pci.0,addr=0x1
> diff --git a/tests/qemuxml2argvdata/pseries-controllers-pciopts.xml b/tests/qemuxml2argvdata/pseries-controllers-pciopts.xml
> new file mode 100644
> index 0000000000..088c7159c3
> --- /dev/null
> +++ b/tests/qemuxml2argvdata/pseries-controllers-pciopts.xml
> @@ -0,0 +1,35 @@
> +<domain type='qemu'>
> +  <name>guest</name>
> +  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
> +  <memory unit='KiB'>1048576</memory>
> +  <vcpu placement='static'>1</vcpu>
> +  <os>
> +    <type arch='ppc64' machine='pseries'>hvm</type>
> +  </os>
> +  <cpu>
> +    <numa>
> +      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
> +    </numa>
> +  </cpu>
> +  <devices>
> +    <emulator>/usr/bin/qemu-system-ppc64</emulator>
> +    <controller type='pci' index='0' model='pci-root'>
> +      <!-- Only targetIndex is valid -->
> +      <target busNr='1' chassis='1' chassisNr='1' index='0' port='1'/>
> +    </controller>
> +    <controller type='pci' index='1' model='pci-root'>
> +      <!-- Only numaNode and targetIndex are valid -->
> +      <target busNr='2' chassis='2' chassisNr='2' index='1' port='2'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='pci' index='2' model='pci-bridge'>
> +      <!-- Only chassisNr is valid -->
> +      <target busNr='3' chassis='3' chassisNr='3' index='3' port='3'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='usb' index='0' model='none'/>
> +    <memballoon model='none'/>
> +  </devices>
> +</domain>
> diff --git a/tests/qemuxml2argvdata/q35-controllers-pciopts.args b/tests/qemuxml2argvdata/q35-controllers-pciopts.args
> new file mode 100644
> index 0000000000..44259f5027
> --- /dev/null
> +++ b/tests/qemuxml2argvdata/q35-controllers-pciopts.args
> @@ -0,0 +1,28 @@
> +LC_ALL=C \
> +PATH=/bin \
> +HOME=/home/test \
> +USER=test \
> +LOGNAME=test \
> +QEMU_AUDIO_DRV=none \
> +/usr/bin/qemu-system-x86_64 \
> +-name guest \
> +-S \
> +-M q35 \
> +-m 1024 \
> +-smp 1,sockets=1,cores=1,threads=1 \
> +-numa node,nodeid=0,cpus=0,mem=1024 \
> +-uuid 496d7ea8-9739-544b-4ebd-ef08be936e8b \
> +-nographic \
> +-nodefaults \
> +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-guest/monitor.sock,\
> +server,nowait \
> +-mon chardev=charmonitor,id=monitor,mode=readline \
> +-no-acpi \
> +-boot c \
> +-global q35-pcihost.pci-hole64-size=1024K \
> +-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \
> +-device pci-bridge,chassis_nr=3,id=pci.2,bus=pci.1,addr=0x0 \
> +-device pxb-pcie,bus_nr=4,id=pci.3,numa_node=0,bus=pcie.0,addr=0x2 \
> +-device pcie-root-port,port=0x5,chassis=5,id=pci.4,bus=pcie.0,addr=0x3 \
> +-device x3130-upstream,id=pci.5,bus=pci.4,addr=0x0 \
> +-device xio3130-downstream,port=0x7,chassis=7,id=pci.6,bus=pci.5,addr=0x0
> diff --git a/tests/qemuxml2argvdata/q35-controllers-pciopts.xml b/tests/qemuxml2argvdata/q35-controllers-pciopts.xml
> new file mode 100644
> index 0000000000..f5b8d1e355
> --- /dev/null
> +++ b/tests/qemuxml2argvdata/q35-controllers-pciopts.xml
> @@ -0,0 +1,60 @@
> +<domain type='qemu'>
> +  <name>guest</name>
> +  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
> +  <memory unit='KiB'>1048576</memory>
> +  <vcpu placement='static'>1</vcpu>
> +  <os>
> +    <type arch='x86_64' machine='q35'>hvm</type>
> +  </os>
> +  <cpu>
> +    <numa>
> +      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
> +    </numa>
> +  </cpu>
> +  <devices>
> +    <emulator>/usr/bin/qemu-system-x86_64</emulator>
> +    <controller type='pci' index='0' model='pcie-root'>
> +      <!-- Only pcihole64 is valid -->
> +      <target busNr='1' chassis='1' chassisNr='1' index='0' port='1'/>
> +      <pcihole64 unit='KiB'>1024</pcihole64>
> +    </controller>
> +    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
> +      <!-- No option is valid -->
> +      <target busNr='2' chassis='2' chassisNr='2' index='2' port='2'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='pci' index='2' model='pci-bridge'>
> +      <!-- Only chassisNr is valid -->
> +      <target busNr='3' chassis='3' chassisNr='3' index='3' port='3'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='pci' index='3' model='pcie-expander-bus'>
> +      <!-- Only busNr and numaNode are valid -->
> +      <target busNr='4' chassis='4' chassisNr='4' index='4' port='4'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='pci' index='4' model='pcie-root-port'>
> +      <!-- Only chassis and port are valid -->
> +      <target busNr='5' chassis='5' chassisNr='5' index='5' port='5'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='pci' index='5' model='pcie-switch-upstream-port'>
> +      <!-- No option is valid -->
> +      <target busNr='6' chassis='6' chassisNr='6' index='6' port='6'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='pci' index='6' model='pcie-switch-downstream-port'>
> +      <!-- Only chassis and port are valid -->
> +      <target busNr='7' chassis='7' chassisNr='7' index='7' port='7'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='usb' index='0' model='none'/>
> +    <memballoon model='none'/>
> +  </devices>
> +</domain>
> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c
> index 688846b9b4..9015eb5fb3 100644
> --- a/tests/qemuxml2argvtest.c
> +++ b/tests/qemuxml2argvtest.c
> @@ -2436,6 +2436,23 @@ mymain(void)
>                          QEMU_CAPS_DEVICE_IOH3420,
>                          QEMU_CAPS_DEVICE_PXB_PCIE);
>  
> +    DO_TEST("i440fx-controllers-pciopts",
> +            QEMU_CAPS_I440FX_PCI_HOLE64_SIZE,
> +            QEMU_CAPS_DEVICE_PCI_BRIDGE,
> +            QEMU_CAPS_DEVICE_PXB);
> +    DO_TEST("q35-controllers-pciopts",
> +            QEMU_CAPS_Q35_PCI_HOLE64_SIZE,
> +            QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE,
> +            QEMU_CAPS_DEVICE_PCI_BRIDGE,
> +            QEMU_CAPS_DEVICE_PXB_PCIE,
> +            QEMU_CAPS_DEVICE_PCIE_ROOT_PORT,
> +            QEMU_CAPS_DEVICE_X3130_UPSTREAM,
> +            QEMU_CAPS_DEVICE_XIO3130_DOWNSTREAM);
> +    DO_TEST("pseries-controllers-pciopts",
> +            QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE,
> +            QEMU_CAPS_SPAPR_PCI_HOST_BRIDGE_NUMA_NODE,
> +            QEMU_CAPS_DEVICE_PCI_BRIDGE);
> +
>      DO_TEST("hostdev-scsi-lsi",
>              QEMU_CAPS_VIRTIO_SCSI, QEMU_CAPS_SCSI_LSI,
>              QEMU_CAPS_DEVICE_SCSI_GENERIC);
> diff --git a/tests/qemuxml2xmloutdata/i440fx-controllers-pciopts.xml b/tests/qemuxml2xmloutdata/i440fx-controllers-pciopts.xml
> new file mode 100644
> index 0000000000..d171d13705
> --- /dev/null
> +++ b/tests/qemuxml2xmloutdata/i440fx-controllers-pciopts.xml
> @@ -0,0 +1,45 @@
> +<domain type='qemu'>
> +  <name>guest</name>
> +  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
> +  <memory unit='KiB'>1048576</memory>
> +  <currentMemory unit='KiB'>1048576</currentMemory>
> +  <vcpu placement='static'>1</vcpu>
> +  <os>
> +    <type arch='x86_64' machine='pc'>hvm</type>
> +    <boot dev='hd'/>
> +  </os>
> +  <cpu>
> +    <numa>
> +      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
> +    </numa>
> +  </cpu>
> +  <clock offset='utc'/>
> +  <on_poweroff>destroy</on_poweroff>
> +  <on_reboot>restart</on_reboot>
> +  <on_crash>destroy</on_crash>
> +  <devices>
> +    <emulator>/usr/bin/qemu-system-x86_64</emulator>
> +    <controller type='pci' index='0' model='pci-root'>
> +      <target chassisNr='1' chassis='1' port='0x1' busNr='1' index='0'/>
> +      <pcihole64 unit='KiB'>1024</pcihole64>
> +    </controller>
> +    <controller type='pci' index='1' model='pci-bridge'>
> +      <model name='pci-bridge'/>
> +      <target chassisNr='2' chassis='2' port='0x2' busNr='2' index='2'>
> +        <node>0</node>
> +      </target>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
> +    </controller>
> +    <controller type='pci' index='2' model='pci-expander-bus'>
> +      <model name='pxb'/>
> +      <target chassisNr='3' chassis='3' port='0x3' busNr='3' index='3'>
> +        <node>0</node>
> +      </target>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
> +    </controller>
> +    <controller type='usb' index='0' model='none'/>
> +    <input type='mouse' bus='ps2'/>
> +    <input type='keyboard' bus='ps2'/>
> +    <memballoon model='none'/>
> +  </devices>
> +</domain>
> diff --git a/tests/qemuxml2xmloutdata/pseries-controllers-pciopts.xml b/tests/qemuxml2xmloutdata/pseries-controllers-pciopts.xml
> new file mode 100644
> index 0000000000..bbe360e25d
> --- /dev/null
> +++ b/tests/qemuxml2xmloutdata/pseries-controllers-pciopts.xml
> @@ -0,0 +1,43 @@
> +<domain type='qemu'>
> +  <name>guest</name>
> +  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
> +  <memory unit='KiB'>1048576</memory>
> +  <currentMemory unit='KiB'>1048576</currentMemory>
> +  <vcpu placement='static'>1</vcpu>
> +  <os>
> +    <type arch='ppc64' machine='pseries'>hvm</type>
> +    <boot dev='hd'/>
> +  </os>
> +  <cpu>
> +    <numa>
> +      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
> +    </numa>
> +  </cpu>
> +  <clock offset='utc'/>
> +  <on_poweroff>destroy</on_poweroff>
> +  <on_reboot>restart</on_reboot>
> +  <on_crash>destroy</on_crash>
> +  <devices>
> +    <emulator>/usr/bin/qemu-system-ppc64</emulator>
> +    <controller type='pci' index='0' model='pci-root'>
> +      <model name='spapr-pci-host-bridge'/>
> +      <target chassisNr='1' chassis='1' port='0x1' busNr='1' index='0'/>
> +    </controller>
> +    <controller type='pci' index='1' model='pci-root'>
> +      <model name='spapr-pci-host-bridge'/>
> +      <target chassisNr='2' chassis='2' port='0x2' busNr='2' index='1'>
> +        <node>0</node>
> +      </target>
> +    </controller>
> +    <controller type='pci' index='2' model='pci-bridge'>
> +      <model name='pci-bridge'/>
> +      <target chassisNr='3' chassis='3' port='0x3' busNr='3' index='3'>
> +        <node>0</node>
> +      </target>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
> +    </controller>
> +    <controller type='usb' index='0' model='none'/>
> +    <memballoon model='none'/>
> +    <panic model='pseries'/>
> +  </devices>
> +</domain>
> diff --git a/tests/qemuxml2xmloutdata/q35-controllers-pciopts.xml b/tests/qemuxml2xmloutdata/q35-controllers-pciopts.xml
> new file mode 100644
> index 0000000000..5ef7aa564c
> --- /dev/null
> +++ b/tests/qemuxml2xmloutdata/q35-controllers-pciopts.xml
> @@ -0,0 +1,76 @@
> +<domain type='qemu'>
> +  <name>guest</name>
> +  <uuid>496d7ea8-9739-544b-4ebd-ef08be936e8b</uuid>
> +  <memory unit='KiB'>1048576</memory>
> +  <currentMemory unit='KiB'>1048576</currentMemory>
> +  <vcpu placement='static'>1</vcpu>
> +  <os>
> +    <type arch='x86_64' machine='q35'>hvm</type>
> +    <boot dev='hd'/>
> +  </os>
> +  <cpu>
> +    <numa>
> +      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
> +    </numa>
> +  </cpu>
> +  <clock offset='utc'/>
> +  <on_poweroff>destroy</on_poweroff>
> +  <on_reboot>restart</on_reboot>
> +  <on_crash>destroy</on_crash>
> +  <devices>
> +    <emulator>/usr/bin/qemu-system-x86_64</emulator>
> +    <controller type='pci' index='0' model='pcie-root'>
> +      <target chassisNr='1' chassis='1' port='0x1' busNr='1' index='0'/>
> +      <pcihole64 unit='KiB'>1024</pcihole64>
> +    </controller>
> +    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
> +      <model name='i82801b11-bridge'/>
> +      <target chassisNr='2' chassis='2' port='0x2' busNr='2' index='2'>
> +        <node>0</node>
> +      </target>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
> +    </controller>
> +    <controller type='pci' index='2' model='pci-bridge'>
> +      <model name='pci-bridge'/>
> +      <target chassisNr='3' chassis='3' port='0x3' busNr='3' index='3'>
> +        <node>0</node>
> +      </target>
> +      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
> +    </controller>
> +    <controller type='pci' index='3' model='pcie-expander-bus'>
> +      <model name='pxb-pcie'/>
> +      <target chassisNr='4' chassis='4' port='0x4' busNr='4' index='4'>
> +        <node>0</node>
> +      </target>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
> +    </controller>
> +    <controller type='pci' index='4' model='pcie-root-port'>
> +      <model name='pcie-root-port'/>
> +      <target chassisNr='5' chassis='5' port='0x5' busNr='5' index='5'>
> +        <node>0</node>
> +      </target>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
> +    </controller>
> +    <controller type='pci' index='5' model='pcie-switch-upstream-port'>
> +      <model name='x3130-upstream'/>
> +      <target chassisNr='6' chassis='6' port='0x6' busNr='6' index='6'>
> +        <node>0</node>
> +      </target>
> +      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
> +    </controller>
> +    <controller type='pci' index='6' model='pcie-switch-downstream-port'>
> +      <model name='xio3130-downstream'/>
> +      <target chassisNr='7' chassis='7' port='0x7' busNr='7' index='7'>
> +        <node>0</node>
> +      </target>
> +      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
> +    </controller>
> +    <controller type='usb' index='0' model='none'/>
> +    <controller type='sata' index='0'>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
> +    </controller>
> +    <input type='mouse' bus='ps2'/>
> +    <input type='keyboard' bus='ps2'/>
> +    <memballoon model='none'/>
> +  </devices>
> +</domain>
> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c
> index 0eb9e6c77a..7b8a160788 100644
> --- a/tests/qemuxml2xmltest.c
> +++ b/tests/qemuxml2xmltest.c
> @@ -1102,6 +1102,23 @@ mymain(void)
>              QEMU_CAPS_DEVICE_IOH3420,
>              QEMU_CAPS_HDA_DUPLEX);
>  
> +    DO_TEST("i440fx-controllers-pciopts",
> +            QEMU_CAPS_I440FX_PCI_HOLE64_SIZE,
> +            QEMU_CAPS_DEVICE_PCI_BRIDGE,
> +            QEMU_CAPS_DEVICE_PXB);
> +    DO_TEST("q35-controllers-pciopts",
> +            QEMU_CAPS_Q35_PCI_HOLE64_SIZE,
> +            QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE,
> +            QEMU_CAPS_DEVICE_PCI_BRIDGE,
> +            QEMU_CAPS_DEVICE_PXB_PCIE,
> +            QEMU_CAPS_DEVICE_PCIE_ROOT_PORT,
> +            QEMU_CAPS_DEVICE_X3130_UPSTREAM,
> +            QEMU_CAPS_DEVICE_XIO3130_DOWNSTREAM);
> +    DO_TEST("pseries-controllers-pciopts",
> +            QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE,
> +            QEMU_CAPS_SPAPR_PCI_HOST_BRIDGE_NUMA_NODE,
> +            QEMU_CAPS_DEVICE_PCI_BRIDGE);
> +
>      DO_TEST("hostdev-scsi-vhost-scsi-ccw",
>              QEMU_CAPS_VIRTIO_SCSI, QEMU_CAPS_DEVICE_VHOST_SCSI,
>              QEMU_CAPS_DEVICE_SCSI_GENERIC, QEMU_CAPS_VIRTIO_CCW);


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3 01/12] tests: Add some tests for PCI controller options
Posted by Andrea Bolognani 7 years, 2 months ago 
On Fri, 2018-02-23 at 14:18 -0500, Laine Stump wrote:
> On 02/21/2018 09:14 AM, Andrea Bolognani wrote:
> > The input configurations set all existing options for all PCI
> > controllers, even those that are not valid for the controller.
> > As we implement validation for PCI controller options, we expect
> > these test to start failing.
> 
> A noble cause, but since multiple options are being tested for multiple
> controllers in the same file, once you have all the proper checks in
> place the tests won't actually be verifying all of the negative tests -
> only the first failure will be noticed - if one of the others is missed,
> it won't "fail extra hard" or anything.

I'm well aware of that.

> Although I hate to explode the number of tests, I think if you want to
> have proper negative testing for every options that doesn't belong on a
> particular controller, then you'll need a separate test case for each
> combination of option and controller model. And since that would make
> for a *lot* of test cases if we tried to extrapolate to all other
> options for all other elements, I don't know that it's worth going down
> that rabbit hole.

So should I just drop this one, or is it still somewhat valuable
to have any sort of test suite coverage for PCI controller options?

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3 01/12] tests: Add some tests for PCI controller options
Posted by Laine Stump 7 years, 2 months ago 
On 02/27/2018 07:17 AM, Andrea Bolognani wrote:
> On Fri, 2018-02-23 at 14:18 -0500, Laine Stump wrote:
>> On 02/21/2018 09:14 AM, Andrea Bolognani wrote:
>>> The input configurations set all existing options for all PCI
>>> controllers, even those that are not valid for the controller.
>>> As we implement validation for PCI controller options, we expect
>>> these test to start failing.
>> A noble cause, but since multiple options are being tested for multiple
>> controllers in the same file, once you have all the proper checks in
>> place the tests won't actually be verifying all of the negative tests -
>> only the first failure will be noticed - if one of the others is missed,
>> it won't "fail extra hard" or anything.
> I'm well aware of that.

Yeah, I'm just trying to be funny.

>
>> Although I hate to explode the number of tests, I think if you want to
>> have proper negative testing for every options that doesn't belong on a
>> particular controller, then you'll need a separate test case for each
>> combination of option and controller model. And since that would make
>> for a *lot* of test cases if we tried to extrapolate to all other
>> options for all other elements, I don't know that it's worth going down
>> that rabbit hole.
> So should I just drop this one, or is it still somewhat valuable
> to have any sort of test suite coverage for PCI controller options?
>

I was thinking that having this test is better than not having this
test. But then I thought about what would happen  if there was a
regression in just a single one of these validations - the negative test
would still "succeed" (i.e. "succeed in detecting a failure") because it
would hit the next disallowed attribute. As a matter of fact, the test
would continue to "succeed" until there was a regression in the
validation of every single attribute, so that no error would be
triggered. So having a negative test that has multiple examples of
failures actually gives us a false sense of security - we believe that
it's verifying we're catching incorrect config, but it won't actually
notify us until *all* of the bad config is missed by validation.


So, I think each negative test should have exactly one piece of
incorrect data. That necessarily means that it's only testing for proper
validation of a single aspect of a single attribute. But making this
generally useful with the current test apparatus would mean a huge
explosion in the number of test files, and I don't think that's
practical. But if we're only testing for one out of a thousand
validations, there's really not much point in it.

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3 01/12] tests: Add some tests for PCI controller options
Posted by Andrea Bolognani 7 years, 2 months ago 
On Tue, 2018-02-27 at 09:59 -0500, Laine Stump wrote:
> > So should I just drop this one, or is it still somewhat valuable
> > to have any sort of test suite coverage for PCI controller options?
> 
> I was thinking that having this test is better than not having this
> test. But then I thought about what would happen  if there was a
> regression in just a single one of these validations - the negative test
> would still "succeed" (i.e. "succeed in detecting a failure") because it
> would hit the next disallowed attribute. As a matter of fact, the test
> would continue to "succeed" until there was a regression in the
> validation of every single attribute, so that no error would be
> triggered. So having a negative test that has multiple examples of
> failures actually gives us a false sense of security - we believe that
> it's verifying we're catching incorrect config, but it won't actually
> notify us until *all* of the bad config is missed by validation.
> 
> 
> So, I think each negative test should have exactly one piece of
> incorrect data. That necessarily means that it's only testing for proper
> validation of a single aspect of a single attribute. But making this
> generally useful with the current test apparatus would mean a huge
> explosion in the number of test files, and I don't think that's
> practical. But if we're only testing for one out of a thousand
> validations, there's really not much point in it.

Okay, I'll drop it and respin then.

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3 01/12] tests: Add some tests for PCI controller options
Posted by Laine Stump 7 years, 2 months ago 
On 02/21/2018 09:14 AM, Andrea Bolognani wrote:
> The input configurations set all existing options for all PCI
> controllers, even those that are not valid for the controller.
> As we implement validation for PCI controller options, we expect
> these test to start failing.

Oh, and s/test/tests/.


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Patchew 2.1-devel-8c64711

© 2016 - 2025 Red Hat, Inc.