From nobody Fri May 16 10:29:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1496419786981268.4417603074852; Fri, 2 Jun 2017 09:09:46 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D7CA340F33; Fri, 2 Jun 2017 16:09:44 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 519247ADCC; Fri, 2 Jun 2017 16:09:44 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9F2964A495; Fri, 2 Jun 2017 16:08:44 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v52G8PEB011653 for ; Fri, 2 Jun 2017 12:08:25 -0400 Received: by smtp.corp.redhat.com (Postfix) id B42248441E; Fri, 2 Jun 2017 16:08:25 +0000 (UTC) Received: from inaba.usersys.redhat.com (dhcp129-60.brq.redhat.com [10.34.129.60]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3B1C38443D for ; Fri, 2 Jun 2017 16:08:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D7CA340F33 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D7CA340F33 From: Andrea Bolognani To: libvir-list@redhat.com Date: Fri, 2 Jun 2017 18:07:48 +0200 Message-Id: <1496419671-7600-24-git-send-email-abologna@redhat.com> In-Reply-To: <1496419671-7600-1-git-send-email-abologna@redhat.com> References: <1496419671-7600-1-git-send-email-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 23/26] conf: Introduce isolation groups X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 02 Jun 2017 16:09:45 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Isolation groups will eventually allow us to make sure certain devices, eg. PCI hostdevs, are assigned to guest PCI buses in a way that guarantees improved isolation, error detection and recovery for machine types and hypervisors that support it, eg. pSeries guest on QEMU. --- src/bhyve/bhyve_device.c | 4 ++-- src/conf/device_conf.h | 4 ++++ src/conf/domain_addr.c | 33 +++++++++++++++++++++++++-------- src/conf/domain_addr.h | 7 ++++++- src/conf/domain_conf.h | 4 ++++ src/qemu/qemu_domain_address.c | 34 +++++++++++++++++----------------- 6 files changed, 58 insertions(+), 28 deletions(-) diff --git a/src/bhyve/bhyve_device.c b/src/bhyve/bhyve_device.c index fdfd512..03aa6c9 100644 --- a/src/bhyve/bhyve_device.c +++ b/src/bhyve/bhyve_device.c @@ -57,7 +57,7 @@ bhyveCollectPCIAddress(virDomainDefPtr def ATTRIBUTE_UNUS= ED, } =20 if (virDomainPCIAddressReserveAddr(addrs, addr, - VIR_PCI_CONNECT_TYPE_PCI_DEVICE) < = 0) { + VIR_PCI_CONNECT_TYPE_PCI_DEVICE, 0)= < 0) { goto cleanup; } =20 @@ -100,7 +100,7 @@ bhyveAssignDevicePCISlots(virDomainDefPtr def, lpc_addr.slot =3D 0x1; =20 if (virDomainPCIAddressReserveAddr(addrs, &lpc_addr, - VIR_PCI_CONNECT_TYPE_PCI_DEVICE) < = 0) { + VIR_PCI_CONNECT_TYPE_PCI_DEVICE, 0)= < 0) { goto error; } =20 diff --git a/src/conf/device_conf.h b/src/conf/device_conf.h index 58b2c9c..4b1377c 100644 --- a/src/conf/device_conf.h +++ b/src/conf/device_conf.h @@ -163,6 +163,10 @@ struct _virDomainDeviceInfo { * assignment, never saved and never reported. */ int pciConnectFlags; /* enum virDomainPCIConnectFlags */ + + /* PCI devices will only be automatically placed on a PCI bus + * that shares the same isolation group */ + int isolationGroup; }; =20 =20 diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index e6a6399..cd26b21 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -365,7 +365,8 @@ virDomainPCIAddressBusSetModel(virDomainPCIAddressBusPt= r bus, static int virDomainPCIAddressSetGrow(virDomainPCIAddressSetPtr addrs, virPCIDeviceAddressPtr addr, - virDomainPCIConnectFlags flags) + virDomainPCIConnectFlags flags, + int isolationGroup) { int add; size_t i; @@ -492,6 +493,9 @@ virDomainPCIAddressSetGrow(virDomainPCIAddressSetPtr ad= drs, for (; i < addrs->nbuses; i++) { if (virDomainPCIAddressBusSetModel(&addrs->buses[i], model) < 0) return -1; + + /* Set isolation group for the new bus */ + addrs->buses[i].isolationGroup =3D isolationGroup; } =20 return add; @@ -534,6 +538,7 @@ static int ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) virDomainPCIAddressReserveAddrInternal(virDomainPCIAddressSetPtr addrs, virPCIDeviceAddressPtr addr, virDomainPCIConnectFlags flags, + int isolationGroup, bool fromConfig) { int ret =3D -1; @@ -546,8 +551,10 @@ virDomainPCIAddressReserveAddrInternal(virDomainPCIAdd= ressSetPtr addrs, goto cleanup; =20 /* Add an extra bus if necessary */ - if (addrs->dryRun && virDomainPCIAddressSetGrow(addrs, addr, flags) < = 0) + if (addrs->dryRun && + virDomainPCIAddressSetGrow(addrs, addr, flags, isolationGroup) < 0= ) { goto cleanup; + } /* Check that the requested bus exists, is the correct type, and we * are asking for a valid slot */ @@ -586,9 +593,11 @@ virDomainPCIAddressReserveAddrInternal(virDomainPCIAdd= ressSetPtr addrs, int virDomainPCIAddressReserveAddr(virDomainPCIAddressSetPtr addrs, virPCIDeviceAddressPtr addr, - virDomainPCIConnectFlags flags) + virDomainPCIConnectFlags flags, + int isolationGroup) { - return virDomainPCIAddressReserveAddrInternal(addrs, addr, flags, true= ); + return virDomainPCIAddressReserveAddrInternal(addrs, addr, flags, + isolationGroup, true); } =20 int @@ -624,7 +633,8 @@ virDomainPCIAddressEnsureAddr(virDomainPCIAddressSetPtr= addrs, goto cleanup; =20 ret =3D virDomainPCIAddressReserveAddrInternal(addrs, &dev->addr.p= ci, - flags, true); + flags, dev->isolation= Group, + true); } else { ret =3D virDomainPCIAddressReserveNextAddr(addrs, dev, flags, -1); } @@ -745,6 +755,7 @@ static int ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) virDomainPCIAddressGetNextAddr(virDomainPCIAddressSetPtr addrs, virPCIDeviceAddressPtr next_addr, virDomainPCIConnectFlags flags, + int isolationGroup, int function) { virPCIDeviceAddress a =3D { 0 }; @@ -765,6 +776,10 @@ virDomainPCIAddressGetNextAddr(virDomainPCIAddressSetP= tr addrs, for (a.bus =3D 0; a.bus < addrs->nbuses; a.bus++) { bool found =3D false; =20 + /* Isolation groups for bus and device must match */ + if (addrs->buses[a.bus].isolationGroup !=3D isolationGroup) + continue; + a.slot =3D addrs->buses[a.bus].minSlot; =20 if (virDomainPCIAddressFindUnusedFunctionOnBus(&addrs->buses[a.bus= ], @@ -780,7 +795,7 @@ virDomainPCIAddressGetNextAddr(virDomainPCIAddressSetPt= r addrs, /* There were no free slots after the last used one */ if (addrs->dryRun) { /* a is already set to the first new bus */ - if (virDomainPCIAddressSetGrow(addrs, &a, flags) < 0) + if (virDomainPCIAddressSetGrow(addrs, &a, flags, isolationGroup) <= 0) goto error; /* this device will use the first slot of the new bus */ a.slot =3D addrs->buses[a.bus].minSlot; @@ -825,10 +840,12 @@ virDomainPCIAddressReserveNextAddr(virDomainPCIAddres= sSetPtr addrs, { virPCIDeviceAddress addr; =20 - if (virDomainPCIAddressGetNextAddr(addrs, &addr, flags, function) < 0) + if (virDomainPCIAddressGetNextAddr(addrs, &addr, flags, + dev->isolationGroup, function) < 0) return -1; =20 - if (virDomainPCIAddressReserveAddrInternal(addrs, &addr, flags, false)= < 0) + if (virDomainPCIAddressReserveAddrInternal(addrs, &addr, flags, + dev->isolationGroup, false)= < 0) return -1; =20 if (!addrs->dryRun) { diff --git a/src/conf/domain_addr.h b/src/conf/domain_addr.h index 1849d2b..23c7900 100644 --- a/src/conf/domain_addr.h +++ b/src/conf/domain_addr.h @@ -100,6 +100,10 @@ typedef struct { * bit is set, that function is in use by a device. */ virDomainPCIAddressSlot slot[VIR_PCI_ADDRESS_SLOT_LAST + 1]; + + /* PCI devices will only be automatically placed on a PCI bus + * that shares the same isolation group */ + int isolationGroup; } virDomainPCIAddressBus; typedef virDomainPCIAddressBus *virDomainPCIAddressBusPtr; =20 @@ -147,7 +151,8 @@ bool virDomainPCIAddressSlotInUse(virDomainPCIAddressSe= tPtr addrs, =20 int virDomainPCIAddressReserveAddr(virDomainPCIAddressSetPtr addrs, virPCIDeviceAddressPtr addr, - virDomainPCIConnectFlags flags) + virDomainPCIConnectFlags flags, + int isolationGroup) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); =20 int virDomainPCIAddressReserveNextAddr(virDomainPCIAddressSetPtr addrs, diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 244977a..507891a 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -786,6 +786,10 @@ struct _virDomainPCIControllerOpts { * item in memory target config) -1 =3D=3D unspecified */ int numaNode; + + /* PCI devices will only be automatically placed on a PCI bus + * that shares the same isolation group */ + int isolationGroup; }; =20 typedef struct _virDomainUSBControllerOpts virDomainUSBControllerOpts; diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 08773c0..384d6fd 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -1032,7 +1032,8 @@ qemuDomainCollectPCIAddress(virDomainDefPtr def ATTRI= BUTE_UNUSED, } =20 if (virDomainPCIAddressReserveAddr(addrs, addr, - info->pciConnectFlags) < 0) { + info->pciConnectFlags, + info->isolationGroup) < 0) { goto cleanup; } =20 @@ -1077,6 +1078,8 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, if (virDomainPCIAddressBusSetModel(&addrs->buses[idx], cont->model= ) < 0) goto error; =20 + addrs->buses[idx].isolationGroup =3D cont->opts.pciopts.isolationG= roup; + if (cont->model =3D=3D VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) hasPCIeRoot =3D true; } @@ -1193,7 +1196,7 @@ qemuDomainValidateDevicePCISlotsPIIX3(virDomainDefPtr= def, if (addrs->nbuses) { memset(&tmp_addr, 0, sizeof(tmp_addr)); tmp_addr.slot =3D 1; - if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags) < 0) + if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags, 0) < 0) goto cleanup; } =20 @@ -1228,7 +1231,7 @@ qemuDomainValidateDevicePCISlotsPIIX3(virDomainDefPtr= def, goto cleanup; } } else { - if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags= ) < 0) + if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags= , 0) < 0) goto cleanup; primaryVideo->info.addr.pci =3D tmp_addr; primaryVideo->info.type =3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE= _PCI; @@ -1253,7 +1256,7 @@ qemuDomainValidateDevicePCISlotsPIIX3(virDomainDefPtr= def, VIR_DEBUG("PCI address 0:0:2.0 in use, future addition of a vi= deo" " device will not be possible without manual" " intervention"); - } else if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags)= < 0) { + } else if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags,= 0) < 0) { goto cleanup; } } @@ -1329,10 +1332,8 @@ qemuDomainValidateDevicePCISlotsQ35(virDomainDefPtr = def, assign =3D true; } if (assign) { - if (virDomainPCIAddressReserveAddr(addrs, - &tmp_addr, flags) <= 0) { + if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, f= lags, 0) < 0) goto cleanup; - } =20 cont->info.type =3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI; cont->info.addr.pci.domain =3D 0; @@ -1354,10 +1355,8 @@ qemuDomainValidateDevicePCISlotsQ35(virDomainDefPtr = def, memset(&tmp_addr, 0, sizeof(tmp_addr)); tmp_addr.slot =3D 0x1E; if (!virDomainPCIAddressSlotInUse(addrs, &tmp_addr)) { - if (virDomainPCIAddressReserveAddr(addrs, - &tmp_addr, flags) <= 0) { + if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, f= lags, 0) < 0) goto cleanup; - } =20 cont->info.type =3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI; cont->info.addr.pci.domain =3D 0; @@ -1380,12 +1379,12 @@ qemuDomainValidateDevicePCISlotsQ35(virDomainDefPtr= def, tmp_addr.slot =3D 0x1F; tmp_addr.function =3D 0; tmp_addr.multi =3D VIR_TRISTATE_SWITCH_ON; - if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags) < 0) + if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags, 0) < 0) goto cleanup; =20 tmp_addr.function =3D 3; tmp_addr.multi =3D VIR_TRISTATE_SWITCH_ABSENT; - if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags) < 0) + if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags, 0) < 0) goto cleanup; } =20 @@ -1419,7 +1418,7 @@ qemuDomainValidateDevicePCISlotsQ35(virDomainDefPtr d= ef, goto cleanup; } } else { - if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags= ) < 0) + if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags= , 0) < 0) goto cleanup; primaryVideo->info.type =3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE= _PCI; primaryVideo->info.addr.pci =3D tmp_addr; @@ -1445,8 +1444,7 @@ qemuDomainValidateDevicePCISlotsQ35(virDomainDefPtr d= ef, " device will not be possible without manual" " intervention"); virResetLastError(); - } else if (virDomainPCIAddressReserveAddr(addrs, - &tmp_addr, flags) < 0) { + } else if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags,= 0) < 0) { goto cleanup; } } @@ -1467,7 +1465,7 @@ qemuDomainValidateDevicePCISlotsQ35(virDomainDefPtr d= ef, !virDeviceInfoPCIAddressWanted(&sound->info)) { continue; } - if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags) < = 0) + if (virDomainPCIAddressReserveAddr(addrs, &tmp_addr, flags, 0)= < 0) goto cleanup; =20 sound->info.type =3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI; @@ -1684,7 +1682,8 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, if (foundAddr) { /* Reserve this function on the slot we found */ if (virDomainPCIAddressReserveAddr(addrs, &addr, - cont->info.pciConnectFl= ags) < 0) { + cont->info.pciConnectFl= ags, + cont->info.isolationGro= up) < 0) { goto error; } =20 @@ -2150,6 +2149,7 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, } dev.type =3D VIR_DOMAIN_DEVICE_CONTROLLER; dev.data.controller =3D def->controllers[contIndex]; + dev.data.controller->opts.pciopts.isolationGroup =3D bus->isol= ationGroup; /* set connect flags so it will be properly addressed */ qemuDomainFillDevicePCIConnectFlags(def, &dev, qemuCaps, drive= r); =20 --=20 2.7.5 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list