From nobody Fri May 16 05:42:21 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1502415962733222.66598267852237; Thu, 10 Aug 2017 18:46:02 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 60DBC28A2AB; Fri, 11 Aug 2017 01:46:01 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 33CA85C6D7; Fri, 11 Aug 2017 01:46:01 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C521C4BB79; Fri, 11 Aug 2017 01:46:00 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v7B1gZOa003182 for ; Thu, 10 Aug 2017 21:42:35 -0400 Received: by smtp.corp.redhat.com (Postfix) id B9E7198145; Fri, 11 Aug 2017 01:42:35 +0000 (UTC) Received: from vhost2.laine.org (ovpn-117-36.phx2.redhat.com [10.3.117.36]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4683E9814B; Fri, 11 Aug 2017 01:42:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 60DBC28A2AB Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=laine.org Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 60DBC28A2AB From: Laine Stump To: libvir-list@redhat.com Date: Thu, 10 Aug 2017 21:42:22 -0400 Message-Id: <20170811014222.29548-8-laine@laine.org> In-Reply-To: <20170811014222.29548-1-laine@laine.org> References: <20170811014222.29548-1-laine@laine.org> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: mprivozn@redhat.com, moshele@mellanox.com Subject: [libvirt] [PATCH v2 7/7] util: check for PF online status earlier in guest startup X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 11 Aug 2017 01:46:01 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" When using a VF from an SRIOV-capable network card in a guest (either in macvtap passthrough mode, or via VFIO PCI device assignment), The associated PF netdev must be online in order for the VF to be usable by the guest. The guest, however, is not able to change the state of the PF. And libvirt *could* set the PF online as needed, but that could lead to the host receiving unexpected IPv6 traffic (since the default for an unconfigured interface is to participate in IPv6 autoconf). For this reason, before assigning a VF to a guest, libvirt verifies that the related PF netdev is online - if it isn't, then we log an error and don't allow the guest startup to continue. Until now, this check was done during virNetDevSetNetConfig(). This works nicely because the same function is called both for macvtap passthrough and for VFIO device assignment. But in the case of VFIO, the VF has already been unbound from its netdev driver by the time we get to virNetDevSetNetConfig(), and in the case of dual port Mellanox NICs that have their VFs setup in single port mode, the *only* way to determine the proper PF netdev to query for online status is via the "phys_port_id" file that is in the VF netdev's sysfs directory. *BUT* if we've unbound the VF from the netdev driver, then it doesn't *have* a netdev sysfs directory. So, in order to check the correct PF netdev for online status, this patch moved the check earlier in the setup, into virNetDevSaveNetConfig(), which is called *before* unbinding the VF from its netdev driver. (Note that this implies that if you are using VFIO device assignment for the VFs of a Mellanox NIC that has the VFs programmed in single port mode, you must let the VFs be bound to their net driver and use "managed=3D'yes'" in the device definition. To be more specific, this is only true if the VFs in single port mode are using port *2* of the PF - if the VFs are using only port 1, then the correct PF netdev will be arrived at by default/chance)) --- New in V2. src/util/virnetdev.c | 59 +++++++++++++++++++++++++++---------------------= ---- 1 file changed, 31 insertions(+), 28 deletions(-) diff --git a/src/util/virnetdev.c b/src/util/virnetdev.c index 8fc643c93..4ea6d5de2 100644 --- a/src/util/virnetdev.c +++ b/src/util/virnetdev.c @@ -1878,8 +1878,9 @@ virNetDevSaveNetConfig(const char *linkdev, int vf, goto cleanup; =20 linkdev =3D vfDevOrig; + saveVlan =3D true; =20 - } else if (saveVlan && virNetDevIsVirtualFunction(linkdev) =3D=3D 1) { + } else if (virNetDevIsVirtualFunction(linkdev) =3D=3D 1) { /* when vf is -1, linkdev might be a standard netdevice (not * SRIOV), or it might be an SRIOV VF. If it's a VF, normalize * it to PF + VFname @@ -1894,6 +1895,34 @@ virNetDevSaveNetConfig(const char *linkdev, int vf, goto cleanup; } =20 + if (pfDevName) { + bool pfIsOnline; + + /* Assure that PF is online before trying to use it to set + * anything up for this VF. It *should* be online already, + * but if it isn't online the changes made to the VF via the + * PF won't take effect, yet there will be no error + * reported. In the case that the PF isn't online, we need to + * fail and report the error, rather than automatically + * setting it online, since setting an unconfigured interface + * online automatically turns on IPv6 autoconfig, which may + * not be what the admin expects, so we require them to + * explicitly enable the PF in the host system network config. + */ + if (virNetDevGetOnline(pfDevName, &pfIsOnline) < 0) + goto cleanup; + + if (!pfIsOnline) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to configure VF %d of PF '%s' " + "because the PF is not online. Please " + "change host network config to put the " + "PF online."), + vf, pfDevName); + goto cleanup; + } + } + if (!(configJSON =3D virJSONValueNewObject())) goto cleanup; =20 @@ -1902,7 +1931,7 @@ virNetDevSaveNetConfig(const char *linkdev, int vf, * on the host) */ =20 - if (pfDevName) { + if (pfDevName && saveVlan) { if (virAsprintf(&filePath, "%s/%s_vf%d", stateDir, pfDevName, vf) = < 0) goto cleanup; =20 @@ -2251,32 +2280,6 @@ virNetDevSetNetConfig(const char *linkdev, int vf, } =20 } else { - bool pfIsOnline; - - /* Assure that PF is online before trying to use it to set - * anything up for this VF. It *should* be online already, - * but if it isn't online the changes made to the VF via the - * PF won't take effect, yet there will be no error - * reported. In the case that the PF isn't online, we need to - * fail and report the error, rather than automatically - * setting it online, since setting an unconfigured interface - * online automatically turns on IPv6 autoconfig, which may - * not be what the admin expects, so we require them to - * explicitly enable the PF in the host system network config. - */ - if (virNetDevGetOnline(pfDevName, &pfIsOnline) < 0) - goto cleanup; - - if (!pfIsOnline) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("Unable to configure VF %d of PF '%s' " - "because the PF is not online. Please " - "change host network config to put the " - "PF online."), - vf, pfDevName); - goto cleanup; - } - if (vlan) { if (vlan->nTags !=3D 1 || vlan->trunk) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", --=20 2.13.3 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list