From nobody Wed Apr 24 07:07:10 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 153978625500496.4841296297949; Wed, 17 Oct 2018 07:24:15 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 72FBF3082B04; Wed, 17 Oct 2018 14:24:12 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AA4C42C7B0; Wed, 17 Oct 2018 14:24:09 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 231B518005B5; Wed, 17 Oct 2018 14:24:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w9HEFRgv003240 for ; Wed, 17 Oct 2018 10:15:27 -0400 Received: by smtp.corp.redhat.com (Postfix) id 90A511057044; Wed, 17 Oct 2018 14:15:27 +0000 (UTC) Received: from domokun.gsslab.fab.redhat.com (dhcp-93.gsslab.fab.redhat.com [10.33.9.93]) by smtp.corp.redhat.com (Postfix) with ESMTP id 071C710018F8; Wed, 17 Oct 2018 14:15:26 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 17 Oct 2018 15:15:21 +0100 Message-Id: <20181017141522.284598-2-berrange@redhat.com> In-Reply-To: <20181017141522.284598-1-berrange@redhat.com> References: <20181017141522.284598-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 1/2] qemu: fix recording of vCPU pids for MTTCG X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Wed, 17 Oct 2018 14:24:13 +0000 (UTC) X-ZohoMail: RDMRC_0 RSF_0 Z_629925259 SPT_0 MTTCG is the new multi-threaded impl of TCG which follows KVM in having one host OS thread per vCPU. Historically we have discarded all PIDs reported for TCG guests, but we must now selectively honour this data. We don't have anything in the domain XML that indicates whether a guest is using TCG or MTTCG. While QEMU does have an option (-accel tcg,thread=3Dsingle|multi), it is not desirable to expose this in libvirt. QEMU will automatically use MTTCG when the host/guest architecture pairing is known to be safe. Only developers of QEMU TCG have a strong reason to override this logic. Thus we use two sanity checks to decide if the vCPU PID information is usable. First we see if the PID duplicates the main emulator PID, and second we see if the PID duplicates any other vCPUs. Signed-off-by: Daniel P. Berrang=C3=A9 --- src/qemu/qemu_domain.c | 81 ++++++++++++++++++++++++++---------------- 1 file changed, 51 insertions(+), 30 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index f00f1b3fdb..c7a0c03e3f 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -10326,9 +10326,10 @@ qemuDomainRefreshVcpuInfo(virQEMUDriverPtr driver, qemuDomainVcpuPrivatePtr vcpupriv; qemuMonitorCPUInfoPtr info =3D NULL; size_t maxvcpus =3D virDomainDefGetVcpusMax(vm->def); - size_t i; + size_t i, j; bool hotplug; bool fast; + bool validTIDs =3D true; int rc; int ret =3D -1; =20 @@ -10336,6 +10337,8 @@ qemuDomainRefreshVcpuInfo(virQEMUDriverPtr driver, fast =3D virQEMUCapsGet(QEMU_DOMAIN_PRIVATE(vm)->qemuCaps, QEMU_CAPS_QUERY_CPUS_FAST); =20 + VIR_DEBUG("Maxvcpus %zu hotplug %d fast query %d", maxvcpus, hotplug, = fast); + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0) return -1; =20 @@ -10348,39 +10351,57 @@ qemuDomainRefreshVcpuInfo(virQEMUDriverPtr driver, if (rc < 0) goto cleanup; =20 + /* + * The query-cpus[-fast] commands return information + * about the vCPUs, including the OS level PID that + * is executing the vCPU. + * + * For KVM there is always a 1-1 mapping between + * vCPUs and host OS PIDs. + * + * For TCG things are a little more complicated. + * + * - In some cases the vCPUs will all have the same + * PID as the main emulator thread. + * - In some cases the first vCPU will have a distinct + * PID, but other vCPUs will share the emulator thread + * + * For MTTCG, things work the same as KVM, with each + * vCPU getting its own PID. + * + * We use the Host OS PIDs for doing vCPU pinning + * and reporting. The TCG data reporting will result + * in bad behaviour such as pinning the wrong PID. + * We must thus detect and discard bogus PID info + * from TCG, while still honouring the modern MTTCG + * impl which we can support. + */ + for (i =3D 0; i < maxvcpus && validTIDs; i++) { + if (info[i].tid =3D=3D vm->pid) { + VIR_DEBUG("vCPU[%zu] PID %llu duplicates process", + i, (unsigned long long)info[i].tid); + validTIDs =3D false; + } + + for (j =3D 0; j < i; j++) { + if (info[i].tid =3D=3D info[j].tid) { + VIR_DEBUG("vCPU[%zu] PID %llu duplicates vCPU[%zu]", + i, (unsigned long long)info[i].tid, j); + validTIDs =3D false; + } + } + + if (validTIDs) + VIR_DEBUG("vCPU[%zu] PID %llu is valid", + i, (unsigned long long)info[i].tid); + } + + VIR_DEBUG("Extracting vCPU information validTIDs=3D%d", validTIDs); for (i =3D 0; i < maxvcpus; i++) { vcpu =3D virDomainDefGetVcpu(vm->def, i); vcpupriv =3D QEMU_DOMAIN_VCPU_PRIVATE(vcpu); =20 - /* - * Current QEMU *can* report info about host threads mapped - * to vCPUs, but it is not in a manner we can correctly - * deal with. The TCG CPU emulation does have a separate vCPU - * thread, but it runs every vCPU in that same thread. So it - * is impossible to setup different affinity per thread. - * - * What's more the 'query-cpus[-fast]' command returns bizarre - * data for the threads. It gives the TCG thread for the - * vCPU 0, but for vCPUs 1-> N, it actually replies with - * the main process thread ID. - * - * The result is that when we try to set affinity for - * vCPU 1, it will actually change the affinity of the - * emulator thread :-( When you try to set affinity for - * vCPUs 2, 3.... it will fail if the affinity was - * different from vCPU 1. - * - * We *could* allow vcpu pinning with TCG, if we made the - * restriction that all vCPUs had the same mask. This would - * at least let us separate emulator from vCPUs threads, as - * we do for KVM. It would need some changes to our cgroups - * CPU layout though, and error reporting for the config - * restrictions. - * - * Just disable CPU pinning with TCG until someone wants - * to try to do this hard work. - */ - if (vm->def->virtType !=3D VIR_DOMAIN_VIRT_QEMU) + if (validTIDs) vcpupriv->tid =3D info[i].tid; =20 vcpupriv->socket_id =3D info[i].socket_id; --=20 2.17.2 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Wed Apr 24 07:07:10 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 153978628104415.151905070801376; Wed, 17 Oct 2018 07:24:41 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1479231500BE; Wed, 17 Oct 2018 14:24:39 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B4AB35E255; Wed, 17 Oct 2018 14:24:38 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 4CC7319746; Wed, 17 Oct 2018 14:24:38 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w9HEFSIB003251 for ; Wed, 17 Oct 2018 10:15:28 -0400 Received: by smtp.corp.redhat.com (Postfix) id 6A25B1057044; Wed, 17 Oct 2018 14:15:28 +0000 (UTC) Received: from domokun.gsslab.fab.redhat.com (dhcp-93.gsslab.fab.redhat.com [10.33.9.93]) by smtp.corp.redhat.com (Postfix) with ESMTP id D2FB310018F8; Wed, 17 Oct 2018 14:15:27 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 17 Oct 2018 15:15:22 +0100 Message-Id: <20181017141522.284598-3-berrange@redhat.com> In-Reply-To: <20181017141522.284598-1-berrange@redhat.com> References: <20181017141522.284598-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 2/2] Revert "qemu: Forbid pinning vCPUs for TCG domain" X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Wed, 17 Oct 2018 14:24:39 +0000 (UTC) X-ZohoMail: RDMRC_0 RSF_0 Z_629925259 SPT_0 This reverts commit 8b035c84d8a7362a87a95e6114b8e7f959685ed9. The MTTCG impl in QEMU does allow pinning vCPUs. When the guest is running we already check if pinning is possible in the qemuDomainPinVcpuLive method, so this check was adding no benefit. When the guest is not running, we cannot know whether the subsequent launch will use MTTCG or TCG, so we must allow the pinning request. If the guest does use TCG on the next launch it will fail, but this is no worse than if the user had done a virDomainDefineXML with an XML doc specifying vCPU pinning. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: John Ferlan --- src/qemu/qemu_driver.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index a52e2495d5..981db01059 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5111,13 +5111,6 @@ qemuDomainPinVcpuFlags(virDomainPtr dom, if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) goto endjob; =20 - if ((def && def->virtType =3D=3D VIR_DOMAIN_VIRT_QEMU) || - (persistentDef && persistentDef->virtType =3D=3D VIR_DOMAIN_VIRT_Q= EMU)) { - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("Virt type 'qemu' does not support vCPU pinning")= ); - goto endjob; - } - if (persistentDef && !(vcpuinfo =3D virDomainDefGetVcpu(persistentDef, vcpu))) { virReportError(VIR_ERR_INVALID_ARG, --=20 2.17.2 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list