From nobody Wed May 14 05:54:48 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1525465814680270.33963475893916; Fri, 4 May 2018 13:30:14 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D179A883C7; Fri, 4 May 2018 20:30:12 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 98A15271BD; Fri, 4 May 2018 20:30:12 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 528364CAB2; Fri, 4 May 2018 20:30:12 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w44KM0hp013789 for ; Fri, 4 May 2018 16:22:00 -0400 Received: by smtp.corp.redhat.com (Postfix) id 05C81601AE; Fri, 4 May 2018 20:22:00 +0000 (UTC) Received: from mx1.redhat.com (ext-mx04.extmail.prod.ext.phx2.redhat.com [10.5.110.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F20AE601A6 for ; Fri, 4 May 2018 20:21:59 +0000 (UTC) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A7F1585545 for ; Fri, 4 May 2018 20:21:58 +0000 (UTC) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w44KFlKH125787 for ; Fri, 4 May 2018 16:21:58 -0400 Received: from e35.co.us.ibm.com (e35.co.us.ibm.com [32.97.110.153]) by mx0b-001b2d01.pphosted.com with ESMTP id 2hrs664vk8-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 04 May 2018 16:21:57 -0400 Received: from localhost by e35.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 4 May 2018 14:21:57 -0600 Received: from b03cxnp08026.gho.boulder.ibm.com (9.17.130.18) by e35.co.us.ibm.com (192.168.1.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 4 May 2018 14:21:54 -0600 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w44KLruY8651254; Fri, 4 May 2018 13:21:53 -0700 Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8FF51136043; Fri, 4 May 2018 14:21:53 -0600 (MDT) Received: from sbct-3.watson.ibm.com (unknown [9.47.158.153]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTP id 391F013603C; Fri, 4 May 2018 14:21:53 -0600 (MDT) From: Stefan Berger To: libvir-list@redhat.com Date: Fri, 4 May 2018 16:21:25 -0400 In-Reply-To: <1525465285-14102-1-git-send-email-stefanb@linux.vnet.ibm.com> References: <1525465285-14102-1-git-send-email-stefanb@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18050420-0012-0000-0000-000016292ACA X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008970; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000258; SDB=6.01027522; UDB=6.00524878; IPR=6.00806655; MB=3.00020932; MTD=3.00000008; XFM=3.00000015; UTC=2018-05-04 20:21:55 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18050420-0013-0000-0000-00005296C840 Message-Id: <1525465285-14102-15-git-send-email-stefanb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-04_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805040185 X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 04 May 2018 20:21:58 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 04 May 2018 20:21:58 +0000 (UTC) for IP:'148.163.158.5' DOMAIN:'mx0b-001b2d01.pphosted.com' HELO:'mx0a-001b2d01.pphosted.com' FROM:'stefanb@linux.vnet.ibm.com' RCPT:'' X-RedHat-Spam-Score: -0.7 (RCVD_IN_DNSWL_LOW) 148.163.158.5 mx0b-001b2d01.pphosted.com 148.163.158.5 mx0b-001b2d01.pphosted.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.28 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 14/14] qemu: Add swtpm to emulator cgroup X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 04 May 2018 20:30:14 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Add the external swtpm to the emulator cgroup so that upper limits of CPU usage can be enforced on the emulated TPM. To enable this we need to have the swtpm write its process id (pid) into a file. We then read it from the file to configure the emulator cgroup. The PID file is created in /var/run/libvirt/qemu/swtpm: [root@localhost swtpm]# ls -lZ /var/run/libvirt/qemu/swtpm/ total 4 -rw-r--r--. 1 tss tss system_u:object_r:qemu_var_run_t:s0 5 Apr = 10 12:26 1-testvm-swtpm.pid srw-rw----. 1 qemu qemu system_u:object_r:svirt_image_t:s0:c597,c632 0 Apr = 10 12:26 1-testvm-swtpm.sock The swtpm command line now looks as follows: root@localhost testvm]# ps auxZ | grep swtpm | grep socket | grep -v grep system_u:system_r:virtd_t:s0:c597,c632 tss 18697 0.0 0.0 28172 3892 ? = Ss 16:46 0:00 /usr/bin/swtpm socket --daemon --ctrl type=3Dunixio,pat= h=3D/var/run/libvirt/qemu/swtpm/1-testvm-swtpm.sock,mode=3D0600 --tpmstate = dir=3D/var/lib/libvirt/swtpm/485d0004-a48f-436a-8457-8a3b73e28568/tpm1.2/ -= -log file=3D/var/log/swtpm/libvirt/qemu/testvm-swtpm.log --pid file=3D/var/= run/libvirt/qemu/swtpm/1-testvm-swtpm.pid Signed-off-by: Stefan Berger --- src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/qemu/qemu_cgroup.c | 53 +++++++++++++++++++++++++++++++++++++++++++= ++++ src/qemu/qemu_cgroup.h | 1 + src/qemu/qemu_extdevice.c | 19 +++++++++++++++++ src/qemu/qemu_process.c | 4 ++++ src/util/virtpm.c | 33 +++++++++++++++++++++++++++++ 7 files changed, 112 insertions(+) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index c98d26a..f542c8e 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2624,6 +2624,7 @@ void virDomainTPMDefFree(virDomainTPMDefPtr def) VIR_FREE(def->data.emulator.source.data.nix.path); VIR_FREE(def->data.emulator.storagepath); VIR_FREE(def->data.emulator.logfile); + VIR_FREE(def->data.emulator.pidfile); break; case VIR_DOMAIN_TPM_TYPE_LAST: break; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 826ff26..49c77f7 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1311,6 +1311,7 @@ struct _virDomainTPMDef { virDomainChrSourceDef source; char *storagepath; char *logfile; + char *pidfile; } emulator; } data; }; diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index 1a5adca..f86ac3f 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -38,6 +38,7 @@ #include "virnuma.h" #include "virsystemd.h" #include "virdevmapper.h" +#include "virpidfile.h" =20 #define VIR_FROM_THIS VIR_FROM_QEMU =20 @@ -1146,6 +1147,58 @@ qemuSetupCgroupCpusetCpus(virCgroupPtr cgroup, =20 =20 int +qemuSetupCgroupForExtDevices(virDomainObjPtr vm) +{ + qemuDomainObjPrivatePtr priv =3D vm->privateData; + virDomainTPMDefPtr tpm =3D vm->def->tpm; + virCgroupPtr cgroup_temp =3D NULL; + pid_t pid; + int ret =3D -1; + + if (priv->cgroup =3D=3D NULL) + return 0; /* Not supported, so claim success */ + + /* + * If CPU cgroup controller is not initialized here, then we need + * neither period nor quota settings. And if CPUSET controller is + * not initialized either, then there's nothing to do anyway. + */ + if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU) && + !virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPUSET= )) + return 0; + + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_EMULATOR, 0, + false, &cgroup_temp) < 0) + goto cleanup; + + if (tpm) { + switch (tpm->type) { + case VIR_DOMAIN_TPM_TYPE_EMULATOR: + if (virPidFileReadPath(tpm->data.emulator.pidfile, &pid) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Could not read swtpm's pidfile %s"), + tpm->data.emulator.pidfile); + goto cleanup; + } + if (virCgroupAddTask(cgroup_temp, pid) < 0) + goto cleanup; + break; + case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH: + case VIR_DOMAIN_TPM_TYPE_LAST: + break; + } + } + + ret =3D 0; + +cleanup: + virCgroupFree(&cgroup_temp); + + return ret; +} + + +int qemuSetupGlobalCpuCgroup(virDomainObjPtr vm) { qemuDomainObjPrivatePtr priv =3D vm->privateData; diff --git a/src/qemu/qemu_cgroup.h b/src/qemu/qemu_cgroup.h index 3b8ff60..478bf7e 100644 --- a/src/qemu/qemu_cgroup.h +++ b/src/qemu/qemu_cgroup.h @@ -69,6 +69,7 @@ int qemuSetupCgroupVcpuBW(virCgroupPtr cgroup, long long quota); int qemuSetupCgroupCpusetCpus(virCgroupPtr cgroup, virBitmapPtr cpumask); int qemuSetupGlobalCpuCgroup(virDomainObjPtr vm); +int qemuSetupCgroupForExtDevices(virDomainObjPtr vm); int qemuRemoveCgroup(virDomainObjPtr vm); =20 typedef struct _qemuCgroupEmulatorAllNodesData qemuCgroupEmulatorAllNodesD= ata; diff --git a/src/qemu/qemu_extdevice.c b/src/qemu/qemu_extdevice.c index eb7220d..779e759 100644 --- a/src/qemu/qemu_extdevice.c +++ b/src/qemu/qemu_extdevice.c @@ -148,6 +148,9 @@ qemuExtTPMStartEmulator(virQEMUDriverPtr driver, virQEMUDriverConfigPtr cfg =3D virQEMUDriverGetConfig(driver); virDomainTPMDefPtr tpm =3D def->tpm; char *shortName =3D virDomainDefGetShortName(def); + char *pidfiledata =3D NULL; + int timeout; + int len; =20 if (!shortName) return -1; @@ -195,6 +198,22 @@ qemuExtTPMStartEmulator(virQEMUDriverPtr driver, goto error; } =20 + /* check that the swtpm has written its pid into the file */ + timeout =3D 1000; /* ms */ + while ((len =3D virFileReadHeaderQuiet(tpm->data.emulator.pidfile, + 10, &pidfiledata)) <=3D 0) { + if (len =3D=3D 0 && timeout > 0) { + timeout -=3D 50; + usleep(50 * 1000); + continue; + } + virReportError(VIR_ERR_INTERNAL_ERROR, + _("swtpm did not write pidfile '%s'"), + tpm->data.emulator.pidfile); + goto error; + } + VIR_FREE(pidfiledata); + ret =3D 0; =20 cleanup: diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 2b07530..bf0e4da 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6076,6 +6076,10 @@ qemuProcessLaunch(virConnectPtr conn, if (qemuProcessSetupEmulator(vm) < 0) goto cleanup; =20 + VIR_DEBUG("Setting cgroup for external devices (if required)"); + if (qemuSetupCgroupForExtDevices(vm) < 0) + goto cleanup; + VIR_DEBUG("Setting up resctrl"); if (qemuProcessResctrlCreate(driver, vm) < 0) goto cleanup; diff --git a/src/util/virtpm.c b/src/util/virtpm.c index 0617326..3d8a195 100644 --- a/src/util/virtpm.c +++ b/src/util/virtpm.c @@ -39,6 +39,7 @@ #include "virlog.h" #include "virtpm.h" #include "virutil.h" +#include "virpidfile.h" #include "configmake.h" =20 #define VIR_FROM_THIS VIR_FROM_NONE @@ -376,6 +377,25 @@ int virTPMEmulatorInitPaths(virDomainTPMDefPtr tpm, } =20 /* + * virTPMCreatePidFilename + */ +static char *virTPMCreatePidFilename(const char *swtpmStateDir, + const char *shortName) +{ + char *pidfile =3D NULL; + char *devname =3D NULL; + + if (virAsprintf(&devname, "%s-swtpm", shortName) < 0) + return NULL; + + pidfile =3D virPidFileBuildPath(swtpmStateDir, devname); + + VIR_FREE(devname); + + return pidfile; +} + +/* * virTPMEmulatorPrepareHost: * * @tpm: tpm definition @@ -442,6 +462,10 @@ int virTPMEmulatorPrepareHost(virDomainTPMDefPtr tpm, goto cleanup; tpm->data.emulator.source.type =3D VIR_DOMAIN_CHR_TYPE_UNIX; =20 + if (!(tpm->data.emulator.pidfile =3D + virTPMCreatePidFilename(swtpmStateDir, shortName))) + goto cleanup; + ret =3D 0; =20 cleanup: @@ -619,6 +643,9 @@ virTPMEmulatorBuildCommand(virDomainTPMDefPtr tpm, cons= t char *vmname, break; } =20 + virCommandAddArg(cmd, "--pid"); + virCommandAddArgFormat(cmd, "file=3D%s", tpm->data.emulator.pidfile); + return cmd; =20 error: @@ -646,6 +673,7 @@ virTPMEmulatorStop(const char *swtpmStateDir, const cha= r *shortName) virCommandPtr cmd; char *pathname; char *errbuf =3D NULL; + char *pidfile; =20 if (virTPMEmulatorInit() < 0) return; @@ -674,6 +702,11 @@ virTPMEmulatorStop(const char *swtpmStateDir, const ch= ar *shortName) unlink(pathname); =20 cleanup: + /* clean up the PID file */ + if ((pidfile =3D virTPMCreatePidFilename(swtpmStateDir, shortName))) { + unlink(pidfile); + VIR_FREE(pidfile); + } VIR_FREE(pathname); VIR_FREE(errbuf); } --=20 2.5.5 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list