From nobody Thu May 15 06:05:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1527193644572764.6064064454033; Thu, 24 May 2018 13:27:24 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2A4D5C109AB1; Thu, 24 May 2018 20:27:23 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E30D110016DC; Thu, 24 May 2018 20:27:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9606918033F2; Thu, 24 May 2018 20:27:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w4OKQYdR005472 for ; Thu, 24 May 2018 16:26:35 -0400 Received: by smtp.corp.redhat.com (Postfix) id 148D117D19; Thu, 24 May 2018 20:26:34 +0000 (UTC) Received: from mx1.redhat.com (ext-mx05.extmail.prod.ext.phx2.redhat.com [10.5.110.29]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0B1B75F90E for ; Thu, 24 May 2018 20:26:34 +0000 (UTC) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 664C723E6EB for ; Thu, 24 May 2018 20:26:32 +0000 (UTC) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4OKNeT4112881 for ; Thu, 24 May 2018 16:26:32 -0400 Received: from e18.ny.us.ibm.com (e18.ny.us.ibm.com [129.33.205.208]) by mx0a-001b2d01.pphosted.com with ESMTP id 2j60rqa1du-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 24 May 2018 16:26:31 -0400 Received: from localhost by e18.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 24 May 2018 16:26:30 -0400 Received: from b01cxnp23034.gho.pok.ibm.com (9.57.198.29) by e18.ny.us.ibm.com (146.89.104.205) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Thu, 24 May 2018 16:26:28 -0400 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w4OKQRRh50528298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 24 May 2018 20:26:27 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C872AB2067; Thu, 24 May 2018 17:28:14 -0400 (EDT) Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AECB5B2064; Thu, 24 May 2018 17:28:14 -0400 (EDT) Received: from sbct-3.pok.ibm.com (unknown [9.47.158.153]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTP; Thu, 24 May 2018 17:28:14 -0400 (EDT) From: Stefan Berger To: libvir-list@redhat.com Date: Thu, 24 May 2018 16:26:07 -0400 In-Reply-To: <20180524202614.3180183-1-stefanb@linux.vnet.ibm.com> References: <20180524202614.3180183-1-stefanb@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18052420-0044-0000-0000-0000041A82F4 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00009065; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000261; SDB=6.01036001; UDB=6.00529949; IPR=6.00815130; MB=3.00021234; MTD=3.00000008; XFM=3.00000015; UTC=2018-05-24 20:26:30 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18052420-0045-0000-0000-0000084CA01A Message-Id: <20180524202614.3180183-12-stefanb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-24_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805240231 X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 24 May 2018 20:26:32 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 24 May 2018 20:26:32 +0000 (UTC) for IP:'148.163.156.1' DOMAIN:'mx0a-001b2d01.pphosted.com' HELO:'mx0a-001b2d01.pphosted.com' FROM:'stefanb@linux.vnet.ibm.com' RCPT:'' X-RedHat-Spam-Score: -0.7 (RCVD_IN_DNSWL_LOW) 148.163.156.1 mx0a-001b2d01.pphosted.com 148.163.156.1 mx0a-001b2d01.pphosted.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.29 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Cc: fiuczy@linux.ibm.com, mhartmay@linux.ibm.com, jtomko@redhat.com Subject: [libvirt] [PATCH v8 11/18] qemu: Add swtpm to emulator cgroup X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 24 May 2018 20:27:23 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Add the external swtpm to the emulator cgroup so that upper limits of CPU usage can be enforced on the emulated TPM. To enable this we need to have the swtpm write its process id (pid) into a file. We then read it from the file to configure the emulator cgroup. The PID file is created in /var/run/libvirt/qemu/swtpm: [root@localhost swtpm]# ls -lZ /var/run/libvirt/qemu/swtpm/ total 4 -rw-r--r--. 1 tss tss system_u:object_r:qemu_var_run_t:s0 5 Apr = 10 12:26 1-testvm-swtpm.pid srw-rw----. 1 qemu qemu system_u:object_r:svirt_image_t:s0:c597,c632 0 Apr = 10 12:26 1-testvm-swtpm.sock The swtpm command line now looks as follows: root@localhost testvm]# ps auxZ | grep swtpm | grep socket | grep -v grep system_u:system_r:virtd_t:s0:c597,c632 tss 18697 0.0 0.0 28172 3892 ? = Ss 16:46 0:00 /usr/bin/swtpm socket --daemon --ctrl type=3Dunixio,pat= h=3D/var/run/libvirt/qemu/swtpm/1-testvm-swtpm.sock,mode=3D0600 --tpmstate = dir=3D/var/lib/libvirt/swtpm/485d0004-a48f-436a-8457-8a3b73e28568/tpm1.2/ -= -log file=3D/var/log/swtpm/libvirt/qemu/testvm-swtpm.log --pid file=3D/var/= run/libvirt/qemu/swtpm/1-testvm-swtpm.pid Signed-off-by: Stefan Berger Reviewed-by: John Ferlan Reviewed-by: J=EF=BF=BDn Tomko --- src/qemu/qemu_cgroup.c | 35 ++++++++++++ src/qemu/qemu_cgroup.h | 2 + src/qemu/qemu_extdevice.c | 26 +++++++++ src/qemu/qemu_extdevice.h | 6 +++ src/qemu/qemu_process.c | 4 ++ src/qemu/qemu_tpm.c | 135 ++++++++++++++++++++++++++++++++++++++++++= ++-- src/qemu/qemu_tpm.h | 6 +++ 7 files changed, 211 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index 54b00a5da5..12b3f3bf40 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -26,6 +26,7 @@ #include "qemu_cgroup.h" #include "qemu_domain.h" #include "qemu_process.h" +#include "qemu_extdevice.h" #include "vircgroup.h" #include "virlog.h" #include "viralloc.h" @@ -1172,6 +1173,40 @@ qemuSetupCgroupCpusetCpus(virCgroupPtr cgroup, } =20 =20 +int +qemuSetupCgroupForExtDevices(virDomainObjPtr vm, + virQEMUDriverPtr driver) +{ + qemuDomainObjPrivatePtr priv =3D vm->privateData; + virCgroupPtr cgroup_temp =3D NULL; + int ret =3D -1; + + if (!qemuExtDevicesHasDevice(vm->def) || + priv->cgroup =3D=3D NULL) + return 0; /* Not supported, so claim success */ + + /* + * If CPU cgroup controller is not initialized here, then we need + * neither period nor quota settings. And if CPUSET controller is + * not initialized either, then there's nothing to do anyway. + */ + if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU) && + !virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPUSET= )) + return 0; + + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_EMULATOR, 0, + false, &cgroup_temp) < 0) + goto cleanup; + + ret =3D qemuExtDevicesSetupCgroup(driver, vm->def, cgroup_temp); + + cleanup: + virCgroupFree(&cgroup_temp); + + return ret; +} + + int qemuSetupGlobalCpuCgroup(virDomainObjPtr vm) { diff --git a/src/qemu/qemu_cgroup.h b/src/qemu/qemu_cgroup.h index 3b8ff6055d..c2fca7fc1d 100644 --- a/src/qemu/qemu_cgroup.h +++ b/src/qemu/qemu_cgroup.h @@ -69,6 +69,8 @@ int qemuSetupCgroupVcpuBW(virCgroupPtr cgroup, long long quota); int qemuSetupCgroupCpusetCpus(virCgroupPtr cgroup, virBitmapPtr cpumask); int qemuSetupGlobalCpuCgroup(virDomainObjPtr vm); +int qemuSetupCgroupForExtDevices(virDomainObjPtr vm, + virQEMUDriverPtr driver); int qemuRemoveCgroup(virDomainObjPtr vm); =20 typedef struct _qemuCgroupEmulatorAllNodesData qemuCgroupEmulatorAllNodesD= ata; diff --git a/src/qemu/qemu_extdevice.c b/src/qemu/qemu_extdevice.c index 790b19be9e..d982922470 100644 --- a/src/qemu/qemu_extdevice.c +++ b/src/qemu/qemu_extdevice.c @@ -30,6 +30,8 @@ #include "virlog.h" #include "virstring.h" #include "virtime.h" +#include "virtpm.h" +#include "virpidfile.h" =20 #define VIR_FROM_THIS VIR_FROM_QEMU =20 @@ -152,3 +154,27 @@ qemuExtDevicesStop(virQEMUDriverPtr driver, if (def->tpm) qemuExtTPMStop(driver, def); } + + +bool +qemuExtDevicesHasDevice(virDomainDefPtr def) +{ + if (def->tpm && def->tpm->type =3D=3D VIR_DOMAIN_TPM_TYPE_EMULATOR) + return true; + + return false; +} + + +int +qemuExtDevicesSetupCgroup(virQEMUDriverPtr driver, + virDomainDefPtr def, + virCgroupPtr cgroup) +{ + int ret =3D 0; + + if (def->tpm) + ret =3D qemuExtTPMSetupCgroup(driver, def, cgroup); + + return ret; +} diff --git a/src/qemu/qemu_extdevice.h b/src/qemu/qemu_extdevice.h index 6de858b2a3..c557778ddb 100644 --- a/src/qemu/qemu_extdevice.h +++ b/src/qemu/qemu_extdevice.h @@ -50,4 +50,10 @@ void qemuExtDevicesStop(virQEMUDriverPtr driver, virDomainDefPtr def) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); =20 +bool qemuExtDevicesHasDevice(virDomainDefPtr def); + +int qemuExtDevicesSetupCgroup(virQEMUDriverPtr driver, + virDomainDefPtr def, + virCgroupPtr cgroup); + #endif /* __QEMU_EXTDEVICE_H__ */ diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 3e98b42933..194cc891d1 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6246,6 +6246,10 @@ qemuProcessLaunch(virConnectPtr conn, if (qemuProcessSetupEmulator(vm) < 0) goto cleanup; =20 + VIR_DEBUG("Setting cgroup for external devices (if required)"); + if (qemuSetupCgroupForExtDevices(vm, driver) < 0) + goto cleanup; + VIR_DEBUG("Setting up resctrl"); if (qemuProcessResctrlCreate(driver, vm) < 0) goto cleanup; diff --git a/src/qemu/qemu_tpm.c b/src/qemu/qemu_tpm.c index a693854b30..201da359d7 100644 --- a/src/qemu/qemu_tpm.c +++ b/src/qemu/qemu_tpm.c @@ -40,6 +40,7 @@ #include "viruuid.h" #include "virfile.h" #include "virstring.h" +#include "virpidfile.h" #include "configmake.h" #include "dirname.h" #include "qemu_tpm.h" @@ -308,6 +309,57 @@ qemuTPMEmulatorInitPaths(virDomainTPMDefPtr tpm, } =20 =20 +/* + * qemuTPMCreatePidFilename + */ +static char * +qemuTPMEmulatorCreatePidFilename(const char *swtpmStateDir, + const char *shortName) +{ + char *pidfile =3D NULL; + char *devicename =3D NULL; + + if (virAsprintf(&devicename, "%s-swtpm", shortName) < 0) + return NULL; + + pidfile =3D virPidFileBuildPath(swtpmStateDir, devicename); + + VIR_FREE(devicename); + + return pidfile; +} + + +/* + * qemuTPMEmulatorGetPid + * + * @swtpmStateDir: the directory where swtpm writes the pidfile into + * @shortName: short name of the domain + * @pid: pointer to pid + * + * Return -errno upon error, or zero on successful reading of the pidfile. + * If the PID was not still alive, zero will be returned, and @pid will be + * set to -1; + */ +static int +qemuTPMEmulatorGetPid(const char *swtpmStateDir, + const char *shortName, + pid_t *pid) +{ + int ret; + char *pidfile =3D qemuTPMEmulatorCreatePidFilename(swtpmStateDir, + shortName); + if (!pidfile) + return -ENOMEM; + + ret =3D virPidFileReadPathIfAlive(pidfile, pid, swtpm_path); + + VIR_FREE(pidfile); + + return ret; +} + + /* * qemuTPMEmulatorPrepareHost: * @@ -490,6 +542,9 @@ qemuTPMEmulatorRunSetup(const char *storagepath, * @privileged: whether we are running in privileged mode * @swtpm_user: The uid for the swtpm to run as (drop privileges to from r= oot) * @swtpm_group: The gid for the swtpm to run as + * @swtpmStateDir: the directory where swtpm writes the pid file and creat= es the + * Unix socket + * @shortName: the short name of the VM * * Create the virCommand use for starting the emulator * Do some initializations on the way, such as creation of storage @@ -501,10 +556,13 @@ qemuTPMEmulatorBuildCommand(virDomainTPMDefPtr tpm, const unsigned char *vmuuid, bool privileged, uid_t swtpm_user, - gid_t swtpm_group) + gid_t swtpm_group, + const char *swtpmStateDir, + const char *shortName) { virCommandPtr cmd =3D NULL; bool created =3D false; + char *pidfile; =20 if (qemuTPMCreateEmulatorStorage(tpm->data.emulator.storagepath, &created, swtpm_user, swtpm_group) < = 0) @@ -548,6 +606,13 @@ qemuTPMEmulatorBuildCommand(virDomainTPMDefPtr tpm, break; } =20 + if (!(pidfile =3D qemuTPMEmulatorCreatePidFilename(swtpmStateDir, shor= tName))) + goto error; + + virCommandAddArg(cmd, "--pid"); + virCommandAddArgFormat(cmd, "file=3D%s", pidfile); + VIR_FREE(pidfile); + return cmd; =20 error: @@ -699,7 +764,8 @@ qemuExtTPMStartEmulator(virQEMUDriverPtr driver, virQEMUDriverConfigPtr cfg; virDomainTPMDefPtr tpm =3D def->tpm; char *shortName =3D virDomainDefGetShortName(def); - int cmdret =3D 0; + int cmdret =3D 0, timeout, rc; + pid_t pid; =20 if (!shortName) return -1; @@ -712,7 +778,8 @@ qemuExtTPMStartEmulator(virQEMUDriverPtr driver, if (!(cmd =3D qemuTPMEmulatorBuildCommand(tpm, def->name, def->uuid, driver->privileged, cfg->swtpm_user, - cfg->swtpm_group))) + cfg->swtpm_group, + cfg->swtpmStateDir, shortName)= )) goto cleanup; =20 if (qemuExtDeviceLogCommand(logCtxt, cmd, "TPM Emulator") < 0) @@ -732,6 +799,22 @@ qemuExtTPMStartEmulator(virQEMUDriverPtr driver, goto cleanup; } =20 + /* check that the swtpm has written its pid into the file */ + timeout =3D 1000; /* ms */ + while (timeout > 0) { + rc =3D qemuTPMEmulatorGetPid(cfg->swtpmStateDir, shortName, &pid); + if (rc < 0) { + timeout -=3D 50; + usleep(50 * 1000); + continue; + } + if (rc =3D=3D 0 && pid =3D=3D (pid_t)-1) + goto error; + break; + } + if (timeout <=3D 0) + goto error; + ret =3D 0; =20 cleanup: @@ -742,6 +825,11 @@ qemuExtTPMStartEmulator(virQEMUDriverPtr driver, virObjectUnref(cfg); =20 return ret; + + error: + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("swtpm failed to start")); + goto cleanup; } =20 =20 @@ -791,3 +879,44 @@ qemuExtTPMStop(virQEMUDriverPtr driver, VIR_FREE(shortName); virObjectUnref(cfg); } + + +int +qemuExtTPMSetupCgroup(virQEMUDriverPtr driver, + virDomainDefPtr def, + virCgroupPtr cgroup) +{ + virQEMUDriverConfigPtr cfg =3D virQEMUDriverGetConfig(driver); + char *pidfile =3D NULL; + char *shortName =3D NULL; + int ret =3D -1, rc; + pid_t pid; + + switch (def->tpm->type) { + case VIR_DOMAIN_TPM_TYPE_EMULATOR: + shortName =3D virDomainDefGetShortName(def); + if (!shortName) + goto cleanup; + rc =3D qemuTPMEmulatorGetPid(cfg->swtpmStateDir, shortName, &pid); + if (rc < 0 || (rc =3D=3D 0 && pid =3D=3D (pid_t)-1)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Could not get process id of swtpm")); + goto cleanup; + } + if (virCgroupAddTask(cgroup, pid) < 0) + goto cleanup; + break; + case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH: + case VIR_DOMAIN_TPM_TYPE_LAST: + break; + } + + ret =3D 0; + + cleanup: + VIR_FREE(pidfile); + VIR_FREE(shortName); + virObjectUnref(cfg); + + return ret; +} diff --git a/src/qemu/qemu_tpm.h b/src/qemu/qemu_tpm.h index 20f3a9ccc4..6eb1294da0 100644 --- a/src/qemu/qemu_tpm.h +++ b/src/qemu/qemu_tpm.h @@ -47,4 +47,10 @@ void qemuExtTPMStop(virQEMUDriverPtr driver, virDomainDefPtr def) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); =20 +int qemuExtTPMSetupCgroup(virQEMUDriverPtr driver, + virDomainDefPtr def, + virCgroupPtr cgroup) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3) + ATTRIBUTE_RETURN_CHECK; + #endif /* __QEMU_TPM_H__ */ --=20 2.14.3 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list