From nobody Wed May 14 11:52:00 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1523415049902560.8932318074203; Tue, 10 Apr 2018 19:50:49 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7CB8D4902F; Wed, 11 Apr 2018 02:50:48 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4CBD56AFE0; Wed, 11 Apr 2018 02:50:48 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0954B180596E; Wed, 11 Apr 2018 02:50:48 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w3B2oSFN009957 for ; Tue, 10 Apr 2018 22:50:28 -0400 Received: by smtp.corp.redhat.com (Postfix) id 684D2662CE; Wed, 11 Apr 2018 02:50:28 +0000 (UTC) Received: from mx1.redhat.com (ext-mx04.extmail.prod.ext.phx2.redhat.com [10.5.110.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 61BAC60CD3 for ; Wed, 11 Apr 2018 02:50:28 +0000 (UTC) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 42D2380469 for ; Wed, 11 Apr 2018 02:50:26 +0000 (UTC) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3B2nfbZ127203 for ; Tue, 10 Apr 2018 22:50:25 -0400 Received: from e37.co.us.ibm.com (e37.co.us.ibm.com [32.97.110.158]) by mx0a-001b2d01.pphosted.com with ESMTP id 2h98dak0ha-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Tue, 10 Apr 2018 22:50:25 -0400 Received: from localhost by e37.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 10 Apr 2018 20:50:24 -0600 Received: from b03cxnp07028.gho.boulder.ibm.com (9.17.130.15) by e37.co.us.ibm.com (192.168.1.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 10 Apr 2018 20:50:21 -0600 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3B2oLqa14811644; Tue, 10 Apr 2018 19:50:21 -0700 Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 61D43BE039; Tue, 10 Apr 2018 20:50:21 -0600 (MDT) Received: from sbct-3.watson.ibm.com (unknown [9.47.158.153]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP id 20E2ABE03E; Tue, 10 Apr 2018 20:50:21 -0600 (MDT) From: Stefan Berger To: libvir-list@redhat.com Date: Tue, 10 Apr 2018 22:50:05 -0400 In-Reply-To: <1523415005-30661-1-git-send-email-stefanb@linux.vnet.ibm.com> References: <1523415005-30661-1-git-send-email-stefanb@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18041102-0024-0000-0000-000018357F6F X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008834; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000256; SDB=6.01016139; UDB=6.00518173; IPR=6.00795386; MB=3.00020511; MTD=3.00000008; XFM=3.00000015; UTC=2018-04-11 02:50:23 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041102-0025-0000-0000-00004F78820B Message-Id: <1523415005-30661-7-git-send-email-stefanb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-11_01:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804110026 X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 11 Apr 2018 02:50:26 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 11 Apr 2018 02:50:26 +0000 (UTC) for IP:'148.163.156.1' DOMAIN:'mx0a-001b2d01.pphosted.com' HELO:'mx0a-001b2d01.pphosted.com' FROM:'stefanb@linux.vnet.ibm.com' RCPT:'' X-RedHat-Spam-Score: -1.71 (RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL) 148.163.156.1 mx0a-001b2d01.pphosted.com 148.163.156.1 mx0a-001b2d01.pphosted.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.28 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2 6/6] tpm: Add swtpm to emulator cgroup X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 11 Apr 2018 02:50:48 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Add the external swtpm to the emulator cgroup so that upper limits of CPU usage can be enforced on the emulated TPM. To enable this we need to have the swtpm write its process id (pid) into a file. We then read it from the file to configure the emulator cgroup. The PID file is created in /var/run/libvirt/qemu/swtpm: [root@localhost swtpm]# ls -lZ /var/run/libvirt/qemu/swtpm/ total 4 -rw-r--r--. 1 tss tss system_u:object_r:qemu_var_run_t:s0 5 Apr = 10 12:26 testvm-swtpm.pid srw-rw----. 1 qemu qemu system_u:object_r:svirt_image_t:s0:c597,c632 0 Apr = 10 12:26 testvm-swtpm.sock The swtpm command line now looks as follows: root@localhost testvm]# ps auxZ | grep swtpm | grep socket | grep -v grep system_u:system_r:virtd_t:s0:c597,c632 tss 18697 0.0 0.0 28172 3892 ? = Ss 16:46 0:00 /usr/bin/swtpm socket --daemon --ctrl type=3Dunixio,pat= h=3D/var/run/libvirt/qemu/swtpm/testvm-swtpm.sock,mode=3D0660 --tpmstate di= r=3D/var/lib/libvirt/swtpm/testvm --log file=3D/var/log/swtpm/libvirt/qemu/= testvm-swtpm.log --pid file=3D/var/run/libvirt/qemu/swtpm/testvm-swtpm.pid Signed-off-by: Stefan Berger --- src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/libvirt_private.syms | 1 + src/qemu/qemu_cgroup.c | 53 +++++++++++++++++++++++++++++++++++++++++++= ++++ src/qemu/qemu_cgroup.h | 1 + src/qemu/qemu_extdevice.c | 19 +++++++++++++++++ src/qemu/qemu_process.c | 4 ++++ src/util/vircgroup.c | 42 +++++++++++++++++++++++++++++++++++++ src/util/vircgroup.h | 1 + src/util/virtpm.c | 33 +++++++++++++++++++++++++++++ 10 files changed, 156 insertions(+) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 0bbb547..e19f7dc 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2620,6 +2620,7 @@ void virDomainTPMDefFree(virDomainTPMDefPtr def) VIR_FREE(def->data.emulator.source.data.nix.path); VIR_FREE(def->data.emulator.storagepath); VIR_FREE(def->data.emulator.logfile); + VIR_FREE(def->data.emulator.pidfile); break; case VIR_DOMAIN_TPM_TYPE_LAST: break; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 80f599c..34bd4a2 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1309,6 +1309,7 @@ struct _virDomainTPMDef { virDomainChrSourceDef source; char *storagepath; char *logfile; + char *pidfile; } emulator; } data; }; diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index af9163f..00cb294 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1467,6 +1467,7 @@ virBufferVasprintf; =20 # util/vircgroup.h virCgroupAddMachineTask; +virCgroupAddProc; virCgroupAddTask; virCgroupAddTaskController; virCgroupAllowAllDevices; diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index bd4859c..859ed55 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -37,6 +37,7 @@ #include "virtypedparam.h" #include "virnuma.h" #include "virsystemd.h" +#include "virpidfile.h" =20 #define VIR_FROM_THIS VIR_FROM_QEMU =20 @@ -1106,6 +1107,58 @@ qemuSetupCgroupCpusetCpus(virCgroupPtr cgroup, =20 =20 int +qemuSetupCgroupForExtDevices(virDomainObjPtr vm) +{ + qemuDomainObjPrivatePtr priv =3D vm->privateData; + virDomainTPMDefPtr tpm =3D vm->def->tpm; + virCgroupPtr cgroup_temp =3D NULL; + pid_t pid; + int ret =3D -1; + + if (priv->cgroup =3D=3D NULL) + return 0; /* Not supported, so claim success */ + + /* + * If CPU cgroup controller is not initialized here, then we need + * neither period nor quota settings. And if CPUSET controller is + * not initialized either, then there's nothing to do anyway. + */ + if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU) && + !virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPUSET= )) + return 0; + + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_EMULATOR, 0, + false, &cgroup_temp) < 0) + goto cleanup; + + if (tpm) { + switch (tpm->type) { + case VIR_DOMAIN_TPM_TYPE_EMULATOR: + if (virPidFileReadPath(tpm->data.emulator.pidfile, &pid) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Could not read swtpm's pidfile %s"), + tpm->data.emulator.pidfile); + goto cleanup; + } + if (virCgroupAddProc(cgroup_temp, pid) < 0) + goto cleanup; + break; + case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH: + case VIR_DOMAIN_TPM_TYPE_LAST: + break; + } + } + + ret =3D 0; + +cleanup: + virCgroupFree(&cgroup_temp); + + return ret; +} + + +int qemuSetupGlobalCpuCgroup(virDomainObjPtr vm) { qemuDomainObjPrivatePtr priv =3D vm->privateData; diff --git a/src/qemu/qemu_cgroup.h b/src/qemu/qemu_cgroup.h index 3b8ff60..478bf7e 100644 --- a/src/qemu/qemu_cgroup.h +++ b/src/qemu/qemu_cgroup.h @@ -69,6 +69,7 @@ int qemuSetupCgroupVcpuBW(virCgroupPtr cgroup, long long quota); int qemuSetupCgroupCpusetCpus(virCgroupPtr cgroup, virBitmapPtr cpumask); int qemuSetupGlobalCpuCgroup(virDomainObjPtr vm); +int qemuSetupCgroupForExtDevices(virDomainObjPtr vm); int qemuRemoveCgroup(virDomainObjPtr vm); =20 typedef struct _qemuCgroupEmulatorAllNodesData qemuCgroupEmulatorAllNodesD= ata; diff --git a/src/qemu/qemu_extdevice.c b/src/qemu/qemu_extdevice.c index ee327ca..23ec310 100644 --- a/src/qemu/qemu_extdevice.c +++ b/src/qemu/qemu_extdevice.c @@ -128,6 +128,9 @@ qemuExtTPMStartEmulator(virQEMUDriverPtr driver, char *errbuf =3D NULL; virQEMUDriverConfigPtr cfg =3D virQEMUDriverGetConfig(driver); virDomainTPMDefPtr tpm =3D def->tpm; + char *pidfiledata =3D NULL; + int timeout; + int len; =20 /* stop any left-over TPM emulator for this VM */ virTPMEmulatorStop(cfg->swtpmStateDir, def->name); @@ -170,6 +173,22 @@ qemuExtTPMStartEmulator(virQEMUDriverPtr driver, goto error; } =20 + /* check that the swtpm has written its pid into the file */ + timeout =3D 1000; /* ms */ + while ((len =3D virFileReadHeaderQuiet(tpm->data.emulator.pidfile, + 10, &pidfiledata)) <=3D 0) { + if (len =3D=3D 0 && timeout > 0) { + timeout -=3D 50; + usleep(50 * 1000); + continue; + } + virReportError(VIR_ERR_INTERNAL_ERROR, + _("swtpm did not write pidfile '%s'"), + tpm->data.emulator.pidfile); + goto error; + } + VIR_FREE(pidfiledata); + ret =3D 0; =20 cleanup: diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 7bf90a4..4877c49 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6072,6 +6072,10 @@ qemuProcessLaunch(virConnectPtr conn, if (qemuProcessSetupEmulator(vm) < 0) goto cleanup; =20 + VIR_DEBUG("Setting cgroup for external devices (if required)"); + if (qemuSetupCgroupForExtDevices(vm) < 0) + goto cleanup; + VIR_DEBUG("Setting up resctrl"); if (qemuProcessResctrlCreate(driver, vm) < 0) goto cleanup; diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index 0a31947..4809f12 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -1245,6 +1245,38 @@ virCgroupAddMachineTask(virCgroupPtr group, pid_t pi= d) return virCgroupAddTaskInternal(group, pid, true); } =20 +/** + * virCgroupAddProc: + * + * @group: The cgroup to add a process to + * @pid: The pid of the process to add + * + * Returns: 0 on success, -1 on error + */ +int +virCgroupAddProc(virCgroupPtr group, pid_t pid) +{ + int ret =3D -1; + size_t i; + + for (i =3D 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + /* Skip over controllers not mounted */ + if (!group->controllers[i].mountPoint) + continue; + + /* We must never add tasks in systemd's hierarchy */ + if (i =3D=3D VIR_CGROUP_CONTROLLER_SYSTEMD) + continue; + + if (virCgroupSetValueU64(group, i, "cgroup.procs", pid) < 0) + goto cleanup; + } + + ret =3D 0; + cleanup: + return ret; +} + =20 /** * virCgroupAddTaskController: @@ -4298,6 +4330,16 @@ virCgroupAddMachineTask(virCgroupPtr group ATTRIBUTE= _UNUSED, =20 =20 int +virCgroupAddProc(virCgroupPtr group ATTRIBUTE_UNUSED, + pid_t pid ATTRIBUTE_UNUSED) +{ + virReportSystemError(ENXIO, "%s", + _("Control groups not supported on this platform"= )); + return -1; +} + + +int virCgroupAddTaskController(virCgroupPtr group ATTRIBUTE_UNUSED, pid_t pid ATTRIBUTE_UNUSED, int controller ATTRIBUTE_UNUSED) diff --git a/src/util/vircgroup.h b/src/util/vircgroup.h index d833927..82b3964 100644 --- a/src/util/vircgroup.h +++ b/src/util/vircgroup.h @@ -132,6 +132,7 @@ int virCgroupPathOfController(virCgroupPtr group, =20 int virCgroupAddTask(virCgroupPtr group, pid_t pid); int virCgroupAddMachineTask(virCgroupPtr group, pid_t pid); +int virCgroupAddProc(virCgroupPtr group, pid_t pid); =20 int virCgroupAddTaskController(virCgroupPtr group, pid_t pid, diff --git a/src/util/virtpm.c b/src/util/virtpm.c index 3bb911e..af2e1d2 100644 --- a/src/util/virtpm.c +++ b/src/util/virtpm.c @@ -39,6 +39,7 @@ #include "virlog.h" #include "virtpm.h" #include "virutil.h" +#include "virpidfile.h" #include "configmake.h" =20 #define VIR_FROM_THIS VIR_FROM_NONE @@ -370,6 +371,25 @@ int virTPMEmulatorInitPaths(virDomainTPMDefPtr tpm, } =20 /* + * virTPMCreatePidfileName + */ +static char *virTPMCreatePidfileName(const char *swtpmStateDir, + const char *vmname) +{ + char *pidfile =3D NULL; + char *devname =3D NULL; + + if (virAsprintf(&devname, "%s-swtpm", vmname) < 0) + return NULL; + + pidfile =3D virPidFileBuildPath(swtpmStateDir, devname); + + VIR_FREE(devname); + + return pidfile; +} + +/* * virTPMEmulatorPrepareHost: * * @tpm: tpm definition @@ -427,6 +447,10 @@ int virTPMEmulatorPrepareHost(virDomainTPMDefPtr tpm, goto cleanup; tpm->data.emulator.source.type =3D VIR_DOMAIN_CHR_TYPE_UNIX; =20 + if (!(tpm->data.emulator.pidfile =3D + virTPMCreatePidfileName(swtpmStateDir, vmname))) + goto cleanup; + ret =3D 0; =20 cleanup: @@ -577,6 +601,9 @@ virTPMEmulatorBuildCommand(virDomainTPMDefPtr tpm, cons= t char *vmname, break; } =20 + virCommandAddArg(cmd, "--pid"); + virCommandAddArgFormat(cmd, "file=3D%s", tpm->data.emulator.pidfile); + return cmd; =20 error: @@ -604,6 +631,7 @@ virTPMEmulatorStop(const char *swtpmStateDir, const cha= r *vmname) virCommandPtr cmd; char *pathname; char *errbuf =3D NULL; + char *pidfile; =20 if (virTPMEmulatorInit() < 0) return; @@ -632,6 +660,11 @@ virTPMEmulatorStop(const char *swtpmStateDir, const ch= ar *vmname) unlink(pathname); =20 cleanup: + /* clean up the PID file */ + if ((pidfile =3D virTPMCreatePidfileName(swtpmStateDir, vmname))) { + unlink(pidfile); + VIR_FREE(pidfile); + } VIR_FREE(pathname); VIR_FREE(errbuf); } --=20 2.5.5 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list