From nobody Tue Dec 16 08:39:17 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=intel.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1528450378336274.7354217498455; Fri, 8 Jun 2018 02:32:58 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C3CE1C057FA0; Fri, 8 Jun 2018 09:32:56 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5F30E60C22; Fri, 8 Jun 2018 09:32:56 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9CB9C4CA83; Fri, 8 Jun 2018 09:32:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w589WpO1010286 for ; Fri, 8 Jun 2018 05:32:51 -0400 Received: by smtp.corp.redhat.com (Postfix) id A10755D9C6; Fri, 8 Jun 2018 09:32:51 +0000 (UTC) Received: from mx1.redhat.com (ext-mx08.extmail.prod.ext.phx2.redhat.com [10.5.110.32]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 95DC15D9C7 for ; Fri, 8 Jun 2018 09:32:48 +0000 (UTC) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 46FA0C057F87 for ; Fri, 8 Jun 2018 09:32:47 +0000 (UTC) Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jun 2018 02:32:46 -0700 Received: from david-unc.bj.intel.com ([10.238.145.77]) by fmsmga004.fm.intel.com with ESMTP; 08 Jun 2018 02:32:45 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,490,1520924400"; d="scan'208";a="61461050" From: Wang Huaqiang To: libvir-list@redhat.com Date: Fri, 8 Jun 2018 17:02:17 +0800 Message-Id: <1528448539-25588-2-git-send-email-huaqiang.wang@intel.com> In-Reply-To: <1528448539-25588-1-git-send-email-huaqiang.wang@intel.com> References: <1528448539-25588-1-git-send-email-huaqiang.wang@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 08 Jun 2018 09:32:47 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 08 Jun 2018 09:32:47 +0000 (UTC) for IP:'192.55.52.120' DOMAIN:'mga04.intel.com' HELO:'mga04.intel.com' FROM:'huaqiang.wang@intel.com' RCPT:'' X-RedHat-Spam-Score: -2.301 (RCVD_IN_DNSWL_MED, SPF_PASS) 192.55.52.120 mga04.intel.com 192.55.52.120 mga04.intel.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.32 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Cc: shaohe.feng@intel.com, Wang Huaqiang , bing.niu@intel.com, jian-feng.ding@intel.com, rui.zang@intel.com Subject: [libvirt] [PATCH 1/3] util: add Intel x86 RDT/CMT support X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 08 Jun 2018 09:32:57 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Add RDT/CMT feature (Intel x86) by interacting with kernel resctrl file sys= tem. Integrate code into util/resctrl. --- src/libvirt_private.syms | 9 ++ src/util/virresctrl.c | 316 +++++++++++++++++++++++++++++++++++++++++++= +++- src/util/virresctrl.h | 44 +++++++ 3 files changed, 367 insertions(+), 2 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index b4ab1f3..e16c3e0 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2634,6 +2634,15 @@ virResctrlAllocSetSize; virResctrlGetInfo; virResctrlInfoGetCache; virResctrlInfoNew; +virResctrlMonNew; +virResctrlMonSetID; +virResctrlMonGetID; +virResctrlMonDeterminePath; +virResctrlMonAddPID; +virResctrlMonCreate; +virResctrlMonRemove; +virResctrlMonIsRunning; +virResctrlMonGetCacheOccupancy; =20 =20 # util/virrotatingfile.h diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index fc11635..e5f5caf 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -224,6 +224,22 @@ struct _virResctrlAlloc { =20 static virClassPtr virResctrlAllocClass; =20 +struct _virResctrlMon { + virObject parent; + + /* pairedalloc: pointer to a resctrl allocaion it paried with. + * NULL for a resctrl monitoring group not associated with + * any allocation. */ + virResctrlAllocPtr pairedalloc; + /* The identifier (any unique string for now) */ + char *id; + /* libvirt-generated path, may be identical to alloction path + * may not if allocation is ready */ + char *path; +}; + +static virClassPtr virResctrlMonClass; + static void virResctrlAllocDispose(void *obj) { @@ -275,7 +291,28 @@ virResctrlAllocOnceInit(void) } =20 =20 +static void +virResctrlMonDispose(void *obj) +{ + virResctrlMonPtr resctrlMon =3D obj; + + VIR_FREE(resctrlMon->id); + VIR_FREE(resctrlMon->path); +} + + +static int +virResctrlMonOnceInit(void) +{ + if (!VIR_CLASS_NEW(virResctrlMon, virClassForObject())) + return -1; + + return 0; +} + + VIR_ONCE_GLOBAL_INIT(virResctrlAlloc) +VIR_ONCE_GLOBAL_INIT(virResctrlMon) =20 =20 virResctrlAllocPtr @@ -288,6 +325,16 @@ virResctrlAllocNew(void) } =20 =20 +virResctrlMonPtr +virResctrlMonNew(void) +{ + if (virResctrlMonInitialize() < 0) + return NULL; + + return virObjectNew(virResctrlMonClass); +} + + /* Common functions */ #ifdef __linux__ static int @@ -329,8 +376,6 @@ virResctrlLockWrite(void) #endif =20 =20 - - static int virResctrlUnlock(int fd) { @@ -1646,3 +1691,270 @@ virResctrlAllocRemove(virResctrlAllocPtr alloc) =20 return ret; } + + +int +virResctrlMonSetID(virResctrlMonPtr mon, + const char *id) +{ + if (!id) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Resctrl mon group 'id' cannot be NULL")); + return -1; + } + + return VIR_STRDUP(mon->id, id); +} + + +const char * +virResctrlMonGetID(virResctrlMonPtr mon) +{ + return mon->id; +} + + +int +virResctrlMonDeterminePath(virResctrlMonPtr mon, + const char *machinename) +{ + + VIR_DEBUG("mon group, mon->path=3D%s\n", mon->path); + if (!mon->id) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Resctrl mon group id must be set before creation")); + return -1; + } + + if (mon->path) + return -1; + + if(mon->pairedalloc) + { + if (virAsprintf(&mon->path, "%s/%s-%s", + SYSFS_RESCTRL_PATH, machinename, mon->id) < 0) + return -1; + } + else + { + if (virAsprintf(&mon->path, "%s/mon_groups/%s-%s", + SYSFS_RESCTRL_PATH, machinename, mon->id) < 0) + return -1; + } + + return 0; +} + + +int +virResctrlMonAddPID(virResctrlMonPtr mon, + pid_t pid) +{ + char *tasks =3D NULL; + char *pidstr =3D NULL; + int ret =3D 0; + + if (!mon->path) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot add pid to non-existing resctrl mon group")); + return -1; + } + + VIR_DEBUG("Add PID %d to domain %s\n", + pid, mon->path); + + if (virAsprintf(&tasks, "%s/tasks", mon->path) < 0) + return -1; + + if (virAsprintf(&pidstr, "%lld", (long long int) pid) < 0) + goto cleanup; + + if (virFileWriteStr(tasks, pidstr, 0) < 0) { + virReportSystemError(errno, + _("Cannot write pid in tasks file '%s'"), + tasks); + goto cleanup; + } + + ret =3D 0; +cleanup: + VIR_FREE(tasks); + VIR_FREE(pidstr); + return ret; +} + + +int +virResctrlMonCreate(virResctrlAllocPtr pairedalloc, + virResctrlMonPtr mon, + const char *machinename) +{ + int ret =3D -1; + int lockfd =3D -1; + + if (!mon) + return 0; + + + if (pairedalloc) + { + if (!virFileExists(pairedalloc->path)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("For paired mon group, the resctrl allocation " + "must be created first")); + goto cleanup; + } + mon->pairedalloc =3D pairedalloc; + + if (virResctrlMonDeterminePath(mon, machinename) < 0) + return -1; + } + else + { + mon->pairedalloc =3D NULL; + + /* resctrl mon group object may use for multiple purpose, + * free mon group path information*/ + VIR_FREE(mon->path); + + if (virResctrlMonDeterminePath(mon, machinename) < 0) + return -1; + + lockfd =3D virResctrlLockWrite(); + if (lockfd < 0) + goto cleanup; + + if (virFileExists(mon->path)) + { + VIR_DEBUG("Removing resctrl mon group %s", mon->path); + if (rmdir(mon->path) !=3D 0 && errno !=3D ENOENT) { + ret =3D -errno; + VIR_ERROR(_("Unable to remove %s (%d)"), mon->path, errno); + goto cleanup; + } + } + + if (virFileMakePath(mon->path) < 0) { + virReportSystemError(errno, + _("Cannot create resctrl directory '%s'"), + mon->path); + goto cleanup; + } + } + + ret =3D 0; +cleanup: + virResctrlUnlock(lockfd); + return ret; +} + + +int +virResctrlMonRemove(virResctrlMonPtr mon) +{ + int ret =3D 0; + + if (!mon->path) + return 0; + + VIR_DEBUG("Removing resctrl mon group %s", mon->path); + if (rmdir(mon->path) !=3D 0 && errno !=3D ENOENT) { + ret =3D -errno; + VIR_ERROR(_("Unable to remove %s (%d)"), mon->path, errno); + } + + return ret; +} + + +bool +virResctrlMonIsRunning(virResctrlMonPtr mon) +{ + bool ret =3D false; + char *tasks =3D NULL; + + if (mon && virFileExists(mon->path)) { + ret =3D virFileReadValueString(&tasks, "%s/tasks", mon->path); + if (ret < 0) + goto cleanup; + + if (!tasks || !tasks[0]) + goto cleanup; + + ret =3D true; + } + +cleanup: + VIR_FREE(tasks); + + return ret; +} + + +int +virResctrlMonGetCacheOccupancy(virResctrlMonPtr mon, + unsigned int * cacheoccu) +{ + DIR *dirp =3D NULL; + int ret =3D -1; + int rv =3D -1; + struct dirent *ent =3D NULL; + unsigned int cachetotal =3D 0; + unsigned int cacheoccyperblock =3D 0; + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + char *pathmondata =3D NULL; + + if (!mon->path) + goto cleanup; + + rv =3D virDirOpenIfExists(&dirp,mon->path); + if (rv <=3D 0) { + goto cleanup; + } + + virBufferAsprintf(&buf, "%s/mon_data", + mon->path); + pathmondata =3D virBufferContentAndReset(&buf); + if (!pathmondata) + goto cleanup; + + VIR_DEBUG("Seek llc_occupancy file from root: %s ", + pathmondata); + + if (virDirOpen(&dirp, pathmondata) < 0) + goto cleanup; + + while ((rv =3D virDirRead(dirp, &ent, pathmondata)) > 0) { + VIR_DEBUG("Parsing file '%s'", ent->d_name); + if (ent->d_type !=3D DT_DIR) + continue; + + if (STRNEQLEN(ent->d_name, "mon_L", 5)) + continue; + + rv =3D virFileReadValueUint(&cacheoccyperblock, + "%s/%s/llc_occupancy", + pathmondata, ent->d_name); + if (rv =3D=3D -2) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "file %s/%s/llc_occupancy does not exist", + pathmondata, ent->d_name); + goto cleanup; + } else if (rv < 0) { + goto cleanup; + } + + VIR_DEBUG("%s/%s/llc_occupancy: occupancy %d bytes", + pathmondata, ent->d_name, cacheoccyperblock); + + cachetotal +=3D cacheoccyperblock; + } + + *cacheoccu =3D cachetotal; + + ret =3D 0; +cleanup: + VIR_FREE(pathmondata); + VIR_DIR_CLOSE(dirp); + return ret; +} diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h index 5368ba2..a23c425 100644 --- a/src/util/virresctrl.h +++ b/src/util/virresctrl.h @@ -35,6 +35,12 @@ typedef enum { =20 VIR_ENUM_DECL(virCache); =20 +typedef enum { + VIR_RESCTRL_MONACT_NONE, + VIR_RESCTRL_MONACT_ENABLE, + VIR_RESCTRL_MONACT_DISABLE +} virResctrlMonAct; + =20 typedef struct _virResctrlInfoPerCache virResctrlInfoPerCache; typedef virResctrlInfoPerCache *virResctrlInfoPerCachePtr; @@ -118,4 +124,42 @@ virResctrlAllocAddPID(virResctrlAllocPtr alloc, int virResctrlAllocRemove(virResctrlAllocPtr alloc); =20 + +/* Monitoring-related things */ +typedef struct _virResctrlMon virResctrlMon; +typedef virResctrlMon *virResctrlMonPtr; + +virResctrlMonPtr +virResctrlMonNew(void); + +int +virResctrlMonSetID(virResctrlMonPtr mon, + const char *id); + +const char * +virResctrlMonGetID(virResctrlMonPtr mon); + +int +virResctrlMonDeterminePath(virResctrlMonPtr mon, + const char *machinename); + +int +virResctrlMonAddPID(virResctrlMonPtr alloc, + pid_t pid); + +int +virResctrlMonCreate(virResctrlAllocPtr pairedalloc, + virResctrlMonPtr mon, + const char *machinename); + +int +virResctrlMonRemove(virResctrlMonPtr mon); + +bool +virResctrlMonIsRunning(virResctrlMonPtr mon); + +int +virResctrlMonGetCacheOccupancy(virResctrlMonPtr mon, + unsigned int * cacheoccu); + #endif /* __VIR_RESCTRL_H__ */ --=20 2.7.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list