From nobody Sat Jul 5 02:20:03 2025 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652932139877765.6674154963786; Wed, 18 May 2022 20:48:59 -0700 (PDT) Received: from localhost ([::1]:53512 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrX9m-0001tf-Kc for importer2@patchew.org; Wed, 18 May 2022 23:48:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nrWyc-0007NM-25 for qemu-devel@nongnu.org; Wed, 18 May 2022 23:37:27 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.222]:54150 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nrWyZ-0008P4-1W for qemu-devel@nongnu.org; Wed, 18 May 2022 23:37:25 -0400 Received: from clientip-36.111.64.84 (unknown [172.18.0.188]) by chinatelecom.cn (HERMES) with SMTP id B813A2800D8; Thu, 19 May 2022 11:37:17 +0800 (CST) Received: from ([172.18.0.188]) by app0023 with ESMTP id 96e80c95898b44fa8ab8a8b901f59087 for qemu-devel@nongnu.org; Thu, 19 May 2022 11:37:20 CST HMM_SOURCE_IP: 172.18.0.188:58888.87199194 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 96e80c95898b44fa8ab8a8b901f59087 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.188 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier , Paolo Bonzini , Hyman Huang Subject: [PATCH v24 7/8] softmmu/dirtylimit: Implement dirty page rate limit Date: Thu, 19 May 2022 11:36:47 +0800 Message-Id: <80195172e377e3d96394b5b94425098260380bfa.1652931128.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.222; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1652932141703100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query dirty page limit for virtual CPU. Meanwhile, introduce corresponding hmp commands "set_vcpu_dirty_limit", "cancel_vcpu_dirty_limit", "info vcpu_dirty_limit" so the feature can be more usable. "query-vcpu-dirty-limit" success depends on enabling dirty page rate limit, so just add it to the list of skipped command to ensure qmp-cmd-test run successfully. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) Acked-by: Markus Armbruster Reviewed-by: Peter Xu --- hmp-commands-info.hx | 13 +++ hmp-commands.hx | 32 ++++++++ include/monitor/hmp.h | 3 + qapi/migration.json | 80 +++++++++++++++++++ softmmu/dirtylimit.c | 194 +++++++++++++++++++++++++++++++++++++++++= ++++ tests/qtest/qmp-cmd-test.c | 2 + 6 files changed, 324 insertions(+) diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx index adfa085..016717d 100644 --- a/hmp-commands-info.hx +++ b/hmp-commands-info.hx @@ -865,6 +865,19 @@ SRST Display the vcpu dirty rate information. ERST =20 + { + .name =3D "vcpu_dirty_limit", + .args_type =3D "", + .params =3D "", + .help =3D "show dirty page limit information of all vCPU", + .cmd =3D hmp_info_vcpu_dirty_limit, + }, + +SRST + ``info vcpu_dirty_limit`` + Display the vcpu dirty page limit information. +ERST + #if defined(TARGET_I386) { .name =3D "sgx", diff --git a/hmp-commands.hx b/hmp-commands.hx index 564f1de..cb688a6 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -1754,3 +1754,35 @@ ERST "\n\t\t\t -b to specify dirty bitmap as method of ca= lculation)", .cmd =3D hmp_calc_dirty_rate, }, + +SRST +``set_vcpu_dirty_limit`` + Set dirty page rate limit on virtual CPU, the information about all the + virtual CPU dirty limit status can be observed with ``info vcpu_dirty_li= mit`` + command. +ERST + + { + .name =3D "set_vcpu_dirty_limit", + .args_type =3D "dirty_rate:l,cpu_index:l?", + .params =3D "dirty_rate [cpu_index]", + .help =3D "set dirty page rate limit, use cpu_index to set l= imit" + "\n\t\t\t\t\t on a specified virtual cpu", + .cmd =3D hmp_set_vcpu_dirty_limit, + }, + +SRST +``cancel_vcpu_dirty_limit`` + Cancel dirty page rate limit on virtual CPU, the information about all t= he + virtual CPU dirty limit status can be observed with ``info vcpu_dirty_li= mit`` + command. +ERST + + { + .name =3D "cancel_vcpu_dirty_limit", + .args_type =3D "cpu_index:l?", + .params =3D "[cpu_index]", + .help =3D "cancel dirty page rate limit, use cpu_index to ca= ncel" + "\n\t\t\t\t\t limit on a specified virtual cpu", + .cmd =3D hmp_cancel_vcpu_dirty_limit, + }, diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h index 96d0148..478820e 100644 --- a/include/monitor/hmp.h +++ b/include/monitor/hmp.h @@ -131,6 +131,9 @@ void hmp_replay_delete_break(Monitor *mon, const QDict = *qdict); void hmp_replay_seek(Monitor *mon, const QDict *qdict); void hmp_info_dirty_rate(Monitor *mon, const QDict *qdict); void hmp_calc_dirty_rate(Monitor *mon, const QDict *qdict); +void hmp_set_vcpu_dirty_limit(Monitor *mon, const QDict *qdict); +void hmp_cancel_vcpu_dirty_limit(Monitor *mon, const QDict *qdict); +void hmp_info_vcpu_dirty_limit(Monitor *mon, const QDict *qdict); void hmp_human_readable_text_helper(Monitor *mon, HumanReadableText *(*qmp_handler)(Erro= r **)); =20 diff --git a/qapi/migration.json b/qapi/migration.json index 6130cd9..9c57cc0 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1884,6 +1884,86 @@ { 'command': 'query-dirty-rate', 'returns': 'DirtyRateInfo' } =20 ## +# @DirtyLimitInfo: +# +# Dirty page rate limit information of a virtual CPU. +# +# @cpu-index: index of a virtual CPU. +# +# @limit-rate: upper limit of dirty page rate (MB/s) for a virtual +# CPU, 0 means unlimited. +# +# @current-rate: current dirty page rate (MB/s) for a virtual CPU. +# +# Since: 7.1 +# +## +{ 'struct': 'DirtyLimitInfo', + 'data': { 'cpu-index': 'int', + 'limit-rate': 'uint64', + 'current-rate': 'uint64' } } + +## +# @set-vcpu-dirty-limit: +# +# Set the upper limit of dirty page rate for virtual CPUs. +# +# Requires KVM with accelerator property "dirty-ring-size" set. +# A virtual CPU's dirty page rate is a measure of its memory load. +# To observe dirty page rates, use @calc-dirty-rate. +# +# @cpu-index: index of a virtual CPU, default is all. +# +# @dirty-rate: upper limit of dirty page rate (MB/s) for virtual CPUs. +# +# Since: 7.1 +# +# Example: +# {"execute": "set-vcpu-dirty-limit"} +# "arguments": { "dirty-rate": 200, +# "cpu-index": 1 } } +# +## +{ 'command': 'set-vcpu-dirty-limit', + 'data': { '*cpu-index': 'int', + 'dirty-rate': 'uint64' } } + +## +# @cancel-vcpu-dirty-limit: +# +# Cancel the upper limit of dirty page rate for virtual CPUs. +# +# Cancel the dirty page limit for the vCPU which has been set with +# set-vcpu-dirty-limit command. Note that this command requires +# support from dirty ring, same as the "set-vcpu-dirty-limit". +# +# @cpu-index: index of a virtual CPU, default is all. +# +# Since: 7.1 +# +# Example: +# {"execute": "cancel-vcpu-dirty-limit"} +# "arguments": { "cpu-index": 1 } } +# +## +{ 'command': 'cancel-vcpu-dirty-limit', + 'data': { '*cpu-index': 'int'} } + +## +# @query-vcpu-dirty-limit: +# +# Returns information about virtual CPU dirty page rate limits, if any. +# +# Since: 7.1 +# +# Example: +# {"execute": "query-vcpu-dirty-limit"} +# +## +{ 'command': 'query-vcpu-dirty-limit', + 'returns': [ 'DirtyLimitInfo' ] } + +## # @snapshot-save: # # Save a VM snapshot diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c index 76d0b44..c5ea241 100644 --- a/softmmu/dirtylimit.c +++ b/softmmu/dirtylimit.c @@ -14,8 +14,12 @@ #include "qapi/error.h" #include "qemu/main-loop.h" #include "qapi/qapi-commands-migration.h" +#include "qapi/qmp/qdict.h" +#include "qapi/error.h" #include "sysemu/dirtyrate.h" #include "sysemu/dirtylimit.h" +#include "monitor/hmp.h" +#include "monitor/monitor.h" #include "exec/memory.h" #include "hw/boards.h" #include "sysemu/kvm.h" @@ -405,3 +409,193 @@ void dirtylimit_vcpu_execute(CPUState *cpu) usleep(cpu->throttle_us_per_full); } } + +static void dirtylimit_init(void) +{ + dirtylimit_state_initialize(); + dirtylimit_change(true); + vcpu_dirty_rate_stat_initialize(); + vcpu_dirty_rate_stat_start(); +} + +static void dirtylimit_cleanup(void) +{ + vcpu_dirty_rate_stat_stop(); + vcpu_dirty_rate_stat_finalize(); + dirtylimit_change(false); + dirtylimit_state_finalize(); +} + +void qmp_cancel_vcpu_dirty_limit(bool has_cpu_index, + int64_t cpu_index, + Error **errp) +{ + if (!kvm_enabled() || !kvm_dirty_ring_enabled()) { + return; + } + + if (has_cpu_index && !dirtylimit_vcpu_index_valid(cpu_index)) { + error_setg(errp, "incorrect cpu index specified"); + return; + } + + if (!dirtylimit_in_service()) { + return; + } + + dirtylimit_state_lock(); + + if (has_cpu_index) { + dirtylimit_set_vcpu(cpu_index, 0, false); + } else { + dirtylimit_set_all(0, false); + } + + if (!dirtylimit_state->limited_nvcpu) { + dirtylimit_cleanup(); + } + + dirtylimit_state_unlock(); +} + +void hmp_cancel_vcpu_dirty_limit(Monitor *mon, const QDict *qdict) +{ + int64_t cpu_index =3D qdict_get_try_int(qdict, "cpu_index", -1); + Error *err =3D NULL; + + qmp_cancel_vcpu_dirty_limit(!!(cpu_index !=3D -1), cpu_index, &err); + if (err) { + hmp_handle_error(mon, err); + return; + } + + monitor_printf(mon, "[Please use 'info vcpu_dirty_limit' to query " + "dirty limit for virtual CPU]\n"); +} + +void qmp_set_vcpu_dirty_limit(bool has_cpu_index, + int64_t cpu_index, + uint64_t dirty_rate, + Error **errp) +{ + if (!kvm_enabled() || !kvm_dirty_ring_enabled()) { + error_setg(errp, "dirty page limit feature requires KVM with" + " accelerator property 'dirty-ring-size' set'"); + return; + } + + if (has_cpu_index && !dirtylimit_vcpu_index_valid(cpu_index)) { + error_setg(errp, "incorrect cpu index specified"); + return; + } + + if (!dirty_rate) { + qmp_cancel_vcpu_dirty_limit(has_cpu_index, cpu_index, errp); + return; + } + + dirtylimit_state_lock(); + + if (!dirtylimit_in_service()) { + dirtylimit_init(); + } + + if (has_cpu_index) { + dirtylimit_set_vcpu(cpu_index, dirty_rate, true); + } else { + dirtylimit_set_all(dirty_rate, true); + } + + dirtylimit_state_unlock(); +} + +void hmp_set_vcpu_dirty_limit(Monitor *mon, const QDict *qdict) +{ + int64_t dirty_rate =3D qdict_get_int(qdict, "dirty_rate"); + int64_t cpu_index =3D qdict_get_try_int(qdict, "cpu_index", -1); + Error *err =3D NULL; + + qmp_set_vcpu_dirty_limit(!!(cpu_index !=3D -1), cpu_index, dirty_rate,= &err); + if (err) { + hmp_handle_error(mon, err); + return; + } + + monitor_printf(mon, "[Please use 'info vcpu_dirty_limit' to query " + "dirty limit for virtual CPU]\n"); +} + +static struct DirtyLimitInfo *dirtylimit_query_vcpu(int cpu_index) +{ + DirtyLimitInfo *info =3D NULL; + + info =3D g_malloc0(sizeof(*info)); + info->cpu_index =3D cpu_index; + info->limit_rate =3D dirtylimit_vcpu_get_state(cpu_index)->quota; + info->current_rate =3D vcpu_dirty_rate_get(cpu_index); + + return info; +} + +static struct DirtyLimitInfoList *dirtylimit_query_all(void) +{ + int i, index; + DirtyLimitInfo *info =3D NULL; + DirtyLimitInfoList *head =3D NULL, **tail =3D &head; + + dirtylimit_state_lock(); + + if (!dirtylimit_in_service()) { + dirtylimit_state_unlock(); + return NULL; + } + + for (i =3D 0; i < dirtylimit_state->max_cpus; i++) { + index =3D dirtylimit_state->states[i].cpu_index; + if (dirtylimit_vcpu_get_state(index)->enabled) { + info =3D dirtylimit_query_vcpu(index); + QAPI_LIST_APPEND(tail, info); + } + } + + dirtylimit_state_unlock(); + + return head; +} + +struct DirtyLimitInfoList *qmp_query_vcpu_dirty_limit(Error **errp) +{ + if (!dirtylimit_in_service()) { + return NULL; + } + + return dirtylimit_query_all(); +} + +void hmp_info_vcpu_dirty_limit(Monitor *mon, const QDict *qdict) +{ + DirtyLimitInfoList *limit, *head, *info =3D NULL; + Error *err =3D NULL; + + if (!dirtylimit_in_service()) { + monitor_printf(mon, "Dirty page limit not enabled!\n"); + return; + } + + info =3D qmp_query_vcpu_dirty_limit(&err); + if (err) { + hmp_handle_error(mon, err); + return; + } + + head =3D info; + for (limit =3D head; limit !=3D NULL; limit =3D limit->next) { + monitor_printf(mon, "vcpu[%"PRIi64"], limit rate %"PRIi64 " (MB/s)= ," + " current rate %"PRIi64 " (MB/s)\n", + limit->value->cpu_index, + limit->value->limit_rate, + limit->value->current_rate); + } + + g_free(info); +} diff --git a/tests/qtest/qmp-cmd-test.c b/tests/qtest/qmp-cmd-test.c index 056b40e..af00712 100644 --- a/tests/qtest/qmp-cmd-test.c +++ b/tests/qtest/qmp-cmd-test.c @@ -110,6 +110,8 @@ static bool query_is_ignored(const char *cmd) "query-sev-capabilities", "query-sgx", "query-sgx-capabilities", + /* Success depends on enabling dirty page rate limit */ + "query-vcpu-dirty-limit", NULL }; int i; --=20 1.8.3.1