From nobody Mon May 6 13:22:19 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1679034552; cv=none; d=zohomail.com; s=zohoarc; b=mDOVbM3x4WnqgFqRDnUmRmaiAb29ZsGpAmUbUGOIYklGJD36ZxK0W/RwKu7dMKX0fo+6xDvVFyiAUFkTLYG93qQviu921iBV0QN4t/GABi72QpLAMzpT1c4Bmu/LW50cEQ5KpDTZm+pF5pfsd/eeVZffwe/7NhYNAnJAAT6Wyfo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1679034552; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DUaA0w6Hlg4CnGtrq1U81HdQH9x+ppG61WKlaTQh/wY=; b=VMFD8OrRj1neRkbX5MvZ4vuXhMGaF3ttx+5FG9LJlPurhw/moEFOwWuKnq/+ODph2VrjQMD9+CpEy5Mc38CMuhFTT+GZyWa20YBbEgjoQXXs0pc4Kr6/ER2af7ZzccCF66IO+fsUaGBm0nhDA/HN71BqZSISFLkVF1pH0UyiFHY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 167903455200517.398062749021392; Thu, 16 Mar 2023 23:29:12 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pd3YX-0001gT-Dd; Fri, 17 Mar 2023 02:27:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd3YS-0001fA-8x for qemu-devel@nongnu.org; Fri, 17 Mar 2023 02:27:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd3YP-0002tQ-JJ for qemu-devel@nongnu.org; Fri, 17 Mar 2023 02:27:08 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-279-gyotC7bbNkuTC8Ti9_PSZw-1; Fri, 17 Mar 2023 02:26:58 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 46F5D3822DF6; Fri, 17 Mar 2023 06:26:57 +0000 (UTC) Received: from gshan.redhat.com (ovpn-12-80.pek2.redhat.com [10.72.12.80]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6CD042166B26; Fri, 17 Mar 2023 06:26:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679034425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DUaA0w6Hlg4CnGtrq1U81HdQH9x+ppG61WKlaTQh/wY=; b=JuWnLdWB8AFy5C9zejHkbtCab/Ifr1l/GVnfjzlLjTkKvke2zyn6Bqxf3Jt8+sjm6mt3sm B+lQXt2E9a+GumthLz54MfiN4dwDY9zcBoksOzU9qnesU4ggQU5KdaKZKDHKrTc/SwOJU3 YRicO5T/mlSxZLq6CekqR9MKm2dqGoI= X-MC-Unique: gyotC7bbNkuTC8Ti9_PSZw-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, qemu-riscv@nongnu.org, rad@semihalf.com, peter.maydell@linaro.org, quic_llindhol@quicinc.com, eduardo@habkost.net, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, thuth@redhat.com, lvivier@redhat.com, pbonzini@redhat.com, imammedo@redhat.com, ajones@ventanamicro.com, berrange@redhat.com, dbarboza@ventanamicro.com, yihyu@redhat.com, shan.gavin@gmail.com Subject: [PATCH v4 1/3] numa: Validate cluster and NUMA node boundary if required Date: Fri, 17 Mar 2023 14:25:40 +0800 Message-Id: <20230317062542.61061-2-gshan@redhat.com> In-Reply-To: <20230317062542.61061-1-gshan@redhat.com> References: <20230317062542.61061-1-gshan@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1679034553813100004 For some architectures like ARM64, multiple CPUs in one cluster can be associated with different NUMA nodes, which is irregular configuration because we shouldn't have this in baremetal environment. The irregular configuration causes Linux guest to misbehave, as the following warning messages indicate. -smp 6,maxcpus=3D6,sockets=3D2,clusters=3D1,cores=3D3,threads=3D1 \ -numa node,nodeid=3D0,cpus=3D0-1,memdev=3Dram0 \ -numa node,nodeid=3D1,cpus=3D2-3,memdev=3Dram1 \ -numa node,nodeid=3D2,cpus=3D4-5,memdev=3Dram2 \ ------------[ cut here ]------------ WARNING: CPU: 0 PID: 1 at kernel/sched/topology.c:2271 build_sched_domain= s+0x284/0x910 Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.14.0-268.el9.aarch64 #1 pstate: 00400005 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=3D--) pc : build_sched_domains+0x284/0x910 lr : build_sched_domains+0x184/0x910 sp : ffff80000804bd50 x29: ffff80000804bd50 x28: 0000000000000002 x27: 0000000000000000 x26: ffff800009cf9a80 x25: 0000000000000000 x24: ffff800009cbf840 x23: ffff000080325000 x22: ffff0000005df800 x21: ffff80000a4ce508 x20: 0000000000000000 x19: ffff000080324440 x18: 0000000000000014 x17: 00000000388925c0 x16: 000000005386a066 x15: 000000009c10cc2e x14: 00000000000001c0 x13: 0000000000000001 x12: ffff00007fffb1a0 x11: ffff00007fffb180 x10: ffff80000a4ce508 x9 : 0000000000000041 x8 : ffff80000a4ce500 x7 : ffff80000a4cf920 x6 : 0000000000000001 x5 : 0000000000000001 x4 : 0000000000000007 x3 : 0000000000000002 x2 : 0000000000001000 x1 : ffff80000a4cf928 x0 : 0000000000000001 Call trace: build_sched_domains+0x284/0x910 sched_init_domains+0xac/0xe0 sched_init_smp+0x48/0xc8 kernel_init_freeable+0x140/0x1ac kernel_init+0x28/0x140 ret_from_fork+0x10/0x20 Improve the situation to warn when multiple CPUs in one cluster have been associated with different NUMA nodes. However, one NUMA node is allowed to be associated with different clusters. Signed-off-by: Gavin Shan Acked-by: Philippe Mathieu-Daud=C3=A9 Acked-by: Alistair Francis Acked-by: Igor Mammedov --- hw/core/machine.c | 42 ++++++++++++++++++++++++++++++++++++++++++ include/hw/boards.h | 1 + 2 files changed, 43 insertions(+) diff --git a/hw/core/machine.c b/hw/core/machine.c index 45e3d24fdc..a2329f975d 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -1255,6 +1255,45 @@ static void machine_numa_finish_cpu_init(MachineStat= e *machine) g_string_free(s, true); } =20 +static void validate_cpu_cluster_to_numa_boundary(MachineState *ms) +{ + MachineClass *mc =3D MACHINE_GET_CLASS(ms); + NumaState *state =3D ms->numa_state; + const CPUArchIdList *possible_cpus =3D mc->possible_cpu_arch_ids(ms); + const CPUArchId *cpus =3D possible_cpus->cpus; + int i, j; + + if (state->num_nodes <=3D 1 || possible_cpus->len <=3D 1) { + return; + } + + /* + * The Linux scheduling domain can't be parsed when the multiple CPUs + * in one cluster have been associated with different NUMA nodes. Howe= ver, + * it's fine to associate one NUMA node with CPUs in different cluster= s. + */ + for (i =3D 0; i < possible_cpus->len; i++) { + for (j =3D i + 1; j < possible_cpus->len; j++) { + if (cpus[i].props.has_socket_id && + cpus[i].props.has_cluster_id && + cpus[i].props.has_node_id && + cpus[j].props.has_socket_id && + cpus[j].props.has_cluster_id && + cpus[j].props.has_node_id && + cpus[i].props.socket_id =3D=3D cpus[j].props.socket_id && + cpus[i].props.cluster_id =3D=3D cpus[j].props.cluster_id && + cpus[i].props.node_id !=3D cpus[j].props.node_id) { + warn_report("CPU-%d and CPU-%d in socket-%ld-cluster-%ld " + "have been associated with node-%ld and node-= %ld " + "respectively. It can cause OSes like Linux t= o " + "misbehave", i, j, cpus[i].props.socket_id, + cpus[i].props.cluster_id, cpus[i].props.node_= id, + cpus[j].props.node_id); + } + } + } +} + MemoryRegion *machine_consume_memdev(MachineState *machine, HostMemoryBackend *backend) { @@ -1340,6 +1379,9 @@ void machine_run_board_init(MachineState *machine, co= nst char *mem_path, Error * numa_complete_configuration(machine); if (machine->numa_state->num_nodes) { machine_numa_finish_cpu_init(machine); + if (machine_class->cpu_cluster_has_numa_boundary) { + validate_cpu_cluster_to_numa_boundary(machine); + } } } =20 diff --git a/include/hw/boards.h b/include/hw/boards.h index 6fbbfd56c8..c9793b2789 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -273,6 +273,7 @@ struct MachineClass { bool nvdimm_supported; bool numa_mem_supported; bool auto_enable_numa; + bool cpu_cluster_has_numa_boundary; SMPCompatProps smp_props; const char *default_ram_id; =20 --=20 2.23.0 From nobody Mon May 6 13:22:19 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1679034512; cv=none; d=zohomail.com; s=zohoarc; b=aCBha3fpnki8fktrN2TITr4pCUubLRpYT9Ue3IWmpdX3eFLcWufIkhN5+vEVMr2tWwwPoDn/UtSymiJqRJOdvHG5rS78RNXvxfiOrZaKl2M4FUWyj+pdLfZYqd3RJRwLz2OFTVV9kagtvMu4tVV3TjSCh2RQPYxIek37edxTlps= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1679034512; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=a5MMJrgBa4SWV2ls7Z2cuU5cgtU/tihOqfmEBAzFuCE=; b=np/E8Wslq/24TE/93dTgoPofVwEEQkzfxWhNqTRYOW5Cy8Zn9r52vfijHP61I+QRCFmCgTQ7qO7hg0+1ZZCyoAndM5592lQsCGJd/GYtHsDrsRydYhx/da1eH5HLuzfx6xbjtNQeaUkW2OL23vs41KVQii3aXxsyb8pzn7HzrUU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1679034512933730.6122454832442; Thu, 16 Mar 2023 23:28:32 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pd3Yb-0001kz-Lq; Fri, 17 Mar 2023 02:27:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd3Ya-0001kk-VX for qemu-devel@nongnu.org; Fri, 17 Mar 2023 02:27:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd3YZ-0002uq-KP for qemu-devel@nongnu.org; Fri, 17 Mar 2023 02:27:16 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-190-XrulB8LHNWuD-QZcA88iVw-1; Fri, 17 Mar 2023 02:27:09 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D17713822DE7; Fri, 17 Mar 2023 06:27:08 +0000 (UTC) Received: from gshan.redhat.com (ovpn-12-80.pek2.redhat.com [10.72.12.80]) by smtp.corp.redhat.com (Postfix) with ESMTP id 218B32166B26; Fri, 17 Mar 2023 06:26:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679034435; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a5MMJrgBa4SWV2ls7Z2cuU5cgtU/tihOqfmEBAzFuCE=; b=WKyJ8IAe6jWZsA9+dYxVjGyZj6uWs5I7kFpttEODtHP6zqUqs4QZqr4tlajoMWtyOcxG6D S3VIEoMnJElJAAIDyi37bIPXKiF+0cu/irxuxwFYJTMF5S/RgbYconakJRQ9RmRVH8uQRv XIUCNfIPP/pj3hP2++eLVFtysPKZQ2o= X-MC-Unique: XrulB8LHNWuD-QZcA88iVw-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, qemu-riscv@nongnu.org, rad@semihalf.com, peter.maydell@linaro.org, quic_llindhol@quicinc.com, eduardo@habkost.net, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, thuth@redhat.com, lvivier@redhat.com, pbonzini@redhat.com, imammedo@redhat.com, ajones@ventanamicro.com, berrange@redhat.com, dbarboza@ventanamicro.com, yihyu@redhat.com, shan.gavin@gmail.com Subject: [PATCH v4 2/3] hw/arm: Validate cluster and NUMA node boundary Date: Fri, 17 Mar 2023 14:25:41 +0800 Message-Id: <20230317062542.61061-3-gshan@redhat.com> In-Reply-To: <20230317062542.61061-1-gshan@redhat.com> References: <20230317062542.61061-1-gshan@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1679034513735100001 Content-Type: text/plain; charset="utf-8" There are two ARM machines where NUMA is aware: 'virt' and 'sbsa-ref'. Both of them are required to follow cluster-NUMA-node boundary. To enable the validation to warn about the irregular configuration where multiple CPUs in one cluster have been associated with different NUMA nodes. Signed-off-by: Gavin Shan Acked-by: Igor Mammedov --- hw/arm/sbsa-ref.c | 2 ++ hw/arm/virt.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index 0b93558dde..efb380e7c8 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -864,6 +864,8 @@ static void sbsa_ref_class_init(ObjectClass *oc, void *= data) mc->possible_cpu_arch_ids =3D sbsa_ref_possible_cpu_arch_ids; mc->cpu_index_to_instance_props =3D sbsa_ref_cpu_index_to_props; mc->get_default_cpu_node_id =3D sbsa_ref_get_default_cpu_node_id; + /* platform instead of architectural choice */ + mc->cpu_cluster_has_numa_boundary =3D true; } =20 static const TypeInfo sbsa_ref_info =3D { diff --git a/hw/arm/virt.c b/hw/arm/virt.c index ac626b3bef..b73ac6eabb 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -3030,6 +3030,8 @@ static void virt_machine_class_init(ObjectClass *oc, = void *data) mc->smp_props.clusters_supported =3D true; mc->auto_enable_numa_with_memhp =3D true; mc->auto_enable_numa_with_memdev =3D true; + /* platform instead of architectural choice */ + mc->cpu_cluster_has_numa_boundary =3D true; mc->default_ram_id =3D "mach-virt.ram"; =20 object_class_property_add(oc, "acpi", "OnOffAuto", --=20 2.23.0 From nobody Mon May 6 13:22:19 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1679034552; cv=none; d=zohomail.com; s=zohoarc; b=WXGljUdB7JbM15a7T43LHoJNED2XDULOwkYBZstxjkTg3sdyY6R3P+ud+fWOFM10EwTTgFSiGcOv0I6oB7b+KiL1UNTbUPaQfrAwEQvU9UDqwHvMC0mMqeTZ3iFFic3CnucDlDabsripUcO1oqGcBXf/zxDxG2CnG8P21706KNw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1679034552; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SMTpAH1VNIj8/8DM28+aXu+qVXcaJh5NWbLcUgKEflQ=; b=SkTlWU7SZ2SiWPvR06fn+O3AKaaLV8e03Rx8B6Y2K9dE5AqLf5WR/db1FmoEUPCpW3cCtql4A7z2YD7JOmM1jhAqwlcqRcz/HevZ/p91ULraR5IpkfCMr93OLBMvW9k1w9tEauPQrpfechX9BjYq0RpNZm1sYN7jfOCRzEi3JmY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 167903455215817.494577623859755; Thu, 16 Mar 2023 23:29:12 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pd3Yo-0001qi-PG; Fri, 17 Mar 2023 02:27:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd3Yn-0001pq-8n for qemu-devel@nongnu.org; Fri, 17 Mar 2023 02:27:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd3Yk-0002w9-U9 for qemu-devel@nongnu.org; Fri, 17 Mar 2023 02:27:29 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-19-Tg79abGfMHGrLApvxnAlBA-1; Fri, 17 Mar 2023 02:27:20 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C742185A588; Fri, 17 Mar 2023 06:27:19 +0000 (UTC) Received: from gshan.redhat.com (ovpn-12-80.pek2.redhat.com [10.72.12.80]) by smtp.corp.redhat.com (Postfix) with ESMTP id A2F222166B26; Fri, 17 Mar 2023 06:27:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679034446; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SMTpAH1VNIj8/8DM28+aXu+qVXcaJh5NWbLcUgKEflQ=; b=R+8N30OUFg3f91d8t7AdbeNbNDeptLZU7ehrPOQIpgzNiGtz2LSpLuZ7FU3toohUMW1mXv mtDMRBIts8pxBGlEKs56wLVUJJlyELs2BMKgaVKzDxi4uII3sbVI47+NlgDQiPT0IV4hdB 6VRPT7zqWlaiVLjW6lD4o1iHvvkw1hk= X-MC-Unique: Tg79abGfMHGrLApvxnAlBA-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, qemu-riscv@nongnu.org, rad@semihalf.com, peter.maydell@linaro.org, quic_llindhol@quicinc.com, eduardo@habkost.net, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, thuth@redhat.com, lvivier@redhat.com, pbonzini@redhat.com, imammedo@redhat.com, ajones@ventanamicro.com, berrange@redhat.com, dbarboza@ventanamicro.com, yihyu@redhat.com, shan.gavin@gmail.com Subject: [PATCH v4 3/3] hw/riscv: Validate cluster and NUMA node boundary Date: Fri, 17 Mar 2023 14:25:42 +0800 Message-Id: <20230317062542.61061-4-gshan@redhat.com> In-Reply-To: <20230317062542.61061-1-gshan@redhat.com> References: <20230317062542.61061-1-gshan@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1679034553788100003 Content-Type: text/plain; charset="utf-8" There are two RISCV machines where NUMA is aware: 'virt' and 'spike'. Both of them are required to follow cluster-NUMA-node boundary. To enable the validation to warn about the irregular configuration where multiple CPUs in one cluster has been associated with multiple NUMA nodes. Signed-off-by: Gavin Shan Reviewed-by: Daniel Henrique Barboza Acked-by: Alistair Francis Acked-by: Igor Mammedov --- hw/riscv/spike.c | 2 ++ hw/riscv/virt.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c index a584d5b3a2..4bf783884b 100644 --- a/hw/riscv/spike.c +++ b/hw/riscv/spike.c @@ -349,6 +349,8 @@ static void spike_machine_class_init(ObjectClass *oc, v= oid *data) mc->cpu_index_to_instance_props =3D riscv_numa_cpu_index_to_props; mc->get_default_cpu_node_id =3D riscv_numa_get_default_cpu_node_id; mc->numa_mem_supported =3D true; + /* platform instead of architectural choice */ + mc->cpu_cluster_has_numa_boundary =3D true; mc->default_ram_id =3D "riscv.spike.ram"; } =20 diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 4e3efbee16..84a2bca460 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -1678,6 +1678,8 @@ static void virt_machine_class_init(ObjectClass *oc, = void *data) mc->cpu_index_to_instance_props =3D riscv_numa_cpu_index_to_props; mc->get_default_cpu_node_id =3D riscv_numa_get_default_cpu_node_id; mc->numa_mem_supported =3D true; + /* platform instead of architectural choice */ + mc->cpu_cluster_has_numa_boundary =3D true; mc->default_ram_id =3D "riscv_virt_board.ram"; assert(!mc->get_hotplug_handler); mc->get_hotplug_handler =3D virt_machine_get_hotplug_handler; --=20 2.23.0