From nobody Sun Jul 13 04:18:28 2025 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1665161036; cv=none; d=zohomail.com; s=zohoarc; b=jgkX6qkvS9+rypn6mRuiU6mLvzOCskDNIp6bLhmAOymnfKW0gYorVD5TTZFxgMbmItBsdZvFarwB8BIqW75RcBOk99m5APDJSfyuUD/2hKUnmYGxIMbq+E2qoQRYH/I/N3L0t0F8NhZnajSSI0KHgkkUxrrBjScIamc8dRzjhXE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665161036; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=KfRjA7bLXInSsF6EtXJzj8gO3KQU8Cidyvo/Oc+B5Rg=; b=Hey2j7RvBvnrhqdTR/mgXwHlYlkGoACGGh0hn8b9iQHLF6ThrB+s/pUcSFmi3P47PB9aFNUANZaSanbe8KQJFSoCvjQUX5SSli0bPRBjblkmY0KCEENAk/2hMRG+TiLd1qX4qKpLuZnx2ZINB4bSXEudVLCrhgW8pOOAlBtzQ4w= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665161036807708.286107652749; Fri, 7 Oct 2022 09:43:56 -0700 (PDT) Received: from localhost ([::1]:47368 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ogqS2-000309-QK for importer2@patchew.org; Fri, 07 Oct 2022 12:43:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38442) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogpCh-0004N7-UL for qemu-devel@nongnu.org; Fri, 07 Oct 2022 11:23:59 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]:2706) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogpCf-0005vl-Tj for qemu-devel@nongnu.org; Fri, 07 Oct 2022 11:23:59 -0400 Received: from fraeml745-chm.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4MkXBj3vx4z67xX3; Fri, 7 Oct 2022 23:23:25 +0800 (CST) Received: from lhrpeml500005.china.huawei.com (7.191.163.240) by fraeml745-chm.china.huawei.com (10.206.15.226) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 7 Oct 2022 17:23:56 +0200 Received: from SecurePC-101-06.china.huawei.com (10.122.247.231) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 7 Oct 2022 16:23:55 +0100 To: , Michael Tsirkin , Ben Widawsky , , Huai-Cheng Kuo , Chris Browy CC: , Subject: [PATCH v7 4/5] hw/mem/cxl-type3: Add CXL CDAT Data Object Exchange Date: Fri, 7 Oct 2022 16:21:55 +0100 Message-ID: <20221007152156.24883-5-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221007152156.24883-1-Jonathan.Cameron@huawei.com> References: <20221007152156.24883-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.122.247.231] X-ClientProxiedBy: lhrpeml100002.china.huawei.com (7.191.160.241) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=jonathan.cameron@huawei.com; helo=frasgout.his.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" Reply-to: Jonathan Cameron From: Jonathan Cameron via X-ZM-MESSAGEID: 1665161037859100001 Content-Type: text/plain; charset="utf-8" From: Huai-Cheng Kuo The CDAT can be specified in two ways. One is to add ",cdat=3D" in "-device cxl-type3"'s command option. The file is required to provide the whole CDAT table in binary mode. The other is to use the default that provides some 'reasonable' numbers based on type of memory and size. The DOE capability supporting CDAT is added to hw/mem/cxl_type3.c with capability offset 0x190. The config read/write to this capability range can be generated in the OS to request the CDAT data. Signed-off-by: Huai-Cheng Kuo Signed-off-by: Chris Browy Signed-off-by: Jonathan Cameron -- Changes since RFC: - Break out type 3 user of library as separate patch. - Change reported data for default to be based on the options provided for the type 3 device. --- hw/mem/cxl_type3.c | 227 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 227 insertions(+) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 568c9d62f5..3fa5d70662 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -12,9 +12,218 @@ #include "qemu/range.h" #include "qemu/rcu.h" #include "sysemu/hostmem.h" +#include "sysemu/numa.h" #include "hw/cxl/cxl.h" #include "hw/pci/msix.h" =20 +#define DWORD_BYTE 4 + +static int ct3_build_cdat_table(CDATSubHeader ***cdat_table, + void *priv) +{ + g_autofree CDATDsmas *dsmas_nonvolatile =3D NULL; + g_autofree CDATDslbis *dslbis_nonvolatile =3D NULL; + g_autofree CDATDsemts *dsemts_nonvolatile =3D NULL; + CXLType3Dev *ct3d =3D priv; + int len =3D 0; + int i =3D 0; + int next_dsmad_handle =3D 0; + int nonvolatile_dsmad =3D -1; + int dslbis_nonvolatile_num =3D 4; + MemoryRegion *mr; + + /* Non volatile aspects */ + if (ct3d->hostmem) { + dsmas_nonvolatile =3D g_malloc(sizeof(*dsmas_nonvolatile)); + if (!dsmas_nonvolatile) { + return -ENOMEM; + } + nonvolatile_dsmad =3D next_dsmad_handle++; + mr =3D host_memory_backend_get_memory(ct3d->hostmem); + if (!mr) { + return -EINVAL; + } + *dsmas_nonvolatile =3D (CDATDsmas) { + .header =3D { + .type =3D CDAT_TYPE_DSMAS, + .length =3D sizeof(*dsmas_nonvolatile), + }, + .DSMADhandle =3D nonvolatile_dsmad, + .flags =3D CDAT_DSMAS_FLAG_NV, + .DPA_base =3D 0, + .DPA_length =3D int128_get64(mr->size), + }; + len++; + + /* For now, no memory side cache, plausiblish numbers */ + dslbis_nonvolatile =3D g_malloc(sizeof(*dslbis_nonvolatile) * dslb= is_nonvolatile_num); + if (!dslbis_nonvolatile) + return -ENOMEM; + + dslbis_nonvolatile[0] =3D (CDATDslbis) { + .header =3D { + .type =3D CDAT_TYPE_DSLBIS, + .length =3D sizeof(*dslbis_nonvolatile), + }, + .handle =3D nonvolatile_dsmad, + .flags =3D HMAT_LB_MEM_MEMORY, + .data_type =3D HMAT_LB_DATA_READ_LATENCY, + .entry_base_unit =3D 10000, /* 10ns base */ + .entry[0] =3D 15, /* 150ns */ + }; + len++; + + dslbis_nonvolatile[1] =3D (CDATDslbis) { + .header =3D { + .type =3D CDAT_TYPE_DSLBIS, + .length =3D sizeof(*dslbis_nonvolatile), + }, + .handle =3D nonvolatile_dsmad, + .flags =3D HMAT_LB_MEM_MEMORY, + .data_type =3D HMAT_LB_DATA_WRITE_LATENCY, + .entry_base_unit =3D 10000, + .entry[0] =3D 25, /* 250ns */ + }; + len++; + =20 + dslbis_nonvolatile[2] =3D (CDATDslbis) { + .header =3D { + .type =3D CDAT_TYPE_DSLBIS, + .length =3D sizeof(*dslbis_nonvolatile), + }, + .handle =3D nonvolatile_dsmad, + .flags =3D HMAT_LB_MEM_MEMORY, + .data_type =3D HMAT_LB_DATA_READ_BANDWIDTH, + .entry_base_unit =3D 1000, /* GB/s */ + .entry[0] =3D 16, + }; + len++; + + dslbis_nonvolatile[3] =3D (CDATDslbis) { + .header =3D { + .type =3D CDAT_TYPE_DSLBIS, + .length =3D sizeof(*dslbis_nonvolatile), + }, + .handle =3D nonvolatile_dsmad, + .flags =3D HMAT_LB_MEM_MEMORY, + .data_type =3D HMAT_LB_DATA_WRITE_BANDWIDTH, + .entry_base_unit =3D 1000, /* GB/s */ + .entry[0] =3D 16, + }; + len++; + + mr =3D host_memory_backend_get_memory(ct3d->hostmem); + if (!mr) { + return -EINVAL; + } + dsemts_nonvolatile =3D g_malloc(sizeof(*dsemts_nonvolatile)); + *dsemts_nonvolatile =3D (CDATDsemts) { + .header =3D { + .type =3D CDAT_TYPE_DSEMTS, + .length =3D sizeof(*dsemts_nonvolatile), + }, + .DSMAS_handle =3D nonvolatile_dsmad, + .EFI_memory_type_attr =3D 2, /* Reserved - the non volatile fr= om DSMAS matters */ + .DPA_offset =3D 0, + .DPA_length =3D int128_get64(mr->size), + }; + len++; + } + + *cdat_table =3D g_malloc0(len * sizeof(*cdat_table)); + /* Header always at start of structure */ + if (dsmas_nonvolatile) { + (*cdat_table)[i++] =3D g_steal_pointer(&dsmas_nonvolatile); + } + if (dslbis_nonvolatile) { + CDATDslbis *dslbis =3D g_steal_pointer(&dslbis_nonvolatile); = =20 + int j; + + for (j =3D 0; j < dslbis_nonvolatile_num; j++) { + (*cdat_table)[i++] =3D (CDATSubHeader *)&dslbis[j]; + } + } + if (dsemts_nonvolatile) { + (*cdat_table)[i++] =3D g_steal_pointer(&dsemts_nonvolatile); + } + =20 + return len; +} + +static void ct3_free_cdat_table(CDATSubHeader **cdat_table, int num, void = *priv) +{ + int i; + + for (i =3D 0; i < num; i++) { + g_free(cdat_table[i]); + } + g_free(cdat_table); +} + +static bool cxl_doe_cdat_rsp(DOECap *doe_cap) +{ + CDATObject *cdat =3D &CXL_TYPE3(doe_cap->pdev)->cxl_cstate.cdat; + uint16_t ent; + void *base; + uint32_t len; + CDATReq *req =3D pcie_doe_get_write_mbox_ptr(doe_cap); + CDATRsp rsp; + + assert(cdat->entry_len); + + /* Discard if request length mismatched */ + if (pcie_doe_get_obj_len(req) < + DIV_ROUND_UP(sizeof(CDATReq), DWORD_BYTE)) { + return false; + } + + ent =3D req->entry_handle; + base =3D cdat->entry[ent].base; + len =3D cdat->entry[ent].length; + + rsp =3D (CDATRsp) { + .header =3D { + .vendor_id =3D CXL_VENDOR_ID, + .data_obj_type =3D CXL_DOE_TABLE_ACCESS, + .reserved =3D 0x0, + .length =3D DIV_ROUND_UP((sizeof(rsp) + len), DWORD_BYTE), + }, + .rsp_code =3D CXL_DOE_TAB_RSP, + .table_type =3D CXL_DOE_TAB_TYPE_CDAT, + .entry_handle =3D (ent < cdat->entry_len - 1) ? + ent + 1 : CXL_DOE_TAB_ENT_MAX, + }; + + memcpy(doe_cap->read_mbox, &rsp, sizeof(rsp)); + memcpy(doe_cap->read_mbox + DIV_ROUND_UP(sizeof(rsp), DWORD_BYTE), + base, len); + + doe_cap->read_mbox_len +=3D rsp.header.length; + + return true; +} + +static uint32_t ct3d_config_read(PCIDevice *pci_dev, uint32_t addr, int si= ze) +{ + CXLType3Dev *ct3d =3D CXL_TYPE3(pci_dev); + uint32_t val; + + if (pcie_doe_read_config(&ct3d->doe_cdat, addr, size, &val)) { + return val; + } + + return pci_default_read_config(pci_dev, addr, size); +} + +static void ct3d_config_write(PCIDevice *pci_dev, uint32_t addr, uint32_t = val, + int size) +{ + CXLType3Dev *ct3d =3D CXL_TYPE3(pci_dev); + + pcie_doe_write_config(&ct3d->doe_cdat, addr, val, size); + pci_default_write_config(pci_dev, addr, val, size); +} + /* * Null value of all Fs suggested by IEEE RA guidelines for use of * EU, OUI and CID @@ -140,6 +349,11 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error = **errp) return true; } =20 +static DOEProtocol doe_cdat_prot[] =3D { + { CXL_VENDOR_ID, CXL_DOE_TABLE_ACCESS, cxl_doe_cdat_rsp }, + { } +}; + static void ct3_realize(PCIDevice *pci_dev, Error **errp) { CXLType3Dev *ct3d =3D CXL_TYPE3(pci_dev); @@ -189,6 +403,14 @@ static void ct3_realize(PCIDevice *pci_dev, Error **er= rp) for (i =3D 0; i < msix_num; i++) { msix_vector_use(pci_dev, i); } + + /* DOE Initailization */ + pcie_doe_init(pci_dev, &ct3d->doe_cdat, 0x190, doe_cdat_prot, true, 0); + + cxl_cstate->cdat.build_cdat_table =3D ct3_build_cdat_table; + cxl_cstate->cdat.free_cdat_table =3D ct3_free_cdat_table; + cxl_cstate->cdat.private =3D ct3d; + cxl_doe_cdat_init(cxl_cstate, errp); } =20 static void ct3_exit(PCIDevice *pci_dev) @@ -197,6 +419,7 @@ static void ct3_exit(PCIDevice *pci_dev) CXLComponentState *cxl_cstate =3D &ct3d->cxl_cstate; ComponentRegisters *regs =3D &cxl_cstate->crb; =20 + cxl_doe_cdat_release(cxl_cstate); g_free(regs->special_ops); address_space_destroy(&ct3d->hostmem_as); } @@ -296,6 +519,7 @@ static Property ct3_props[] =3D { DEFINE_PROP_LINK("lsa", CXLType3Dev, lsa, TYPE_MEMORY_BACKEND, HostMemoryBackend *), DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL), + DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename), DEFINE_PROP_END_OF_LIST(), }; =20 @@ -361,6 +585,9 @@ static void ct3_class_init(ObjectClass *oc, void *data) pc->device_id =3D 0xd93; /* LVF for now */ pc->revision =3D 1; =20 + pc->config_write =3D ct3d_config_write; + pc->config_read =3D ct3d_config_read; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); dc->desc =3D "CXL PMEM Device (Type 3)"; dc->reset =3D ct3d_reset; --=20 2.37.2