From nobody Fri Mar 29 09:21:21 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=oracle.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1541415619363171.75148600414752; Mon, 5 Nov 2018 03:00:19 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B574B1556B; Mon, 5 Nov 2018 11:00:16 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BB7E7611C3; Mon, 5 Nov 2018 11:00:15 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3E9724BB79; Mon, 5 Nov 2018 11:00:09 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wA5AoeUV016160 for ; Mon, 5 Nov 2018 05:50:40 -0500 Received: by smtp.corp.redhat.com (Postfix) id DB3684BF; Mon, 5 Nov 2018 10:50:40 +0000 (UTC) Received: from mx1.redhat.com (ext-mx01.extmail.prod.ext.phx2.redhat.com [10.5.110.25]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3DE8E18228; Mon, 5 Nov 2018 10:50:35 +0000 (UTC) Received: from userp2130.oracle.com (userp2130.oracle.com [156.151.31.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AFBCD81DE1; Mon, 5 Nov 2018 10:50:33 +0000 (UTC) Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wA5AoCM0160394; Mon, 5 Nov 2018 10:50:32 GMT Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2nh33tp0ev-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 05 Nov 2018 10:50:32 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wA5AoVTd026802 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Nov 2018 10:50:31 GMT Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wA5AoU4k002484; Mon, 5 Nov 2018 10:50:30 GMT Received: from localhost.localdomain (/77.138.186.148) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 05 Nov 2018 02:50:30 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id; s=corp-2018-07-02; bh=YT+EbXp1M9KaEwHmOm9y2wOUwJiUygN/fNjRorvVnkA=; b=m6MMGV8dSDP3YHEQQ8a3Ml6kM9xhbosWAvYCQ+YH2ZEpMV8iNXrSAcrvZIcHg+74mucm ZC0zqYlKsQgX4CwjJz8JhMKygXmLIh4hHblPJAMZWsK7CxGejMsLJtzOwM6qxDTa0CeY fV+LjurpuJk0i2OJiCBldiWSye3+UJBGElp0i5ejSdxe6ldyTSzprNVJd8eF/RhJ6Q2p sn9bSJrEFGmgBPC9mubL2mg0cZnhey42zuTUDu1iFIkw1ygWRaN+/2jwJ7ADOwvSaaVS UpZeCi87ogH0WwiE9Un2uRJSVME6p5oMpj7nu7TB9sIbyGbb5TCBBpTv/+cxnYL3xKoa fg== From: Yuval Shaia To: libvir-list@redhat.com, berrange@redhat.com, laine@redhat.com, mprivozn@redhat.com, marcel.apfelbaum@gmail.com Date: Mon, 5 Nov 2018 12:50:02 +0200 Message-Id: <20181105105002.4998-1-yuval.shaia@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9067 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1811050101 X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 216 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 05 Nov 2018 10:50:34 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 05 Nov 2018 10:50:34 +0000 (UTC) for IP:'156.151.31.86' DOMAIN:'userp2130.oracle.com' HELO:'userp2130.oracle.com' FROM:'yuval.shaia@oracle.com' RCPT:'' X-RedHat-Spam-Score: -103.833 (DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_MED, SPF_HELO_PASS, SPF_PASS, UNPARSEABLE_RELAY, USER_IN_WHITELIST) 156.151.31.86 userp2130.oracle.com 156.151.31.86 userp2130.oracle.com X-Scanned-By: MIMEDefang 2.83 on 10.5.110.25 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Yuval Shaia Subject: [libvirt] [PATCH RESEND] qemu: Process RDMA GID state change event X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 05 Nov 2018 11:00:18 +0000 (UTC) X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This event is emitted on the monitor when a GID table in pvrdma device is modified and the change needs to be propagate to the backend RDMA device's GID table. The control over the RDMA device's GID table is done by updating the device's Ethernet function addresses. Usually the first GID entry is determine by the MAC address, the second by the first IPv6 address and the third by the IPv4 address. Other entries can be added by adding more IP addresses. The opposite is the same, i.e. whenever an address is removed, the corresponding GID entry is removed. The process is done by the network and RDMA stacks. Whenever an address is added the ib_core driver is notified and calls the device driver's add_gid function which in turn update the device. To support this in pvrdma device we need to hook into the create_bind and destroy_bind HW commands triggered by pvrdma driver in guest. Whenever a changed is made to the pvrdma device's GID table a special QMP messages is sent to be processed by libvirt to update the address of the backend Ethernet device. Signed-off-by: Yuval Shaia --- src/qemu/qemu_domain.c | 3 ++ src/qemu/qemu_domain.h | 15 +++++++++ src/qemu/qemu_driver.c | 40 ++++++++++++++++++++++++ src/qemu/qemu_monitor.c | 28 +++++++++++++++++ src/qemu/qemu_monitor.h | 13 ++++++++ src/qemu/qemu_monitor_json.c | 36 ++++++++++++++++++++++ src/qemu/qemu_process.c | 59 ++++++++++++++++++++++++++++++++++++ 7 files changed, 194 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index ba3fff607a..8da54c7ee9 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -13479,6 +13479,9 @@ qemuProcessEventFree(struct qemuProcessEvent *event) case QEMU_PROCESS_EVENT_GUESTPANIC: qemuMonitorEventPanicInfoFree(event->data); break; + case QEMU_PROCESS_EVENT_RDMA_GID_STATUS_CHANGED: + qemuMonitorEventRdmaGidStatusFree(event->data); + break; case QEMU_PROCESS_EVENT_WATCHDOG: case QEMU_PROCESS_EVENT_DEVICE_DELETED: case QEMU_PROCESS_EVENT_NIC_RX_FILTER_CHANGED: diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 80bd4bde91..1b188843e3 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -478,6 +478,18 @@ struct _qemuDomainVsockPrivate { }; =20 =20 +typedef struct _qemuDomainRdmaGidStatusChangedPrivate qemuDomainRdmaGidSta= tusChangedPrivate; +typedef qemuDomainRdmaGidStatusChangedPrivate *qemuDomainRdmaGidStatusChan= gedPrivatePtr; +struct _qemuDomainRdmaGidStatusChangedPrivate { + virObject parent; + + char *netdev; + bool gid_status; + uint64_t subnet_prefix; + uint64_t interface_id; +}; + + typedef enum { QEMU_PROCESS_EVENT_WATCHDOG =3D 0, QEMU_PROCESS_EVENT_GUESTPANIC, @@ -487,6 +499,7 @@ typedef enum { QEMU_PROCESS_EVENT_BLOCK_JOB, QEMU_PROCESS_EVENT_MONITOR_EOF, QEMU_PROCESS_EVENT_PR_DISCONNECT, + QEMU_PROCESS_EVENT_RDMA_GID_STATUS_CHANGED, =20 QEMU_PROCESS_EVENT_LAST } qemuProcessEventType; @@ -499,6 +512,8 @@ struct qemuProcessEvent { void *data; }; =20 +void qemuMonitorEventRdmaGidStatusFree(qemuDomainRdmaGidStatusChangedPriva= tePtr info); + void qemuProcessEventFree(struct qemuProcessEvent *event); =20 typedef struct _qemuDomainLogContext qemuDomainLogContext; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index a52e2495d5..dc088d844f 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -4788,6 +4788,43 @@ processPRDisconnectEvent(virDomainObjPtr vm) } =20 =20 +static void +processRdmaGidStatusChangedEvent(virDomainObjPtr vm, + qemuDomainRdmaGidStatusChangedPrivatePtr = info) +{ + unsigned int prefix_len; + virSocketAddr addr =3D {0}; + int rc; + + if (!virDomainObjIsActive(vm)) + return; + + VIR_DEBUG("netdev=3D%s,gid_status=3D%d,subnet_prefix=3D0x%lx,interface= _id=3D0x%lx", + info->netdev, info->gid_status, info->subnet_prefix, + info->interface_id); + + if (info->subnet_prefix) { + prefix_len =3D 64; + uint32_t ipv6[4]; + memcpy(&ipv6[0], &info->subnet_prefix, sizeof(info->subnet_prefix)= ); + memcpy(&ipv6[2], &info->interface_id, sizeof(info->subnet_prefix)); + virSocketAddrSetIPv6AddrNetOrder(&addr, ipv6); + } else { + prefix_len =3D 24; + virSocketAddrSetIPv4AddrNetOrder(&addr, info->interface_id >> 32); + } + + if (info->gid_status) + rc =3D virNetDevIPAddrAdd(info->netdev, &addr, NULL, prefix_len); + else + rc =3D virNetDevIPAddrDel(info->netdev, &addr, prefix_len); + + if (rc) + VIR_ERROR(_("Fail to update address 0x%lx to %s"), info->interface= _id, + info->netdev); +} + + static void qemuProcessEventHandler(void *data, void *opaque) { struct qemuProcessEvent *processEvent =3D data; @@ -4828,6 +4865,9 @@ static void qemuProcessEventHandler(void *data, void = *opaque) case QEMU_PROCESS_EVENT_PR_DISCONNECT: processPRDisconnectEvent(vm); break; + case QEMU_PROCESS_EVENT_RDMA_GID_STATUS_CHANGED: + processRdmaGidStatusChangedEvent(vm, processEvent->data); + break; case QEMU_PROCESS_EVENT_LAST: break; } diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 7f7013e115..375ed7ceaf 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -1686,6 +1686,23 @@ qemuMonitorEmitPRManagerStatusChanged(qemuMonitorPtr= mon, } =20 =20 +int +qemuMonitorEmitRdmaGidStatusChanged(qemuMonitorPtr mon, const char *netdev, + bool gid_status, uint64_t subnet_prefi= x, + uint64_t interface_id) +{ + int ret =3D -1; + VIR_DEBUG("mon=3D%p", mon); + VIR_DEBUG("netdev=3D'%s',gid_status=3D%d,subnet_prefix=3D0x%lx,interfa= ce_id=3D0x%lx", + netdev, gid_status, subnet_prefix, interface_id); + + QEMU_MONITOR_CALLBACK(mon, ret, domainRdmaGidStatusChanged, mon->vm, n= etdev, + gid_status, subnet_prefix, interface_id); + + return ret; +} + + int qemuMonitorSetCapabilities(qemuMonitorPtr mon) { @@ -4298,6 +4315,17 @@ qemuMonitorEventPanicInfoFree(qemuMonitorEventPanicI= nfoPtr info) } =20 =20 +void +qemuMonitorEventRdmaGidStatusFree(qemuDomainRdmaGidStatusChangedPrivatePtr= info) +{ + if (!info) + return; + + VIR_FREE(info->netdev); + VIR_FREE(info); +} + + int qemuMonitorSetWatchdogAction(qemuMonitorPtr mon, const char *action) diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 48b142a4f4..f5affe615a 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -281,6 +281,14 @@ typedef int (*qemuMonitorDomainPRManagerStatusChangedC= allback)(qemuMonitorPtr mo bool connec= ted, void *opaqu= e); =20 +typedef int (*qemuMonitorDomainRdmaGidStatusChangedCallback)(qemuMonitorPt= r mon, + virDomainObjP= tr vm, + const char *n= etdev, + bool gid_stat= us, + uint64_t subn= et_prefix, + uint64_t inte= rface_id, + void *opaque); + typedef struct _qemuMonitorCallbacks qemuMonitorCallbacks; typedef qemuMonitorCallbacks *qemuMonitorCallbacksPtr; struct _qemuMonitorCallbacks { @@ -314,6 +322,7 @@ struct _qemuMonitorCallbacks { qemuMonitorDomainBlockThresholdCallback domainBlockThreshold; qemuMonitorDomainDumpCompletedCallback domainDumpCompleted; qemuMonitorDomainPRManagerStatusChangedCallback domainPRManagerStatusC= hanged; + qemuMonitorDomainRdmaGidStatusChangedCallback domainRdmaGidStatusChang= ed; }; =20 char *qemuMonitorEscapeArg(const char *in); @@ -448,6 +457,10 @@ int qemuMonitorEmitPRManagerStatusChanged(qemuMonitorP= tr mon, const char *prManager, bool connected); =20 +int qemuMonitorEmitRdmaGidStatusChanged(qemuMonitorPtr mon, const char *ne= tdev, + bool gid_status, uint64_t subnet_p= refix, + uint64_t interface_id); + int qemuMonitorStartCPUs(qemuMonitorPtr mon); int qemuMonitorStopCPUs(qemuMonitorPtr mon); =20 diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index 3de298c9e2..8df9b426ba 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -91,6 +91,7 @@ static void qemuMonitorJSONHandleAcpiOstInfo(qemuMonitorP= tr mon, virJSONValuePtr static void qemuMonitorJSONHandleBlockThreshold(qemuMonitorPtr mon, virJSO= NValuePtr data); static void qemuMonitorJSONHandleDumpCompleted(qemuMonitorPtr mon, virJSON= ValuePtr data); static void qemuMonitorJSONHandlePRManagerStatusChanged(qemuMonitorPtr mon= , virJSONValuePtr data); +static void qemuMonitorJSONHandleRdmaGidStatusChanged(qemuMonitorPtr mon, = virJSONValuePtr data); =20 typedef struct { const char *type; @@ -114,6 +115,7 @@ static qemuEventHandler eventHandlers[] =3D { { "NIC_RX_FILTER_CHANGED", qemuMonitorJSONHandleNicRxFilterChanged, }, { "POWERDOWN", qemuMonitorJSONHandlePowerdown, }, { "PR_MANAGER_STATUS_CHANGED", qemuMonitorJSONHandlePRManagerStatusCha= nged, }, + { "RDMA_GID_STATUS_CHANGED", qemuMonitorJSONHandleRdmaGidStatusChanged= , }, { "RESET", qemuMonitorJSONHandleReset, }, { "RESUME", qemuMonitorJSONHandleResume, }, { "RTC_CHANGE", qemuMonitorJSONHandleRTCChange, }, @@ -1351,6 +1353,40 @@ static void qemuMonitorJSONHandlePRManagerStatusChan= ged(qemuMonitorPtr mon, } =20 =20 +static void qemuMonitorJSONHandleRdmaGidStatusChanged(qemuMonitorPtr mon, + virJSONValuePtr data) +{ + const char *netdev; + bool gid_status; + unsigned long long subnet_prefix, interface_id; + + if (!(netdev =3D virJSONValueObjectGetString(data, "netdev"))) { + VIR_WARN("missing netdev in GID_STATUS_CHANGED event"); + return; + } + + if (virJSONValueObjectGetBoolean(data, "gid-status", &gid_status)) { + VIR_WARN("missing gid-status in GID_STATUS_CHANGED event"); + return; + } + + if (virJSONValueObjectGetNumberUlong(data, "subnet-prefix", + &subnet_prefix)) { + VIR_WARN("missing subnet-prefix in GID_STATUS_CHANGED event"); + return; + } + + if (virJSONValueObjectGetNumberUlong(data, "interface-id", + &interface_id)) { + VIR_WARN("missing interface-id in GID_STATUS_CHANGED event"); + return; + } + + qemuMonitorEmitRdmaGidStatusChanged(mon, netdev, gid_status, subnet_pr= efix, + interface_id); +} + + int qemuMonitorJSONHumanCommandWithFd(qemuMonitorPtr mon, const char *cmd_str, diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 9cf971808c..45fc155d31 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -1703,6 +1703,64 @@ qemuProcessHandlePRManagerStatusChanged(qemuMonitorP= tr mon ATTRIBUTE_UNUSED, } =20 =20 +static int +qemuProcessHandleRdmaGidStatusChanged(qemuMonitorPtr mon ATTRIBUTE_UNUSED, + virDomainObjPtr vm, const char *netd= ev, + bool gid_status, uint64_t subnet_pre= fix, + uint64_t interface_id, void *opaque) +{ + virQEMUDriverPtr driver =3D opaque; + qemuDomainObjPrivatePtr priv; + struct qemuProcessEvent *processEvent =3D NULL; + qemuDomainRdmaGidStatusChangedPrivatePtr rdmaGitStatusChangedPriv; + int ret =3D -1; + + virObjectLock(vm); + + VIR_DEBUG("netdev=3D%s,gid_status=3D%d,subnet_prefix=3D0x%lx,interface= _id=3D0x%lx", + netdev, gid_status, subnet_prefix, interface_id); + + priv =3D vm->privateData; + priv->prDaemonRunning =3D false; + + if (VIR_ALLOC(rdmaGitStatusChangedPriv) < 0) + goto out_unlock; + + if (VIR_STRDUP(rdmaGitStatusChangedPriv->netdev, netdev) < 0) + goto out_free_rdma_priv; + + rdmaGitStatusChangedPriv->gid_status =3D gid_status; + rdmaGitStatusChangedPriv->subnet_prefix =3D subnet_prefix; + rdmaGitStatusChangedPriv->interface_id =3D interface_id; + + if (VIR_ALLOC(processEvent) < 0) + goto out_free_rdma_priv_netdev; + + processEvent->eventType =3D QEMU_PROCESS_EVENT_RDMA_GID_STATUS_CHANGED; + processEvent->vm =3D virObjectRef(vm); + processEvent->data =3D rdmaGitStatusChangedPriv; + + if (virThreadPoolSendJob(driver->workerPool, 0, processEvent) < 0) { + qemuProcessEventFree(processEvent); + virObjectUnref(vm); + goto out_free_rdma_priv_netdev; + } + + ret =3D 0; + goto out_unlock; + + out_free_rdma_priv_netdev: + VIR_FREE(rdmaGitStatusChangedPriv->netdev); + + out_free_rdma_priv: + VIR_FREE(rdmaGitStatusChangedPriv); + + out_unlock: + virObjectUnlock(vm); + return ret; +} + + static qemuMonitorCallbacks monitorCallbacks =3D { .eofNotify =3D qemuProcessHandleMonitorEOF, .errorNotify =3D qemuProcessHandleMonitorError, @@ -1732,6 +1790,7 @@ static qemuMonitorCallbacks monitorCallbacks =3D { .domainBlockThreshold =3D qemuProcessHandleBlockThreshold, .domainDumpCompleted =3D qemuProcessHandleDumpCompleted, .domainPRManagerStatusChanged =3D qemuProcessHandlePRManagerStatusChan= ged, + .domainRdmaGidStatusChanged =3D qemuProcessHandleRdmaGidStatusChanged, }; =20 static void --=20 2.17.2 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list