From nobody Tue May 13 18:49:05 2025 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677256516; cv=none; d=zohomail.com; s=zohoarc; b=YXjwDLgh98JnzdKmR0GRByGHW6HaYGLAeNxJA9+uc61gMaB8KoBjI+LsdZ6k5B/wDYP86lmUMxFQWYXpQD8h+LjmAx8NAaEc0fIZ3qX30aYnyXNHvlc5kR4hMv13hPfQw9qCWzHSZvJnKE4SqctqQY6YLGbbJHiY92LAHFk7upE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677256516; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BdfE0nCxLA0iEBNRdfCG3ox5I4m2523p5+WhZcH/Tzs=; b=aLCNNaNOhoSv8ApzL/Dvk2R0a1gvy37ZBeYG5EkoanuUCjkcNgri1VWm43/C1D+BHaRf+5M21auzpUIHJl/o1JhZJW3U/FzDCWjp8es17l5+58gtjvq6BjQzLny4Pp7jGCV3+IK9P32lvdodSC8VbDz+PzznVJ7ivESAkIMobUw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677256516127329.7711729264836; Fri, 24 Feb 2023 08:35:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQB-00061J-CQ; Fri, 24 Feb 2023 10:55:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPn-0005cZ-7F for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPg-0005sL-DL for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:14 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-311-CBFGFpjNNOycquzJM8dPCA-1; Fri, 24 Feb 2023 10:55:08 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2554F1C05B02; Fri, 24 Feb 2023 15:55:07 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 478BCC15BA0; Fri, 24 Feb 2023 15:55:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254111; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BdfE0nCxLA0iEBNRdfCG3ox5I4m2523p5+WhZcH/Tzs=; b=HUMqf3jhzQdDNFz12RLuZoNMjKtSl64XiSEnkRc8V1KS67cm2WB/cO6SR6gNAWJkEi7lVU ntXJhN/SV4dYQXTZsr6VBZ4ZVcNbe+cqYEQSkDPmsWv+ZbnjVmNI6yX05h5R4FA8NbTQ2a Vc8YnS38kORwHL7MyzoWRg+RH5OwvoI= X-MC-Unique: CBFGFpjNNOycquzJM8dPCA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 07/15] vdpa: add vhost_vdpa_suspend Date: Fri, 24 Feb 2023 16:54:30 +0100 Message-Id: <20230224155438.112797-8-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer2=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677256518329100003 The function vhost.c:vhost_dev_stop fetches the vring base so the vq state can be migrated to other devices. However, this is unreliable in vdpa, since we didn't signal the device to suspend the queues, making the value fetched useless. Suspend the device if possible before fetching first and subsequent vring bases. Moreover, vdpa totally reset and wipes the device at the last device before fetch its vrings base, making that operation useless in the last device. This will be fixed in later patches of this series. Signed-off-by: Eugenio P=C3=A9rez --- v4: * Look for _F_SUSPEND at vhost_dev->backend_cap, not backend_features * Fall back on reset & fetch used idx from guest's memory --- hw/virtio/vhost-vdpa.c | 25 +++++++++++++++++++++++++ hw/virtio/trace-events | 1 + 2 files changed, 26 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 228677895a..f542960a64 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -712,6 +712,7 @@ static int vhost_vdpa_reset_device(struct vhost_dev *de= v) =20 ret =3D vhost_vdpa_call(dev, VHOST_VDPA_SET_STATUS, &status); trace_vhost_vdpa_reset_device(dev, status); + v->suspended =3D false; return ret; } =20 @@ -1109,6 +1110,29 @@ static void vhost_vdpa_svqs_stop(struct vhost_dev *d= ev) } } =20 +static void vhost_vdpa_suspend(struct vhost_dev *dev) +{ + struct vhost_vdpa *v =3D dev->opaque; + int r; + + if (!vhost_vdpa_first_dev(dev)) { + return; + } + + if (!(dev->backend_cap & BIT_ULL(VHOST_BACKEND_F_SUSPEND))) { + trace_vhost_vdpa_suspend(dev); + r =3D ioctl(v->device_fd, VHOST_VDPA_SUSPEND); + if (unlikely(r)) { + error_report("Cannot suspend: %s(%d)", g_strerror(errno), errn= o); + } else { + v->suspended =3D true; + return; + } + } + + vhost_vdpa_reset_device(dev); +} + static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) { struct vhost_vdpa *v =3D dev->opaque; @@ -1123,6 +1147,7 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev= , bool started) } vhost_vdpa_set_vring_ready(dev); } else { + vhost_vdpa_suspend(dev); vhost_vdpa_svqs_stop(dev); vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); } diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index a87c5f39a2..8f8d05cf9b 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -50,6 +50,7 @@ vhost_vdpa_set_vring_ready(void *dev) "dev: %p" vhost_vdpa_dump_config(void *dev, const char *line) "dev: %p %s" vhost_vdpa_set_config(void *dev, uint32_t offset, uint32_t size, uint32_t = flags) "dev: %p offset: %"PRIu32" size: %"PRIu32" flags: 0x%"PRIx32 vhost_vdpa_get_config(void *dev, void *config, uint32_t config_len) "dev: = %p config: %p config_len: %"PRIu32 +vhost_vdpa_suspend(void *dev) "dev: %p" vhost_vdpa_dev_start(void *dev, bool started) "dev: %p started: %d" vhost_vdpa_set_log_base(void *dev, uint64_t base, unsigned long long size,= int refcnt, int fd, void *log) "dev: %p base: 0x%"PRIx64" size: %llu refcn= t: %d fd: %d log: %p" vhost_vdpa_set_vring_addr(void *dev, unsigned int index, unsigned int flag= s, uint64_t desc_user_addr, uint64_t used_user_addr, uint64_t avail_user_ad= dr, uint64_t log_guest_addr) "dev: %p index: %u flags: 0x%x desc_user_addr:= 0x%"PRIx64" used_user_addr: 0x%"PRIx64" avail_user_addr: 0x%"PRIx64" log_g= uest_addr: 0x%"PRIx64 --=20 2.31.1