From nobody Thu May 15 08:29:35 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1510001602197481.6414448498274; Mon, 6 Nov 2017 12:53:22 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 361805F7AD; Mon, 6 Nov 2017 20:53:17 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1671C5D6A3; Mon, 6 Nov 2017 20:53:17 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6EE7B1800BDC; Mon, 6 Nov 2017 20:53:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id vA6KrEVF002436 for ; Mon, 6 Nov 2017 15:53:14 -0500 Received: by smtp.corp.redhat.com (Postfix) id B9FB2190C3; Mon, 6 Nov 2017 20:53:14 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-90.phx2.redhat.com [10.3.116.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6E8D1190C2; Mon, 6 Nov 2017 20:53:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 361805F7AD Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=libvir-list-bounces@redhat.com From: John Ferlan To: libvir-list@redhat.com Date: Mon, 6 Nov 2017 15:53:08 -0500 Message-Id: <20171106205309.22525-2-jferlan@redhat.com> In-Reply-To: <20171106205309.22525-1-jferlan@redhat.com> References: <20171106205309.22525-1-jferlan@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: cbosdonnat@suse.com Subject: [libvirt] [PATCH 1/2] storage: Resolve storage driver crash X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Mon, 06 Nov 2017 20:53:17 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Resolve a storage driver crash as a result of a long running storageVolCreateXML when the virStorageVolPoolRefreshThread is run as a result of a storageVolUpload complete and ran the virStoragePoolObjClearVols without checking if the creation code was currently processing a buildVol after incrementing the driver->asyncjob count. The refreshThread needs to wait until all creation threads are completed so as to not alter the volume list while the volume is being built. Crash from valgrind is as follows (with a bit of editing): =3D=3D21309=3D=3D Invalid read of size 8 =3D=3D21309=3D=3D at 0x153E47AF: storageBackendUpdateVolTargetInfo =3D=3D21309=3D=3D by 0x153E4C30: virStorageBackendUpdateVolInfo =3D=3D21309=3D=3D by 0x153E52DE: virStorageBackendVolRefreshLocal =3D=3D21309=3D=3D by 0x153DE29E: storageVolCreateXML =3D=3D21309=3D=3D by 0x562035B: virStorageVolCreateXML =3D=3D21309=3D=3D by 0x147366: remoteDispatchStorageVolCreateXML ... =3D=3D21309=3D=3D Address 0x2590a720 is 64 bytes inside a block of size 33= 6 free'd =3D=3D21309=3D=3D at 0x4C2F2BB: free =3D=3D21309=3D=3D by 0x54CB9FA: virFree =3D=3D21309=3D=3D by 0x55BC800: virStorageVolDefFree =3D=3D21309=3D=3D by 0x55BF1D8: virStoragePoolObjClearVols =3D=3D21309=3D=3D by 0x153D967E: virStorageVolPoolRefreshThread ... =3D=3D21309=3D=3D Block was alloc'd at =3D=3D21309=3D=3D at 0x4C300A5: calloc =3D=3D21309=3D=3D by 0x54CB483: virAlloc =3D=3D21309=3D=3D by 0x55BDC1F: virStorageVolDefParseXML =3D=3D21309=3D=3D by 0x55BDC1F: virStorageVolDefParseNode =3D=3D21309=3D=3D by 0x55BE5A4: virStorageVolDefParse =3D=3D21309=3D=3D by 0x153DDFF1: storageVolCreateXML =3D=3D21309=3D=3D by 0x562035B: virStorageVolCreateXML =3D=3D21309=3D=3D by 0x147366: remoteDispatchStorageVolCreateXML ... Signed-off-by: John Ferlan --- src/storage/storage_driver.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c index b0edf9f885..5e920fc14c 100644 --- a/src/storage/storage_driver.c +++ b/src/storage/storage_driver.c @@ -2257,6 +2257,7 @@ virStorageVolPoolRefreshThread(void *opaque) virStorageBackendPtr backend; virObjectEventPtr event =3D NULL; =20 + retry: storageDriverLock(); if (cbdata->vol_path) { if (virStorageBackendPloopRestoreDesc(cbdata->vol_path) < 0) @@ -2270,6 +2271,14 @@ virStorageVolPoolRefreshThread(void *opaque) if (!(backend =3D virStorageBackendForType(def->type))) goto cleanup; =20 + /* Some thread is creating a new volume in the pool, we need to retry = */ + if (virStoragePoolObjGetAsyncjobs(obj) > 0) { + virStoragePoolObjUnlock(obj); + storageDriverUnlock(); + usleep(100 * 1000); + goto retry; + } + virStoragePoolObjClearVols(obj); if (backend->refreshPool(NULL, obj) < 0) VIR_DEBUG("Failed to refresh storage pool"); --=20 2.13.6 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list