From nobody Thu Apr 25 15:21:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=canonical.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1534238872467626.02385511858; Tue, 14 Aug 2018 02:27:52 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 17574307DA3E; Tue, 14 Aug 2018 09:27:50 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C346C308BDBC; Tue, 14 Aug 2018 09:27:49 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6DDB93F7FA; Tue, 14 Aug 2018 09:27:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w7E9RdVt027047 for ; Tue, 14 Aug 2018 05:27:39 -0400 Received: by smtp.corp.redhat.com (Postfix) id 690C2101F94C; Tue, 14 Aug 2018 09:27:39 +0000 (UTC) Received: from mx1.redhat.com (ext-mx20.extmail.prod.ext.phx2.redhat.com [10.5.110.49]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 592D9100190D; Tue, 14 Aug 2018 09:27:39 +0000 (UTC) Received: from youngberry.canonical.com (youngberry.canonical.com [91.189.89.112]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1943F3086248; Tue, 14 Aug 2018 09:27:38 +0000 (UTC) Received: from 1.general.paelzer.uk.vpn ([10.172.196.172] helo=lap.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1fpVc0-00033p-P7; Tue, 14 Aug 2018 09:27:36 +0000 From: Christian Ehrhardt To: Alex Williamson , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , libvir-list@redhat.com Date: Tue, 14 Aug 2018 11:27:33 +0200 Message-Id: <20180814092734.14622-2-christian.ehrhardt@canonical.com> In-Reply-To: <20180814092734.14622-1-christian.ehrhardt@canonical.com> References: <20180814092734.14622-1-christian.ehrhardt@canonical.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 212 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Tue, 14 Aug 2018 09:27:38 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Tue, 14 Aug 2018 09:27:38 +0000 (UTC) for IP:'91.189.89.112' DOMAIN:'youngberry.canonical.com' HELO:'youngberry.canonical.com' FROM:'christian.ehrhardt@canonical.com' RCPT:'' X-RedHat-Spam-Score: -5 (RCVD_IN_DNSWL_HI) 91.189.89.112 youngberry.canonical.com 91.189.89.112 youngberry.canonical.com X-Scanned-By: MIMEDefang 2.84 on 10.5.110.49 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: libvir-list@redhat.com Cc: Christian Ehrhardt Subject: [libvirt] [PATCH v3 1/2] process: wait longer on kill per assigned Hostdev X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk Reply-To: CAATJJ0KUxiPdD=v22zqL9OZayKecWCv=TU47WFkxvfVCkcBQtg@mail.gmail.com List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Tue, 14 Aug 2018 09:27:50 +0000 (UTC) X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" It was found that in cases with host devices virProcessKillPainfully might be able to send signal zero to the target PID for quite a while with the process already being gone from /proc/. That is due to cleanup and reset of devices which might include a secondary bus reset that on top of the actions taken has a 1s delay to let the bus settle. Due to that guests with plenty of Host devices could easily exceed the default timeouts. To solve that, this adds an extra delay of 2s per hostdev that is associated to a VM. Signed-off-by: Christian Ehrhardt --- src/libvirt_private.syms | 1 + src/qemu/qemu_process.c | 5 +++-- src/util/virprocess.c | 18 +++++++++++++++--- src/util/virprocess.h | 1 + 4 files changed, 20 insertions(+), 5 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index ca4a192a4a..47ea35f864 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2605,6 +2605,7 @@ virProcessGetPids; virProcessGetStartTime; virProcessKill; virProcessKillPainfully; +virProcessKillPainfullyDelay; virProcessNamespaceAvailable; virProcessRunInMountNamespace; virProcessSchedPolicyTypeFromString; diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 02fdc55156..b7bf8813da 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6814,8 +6814,9 @@ qemuProcessKill(virDomainObjPtr vm, unsigned int flag= s) return 0; } =20 - ret =3D virProcessKillPainfully(vm->pid, - !!(flags & VIR_QEMU_PROCESS_KILL_FORCE)); + ret =3D virProcessKillPainfullyDelay(vm->pid, + !!(flags & VIR_QEMU_PROCESS_KILL_FO= RCE), + vm->def->nhostdevs * 2); =20 return ret; } diff --git a/src/util/virprocess.c b/src/util/virprocess.c index ecea27a2d4..46360cc051 100644 --- a/src/util/virprocess.c +++ b/src/util/virprocess.c @@ -341,15 +341,19 @@ int virProcessKill(pid_t pid, int sig) * Returns 0 if it was killed gracefully, 1 if it * was killed forcibly, -1 if it is still alive, * or another error occurred. + * + * Callers can proide an extra delay to wait longer + * than the default. */ int -virProcessKillPainfully(pid_t pid, bool force) +virProcessKillPainfullyDelay(pid_t pid, bool force, unsigned int extradela= y) { size_t i; int ret =3D -1; + unsigned int delay =3D 75 + (extradelay*5); const char *signame =3D "TERM"; =20 - VIR_DEBUG("vpid=3D%lld force=3D%d", (long long)pid, force); + VIR_DEBUG("vpid=3D%lld force=3D%d delay=3D%u", (long long)pid, force, = pid); =20 /* This loop sends SIGTERM, then waits a few iterations (10 seconds) * to see if it dies. If the process still hasn't exited, and @@ -357,9 +361,12 @@ virProcessKillPainfully(pid_t pid, bool force) * wait up to 5 seconds more for the process to exit before * returning. * + * An extra delay can be specified for cases that are expected to clean + * up slower than usual. + * * Note that setting @force could result in dataloss for the process. */ - for (i =3D 0; i < 75; i++) { + for (i =3D 0; i < delay; i++) { int signum; if (i =3D=3D 0) { signum =3D SIGTERM; /* kindly suggest it should exit */ @@ -402,6 +409,11 @@ virProcessKillPainfully(pid_t pid, bool force) } =20 =20 +int virProcessKillPainfully(pid_t pid, bool force) +{ + return virProcessKillPainfullyDelay(pid, force, 0); +} + #if HAVE_SCHED_GETAFFINITY =20 int virProcessSetAffinity(pid_t pid, virBitmapPtr map) diff --git a/src/util/virprocess.h b/src/util/virprocess.h index 3c5a882772..b72603ca8e 100644 --- a/src/util/virprocess.h +++ b/src/util/virprocess.h @@ -55,6 +55,7 @@ virProcessWait(pid_t pid, int *exitstatus, bool raw) int virProcessKill(pid_t pid, int sig); =20 int virProcessKillPainfully(pid_t pid, bool force); +int virProcessKillPainfullyDelay(pid_t pid, bool force, unsigned int); =20 int virProcessSetAffinity(pid_t pid, virBitmapPtr map); =20 --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu Apr 25 15:21:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=canonical.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1534238865570476.1445928461545; Tue, 14 Aug 2018 02:27:45 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0CE4330E684C; Tue, 14 Aug 2018 09:27:43 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5F45170BAB; Tue, 14 Aug 2018 09:27:42 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id EB3C818037ED; Tue, 14 Aug 2018 09:27:40 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w7E9RevV027052 for ; Tue, 14 Aug 2018 05:27:40 -0400 Received: by smtp.corp.redhat.com (Postfix) id 097A318BA6; Tue, 14 Aug 2018 09:27:40 +0000 (UTC) Received: from mx1.redhat.com (ext-mx19.extmail.prod.ext.phx2.redhat.com [10.5.110.48]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EFBFE18B9F; Tue, 14 Aug 2018 09:27:39 +0000 (UTC) Received: from youngberry.canonical.com (youngberry.canonical.com [91.189.89.112]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9B496307D858; Tue, 14 Aug 2018 09:27:38 +0000 (UTC) Received: from 1.general.paelzer.uk.vpn ([10.172.196.172] helo=lap.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1fpVc1-00033p-F3; Tue, 14 Aug 2018 09:27:37 +0000 From: Christian Ehrhardt To: Alex Williamson , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , libvir-list@redhat.com Date: Tue, 14 Aug 2018 11:27:34 +0200 Message-Id: <20180814092734.14622-3-christian.ehrhardt@canonical.com> In-Reply-To: <20180814092734.14622-1-christian.ehrhardt@canonical.com> References: <20180814092734.14622-1-christian.ehrhardt@canonical.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 212 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Tue, 14 Aug 2018 09:27:38 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Tue, 14 Aug 2018 09:27:38 +0000 (UTC) for IP:'91.189.89.112' DOMAIN:'youngberry.canonical.com' HELO:'youngberry.canonical.com' FROM:'christian.ehrhardt@canonical.com' RCPT:'' X-RedHat-Spam-Score: -5 (RCVD_IN_DNSWL_HI) 91.189.89.112 youngberry.canonical.com 91.189.89.112 youngberry.canonical.com X-Scanned-By: MIMEDefang 2.84 on 10.5.110.48 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Cc: Christian Ehrhardt Subject: [libvirt] [PATCH v3 2/2] process: wait longer 5->30s on hard shutdown X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk Reply-To: CAATJJ0KUxiPdD=v22zqL9OZayKecWCv=TU47WFkxvfVCkcBQtg@mail.gmail.com List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 14 Aug 2018 09:27:43 +0000 (UTC) X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" In cases where virProcessKillPainfully already reailizes that SIGTERM wasn't enough we are partially on a bad path already. Maybe the system is overloaded or having serious trouble to free and reap resources in time. In those case give the SIGKILL that was sent after 10 seconds some more time to take effect if force was set (only then we are falling back to SIGKILL anyway). Signed-off-by: Christian Ehrhardt Reviewed-by: Daniel P. Berrang=C3=A9 --- src/util/virprocess.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/util/virprocess.c b/src/util/virprocess.c index 46360cc051..dda8916284 100644 --- a/src/util/virprocess.c +++ b/src/util/virprocess.c @@ -350,7 +350,7 @@ virProcessKillPainfullyDelay(pid_t pid, bool force, uns= igned int extradelay) { size_t i; int ret =3D -1; - unsigned int delay =3D 75 + (extradelay*5); + unsigned int delay =3D (force ? 200 : 75) + (extradelay*5); const char *signame =3D "TERM"; =20 VIR_DEBUG("vpid=3D%lld force=3D%d delay=3D%u", (long long)pid, force, = pid); @@ -358,7 +358,7 @@ virProcessKillPainfullyDelay(pid_t pid, bool force, uns= igned int extradelay) /* This loop sends SIGTERM, then waits a few iterations (10 seconds) * to see if it dies. If the process still hasn't exited, and * @force is requested, a SIGKILL will be sent, and this will - * wait up to 5 seconds more for the process to exit before + * wait up to 30 seconds more for the process to exit before * returning. * * An extra delay can be specified for cases that are expected to clean --=20 2.17.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list