From nobody Thu Jul 3 23:56:32 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 152890817289740.88275784441555; Wed, 13 Jun 2018 09:42:52 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7AF43C04B31E; Wed, 13 Jun 2018 16:42:51 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 36CA7183F0; Wed, 13 Jun 2018 16:42:51 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id CFEB24CA81; Wed, 13 Jun 2018 16:42:50 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w5DGgWEu029586 for ; Wed, 13 Jun 2018 12:42:32 -0400 Received: by smtp.corp.redhat.com (Postfix) id 8D6362024CA4; Wed, 13 Jun 2018 16:42:32 +0000 (UTC) Received: from red.redhat.com (ovpn-120-103.rdu2.redhat.com [10.10.120.103]) by smtp.corp.redhat.com (Postfix) with ESMTP id B474F2024CA1; Wed, 13 Jun 2018 16:42:31 +0000 (UTC) From: Eric Blake To: libvir-list@redhat.com Date: Wed, 13 Jun 2018 11:42:23 -0500 Message-Id: <20180613164229.1379979-3-eblake@redhat.com> In-Reply-To: <20180613164229.1379979-1-eblake@redhat.com> References: <20180613164229.1379979-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-loop: libvir-list@redhat.com Cc: vsementsov@virtuozzo.com, mnestratov@virtuozzo.com, nsoffer@redhat.com, pkrempa@redhat.com, nshirokovskiy@virtuozzo.com, den@openvz.org, jsnow@redhat.com Subject: [libvirt] [PATCH 2/8] backup: Document nuances between different state capture APIs X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 13 Jun 2018 16:42:52 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Upcoming patches will add support for incremental backups via a new API; but first, we need a landing page that gives an overview of capturing various pieces of guest state, and which APIs are best suited to which tasks. Signed-off-by: Eric Blake --- docs/docs.html.in | 5 ++ docs/domainstatecapture.html.in | 190 ++++++++++++++++++++++++++++++++++++= ++++ docs/formatsnapshot.html.in | 2 + 3 files changed, 197 insertions(+) create mode 100644 docs/domainstatecapture.html.in diff --git a/docs/docs.html.in b/docs/docs.html.in index 40e0e3b82e..4c46b74980 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -120,6 +120,11 @@
Secure usage
Secure usage of the libvirt APIs
+ +
Domain state + capture
+
Comparison between different methods of capturing domain + state
diff --git a/docs/domainstatecapture.html.in b/docs/domainstatecapture.html= .in new file mode 100644 index 0000000000..00ab7e8ee1 --- /dev/null +++ b/docs/domainstatecapture.html.in @@ -0,0 +1,190 @@ + + + + + +

Domain state capture using Libvirt

+ +
    + +

    + This page compares the different means for capturing state + related to a domain managed by libvirt, in order to aid + application developers to choose which operations best suit + their needs. +

    + +

    State capture trade-offs

    + +

    One of the features made possible with virtual machines is live + migration, or transferring all state related to the guest from + one host to another, with minimal interruption to the guest's + activity. A clever observer will then note that if all state is + available for live migration, there is nothing stopping a user + from saving that state at a given point of time, to be able to + later rewind guest execution back to the state it previously + had. There are several different libvirt APIs associated with + capturing the state of a guest, such that the captured state can + later be used to rewind that guest to the conditions it was in + earlier. But since there are multiple APIs, it is best to + understand the tradeoffs and differences between them, in order + to choose the best API for a given task. +

    + +
    +
    Timing
    +
    Capturing state can be a lengthy process, so while the + captured state ideally represents an atomic point in time + correpsonding to something the guest was actually executing, + some interfaces require up-front preparation (the state + captured is not complete until the API ends, which may be some + time after the command was first started), while other + interfaces track the state when the command was first issued + even if it takes some time to finish capturing the state. + While it is possible to freeze guest I/O around either point + in time (so that the captured state is fully consistent, + rather than just crash-consistent), knowing whether the state + is captured at the start or end of the command may determine + which approach to use. A related concept is the amount of + downtime the guest will experience during the capture, + particularly since freezing guest I/O has time + constraints.
    + +
    Amount of state
    +
    For an offline guest, only the contents of the guest disks + needs to be captured; restoring that state is merely a fresh + boot with the disks restored to that state. But for an online + guest, there is a choice between storing the guest's memory + (all that is needed during live migration where the storage is + shared between source and destination), the guest's disk state + (all that is needed if there are no pending guest I/O + transactions that would be lost without the corresponding + memory state), or both together. Unless guest I/O is quiesced + prior to capturing state, then reverting to captured disk + state of a live guest without the corresponding memory state + is comparable to booting a machine that previously lost power + without a clean shutdown; but for a guest that uses + appropriate journaling methods, this crash-consistent state + may be sufficient to avoid the additional storage and time + needed to capture memory state.
    + +
    Quantity of files
    +
    When capturing state, some approaches store all state within + the same file (internal), while others expand a chain of + related files that must be used together (external), for more + files that a management application must track. There are + also differences depending on whether the state is captured in + the same file in use by a running guest, or whether the state + is captured to a distinct file without impacting the files + used to run the guest.
    + +
    Third-party integration
    +
    When capturing state, particularly for a running, there are + tradeoffs to how much of the process must be done directly by + the hypervisor, and how much can be off-loaded to third-party + software. Since capturing state is not instantaneous, it is + essential that any third-party integration see consistent data + even if the running guest continues to modify that data after + the point in time of the capture.
    + +
    Full vs. partial
    +
    When capturing state, it is useful to minimize the amount of + state that must be captured in relation to a previous capture, + by focusing only on the portions of the disk that the guest + has modified since the previous capture. Some approaches are + able to take advantage of checkpoints to provide an + incremental backup, while others are only capable of a full + backup including portions of the disk that have not changed + since the previous state capture.
    +
    + +

    State capture APIs

    +

    With those definitions, the following libvirt APIs have these + properties:

    +
    +
    virDomainSnapshotCreateXML()
    +
    This API wraps several approaches for capturing guest state, + with a general premise of creating a snapshot (where the + current guest resources are frozen in time and a new wrapper + layer is opened for tracking subsequent guest changes). It + can operate on both offline and running guests, can choose + whether to capture the state of memory, disk, or both when + used on a running guest, and can choose between internal and + external storage for captured state. However, it is geared + towards post-event captures (when capturing both memory and + disk state, the disk state is not captured until all memory + state has been collected first). For qemu as the hypervisor, + internal snapshots currently have lengthy downtime that is + incompatible with freezing guest I/O, but external snapshots + are quick. Since creating an external snapshot changes which + disk image resource is in use by the guest, this API can be + coupled with virDomainBlockCommit() to restore + things back to the guest using its original disk image, where + a third-party tool can read the backing file prior to the live + commit. See also the XML + details used with this command.
    +
    virDomainBlockCopy()
    +
    This API wraps approaches for capturing the state of disks + of a running guest, but does not track accompanying guest + memory state, and can only operate on one block device per job + (to get a consistent copy of multiple disks, the domain must + be paused before ending the multiple jobs). The capture is + consistent only at the end of the operation, with a choice to + either pivot to the new file that contains the copy (leaving + the old file as the backup), or to return to the original file + (leaving the new file as the backup).
    +
    virDomainBackupBegin()
    +
    This API wraps approaches for capturing the state of disks + of a running guest, but does not track accompanying guest + memory state. The capture is consistent to the start of the + operation, where the captured state is stored independently + from the disk image in use with the guest, and where it can be + easily integrated with a third-party for capturing the disk + state. Since the backup operation is stored externally from + the guest resources, there is no need to commit data back in + at the completion of the operation. When coupled with + checkpoints, this can be used to capture incremental backups + instead of full.
    +
    virDomainCheckpointCreateXML()
    +
    This API does not actually capture guest state, so much as + make it possible to track which portions of guest disks have + change between checkpoints or between a current checkpoint and + the live execution of the guest. When performing incremental + backups, it is easier to create a new checkpoint at the same + time as a new backup, so that the next incremental backup can + refer to the incremental state since the checkpoint created + during the current backup. Guest state is then actually + captured using virDomainBackupBegin().
    +
    + +

    Examples

    +

    The following two sequences both capture the disk state of a + running guest, then complete with the guest running on its + original disk image; but with a difference that an unexpected + interruption during the first mode leaves a temporary wrapper + file that must be accounted for, while interruption of the + second mode has no impact to the guest.

    +

    1. Backup via temporary snapshot + 

    +virDomainFSFreeze()
+virDomainSnapshotCreateXML(VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY)
+virDomainFSThaw()
+third-party copy the backing file to backup storage # most time spent here
+virDomainBlockCommit(VIR_DOMAIN_BLOCK_COMMIT_ACTIVE) per disk
+wait for commit ready event per disk
+virDomainBlockJobAbort() per disk
+

    + +

    2. Direct backup + 

    +virDomainFSFreeze()
+virDomainBackupBegin()
+virDomainFSThaw()
+wait for push mode event, or pull data over NBD # most time spent here
+virDomainBackeupEnd()
+

    + + + diff --git a/docs/formatsnapshot.html.in b/docs/formatsnapshot.html.in index f2e51df5ab..d7051683a5 100644 --- a/docs/formatsnapshot.html.in +++ b/docs/formatsnapshot.html.in @@ -9,6 +9,8 @@

    Snapshot XML

    + Snapshots are one form + of domain state capture. There are several types of snapshots:

    --=20 2.14.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list