What?

This patch series adds the live migration cpr-transfer mode, which
allows the user to transfer a guest to a new QEMU instance on the same
host.  It is identical to cpr-exec in most respects, except as described
below. 

The new user-visible interfaces are:
  * cpr-transfer (MigMode migration parameter)
  * cpr-uri (migration parameter)
  * cpr-uri (command-line argument)

In this mode, the user starts new QEMU on the same host as old QEMU, with
the same arguments as old QEMU, plus the -incoming and the -cpr-uri options.
The user issues the migrate command to old QEMU, which stops the VM, saves
state to the migration channels, and enters the postmigrate state.  Execution
resumes in new QEMU.

This mode requires a second migration channel, specified by the cpr-uri
migration property on the outgoing side, and by the cpr-uri QEMU command-line
option on the incoming side.  The channel must be a type, such as unix socket,
that supports SCM_RIGHTS.

This series depends on the series "Live update: cpr-exec mode".

Why?

cpr-transfer offers the same benefits as cpr-exec mode, but with a model
for launching new QEMU that may be more natural for some management packages.

How?

The file descriptors are kept open by sending them to new QEMU via the
cpr-uri, which must support SCM_RIGHTS.

Example:

In this example, we simply restart the same version of QEMU, but in
a real scenario one would use a new QEMU binary path in terminal 2.

  Terminal 1: start old QEMU
  # qemu-kvm -monitor stdio -object
  memory-backend-file,id=ram0,size=4G,mem-path=/dev/shm/ram0,share=on
  -m 4G -machine anon-alloc=memfd ...

  Terminal 2: start new QEMU
  # qemu-kvm ... -incoming unix:vm.sock -cpr-uri unix:cpr.sock

  Terminal 1:
  QEMU 9.1.50 monitor - type 'help' for more information
  (qemu) info status
  VM status: running
  (qemu) migrate_set_parameter mode cpr-transfer
  (qemu) migrate_set_parameter cpr-uri unix:cpr.sock
  (qemu) migrate -d unix:vm.sock
  (qemu) info status
  VM status: paused (postmigrate)

  Terminal 2:
  QEMU 9.1.50 monitor - type 'help' for more information
  (qemu) info status
  VM status: running

Steve Sistare (6):
  migration: SCM_RIGHTS for QEMUFile
  migration: VMSTATE_FD
  migration: cpr-transfer save and load
  migration: cpr-uri parameter
  migration: cpr-uri option
  migration: cpr-transfer mode

 include/migration/cpr.h        |  4 ++
 include/migration/vmstate.h    |  9 +++++
 migration/cpr-transfer.c       | 81 +++++++++++++++++++++++++++++++++++++++++
 migration/cpr.c                | 16 +++++++-
 migration/meson.build          |  1 +
 migration/migration-hmp-cmds.c | 10 +++++
 migration/migration.c          | 37 +++++++++++++++++++
 migration/options.c            | 29 +++++++++++++++
 migration/options.h            |  1 +
 migration/qemu-file.c          | 83 ++++++++++++++++++++++++++++++++++++++++--
 migration/qemu-file.h          |  2 +
 migration/ram.c                |  1 +
 migration/trace-events         |  2 +
 migration/vmstate-types.c      | 33 +++++++++++++++++
 qapi/migration.json            | 38 ++++++++++++++++++-
 qemu-options.hx                |  8 ++++
 stubs/vmstate.c                |  7 ++++
 system/vl.c                    |  3 ++
 18 files changed, 359 insertions(+), 6 deletions(-)
 create mode 100644 migration/cpr-transfer.c

-- 
1.8.3.1

On Sun, Jun 30, 2024 at 12:44:02PM -0700, Steve Sistare wrote:
> What?
> 
> This patch series adds the live migration cpr-transfer mode, which
> allows the user to transfer a guest to a new QEMU instance on the same
> host.  It is identical to cpr-exec in most respects, except as described
> below. 

I definitely prefer this one more than the exec solution, thanks for trying
this out.  It's a matter of whether we'll need both, my answer would be
no..

> 
> The new user-visible interfaces are:
>   * cpr-transfer (MigMode migration parameter)
>   * cpr-uri (migration parameter)

I wonder whether this parameter can be avoided already, maybe we can let
cpr-transfer depend on unix socket in -incoming, then integrate fd sharing
in the same channel?

>   * cpr-uri (command-line argument)
> 
> In this mode, the user starts new QEMU on the same host as old QEMU, with
> the same arguments as old QEMU, plus the -incoming and the -cpr-uri options.
> The user issues the migrate command to old QEMU, which stops the VM, saves
> state to the migration channels, and enters the postmigrate state.  Execution
> resumes in new QEMU.
> 
> This mode requires a second migration channel, specified by the cpr-uri
> migration property on the outgoing side, and by the cpr-uri QEMU command-line
> option on the incoming side.  The channel must be a type, such as unix socket,
> that supports SCM_RIGHTS.
> 
> This series depends on the series "Live update: cpr-exec mode".
> 
> Why?
> 
> cpr-transfer offers the same benefits as cpr-exec mode, but with a model
> for launching new QEMU that may be more natural for some management packages.
> 
> How?
> 
> The file descriptors are kept open by sending them to new QEMU via the
> cpr-uri, which must support SCM_RIGHTS.
> 
> Example:
> 
> In this example, we simply restart the same version of QEMU, but in
> a real scenario one would use a new QEMU binary path in terminal 2.
> 
>   Terminal 1: start old QEMU
>   # qemu-kvm -monitor stdio -object
>   memory-backend-file,id=ram0,size=4G,mem-path=/dev/shm/ram0,share=on
>   -m 4G -machine anon-alloc=memfd ...
> 
>   Terminal 2: start new QEMU
>   # qemu-kvm ... -incoming unix:vm.sock -cpr-uri unix:cpr.sock
> 
>   Terminal 1:
>   QEMU 9.1.50 monitor - type 'help' for more information
>   (qemu) info status
>   VM status: running
>   (qemu) migrate_set_parameter mode cpr-transfer
>   (qemu) migrate_set_parameter cpr-uri unix:cpr.sock
>   (qemu) migrate -d unix:vm.sock
>   (qemu) info status
>   VM status: paused (postmigrate)
> 
>   Terminal 2:
>   QEMU 9.1.50 monitor - type 'help' for more information
>   (qemu) info status
>   VM status: running
> 
> Steve Sistare (6):
>   migration: SCM_RIGHTS for QEMUFile
>   migration: VMSTATE_FD
>   migration: cpr-transfer save and load
>   migration: cpr-uri parameter
>   migration: cpr-uri option
>   migration: cpr-transfer mode
> 
>  include/migration/cpr.h        |  4 ++
>  include/migration/vmstate.h    |  9 +++++
>  migration/cpr-transfer.c       | 81 +++++++++++++++++++++++++++++++++++++++++
>  migration/cpr.c                | 16 +++++++-
>  migration/meson.build          |  1 +
>  migration/migration-hmp-cmds.c | 10 +++++
>  migration/migration.c          | 37 +++++++++++++++++++
>  migration/options.c            | 29 +++++++++++++++
>  migration/options.h            |  1 +
>  migration/qemu-file.c          | 83 ++++++++++++++++++++++++++++++++++++++++--
>  migration/qemu-file.h          |  2 +
>  migration/ram.c                |  1 +
>  migration/trace-events         |  2 +
>  migration/vmstate-types.c      | 33 +++++++++++++++++
>  qapi/migration.json            | 38 ++++++++++++++++++-
>  qemu-options.hx                |  8 ++++
>  stubs/vmstate.c                |  7 ++++
>  system/vl.c                    |  3 ++
>  18 files changed, 359 insertions(+), 6 deletions(-)
>  create mode 100644 migration/cpr-transfer.c
> 
> -- 
> 1.8.3.1
> 

-- 
Peter Xu