[libvirt] [PATCH 00/11] Allow modification of IOThread polling values (redux)

John Ferlan posted 11 patches 5 years, 6 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20181007130024.1750-1-jferlan@redhat.com
There is a newer version of this series
include/libvirt/libvirt-domain.h              |  45 ++
src/driver-hypervisor.h                       |   8 +
src/libvirt-domain.c                          | 109 +++++
src/libvirt_public.syms                       |   5 +
src/qemu/qemu_capabilities.c                  |   2 +
src/qemu/qemu_capabilities.h                  |   1 +
src/qemu/qemu_driver.c                        | 393 ++++++++++++++++--
src/qemu/qemu_monitor.c                       |  19 +
src/qemu/qemu_monitor.h                       |   6 +
src/qemu/qemu_monitor_json.c                  |  48 +++
src/qemu/qemu_monitor_json.h                  |   4 +
src/remote/remote_driver.c                    |   1 +
src/remote/remote_protocol.x                  |  21 +-
src/remote_protocol-structs                   |  10 +
.../caps_2.10.0.aarch64.xml                   |   1 +
.../caps_2.10.0.ppc64.xml                     |   1 +
.../caps_2.10.0.s390x.xml                     |   1 +
.../caps_2.10.0.x86_64.xml                    |   1 +
.../caps_2.11.0.s390x.xml                     |   1 +
.../caps_2.11.0.x86_64.xml                    |   1 +
.../caps_2.12.0.aarch64.xml                   |   1 +
.../caps_2.12.0.ppc64.xml                     |   1 +
.../caps_2.12.0.s390x.xml                     |   1 +
.../caps_2.12.0.x86_64.xml                    |   1 +
.../qemucapabilitiesdata/caps_2.9.0.ppc64.xml |   1 +
.../qemucapabilitiesdata/caps_2.9.0.s390x.xml |   1 +
.../caps_2.9.0.x86_64.xml                     |   1 +
.../qemucapabilitiesdata/caps_3.0.0.ppc64.xml |   1 +
.../caps_3.0.0.riscv32.xml                    |   1 +
.../caps_3.0.0.riscv64.xml                    |   1 +
.../qemucapabilitiesdata/caps_3.0.0.s390x.xml |   1 +
.../caps_3.0.0.x86_64.xml                     |   1 +
tools/virsh-domain-monitor.c                  |   7 +
tools/virsh-domain.c                          | 110 +++++
tools/virsh.pod                               |  47 ++-
35 files changed, 810 insertions(+), 44 deletions(-)
[libvirt] [PATCH 00/11] Allow modification of IOThread polling values (redux)
Posted by John Ferlan 5 years, 6 months ago
This series attempts to resurrect the concept of being able to modify
the IOThread polling parameters; however, in a slightly different
manner than the previous series posted by posted by Pavel Hrdina
<phrdina@redhat.com>:

  https://www.redhat.com/archives/libvir-list/2017-February/msg01047.html

The work is prompted by continued pleas found in the bz:

  https://bugzilla.redhat.com/show_bug.cgi?id=1545732

to provide some way to modify the paremters without needing to supply
QEMU command line pass through values.

It's accepted that the values being changed are fairly or extremely
low level type knobs; however, it's also shown that by being able to
turn the knob it is possible for certain, specific appliances to be
able to gain a performance benefit for the thread at the expense of
other competing threads.

Unlike the previous series, this series does not attempt to save the
polling values in the guest XML. Rather, it only modifies the live
guest's IOThread with new values. It also doesn't provide the polling
values in a virsh iothread* command, rather it uses the domstats
in order to fetch and display the values. The theory being this
leaves the onus on the higher level appliance/application to provide
the "proper guidance" and "concerns" related to changing the values
to the consumer. Not saving the values means whatever values that
are chosen do not "live" in perpetuity. Once the guest is shut down
or the IOThread removed from guest, the hypervisor default values
take over again. Perhaps not a perfect situation in terms of what
the bz requests; however, storage of default values that could
cause performance issues is not an optimal situation. So this I
figured is a "comprimise" of sorts.

If it's still felt that no we don't want to do this, then fine,
but please in doing so own the bz, state your case, and close it.
I'm 50/50 on it, but figured at least I'd present this option and
see what the concensus was.

John Ferlan (11):
  qemu: Check for and return IOThread polling values if available
  qemu: Split qemuDomainGetIOThreadsLive
  qemu: Implement the ability to return IOThread stats
  virsh: Add ability to display IOThread stats
  lib: Introduce virDomainSetIOThreadParams
  qemu: Add monitor functions to set IOThread params
  qemu: Alter qemuDomainChgIOThread to take enum instead of bool
  qemu: Alter qemuDomainChgIOThread to take qemuMonitorIOThreadInfo
  qemu: Detect whether iothread polling is supported
  qemu: Introduce qemuDomainSetIOThreadParams
  tools: Add virsh iothreadset command

 include/libvirt/libvirt-domain.h              |  45 ++
 src/driver-hypervisor.h                       |   8 +
 src/libvirt-domain.c                          | 109 +++++
 src/libvirt_public.syms                       |   5 +
 src/qemu/qemu_capabilities.c                  |   2 +
 src/qemu/qemu_capabilities.h                  |   1 +
 src/qemu/qemu_driver.c                        | 393 ++++++++++++++++--
 src/qemu/qemu_monitor.c                       |  19 +
 src/qemu/qemu_monitor.h                       |   6 +
 src/qemu/qemu_monitor_json.c                  |  48 +++
 src/qemu/qemu_monitor_json.h                  |   4 +
 src/remote/remote_driver.c                    |   1 +
 src/remote/remote_protocol.x                  |  21 +-
 src/remote_protocol-structs                   |  10 +
 .../caps_2.10.0.aarch64.xml                   |   1 +
 .../caps_2.10.0.ppc64.xml                     |   1 +
 .../caps_2.10.0.s390x.xml                     |   1 +
 .../caps_2.10.0.x86_64.xml                    |   1 +
 .../caps_2.11.0.s390x.xml                     |   1 +
 .../caps_2.11.0.x86_64.xml                    |   1 +
 .../caps_2.12.0.aarch64.xml                   |   1 +
 .../caps_2.12.0.ppc64.xml                     |   1 +
 .../caps_2.12.0.s390x.xml                     |   1 +
 .../caps_2.12.0.x86_64.xml                    |   1 +
 .../qemucapabilitiesdata/caps_2.9.0.ppc64.xml |   1 +
 .../qemucapabilitiesdata/caps_2.9.0.s390x.xml |   1 +
 .../caps_2.9.0.x86_64.xml                     |   1 +
 .../qemucapabilitiesdata/caps_3.0.0.ppc64.xml |   1 +
 .../caps_3.0.0.riscv32.xml                    |   1 +
 .../caps_3.0.0.riscv64.xml                    |   1 +
 .../qemucapabilitiesdata/caps_3.0.0.s390x.xml |   1 +
 .../caps_3.0.0.x86_64.xml                     |   1 +
 tools/virsh-domain-monitor.c                  |   7 +
 tools/virsh-domain.c                          | 110 +++++
 tools/virsh.pod                               |  47 ++-
 35 files changed, 810 insertions(+), 44 deletions(-)

-- 
2.17.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 00/11] Allow modification of IOThread polling values (redux)
Posted by John Ferlan 5 years, 6 months ago
ping?

Any takers or thoughts?

Tks,

John


On 10/7/18 9:00 AM, John Ferlan wrote:
> This series attempts to resurrect the concept of being able to modify
> the IOThread polling parameters; however, in a slightly different
> manner than the previous series posted by posted by Pavel Hrdina
> <phrdina@redhat.com>:
> 
>   https://www.redhat.com/archives/libvir-list/2017-February/msg01047.html
> 
> The work is prompted by continued pleas found in the bz:
> 
>   https://bugzilla.redhat.com/show_bug.cgi?id=1545732
> 
> to provide some way to modify the paremters without needing to supply
> QEMU command line pass through values.
> 
> It's accepted that the values being changed are fairly or extremely
> low level type knobs; however, it's also shown that by being able to
> turn the knob it is possible for certain, specific appliances to be
> able to gain a performance benefit for the thread at the expense of
> other competing threads.
> 
> Unlike the previous series, this series does not attempt to save the
> polling values in the guest XML. Rather, it only modifies the live
> guest's IOThread with new values. It also doesn't provide the polling
> values in a virsh iothread* command, rather it uses the domstats
> in order to fetch and display the values. The theory being this
> leaves the onus on the higher level appliance/application to provide
> the "proper guidance" and "concerns" related to changing the values
> to the consumer. Not saving the values means whatever values that
> are chosen do not "live" in perpetuity. Once the guest is shut down
> or the IOThread removed from guest, the hypervisor default values
> take over again. Perhaps not a perfect situation in terms of what
> the bz requests; however, storage of default values that could
> cause performance issues is not an optimal situation. So this I
> figured is a "comprimise" of sorts.
> 
> If it's still felt that no we don't want to do this, then fine,
> but please in doing so own the bz, state your case, and close it.
> I'm 50/50 on it, but figured at least I'd present this option and
> see what the concensus was.
> 
> John Ferlan (11):
>   qemu: Check for and return IOThread polling values if available
>   qemu: Split qemuDomainGetIOThreadsLive
>   qemu: Implement the ability to return IOThread stats
>   virsh: Add ability to display IOThread stats
>   lib: Introduce virDomainSetIOThreadParams
>   qemu: Add monitor functions to set IOThread params
>   qemu: Alter qemuDomainChgIOThread to take enum instead of bool
>   qemu: Alter qemuDomainChgIOThread to take qemuMonitorIOThreadInfo
>   qemu: Detect whether iothread polling is supported
>   qemu: Introduce qemuDomainSetIOThreadParams
>   tools: Add virsh iothreadset command
> 
>  include/libvirt/libvirt-domain.h              |  45 ++
>  src/driver-hypervisor.h                       |   8 +
>  src/libvirt-domain.c                          | 109 +++++
>  src/libvirt_public.syms                       |   5 +
>  src/qemu/qemu_capabilities.c                  |   2 +
>  src/qemu/qemu_capabilities.h                  |   1 +
>  src/qemu/qemu_driver.c                        | 393 ++++++++++++++++--
>  src/qemu/qemu_monitor.c                       |  19 +
>  src/qemu/qemu_monitor.h                       |   6 +
>  src/qemu/qemu_monitor_json.c                  |  48 +++
>  src/qemu/qemu_monitor_json.h                  |   4 +
>  src/remote/remote_driver.c                    |   1 +
>  src/remote/remote_protocol.x                  |  21 +-
>  src/remote_protocol-structs                   |  10 +
>  .../caps_2.10.0.aarch64.xml                   |   1 +
>  .../caps_2.10.0.ppc64.xml                     |   1 +
>  .../caps_2.10.0.s390x.xml                     |   1 +
>  .../caps_2.10.0.x86_64.xml                    |   1 +
>  .../caps_2.11.0.s390x.xml                     |   1 +
>  .../caps_2.11.0.x86_64.xml                    |   1 +
>  .../caps_2.12.0.aarch64.xml                   |   1 +
>  .../caps_2.12.0.ppc64.xml                     |   1 +
>  .../caps_2.12.0.s390x.xml                     |   1 +
>  .../caps_2.12.0.x86_64.xml                    |   1 +
>  .../qemucapabilitiesdata/caps_2.9.0.ppc64.xml |   1 +
>  .../qemucapabilitiesdata/caps_2.9.0.s390x.xml |   1 +
>  .../caps_2.9.0.x86_64.xml                     |   1 +
>  .../qemucapabilitiesdata/caps_3.0.0.ppc64.xml |   1 +
>  .../caps_3.0.0.riscv32.xml                    |   1 +
>  .../caps_3.0.0.riscv64.xml                    |   1 +
>  .../qemucapabilitiesdata/caps_3.0.0.s390x.xml |   1 +
>  .../caps_3.0.0.x86_64.xml                     |   1 +
>  tools/virsh-domain-monitor.c                  |   7 +
>  tools/virsh-domain.c                          | 110 +++++
>  tools/virsh.pod                               |  47 ++-
>  35 files changed, 810 insertions(+), 44 deletions(-)
> 

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 00/11] Allow modification of IOThread polling values (redux)
Posted by Christian Borntraeger 5 years, 6 months ago
On 10/15/2018 04:28 PM, John Ferlan wrote:
> 
> ping?
> 
> Any takers or thoughts?

No review, but I think it makes a lot of sense to expose these tuneables.

> 
> Tks,
> 
> John
> 
> 
> On 10/7/18 9:00 AM, John Ferlan wrote:
>> This series attempts to resurrect the concept of being able to modify
>> the IOThread polling parameters; however, in a slightly different
>> manner than the previous series posted by posted by Pavel Hrdina
>> <phrdina@redhat.com>:
>>
>>   https://www.redhat.com/archives/libvir-list/2017-February/msg01047.html
>>
>> The work is prompted by continued pleas found in the bz:
>>
>>   https://bugzilla.redhat.com/show_bug.cgi?id=1545732
>>
>> to provide some way to modify the paremters without needing to supply
>> QEMU command line pass through values.
>>
>> It's accepted that the values being changed are fairly or extremely
>> low level type knobs; however, it's also shown that by being able to
>> turn the knob it is possible for certain, specific appliances to be
>> able to gain a performance benefit for the thread at the expense of
>> other competing threads.
>>
>> Unlike the previous series, this series does not attempt to save the
>> polling values in the guest XML. Rather, it only modifies the live
>> guest's IOThread with new values. It also doesn't provide the polling
>> values in a virsh iothread* command, rather it uses the domstats
>> in order to fetch and display the values. The theory being this
>> leaves the onus on the higher level appliance/application to provide
>> the "proper guidance" and "concerns" related to changing the values
>> to the consumer. Not saving the values means whatever values that
>> are chosen do not "live" in perpetuity. Once the guest is shut down
>> or the IOThread removed from guest, the hypervisor default values
>> take over again. Perhaps not a perfect situation in terms of what
>> the bz requests; however, storage of default values that could
>> cause performance issues is not an optimal situation. So this I
>> figured is a "comprimise" of sorts.
>>
>> If it's still felt that no we don't want to do this, then fine,
>> but please in doing so own the bz, state your case, and close it.
>> I'm 50/50 on it, but figured at least I'd present this option and
>> see what the concensus was.
>>
>> John Ferlan (11):
>>   qemu: Check for and return IOThread polling values if available
>>   qemu: Split qemuDomainGetIOThreadsLive
>>   qemu: Implement the ability to return IOThread stats
>>   virsh: Add ability to display IOThread stats
>>   lib: Introduce virDomainSetIOThreadParams
>>   qemu: Add monitor functions to set IOThread params
>>   qemu: Alter qemuDomainChgIOThread to take enum instead of bool
>>   qemu: Alter qemuDomainChgIOThread to take qemuMonitorIOThreadInfo
>>   qemu: Detect whether iothread polling is supported
>>   qemu: Introduce qemuDomainSetIOThreadParams
>>   tools: Add virsh iothreadset command
>>
>>  include/libvirt/libvirt-domain.h              |  45 ++
>>  src/driver-hypervisor.h                       |   8 +
>>  src/libvirt-domain.c                          | 109 +++++
>>  src/libvirt_public.syms                       |   5 +
>>  src/qemu/qemu_capabilities.c                  |   2 +
>>  src/qemu/qemu_capabilities.h                  |   1 +
>>  src/qemu/qemu_driver.c                        | 393 ++++++++++++++++--
>>  src/qemu/qemu_monitor.c                       |  19 +
>>  src/qemu/qemu_monitor.h                       |   6 +
>>  src/qemu/qemu_monitor_json.c                  |  48 +++
>>  src/qemu/qemu_monitor_json.h                  |   4 +
>>  src/remote/remote_driver.c                    |   1 +
>>  src/remote/remote_protocol.x                  |  21 +-
>>  src/remote_protocol-structs                   |  10 +
>>  .../caps_2.10.0.aarch64.xml                   |   1 +
>>  .../caps_2.10.0.ppc64.xml                     |   1 +
>>  .../caps_2.10.0.s390x.xml                     |   1 +
>>  .../caps_2.10.0.x86_64.xml                    |   1 +
>>  .../caps_2.11.0.s390x.xml                     |   1 +
>>  .../caps_2.11.0.x86_64.xml                    |   1 +
>>  .../caps_2.12.0.aarch64.xml                   |   1 +
>>  .../caps_2.12.0.ppc64.xml                     |   1 +
>>  .../caps_2.12.0.s390x.xml                     |   1 +
>>  .../caps_2.12.0.x86_64.xml                    |   1 +
>>  .../qemucapabilitiesdata/caps_2.9.0.ppc64.xml |   1 +
>>  .../qemucapabilitiesdata/caps_2.9.0.s390x.xml |   1 +
>>  .../caps_2.9.0.x86_64.xml                     |   1 +
>>  .../qemucapabilitiesdata/caps_3.0.0.ppc64.xml |   1 +
>>  .../caps_3.0.0.riscv32.xml                    |   1 +
>>  .../caps_3.0.0.riscv64.xml                    |   1 +
>>  .../qemucapabilitiesdata/caps_3.0.0.s390x.xml |   1 +
>>  .../caps_3.0.0.x86_64.xml                     |   1 +
>>  tools/virsh-domain-monitor.c                  |   7 +
>>  tools/virsh-domain.c                          | 110 +++++
>>  tools/virsh.pod                               |  47 ++-
>>  35 files changed, 810 insertions(+), 44 deletions(-)
>>
> 
> --
> libvir-list mailing list
> libvir-list@redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list
> 


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list