[PATCH v4 00/15] qemu_iotests: improve debugging options

Emanuele Giuseppe Esposito posted 15 patches 2 years, 11 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
docs/devel/testing.rst        | 30 +++++++++++++++++++
python/qemu/machine.py        |  7 +++--
python/qemu/qtest.py          |  9 ++++--
tests/qemu-iotests/check      | 15 +++++++---
tests/qemu-iotests/common.rc  |  8 ++++-
tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++----
tests/qemu-iotests/testenv.py | 25 ++++++++++++++--
7 files changed, 132 insertions(+), 18 deletions(-)
[PATCH v4 00/15] qemu_iotests: improve debugging options
Posted by Emanuele Giuseppe Esposito 2 years, 11 months ago
This series adds the option to attach gdbserver and valgrind
to the QEMU binary running in qemu_iotests.
It also allows to redirect QEMU binaries output of the python tests
to the stdout, instead of a log file.

Patches 1-6 introduce the -gdb option to both python and bash tests, 
7-10 extend the already existing -valgrind flag to work also on 
python tests, and patch 11 introduces -p to enable logging to stdout.

In particular, patches 1,5,10 focus on extending the QMP socket timers
when using gdb/valgrind, otherwise the python tests will fail due to
delays in the QMP responses.

This series is tested on the previous serie
"qemu-iotests: quality of life improvements"
but independent from it, so it can be applied separately.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
---
v4:
* Rename environment variable from GDB_QEMU to GDB_OPTIONS
* This time test 297 (pylint) passes [Max]
* Refactor the qmp_timer field in machine.py, and add a new 
  parameter in machine.py and subclasses constructor [John]
* Add additional check in patch 4 to cover the case where
  GDB_OPTIONS is empty

Emanuele Giuseppe Esposito (15):
  python: qemu: add timer parameter for qmp.accept socket
  python: qemu: pass the wrapper field from QEMUQtestmachine to
    QEMUMachine
  docs/devel/testing: add debug section to the QEMU iotests chapter
  qemu-iotests: add option to attach gdbserver
  qemu-iotests: delay QMP socket timers
  qemu_iotests: insert gdbserver command line as wrapper for qemu binary
  qemu-iotests: add gdbserver option to script tests too
  docs/devel/testing: add -gdb option to the debugging section of QEMU
    iotests
  qemu-iotests: extend the check script to support valgrind for python
    tests
  qemu-iotests: extent QMP socket timeout when using valgrind
  qemu-iotests: allow valgrind to read/delete the generated log file
  qemu-iotests: insert valgrind command line as wrapper for qemu binary
  docs/devel/testing: add -valgrind option to the debug section of QEMU
    iotests
  qemu-iotests: add option to show qemu binary logs on stdout
  docs/devel/testing: add -p option to the debug section of QEMU iotests

 docs/devel/testing.rst        | 30 +++++++++++++++++++
 python/qemu/machine.py        |  7 +++--
 python/qemu/qtest.py          |  9 ++++--
 tests/qemu-iotests/check      | 15 +++++++---
 tests/qemu-iotests/common.rc  |  8 ++++-
 tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++----
 tests/qemu-iotests/testenv.py | 25 ++++++++++++++--
 7 files changed, 132 insertions(+), 18 deletions(-)

-- 
2.30.2


Re: [PATCH v4 00/15] qemu_iotests: improve debugging options
Posted by Vladimir Sementsov-Ogievskiy 2 years, 11 months ago
20.05.2021 10:52, Emanuele Giuseppe Esposito wrote:
> This series adds the option to attach gdbserver and valgrind
> to the QEMU binary running in qemu_iotests.
> It also allows to redirect QEMU binaries output of the python tests
> to the stdout, instead of a log file.
> 
> Patches 1-6 introduce the -gdb option to both python and bash tests,

Could you describe how to use it?

I often have to debug qemu iotests with gdb, and can say the following:

1. Test runs different qemu binaries (not only qemu, but also qemu-nbd, qemu-img, qemu-io)

2. Test can run qemu binaries several times, or even in parallel.

So, with a new option, how will I choice, which qemu (or qemu-nbd, etc) process I want to be attached to gdb?

Currently, I do the following: I recompile with  "sleep(15);" at the place where I want to attach, and start the test. During the sleep, I find the needed process in "ps aux" output and start "gdb -p PID".. Sometimes there may be troubles if the place where I inserted the sleep is triggered on another code path.

So, with new option, can my workflow be improved or not?

> 7-10 extend the already existing -valgrind flag to work also on
> python tests, and patch 11 introduces -p to enable logging to stdout.
> 
> In particular, patches 1,5,10 focus on extending the QMP socket timers
> when using gdb/valgrind, otherwise the python tests will fail due to
> delays in the QMP responses.
> 
> This series is tested on the previous serie
> "qemu-iotests: quality of life improvements"
> but independent from it, so it can be applied separately.
> 
> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> ---
> v4:
> * Rename environment variable from GDB_QEMU to GDB_OPTIONS
> * This time test 297 (pylint) passes [Max]
> * Refactor the qmp_timer field in machine.py, and add a new
>    parameter in machine.py and subclasses constructor [John]
> * Add additional check in patch 4 to cover the case where
>    GDB_OPTIONS is empty
> 
> Emanuele Giuseppe Esposito (15):
>    python: qemu: add timer parameter for qmp.accept socket
>    python: qemu: pass the wrapper field from QEMUQtestmachine to
>      QEMUMachine
>    docs/devel/testing: add debug section to the QEMU iotests chapter
>    qemu-iotests: add option to attach gdbserver
>    qemu-iotests: delay QMP socket timers
>    qemu_iotests: insert gdbserver command line as wrapper for qemu binary
>    qemu-iotests: add gdbserver option to script tests too
>    docs/devel/testing: add -gdb option to the debugging section of QEMU
>      iotests
>    qemu-iotests: extend the check script to support valgrind for python
>      tests
>    qemu-iotests: extent QMP socket timeout when using valgrind
>    qemu-iotests: allow valgrind to read/delete the generated log file
>    qemu-iotests: insert valgrind command line as wrapper for qemu binary
>    docs/devel/testing: add -valgrind option to the debug section of QEMU
>      iotests
>    qemu-iotests: add option to show qemu binary logs on stdout
>    docs/devel/testing: add -p option to the debug section of QEMU iotests
> 
>   docs/devel/testing.rst        | 30 +++++++++++++++++++
>   python/qemu/machine.py        |  7 +++--
>   python/qemu/qtest.py          |  9 ++++--
>   tests/qemu-iotests/check      | 15 +++++++---
>   tests/qemu-iotests/common.rc  |  8 ++++-
>   tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++----
>   tests/qemu-iotests/testenv.py | 25 ++++++++++++++--
>   7 files changed, 132 insertions(+), 18 deletions(-)
> 


-- 
Best regards,
Vladimir

Re: [PATCH v4 00/15] qemu_iotests: improve debugging options
Posted by Emanuele Giuseppe Esposito 2 years, 11 months ago

On 26/05/2021 13:32, Vladimir Sementsov-Ogievskiy wrote:
> 20.05.2021 10:52, Emanuele Giuseppe Esposito wrote:
>> This series adds the option to attach gdbserver and valgrind
>> to the QEMU binary running in qemu_iotests.
>> It also allows to redirect QEMU binaries output of the python tests
>> to the stdout, instead of a log file.
>>
>> Patches 1-6 introduce the -gdb option to both python and bash tests,
> 
> Could you describe how to use it?
> 
> I often have to debug qemu iotests with gdb, and can say the following:
> 
> 1. Test runs different qemu binaries (not only qemu, but also qemu-nbd, 
> qemu-img, qemu-io)
> 
> 2. Test can run qemu binaries several times, or even in parallel.
> 
> So, with a new option, how will I choice, which qemu (or qemu-nbd, etc) 
> process I want to be attached to gdb?
> 
> Currently, I do the following: I recompile with  "sleep(15);" at the 
> place where I want to attach, and start the test. During the sleep, I 
> find the needed process in "ps aux" output and start "gdb -p PID".. 
> Sometimes there may be troubles if the place where I inserted the sleep 
> is triggered on another code path.
> 
> So, with new option, can my workflow be improved or not?

Unfortunately not, unless you want to debug a qemu_system binary.
The usage should be clear in the doc patches of this series (please 
check them too if you haven't done already), but the idea is to just 
wrap gdb around every qemu binary usage.

If you apply these patches together with Paolo's "qemu-iotests: quality 
of life improvements", you will get something much better, where you can 
choose which test function to run under gdb.

If I recall correctly, you should be able to do something like:
check -gdb -- ../../../tests/qemu-iotests/055 TestCompressedToQcow2

Still, if the test function has multiple qemu binary invocations or in 
parallel, you gdb will be applied to all of them. I don't think 
test/debug granularity can be reduced any further, at least not so easily.

Thank you,
Emanuele

> 
>> 7-10 extend the already existing -valgrind flag to work also on
>> python tests, and patch 11 introduces -p to enable logging to stdout.
>>
>> In particular, patches 1,5,10 focus on extending the QMP socket timers
>> when using gdb/valgrind, otherwise the python tests will fail due to
>> delays in the QMP responses.
>>
>> This series is tested on the previous serie
>> "qemu-iotests: quality of life improvements"
>> but independent from it, so it can be applied separately.
>>
>> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
>> ---
>> v4:
>> * Rename environment variable from GDB_QEMU to GDB_OPTIONS
>> * This time test 297 (pylint) passes [Max]
>> * Refactor the qmp_timer field in machine.py, and add a new
>>    parameter in machine.py and subclasses constructor [John]
>> * Add additional check in patch 4 to cover the case where
>>    GDB_OPTIONS is empty
>>
>> Emanuele Giuseppe Esposito (15):
>>    python: qemu: add timer parameter for qmp.accept socket
>>    python: qemu: pass the wrapper field from QEMUQtestmachine to
>>      QEMUMachine
>>    docs/devel/testing: add debug section to the QEMU iotests chapter
>>    qemu-iotests: add option to attach gdbserver
>>    qemu-iotests: delay QMP socket timers
>>    qemu_iotests: insert gdbserver command line as wrapper for qemu binary
>>    qemu-iotests: add gdbserver option to script tests too
>>    docs/devel/testing: add -gdb option to the debugging section of QEMU
>>      iotests
>>    qemu-iotests: extend the check script to support valgrind for python
>>      tests
>>    qemu-iotests: extent QMP socket timeout when using valgrind
>>    qemu-iotests: allow valgrind to read/delete the generated log file
>>    qemu-iotests: insert valgrind command line as wrapper for qemu binary
>>    docs/devel/testing: add -valgrind option to the debug section of QEMU
>>      iotests
>>    qemu-iotests: add option to show qemu binary logs on stdout
>>    docs/devel/testing: add -p option to the debug section of QEMU iotests
>>
>>   docs/devel/testing.rst        | 30 +++++++++++++++++++
>>   python/qemu/machine.py        |  7 +++--
>>   python/qemu/qtest.py          |  9 ++++--
>>   tests/qemu-iotests/check      | 15 +++++++---
>>   tests/qemu-iotests/common.rc  |  8 ++++-
>>   tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++----
>>   tests/qemu-iotests/testenv.py | 25 ++++++++++++++--
>>   7 files changed, 132 insertions(+), 18 deletions(-)
>>
> 
>