tests/vm/freebsd: Increase code coverage

[PATCH] tests/vm/freebsd: Increase code coverage

Posted by Philippe Mathieu-Daudé 2 years, 10 months ago

Install more dependencies to increase code coverage.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/vm/freebsd | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 6e20e843228..85049b43136 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -46,7 +46,9 @@ class FreeBSDVM(basevm.BaseVM):
         "gettext",
 
         # libs: crypto
+        "cyrus-sasl",
         "gnutls",
+        "nettle",
 
         # libs: images
         "jpeg-turbo",
@@ -56,6 +58,7 @@ class FreeBSDVM(basevm.BaseVM):
         "sdl2",
         "gtk3",
         "libxkbcommon",
+        "pixman",
 
         # libs: opengl
         "libepoxy",
@@ -63,6 +66,8 @@ class FreeBSDVM(basevm.BaseVM):
 
         # libs: migration
         "zstd",
+
+        "usbredir",
     ]
 
     # TODO: Enable gnutls again once FreeBSD's libtasn1 got fixed
-- 
2.26.3

Re: [PATCH] tests/vm/freebsd: Increase code coverage

Posted by Wainer dos Santos Moschetta 2 years, 10 months ago

Hi,

On 5/31/21 7:03 AM, Philippe Mathieu-Daudé wrote:
> Install more dependencies to increase code coverage.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   tests/vm/freebsd | 5 +++++
>   1 file changed, 5 insertions(+)

With or without this patch I got an error when `make vm-build-freebsd`. 
It fails to install packages.

For example, with this patch I got:

< Output omitted>

### Installing packages ...
Failed to prepare guest environment
Traceback (most recent call last):
   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 634, in main
     return vm.build_image(args.image)
   File "/home/wmoschet/src/qemu/tests/vm/freebsd", line 206, in build_image
     self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 255, in 
ssh_root_check
     self._ssh_do(self._config["root_user"], cmd, True)
   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 242, in _ssh_do
     raise Exception("SSH command failed: %s" % cmd)
Exception: SSH command failed: pkg install -y git pkgconf bzip2 python37 
ninja bash gmake gsed gettext cyrus-sasl gnutls nettle jpeg-turbo png 
sdl2 gtk3 libxkbcommon pixman libepoxy mesa-libs zstd usbredir

Is it a known issue?

>
> diff --git a/tests/vm/freebsd b/tests/vm/freebsd
> index 6e20e843228..85049b43136 100755
> --- a/tests/vm/freebsd
> +++ b/tests/vm/freebsd
> @@ -46,7 +46,9 @@ class FreeBSDVM(basevm.BaseVM):
>           "gettext",
>   
>           # libs: crypto
> +        "cyrus-sasl",
>           "gnutls",
> +        "nettle",
>   
>           # libs: images
>           "jpeg-turbo",
> @@ -56,6 +58,7 @@ class FreeBSDVM(basevm.BaseVM):
>           "sdl2",
>           "gtk3",
>           "libxkbcommon",
> +        "pixman",
>   
>           # libs: opengl
>           "libepoxy",
> @@ -63,6 +66,8 @@ class FreeBSDVM(basevm.BaseVM):
>   
>           # libs: migration
>           "zstd",
> +
> +        "usbredir",
>       ]
>   
>       # TODO: Enable gnutls again once FreeBSD's libtasn1 got fixed

Re: [PATCH] tests/vm/freebsd: Increase code coverage

Posted by Daniel P. Berrangé 2 years, 10 months ago

On Mon, May 31, 2021 at 05:53:25PM -0300, Wainer dos Santos Moschetta wrote:
> Hi,
> 
> On 5/31/21 7:03 AM, Philippe Mathieu-Daudé wrote:
> > Install more dependencies to increase code coverage.
> > 
> > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> > ---
> >   tests/vm/freebsd | 5 +++++
> >   1 file changed, 5 insertions(+)
> 
> With or without this patch I got an error when `make vm-build-freebsd`. It
> fails to install packages.
> 
> For example, with this patch I got:
> 
> < Output omitted>
> 
> ### Installing packages ...
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 634, in main
>     return vm.build_image(args.image)
>   File "/home/wmoschet/src/qemu/tests/vm/freebsd", line 206, in build_image
>     self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
>   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 255, in
> ssh_root_check
>     self._ssh_do(self._config["root_user"], cmd, True)
>   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 242, in _ssh_do
>     raise Exception("SSH command failed: %s" % cmd)
> Exception: SSH command failed: pkg install -y git pkgconf bzip2 python37
> ninja bash gmake gsed gettext cyrus-sasl gnutls nettle jpeg-turbo png sdl2
> gtk3 libxkbcommon pixman libepoxy mesa-libs zstd usbredir
> 
> Is it a known issue?

Hard to actually tell what the error really is. This message is
only giving the command that was invoked, but seems to have thrown
away stdout/stderr which would have the messages telling us what
went wrong.  This lack of error reporting in basevm.py so badly
needs to be fixed, otherwise we're working blind when debugging
failures.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] tests/vm/freebsd: Increase code coverage

Posted by Daniel P. Berrangé 2 years, 10 months ago

On Tue, Jun 15, 2021 at 06:02:02PM +0100, Daniel P. Berrangé wrote:
> On Mon, May 31, 2021 at 05:53:25PM -0300, Wainer dos Santos Moschetta wrote:
> > Hi,
> > 
> > On 5/31/21 7:03 AM, Philippe Mathieu-Daudé wrote:
> > > Install more dependencies to increase code coverage.
> > > 
> > > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> > > ---
> > >   tests/vm/freebsd | 5 +++++
> > >   1 file changed, 5 insertions(+)
> > 
> > With or without this patch I got an error when `make vm-build-freebsd`. It
> > fails to install packages.
> > 
> > For example, with this patch I got:
> > 
> > < Output omitted>
> > 
> > ### Installing packages ...
> > Failed to prepare guest environment
> > Traceback (most recent call last):
> >   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 634, in main
> >     return vm.build_image(args.image)
> >   File "/home/wmoschet/src/qemu/tests/vm/freebsd", line 206, in build_image
> >     self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
> >   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 255, in
> > ssh_root_check
> >     self._ssh_do(self._config["root_user"], cmd, True)
> >   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 242, in _ssh_do
> >     raise Exception("SSH command failed: %s" % cmd)
> > Exception: SSH command failed: pkg install -y git pkgconf bzip2 python37
> > ninja bash gmake gsed gettext cyrus-sasl gnutls nettle jpeg-turbo png sdl2
> > gtk3 libxkbcommon pixman libepoxy mesa-libs zstd usbredir
> > 
> > Is it a known issue?
> 
> Hard to actually tell what the error really is. This message is
> only giving the command that was invoked, but seems to have thrown
> away stdout/stderr which would have the messages telling us what
> went wrong.  This lack of error reporting in basevm.py so badly
> needs to be fixed, otherwise we're working blind when debugging
> failures.

Hmm, when I try the same command, it *does* print all the output
so you can see what's going on, but it doesn't actually fail for
me either (without this patch).

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] tests/vm/freebsd: Increase code coverage

Posted by Wainer dos Santos Moschetta 2 years, 10 months ago

Hi,

On 6/15/21 2:19 PM, Daniel P. Berrangé wrote:
> On Tue, Jun 15, 2021 at 06:02:02PM +0100, Daniel P. Berrangé wrote:
>> On Mon, May 31, 2021 at 05:53:25PM -0300, Wainer dos Santos Moschetta wrote:
>>> Hi,
>>>
>>> On 5/31/21 7:03 AM, Philippe Mathieu-Daudé wrote:
>>>> Install more dependencies to increase code coverage.
>>>>
>>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>>> ---
>>>>    tests/vm/freebsd | 5 +++++
>>>>    1 file changed, 5 insertions(+)
>>> With or without this patch I got an error when `make vm-build-freebsd`. It
>>> fails to install packages.
>>>
>>> For example, with this patch I got:
>>>
>>> < Output omitted>
>>>
>>> ### Installing packages ...
>>> Failed to prepare guest environment
>>> Traceback (most recent call last):
>>>    File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 634, in main
>>>      return vm.build_image(args.image)
>>>    File "/home/wmoschet/src/qemu/tests/vm/freebsd", line 206, in build_image
>>>      self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
>>>    File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 255, in
>>> ssh_root_check
>>>      self._ssh_do(self._config["root_user"], cmd, True)
>>>    File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 242, in _ssh_do
>>>      raise Exception("SSH command failed: %s" % cmd)
>>> Exception: SSH command failed: pkg install -y git pkgconf bzip2 python37
>>> ninja bash gmake gsed gettext cyrus-sasl gnutls nettle jpeg-turbo png sdl2
>>> gtk3 libxkbcommon pixman libepoxy mesa-libs zstd usbredir
>>>
>>> Is it a known issue?
>> Hard to actually tell what the error really is. This message is
>> only giving the command that was invoked, but seems to have thrown
>> away stdout/stderr which would have the messages telling us what
>> went wrong.  This lack of error reporting in basevm.py so badly
>> needs to be fixed, otherwise we're working blind when debugging
>> failures.
> Hmm, when I try the same command, it *does* print all the output
> so you can see what's going on, but it doesn't actually fail for
> me either (without this patch).

Thomas sent me a message in private while ago that the error could be 
related with the amount of ssh keys in my agent-ssh. I didn' t check 
that hypothesis yet; will debug a littler further and let you all posted.

ah, if this problem cannot be reproduced on other machines....I see no 
reason to hold this patch.

- Wainer

>
> Regards,
> Daniel

Re: [PATCH] tests/vm/freebsd: Increase code coverage

Posted by Philippe Mathieu-Daudé 2 years, 10 months ago

On 5/31/21 10:53 PM, Wainer dos Santos Moschetta wrote:
> Hi,
> 
> On 5/31/21 7:03 AM, Philippe Mathieu-Daudé wrote:
>> Install more dependencies to increase code coverage.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>>   tests/vm/freebsd | 5 +++++
>>   1 file changed, 5 insertions(+)
> 
> With or without this patch I got an error when `make vm-build-freebsd`.
> It fails to install packages.
> 
> For example, with this patch I got:
> 
> < Output omitted>
> 
> ### Installing packages ...
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 634, in main
>     return vm.build_image(args.image)
>   File "/home/wmoschet/src/qemu/tests/vm/freebsd", line 206, in build_image
>     self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
>   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 255, in
> ssh_root_check
>     self._ssh_do(self._config["root_user"], cmd, True)
>   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 242, in _ssh_do
>     raise Exception("SSH command failed: %s" % cmd)
> Exception: SSH command failed: pkg install -y git pkgconf bzip2 python37
> ninja bash gmake gsed gettext cyrus-sasl gnutls nettle jpeg-turbo png
> sdl2 gtk3 libxkbcommon pixman libepoxy mesa-libs zstd usbredir
> 
> Is it a known issue?

I'll defer that to Warner, I suppose the package repository got updated
and I am not hitting that because my VM installed them earlier? (Similar
to the problems with Docker packages). Is there a way to use a snapshot
mirror of FreeBSD packages?

Re: [PATCH] tests/vm/freebsd: Increase code coverage

Posted by Warner Losh 2 years, 10 months ago


> On Jun 15, 2021, at 10:55 AM, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> 
> On 5/31/21 10:53 PM, Wainer dos Santos Moschetta wrote:
>> Hi,
>> 
>> On 5/31/21 7:03 AM, Philippe Mathieu-Daudé wrote:
>>> Install more dependencies to increase code coverage.
>>> 
>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>> ---
>>>   tests/vm/freebsd | 5 +++++
>>>   1 file changed, 5 insertions(+)
>> 
>> With or without this patch I got an error when `make vm-build-freebsd`.
>> It fails to install packages.
>> 
>> For example, with this patch I got:
>> 
>> < Output omitted>
>> 
>> ### Installing packages ...
>> Failed to prepare guest environment
>> Traceback (most recent call last):
>>   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 634, in main
>>     return vm.build_image(args.image)
>>   File "/home/wmoschet/src/qemu/tests/vm/freebsd", line 206, in build_image
>>     self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
>>   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 255, in
>> ssh_root_check
>>     self._ssh_do(self._config["root_user"], cmd, True)
>>   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 242, in _ssh_do
>>     raise Exception("SSH command failed: %s" % cmd)
>> Exception: SSH command failed: pkg install -y git pkgconf bzip2 python37
>> ninja bash gmake gsed gettext cyrus-sasl gnutls nettle jpeg-turbo png
>> sdl2 gtk3 libxkbcommon pixman libepoxy mesa-libs zstd usbredir
>> 
>> Is it a known issue?
> 
> I'll defer that to Warner, I suppose the package repository got updated
> and I am not hitting that because my VM installed them earlier? (Similar
> to the problems with Docker packages). Is there a way to use a snapshot
> mirror of FreeBSD packages?

I’m still trying to replicate this, but I do know FreeBSD’s packages moved
from python 3.7 to 3.8 by default recently….

Warner

Re: [PATCH] tests/vm/freebsd: Increase code coverage

Posted by Warner Losh 2 years, 10 months ago


> On Jun 15, 2021, at 2:09 PM, Warner Losh <wlosh@bsdimp.com> wrote:
> 
> 
> 
>> On Jun 15, 2021, at 10:55 AM, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>> 
>> On 5/31/21 10:53 PM, Wainer dos Santos Moschetta wrote:
>>> Hi,
>>> 
>>> On 5/31/21 7:03 AM, Philippe Mathieu-Daudé wrote:
>>>> Install more dependencies to increase code coverage.
>>>> 
>>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>>> ---
>>>>  tests/vm/freebsd | 5 +++++
>>>>  1 file changed, 5 insertions(+)
>>> 
>>> With or without this patch I got an error when `make vm-build-freebsd`.
>>> It fails to install packages.
>>> 
>>> For example, with this patch I got:
>>> 
>>> < Output omitted>
>>> 
>>> ### Installing packages ...
>>> Failed to prepare guest environment
>>> Traceback (most recent call last):
>>>  File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 634, in main
>>>    return vm.build_image(args.image)
>>>  File "/home/wmoschet/src/qemu/tests/vm/freebsd", line 206, in build_image
>>>    self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
>>>  File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 255, in
>>> ssh_root_check
>>>    self._ssh_do(self._config["root_user"], cmd, True)
>>>  File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 242, in _ssh_do
>>>    raise Exception("SSH command failed: %s" % cmd)
>>> Exception: SSH command failed: pkg install -y git pkgconf bzip2 python37
>>> ninja bash gmake gsed gettext cyrus-sasl gnutls nettle jpeg-turbo png
>>> sdl2 gtk3 libxkbcommon pixman libepoxy mesa-libs zstd usbredir
>>> 
>>> Is it a known issue?
>> 
>> I'll defer that to Warner, I suppose the package repository got updated
>> and I am not hitting that because my VM installed them earlier? (Similar
>> to the problems with Docker packages). Is there a way to use a snapshot
>> mirror of FreeBSD packages?
> 
> I’m still trying to replicate this, but I do know FreeBSD’s packages moved
> from python 3.7 to 3.8 by default recently….

It’s not this… I’m seeing inability to pull from the pkg repo, I think, as the root cause… Still unclear why… Sometimes it dies part way into the download. I’ve not yet worked out why it gets to different spots different times.

Warner