[edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address

Brijesh Singh posted 14 patches 7 years, 4 months ago
Failed in applying to current master (apply log)
OvmfPkg/Include/Library/VirtioLib.h             | 198 ++++++++++++-
OvmfPkg/Include/Protocol/VirtioDevice.h         | 121 ++++++++
OvmfPkg/VirtioBlkDxe/VirtioBlk.h                |   1 +
OvmfPkg/VirtioNetDxe/VirtioNet.h                |  25 +-
OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.h    |  34 +++
OvmfPkg/VirtioRngDxe/VirtioRng.h                |   1 +
OvmfPkg/VirtioScsiDxe/VirtioScsi.h              |   1 +
OvmfPkg/Library/VirtioLib/VirtioLib.c           | 296 +++++++++++++++++++-
OvmfPkg/Virtio10Dxe/Virtio10.c                  | 114 +++++++-
OvmfPkg/VirtioBlkDxe/VirtioBlk.c                | 104 ++++++-
OvmfPkg/VirtioGpuDxe/Commands.c                 |   7 +-
OvmfPkg/VirtioNetDxe/Events.c                   |  24 ++
OvmfPkg/VirtioNetDxe/SnpGetStatus.c             |  19 +-
OvmfPkg/VirtioNetDxe/SnpInitialize.c            | 102 +++++--
OvmfPkg/VirtioNetDxe/SnpSharedHelpers.c         | 119 +++++++-
OvmfPkg/VirtioNetDxe/SnpShutdown.c              |  16 +-
OvmfPkg/VirtioNetDxe/SnpTransmit.c              |  22 +-
OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.c    |   7 +-
OvmfPkg/VirtioPciDeviceDxe/VirtioPciFunctions.c |  66 +++++
OvmfPkg/VirtioRngDxe/VirtioRng.c                |  54 +++-
OvmfPkg/VirtioScsiDxe/VirtioScsi.c              | 121 +++++++-
21 files changed, 1378 insertions(+), 74 deletions(-)
[edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Brijesh Singh 7 years, 4 months ago
Currently, virtio drivers provides the system physical address to the device.
However, some systems may feature an IOMMU that requires the drivers to pass
the device addresses to the device - which are then translated by the IOMMU 
into physical addresses in memory. The patch series introduces new member
functions in VIRTIO_DEVICE_PROTOCOL which can be used for mapping a system
physical address to device address.

The approach that this patch series takes is to maps the system physical
address to device address for buffers (including rings, device specifc
request and response  pointed by vring descriptor, and any further memory 
reference by those request and response).

Patch 1 - 3:
 Defines and implements new member functions to map a system physical address
 to device address. The patch implements Laszlo's suggestion [1].

Patch 4 - 7:
 Allocates the vring buffer using newly added member functions and provides
 some helper functions.

Patch 8:
 Update the virtio-blk driver to use newly added member functions to map the
 addresses.
 Verified using the following qemu cli

 # $QEMU \
    -drive file=${IMAGE},if=none,id=disk0 \
    -device virtio-blk-pci,drive=disk0

 # $QEMU \
    -drive file=${IMAGE},if=none,id=disk0 \
    -device virtio-blk-pci,drive=disk0,disable-legacy=on

Patch 9:
 Update the virtio-scsi driver to use newly added member functions to map the
 addresses.
 Verified using the following qemu cli

 # $QEMU \
    -drive file=${IMAGE},if=none,id=disk0 \
    -device scsi-hd,drive=disk0
    -device virtio-scsi-pci,id=scsi \

 # $QEMU \
    -drive file=${IMAGE},if=none,id=disk0 \
    -device scsi-hd,drive=disk0 \
    -device virtio-scsi-pci,id=scsi,disable-legacy=on \

Patch 10 - 13:
 Update the virtio-net driver to use newly added member functions to map the
 addresses.
 Verified using the following qemu cli

 # $QEMU \
    -netdev type=tap,id=net0 \
    -device virtio-net-pci,netdev=net0,romfile=

 # $QEMU \
    -netdev type=tap,id=net0 \
    -device virtio-net-pci,netdev=net0,disable-legacy=on,romfile=

Patch 14:
 Update the virtio-rng driver to use newly added member functions to map the
 addresses.
 Verified using the following qemu cli

 # $QEMU \
    -device virtio-rng-pci

 # $QEMU \
    -device virtio-rng-pci,disable-legacy=on

 And succesfully ran RngTest.efi from SecurityPkg/Application

Repo: https://github.com/codomania/edk2
Branch: virtio-support

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Laszlo Ersek <lersek@redhat.com>

Brijesh Singh (14):
  OvmfPkg/Virtio: Introduce new member functions in
    VIRTIO_DEVICE_PROTOCOL
  OvmfPkg/Virtio10Dxe: Implement new member functions
  OvmfPkg/VirtioPciDeviceDxe: Implement new member functions
  OvmfPkg/VirtioLib: Add SharedBuffer helper functions
  OvmfPkg/VirtioLib: Pass VirtIo instance in VringInit/Uinit()
  OvmfPkg/VirtioLib: Add functions to map/unmap VRING
  OvmfPkg/VirtioLib: Use AllocateShared() to allocate vring buffer
  OvmfPkg/VirtioBlkDxe: Use DeviceAddresses in vring descriptors
  OvmfPkg/VirtioScsiDxe: Use DeviceAddresses in vring descriptors
  OvmfPkg/VirtioNetDxe: Allocate Tx and Rx ring using
    AllocateSharedPage()
  OvmfPkg/VirtioNetDxe: Allocate RxBuf using AllocateSharedPages()
  OvmfPkg/VirtioNetDxe: Dynamically allocate transmit header
  OvmfPkg/VirtioNetDxe: Use DeviceAddress in transmit vring descriptors
  OvmfPkg/VirtioRngDxe: Use DeviceAddresses in vring descriptors

 OvmfPkg/Include/Library/VirtioLib.h             | 198 ++++++++++++-
 OvmfPkg/Include/Protocol/VirtioDevice.h         | 121 ++++++++
 OvmfPkg/VirtioBlkDxe/VirtioBlk.h                |   1 +
 OvmfPkg/VirtioNetDxe/VirtioNet.h                |  25 +-
 OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.h    |  34 +++
 OvmfPkg/VirtioRngDxe/VirtioRng.h                |   1 +
 OvmfPkg/VirtioScsiDxe/VirtioScsi.h              |   1 +
 OvmfPkg/Library/VirtioLib/VirtioLib.c           | 296 +++++++++++++++++++-
 OvmfPkg/Virtio10Dxe/Virtio10.c                  | 114 +++++++-
 OvmfPkg/VirtioBlkDxe/VirtioBlk.c                | 104 ++++++-
 OvmfPkg/VirtioGpuDxe/Commands.c                 |   7 +-
 OvmfPkg/VirtioNetDxe/Events.c                   |  24 ++
 OvmfPkg/VirtioNetDxe/SnpGetStatus.c             |  19 +-
 OvmfPkg/VirtioNetDxe/SnpInitialize.c            | 102 +++++--
 OvmfPkg/VirtioNetDxe/SnpSharedHelpers.c         | 119 +++++++-
 OvmfPkg/VirtioNetDxe/SnpShutdown.c              |  16 +-
 OvmfPkg/VirtioNetDxe/SnpTransmit.c              |  22 +-
 OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.c    |   7 +-
 OvmfPkg/VirtioPciDeviceDxe/VirtioPciFunctions.c |  66 +++++
 OvmfPkg/VirtioRngDxe/VirtioRng.c                |  54 +++-
 OvmfPkg/VirtioScsiDxe/VirtioScsi.c              | 121 +++++++-
 21 files changed, 1378 insertions(+), 74 deletions(-)

-- 
2.7.4

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Laszlo Ersek 7 years, 4 months ago
On 08/07/17 13:58, Brijesh Singh wrote:
> Currently, virtio drivers provides the system physical address to the device.
> However, some systems may feature an IOMMU that requires the drivers to pass
> the device addresses to the device - which are then translated by the IOMMU 
> into physical addresses in memory. The patch series introduces new member
> functions in VIRTIO_DEVICE_PROTOCOL which can be used for mapping a system
> physical address to device address.
> 
> The approach that this patch series takes is to maps the system physical
> address to device address for buffers (including rings, device specifc
> request and response  pointed by vring descriptor, and any further memory 
> reference by those request and response).
> 
> Patch 1 - 3:
>  Defines and implements new member functions to map a system physical address
>  to device address. The patch implements Laszlo's suggestion [1].

(1) I guess under [1] you meant the following message:

http://mid.mail-archive.com/841bec5f-6f6e-8b1f-25ba-0fd37a915b72@redhat.com

If you want, you can add that link to the commit message of patch #1.

(2) But, that's not my main point here. Before I forget, I'd like to
point out that you missed one of the three virtio protocol
implementations -- see my point (5.4) in the above-referenced message
--, namely:

> - "ArmVirtPkg/VirtioFdtDxe" (via
>   "OvmfPkg/Library/VirtioMmioDeviceLib") binds virtio-mmio devices,
>    and offers virtio 0.9.5 semantics.

So please replicate patch v1 03/14 to
"OvmfPkg/Library/VirtioMmioDeviceLib". Otherwise, the modified virtio
device drivers will crash when they are built for ArmVirtPkg and used
over virtio-mmio transports.

(3) Starting with your v2, please add a reminder to your blurb -- for
Ard and myself -- that before merging this, we should regression-test it
on aarch64.

Thanks!
Laszlo

> 
> Patch 4 - 7:
>  Allocates the vring buffer using newly added member functions and provides
>  some helper functions.
> 
> Patch 8:
>  Update the virtio-blk driver to use newly added member functions to map the
>  addresses.
>  Verified using the following qemu cli
> 
>  # $QEMU \
>     -drive file=${IMAGE},if=none,id=disk0 \
>     -device virtio-blk-pci,drive=disk0
> 
>  # $QEMU \
>     -drive file=${IMAGE},if=none,id=disk0 \
>     -device virtio-blk-pci,drive=disk0,disable-legacy=on
> 
> Patch 9:
>  Update the virtio-scsi driver to use newly added member functions to map the
>  addresses.
>  Verified using the following qemu cli
> 
>  # $QEMU \
>     -drive file=${IMAGE},if=none,id=disk0 \
>     -device scsi-hd,drive=disk0
>     -device virtio-scsi-pci,id=scsi \
> 
>  # $QEMU \
>     -drive file=${IMAGE},if=none,id=disk0 \
>     -device scsi-hd,drive=disk0 \
>     -device virtio-scsi-pci,id=scsi,disable-legacy=on \
> 
> Patch 10 - 13:
>  Update the virtio-net driver to use newly added member functions to map the
>  addresses.
>  Verified using the following qemu cli
> 
>  # $QEMU \
>     -netdev type=tap,id=net0 \
>     -device virtio-net-pci,netdev=net0,romfile=
> 
>  # $QEMU \
>     -netdev type=tap,id=net0 \
>     -device virtio-net-pci,netdev=net0,disable-legacy=on,romfile=
> 
> Patch 14:
>  Update the virtio-rng driver to use newly added member functions to map the
>  addresses.
>  Verified using the following qemu cli
> 
>  # $QEMU \
>     -device virtio-rng-pci
> 
>  # $QEMU \
>     -device virtio-rng-pci,disable-legacy=on
> 
>  And succesfully ran RngTest.efi from SecurityPkg/Application
> 
> Repo: https://github.com/codomania/edk2
> Branch: virtio-support
> 
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> 
> Brijesh Singh (14):
>   OvmfPkg/Virtio: Introduce new member functions in
>     VIRTIO_DEVICE_PROTOCOL
>   OvmfPkg/Virtio10Dxe: Implement new member functions
>   OvmfPkg/VirtioPciDeviceDxe: Implement new member functions
>   OvmfPkg/VirtioLib: Add SharedBuffer helper functions
>   OvmfPkg/VirtioLib: Pass VirtIo instance in VringInit/Uinit()
>   OvmfPkg/VirtioLib: Add functions to map/unmap VRING
>   OvmfPkg/VirtioLib: Use AllocateShared() to allocate vring buffer
>   OvmfPkg/VirtioBlkDxe: Use DeviceAddresses in vring descriptors
>   OvmfPkg/VirtioScsiDxe: Use DeviceAddresses in vring descriptors
>   OvmfPkg/VirtioNetDxe: Allocate Tx and Rx ring using
>     AllocateSharedPage()
>   OvmfPkg/VirtioNetDxe: Allocate RxBuf using AllocateSharedPages()
>   OvmfPkg/VirtioNetDxe: Dynamically allocate transmit header
>   OvmfPkg/VirtioNetDxe: Use DeviceAddress in transmit vring descriptors
>   OvmfPkg/VirtioRngDxe: Use DeviceAddresses in vring descriptors
> 
>  OvmfPkg/Include/Library/VirtioLib.h             | 198 ++++++++++++-
>  OvmfPkg/Include/Protocol/VirtioDevice.h         | 121 ++++++++
>  OvmfPkg/VirtioBlkDxe/VirtioBlk.h                |   1 +
>  OvmfPkg/VirtioNetDxe/VirtioNet.h                |  25 +-
>  OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.h    |  34 +++
>  OvmfPkg/VirtioRngDxe/VirtioRng.h                |   1 +
>  OvmfPkg/VirtioScsiDxe/VirtioScsi.h              |   1 +
>  OvmfPkg/Library/VirtioLib/VirtioLib.c           | 296 +++++++++++++++++++-
>  OvmfPkg/Virtio10Dxe/Virtio10.c                  | 114 +++++++-
>  OvmfPkg/VirtioBlkDxe/VirtioBlk.c                | 104 ++++++-
>  OvmfPkg/VirtioGpuDxe/Commands.c                 |   7 +-
>  OvmfPkg/VirtioNetDxe/Events.c                   |  24 ++
>  OvmfPkg/VirtioNetDxe/SnpGetStatus.c             |  19 +-
>  OvmfPkg/VirtioNetDxe/SnpInitialize.c            | 102 +++++--
>  OvmfPkg/VirtioNetDxe/SnpSharedHelpers.c         | 119 +++++++-
>  OvmfPkg/VirtioNetDxe/SnpShutdown.c              |  16 +-
>  OvmfPkg/VirtioNetDxe/SnpTransmit.c              |  22 +-
>  OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.c    |   7 +-
>  OvmfPkg/VirtioPciDeviceDxe/VirtioPciFunctions.c |  66 +++++
>  OvmfPkg/VirtioRngDxe/VirtioRng.c                |  54 +++-
>  OvmfPkg/VirtioScsiDxe/VirtioScsi.c              | 121 +++++++-
>  21 files changed, 1378 insertions(+), 74 deletions(-)
> 

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Brijesh Singh 7 years, 4 months ago

On 08/09/2017 09:39 AM, Laszlo Ersek wrote:
> On 08/07/17 13:58, Brijesh Singh wrote:
>> Currently, virtio drivers provides the system physical address to the device.
>> However, some systems may feature an IOMMU that requires the drivers to pass
>> the device addresses to the device - which are then translated by the IOMMU
>> into physical addresses in memory. The patch series introduces new member
>> functions in VIRTIO_DEVICE_PROTOCOL which can be used for mapping a system
>> physical address to device address.
>>
>> The approach that this patch series takes is to maps the system physical
>> address to device address for buffers (including rings, device specifc
>> request and response  pointed by vring descriptor, and any further memory
>> reference by those request and response).
>>
>> Patch 1 - 3:
>>   Defines and implements new member functions to map a system physical address
>>   to device address. The patch implements Laszlo's suggestion [1].
> 
> (1) I guess under [1] you meant the following message:
> 
> http://mid.mail-archive.com/841bec5f-6f6e-8b1f-25ba-0fd37a915b72@redhat.com
> 

Yes, thank you :) I did not realized that I forgot adding the link.


> If you want, you can add that link to the commit message of patch #1.
> 
> (2) But, that's not my main point here. Before I forget, I'd like to
> point out that you missed one of the three virtio protocol
> implementations -- see my point (5.4) in the above-referenced message
> --, namely:
> 
>> - "ArmVirtPkg/VirtioFdtDxe" (via
>>    "OvmfPkg/Library/VirtioMmioDeviceLib") binds virtio-mmio devices,
>>     and offers virtio 0.9.5 semantics.
> 
> So please replicate patch v1 03/14 to
> "OvmfPkg/Library/VirtioMmioDeviceLib". Otherwise, the modified virtio
> device drivers will crash when they are built for ArmVirtPkg and used
> over virtio-mmio transports.
> 

Sure, I will make the necessary changes in VirtioMmioDeviceLib and try
do the build test but I don't have aarch64 platform to verify at the
runtime.

> (3) Starting with your v2, please add a reminder to your blurb -- for
> Ard and myself -- that before merging this, we should regression-test it
> on aarch64.
> 

Will do
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Laszlo Ersek 7 years, 4 months ago
On 08/09/17 19:35, Brijesh Singh wrote:
> 
> 
> On 08/09/2017 09:39 AM, Laszlo Ersek wrote:
>> On 08/07/17 13:58, Brijesh Singh wrote:
>>> Currently, virtio drivers provides the system physical address to the
>>> device.
>>> However, some systems may feature an IOMMU that requires the drivers
>>> to pass
>>> the device addresses to the device - which are then translated by the
>>> IOMMU
>>> into physical addresses in memory. The patch series introduces new
>>> member
>>> functions in VIRTIO_DEVICE_PROTOCOL which can be used for mapping a
>>> system
>>> physical address to device address.
>>>
>>> The approach that this patch series takes is to maps the system physical
>>> address to device address for buffers (including rings, device specifc
>>> request and response  pointed by vring descriptor, and any further
>>> memory
>>> reference by those request and response).
>>>
>>> Patch 1 - 3:
>>>   Defines and implements new member functions to map a system
>>> physical address
>>>   to device address. The patch implements Laszlo's suggestion [1].
>>
>> (1) I guess under [1] you meant the following message:
>>
>> http://mid.mail-archive.com/841bec5f-6f6e-8b1f-25ba-0fd37a915b72@redhat.com
>>
>>
> 
> Yes, thank you :) I did not realized that I forgot adding the link.
> 
> 
>> If you want, you can add that link to the commit message of patch #1.
>>
>> (2) But, that's not my main point here. Before I forget, I'd like to
>> point out that you missed one of the three virtio protocol
>> implementations -- see my point (5.4) in the above-referenced message
>> --, namely:
>>
>>> - "ArmVirtPkg/VirtioFdtDxe" (via
>>>    "OvmfPkg/Library/VirtioMmioDeviceLib") binds virtio-mmio devices,
>>>     and offers virtio 0.9.5 semantics.
>>
>> So please replicate patch v1 03/14 to
>> "OvmfPkg/Library/VirtioMmioDeviceLib". Otherwise, the modified virtio
>> device drivers will crash when they are built for ArmVirtPkg and used
>> over virtio-mmio transports.
>>
> 
> Sure, I will make the necessary changes in VirtioMmioDeviceLib and try
> do the build test but I don't have aarch64 platform to verify at the
> runtime.

Actually, dependent on your GNU/Linux distribution, it is pretty easy to
do on x86_64 too. It comes together from two parts:

(1) installing an aarch64 cross compiler
(2) installing qemu-system-aarch64 (from source or distro package),
    and then either using it directly (from the cmdline) or with the
    libvirt toolstack

The only "real" difference is that it's going to use TCG and not KVM
(software emulation rather than hardware virtualization), so it will be
slower, but that's not really a problem if you only care about your VM
until the firmware boots the OS :)

On Fedora, the cross-compiler (and cross-binutils) packages are built
from the following SRPMs:

  https://koji.fedoraproject.org/koji/buildinfo?buildID=921790
  https://koji.fedoraproject.org/koji/buildinfo?buildID=912429

("cross-gcc", "cross-binutils")

Linaro distributes distro-independent cross compilers:

  http://www.linaro.org/downloads/

Build instructions for ArmVirtQemu, and usage hints for the QEMU command
line, can be found in the Linaro Wiki:

  https://wiki.linaro.org/LEG/UEFIforQEMU

Thanks
Laszlo

>> (3) Starting with your v2, please add a reminder to your blurb -- for
>> Ard and myself -- that before merging this, we should regression-test it
>> on aarch64.
>>
> 
> Will do

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Laszlo Ersek 7 years, 4 months ago
On 08/09/17 19:56, Laszlo Ersek wrote:

> Build instructions for ArmVirtQemu, and usage hints for the QEMU command
> line, can be found in the Linaro Wiki:
> 
>   https://wiki.linaro.org/LEG/UEFIforQEMU

It's visible in the Wiki article, but I'd like to point it out
nevertheless, that you can select the virtio-mmio transport for a given
virtio device model by typing

  -device virtio-whatever-device

rather than

  -device virtio-whatever-pci

Thanks
Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Brijesh Singh 7 years, 4 months ago

On 08/09/2017 12:56 PM, Laszlo Ersek wrote:

>>
>> Sure, I will make the necessary changes in VirtioMmioDeviceLib and try
>> do the build test but I don't have aarch64 platform to verify at the
>> runtime.
> 
> Actually, dependent on your GNU/Linux distribution, it is pretty easy to
> do on x86_64 too. It comes together from two parts:
> 
> (1) installing an aarch64 cross compiler
> (2) installing qemu-system-aarch64 (from source or distro package),
>      and then either using it directly (from the cmdline) or with the
>      libvirt toolstack
> 
> The only "real" difference is that it's going to use TCG and not KVM
> (software emulation rather than hardware virtualization), so it will be
> slower, but that's not really a problem if you only care about your VM
> until the firmware boots the OS :)
> 
> On Fedora, the cross-compiler (and cross-binutils) packages are built
> from the following SRPMs:
> 
>    https://koji.fedoraproject.org/koji/buildinfo?buildID=921790
>    https://koji.fedoraproject.org/koji/buildinfo?buildID=912429
> 
> ("cross-gcc", "cross-binutils")
> 
> Linaro distributes distro-independent cross compilers:
> 
>    http://www.linaro.org/downloads/
> 
> Build instructions for ArmVirtQemu, and usage hints for the QEMU command
> line, can be found in the Linaro Wiki:
> 
>    https://wiki.linaro.org/LEG/UEFIforQEMU

v2 is almost ready for review, I have made all the necessary changes in
VirtioMmioDeviceLib and it builds fine for aarch64 but I am having trouble
booting the qemu-aarch64 in general. On serial console I see the UEFI debug
messages but it never reaches to UEFI shell.

I have been following the steps from  https://wiki.linaro.org/LEG/UEFIforQEMU

qemu-system-aarch64 \
  -m 1024 \
  -cpu cortex-a57 \
  -M virt  \
  -bios QEMU_EFI.fd \
  -serial stdio

I tried this steps with and without my patches and it resulted in the same.
It seems like I am missing something in the qemu cli, do I need to pass
special dtb file or something similar ?

P.S: the wiki talks about a prebuilt qcow2 image but the link is dead, instead
I downloaded a qcow2 file from Ubuntu repo.

-Brijesh
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Laszlo Ersek 7 years, 4 months ago
On 08/12/17 00:22, Brijesh Singh wrote:
> 
> 
> On 08/09/2017 12:56 PM, Laszlo Ersek wrote:
> 
>>>
>>> Sure, I will make the necessary changes in VirtioMmioDeviceLib and try
>>> do the build test but I don't have aarch64 platform to verify at the
>>> runtime.
>>
>> Actually, dependent on your GNU/Linux distribution, it is pretty easy to
>> do on x86_64 too. It comes together from two parts:
>>
>> (1) installing an aarch64 cross compiler
>> (2) installing qemu-system-aarch64 (from source or distro package),
>>      and then either using it directly (from the cmdline) or with the
>>      libvirt toolstack
>>
>> The only "real" difference is that it's going to use TCG and not KVM
>> (software emulation rather than hardware virtualization), so it will be
>> slower, but that's not really a problem if you only care about your VM
>> until the firmware boots the OS :)
>>
>> On Fedora, the cross-compiler (and cross-binutils) packages are built
>> from the following SRPMs:
>>
>>    https://koji.fedoraproject.org/koji/buildinfo?buildID=921790
>>    https://koji.fedoraproject.org/koji/buildinfo?buildID=912429
>>
>> ("cross-gcc", "cross-binutils")
>>
>> Linaro distributes distro-independent cross compilers:
>>
>>    http://www.linaro.org/downloads/
>>
>> Build instructions for ArmVirtQemu, and usage hints for the QEMU command
>> line, can be found in the Linaro Wiki:
>>
>>    https://wiki.linaro.org/LEG/UEFIforQEMU
> 
> v2 is almost ready for review, I have made all the necessary changes in
> VirtioMmioDeviceLib and it builds fine for aarch64 but I am having trouble
> booting the qemu-aarch64 in general. On serial console I see the UEFI debug
> messages but it never reaches to UEFI shell.
> 
> I have been following the steps from 
> https://wiki.linaro.org/LEG/UEFIforQEMU
> 
> qemu-system-aarch64 \
>  -m 1024 \
>  -cpu cortex-a57 \
>  -M virt  \
>  -bios QEMU_EFI.fd \
>  -serial stdio
> 
> I tried this steps with and without my patches and it resulted in the same.
> It seems like I am missing something in the qemu cli, do I need to pass
> special dtb file or something similar ?

The above command line is not right ("-bios"). Please scroll down the
wiki page, to the section heading saying "Using persistent UEFI
variables". There it explains how to pad the images and how to use two
-pflash options. ... Perhaps even that part of the article is a bit
out-of-date now.

Basically, today ArmVirtQemu should be used the same way as OVMF, except
for the padding. The build produces two files:
- QEMU_EFI.fd (fw binary)
- QEMU_VARS.fd (varstore template)

Each should be padded to 64MiB with zeros at the end (write a small
script for that), then use them with two pflash drives similarly to OVMF.

Thanks!
Laszlo

> P.S: the wiki talks about a prebuilt qcow2 image but the link is dead,
> instead
> I downloaded a qcow2 file from Ubuntu repo.
> 
> -Brijesh

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Brijesh Singh 7 years, 4 months ago
Hi Laszlo,

On 08/15/2017 05:42 AM, Laszlo Ersek wrote:
[snip]

>>
>> I have been following the steps from
>> https://wiki.linaro.org/LEG/UEFIforQEMU
>>
>> qemu-system-aarch64 \
>>   -m 1024 \
>>   -cpu cortex-a57 \
>>   -M virt  \
>>   -bios QEMU_EFI.fd \
>>   -serial stdio
>>
>> I tried this steps with and without my patches and it resulted in the same.
>> It seems like I am missing something in the qemu cli, do I need to pass
>> special dtb file or something similar ?
> 
> The above command line is not right ("-bios"). Please scroll down the
> wiki page, to the section heading saying "Using persistent UEFI
> variables". There it explains how to pad the images and how to use two
> -pflash options. ... Perhaps even that part of the article is a bit
> out-of-date now.
> 
> Basically, today ArmVirtQemu should be used the same way as OVMF, except
> for the padding. The build produces two files:
> - QEMU_EFI.fd (fw binary)
> - QEMU_VARS.fd (varstore template)
> 
> Each should be padded to 64MiB with zeros at the end (write a small
> script for that), then use them with two pflash drives similarly to OVMF.
> 

Still no luck, you can see my log error [1]. I never get to UEFI shell, I have
tried with and without virtio disk.

https://gist.github.com/codomania/0aed024702b817761ee55fd30929200a

I will continuing googling ...

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Laszlo Ersek 7 years, 4 months ago
On 08/15/17 21:32, Brijesh Singh wrote:
> Hi Laszlo,
> 
> On 08/15/2017 05:42 AM, Laszlo Ersek wrote:
> [snip]
> 
>>>
>>> I have been following the steps from
>>> https://wiki.linaro.org/LEG/UEFIforQEMU
>>>
>>> qemu-system-aarch64 \
>>>   -m 1024 \
>>>   -cpu cortex-a57 \
>>>   -M virt  \
>>>   -bios QEMU_EFI.fd \
>>>   -serial stdio
>>>
>>> I tried this steps with and without my patches and it resulted in the
>>> same.
>>> It seems like I am missing something in the qemu cli, do I need to pass
>>> special dtb file or something similar ?
>>
>> The above command line is not right ("-bios"). Please scroll down the
>> wiki page, to the section heading saying "Using persistent UEFI
>> variables". There it explains how to pad the images and how to use two
>> -pflash options. ... Perhaps even that part of the article is a bit
>> out-of-date now.
>>
>> Basically, today ArmVirtQemu should be used the same way as OVMF, except
>> for the padding. The build produces two files:
>> - QEMU_EFI.fd (fw binary)
>> - QEMU_VARS.fd (varstore template)
>>
>> Each should be padded to 64MiB with zeros at the end (write a small
>> script for that), then use them with two pflash drives similarly to OVMF.
>>
> 
> Still no luck, you can see my log error [1]. I never get to UEFI shell,
> I have
> tried with and without virtio disk.
> 
> https://gist.github.com/codomania/0aed024702b817761ee55fd30929200a
> 
> I will continuing googling ...

In order to get as detailed as possible logs, I suggest adding the
following option to the ArmVirtQemu build command line:

  -D DEBUG_PRINT_ERROR_LEVEL=0x8040004F

The current log looks quite strange to me in places, but I'm not sure if
that's because there are problems in those parts, or because the log
does not contain DEBUG_VERBOSE entries.

Thanks
Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Brijesh Singh 7 years, 4 months ago

On 08/15/2017 02:48 PM, Laszlo Ersek wrote:
> On 08/15/17 21:32, Brijesh Singh wrote:
>> Hi Laszlo,
>>
>> On 08/15/2017 05:42 AM, Laszlo Ersek wrote:
>> [snip]
>>
>>>>
>>>> I have been following the steps from
>>>> https://wiki.linaro.org/LEG/UEFIforQEMU
>>>>
>>>> qemu-system-aarch64 \
>>>>    -m 1024 \
>>>>    -cpu cortex-a57 \
>>>>    -M virt  \
>>>>    -bios QEMU_EFI.fd \
>>>>    -serial stdio
>>>>
>>>> I tried this steps with and without my patches and it resulted in the
>>>> same.
>>>> It seems like I am missing something in the qemu cli, do I need to pass
>>>> special dtb file or something similar ?
>>>
>>> The above command line is not right ("-bios"). Please scroll down the
>>> wiki page, to the section heading saying "Using persistent UEFI
>>> variables". There it explains how to pad the images and how to use two
>>> -pflash options. ... Perhaps even that part of the article is a bit
>>> out-of-date now.
>>>
>>> Basically, today ArmVirtQemu should be used the same way as OVMF, except
>>> for the padding. The build produces two files:
>>> - QEMU_EFI.fd (fw binary)
>>> - QEMU_VARS.fd (varstore template)
>>>
>>> Each should be padded to 64MiB with zeros at the end (write a small
>>> script for that), then use them with two pflash drives similarly to OVMF.
>>>
>>
>> Still no luck, you can see my log error [1]. I never get to UEFI shell,
>> I have
>> tried with and without virtio disk.
>>
>> https://gist.github.com/codomania/0aed024702b817761ee55fd30929200a
>>
>> I will continuing googling ...
> 
> In order to get as detailed as possible logs, I suggest adding the
> following option to the ArmVirtQemu build command line:
> 
>    -D DEBUG_PRINT_ERROR_LEVEL=0x8040004F
> 
> The current log looks quite strange to me in places, but I'm not sure if
> that's because there are problems in those parts, or because the log
> does not contain DEBUG_VERBOSE entries.
> 


https://gist.github.com/codomania/8b2fc5424fda259236405c5e257d8f47

I am using Ubuntu 16.04 for builds and runs

$ qemu-system-aarch64 --version
QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.14), Copyright (c) 2003-2008 Fabrice Bellard

-Brijesh
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Laszlo Ersek 7 years, 4 months ago
On 08/15/17 22:26, Brijesh Singh wrote:
> 
> 
> On 08/15/2017 02:48 PM, Laszlo Ersek wrote:
>> On 08/15/17 21:32, Brijesh Singh wrote:
>>> Hi Laszlo,
>>>
>>> On 08/15/2017 05:42 AM, Laszlo Ersek wrote:
>>> [snip]
>>>
>>>>>
>>>>> I have been following the steps from
>>>>> https://wiki.linaro.org/LEG/UEFIforQEMU
>>>>>
>>>>> qemu-system-aarch64 \
>>>>>    -m 1024 \
>>>>>    -cpu cortex-a57 \
>>>>>    -M virt  \
>>>>>    -bios QEMU_EFI.fd \
>>>>>    -serial stdio
>>>>>
>>>>> I tried this steps with and without my patches and it resulted in the
>>>>> same.
>>>>> It seems like I am missing something in the qemu cli, do I need to
>>>>> pass
>>>>> special dtb file or something similar ?
>>>>
>>>> The above command line is not right ("-bios"). Please scroll down the
>>>> wiki page, to the section heading saying "Using persistent UEFI
>>>> variables". There it explains how to pad the images and how to use two
>>>> -pflash options. ... Perhaps even that part of the article is a bit
>>>> out-of-date now.
>>>>
>>>> Basically, today ArmVirtQemu should be used the same way as OVMF,
>>>> except
>>>> for the padding. The build produces two files:
>>>> - QEMU_EFI.fd (fw binary)
>>>> - QEMU_VARS.fd (varstore template)
>>>>
>>>> Each should be padded to 64MiB with zeros at the end (write a small
>>>> script for that), then use them with two pflash drives similarly to
>>>> OVMF.
>>>>
>>>
>>> Still no luck, you can see my log error [1]. I never get to UEFI shell,
>>> I have
>>> tried with and without virtio disk.
>>>
>>> https://gist.github.com/codomania/0aed024702b817761ee55fd30929200a
>>>
>>> I will continuing googling ...
>>
>> In order to get as detailed as possible logs, I suggest adding the
>> following option to the ArmVirtQemu build command line:
>>
>>    -D DEBUG_PRINT_ERROR_LEVEL=0x8040004F
>>
>> The current log looks quite strange to me in places, but I'm not sure if
>> that's because there are problems in those parts, or because the log
>> does not contain DEBUG_VERBOSE entries.
>>
> 
> 
> https://gist.github.com/codomania/8b2fc5424fda259236405c5e257d8f47
> 
> I am using Ubuntu 16.04 for builds and runs
> 
> $ qemu-system-aarch64 --version
> QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.14), Copyright
> (c) 2003-2008 Fabrice Bellard

What is your complete QEMU command line?

Thanks
Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Brijesh Singh 7 years, 4 months ago

On 08/15/2017 03:39 PM, Laszlo Ersek wrote:
> On 08/15/17 22:26, Brijesh Singh wrote:
>>
>>
>> On 08/15/2017 02:48 PM, Laszlo Ersek wrote:
>>> On 08/15/17 21:32, Brijesh Singh wrote:
>>>> Hi Laszlo,
>>>>
>>>> On 08/15/2017 05:42 AM, Laszlo Ersek wrote:
>>>> [snip]
>>>>
>>>>>>
>>>>>> I have been following the steps from
>>>>>> https://wiki.linaro.org/LEG/UEFIforQEMU
>>>>>>
>>>>>> qemu-system-aarch64 \
>>>>>>     -m 1024 \
>>>>>>     -cpu cortex-a57 \
>>>>>>     -M virt  \
>>>>>>     -bios QEMU_EFI.fd \
>>>>>>     -serial stdio
>>>>>>
>>>>>> I tried this steps with and without my patches and it resulted in the
>>>>>> same.
>>>>>> It seems like I am missing something in the qemu cli, do I need to
>>>>>> pass
>>>>>> special dtb file or something similar ?
>>>>>
>>>>> The above command line is not right ("-bios"). Please scroll down the
>>>>> wiki page, to the section heading saying "Using persistent UEFI
>>>>> variables". There it explains how to pad the images and how to use two
>>>>> -pflash options. ... Perhaps even that part of the article is a bit
>>>>> out-of-date now.
>>>>>
>>>>> Basically, today ArmVirtQemu should be used the same way as OVMF,
>>>>> except
>>>>> for the padding. The build produces two files:
>>>>> - QEMU_EFI.fd (fw binary)
>>>>> - QEMU_VARS.fd (varstore template)
>>>>>
>>>>> Each should be padded to 64MiB with zeros at the end (write a small
>>>>> script for that), then use them with two pflash drives similarly to
>>>>> OVMF.
>>>>>
>>>>
>>>> Still no luck, you can see my log error [1]. I never get to UEFI shell,
>>>> I have
>>>> tried with and without virtio disk.
>>>>
>>>> https://gist.github.com/codomania/0aed024702b817761ee55fd30929200a
>>>>
>>>> I will continuing googling ...
>>>
>>> In order to get as detailed as possible logs, I suggest adding the
>>> following option to the ArmVirtQemu build command line:
>>>
>>>     -D DEBUG_PRINT_ERROR_LEVEL=0x8040004F
>>>
>>> The current log looks quite strange to me in places, but I'm not sure if
>>> that's because there are problems in those parts, or because the log
>>> does not contain DEBUG_VERBOSE entries.
>>>
>>
>>
>> https://gist.github.com/codomania/8b2fc5424fda259236405c5e257d8f47
>>
>> I am using Ubuntu 16.04 for builds and runs
>>
>> $ qemu-system-aarch64 --version
>> QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.14), Copyright
>> (c) 2003-2008 Fabrice Bellard
> 
> What is your complete QEMU command line?
> 

I have been using the following two qemu cli

# qemu-system-aarch64 -m 2048 -cpu cortex-a57 -M virt \
      -pflash flash0.img -pflash flash1.img \
      -nographic

# qemu-system-aarch64 -m 2048 -cpu cortex-a57 -M virt \
       -pflash flash0.img -pflash flash1.img \
      -drive if=none,file=/home/brijesh/xenial-server-cloudimg-arm64.img,id=hd0,format=raw -device virtio-blk-device,drive=hd0 \
      -nographic
   


_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Laszlo Ersek 7 years, 4 months ago
On 08/15/17 22:44, Brijesh Singh wrote:
> 
> 
> On 08/15/2017 03:39 PM, Laszlo Ersek wrote:
>> On 08/15/17 22:26, Brijesh Singh wrote:
>>>
>>>
>>> On 08/15/2017 02:48 PM, Laszlo Ersek wrote:
>>>> On 08/15/17 21:32, Brijesh Singh wrote:
>>>>> Hi Laszlo,
>>>>>
>>>>> On 08/15/2017 05:42 AM, Laszlo Ersek wrote:
>>>>> [snip]
>>>>>
>>>>>>>
>>>>>>> I have been following the steps from
>>>>>>> https://wiki.linaro.org/LEG/UEFIforQEMU
>>>>>>>
>>>>>>> qemu-system-aarch64 \
>>>>>>>     -m 1024 \
>>>>>>>     -cpu cortex-a57 \
>>>>>>>     -M virt  \
>>>>>>>     -bios QEMU_EFI.fd \
>>>>>>>     -serial stdio
>>>>>>>
>>>>>>> I tried this steps with and without my patches and it resulted in
>>>>>>> the
>>>>>>> same.
>>>>>>> It seems like I am missing something in the qemu cli, do I need to
>>>>>>> pass
>>>>>>> special dtb file or something similar ?
>>>>>>
>>>>>> The above command line is not right ("-bios"). Please scroll down the
>>>>>> wiki page, to the section heading saying "Using persistent UEFI
>>>>>> variables". There it explains how to pad the images and how to use
>>>>>> two
>>>>>> -pflash options. ... Perhaps even that part of the article is a bit
>>>>>> out-of-date now.
>>>>>>
>>>>>> Basically, today ArmVirtQemu should be used the same way as OVMF,
>>>>>> except
>>>>>> for the padding. The build produces two files:
>>>>>> - QEMU_EFI.fd (fw binary)
>>>>>> - QEMU_VARS.fd (varstore template)
>>>>>>
>>>>>> Each should be padded to 64MiB with zeros at the end (write a small
>>>>>> script for that), then use them with two pflash drives similarly to
>>>>>> OVMF.
>>>>>>
>>>>>
>>>>> Still no luck, you can see my log error [1]. I never get to UEFI
>>>>> shell,
>>>>> I have
>>>>> tried with and without virtio disk.
>>>>>
>>>>> https://gist.github.com/codomania/0aed024702b817761ee55fd30929200a
>>>>>
>>>>> I will continuing googling ...
>>>>
>>>> In order to get as detailed as possible logs, I suggest adding the
>>>> following option to the ArmVirtQemu build command line:
>>>>
>>>>     -D DEBUG_PRINT_ERROR_LEVEL=0x8040004F
>>>>
>>>> The current log looks quite strange to me in places, but I'm not
>>>> sure if
>>>> that's because there are problems in those parts, or because the log
>>>> does not contain DEBUG_VERBOSE entries.
>>>>
>>>
>>>
>>> https://gist.github.com/codomania/8b2fc5424fda259236405c5e257d8f47
>>>
>>> I am using Ubuntu 16.04 for builds and runs
>>>
>>> $ qemu-system-aarch64 --version
>>> QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.14), Copyright
>>> (c) 2003-2008 Fabrice Bellard
>>
>> What is your complete QEMU command line?
>>
> 
> I have been using the following two qemu cli
> 
> # qemu-system-aarch64 -m 2048 -cpu cortex-a57 -M virt \
>      -pflash flash0.img -pflash flash1.img \
>      -nographic
> 
> # qemu-system-aarch64 -m 2048 -cpu cortex-a57 -M virt \
>       -pflash flash0.img -pflash flash1.img \
>      -drive
> if=none,file=/home/brijesh/xenial-server-cloudimg-arm64.img,id=hd0,format=raw
> -device virtio-blk-device,drive=hd0 \
>      -nographic

I can reproduce the boot hang. It looks like an edk2 regression. I'm
currently bisecting the issue.

Thanks
Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Laszlo Ersek 7 years, 4 months ago
On 08/07/17 13:58, Brijesh Singh wrote:
> Currently, virtio drivers provides the system physical address to the device.
> However, some systems may feature an IOMMU that requires the drivers to pass
> the device addresses to the device - which are then translated by the IOMMU 
> into physical addresses in memory. The patch series introduces new member
> functions in VIRTIO_DEVICE_PROTOCOL which can be used for mapping a system
> physical address to device address.
> 
> The approach that this patch series takes is to maps the system physical
> address to device address for buffers (including rings, device specifc
> request and response  pointed by vring descriptor, and any further memory 
> reference by those request and response).
> 
> Patch 1 - 3:
>  Defines and implements new member functions to map a system physical address
>  to device address. The patch implements Laszlo's suggestion [1].
> 
> Patch 4 - 7:
>  Allocates the vring buffer using newly added member functions and provides
>  some helper functions.
> 
> Patch 8:
>  Update the virtio-blk driver to use newly added member functions to map the
>  addresses.
>  Verified using the following qemu cli
> 
>  # $QEMU \
>     -drive file=${IMAGE},if=none,id=disk0 \
>     -device virtio-blk-pci,drive=disk0
> 
>  # $QEMU \
>     -drive file=${IMAGE},if=none,id=disk0 \
>     -device virtio-blk-pci,drive=disk0,disable-legacy=on
> 
> Patch 9:
>  Update the virtio-scsi driver to use newly added member functions to map the
>  addresses.
>  Verified using the following qemu cli
> 
>  # $QEMU \
>     -drive file=${IMAGE},if=none,id=disk0 \
>     -device scsi-hd,drive=disk0
>     -device virtio-scsi-pci,id=scsi \
> 
>  # $QEMU \
>     -drive file=${IMAGE},if=none,id=disk0 \
>     -device scsi-hd,drive=disk0 \
>     -device virtio-scsi-pci,id=scsi,disable-legacy=on \
> 
> Patch 10 - 13:
>  Update the virtio-net driver to use newly added member functions to map the
>  addresses.
>  Verified using the following qemu cli
> 
>  # $QEMU \
>     -netdev type=tap,id=net0 \
>     -device virtio-net-pci,netdev=net0,romfile=
> 
>  # $QEMU \
>     -netdev type=tap,id=net0 \
>     -device virtio-net-pci,netdev=net0,disable-legacy=on,romfile=
> 
> Patch 14:
>  Update the virtio-rng driver to use newly added member functions to map the
>  addresses.
>  Verified using the following qemu cli
> 
>  # $QEMU \
>     -device virtio-rng-pci
> 
>  # $QEMU \
>     -device virtio-rng-pci,disable-legacy=on
> 
>  And succesfully ran RngTest.efi from SecurityPkg/Application
> 
> Repo: https://github.com/codomania/edk2
> Branch: virtio-support
> 
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> 
> Brijesh Singh (14):
>   OvmfPkg/Virtio: Introduce new member functions in
>     VIRTIO_DEVICE_PROTOCOL
>   OvmfPkg/Virtio10Dxe: Implement new member functions
>   OvmfPkg/VirtioPciDeviceDxe: Implement new member functions
>   OvmfPkg/VirtioLib: Add SharedBuffer helper functions
>   OvmfPkg/VirtioLib: Pass VirtIo instance in VringInit/Uinit()
>   OvmfPkg/VirtioLib: Add functions to map/unmap VRING
>   OvmfPkg/VirtioLib: Use AllocateShared() to allocate vring buffer
>   OvmfPkg/VirtioBlkDxe: Use DeviceAddresses in vring descriptors
>   OvmfPkg/VirtioScsiDxe: Use DeviceAddresses in vring descriptors
>   OvmfPkg/VirtioNetDxe: Allocate Tx and Rx ring using
>     AllocateSharedPage()
>   OvmfPkg/VirtioNetDxe: Allocate RxBuf using AllocateSharedPages()
>   OvmfPkg/VirtioNetDxe: Dynamically allocate transmit header
>   OvmfPkg/VirtioNetDxe: Use DeviceAddress in transmit vring descriptors
>   OvmfPkg/VirtioRngDxe: Use DeviceAddresses in vring descriptors
> 
>  OvmfPkg/Include/Library/VirtioLib.h             | 198 ++++++++++++-
>  OvmfPkg/Include/Protocol/VirtioDevice.h         | 121 ++++++++
>  OvmfPkg/VirtioBlkDxe/VirtioBlk.h                |   1 +
>  OvmfPkg/VirtioNetDxe/VirtioNet.h                |  25 +-
>  OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.h    |  34 +++
>  OvmfPkg/VirtioRngDxe/VirtioRng.h                |   1 +
>  OvmfPkg/VirtioScsiDxe/VirtioScsi.h              |   1 +
>  OvmfPkg/Library/VirtioLib/VirtioLib.c           | 296 +++++++++++++++++++-
>  OvmfPkg/Virtio10Dxe/Virtio10.c                  | 114 +++++++-
>  OvmfPkg/VirtioBlkDxe/VirtioBlk.c                | 104 ++++++-
>  OvmfPkg/VirtioGpuDxe/Commands.c                 |   7 +-
>  OvmfPkg/VirtioNetDxe/Events.c                   |  24 ++
>  OvmfPkg/VirtioNetDxe/SnpGetStatus.c             |  19 +-
>  OvmfPkg/VirtioNetDxe/SnpInitialize.c            | 102 +++++--
>  OvmfPkg/VirtioNetDxe/SnpSharedHelpers.c         | 119 +++++++-
>  OvmfPkg/VirtioNetDxe/SnpShutdown.c              |  16 +-
>  OvmfPkg/VirtioNetDxe/SnpTransmit.c              |  22 +-
>  OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.c    |   7 +-
>  OvmfPkg/VirtioPciDeviceDxe/VirtioPciFunctions.c |  66 +++++
>  OvmfPkg/VirtioRngDxe/VirtioRng.c                |  54 +++-
>  OvmfPkg/VirtioScsiDxe/VirtioScsi.c              | 121 +++++++-
>  21 files changed, 1378 insertions(+), 74 deletions(-)
> 

The conversion for VirtioGpuDxe seems to be missing; is it on your todo
list? (Not that it's urgent at all; in fact I'll suggest delaying the
more complex drivers in a minute.)

Thanks,
Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Brijesh Singh 7 years, 4 months ago

On 08/09/2017 05:38 PM, Laszlo Ersek wrote:
> On 08/07/17 13:58, Brijesh Singh wrote:
>> Currently, virtio drivers provides the system physical address to the device.
>> However, some systems may feature an IOMMU that requires the drivers to pass
>> the device addresses to the device - which are then translated by the IOMMU
>> into physical addresses in memory. The patch series introduces new member
>> functions in VIRTIO_DEVICE_PROTOCOL which can be used for mapping a system
>> physical address to device address.
>>
>> The approach that this patch series takes is to maps the system physical
>> address to device address for buffers (including rings, device specifc
>> request and response  pointed by vring descriptor, and any further memory
>> reference by those request and response).
>>
>> Patch 1 - 3:
>>   Defines and implements new member functions to map a system physical address
>>   to device address. The patch implements Laszlo's suggestion [1].
>>
>> Patch 4 - 7:
>>   Allocates the vring buffer using newly added member functions and provides
>>   some helper functions.
>>
>> Patch 8:
>>   Update the virtio-blk driver to use newly added member functions to map the
>>   addresses.
>>   Verified using the following qemu cli
>>
>>   # $QEMU \
>>      -drive file=${IMAGE},if=none,id=disk0 \
>>      -device virtio-blk-pci,drive=disk0
>>
>>   # $QEMU \
>>      -drive file=${IMAGE},if=none,id=disk0 \
>>      -device virtio-blk-pci,drive=disk0,disable-legacy=on
>>
>> Patch 9:
>>   Update the virtio-scsi driver to use newly added member functions to map the
>>   addresses.
>>   Verified using the following qemu cli
>>
>>   # $QEMU \
>>      -drive file=${IMAGE},if=none,id=disk0 \
>>      -device scsi-hd,drive=disk0
>>      -device virtio-scsi-pci,id=scsi \
>>
>>   # $QEMU \
>>      -drive file=${IMAGE},if=none,id=disk0 \
>>      -device scsi-hd,drive=disk0 \
>>      -device virtio-scsi-pci,id=scsi,disable-legacy=on \
>>
>> Patch 10 - 13:
>>   Update the virtio-net driver to use newly added member functions to map the
>>   addresses.
>>   Verified using the following qemu cli
>>
>>   # $QEMU \
>>      -netdev type=tap,id=net0 \
>>      -device virtio-net-pci,netdev=net0,romfile=
>>
>>   # $QEMU \
>>      -netdev type=tap,id=net0 \
>>      -device virtio-net-pci,netdev=net0,disable-legacy=on,romfile=
>>
>> Patch 14:
>>   Update the virtio-rng driver to use newly added member functions to map the
>>   addresses.
>>   Verified using the following qemu cli
>>
>>   # $QEMU \
>>      -device virtio-rng-pci
>>
>>   # $QEMU \
>>      -device virtio-rng-pci,disable-legacy=on
>>
>>   And succesfully ran RngTest.efi from SecurityPkg/Application
>>
>> Repo: https://github.com/codomania/edk2
>> Branch: virtio-support
>>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>> Cc: Tom Lendacky <thomas.lendacky@amd.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>>
>> Brijesh Singh (14):
>>    OvmfPkg/Virtio: Introduce new member functions in
>>      VIRTIO_DEVICE_PROTOCOL
>>    OvmfPkg/Virtio10Dxe: Implement new member functions
>>    OvmfPkg/VirtioPciDeviceDxe: Implement new member functions
>>    OvmfPkg/VirtioLib: Add SharedBuffer helper functions
>>    OvmfPkg/VirtioLib: Pass VirtIo instance in VringInit/Uinit()
>>    OvmfPkg/VirtioLib: Add functions to map/unmap VRING
>>    OvmfPkg/VirtioLib: Use AllocateShared() to allocate vring buffer
>>    OvmfPkg/VirtioBlkDxe: Use DeviceAddresses in vring descriptors
>>    OvmfPkg/VirtioScsiDxe: Use DeviceAddresses in vring descriptors
>>    OvmfPkg/VirtioNetDxe: Allocate Tx and Rx ring using
>>      AllocateSharedPage()
>>    OvmfPkg/VirtioNetDxe: Allocate RxBuf using AllocateSharedPages()
>>    OvmfPkg/VirtioNetDxe: Dynamically allocate transmit header
>>    OvmfPkg/VirtioNetDxe: Use DeviceAddress in transmit vring descriptors
>>    OvmfPkg/VirtioRngDxe: Use DeviceAddresses in vring descriptors
>>
>>   OvmfPkg/Include/Library/VirtioLib.h             | 198 ++++++++++++-
>>   OvmfPkg/Include/Protocol/VirtioDevice.h         | 121 ++++++++
>>   OvmfPkg/VirtioBlkDxe/VirtioBlk.h                |   1 +
>>   OvmfPkg/VirtioNetDxe/VirtioNet.h                |  25 +-
>>   OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.h    |  34 +++
>>   OvmfPkg/VirtioRngDxe/VirtioRng.h                |   1 +
>>   OvmfPkg/VirtioScsiDxe/VirtioScsi.h              |   1 +
>>   OvmfPkg/Library/VirtioLib/VirtioLib.c           | 296 +++++++++++++++++++-
>>   OvmfPkg/Virtio10Dxe/Virtio10.c                  | 114 +++++++-
>>   OvmfPkg/VirtioBlkDxe/VirtioBlk.c                | 104 ++++++-
>>   OvmfPkg/VirtioGpuDxe/Commands.c                 |   7 +-
>>   OvmfPkg/VirtioNetDxe/Events.c                   |  24 ++
>>   OvmfPkg/VirtioNetDxe/SnpGetStatus.c             |  19 +-
>>   OvmfPkg/VirtioNetDxe/SnpInitialize.c            | 102 +++++--
>>   OvmfPkg/VirtioNetDxe/SnpSharedHelpers.c         | 119 +++++++-
>>   OvmfPkg/VirtioNetDxe/SnpShutdown.c              |  16 +-
>>   OvmfPkg/VirtioNetDxe/SnpTransmit.c              |  22 +-
>>   OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.c    |   7 +-
>>   OvmfPkg/VirtioPciDeviceDxe/VirtioPciFunctions.c |  66 +++++
>>   OvmfPkg/VirtioRngDxe/VirtioRng.c                |  54 +++-
>>   OvmfPkg/VirtioScsiDxe/VirtioScsi.c              | 121 +++++++-
>>   21 files changed, 1378 insertions(+), 74 deletions(-)
>>
> 
> The conversion for VirtioGpuDxe seems to be missing; is it on your todo
> list? (Not that it's urgent at all; in fact I'll suggest delaying the
> more complex drivers in a minute.)
> 

Actually I was thinking to delay it for now, once we get a basic support
enabled then we can revisit the VirtioGpuDxe, I still need to figure out
qemu cli and Uefi app to test it. I will add in TODO list so that.


> Thanks,
> Laszlo
> 
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH v1 00/14] OvmfPkg/Virtio: Add APIs to map system physical to device address
Posted by Laszlo Ersek 7 years, 4 months ago
On 08/10/17 00:44, Brijesh Singh wrote:
> 
> 
> On 08/09/2017 05:38 PM, Laszlo Ersek wrote:
>> On 08/07/17 13:58, Brijesh Singh wrote:
>>> Currently, virtio drivers provides the system physical address to the
>>> device.
>>> However, some systems may feature an IOMMU that requires the drivers
>>> to pass
>>> the device addresses to the device - which are then translated by the
>>> IOMMU
>>> into physical addresses in memory. The patch series introduces new
>>> member
>>> functions in VIRTIO_DEVICE_PROTOCOL which can be used for mapping a
>>> system
>>> physical address to device address.
>>>
>>> The approach that this patch series takes is to maps the system physical
>>> address to device address for buffers (including rings, device specifc
>>> request and response  pointed by vring descriptor, and any further
>>> memory
>>> reference by those request and response).
>>>
>>> Patch 1 - 3:
>>>   Defines and implements new member functions to map a system
>>> physical address
>>>   to device address. The patch implements Laszlo's suggestion [1].
>>>
>>> Patch 4 - 7:
>>>   Allocates the vring buffer using newly added member functions and
>>> provides
>>>   some helper functions.
>>>
>>> Patch 8:
>>>   Update the virtio-blk driver to use newly added member functions to
>>> map the
>>>   addresses.
>>>   Verified using the following qemu cli
>>>
>>>   # $QEMU \
>>>      -drive file=${IMAGE},if=none,id=disk0 \
>>>      -device virtio-blk-pci,drive=disk0
>>>
>>>   # $QEMU \
>>>      -drive file=${IMAGE},if=none,id=disk0 \
>>>      -device virtio-blk-pci,drive=disk0,disable-legacy=on
>>>
>>> Patch 9:
>>>   Update the virtio-scsi driver to use newly added member functions
>>> to map the
>>>   addresses.
>>>   Verified using the following qemu cli
>>>
>>>   # $QEMU \
>>>      -drive file=${IMAGE},if=none,id=disk0 \
>>>      -device scsi-hd,drive=disk0
>>>      -device virtio-scsi-pci,id=scsi \
>>>
>>>   # $QEMU \
>>>      -drive file=${IMAGE},if=none,id=disk0 \
>>>      -device scsi-hd,drive=disk0 \
>>>      -device virtio-scsi-pci,id=scsi,disable-legacy=on \
>>>
>>> Patch 10 - 13:
>>>   Update the virtio-net driver to use newly added member functions to
>>> map the
>>>   addresses.
>>>   Verified using the following qemu cli
>>>
>>>   # $QEMU \
>>>      -netdev type=tap,id=net0 \
>>>      -device virtio-net-pci,netdev=net0,romfile=
>>>
>>>   # $QEMU \
>>>      -netdev type=tap,id=net0 \
>>>      -device virtio-net-pci,netdev=net0,disable-legacy=on,romfile=
>>>
>>> Patch 14:
>>>   Update the virtio-rng driver to use newly added member functions to
>>> map the
>>>   addresses.
>>>   Verified using the following qemu cli
>>>
>>>   # $QEMU \
>>>      -device virtio-rng-pci
>>>
>>>   # $QEMU \
>>>      -device virtio-rng-pci,disable-legacy=on
>>>
>>>   And succesfully ran RngTest.efi from SecurityPkg/Application
>>>
>>> Repo: https://github.com/codomania/edk2
>>> Branch: virtio-support
>>>
>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>>> Cc: Tom Lendacky <thomas.lendacky@amd.com>
>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>
>>> Brijesh Singh (14):
>>>    OvmfPkg/Virtio: Introduce new member functions in
>>>      VIRTIO_DEVICE_PROTOCOL
>>>    OvmfPkg/Virtio10Dxe: Implement new member functions
>>>    OvmfPkg/VirtioPciDeviceDxe: Implement new member functions
>>>    OvmfPkg/VirtioLib: Add SharedBuffer helper functions
>>>    OvmfPkg/VirtioLib: Pass VirtIo instance in VringInit/Uinit()
>>>    OvmfPkg/VirtioLib: Add functions to map/unmap VRING
>>>    OvmfPkg/VirtioLib: Use AllocateShared() to allocate vring buffer
>>>    OvmfPkg/VirtioBlkDxe: Use DeviceAddresses in vring descriptors
>>>    OvmfPkg/VirtioScsiDxe: Use DeviceAddresses in vring descriptors
>>>    OvmfPkg/VirtioNetDxe: Allocate Tx and Rx ring using
>>>      AllocateSharedPage()
>>>    OvmfPkg/VirtioNetDxe: Allocate RxBuf using AllocateSharedPages()
>>>    OvmfPkg/VirtioNetDxe: Dynamically allocate transmit header
>>>    OvmfPkg/VirtioNetDxe: Use DeviceAddress in transmit vring descriptors
>>>    OvmfPkg/VirtioRngDxe: Use DeviceAddresses in vring descriptors
>>>
>>>   OvmfPkg/Include/Library/VirtioLib.h             | 198 ++++++++++++-
>>>   OvmfPkg/Include/Protocol/VirtioDevice.h         | 121 ++++++++
>>>   OvmfPkg/VirtioBlkDxe/VirtioBlk.h                |   1 +
>>>   OvmfPkg/VirtioNetDxe/VirtioNet.h                |  25 +-
>>>   OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.h    |  34 +++
>>>   OvmfPkg/VirtioRngDxe/VirtioRng.h                |   1 +
>>>   OvmfPkg/VirtioScsiDxe/VirtioScsi.h              |   1 +
>>>   OvmfPkg/Library/VirtioLib/VirtioLib.c           | 296
>>> +++++++++++++++++++-
>>>   OvmfPkg/Virtio10Dxe/Virtio10.c                  | 114 +++++++-
>>>   OvmfPkg/VirtioBlkDxe/VirtioBlk.c                | 104 ++++++-
>>>   OvmfPkg/VirtioGpuDxe/Commands.c                 |   7 +-
>>>   OvmfPkg/VirtioNetDxe/Events.c                   |  24 ++
>>>   OvmfPkg/VirtioNetDxe/SnpGetStatus.c             |  19 +-
>>>   OvmfPkg/VirtioNetDxe/SnpInitialize.c            | 102 +++++--
>>>   OvmfPkg/VirtioNetDxe/SnpSharedHelpers.c         | 119 +++++++-
>>>   OvmfPkg/VirtioNetDxe/SnpShutdown.c              |  16 +-
>>>   OvmfPkg/VirtioNetDxe/SnpTransmit.c              |  22 +-
>>>   OvmfPkg/VirtioPciDeviceDxe/VirtioPciDevice.c    |   7 +-
>>>   OvmfPkg/VirtioPciDeviceDxe/VirtioPciFunctions.c |  66 +++++
>>>   OvmfPkg/VirtioRngDxe/VirtioRng.c                |  54 +++-
>>>   OvmfPkg/VirtioScsiDxe/VirtioScsi.c              | 121 +++++++-
>>>   21 files changed, 1378 insertions(+), 74 deletions(-)
>>>
>>
>> The conversion for VirtioGpuDxe seems to be missing; is it on your todo
>> list? (Not that it's urgent at all; in fact I'll suggest delaying the
>> more complex drivers in a minute.)
>>
> 
> Actually I was thinking to delay it for now,

OK, that makes sense even from the complexity standpoint.

> once we get a basic support
> enabled then we can revisit the VirtioGpuDxe, I still need to figure out
> qemu cli and Uefi app to test it.

You can read about my original test cases and test instructions in the
following message:

[2] http://mid.mail-archive.com/20160819124932.29711-1-lersek@redhat.com

(

Some background:

The virtio-vga device combines the Virtio GPU and the traditional VGA
framebuffer that is also seen on QXL and stdvga ("Bochs"). If you use
"-device virtio-vga", then QemuVideoDxe will bind the device (this is
intended). So, the right option for specifically using the virtio GPU is
"-device virtio-gpu-pci".

You can read more about this distinction in the following libvirt commit:

[3] http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=706b5b627719

)

Regarding a UEFI app for testing, the UEFI shell and grub suffice:
- do you see the TianoCore boot logo?
- does the UEFI shell work?
- does the shell scroll fine when you press Enter at the bottom of the
  screen?
- does the MODE command work in the shell?
- typing on the serial port, can you disconnect / reconnect the display?
- does grub work?

See again my message [2].

> I will add in TODO list so that.

Sounds good. Once we lay down the foundation (and I can find some time),
I might be able to take on the conversion of VirtioGpuDxe. (If you're OK
with that.)

Thanks!
Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel