The patch series provides support for AMD's new Secure Encrypted
Virtualization (SEV) feature.

SEV is an extension to the AMD-V architecture which supports running
multiple VMs under the control of a hypervisor. The SEV feature allows
the memory contents of a virtual machine (VM) to be transparently encrypted
with a key unique to the guest VM. The memory controller contains a
high performance encryption engine which can be programmed with multiple
keys for use by a different VMs in the system. The programming and
management of these keys is handled by the AMD Secure Processor firmware
which exposes a commands for these tasks.

SEV guest VMs have the concept of private and shared memory.  Private memory is
encrypted with the guest-specific key, while shared memory may be encrypted
with hypervisor key.  Certain types of memory (namely instruction pages and
guest page tables) are always treated as private memory by the hardware.
For data memory, SEV guest VMs can choose which pages they would like to be
private. The choice is done using the standard CPU page tables using the C-bit,
and is fully controlled by the guest. Due to security reasons all the DMA
operations inside the  guest must be performed on shared pages (C-bit clear).
Note that since C-bit is only controllable by the guest OS when it is operating
in 64-bit or 32-bit PAE mode, in all other modes the SEV hardware forces the
C-bit to a 1.

The following links provide additional details:

AMD Memory Encryption whitepaper:
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf

AMD64 Architecture Programmer's Manual:
    http://support.amd.com/TechDocs/24593.pdf
    SME is section 7.10
    SEV is section 15.34

Secure Encrypted Virutualization Key Management:
http://support.amd.com/TechDocs/55766_SEV-KM API_Specification.pdf

KVM Forum Presentation:
http://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf

[1] http://marc.info/?l=linux-mm&m=148846752931115&w=2

---

Patch series is based on commit aff463c825a3
 (Vlv2TbltDevicePkg/FvbRuntimeDxe: correct NumOfLba vararg type in EraseBlocks())

https://github.com/codomania/edk2/tree/v6

The patch series is tested with OvmfIa32.dsc, OvmfIa32X64.dsc and OvmfX64.dsc.
Since memory encryption bit is not accessiable when processor is in 32-bit mode
hence any DMA access in this mode would cause assert. I have also tested the
suspend and resume path, it seems to be working fine. I still need to work to
finish adding the SEV Dma support in QemuFwCfgS3Lib package (see TODO).

Changes since v5:
 - add placeholder gIoMmuAbsentProtocolGuid
 - add PlatformHasIoMmuLib
 - fix indentation

Changes since v4:
 - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
   IoMmuDxe driver. And introduce a placeholder protocol to provide the
   dependency support for the dependent modules.
 - update debug messages to use gEfiCallerBaseName where applicable.
 - fix QemuFwCfgSecLib build errors and simplify SEV support
 - update QemuFwCfgDxeLib to assert when failed to locate IOMMU
 - update comments "host buffer" to " host buffer"

Changes since v3:
 - update AmdSevDxe driver to produce IOMMU protocol
 - remove BmDmaLib dependency
 - update QemuFwCfgLib to use IOMMU protocol to allocate SEV DMA buffer

Changes since v2:
 - move memory encryption CPUID and MSR definition into UefiCpuPkg
 - fix the argument order for SUB instruction in ResetVector and add more
   comments
 - update PlatformPei to use BaseMemEncryptSevLib
 - break the overlong comment lines to 79 chars
 - variable aligment and other formating fixes
 - split the SEV DMA support patch for QemuFwCfgLib into multiple patches as
   recommended by Laszlo
 - add AmdSevDxe driver which runs very early in DXE phase and clear the C-bit
   from MMIO memory region
 - drop 'QemuVideoDxe: Clear C-bit from framebuffer' patch since AmdSevDxe
   driver takes care of clearing the C-bit from MMIO region
 - Verified that Qemu PFLASH works fine with SEV guest, Found a KVM driver issue
   which was causing #PF when PFLASH was enabled. I have submitted patch to
   fix it in upstream http://marc.info/?l=kvm&m=149304930814202&w=2

Changes since v1:
 - bug fixes in OvmfPkg/ResetVector (pointed by Tom Lendacky)
 - add SEV CPUID and MSR register definition in standard include file
 - remove the MemEncryptLib dependency from PlatformPei. Move AmdSevInitialize()
   implementation in local file inside the PlatformPei package
 - rename MemCryptSevLib to MemEncryptSevLib and add functions to set or
   clear memory encryption attribute on memory region
 - integerate SEV support in BmDmaLib
 - split QemuFwCfgDxePei.c into QemuFwCfgDxe.c and QemuFwCfgPei.c to
   allow building seperate QemuFwCfgLib for Dxe and Pei phase
   (recommended by Laszlo Ersek)
 - add SEV support in QemuFwCfgLib
 - clear the memory encryption attribute from framebuffer memory region


TODO:
(Will add these features after basic SEV support patches are accepted in upstream)
 - add support for DMA operation in QemuFwCfgS3Lib when SEV is enabled
 - investigate SMM/SMI support

Cc: Jeff Fan <jeff.fan@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leo Duran <leo.duran@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leo Duran <leo.duran@amd.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <Thomas.Lendacky@amd.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>

Brijesh Singh (17):
  UefiCpuPkg: Define AMD Memory Encryption specific CPUID and MSR
  OvmfPkg/ResetVector: Set C-bit when building initial page table
  OvmfPkg: Update dsc to use IoLib from BaseIoLibIntrinsicSev.inf
  OvmfPkg/BaseMemcryptSevLib: Add SEV helper library
  OvmfPkg/PlatformPei: Set memory encryption PCD when SEV is enabled
  OvmfPkg: Add AmdSevDxe driver
  OvmfPkg: Introduce IoMmuAbsent Protocol GUID
  OvmfPkg: Add PlatformHasIoMmuLib
  OvmfPkg: Add IoMmuDxe driver
  OvmfPkg/QemuFwCfgLib: Provide Pei and Dxe specific library
  OvmfPkg/QemuFwCfgLib: Prepare for SEV support
  OvmfPkg/QemuFwCfgLib: Implement SEV internal function for SEC phase
  OvmfPkg/QemuFwCfgLib: Implement SEV internal functions for PEI phase
  OvmfPkg/QemuFwCfgLib: Implement SEV internal function for Dxe phase
  OvmfPkg/QemuFwCfgLib: Add option to dynamic alloc FW_CFG_DMA Access
  OvmfPkg/QemuFwCfgLib: Add SEV support
  OvmfPkg: update PciHostBridgeDxe to use PlatformHasIoMmuLib

 OvmfPkg/OvmfPkg.dec                                                    |   1 +
 OvmfPkg/OvmfPkgIa32.dsc                                                |  11 +-
 OvmfPkg/OvmfPkgIa32X64.dsc                                             |  12 +-
 OvmfPkg/OvmfPkgX64.dsc                                                 |  12 +-
 OvmfPkg/OvmfPkgIa32.fdf                                                |   1 +
 OvmfPkg/OvmfPkgIa32X64.fdf                                             |   3 +
 OvmfPkg/OvmfPkgX64.fdf                                                 |   3 +
 OvmfPkg/AmdSevDxe/AmdSevDxe.inf                                        |  43 ++
 OvmfPkg/IoMmuDxe/IoMmuDxe.inf                                          |  49 +++
 OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf          |  50 +++
 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf            |  37 ++
 OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} |  15 +-
 OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} |   9 +-
 OvmfPkg/PlatformPei/PlatformPei.inf                                    |   3 +
 OvmfPkg/Include/Library/MemEncryptSevLib.h                             |  81 ++++
 OvmfPkg/IoMmuDxe/AmdSevIoMmu.h                                         |  43 ++
 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h               | 184 ++++++++
 OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLibInternal.h                    |  37 ++
 OvmfPkg/PlatformPei/Platform.h                                         |   5 +
 UefiCpuPkg/Include/Register/Amd/Cpuid.h                                | 162 +++++++
 UefiCpuPkg/Include/Register/Amd/Fam17Msr.h                             |  62 +++
 UefiCpuPkg/Include/Register/Amd/Msr.h                                  |  29 ++
 OvmfPkg/AmdSevDxe/AmdSevDxe.c                                          |  75 ++++
 OvmfPkg/IoMmuDxe/AmdSevIoMmu.c                                         | 459 ++++++++++++++++++++
 OvmfPkg/IoMmuDxe/IoMmuDxe.c                                            |  53 +++
 OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c           |  84 ++++
 OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c        |  90 ++++
 OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c            |  84 ++++
 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c               | 439 +++++++++++++++++++
 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c              |  32 ++
 OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c                            | 230 ++++++++++
 OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLib.c                            |  67 ++-
 OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c}     |  72 ++-
 OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgSec.c                            |  57 +++
 OvmfPkg/PlatformPei/AmdSev.c                                           |  62 +++
 OvmfPkg/PlatformPei/Platform.c                                         |   1 +
 OvmfPkg/ResetVector/Ia32/PageTables64.asm                              |  70 ++-
 37 files changed, 2703 insertions(+), 24 deletions(-)
 create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.inf
 create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.inf
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf
 create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf
 copy OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} (71%)
 rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} (80%)
 create mode 100644 OvmfPkg/Include/Library/MemEncryptSevLib.h
 create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.h
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h
 create mode 100644 UefiCpuPkg/Include/Register/Amd/Cpuid.h
 create mode 100644 UefiCpuPkg/Include/Register/Amd/Fam17Msr.h
 create mode 100644 UefiCpuPkg/Include/Register/Amd/Msr.h
 create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.c
 create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.c
 create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c
 create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c
 create mode 100644 OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c
 rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c} (61%)
 create mode 100644 OvmfPkg/PlatformPei/AmdSev.c

-- 
2.7.4

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Hi Jordan and Laszlo,

Ping.

It has been a while, Do you have any further feedbacks on this series ?
If you want then I can rebase the patches before you commit into upstream repos.

-Brijesh

On 05/26/2017 09:43 AM, Brijesh Singh wrote:
> The patch series provides support for AMD's new Secure Encrypted
> Virtualization (SEV) feature.
> 
> SEV is an extension to the AMD-V architecture which supports running
> multiple VMs under the control of a hypervisor. The SEV feature allows
> the memory contents of a virtual machine (VM) to be transparently encrypted
> with a key unique to the guest VM. The memory controller contains a
> high performance encryption engine which can be programmed with multiple
> keys for use by a different VMs in the system. The programming and
> management of these keys is handled by the AMD Secure Processor firmware
> which exposes a commands for these tasks.
> 
> SEV guest VMs have the concept of private and shared memory.  Private memory is
> encrypted with the guest-specific key, while shared memory may be encrypted
> with hypervisor key.  Certain types of memory (namely instruction pages and
> guest page tables) are always treated as private memory by the hardware.
> For data memory, SEV guest VMs can choose which pages they would like to be
> private. The choice is done using the standard CPU page tables using the C-bit,
> and is fully controlled by the guest. Due to security reasons all the DMA
> operations inside the  guest must be performed on shared pages (C-bit clear).
> Note that since C-bit is only controllable by the guest OS when it is operating
> in 64-bit or 32-bit PAE mode, in all other modes the SEV hardware forces the
> C-bit to a 1.
> 
> The following links provide additional details:
> 
> AMD Memory Encryption whitepaper:
> http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf
> 
> AMD64 Architecture Programmer's Manual:
>      http://support.amd.com/TechDocs/24593.pdf
>      SME is section 7.10
>      SEV is section 15.34
> 
> Secure Encrypted Virutualization Key Management:
> http://support.amd.com/TechDocs/55766_SEV-KM API_Specification.pdf
> 
> KVM Forum Presentation:
> http://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf
> 
> [1] http://marc.info/?l=linux-mm&m=148846752931115&w=2
> 
> ---
> 
> Patch series is based on commit aff463c825a3
>   (Vlv2TbltDevicePkg/FvbRuntimeDxe: correct NumOfLba vararg type in EraseBlocks())
> 
> https://github.com/codomania/edk2/tree/v6
> 
> The patch series is tested with OvmfIa32.dsc, OvmfIa32X64.dsc and OvmfX64.dsc.
> Since memory encryption bit is not accessiable when processor is in 32-bit mode
> hence any DMA access in this mode would cause assert. I have also tested the
> suspend and resume path, it seems to be working fine. I still need to work to
> finish adding the SEV Dma support in QemuFwCfgS3Lib package (see TODO).
> 
> Changes since v5:
>   - add placeholder gIoMmuAbsentProtocolGuid
>   - add PlatformHasIoMmuLib
>   - fix indentation
> 
> Changes since v4:
>   - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>     IoMmuDxe driver. And introduce a placeholder protocol to provide the
>     dependency support for the dependent modules.
>   - update debug messages to use gEfiCallerBaseName where applicable.
>   - fix QemuFwCfgSecLib build errors and simplify SEV support
>   - update QemuFwCfgDxeLib to assert when failed to locate IOMMU
>   - update comments "host buffer" to " host buffer"
> 
> Changes since v3:
>   - update AmdSevDxe driver to produce IOMMU protocol
>   - remove BmDmaLib dependency
>   - update QemuFwCfgLib to use IOMMU protocol to allocate SEV DMA buffer
> 
> Changes since v2:
>   - move memory encryption CPUID and MSR definition into UefiCpuPkg
>   - fix the argument order for SUB instruction in ResetVector and add more
>     comments
>   - update PlatformPei to use BaseMemEncryptSevLib
>   - break the overlong comment lines to 79 chars
>   - variable aligment and other formating fixes
>   - split the SEV DMA support patch for QemuFwCfgLib into multiple patches as
>     recommended by Laszlo
>   - add AmdSevDxe driver which runs very early in DXE phase and clear the C-bit
>     from MMIO memory region
>   - drop 'QemuVideoDxe: Clear C-bit from framebuffer' patch since AmdSevDxe
>     driver takes care of clearing the C-bit from MMIO region
>   - Verified that Qemu PFLASH works fine with SEV guest, Found a KVM driver issue
>     which was causing #PF when PFLASH was enabled. I have submitted patch to
>     fix it in upstream http://marc.info/?l=kvm&m=149304930814202&w=2
> 
> Changes since v1:
>   - bug fixes in OvmfPkg/ResetVector (pointed by Tom Lendacky)
>   - add SEV CPUID and MSR register definition in standard include file
>   - remove the MemEncryptLib dependency from PlatformPei. Move AmdSevInitialize()
>     implementation in local file inside the PlatformPei package
>   - rename MemCryptSevLib to MemEncryptSevLib and add functions to set or
>     clear memory encryption attribute on memory region
>   - integerate SEV support in BmDmaLib
>   - split QemuFwCfgDxePei.c into QemuFwCfgDxe.c and QemuFwCfgPei.c to
>     allow building seperate QemuFwCfgLib for Dxe and Pei phase
>     (recommended by Laszlo Ersek)
>   - add SEV support in QemuFwCfgLib
>   - clear the memory encryption attribute from framebuffer memory region
> 
> 
> TODO:
> (Will add these features after basic SEV support patches are accepted in upstream)
>   - add support for DMA operation in QemuFwCfgS3Lib when SEV is enabled
>   - investigate SMM/SMI support
> 
> Cc: Jeff Fan <jeff.fan@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Leo Duran <leo.duran@amd.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leo Duran <leo.duran@amd.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <Thomas.Lendacky@amd.com>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
> 
> Brijesh Singh (17):
>    UefiCpuPkg: Define AMD Memory Encryption specific CPUID and MSR
>    OvmfPkg/ResetVector: Set C-bit when building initial page table
>    OvmfPkg: Update dsc to use IoLib from BaseIoLibIntrinsicSev.inf
>    OvmfPkg/BaseMemcryptSevLib: Add SEV helper library
>    OvmfPkg/PlatformPei: Set memory encryption PCD when SEV is enabled
>    OvmfPkg: Add AmdSevDxe driver
>    OvmfPkg: Introduce IoMmuAbsent Protocol GUID
>    OvmfPkg: Add PlatformHasIoMmuLib
>    OvmfPkg: Add IoMmuDxe driver
>    OvmfPkg/QemuFwCfgLib: Provide Pei and Dxe specific library
>    OvmfPkg/QemuFwCfgLib: Prepare for SEV support
>    OvmfPkg/QemuFwCfgLib: Implement SEV internal function for SEC phase
>    OvmfPkg/QemuFwCfgLib: Implement SEV internal functions for PEI phase
>    OvmfPkg/QemuFwCfgLib: Implement SEV internal function for Dxe phase
>    OvmfPkg/QemuFwCfgLib: Add option to dynamic alloc FW_CFG_DMA Access
>    OvmfPkg/QemuFwCfgLib: Add SEV support
>    OvmfPkg: update PciHostBridgeDxe to use PlatformHasIoMmuLib
> 
>   OvmfPkg/OvmfPkg.dec                                                    |   1 +
>   OvmfPkg/OvmfPkgIa32.dsc                                                |  11 +-
>   OvmfPkg/OvmfPkgIa32X64.dsc                                             |  12 +-
>   OvmfPkg/OvmfPkgX64.dsc                                                 |  12 +-
>   OvmfPkg/OvmfPkgIa32.fdf                                                |   1 +
>   OvmfPkg/OvmfPkgIa32X64.fdf                                             |   3 +
>   OvmfPkg/OvmfPkgX64.fdf                                                 |   3 +
>   OvmfPkg/AmdSevDxe/AmdSevDxe.inf                                        |  43 ++
>   OvmfPkg/IoMmuDxe/IoMmuDxe.inf                                          |  49 +++
>   OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf          |  50 +++
>   OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf            |  37 ++
>   OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} |  15 +-
>   OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} |   9 +-
>   OvmfPkg/PlatformPei/PlatformPei.inf                                    |   3 +
>   OvmfPkg/Include/Library/MemEncryptSevLib.h                             |  81 ++++
>   OvmfPkg/IoMmuDxe/AmdSevIoMmu.h                                         |  43 ++
>   OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h               | 184 ++++++++
>   OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLibInternal.h                    |  37 ++
>   OvmfPkg/PlatformPei/Platform.h                                         |   5 +
>   UefiCpuPkg/Include/Register/Amd/Cpuid.h                                | 162 +++++++
>   UefiCpuPkg/Include/Register/Amd/Fam17Msr.h                             |  62 +++
>   UefiCpuPkg/Include/Register/Amd/Msr.h                                  |  29 ++
>   OvmfPkg/AmdSevDxe/AmdSevDxe.c                                          |  75 ++++
>   OvmfPkg/IoMmuDxe/AmdSevIoMmu.c                                         | 459 ++++++++++++++++++++
>   OvmfPkg/IoMmuDxe/IoMmuDxe.c                                            |  53 +++
>   OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c           |  84 ++++
>   OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c        |  90 ++++
>   OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c            |  84 ++++
>   OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c               | 439 +++++++++++++++++++
>   OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c              |  32 ++
>   OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c                            | 230 ++++++++++
>   OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLib.c                            |  67 ++-
>   OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c}     |  72 ++-
>   OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgSec.c                            |  57 +++
>   OvmfPkg/PlatformPei/AmdSev.c                                           |  62 +++
>   OvmfPkg/PlatformPei/Platform.c                                         |   1 +
>   OvmfPkg/ResetVector/Ia32/PageTables64.asm                              |  70 ++-
>   37 files changed, 2703 insertions(+), 24 deletions(-)
>   create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.inf
>   create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.inf
>   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf
>   create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf
>   copy OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} (71%)
>   rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} (80%)
>   create mode 100644 OvmfPkg/Include/Library/MemEncryptSevLib.h
>   create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.h
>   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h
>   create mode 100644 UefiCpuPkg/Include/Register/Amd/Cpuid.h
>   create mode 100644 UefiCpuPkg/Include/Register/Amd/Fam17Msr.h
>   create mode 100644 UefiCpuPkg/Include/Register/Amd/Msr.h
>   create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.c
>   create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.c
>   create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.c
>   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
>   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c
>   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c
>   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c
>   create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c
>   create mode 100644 OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c
>   rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c} (61%)
>   create mode 100644 OvmfPkg/PlatformPei/AmdSev.c
> 
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 07/06/17 00:31, Brijesh Singh wrote:
> Hi Jordan and Laszlo,
> 
> Ping.
> 
> It has been a while, Do you have any further feedbacks on this series ?
> If you want then I can rebase the patches before you commit into
> upstream repos.

From my side, refreshing the v7 series (posted on 2017-Jun-22) against
current master would be welcome, just to sort out any context
differences that may have crept in meanwhile.

Other than that, I'm ready to merge v7 (and have been ready since my v7
review, posted on 2017-Jun-23).

Thanks,
Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 07/05/2017 06:38 PM, Laszlo Ersek wrote:
> On 07/06/17 00:31, Brijesh Singh wrote:
>> Hi Jordan and Laszlo,
>>
>> Ping.
>>
>> It has been a while, Do you have any further feedbacks on this series ?
>> If you want then I can rebase the patches before you commit into
>> upstream repos.
> 
>  From my side, refreshing the v7 series (posted on 2017-Jun-22) against
> current master would be welcome, just to sort out any context
> differences that may have crept in meanwhile.
> 
> Other than that, I'm ready to merge v7 (and have been ready since my v7
> review, posted on 2017-Jun-23).
> 

Thanks Laszlo, I just pulled latest and there was one hunk failure when applying v7 series.
I will send v8 which applies cleanly on top of latest. I also noticed that Jeff Fan has
already applied the first patch (cpuid) from this series and I will drop it in v8.


> Thanks,
> Laszlo
> 
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 2017-07-05 15:31:20, Brijesh Singh wrote:
> Hi Jordan and Laszlo,
> 
> Ping.
> 
> It has been a while, Do you have any further feedbacks on this series ?
> If you want then I can rebase the patches before you commit into upstream repos.
> 

I'm still dissappointed by the APRIORI usage.

As I understand it, you are also dissatisfied with this approach and
you hope to improve things by somehow hooking into DXE Core. Is that
true? If so, can you create a bugzilla regarding this feature? When
would you plan to work to address that?

I guess with that resolved, you could add an Acked-by from me.

In general, it'd also be nice to move the processor features to more
generic places, although that may be challenging if the next step is
some kind of platform hook from DXE Core. Maybe if the DXE Core calls
out to some protocol or signals an event then a driver in UefiCpuPkg
could handle the protocol implementation to modify the page tables.

-Jordan

> 
> On 05/26/2017 09:43 AM, Brijesh Singh wrote:
> > The patch series provides support for AMD's new Secure Encrypted
> > Virtualization (SEV) feature.
> > 
> > SEV is an extension to the AMD-V architecture which supports running
> > multiple VMs under the control of a hypervisor. The SEV feature allows
> > the memory contents of a virtual machine (VM) to be transparently encrypted
> > with a key unique to the guest VM. The memory controller contains a
> > high performance encryption engine which can be programmed with multiple
> > keys for use by a different VMs in the system. The programming and
> > management of these keys is handled by the AMD Secure Processor firmware
> > which exposes a commands for these tasks.
> > 
> > SEV guest VMs have the concept of private and shared memory.  Private memory is
> > encrypted with the guest-specific key, while shared memory may be encrypted
> > with hypervisor key.  Certain types of memory (namely instruction pages and
> > guest page tables) are always treated as private memory by the hardware.
> > For data memory, SEV guest VMs can choose which pages they would like to be
> > private. The choice is done using the standard CPU page tables using the C-bit,
> > and is fully controlled by the guest. Due to security reasons all the DMA
> > operations inside the  guest must be performed on shared pages (C-bit clear).
> > Note that since C-bit is only controllable by the guest OS when it is operating
> > in 64-bit or 32-bit PAE mode, in all other modes the SEV hardware forces the
> > C-bit to a 1.
> > 
> > The following links provide additional details:
> > 
> > AMD Memory Encryption whitepaper:
> > http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf
> > 
> > AMD64 Architecture Programmer's Manual:
> >      http://support.amd.com/TechDocs/24593.pdf
> >      SME is section 7.10
> >      SEV is section 15.34
> > 
> > Secure Encrypted Virutualization Key Management:
> > http://support.amd.com/TechDocs/55766_SEV-KM API_Specification.pdf
> > 
> > KVM Forum Presentation:
> > http://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf
> > 
> > [1] http://marc.info/?l=linux-mm&m=148846752931115&w=2
> > 
> > ---
> > 
> > Patch series is based on commit aff463c825a3
> >   (Vlv2TbltDevicePkg/FvbRuntimeDxe: correct NumOfLba vararg type in EraseBlocks())
> > 
> > https://github.com/codomania/edk2/tree/v6
> > 
> > The patch series is tested with OvmfIa32.dsc, OvmfIa32X64.dsc and OvmfX64.dsc.
> > Since memory encryption bit is not accessiable when processor is in 32-bit mode
> > hence any DMA access in this mode would cause assert. I have also tested the
> > suspend and resume path, it seems to be working fine. I still need to work to
> > finish adding the SEV Dma support in QemuFwCfgS3Lib package (see TODO).
> > 
> > Changes since v5:
> >   - add placeholder gIoMmuAbsentProtocolGuid
> >   - add PlatformHasIoMmuLib
> >   - fix indentation
> > 
> > Changes since v4:
> >   - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
> >     IoMmuDxe driver. And introduce a placeholder protocol to provide the
> >     dependency support for the dependent modules.
> >   - update debug messages to use gEfiCallerBaseName where applicable.
> >   - fix QemuFwCfgSecLib build errors and simplify SEV support
> >   - update QemuFwCfgDxeLib to assert when failed to locate IOMMU
> >   - update comments "host buffer" to " host buffer"
> > 
> > Changes since v3:
> >   - update AmdSevDxe driver to produce IOMMU protocol
> >   - remove BmDmaLib dependency
> >   - update QemuFwCfgLib to use IOMMU protocol to allocate SEV DMA buffer
> > 
> > Changes since v2:
> >   - move memory encryption CPUID and MSR definition into UefiCpuPkg
> >   - fix the argument order for SUB instruction in ResetVector and add more
> >     comments
> >   - update PlatformPei to use BaseMemEncryptSevLib
> >   - break the overlong comment lines to 79 chars
> >   - variable aligment and other formating fixes
> >   - split the SEV DMA support patch for QemuFwCfgLib into multiple patches as
> >     recommended by Laszlo
> >   - add AmdSevDxe driver which runs very early in DXE phase and clear the C-bit
> >     from MMIO memory region
> >   - drop 'QemuVideoDxe: Clear C-bit from framebuffer' patch since AmdSevDxe
> >     driver takes care of clearing the C-bit from MMIO region
> >   - Verified that Qemu PFLASH works fine with SEV guest, Found a KVM driver issue
> >     which was causing #PF when PFLASH was enabled. I have submitted patch to
> >     fix it in upstream http://marc.info/?l=kvm&m=149304930814202&w=2
> > 
> > Changes since v1:
> >   - bug fixes in OvmfPkg/ResetVector (pointed by Tom Lendacky)
> >   - add SEV CPUID and MSR register definition in standard include file
> >   - remove the MemEncryptLib dependency from PlatformPei. Move AmdSevInitialize()
> >     implementation in local file inside the PlatformPei package
> >   - rename MemCryptSevLib to MemEncryptSevLib and add functions to set or
> >     clear memory encryption attribute on memory region
> >   - integerate SEV support in BmDmaLib
> >   - split QemuFwCfgDxePei.c into QemuFwCfgDxe.c and QemuFwCfgPei.c to
> >     allow building seperate QemuFwCfgLib for Dxe and Pei phase
> >     (recommended by Laszlo Ersek)
> >   - add SEV support in QemuFwCfgLib
> >   - clear the memory encryption attribute from framebuffer memory region
> > 
> > 
> > TODO:
> > (Will add these features after basic SEV support patches are accepted in upstream)
> >   - add support for DMA operation in QemuFwCfgS3Lib when SEV is enabled
> >   - investigate SMM/SMI support
> > 
> > Cc: Jeff Fan <jeff.fan@intel.com>
> > Cc: Liming Gao <liming.gao@intel.com>
> > Cc: Leo Duran <leo.duran@amd.com>
> > Cc: Jordan Justen <jordan.l.justen@intel.com>
> > Cc: Laszlo Ersek <lersek@redhat.com>
> > Cc: Leo Duran <leo.duran@amd.com>
> > Cc: Jiewen Yao <jiewen.yao@intel.com>
> > Cc: Tom Lendacky <Thomas.Lendacky@amd.com>
> > Contributed-under: TianoCore Contribution Agreement 1.0
> > Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
> > 
> > Brijesh Singh (17):
> >    UefiCpuPkg: Define AMD Memory Encryption specific CPUID and MSR
> >    OvmfPkg/ResetVector: Set C-bit when building initial page table
> >    OvmfPkg: Update dsc to use IoLib from BaseIoLibIntrinsicSev.inf
> >    OvmfPkg/BaseMemcryptSevLib: Add SEV helper library
> >    OvmfPkg/PlatformPei: Set memory encryption PCD when SEV is enabled
> >    OvmfPkg: Add AmdSevDxe driver
> >    OvmfPkg: Introduce IoMmuAbsent Protocol GUID
> >    OvmfPkg: Add PlatformHasIoMmuLib
> >    OvmfPkg: Add IoMmuDxe driver
> >    OvmfPkg/QemuFwCfgLib: Provide Pei and Dxe specific library
> >    OvmfPkg/QemuFwCfgLib: Prepare for SEV support
> >    OvmfPkg/QemuFwCfgLib: Implement SEV internal function for SEC phase
> >    OvmfPkg/QemuFwCfgLib: Implement SEV internal functions for PEI phase
> >    OvmfPkg/QemuFwCfgLib: Implement SEV internal function for Dxe phase
> >    OvmfPkg/QemuFwCfgLib: Add option to dynamic alloc FW_CFG_DMA Access
> >    OvmfPkg/QemuFwCfgLib: Add SEV support
> >    OvmfPkg: update PciHostBridgeDxe to use PlatformHasIoMmuLib
> > 
> >   OvmfPkg/OvmfPkg.dec                                                    |   1 +
> >   OvmfPkg/OvmfPkgIa32.dsc                                                |  11 +-
> >   OvmfPkg/OvmfPkgIa32X64.dsc                                             |  12 +-
> >   OvmfPkg/OvmfPkgX64.dsc                                                 |  12 +-
> >   OvmfPkg/OvmfPkgIa32.fdf                                                |   1 +
> >   OvmfPkg/OvmfPkgIa32X64.fdf                                             |   3 +
> >   OvmfPkg/OvmfPkgX64.fdf                                                 |   3 +
> >   OvmfPkg/AmdSevDxe/AmdSevDxe.inf                                        |  43 ++
> >   OvmfPkg/IoMmuDxe/IoMmuDxe.inf                                          |  49 +++
> >   OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf          |  50 +++
> >   OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf            |  37 ++
> >   OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} |  15 +-
> >   OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} |   9 +-
> >   OvmfPkg/PlatformPei/PlatformPei.inf                                    |   3 +
> >   OvmfPkg/Include/Library/MemEncryptSevLib.h                             |  81 ++++
> >   OvmfPkg/IoMmuDxe/AmdSevIoMmu.h                                         |  43 ++
> >   OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h               | 184 ++++++++
> >   OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLibInternal.h                    |  37 ++
> >   OvmfPkg/PlatformPei/Platform.h                                         |   5 +
> >   UefiCpuPkg/Include/Register/Amd/Cpuid.h                                | 162 +++++++
> >   UefiCpuPkg/Include/Register/Amd/Fam17Msr.h                             |  62 +++
> >   UefiCpuPkg/Include/Register/Amd/Msr.h                                  |  29 ++
> >   OvmfPkg/AmdSevDxe/AmdSevDxe.c                                          |  75 ++++
> >   OvmfPkg/IoMmuDxe/AmdSevIoMmu.c                                         | 459 ++++++++++++++++++++
> >   OvmfPkg/IoMmuDxe/IoMmuDxe.c                                            |  53 +++
> >   OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c           |  84 ++++
> >   OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c        |  90 ++++
> >   OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c            |  84 ++++
> >   OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c               | 439 +++++++++++++++++++
> >   OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c              |  32 ++
> >   OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c                            | 230 ++++++++++
> >   OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLib.c                            |  67 ++-
> >   OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c}     |  72 ++-
> >   OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgSec.c                            |  57 +++
> >   OvmfPkg/PlatformPei/AmdSev.c                                           |  62 +++
> >   OvmfPkg/PlatformPei/Platform.c                                         |   1 +
> >   OvmfPkg/ResetVector/Ia32/PageTables64.asm                              |  70 ++-
> >   37 files changed, 2703 insertions(+), 24 deletions(-)
> >   create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.inf
> >   create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.inf
> >   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf
> >   create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf
> >   copy OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} (71%)
> >   rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} (80%)
> >   create mode 100644 OvmfPkg/Include/Library/MemEncryptSevLib.h
> >   create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.h
> >   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h
> >   create mode 100644 UefiCpuPkg/Include/Register/Amd/Cpuid.h
> >   create mode 100644 UefiCpuPkg/Include/Register/Amd/Fam17Msr.h
> >   create mode 100644 UefiCpuPkg/Include/Register/Amd/Msr.h
> >   create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.c
> >   create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.c
> >   create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.c
> >   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
> >   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c
> >   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c
> >   create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c
> >   create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c
> >   create mode 100644 OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c
> >   rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c} (61%)
> >   create mode 100644 OvmfPkg/PlatformPei/AmdSev.c
> > 
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 07/06/2017 11:45 AM, Jordan Justen wrote:
> On 2017-07-05 15:31:20, Brijesh Singh wrote:
>> Hi Jordan and Laszlo,
>>
>> Ping.
>>
>> It has been a while, Do you have any further feedbacks on this series ?
>> If you want then I can rebase the patches before you commit into upstream repos.
>>
> 
> I'm still dissappointed by the APRIORI usage.
> 
> As I understand it, you are also dissatisfied with this approach and
> you hope to improve things by somehow hooking into DXE Core. Is that
> true? If so, can you create a bugzilla regarding this feature? When
> would you plan to work to address that?
> 

I think we agree in that this particular use-case has shown the need for re-thinking
the existing GCD interface. However, the problem we are trying to solve with this
patch-set is enabling the SEV feature. As it turns out, we can do so within the
existing GCD framework by simply leveraging the APRIORI hook already in use by OvmfPkg.

In that context, our proposal is that we limit the scope of this patch-set to simply
enabling the SEV feature, and then allow the 'GCD experts' to separately propose updates
to the framework.


> I guess with that resolved, you could add an Acked-by from me.
> 
> In general, it'd also be nice to move the processor features to more
> generic places, although that may be challenging if the next step is
> some kind of platform hook from DXE Core. Maybe if the DXE Core calls
> out to some protocol or signals an event then a driver in UefiCpuPkg
> could handle the protocol implementation to modify the page tables.
> 
> -Jordan
> 
>
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 07/06/17 22:11, Brijesh Singh wrote:
>
>
> On 07/06/2017 11:45 AM, Jordan Justen wrote:
>> On 2017-07-05 15:31:20, Brijesh Singh wrote:
>>> Hi Jordan and Laszlo,
>>>
>>> Ping.
>>>
>>> It has been a while, Do you have any further feedbacks on this
>>> series ? If you want then I can rebase the patches before you commit
>>> into upstream repos.
>>>
>>
>> I'm still dissappointed by the APRIORI usage.
>>
>> As I understand it, you are also dissatisfied with this approach and
>> you hope to improve things by somehow hooking into DXE Core. Is that
>> true? If so, can you create a bugzilla regarding this feature? When
>> would you plan to work to address that?
>>
>
> I think we agree in that this particular use-case has shown the need
> for re-thinking the existing GCD interface. However, the problem we
> are trying to solve with this patch-set is enabling the SEV feature.

I agree.

> As it turns out, we can do so within the existing GCD framework by
> simply leveraging the APRIORI hook already in use by OvmfPkg.

Not just in OvmfPkg. CorebootPayloadPkg, DuetPkg, EmulatorPkg, Nt32Pkg
and Vlv2TbltDevicePkg also use "APRIORI DXE".

See e.g. commit 70420e31a04b ("Nt32Pkg FDF: Move StatusCode Handler run
earlier in DXE phase", 2017-01-20).

> In that context, our proposal is that we limit the scope of this
> patch-set to simply enabling the SEV feature, and then allow the 'GCD
> experts' to separately propose updates to the framework.

I agree. Based on the past discussions, even said experts and edk2
maintainers aren't united on the optimal approach here.

I think filing a BZ (or even a PI spec ticket!) for figuring out the
best scope and location for the platformization of GCD, or for the same
of the page table setup, is justified.

However, I disagree that such a ticket should block this series, or that
Brijesh should prioritize such a ticket after this initial OVMF series
is merged. Brijesh has been handling SEV enablement in other projects as
well, such as KVM, and there's still a whole lot to do after these
initial OVMF patches are merged.

Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 2017-07-06 13:11:03, Brijesh Singh wrote:
> 
> 
> On 07/06/2017 11:45 AM, Jordan Justen wrote:
> > On 2017-07-05 15:31:20, Brijesh Singh wrote:
> >> Hi Jordan and Laszlo,
> >>
> >> Ping.
> >>
> >> It has been a while, Do you have any further feedbacks on this series ?
> >> If you want then I can rebase the patches before you commit into upstream repos.
> >>
> > 
> > I'm still dissappointed by the APRIORI usage.
> > 
> > As I understand it, you are also dissatisfied with this approach and
> > you hope to improve things by somehow hooking into DXE Core. Is that
> > true? If so, can you create a bugzilla regarding this feature? When
> > would you plan to work to address that?
> > 
> 
> I think we agree in that this particular use-case has shown the need for re-thinking
> the existing GCD interface. However, the problem we are trying to solve with this
> patch-set is enabling the SEV feature. As it turns out, we can do so within the
> existing GCD framework by simply leveraging the APRIORI hook already in use by OvmfPkg.
> 
> In that context, our proposal is that we limit the scope of this patch-set to simply
> enabling the SEV feature, and then allow the 'GCD experts' to separately propose updates
> to the framework.

This sounds like you don't plan to work on this, but will just leave
it to the 'GCD experts'. Is that right?

I am asking that you file and own a bugzilla for this. You'd obviously
need to work with the package owners though. Unless you drive this, I
don't think anyone will be motivated enough to get it fixed.

-Jordan

> 
> > I guess with that resolved, you could add an Acked-by from me.
> > 
> > In general, it'd also be nice to move the processor features to more
> > generic places, although that may be challenging if the next step is
> > some kind of platform hook from DXE Core. Maybe if the DXE Core calls
> > out to some protocol or signals an event then a driver in UefiCpuPkg
> > could handle the protocol implementation to modify the page tables.
> > 
> > -Jordan
> > 
> >
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Jordan,

> -----Original Message-----
> From: Jordan Justen [mailto:jordan.l.justen@intel.com]
> Sent: Thursday, July 06, 2017 4:43 PM
> To: Singh, Brijesh <brijesh.singh@amd.com>; edk2-devel@lists.01.org
> Cc: Singh, Brijesh <brijesh.singh@amd.com>; Lendacky, Thomas
> <Thomas.Lendacky@amd.com>; Duran, Leo <leo.duran@amd.com>; Jeff
> Fan <jeff.fan@intel.com>; Liming Gao <liming.gao@intel.com>; Laszlo Ersek
> <lersek@redhat.com>; Jiewen Yao <jiewen.yao@intel.com>; Michael D
> Kinney <michael.d.kinney@intel.com>
> Subject: Re: [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
> 
> On 2017-07-06 13:11:03, Brijesh Singh wrote:
> >
> >
> > On 07/06/2017 11:45 AM, Jordan Justen wrote:
> > > On 2017-07-05 15:31:20, Brijesh Singh wrote:
> > >> Hi Jordan and Laszlo,
> > >>
> > >> Ping.
> > >>
> > >> It has been a while, Do you have any further feedbacks on this series ?
> > >> If you want then I can rebase the patches before you commit into
> upstream repos.
> > >>
> > >
> > > I'm still dissappointed by the APRIORI usage.
> > >
> > > As I understand it, you are also dissatisfied with this approach and
> > > you hope to improve things by somehow hooking into DXE Core. Is that
> > > true? If so, can you create a bugzilla regarding this feature? When
> > > would you plan to work to address that?
> > >
> >
> > I think we agree in that this particular use-case has shown the need
> > for re-thinking the existing GCD interface. However, the problem we
> > are trying to solve with this patch-set is enabling the SEV feature.
> > As it turns out, we can do so within the existing GCD framework by simply
> leveraging the APRIORI hook already in use by OvmfPkg.
> >
> > In that context, our proposal is that we limit the scope of this
> > patch-set to simply enabling the SEV feature, and then allow the 'GCD
> > experts' to separately propose updates to the framework.
> 
> This sounds like you don't plan to work on this, but will just leave it to the
> 'GCD experts'. Is that right?
> 
> I am asking that you file and own a bugzilla for this. You'd obviously need to
> work with the package owners though. Unless you drive this, I don't think
> anyone will be motivated enough to get it fixed.
> 
[Duran, Leo] 
OK, we will file the BZ... But please don't let that keep you from moving these patches forward.
Thanks! :-).

> -Jordan
> 
> >
> > > I guess with that resolved, you could add an Acked-by from me.
> > >
> > > In general, it'd also be nice to move the processor features to more
> > > generic places, although that may be challenging if the next step is
> > > some kind of platform hook from DXE Core. Maybe if the DXE Core
> > > calls out to some protocol or signals an event then a driver in
> > > UefiCpuPkg could handle the protocol implementation to modify the
> page tables.
> > >
> > > -Jordan
> > >
> > >
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

> On Jul 6, 2017, at 2:42 PM, Jordan Justen <jordan.l.justen@intel.com> wrote:
> 
> On 2017-07-06 13:11:03, Brijesh Singh wrote:
>> 
>> 
>> On 07/06/2017 11:45 AM, Jordan Justen wrote:
>>> On 2017-07-05 15:31:20, Brijesh Singh wrote:
>>>> Hi Jordan and Laszlo,
>>>> 
>>>> Ping.
>>>> 
>>>> It has been a while, Do you have any further feedbacks on this series ?
>>>> If you want then I can rebase the patches before you commit into upstream repos.
>>>> 
>>> 
>>> I'm still dissappointed by the APRIORI usage.
>>> 
>>> As I understand it, you are also dissatisfied with this approach and
>>> you hope to improve things by somehow hooking into DXE Core. Is that
>>> true? If so, can you create a bugzilla regarding this feature? When
>>> would you plan to work to address that?
>>> 
>> 
>> I think we agree in that this particular use-case has shown the need for re-thinking
>> the existing GCD interface. However, the problem we are trying to solve with this
>> patch-set is enabling the SEV feature. As it turns out, we can do so within the
>> existing GCD framework by simply leveraging the APRIORI hook already in use by OvmfPkg.
>> 
>> In that context, our proposal is that we limit the scope of this patch-set to simply
>> enabling the SEV feature, and then allow the 'GCD experts' to separately propose updates
>> to the framework.
> 
> This sounds like you don't plan to work on this, but will just leave
> it to the 'GCD experts'. Is that right?
> 
> I am asking that you file and own a bugzilla for this. You'd obviously
> need to work with the package owners though. Unless you drive this, I
> don't think anyone will be motivated enough to get it fixed.
> 

If some one will make a write up on this mailing list summarizing the issue with the GCD design, and what features are needed I can start a conversation on the PI working group list. 

Thanks,

Andrew Fish

> -Jordan
> 
>> 
>>> I guess with that resolved, you could add an Acked-by from me.
>>> 
>>> In general, it'd also be nice to move the processor features to more
>>> generic places, although that may be challenging if the next step is
>>> some kind of platform hook from DXE Core. Maybe if the DXE Core calls
>>> out to some protocol or signals an event then a driver in UefiCpuPkg
>>> could handle the protocol implementation to modify the page tables.
>>> 
>>> -Jordan
>>> 
>>> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Hi Andrew, 

> -----Original Message-----
> From: afish@apple.com [mailto:afish@apple.com]
> Sent: Thursday, July 06, 2017 4:46 PM
> To: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Singh, Brijesh <brijesh.singh@amd.com>; edk2-devel-01 <edk2-
> devel@lists.01.org>; Lendacky, Thomas <Thomas.Lendacky@amd.com>;
> Liming Gao <liming.gao@intel.com>; Duran, Leo <leo.duran@amd.com>;
> Mike Kinney <michael.d.kinney@intel.com>; Jiewen Yao
> <jiewen.yao@intel.com>; Laszlo Ersek <lersek@redhat.com>; Jeff Fan
> <jeff.fan@intel.com>
> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization
> (AMD)
> 
> 
> > On Jul 6, 2017, at 2:42 PM, Jordan Justen <jordan.l.justen@intel.com>
> wrote:
> >
> > On 2017-07-06 13:11:03, Brijesh Singh wrote:
> >>
> >>
> >> On 07/06/2017 11:45 AM, Jordan Justen wrote:
> >>> On 2017-07-05 15:31:20, Brijesh Singh wrote:
> >>>> Hi Jordan and Laszlo,
> >>>>
> >>>> Ping.
> >>>>
> >>>> It has been a while, Do you have any further feedbacks on this series ?
> >>>> If you want then I can rebase the patches before you commit into
> upstream repos.
> >>>>
> >>>
> >>> I'm still dissappointed by the APRIORI usage.
> >>>
> >>> As I understand it, you are also dissatisfied with this approach and
> >>> you hope to improve things by somehow hooking into DXE Core. Is that
> >>> true? If so, can you create a bugzilla regarding this feature? When
> >>> would you plan to work to address that?
> >>>
> >>
> >> I think we agree in that this particular use-case has shown the need
> >> for re-thinking the existing GCD interface. However, the problem we
> >> are trying to solve with this patch-set is enabling the SEV feature.
> >> As it turns out, we can do so within the existing GCD framework by simply
> leveraging the APRIORI hook already in use by OvmfPkg.
> >>
> >> In that context, our proposal is that we limit the scope of this
> >> patch-set to simply enabling the SEV feature, and then allow the 'GCD
> >> experts' to separately propose updates to the framework.
> >
> > This sounds like you don't plan to work on this, but will just leave
> > it to the 'GCD experts'. Is that right?
> >
> > I am asking that you file and own a bugzilla for this. You'd obviously
> > need to work with the package owners though. Unless you drive this, I
> > don't think anyone will be motivated enough to get it fixed.
> >
> 
> If some one will make a write up on this mailing list summarizing the issue
> with the GCD design, and what features are needed I can start a
> conversation on the PI working group list.
> 
> Thanks,
> 
> Andrew Fish
[Duran, Leo] 
Excellent proposal, thanks!

How about we do that on a separate thread (maybe with a reference back to this one, if needed for context)?
Basically, we would like these patch-set to move upstream independent of the GCD write-up.

I hope that's reasonable & agreeable by all.
Leo.
> 
> > -Jordan
> >
> >>
> >>> I guess with that resolved, you could add an Acked-by from me.
> >>>
> >>> In general, it'd also be nice to move the processor features to more
> >>> generic places, although that may be challenging if the next step is
> >>> some kind of platform hook from DXE Core. Maybe if the DXE Core
> >>> calls out to some protocol or signals an event then a driver in
> >>> UefiCpuPkg could handle the protocol implementation to modify the
> page tables.
> >>>
> >>> -Jordan
> >>>
> >>>
> > _______________________________________________
> > edk2-devel mailing list
> > edk2-devel@lists.01.org
> > https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 2017-07-06 14:49:44, Duran, Leo wrote:
> 
> > -----Original Message-----
> > From: afish@apple.com [mailto:afish@apple.com]
> > Sent: Thursday, July 06, 2017 4:46 PM
> > 
> > > On Jul 6, 2017, at 2:42 PM, Jordan Justen <jordan.l.justen@intel.com>
> > wrote:
> > >
> > > On 2017-07-06 13:11:03, Brijesh Singh wrote:
> > >>
> > >>
> > >> On 07/06/2017 11:45 AM, Jordan Justen wrote:
> > >
> > > I am asking that you file and own a bugzilla for this. You'd obviously
> > > need to work with the package owners though. Unless you drive this, I
> > > don't think anyone will be motivated enough to get it fixed.
> > >
> > 
> > If some one will make a write up on this mailing list summarizing the issue
> > with the GCD design, and what features are needed I can start a
> > conversation on the PI working group list.
> > 
> > Thanks,
> > 
> > Andrew Fish
> [Duran, Leo] 
> Excellent proposal, thanks!
> 
> How about we do that on a separate thread (maybe with a reference back to this one, if needed for context)?
> Basically, we would like these patch-set to move upstream independent of the GCD write-up.

This sounds reasonable, but can you file the bugzilla ticket? I think
it would be fine to add more details to the bugzilla later.

It sounds like it'd be good to note in the ticker that it may require
discussion in the PIWG.

If you document what is needed in the bugzilla, then maybe Andrew can
pull the information from there.

-Jordan
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Hi Jordan and Andrew,

On 07/07/2017 12:28 AM, Jordan Justen wrote:
> On 2017-07-06 14:49:44, Duran, Leo wrote:
>>
>>> -----Original Message-----
>>> From: afish@apple.com [mailto:afish@apple.com]
>>> Sent: Thursday, July 06, 2017 4:46 PM
>>>
>>>> On Jul 6, 2017, at 2:42 PM, Jordan Justen <jordan.l.justen@intel.com>
>>> wrote:
>>>>
>>>> On 2017-07-06 13:11:03, Brijesh Singh wrote:
>>>>>
>>>>>
>>>>> On 07/06/2017 11:45 AM, Jordan Justen wrote:
>>>>
>>>> I am asking that you file and own a bugzilla for this. You'd obviously
>>>> need to work with the package owners though. Unless you drive this, I
>>>> don't think anyone will be motivated enough to get it fixed.
>>>>
>>>
>>> If some one will make a write up on this mailing list summarizing the issue
>>> with the GCD design, and what features are needed I can start a
>>> conversation on the PI working group list.
>>>
>>> Thanks,
>>>
>>> Andrew Fish
>> [Duran, Leo]
>> Excellent proposal, thanks!
>>
>> How about we do that on a separate thread (maybe with a reference back to this one, if needed for context)?
>> Basically, we would like these patch-set to move upstream independent of the GCD write-up.
> 
> This sounds reasonable, but can you file the bugzilla ticket? I think
> it would be fine to add more details to the bugzilla later.
> 
> It sounds like it'd be good to note in the ticker that it may require
> discussion in the PIWG.
> 
> If you document what is needed in the bugzilla, then maybe Andrew can
> pull the information from there.
> 

I have created a bugzilla [1]. I have tried to give some background on what we are trying to do.
Lets take the GCD framework discussion on bugzilla (or separate thread).

[1] https://bugzilla.tianocore.org/show_bug.cgi?id=623


> -Jordan
> 
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 2017-07-07 11:29:20, Brijesh Singh wrote:
> I have created a bugzilla [1]. I have tried to give some background on what we are trying to do.
> Lets take the GCD framework discussion on bugzilla (or separate thread).
> 
> [1] https://bugzilla.tianocore.org/show_bug.cgi?id=623
> 

Thanks.

I'll take a final look over v8 and push them. (I'll add
Reviewed-by/Acked-by to more patches.) I expect I should finish within
the next day, or Monday at the latest.

-Jordan
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 2017-05-26 07:43:48, Brijesh Singh wrote:
> Changes since v4:
>  - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>    IoMmuDxe driver. And introduce a placeholder protocol to provide the
>    dependency support for the dependent modules.

I think you split IoMmuDxe out from AmdSevDxe based on my feedback
regarding APRIORI, but I don't think this helped.

Ideally I would like to see one driver named IoMmuDxe that is *not* in
APRIORI.

I think because of QemuFlashFvbServicesRuntimeDxe in APRIORI, we also
need IoMmuDxe in the APRIORI. Hopefully we can figure our how to
remove QemuFlashFvbServicesRuntimeDxe and then be able to remove
IoMmuDxe.

-Jordan

>  - update debug messages to use gEfiCallerBaseName where applicable.
>  - fix QemuFwCfgSecLib build errors and simplify SEV support
>  - update QemuFwCfgDxeLib to assert when failed to locate IOMMU
>  - update comments "host buffer" to " host buffer"
> 
> Changes since v3:
>  - update AmdSevDxe driver to produce IOMMU protocol
>  - remove BmDmaLib dependency
>  - update QemuFwCfgLib to use IOMMU protocol to allocate SEV DMA buffer
> 
> Changes since v2:
>  - move memory encryption CPUID and MSR definition into UefiCpuPkg
>  - fix the argument order for SUB instruction in ResetVector and add more
>    comments
>  - update PlatformPei to use BaseMemEncryptSevLib
>  - break the overlong comment lines to 79 chars
>  - variable aligment and other formating fixes
>  - split the SEV DMA support patch for QemuFwCfgLib into multiple patches as
>    recommended by Laszlo
>  - add AmdSevDxe driver which runs very early in DXE phase and clear the C-bit
>    from MMIO memory region
>  - drop 'QemuVideoDxe: Clear C-bit from framebuffer' patch since AmdSevDxe
>    driver takes care of clearing the C-bit from MMIO region
>  - Verified that Qemu PFLASH works fine with SEV guest, Found a KVM driver issue
>    which was causing #PF when PFLASH was enabled. I have submitted patch to
>    fix it in upstream http://marc.info/?l=kvm&m=149304930814202&w=2
> 
> Changes since v1:
>  - bug fixes in OvmfPkg/ResetVector (pointed by Tom Lendacky)
>  - add SEV CPUID and MSR register definition in standard include file
>  - remove the MemEncryptLib dependency from PlatformPei. Move AmdSevInitialize()
>    implementation in local file inside the PlatformPei package
>  - rename MemCryptSevLib to MemEncryptSevLib and add functions to set or
>    clear memory encryption attribute on memory region
>  - integerate SEV support in BmDmaLib
>  - split QemuFwCfgDxePei.c into QemuFwCfgDxe.c and QemuFwCfgPei.c to
>    allow building seperate QemuFwCfgLib for Dxe and Pei phase
>    (recommended by Laszlo Ersek)
>  - add SEV support in QemuFwCfgLib
>  - clear the memory encryption attribute from framebuffer memory region
> 
> 
> TODO:
> (Will add these features after basic SEV support patches are accepted in upstream)
>  - add support for DMA operation in QemuFwCfgS3Lib when SEV is enabled
>  - investigate SMM/SMI support
> 
> Cc: Jeff Fan <jeff.fan@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Leo Duran <leo.duran@amd.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leo Duran <leo.duran@amd.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <Thomas.Lendacky@amd.com>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
> 
> Brijesh Singh (17):
>   UefiCpuPkg: Define AMD Memory Encryption specific CPUID and MSR
>   OvmfPkg/ResetVector: Set C-bit when building initial page table
>   OvmfPkg: Update dsc to use IoLib from BaseIoLibIntrinsicSev.inf
>   OvmfPkg/BaseMemcryptSevLib: Add SEV helper library
>   OvmfPkg/PlatformPei: Set memory encryption PCD when SEV is enabled
>   OvmfPkg: Add AmdSevDxe driver
>   OvmfPkg: Introduce IoMmuAbsent Protocol GUID
>   OvmfPkg: Add PlatformHasIoMmuLib
>   OvmfPkg: Add IoMmuDxe driver
>   OvmfPkg/QemuFwCfgLib: Provide Pei and Dxe specific library
>   OvmfPkg/QemuFwCfgLib: Prepare for SEV support
>   OvmfPkg/QemuFwCfgLib: Implement SEV internal function for SEC phase
>   OvmfPkg/QemuFwCfgLib: Implement SEV internal functions for PEI phase
>   OvmfPkg/QemuFwCfgLib: Implement SEV internal function for Dxe phase
>   OvmfPkg/QemuFwCfgLib: Add option to dynamic alloc FW_CFG_DMA Access
>   OvmfPkg/QemuFwCfgLib: Add SEV support
>   OvmfPkg: update PciHostBridgeDxe to use PlatformHasIoMmuLib
> 
>  OvmfPkg/OvmfPkg.dec                                                    |   1 +
>  OvmfPkg/OvmfPkgIa32.dsc                                                |  11 +-
>  OvmfPkg/OvmfPkgIa32X64.dsc                                             |  12 +-
>  OvmfPkg/OvmfPkgX64.dsc                                                 |  12 +-
>  OvmfPkg/OvmfPkgIa32.fdf                                                |   1 +
>  OvmfPkg/OvmfPkgIa32X64.fdf                                             |   3 +
>  OvmfPkg/OvmfPkgX64.fdf                                                 |   3 +
>  OvmfPkg/AmdSevDxe/AmdSevDxe.inf                                        |  43 ++
>  OvmfPkg/IoMmuDxe/IoMmuDxe.inf                                          |  49 +++
>  OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf          |  50 +++
>  OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf            |  37 ++
>  OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} |  15 +-
>  OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} |   9 +-
>  OvmfPkg/PlatformPei/PlatformPei.inf                                    |   3 +
>  OvmfPkg/Include/Library/MemEncryptSevLib.h                             |  81 ++++
>  OvmfPkg/IoMmuDxe/AmdSevIoMmu.h                                         |  43 ++
>  OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h               | 184 ++++++++
>  OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLibInternal.h                    |  37 ++
>  OvmfPkg/PlatformPei/Platform.h                                         |   5 +
>  UefiCpuPkg/Include/Register/Amd/Cpuid.h                                | 162 +++++++
>  UefiCpuPkg/Include/Register/Amd/Fam17Msr.h                             |  62 +++
>  UefiCpuPkg/Include/Register/Amd/Msr.h                                  |  29 ++
>  OvmfPkg/AmdSevDxe/AmdSevDxe.c                                          |  75 ++++
>  OvmfPkg/IoMmuDxe/AmdSevIoMmu.c                                         | 459 ++++++++++++++++++++
>  OvmfPkg/IoMmuDxe/IoMmuDxe.c                                            |  53 +++
>  OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c           |  84 ++++
>  OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c        |  90 ++++
>  OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c            |  84 ++++
>  OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c               | 439 +++++++++++++++++++
>  OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c              |  32 ++
>  OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c                            | 230 ++++++++++
>  OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLib.c                            |  67 ++-
>  OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c}     |  72 ++-
>  OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgSec.c                            |  57 +++
>  OvmfPkg/PlatformPei/AmdSev.c                                           |  62 +++
>  OvmfPkg/PlatformPei/Platform.c                                         |   1 +
>  OvmfPkg/ResetVector/Ia32/PageTables64.asm                              |  70 ++-
>  37 files changed, 2703 insertions(+), 24 deletions(-)
>  create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.inf
>  create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.inf
>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf
>  create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf
>  copy OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} (71%)
>  rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} (80%)
>  create mode 100644 OvmfPkg/Include/Library/MemEncryptSevLib.h
>  create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.h
>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h
>  create mode 100644 UefiCpuPkg/Include/Register/Amd/Cpuid.h
>  create mode 100644 UefiCpuPkg/Include/Register/Amd/Fam17Msr.h
>  create mode 100644 UefiCpuPkg/Include/Register/Amd/Msr.h
>  create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.c
>  create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.c
>  create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.c
>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c
>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c
>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c
>  create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c
>  create mode 100644 OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c
>  rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c} (61%)
>  create mode 100644 OvmfPkg/PlatformPei/AmdSev.c
> 
> -- 
> 2.7.4
> 
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

(looks like I was the one to comment as second reviewer after all :) )

On 05/26/17 23:05, Jordan Justen wrote:
> On 2017-05-26 07:43:48, Brijesh Singh wrote:
>> Changes since v4:
>>  - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>>    IoMmuDxe driver. And introduce a placeholder protocol to provide the
>>    dependency support for the dependent modules.
> 
> I think you split IoMmuDxe out from AmdSevDxe based on my feedback
> regarding APRIORI, but I don't think this helped.
> 
> Ideally I would like to see one driver named IoMmuDxe that is *not* in
> APRIORI.

There are two separate goals here:

(1) Make sure that any driver that adds MMIO ranges will automatically
add those ranges with the C bit cleared in the PTEs, without actually
knowing about SEV.

(2) Provide an IOMMU protocol for those drivers that explicitly map /
unmap memory for DMA(-like) transfers.

Goal (2) is covered by the IOMMU protocol, and dependent drivers in
category (2) can be nicely ordered against IoMmuDxe with the OR depex (=
depend on the IOMMU protocol or the placeholder).

Goal (1) is a separate question. For covering goal (1), there are two
options:

(1a) modify all MMIO-adding drivers individually, to clear the C-bit.
(1b) do something general that will affect all MMIO additions at once,
without modifying individual drivers.

(1a) is ugly and it doesn't scale.

For (1b), there are again two possibilities:

(1b1) modify the GCD memory space map gDS services, so they handle SEV
internally,

(1b2) right after we enter the DXE phase (before any MMIO-adding driver
gets a chance to be dispatched), clear the C bit for all NonExistent
ranges (where MMIO might be added later) and for all currently existing
MMIO ranges (which come from the DXE core initialization, according to
MMIO memory descriptor HOBs from PEI).

I don't think we can express "right after we enter the DXE phase"
without an APRIORI entry; DEPEXes don't seem realistic for this.

Option (1b1) could still work, but as far as I remember (I could be
wrong!), Jiewen didn't like that, and he suggested (1b2) as the alternative.

> I think because of QemuFlashFvbServicesRuntimeDxe in APRIORI, we also
> need IoMmuDxe in the APRIORI.

QemuFlashFvbServicesRuntimeDxe adds the flash range as runtime,
uncacheable, system memory to the GCD memory space map (and then
allocates it at once). In practice it is used like MMIO, and therefore
the driver falls under category (1). Consequently, it should be
addressed with:
- either (1a) -- clear the C bit manually,
- or (1b1) -- modify the gDS services --> automatic C-bit clearing,
- or (1b2) -- assign the area with the C-bit pre-cleared.
- (or even (2) -- rework the driver to consume the IOMMU protocol
explicitly.)

Are you suggesting the last option -- rework
QemuFlashFvbServicesRuntimeDxe so that it consumes the IOMMU protocol?

In my opinion, that option is conceptually not much different from (1a)
-- which I don't like --, because it again implements a case-by-case
modification of an MMIO driver. Instead, I prefer SEV to be transparent
to drivers that don't already do *explicit* DMA.

> Hopefully we can figure our how to
> remove QemuFlashFvbServicesRuntimeDxe and then be able to remove
> IoMmuDxe.

The reason why QemuFlashFvbServicesRuntimeDxe is in APRIORI, and the
reason why AmdSevDxe is in APRIORI, are different.

* QemuFlashFvbServicesRuntimeDxe is in APRIORI -- and that's only when
SMM_REQUIRE is FALSE -- because it competes with
EmuVariableFvbRuntimeDxe, and QemuFlashFvbServicesRuntimeDxe takes
priority. EmuVariableFvbRuntimeDxe is only part of the build when
SMM_REQUIRE is false. When SMM_REQUIRE is TRUE, there is no competition,
and QemuFlashFvbServicesRuntimeDxe is not in APRIORI.

* AmdSevDxe is part of APRIORI because it must pre-clear the C bit for
*all* MMIO(-like) drivers.

If you move QemuFlashFvbServicesRuntimeDxe out of APRIORI (for example,
either by defining SMM_REQUIRE, or by inventing a different
serialization vehicle against EmuVariableFvbRuntimeDxe), AmdSevDxe
*still* has to remain in APRIORI, because it still must clear the C bit
for *all* MMIO(-like) drivers.

In my opinion, the current version of the patch set (v6) accurately
reflects option (1b2). And option (1b2) requires APRIORI usage.

If you dislike that, I think we'll have to go back to another option:
(1a), which I dislike; (1b1), which Jiewen dislikes (IIRC); or (2),
which I again dislike for all MMIO drivers that don't already use
explicit DMA.

Thanks
Laszlo

>>  - update debug messages to use gEfiCallerBaseName where applicable.
>>  - fix QemuFwCfgSecLib build errors and simplify SEV support
>>  - update QemuFwCfgDxeLib to assert when failed to locate IOMMU
>>  - update comments "host buffer" to " host buffer"
>>
>> Changes since v3:
>>  - update AmdSevDxe driver to produce IOMMU protocol
>>  - remove BmDmaLib dependency
>>  - update QemuFwCfgLib to use IOMMU protocol to allocate SEV DMA buffer
>>
>> Changes since v2:
>>  - move memory encryption CPUID and MSR definition into UefiCpuPkg
>>  - fix the argument order for SUB instruction in ResetVector and add more
>>    comments
>>  - update PlatformPei to use BaseMemEncryptSevLib
>>  - break the overlong comment lines to 79 chars
>>  - variable aligment and other formating fixes
>>  - split the SEV DMA support patch for QemuFwCfgLib into multiple patches as
>>    recommended by Laszlo
>>  - add AmdSevDxe driver which runs very early in DXE phase and clear the C-bit
>>    from MMIO memory region
>>  - drop 'QemuVideoDxe: Clear C-bit from framebuffer' patch since AmdSevDxe
>>    driver takes care of clearing the C-bit from MMIO region
>>  - Verified that Qemu PFLASH works fine with SEV guest, Found a KVM driver issue
>>    which was causing #PF when PFLASH was enabled. I have submitted patch to
>>    fix it in upstream http://marc.info/?l=kvm&m=149304930814202&w=2
>>
>> Changes since v1:
>>  - bug fixes in OvmfPkg/ResetVector (pointed by Tom Lendacky)
>>  - add SEV CPUID and MSR register definition in standard include file
>>  - remove the MemEncryptLib dependency from PlatformPei. Move AmdSevInitialize()
>>    implementation in local file inside the PlatformPei package
>>  - rename MemCryptSevLib to MemEncryptSevLib and add functions to set or
>>    clear memory encryption attribute on memory region
>>  - integerate SEV support in BmDmaLib
>>  - split QemuFwCfgDxePei.c into QemuFwCfgDxe.c and QemuFwCfgPei.c to
>>    allow building seperate QemuFwCfgLib for Dxe and Pei phase
>>    (recommended by Laszlo Ersek)
>>  - add SEV support in QemuFwCfgLib
>>  - clear the memory encryption attribute from framebuffer memory region
>>
>>
>> TODO:
>> (Will add these features after basic SEV support patches are accepted in upstream)
>>  - add support for DMA operation in QemuFwCfgS3Lib when SEV is enabled
>>  - investigate SMM/SMI support
>>
>> Cc: Jeff Fan <jeff.fan@intel.com>
>> Cc: Liming Gao <liming.gao@intel.com>
>> Cc: Leo Duran <leo.duran@amd.com>
>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Leo Duran <leo.duran@amd.com>
>> Cc: Jiewen Yao <jiewen.yao@intel.com>
>> Cc: Tom Lendacky <Thomas.Lendacky@amd.com>
>> Contributed-under: TianoCore Contribution Agreement 1.0
>> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
>>
>> Brijesh Singh (17):
>>   UefiCpuPkg: Define AMD Memory Encryption specific CPUID and MSR
>>   OvmfPkg/ResetVector: Set C-bit when building initial page table
>>   OvmfPkg: Update dsc to use IoLib from BaseIoLibIntrinsicSev.inf
>>   OvmfPkg/BaseMemcryptSevLib: Add SEV helper library
>>   OvmfPkg/PlatformPei: Set memory encryption PCD when SEV is enabled
>>   OvmfPkg: Add AmdSevDxe driver
>>   OvmfPkg: Introduce IoMmuAbsent Protocol GUID
>>   OvmfPkg: Add PlatformHasIoMmuLib
>>   OvmfPkg: Add IoMmuDxe driver
>>   OvmfPkg/QemuFwCfgLib: Provide Pei and Dxe specific library
>>   OvmfPkg/QemuFwCfgLib: Prepare for SEV support
>>   OvmfPkg/QemuFwCfgLib: Implement SEV internal function for SEC phase
>>   OvmfPkg/QemuFwCfgLib: Implement SEV internal functions for PEI phase
>>   OvmfPkg/QemuFwCfgLib: Implement SEV internal function for Dxe phase
>>   OvmfPkg/QemuFwCfgLib: Add option to dynamic alloc FW_CFG_DMA Access
>>   OvmfPkg/QemuFwCfgLib: Add SEV support
>>   OvmfPkg: update PciHostBridgeDxe to use PlatformHasIoMmuLib
>>
>>  OvmfPkg/OvmfPkg.dec                                                    |   1 +
>>  OvmfPkg/OvmfPkgIa32.dsc                                                |  11 +-
>>  OvmfPkg/OvmfPkgIa32X64.dsc                                             |  12 +-
>>  OvmfPkg/OvmfPkgX64.dsc                                                 |  12 +-
>>  OvmfPkg/OvmfPkgIa32.fdf                                                |   1 +
>>  OvmfPkg/OvmfPkgIa32X64.fdf                                             |   3 +
>>  OvmfPkg/OvmfPkgX64.fdf                                                 |   3 +
>>  OvmfPkg/AmdSevDxe/AmdSevDxe.inf                                        |  43 ++
>>  OvmfPkg/IoMmuDxe/IoMmuDxe.inf                                          |  49 +++
>>  OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf          |  50 +++
>>  OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf            |  37 ++
>>  OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} |  15 +-
>>  OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} |   9 +-
>>  OvmfPkg/PlatformPei/PlatformPei.inf                                    |   3 +
>>  OvmfPkg/Include/Library/MemEncryptSevLib.h                             |  81 ++++
>>  OvmfPkg/IoMmuDxe/AmdSevIoMmu.h                                         |  43 ++
>>  OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h               | 184 ++++++++
>>  OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLibInternal.h                    |  37 ++
>>  OvmfPkg/PlatformPei/Platform.h                                         |   5 +
>>  UefiCpuPkg/Include/Register/Amd/Cpuid.h                                | 162 +++++++
>>  UefiCpuPkg/Include/Register/Amd/Fam17Msr.h                             |  62 +++
>>  UefiCpuPkg/Include/Register/Amd/Msr.h                                  |  29 ++
>>  OvmfPkg/AmdSevDxe/AmdSevDxe.c                                          |  75 ++++
>>  OvmfPkg/IoMmuDxe/AmdSevIoMmu.c                                         | 459 ++++++++++++++++++++
>>  OvmfPkg/IoMmuDxe/IoMmuDxe.c                                            |  53 +++
>>  OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c           |  84 ++++
>>  OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c        |  90 ++++
>>  OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c            |  84 ++++
>>  OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c               | 439 +++++++++++++++++++
>>  OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c              |  32 ++
>>  OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c                            | 230 ++++++++++
>>  OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLib.c                            |  67 ++-
>>  OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c}     |  72 ++-
>>  OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgSec.c                            |  57 +++
>>  OvmfPkg/PlatformPei/AmdSev.c                                           |  62 +++
>>  OvmfPkg/PlatformPei/Platform.c                                         |   1 +
>>  OvmfPkg/ResetVector/Ia32/PageTables64.asm                              |  70 ++-
>>  37 files changed, 2703 insertions(+), 24 deletions(-)
>>  create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.inf
>>  create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.inf
>>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/BaseMemEncryptSevLib.inf
>>  create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf
>>  copy OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgDxeLib.inf} (71%)
>>  rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgLib.inf => QemuFwCfgPeiLib.inf} (80%)
>>  create mode 100644 OvmfPkg/Include/Library/MemEncryptSevLib.h
>>  create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.h
>>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h
>>  create mode 100644 UefiCpuPkg/Include/Register/Amd/Cpuid.h
>>  create mode 100644 UefiCpuPkg/Include/Register/Amd/Fam17Msr.h
>>  create mode 100644 UefiCpuPkg/Include/Register/Amd/Msr.h
>>  create mode 100644 OvmfPkg/AmdSevDxe/AmdSevDxe.c
>>  create mode 100644 OvmfPkg/IoMmuDxe/AmdSevIoMmu.c
>>  create mode 100644 OvmfPkg/IoMmuDxe/IoMmuDxe.c
>>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
>>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/MemEncryptSevLibInternal.c
>>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c
>>  create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.c
>>  create mode 100644 OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.c
>>  create mode 100644 OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c
>>  rename OvmfPkg/Library/QemuFwCfgLib/{QemuFwCfgPeiDxe.c => QemuFwCfgPei.c} (61%)
>>  create mode 100644 OvmfPkg/PlatformPei/AmdSev.c
>>
>> -- 
>> 2.7.4
>>

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 2017-05-29 04:16:15, Laszlo Ersek wrote:
> (looks like I was the one to comment as second reviewer after all :) )
> 
> On 05/26/17 23:05, Jordan Justen wrote:
> > On 2017-05-26 07:43:48, Brijesh Singh wrote:
> >> Changes since v4:
> >>  - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
> >>    IoMmuDxe driver. And introduce a placeholder protocol to provide the
> >>    dependency support for the dependent modules.
> > 
> > I think you split IoMmuDxe out from AmdSevDxe based on my feedback
> > regarding APRIORI, but I don't think this helped.
> > 
> > Ideally I would like to see one driver named IoMmuDxe that is *not* in
> > APRIORI.
> 
> There are two separate goals here:
> 
> (1) Make sure that any driver that adds MMIO ranges will automatically
> add those ranges with the C bit cleared in the PTEs, without actually
> knowing about SEV.

Ok, this sounds reasonable.

The APRIORI method looks like a hack. Why is this not being handled at
the time the page tables are being built, in DxeIpl? Couldn't we
define a platform Page Tables library to allow a platform to somehow
modify the page tables as they are built? Or, maybe just after? This
would also make sure it happens before DXE runs.

-Jordan
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 5/29/17 3:38 PM, Jordan Justen wrote:
> On 2017-05-29 04:16:15, Laszlo Ersek wrote:
>> (looks like I was the one to comment as second reviewer after all :) )
>>
>> On 05/26/17 23:05, Jordan Justen wrote:
>>> On 2017-05-26 07:43:48, Brijesh Singh wrote:
>>>> Changes since v4:
>>>>  - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>>>>    IoMmuDxe driver. And introduce a placeholder protocol to provide the
>>>>    dependency support for the dependent modules.
>>> I think you split IoMmuDxe out from AmdSevDxe based on my feedback
>>> regarding APRIORI, but I don't think this helped.
>>>
>>> Ideally I would like to see one driver named IoMmuDxe that is *not* in
>>> APRIORI.
>> There are two separate goals here:
>>
>> (1) Make sure that any driver that adds MMIO ranges will automatically
>> add those ranges with the C bit cleared in the PTEs, without actually
>> knowing about SEV.
> Ok, this sounds reasonable.
>
> The APRIORI method looks like a hack. Why is this not being handled at
> the time the page tables are being built, in DxeIpl? Couldn't we
> define a platform Page Tables library to allow a platform to somehow
> modify the page tables as they are built? Or, maybe just after? This
> would also make sure it happens before DXE runs.

Before introducing  AmdSevDxe driver, we did proposed patches to clear
the C-bit during the page table creation time. In the first patch [1],
Leo tried to  teach gcd.c to clear the C-bit from MMIO. IIRC, the main
concern was -- typically Dxecore does not do any CPU specific thing
hence we should try to find some alternative approach. In second patch
[2], Leo tried to introduce a new notify protocol to get MMIO add/remove
events. During discussion Jiewen suggested to look into adding a new
platform driver into APRIORI to avoid the need for any modifications
inside the Gcdcore - this seems workable solution which did not require
adding any CPU specific code inside the Gcd.

[1] https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
[2] https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 2017-05-29 14:59:46, Brijesh Singh wrote:
> 
> 
> On 5/29/17 3:38 PM, Jordan Justen wrote:
> > On 2017-05-29 04:16:15, Laszlo Ersek wrote:
> >> (looks like I was the one to comment as second reviewer after all :) )
> >>
> >> On 05/26/17 23:05, Jordan Justen wrote:
> >>> On 2017-05-26 07:43:48, Brijesh Singh wrote:
> >>>> Changes since v4:
> >>>>  - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
> >>>>    IoMmuDxe driver. And introduce a placeholder protocol to provide the
> >>>>    dependency support for the dependent modules.
> >>> I think you split IoMmuDxe out from AmdSevDxe based on my feedback
> >>> regarding APRIORI, but I don't think this helped.
> >>>
> >>> Ideally I would like to see one driver named IoMmuDxe that is *not* in
> >>> APRIORI.
> >> There are two separate goals here:
> >>
> >> (1) Make sure that any driver that adds MMIO ranges will automatically
> >> add those ranges with the C bit cleared in the PTEs, without actually
> >> knowing about SEV.
> > Ok, this sounds reasonable.
> >
> > The APRIORI method looks like a hack. Why is this not being handled at
> > the time the page tables are being built, in DxeIpl? Couldn't we
> > define a platform Page Tables library to allow a platform to somehow
> > modify the page tables as they are built? Or, maybe just after? This
> > would also make sure it happens before DXE runs.
> 
> Before introducing  AmdSevDxe driver, we did proposed patches to clear
> the C-bit during the page table creation time. In the first patch [1],
> Leo tried to  teach gcd.c to clear the C-bit from MMIO. IIRC, the main
> concern was -- typically Dxecore does not do any CPU specific thing
> hence we should try to find some alternative approach.

DxeCore doesn't build the page tables. DxeIpl builds them. I agree
that DxeCore is not the right place to handle this. In
https://lists.01.org/pipermail/edk2-devel/2017-March/008987.html
Jiewen suggested that DxeIpl could be updated during page table
creation time.

In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
Leo said that DxeIpl won't work because new I/O ranges might be added.
I don't understand this, because isn't DxeIpl and an early APRIORI
entry are roughly equivalent in the boot sequence?

-Jordan

> In second patch
> [2], Leo tried to introduce a new notify protocol to get MMIO add/remove
> events. During discussion Jiewen suggested to look into adding a new
> platform driver into APRIORI to avoid the need for any modifications
> inside the Gcdcore - this seems workable solution which did not require
> adding any CPU specific code inside the Gcd.
> 
> [1] https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
> [2] https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
> 
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 06/01/17 09:40, Jordan Justen wrote:
> On 2017-05-29 14:59:46, Brijesh Singh wrote:
>>
>>
>> On 5/29/17 3:38 PM, Jordan Justen wrote:
>>> On 2017-05-29 04:16:15, Laszlo Ersek wrote:
>>>> (looks like I was the one to comment as second reviewer after all :) )
>>>>
>>>> On 05/26/17 23:05, Jordan Justen wrote:
>>>>> On 2017-05-26 07:43:48, Brijesh Singh wrote:
>>>>>> Changes since v4:
>>>>>>  - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>>>>>>    IoMmuDxe driver. And introduce a placeholder protocol to provide the
>>>>>>    dependency support for the dependent modules.
>>>>> I think you split IoMmuDxe out from AmdSevDxe based on my feedback
>>>>> regarding APRIORI, but I don't think this helped.
>>>>>
>>>>> Ideally I would like to see one driver named IoMmuDxe that is *not* in
>>>>> APRIORI.
>>>> There are two separate goals here:
>>>>
>>>> (1) Make sure that any driver that adds MMIO ranges will automatically
>>>> add those ranges with the C bit cleared in the PTEs, without actually
>>>> knowing about SEV.
>>> Ok, this sounds reasonable.
>>>
>>> The APRIORI method looks like a hack. Why is this not being handled at
>>> the time the page tables are being built, in DxeIpl? Couldn't we
>>> define a platform Page Tables library to allow a platform to somehow
>>> modify the page tables as they are built? Or, maybe just after? This
>>> would also make sure it happens before DXE runs.
>>
>> Before introducing  AmdSevDxe driver, we did proposed patches to clear
>> the C-bit during the page table creation time. In the first patch [1],
>> Leo tried to  teach gcd.c to clear the C-bit from MMIO. IIRC, the main
>> concern was -- typically Dxecore does not do any CPU specific thing
>> hence we should try to find some alternative approach.
> 
> DxeCore doesn't build the page tables. DxeIpl builds them. I agree
> that DxeCore is not the right place to handle this. In
> https://lists.01.org/pipermail/edk2-devel/2017-March/008987.html
> Jiewen suggested that DxeIpl could be updated during page table
> creation time.
> 
> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
> Leo said that DxeIpl won't work because new I/O ranges might be added.
> I don't understand this, because isn't DxeIpl and an early APRIORI
> entry are roughly equivalent in the boot sequence?

I think you are right. I believe a patch for this exact idea hasn't been
posted yet. Jiewen's message that you linked above contains the expression

    always clear SEV mask for MMIO *and all rest*

(emphasis mine), which I think we may have missed *in combination with*
the DxeIpl.

So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
Keep the C bit set for system memory regions. Clear the C bit for MMIO
regions that are known from the HOB list. Also clear the C bit
everywhere else in the address space (known from the CPU HOB) where no
coverage is provided by any memory resource descriptor HOB.

This is going to be harder than the current approach, because:

- The current approach can work off of the GCD memory space map, which
provides explicit NonExistent entries, covering the entire address space
(according to the CPU HOB).

- However, the DxeIpl method would take place before entering DXE, so no
GCD memory space map would be available -- the "NonExistent" entries
would have to be synthesized manually from the address space size (known
from the CPU HOB) and the lack of coverage by memory resource descriptor
HOBs.

Basically, in order to move the current GCD memory space map traversal
from early DXE to late PEI, the memory space map building logic of the
DXE Core would have to be duplicated in the DxeIpl PEIM. If I understand
correctly. (The DxeIpl PEIM may already contain very similar code, for
the page table building, which might not be difficult to extend like
this -- I haven't looked.)

Is this what you have in mind?

Thanks
Laszlo

> -Jordan
> 
>> In second patch
>> [2], Leo tried to introduce a new notify protocol to get MMIO add/remove
>> events. During discussion Jiewen suggested to look into adding a new
>> platform driver into APRIORI to avoid the need for any modifications
>> inside the Gcdcore - this seems workable solution which did not require
>> adding any CPU specific code inside the Gcd.
>>
>> [1] https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>> [2] https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Laszlo,

The current design is DXE IPL and gEfiCpuArchProtocolGuid abstract the CPU specifics from the DXE Core. 

https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Dxe/Gcd/Gcd.c#L866
  if (Operation == GCD_SET_ATTRIBUTES_MEMORY_OPERATION) {
    //
    // Call CPU Arch Protocol to attempt to set attributes on the range
    //
    CpuArchAttributes = ConverToCpuArchAttributes (Attributes);
    if (CpuArchAttributes != INVALID_CPU_ARCH_ATTRIBUTES) {
      if (gCpu == NULL) {
        Status = EFI_NOT_AVAILABLE_YET;
      } else {
        Status = gCpu->SetMemoryAttributes (
                         gCpu,
                         BaseAddress,
                         Length,
                         CpuArchAttributes
                         );
      }
      if (EFI_ERROR (Status)) {
        CoreFreePool (TopEntry);
        CoreFreePool (BottomEntry);
        goto Done;
      }
    }
  }

Maybe the issue is there is an attempt to change attributes too early and they currently get sent to the bit bucket? I guess they could get queued up and replayed after gEfiCpuArchProtocolGuid is preset?

Thanks,

Andrew Fish


> On Jun 1, 2017, at 2:10 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> 
> On 06/01/17 09:40, Jordan Justen wrote:
>> On 2017-05-29 14:59:46, Brijesh Singh wrote:
>>> 
>>> 
>>> On 5/29/17 3:38 PM, Jordan Justen wrote:
>>>> On 2017-05-29 04:16:15, Laszlo Ersek wrote:
>>>>> (looks like I was the one to comment as second reviewer after all :) )
>>>>> 
>>>>> On 05/26/17 23:05, Jordan Justen wrote:
>>>>>> On 2017-05-26 07:43:48, Brijesh Singh wrote:
>>>>>>> Changes since v4:
>>>>>>> - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>>>>>>>   IoMmuDxe driver. And introduce a placeholder protocol to provide the
>>>>>>>   dependency support for the dependent modules.
>>>>>> I think you split IoMmuDxe out from AmdSevDxe based on my feedback
>>>>>> regarding APRIORI, but I don't think this helped.
>>>>>> 
>>>>>> Ideally I would like to see one driver named IoMmuDxe that is *not* in
>>>>>> APRIORI.
>>>>> There are two separate goals here:
>>>>> 
>>>>> (1) Make sure that any driver that adds MMIO ranges will automatically
>>>>> add those ranges with the C bit cleared in the PTEs, without actually
>>>>> knowing about SEV.
>>>> Ok, this sounds reasonable.
>>>> 
>>>> The APRIORI method looks like a hack. Why is this not being handled at
>>>> the time the page tables are being built, in DxeIpl? Couldn't we
>>>> define a platform Page Tables library to allow a platform to somehow
>>>> modify the page tables as they are built? Or, maybe just after? This
>>>> would also make sure it happens before DXE runs.
>>> 
>>> Before introducing  AmdSevDxe driver, we did proposed patches to clear
>>> the C-bit during the page table creation time. In the first patch [1],
>>> Leo tried to  teach gcd.c to clear the C-bit from MMIO. IIRC, the main
>>> concern was -- typically Dxecore does not do any CPU specific thing
>>> hence we should try to find some alternative approach.
>> 
>> DxeCore doesn't build the page tables. DxeIpl builds them. I agree
>> that DxeCore is not the right place to handle this. In
>> https://lists.01.org/pipermail/edk2-devel/2017-March/008987.html
>> Jiewen suggested that DxeIpl could be updated during page table
>> creation time.
>> 
>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>> I don't understand this, because isn't DxeIpl and an early APRIORI
>> entry are roughly equivalent in the boot sequence?
> 
> I think you are right. I believe a patch for this exact idea hasn't been
> posted yet. Jiewen's message that you linked above contains the expression
> 
>    always clear SEV mask for MMIO *and all rest*
> 
> (emphasis mine), which I think we may have missed *in combination with*
> the DxeIpl.
> 
> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
> Keep the C bit set for system memory regions. Clear the C bit for MMIO
> regions that are known from the HOB list. Also clear the C bit
> everywhere else in the address space (known from the CPU HOB) where no
> coverage is provided by any memory resource descriptor HOB.
> 
> This is going to be harder than the current approach, because:
> 
> - The current approach can work off of the GCD memory space map, which
> provides explicit NonExistent entries, covering the entire address space
> (according to the CPU HOB).
> 
> - However, the DxeIpl method would take place before entering DXE, so no
> GCD memory space map would be available -- the "NonExistent" entries
> would have to be synthesized manually from the address space size (known
> from the CPU HOB) and the lack of coverage by memory resource descriptor
> HOBs.
> 
> Basically, in order to move the current GCD memory space map traversal
> from early DXE to late PEI, the memory space map building logic of the
> DXE Core would have to be duplicated in the DxeIpl PEIM. If I understand
> correctly. (The DxeIpl PEIM may already contain very similar code, for
> the page table building, which might not be difficult to extend like
> this -- I haven't looked.)
> 
> Is this what you have in mind?
> 
> Thanks
> Laszlo
> 
>> -Jordan
>> 
>>> In second patch
>>> [2], Leo tried to introduce a new notify protocol to get MMIO add/remove
>>> events. During discussion Jiewen suggested to look into adding a new
>>> platform driver into APRIORI to avoid the need for any modifications
>>> inside the Gcdcore - this seems workable solution which did not require
>>> adding any CPU specific code inside the Gcd.
>>> 
>>> [1] https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>> [2] https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>> 
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 06/01/17 15:48, Andrew Fish wrote:
> Laszlo,
> 
> The current design is DXE IPL and gEfiCpuArchProtocolGuid abstract the CPU specifics from the DXE Core. 
> 
> https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Dxe/Gcd/Gcd.c#L866
>   if (Operation == GCD_SET_ATTRIBUTES_MEMORY_OPERATION) {
>     //
>     // Call CPU Arch Protocol to attempt to set attributes on the range
>     //
>     CpuArchAttributes = ConverToCpuArchAttributes (Attributes);
>     if (CpuArchAttributes != INVALID_CPU_ARCH_ATTRIBUTES) {
>       if (gCpu == NULL) {
>         Status = EFI_NOT_AVAILABLE_YET;
>       } else {
>         Status = gCpu->SetMemoryAttributes (
>                          gCpu,
>                          BaseAddress,
>                          Length,
>                          CpuArchAttributes
>                          );
>       }
>       if (EFI_ERROR (Status)) {
>         CoreFreePool (TopEntry);
>         CoreFreePool (BottomEntry);
>         goto Done;
>       }
>     }
>   }
> 
> Maybe the issue is there is an attempt to change attributes too early
> and they currently get sent to the bit bucket? I guess they could get
> queued up and replayed after gEfiCpuArchProtocolGuid is preset?
The problem we are facing is not a technical one, the method implemented
in this series works. AIUI we're looking for the best component and best
phase for clearing the C bit for MMIO ranges that are either installed
by the PEI phase (via HOBs) or by gDS->AddMemorySpace() calls in DXE.

One idea was to incorporate the C-bit's management, for both kinds of
MMIO additions, into the DXE Core. (The DXE Core precedes all DXE
drivers, and it processes the memory descriptor HOBs anyway, for
initializing the GCD memory space map.)

If we can push down the C-bit's management to the CPU Arch protocol
implementation, because that's what the DXE Core calls out to, upon
memory space addition, that could likely take care of the
gDS->AddMemorySpace() calls.

Not sure how the other category would be handled via the CPU Arch
protocol though, as a DXE driver directly accessing an MMIO range that
was described in PEI with a HOB could be dispatched before the CPU Arch
protocol becomes available. So acting upon those MMIO HOBs only after
the CPU Arch proto is up could be too late.

Thanks
Laszlo


> 
>> On Jun 1, 2017, at 2:10 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>>
>> On 06/01/17 09:40, Jordan Justen wrote:
>>> On 2017-05-29 14:59:46, Brijesh Singh wrote:
>>>>
>>>>
>>>> On 5/29/17 3:38 PM, Jordan Justen wrote:
>>>>> On 2017-05-29 04:16:15, Laszlo Ersek wrote:
>>>>>> (looks like I was the one to comment as second reviewer after all :) )
>>>>>>
>>>>>> On 05/26/17 23:05, Jordan Justen wrote:
>>>>>>> On 2017-05-26 07:43:48, Brijesh Singh wrote:
>>>>>>>> Changes since v4:
>>>>>>>> - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>>>>>>>>   IoMmuDxe driver. And introduce a placeholder protocol to provide the
>>>>>>>>   dependency support for the dependent modules.
>>>>>>> I think you split IoMmuDxe out from AmdSevDxe based on my feedback
>>>>>>> regarding APRIORI, but I don't think this helped.
>>>>>>>
>>>>>>> Ideally I would like to see one driver named IoMmuDxe that is *not* in
>>>>>>> APRIORI.
>>>>>> There are two separate goals here:
>>>>>>
>>>>>> (1) Make sure that any driver that adds MMIO ranges will automatically
>>>>>> add those ranges with the C bit cleared in the PTEs, without actually
>>>>>> knowing about SEV.
>>>>> Ok, this sounds reasonable.
>>>>>
>>>>> The APRIORI method looks like a hack. Why is this not being handled at
>>>>> the time the page tables are being built, in DxeIpl? Couldn't we
>>>>> define a platform Page Tables library to allow a platform to somehow
>>>>> modify the page tables as they are built? Or, maybe just after? This
>>>>> would also make sure it happens before DXE runs.
>>>>
>>>> Before introducing  AmdSevDxe driver, we did proposed patches to clear
>>>> the C-bit during the page table creation time. In the first patch [1],
>>>> Leo tried to  teach gcd.c to clear the C-bit from MMIO. IIRC, the main
>>>> concern was -- typically Dxecore does not do any CPU specific thing
>>>> hence we should try to find some alternative approach.
>>>
>>> DxeCore doesn't build the page tables. DxeIpl builds them. I agree
>>> that DxeCore is not the right place to handle this. In
>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008987.html
>>> Jiewen suggested that DxeIpl could be updated during page table
>>> creation time.
>>>
>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>> I don't understand this, because isn't DxeIpl and an early APRIORI
>>> entry are roughly equivalent in the boot sequence?
>>
>> I think you are right. I believe a patch for this exact idea hasn't been
>> posted yet. Jiewen's message that you linked above contains the expression
>>
>>    always clear SEV mask for MMIO *and all rest*
>>
>> (emphasis mine), which I think we may have missed *in combination with*
>> the DxeIpl.
>>
>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>> Keep the C bit set for system memory regions. Clear the C bit for MMIO
>> regions that are known from the HOB list. Also clear the C bit
>> everywhere else in the address space (known from the CPU HOB) where no
>> coverage is provided by any memory resource descriptor HOB.
>>
>> This is going to be harder than the current approach, because:
>>
>> - The current approach can work off of the GCD memory space map, which
>> provides explicit NonExistent entries, covering the entire address space
>> (according to the CPU HOB).
>>
>> - However, the DxeIpl method would take place before entering DXE, so no
>> GCD memory space map would be available -- the "NonExistent" entries
>> would have to be synthesized manually from the address space size (known
>> from the CPU HOB) and the lack of coverage by memory resource descriptor
>> HOBs.
>>
>> Basically, in order to move the current GCD memory space map traversal
>> from early DXE to late PEI, the memory space map building logic of the
>> DXE Core would have to be duplicated in the DxeIpl PEIM. If I understand
>> correctly. (The DxeIpl PEIM may already contain very similar code, for
>> the page table building, which might not be difficult to extend like
>> this -- I haven't looked.)
>>
>> Is this what you have in mind?
>>
>> Thanks
>> Laszlo
>>
>>> -Jordan
>>>
>>>> In second patch
>>>> [2], Leo tried to introduce a new notify protocol to get MMIO add/remove
>>>> events. During discussion Jiewen suggested to look into adding a new
>>>> platform driver into APRIORI to avoid the need for any modifications
>>>> inside the Gcdcore - this seems workable solution which did not require
>>>> adding any CPU specific code inside the Gcd.
>>>>
>>>> [1] https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>> [2] https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>
>>
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org
>> https://lists.01.org/mailman/listinfo/edk2-devel
> 
> 

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Hi Andrew,

The goal is to clear the "C" bit in PTE for all the MMIO areas in the GCD memory
space map. I think Leo looked at SetMemoryAttributes() based on Mike's feedback,
but I believe SetMemoryAttribute may get called on any range without specifying
types (we are interested in MMIO ranges, which are specified in ADD/REMOVE operations).

-Brijesh

On 06/01/2017 08:48 AM, Andrew Fish wrote:
> Laszlo,
> 
> The current design is DXE IPL and gEfiCpuArchProtocolGuid abstract the CPU specifics from the DXE Core.
> 
> https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Dxe/Gcd/Gcd.c#L866
>    if (Operation == GCD_SET_ATTRIBUTES_MEMORY_OPERATION) {
>      //
>      // Call CPU Arch Protocol to attempt to set attributes on the range
>      //
>      CpuArchAttributes = ConverToCpuArchAttributes (Attributes);
>      if (CpuArchAttributes != INVALID_CPU_ARCH_ATTRIBUTES) {
>        if (gCpu == NULL) {
>          Status = EFI_NOT_AVAILABLE_YET;
>        } else {
>          Status = gCpu->SetMemoryAttributes (
>                           gCpu,
>                           BaseAddress,
>                           Length,
>                           CpuArchAttributes
>                           );
>        }
>        if (EFI_ERROR (Status)) {
>          CoreFreePool (TopEntry);
>          CoreFreePool (BottomEntry);
>          goto Done;
>        }
>      }
>    }
> 
> Maybe the issue is there is an attempt to change attributes too early and they currently get sent to the bit bucket? I guess they could get queued up and replayed after gEfiCpuArchProtocolGuid is preset?
> 
> Thanks,
> 
> Andrew Fish
> 
> 
>> On Jun 1, 2017, at 2:10 AM, Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>> wrote:
>>
>> On 06/01/17 09:40, Jordan Justen wrote:
>>> On 2017-05-29 14:59:46, Brijesh Singh wrote:
>>>>
>>>>
>>>> On 5/29/17 3:38 PM, Jordan Justen wrote:
>>>>> On 2017-05-29 04:16:15, Laszlo Ersek wrote:
>>>>>> (looks like I was the one to comment as second reviewer after all :) )
>>>>>>
>>>>>> On 05/26/17 23:05, Jordan Justen wrote:
>>>>>>> On 2017-05-26 07:43:48, Brijesh Singh wrote:
>>>>>>>> Changes since v4:
>>>>>>>> - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>>>>>>>>   IoMmuDxe driver. And introduce a placeholder protocol to provide the
>>>>>>>>   dependency support for the dependent modules.
>>>>>>> I think you split IoMmuDxe out from AmdSevDxe based on my feedback
>>>>>>> regarding APRIORI, but I don't think this helped.
>>>>>>>
>>>>>>> Ideally I would like to see one driver named IoMmuDxe that is *not* in
>>>>>>> APRIORI.
>>>>>> There are two separate goals here:
>>>>>>
>>>>>> (1) Make sure that any driver that adds MMIO ranges will automatically
>>>>>> add those ranges with the C bit cleared in the PTEs, without actually
>>>>>> knowing about SEV.
>>>>> Ok, this sounds reasonable.
>>>>>
>>>>> The APRIORI method looks like a hack. Why is this not being handled at
>>>>> the time the page tables are being built, in DxeIpl? Couldn't we
>>>>> define a platform Page Tables library to allow a platform to somehow
>>>>> modify the page tables as they are built? Or, maybe just after? This
>>>>> would also make sure it happens before DXE runs.
>>>>
>>>> Before introducing  AmdSevDxe driver, we did proposed patches to clear
>>>> the C-bit during the page table creation time. In the first patch [1],
>>>> Leo tried to  teach gcd.c to clear the C-bit from MMIO. IIRC, the main
>>>> concern was -- typically Dxecore does not do any CPU specific thing
>>>> hence we should try to find some alternative approach.
>>>
>>> DxeCore doesn't build the page tables. DxeIpl builds them. I agree
>>> that DxeCore is not the right place to handle this. In
>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008987.html
>>> Jiewen suggested that DxeIpl could be updated during page table
>>> creation time.
>>>
>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>> I don't understand this, because isn't DxeIpl and an early APRIORI
>>> entry are roughly equivalent in the boot sequence?
>>
>> I think you are right. I believe a patch for this exact idea hasn't been
>> posted yet. Jiewen's message that you linked above contains the expression
>>
>>    always clear SEV mask for MMIO *and all rest*
>>
>> (emphasis mine), which I think we may have missed *in combination with*
>> the DxeIpl.
>>
>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>> Keep the C bit set for system memory regions. Clear the C bit for MMIO
>> regions that are known from the HOB list. Also clear the C bit
>> everywhere else in the address space (known from the CPU HOB) where no
>> coverage is provided by any memory resource descriptor HOB.
>>
>> This is going to be harder than the current approach, because:
>>
>> - The current approach can work off of the GCD memory space map, which
>> provides explicit NonExistent entries, covering the entire address space
>> (according to the CPU HOB).
>>
>> - However, the DxeIpl method would take place before entering DXE, so no
>> GCD memory space map would be available -- the "NonExistent" entries
>> would have to be synthesized manually from the address space size (known
>> from the CPU HOB) and the lack of coverage by memory resource descriptor
>> HOBs.
>>
>> Basically, in order to move the current GCD memory space map traversal
>> from early DXE to late PEI, the memory space map building logic of the
>> DXE Core would have to be duplicated in the DxeIpl PEIM. If I understand
>> correctly. (The DxeIpl PEIM may already contain very similar code, for
>> the page table building, which might not be difficult to extend like
>> this -- I haven't looked.)
>>
>> Is this what you have in mind?
>>
>> Thanks
>> Laszlo
>>
>>> -Jordan
>>>
>>>> In second patch
>>>> [2], Leo tried to introduce a new notify protocol to get MMIO add/remove
>>>> events. During discussion Jiewen suggested to look into adding a new
>>>> platform driver into APRIORI to avoid the need for any modifications
>>>> inside the Gcdcore - this seems workable solution which did not require
>>>> adding any CPU specific code inside the Gcd.
>>>>
>>>> [1] https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>> [2] https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>
>>
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>
>> https://lists.01.org/mailman/listinfo/edk2-devel
> 
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

> On Jun 1, 2017, at 8:01 AM, Brijesh Singh <brijesh.singh@amd.com> wrote:
> 
> Hi Andrew,
> 
> The goal is to clear the "C" bit in PTE for all the MMIO areas in the GCD memory
> space map. I think Leo looked at SetMemoryAttributes() based on Mike's feedback,
> but I believe SetMemoryAttribute may get called on any range without specifying
> types (we are interested in MMIO ranges, which are specified in ADD/REMOVE operations).
> 

Brijesh,

Sorry pre coffee...

I gest I'm trying to ask more generic questions.

1) Should the DXE Core queue GCD_SET_ATTRIBUTES_MEMORY_OPERATIONs so you can update things from PEI, or not require a priori dispatch? 
2) Is there an issue in the GCD implementation or architecture. 

For example I recently ran into an issue use EfiGcdMemoryTypeMemoryMappedIo as it was causing massive amounts of virtual mapping requests to the OS. It looks like by default EfiGcdMemoryTypeMemoryMappedIo sets the EFI_MEMORY_RUNTIME Capability. I'm not exactly sure that is what the PI Spec requires?  It makes sense to me that from EFI MemoryMappedIo things require the EFI_MEMORY_RUNTIME, but for GCD seems like you might have massive ranges of MMIO that need to get mapped, and those regions get owned by the OS at runtime? 

Anyway not trying to derail your solution, or block progress. I just want to make sure we have the conversation as to why this ended up being so hard to make sure we don't have an implementation or spec issue around things GCD. 

PI Spec on EfiGcdMemoryTypeMemoryMappedIo

The GetMemoryMap() implementation must include into the UEFI memory map all GCD map entries of types EfiGcdMemoryTypeReserved and EfiPersistentMemory, and all GCD map entries of type EfiGcdMemoryTypeMemoryMappedIo that have EFI_MEMORY_RUNTIME attribute set.

vs.

The GetMemoryMap() implementation must include all GCD map entries of types EfiGcdMemoryTypeReserved and EfiGcdMemoryTypeMemoryMappedIo into the UEFI memory map.

Thanks,

Andrew Fish

> -Brijesh
> 
> On 06/01/2017 08:48 AM, Andrew Fish wrote:
>> Laszlo,
>> The current design is DXE IPL and gEfiCpuArchProtocolGuid abstract the CPU specifics from the DXE Core.
>> https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Dxe/Gcd/Gcd.c#L866
>>   if (Operation == GCD_SET_ATTRIBUTES_MEMORY_OPERATION) {
>>     //
>>     // Call CPU Arch Protocol to attempt to set attributes on the range
>>     //
>>     CpuArchAttributes = ConverToCpuArchAttributes (Attributes);
>>     if (CpuArchAttributes != INVALID_CPU_ARCH_ATTRIBUTES) {
>>       if (gCpu == NULL) {
>>         Status = EFI_NOT_AVAILABLE_YET;
>>       } else {
>>         Status = gCpu->SetMemoryAttributes (
>>                          gCpu,
>>                          BaseAddress,
>>                          Length,
>>                          CpuArchAttributes
>>                          );
>>       }
>>       if (EFI_ERROR (Status)) {
>>         CoreFreePool (TopEntry);
>>         CoreFreePool (BottomEntry);
>>         goto Done;
>>       }
>>     }
>>   }
>> Maybe the issue is there is an attempt to change attributes too early and they currently get sent to the bit bucket? I guess they could get queued up and replayed after gEfiCpuArchProtocolGuid is preset?
>> Thanks,
>> Andrew Fish
>>> On Jun 1, 2017, at 2:10 AM, Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com> <mailto:lersek@redhat.com <mailto:lersek@redhat.com>>> wrote:
>>> 
>>> On 06/01/17 09:40, Jordan Justen wrote:
>>>> On 2017-05-29 14:59:46, Brijesh Singh wrote:
>>>>> 
>>>>> 
>>>>> On 5/29/17 3:38 PM, Jordan Justen wrote:
>>>>>> On 2017-05-29 04:16:15, Laszlo Ersek wrote:
>>>>>>> (looks like I was the one to comment as second reviewer after all :) )
>>>>>>> 
>>>>>>> On 05/26/17 23:05, Jordan Justen wrote:
>>>>>>>> On 2017-05-26 07:43:48, Brijesh Singh wrote:
>>>>>>>>> Changes since v4:
>>>>>>>>> - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>>>>>>>>>  IoMmuDxe driver. And introduce a placeholder protocol to provide the
>>>>>>>>>  dependency support for the dependent modules.
>>>>>>>> I think you split IoMmuDxe out from AmdSevDxe based on my feedback
>>>>>>>> regarding APRIORI, but I don't think this helped.
>>>>>>>> 
>>>>>>>> Ideally I would like to see one driver named IoMmuDxe that is *not* in
>>>>>>>> APRIORI.
>>>>>>> There are two separate goals here:
>>>>>>> 
>>>>>>> (1) Make sure that any driver that adds MMIO ranges will automatically
>>>>>>> add those ranges with the C bit cleared in the PTEs, without actually
>>>>>>> knowing about SEV.
>>>>>> Ok, this sounds reasonable.
>>>>>> 
>>>>>> The APRIORI method looks like a hack. Why is this not being handled at
>>>>>> the time the page tables are being built, in DxeIpl? Couldn't we
>>>>>> define a platform Page Tables library to allow a platform to somehow
>>>>>> modify the page tables as they are built? Or, maybe just after? This
>>>>>> would also make sure it happens before DXE runs.
>>>>> 
>>>>> Before introducing  AmdSevDxe driver, we did proposed patches to clear
>>>>> the C-bit during the page table creation time. In the first patch [1],
>>>>> Leo tried to  teach gcd.c to clear the C-bit from MMIO. IIRC, the main
>>>>> concern was -- typically Dxecore does not do any CPU specific thing
>>>>> hence we should try to find some alternative approach.
>>>> 
>>>> DxeCore doesn't build the page tables. DxeIpl builds them. I agree
>>>> that DxeCore is not the right place to handle this. In
>>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008987.html
>>>> Jiewen suggested that DxeIpl could be updated during page table
>>>> creation time.
>>>> 
>>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>>> I don't understand this, because isn't DxeIpl and an early APRIORI
>>>> entry are roughly equivalent in the boot sequence?
>>> 
>>> I think you are right. I believe a patch for this exact idea hasn't been
>>> posted yet. Jiewen's message that you linked above contains the expression
>>> 
>>>   always clear SEV mask for MMIO *and all rest*
>>> 
>>> (emphasis mine), which I think we may have missed *in combination with*
>>> the DxeIpl.
>>> 
>>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>>> Keep the C bit set for system memory regions. Clear the C bit for MMIO
>>> regions that are known from the HOB list. Also clear the C bit
>>> everywhere else in the address space (known from the CPU HOB) where no
>>> coverage is provided by any memory resource descriptor HOB.
>>> 
>>> This is going to be harder than the current approach, because:
>>> 
>>> - The current approach can work off of the GCD memory space map, which
>>> provides explicit NonExistent entries, covering the entire address space
>>> (according to the CPU HOB).
>>> 
>>> - However, the DxeIpl method would take place before entering DXE, so no
>>> GCD memory space map would be available -- the "NonExistent" entries
>>> would have to be synthesized manually from the address space size (known
>>> from the CPU HOB) and the lack of coverage by memory resource descriptor
>>> HOBs.
>>> 
>>> Basically, in order to move the current GCD memory space map traversal
>>> from early DXE to late PEI, the memory space map building logic of the
>>> DXE Core would have to be duplicated in the DxeIpl PEIM. If I understand
>>> correctly. (The DxeIpl PEIM may already contain very similar code, for
>>> the page table building, which might not be difficult to extend like
>>> this -- I haven't looked.)
>>> 
>>> Is this what you have in mind?
>>> 
>>> Thanks
>>> Laszlo
>>> 
>>>> -Jordan
>>>> 
>>>>> In second patch
>>>>> [2], Leo tried to introduce a new notify protocol to get MMIO add/remove
>>>>> events. During discussion Jiewen suggested to look into adding a new
>>>>> platform driver into APRIORI to avoid the need for any modifications
>>>>> inside the Gcdcore - this seems workable solution which did not require
>>>>> adding any CPU specific code inside the Gcd.
>>>>> 
>>>>> [1] https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>>> [2] https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>> 
>>> 
>>> _______________________________________________
>>> edk2-devel mailing list
>>> edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> <mailto:edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>>
>>> https://lists.01.org/mailman/listinfo/edk2-devel <https://lists.01.org/mailman/listinfo/edk2-devel>
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>
> https://lists.01.org/mailman/listinfo/edk2-devel <https://lists.01.org/mailman/listinfo/edk2-devel>
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Hi Jordan,


On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
> On 06/01/17 09:40, Jordan Justen wrote:
>> On 2017-05-29 14:59:46, Brijesh Singh wrote:
>>>
>>>
>>> On 5/29/17 3:38 PM, Jordan Justen wrote:
>>>> On 2017-05-29 04:16:15, Laszlo Ersek wrote:
>>>>> (looks like I was the one to comment as second reviewer after all :) )
>>>>>
>>>>> On 05/26/17 23:05, Jordan Justen wrote:
>>>>>> On 2017-05-26 07:43:48, Brijesh Singh wrote:
>>>>>>> Changes since v4:
>>>>>>>   - decouple IoMmu protocol implementation from AmdSevDxe into a seperate
>>>>>>>     IoMmuDxe driver. And introduce a placeholder protocol to provide the
>>>>>>>     dependency support for the dependent modules.
>>>>>> I think you split IoMmuDxe out from AmdSevDxe based on my feedback
>>>>>> regarding APRIORI, but I don't think this helped.
>>>>>>
>>>>>> Ideally I would like to see one driver named IoMmuDxe that is *not* in
>>>>>> APRIORI.
>>>>> There are two separate goals here:
>>>>>
>>>>> (1) Make sure that any driver that adds MMIO ranges will automatically
>>>>> add those ranges with the C bit cleared in the PTEs, without actually
>>>>> knowing about SEV.
>>>> Ok, this sounds reasonable.
>>>>
>>>> The APRIORI method looks like a hack. Why is this not being handled at
>>>> the time the page tables are being built, in DxeIpl? Couldn't we
>>>> define a platform Page Tables library to allow a platform to somehow
>>>> modify the page tables as they are built? Or, maybe just after? This
>>>> would also make sure it happens before DXE runs.
>>>
>>> Before introducing  AmdSevDxe driver, we did proposed patches to clear
>>> the C-bit during the page table creation time. In the first patch [1],
>>> Leo tried to  teach gcd.c to clear the C-bit from MMIO. IIRC, the main
>>> concern was -- typically Dxecore does not do any CPU specific thing
>>> hence we should try to find some alternative approach.
>>
>> DxeCore doesn't build the page tables. DxeIpl builds them. I agree
>> that DxeCore is not the right place to handle this. In
>> https://lists.01.org/pipermail/edk2-devel/2017-March/008987.html
>> Jiewen suggested that DxeIpl could be updated during page table
>> creation time.
>>
>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>> I don't understand this, because isn't DxeIpl and an early APRIORI
>> entry are roughly equivalent in the boot sequence?
> 
> I think you are right. I believe a patch for this exact idea hasn't been
> posted yet. Jiewen's message that you linked above contains the expression
> 
>      always clear SEV mask for MMIO *and all rest*
> 
> (emphasis mine), which I think we may have missed *in combination with*
> the DxeIpl.
> 
> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
> Keep the C bit set for system memory regions. Clear the C bit for MMIO
> regions that are known from the HOB list. Also clear the C bit
> everywhere else in the address space (known from the CPU HOB) where no
> coverage is provided by any memory resource descriptor HOB.
> 
> This is going to be harder than the current approach, because:
> 
> - The current approach can work off of the GCD memory space map, which
> provides explicit NonExistent entries, covering the entire address space
> (according to the CPU HOB).
> 
> - However, the DxeIpl method would take place before entering DXE, so no
> GCD memory space map would be available -- the "NonExistent" entries
> would have to be synthesized manually from the address space size (known
> from the CPU HOB) and the lack of coverage by memory resource descriptor
> HOBs.
> 
> Basically, in order to move the current GCD memory space map traversal
> from early DXE to late PEI, the memory space map building logic of the
> DXE Core would have to be duplicated in the DxeIpl PEIM. If I understand
> correctly. (The DxeIpl PEIM may already contain very similar code, for
> the page table building, which might not be difficult to extend like
> this -- I haven't looked.)
> 
> Is this what you have in mind?
> 

Do you have any further thought on this?

In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to see
if I can invoke a platform dependent library to clear C-bit before DxeMain
finishes its execution. As Laszlo pointed, current approach is using GCD memory
space map to get MMIO and NonExistent entries. I have pushed two patches
in my development branch to show what I have been doing:

1) add a new null DxeGcdCorePlatformHookLib

https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94a920d7ad72d

The library provides a function "DxeGcdCorePlatformHookReady" which can be called
by DxeMain just after it initializes the GcdServices (which will guarantee that
Gcd memory space map is available).

2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
  SEV is detected.

https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba53c95949f659

I've tested the approach and it seems to work. Is this something aligned with your
thinking?


> Thanks
> Laszlo
> 
>> -Jordan
>>
>>> In second patch
>>> [2], Leo tried to introduce a new notify protocol to get MMIO add/remove
>>> events. During discussion Jiewen suggested to look into adding a new
>>> platform driver into APRIORI to avoid the need for any modifications
>>> inside the Gcdcore - this seems workable solution which did not require
>>> adding any CPU specific code inside the Gcd.
>>>
>>> [1] https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>> [2] https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>
> 
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 2017-06-05 14:56:04, Brijesh Singh wrote:
> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
> > On 06/01/17 09:40, Jordan Justen wrote:
> >> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
> >> Leo said that DxeIpl won't work because new I/O ranges might be added.
> >> I don't understand this, because isn't DxeIpl and an early APRIORI
> >> entry are roughly equivalent in the boot sequence?
> > 
> > I think you are right. I believe a patch for this exact idea hasn't been
> > posted yet. Jiewen's message that you linked above contains the expression
> > 
> >      always clear SEV mask for MMIO *and all rest*
> > 
> > (emphasis mine), which I think we may have missed *in combination with*
> > the DxeIpl.
> > 
> > So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
> > Keep the C bit set for system memory regions. Clear the C bit for MMIO
> > regions that are known from the HOB list. Also clear the C bit
> > everywhere else in the address space (known from the CPU HOB) where no
> > coverage is provided by any memory resource descriptor HOB.
> > 
> > This is going to be harder than the current approach, because:
> > 
> > - The current approach can work off of the GCD memory space map, which
> > provides explicit NonExistent entries, covering the entire address space
> > (according to the CPU HOB).
> > 
> > - However, the DxeIpl method would take place before entering DXE, so no
> > GCD memory space map would be available -- the "NonExistent" entries
> > would have to be synthesized manually from the address space size (known
> > from the CPU HOB) and the lack of coverage by memory resource descriptor
> > HOBs.
> > 
> > Basically, in order to move the current GCD memory space map traversal
> > from early DXE to late PEI, the memory space map building logic of the
> > DXE Core would have to be duplicated in the DxeIpl PEIM. If I understand
> > correctly. (The DxeIpl PEIM may already contain very similar code, for
> > the page table building, which might not be difficult to extend like
> > this -- I haven't looked.)
> > 
> > Is this what you have in mind?
> > 
> 
> Do you have any further thought on this?

Regarding Laszlo's feedback, I'm not convinced that it would be
excessively difficult to accomplish this in DxeIpl. (I'm not saying
that I couldn't be convinced. :)

As far as I can see, this is an architecturally defined AMD feature.
(Is this true, or is BaseMemcryptSevLib actually OVMF specific?)

You've asserted that it should work (SEV would not be detected) with
any Intel processor as well. Therefore, I don't see a good reason that
we shouldn't be able to support it in modules that already have
IA32/X64 specific code. (I'm recalling
881813d7a93d9009c873515b043c41c4554779e4.)

Since DxeIpl builds the IA32/X64 page tables, and you need to modify
the page tables for this feature (correct?), I think we should try to
support the feature there if it is feasible. I can understand the
argument that this doesn't apply to all non-VM platforms, so I think
we could add a PCD which disables this support by default.

I don't know that the owners of MdeModulePkg and UefiCpuPkg will agree
with me though.

> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to see
> if I can invoke a platform dependent library to clear C-bit before DxeMain
> finishes its execution. As Laszlo pointed, current approach is using GCD memory
> space map to get MMIO and NonExistent entries. I have pushed two patches
> in my development branch to show what I have been doing:
> 
> 1) add a new null DxeGcdCorePlatformHookLib
> 
> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94a920d7ad72d
> 
> The library provides a function "DxeGcdCorePlatformHookReady" which can be called
> by DxeMain just after it initializes the GcdServices (which will guarantee that
> Gcd memory space map is available).

Regarding hooking into DxeCore, I don't think it is the best approach,
but it is better than APRIORI. I wonder if the MdeModulePkg owners
could jump in with an opinion. (Hopefully besides just pushing the
problem away via APRIORI.)

-Jordan

> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
>   SEV is detected.
> 
> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba53c95949f659
> 
> I've tested the approach and it seems to work. Is this something aligned with your
> thinking?
> 
> 
> > Thanks
> > Laszlo
> > 
> >> -Jordan
> >>
> >>> In second patch
> >>> [2], Leo tried to introduce a new notify protocol to get MMIO add/remove
> >>> events. During discussion Jiewen suggested to look into adding a new
> >>> platform driver into APRIORI to avoid the need for any modifications
> >>> inside the Gcdcore - this seems workable solution which did not require
> >>> adding any CPU specific code inside the Gcd.
> >>>
> >>> [1] https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
> >>> [2] https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
> >>>
> > 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

I was not tracking this thread.
Jiewen will help give comments about the potential change in MdeModulePkg.

Thanks,
Star
-----Original Message-----
From: Justen, Jordan L 
Sent: Tuesday, June 6, 2017 9:12 AM
To: Brijesh Singh <brijesh.singh@amd.com>; Laszlo Ersek <lersek@redhat.com>; edk2-devel@lists.01.org; Zeng, Star <star.zeng@intel.com>; Dong, Eric <eric.dong@intel.com>
Cc: Thomas.Lendacky@amd.com; Gao, Liming <liming.gao@intel.com>; leo.duran@amd.com; Yao, Jiewen <jiewen.yao@intel.com>; Fan, Jeff <jeff.fan@intel.com>
Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)

On 2017-06-05 14:56:04, Brijesh Singh wrote:
> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
> > On 06/01/17 09:40, Jordan Justen wrote:
> >> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
> >> Leo said that DxeIpl won't work because new I/O ranges might be added.
> >> I don't understand this, because isn't DxeIpl and an early APRIORI 
> >> entry are roughly equivalent in the boot sequence?
> > 
> > I think you are right. I believe a patch for this exact idea hasn't 
> > been posted yet. Jiewen's message that you linked above contains the 
> > expression
> > 
> >      always clear SEV mask for MMIO *and all rest*
> > 
> > (emphasis mine), which I think we may have missed *in combination 
> > with* the DxeIpl.
> > 
> > So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
> > Keep the C bit set for system memory regions. Clear the C bit for 
> > MMIO regions that are known from the HOB list. Also clear the C bit 
> > everywhere else in the address space (known from the CPU HOB) where 
> > no coverage is provided by any memory resource descriptor HOB.
> > 
> > This is going to be harder than the current approach, because:
> > 
> > - The current approach can work off of the GCD memory space map, 
> > which provides explicit NonExistent entries, covering the entire 
> > address space (according to the CPU HOB).
> > 
> > - However, the DxeIpl method would take place before entering DXE, 
> > so no GCD memory space map would be available -- the "NonExistent" 
> > entries would have to be synthesized manually from the address space 
> > size (known from the CPU HOB) and the lack of coverage by memory 
> > resource descriptor HOBs.
> > 
> > Basically, in order to move the current GCD memory space map 
> > traversal from early DXE to late PEI, the memory space map building 
> > logic of the DXE Core would have to be duplicated in the DxeIpl 
> > PEIM. If I understand correctly. (The DxeIpl PEIM may already 
> > contain very similar code, for the page table building, which might 
> > not be difficult to extend like this -- I haven't looked.)
> > 
> > Is this what you have in mind?
> > 
> 
> Do you have any further thought on this?

Regarding Laszlo's feedback, I'm not convinced that it would be excessively difficult to accomplish this in DxeIpl. (I'm not saying that I couldn't be convinced. :)

As far as I can see, this is an architecturally defined AMD feature.
(Is this true, or is BaseMemcryptSevLib actually OVMF specific?)

You've asserted that it should work (SEV would not be detected) with any Intel processor as well. Therefore, I don't see a good reason that we shouldn't be able to support it in modules that already have
IA32/X64 specific code. (I'm recalling
881813d7a93d9009c873515b043c41c4554779e4.)

Since DxeIpl builds the IA32/X64 page tables, and you need to modify the page tables for this feature (correct?), I think we should try to support the feature there if it is feasible. I can understand the argument that this doesn't apply to all non-VM platforms, so I think we could add a PCD which disables this support by default.

I don't know that the owners of MdeModulePkg and UefiCpuPkg will agree with me though.

> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to 
> see if I can invoke a platform dependent library to clear C-bit before 
> DxeMain finishes its execution. As Laszlo pointed, current approach is 
> using GCD memory space map to get MMIO and NonExistent entries. I have 
> pushed two patches in my development branch to show what I have been doing:
> 
> 1) add a new null DxeGcdCorePlatformHookLib
> 
> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94
> a920d7ad72d
> 
> The library provides a function "DxeGcdCorePlatformHookReady" which 
> can be called by DxeMain just after it initializes the GcdServices 
> (which will guarantee that Gcd memory space map is available).

Regarding hooking into DxeCore, I don't think it is the best approach, but it is better than APRIORI. I wonder if the MdeModulePkg owners could jump in with an opinion. (Hopefully besides just pushing the problem away via APRIORI.)

-Jordan

> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
>   SEV is detected.
> 
> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba5
> 3c95949f659
> 
> I've tested the approach and it seems to work. Is this something 
> aligned with your thinking?
> 
> 
> > Thanks
> > Laszlo
> > 
> >> -Jordan
> >>
> >>> In second patch
> >>> [2], Leo tried to introduce a new notify protocol to get MMIO 
> >>> add/remove events. During discussion Jiewen suggested to look into 
> >>> adding a new platform driver into APRIORI to avoid the need for 
> >>> any modifications inside the Gcdcore - this seems workable 
> >>> solution which did not require adding any CPU specific code inside the Gcd.
> >>>
> >>> [1] 
> >>> https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
> >>> [2] 
> >>> https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
> >>>
> > 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Hi Jordan,


On 6/5/17 9:08 PM, Zeng, Star wrote:
> I was not tracking this thread.
> Jiewen will help give comments about the potential change in MdeModulePkg.
>
> Thanks,
> Star
> -----Original Message-----
> From: Justen, Jordan L 
> Sent: Tuesday, June 6, 2017 9:12 AM
> To: Brijesh Singh <brijesh.singh@amd.com>; Laszlo Ersek <lersek@redhat.com>; edk2-devel@lists.01.org; Zeng, Star <star.zeng@intel.com>; Dong, Eric <eric.dong@intel.com>
> Cc: Thomas.Lendacky@amd.com; Gao, Liming <liming.gao@intel.com>; leo.duran@amd.com; Yao, Jiewen <jiewen.yao@intel.com>; Fan, Jeff <jeff.fan@intel.com>
> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>
> On 2017-06-05 14:56:04, Brijesh Singh wrote:
>> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
>>> On 06/01/17 09:40, Jordan Justen wrote:
>>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>>> I don't understand this, because isn't DxeIpl and an early APRIORI 
>>>> entry are roughly equivalent in the boot sequence?
>>> I think you are right. I believe a patch for this exact idea hasn't 
>>> been posted yet. Jiewen's message that you linked above contains the 
>>> expression
>>>
>>>      always clear SEV mask for MMIO *and all rest*
>>>
>>> (emphasis mine), which I think we may have missed *in combination 
>>> with* the DxeIpl.
>>>
>>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>>> Keep the C bit set for system memory regions. Clear the C bit for 
>>> MMIO regions that are known from the HOB list. Also clear the C bit 
>>> everywhere else in the address space (known from the CPU HOB) where 
>>> no coverage is provided by any memory resource descriptor HOB.
>>>
>>> This is going to be harder than the current approach, because:
>>>
>>> - The current approach can work off of the GCD memory space map, 
>>> which provides explicit NonExistent entries, covering the entire 
>>> address space (according to the CPU HOB).
>>>
>>> - However, the DxeIpl method would take place before entering DXE, 
>>> so no GCD memory space map would be available -- the "NonExistent" 
>>> entries would have to be synthesized manually from the address space 
>>> size (known from the CPU HOB) and the lack of coverage by memory 
>>> resource descriptor HOBs.
>>>
>>> Basically, in order to move the current GCD memory space map 
>>> traversal from early DXE to late PEI, the memory space map building 
>>> logic of the DXE Core would have to be duplicated in the DxeIpl 
>>> PEIM. If I understand correctly. (The DxeIpl PEIM may already 
>>> contain very similar code, for the page table building, which might 
>>> not be difficult to extend like this -- I haven't looked.)
>>>
>>> Is this what you have in mind?
>>>
>> Do you have any further thought on this?
> Regarding Laszlo's feedback, I'm not convinced that it would be excessively difficult to accomplish this in DxeIpl. (I'm not saying that I couldn't be convinced. :)
>
> As far as I can see, this is an architecturally defined AMD feature.
> (Is this true, or is BaseMemcryptSevLib actually OVMF specific?)

Yes, SEV is AMD-V architecture extension and its applicable to
virtualization platform only (we can says BaseMemEncryptSevLib is OVMF
specific).
> You've asserted that it should work (SEV would not be detected) with any Intel processor as well. Therefore, I don't see a good reason that we shouldn't be able to support it in modules that already have
> IA32/X64 specific code. (I'm recalling
> 881813d7a93d9009c873515b043c41c4554779e4.)
>
> Since DxeIpl builds the IA32/X64 page tables, and you need to modify the page tables for this feature (correct?), I think we should try to support the feature there if it is feasible. I can understand the argument that this doesn't apply to all non-VM platforms, so I think we could add a PCD which disables this support by default.
>
> I don't know that the owners of MdeModulePkg and UefiCpuPkg will agree with me though.

I am flexible to implement APRIORI or Platform hooks Lib. But one thing
I want to highlight is: I'll prefer clearing C-bit  through
BaseMemEncryptSevLib functions. One of the main reason for doing so - In
future when we add migration support for the SEV guest then we will be
required to notify the unencrypted page range to hypevisor ( through
hypercall). During migration phase, Hypervisor will use this information
to make decision on whether to invoke the SEV firmware to encrypt the
memory region for transport purposes. If clearing C-bit logic is
contained inside BaseMemEncryptSevLib then it will make life much easier.

>> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to 
>> see if I can invoke a platform dependent library to clear C-bit before 
>> DxeMain finishes its execution. As Laszlo pointed, current approach is 
>> using GCD memory space map to get MMIO and NonExistent entries. I have 
>> pushed two patches in my development branch to show what I have been doing:
>>
>> 1) add a new null DxeGcdCorePlatformHookLib
>>
>> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94
>> a920d7ad72d
>>
>> The library provides a function "DxeGcdCorePlatformHookReady" which 
>> can be called by DxeMain just after it initializes the GcdServices 
>> (which will guarantee that Gcd memory space map is available).
> Regarding hooking into DxeCore, I don't think it is the best approach, but it is better than APRIORI. I wonder if the MdeModulePkg owners could jump in with an opinion. (Hopefully besides just pushing the problem away via APRIORI.)

Jiewen, any comments ?

> -Jordan
>
>> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
>>   SEV is detected.
>>
>> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba5
>> 3c95949f659
>>
>> I've tested the approach and it seems to work. Is this something 
>> aligned with your thinking?
>>
>>
>>> Thanks
>>> Laszlo
>>>
>>>> -Jordan
>>>>
>>>>> In second patch
>>>>> [2], Leo tried to introduce a new notify protocol to get MMIO 
>>>>> add/remove events. During discussion Jiewen suggested to look into 
>>>>> adding a new platform driver into APRIORI to avoid the need for 
>>>>> any modifications inside the Gcdcore - this seems workable 
>>>>> solution which did not require adding any CPU specific code inside the Gcd.
>>>>>
>>>>> [1] 
>>>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>>> [2] 
>>>>> https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>>
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org
>> https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Hi
It takes me some time to read all email below. I believe all of us have a clean understanding on what problem we have now and the possible solutions to clear C bit are below

1)       In DxeIpl, when it builds page table.

2)       In DxeCore

a)         By use CpuArch

b)         By use page table lib

c)         By use a GCD update callback

d)         By use PlatformHook lib

3)       In a standalone AmdSev driver.

Here is my thought:
2.a) is not possible, per Leo’s investigation.
2.b) is not a good design, because we do not introduce any Cpu Specific thing to DxeCore so far.
2.c) and 2.d) are same. I do not suggest we add a private interface to the core just to support one specific feature.

1) is one possible solution, I suggested before. But if Leo/Laszlo think it is too hard to implement, I am OK.

If 1) cannot be chosen, I still think 3) is the best idea.
It makes the code very clean by introducing a standalone driver to resolve the problem.
Zero impact on existing platform.
If this feature is not needed, just remove the driver.

I do not see any issue on using a priori, because: A) “a priori” is clearly defined in PI spec, B) “a priori” has already been widely used in current platform in EDKII open source, as well as close source platform.

Thank you
Yao Jiewen



From: Brijesh Singh [mailto:brijesh.singh@amd.com]
Sent: Tuesday, June 6, 2017 11:51 AM
To: Zeng, Star <star.zeng@intel.com>; Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>; edk2-devel@lists.01.org; Dong, Eric <eric.dong@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>
Cc: Thomas.Lendacky@amd.com; Gao, Liming <liming.gao@intel.com>; leo.duran@amd.com; Fan, Jeff <jeff.fan@intel.com>
Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)

Hi Jordan,


On 6/5/17 9:08 PM, Zeng, Star wrote:
> I was not tracking this thread.
> Jiewen will help give comments about the potential change in MdeModulePkg.
>
> Thanks,
> Star
> -----Original Message-----
> From: Justen, Jordan L
> Sent: Tuesday, June 6, 2017 9:12 AM
> To: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com>>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com>>
> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; leo.duran@amd.com<mailto:leo.duran@amd.com>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com>>
> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>
> On 2017-06-05 14:56:04, Brijesh Singh wrote:
>> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
>>> On 06/01/17 09:40, Jordan Justen wrote:
>>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>>> I don't understand this, because isn't DxeIpl and an early APRIORI
>>>> entry are roughly equivalent in the boot sequence?
>>> I think you are right. I believe a patch for this exact idea hasn't
>>> been posted yet. Jiewen's message that you linked above contains the
>>> expression
>>>
>>>      always clear SEV mask for MMIO *and all rest*
>>>
>>> (emphasis mine), which I think we may have missed *in combination
>>> with* the DxeIpl.
>>>
>>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>>> Keep the C bit set for system memory regions. Clear the C bit for
>>> MMIO regions that are known from the HOB list. Also clear the C bit
>>> everywhere else in the address space (known from the CPU HOB) where
>>> no coverage is provided by any memory resource descriptor HOB.
>>>
>>> This is going to be harder than the current approach, because:
>>>
>>> - The current approach can work off of the GCD memory space map,
>>> which provides explicit NonExistent entries, covering the entire
>>> address space (according to the CPU HOB).
>>>
>>> - However, the DxeIpl method would take place before entering DXE,
>>> so no GCD memory space map would be available -- the "NonExistent"
>>> entries would have to be synthesized manually from the address space
>>> size (known from the CPU HOB) and the lack of coverage by memory
>>> resource descriptor HOBs.
>>>
>>> Basically, in order to move the current GCD memory space map
>>> traversal from early DXE to late PEI, the memory space map building
>>> logic of the DXE Core would have to be duplicated in the DxeIpl
>>> PEIM. If I understand correctly. (The DxeIpl PEIM may already
>>> contain very similar code, for the page table building, which might
>>> not be difficult to extend like this -- I haven't looked.)
>>>
>>> Is this what you have in mind?
>>>
>> Do you have any further thought on this?
> Regarding Laszlo's feedback, I'm not convinced that it would be excessively difficult to accomplish this in DxeIpl. (I'm not saying that I couldn't be convinced. :)
>
> As far as I can see, this is an architecturally defined AMD feature.
> (Is this true, or is BaseMemcryptSevLib actually OVMF specific?)

Yes, SEV is AMD-V architecture extension and its applicable to
virtualization platform only (we can says BaseMemEncryptSevLib is OVMF
specific).
> You've asserted that it should work (SEV would not be detected) with any Intel processor as well. Therefore, I don't see a good reason that we shouldn't be able to support it in modules that already have
> IA32/X64 specific code. (I'm recalling
> 881813d7a93d9009c873515b043c41c4554779e4.)
>
> Since DxeIpl builds the IA32/X64 page tables, and you need to modify the page tables for this feature (correct?), I think we should try to support the feature there if it is feasible. I can understand the argument that this doesn't apply to all non-VM platforms, so I think we could add a PCD which disables this support by default.
>
> I don't know that the owners of MdeModulePkg and UefiCpuPkg will agree with me though.

I am flexible to implement APRIORI or Platform hooks Lib. But one thing
I want to highlight is: I'll prefer clearing C-bit  through
BaseMemEncryptSevLib functions. One of the main reason for doing so - In
future when we add migration support for the SEV guest then we will be
required to notify the unencrypted page range to hypevisor ( through
hypercall). During migration phase, Hypervisor will use this information
to make decision on whether to invoke the SEV firmware to encrypt the
memory region for transport purposes. If clearing C-bit logic is
contained inside BaseMemEncryptSevLib then it will make life much easier.

>> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to
>> see if I can invoke a platform dependent library to clear C-bit before
>> DxeMain finishes its execution. As Laszlo pointed, current approach is
>> using GCD memory space map to get MMIO and NonExistent entries. I have
>> pushed two patches in my development branch to show what I have been doing:
>>
>> 1) add a new null DxeGcdCorePlatformHookLib
>>
>> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94
>> a920d7ad72d
>>
>> The library provides a function "DxeGcdCorePlatformHookReady" which
>> can be called by DxeMain just after it initializes the GcdServices
>> (which will guarantee that Gcd memory space map is available).
> Regarding hooking into DxeCore, I don't think it is the best approach, but it is better than APRIORI. I wonder if the MdeModulePkg owners could jump in with an opinion. (Hopefully besides just pushing the problem away via APRIORI.)

Jiewen, any comments ?

> -Jordan
>
>> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
>>   SEV is detected.
>>
>> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba5
>> 3c95949f659
>>
>> I've tested the approach and it seems to work. Is this something
>> aligned with your thinking?
>>
>>
>>> Thanks
>>> Laszlo
>>>
>>>> -Jordan
>>>>
>>>>> In second patch
>>>>> [2], Leo tried to introduce a new notify protocol to get MMIO
>>>>> add/remove events. During discussion Jiewen suggested to look into
>>>>> adding a new platform driver into APRIORI to avoid the need for
>>>>> any modifications inside the Gcdcore - this seems workable
>>>>> solution which did not require adding any CPU specific code inside the Gcd.
>>>>>
>>>>> [1]
>>>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>>> [2]
>>>>> https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>>
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
>> https://lists.01.org/mailman/listinfo/edk2-devel
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

> On Jun 6, 2017, at 7:54 AM, Yao, Jiewen <jiewen.yao@intel.com> wrote:
> 
> Hi
> It takes me some time to read all email below. I believe all of us have a clean understanding on what problem we have now and the possible solutions to clear C bit are below
> 
> 1)       In DxeIpl, when it builds page table.
> 
> 2)       In DxeCore
> 
> a)         By use CpuArch
> 
> b)         By use page table lib
> 
> c)         By use a GCD update callback
> 
> d)         By use PlatformHook lib
> 
> 3)       In a standalone AmdSev driver.
> 
> Here is my thought:
> 2.a) is not possible, per Leo’s investigation.
> 2.b) is not a good design, because we do not introduce any Cpu Specific thing to DxeCore so far.
> 2.c) and 2.d) are same. I do not suggest we add a private interface to the core just to support one specific feature.
> 
> 1) is one possible solution, I suggested before. But if Leo/Laszlo think it is too hard to implement, I am OK.
> 
> If 1) cannot be chosen, I still think 3) is the best idea.
> It makes the code very clean by introducing a standalone driver to resolve the problem.
> Zero impact on existing platform.
> If this feature is not needed, just remove the driver.
> 
> I do not see any issue on using a priori, because: A) “a priori” is clearly defined in PI spec, B) “a priori” has already been widely used in current platform in EDKII open source, as well as close source platform.
> 

Jiewen,

I agree that "a priori" is part of the architecture so it is OK to use it, but "a priori" was never really intended as a way to add basic features. it was more for debugging and work arounds. It seems like a feature like this should not require a work around....

'So I think it is OK to accept this patch to get the feature enabled, but we need to look at the GCD implementation and PI architecture to figure out why there is not a cleaner way to add this feature. Maybe we need to change the implementation, and/or the PI Spec?

Thanks,

Andrew Fish

> Thank you
> Yao Jiewen
> 
> 
> 
> From: Brijesh Singh [mailto:brijesh.singh@amd.com]
> Sent: Tuesday, June 6, 2017 11:51 AM
> To: Zeng, Star <star.zeng@intel.com>; Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>; edk2-devel@lists.01.org; Dong, Eric <eric.dong@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>
> Cc: Thomas.Lendacky@amd.com; Gao, Liming <liming.gao@intel.com>; leo.duran@amd.com; Fan, Jeff <jeff.fan@intel.com>
> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
> 
> Hi Jordan,
> 
> 
> On 6/5/17 9:08 PM, Zeng, Star wrote:
>> I was not tracking this thread.
>> Jiewen will help give comments about the potential change in MdeModulePkg.
>> 
>> Thanks,
>> Star
>> -----Original Message-----
>> From: Justen, Jordan L
>> Sent: Tuesday, June 6, 2017 9:12 AM
>> To: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com>>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com>>
>> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; leo.duran@amd.com<mailto:leo.duran@amd.com>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com>>
>> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>> 
>> On 2017-06-05 14:56:04, Brijesh Singh wrote:
>>> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
>>>> On 06/01/17 09:40, Jordan Justen wrote:
>>>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>>>> I don't understand this, because isn't DxeIpl and an early APRIORI
>>>>> entry are roughly equivalent in the boot sequence?
>>>> I think you are right. I believe a patch for this exact idea hasn't
>>>> been posted yet. Jiewen's message that you linked above contains the
>>>> expression
>>>> 
>>>>     always clear SEV mask for MMIO *and all rest*
>>>> 
>>>> (emphasis mine), which I think we may have missed *in combination
>>>> with* the DxeIpl.
>>>> 
>>>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>>>> Keep the C bit set for system memory regions. Clear the C bit for
>>>> MMIO regions that are known from the HOB list. Also clear the C bit
>>>> everywhere else in the address space (known from the CPU HOB) where
>>>> no coverage is provided by any memory resource descriptor HOB.
>>>> 
>>>> This is going to be harder than the current approach, because:
>>>> 
>>>> - The current approach can work off of the GCD memory space map,
>>>> which provides explicit NonExistent entries, covering the entire
>>>> address space (according to the CPU HOB).
>>>> 
>>>> - However, the DxeIpl method would take place before entering DXE,
>>>> so no GCD memory space map would be available -- the "NonExistent"
>>>> entries would have to be synthesized manually from the address space
>>>> size (known from the CPU HOB) and the lack of coverage by memory
>>>> resource descriptor HOBs.
>>>> 
>>>> Basically, in order to move the current GCD memory space map
>>>> traversal from early DXE to late PEI, the memory space map building
>>>> logic of the DXE Core would have to be duplicated in the DxeIpl
>>>> PEIM. If I understand correctly. (The DxeIpl PEIM may already
>>>> contain very similar code, for the page table building, which might
>>>> not be difficult to extend like this -- I haven't looked.)
>>>> 
>>>> Is this what you have in mind?
>>>> 
>>> Do you have any further thought on this?
>> Regarding Laszlo's feedback, I'm not convinced that it would be excessively difficult to accomplish this in DxeIpl. (I'm not saying that I couldn't be convinced. :)
>> 
>> As far as I can see, this is an architecturally defined AMD feature.
>> (Is this true, or is BaseMemcryptSevLib actually OVMF specific?)
> 
> Yes, SEV is AMD-V architecture extension and its applicable to
> virtualization platform only (we can says BaseMemEncryptSevLib is OVMF
> specific).
>> You've asserted that it should work (SEV would not be detected) with any Intel processor as well. Therefore, I don't see a good reason that we shouldn't be able to support it in modules that already have
>> IA32/X64 specific code. (I'm recalling
>> 881813d7a93d9009c873515b043c41c4554779e4.)
>> 
>> Since DxeIpl builds the IA32/X64 page tables, and you need to modify the page tables for this feature (correct?), I think we should try to support the feature there if it is feasible. I can understand the argument that this doesn't apply to all non-VM platforms, so I think we could add a PCD which disables this support by default.
>> 
>> I don't know that the owners of MdeModulePkg and UefiCpuPkg will agree with me though.
> 
> I am flexible to implement APRIORI or Platform hooks Lib. But one thing
> I want to highlight is: I'll prefer clearing C-bit  through
> BaseMemEncryptSevLib functions. One of the main reason for doing so - In
> future when we add migration support for the SEV guest then we will be
> required to notify the unencrypted page range to hypevisor ( through
> hypercall). During migration phase, Hypervisor will use this information
> to make decision on whether to invoke the SEV firmware to encrypt the
> memory region for transport purposes. If clearing C-bit logic is
> contained inside BaseMemEncryptSevLib then it will make life much easier.
> 
>>> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to
>>> see if I can invoke a platform dependent library to clear C-bit before
>>> DxeMain finishes its execution. As Laszlo pointed, current approach is
>>> using GCD memory space map to get MMIO and NonExistent entries. I have
>>> pushed two patches in my development branch to show what I have been doing:
>>> 
>>> 1) add a new null DxeGcdCorePlatformHookLib
>>> 
>>> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94
>>> a920d7ad72d
>>> 
>>> The library provides a function "DxeGcdCorePlatformHookReady" which
>>> can be called by DxeMain just after it initializes the GcdServices
>>> (which will guarantee that Gcd memory space map is available).
>> Regarding hooking into DxeCore, I don't think it is the best approach, but it is better than APRIORI. I wonder if the MdeModulePkg owners could jump in with an opinion. (Hopefully besides just pushing the problem away via APRIORI.)
> 
> Jiewen, any comments ?
> 
>> -Jordan
>> 
>>> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
>>>  SEV is detected.
>>> 
>>> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba5
>>> 3c95949f659
>>> 
>>> I've tested the approach and it seems to work. Is this something
>>> aligned with your thinking?
>>> 
>>> 
>>>> Thanks
>>>> Laszlo
>>>> 
>>>>> -Jordan
>>>>> 
>>>>>> In second patch
>>>>>> [2], Leo tried to introduce a new notify protocol to get MMIO
>>>>>> add/remove events. During discussion Jiewen suggested to look into
>>>>>> adding a new platform driver into APRIORI to avoid the need for
>>>>>> any modifications inside the Gcdcore - this seems workable
>>>>>> solution which did not require adding any CPU specific code inside the Gcd.
>>>>>> 
>>>>>> [1]
>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>>>> [2]
>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>>> 
>>> _______________________________________________
>>> edk2-devel mailing list
>>> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
>>> https://lists.01.org/mailman/listinfo/edk2-devel
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Hi Andrew
Yes, I agree. If we could figure out a cleaner way to resolve the problem, we should use the cleaner way.



If we really really do not want to use a priori for AmdSec, we can let AmdSec to publish a special protocol, and let the driver depend that protocol, if this driver need add MMIO region.
Because AmdSec is inside of OvmfPkg, this special protocol can be in OvmfPkg.

That is just another option.

Thank you
Yao Jiewen

From: afish@apple.com [mailto:afish@apple.com]
Sent: Tuesday, June 6, 2017 11:25 PM
To: Yao, Jiewen <jiewen.yao@intel.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>; Zeng, Star <star.zeng@intel.com>; Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>; edk2-devel@lists.01.org; Dong, Eric <eric.dong@intel.com>; Thomas.Lendacky@amd.com; leo.duran@amd.com; Fan, Jeff <jeff.fan@intel.com>; Gao, Liming <liming.gao@intel.com>
Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)


> On Jun 6, 2017, at 7:54 AM, Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>> wrote:
>
> Hi
> It takes me some time to read all email below. I believe all of us have a clean understanding on what problem we have now and the possible solutions to clear C bit are below
>
> 1)       In DxeIpl, when it builds page table.
>
> 2)       In DxeCore
>
> a)         By use CpuArch
>
> b)         By use page table lib
>
> c)         By use a GCD update callback
>
> d)         By use PlatformHook lib
>
> 3)       In a standalone AmdSev driver.
>
> Here is my thought:
> 2.a) is not possible, per Leo’s investigation.
> 2.b) is not a good design, because we do not introduce any Cpu Specific thing to DxeCore so far.
> 2.c) and 2.d) are same. I do not suggest we add a private interface to the core just to support one specific feature.
>
> 1) is one possible solution, I suggested before. But if Leo/Laszlo think it is too hard to implement, I am OK.
>
> If 1) cannot be chosen, I still think 3) is the best idea.
> It makes the code very clean by introducing a standalone driver to resolve the problem.
> Zero impact on existing platform.
> If this feature is not needed, just remove the driver.
>
> I do not see any issue on using a priori, because: A) “a priori” is clearly defined in PI spec, B) “a priori” has already been widely used in current platform in EDKII open source, as well as close source platform.
>

Jiewen,

I agree that "a priori" is part of the architecture so it is OK to use it, but "a priori" was never really intended as a way to add basic features. it was more for debugging and work arounds. It seems like a feature like this should not require a work around....

'So I think it is OK to accept this patch to get the feature enabled, but we need to look at the GCD implementation and PI architecture to figure out why there is not a cleaner way to add this feature. Maybe we need to change the implementation, and/or the PI Spec?

Thanks,

Andrew Fish

> Thank you
> Yao Jiewen
>
>
>
> From: Brijesh Singh [mailto:brijesh.singh@amd.com]
> Sent: Tuesday, June 6, 2017 11:51 AM
> To: Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com>>; Justen, Jordan L <jordan.l.justen@intel.com<mailto:jordan.l.justen@intel.com>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com>>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>
> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; leo.duran@amd.com<mailto:leo.duran@amd.com>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com>>
> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>
> Hi Jordan,
>
>
> On 6/5/17 9:08 PM, Zeng, Star wrote:
>> I was not tracking this thread.
>> Jiewen will help give comments about the potential change in MdeModulePkg.
>>
>> Thanks,
>> Star
>> -----Original Message-----
>> From: Justen, Jordan L
>> Sent: Tuesday, June 6, 2017 9:12 AM
>> To: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com<mailto:brijesh.singh@amd.com%3cmailto:brijesh.singh@amd.com>>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com<mailto:lersek@redhat.com%3cmailto:lersek@redhat.com>>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org>>; Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com<mailto:star.zeng@intel.com%3cmailto:star.zeng@intel.com>>>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com<mailto:eric.dong@intel.com%3cmailto:eric.dong@intel.com>>>
>> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com%3cmailto:Thomas.Lendacky@amd.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com<mailto:liming.gao@intel.com%3cmailto:liming.gao@intel.com>>>; leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com<mailto:jiewen.yao@intel.com%3cmailto:jiewen.yao@intel.com>>>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com<mailto:jeff.fan@intel.com%3cmailto:jeff.fan@intel.com>>>
>> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>>
>> On 2017-06-05 14:56:04, Brijesh Singh wrote:
>>> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
>>>> On 06/01/17 09:40, Jordan Justen wrote:
>>>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>>>> I don't understand this, because isn't DxeIpl and an early APRIORI
>>>>> entry are roughly equivalent in the boot sequence?
>>>> I think you are right. I believe a patch for this exact idea hasn't
>>>> been posted yet. Jiewen's message that you linked above contains the
>>>> expression
>>>>
>>>>     always clear SEV mask for MMIO *and all rest*
>>>>
>>>> (emphasis mine), which I think we may have missed *in combination
>>>> with* the DxeIpl.
>>>>
>>>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>>>> Keep the C bit set for system memory regions. Clear the C bit for
>>>> MMIO regions that are known from the HOB list. Also clear the C bit
>>>> everywhere else in the address space (known from the CPU HOB) where
>>>> no coverage is provided by any memory resource descriptor HOB.
>>>>
>>>> This is going to be harder than the current approach, because:
>>>>
>>>> - The current approach can work off of the GCD memory space map,
>>>> which provides explicit NonExistent entries, covering the entire
>>>> address space (according to the CPU HOB).
>>>>
>>>> - However, the DxeIpl method would take place before entering DXE,
>>>> so no GCD memory space map would be available -- the "NonExistent"
>>>> entries would have to be synthesized manually from the address space
>>>> size (known from the CPU HOB) and the lack of coverage by memory
>>>> resource descriptor HOBs.
>>>>
>>>> Basically, in order to move the current GCD memory space map
>>>> traversal from early DXE to late PEI, the memory space map building
>>>> logic of the DXE Core would have to be duplicated in the DxeIpl
>>>> PEIM. If I understand correctly. (The DxeIpl PEIM may already
>>>> contain very similar code, for the page table building, which might
>>>> not be difficult to extend like this -- I haven't looked.)
>>>>
>>>> Is this what you have in mind?
>>>>
>>> Do you have any further thought on this?
>> Regarding Laszlo's feedback, I'm not convinced that it would be excessively difficult to accomplish this in DxeIpl. (I'm not saying that I couldn't be convinced. :)
>>
>> As far as I can see, this is an architecturally defined AMD feature.
>> (Is this true, or is BaseMemcryptSevLib actually OVMF specific?)
>
> Yes, SEV is AMD-V architecture extension and its applicable to
> virtualization platform only (we can says BaseMemEncryptSevLib is OVMF
> specific).
>> You've asserted that it should work (SEV would not be detected) with any Intel processor as well. Therefore, I don't see a good reason that we shouldn't be able to support it in modules that already have
>> IA32/X64 specific code. (I'm recalling
>> 881813d7a93d9009c873515b043c41c4554779e4.)
>>
>> Since DxeIpl builds the IA32/X64 page tables, and you need to modify the page tables for this feature (correct?), I think we should try to support the feature there if it is feasible. I can understand the argument that this doesn't apply to all non-VM platforms, so I think we could add a PCD which disables this support by default.
>>
>> I don't know that the owners of MdeModulePkg and UefiCpuPkg will agree with me though.
>
> I am flexible to implement APRIORI or Platform hooks Lib. But one thing
> I want to highlight is: I'll prefer clearing C-bit  through
> BaseMemEncryptSevLib functions. One of the main reason for doing so - In
> future when we add migration support for the SEV guest then we will be
> required to notify the unencrypted page range to hypevisor ( through
> hypercall). During migration phase, Hypervisor will use this information
> to make decision on whether to invoke the SEV firmware to encrypt the
> memory region for transport purposes. If clearing C-bit logic is
> contained inside BaseMemEncryptSevLib then it will make life much easier.
>
>>> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to
>>> see if I can invoke a platform dependent library to clear C-bit before
>>> DxeMain finishes its execution. As Laszlo pointed, current approach is
>>> using GCD memory space map to get MMIO and NonExistent entries. I have
>>> pushed two patches in my development branch to show what I have been doing:
>>>
>>> 1) add a new null DxeGcdCorePlatformHookLib
>>>
>>> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94
>>> a920d7ad72d
>>>
>>> The library provides a function "DxeGcdCorePlatformHookReady" which
>>> can be called by DxeMain just after it initializes the GcdServices
>>> (which will guarantee that Gcd memory space map is available).
>> Regarding hooking into DxeCore, I don't think it is the best approach, but it is better than APRIORI. I wonder if the MdeModulePkg owners could jump in with an opinion. (Hopefully besides just pushing the problem away via APRIORI.)
>
> Jiewen, any comments ?
>
>> -Jordan
>>
>>> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
>>>  SEV is detected.
>>>
>>> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba5
>>> 3c95949f659
>>>
>>> I've tested the approach and it seems to work. Is this something
>>> aligned with your thinking?
>>>
>>>
>>>> Thanks
>>>> Laszlo
>>>>
>>>>> -Jordan
>>>>>
>>>>>> In second patch
>>>>>> [2], Leo tried to introduce a new notify protocol to get MMIO
>>>>>> add/remove events. During discussion Jiewen suggested to look into
>>>>>> adding a new platform driver into APRIORI to avoid the need for
>>>>>> any modifications inside the Gcdcore - this seems workable
>>>>>> solution which did not require adding any CPU specific code inside the Gcd.
>>>>>>
>>>>>> [1]
>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>>>> [2]
>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>>>
>>> _______________________________________________
>>> edk2-devel mailing list
>>> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org>>
>>> https://lists.01.org/mailman/listinfo/edk2-devel
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
> https://lists.01.org/mailman/listinfo/edk2-devel
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Thank you all for the feedback… The summary of what I’m hearing is:

1)      For now, use APRIORI in the stand-alone (SEV) driver in OvmfPkg (albeit not as clean as desired, but least intrusive solution.).

2)      Let’s revisit the existing GCD implementation to allow for cleaner of features like this.

Leo.


From: Yao, Jiewen [mailto:jiewen.yao@intel.com]
Sent: Tuesday, June 06, 2017 10:43 AM
To: afish@apple.com
Cc: Singh, Brijesh <brijesh.singh@amd.com>; Zeng, Star <star.zeng@intel.com>; Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>; edk2-devel@lists.01.org; Dong, Eric <eric.dong@intel.com>; Lendacky, Thomas <Thomas.Lendacky@amd.com>; Duran, Leo <leo.duran@amd.com>; Fan, Jeff <jeff.fan@intel.com>; Gao, Liming <liming.gao@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>
Subject: RE: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)

Hi Andrew
Yes, I agree. If we could figure out a cleaner way to resolve the problem, we should use the cleaner way.



If we really really do not want to use a priori for AmdSec, we can let AmdSec to publish a special protocol, and let the driver depend that protocol, if this driver need add MMIO region.
Because AmdSec is inside of OvmfPkg, this special protocol can be in OvmfPkg.

That is just another option.

Thank you
Yao Jiewen

From: afish@apple.com<mailto:afish@apple.com> [mailto:afish@apple.com]
Sent: Tuesday, June 6, 2017 11:25 PM
To: Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>
Cc: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>; Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com>>; Justen, Jordan L <jordan.l.justen@intel.com<mailto:jordan.l.justen@intel.com>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com>>; Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com>; leo.duran@amd.com<mailto:leo.duran@amd.com>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)


> On Jun 6, 2017, at 7:54 AM, Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>> wrote:
>
> Hi
> It takes me some time to read all email below. I believe all of us have a clean understanding on what problem we have now and the possible solutions to clear C bit are below
>
> 1)       In DxeIpl, when it builds page table.
>
> 2)       In DxeCore
>
> a)         By use CpuArch
>
> b)         By use page table lib
>
> c)         By use a GCD update callback
>
> d)         By use PlatformHook lib
>
> 3)       In a standalone AmdSev driver.
>
> Here is my thought:
> 2.a) is not possible, per Leo’s investigation.
> 2.b) is not a good design, because we do not introduce any Cpu Specific thing to DxeCore so far.
> 2.c) and 2.d) are same. I do not suggest we add a private interface to the core just to support one specific feature.
>
> 1) is one possible solution, I suggested before. But if Leo/Laszlo think it is too hard to implement, I am OK.
>
> If 1) cannot be chosen, I still think 3) is the best idea.
> It makes the code very clean by introducing a standalone driver to resolve the problem.
> Zero impact on existing platform.
> If this feature is not needed, just remove the driver.
>
> I do not see any issue on using a priori, because: A) “a priori” is clearly defined in PI spec, B) “a priori” has already been widely used in current platform in EDKII open source, as well as close source platform.
>

Jiewen,

I agree that "a priori" is part of the architecture so it is OK to use it, but "a priori" was never really intended as a way to add basic features. it was more for debugging and work arounds. It seems like a feature like this should not require a work around....

'So I think it is OK to accept this patch to get the feature enabled, but we need to look at the GCD implementation and PI architecture to figure out why there is not a cleaner way to add this feature. Maybe we need to change the implementation, and/or the PI Spec?

Thanks,

Andrew Fish

> Thank you
> Yao Jiewen
>
>
>
> From: Brijesh Singh [mailto:brijesh.singh@amd.com]
> Sent: Tuesday, June 6, 2017 11:51 AM
> To: Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com>>; Justen, Jordan L <jordan.l.justen@intel.com<mailto:jordan.l.justen@intel.com>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com>>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>
> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; leo.duran@amd.com<mailto:leo.duran@amd.com>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com>>
> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>
> Hi Jordan,
>
>
> On 6/5/17 9:08 PM, Zeng, Star wrote:
>> I was not tracking this thread.
>> Jiewen will help give comments about the potential change in MdeModulePkg.
>>
>> Thanks,
>> Star
>> -----Original Message-----
>> From: Justen, Jordan L
>> Sent: Tuesday, June 6, 2017 9:12 AM
>> To: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com<mailto:brijesh.singh@amd.com%3cmailto:brijesh.singh@amd.com>>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com<mailto:lersek@redhat.com%3cmailto:lersek@redhat.com>>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org>>; Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com<mailto:star.zeng@intel.com%3cmailto:star.zeng@intel.com>>>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com<mailto:eric.dong@intel.com%3cmailto:eric.dong@intel.com>>>
>> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com%3cmailto:Thomas.Lendacky@amd.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com<mailto:liming.gao@intel.com%3cmailto:liming.gao@intel.com>>>; leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com<mailto:jiewen.yao@intel.com%3cmailto:jiewen.yao@intel.com>>>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com<mailto:jeff.fan@intel.com%3cmailto:jeff.fan@intel.com>>>
>> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>>
>> On 2017-06-05 14:56:04, Brijesh Singh wrote:
>>> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
>>>> On 06/01/17 09:40, Jordan Justen wrote:
>>>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>>>> I don't understand this, because isn't DxeIpl and an early APRIORI
>>>>> entry are roughly equivalent in the boot sequence?
>>>> I think you are right. I believe a patch for this exact idea hasn't
>>>> been posted yet. Jiewen's message that you linked above contains the
>>>> expression
>>>>
>>>>     always clear SEV mask for MMIO *and all rest*
>>>>
>>>> (emphasis mine), which I think we may have missed *in combination
>>>> with* the DxeIpl.
>>>>
>>>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>>>> Keep the C bit set for system memory regions. Clear the C bit for
>>>> MMIO regions that are known from the HOB list. Also clear the C bit
>>>> everywhere else in the address space (known from the CPU HOB) where
>>>> no coverage is provided by any memory resource descriptor HOB.
>>>>
>>>> This is going to be harder than the current approach, because:
>>>>
>>>> - The current approach can work off of the GCD memory space map,
>>>> which provides explicit NonExistent entries, covering the entire
>>>> address space (according to the CPU HOB).
>>>>
>>>> - However, the DxeIpl method would take place before entering DXE,
>>>> so no GCD memory space map would be available -- the "NonExistent"
>>>> entries would have to be synthesized manually from the address space
>>>> size (known from the CPU HOB) and the lack of coverage by memory
>>>> resource descriptor HOBs.
>>>>
>>>> Basically, in order to move the current GCD memory space map
>>>> traversal from early DXE to late PEI, the memory space map building
>>>> logic of the DXE Core would have to be duplicated in the DxeIpl
>>>> PEIM. If I understand correctly. (The DxeIpl PEIM may already
>>>> contain very similar code, for the page table building, which might
>>>> not be difficult to extend like this -- I haven't looked.)
>>>>
>>>> Is this what you have in mind?
>>>>
>>> Do you have any further thought on this?
>> Regarding Laszlo's feedback, I'm not convinced that it would be excessively difficult to accomplish this in DxeIpl. (I'm not saying that I couldn't be convinced. :)
>>
>> As far as I can see, this is an architecturally defined AMD feature.
>> (Is this true, or is BaseMemcryptSevLib actually OVMF specific?)
>
> Yes, SEV is AMD-V architecture extension and its applicable to
> virtualization platform only (we can says BaseMemEncryptSevLib is OVMF
> specific).
>> You've asserted that it should work (SEV would not be detected) with any Intel processor as well. Therefore, I don't see a good reason that we shouldn't be able to support it in modules that already have
>> IA32/X64 specific code. (I'm recalling
>> 881813d7a93d9009c873515b043c41c4554779e4.)
>>
>> Since DxeIpl builds the IA32/X64 page tables, and you need to modify the page tables for this feature (correct?), I think we should try to support the feature there if it is feasible. I can understand the argument that this doesn't apply to all non-VM platforms, so I think we could add a PCD which disables this support by default.
>>
>> I don't know that the owners of MdeModulePkg and UefiCpuPkg will agree with me though.
>
> I am flexible to implement APRIORI or Platform hooks Lib. But one thing
> I want to highlight is: I'll prefer clearing C-bit  through
> BaseMemEncryptSevLib functions. One of the main reason for doing so - In
> future when we add migration support for the SEV guest then we will be
> required to notify the unencrypted page range to hypevisor ( through
> hypercall). During migration phase, Hypervisor will use this information
> to make decision on whether to invoke the SEV firmware to encrypt the
> memory region for transport purposes. If clearing C-bit logic is
> contained inside BaseMemEncryptSevLib then it will make life much easier.
>
>>> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to
>>> see if I can invoke a platform dependent library to clear C-bit before
>>> DxeMain finishes its execution. As Laszlo pointed, current approach is
>>> using GCD memory space map to get MMIO and NonExistent entries. I have
>>> pushed two patches in my development branch to show what I have been doing:
>>>
>>> 1) add a new null DxeGcdCorePlatformHookLib
>>>
>>> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94
>>> a920d7ad72d
>>>
>>> The library provides a function "DxeGcdCorePlatformHookReady" which
>>> can be called by DxeMain just after it initializes the GcdServices
>>> (which will guarantee that Gcd memory space map is available).
>> Regarding hooking into DxeCore, I don't think it is the best approach, but it is better than APRIORI. I wonder if the MdeModulePkg owners could jump in with an opinion. (Hopefully besides just pushing the problem away via APRIORI.)
>
> Jiewen, any comments ?
>
>> -Jordan
>>
>>> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
>>>  SEV is detected.
>>>
>>> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba5
>>> 3c95949f659
>>>
>>> I've tested the approach and it seems to work. Is this something
>>> aligned with your thinking?
>>>
>>>
>>>> Thanks
>>>> Laszlo
>>>>
>>>>> -Jordan
>>>>>
>>>>>> In second patch
>>>>>> [2], Leo tried to introduce a new notify protocol to get MMIO
>>>>>> add/remove events. During discussion Jiewen suggested to look into
>>>>>> adding a new platform driver into APRIORI to avoid the need for
>>>>>> any modifications inside the Gcdcore - this seems workable
>>>>>> solution which did not require adding any CPU specific code inside the Gcd.
>>>>>>
>>>>>> [1]
>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>>>> [2]
>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>>>
>>> _______________________________________________
>>> edk2-devel mailing list
>>> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org>>
>>> https://lists.01.org/mailman/listinfo/edk2-devel
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
> https://lists.01.org/mailman/listinfo/edk2-devel
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 06/06/17 17:54, Duran, Leo wrote:
> Thank you all for the feedback… The summary of what I’m hearing is:
> 
> 1)      For now, use APRIORI in the stand-alone (SEV) driver in OvmfPkg (albeit not as clean as desired, but least intrusive solution.).
> 
> 2)      Let’s revisit the existing GCD implementation to allow for cleaner of features like this.

Personally I'm OK with this as well.

Thanks
Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 06/06/17 17:43, Yao, Jiewen wrote:
> Hi Andrew
> Yes, I agree. If we could figure out a cleaner way to resolve the problem, we should use the cleaner way.
> 
> 
> 
> If we really really do not want to use a priori for AmdSec, we can
> let AmdSec to publish a special protocol, and let the driver depend
> that protocol, if this driver need add MMIO region. Because AmdSec is
> inside of OvmfPkg, this special protocol can be in OvmfPkg.
> 
> That is just another option.
We considered this approach before, and it is something that -- for a
change :) -- *I* happen to dislike.

My issue with this is that it doesn't scale. Assume that in a few months
we'd like to pull another driver from MdeModulePkg into the OVMF binary,
and that QEMU implemented emulation for that device. In virtual machines
without SEV, the driver could just work (we might have to set a few PCDs
for it, but that would be all). For making the driver work in SEV guests
however, we'd have to clone the driver to OvmfPkg, and add code to the
driver to explicitly use the special protocol. Not good. :(

Your suggestion is actually a variant of the IOMMU protocol. The IOMMU
protocol is different however, because PCI physical address space and
CPU physical address space are *already* different, and for this mapping
to work, all drivers that use PCI DMA must *already* use the IOMMU
protocol, even if they run on physical hardware, and even if they live
under MdeModulePkg. Therefore if we provide a SEV-specific IOMMU driver
in OvmfPkg, those PCI DMA drivers in MdeModulePkg will automatically
start using it; no change needed.

However, any drivers under MdeModulePkg (current and future) that use
MMIO directly, without going through PCI DMA, would have to be cloned
and modified in OvmfPkg.

Thanks
Laszlo


> 
> Thank you
> Yao Jiewen
> 
> From: afish@apple.com [mailto:afish@apple.com]
> Sent: Tuesday, June 6, 2017 11:25 PM
> To: Yao, Jiewen <jiewen.yao@intel.com>
> Cc: Brijesh Singh <brijesh.singh@amd.com>; Zeng, Star <star.zeng@intel.com>; Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>; edk2-devel@lists.01.org; Dong, Eric <eric.dong@intel.com>; Thomas.Lendacky@amd.com; leo.duran@amd.com; Fan, Jeff <jeff.fan@intel.com>; Gao, Liming <liming.gao@intel.com>
> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
> 
> 
>> On Jun 6, 2017, at 7:54 AM, Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>> wrote:
>>
>> Hi
>> It takes me some time to read all email below. I believe all of us have a clean understanding on what problem we have now and the possible solutions to clear C bit are below
>>
>> 1)       In DxeIpl, when it builds page table.
>>
>> 2)       In DxeCore
>>
>> a)         By use CpuArch
>>
>> b)         By use page table lib
>>
>> c)         By use a GCD update callback
>>
>> d)         By use PlatformHook lib
>>
>> 3)       In a standalone AmdSev driver.
>>
>> Here is my thought:
>> 2.a) is not possible, per Leo’s investigation.
>> 2.b) is not a good design, because we do not introduce any Cpu Specific thing to DxeCore so far.
>> 2.c) and 2.d) are same. I do not suggest we add a private interface to the core just to support one specific feature.
>>
>> 1) is one possible solution, I suggested before. But if Leo/Laszlo think it is too hard to implement, I am OK.
>>
>> If 1) cannot be chosen, I still think 3) is the best idea.
>> It makes the code very clean by introducing a standalone driver to resolve the problem.
>> Zero impact on existing platform.
>> If this feature is not needed, just remove the driver.
>>
>> I do not see any issue on using a priori, because: A) “a priori” is clearly defined in PI spec, B) “a priori” has already been widely used in current platform in EDKII open source, as well as close source platform.
>>
> 
> Jiewen,
> 
> I agree that "a priori" is part of the architecture so it is OK to use it, but "a priori" was never really intended as a way to add basic features. it was more for debugging and work arounds. It seems like a feature like this should not require a work around....
> 
> 'So I think it is OK to accept this patch to get the feature enabled, but we need to look at the GCD implementation and PI architecture to figure out why there is not a cleaner way to add this feature. Maybe we need to change the implementation, and/or the PI Spec?
> 
> Thanks,
> 
> Andrew Fish
> 
>> Thank you
>> Yao Jiewen
>>
>>
>>
>> From: Brijesh Singh [mailto:brijesh.singh@amd.com]
>> Sent: Tuesday, June 6, 2017 11:51 AM
>> To: Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com>>; Justen, Jordan L <jordan.l.justen@intel.com<mailto:jordan.l.justen@intel.com>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com>>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>
>> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; leo.duran@amd.com<mailto:leo.duran@amd.com>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com>>
>> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>>
>> Hi Jordan,
>>
>>
>> On 6/5/17 9:08 PM, Zeng, Star wrote:
>>> I was not tracking this thread.
>>> Jiewen will help give comments about the potential change in MdeModulePkg.
>>>
>>> Thanks,
>>> Star
>>> -----Original Message-----
>>> From: Justen, Jordan L
>>> Sent: Tuesday, June 6, 2017 9:12 AM
>>> To: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com<mailto:brijesh.singh@amd.com%3cmailto:brijesh.singh@amd.com>>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com<mailto:lersek@redhat.com%3cmailto:lersek@redhat.com>>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org>>; Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com<mailto:star.zeng@intel.com%3cmailto:star.zeng@intel.com>>>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com<mailto:eric.dong@intel.com%3cmailto:eric.dong@intel.com>>>
>>> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com%3cmailto:Thomas.Lendacky@amd.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com<mailto:liming.gao@intel.com%3cmailto:liming.gao@intel.com>>>; leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com<mailto:jiewen.yao@intel.com%3cmailto:jiewen.yao@intel.com>>>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com<mailto:jeff.fan@intel.com%3cmailto:jeff.fan@intel.com>>>
>>> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>>>
>>> On 2017-06-05 14:56:04, Brijesh Singh wrote:
>>>> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
>>>>> On 06/01/17 09:40, Jordan Justen wrote:
>>>>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>>>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>>>>> I don't understand this, because isn't DxeIpl and an early APRIORI
>>>>>> entry are roughly equivalent in the boot sequence?
>>>>> I think you are right. I believe a patch for this exact idea hasn't
>>>>> been posted yet. Jiewen's message that you linked above contains the
>>>>> expression
>>>>>
>>>>>     always clear SEV mask for MMIO *and all rest*
>>>>>
>>>>> (emphasis mine), which I think we may have missed *in combination
>>>>> with* the DxeIpl.
>>>>>
>>>>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>>>>> Keep the C bit set for system memory regions. Clear the C bit for
>>>>> MMIO regions that are known from the HOB list. Also clear the C bit
>>>>> everywhere else in the address space (known from the CPU HOB) where
>>>>> no coverage is provided by any memory resource descriptor HOB.
>>>>>
>>>>> This is going to be harder than the current approach, because:
>>>>>
>>>>> - The current approach can work off of the GCD memory space map,
>>>>> which provides explicit NonExistent entries, covering the entire
>>>>> address space (according to the CPU HOB).
>>>>>
>>>>> - However, the DxeIpl method would take place before entering DXE,
>>>>> so no GCD memory space map would be available -- the "NonExistent"
>>>>> entries would have to be synthesized manually from the address space
>>>>> size (known from the CPU HOB) and the lack of coverage by memory
>>>>> resource descriptor HOBs.
>>>>>
>>>>> Basically, in order to move the current GCD memory space map
>>>>> traversal from early DXE to late PEI, the memory space map building
>>>>> logic of the DXE Core would have to be duplicated in the DxeIpl
>>>>> PEIM. If I understand correctly. (The DxeIpl PEIM may already
>>>>> contain very similar code, for the page table building, which might
>>>>> not be difficult to extend like this -- I haven't looked.)
>>>>>
>>>>> Is this what you have in mind?
>>>>>
>>>> Do you have any further thought on this?
>>> Regarding Laszlo's feedback, I'm not convinced that it would be excessively difficult to accomplish this in DxeIpl. (I'm not saying that I couldn't be convinced. :)
>>>
>>> As far as I can see, this is an architecturally defined AMD feature.
>>> (Is this true, or is BaseMemcryptSevLib actually OVMF specific?)
>>
>> Yes, SEV is AMD-V architecture extension and its applicable to
>> virtualization platform only (we can says BaseMemEncryptSevLib is OVMF
>> specific).
>>> You've asserted that it should work (SEV would not be detected) with any Intel processor as well. Therefore, I don't see a good reason that we shouldn't be able to support it in modules that already have
>>> IA32/X64 specific code. (I'm recalling
>>> 881813d7a93d9009c873515b043c41c4554779e4.)
>>>
>>> Since DxeIpl builds the IA32/X64 page tables, and you need to modify the page tables for this feature (correct?), I think we should try to support the feature there if it is feasible. I can understand the argument that this doesn't apply to all non-VM platforms, so I think we could add a PCD which disables this support by default.
>>>
>>> I don't know that the owners of MdeModulePkg and UefiCpuPkg will agree with me though.
>>
>> I am flexible to implement APRIORI or Platform hooks Lib. But one thing
>> I want to highlight is: I'll prefer clearing C-bit  through
>> BaseMemEncryptSevLib functions. One of the main reason for doing so - In
>> future when we add migration support for the SEV guest then we will be
>> required to notify the unencrypted page range to hypevisor ( through
>> hypercall). During migration phase, Hypervisor will use this information
>> to make decision on whether to invoke the SEV firmware to encrypt the
>> memory region for transport purposes. If clearing C-bit logic is
>> contained inside BaseMemEncryptSevLib then it will make life much easier.
>>
>>>> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to
>>>> see if I can invoke a platform dependent library to clear C-bit before
>>>> DxeMain finishes its execution. As Laszlo pointed, current approach is
>>>> using GCD memory space map to get MMIO and NonExistent entries. I have
>>>> pushed two patches in my development branch to show what I have been doing:
>>>>
>>>> 1) add a new null DxeGcdCorePlatformHookLib
>>>>
>>>> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94
>>>> a920d7ad72d
>>>>
>>>> The library provides a function "DxeGcdCorePlatformHookReady" which
>>>> can be called by DxeMain just after it initializes the GcdServices
>>>> (which will guarantee that Gcd memory space map is available).
>>> Regarding hooking into DxeCore, I don't think it is the best approach, but it is better than APRIORI. I wonder if the MdeModulePkg owners could jump in with an opinion. (Hopefully besides just pushing the problem away via APRIORI.)
>>
>> Jiewen, any comments ?
>>
>>> -Jordan
>>>
>>>> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
>>>>  SEV is detected.
>>>>
>>>> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba5
>>>> 3c95949f659
>>>>
>>>> I've tested the approach and it seems to work. Is this something
>>>> aligned with your thinking?
>>>>
>>>>
>>>>> Thanks
>>>>> Laszlo
>>>>>
>>>>>> -Jordan
>>>>>>
>>>>>>> In second patch
>>>>>>> [2], Leo tried to introduce a new notify protocol to get MMIO
>>>>>>> add/remove events. During discussion Jiewen suggested to look into
>>>>>>> adding a new platform driver into APRIORI to avoid the need for
>>>>>>> any modifications inside the Gcdcore - this seems workable
>>>>>>> solution which did not require adding any CPU specific code inside the Gcd.
>>>>>>>
>>>>>>> [1]
>>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>>>>> [2]
>>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>>>>
>>>> _______________________________________________
>>>> edk2-devel mailing list
>>>> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org>>
>>>> https://lists.01.org/mailman/listinfo/edk2-devel
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
>> https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

On 06/06/17 16:54, Yao, Jiewen wrote:
> Hi
> It takes me some time to read all email below. I believe all of us have a clean understanding on what problem we have now and the possible solutions to clear C bit are below
> 
> 1)       In DxeIpl, when it builds page table.
> 
> 2)       In DxeCore
> 
> a)         By use CpuArch
> 
> b)         By use page table lib
> 
> c)         By use a GCD update callback
> 
> d)         By use PlatformHook lib
> 
> 3)       In a standalone AmdSev driver.
> 
> Here is my thought:
> 2.a) is not possible, per Leo’s investigation.
> 2.b) is not a good design, because we do not introduce any Cpu Specific thing to DxeCore so far.
> 2.c) and 2.d) are same. I do not suggest we add a private interface to the core just to support one specific feature.
> 
> 1) is one possible solution, I suggested before. But if Leo/Laszlo think it is too hard to implement, I am OK.

I think that implementing the logic in the DXE IPL PEIM will be more
difficult than doing the same somewhere in the DXE phase. But, I
wouldn't oppose the idea. The implementation difficulty is not for me to
overcome: it affects Brijesh and Leo (as the implementors) and the DXE
IPL PEIM maintainers (in review).

NB, Brijesh mentioned up-thread that wherever (= in whichever firmware
phase) they massaged the C-bit, they'd like to use BaseMemEncryptSevLib
for it, because that would help with live migrating VMs. I can't say
off-hand whether the current library instance is appropriate for use in
the DXE IPL PEIM -- for example: repeated memory allocation and freeing
isn't very good for PEIMs, since no freeing occurs in PEI --, so that's
something for Brijesh & Leo to evaluate.

Anyhow, if the DXE IPL PEIM approach can be made work, technically, then
I'm 100% fine with it personally.

Thanks
Laszlo

> 
> If 1) cannot be chosen, I still think 3) is the best idea.
> It makes the code very clean by introducing a standalone driver to resolve the problem.
> Zero impact on existing platform.
> If this feature is not needed, just remove the driver.
> 
> I do not see any issue on using a priori, because: A) “a priori” is clearly defined in PI spec, B) “a priori” has already been widely used in current platform in EDKII open source, as well as close source platform.
> 
> Thank you
> Yao Jiewen
> 
> 
> 
> From: Brijesh Singh [mailto:brijesh.singh@amd.com]
> Sent: Tuesday, June 6, 2017 11:51 AM
> To: Zeng, Star <star.zeng@intel.com>; Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>; edk2-devel@lists.01.org; Dong, Eric <eric.dong@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>
> Cc: Thomas.Lendacky@amd.com; Gao, Liming <liming.gao@intel.com>; leo.duran@amd.com; Fan, Jeff <jeff.fan@intel.com>
> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
> 
> Hi Jordan,
> 
> 
> On 6/5/17 9:08 PM, Zeng, Star wrote:
>> I was not tracking this thread.
>> Jiewen will help give comments about the potential change in MdeModulePkg.
>>
>> Thanks,
>> Star
>> -----Original Message-----
>> From: Justen, Jordan L
>> Sent: Tuesday, June 6, 2017 9:12 AM
>> To: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Zeng, Star <star.zeng@intel.com<mailto:star.zeng@intel.com>>; Dong, Eric <eric.dong@intel.com<mailto:eric.dong@intel.com>>
>> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; leo.duran@amd.com<mailto:leo.duran@amd.com>; Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>; Fan, Jeff <jeff.fan@intel.com<mailto:jeff.fan@intel.com>>
>> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization (AMD)
>>
>> On 2017-06-05 14:56:04, Brijesh Singh wrote:
>>> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
>>>> On 06/01/17 09:40, Jordan Justen wrote:
>>>>> In https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
>>>>> Leo said that DxeIpl won't work because new I/O ranges might be added.
>>>>> I don't understand this, because isn't DxeIpl and an early APRIORI
>>>>> entry are roughly equivalent in the boot sequence?
>>>> I think you are right. I believe a patch for this exact idea hasn't
>>>> been posted yet. Jiewen's message that you linked above contains the
>>>> expression
>>>>
>>>>      always clear SEV mask for MMIO *and all rest*
>>>>
>>>> (emphasis mine), which I think we may have missed *in combination
>>>> with* the DxeIpl.
>>>>
>>>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
>>>> Keep the C bit set for system memory regions. Clear the C bit for
>>>> MMIO regions that are known from the HOB list. Also clear the C bit
>>>> everywhere else in the address space (known from the CPU HOB) where
>>>> no coverage is provided by any memory resource descriptor HOB.
>>>>
>>>> This is going to be harder than the current approach, because:
>>>>
>>>> - The current approach can work off of the GCD memory space map,
>>>> which provides explicit NonExistent entries, covering the entire
>>>> address space (according to the CPU HOB).
>>>>
>>>> - However, the DxeIpl method would take place before entering DXE,
>>>> so no GCD memory space map would be available -- the "NonExistent"
>>>> entries would have to be synthesized manually from the address space
>>>> size (known from the CPU HOB) and the lack of coverage by memory
>>>> resource descriptor HOBs.
>>>>
>>>> Basically, in order to move the current GCD memory space map
>>>> traversal from early DXE to late PEI, the memory space map building
>>>> logic of the DXE Core would have to be duplicated in the DxeIpl
>>>> PEIM. If I understand correctly. (The DxeIpl PEIM may already
>>>> contain very similar code, for the page table building, which might
>>>> not be difficult to extend like this -- I haven't looked.)
>>>>
>>>> Is this what you have in mind?
>>>>
>>> Do you have any further thought on this?
>> Regarding Laszlo's feedback, I'm not convinced that it would be excessively difficult to accomplish this in DxeIpl. (I'm not saying that I couldn't be convinced. :)
>>
>> As far as I can see, this is an architecturally defined AMD feature.
>> (Is this true, or is BaseMemcryptSevLib actually OVMF specific?)
> 
> Yes, SEV is AMD-V architecture extension and its applicable to
> virtualization platform only (we can says BaseMemEncryptSevLib is OVMF
> specific).
>> You've asserted that it should work (SEV would not be detected) with any Intel processor as well. Therefore, I don't see a good reason that we shouldn't be able to support it in modules that already have
>> IA32/X64 specific code. (I'm recalling
>> 881813d7a93d9009c873515b043c41c4554779e4.)
>>
>> Since DxeIpl builds the IA32/X64 page tables, and you need to modify the page tables for this feature (correct?), I think we should try to support the feature there if it is feasible. I can understand the argument that this doesn't apply to all non-VM platforms, so I think we could add a PCD which disables this support by default.
>>
>> I don't know that the owners of MdeModulePkg and UefiCpuPkg will agree with me though.
> 
> I am flexible to implement APRIORI or Platform hooks Lib. But one thing
> I want to highlight is: I'll prefer clearing C-bit  through
> BaseMemEncryptSevLib functions. One of the main reason for doing so - In
> future when we add migration support for the SEV guest then we will be
> required to notify the unencrypted page range to hypevisor ( through
> hypercall). During migration phase, Hypervisor will use this information
> to make decision on whether to invoke the SEV firmware to encrypt the
> memory region for transport purposes. If clearing C-bit logic is
> contained inside BaseMemEncryptSevLib then it will make life much easier.
> 
>>> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain to
>>> see if I can invoke a platform dependent library to clear C-bit before
>>> DxeMain finishes its execution. As Laszlo pointed, current approach is
>>> using GCD memory space map to get MMIO and NonExistent entries. I have
>>> pushed two patches in my development branch to show what I have been doing:
>>>
>>> 1) add a new null DxeGcdCorePlatformHookLib
>>>
>>> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271a94
>>> a920d7ad72d
>>>
>>> The library provides a function "DxeGcdCorePlatformHookReady" which
>>> can be called by DxeMain just after it initializes the GcdServices
>>> (which will guarantee that Gcd memory space map is available).
>> Regarding hooking into DxeCore, I don't think it is the best approach, but it is better than APRIORI. I wonder if the MdeModulePkg owners could jump in with an opinion. (Hopefully besides just pushing the problem away via APRIORI.)
> 
> Jiewen, any comments ?
> 
>> -Jordan
>>
>>> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-bit when
>>>   SEV is detected.
>>>
>>> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596ba5
>>> 3c95949f659
>>>
>>> I've tested the approach and it seems to work. Is this something
>>> aligned with your thinking?
>>>
>>>
>>>> Thanks
>>>> Laszlo
>>>>
>>>>> -Jordan
>>>>>
>>>>>> In second patch
>>>>>> [2], Leo tried to introduce a new notify protocol to get MMIO
>>>>>> add/remove events. During discussion Jiewen suggested to look into
>>>>>> adding a new platform driver into APRIORI to avoid the need for
>>>>>> any modifications inside the Gcdcore - this seems workable
>>>>>> solution which did not require adding any CPU specific code inside the Gcd.
>>>>>>
>>>>>> [1]
>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
>>>>>> [2]
>>>>>> https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
>>>>>>
>>> _______________________________________________
>>> edk2-devel mailing list
>>> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
>>> https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Lazlo, et al,

Calling back from DxeIpl after page tables are created seems like a major hack:
- Duplicates logic already in GCD
- Would need to  pass the page-table root pointer

So, as stated earlier, I'm favor of:  using APRIORI in the interim, and reviewing GCD to support a more correct framework.
Leo.

> -----Original Message-----
> From: Laszlo Ersek [mailto:lersek@redhat.com]
> Sent: Tuesday, June 06, 2017 1:30 PM
> To: Yao, Jiewen <jiewen.yao@intel.com>; Singh, Brijesh
> <brijesh.singh@amd.com>; Zeng, Star <star.zeng@intel.com>; Justen,
> Jordan L <jordan.l.justen@intel.com>; edk2-devel@lists.01.org; Dong, Eric
> <eric.dong@intel.com>
> Cc: Lendacky, Thomas <Thomas.Lendacky@amd.com>; Gao, Liming
> <liming.gao@intel.com>; Duran, Leo <leo.duran@amd.com>; Fan, Jeff
> <jeff.fan@intel.com>
> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted Virtualization
> (AMD)
> 
> On 06/06/17 16:54, Yao, Jiewen wrote:
> > Hi
> > It takes me some time to read all email below. I believe all of us
> > have a clean understanding on what problem we have now and the
> > possible solutions to clear C bit are below
> >
> > 1)       In DxeIpl, when it builds page table.
> >
> > 2)       In DxeCore
> >
> > a)         By use CpuArch
> >
> > b)         By use page table lib
> >
> > c)         By use a GCD update callback
> >
> > d)         By use PlatformHook lib
> >
> > 3)       In a standalone AmdSev driver.
> >
> > Here is my thought:
> > 2.a) is not possible, per Leo’s investigation.
> > 2.b) is not a good design, because we do not introduce any Cpu Specific
> thing to DxeCore so far.
> > 2.c) and 2.d) are same. I do not suggest we add a private interface to the
> core just to support one specific feature.
> >
> > 1) is one possible solution, I suggested before. But if Leo/Laszlo think it is
> too hard to implement, I am OK.
> 
> I think that implementing the logic in the DXE IPL PEIM will be more difficult
> than doing the same somewhere in the DXE phase. But, I wouldn't oppose
> the idea. The implementation difficulty is not for me to
> overcome: it affects Brijesh and Leo (as the implementors) and the DXE IPL
> PEIM maintainers (in review).
> 
> NB, Brijesh mentioned up-thread that wherever (= in whichever firmware
> phase) they massaged the C-bit, they'd like to use BaseMemEncryptSevLib
> for it, because that would help with live migrating VMs. I can't say off-hand
> whether the current library instance is appropriate for use in the DXE IPL
> PEIM -- for example: repeated memory allocation and freeing isn't very good
> for PEIMs, since no freeing occurs in PEI --, so that's something for Brijesh &
> Leo to evaluate.
> 
> Anyhow, if the DXE IPL PEIM approach can be made work, technically, then
> I'm 100% fine with it personally.
> 
> Thanks
> Laszlo
> 
> >
> > If 1) cannot be chosen, I still think 3) is the best idea.
> > It makes the code very clean by introducing a standalone driver to resolve
> the problem.
> > Zero impact on existing platform.
> > If this feature is not needed, just remove the driver.
> >
> > I do not see any issue on using a priori, because: A) “a priori” is clearly
> defined in PI spec, B) “a priori” has already been widely used in current
> platform in EDKII open source, as well as close source platform.
> >
> > Thank you
> > Yao Jiewen
> >
> >
> >
> > From: Brijesh Singh [mailto:brijesh.singh@amd.com]
> > Sent: Tuesday, June 6, 2017 11:51 AM
> > To: Zeng, Star <star.zeng@intel.com>; Justen, Jordan L
> > <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>;
> > edk2-devel@lists.01.org; Dong, Eric <eric.dong@intel.com>; Yao, Jiewen
> > <jiewen.yao@intel.com>
> > Cc: Thomas.Lendacky@amd.com; Gao, Liming <liming.gao@intel.com>;
> > leo.duran@amd.com; Fan, Jeff <jeff.fan@intel.com>
> > Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted
> > Virtualization (AMD)
> >
> > Hi Jordan,
> >
> >
> > On 6/5/17 9:08 PM, Zeng, Star wrote:
> >> I was not tracking this thread.
> >> Jiewen will help give comments about the potential change in
> MdeModulePkg.
> >>
> >> Thanks,
> >> Star
> >> -----Original Message-----
> >> From: Justen, Jordan L
> >> Sent: Tuesday, June 6, 2017 9:12 AM
> >> To: Brijesh Singh
> >> <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>; Laszlo Ersek
> >> <lersek@redhat.com<mailto:lersek@redhat.com>>;
> >> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Zeng, Star
> >> <star.zeng@intel.com<mailto:star.zeng@intel.com>>; Dong, Eric
> >> <eric.dong@intel.com<mailto:eric.dong@intel.com>>
> >> Cc: Thomas.Lendacky@amd.com<mailto:Thomas.Lendacky@amd.com>;
> Gao,
> >> Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>;
> >> leo.duran@amd.com<mailto:leo.duran@amd.com>; Yao, Jiewen
> >> <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>; Fan, Jeff
> >> <jeff.fan@intel.com<mailto:jeff.fan@intel.com>>
> >> Subject: Re: [edk2] [PATCH v6 00/17] x86: Secure Encrypted
> >> Virtualization (AMD)
> >>
> >> On 2017-06-05 14:56:04, Brijesh Singh wrote:
> >>> On 06/01/2017 04:10 AM, Laszlo Ersek wrote:
> >>>> On 06/01/17 09:40, Jordan Justen wrote:
> >>>>> In
> >>>>> https://lists.01.org/pipermail/edk2-devel/2017-April/009883.html
> >>>>> Leo said that DxeIpl won't work because new I/O ranges might be
> added.
> >>>>> I don't understand this, because isn't DxeIpl and an early APRIORI
> >>>>> entry are roughly equivalent in the boot sequence?
> >>>> I think you are right. I believe a patch for this exact idea hasn't
> >>>> been posted yet. Jiewen's message that you linked above contains
> >>>> the expression
> >>>>
> >>>>      always clear SEV mask for MMIO *and all rest*
> >>>>
> >>>> (emphasis mine), which I think we may have missed *in combination
> >>>> with* the DxeIpl.
> >>>>
> >>>> So the idea would be to iterate over all the HOBs in the DxeIpl PEIM.
> >>>> Keep the C bit set for system memory regions. Clear the C bit for
> >>>> MMIO regions that are known from the HOB list. Also clear the C bit
> >>>> everywhere else in the address space (known from the CPU HOB)
> where
> >>>> no coverage is provided by any memory resource descriptor HOB.
> >>>>
> >>>> This is going to be harder than the current approach, because:
> >>>>
> >>>> - The current approach can work off of the GCD memory space map,
> >>>> which provides explicit NonExistent entries, covering the entire
> >>>> address space (according to the CPU HOB).
> >>>>
> >>>> - However, the DxeIpl method would take place before entering DXE,
> >>>> so no GCD memory space map would be available -- the "NonExistent"
> >>>> entries would have to be synthesized manually from the address
> >>>> space size (known from the CPU HOB) and the lack of coverage by
> >>>> memory resource descriptor HOBs.
> >>>>
> >>>> Basically, in order to move the current GCD memory space map
> >>>> traversal from early DXE to late PEI, the memory space map building
> >>>> logic of the DXE Core would have to be duplicated in the DxeIpl
> >>>> PEIM. If I understand correctly. (The DxeIpl PEIM may already
> >>>> contain very similar code, for the page table building, which might
> >>>> not be difficult to extend like this -- I haven't looked.)
> >>>>
> >>>> Is this what you have in mind?
> >>>>
> >>> Do you have any further thought on this?
> >> Regarding Laszlo's feedback, I'm not convinced that it would be
> >> excessively difficult to accomplish this in DxeIpl. (I'm not saying
> >> that I couldn't be convinced. :)
> >>
> >> As far as I can see, this is an architecturally defined AMD feature.
> >> (Is this true, or is BaseMemcryptSevLib actually OVMF specific?)
> >
> > Yes, SEV is AMD-V architecture extension and its applicable to
> > virtualization platform only (we can says BaseMemEncryptSevLib is OVMF
> > specific).
> >> You've asserted that it should work (SEV would not be detected) with
> >> any Intel processor as well. Therefore, I don't see a good reason
> >> that we shouldn't be able to support it in modules that already have
> >> IA32/X64 specific code. (I'm recalling
> >> 881813d7a93d9009c873515b043c41c4554779e4.)
> >>
> >> Since DxeIpl builds the IA32/X64 page tables, and you need to modify the
> page tables for this feature (correct?), I think we should try to support the
> feature there if it is feasible. I can understand the argument that this doesn't
> apply to all non-VM platforms, so I think we could add a PCD which disables
> this support by default.
> >>
> >> I don't know that the owners of MdeModulePkg and UefiCpuPkg will
> agree with me though.
> >
> > I am flexible to implement APRIORI or Platform hooks Lib. But one
> > thing I want to highlight is: I'll prefer clearing C-bit  through
> > BaseMemEncryptSevLib functions. One of the main reason for doing so -
> > In future when we add migration support for the SEV guest then we will
> > be required to notify the unencrypted page range to hypevisor (
> > through hypercall). During migration phase, Hypervisor will use this
> > information to make decision on whether to invoke the SEV firmware to
> > encrypt the memory region for transport purposes. If clearing C-bit
> > logic is contained inside BaseMemEncryptSevLib then it will make life much
> easier.
> >
> >>> In meantime, I have been looking into MdeModule/Core/Dxe/DxeMain
> to
> >>> see if I can invoke a platform dependent library to clear C-bit
> >>> before DxeMain finishes its execution. As Laszlo pointed, current
> >>> approach is using GCD memory space map to get MMIO and NonExistent
> >>> entries. I have pushed two patches in my development branch to show
> what I have been doing:
> >>>
> >>> 1) add a new null DxeGcdCorePlatformHookLib
> >>>
> >>>
> https://github.com/codomania/edk2/commit/171f816376b3b0677cbfb90271
> a
> >>> 94
> >>> a920d7ad72d
> >>>
> >>> The library provides a function "DxeGcdCorePlatformHookReady" which
> >>> can be called by DxeMain just after it initializes the GcdServices
> >>> (which will guarantee that Gcd memory space map is available).
> >> Regarding hooking into DxeCore, I don't think it is the best
> >> approach, but it is better than APRIORI. I wonder if the MdeModulePkg
> >> owners could jump in with an opinion. (Hopefully besides just pushing
> >> the problem away via APRIORI.)
> >
> > Jiewen, any comments ?
> >
> >> -Jordan
> >>
> >>> 2) override DxeGcdCorePlatformHookLib inside the Ovmf to clear the C-
> bit when
> >>>   SEV is detected.
> >>>
> >>>
> https://github.com/codomania/edk2/commit/914ce904ca1b7647c966562596
> b
> >>> a5
> >>> 3c95949f659
> >>>
> >>> I've tested the approach and it seems to work. Is this something
> >>> aligned with your thinking?
> >>>
> >>>
> >>>> Thanks
> >>>> Laszlo
> >>>>
> >>>>> -Jordan
> >>>>>
> >>>>>> In second patch
> >>>>>> [2], Leo tried to introduce a new notify protocol to get MMIO
> >>>>>> add/remove events. During discussion Jiewen suggested to look
> >>>>>> into adding a new platform driver into APRIORI to avoid the need
> >>>>>> for any modifications inside the Gcdcore - this seems workable
> >>>>>> solution which did not require adding any CPU specific code inside
> the Gcd.
> >>>>>>
> >>>>>> [1]
> >>>>>> https://lists.01.org/pipermail/edk2-devel/2017-March/008974.html
> >>>>>> [2]
> >>>>>> https://lists.01.org/pipermail/edk2-devel/2017-April/009852.html
> >>>>>>
> >>> _______________________________________________
> >>> edk2-devel mailing list
> >>> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
> >>> https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel