[edk2] [Patch] Build spec: add description for build with binary cache

Yonghong Zhu posted 1 patch 7 years, 3 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
.../82_auto-generation_process.md                    | 20 ++++++++++++++++++++
README.md                                            |  1 +
appendix_d_buildexe_command/d4_usage.md              | 19 +++++++++++++++----
3 files changed, 36 insertions(+), 4 deletions(-)
[edk2] [Patch] Build spec: add description for build with binary cache
Posted by Yonghong Zhu 7 years, 3 months ago
fixes:https://bugzilla.tianocore.org/show_bug.cgi?id=689
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Kevin W Shaw <kevin.w.shaw@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
---
 .../82_auto-generation_process.md                    | 20 ++++++++++++++++++++
 README.md                                            |  1 +
 appendix_d_buildexe_command/d4_usage.md              | 19 +++++++++++++++----
 3 files changed, 36 insertions(+), 4 deletions(-)

diff --git a/8_pre-build_autogen_stage/82_auto-generation_process.md b/8_pre-build_autogen_stage/82_auto-generation_process.md
index 671a7d5..f1d7158 100644
--- a/8_pre-build_autogen_stage/82_auto-generation_process.md
+++ b/8_pre-build_autogen_stage/82_auto-generation_process.md
@@ -1031,10 +1031,30 @@ maximum command line length.  The default value is 4096.
 **Note:** The following `FLAGS` options are included in the response file:
 `PP_FLAGS`, `CC_FLAGS`, `VFRPP_FLAGS`, `APP_FLAGS`, `ASLPP_FLAGS`, `ASLCC_FLAGS`,
 and `ASM_FLAGS`.
 **********
 
+#### 8.2.4.15 Build with Binary Cache
+
+**build** tool provides three new options for binary cache feature.
+--hash enables hash-based caching during build process. when --hash is enabled,
+build tool will base on the module hash value to do the incremental build, without
+--hash, build tool will base on the timestamp to do the incremental build. --hash
+option use md5 method to get every hash value, DSC/FDF, tools_def.txt, build_rule.txt
+and build command are calculated as global hash value, Package DEC and its include
+header files are calculated as package hash value, Module source files and its INF
+file are calculated as module hash value. Library hash value will combine the global
+hash value and its dependent package hash value. Driver hash value will combine the
+global hash value, its dependent package hash value and its linked library hash value.
+When --hash and --bindest are specified, build tool will copy the generated binary 
+files for each module into the directory specified by bindest at the build phase.
+Bindest caches all generated binary files.
+When --hash and --binsource are specified, build tool will try to get the binary
+files from the binary source directory at the build phase. If the cached binary has
+the same hash value, it will be directly used. Otherwise, build tool will compile the
+source files and generate the binary files.
+
 ### 8.2.5 Post processing
 
 Once all files are parsed, the build tools will do following work for each EDK
 II module:
 
diff --git a/README.md b/README.md
index 52abb6a..ca59a35 100644
--- a/README.md
+++ b/README.md
@@ -215,5 +215,6 @@ Copyright (c) 2008-2017, Intel Corporation. All rights reserved.
 |            | [#523](https://bugzilla.tianocore.org/show_bug.cgi?id=523) Build spec: add EBNF for the --pcd syntax in the Section D.4                                                                                                                                                                                                                                             |               |
 |            | [#517](https://bugzilla.tianocore.org/show_bug.cgi?id=517) Build spec: chapter 5.2.2 Guided Tools add description for Pkcs7Sign tool and BrotliCompress tool                                                                                                                                                                                                        |               |
 |            | [#481](https://bugzilla.tianocore.org/show_bug.cgi?id=481) Build Spec: add clarification for not used Pcd that build tool will not do additional checks on its value                                                                                                                                                                                                |               |
 |            | [#518](https://bugzilla.tianocore.org/show_bug.cgi?id=518) Build Spec: Update Precedence of PCD Values                                                                                                                                                                                                                                                              |               |
 |            | [#669](https://bugzilla.tianocore.org/show_bug.cgi?id=669) Build Spec: Add multi-arg support to PREBUILD/POSTBUILD                                                                                                                                                                                                                                                  |               |
+|            | [#689](https://bugzilla.tianocore.org/show_bug.cgi?id=689) Build spec: add description for build with binary cache                                                                                                                                                                                                                                                  |               |
diff --git a/appendix_d_buildexe_command/d4_usage.md b/appendix_d_buildexe_command/d4_usage.md
index b71f2d0..6a91c3a 100644
--- a/appendix_d_buildexe_command/d4_usage.md
+++ b/appendix_d_buildexe_command/d4_usage.md
@@ -32,19 +32,20 @@
 ## D.4 Usage
 
 ```ini
 Usage: build.exe [options]
 [all|fds|genc|genmake|clean|cleanall|cleanlib|modules|libraries|run]
-Copyright (c) 2007 - 2014, Intel Corporation All rights reserved.
+Copyright (c) 2007 - 2017, Intel Corporation All rights reserved.
 
 Options:
   --version             show program's version number and exit
   -h, --help            show this help message and exit
   -a TARGETARCH, --arch=TARGETARCH
-                        ARCHS is one of list: IA32, X64, IPF, ARM, or EBC,
-                        which overrides target.txt's TARGET_ARCH definition.
-                        To specify more archs, please repeat this option.
+                        ARCHS is one of list: IA32, X64, IPF, ARM, AARCH64 or
+                        EBC, which overrides target.txt's TARGET_ARCH
+                        definition. To specify more archs, please repeat this
+                        option.
   -p PLATFORMFILE, --platform=PLATFORMFILE
                         Build the platform specified by the DSC file name
                         argument, overriding target.txt's ACTIVE_PLATFORM
                         definition.
   -m MODULEFILE, --module=MODULEFILE
@@ -112,10 +113,20 @@ Options:
   -N, --no-cache        Disable build cache mechanism
   --conf=CONFDIRECTORY  Specify the customized Conf directory.
   --check-usage         Check usage content of entries listed in INF file.
   --ignore-sources      Focus to a binary build and ignore all source files
   --pcd=OPTIONPCD       Set PCD value by command line. Format: "PcdName=Value"
+  -l COMMANDLENGTH, --cmd-len=COMMANDLENGTH
+                        Specify the maximum line length of build command.
+                        Default is 4096.
+  --hash                Enable hash-based caching during build process.
+  --bindest=BINCACHEDEST
+                        Generate a cache of binary files in the specified
+                        directory.
+  --binsource=BINCACHESOURCE
+                        Consume a cache of binary files from the specified
+                        directory.
 ```
 
 ### D.4.1 Debug Levels
 
 The numeric debug levels are defined as integer values 0-9.
-- 
2.6.1.windows.1

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [Patch] Build spec: add description for build with binary cache
Posted by Gao, Liming 7 years, 3 months ago
Reviewed-by: Liming Gao <liming.gao@intel.com>

>-----Original Message-----
>From: Zhu, Yonghong
>Sent: Monday, September 04, 2017 4:43 PM
>To: edk2-devel@lists.01.org
>Cc: Gao, Liming <liming.gao@intel.com>; Kinney, Michael D
><michael.d.kinney@intel.com>; Shaw, Kevin W <kevin.w.shaw@intel.com>
>Subject: [Patch] Build spec: add description for build with binary cache
>
>fixes:https://bugzilla.tianocore.org/show_bug.cgi?id=689
>Cc: Liming Gao <liming.gao@intel.com>
>Cc: Michael Kinney <michael.d.kinney@intel.com>
>Cc: Kevin W Shaw <kevin.w.shaw@intel.com>
>Contributed-under: TianoCore Contribution Agreement 1.1
>Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
>---
> .../82_auto-generation_process.md                    | 20 ++++++++++++++++++++
> README.md                                            |  1 +
> appendix_d_buildexe_command/d4_usage.md              | 19
>+++++++++++++++----
> 3 files changed, 36 insertions(+), 4 deletions(-)
>
>diff --git a/8_pre-build_autogen_stage/82_auto-generation_process.md
>b/8_pre-build_autogen_stage/82_auto-generation_process.md
>index 671a7d5..f1d7158 100644
>--- a/8_pre-build_autogen_stage/82_auto-generation_process.md
>+++ b/8_pre-build_autogen_stage/82_auto-generation_process.md
>@@ -1031,10 +1031,30 @@ maximum command line length.  The default
>value is 4096.
> **Note:** The following `FLAGS` options are included in the response file:
> `PP_FLAGS`, `CC_FLAGS`, `VFRPP_FLAGS`, `APP_FLAGS`, `ASLPP_FLAGS`,
>`ASLCC_FLAGS`,
> and `ASM_FLAGS`.
> **********
>
>+#### 8.2.4.15 Build with Binary Cache
>+
>+**build** tool provides three new options for binary cache feature.
>+--hash enables hash-based caching during build process. when --hash is
>enabled,
>+build tool will base on the module hash value to do the incremental build,
>without
>+--hash, build tool will base on the timestamp to do the incremental build. --
>hash
>+option use md5 method to get every hash value, DSC/FDF, tools_def.txt,
>build_rule.txt
>+and build command are calculated as global hash value, Package DEC and its
>include
>+header files are calculated as package hash value, Module source files and its
>INF
>+file are calculated as module hash value. Library hash value will combine the
>global
>+hash value and its dependent package hash value. Driver hash value will
>combine the
>+global hash value, its dependent package hash value and its linked library
>hash value.
>+When --hash and --bindest are specified, build tool will copy the generated
>binary
>+files for each module into the directory specified by bindest at the build
>phase.
>+Bindest caches all generated binary files.
>+When --hash and --binsource are specified, build tool will try to get the
>binary
>+files from the binary source directory at the build phase. If the cached binary
>has
>+the same hash value, it will be directly used. Otherwise, build tool will
>compile the
>+source files and generate the binary files.
>+
> ### 8.2.5 Post processing
>
> Once all files are parsed, the build tools will do following work for each EDK
> II module:
>
>diff --git a/README.md b/README.md
>index 52abb6a..ca59a35 100644
>--- a/README.md
>+++ b/README.md
>@@ -215,5 +215,6 @@ Copyright (c) 2008-2017, Intel Corporation. All rights
>reserved.
> |            | [#523](https://bugzilla.tianocore.org/show_bug.cgi?id=523) Build
>spec: add EBNF for the --pcd syntax in the Section D.4
>|               |
> |            | [#517](https://bugzilla.tianocore.org/show_bug.cgi?id=517) Build
>spec: chapter 5.2.2 Guided Tools add description for Pkcs7Sign tool and
>BrotliCompress tool
>|               |
> |            | [#481](https://bugzilla.tianocore.org/show_bug.cgi?id=481) Build
>Spec: add clarification for not used Pcd that build tool will not do additional
>checks on its value
>|               |
> |            | [#518](https://bugzilla.tianocore.org/show_bug.cgi?id=518) Build
>Spec: Update Precedence of PCD Values
>|               |
> |            | [#669](https://bugzilla.tianocore.org/show_bug.cgi?id=669) Build
>Spec: Add multi-arg support to PREBUILD/POSTBUILD
>|               |
>+|            | [#689](https://bugzilla.tianocore.org/show_bug.cgi?id=689) Build
>spec: add description for build with binary cache
>|               |
>diff --git a/appendix_d_buildexe_command/d4_usage.md
>b/appendix_d_buildexe_command/d4_usage.md
>index b71f2d0..6a91c3a 100644
>--- a/appendix_d_buildexe_command/d4_usage.md
>+++ b/appendix_d_buildexe_command/d4_usage.md
>@@ -32,19 +32,20 @@
> ## D.4 Usage
>
> ```ini
> Usage: build.exe [options]
> [all|fds|genc|genmake|clean|cleanall|cleanlib|modules|libraries|run]
>-Copyright (c) 2007 - 2014, Intel Corporation All rights reserved.
>+Copyright (c) 2007 - 2017, Intel Corporation All rights reserved.
>
> Options:
>   --version             show program's version number and exit
>   -h, --help            show this help message and exit
>   -a TARGETARCH, --arch=TARGETARCH
>-                        ARCHS is one of list: IA32, X64, IPF, ARM, or EBC,
>-                        which overrides target.txt's TARGET_ARCH definition.
>-                        To specify more archs, please repeat this option.
>+                        ARCHS is one of list: IA32, X64, IPF, ARM, AARCH64 or
>+                        EBC, which overrides target.txt's TARGET_ARCH
>+                        definition. To specify more archs, please repeat this
>+                        option.
>   -p PLATFORMFILE, --platform=PLATFORMFILE
>                         Build the platform specified by the DSC file name
>                         argument, overriding target.txt's ACTIVE_PLATFORM
>                         definition.
>   -m MODULEFILE, --module=MODULEFILE
>@@ -112,10 +113,20 @@ Options:
>   -N, --no-cache        Disable build cache mechanism
>   --conf=CONFDIRECTORY  Specify the customized Conf directory.
>   --check-usage         Check usage content of entries listed in INF file.
>   --ignore-sources      Focus to a binary build and ignore all source files
>   --pcd=OPTIONPCD       Set PCD value by command line. Format:
>"PcdName=Value"
>+  -l COMMANDLENGTH, --cmd-len=COMMANDLENGTH
>+                        Specify the maximum line length of build command.
>+                        Default is 4096.
>+  --hash                Enable hash-based caching during build process.
>+  --bindest=BINCACHEDEST
>+                        Generate a cache of binary files in the specified
>+                        directory.
>+  --binsource=BINCACHESOURCE
>+                        Consume a cache of binary files from the specified
>+                        directory.
> ```
>
> ### D.4.1 Debug Levels
>
> The numeric debug levels are defined as integer values 0-9.
>--
>2.6.1.windows.1

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel