[edk2] [PATCH] BaseTools: Fix --hash Package and Module hash value.

Lin, Derek (HPS UEFI Dev) posted 1 patch 5 years, 11 months ago
Failed in applying to current master (apply log)
BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
[edk2] [PATCH] BaseTools: Fix --hash Package and Module hash value.
Posted by Lin, Derek (HPS UEFI Dev) 5 years, 11 months ago
From: Lin, Derek (HPS UEFI Dev) 
Sent: Wednesday, May 9, 2018 3:39 PM
To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.

The order of List enumeration is arbitrary.
Need to be sorted while calculating Package/Module hash, otherwise it generate different hash value even nothing changes.
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 54f6b1f173..90704dcae4 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -2,6 +2,7 @@
 # Generate AutoGen.h, AutoGen.c and *.depex files  #  # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+#  Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
 # This program and the accompanying materials  # are licensed and made available under the terms and conditions of the BSD License  # which accompanies this distribution.  The full text of the license may be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
         return True
 
     def _GenPkgLevelHash(self, Pkg):
+        if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
+            return
+
         PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
         CreateDirectory(PkgDir)
         HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@ -681,17 +685,16 @@ class WorkspaceAutoGen(AutoGen):
         m.update(Content)
         # Get include files hash value
         if Pkg.Includes:
-            for inc in Pkg.Includes:
+            for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
                 for Root, Dirs, Files in os.walk(str(inc)):
-                    for File in Files:
+                    for File in sorted(Files):
                         File_Path = os.path.join(Root, File)
                         f = open(File_Path, 'r')
                         Content = f.read()
                         f.close()
                         m.update(Content)
         SaveFileOnChange(HashFile, m.hexdigest(), True)
-        if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
-            GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = m.hexdigest()
+        GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = 
+ m.hexdigest()
 
     def _GetMetaFiles(self, Target, Toolchain, Arch):
         AllWorkSpaceMetaFiles = set()
@@ -4432,13 +4435,13 @@ class ModuleAutoGen(AutoGen):
         m.update(GlobalData.gPlatformHash)
         # Add Package level hash
         if self.DependentPackageList:
-            for Pkg in self.DependentPackageList:
+            for Pkg in sorted(self.DependentPackageList, key=lambda x: x.PackageName):
                 if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
                     m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
 
         # Add Library hash
         if self.LibraryAutoGenList:
-            for Lib in self.LibraryAutoGenList:
+            for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
                 if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
                     Lib.GenModuleHash()
                 m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
@@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
         m.update(Content)
         # Add Module's source files
         if self.SourceFileList:
-            for File in self.SourceFileList:
+            for File in sorted(self.SourceFileList, key=lambda x: str(x)):
                 f = open(str(File), 'r')
                 Content = f.read()
                 f.close()
--
2.15.1.windows.2

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH] BaseTools: Fix --hash Package and Module hash value.
Posted by Zhu, Yonghong 5 years, 11 months ago
Hi Derek,

Please refer below link to add Contributed-under and Signed-off-by info.
We can use BaseTools\Scripts\PatchCheck.py  to check the patch format. Thanks.
https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format 

Best Regards,
Zhu Yonghong


-----Original Message-----
From: Lin, Derek (HPS UEFI Dev) [mailto:derek.lin2@hpe.com] 
Sent: Wednesday, May 09, 2018 3:49 PM
To: edk2-devel@lists.01.org
Cc: Zhu, Yonghong <yonghong.zhu@intel.com>; Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.

From: Lin, Derek (HPS UEFI Dev) 
Sent: Wednesday, May 9, 2018 3:39 PM
To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.

The order of List enumeration is arbitrary.
Need to be sorted while calculating Package/Module hash, otherwise it generate different hash value even nothing changes.
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 54f6b1f173..90704dcae4 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -2,6 +2,7 @@
 # Generate AutoGen.h, AutoGen.c and *.depex files  #  # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+#  Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
 # This program and the accompanying materials  # are licensed and made available under the terms and conditions of the BSD License  # which accompanies this distribution.  The full text of the license may be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
         return True
 
     def _GenPkgLevelHash(self, Pkg):
+        if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
+            return
+
         PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
         CreateDirectory(PkgDir)
         HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@ -681,17 +685,16 @@ class WorkspaceAutoGen(AutoGen):
         m.update(Content)
         # Get include files hash value
         if Pkg.Includes:
-            for inc in Pkg.Includes:
+            for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
                 for Root, Dirs, Files in os.walk(str(inc)):
-                    for File in Files:
+                    for File in sorted(Files):
                         File_Path = os.path.join(Root, File)
                         f = open(File_Path, 'r')
                         Content = f.read()
                         f.close()
                         m.update(Content)
         SaveFileOnChange(HashFile, m.hexdigest(), True)
-        if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
-            GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = m.hexdigest()
+        GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = 
+ m.hexdigest()
 
     def _GetMetaFiles(self, Target, Toolchain, Arch):
         AllWorkSpaceMetaFiles = set()
@@ -4432,13 +4435,13 @@ class ModuleAutoGen(AutoGen):
         m.update(GlobalData.gPlatformHash)
         # Add Package level hash
         if self.DependentPackageList:
-            for Pkg in self.DependentPackageList:
+            for Pkg in sorted(self.DependentPackageList, key=lambda x: x.PackageName):
                 if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
                     m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
 
         # Add Library hash
         if self.LibraryAutoGenList:
-            for Lib in self.LibraryAutoGenList:
+            for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
                 if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
                     Lib.GenModuleHash()
                 m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
@@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
         m.update(Content)
         # Add Module's source files
         if self.SourceFileList:
-            for File in self.SourceFileList:
+            for File in sorted(self.SourceFileList, key=lambda x: str(x)):
                 f = open(str(File), 'r')
                 Content = f.read()
                 f.close()
--
2.15.1.windows.2


_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] [PATCH] BaseTools: Fix --hash Package and Module hash value.
Posted by Lin, Derek (HPS UEFI Dev) 5 years, 11 months ago
Hi Yonghong,

Thanks for notify about this.
I resend the patch.

Thanks,
Derek


-----Original Message-----
From: Zhu, Yonghong [mailto:yonghong.zhu@intel.com] 
Sent: Wednesday, May 9, 2018 4:35 PM
To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>; edk2-devel@lists.01.org
Cc: Zhu, Yonghong <yonghong.zhu@intel.com>
Subject: RE: [PATCH] BaseTools: Fix --hash Package and Module hash value.

Hi Derek,

Please refer below link to add Contributed-under and Signed-off-by info.
We can use BaseTools\Scripts\PatchCheck.py  to check the patch format. Thanks.
https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format 

Best Regards,
Zhu Yonghong


-----Original Message-----
From: Lin, Derek (HPS UEFI Dev) [mailto:derek.lin2@hpe.com] 
Sent: Wednesday, May 09, 2018 3:49 PM
To: edk2-devel@lists.01.org
Cc: Zhu, Yonghong <yonghong.zhu@intel.com>; Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.

From: Lin, Derek (HPS UEFI Dev) 
Sent: Wednesday, May 9, 2018 3:39 PM
To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.

The order of List enumeration is arbitrary.
Need to be sorted while calculating Package/Module hash, otherwise it generate different hash value even nothing changes.
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 54f6b1f173..90704dcae4 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -2,6 +2,7 @@
 # Generate AutoGen.h, AutoGen.c and *.depex files  #  # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+#  Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
 # This program and the accompanying materials  # are licensed and made available under the terms and conditions of the BSD License  # which accompanies this distribution.  The full text of the license may be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
         return True
 
     def _GenPkgLevelHash(self, Pkg):
+        if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
+            return
+
         PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
         CreateDirectory(PkgDir)
         HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@ -681,17 +685,16 @@ class WorkspaceAutoGen(AutoGen):
         m.update(Content)
         # Get include files hash value
         if Pkg.Includes:
-            for inc in Pkg.Includes:
+            for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
                 for Root, Dirs, Files in os.walk(str(inc)):
-                    for File in Files:
+                    for File in sorted(Files):
                         File_Path = os.path.join(Root, File)
                         f = open(File_Path, 'r')
                         Content = f.read()
                         f.close()
                         m.update(Content)
         SaveFileOnChange(HashFile, m.hexdigest(), True)
-        if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
-            GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = m.hexdigest()
+        GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = 
+ m.hexdigest()
 
     def _GetMetaFiles(self, Target, Toolchain, Arch):
         AllWorkSpaceMetaFiles = set()
@@ -4432,13 +4435,13 @@ class ModuleAutoGen(AutoGen):
         m.update(GlobalData.gPlatformHash)
         # Add Package level hash
         if self.DependentPackageList:
-            for Pkg in self.DependentPackageList:
+            for Pkg in sorted(self.DependentPackageList, key=lambda x: x.PackageName):
                 if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
                     m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
 
         # Add Library hash
         if self.LibraryAutoGenList:
-            for Lib in self.LibraryAutoGenList:
+            for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
                 if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
                     Lib.GenModuleHash()
                 m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
@@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
         m.update(Content)
         # Add Module's source files
         if self.SourceFileList:
-            for File in self.SourceFileList:
+            for File in sorted(self.SourceFileList, key=lambda x: str(x)):
                 f = open(str(File), 'r')
                 Content = f.read()
                 f.close()
--
2.15.1.windows.2


_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel