1. Patchew
  2. Patchew
  3. View series

[Patchew-devel] [PATCH v2] applier: synchronize tags lazily into the cache

Paolo Bonzini posted 1 patch 5 years, 3 months ago 
patchew-cli | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
[Patchew-devel] [PATCH v2] applier: synchronize tags lazily into the cache
Posted by Paolo Bonzini 5 years, 3 months ago 
Every time the patchew's applier mode is restarted, it repopulates a cache
of git repositories.  For QEMU, the git repo has thousands of tags
which take a very long time to create.  Synchronize refs/heads/* only
(--mirror=fetch would synchronize all of refs/*); the tags are
only used by patchew-cli when a Based-on trailer is found, and can
be populated lazily.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
	v1->v2: always fetch refs/heads, so that HEAD contains a valid
		reference

 patchew-cli | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/patchew-cli b/patchew-cli
index 5a39f79..388d197 100755
--- a/patchew-cli
+++ b/patchew-cli
@@ -43,9 +43,18 @@ def git_clone_repo(clone, remote, head, logf, checkout=True, recursive=False):
                     cwd=cache_repo,
                     stdout=subprocess.PIPE,
                     stderr=subprocess.PIPE)
-    subprocess.check_call(["git", "remote", "add", "-f", "--mirror=fetch",
+    subprocess.check_call(["git", "remote", "add",
                           remote_name, remote], cwd=cache_repo,
                           stdout=logf, stderr=logf)
+    subprocess.check_call(["git", "config",
+                           "remote.%s.fetch" % (remote_name,),
+                           "+refs/heads/*:refs/heads/*"], cwd=cache_repo,
+                          stdout=logf, stderr=logf)
+    subprocess.check_call(["git", "fetch", remote_name], cwd=cache_repo,
+                          stdout=logf, stderr=logf)
+    if head.startswith("refs/tags/"):
+        subprocess.check_call(["git", "fetch", remote_name, "+%s:%s" % (head, head)],
+                              cwd=cache_repo, stdout=logf, stderr=logf)
     clone_cmd = ["git", "clone", "-q"]
     if recursive:
         clone_cmd += ["--recursive"]
-- 
2.20.1

_______________________________________________
Patchew-devel mailing list
Patchew-devel@redhat.com
https://www.redhat.com/mailman/listinfo/patchew-devel
Re: [Patchew-devel] [PATCH v2] applier: synchronize tags lazily into the cache
Posted by Fam Zheng 5 years, 3 months ago 

> On Jan 13, 2019, at 23:29, Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
> Every time the patchew's applier mode is restarted, it repopulates a cache
> of git repositories.  For QEMU, the git repo has thousands of tags
> which take a very long time to create.  Synchronize refs/heads/* only
> (--mirror=fetch would synchronize all of refs/*); the tags are
> only used by patchew-cli when a Based-on trailer is found, and can
> be populated lazily.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> 	v1->v2: always fetch refs/heads, so that HEAD contains a valid
> 		reference
> 
> patchew-cli | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/patchew-cli b/patchew-cli
> index 5a39f79..388d197 100755
> --- a/patchew-cli
> +++ b/patchew-cli
> @@ -43,9 +43,18 @@ def git_clone_repo(clone, remote, head, logf, checkout=True, recursive=False):
>                     cwd=cache_repo,
>                     stdout=subprocess.PIPE,
>                     stderr=subprocess.PIPE)
> -    subprocess.check_call(["git", "remote", "add", "-f", "--mirror=fetch",
> +    subprocess.check_call(["git", "remote", "add",
>                           remote_name, remote], cwd=cache_repo,
>                           stdout=logf, stderr=logf)
> +    subprocess.check_call(["git", "config",
> +                           "remote.%s.fetch" % (remote_name,),
> +                           "+refs/heads/*:refs/heads/*"], cwd=cache_repo,
> +                          stdout=logf, stderr=logf)
> +    subprocess.check_call(["git", "fetch", remote_name], cwd=cache_repo,
> +                          stdout=logf, stderr=logf)
> +    if head.startswith("refs/tags/"):
> +        subprocess.check_call(["git", "fetch", remote_name, "+%s:%s" % (head, head)],
> +                              cwd=cache_repo, stdout=logf, stderr=logf)

Will “+” prefix in the refspec work on older git?

Fam

>     clone_cmd = ["git", "clone", "-q"]
>     if recursive:
>         clone_cmd += ["--recursive"]
> -- 
> 2.20.1
> 
> _______________________________________________
> Patchew-devel mailing list
> Patchew-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/patchew-devel



_______________________________________________
Patchew-devel mailing list
Patchew-devel@redhat.com
https://www.redhat.com/mailman/listinfo/patchew-devel
Re: [Patchew-devel] [PATCH v2] applier: synchronize tags lazily into the cache
Posted by Paolo Bonzini 5 years, 3 months ago 
On 14/01/19 04:10, Fam Zheng wrote:
> 
> 
>> On Jan 13, 2019, at 23:29, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>> Every time the patchew's applier mode is restarted, it repopulates a cache
>> of git repositories.  For QEMU, the git repo has thousands of tags
>> which take a very long time to create.  Synchronize refs/heads/* only
>> (--mirror=fetch would synchronize all of refs/*); the tags are
>> only used by patchew-cli when a Based-on trailer is found, and can
>> be populated lazily.
>>
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>> 	v1->v2: always fetch refs/heads, so that HEAD contains a valid
>> 		reference
>>
>> patchew-cli | 11 ++++++++++-
>> 1 file changed, 10 insertions(+), 1 deletion(-)
>>
>> diff --git a/patchew-cli b/patchew-cli
>> index 5a39f79..388d197 100755
>> --- a/patchew-cli
>> +++ b/patchew-cli
>> @@ -43,9 +43,18 @@ def git_clone_repo(clone, remote, head, logf, checkout=True, recursive=False):
>>                     cwd=cache_repo,
>>                     stdout=subprocess.PIPE,
>>                     stderr=subprocess.PIPE)
>> -    subprocess.check_call(["git", "remote", "add", "-f", "--mirror=fetch",
>> +    subprocess.check_call(["git", "remote", "add",
>>                           remote_name, remote], cwd=cache_repo,
>>                           stdout=logf, stderr=logf)
>> +    subprocess.check_call(["git", "config",
>> +                           "remote.%s.fetch" % (remote_name,),
>> +                           "+refs/heads/*:refs/heads/*"], cwd=cache_repo,
>> +                          stdout=logf, stderr=logf)
>> +    subprocess.check_call(["git", "fetch", remote_name], cwd=cache_repo,
>> +                          stdout=logf, stderr=logf)
>> +    if head.startswith("refs/tags/"):
>> +        subprocess.check_call(["git", "fetch", remote_name, "+%s:%s" % (head, head)],
>> +                              cwd=cache_repo, stdout=logf, stderr=logf)
> 
> Will “+” prefix in the refspec work on older git?

Yes, it's been there forever.  I think I was using it in 2007 or so.

Paolo

_______________________________________________
Patchew-devel mailing list
Patchew-devel@redhat.com
https://www.redhat.com/mailman/listinfo/patchew-devel
Re: [Patchew-devel] [PATCH v2] applier: synchronize tags lazily into the cache
Posted by Fam Zheng 5 years, 3 months ago 

> On Jan 13, 2019, at 23:29, Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
> Every time the patchew's applier mode is restarted, it repopulates a cache
> of git repositories.  For QEMU, the git repo has thousands of tags
> which take a very long time to create.  Synchronize refs/heads/* only
> (--mirror=fetch would synchronize all of refs/*); the tags are
> only used by patchew-cli when a Based-on trailer is found, and can
> be populated lazily.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Fam Zheng <fam@euphon.net>

> ---
> 	v1->v2: always fetch refs/heads, so that HEAD contains a valid
> 		reference
> 
> patchew-cli | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/patchew-cli b/patchew-cli
> index 5a39f79..388d197 100755
> --- a/patchew-cli
> +++ b/patchew-cli
> @@ -43,9 +43,18 @@ def git_clone_repo(clone, remote, head, logf, checkout=True, recursive=False):
>                     cwd=cache_repo,
>                     stdout=subprocess.PIPE,
>                     stderr=subprocess.PIPE)
> -    subprocess.check_call(["git", "remote", "add", "-f", "--mirror=fetch",
> +    subprocess.check_call(["git", "remote", "add",
>                           remote_name, remote], cwd=cache_repo,
>                           stdout=logf, stderr=logf)
> +    subprocess.check_call(["git", "config",
> +                           "remote.%s.fetch" % (remote_name,),
> +                           "+refs/heads/*:refs/heads/*"], cwd=cache_repo,
> +                          stdout=logf, stderr=logf)
> +    subprocess.check_call(["git", "fetch", remote_name], cwd=cache_repo,
> +                          stdout=logf, stderr=logf)
> +    if head.startswith("refs/tags/"):
> +        subprocess.check_call(["git", "fetch", remote_name, "+%s:%s" % (head, head)],
> +                              cwd=cache_repo, stdout=logf, stderr=logf)
>     clone_cmd = ["git", "clone", "-q"]
>     if recursive:
>         clone_cmd += ["--recursive"]
> -- 
> 2.20.1
> 
> _______________________________________________
> Patchew-devel mailing list
> Patchew-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/patchew-devel



_______________________________________________
Patchew-devel mailing list
Patchew-devel@redhat.com
https://www.redhat.com/mailman/listinfo/patchew-devel

Patchew 2.1-devel-8c64711

© 2016 - 2024 Red Hat, Inc.