From nobody Sat May 11 03:05:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=patchew-devel-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=patchew-devel-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 152049605937146.9192381955487; Thu, 8 Mar 2018 00:00:59 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7F22AC04BD5A; Thu, 8 Mar 2018 08:00:58 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 701505C25F; Thu, 8 Mar 2018 08:00:58 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 62E674CA9C; Thu, 8 Mar 2018 08:00:58 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w2880vdh032398 for ; Thu, 8 Mar 2018 03:00:57 -0500 Received: by smtp.corp.redhat.com (Postfix) id 3CE815C25A; Thu, 8 Mar 2018 08:00:57 +0000 (UTC) Received: from mx1.redhat.com (ext-mx09.extmail.prod.ext.phx2.redhat.com [10.5.110.38]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 33E065C259 for ; Thu, 8 Mar 2018 08:00:53 +0000 (UTC) Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AA95C4E8AC for ; Thu, 8 Mar 2018 08:00:51 +0000 (UTC) Received: by mail-wm0-f54.google.com with SMTP id w128so9550655wmw.0 for ; Thu, 08 Mar 2018 00:00:51 -0800 (PST) Received: from donizetti.lan (94-36-191-219.adsl-ull.clienti.tiscali.it. [94.36.191.219]) by smtp.gmail.com with ESMTPSA id b9sm21755912wrh.68.2018.03.08.00.00.48 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 08 Mar 2018 00:00:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:date:message-id:in-reply-to:references; bh=7+eUFXhQDIjkYvjD1vGRCDe0w+Uav0VKq9GLoL1aGtI=; b=CB5H7cP7g8IHDHRGjS0jXmVf7w4E8PuXzS+AVV4iwrMrmsp1fndqJQXVi2YxWKaINI AUItCH8pywt9eF5Ph+w61GV1X/24cvfvOBOqQYz3bww+n3sdnZFk/g9CN7GPJfu1g08c V4HlzphbAItHy+YEDa3mn+PgT9NjyiLmocJwS21d/E9N9L3R7OpHT1QkvFmJc3b5TNGt p4jBOUwobFf7w5v7owPE9lkVQfRAhfJvjVDwSR+dwp6/8+rLQMBE1UWg1JXMd9/N2uZH WgXLo2SVxBwbJyanv3QooNiyiCQ79qvuLPWXp6LRxDUCJ47UH03DYMKxTXzMevyEkVEP 8fxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:date:message-id :in-reply-to:references; bh=7+eUFXhQDIjkYvjD1vGRCDe0w+Uav0VKq9GLoL1aGtI=; b=m48N+Uc58djtEJEudveC22Zxcj4wtqpkDmINw5t8lUyDA31NEREsR1IuWCQCDhqXPk q2pc7cvq59Fqw8NPPbjT+S2w6PdSd5ze5yLE+P1qiFNyo/5LmD592aZMrltX4Ku3b2ML F+9cnOMIJrluPj5tia/FVnyH/Szja9Skzb1Nfan43b2+5tL0zOO2tlxge/9Wa1CihYyD 364XbKD9plI84/kwSm7nUubMgYI8U0t0XPGn2gLEEPSv6pF9UEQn+G8Wt1qFXixpYvFv hgjwmE8plkXWJSoHWhpr0l6eVjKZzmN+KTZtXTuthiK86Ovhi0URDLBkIawm2cF14DOR gAdQ== X-Gm-Message-State: AElRT7FeCSCGJI6iVW08FEK7EBmtJ1uX1wSvEUBC4zWe1w5mOO67F52d 7C6YxJAGLqLVzdK4uCVuWk8KQhQc X-Google-Smtp-Source: AG47ELtBJlNASTBolY/Tj8sbZXWTf829Dj9Vlv8xlxu0TizNtOlPbm4/F6b1eEl32d4L7Et5LVeI4A== X-Received: by 10.28.163.5 with SMTP id m5mr15537129wme.7.1520496049747; Thu, 08 Mar 2018 00:00:49 -0800 (PST) From: Paolo Bonzini To: patchew-devel@redhat.com Date: Thu, 8 Mar 2018 09:00:39 +0100 Message-Id: <20180308080042.19342-5-pbonzini@redhat.com> In-Reply-To: <20180308080042.19342-1-pbonzini@redhat.com> References: <20180308080042.19342-1-pbonzini@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 08 Mar 2018 08:00:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 08 Mar 2018 08:00:52 +0000 (UTC) for IP:'74.125.82.54' DOMAIN:'mail-wm0-f54.google.com' HELO:'mail-wm0-f54.google.com' FROM:'paolo.bonzini@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.469 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_PASS) 74.125.82.54 mail-wm0-f54.google.com 74.125.82.54 mail-wm0-f54.google.com X-RedHat-Possible-Forgery: Paolo Bonzini X-Scanned-By: MIMEDefang 2.78 on 10.5.110.38 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: patchew-devel@redhat.com Subject: [Patchew-devel] [PATCH 4/7] add custom template tags for manipulating testing logs X-BeenThere: patchew-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Patchew development and discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: patchew-devel-bounces@redhat.com Errors-To: patchew-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 08 Mar 2018 08:00:58 +0000 (UTC) X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" These can be used to make test report emails shorter and more readable. Signed-off-by: Paolo Bonzini --- patchew/tags.py | 77 ++++++++++++++++++++++++++++++++++ tests/test_custom_tags.py | 102 ++++++++++++++++++++++++++++++++++++++++++= ++++ 2 files changed, 179 insertions(+) diff --git a/patchew/tags.py b/patchew/tags.py index 22d64b0..4847cb4 100644 --- a/patchew/tags.py +++ b/patchew/tags.py @@ -10,6 +10,57 @@ =20 from django import template from patchew import logviewer +from collections import deque +import io +import operator +import re + +# The basic implementation uses generators. The filters simply apply join +# to the result of the generators. + +def lines_iter(value): + if not isinstance(value, io.IOBase): + # StringIO provides a generator to split lines. + value =3D io.StringIO(value) + # "chomp" the newlines on each line. Using operator and map does + # everything in the interpreter, avoiding the overhead of a generator. + return map(operator.methodcaller('rstrip', '\r\n'), value) + +# To understand grep_iter, it may help to first study this implementation +# of a "tail" iterator, which is based on the same circular array idea: +# +# def tail_lines_iter(value, n): +# lines =3D [None] * n +# lineno =3D 0 +# for line in lines_iter(value): +# lines[lineno % n] =3D line +# lineno +=3D 1 +# +# for i in range(max(lineno - n, 0), lineno): +# yield lines[i % n] +# +# Basic "grep" prints one line when the match is on the last line, so +# "grep" is a variation on tail_lines with n=3D1; likewise, "grep -B1" is +# a variation on tail_lines with n=3D2, etc. + +def grep_iter(value, regex, n_before, n_after, sep): + n =3D n_before + 1 + lines =3D [None] * n + stop =3D lineno =3D 0 + for line in lines_iter(value): + # Print the (lineno - n)-th line. Each element of lines[] is used + # just before it is thrown away. + if lineno - n >=3D 0 and lineno - n < stop: + yield lines[lineno % n] + if re.search(regex, line): + if lineno - n >=3D stop and sep is not None and stop > 0: + yield sep + stop =3D lineno + n_after + 1 + lines[lineno % n] =3D line + lineno +=3D 1 + + for i in range(max(lineno - n, 0), min(stop, lineno)): + yield lines[i % n] =20 register =3D template.Library() =20 @@ -17,3 +68,29 @@ register =3D template.Library() @register.filter def ansi2text(value): return ''.join(logviewer.ansi2text(value)) + +@register.simple_tag +@register.filter +def tail_lines(value, n): + lines =3D deque(lines_iter(value), n) + return '\n'.join(lines) + +@register.simple_tag +@register.filter +def grep(value, regex, sep=3DNone): + return '\n'.join(grep_iter(value, regex, 0, 0, sep)) + +@register.simple_tag +@register.filter +def grep_A(value, regex, n=3D3, sep=3D'---'): + return '\n'.join(grep_iter(value, regex, 0, n, sep)) + +@register.simple_tag +@register.filter +def grep_B(value, regex, n=3D3, sep=3D'---'): + return '\n'.join(grep_iter(value, regex, n, 0, sep)) + +@register.simple_tag +@register.filter +def grep_C(value, regex, n=3D3, sep=3D'---'): + return '\n'.join(grep_iter(value, regex, n, n, sep)) diff --git a/tests/test_custom_tags.py b/tests/test_custom_tags.py index ec875c8..aa828be 100755 --- a/tests/test_custom_tags.py +++ b/tests/test_custom_tags.py @@ -9,6 +9,7 @@ # http://opensource.org/licenses/MIT. =20 from django.template import Context, Template +from patchew.tags import tail_lines, grep_A, grep_B, grep_C, grep import patchewtest import unittest =20 @@ -19,9 +20,110 @@ class CustomTagsTest(unittest.TestCase): =20 def test_template_filters(self): self.assertTemplate('{{s|ansi2text}}', 'dbc', s=3D'abc\rd') + self.assertTemplate('{{s|grep:"[0-9]"}}', '0\n9', s=3D'0\na\n9') + self.assertTemplate('{{s|grep_A:"b"}}', + 'b\nc\nd\ne\n---\nb', + s=3D'a\nb\nc\nd\ne\nf\nx\ny\nz\nb') + self.assertTemplate('{{s|grep_B:"b"}}', + 'a\nb\n---\nx\ny\nz\nb', + s=3D'a\nb\nc\nd\ne\nf\nx\ny\nz\nb') + self.assertTemplate('{{s|grep_C:"b"}}', + 'a\nb\nc\nd\ne\n---\nx\ny\nz\nb', + s=3D'a\nb\nc\nd\ne\nf\nx\ny\nz\nb') + self.assertTemplate('{{s|tail_lines:3}}', 'b\nc\nd', s=3D'a\nb\nc\= nd') =20 def test_template_tags(self): self.assertTemplate('{% ansi2text s %}', 'dbc', s=3D'abc\rd') + self.assertTemplate('{% grep s "[0-9]" %}', '0\n9', s=3D'0\na\n9') + self.assertTemplate('{% grep_A s regex=3D"[bc]" n=3D1 %}', 'b\nc\n= d', s=3D'a\nb\nc\nd') + self.assertTemplate('{% grep_B s regex=3D"[bc]" n=3D1 %}', 'a\nb\n= c', s=3D'a\nb\nc\nd') + self.assertTemplate('{% grep_C s "b" n=3D1 %}', 'a\nb\nc', s=3D'a\= nb\nc\nd') + self.assertTemplate('{% tail_lines s n=3D3 %}', 'b\nc\nd', s=3D'a\= nb\nc\nd') + + def test_grep(self): + self.assertEqual(grep('0\na\n9', '[0-9]'), '0\n9') + self.assertEqual(grep('0\na\n9', '[0-9]', '---'), '0\n---\n9') + + def test_grep_A(self): + self.assertEqual(grep_A('a\nb\nc\nd', 'b', 1, None), 'b\nc') + self.assertEqual(grep_A('a\nb\nc\nd', 'b', 2, None), 'b\nc\nd') + self.assertEqual(grep_A('a\nb\nc\nd\nb\ne', 'b', 1, None), 'b\nc\n= b\ne') + self.assertEqual(grep_A('a\nb\nc\nd\nb\ne', 'b', 2, None), 'b\nc\n= d\nb\ne') + self.assertEqual(grep_A('a\nb\nc\nd\nz\nb\ne', 'b', 1, None), 'b\n= c\nb\ne') + self.assertEqual(grep_A('a\nb\nc\nd\nz\nb\ne', 'b', 2, None), 'b\n= c\nd\nb\ne') + self.assertEqual(grep_A('a\nb\nc\nz\nb\nb\ne', 'b', 1, None), 'b\n= c\nb\nb\ne') + self.assertEqual(grep_A('b\nc\nz\nb\nb\ne', 'b', 1, None), 'b\nc\n= b\nb\ne') + self.assertEqual(grep_A('b\n', 'b', 1, None), 'b') + + def test_grep_A_sep(self): + self.assertEqual(grep_A('a\nb\nc\nd', 'b', 1), 'b\nc') + self.assertEqual(grep_A('a\nb\nc\nd', 'b', 2), 'b\nc\nd') + self.assertEqual(grep_A('a\nb\nc\nd\nb\ne', 'b', 1), 'b\nc\n---\nb= \ne') + self.assertEqual(grep_A('a\nb\nc\nd\nb\ne', 'b', 2), 'b\nc\nd\nb\n= e') + self.assertEqual(grep_A('a\nb\nc\nd\nz\nb\ne', 'b', 1), 'b\nc\n---= \nb\ne') + self.assertEqual(grep_A('a\nb\nc\nd\nz\nb\nb\ne', 'b', 1), 'b\nc\n= ---\nb\nb\ne') + self.assertEqual(grep_A('b\nc\nz\nb\nb\ne', 'b', 1), 'b\nc\n---\nb= \nb\ne') + self.assertEqual(grep_A('b\n', 'b', 1), 'b') + + def test_grep_B(self): + self.assertEqual(grep_B('a\nb\nc\nd', 'b', 1, None), 'a\nb') + self.assertEqual(grep_B('a\nb\nc\nd', 'b', 2, None), 'a\nb') + self.assertEqual(grep_B('a\nb\nc\nd\nb\ne', 'b', 1, None), 'a\nb\n= d\nb') + self.assertEqual(grep_B('a\nb\nc\nd\nz\nb\ne', 'b', 1, None), 'a\n= b\nz\nb') + self.assertEqual(grep_B('a\nb\nc\nd\nz\nb\ne', 'b', 2, None), 'a\n= b\nd\nz\nb') + self.assertEqual(grep_B('a\nb\nc\nz\nb\nb\ne', 'b', 1, None), 'a\n= b\nz\nb\nb') + self.assertEqual(grep_B('b\nc\nz\nb\nb\ne', 'b', 1, None), 'b\nz\n= b\nb') + self.assertEqual(grep_B('b\n', 'b', 1, None), 'b') + + def test_grep_B_sep(self): + self.assertEqual(grep_B('a\nb\nc\nd', 'b', 1), 'a\nb') + self.assertEqual(grep_B('a\nb\nc\nd', 'b', 2), 'a\nb') + self.assertEqual(grep_B('a\nb\nc\nd\nb\ne', 'b', 1), 'a\nb\n---\nd= \nb') + self.assertEqual(grep_B('a\nb\nc\nd\nz\nb\ne', 'b', 1), 'a\nb\n---= \nz\nb') + self.assertEqual(grep_B('a\nb\nc\nd\nz\nb\ne', 'b', 2), 'a\nb\n---= \nd\nz\nb') + self.assertEqual(grep_B('a\nb\nc\nz\nb\nb\ne', 'b', 1), 'a\nb\n---= \nz\nb\nb') + self.assertEqual(grep_B('b\nc\nz\nb\nb\ne', 'b', 1), 'b\n---\nz\nb= \nb') + self.assertEqual(grep_B('b\n', 'b', 1), 'b') + + def test_grep_C(self): + self.assertEqual(grep_C('a\nb\nc\nd', 'b', 1, None), 'a\nb\nc') + self.assertEqual(grep_C('a\nb\nc\nd', 'b', 2, None), 'a\nb\nc\nd') + self.assertEqual(grep_C('a\nb\nc\nd\nb\ne', 'b', 1, None), 'a\nb\n= c\nd\nb\ne') + self.assertEqual(grep_C('a\nb\nc\nd\nz\nb\ne', 'b', 1, None), 'a\n= b\nc\nz\nb\ne') + self.assertEqual(grep_C('a\nb\nc\nd\nz\nb\ne', 'b', 2, None), 'a\n= b\nc\nd\nz\nb\ne') + self.assertEqual(grep_C('a\nb\nc\nz\nb\nb\ne', 'b', 1, None), 'a\n= b\nc\nz\nb\nb\ne') + self.assertEqual(grep_C('b\nc\nz\nb\nb\ne', 'b', 1, None), 'b\nc\n= z\nb\nb\ne') + self.assertEqual(grep_C('b\n', 'b', 1, None), 'b') + + def test_grep_C_sep(self): + self.assertEqual(grep_C('a\nb\nc\nd', 'b', 1), 'a\nb\nc') + self.assertEqual(grep_C('a\nb\nc\nd', 'b', 2), 'a\nb\nc\nd') + self.assertEqual(grep_C('a\nb\nc\nd\nb\ne', 'b', 1), 'a\nb\nc\nd\n= b\ne') + self.assertEqual(grep_C('a\nb\nc\nd\nz\nb\ne', 'b', 1), 'a\nb\nc\n= ---\nz\nb\ne') + self.assertEqual(grep_C('a\nb\nc\nd\nz\nb\ne', 'b', 2), 'a\nb\nc\n= d\nz\nb\ne') + self.assertEqual(grep_C('a\nb\nc\nz\nb\nb\ne', 'b', 1), 'a\nb\nc\n= z\nb\nb\ne') + self.assertEqual(grep_C('b\nc\nz\nb\nb\ne', 'b', 1), 'b\nc\nz\nb\n= b\ne') + self.assertEqual(grep_C('b\n', 'b', 1), 'b') + + def test_tail_lines(self): + self.assertEqual(tail_lines('', 0), '') + self.assertEqual(tail_lines('', 1), '') + self.assertEqual(tail_lines('', 2), '') + self.assertEqual(tail_lines('', 4), '') + + self.assertEqual(tail_lines('a\nb\n', 0), '') + self.assertEqual(tail_lines('a\nb\n', 1), 'b') + self.assertEqual(tail_lines('a\nb\n', 2), 'a\nb') + + self.assertEqual(tail_lines('a\nb\nc\n', 2), 'b\nc') + self.assertEqual(tail_lines('a\nb\nc\n', 4), 'a\nb\nc') + + self.assertEqual(tail_lines('a\nb\nc\nd\n', 2), 'c\nd') + + self.assertEqual(tail_lines('\n\n\n', 2), '\n') + self.assertEqual(tail_lines('\n\n\nbc', 2), '\nbc') + self.assertEqual(tail_lines('\n\nbc', 3), '\n\nbc') + self.assertEqual(tail_lines('\n\n\n\nbc', 3), '\n\nbc') =20 if __name__ =3D=3D '__main__': unittest.main() --=20 2.14.3 _______________________________________________ Patchew-devel mailing list Patchew-devel@redhat.com https://www.redhat.com/mailman/listinfo/patchew-devel