From nobody Tue May 13 19:29:18 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=famz@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by mx.zohomail.com with SMTPS id 1519700459720714.8233749214631; Mon, 26 Feb 2018 19:00:59 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7E94D4036114 for ; Tue, 27 Feb 2018 03:00:58 +0000 (UTC) Received: from localhost (ovpn-12-53.pek2.redhat.com [10.72.12.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1A65C10B0F22 for ; Tue, 27 Feb 2018 03:00:56 +0000 (UTC) Received: by 10.103.98.66 with SMTP id w63csp3595243vsb; Mon, 26 Feb 2018 03:27:25 -0800 (PST) Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id v49si8393262qtc.31.2018.02.26.03.27.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Feb 2018 03:27:25 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2760A8182D06 for ; Mon, 26 Feb 2018 11:27:25 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 22DBF2026E04; Mon, 26 Feb 2018 11:27:25 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-117-126.ams2.redhat.com [10.36.117.126]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7D5A42026DFD; Mon, 26 Feb 2018 11:27:24 +0000 (UTC) Resent-From: Fam Zheng Resent-Date: Tue, 27 Feb 2018 11:00:55 +0800 Resent-Message-ID: <20180227030055.GN21035@lemon.usersys.redhat.com> Resent-To: importer@patchew.org X-Google-Smtp-Source: AG47ELv8uZY+NV3fzSeZvBUF7iNjAy0QUN/El5CzpP0r8mvS7tl+xe8nb2uwR2bRBWBavNKWu0lS X-Received: by 10.200.81.215 with SMTP id d23mr16157883qtn.272.1519644445429; Mon, 26 Feb 2018 03:27:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519644445; cv=none; d=google.com; s=arc-20160816; b=fQ8yt/+IdCYpESynV7wnTSYRAAtWk13u17Ym2rC8fG3U6YrpRUuG5gNBHUf4SNtuef Slv4oEw7qzyuC9kGX7WKwg6o1/dgd1BVGLDKNsgIH44hPu4o4LRVBMB1vDgCkc7ZYliA B9otqO8bQdfq+Muz8KomU6P5ZyFy3HMAYgilXCEnXDkEFDDHAWQY3amXdUyrNndln3Iv yMuPqq0Kxrw6KlAoEUJltoXJKzh5E/vQRpXlRS9uY46Jf+KSN/x8A07Y5WcE5e4tB/ez iWAOGNnJcjjl/sX3QH9Fh72fWXhsRiKJKV5ZYkcQyZbfFmh9Ka3cxJ7c4VpuwfSBb9ue fEiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :delivered-to:arc-authentication-results; bh=K3ASak9fEVEDon1Kh2RjgjTIw5aQhsh3HUEGfLuZhDc=; b=vc1F+n0eDEdYaU55agj2SLK5S/aGaalk1VN839s4aXHKNZCdKD5JxoZ/lG58x3B8MZ NP4Y8Wb3lPbP9WsKq/I01PJMkrdzN914En2MMVKWhwqWb0Ib5IzlYNUUgejn5kjEXSWS BqnP4GAh6okjognYZiaa3mD0oAnQlSjudO7dejJt5a6E/zg7qcLPt9fkE5nYqM1IOiRm B5tbGYaAx/UP2x8/0nZ7OgVgrB991xYV3PGZn+T1A2H+OIsQ4FRPpOybMWcVwAeK2ODQ tolL/SRKU3ZcMeouol85ZSngVjUtlm1SM68OISMUTIBi6RDWBPvj2jA7ElFjBXpmEdl1 zYUg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain gapps.redhat.com configured 66.187.233.73 as internal address) smtp.mailfrom=pbonzini@redhat.com Received-SPF: pass (zoho.com: domain of redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; envelope-from=famz@redhat.com; helo=mx1.redhat.com; Received-SPF: pass (google.com: domain gapps.redhat.com configured 66.187.233.73 as internal address) Authentication-Results: mx.google.com; spf=pass (google.com: domain gapps.redhat.com configured 66.187.233.73 as internal address) smtp.mailfrom=pbonzini@redhat.com From: Paolo Bonzini To: patchew-devel@freelists.org Cc: famz@redhat.com Subject: [PATCH 2/9] ansi2html: add basic parsing and cursor movement support Date: Mon, 26 Feb 2018 12:27:15 +0100 Message-Id: <20180226112722.19488-3-pbonzini@redhat.com> In-Reply-To: <20180226112722.19488-1-pbonzini@redhat.com> References: <20180226112722.19488-1-pbonzini@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Mon, 26 Feb 2018 11:27:25 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Mon, 26 Feb 2018 11:27:25 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'pbonzini@redhat.com' RCPT:'' X-TUID: V9JeZuKU+8uI X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 27 Feb 2018 03:00:58 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 27 Feb 2018 03:00:58 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add the code to track cursor movement and overwrites within a line, and a simple lexer based on re.finditer. Use it to implement \b, \t, \r and \f. Signed-off-by: Paolo Bonzini --- patchew/logviewer.py | 82 +++++++++++++++++++++++++++++++++++++++++++++= ++-- tests/test_ansi2html.py | 31 +++++++++++++++++++ 2 files changed, 110 insertions(+), 3 deletions(-) diff --git a/patchew/logviewer.py b/patchew/logviewer.py index 4438fa4..1e3495b 100644 --- a/patchew/logviewer.py +++ b/patchew/logviewer.py @@ -4,30 +4,106 @@ # # Author: Paolo Bonzini =20 +# Entity table based on ansi2html.c from colorized-logs. + +import re import abc import sys =20 from django.views import View from django.http import HttpResponse, StreamingHttpResponse -from django.utils.html import format_html from django.utils.safestring import mark_safe =20 class ANSI2HTMLConverter(object): + RE_STRING =3D '[^\b\t\n\f\r\x1B]+' + RE_NUMS =3D '[0-9]+(?:;[0-9]+)*' + RE_CSI =3D r'\[\??(?:' + RE_NUMS + ')?[^;0-9]' + RE_OSC =3D r'].*?(?:\x1B\\|\x07)' + RE_CONTROL =3D '\x1B(?:%s|%s|[^][])|[\b\t\n\f\r]' % (RE_CSI, RE_OSC) + RE =3D re.compile('(%s)|(%s)' % (RE_STRING, RE_CONTROL)) + + ENTITIES =3D { + '\x00' : '␀', '\x01' : '␁', '\x02' : '␂', + '\x03' : '␃', '\x04' : '␄', '\x05' : '␅', + '\x06' : '␆', '\x07' : '🔔', '\x0B' : '␋', + '\x0E' : '␎', '\x0F' : '␏', '\x10' : '␐', + '\x11' : '␑', '\x12' : '␒', '\x13' : '␓', + '\x14' : '␔', '\x15' : '␕', '\x16' : '␖', + '\x17' : '␗', '\x18' : '␘', '\x19' : '␙', + '\x1A' : '␚', '\x1B' : '␛', '\x1C' : '␜', + '\x1D' : '␝', '\x1E' : '␞', '\x1F' : '␟', + '<' : '<', '>' : '>', '&' : '&', + '\x7F' : '⌦' + } + RE_ENTITIES =3D re.compile('[\x00-\x1F<>&\x7F]') + def __init__(self, white_bg=3DFalse): self.prefix =3D '
'
+        self._reset()
+
+    def _reset(self):
+        self.line =3D []
+        self.pos =3D 0
+
+    # self.line holds the characters for the current line.
+    # Writing can overwrite some characters if self.pos is
+    # not pointing to the end of the line, and then appends.
+    # Moving the cursor right can add spaces to the end.
+
+    def _write(self, chars):
+        cur_len =3D len(self.line)
+        if self.pos < cur_len:
+            last =3D min(cur_len - self.pos, len(chars))
+            self.line[self.pos:self.pos+last] =3D list(chars[0:last])
+        else:
+            last =3D 0
+
+        if len(chars) > last:
+            self.line +=3D list(chars[last:])
+        self.pos +=3D len(chars)
+
+    def _set_pos(self, pos):
+        self.pos =3D pos
+        if self.pos > len(self.line):
+            num =3D self.pos - len(self.line)
+            self.line +=3D [' '] * num
=20
     def _write_prefix(self):
         if self.prefix !=3D '':
             yield self.prefix
             self.prefix =3D ''
=20
+    def _write_span(self, text):
+        yield self.RE_ENTITIES.sub(lambda x: self.ENTITIES[x.group(0)], te=
xt)
+
+    def _write_line(self, suffix):
+        text =3D "".join(self.line)
+        yield from self._write_span(text)
+        yield suffix
+        self._reset()
+
     def convert(self, input):
         yield from self._write_prefix()
-        yield format_html('{}', input)
+        for m in self.RE.finditer(input):
+            if m.group(1):
+                self._write(m.group(1))
+            else:
+                seq =3D m.group(2)
+                if seq =3D=3D '\n':
+                    yield from self._write_line('\n')
+                elif seq =3D=3D '\f':
+                    yield from self._write_line('\n
')
+                elif seq =3D=3D '\b':
+                    if self.pos > 0:
+                        self.pos -=3D 1
+                elif seq =3D=3D '\t':
+                    self._set_pos(self.pos + (8 - self.pos % 8))
+                elif seq =3D=3D '\r':
+                    self.pos =3D 0
=20
     def finish(self):
         yield from self._write_prefix()
-        yield '
' + yield from self._write_line('') self.prefix =3D '
'
=20
 def ansi2html(input, white_bg=3DFalse):
diff --git a/tests/test_ansi2html.py b/tests/test_ansi2html.py
index 5ad993b..731f243 100644
--- a/tests/test_ansi2html.py
+++ b/tests/test_ansi2html.py
@@ -22,8 +22,39 @@ class ANSI2HTMLTest(unittest.TestCase):
=20
     # basic formatting tests
     def test_basic(self):
+        self.assertBlackBg('\tb', '        b')
+        self.assertBlackBg('\t\ta', '                a')
+        self.assertBlackBg('a\tb', 'a       b')
+        self.assertBlackBg('ab\tc', 'ab      c')
         self.assertBlackBg('a\nbc', 'a\nbc')
+        self.assertBlackBg('a\f', 'a\n
')
+        self.assertBlackBg('a\n\f', 'a\n\n
')
         self.assertBlackBg('<', '<')
+        self.assertBlackBg('\x07', '🔔')
+
+    # backspace and carriage return
+    def test_set_pos(self):
+        self.assertBlackBg('abc\b\bBC', 'aBC')
+        self.assertBlackBg('a\b<', '<')
+        self.assertBlackBg('<\ba', 'a')
+        self.assertBlackBg('a\b\bbc', 'bc')
+        self.assertBlackBg('a\rbc', 'bc')
+        self.assertBlackBg('a\nb\bc', 'a\nc')
+        self.assertBlackBg('a\t\bb', 'a      b')
+        self.assertBlackBg('a\tb\b\bc', 'a      cb')
+        self.assertBlackBg('01234567\r\tb', '01234567b')
+
+    # Escape sequences
+    def test_esc_parsing(self):
+        self.assertBlackBg('{\x1b%}', '{}')
+        self.assertBlackBg('{\x1b[0m}', '{}')
+        self.assertBlackBg('{\x1b[m}', '{}')
+        self.assertBlackBg('{\x1b[0;1;7;0m}', '{}')
+        self.assertBlackBg('{\x1b[1;7m\x1b[m}', '{}')
+        self.assertBlackBg('{\x1b]test\x1b\\}', '{}')
+        self.assertBlackBg('{\x1b]test\x07}', '{}')
+        self.assertBlackBg('{\x1b]test\x1b[0m\x07}', '{}')
+        self.assertBlackBg('{\x1b]test\x1b[7m\x07}', '{}')
=20
=20
 if __name__ =3D=3D '__main__':
--=20
2.14.3