From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621451757; cv=none; d=zohomail.com; s=zohoarc; b=YvWPfamBshl70oKLjf1WW+eXNpHLzzN160ehGbNYhtP2xjmymSE+oUzg/1uqjWsQ1lCdfo7Zx4aWvLBHe+rjbl6P2oADrypm1vEKHlielxKM51BqRNBSfurEag5fqDT/Kn+LyZCw8R4Iw1XFaj1E3zH3dNqbBXe1XGSkN39jU6I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621451757; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Cfw1pJJnQNieEEbqPWCtUB8RGpx680d1YC/REGKv2AE=; b=hMUDVUDbx6XsSFTi+ymgb8FlyG6FaibjPLHNTJuWj1woRzaM1ZIAbfAZydUgjM0DDQ2UOaU/SwIuMZBcQpXWd90Tb9J5PyV4AOAtwdpeWUijj958FI/YyboRuKZ/O0ebjJhz+Ml3rt9pn5tND7zE/UkngXb8dMvtnlEGvsWdAmk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621451757314396.59203453124235; Wed, 19 May 2021 12:15:57 -0700 (PDT) Received: from localhost ([::1]:56250 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRfg-0003Ff-6z for importer2@patchew.org; Wed, 19 May 2021 15:15:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55270) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR73-00061u-NY for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:34283) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6w-0001z9-2w for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:09 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-489-t7DRotyuPOykSZ8fmWvw4g-1; Wed, 19 May 2021 14:39:55 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 64C5C501F4; Wed, 19 May 2021 18:39:54 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id A7CD95944C; Wed, 19 May 2021 18:39:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449597; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cfw1pJJnQNieEEbqPWCtUB8RGpx680d1YC/REGKv2AE=; b=N1mzkaVNKcrmfWxm2K5T1LW4b3brkZTDy/rl4QEM3M0S1aT8gJNcJqD7Vos3MO7/Vptv1e PEp8O+XS1kJtkJHv1u2+zk1oMxwlt23LhY9ZqMYfu0irqxYGiVrNCMBRAe+7bVVZtsdlI1 qfHt3qXmokauTJuaYgB7XRxXbpVXg7Q= X-MC-Unique: t7DRotyuPOykSZ8fmWvw4g-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 01/15] qapi/parser: Don't try to handle file errors Date: Wed, 19 May 2021 14:39:37 -0400 Message-Id: <20210519183951.3946870-2-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Fixes: f5d4361cda Fixes: 52a474180a Fixes: 46f49468c6 Remove the try/except block that handles file-opening errors in QAPISchemaParser.__init__() and add one each to QAPISchemaParser._include() and QAPISchema.__init__() respectively. This simultaneously fixes the typing of info.fname (f5d4361cda), A static typing violation in test-qapi (46f49468c6), and a regression of an error message (52a474180a). The short-ish version of what motivates this patch is: - It's hard to write a good error message in the init method, because we need to determine the context of our caller to do so. It's easier to just let the caller write the message. - We don't want to allow QAPISourceInfo(None, None, None) to exist. The typing introduced by commit f5d4361cda types the 'fname' field as (non-optional) str, which was premature until the removal of this construct. - Errors made using such an object are currently incorrect (since 52a474180a) - It's not technically a semantic error if we cannot open the schema. - There are various typing constraints that make mixing these two cases undesirable for a single special case. - test-qapi's code handling an fname of 'None' is now dead, drop it. Additionally, Not all QAPIError objects have an 'info' field (since 46f49468), so deleting this stanza corrects a typing oversight in test-qapi introduced by that commit. Other considerations: - open() is moved to a 'with' block to ensure file pointers are cleaned up deterministically. - Python 3.3 deprecated IOError and made it a synonym for OSError. Avoid the misleading perception these exception handlers are narrower than they really are. The long version: The error message here is incorrect (since commit 52a474180a): > python3 qapi-gen.py 'fake.json' qapi-gen.py: qapi-gen.py: can't read schema file 'fake.json': No such file = or directory In pursuing it, we find that QAPISourceInfo has a special accommodation for when there's no filename. Meanwhile, the intent when QAPISourceInfo was typed (f5d4361cda) was non-optional 'str'. This usage was overlooked. To remove this, I'd want to avoid having a "fake" QAPISourceInfo object. I also don't want to explicitly begin accommodating QAPISourceInfo itself being None, because we actually want to eventually prove that this can never happen -- We don't want to confuse "The file isn't open yet" with "This error stems from a definition that wasn't defined in any file". (An earlier series tried to create a dummy info object, but it was tough to prove in review that it worked correctly without creating new regressions. This patch avoids that distraction. We would like to first prove that we never raise QAPISemError for any built-in object before we add "special" info objects. We aren't ready to do that yet.) So, which way out of the labyrinth? Here's one way: Don't try to handle errors at a level with "mixed" semantic contexts; i.e. don't mix inclusion errors (should report a source line where the include was triggered) and command line errors (where we specified a file we couldn't read). Remove the error handling from the initializer of the parser. Pythonic! Now it's the caller's job to figure out what to do about it. Handle the error in QAPISchemaParser._include() instead, where we can write a targeted error message where we are guaranteed to have an 'info' context to report with. The root level error can similarly move to QAPISchema.__init__(), where we know we'll never have an info context to report with, so we use a more abstract error type. Now the error looks sensible again: > python3 qapi-gen.py 'fake.json' qapi-gen.py: can't read schema file 'fake.json': No such file or directory With these error cases separated, QAPISourceInfo can be solidified as never having placeholder arguments that violate our desired types. Clean up test-qapi along similar lines. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 18 +++++++++--------- scripts/qapi/schema.py | 11 +++++++++-- scripts/qapi/source.py | 3 --- tests/qapi-schema/test-qapi.py | 3 --- 4 files changed, 18 insertions(+), 17 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index ca5e8e18e00..a53b735e7de 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -40,15 +40,9 @@ def __init__(self, fname, previously_included=3DNone, in= cl_info=3DNone): previously_included =3D previously_included or set() previously_included.add(os.path.abspath(fname)) =20 - try: - fp =3D open(fname, 'r', encoding=3D'utf-8') + # May raise OSError; allow the caller to handle it. + with open(fname, 'r', encoding=3D'utf-8') as fp: self.src =3D fp.read() - except IOError as e: - raise QAPISemError(incl_info or QAPISourceInfo(None, None, Non= e), - "can't read %s file '%s': %s" - % ("include" if incl_info else "schema", - fname, - e.strerror)) =20 if self.src =3D=3D '' or self.src[-1] !=3D '\n': self.src +=3D '\n' @@ -129,7 +123,13 @@ def _include(self, include, info, incl_fname, previous= ly_included): if incl_abs_fname in previously_included: return None =20 - return QAPISchemaParser(incl_fname, previously_included, info) + try: + return QAPISchemaParser(incl_fname, previously_included, info) + except OSError as err: + raise QAPISemError( + info, + f"can't read include file '{incl_fname}': {err.strerror}" + ) from err =20 def _check_pragma_list_of_str(self, name, value, info): if (not isinstance(value, list) diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py index 3a4172fb749..d1d27ff7ee8 100644 --- a/scripts/qapi/schema.py +++ b/scripts/qapi/schema.py @@ -20,7 +20,7 @@ from typing import Optional =20 from .common import POINTER_SUFFIX, c_name -from .error import QAPISemError, QAPISourceError +from .error import QAPIError, QAPISemError, QAPISourceError from .expr import check_exprs from .parser import QAPISchemaParser =20 @@ -849,7 +849,14 @@ def visit(self, visitor): class QAPISchema: def __init__(self, fname): self.fname =3D fname - parser =3D QAPISchemaParser(fname) + + try: + parser =3D QAPISchemaParser(fname) + except OSError as err: + raise QAPIError( + f"can't read schema file '{fname}': {err.strerror}" + ) from err + exprs =3D check_exprs(parser.exprs) self.docs =3D parser.docs self._entity_list =3D [] diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py index 03b6ede0828..1ade864d7b9 100644 --- a/scripts/qapi/source.py +++ b/scripts/qapi/source.py @@ -10,7 +10,6 @@ # See the COPYING file in the top-level directory. =20 import copy -import sys from typing import List, Optional, TypeVar =20 =20 @@ -53,8 +52,6 @@ def next_line(self: T) -> T: return info =20 def loc(self) -> str: - if self.fname is None: - return sys.argv[0] ret =3D self.fname if self.line is not None: ret +=3D ':%d' % self.line diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py index e8db9d09d91..f1c4deb9a51 100755 --- a/tests/qapi-schema/test-qapi.py +++ b/tests/qapi-schema/test-qapi.py @@ -128,9 +128,6 @@ def test_and_diff(test_name, dir_name, update): try: test_frontend(os.path.join(dir_name, test_name + '.json')) except QAPIError as err: - if err.info.fname is None: - print("%s" % err, file=3Dsys.stderr) - return 2 errstr =3D str(err) + '\n' if dir_name: errstr =3D errstr.replace(dir_name + '/', '') --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621451760; cv=none; d=zohomail.com; s=zohoarc; b=Bd7hOin8EsHIn4qPPUDW1ifNO/t5P3mOtoLepmxPgL8wsjMsDJwYUgHcHrpxUTqbueysD63Xdx8dXCeVivqp0Pv2mQE3u/zAbsykY/oIFQO8uuHIfH6KhWFgH+/HkLq9q+TUwrCxftc+zxGzE6/fcWk4iMxj3UdhATVq9BdIqCE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621451760; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2Zw/1IFeRgebH6yrdkTw5SelFOgt1TKw/afQTI5DhMA=; b=BAf3mcjhVbXSxCzKkKEA+YBAkZ6mLzz7r5gvvhNGdK2bm6EkL7c7vaShnPSadDSuas5kzHy4cSlggyBUArQsWZTq79R+gPVvymkoFYoSlNMNEhGELPgAbnrZ7z1li1Mqpz5z9gYl1CpdydKHsPV3UfgD39MQ1pkSFkP0FW+j35w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621451760209514.3803004606588; Wed, 19 May 2021 12:16:00 -0700 (PDT) Received: from localhost ([::1]:56684 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRfj-0003XV-8N for importer2@patchew.org; Wed, 19 May 2021 15:15:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55218) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR70-0005z9-UV for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:28375) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6w-0001zK-D0 for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:06 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-600-t0zKAzBDPyy6pDoRY9tE8A-1; Wed, 19 May 2021 14:39:56 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4858680ED8D; Wed, 19 May 2021 18:39:55 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C8E259443; Wed, 19 May 2021 18:39:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Zw/1IFeRgebH6yrdkTw5SelFOgt1TKw/afQTI5DhMA=; b=ZO8bf3nFrWdUHy4PG0v0LNrCVZC6OrA3fDwIoBOEcSzesaUd0JG7RRuPWADrk6eoWKYy2e uCLHVLwDk7x4B6/4zoCSDJ2ihPVWiSGq00wFvV1sNbloTIAb5wwXFcpbXh+9Td8q4Yd/Kd ccEO6ja/BoKpwORY+pj/3/tpSR90veI= X-MC-Unique: t0zKAzBDPyy6pDoRY9tE8A-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 02/15] qapi: Add test for nonexistent schema file Date: Wed, 19 May 2021 14:39:38 -0400 Message-Id: <20210519183951.3946870-3-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This tests the error-return pathway introduced in the previous commit. (Thanks to Paolo for the help with the Meson magic.) Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- This went after the previous patch instead of before because prior to removing the sys.argv[0] bit from QAPISourceInfo, I can't filter the test to pass the diff. Instead of writing something new to get a better patch ordering, just add the test after. Signed-off-by: John Snow --- tests/qapi-schema/meson.build | 7 ++++++- tests/qapi-schema/missing-schema.err | 1 + tests/qapi-schema/missing-schema.out | 0 3 files changed, 7 insertions(+), 1 deletion(-) create mode 100644 tests/qapi-schema/missing-schema.err create mode 100644 tests/qapi-schema/missing-schema.out diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build index d7163e6601c..dc448e8f74d 100644 --- a/tests/qapi-schema/meson.build +++ b/tests/qapi-schema/meson.build @@ -199,11 +199,16 @@ schemas =3D [ 'unknown-escape.json', 'unknown-expr-key.json', ] +schemas =3D files(schemas) + +# Intentionally missing schema file test -- not passed through files(): +schemas +=3D [meson.current_source_dir() / 'missing-schema.json'] =20 # Because people may want to use test-qapi.py from the command line, we # are not using the "#! /usr/bin/env python3" trick here. See # docs/devel/build-system.txt -test('QAPI schema regression tests', python, args: files('test-qapi.py', s= chemas), +test('QAPI schema regression tests', python, + args: files('test-qapi.py') + schemas, env: test_env, suite: ['qapi-schema', 'qapi-frontend']) =20 diff =3D find_program('diff') diff --git a/tests/qapi-schema/missing-schema.err b/tests/qapi-schema/missi= ng-schema.err new file mode 100644 index 00000000000..b4d9ff1fb2b --- /dev/null +++ b/tests/qapi-schema/missing-schema.err @@ -0,0 +1 @@ +can't read schema file 'missing-schema.json': No such file or directory diff --git a/tests/qapi-schema/missing-schema.out b/tests/qapi-schema/missi= ng-schema.out new file mode 100644 index 00000000000..e69de29bb2d --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621452232; cv=none; d=zohomail.com; s=zohoarc; b=jAhUnjdyrARUjiyCbp+AkrbgUhrGIppMJyhg5wvy+UpU6i5D9kV16VuEHvcpHJ71sToWHPTUp6OCwtiNq/5C2gAEGhtEIOXpe6sYQIWGEtUcJrn8nN/Cbj7yGCG3s4ZK2q3t+MHBeb+YfNfYtiGZlRPTpf6wg6Q/v2cUpAKYqpg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621452232; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XcLCBJp2GifzvxOzMbM4f59L5eLSU4faCfjRnldxOV4=; b=OXfjZa45sTWPWsLFglM++1fOQVyTWZAfl+6owNU+LCXF7b6lqVK50cU5ll3vZKLSzdLrr1mXJjesXwXWmleZ1KRBUmYHRl332Vp/PjhMqnh2VZNMsE6YOoQshJaUXkmF3yGDLjpOms9gABNk+f8OFKOXre06JsjhMn3OGDl8f1s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621452232700436.24652016718994; Wed, 19 May 2021 12:23:52 -0700 (PDT) Received: from localhost ([::1]:53292 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRnL-0004S0-Fg for importer2@patchew.org; Wed, 19 May 2021 15:23:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55310) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR78-00062s-PM for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:31570) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6x-00020y-KB for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:14 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-414-MpV5hP4wM_yT575qcpcLQA-1; Wed, 19 May 2021 14:39:57 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 378F3501ED; Wed, 19 May 2021 18:39:56 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6FC6A59443; Wed, 19 May 2021 18:39:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449602; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XcLCBJp2GifzvxOzMbM4f59L5eLSU4faCfjRnldxOV4=; b=E9K5JHnjW0kljBHHoTT7prS5WVpYhyq7+XjcE1ds/TXGmpiibbRAtjb1va9oO6Sz9krK/C TqyEvCIRXt2kA3iSbWvEZuJ2WwD2SPBn3a4BJGVtvEYf9K+forLVmvDSbPt3EfB5vrdJw5 upp8aRTZzfyLVYbq1+wiy41Ln3vxmEg= X-MC-Unique: MpV5hP4wM_yT575qcpcLQA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 03/15] qapi/source: Remove line number from QAPISourceInfo initializer Date: Wed, 19 May 2021 14:39:39 -0400 Message-Id: <20210519183951.3946870-4-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" With the QAPISourceInfo(None, None, None) construct gone, there's no longer any reason to have to specify that a file starts on the first line. Remove it from the initializer and default it to 1. Remove the last vestiges where we check for 'line' being unset, that can't happen, now. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 2 +- scripts/qapi/source.py | 10 +++------- 2 files changed, 4 insertions(+), 8 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index a53b735e7de..39dbcc4eacc 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -47,7 +47,7 @@ def __init__(self, fname, previously_included=3DNone, inc= l_info=3DNone): if self.src =3D=3D '' or self.src[-1] !=3D '\n': self.src +=3D '\n' self.cursor =3D 0 - self.info =3D QAPISourceInfo(fname, 1, incl_info) + self.info =3D QAPISourceInfo(fname, incl_info) self.line_pos =3D 0 self.exprs =3D [] self.docs =3D [] diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py index 1ade864d7b9..04193cc9643 100644 --- a/scripts/qapi/source.py +++ b/scripts/qapi/source.py @@ -31,10 +31,9 @@ def __init__(self) -> None: class QAPISourceInfo: T =3D TypeVar('T', bound=3D'QAPISourceInfo') =20 - def __init__(self, fname: str, line: int, - parent: Optional['QAPISourceInfo']): + def __init__(self, fname: str, parent: Optional['QAPISourceInfo']): self.fname =3D fname - self.line =3D line + self.line =3D 1 self.parent =3D parent self.pragma: QAPISchemaPragma =3D ( parent.pragma if parent else QAPISchemaPragma() @@ -52,10 +51,7 @@ def next_line(self: T) -> T: return info =20 def loc(self) -> str: - ret =3D self.fname - if self.line is not None: - ret +=3D ':%d' % self.line - return ret + return f"{self.fname}:{self.line}" =20 def in_defn(self) -> str: if self.defn_name: --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621451883; cv=none; d=zohomail.com; s=zohoarc; b=Of7SKtKjP860u/gwV4xuk/8167J7yEZln9/HiiR3AhD7un35KYecqffMcf5NDQwZeBAO27qUo3fV6YB/NTIT4WTZurZhPmxykre5/Jm2YJhO844/5cRL+tom2/2IeDYxHyxsiv2+b+9Z8B3X/Y222Oh6+qG6GgMJShQPQZNxlHc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621451883; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jp8Frl7879ztIXD/N9WVnUKG+uz2E5Qwvhv8zguDcGA=; b=bCJtRYNlNlf3wJH2WXvDGrIYPuFPUyv2QKz1H3vT773QZUuJ4AxySA2MRXbm8CvDeVvk9ShppgYlBLp9ePHoTon6tFHAbEgin+H56066a6xD5xLiNwGtYRMpC/NbrNZk1jCZ6WqNqp13IlUvrsTDUVTpn65LnnRtvZsnbsLfbRs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621451883443411.9002218636134; Wed, 19 May 2021 12:18:03 -0700 (PDT) Received: from localhost ([::1]:36844 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRhi-0000xE-AX for importer2@patchew.org; Wed, 19 May 2021 15:18:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55244) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR71-000612-O3 for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:23600) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6w-0001zN-TP for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:07 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-162-1nkSOxKNNSChjWtj7_h0nw-1; Wed, 19 May 2021 14:39:58 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0DEE5100747E; Wed, 19 May 2021 18:39:57 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5378B59443; Wed, 19 May 2021 18:39:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jp8Frl7879ztIXD/N9WVnUKG+uz2E5Qwvhv8zguDcGA=; b=OGcYhtqtJMweNzLOvpqeYnXxgJT5hpqcCKS0+qpw+EiswLuk2pus4MCwRGAFDmGHAvDFFO SpOgxKLfhplkS7H3QTv86leKuMdZVgPHS5B0EPl7vGp79PALQchfB88BTAM72I+0I20LKp spSdw0CtsF3BKf2rIWZ7aUwYEoj7a9Q= X-MC-Unique: 1nkSOxKNNSChjWtj7_h0nw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 04/15] qapi/parser: factor parsing routine into method Date: Wed, 19 May 2021 14:39:40 -0400 Message-Id: <20210519183951.3946870-5-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" For the sake of keeping __init__ smaller (and treating it more like a gallery of what state variables we can expect to see), put the actual parsing action into a parse method. It remains invoked from the init method to reduce churn. To accomplish this, @previously_included becomes the private data member ._included, and the filename is stashed as ._fname. Add any missing declarations to the init method, and group them by function so they can be understood quickly at a glance. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 40 ++++++++++++++++++++++++++++------------ 1 file changed, 28 insertions(+), 12 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 39dbcc4eacc..d620706fffb 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -37,23 +37,39 @@ def __init__(self, parser, msg): class QAPISchemaParser: =20 def __init__(self, fname, previously_included=3DNone, incl_info=3DNone= ): - previously_included =3D previously_included or set() - previously_included.add(os.path.abspath(fname)) + self._fname =3D fname + self._included =3D previously_included or set() + self._included.add(os.path.abspath(self._fname)) + self.src =3D '' =20 - # May raise OSError; allow the caller to handle it. - with open(fname, 'r', encoding=3D'utf-8') as fp: - self.src =3D fp.read() - - if self.src =3D=3D '' or self.src[-1] !=3D '\n': - self.src +=3D '\n' + # Lexer state (see `accept` for details): + self.info =3D QAPISourceInfo(self._fname, incl_info) + self.tok =3D None + self.pos =3D 0 self.cursor =3D 0 - self.info =3D QAPISourceInfo(fname, incl_info) + self.val =3D None self.line_pos =3D 0 + + # Parser output: self.exprs =3D [] self.docs =3D [] - self.accept() + + # Showtime! + self._parse() + + def _parse(self): cur_doc =3D None =20 + # May raise OSError; allow the caller to handle it. + with open(self._fname, 'r', encoding=3D'utf-8') as fp: + self.src =3D fp.read() + if self.src =3D=3D '' or self.src[-1] !=3D '\n': + self.src +=3D '\n' + + # Prime the lexer: + self.accept() + + # Parse until done: while self.tok is not None: info =3D self.info if self.tok =3D=3D '#': @@ -71,12 +87,12 @@ def __init__(self, fname, previously_included=3DNone, i= ncl_info=3DNone): if not isinstance(include, str): raise QAPISemError(info, "value of 'include' must be a strin= g") - incl_fname =3D os.path.join(os.path.dirname(fname), + incl_fname =3D os.path.join(os.path.dirname(self._fname), include) self.exprs.append({'expr': {'include': incl_fname}, 'info': info}) exprs_include =3D self._include(include, info, incl_fname, - previously_included) + self._included) if exprs_include: self.exprs.extend(exprs_include.exprs) self.docs.extend(exprs_include.docs) --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621451540; cv=none; d=zohomail.com; s=zohoarc; b=JaWDDIQ7ScpPv+uyqpZcAfAq8YUAiUeEoqpQFfopefxgNUd/Jkt2ckHBTQ9y8gb6UGmg8g8SbvBBp0E0Z2j5WgBEn1EpBX3E6dLcNmOOOI+NZ8wlzuOEpd0v0UjAZXwHI7YcR3C2hlMOVQ0vNspo24MTL+fDGT8Vbs+ThCi7zJM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621451540; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=aI7W7IlWiQpToC4v4jSUvx7Ftf+CA8yBdCNhfT36HQw=; b=CqafTxjInGXRvVe7huEVbDstCsIjhpA8bH165QfTBnDQl8y61ElcagKohtzAu5V4v0UU4uP81grroh+vDtIdJN66OgxuUtuTm4edux8ciGeaqnOoMGujz9mkXx/smEVfkpGgGvxhmJvn1+0RP5l7HFxqbc950jxVjJ+MHFkM5Ek= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621451540462143.48507239424748; Wed, 19 May 2021 12:12:20 -0700 (PDT) Received: from localhost ([::1]:47848 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRcB-0005kO-C9 for importer2@patchew.org; Wed, 19 May 2021 15:12:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55234) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR71-00060H-Gn for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37856) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6w-0001zO-Oh for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:07 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-62-u6jIMFu_OqWXL29boiuAKg-1; Wed, 19 May 2021 14:39:59 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 53A18800D62; Wed, 19 May 2021 18:39:58 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3594859443; Wed, 19 May 2021 18:39:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aI7W7IlWiQpToC4v4jSUvx7Ftf+CA8yBdCNhfT36HQw=; b=XTd8h8nnUE67Hl+ThiNnhfKy+IHi8Xg0/LP+Z7w1H/KWHWBxVqjrNQVpYz9uW/U+ujW8ij 4IyfxoPCaVpXjoKsjpC8IVHeMwPkOV9AVKJwLGwAMpVTYAEGFBmsgTuSkHU9/0cKOR7axs KM+osL266xOeShDdWBexoRDesMv7jPs= X-MC-Unique: u6jIMFu_OqWXL29boiuAKg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 05/15] qapi/parser: Assert lexer value is a string Date: Wed, 19 May 2021 14:39:41 -0400 Message-Id: <20210519183951.3946870-6-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The type checker can't narrow the type of the token value to string, because it's only loosely correlated with the return token. We know that a token of '#' should always have a "str" value. Add an assertion. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index d620706fffb..0bc852eda72 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -304,6 +304,7 @@ def get_doc(self, info): cur_doc =3D QAPIDoc(self, info) self.accept(False) while self.tok =3D=3D '#': + assert isinstance(self.val, str) if self.val.startswith('##'): # End of doc comment if self.val !=3D '##': --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621452338; cv=none; d=zohomail.com; s=zohoarc; b=nbjTbL5CNF74zeMyvDJX1UDEXlTLj1Hd/uoyqA1jCifDFkXrNC4jYR9KWYYC1Cleh/pDeEBEfk2IQ/pI0zNg/n2lAjDQ0Ifr1FwCzZ0BgD/27UC1nLUvoePHMS587RJVxFuA8PJGqOJRrDuYld4rz9eN6Evgo+GKTSMsTu+hFuI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621452338; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=q8AaRF//0zi4W5LuE7KrmpqS8mCUzfo33bp8znBSemc=; b=TsE9SQAMeg8qefV+ANduGW+qYv3qT0HZNnYUUrthGYtGBAhrWP+qVidn4ZDsAdTACZro9MHQtcYugLuKpGEx8B+aLtD0GdOhaYVN2/47uwQ1g+EzR/9UjwkGscgSWq+I693eSZNQ6sF2gypLWMIGSuEAgt33Rt3BpwD/HzJ1cgs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621452338905691.5454748133054; Wed, 19 May 2021 12:25:38 -0700 (PDT) Received: from localhost ([::1]:33496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRp3-0001uD-Ny for importer2@patchew.org; Wed, 19 May 2021 15:25:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55330) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR7B-00063s-5U for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21105) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6x-00020a-2f for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:16 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-61-rYeGwbEbMga9z8KQv7Yu9w-1; Wed, 19 May 2021 14:40:00 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 752848015C6; Wed, 19 May 2021 18:39:59 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9618560BD9; Wed, 19 May 2021 18:39:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449602; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q8AaRF//0zi4W5LuE7KrmpqS8mCUzfo33bp8znBSemc=; b=hHsmETJBkG1Z9gMA8A0SCIihXdkcOzmD+P/t4xW48I5wbmP9PVgfUsrUKLASrThkhw8YvG C+x8aJydE88ePo1OhGOdLqrsjWt8SR3WAhWLUZp01iJ0NzAKtCxo8tV0QNqUhSdy78ygO8 aGs5DE+FWHM4HeZ2Yl5nVcm5WWhmaIU= X-MC-Unique: rYeGwbEbMga9z8KQv7Yu9w-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 06/15] qapi/parser: enforce all top-level expressions must be dict in _parse() Date: Wed, 19 May 2021 14:39:42 -0400 Message-Id: <20210519183951.3946870-7-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Instead of using get_expr nested=3DFalse, allow get_expr to always return any expression. In exchange, add a new error message to the top-level parser that explains the semantic error: Top-level expressions must always be JSON objects. This helps mypy understand the rest of this function which assumes that get_expr did indeed return a dict. The exception type changes from QAPIParseError to QAPISemError as a result, and the error message in two tests now changes. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- Thanks Markus, I like this quite a bit better. I think I swung the pendulum back too far away from "try not to change anything". This is cleaner. Signed-off-by: John Snow --- scripts/qapi/parser.py | 14 ++++++++------ tests/qapi-schema/non-objects.err | 2 +- tests/qapi-schema/quoted-structural-chars.err | 2 +- 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 0bc852eda72..ffdd4298b6b 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -78,7 +78,11 @@ def _parse(self): self.docs.append(cur_doc) continue =20 - expr =3D self.get_expr(False) + expr =3D self.get_expr() + if not isinstance(expr, dict): + raise QAPISemError( + info, "top-level expression must be an object") + if 'include' in expr: self.reject_expr_doc(cur_doc) if len(expr) !=3D 1: @@ -251,7 +255,7 @@ def get_members(self): self.accept() if key in expr: raise QAPIParseError(self, "duplicate key '%s'" % key) - expr[key] =3D self.get_expr(True) + expr[key] =3D self.get_expr() if self.tok =3D=3D '}': self.accept() return expr @@ -270,7 +274,7 @@ def get_values(self): raise QAPIParseError( self, "expected '{', '[', ']', string, or boolean") while True: - expr.append(self.get_expr(True)) + expr.append(self.get_expr()) if self.tok =3D=3D ']': self.accept() return expr @@ -278,9 +282,7 @@ def get_values(self): raise QAPIParseError(self, "expected ',' or ']'") self.accept() =20 - def get_expr(self, nested): - if self.tok !=3D '{' and not nested: - raise QAPIParseError(self, "expected '{'") + def get_expr(self): if self.tok =3D=3D '{': self.accept() expr =3D self.get_members() diff --git a/tests/qapi-schema/non-objects.err b/tests/qapi-schema/non-obje= cts.err index 3a4ea36966e..23bdb69c711 100644 --- a/tests/qapi-schema/non-objects.err +++ b/tests/qapi-schema/non-objects.err @@ -1 +1 @@ -non-objects.json:1:1: expected '{' +non-objects.json:1: top-level expression must be an object diff --git a/tests/qapi-schema/quoted-structural-chars.err b/tests/qapi-sch= ema/quoted-structural-chars.err index 07d1561d1f7..af6c1e173db 100644 --- a/tests/qapi-schema/quoted-structural-chars.err +++ b/tests/qapi-schema/quoted-structural-chars.err @@ -1 +1 @@ -quoted-structural-chars.json:1:1: expected '{' +quoted-structural-chars.json:1: top-level expression must be an object --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621451878; cv=none; d=zohomail.com; s=zohoarc; b=DP1LJrlURr1dLqhlTwmEuVu/pRZmq9NF6cshv7iezTA0qcxcFDQ0kAYyRJP/dwLmqnpJ0tPTNmysLKF68Y0GIVJRhHGNXuJOaL4+jJhCaNQ4K3IX4hPqGcMrxVEEjE+PGvWFn2cW3yjTTa5xkvXA6LboFk0vDDanZ/b4J/yJ+m4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621451878; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Cv0ANjeTyt53VFYW0L9D5b7rB0+9bbslIVfOzPEcSuo=; b=UQgszdA9Gd4rmP9IfGbIuMb/2NBalFkXFi0z5nyrr5q4JSb2M8sfNJTq165BgbBaWNkZnFsGQggPQMkzdeaCf16VgVL02ncNnVRE3s+0r+MbK7aF6Fn3a0zkSVeLNP40ERsABGrNpENYO/5rWlM0gI0J1qZiHaCyRl+UayeTgyA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621451878672299.1182284827969; Wed, 19 May 2021 12:17:58 -0700 (PDT) Received: from localhost ([::1]:36424 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRhd-0000g2-LS for importer2@patchew.org; Wed, 19 May 2021 15:17:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55282) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR75-00062V-I9 for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:54452) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6x-00021M-QJ for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:11 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-131-D6yMNMGkOmWJWnye3VKeLA-1; Wed, 19 May 2021 14:40:01 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4BAEE100747A; Wed, 19 May 2021 18:40:00 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8F85C5944C; Wed, 19 May 2021 18:39:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449603; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cv0ANjeTyt53VFYW0L9D5b7rB0+9bbslIVfOzPEcSuo=; b=X2taRPQCtiv/oiP5etIdW6rXCim4ybIMobO6g/APIM5EouNMfw4856D5c6r7qWWEjNs6sW AbXAWkpUtGTF31SoMKzY0hBk29PjeaBtDG7dS1NrvD0+lIgMUM3qsNRIfJbjoznyADTphx oehq9VJIuDK1BLpxWvsuXP2dvquj5VQ= X-MC-Unique: D6yMNMGkOmWJWnye3VKeLA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 07/15] qapi/parser: assert object keys are strings Date: Wed, 19 May 2021 14:39:43 -0400 Message-Id: <20210519183951.3946870-8-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The single quote token implies the value is a string. Assert this to be the case, to allow us to write an accurate return type for get_members. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index ffdd4298b6b..4959630ce64 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -249,6 +249,8 @@ def get_members(self): raise QAPIParseError(self, "expected string or '}'") while True: key =3D self.val + assert isinstance(key, str) # Guaranteed by tok =3D=3D "'" + self.accept() if self.tok !=3D ':': raise QAPIParseError(self, "expected ':'") --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621450729; cv=none; d=zohomail.com; s=zohoarc; b=ftP5nsh5g5Vpw2EfVYRE9pq5u0Sz+wiFohoVh8OLNypgJJx8Sr0ZaKM6gwM+7GiJZ7pdiQ8WzqzhW9z0VRRwmuHcJZNLEqqmmvM8XT+of4i7GoLNpWO5spB3xVI7UMQZxCxjCOS0T+FIzZ65T7N5tUYqf9Ubi6O3NFiHi8kwtIg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621450729; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FU4XspzLRiY8Y54MOfT8KSbc5TquWno86xK6eyBoFc4=; b=mXjvg/SzEtKAkAybRRnvAyRib0Fs6AxDbfJMDbO059PpJMkaoSSF/1Ayp9xeYK4LxWbmj+eSsO7OmIMeaEP9vGc8sgSTO+arpjaSzoSfS2mB6isNr9IwHhuqcETPesVXXRrjrkg0PyiywcE4MTirVcIGCOgs+fikVM05jfXj3dg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 162145072940232.807875771144836; Wed, 19 May 2021 11:58:49 -0700 (PDT) Received: from localhost ([::1]:38026 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRP6-0005CW-CQ for importer2@patchew.org; Wed, 19 May 2021 14:58:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55340) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR7G-00064Z-B4 for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45467) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6z-00021o-Lv for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:20 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-26--ct56z9RMUuEz6UdPvf1CA-1; Wed, 19 May 2021 14:40:02 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 34A9480ED91; Wed, 19 May 2021 18:40:01 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7507E5944C; Wed, 19 May 2021 18:40:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FU4XspzLRiY8Y54MOfT8KSbc5TquWno86xK6eyBoFc4=; b=iMP+ztJef4H5t/Ca+m/nCoYRTsYXHb3JBQN8Yf3bFldp2cd1t+pstu8EZq4lK6a5kO1dqm AmMgmw3FCLBF7TdICKrowm0UYOVvinNvGmNe7HI+1Tw53XJ5tDTDJeKkkSNsLHZHoEzn/2 XZ0/TANlP4VzJV9DuRjo2emBt5I33ZA= X-MC-Unique: -ct56z9RMUuEz6UdPvf1CA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 08/15] qapi/parser: Use @staticmethod where appropriate Date: Wed, 19 May 2021 14:39:44 -0400 Message-Id: <20210519183951.3946870-9-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" No self, no thank you! (Quiets pylint warnings.) Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 4959630ce64..7c718661950 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -130,7 +130,8 @@ def reject_expr_doc(doc): "documentation for '%s' is not followed by the definition" % doc.symbol) =20 - def _include(self, include, info, incl_fname, previously_included): + @staticmethod + def _include(include, info, incl_fname, previously_included): incl_abs_fname =3D os.path.abspath(incl_fname) # catch inclusion cycle inf =3D info @@ -151,7 +152,8 @@ def _include(self, include, info, incl_fname, previousl= y_included): f"can't read include file '{incl_fname}': {err.strerror}" ) from err =20 - def _check_pragma_list_of_str(self, name, value, info): + @staticmethod + def _check_pragma_list_of_str(name, value, info): if (not isinstance(value, list) or any([not isinstance(elt, str) for elt in value])): raise QAPISemError( --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621452117; cv=none; d=zohomail.com; s=zohoarc; b=fuBiJ3tlABJrhRp0g95YVXNuR0zBL5EYZJqyln2mgCD4iS7RzQHSD26ns+juxmeCAst5PBKus+8cuAO/81jNPMcKn6jgM/0PLjbxN0nCceCoF73wU8EaJ466hIuRY8zRhc94fJ2zztN/AZLhPR3GFWLMQ73SEuYxDjK/nT4GixU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621452117; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jVjMKiP7V5w3KL4bbzxvXjiaw3A5l5JPcZh2NCkf92s=; b=jaKFpQX2lZiS2Lb8xvOu/f39c11VuFOvF+39KLoLBdbmETF8uCpag3hU0OBUue1Q8XJls3VL/MgCn9qJzxzq+6kQ+Oy32P0TtSC9Jx5HRBE1Wft660/Y2zRYbaS8nMaj+kC2CycYgElrq6LoZYEZgsUaX0euXgPMTV2mgnSAa9A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621452117343529.1612968026002; Wed, 19 May 2021 12:21:57 -0700 (PDT) Received: from localhost ([::1]:45048 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRlT-0006eq-5n for importer2@patchew.org; Wed, 19 May 2021 15:21:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55294) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR77-00062j-Ke for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:32050) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6z-00021w-Of for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:13 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-48-YNIOhCC-NdK3FoIEzTtf-w-1; Wed, 19 May 2021 14:40:03 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 76FFA501ED; Wed, 19 May 2021 18:40:02 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5BE31690FB; Wed, 19 May 2021 18:40:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jVjMKiP7V5w3KL4bbzxvXjiaw3A5l5JPcZh2NCkf92s=; b=fKlTqGHp+rXgKRN6RqpZuJtskg1QhDtL5AX/y/PNRL7vj+YM+elbs7NJEh+Vy+c506MYK6 x89etroY1S4iU/6o7RFaCuZE/ULkrOquhZmh7uzFFim8Aeq/tXFJKOvM8wToaBRkhpQ8fq q9H56S2GGkW1+7jEuzV66uZvQd9AuKo= X-MC-Unique: YNIOhCC-NdK3FoIEzTtf-w-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 09/15] qapi: add must_match helper Date: Wed, 19 May 2021 14:39:45 -0400 Message-Id: <20210519183951.3946870-10-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Mypy cannot generally understand that these regex functions cannot possibly fail. Add a "must_match" helper that makes this clear for mypy. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/common.py | 8 +++++++- scripts/qapi/main.py | 6 ++---- scripts/qapi/parser.py | 13 +++++++------ 3 files changed, 16 insertions(+), 11 deletions(-) diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py index cbd3fd81d36..6ad1eeb61d4 100644 --- a/scripts/qapi/common.py +++ b/scripts/qapi/common.py @@ -12,7 +12,7 @@ # See the COPYING file in the top-level directory. =20 import re -from typing import Optional, Sequence +from typing import Match, Optional, Sequence =20 =20 #: Magic string that gets removed along with all space to its right. @@ -210,3 +210,9 @@ def gen_endif(ifcond: Sequence[str]) -> str: #endif /* %(cond)s */ ''', cond=3Difc) return ret + + +def must_match(pattern: str, string: str) -> Match[str]: + match =3D re.match(pattern, string) + assert match is not None + return match diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py index 703e7ed1ed5..f2ea6e0ce4a 100644 --- a/scripts/qapi/main.py +++ b/scripts/qapi/main.py @@ -8,11 +8,11 @@ """ =20 import argparse -import re import sys from typing import Optional =20 from .commands import gen_commands +from .common import must_match from .error import QAPIError from .events import gen_events from .introspect import gen_introspect @@ -22,9 +22,7 @@ =20 =20 def invalid_prefix_char(prefix: str) -> Optional[str]: - match =3D re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', prefix) - # match cannot be None, but mypy cannot infer that. - assert match is not None + match =3D must_match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', prefix) if match.end() !=3D len(prefix): return prefix[match.end()] return None diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 7c718661950..48137d3fbec 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -18,6 +18,7 @@ import os import re =20 +from .common import must_match from .error import QAPISemError, QAPISourceError from .source import QAPISourceInfo =20 @@ -238,8 +239,8 @@ def accept(self, skip_comment=3DTrue): elif not self.tok.isspace(): # Show up to next structural, whitespace or quote # character - match =3D re.match('[^[\\]{}:,\\s\'"]+', - self.src[self.cursor-1:]) + match =3D must_match('[^[\\]{}:,\\s\'"]+', + self.src[self.cursor-1:]) raise QAPIParseError(self, "stray '%s'" % match.group(0)) =20 def get_members(self): @@ -369,7 +370,7 @@ def append(self, line): # Strip leading spaces corresponding to the expected indent le= vel # Blank lines are always OK. if line: - indent =3D re.match(r'\s*', line).end() + indent =3D must_match(r'\s*', line).end() if indent < self._indent: raise QAPIParseError( self._parser, @@ -505,7 +506,7 @@ def _append_args_line(self, line): # from line and replace it with spaces so that 'f' has the # same index as it did in the original line and can be # handled the same way we will handle following lines. - indent =3D re.match(r'@\S*:\s*', line).end() + indent =3D must_match(r'@\S*:\s*', line).end() line =3D line[indent:] if not line: # Line was just the "@arg:" header; following lines @@ -540,7 +541,7 @@ def _append_features_line(self, line): # from line and replace it with spaces so that 'f' has the # same index as it did in the original line and can be # handled the same way we will handle following lines. - indent =3D re.match(r'@\S*:\s*', line).end() + indent =3D must_match(r'@\S*:\s*', line).end() line =3D line[indent:] if not line: # Line was just the "@arg:" header; following lines @@ -586,7 +587,7 @@ def _append_various_line(self, line): # from line and replace it with spaces so that 'f' has the # same index as it did in the original line and can be # handled the same way we will handle following lines. - indent =3D re.match(r'\S*:\s*', line).end() + indent =3D must_match(r'\S*:\s*', line).end() line =3D line[indent:] if not line: # Line was just the "Section:" header; following lines --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621450620; cv=none; d=zohomail.com; s=zohoarc; b=gjNwgEW+lAUOkol43u/TzBLQwnSU0IedqYG3glxQg5fkLJ7jDlqllCUeDJV+YQ9WMgnUxb8EJ+Dc3qvXrwFV7X5f2mN35f+9MkRzE9xp4UuaF2iIa+Q3ygAlTFgttYbRJkJu1mDUhVkU5JbTuayNlR8h+KJxrgm5NAW1YXbNTv0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621450620; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=gPG/1IUzw5ZpqclOqI/iTxWDhn2n18qGcScieeJ3hL8=; b=aBsEp5D3zSbJ8GzvDAFQPBo4gZFaSzro8YQQjgliuKib4ey2w3FxrkP95SHjGWzessfn5NGB4vNSysYF9zTSPW2H8LC0A7UF7yYo/TaFy1V4EDL+KZWLV49LnzvIG1cqjJxDyjceZX7PEZbDra+0F332uGUwp4ym7I3Kpliuz3c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621450620233555.8543719899303; Wed, 19 May 2021 11:57:00 -0700 (PDT) Received: from localhost ([::1]:57624 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRNL-0007pI-5H for importer2@patchew.org; Wed, 19 May 2021 14:56:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55354) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR7H-00064k-RF for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:56415) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR75-00024N-9z for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:23 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-33-r7zBPsnsP8u95wgO54FHSg-1; Wed, 19 May 2021 14:40:08 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 60815801107; Wed, 19 May 2021 18:40:07 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id AC28769FA4; Wed, 19 May 2021 18:40:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449610; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gPG/1IUzw5ZpqclOqI/iTxWDhn2n18qGcScieeJ3hL8=; b=VeZQjC4K4BnvLjzoobj8BToXTYmvcJaaOOGjIfmtfb64dEUcFb39w1xaaCuzK54LTPz8+s N3XYgF5Mn7M29j710YvHSaajVYSqNtJA38la8jD80BmN786Kzy2Q5ONXTWT+fEl8au7pi4 9uLY8i3E40KuWqO1lhyIGddiTOBkYTk= X-MC-Unique: r7zBPsnsP8u95wgO54FHSg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 10/15] qapi/parser: Fix token membership tests when token can be None Date: Wed, 19 May 2021 14:39:46 -0400 Message-Id: <20210519183951.3946870-11-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" When the token can be None (EOF), we can't use 'x in "abc"' style membership tests to group types of tokens together, because 'None in "abc"' is a TypeError. Easy enough to fix. (Use a tuple: It's neither a static typing error nor a runtime error to check for None in Tuple[str, ...]) Add tests to prevent a regression. (Note: they cannot be added prior to this fix, as the unhandled stack trace will not match test output in the CI system.) Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 5 +++-- tests/qapi-schema/meson.build | 2 ++ tests/qapi-schema/missing-array-rsqb.err | 1 + tests/qapi-schema/missing-array-rsqb.json | 1 + tests/qapi-schema/missing-array-rsqb.out | 0 tests/qapi-schema/missing-object-member-element.err | 1 + tests/qapi-schema/missing-object-member-element.json | 1 + tests/qapi-schema/missing-object-member-element.out | 0 8 files changed, 9 insertions(+), 2 deletions(-) create mode 100644 tests/qapi-schema/missing-array-rsqb.err create mode 100644 tests/qapi-schema/missing-array-rsqb.json create mode 100644 tests/qapi-schema/missing-array-rsqb.out create mode 100644 tests/qapi-schema/missing-object-member-element.err create mode 100644 tests/qapi-schema/missing-object-member-element.json create mode 100644 tests/qapi-schema/missing-object-member-element.out diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 48137d3fbec..9f980f75139 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -275,7 +275,7 @@ def get_values(self): if self.tok =3D=3D ']': self.accept() return expr - if self.tok not in "{['tf": + if self.tok not in tuple("{['tf"): raise QAPIParseError( self, "expected '{', '[', ']', string, or boolean") while True: @@ -294,7 +294,8 @@ def get_expr(self): elif self.tok =3D=3D '[': self.accept() expr =3D self.get_values() - elif self.tok in "'tf": + elif self.tok in tuple("'tf"): + assert isinstance(self.val, (str, bool)) expr =3D self.val self.accept() else: diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build index dc448e8f74d..9e8f658ce38 100644 --- a/tests/qapi-schema/meson.build +++ b/tests/qapi-schema/meson.build @@ -134,9 +134,11 @@ schemas =3D [ 'indented-expr.json', 'leading-comma-list.json', 'leading-comma-object.json', + 'missing-array-rsqb.json', 'missing-colon.json', 'missing-comma-list.json', 'missing-comma-object.json', + 'missing-object-member-element.json', 'missing-type.json', 'nested-struct-data.json', 'nested-struct-data-invalid-dict.json', diff --git a/tests/qapi-schema/missing-array-rsqb.err b/tests/qapi-schema/m= issing-array-rsqb.err new file mode 100644 index 00000000000..b5f58b8c12a --- /dev/null +++ b/tests/qapi-schema/missing-array-rsqb.err @@ -0,0 +1 @@ +missing-array-rsqb.json:1:44: expected '{', '[', string, or boolean diff --git a/tests/qapi-schema/missing-array-rsqb.json b/tests/qapi-schema/= missing-array-rsqb.json new file mode 100644 index 00000000000..7fca1df923c --- /dev/null +++ b/tests/qapi-schema/missing-array-rsqb.json @@ -0,0 +1 @@ +['Daisy,', 'Daisy,', 'Give me your answer', diff --git a/tests/qapi-schema/missing-array-rsqb.out b/tests/qapi-schema/m= issing-array-rsqb.out new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/qapi-schema/missing-object-member-element.err b/tests/qa= pi-schema/missing-object-member-element.err new file mode 100644 index 00000000000..c08a3dc307f --- /dev/null +++ b/tests/qapi-schema/missing-object-member-element.err @@ -0,0 +1 @@ +missing-object-member-element.json:1:8: expected '{', '[', string, or bool= ean diff --git a/tests/qapi-schema/missing-object-member-element.json b/tests/q= api-schema/missing-object-member-element.json new file mode 100644 index 00000000000..f52d0106f31 --- /dev/null +++ b/tests/qapi-schema/missing-object-member-element.json @@ -0,0 +1 @@ +{'key': diff --git a/tests/qapi-schema/missing-object-member-element.out b/tests/qa= pi-schema/missing-object-member-element.out new file mode 100644 index 00000000000..e69de29bb2d --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621452483; cv=none; d=zohomail.com; s=zohoarc; b=bgeeuOuTq04+LMTrwvOpO4/frfva5Soqcox3WSFqsjOm2r8guX7lw+uTsdCQh6ytnoSl2yVxJfvVTlwp4dtww5S7uz+N/P7aWHSxwafCn/8Fq9VK9uho7K4Ig7jEHPxu9yAt/qVzZ+k6cwmo8Y5K2JAyZ0Y90Ft4x5kg8MB9ey4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621452483; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5Won4SmCvj7mYBAtSANGNEVYeQbkzdlBm3wPfV06Ta4=; b=PoVdkcid1OhbhPkfXZd69LUl1Cs6Sl8vZlJrmGBwRAa35ecqX21b2XLnhHMKclnMa7qXXfKzo93XGb7EGmdSllPy1DFal/51zPbH3RqmNx3jBRcFFgE1Fxom75YdYa/0QGb+52YfYJioVfsYd05ELaYkkUftkp3tOy4ZmU18IRg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621452483536634.7926365756555; Wed, 19 May 2021 12:28:03 -0700 (PDT) Received: from localhost ([::1]:42080 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRrO-0007gy-BF for importer2@patchew.org; Wed, 19 May 2021 15:28:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55358) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR7H-00064m-T0 for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:25722) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR77-00025A-Bl for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:23 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-173-qm4z5I1QOxyxnI7Cl7jRnw-1; Wed, 19 May 2021 14:40:09 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 416411007477; Wed, 19 May 2021 18:40:08 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 88EC450FA2; Wed, 19 May 2021 18:40:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5Won4SmCvj7mYBAtSANGNEVYeQbkzdlBm3wPfV06Ta4=; b=DAdU2Jt/x58Qu1xw985/I6rZvB42ZzVSfUzzQy4OzRr9GeqlmHXTDDidMvH3MFvjYP4+2R eIlk0lywq6dfM6HY2/Mv0IEEki7efRh/Mj/7tqRTDdj8RR8uqp7pN+9EZboNBBlZi6UOUF QGjLjwvx/258q2YH2dXr1KaknLFpUMc= X-MC-Unique: qm4z5I1QOxyxnI7Cl7jRnw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 11/15] qapi/parser: Rework _check_pragma_list_of_str as a TypeGuard Date: Wed, 19 May 2021 14:39:47 -0400 Message-Id: <20210519183951.3946870-12-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" TypeGuards wont exist in Python proper until 3.10. Ah well. We can hack up our own by declaring this function to return the type we claim it checks for and using this to safely downcast object -> List[str]. In so doing, I bring this function under _pragma so it can use the 'info' object in its closure. Having done this, _pragma also now no longer needs to take a 'self' parameter, so drop it. To help with line-length, and with the context evident from its new scope, rename the function to the shorter check_list_str(). Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- I left (name, value) as args to avoid creating a fully magic "macro", because this looked simply too weird: info.pragma.foobar =3D check_list_str() and it looked more reasonable as: info.pragma.foobar =3D check_list_str(name, value) Signed-off-by: John Snow --- scripts/qapi/parser.py | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 9f980f75139..8a58e1228f0 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -17,6 +17,7 @@ from collections import OrderedDict import os import re +from typing import List =20 from .common import must_match from .error import QAPISemError, QAPISourceError @@ -154,28 +155,29 @@ def _include(include, info, incl_fname, previously_in= cluded): ) from err =20 @staticmethod - def _check_pragma_list_of_str(name, value, info): - if (not isinstance(value, list) - or any([not isinstance(elt, str) for elt in value])): - raise QAPISemError( - info, - "pragma %s must be a list of strings" % name) + def _pragma(name, value, info): + + def check_list_str(name, value) -> List[str]: + if (not isinstance(value, list) or + any([not isinstance(elt, str) for elt in value])): + raise QAPISemError( + info, + "pragma %s must be a list of strings" % name) + return value + + pragma =3D info.pragma =20 - def _pragma(self, name, value, info): if name =3D=3D 'doc-required': if not isinstance(value, bool): raise QAPISemError(info, "pragma 'doc-required' must be boolean") - info.pragma.doc_required =3D value + pragma.doc_required =3D value elif name =3D=3D 'command-name-exceptions': - self._check_pragma_list_of_str(name, value, info) - info.pragma.command_name_exceptions =3D value + pragma.command_name_exceptions =3D check_list_str(name, value) elif name =3D=3D 'command-returns-exceptions': - self._check_pragma_list_of_str(name, value, info) - info.pragma.command_returns_exceptions =3D value + pragma.command_returns_exceptions =3D check_list_str(name, val= ue) elif name =3D=3D 'member-name-exceptions': - self._check_pragma_list_of_str(name, value, info) - info.pragma.member_name_exceptions =3D value + pragma.member_name_exceptions =3D check_list_str(name, value) else: raise QAPISemError(info, "unknown pragma '%s'" % name) =20 --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621451493; cv=none; d=zohomail.com; s=zohoarc; b=KSAjEouZ1Xg98BqtvyKCJl8qkl1cE2IOzJhkKc8BmEZbWr6FugdScGC9PHwGKw6u5PV0ClSGTWeNT7XpJdo4oGKXLgL3R51fjz5lEFcAWym6frvjBc4pQBgnQ6ZGnZDPOZGv8gd+qrDNcTOKwqqFkviHNhyjdHJ3+QKht+AyffY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621451493; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CaB/dKZi3LA71EToFZxutHlVaVmHVneNDElkORGEuDI=; b=h+17yTYk2COB5+3uu1p1LS7Yl6wlT9V2IczZGIYfI09YTEQuFk/3EIjh1Tx6BHGN7HaswrUh7+RRrmDM/GF2VT0AR+ZAbO0wkHjCepul/nPy4G/kDcP8RDODAalu0F+33JgWSrSzNDDW5qgotOvHIYyHgR75JP7ZO5OGF/aL1K8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621451493856854.3580013530768; Wed, 19 May 2021 12:11:33 -0700 (PDT) Received: from localhost ([::1]:44324 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRbQ-0003Cv-Ki for importer2@patchew.org; Wed, 19 May 2021 15:11:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55390) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR7K-00067g-C0 for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47750) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR78-00025M-Fl for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:25 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-303-zSc9SlF5OsGj90YO8vgqzw-1; Wed, 19 May 2021 14:40:10 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 74858501E8; Wed, 19 May 2021 18:40:09 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 67F6D50FA2; Wed, 19 May 2021 18:40:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CaB/dKZi3LA71EToFZxutHlVaVmHVneNDElkORGEuDI=; b=KAafdv6YT6iX8KBxZC2a9+Uujy5tshozp9/CU77t/tl2lVhG53Bqj42IDCnM6dxyNbf10t UdglRpSa/Uf0b5+FMnuUPns/YyGXFRmjMCBQZ6sSt0+oEQz9krhdJXuRF55GoYubhdHB9p moiRj/y1cr9T5+gkAVCcHrh8bd8QZ4Y= X-MC-Unique: zSc9SlF5OsGj90YO8vgqzw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 12/15] qapi/parser: add type hint annotations Date: Wed, 19 May 2021 14:39:48 -0400 Message-Id: <20210519183951.3946870-13-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Annotations do not change runtime behavior. This commit *only* adds annotations. (Annotations for QAPIDoc are in a forthcoming commit.) Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 58 +++++++++++++++++++++++++++--------------- 1 file changed, 38 insertions(+), 20 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 8a58e1228f0..419e36c8702 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -17,16 +17,26 @@ from collections import OrderedDict import os import re -from typing import List +from typing import ( + Dict, + List, + Optional, + Set, + Union, +) =20 from .common import must_match from .error import QAPISemError, QAPISourceError from .source import QAPISourceInfo =20 =20 +# Return value alias for get_expr(). +_ExprValue =3D Union[List[object], Dict[str, object], str, bool] + + class QAPIParseError(QAPISourceError): """Error class for all QAPI schema parsing errors.""" - def __init__(self, parser, msg): + def __init__(self, parser: 'QAPISchemaParser', msg: str): col =3D 1 for ch in parser.src[parser.line_pos:parser.pos]: if ch =3D=3D '\t': @@ -38,7 +48,10 @@ def __init__(self, parser, msg): =20 class QAPISchemaParser: =20 - def __init__(self, fname, previously_included=3DNone, incl_info=3DNone= ): + def __init__(self, + fname: str, + previously_included: Optional[Set[str]] =3D None, + incl_info: Optional[QAPISourceInfo] =3D None): self._fname =3D fname self._included =3D previously_included or set() self._included.add(os.path.abspath(self._fname)) @@ -46,20 +59,20 @@ def __init__(self, fname, previously_included=3DNone, i= ncl_info=3DNone): =20 # Lexer state (see `accept` for details): self.info =3D QAPISourceInfo(self._fname, incl_info) - self.tok =3D None + self.tok: Union[None, str] =3D None self.pos =3D 0 self.cursor =3D 0 - self.val =3D None + self.val: Optional[Union[bool, str]] =3D None self.line_pos =3D 0 =20 # Parser output: - self.exprs =3D [] - self.docs =3D [] + self.exprs: List[Dict[str, object]] =3D [] + self.docs: List[QAPIDoc] =3D [] =20 # Showtime! self._parse() =20 - def _parse(self): + def _parse(self) -> None: cur_doc =3D None =20 # May raise OSError; allow the caller to handle it. @@ -125,7 +138,7 @@ def _parse(self): self.reject_expr_doc(cur_doc) =20 @staticmethod - def reject_expr_doc(doc): + def reject_expr_doc(doc: Optional['QAPIDoc']) -> None: if doc and doc.symbol: raise QAPISemError( doc.info, @@ -133,10 +146,14 @@ def reject_expr_doc(doc): % doc.symbol) =20 @staticmethod - def _include(include, info, incl_fname, previously_included): + def _include(include: str, + info: QAPISourceInfo, + incl_fname: str, + previously_included: Set[str] + ) -> Optional['QAPISchemaParser']: incl_abs_fname =3D os.path.abspath(incl_fname) # catch inclusion cycle - inf =3D info + inf: Optional[QAPISourceInfo] =3D info while inf: if incl_abs_fname =3D=3D os.path.abspath(inf.fname): raise QAPISemError(info, "inclusion loop for %s" % include) @@ -155,9 +172,9 @@ def _include(include, info, incl_fname, previously_incl= uded): ) from err =20 @staticmethod - def _pragma(name, value, info): + def _pragma(name: str, value: object, info: QAPISourceInfo) -> None: =20 - def check_list_str(name, value) -> List[str]: + def check_list_str(name: str, value: object) -> List[str]: if (not isinstance(value, list) or any([not isinstance(elt, str) for elt in value])): raise QAPISemError( @@ -181,7 +198,7 @@ def check_list_str(name, value) -> List[str]: else: raise QAPISemError(info, "unknown pragma '%s'" % name) =20 - def accept(self, skip_comment=3DTrue): + def accept(self, skip_comment: bool =3D True) -> None: while True: self.tok =3D self.src[self.cursor] self.pos =3D self.cursor @@ -245,8 +262,8 @@ def accept(self, skip_comment=3DTrue): self.src[self.cursor-1:]) raise QAPIParseError(self, "stray '%s'" % match.group(0)) =20 - def get_members(self): - expr =3D OrderedDict() + def get_members(self) -> Dict[str, object]: + expr: Dict[str, object] =3D OrderedDict() if self.tok =3D=3D '}': self.accept() return expr @@ -272,8 +289,8 @@ def get_members(self): if self.tok !=3D "'": raise QAPIParseError(self, "expected string") =20 - def get_values(self): - expr =3D [] + def get_values(self) -> List[object]: + expr: List[object] =3D [] if self.tok =3D=3D ']': self.accept() return expr @@ -289,7 +306,8 @@ def get_values(self): raise QAPIParseError(self, "expected ',' or ']'") self.accept() =20 - def get_expr(self): + def get_expr(self) -> _ExprValue: + expr: _ExprValue if self.tok =3D=3D '{': self.accept() expr =3D self.get_members() @@ -305,7 +323,7 @@ def get_expr(self): self, "expected '{', '[', string, or boolean") return expr =20 - def get_doc(self, info): + def get_doc(self, info: QAPISourceInfo) -> List['QAPIDoc']: if self.val !=3D '##': raise QAPIParseError( self, "junk after '##' at start of documentation comment") --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621450913; cv=none; d=zohomail.com; s=zohoarc; b=fMwy3hsXf3NK6XpcN91TwL2lHZOUuvIukeloT8gQp0O440lLd0M1P7zXuoA6hRsOTEQNe4kIKNqJha0B2KKTa5Wlf4zeSw3KeWXGy3AWzbpjjROQP27aSw9JM0TK9wbub+jwiZAiwVWLYslq81tO6PhuNalC+BNeeH1JmT67NBY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621450913; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=B6Y8ZWp6g+1MHAY0qscGX+ZYMYfMFTLfF/Td92r+1fg=; b=oJU5uRy+BMnL88/GIs2Mwvbhs44HInNfXVvd7/7ePzV/Bah1tiIiJ6gIH5daiC2f1XCYA8riuSJg6o4kuUNSjMp6WC6tnE2nMDRx4CZjg8WI0VGdWsLjN3fY3POmfX7m3mXdf5d/1p6BtByU3PmDktpeFbcHIsrCacTsZRGtagM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 162145091348562.75835897871957; Wed, 19 May 2021 12:01:53 -0700 (PDT) Received: from localhost ([::1]:46614 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRS4-0002Qp-CK for importer2@patchew.org; Wed, 19 May 2021 15:01:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55392) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR7K-00068U-UL for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:20036) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR77-00025D-Hk for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:26 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-549-WeQ77B27OEeTeZlLphSvYA-1; Wed, 19 May 2021 14:40:11 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 37221100747E; Wed, 19 May 2021 18:40:10 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E7D019CAD; Wed, 19 May 2021 18:40:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B6Y8ZWp6g+1MHAY0qscGX+ZYMYfMFTLfF/Td92r+1fg=; b=TcjcwqCTqrOcv0Az/pcLSuwt7vwK1tZsKjHOkWLGaZXQ6TwEaATMMUtWVqJOv7BXP9ECMz SJ8KT7iuJcupbO5aZvGL9i2pp7JSbVh4F1AlY3DFWFypvn5/9sStPCr6FtwzlyYpKw5VhF 5961azmNx/pZ0JoLkJu0iNL2H7WRAL4= X-MC-Unique: WeQ77B27OEeTeZlLphSvYA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 13/15] qapi/parser: Remove superfluous list comprehension Date: Wed, 19 May 2021 14:39:49 -0400 Message-Id: <20210519183951.3946870-14-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" A generator suffices (and quiets a pylint warning). Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 419e36c8702..b0f18c0d176 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -176,7 +176,7 @@ def _pragma(name: str, value: object, info: QAPISourceI= nfo) -> None: =20 def check_list_str(name: str, value: object) -> List[str]: if (not isinstance(value, list) or - any([not isinstance(elt, str) for elt in value])): + any(not isinstance(elt, str) for elt in value)): raise QAPISemError( info, "pragma %s must be a list of strings" % name) --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621452227; cv=none; d=zohomail.com; s=zohoarc; b=Y+vphI8spRlpZTT3dum6qVNubStJZWVF8x+OdqizFk/eWrTYgYdwXdBp4jCS0cEbpD8KHB9x2opcsuTvXZgiQbttt2rjKgbarpbmQHiIVttiQrxa+jfVVSwQt0OzsCoMfZe/uus9i+ItrwEdK3QAcVo+S2inaaHYZP18qGjZXck= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621452227; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8QEcYwPpiAdzGnQtGtm+YvWw/XHUAr6+X1gfe0OtI4w=; b=QAXKNVoFeisKkXrm+1+e/QaN1Gi5lEB2WEBeFyuo7dD8TD0aw5axBZJlDdz7VeDGK0S6axNUIApUxNMEXg2WLh4zw191YKYiLcuj3pwjST0ZtV8fJHEWDpjzHKEfKxs4aoqJEuuhPuJIA2Ad2uI/vHjw4Xg2+cMJkVXDNlSL8r8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621452227560761.426045779867; Wed, 19 May 2021 12:23:47 -0700 (PDT) Received: from localhost ([::1]:52886 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRnG-0004Ax-Ha for importer2@patchew.org; Wed, 19 May 2021 15:23:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55424) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR7O-0006CL-3C for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:30922) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR78-00025O-HO for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:29 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-596-w66oqIL3Muu5B1qwVnkNMQ-1; Wed, 19 May 2021 14:40:12 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3613480ED96; Wed, 19 May 2021 18:40:11 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5DD0350FA2; Wed, 19 May 2021 18:40:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8QEcYwPpiAdzGnQtGtm+YvWw/XHUAr6+X1gfe0OtI4w=; b=BUDpdAKQZpRP02PXJZOmpg/Ml8XEN4FXUPCZg32nWDBl4UdhaiG6mcdE2htIqr8wSoqv7r iniSZ8bhsGNlRkIbJ6rxrrfgfQdq3xj/X98pPWVizdjhckqWwjqv2PHCdx37fnvmjlU6EZ Ruz7U45c/9YfU5QtZ+KRUDuJ7m0WQQw= X-MC-Unique: w66oqIL3Muu5B1qwVnkNMQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 14/15] qapi/parser: allow 'ch' variable name Date: Wed, 19 May 2021 14:39:50 -0400 Message-Id: <20210519183951.3946870-15-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We can have a two-letter variable name, as a treat. Signed-off-by: John Snow -- I don't want to use 'chr' or 'char', and in context 'ch' works well enough. I will assume that any possible future uses will also be obvious enough. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/pylintrc | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc index 88efbf71cb2..c5275d5f59b 100644 --- a/scripts/qapi/pylintrc +++ b/scripts/qapi/pylintrc @@ -43,6 +43,7 @@ good-names=3Di, _, fp, # fp =3D open(...) fd, # fd =3D os.open(...) + ch, =20 [VARIABLES] =20 --=20 2.30.2 From nobody Mon May 6 16:27:47 2024 Delivered-To: importer2@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621452107; cv=none; d=zohomail.com; s=zohoarc; b=ig5ezi5a970wAgIcK79QoyqgQwomHEniddAMeX+1R212zO5/xg1wlHXZgDCw5iPEUTL/CGyCX8IydoeW9NXFOOWX3pSGVQy4gWy7K8GSUDjWJjIksFxzXmKVCgMmqVtgnC7xR70ednVjfCkppSgr66M9nrCcdXfIVk8a0kr6pfY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621452107; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=sxJhdCOrXufnbDk8jQ94+7NPvX3C7enbjBd7xpEQY/Q=; b=c3s2VjQoiuxK9lAzb5xnSqGhFh+RGEp2PHc2iXnLF/tpSE2s1XUS/RnaVFqLyNBi8wKomiH3QDsYp2X8V81PZuGRFNdUtGr+Wqv//4KF2YCnXqhWulwdHl10n+KWj2Vi0z78Tamz1JKcD6gQeFNRFUq1TClPqQx9UW6YYG5/dcE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer2=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621452107222694.8571866974983; Wed, 19 May 2021 12:21:47 -0700 (PDT) Received: from localhost ([::1]:44614 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRlJ-0006NP-7C for importer2@patchew.org; Wed, 19 May 2021 15:21:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55404) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR7M-00069i-8Q for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:36758) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR7A-00025q-74 for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-519-RNRtnR4hPGOlq76xCuk6tw-1; Wed, 19 May 2021 14:40:13 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 97699501F4; Wed, 19 May 2021 18:40:12 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 77BEC690EF; Wed, 19 May 2021 18:40:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449615; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sxJhdCOrXufnbDk8jQ94+7NPvX3C7enbjBd7xpEQY/Q=; b=McTcNxopi6EIKHOT1P3UZsfbfON/xBvVMw6JEicni3PculjfoTNZHevXPCuORuJ6QB1w/6 XJQIzlfMeAa4cfvyzxk9p0uGqWfKX2KOrAo9IpVTv/wsBi0La0mCJccCFOA+fWKKVNpyuZ dfI2chuBTXCSpOf/WL5Yx7ZnXGl8V5U= X-MC-Unique: RNRtnR4hPGOlq76xCuk6tw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 15/15] qapi/parser: add docstrings Date: Wed, 19 May 2021 14:39:51 -0400 Message-Id: <20210519183951.3946870-16-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer2=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+importer2=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- scripts/qapi/parser.py | 69 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index b0f18c0d176..06167ed3e0a 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -47,7 +47,27 @@ def __init__(self, parser: 'QAPISchemaParser', msg: str): =20 =20 class QAPISchemaParser: + """ + Parse QAPI schema source. =20 + Parse a JSON-esque schema file and process directives. See + qapi-code-gen.txt section "Schema Syntax" for the exact syntax. + Grammatical validation is handled later by `expr.check_exprs()`. + + :param fname: Source file name. + :param previously_included: + The absolute names of previously included source files, + if being invoked from another parser. + :param incl_info: + `QAPISourceInfo` belonging to the parent module. + ``None`` implies this is the root module. + + :ivar exprs: Resulting parsed expressions. + :ivar docs: Resulting parsed documentation blocks. + + :raise OSError: For problems reading the root schema document. + :raise QAPIError: For errors in the schema source. + """ def __init__(self, fname: str, previously_included: Optional[Set[str]] =3D None, @@ -73,6 +93,11 @@ def __init__(self, self._parse() =20 def _parse(self) -> None: + """ + Parse the QAPI schema document. + + :return: None. Results are stored in ``.exprs`` and ``.docs``. + """ cur_doc =3D None =20 # May raise OSError; allow the caller to handle it. @@ -199,6 +224,50 @@ def check_list_str(name: str, value: object) -> List[s= tr]: raise QAPISemError(info, "unknown pragma '%s'" % name) =20 def accept(self, skip_comment: bool =3D True) -> None: + """ + Read and store the next token. + + :param skip_comment: + When false, return COMMENT tokens ("#"). + This is used when reading documentation blocks. + + :return: + None. Several instance attributes are updated instead: + + - ``.tok`` represents the token type. See below for values. + - ``.info`` describes the token's source location. + - ``.val`` is the token's value, if any. See below. + - ``.pos`` is the buffer index of the first character of + the token. + + * Single-character tokens: + + These are "{", "}", ":", ",", "[", and "]". ``.tok`` holds + the single character and ``.val`` is None. + + * Multi-character tokens: + + * COMMENT: + + This token is not normally returned by the lexer, but it can + be when ``skip_comment`` is False. ``.tok`` is "#", and + ``.val`` is a string including all chars until end-of-line, + including the "#" itself. + + * STRING: + + ``.tok`` is "'", the single quote. ``.val`` contains the + string, excluding the surrounding quotes. + + * TRUE and FALSE: + + ``.tok`` is either "t" or "f", ``.val`` will be the + corresponding bool value. + + * EOF: + + ``.tok`` and ``.val`` will both be None at EOF. + """ while True: self.tok =3D self.src[self.cursor] self.pos =3D self.cursor --=20 2.30.2