[Patchew-devel] [PATCH v3] Text to json convertor for mbox

Shubham Jain posted 1 patch 5 years, 12 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/patchew-ci tags/patchew/20180428095017.46366-1-shubhamjain7495@gmail.com
api/rest.py        |   4 +-
mbox.py            |  18 +++++++++
tests/test_mbox.py | 110 +++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 129 insertions(+), 3 deletions(-)
[Patchew-devel] [PATCH v3] Text to json convertor for mbox
Posted by Shubham Jain 5 years, 12 months ago
- Mbox would now return a dictionary object which would help REST API deserialize easily into model object.
- Added test for text to json converter.

[WIP] POST endpoint for messages.

- It's a work in progress for creating new message which is improvement of legacy endpoint: "import"
---
 api/rest.py        |   4 +-
 mbox.py            |  18 +++++++++
 tests/test_mbox.py | 110 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 129 insertions(+), 3 deletions(-)

diff --git a/api/rest.py b/api/rest.py
index 381b569..cced462 100644
--- a/api/rest.py
+++ b/api/rest.py
@@ -286,7 +286,6 @@ class MessageSerializer(BaseMessageSerializer):
     class Meta:
         model = Message
         fields = BaseMessageSerializer.Meta.fields + ('mbox', )
-
     def get_mbox(self, obj):
         return obj.get_mbox()
     mbox = SerializerMethodField()
@@ -311,9 +310,8 @@ class StaticTextRenderer(renderers.BaseRenderer):
             return data
 
 class MessagesViewSet(ProjectMessagesViewSetMixin,
-                      BaseMessageViewSet):
+                      BaseMessageViewSet, mixins.CreateModelMixin):
     serializer_class = MessageSerializer
-
     @detail_route(renderer_classes=[StaticTextRenderer])
     def mbox(self, request, *args, **kwargs):
         message = self.get_object()
diff --git a/mbox.py b/mbox.py
index fe108f3..c391fb0 100644
--- a/mbox.py
+++ b/mbox.py
@@ -269,3 +269,21 @@ class MboxMessage(object):
         if c == 0:
             return True
         return False
+
+    def get_json(self):
+        """Return the JSON format of the mbox """
+        msg = {}
+        msg['message_id'] = self.get_message_id()
+        msg['in_reply_to'] = self.get_in_reply_to() or ""
+        msg['date'] = self.get_date()
+        msg['subject'] = self.get_subject()
+        msg['stripped_subject'] = self.get_subject(strip_tags=True)
+        msg['version'] = self.get_version()
+        msg['sender'] = self.get_from()
+        msg['recipients'] = self.get_to() + self.get_cc()
+        msg['prefixes']= self.get_prefixes()
+        msg['is_series_head'] = self.is_series_head()
+        msg['is_patch'] = self.is_patch()
+        msg['patch_num'] = self.get_num()[0]
+        msg['mbox'] = self.get_mbox()
+        return msg
diff --git a/tests/test_mbox.py b/tests/test_mbox.py
index 8493df7..5b7f10e 100755
--- a/tests/test_mbox.py
+++ b/tests/test_mbox.py
@@ -13,6 +13,7 @@ import sys
 import mbox
 sys.path.append(os.path.dirname(__file__))
 from patchewtest import PatchewTestCase, main
+import datetime
 
 class MboxTest(PatchewTestCase):
 
@@ -52,5 +53,114 @@ Virtualization:  qemu.org | libvirt.org
             msg = mbox.MboxMessage(f.read())
         self.assertTrue(msg.is_patch())
 
+    def test_get_json(self):
+        expected = {'message_id': '20160628014747.20971-1-famz@redhat.com', 
+                    'in_reply_to': '', 
+                    'date': datetime.datetime(2016, 6, 28, 1, 47, 47), 
+                    'subject': '[Qemu-devel] [PATCH] quorum: Only compile when supported', 
+                    'stripped_subject': 'quorum: Only compile when supported', 
+                    'version': 1, 
+                    'sender': ('Fam Zheng', 'famz@redhat.com'), 
+                    'recipients': [('qemu-devel@nongnu.org', 'qemu-devel@nongnu.org'), 
+                                   ('Kevin Wolf', 'kwolf@redhat.com'), 
+                                   ('Alberto Garcia', 'berto@igalia.com'),
+                                   ('qemu-block@nongnu.org', 'qemu-block@nongnu.org'), 
+                                   ('Max Reitz', 'mreitz@redhat.com')],
+                    'prefixes': ['Qemu-devel', 'PATCH'],
+                    'is_series_head': True, 
+                    'is_patch': True, 
+                    'patch_num': None,
+                    'mbox': 'Delivered-To: importer@patchew.org\nReceived-SPF: '
+                            'Pass (zoho.com: domain of qemu-devel-bounces@nongnu.org '
+                            'designates 208.118.235.17 as permitted sender )  client-ip: '
+                            '208.118.235.17\nReceived-SPF: pass (zoho.com: domain of '
+                            'gnu.org designates 208.118.235.17 as permitted sender) '
+                            'client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+'
+                            'importer=patchew.org@nongnu.org; helo=lists.gnu.org;\n'
+                            'Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>'
+                            '\nReceived: from lists.gnu.org (lists.gnu.org [208.118.235.17]) '
+                            'by mx.zohomail.com\n\twith SMTPS id 1467078971424862.8927889595075;'
+                            ' Mon, 27 Jun 2016 18:56:11 -0700 (PDT)\nReceived: from localhost '
+                            '([::1]:33689 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp '
+                            '(Exim 4.71)\n\t(envelope-from <qemu-devel-bounces+importer='
+                            'patchew.org@nongnu.org>)\n\tid 1bHi94-0006LP-Ok\n\tfor '
+                            'importer@patchew.org; Mon, 27 Jun 2016 21:48:58 -0400\nReceived: '
+                            'from eggs.gnu.org ([2001:4830:134:3::10]:53270)\n\tby lists.gnu.org'
+                            ' with esmtp (Exim 4.71)\n\t(envelope-from <famz@redhat.com>) id '
+                            '1bHi8E-0002Lm-KR\n\tfor qemu-devel@nongnu.org; Mon, 27 Jun 2016 '
+                            '21:48:07 -0400\nReceived: from Debian-exim by eggs.gnu.org with '
+                            'spam-scanned (Exim 4.71)\n\t(envelope-from <famz@redhat.com>) '
+                            'id 1bHi8D-0008T4-N7\n\tfor qemu-devel@nongnu.org; Mon, 27 Jun '
+                            '2016 21:48:06 -0400\nReceived: from mx1.redhat.com '
+                            '([209.132.183.28]:47972)\n\tby eggs.gnu.org with esmtp '
+                            '(Exim 4.71)\n\t(envelope-from <famz@redhat.com>)\n\tid '
+                            '1bHi86-0008SN-IZ; Mon, 27 Jun 2016 21:47:58 -0400\nReceived: '
+                            'from int-mx10.intmail.prod.int.phx2.redhat.com\n\t'
+                            '(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])\n\t'
+                            '(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 '
+                            '(256/256 bits))\n\t(No client certificate requested)\n\tby '
+                            'mx1.redhat.com (Postfix) with ESMTPS id BDB007F088;\n\tTue, '
+                            '28 Jun 2016 01:47:57 +0000 (UTC)\nReceived: '
+                            'from ad.usersys.redhat.com (dhcp-15-133.nay.redhat.com\n\t'
+                            '[10.66.15.133])\n\tby int-mx10.intmail.prod.int.phx2.redhat.com'
+                            ' (8.14.4/8.14.4) with ESMTP\n\tid u5S1lssT024908; Mon, 27 Jun '
+                            '2016 21:47:55 -0400\nFrom: Fam Zheng <famz@redhat.com>\nTo: '
+                            'qemu-devel@nongnu.org\nDate: Tue, 28 Jun 2016 09:47:47 '
+                            '+0800\nMessage-Id: <20160628014747.20971-1-famz@redhat.com>\n'
+                            'X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23\nX-Greylist: '
+                            'Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t'
+                            '(mx1.redhat.com [10.5.110.26]);\n\tTue, 28 Jun 2016 01:47:57 '
+                            '+0000 (UTC)\nX-detected-operating-system: by eggs.gnu.org: '
+                            'GNU/Linux 2.2.x-3.x [generic]\nX-Received-From: 209.132.183.28\n'
+                            'Subject: [Qemu-devel] [PATCH] quorum: Only compile when supported\n'
+                            'X-BeenThere: qemu-devel@nongnu.org\nX-Mailman-Version: 2.1.21\n'
+                            'Precedence: list\nList-Id: <qemu-devel.nongnu.org>\nList-Unsubscribe:'
+                            ' <https://lists.nongnu.org/mailman/options/qemu-devel>,\n\t'
+                            '<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>\n'
+                            'List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel/>\n'
+                            'List-Post: <mailto:qemu-devel@nongnu.org>\nList-Help: '
+                            '<mailto:qemu-devel-request@nongnu.org?subject=help>\nList-Subscribe:'
+                            ' <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,\n\t'
+                            '<mailto:qemu-devel-request@nongnu.org?subject=subscribe>\nCc:'
+                            ' Kevin Wolf <kwolf@redhat.com>, Alberto Garcia <berto@igalia.com>,'
+                            '\n\tqemu-block@nongnu.org, Max Reitz <mreitz@redhat.com>\nErrors-To:'
+                            ' qemu-devel-bounces+importer=patchew.org@nongnu.org\nSender: '
+                            '"Qemu-devel" <qemu-devel-bounces+importer=patchew.org@nongnu.org>\n'
+                            'X-ZohoMail-Owner: <20160628014747.20971-1-famz@redhat.com>+zmo_0_'
+                            '<qemu-devel-bounces+importer=patchew.org@nongnu.org>\n'
+                            'X-ZohoMail-Sender: 209.132.183.28\nX-ZohoMail: RSF_0  Z_629925259 '
+                            'SPT_1 Z_629926901 SPT_1  SS_1 SFPD SFPP UW2468 UB2468 ZFF-EB_1'
+                            ' COSF  ODL   SGR3_1_2_0_27046_53\nX-Zoho-Virus-Status: 2\n\n'
+                            'This was the only exceptional module init function that does '
+                            'something\nelse than a simple list of bdrv_register() calls, '
+                            'in all the block\ndrivers.\n\nThe qcrypto_hash_supports is actually'
+                            ' a static check, determined at\ncompile time.  Follow the '
+                            'block-job-$(CONFIG_FOO) convention for\nconsistency.\n\n'
+                            'Signed-off-by: Fam Zheng <famz@redhat.com>\n---\n block/'
+                            'Makefile.objs | 2 +-\n block/quorum.c      | 4 ----\n 2 files'
+                            ' changed, 1 insertion(+), 5 deletions(-)\n\ndiff --git a/block/'
+                            'Makefile.objs b/block/Makefile.objs\nindex 44a5416..c87d605 '
+                            '100644\n--- a/block/Makefile.objs\n+++ b/block/Makefile.objs\n@@'
+                            ' -3,7 +3,7 @@ block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o'
+                            ' qcow2-snapshot.o qcow2-c\n block-obj-y += qed.o qed-gencb.o '
+                            'qed-l2-cache.o qed-table.o qed-cluster.o\n block-obj-y += '
+                            'qed-check.o\n block-obj-$(CONFIG_VHDX) += vhdx.o vhdx-endian.o'
+                            ' vhdx-log.o\n-block-obj-y += quorum.o\n+block-obj-$(CONFIG_GNUTLS_HASH)'
+                            ' += quorum.o\n block-obj-y += parallels.o blkdebug.o blkverify.o '
+                            'blkreplay.o\n block-obj-y += block-backend.o snapshot.o qapi.o\n '
+                            'block-obj-$(CONFIG_WIN32) += raw-win32.o win32-aio.o\ndiff --git '
+                            'a/block/quorum.c b/block/quorum.c\nindex 331b726..18fbed8 100644\n'
+                            '--- a/block/quorum.c\n+++ b/block/quorum.c\n@@ -1113,10 +1113,6 @@'
+                            ' static BlockDriver bdrv_quorum = {\n \n static void bdrv_quorum_init'
+                            '(void)\n {\n-    if (!qcrypto_hash_supports(QCRYPTO_HASH_ALG_SHA256))'
+                            ' {\n-        /* SHA256 hash support is required for quorum device */\n-'
+                            '        return;\n-    }\n     bdrv_register(&bdrv_quorum);\n }\n \n-- \n'
+                            '2.9.0\n\n\n'}
+        dp = self.get_data_path("0001-simple-patch.mbox.gz")
+        with open(dp, "r") as f:
+            msg = mbox.MboxMessage(f.read()).get_json()
+        self.assertEqual(msg, expected)
+
+
 if __name__ == '__main__':
     main()
-- 
2.14.3 (Apple Git-98)

_______________________________________________
Patchew-devel mailing list
Patchew-devel@redhat.com
https://www.redhat.com/mailman/listinfo/patchew-devel
Re: [Patchew-devel] [PATCH v3] Text to json convertor for mbox
Posted by Shubham Jain 5 years, 11 months ago
I noticed two things after sending this.
First, we are returning a dictionary rather than a json object for text to
json converter. Returning a json dump of dictionary gives error of date
(date time object not json serialisable). Though POST can accept dictionary
object (I think), I should change the name to get_dictionary rather than
get_json.

Second, should we change the format of recipients to
[{"name":..,"address":..},..] while extracting recipients from mbox? and
similarly with sender? And the corresponding custom serialiser would like
something like this
recipients = AddressSerializer()
class AddressSerializer(serializers.Field):
    def to_representation(self,obj):
        ret = []
        for item in obj:
            ret.append({"name":item[0],"address":item[1]})
        return ret
    def to_internal_value(self, data):
        return [[x['name'],x['address']] for x in data]


On Sat, Apr 28, 2018 at 3:20 PM, Shubham Jain <shubhamjain7495@gmail.com>
wrote:

> - Mbox would now return a dictionary object which would help REST API
> deserialize easily into model object.
> - Added test for text to json converter.
>
> [WIP] POST endpoint for messages.
>
> - It's a work in progress for creating new message which is improvement of
> legacy endpoint: "import"
> ---
>  api/rest.py        |   4 +-
>  mbox.py            |  18 +++++++++
>  tests/test_mbox.py | 110 ++++++++++++++++++++++++++++++
> +++++++++++++++++++++++
>  3 files changed, 129 insertions(+), 3 deletions(-)
>
> diff --git a/api/rest.py b/api/rest.py
> index 381b569..cced462 100644
> --- a/api/rest.py
> +++ b/api/rest.py
> @@ -286,7 +286,6 @@ class MessageSerializer(BaseMessageSerializer):
>      class Meta:
>          model = Message
>          fields = BaseMessageSerializer.Meta.fields + ('mbox', )
> -
>      def get_mbox(self, obj):
>          return obj.get_mbox()
>      mbox = SerializerMethodField()
> @@ -311,9 +310,8 @@ class StaticTextRenderer(renderers.BaseRenderer):
>              return data
>
>  class MessagesViewSet(ProjectMessagesViewSetMixin,
> -                      BaseMessageViewSet):
> +                      BaseMessageViewSet, mixins.CreateModelMixin):
>      serializer_class = MessageSerializer
> -
>      @detail_route(renderer_classes=[StaticTextRenderer])
>      def mbox(self, request, *args, **kwargs):
>          message = self.get_object()
> diff --git a/mbox.py b/mbox.py
> index fe108f3..c391fb0 100644
> --- a/mbox.py
> +++ b/mbox.py
> @@ -269,3 +269,21 @@ class MboxMessage(object):
>          if c == 0:
>              return True
>          return False
> +
> +    def get_json(self):
> +        """Return the JSON format of the mbox """
> +        msg = {}
> +        msg['message_id'] = self.get_message_id()
> +        msg['in_reply_to'] = self.get_in_reply_to() or ""
> +        msg['date'] = self.get_date()
> +        msg['subject'] = self.get_subject()
> +        msg['stripped_subject'] = self.get_subject(strip_tags=True)
> +        msg['version'] = self.get_version()
> +        msg['sender'] = self.get_from()
> +        msg['recipients'] = self.get_to() + self.get_cc()
> +        msg['prefixes']= self.get_prefixes()
> +        msg['is_series_head'] = self.is_series_head()
> +        msg['is_patch'] = self.is_patch()
> +        msg['patch_num'] = self.get_num()[0]
> +        msg['mbox'] = self.get_mbox()
> +        return msg
> diff --git a/tests/test_mbox.py b/tests/test_mbox.py
> index 8493df7..5b7f10e 100755
> --- a/tests/test_mbox.py
> +++ b/tests/test_mbox.py
> @@ -13,6 +13,7 @@ import sys
>  import mbox
>  sys.path.append(os.path.dirname(__file__))
>  from patchewtest import PatchewTestCase, main
> +import datetime
>
>  class MboxTest(PatchewTestCase):
>
> @@ -52,5 +53,114 @@ Virtualization:  qemu.org | libvirt.org
>              msg = mbox.MboxMessage(f.read())
>          self.assertTrue(msg.is_patch())
>
> +    def test_get_json(self):
> +        expected = {'message_id': '20160628014747.20971-1-famz@redhat.com',
>
> +                    'in_reply_to': '',
> +                    'date': datetime.datetime(2016, 6, 28, 1, 47, 47),
> +                    'subject': '[Qemu-devel] [PATCH] quorum: Only compile
> when supported',
> +                    'stripped_subject': 'quorum: Only compile when
> supported',
> +                    'version': 1,
> +                    'sender': ('Fam Zheng', 'famz@redhat.com'),
> +                    'recipients': [('qemu-devel@nongnu.org', '
> qemu-devel@nongnu.org'),
> +                                   ('Kevin Wolf', 'kwolf@redhat.com'),
> +                                   ('Alberto Garcia', 'berto@igalia.com
> '),
> +                                   ('qemu-block@nongnu.org', '
> qemu-block@nongnu.org'),
> +                                   ('Max Reitz', 'mreitz@redhat.com')],
> +                    'prefixes': ['Qemu-devel', 'PATCH'],
> +                    'is_series_head': True,
> +                    'is_patch': True,
> +                    'patch_num': None,
> +                    'mbox': 'Delivered-To: importer@patchew.org\nReceived-SPF:
> '
> +                            'Pass (zoho.com: domain of
> qemu-devel-bounces@nongnu.org '
> +                            'designates 208.118.235.17 as permitted
> sender )  client-ip: '
> +                            '208.118.235.17\nReceived-SPF: pass (zoho.com:
> domain of '
> +                            'gnu.org designates 208.118.235.17 as
> permitted sender) '
> +                            'client-ip=208.118.235.17;
> envelope-from=qemu-devel-bounces+'
> +                            'importer=patchew.org@nongnu.org; helo=
> lists.gnu.org;\n'
> +                            'Return-Path: <qemu-devel-bounces+importer=p
> atchew.org@nongnu.org>'
> +                            '\nReceived: from lists.gnu.org (
> lists.gnu.org [208.118.235.17]) '
> +                            'by mx.zohomail.com\n\twith SMTPS id
> 1467078971424862.8927889595075;'
> +                            ' Mon, 27 Jun 2016 18:56:11 -0700
> (PDT)\nReceived: from localhost '
> +                            '([::1]:33689 helo=lists.gnu.org)\n\tby
> lists.gnu.org with esmtp '
> +                            '(Exim 4.71)\n\t(envelope-from
> <qemu-devel-bounces+importer='
> +                            'patchew.org@nongnu.org>)\n\tid
> 1bHi94-0006LP-Ok\n\tfor '
> +                            'importer@patchew.org; Mon, 27 Jun 2016
> 21:48:58 -0400\nReceived: '
> +                            'from eggs.gnu.org
> ([2001:4830:134:3::10]:53270)\n\tby lists.gnu.org'
> +                            ' with esmtp (Exim 4.71)\n\t(envelope-from <
> famz@redhat.com>) id '
> +                            '1bHi8E-0002Lm-KR\n\tfor
> qemu-devel@nongnu.org; Mon, 27 Jun 2016 '
> +                            '21:48:07 -0400\nReceived: from Debian-exim
> by eggs.gnu.org with '
> +                            'spam-scanned (Exim 4.71)\n\t(envelope-from <
> famz@redhat.com>) '
> +                            'id 1bHi8D-0008T4-N7\n\tfor
> qemu-devel@nongnu.org; Mon, 27 Jun '
> +                            '2016 21:48:06 -0400\nReceived: from
> mx1.redhat.com '
> +                            '([209.132.183.28]:47972)\n\tby eggs.gnu.org
> with esmtp '
> +                            '(Exim 4.71)\n\t(envelope-from <
> famz@redhat.com>)\n\tid '
> +                            '1bHi86-0008SN-IZ; Mon, 27 Jun 2016 21:47:58
> -0400\nReceived: '
> +                            'from int-mx10.intmail.prod.int.
> phx2.redhat.com\n\t'
> +                            '(int-mx10.intmail.prod.int.phx2.redhat.com
> [10.5.11.23])\n\t'
> +                            '(using TLSv1.2 with cipher
> ECDHE-RSA-AES256-GCM-SHA384 '
> +                            '(256/256 bits))\n\t(No client certificate
> requested)\n\tby '
> +                            'mx1.redhat.com (Postfix) with ESMTPS id
> BDB007F088;\n\tTue, '
> +                            '28 Jun 2016 01:47:57 +0000 (UTC)\nReceived: '
> +                            'from ad.usersys.redhat.com (
> dhcp-15-133.nay.redhat.com\n\t'
> +                            '[10.66.15.133])\n\tby
> int-mx10.intmail.prod.int.phx2.redhat.com'
> +                            ' (8.14.4/8.14.4) with ESMTP\n\tid
> u5S1lssT024908; Mon, 27 Jun '
> +                            '2016 21:47:55 -0400\nFrom: Fam Zheng <
> famz@redhat.com>\nTo: '
> +                            'qemu-devel@nongnu.org\nDate: Tue, 28 Jun
> 2016 09:47:47 '
> +                            '+0800\nMessage-Id: <
> 20160628014747.20971-1-famz@redhat.com>\n'
> +                            'X-Scanned-By: MIMEDefang 2.68 on
> 10.5.11.23\nX-Greylist: '
> +                            'Sender IP whitelisted, not delayed by
> milter-greylist-4.5.16\n\t'
> +                            '(mx1.redhat.com [10.5.110.26]);\n\tTue, 28
> Jun 2016 01:47:57 '
> +                            '+0000 (UTC)\nX-detected-operating-system:
> by eggs.gnu.org: '
> +                            'GNU/Linux 2.2.x-3.x
> [generic]\nX-Received-From: 209.132.183.28\n'
> +                            'Subject: [Qemu-devel] [PATCH] quorum: Only
> compile when supported\n'
> +                            'X-BeenThere: qemu-devel@nongnu.org\nX-Mailman-Version:
> 2.1.21\n'
> +                            'Precedence: list\nList-Id: <
> qemu-devel.nongnu.org>\nList-Unsubscribe:'
> +                            ' <https://lists.nongnu.org/
> mailman/options/qemu-devel>,\n\t'
> +                            '<mailto:qemu-devel-request@nongnu.org
> ?subject=unsubscribe>\n'
> +                            'List-Archive: <http://lists.nongnu.org/
> archive/html/qemu-devel/>\n'
> +                            'List-Post: <mailto:qemu-devel@nongnu.org>\nList-Help:
> '
> +                            '<mailto:qemu-devel-request@nongnu.org
> ?subject=help>\nList-Subscribe:'
> +                            ' <https://lists.nongnu.org/
> mailman/listinfo/qemu-devel>,\n\t'
> +                            '<mailto:qemu-devel-request@nongnu.org
> ?subject=subscribe>\nCc:'
> +                            ' Kevin Wolf <kwolf@redhat.com>, Alberto
> Garcia <berto@igalia.com>,'
> +                            '\n\tqemu-block@nongnu.org, Max Reitz <
> mreitz@redhat.com>\nErrors-To:'
> +                            ' qemu-devel-bounces+importer=pa
> tchew.org@nongnu.org\nSender: '
> +                            '"Qemu-devel" <qemu-devel-bounces+importer=p
> atchew.org@nongnu.org>\n'
> +                            'X-ZohoMail-Owner: <
> 20160628014747.20971-1-famz@redhat.com>+zmo_0_'
> +                            '<qemu-devel-bounces+importer=
> patchew.org@nongnu.org>\n'
> +                            'X-ZohoMail-Sender:
> 209.132.183.28\nX-ZohoMail: RSF_0  Z_629925259 '
> +                            'SPT_1 Z_629926901 SPT_1  SS_1 SFPD SFPP
> UW2468 UB2468 ZFF-EB_1'
> +                            ' COSF  ODL   SGR3_1_2_0_27046_53\nX-Zoho-Virus-Status:
> 2\n\n'
> +                            'This was the only exceptional module init
> function that does '
> +                            'something\nelse than a simple list of
> bdrv_register() calls, '
> +                            'in all the block\ndrivers.\n\nThe
> qcrypto_hash_supports is actually'
> +                            ' a static check, determined at\ncompile
> time.  Follow the '
> +                            'block-job-$(CONFIG_FOO) convention
> for\nconsistency.\n\n'
> +                            'Signed-off-by: Fam Zheng <famz@redhat.com>\n---\n
> block/'
> +                            'Makefile.objs | 2 +-\n block/quorum.c      |
> 4 ----\n 2 files'
> +                            ' changed, 1 insertion(+), 5
> deletions(-)\n\ndiff --git a/block/'
> +                            'Makefile.objs b/block/Makefile.objs\nindex
> 44a5416..c87d605 '
> +                            '100644\n--- a/block/Makefile.objs\n+++
> b/block/Makefile.objs\n@@'
> +                            ' -3,7 +3,7 @@ block-obj-y += qcow2.o
> qcow2-refcount.o qcow2-cluster.o'
> +                            ' qcow2-snapshot.o qcow2-c\n block-obj-y +=
> qed.o qed-gencb.o '
> +                            'qed-l2-cache.o qed-table.o qed-cluster.o\n
> block-obj-y += '
> +                            'qed-check.o\n block-obj-$(CONFIG_VHDX) +=
> vhdx.o vhdx-endian.o'
> +                            ' vhdx-log.o\n-block-obj-y +=
> quorum.o\n+block-obj-$(CONFIG_GNUTLS_HASH)'
> +                            ' += quorum.o\n block-obj-y += parallels.o
> blkdebug.o blkverify.o '
> +                            'blkreplay.o\n block-obj-y += block-backend.o
> snapshot.o qapi.o\n '
> +                            'block-obj-$(CONFIG_WIN32) += raw-win32.o
> win32-aio.o\ndiff --git '
> +                            'a/block/quorum.c b/block/quorum.c\nindex
> 331b726..18fbed8 100644\n'
> +                            '--- a/block/quorum.c\n+++ b/block/quorum.c\n@@
> -1113,10 +1113,6 @@'
> +                            ' static BlockDriver bdrv_quorum = {\n \n
> static void bdrv_quorum_init'
> +                            '(void)\n {\n-    if (!qcrypto_hash_supports(
> QCRYPTO_HASH_ALG_SHA256))'
> +                            ' {\n-        /* SHA256 hash support is
> required for quorum device */\n-'
> +                            '        return;\n-    }\n
>  bdrv_register(&bdrv_quorum);\n }\n \n-- \n'
> +                            '2.9.0\n\n\n'}
> +        dp = self.get_data_path("0001-simple-patch.mbox.gz")
> +        with open(dp, "r") as f:
> +            msg = mbox.MboxMessage(f.read()).get_json()
> +        self.assertEqual(msg, expected)
> +
> +
>  if __name__ == '__main__':
>      main()
> --
> 2.14.3 (Apple Git-98)
>
>
_______________________________________________
Patchew-devel mailing list
Patchew-devel@redhat.com
https://www.redhat.com/mailman/listinfo/patchew-devel
Re: [Patchew-devel] [PATCH v3] Text to json convertor for mbox
Posted by Paolo Bonzini 5 years, 11 months ago
On 28/04/2018 18:20, Shubham Jain wrote:
> I noticed two things after sending this. 
> First, we are returning a dictionary rather than a json object for text
> to json converter. Returning a json dump of dictionary gives error of
> date (date time object not json serialisable). Though POST can accept
> dictionary object (I think), I should change the name to get_dictionary
> rather than get_json.

Nicely spotted.  I think the date time object would not be accepted by
the REST API so it would be better to have a real JSON.  You can use
rest_framework itself to format the date/time.  For example

rest_framework.fields.DateTimeField().to_representation(datetime.datetime.now())

gives '2018-04-29T21:19:17.317160'.

Apart from this ussue, the "get_json()" part is more or less okay.
Let's move to the next part and then we'll pick it up again and put it
all together.

You can leave this patch in your branch and go on with the next...

> Second, should we change the format of recipients to
> [{"name":..,"address":..},..] while extracting recipients from mbox? and
> similarly with sender? And the corresponding custom serialiser would
> like something like this
> recipients = AddressSerializer()
> class AddressSerializer(serializers.Field):
>     def to_representation(self,obj):
>         ret = []
>         for item in obj:
>             ret.append({"name":item[0],"address":item[1]})
>         return ret
>     def to_internal_value(self, data):
>         return [[x['name'],x['address']] for x in data]

Yes, indeed!  That's the next step.  A couple things:

- deriving from Field means that you have to implement validation and
that's quite a lot of code.  Instead, you can derive from Serializer,
and the validation is implemented simply by declaring "name" and
"address" fields in the class---just like with any other serializer.
There is only one difference.  For Fields, "to_internal_vlaue_ converts
JSON input to a Python representation, but Serializers keep the
dictionary format after "to_internal_value".  See this example:

    import rest_framework.serializers
    import rest_framework.fields
    class TestSerializer(rest_framework.serializers.Serializer):
        x = rest_framework.serializers.DateTimeField()

    d = {'x': '2018-04-29T21:19:17.317160'}
    s = TestSerializer(data=d)
    s.is_valid()     # -> True
    s.data           # -> {'x': '2018-04-29T21:19:17.317160'}
    s.validated_data # -> x now is the key to a datetime.datetime object

In your case, Serializer's default implementation of "to_internal_value"
is good already, but you'll have to implement "create" , which converts
the Serializer's validated_data to the JSON array.

- you don't need the "for" loops: just declare

    recipients = AddressSerializer(many=True)
    sender = AddressSerializer()

and DRF will take care of serializing/deserializing lists! :)

- there is a special case where ['foo@bar.com', 'foo@bar.com'] leaves
out the "name" key.  So the serializer should behave like this:

    >>> import api.rest
    >>> first = ['Paolo', 'pbonzini@redhat.com']
    >>> s1 = api.rest.AddressSerializer(first)
    >>> data1 = s1.data
    >>> data1
          {'name': 'Paolo', 'address': 'pbonzini@redhat.com'}
    >>> second = ['pbonzini@redhat.com', 'pbonzini@redhat.com']
    >>> s2 = api.rest.AddressSerializer(second)
    >>> data2 = s2.data
    >>> data2
          {'address': 'pbonzini@redhat.com'}
    >>>
    >>> s1 = api.rest.AddressSerializer(data=data1)
    >>> s1.is_valid()
          True
    >>> s1.validated_data
          OrderedDict([('name', 'Paolo'),
                       ('address', 'pbonzini@redhat.com')])
    >>> s1.save()
          ['Paolo', 'pbonzini@redhat.com']
    >>> s2 = api.rest.AddressSerializer(data=data2)
    >>> s2.is_valid()
          True
    >>> s2.validated_data
          OrderedDict([('address', 'pbonzini@redhat.com')])

Now, apply to your tree my patch for JSONField; you can get it with

  patchew -s https://patchew.org/ \
     apply id:20180426083340.9816-1-pbonzini@redhat.com

... and then switching sender and recipients to AddressSerializer should
work with no other changes required.  The above sample interaction can
be easily converted to a testcase.  This will be your next change!

Thanks,

Paolo

_______________________________________________
Patchew-devel mailing list
Patchew-devel@redhat.com
https://www.redhat.com/mailman/listinfo/patchew-devel