From nobody Fri Apr 19 05:33:00 2024 Delivered-To: importer2@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=patchew-devel-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=patchew-devel-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1648126634; cv=none; d=zohomail.com; s=zohoarc; b=eyis6NSSBQEuYlSSyZpI0sIvTrcQV7N3ieiweutG35DEVS8rXy6r5LJPGSQsjUclEEwFBqobjYDHRpqtM4XDcvKTTrUzsaUfx9HzAn3BtVDJlRBeIdwSYN9plWGnnVxRSDJlPCXEvLC9QyoyeBLEd3q+R8w78S6/jMIV8wfTINY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1648126634; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=IK+33M5usbimglh4dLP4Xs8VNFtTnPAUNKV2PN1tF3A=; b=bNhiIiiSsXBwAdnGacA4XCoQuwVatUObgnzq873soCt+0fzmM662I3/2eDhmsqW98/pVL11LB0Lta/uTHy2lvFKaBazp0DaLDJhmD5l3jIvp4XrExQjIdifAfBIEetRIMydt0Vdm4mn5bRyykl6k1Gb18i9NABZPifySImxHCQA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=patchew-devel-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 16481266349831013.9254967471354; Thu, 24 Mar 2022 05:57:14 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-49-33IVSeiIOA-a-k5TrS052w-1; Thu, 24 Mar 2022 08:57:09 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 48ADF3C163EA; Thu, 24 Mar 2022 12:57:09 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 410507BA720; Thu, 24 Mar 2022 12:57:08 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8E61C194034E; Thu, 24 Mar 2022 12:57:08 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 97D71194034B for ; Thu, 24 Mar 2022 12:57:07 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 60DD1C44AE7; Thu, 24 Mar 2022 12:57:07 +0000 (UTC) Received: from avogadro.lan (unknown [10.39.195.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id BEC10C44AE1 for ; Thu, 24 Mar 2022 12:57:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648126633; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=IK+33M5usbimglh4dLP4Xs8VNFtTnPAUNKV2PN1tF3A=; b=il68wp0LA2UHVYqkVOCzDFqfPNHi+gG/OSRVUPBOv3MgijM+iOl4FcOSuxMcLXz5EoMgZG OjrqUsj6uZLVkCRXFZ6MonWmOzvvixnmGrvQOg1SYOh193V8IyKJDv+ORuX24LpxnbBFdN Bbhr4qOLhizPYmouVUVlulIvp+x/v6s= X-MC-Unique: 33IVSeiIOA-a-k5TrS052w-1 X-Original-To: patchew-devel@listman.corp.redhat.com From: Paolo Bonzini To: patchew-devel@redhat.com Date: Thu, 24 Mar 2022 13:57:05 +0100 Message-Id: <20220324125705.172568-1-pbonzini@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 Subject: [Patchew-devel] [PATCH] rest: remove count from pagination output X-BeenThere: patchew-devel@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchew development and discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchew-devel-bounces@redhat.com Sender: "Patchew-devel" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=patchew-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1648126636215100001 Content-Type: text/plain; charset="utf-8" The same problem as b67dbe8 ("series-list: optimize generation of page links", 2022-02-23) is also there in the REST output. Remove the overhead of the count query from REST API responses. --- api/{rest.py =3D> rest/__init__.py} | 7 +++-- api/rest/pagination.py | 49 +++++++++++++++++++++++++++++++ patchew/settings.py | 2 +- tests/test_diff.py | 2 +- tests/test_rest.py | 28 +++++++++--------- 5 files changed, 69 insertions(+), 19 deletions(-) rename api/{rest.py =3D> rest/__init__.py} (99%) create mode 100644 api/rest/pagination.py diff --git a/api/rest.py b/api/rest/__init__.py similarity index 99% rename from api/rest.py rename to api/rest/__init__.py index 8117ddc..4a4b047 100644 --- a/api/rest.py +++ b/api/rest/__init__.py @@ -14,8 +14,8 @@ from django.http import Http404, HttpResponseRedirect from django.template import loader =20 from mod import dispatch_module_hook -from .models import Project, ProjectResult, Message, MessageResult, Result -from .search import SearchEngine +from ..models import Project, ProjectResult, Message, MessageResult, Result +from ..search import SearchEngine from rest_framework import ( permissions, serializers, @@ -34,6 +34,7 @@ from rest_framework.fields import ( EmailField, ListField, ) +from rest_framework.pagination import LimitOffsetPagination from rest_framework.relations import HyperlinkedIdentityField from rest_framework.response import Response from rest_framework.views import APIView @@ -699,7 +700,7 @@ class MessagesViewSet(BaseMessageViewSet): pass # Fake paginator response. Note that there is no Location header. return Response( - OrderedDict([("count", len(results)), ("results", results)]), + OrderedDict([("results", results)]), status=3Dstatus.HTTP_201_CREATED if results else status.HTTP_2= 00_OK, ) =20 diff --git a/api/rest/pagination.py b/api/rest/pagination.py new file mode 100644 index 0000000..844ef6b --- /dev/null +++ b/api/rest/pagination.py @@ -0,0 +1,49 @@ +#!/usr/bin/env python3 +# +# Copyright 2022 Red Hat, Inc. +# +# Authors: +# Paolo Bonzini +# +# This work is licensed under the MIT License. Please see the LICENSE fil= e or +# http://opensource.org/licenses/MIT. + +from rest_framework.pagination import LimitOffsetPagination +from rest_framework.response import Response + + +# remove count from paginator + +# This is referred to in settings.py, putting it with the rest of api/rest= .py +# results in a circular dependency between modules. + + +class PatchewPagination(LimitOffsetPagination): + def get_paginated_response(self, data): + return Response( + { + "next": self.get_next_link(), + "previous": self.get_previous_link(), + "results": data, + } + ) + + def paginate_queryset(self, queryset, request, view=3DNone): + self.offset =3D self.get_offset(request) + self.limit =3D self.get_limit(request) + + # Get one extra element to check if there is a "next" page + q =3D list(queryset[self.offset : self.offset + self.limit + 1]) + self.count =3D self.offset + len(q) if len(q) else self.offset - 1 + q.pop() + + self.request =3D request + if self.count > self.limit and self.template is not None: + self.display_page_controls =3D True + + return q + + def get_paginated_response_schema(self, schema): + ret =3D super(PatchewPagination, self).get_paginated_response_sche= ma(schema) + del ret["properties"]["count"] + return ret diff --git a/patchew/settings.py b/patchew/settings.py index 8b19d19..0b8d695 100644 --- a/patchew/settings.py +++ b/patchew/settings.py @@ -70,7 +70,7 @@ REST_FRAMEWORK =3D { "rest_framework.authentication.TokenAuthentication", "rest_framework.authentication.SessionAuthentication", ), - "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.LimitOffsetPagi= nation", + "DEFAULT_PAGINATION_CLASS": "api.rest.pagination.PatchewPagination", "URL_FIELD_NAME": "resource_uri", "PAGE_SIZE": 50, "UPLOADED_FILES_USE_URL": True, diff --git a/tests/test_diff.py b/tests/test_diff.py index 6d3f311..dd74ec4 100755 --- a/tests/test_diff.py +++ b/tests/test_diff.py @@ -37,8 +37,8 @@ class DiffTest(PatchewTestCase): self.cli_import("0009-obsolete-by.mbox.gz") =20 resp =3D self.api_client.get(self.REST_BASE + "series/?q=3Dquorum") - self.assertEqual(resp.data["count"], 3) results =3D sorted(resp.data["results"], key=3Dlambda y: y["versio= n"]) + self.assertEqual(len(results), 3) self.assertEqual(results[0]["version"], 1) self.assertEqual(results[1]["version"], 2) self.assertEqual(results[2]["version"], 3) diff --git a/tests/test_rest.py b/tests/test_rest.py index 9d96fa4..d7877f6 100755 --- a/tests/test_rest.py +++ b/tests/test_rest.py @@ -53,7 +53,7 @@ class RestTest(PatchewTestCase): =20 def test_users(self): resp =3D self.api_client.get(self.REST_BASE + "users/") - self.assertEquals(resp.data["count"], 1) + self.assertEquals(len(resp.data["results"]), 1) self.assertEquals(resp.data["results"][0]["resource_uri"], self.US= ER_BASE) self.assertEquals(resp.data["results"][0]["username"], self.admin.= username) =20 @@ -64,7 +64,7 @@ class RestTest(PatchewTestCase): =20 def test_projects(self): resp =3D self.api_client.get(self.REST_BASE + "projects/") - self.assertEquals(resp.data["count"], 3) + self.assertEquals(len(resp.data["results"]), 3) self.assertEquals(resp.data["results"][0]["resource_uri"], self.PR= OJECT_BASE) self.assertEquals(resp.data["results"][0]["name"], "QEMU") self.assertEquals( @@ -231,7 +231,7 @@ class RestTest(PatchewTestCase): def test_project_results_list(self): resp1 =3D self.api_client.get(self.PROJECT_BASE) resp =3D self.api_client.get(resp1.data["results"]) - self.assertEqual(resp.data["count"], len(resp.data["results"])) + self.assertEqual(len(resp.data["results"]), len(resp.data["results= "])) =20 def test_series_single(self): resp =3D self.apply_and_retrieve( @@ -371,10 +371,10 @@ class RestTest(PatchewTestCase): ) =20 resp =3D self.api_client.get(self.REST_BASE + "series/") - self.assertEqual(resp.data["count"], 2) + self.assertEqual(len(resp.data["results"]), 2) =20 resp =3D self.api_client.get(self.PROJECT_BASE + "series/") - self.assertEqual(resp.data["count"], 2) + self.assertEqual(len(resp.data["results"]), 2) =20 resp =3D self.api_client.get(self.REST_BASE + "projects/12345/seri= es/") self.assertEqual(resp.status_code, 404) @@ -386,7 +386,7 @@ class RestTest(PatchewTestCase): "20160628014747.20971-1-famz@redhat.com", ) resp =3D self.api_client.get(resp1.data["results"]) - self.assertEqual(resp.data["count"], len(resp.data["results"])) + self.assertEqual(len(resp.data["results"]), len(resp.data["results= "])) =20 def test_series_search(self): resp1 =3D self.apply_and_retrieve( @@ -401,7 +401,7 @@ class RestTest(PatchewTestCase): ) =20 resp =3D self.api_client.get(self.REST_BASE + "series/?q=3Dquorum") - self.assertEqual(resp.data["count"], 1) + self.assertEqual(len(resp.data["results"]), 1) self.assertEqual( resp.data["results"][0]["resource_uri"], resp2.data["resource_= uri"] ) @@ -410,7 +410,7 @@ class RestTest(PatchewTestCase): self.assertEqual("patches" in resp.data["results"][0], False) =20 resp =3D self.api_client.get(self.REST_BASE + "series/?q=3Dproject= :QEMU") - self.assertEqual(resp.data["count"], 2) + self.assertEqual(len(resp.data["results"]), 2) =20 def cmp_result(a, expected): self.assertEqual(a["resource_uri"], expected["resource_uri"]) @@ -579,7 +579,7 @@ class RestTest(PatchewTestCase): self.REST_BASE + "messages/", data, content_type=3D"applicatio= n/json" ) self.assertEqual(resp.status_code, 201) - self.assertEqual(resp.data["count"], 2) + self.assertEqual(len(resp.data["results"]), 2) resp_get =3D self.api_client.get( self.PROJECT_BASE + "messages/20180223132311.26555-2-marcandre.lureau@redhat.com= /" @@ -604,7 +604,7 @@ class RestTest(PatchewTestCase): self.REST_BASE + "messages/", data, content_type=3D"message/rf= c822" ) self.assertEqual(resp.status_code, 201) - self.assertEqual(resp.data["count"], 2) + self.assertEqual(len(resp.data["results"]), 2) resp_get =3D self.api_client.get( self.PROJECT_BASE + "messages/20180223132311.26555-2-marcandre.lureau@redhat.com= /" @@ -639,7 +639,7 @@ class RestTest(PatchewTestCase): self.REST_BASE + "messages/", data, content_type=3D"message/rf= c822" ) self.assertEqual(resp.status_code, 200) - self.assertEqual(resp.data["count"], 0) + self.assertEqual(len(resp.data["results"]), 0) resp_get =3D self.api_client.get( self.PROJECT_BASE + "messages/20180223132311.26555-2-marcandre.lureau@redhat.com= /" @@ -662,7 +662,7 @@ class RestTest(PatchewTestCase): self.REST_BASE + "messages/", data, content_type=3D"message/rf= c822" ) self.assertEqual(resp.status_code, 201) - self.assertEqual(resp.data["count"], 1) + self.assertEqual(len(resp.data["results"]), 1) resp_get =3D self.api_client.get( self.PROJECT_BASE + "messages/20180223132311.26555-2-marcandre.lureau@redhat.com= /" @@ -684,7 +684,7 @@ class RestTest(PatchewTestCase): self.REST_BASE + "messages/", data, content_type=3D"message/rf= c822" ) self.assertEqual(resp.status_code, 201) - self.assertEqual(resp.data["count"], 2) + self.assertEqual(len(resp.data["results"]), 2) resp_get =3D self.api_client.get( self.PROJECT_BASE + "messages/20180223132311.26555-2-marcandre.lureau@redhat.com= /" @@ -754,7 +754,7 @@ class RestTest(PatchewTestCase): =20 message =3D series.data["message"] resp =3D self.api_client.get(message + "replies/") - self.assertEqual(resp.data["count"], 4) + self.assertEqual(len(resp.data["results"]), 4) self.assertEqual( resp.data["results"][0]["resource_uri"], self.PROJECT_BASE --=20 2.35.1 _______________________________________________ Patchew-devel mailing list Patchew-devel@redhat.com https://listman.redhat.com/mailman/listinfo/patchew-devel