From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1SHH6I-0000wC-GB for garchives@archives.gentoo.org; Mon, 09 Apr 2012 16:05:54 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 817B5E0C73; Mon, 9 Apr 2012 16:05:17 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id 4F16BE0C73 for ; Mon, 9 Apr 2012 16:05:17 +0000 (UTC) Received: from hornbill.gentoo.org (hornbill.gentoo.org [94.100.119.163]) (using TLSv1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 8C2E51B403D for ; Mon, 9 Apr 2012 16:05:16 +0000 (UTC) Received: from localhost.localdomain (localhost [127.0.0.1]) by hornbill.gentoo.org (Postfix) with ESMTP id 56B62E5402 for ; Mon, 9 Apr 2012 16:05:15 +0000 (UTC) From: "Kent Fredric" To: gentoo-commits@lists.gentoo.org Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Kent Fredric" Message-ID: <1333985366.7263e878fa21e0e29186e1586f2dd3fd255d1e3d.kent@gentoo> Subject: [gentoo-commits] proj/perl-overlay:master commit in: scripts/ X-VCS-Repository: proj/perl-overlay X-VCS-Files: scripts/package_map_all.pl X-VCS-Directories: scripts/ X-VCS-Committer: kent X-VCS-Committer-Name: Kent Fredric X-VCS-Revision: 7263e878fa21e0e29186e1586f2dd3fd255d1e3d X-VCS-Branch: master Date: Mon, 9 Apr 2012 16:05:15 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: quoted-printable X-Archives-Salt: b276f726-b49e-4592-9902-fa1213dd8702 X-Archives-Hash: 724b846ab747c0483536937ce51954b1 commit: 7263e878fa21e0e29186e1586f2dd3fd255d1e3d Author: Kent Fredric gmail com> AuthorDate: Mon Apr 9 15:29:26 2012 +0000 Commit: Kent Fredric gmail com> CommitDate: Mon Apr 9 15:29:26 2012 +0000 URL: http://git.overlays.gentoo.org/gitweb/?p=3Dproj/perl-overlay.= git;a=3Dcommit;h=3D7263e878 [scripts] Turn on GZIP encoding to make package_map_all faster --- scripts/package_map_all.pl | 14 +++++++++----- 1 files changed, 9 insertions(+), 5 deletions(-) diff --git a/scripts/package_map_all.pl b/scripts/package_map_all.pl index a22355b..8b37bd2 100755 --- a/scripts/package_map_all.pl +++ b/scripts/package_map_all.pl @@ -33,8 +33,6 @@ my $size =3D 300; my $metadata =3D $root->subdir( 'metadata', 'perl' ); my $distmap =3D $metadata->subdir('distmap'); =20 -#my $distinfo =3D $metadata->subdir('distinfo'); -$distinfo->mkpath(); my (@json_files) =3D grep { not $_->is_dir and $_->basename =3D~ /\.json= $/ } $distmap->children(); =20 use JSON; @@ -90,7 +88,10 @@ $ENV{WWW_MECH_NOCACHE} =3D 1; my $results_string =3D mcpan->ua->request( 'POST', mcpan->base_url . 'release/_search?search_type=3Dscan&scroll=3D30s&siz= e=3D' . $size, - { content =3D> $encoder->encode($search), } + { + headers =3D> { 'Accept-Encoding' =3D> 'gzip', }, + content =3D> $encoder->encode($search), + } ); =20 say $results_string->{content}; @@ -125,8 +126,11 @@ exit 0; =20 sub scroll { my ($id) =3D @_; - my $result =3D - mcpan->ua->request( 'GET', 'http://api.metacpan.org/_search/scroll/?= scroll=3D30s&size=3D' . $size . '&scroll_id=3D' . $id ); + my $result =3D mcpan->ua->request( + 'GET', + 'http://api.metacpan.org/_search/scroll/?scroll=3D30s&size=3D' . $si= ze . '&scroll_id=3D' . $id, + { headers =3D> { 'Accept-Encoding' =3D> 'gzip', } } + ); =20 my $data =3D $decoder->decode( $result->{content} ); return $data, $data->{_scroll_id};