From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <gentoo-commits+bounces-1631970-garchives=archives.gentoo.org@lists.gentoo.org>
Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by finch.gentoo.org (Postfix) with ESMTPS id 2B4D91581D3
for <garchives@archives.gentoo.org>; Fri, 17 May 2024 06:26:05 +0000 (UTC)
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id 301C0E2A80;
Fri, 17 May 2024 06:26:03 +0000 (UTC)
Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits))
(No client certificate requested)
by pigeon.gentoo.org (Postfix) with ESMTPS id 170AFE2A80
for <gentoo-commits@lists.gentoo.org>; Fri, 17 May 2024 06:26:03 +0000 (UTC)
Received: from oystercatcher.gentoo.org (oystercatcher.gentoo.org [148.251.78.52])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits))
(No client certificate requested)
by smtp.gentoo.org (Postfix) with ESMTPS id 3F56933C84E
for <gentoo-commits@lists.gentoo.org>; Fri, 17 May 2024 06:26:02 +0000 (UTC)
Received: from localhost.localdomain (localhost [IPv6:::1])
by oystercatcher.gentoo.org (Postfix) with ESMTP id ECAC61AE6
for <gentoo-commits@lists.gentoo.org>; Fri, 17 May 2024 06:25:58 +0000 (UTC)
From: "Michał Górny" <mgorny@gentoo.org>
To: gentoo-commits@lists.gentoo.org
Content-Transfer-Encoding: 8bit
Content-type: text/plain; charset=UTF-8
Reply-To: gentoo-dev@lists.gentoo.org, "Michał Górny" <mgorny@gentoo.org>
Message-ID: <1715927148.bf9a8b7af96bb7161956fc8276d9d07f021bda8a.mgorny@gentoo>
Subject: [gentoo-commits] repo/gentoo:master commit in: eclass/
X-VCS-Repository: repo/gentoo
X-VCS-Files: eclass/cargo.eclass
X-VCS-Directories: eclass/
X-VCS-Committer: mgorny
X-VCS-Committer-Name: Michał Górny
X-VCS-Revision: bf9a8b7af96bb7161956fc8276d9d07f021bda8a
X-VCS-Branch: master
Date: Fri, 17 May 2024 06:25:58 +0000 (UTC)
Precedence: bulk
List-Post: <mailto:gentoo-commits@lists.gentoo.org>
List-Help: <mailto:gentoo-commits+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-commits+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-commits+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-commits.gentoo.org>
X-BeenThere: gentoo-commits@lists.gentoo.org
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-Archives-Salt: 62e29e99-852d-4f6b-9a1b-6ac8a1fd4643
X-Archives-Hash: 91eb9e9313a480d28a0ff6ce7aabbc7b
commit: bf9a8b7af96bb7161956fc8276d9d07f021bda8a
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sat May 11 18:53:28 2024 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Fri May 17 06:25:48 2024 +0000
URL: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bf9a8b7a
cargo.eclass: Optimize crate unpacking
Unpack crates in parallel using xargs to utilize multicore systems
better. Perform checksumming via a single sha256sum invocation.
For dev-python/watchfiles, this speeds up unpacking on my machine
from 2.6 s to 0.75 s (warm cache).
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
Closes: https://github.com/gentoo/gentoo/pull/36645
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
eclass/cargo.eclass | 56 +++++++++++++++++++++++++++++++----------------------
1 file changed, 33 insertions(+), 23 deletions(-)
diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index 0f2da982f60c..a685cd99fb38 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -329,40 +329,50 @@ _cargo_gen_git_config() {
cargo_src_unpack() {
debug-print-function ${FUNCNAME} "$@"
- mkdir -p "${ECARGO_VENDOR}" || die
- mkdir -p "${S}" || die
+ mkdir -p "${ECARGO_VENDOR}" "${S}" || die
local archive shasum pkg
+ local crates=()
for archive in ${A}; do
case "${archive}" in
*.crate)
- # when called by pkgdiff-mg, do not unpack crates
- [[ ${PKGBUMPING} == ${PVR} ]] && continue
-
- ebegin "Loading ${archive} into Cargo registry"
- tar -xf "${DISTDIR}"/${archive} -C "${ECARGO_VENDOR}/" || die
- # generate sha256sum of the crate itself as cargo needs this
- shasum=$(sha256sum "${DISTDIR}"/${archive} | cut -d ' ' -f 1)
- pkg=$(basename ${archive} .crate)
- cat <<- EOF > ${ECARGO_VENDOR}/${pkg}/.cargo-checksum.json
- {
- "package": "${shasum}",
- "files": {}
- }
- EOF
- # if this is our target package we need it in ${WORKDIR} too
- # to make ${S} (and handle any revisions too)
- if [[ ${P} == ${pkg}* ]]; then
- tar -xf "${DISTDIR}"/${archive} -C "${WORKDIR}" || die
- fi
- eend $?
+ crates+=( "${archive}" )
;;
*)
- unpack ${archive}
+ unpack "${archive}"
;;
esac
done
+ if [[ ${PKGBUMPING} != ${PVR} && ${crates[@]} ]]; then
+ pushd "${DISTDIR}" >/dev/null || die
+
+ ebegin "Unpacking crates"
+ printf '%s\0' "${crates[@]}" |
+ xargs -0 -P "$(makeopts_jobs)" -n 1 -t -- \
+ tar -x -C "${ECARGO_VENDOR}" -f
+ assert
+ eend $?
+
+ while read -d '' -r shasum archive; do
+ pkg=${archive%.crate}
+ cat <<- EOF > ${ECARGO_VENDOR}/${pkg}/.cargo-checksum.json || die
+ {
+ "package": "${shasum}",
+ "files": {}
+ }
+ EOF
+
+ # if this is our target package we need it in ${WORKDIR} too
+ # to make ${S} (and handle any revisions too)
+ if [[ ${P} == ${pkg}* ]]; then
+ tar -xf "${archive}" -C "${WORKDIR}" || die
+ fi
+ done < <(sha256sum -z "${crates[@]}" || die)
+
+ popd >/dev/null || die
+ fi
+
cargo_gen_config
}