From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <gentoo-commits+bounces-1182891-garchives=archives.gentoo.org@lists.gentoo.org>
Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by finch.gentoo.org (Postfix) with ESMTPS id A49161382C5
for <garchives@archives.gentoo.org>; Sun, 28 Jun 2020 10:46:50 +0000 (UTC)
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id EC204E089E;
Sun, 28 Jun 2020 10:46:49 +0000 (UTC)
Received: from smtp.gentoo.org (woodpecker.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by pigeon.gentoo.org (Postfix) with ESMTPS id C830FE089E
for <gentoo-commits@lists.gentoo.org>; Sun, 28 Jun 2020 10:46:49 +0000 (UTC)
Received: from oystercatcher.gentoo.org (unknown [IPv6:2a01:4f8:202:4333:225:90ff:fed9:fc84])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by smtp.gentoo.org (Postfix) with ESMTPS id BFB7734EBD3
for <gentoo-commits@lists.gentoo.org>; Sun, 28 Jun 2020 10:46:48 +0000 (UTC)
Received: from localhost.localdomain (localhost [IPv6:::1])
by oystercatcher.gentoo.org (Postfix) with ESMTP id 7C41D2B8
for <gentoo-commits@lists.gentoo.org>; Sun, 28 Jun 2020 10:46:47 +0000 (UTC)
From: "Ulrich Müller" <ulm@gentoo.org>
To: gentoo-commits@lists.gentoo.org
Content-Transfer-Encoding: 8bit
Content-type: text/plain; charset=UTF-8
Reply-To: gentoo-dev@lists.gentoo.org, "Ulrich Müller" <ulm@gentoo.org>
Message-ID: <1593334887.039b450670c4b7c632c0c4b52f3f4dfed3e7ae15.ulm@gentoo>
Subject: [gentoo-commits] proj/pms:master commit in: /
X-VCS-Repository: proj/pms
X-VCS-Files: dependencies.tex ebuild-env-vars.tex
X-VCS-Directories: /
X-VCS-Committer: ulm
X-VCS-Committer-Name: Ulrich Müller
X-VCS-Revision: 039b450670c4b7c632c0c4b52f3f4dfed3e7ae15
X-VCS-Branch: master
Date: Sun, 28 Jun 2020 10:46:47 +0000 (UTC)
Precedence: bulk
List-Post: <mailto:gentoo-commits@lists.gentoo.org>
List-Help: <mailto:gentoo-commits+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-commits+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-commits+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-commits.gentoo.org>
X-BeenThere: gentoo-commits@lists.gentoo.org
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-Archives-Salt: 09596a9b-2b23-4d23-b5c0-4d43def58900
X-Archives-Hash: 8900447cf6af4d1ad1e6efedeb12f6e1
commit: 039b450670c4b7c632c0c4b52f3f4dfed3e7ae15
Author: James Le Cuirot <chewi <AT> gentoo <DOT> org>
AuthorDate: Sun Jul 28 21:37:35 2019 +0000
Commit: Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Sun Jun 28 09:01:27 2020 +0000
URL: https://gitweb.gentoo.org/proj/pms.git/commit/?id=039b4506
Correct the definition of ESYSROOT as EPREFIX isn't always applicable
It was originally envisaged (but not stated in PMS) that SYSROOT would
only ever need to equal / or ROOT as a distinct SYSROOT would have no
benefit. A check was added to Portage to ensure this held. Myself, the
ChromiumOS team, and others have since been caught out by this check
when trying to bootstrap brand new systems from scratch. You cannot
bootstrap with no headers at all! The check will therefore be adjusted
to merely ensure that SYSROOT is / when ROOT is /.
There were differing assumptions about how prefixes applied to the
above. EPREFIX is traditionally something the user sets so some
thought that it would be applied to SYSROOT, regardless of the
latter's value. In order to honor the rule about there being no
distinct SYSROOT, this would mean that if SYSROOT is / then EPREFIX
would have to match BROOT. Despite that limitation, ESYSROOT was
written into PMS with a fixed value of ${SYSROOT}${EPREFIX}. Being
somewhat unfamiliar with prefix at the time, I didn't realise that
this view didn't align with what I'd had in mind and it was only when
I came to need a distinct SYSROOT that I realised there was a problem.
crossdev toolchains are installed to ${EPREFIX}/usr/${CHOST} but have
no further prefix appended and packages subsequently installed with
cross-emerge are placed in this location by setting ROOT. Bug #642604
recently revealed that the build system's prefix was being erroneously
duplicated on the end but I have now fixed this.
What if we want to bootstrap a brand new prefixed system using the
crossdev system as SYSROOT? This is the distinct SYSROOT case. The
problem is that there is no distinct variable for SYSROOT's prefix
and, as already stated, ESYSROOT is always ${SYSROOT}${EPREFIX}. We
therefore cannot do it! If the crossdev prefix is blank then ROOT's
must be blank too.
I also never intended to have the aforementioned limitation where
EPREFIX must match BROOT when SYSROOT is /. These are both entirely
artificial restrictions.
So how should it work instead? We originally intended for SYSROOT to
equal either / or ROOT so I imagined the prefix would automatically be
adjusted to match the prefix applicable at the matching location,
namely BROOT or EPREFIX. This is obviously more flexible than forcing
it to match EPREFIX.
What about the distinct SYSROOT case? With no distinct variable, we
have no way to explicitly set a prefix but this is likely only needed
when bootstrapping against crossdev systems, which are unprefixed by
nature. We therefore simply assume that the prefix is blank in this
case.
What about the cross-prefix case? Here, SYSROOT matches both / and
ROOT so which prefix do we choose? The bootstrap-prefix.sh script sets
flags to build against the target prefix so EPREFIX is used in this
case. This happens to fit the current definition of ESYSROOT anyway.
Legitimate concerns have been raised about building for a system with
a different prefix to the one you're building against. The only
binaries that leak from SYSROOT to ROOT are static libraries. Headers
from SYSROOT will obviously also influence how ROOT's binaries are
built. It is entirely possible that SYSROOT's prefix may leak through
a header but grepping /usr/include on my own main system reveals only
a few paths from a small handful of packages. pkg-config files
invariably include paths but these are almost always used at build
time, not runtime. A differing prefix would likely only occur in cases
involving core packages like the libc and kernel headers anyway. Also
consider that we have never prevented this from happening in the
past. It has always been possible to do "EPREFIX=/foo emerge bar" from
some system with a different prefix or no prefix at all. All we're
doing here is including the prefix (if any) in the ESYSROOT variable.
Should this warrant a new EAPI? I don't think so. All existing usage
of ESYSROOT that I have seen still fits with this new definition and
most of that usage has come from me. We're not even changing what the
variable is used for, just loosening the constraints around what it
can be set to.
If you have doubts about whether this makes sense or actually works in
practise, I have experimented with a prefixed system using all the
different combinations I could think of, including cross-compiling,
and it all worked as expected. Keep in mind that ESYSROOT is not magic
and currently isn't used very much. As such, neither the toolchain nor
pkg-config will use this sysroot if you don't explicitly tell them
to. For the former, I find CC="${CHOST}-gcc --sysroot=${ESYSROOT}"
works well. For the latter, crossdev installs a cross-pkg-config
wrapper but it is completely lacking prefix support at the moment. I
have fixes waiting on this change.
Signed-off-by: James Le Cuirot <chewi <AT> gentoo.org>
[Replaced "/" by "empty", reworded table cell in ebuild-env-vars.tex]
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>
dependencies.tex | 16 +++++++++++++++-
ebuild-env-vars.tex | 6 +++---
2 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/dependencies.tex b/dependencies.tex
index 3305b86..a641014 100644
--- a/dependencies.tex
+++ b/dependencies.tex
@@ -36,13 +36,27 @@
\midrule
Binary compatible with & \t{CBUILD} & \t{CHOST} & \t{CHOST} \\
Base unprefixed path & \t{/} & \t{\$\{SYSROOT\}} & \t{\$\{ROOT\}} \\
- Relevant offset-prefix & \t{\$\{BROOT\}} & \t{\$\{EPREFIX\}} & \t{\$\{EPREFIX\}} \\
+ Relevant offset-prefix & \t{\$\{BROOT\}} & See table~\ref{tab:depend-prefix}
+ & \t{\$\{EPREFIX\}} \\
Path combined with prefix & \t{\$\{BROOT\}} & \t{\$\{ESYSROOT\}} & \t{\$\{EROOT\}} \\
PM query command option & \t{-b} & \t{-d} & \t{-r} \\
\bottomrule
\end{tabular}
\end{centertable}
+\begin{centertable}{Prefix values for \t{DEPEND}}
+ \label{tab:depend-prefix}
+ \begin{tabular}{llll}
+ \toprule
+ If \t{SYSROOT} is: & \t{\$\{ROOT\}} & Empty, and \t{ROOT} is non-empty
+ & Other \\
+ \midrule
+ Then offset-prefix is: & \t{\$\{EPREFIX\}} & \t{\$\{BROOT\}} & Empty \\
+ And \t{ESYSROOT} is: & \t{\$\{EROOT\}} & \t{\$\{BROOT\}} & \t{\$\{SYSROOT\}} \\
+ \bottomrule
+ \end{tabular}
+\end{centertable}
+
There are three classes of dependencies supported by ebuilds:
\begin{compactitem}
diff --git a/ebuild-env-vars.tex b/ebuild-env-vars.tex
index ae8bd7d..117a19d 100644
--- a/ebuild-env-vars.tex
+++ b/ebuild-env-vars.tex
@@ -141,9 +141,9 @@ variable.
\t{ESYSROOT} &
Ditto &
No &
- Contains the concatenation of the paths in the \t{SYSROOT} and \t{EPREFIX} variables,
- for convenience. See also the \t{EPREFIX} variable. Only for EAPIs listed
- in table~\ref{tab:offset-env-vars-table} as supporting \t{ESYSROOT}. \\
+ Contains the concatenation of the \t{SYSROOT} path and applicable prefix value, as determined
+ by table~\ref{tab:depend-prefix}. Only for EAPIs listed in table~\ref{tab:offset-env-vars-table}
+ as supporting \t{ESYSROOT}. \\
\t{BROOT} &
Ditto &
No &
From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <gentoo-commits+bounces-1281135-garchives=archives.gentoo.org@lists.gentoo.org>
Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by finch.gentoo.org (Postfix) with ESMTPS id 301811382C5
for <garchives@archives.gentoo.org>; Thu, 13 May 2021 11:42:29 +0000 (UTC)
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id 86FA9E08D3;
Thu, 13 May 2021 11:42:28 +0000 (UTC)
Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by pigeon.gentoo.org (Postfix) with ESMTPS id 19C7DE08C3
for <gentoo-commits@lists.gentoo.org>; Thu, 13 May 2021 11:42:28 +0000 (UTC)
Received: from oystercatcher.gentoo.org (unknown [IPv6:2a01:4f8:202:4333:225:90ff:fed9:fc84])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by smtp.gentoo.org (Postfix) with ESMTPS id EF9ED335CE9
for <gentoo-commits@lists.gentoo.org>; Thu, 13 May 2021 11:42:26 +0000 (UTC)
Received: from localhost.localdomain (localhost [IPv6:::1])
by oystercatcher.gentoo.org (Postfix) with ESMTP id 6892775F
for <gentoo-commits@lists.gentoo.org>; Thu, 13 May 2021 11:42:25 +0000 (UTC)
From: "Ulrich Müller" <ulm@gentoo.org>
To: gentoo-commits@lists.gentoo.org
Content-Transfer-Encoding: 8bit
Content-type: text/plain; charset=UTF-8
Reply-To: gentoo-dev@lists.gentoo.org, "Ulrich Müller" <ulm@gentoo.org>
Message-ID: <1593334887.039b450670c4b7c632c0c4b52f3f4dfed3e7ae15.ulm@gentoo>
Subject: [gentoo-commits] proj/pms:eapi-8 commit in: /
X-VCS-Repository: proj/pms
X-VCS-Files: dependencies.tex ebuild-env-vars.tex
X-VCS-Directories: /
X-VCS-Committer: ulm
X-VCS-Committer-Name: Ulrich Müller
X-VCS-Revision: 039b450670c4b7c632c0c4b52f3f4dfed3e7ae15
X-VCS-Branch: eapi-8
Date: Thu, 13 May 2021 11:42:25 +0000 (UTC)
Precedence: bulk
List-Post: <mailto:gentoo-commits@lists.gentoo.org>
List-Help: <mailto:gentoo-commits+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-commits+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-commits+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-commits.gentoo.org>
X-BeenThere: gentoo-commits@lists.gentoo.org
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-Archives-Salt: 4adefd58-4a96-47e0-af47-01871e1fd519
X-Archives-Hash: ae7fb8a606c423aba38c1f1cabfa2efa
Message-ID: <20210513114225.3WdtsUJe0BkarNBpUawZA7R3FAgUpOZjEq2-Gd0HoOY@z>
commit: 039b450670c4b7c632c0c4b52f3f4dfed3e7ae15
Author: James Le Cuirot <chewi <AT> gentoo <DOT> org>
AuthorDate: Sun Jul 28 21:37:35 2019 +0000
Commit: Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Sun Jun 28 09:01:27 2020 +0000
URL: https://gitweb.gentoo.org/proj/pms.git/commit/?id=039b4506
Correct the definition of ESYSROOT as EPREFIX isn't always applicable
It was originally envisaged (but not stated in PMS) that SYSROOT would
only ever need to equal / or ROOT as a distinct SYSROOT would have no
benefit. A check was added to Portage to ensure this held. Myself, the
ChromiumOS team, and others have since been caught out by this check
when trying to bootstrap brand new systems from scratch. You cannot
bootstrap with no headers at all! The check will therefore be adjusted
to merely ensure that SYSROOT is / when ROOT is /.
There were differing assumptions about how prefixes applied to the
above. EPREFIX is traditionally something the user sets so some
thought that it would be applied to SYSROOT, regardless of the
latter's value. In order to honor the rule about there being no
distinct SYSROOT, this would mean that if SYSROOT is / then EPREFIX
would have to match BROOT. Despite that limitation, ESYSROOT was
written into PMS with a fixed value of ${SYSROOT}${EPREFIX}. Being
somewhat unfamiliar with prefix at the time, I didn't realise that
this view didn't align with what I'd had in mind and it was only when
I came to need a distinct SYSROOT that I realised there was a problem.
crossdev toolchains are installed to ${EPREFIX}/usr/${CHOST} but have
no further prefix appended and packages subsequently installed with
cross-emerge are placed in this location by setting ROOT. Bug #642604
recently revealed that the build system's prefix was being erroneously
duplicated on the end but I have now fixed this.
What if we want to bootstrap a brand new prefixed system using the
crossdev system as SYSROOT? This is the distinct SYSROOT case. The
problem is that there is no distinct variable for SYSROOT's prefix
and, as already stated, ESYSROOT is always ${SYSROOT}${EPREFIX}. We
therefore cannot do it! If the crossdev prefix is blank then ROOT's
must be blank too.
I also never intended to have the aforementioned limitation where
EPREFIX must match BROOT when SYSROOT is /. These are both entirely
artificial restrictions.
So how should it work instead? We originally intended for SYSROOT to
equal either / or ROOT so I imagined the prefix would automatically be
adjusted to match the prefix applicable at the matching location,
namely BROOT or EPREFIX. This is obviously more flexible than forcing
it to match EPREFIX.
What about the distinct SYSROOT case? With no distinct variable, we
have no way to explicitly set a prefix but this is likely only needed
when bootstrapping against crossdev systems, which are unprefixed by
nature. We therefore simply assume that the prefix is blank in this
case.
What about the cross-prefix case? Here, SYSROOT matches both / and
ROOT so which prefix do we choose? The bootstrap-prefix.sh script sets
flags to build against the target prefix so EPREFIX is used in this
case. This happens to fit the current definition of ESYSROOT anyway.
Legitimate concerns have been raised about building for a system with
a different prefix to the one you're building against. The only
binaries that leak from SYSROOT to ROOT are static libraries. Headers
from SYSROOT will obviously also influence how ROOT's binaries are
built. It is entirely possible that SYSROOT's prefix may leak through
a header but grepping /usr/include on my own main system reveals only
a few paths from a small handful of packages. pkg-config files
invariably include paths but these are almost always used at build
time, not runtime. A differing prefix would likely only occur in cases
involving core packages like the libc and kernel headers anyway. Also
consider that we have never prevented this from happening in the
past. It has always been possible to do "EPREFIX=/foo emerge bar" from
some system with a different prefix or no prefix at all. All we're
doing here is including the prefix (if any) in the ESYSROOT variable.
Should this warrant a new EAPI? I don't think so. All existing usage
of ESYSROOT that I have seen still fits with this new definition and
most of that usage has come from me. We're not even changing what the
variable is used for, just loosening the constraints around what it
can be set to.
If you have doubts about whether this makes sense or actually works in
practise, I have experimented with a prefixed system using all the
different combinations I could think of, including cross-compiling,
and it all worked as expected. Keep in mind that ESYSROOT is not magic
and currently isn't used very much. As such, neither the toolchain nor
pkg-config will use this sysroot if you don't explicitly tell them
to. For the former, I find CC="${CHOST}-gcc --sysroot=${ESYSROOT}"
works well. For the latter, crossdev installs a cross-pkg-config
wrapper but it is completely lacking prefix support at the moment. I
have fixes waiting on this change.
Signed-off-by: James Le Cuirot <chewi <AT> gentoo.org>
[Replaced "/" by "empty", reworded table cell in ebuild-env-vars.tex]
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>
dependencies.tex | 16 +++++++++++++++-
ebuild-env-vars.tex | 6 +++---
2 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/dependencies.tex b/dependencies.tex
index 3305b86..a641014 100644
--- a/dependencies.tex
+++ b/dependencies.tex
@@ -36,13 +36,27 @@
\midrule
Binary compatible with & \t{CBUILD} & \t{CHOST} & \t{CHOST} \\
Base unprefixed path & \t{/} & \t{\$\{SYSROOT\}} & \t{\$\{ROOT\}} \\
- Relevant offset-prefix & \t{\$\{BROOT\}} & \t{\$\{EPREFIX\}} & \t{\$\{EPREFIX\}} \\
+ Relevant offset-prefix & \t{\$\{BROOT\}} & See table~\ref{tab:depend-prefix}
+ & \t{\$\{EPREFIX\}} \\
Path combined with prefix & \t{\$\{BROOT\}} & \t{\$\{ESYSROOT\}} & \t{\$\{EROOT\}} \\
PM query command option & \t{-b} & \t{-d} & \t{-r} \\
\bottomrule
\end{tabular}
\end{centertable}
+\begin{centertable}{Prefix values for \t{DEPEND}}
+ \label{tab:depend-prefix}
+ \begin{tabular}{llll}
+ \toprule
+ If \t{SYSROOT} is: & \t{\$\{ROOT\}} & Empty, and \t{ROOT} is non-empty
+ & Other \\
+ \midrule
+ Then offset-prefix is: & \t{\$\{EPREFIX\}} & \t{\$\{BROOT\}} & Empty \\
+ And \t{ESYSROOT} is: & \t{\$\{EROOT\}} & \t{\$\{BROOT\}} & \t{\$\{SYSROOT\}} \\
+ \bottomrule
+ \end{tabular}
+\end{centertable}
+
There are three classes of dependencies supported by ebuilds:
\begin{compactitem}
diff --git a/ebuild-env-vars.tex b/ebuild-env-vars.tex
index ae8bd7d..117a19d 100644
--- a/ebuild-env-vars.tex
+++ b/ebuild-env-vars.tex
@@ -141,9 +141,9 @@ variable.
\t{ESYSROOT} &
Ditto &
No &
- Contains the concatenation of the paths in the \t{SYSROOT} and \t{EPREFIX} variables,
- for convenience. See also the \t{EPREFIX} variable. Only for EAPIs listed
- in table~\ref{tab:offset-env-vars-table} as supporting \t{ESYSROOT}. \\
+ Contains the concatenation of the \t{SYSROOT} path and applicable prefix value, as determined
+ by table~\ref{tab:depend-prefix}. Only for EAPIs listed in table~\ref{tab:offset-env-vars-table}
+ as supporting \t{ESYSROOT}. \\
\t{BROOT} &
Ditto &
No &