[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Göktürk Yüksek]

commit:     727424254a8bff2c1339569b9e0091809e764c6c
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun May  7 18:58:56 2017 +0000
Commit:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 18:58:01 2017 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=72742425

ebuild-writing/.../src_test: Improve the broken test suite explanation

Broken test suite is *not* worse than no tests. It just requires
significantly more effort from the developers analazying the results.

 ebuild-writing/functions/src_test/text.xml | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index f43fae0..f058006 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -113,9 +113,9 @@ setting <c>RESTRICT="test"</c> in the ebuild.
 
 <note>
 If upstream provide a test suite which doesn't work, consider talking
-to them about getting it fixed. A broken test suite is worse than no
-test suite at all, since we are unable to tell whether a test failure
-indicates a genuine fault.
+to them about getting it fixed. A broken test suite requires developers
+to investigate each test failure in order to determine whether it is
+genuine or indicates a broken test.
 </note>
 </body>
 </section>

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Göktürk Yüksek]

commit:     53b55350b50b480aae6f4e2ae9c93b023d8b02c1
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun May  7 18:51:48 2017 +0000
Commit:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 18:57:15 2017 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=53b55350

ebuild-writing/.../src_test: Do not encourage make, use 'emake -j1' instead

Replace the call to 'make' with 'emake -j1'. This was an old, bad
recommendation. Furthermore, remove the whole paragraph suggesting
not to use emake -- nowadays, most of the test suites work fine
in parallel.

 ebuild-writing/functions/src_test/text.xml | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index 70b6212..ca13a07 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -65,8 +65,8 @@ src_test() {
     sed -i -e 's~test49.out~~g' Makefile || die
 
     # Try to run the non-gui tests only
-    make test-nongui \
-            || die "At least one test failed"
+    # pass -j1 if tests do not support being run in parallel
+    emake -j1 test-nongui
 }
 </codesample>
 </body>
@@ -96,12 +96,6 @@ using <c>sed</c> or skipping a particular <c>make</c> target is
 sufficient.
 </p>
 
-<note>
-<c>emake</c> should not be used for <c>src_test</c> <d/> trying to
-parallelise tests unless the <c>Makefile</c> was specifically designed
-for this can cause all sorts of strange problems.
-</note>
-
 <p>
 Try to ensure that tests work properly for your ebuild. A good test
 suite is extremely helpful for arch maintainers.

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Göktürk Yüksek]

commit:     ed826e136273c8d5beeec8f64c5bd87006c8f791
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun May  7 18:53:15 2017 +0000
Commit:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 18:57:34 2017 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=ed826e13

ebuild-writing/.../src_test: Remove use of X11 from reasons to skip tests

If tests require X11, virtualx.eclass can be used to run them on top
of Xvfb. There is no reason to skip those tests altogether.

 ebuild-writing/functions/src_test/text.xml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index ca13a07..3790195 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -82,7 +82,6 @@ portage environment. Reasons for such a failure could include:
 </p>
 
 <ul>
-  <li>Needing to use X.</li>
   <li>
     Needing to work with files which are disallowed by the sandbox.
   </li>

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Göktürk Yüksek]

commit:     eb9ba924567615e02a0e4e9f089ad93326b7a3e8
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun May  7 18:54:04 2017 +0000
Commit:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 18:57:47 2017 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=eb9ba924

ebuild-writing/.../src_test: Do not suggest blanking src_test()

The only reasonable reason to blank src_test() out would be if the check
or test target represented something completely different from tests.
If the tests are broken, the only proper way to disable them is using
RESTRICT=test, as it clearly indicates that they were explicitly
disabled.

 ebuild-writing/functions/src_test/text.xml | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index 3790195..f43fae0 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -107,19 +107,8 @@ suite is extremely helpful for arch maintainers.
 <title>Skipping Tests</title>
 <body>
 <p>
-Sometimes it is necessary to skip tests entirely. This can be done
-using a dummy <c>src_test</c> function:
-</p>
-
-<codesample lang="ebuild">
-src_test() {
-    # Tests don't even remotely work inside portage
-    true
-}
-</codesample>
-
-<p>
-Another option would be to set <c>RESTRICT="test"</c> in the ebuild.
+Sometimes it is necessary to skip tests entirely. This can be done by
+setting <c>RESTRICT="test"</c> in the ebuild.
 </p>
 
 <note>

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Göktürk Yüksek]

commit:     ed7001edd33076fa8517ad62dd1cf3f3f46d21d7
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun May  7 18:51:28 2017 +0000
Commit:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 18:56:22 2017 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=ed7001ed

ebuild-writing/.../src_test: Add missing ||die into the example

 ebuild-writing/functions/src_test/text.xml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index dd02d4b..70b6212 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -59,10 +59,10 @@ src_test() {
 <body>
 <codesample lang="ebuild">
 src_test() {
-    cd "${S}"/src/testdir
+    cd "${S}"/src/testdir || die
 
     # Test 49 won't work inside a portage environment
-    sed -i -e 's~test49.out~~g' Makefile
+    sed -i -e 's~test49.out~~g' Makefile || die
 
     # Try to run the non-gui tests only
     make test-nongui \

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Göktürk Yüksek]

commit:     96ce954fee8e464edadc473a5ba2da8b8352b6cb
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 22 08:51:57 2017 +0000
Commit:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 19:00:01 2017 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=96ce954f

ebuild-writing/.../src_test: Update the default implementation

 ebuild-writing/functions/src_test/text.xml | 20 +++++---------------
 1 file changed, 5 insertions(+), 15 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index f058006..5d1ba3f 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -31,23 +31,13 @@
 <section>
 <title>Default <c>src_test</c></title>
 <body>
-<p> From sys-apps/portage-2.1.2.9 </p>
+<p>The default test phase in EAPI 6 is equivalent to the following:</p>
 <codesample lang="ebuild">
 src_test() {
-	if emake -j1 check -n &amp;> /dev/null; then
-		vecho ">>> Test phase [check]: ${CATEGORY}/${PF}"
-		if ! emake -j1 check; then
-			hasq test $FEATURES &amp;&amp; die "Make check failed. See above for details."
-			hasq test $FEATURES || eerror "Make check failed. See above for details."
-		fi
-	elif emake -j1 test -n &amp;> /dev/null; then
-		vecho ">>> Test phase [test]: ${CATEGORY}/${PF}"
-		if ! emake -j1 test; then
-			hasq test $FEATURES &amp;&amp; die "Make test failed. See above for details."
-			hasq test $FEATURES || eerror "Make test failed. See above for details."
-		fi
-	else
-		vecho ">>> Test phase [none]: ${CATEGORY}/${PF}"
+	if emake check -n &amp;> /dev/null; then
+		emake check
+	elif emake test -n &amp;> /dev/null; then
+		emake test
 	fi
 }
 </codesample>

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Göktürk Yüksek]

commit:     084a5771a359da6a083e8f11169ccfdff6066abd
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 22 09:02:51 2017 +0000
Commit:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 19:00:22 2017 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=084a5771

ebuild-writing/.../src_test: Provide a detailed solution for X11 requirement

 ebuild-writing/functions/src_test/text.xml | 44 ++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index 5d1ba3f..bd8f297 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -63,6 +63,50 @@ src_test() {
 </section>
 
 <section>
+<title>Tests that require X11</title>
+<body>
+<p>
+Some packages include tests (or other build-time applications) that
+attempt to use the user's X11 session and fail being unable to connect
+to it. Those tests need to be fixed to work independently of the X11
+server that might or might not be running when packages are being built.
+</p>
+
+<p>
+If the program in question does not strictly need X11 but merely
+attempts to take opportunity of the <c>DISPLAY</c> variable being set,
+the best solution is to simply unset this variable in the ebuild.
+</p>
+
+<codesample lang="ebuild">
+src_test() {
+	# tests attempt to connect to X11 and fail when it is set
+	# however, they work just fine without X11
+	unset DISPLAY
+
+	default
+}
+</codesample>
+
+<p>
+If the package actually requires a running X11 server to run
+the complete test suite, you can use the <c>virtualx</c> eclass to
+provide an isolated Xvfb environment for the tests to use. It provides
+a virtual X11 display that is not connected to any physical device
+and that programs can use reliably.
+</p>
+
+<codesample lang="ebuild">
+inherit virtualx
+
+src_test() {
+	virtx default
+}
+</codesample>
+</body>
+</section>
+
+<section>
 <title>Common <c>src_test</c> Tasks</title>
 <body>
 <p>

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Göktürk Yüksek]

commit:     1118c727fbb60fb6b26fd1e2845ecca51685a7ee
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 22 09:44:48 2017 +0000
Commit:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 19:00:37 2017 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=1118c727

ebuild-writing/.../src_test: Expand on network & service access problems

 ebuild-writing/functions/src_test/text.xml | 101 +++++++++++++++++++++++++++++
 1 file changed, 101 insertions(+)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index bd8f297..6ce3b79 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -63,6 +63,107 @@ src_test() {
 </section>
 
 <section>
+<title>Tests that require network or service access</title>
+<body>
+<p>
+Sometimes test suites (and other build-time programs) attempt to use
+remote or local network, or production servers running on the host. All
+of these are strictly forbidden. Developers should either fix such tests
+to work in an isolated environment, or disable them completely unless
+explicitly allowed by the user. At the bare minimum, the tests must
+not fail with <c>FEATURES=network-sandbox</c> being enabled.
+</p>
+
+<p>
+Internet access within the build procedure is forbidden for
+the following reasons:
+</p>
+<ul>
+  <li>
+    the build may be running in an environment with no or restricted
+    Internet access, and this must not cause the tests (build) to fail;
+  </li>
+
+  <li>
+    the Internet connection may be unstable (e.g. poor reception)
+    in which case an interrupted connection or packet loss must not
+    cause the tests to fail or hang, and it should not cause unnecessary
+    delays;
+  </li>
+
+  <li>
+    the Internet connection may be running on a limited data plan
+    in which case the additional network use may cause additional
+    charges or other inconveniences to the user;
+  </li>
+
+  <li>
+    the remote network services used by the tests may become unavailable
+    temporarily or permanently, causing unexpected test failures;
+  </li>
+
+  <li>
+    accessing remote sites always poses a privacy issue, and possibly
+    a threat to security (e.g. through inadvertently exposing
+    information about the system).
+  </li>
+</ul>
+
+<p>
+Fixing tests that require Internet access usually requires cooperation
+with upstream, and porting the tests to use test techniques such as
+mocking or using replay data. For this reason, developers report
+the issue upstream and skip tests that require network access.
+It is recommended to explicitly leave a note as to why the tests are
+skipped, so that other developers can re-enable them locally to run
+a more complete test suite.
+</p>
+
+<p>
+Local server access within the build procedure is additionally
+forbidden for the following reasons:
+</p>
+<ul>
+  <li>
+    tests must run reliably independently of whether a particular
+    server is running throughout the build process or not,
+  </li>
+
+  <li>
+    using production services for running tests is extremely
+    <b>dangerous</b> as it may inadvertently expose bugs in those
+    services, causing instability, data loss or even exposing security
+    vulnerabilities.
+  </li>
+</ul>
+
+<p>
+Fixing tests that require access to local services is usually done
+via starting additional isolated instances of those services during
+the test phase. Those services must either be running on a UNIX
+socket or on the loopback interface, to reliably prevent remote access.
+</p>
+
+<p>
+For all networked services exposed during the test phase (either by
+the ebuild or the tests themselves), UNIX sockets are strongly preferred
+over IP sockets as they provide better means for unique naming
+and access control mechanisms. IP sockets can be subject to port
+collisions with other local services and they can be accessed by local
+system users who may exploit a vulnerability through the tests.
+</p>
+
+<p>
+Additional protection against those issues is provided through
+<c>FEATURES=network-sandbox</c>. However, this is only an optional
+Portage feature relying on specific Linux kernel namespace mechanisms
+and developers should not rely on it being enabled.
+</p>
+
+</body>
+</section>
+
+<section>
 <title>Tests that require X11</title>
 <body>
 <p>

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Göktürk Yüksek]

commit:     7fda20c5d037d292ea19d2ed7f032a81486c3bd2
Author:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
AuthorDate: Thu Aug 24 19:06:26 2017 +0000
Commit:     Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 19:06:26 2017 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=7fda20c5

ebuild-writing/functions/src_test: s/provide/provides/, s/which/that/

 ebuild-writing/functions/src_test/text.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index 6ce3b79..56dd113 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -247,7 +247,7 @@ setting <c>RESTRICT="test"</c> in the ebuild.
 </p>
 
 <note>
-If upstream provide a test suite which doesn't work, consider talking
+If upstream provides a test suite that doesn't work, consider talking
 to them about getting it fixed. A broken test suite requires developers
 to investigate each test failure in order to determine whether it is
 genuine or indicates a broken test.

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Ulrich Müller]

commit:     b2ad7639d04e296312820025c9b8f77d22272028
Author:     Florian Schmaus <flo <AT> geekplace <DOT> eu>
AuthorDate: Sun Aug  2 17:17:15 2020 +0000
Commit:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Wed Aug 19 17:29:42 2020 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=b2ad7639

ebuild-writing/functions/src_test: re-organize, merge sections

Merge "Common src_test Tasks" and "Skipping Tests" sections into a new
section right after "Sample src_test".

Also forward the reader to the related paragraph on test-only
dependencies in this new section. Before that, the src_test
documentation page was silent about that, rather important,
topic. This also includes the RESTRICT="!test? ( test )" idiom.

Signed-off-by: Florian Schmaus <flo <AT> geekplace.eu>
Closes: https://bugs.gentoo.org/735570
[Whitespace changes]
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>

 ebuild-writing/functions/src_test/text.xml | 108 ++++++++++++++++-------------
 1 file changed, 60 insertions(+), 48 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index c278aa0..fbd0751 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -62,6 +62,66 @@ src_test() {
 </body>
 </section>
 
+<section>
+<title>Supporting test suites in packages</title>
+<body>
+
+<p>
+If the packaged software is equipped with a test suite, it is sensible
+to make the package aware of it. This allows the package manager to
+run the software's unit tests. Packages with a test suite must
+announce the "test" USE flag.
+</p>
+
+<codesample lang="ebuild">
+IUSE="test"
+</codesample>
+
+<p>
+The next important aspect to consider are test-only dependencies, i.e.
+dependencies that are only required to run the test suite. Test-only
+dependencies should be specified in a DEPEND behind a USE flag.
+Please refer to the section on
+<uri link="::general-concepts/dependencies/#Test Dependencies"/>
+for more information.
+</p>
+
+<p>
+Often the default <c>src_test</c> is fine. Sometimes it is necessary
+to remove certain tests from the list if they cannot be used with a
+portage environment. Reasons for such a failure could include:
+</p>
+
+<ul>
+  <li>
+    Needing to work with files which are disallowed by the sandbox.
+  </li>
+  <li>Requiring user input (<c>src_test</c> must not be interactive).</li>
+  <li>Requiring root privileges.</li>
+</ul>
+
+<p>
+Usually, removing the relevant test from the <c>Makefile</c>
+using <c>sed</c> or skipping a particular <c>make</c> target is
+sufficient.
+</p>
+
+<p>
+Try to ensure that tests work properly for your ebuild. A good test
+suite is extremely helpful for arch maintainers.
+Sometimes it is necessary to skip tests entirely. This can be done by
+setting <c>RESTRICT="test"</c> in the ebuild.
+</p>
+
+<note>
+If upstream provides a test suite that doesn't work, consider talking
+to them about getting it fixed. A broken test suite requires developers
+to investigate each test failure in order to determine whether it is
+genuine or indicates a broken test.
+</note>
+</body>
+</section>
+
 <section>
 <title>Tests that require network or service access</title>
 <body>
@@ -207,53 +267,5 @@ src_test() {
 </body>
 </section>
 
-<section>
-<title>Common <c>src_test</c> Tasks</title>
-<body>
-<p>
-Often the default <c>src_test</c> is fine. Sometimes it is necessary
-to remove certain tests from the list if they cannot be used with a
-portage environment. Reasons for such a failure could include:
-</p>
-
-<ul>
-  <li>
-    Needing to work with files which are disallowed by the sandbox.
-  </li>
-  <li>Requiring user input (src_test must not be interactive).</li>
-  <li>Requiring root privileges.</li>
-</ul>
-
-<p>
-Usually, removing the relevant test from the <c>Makefile</c>
-using <c>sed</c> or skipping a particular <c>make</c> target is
-sufficient.
-</p>
-
-<p>
-Try to ensure that tests work properly for your ebuild. A good test
-suite is extremely helpful for arch maintainers.
-</p>
-
-</body>
-</section>
-
-<section>
-<title>Skipping Tests</title>
-<body>
-<p>
-Sometimes it is necessary to skip tests entirely. This can be done by
-setting <c>RESTRICT="test"</c> in the ebuild.
-</p>
-
-<note>
-If upstream provides a test suite that doesn't work, consider talking
-to them about getting it fixed. A broken test suite requires developers
-to investigate each test failure in order to determine whether it is
-genuine or indicates a broken test.
-</note>
-</body>
-</section>
-
 </chapter>
 </guide>

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Ulrich Müller]

commit:     929b5a3a85c3cf34cf967db579a45d8f3ee88564
Author:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
AuthorDate: Fri Aug 14 12:15:27 2020 +0000
Commit:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Fri Aug 28 18:59:51 2020 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=929b5a3a

ebuild-writing/functions/src_test: Add nonfatal to if conditions.

Closes: https://bugs.gentoo.org/722418
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>

 ebuild-writing/functions/src_test/text.xml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index fbd0751..d214b15 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -34,9 +34,9 @@
 <p>The default test phase in EAPI 6 is equivalent to the following:</p>
 <codesample lang="ebuild">
 src_test() {
-	if emake check -n &amp;&gt; /dev/null; then
+	if nonfatal emake check -n &amp;&gt; /dev/null; then
 		emake check
-	elif emake test -n &amp;&gt; /dev/null; then
+	elif nonfatal emake test -n &amp;&gt; /dev/null; then
 		emake test
 	fi
 }

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Ulrich Müller]

commit:     7d12a219103e5951c0c5c25cf12abdc306d29868
Author:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
AuthorDate: Tue Dec 15 10:24:48 2020 +0000
Commit:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Sun Jan  3 13:57:24 2021 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=7d12a219

ebuild-writing/functions/src_test: IUSE=test is not required.

Closes: https://bugs.gentoo.org/760051
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>

 ebuild-writing/functions/src_test/text.xml | 20 +++++---------------
 1 file changed, 5 insertions(+), 15 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index d214b15..baf0ad6 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -68,21 +68,11 @@ src_test() {
 
 <p>
 If the packaged software is equipped with a test suite, it is sensible
-to make the package aware of it. This allows the package manager to
-run the software's unit tests. Packages with a test suite must
-announce the "test" USE flag.
-</p>
-
-<codesample lang="ebuild">
-IUSE="test"
-</codesample>
-
-<p>
-The next important aspect to consider are test-only dependencies, i.e.
-dependencies that are only required to run the test suite. Test-only
-dependencies should be specified in a DEPEND behind a USE flag.
-Please refer to the section on
-<uri link="::general-concepts/dependencies/#Test Dependencies"/>
+to run it. Sometimes the package will need additional dependencies for this,
+i.e., dependencies that are only required to run the test suite. Such test-only
+dependencies should be specified in DEPEND or BDEPEND behind a USE flag;
+often, the <c>test</c> USE flag will be used for this. Please refer to the
+section on <uri link="::general-concepts/dependencies/#Test Dependencies"/>
 for more information.
 </p>
 

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Ulrich Müller]

commit:     6bf4fb565b28b8041951f3a85d8a75e19ab2042a
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Wed Jun 23 07:18:16 2021 +0000
Commit:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Tue Jul  6 08:02:49 2021 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=6bf4fb56

ebuild-writing/functions/src_test: explicitly say USE=test is for only some cases

It's only needed where we have test dependencies, we want to conditionally
build tests, conditionally fetch files, or otherwise alter behaviour based
on the fact we're running tests (e.g. usage in RESTRICT).

Closes: https://github.com/gentoo/devmanual/pull/231
Suggested-by: Florian Schmaus <flow <AT> gentoo.org>
Signed-off-by: Sam James <sam <AT> gentoo.org>
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>

 ebuild-writing/functions/src_test/text.xml | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index 0dae1f4..1822c33 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -76,6 +76,14 @@ section on <uri link="::general-concepts/dependencies/#Test Dependencies"/>
 for more information.
 </p>
 
+<p>
+Note that the <c>test</c> USE flag is only necessary if there are
+test dependencies, tests are being built conditionally, or it is desirable
+to vary behaviour based on whether tests will be run (e.g. downloading large
+files in SRC_URI). It is not appropriate to use the flag to simply indicate that
+tests exist and are expected to pass.
+</p>
+
 <p>
 Often the default <c>src_test</c> is fine. Sometimes it is necessary
 to remove certain tests from the list if they cannot be used with a

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Ulrich Müller]

commit:     23a5480394d2202081f76a1bf78b3785e7f3a4c6
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Wed May  5 07:48:35 2021 +0000
Commit:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Sun Aug  1 09:50:55 2021 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=23a54803

ebuild-writing/functions/src_test: Clarify how localhost can be used

Closes: https://github.com/gentoo/devmanual/pull/227
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
[Reword as suggested by Sam James <sam <AT> gentoo.org>]
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>

 ebuild-writing/functions/src_test/text.xml | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index 1822c33..3b87060 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -178,9 +178,17 @@ a more complete test suite.
 </p>
 
 <p>
-Local server access within the build procedure is additionally
-forbidden for the following reasons:
+It is generally considered acceptable to rely on IPv4 <c>localhost</c> being
+resolvable and available for binding to and connecting to services started
+within the phase. However, it is not acceptable to connect to the daemons
+run outside the test environment.
 </p>
+
+<p>
+Local server access within the build procedure is forbidden for the following
+reasons:
+</p>
+
 <ul>
   <li>
     tests must run reliably independently of whether a particular

[gentoo-commits] proj/devmanual:master commit in: ebuild-writing/functions/src_test/

[Ulrich Müller]

commit:     1779bc63ec5bce6e0c117b1fe90065e9d1622df0
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Wed May  5 07:48:35 2021 +0000
Commit:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Sun Aug  1 09:54:50 2021 +0000
URL:        https://gitweb.gentoo.org/proj/devmanual.git/commit/?id=1779bc63

ebuild-writing/functions/src_test: Clarify how localhost can be used

Closes: https://github.com/gentoo/devmanual/pull/227
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
[Reword as suggested by Sam James and Florian Schmaus]
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>

 ebuild-writing/functions/src_test/text.xml | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/ebuild-writing/functions/src_test/text.xml b/ebuild-writing/functions/src_test/text.xml
index 1822c33..3fa05d5 100644
--- a/ebuild-writing/functions/src_test/text.xml
+++ b/ebuild-writing/functions/src_test/text.xml
@@ -178,9 +178,17 @@ a more complete test suite.
 </p>
 
 <p>
-Local server access within the build procedure is additionally
-forbidden for the following reasons:
+It is generally considered acceptable to rely on IPv4 <c>localhost</c> being
+resolvable and available for binding. Tests should only connect to services
+that are started as part of the testsuite. It is not acceptable to connect
+to daemons run outside the test environment.
 </p>
+
+<p>
+Local server access within the build procedure is forbidden for the following
+reasons:
+</p>
+
 <ul>
   <li>
     tests must run reliably independently of whether a particular