[gentoo-commits] proj/portage:master commit in: pym/portage/tests/util/futures/asyncio/

[gentoo-commits] proj/portage:master commit in: pym/portage/tests/util/futures/asyncio/

[Zac Medico]

commit:     c23d093d0330a9318534a1c521fb00d6360fabc0
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Apr 23 18:20:25 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Apr 23 18:24:11 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=c23d093d

tests: prove run_until_complete executes done callbacks

Prove that done callbacks are executed before run_until_complete
returns, which is how asyncio's default event loop behaves. This
behavior was implemented in portage's internal event loop in commit
25245d7eb86ed197b7d7cfead0dbe4ce8ad4bc5b.

Bug: https://bugs.gentoo.org/591760

 .../futures/asyncio/test_run_until_complete.py     | 29 ++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/pym/portage/tests/util/futures/asyncio/test_run_until_complete.py b/pym/portage/tests/util/futures/asyncio/test_run_until_complete.py
new file mode 100644
index 000000000..fc8f198ca
--- /dev/null
+++ b/pym/portage/tests/util/futures/asyncio/test_run_until_complete.py
@@ -0,0 +1,29 @@
+# Copyright 2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+from portage.tests import TestCase
+from portage.util.futures import asyncio
+from portage.util.futures.unix_events import DefaultEventLoopPolicy
+
+
+class RunUntilCompleteTestCase(TestCase):
+	def test_add_done_callback(self):
+		initial_policy = asyncio.get_event_loop_policy()
+		if not isinstance(initial_policy, DefaultEventLoopPolicy):
+			asyncio.set_event_loop_policy(DefaultEventLoopPolicy())
+
+		try:
+			loop = asyncio.get_event_loop()
+			f1 = loop.create_future()
+			f2 = loop.create_future()
+			f1.add_done_callback(f2.set_result)
+			loop.call_soon(lambda: f1.set_result(None))
+			loop.run_until_complete(f1)
+			self.assertEqual(f1.done(), True)
+
+			# This proves that done callbacks of f1 are executed before
+			# loop.run_until_complete(f1) returns, which is how asyncio's
+			# default event loop behaves.
+			self.assertEqual(f2.done(), True)
+		finally:
+			asyncio.set_event_loop_policy(initial_policy)

[gentoo-commits] proj/portage:master commit in: pym/portage/tests/util/futures/asyncio/

[Zac Medico]

commit:     10253819aae4cefee80f377e167ad521516d66a2
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Thu May  3 01:18:39 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Thu May  3 01:18:39 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=10253819

WriterPipeClosedTestCase: add_writer after pipe is filled

Hopefully this supresses a spurious writer callback observed
once for pypy in travis.

See: https://travis-ci.org/gentoo/portage/jobs/373734825

 pym/portage/tests/util/futures/asyncio/test_pipe_closed.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/pym/portage/tests/util/futures/asyncio/test_pipe_closed.py b/pym/portage/tests/util/futures/asyncio/test_pipe_closed.py
index 1ecddab78..5398ca35c 100644
--- a/pym/portage/tests/util/futures/asyncio/test_pipe_closed.py
+++ b/pym/portage/tests/util/futures/asyncio/test_pipe_closed.py
@@ -105,7 +105,6 @@ class WriterPipeClosedTestCase(_PipeClosedTestCase, TestCase):
 
 			writer_callback.called = loop.create_future()
 			_set_nonblocking(write_end.fileno())
-			loop.add_writer(write_end.fileno(), writer_callback)
 
 			# Fill up the pipe, so that no writer callbacks should be
 			# received until the state has changed.
@@ -117,6 +116,11 @@ class WriterPipeClosedTestCase(_PipeClosedTestCase, TestCase):
 						raise
 					break
 
+			# We've seen at least one spurious writer callback when
+			# this was registered before the pipe was filled, so
+			# register it afterwards.
+			loop.add_writer(write_end.fileno(), writer_callback)
+
 			# Allow the loop to check for IO events, and assert
 			# that our future is still not done.
 			loop.run_until_complete(asyncio.sleep(0, loop=loop))

[gentoo-commits] proj/portage:master commit in: pym/portage/tests/util/futures/asyncio/

[Zac Medico]

commit:     87aeab1a62cc6fa1d48354a42ec4fa787dbe9603
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun May  6 01:19:08 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun May  6 01:26:50 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=87aeab1a

WriterPipeClosedTestCase: retry filling pipe

This should suppress spurious writer callback observed
twice for pypy in travis.

See: https://travis-ci.org/gentoo/portage/jobs/375411936
See: https://travis-ci.org/gentoo/portage/jobs/373734825

 .../tests/util/futures/asyncio/test_pipe_closed.py | 39 +++++++++++++---------
 1 file changed, 23 insertions(+), 16 deletions(-)

diff --git a/pym/portage/tests/util/futures/asyncio/test_pipe_closed.py b/pym/portage/tests/util/futures/asyncio/test_pipe_closed.py
index 5398ca35c..e63829888 100644
--- a/pym/portage/tests/util/futures/asyncio/test_pipe_closed.py
+++ b/pym/portage/tests/util/futures/asyncio/test_pipe_closed.py
@@ -105,25 +105,32 @@ class WriterPipeClosedTestCase(_PipeClosedTestCase, TestCase):
 
 			writer_callback.called = loop.create_future()
 			_set_nonblocking(write_end.fileno())
+			loop.add_writer(write_end.fileno(), writer_callback)
 
-			# Fill up the pipe, so that no writer callbacks should be
-			# received until the state has changed.
-			while True:
-				try:
-					os.write(write_end.fileno(), 512 * b'0')
-				except EnvironmentError as e:
-					if e.errno != errno.EAGAIN:
-						raise
+			# With pypy we've seen intermittent spurious writer callbacks
+			# here, so retry until the correct state is achieved.
+			tries = 10
+			while tries:
+				tries -= 1
+
+				# Fill up the pipe, so that no writer callbacks should be
+				# received until the state has changed.
+				while True:
+					try:
+						os.write(write_end.fileno(), 512 * b'0')
+					except EnvironmentError as e:
+						if e.errno != errno.EAGAIN:
+							raise
+						break
+
+				# Allow the loop to check for IO events, and assert
+				# that our future is still not done.
+				loop.run_until_complete(asyncio.sleep(0, loop=loop))
+				if writer_callback.called.done():
+					writer_callback.called = loop.create_future()
+				else:
 					break
 
-			# We've seen at least one spurious writer callback when
-			# this was registered before the pipe was filled, so
-			# register it afterwards.
-			loop.add_writer(write_end.fileno(), writer_callback)
-
-			# Allow the loop to check for IO events, and assert
-			# that our future is still not done.
-			loop.run_until_complete(asyncio.sleep(0, loop=loop))
 			self.assertFalse(writer_callback.called.done())
 
 			# Demonstrate that the callback is called afer the

[gentoo-commits] proj/portage:master commit in: pym/portage/tests/util/futures/asyncio/

[Zac Medico]

commit:     0c96d2d0b18036cb94d68c42783441d32af2d9d3
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Wed May  9 07:54:26 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed May  9 07:59:56 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=0c96d2d0

SubprocessExecTestCase: fix unintended skipTest

The internal asyncio shim does not provide a
create_subprocess_exec attribute, so access it
directly from the real asyncio module.

Fixes 82a3cda6f1ff ("Minimize _asyncio_wrapper usage (bug 654390)")

 .../util/futures/asyncio/test_subprocess_exec.py    | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/pym/portage/tests/util/futures/asyncio/test_subprocess_exec.py b/pym/portage/tests/util/futures/asyncio/test_subprocess_exec.py
index 534d79c53..5a812ba6a 100644
--- a/pym/portage/tests/util/futures/asyncio/test_subprocess_exec.py
+++ b/pym/portage/tests/util/futures/asyncio/test_subprocess_exec.py
@@ -4,6 +4,11 @@
 import os
 import subprocess
 
+try:
+	from asyncio import create_subprocess_exec
+except ImportError:
+	create_subprocess_exec = None
+
 from portage.process import find_binary
 from portage.tests import TestCase
 from portage.util._eventloop.global_event_loop import global_event_loop
@@ -71,7 +76,7 @@ class SubprocessExecTestCase(TestCase):
 				self.assertFalse(global_event_loop().is_closed())
 
 	def testEcho(self):
-		if not hasattr(asyncio, 'create_subprocess_exec'):
+		if create_subprocess_exec is None:
 			self.skipTest('create_subprocess_exec not implemented for python2')
 
 		args_tuple = (b'hello', b'world')
@@ -91,7 +96,7 @@ class SubprocessExecTestCase(TestCase):
 			try:
 				with open(os.devnull, 'rb', 0) as devnull:
 					proc = loop.run_until_complete(
-						asyncio.create_subprocess_exec(
+						create_subprocess_exec(
 						echo_binary, *args_tuple,
 						stdin=devnull, stdout=stdout_pw, stderr=stdout_pw))
 
@@ -114,7 +119,7 @@ class SubprocessExecTestCase(TestCase):
 		self._run_test(test)
 
 	def testCat(self):
-		if not hasattr(asyncio, 'create_subprocess_exec'):
+		if create_subprocess_exec is None:
 			self.skipTest('create_subprocess_exec not implemented for python2')
 
 		stdin_data = b'hello world'
@@ -138,7 +143,7 @@ class SubprocessExecTestCase(TestCase):
 			output = None
 			try:
 				proc = loop.run_until_complete(
-					asyncio.create_subprocess_exec(
+					create_subprocess_exec(
 					cat_binary,
 					stdin=stdin_pr, stdout=stdout_pw, stderr=stdout_pw))
 
@@ -173,7 +178,7 @@ class SubprocessExecTestCase(TestCase):
 		requires an AbstractEventLoop.connect_read_pipe implementation
 		(and a ReadTransport implementation for it to return).
 		"""
-		if not hasattr(asyncio, 'create_subprocess_exec'):
+		if create_subprocess_exec is None:
 			self.skipTest('create_subprocess_exec not implemented for python2')
 
 		args_tuple = (b'hello', b'world')
@@ -184,7 +189,7 @@ class SubprocessExecTestCase(TestCase):
 		def test(loop):
 			with open(os.devnull, 'rb', 0) as devnull:
 				proc = loop.run_until_complete(
-					asyncio.create_subprocess_exec(
+					create_subprocess_exec(
 					echo_binary, *args_tuple,
 					stdin=devnull,
 					stdout=subprocess.PIPE, stderr=subprocess.STDOUT))
@@ -202,7 +207,7 @@ class SubprocessExecTestCase(TestCase):
 		requires an AbstractEventLoop.connect_write_pipe implementation
 		(and a WriteTransport implementation for it to return).
 		"""
-		if not hasattr(asyncio, 'create_subprocess_exec'):
+		if create_subprocess_exec is None:
 			self.skipTest('create_subprocess_exec not implemented for python2')
 
 		stdin_data = b'hello world'
@@ -212,7 +217,7 @@ class SubprocessExecTestCase(TestCase):
 
 		def test(loop):
 			proc = loop.run_until_complete(
-				asyncio.create_subprocess_exec(
+				create_subprocess_exec(
 				cat_binary,
 				stdin=subprocess.PIPE,
 				stdout=subprocess.PIPE, stderr=subprocess.STDOUT))

[gentoo-commits] proj/portage:master commit in: pym/portage/tests/util/futures/asyncio/

[Zac Medico]

commit:     3a55ecd1f79c31f477d7bdd0b9f0e97d8a15eb9e
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Wed May  9 14:19:11 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed May  9 14:19:11 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=3a55ecd1

DefaultEventLoopPolicy: test NotImplementedError due to recursion

 .../asyncio/test_policy_wrapper_recursion.py       | 29 ++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/pym/portage/tests/util/futures/asyncio/test_policy_wrapper_recursion.py b/pym/portage/tests/util/futures/asyncio/test_policy_wrapper_recursion.py
new file mode 100644
index 000000000..d3cd94b35
--- /dev/null
+++ b/pym/portage/tests/util/futures/asyncio/test_policy_wrapper_recursion.py
@@ -0,0 +1,29 @@
+# Copyright 2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+try:
+	import asyncio
+except ImportError:
+	asyncio = None
+
+from portage.tests import TestCase
+from portage.util.futures.unix_events import DefaultEventLoopPolicy
+
+
+class PolicyWrapperRecursionTestCase(TestCase):
+	def testPolicyWrapperRecursion(self):
+		if asyncio is None:
+			self.skipTest('asyncio is not available')
+
+		initial_policy = asyncio.get_event_loop_policy()
+		if not isinstance(initial_policy, DefaultEventLoopPolicy):
+			asyncio.set_event_loop_policy(DefaultEventLoopPolicy())
+
+		try:
+			with self.assertRaises(NotImplementedError):
+				asyncio.get_event_loop()
+
+			with self.assertRaises(NotImplementedError):
+				asyncio.get_child_watcher()
+		finally:
+			asyncio.set_event_loop_policy(initial_policy)