From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1O23SC-0003hV-BF for garchives@archives.gentoo.org; Wed, 14 Apr 2010 14:20:32 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 096C5E0923; Wed, 14 Apr 2010 14:20:27 +0000 (UTC) Received: from mail-bw0-f217.google.com (mail-bw0-f217.google.com [209.85.218.217]) by pigeon.gentoo.org (Postfix) with ESMTP id 651FEE091B for ; Wed, 14 Apr 2010 14:19:56 +0000 (UTC) Received: by bwz9 with SMTP id 9so190933bwz.29 for ; Wed, 14 Apr 2010 07:19:55 -0700 (PDT) Received: by 10.204.136.15 with SMTP id p15mr8443544bkt.172.1271254795684; Wed, 14 Apr 2010 07:19:55 -0700 (PDT) Received: from pomiot.lan (213-238-106-44.adsl.inetia.pl [213.238.106.44]) by mx.google.com with ESMTPS id 13sm498860bwz.7.2010.04.14.07.19.53 (version=SSLv3 cipher=RC4-MD5); Wed, 14 Apr 2010 07:19:54 -0700 (PDT) Sender: Spam Box Date: Wed, 14 Apr 2010 16:20:18 +0200 From: =?UTF-8?B?TWljaGHFgiBHw7Nybnk=?= To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] Multiple emerges in parallel (was: [RFC] RESTRICT=parallel for builds that can't be executed in parallel) Message-ID: <20100414162018.26e0524c@pomiot.lan> In-Reply-To: <20100414061016.GA30025@hrair> References: <4BC52478.3020303@gentoo.org> <20100414074520.339dd0ed@pomiot.lan> <20100414061016.GA30025@hrair> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.9; x86_64-pc-linux-gnu) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/PilI3P=Zys8XhB7Ghg2zJS8"; protocol="application/pgp-signature" X-Archives-Salt: 5756d161-a514-4897-919c-f1055f860427 X-Archives-Hash: dfb02c9620ad7d4fff2d816d5153026e --Sig_/PilI3P=Zys8XhB7Ghg2zJS8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue, 13 Apr 2010 23:10:16 -0700 Brian Harring wrote: > Running multiple emerges in parallel is already a bad idea. The=20 > solution for that case is for the new/second emerge to feed the=20 > request into the original emerge (or a daemon). Although such solution will be useful in many cases indeed, there are still many advantages of having few separate emerge calls running in parallel. First of all, the 'emerge daemon' would have to be able to transparently inject the additional packages onto the dependency tree. And just appending missing dependencies and the real package at the end doesn't seem sufficient. For example, what if such operation would result in a conflict? If that's only that simple as 'A and B can not be installed at the same time', the second emerge call could reject appending the packages, explaining the reason for that. But what if the conflict could be resolved through pulling in some other version of one of the packages? We could try to replace the package in the dependency tree silently if it didn't yet start being merged; but what if it did? Should we reject adding the packages, encouraging user to 'try again later'? Or maybe abort the running builds to merge another version of the depend? Same goes for virtual dependencies which can be fulfilled by many different packages. Of course, that's just implementation-wide issue and running separate emerge instances could result in even worse results of this problem. But apart from that, there are side uses of multiple emerges which would be hard to implement with your idea. For example, I often use different PORTAGE_TMPDIR to build large packages like xulrunner. Although it doesn't seem that hard implementing the support for running merges in different temporary directories at the same time, should we really waste time implementing that? What about other possible configuration changes? What about 'USE' overrides within the environment? These ones should certainly be respected. So, now our daemon not only has to be able to efficiently mangle dependency trees at runtime but also to support partial runtime configuration changes. The next thing is aborting merges. When running multiple emerges, aborting one of them is as simple as pressing ^c. With daemon, we would have to implement an ability of aborting/removing packages in runtime -- and that would be another example of dependency tree mangling. But in fact, implementing (at least limited) the support of such mangling would be useful in some cases. With '--keep-going', it would allow to drop the packages and continue merging the remaining ones without waiting for current builds to complete. And looking from the other side, the support for removing single packages from the running emerge dependency tree would be useful too with current emerge concept. Even if it just worked like '--keep-going', it would be still better than the current workaround -- waiting for particular emerge to start and removing its ${WORKDIR} to force failure. --=20 Best regards, Micha=C5=82 G=C3=B3rny --Sig_/PilI3P=Zys8XhB7Ghg2zJS8 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (GNU/Linux) iEYEARECAAYFAkvFzysACgkQnGSe5QXeB7tgZwCdGVssaVsh9JES6k9faedON/Fz PSUAn3fVdOm+LZEEGIktusy4thVH80/B =5HNH -----END PGP SIGNATURE----- --Sig_/PilI3P=Zys8XhB7Ghg2zJS8--