From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id B32B71382C5 for ; Tue, 19 May 2020 22:55:41 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id CB837E0878; Tue, 19 May 2020 22:55:38 +0000 (UTC) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id A75E3E0878 for ; Tue, 19 May 2020 22:55:38 +0000 (UTC) Received: from oystercatcher.gentoo.org (unknown [IPv6:2a01:4f8:202:4333:225:90ff:fed9:fc84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 540B634F1E7 for ; Tue, 19 May 2020 22:55:37 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id 60E9D260 for ; Tue, 19 May 2020 22:55:34 +0000 (UTC) From: "Matt Turner" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Matt Turner" Message-ID: <1589928861.709612b50e6c25cbfa0275cdaffa8ec80bd2f146.mattst88@gentoo> Subject: [gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/, catalyst/base/, etc/, doc/ X-VCS-Repository: proj/catalyst X-VCS-Files: catalyst/base/stagebase.py catalyst/defaults.py doc/catalyst-config.5.txt etc/catalystrc X-VCS-Directories: etc/ catalyst/base/ catalyst/ doc/ X-VCS-Committer: mattst88 X-VCS-Committer-Name: Matt Turner X-VCS-Revision: 709612b50e6c25cbfa0275cdaffa8ec80bd2f146 X-VCS-Branch: wip/mattst88 Date: Tue, 19 May 2020 22:55:34 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-Archives-Salt: 03333c39-a4d6-43cd-9db4-d97eb55668d3 X-Archives-Hash: dc8bbaac674703ebbc95954cf51c323d commit: 709612b50e6c25cbfa0275cdaffa8ec80bd2f146 Author: Matt Turner gentoo org> AuthorDate: Thu May 14 02:37:11 2020 +0000 Commit: Matt Turner gentoo org> CommitDate: Tue May 19 22:54:21 2020 +0000 URL: https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=709612b5 catalyst: Set jobs/load-average via catalyst.conf We currently have two mechanisms of setting MAKEOPTS: in spec files and in catalystrc. Setting makeopts in spec files doesn't make sense. The spec should describe the thing that's being built and not contain options that are specific to the build system. Setting makeopts via catalystrc is better, but it only applies to the actual build system invocations, leaving emerge to run jobs serially or again requiring configuration specific to the build machine to be put into the spec file. For example: update_seed_command: ... --jobs 5 --load-average 5 With jobs and load-average specified in catalyst.conf, catalyst has the information required to configure both emerge and the build systems emerge executes. This removes the undocumented makeopts spec file option and replaces it with jobs and load-average settings in catalyst.conf. Signed-off-by: Matt Turner gentoo.org> catalyst/base/stagebase.py | 12 +++++------- catalyst/defaults.py | 2 ++ doc/catalyst-config.5.txt | 24 +++++++++++++++--------- etc/catalystrc | 3 --- 4 files changed, 22 insertions(+), 19 deletions(-) diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py index 537ab752..2e6b349a 100644 --- a/catalyst/base/stagebase.py +++ b/catalyst/base/stagebase.py @@ -56,7 +56,6 @@ class StageBase(TargetBase, ClearBase, GenBase): "hostuse", "kerncache_path", "ldflags", - "makeopts", "pkgcache_path", "portage_confdir", "portage_overlay", @@ -1308,12 +1307,11 @@ class StageBase(TargetBase, ClearBase, GenBase): else: self.env[varname] = "false" - if "makeopts" in self.settings: - if isinstance(self.settings["makeopts"], str): - self.env["MAKEOPTS"] = self.settings["makeopts"] - else: - # ensure makeopts is a string - self.env["MAKEOPTS"] = ' '.join(self.settings["makeopts"]) + makeopts = [] + for flag, setting in {'j': 'jobs', 'l': 'load-average'}.items(): + if setting in self.settings: + makeopts.append(f'-{flag}{self.settings[setting]}') + self.env['MAKEOPTS'] = ' '.join(makeopts) log.debug('setup_environment(); env = %r', self.env) diff --git a/catalyst/defaults.py b/catalyst/defaults.py index 14f671fe..b31d5b50 100644 --- a/catalyst/defaults.py +++ b/catalyst/defaults.py @@ -12,6 +12,8 @@ valid_config_file_values = frozenset([ "digests", "distdir", "envscript", + "jobs", + "load-average", "options", "port_logdir", "repo_basedir", diff --git a/doc/catalyst-config.5.txt b/doc/catalyst-config.5.txt index 7ac9a2a3..4ea4690f 100644 --- a/doc/catalyst-config.5.txt +++ b/doc/catalyst-config.5.txt @@ -48,9 +48,9 @@ $ python3 -c 'import hashlib; print(hashlib.algorithms_available)' *envscript*:: Environment script location, which allows users to set options such as -HTTP proxies, `MAKEOPTS`, `GENTOO_MIRRORS`, or any other environment -variables needed for building. The envscript file sets environment -variables using POSIX shell notation: +HTTP proxies, `GENTOO_MIRRORS`, or any other environment variables +needed for building. The envscript file sets environment variables +using POSIX shell notation: + --------------------------------- export FOO="bar" @@ -136,6 +136,18 @@ written to the target's make.conf if it is not the default value of Other settings ~~~~~~~~~~~~~~ +*jobs*:: +Number of jobs to execute simultaneously. + +*load-average*:: +Load-average + +*port_logdir*:: +Location for build logs (example: `/var/tmp/catalyst/tmp`). This dir +will be automatically cleaned of all logs over 30 days old. If left +undefined the logs will remain in the build directory as usual and get +cleaned every time a stage build is restarted. + *sharedir*:: Catalyst runtime script location. `/usr/share/catalyst` should work for most default installations. If you are running catalyst from a Git @@ -145,12 +157,6 @@ checkout, you should change this to point to your checkout directory. Location for built seeds, temporary files, and caches (example: `/var/tmp/catalyst`). -*port_logdir*:: -Location for build logs (example: `/var/tmp/catalyst/tmp`). This dir -will be automatically cleaned of all logs over 30 days old. If left -undefined the logs will remain in the build directory as usual and get -cleaned every time a stage build is restarted. - *var_tmpfs_portage*:: Set the size of a `/var/tmp/portage` tmpfs in gigabytes (example: `16`). If set, this mounts a tmpfs for `/var/tmp/portage` so building diff --git a/etc/catalystrc b/etc/catalystrc index bcd729af..e7904128 100755 --- a/etc/catalystrc +++ b/etc/catalystrc @@ -1,5 +1,2 @@ #!/bin/bash # This is an example catalystrc. As such, it doesn't actually *do* anything. - -# Uncomment the following to increase the number of threads used to compile. -# export MAKEOPTS="-j16"