Commit Graph

98 Commits (e37779c11db42104408d6c2985d21da679df9c44)

Author SHA1 Message Date
Leah Rowe e37779c11d trees: don't continue if no argument given
main() used to be the only function executed from
outside of main(), in this script, but now we source
a config file and then run the build afterward.

when a flag is provided without OPTARG, this means
that we are continuing such action erroneously. to
mitigate this, return 1 in that instance, and handle
it in the line that calls main(), making it exit with
zero status (success).

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-11 00:28:32 +01:00
Leah Rowe 3263eeb6b6 trees: general code cleanup
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 23:11:23 +01:00
Leah Rowe 17f3e61a0e trees: merge build_targets() with handle_targets()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 21:43:13 +01:00
Leah Rowe c0a4df688c trees: use wrapper for dry-running make commands
instead of using lots of if/else conditions, do that once
and set a variable, dry, to :

if not doing a dry run, the variable is empty. prefix this
variable in places where you don't want a certain action to
be performed, on dry runs.

more specifically, : does *nothing* and always returns with
zero status (success).

this results in cleaner code, and a small sloccount reduction.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 21:00:16 +01:00
Leah Rowe 459db1cc2e trees: remove project-specific hacks
move the coreboot-specific includes into mkhelper.cfg
for that project.

on some projects, we need variables from mkhelper.cfg
to be global, so I was including serprog and coreboot
mkhelper.cfg files in this script.

instead, set a new variable "mkhelpercfg" pointing to
the config file. if it doesn't exist, create and then
point to a temporary (empty) mkhelper.cfg file.

the rom.sh include has been moved to coreboot mkhelper.cfg

The only remaining project-specific logic, in this trees
script, is now the coreboot crossgcc handling, but this
needs to be there as it's also used to build U-Boot.

The way this now works, certain includes are done twice.
For example, include/rom.sh will be included once globally,
outside of main(), and then again in configure_project().

This means that certain functions will be defined twice.
I'm uncertain if shell has anything equivalent to an ifdef
guard as in C, but we actually want this here anyway, and
it shouldn't cause any problems. It's a bit of a hack, but
otherwise results in much cleaner code.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 13:41:15 +01:00
Leah Rowe 469cc64f99 trees: fix bad comparison
pro-tip: don't do this at 3AM

do massive changes like this, no later than 1AM.

the intent anyway is for -d to cause no build dependencies
to be handled, but the current logic says to only handle
them if -d is set! fix it by removing the ! part

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 01:11:14 +01:00
Leah Rowe 9646172145 trees: support -d (dry run) for custom build logic
-d does the same as -b, except for actually building
anything! in effect, it does the same as -f (fetch)
except that the resulting variable assignments will
not be recursive (as with -f).

if -d is passed, configuration is still loaded, defconfig
files are still cycled through, and more importantly:

helper functions are still processed.

the grub, serprog and coreboot helper functions have
been modified to return early (zero status) if -d is
passed.

this behaviour will be used to integrate vendor.sh
logic in with the trees script, for cases where the
user wants to only handle vendor files. e.g.:

./update trees -b coreboot x230_12mb

this would download the files as usual, build coreboot,
with those files, and then build the payloads. but:

./update trees -d coreboot x230_12mb

this would download the files, NOT build coreboot, and
NOT build the payloads.

this change increases the sloccount a bit, but i'm relying
on the fact that the vendor.sh script already re-implements
config handling wastefully; the plan is to only use trees.

for now, simply stub the same ./vendor download command.

there is one additional benefit to doing it this way:

this method is *per-kconfig* rather than per-target.
this way, one kconfig might specify a given vendor file
that is not specified in the other. although the stub
still simply handles this per target, it's done in premake,
which means that the given .config file has been copied.

this means that when i properly re-integrate the logic
into script/trees, i'll be able to go for it per-kconfig.

the utils command has been removed, e.g.
./update trees -b coreboot utils default

the equivalent is now:
./update trees -d coreboot default

this would technically download vendor files, but here
we are specifying a target for which no kconfigs exist;
a check is also in place, to avoid running the vendor file
download logic if tree==target

the overall effect of this change is that the trees script
no longer contains any project-specific logic, except for
the crossgcc build logic.

it does include some config/data mkhelper files at the top,
for serprog and coreboot, so that those variables defined in
those files can be global, but another solution to mitigate
that will also be implemented in a future commit.

the purpose of this and other revisions (in the final push
to complete lbmk audit 6 / cbmk audit 2) is to generalise as
much logic as possible, removing various ugly hacks.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-09 02:55:20 +01:00
Leah Rowe 2f3cc5d379 rom.sh: new file, to replace script/roms
stub it from the trees script. the way it works now,
there is less code in the build system.

./build roms

this is no longer a thing

./build roms serprog

this is also no longer a thing. instead, do:

./update trees -b coreboot targetnamehere

./update trees -b pico-serprog

./update trees -b stm32-vserprog

the old commands still works, which causes the new
commands to run

coreboot roms now appear in elf/, not bin/, as before,
but those images now contain payloads.

NOTE: to contradict the above: ./build roms is no
longer a thing, in that it's now deprecated, but
backward compatibility is present for now. it will
be removed in a future release.

./build roms list also still works! it will do:
./update trees -b coreboot list

also:
./update trees -b grub list
this is now possible too

if a target "list" is provided, for multi-tree sources,
the targets are shown.

there is another difference: seagrub roms are now seagrub_,
instead of seabios_withgrub.

seabios-only roms are no longer provided, where grub is also
enabled; only seagrub is used. the user can easily remove
the bootorder file, if they want seabios to not try grub first.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-07 21:25:07 +01:00
Leah Rowe 6d4fb200ac trees: rename load_target_config()
rename it to configure_project, because the function now
also handles building (a little bit), not just mere loading
of configuration files.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-06 11:34:27 +01:00
Leah Rowe 87df6ae369 trees: support build dependencies, not just fetch
We already have this to an extent, e.g. the xtree variable.

The xtree variable could probably be removed, in favour of
this, and used for the same purpose.

It works like this, for example:

build_depend="coreboot/default grub/xhci seabios u-boot/gru_bob flashprog"

the "/" denotes a tree, if it's a multi-tree project. However, specifying
the entire multi-tree project without slash is possible, for example:

build_depend="coreboot"

this would specify that all coreboot trees must be built.

This functionality will be used in follow-up commits, centralising
script/trees into mk on the main directory, repacing "build".

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-06 11:34:19 +01:00
Leah Rowe 065453b72e trees: just do makeargs on coreboot, not cbmakearg
stick the makeargs in mkhelper

i previously did cbmakeargs because the old revisions
had to define makeargs per-target otherwise. mkhelper
was done specifically to solve that problem.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-05 18:17:23 +01:00
Leah Rowe 2ab1d99490 trees: fix bad rm -Rf command (-Rf, not Rf)
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-01 13:03:04 +01:00
Leah Rowe 3ef84af9fd trees: allow global config on multi-tree projects
config/data/PROJECT/mkhelper.cfg can be provided, for
configuration, and it is loaded *before* target.cfg

there are certain instances where we repeat a lot of
config per tree, in multi-tree projects.

for example, we have the exact same config per grub
tree, besides tree name and revision number, for things
like autoconf arguments.

this last problem will be addressed, in a follow-up
patch, and then expanded upon for other projects.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 22:22:15 +01:00
Leah Rowe 2b49714a6a trees: handle all projects if project undefined
e.g. ./update trees -f

if passed, this command would download every tree

similarly, the -c option can be used in this way. this
solves a longstanding issue: on the current, much more
efficient design, it was not possible to systematically
clean every project.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 21:33:30 +01:00
Leah Rowe 67421a21fd trees and git.sh: tidy up global variables
some of the variables only initialised in git.sh are
also used in the trees script, which is technically ok
because git.sh is included from the trees script, but
it makes more sense to declare them in the latter.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 18:10:22 +01:00
Leah Rowe 0cb84a8dd1 trees: simplified distclean directory check
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 17:13:49 +01:00
Leah Rowe c62dbdbe82 trees: only do bare multi-tree clone on git
i forgot to add the check for CHANGELOG

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 16:23:32 +01:00
Leah Rowe 89a81a2910 trees: simplified multi-tree bare repo clone
the git_prep function already creates the given
directory where source code goes, so we don't
need to do it from the trees script.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 16:16:07 +01:00
Leah Rowe f937a1142d trees run_make_command: simpler distclean handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 15:46:03 +01:00
Leah Rowe 88f741db92 trees: condense copy_elf() a bit
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 15:39:41 +01:00
Leah Rowe d5928c9ba8 trees: add return to handle_defconfig()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 14:39:20 +01:00
Leah Rowe 1a7c8acb0e trees: rename check_config to check_defconfig
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 14:13:52 +01:00
Leah Rowe ba7b3a3b21 trees: remove variable "config_name"
it's only ever used once, so just use the value that
it's set to, without declaring the variable itself

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 14:10:49 +01:00
Leah Rowe 030f1afd8e trees: rename variable "config" to defconfig
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 14:08:22 +01:00
Leah Rowe 0c4ad24b05 trees: remove unnecessary commonts
behaviour described there is obvious just
from analysing the while loop condition

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 13:46:31 +01:00
Leah Rowe eacc41e82a trees: condense run_make_command() a bit
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 13:42:42 +01:00
Leah Rowe 5639481419 trees: condense handle_makefile() a bit
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 13:30:35 +01:00
Leah Rowe 1ec9fc3368 trees: mv load_project_config load_target_config
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 01:07:59 +01:00
Leah Rowe 789631ccb8 trees, multi: download bare project *before* trees
when downloading multi-tree projects, the rev can be reset
to HEAD instead of the actual rev for a given target. this
occurs when the bare repo (e.g. src/coreboot/coreboot) does
not exist and has to be downloaded first.

bare repository downloading does not rely on target.cfg, in
this context, only pkg.cfg, but it uses the same variable
names (e.g. "rev").

instead of using a separate variable name, thus increasing
code complexity (which is the exact opposite of what i want
to do), do the bare repository download first.

this means that the git.sh script is much cleaner now, for
multi-tree projects, in that it *only* copies the bare repo
then runs git_prep; in that context, the bare repo is cloned
directly by calling the relevant function from script/trees,
which is the same behaviour as when cloning single-tree
project sources.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 00:40:30 +01:00
Leah Rowe a5e724e51e trees: unified multi-tree configuration handling
the same function that loads configurations for single-tree
projects has been merged with the function for multi-tree
configs in git.sh, and that functionality has been removed
from git.sh; now it is all unified in the trees script.

as the saying goes: write one program to do one thing well.

the purpose of git.sh is to download source code, but not
to handle configuration files; the latter is meant to be
handled by the trees script, which then calls into git.sh
before running the build logic for that given project.

additionally: the "seen" files are no longer handled, at all.
the logic there was added ages ago, because at the time, i was
considering whether to separate configuration into a new
repository, so that users could more easily make their own
configuration, so it was a guard against misconfiguration.
however, that decision was canceled and we're always very
careful not to introduce a loop; if a loop does occur, the
worst that can possibly happen is you waste some CPU cycles.

Instead, print (on standard output) what config file is being
used, so the operator can see when an infinite loop occurs.

ALSO:

remove _setcfgarg in load_project_config()

it was used to skip when a target.cfg file didn't exist,
specifically on single-tree projects, but this is now
handled using -f instead, on the while loop inside that
function, so _setcfgarg is now a redundant variable.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 00:02:03 +01:00
Leah Rowe cfc9f62ffe trees: unified handling of source downloads
run it from load_project_config()

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 22:25:54 +01:00
Leah Rowe 7bf2835207 git.sh: rename Fetch_project_trees fetch_targets
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:59:16 +01:00
Leah Rowe e0c244f37b git.sh: rename fetch_project_repo to fetch_project
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:58:03 +01:00
Leah Rowe 02f7417952 trees: better skip-clean string in handle_src_tree
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:47:25 +01:00
Leah Rowe f7ec42d379 trees: add return to check_coreboot_utils()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:46:07 +01:00
Leah Rowe 8d02adfbb5 trees: simplify "utilmode" variable initialisation
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:44:47 +01:00
Leah Rowe fbe36b01ac trees mkpayload_grub: remove unnecessary check
these variables are initialised empty, then populated
by reading a configuration file.

it may be that in some cases, we want these variables
to be empty. besides that, the "setcfg" command before
it will throw an error if the module file is missing,
and it is assumed that the variables would be set there.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 18:39:25 +01:00
Leah Rowe 3f37c3788b tidy up some setvars lists
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28 15:02:11 +01:00
Leah Rowe afc0030682 remove more unnecessary checks on trees commands
the trees script already performs these checks

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28 14:51:32 +01:00
Leah Rowe 136787185d trees: don't hardcode use of mkpayload_grub
instead, make it a helper function, defined in target.cfg

this means that we can also do the same with other projects
in the future, and it is expected that we will have to.

these helper functions are used in cases where we want
additional actions to be performed.

actually, the helper could be anything. for example, you
could write:

mkhelper="./build foo bar"

and it would do that (at the point of execution, PWD
is the root directory of the build system)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:51:48 +01:00
Leah Rowe c59fbb3651 trees: simplify single/multi handling in main()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:43:35 +01:00
Leah Rowe 902b98d1ac trees: rename build_projects to build_project
the script used to support building multiple single-tree
projects, but this behaviour was buggy and unused, so it
was removed.

rename the build_projects variable accordingly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:41:23 +01:00
Leah Rowe e1e04aa80d trees: err if target.cfg not given if multi-tree
it was always by design that an error should occur, if a
target.cfg file does not exist on multi-tree projects,
but we previously did not support target.cfg files on
single-tree projects.

single-tree target.cfg support was later added, and it was
done by making target.cfg optional there, but i accidentally
made it optional on multi-tree projects.

in practise, all multi-tree projects included target.cfg,
but this was not being enforced in code.

this patch should fix the issue.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:38:49 +01:00
Leah Rowe 93ff80d96d trees: set btype if target.cfg doesn't exist
set it to "auto", because otherwise it'll be unset,
which means that kconfig type is assumed.

the build system is designed in such a way that multi-tree
is assumed, if the target build system uses kconfig files.

target.cfg is optional on single-tree but not multi-tree,
so it's ok to set something here.

basically, kconfig-type projects will never be single-tree.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:33:30 +01:00
Leah Rowe 7a15ba18cb trees: avoid kconfig make commands generically
don't hardcode the check based on whether the current
project is grub. instead, define "btype" in target.cfg

if unset, we assume kconfig and permit kconfig commands
e.g. make menuconfig, make silentoldconfig, etc

this is to avoid the deadliest of sins:
project-specific hacks

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 15:55:56 +01:00
Leah Rowe 144b3c93a4 trees: condense elfcheck() a bit
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 14:05:19 +01:00
Leah Rowe ffdecb0ce1 trees: shorten the final confirmation message
must stay below 80 characters per line

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 03:36:32 +01:00
Leah Rowe 19bc5845a6 trees: remove redundant space in printf
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 02:45:11 +01:00
Leah Rowe 9154ba5b85 trees: explicitly err if OPTARG is not set
we currently rely on -e to make this happen

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 02:44:01 +01:00
Leah Rowe f383b1ad70 trees: only permit one single-tree project
the current logic for handling multiple single-tree projects
is quite error-prone, and uses recursion.

since we don't actually use it this way, remove that feature.

the most correct way to do it is with a for loop.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 02:42:59 +01:00