Commit Graph

67 Commits (7bf28352078951e9eb78ec1aa467346031c8628e)

Author SHA1 Message Date
Leah Rowe 7bf2835207 git.sh: rename Fetch_project_trees fetch_targets
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:59:16 +01:00
Leah Rowe e0c244f37b git.sh: rename fetch_project_repo to fetch_project
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:58:03 +01:00
Leah Rowe 02f7417952 trees: better skip-clean string in handle_src_tree
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:47:25 +01:00
Leah Rowe f7ec42d379 trees: add return to check_coreboot_utils()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:46:07 +01:00
Leah Rowe 8d02adfbb5 trees: simplify "utilmode" variable initialisation
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:44:47 +01:00
Leah Rowe fbe36b01ac trees mkpayload_grub: remove unnecessary check
these variables are initialised empty, then populated
by reading a configuration file.

it may be that in some cases, we want these variables
to be empty. besides that, the "setcfg" command before
it will throw an error if the module file is missing,
and it is assumed that the variables would be set there.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 18:39:25 +01:00
Leah Rowe 3f37c3788b tidy up some setvars lists
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28 15:02:11 +01:00
Leah Rowe afc0030682 remove more unnecessary checks on trees commands
the trees script already performs these checks

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28 14:51:32 +01:00
Leah Rowe 136787185d trees: don't hardcode use of mkpayload_grub
instead, make it a helper function, defined in target.cfg

this means that we can also do the same with other projects
in the future, and it is expected that we will have to.

these helper functions are used in cases where we want
additional actions to be performed.

actually, the helper could be anything. for example, you
could write:

mkhelper="./build foo bar"

and it would do that (at the point of execution, PWD
is the root directory of the build system)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:51:48 +01:00
Leah Rowe c59fbb3651 trees: simplify single/multi handling in main()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:43:35 +01:00
Leah Rowe 902b98d1ac trees: rename build_projects to build_project
the script used to support building multiple single-tree
projects, but this behaviour was buggy and unused, so it
was removed.

rename the build_projects variable accordingly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:41:23 +01:00
Leah Rowe e1e04aa80d trees: err if target.cfg not given if multi-tree
it was always by design that an error should occur, if a
target.cfg file does not exist on multi-tree projects,
but we previously did not support target.cfg files on
single-tree projects.

single-tree target.cfg support was later added, and it was
done by making target.cfg optional there, but i accidentally
made it optional on multi-tree projects.

in practise, all multi-tree projects included target.cfg,
but this was not being enforced in code.

this patch should fix the issue.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:38:49 +01:00
Leah Rowe 93ff80d96d trees: set btype if target.cfg doesn't exist
set it to "auto", because otherwise it'll be unset,
which means that kconfig type is assumed.

the build system is designed in such a way that multi-tree
is assumed, if the target build system uses kconfig files.

target.cfg is optional on single-tree but not multi-tree,
so it's ok to set something here.

basically, kconfig-type projects will never be single-tree.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 16:33:30 +01:00
Leah Rowe 7a15ba18cb trees: avoid kconfig make commands generically
don't hardcode the check based on whether the current
project is grub. instead, define "btype" in target.cfg

if unset, we assume kconfig and permit kconfig commands
e.g. make menuconfig, make silentoldconfig, etc

this is to avoid the deadliest of sins:
project-specific hacks

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 15:55:56 +01:00
Leah Rowe 144b3c93a4 trees: condense elfcheck() a bit
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 14:05:19 +01:00
Leah Rowe ffdecb0ce1 trees: shorten the final confirmation message
must stay below 80 characters per line

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 03:36:32 +01:00
Leah Rowe 19bc5845a6 trees: remove redundant space in printf
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 02:45:11 +01:00
Leah Rowe 9154ba5b85 trees: explicitly err if OPTARG is not set
we currently rely on -e to make this happen

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 02:44:01 +01:00
Leah Rowe f383b1ad70 trees: only permit one single-tree project
the current logic for handling multiple single-tree projects
is quite error-prone, and uses recursion.

since we don't actually use it this way, remove that feature.

the most correct way to do it is with a for loop.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 02:42:59 +01:00
Leah Rowe a64e2db589 trees: call err if multiple flags are provided
this script is designed to only run a single flag.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 02:33:45 +01:00
Leah Rowe 0e97e98fe1 trees: explicitly set mode for -b
doing nothing means that if a flag is passed, and then
another flag overriding it, the resulting action will
not be correct; only one flag should be provided anyway,
but some users may feel a bit more adventurous.

mitigate it.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27 02:31:54 +01:00
Leah Rowe 4e48fa808e lib.sh: simplify use of environment variables
don't have a separate variable for them.

just export them directly and use them directly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-25 00:12:18 +01:00
Leah Rowe 19e7c1eabb trees: try xgcc build twice if first attempt fails
sometimes buildgcc just fails for like no reason. we had this
the other day and another fix was made to the trees script, to
mitigate; the user ran it again and buildgcc worked just fine.

run it twice, and then call err only if the second one fails.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24 14:23:56 +01:00
Leah Rowe 6468bdb3aa trees: don't check if xgcc is already built
just run the make crossgcc command anyway.
coreboot's own build system checks itself, and much
more reliably, but the check is more thorough and a bit
slower.

in rare cases, lbmk may come into build issues with xgcc,
and if you run the build again, it will always fail every
time because the checks is based on whether the xgcc
directory exists, rather than checking each
individual crossgcc binary.

checking every binary is also possible, but as i said,
the coreboot build system already does that, so let's defer
to coreboot's own handling of it.

remove the directory check. this will slow down the build
process a little bit, but should improve reliability under
fault conditions.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-23 17:24:28 +01:00
Leah Rowe d4d5d2902c use backticks on eval commands, not subshells
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22 13:46:20 +01:00
Leah Rowe c1527b6114 lib.sh: remove badcmd()
it's bloat. telling the user to rtfm is something that
we already do on irc; they will still ask how to do
everything, and ignore the message from badcmd(), or
they will automatically know to rtfm.

i'm on a massive purge, removing bloat from lbmk as
part of Libreboot Build System Audit 6.

all bloat must go.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22 13:46:20 +01:00
Leah Rowe fc7ae3e590 lib.sh: more unified config handling
replace it with logic that simply uses "." to load
files directly. for this, "vcfg" is added as a variable
in coreboot target.cfg files, referring to a directory
in config/vendor/ containing a file named pkg.cfg, and
this file then contains the same variables as the
erstwhile config/vendor/sources

config/git files are now directories, also containing
pkg.cfg files each with the same variables as before,
such as repository link and commit hash

this change results in a noticeable reduction in code
complexity within the build system.

unified reading of config files: new function setcfg()
added to lib.sh

setcfg checks if a config exists. if a 2nd argument is
passed, it is used as a return value for eval, otherwise
a string calling err is passed. setcfg output is passed
through eval, to set strings based on config; eval must
be used, so that the variables are set within the same
scope, otherwise they'd be set within setcfg which could
lead to some whacky results.

there's still a bit more more to do, but this single change
results in a substantial reduction in code complexity.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22 13:44:27 +01:00
Leah Rowe ad1602569e trees: more robust check to avoid "make fetch"
do not use shorthand here. the test was failing to
produce the desired result under some circumstances,
for example when i did "./update release" i got this:

make: Entering directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog'
make: *** No rule to make target 'fetch'.  Stop.
make: Leaving directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog'
ERROR script/trees: !mk src/stm32-vserprog fetch
ERROR ./update: excmd: script/trees -f
ERROR script/roms: Unhandled non-zero exit: ./update
ERROR ./build: excmd: script/roms serprog
ERROR ./update: build_release release/20240612-62-ga6b1a6bd: stm32
ERROR ./update: can't build rom images

in the above circumstance, run_make_command was executed,
which is not the desired behaviour; rather, fetch_project_trees
or fetch_project_repo should be called, and then the script
should immediately exit. it should also exit, without downloading
anything, if a changelog file exists as in release archives.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-21 03:10:46 +01:00
Leah Rowe 808458ced5 minor code cleanup in the build system
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-20 01:15:06 +01:00
Leah Rowe e67628c6a5 unify coreboot elfdir (DO_NOT_FLASH)
use a common string when setting this path

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-16 17:46:58 +01:00
Leah Rowe 0812d53212 trees: simplify copy_elf()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15 23:15:27 +01:00
Leah Rowe 331c4097ff trees: remove unnecessary check in copy_elf
we don't need to check whether this variable is set,
because checking an empty path will also cause the
same return in the next line.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15 23:15:27 +01:00
Leah Rowe 340eea0b1c grub: insert background in memdisk instead
the background is only a few kb. the whole rationale
before was to limit the space used in memdisk, but this
decision was made when the background was much bigger;
it has since been optimised greatly, and the grub modules
were heavily reduce, so it should be safe.

grub's memdisk breaks when you add too much data to it.
as part of simplifying the rest of lbmk, this change removes
some more bloat from the rest of lbmk. handling this in the
memdisk is much simpler than handling it with cbfstool.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15 23:15:27 +01:00
Leah Rowe 167e1a0fbe unified checks for variable initialisation
new function chkvars() does the job

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-14 13:37:03 +01:00
Leah Rowe 98724d701b lib.sh: remove the items() function
it's pretty much just doing the same thing as ls -1

remove it!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-14 13:19:25 +01:00
Leah Rowe c5441bb9f5 re-add ability to use cbfs grub.cfg as default
i removed this before, when making grub multi-tree,
because the design i used in an earlier version of
the patch actually added the grub.elf generation
to grub source itself, but then i decided to hack
around the grub build system from lbmk/cbmk instead

re-add this functionality, so that users can easily
insert their own custom grub.cfg into cbfs without
needing to re-build their image.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09 16:06:07 +01:00
Leah Rowe d33556c6ae trees: exit with error if project undefined
i was originally looser about this, because i also wanted
the trees script to generically run "make" from any
directory, but this behaviour was error-prone and it is
no longer used in the build system.

disable it, in the interest of stability.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09 15:58:09 +01:00
Leah Rowe a0eb79dfd8 add crossgcc tarballs to config/submodules/
support redundant downloads, and enable inclusion of these
tarballs inside release archives, for offline builds.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09 06:55:25 +01:00
Leah Rowe 668bcbf69c trees: simplified copy_elf() handling
don't create elfdir, create dest_dir, which is elfdir
plus the location within it

only create dest_dir within copy_elf, which is only
called if actually compiling the code

this avoids creating empty elf directories, and it
generally cleans up all handling, unifying the
handling of directories into a single function,
namely copy_elf() which already exists

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 15:04:24 +01:00
Leah Rowe d43247683b git.sh: download "depend" projects *before*
don't do it after, because that means the main project
is saved under src/ before we know whether the subrepo
was downloaded.

the "depend" variable (in config/git/) is no longer used
for projects that go in subdirectories of a parent; now,
we use config/submodules/ for this type of dependency.

download the "depend" projects (as per config/git/) first.
this way, if they fail, the main one will fail, but if
they succeed and main fails, you can just run the main
download again and it won't fail.

this fixes a bug where, depending on how you download a
set of projects and depending on the order which you do so,
a given project can become un-downloadable on current design,
because git will complain that a directory already exists.

this fix is done not only in code (by this commit), but
by prior configuration changes.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 14:16:58 +01:00
Leah Rowe 7bfb1d621f trees: don't check empty path in copy_elf()
it could lead to some whackiness later on

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 12:04:28 +01:00
Leah Rowe 0b7566cb78 trees: fix build issue caused by bad elf check
we're not checking for bad elfs, but the check itself was bad

due to a quirk in how sh works. really, really obscure bug.

fixed now!

if the given directory didn't actually exist, or there were no
files in it, it'd be searching for the file named "*"

which is obviously wrong

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 11:29:36 +01:00
Leah Rowe 7aa9f22409 trees: fix listfile check in copy_elf()
don't check that the variable is empty

check that the file itself exists or not

this should fix the recent build issues

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 10:35:38 +01:00
Leah Rowe 06c78e1321 trees: don't say check elf/ if build.list missing
(on single tree projects. this complements the last patch)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 10:32:39 +01:00
Leah Rowe dea41f138a trees: don't do elfcheck if build.list missing
otherwise, some checks are done too soon, and nothing
gets built.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 10:31:01 +01:00
Leah Rowe d0f99c2f98 trees: unified coreboot makeargs
in particular, the coreboot build system may auto-download
submodules when building cbfstool; vboot for instance.

we do not want such unpredictable behaviour, so now we
use UPDATED_SUBMODULES=1 when building coreboot utilities.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 22:21:59 +01:00
Leah Rowe a7889c5af8 trees: use multiple threads to build cbutils
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 22:20:09 +01:00
Leah Rowe d41658f17e move handle_coreboot_utils to script/trees
it doesn't really make sense placed in lib.sh,
because it's only called from script/trees

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 22:17:16 +01:00
Leah Rowe 7e49fe4b9b trees: skip single-tree build if a build exists
this replicates the same behaviour as multi-tree builds,
checking for files inside the relevant elf/ directory

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 09:20:45 +01:00
Leah Rowe 48cbb30db1 trees: also print "DONE! check elf/dir" on single
e.g. ./update trees -b flashprog

tell the user to check elf/flashprog

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 08:53:29 +01:00