we're building it per coreboot configuration file, rather
than per-target; the latter is more appropriate, and saves
on compilation time.
do it per-target.cfg, not per coreboot configuration.
this works because the trees script compiles all images
per target, for each given coreboot configuration within
that target, e.g. libgfxinit _corebootfb and _txtmode.
Signed-off-by: Leah Rowe <leah@libreboot.org>
sometimes buildgcc just fails for like no reason. we had this
the other day and another fix was made to the trees script, to
mitigate; the user ran it again and buildgcc worked just fine.
run it twice, and then call err only if the second one fails.
Signed-off-by: Leah Rowe <leah@libreboot.org>
just run the make crossgcc command anyway.
coreboot's own build system checks itself, and much
more reliably, but the check is more thorough and a bit
slower.
in rare cases, lbmk may come into build issues with xgcc,
and if you run the build again, it will always fail every
time because the checks is based on whether the xgcc
directory exists, rather than checking each
individual crossgcc binary.
checking every binary is also possible, but as i said,
the coreboot build system already does that, so let's defer
to coreboot's own handling of it.
remove the directory check. this will slow down the build
process a little bit, but should improve reliability under
fault conditions.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Never, ever build images where GRUB is the primary payload.
These options have been removed from target.cfg handling:
* seabios_withgrub
* grub_withseabios
The "payload_grub" variable now does the same thing as
the old "seabios_withgrub" variable, if set.
The "grubonly" configuration is retained, and enabled by
default when SeaGRUB is enabled (non-grubonly also available).
Due to lbmk issue #216, it is no longer Libreboot policy to
make GRUB the primary payload on any board. GRUB's sheer size
and complexity, plus the large number of memory corruption issues
similar to it that *have* been fixed over the years, tells me
that GRUB is a liability when it is the primary payload.
SeaBIOS is a much safer payload to run as primary, on x86, due
to its smaller size and much more conservative development; it
is simply far less likely to break.
If GRUB breaks in the future, the user's machine is not
bricked. This is because SeaBIOS is the default payload.
Since I no longer wish to ever provide GRUB as a primary
payload, supporting it in lbmk adds needless bloat that
will later probably break anyway due to lack of testing,
so let's just assume SeaGRUB in all cases where the user
wants to use a GRUB payload.
You can mitigate potential security issues with SeaBIOS
by disabling option ROM execution, which can be done at
runtime by inserting integers into CBFS. The SeaBIOS
documentation says how to do this.
Libreboot's GRUB hardening guide still says how to add
a bootorder file in CBFS, making SeaBIOS only load GRUB
from CBFS, and nothing else. This, combined with the
disablement of option ROM execution (if using Intel
graphics), pretty much provides the same security benefits
as GRUB-as-primary, for example when setting a GRUB password
and GPG checks, with encrypted /boot as in the hardening guide.
Signed-off-by: Leah Rowe <leah@libreboot.org>
due to lbmk issue #216, it is now unwise to use grub as the
primary payload on any machine; the sheer complexity of grub
and the number of memory corruption bugs that have been fixed
due to auditing over the years, means more such bugs exist.
we now provide seabios as the primary payload on all x86 ports,
but provide a "grubfirst" configuration where a bootorder file
in seabios can be added via cbfs, which tells seabios to load
grub from cbfs first, while still allowing use of the boot select
menu by pressing esc in seabios.
well, the "grubonly" option also disables the seabios esc menu,
so that *only* grub runs. there is no point in using this unless
you want to harden your setup, for example if you want to set up
encrypted /boot and boot that from grub, and have a grub password
disallowing unauthorised bootup of your machine.
see grub hardening guide;
https://libreboot.org/docs/linux/grub_hardening.html
at least as of today, 22 June 2024, that page already says
how to manually disable the seabios menu in the same way, if that
is the setup you want. alternatively, a user may be wily
enough to edit target.cfg for their board and compile a rom
that only has the grub payload in it, if that is what the user
wishes to do.
regardless, the default configurations provided by lbmk must never
be unsafe, norc should the build system support such unsafe
settings;
yes, grub as primary payload is technically still supported in
lbmk. actually, at the time of this revision, i have half a mind
to remove that functionality altogether, so that only seabios is
allowed as primary payload, when compiling a rom image that also
has grub, chainloading grub from the seabios menu instead.
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's bloat. telling the user to rtfm is something that
we already do on irc; they will still ask how to do
everything, and ignore the message from badcmd(), or
they will automatically know to rtfm.
i'm on a massive purge, removing bloat from lbmk as
part of Libreboot Build System Audit 6.
all bloat must go.
Signed-off-by: Leah Rowe <leah@libreboot.org>
replace it with logic that simply uses "." to load
files directly.
config/git files are now directories, also containing
pkg.cfg files each with the same variables as before,
such as repository link and commit hash
this change results in a noticeable reduction in code
complexity within the build system.
unified reading of config files: new function setcfg()
added to lib.sh
setcfg checks if a config exists. if a 2nd argument is
passed, it is used as a return value for eval, otherwise
a string calling err is passed. setcfg output is passed
through eval, to set strings based on config; eval must
be used, so that the variables are set within the same
scope, otherwise they'd be set within setcfg which could
lead to some whacky results.
there's still a bit more more to do, but this single change
results in a substantial reduction in code complexity.
Signed-off-by: Leah Rowe <leah@libreboot.org>
do not use shorthand here. the test was failing to
produce the desired result under some circumstances,
for example when i did "./update release" i got this:
make: Entering directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog'
make: *** No rule to make target 'fetch'. Stop.
make: Leaving directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog'
ERROR script/trees: !mk src/stm32-vserprog fetch
ERROR ./update: excmd: script/trees -f
ERROR script/roms: Unhandled non-zero exit: ./update
ERROR ./build: excmd: script/roms serprog
ERROR ./update: build_release release/20240612-62-ga6b1a6bd: stm32
ERROR ./update: can't build rom images
in the above circumstance, run_make_command was executed,
which is not the desired behaviour; rather, fetch_project_trees
or fetch_project_repo should be called, and then the script
should immediately exit. it should also exit, without downloading
anything, if a changelog file exists as in release archives.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This reverts commit 3610667e3db08ff1db4f7784ff5a879d8aebce9d.
The output of some functions in the roms script are used as
an argument in cp and mv commands, also cbfstool. I overlooked
this fact in a previous code optimisation.
Revert it. The change only reduced sloccount by a few lines
anyway.
this is bloat, because it's something the user can already
do at runtime configuration anyway.
set it to a reasonable default of 8 seconds instead of 5,
and don't honour the timeout variable in target.cfg.
this will be documented in the next release.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i disabled a check in the script, while testing a prior
modification. re-introoduce the check, which is put there
to yield an error condition if no targets were compiled.
Signed-off-by: Leah Rowe <leah@libreboot.org>
payloads are compiled before coreboot, but it doesn't matter
to the build speed whether this is done first.
reduce the lines of code by checking payload builds *while*
adding them to the coreboot images. this means that coreboot
is now compiled first, before the payloads.
Signed-off-by: Leah Rowe <leah@libreboot.org>
nowadays, we don't insert GRUB keymaps automatically, for
sake of efficiency; without one, the default is US QWERTY.
Signed-off-by: Leah Rowe <leah@libreboot.org>
there are two for loops that use x as a variable anme,
and an idiosyncrasy of certain sh implementations is
that these become global;
the result in this case was that when you finish building
every target in "./build roms", it would print "libgfxinit"
repeatedly, comma separated, instead of a comma-separated
list of the targets that were built.
work around it by renaming the variable in one of the loops.
Signed-off-by: Leah Rowe <leah@libreboot.org>
cbcfg is already a global variable, so there's no reason
to set it again at the start of this function.
remove the check for whether the given coreboot config
exists, to the calling function instead of build_roms().
Signed-off-by: Leah Rowe <leah@libreboot.org>
we don't need to call mktemp everytime.
just use a staticly named file in tmpdir
and keep overwriting it.
these files are only small, and they get deleted
when the build system exits later on.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we don't need to check whether this variable is set,
because checking an empty path will also cause the
same return in the next line.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the background is only a few kb. the whole rationale
before was to limit the space used in memdisk, but this
decision was made when the background was much bigger;
it has since been optimised greatly, and the grub modules
were heavily reduce, so it should be safe.
grub's memdisk breaks when you add too much data to it.
as part of simplifying the rest of lbmk, this change removes
some more bloat from the rest of lbmk. handling this in the
memdisk is much simpler than handling it with cbfstool.
Signed-off-by: Leah Rowe <leah@libreboot.org>
if not inserted, the default keymap is usqwerty.
don't waste ssd write cycles copying so many images,
or cpu time compressing so many. the user can simply
add a keymap.gkb file to cbfs and it will work fine.
this will be documented in the next release.
Signed-off-by: Leah Rowe <leah@libreboot.org>
rely on return status per each of the three main rom
functions, to then update the "targets" variable.
use this as the basis to determine which targets were
built, during final confirmation when the script exits.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the current validation check is extremely over-engineered,
because the user override is no longer available and we're
always very careful in how we modify target.cfg per board.
remove the redundant code. trust that target.cfg is correct.
Signed-off-by: Leah Rowe <leah@libreboot.org>
p = payload
s = grub_scan_disk
d = displaymode
setting the payload is no longer safe, due to issue 216
and similar issues that might pop up in the future; it's
best left only to target.cfg, per board, so that we know
what config is safe/tested. don't let the user override it.
scandisk isn't safe to override because the given machine
may not have the type of device that the user specifies
displaymode is actually ok to set, because it simply whitelists
what configs pre-existing to actually use, but it's bloat
basically, the rule is this:
don't make it easy for the user to brick their hardware.
make it harder instead.
a user wily enough to go modifying their payload will probably
have read docs/maintain/ anyway and knows how to edit target.cfg
if they want another board configuration.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i removed this before, when making grub multi-tree,
because the design i used in an earlier version of
the patch actually added the grub.elf generation
to grub source itself, but then i decided to hack
around the grub build system from lbmk/cbmk instead
re-add this functionality, so that users can easily
insert their own custom grub.cfg into cbfs without
needing to re-build their image.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i was originally looser about this, because i also wanted
the trees script to generically run "make" from any
directory, but this behaviour was error-prone and it is
no longer used in the build system.
disable it, in the interest of stability.
Signed-off-by: Leah Rowe <leah@libreboot.org>
support redundant downloads, and enable inclusion of these
tarballs inside release archives, for offline builds.
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't create elfdir, create dest_dir, which is elfdir
plus the location within it
only create dest_dir within copy_elf, which is only
called if actually compiling the code
this avoids creating empty elf directories, and it
generally cleans up all handling, unifying the
handling of directories into a single function,
namely copy_elf() which already exists
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't do it after, because that means the main project
is saved under src/ before we know whether the subrepo
was downloaded.
the "depend" variable (in config/git/) is no longer used
for projects that go in subdirectories of a parent; now,
we use config/submodules/ for this type of dependency.
download the "depend" projects (as per config/git/) first.
this way, if they fail, the main one will fail, but if
they succeed and main fails, you can just run the main
download again and it won't fail.
this fixes a bug where, depending on how you download a
set of projects and depending on the order which you do so,
a given project can become un-downloadable on current design,
because git will complain that a directory already exists.
this fix is done not only in code (by this commit), but
by prior configuration changes.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we're not checking for bad elfs, but the check itself was bad
due to a quirk in how sh works. really, really obscure bug.
fixed now!
if the given directory didn't actually exist, or there were no
files in it, it'd be searching for the file named "*"
which is obviously wrong
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't check that the variable is empty
check that the file itself exists or not
this should fix the recent build issues
Signed-off-by: Leah Rowe <leah@libreboot.org>
in particular, the coreboot build system may auto-download
submodules when building cbfstool; vboot for instance.
we do not want such unpredictable behaviour, so now we
use UPDATED_SUBMODULES=1 when building coreboot utilities.
Signed-off-by: Leah Rowe <leah@libreboot.org>
one directory per util, under elf/
e.g. elf/cbfstool/
further split by tree name, e.g.:
elf/cbfstool/default/
elf/cbfstool/foo/
Signed-off-by: Leah Rowe <leah@libreboot.org>
this replicates the same behaviour as multi-tree builds,
checking for files inside the relevant elf/ directory
Signed-off-by: Leah Rowe <leah@libreboot.org>
the previous change makes memtest.bin get cached in elf/
but the path was being prefixed with src/ by script/roms
do away with the prefix
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's also used from script/roms, in addition to trees
move these variables to a common file used everywhere
Signed-off-by: Leah Rowe <leah@libreboot.org>
certain code checks for build.list, to skip it, for
example in items()
we already use config/data/grub to store grub config data
that applied to all trees
create these directories too:
config/data/coreboot
config/data/u-boot
config/data/seabios
move the respective build.list files in here, and also
to config/data/grub
now multi-tree projects contain, per directory, just the
target.cfg file and the patches directory. this is much
cleaner, because some of the logic can be simplified more
Signed-off-by: Leah Rowe <leah@libreboot.org>
instead, check for the presence of target.cfg files
not in config/project/ but config/project/tree/
the way this check is done, it merely returns 1 if
config/project/*/target.cfg is detected, and returns
0 in all other cases, even if config/project/target.cfg
exists
that way, if the maintainer accidentally adds a
target.cfg in the main directory, the given multi-tree
project will not break
Signed-off-by: Leah Rowe <leah@libreboot.org>
adding help again is a bad idea. code should never
document itself; that's what documentation is for.
so, make the code do a better job telling the user
where to find documentation.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The xHCI patches were removed because they caused issues
on Sandybridge-based Dell Latitude laptops. See:
https://codeberg.org/libreboot/lbmk/issues/216
The issue was not reported elsewhere, but we still don't
need xHCI support in Canoeboot's GRUB because none of the
available coreboot targets have xHCI support. However, we
may want it in the future and it helps to keep Canoeboot
in sync with Libreboot (this patch is adapted from lbmk).
Each given coreboot target can say which GRUB tree to use
by setting this in target.cfg:
grubtree="xhci"
In the above example, the "xhci" tree would be used. Some
generic GRUB config has been moved to config/data/grub/
and config/grub/ now looks like config/coreboot/ - also,
the grub.cfg file (named "payload" in each tree) is copied
to the GRUB source tree as ".config", then added to GRUB's
memdisk in the same way, as grub.cfg.
Several other design changes had to be made because of this:
* grub.cfg in memdisk no longer automatically jumps to one
in CBFS, but now shows a menuentry for it if available
* Certain commands in script/trees are disabled for GRUB,
such as *config make commands.
* gnulib is now defined in config/submodule/grub/, instead
of config/git/grub - and this mitigates an existing bug
where downloading gnulib first would make grub no longer
possible to download in lbmk.
There is another reason for merging this design change from
lbmk, and that reasoning also applies to lbmk. Specifically:
This change will enable per-board GRUB optimisation in the
future. For example, we hardcode what partitions and LVMs
GRUB scans because * is slow on ICH7-based machines, due
to GRUB's design. On other machines, * is reasonably fast,
for automatically enumerating the list of devices for boot.
Use of * (and other wildcards) could enable our GRUB payload
to automatically boot more distros, with minimal fuss. This
can be done at a later date, in subsequent revisions.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this effectively lets you change the boot order. example:
./build roms -s "nvme ata" t1650_12mb
the above example would set:
grub_scan_disk="nvme ata"
another example:
./build roms -s nvme t1650_12mb
this would set:
grub_scan_disk="nvme"
this overrides what's set in target.cfg for the given
target. useful for quick reconfiguration if building
from source
Signed-off-by: Leah Rowe <leah@libreboot.org>
i already do this on crossgcc, but overlooked it on regular
builds where i just use -j, but coreboot's build system
makes use of the CPUS= option in make
use XBMK_THREADS for this
Signed-off-by: Leah Rowe <leah@libreboot.org>
Previously, grub_scan_disk could set ata, ahci or "both",
which would make both be tried (ahci first). This worked
when we only dealt with ata and ahci devices, but now we
support nvme devices so the logic is inherently flawed.
Instead, use grub_scan_disk to store the boot order, e.g.:
grub_scan_disk="ahci nvme ata"
grub_scan_disk="nvme ata"
In the first example, it would make GRUB scan ahci first,
then nvme and then ata.
In the secontd example, it would make GRUB scan nvme first,
and then ata.
If "both" is set, or anything other than ahci/ata/nvme,
grub_scan_disk is now changed to "nvme ahci ata".
Actual grub_scan_disk entries in target.cfg files will now
be modified, to match each machine.
Signed-off-by: Leah Rowe <leah@libreboot.org>
rather than if seabios_grubonly=y
if grubonly=y, still make the grubonly rom
this complements the previous commit
Signed-off-by: Leah Rowe <leah@libreboot.org>
as stated in the previous commit, i'm adding this function
to lbmk because there are files i want to systematically
delete in libreboot releases, not just canoeboot releases,
but libreboot releases delete things such as unlicensed
readme files, or poorly licensed other files.
i initially moved the nuke function to its own file so as
to reduce the number of merge conflicts when merging
changes to git.sh between cbmk and lbmk, but if they're
going to both contain this file, then it makes sense
to have this in git.sh once again.
Signed-off-by: Leah Rowe <info@minifree.org>
blobs.list is now nuke.list
this is because i'm going to import this functionality
into lbmk (libreboot build system).
libreboot will not do full deblobbing like canoeboot does,
but there are still certain files that i like to delete
in releases, such as u-boot's strlcat.c file under tests
calling it "nukeblobs" in libreboot makes no sense, but
i like to avoid merge conflicts when cherry-picking
patches between cbmk and lbmk, so i like to make sure
that functions and variables common to both are named
the name.
simply calling it "nuke" or calling the files "nuke.list"
is probably inoffensive while conveying the same meaning.
Signed-off-by: Leah Rowe <info@minifree.org>
the inject script is from lbmk and referenced here, but
cbmk does not have this script.
remove the reference.
Signed-off-by: Leah Rowe <info@minifree.org>
Leah Rowe