Commit Graph

401 Commits (668bcbf69cbc250a49134056c1bf870af62e36e1)

Author SHA1 Message Date
Leah Rowe 668bcbf69c trees: simplified copy_elf() handling
don't create elfdir, create dest_dir, which is elfdir
plus the location within it

only create dest_dir within copy_elf, which is only
called if actually compiling the code

this avoids creating empty elf directories, and it
generally cleans up all handling, unifying the
handling of directories into a single function,
namely copy_elf() which already exists

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 15:04:24 +01:00
Leah Rowe d43247683b git.sh: download "depend" projects *before*
don't do it after, because that means the main project
is saved under src/ before we know whether the subrepo
was downloaded.

the "depend" variable (in config/git/) is no longer used
for projects that go in subdirectories of a parent; now,
we use config/submodules/ for this type of dependency.

download the "depend" projects (as per config/git/) first.
this way, if they fail, the main one will fail, but if
they succeed and main fails, you can just run the main
download again and it won't fail.

this fixes a bug where, depending on how you download a
set of projects and depending on the order which you do so,
a given project can become un-downloadable on current design,
because git will complain that a directory already exists.

this fix is done not only in code (by this commit), but
by prior configuration changes.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 14:16:58 +01:00
Leah Rowe 7bfb1d621f trees: don't check empty path in copy_elf()
it could lead to some whackiness later on

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 12:04:28 +01:00
Leah Rowe 0b7566cb78 trees: fix build issue caused by bad elf check
we're not checking for bad elfs, but the check itself was bad

due to a quirk in how sh works. really, really obscure bug.

fixed now!

if the given directory didn't actually exist, or there were no
files in it, it'd be searching for the file named "*"

which is obviously wrong

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 11:29:36 +01:00
Leah Rowe 7aa9f22409 trees: fix listfile check in copy_elf()
don't check that the variable is empty

check that the file itself exists or not

this should fix the recent build issues

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 10:35:38 +01:00
Leah Rowe 06c78e1321 trees: don't say check elf/ if build.list missing
(on single tree projects. this complements the last patch)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 10:32:39 +01:00
Leah Rowe dea41f138a trees: don't do elfcheck if build.list missing
otherwise, some checks are done too soon, and nothing
gets built.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 10:31:01 +01:00
Leah Rowe d0f99c2f98 trees: unified coreboot makeargs
in particular, the coreboot build system may auto-download
submodules when building cbfstool; vboot for instance.

we do not want such unpredictable behaviour, so now we
use UPDATED_SUBMODULES=1 when building coreboot utilities.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 22:21:59 +01:00
Leah Rowe a7889c5af8 trees: use multiple threads to build cbutils
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 22:20:09 +01:00
Leah Rowe d41658f17e move handle_coreboot_utils to script/trees
it doesn't really make sense placed in lib.sh,
because it's only called from script/trees

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 22:17:16 +01:00
Leah Rowe c0822ac458 put coreboot utils in elf/, not cbutils/
one directory per util, under elf/

e.g. elf/cbfstool/

further split by tree name, e.g.:

elf/cbfstool/default/
elf/cbfstool/foo/

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 21:51:37 +01:00
Leah Rowe 7e49fe4b9b trees: skip single-tree build if a build exists
this replicates the same behaviour as multi-tree builds,
checking for files inside the relevant elf/ directory

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 09:20:45 +01:00
Leah Rowe 1277427478 use correct memtest86plus path in script/roms
the previous change makes memtest.bin get cached in elf/
but the path was being prefixed with src/ by script/roms

do away with the prefix

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 09:11:39 +01:00
Leah Rowe 8511615e1f put memtest86plus builds in elf/memtest86plus/
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 09:05:16 +01:00
Leah Rowe 48cbb30db1 trees: also print "DONE! check elf/dir" on single
e.g. ./update trees -b flashprog

tell the user to check elf/flashprog

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 08:53:29 +01:00
Leah Rowe 315fed5f48 trees: handle build-test on multi-tree projects
for example, now flashprog binaries could be placed
in the elf/ directory, under elf/flashprog/

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 03:13:52 +01:00
Leah Rowe 810ad480da move cfgsdir/datadir variables to lib.sh
it's also used from script/roms, in addition to trees

move these variables to a common file used everywhere

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 02:43:27 +01:00
Leah Rowe ba36f26d62 handle build.list from config/data/, not config/
certain code checks for build.list, to skip it, for
example in items()

we already use config/data/grub to store grub config data
that applied to all trees

create these directories too:

config/data/coreboot
config/data/u-boot
config/data/seabios

move the respective build.list files in here, and also
to config/data/grub

now multi-tree projects contain, per directory, just the
target.cfg file and the patches directory. this is much
cleaner, because some of the logic can be simplified more

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 02:35:36 +01:00
Leah Rowe bea089bbe4 don't use build.list to detect multi-tree projects
instead, check for the presence of target.cfg files
not in config/project/ but config/project/tree/

the way this check is done, it merely returns 1 if
config/project/*/target.cfg is detected, and returns
0 in all other cases, even if config/project/target.cfg
exists

that way, if the maintainer accidentally adds a
target.cfg in the main directory, the given multi-tree
project will not break

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06 01:01:22 +01:00
Leah Rowe 00653aab1e better help text on invalid commands
adding help again is a bad idea. code should never
document itself; that's what documentation is for.

so, make the code do a better job telling the user
where to find documentation.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-03 00:17:36 +01:00
Leah Rowe 429e91f908 make GRUB multi-tree and re-add xhci patches
Re-add xHCI only on haswell and broadwell machines, where
they are needed. Otherwise, keep the same GRUB code.

The xHCI patches were removed because they caused issues
on Sandybridge-based Dell Latitude laptops. See:
https://codeberg.org/libreboot/lbmk/issues/216

The issue was not reported elsewhere, including on the
Haswell/Broadwell hardware where they are needed, but the
build system could only build one version of GRUB.

The older machines do not need xHCI patches, because they
either do not have xHCI patches, or work (in GRUB) because
they're in EHCI mode when running the payload.

So, the problem is that we need the xHCI patches for GRUB
on Haswell/Broadwell hardware, but the patches break
Sandybridge hardware, and we only had the one build of GRUB.
To mitigate this problem, the build system now supports
building multiple revisions of GRUB, with different patches,
and each given coreboot target can say which GRUB tree to use
by setting this in target.cfg:

grubtree="xhci"

In the above example, the "xhci" tree would be used. Some
generic GRUB config has been moved to config/data/grub/
and config/grub/ now looks like config/coreboot/ - also,
the grub.cfg file (named "payload" in each tree) is copied
to the GRUB source tree as ".config", then added to GRUB's
memdisk in the same way, as grub.cfg.

Several other design changes had to be made because of this:

* grub.cfg in memdisk no longer automatically jumps to one
  in CBFS, but now shows a menuentry for it if available

* Certain commands in script/trees are disabled for GRUB,
  such as *config make commands.

* gnulib is now defined in config/submodule/grub/, instead
  of config/git/grub - and this mitigates an existing bug
  where downloading gnulib first would make grub no longer
  possible to download in lbmk.

The coreboot option CONFIG_FINALIZE_USB_ROUTE_XHCI has been
re-enabled on: Dell OptiPlex 9020 MT, Dell OptiPlex 9020 SFF,
Lenovo ThinkPad T440p and Lenovo ThinkPad W541 - now USB should
work again in GRUB.

The GRUB payload has been re-enabled on HP EliteBook 820 G2.

This change will enable per-board GRUB optimisation in the
future. For example, we hardcode what partitions and LVMs
GRUB scans because * is slow on ICH7-based machines, due
to GRUB's design. On other machines, * is reasonably fast,
for automatically enumerating the list of devices for boot.

Use of * (and other wildcards) could enable our GRUB payload
to automatically boot more distros, with minimal fuss. This
can be done at a later date, in subsequent revisions.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-02 19:58:50 +01:00
Leah Rowe a4bd49ded6 roms: allow user override of grub_scan_disk
this effectively lets you change the boot order. example:

./build roms -s "nvme ata" t1650_12mb

the above example would set:
grub_scan_disk="nvme ata"

another example:

./build roms -s nvme t1650_12mb

this would set:
grub_scan_disk="nvme"

this overrides what's set in target.cfg for the given
target. useful for quick reconfiguration if building
from source

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-28 23:47:13 +01:00
Leah Rowe 4488745cb6 trees: use CPUS=x on regular coreboot make
i already do this on crossgcc, but overlooked it on regular
builds where i just use -j, but coreboot's build system
makes use of the CPUS= option in make

use XBMK_THREADS for this

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-28 19:57:35 +01:00
Leah Rowe b78f62c78b roms: fix bad eval when comparing options
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-28 01:23:43 +01:00
Leah Rowe e1883f1d5a grub.cfg: use grub_scan_disk to set boot order
Previously, grub_scan_disk could set ata, ahci or "both",
which would make both be tried (ahci first). This worked
when we only dealt with ata and ahci devices, but now we
support nvme devices so the logic is inherently flawed.

Instead, use grub_scan_disk to store the boot order, e.g.:

grub_scan_disk="ahci nvme ata"

grub_scan_disk="nvme ata"

In the first example, it would make GRUB scan ahci first,
then nvme and then ata.

In the secontd example, it would make GRUB scan nvme first,
and then ata.

If "both" is set, or anything other than ahci/ata/nvme,
grub_scan_disk is now changed to "nvme ahci ata".

Actual grub_scan_disk entries in target.cfg files will now
be modified, to match each machine.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-27 20:36:58 +01:00
Leah Rowe ff2997d6ec minor correction
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-27 15:05:33 +01:00
Leah Rowe d855408a58 roms: make grubfirst if seabios_withgrub=y
rather than if seabios_grubonly=y

if grubonly=y, still make the grubonly rom

this complements the previous commit

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-27 14:42:16 +01:00
Leah Rowe 914852dd40 rename include/option.sh to include/lib.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-26 12:10:27 +01:00
Leah Rowe dc7b72f3f4 roms: rename bstr variable
i forgot to do this in the recent cleanup. it is now
initialised as a variable named "x"

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-26 09:22:39 +01:00
Leah Rowe 5c14e8e1bc general code cleanup in the build system
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-26 07:22:44 +01:00
Leah Rowe d32968c731 trees: reset makeargs per target/project
it wasn't being reset before. when coreboot is being
built, i add to makeargs every time. if multiple targets
are being built, the make command would end up looking
something like:

make -C src/coreboot/default UPDATED_SUBMODULES=1 \
    UPDATED_SUBMODULES=1

(the parameter would be printed twice)

of course, this doesn't check whether that parameter is
added already in target.cfg for a given target, but that's
ok because i won't add that one in target.cfg

i baked it into the code, only when handling coreboot,
because that was easier than either putting it in makeargs
for every coreboot target.cfg, or again modifying the code to
handle that; the current solution is the cleanest.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25 14:32:06 +01:00
Leah Rowe 7bab0cf9ed trees: also use UPDATED_SUBMODULES=1 on crossgcc
i overlooked this in the previous revision

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25 00:37:26 +01:00
Leah Rowe 0a50eaf227 trees: add UPDATED_SUBMODULES to coreboot make
we do not want submodules to be downloaded after the fact.

we only handle this on ./update trees -f coreboot

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25 00:22:34 +01:00
Leah Rowe ff0840bdf8 trees: write -C on the make command first not last
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25 00:18:40 +01:00
Leah Rowe 87c361f3df update/trees: remove unused variable
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-23 00:01:20 +01:00
Leah Rowe e5a5935d8d fix building coreboot images on i686 hosts
firstly, memtest86+ is currently not cross compiled and
relies on 64-bit headers (x86_64 only). a 32-bit distro
is unlikely to be able to build 64-bit binaries.

secondly: vboot throws a build error due to -Werror when
building on 32-bit hosts. we rely on vboot code to build
cbfstool, so turn off -Werror on vboot

that's all. 32-bit hosts are not recommended; it is assumed
that you are building on an x86_64 host. work will go into
the build system at a later date to make it more portable,
by cross compiling everything, but this should fix 32-bit
for now.

there are some x60/t60 users who still want to build roms,
so let's allow them that possibility.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-21 23:40:25 +01:00
Leah Rowe 245b4eb21d build/roms: skip target if config/ dir missing
fixes a regression when running ./build roms all

now it should work again

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19 22:43:52 +01:00
Leah Rowe 1fe9c4b899 option.sh: mktar_release to mkrom_tarball
that's all it's used for, to compress the rom images

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19 06:45:38 +01:00
Leah Rowe cc7ed6926b build/roms: rename moverom to copyrom
it copies, it doesn't move, so name it right

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19 06:43:37 +01:00
Leah Rowe b40118ae59 minor code cleanup in the build system
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19 06:30:32 +01:00
Leah Rowe 998f30ad4d build/roms: simplify serprog list command
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18 22:17:12 +01:00
Leah Rowe 21a7efaa95 build/roms: simplified config payload checks
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18 22:13:20 +01:00
Leah Rowe 839ef680cd lbmk: allow easier sync with cbmk
an equivalent change has been made in cbmk.

certain lbmk-specific variable names have been made
generic, with certain functions and other variables
moved around.

i maintain sync between libreboot and canoeboot, where
both projects can have the same behaviours, and most of
the merge conflicts have to do with variable names
containing "LBMK", "lbmk", "cbmk" or "CBMK", or
indeed "canoeboot" and "libreboot"

LBMK/lbmk/CBMK/cbmk variables between canoeboot and
libreboot now contain the string XBMK/xbmk

it should now be *much* easier to merge build system
changes between lbmk and cbmk.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-16 03:56:52 +01:00
Leah Rowe 885fcebd81 remove help commands (user should read docs)
i always say, code should never document itself.

that's what documentation is for. the releases
contain documentation under docs/ but the git
repository does not; for that, use the website.

(in practise, lbmk usually needs internet anyway)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 04:30:42 +01:00
Leah Rowe e281966f22 remove check_project() (always set variables)
in lbmk, we call check_project() to set variables
such as projectname, version, version date

this is unnecessary, because all main scripts use
this functionality anyway

do it by default

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 03:04:41 +01:00
Leah Rowe a40a61292d build/roms: clean up tarball handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 02:36:24 +01:00
Leah Rowe 189b70ddcb build/roms: create full release tarball name
set relname from option.sh under check_project()

now the release logic simply has to move a directory

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 01:27:45 +01:00
Leah Rowe 08c9f94a1a unified sha512sum creation for tarballs
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 00:37:12 +01:00
Leah Rowe 1ce7e33976 move rom tarball creation to script/roms
export LBMK_RELEASE="y"

if this is done, the tarball is created instead
of a directory, and the rom images are nuked using
./vendor inject with the nuke option, inserting the
correct version files; the rom directory is deleted

now the release script logic simple renames existing
tarballs. the benefit of this change is fewer lines of
code, and now lbmk doesn't use an insane amount of disk
space when building a *lot* of release images (the
uncompressed directories are deleted after each build)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-14 23:44:28 +01:00
Leah Rowe 05fbd39298 remove all status checks. only handle release.
the release variable is all we need, turning a target on
or off for a given release.

the status checks were prone to bugs, and unnecessary; it
also broke certain benchmark scripts.

it's better to keep the lbmk logic simpler. board status
will be moved to the documentation instead.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11 18:53:12 +01:00