i removed this before, when making grub multi-tree,
because the design i used in an earlier version of
the patch actually added the grub.elf generation
to grub source itself, but then i decided to hack
around the grub build system from lbmk/cbmk instead
re-add this functionality, so that users can easily
insert their own custom grub.cfg into cbfs without
needing to re-build their image.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i was originally looser about this, because i also wanted
the trees script to generically run "make" from any
directory, but this behaviour was error-prone and it is
no longer used in the build system.
disable it, in the interest of stability.
Signed-off-by: Leah Rowe <leah@libreboot.org>
support redundant downloads, and enable inclusion of these
tarballs inside release archives, for offline builds.
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't create elfdir, create dest_dir, which is elfdir
plus the location within it
only create dest_dir within copy_elf, which is only
called if actually compiling the code
this avoids creating empty elf directories, and it
generally cleans up all handling, unifying the
handling of directories into a single function,
namely copy_elf() which already exists
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't do it after, because that means the main project
is saved under src/ before we know whether the subrepo
was downloaded.
the "depend" variable (in config/git/) is no longer used
for projects that go in subdirectories of a parent; now,
we use config/submodules/ for this type of dependency.
download the "depend" projects (as per config/git/) first.
this way, if they fail, the main one will fail, but if
they succeed and main fails, you can just run the main
download again and it won't fail.
this fixes a bug where, depending on how you download a
set of projects and depending on the order which you do so,
a given project can become un-downloadable on current design,
because git will complain that a directory already exists.
this fix is done not only in code (by this commit), but
by prior configuration changes.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we're not checking for bad elfs, but the check itself was bad
due to a quirk in how sh works. really, really obscure bug.
fixed now!
if the given directory didn't actually exist, or there were no
files in it, it'd be searching for the file named "*"
which is obviously wrong
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't check that the variable is empty
check that the file itself exists or not
this should fix the recent build issues
Signed-off-by: Leah Rowe <leah@libreboot.org>
in particular, the coreboot build system may auto-download
submodules when building cbfstool; vboot for instance.
we do not want such unpredictable behaviour, so now we
use UPDATED_SUBMODULES=1 when building coreboot utilities.
Signed-off-by: Leah Rowe <leah@libreboot.org>
one directory per util, under elf/
e.g. elf/cbfstool/
further split by tree name, e.g.:
elf/cbfstool/default/
elf/cbfstool/foo/
Signed-off-by: Leah Rowe <leah@libreboot.org>
this replicates the same behaviour as multi-tree builds,
checking for files inside the relevant elf/ directory
Signed-off-by: Leah Rowe <leah@libreboot.org>
the previous change makes memtest.bin get cached in elf/
but the path was being prefixed with src/ by script/roms
do away with the prefix
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's also used from script/roms, in addition to trees
move these variables to a common file used everywhere
Signed-off-by: Leah Rowe <leah@libreboot.org>
certain code checks for build.list, to skip it, for
example in items()
we already use config/data/grub to store grub config data
that applied to all trees
create these directories too:
config/data/coreboot
config/data/u-boot
config/data/seabios
move the respective build.list files in here, and also
to config/data/grub
now multi-tree projects contain, per directory, just the
target.cfg file and the patches directory. this is much
cleaner, because some of the logic can be simplified more
Signed-off-by: Leah Rowe <leah@libreboot.org>
instead, check for the presence of target.cfg files
not in config/project/ but config/project/tree/
the way this check is done, it merely returns 1 if
config/project/*/target.cfg is detected, and returns
0 in all other cases, even if config/project/target.cfg
exists
that way, if the maintainer accidentally adds a
target.cfg in the main directory, the given multi-tree
project will not break
Signed-off-by: Leah Rowe <leah@libreboot.org>
adding help again is a bad idea. code should never
document itself; that's what documentation is for.
so, make the code do a better job telling the user
where to find documentation.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Re-add xHCI only on haswell and broadwell machines, where
they are needed. Otherwise, keep the same GRUB code.
The xHCI patches were removed because they caused issues
on Sandybridge-based Dell Latitude laptops. See:
https://codeberg.org/libreboot/lbmk/issues/216
The issue was not reported elsewhere, including on the
Haswell/Broadwell hardware where they are needed, but the
build system could only build one version of GRUB.
The older machines do not need xHCI patches, because they
either do not have xHCI patches, or work (in GRUB) because
they're in EHCI mode when running the payload.
So, the problem is that we need the xHCI patches for GRUB
on Haswell/Broadwell hardware, but the patches break
Sandybridge hardware, and we only had the one build of GRUB.
To mitigate this problem, the build system now supports
building multiple revisions of GRUB, with different patches,
and each given coreboot target can say which GRUB tree to use
by setting this in target.cfg:
grubtree="xhci"
In the above example, the "xhci" tree would be used. Some
generic GRUB config has been moved to config/data/grub/
and config/grub/ now looks like config/coreboot/ - also,
the grub.cfg file (named "payload" in each tree) is copied
to the GRUB source tree as ".config", then added to GRUB's
memdisk in the same way, as grub.cfg.
Several other design changes had to be made because of this:
* grub.cfg in memdisk no longer automatically jumps to one
in CBFS, but now shows a menuentry for it if available
* Certain commands in script/trees are disabled for GRUB,
such as *config make commands.
* gnulib is now defined in config/submodule/grub/, instead
of config/git/grub - and this mitigates an existing bug
where downloading gnulib first would make grub no longer
possible to download in lbmk.
The coreboot option CONFIG_FINALIZE_USB_ROUTE_XHCI has been
re-enabled on: Dell OptiPlex 9020 MT, Dell OptiPlex 9020 SFF,
Lenovo ThinkPad T440p and Lenovo ThinkPad W541 - now USB should
work again in GRUB.
The GRUB payload has been re-enabled on HP EliteBook 820 G2.
This change will enable per-board GRUB optimisation in the
future. For example, we hardcode what partitions and LVMs
GRUB scans because * is slow on ICH7-based machines, due
to GRUB's design. On other machines, * is reasonably fast,
for automatically enumerating the list of devices for boot.
Use of * (and other wildcards) could enable our GRUB payload
to automatically boot more distros, with minimal fuss. This
can be done at a later date, in subsequent revisions.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this effectively lets you change the boot order. example:
./build roms -s "nvme ata" t1650_12mb
the above example would set:
grub_scan_disk="nvme ata"
another example:
./build roms -s nvme t1650_12mb
this would set:
grub_scan_disk="nvme"
this overrides what's set in target.cfg for the given
target. useful for quick reconfiguration if building
from source
Signed-off-by: Leah Rowe <leah@libreboot.org>
i already do this on crossgcc, but overlooked it on regular
builds where i just use -j, but coreboot's build system
makes use of the CPUS= option in make
use XBMK_THREADS for this
Signed-off-by: Leah Rowe <leah@libreboot.org>
Previously, grub_scan_disk could set ata, ahci or "both",
which would make both be tried (ahci first). This worked
when we only dealt with ata and ahci devices, but now we
support nvme devices so the logic is inherently flawed.
Instead, use grub_scan_disk to store the boot order, e.g.:
grub_scan_disk="ahci nvme ata"
grub_scan_disk="nvme ata"
In the first example, it would make GRUB scan ahci first,
then nvme and then ata.
In the secontd example, it would make GRUB scan nvme first,
and then ata.
If "both" is set, or anything other than ahci/ata/nvme,
grub_scan_disk is now changed to "nvme ahci ata".
Actual grub_scan_disk entries in target.cfg files will now
be modified, to match each machine.
Signed-off-by: Leah Rowe <leah@libreboot.org>
rather than if seabios_grubonly=y
if grubonly=y, still make the grubonly rom
this complements the previous commit
Signed-off-by: Leah Rowe <leah@libreboot.org>
it wasn't being reset before. when coreboot is being
built, i add to makeargs every time. if multiple targets
are being built, the make command would end up looking
something like:
make -C src/coreboot/default UPDATED_SUBMODULES=1 \
UPDATED_SUBMODULES=1
(the parameter would be printed twice)
of course, this doesn't check whether that parameter is
added already in target.cfg for a given target, but that's
ok because i won't add that one in target.cfg
i baked it into the code, only when handling coreboot,
because that was easier than either putting it in makeargs
for every coreboot target.cfg, or again modifying the code to
handle that; the current solution is the cleanest.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we do not want submodules to be downloaded after the fact.
we only handle this on ./update trees -f coreboot
Signed-off-by: Leah Rowe <leah@libreboot.org>
firstly, memtest86+ is currently not cross compiled and
relies on 64-bit headers (x86_64 only). a 32-bit distro
is unlikely to be able to build 64-bit binaries.
secondly: vboot throws a build error due to -Werror when
building on 32-bit hosts. we rely on vboot code to build
cbfstool, so turn off -Werror on vboot
that's all. 32-bit hosts are not recommended; it is assumed
that you are building on an x86_64 host. work will go into
the build system at a later date to make it more portable,
by cross compiling everything, but this should fix 32-bit
for now.
there are some x60/t60 users who still want to build roms,
so let's allow them that possibility.
Signed-off-by: Leah Rowe <leah@libreboot.org>
an equivalent change has been made in cbmk.
certain lbmk-specific variable names have been made
generic, with certain functions and other variables
moved around.
i maintain sync between libreboot and canoeboot, where
both projects can have the same behaviours, and most of
the merge conflicts have to do with variable names
containing "LBMK", "lbmk", "cbmk" or "CBMK", or
indeed "canoeboot" and "libreboot"
LBMK/lbmk/CBMK/cbmk variables between canoeboot and
libreboot now contain the string XBMK/xbmk
it should now be *much* easier to merge build system
changes between lbmk and cbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i always say, code should never document itself.
that's what documentation is for. the releases
contain documentation under docs/ but the git
repository does not; for that, use the website.
(in practise, lbmk usually needs internet anyway)
Signed-off-by: Leah Rowe <leah@libreboot.org>
in lbmk, we call check_project() to set variables
such as projectname, version, version date
this is unnecessary, because all main scripts use
this functionality anyway
do it by default
Signed-off-by: Leah Rowe <leah@libreboot.org>
export LBMK_RELEASE="y"
if this is done, the tarball is created instead
of a directory, and the rom images are nuked using
./vendor inject with the nuke option, inserting the
correct version files; the rom directory is deleted
now the release script logic simple renames existing
tarballs. the benefit of this change is fewer lines of
code, and now lbmk doesn't use an insane amount of disk
space when building a *lot* of release images (the
uncompressed directories are deleted after each build)
Signed-off-by: Leah Rowe <leah@libreboot.org>
the release variable is all we need, turning a target on
or off for a given release.
the status checks were prone to bugs, and unnecessary; it
also broke certain benchmark scripts.
it's better to keep the lbmk logic simpler. board status
will be moved to the documentation instead.
Signed-off-by: Leah Rowe <leah@libreboot.org>
there are only two scripts under script/ now, and there
probably won't be many more. lbmk's design has simplified
to such a degree that the two-level directory structure is
no longer necessary.
the existing command structure has not changed. for example:
./build roms list
./update trees -f coreboot default
these will still work, but the symlinks to "build" are now
strictly for backwards compatibility; they may be removed
at a later date, but i'll keep the current design for now.
this also leads to a quirk, for example:
./build roms all
./update roms all
these now do the exact same thing, whereas "./update roms all"
would have previously been an invalid command.
Signed-off-by: Leah Rowe <leah@libreboot.org>
stub it from the main build script
the commands remain identical:
./vendor download arguments_here
./vendor inject arguments_here
Signed-off-by: Leah Rowe <leah@libreboot.org>
the main script isn't that big, and since the main
purpose of lbmk is geared toward the releases, it
makes sense to reduce the number of scripts by
merging into the main one
the way this works, "./update release" still works
afterward
so, the way lbmk is used shall remain unchanged
Signed-off-by: Leah Rowe <leah@libreboot.org>
previous command:
./build serprog
now it is:
./build roms serprog
after that, it's the same arguments e.g.
./build roms serprog stm32
./build roms serprog rp2040
further cleanup to commence
Signed-off-by: Leah Rowe <leah@libreboot.org>
x is part of the for loop in main() and may or not
still be available from handle_target, depending on
your implementation of sh, but this should not be assumed
do it properly
Signed-off-by: Leah Rowe <leah@libreboot.org>
for example:
./build roms list stable
this lists all images that are marked "stable"
now:
./build roms list _stable
this lists all images that are *not* marked stable
this will help me keep track during development
Signed-off-by: Leah Rowe <leah@libreboot.org>
This is useful on desktops, where you want GRUB to
automatically start, but you still want access to the
GRUB menu, in the case where you rely on SeaBIOS to
execute the VGA ROM inside your graphics card.
Signed-off-by: Leah Rowe <leah@libreboot.org>
just to ensure that nothing goes wrong. we don't rely on
the status variable for releases, because there is another
variable, release, that target.cfg files declare, e.g.
release="n"
release="y"
you can just omit the variable, because it defaults to y, so
you only need declare it when it needs to be "n"
Signed-off-by: Leah Rowe <leah@libreboot.org>
export LBMK_STATUS=n
if not set, the status checks and confirmation dialogs
persist. if set to y they persist.
if you set it to n, all checks are disabled, so e.g.:
./build roms all
this would once again build all targets, regardless
of status. this is if you want the old behaviour.
Signed-off-by: Leah Rowe <leah@libreboot.org>
for example:
./build roms list
this will list every now, still. same behaviour. now see:
./build roms list stable
this will list all stable roms
./build roms list untested
this lists untested roms. but wait!
./build roms list untested broken unstable
./build roms list broken unstable
yes. it works this way. now you can use lbmk to easily
see what rom status are, during maintenance.
Signed-off-by: Leah Rowe <leah@libreboot.org>
export LBMK_VERSION_TYPE=x
x can be: stable, unstable
in target.cfg files, specify:
status=x
x can be: stable, unstable, broken, untested
if unset, lbmk defaults to "unknown"
if LBMK_VERSION_TYPE is set, no confirmation is asked
if the given target matches what's set (but what's set
in that environmental variable can only be stable or
unstable)
if LBMK_RELEASE="y", no confirmation is asked, unless
the target is something other than stable/unstable
"unstable" means it works, but has a few non-breaking
bugs, e.g. broken s3 on dell e6400
whereas, if raminit regularly fails or it is so absolutely
unreliable as to be unusable, then the board should be
declared "broken"
untested means: it has not been tested
With this change, it should now be easier to track whether
a given board is tested, in preparation for releases. When
working on trees/boards, status can be set for targets.
Also: in the board directory, you can add a "warn.txt" file
which will display a message. For example, if a board has a
particular quirk to watch out for, write that there. The message
will be printed during the build process, to stdout.
If status is anything *other* than stable, or it is unstable
but LBMK_VERSION_TYPE is not set to "unstable", and not building
a release, a confirmation is passed.
If the board is not specified as stable or unstable, during
a release build, the build is skipped and the ROM is not
provided in that release; this is in *addition* to
release="n" or release="y" that can be set in target.cfg,
which will skip the release build for that target if "n"
Signed-off-by: Leah Rowe <leah@libreboot.org>
the temporary rom per build was not being deleted after
finishing the current target. this adds up in /tmp during
large builds, when building for many targets. fix this!
Signed-off-by: Leah Rowe <leah@libreboot.org>
release="n" is set in target.cfg on haswell build targets
that use mrc.bin
script/update/release exports LBMK_RELEASE="y"
script/build/roms skips building a given target if release="n"
in target.cfg *and* LBMK_RELEASE="y"
you could also do the export yourself before running ./build roms,
for example:
export LBMK_RELEASE="y"
./build roms all
This would skip these ROM images. The native haswell raminit is
now stable enough in my testing, that I wish to delete the MRC-based
targets. This is in line with Libreboot's Binary Blob Reduction Policy,
which states: if a blob can be avoided, it should be avoided.
The problem is that users often run the inject script in *lbmk* from
Git, instead of from the src release archive. I forsee some users
running this on modern lbmk with older release images. If the mrc-based
target isn't there, the user may use an NRI-based target name, and
think it works; they will insert without MRC. I foresaw this ages
ago, which is why Caleb and I ensured that the script checks hashes,
and hashes are included in releases.
Therefore: for the time being, keep the MRC-based configs in lbmk
but do not include images for them in releases. This can be done
indefinitely, but I'll probably remove those configs entirely at
some point.
On the following boards, Libreboot now will *only* provide NRI-based
ROM images for the following machines:
* Dell OptiPlex 9020 SFF
* Dell OptiPlex 9020 MT
* Lenovo ThinkPad T440p
* Lenovo ThinkPad W541/W540
I now recommend exclusive use of NRI-based images, on Haswell
hardware. It's stable enough in my testing, and now supports S3.
Signed-off-by: Leah Rowe <leah@libreboot.org>
lbmk otherwise uses nproc to set the number of build threads,
in these places:
* generic make commands in script/update/trees
* crossgcc make command in script/update/trees
the -T0 option is also used in script/update/release, when running
tar.
with this change, you can do:
export LBMK_THREADS=x
where x is the number of threads. when you then run
lbmk, your chosen number of threads will override
the default. this may be useful on a host that does
not have a lot of memory.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in shell scripts, a function named the same as a program included in
the $PATH will override that program. for example, you could make a
function called ls() and this would override the standand "ls".
in lbmk, a part of it was first trying to run the "fail" command,
deferring to "err", because some scripts call fail() which does
some minor cleanup before calling err.
in most cases, fail() is not defined, and it's possible that the user
could have a program called "fail" in their $PATH, the behaviour of
which we could not determine, and it could have disastrous effects.
lbmk error handling has been re-engineered in such a way that the
err function is defined in a variable, which defaults to err_ which
calls err_, so defined under include/err.sh.
in functions that require cleanup prior to error handling, a fail()
function is still defined, and err is overridden, thus:
err="fail"
this change has made xx_() obsolete, so now only x_ is used. the x_
function is a wrapper that can be used to run a command and exit with
non-zero status (from lbmk) if the command fails. the xx_ command
did the same thing, but called fail() which would have called err();
now everything is $err
example:
rm -f "$filename" || err "could not delete file"
this would now be:
rm -f "$filename" || $err "could not delete file"
overriding of err= must be done *after* including err.sh. for
example:
err="fail"
. "include/err.sh"
^ this is wrong. instead, one must do:
. "include/err.sh"
err="fail"
this is because err is set as a global variable under err.sh
the new error handling is much cleaner, and safer. it also reduces
the chance of mistakes such as: calling err when you meant to
call fail. this is because the standard way is now to call $err,
so you set err="fail" at the top of the script and all is well.
Signed-off-by: Leah Rowe <leah@libreboot.org>
./update release -m u-boot
if someone just wants to make u-boot, they can
use this and it tars up all the trees.
Signed-off-by: Leah Rowe <info@minifree.org>
in some cases, the build system was needlessly, and sometimes
erroneously, creating crossgcc symlinks, which then caused an
issue, namely:
in lbmk release builds, dell e6400 is build before fam15h boards,
and it sets xtree, but fam15h_rdimm doesn't, and later this would
cause fam15h_rdimm boards to use xtree="default" (because they don't
set xtree), causing the newer toolchain to be used on coreboot 4.11.
this patch fixes the issue. quite a simple problem, actually.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the boarddir variable is only set *after* detect_board
is run, and is in fact checked after that. this check,
removed by this patch, is too early and causes lbmk
to exit with error states. this patch fixes the error.
the error was that lbmk was then searching for a file
that is at an empty path.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the changelog file is only present in releases, so
use the presence of this file for the test.
someone who wants to fetch projects within a release
archive can simply use the git repo, or delete the file.
Signed-off-by: Leah Rowe <leah@libreboot.org>
use the git log, as follows:
git log --graph --pretty=format:'%Cred%h%Creset %s %Creset' --abbrev-commit
this creates a nice, uniform list of changes.
Signed-off-by: Leah Rowe <leah@libreboot.org>
let them specific it, rather than falling back
to coreboot/default (can also be used for coreboot boards)
Signed-off-by: Leah Rowe <leah@libreboot.org>
There is no need to add multiple keymap files, because
GRUB can load keymaps from CBFS. The current build logic
is designed to avoid building multiple GRUB binaries,
which are expensive computationally because each one
would then have to be compressed for each board.
This patch provides the best of both worlds: less space
used in flash like in the old lbmk design (1 keymap per
board), but retaining the current build speeds and therefore
not re-introducing the slowness of lbmk's previous GRUB
build logic.
The grub.cfg file has been modified, accordingly. It now
only loads a keymap.gkb file from CBFS, by default. It does
this, only if that file exists; if not, GRUB already defaults
to US Qwerty layout anyway.
ALSO: compress all keymap gkb files with xz -6
GRUB automatically decompresses files when accessed.
This results in about 2KB of flash space saved in CBFS.
Here is real-world data, showing the increased flash space:
< fallback/payload 0x3eb80 simple elf 548821 none
< keymap.cfg 0xc4bc0 raw 16 none
< (empty) 0xc4c00 null 11633316 none
---
> fallback/payload 0x3eb80 simple elf 546787 none
> keymap.gkb 0xc43c0 raw 344 none
> (empty) 0xc4540 null 11635044 none
This was taken by diffing the cbfstool "print" output,
both before and after. The *after* result is with this change.
11633316. In this example, 1728 bytes have been saved. Therefore,
with compression taken into account, this patch saves about 1.7KB
of space in CBFS.
This change means that lbmk can now scale to support hundreds
of keymaps, without increasing the amount of flash space used,
in each given image. Since the keymap files are compressed in
lbmk.git, in advance, we spend no additional time on compression
at build time. The resulting change in build speed in negligible.
Adding your own keymap.gkb file was already possible, for changing
the keymap in libreboot images, if you didn't want to change the
memdisk (and thus re-compile grub.elf). Now, this is the default
behaviour, and the only way to do it. It's much more efficient.
The original keymap files can be restored, by running unxz.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the "kmapdir" variable was removed in an earlier audit,
but was overlooked for -k because that option was untested.
rather than initialise the variable, re-use grubcfgsdir.
this fix enables e.g. "-k usdvorak" to work again.
Signed-off-by: Leah Rowe <leah@libreboot.org>
with neutered ME, fan control fails. while there are
ways to mitigate it, many users will not, and will
likely see their system overheat, which is very
dangerous.
this bug (failed fan control on neutered ME) only
affects arrandale machines such as lenovo x201.
the newer machines are not affected by this.
other arrandale machines will probably not be added
to libreboot because of this, or they will be subject
to further testing.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This is of Broadwell platform, one generation above Haswell.
Of note: this uses HP Sure Start. Although the flash is 16MB,
our CBFS section (and IFD configuration) assumes 12MB flash,
so the final 4MB will be left unflashed on installation,
after blanking the private flash. The coreboot documents have
more information about this.
Some minor design changes in lbmk were made, to accomodate
this port:
Support for extracting refcode binaries added (pulled from
Google recovery images). The refcode file is an ELF that
initialises the MRC and the PCH. It is also responsible for
enabling or disabling the Intel GbE device, where Google
does not enable it, but lbmk modifies it per the instructions
on the coreboot documentation, so as to enable Intel GbE.
Google's recovery image stores the refcode as a stage file,
but coreboot changed the format (for CBFS files) after 4.13
so coreboot 4.13's cbfstool is used to extract refcode. This
realisation made me also change the script logic to use a
cbfstool and ifdtool version matching the coreboot tree, for
all parts of lbmk, whereas lbmk previously used only the
default tree for cbfstool/ifdtool, on insertion and deletion
of vendor files - it was 81dc20e744 that broke extraction of
refcode on google's recovery images, where google used an older
version of cbfstool to insert the files in their coreboot ROMs.
A further backported patch has been added, copying coreboot
revision f22f408956 which is a build fix from Nico Huber.
Iru Cai submitted an ACPI bugfix after the revision lbmk
currently uses, for coreboot/default, and this fix is
needed for rebooting to work on Linux 6.1 or higher. This
patch has been backported to lbmk, while it still uses the
same October 2023 revision of coreboot.
Broadwell MRC is inserted at the same offset as Haswell,
so I didn't need to tweak that.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the update/trees script checks this binary itself, before
deciding whether to recompile/compile, so we don't need
to do such checks here.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this is now used in grub, for the FS_PAYLOAD_MODULES
option in the make command
lbmk should generalise as much logic as possible. in
some parts of it, logic is hurrently hardcoded, specific
to a given project that lbmk uses, but lbmk is essentially
a source-based package manager, like what you might find
on a small linux distro, so we need to try to
be as generic as possible.
lbmk is the "build system of build systems", so it has to
work generically with as many of them as possible
Signed-off-by: Leah Rowe <leah@libreboot.org>
it is no longer hardcoded just to be handled for uefiextract.
it is now defined as cmakedir in target.cfg, for a single or
multi tree project. if multi tree, it is applied to the specific
tree, and has to be defined per tree
the way it works is: as per cmakelist, a project will define
which directory is to be built, and it will then generate
a makefile in the main source tree (the build tree in cmake
language, where the main CMakeLists.txt file exists)
when the makefile has been generated, the project is then treated
like any other project. the way cmake works, if a makefile has
already been generated by it, in a given directory, running it
again will fail and not affect anything; if it fails but the
makefile doesn't exist, then something is wrong, but if the
makefile does exist, then it's all fine and nothing happens
at present, this is only used for uefiextract, which is part
of src/uefitool
Signed-off-by: Leah Rowe <leah@libreboot.org>
the logic of the previous commit was correct, but one
of the functions was named the same as another function
used in this file, causing a namespace conflict, and
a build error
Signed-off-by: Leah Rowe <leah@libreboot.org>
at present, the bootstrap and configure script is only
directly executed for grub, because grub is the only
project that uses them in lbmk
however, when i start adding linuxboot support, i will
have to start building a lot of projects, some of which
make use autoconf and bootstrap scripts
e.g.
./bootstrap --foo
./configure --bar
the "bootstrap" script is often used on GNU programs,
because they like to over-engineer absolutely everything
Signed-off-by: Leah Rowe <leah@libreboot.org>
the script can now also handle autoconf build systems,
whereas this could previously only be done for grub.
with this change, the overall sloccount is also lower
Signed-off-by: Leah Rowe <leah@libreboot.org>
arch no longer needs to be set, on multi-tree projects,
and it has been renamed to xarch
the new behaviour is: if xarch is set, treat it as a
list of crossgcc targets and go through the list. set
the first one as the target, for what lbmk builds, but
build all of the defined crossgccc targets
crossgcc_ada is now xlang, and defines which languages
to build, rather than whether to build gcc-gnat
Signed-off-by: Leah Rowe <leah@libreboot.org>
while seemingly pedantic, this does actually make code
easier to read. mostly just switching to shorthand for
variable names, where no expansions or patterns are used
Signed-off-by: Leah Rowe <leah@libreboot.org>
the rom functions print a path to the rom they built,
which is then used, but these are called inside what
are essentially subshells, and we had no error handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
cros roms are always using libgfxinit, with a coreboot
framebuffer, so the "normal" initmode is never used.
Signed-off-by: Leah Rowe <leah@libreboot.org>
lbmk used to set version/versiondate directly in
err.sh, but now it's handled there by a function,
which is called by the main script.
script/update/release hadn't yet been adapted. the
only change necessary is to call check_project()
script/update/trees also makes use of it
script/build/roms is using "projectname"
Signed-off-by: Leah Rowe <leah@libreboot.org>
when make-all is being executed on a coreboot tree,
the "./vendor download target" command is used, where
target is the tree/board name.
that script then checks whether cbfstool and ifdtool
are built, and if they're not, they then call
./update trees -b coreboot utils bla bla bla
in this scenario, project=coreboot and mode="", meaning
make-all, and the same check that checks whether the
vendor download script should be run, is executed,
which in turn then checks cbutils again
fix the infinite loop by checking whether it was coreboot
utils, as opposed to *firmware*, that is to be built, before
running ./vendor download
Signed-off-by: Leah Rowe <leah@libreboot.org>
the x_ function doesn't handle arguments with spaces
well, and this cd command is going to an asterisk, so
it's unknown what the resultant string will be.
Signed-off-by: Leah Rowe <leah@libreboot.org>
at present, lbmk can remove microcode updates on images for
a given target, if the target specifies
microcode_required="n" in target.cfg
lbmk then provides images with microcode, and images without,
in a given release. although the user can also remove them
manually, this just makes it a bit more convenient, for those
users who do wish to run without the updates. this functionality
is provided only on those platforms where no-microcode is tested.
well, this behaviour implements a compromise on libreboot policy,
which is to always include microcode updates by default. see:
Binary Blob Reduction Policy
the *canoeboot* project now exists, developed in parallel with
libreboot, and it ships without microcode updates, on the same
targets where lbmk also handled this.
running without microcode updates is foolish, and should not
be encouraged. clean up lbmk by not providing this kludge.
the libreboot documentation will be updated, telling such users
to try canoeboot instead, or to remove the update from a given
libreboot rom - this is still possible, and mitigations such as
PECI disablement on GM45 are still in place (and will be kept),
so that this continues to work well.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the purpose of script/update/release is not to test the
build system, but to build release archives.
testing of lbmk is done during the course of development.
remove this bloat from the release script. we run the nuke
mode anyway, to scrub blobs from releases, which will more
or less test the logic in that script (the only difference
is that it runs e.g. ifdtool --nuke instead of -i).
Signed-off-by: Leah Rowe <leah@libreboot.org>
why are we distributing gcc at all?
the coreboot build system downloads it at build time,
and the GNU rsync mirrors aren't going anywhere.
simplify script/update/release by not handling gcc.
this means: release archives will no longer contain gcc.
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't handle "romtype" at all, in board target.cfg files
add /dev/null as pike2008 rom on amd boards. this serves
the same purpose, adding them as empty vga roms, to add
an empty rom in cbfs. pike2008 cards cause seabios to hang,
when their oproms are executed, so we insert a fake rom
on i945 thinkpads, use the coreboot config option:
CONFIG_INTEL_ADD_TOP_SWAP_BOOTBLOCK
when set, this enables the same bootblock copy, for use
with bucts. these two cases, namely pike2008 roms and
i945 bootblock copies, no longer need to be handled in code
Signed-off-by: Leah Rowe <leah@libreboot.org>
only call crossgcc for coreboot and u-boot, but use
hostcc for everything else. simplify the checking of
which architecture to compile for. "arch" in target.cfg
files has been modified, to allow further simplification.
without this patch, the logic currently only *barely* avoids
using crossgcc on things like utils, and only works in practise
because, in practise, lbmk only works on x86_64 anyway.
the new logic, as per this patch, is simpler and more robust.
Signed-off-by: Leah Rowe <leah@libreboot.org>
remove unnecessary "continue" command. it's written
at the end of a for loop, where it'll continue anyway
Signed-off-by: Leah Rowe <leah@libreboot.org>
all it did was set -v in the shell, which doesn't yield
very useful results. this is a relic of very old design
in the libreboot build system, that is no longer needed.
Signed-off-by: Leah Rowe <leah@libreboot.org>
lbmk didn't quote certain arguments in commands, or
used ! -z instead of -n, things like that. simple fixes.
Signed-off-by: Leah Rowe <leah@libreboot.org>
most of these are probably redundant, and will never
be called, but lbmk needs to be as safe as possible
under fault conditions. fail early, fail hard.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in cases where lbmk must always return from a function,
there are some cases where it relies on non-zero exit
status, which in practise is always the case, but may
change in the future if the relevant part is modified
e.g. do_something && return 0
the proper form is:
do_something
return 0
also do this for unconditional exits
Signed-off-by: Leah Rowe <leah@libreboot.org>
it didn't work in the past, but it does work nowadays;
specifically, it only worked with libgfxinit in the past,
but not on VGA ROMs.
now it does work on VGA ROMs, tested on e6400 and t1650 so
it was enabled there.
in this setup, a special image is provided where SeaBIOS is
the main payload, but it only loads GRUB; nothing else, every.
this is called SeaGRUB. this setup is useful in cases where
the user only has a GPU that lacks libgfxinit support.
Signed-off-by: Leah Rowe <leah@libreboot.org>
when printing the name of the rom being created, it's
done before the check to rename based on vendorfiles
in target.cfg. this patch fixes that bug.
Signed-off-by: Leah Rowe <leah@libreboot.org>
note: me6_update_parser needs to be written, similar
to me7_update_parser, to generate the partition
tables within intel me6 on lenovo bios updates.
the current logic in lbmk goes like this:
mkdir -p vendorfiles/cache/
and save your factory dump as:
vendorfiles/cache/x201_factory.rom
the build system has been modified, in such a way
as to support extracting me.bin (which is the full
one) and then neutering from this.
this is done automatically, if the file is present,
but you must first insert that file there, which means
you'll need a dump of the original boot flash on your
thinkpad x201
Signed-off-by: Leah Rowe <leah@libreboot.org>
it wasn't being copied right
the roms under elf/ were being copied, but not the ones
under bin/ - i need to audit it further
for now, i run modify_coreboot_roms from build/roms
instead of update/trees
so, the ones under elf/ no longer have bootblocks copied.
it's only done in bin/
Signed-off-by: Leah Rowe <leah@libreboot.org>
when building only for u-boot, the current script
works just fine. however, when building for other
payloads in additional to u-boot, the final u-boot
stage fails because other payloads are already
inserted via cbfs.
when we build u-boot, we do that last because we want
u-boot setups to only be u-boot, nothing else.
this patch enables qemu x86 to build properly with
u-boot.
Signed-off-by: Leah Rowe <leah@libreboot.org>
keymaps weren't being set in keymay.cfg of cbfs, due
to use of x_ in the rom script, and x_ doesn't handle
quotes or spaces in arguments well.
i'm going to remove use of x_ and xx_ (it's in my todo),
for next release.
for now, hot patch the release. i've gone through and
replaced use of x_ with || err, in some places.
not just the keymap.cfg command, but others too. in case
there are more issues we missed.
this commit is being tagged "20231021fix" and i'm using
this tag to re-build the 20231021 release. i'll just
replace the tarballs in rsync and add errata to the news
page announcing the release. all i did was break peoples
umlauts, i didn't brick their machines fortunately!
very minor bug. anyway, x_/xx_ is a great idea, but sh
isn't really designed for that style of programming. i'll
go back to using just || err in the next release.
Signed-off-by: Leah Rowe <leah@libreboot.org>
clean it up after copying the tarballs
i really hate how this logic is written, it's clunky
but it should work; the only issue is that it's quite
slow, and inefficient on use of disk space.
however, i've not yet figured out how to reproducible
add files to a tarball, once the tarball has been created,
and i rely on sorting (of file names) when creating them.
it's really not a problem because normal people won't
use this script, only i or anyone who wants to test out
the libreboot release infrastructure. this script is
largely intended to *work*
but i'm still annoyed by how crappy it is. i'll fix it
after the Libreboot 20231021 release.
Signed-off-by: Leah Rowe <leah@libreboot.org>
everything downloaded, then tarballed, then built,
now crossgcc is downloaded by coreboot.
now extract, copy crossgcc tarballs, re-compress.
TODO: simply add files to the archive, without re-
compressing the whole thing.
this is still more efficient than the old way: build
everything, then clean and compress, making another
build test on the release archive necessary; with this,
there is still only one build test per release.
with this, and the previous revisions dealing with
submodules, the source archives should now be complete.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in release archives, .git is excluded but the version
and versiondate files are included. from these, the
git history is re-created with the exact date (but not
taking into account timezone, at present).
in this way, lbmk will have git history in a release
archive. some build systems, like coreboot, prefer that
there be git history available, so this is a nice
workaround on those build systems.
Signed-off-by: Leah Rowe <leah@libreboot.org>
do it using find -exec
this is more robust, and it will never need to be
maintained over time (famous last words).
this is done because now we download submodules
for all git projects, so it's hard to predict.
Signed-off-by: Leah Rowe <leah@libreboot.org>
when [] is used right at the end of a function, or
certain loops/subshells, some sh implementations will
just return a non-zero exit
Signed-off-by: Leah Rowe <leah@libreboot.org>
config/git has been re-arranged in a prior revision,
ensuring that each file only refers to a main source
tree defined within those files.
the erstwhile "./build clean all" functionality is now
once again possible in lbmk
Signed-off-by: Leah Rowe <leah@libreboot.org>
in some cases, use of x_ or xx_ can be error-prone,
due to the way $@ is handled; commands requiring
quotes, or with funny file names as arguments such
as spaces in the file name, or other special
characters, can make the x/xx functions break.
in those cases, where x/xx must not be used, the
commands use || err instead
in other cases, use of x/xx is superfluous, and has
been removed in some commands.
Signed-off-by: Leah Rowe <leah@libreboot.org>
as opposed to the current 3-level structure.
recent build system simplifications have enabled
this change, thus:
./build fw coreboot -> ./build roms
./build fw grub -> ./build grub
./build fw serprog -> ./build serprog
./update project release -> ./update release
./update project trees -> ./update trees
./update vendor download -> ./vendor download
./update vendor inject -> ./vendor inject
alper criticised that the commands were too long,
so i made them shorter!
Signed-off-by: Leah Rowe <leah@libreboot.org>
i forgot to include option.sh in this script,
during previous re-factoring. the cbfstoos variable
is now defined exclusively in option.sh, but other
scripts can set it to something else.
Signed-off-by: Leah Rowe <leah@libreboot.org>
move it all to other files where items are used, and not
used anywhere else. this reduces the size of vendor.sh.
also remove a few redundant variables, or variables that
are not meaningfully used.
a few items have been moved to include/option.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
they are the functions only used by the download
script, so they don't belong in vendor.sh
an include file should only contain variables and
functions used by multiple main scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
We don't really need a custom coreboot tree for Chromebooks. I had added
one, because at a cursory glance to the available config/coreboot/board
subdirectories I had the impression that I should. But upstreams have
one tree for every board and I think we should move towards that too.
Move the one important BL31 makefile patch into the default coreboot
patches, update the gru boards' configs by running savedefconfig in the
cros tree and then running olddefconfig in the default tree.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Add an "-s" flag for "make savedefconfig", "-l" for "make olddefconfig"
and "-n" for "make nconfig" to the update script. The first two are
mainly useful for U-Boot, to compare our configs to the upstream
defconfigs and stay in sync with any upstream changes. The latter is
because the ncurses one has a nice "Symbol Search" that can point out
the menu entry for a config symbol we know.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
The "u-boot.bin" file generated by U-Boot builds is a raw binary. When
adding payloads to a CBFS, we need to use ELF files with add-payload
or manually pass the entry point and load address of the payload binary
with add-flat-binary.
We primarily use the "u-boot.elf" which gets build with the REMAKE_ELF
option, as it also has the necessary device-tree binary that U-Boot
usually needs to work. When the option is not set (e.g. for QEMU), we
need to use the "u-boot" file which is an ELF.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
i wasn't getting the very first line of tar --version,
so it wasn't doing the check properly.
further sort the files by name within the tar archive.
for reliability, don't bother using versiondate anymore:
set a *fixed* date, and fixed timezone, to ensure
that it works reliably for reproducible tarball creation.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This way, the handling of configs is unified into one
script, which reduces the possibility of bugs later,
and it reduces the repetition of code.
Signed-off-by: Leah Rowe <leah@libreboot.org>
use find and touch, to force all files, directories and
links to the desired timestamp (versiondate file)
Signed-off-by: Leah Rowe <leah@libreboot.org>
e.g. src/coreboot/coreboot must not appear in a release,
because we instead have directories like
src/coreboot/default or src/coreboot/cros
lbmk resets src/coreboot/coreboot to HEAD, but then resets
revisions properly in copies of it
therefore, for reproducibility, we must not include
src/coreboot/coreboot, src/u-boot/u-boot or
src/seabios/seabios into libreboot releases
Signed-off-by: Leah Rowe <leah@libreboot.org>
with --mtime, files added to the archive can be set
to a static date (in this case, the unix epoch)
the one used here is derived from git commit dates,
and it is static; if not being handled in lbmk.git,
the versiondate file never changes
this is the first patch in a series of patches designed
to bring about reproducible builds in libreboot
a solution will need to be found, for non-GNU tar
implementations, because they did not have an
equivalent option according to their manpages.
for example, BSD tar implementations.
perhaps i could systematically go around changing
file dates, on each file, as a fallback behaviour?
Signed-off-by: Leah Rowe <leah@libreboot.org>
this way, the src tarball is guaranteed to be clean.
the downside is that lbmk itself does not currently
handle crossgcc downloads, and there may be some
stragglers such as third party modules automatically
downloaded by certain codebases that libreboot uses.
this will have to be audited later (and it will be).
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's sometimes done unconditionally. this change
ensures that it is not repeated needlessly.
i observed otherwise that cbfstool would be
re-built from time to time, even if it was built.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Riku's mSATA patch for HP8300USDT was merged upstream, so the
patch has been dropped from lbmk because it is contained within
this new coreboot revision.
Signed-off-by: Leah Rowe <leah@libreboot.org>
coreboot closely matches upstream, whose current release
is version 1.2 from 2018, and coreboot has not changed it
in any meaningful way.
the upstream did add patches since, but they are documentation
patches only.
this means: we do not need to use the upstream version
Signed-off-by: Leah Rowe <leah@libreboot.org>
also rename elf/coreboot to something scary
some users were flashing roms built under elf/, which
lack payloads. lbmk builds no-payload roms (and payloads)
under elf/ then inserts them, creating full (flashable)
images under bin/
Signed-off-by: Leah Rowe <leah@libreboot.org>
some users reported build errors. technically, there's
nothing wrong with lbmk but it relies on hostcc, and
hostcc is hit or miss when it comes to cross compiling
32-bit, depending on the build system of whatever project.
lbmk needs to handle cross compilation. for now, i'm just
disabling memtest86plus on non-64-bit hosts.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The logic has been re-written, where source archives are
concerned. This clones the current repository, and starts
a new build from scratch. A custom release directory is
possible, by passing -d
This eliminates a step during build-testing, saving hours
of time, because it builds the release archive *inside* the
release archive, with git files removed, thus replicating
the same setup that the user would have.
This also makes everything a bit more consistent, because
it's guaranteed that a release archive will always have
the same files; previously, the release build script would
only copy what was already built, without building anything.
Now, this script builds everything itself.
The script also builds serprog images, not just coreboot.
Usage:
./update project release
If -d is not passed, release/ is used inside lbmk.
Otherwise, you could do:
./update project release -d /path/to/directory
If the directory exists, this script will exit (error).
Other minor fixes: build/fw/coreboot: make version in
coreboot-version (file) not contain hyphens, to work
around a quirk in coreboot's build system when not building
on regular libreboot releases. this quirk only appears
when lbmk is not being compiled under git.
The other main benefit of this change is that the new
script will probably require a lot less maintenance.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the script used to be called once per target, now it
handles every target. the grub background image wasn't
being set, so if it changed at build time, it would
stay changed.
keep the default in place for each run, while still
allowing target.cfg files to change it per target.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Leah Rowe