Commit Graph

297 Commits (042c7877e91032c46220609c8e8ab9df2f109ee3)

Author SHA1 Message Date
Leah Rowe c477599c78 build/boot/roms_helper: general code cleanup
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 22:27:39 +01:00
Leah Rowe 26fc3f1325 general code formatting cleanup in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 20:06:29 +01:00
Leah Rowe 0a0defd325 simplify initialising variables in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 19:09:25 +01:00
Leah Rowe 49b266eb42 build/boot/roms: only do 1 custom kbd/payload/mode
-k, -p and -d let you set keymap, payload and displaymode
respectively, but the handling for this is buggy when
passing multiple arguments.

Support only one argument, for simplicity. This is how
people use them anyway, and it makes lbmk less buggy.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 16:14:27 +01:00
Leah Rowe d268f5eb28 build/boot/roms: move usage() to include/boot.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 15:36:53 +01:00
Leah Rowe 7922b6e0e5 build/boot/*: unified main() function
The *same* main() function is now used on both scripts.

However, merging both scripts together would be less efficient
on sloccount, and would be error-prone. The purpose of having
roms_helper is that the variables get re-initialised the same
way each time, for each board, automatically.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 15:28:52 +01:00
Leah Rowe f3c4f208d0 build/boot/roms: split up handle_targets()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 14:13:11 +01:00
Leah Rowe 4afa0aaa3c build/boot/roms: check all targets before building
If one of them doesn't exist, error out.

Previously, a build would start but then it would
error out later on. This implements the mentality:

fail early, fail hard

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 13:22:19 +01:00
Leah Rowe 6125d3418f build/boot/roms: merge handle_targets/build_target
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 13:18:11 +01:00
Leah Rowe 13f5a4322b build/boot/roms: only run confirm_targets once
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 13:11:48 +01:00
Leah Rowe 5462bf1ca0 build/boot/roms: rename buildrom to build_target
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 13:06:56 +01:00
Leah Rowe fc097b3e0f build/boot/roms: split up main()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 13:05:57 +01:00
Leah Rowe 895073d765 build/boot/roms: simplify buildrom() handling
Only one for loop is required.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 12:59:20 +01:00
Leah Rowe df7305a5d9 build/boot/roms: support "all" if argument passed
e.g. -k ukqwerty

previously, this would not work:
./build boot roms -k ukqwerty all

only this would work:
./build boot roms all

this patch fixes the bug.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 12:51:40 +01:00
Leah Rowe b3e69cd9ac build/boot/roms: move help() to bottom of file
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 12:41:41 +01:00
Leah Rowe 385eb90c8c update/*/*: unified scanning of revisions/sources
update/blobs/download and update/project/repo both use
the same logic, for setting variables with awk and a
specially formatted configuration file.

unify this logic under include/option.sh, and use that.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 12:31:30 +01:00
Leah Rowe 9f5a5450af blobs/download: move helpers to include/blobutil
mkdirs() should be in include/blobutil.sh, as should
extract_archive(), because that is primarily where
they are used.

script/update/blobs/download calls these functions
aswell, but it sources include/blobutil.sh so it's OK.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 23:40:57 +01:00
Leah Rowe 416704fb4e include/blobutil: try curl first, then wget
Don't use only wget. Some systems may only have curl.

The user can always install wget anyway, but why not
support both? I've added the right user agent string.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 23:04:30 +01:00
Leah Rowe 6519cea9cf include/blobutil: simplify check_defconfig()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 22:34:34 +01:00
Leah Rowe ac05e5ff17 blobs/download: do IntelME extract in one function
Just one function.

Just one.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 22:12:55 +01:00
Leah Rowe 9b94df5dbb blobs/download: do final check of _dest in fetch()
This way, the file is checked regardless of what type of
blob is handled, not just Intel ME.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 21:44:51 +01:00
Leah Rowe 9a7bf4af0f blobs/download: don't pass dl_path as argument
Because fetch() is only called now from blobs/download,
we can reliably know what dl_path should be.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 20:43:24 +01:00
Leah Rowe 2b7ae8e204 blob scripts: unified handling of blob destination
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 20:29:13 +01:00
Leah Rowe 8ea62a1661 remove unused variables in blob scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 18:44:27 +01:00
Leah Rowe 32da4e319b merge include/fetch.sh, blobutil.sh, defconfig.sh
They are only ever used by script/update/blobs/*, so
put them all in blobutil.sh. This cuts down on the
number of scripts in lbmk.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 17:23:47 +01:00
Leah Rowe 710171f92b update/blobs/*: simplify mrc.bin handling
mrc.bin is now handled by include/mrc.sh, adapted
from now-deleted script/update/blobs/mrc

much of the logic has been re-written or adapted for
inside script/update/blobs/download

mrc links/hashes now defined in config/blobs/sources

the new code is simpler (and smaller). in addition,
lbmk can now easily handle mrc.bin files for other
platforms such as broadwell. watch this space.

the full .zip download is now cached, like with other
vendor downloads. this means it won't be re-downloaded
if it was already downloaded before.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 16:40:49 +01:00
Leah Rowe 0bb3c59620 update/blobs/*: unified download/checksum logic
Use the same logic between blobs/download and blobs/mrc.

The logic is taken from blobs/download.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 04:03:18 +01:00
Leah Rowe 5d934be7b0 blobs/download: remove unnecessary linebreaks
generally condense the code, but not in a way that
makes the code unreadable.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 03:37:05 +01:00
Leah Rowe 3256ef3e24 blobs/download: remove unnecessary messages
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 02:54:46 +01:00
Leah Rowe e9e1a3b4ae blobs/download: simplify downloading of files
individual functions for downloading each archive have
been removed. instead, eval is used in fetch_update(),
which is now renamed to fetch().

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 01:21:58 +01:00
Leah Rowe 781d0a8091 blobs/download: remove unnecessary error pipes
the called functions directly call err() under fault condition,
so this additional handling is redundant.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 00:46:04 +01:00
Leah Rowe 9aef57dfb7 blobs/download: unified archive extraction
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 00:29:54 +01:00
Leah Rowe 74c48a881d move build/command/options to include/option.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 22:31:24 +01:00
Leah Rowe a00b43375a build/release/roms: simplify strip_rom_image()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 21:33:19 +01:00
Leah Rowe 3b9442f7b2 blobs/download: unified blobdir handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 20:59:07 +01:00
Leah Rowe 373c84e472 blobs/download: unified archive extraction
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 20:41:21 +01:00
Leah Rowe 1e92abb177 blobs/download: remove errant debug line
i added that echo command when working on the function
in question, but it's not needed now.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 19:15:52 +01:00
Leah Rowe e73306ba8b remove script/update/blobs/extract
This script is incomplete, buggy and its use is ill advised.

This script can be re-added later, when more work is done.

The download and/or inject script is recommended.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 17:01:41 +01:00
Leah Rowe 16235cb6f7 blobs/download: simplify fetch_update()
Do not specifically name types of firmware. Instead,
pass the URLs and checksum as direct arguments.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 16:51:00 +01:00
Leah Rowe d023327f98 blobs/download: greatly simplify sources handling
remove the giant case/esac list, and set variables directly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 16:10:48 +01:00
Leah Rowe 6b17cda137 blobs/download: simplify defconfig handling
use the variable names directly, as defined in defconfig.

do not hardcode the if/else chain in detect_firmware, use
eval instead.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 15:01:49 +01:00
Leah Rowe b5628131ba handle/make/config: check project in main()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-26 02:01:43 +01:00
Leah Rowe f052f61fb7 handle/make/config: split up main()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-26 01:59:21 +01:00
Leah Rowe 67f4919ffe simplify getopts loops in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-26 01:35:15 +01:00
Leah Rowe 36b7f01a8a only update git submodules in project/trees
do not update them in project/repos - despite what
the previous commit message says, this behaviour is
error prone and should be avoided.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 12:46:54 +01:00
Leah Rowe 81d073d5a1 update/project/*: unified git reset handling
With this change, lbmk now also updates submodules on
simple git clones, not just multi-tree clones.

This is OK, because git does not return non-zero status
when git submodule update is ran, where git submodules
are not actually defined.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 12:21:23 +01:00
Leah Rowe 3738ec90ec update/project/*: unified patch handling
Handle patches by a function at include/git.sh

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 11:49:13 +01:00
Leah Rowe cd3225d845 update/project/trees: remove extra.sh handling
This functionality has never been used, except in the
erstwhile osboot project, and even then only experimentally.

It was intended for use with coreboot's gerrit site, but
it became Libreboot project policy that this not be relied
upon, instead preferring to include patches directly within
lbmk. This functionality can be re-added, if necessary.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 10:57:10 +01:00
Leah Rowe 42c9d7d28c build/grub/*: move common strings to variables
also general cleanup of these scripts

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 03:06:43 +01:00
Leah Rowe e94ba1f7c0 build/grub/payload: split up main()
also: the grub-mkstandalone command didn't have
a || at the end, even though it did specify an err
call. This has been corrected, so that the command
now defers to err() under fault conditions.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:57:45 +01:00
Leah Rowe 208620198c Update email address for Leah Rowe copyrights
also, some of them were out of date; years now updated.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:27:26 +01:00
Leah Rowe cc1642096e Use SPDX license headers on all scripts
This results in much cleaner copyright and license declarations.
SPDX headers are legally recognised and make auditing easier.

Also, remove descriptions of each script, from each script.
Libreboot documentation at docs/maintain/ describes them.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:19:48 +01:00
Leah Rowe a7b767a47b update/repos: concatenate multiple revision files
With this change, it's still possible to have a single
file at config/git/revisions, but this has been scrapped.

Instead, multiple files now exist under config/git/ with
the same modules declared, but the files are separated
logically. List of files under config/git:

* bios_extract
* biosutilities
* coreboot
* flashrom
* grub (gnulib also defined here)
* me_cleaner
* memtest86plus
* seabios
* serprog (multiple projects defined)
* u-boot
* uefitool

The rationale behind this change is simple: in the future,
we will stop relying on build systems within imported
projects for the import of git submodules. Instead, we
will handle them directly in lbmk.

Additionally, a Linux payload is planned for Libreboot, made
easier by the recent audit (script handle/make/config makes
it easy to integrate Linux, and handle cross-compilers for
userland utilities); a "linux" file under config/git/ could
also define rules for each project besides linux, such as
musl libc, busybox and other utilities.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 00:51:03 +01:00
Leah Rowe 7966f9111d handle/make/config: run fail() on error, not err()
This was an oversight, during a previous audit.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-24 18:57:01 +01:00
Riku Viitanen 3c30e1e3dd merge serprog scripts
13 sloc reduction

Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-16 21:49:45 +03:00
Leah Rowe 3400e5a12b rel/src: fix multi-line command
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-12 16:25:20 +01:00
Riku Viitanen c3ac62b173 serprog: list available boards
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-12 02:53:23 +03:00
Riku Viitanen 24185bca42 fix typo serprog -> vserprog
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-11 14:42:43 +03:00
Riku Viitanen ccb36aa652 make libopencm3 correctly
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-11 14:35:00 +03:00
Riku Viitanen 5737abf0ed make clean libopencm3
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-11 11:58:16 +03:00
Riku Viitanen 3d77b8a0b9 download and copy serprog related src
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-11 01:08:34 +03:00
Riku Viitanen 7dc8632514 clean up pico-serprog for release
moved cmake files into a separate build directory.
this can just be deleted for the source release.

might as well use cmake for the actual build too.
that makes repeated builds faster for some reason.

Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-11 01:01:32 +03:00
Riku Viitanen 34d3629ebf make clean stm32-vserprog for release
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-10 22:52:26 +03:00
Leah Rowe 093d40eec2 build/release/src: be more thorough deleting .git
a few were missed. nuke all of it from orbit.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 17:45:56 +01:00
Leah Rowe 630a65462f build/release/src: delete elf/ in srcdir
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 17:23:37 +01:00
Leah Rowe 0543350d44 handle/make/file: run make-clean first
flashrom distclean resulted in zero status upon exit,
but did not remove the actual flashrom binary.

our logic was to run distclean and defer to clean;
now, we run clean and *then* run distclean, but we
do not throw an error if distclean fails. (we do
throw one if clean fails)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 17:17:33 +01:00
Leah Rowe 12f9afe622 build/release/src: remove cbutils/ in srcdir
the builds were being created within that srcdir,
because build/release/src runs lbmk commands within
it, and one of them is building (re-building) it.

there's no point addressing this, other than rm -Rf

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 17:08:08 +01:00
Leah Rowe fe00ab4e91 build/release/src: remove errant code
the main lbmk script already creates these files,
and these files are then copied by build/release/src
so we don't need to re-create them here

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 17:04:59 +01:00
Leah Rowe be4ed54023 handle/make/config: distclean once per tree
previously, it was possible that the distclean or
crossgcc-clean modes were being executed on the same
project tree, needlessly. this patch fixes that.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 16:55:36 +01:00
Leah Rowe f227cc081e handle/make/config: fix distclean/crossgcc-clean
these commands weren't being run at all, leading
to binaries (such as xgcc) not being removed, and
thus they were present in tested release archives.

this bug did not affect libreboot 20230625. it
appeared during my audit, post-20230625.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 15:55:02 +01:00
Leah Rowe 669c9770cd handle/make/config: fix whitespace and 80-line bug
fit in 80 lines, and remove whitespace

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 15:45:56 +01:00
Leah Rowe d28ad6aa78 build/release/roms: use -T0 on serprog tarballs
xz supports using multiple threads

so use multiple threads

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 15:12:38 +01:00
Leah Rowe 308c21dd43 build/boot/roms stragglers: properly handle errors
there were a few missing err calls

i actually went through all of lbmk and found no
instances where err calls were missing except in
build/boot/roms_helper

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 15:05:19 +01:00
Leah Rowe c16b28efad build/release/src: re-create symlinks, don't copy
if you copy a symlink, you create a whole new file with the
contents of what that symlink points to.

what we need to do instead is re-create the symlinks. this
is relevant for all symlinks to the main lbmk script, from
the main directory of lbmk.git.

this avoids there being multiple copies of the main lbmk
script, in release archives.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 01:21:12 +01:00
Leah Rowe 5aef8156b5 scripts: use printf, not echo, where appropriate
printf has more universal behaviour, across various
implementations of sh, so it's better to use this.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 23:31:20 +01:00
Leah Rowe 76e12cd4a9 update/blobs printf statements: use double quotes
single quotes are not valid

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 23:27:44 +01:00
Leah Rowe 84bf47b5b9 scripts: better handling of printf: stdout/stderr
in some cases, messages that should be considered errors
or warnings, were being written to the standard output,
rather than written as error messages.

also: one or two printf statements should specifically
avoid printing errors (to any file); in these cases,
stdout has been redirected to /dev/null

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 23:15:17 +01:00
Leah Rowe f45f5e625d update/project/*: remove redundant checks
these scripts used to be in the main directory of
lbmk, and thus needed to check for root user, and
also git credentials. now they are called by the main
lbmk script, which also runs the same checks.

avoid waste of resources by not running the same
check twice.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 22:30:22 +01:00
Leah Rowe 3e76e70d83 blobs/download: don't use the -B option in make
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 22:14:02 +01:00
Leah Rowe 877c691ef5 build/release/roms: remove errant line break
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 22:11:28 +01:00
Leah Rowe f03efbc27f blobs/inject: add error condition on rm command
this was overlooked, earlier on in lbmk audit 2

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 20:15:50 +01:00
Leah Rowe 20be007f5b blobs/inject: fix checksum validation if no-ucode
on e6400_4mb, the release build scripts remove nvidia's vga
rom which is used on dgpu models. however, microcode is also
removed in separately copied rom images

the inject script was inserting vgaroms directly into these
no-microcode roms, but the microcode blob is bigger than the
vga rom, and cbfstool inserts into the first available free
spot within cbfs, so it was inserting into the spot where
cpu microcode went. this caused the rom checksum to not match
what was generated during build/release/roms being executed

the only real fix is to guarantee offsets within cbfs for all
files, by recording what offsets were used and then calculating
that during insertion

so this patch is a workaround, but fixes the issue. the workaround
is: don't insert blobs directly on no-microcode roms, instead
insert only on microcode-based roms, then re-copy those roms
and remove microcode in aptly named copies

it's a bit more convoluted, but works perfectly fine.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 20:11:11 +01:00
Leah Rowe 878550d519 use sha512sum to check downloads, not sha1sum
sha-1 has known collision issues, which may not be readily
exploitable yet (in our context), but we should ideally use
a more secure method for checking file integrity.

therefore, use sha-2 (sha512sum) for checking files. this is
slower than sha-1, but checksum verification is only a minor
part of what lbmk does, so the overall effect on build times
is quite negligible.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 16:39:26 +01:00
Leah Rowe 022e0200df Merge pull request 'Add stm32-vserprog' (#129) from Riku_V/lbmk:stm32 into master
Reviewed-on: https://codeberg.org/libreboot/lbmk/pulls/129
2023-09-09 14:13:05 +00:00
Riku Viitanen bed444ff90 Add stm32-vserprog
Tested on a Nucleo-F042K6.

That has an onboard stlink:
`st-flash --format ihex write bin/serprog_stm32/serprog_nucleo-f042k6.hex`

The usb port used for flashing is separate, its is exposed on
the pin header instead. Check boards/nucleo-f042k6.h for usb pinout.

Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-09 16:55:06 +03:00
Leah Rowe e9e4ada55f build/boot/rom: only insert scan.cfg if needed
where it is set to "both" (grub_scan_disk), inserting
scan.cfg is superfluous, because grub.cfg defaults to
both anyway, unless otherwise specified by scan.cfg,
and only if that file exists within cbfs.

thus, save a bit of build time (only a slight saving)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 00:10:23 +01:00
Leah Rowe 0e3f3efcaa build/boot/roms: delete tmpcfg when done
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 00:02:00 +01:00
Leah Rowe a69e8548b3 set grub.cfg timeout to 5s (10 on some boards)
target.cfg can now specify e.g.

grub_timeout=20

this would then be inserted as timeout.cfg in cbfs,
containing the instruction:
set timeout=20

HP laptops need a bit of extra time, due to the delay
caused by the EC bug workaround deployed in GRUB

desktops in general need extra time. this too is set to
10s, like the HP laptops.

only insert timeout.cfg if actually needed (declared in
target.cfg), otherwise grub.cfg will default to 5s

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 00:01:22 +01:00
Riku Viitanen 7b6fb95897 Build pico-serprog binary release archive
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-08 20:43:53 +03:00
Riku Viitanen c292e01b00 Build for all pico board, not just the "original"
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-08 20:43:53 +03:00
Riku Viitanen 1bde6bb3c4 Support multiple dependencies per project
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-08 20:43:53 +03:00
Riku Viitanen 4d3b16da38 Cleaner parent directory creation
My previous patch b0rked memtest and others because when making sure
their parent directory (the project root) exists, it would instead create
the project directory (memtest86lus). The later move would then put the
git repo inside that (memtest86plus/memtest86plus_123456).

We just need to make sure we don't create the target directory itself.
This way, there's no need to hardcode any project names.

Tested by ./updating rpi-pico-serprog, memtest86plus, grub and seabios.

Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-08 00:25:58 +03:00
Leah Rowe 8fc5f6ed53 update/blobs/inject: use tmp/romdir, not TMPDIR
we are copying large numbers of ROM images, and the
host system may have /tmp under a tmpfs; that same
host system may or may not have a lot of memory.

respect the user's machine.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-05 01:49:35 +01:00
Leah Rowe da991262a3 build/release/roms: use tmp/romdir, not TMPDIR
we must conserve memory usage, in the event that the
user's /tmp is a tmpfs. copying of ROM images into
tmpfs is ill advised; we must copy them, due to how
the release process works (e.g. stripping of blobs,
but this must be done in a way so as to not interfere
with regular builds, thus they are copied instead)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-05 01:39:36 +01:00
Leah Rowe 15081ed9ef grub: make backgrounds configurable in target.cfg
now under coreboot mainboards, target.cfg can specify
a background. if not specified, the 1280x800 one is
assumed, and used by default. it can be overridden.
the path should be relative to:
config/grub/background/

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-05 00:53:28 +01:00
Leah Rowe 0d315c3a4e curl/wget downloads: set re-try count to 3
explicitly set the count to 3, so that a maximum of 3
attemps are made per download, barring fatal errors such
as http 404.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 22:12:02 +01:00
Leah Rowe bdf171e3ec don't use /tmp/ for git clones. use tmp/ instead.
the /tmp/ file system may be a tmpfs, with conservative
memory limits, depending on host system.

it's more likely that the user will have enough disk space
under tmp/ within lbmk (if they don't, they can't use
lbmk anyway). that is to say: more likely that they would
have the disk space, but not the memory.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 21:45:29 +01:00
Leah Rowe 196f293a27 build/release/roms: fix ucode handling
microcode_required wasn't being reset per target,
leading to unreliable results. this fixes that.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 21:23:18 +01:00
Leah Rowe c0c7f3ae15 build/release/roms: simplify defcongic handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 21:19:27 +01:00
Leah Rowe a56cad71c0 update/blobs: unify global variables
they all more or less use the same variables, so put
them all under include/blobutil.sh

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 18:12:49 +01:00
Leah Rowe 2cbc7eea95 update/blobs/*: unify checking of defconfig files
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 16:16:53 +01:00
Leah Rowe 52677309c5 update/blobs/extract: replace errant target code
check based on whether defconfigs are available, which
are used extensively, rather than checking based on
whether target.cfg is available, which is not used

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 15:57:48 +01:00
Leah Rowe ea7fae97bd build/boot/roms: don't create empty bin/ directory
also: only return zero status if rom images were succesfully
built, and print a list of each rom image directory based on
what was actually compiled, rather than just saying that the
rom images are stored under bin/

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 14:54:36 +01:00
Leah Rowe c62a423909 update/blobs/inject: remove errant target handling
just like the last patch, target.cfg handling is not
required in this script either. remove it.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 14:17:09 +01:00
Leah Rowe 950166da7b update/blobs/download: remove errant code
the handling of target.cfg is *not* required, in
this script. other mechanisms are also used for
error checking. this script only uses defconfigs.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 13:44:07 +01:00
Leah Rowe 0668d234f0 add checkversion to build/release/src
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 12:39:22 +01:00
Leah Rowe 33e6088a16 move script/misc/versioncheck to main directory
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 03:28:11 +01:00
Leah Rowe 2c769dc136 move me7_update_parser.py to util/
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 03:24:46 +01:00
Leah Rowe da3c9bb3c5 merge config/ and resources/
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 02:47:25 +01:00
Leah Rowe a05010503f blobs/download: don't handle ifd/gbe files
they weren't even handled at all, but they were referenced
under coreboot configuration

they don't need to be handled. lbmk simply includes these files.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 02:12:05 +01:00
Leah Rowe 03788d14fb move ifd/gbe configs into config/ifd/
it doesn't really make sense for them to be under
blobs/ - nominally, they are blobs, but they are
well-understood data files containing config data,
that is easily parsed by tools like ich9show or
ifdtool (and tools like bincfg or nvmutil)

blobs/ has been re-purposed: this directory no longer
exists in lbmk, but it is created (and on .gitignore)
when needed, by blobutil

thus, the blobs/ directory shall only contain vendor
files, and only those files that libreboot scrubs from
releases. therefore, build/release/src can (and has
been) simplified; it currently copies just the ifd and
gbe files from blobs/, selectively, and this logic is
quite error prone, requiring maintenance. now, the
build/release/src script simply copies config/ (which
only ever contains distributable files) and entirely
ignores the blobs/ directory

the blob download script already creates the required
directory, except for the sch5545 download; this is
now fixed

lbmk code size is slightly smaller, due to this patch

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 01:38:08 +01:00
Leah Rowe 19efdf9eeb ich9m mainboards: use pre-assembled ifd/gbe files
This cuts down on build time, and it will allow libreboot
to remove large chunks of code.

these ifd/gbe configs are just binary-encoded config files,
in a format well-understood. they can easily be opened up
and displayed, using ich9show or ifdtool, and manipulated
by these tools; bincfg can generate them from scratch, and
nvmutil can change mac addresses, for example.

so, do this and remove from lbmk the following:

* ich9utils (which contains ich9gen) - not needed anymore
* code in lbmk for handling ich9gen and insertions; the
  coreboot build system is now used, for this same purpose,
  so remove such code from lbmk

this results in a massive code size reduction (thousands of
lines) in lbmk; smaller when only looking at the build
system, but much larger when you consider that ich9utils
is also removed (about 3k sloc)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 22:40:33 +01:00
Leah Rowe af8d8cda7d add ich9m ifd/gbe files
a follow-up patch will make use of these, rather than ich9gen,
and ich9gen will be deleted.

these files were in fact generated *by* ich9gen.

coreboot has ifdtool and bincfg, the latter of which can
generate both ifd and gbe files for ich9m. that, and nvmutil
which is part of libreboot, can change gbe mac addresses.

i was going to replace ich9gen with a script that would run
bincfg, ifdtool and nvmutil, to greatly reduce code size,
because ich9gen is about 3k sloc.

however, in practise we would always generate the same ifd
config, and basically only change the mac address if that's
what the user wants; nvmutil can already do that just fine.

so, just include the binaries directly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 19:19:50 +01:00
Leah Rowe d554efae81 build/release/src: copy e6430 ifd/gbe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 18:36:25 +01:00
Leah Rowe 09aae7be45 build/rpi-pico-serprog: better error handling
use a subshell for changing directory, and use more
verbose error messages under fault conditions

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 17:46:59 +01:00
Leah Rowe 1dc54608a1 fix rpi-pico builds when running it twice
it needed to be make-cleaned

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 17:36:42 +01:00
Leah Rowe c63052cf19 fix memtest86plus download/build
the mkdir command in update/project/repo, added for
pico-pi integration, broke a bunch of other downloads.

the fix is a bit of a hack but it should hold for now.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 17:28:10 +01:00
Riku Viitanen ef3fb05d66 Less cat abuse
More than 90% of cats were thus terminated.
read (shell built-in) is better at reading, and dogs are better pets.

Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-03 17:00:21 +03:00
Riku Viitanen e369e8fb4a automate rpi-pico serprog builds
./build rpi-pico serprog
build bin/serprog/rpi-pico-serprog.uf2

Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-03 04:00:42 +03:00
Leah Rowe 92b4db6992 build/release/src: only clean kbc1126 if it exists
the way this script works, it only copies what was built,
but it currently operatios as though coreboot/default
always exists, and then cleans the kbc1126 util

this patch fixes such buggy behaviour

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 01:53:35 +01:00
Leah Rowe 7c6b35cf95 unify build/clean scripts: use handle/make instead
The -c option is added for distclean, and -x for crossgcc-clean,
in handle/make/config

about 100 sloc removed from lbmk

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 01:53:35 +01:00
Leah Rowe cec37747b7 build/release/*: use -T0 in xz, for multithreading
The -T option specifies how many threads xz shall use.

The -T value of zero shall dictate that xz use so many
threads as there are CPUs, on the host system.

This will probably speed up the release process a bit.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 01:53:35 +01:00
Leah Rowe b30c7e330b coreboot/e6400: support nvidia models
The same ROM images that you flash on Intel GPU variants,
are now flashed on Nvidia models. The same ROM will work
on both. If an Intel GPU variant is present, libgfxinit
is used, and the VGA ROM is used if an Nvidia GPU variant;
however, release ROMs will scrub the nvidia option ROM,
so release ROMs will only work on Intel GPUs unless you
run the blobutil inject command.

I decided to no longer have this under WIP, but to put
it in master. The issue with it pertains to video drivers,
which is not Libreboot's problem.

Nouveau crashes under Linux, so use "nomodeset" if it does.
The "nv" drivers in BSD systems work very well.

The nvidia model of E6400 isn't recommended for other
reasons, namely: poor thermal cooling (thermal pad on
the GPU) and that Nvidia GPU doesn't get very good
performance on any libre drivers anyway. The Intel GPU
variant is better, in terms of power efficiency and
software support; the intel variant also works with
native graphics initialisation in coreboot.

This board port already only enables SeaBIOS, which will
simply execute the VGA ROM. Blobutil already supports
reading the config, detecting that a VGA ROM is needed,
because that part of the WIP E6400 branch was already
merged in lbmk master.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-02 17:40:49 +01:00
Leah Rowe 436b2ccb5a handle/make/config -m/-u: actually copy configs
they weren't being copied back, after running the
make command. i overlooked this when testing in
the previous optimisations, because i only tested
building, not modification or updating of configs

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-02 17:02:17 +01:00
Leah Rowe 3c7b09ace9 handle/make/config: properly handle cbutils
it wasn't being checked for, to run cbfstool.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-02 12:04:50 +01:00
Leah Rowe a3bc7ccd71 handle/make/file: fix uefitool builds
the current check only worked if it had already
been built, when checking for the Makefile

however, running this during build/release/src
caused problems, hence the current check

so: perform the same check, but as a fallback for
cmake failing (and if that check fails, only then
will err be called)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-02 10:43:35 +01:00
Leah Rowe 56f16bc883 don't do cmake on uefitool if the Makefile exists
nasty little hack to fix another nasty little hack,
which i call script/build/release/src

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 11:07:00 +01:00
Leah Rowe 98d1ea5ae7 build/release/src: bugfix: actually copy cb/ub/sb
coreboot, seabios and u-boot were not being copied at all

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 11:02:47 +01:00
Leah Rowe 755f925ad9 build/release/src: copy handle symlink
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 10:48:33 +01:00
Leah Rowe 3ad29d2d2a build/release/src: remove Makefile reference
lbmk never needed a makefile, because the build system
is all shell scripting; the former makefile simply called
those scripts, in a way that was mostly superfluous

build/release/src was still trying to copy it, so let's
remove it from that file

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 10:43:33 +01:00
Leah Rowe d69c231e24 build/release/src: fix bad variable reference
it's i, not 1

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 10:27:14 +01:00
Leah Rowe 38440153b5 update build/release/src based on lbmk changes
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 10:19:14 +01:00
Leah Rowe 0e782e7ee5 update the fetch scripts themselves
the fetch scripts call themselves, for dependencies

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 09:38:24 +01:00
Leah Rowe 98f30b6dbe build/coreboot/utils: exit 1 if target.cfg missing
it was previously trying to "continue", despite not being
inside a loop. the correct instruction would have
been "return 0", but then I thought it'd be better to
err here

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 08:46:24 +01:00
Leah Rowe b9662fbe36 handle project downloads in main lbmk script
this means the unified /tmp handling is now provided for
in both the former "fetch" and "fetch_trees" script, which
are now (respectively):

./update project repo
./update project trees

if the fetch scripts weren't cleaning /tmp before, they
now are, because lbmk handles it

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 08:30:08 +01:00
Leah Rowe 225e2609fa only remove microcode in build/release/roms
libreboot's build system, lbmk, *is* available to use
in releases aswell (use the _src tarball), but it is
mostly intended for development, in lbmk.git

well, there's not much point wasting time / disk space
generating no-microcode roms within lbmk

they should be generated only at release time, alongside
the default ones

this patch implements that, thus speeding up the build
process and saving disk usage during development

the other alternative was to add a new option in
build/boot/roms, -m, that would opt in to removing them,
but this is extra complexity for something that is ill
advised and only provided to appease certain people

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-31 23:30:56 +01:00
Leah Rowe bf774acf1d move build/boot/rom moverom to handle/make/config
most of these steps do not need to be repeated, per image.

move it into handle/make/config, so that the steps are
performed on files that go under elf/coreboot (this will
save on build time).

the logic for handling 4MB ROM images on sandy/ivy was unused,
and has been removed.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-31 21:12:35 +01:00
Leah Rowe e5546128ea build/release/roms: fix syntax error
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-31 17:47:56 +01:00
Leah Rowe fbda0f04db re-add /dev/null redirect on . ${1}
the error messages that it shows are benign, but users
see them and worry that something went wrong

this patch reduces the number of people asking pointless
questions on irc

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-31 17:16:54 +01:00
Leah Rowe b2bad5a030 build/release/src: copy the include/ directory
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-29 13:15:13 +01:00
Leah Rowe eb54e427e6 grub: all one grub.elf containing keymaps and cfg
new behaviour:
* grub.cfg and grubtest.cfg no longer inserted to cbfs
* grub.cfg in memdisk instead
* grub.cfg in memdisk defers to cbfs/grub.cfg if added
  (not added by default, anymore)
* does not defer to grubtest.cfg even if available
* only shows link to grubtest.cfg if available,
  as a menuentry item

keymaps:
if /keymap.gkb exists in cbfs, it uses that by default,
but by default this isn't added. instead, it looks for
a file named keymap.cfg and sources that, which then
sets the keymap to one that is located under memdisk.
this file is inserted for each rom, per layout.
if keymap.gkb and keymap.cfg both absent, grub.cfg in
memdisk shall defer to usqwerty as the default keymap

grub_scan_disk: grub.cfg looks for cbfs file "scan.cfg"
and sources that if found, which will be inserted with
the string: set grub_scandisk=setting_goes_here (based
on target.cfg, generated by build/boot/roms automatically).
If no scan.cfg is found, it defaults to "both"

The "background.png" file remains unchanged, and present in
CBFS, used by grub.cfg if present (and it is, by default)

This change actually *saves* space in CBFS, due to compression,
and means that the grub.cfg is now compressed heavily. This
is also safer, because now the user overrides grub.cfg by
adding it, and they can still add grubtest.cfg for testing
first. If they accidentally delete both configs from cbfs,
Libreboot will fall back to the one in memdisk which would
presumably not be deleted.

This also means that lbmk can now more easily be used by
other build systems, that just want the GRUB part to re-use
in their own project. For example, people who want to build
custom coreboot images without using Libreboot's build system.

This change also *speeds* up the build process considerably,
on the parts where ROM images are copied. It's less than half
a second now, whereas previously it took about 30-45 seconds
for ROM images to copy, because of grub.elf being re-added in
each ROM via cbfstool, where compression is used; I believe
the compression part is what caused slowness.

Much, much faster, more versatile builds.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-29 02:03:16 +01:00
Leah Rowe 6722624dfc build/boot/roms: fix bad variable assignment
this was an oversight, in a previous commit.
there was a space, between variable name and
the equals sign, and then another space, so it
was trying to *execute* the rom

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-28 13:51:17 +01:00
Leah Rowe 6dbddf852c build/boot/roms: simplify ich9m ifd handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-28 11:41:38 +01:00
Leah Rowe f5787c9e50 build/boot/roms ich9m ifd: use fast dd command
bs=block size and count=1,
rather than bs=1 and count=block size

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-28 11:28:41 +01:00
Leah Rowe d9292cec6a build/boot/roms: use the new coding style
lbmk's new style is inspired by the bsd coding styles:
top-down logic, main simplified to a skeleton showing
overall program structure, variables well-defined,
rigorous (yet deceptively simple) error checking.

this was attempted before, but caused problems; coreboot
wasn't being cleaned properly, and rather than audit it,
i simply reverted this back to the old style.

this is actually attempt number 5, because i made 3 more
attempts between then and this one. i've build-tested this
using "./build boot roms all" (which is what b0rked on
the first attempt, months ago). it should be stable(tm).

the code is much nicer to read / work on now. this is the
beating heart of lbmk. get this script wrong, and you break
all of libreboot.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-28 11:23:23 +01:00
Leah Rowe 4623f3f2b2 Remove superfluous GRUB modules (save CBFS space)
With this change, about 54KB of compressed space is saved
inside of CBFS, on setups that use the GRUB payload.

The uncompressed saving is about 720KB, but payloads are
compressed inside each coreboot image, so the compressed
saving is much smaller. That 54KB saving means a lot,
especially on small (1MB or smaller) flash sizes.

The following modules were removed:

adler32, afsplitter, aout, archelp, backtrace, blocklist,
bswap_test, cat, cmdline_cat_test, cmosdump, cmostest, cmp,
cmp_test, cpuid, cs5536, ctz_test, date, datehook, datetime,
disk, diskfilter, div, div_test, dm_nv, efiemu, eval,
exfctest, extcmd, file, fshelp, functional_test, gdb,
gettext, gptsync, hashsum, hdparm, hello, hfspluscomp, http,
json, json, ldm, loadenv, macbless, macho, mda_text, morse,
mpi, msdospart, mul_test, net, ntfscomp, offsetio,
part_acorn, part_amiga, part_apple, part_dvh, part_plan,
part_sun, part_sunpc, parttool, pbkdf2, pbkdf2_test, pci,
play, priority_queue, probe, progress, random, rdmsr, read,
relocator, setjmp, setjmp_test, shift_test, signature_test,
sleep, sleep_test, smbios, strtoull_test, terminal,
terminfo, test_blockarg, testload, testspeed, tftp, tga,
time, tr, trig, usbtest, video_bochs, video_cirrus,
videoinfo, videotest, videotest_checksum, wrmsr, xnu_uuid,
xnu_uuid_test

These were retained, but moved to modules instead of
install modules:

geli, udf, ufs1, ufs1_be, ufs2

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-28 01:25:10 +01:00
Leah Rowe 355eb765ff move resources/scripts/ to script/
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-27 17:19:36 +01:00