Commit Graph

1827 Commits (5e2308ce054f52e39644769728a5b6e8e9cb4426)

Author SHA1 Message Date
Leah Rowe 9f5a5450af blobs/download: move helpers to include/blobutil
mkdirs() should be in include/blobutil.sh, as should
extract_archive(), because that is primarily where
they are used.

script/update/blobs/download calls these functions
aswell, but it sources include/blobutil.sh so it's OK.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 23:40:57 +01:00
Leah Rowe 416704fb4e include/blobutil: try curl first, then wget
Don't use only wget. Some systems may only have curl.

The user can always install wget anyway, but why not
support both? I've added the right user agent string.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 23:04:30 +01:00
Leah Rowe 6519cea9cf include/blobutil: simplify check_defconfig()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 22:34:34 +01:00
Leah Rowe ac05e5ff17 blobs/download: do IntelME extract in one function
Just one function.

Just one.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 22:12:55 +01:00
Leah Rowe 9b94df5dbb blobs/download: do final check of _dest in fetch()
This way, the file is checked regardless of what type of
blob is handled, not just Intel ME.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 21:44:51 +01:00
Leah Rowe 9a7bf4af0f blobs/download: don't pass dl_path as argument
Because fetch() is only called now from blobs/download,
we can reliably know what dl_path should be.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 20:43:24 +01:00
Leah Rowe 2b7ae8e204 blob scripts: unified handling of blob destination
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 20:29:13 +01:00
Leah Rowe 8ea62a1661 remove unused variables in blob scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 18:44:27 +01:00
Leah Rowe 32da4e319b merge include/fetch.sh, blobutil.sh, defconfig.sh
They are only ever used by script/update/blobs/*, so
put them all in blobutil.sh. This cuts down on the
number of scripts in lbmk.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 17:23:47 +01:00
Leah Rowe 710171f92b update/blobs/*: simplify mrc.bin handling
mrc.bin is now handled by include/mrc.sh, adapted
from now-deleted script/update/blobs/mrc

much of the logic has been re-written or adapted for
inside script/update/blobs/download

mrc links/hashes now defined in config/blobs/sources

the new code is simpler (and smaller). in addition,
lbmk can now easily handle mrc.bin files for other
platforms such as broadwell. watch this space.

the full .zip download is now cached, like with other
vendor downloads. this means it won't be re-downloaded
if it was already downloaded before.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 16:40:49 +01:00
Leah Rowe 0bb3c59620 update/blobs/*: unified download/checksum logic
Use the same logic between blobs/download and blobs/mrc.

The logic is taken from blobs/download.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 04:03:18 +01:00
Leah Rowe 5d934be7b0 blobs/download: remove unnecessary linebreaks
generally condense the code, but not in a way that
makes the code unreadable.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 03:37:05 +01:00
Leah Rowe 3256ef3e24 blobs/download: remove unnecessary messages
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 02:54:46 +01:00
Leah Rowe 178b888aa0 include/blobutil: properly set global variables
I was setting certain global variables inside for loops,
but some sh implementations won't like this.

Instead, don't run eval inside the for loops. Set a string
for eval inside the for loops, then execute eval outside of
the loops. This should work on every shell.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 02:28:51 +01:00
Leah Rowe e9e1a3b4ae blobs/download: simplify downloading of files
individual functions for downloading each archive have
been removed. instead, eval is used in fetch_update(),
which is now renamed to fetch().

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 01:21:58 +01:00
Leah Rowe 781d0a8091 blobs/download: remove unnecessary error pipes
the called functions directly call err() under fault condition,
so this additional handling is redundant.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 00:46:04 +01:00
Leah Rowe 9aef57dfb7 blobs/download: unified archive extraction
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 00:29:54 +01:00
Leah Rowe 74c48a881d move build/command/options to include/option.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 22:31:24 +01:00
Leah Rowe a00b43375a build/release/roms: simplify strip_rom_image()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 21:33:19 +01:00
Leah Rowe 3b9442f7b2 blobs/download: unified blobdir handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 20:59:07 +01:00
Leah Rowe 373c84e472 blobs/download: unified archive extraction
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 20:41:21 +01:00
Leah Rowe 1e92abb177 blobs/download: remove errant debug line
i added that echo command when working on the function
in question, but it's not needed now.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 19:15:52 +01:00
Leah Rowe e73306ba8b remove script/update/blobs/extract
This script is incomplete, buggy and its use is ill advised.

This script can be re-added later, when more work is done.

The download and/or inject script is recommended.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 17:01:41 +01:00
Leah Rowe 16235cb6f7 blobs/download: simplify fetch_update()
Do not specifically name types of firmware. Instead,
pass the URLs and checksum as direct arguments.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 16:51:00 +01:00
Leah Rowe d023327f98 blobs/download: greatly simplify sources handling
remove the giant case/esac list, and set variables directly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 16:10:48 +01:00
Leah Rowe 65a3269835 include/blobutil: simplify setting empty strings
use a for loop and eval to set them.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 15:26:17 +01:00
Leah Rowe 6b17cda137 blobs/download: simplify defconfig handling
use the variable names directly, as defined in defconfig.

do not hardcode the if/else chain in detect_firmware, use
eval instead.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 15:01:49 +01:00
Leah Rowe b5628131ba handle/make/config: check project in main()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-26 02:01:43 +01:00
Leah Rowe f052f61fb7 handle/make/config: split up main()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-26 01:59:21 +01:00
Leah Rowe 67f4919ffe simplify getopts loops in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-26 01:35:15 +01:00
Leah Rowe 36b7f01a8a only update git submodules in project/trees
do not update them in project/repos - despite what
the previous commit message says, this behaviour is
error prone and should be avoided.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 12:46:54 +01:00
Leah Rowe 81d073d5a1 update/project/*: unified git reset handling
With this change, lbmk now also updates submodules on
simple git clones, not just multi-tree clones.

This is OK, because git does not return non-zero status
when git submodule update is ran, where git submodules
are not actually defined.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 12:21:23 +01:00
Leah Rowe eae173ec13 split up grub patches into subdirectories
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 12:03:07 +01:00
Leah Rowe a823bab365 include/git: support applying patch subdirectories
This is done recursively, with the following rule:
files first, then directories.

Where all patch files are applied from within the
patch directory, subdirectories (within the patch
directory) are then tried in alphanumerical order.

Then, within each subdirectory tried, the same rule
is once again applied. This is done recursively,
until every patch file is applied.

The code no longer applies *.patch, but instead any
file. Additionally, symlinks are avoided.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 12:00:43 +01:00
Leah Rowe 3738ec90ec update/project/*: unified patch handling
Handle patches by a function at include/git.sh

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 11:49:13 +01:00
Leah Rowe cd3225d845 update/project/trees: remove extra.sh handling
This functionality has never been used, except in the
erstwhile osboot project, and even then only experimentally.

It was intended for use with coreboot's gerrit site, but
it became Libreboot project policy that this not be relied
upon, instead preferring to include patches directly within
lbmk. This functionality can be re-added, if necessary.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 10:57:10 +01:00
Leah Rowe 42c9d7d28c build/grub/*: move common strings to variables
also general cleanup of these scripts

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 03:06:43 +01:00
Leah Rowe e94ba1f7c0 build/grub/payload: split up main()
also: the grub-mkstandalone command didn't have
a || at the end, even though it did specify an err
call. This has been corrected, so that the command
now defers to err() under fault conditions.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:57:45 +01:00
Leah Rowe b727f9666b util/: use SPDX license and copyright headers
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:49:34 +01:00
Leah Rowe 208620198c Update email address for Leah Rowe copyrights
also, some of them were out of date; years now updated.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:27:26 +01:00
Leah Rowe cc1642096e Use SPDX license headers on all scripts
This results in much cleaner copyright and license declarations.
SPDX headers are legally recognised and make auditing easier.

Also, remove descriptions of each script, from each script.
Libreboot documentation at docs/maintain/ describes them.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:19:48 +01:00
Leah Rowe a7b767a47b update/repos: concatenate multiple revision files
With this change, it's still possible to have a single
file at config/git/revisions, but this has been scrapped.

Instead, multiple files now exist under config/git/ with
the same modules declared, but the files are separated
logically. List of files under config/git:

* bios_extract
* biosutilities
* coreboot
* flashrom
* grub (gnulib also defined here)
* me_cleaner
* memtest86plus
* seabios
* serprog (multiple projects defined)
* u-boot
* uefitool

The rationale behind this change is simple: in the future,
we will stop relying on build systems within imported
projects for the import of git submodules. Instead, we
will handle them directly in lbmk.

Additionally, a Linux payload is planned for Libreboot, made
easier by the recent audit (script handle/make/config makes
it easy to integrate Linux, and handle cross-compilers for
userland utilities); a "linux" file under config/git/ could
also define rules for each project besides linux, such as
musl libc, busybox and other utilities.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 00:51:03 +01:00
Leah Rowe 7966f9111d handle/make/config: run fail() on error, not err()
This was an oversight, during a previous audit.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-24 18:57:01 +01:00
Leah Rowe 2d0e978c94 update grub revision
It's now 44 revisions above 2.12-rc1, not 17 above.

The additional patches (in GRUB master) contain several
important fixes, including cryptodisk and ZFS fixes plus
a few other interesting changes, namely:

14c95e57fddb6c826bee7755232de62efc8eb45b:
kern/misc: Make grub_vsnprintf() C99/POSIX conformant

296d3ec835ed6e3b90d740e497bb534f14fe4b79:
disk/cryptodisk: Fix missing change when updating to use grub_uuidcasecmp()

42a831d7462ec3a114156d56ef8a03e1d47f19e7:
ZFS: support inode type embed into its ID

96446ce14e2d1fe9f5b36ec4ac45a2efd92a40d1:
ZFS: Fix invalid memcmp

444089eec6042250ce3a7184cb09bd8a2ab16808:
ZFS: Don't iterate over null objsets

7ce5b4911005b2a0bfd716d92466b6711844068c:
ZFS: Check bonustype in addition to dnode type

There are more patches than this, but these are the
ones that strike me as interesting for Libreboot.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-24 18:03:14 +01:00
Leah Rowe 905f3d8e7b util/nvmutil: remove xorswap() macro
it's only used once, so just do it once.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-19 16:51:37 +01:00
Leah Rowe 231015ef89 util/nvmutil: make setWord a macro
253 sloccount on nvmutil.c now, versus 258

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-19 16:18:13 +01:00
Leah Rowe d9bed11501 util/nvmutil: further optimise swap command
don't swap pointers at all. handle it in the for loop.

258 sloccount now, versus 261.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-19 16:05:34 +01:00
Leah Rowe 5e8013601a util/nvmutil: use correct comparisons on pointers
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-19 16:01:26 +01:00
Leah Rowe 137a548b04 util/nvmutil: optimise swap command
handle it exclusively in writeGbeFile()

this reduces nvmutil.c sloccount to 261, versus 265

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-19 15:56:55 +01:00
Leah Rowe 4d44820163 util/nvmutil: don't use err_if on argc check
at this stage in the code, the file name will be NULL
value, so it would be improper to use it in a string.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-18 16:48:26 +01:00