Commit Graph

62 Commits (3c7e37b15d84cbdac6777c5cdb9f050af1c8b16b)

Author SHA1 Message Date
Leah Rowe 3c7e37b15d update/blobs: correct utils paths check
some utils were being needlessly re-downloaded.
fix that!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-07 00:47:42 +01:00
Leah Rowe 5e81024e97 update/blobs: don't hardcode kbc1126 util check
use the variable that is set up in blobutil.sh

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-07 00:46:38 +01:00
Leah Rowe 9f8f230b61 update/blobs: don't needlessly re-build uefitool
during the switch to src/ for all downloads, i
overlooked that the path check was hardcoded.

now the check for this binary is corrected.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-07 00:45:34 +01:00
Leah Rowe 4e39d5a5a8 put all src downloads under src/
build/release/src was partly re-written to accomodate this

memtest86plus was patched to have a central Makefile, and
lbmk modified to use that, rather than mess with build32
and build64. the central Makefile just builds both targets
or cleans both targets

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-07 00:11:21 +01:00
Leah Rowe 2d16e1ee47 rename build/project/trees to update/project/build
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-06 01:59:13 +01:00
Leah Rowe 176722a841 unify handle/make/* into build/project/trees
Just one script.

Just one.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-06 01:16:39 +01:00
Leah Rowe 5de8eda21c general code cleanup in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-03 12:59:49 +01:00
Leah Rowe 0db6c0a4a8 update/blobs/download: remove errant comment
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-02 22:39:25 +01:00
Leah Rowe 8c03b886c4 Greatly simplify error handling in shell scripts
Instead of having detailed error messages, run most
commands through a function that calls err() under
fault conditions.

Where detail is still required, err() is still called
manually. Where it isn't, the error message is simply
whatever command was executed to cause the error.

This results in a massive sloccount reduction for lbmk;
specifically, 178 sloc reduction, or a 8.1% reduction.
The total sloccount is now 2022, for shell scripts.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-01 22:47:02 +01:00
Leah Rowe 26fc3f1325 general code formatting cleanup in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 20:06:29 +01:00
Leah Rowe 0a0defd325 simplify initialising variables in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 19:09:25 +01:00
Leah Rowe 385eb90c8c update/*/*: unified scanning of revisions/sources
update/blobs/download and update/project/repo both use
the same logic, for setting variables with awk and a
specially formatted configuration file.

unify this logic under include/option.sh, and use that.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30 12:31:30 +01:00
Leah Rowe 9f5a5450af blobs/download: move helpers to include/blobutil
mkdirs() should be in include/blobutil.sh, as should
extract_archive(), because that is primarily where
they are used.

script/update/blobs/download calls these functions
aswell, but it sources include/blobutil.sh so it's OK.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 23:40:57 +01:00
Leah Rowe 416704fb4e include/blobutil: try curl first, then wget
Don't use only wget. Some systems may only have curl.

The user can always install wget anyway, but why not
support both? I've added the right user agent string.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 23:04:30 +01:00
Leah Rowe 6519cea9cf include/blobutil: simplify check_defconfig()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 22:34:34 +01:00
Leah Rowe ac05e5ff17 blobs/download: do IntelME extract in one function
Just one function.

Just one.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 22:12:55 +01:00
Leah Rowe 9b94df5dbb blobs/download: do final check of _dest in fetch()
This way, the file is checked regardless of what type of
blob is handled, not just Intel ME.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 21:44:51 +01:00
Leah Rowe 9a7bf4af0f blobs/download: don't pass dl_path as argument
Because fetch() is only called now from blobs/download,
we can reliably know what dl_path should be.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 20:43:24 +01:00
Leah Rowe 2b7ae8e204 blob scripts: unified handling of blob destination
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 20:29:13 +01:00
Leah Rowe 8ea62a1661 remove unused variables in blob scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 18:44:27 +01:00
Leah Rowe 32da4e319b merge include/fetch.sh, blobutil.sh, defconfig.sh
They are only ever used by script/update/blobs/*, so
put them all in blobutil.sh. This cuts down on the
number of scripts in lbmk.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 17:23:47 +01:00
Leah Rowe 710171f92b update/blobs/*: simplify mrc.bin handling
mrc.bin is now handled by include/mrc.sh, adapted
from now-deleted script/update/blobs/mrc

much of the logic has been re-written or adapted for
inside script/update/blobs/download

mrc links/hashes now defined in config/blobs/sources

the new code is simpler (and smaller). in addition,
lbmk can now easily handle mrc.bin files for other
platforms such as broadwell. watch this space.

the full .zip download is now cached, like with other
vendor downloads. this means it won't be re-downloaded
if it was already downloaded before.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 16:40:49 +01:00
Leah Rowe 0bb3c59620 update/blobs/*: unified download/checksum logic
Use the same logic between blobs/download and blobs/mrc.

The logic is taken from blobs/download.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29 04:03:18 +01:00
Leah Rowe 5d934be7b0 blobs/download: remove unnecessary linebreaks
generally condense the code, but not in a way that
makes the code unreadable.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 03:37:05 +01:00
Leah Rowe 3256ef3e24 blobs/download: remove unnecessary messages
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 02:54:46 +01:00
Leah Rowe e9e1a3b4ae blobs/download: simplify downloading of files
individual functions for downloading each archive have
been removed. instead, eval is used in fetch_update(),
which is now renamed to fetch().

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 01:21:58 +01:00
Leah Rowe 781d0a8091 blobs/download: remove unnecessary error pipes
the called functions directly call err() under fault condition,
so this additional handling is redundant.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 00:46:04 +01:00
Leah Rowe 9aef57dfb7 blobs/download: unified archive extraction
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28 00:29:54 +01:00
Leah Rowe 74c48a881d move build/command/options to include/option.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 22:31:24 +01:00
Leah Rowe 3b9442f7b2 blobs/download: unified blobdir handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 20:59:07 +01:00
Leah Rowe 373c84e472 blobs/download: unified archive extraction
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 20:41:21 +01:00
Leah Rowe 1e92abb177 blobs/download: remove errant debug line
i added that echo command when working on the function
in question, but it's not needed now.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 19:15:52 +01:00
Leah Rowe e73306ba8b remove script/update/blobs/extract
This script is incomplete, buggy and its use is ill advised.

This script can be re-added later, when more work is done.

The download and/or inject script is recommended.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 17:01:41 +01:00
Leah Rowe 16235cb6f7 blobs/download: simplify fetch_update()
Do not specifically name types of firmware. Instead,
pass the URLs and checksum as direct arguments.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 16:51:00 +01:00
Leah Rowe d023327f98 blobs/download: greatly simplify sources handling
remove the giant case/esac list, and set variables directly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 16:10:48 +01:00
Leah Rowe 6b17cda137 blobs/download: simplify defconfig handling
use the variable names directly, as defined in defconfig.

do not hardcode the if/else chain in detect_firmware, use
eval instead.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 15:01:49 +01:00
Leah Rowe 67f4919ffe simplify getopts loops in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-26 01:35:15 +01:00
Leah Rowe 208620198c Update email address for Leah Rowe copyrights
also, some of them were out of date; years now updated.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:27:26 +01:00
Leah Rowe cc1642096e Use SPDX license headers on all scripts
This results in much cleaner copyright and license declarations.
SPDX headers are legally recognised and make auditing easier.

Also, remove descriptions of each script, from each script.
Libreboot documentation at docs/maintain/ describes them.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:19:48 +01:00
Leah Rowe 308c21dd43 build/boot/roms stragglers: properly handle errors
there were a few missing err calls

i actually went through all of lbmk and found no
instances where err calls were missing except in
build/boot/roms_helper

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 15:05:19 +01:00
Leah Rowe 5aef8156b5 scripts: use printf, not echo, where appropriate
printf has more universal behaviour, across various
implementations of sh, so it's better to use this.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 23:31:20 +01:00
Leah Rowe 76e12cd4a9 update/blobs printf statements: use double quotes
single quotes are not valid

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 23:27:44 +01:00
Leah Rowe 84bf47b5b9 scripts: better handling of printf: stdout/stderr
in some cases, messages that should be considered errors
or warnings, were being written to the standard output,
rather than written as error messages.

also: one or two printf statements should specifically
avoid printing errors (to any file); in these cases,
stdout has been redirected to /dev/null

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 23:15:17 +01:00
Leah Rowe 3e76e70d83 blobs/download: don't use the -B option in make
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 22:14:02 +01:00
Leah Rowe f03efbc27f blobs/inject: add error condition on rm command
this was overlooked, earlier on in lbmk audit 2

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 20:15:50 +01:00
Leah Rowe 20be007f5b blobs/inject: fix checksum validation if no-ucode
on e6400_4mb, the release build scripts remove nvidia's vga
rom which is used on dgpu models. however, microcode is also
removed in separately copied rom images

the inject script was inserting vgaroms directly into these
no-microcode roms, but the microcode blob is bigger than the
vga rom, and cbfstool inserts into the first available free
spot within cbfs, so it was inserting into the spot where
cpu microcode went. this caused the rom checksum to not match
what was generated during build/release/roms being executed

the only real fix is to guarantee offsets within cbfs for all
files, by recording what offsets were used and then calculating
that during insertion

so this patch is a workaround, but fixes the issue. the workaround
is: don't insert blobs directly on no-microcode roms, instead
insert only on microcode-based roms, then re-copy those roms
and remove microcode in aptly named copies

it's a bit more convoluted, but works perfectly fine.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 20:11:11 +01:00
Leah Rowe 878550d519 use sha512sum to check downloads, not sha1sum
sha-1 has known collision issues, which may not be readily
exploitable yet (in our context), but we should ideally use
a more secure method for checking file integrity.

therefore, use sha-2 (sha512sum) for checking files. this is
slower than sha-1, but checksum verification is only a minor
part of what lbmk does, so the overall effect on build times
is quite negligible.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 16:39:26 +01:00
Leah Rowe 8fc5f6ed53 update/blobs/inject: use tmp/romdir, not TMPDIR
we are copying large numbers of ROM images, and the
host system may have /tmp under a tmpfs; that same
host system may or may not have a lot of memory.

respect the user's machine.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-05 01:49:35 +01:00
Leah Rowe 0d315c3a4e curl/wget downloads: set re-try count to 3
explicitly set the count to 3, so that a maximum of 3
attemps are made per download, barring fatal errors such
as http 404.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 22:12:02 +01:00
Leah Rowe a56cad71c0 update/blobs: unify global variables
they all more or less use the same variables, so put
them all under include/blobutil.sh

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 18:12:49 +01:00