Commit Graph

146 Commits (493ebdfb74a7b0f4a5b23b4bdf80936247a67a47)

Author SHA1 Message Date
Leah Rowe 74c48a881d move build/command/options to include/option.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 22:31:24 +01:00
Leah Rowe 3b9442f7b2 blobs/download: unified blobdir handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 20:59:07 +01:00
Leah Rowe 373c84e472 blobs/download: unified archive extraction
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 20:41:21 +01:00
Leah Rowe 1e92abb177 blobs/download: remove errant debug line
i added that echo command when working on the function
in question, but it's not needed now.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 19:15:52 +01:00
Leah Rowe e73306ba8b remove script/update/blobs/extract
This script is incomplete, buggy and its use is ill advised.

This script can be re-added later, when more work is done.

The download and/or inject script is recommended.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 17:01:41 +01:00
Leah Rowe 16235cb6f7 blobs/download: simplify fetch_update()
Do not specifically name types of firmware. Instead,
pass the URLs and checksum as direct arguments.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 16:51:00 +01:00
Leah Rowe d023327f98 blobs/download: greatly simplify sources handling
remove the giant case/esac list, and set variables directly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 16:10:48 +01:00
Leah Rowe 6b17cda137 blobs/download: simplify defconfig handling
use the variable names directly, as defined in defconfig.

do not hardcode the if/else chain in detect_firmware, use
eval instead.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27 15:01:49 +01:00
Leah Rowe 67f4919ffe simplify getopts loops in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-26 01:35:15 +01:00
Leah Rowe 36b7f01a8a only update git submodules in project/trees
do not update them in project/repos - despite what
the previous commit message says, this behaviour is
error prone and should be avoided.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 12:46:54 +01:00
Leah Rowe 81d073d5a1 update/project/*: unified git reset handling
With this change, lbmk now also updates submodules on
simple git clones, not just multi-tree clones.

This is OK, because git does not return non-zero status
when git submodule update is ran, where git submodules
are not actually defined.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 12:21:23 +01:00
Leah Rowe 3738ec90ec update/project/*: unified patch handling
Handle patches by a function at include/git.sh

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 11:49:13 +01:00
Leah Rowe cd3225d845 update/project/trees: remove extra.sh handling
This functionality has never been used, except in the
erstwhile osboot project, and even then only experimentally.

It was intended for use with coreboot's gerrit site, but
it became Libreboot project policy that this not be relied
upon, instead preferring to include patches directly within
lbmk. This functionality can be re-added, if necessary.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 10:57:10 +01:00
Leah Rowe 208620198c Update email address for Leah Rowe copyrights
also, some of them were out of date; years now updated.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:27:26 +01:00
Leah Rowe cc1642096e Use SPDX license headers on all scripts
This results in much cleaner copyright and license declarations.
SPDX headers are legally recognised and make auditing easier.

Also, remove descriptions of each script, from each script.
Libreboot documentation at docs/maintain/ describes them.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 02:19:48 +01:00
Leah Rowe a7b767a47b update/repos: concatenate multiple revision files
With this change, it's still possible to have a single
file at config/git/revisions, but this has been scrapped.

Instead, multiple files now exist under config/git/ with
the same modules declared, but the files are separated
logically. List of files under config/git:

* bios_extract
* biosutilities
* coreboot
* flashrom
* grub (gnulib also defined here)
* me_cleaner
* memtest86plus
* seabios
* serprog (multiple projects defined)
* u-boot
* uefitool

The rationale behind this change is simple: in the future,
we will stop relying on build systems within imported
projects for the import of git submodules. Instead, we
will handle them directly in lbmk.

Additionally, a Linux payload is planned for Libreboot, made
easier by the recent audit (script handle/make/config makes
it easy to integrate Linux, and handle cross-compilers for
userland utilities); a "linux" file under config/git/ could
also define rules for each project besides linux, such as
musl libc, busybox and other utilities.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25 00:51:03 +01:00
Leah Rowe 308c21dd43 build/boot/roms stragglers: properly handle errors
there were a few missing err calls

i actually went through all of lbmk and found no
instances where err calls were missing except in
build/boot/roms_helper

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10 15:05:19 +01:00
Leah Rowe 5aef8156b5 scripts: use printf, not echo, where appropriate
printf has more universal behaviour, across various
implementations of sh, so it's better to use this.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 23:31:20 +01:00
Leah Rowe 76e12cd4a9 update/blobs printf statements: use double quotes
single quotes are not valid

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 23:27:44 +01:00
Leah Rowe 84bf47b5b9 scripts: better handling of printf: stdout/stderr
in some cases, messages that should be considered errors
or warnings, were being written to the standard output,
rather than written as error messages.

also: one or two printf statements should specifically
avoid printing errors (to any file); in these cases,
stdout has been redirected to /dev/null

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 23:15:17 +01:00
Leah Rowe f45f5e625d update/project/*: remove redundant checks
these scripts used to be in the main directory of
lbmk, and thus needed to check for root user, and
also git credentials. now they are called by the main
lbmk script, which also runs the same checks.

avoid waste of resources by not running the same
check twice.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 22:30:22 +01:00
Leah Rowe 3e76e70d83 blobs/download: don't use the -B option in make
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 22:14:02 +01:00
Leah Rowe f03efbc27f blobs/inject: add error condition on rm command
this was overlooked, earlier on in lbmk audit 2

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 20:15:50 +01:00
Leah Rowe 20be007f5b blobs/inject: fix checksum validation if no-ucode
on e6400_4mb, the release build scripts remove nvidia's vga
rom which is used on dgpu models. however, microcode is also
removed in separately copied rom images

the inject script was inserting vgaroms directly into these
no-microcode roms, but the microcode blob is bigger than the
vga rom, and cbfstool inserts into the first available free
spot within cbfs, so it was inserting into the spot where
cpu microcode went. this caused the rom checksum to not match
what was generated during build/release/roms being executed

the only real fix is to guarantee offsets within cbfs for all
files, by recording what offsets were used and then calculating
that during insertion

so this patch is a workaround, but fixes the issue. the workaround
is: don't insert blobs directly on no-microcode roms, instead
insert only on microcode-based roms, then re-copy those roms
and remove microcode in aptly named copies

it's a bit more convoluted, but works perfectly fine.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 20:11:11 +01:00
Leah Rowe 878550d519 use sha512sum to check downloads, not sha1sum
sha-1 has known collision issues, which may not be readily
exploitable yet (in our context), but we should ideally use
a more secure method for checking file integrity.

therefore, use sha-2 (sha512sum) for checking files. this is
slower than sha-1, but checksum verification is only a minor
part of what lbmk does, so the overall effect on build times
is quite negligible.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09 16:39:26 +01:00
Riku Viitanen 1bde6bb3c4 Support multiple dependencies per project
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-08 20:43:53 +03:00
Riku Viitanen 4d3b16da38 Cleaner parent directory creation
My previous patch b0rked memtest and others because when making sure
their parent directory (the project root) exists, it would instead create
the project directory (memtest86lus). The later move would then put the
git repo inside that (memtest86plus/memtest86plus_123456).

We just need to make sure we don't create the target directory itself.
This way, there's no need to hardcode any project names.

Tested by ./updating rpi-pico-serprog, memtest86plus, grub and seabios.

Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-08 00:25:58 +03:00
Leah Rowe 8fc5f6ed53 update/blobs/inject: use tmp/romdir, not TMPDIR
we are copying large numbers of ROM images, and the
host system may have /tmp under a tmpfs; that same
host system may or may not have a lot of memory.

respect the user's machine.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-05 01:49:35 +01:00
Leah Rowe 0d315c3a4e curl/wget downloads: set re-try count to 3
explicitly set the count to 3, so that a maximum of 3
attemps are made per download, barring fatal errors such
as http 404.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 22:12:02 +01:00
Leah Rowe bdf171e3ec don't use /tmp/ for git clones. use tmp/ instead.
the /tmp/ file system may be a tmpfs, with conservative
memory limits, depending on host system.

it's more likely that the user will have enough disk space
under tmp/ within lbmk (if they don't, they can't use
lbmk anyway). that is to say: more likely that they would
have the disk space, but not the memory.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 21:45:29 +01:00
Leah Rowe a56cad71c0 update/blobs: unify global variables
they all more or less use the same variables, so put
them all under include/blobutil.sh

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 18:12:49 +01:00
Leah Rowe 2cbc7eea95 update/blobs/*: unify checking of defconfig files
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 16:16:53 +01:00
Leah Rowe 52677309c5 update/blobs/extract: replace errant target code
check based on whether defconfigs are available, which
are used extensively, rather than checking based on
whether target.cfg is available, which is not used

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 15:57:48 +01:00
Leah Rowe c62a423909 update/blobs/inject: remove errant target handling
just like the last patch, target.cfg handling is not
required in this script either. remove it.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 14:17:09 +01:00
Leah Rowe 950166da7b update/blobs/download: remove errant code
the handling of target.cfg is *not* required, in
this script. other mechanisms are also used for
error checking. this script only uses defconfigs.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 13:44:07 +01:00
Leah Rowe 2c769dc136 move me7_update_parser.py to util/
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 03:24:46 +01:00
Leah Rowe da3c9bb3c5 merge config/ and resources/
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 02:47:25 +01:00
Leah Rowe a05010503f blobs/download: don't handle ifd/gbe files
they weren't even handled at all, but they were referenced
under coreboot configuration

they don't need to be handled. lbmk simply includes these files.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 02:12:05 +01:00
Leah Rowe 03788d14fb move ifd/gbe configs into config/ifd/
it doesn't really make sense for them to be under
blobs/ - nominally, they are blobs, but they are
well-understood data files containing config data,
that is easily parsed by tools like ich9show or
ifdtool (and tools like bincfg or nvmutil)

blobs/ has been re-purposed: this directory no longer
exists in lbmk, but it is created (and on .gitignore)
when needed, by blobutil

thus, the blobs/ directory shall only contain vendor
files, and only those files that libreboot scrubs from
releases. therefore, build/release/src can (and has
been) simplified; it currently copies just the ifd and
gbe files from blobs/, selectively, and this logic is
quite error prone, requiring maintenance. now, the
build/release/src script simply copies config/ (which
only ever contains distributable files) and entirely
ignores the blobs/ directory

the blob download script already creates the required
directory, except for the sch5545 download; this is
now fixed

lbmk code size is slightly smaller, due to this patch

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04 01:38:08 +01:00
Leah Rowe c63052cf19 fix memtest86plus download/build
the mkdir command in update/project/repo, added for
pico-pi integration, broke a bunch of other downloads.

the fix is a bit of a hack but it should hold for now.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-03 17:28:10 +01:00
Riku Viitanen e369e8fb4a automate rpi-pico serprog builds
./build rpi-pico serprog
build bin/serprog/rpi-pico-serprog.uf2

Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-03 04:00:42 +03:00
Leah Rowe b30c7e330b coreboot/e6400: support nvidia models
The same ROM images that you flash on Intel GPU variants,
are now flashed on Nvidia models. The same ROM will work
on both. If an Intel GPU variant is present, libgfxinit
is used, and the VGA ROM is used if an Nvidia GPU variant;
however, release ROMs will scrub the nvidia option ROM,
so release ROMs will only work on Intel GPUs unless you
run the blobutil inject command.

I decided to no longer have this under WIP, but to put
it in master. The issue with it pertains to video drivers,
which is not Libreboot's problem.

Nouveau crashes under Linux, so use "nomodeset" if it does.
The "nv" drivers in BSD systems work very well.

The nvidia model of E6400 isn't recommended for other
reasons, namely: poor thermal cooling (thermal pad on
the GPU) and that Nvidia GPU doesn't get very good
performance on any libre drivers anyway. The Intel GPU
variant is better, in terms of power efficiency and
software support; the intel variant also works with
native graphics initialisation in coreboot.

This board port already only enables SeaBIOS, which will
simply execute the VGA ROM. Blobutil already supports
reading the config, detecting that a VGA ROM is needed,
because that part of the WIP E6400 branch was already
merged in lbmk master.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-02 17:40:49 +01:00
Leah Rowe 0e782e7ee5 update the fetch scripts themselves
the fetch scripts call themselves, for dependencies

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 09:38:24 +01:00
Leah Rowe b9662fbe36 handle project downloads in main lbmk script
this means the unified /tmp handling is now provided for
in both the former "fetch" and "fetch_trees" script, which
are now (respectively):

./update project repo
./update project trees

if the fetch scripts weren't cleaning /tmp before, they
now are, because lbmk handles it

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-01 08:30:08 +01:00
Leah Rowe fbda0f04db re-add /dev/null redirect on . ${1}
the error messages that it shows are benign, but users
see them and worry that something went wrong

this patch reduces the number of people asking pointless
questions on irc

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-31 17:16:54 +01:00
Leah Rowe 355eb765ff move resources/scripts/ to script/
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-27 17:19:36 +01:00