% Libreboot Build System Audit 2 % Leah Rowe % 11 September 2023 Introduction ============ This article pertains to the current Libreboot (lbmk) revision at this time of writing, which is `c400916e33b8870384c30b83a9ecb9cdf7121917` from 11 September 2023. A lot of work has gone into Libreboot since the [20230625 release](libreboot20230625.md), the main focus of which has been yet another audit; the [previous audit](audit.md) was mild by comparison. Great care is always taken when writing for [lbmk](../docs/maintain/), which is Libreboot's build system, but in the past, auditing was always done ad-hoc. Since the start of 2023, auditing is now a main aspect of Libreboot development, and the result has been that the *quality* of Libreboot's build system has improved greatly. The same care has been given to documentation aswell. Many bugs have been fixed, and the Libreboot build system (lbmk) is generally much more efficient now, compared to the Libreboot 20230625 release. The main focus has been this audit, now concluded, and the next focus shall once again be adding more mainboards to Libreboot, with a view to making a full new release some time during September 2023. Before diving into details, here is a brief summary of the recent audit, and the changes made under it, which accounts for almost *200* individual changes. This is the *summarised* list, and may not list all changes, but it's pretty exhaustive: * Very general auditing has been done, finding and fixing bugs. * Reduced the number of scripts significantly. There were about 50 scripts in the Libreboot 20230625 build system. There are closer to *20* in today's Libreboot revision. * *Massively reduced the size of the build system*: where only shell scripts are concerned, the total sloccount in Libreboot 20230625 was 3388 source lines. As of *today*, that figure stands at 2644 source lines, a *22% reduction in code size*. * Many scripts that were separate are now unified. For example: the scripts handling defconfigs files on SeaBIOS, u-Boot and coreboot have now been merged into a single script, performing the same work *better* in less code. * Ditto many other scripts; repeated logic unified, logic generalised. The logic for *downloading* coreboot and u-boot was unified into one script, basing off of the coreboot one, and then expanding to also cover SeaBIOS. Most building (e.g. handling of Makefiles) is now done in a single script. * Far superior error handling; in many scripts, the `-e` option in `sh` was heavily relied upon to catch errors, but now errors are handled much more verbosely. *Many* fault conditions previously did not make lbmk *exit* at all, let alone with non-zero status, and zero status was sometimes being returned under some edge cases that were tested. Error handling is more robust now. * `util/ich9utils` (containing `ich9gen`) was *removed*, thus eliminating about 3000 source lines (of C code) from lbmk. The `nvmutil` program, also provided by and originating from the Libreboot project, can already change GbE MAC addresses. Coreboot's bincfg can generate ich9m descriptors, and ifdtool can manipulate them; so the features provided by ich9utils were superfluous, since they are available in other projects that we ship. We now ship pre-built ifd/gbe configs on these machines, which can be modified or re-assembled manually if you want to. This eliminates a moving part from Libreboot, and speeds up the build a little bit. * ROM images (of coreboot) build *much faster*: no-payload coreboot ROMs are cached on disk, as are payloads, where previously only the latter was cached. These cached images have as much inserted into them as possible, to eliminate redundant steps in the build process. The `elf` directory contains these, and the existing `bin` directory still holds the full ROM images (containing payloads) when compiled. * GRUB payload: vastly reduced the size of the payload, by eliminating GRUB modules that were not needed. About 100KB of compressed space saved in flash! * GRUB payload: [argon2 key derivation supported](argon2.md) - this means LUKS2 decryption is now possible in GRUB. This work was performed by Nicholas Johnson, rebasing from Axel's AUR patch for GRUB 2.06 (Libreboot currently uses GRUB 2.12). * Blobutil: generally more reliable now at downloading vendor files, especially under fault conditions; for example, if a download failed before, it'd try a backup link, but now it also tries the backup link if main download succeeds but checksum verification didn't; and SHA512 checksums are now used, for greater security, whereas Libreboot 20230625 used sha1sum (now we use sha512sum). A user agent is specified in wegt, matching that used by Tor Browser (which in turn mimics Firefox running on Windows). This is needed for some vendors, which seem to dislike wget's default user agent. * The *new* coding style is now used on many more scripts, including the `build/boot/roms_helper` script - the new style is much cleaner, mandating that logic be top-down, with a `main()` function defined; it's basically inspired by the OpenBSD coding style for C programs, adapted to shell scripts. * All GRUB keymaps now included; a single `grub.elf` is now used on all ROM images. The `grub.cfg` goes in GRUB memdisk now, but can be overridden by inserting a `grub.cfg` in CBFS; many behaviours are also controlled this way, for example to change keymaps and other behaviours. This results in *much* faster builds, because a different GRUB payload doesn't have to be added to each new ROM image; such takes time, due to time-expensive LZMA compression. This, plus the optimised set of GRUB modules, also makes GRUB itself load much faster. All of the fat has been trimmed, though still quite a lot more than a Crumb. * A lot of scripts have been removed entirely, and their logic not replaced; in many cases, Libreboot's build system contained logic that had gone unused for many years. * More reliable configs now used on desktop mainboards: SeaBIOS-only for start, but GRUB still available where feasible (in the SeaBIOS menu). This makes it more fool proof for a user who might use integrated graphics and then switch to a graphics card; the very same images will work. * TMPDIR environmental variable now set, and exported from main parent process when running lbmk; child processes inherit it, and a single tmp dir is used. This is then automatically cleaned, upon exit from lbmk; previously, lbmk did not cleanly handle `/tmp` at all, but now it's pretty reliable. Several other features have been added too, despite the overall code size reduction; for example, Riku Viitanen implemented support for building RP2040 images e.g. Pico Pi, and stm32-vserprog images, for use in small MCU-based SPI flashers (via serprog protocol). FULL list of changes (from lbmk git log) --------------------------------------- Almost all of the changes are post-20230625 release, and almost all of them are audit-related, so I've simply pasted every commit between the last release and now. A few of them, e.g. the new mainboard ports, are not audited-related. Entries that give time frames such as *"14 hours ago"* are relative to today, 11 September 2023, at some time around 3PM UK time. Changes on top are newer, and changes further down are older: ``` * c400916e - (HEAD -> master) coreboot/hp8200sff_4mb: fix bad ifd path in config (14 hours ago) * 087f0e06 - make lbmk help text actually vaguely helpful (19 hours ago) * 093d40ee - build/release/src: be more thorough deleting .git (21 hours ago) * 630a6546 - build/release/src: delete elf/ in srcdir (21 hours ago) * 0543350d - handle/make/file: run make-clean first (21 hours ago) * 12f9afe6 - build/release/src: remove cbutils/ in srcdir (21 hours ago) * fe00ab4e - build/release/src: remove errant code (21 hours ago) * be4ed540 - handle/make/config: distclean once per tree (21 hours ago) * f227cc08 - handle/make/config: fix distclean/crossgcc-clean (22 hours ago) * 669c9770 - handle/make/config: fix whitespace and 80-line bug (23 hours ago) * d28ad6aa - build/release/roms: use -T0 on serprog tarballs (23 hours ago) * 308c21dd - build/boot/roms stragglers: properly handle errors (23 hours ago) * c16b28ef - build/release/src: re-create symlinks, don't copy (2 days ago) * 32dcf9e5 - coreboot/qemu_x86_12mb: re-add this mainboard (2 days ago) * 5aef8156 - scripts: use printf, not echo, where appropriate (2 days ago) * 76e12cd4 - update/blobs printf statements: use double quotes (2 days ago) * 84bf47b5 - scripts: better handling of printf: stdout/stderr (2 days ago) * b78009e2 - checkgit: properly print output to stderr (2 days ago) * f45f5e62 - update/project/*: remove redundant checks (2 days ago) * 3e76e70d - blobs/download: don't use the -B option in make (2 days ago) * 877c691e - build/release/roms: remove errant line break (2 days ago) * f03efbc2 - blobs/inject: add error condition on rm command (2 days ago) * 20be007f - blobs/inject: fix checksum validation if no-ucode (2 days ago) * f989d5b4 - blobs/sources: fix backup links on some files (2 days ago) * 878550d5 - use sha512sum to check downloads, not sha1sum (2 days ago) * 022e0200 - Merge pull request 'Add stm32-vserprog' (#129) from Riku_V/lbmk:stm32 into master (2 days ago) |\ | * bed444ff - Add stm32-vserprog (2 days ago) * | e9e4ada5 - build/boot/rom: only insert scan.cfg if needed (3 days ago) * | 0e3f3efc - build/boot/roms: delete tmpcfg when done (3 days ago) * | a69e8548 - set grub.cfg timeout to 5s (10 on some boards) (3 days ago) * | 4a459b02 - Merge pull request 'pico-serprog improvements' (#128) from Riku_V/lbmk:master into master (3 days ago) |\| | * 7b6fb958 - Build pico-serprog binary release archive (3 days ago) | * c292e01b - Build for all pico board, not just the "original" (3 days ago) | * 1bde6bb3 - Support multiple dependencies per project (3 days ago) | * 4d3b16da - Cleaner parent directory creation (4 days ago) |/ * 7e8465be - grub: re-add module: play (5 days ago) * e3b9dfc9 - util/nvmutil: put code all in nvmutil.c (6 days ago) * 8fc5f6ed - update/blobs/inject: use tmp/romdir, not TMPDIR (7 days ago) * da991262 - build/release/roms: use tmp/romdir, not TMPDIR (7 days ago) * 15081ed9 - grub: make backgrounds configurable in target.cfg (7 days ago) * 0d315c3a - curl/wget downloads: set re-try count to 3 (7 days ago) * bdf171e3 - don't use /tmp/ for git clones. use tmp/ instead. (7 days ago) * 196f293a - build/release/roms: fix ucode handling (7 days ago) * c0c7f3ae - build/release/roms: simplify defcongic handling (7 days ago) * a56cad71 - update/blobs: unify global variables (7 days ago) * 2cbc7eea - update/blobs/*: unify checking of defconfig files (7 days ago) * 52677309 - update/blobs/extract: replace errant target code (7 days ago) * ea7fae97 - build/boot/roms: don't create empty bin/ directory (7 days ago) * c62a4239 - update/blobs/inject: remove errant target handling (7 days ago) * 950166da - update/blobs/download: remove errant code (7 days ago) * 0668d234 - add checkversion to build/release/src (7 days ago) * c92a596c - grub: remove xnu module (7 days ago) * e659ddd8 - grub: remove legacy file system modules (7 days ago) * cf535785 - re-add grub modules cat, eval and pbkdf2 (7 days ago) * 33e6088a - move script/misc/versioncheck to main directory (7 days ago) * 2c769dc1 - move me7_update_parser.py to util/ (7 days ago) * da3c9bb3 - merge config/ and resources/ (8 days ago) * a0501050 - blobs/download: don't handle ifd/gbe files (8 days ago) * 03788d14 - move ifd/gbe configs into config/ifd/ (8 days ago) * 6ddb0e09 - run make oldconfig on coreboot/default mainboards (8 days ago) * 19efdf9e - ich9m mainboards: use pre-assembled ifd/gbe files (8 days ago) * af8d8cda - add ich9m ifd/gbe files (8 days ago) * d554efae - build/release/src: copy e6430 ifd/gbe (8 days ago) * 09aae7be - build/rpi-pico-serprog: better error handling (8 days ago) * 1dc54608 - fix rpi-pico builds when running it twice (8 days ago) * c63052cf - fix memtest86plus download/build (8 days ago) * fb4e6834 - Merge pull request 'Add Dell Latitude E6430' (#124) from nic3-14159/lbmk:e6430 into master (8 days ago) |\ | * ebc04e52 - Add Dell Latitude E6430 (8 days ago) * | 71d361aa - Merge pull request 'Less cat abuse' (#123) from Riku_V/lbmk:cat into master (8 days ago) |\ \ | * | ef3fb05d - Less cat abuse (8 days ago) * | | eebf7133 - switch repo links for pico-serprog (8 days ago) | |/ |/| * | 9ef8a7ea - Merge pull request 'Automate pico-serprog builds' (#122) from Riku_V/lbmk:mkserprog into master (9 days ago) |\| | * e369e8fb - automate rpi-pico serprog builds (9 days ago) |/ * 92b4db69 - build/release/src: only clean kbc1126 if it exists (9 days ago) * 7c6b35cf - unify build/clean scripts: use handle/make instead (9 days ago) * cec37747 - build/release/*: use -T0 in xz, for multithreading (9 days ago) * b4b63adb - don't support ucode removal on untested targets (9 days ago) * b30c7e33 - coreboot/e6400: support nvidia models (9 days ago) * 436b2ccb - handle/make/config -m/-u: actually copy configs (9 days ago) * 3c7b09ac - handle/make/config: properly handle cbutils (9 days ago) * a3bc7ccd - handle/make/file: fix uefitool builds (9 days ago) * 4885c796 - handle TMPDIR from include/export.sh (9 days ago) * 56f16bc8 - don't do cmake on uefitool if the Makefile exists (10 days ago) * 98d1ea5a - build/release/src: bugfix: actually copy cb/ub/sb (10 days ago) * 755f925a - build/release/src: copy handle symlink (10 days ago) * 3ad29d2d - build/release/src: remove Makefile reference (10 days ago) * d69c231e - build/release/src: fix bad variable reference (10 days ago) * 38440153 - update build/release/src based on lbmk changes (10 days ago) * 0e782e7e - update the fetch scripts themselves (10 days ago) * 98f30b6d - build/coreboot/utils: exit 1 if target.cfg missing (10 days ago) * b9662fbe - handle project downloads in main lbmk script (10 days ago) * 12b33eb8 - lbmk script: always clean up /tmp files (11 days ago) * 225e2609 - only remove microcode in build/release/roms (11 days ago) * bf774acf - move build/boot/rom moverom to handle/make/config (11 days ago) * e5546128 - build/release/roms: fix syntax error (11 days ago) * fbda0f04 - re-add /dev/null redirect on . ${1} (11 days ago) * b2bad5a0 - build/release/src: copy the include/ directory (13 days ago) * eb54e427 - grub: all one grub.elf containing keymaps and cfg (2 weeks ago) * c6fd4d2a - lbmk: run ./build dependencies *before* root check (2 weeks ago) * 6722624d - build/boot/roms: fix bad variable assignment (2 weeks ago) * 55be6dda - dependencies/ubuntu2004: update based on debian (2 weeks ago) * 0052f9d0 - fix: don't require git config for dependencies (2 weeks ago) * 6dbddf85 - build/boot/roms: simplify ich9m ifd handling (2 weeks ago) * f5787c9e - build/boot/roms ich9m ifd: use fast dd command (2 weeks ago) * d9292cec - build/boot/roms: use the new coding style (2 weeks ago) * 4623f3f2 - Remove superfluous GRUB modules (save CBFS space) (2 weeks ago) * 623c3389 - fix typo in error message ("as not permitted") (2 weeks ago) * 4a280c62 - .gitcheck: re-write entirely. force global config. (2 weeks ago) * 355eb765 - move resources/scripts/ to script/ (2 weeks ago) * eed34d3e - enable memtest86plus on various boards (2 weeks ago) * bc0fb51d - x86 desktops: only enable seabios_withgrub (2 weeks ago) * 9457d6be - unified list command for all scripts (2 weeks ago) * 93d2dcad - handle/make/config: add missing pipes for err (2 weeks ago) * 0e6851c8 - delete the Makefile (2 weeks ago) * ebbefa60 - handle/config/file: rename to handle/make/config (2 weeks ago) * df6db1c6 - handle/config: fix errant "handle src for" call (2 weeks ago) * 6874bc39 - "handle src for" - change to handle make file (2 weeks ago) * 798ce03a - handle/config: add missing error handle (2 weeks ago) * 29a8193e - build/src/for: rename to handle/make/file (2 weeks ago) * 27c67295 - handle/config/file: unified distclean handling (2 weeks ago) * 197464bc - build/src/for: use -j for multithreaded builds (2 weeks ago) * 95f290d9 - build/release/src: update based on recent changes (2 weeks ago) * 5a47c01b - scripts: put quotes around file/directory names (2 weeks ago) * 1c8401be - much, much stricter, more verbose error handling (2 weeks ago) * 50c395df - .gitcheck: continue if no .git (don't break) (3 weeks ago) * be7a5b0c - .gitcheck: must stricter error handling (3 weeks ago) * 3a5ba57f - .gitcheck: only redirect stdout to /dev/null (3 weeks ago) * 8f4f0e00 - use the new coding style in scripts (3 weeks ago) * 4c6c7d10 - scripts: never exit 1, always call err instead (3 weeks ago) * 52f3fd35 - blobs/download: copy dl_path, don't move it (3 weeks ago) * 57adbc6e - unify err functions across scripts (3 weeks ago) * b3fbcdf6 - .gitignore: ignore *all* seen files (3 weeks ago) * 24f09335 - Merge pull request 'hp8300usdt: enable mSATA' (#118) from Riku_V/lbmk:master into master (3 weeks ago) |\ | * df1e8913 - hp8300usdt: enable mSATA (3 weeks ago) * | dfb93166 - Merge pull request 'memtest86+ v6.20' (#116) from Riku_V/lbmk:memtest into master (3 weeks ago) |\| | * fa926632 - memtest86+ v6.20 (3 weeks ago) |/ * 1bd84209 - Merge pull request 'osbmk->lbmk' (#117) from Riku_V/lbmk:osbmk-lbmk into master (3 weeks ago) |\ | * 7be203dd - osbmk->lbmk (3 weeks ago) * | 04ee2672 - also clean up the main scripts (3 weeks ago) * | 62f23123 - general code cleanup on lbmk shell scripts (3 weeks ago) |/ * 7be47065 - unify build/defconfig and modify/defconfig (3 weeks ago) * 0faf2a0c - main lbmk script: exit non-zero if argc is wrong (3 weeks ago) * 6e92d9a3 - fix "./build help" (3 weeks ago) * 9031bb7b - unify dependencies scripts (3 weeks ago) * 023d6b69 - unify build/clean into ./build release src (3 weeks ago) * f893a29b - unify most module build scripts (3 weeks ago) * c83d1a8d - unify grub scripts under one directory (3 weeks ago) * 438bf2c9 - grub/modules.list: add argon2 (3 weeks ago) * fd602532 - grub: import phc argon2 implementation (for luks2) (3 weeks ago) * 2c0c521e - bump grub revision a bit (3 weeks ago) * e076d893 - unify update/modify coreboot/u-boot/seabios script (3 weeks ago) * e25984d7 - remove board: qemu_x86_12mb (not usable for now) (3 weeks ago) * e5b898f6 - consolidate u-boot/seabios/coreboot build scripts (3 weeks ago) * 673b144a - coreboot/fam15h: fix for gcc/gnat building (4 weeks ago) * 63b0e99f - don't call blobutil directly from lbmk (4 weeks ago) * 08486227 - remove download scripts, consolidate into script (4 weeks ago) * 8459e33b - improve user feedback in blobutil (4 weeks ago) * 59dba6cf - merge coreboot/u-boot download logic to one script (4 weeks ago) * 2453c303 - gitclone: always clean up /tmp (4 weeks ago) * adeb065c - fix permissions on arch dependencies script (4 weeks ago) * 6075fed8 - NEW BOARD: HP EliteBook 8470p (Intel GPU) (4 weeks ago) * f9afeb6f - NEW BOARD: Dell Precision T1650 (4 weeks ago) * f8f77cb2 - NEW BOARD: HP EliteBook 2170p (5 weeks ago) * c5c89467 - Merge pull request 'Update 'README.md'' (#89) from ewpr5kwu/lbmk:master into master (5 weeks ago) |\ | * 5204f0a9 - Update 'README.md' (2 months ago) * cb8bf380 - bump seabios revision to 30 May 2023 (5 weeks ago) * 27ee975e - bump grub revision to 2.12-rc1 (5 weeks ago) * 705149a3 - coreboot/default: bump revision to 2 August 2023 (5 weeks ago) * 22ee7f74 - blobs/download: save ME file to correct location (5 weeks ago) * cdd83ab1 - blobs/download: try backup if bad hash on main (5 weeks ago) * f18b1859 - blobs/download: support more formats on ME extract (5 weeks ago) * f0efaf79 - add unar to dependencies scripts (5 weeks ago) * e8ba0f87 - blobs/download: declare full user agent (6 weeks ago) * 4875eef1 - blobs/download: properly handle backup/main url (6 weeks ago) * cca93ca3 - blobs/download: don't download backup on main (6 weeks ago) * 3aeefaa7 - blobs/download: set common user agent string (6 weeks ago) * 5e83d2bc - blobs/download: simplify for loop (6 weeks ago) * 8f1d3ad1 - scripts: fix indentation in switch/case blocks (6 weeks ago) * 748e0972 - blobutil/ec: abort if kbc1126 ec extraction fails (7 weeks ago) * e594ac16 - coreboot/fam15h: remove unused files (7 weeks ago) * 44bd077a - Revert "build/boot/roms mkUBootRoms: initialise variables" (8 weeks ago) * 7c90a407 - build/boot/roms mkUBootRoms: initialise variables (8 weeks ago) * d918139f - coreboot/fam15h: re-enable microcode updates (8 weeks ago) * 8c777428 - board/qemu_x86: don't enable u-boot (8 weeks ago) * fb44c349 - coreboot/haswell: fix acpica downloads (8 weeks ago) * af084014 - coreboot: re-add asus kgpe-d16/kcma-d8/kfsn4-dre (8 weeks ago) * e6002b91 - coreboot/cros: fix acpica downloads (8 weeks ago) * f34e07ae - build/boot/roms: fix coreboot-version in releases (9 weeks ago) ``` A new Libreboot release is planned soon.