diff --git a/site/news/MANIFEST b/site/news/MANIFEST index 189b055..4bd050e 100644 --- a/site/news/MANIFEST +++ b/site/news/MANIFEST @@ -1,3 +1,4 @@ +canoeboot0.1.md canoeboot20231107.md canoeboot20231103.md canoeboot20231101.md diff --git a/site/news/canoeboot0.1.md b/site/news/canoeboot0.1.md new file mode 100644 index 0000000..9a5b570 --- /dev/null +++ b/site/news/canoeboot0.1.md @@ -0,0 +1,911 @@ +% Canoeboot v0.1 released! +% Leah Rowe in Canoe Leah Mode™ +% 27 January 2024 + +This is a special release. It is *not* based on the recent +[Libreboot 20240126 release](https://libreboot.org/news/libreboot20240126.md). +No, it is very special indeed. + +I have recently sent GNU Boot some patches for their 0.1 RC3 release, fixing +it so that it compiles on modern distros. Their release only compiled on really +old distros like Debian 10 or Trisquel 10. I made it compile on the latest +Gentoo, Arch and Debian(Sid) as of 14 January 2024. I also added Dell Latitude +E6400, gru\_bob and gru\_kevin chromebooks. I also added several initialisation +fixes for keyboards in their GRUB payload, in addition to EFI System Partition +support in their `grub.cfg` - in other words, I've backported several fixes +and improvements from Canoeboot and Libreboot, to their project. + +I did this, purely for fun, to see if it was technically feasible. And it was. +I sent these patches and they are now under review by the GNU people. + +As you may know from reading this Canoeboot website, Canoeboot is vastly more +up to date than GNU Boot, using revisions 2 years newer (from 2023), whereas +GNU Boot uses old 2021 coreboot, GRUB and SeaBIOS revisions. It does not contain +improvements such as GRUB argon2 support. + +Well, purely for fun, I made this special Canoeboot v0.1 release, which re-uses +the same *old* 2021 revisions as GNU Boot 0.1 RC3, but with my special fixes +as mentioned above (so, it has E6400/gru\_bob/gru\_kevin, and builds on modern +distros). However, that release is compiled using Canoeboot's build system, +which is vastly more efficient than the GNU Boot one (about twice as fast, and +less error prone, due to optimisations made during the four Libreboot build +system audits of 2023). + +You can find the Canoeboot v0.1 release on the mirrors, alongside regular +releases. It should boot and work perfectly, albeit it on those very old code +revisions. It is advised that you still use the November 2023 Canoeboot +release, for the time being. A proper Canoeboot update, based on +Libreboot 20240126 (which uses Coreboot revisions from January 2024) will be +done at a date in the near future. + +Anyway, the fixes that I did were sent to the GNU Boot mailing list. Check +the `gnuboot-patches` mailing list archive from 16 January 2024. + +GNU Boot 0.1 RC3 fixes: + + +Canoeboot v0.1 branch: + + +I also did another GNU Boot branch for fun, that updates it to the +October 2023 revisions used in Libreboot/Canoeboot releases from November 2023: + \ +...these patches were also sent, but it seems they still prefer to use my +Libreboot 20220710 release. + +The GNU Boot 0.1 RC3 release is essentially Libreboot 20220710, with a few +minor changes, and Canoeboot v0.1 is essentially Libreboot 20220710 aswell, +but with *substantial* build system design changes (but the overall code +is identical, when analysing the binaries). + +PS: I use a new GPG signing key on Libreboot releases now. Check the Libreboot +download page for it. At the time of writing, the new key is not listed on +the Canoeboot page but I used that key. + +Introduction +============ + +*This* new release, Canoeboot 20231026, released today 26 October 2023, is +based on the [Libreboot 20231021](https://libreboot.org/news/libreboot20231021.html) +release, porting changes in it on top of +[nonGeNUine Boot 20230717](nongenuineboot20230717.md) as a base. The previous +release was nonGeNUine Boot 20230717, released on 17 July 2023; the project +named *nonGeNUine Boot* has been renamed to Canoeboot, in this release, which +is the first ever release under the name *Canoeboot*. + +Canoeboot provides boot firmware for supported x86/ARM machines, starting a +bootloader that then loads your operating system. It replaces proprietary +BIOS/UEFI firmware on x86 machines, and provides an *improved* configuration +on [ARM-based chromebooks](../docs/install/chromebooks.html) supported +(U-Boot bootloader, instead of Google's depthcharge bootloader). On x86 +machines, the GRUB and SeaBIOS coreboot +payloads are officially supported, provided in varying configurations per +machine. It provides an [automated build system](../docs/maintain/) for the +[configuration](../docs/build/) and [installation](../docs/install/) of coreboot +ROM images, making coreboot easier to use for non-technical people. You can find +the [list of supported hardware](../docs/hardware/) in Canoeboot documentation. + +Canoeboot's main benefit is *higher boot speed*, +[better](../docs/gnulinux/encryption.md) +[security](../docs/gnulinux/grub_hardening.md) and more +customisation options compared to most proprietary firmware. As a +[libre](https://writefreesoftware.org/learn) software project, the code can be +audited, and coreboot does +regularly audit code. The other main benefit is [*freedom* to study, adapt and +share the code](https://writefreesoftware.org/), a freedom denied by most boot +firmware, but not Canoeboot! Booting Linux/BSD is also [well](../docs/gnulinux/) +[supported](../docs/bsd/). + +Canoeboot is maintained in parallel with Libreboot, and by the same developer, +Leah Rowe, who maintains both projects; Canoeboot implements the [GNU Free +System Distribution Guideline](https://www.gnu.org/distros/free-system-distribution-guidelines.en.html) +as policy, whereas Libreboot implements its own [Binary Blob Reduction +Policy](https://libreboot.org/news/policy.html). + +Work done since last release +============================ + +No new mainboards have been added in Canoeboot 20231026, versus nonGeNUine +Boot 20230717, but a slew of build system enhancements and new features have +been ported from Libreboot. + +However, the *following* mainboards added in Libreboot 20231021 have *been +excluded* in this Canoeboot release, due to the GNU FSDG policy: HP +EliteBook 2170p, HP EliteBook 8470p, Dell Precision T1650 and Dell +Latitude E6430. + +GRUB LUKS2 now supported (with argon2 key derivation) +--------------------------------------------------- + +*This* new Canoeboot release imports the [PHC argon2 +implementation](https://github.com/P-H-C/phc-winner-argon2) into GRUB, +courtesy of [Axel](https://axelen.xyz/) who initially ported the code to run +under GRUB *2.06*, but this Canoeboot release uses GRUB *2.12* (an RC revision +from git, at present). + +Axel's code was published to [this AUR repository](https://aur.archlinux.org/cgit/aur.git/tree/?h=grub-improved-luks2-git&id=1c7932d90f1f62d0fd5485c5eb8ad79fa4c2f50d) +which [Nicholas Johnson](https://nicholasjohnson.ch/) then rebased on top of +GRUB *2.12*, and I then imported the work into Libreboot, with Johnson's +blessing; Canoeboot has inherited this work in full. + +These libreboot patches added argon2 support, and have been ported to Canoeboot +in this 20231026 release: + +* +* +* + +This means that you can now boot from encrypted `/boot` partitions. I'm very +grateful to everyone who made this possible! + +Simplified commands (build system) +------------------------- + +Simply put, cbmk (the Canoeboot build system) is now *easier to use* than +gbmk (the nonGeNUine Boot 20230717 build system) was; there +are only *9* shell scripts in this release, versus 50 or so in the +nonGeNU 20230717 release, and the command structure has been simplified. + +You can find information about *using* the build system in +the [Canoeboot build instructions](../docs/build/) and in the [cbmk +maintenance manual](../docs/maintain/). + +The Libreboot 20231021 release has *12* scripts, bacause there are 3 more +scripts there for handling the downloading of vendor code; since Canoeboot +intentionally avoids doing that, those scripts are not needed in Canoeboot +and have therefore been excluded. They are: `script/vendor/download`, +`script/vendor/inject` and `include/mrc.sh`. + +TWO massive audits. 50% code size reduction in lbmk. +-------------------------------------------- + +Libreboot's build system, lbmk, is written entirely in shell scripts. It is +an automatic build system that downloads, patches, configures and compiles +source trees such as coreboot and various payloads, to build complete ROM +images that are easier to install. More info about that is available in +the [lbmk maintenance manual](https://libreboot.org/docs/maintain/) - and you +can read the [cbmk maintenance manual](../docs/maintain/) for comparison. + +The primary focus of Libreboot 20231021 cultiminated in two *audits*, namely +[Libreboot Build System Audit 2](https://libreboot.org/news/audit2.html) and +then [Libreboot Build System Audit 3](https://libreboot.org/news/audit3.html). + +The changes in those audits have been ported to this *Canoeboot* release. + +Changes include things like vastly reduced code complexity (while not +sacrificing functionality), greater speed (at compiling, and boot speeds are +higher when you use the GRUB payload), many bug fixes and more. + +Serprog firmware building (RP2040 and STM32) +----------------------------------- + +In addition to coreboot firmware, the Canoeboot build system (lbmk) can now +build *serprog* firmware, specifically `pico-serprog` and `stm32-vserprog`, on +all devices that these projects support. + +The *serprog* protocol is supported by flashrom, to provide SPI flashing. It +can be used to set up an external SPI flasher, for [flashing Canoeboot +externally](../docs/install/spi.md). This too has been ported from Libreboot. + +Pre-compiled firmware images are available, for many of these devices, under +the `roms/` directory in this Canoeboot 20231026 release! Riku Viitanen is the +one who added this capability to Libreboot, which was then ported to Canoeboot. + +Updated U-Boot revision (2023.10) +---------------------------- + +Alper Nebi Yasak submitted patches that update the U-Boot revision in +Libreboot, on `gru_bob` and `gru_kevin` chromebooks. Additionally, the `cros` +coreboot tree has merged there with the `default` tree instead (and the `default` +tree has been updated to coreboot from 12 October 2023). + +Many improvements were made to these boards, which you can learn about by +reading these diffs: + +* +* +* +* +* +* +* +* +* +* +* +* +* +* + +All of these patches have been ported to this Canoeboot release. + +Coreboot, GRUB, U-Boot and SeaBIOS revisions +------------------------------------ + +In Canoeboot 20231026 (*this release*): + +* Coreboot (default): commit ID `d862695f5f432b5c78dada5f16c293a4c3f9fce6`, 12 October 2023 +* Coreboot (cros): MERGED WITH `coreboot/default` (see above) +* Coreboot (fam15h\_udimm): commit ID `1c13f8d85c7306213cd525308ee8973e5663a3f8`, 16 June 2021 +* GRUB: commit ID `e58b870ff926415e23fc386af41ff81b2f588763`, 3 October 2023 +* SeaBIOS: commit ID `1e1da7a963007d03a4e0e9a9e0ff17990bb1608d`, 24 August 2023 +* U-Boot: commit ID `4459ed60cb1e0562bc5b40405e2b4b9bbf766d57`, 2 October 2023 + +In nonGeNUine Boot 20230717 (*previous release*): + +* Coreboot (default): commit ID `e70bc423f9a2e1d13827f2703efe1f9c72549f20`, 17 February 2023 +* Coreboot (cros): commit ID `8da4bfe5b573f395057fbfb5a9d99b376e25c2a4` 2 June 2022 +* Coreboot (fam15h\_udimm): DID NOT EXIST +* GRUB: commit ID `f7564844f82b57078d601befadc438b5bc1fa01b`, 14 February 2023 +* SeaBIOS: commit ID `ea1b7a0733906b8425d948ae94fba63c32b1d425`, 20 January 2023 +* U-Boot (for coreboot/cros): commit ID `890233ca5569e5787d8407596a12b9fca80952bf`, 9 January 2023 + +As you can see, all revisions are quite new in this release. + +Build system tweaks +=================== + +resources/ now config/ +---------------------- + +The `resources/scripts/` directory is now `script/`, and what was `resources/` +now only contains configuration data plus code patches for various projects, +so it has been renamed to `config/` - I considered splitting patches +into `patch/`, but the current directory structure for patches is not a problem +so I left it alone. + +Also, the IFD/GbE files have been moved here, under `config/ifd/`. These can +always be ge-generated if the user wants to, using ich9gen, or using a +combination of bincfg and ifdtool from coreboot, and nvmutil (to change the +mac address) from Canoeboot or Libreboot. + +Full list of changes (detail) +-------------------- + +These changes have been ported from the Libreboot 20231021 release, which are +mostly the results of the two audits (mentioned above): + +* Much stricter, more robust error handling; too many changes to list here, so + check the git log. Also, errors that *are not errors* are no longer treated as + such; nonGeNUine Boot 20230717's build system was actually too strict, sometimes. +* Most logic has been unified in single scripts that perform once type of task + each, instead of multiple scripts performing the same type of talk; for + example, defconfig-based projects now handled with the same scripts, and + preparing trees for them is done the same. These unifications have been done + carefully and incrementally, with great thought so as to prevent *spaghetti*. + The code is clean, and small. +* GitHub is no longer used on main Git repository links, instead only as backup +* Backup repositories now defined, for all main repos under `config/git/` +* Single-tree projects are no longer needlessly re-downloaded when they already + have been downloaded. +* GRUB LUKS2 support now available, with argon2 key derivation; previously, only + PBKDF2 worked so most LUKS2 setups were unbootable in Canoeboot. This is fixed. +* Vastly reduced number of modules in GRUB, keeping only what is required. +* Use `--mtime` and option options in GNU Tar (if it is actually GNU Tar), when + creating Tar archives. This results in partially reproducible source archives, + and consistent hashes were seen in testing, but not between distros. +* Always re-inialitise `.git` within lbmk, for the build system itself, if + Git history was removed as in releases. This work around some build systems + like coreboot that use Git extensively, and are error-prone without it. +* More robust makefile handling in source trees; if one doesn't exist, error + out but also check other makefile name combinations, and only error out if + the command was to actually build. +* ROMs build script: support the "all" argument, even when getopt options are + used e.g. `-k` +* Disabled the pager in `grub.cfg`, because it causes trouble in some + non-interactive setups where the user sees an errant message on the screen + and has to press enter. This fixes boot interruptions in some cases, allowing + normal use of the machine. The pager was initially enabled many years ago, + to make use of cat a bit easier in the GRUB shell, but the user can just + enable the pager themselves if they really want to. +* U-Boot can now be compiled standalone, without using the ROMs build script, + because crossgcc handling is provided for U-Boot now in addition to coreboot. +* All helper scripts are now under `include/`, and main scripts in `script/`, + called by the main `build` script +* Generally purge unused variables in shell scripts +* Simplified initialisation of variables in shell scripts, using the `setvars` + function defined under `include/err.sh` +* Support patch subdirectories, when applying patches. This is done recursively, + making it possible to split up patch files into smaller sets inside sub + directories, per each source tree (or target of each source tree, where a + project is multi-tree within lbmk) +* SPDX license headers now used, almost universally, in all parts of cbmk. +* Files such as those under `config/git` are now + concatenated, traversing recursively through the target directory; files first, + then directories in order, and for each directory, follow the same pattern + until all files are concatenated. This same logic is also used for patches. + This now enables use of subdirectories, in some config/patch directories. +* General code cleanup on `util/nvmutil` +* Git histories are more thoroughly deleted, in third party source trees during + release time. +* Symlinks in release archives are no longer hard copies; the symlinks are + re-created by the release script, because it clones the current lbmk work + directory via Git (local git clone), rather than just using `cp` to copy links. +* Properly output to stderr, on printf commands in scripts where it is either + a warning prior to calling `err`, or just something that belongs on the error + output (instead of standard output). +* Don't use the `-B` option in make commands. +* SECURITY: Use sha512sum (not sha1sum) when verifying certain downloads. This + reduces the chance for collisions, during checksum verification. +* Set GRUB timout to 5s by default, but allow override and set to 10s or 15s + on some mainboards. +* Support both curl and wget, where files are downloaded outside of Git; defer + to Wget when Curl fails, and try each program three times before failing. This + results in more resilient downloading, on wobbly internet connections. +* Don't clone Git repositories into `/tmp`, because it might be a tmpfs with + little memory available; clone into `tmp/gitclone` instead, within lbmk, + and `mv` it to avoid unnecessary additional writes (`mv` is much more efficient + than `cp`, for this purpose). +* Removed unused `target.cfg` handling in vendor scripts, because they use + the concatenated config format instead (they always have). +* Coreboot builds: automatically run make-oldconfig, to mitigate use of raw + coreboot config where a revision was updated but the config was untouched. + This may still result in a confirmation dialog, and it's still recommended + that the configs be updated per revision (or switch them to defconfigs). +* Vastly simplified directory structure; `resources/scripts/` is now `script/`, + and `resources/` was renamed to `config/`; ifd and gbe files were also moved + to `config/ifd/`. Commands are now 1-argument instead of 2, for example + the `./build boot roms` command is now `./build roms`. +* memtest86plus: only build it on 64-bit hosts, for now (32-bit building is + broken on a lot of distros nowadays, and lbmk doesn't properly handle cross + compilation except on coreboot or U-Boot) +* (courtesy of Riku Viitanen) don't use cat on loops that handle lines of text. + Instead, use the `read` command that is built into `sh`, reading each line. + This is more efficient, and provides more robust handling on lines with + spaces in them. +* *ALL* projects now have submodules downloaded at build time, not just multi + tree projects such as coreboot - and a few projects under `config/git` have + had certain `depend` items removed, if a given project already defines it + under `.gitmodules` (within its repository). +* Improved cbutils handling; it's now even less likely to needlessly re-build + if it was already built. +* The release build script no longer archives what was already built, but + instead builds from scratch, creating an archive from source downloads + first before building the ROM archives. This saves time because it enables + a single build test per release, whereas at was previously necessary to test + the Git repository and then the release archive. Testing both is still desired, + but this behaviour also means that whatever is built at release time is + guaranteed to be the same as what the user would build (from archives). +* Improved handling of `target.cfg` files in multi-tree projects coreboot, + SeaBIOS and U-Boot. Unified to all such projects, under one script, and + with improved error handling. +* GRUB payload: all ROM images now contain the same ELF, with all keymaps + inserted. This speeds up the build process, and enables easier configuration + when changing the keyboard layout because less re-flashing is needed. +* Simplified IFD handling on ICH9M platforms (e.g. X200/T400 thinkpads); the + ich9gen utility wasn't needed anymore so ich9utils has been removed, and now + the IFD/GbE files are included pre-assembled (generated by ich9gen). Ich9gen + can still be used, or you can re-generate with coreboot's bincfg; the ifdtool + util can be used to edit IFD and nvmutil (part of Canoeboot) can change MAC + addresses. The ich9utils code was always redundant for the last few years, + especially since 2022 when nvmutil was first written. +* Running as root is now forbidden, for most commands; lbmk will exit with + non-zero status if you try. The `./build dependencies x` commands still work + as root (they're the only commands available as root). +* Enabled memtest86plus on more boards, where it wasn't previously enabled. +* Only enable SeaBIOS as first payload on desktops, but still enable GRUB as + second payload where GRUB is known to work (on each given host). The text + mode and coreboot framebuffer modes are provided in each case, where feasible. +* The `list` command has been mostly unified, making it easier to tell (from + lbmk) what commands are available, without having to manually poke around + under `script/`. +* The `-T0` flag is now used, universally, on xz commands. This makes `xz` run + on multiple threads, greatly speeding up the creation of large tar archives. +* Universally use `-j` in make commands, for multi-threading, but it relies + on `nproc` to get thread count, so this only works if you have `nproc` (you + probably don't, if you run BSD; BSD porting is still on TODO for Canoeboot) +* File names as arguments now universally have quotes wrapped around them, and + similar auditing has been done to all variables used as arguments everywhere + in lbmk. There were cases where multiple arguments were wrongly quoted then + treated as a single argument, and vice versa. This is now fixed. +* Re-wrote `.gitcheck`; now, a global git name/email config is always required. + The only behaviour (setting local config, and unsetting) was quite error-prone + under fault conditions, where cleanup may not have been provided, or when + execution was interrupted, resulting sometimes in accidentally committing + to `lbmk.git` as author named `lbmkplaceholder`. +* The new BSD-like coding style is now used on *all* shell scripts in lbmk. A + few scripts still used the old lbmk coding style, as of audit 2. +* Scripts no longer directly exit with non-zero status, under fault conditions; + instead, `x_` or `err` is used to provide such behaviour. This results in all + exits from lbmk being consolidated to `err`, under fault conditions. - zero + exits are also consolidated, going only through the main script, which has its + own exit function called `lbmk_exit` that provides `TMPDIR` cleanup. +* BSD-style error handling implemented, with an `err` function (and functions + that use it) inside `include/err.sh`; there is also `x_` which can be used + to run a command and exit automatically with non-zero status, useful because + it provides more verbose output than if you just relied on `set -e`, and it + still works when a script *does not* use `set -e` - however, it is not used + on all functions, because it works by executing `$@` directly, which can break + depending on arguments. Therefore, some scripts just default to `|| err` for + providing breakage in scripts. +* Memtest *6.2* now used (instead of *5.x* releases). This is essentially a + re-write, and it works on the coreboot framebuffer, whereas previous revisions + only worked on text mode setups. +* NO MAKEFILE. The Makefile in lbmk has been removed. It was never meaningfully + used because all it did was run lbmk commands, without implementing any logic + itself. A Makefile may be added again in the future, but with a view to + installing *just the build system* onto the host system, to then build ROM + images under any number of directories. Lbmk's design is strictly no-Makefile, + but it uses Makefiles provided by third party source trees when building them. +* Safer GRUB configuration file handling between GRUB memdisk and coreboot CBFS; + it is no longer possible to boot without a GRUB config, because the one in + GRUB memdisk is provided as a failsafe, overridden by *inserting* one in CBFS, + but there is no config in CBFS by default anymore. +* The build system *warns* users about `elf/` vs `bin/`, when it comes to + flashing coreboot ROM images; it tells them to use `bin/` because those + images do contain payloads, whereas the ones under `elf/` do not. +* VASTLY more efficient build process; all coreboot ROMs without payload are + now cached under `elf/`, as are payloads, then they are joined separately by + the usual ROMs build script, and these cached ROMs contain many changes in + them that were previously handled by `moverom` in the main ROM build script. + Under the new design, repetitive steps are avoided; payloads are inserted into + a copy of the cached ROMs under `TMPDIR`, *before* being copied for keymaps + and small files; this eliminates delays caused by slow compression (LZMA is + always used, when inserting payloads). After crossgcc and the payloads are + compiled, the ROM with coreboot builds in under a minute, whereas it would + have previously taken several minutes on most Canoeboot-supported hardware. +* VASTLY reduced GRUB payload size; modules that aren't needed have been removed + resulting in much smaller GRUB payloads, that also boot faster. +* ALL defconfig creation, updating and modification are handled by the same + script that *also* handles compiling, as mentioned in the bullet-point below. +* ALL main source trees are now compiled, downloaded, configured and cleaned + using the same script. The *download* (Git) logic is a separate file + under `include/` and its functions are called by the main build script, which + provides a stub for this. +* Scripts are no longer executed directly, ever, except the main script. All + scripts are otherwise executed from `script/`, inheriting the `TMPDIR` + variable set (and exported) by lbmk. +* Generally improved user feedback in scripts, especially the vendor scripts. +* Coreboot, U-Boot and SeaBIOS are now downloaded, configured and compiled using + the exact same script. Although these codebases differ wildly, their build + systems use the same design, and they are compatible from a user-interface + perspective. +* Vastly improved `/tmp` handling; a universal `TMPDIR` is set (environmental + variable) and exported to all child processes running lbmk scripts. On exit, + the main tmp directory is purged, cleaning all tmp directories under it. +* General simplification of coding style on all shell scripts. +* Fixed some variable initialisations in the coreboot ROM image build script +* Don't enable u-boot on QEMU x86 images (due to buggy builds, untested) +* Fixed coreboot-version file inserted into coreboot trees, when compiled + on Canoeboot release archives. +* Very general auditing has been done, finding and fixing bugs. +* Reduced the number of scripts significantly. There were about 50 scripts in + the nonGeNUine Boot 20230717 build system. There are closer to *20* in today's + Canoeboot 20231026 revision. +* Many scripts that were separate are now unified. For example: the scripts + handling defconfigs files on SeaBIOS, u-Boot and coreboot have now been + merged into a single script, performing the same work *better* in less code. +* Ditto many other scripts; repeated logic unified, logic generalised. The + logic for *downloading* coreboot and u-boot was unified into one script, + basing off of the coreboot one, and then expanding to also cover SeaBIOS. + Most building (e.g. handling of Makefiles) is now done in a single script. +* Far superior error handling; in many scripts, the `-e` option in `sh` was + heavily relied upon to catch errors, but now errors are handled much more + verbosely. *Many* fault conditions previously did not make lbmk *exit* at all, + let alone with non-zero status, and zero status was sometimes being returned + under some edge cases that were tested. Error handling is more robust now. +* `util/ich9utils` (containing `ich9gen`) was *removed*, thus eliminating about + 3000 source lines (of C code) from lbmk. The `nvmutil` program, also provided + by and originating from the Canoeboot project, can already change GbE MAC + addresses. Coreboot's bincfg can generate ich9m descriptors, and ifdtool can + manipulate them; so the features provided by ich9utils were superfluous, since + they are available in other projects that we ship. We now ship pre-built + ifd/gbe configs on these machines, which can be modified or re-assembled + manually if you want to. This eliminates a moving part from Canoeboot, and + speeds up the build a little bit. +* ROM images (of coreboot) build *much faster*: no-payload coreboot ROMs are + cached on disk, as are payloads, where previously only the latter was cached. + These cached images have as much inserted into them as possible, to eliminate + redundant steps in the build process. The `elf` directory contains these, and + the existing `bin` directory still holds the full ROM images (containing + payloads) when compiled. +* GRUB payload: vastly reduced the size of the payload, by eliminating GRUB + modules that were not needed. About 100KB of compressed space saved in flash! +* GRUB payload: [argon2 key derivation supported](argon2.md) - this means LUKS2 + decryption is now possible in GRUB. This work was performed by Nicholas + Johnson, rebasing from Axel's AUR patch for GRUB 2.06 (Canoeboot currently + uses GRUB 2.12). +* The *new* coding style is now used on many more scripts, including + the `build/boot/roms_helper` script - the new style is much cleaner, + mandating that logic be top-down, with a `main()` function defined; it's + basically inspired by the OpenBSD coding style for C programs, adapted to + shell scripts. +* All GRUB keymaps now included; a single `grub.elf` is now used on all ROM + images. The `grub.cfg` goes in GRUB memdisk now, but can be overridden by + inserting a `grub.cfg` in CBFS; many behaviours are also controlled this way, + for example to change keymaps and other behaviours. This results in *much* + faster builds, because a different GRUB payload doesn't have to be added to + each new ROM image; such takes time, due to time-expensive LZMA compression. + This, plus the optimised set of GRUB modules, also makes GRUB itself load + much faster. All of the fat has been trimmed, though still quite a lot more + than a Crumb. +* A lot of scripts have been removed entirely, and their logic not replaced; + in many cases, Canoeboot's build system contained logic that had gone unused + for many years. +* More reliable configs now used on desktop mainboards: SeaBIOS-only for start, + but GRUB still available where feasible (in the SeaBIOS menu). This makes it + more fool proof for a user who might use integrated graphics and then switch + to a graphics card; the very same images will work. +* TMPDIR environmental variable now set, and exported from main parent process + when running lbmk; child processes inherit it, and a single tmp dir is used. + This is then automatically cleaned, upon exit from lbmk; previously, lbmk did + not cleanly handle `/tmp` at all, but now it's pretty reliable. + +Hardware supported in this release +================================== + +All of the following are believed to *boot*, but if you have any issues, +please contact the Canoeboot project. They are: + +### Servers (AMD, x86) + +- [ASUS KFSN4-DRE motherboard](../docs/hardware/kfsn4-dre.md) +- [ASUS KGPE-D16 motherboard](../docs/hardware/kgpe-d16.md) + +Desktops (AMD, Intel, x86) +----------------------- + +- [Gigabyte GA-G41M-ES2L motherboard](../docs/hardware/ga-g41m-es2l.md) +- [Acer G43T-AM3](../docs/hardware/acer_g43t-am3.md) +- [Intel D510MO and D410PT motherboards](../docs/hardware/d510mo.md) +- [Apple iMac 5,2](../docs/hardware/imac52.md) +- [ASUS KCMA-D8 motherboard](../docs/hardware/kcma-d8.md) + +### Laptops (Intel, x86) + +- **[Dell Latitude E6400](../docs/hardware/e6400.md) (easy to flash, no disassembly, similar + hardware to X200/T400)** +- ThinkPad X60 / X60S / X60 Tablet +- ThinkPad T60 (with Intel GPU) +- [Lenovo ThinkPad X200 / X200S / X200 Tablet](../docs/hardware/x200.md) +- Lenovo ThinkPad X301 +- [Lenovo ThinkPad R400](../docs/hardware/r400.md) +- [Lenovo ThinkPad T400 / T400S](../docs/hardware/t400.md) +- [Lenovo ThinkPad T500](../docs/hardware/t500.md) +- [Lenovo ThinkPad W500](../docs/hardware/t500.md) +- [Lenovo ThinkPad R500](../docs/hardware/r500.md) +- [Apple MacBook1,1 and MacBook2,1](../docs/hardware/macbook21.md) + +### Laptops (ARM, with U-Boot payload) + +- [ASUS Chromebook Flip C101 (gru-bob)](../docs/install/chromebooks.md) +- [Samsung Chromebook Plus (v1) (gru-kevin)](../docs/install/chromebooks.md) + +Downloads +========= + +You can find this release on the downloads page. At the time of this +announcement, some of the rsync mirrors may not have it yet, so please check +another one if your favourite one doesn't have it. + +Special changes +=============== + +Besides deblobbing, there are two critical differences in how Canoeboot's +build system works in this release, versus the Libreboot 20231021 build system: + +* Single-tree git submodules are not downloaded in Canoeboot; none of them are + used in the Libreboot release, but using them simplified `config/git/` because + many of those entries were defined as submodules by each given project; in + some serprog-related repositories, proprietary drivers get downloaded that are + never actually compiled or executed in any way. Rather than deblob these in + Canoeboot, the Canoeboot build system simply skips downloading those + repositories altogether. +* Thus, several entries in under `config/git/` for Canoeboot 20231026, that do + not exist under Libreboot 20231021. + +This quirk is only a minor difference. Severals scripts that handled +dependencies for building non-FSDG-compliant boards (such as blob download +scripts) have been *excluded* in this Canoeboot release, because they are +not needed. + +As a result, the Canoeboot build system is about 1250 sloc when counting shell +scripts of the build system; the Libreboot build system is about 1750. This +comparison is between Canoeboot 20231026 and Libreboot 20231021 - by contrast, +Libreboot 20230625 was 3388 sloc, and GNU Boot 0.1 RC is 2111 sloc (counting +shell scripts, because it uses the same design as lbmk and cbmk). + +That ~1250 sloc in Canoeboot is *with* all the extra features such as serprog +integration and U-Boot support (on actual mainboards, that you can flash it +with). The build system in Canoeboot 20231026 is *[extremely +efficient](../docs/maintain/)*. + +Backports +========= + +In addition to the Libreboot 20231021 changes, the following Libreboot patches +were backported into this Canoeboot release, from Libreboot revisions pushed +after the Libreboot 20231021 release came out: + +* +* +* +* +* +* +* +* +* + +Changes NOT included in this release +==================================== + +These entries are from the Libreboot 20231021 change log, but these changes +are *not* present in the Canoeboot 20231026 release: + +* Better integrity checking when downloading vendor files +* Scrubbing of vendor files *now* handled by the inject script, rather than + the release script. This enables more robust handling of configs pertaining + to vendor files, that tell lbmk where the files are and how to insert them; it + therefore follows that this same script should be used to delete them. +* Unified handling of git/vendor config files, containing URLs, revisions, + checksums and so on. This is handled by a single function + under `include/option.sh` +* Intel ME extraction is now provided in one function, instead of two, when + downloading vendor files per mainboard, before running it + through `me_cleaner` +* Unified checking of the destination file, when downloading vendor updates. + This results in more reliable checking of whether a vendor file has already + been downloaded or not, where it is only handled if missing. +* Vendor scripts: archive extraction is now unified, the same method used for + each archive. This enables more robust checking of hashes and so on. +* More deeply integrated the Intel MRC download script (from coreboot) into + Canoeboot's vendor scripts, removing its download logic and re-using that + from Canoeboot's scripts instead; now, the MRC script only contains extraction + logic, and it is an *include* file, rather than a standalone script. +* Where no-microcode ROM images are provided, ensure that the ROM hashes still + match when running the vendor inject script. This is only useful on the + Dell Latitude E6400, which is otherwise FSDG-compatible but (in Canoeboot) + comes with or without microcode updates, and with or without the Nvidia VGA + ROM (handled by vendor inject/download scripts) for dGPU variants. Verification + previously failed, under certain conditions, when inserting that VGA ROM. +* Vendor scripts: don't use `/tmp` for ROM images when inserting vendor files. + In case `/tmp` is a tmpfs and not much RAM is available, it is paramount that + the user's file system is used instead, where there is likely greater capacity; + it is done under `tmp/` in lbmk (not to be confused with `/tmp`). +* move `me7_updater_parser.py` to `util/` (not under `script/`) +* The directory containing vendor files no longer exists in lbmk, because it + is instead created when needed; the ifd/gbe files were moved to `config/ifd` + so the vendorfile directory became redundant. +* Don't support removal of microcode (during release time) on untested targets. + Set `microcode_required="y"` on most boards, but leave it set to `"n"` on + platfroms such as GM45 (ThinkPad X200/T400, Dell E6400, etc); anything FSDG + compatible, in other words. +* Improved Dell Latitude E6400 support; the same image now provides iGPU and + dGPU support, since it's SeaBIOS-only anyway, so a VGA ROM is inserted into + the same ROM that also enables libgfxinit, enabling the Intel or Nvidia GPU + to be used (if the VGA ROM is missing, only the Intel GPU will work) +* Only remove microcode (where that behaviour is enabled per board) in release + ROMs, but not during build time. This results in reduced disk usage during + development, but release archives still contain the no-microcode option if + you want to use that; manual removal is also still possible, during development. +* *Copy* `dl_path`, don't move it, when downloading and extracting a vendor + file. This reduces the change of it being missing later when lbmk is run again. +* Improved handling of vendor file hashes; previously, the backup would only + be tried if the first one failed to download, but if the first file succeeded + and yet had a bad hash, the backup would not be tried. Now the backup is tried + when either the first download fails OR it has a bad hash, making downloads + of vendor files more resilient to network failure. +* When extracting ME files from vendors, more types of archives are supported + for decompression at build time. +* Fixed bug where vendor files were always being downloaded from backup URLs + at build time. +* Spoof the user agent string mimicking that of Tor Browser, when downloading + vendor files at build time. This circumvents restrictions based on user agent + string, when lbmk interacts with certain HTTP servers. +* Abort (with non-zero exit) if KBC1126 EC firmware fails to download at build + time. +* Haswell (libre MRC) coreboot tree: fixed acpica downloads, which no longer + work on the upstream URL. Old acpica binaries now hosted on Canoeboot rsync. +* Blobutil: generally more reliable now at downloading vendor files, especially + under fault conditions; for example, if a download failed before, it'd try + a backup link, but now it also tries the backup link if main download succeeds + but checksum verification didn't; and SHA512 checksums are now used, for + greater security, whereas nonGeNUine Boot 20230717 used sha1sum (now we use + sha512sum). A user agent is specified in wegt, matching that used by Tor + Browser (which in turn mimics Firefox running on Windows). This is needed + for some vendors, which seem to dislike wget's default user agent. + +Excluded mainboards +=================== + +The following boards are *missing* in Canoeboot 20231026, but are supported in +the Libreboot 20231021 release; this is because they do not comply with FSDG +policy: + +* Dell Latitude E6430 +* Dell Precision T1650 +* HP EliteBook 2170p +* HP EliteBook 2560p +* HP EliteBook 2570p +* HP EliteBook 8470p +* HP 8200 SFF +* HP 8300 USDT +* HP EliteBook 9470m +* Lenovo ThinkPad T420 +* Lenovo ThinkPad T420S +* Lenovo ThinkPad T430 +* Lenovo ThinkPad T440p +* Lenovo ThinkPad T520 +* Lenovo ThinkPad T530 +* Lenovo ThinkPad W530 +* Lenovo ThinkPad W541 +* Lenovo ThinkPad X220/X220T +* Lenovo ThinkPad X230/X230T + +Removed/modified code, in the build system +------------------------------------------- + +Again, certain features cannot be merged from Libreboot and into Canoeboot, +because of the restrictions set by Canoeboot policy (adhering to GNU FSDG). Here +is an overview of the code present in Libreboot 20231021 that is *missing* in +Canoeboot 20231026: + +* **coreboot and u-boot download scripts:** Binary blobs are now removed during + download. A list of blobs is programmed into the build system, based on + scanning of each tree with the linux-libre `deblob-check` script. (yes, it + works on other code bases, besides Linux). **This means that most mainboards + no longer compile, in coreboot, and many u-boot targets no longer compile.** +* **`build/roms`:** These scripts build ROM images. For **zero-blob boards**, + in other words boards that do not require binary blobs, *regular* Libreboot + inserts **CPU microcode** by default, but copies each ROM to produce a + corresponding, parallel zero-blobs version **without** CPU microcode. **This** + censored version of Libreboot modifies the script in the following way: since + the coreboot and uboot download scripts **remove blobs** anyway, including CPU + microcode, the default compiled ROMs exclude microcode. Therefore, *this* + version simply removes that logic, because it's not needed. +* **`blobutil`:** Anything pertaining to the downloading of vendor blobs + has been removed. This includes `me_cleaner`, `ME7 Update Parser` and the like. + It is not needed, in this version of Libreboot. Directories such + as `resources/blobs/` (containing code and config data) has been removed. + In regular Libreboot, there are certain required binary blobs that we cannot + legally distribute on certain mainboards, so `blobutil` auto-downloads them + from the vendor while compiling ROM images, then it processes them (if needed) + and inserts them; the scripts that produce release archives will *delete* + these blobs, for the release, and those same scripts can be re-run on release + ROMs, to re-insert binary blobs. It is *completely automated*, removing any + need for manual intervention by the user, thus saving hours of time in some + cases. Blobutil snaps them up like *that* and everything *Just Works*. + It does this for *many* different types of blobs, including: Intel ME, Intel + MRC, HP KBC1126 EC firmware, VGA ROMs - you just run 1 command on 1 ROM (or + an entire collection of ROMs) and it does it, automatically detecting what + is needed for the given ROM image, per mainboard definition. Very easy to use. + This *highly innovative* technology does not exist in Censored Libreboot. +* Blobs: Removed Intel Flash Descriptors and GbE configuration files. These are + non-copyrightable, non-software blobs, just binary-encoded config. They are + not needed, in this Libreboot version. NOTE: ICH9M ones remain, because they + are needed (but they are not software). +* Blobs: Anything downloaded and inserted by `blobutil`, during the build + process or post-release in the Libreboot build system. This includes: + Intel ME firmware, Intel MRC firmware, HP KBC1126 EC firmware and VGA option + ROM for Nvidia GPU variant of Dell Latitude E6400. +* `lbmk`: Code that executes blob-related scripts has been removed. +* Patches: Any custom coreboot patches, for mainboards that require binary + blobs, have been removed. They are not needed in this Libreboot version. +* `update/release`: correspondingly deleted files + are no longer copied by these scripts (they are the scripts that generate + tar archives for Libreboot releases, after everything is compiled). The build + logic no longer bothers to scrub non-redistributable inserted binary blobs + from certain ROM images, because 1) those corresponding mainboards are no + longer supported anyway and 2) the logic for downloading/inserting those + blobs no longer exists. So there's nothing to do. + +It's not actually a lot of code that was removed. The actual diff that did this +is very large, because it also removed the coreboot configs for the removed +boards, and those configs are very large. + +Libreboot is superior to Canoeboot, in every way. You should use Libreboot. +Use of Canoeboot is even *dangerous*, because lack of microcode updates in +Canoeboot could potentially lead to data loss due to memory corruption. + +Read more about the [Binary Blob Reduction +Policy](https://libreboot.org/news/policy.html) of the Libreboot project. The +Canoeboot project is provided as a proof of concept, to demonstrate just how +awful Libreboot used to be, before it implement the new policy in November 2022. + +Canoeboot is a worthless project, but engineered to a high standard. It's +necessary to do this, because there are some people who won't adequately see +the problem unless it actually exists; Canoeboot is not a problem, because it's +not the only choice, but there was a time when osboot didn't even exist, let +alone the new Libreboot, and the other more pragmatic coreboot distros do not +support as much hardware as Libreboot does today. + +You should use Libreboot, even if your hardware is compatible with Canoeboot. +I make these Canoeboot releases, specifically so that I have something to crap +all over. I could criticise GNU Boot more heavily, but GNU Boot is even more +inferior; I make Canoeboot as good as it can feasibly be at any point in time, +and criticise *that* result. My arguments are stronger when an *example* exists, +especially a strong example such as Canoeboot. If the best possible solution is +still inferior, then that will further invalidate the even lesser solutions, +and that is the entire purpose of Canoeboot; I do Canoeboot releases, specifically +so that I can crap all over them. I'm allowed to do that if it's mine. + +I say again. Canoeboot is inferior. + +[Download Libreboot 20231021 instead](https://libreboot.org/news/libreboot20231021.html). + +Censored Libreboot 20230710 release +=================================== + +On this day, the websites of Censored Libreboot and nonGeNUine Boot are being +redirected (HTTP 301 return) to the Canoeboot website. + +An archive of nonGeNU 20230717's announcement is contained on this website, +but not Censored Libreboot 20230717; it was virtually identical to +nonGeNUine Boot 20230717, the latter of which was just a re-brand of +Censored Libreboot. + +If you do want to see either nonGeNU or C-Libreboot, go to these links: + +* +* +* + +And for nonGeNUine Boot, though the code (website and code) is included in +the Canoeboot repositories, here are the original repositories: + +* +* +* + +You can find the actual software release archives for nonGeNUine Boot 20230717 +and Censored Libreboot 20230710 under Libreboot rsync mirrors, +in the `canoeboot` directory. They have been moved there, from where they +were previously hosted. + +Post-release errata +=================== + +The following binary blobs were overlooked, and are still present in the +release archive for Canoeboot 20231101 and 20231026; this mistake was +corrected, in the [Canoeboot 20231103 release](canoeboot20231103.md), so you +should use that if you don't want these files. They are, thus: + +* `src/coreboot/default/3rdparty/stm/Test/FrmPkg/Core/Init/Dmar.h` +* `src/coreboot/fam15h_rdimm/src/vendorcode/intel/fsp1_0/baytrail/absf/minnowmax_1gb.absf` +* `src/coreboot/fam15h_rdimm/src/vendorcode/intel/fsp1_0/baytrail/absf/minnowmax_2gb.absf` +* `src/coreboot/fam15h_udimm/src/vendorcode/intel/fsp1_0/baytrail/absf/minnowmax_1gb.absf` +* `src/coreboot/fam15h_udimm/src/vendorcode/intel/fsp1_0/baytrail/absf/minnowmax_2gb.absf` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_err.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_gap.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_gatt.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_gattc.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_gatts.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_hci.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_l2cap.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_ranges.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_types.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_error.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_error_sdm.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_error_soc.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_nvic.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_sdm.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_soc.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_svc.h` +* `src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf52/nrf_mbr.h` + +Thanks go to Craig Topham, who is the Copyright and Licensing Associate at the +Free Software Foundation; you can find his entry on the [FSF staff +page](https://www.fsf.org/about/staff-and-board). Craig is the one who reported +these. + +The Canoeboot 20231026 and 20231101 release tarballs will not be altered, but +errata has now been added to the announcement pages for those releases, to let +people know of the above issue. + +You are advised, therefore, to use the [Canoeboot 20231103 +release](canoeboot20231103.md). + +Update on 12 November 2023: +--------------------------- + +This file was also overlooked, and is still present in the release tarball: + +* `src/vendorcode/amd/agesa/f12/Proc/GNB/Nb/Family/LN/F12NbSmuFirmware.h` + +This has now been removed, in the Canoeboot git repository (`cbmk.git`), and +this file will absent, in the next release after Canoeboot 20231107. Thanks go +to Denis Carikli who reported this. The patch to fix it is here: + +