Signed-off-by: Leah Rowe <info@minifree.org>
master
Leah Rowe 2024-05-28 00:08:08 +01:00
parent 66aeddb1bc
commit 67bcda1dda
6 changed files with 183 additions and 221 deletions

View File

@ -25,6 +25,20 @@ canoeboot from the available source code.
The following document describes how `cbmk` works, and how you can make changes
to it: [canoeboot maintenance manual](../maintain/)
Multi-threaded builds
=====================
Canoeboot's build system defaults to a single build thread, but you can change
it by doing e.g.
export XBMK_THREADS=4
This would make cbmk run on 4 threads.
More specifically: when compiling source trees via `script/trees`, `-jTHREADS`
is passed, where THREADS is the number of threads. This is also set when running
xz commands for compression, using the `-t` option.
Environmental variables
=======================

View File

@ -17,13 +17,27 @@ images (containing payloads) in `bin/`. This design is more efficient, and
permits many configurations without needless duplication of work. More info
is available in the [cbmk maintenance manual](../maintain/)**
Multi-threaded builds
=====================
Canoeboot's build system defaults to a single build thread, but you can change
it by doing e.g.
export XBMK_THREADS=4
This would make cbmk run on 4 threads.
More specifically: when compiling source trees via `script/trees`, `-jTHREADS`
is passed, where THREADS is the number of threads. This is also set when running
xz commands for compression, using the `-t` option.
Environmental variables
=======================
Please read about environmental variables in [the build
instructions](../maintain/), before running cbmk. You should set
your variables accordingly, though you do not technically need to; some
of them may be useful, e.g. `CBMK_THREADS` (sets the number of build threads).
of them may be useful, e.g. `XBMK_THREADS` (sets the number of build threads).
Introduction
============

View File

@ -40,13 +40,6 @@ folder, this is the command we would run:
That's it! You should now be able to boot the installer from your USB drive
(the instructions for doing so will be given later).
## GRUB2 config on external media
Pick the menu option: *Search for GRUB2 configuration on external media*
If the distro installer image has a `grub.cfg` file inside, this menuentry is
scripted to find it. This works well for many distros.
## Prepare the USB drive in NetBSD
[This page](https://wiki.netbsd.org/tutorials/how_to_install_netbsd_from_an_usb_memory_stick/)
on the NetBSD website shows how to create a NetBSD bootable USB drive, from
@ -85,6 +78,14 @@ the OpenBSD installer to it with `dd`. Here's an example:
That's it! You should now be able to boot the installer from your USB drive
(the instructions for doing so will be given later).
## GRUB2 config on external media
NOTE: This will also apply to Trisquel GNU+Linux (an Ubuntu-based distro).
Pick the menu option: *Search for GRUB2 configuration on external media*
If the distro installer image has a `grub.cfg` file inside, this menuentry is
scripted to find it. This works well for many distros.
## Debian or Devuan net install
NOTE: This will also apply to Trisquel GNU+Linux (an Ubuntu-based distro).

View File

@ -3,6 +3,12 @@ title: Dell Latitude E6400
x-toc-enable: true
...
**Thermal safety**: this machine shuts down very quickly, when the machine
exceeds 80c CPU temperature, which is far more conservative than on most
laptops (non-Dell ones), so you should make sure that your thermals are
excellent. More info available [here](dell_thermal.md). This is a known bug,
but otherwise the machine will be mostly stable.
<div class="specs">
<center>
<img tabindex=1 alt="Dell Latitude E6400" class="p" src="https://av.canoeboot.org/e6400/e6400-seabios.jpg" /><span class="f"><img src="https://av.canoeboot.org/e6400/e6400-seabios.jpg" /></span> <img tabindex=1 alt="Dell Latitude E6400 XFR" class="p" style="max-width:24em" src="https://av.canoeboot.org/e6400/e6400xfr-seabios.jpg" /><span class="f"><img src="https://av.canoeboot.org/e6400/e6400xfr-seabios.jpg" /></span>

View File

@ -29,21 +29,23 @@ with `seabios_` at the start of the file name, and `grubfirst` at the end;
these place a bootorder file in CBFS, so that SeaBIOS loads GRUB first, but
you can still press ESC to bring up the SeaBIOS boot select menu.
The error above, in the linked bug report, occurs on `grub_free()`
or `grub_realloc()` and could be caused by anything in GRUB; the user tested
on memtest86plus and got no errors, and SeaBIOS works; BIOS GRUB (from a distro)
also works, where GRUB doesn't need as many drivers; we suspect this may be
a use-after-free in a GRUB driver, or otherwise something writing to heap memory
wrongly. The bug is not yet known, only the symptom (the fatal error message).
*This warning applies to Canoeboot 20240504/20240510 and other recent releases.*
**We have since fully mitigated this bug**; SeaBIOS is now the primary payload on
all boards, with GRUB still available in the boot select menu, and we have
identified that it was caused by the xHCI driver which has since been removed
for the affected machines(machines which don't have xHCI anyway, but they
touch code that does run on the given machines). The xHCI support works fine
on some newer machines and will be re-added there by making GRUB multi-tree,
so that different boards can use different versions of GRUB. This will be done,
and present in the next Canoeboot release after 20240510, in addition to fixing
the actual bug itself. **For now, there are no problems!**
Canoeboot releases after 20240510 will *only* (on x86) contain ROM images where
SeaBIOS is the first payload, without disabling the SeaBIOS menu (no `grubonly`). You'll still be able to use GRUB, either by pressing ESC for the boot
select menu, and/or using an image with `grubfirst` in the file name so that
SeaBIOS loads it first (while still permitting boot select via ESC keypress).
Canoeboot releases after 20240510 will *only* (on x86) contain ROM images where
SeaBIOS is the first payload, without disabling the SeaBIOS menu (no `grubonly`).
GRUB's code is vast, and complicated, so this policy change is permanent,
until GRUB can be well-audited (likely forked, with dead/legacy code removed).
SeaBIOS code is much smaller and more robust. Remember always: code equals bugs.

View File

@ -17,14 +17,13 @@ Automated coreboot build system
This document describes the entire Canoeboot build system, its design philosophy
and how it's used to prepare Canoeboot releases; it is provided as a *reference*
for *Canoeboot development*, pertaining to the current development branch of
Canoeboot's build system (called *cbmk*, short
for **C**anoe**B**oot **M**a**K**e).
Canoeboot's build system (called *cbmk*).
The homepage of Canoeboot says that Canoeboot is a *coreboot distro*, providing
the necessary integration of coreboot, payloads and utilities so as to provide
releases, much like GNU+Linux distros do for your operating system, but here we are
releases, much like Linux distros do for your operating system, but here we are
concerned about the *boot firmware* instead. Canoeboot is to coreboot, what
Trisquel is to GNU+Linux. It provides easier, more automated configuration and
Debian is to Linux. It provides easier, more automated configuration and
installation.
The build system, cbmk, *is* that coreboot distro, at its very core. You can
@ -38,40 +37,6 @@ check itself when running *any* command; if another command had to be executed
first, it will do so automatically. Therefore, you can run any part of cbmk
on its own, and the entire design is modular.
Environmental variables
=======================
CBMK\_THREADS
-------------
For example:
export CBMK_THREADS=2
This would build on two threads, when running cbmk. It defaults to 1.
Previous revisions of cbmk used `nproc` by default, but this was set to 1
instead, because nproc is not available on every operating system.
CBMK\_STATUS
------------
By default, the user is asked to confirm when building for a given mainboard,
if that mainboard is not marked *stable* in `target.cfg`. To disable such
dialogs, do this:
export CBMK_STATUS=n
CBMK\_RELEASE
-------------
If set to `y`, it signals to `script/build/roms` that a release is being built,
and it will honour `release="n"` in target.cfg files. You could also set this
yourself, when doing regular builds, if you wanted to test how `./build roms`
behaves running it in release mode. Do this if you want to:
export CBMK_RELEASE=y
Best practises for learning cbmk
================================
@ -97,10 +62,35 @@ Don't be deceived by simplicity
Canoeboot's build system is powerful, and highly configurable, yet deceptively
simple at the same time. Remember this rule, a rule that applies to *all*
software projects: code equals bugs, so smaller codebases will yield fewer bugs.
Canoeboot benefits from regular auditing in the *Libreboot* build system, where
the improvements are ported to Canoeboot after each Libreboot release.
Canoeboot is [regularly](../../news/audit.md) [audited](../../news/audit2.md).
You'll be surprised by how much can be done with so little. Continue reading!
Many people will be shocked by how *small* Canoeboot is, at its core. You will
be surprised by just how much can be done with so little. Continue reading!
Environmental variables
=======================
XBMK\_THREADS
-------------
For example:
export XBMK_THREADS=2
This would build on two threads, when running cbmk. It defaults to 1.
Previous revisions of cbmk used `nproc` by default, but this was set to 1
instead, because nproc is not available on every operating system.
XBMK\_RELEASE
-------------
If set to `y`, it signals to `script/roms` that a release is being built,
and it will honour `release="n"` in target.cfg files. You could also set this
yourself, when doing regular builds, if you wanted to test how `./build roms`
behaves running it in release mode. Do this if you want to:
export XBMK_RELEASE=y
Projects/files downloaded/generated by cbmk
===========================================
@ -162,12 +152,32 @@ releases (only the images under `bin/` are provided).
release/
---------------
The script at `script/update/release` create tarballs in here, which
The script at `build` create tarballs in here, which
constitute regular Canoeboot releases. It is meticulously maintained, as per
current cbmk behaviour, and executed so as to provide Canoeboot release
archives.
This provides source tarballs and ROM images.
This provides source tarballs, and ROM images.
You can create release archives by doing:
./update release
By default, this creates a release under `release/`, but you can change the
directory, for example:
./update release -d path
You can also specify that only a *source archive* be created, like so:
./update release -m src
Or with a custom directory:
./update release -d path -m src
The build system expects there to be a *git tag*, so make sure there is one.
This is used to create the version number for a given release.
src/
----
@ -200,7 +210,7 @@ src/flashprog/
Please also visit: <https://flashprog.org/>
Although currently unused by any part of cbmk, we provide flashprog for the
convenience of users, and this is copied to release archives. Flashprog is the
convenience of users, and this is copied to release archives. Flashrom is the
program that you will use to read, erase and write the flash, containing
coreboot firmware.
@ -308,9 +318,6 @@ desirable, `cbmk.git` provides a few utilities as part of itself, namely:
util/dell-flash-unlock/
---------------
**NOTE: The util is now called `dell-flash-unlock`, but it
was previously called `e6400-flash-unlock`. Links have been updated.**
This program, written by Nicholas Chin, unlocks the boot flash on Dell Latitude
E6400; it permits internal flashing, from factory firmware to Canoeboot, so that
the user need not disassemble and flash externally.
@ -350,7 +357,7 @@ directly into `cbmk.git`, and thoroughly cleaned. The cbmk version has been
more or less re-written, using the original logic as a base; variables are
more clearly named. A top-down, OpenBSD-inspired coding style is used,
replacing the GNU coding style implemented in the original code. The [OpenBSD
coding style](https://man.openbsd.org/style.9) is much easier to read.
coding style][https://man.openbsd.org/style.9] is much easier to read.
This code has been modified to make use of the `pledge()` system call, when used
on [OpenBSD](https://www.openbsd.org/); the original version from GRUB did not
@ -371,18 +378,28 @@ by cbmk:
config/
=======
config/PROJECT\*/blobs.list
---------------------------
The script `include/git.sh` handles deletion of binary blobs, for downloaded
projects, based on a `blobs.list` file that can (for single-tree projects) be
included at `config/PROJECT/blobs.list` or (multi-tree project)
at `config/PROJECT/TREE/blobs.list`. Learn more about how de-blobbing is
handled by reading the [about page](../../about.md).
This directory contains configuration files, used by the Canoeboot build
system. These next sections will cover specific configuration files.
config/PROJECT\*/nuke.list
--------------------------
The script `include/git.sh` handles deletion of certain files, for downloaded
projects, based on a `nuke.list` file that can (for single-tree projects) be
included at `config/PROJECT/nuke.list` or (multi-tree project)
at `config/PROJECT/TREE/nuke.list` (entries are relative links from the root
directory of the given source tree e.g. `src/coreboot/default/`).
So, if `src/coreboot/default/` contained foo/bar.txt, you could add to
the `nuke.list` file as follows:
```
foo/bar.txt
```
Ditto `src/flashprog/`, if you wanted to delete a file from in there, as one
other example. Deletions occur when the source tree is created.
config/coreboot
---------------
@ -392,7 +409,7 @@ When a given coreboot tree is compiled, for a given target, this file defines
which files to copy from the coreboot directory, which are then copied to
a location under `elf/coreboot`.
The presence of this file affects behaviour in `script/update/release`;
The presence of this file affects behaviour in `./update release` commands;
specifically, PROJECT is then downloaded to `src/PROJECT/PROJECT`, and files
under `config/PROJECT/TARGET/target.cfg` define which tree to use, which then
looks under `config/PROJECT/TREE/target.cfg` to get the git revision; then
@ -442,7 +459,6 @@ as:
* `grub_scan_disk="ata"`
* `uboot_config=default` (specify which U-Boot tree to use)
* `release="n"` (example entry)
* `status=stable` (example entry)
* `xtree="default"` (example entry)
* `tree_depend="default"` (example entry)
@ -490,25 +506,17 @@ other than `default`, which is the default if the option is missing.
The `grub_scan_disk` option specifies can be `ahci`, `ata` or `both`, and it
determines which types of disks are to be scanned, when the `grub.cfg` file in
GRUB payloads tries to automatically find other `grub.cfg` files supplied by
your GNU+Linux distribution. On some machines, setting it to `ata` or `ahci`
your Linux distribution. On some machines, setting it to `ata` or `ahci`
can improve boot speed by reducing delays; for example, trying to scan `ata0`
on a ThinkPad X60 with the optical drive may cause GRUB to hang, so on that
machine it is advisable to set this option to `ahci` (becuse the default HDD
slot is AHCI).
The `release` variable can be set to n, which makes the `script/update/release`
script skip that target, when creating release images. For example, a given
The `release` variable can be set to n, which makes the `./update release`
call skip that target, when creating release images. For example, a given
board may not be stable and you don't want images for it to be included in the
release.
The `status` variable can be set to whatever you want, but anything other
than `stable` will make `script/build/roms` ask for y/n confirmation if
not building images using `script/update/release`.
Recommended strings for `status` could be: `stable`, `unstable`, `broken`
or `untested`. Alternatively, you might state `wip`. You can set whatever
string you want here.
The `xtree` option specifies that a given tree with use a specific coreboot
tree for compiling crossgcc. This can be used to skip building gcc if OK on
a given board; two trees may use the same crossgcc as each other.
@ -516,13 +524,6 @@ a given board; two trees may use the same crossgcc as each other.
The `tree_depend` option means that a given tree needs another tree, defined
by this variable, to also be present.
### config/coreboot/BOARDNAME/warn.txt
Additionally: the `warn.txt` file can be included alongside target.cfg, to
provide warning of any potential issues or quirks. For example, raminit may
only be reliable with certain modules. This is printed on the user's terminal
when building that target.
### config/coreboot/BOARDNAME/config/
Files in this directory are *coreboot* configuration files.
@ -934,38 +935,28 @@ Scripts in root directory of cbmk
build
---------------
This is the main script in cbmk, Canoeboot's build system. It is what executes
all other parts of the Canoeboot build system. The rules are as follows:
This is the main script. The symlink named `update` also point to it.
* Argument zero, representing the name of the symlink, will be used to
execute `script/LINKNAME/COMMAND` - for example: `./build roms all`
would execute `script/build/roms all` in `sh`.
* In the above example, `LINKNAME` could also be `update`. In examples below,
symlinks are described pointing to `build` (the actual script). The script
works by checking argument zero, so it would look in a different directory
under `script/` matching `LINKNAME` - in this case, `script/update/`
* `TMPDIR` is exclicitly set, providing a constant location where temporary
files and directories can be made. `TMPDIR` is exported by the parent to
all children; for example, `./build roms all` would export it
to `script/build/roms`, and then anything called by *that* will also
inherit it - the main parent process running `cbmk` will then clean up this
`TMPDIR` directory upon any exit.
* All exits from cbmk are handled by this script. *All* exits, zero or non-zero,
are engineered such that *this* script, in the parent process (the very first
instance) is what ultimately exits back to the user's shell prompt.
* This script is programmed to *exit* with non-zero status, when run as root,
unless the `./build dependencies *` commands are used,
referencing files under `config/dependencies/`
* Under fault conditions, each child process shall output to stderr, and the
main parent process running `cbmk` will output the final error message.
Take any given file under `script/` and you can do:
tl;dr break this script and you *break Canoeboot*.
./build file # (THIS IS NOT A VALID COMMAND)
update
---------------
For example:
Symbolic link, pointing to the `build` script. This is executed by the user, or
by cbmk, referencing scripts under `script/update/`.
./build roms
./update trees
Special commands available (not provided by files under `script/`):
./update release
Information about `./update release` is written elsewhere on this page.
You can also know what build system revision you have by running:
./build version
This script is the beating heart of Libreboot. Break it and you break Libreboot.
include/
===============
@ -974,35 +965,20 @@ This directory contains *helper scripts*, to be included
by main scripts using the `.` command (called the `source`
command in `bash`, but we rely upon posix `sh` only).
include/err.sh
include/git.sh
--------------
These functions in here previously existed as independent scripts, but they
were unified here, and they are used when you pass the `-f` argument
to `script/update/trees` (e.g. `./update trees -f coreboot`).
These functions deal with git cloning, submodule updates, revision resets and
the application of patch files via `git am`. *Every* git repository downloaded
by cbmk is handled by the functions in this file.
include/lib.sh
---------------
Generic error handling, used by all cbmk scripts.
This also contains functions to verify the current canoeboot version, and check
whether Git is properly initialised on the host system. It also contains
the `setvars` function, which provides a shorthand way of initialising many
variables (combined with use of `eval`), which cbmk uses heavily.
This function also contains `x_` and `xx_` which cbmk uses to execute commands
and ensure that they cause an exit (with non-zero status) from cbmk, if they
return an error state; the `xx_` function calls `fail()` which a script must
provide, to perform some action before calling `err` which in turn prints an
error message provided as argument. It is used similarly to the C
function `err()` in BSD libc. The `x_` function simply calls `err`.
This entire file is heavily inspired by `err.h` in BSD libc code. This file is
heavily used by cbmk (it's used by every script), to provide clean error
handling in `sh`.
include/option.sh
---------------
Functions used by scripts under `script/vendor/`, for checking defconfig
files. These files are checked because the scripts need to know whether a given
file is used; if it is, a path is then specified in defconfig, telling the vendor
script either where it is, or where it should be downloaded to.
Several other parts of cbmk also use this file. It is added to as little as
possible, and contains miscallaneous functions that don't belong anywhere else.
@ -1011,7 +987,7 @@ them to set variables and so on.
This file also contains generic error handling, used by all cbmk scripts.
This also contains functions to verify the current canoeboot version, and check
This also contains functions to verify the current Canoeboot version, and check
whether Git is properly initialised on the host system. It also contains
the `setvars` function, which provides a shorthand way of initialising many
variables (combined with use of `eval`), which cbmk uses heavily.
@ -1023,20 +999,8 @@ return an error state.
script/
=======
*All* scripts under `script/` are executed only by the main `cbmk` script,
conforming to the standard `buildpath/option` e.g. `build/roms` - so,
running `./build roms` would run `script/build/roms`.
script/build/
---------------
These are highly specialised build scripts, written for specific tasks, almost
entirely in the context of building firmware images themselves, but some utils
are also handled.
The scripts that create release archives are also located under this directory.
### script/build/roms
script/roms
-----------
This builds coreboot ROM images.
@ -1055,6 +1019,11 @@ To build *all* targets, specify:
./build roms all
This script can build images for x86 *and* ARM targets.
The *ARM* targets are ChromeOS devices (chromebooks and such); Canoeboot uses
the *U-Boot* payload, rather than Google's *depthcharge* bootloader. In this
setup, U-Boot is running on the bare metal, as enabled by *coreboot*.
For x86 targets, these scripts build with the GRUB and/or SeaBIOS payloads
inserted into the ROM images; secondary payloads like Memtest86+ are also
handled and inserted here.
@ -1073,28 +1042,6 @@ It creates ROM images with GRUB, SeaBIOS, U-Boot, optionally with Memtest86+
also included, in various separate configurations in many different ROM images
for user installation.
The `romtype` entry in `target.cfg` tells this script what to do with the ROM,
after it has been built. Currently, it operates based on these possible values
for `romtype`:
* `d8d16sas` will cause *fake* (empty) files named `pci1000,0072.rom`
and `pci1000,3050.rom` to be inserted in CBFS. This prevents SeaBIOS from
loading or executing the option ROM stored on PIKE2008 modules, present on
certain configurations with the ASUS KCMA-D8 or KGPE-D16 mainboards. Those
option ROMs cause the system to hang, so they should never be executed (this
means however that booting Linux kernels from SAS devices is impossible on
those boards, unless a Linux payload is used; Linux can use those SAS drives,
without relying on the PIKE2008 option ROMs). When SeaBIOS runs, it will
default to loading the corresponding option ROM from CBFS, if it exists, for
a given PCI device, overriding whatever option ROM is present on the device
itself, but if the option ROM is invalid/empty, SeaBIOS will not attempt to
load another one, until the empty/invalid one (in CBFS) is deleted.
* `i945 laptop`: in this configuration, the upper 64KB section of the ROM is
copied into the 64KB section below that. This results in there being two
bootblocks in the ROM, and you can decide which one is used by setting `bucts`
* If no declaration is made, or a declaration is made contrary to the above,
no special modifications will be made.
If no payload is defined in `target.cfg`, the `build/roms` script will exit
with error status.
@ -1123,51 +1070,25 @@ When the ROM is finished compiling, it will appear under a directory in `bin/`
This script is the beating heart of Canoeboot. Break it, and you break
Canoeboot!
### script/build/serprog
Serprog images:
Build firmware images for serprog-based SPI programmers, where they use an
STM32 MCU. It also builds for RP2040-based programmers like Raspberry Pi Pico.
Example command: `./build serprog stm32`
Example command: `./build roms serprog stm32`
Example command: `./build serprog rp2040`
Example command: `./build roms serprog rp2040`
The `list` argument is available:
./build serprog stm32 list
./build roms serprog stm32 list
./build roms serprog rp2040 list
Without arguments, all targets would be compiled, but you can specify a short
list of targets instead, based on the output of `list`.
script/update/
--------------
This handles most actual building of source trees, called into by scripts
under `script/update/`.
### script/update/release
This script builds the release archives, which are then provided in a new
Canoeboot release. Most users do not need to look at this file at all, but it
is provided under free license for curious souls.
Command: `./update release`
NOTE: if the `-d` option is used, you can specify a directory other
than `release`. For example:
./update release -d /media/stuff/canoeboot_release_test
If `-d` is not passed, they will go under `release/` in your cbmk repository.
The script is engineered to re-initialise git if ran from a release archive.
This builds release archives, containing ROM images for coreboot and/or serprog
programmers. It works simply: cbmk clones *itself*, and builds itself in its
clone, then cleans itself up and creates tarballs. If you run this script, you
should expect it to take at least 4 hours; slower on really old systems. On
really fast systems, it might take 2-3 hours.
### script/update/trees
script/trees
------------
*This* is the other beating heart of Canoeboot. Used heavily by Canoeboot, this
script is what handles defconfig files for SeaBIOS, U-Boot *and* coreboot; it
@ -1243,8 +1164,7 @@ As for *projectname", this can either be `coreboot`, `u-boot` or `seabios`.
Example commands:
./update trees -b coreboot
./update trees -b coreboot x200_8mb
./update trees -b coreboot x230_12mb x220_8mb t1650_12mb
./update trees -b coreboot x200_8mb t60_intelgpu
./update trees -x coreboot default
./update trees -u seabios
./update trees -m u-boot gru_bob
@ -1282,3 +1202,8 @@ SeaBIOS which are *multi* downloads. The *other* requirement is that defconfigs
be used, though this could be worked around in the future if a *multi* setup is
needed on a project that *does not use defconfigs* (this is not yet the case in
cbmk).
All of this used to about 20 different scripts, all with much-duplicated logic.
Now it is unified, efficiently, under a single script.
Remember: code equals bugs, so less code equals fewer bugs.