diff --git a/site/docs/build/index.md b/site/docs/build/index.md index 29eb592..770667b 100644 --- a/site/docs/build/index.md +++ b/site/docs/build/index.md @@ -25,6 +25,20 @@ canoeboot from the available source code. The following document describes how `cbmk` works, and how you can make changes to it: [canoeboot maintenance manual](../maintain/) +Multi-threaded builds +===================== + +Canoeboot's build system defaults to a single build thread, but you can change +it by doing e.g. + + export XBMK_THREADS=4 + +This would make cbmk run on 4 threads. + +More specifically: when compiling source trees via `script/trees`, `-jTHREADS` +is passed, where THREADS is the number of threads. This is also set when running +xz commands for compression, using the `-t` option. + Environmental variables ======================= diff --git a/site/docs/build/index.uk.md b/site/docs/build/index.uk.md index cedfebf..45d841b 100644 --- a/site/docs/build/index.uk.md +++ b/site/docs/build/index.uk.md @@ -17,13 +17,27 @@ images (containing payloads) in `bin/`. This design is more efficient, and permits many configurations without needless duplication of work. More info is available in the [cbmk maintenance manual](../maintain/)** +Multi-threaded builds +===================== + +Canoeboot's build system defaults to a single build thread, but you can change +it by doing e.g. + + export XBMK_THREADS=4 + +This would make cbmk run on 4 threads. + +More specifically: when compiling source trees via `script/trees`, `-jTHREADS` +is passed, where THREADS is the number of threads. This is also set when running +xz commands for compression, using the `-t` option. + Environmental variables ======================= Please read about environmental variables in [the build instructions](../maintain/), before running cbmk. You should set your variables accordingly, though you do not technically need to; some -of them may be useful, e.g. `CBMK_THREADS` (sets the number of build threads). +of them may be useful, e.g. `XBMK_THREADS` (sets the number of build threads). Introduction ============ diff --git a/site/docs/gnulinux/grub_boot_installer.md b/site/docs/gnulinux/grub_boot_installer.md index 5f71163..78e573c 100644 --- a/site/docs/gnulinux/grub_boot_installer.md +++ b/site/docs/gnulinux/grub_boot_installer.md @@ -40,13 +40,6 @@ folder, this is the command we would run: That's it! You should now be able to boot the installer from your USB drive (the instructions for doing so will be given later). -## GRUB2 config on external media - -Pick the menu option: *Search for GRUB2 configuration on external media* - -If the distro installer image has a `grub.cfg` file inside, this menuentry is -scripted to find it. This works well for many distros. - ## Prepare the USB drive in NetBSD [This page](https://wiki.netbsd.org/tutorials/how_to_install_netbsd_from_an_usb_memory_stick/) on the NetBSD website shows how to create a NetBSD bootable USB drive, from @@ -85,6 +78,14 @@ the OpenBSD installer to it with `dd`. Here's an example: That's it! You should now be able to boot the installer from your USB drive (the instructions for doing so will be given later). +## GRUB2 config on external media + +NOTE: This will also apply to Trisquel GNU+Linux (an Ubuntu-based distro). +Pick the menu option: *Search for GRUB2 configuration on external media* + +If the distro installer image has a `grub.cfg` file inside, this menuentry is +scripted to find it. This works well for many distros. + ## Debian or Devuan net install NOTE: This will also apply to Trisquel GNU+Linux (an Ubuntu-based distro). diff --git a/site/docs/hardware/e6400.md b/site/docs/hardware/e6400.md index 2ff6035..4128abe 100644 --- a/site/docs/hardware/e6400.md +++ b/site/docs/hardware/e6400.md @@ -3,6 +3,12 @@ title: Dell Latitude E6400 x-toc-enable: true ... +**Thermal safety**: this machine shuts down very quickly, when the machine +exceeds 80c CPU temperature, which is far more conservative than on most +laptops (non-Dell ones), so you should make sure that your thermals are +excellent. More info available [here](dell_thermal.md). This is a known bug, +but otherwise the machine will be mostly stable. +
Dell Latitude E6400 Dell Latitude E6400 XFR diff --git a/site/docs/install/index.md b/site/docs/install/index.md index 7fa5791..7b503ea 100644 --- a/site/docs/install/index.md +++ b/site/docs/install/index.md @@ -29,21 +29,23 @@ with `seabios_` at the start of the file name, and `grubfirst` at the end; these place a bootorder file in CBFS, so that SeaBIOS loads GRUB first, but you can still press ESC to bring up the SeaBIOS boot select menu. -The error above, in the linked bug report, occurs on `grub_free()` -or `grub_realloc()` and could be caused by anything in GRUB; the user tested -on memtest86plus and got no errors, and SeaBIOS works; BIOS GRUB (from a distro) -also works, where GRUB doesn't need as many drivers; we suspect this may be -a use-after-free in a GRUB driver, or otherwise something writing to heap memory -wrongly. The bug is not yet known, only the symptom (the fatal error message). +*This warning applies to Canoeboot 20240504/20240510 and other recent releases.* + +**We have since fully mitigated this bug**; SeaBIOS is now the primary payload on +all boards, with GRUB still available in the boot select menu, and we have +identified that it was caused by the xHCI driver which has since been removed +for the affected machines(machines which don't have xHCI anyway, but they +touch code that does run on the given machines). The xHCI support works fine +on some newer machines and will be re-added there by making GRUB multi-tree, +so that different boards can use different versions of GRUB. This will be done, +and present in the next Canoeboot release after 20240510, in addition to fixing +the actual bug itself. **For now, there are no problems!** Canoeboot releases after 20240510 will *only* (on x86) contain ROM images where SeaBIOS is the first payload, without disabling the SeaBIOS menu (no `grubonly`). You'll still be able to use GRUB, either by pressing ESC for the boot select menu, and/or using an image with `grubfirst` in the file name so that SeaBIOS loads it first (while still permitting boot select via ESC keypress). -Canoeboot releases after 20240510 will *only* (on x86) contain ROM images where -SeaBIOS is the first payload, without disabling the SeaBIOS menu (no `grubonly`). - GRUB's code is vast, and complicated, so this policy change is permanent, until GRUB can be well-audited (likely forked, with dead/legacy code removed). SeaBIOS code is much smaller and more robust. Remember always: code equals bugs. diff --git a/site/docs/maintain/index.md b/site/docs/maintain/index.md index 06b8043..5fb56b2 100644 --- a/site/docs/maintain/index.md +++ b/site/docs/maintain/index.md @@ -17,14 +17,13 @@ Automated coreboot build system This document describes the entire Canoeboot build system, its design philosophy and how it's used to prepare Canoeboot releases; it is provided as a *reference* for *Canoeboot development*, pertaining to the current development branch of -Canoeboot's build system (called *cbmk*, short -for **C**anoe**B**oot **M**a**K**e). +Canoeboot's build system (called *cbmk*). The homepage of Canoeboot says that Canoeboot is a *coreboot distro*, providing the necessary integration of coreboot, payloads and utilities so as to provide -releases, much like GNU+Linux distros do for your operating system, but here we are +releases, much like Linux distros do for your operating system, but here we are concerned about the *boot firmware* instead. Canoeboot is to coreboot, what -Trisquel is to GNU+Linux. It provides easier, more automated configuration and +Debian is to Linux. It provides easier, more automated configuration and installation. The build system, cbmk, *is* that coreboot distro, at its very core. You can @@ -38,40 +37,6 @@ check itself when running *any* command; if another command had to be executed first, it will do so automatically. Therefore, you can run any part of cbmk on its own, and the entire design is modular. -Environmental variables -======================= - -CBMK\_THREADS -------------- - -For example: - - export CBMK_THREADS=2 - -This would build on two threads, when running cbmk. It defaults to 1. - -Previous revisions of cbmk used `nproc` by default, but this was set to 1 -instead, because nproc is not available on every operating system. - -CBMK\_STATUS ------------- - -By default, the user is asked to confirm when building for a given mainboard, -if that mainboard is not marked *stable* in `target.cfg`. To disable such -dialogs, do this: - - export CBMK_STATUS=n - -CBMK\_RELEASE -------------- - -If set to `y`, it signals to `script/build/roms` that a release is being built, -and it will honour `release="n"` in target.cfg files. You could also set this -yourself, when doing regular builds, if you wanted to test how `./build roms` -behaves running it in release mode. Do this if you want to: - - export CBMK_RELEASE=y - Best practises for learning cbmk ================================ @@ -97,10 +62,35 @@ Don't be deceived by simplicity Canoeboot's build system is powerful, and highly configurable, yet deceptively simple at the same time. Remember this rule, a rule that applies to *all* software projects: code equals bugs, so smaller codebases will yield fewer bugs. -Canoeboot benefits from regular auditing in the *Libreboot* build system, where -the improvements are ported to Canoeboot after each Libreboot release. +Canoeboot is [regularly](../../news/audit.md) [audited](../../news/audit2.md). -You'll be surprised by how much can be done with so little. Continue reading! +Many people will be shocked by how *small* Canoeboot is, at its core. You will +be surprised by just how much can be done with so little. Continue reading! + +Environmental variables +======================= + +XBMK\_THREADS +------------- + +For example: + + export XBMK_THREADS=2 + +This would build on two threads, when running cbmk. It defaults to 1. + +Previous revisions of cbmk used `nproc` by default, but this was set to 1 +instead, because nproc is not available on every operating system. + +XBMK\_RELEASE +------------- + +If set to `y`, it signals to `script/roms` that a release is being built, +and it will honour `release="n"` in target.cfg files. You could also set this +yourself, when doing regular builds, if you wanted to test how `./build roms` +behaves running it in release mode. Do this if you want to: + + export XBMK_RELEASE=y Projects/files downloaded/generated by cbmk =========================================== @@ -162,12 +152,32 @@ releases (only the images under `bin/` are provided). release/ --------------- -The script at `script/update/release` create tarballs in here, which +The script at `build` create tarballs in here, which constitute regular Canoeboot releases. It is meticulously maintained, as per current cbmk behaviour, and executed so as to provide Canoeboot release archives. -This provides source tarballs and ROM images. +This provides source tarballs, and ROM images. + +You can create release archives by doing: + + ./update release + +By default, this creates a release under `release/`, but you can change the +directory, for example: + + ./update release -d path + +You can also specify that only a *source archive* be created, like so: + + ./update release -m src + +Or with a custom directory: + + ./update release -d path -m src + +The build system expects there to be a *git tag*, so make sure there is one. +This is used to create the version number for a given release. src/ ---- @@ -200,7 +210,7 @@ src/flashprog/ Please also visit: Although currently unused by any part of cbmk, we provide flashprog for the -convenience of users, and this is copied to release archives. Flashprog is the +convenience of users, and this is copied to release archives. Flashrom is the program that you will use to read, erase and write the flash, containing coreboot firmware. @@ -308,9 +318,6 @@ desirable, `cbmk.git` provides a few utilities as part of itself, namely: util/dell-flash-unlock/ --------------- -**NOTE: The util is now called `dell-flash-unlock`, but it -was previously called `e6400-flash-unlock`. Links have been updated.** - This program, written by Nicholas Chin, unlocks the boot flash on Dell Latitude E6400; it permits internal flashing, from factory firmware to Canoeboot, so that the user need not disassemble and flash externally. @@ -350,7 +357,7 @@ directly into `cbmk.git`, and thoroughly cleaned. The cbmk version has been more or less re-written, using the original logic as a base; variables are more clearly named. A top-down, OpenBSD-inspired coding style is used, replacing the GNU coding style implemented in the original code. The [OpenBSD -coding style](https://man.openbsd.org/style.9) is much easier to read. +coding style][https://man.openbsd.org/style.9] is much easier to read. This code has been modified to make use of the `pledge()` system call, when used on [OpenBSD](https://www.openbsd.org/); the original version from GRUB did not @@ -371,18 +378,28 @@ by cbmk: config/ ======= -config/PROJECT\*/blobs.list ---------------------------- - -The script `include/git.sh` handles deletion of binary blobs, for downloaded -projects, based on a `blobs.list` file that can (for single-tree projects) be -included at `config/PROJECT/blobs.list` or (multi-tree project) -at `config/PROJECT/TREE/blobs.list`. Learn more about how de-blobbing is -handled by reading the [about page](../../about.md). - This directory contains configuration files, used by the Canoeboot build system. These next sections will cover specific configuration files. +config/PROJECT\*/nuke.list +-------------------------- + +The script `include/git.sh` handles deletion of certain files, for downloaded +projects, based on a `nuke.list` file that can (for single-tree projects) be +included at `config/PROJECT/nuke.list` or (multi-tree project) +at `config/PROJECT/TREE/nuke.list` (entries are relative links from the root +directory of the given source tree e.g. `src/coreboot/default/`). + +So, if `src/coreboot/default/` contained foo/bar.txt, you could add to +the `nuke.list` file as follows: + +``` +foo/bar.txt +``` + +Ditto `src/flashprog/`, if you wanted to delete a file from in there, as one +other example. Deletions occur when the source tree is created. + config/coreboot --------------- @@ -392,7 +409,7 @@ When a given coreboot tree is compiled, for a given target, this file defines which files to copy from the coreboot directory, which are then copied to a location under `elf/coreboot`. -The presence of this file affects behaviour in `script/update/release`; +The presence of this file affects behaviour in `./update release` commands; specifically, PROJECT is then downloaded to `src/PROJECT/PROJECT`, and files under `config/PROJECT/TARGET/target.cfg` define which tree to use, which then looks under `config/PROJECT/TREE/target.cfg` to get the git revision; then @@ -442,7 +459,6 @@ as: * `grub_scan_disk="ata"` * `uboot_config=default` (specify which U-Boot tree to use) * `release="n"` (example entry) -* `status=stable` (example entry) * `xtree="default"` (example entry) * `tree_depend="default"` (example entry) @@ -490,25 +506,17 @@ other than `default`, which is the default if the option is missing. The `grub_scan_disk` option specifies can be `ahci`, `ata` or `both`, and it determines which types of disks are to be scanned, when the `grub.cfg` file in GRUB payloads tries to automatically find other `grub.cfg` files supplied by -your GNU+Linux distribution. On some machines, setting it to `ata` or `ahci` +your Linux distribution. On some machines, setting it to `ata` or `ahci` can improve boot speed by reducing delays; for example, trying to scan `ata0` on a ThinkPad X60 with the optical drive may cause GRUB to hang, so on that machine it is advisable to set this option to `ahci` (becuse the default HDD slot is AHCI). -The `release` variable can be set to n, which makes the `script/update/release` -script skip that target, when creating release images. For example, a given +The `release` variable can be set to n, which makes the `./update release` +call skip that target, when creating release images. For example, a given board may not be stable and you don't want images for it to be included in the release. -The `status` variable can be set to whatever you want, but anything other -than `stable` will make `script/build/roms` ask for y/n confirmation if -not building images using `script/update/release`. - -Recommended strings for `status` could be: `stable`, `unstable`, `broken` -or `untested`. Alternatively, you might state `wip`. You can set whatever -string you want here. - The `xtree` option specifies that a given tree with use a specific coreboot tree for compiling crossgcc. This can be used to skip building gcc if OK on a given board; two trees may use the same crossgcc as each other. @@ -516,13 +524,6 @@ a given board; two trees may use the same crossgcc as each other. The `tree_depend` option means that a given tree needs another tree, defined by this variable, to also be present. -### config/coreboot/BOARDNAME/warn.txt - -Additionally: the `warn.txt` file can be included alongside target.cfg, to -provide warning of any potential issues or quirks. For example, raminit may -only be reliable with certain modules. This is printed on the user's terminal -when building that target. - ### config/coreboot/BOARDNAME/config/ Files in this directory are *coreboot* configuration files. @@ -934,38 +935,28 @@ Scripts in root directory of cbmk build --------------- -This is the main script in cbmk, Canoeboot's build system. It is what executes -all other parts of the Canoeboot build system. The rules are as follows: +This is the main script. The symlink named `update` also point to it. -* Argument zero, representing the name of the symlink, will be used to - execute `script/LINKNAME/COMMAND` - for example: `./build roms all` - would execute `script/build/roms all` in `sh`. -* In the above example, `LINKNAME` could also be `update`. In examples below, - symlinks are described pointing to `build` (the actual script). The script - works by checking argument zero, so it would look in a different directory - under `script/` matching `LINKNAME` - in this case, `script/update/` -* `TMPDIR` is exclicitly set, providing a constant location where temporary - files and directories can be made. `TMPDIR` is exported by the parent to - all children; for example, `./build roms all` would export it - to `script/build/roms`, and then anything called by *that* will also - inherit it - the main parent process running `cbmk` will then clean up this - `TMPDIR` directory upon any exit. -* All exits from cbmk are handled by this script. *All* exits, zero or non-zero, - are engineered such that *this* script, in the parent process (the very first - instance) is what ultimately exits back to the user's shell prompt. -* This script is programmed to *exit* with non-zero status, when run as root, - unless the `./build dependencies *` commands are used, - referencing files under `config/dependencies/` -* Under fault conditions, each child process shall output to stderr, and the - main parent process running `cbmk` will output the final error message. +Take any given file under `script/` and you can do: -tl;dr break this script and you *break Canoeboot*. + ./build file # (THIS IS NOT A VALID COMMAND) -update ---------------- +For example: -Symbolic link, pointing to the `build` script. This is executed by the user, or -by cbmk, referencing scripts under `script/update/`. + ./build roms + ./update trees + +Special commands available (not provided by files under `script/`): + + ./update release + +Information about `./update release` is written elsewhere on this page. + +You can also know what build system revision you have by running: + + ./build version + +This script is the beating heart of Libreboot. Break it and you break Libreboot. include/ =============== @@ -974,35 +965,20 @@ This directory contains *helper scripts*, to be included by main scripts using the `.` command (called the `source` command in `bash`, but we rely upon posix `sh` only). -include/err.sh +include/git.sh +-------------- + +These functions in here previously existed as independent scripts, but they +were unified here, and they are used when you pass the `-f` argument +to `script/update/trees` (e.g. `./update trees -f coreboot`). + +These functions deal with git cloning, submodule updates, revision resets and +the application of patch files via `git am`. *Every* git repository downloaded +by cbmk is handled by the functions in this file. + +include/lib.sh --------------- -Generic error handling, used by all cbmk scripts. - -This also contains functions to verify the current canoeboot version, and check -whether Git is properly initialised on the host system. It also contains -the `setvars` function, which provides a shorthand way of initialising many -variables (combined with use of `eval`), which cbmk uses heavily. - -This function also contains `x_` and `xx_` which cbmk uses to execute commands -and ensure that they cause an exit (with non-zero status) from cbmk, if they -return an error state; the `xx_` function calls `fail()` which a script must -provide, to perform some action before calling `err` which in turn prints an -error message provided as argument. It is used similarly to the C -function `err()` in BSD libc. The `x_` function simply calls `err`. - -This entire file is heavily inspired by `err.h` in BSD libc code. This file is -heavily used by cbmk (it's used by every script), to provide clean error -handling in `sh`. - -include/option.sh ---------------- - -Functions used by scripts under `script/vendor/`, for checking defconfig -files. These files are checked because the scripts need to know whether a given -file is used; if it is, a path is then specified in defconfig, telling the vendor -script either where it is, or where it should be downloaded to. - Several other parts of cbmk also use this file. It is added to as little as possible, and contains miscallaneous functions that don't belong anywhere else. @@ -1011,7 +987,7 @@ them to set variables and so on. This file also contains generic error handling, used by all cbmk scripts. -This also contains functions to verify the current canoeboot version, and check +This also contains functions to verify the current Canoeboot version, and check whether Git is properly initialised on the host system. It also contains the `setvars` function, which provides a shorthand way of initialising many variables (combined with use of `eval`), which cbmk uses heavily. @@ -1023,20 +999,8 @@ return an error state. script/ ======= -*All* scripts under `script/` are executed only by the main `cbmk` script, -conforming to the standard `buildpath/option` e.g. `build/roms` - so, -running `./build roms` would run `script/build/roms`. - -script/build/ ---------------- - -These are highly specialised build scripts, written for specific tasks, almost -entirely in the context of building firmware images themselves, but some utils -are also handled. - -The scripts that create release archives are also located under this directory. - -### script/build/roms +script/roms +----------- This builds coreboot ROM images. @@ -1055,6 +1019,11 @@ To build *all* targets, specify: ./build roms all +This script can build images for x86 *and* ARM targets. +The *ARM* targets are ChromeOS devices (chromebooks and such); Canoeboot uses +the *U-Boot* payload, rather than Google's *depthcharge* bootloader. In this +setup, U-Boot is running on the bare metal, as enabled by *coreboot*. + For x86 targets, these scripts build with the GRUB and/or SeaBIOS payloads inserted into the ROM images; secondary payloads like Memtest86+ are also handled and inserted here. @@ -1073,28 +1042,6 @@ It creates ROM images with GRUB, SeaBIOS, U-Boot, optionally with Memtest86+ also included, in various separate configurations in many different ROM images for user installation. -The `romtype` entry in `target.cfg` tells this script what to do with the ROM, -after it has been built. Currently, it operates based on these possible values -for `romtype`: - -* `d8d16sas` will cause *fake* (empty) files named `pci1000,0072.rom` - and `pci1000,3050.rom` to be inserted in CBFS. This prevents SeaBIOS from - loading or executing the option ROM stored on PIKE2008 modules, present on - certain configurations with the ASUS KCMA-D8 or KGPE-D16 mainboards. Those - option ROMs cause the system to hang, so they should never be executed (this - means however that booting Linux kernels from SAS devices is impossible on - those boards, unless a Linux payload is used; Linux can use those SAS drives, - without relying on the PIKE2008 option ROMs). When SeaBIOS runs, it will - default to loading the corresponding option ROM from CBFS, if it exists, for - a given PCI device, overriding whatever option ROM is present on the device - itself, but if the option ROM is invalid/empty, SeaBIOS will not attempt to - load another one, until the empty/invalid one (in CBFS) is deleted. -* `i945 laptop`: in this configuration, the upper 64KB section of the ROM is - copied into the 64KB section below that. This results in there being two - bootblocks in the ROM, and you can decide which one is used by setting `bucts` -* If no declaration is made, or a declaration is made contrary to the above, - no special modifications will be made. - If no payload is defined in `target.cfg`, the `build/roms` script will exit with error status. @@ -1123,51 +1070,25 @@ When the ROM is finished compiling, it will appear under a directory in `bin/` This script is the beating heart of Canoeboot. Break it, and you break Canoeboot! -### script/build/serprog +Serprog images: Build firmware images for serprog-based SPI programmers, where they use an STM32 MCU. It also builds for RP2040-based programmers like Raspberry Pi Pico. -Example command: `./build serprog stm32` +Example command: `./build roms serprog stm32` -Example command: `./build serprog rp2040` +Example command: `./build roms serprog rp2040` The `list` argument is available: - ./build serprog stm32 list + ./build roms serprog stm32 list + ./build roms serprog rp2040 list Without arguments, all targets would be compiled, but you can specify a short list of targets instead, based on the output of `list`. -script/update/ --------------- - -This handles most actual building of source trees, called into by scripts -under `script/update/`. - -### script/update/release - -This script builds the release archives, which are then provided in a new -Canoeboot release. Most users do not need to look at this file at all, but it -is provided under free license for curious souls. - -Command: `./update release` - -NOTE: if the `-d` option is used, you can specify a directory other -than `release`. For example: - - ./update release -d /media/stuff/canoeboot_release_test - -If `-d` is not passed, they will go under `release/` in your cbmk repository. -The script is engineered to re-initialise git if ran from a release archive. - -This builds release archives, containing ROM images for coreboot and/or serprog -programmers. It works simply: cbmk clones *itself*, and builds itself in its -clone, then cleans itself up and creates tarballs. If you run this script, you -should expect it to take at least 4 hours; slower on really old systems. On -really fast systems, it might take 2-3 hours. - -### script/update/trees +script/trees +------------ *This* is the other beating heart of Canoeboot. Used heavily by Canoeboot, this script is what handles defconfig files for SeaBIOS, U-Boot *and* coreboot; it @@ -1243,8 +1164,7 @@ As for *projectname", this can either be `coreboot`, `u-boot` or `seabios`. Example commands: ./update trees -b coreboot - ./update trees -b coreboot x200_8mb - ./update trees -b coreboot x230_12mb x220_8mb t1650_12mb + ./update trees -b coreboot x200_8mb t60_intelgpu ./update trees -x coreboot default ./update trees -u seabios ./update trees -m u-boot gru_bob @@ -1282,3 +1202,8 @@ SeaBIOS which are *multi* downloads. The *other* requirement is that defconfigs be used, though this could be worked around in the future if a *multi* setup is needed on a project that *does not use defconfigs* (this is not yet the case in cbmk). + +All of this used to about 20 different scripts, all with much-duplicated logic. +Now it is unified, efficiently, under a single script. + +Remember: code equals bugs, so less code equals fewer bugs.