Commit Graph

256 Commits (b179ef63edad3f0f1812bef20ea2183610e97db2)

Author SHA1 Message Date
Leah Rowe f102e21ab6 build/roms: simplify serprog list command
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18 22:21:23 +01:00
Leah Rowe 7a565c9f43 build/roms: simplified config payload checks
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18 22:21:11 +01:00
Leah Rowe 3e5db248dd cbmk: allow easier sync with lbmk
an equivalent change has been made in lbmk.

certain cbmk-specific variable names have been made
generic, with certain functions and other variables
moved around.

i maintain sync between libreboot and canoeboot, where
both projects can have the same behaviours, and most of
the merge conflicts have to do with variable names
containing "LBMK", "lbmk", "cbmk" or "CBMK", or
indeed "canoeboot" and "libreboot"

LBMK/lbmk/CBMK/cbmk variables between canoeboot and
libreboot now contain the string XBMK/xbmk

it should now be *much* easier to merge build system
changes between lbmk and cbmk.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-16 03:59:23 +01:00
Leah Rowe e71189420f remove help commands (user should read docs)
i always say, code should never document itself.

that's what documentation is for. the releases
contain documentation under docs/ but the git
repository does not; for that, use the website.

(in practise, cbmk usually needs internet anyway)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 07:55:54 +01:00
Leah Rowe 0794127986 remove check_project() (always set variables)
in cbmk, we call check_project() to set variables
such as projectname, version, version date

this is unnecessary, because all main scripts use
this functionality anyway

do it by default

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 07:49:55 +01:00
Leah Rowe ae44676727 build/roms: clean up tarball handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 07:45:02 +01:00
Leah Rowe 21436c6a8f build/roms: create full release tarball name
set relname from option.sh under check_project()

now the release logic simply has to move a directory

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 07:40:53 +01:00
Leah Rowe 09fcc343a3 move rom tarball creation to script/roms
export CBMK_RELEASE="y"

if this is done, the tarball is created instead
of a directory, and the rom images are nuked using
./vendor inject with the nuke option, inserting the
correct version files; the rom directory is deleted

now the release script logic simple renames existing
tarballs. the benefit of this change is fewer lines of
code, and now cbmk doesn't use an insane amount of disk
space when building a *lot* of release images (the
uncompressed directories are deleted after each build)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 07:27:20 +01:00
Leah Rowe d423421995 remove all status checks. only handle release.
the release variable is all we need, turning a target on
or off for a given release.

the status checks were prone to bugs, and unnecessary; it
also broke certain benchmark scripts.

it's better to keep the cbmk logic simpler. board status
will be moved to the documentation instead.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-13 02:07:22 +01:00
Leah Rowe 541430016f move script/*/* to script/
there are only two scripts under script/ now, and there
probably won't be many more. cbmk's design has simplified
to such a degree that the two-level directory structure is
no longer necessary.

the existing command structure has not changed. for example:

./build roms list
./update trees -f coreboot default

these will still work, but the symlinks to "build" are now
strictly for backwards compatibility; they may be removed
at a later date, but i'll keep the current design for now.

this also leads to a quirk, for example:

./build roms all
./update roms all

these now do the exact same thing, whereas "./update roms all"
would have previously been an invalid command.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-13 01:57:12 +01:00
Leah Rowe f12c2f284f merge script/update/release into build
the main script isn't that big, and since the main
purpose of cbmk is geared toward the releases, it
makes sense to reduce the number of scripts by
merging into the main one

the way this works, "./update release" still works
afterward

so, the way cbmk is used shall remain unchanged

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-13 01:54:56 +01:00
Leah Rowe 41f4ee3c2d Canoeboot 20240510 release
Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-10 08:39:24 +01:00
Leah Rowe eb8a02e808 build/roms: print serprog help
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-10 05:29:53 +01:00
Leah Rowe a398011180 merge script/build/serprog with script/build/roms
previous command:

./build serprog

now it is:

./build roms serprog

after that, it's the same arguments e.g.

./build roms serprog stm32
./build roms serprog rp2040

further cleanup to commence

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-10 05:29:25 +01:00
Leah Rowe cd5c2573ac build/roms: remove unnecessary command
there is no need to return 0 at the end of a function.

sh does that anyway

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-10 05:29:14 +01:00
Leah Rowe da748de455 merge include/err.sh with include/option.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-10 05:25:23 +01:00
Leah Rowe 6bdbb70dbc build/roms: don't rely on x in handle_target
x is part of the for loop in main() and may or not
still be available from handle_target, depending on
your implementation of sh, but this should not be assumed

do it properly

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-10 05:19:53 +01:00
Leah Rowe 1c84d0fc9d build/roms: don't use exit status from skip_board
the printf could potentially return non-zero, which might
make the script not skip a given target

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-10 05:19:48 +01:00
Leah Rowe 0ada63b629 build/roms: split up main()
it's starting to get a big big, so break it up

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-10 05:19:34 +01:00
Leah Rowe 5cecd9e394 build/roms: allow searching status by mismatch
for example:

./build roms list stable

this lists all images that are marked "stable"

now:

./build roms list _stable

this lists all images that are *not* marked stable

this will help me keep track during development

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-10 05:19:24 +01:00
Leah Rowe 29d6d294ab Canoeboot 20240504 release
Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-04 10:26:51 +01:00
Leah Rowe 8a1255634e build/roms: more useful status warnings
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-04 10:22:40 +01:00
Leah Rowe fc0972fe6d update/release: purge test/lib/strlcat.c in u-boot
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-04 10:20:50 +01:00
Leah Rowe 2074d33712 update/release: say when an archive is being made
without this change, the user might think lbmk crashed

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-04 10:20:20 +01:00
Leah Rowe 2672393bf5 build/roms: simplified list handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:40:48 +01:00
Leah Rowe 4876f32159 build/roms: if release, allow all non-broken roms
this includes untested roms

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:38:41 +01:00
Leah Rowe 3ec6607807 build/roms: always display warnings
(even if status=stable)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:35:31 +01:00
Leah Rowe 3eb59000a5 build/roms: reduce indentation in skip_board()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:30:33 +01:00
Leah Rowe 3aa148aa2e build/roms: simplified status handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:27:37 +01:00
Leah Rowe 96eb7913a5 build/roms: simplified seagrub handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:22:15 +01:00
Leah Rowe cbf673fc46 build/roms: support SeaGRUB *with menu enabled*
This is useful on desktops, where you want GRUB to
automatically start, but you still want access to the
GRUB menu, in the case where you rely on SeaBIOS to
execute the VGA ROM inside your graphics card.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:21:48 +01:00
Leah Rowe 79b1a1fee2 update/trees: simplified defconfig copying
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:21:18 +01:00
Leah Rowe ee853c20e0 update/release: disable status checking
just to ensure that nothing goes wrong. we don't rely on
the status variable for releases, because there is another
variable, release, that target.cfg files declare, e.g.

release="n"
release="y"

you can just omit the variable, because it defaults to y, so
you only need declare it when it needs to be "n"

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:14:54 +01:00
Leah Rowe 8cf0d56f66 build/roms: tell the user how to ignore status
Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-03 01:12:29 +01:00
Leah Rowe c7513ce31c allow disabling status checks during builds
export CBMK_STATUS=n

if not set, the status checks and confirmation dialogs
persist. if set to y they persist.

if you set it to n, all checks are disabled, so e.g.:

./build roms all

this would once again build all targets, regardless
of status. this is if you want the old behaviour.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 01:03:02 +01:00
Leah Rowe b707b7a8cc build/roms: update help, pertaining to status
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 00:49:24 +01:00
Leah Rowe eda035eb23 build/roms: let "list" specify status types
for example:

./build roms list

this will list every now, still. same behaviour. now see:

./build roms list stable

this will list all stable roms

./build roms list untested

this lists untested roms. but wait!

./build roms list untested broken unstable

./build roms list broken unstable

yes. it works this way. now you can use lbmk to easily
see what rom status are, during maintenance.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03 00:48:14 +01:00
Leah Rowe b56f79578a build/roms: report status when building images
export CBMK_VERSION_TYPE=x
x can be: stable, unstable

in target.cfg files, specify:
status=x
x can be: stable, unstable, broken, untested
if unset, cbmk defaults to "unknown"

if CBMK_VERSION_TYPE is set, no confirmation is asked
if the given target matches what's set (but what's set
in that environmental variable can only be stable or
unstable)

if CBMK_RELEASE="y", no confirmation is asked, unless
the target is something other than stable/unstable

"unstable" means it works, but has a few non-breaking
bugs, e.g. broken s3 on dell e6400

whereas, if raminit regularly fails or it is so absolutely
unreliable as to be unusable, then the board should be
declared "broken"

untested means: it has not been tested

With this change, it should now be easier to track whether
a given board is tested, in preparation for releases. When
working on trees/boards, status can be set for targets.

Also: in the board directory, you can add a "warn.txt" file
which will display a message. For example, if a board has a
particular quirk to watch out for, write that there. The message
will be printed during the build process, to stdout.

If status is anything *other* than stable, or it is unstable
but CBMK_VERSION_TYPE is not set to "unstable", and not building
a release, a confirmation is passed.

If the board is not specified as stable or unstable, during
a release build, the build is skipped and the ROM is not
provided in that release; this is in *addition* to
release="n" or release="y" that can be set in target.cfg,
which will skip the release build for that target if "n"

Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-03 00:45:50 +01:00
Leah Rowe c8bb465d1a build/roms: add missing deletion of tmp file
the temporary rom per build was not being deleted after
finishing the current target. this adds up in /tmp during
large builds, when building for many targets. fix this!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-02 23:17:45 +01:00
Leah Rowe a1ac872240 permit excluding certain coreboot roms on releases
release="n" can be set in target.cfg on coreboot targets

script/update/release exports CBMK_RELEASE="y"

script/build/roms skips building a given target if release="n"
in target.cfg *and* CBMK_RELEASE="y"

you could also do the export yourself before running ./build roms,
for example:

export CBMK_RELEASE="y"
./build roms all

this will be used in subsequent revisions, to exclude certain
targets from the next canoeboot release.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-02 23:15:50 +01:00
Leah Rowe 187a3ea484 allow users to specify number of build threads
cbmk otherwise uses nproc to set the number of build threads,
in these places:

* generic make commands in script/update/trees
* crossgcc make command in script/update/trees

the -T0 option is also used in script/update/release, when running
tar.

with this change, you can do:

export CBMK_THREADS=x

where x is the number of threads. when you then run
cbmk, your chosen number of threads will override
the default. this may be useful on a host that does
not have a lot of memory.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-02 22:44:59 +01:00
Leah Rowe 226b2301db safer, simpler error handling in cbmk
in shell scripts, a function named the same as a program included in
the $PATH will override that program. for example, you could make a
function called ls() and this would override the standand "ls".

in cbmk, a part of it was first trying to run the "fail" command,
deferring to "err", because some scripts call fail() which does
some minor cleanup before calling err.

in most cases, fail() is not defined, and it's possible that the user
could have a program called "fail" in their $PATH, the behaviour of
which we could not determine, and it could have disastrous effects.

cbmk error handling has been re-engineered in such a way that the
err function is defined in a variable, which defaults to err_ which
calls err_, so defined under include/err.sh.

in functions that require cleanup prior to error handling, a fail()
function is still defined, and err is overridden, thus:

err="fail"

this change has made xx_() obsolete, so now only x_ is used. the x_
function is a wrapper that can be used to run a command and exit with
non-zero status (from cbmk) if the command fails. the xx_ command
did the same thing, but called fail() which would have called err();
now everything is $err

example:

	rm -f "$filename" || err "could not delete file"

this would now be:

	rm -f "$filename" || $err "could not delete file"

overriding of err= must be done *after* including err.sh. for
example:

err="fail"
. "include/err.sh"

^ this is wrong. instead, one must do:

. "include/err.sh"
err="fail"

this is because err is set as a global variable under err.sh

the new error handling is much cleaner, and safer. it also reduces
the chance of mistakes such as: calling err when you meant to
call fail. this is because the standard way is now to call $err,
so you set err="fail" at the top of the script and all is well.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-02 22:41:35 +01:00
Leah Rowe 5447f8be00 support making u-boot-only tarballs in releases
./update release -m u-boot

if someone just wants to make u-boot, they can
use this and it tars up all the trees.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-02-07 00:26:31 +00:00
Leah Rowe f1549872b8 update/trees: reset xtree/tree_depend before build
in some cases, the build system was needlessly, and sometimes
erroneously, creating crossgcc symlinks, which then caused an
issue, namely:

in lbmk release builds, dell e6400 is build before fam15h boards,
and it sets xtree, but fam15h_rdimm doesn't, and later this would
cause fam15h_rdimm boards to use xtree="default" (because they don't
set xtree), causing the newer toolchain to be used on coreboot 4.11.

this patch fixes the issue. quite a simple problem, actually.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-26 09:31:52 +00:00
Leah Rowe 5f9ed1e130 don't download projects on release archives
the changelog file is only present in releases, so
use the presence of this file for the test.

someone who wants to fetch projects within a release
archive can simply use the git repo, or delete the file.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-21 22:12:26 +00:00
Leah Rowe aa11288a09 update/release: generate changelogs
use the git log, as follows:

git log --graph --pretty=format:'%Cred%h%Creset %s %Creset' --abbrev-commit

this creates a nice, uniform list of changes.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-21 22:06:39 +00:00
Leah Rowe 2e6dec0c27 allow multitree projects to define xgcc tree
let them specific it, rather than falling back
to coreboot/default (can also be used for coreboot boards)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-21 06:28:08 +00:00
Leah Rowe 849eb62d9c GRUB: insert only 1 keymap per board, in cbfs
There is no need to add multiple keymap files, because
GRUB can load keymaps from CBFS. The current build logic
is designed to avoid building multiple GRUB binaries,
which are expensive computationally because each one
would then have to be compressed for each board.

This patch provides the best of both worlds: less space
used in flash like in the old lbmk design (1 keymap per
board), but retaining the current build speeds and therefore
not re-introducing the slowness of lbmk's previous GRUB
build logic.

The grub.cfg file has been modified, accordingly. It now
only loads a keymap.gkb file from CBFS, by default. It does
this, only if that file exists; if not, GRUB already defaults
to US Qwerty layout anyway.

ALSO: compress all keymap gkb files with xz -6

GRUB automatically decompresses files when accessed.
This results in about 2KB of flash space saved in CBFS.

Here is real-world data, showing the increased flash space:

< fallback/payload               0x3eb80    simple elf     548821 none
< keymap.cfg                     0xc4bc0    raw                16 none
< (empty)                        0xc4c00    null         11633316 none
---
> fallback/payload               0x3eb80    simple elf     546787 none
> keymap.gkb                     0xc43c0    raw               344 none
> (empty)                        0xc4540    null         11635044 none

This was taken by diffing the cbfstool "print" output,
both before and after. The *after* result is with this change.
11633316. In this example, 1728 bytes have been saved. Therefore,
with compression taken into account, this patch saves about 1.7KB
of space in CBFS.

This change means that lbmk can now scale to support hundreds
of keymaps, without increasing the amount of flash space used,
in each given image. Since the keymap files are compressed in
lbmk.git, in advance, we spend no additional time on compression
at build time. The resulting change in build speed in negligible.

Adding your own keymap.gkb file was already possible, for changing
the keymap in libreboot images, if you didn't want to change the
memdisk (and thus re-compile grub.elf). Now, this is the default
behaviour, and the only way to do it. It's much more efficient.

The original keymap files can be restored, by running unxz.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-21 00:15:36 +00:00
Leah Rowe 1d4e757253 build/roms: err if -k layout doesn't exist
if the user defines a layout that doesn't exist, throw
an error in lbmk.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-21 00:15:25 +00:00
Leah Rowe acf3d81ccf build/roms: regression fix: uninitialised variable
the "kmapdir" variable was removed in an earlier audit,
but was overlooked for -k because that option was untested.

rather than initialise the variable, re-use grubcfgsdir.
this fix enables e.g. "-k usdvorak" to work again.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-21 00:15:20 +00:00
Leah Rowe 102ce12cea rebase cbmk 9429287 per lbmk c4d90087..f5b04fa5
cbmk 9429287 is the present canoeboot revision, on this day,
two commits after canoeboot 20231107

the cbmk revision was based on lbmk c4d90087, but lbmk
has developed a lot since, right up to f5b04fa5. lbmk
c4d90087 was four commits after libreboot 20231106

this patch brings cbmk up to date, versus lbmk f5b04fa5,
which is 135 commits after libreboot 20231106 (not 4)

therefore, the next canoeboot release shall import lbmk
changes made *after* lbmk revision f5b04fa5. good day!

In English (the above is for my reference, next time
I make a new canoeboot release):

This imports all of the numerous improvements from
Libreboot, sans the non-FSDG-compliant changes. You
can find a full list of such changes in the audit4 page:

https://libreboot.org/news/audit4.html

A full canoeboot-ised changelog will be available in
the next canoeboot release, with these and subsequent
changes. Most notable here is the update to the new
GRUB 2.12 release (instead of 2.12-rc1), and the
improvements Riku made to pico-serprog. And the build
system improvements from lbmk, such as improved, more
generic cmake and autoconf handling.

Canoeboot-specific changes: I also tweaked the deblob
logic, to make it less error-prone. The new design
changes imported into cbmk (based on latest lbmk) somewhat
broke the deblob logic; it was constantly reminding the
user that blobs.list was missing for coreboot,
at config/coreboot/blobs.list - coreboot is a multi-tree
project in both cbmk and lbmk, and the deblob logic was
tuned for single/multi, but was treating coreboot as both.
for simplicity, i removed the check for whether blobs.list
is present. this means that the operator must ensure that
these files are present, in any given revision, where they
are required on a given set of projects (and the files are
all present, in this update to cbmk)

Also of note: the grub.cfg improvements are included in this
cbmk update. The improved grub.cfg can find grub/syslinux
configs by default, not just grub anymore, also finds extlinux,
and will also find them on EFI System Partition - in addition,
UEFI-based install media is also more robust; although cbmk
doesn't provide UEFI configurations on x86, our GRUB palyoad
does still need to work with distro install media, and many
of them now use UEFI-based GRUB configurations in their
installation media, which just happen to work with our GRUB

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-02 11:55:45 +00:00
Leah Rowe 6adbc8680f Canoeboot 20231107 release
changes after libreboot 20231101 were imported,
up to libreboot 20231106, and then to revision:
c4d90087535617d4fb31ca94803f9426010cfec5

i945 and gm45 configs were re-done, and dell e6400
was moved to its own tree with the ddr2 fix moved
there, to prevent breakage on ddr3-based gm45 boards
(look at libreboot 20231106 for more info)

several fixes are present in this canoeboot release,
that were only merged in libreboot *after* the
libreboot 20231106 release, and they are:

* c4d90087 add grub mods: diskfilter,hashsum,loadenv,setjmp
* d0d6decb re-add grub modules: f2fs, json, read, scsi, sleep
* 86608721 nvmutil: print usage
* f12f5c3a nvmutil: fix makefile

the release documentation has also been updated,
pulling down newer cbwww and cbwww-img based on
the new canoeboot 20231107 release announcement

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-11-07 17:19:04 +00:00
Leah Rowe 75636d4338 Canoeboot 20231103
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-11-03 18:37:59 +00:00
Leah Rowe 8db1b6da37 delete overlooked blobs in src/pico-sdk
thanks go to craig topman for finding these!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-11-03 17:40:08 +00:00
Leah Rowe b78ac6bf83 Canoeboot 20231101
Pull all suitable changes from Libreboot 20231101.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-11-01 16:34:22 +00:00
Leah Rowe 58ec3ca34f Canoeboot 20231026 release
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-27 08:21:04 +01:00