as stated in the previous commit, i'm adding this function
to lbmk because there are files i want to systematically
delete in libreboot releases, not just canoeboot releases,
but libreboot releases delete things such as unlicensed
readme files, or poorly licensed other files.
i initially moved the nuke function to its own file so as
to reduce the number of merge conflicts when merging
changes to git.sh between cbmk and lbmk, but if they're
going to both contain this file, then it makes sense
to have this in git.sh once again.
Signed-off-by: Leah Rowe <info@minifree.org>
blobs.list is now nuke.list
this is because i'm going to import this functionality
into lbmk (libreboot build system).
libreboot will not do full deblobbing like canoeboot does,
but there are still certain files that i like to delete
in releases, such as u-boot's strlcat.c file under tests
calling it "nukeblobs" in libreboot makes no sense, but
i like to avoid merge conflicts when cherry-picking
patches between cbmk and lbmk, so i like to make sure
that functions and variables common to both are named
the name.
simply calling it "nuke" or calling the files "nuke.list"
is probably inoffensive while conveying the same meaning.
Signed-off-by: Leah Rowe <info@minifree.org>
the inject script is from lbmk and referenced here, but
cbmk does not have this script.
remove the reference.
Signed-off-by: Leah Rowe <info@minifree.org>
it wasn't being reset before. when coreboot is being
built, i add to makeargs every time. if multiple targets
are being built, the make command would end up looking
something like:
make -C src/coreboot/default UPDATED_SUBMODULES=1 \
UPDATED_SUBMODULES=1
(the parameter would be printed twice)
of course, this doesn't check whether that parameter is
added already in target.cfg for a given target, but that's
ok because i won't add that one in target.cfg
i baked it into the code, only when handling coreboot,
because that was easier than either putting it in makeargs
for every coreboot target.cfg, or again modifying the code to
handle that; the current solution is the cleanest.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we do not want submodules to be downloaded after the fact.
we only handle this on ./update trees -f coreboot
Signed-off-by: Leah Rowe <leah@libreboot.org>
i'm importing some changes from lbmk and they go at the
end of git.sh, in the diffs. moving the deblob function
to its own file will allow me to cherry-pick with fewer
merge conflicts.
Signed-off-by: Leah Rowe <info@minifree.org>
firstly, memtest86+ is currently not cross compiled and
relies on 64-bit headers (x86_64 only). a 32-bit distro
is unlikely to be able to build 64-bit binaries.
secondly: vboot throws a build error due to -Werror when
building on 32-bit hosts. we rely on vboot code to build
cbfstool, so turn off -Werror on vboot
that's all. 32-bit hosts are not recommended; it is assumed
that you are building on an x86_64 host. work will go into
the build system at a later date to make it more portable,
by cross compiling everything, but this should fix 32-bit
for now.
there are some x60/t60 users who still want to build roms,
so let's allow them that possibility.
Signed-off-by: Leah Rowe <leah@libreboot.org>
an equivalent change has been made in lbmk.
certain cbmk-specific variable names have been made
generic, with certain functions and other variables
moved around.
i maintain sync between libreboot and canoeboot, where
both projects can have the same behaviours, and most of
the merge conflicts have to do with variable names
containing "LBMK", "lbmk", "cbmk" or "CBMK", or
indeed "canoeboot" and "libreboot"
LBMK/lbmk/CBMK/cbmk variables between canoeboot and
libreboot now contain the string XBMK/xbmk
it should now be *much* easier to merge build system
changes between lbmk and cbmk.
Signed-off-by: Leah Rowe <info@minifree.org>
i always say, code should never document itself.
that's what documentation is for. the releases
contain documentation under docs/ but the git
repository does not; for that, use the website.
(in practise, cbmk usually needs internet anyway)
Signed-off-by: Leah Rowe <leah@libreboot.org>
in cbmk, we call check_project() to set variables
such as projectname, version, version date
this is unnecessary, because all main scripts use
this functionality anyway
do it by default
Signed-off-by: Leah Rowe <leah@libreboot.org>
export CBMK_RELEASE="y"
if this is done, the tarball is created instead
of a directory, and the rom images are nuked using
./vendor inject with the nuke option, inserting the
correct version files; the rom directory is deleted
now the release script logic simple renames existing
tarballs. the benefit of this change is fewer lines of
code, and now cbmk doesn't use an insane amount of disk
space when building a *lot* of release images (the
uncompressed directories are deleted after each build)
Signed-off-by: Leah Rowe <leah@libreboot.org>
the release variable is all we need, turning a target on
or off for a given release.
the status checks were prone to bugs, and unnecessary; it
also broke certain benchmark scripts.
it's better to keep the cbmk logic simpler. board status
will be moved to the documentation instead.
Signed-off-by: Leah Rowe <leah@libreboot.org>
there are only two scripts under script/ now, and there
probably won't be many more. cbmk's design has simplified
to such a degree that the two-level directory structure is
no longer necessary.
the existing command structure has not changed. for example:
./build roms list
./update trees -f coreboot default
these will still work, but the symlinks to "build" are now
strictly for backwards compatibility; they may be removed
at a later date, but i'll keep the current design for now.
this also leads to a quirk, for example:
./build roms all
./update roms all
these now do the exact same thing, whereas "./update roms all"
would have previously been an invalid command.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the main script isn't that big, and since the main
purpose of cbmk is geared toward the releases, it
makes sense to reduce the number of scripts by
merging into the main one
the way this works, "./update release" still works
afterward
so, the way cbmk is used shall remain unchanged
Signed-off-by: Leah Rowe <leah@libreboot.org>
previous command:
./build serprog
now it is:
./build roms serprog
after that, it's the same arguments e.g.
./build roms serprog stm32
./build roms serprog rp2040
further cleanup to commence
Signed-off-by: Leah Rowe <leah@libreboot.org>
x is part of the for loop in main() and may or not
still be available from handle_target, depending on
your implementation of sh, but this should not be assumed
do it properly
Signed-off-by: Leah Rowe <leah@libreboot.org>
for example:
./build roms list stable
this lists all images that are marked "stable"
now:
./build roms list _stable
this lists all images that are *not* marked stable
this will help me keep track during development
Signed-off-by: Leah Rowe <leah@libreboot.org>
This is useful on desktops, where you want GRUB to
automatically start, but you still want access to the
GRUB menu, in the case where you rely on SeaBIOS to
execute the VGA ROM inside your graphics card.
Signed-off-by: Leah Rowe <leah@libreboot.org>
just to ensure that nothing goes wrong. we don't rely on
the status variable for releases, because there is another
variable, release, that target.cfg files declare, e.g.
release="n"
release="y"
you can just omit the variable, because it defaults to y, so
you only need declare it when it needs to be "n"
Signed-off-by: Leah Rowe <leah@libreboot.org>
export CBMK_STATUS=n
if not set, the status checks and confirmation dialogs
persist. if set to y they persist.
if you set it to n, all checks are disabled, so e.g.:
./build roms all
this would once again build all targets, regardless
of status. this is if you want the old behaviour.
Signed-off-by: Leah Rowe <leah@libreboot.org>
for example:
./build roms list
this will list every now, still. same behaviour. now see:
./build roms list stable
this will list all stable roms
./build roms list untested
this lists untested roms. but wait!
./build roms list untested broken unstable
./build roms list broken unstable
yes. it works this way. now you can use lbmk to easily
see what rom status are, during maintenance.
Signed-off-by: Leah Rowe <leah@libreboot.org>
export CBMK_VERSION_TYPE=x
x can be: stable, unstable
in target.cfg files, specify:
status=x
x can be: stable, unstable, broken, untested
if unset, cbmk defaults to "unknown"
if CBMK_VERSION_TYPE is set, no confirmation is asked
if the given target matches what's set (but what's set
in that environmental variable can only be stable or
unstable)
if CBMK_RELEASE="y", no confirmation is asked, unless
the target is something other than stable/unstable
"unstable" means it works, but has a few non-breaking
bugs, e.g. broken s3 on dell e6400
whereas, if raminit regularly fails or it is so absolutely
unreliable as to be unusable, then the board should be
declared "broken"
untested means: it has not been tested
With this change, it should now be easier to track whether
a given board is tested, in preparation for releases. When
working on trees/boards, status can be set for targets.
Also: in the board directory, you can add a "warn.txt" file
which will display a message. For example, if a board has a
particular quirk to watch out for, write that there. The message
will be printed during the build process, to stdout.
If status is anything *other* than stable, or it is unstable
but CBMK_VERSION_TYPE is not set to "unstable", and not building
a release, a confirmation is passed.
If the board is not specified as stable or unstable, during
a release build, the build is skipped and the ROM is not
provided in that release; this is in *addition* to
release="n" or release="y" that can be set in target.cfg,
which will skip the release build for that target if "n"
Signed-off-by: Leah Rowe <info@minifree.org>
the temporary rom per build was not being deleted after
finishing the current target. this adds up in /tmp during
large builds, when building for many targets. fix this!
Signed-off-by: Leah Rowe <leah@libreboot.org>
release="n" can be set in target.cfg on coreboot targets
script/update/release exports CBMK_RELEASE="y"
script/build/roms skips building a given target if release="n"
in target.cfg *and* CBMK_RELEASE="y"
you could also do the export yourself before running ./build roms,
for example:
export CBMK_RELEASE="y"
./build roms all
this will be used in subsequent revisions, to exclude certain
targets from the next canoeboot release.
Signed-off-by: Leah Rowe <info@minifree.org>
cbmk otherwise uses nproc to set the number of build threads,
in these places:
* generic make commands in script/update/trees
* crossgcc make command in script/update/trees
the -T0 option is also used in script/update/release, when running
tar.
with this change, you can do:
export CBMK_THREADS=x
where x is the number of threads. when you then run
cbmk, your chosen number of threads will override
the default. this may be useful on a host that does
not have a lot of memory.
Signed-off-by: Leah Rowe <info@minifree.org>
in shell scripts, a function named the same as a program included in
the $PATH will override that program. for example, you could make a
function called ls() and this would override the standand "ls".
in cbmk, a part of it was first trying to run the "fail" command,
deferring to "err", because some scripts call fail() which does
some minor cleanup before calling err.
in most cases, fail() is not defined, and it's possible that the user
could have a program called "fail" in their $PATH, the behaviour of
which we could not determine, and it could have disastrous effects.
cbmk error handling has been re-engineered in such a way that the
err function is defined in a variable, which defaults to err_ which
calls err_, so defined under include/err.sh.
in functions that require cleanup prior to error handling, a fail()
function is still defined, and err is overridden, thus:
err="fail"
this change has made xx_() obsolete, so now only x_ is used. the x_
function is a wrapper that can be used to run a command and exit with
non-zero status (from cbmk) if the command fails. the xx_ command
did the same thing, but called fail() which would have called err();
now everything is $err
example:
rm -f "$filename" || err "could not delete file"
this would now be:
rm -f "$filename" || $err "could not delete file"
overriding of err= must be done *after* including err.sh. for
example:
err="fail"
. "include/err.sh"
^ this is wrong. instead, one must do:
. "include/err.sh"
err="fail"
this is because err is set as a global variable under err.sh
the new error handling is much cleaner, and safer. it also reduces
the chance of mistakes such as: calling err when you meant to
call fail. this is because the standard way is now to call $err,
so you set err="fail" at the top of the script and all is well.
Signed-off-by: Leah Rowe <info@minifree.org>
./update release -m u-boot
if someone just wants to make u-boot, they can
use this and it tars up all the trees.
Signed-off-by: Leah Rowe <info@minifree.org>
in some cases, the build system was needlessly, and sometimes
erroneously, creating crossgcc symlinks, which then caused an
issue, namely:
in lbmk release builds, dell e6400 is build before fam15h boards,
and it sets xtree, but fam15h_rdimm doesn't, and later this would
cause fam15h_rdimm boards to use xtree="default" (because they don't
set xtree), causing the newer toolchain to be used on coreboot 4.11.
this patch fixes the issue. quite a simple problem, actually.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the changelog file is only present in releases, so
use the presence of this file for the test.
someone who wants to fetch projects within a release
archive can simply use the git repo, or delete the file.
Signed-off-by: Leah Rowe <leah@libreboot.org>
use the git log, as follows:
git log --graph --pretty=format:'%Cred%h%Creset %s %Creset' --abbrev-commit
this creates a nice, uniform list of changes.
Signed-off-by: Leah Rowe <leah@libreboot.org>
let them specific it, rather than falling back
to coreboot/default (can also be used for coreboot boards)
Signed-off-by: Leah Rowe <leah@libreboot.org>
There is no need to add multiple keymap files, because
GRUB can load keymaps from CBFS. The current build logic
is designed to avoid building multiple GRUB binaries,
which are expensive computationally because each one
would then have to be compressed for each board.
This patch provides the best of both worlds: less space
used in flash like in the old lbmk design (1 keymap per
board), but retaining the current build speeds and therefore
not re-introducing the slowness of lbmk's previous GRUB
build logic.
The grub.cfg file has been modified, accordingly. It now
only loads a keymap.gkb file from CBFS, by default. It does
this, only if that file exists; if not, GRUB already defaults
to US Qwerty layout anyway.
ALSO: compress all keymap gkb files with xz -6
GRUB automatically decompresses files when accessed.
This results in about 2KB of flash space saved in CBFS.
Here is real-world data, showing the increased flash space:
< fallback/payload 0x3eb80 simple elf 548821 none
< keymap.cfg 0xc4bc0 raw 16 none
< (empty) 0xc4c00 null 11633316 none
---
> fallback/payload 0x3eb80 simple elf 546787 none
> keymap.gkb 0xc43c0 raw 344 none
> (empty) 0xc4540 null 11635044 none
This was taken by diffing the cbfstool "print" output,
both before and after. The *after* result is with this change.
11633316. In this example, 1728 bytes have been saved. Therefore,
with compression taken into account, this patch saves about 1.7KB
of space in CBFS.
This change means that lbmk can now scale to support hundreds
of keymaps, without increasing the amount of flash space used,
in each given image. Since the keymap files are compressed in
lbmk.git, in advance, we spend no additional time on compression
at build time. The resulting change in build speed in negligible.
Adding your own keymap.gkb file was already possible, for changing
the keymap in libreboot images, if you didn't want to change the
memdisk (and thus re-compile grub.elf). Now, this is the default
behaviour, and the only way to do it. It's much more efficient.
The original keymap files can be restored, by running unxz.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the "kmapdir" variable was removed in an earlier audit,
but was overlooked for -k because that option was untested.
rather than initialise the variable, re-use grubcfgsdir.
this fix enables e.g. "-k usdvorak" to work again.
Signed-off-by: Leah Rowe <leah@libreboot.org>
cbmk 9429287 is the present canoeboot revision, on this day,
two commits after canoeboot 20231107
the cbmk revision was based on lbmk c4d90087, but lbmk
has developed a lot since, right up to f5b04fa5. lbmk
c4d90087 was four commits after libreboot 20231106
this patch brings cbmk up to date, versus lbmk f5b04fa5,
which is 135 commits after libreboot 20231106 (not 4)
therefore, the next canoeboot release shall import lbmk
changes made *after* lbmk revision f5b04fa5. good day!
In English (the above is for my reference, next time
I make a new canoeboot release):
This imports all of the numerous improvements from
Libreboot, sans the non-FSDG-compliant changes. You
can find a full list of such changes in the audit4 page:
https://libreboot.org/news/audit4.html
A full canoeboot-ised changelog will be available in
the next canoeboot release, with these and subsequent
changes. Most notable here is the update to the new
GRUB 2.12 release (instead of 2.12-rc1), and the
improvements Riku made to pico-serprog. And the build
system improvements from lbmk, such as improved, more
generic cmake and autoconf handling.
Canoeboot-specific changes: I also tweaked the deblob
logic, to make it less error-prone. The new design
changes imported into cbmk (based on latest lbmk) somewhat
broke the deblob logic; it was constantly reminding the
user that blobs.list was missing for coreboot,
at config/coreboot/blobs.list - coreboot is a multi-tree
project in both cbmk and lbmk, and the deblob logic was
tuned for single/multi, but was treating coreboot as both.
for simplicity, i removed the check for whether blobs.list
is present. this means that the operator must ensure that
these files are present, in any given revision, where they
are required on a given set of projects (and the files are
all present, in this update to cbmk)
Also of note: the grub.cfg improvements are included in this
cbmk update. The improved grub.cfg can find grub/syslinux
configs by default, not just grub anymore, also finds extlinux,
and will also find them on EFI System Partition - in addition,
UEFI-based install media is also more robust; although cbmk
doesn't provide UEFI configurations on x86, our GRUB palyoad
does still need to work with distro install media, and many
of them now use UEFI-based GRUB configurations in their
installation media, which just happen to work with our GRUB
Signed-off-by: Leah Rowe <leah@libreboot.org>
changes after libreboot 20231101 were imported,
up to libreboot 20231106, and then to revision:
c4d90087535617d4fb31ca94803f9426010cfec5
i945 and gm45 configs were re-done, and dell e6400
was moved to its own tree with the ddr2 fix moved
there, to prevent breakage on ddr3-based gm45 boards
(look at libreboot 20231106 for more info)
several fixes are present in this canoeboot release,
that were only merged in libreboot *after* the
libreboot 20231106 release, and they are:
* c4d90087 add grub mods: diskfilter,hashsum,loadenv,setjmp
* d0d6decb re-add grub modules: f2fs, json, read, scsi, sleep
* 86608721 nvmutil: print usage
* f12f5c3a nvmutil: fix makefile
the release documentation has also been updated,
pulling down newer cbwww and cbwww-img based on
the new canoeboot 20231107 release announcement
Signed-off-by: Leah Rowe <leah@libreboot.org>
Leah Rowe