The build system already deletes .git in all source
directories for each given release, but does so at
the very end; it still does, but now it is deleted
one by one per project, to save space during very
large builds (release sizes vary wildly, depending
on how many trees exist for coreboot basically).
If you're building entirely in tmpfs (as I do), this
could be a problem if you have lots of .git/ directories.
This change reduces disk usage, or in the above example,
memory usage when running the build system from tmpfs.
This complements another recent change, where ROM images
are compressed per target during release builds, rather
than all at the very end of the process. It is part of a
series of optimisations, to reduce the memory and disk
usage of the build system, and to reduce I/O wastage
in general.
This change will not be the last of such changes!
Signed-off-by: Leah Rowe <leah@libreboot.org>
an equivalent change has been made in lbmk.
certain cbmk-specific variable names have been made
generic, with certain functions and other variables
moved around.
i maintain sync between libreboot and canoeboot, where
both projects can have the same behaviours, and most of
the merge conflicts have to do with variable names
containing "LBMK", "lbmk", "cbmk" or "CBMK", or
indeed "canoeboot" and "libreboot"
LBMK/lbmk/CBMK/cbmk variables between canoeboot and
libreboot now contain the string XBMK/xbmk
it should now be *much* easier to merge build system
changes between lbmk and cbmk.
Signed-off-by: Leah Rowe <info@minifree.org>
i always say, code should never document itself.
that's what documentation is for. the releases
contain documentation under docs/ but the git
repository does not; for that, use the website.
(in practise, cbmk usually needs internet anyway)
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's only used from main() in the main build script,
and it's very small, as is main()
therefore, move the logic into main()
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's a pointless feature, initially added just to one-up
gnuboot and only intended for canoeboot, to provide u-boot
tarballs with deblobbing. this was done, because the parabola
build system has certain limitations so the idea is to provide
them with tarballs. but why? they can just fix their build system...
delete this bloat from cbmk. we only need to provide full sources,
and rom images.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in cbmk, we call check_project() to set variables
such as projectname, version, version date
this is unnecessary, because all main scripts use
this functionality anyway
do it by default
Signed-off-by: Leah Rowe <leah@libreboot.org>
this function is not needed, because it's only called
once and it's very small.
furthermore: insert_version_files does ntot need to be called here,
because they same files are generated immediately afterward when
running that version of lbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
export CBMK_RELEASE="y"
if this is done, the tarball is created instead
of a directory, and the rom images are nuked using
./vendor inject with the nuke option, inserting the
correct version files; the rom directory is deleted
now the release script logic simple renames existing
tarballs. the benefit of this change is fewer lines of
code, and now cbmk doesn't use an insane amount of disk
space when building a *lot* of release images (the
uncompressed directories are deleted after each build)
Signed-off-by: Leah Rowe <leah@libreboot.org>
the release variable is all we need, turning a target on
or off for a given release.
the status checks were prone to bugs, and unnecessary; it
also broke certain benchmark scripts.
it's better to keep the cbmk logic simpler. board status
will be moved to the documentation instead.
Signed-off-by: Leah Rowe <leah@libreboot.org>
there are only two scripts under script/ now, and there
probably won't be many more. cbmk's design has simplified
to such a degree that the two-level directory structure is
no longer necessary.
the existing command structure has not changed. for example:
./build roms list
./update trees -f coreboot default
these will still work, but the symlinks to "build" are now
strictly for backwards compatibility; they may be removed
at a later date, but i'll keep the current design for now.
this also leads to a quirk, for example:
./build roms all
./update roms all
these now do the exact same thing, whereas "./update roms all"
would have previously been an invalid command.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the main script isn't that big, and since the main
purpose of cbmk is geared toward the releases, it
makes sense to reduce the number of scripts by
merging into the main one
the way this works, "./update release" still works
afterward
so, the way cbmk is used shall remain unchanged
Signed-off-by: Leah Rowe <leah@libreboot.org>
this is a follow-up to the previous commit
again, there's no posix way to check the path to the
file at argument 0, because readlink (utility) isn't
defined in posix (the C function is defined, but not
the utility included on many unices)
check whether "build" (file) exists, and whether it
is a symlink; if the latter, then we are definitely
not in the cbmk work directory!
Signed-off-by: Leah Rowe <leah@libreboot.org>
there's no portable(posix) way to check when running
from a symlink to cbmk in the current work directory
for example:
ln -s cbmk/build cbmktest
./cbmktest roms list
this would pass the new test, and first try to
include option.sh. in practise, the user probably doesn't
happen to have include/option.sh in their current path
i can use readlink here, but again not portable
the current check will suffice. it also works when
the symlink is called from $PATH
e.g. /usr/bin/cbmktest exists and you do:
cbmktest roms list
Signed-off-by: Leah Rowe <leah@libreboot.org>
Signed-off-by: Leah Rowe <info@minifree.org>
LC_COLLATE and CBMK_RELEASE are important variables. we want
to make sure that these are seen by everything.
since err.sh is included from all scripts, doing it there will
accomplish just that.
Signed-off-by: Leah Rowe <leah@libreboot.org>
cbmk sets TMPDIR to /tmp, and then creates a tmpdir, then
exports *that* as the value of TMPDIR. this unified TMPDIR
location then contains all subsequent files and directories,
when any script or program makes use of /tmp, via mktemp. at
least, that's the theory!
in practise, because it was only being properly exported from
the main build scripts, subscripts that are then called were
not exporting it, at least that is my assumption because in
some cases, i found that the coreboot build system was leaving
errant files behind outside of our own TMPDIR, and that build
system did not seem to be setting TMPDIR itself; more debugging
is needed.
anyway: use the exact same logic, but do it from err.sh. since
err.sh is included from every cbmk script, that means it will
always be exported when running every single part of cbmk. this
should reduce the chance that mktemp creates files and directories
outside of our custom TMPDIR location.
this is because in cbmk, we mitigate unhandled tmpdirs/files by
unifying it in the manner described, then deleting the entire
TMPDIR on exit from the main cbmk parent process (the main
script that the user called from, which is always the "build"
file).
in cbmk, effort is made to clean up temporary files properly,
without relying on this catch-all, but we can't rely on that.
the catch-all should also be as robust as possible.
Signed-off-by: Leah Rowe <info@minifree.org>
release="n" can be set in target.cfg on coreboot targets
script/update/release exports CBMK_RELEASE="y"
script/build/roms skips building a given target if release="n"
in target.cfg *and* CBMK_RELEASE="y"
you could also do the export yourself before running ./build roms,
for example:
export CBMK_RELEASE="y"
./build roms all
this will be used in subsequent revisions, to exclude certain
targets from the next canoeboot release.
Signed-off-by: Leah Rowe <info@minifree.org>
in shell scripts, a function named the same as a program included in
the $PATH will override that program. for example, you could make a
function called ls() and this would override the standand "ls".
in cbmk, a part of it was first trying to run the "fail" command,
deferring to "err", because some scripts call fail() which does
some minor cleanup before calling err.
in most cases, fail() is not defined, and it's possible that the user
could have a program called "fail" in their $PATH, the behaviour of
which we could not determine, and it could have disastrous effects.
cbmk error handling has been re-engineered in such a way that the
err function is defined in a variable, which defaults to err_ which
calls err_, so defined under include/err.sh.
in functions that require cleanup prior to error handling, a fail()
function is still defined, and err is overridden, thus:
err="fail"
this change has made xx_() obsolete, so now only x_ is used. the x_
function is a wrapper that can be used to run a command and exit with
non-zero status (from cbmk) if the command fails. the xx_ command
did the same thing, but called fail() which would have called err();
now everything is $err
example:
rm -f "$filename" || err "could not delete file"
this would now be:
rm -f "$filename" || $err "could not delete file"
overriding of err= must be done *after* including err.sh. for
example:
err="fail"
. "include/err.sh"
^ this is wrong. instead, one must do:
. "include/err.sh"
err="fail"
this is because err is set as a global variable under err.sh
the new error handling is much cleaner, and safer. it also reduces
the chance of mistakes such as: calling err when you meant to
call fail. this is because the standard way is now to call $err,
so you set err="fail" at the top of the script and all is well.
Signed-off-by: Leah Rowe <info@minifree.org>
cbmk 9429287 is the present canoeboot revision, on this day,
two commits after canoeboot 20231107
the cbmk revision was based on lbmk c4d90087, but lbmk
has developed a lot since, right up to f5b04fa5. lbmk
c4d90087 was four commits after libreboot 20231106
this patch brings cbmk up to date, versus lbmk f5b04fa5,
which is 135 commits after libreboot 20231106 (not 4)
therefore, the next canoeboot release shall import lbmk
changes made *after* lbmk revision f5b04fa5. good day!
In English (the above is for my reference, next time
I make a new canoeboot release):
This imports all of the numerous improvements from
Libreboot, sans the non-FSDG-compliant changes. You
can find a full list of such changes in the audit4 page:
https://libreboot.org/news/audit4.html
A full canoeboot-ised changelog will be available in
the next canoeboot release, with these and subsequent
changes. Most notable here is the update to the new
GRUB 2.12 release (instead of 2.12-rc1), and the
improvements Riku made to pico-serprog. And the build
system improvements from lbmk, such as improved, more
generic cmake and autoconf handling.
Canoeboot-specific changes: I also tweaked the deblob
logic, to make it less error-prone. The new design
changes imported into cbmk (based on latest lbmk) somewhat
broke the deblob logic; it was constantly reminding the
user that blobs.list was missing for coreboot,
at config/coreboot/blobs.list - coreboot is a multi-tree
project in both cbmk and lbmk, and the deblob logic was
tuned for single/multi, but was treating coreboot as both.
for simplicity, i removed the check for whether blobs.list
is present. this means that the operator must ensure that
these files are present, in any given revision, where they
are required on a given set of projects (and the files are
all present, in this update to cbmk)
Also of note: the grub.cfg improvements are included in this
cbmk update. The improved grub.cfg can find grub/syslinux
configs by default, not just grub anymore, also finds extlinux,
and will also find them on EFI System Partition - in addition,
UEFI-based install media is also more robust; although cbmk
doesn't provide UEFI configurations on x86, our GRUB palyoad
does still need to work with distro install media, and many
of them now use UEFI-based GRUB configurations in their
installation media, which just happen to work with our GRUB
Signed-off-by: Leah Rowe <leah@libreboot.org>
Leah Rowe