this is in prep for the next change, where non-init
functions will be moved to another file, again named
include/lib.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
a lot of init code was handled outside of any function. the
coding style used in the rest of the build system has now
been introduced, with xbmk_init being the main function.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this was used alongside the xgcc linking, so that coreboot
trees could specify that another tree was to be downloaded.
since this variable will no longer be used, it should be
removed, to avoid dead code bloat.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the "xtree" variable is used by projects such as u-boot,
to export a CROSS_COMPILE variable specifying prefix for
gnu compilers, and for building the named coreboot tree.
for example, xtree can be "default", which is then the
coreboot tree downloaded, for use of crossgcc.
however, it is also used to symlink identical versions
of crossgcc between coreboot trees. this latter feature
was only needed for fam15h boards which were previously
split between two mostly identical coreboot trees, that
were later merged into a single tree, and this feature
is therefore no longer used.
remove this dead code, to reduce bloat in the build system.
Signed-off-by: Leah Rowe <leah@libreboot.org>
In the previous revision, I make hardcoded use of
/usr/local/bin and /usr/bin as search locations, instead
of relying on PATH, when the user has a python venv, because
in those cases, we cannot rely on PATH so we use a python
command to detect the venv and then force use of the
normal system path for python.
However, there's no guarantee that the real Python will
indeed live at these locations. For example, some distros
like Nix or Guix will use many locations for different
versions of a given package, and it's for the birds as to
what given package version the user might be running.
Therefore, this patch retains that current hardcoded
assumption of /usr/local/bin and /usr/bin but *only* as
a fallback solution, instead checking realpath first.
The "realpath" command isn't technically POSIX standard,
but in practise it is available on GNU coreutils, Busybox,
and the various BSD userlands.
I could perhaps *import* a realpath utility, and use that,
but this should be fine.
Signed-off-by: Leah Rowe <leah@libreboot.org>
If the user has a virtual environment, the current logic
will cause lbmk to hang. A useful workaround is to force
use of the direct path to the system binary of python.
This works by detecting a virtual environment first, and
deferring to the old behaviour if no venv is found. If one
is found, then it will not rely on PATH, but instead only
search the standard locations /usr/local/bin and /usr/bin.
Signed-off-by: Leah Rowe <leah@libreboot.org>
alper made a fix to this file a few hours ago, but
forgot to update the copyright header
i'm doing it for alper, as a courtesy
Signed-off-by: Leah Rowe <leah@libreboot.org>
Properly set $pyver to "3" when we detect we can use python3. In the
following version checks, use the $python we detected instead of a
'python' from PATH because the latter might be a python2 while still
co-existing with a python3.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
initialising variables, setting PWD, setting version,
this is all unnecessary before the root check, because
the dependencies commands use none of these.
Signed-off-by: Leah Rowe <leah@libreboot.org>
cbmk creates TMPDIR as /tmp/xbmk_*, but it's theoretically
possible that something could re-export it by mistake.
this change retains the same initialisation, but further
use is now via a new variable "xbmktmp", that stores the
value of TMPDIR upon cbmk's initialisation of it.
this reduces the chance of such a bug in the future, as
described above, so it is a preemptive/preventative fix.
Signed-off-by: Leah Rowe <leah@libreboot.org>
merge it with git_prep, since it's only a small
function and only called from there. the merged
code still makes sense and its purpose is still
quite clear on casual reading.
Signed-off-by: Leah Rowe <leah@libreboot.org>
merge it with git_prep, since it's only a tiny
function and only called from there. the for
loop moved to the if block still makes sense
on casual reading.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the "u" argument can actually be any thing. git_prep
handles git submodules only for single-tree projects,
under any candition, or on multi-tree projects if
the number of arguments to git_prep is above four.
"u" is the 5th argument, meant to enable submodule
downloads. it really doesn't matter what this string
says, so let's just make it as clear as possible.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the cbfs function will call cbfstool, which will perform
the same check, and the same error condition would cause
the same exit behaviour in lbmk. the error message would
also provide output that is just as useful for debugging.
Signed-off-by: Leah Rowe <leah@libreboot.org>
There was no error handling, *at all*, on the actual tar
command, due to the lack of set -o pipefail, which we cannot
rely on in sh.
The x_ wrapper can be used in this case, as a mitigation.
Signed-off-by: Leah Rowe <leah@libreboot.org>
x_ cannot be used, where output is redirected to a file;
only the convention piping can be used, for errors.
relying on x_ in these cases will cause unpredictable bugs.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i can't call $err (variable), because it's set
to fail_inject. fix this infinite loop, which
was an oversight in the previous commit.
Signed-off-by: Leah Rowe <leah@libreboot.org>
I was using a complicated method of knowing whether
the current instance was parent or a child, to know
whether the lock file and TMPDIR needed to be purged.
It was quite error-prone too. Instead, I'm now handling
it directly from within the if statement that previously
initialised xbmk_parent=y, forking ./mk from there.
The forked instance would not trigger that if clause
again, since then TMPDIR is created, thus avoiding
recursion.
This is an improvement because it doesn't rely on how
the parent handles exit statuses, and it ensures that
the lock/tmp files are never accidentally deleted.
Even if a given program/script that cbmk runs would
export TMPDIR, it doesn't matter because cbmk doesn't,
so it would be unaffected.
Signed-off-by: Leah Rowe <leah@libreboot.org>
because the top-down function order isn't as reliable
in lib.sh, since this is what first runs, included
in every other script
Signed-off-by: Leah Rowe <leah@libreboot.org>
write to .version and .versiondate, instead
of version and versiondate.
this will hide them to avoid visual clutter while
analysing files within cbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
instead of running pwd all the time, run it once in lib.sh,
and export PWD.
for cbmk-specific use of PWD, use xbmkpwd, which contains
the value of PWD as was set by the pwd utility in lib.sh.
many parts of cbmk rely on pwd, and it *must* be correct.
this change adds basic error handling, since pwd can in
fact return errors in some cases.
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's incorrect for PATH not to be set, but some users
may foolishly blank it out before running cbmk.
prevent such issues, by initialising it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
PWD could be anything, if the user manually exported
it before running cbmk.
always run pwd instead, to get the real string.
Signed-off-by: Leah Rowe <leah@libreboot.org>
several code lines were condensed together, which
make them less readable. make the code more readable
by having separate commands on separate lines.
i previously did this during my manic build system
audits of 2023 and 2024; condensing lines like this
is overly pedantic and serves no real purpose.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This is based on include/vendor.sh from this lbmk
revision:
3c9f4be76f61c80060b4238eff96ef268272cffb
This version doesn't support downloading/injecting
vendor files such as Intel ME; that's what the lbmk
version is for.
If you try to run this on a Libreboot archive that
uses vendor files, the script will see that there is
a hash file present, and not inject a new MAC.
HOWEVER: if the hash file is not present, it will
work just fine, but again only change the MAC. That
way, you can use the "./mk inject" command from lbmk,
to insert files such as Intel ME. In practise, due to
the design checking out a specific cbfstool version
based on the board config, you can only use a config
in this way that's present on both Libreboot and
Canoeboot, such as the E6400 images; the E6400 images
on Libreboot insert an Nvidia GPU ROM, but Canoeboot
does not.
You don't need to run this on Libreboot tarballs, because
the Libreboot version can be used anyway. Canoeboot is
mostly a pointless project, but I maintain it for fun. I
make it adhere to GNU FSDG for fun, even though I disagree
with it; Libreboot's binary blob reduction policy is better.
The reason for this design is because of GNU FSDG,
which Canoeboot complies with to the letter. It states
that any such project must not distribute, promote or
otherwise boost proprietary software in any way; it must
steer the user only towards entirely free software.
It also doesn't support nuking. It only sets MAC
addresses; the "setmac keep" command is not present,
because it's pointless, but these work, e.g.:
./mk inject tarball.tar.xz
./mk inject tarball.tar.xz setmac
./mk inject tarball.tar.xz setmac restore
./mk inject tarball.tar.xz MACADDRESS
./mk inject tarball.tar.xz ??:aa:bb:??:22:01
etc
Same command structure as setmac for lbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
We were previously not handling picotool at all, and
pico-sdk would download picotool itself, at build time.
This means that the source archive, if created, would
not contain picotool. While not strictly required, for
complete corresponding source, since it's a toolchain
and not the actual pico-serprog firmware, it is my policy
that releases must include full corresponding source code,
when it is feasible to do so.
I must say, I intensely dislike cmake, with such burning
passion; I am thoroughly displeased by how hacky this is,
but it works and now nothing is in my way for a Libreboot
20241206 rev8 release!
Signed-off-by: Leah Rowe <leah@libreboot.org>
See:
https://docs.python.org/3/library/sys.html#sys.version_info
The sys.version_info tuple is a more reliable way to
get the version. Our previous logic assumed that Python
would always output "Python versionnumber", but this may
not always be how it works. We've seen this for example
where Debian modifies some GNU toolchains to include Debian
something in the output.
Python has a standard method built in for outputting exact
the information we need. In my system, what I got was this:
(3, 11, 2, 'final', 0)
That output was from running this command:
python -c 'import sys; print(sys.version_info[:])'
This is much more robust, so use this instead.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we already check the python version, and set a variable
for it, so that we can reliably use python3, even if
python in PATH doesn't correspond to python3. for
example if a system has python as python2 and python3
as python3
well, we use that when running deguard for example, but
various upstream projects that we use may need python,
and all of them use python3, not 2
so, re-use the python variable set up by lbmk, and
set it up in PATH accordingly. this now makes the note
about python3 obsolete, on docs/build.md in lbwww.git
Signed-off-by: Leah Rowe <leah@libreboot.org>
it should fix more build errors that might have appeared
in the aforementioned revision, mentioned in the previous
commit message
Signed-off-by: Leah Rowe <leah@libreboot.org>
the bug was actually caused by chkvars
add an escape for the quotes and bam. fixed.
without this, i got the following e.g.
For command: ./mk dependencies debian
Output:
./mk: 1: [: apt-get: unexpected operator
ERROR ./mk: pkg_add unset
Someone reported a similar issue with the Arch one,
which is also now fixed. This regression was caused
by the previous commit:
commit 0cf58c22734b19293f4cbef83add59b031ca1773
Author: Leah Rowe <leah@libreboot.org>
Date: Thu Jan 2 23:52:45 2025 +0000
fix lbmk shellcheck errors
I forgot to escape the double quotes in an eval.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This check is a good idea, but not viable here,
because the modules naturally aren't set in all
circumstances, so it just causes a build error.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i also removed that printf, because the path it prints is
actually wrong sometimes; in the recent re-write of vendor.sh,
it prints the correct path instead
Signed-off-by: Leah Rowe <leah@libreboot.org>
There was also a condition in run_make_command that is now
an OR, where it was an AND, on script/trees, to fix the use
of mixed (and erroneous) OR/AND operators.
I'm planning a much more invasive audit than this. These are
light fixes, intended for Libreboot 20241206 rev8.
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't make sha512 files for tar archives, because it
is my intention to add the ./mk inject command to
canoeboot in a future commit, but without the vendor
file download/inject functionality, just the mac
address changer.
this commit is based on lbmk commit 41275d699ca
Signed-off-by: Leah Rowe <leah@libreboot.org>
./mk dependencies debian --reinstall
Add --reinstall and it'll do:
apt-get install --reinstall
This can be useful when updating from a stable release
to a testing release. The variable, "reinstall" can be
configured for other distros, but it's currently only
configured for Debian-based distros.
Also, it can be anything. For example, you could add -y;
however, a 4th argument will not be accepted. For example,
you cannot do:
./mk dependencies debian --reinstall -y
If you do this, it'll only see --reinstall; similarly, if
you did this command:
./mk dependencies debian -y --reinstall
then -y would be passed, but not --reinstall. This is an
intentional design decision, in case you accidentally pasted
or subshelled something that outputted something undesirable,
to prevent possible abuse.
Signed-off-by: Leah Rowe <leah@libreboot.org>
When doing ./mk release, the build system would create
symlinks inside xbmkpath/ relative to the current work tree,
which will differ from what's in PATH.
Since XBMK_CACHE is already set globally, from the main work
tree and the release-build work tree, that means we can know
reliably that PATH is always correct if we put xbmkpath/
inside XBMK_CACHE.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Remove all symlinks each time, to ensure that no
stragglers are left behind, since they are being
re-generated each time anyway.
The code for determining version numbers has now
been unified under gnu_setver()
Signed-off-by: Leah Rowe <leah@libreboot.org>
We were checking the shorthand version number, but
the precise version numbers need to match.
Also: when we searched $PATH/gnat-$gccver, we assumed
that the full version would then match, without checking
it, so now it is checked precisely.
Signed-off-by: Leah Rowe <leah@libreboot.org>
When doing e.g. $@ we should use double quotes to prevent globbing.
Thanks go to XRevan86 for pointing this out.
Signed-off-by: Leah Rowe <leah@libreboot.org>
When I tested Debian Trixie, and Debian Sid, I saw that
GCC in PATH pointed to gcc-14, but gnat in path pointed
to GNAT-13, even if you manually install gnat-14.
GNAT 14 was marked experimental, but GCC 14 was marked
for use, in the apt repositories.
So this patch doesn't address the mismatch when doing e.g.
apt-get install gcc gnat
I will address the actual package dependency in a follow-up
patch, on the Debian dependencies config.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Previously serprog_rp2040, but we now also support
the RP2530 boards.
Therefore, serprog_pico is a nice generic name. The
directory on release archives will now be serprog_pico
instead of serprog_rp2040; it will contain serprog images
for both RP2040 and RP2530 devices.
Signed-off-by: Leah Rowe <leah@libreboot.org>
rp2040 and rp2530 platforms can't share a cmake build directory. we
could just delete the build directory after every compilation, but that
would be really wasteful (every tool would need to be recomiled every
time. instead create new build directories as new plaforms are found
and symlink them to the point where the build directory used to be.
to find out which platform we're compiling for, we crudely parse the
board headers file.
there surely would be better ways to do this, but this hack works
with all the boards in pico-sdk 2.1.0.
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
set this variable in the tmpclone function. otherwise,
certain submodules might always download every time,
when handling multiple projects.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The exit was dependent upon install_packages returning
zero status, which it always would in practise, due to
its design, but this exit must always be observed, so
the code has been modified to honour this design.
A direct exit violates lbmk's design in most instances,
where a temporary directory and lock file has already
been created; at this stage, no such act was performed,
so a direct exit is perfectly acceptable.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in this setup, seabios is never the default payload, grub is,
but only if grub is enabled.
set this in target.cfg:
payload_grubsea="y"
if payload_grub isn't enabled, this is auto-set to n
ditto if initmode=normal
NOTE: if flashing libgfx setups, you should make sure
that you're not booting with a graphics card, only intel
graphics. this setting will intentionally not be documented,
because it's not recommended, but is being implemented for
testing purposes (and i implemented it for some guy who i
think is cool). i'll probably also use this myself, since
i already do grub-only setups on all my own machines.
seagrub is the default on x86 because of past instabilities
with grub. to mitigate in case of future issues, since seabios
is always stable, we reduce the chance of bricks.
Signed-off-by: Leah Rowe <leah@libreboot.org>
for some reason, when the background is in memdisk, inserting
it into cbfs afterward doesn't override, despite this
being the behaviour in grub.cfg
put it in cbfs explicitly, and skip inserting into memdisk
Signed-off-by: Leah Rowe <leah@libreboot.org>
Since U-Boot must be inserted at a specific offset, it's
theoretically possible that other files might overlap, but
cbfstool will work around wherever U-Boot was inserted if
it was inserted first; we don't use specific offsets for
the other files.
This is technically a preventative bug fix, but it fixes
a bug that would probably never occur in practise.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This is not a main script, and should not be treated as such;
it must never be directly executed by the user.
This script was only ever used inside other scripts, so the
shebang didn't seem to do much at all, but it shouldn't be
there anyway.
Remove it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The "normal" mode in lbmk is where no built-in GPU exists,
or no libgfxinit is used, and SeaBIOS is the first payload,
and SeaBIOS executes VGA ROMs (can't know if it'll start
in VESA or text mode).
U-Boot needs a VESA framebuffer or native coreboot
framebuffer to work correctly.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Same concept as SeaGRUB, but for U-Boot. SeaBIOS starts, but
has a bootorder file loading U-Boot first, from flash.
You can interrupt it with the ESC menu, to boot something else
in SeaBIOS, including GRUB.
With this, we can effectively provide extremely user-friendly
UEFI-first setups in Canoeboot.
Take that, edk2!
Signed-off-by: Leah Rowe <leah@libreboot.org>
For some reason, 32-bit U-Boot only works when executed from
GRUB, but not SeaBIOS; 64-bit U-Boot only works from SeaBIOS!
This will have to be investigated. Standalone U-Boot, where
U-Boot is the primary payload, has not yet been tested in
Libreboot, and will not be provided for some time due to
stability concerns. More testing is needed!
Signed-off-by: Leah Rowe <leah@libreboot.org>
Currently seems to stall when booted from the GRUB
payload, but works when booted from the SeaBIOS menu.
I also tested it as a standalone payload and it seems
to boot. Will test on hardware next, and start adding
it to more mainboards.
Signed-off-by: Leah Rowe <leah@libreboot.org>
NOTE: Support added for xarch target x86_64-elf,
but U-Boot failed to build with this error:
OBJCOPY lib/efi_loader/helloworld.efi
x86_64-elf-objcopy: lib/efi_loader/helloworld_efi.so: invalid bfd target
make[2]: *** [scripts/Makefile.lib:476: lib/efi_loader/helloworld.efi] Error 1
Since I'm building U-Boot for x86_64 *on* an x86-64
host, and since that is currently the recommended type
of machine to use for cbmk development, and since the
other x86 payloads currently don't cross compile anyway,
this is an acceptable compromise for now. This is because
at present, I'm not making U-Boot the primary payload on x86,
instead preferring to chain it from GRUB and SeaBIOS.
The target.cfg file for x86 u-boot shows xarch/xtree commented.
Uncomment these to compile on crossgcc instead of hostcc.
I mention 64-bit because I initially did this first, but decided
to do 32-bit first. I'll work on the 64-bit one next (SPL).
It's only enabled in QEMU for now.
Signed-off-by: Leah Rowe <leah@libreboot.org>
GRUB-as-primary was temporarily allowed in lbmk, because of
a temporary SeaBIOS bug on a machine that canoeboot doesn't
actually support yet, namely the 3050 Micro.
This same diff was also applied to lbmk, but lbmk also applied
changes to a coreboot config for the aforementioned mainboard.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the .git directory never exists anyway, when doing a release,
so the purpose this is intended is defeated by lbmk's design.
individual headers say "pcsx-redux team" as copyright anyway,
and the code for generating that COPYING file, with MIT license
and correct years (matching the entire source code for the
open bios) remains correct.
a mitigation instead of this patch might be to maintain a hardcoded
list of authors, and manually update it over time, but this is not
required. however, it may be good practise for upstream to maintain
such a file. perhaps i should contact them?
Signed-off-by: Leah Rowe <leah@libreboot.org>
I also checked the copyright declarations in the
directory src/mips/openbios where the PCSX-Redux BIOS
is, gleaning all the copyright years: 2019-2024 at this
time.
The years will be updated as and when PCSX-Redux is
updated in lbmk. Their BIOS is under MIT so I made lbmk
generate an appropriate COPYING file alongside the binary,
containing:
Copyright (c) 2019-2024 PCSX-Redux authors
Along with the actual text of the MIT license. With all
of this, the PCSX-Redux BIOS can now be included in
Libreboot releases.
No actual tarball is created. The release script in lbmk
simply copies the bin/ directory to ../roms
I'm leaving the PCSX-Redux BIOS release uncompressed,
because, and this will sound patronising because that is
my precise intention: Windows users don't know how to do
anything. If I provide a tarball to Windows users, they
won't know what to do. Libreboot releases always go on rsync
mirrors, which also have HTTP servers with indexing enabled,
for browsing release files.
I mention Windows users, because most people who use the PCSX
Redux BIOS will probably use it on a PlayStation emulator, and
most emulator users are on Windows. I can't really be bothered
to provide it as a .zip archive, and it's only 512kb, so just
provide it uncompressed in Libreboot releases!
Releases were already possible under this scheme, so this
patch really just adds the COPYING file. It's simply a courtesy
to the PCSX-Redux developers, providing proper credit to them.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the way it worked, the roms were still named seagrub
and the seabios rom would be compiled, but with the wrong
path, so seabios wouldn't be executed; seabios would hang
anyway, on this board.
instead, engineer it in such a way as to disable seabios_
images on such setups. also, rename seagrub_ to grub_.
i normally only permit seagrub, and not grub, but i make an
exception for 3050micro because we know grub works, but seabios
currently hangs on this board (which means no bsd).
dell optiplex 3050 micro isn't actually supported in canoeboot,
but the workaround patch for enabling grub as primary payload
was added so that cbmk will maintain parity with lbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Dell OptiPlex 3050 Micro support was added to Libreboot,
which we can't add in Canoeboot, but SeaBIOS hung on that
board so we made GRUB the primary payload on that board.
SeaGRUB is still enforced on all Canoeboot targets at
present, but we want cbmk to maintain parity with lbmk.
Therefore, import this functionality into cbmk, but without
actually using it.
Add two variable options for target.cfg files:
* seabiosname
* grubname
This string defines where it would be located in CBFS.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in some cases, on a fresh clone, the cached repo already
exists but lbmk tries to download it again. work around
this by checking that the directory exists; it's in the
main if statement, so that the "else" still applies. as
a result, the fallback to a live repo would un-fall back
to doing git-pull if the cached directory exists exists.
if it doesn't seem to make sense, it's because it doesn't.
this whole function needs to be rewritten better.
Signed-off-by: Leah Rowe <leah@libreboot.org>
I also added a "cleanargs" argument, similar to the makeargs
argument, to work around a build error.
This builds the PCSX-Redux PS1 BIOS. They reverse engineered
the Sony PS1 BIOS and wrote a free one under MIT license.
Run this:
./mk -b pcsx-redux
The file will appear: bin/playstation/openbios.bin
Yes. PlayStation support. It's even RYF-friendly:
* Replace BIOS chip with PCSX-Redux's open BIOS
* Install PsNee modchip (libre modchip firmware)
* Install PicoStation (libre optical disc drive emulator)
* Libre SDKs are available e.g. PSn00bSDK
I added this to Libreboot, because I'm working on a fork of
DuckStation. DuckStation recently went proprietary, so I'm making
a fork (soon to be launched) that is based on the libre version,
and I was told of PCSX-Redux's Open BIOS, so I decided to add it
to the *boot projects.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Signed-off-by: Leah Rowe <info@minifree.org>
single-tree projects cannot be handled in bulk, e.g.
./mk -f project1 project2 project3
that is still the case, from the shell, but internally
it is now possible:
mk -f project1 project2 project3
mk() is a function that simply handles the given flag,
and all projects specified.
it does not handle cases without argument, for example
you cannot do:
mk -f
arguments must be provided. it can be used internally,
to simplify cases where multiple single-tree projects
must be handled, but *also* allows multi-tree projects
to be specified, without being able to actually handle
trees within that multi-tree project; so for example,
you can only specify coreboot, and then it would run
on every coreboot tree.
Signed-off-by: Leah Rowe <leah@libreboot.org>
same as the last change. make the main function a wrapper
that dry-runs the real function.
if the "dry" variable is blank, it executes.
Signed-off-by: Leah Rowe <leah@libreboot.org>
when badhash=y, the utils should be deleted, but
the check is deleting if badhash isn't n. if the
hash check isn't being performed, then this will
always be the case and the utils are always deleted.
make it positively delete the file only if badhash=y,
not when it isn't n. while this may not sound very
different, it will prevent the utils being deleted and
re-build endlessly in other cases, like when building
release archives and running the inject --nuke mode
on every image that gets built.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we want multiple seagrub images made, with different
keymaps, but we only want one non-seagrub image.
however, we also want grub in the non-seagrub image.
it just means that seabios is primarily what the user
wants, and they might occasionally use grub, whereas
the seagrub images are for people who primarily want
grub but may occasionally access the seabios menu.
right now, the seabios images really only contain seabios,
but there's no harm in adding grub to them.
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't rely on build/coreboot.rom staying in place,
because sometimes it can get purged under certain
conditions, due to idiosyncrasies in the coreboot
build system, even when we don't explicitly clean it
Signed-off-by: Leah Rowe <leah@libreboot.org>
this time, only handle multiple keymaps on seagrub
images. for images where seabios is first but does
not immediately load grub, whether grub is still
available in flash, just do one image (US Qwerty)
this still results in fewer images per target than
Libreboot 20240612, but should prevent most users
from being annoyed. i got a few people asking
repeatedly, and i hadn't documented yet how to add
keymap.gkb or how to remove bootorder, to get a
different keymap or disable seagrub respectively.
i anticipate that i'll get such questions a lot, even
if i do document it, so i'm reversing that decision.
it doesn't result in much extra code. the new design
in lbmk makes this sort of thing much simpler.
Signed-off-by: Leah Rowe <leah@libreboot.org>
XBMK_CACHE is now used, instead of hardcoding cache/
this is exported initialised to cache/, if unset.
this means you can set your own directory, and it means
./update release will use the same directory.
this means bandwidth wastage is further avoided.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the || : condition should be used, whereas i just
wrote : by mistake. this was done in a previous change.
fix it now.
Signed-off-by: Leah Rowe <leah@libreboot.org>
a previous change made it more redundant, falling back
on old behaviour (direct downloading, not cached), but
the way it's done means that the function never returns
an error condition in practise.
this patch fixes it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
lbmk must still define payloads, but specific configs
may use coreboot's build system instead.
you might use this to add your own config with, say,
tianocore payload, using coreboot.git to build it,
rather than using lbmk's choice of payloads.
Signed-off-by: Leah Rowe <leah@libreboot.org>
lib.sh download() is used by subfile handling in git.sh,
e.g. crossgcc tarballs.
they are not currently cached, but are downloaded directly
in place.
cache them, under cache/file/, saved with the name equal
to the checksum, so: cache/file/CHECKSUM
if the given cached file exists, use it as-is for simple
copy, instead of curl. this avoids re-downloading a lot of
crossgcc tarballs, where different coreboot trees may use
some archives that are the same throughout.
Signed-off-by: Leah Rowe <leah@libreboot.org>
if doing a retry, the directory may still exist, which
would make git clone yield an error response; the existing
directory will have been the one that failed to reset, so
let's delete it.
the one deleted is not the cache (repo/PROJECT/), thus
otherwise maintaining current behaviour.
Signed-off-by: Leah Rowe <leah@libreboot.org>
normally, a project is cached at repo/PROJECT/, and
cloned from there to the final destination.
errors lead to a calling of $err, but this will result
in a return if done from inside a subshell, of non-zero
value, so use this to re-try with a 6th argument when
calling tmpclone().
in most cases, this fallback will never kick in, but
it will kick in resetting or patching the cached clone
fails; specifically, we are interested in the reset part.
a given project name may change repositories in lbmk at
a given time. if this happens, and the old one is cached,
the overall result of this patch is that lbmk will fall
back to the old behaviour, where git urls are tried
directly, without caching.
Signed-off-by: Leah Rowe <leah@libreboot.org>
actual source code is not scanned, but config directories are
scanned. simply get the checksum of each file under config/
pertaining to a given project/tree, and also for the given
target. coreboot utilities are also handled.
if it changes, in any way, delete and re-build automatically.
such deletions should probably still be done manually, as part
of understanding the build system, but this change should make
the build system much easier to use during development.
Signed-off-by: Leah Rowe <leah@libreboot.org>
re-use repo/project/
this means that single- and multi-tree projects now
have a unified cached git repo location, as per the
new rules, thus saving on disk space usage.
Signed-off-by: Leah Rowe <leah@libreboot.org>
do it based on the URL, e.g. https://review.coreboot.org/coreboot
becomes repo/coreboot
the downside is if you have two projects with repo urls specifying
the same string at the end, but this isn't the case at the moment
and likely won't be the case, but it's a theoretical issue.
this saves on bandwidth when downloading identical submodule repos
between multiple trees within the same multi-tree project
for example, coreboot 3rdparty/vboot is no longer downloaded more
than once, instead cloned locally on subsequent downloads.
if repo/DIR exists, git-pull is attempted, but errors do not result
in a non-zero exit, by design.
Signed-off-by: Leah Rowe <leah@libreboot.org>
instead of using lots of if/else conditions, do that once
and set a variable, dry, to :
if not doing a dry run, the variable is empty. prefix this
variable in places where you don't want a certain action to
be performed, on dry runs.
more specifically, : does *nothing* and always returns with
zero status (success).
this results in cleaner code, and a small sloccount reduction.
Signed-off-by: Leah Rowe <leah@libreboot.org>
otherwise, due to the idiosyncratic nature of the coreboot
build system, the coreboot.rom gets wiped out.
cbutils is still handled by premake. ensure that payloads are
only inserted just after running the coreboot make command.
fixes a build issues introduced on 9020sff, previously unhandled.
Signed-off-by: Leah Rowe <leah@libreboot.org>
-d does the same as -b, except for actually building
anything! in effect, it does the same as -f (fetch)
except that the resulting variable assignments will
not be recursive (as with -f).
if -d is passed, configuration is still loaded, defconfig
files are still cycled through, and more importantly:
helper functions are still processed.
the grub, serprog and coreboot helper functions have
been modified to return early (zero status) if -d is
passed.
example usage:
./update trees -d coreboot x230_12mb
this would download the files, NOT build coreboot, and
NOT build the payloads.
there is one additional benefit to doing it this way:
the utils command has been removed, e.g.
./update trees -b coreboot utils default
the equivalent is now:
./update trees -d coreboot default
the overall effect of this change is that the trees script
no longer contains any project-specific logic, except for
the crossgcc build logic.
it does include some config/data mkhelper files at the top,
for serprog and coreboot, so that those variables defined in
those files can be global, but another solution to mitigate
that will also be implemented in a future commit.
the purpose of this and other revisions (in the final push
to complete lbmk audit 6 / cbmk audit 2) is to generalise as
much logic as possible, removing various ugly hacks.
Signed-off-by: Leah Rowe <leah@libreboot.org>
stub it from the trees script. the way it works now,
there is less code in the build system.
./build roms
this is no longer a thing
./build roms serprog
this is also no longer a thing. instead, do:
./update trees -b coreboot targetnamehere
./update trees -b pico-serprog
./update trees -b stm32-vserprog
the old commands still works, which causes the new
commands to run
coreboot roms now appear in elf/, not bin/, as before,
but those images now contain payloads.
NOTE: to contradict the above: ./build roms is no
longer a thing, in that it's now deprecated, but
backward compatibility is present for now. it will
be removed in a future release.
./build roms list also still works! it will do:
./update trees -b coreboot list
also:
./update trees -b grub list
this is now possible too
if a target "list" is provided, for multi-tree sources,
the targets are shown.
there is another difference: seagrub roms are now seagrub_,
instead of seabios_withgrub.
seabios-only roms are no longer provided, where grub is also
enabled; only seagrub is used. the user can easily remove
the bootorder file, if they want seabios to not try grub first.
Signed-off-by: Leah Rowe <leah@libreboot.org>
some of the variables only initialised in git.sh are
also used in the trees script, which is technically ok
because git.sh is included from the trees script, but
it makes more sense to declare them in the latter.
Signed-off-by: Leah Rowe <leah@libreboot.org>
when downloading multi-tree projects, the rev can be reset
to HEAD instead of the actual rev for a given target. this
occurs when the bare repo (e.g. src/coreboot/coreboot) does
not exist and has to be downloaded first.
bare repository downloading does not rely on target.cfg, in
this context, only pkg.cfg, but it uses the same variable
names (e.g. "rev").
instead of using a separate variable name, thus increasing
code complexity (which is the exact opposite of what i want
to do), do the bare repository download first.
this means that the git.sh script is much cleaner now, for
multi-tree projects, in that it *only* copies the bare repo
then runs git_prep; in that context, the bare repo is cloned
directly by calling the relevant function from script/trees,
which is the same behaviour as when cloning single-tree
project sources.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the same function that loads configurations for single-tree
projects has been merged with the function for multi-tree
configs in git.sh, and that functionality has been removed
from git.sh; now it is all unified in the trees script.
as the saying goes: write one program to do one thing well.
the purpose of git.sh is to download source code, but not
to handle configuration files; the latter is meant to be
handled by the trees script, which then calls into git.sh
before running the build logic for that given project.
additionally: the "seen" files are no longer handled, at all.
the logic there was added ages ago, because at the time, i was
considering whether to separate configuration into a new
repository, so that users could more easily make their own
configuration, so it was a guard against misconfiguration.
however, that decision was canceled and we're always very
careful not to introduce a loop; if a loop does occur, the
worst that can possibly happen is you waste some CPU cycles.
Instead, print (on standard output) what config file is being
used, so the operator can see when an infinite loop occurs.
ALSO:
remove _setcfgarg in load_project_config()
it was used to skip when a target.cfg file didn't exist,
specifically on single-tree projects, but this is now
handled using -f instead, on the while loop inside that
function, so _setcfgarg is now a redundant variable.
Signed-off-by: Leah Rowe <leah@libreboot.org>
testing +x is all well and good, but the variable string
may be empty, even if set. some of the checks in the build
system are relying on the latter, so handle it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the trees script itself will check that the directory
exists, and exit with zero status if it does, without
doing anything else other than the return.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we don't want the user to flash coreboot from elf/, because
those images do not contain payloads. the user must flash from
bin/
ample warning is given, at build time, but the warning is written
in english. therefore, some people may not understand it, because
they may not even speak english.
hide the coreboot elf/ directory, to mitigate this possibility.
in most cases, this will probably prevent the average user from
flashing those images, since they likely won't see it.
the "DO NOT FLASH" warning is still included in that directory
name, while creating it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
error out if it's not set. ditto projectsite.
that way, if the files are accidentally deleted, or not
added in a derivative of the build system, you'll know.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the e() and setvars() functions need to be declared before
the dependencies function.
also: after calling install_packages, it was doing a return
when it should have done an exit.
this is all fixed now. i apologise to anyone who previously
ran into trouble with this!
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's bloat. telling the user to rtfm is something that
we already do on irc; they will still ask how to do
everything, and ignore the message from badcmd(), or
they will automatically know to rtfm.
i'm on a massive purge, removing bloat from lbmk as
part of Libreboot Build System Audit 6.
all bloat must go.
Signed-off-by: Leah Rowe <leah@libreboot.org>
replace it with logic that simply uses "." to load
files directly.
config/git files are now directories, also containing
pkg.cfg files each with the same variables as before,
such as repository link and commit hash
this change results in a noticeable reduction in code
complexity within the build system.
unified reading of config files: new function setcfg()
added to lib.sh
setcfg checks if a config exists. if a 2nd argument is
passed, it is used as a return value for eval, otherwise
a string calling err is passed. setcfg output is passed
through eval, to set strings based on config; eval must
be used, so that the variables are set within the same
scope, otherwise they'd be set within setcfg which could
lead to some whacky results.
there's still a bit more more to do, but this single change
results in a substantial reduction in code complexity.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i tried to be clever with this one, but it just made
the script exit with an error.
revert back to the old check (check whether one of
either repo or repo backup is set)
Signed-off-by: Leah Rowe <leah@libreboot.org>
nowadays, we don't insert GRUB keymaps automatically, for
sake of efficiency; without one, the default is US QWERTY.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Leah Rowe