only call crossgcc for coreboot and u-boot, but use
hostcc for everything else. simplify the checking of
which architecture to compile for. "arch" in target.cfg
files has been modified, to allow further simplification.
without this patch, the logic currently only *barely* avoids
using crossgcc on things like utils, and only works in practise
because, in practise, lbmk only works on x86_64 anyway.
the new logic, as per this patch, is simpler and more robust.
Signed-off-by: Leah Rowe <leah@libreboot.org>
remove unnecessary "continue" command. it's written
at the end of a for loop, where it'll continue anyway
Signed-off-by: Leah Rowe <leah@libreboot.org>
all it did was set -v in the shell, which doesn't yield
very useful results. this is a relic of very old design
in the libreboot build system, that is no longer needed.
Signed-off-by: Leah Rowe <leah@libreboot.org>
lbmk didn't quote certain arguments in commands, or
used ! -z instead of -n, things like that. simple fixes.
Signed-off-by: Leah Rowe <leah@libreboot.org>
most of these are probably redundant, and will never
be called, but lbmk needs to be as safe as possible
under fault conditions. fail early, fail hard.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in cases where lbmk must always return from a function,
there are some cases where it relies on non-zero exit
status, which in practise is always the case, but may
change in the future if the relevant part is modified
e.g. do_something && return 0
the proper form is:
do_something
return 0
also do this for unconditional exits
Signed-off-by: Leah Rowe <leah@libreboot.org>
it didn't work in the past, but it does work nowadays;
specifically, it only worked with libgfxinit in the past,
but not on VGA ROMs.
now it does work on VGA ROMs, tested on e6400 and t1650 so
it was enabled there.
in this setup, a special image is provided where SeaBIOS is
the main payload, but it only loads GRUB; nothing else, every.
this is called SeaGRUB. this setup is useful in cases where
the user only has a GPU that lacks libgfxinit support.
Signed-off-by: Leah Rowe <leah@libreboot.org>
when printing the name of the rom being created, it's
done before the check to rename based on vendorfiles
in target.cfg. this patch fixes that bug.
Signed-off-by: Leah Rowe <leah@libreboot.org>
note: me6_update_parser needs to be written, similar
to me7_update_parser, to generate the partition
tables within intel me6 on lenovo bios updates.
the current logic in lbmk goes like this:
mkdir -p vendorfiles/cache/
and save your factory dump as:
vendorfiles/cache/x201_factory.rom
the build system has been modified, in such a way
as to support extracting me.bin (which is the full
one) and then neutering from this.
this is done automatically, if the file is present,
but you must first insert that file there, which means
you'll need a dump of the original boot flash on your
thinkpad x201
Signed-off-by: Leah Rowe <leah@libreboot.org>
it wasn't being copied right
the roms under elf/ were being copied, but not the ones
under bin/ - i need to audit it further
for now, i run modify_coreboot_roms from build/roms
instead of update/trees
so, the ones under elf/ no longer have bootblocks copied.
it's only done in bin/
Signed-off-by: Leah Rowe <leah@libreboot.org>
when building only for u-boot, the current script
works just fine. however, when building for other
payloads in additional to u-boot, the final u-boot
stage fails because other payloads are already
inserted via cbfs.
when we build u-boot, we do that last because we want
u-boot setups to only be u-boot, nothing else.
this patch enables qemu x86 to build properly with
u-boot.
Signed-off-by: Leah Rowe <leah@libreboot.org>
keymaps weren't being set in keymay.cfg of cbfs, due
to use of x_ in the rom script, and x_ doesn't handle
quotes or spaces in arguments well.
i'm going to remove use of x_ and xx_ (it's in my todo),
for next release.
for now, hot patch the release. i've gone through and
replaced use of x_ with || err, in some places.
not just the keymap.cfg command, but others too. in case
there are more issues we missed.
this commit is being tagged "20231021fix" and i'm using
this tag to re-build the 20231021 release. i'll just
replace the tarballs in rsync and add errata to the news
page announcing the release. all i did was break peoples
umlauts, i didn't brick their machines fortunately!
very minor bug. anyway, x_/xx_ is a great idea, but sh
isn't really designed for that style of programming. i'll
go back to using just || err in the next release.
Signed-off-by: Leah Rowe <leah@libreboot.org>
clean it up after copying the tarballs
i really hate how this logic is written, it's clunky
but it should work; the only issue is that it's quite
slow, and inefficient on use of disk space.
however, i've not yet figured out how to reproducible
add files to a tarball, once the tarball has been created,
and i rely on sorting (of file names) when creating them.
it's really not a problem because normal people won't
use this script, only i or anyone who wants to test out
the libreboot release infrastructure. this script is
largely intended to *work*
but i'm still annoyed by how crappy it is. i'll fix it
after the Libreboot 20231021 release.
Signed-off-by: Leah Rowe <leah@libreboot.org>
everything downloaded, then tarballed, then built,
now crossgcc is downloaded by coreboot.
now extract, copy crossgcc tarballs, re-compress.
TODO: simply add files to the archive, without re-
compressing the whole thing.
this is still more efficient than the old way: build
everything, then clean and compress, making another
build test on the release archive necessary; with this,
there is still only one build test per release.
with this, and the previous revisions dealing with
submodules, the source archives should now be complete.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in release archives, .git is excluded but the version
and versiondate files are included. from these, the
git history is re-created with the exact date (but not
taking into account timezone, at present).
in this way, lbmk will have git history in a release
archive. some build systems, like coreboot, prefer that
there be git history available, so this is a nice
workaround on those build systems.
Signed-off-by: Leah Rowe <leah@libreboot.org>
do it using find -exec
this is more robust, and it will never need to be
maintained over time (famous last words).
this is done because now we download submodules
for all git projects, so it's hard to predict.
Signed-off-by: Leah Rowe <leah@libreboot.org>
when [] is used right at the end of a function, or
certain loops/subshells, some sh implementations will
just return a non-zero exit
Signed-off-by: Leah Rowe <leah@libreboot.org>
config/git has been re-arranged in a prior revision,
ensuring that each file only refers to a main source
tree defined within those files.
the erstwhile "./build clean all" functionality is now
once again possible in lbmk
Signed-off-by: Leah Rowe <leah@libreboot.org>
in some cases, use of x_ or xx_ can be error-prone,
due to the way $@ is handled; commands requiring
quotes, or with funny file names as arguments such
as spaces in the file name, or other special
characters, can make the x/xx functions break.
in those cases, where x/xx must not be used, the
commands use || err instead
in other cases, use of x/xx is superfluous, and has
been removed in some commands.
Signed-off-by: Leah Rowe <leah@libreboot.org>
as opposed to the current 3-level structure.
recent build system simplifications have enabled
this change, thus:
./build fw coreboot -> ./build roms
./build fw grub -> ./build grub
./build fw serprog -> ./build serprog
./update project release -> ./update release
./update project trees -> ./update trees
./update vendor download -> ./vendor download
./update vendor inject -> ./vendor inject
alper criticised that the commands were too long,
so i made them shorter!
Signed-off-by: Leah Rowe <leah@libreboot.org>
i forgot to include option.sh in this script,
during previous re-factoring. the cbfstoos variable
is now defined exclusively in option.sh, but other
scripts can set it to something else.
Signed-off-by: Leah Rowe <leah@libreboot.org>
move it all to other files where items are used, and not
used anywhere else. this reduces the size of vendor.sh.
also remove a few redundant variables, or variables that
are not meaningfully used.
a few items have been moved to include/option.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
they are the functions only used by the download
script, so they don't belong in vendor.sh
an include file should only contain variables and
functions used by multiple main scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
We don't really need a custom coreboot tree for Chromebooks. I had added
one, because at a cursory glance to the available config/coreboot/board
subdirectories I had the impression that I should. But upstreams have
one tree for every board and I think we should move towards that too.
Move the one important BL31 makefile patch into the default coreboot
patches, update the gru boards' configs by running savedefconfig in the
cros tree and then running olddefconfig in the default tree.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Add an "-s" flag for "make savedefconfig", "-l" for "make olddefconfig"
and "-n" for "make nconfig" to the update script. The first two are
mainly useful for U-Boot, to compare our configs to the upstream
defconfigs and stay in sync with any upstream changes. The latter is
because the ncurses one has a nice "Symbol Search" that can point out
the menu entry for a config symbol we know.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
The "u-boot.bin" file generated by U-Boot builds is a raw binary. When
adding payloads to a CBFS, we need to use ELF files with add-payload
or manually pass the entry point and load address of the payload binary
with add-flat-binary.
We primarily use the "u-boot.elf" which gets build with the REMAKE_ELF
option, as it also has the necessary device-tree binary that U-Boot
usually needs to work. When the option is not set (e.g. for QEMU), we
need to use the "u-boot" file which is an ELF.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
i wasn't getting the very first line of tar --version,
so it wasn't doing the check properly.
further sort the files by name within the tar archive.
for reliability, don't bother using versiondate anymore:
set a *fixed* date, and fixed timezone, to ensure
that it works reliably for reproducible tarball creation.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This way, the handling of configs is unified into one
script, which reduces the possibility of bugs later,
and it reduces the repetition of code.
Signed-off-by: Leah Rowe <leah@libreboot.org>
use find and touch, to force all files, directories and
links to the desired timestamp (versiondate file)
Signed-off-by: Leah Rowe <leah@libreboot.org>
e.g. src/coreboot/coreboot must not appear in a release,
because we instead have directories like
src/coreboot/default or src/coreboot/cros
lbmk resets src/coreboot/coreboot to HEAD, but then resets
revisions properly in copies of it
therefore, for reproducibility, we must not include
src/coreboot/coreboot, src/u-boot/u-boot or
src/seabios/seabios into libreboot releases
Signed-off-by: Leah Rowe <leah@libreboot.org>
with --mtime, files added to the archive can be set
to a static date (in this case, the unix epoch)
the one used here is derived from git commit dates,
and it is static; if not being handled in lbmk.git,
the versiondate file never changes
this is the first patch in a series of patches designed
to bring about reproducible builds in libreboot
a solution will need to be found, for non-GNU tar
implementations, because they did not have an
equivalent option according to their manpages.
for example, BSD tar implementations.
perhaps i could systematically go around changing
file dates, on each file, as a fallback behaviour?
Signed-off-by: Leah Rowe <leah@libreboot.org>
this way, the src tarball is guaranteed to be clean.
the downside is that lbmk itself does not currently
handle crossgcc downloads, and there may be some
stragglers such as third party modules automatically
downloaded by certain codebases that libreboot uses.
this will have to be audited later (and it will be).
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's sometimes done unconditionally. this change
ensures that it is not repeated needlessly.
i observed otherwise that cbfstool would be
re-built from time to time, even if it was built.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Riku's mSATA patch for HP8300USDT was merged upstream, so the
patch has been dropped from lbmk because it is contained within
this new coreboot revision.
Signed-off-by: Leah Rowe <leah@libreboot.org>
coreboot closely matches upstream, whose current release
is version 1.2 from 2018, and coreboot has not changed it
in any meaningful way.
the upstream did add patches since, but they are documentation
patches only.
this means: we do not need to use the upstream version
Signed-off-by: Leah Rowe <leah@libreboot.org>
also rename elf/coreboot to something scary
some users were flashing roms built under elf/, which
lack payloads. lbmk builds no-payload roms (and payloads)
under elf/ then inserts them, creating full (flashable)
images under bin/
Signed-off-by: Leah Rowe <leah@libreboot.org>
some users reported build errors. technically, there's
nothing wrong with lbmk but it relies on hostcc, and
hostcc is hit or miss when it comes to cross compiling
32-bit, depending on the build system of whatever project.
lbmk needs to handle cross compilation. for now, i'm just
disabling memtest86plus on non-64-bit hosts.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The logic has been re-written, where source archives are
concerned. This clones the current repository, and starts
a new build from scratch. A custom release directory is
possible, by passing -d
This eliminates a step during build-testing, saving hours
of time, because it builds the release archive *inside* the
release archive, with git files removed, thus replicating
the same setup that the user would have.
This also makes everything a bit more consistent, because
it's guaranteed that a release archive will always have
the same files; previously, the release build script would
only copy what was already built, without building anything.
Now, this script builds everything itself.
The script also builds serprog images, not just coreboot.
Usage:
./update project release
If -d is not passed, release/ is used inside lbmk.
Otherwise, you could do:
./update project release -d /path/to/directory
If the directory exists, this script will exit (error).
Other minor fixes: build/fw/coreboot: make version in
coreboot-version (file) not contain hyphens, to work
around a quirk in coreboot's build system when not building
on regular libreboot releases. this quirk only appears
when lbmk is not being compiled under git.
The other main benefit of this change is that the new
script will probably require a lot less maintenance.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the script used to be called once per target, now it
handles every target. the grub background image wasn't
being set, so if it changed at build time, it would
stay changed.
keep the default in place for each run, while still
allowing target.cfg files to change it per target.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Just one script.
Just one!
Well, two, but the 2nd one already existed:
logic in update/project/trees and
update/project/repo was merged into
include/git.sh and update/project/build
was renamed to update/project/trees; an -f
option was added, which calls the functions
under git.sh
so git clones are now handled by the main build
script (for handling makefiles and defconfigs)
but the logic there is a stub, where git.sh
does all the actual heavy lifting
this cuts the file count down by two, and reduces
sloccount a reasonable amount because much of
the logic already exists in the build script, when
it comes to handling targets. git.sh was adjusted
to integrate with this, rather than act standalone
Signed-off-by: Leah Rowe <leah@libreboot.org>
otherwise, if src/grub/ was already compiled, this
would not print anything on the screen. however, the
files will have been created under elf/grub
this message just makes lbmk a bit more user friendly
Signed-off-by: Leah Rowe <leah@libreboot.org>
The benefit now is that it can be cleaned. E.g.
./update project build -b coreboot utils
./update project build -b coreboot utils default
./update project build -c coreboot utils
./update project build -c coreboot utils default
the update/project/build script checks when arguments
are provided after the project name. if the first one
is "utils", then it acts in the same way as the old
build/coreboot/util script
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's buggy. "./build fw coreboot" was made to work,
but it caused lots of unknown issues when mixing other
args
the old way wasn't broken. now, once again, you must
pass the "all" argument. e.g.:
./build fw coreboot all
Also, the confirmation messages at the end are a bit
clearer, when listing which ROM images were compiled.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in the future, we may start downloading files that aren't
blobs, such as mxm port configs (on mainboards that use
MXM graphics)
this directory will contain all of those files
generally change the language used, across lbmk, to make
use of "vendorfile" instead of "blob"
Signed-off-by: Leah Rowe <leah@libreboot.org>
during the switch to src/ for all downloads, i
overlooked that the path check was hardcoded.
now the check for this binary is corrected.
Signed-off-by: Leah Rowe <leah@libreboot.org>
build/release/src was partly re-written to accomodate this
memtest86plus was patched to have a central Makefile, and
lbmk modified to use that, rather than mess with build32
and build64. the central Makefile just builds both targets
or cleans both targets
Signed-off-by: Leah Rowe <leah@libreboot.org>
return with error status if no images were compiled
if a rom image fails to compile, then it will also
exit with error status, but sometimes you can pass
argument "cros" or "default", and it would not give
you rom images due to no target.cfg files, but these
are also ignored because of that.
this restores the same behaviour that existed before,
for this final error check.
Signed-off-by: Leah Rowe <leah@libreboot.org>
for the first time ever, this is a single script.
with recent simplifications in how variables are
handled, and techniques i've developed during
auditing, it's now feasible design-wise for this
to be a single script, without a helper script.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Previously, this script only checked for "Makefile",
but "makefile" is another valid name; additionally, if
GNU Make is used, "GNUmakefile" is an accepted default.
Signed-off-by: Leah Rowe <leah@libreboot.org>
There is no reason to err if no Makefile exists.
Just exit with zero status. This makes the following
command work:
./handle make file -c util/*
Within util/, there is me7 update parser which does
not have a makefile (it's a python script).
Signed-off-by: Leah Rowe <leah@libreboot.org>
The previous patch to the file was correct, except for
off by one at the end, resulting in no argument being
passed for project names.
Now the extra commands are run *before* handle_dependencies,
instead of running at the end of main. This prevents error.
Signed-off-by: Leah Rowe <leah@libreboot.org>
At the end of the function, this script will now
run itself again if there are more arguments. This
enables the following:
./handle make file -c project1 project2 project3
Whereas previously, it could only do this:
./handle make file -c project1
Substitude -b and it's the same.
Signed-off-by: Leah Rowe <leah@libreboot.org>
some x_ calls are made that aren't needed. this is now
corrected. additionally, some x_ calls were being made
that are quite error-prone, like ones that use $PWD.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the one at the end of main is unnecessary, because
it's handled inside the for loop.
this file isn't used anywhere else, so it's OK.
Signed-off-by: Leah Rowe <leah@libreboot.org>
as it turns out, i delete "seen" inside the for loop,
which is a more thorough way to do it.
thus, the first rm command is unnecessary.
Signed-off-by: Leah Rowe <leah@libreboot.org>
slight sloccount reduction. light renaming of
functions between the two scripts, placing more
logic in main() under include/boot.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
errors are not defined for mktemp, and the /tmp file
system should be assumed reliable.
if /tmp is *unreliable*, then this is not something that
lbmk either can or should fix; the user clearly has
bigger problems.
manpages for mktemp do not define errors. it is assumed
to be completely reliable.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Instead of having detailed error messages, run most
commands through a function that calls err() under
fault conditions.
Where detail is still required, err() is still called
manually. Where it isn't, the error message is simply
whatever command was executed to cause the error.
This results in a massive sloccount reduction for lbmk;
specifically, 178 sloc reduction, or a 8.1% reduction.
The total sloccount is now 2022, for shell scripts.
Signed-off-by: Leah Rowe <leah@libreboot.org>
also: further reduce the number of arguments passed,
to certain functions as and when feasible, in cases
where those are global variables that never change.
the cbfstool argument in mkUbootRom wasn't even used.
that function was only using the global variable, which
again is only set once.
i also shortened a few messages, removed a few errant
line breaks and reduced sloccount by exactly 1 in main()
by re-arranging how the shift command is used.
it's mainly about shortening variable names, to then
reduce the number of line breaks, but it's a surgical
code size reduction in build/boot/roms.
Signed-off-by: Leah Rowe <leah@libreboot.org>
These are only ever initialised globally, and set once.
Other instances where they are set are only in cases
where they are passed as argument, at the start of
a function, so they are being *needlessly* re-set.
Set them only once and use them globally.
Signed-off-by: Leah Rowe <leah@libreboot.org>
-k, -p and -d let you set keymap, payload and displaymode
respectively, but the handling for this is buggy when
passing multiple arguments.
Support only one argument, for simplicity. This is how
people use them anyway, and it makes lbmk less buggy.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The *same* main() function is now used on both scripts.
However, merging both scripts together would be less efficient
on sloccount, and would be error-prone. The purpose of having
roms_helper is that the variables get re-initialised the same
way each time, for each board, automatically.
Signed-off-by: Leah Rowe <leah@libreboot.org>
If one of them doesn't exist, error out.
Previously, a build would start but then it would
error out later on. This implements the mentality:
fail early, fail hard
Signed-off-by: Leah Rowe <leah@libreboot.org>
e.g. -k ukqwerty
previously, this would not work:
./build boot roms -k ukqwerty all
only this would work:
./build boot roms all
this patch fixes the bug.
Signed-off-by: Leah Rowe <leah@libreboot.org>
update/blobs/download and update/project/repo both use
the same logic, for setting variables with awk and a
specially formatted configuration file.
unify this logic under include/option.sh, and use that.
Signed-off-by: Leah Rowe <leah@libreboot.org>
mkdirs() should be in include/blobutil.sh, as should
extract_archive(), because that is primarily where
they are used.
script/update/blobs/download calls these functions
aswell, but it sources include/blobutil.sh so it's OK.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Don't use only wget. Some systems may only have curl.
The user can always install wget anyway, but why not
support both? I've added the right user agent string.
Signed-off-by: Leah Rowe <leah@libreboot.org>
They are only ever used by script/update/blobs/*, so
put them all in blobutil.sh. This cuts down on the
number of scripts in lbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
mrc.bin is now handled by include/mrc.sh, adapted
from now-deleted script/update/blobs/mrc
much of the logic has been re-written or adapted for
inside script/update/blobs/download
mrc links/hashes now defined in config/blobs/sources
the new code is simpler (and smaller). in addition,
lbmk can now easily handle mrc.bin files for other
platforms such as broadwell. watch this space.
the full .zip download is now cached, like with other
vendor downloads. this means it won't be re-downloaded
if it was already downloaded before.
Signed-off-by: Leah Rowe <leah@libreboot.org>
individual functions for downloading each archive have
been removed. instead, eval is used in fetch_update(),
which is now renamed to fetch().
Signed-off-by: Leah Rowe <leah@libreboot.org>
the called functions directly call err() under fault condition,
so this additional handling is redundant.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Leah Rowe