we no longer need to remove cache/ per project, because
it's removed in bulk at the end, in the main build script,
when generating release archives.
Signed-off-by: Leah Rowe <leah@libreboot.org>
lib.sh download() is used by subfile handling in git.sh,
e.g. crossgcc tarballs, and also the vendor scripts.
vendor files are cached, but not subfiles for repos.
cache both, under cache/file/, saved with the name equal
to the checksum, so: cache/file/CHECKSUM
also move vendorfiles/app/ to cache/app/ in this change.
Signed-off-by: Leah Rowe <leah@libreboot.org>
if doing a retry, the directory may still exist, which
would make git clone yield an error response; the existing
directory will have been the one that failed to reset, so
let's delete it.
the one deleted is not the cache (repo/PROJECT/), thus
otherwise maintaining current behaviour.
Signed-off-by: Leah Rowe <leah@libreboot.org>
normally, a project is cached at repo/PROJECT/, and
cloned from there to the final destination.
errors lead to a calling of $err, but this will result
in a return if done from inside a subshell, of non-zero
value, so use this to re-try with a 6th argument when
calling tmpclone().
in most cases, this fallback will never kick in, but
it will kick in resetting or patching the cached clone
fails; specifically, we are interested in the reset part.
a given project name may change repositories in lbmk at
a given time. if this happens, and the old one is cached,
the overall result of this patch is that lbmk will fall
back to the old behaviour, where git urls are tried
directly, without caching.
Signed-off-by: Leah Rowe <leah@libreboot.org>
actual source code is not scanned, but config directories are
scanned. simply get the checksum of each file under config/
pertaining to a given project/tree, and also for the given
target. coreboot utilities are also handled.
if it changes, in any way, delete and re-build automatically.
such deletions should probably still be done manually, as part
of understanding the build system, but this change should make
the build system much easier to use during development.
Signed-off-by: Leah Rowe <leah@libreboot.org>
single-tree repos were not previously cached, but now
they are and they have to be handled.
this, as also alluded to in the previous commit, is done
when preparing release archives (XBMK_RELEASE=y)
Signed-off-by: Leah Rowe <leah@libreboot.org>
repo/p/ does not have its revision reset, so it
changes unpredictably, and it's not used in builds.
this used to be src/p/p/ - the context here is multi-tree
projects, in source archives.
Signed-off-by: Leah Rowe <leah@libreboot.org>
re-use repo/project/
this means that single- and multi-tree projects now
have a unified cached git repo location, as per the
new rules, thus saving on disk space usage.
Signed-off-by: Leah Rowe <leah@libreboot.org>
do it based on the URL, e.g. https://review.coreboot.org/coreboot
becomes repo/coreboot
the downside is if you have two projects with repo urls specifying
the same string at the end, but this isn't the case at the moment
and likely won't be the case, but it's a theoretical issue.
this saves on bandwidth when downloading identical submodule repos
between multiple trees within the same multi-tree project
for example, coreboot 3rdparty/vboot is no longer downloaded more
than once, instead cloned locally on subsequent downloads.
if repo/DIR exists, git-pull is attempted, but errors do not result
in a non-zero exit, by design.
Signed-off-by: Leah Rowe <leah@libreboot.org>
upstream has merged all of the changes that it contained,
so we don't need this anymore. we'll have the newer upstream
changes on the next general revision updates for coreboot,
within config/coreboot/
Signed-off-by: Leah Rowe <leah@libreboot.org>
configure_project is a bit big. move the dependencies
build logic to a new function.
it may be desirable in future to make the way that
function works the way all build commands are done.
for example:
./update trees -b coreboot x230_12mb
would become:
./update trees -b coreboot/x230_12mb
this would enable to mix and match multi/single tree
projects. for now, leave things as they are.
Signed-off-by: Leah Rowe <leah@libreboot.org>
U-Boot has migrated to using upstream device-tree files for gru boards,
but the clock driver doesn't yet support setting rates for a certain
clock that upstream uses for the eDP display. It happens to work without
it, so for now remove the clock setting until the driver is fixed.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Commit 46e01c0e1d ("u-boot: Avoid building U-Boot-only binman images")
added a patch that prevents an error while building U-Boot, due to some
U-Boot images needing a copy of BL31 that we are not passing in.
Removing build instructions for these images isn't really necessary,
when we can instead tell the build tool that it shouldn't exit with an
error. It checks a BINMAN_ALLOW_MISSING environment variable for this,
but just unconditionally replace the check with the argument.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Set default U-Boot revision to v2024.07 and rebase patches on top of
that. One patch that fixes drawing box characters (UTF-8 to CP437) had
an alternative merged, another hack we have to fix regulator issues is
no longer neccessary as the issue is fixed, and my QEMU patches were
merged upstream, so drop these patches. One patch we have to disable
binman images can be replaced by a simpler alternative so drop it too.
Upstream kconfig status is still unstable, so updating configs with
`make oldconfig` would miss important upstream changes, since they rely
on carrying defaults via upstream defconfigs. Update the configs as
such, like before:
- Turn old configs into defconfigs (./update trees -s u-boot)
- Save the diff from old upstream defconfig (diffconfig $theirs $ours)
- Update U-Boot revision, rebase patches, and clean old trees
- Prepare new U-Boot tree (./update trees -f u-boot)
- Review the diffconfigs to see if any options were renamed upstream
- Copy over the new upstream defconfigs and apply earlier diff
- Turn new defconfigs into configs (./update trees -l u-boot)
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
we used to set cmd only to these values:
build_project
build_targets
however, now we set them to:
build_project
build_targets $@
the latter cannot be measured reliably, but
we were checking whether cmd equalled:
build_targets
now we instead check that it does not equal:
build_project
Signed-off-by: Leah Rowe <leah@libreboot.org>
main() used to be the only function executed from
outside of main(), in this script, but now we source
a config file and then run the build afterward.
when a flag is provided without OPTARG, this means
that we are continuing such action erroneously. to
mitigate this, return 1 in that instance, and handle
it in the line that calls main(), making it exit with
zero status (success).
Signed-off-by: Leah Rowe <leah@libreboot.org>
instead of using lots of if/else conditions, do that once
and set a variable, dry, to :
if not doing a dry run, the variable is empty. prefix this
variable in places where you don't want a certain action to
be performed, on dry runs.
more specifically, : does *nothing* and always returns with
zero status (success).
this results in cleaner code, and a small sloccount reduction.
Signed-off-by: Leah Rowe <leah@libreboot.org>
move the coreboot-specific includes into mkhelper.cfg
for that project.
on some projects, we need variables from mkhelper.cfg
to be global, so I was including serprog and coreboot
mkhelper.cfg files in this script.
instead, set a new variable "mkhelpercfg" pointing to
the config file. if it doesn't exist, create and then
point to a temporary (empty) mkhelper.cfg file.
the rom.sh include has been moved to coreboot mkhelper.cfg
The only remaining project-specific logic, in this trees
script, is now the coreboot crossgcc handling, but this
needs to be there as it's also used to build U-Boot.
The way this now works, certain includes are done twice.
For example, include/rom.sh will be included once globally,
outside of main(), and then again in configure_project().
This means that certain functions will be defined twice.
I'm uncertain if shell has anything equivalent to an ifdef
guard as in C, but we actually want this here anyway, and
it shouldn't cause any problems. It's a bit of a hack, but
otherwise results in much cleaner code.
Signed-off-by: Leah Rowe <leah@libreboot.org>
otherwise, due to the idiosyncratic nature of the coreboot
build system, the coreboot.rom gets wiped out.
cbutils is still handled by premake. ensure that payloads are
only inserted just after running the coreboot make command.
fixes a build issues introduced on 9020sff, previously unhandled.
Signed-off-by: Leah Rowe <leah@libreboot.org>
pro-tip: don't do this at 3AM
do massive changes like this, no later than 1AM.
the intent anyway is for -d to cause no build dependencies
to be handled, but the current logic says to only handle
them if -d is set! fix it by removing the ! part
Signed-off-by: Leah Rowe <leah@libreboot.org>
-d does the same as -b, except for actually building
anything! in effect, it does the same as -f (fetch)
except that the resulting variable assignments will
not be recursive (as with -f).
if -d is passed, configuration is still loaded, defconfig
files are still cycled through, and more importantly:
helper functions are still processed.
the grub, serprog and coreboot helper functions have
been modified to return early (zero status) if -d is
passed.
this behaviour will be used to integrate vendor.sh
logic in with the trees script, for cases where the
user wants to only handle vendor files. e.g.:
./update trees -b coreboot x230_12mb
this would download the files as usual, build coreboot,
with those files, and then build the payloads. but:
./update trees -d coreboot x230_12mb
this would download the files, NOT build coreboot, and
NOT build the payloads.
this change increases the sloccount a bit, but i'm relying
on the fact that the vendor.sh script already re-implements
config handling wastefully; the plan is to only use trees.
for now, simply stub the same ./vendor download command.
there is one additional benefit to doing it this way:
this method is *per-kconfig* rather than per-target.
this way, one kconfig might specify a given vendor file
that is not specified in the other. although the stub
still simply handles this per target, it's done in premake,
which means that the given .config file has been copied.
this means that when i properly re-integrate the logic
into script/trees, i'll be able to go for it per-kconfig.
the utils command has been removed, e.g.
./update trees -b coreboot utils default
the equivalent is now:
./update trees -d coreboot default
this would technically download vendor files, but here
we are specifying a target for which no kconfigs exist;
a check is also in place, to avoid running the vendor file
download logic if tree==target
the overall effect of this change is that the trees script
no longer contains any project-specific logic, except for
the crossgcc build logic.
it does include some config/data mkhelper files at the top,
for serprog and coreboot, so that those variables defined in
those files can be global, but another solution to mitigate
that will also be implemented in a future commit.
the purpose of this and other revisions (in the final push
to complete lbmk audit 6 / cbmk audit 2) is to generalise as
much logic as possible, removing various ugly hacks.
Signed-off-by: Leah Rowe <leah@libreboot.org>
stub it from the trees script. the way it works now,
there is less code in the build system.
./build roms
this is no longer a thing
./build roms serprog
this is also no longer a thing. instead, do:
./update trees -b coreboot targetnamehere
./update trees -b pico-serprog
./update trees -b stm32-vserprog
the old commands still works, which causes the new
commands to run
coreboot roms now appear in elf/, not bin/, as before,
but those images now contain payloads.
NOTE: to contradict the above: ./build roms is no
longer a thing, in that it's now deprecated, but
backward compatibility is present for now. it will
be removed in a future release.
./build roms list also still works! it will do:
./update trees -b coreboot list
also:
./update trees -b grub list
this is now possible too
if a target "list" is provided, for multi-tree sources,
the targets are shown.
there is another difference: seagrub roms are now seagrub_,
instead of seabios_withgrub.
seabios-only roms are no longer provided, where grub is also
enabled; only seagrub is used. the user can easily remove
the bootorder file, if they want seabios to not try grub first.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we no longer need to call trees -b for payloads, because
build_depend is set in coreboot target.cfg files
Signed-off-by: Leah Rowe <leah@libreboot.org>
rename it to configure_project, because the function now
also handles building (a little bit), not just mere loading
of configuration files.
Signed-off-by: Leah Rowe <leah@libreboot.org>
We already have this to an extent, e.g. the xtree variable.
The xtree variable could probably be removed, in favour of
this, and used for the same purpose.
It works like this, for example:
build_depend="coreboot/default grub/xhci seabios u-boot/gru_bob flashprog"
the "/" denotes a tree, if it's a multi-tree project. However, specifying
the entire multi-tree project without slash is possible, for example:
build_depend="coreboot"
this would specify that all coreboot trees must be built.
This functionality will be used in follow-up commits, centralising
script/trees into mk on the main directory, repacing "build".
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't put it in the install modules.
this works around a hanging issue on haswell thinkpads.
when any usb device is inserted, GRUB will sometimes
hang if started from the SeaBIOS payload, *while* the
USB device is plugged in.
plugging in the USB device after GRUB starts worked.
it will have to be investigated more at a later date,
but this simply configuration change works.
the xhci module is already loaded explicitly, in grub.cfg
Signed-off-by: Leah Rowe <leah@libreboot.org>
stick the makeargs in mkhelper
i previously did cbmakeargs because the old revisions
had to define makeargs per-target otherwise. mkhelper
was done specifically to solve that problem.
Signed-off-by: Leah Rowe <leah@libreboot.org>
config/data/PROJECT/mkhelper.cfg can be provided, for
configuration, and it is loaded *before* target.cfg
there are certain instances where we repeat a lot of
config per tree, in multi-tree projects.
for example, we have the exact same config per grub
tree, besides tree name and revision number, for things
like autoconf arguments.
this last problem will be addressed, in a follow-up
patch, and then expanded upon for other projects.
Signed-off-by: Leah Rowe <leah@libreboot.org>
e.g. ./update trees -f
if passed, this command would download every tree
similarly, the -c option can be used in this way. this
solves a longstanding issue: on the current, much more
efficient design, it was not possible to systematically
clean every project.
Signed-off-by: Leah Rowe <leah@libreboot.org>
some of the variables only initialised in git.sh are
also used in the trees script, which is technically ok
because git.sh is included from the trees script, but
it makes more sense to declare them in the latter.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Leah Rowe