Commit Graph

447 Commits (8a435355135daf19040fcc6b02e6570d918958fd)

Author SHA1 Message Date
Leah Rowe b5da9feba3 vendor.sh: Print useful message on ./mk inject
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-26 22:25:07 +00:00
Leah Rowe 12c6259cb2 vendor.sh: Handle FSP insertion post-release
The Libreboot 20241206 release provided FSP pre-assembled
and inserted into the ROM images; the only file inserted
by vendor.sh was the Intel ME.

Direct distribution of an unmodified FSP image is permitted
by Intel, provided that the license notice is given among
other requirements. Due to how coreboot works, it must split
up the FSP into subcomponents, and adjust certain pointers
within the -M component (for raminit).

Such build-time modifications are perfectly fine in a coreboot
context, where it is expected that you are building from source.
The end result is simply what you use.

In a distribution such as Libreboot, where we provide pre-built
images, this becomes problematic. It's a technicality of the
license, and it seems that Intel themselves probably intended
for Libreboot to use the FSP this way anyway, since it is they
who seem to be the author of SplitFspBin.py, which is the
utility that coreboot uses for splitting up the FSP image.

Due to the technicality of the licensing, the FSP shall now
be scrubbed from releases, and re-inserted.

Coreboot was inserting the -S component with LZ4 compression,
which is bad news for ./mk inject beacuse the act of compression
is currently not reproducible. Therefore, coreboot has been
modified not to compress this section, and the inject command
doesn't compress it either. This means that the S file is using
about 180KB in flash, instead of about 140KB. This is totally OK.

The _fsp targets are retained, but set to release=n, because these
targets *still* don't scrub fsp.bin; if released, they would
include fsp files, so they've been set to release=n. These can
be used on older Libreboot release archives, for compatibility.

The new ROM images released for the affected machines are:

t480_vfsp_16mb
t480s_vfsp_16mb
dell3050micro_vfsp_16mb

Note the use of _vfsp instead of _fsp. These images are released,
unlike _fsp, and they lack fspm/fsps in the image. FSP S/M must
be inserted using ./mk inject.

This has been tested and confirmed to boot just fine.
The 20241206 images will be re-compiled and re-uploaded with this
and other recent changes, to make Libreboot 20241206 rev8.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-26 22:05:16 +00:00
Leah Rowe 07037561bd lbmk: remove use of deprecated ./vendor command
use ./mk instead, because in a future change to lbmk,
only ./mk will be used and the other commands will
be removed.

with this change, the ./vendor, ./build and ./update
commands are no longer used. these commands still work,
for backwards compatibility, but they are deprecated.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-24 16:58:23 +00:00
Leah Rowe 5d1f182306 vendor.sh: Safer exit when vendorfiles not needed
When vendor files were not needed on a given board,
the script would directly exit. This is bad, because
the inject functions are called directly from the main
script, which means the parent instance of lbmk.

This means that the lock file and temporary files were
not being removed on exit. On a subsequent run, this
would cause the error stating that a lock file is present,
which would cause further error, making the user believe
something is broken in lbmk.

Modify the behaviour accordingly; exits are now returns,
and these are handled in the calling functions, in such
a way that a proper exit occurs, whereby temporary files
and the lock file are deleted.

For context, please read the main "build" script where
it calls vendor_inject and vendor_download. At the end
of that script, it calls tmp_cleanup, which removes the
TMPDIR that was created, and the lock file. In lbmk,
the TMPDIR is not /tmp, but rather a subdirectory
under /tmp, so that further calls to mktemp create
everything under one single temporary directory, which
lbmk automatically removes on exit.

Therefore, this patch also avoids leaving temporary files
laying around on the disk.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-24 14:09:29 +00:00
Leah Rowe ee8f53b96f lib.sh: Safer exit from ./mk dependencies
The exit was dependent upon install_packages returning
zero status, which it always would in practise, due to
its design, but this exit must always be observed, so
the code has been modified to honour this design.

A direct exit violates lbmk's design in most instances,
where a temporary directory and lock file has already
been created; at this stage, no such act was performed,
so a direct exit is perfectly acceptable.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-24 12:39:05 +00:00
Leah Rowe a8b35c88cf remove geteltorito and mtools from lbmk
we needed these for extracting intel vga roms from
lenovoo updates, for t480, very briefly. about an hour
after i pushed that patch, mate kukri fixed libgfxinit
and then i removed the vgarom integration because it
wasn't needed anymore.

however, i forgot to remove geteltorito/mtools from
dependencies. some distros like fedora were problematic
about it.

the best thing about bugs is when you don't have to fix them.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-22 23:13:43 +00:00
Leah Rowe 1dd32ea548 rom.sh: support grub-first setups
in this setup, seabios is never the default payload, grub is,
but only if grub is enabled.

set this in target.cfg:

payload_grubsea="y"

if payload_grub isn't enabled, this is auto-set to n

ditto if initmode=normal

NOTE: if flashing libgfx setups, you should make sure
that you're not booting with a graphics card, only intel
graphics. this setting will intentionally not be documented,
because it's not recommended, but is being implemented for
testing purposes (and i implemented it for some guy who i
think is cool). i'll probably also use this myself, since
i already do grub-only setups on all my own machines.

seagrub is the default on x86 because of past instabilities
with grub. to mitigate in case of future issues, since seabios
is always stable, we reduce the chance of bricks.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-18 07:15:18 +00:00
Leah Rowe f7801ef477 vendor.sh: delete old tb.bin first, just in case
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-18 03:49:58 +00:00
Leah Rowe 02cbf8a729 vendor.sh: make TBFW pad size configurable
we encountered 1MB flash so far, but we may encounter other
sizes on other machines when added to libreboot later on

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-18 03:42:45 +00:00
Leah Rowe 9884e5ed1b T480/T480S: Support fetching ThunderBolt firmware
Though not used in coreboot builds, and not injected into the
builds in any way, these files are now created seperately when
handling T480/T480s vendor files:

vendorfiles/t480/tb.bin
vendorfiles/t480s/tb.bin

These are created by extracting Lenovo's ThunderBolt firmware
from update files. The updated firmware fixes a bug; older firmware
enabled debug commands that wrote logs to the TB controller's
own flash IC, and it'd get full up with logs, bricking the controller.
If you've already been screwed by this, you must flash externally,
using a padded firmware from Lenovo's updates.

Lenovo's own updater requires creating a boot CD or booting
Windows. This patch in lbmk auto-downloads just the firmware,
and you can flash it externally.

You could simply do this as a matter of course, when installing
Libreboot. You are recommended to update the Lenovo UEFI/EC firmwares
first, before installing Libreboot; please look at the Libreboot
documentation to know exactly which versions.

Then dump the ThunderBolt firmware first, to be sure, and then you
can flash these files. Flashing these updates will prevent the bug
described here:

https://pcsupport.lenovo.com/us/en/products/laptops-and-netbooks/thinkpad-t-series-laptops/thinkpad-t480-type-20l5-20l6/20l5/solutions/ht508988

You can download Lenovo's installers for various ThinkPad models
there, including T480s/T480s. It is these downloads that this lbmk
patch uses, to extract those files directly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-18 02:28:29 +00:00
Leah Rowe 44969c73bd rom.sh: insert grub background in cbfs not memdisk
for some reason, when the background is in memdisk, inserting
it into cbfs afterward doesn't override, despite this
being the behaviour in grub.cfg

put it in cbfs explicitly, and skip inserting into memdisk

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-17 01:02:03 +00:00
Leah Rowe b910424b5d fix another very stupid mistake
the last revision disabled building arm64 images!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-08 18:24:57 +00:00
Leah Rowe e3b77b132e fix the stupidest bug ever
no context given, but every rom needs to be re-built.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-08 18:04:51 +00:00
Leah Rowe e80261dd54 Revert "vendor.sh: avoid unnecessary directory copy"
Nope. It was correct before. fml

This reverts commit 2d96fe2a1d.
2024-12-06 10:34:36 +00:00
Leah Rowe 2d96fe2a1d vendor.sh: avoid unnecessary directory copy
the previous commit changed an mv to a cp. what it hacked
was actually a relic of the vgarom download patch that i
did for t480, before mate got native video init working.

this patch is the better fix. i double checked to be sure,
and nothing was using the files at the copied location.
the _extracted directory under cache gets deleted later on,
so it's perfectly acceptable to keep.

the other alternative would have been to simply change
the path in the sch5545 function to appdir, instead of
the cache dir, but who really cares?

this patch removes bloat from lbmk.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-06 01:53:44 +00:00
Leah Rowe 2dc7c5fa72 vendor.sh: fix minor release bug
I should have copied the extract directory, in cases
where it appears as filename_extracted/ under cache/,
but I was moving it instead.

Both locations (cache/file/*_extracted/
and vendorfiles/appdir/) get deleted, on every run of
the vendor script, per target, so this is OK.

The only sin is additional use of disk space, for
archives that are mostly very small and get immediately
deleted anyway.

This one lbmk bug, minor though it may be, prevented
the Libreboot 20241205 release, which (since it's now
the 6th of December) will become Libreboot 20241206
instead - and that gives me time to contemplate whether
I want to do one more change that I had planned for the 5th!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-06 01:24:35 +00:00
Leah Rowe 9dc3c86ae3 vendor.sh: Remove T480 VGA ROM download handling
Libreboot's binary blob reduction policy is crystal clear:

If a blob can be avoided, it must be avoided.

The ThinkPad T480 was using Intel's VGA ROM for graphics
initialisation very briefly, before Mate fixed libgfxinit.

Since libgfxinit is fixed, the Intel VGA ROM is obsolete,
so we should not be handling this at all.

Similarly, the Nvidia ROM handling has been removed, because
Mate is hard-disabling that in the coreboot code anyway, since
the Nvidia dGPU didn't work when tested anyway.

Even if it did, Libreboot's blob policy makes it clear
that Intel graphics with native init from coreboot is to
be the preferred option.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-02 06:12:59 +00:00
Leah Rowe 264928c6cd NEW MAINBOARD: ThinkPad T480
This uses the excellent deguard utility, written by
the excellent Mate Kukri.

A few bugs but it mostly works. Documentation to come
shortly, in lbwww.git.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-01 23:51:20 +00:00
Leah Rowe 28d8dc93a5 vendor.sh: Use the new deguard for 3050micro
I'm adding ThinkPad T480 support next, which requires
the new revision of deguard. Mate Kukri changed the way
deguard is used, in a rewrite of the project, so lbmk
has to change too.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-12-01 01:44:45 +00:00
Leah Rowe f4b6aeea28 rom.sh: Add U-Boot before SeaBIOS and GRUB (x86)
Since U-Boot must be inserted at a specific offset, it's
theoretically possible that other files might overlap, but
cbfstool will work around wherever U-Boot was inserted if
it was inserted first; we don't use specific offsets for
the other files.

This is technically a preventative bug fix, but it fixes
a bug that would probably never occur in practise.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-11-27 02:41:39 +00:00
Leah Rowe 7014b303f9 rom.sh: Remove unnecessary shebang
This is not a main script, and should not be treated as such;
it must never be directly executed by the user.

This script was only ever used inside other scripts, so the
shebang didn't seem to do much at all, but it shouldn't be
there anyway.

Remove it.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-11-26 23:28:42 +00:00
Leah Rowe eea9fc4097 rom.sh: unset displaymode on normal initmode
Otherwise, you get "normal_normal" in the image name.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-11-21 18:34:08 +00:00
Leah Rowe f312e6026f rom.sh: Don't build U-Boot on normal initmode
The "normal" mode in lbmk is where no built-in GPU exists,
or no libgfxinit is used, and SeaBIOS is the first payload,
and SeaBIOS executes VGA ROMs (can't know if it'll start
in VESA or text mode).

U-Boot needs a VESA framebuffer or native coreboot
framebuffer to work correctly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-11-21 18:24:50 +00:00
Leah Rowe 587d588fe4 rom.sh: Don't build txtmode U-Boot images
U-Boot needs a VESA framebuffer or native coreboot
framebuffer to work properly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-11-21 18:07:33 +00:00
Leah Rowe 7d1fd1cc6a rom.sh: Support SeaUBoot for 64-bit x86 U-Boot
Same concept as SeaGRUB, but for U-Boot. SeaBIOS starts, but
has a bootorder file loading U-Boot first, from flash.

You can interrupt it with the ESC menu, to boot something else
in SeaBIOS, including GRUB.

With this, we can effectively provide extremely user-friendly
UEFI-first setups in Libreboot.

Take that, edk2!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-11-21 12:29:05 +00:00
Leah Rowe eb14a176bc Only boot 32-bit u-boot from grub, 64 from seabios
For some reason, 32-bit U-Boot only works when executed from
GRUB, but not SeaBIOS; 64-bit U-Boot only works from SeaBIOS!

This will have to be investigated. Standalone U-Boot, where
U-Boot is the primary payload, has not yet been tested in
Libreboot, and will not be provided for some time due to
stability concerns. More testing is needed!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-11-20 01:19:27 +00:00
Leah Rowe 747b6514ea Add U-Boot x86_64 payload
Currently seems to stall when booted from the GRUB
payload, but works when booted from the SeaBIOS menu.

I also tested it as a standalone payload and it seems
to boot. Will test on hardware next, and start adding
it to more mainboards.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-11-19 02:04:50 +00:00
Leah Rowe c0017c7357 Experimental U-Boot payload (32-bit dtb, U-Boot)
NOTE: Support added for xarch target x86_64-elf,
but U-Boot failed to build with this error:

OBJCOPY lib/efi_loader/helloworld.efi
x86_64-elf-objcopy: lib/efi_loader/helloworld_efi.so: invalid bfd target
make[2]: *** [scripts/Makefile.lib:476: lib/efi_loader/helloworld.efi] Error 1

Since I'm building U-Boot for x86_64 *on* an x86-64
host, and since that is currently the recommended type
of machine to use for lbmk development, and since the
other x86 payloads currently don't cross compile anyway,
this is an acceptable compromise for now. This is because
at present, I'm not making U-Boot the primary payload on x86,
instead preferring to chain it from GRUB and SeaBIOS.

The target.cfg file for x86 u-boot shows xarch/xtree commented.
Uncomment these to compile on crossgcc instead of hostcc.

I mention 64-bit because I initially did this first, but decided
to do 32-bit first. I'll work on the 64-bit one next (SPL).

It's only enabled in QEMU for now.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-11-03 09:22:52 +00:00
Leah Rowe 9bdec645a3 3050micro: Re-enable SeaGRUB
Remove what is now unnecessary bloat, for ensuring that
GRUB is the primary payload; SeaGRUB is the only preference,
as per lbmk design.

The SeaBIOS hanging issue was fixed, so SeaGRUB is OK now.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-10-27 19:32:36 +00:00
Leah Rowe 3f63c6d12f rom.sh: remove unnecessary logic from copyps1bios
the .git directory never exists anyway, when doing a release,
so the purpose this is intended is defeated by lbmk's design.

individual headers say "pcsx-redux team" as copyright anyway,
and the code for generating that COPYING file, with MIT license
and correct years (matching the entire source code for the
open bios) remains correct.

a mitigation instead of this patch might be to maintain a hardcoded
list of authors, and manually update it over time, but this is not
required. however, it may be good practise for upstream to maintain
such a file. perhaps i should contact them?

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-10-20 00:13:35 +01:00
Leah Rowe 34f851742e vendor.sh: Don't use x_ for image MAC address mod
The path might contain spaces and such, which breaks when
using the x_ prefix.

Call err instead.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-10-16 15:48:43 +01:00
Leah Rowe 3b92b7b723 vendor.sh: Handle error status on RUNME.sh
The deguard utility is executed within a subshell, and
the subshell does not handle error status. This patch
fixes that, so that the main shell also exits non-zero.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-10-14 01:15:44 +01:00
Leah Rowe d845791d6f rom.sh: support making pcsx-redux bios release
I also checked the copyright declarations in the
directory src/mips/openbios where the PCSX-Redux BIOS
is, gleaning all the copyright years: 2019-2024 at this
time.

The years will be updated as and when PCSX-Redux is
updated in lbmk. Their BIOS is under MIT so I made lbmk
generate an appropriate COPYING file alongside the binary,
containing:

Copyright (c) 2019-2024 PCSX-Redux authors

Along with the actual text of the MIT license. With all
of this, the PCSX-Redux BIOS can now be included in
Libreboot releases.

No actual tarball is created. The release script in lbmk
simply copies the bin/ directory to ../roms

I'm leaving the PCSX-Redux BIOS release uncompressed,
because, and this will sound patronising because that is
my precise intention: Windows users don't know how to do
anything. If I provide a tarball to Windows users, they
won't know what to do. Libreboot releases always go on rsync
mirrors, which also have HTTP servers with indexing enabled,
for browsing release files.

I mention Windows users, because most people who use the PCSX
Redux BIOS will probably use it on a PlayStation emulator, and
most emulator users are on Windows. I can't really be bothered
to provide it as a .zip archive, and it's only 512kb, so just
provide it uncompressed in Libreboot releases!

Releases were already possible under this scheme, so this
patch really just adds the COPYING file. It's simply a courtesy
to the PCSX-Redux developers, providing proper credit to them.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-10-08 13:27:08 +01:00
Leah Rowe f4de640e45 rom.sh: disable seabios-as-primary if grub is main
on 3050micro, we disable seabios as a primary payload,
making grub a pribary payload instead.

the way it worked, the roms were still named seagrub
and the seabios rom would be compiled, but with the wrong
path, so seabios wouldn't be executed; seabios would hang
anyway, on this board.

instead, engineer it in such a way as to disable seabios_
images on this board. also, rename seagrub_ to grub_.

i normally only permit seagrub, and not grub, but i make an
exception for 3050micro because we know grub works, but seabios
currently hangs on this board (which means no bsd).

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-10-06 10:31:36 +01:00
Leah Rowe c99dced5b1 dell3050micro: make GRUB the primary payload
SeaBIOS is known to hang on this board. It is being investigated.

Add two variable options for target.cfg files:

* seabiosname
* grubname

This string defines where it would be located in CBFS.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-10-06 09:22:21 +01:00
Leah Rowe 44473d6832 git.sh: fix error with cache re-download
in some cases, on a fresh clone, the cached repo already
exists but lbmk tries to download it again. work around
this by checking that the directory exists; it's in the
main if statement, so that the "else" still applies. as
a result, the fallback to a live repo would un-fall back
to doing git-pull if the cached directory exists exists.

if it doesn't seem to make sense, it's because it doesn't.
this whole function needs to be rewritten better.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-10-05 03:52:50 +01:00
Leah Rowe c3ef0a8639 Add config for Dell OptiPlex 3050 Micro
This is using Mate Kukri's port, which was added in
previous lbmk revisions. I've added an IFD that sets
the HAP bit, and unlocks regions as standard.

vcfg is set to 3050micro, which defines downloading
of the MEv11 image and it will run deguard automatically.

I made a small adjustment to vendor.sh, because the hotpatch
logic for deguard uses -C in git, and when doing that, the
specified directory path is relative to that Git repository;
the .patch path has been adjusted accordingly.

Also add 3rdparty/fsp to coreboot/default modules.

This board requires the ifdtool option: -p sklkbl

The -p option tells flashrom what quirks are present in a
given IFD. We don't normally need this on other Libreboot
targets that we currently support. The -p option was needed
for creating this modified IFD, and it is therefore needed in
the inject script. Therefore, an "IFD_platform" option is
specified in a given board's target.cfg file. If this is set,
another variable is set that makes -p be used.

In this case, 3050's target.cfg says:

IFD_platform="sklkbl"

This option enables quirks for skylake/kabylake descriptors,
as required when using ifdtool.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-10-05 03:32:06 +01:00
Leah Rowe 02e76d09c1 add swig to fedora dependencies
needed when compiling u-boot

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-09-30 09:35:46 +01:00
Leah Rowe ff9c250a3e Add Sony PlayStation support to Libreboot
I also added a "cleanargs" argument, similar to the makeargs
argument, to work around a build error.

This builds the PCSX-Redux PS1 BIOS. They reverse engineered
the Sony PS1 BIOS and wrote a free one under MIT license.

Run this:

./mk -b pcsx-redux

The file will appear: bin/playstation/openbios.bin

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-09-26 00:35:18 +01:00
Leah Rowe e7c0109f5d Add deguard logic for Dell OptiPlex 3050 Micro
Copy the downloaded deguard source code into appdir,
and patch it to run as part of lbmk, instead of
standalone. The archived one in src/ is not directly
used; instead, the hotpatched version is used.

This is because the standalone version already has
download logic for the .zip file, but we already
cache that file in cache/ and use that.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-09-24 16:53:48 +01:00
Leah Rowe 242b79aa25 Revert "vendor.sh: print extract errors to /dev/null"
This reverts commit 72fa467cb7.
2024-09-05 19:46:37 +01:00
Leah Rowe 72fa467cb7 vendor.sh: print extract errors to /dev/null
the output isn't really super critical, because it pertains
to files that would just result in a coreboot build error
if they didn't extract, which would still allow me to know
if a given extract function failed.

however, the extract function shows a lot of error output
because it literally bruteforces various extract methods,
when dealing with vendor files.

mitigate this by just printing the errors to /dev/null. this
will prevent users from erroneously thinking that lbmk is
operating under error condition, when it isn't. we do sometimes
get questions about it on irc.

fewer questions on irc is better.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-08-31 01:17:57 +01:00
Leah Rowe 7ab22503ab vendor.sh: use readkconfig on inject too
same as the last change. we must avoid use of make variables,
in sh specifically, when handling these configuration files.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-08-11 22:06:18 +01:00
Leah Rowe d66f6e0d5b vendor.sh: don't load entire coreboot configs
instead, only grep for the entries required, such
as Intel ME paths.

some variables in coreboot configs use $(), which
is used in *make*, on the coreboot build system, and
there refers to variables.

here, we are sourcing them from sh, which treats this
as a mini subshell to run a command; for example
CONFIG_FOO would be executed, which is bad.

The current logic still theoretically has this problem,
with this patch, but the entries we scan from the configs
do not currently have variable names in the strings.

So: filter out just what we need, into a temporary config,
when scanning for vendor files in coreboot configs, and
use the temporary config.

This fixes a build error when compiling for e5520_6mb.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-08-11 21:22:26 +01:00
Leah Rowe aae8cabe73 lib.sh: more verbose error in x_
part of the command was cut off in the output

Signed-off-by: Leah Rowe <info@minifree.org>
2024-08-11 21:22:23 +01:00
Leah Rowe 59894ed555 lib.sh: new function mk() to handle trees in bulk
single-tree projects cannot be handled in bulk, e.g.
./mk -f project1 project2 project3

that is still the case, from the shell, but internally
it is now possible:
mk -f project1 project2 project3

mk() is a function that simply handles the given flag,
and all projects specified.

it does not handle cases without argument, for example
you cannot do:
mk -f

arguments must be provided. it can be used internally,
to simplify cases where multiple single-tree projects
must be handled, but *also* allows multi-tree projects
to be specified, without being able to actually handle
trees within that multi-tree project; so for example,
you can only specify coreboot, and then it would run
on every coreboot tree.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-28 13:35:31 +01:00
Leah Rowe 7fa6052de0 general code cleanup in the build system
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-26 15:49:49 +01:00
Leah Rowe 3bd290f6a6 rom.sh: don't dry-run mkcoreboottar
same as the last change. make the main function a wrapper
that dry-runs the real function.

if the "dry" variable is blank, it executes.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22 23:46:53 +01:00
Leah Rowe a91751a86b rom.sh: don't run mkcorebootbin on trees -d
don't let it execute during dry builds

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22 23:39:36 +01:00
Leah Rowe faefcdf3df rom.sh: fix buggy deletion of cbutils
when badhash=y, the utils should be deleted, but
the check is deleting if badhash isn't n. if the
hash check isn't being performed, then this will
always be the case and the utils are always deleted.

make it positively delete the file only if badhash=y,
not when it isn't n. while this may not sound very
different, it will prevent the utils being deleted and
re-build endlessly in other cases, like when building
release archives and running the inject --nuke mode
on every image that gets built.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22 19:34:19 +01:00
Leah Rowe 40dd0a7cf3 rom.sh: also add grub to seabios images
we want multiple seagrub images made, with different
keymaps, but we only want one non-seagrub image.

however, we also want grub in the non-seagrub image.
it just means that seabios is primarily what the user
wants, and they might occasionally use grub, whereas
the seagrub images are for people who primarily want
grub but may occasionally access the seabios menu.

right now, the seabios images really only contain seabios,
but there's no harm in adding grub to them.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22 18:47:38 +01:00
Leah Rowe d070eb1fef rom.sh: copy tmprom to TMPDIR for modification
don't rely on build/coreboot.rom staying in place,
because sometimes it can get purged under certain
conditions, due to idiosyncrasies in the coreboot
build system, even when we don't explicitly clean it

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22 18:40:42 +01:00
Leah Rowe f85cb69ce0 rom.sh: re-add seabios-only roms and grub keymaps
this time, only handle multiple keymaps on seagrub
images. for images where seabios is first but does
not immediately load grub, whether grub is still
available in flash, just do one image (US Qwerty)

this still results in fewer images per target than
Libreboot 20240612, but should prevent most users
from being annoyed. i got a few people asking
repeatedly, and i hadn't documented yet how to add
keymap.gkb or how to remove bootorder, to get a
different keymap or disable seagrub respectively.

i anticipate that i'll get such questions a lot, even
if i do document it, so i'm reversing that decision.

it doesn't result in much extra code. the new design
in lbmk makes this sort of thing much simpler.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22 09:30:47 +01:00
Leah Rowe ba4278e0c6 include/rom.sh: use ccache when building coreboot
ccache now required, in build dependencies

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-21 07:39:47 +01:00
Leah Rowe 31f1e4dadf vendor.sh: don't use XBMK_CACHE for appdir
the me_extract function prefixes it with PWD in
some cases, but we can't predict where appdir
will point to.

the "app" directory is not intended to be a cache
anyway, so it doesn't make sense to put it in
the cache directory.

it's essentially scratch memory.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-19 00:48:09 +01:00
Leah Rowe 2b50b3ea9a put cachedir in environmental variable
XBMK_CACHE is now used, instead of hardcoding cache/

this is exported initialised to cache/, if unset.
this means you can set your own directory, and it means
./update release will use the same directory.

this means bandwidth wastage is further avoided.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-19 00:37:11 +01:00
Leah Rowe 0e0b12a630 git.sh: warn when a cached clone fails
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-18 23:23:50 +01:00
Leah Rowe 9b1b955767 git.sh: fix typo in git command
the || : condition should be used, whereas i just
wrote : by mistake. this was done in a previous change.

fix it now.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-18 23:11:45 +01:00
Leah Rowe 82bdf27072 git.sh: fix lack of error exits on fault
a previous change made it more redundant, falling back
on old behaviour (direct downloading, not cached), but
the way it's done means that the function never returns
an error condition in practise.

this patch fixes it.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-18 23:09:59 +01:00
Leah Rowe c6e6d96ce1 allow using coreboot's build system to add payload
lbmk must still define payloads, but specific configs
may use coreboot's build system instead.

you might use this to add your own config with, say,
tianocore payload, using coreboot.git to build it,
rather than using lbmk's choice of payloads.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17 19:06:53 +01:00
Leah Rowe 79f50b3791 unify caching of files into cache/
hash/ becomes cache/hash/

repo/ becomes cache/repo/

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17 18:39:42 +01:00
Leah Rowe a9f9de885c unified cache file handling for vendorfile/subfile
lib.sh download() is used by subfile handling in git.sh,
e.g. crossgcc tarballs, and also the vendor scripts.

vendor files are cached, but not subfiles for repos.

cache both, under cache/file/, saved with the name equal
to the checksum, so: cache/file/CHECKSUM

also move vendorfiles/app/ to cache/app/ in this change.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17 18:15:52 +01:00
Leah Rowe ef867eccca git.sh: remove previous tmprepo if args>5
if doing a retry, the directory may still exist, which
would make git clone yield an error response; the existing
directory will have been the one that failed to reset, so
let's delete it.

the one deleted is not the cache (repo/PROJECT/), thus
otherwise maintaining current behaviour.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17 17:28:27 +01:00
Leah Rowe 4d5385a14d git.sh: try direct clone if cached git fails
normally, a project is cached at repo/PROJECT/, and
cloned from there to the final destination.

errors lead to a calling of $err, but this will result
in a return if done from inside a subshell, of non-zero
value, so use this to re-try with a 6th argument when
calling tmpclone().

in most cases, this fallback will never kick in, but
it will kick in resetting or patching the cached clone
fails; specifically, we are interested in the reset part.

a given project name may change repositories in lbmk at
a given time. if this happens, and the old one is cached,
the overall result of this patch is that lbmk will fall
back to the old behaviour, where git urls are tried
directly, without caching.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17 17:09:34 +01:00
Leah Rowe 0cd52fc7fd git.sh: re-try git pull three times
mitigate jittery internet connections

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17 16:55:05 +01:00
Leah Rowe 0b9cd77fe7 trees: auto-delete+auto-rebuild if project changes
actual source code is not scanned, but config directories are
scanned. simply get the checksum of each file under config/
pertaining to a given project/tree, and also for the given
target. coreboot utilities are also handled.

if it changes, in any way, delete and re-build automatically.

such deletions should probably still be done manually, as part
of understanding the build system, but this change should make
the build system much easier to use during development.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17 16:34:34 +01:00
Leah Rowe 583502027e git.sh: don't download to src/project/project/
re-use repo/project/

this means that single- and multi-tree projects now
have a unified cached git repo location, as per the
new rules, thus saving on disk space usage.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17 13:20:51 +01:00
Leah Rowe 9f09728aca git.sh: cache git downloads to repo/
do it based on the URL, e.g. https://review.coreboot.org/coreboot
becomes repo/coreboot

the downside is if you have two projects with repo urls specifying
the same string at the end, but this isn't the case at the moment
and likely won't be the case, but it's a theoretical issue.

this saves on bandwidth when downloading identical submodule repos
between multiple trees within the same multi-tree project

for example, coreboot 3rdparty/vboot is no longer downloaded more
than once, instead cloned locally on subsequent downloads.

if repo/DIR exists, git-pull is attempted, but errors do not result
in a non-zero exit, by design.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17 13:01:12 +01:00
Leah Rowe 3263eeb6b6 trees: general code cleanup
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 23:11:23 +01:00
Leah Rowe c0a4df688c trees: use wrapper for dry-running make commands
instead of using lots of if/else conditions, do that once
and set a variable, dry, to :

if not doing a dry run, the variable is empty. prefix this
variable in places where you don't want a certain action to
be performed, on dry runs.

more specifically, : does *nothing* and always returns with
zero status (success).

this results in cleaner code, and a small sloccount reduction.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 21:00:16 +01:00
Leah Rowe e9f66ec4a8 remove executable permission on include/
files under include/ should never be executed directly

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 02:14:25 +01:00
Leah Rowe 91994b7852 rom.sh: actually say if a cbutil exists
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 02:04:25 +01:00
Leah Rowe 97b777715b rom.sh: avoid re-building cbutils if built
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 01:37:27 +01:00
Leah Rowe 494b94799e rom.sh: only before cbutils before coreboot
otherwise, due to the idiosyncratic nature of the coreboot
build system, the coreboot.rom gets wiped out.

cbutils is still handled by premake. ensure that payloads are
only inserted just after running the coreboot make command.

fixes a build issues introduced on 9020sff, previously unhandled.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10 01:24:05 +01:00
Leah Rowe 5a1d2401c2 minor cleanup
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-09 03:09:49 +01:00
Leah Rowe 9646172145 trees: support -d (dry run) for custom build logic
-d does the same as -b, except for actually building
anything! in effect, it does the same as -f (fetch)
except that the resulting variable assignments will
not be recursive (as with -f).

if -d is passed, configuration is still loaded, defconfig
files are still cycled through, and more importantly:

helper functions are still processed.

the grub, serprog and coreboot helper functions have
been modified to return early (zero status) if -d is
passed.

this behaviour will be used to integrate vendor.sh
logic in with the trees script, for cases where the
user wants to only handle vendor files. e.g.:

./update trees -b coreboot x230_12mb

this would download the files as usual, build coreboot,
with those files, and then build the payloads. but:

./update trees -d coreboot x230_12mb

this would download the files, NOT build coreboot, and
NOT build the payloads.

this change increases the sloccount a bit, but i'm relying
on the fact that the vendor.sh script already re-implements
config handling wastefully; the plan is to only use trees.

for now, simply stub the same ./vendor download command.

there is one additional benefit to doing it this way:

this method is *per-kconfig* rather than per-target.
this way, one kconfig might specify a given vendor file
that is not specified in the other. although the stub
still simply handles this per target, it's done in premake,
which means that the given .config file has been copied.

this means that when i properly re-integrate the logic
into script/trees, i'll be able to go for it per-kconfig.

the utils command has been removed, e.g.
./update trees -b coreboot utils default

the equivalent is now:
./update trees -d coreboot default

this would technically download vendor files, but here
we are specifying a target for which no kconfigs exist;
a check is also in place, to avoid running the vendor file
download logic if tree==target

the overall effect of this change is that the trees script
no longer contains any project-specific logic, except for
the crossgcc build logic.

it does include some config/data mkhelper files at the top,
for serprog and coreboot, so that those variables defined in
those files can be global, but another solution to mitigate
that will also be implemented in a future commit.

the purpose of this and other revisions (in the final push
to complete lbmk audit 6 / cbmk audit 2) is to generalise as
much logic as possible, removing various ugly hacks.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-09 02:55:20 +01:00
Leah Rowe e850c06bd2 rom.sh: only make rom tarball if release=y
Signed-off-by: Leah Rowe <info@minifree.org>
2024-07-08 02:22:22 +01:00
Leah Rowe 2f3cc5d379 rom.sh: new file, to replace script/roms
stub it from the trees script. the way it works now,
there is less code in the build system.

./build roms

this is no longer a thing

./build roms serprog

this is also no longer a thing. instead, do:

./update trees -b coreboot targetnamehere

./update trees -b pico-serprog

./update trees -b stm32-vserprog

the old commands still works, which causes the new
commands to run

coreboot roms now appear in elf/, not bin/, as before,
but those images now contain payloads.

NOTE: to contradict the above: ./build roms is no
longer a thing, in that it's now deprecated, but
backward compatibility is present for now. it will
be removed in a future release.

./build roms list also still works! it will do:
./update trees -b coreboot list

also:
./update trees -b grub list
this is now possible too

if a target "list" is provided, for multi-tree sources,
the targets are shown.

there is another difference: seagrub roms are now seagrub_,
instead of seabios_withgrub.

seabios-only roms are no longer provided, where grub is also
enabled; only seagrub is used. the user can easily remove
the bootorder file, if they want seabios to not try grub first.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-07 21:25:07 +01:00
Leah Rowe b7f12ade03 vendor.sh: remove mkdirs()
merge it into the only calling function

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 19:22:58 +01:00
Leah Rowe 0018600d6e git.sh: simpler for loop in git_am_patches()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 18:23:15 +01:00
Leah Rowe 5882056a27 git.sh: merge for loops in git_am_patches()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 18:16:34 +01:00
Leah Rowe 67421a21fd trees and git.sh: tidy up global variables
some of the variables only initialised in git.sh are
also used in the trees script, which is technically ok
because git.sh is included from the trees script, but
it makes more sense to declare them in the latter.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 18:10:22 +01:00
Leah Rowe 55dbd72aaf git.sh: simplified initialisation of "loc"
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 17:56:32 +01:00
Leah Rowe 5ba0433b5f git.sh: condense fetch_targets() a bit
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 17:04:24 +01:00
Leah Rowe 4772186b19 git.sh: short git_prep command in fetch_targets()
loc is already set, and will correspond to the same
path, so we can quite conveniently use it.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 17:02:44 +01:00
Leah Rowe 017fd8259e git.sh: merge prepare_new_tree with fetch_targets
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 15:49:35 +01:00
Leah Rowe 7394960619 git.sh: remove duplicate "xtree" variable
it's already defined inside the trees script

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 14:00:19 +01:00
Leah Rowe 789631ccb8 trees, multi: download bare project *before* trees
when downloading multi-tree projects, the rev can be reset
to HEAD instead of the actual rev for a given target. this
occurs when the bare repo (e.g. src/coreboot/coreboot) does
not exist and has to be downloaded first.

bare repository downloading does not rely on target.cfg, in
this context, only pkg.cfg, but it uses the same variable
names (e.g. "rev").

instead of using a separate variable name, thus increasing
code complexity (which is the exact opposite of what i want
to do), do the bare repository download first.

this means that the git.sh script is much cleaner now, for
multi-tree projects, in that it *only* copies the bare repo
then runs git_prep; in that context, the bare repo is cloned
directly by calling the relevant function from script/trees,
which is the same behaviour as when cloning single-tree
project sources.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 00:40:30 +01:00
Leah Rowe a5e724e51e trees: unified multi-tree configuration handling
the same function that loads configurations for single-tree
projects has been merged with the function for multi-tree
configs in git.sh, and that functionality has been removed
from git.sh; now it is all unified in the trees script.

as the saying goes: write one program to do one thing well.

the purpose of git.sh is to download source code, but not
to handle configuration files; the latter is meant to be
handled by the trees script, which then calls into git.sh
before running the build logic for that given project.

additionally: the "seen" files are no longer handled, at all.
the logic there was added ages ago, because at the time, i was
considering whether to separate configuration into a new
repository, so that users could more easily make their own
configuration, so it was a guard against misconfiguration.
however, that decision was canceled and we're always very
careful not to introduce a loop; if a loop does occur, the
worst that can possibly happen is you waste some CPU cycles.

Instead, print (on standard output) what config file is being
used, so the operator can see when an infinite loop occurs.

ALSO:

remove _setcfgarg in load_project_config()

it was used to skip when a target.cfg file didn't exist,
specifically on single-tree projects, but this is now
handled using -f instead, on the while loop inside that
function, so _setcfgarg is now a redundant variable.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30 00:02:03 +01:00
Leah Rowe 7bf2835207 git.sh: rename Fetch_project_trees fetch_targets
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:59:16 +01:00
Leah Rowe e0c244f37b git.sh: rename fetch_project_repo to fetch_project
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 19:58:03 +01:00
Leah Rowe d1b7882c5a vendor.sh readcfg: split the -b coreboot command
just add a line break to make the code more readable

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 18:55:41 +01:00
Leah Rowe 088a79918b vendor.sh: remove unnecessary check
the next command is a copy, which would give us the error
if the file doesn't exist, and an appropriate message

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 03:47:57 +01:00
Leah Rowe 8c5f78d3d3 vendor.sh: condense inject() a bit
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 03:27:31 +01:00
Leah Rowe 42925ce79f vendor.sh extract_kbc1126: use quote on file check
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 03:09:03 +01:00
Leah Rowe ddcc74777e vendor.sh extract_kbc1126ec: simplify build check
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 03:06:58 +01:00
Leah Rowe 7a12cd7704 vendor.sh: simplify e6400 vga rom file check
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 02:25:22 +01:00
Leah Rowe 163bcfcdfd vendor.sh: simplify variable checks for e6400vga
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 02:23:02 +01:00
Leah Rowe cbac2087cc vendor.sh: condense fetch() a bit more
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 02:21:00 +01:00
Leah Rowe 675d2c0376 vendor.sh: remove unnecessary check
this is over-engineering, because we do not allow just
about any path to be provided; it's not provided as an
argument in a command, for example.

this is dictated by a configuration file, which we control.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29 02:14:00 +01:00