replace it with logic that simply uses "." to load
files directly. for this, "vcfg" is added as a variable
in coreboot target.cfg files, referring to a directory
in config/vendor/ containing a file named pkg.cfg, and
this file then contains the same variables as the
erstwhile config/vendor/sources
config/git files are now directories, also containing
pkg.cfg files each with the same variables as before,
such as repository link and commit hash
this change results in a noticeable reduction in code
complexity within the build system.
unified reading of config files: new function setcfg()
added to lib.sh
setcfg checks if a config exists. if a 2nd argument is
passed, it is used as a return value for eval, otherwise
a string calling err is passed. setcfg output is passed
through eval, to set strings based on config; eval must
be used, so that the variables are set within the same
scope, otherwise they'd be set within setcfg which could
lead to some whacky results.
there's still a bit more more to do, but this single change
results in a substantial reduction in code complexity.
Signed-off-by: Leah Rowe <leah@libreboot.org>
again: the "depend" variable must never be used for subprojects
that point to a subdirectory of the main project, because there's
no clean way of handling this in case of error conditions.
make it a submodule under config/submodules/. this is for the
documentation, including static site generator documentation,
and image files (photos).
as of this revision, there are now only those "depend" projects
defined in config/git/, where the destination directory of the
subject is not a subdirectory of the main project, so:
in a subsequest revision, i will mitigate an existing bug whereby
failure of the dependency project leaves the main one still
intact, breaking builds; this revision enables that to be done.
from now on, subproject-to-subdirectory-of-main-project will
be avoided in config/git/; config/submodules/ will be used.
Signed-off-by: Leah Rowe <leah@libreboot.org>
same as the previous patch, we must no longer use "define"
variables in config/git/ when the path is a subdirectory of
a given project, because it means that the download can only
happen after the main one, and currently if that fails, the
download of the main repo would remain intact, breaking future
builds in ways that we can't control - to be clear, it could
be controlled, but with added code complexity in the build
system, so:
put it in config/submodules/
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't define it as a "depend" variable in config/git/,
because it means putting the files in a subdirectory of
an existing project was was already then downloaded, and
that means it can't be downloaded first; if the download
of it fails, the old download is left intact.
this bug isn't currently fixed in the build system, at all,
so this and other patches are being made to mitigate it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this brings the handling of serprog projects in sync
with canoeboot, which relies on the "depend" variable
to get the needed submodules, because cbmk does not
download submodules for these projects
lbmk does download submodules. i want it in sync with
cbmk for this, to make merging easier between both
projects, because i'm going to make a change on both
projects, whereby config/submodules/ is used exclusively
Signed-off-by: Leah Rowe <leah@libreboot.org>
it imports the same environmental variable fix because
i had the same buggy TMPDIR check there. i fixed that
upstream in untitled.
import the new untitled revision.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Re-add xHCI only on haswell and broadwell machines, where
they are needed. Otherwise, keep the same GRUB code.
The xHCI patches were removed because they caused issues
on Sandybridge-based Dell Latitude laptops. See:
https://codeberg.org/libreboot/lbmk/issues/216
The issue was not reported elsewhere, including on the
Haswell/Broadwell hardware where they are needed, but the
build system could only build one version of GRUB.
The older machines do not need xHCI patches, because they
either do not have xHCI patches, or work (in GRUB) because
they're in EHCI mode when running the payload.
So, the problem is that we need the xHCI patches for GRUB
on Haswell/Broadwell hardware, but the patches break
Sandybridge hardware, and we only had the one build of GRUB.
To mitigate this problem, the build system now supports
building multiple revisions of GRUB, with different patches,
and each given coreboot target can say which GRUB tree to use
by setting this in target.cfg:
grubtree="xhci"
In the above example, the "xhci" tree would be used. Some
generic GRUB config has been moved to config/data/grub/
and config/grub/ now looks like config/coreboot/ - also,
the grub.cfg file (named "payload" in each tree) is copied
to the GRUB source tree as ".config", then added to GRUB's
memdisk in the same way, as grub.cfg.
Several other design changes had to be made because of this:
* grub.cfg in memdisk no longer automatically jumps to one
in CBFS, but now shows a menuentry for it if available
* Certain commands in script/trees are disabled for GRUB,
such as *config make commands.
* gnulib is now defined in config/submodule/grub/, instead
of config/git/grub - and this mitigates an existing bug
where downloading gnulib first would make grub no longer
possible to download in lbmk.
The coreboot option CONFIG_FINALIZE_USB_ROUTE_XHCI has been
re-enabled on: Dell OptiPlex 9020 MT, Dell OptiPlex 9020 SFF,
Lenovo ThinkPad T440p and Lenovo ThinkPad W541 - now USB should
work again in GRUB.
The GRUB payload has been re-enabled on HP EliteBook 820 G2.
This change will enable per-board GRUB optimisation in the
future. For example, we hardcode what partitions and LVMs
GRUB scans because * is slow on ICH7-based machines, due
to GRUB's design. On other machines, * is reasonably fast,
for automatically enumerating the list of devices for boot.
Use of * (and other wildcards) could enable our GRUB payload
to automatically boot more distros, with minimal fuss. This
can be done at a later date, in subsequent revisions.
Signed-off-by: Leah Rowe <leah@libreboot.org>
I've rebased the workaround-mx patch as follows. See:
commit 9a11cbf21a5078bcdb8db7584c44a9ee17020db4
Author: Nico Huber <nico.h@gmx.de>
Date: Fri Jan 13 01:19:07 2023 +0100
Let the flash context directly point to the used master
This change, now upstream in flashprog, made me have to do this in
the patch. I changed this:
flash->mst->spi.command(flash, sizeof(cmd), sizeof(buf), cmd, buf);
to this:
flash->mst.spi->command(flash, sizeof(cmd), sizeof(buf), cmd, buf);
It should work fine. This update imports the following upstream
patches from flashprog:
* 5b4fdd1 z60_flashprog.rules: Add udev rule for CH347
* 72c9e40 meson: Check for CPU families with known raw mem access
* 3458220 platform/meson: Port pciutils/pci.h workaround to Meson
* f279762 platform/meson: Check for libi386 on NetBSD
* 14da5f7 README: Convert to Markdown
* 8ddea57 README: Document branching and release policy
* 2522456 util/list_yet_unsupported_chips.sh: Fix path
* cbf9c11 spi: Don't cross 16MiB boundaries with long writes
* 823a704 dediprog: Skip warning on first attempt to read device string
* e8463c8 dediprog: Revise prefix check for given programmer id
* 38af1a1 dediprog: Revise id matching
* 4661e7c amd_spi100: Use flashprog_read_chunked() for progress reporting
* cdcfda2 read_memmapped: Use flashprog_read_chunked() for progress reporting
* 7679b5c spi25: Replace spi_read_chunked() with more abstract version
* ca1c7fd spi25: Normalize parameters of spi_nbyte_read()
* e36e3dc dediprog: Use default_spi_write_256
* 522a86d linux_spi: Use default_spi_read()/_write_256()
* 806509b cli_classic: Turn progress reporting into a progress bar
* 842d678 libflashrom: Return progress state to the library user
* aa714dd flashprog.c: Let select_erase_functions() return byte count
* 2eed4cf serprog: Add SPI Mode and CS Mode commands
* 821a085 dediprog: Implement id reading for SF600 and later
* 274e655 dediprog: Read device string early
* 0057822 dediprog: Add protocol detection for SF700 & SF600Plus-G2
* fb176d2 dediprog: Use more general 4BA write mode for newer protocols
* 0ab5c3d dediprog: Split device type and version parsing
* bdef5c2 dediprog: Use unsigned conversions to parse device string
* 5262e29 dediprog: Try to request 32B device string (instead of 16B)
* e76e21f dediprog: Get rid of some unnecessary hex constants
* 5a09d1e udelay: Lower the sleep vs delay threshold
* 03ad4a4 linux_mtd: Provide no-op delay implementation
* 211c6ec serprog: Refine flushing before synchronization
* 383b7fe serprog: Test synchronicity before trying to synchronize
* d7318ea serprog: Move synchronicity test into separate function
* 9a11cbf Let the flash context directly point to the used master
* aabb3e0 writeprotect: Hook wp functions into the chip driver
* 89569d6 memory_mapped: Reduce `decode_sizes` to a single `max_rom_decode`
* 929d2e1 internal: Pass programmer context down into chipset enables
* 7c717c3 internal: Pass programmer context down into board enables
* e3a2688 Pass programmer context to programmer->init()
* 2b66ad9 Start implementing struct flashprog_programmer
* 4517e92 memory_bus: Drop stale `size == 0` workaround and FIXME
* b197402 memory_bus: Split register mapping into own function
* 0e76d99 memory_bus: Move (un)map_flash_region into par master
* 9eec407 Perform default mapping only for respective chips
* 56b53dd wbsio_spi: Request memory mapping locally
* 5596190 it87spi: Request memory mapping locally
* 46449b4 spi25: Drop stale `bus == SPI` guards
* ab6b18f spi25: Move 4BA preparations into spi_prepare_4ba() hook
* 901fb95 Add prepare/finish_access() hooks for chip drivers
* a96aaa3 dediprog: Support long writes of 16MiB and more
* 1338936 Consider 4BA support when filtering erase functions
* 8d36db6 flashprog.8: Fix up serprog example
* d2ac303 flashprog.8: document new serprog cs parameter
* d1b9153 chipset_enable.c: Add Genoa to mendocino entry
Signed-off-by: Leah Rowe <leah@libreboot.org>
I'm on a schedule here and don't have time to do the
release changelog before actually compiling the release.
I'm pushing the release changelog / news announcement
*while the release is building*. Therefore, the actual
release archive will contain Libreboot documentation, but
from the lbwww revision just before the release announcement.
(a changelog file is still generated from Git, and included
in releases)
Signed-off-by: Leah Rowe <leah@libreboot.org>
GRUB has not pushed many patches to master since the recent 2.12
release, but there are a number of interesting fixes.
libreboot is doing a release soon. bump to latest grub revision.
Some of the new patches in GRUB are interesting:
XFS fixes:
"fs/xfs: Handle non-continuous data blocks in directory extents"
68dd65cfdaad08b1f8ec01b84949b0bf88bc0d8c
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2254370
Apparently, XFS could not boot in some reports, though this was
likely with BIOS or UEFI GRUB; no such reports were made to libreboot
"gfxmenu/view: Resolve false grub_errno disrupting boot process"
39c927df66c7ca62d97905d1385054ac9ce67209
"util/grub-fstest: Add a new command zfs-bootfs"
28c4405208cfb6e2cea737f6cbaf17e631bac6cd
The gnulib revision does not need to be updated at this time.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this merges the fix from:
https://codeberg.org/libreboot/pico-serprog/pulls/1
however, PRs are not to be sent there. riku merged it in
his repository, and i pulled it in the mirror hosted
on libreboot codeberg
Signed-off-by: Leah Rowe <leah@libreboot.org>
Riku introduced three new patches:
* Add support for multiple chip selects. This allows you to
control multiple chips from the same clip, on systems with
dual flash setups, at least theoretically.
* Enable pull-up on unused chip selects - pull them high so
that chips you connect that to are deactivated while flashing
the target chip. This could be used on thinkpad W541 for
instance, where miso/mosi have 0ohm between them via the two
flash ICs. You could pull the other chip select high.
* Documentation for the above, in the pico-serprog readme.
This goes in tandem with a patch from Riku, present in the
recently integrated flashprog project, namely:
commit ddb6d926783d4f9cbee04c7392718ed8f89daa0e
Author: Riku Viitanen <riku.viitanen@protonmail.com>
Date: Mon Jan 15 19:15:49 2024 +0200
serprog: Add support for multiple SPI chip selects
This functionality will therefore be present in the next
release of Libreboot.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Nico Huber is the rightful project lead. I do not support
the coup that occured within the flashrom project. Nico
has always been of great service to the Libreboot project,
by virtue of his work on both coreboot and flashrom.
Nico Huber was unfairly removed from the flashrom project
infrastructure, due to unfounded accusations hurled at him
by flashrom's new project lead. The accusations are unfounded
because no evidence was given.
Use Nico Huber's fork, named flashprog. We will work with
flashprog from now on.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Actually, it's 2 commits after 2.12, because there was a
patch added afterwards, fixing a build issue on Gentoo.
These changes are present in GRUB 2.12, relative to the
revision that we previously used on lbmk:
* b835601c7 build: Include grub-core/extra_deps.lst in dist
* 8961305b4 Bump version to 2.13
* 5ca9db22e Release 2.12
* 477a0dbd5 efi: Add support for reproducible builds
* dcc1af5d6 efi: Generate stack protector canary at build time if urandom is available
* e424e945c efi: Initialize canary to non-zero value
* 7c8ae7dcb gfxmenu/gui_image: Fix double free of bitmap
* 63fc253fc commands/acpi: Fix calculation of ACPI tables addresses when processing RSDT and XSDT
* f20123072 libnvpair: Support prefixed nvlist symbol names as found on NetBSD
* a13df3d15 bootstrap: Don't check gettext version
* 6d2aa7ee0 kern/mm: Use %x and cast for displaying sizeof()
* b3d49a697 configure: Add RPATH for freetype on NetBSD
* 52dbf66ea configure: Add *BSD font paths
* 2d6a89980 autogen: Accept python3.10 as a python alternative
* 3d4cb5a43 build: Rename HAVE_LIBZFS to USE_LIBZFS
* e4dbe5cfa gnulib: Tolerate always_inline attribute being ignored
* 31e47cfe2 util/editenv: Don't use %m formatter
* f5905f656 osdep/bsd/hostdisk: Fix NetBSD compilation
* cb1824a87 osdep/generic/blocklist: Fix compilation
* 2f3faf02c disk/diskfilter: Remove unused variable
* 3815acc57 build: Tolerate unused-but-set in generated lexer/bison files
* c129e44e7 loader/i386/bsdXX: Fix loading after unaligned module
* 89fbe0cac grub-core/Makefile.am: Make path to extra_deps.lst relative to $(top_srcdir)/grub-core
* 353beb80c util/grub-install: Move platdir path canonicalization after files were copied to grubdir
* f18a899ab util/grub-mkstandalone: Ensure deterministic tar file creation by sorting contents
* ed74bc376 util/grub-mkstandalone: Ensure stable timestamps for generated images
* 069cc46c9 net/http: Fix gcc-13 errors relating to type signedness
* e7a831963 templates: Reinstate unused version comparison functions with warning
* 3f9eace2d util/grub-install: Delay copying files to {grubdir,platdir} after install_device was validated
* e60015f57 efi: Set shim_lock_enabled even if validation is disabled
* e35683317 docs: Improve bli module documentation
* 57059ccb6 bli: Add explicit dependency on the part_gpt module
* 154dcb1ae build: Allow explicit module dependencies
* 17c68472d kern/ieee1275/init/ppc64: Display upper_mem_limit when debugging
* 5f8e091b6 kern/ieee1275/init/ppc64: Fix a comment
* dc569b077 kern/ieee1275/ieee1275: Display successful memory claims when debugging
* 0ac3d938a loader/powerpc/ieee1275: Use new allocation function for kernel and initrd
* 2a9a8518e kern/ieee1275/cmain/ppc64: Introduce flags to identify KVM and PowerVM
* 679691a13 kern/ieee1275/init/ppc64: Rename regions_claim() to grub_regions_claim()
* d49e86db2 kern/ieee1275/init/ppc64: Add support for alignment requirements
* fe5d5e857 kern/ieee1275/init/ppc64: Return allocated address using context
* ea2c93484 kern/ieee1275/init/ppc64: Decide by request whether to initialize region
* 0bb59fa9a kern/ieee1275/init/ppc64: Introduce a request for regions_claim()
* aa7c13226 fs/xfs: Add large extent counters incompat feature support
Most notable in the above log, that are beneficial to Libreboot
users, are:
aa7c13226 which improves XFS support (large extents), which is default
now on many setups.
ed74bc376 which introduces more stable timestamp generation when using
grub-mkstandalone. this is what lbmk uses to generate grub.elf, whereas
grub previously only implemented this fix on mkimage which we don't use
f18a899ab which ensures deterministic (reproducible) tar file creation
by sorting contents (file names / directories). this is done by sorting
the entries
f5905f656 which improves grub build system reliability on netbsd and
openbsd systems - useful for us because an ambition of lbmk is to port
the build system to run on bsd systems, and we will still want grub -
several other of the changes here are beneficial for BSD aswell, all
or most of them by Vladimir Serbinenko
Signed-off-by: Leah Rowe <leah@libreboot.org>
riku committed a new patch, that fixes build errors
when PICO_DEFAULT_LED_PIN is not defined, on a given
board. in such cases, riku's new patch just disables
handling of the status LED, but LEDs continue to work
on boards where it is defined.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the new revision sets drive level to 12mA instead
of the default 4mA. 16-20mA is the maximum tolerated
level for data lines, on most flash ICs, so 12mA is
relatively safe.
riku did this a while ago, tested on pico pi.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Now the revision is:
64e3cee72ab8f5876abfebb263b5e6cf7c4a9a4e
The old revision was:
e58b870ff926415e23fc386af41ff81b2f588763
With this new revision update, the following patches have
been imported from the upstream GRUB project:
* 64e3cee72 gpt: Add compile time asserts for guid and gpt_partentry sizes
* 7de6fe963 types: Split aligned and packed guids
* 5fc985bfd gpt_partition: Mark grub_gpt_partentry as having natural alignment
* 7ad30299d efi: Deduplicate configuration table search function
* c6cf807fc lsefi: Add missing static qualifier
* a964e359b types: Fix typo
* 3f79e3b15 util/grub-mount: Check file path sanity
* 85e40b36e configure: Make the DJVU_FONT_SOURCE configurable with --with-dejavufont=FILE
* 2d6631d2a configure: Make the Unifont FONT_SOURCE configurable with --with-unifont=FILE
* 07318ee7e fs/xfs: Fix XFS directory extent parsing
* ad7fb8e2e fs/xfs: Incorrect short form directory data boundary check
* 4e10213de Revert "zfsinfo: Correct a check for error allocating memory"
* 4266fd2bb disk/i386/pc/biosdisk: Read up to 63 sectors in LBA mode
* cab04dcda kern/i386/pc/init: Flush cache only on VIA C3 and earlier
* 3c7e84257 fs/btrfs: Zero file data not backed by extents
* 4bcf6f747 kern/ieee1275/init: Restrict high memory in presence of fadump on ppc64
* cf58eca2a tests/util/grub-shell: Enable RNG device to better test stack smashing
* c3bdf263f kern/efi/init: Disable stack smashing protection on grub_efi_init()
* 95963d97f disk/cryptodisk: Add support for LUKS2 in (proc)/luks_script
* 016f14257 disk/cryptodisk: Optimize luks_script_get()
* f7a663c00 term/serial: Ensure proper NULL termination after grub_strncpy()
* a19e47ca4 commands/efi/lsefisystab: Print the UEFI specification revision in human readable form
Signed-off-by: Leah Rowe <leah@libreboot.org>
now the docs are complete, in releases. they
contain the libreboot site, libreboot images,
the untitled static site generator and untitled
static site generator documentation.
Signed-off-by: Leah Rowe <leah@libreboot.org>
with this, you can just do:
cd src/docs
./build
the html files would then be available for
publishing, if you wish, or you could set up
a local httpd to view them.
if you have pandoc installed, this will build the
markdown files into html
untitled static site generator is what generates
the html files, from the markdown files, on the
website. it will now also be included in releases.
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's not used by anywhere else in lbmk, but the release build
script will automatically download each project named as per
file names in config/git/
this is a stupidly simply way to prove documentation in
libreboot releases, and i've used current revisions corresponding
to the Libreboot 20231021 release, for this 20231021 release
of lbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
notabug is unreliable, even as a backup.
why, just today, it was offline! all day.
i originally moved libreboot away from notabug,
to codeberg instead, but kept the notabug account
online, and i still push to it when it's online.
however, notabug seems to be in a terminal state
of neglect by its admins, so lbmk should not use it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
flashrom-stable isn't really going anywhere
i'll decide at some future point what to do
with flashrom. for now, just give latest rev
Signed-off-by: Leah Rowe <leah@libreboot.org>
the grub backup was the same gnu server
i decided to host grub on codeberg, as backup
(gnu links as primary is ok)
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's ok for now to use it as a backup.
where only github was specified, i mirrored each
given repository to codeberg as main repo for lbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's not actually needed in lbmk
flashrom can be downloaded separately by the user,
if they want to flash their chip
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't put multiple downloads in the same files, except
when they are dependencies that go inside the directory
of another download.
by doing this, the following functionality will become
possible: clean every project or build every project,
or maybe fetch every project, based entirely on the
names of these files.
this will be used later to simplify the release script.
Signed-off-by: Leah Rowe <leah@libreboot.org>
coreboot closely matches upstream, whose current release
is version 1.2 from 2018, and coreboot has not changed it
in any meaningful way.
the upstream did add patches since, but they are documentation
patches only.
this means: we do not need to use the upstream version
Signed-off-by: Leah Rowe <leah@libreboot.org>
most of the changes since last revision aren't very
useful to us; most of them pertain to fs/ntfs, but
there is one that is interesting:
48f569c78a496d3e11a4605b0999bc34fa5bc977
kern/acpi: Skip NULL entries in RSDT and XSDT
Signed-off-by: Leah Rowe <leah@libreboot.org>
build/release/src was partly re-written to accomodate this
memtest86plus was patched to have a central Makefile, and
lbmk modified to use that, rather than mess with build32
and build64. the central Makefile just builds both targets
or cleans both targets
Signed-off-by: Leah Rowe <leah@libreboot.org>
With this change, it's still possible to have a single
file at config/git/revisions, but this has been scrapped.
Instead, multiple files now exist under config/git/ with
the same modules declared, but the files are separated
logically. List of files under config/git:
* bios_extract
* biosutilities
* coreboot
* flashrom
* grub (gnulib also defined here)
* me_cleaner
* memtest86plus
* seabios
* serprog (multiple projects defined)
* u-boot
* uefitool
The rationale behind this change is simple: in the future,
we will stop relying on build systems within imported
projects for the import of git submodules. Instead, we
will handle them directly in lbmk.
Additionally, a Linux payload is planned for Libreboot, made
easier by the recent audit (script handle/make/config makes
it easy to integrate Linux, and handle cross-compilers for
userland utilities); a "linux" file under config/git/ could
also define rules for each project besides linux, such as
musl libc, busybox and other utilities.
Signed-off-by: Leah Rowe <leah@libreboot.org>
It's now 44 revisions above 2.12-rc1, not 17 above.
The additional patches (in GRUB master) contain several
important fixes, including cryptodisk and ZFS fixes plus
a few other interesting changes, namely:
14c95e57fddb6c826bee7755232de62efc8eb45b:
kern/misc: Make grub_vsnprintf() C99/POSIX conformant
296d3ec835ed6e3b90d740e497bb534f14fe4b79:
disk/cryptodisk: Fix missing change when updating to use grub_uuidcasecmp()
42a831d7462ec3a114156d56ef8a03e1d47f19e7:
ZFS: support inode type embed into its ID
96446ce14e2d1fe9f5b36ec4ac45a2efd92a40d1:
ZFS: Fix invalid memcmp
444089eec6042250ce3a7184cb09bd8a2ab16808:
ZFS: Don't iterate over null objsets
7ce5b4911005b2a0bfd716d92466b6711844068c:
ZFS: Check bonustype in addition to dnode type
There are more patches than this, but these are the
ones that strike me as interesting for Libreboot.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Leah Rowe