this means the unified /tmp handling is now provided for
in both the former "fetch" and "fetch_trees" script, which
are now (respectively):
./update project repo
./update project trees
if the fetch scripts weren't cleaning /tmp before, they
now are, because lbmk handles it
Signed-off-by: Leah Rowe <leah@libreboot.org>
export TMPDIR to scripts, and handle it in a way that
we know lbmk set it
delete it at the end of the parent process, but not child
processes; when the lbmk script calls itself, child processes
will not delete the tmp directory.
some scripts in lbmk weren't cleaning up the tmpfiles they
made, and they still don't, but this mitigates that.
now in follow-up commits, i can start cleaning up those
scripts too.
not handled by this patch:
if the user cancels lbmk (ctrl+c), the tmp directory will
still be there. this too will be handled, in subsequent
patches
Signed-off-by: Leah Rowe <leah@libreboot.org>
libreboot's build system, lbmk, *is* available to use
in releases aswell (use the _src tarball), but it is
mostly intended for development, in lbmk.git
well, there's not much point wasting time / disk space
generating no-microcode roms within lbmk
they should be generated only at release time, alongside
the default ones
this patch implements that, thus speeding up the build
process and saving disk usage during development
the other alternative was to add a new option in
build/boot/roms, -m, that would opt in to removing them,
but this is extra complexity for something that is ill
advised and only provided to appease certain people
Signed-off-by: Leah Rowe <leah@libreboot.org>
most of these steps do not need to be repeated, per image.
move it into handle/make/config, so that the steps are
performed on files that go under elf/coreboot (this will
save on build time).
the logic for handling 4MB ROM images on sandy/ivy was unused,
and has been removed.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the error messages that it shows are benign, but users
see them and worry that something went wrong
this patch reduces the number of people asking pointless
questions on irc
Signed-off-by: Leah Rowe <leah@libreboot.org>
new behaviour:
* grub.cfg and grubtest.cfg no longer inserted to cbfs
* grub.cfg in memdisk instead
* grub.cfg in memdisk defers to cbfs/grub.cfg if added
(not added by default, anymore)
* does not defer to grubtest.cfg even if available
* only shows link to grubtest.cfg if available,
as a menuentry item
keymaps:
if /keymap.gkb exists in cbfs, it uses that by default,
but by default this isn't added. instead, it looks for
a file named keymap.cfg and sources that, which then
sets the keymap to one that is located under memdisk.
this file is inserted for each rom, per layout.
if keymap.gkb and keymap.cfg both absent, grub.cfg in
memdisk shall defer to usqwerty as the default keymap
grub_scan_disk: grub.cfg looks for cbfs file "scan.cfg"
and sources that if found, which will be inserted with
the string: set grub_scandisk=setting_goes_here (based
on target.cfg, generated by build/boot/roms automatically).
If no scan.cfg is found, it defaults to "both"
The "background.png" file remains unchanged, and present in
CBFS, used by grub.cfg if present (and it is, by default)
This change actually *saves* space in CBFS, due to compression,
and means that the grub.cfg is now compressed heavily. This
is also safer, because now the user overrides grub.cfg by
adding it, and they can still add grubtest.cfg for testing
first. If they accidentally delete both configs from cbfs,
Libreboot will fall back to the one in memdisk which would
presumably not be deleted.
This also means that lbmk can now more easily be used by
other build systems, that just want the GRUB part to re-use
in their own project. For example, people who want to build
custom coreboot images without using Libreboot's build system.
This change also *speeds* up the build process considerably,
on the parts where ROM images are copied. It's less than half
a second now, whereas previously it took about 30-45 seconds
for ROM images to copy, because of grub.elf being re-added in
each ROM via cbfstool, where compression is used; I believe
the compression part is what caused slowness.
Much, much faster, more versatile builds.
Signed-off-by: Leah Rowe <leah@libreboot.org>
After that, do not allow anything to run if the user is
root. This logic flow is more robust, and reduces the
chance of bugs in the future.
We must not permit the user to run lbmk as root.
Running it as root *is* possible, by just removing
the check, and wily enough users will do that, but
this behaviour in lbmk is good practise because it
prevents accidentally running as root. If the user
went into root just for installing dependencies, they
might accidentally forget to switch back. This is a
safeguard against such folly.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this was an oversight, in a previous commit.
there was a space, between variable name and
the equals sign, and then another space, so it
was trying to *execute* the rom
Signed-off-by: Leah Rowe <leah@libreboot.org>
this was an oversight on my part. the script cannot be
run as root, except to install distro dependencies e.g.:
as root: ./build dependencies debian
however, ./checkgit was being run *before* checking that,
making it required to set git config as root.
this patch fixes that bug.
Signed-off-by: Leah Rowe <leah@libreboot.org>
lbmk's new style is inspired by the bsd coding styles:
top-down logic, main simplified to a skeleton showing
overall program structure, variables well-defined,
rigorous (yet deceptively simple) error checking.
this was attempted before, but caused problems; coreboot
wasn't being cleaned properly, and rather than audit it,
i simply reverted this back to the old style.
this is actually attempt number 5, because i made 3 more
attempts between then and this one. i've build-tested this
using "./build boot roms all" (which is what b0rked on
the first attempt, months ago). it should be stable(tm).
the code is much nicer to read / work on now. this is the
beating heart of lbmk. get this script wrong, and you break
all of libreboot.
Signed-off-by: Leah Rowe <leah@libreboot.org>
With this change, about 54KB of compressed space is saved
inside of CBFS, on setups that use the GRUB payload.
The uncompressed saving is about 720KB, but payloads are
compressed inside each coreboot image, so the compressed
saving is much smaller. That 54KB saving means a lot,
especially on small (1MB or smaller) flash sizes.
The following modules were removed:
adler32, afsplitter, aout, archelp, backtrace, blocklist,
bswap_test, cat, cmdline_cat_test, cmosdump, cmostest, cmp,
cmp_test, cpuid, cs5536, ctz_test, date, datehook, datetime,
disk, diskfilter, div, div_test, dm_nv, efiemu, eval,
exfctest, extcmd, file, fshelp, functional_test, gdb,
gettext, gptsync, hashsum, hdparm, hello, hfspluscomp, http,
json, json, ldm, loadenv, macbless, macho, mda_text, morse,
mpi, msdospart, mul_test, net, ntfscomp, offsetio,
part_acorn, part_amiga, part_apple, part_dvh, part_plan,
part_sun, part_sunpc, parttool, pbkdf2, pbkdf2_test, pci,
play, priority_queue, probe, progress, random, rdmsr, read,
relocator, setjmp, setjmp_test, shift_test, signature_test,
sleep, sleep_test, smbios, strtoull_test, terminal,
terminfo, test_blockarg, testload, testspeed, tftp, tga,
time, tr, trig, usbtest, video_bochs, video_cirrus,
videoinfo, videotest, videotest_checksum, wrmsr, xnu_uuid,
xnu_uuid_test
These were retained, but moved to modules instead of
install modules:
geli, udf, ufs1, ufs1_be, ufs2
Signed-off-by: Leah Rowe <leah@libreboot.org>
the way the old script worked was extremely hacky
it's cleaner just to make the user configure git
i haven't used anything from the old .gitcheck script,
which is now deleted. i completely re-wrote this, in
a much simpler way.
this is less maintenance now, when things change in
the upstream projects. coreboot makes heavy use of git
within its build system
Signed-off-by: Leah Rowe <leah@libreboot.org>
and only where grub was already enabled; on boards
that did not enable grub, grub is still disabled
on desktops, it's possible that the user may insert
a graphics card. if their first payload was grub,
it won't work because lbmk doesn't configure coreboot
itself to execute vga roms at present
i found when testing t1650 (dell) that if a vgarom is
loaded from seabios (from a graphics card), the grub
payload still works; if booting in corebootfb mode,
text mode is still used when booting with the card
to decrease the probability of bricks with any given
set of users, make seabios the only payload that starts
first, but make grub available in the esc menu on seabios
it's possible to add a bootorder file and disable the
seabios menu, if you only want a grub payload accessible
Signed-off-by: Leah Rowe <leah@libreboot.org>
e.g. ./build boot roms list
./update blobs inject listboards
./build boot list
./build clean list
also this is now possible:
./build list
or maybe
./update list
^ would list directories in resources/scripts/build
and resources/scripts/update respectively
this script is added:
resources/scripts/build/command/options
call it like so, e.g.
./build command options resources/coreboot
this script is now used, for list functions in
other scripts.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we don't use it in lbmk. it's there mostly because
it was technically feasible, and it still is
however, i've been doing massive re-factoring of
lbmk and the makefile and i just don't feel like
constantly patching up the makefile
if someone wants to re-add it, that's fine. but i
don't see the point in maintaining something that
we don't need.
the makefile is not needed. all it did was call
lbmk directly. the makefile had no logic itself.
Signed-off-by: Leah Rowe <leah@libreboot.org>
lbmk is much more likely to crash now, in error conditions,
which is a boon for further auditing.
also: in "fetch", remove the downloaded program
if fail() was called.
this would also be done for gnulib, when downloading
grub, but done in such a way that gnulib goes first.
where calls to err write "ERROR" in the string, they
no longer say "ERROR" because the "err" function itself
now does that automatically.
also: listmodes/listoptions (in "lbmk") now reports an
error if no scripts and/or directories are found.
also: where a warning is given, but not an error, i've
gone through in some places and redirected the output
to stderr, not stdout
as part of error checks: running anything as root, except
for the "./build dependencies *" commands, is no longer
permitted and lbmk will throw an error
mrc downloads: debugfs output no longer redirected to /dev/null,
and stderr no longer redirected to stdout. everything is verbose.
certain non-error states are also more verbose. for example,
patch_rom in blobs/inject will now state when injection succeeds
certain actual errors(bugs) were fixed:
for example, build/release/roms now correctly prepares the blobs
hash files for a given target, containing only the files and
checksums in the list. Previously, a printf message was included.
Now, with this new code: blobutil/inject rightly verifies hashes.
doing all of this in one giant patch is cleaner
than 100 patches changing each file. even this is yet part
of a much larger audit going on in the Libreboot project.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the user may have re-downloaded a coreboot tree,
in a release. this is supported. therefore, some
may have .git, and some will not
Signed-off-by: Leah Rowe <leah@libreboot.org>
we also run it in releases, so to compensate:
it now checks for .git/, but only in project
directories, not the main lbmk directory of
the git repository or a release.
this is because in a release, it's possible
that the user may still delete coreboot/
directories and re-download coreboot trees
this is not intended, but we must not assume
that users use libreboot the way it's intended!
"much stricter" because there was previously
none, intentionally, due to the above fact. the
checking of .git/ should mitigate this (the
script will exit with zero status if it isn't
there)
Signed-off-by: Leah Rowe <leah@libreboot.org>
there were stragglers left over from the last audit,
and these stragglers still exist even after all the
major re-factoring as of late
the new style is: bsd-like coding style and error
handling. verbose yet simple error handling. we use
an "err" function in a way reminiscent of most C
programs that you see in openbsd base (err.h)
this style is very clean, resulting in readable code
Signed-off-by: Leah Rowe <leah@libreboot.org>
this same change has been applied, selectively, to
certain return statements. the general rule is this:
the return statement should only be used to direct
logic within a script, where certain non-errors
states are used to skip certain actions; the exit
command should *never* be used to return non-zero,
except by err(). in so doing, we ensure easier
debugging of the build system
also: strip_rom_image in build/release/roms was
running "continue" when a rom file didn't exist,
despite not being a while/for loop. i make it
return (non-error condition) instead
it's ok for a script to exit 0, where appropriate,
but perhaps a function could also be written for it
Signed-off-by: Leah Rowe <leah@libreboot.org>
moving it defeats the purpose of the caching mechanism
that's in place. this should avoid unnecessary downloads
Signed-off-by: Leah Rowe <leah@libreboot.org>
u-boot and seabios are now handled by the same logic
as coreboot, in lbmk, and these files are used for
recursive downloads in the build system
Signed-off-by: Leah Rowe <leah@libreboot.org>
uses 32-bit variant for x86_32 arch. 64-bit for x86_64.
resources/scripts/build/src/for:
modified it a bit. when building e.g. "memtest86plus/build32"
it correctly fetches "memtest86plus" instead.
but builds memtest86plus/build32, which is inside that git repo
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
in update/blobs/download, i saw instances where
appdir was being deleted with rm -r, but the more
appropriate command would rm -Rf. this is now fixed.
other than that, i've mostly just simplified a bunch
of if statements and consolidated some duplicated
logic (e.g. if/else block for dependencies in
build_dependencies() of update/blobs/download
one or two functions and/or variables have been
renamed, for greater clarity in the code, also
removed a few messages that were redundant
used printf instead of echo, in a few places, also
fixed up the indentation in a few places
Signed-off-by: Leah Rowe <leah@libreboot.org>
Leah Rowe