Commit Graph

44 Commits (029291e5495bf679cb36a7500b73ff9fa1dfff35)

Author SHA1 Message Date
Leah Rowe 029291e549 merge script/vendor/* into include/vendor.sh
stub it from the main build script

the commands remain identical:
./vendor download arguments_here
./vendor inject arguments_here

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11 05:57:49 +01:00
Leah Rowe c8fb24bb7b build: print usage for special commands
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11 05:19:48 +01:00
Leah Rowe 5f63b594fa merge script/update/release into build
the main script isn't that big, and since the main
purpose of lbmk is geared toward the releases, it
makes sense to reduce the number of scripts by
merging into the main one

the way this works, "./update release" still works
afterward

so, the way lbmk is used shall remain unchanged

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11 04:15:30 +01:00
Leah Rowe 052414c0ca build: further prevent non-lbmk-work-directory
this is a follow-up to the previous commit

again, there's no posix way to check the path to the
file at argument 0, because readlink (utility) isn't
defined in posix (the C function is defined, but not
the utility included on many unices)

check whether "build" (file) exists, and whether it
is a symlink; if the latter, then we are definitely
not in the lbmk work directory!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-09 14:48:14 +01:00
Leah Rowe fb8d0c86c4 build: exit if not running from lbmk directory
there's no portable(posix) way to check when running
from a symlink to lbmk in the current work directory

for example:

ln -s lbmk/build lbmktest
./lbmktest roms list

this would pass the new test, and first try to
include option.sh. in practise, the user probably  doesn't
happen to have include/option.sh in their current path

i can use readlink here, but again not portable

the current check will suffice. it also works when
the symlink is called from $PATH

e.g. /usr/bin/lbmktest exists and you do:
lbmktest roms list

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-09 14:31:43 +01:00
Leah Rowe 5e4009b539 merge include/err.sh with include/option.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-06 22:54:55 +01:00
Leah Rowe 64177dbb8e exports variables from err.sh, not build
LC_COLLATE and LBMK_RELEASE are important variables. we want
to make sure that these are seen by everything.

since err.sh is included from all scripts, doing it there will
accomplish just that.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-26 08:20:19 +01:00
Leah Rowe 08859bb4a5 lbmk: export TMPDIR from err.sh, not build
lbmk sets TMPDIR to /tmp, and then creates a tmpdir, then
exports *that* as the value of TMPDIR. this unified TMPDIR
location then contains all subsequent files and directories,
when any script or program makes use of /tmp, via mktemp. at
least, that's the theory!

in practise, because it was only being properly exported from
the main build scripts, subscripts that are then called were
not exporting it, at least that is my assumption because in
some cases, i found that the coreboot build system was leaving
errant files behind outside of our own TMPDIR, and that build
system did not seem to be setting TMPDIR itself; more debugging
is needed.

anyway: use the exact same logic, but do it from err.sh. since
err.sh is included from every lbmk script, that means it will
always be exported when running every single part of lbmk. this
should reduce the chance that mktemp creates files and directories
outside of our custom TMPDIR location.

this is because in lbmk, we mitigate unhandled tmpdirs/files by
unifying it in the manner described, then deleting the entire
TMPDIR on exit from the main lbmk parent process (the main
script that the user called from, which is always the "build"
file).

in lbmk, effort is made to clean up temporary files properly,
without relying on this catch-all, but we can't rely on that.
the catch-all should also be as robust as possible.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-25 19:08:53 +01:00
Leah Rowe ed0678ae2e haswell: only provide NRI-based ROMs in releases
release="n" is set in target.cfg on haswell build targets
that use mrc.bin

script/update/release exports LBMK_RELEASE="y"

script/build/roms skips building a given target if release="n"
in target.cfg *and* LBMK_RELEASE="y"

you could also do the export yourself before running ./build roms,
for example:

export LBMK_RELEASE="y"
./build roms all

This would skip these ROM images. The native haswell raminit is
now stable enough in my testing, that I wish to delete the MRC-based
targets. This is in line with Libreboot's Binary Blob Reduction Policy,
which states: if a blob can be avoided, it should be avoided.

The problem is that users often run the inject script in *lbmk* from
Git, instead of from the src release archive. I forsee some users
running this on modern lbmk with older release images. If the mrc-based
target isn't there, the user may use an NRI-based target name, and
think it works; they will insert without MRC. I foresaw this ages
ago, which is why Caleb and I ensured that the script checks hashes,
and hashes are included in releases.

Therefore: for the time being, keep the MRC-based configs in lbmk
but do not include images for them in releases. This can be done
indefinitely, but I'll probably remove those configs entirely at
some point.

On the following boards, Libreboot now will *only* provide NRI-based
ROM images for the following machines:

* Dell OptiPlex 9020 SFF
* Dell OptiPlex 9020 MT
* Lenovo ThinkPad T440p
* Lenovo ThinkPad W541/W540

I now recommend exclusive use of NRI-based images, on Haswell
hardware. It's stable enough in my testing, and now supports S3.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-24 05:27:27 +01:00
Leah Rowe 6ebab10caa safer, simpler error handling in lbmk
in shell scripts, a function named the same as a program included in
the $PATH will override that program. for example, you could make a
function called ls() and this would override the standand "ls".

in lbmk, a part of it was first trying to run the "fail" command,
deferring to "err", because some scripts call fail() which does
some minor cleanup before calling err.

in most cases, fail() is not defined, and it's possible that the user
could have a program called "fail" in their $PATH, the behaviour of
which we could not determine, and it could have disastrous effects.

lbmk error handling has been re-engineered in such a way that the
err function is defined in a variable, which defaults to err_ which
calls err_, so defined under include/err.sh.

in functions that require cleanup prior to error handling, a fail()
function is still defined, and err is overridden, thus:

err="fail"

this change has made xx_() obsolete, so now only x_ is used. the x_
function is a wrapper that can be used to run a command and exit with
non-zero status (from lbmk) if the command fails. the xx_ command
did the same thing, but called fail() which would have called err();
now everything is $err

example:

	rm -f "$filename" || err "could not delete file"

this would now be:

	rm -f "$filename" || $err "could not delete file"

overriding of err= must be done *after* including err.sh. for
example:

err="fail"
. "include/err.sh"

^ this is wrong. instead, one must do:

. "include/err.sh"
err="fail"

this is because err is set as a global variable under err.sh

the new error handling is much cleaner, and safer. it also reduces
the chance of mistakes such as: calling err when you meant to
call fail. this is because the standard way is now to call $err,
so you set err="fail" at the top of the script and all is well.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-03-27 01:50:31 +00:00
Leah Rowe 0aca6332ee lbmk scripts: shorter code lines
while seemingly pedantic, this does actually make code
easier to read. mostly just switching to shorthand for
variable names, where no expansions or patterns are used

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-12-24 09:04:36 +00:00
Leah Rowe 746d9cadda build: remove test command
i left this in here during the last change

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-12-23 18:02:09 +00:00
Leah Rowe 655d3cdc88 lbmk scripts: general code cleanup/optimisation
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-12-23 17:51:32 +00:00
Leah Rowe 9d5d98ebae set version/projectname properly
lbmk used to set version/versiondate directly in
err.sh, but now it's handled there by a function,
which is called by the main script.

script/update/release hadn't yet been adapted. the
only change necessary is to call check_project()

script/update/trees also makes use of it

script/build/roms is using "projectname"

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-12-23 08:54:45 +00:00
Leah Rowe 2f98ca6dab build: simplified TMPDIR handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-12-22 10:50:17 +00:00
Leah Rowe 124b5bebd6 build initialise_command: simplify handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-12-19 02:52:46 +00:00
Leah Rowe 39a3de574a remove DEBUG handling in lbmk (not needed)
all it did was set -v in the shell, which doesn't yield
very useful results. this is a relic of very old design
in the libreboot build system, that is no longer needed.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-12-16 07:58:13 +00:00
Leah Rowe ce1176f521 fix typo in help text
say whot?

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-11-09 16:40:36 +00:00
Leah Rowe 70882902b5 build: set --author when running git init
set it to a generic name:
lbmk <lbmk@libreboot.org>

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-11-09 16:34:11 +00:00
Leah Rowe 64f9337470 lbmk: support showing the revision in help text
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-11-08 06:21:25 +00:00
Leah Rowe f4b2a588e2 build: don't generate version/versiondate as root
don't run it directly at the bottom of err.sh,
because otherwise the version and versiondate
files will be generated when running "./build
dependencies distroname" which would then create
these files, but as root because the user runs
that specific command as root.

the rest of lbmk, for any other command, prevents
use of the root account, so running check_project
during "./build dependencies distroname" will cause
the build system to fail (because as non-root user,
the user will run lbmk and it will try to update
those files, and fail because it can't, due to lack
of permissions)

this patch fixes the issue, by only generating those
files if the user is *not* root

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-11-08 06:11:24 +00:00
Leah Rowe dfc5423cad export LC_COLLATE=C and LC_ALL=C
this is to ensure alphanumeric sorting, with
capital letters first; and numbers before letters.

we always relied on this, but until now lbmk would
just assume the host is configured this way.

this fixes a longstanding design flaw in lbmk.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-11-01 08:02:59 +00:00
Leah Rowe e90bffff3c move git_init to the main build script
also, don't use x_ because it totally b0rks on
these commands. handle exit status directly.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-20 21:10:26 +01:00
Leah Rowe 54a05fc167 always re-generate .git in lbmk
in release archives, .git is excluded but the version
and versiondate files are included. from these, the
git history is re-created with the exact date (but not
taking into account timezone, at present).

in this way, lbmk will have git history in a release
archive. some build systems, like coreboot, prefer that
there be git history available, so this is a nice
workaround on those build systems.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-20 11:16:25 +01:00
Leah Rowe 8d9aeef3de lbmk: use 2-level directory structure in script/
as opposed to the current 3-level structure.

recent build system simplifications have enabled
this change, thus:

./build fw coreboot -> ./build roms
./build fw grub -> ./build grub
./build fw serprog -> ./build serprog
./update project release -> ./update release
./update project trees -> ./update trees
./update vendor download -> ./vendor download
./update vendor inject -> ./vendor inject

alper criticised that the commands were too long,
so i made them shorter!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-20 01:00:38 +01:00
Leah Rowe 0b98c9b00c minor code cleanup in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-19 23:36:56 +01:00
Leah Rowe 96fd88c5b3 build: fix bad command in help text
lbmk was massively re-written, very recently.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-10 23:52:42 +01:00
Leah Rowe fa8e204f14 unified projectname/version/versiondate handling
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-07 02:46:47 +01:00
Leah Rowe 24584296a8 put include/export.sh in build script
remove include/export.sh

it's not a lot of code, and build is the only
file that uses it

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-07 02:10:14 +01:00
Leah Rowe 62cc895c3c rename blob/ to vendor/
in the future, we may start downloading files that aren't
blobs, such as mxm port configs (on mainboards that use
MXM graphics)

this directory will contain all of those files

generally change the language used, across lbmk, to make
use of "vendorfile" instead of "blob"

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-07 01:23:11 +01:00
Leah Rowe 4e39d5a5a8 put all src downloads under src/
build/release/src was partly re-written to accomodate this

memtest86plus was patched to have a central Makefile, and
lbmk modified to use that, rather than mess with build32
and build64. the central Makefile just builds both targets
or cleans both targets

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-07 00:11:21 +01:00
Leah Rowe 965b6a7ed7 rename build/firmware/ to build/fw/
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-06 03:32:38 +01:00
Leah Rowe 315d0c4572 mv build/fw/serprog,build/boot/roms build/firmware
./build boot roms is now: ./build firmware coreboot

./build fw serprog is now: ./build firmware serprog

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-06 02:26:18 +01:00
Leah Rowe 863081c369 remove build symlink, rename lbmk to build
re-link update to build

build/update are the only two build modes now

i'm on a crusade to reduce the number
of files and directories, and reduce the number
of source lines, while not reducing functionality

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-10-06 02:12:52 +01:00
Leah Rowe 092600d163 unify these scripts: build, modify and update
unify them, by turning them into symlinks pointing
to a generic script named lbmk

the script named lbmk is a fork of the script
named "build", which just checks argument 0 and adapts
accordingly

all of these core scripts had the exact same overall
logic, and they are thus compatible

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-05-27 10:54:50 +01:00
Leah Rowe a4ea286731 Remove most of Ferass's lbmk contributions
The primary purpose of my intense auditing has
been to improve lbmk's coding style and fix bugs
but there is a secondary purpose: know precisely
who owns what, because I want to re-license as
much as possible of lbmk under *MIT*, instead of
the current GNU licensing. MIT is vastly superior,
because it grants *actual* freedom to the user,
permits *sublicensing* and it is vastly more
compatible with other GPL combinations; for
example, MIT license is compatible with GPL2-only
whereas lbmk's current mix of GPLv3-or-later and
GPLv3-only is legally incompatible with GPLv2-only.

Re-licensing under MIT will most likely result in
more contributions to Libreboot's build system in
the future, especially as it will attract a lot
more commercial interest. Contrary to the popular
arguments, copyleft is a liability to the free
software movement and results in less code being
written; in practise, permissively licensed code
gets more public contributions, including from
commercial entities, even if companies can
theoretically make something proprietary out of
it (in practise, anyone inclined can just use the
upstream and proprietary forks almost always die).

Copyleft propaganda is fundamentally flawed. See:
<https://unixsheikh.com/articles/the-problems-with-the-gpl.html>

Anyway, I've been doing a combination of:

* Seeking permission from other copyright holders,
  for re-licensing
* Deleting, or moving, other contributions; for
  example, splitting certain contributions into
  separate files so that originally modified files
  become unencumbered. This latter solution is a
  result of *code cleanup* arising from the audit.

For Ferass's contributions, I opted to seek
*permission*, and permission was denied. In full compliance
with this legal imperative, I'm acting accordingly; this
commit removes all of Ferass's changes that converted lbmk
to posix shell scripts, thus removing his copyright on the
affected files, bypassing his authority entirely. Therefore,
lbmk is largely now bash-dependent. In practise, nobody is
going to use anything other than a GNU system to build
Libreboot, because many projects that Libreboot makes use
of rely heavily on GNU; for example, coreboot's build
system makes heavy use of GNU-specific extensions in *GNU
Make*, and likely contains many bashisms. Of course,
Libreboot also compiles GNU GRUB.

I would much rather have MIT-licensed Bash scripts
than GPL-licensed posix SCL scripts.

This reverts the changes from Ferass El Hafidi,
for the following commits, with some exceptions:

* 7f5dfebf7d
* f787044642

Exception:

download/mrc not reverted, because that was
already a fork of an existing script under
coreboot's build system, and their script was
GPLv2. i cannot/will not re-license this file
(ergo,
7f5dfebf7d
change remains intact, on this file)

resources/scripts/build/boot/roms_helper, these changes
have been kept:
* 7e6691e9 - Add ARMv7 and AArch64 support
* dec2d720 - add myself in the build/roms_helper script
	(added 2021 copyright for the change below)
* b7405656 - Workaround for grub's slow boot
^ these changes will be re-factored, splitting them
  out of the file into a new file. This will be done in
  a future lbmk revision. (in some cases, it makes sense
  to keep a change but split it, allowing the main file to
  be re-licensed without the change in it)

This is part of a much larger series of
licensing audits. It's likely that lbmk will
be posix-compliant (in its shell scripts)
again some day, because I'm planning to rewrite
most of these scripts (the ones modified in this
patch), and many of them (e.g. individual download
scripts) are subject to future deletion in a planned
overhaul of the download logic for third party
projects.

In addition: these changes are being kept (no attempt
to re-license them will be made):

* cff081c6 - Fix grub's slow boot (1 year, 5 months ago) <Vitali64>
* 4c851889 - Add macbook*1 16mb configs (1 year, 6 months ago) <Vitali64>

Ferass's work that remains will be split into dedicated
files containing them, where feasible.

In the case of grub.cfg (for GNU GRUB), I don't care
because it's a script for an engine (GRUB shell) that's
under GPL anyway, so who really cares about MIT license.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-05-27 08:10:50 +01:00
Leah Rowe 01a2ab3756 use env in shell scripts
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-05-18 13:17:28 +01:00
Leah Rowe 34df727c98 build: cleaner coding style
main() on top

top-down logic

reduced indentation

Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-05-18 10:49:47 +01:00
Leah Rowe 1a062bb628 build: reduce code to less than 80 chars per line
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-05-18 10:21:54 +01:00
Ferass 'Vitali64' EL HAFIDI f787044642 Do not rely on bashisms and behaviour undefined by the POSIX specification.
By making lbmk fully POSIX-compliant, it will be easier to port lbmk to
other systems implementing POSIX such as Alpine Linux and FreeBSD.

Signed-off-by: Ferass 'Vitali64' EL HAFIDI <vitali64pmemail@protonmail.com>
2022-12-27 15:50:41 +00:00
shmalebx9 0b3b7f93b7 added workaround for git credentials
this is cherry-picked from osbmk. the cherry-pick was
performed by i, leah rowe. this is adapted from shmalebx's
patch there, in osboot

specifically, these patches from osbmk are being imported:

327a39ef058d5385bf8c1a1b09bac8db6a51b016

5139ad4be4df1835ce154f39161eef4f7c31ee1a
2022-03-16 04:40:59 +00:00
Leah Rowe 67e2365a6d also change the build and update scripts to env bash 2021-06-03 12:59:22 +01:00
Leah Rowe 5573dcd103 build: don't run the versioncheck script if running ./build dependencies
The "dependencies" script are to be run as root. Without this fix, root will
create the version/versiondate file and then the rest of the build system will
break due to lack of root privilege.
2021-05-19 17:58:36 +01:00
Leah Rowe 89517ed6b9 libreboot!
this is forked from the "libre" branch in osboot, which is itself a libre,
deblobbed fork of osboot, a blobbed up fork of libreboot

libreboot needed to be purged clean. this is the new libreboot development
repository. the old one has been abandoned
2021-05-18 13:56:12 +01:00