Commit Graph

40 Commits (ed921d78900093d7e734e961001a9ac0d6e5d34a)

Author SHA1 Message Date
Leah Rowe ed921d7890 trees: simplify copy_elf()
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15 23:19:02 +01:00
Leah Rowe 293777fb99 trees: remove unnecessary check in copy_elf
we don't need to check whether this variable is set,
because checking an empty path will also cause the
same return in the next line.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15 23:18:57 +01:00
Leah Rowe b50a588cba grub: insert background in memdisk instead
the background is only a few kb. the whole rationale
before was to limit the space used in memdisk, but this
decision was made when the background was much bigger;
it has since been optimised greatly, and the grub modules
were heavily reduce, so it should be safe.

grub's memdisk breaks when you add too much data to it.
as part of simplifying the rest of lbmk, this change removes
some more bloat from the rest of lbmk. handling this in the
memdisk is much simpler than handling it with cbfstool.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15 23:18:51 +01:00
Leah Rowe bb7b04cb59 unified checks for variable initialisation
new function chkvars() does the job

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15 23:17:27 +01:00
Leah Rowe 1399f2137f lib.sh: remove the items() function
it's pretty much just doing the same thing as ls -1

remove it!

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15 23:17:06 +01:00
Leah Rowe 070aee6728 re-add ability to use cbfs grub.cfg as default
i removed this before, when making grub multi-tree,
because the design i used in an earlier version of
the patch actually added the grub.elf generation
to grub source itself, but then i decided to hack
around the grub build system from lbmk/cbmk instead

re-add this functionality, so that users can easily
insert their own custom grub.cfg into cbfs without
needing to re-build their image.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09 16:35:12 +01:00
Leah Rowe b4acd0f73c trees: exit with error if project undefined
i was originally looser about this, because i also wanted
the trees script to generically run "make" from any
directory, but this behaviour was error-prone and it is
no longer used in the build system.

disable it, in the interest of stability.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09 16:35:07 +01:00
Leah Rowe 62b2310a28 add crossgcc tarballs to config/submodules/
support redundant downloads, and enable inclusion of these
tarballs inside release archives, for offline builds.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09 07:38:03 +01:00
Leah Rowe 573199c07d trees: simplified copy_elf() handling
don't create elfdir, create dest_dir, which is elfdir
plus the location within it

only create dest_dir within copy_elf, which is only
called if actually compiling the code

this avoids creating empty elf directories, and it
generally cleans up all handling, unifying the
handling of directories into a single function,
namely copy_elf() which already exists

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:41:25 +01:00
Leah Rowe 591c7d28e0 git.sh: download "depend" projects *before*
don't do it after, because that means the main project
is saved under src/ before we know whether the subrepo
was downloaded.

the "depend" variable (in config/git/) is no longer used
for projects that go in subdirectories of a parent; now,
we use config/submodules/ for this type of dependency.

download the "depend" projects (as per config/git/) first.
this way, if they fail, the main one will fail, but if
they succeed and main fails, you can just run the main
download again and it won't fail.

this fixes a bug where, depending on how you download a
set of projects and depending on the order which you do so,
a given project can become un-downloadable on current design,
because git will complain that a directory already exists.

this fix is done not only in code (by this commit), but
by prior configuration changes.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:41:12 +01:00
Leah Rowe 8d5edd4f06 trees: don't check empty path in copy_elf()
it could lead to some whackiness later on

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:39:20 +01:00
Leah Rowe c1176bbd28 trees: fix build issue caused by bad elf check
we're not checking for bad elfs, but the check itself was bad

due to a quirk in how sh works. really, really obscure bug.

fixed now!

if the given directory didn't actually exist, or there were no
files in it, it'd be searching for the file named "*"

which is obviously wrong

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:39:15 +01:00
Leah Rowe c88fb8c129 trees: fix listfile check in copy_elf()
don't check that the variable is empty

check that the file itself exists or not

this should fix the recent build issues

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:39:10 +01:00
Leah Rowe 9168d33741 trees: don't say check elf/ if build.list missing
(on single tree projects. this complements the last patch)

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:39:04 +01:00
Leah Rowe db09530905 trees: don't do elfcheck if build.list missing
otherwise, some checks are done too soon, and nothing
gets built.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:38:58 +01:00
Leah Rowe e4eb82e089 trees: unified coreboot makeargs
in particular, the coreboot build system may auto-download
submodules when building cbfstool; vboot for instance.

we do not want such unpredictable behaviour, so now we
use UPDATED_SUBMODULES=1 when building coreboot utilities.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:33:25 +01:00
Leah Rowe f7170092c8 trees: use multiple threads to build cbutils
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:33:20 +01:00
Leah Rowe 1d7a6f04c9 move handle_coreboot_utils to script/trees
it doesn't really make sense placed in lib.sh,
because it's only called from script/trees

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:33:14 +01:00
Leah Rowe a30bfd334f trees: skip single-tree build if a build exists
this replicates the same behaviour as multi-tree builds,
checking for files inside the relevant elf/ directory

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:31:11 +01:00
Leah Rowe 7fe0106fa0 trees: also print "DONE! check elf/dir" on single
e.g. ./update trees -b flashprog

tell the user to check elf/flashprog

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:30:51 +01:00
Leah Rowe 74759d876a trees: handle build-test on multi-tree projects
for example, now flashprog binaries could be placed
in the elf/ directory, under elf/flashprog/

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:30:40 +01:00
Leah Rowe cb446e7d24 move cfgsdir/datadir variables to lib.sh
it's also used from script/roms, in addition to trees

move these variables to a common file used everywhere

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:28:16 +01:00
Leah Rowe 7d99786a1a handle build.list from config/data/, not config/
certain code checks for build.list, to skip it, for
example in items()

we already use config/data/grub to store grub config data
that applied to all trees

create these directories too:

config/data/coreboot
config/data/u-boot
config/data/seabios

move the respective build.list files in here, and also
to config/data/grub

now multi-tree projects contain, per directory, just the
target.cfg file and the patches directory. this is much
cleaner, because some of the logic can be simplified more

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:27:45 +01:00
Leah Rowe a61794dfca don't use build.list to detect multi-tree projects
instead, check for the presence of target.cfg files
not in config/project/ but config/project/tree/

the way this check is done, it merely returns 1 if
config/project/*/target.cfg is detected, and returns
0 in all other cases, even if config/project/target.cfg
exists

that way, if the maintainer accidentally adds a
target.cfg in the main directory, the given multi-tree
project will not break

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07 17:27:40 +01:00
Leah Rowe 1d866d17d8 better help text on invalid commands
adding help again is a bad idea. code should never
document itself; that's what documentation is for.

so, make the code do a better job telling the user
where to find documentation.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-03 11:09:39 +01:00
Leah Rowe eb4ac3c334 make GRUB multi-tree and re-add xhci patches
The xHCI patches were removed because they caused issues
on Sandybridge-based Dell Latitude laptops. See:
https://codeberg.org/libreboot/lbmk/issues/216

The issue was not reported elsewhere, but we still don't
need xHCI support in Canoeboot's GRUB because none of the
available coreboot targets have xHCI support. However, we
may want it in the future and it helps to keep Canoeboot
in sync with Libreboot (this patch is adapted from lbmk).

Each given coreboot target can say which GRUB tree to use
by setting this in target.cfg:

grubtree="xhci"

In the above example, the "xhci" tree would be used. Some
generic GRUB config has been moved to config/data/grub/
and config/grub/ now looks like config/coreboot/ - also,
the grub.cfg file (named "payload" in each tree) is copied
to the GRUB source tree as ".config", then added to GRUB's
memdisk in the same way, as grub.cfg.

Several other design changes had to be made because of this:

* grub.cfg in memdisk no longer automatically jumps to one
  in CBFS, but now shows a menuentry for it if available

* Certain commands in script/trees are disabled for GRUB,
  such as *config make commands.

* gnulib is now defined in config/submodule/grub/, instead
  of config/git/grub - and this mitigates an existing bug
  where downloading gnulib first would make grub no longer
  possible to download in lbmk.

There is another reason for merging this design change from
lbmk, and that reasoning also applies to lbmk. Specifically:

This change will enable per-board GRUB optimisation in the
future. For example, we hardcode what partitions and LVMs
GRUB scans because * is slow on ICH7-based machines, due
to GRUB's design. On other machines, * is reasonably fast,
for automatically enumerating the list of devices for boot.

Use of * (and other wildcards) could enable our GRUB payload
to automatically boot more distros, with minimal fuss. This
can be done at a later date, in subsequent revisions.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-02 22:41:46 +01:00
Leah Rowe 2887b77ae4 trees: use CPUS=x on regular coreboot make
i already do this on crossgcc, but overlooked it on regular
builds where i just use -j, but coreboot's build system
makes use of the CPUS= option in make

use XBMK_THREADS for this

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-29 10:25:50 +01:00
Leah Rowe d147c5d915 rename include/option.sh to include/lib.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-26 12:11:40 +01:00
Leah Rowe f534b0e973 merge nuke() back into git.sh
as stated in the previous commit, i'm adding this function
to lbmk because there are files i want to systematically
delete in libreboot releases, not just canoeboot releases,
but libreboot releases delete things such as unlicensed
readme files, or poorly licensed other files.

i initially moved the nuke function to its own file so as
to reduce the number of merge conflicts when merging
changes to git.sh between cbmk and lbmk, but if they're
going to both contain this file, then it makes sense
to have this in git.sh once again.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-26 09:37:28 +01:00
Leah Rowe a02b152f44 rename nukeblobs to a more generic name
blobs.list is now nuke.list

this is because i'm going to import this functionality
into lbmk (libreboot build system).

libreboot will not do full deblobbing like canoeboot does,
but there are still certain files that i like to delete
in releases, such as u-boot's strlcat.c file under tests

calling it "nukeblobs" in libreboot makes no sense, but
i like to avoid merge conflicts when cherry-picking
patches between cbmk and lbmk, so i like to make sure
that functions and variables common to both are named
the name.

simply calling it "nuke" or calling the files "nuke.list"
is probably inoffensive while conveying the same meaning.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-26 09:33:27 +01:00
Leah Rowe cbb2f4f8a9 general code cleanup in the build system
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-26 08:14:23 +01:00
Leah Rowe 3554593fd8 trees: reset makeargs per target/project
it wasn't being reset before. when coreboot is being
built, i add to makeargs every time. if multiple targets
are being built, the make command would end up looking
something like:

make -C src/coreboot/default UPDATED_SUBMODULES=1 \
    UPDATED_SUBMODULES=1

(the parameter would be printed twice)

of course, this doesn't check whether that parameter is
added already in target.cfg for a given target, but that's
ok because i won't add that one in target.cfg

i baked it into the code, only when handling coreboot,
because that was easier than either putting it in makeargs
for every coreboot target.cfg, or again modifying the code to
handle that; the current solution is the cleanest.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25 14:39:22 +01:00
Leah Rowe b09261a901 trees: also use UPDATED_SUBMODULES=1 on crossgcc
i overlooked this in the previous revision

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25 11:51:02 +01:00
Leah Rowe 698548ac59 trees: add UPDATED_SUBMODULES to coreboot make
we do not want submodules to be downloaded after the fact.

we only handle this on ./update trees -f coreboot

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25 11:50:52 +01:00
Leah Rowe c8c516703f trees: write -C on the make command first not last
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25 11:50:44 +01:00
Leah Rowe 7ec023907b update/trees: remove unused variable
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25 11:37:47 +01:00
Leah Rowe 0d7c249c9b move deblob function to new file "deblob.sh"
i'm importing some changes from lbmk and they go at the
end of git.sh, in the diffs. moving the deblob function
to its own file will allow me to cherry-pick with fewer
merge conflicts.

Signed-off-by: Leah Rowe <info@minifree.org>
2024-05-25 11:35:50 +01:00
Leah Rowe a39c95cfac minor code cleanup in the build system
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19 06:31:46 +01:00
Leah Rowe 0794127986 remove check_project() (always set variables)
in cbmk, we call check_project() to set variables
such as projectname, version, version date

this is unnecessary, because all main scripts use
this functionality anyway

do it by default

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15 07:49:55 +01:00
Leah Rowe 541430016f move script/*/* to script/
there are only two scripts under script/ now, and there
probably won't be many more. cbmk's design has simplified
to such a degree that the two-level directory structure is
no longer necessary.

the existing command structure has not changed. for example:

./build roms list
./update trees -f coreboot default

these will still work, but the symlinks to "build" are now
strictly for backwards compatibility; they may be removed
at a later date, but i'll keep the current design for now.

this also leads to a quirk, for example:

./build roms all
./update roms all

these now do the exact same thing, whereas "./update roms all"
would have previously been an invalid command.

Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-13 01:57:12 +01:00