Commit Graph

1108 Commits (5843daf7a1cdf15cbc76952c60a1d822414b2ef3)

Author SHA1 Message Date
Timo Teräs 209201bc5d add abstraction to adb "walking" - a sax like API to enumerate whole db
This converts 'adbdump' applet to generate adb_walk api callbacks,
and implement gentext backend to generate the yaml-like text output.
2021-05-26 12:47:48 +03:00
Timo Teräs 7167bc9e7b solver: don't consider requirer count for preference
The original intent was to choose packages to which there is most
dependencies. However, since the code has evolved this is has been
mostly obsolete. And in fact now interferes with the provides and
provides priority mechanism. Remove this as obsolete.

Fixes #10742
2021-05-19 16:29:55 +03:00
Timo Teräs f7143c1766 io_archive: add bounds limit for uname and gname tar header fields
Modify apk_resolve_[ug]id to take the user/groupname as a blob, so
proper length checking is done and honored.

==31584== Conditional jump or move depends on uninitialised value(s)
==31584==    at 0x5C8CA5: strlen (strlen.c:17)
==31584==    by 0x432575: APK_BLOB_STR (apk_blob.h:79)
==31584==    by 0x4350EB: apk_resolve_uid (io.c:1112)
==31584==    by 0x43696C: apk_tar_parse (io_archive.c:152)
==31584==    by 0x4271BC: apk_pkg_read (package.c:929)
==31584==    by 0x402D75: add_main (app_add.c:163)
==31584==    by 0x40D5FF: main (apk-static.c:516)

Fixes a potential crash (DoS) on a crafted TAR file. CVE-2021-30139.

Reported-by: Sören Tempel <soeren+git@soeren-tempel.net>
Reviewed-by: Ariadne Conill <ariadne@dereferenced.org>
2021-04-12 15:30:14 +03:00
Timo Teräs 4bcd792194 io: fix fd leak in error handling paths
apk_dir_foreach_file and apk_resolve_[ug]id needs to free the fd in
case fdopen/fdopendir fails. Additionally this does not rely on fdopen
to fail if openat() returned -1, making sure that we don't call any
syscalls with invalid file handle.
2021-04-11 15:06:49 +03:00
Sören Tempel 1b954e4120 Fix segfault in log_internal if prefix is APK_OUT_LOG_ONLY
This commit fixes a regression which was introduced in changeset
646c834492. If apk_out_fmt() is called
while out->log is set and prefix is set to APK_OUT_LOG_ONLY, then
apk_out_fmt() would pass this prefix to log_internal() which would, in
turn, attempt to write it to standard out using fprintf().
Unfortunately, doing so wont work as intended if prefix is ((char*)-1)
(i.e. APK_OUT_LOG_ONLY) and will cause a segmentation fault instead.

This commit fixes this segmentation fault by not printing the prefix in
log_internal() if it is either NULL or APK_OUT_LOG_ONLY.
2021-04-02 12:22:25 +02:00
Drew DeVault 646c834492 Log to /var/log/apk.log
This adds a log file at /var/log/apk.log. On each run, apk's version
information and the current date & time are written to this file,
followed by any normal apk output.
2021-03-19 12:26:15 +00:00
Ariadne Conill 4fe5ac8328 database: do not chroot(".") unless actually necessary
If we use default root (/), then we do not have to chroot to run scripts.
Use APK_NO_CHROOT flag for this scenario to avoid the chroot.  This helps
with using apk with bwrap and OSTree.

Closes #10736.
2021-03-19 10:28:55 +00:00
Timo Teräs 511621438d del: report correctly package's provides names
The code assumed that when package is in world, it would be there
by it's primary name. The code is now updated to properly print the
package names that are actually present in world.

fixes #10718
2021-03-19 12:08:46 +02:00
Timo Teräs 361eb063c6 db: fix control character check to use uint8_t
fixes #10737
2021-02-07 23:43:48 +02:00
Timo Teräs 1a4f2e94dd db: consider control characters in filename as malicious
Especially a newline can produce havoc in the database file as
the filename is written there as-is. This hardenes the extraction
to consider any control character as malicious. Additional
hardening is added to database loading to better detect corrupt
state and return proper error code about it.

Reported-by: Luca Weiss <luca@z3ntu.xyz>
2021-02-04 22:30:26 +02:00
Olliver Schinagl f6656f9d8e io: Handle really long lines
While commit 18b0b45 (io: Handle long lines, Thu Jan 7 17:25:23 2021
+0100) did attempt to address this issue, the buffer really is still to
small when dealing with big-big dependency lists.

Lets make it sufficiently large for now, until the new APKINDEX format
can support multi-line dependencies, making this not needed any more.

[TT: Originally the buffer size was conservative to run on resource
constrained embedded platforms. But since the available memory on those
has also increased much, the adjustment to 128kB makes sense also to
increase performance a little bit. Removing also the iolimit test.]

Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
2021-02-04 21:45:58 +02:00
Timo Teräs b43da45bc3 gunzip: fix false end-of-file condition in rare circumstances
It turns out inflate() can output zero bytes, even if it consumed
data. This had the unfortunate side effect of returning zero bytes
(end-of-file) condition before calling the boundary callbacks. This
fixes the logic to not return zero reads on gzip boundary.

In practice this fixes the seldom seen issues of apk reporting
bad signature (when it was correct).
2021-02-04 21:38:30 +02:00
Rosen Penev c4c8aa5ba0 fix compilation without deprecated OpenSSL APIs
(De)initialization is deprecated under OpenSSL 1.0 and above.

[TT: Some simplifications, and additional edits.]

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2021-01-27 09:13:07 +02:00
thibault.ferrante 7a3e8f4214 database.c: Fixed package DESCRIPTION parsing
Regression introduced by 0fb0d30 which makes parsing a description
a critical failure.

[TT: Minor stylistic change. Fix also missing final line change
from the earlier commit]
2021-01-14 16:19:26 +02:00
thibault.ferrante 18b0b45b5b io: Handle long lines
As an APKINDEX can become arbitrarely long due to dependencies and
other parameters, increasing the buffer size make this less likely
to happens.

Closes #10731.
2021-01-11 11:26:49 +02:00
thibault.ferrante 0fb0d30477 database: Propagate errors when loading an APKINDEX
In case of failure when loading an APKINDEX, no errors are
propagated to the user which may uncorrectly interpret the
current problem.
2021-01-11 11:26:49 +02:00
Alex Denes 9e9a9a2e71 apk_update: use URL_PRINTF 2020-12-29 13:52:29 +02:00
Timo Teräs da8d83338b add missing apk_trust.h 2020-10-09 16:18:23 +03:00
Timo Teräs 705e002bb0 rename adb_trust to apk_trust, and use it as package signature keystore too 2020-10-09 16:09:19 +03:00
Timo Teräs a627ab8eb8 make apk_database optional for applets that don't need it
The new v3 applets don't use it, and eventually all applets will
use the new formats only.

This prepares the code for this, and moves trust, id_cache, and
root_fd to apk_ctx which is enough for v3 applets at this time.

The generic code is made to not initialize apk_database if it's
not needed.
2020-10-09 16:09:19 +03:00
Timo Teräs 354713d2f7 rename apk_db_options to apk_ctx, rework logging
makes apk_verbosity non-global

fixes #10682
2020-10-09 16:09:19 +03:00
Timo Teräs 7a7eca8670 make apk_flags non-global, make progress printing state non-global
ref #10682
2020-10-09 16:09:19 +03:00
Timo Teräs 9afb4e8fd1 make apk_force non-global, remove left-over apk_arch
ref #10682
2020-10-09 16:09:19 +03:00
Timo Teräs b2af872fff db: convert repository list to a string array 2020-10-09 16:09:19 +03:00
Timo Teräs 010497cb5a add separate vertest applet for version string testing
start adding meson based tests
2020-10-09 16:09:19 +03:00
Timo Teräs 6d71f49c11 io: make ostream_file always use tmpname 2020-10-09 16:09:19 +03:00
Timo Teräs 2156107afc minor performance improvements on build and code 2020-10-09 16:09:19 +03:00
Timo Teräs f9f8594069 adb: improve sorting features, sort installed-db package listing 2020-10-09 16:09:19 +03:00
Timo Teräs e9caeff22b build: adopt meson build for v3 branch 2020-10-09 16:09:19 +03:00
Rasmus Thomsen 0cfa6cfb83 build: make sure to use helper scripts from git tree and not system 2020-10-09 16:09:19 +03:00
Rasmus Thomsen 1b168696aa build: use tabs for meson files 2020-10-09 16:09:19 +03:00
Rasmus Thomsen 1bbdc8eb37 build: add option to build apk.static binary 2020-10-09 16:09:19 +03:00
Rasmus Thomsen 8a11ac3952 build: add atom.c to meson 2020-10-09 16:09:19 +03:00
Rasmus Thomsen f6795011ca build: allow building without help in meson 2020-10-09 16:09:19 +03:00
Rasmus Thomsen 718e07f78a build: build help.h with meson 2020-10-09 16:09:19 +03:00
Rasmus Thomsen fe1de720d0 build: add support for building with meson 2020-10-09 16:09:19 +03:00
Timo Teräs efe0c4afec adb: introduce apk-tools database format, and few applets
This is a flat buffers inspired format that allows fast
mmaped access to the data with low overhead, signature support
and relatively good forward support.
2020-10-09 16:09:19 +03:00
Timo Teräs 5f66b618ef various changes to make clang not give warnings 2020-10-07 21:16:35 +03:00
Timo Teräs 3fae0c693f db: make the --repositories-file change more announced
Document the version when changed. And print error with similar note
if the given repositories-file cannot be read.
2020-10-06 14:27:59 +03:00
Paul Spooren dac30d5049 database: automatically create missing cache dir
On some systems the `/var/` dir is mounted in a tmpfs which is reseted
after each reboot. For that reason no post-install script can handle the
creation of the cache dir at `/var/cache/apk`.

Check on database opnening if the folder is available, if not create it.
Fixes #10715

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-10-06 10:02:21 +03:00
Timo Teräs 7c2a3657fb db: make --repositories-file relative to host root
It used to be relative to the --root specified root, but that
causes issues with relative command line filenames and is unintuitive.
Update documentation accordingly. Fixes #10702.
2020-10-05 16:49:57 +03:00
Timo Teräs 6cedfe27ac db: mask password component of printed URLs
fixes #10710
2020-10-05 12:58:46 +03:00
Timo Teräs 8a794021c4 commit: rephrase the error messages more understandable
fixes #10703
2020-10-03 13:58:30 +03:00
Timo Teräs c269e9c24d db: check cache only if some repositories are enabled 2020-10-01 14:31:41 +03:00
Timo Teräs c538509a2a Implement upgrade --prune to remove stale world dependencies 2020-09-28 17:38:07 +03:00
Ariadne Conill fa63e46a75 database: clarify the deprecation notice for checksum-less packages
Closes #10708.
2020-08-29 20:37:57 +03:00
Timo Teräs fb0109fdb1 fix --repository short option to be -X as before
unintentional regression from commit edb45ae464
fixes #10707
2020-08-26 23:05:03 +03:00
Timo Teräs 2408ad4f95 build: make soname explicit and simplify and fix soname link 2020-08-25 15:35:46 +03:00
Timo Teräs e2afc7e7a9 fix, simplify and document upgrade --ignore 2020-08-24 15:26:25 +03:00
Oliver Smith f3cf824948 solver: add more dbg_printfs 2020-08-24 14:25:40 +03:00
Timo Teräs edb45ae464 enforce options definitions to bind the enum and the descriptor
This uses some macro trickery to make sure that there's one-to-one
mapping with the option index enum and the descriptor. The down
side is that enum's are generated via #define's and editors might
not pick them up for auto completion, but the benefits are more:
it's no longer possible have mismatching enum value and descriptor
index, and the amount of source code lines is less.
2020-08-24 13:35:36 +03:00
Ariadne Conill bef1faf1cb upgrade: allow for specified package upgrades
Fixes #10667 and #10700
2020-07-30 11:41:35 +03:00
Ariadne Conill f4c3f0ad2a database: create additional device nodes when initializing root fs
Some triggers and maintainer scripts depend on /dev/random, /dev/urandom,
/dev/zero and /dev/console.
2020-05-27 08:29:01 +03:00
Fredrik Gustafsson 4d69d123f5 fix logic to not compile lua module when it was not asked
fixes 12fdf6fc "allow building without help..."
2020-05-19 17:03:25 +03:00
Timo Teräs d0edeec8fb make the atom functions not use global state
This greatly helps with memory management on applications that
may want to daemonize and open/close database several times.

Also the lifetime and "owner" of memory for all data is now
explicitly bound to owning struct apk_database, which might
be helpful when writing language bindings. As side effect, the
interned "atoms" are unique only within what apk_database, so
comparing packages from different apk_database may not work
as expected.

Fixes #10697
2020-05-19 12:02:56 +03:00
Timo Teräs 12fdf6fc21 allow building without help when lua interpreter is not available
fixes #10696
2020-05-19 10:50:57 +03:00
Ariadne Conill 0732976dbe define APK_DEFAULT_ARCH for riscv32 and riscv64 targets 2020-05-17 00:54:51 -06:00
Timo Teräs 7b76182f39 fix apk_blob_pull_csum to always initialize apk_checksum
Fixes #10686 to not use uninitialized value in the error paths.
2020-05-07 13:28:24 +03:00
TBK 5d796b5678 use SPDX-License-Identifier in source files 2020-05-07 10:45:42 +03:00
Fredrik Gustafsson 6863928677 index: add argument --no-warnings
When creating an index apk warns if a dependency is missing a provider.
However when using a multi-arch repository, it's not an error that
a certain architecture is missing a dependency because that dependency
could be in an other architecture. Since apk index doesn't know about
this, add an argument to disable that warning.

Maintainer note: rebased for new option handling, and minor stylistic
adjustments.

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
2020-05-06 15:49:33 +03:00
Timo Teräs 1d7123d837 rewrite option descriptors to be single string
This reduces the number of relocations on PIE binaries, and also
reduces the executable size. Parsing of the options is slightly
sped up as only the exact matching option group parser is called.
2020-05-06 13:05:25 +03:00
Timo Teräs 5258b484bf add script to autogenerate help from man pages
This creates main help like:
--
usage: apk [<OPTIONS>...] COMMAND [<ARGUMENTS>...]

Package installation and removal:
  add        Add packages to WORLD and commit changes
  del        Remove packages from WORLD and commit changes

System maintenance:
  fix        Check WORLD against the system and ensure consistency
  update     Update repository indexes
  upgrade    Install upgrades available from repositories
  cache      Commands related to the management of an offline package cache

Querying package information:
  info       Give detailed information about packages or repositories
  list       List packages matching a pattern or other criteria
  dot        Generate graphviz graphs
  policy     Show repository policy for packages

Repository maintenance:
  index      Create repository index file from packages
  fetch      Download packages from global repositories to a local directory
  manifest   Show checksums of package contents
  verify     Verify package integrity and signature

Miscellaneous:
  audit      Audit directories for changes
  stats      Show statistics about repositories and installations
  version    Compare package versions or perform tests on version strings

This apk has coffee making abilities.
--

And applet specific help like:
--
usage: apk add [<OPTIONS>...] PACKAGES...

Description:
  apk add adds the requested packages to WORLD and installs (or upgrades)
  them if not already present, ensuring all dependencies are met.

Options:
  --initdb         Initialize a new package database
  -l, --latest     Disables normal heuristics for choosing which repository to install a
  -u, --upgrade    When adding packages which are already installed, upgrade them rather
  -t, --virtual NAME
                   Instead of adding the specified packages to WORLD, create a new
  --no-chown       Do not change file owner or group
--
2020-05-06 13:05:19 +03:00
Fredrik Gustafsson d61c009f7a apk: do not manage file ownership as non-root or when asked so
If apk is run as a non-root user, it's not possible to chown files.

Maintainers note: minor wording changes on commit log and man page.

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
2020-04-28 14:48:49 +03:00
Rasmus Thomsen 47053ede6b build: fix includedir in pkgconfig file
We want to include via <apk/$headerfile> and not via <$headerfile>,
so we want to add an include statement for the dir the apk folder
which contains the headers is contained in.
2020-04-01 19:14:28 +03:00
Timo Teräs 845b6397ab db: fix database reading without scripts or triggers
Most notably this happens after blank database is initialized with --initdb,
but can happen also in other conditions.

The error checking changes modified behaviour if the file does not exist.
Explicitly check for ENOENT and ignore it. But the behaviour is improved
from earlier as now e.g. EPERM will be detected and reported correctly.

Fixes #10679

Fixes: 6da3e8eb15 "istream, archive, db: convert db and tar function to use istream"
2020-04-01 10:47:54 +03:00
Rasmus Thomsen 104c65ab92 build: add apk.pc.in 2020-03-23 13:44:42 +02:00
Antoine Fontaine e8522411c1 lower min-width to 25 columns
Some screen size are quite small. For example, the default phosh
terminal is less than 50 character wide on Pinephone. This lowers the minimum
loading bar size to 25 characters.

For comparison, 25 character wide is just as wide as "apk add firefox
linux-lts" without the quotes.

Here's a bad picture to illustrate the result
gitlab.alpine.org/uploads/48c20f746fbf685b62b6bd73585ecbf2/pinephone-phosh.png
2020-03-21 17:41:49 +02:00
TBK a9916c2d4a fix glibc build
fixes #10677

src/apk_defines.h:152:15: error: unknown type name 'uint32_t'
 static inline uint32_t get_unaligned32(const void *ptr)
               ^~~~~~~~
2020-03-21 17:37:47 +02:00
Rasmus Thomsen 4966bc5ede build: enable building of development package
- always create .so and .a with the apk libary code
- create .pc file
- install headers and above mentioned files

Maintainers edit: merged commits, cleaned up sed script logic,
and few other minor fixes.
2020-03-16 15:36:46 +02:00
Timo Teräs c054fbc11e db: fix unaligned memory access in csum_hash() 2020-02-21 09:33:58 +02:00
Timo Teräs 3694dc5fa2 fix murmur3 hash unaligned memory access
- do not do unaligned accesses on non-x86 hardware
- clean up the code a little bit
2020-02-17 21:36:28 +02:00
Timo Teräs 6d11ec36e6 rename all iostream source to io_*.c 2020-02-14 16:53:12 +02:00
Timo Teräs 60b87557e5 rename all applets sources to app_*.c 2020-02-14 16:49:55 +02:00
Timo Teräs 72be813930 io: add stream copy helper 2020-02-14 16:45:13 +02:00
Timo Teräs d60477751f defines: add typeof() as it's gcc built-in
Related to MR !15
2020-02-04 10:36:05 +02:00
Timo Teräs 45d313c51c remove apk_time() as it is causing problems with shared objects
Instead, to make sure test mode produces same output, redefine
time() for the test mode binary.

Reverts parts of 0b82bcc53e.
2020-02-04 10:31:10 +02:00
Drew DeVault b62d252b7b Update apk to make man pages sole source of truth
Detailed docs have been removed from the apk binaries, in favor of
git-style short summaries of each command.
2020-01-26 09:03:18 +02:00
Drew DeVault 7c2c68df69 add apk-manifest(8) 2020-01-26 09:01:26 +02:00
Reid Rankin c7c8ab7c57 don't use hardcoded checksum buffer sizes 2020-01-25 16:35:31 +02:00
Reid Rankin aa882df511 document mysterious SHA-1 blob in database.c 2020-01-25 16:33:54 +02:00
Reid Rankin 6cc3e6a1de don't ignore md parameter to apk_fileinfo_hash_xattr_array() 2020-01-25 16:33:17 +02:00
Timo Teräs 9a76f0d6a6 db: additional clean up and hardening for apk extraction
This enforces all scripts to be in the control block, and
all data files to be in data block. Ignoring of dot files in
root is added back: packages without any real files will
ship one ".dummy" item in the data block to trigger processing
and validation to work.
2020-01-24 10:39:01 +02:00
Reid Rankin d25e5e3879 Harden signature verification process
This mostly boils down to making sure control_started and
data_started are consistently used to gate actions, instead of
relying whether on file names start with a '.'.

None of the weaknesses this fixes are exploitable, but they
might have become so after changes to seemingly-unrelated code,
so it's good to clean them up.
2020-01-24 09:28:48 +02:00
Reid Rankin 093c4b8077 Improve documentation of signature verification process 2020-01-24 09:28:00 +02:00
Reid Rankin 1f9e56d8a4 manifest: fix package file processing
This change ensures that apk_sign_ctx_process_file() and
apk_sign_ctx_parse_pkginfo_line() are called during archive
processing, allowing discovery of signatures and the data section
checksum.

Fixes a bug uncovered by commit f123d77e.
2020-01-24 05:11:58 +02:00
Timo Teräs 67696b2ac6 io: use min() instead of MIN() 2020-01-11 11:32:51 +02:00
Timo Teräs f123d77e0b archive: make apk_tar_parse check and close input stream
simplifies other code quite a bit
2020-01-11 11:32:51 +02:00
Timo Teräs 1de9ef422c io: convert bstream mmap to istream, remove the now obsolete bstream machinery 2020-01-11 11:32:21 +02:00
Timo Teräs 7caa217731 convert remaining locations to use istream instead of bstream 2020-01-11 11:20:48 +02:00
Timo Teräs 6da3e8eb15 istream, archive, db: convert db and tar function to use istream 2020-01-11 11:20:45 +02:00
Timo Teräs 7ca0d146ec istream: add buffering capability
Convert all implementations to do buffering. This is in preparation
to remove bstream interface as redundant.

istream_read() will return full reads unless end-of-file. The backends
can return short reads to optimize buffering or due to other reasons
like boundary change for gz.
2020-01-11 03:44:23 +02:00
Timo Teräs 9dda2d3c21 archive: remove support for old packages without xattr checksums 2020-01-11 03:43:38 +02:00
Timo Teräs 6fae74e1da remove obsolete md5.h 2020-01-10 03:48:29 +02:00
Drew DeVault 7e2e440d4c database.c: drop GNU extension for fnmatch
The non-wildcard version of the function is case-sensitive anyway.
2020-01-09 08:47:31 +02:00
Timo Teräs 3cd7d1e077 io: remove the now unused pid association with istream 2020-01-06 01:17:08 +02:00
Timo Teräs e39334e44f io: remove unused size parameter from bstream close 2020-01-06 00:27:17 +02:00
Matthew McGill e4aae45f96 info: Remove short options for --install-if, --rinstall-if
The "apk info" option --install-if, which had the short option -i,
was being shadowed by the apk global option --interactive, which
has the same short option. To fix this issue the short option for
--install-if was removed, as has been done in other cases where an
applet short option would be shadowed by a global short option. The
short option for --rinstall-if was also removed for consistency,
since it is the complement of --install-if.
2020-01-05 23:28:43 +02:00
Sören Tempel 7d930f942c commit: make use of the apk_get_human_unit function 2020-01-05 23:26:50 +02:00
Sören Tempel a6039e38a4 Humanize size output of `apk info`
This commits adds a function for receiving the closet byte unit and the
size in that unit for a given size_t. The function doesn't return a
string since dynamic memory allocation (or a static buffer) would be
required to implement this properly.

Discussion: It might be useful to add a command line flag for disabling
this behaviour in order to retain compatibility with the previous apk
info output format.
2020-01-05 23:26:50 +02:00
Timo Teräs 0b82bcc53e lua: fix build and update to lua5.3
Fixes #8360
2019-12-30 16:17:14 +02:00