Commit Graph

1231 Commits (272784636112c84b57b3bd4742ab9f0235abe5b8)

Author SHA1 Message Date
Ariadne Conill ae4008c4f2 database: refactor cache remounting 2021-12-29 20:37:28 +02:00
Ariadne Conill 4fa6a9fd73 database: refactor tmpfs root detection and cache setup 2021-12-29 20:37:25 +02:00
Ariadne Conill 2c3cef8787 portability: take over strlcpy 2021-12-29 20:14:41 +02:00
Ariadne Conill 03a5e6d9b9 io: use opendirectory for UID/GID lookups on macOS
macOS has no concept of a chroot-specific UID/GID database, as the database is actually
LDAP.

ref #10794
2021-12-29 20:14:41 +02:00
Ariadne Conill 9d07d07fe4 abstract differences between GNU and Apple xattr functions 2021-12-29 20:14:41 +02:00
Ariadne Conill 6344a0eedb database: isolate linux-specific mountpoint headers 2021-12-29 20:14:41 +02:00
Ariadne Conill de15d4adbb legacy build system: include libportability headers 2021-12-29 20:14:41 +02:00
Ariadne Conill a7a0c0a6ca portability: implement skeleton with memrchr function 2021-12-29 20:11:17 +02:00
Timo Teräs 9ed4785f1c apk_adb: add missing compare functions
fixes #10803
2021-12-29 19:36:34 +02:00
Ariadne Conill 8d5784aaf5 database: use SA_RESETHAND instead of SA_ONESHOT when registering a SIGALRM handler
SA_ONESHOT is a GNU-specific alias for POSIX SA_RESETHAND.
2021-12-29 10:36:59 +02:00
Ariadne Conill 2a9c0277c0 use fstatat, not fstatat64
the stat64 family of functions were provided as transitional functions,
but when building on glibc with _GNU_SOURCE, or any other supported system,
the stat functions are equivalent to their stat64 counterparts
2021-12-29 07:52:48 +00:00
Paul Spooren f2a5959139 apk: add --preserve-env option
This options is useful for (post)install scripts to run with the same
environment variables as apk is executed.

[TT: minor stylistic changes]

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-12-29 09:50:50 +02:00
Paul Spooren 2da3f347d8 apk: add --force-no-chroot option
[TT: reoder changes to alphabetically sorted locations]

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-12-29 09:43:13 +02:00
Timo Teräs 70fe7b121a make stderr line buffered 2021-12-27 14:37:10 +02:00
Timo Teräs a662047e2c print: improve indented printing api
- make sure all commit errors go to stderr
- make it a bit more api like
2021-12-27 14:34:01 +02:00
Timo Teräs 6117de6eae list: sort options alphabetically 2021-12-27 10:34:17 +02:00
Paul Spooren f3fc0105f8 list --manifest
print installed packages in `<name> <version>` format. The format is
currently used in OpenWrt and allows downstream tools to compare what's
installed in the firmware.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-12-27 10:29:10 +02:00
Timo Teräs 0baf59627b io: fix tee error handling path
Make sure the from is not dereferenced/used on error path, and
cancel the ostream.

fixes #10800
2021-12-27 10:20:03 +02:00
Daniel Golle 4ccf038a9f add option --no-logfile 2021-12-23 08:07:42 +00:00
Paul Spooren af489f3b6f io.c: add missing limits.h
Fixes compilation on MacOS X
2021-12-20 12:01:41 +00:00
Ariadne Conill 6941f8bd61 adb: use sys/uio.h for iovec operations
readv/writev and struct iovec are declared in sys/uio.h per POSIX.
using it without sys/uio.h is a GNU extension.

ref #10794
2021-12-14 13:06:26 -06:00
Ariadne Conill 36fcbeed1a applet: suppress is_error warning on clang
clang does not ignore inline functions when checking for unused functions

ref #10794
2021-12-14 18:54:16 +00:00
Ariadne Conill 5bd3b1e3fc uvol: cast size data to intmax_t
this ensures the %ju format parameter is always passed an integer that is
appropriately sized, e.g. on mac where off_t is normally 32-bit.

ref #10794
2021-12-14 18:49:11 +00:00
Ariadne Conill 6df9f0ed59 defines: provide APKE_REMOTE_IO as portable alternative to EREMOTEIO 2021-12-14 18:48:18 +00:00
Ariadne Conill 46a2049d46 meson: fix underlinking issue with lua module 2021-12-14 18:46:18 +00:00
Ariadne Conill 3fd120db5c applet: rework APK_DEFINE_APPLET to use constructor attribute
this allows the applet registration to work in a portable way, without having to
weird things with the linker.

ref #10794
2021-12-14 12:05:59 -06:00
Ariadne Conill f4f7796ccc everywhere: use stdlib.h for malloc(3) definition, not GNU-specific malloc.h
musl implements support for malloc.h, but it is only a stub.  we do not use
any of the GNU-specific malloc interfaces, so just use POSIX stdlib.h instead.

ref #10794
2021-12-14 16:23:29 +00:00
Ariadne Conill b2d3d63184 lua: remove features.h
features.h is a GNU-specific header, and is not required for POSIX-compatible code

macOS does not provide features.h

ref #10794
2021-12-13 14:44:21 -06:00
Timo Teräs 8c6c44d3eb extract_v3: make sure empty file's hash is copied too
fixes #10785
2021-12-03 15:06:49 +02:00
Timo Teräs 1ab81fdd4c db: convert v3 scripts to ipkg on install
fixes #10796
2021-12-03 14:53:30 +02:00
Timo Teräs 4dde7e7e0f extract: fix uvol to not be called for uvol directory 2021-12-02 18:04:43 +02:00
Timo Teräs 845fa93814 db: synthesize symlink hashes for v3 packages
The v3 format will not require the hash, but synthesize it to the
DB so v3 packages can get installed properly.

fixes #10785
2021-11-22 13:41:22 +02:00
Timo Teräs cce6239074 db: fix regression in creating .apk-new files
The apk_fsdir_file_digest() servers dual purpose: to calculate
the checksum, but also to determine if the file exists. Move
it's error check where it originally was.

fixes #10791
2021-11-22 10:35:18 +02:00
Timo Teräs 83b71f0baf db: lazy create directories with --initdb
fixes #10778
2021-11-15 15:56:48 +02:00
Timo Teräs 061bd69b80 fs_fsys: fix extraction of symlinks
Symlinks should use the real target filename instead of
the temporary extraction name.
2021-11-15 15:55:15 +02:00
Timo Teräs de281f0f83 extract_v2: handle "untrusted package" as normal error
The "untrusted" error is higher priority than malformed package.
But given that we expect valid .apk or index as argument, the
untrusted error is likely more accurate than having malformed
package.
2021-11-15 15:41:59 +02:00
Timo Teräs 25c152e653 commit: print download size of packages in interactive mode
ref #10788
2021-11-15 14:37:25 +02:00
Timo Teräs 16e0f6df7f fix fetching of depdencies only packages
Remove the APK_REPOSITORY_CACHED bit from dependencies only
packages (that is, installed_size == 0). For fetch, the problem
is that apk_db_select_repo() would return the cache repository,
but the package would not be there. Update also the locations
needed to handle these packages correctly without the cached
repository bit being set.
2021-11-15 13:35:59 +02:00
Timo Teräs 3cb5ce2a37 fetch: add --url option to print the download URLs 2021-11-12 13:10:31 +02:00
Timo Teräs 3e6261392f pkg: fix reading of v3 package meta data
The adb object for apk_pkg_from_adb() was changed in commit 7d6de220
"database: support loading v3 indexes" which also addressed the broken
site in question but omitted updating the argument.

fixes #10783
2021-11-11 11:41:29 +02:00
Timo Teräs 97e3647e1d pathbuilder: fix push return value
Always return the original length; not the one with trailing '/'
amended.

fixes c60b7424 "optimize apk_pathbuilder_pop to get the old length"
ref #10784
2021-11-11 11:24:49 +02:00
Timo Teräs 9d7b4bd253 extract_v3: fix extraction of zero length files
The apk_istream is null for zero length files, and needs
special handling.

fixes #10784
2021-11-11 11:22:06 +02:00
Timo Teräs c60b7424a0 optimize apk_pathbuilder_pop to get the old length
avoids memrchr
2021-11-09 21:50:11 +02:00
Timo Teräs 7c8f872572 adbdump: fix dumping signatures
give a fake trust root allowing anything. adbdump code will verify
and dump each signature individually.
2021-11-09 21:50:11 +02:00
Timo Teräs 7d6de2203e database: support loading v3 indexes 2021-11-09 21:50:11 +02:00
Timo Teräs a673653200 database: implement uvol support
by adding an abstraction layer to the file system
2021-11-09 21:50:11 +02:00
Timo Teräs d441cf523c db: purge temporary files without explicit modification check
When extraction failed, the user has had no opportunity to edit
any files. Just clean up.
2021-11-03 16:06:04 +02:00
Timo Teräs 5c0c249b78 extract: move uvol extraction code to extract
fix also the progress callback on uvol extraction
2021-11-03 15:37:10 +02:00
Timo Teräs d821265865 db, extract: handle waitpid returning EINTR 2021-11-03 15:37:10 +02:00
Timo Teräs 6941aac4c6 db, audit: support sha256-160 hashes for v3 pkg compat 2021-11-03 15:37:10 +02:00
Timo Teräs 40f0866367 db, add: support adding v3 package files 2021-10-27 16:14:45 +03:00
Timo Teräs 9bd1e95e5b mkpkg: make unique-id field a blob 2021-10-25 15:48:59 +03:00
Timo Teräs 50ab589e9a version: increase number of digits supported in version component
Report also version numbers as invalid if there's more than 18
digits.

fixes #10774
2021-10-25 13:30:40 +03:00
Timo Teräs d38e2dd65a enable uvol support by default 2021-10-04 15:05:38 +03:00
Timo Teräs 9f07a3447e adb: fix --allow-untrusted to work again
and fix the error code if untrusted adb is seen
2021-09-13 13:18:17 +03:00
Timo Teräs 984cd3e0b6 extract: remove redundant file unlink on failure path
apk_extract_file() already calls unlink if the error is fatal.
2021-09-03 13:47:04 +03:00
Timo Teräs fd42c073bf extract: pass root object instead of database 2021-09-03 13:44:48 +03:00
Alex Xu (Hello71) bd15683670 add: warn when creating empty virtual packages
see https://gitlab.alpinelinux.org/alpine/aports/-/issues/11736

[TT: minor stylistic change]
2021-09-03 13:24:39 +03:00
Paul Spooren 9b3e68d4ba tar: include missing limits.h
Without this header the build fails when used as a OpenWrt build
dependency.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-08-24 11:54:24 -10:00
Timo Teräs 7436193885 mkpkg: add support for scripts and triggers 2021-08-23 17:02:50 +03:00
Timo Teräs 99fa1fb797 trust: always use have valid struct apk_trust *
Make sure we always have valid struct apk_trust * for code using it.

Load the signing keys directly when being specified to produce
sane error message if loading them fails.
2021-08-23 17:02:50 +03:00
Timo Teräs 72d8cb8937 remove IS_ERR_OR_NULL
In most places where pointer can be an 'error' it cannot be null
pointer. Further, in those cases just calling PTR_ERR() is not enough
to handle the null case. Simplify code by removing this case.

If NULL case needs to be handled, it's better to add separate check
and return fixed error code in that case.
2021-08-23 17:02:50 +03:00
Timo Teräs 91085a4874 extract: support v3 index verification 2021-08-23 17:02:50 +03:00
Timo Teräs da7264733d tar: improve compatibility
- check magic field for 'ustar' on read
- harden get_octal to report errors on non-octal characters
  (e.g. GNU base256 encoding), fixes #10757
- fix mtime and size octal fields to not have zero terminator
2021-08-23 17:02:46 +03:00
Timo Teräs 33c1de8ea1 db: honor SOURCE_DATE_EPOCH for scriptdb
fixes #10762
2021-08-23 11:11:29 +03:00
Timo Teräs ba0e2b579d list: recognize both --upgradable and --upgradeable
fixes #10759
2021-08-23 11:07:50 +03:00
Timo Teräs a46043bcc4 extract: fix directory handling
'is' is null for directories
2021-08-17 13:58:20 +03:00
Timo Teräs f446f2253d extract: fix --force-overwrite
Directories are handled specially in package installation code, but
extract applet uses also apk_extract_file() to create directories.
These should not be unlinked as that fails with unexpected error code
terminating the extraction.
2021-08-03 21:47:19 +03:00
Timo Teräs aa44cfac05 extract: fix extracting untrusted package 2021-08-03 21:21:43 +03:00
Timo Teräs a0e59e0897 manifest: support v3 packages 2021-07-30 18:30:44 +03:00
Timo Teräs 727ecc440c verify: support v3 packages 2021-07-30 18:06:34 +03:00
Timo Teräs d3e71bb406 mkndx: support v3 packages 2021-07-30 17:54:30 +03:00
Timo Teräs f61eba0627 extract: use extraction api, and implement it for v3 packages
The extract applet now works with both v2 and v3 packages.
2021-07-30 17:54:25 +03:00
Timo Teräs 5843daf7a1 Further refactor extract API to have separate ops vtable
This splits the callbacks by type, and further prepares the API
to be usable for v3 files too.
2021-07-27 16:28:22 +03:00
Timo Teräs 9c843e4ecd Refactor .apk extraction code
This moves and isolates the tar code to tar.c. And the actual
file extraction to disk is moved to extract.c.

A new API is introduced and used for v2 file extraction. This
essentially moves and isolates the apk_sign_ctx_* beast into
extract_v2.c and offers a saner interface to handling packages.

A place holder is added for v3 extraction.
2021-07-27 14:09:38 +03:00
Timo Teräs 2d4e88aeb1 extract: fix handling of error of regular file extraction
fix the error checking, allow --force-overwrite to work and
do not delete existing file in case of error
2021-07-26 21:08:09 +03:00
Timo Teräs 003e713502 adb: fix some error handling paths 2021-07-26 14:23:49 +03:00
Timo Teräs 083ea5a13b db: add some error checking to database state write
Make errors more observable. Unfortunately full rollback is
non-trivial to implement. This is something to be fixed with
the v3 database format.
2021-07-26 14:10:07 +03:00
Samanta Navarro d212aca070 db: fix installed db writing with long names
Packages containing files with path names longer than 1024 characters
cannot fit into the buffer which is used to write "installed" database.
This leads to bbuf being APK_BLOB_NULL in apk_db_write_fdb because
apk_blob_push_blob notices the condition and correctly handles it.

The problem occurs when arguments to apk_ostream_write are manually
calculated by pointer arithmetics. Since bbuf.ptr is NULL in such a
case, bbuf.ptr - buf leads to a huge size value while buf still points
into the stack.

fixes #10751

[TT: minor edit to commit and abbreviating the commit message]
2021-07-26 13:36:28 +03:00
Timo Teräs aca77e8410 Use __attribute__ ((format)) where possible and fix issues found by it 2021-07-26 10:25:23 +03:00
Timo Teräs 62e1cba691 adb: adb_walk_adb fix out of boundary write
If a signature is longer than max allowed adb signature length
then adb_walk_block writes out of boundary of stack variable tmp.

The len += snprintf is not safe per standard snprintf implementation
(kernel does it differently). Introduce and use apk_blob_push_fmt
which does the checking better.

Fixes #10752

Reported-by: Samanta Navarro <ferivoz@riseup.net>
2021-07-26 10:19:20 +03:00
kpcyrd 90228c4d26 io_archive: Use SOURCE_DATE_EPOCH for meta files instead of current time
[TT: minor stylistic changes]
2021-07-25 09:01:52 +03:00
Timo Teräs c1405f9311 adb: fix error handling of non-adb files 2021-07-25 08:52:43 +03:00
Timo Teräs 3d203e8f5f db: allow read-only operations without cache
fixes #10748
2021-07-23 14:39:38 +03:00
Timo Teräs a0e9c909f7 extract: rework uvol name logic
Add uvol_name to struct apk_file_info so it can be passed down
the extract callbacks in future work. Modify uvol name to not
include the path, but just the filename portion.
2021-07-23 13:51:12 +03:00
Daniel Golle 6c41c6f32e extract: fix uvol parameter order 2021-07-23 13:37:17 +03:00
Timo Teräs 33e85fb1f1 extract: remove unneeded argument from uvol_extract 2021-07-22 16:14:49 +03:00
Timo Teräs e6b7fa49ce adb: add a small adb blob header with versions and root object 2021-07-22 16:14:49 +03:00
Timo Teräs e2ebd761a5 adb: refactor struct adb_header to adb_file_header
In struct adb, do not keep the whole header, just the schema
in host byte order.
2021-07-22 16:14:49 +03:00
Timo Teräs 2f1186aa40 adb: make adb mmap/stream parser more united, add ADB_BLOCK_DATAX
Harden the error checking of expected block types and their order.
Add ADB_BLOCK_DATAX as reserved for >1GB blocks.
2021-07-22 16:14:49 +03:00
Timo Teräs 05364528bb adb, convndx: check istream_close result 2021-07-22 15:30:08 +03:00
Timo Teräs 7e585512f4 io: make apk_istream_get/read() fail on incomplete read 2021-07-22 15:30:08 +03:00
Timo Teräs 395e92b66e io: formalize apk_ostream_write() always writing full data 2021-07-22 15:30:08 +03:00
Timo Teräs 796d298313 rework apk_istream_splice and apk_istream_tee
- apk_istream_splice usage is converted to apk_stream_copy which
  is the newer variant. With caching enabled by default, this
  makes more sense mmapping or using separate buffers.

- apk_istream_tee is reworked to write to apk_ostream, which simplifies
  quite a bit of various things
2021-07-22 15:30:08 +03:00
Timo Teräs b559a81694 io: rework apk_istream_get_* to not return erros in blob value
The interface was slightly cumbersome, so replace these functions
to return explicit error, and make the return blob a pointer arg.
2021-07-22 15:30:08 +03:00
Timo Teräs 94c5e01038 adb: remove the now unused get_default_int() hook
It is no longer needed, and can be later reintroduced if needed.
2021-07-22 15:30:08 +03:00
Timo Teräs 3b00c0dc80 adb: unify various interfaces to adb_m_process
Removes code duplication, and puts important checks in one place.
Support seamless decompression in adbdump.
2021-07-22 15:30:08 +03:00
Timo Teräs 69bcdd23b9 adbsign: support compressed adb files 2021-07-22 15:30:08 +03:00
Timo Teräs 7af34db6cf adb: support seamless de/compression of adb files
Add compression header of adb files. Support uncompressed and
deflate compression at this time.
2021-07-22 15:30:08 +03:00
Timo Teräs 7b58f909fa convndx: add error reporting 2021-07-22 15:30:03 +03:00