Especially a newline can produce havoc in the database file as
the filename is written there as-is. This hardenes the extraction
to consider any control character as malicious. Additional
hardening is added to database loading to better detect corrupt
state and return proper error code about it.
Reported-by: Luca Weiss <luca@z3ntu.xyz>
Regression introduced by 0fb0d30 which makes parsing a description
a critical failure.
[TT: Minor stylistic change. Fix also missing final line change
from the earlier commit]
The new v3 applets don't use it, and eventually all applets will
use the new formats only.
This prepares the code for this, and moves trust, id_cache, and
root_fd to apk_ctx which is enough for v3 applets at this time.
The generic code is made to not initialize apk_database if it's
not needed.
This is a flat buffers inspired format that allows fast
mmaped access to the data with low overhead, signature support
and relatively good forward support.
On some systems the `/var/` dir is mounted in a tmpfs which is reseted
after each reboot. For that reason no post-install script can handle the
creation of the cache dir at `/var/cache/apk`.
Check on database opnening if the folder is available, if not create it.
Fixes#10715
Signed-off-by: Paul Spooren <mail@aparcar.org>
It used to be relative to the --root specified root, but that
causes issues with relative command line filenames and is unintuitive.
Update documentation accordingly. Fixes#10702.
This greatly helps with memory management on applications that
may want to daemonize and open/close database several times.
Also the lifetime and "owner" of memory for all data is now
explicitly bound to owning struct apk_database, which might
be helpful when writing language bindings. As side effect, the
interned "atoms" are unique only within what apk_database, so
comparing packages from different apk_database may not work
as expected.
Fixes#10697
If apk is run as a non-root user, it's not possible to chown files.
Maintainers note: minor wording changes on commit log and man page.
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Most notably this happens after blank database is initialized with --initdb,
but can happen also in other conditions.
The error checking changes modified behaviour if the file does not exist.
Explicitly check for ENOENT and ignore it. But the behaviour is improved
from earlier as now e.g. EPERM will be detected and reported correctly.
Fixes#10679
Fixes: 6da3e8eb15 "istream, archive, db: convert db and tar function to use istream"
This enforces all scripts to be in the control block, and
all data files to be in data block. Ignoring of dot files in
root is added back: packages without any real files will
ship one ".dummy" item in the data block to trigger processing
and validation to work.
This mostly boils down to making sure control_started and
data_started are consistently used to gate actions, instead of
relying whether on file names start with a '.'.
None of the weaknesses this fixes are exploitable, but they
might have become so after changes to seemingly-unrelated code,
so it's good to clean them up.
When unpacking a file that is in root, it got a temporary file
name /.apk... however if the --root option was used it should
have the name root/.apk... otherwise unpacking will fail if the
user does not have write access to /.
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
A crafted .apk file could to trick apk writing unverified data to
an unexpected file during temporary file creation due to bugs in handling
long link target name and the way a regular file is extracted.
Several hardening steps are implemented to avoid this:
- the temporary file is now always first unlinked (apk thus reserved
all filenames .apk.* to be it's working files)
- the temporary file is after that created with O_EXCL to avoid races
- the temporary file is no longer directly the archive entry name
and thus directly controlled by potentially untrusted data
- long file names and link target names are now rejected
- hard link targets are now more rigorously checked
- various additional checks added for the extraction process to
error out early in case of malformed (or old legacy) file
Reported-by: Max Justicz <max@justi.cz>
During netboot on systems without RTC, time() will be near zero,
and the index fill not exist. Thus the plain test of st.st_mtime
against system time failed. Verify that fstatat() succeeds.
This modifies apk cache for indexes to be automatically refreshed
periodically without explicit 'update' or '--update-cache' usage.
The default is to do if-modified-since request if the local copy
is older than 4 hours. This age can be changed with --cache-max-age.
Using --update-cache will change this age to 60 seconds to make
sure the cached copy is relatively new. The small age is in order
to try to avoid downloading indexes second time when apk-tools is
upgraded and apk re-execs after self-upgrade.
Accordingly using explicitly 'apk update' will now enforce
--force-refresh and request the very latest index by requesting
any potential http proxy to do refresh too.
This unloads --force as several of the things are really not wanted
together. E.g. --force-refresh is a lot different from --force-broken-world
and doing --force to get the other might introduce unwanted behaviour.
--force is still kept for backwards compatibility and it enables
most things --force was used for.
This change just changes to keep deleted directory items in
the hash with ref count zero and modified flag set. Those entries
are reused when needed. The side effect is that fire_triggers()
will now see those removed direcotries and reports them. Other
enumerators of the directories hash are protected to skip removed
directories when appropriate.
Depending how the directory entries are ordered, the cached dir
instance might not have been updated correctly. This has not been
a problem as the entries have been ordered, but is now triggered
on ppc.
fixes#5616
The original intention was not use unnecessary space on tmpfs
e.g. if the cache directory is a mount point, but accidentally
left unmounted. But there are valid cases when packages are
intentionally wanted to be cached on tmpfs. If caching is not
desired, the user can just remove the cache directory.
This allows for instance integration of etckeeper
[TT: Reorganized code a bit, and modified to use single
directory commit_hooks.d with argument for script of stage.]
Implement --no-cache. The index is read directly from network and not
cached. This is useful for docker, where you install a set of packages
and directly after purge the cache. (see
1fc9e59d16/builder/scripts/apk-install)
fixes#4905
preserve [am]time for all packages and indexes. this fixes the caching
error that 'apk update' is after new index is generated, but before
the used mirror is synchronized. this caused local apkindex timestamp
to be newer than file in mirror, when in fact it was outdated index.
this also fixes fetched files to have build timestamp so that files
going to .iso or custom images have proper timestamps (rsync with
appropriate --modify-window now works)
the problem is that var/lock is on root installs symlink to /run/lock
(on tmpfs) and does not exist if doing chroot() to that root. fixes
apk to work when chrooted to existing rootfs install.
resolve reverse dependencies after all packages have been loaded,
and avoid traversing the reverse name lists. now that we use
automatic virtual packages (soname, pkg-config, etc.) the reverse
dependency chains can become considerable longer than what it was
when the rdependency construction code was originally written.
the security implications are not as high as compared to regular
tar/unzip archiver. this is because you are anyway trusting
the package to install files anywhere in the filesystem.
this serves rather as a sanity to check against errors in created
package.
user xattrs on tmpfs are not supported no non-grsec kernels,
and many times root fs is mounted without user_xattr. Thus
to allow things to go smoothly on non-grsec kernels xattr
unsupported errors are now hidden.
xattrs can be fixed still now with "apk fix --xattrs"
Package pinning was first implemented with 'p' tag. However, it
was before any release renamed to 's', and 'p' was reserved for
package provides support for which is used now.
there are only few combinations for that triplet, and they
occur multiple times reducing the struct sizes a bit. make
sane defaults and prepare to not write defaults to disk
to reduce on-disk installed db size.
Apk used to reset directory permissions always, but this is undesirable
if user has modified the permissions - especially during tmpfs boot.
Though, it is desirable to update the permissions when packaging has
changed permissions, or a new package is installed and the merged
permission mask / owner changes.
Thus the new code updates the permissions only if:
1) We are booting and directory is not in apkovl
2) The directory is modified by a package install/remove/upgrade
3) The filesystem directory permission matched database
Additionally "apk fix --directory-permissions" can be used to reset
all directory permissions to the database defaults.
Fixes#2966
Allows one arch index files to refer to other arch packages. Mostly
useful with noarch packages, but could be used e.g. to ship build
with some of packages optimized for specific cpu generation and
share most packages with the standard build.
basically this makes packager's life simpler as there is no need
to list the subpkg names in replaces. this was also very error
prone and tedious job to do properly.
This implements a new protected path flag '!' to include always
matching entries in the backup mode (overlay). This is also turned
on for etc/apk to include everything there in overlay, because
the full list of repositories and signing keys need to be in
overlay - just getting them from a package is not enough during
bootstrap.
Timo Teräs
Ariadne Conill