resolve reverse dependencies after all packages have been loaded,
and avoid traversing the reverse name lists. now that we use
automatic virtual packages (soname, pkg-config, etc.) the reverse
dependency chains can become considerable longer than what it was
when the rdependency construction code was originally written.
In practice this should fix to e.g. not wipe out /etc/apk/world if
final flush to /etc/apk/world.new fails.
This was prompted by an incident the other day where I ran the root
partition of an Alpine box out of space using 'apk add', and apk
helpfully wiped the contents of /etc/apk/world at the same time.
It might be tricky to try to reproduce exactly the same failure,
but from an examination of the code, setting 'rc' before the final
call to fdo_flush rather than after is one possible cause of this
behavior. (If the entire contents of /etc/apk/world.new are buffered,
and all get written out in the final fdo_flush call, and that call
fails, fdo_close will still happily rename /etc/apk/world.new to
/etc/apk/world.)
the security implications are not as high as compared to regular
tar/unzip archiver. this is because you are anyway trusting
the package to install files anywhere in the filesystem.
this serves rather as a sanity to check against errors in created
package.
fetch_maperror() translates error codes returned by libfetch to our error
codes. Handle those in apk_error_str(), returning error messages which
advise the user of the most likely fix.
A custom error code, EAPKSTALEINDEX, has been added for cases where
retrieving a package fails due to a HTTP error 404 or similar.
[TimoT: add also EAPKBADURL, as well as organize a bit better where the
EAPKSTALEINDEX is generated]
According to the C standards, uint32_t is defined in stdint.h.
Presumably apk is usually built against C libraries where
stdint.h is indirectly included through another header file,
but this isn't the case with the version of glibc which I am using.
user xattrs on tmpfs are not supported no non-grsec kernels,
and many times root fs is mounted without user_xattr. Thus
to allow things to go smoothly on non-grsec kernels xattr
unsupported errors are now hidden.
xattrs can be fixed still now with "apk fix --xattrs"
on arm char is by default unsigned, so this caused crashes
as the ERR_PTR mechanism did not work as expected with unsigned
types. extend the array type to be signed short explicitly.
Package pinning was first implemented with 'p' tag. However, it
was before any release renamed to 's', and 'p' was reserved for
package provides support for which is used now.
this makes 'lbu diff' and aaudit diffs nice when a world
dependency is added or removed. sorting also makes the ordering
more deterministic as the world targets constraints are always
applied in the same order. test suite updated accordingly.
when removing large sets of packets, the ordering of removal
was not quaranteed to honor dependencies. this fixes the removal
order to be in reverse dependency order as far as possible.
there are only few combinations for that triplet, and they
occur multiple times reducing the struct sizes a bit. make
sane defaults and prepare to not write defaults to disk
to reduce on-disk installed db size.
Apk used to reset directory permissions always, but this is undesirable
if user has modified the permissions - especially during tmpfs boot.
Though, it is desirable to update the permissions when packaging has
changed permissions, or a new package is installed and the merged
permission mask / owner changes.
Thus the new code updates the permissions only if:
1) We are booting and directory is not in apkovl
2) The directory is modified by a package install/remove/upgrade
3) The filesystem directory permission matched database
Additionally "apk fix --directory-permissions" can be used to reset
all directory permissions to the database defaults.
Fixes#2966
This makes sure any conflicted packages will be removed first.
Useful if we know there are conflicting files, and want to avoid
adding potentially harmful replaces line. Add a test case for
this too.
Allows one arch index files to refer to other arch packages. Mostly
useful with noarch packages, but could be used e.g. to ship build
with some of packages optimized for specific cpu generation and
share most packages with the standard build.
the dynamic applet registration never worked with PIE, and as
a temporary hack -nopie was added to default link flags in 2008.
this commit reworks the applet registration mechanism to something
that is compatible with PIE, and removes the hack. finally!
basically this makes packager's life simpler as there is no need
to list the subpkg names in replaces. this was also very error
prone and tedious job to do properly.
noticeable fixes upgrading of packages which get pulled in only
by install_if rule. this also simplifies the inheritance calculation
for packages, as well as the place where it is done for install_if
triggered packages.
'available' really means 'available in one or more configured
repository'. Cache is not a repository we track, so those are
only available for installation, but not available as preferred
to be installed from repository. fixes#2831.
This implements a new protected path flag '!' to include always
matching entries in the backup mode (overlay). This is also turned
on for etc/apk to include everything there in overlay, because
the full list of repositories and signing keys need to be in
overlay - just getting them from a package is not enough during
bootstrap.
We should not disqualify non-selectable packages as it would mark it as
dirty, forcing it to reconsider the name again, which could end up in
an endless loop.
fixes#2135
- do not link with -llua. We should not link apk or lua module with it.
- default to lua 5.2
- allow override the lua version with LUA_VERSION
- allow override the lua pkg-config package with LUA_PC
- only add the pkg-config lua --cflags to the lua module
Wildcard matching with no names should match all packages only for
info and search applet. "apk del" would otherwise try to delete
everything, etc.
Fix also interactive mode to ask questions only if we are actually
changing something.
If name N is required, and all providers of A also provide B, it
means that only instances of B can be selected that provide N. This
is strong help with cases when so:libfoo.so.1 is updated to
so:libfoo.so.2 and not everything is recompiled.
- let the apk_print functions deal with the forced print itself. We
avoid that the callbacks need to deal with the force flag. We can
also get rid of the APK_PRINT_PROGRESS_* defines.
- let the reader of --progress-fd decide how often things are updated
rather than having a fixed granularity off 1/100 (percent)
- avoid detect screen size and percent/bar calculations in case the
--no-progress was given
- track satistics for both the ascii bar and percent info and update bar
only if either percent or bar changes. This makes the bar go smoother
when width is wider than 100 chars and it makes the percent counter
go smooth when screen width is less thann 100 chars. It also
simplifies the callbacks as they no longer need to deal with update
granularity.
Select latest version of package (if it is not pinned), and print
error if it cannot be installed due to other dependencies.
Together with --available, it selects the latest package which is
present at least in some repository.
This also fixes few solver issues with ordering of package selection
that got quite apparent with this flag. Namely, we cannot "lock"
a package until it's reverse dependencies are locked or not all of
the solver flags are propagated properly.
the pinning11 changed when @repo got the leading @ on error messages.
analyze_dep() now properly ignores conflict dependencies, as those
names are usually intentionally left unassigned.
So same package it is possible to not match same package multiple
times. Use generation count, so this is handled cleanly during
recursion, like in the use case of search applet.
Instead of the dependency oriented logic, switch to print them
for each package or name needed. Might give a bit more readable
errors now. There's still few corner cases that proper error is
not output, which are cought by the test cases.
- try harder to not change anything during self-upgrade
- also honor locking to packages that where earlier used in
merging common dependencies
- clarify upgrade applet help messages
fixes#1476
Instead of:
ERROR: Index generation failed: Success
The following is now printed:
ERROR: Metadata for package timo-1.0-r0 is too long.
ERROR: Index generation failed: No buffer space available
make cache a special kind of repository, and automatically cache
special packages (virtual packages, or ones installed from command
line). add test cases for handling virtual packages. fixes#1617.
It is incorrect optimization causing valid solutions to be skipped.
Any performance it might've gained, should be fixed by reintroduction
of the minimum penalty logic added in previous commit.
Enabled when all attempts to satisfy a name failed, we know that we
can ignore all decisions until we find a decision affecting the name
we wanted to satisfy.
We can't just use the primary name, as that would mess up
backtracking. We need to record the name which caused the name
to get considered - that way the right last_touched_decision is
used on backtracking.
* push_decision expects to always get the package primary 'name'
as apk_name. ASSERT that and fix problem cases.
(though - this might need to be reverted, and store the non
primary name in apk_decision instead to accomodate for better
backtracking optimizations)
* fix error reporting of virtual package names
* make 'assign_name' errors soft. the incorrect packages just are
no longer consider instead of aborting whole calculation.
* fix backtracking of virtual packages that are not depended
directly
fchmodat does not support this flag - symlinks do not have permissions.
Sysadmin probably does not expect us to not follow symlinks either:
if /var -> /mnt/foo/var, we should be making sure the permissions
and ownership is correct on the target directory, not on the symlink.
Since fchmodat never returned ENOENT with AT_SYMLINK_NOFOLLOW, this
also fixes directory re-creation if it does not exist. fixes#1348.
If a directory has protection mask, but does not exist in db, we
do not handle it right unless we calculate the protection mask by
hand, or create temporary db dir entry for it. For simplicity create
always the db dir entry -- depending on audit type we likely need
to create it anyway. This commit also caches the db dir entry in the
audit tree context to avoid duplicate lookups. ref #1241.
Do not print version numbers or compare result char when in quiet
mode. This makes the output suitable for be used in scripts or pipes.
For example:
# Upgrade all packages that matches a given regexp
apk version --limit '<' --quiet | grep $regexp | apk fix --reinstall
# Delete all packages that are removed from repository
apk version --limit '?' --quiet | xargs apk del
apk_name_state is now quite small; and we avoid overhead of two
pointers (+ malloc overhead) when we just make it part of apk_name.
It also fixes some problems (that got introduced) where apk_name_state
was not allocated.
Solver will now never report partial solution where a conflict
constraint is not satisfied. The is because with --force we might
install the partial solution; and if conflicted packages were to
be installed we might have extra trouble.
Required for provides support as package might be pulled in via
non-primary package name. This allows relatively easily to pass
through inherited flags via the provided names. ref #574.
Reasoning:
- it is less useful now that we do not do common dependency merging
- provides support would make the required logic overly complicated
- callgrind reports that depending on the case it can improve or
decrease performance (the overhead pays off only in some cases);
the difference is not large either way
Otherwise we might start to change packages unexpectedly when not
upgrading. This also fixes some other things the solver might've
decided to do.
Add also few test cases to detect bad behaviour.
In case someone prefers extra quesions while running apk in a
terminal. The file is always from the real root; not from --root
so that we will not accidentally enable interactive mode when in
initramfs bootstrap.
fixes#607.
audit is now mostly rewritten for the new functionality. And
has new features like --check-permissions, --recursive and
--packages.
$ROOT/etc/apk/protected_files.d/*.list can now contain additional
protected paths, one path per line:
+etc
@etc/init.d
-tmp
+ will include the directory as protected configuration directory.
@ considers the directory protected, but will backup only symlinks.
- removes any protection
lbu should be modified to put include and exclude paths in
etc/apk/protected_files.d/lbu.list. Additionally, some packages
might provide their own listings.
E.g. ssh might want to provide ssh.list with something like:
+root/.ssh
+home/*/.ssh
In case someone did "fix --force" for package for which we have no
APK available, we would uninstall the package instead of silently
ignoring the request. This could mean worse things.
So now we just consider unavailable packages a bad deal for reinstall
requests. And will downgrade if necessary. But if we really don't
have any APK available, we just skip the request but report it.
If n+1 packages depend A, and A depend on B. Add n+1 dependencies
to B. Otherwise if someone conflicts B, B might be left out.
Leaving package unassigned is no longer a non-preferred action,
this fixes the final test case that was failing.
And with --force we might even install that scenario.
Add also some debug checks.
callgrind says it's more overhead than improvement. back jumping
effectively prunes all bad trees. but can be added later if it
becomes needed; due to e.g. provides support.
It is faster to just scan the cache directory for existing packages
at startup than trying to faccessat() them on demand. It also makes
quite a few parts of the code more readable and simpler.
* properly do absolute scoring now, the previous scoring where
preference could get reduced could have caused incorrect early
pruning of search tree
* backtracking is now separated from package state, and first
branching point is the decision if a name is left unassigned
or if something _has_ to be assigned. this allows multiple future
search tree optimizations like handling of common dependencies
early.
* merge common dependency names early to provide deeper forward
checking.
Instead of "skipping" certain packages, we include them as-if required,
and at expansion time we decide if they actually need to be considered
for installation. This cleans up the expansion main loop a little bit
and makes the code work together better.
* cleaned up little bit on the internal state machine
* the decision applying mechanism now aborts early to avoid work
if we are approaching bad solution candidate
* package availability checking is now done on-demand; which
could still be improved
* solver internally calculates now using tags; not repository masks
* installeddb now contains the tag name where the package came from
-> we can now handle upgrades properly
* the pinning is still a preference, and not strictly enforced;
versioned dependencies may overrule preference
Only show a warning if we have a non-existing repository or bad
repository signature but don't abort.
This is not to break compat with behaviour in previous releases.
Forgot to reset per-name penalty when it got locked by apply_decision.
This also fine tunes compare_package_preference() to always prefer
packages specified on command line speeding up calculation certain
complicated solutions.
* upgrade needs explicit check so we don't try self-upgrade
(which would print additional messages on screen)
* add can fix problems, so check against the new world
* merge the code in few places
Previously we would cache the penalty when evaluating the final
solution, and adding that until we backtrack to first topology
position changing that penalty. However, we can just keep track
of minimum penalty based on name state, and add it. This allows
us to bail out early on bad branches because we know in advance
how things will turn out.
Previously we would not upgrade just by doing "apk add foo@tag" if
foo was already installed. It required explicit '-u'. This allows
'apk add' to explicitly prefer the newly specified pinning.
Timo Teräs