Instead of the dependency oriented logic, switch to print them
for each package or name needed. Might give a bit more readable
errors now. There's still few corner cases that proper error is
not output, which are cought by the test cases.
- try harder to not change anything during self-upgrade
- also honor locking to packages that where earlier used in
merging common dependencies
- clarify upgrade applet help messages
fixes#1476
Instead of:
ERROR: Index generation failed: Success
The following is now printed:
ERROR: Metadata for package timo-1.0-r0 is too long.
ERROR: Index generation failed: No buffer space available
make cache a special kind of repository, and automatically cache
special packages (virtual packages, or ones installed from command
line). add test cases for handling virtual packages. fixes#1617.
It is incorrect optimization causing valid solutions to be skipped.
Any performance it might've gained, should be fixed by reintroduction
of the minimum penalty logic added in previous commit.
Enabled when all attempts to satisfy a name failed, we know that we
can ignore all decisions until we find a decision affecting the name
we wanted to satisfy.
We can't just use the primary name, as that would mess up
backtracking. We need to record the name which caused the name
to get considered - that way the right last_touched_decision is
used on backtracking.
* push_decision expects to always get the package primary 'name'
as apk_name. ASSERT that and fix problem cases.
(though - this might need to be reverted, and store the non
primary name in apk_decision instead to accomodate for better
backtracking optimizations)
* fix error reporting of virtual package names
* make 'assign_name' errors soft. the incorrect packages just are
no longer consider instead of aborting whole calculation.
* fix backtracking of virtual packages that are not depended
directly
fchmodat does not support this flag - symlinks do not have permissions.
Sysadmin probably does not expect us to not follow symlinks either:
if /var -> /mnt/foo/var, we should be making sure the permissions
and ownership is correct on the target directory, not on the symlink.
Since fchmodat never returned ENOENT with AT_SYMLINK_NOFOLLOW, this
also fixes directory re-creation if it does not exist. fixes#1348.
If a directory has protection mask, but does not exist in db, we
do not handle it right unless we calculate the protection mask by
hand, or create temporary db dir entry for it. For simplicity create
always the db dir entry -- depending on audit type we likely need
to create it anyway. This commit also caches the db dir entry in the
audit tree context to avoid duplicate lookups. ref #1241.
Do not print version numbers or compare result char when in quiet
mode. This makes the output suitable for be used in scripts or pipes.
For example:
# Upgrade all packages that matches a given regexp
apk version --limit '<' --quiet | grep $regexp | apk fix --reinstall
# Delete all packages that are removed from repository
apk version --limit '?' --quiet | xargs apk del
apk_name_state is now quite small; and we avoid overhead of two
pointers (+ malloc overhead) when we just make it part of apk_name.
It also fixes some problems (that got introduced) where apk_name_state
was not allocated.
Solver will now never report partial solution where a conflict
constraint is not satisfied. The is because with --force we might
install the partial solution; and if conflicted packages were to
be installed we might have extra trouble.
Required for provides support as package might be pulled in via
non-primary package name. This allows relatively easily to pass
through inherited flags via the provided names. ref #574.
Reasoning:
- it is less useful now that we do not do common dependency merging
- provides support would make the required logic overly complicated
- callgrind reports that depending on the case it can improve or
decrease performance (the overhead pays off only in some cases);
the difference is not large either way
Otherwise we might start to change packages unexpectedly when not
upgrading. This also fixes some other things the solver might've
decided to do.
Add also few test cases to detect bad behaviour.
In case someone prefers extra quesions while running apk in a
terminal. The file is always from the real root; not from --root
so that we will not accidentally enable interactive mode when in
initramfs bootstrap.
fixes#607.
audit is now mostly rewritten for the new functionality. And
has new features like --check-permissions, --recursive and
--packages.
$ROOT/etc/apk/protected_files.d/*.list can now contain additional
protected paths, one path per line:
+etc
@etc/init.d
-tmp
+ will include the directory as protected configuration directory.
@ considers the directory protected, but will backup only symlinks.
- removes any protection
lbu should be modified to put include and exclude paths in
etc/apk/protected_files.d/lbu.list. Additionally, some packages
might provide their own listings.
E.g. ssh might want to provide ssh.list with something like:
+root/.ssh
+home/*/.ssh
In case someone did "fix --force" for package for which we have no
APK available, we would uninstall the package instead of silently
ignoring the request. This could mean worse things.
So now we just consider unavailable packages a bad deal for reinstall
requests. And will downgrade if necessary. But if we really don't
have any APK available, we just skip the request but report it.
If n+1 packages depend A, and A depend on B. Add n+1 dependencies
to B. Otherwise if someone conflicts B, B might be left out.
Leaving package unassigned is no longer a non-preferred action,
this fixes the final test case that was failing.
And with --force we might even install that scenario.
Add also some debug checks.
callgrind says it's more overhead than improvement. back jumping
effectively prunes all bad trees. but can be added later if it
becomes needed; due to e.g. provides support.
It is faster to just scan the cache directory for existing packages
at startup than trying to faccessat() them on demand. It also makes
quite a few parts of the code more readable and simpler.
* properly do absolute scoring now, the previous scoring where
preference could get reduced could have caused incorrect early
pruning of search tree
* backtracking is now separated from package state, and first
branching point is the decision if a name is left unassigned
or if something _has_ to be assigned. this allows multiple future
search tree optimizations like handling of common dependencies
early.
* merge common dependency names early to provide deeper forward
checking.
Instead of "skipping" certain packages, we include them as-if required,
and at expansion time we decide if they actually need to be considered
for installation. This cleans up the expansion main loop a little bit
and makes the code work together better.
* cleaned up little bit on the internal state machine
* the decision applying mechanism now aborts early to avoid work
if we are approaching bad solution candidate
* package availability checking is now done on-demand; which
could still be improved
* solver internally calculates now using tags; not repository masks
* installeddb now contains the tag name where the package came from
-> we can now handle upgrades properly
* the pinning is still a preference, and not strictly enforced;
versioned dependencies may overrule preference
Only show a warning if we have a non-existing repository or bad
repository signature but don't abort.
This is not to break compat with behaviour in previous releases.
Forgot to reset per-name penalty when it got locked by apply_decision.
This also fine tunes compare_package_preference() to always prefer
packages specified on command line speeding up calculation certain
complicated solutions.
* upgrade needs explicit check so we don't try self-upgrade
(which would print additional messages on screen)
* add can fix problems, so check against the new world
* merge the code in few places
Previously we would cache the penalty when evaluating the final
solution, and adding that until we backtrack to first topology
position changing that penalty. However, we can just keep track
of minimum penalty based on name state, and add it. This allows
us to bail out early on bad branches because we know in advance
how things will turn out.
Previously we would not upgrade just by doing "apk add foo@tag" if
foo was already installed. It required explicit '-u'. This allows
'apk add' to explicitly prefer the newly specified pinning.
Improves /etc/apk/repositories format so you can say:
http://nl.alpinelinux.org/alpine/v2.3/main
@edge http://nl.alpinelinux.org/alpine/edge/main
@testing http://nl.alpinelinux.org/alpine/edge/testing
After which you can pin dependencies to these tags using:
apk add stableapp newapp@edge bleedingapp@testing
Apk will now by default only use the untagged repositories,
but adding a tag to specific dependency:
1. will prefer that tag for the name
2. allowing pulling in dependencies from that tag (though,
it prefers untagged packages to satisfy deps if possible)
fixes#575
If two packages replace each other, the one with highes priority
will keep the file. Additionally, if we have a package overriding
another's file it's remembered and handled properly. This is
essentially to allow "policy packages" which just overwrite certain
(configuration) files from other package(s).
"replaces" is now turned to a full dependency type list, so you can
make package overwrite files only certain versions of the package
(though, we should probably take this into account already at solution
calculation phase).
Also make 'info --replaces' print the "replaces" of the package.
This is in preparation for the policy package support, which still
requires "replacement priority" field to decide which packages' files
get the preference.
The array struct aligned size to 64 bit on x86_64 which caused bad things
to happen.
We use size_t to make sure the size element is correct regardless arch.
Solution found by Timo.
allow per-name solver flags to be inheritable, and use them in
self-upgrade, add -u and the fix applet. this gives more familiar
behaviour for the upgrades.
the only bit of information needed in solver commit is the "hard"
topology sorting information for trigger ordering. fixes a bug in
"apk del" which uses the state pointers to do intermediate
calculations between solution solving and commit.
Allow to select packages that conflict in case we are looking for
errors. This allows 'add --force' to install (on boot) the set of
packages with minimum conflicts.
* each package name has two sorting positions, one which causes
install_if triggers to be run, and other for bulk dependencies
* fix also inverted ordering of package installations
* the solver no longer does look-ahead locking of names
(could be possibly optimized later); instead names are now
always ordered strictly to properly detect the package names
which are unsolveable
* basic error tests added, so we can see the most likely problem
in dependencies easily
* basic code for a backtracking, forward checking dependency satisfier
* works better when there are tricky dependencies to solve
(when can't just upgrade everything to most preferred versions)
* the new code always evaluates all of 'world' constraints
(old code just does incremental updates based on heuristics)
* is probably somewhat slower than old code (probably unnoticeable
difference in most cases)
* makes easier to write support for provides and repository pinning
* test applet and a bunch of test cases added which uses the new code
* from the old feature set install_if is not yet implemented
This applet can be used to generate data for graphviz tools.
Useful to visualize package dependencies, and possible errors
in the repository.
Usage examples:
apk dot gnome-desktop | tred | dot -Tpng gnome-desktop.png
-- to generate simplified dependency chart of gnome-desktop
apk dot --errors | dot -Tpng index-errors.png
-- to generate chart visualizing dependency errors in index
Make indexer keep noise about errors that prevent index generation.
Detect certain errors in the APKs better. And also have the applet
return error in these scenarios.
commit 4e72075fba introduced a bug where package installation might happen
in wrong order (reminder for self to separate the package version deduction
to separate step from installation ordering). this restricts the earlier
commit to not mingle with the install order.
Our build infra does not yet handle properly noarch, so for the
time being we will rewrite them as native packages in index. This
allows the package to be fetched from the proper URL. This feature
will be removed once abuild and the build infra handle noarch
properly.
commit 4e72075fba introduced late locking for top level packages,
but used the wrong package's top level flag for the check. this
fixes a problem that dependencies might not get pulled in.
move all files therein to other places. this allows /var to be
mounted from harddisk, but rest of system be run from ramdisk.
this also removes support for historical version of the scripts
database which was obsoleted in 2.0_pre16 (in July 2009).
If GNU wget is available, then we want use it as it gives the user
possibility to set misc options, such as bandwidth limit in wgetrc.
It might also be that busybox is not available in case bootstrapping
alpine from other distros (think debian vserver host creating an
alpine guest)
In any case we, fall back to busybox wget in case wget libs are getting
upgraded. (see http://redmine.alpinelinux.org/issues/347)
Packages without architecture should get the platforms default
architecture (so we are backwards compatible). Only the virtual
packages should get 'noarch' by default. Also print full path
to the index file which failed.
Parse install_if from package metadata and include it in the
indexes. Also setup the reverse install_if dependencies when
loading a database. ref #443.
Actual install_if functionality is not yet implemented.
Just disable installation of packages using the new stuff. Also
flag lower case package info fields as non-critical and allow
installation even if that features is not supported.
When package is installed from commandline, we should always
install that specific instance of package (never favor repository
version if it has difference identity). Otherwise we might not
always end-up installing the .apk given on command line. The
dependency is now against specific checksum identity (marked
with >< dependency comparison). Fixes#492.
I think there used to be problems with using http_proxy on bb wget,
but those have been fixed for quite some time. This fixes#347.
We should probably use libcurl or similar library eventually, but
I'm not entirely sure if/when we want that.
The boundary callback should not happen until all the uncompressed
data has been consumed. This previously seems to have worked
because normally gzip library returns "no error" instead of the
"stream end" if we extract exactly the amount of bytes remaining
in the archive. (Perhaps this was changed in new zlib.) In any
case, verification was broken with some apks due to this callback
ordering issue.
- implement a hash table for commonly shared fields such as
license, version and architecture
- use macroes to print blobs or pkgname-pkgver strings
- fix some old cruft
The hash is 'unsigned long' everywhere which is 64-bits (on 64-bit
boxes). Fix the one instance of it being 'unsigned int' since that
would be 32-bit and cause lookup failures on 64-bit boxes.
We want remove dirs when they are unreferenced so we remove all dirs on
apk del, but we don't want remove dirs when closing database. So we make
removing dir optional when unreferencing it.
This partially reverts commit c7ffc96a16.
fixes#406
Make the libapk and lua module optional at buildtime and off by default.
This is because enabling libapk requires that mkinitfs also copies the
libapk.so or the initramfs will not work.
To build libapk:
make LIBAPK=yes
To build lua module (will enable LIBAPK=yes):
make LUAAPK
To link apk to libapk (will enable LIBAPK=yes):
make SHARED_LIBAPK=yes
All the above applies to make install target as well.
The triggers are read during apk_db_unpack_pkg(). If we delete the
triggers list after then unpack we delete the triggers which is not
what we want.
This fixes bug introduced in ce3cf8bff9
Instead of having a null pointer, use a dummy array which just
says the array is empty. This helps in multiple places of the code
which would otherwise need explicitly need to check first if the
array exists. This has been cause of multiple seg.faults in the
past as the array check is easily omitted.
This also removes (or fixes) all existing checks accordingly.
In addition to autocleaning dependencies on deletion, we need to
autoclean the old package on upgrade too. This is to make sure that
obsoleted dependencies (existed previously, but not in new package)
are removed where appropriate.
Doing "apk add file.apk" adds a versioned dependency to allow
downgrading, and sticky version. Most often this is to install
single packages from newer repository. So it would make sense
to reset them to non-versioned if doing:
apk upgrade -a
fixes#346
even if we have only one package as candidate, we need to check
it's availability. otherwise we can endup with bad changeset
referring to unavailable package.
to never ever overwrite a while in the filesystem the user knows
about. it gives the impression of extraction succeeding even though
nothing was done. this is inteded to be used only for bootstrapping
with overlay.
the old code treated a symlink to directory as file; it tried
to calculate regular has of it. fix this by: 1) using no follow
on migration and pruning stats, and 2) the helper function to
check if it's point to directory and not calculate hash in that
case. fixes#188.
this makes the database package entry smaller, and we propbably
get more fields to installed_package later too. this cleans up
the way scripts are stored and is a preparation for supporting
triggers. some parsing for trigger meta-data. ref #45.
it's a warning not an error to have non-existant repository.
bad signature means tampering; but we might want to just ignore
those. this is especially important if we have http repositories
and we are bootstrapping (without network).
previously they might have been skipped on certain situations.
this also fixes some other reverse dependency enforcements and
implements new "pending" state for locked name.
make the system audit compare only installed files, instead of
scanning the whole file system. also make it print only package
names with -q, so reinstalling modified packages becomes easy.
this way we never change cwd, and relative filenames are always
parsed consistently. this also helps filename construction in many
places. this patch also changes '--root' to override location of
all configuration to be in the new root. previously it depended
on the file which one was used.
apparently it needs to have both PROT_READ and PROT_WRITE. and
it needs to be MAP_SHARED for the writing to be effective. oh,
and the data needs to be preallocated with ftruncate; otherwise,
one gets SIGBUS.
Timo Teräs