Commit Graph

118 Commits (272784636112c84b57b3bd4742ab9f0235abe5b8)

Author SHA1 Message Date
Henrik Riomar 349c61c961 add support for pre and post commit hooks
This allows for instance integration of etckeeper

[TT: Reorganized code a bit, and modified to use single
directory commit_hooks.d with argument for script of stage.]
2017-02-15 13:44:04 +02:00
Timo Teräs cce4cff553 io, database: preserve [am]time for cached and fetched files
preserve [am]time for all packages and indexes. this fixes the caching
error that 'apk update' is after new index is generated, but before
the used mirror is synchronized. this caused local apkindex timestamp
to be newer than file in mirror, when in fact it was outdated index.

this also fixes fetched files to have build timestamp so that files
going to .iso or custom images have proper timestamps (rsync with
appropriate --modify-window now works)
2015-11-09 12:51:01 +02:00
Natanael Copa 707b51e0c6 io: fix posix_fallocate failure handling
We need fall back to a splice buffer if posix_fallocate call fails due
to file being a device (eg tty) or a pipe. This fixes apk fetch --stdout.
2015-10-08 11:42:23 +03:00
Timo Teräs 9ffa38222b io: use posix_fallocate to allocate disk space
ftruncate does not allocate it, and subsequent access to mmaped
file will result in SIGBUS. this fixes to properly report disk
full errors.
2015-09-03 13:15:18 +03:00
Timo Teräs 6b140ec5c7 fix bstream_from_fd to use mmap when available 2015-06-12 09:57:52 +03:00
Alex Dowad 4c3712ecb4 detect failures in writing to file during final flush of buffers
In practice this should fix to e.g. not wipe out /etc/apk/world if
final flush to /etc/apk/world.new fails.

This was prompted by an incident the other day where I ran the root
partition of an Alpine box out of space using 'apk add', and apk
helpfully wiped the contents of /etc/apk/world at the same time.

It might be tricky to try to reproduce exactly the same failure,
but from an examination of the code, setting 'rc' before the final
call to fdo_flush rather than after is one possible cause of this
behavior. (If the entire contents of /etc/apk/world.new are buffered,
and all get written out in the final fdo_flush call, and that call
fails, fdo_close will still happily rename /etc/apk/world.new to
/etc/apk/world.)
2015-05-26 08:38:45 +03:00
Alex Dowad be31eb24d8 io: fix compiler error by including stdint.h
According to the C standards, uint32_t is defined in stdint.h.
Presumably apk is usually built against C libraries where
stdint.h is indirectly included through another header file,
but this isn't the case with the version of glibc which I am using.
2015-04-22 10:50:31 +03:00
Timo Teräs 3490ff789a fix tee io error handling
use ERR_PTR mechanism, and handle it at all places.
2015-04-13 09:42:27 +03:00
Timo Teräs 60dd5798c9 use memmove for copying buffer leftovers, as the ranges may overlap
issue cought by fortify
2015-04-08 16:58:20 +03:00
Timo Teräs 9ed5a60dec unbreak audit after xattr support 2015-04-08 11:16:46 +03:00
Timo Teräs 83ab022301 audit xattrs
ref #3027
2015-04-08 10:27:49 +03:00
Timo Teräs 8d1ec4c5bc calculate and store checksum of xattrs
ref #3027
2015-03-11 16:10:33 +02:00
Timo Teräs da6e455f70 rename file info related functions for consistency 2015-03-10 15:47:13 +02:00
Timo Teräs be8e133c0b extract xattrs from packages
ref #3027
2015-03-10 14:38:06 +02:00
Timo Teräs 255fd81d79 rework error handling for write streams 2015-03-10 13:15:58 +02:00
Timo Teräs 2a6896b2b4 rework error handling for read streams 2015-03-10 13:15:31 +02:00
Timo Teräs c553970061 remove support for old database location in /var
the location changed in apk-tools 2.1.0 (March 2011) which was
used in Alpine Linux 2.2.
2015-01-30 14:35:02 +02:00
Timo Teräs 32627939f5 io,url,db: support for if-modified-since 2014-10-08 11:13:21 +03:00
Timo Teräs 1bbca16333 io: fix few error path leaks 2014-10-07 17:04:38 +03:00
Timo Teräs 26558bd126 io: use fget{pw,gr}ent_r only on uclibc and glibc
musl does not have those.
2013-07-04 10:36:47 +03:00
Timo Teräs 656190b0a0 io: get rid of APK_PROGRESS_SCALE
no need to muldiv all the time, just pass the current amount of bytes
done, and let callbacks use that directly.
2013-06-17 17:28:03 +03:00
Timo Teräs ade8d0b4e9 cache: implement progress bar (ref #1170) 2013-06-17 17:13:14 +03:00
Timo Teräs dbb642206d io: fix splice for copying unknown lengths 2013-06-17 14:23:14 +03:00
Timo Teräs 7392acb95e db: keep architecture in $ROOT/etc/apk/arch
This we use proper arch in case modifying chroot installation.
2012-02-23 17:04:51 +02:00
Timo Teräs ba3ee3f863 audit, db: add etc/apk/protected_files.d and new audit features
fixes #607.

audit is now mostly rewritten for the new functionality. And
has new features like --check-permissions, --recursive and
--packages.

$ROOT/etc/apk/protected_files.d/*.list can now contain additional
protected paths, one path per line:

+etc
@etc/init.d
-tmp

+ will include the directory as protected configuration directory.
@ considers the directory protected, but will backup only symlinks.
- removes any protection

lbu should be modified to put include and exclude paths in
etc/apk/protected_files.d/lbu.list. Additionally, some packages
might provide their own listings.

E.g. ssh might want to provide ssh.list with something like:
+root/.ssh
+home/*/.ssh
2012-02-23 15:05:06 +02:00
Timo Teräs 77e203bf32 db, io: load repositories also from etc/apk/repositories.d/*.list
Load additional repositories from $ROOT/etc/apk/repositories.d/*.list
unless --repositories-file is given as parameter.
2012-02-22 13:57:05 +02:00
Timo Teräs bf82e2e5fd db, solver, io: scan cache items at startup
It is faster to just scan the cache directory for existing packages
at startup than trying to faccessat() them on demand. It also makes
quite a few parts of the code more readable and simpler.
2012-02-22 08:45:40 +02:00
Timo Teräs 1817b0a3c3 all: update copyright year statement 2011-09-13 11:53:01 +03:00
Timo Teräs 93eb38a31a db: relocate from /var/lib/apk
move all files therein to other places. this allows /var to be
mounted from harddisk, but rest of system be run from ramdisk.

this also removes support for historical version of the scripts
database which was obsoleted in 2.0_pre16 (in July 2009).
2011-03-16 16:53:07 +02:00
Timo Teräs d92df52079 io: enhance istream/bstreams with pipe to forked child
* prunes the child pid to avoid zombies
 * handles the errors so e.g. file-not-found is reported properly
2010-12-09 10:47:09 +02:00
Timo Teräs 6398d338c6 db: fix the previous commit
id cache needs to be initialized early. and group database filename
is surprisingly 'group'.
2010-10-08 16:09:03 +03:00
Timo Teräs 19419f1a88 io, db: id cache should be specific to database root, not system root
Otherwise we end up using wrong uid/gid mappings when doing install
to alternate system root. Fixes #434.
2010-10-08 15:36:54 +03:00
Natanael Copa b4adf7645f Fix building on eglibc
Seems like recent eglibc requires that you include sys/stat.h
2010-08-30 13:04:25 +00:00
Timo Teräs 3062d681f9 archive: honor username/groupname instead of uid/gid
Take the uid/gid from passwd and group.
2010-06-12 13:43:29 +03:00
Timo Teräs 43cb554c3f various: use O_CLOEXEC and add some error checking 2010-06-11 13:42:21 +03:00
Timo Teras a7360395ea db: fix migration and pruning of symlinks to dirs
the old code treated a symlink to directory as file; it tried
to calculate regular has of it. fix this by: 1) using no follow
on migration and pruning stats, and 2) the helper function to
check if it's point to directory and not calculate hash in that
case. fixes #188.
2009-10-26 09:46:09 +02:00
Timo Teras dee6ffa492 io: better error handling when writing stuff out
also have the output stream support writing to temporary file
and do renameat/unlinkat on close depending on if all writes
succeeded or not.
2009-08-12 11:05:09 +03:00
Timo Teras addae04c26 db, audit: audit symlinks (by hash of the link target) 2009-08-11 19:02:22 +03:00
Timo Teras 46e9329568 io: flag for following symlinks on fstat
usually we are interested on the actual file's length. but
audit is interested about the link. so add a flag for this and
use it in audit.
2009-08-05 13:13:52 +03:00
Timo Teras ea90152664 apk: use *at instead of chdir+normal file syscall
this way we never change cwd, and relative filenames are always
parsed consistently. this also helps filename construction in many
places. this patch also changes '--root' to override location of
all configuration to be in the new root. previously it depended
on the file which one was used.
2009-07-31 16:08:09 +03:00
Timo Teras 67108bf07a io: fix corruption of big files on mmap write
remember to increment destination pointer; and munmap the proper
base address.
2009-07-31 10:50:55 +03:00
Timo Teras 5b48b85560 audit: protection mask for "symlinks only"
and use it for /etc/init.d by default. fixes #99.
2009-07-30 10:42:20 +03:00
Timo Teras 60c668f1dc io: keep static pointer to copying buffer
so we avoid some malloc/free calls.
2009-07-30 09:11:14 +03:00
Timo Teras 8e4075e6b1 io: fix mmap writing to actually work
apparently it needs to have both PROT_READ and PROT_WRITE. and
it needs to be MAP_SHARED for the writing to be effective. oh,
and the data needs to be preallocated with ftruncate; otherwise,
one gets SIGBUS.
2009-07-30 09:07:35 +03:00
Timo Teras 2ff4059b9a io: use mmap to write archive entries to disk
avoids some copying and system calls.
2009-07-29 19:33:52 +03:00
Timo Teras 9b77c053e8 pkg: cleanup the signing code
smaller callback and less cases to check. also reintroduce the
oneshot digest flag, hopefully correct this time.
2009-07-22 19:56:13 +03:00
Timo Teras 93f0b3524c various: more informative error messages 2009-07-22 14:56:27 +03:00
Timo Teras e30834fdcb digest: use oneshot context flag where approriate
speeds up digest calculation on some cases.
2009-07-17 15:56:09 +03:00
Timo Teras 6b471bb614 various: new style index generation
change the index generation to do old index, or the new style index
where package identity is sha1 of control block and it's contained
within an .tar.gz to allow signing in future.
2009-07-16 15:16:05 +03:00
Timo Teras 0f6d96a4f5 gzip: always autoclose the inner stream 2009-07-16 13:47:26 +03:00
Timo Teras 981bc118f8 db: live with sha1 and md5
this also convers scripts file to a tar archive.
2009-07-14 19:14:05 +03:00
Timo Teras 4562f44f9b bstream: make tokenizable and load index using bstream
some fixes on index reading code too.
2009-07-14 09:33:47 +03:00
Timo Teras e69b81f525 io: move csumming away from bstream to gunzip
in future we want to checksum on gzip boundary basis, not the
full file.
2009-07-13 20:37:03 +03:00
Timo Teras 9858ffff69 cache: new applet (ref #49)
Administrative tool to download or delete files to/from the cache.
2009-06-29 11:22:55 +03:00
Timo Teras 233918e518 db: cache packages (ref #49)
If /etc/apk/cache is a symlink to directory, a copy of all installed
packages is stored there, and the index of remote repositories will
be there instead of /var/lib/apk. This enables to reconstruct running
system during boot.

Left as todo: remove cached copy when the package is removed, and
additional apk applet to download missing packages to cache and/or
remove extra items.
2009-06-29 09:29:57 +03:00
Timo Teras 49c904c993 io: fix mmap bstream fd leak
We need to close the fd on destruction. This is what the corresponding
istream variant does too.
2009-06-11 13:03:10 +03:00
Timo Teras ec5b63c5e4 pkg: clean up writing of dependencies 2009-04-16 17:05:27 +03:00
Timo Teras 207bff6e6c io: implement write cache
speeds up writing of package / file database.
2009-03-17 14:17:35 +02:00
Timo Teras 052fbe3f86 various: make fancy progress bar and update todo 2009-01-07 21:45:11 +02:00
Timo Teras 5e2aca2678 io: fix some memory leaks 2009-01-06 21:23:26 +02:00
Timo Teras dc2ffc30e8 io: apk_ostream stuff 2008-11-28 16:28:54 +02:00
Timo Teras 4bafa681fe io: prepartions for url handling 2008-11-28 15:03:27 +02:00
Timo Teras a96572fba8 db: compressed index file 2008-11-27 21:06:45 +02:00
Timo Teras 1a7f3e3678 various: use apk_istream api 2008-11-27 16:59:04 +02:00
Timo Teras 8e23a2ba4e db: checksum installed files, protect config files
Checksum of installed is computed on the fly when extracting them
and it'll be saved to fdb. When installing config files those are
diverted with suffix .apk-new if earlier version of same file with
local changes exist.
2008-11-14 14:26:59 +02:00
Timo Teras 791cf6f05b pkg: speed up indexing of version 2 .apks 2008-11-07 19:18:53 +02:00
Timo Teras 0bf7a1a2cc io: implement mmap(2) for reading pkgs 2008-11-07 19:06:58 +02:00
Timo Teras aef0f036f0 use zlib internally to decompress 2008-11-07 17:11:08 +02:00