---
title: "Bits related to Alpine Security Initiatives in September"
date: "2021-10-01"
---

The past month has been quite busy as we prepare to wrap up major security-related initiatives for the Alpine 3.15 release. Some progress has been made on long-term initiatives as well.

## OpenSSL 3 migration

As I noted in [my last status update](https://ariadne.space/2021/09/07/bits-relating-to-alpine-security-initiatives-in-august/), we began the process to migrate the distribution to using OpenSSL 3. As a result of this, we have found and mitigated a few interesting bugs, for example, [`ca-certificates-bundle` was being pulled into the base system as a side effect](https://git.alpinelinux.org/aports/commit/main/apk-tools/APKBUILD?id=1ac3c1bb29eeff083c621cf6b27ad12ab93cb73a) rather than intentionally, despite apk-tools explicitly needing it for validating the TLS certificates used by Fastly for our CDN.

Migrating to OpenSSL 3 has not been without its share of difficulties however, as I noted in [a blog post earlier in the month discussing some of these difficulties](https://ariadne.space/2021/09/16/the-long-term-consequences-of-maintainers-actions/). I hope to be able to fully finish the OpenSSL 3 migration during the Alpine 3.16 development cycle as the last remaining packages such as `mariadb` and `php` make releases which support the new API. One other major issue needing to be addressed is updating `wpa_supplicant` and `hostap` to use the new OpenSSL APIs, but WPA requires the use of RC4 which has been moved to the legacy provider, so this will [require programmatic loading of the legacy OpenSSL provider](https://gitlab.alpinelinux.org/alpine/aports/-/issues/13043). Accordingly, we moved it back to OpenSSL 1.1 for now until upstream releases an update to address these problems.

OpenSSL 3 also introduces some unintended regressions. Specifically, a bug was reported against `apk-tools` where using `apk --allow-untrusted` would result in a crash. After some debugging work, [I was able to reduce the issue to a simple reproducer: the `EVP_md_null` digest family was never updated to be compatible with the new OpenSSL 3 provider](https://github.com/openssl/openssl/issues/16660) APIs, and so attempting to use it results in a crash, as the prerequisite function pointers never get set up on the `EVP_MD_CTX` context. This means that `apk-tools` is still using OpenSSL 1.1 for now, despite otherwise working with OpenSSL 3.

Coordinating the OpenSSL 3 migration consumed a lot of my time in September, for example, [I spent a few days investigating OpenSSL 3 regressions](https://gitlab.alpinelinux.org/alpine/aports/-/issues/13016) on CPUs which claim to be Pentium-compatible but actually lack support for the `lock cmpxchg8b` instruction, and CPUs which claim to be Pentium 3-compatible, but lack the `CMOV` instructions. This investigation was quite frustrating, but also interesting, as the Vortex86DX3+ looks like a nice x86 CPU that is not vulnerable to Meltdown or Spectre due to not supporting speculation.

## Rust in Alpine `main` for 3.16

As I noted in my previous blog about the OpenSSL 3 transition, we had to migrate Ansible from `main` to `community` due to Rust presently being in the `community` repository. This is largely because `main` has a general policy that is similar to other distributions: once we cut a new release series, we generally don't do upgrades of packages in `main`, instead preferring to backport security and reliability fixes, unless we are certain they won't cause regressions. Normally, this is a good thing, as new versions of software frequently bring ABI/API changes that are not backwards compatible, and upstream developers sometimes forget to update their `SONAME` versions to reflect those changes.

Distributions traditionally have to provide a maintenance lifecycle which is several years long without breaking their users' environments, and so tend to be conservative in what post-release updates are made. Alpine takes a more "hybrid" approach and thus has more flexibility, but we still prefer to err on the side of caution. In the case of Rust, this meant that we wanted a working relationship that allowed us to have a high level of confidence in the Rust toolchains we were delivering to our users.

After almost a year of ongoing discussion, [and a Rust MCP](https://github.com/rust-lang/compiler-team/issues/422), we have [come up with a plan in concert with Rust upstream](https://gitlab.alpinelinux.org/alpine/tsc/-/issues/21) which allows us to deliver production-quality Rust toolchains for Alpine users, and keep them continuously updated in Alpine releases. I expect the Alpine TSC to approve this for Alpine 3.16 and later. And yes, for other languages we are willing to offer a similar proposal, if there is a proven track record of maintaining backward compatibility and portability. Please feel free to reach out.

The more interesting development is that this allows for using components written in Rust for the base system in the future. While we do not plan to start evaluating these options until the 3.18 release cycle at the earliest, this does open a path to enabling `rustls` and `hyper` to replace OpenSSL and `libfetch` in `apk-tools` at some point, which could potentially be an interesting development. It also opens a path for components of `apk-tools` to eventually be written in Rust as well.

## Distroless images

[Another project](https://github.com/witchery-project/witchery) I [started recently is Witchery](https://ariadne.space/2021/09/09/introducing-witchery-tools-for-building-distroless-images-with-alpine/), a build system for generating distroless images using Alpine components. This allows a user to easily build a distroless image for their application by leveraging `apk-tools` to do the work. [Distroless images are interesting from a security perspective as they contain less moving parts](https://github.com/GoogleContainerTools/distroless), in most cases, a distroless image built with Witchery will only contain `musl`, some data files in `/etc` and your application. By avoiding other components like `busybox` and `apk-tools`, the attack surface of an image is reduced, as there is nothing available outside your application payload for an attacker to use.

There is still a lot of work to do on Witchery, as it is presently in the proof of concept stage, and I plan on doing a second blog post about it soon. I believe that there is intrinsic value to deploying applications built against `musl` from a security point of view over `glibc`, as there is much more hardening in `musl`.

## Acknowledgement

My activities relating to Alpine security work are presently sponsored by Google and the Linux Foundation. Without their support, I would not be able to work on security full time in Alpine, so thanks!
