84 lines
6.5 KiB
Markdown
84 lines
6.5 KiB
Markdown
|
---
|
|||
|
|
title: "Efficient service isolation on Alpine with VRFs"
|
|||
|
|
date: "2021-09-13"
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
Over the weekend, a reader of my blog contacted me basically asking about firewalls. Firewalls themselves are boring in my opinion, so let’s talk about something Alpine can do that, as far as I know, no other distribution can easily do out of the box yet: service isolation using the base networking stack itself instead of netfilter.
|
|||
|
|
|
|||
|
|
## A note on netfilter
|
|||
|
|
|
|||
|
|
Linux comes with a powerful network filtering framework, called [netfilter](https://www.netfilter.org). In most cases, netfilter is performant enough to be used for packet filtering, and the newer `nftables` project provides a mostly user friendly interface to the system, by comparison to the older `iptables` and ip6tables commands.
|
|||
|
|
|
|||
|
|
However, when trying to isolate individual services, a firewall is sometimes more complicated to use, especially when it concerns services which connect outbound. Netfilter does provide hooks that allow for matching via `cgroup` or `PID`, but these are complicated to use and carry a significant performance penalty.
|
|||
|
|
|
|||
|
|
## seccomp and network namespaces
|
|||
|
|
|
|||
|
|
Two other frequently cited options are to use seccomp and network namespaces for this task. Seccomp is a natural solution to consider, but again has significant overhead, since all syscalls must be audited by the attached seccomp handler. Although seccomp handlers are eBPF programs and may be JIT compiled, the performance cost isn’t zero.
|
|||
|
|
|
|||
|
|
Similarly, one may find network namespaces to be a solution here. And indeed, network namespaces are a very powerful tool. But because of the flexibility afforded, network namespaces also require a lot of effort to set up. Importantly though, network namespaces allow for the use of an alternate routing table, one that can be restricted to say, the management LAN.
|
|||
|
|
|
|||
|
|
## Introducing VRFs
|
|||
|
|
|
|||
|
|
Any network engineer with experience will surely be aware of VRFs. The VRF name is an acronym which stands for virtual routing and forwarding. On a router, these are interfaces that, when packets are forwarded to them, use an alternative routing table for finding the next destination. In that way, they are similar to Linux’s network namespaces, but are a lot simpler.
|
|||
|
|
|
|||
|
|
Thanks to the work of Cumulus Networks, Linux [gained support for VRF interfaces in Linux 4.3](https://lwn.net/Articles/632522/). And since Alpine 3.13, we have supported managing VRFs and binding services to them, primarily for the purpose of low-cost service isolation. Let’s look at an example.
|
|||
|
|
|
|||
|
|
## Setting **up** the VRF
|
|||
|
|
|
|||
|
|
On our example server, we will have a management LAN of `10.20.30.0/24`. A gateway will exist at `10.20.30.1` as expected. The server itself will have an IP of `10.20.30.40`. We will a single VRF, in conjunction with the system’s default route table.
|
|||
|
|
|
|||
|
|
### Installing the needed tools
|
|||
|
|
|
|||
|
|
By default, Alpine comes with Busybox’s iproute2 implementation. While good for basic networking use cases, it is recommended to install the real iproute2 for production servers. To use VRFs, you will need to install the real iproute2, using `apk add iproute2-minimal`, which will cause the corresponding ifupdown-ng modules to be installed as well.
|
|||
|
|
|
|||
|
|
### Configuring `/etc/network/interfaces`
|
|||
|
|
|
|||
|
|
We will assume the server’s ethernet port is the venerable `eth0` interface in Alpine. First, we will want to set up the interface itself and it’s default route. If you’ve used the Alpine installer, this part should already be done, but we will include the configuration snippet for those following along.
|
|||
|
|
|
|||
|
|
auto eth0
|
|||
|
|
iface eth0
|
|||
|
|
address 10.20.30.40/24
|
|||
|
|
gateway 10.20.30.1
|
|||
|
|
|
|||
|
|
The next step is to configure a VRF. In this case, we want to limit the network to just the management LAN, `10.20.30.0/24`. At the moment, ifupdown-ng does not support configuring interface-specific routes out of the box, but it’s coming in the next version. Accordingly, we will use iproute2 directly with a `post-up` directive.
|
|||
|
|
|
|||
|
|
auto vrf-management
|
|||
|
|
iface vrf-management
|
|||
|
|
requires eth0
|
|||
|
|
vrf-table 1
|
|||
|
|
pre-up ip -4 rule add pref 32765 table local
|
|||
|
|
pre-up ip -6 rule add pref 32765 table local
|
|||
|
|
pre-up ip -4 rule del pref 0
|
|||
|
|
pre-up ip -6 rule del pref 0
|
|||
|
|
pre-up ip -4 rule add pref 2000 l3mdev unreachable
|
|||
|
|
post-up ip route add 10.20.30.0/24 dev eth0 table 1
|
|||
|
|
|
|||
|
|
This does four things: first it creates the management VRF, `vrf-management` using the second kernel route table (each network namespace may have up to 4,096 routing tables). It also asserts that the `eth0` interface must be present and configured before the VRF is configured. Next, it removes the default route lookup rules and moves them so that the VRFs will be checked first. Finally, it then adds a route defining that the management LAN can be accessed through `eth0`. This allows egress packets to make their way back to clients on the management LAN.
|
|||
|
|
|
|||
|
|
In future versions of ifupdown-ng, the routing rule setup will be handled automatically.
|
|||
|
|
|
|||
|
|
### Verifying the VRF works as expected
|
|||
|
|
|
|||
|
|
Once a VRF is configured, you can use the ip vrf exec command to run a program in the specified VRF context. In our case, the management VRF lacks a default route, so we should be able to observe a failure trying to ping hosts outside the management VRF, using `ip vrf exec vrf-management ping 8.8.8.8` for example:
|
|||
|
|
|
|||
|
|
localhost:~# ping 8.8.8.8
|
|||
|
|
PING 8.8.8.8 (8.8.8.8): 56 data bytes
|
|||
|
|
64 bytes from 8.8.8.8: seq=0 ttl=121 time=1.287 ms
|
|||
|
|
^C
|
|||
|
|
\--- 8.8.8.8 ping statistics ---
|
|||
|
|
1 packets transmitted, 1 packets received, 0% packet loss
|
|||
|
|
round-trip min/avg/max = 1.287/1.287/1.287 ms
|
|||
|
|
localhost:~# ip vrf exec vrf-management ping 8.8.8.8
|
|||
|
|
PING 8.8.8.8 (8.8.8.8): 56 data bytes
|
|||
|
|
ping: sendto: Network unreachable
|
|||
|
|
|
|||
|
|
Success! Now we know that using `ip vrf exec`, we can launch services with an alternate routing table.
|
|||
|
|
|
|||
|
|
## Integration with OpenRC
|
|||
|
|
|
|||
|
|
Alpine presently uses OpenRC as its service manager, [although we plan to switch to s6-rc in the future](https://ariadne.space/2021/03/25/lets-build-a-new-service-manager-for-alpine/). Our branch of OpenRC has support for using VRFs, for the declarative units. In any of those files, you just add `vrf=vrf-management`, to the appropriate `/etc/conf.d` file, for example `/etc/conf.d/sshd` for the SSH daemon.
|
|||
|
|
|
|||
|
|
For services which have not been converted to use the declarative format, you will need to patch them to use `ip vrf exec` by hand. In most cases, all you should need to do is use `${RC_VRF_EXEC}` in the appropriate place.
|
|||
|
|
|
|||
|
|
As for performance, this is _much_ more efficient than depending on netfilter, although the setup process is not as clean as I'd like it to be.
|