My Home Network Setup

March 30, 2022

My home network is rather unconventional. It is somewhat complicated, because I run my server as a firewall and my family has lots of devices to connect to it. I've chosen to make it much more complicated than most home networks, because I want to provide a level of security that protects my devices from rogue Apple and Google products. I also want to protect my server's management interface, and I don't want my virtual machines accessing anything but each other on the network.

Originally, this was just going to be more of an internal reference, but maybe my setup will be useful for someone else too. In this post, I'll document the way my home network is set up. I won't explain why I've done things the way I have too much, this is just to show how they are set up.

First of all, my firewall runs OpenBSD. It is an HP ProLiant DL380P, with 9 gigabit network interfaces---4 are on the motherboard, and 5 are provided via 3 PCI cards connected to the riser board. This handy diagram details how they are physically arranged, and how OpenBSD identifies them:

                            bge4

                     em0     em1

                     em2     em3

    bge3    bge2    bge1    bge0

I figured this out by repeatedly looking at the output of ifconfig(8) while plugging a switch into one port at a time, and seeing which one got a link. Here's what the physical ports are being used for:

    [bge3] The egress port; this is connected to the broadband
    modem.

    [bge2] The physical LAN for devices out of my control. My
    family has a WiFi access point connected here.

    [bge0] This HP server has a dedicated iLO port. I run a
    patch cable from that port into this one, so I can access
    iLO without having to physically connect to it each time.

    [bge4] I run Proxmox for testing and staging. My Proxmox
    machine is connected directly to this interface.

    [em1] My personal computers are hooked up to a switch that
    runs into this interface.

I have a 24-port switch that I used to have everything on, including iLO and Proxmox, but I wanted to provide a little more security for servers such as iLO and Proxmox that don't need to be accessed by my family. I want my OpenBSD firewall to sit between everything on my network, which is why I've added so many network interfaces to my server.

So that is the physical setup. I also have a few virtual network interfaces:

    [vether0] Used to route traffic from my virtual machines
    running on the server itself through vmd(8). This is connected
    to a virtual switch, bridge0.

    [enc0] Used to route VPN traffic. iked(8) automatically
    encapsulates traffic sent to and from this interface, so
    even though VPN traffic is encrypted when it arrives and
    leaves the firewall, pf(4) can filter it when it is
    unencrypted.

I'm giving each interface a 10.x.0.0/16 subnet, where x is just an arbitrary counter starting at 0. I previously made the mistake of using a different private network for each interface, and I'd allocate the entire range to it, which meant a lot of addresses were wasted and things were rather messy. But now, everything resides in 10.0.0.0/8, each interface just gets a different subnet. So here's how the addresses are allocated:

    [10.0.0.0/16] The physical LAN that my family uses with
    their WiFi access point. dhcpd(8) serves this range.

    [10.1.0.0/16] Used for the VPN. iked(8) manages the address
    allocation within this range.

    [10.2.0.0/16] Proxmox uses this subnet. Proxmox works in
    bridge mode by default, so all virtual machines are connected
    as if they were physically attached to the same switch as
    the Proxmox host.  This allows me to use dhcpd(8) on the
    firewall to issue addresses within this range.

    [10.3.0.0/16] The virtual machines on the firewall use this
    subnet.  It is serviced by dhcpd(8).

    [10.4.0.0/16] A physical LAN for my own personal devices
    that are isolated from the rest of the network. This allows
    me to securely use FTP to my server and other plain-text
    protocols without leaking data to the rest of the network.
    Only devices I trust (otherwise known as only devices that
    run OpenBSD) are allowed on this network.

    [10.254.254.0/30] iLO is set up to use a static IP address
    and netmask. I gave it a subnet that is big enough for
    exactly two hosts, and made sure it was well out of the way
    of everything else. The firewall gets 10.254.254.1, and iLO
    has 10.254.254.2. In the event that the firewall becomes
    unavailable and I need access to iLO, I can simply unplug
    the patch cable that connects the iLO port to bge0, and
    plug in a laptop configured with a static IP and netmask.

Everything I need to make this network setup work is available in OpenBSD:

    hostname.if(5), dhcpd(8), unbound(8), pf.conf(5)

To make things a bit nicer, I'll also be using these:

    relayd(8), tftpd(8)

The virtual machines that run on the firewall utilize these:

    mountd(8), nfsd(8), portmap(8), vmd(8)

So there are a lot of things to configure here. At some point, I'll write a script to automate the setup of all of this, but until then, everything is configured and managed by hand.

I'm not going to document exactly how all of this translates into OpenBSD configuration files—that job is best left to the man pages, which are very thorough and detailed. This is more of just a reference for me if I ever get confused as to what's where on my home network.