Setting up an isolated work VLAN with VyOS

I treat employer-provided hardware as hostile entities on my network. I have no control over them and I have no idea what invisible scanning software they've installed.

Previous to now I set up a dedicated guest wi-fi network which provides a sort of client isolation, but this wasn't quite enough because I want my work machine to also have an ethernet connection. Therefore, I went about setting up a VLAN for the first time.

Topology

My network is unfortunately somewhat complicated due to how my house is laid out. The path from my workstation to my WAN modem goes like this:

1. TP-Link Omada access point
2. TP-Link standalone L2+ switch
3. UniFi Flex switch
4. TP-Link Omada switch
5. VyOS running on an old HP SFF

I set up the VLAN on Omada, UniFi, and the standalone switch as a tagged VLAN on every port so that it could pass through unperturbed to VyOS. This was probably somewhat aggressive but I don't see any harm for my situation. Plus, I don't have my uplink ports written down anywhere and I was lazy.

The way this works in Omada and UniFi is identical, you just add a new LAN network of type VLAN, set the tag, and then ensure that every port is set to the automatically generated All port profile. That profile just sets the new VLAN as tagged (allowed to pass). On Omada I also set the work-specific guest network to have the correct VLAN tag.

On the standalone switch it was a little more manual, in so far as I defined the VLAN and then manually selected every port to be tagged, but overall it was similar.

VyOS

This is where it gets complicated. I'm running VyOS 1.4.x rolling releases and, as is the nature with a rolling release model where you just run the tip of the development branch, there's sometimes bugs.

For the longest time, though, I thought I was holding something wrong. Here's my bridge config:

bridge br0 {
    address 10.73.95.1/24
    enable-vlan
    member {
        interface eth0 {
            allowed-vlan 74
            native-vlan 1
        }
        interface eth2 {
            allowed-vlan 74
            native-vlan 1
        }
    }
    stp
    vif 74 {
        address 10.74.1.1/24
        firewall {
            in {
                name WORK-OUT
            }
        }
    }
}

My VyOS box, for purposes of this blog post, has three ethernet interfaces: one WAN and two LAN. I have the LAN interfaces set up with a Linux bridge, which is basically a network switch in software.

Both member interfaces of the bridge have native-vlan 1, which means that packets coming in without a VLAN tag are automatically assigned VLAN 1 and packets leaving with the VLAN 1 tag will have it stripped on the way out.

Both member interfaces also have allowed-vlan 74, which just means that packets with that tag are allowed to pass through those interfaces.

Then I have vif 74 which sets up a sub-interface on the bridge specific to VLAN 74. That sub-bridge only deals with packets on VLAN 74 and ignores everything else. It has an address in a different subnet along with a firewall that I'll get to in a bit.

This all worked, to a point, but when I tried to do anything on a client machine attached to VLAN 74 I couldn't get anywhere. Every packet said "no route to host".

After a bunch of time with tcpdump I noticed something odd: ARP requests for 10.74.1.1 would get all the way to vif 74, which would dutifully issue an ARP reply, which would then be visible on br0 but not on either one of the ethernet interfaces and would never make it back to the client.

This is extremely weird behavior!

I asked for help on reddit, twitter, various Slacks, but then I remembered that VyOS has a slack and asked there. After a bit of debugging one of the VyOS developers noticed the problem: br0 wasn't getting VLAN 74 set as allowed by the bridge management code within VyOS:

vyos@vyos:~$ sudo bridge -c vlan show
port              vlan-id
eth2              1 PVID Egress Untagged
                  74
eth0              1 PVID Egress Untagged
                  74
br0               1 PVID Egress Untagged

The magic command to fix it is:

vyos@vyos:~$ sudo bridge vlan add vid 74 dev br0 self

and then when we run the show again:

vyos@vyos:~$ sudo bridge -c vlan show
port              vlan-id
eth2              1 PVID Egress Untagged
                  74
eth0              1 PVID Egress Untagged
                  74
br0               1 PVID Egress Untagged
                  74

As soon as I ran that add command everything started working great. I've added it to /config/scripts/vyos-postconfig-bootup.script and I trust that the VyOS development group will get around to fixing the bug at some point in the near future.

Firewall

Now that I had the VLAN set up I could access the internet from it, but also I could access the rest of my home network. VyOS assumes unless told otherwise that it should route packets as much as possible. You can use firewall rules to restrict that, however, so here's what that WORK-OUT rule looks like:

firewall {
    name WORK-OUT {
        default-action accept
        rule 10 {
            action drop
            destination {
                address 10.0.0.0/8
            }
        }
    }
}

It's really simple: drop everything that is destined for an address in the 10.0.0.0/8 subnet and accept everything else. This is maximally restrictive so that I can expand my network with other subnets and VLANs and not have to worry about updating this rule.