wiki/projects/alamesh/setting_up_bird.md

Setting up BIRD

BIRD, or the "BIRD Internet Routing Daemon", is a daemon that turns a dumb linux box into a fully fledged router that can support many different routing protocols. It speaks those protocols with other routers and builds a dynamic routing table that is then sent to the kernel (which does the actual packet forwarding).

This doc tracks my learnings from installing and setting up bird on a cheap, $4/month DO droplet. Actually using it with another router will come in an edit or separate post.

My Network

I'm not trying to to any sort of dynamic routing yet, but here's what I have working right now. My current network is the entire 0200::/7 address space (all addresses prefixed with 0x0(2|3)).

┌──────────┐    wg tunnel
│ hyperion ◄───────────────┐
│  200::3  │               │
└──────────┘               │
                  ┌────────▼────────────┐
                  │ router running bird │
                  │       200::1        │
┌─────────┐       └────────▲────────────┘
│  lykos  │                │
│ 200::55 ◄────────────────┘
└─────────┘     wg tunnel

Installing bird

Ubuntu's apt repo only has version 1.6.8 and I think we want the newer 2.0.10. This means compiling it from source which wasn't too bad.

First we need some dependencies:

sudo apt install gcc flex bison make m4 libncurses5-dev libncursesw5-dev libreadline6-dev

Actually compiling it:

curl -LO https://bird.network.cz/download/bird-2.0.10.tar.gz
tar -xvf bird-2.0.10.tar.gz
cd bird
./configure
make
make install
bird --version

I moved my bin to /usr/local/bin/bird after having some weird env issues.

Now we need to create a user with CAP_NET_* privileges that bird can run as. We actually don't need to do that immediately. We can use -u to drop to a user like nobody and bird will only give itself the privileges that it needs. We may want a specific bird user in the future.

Configuration

Bird is configured with a pretty simple file (or list of files). You can use the global /etc/bird/bird.conf but I've chosen to use a local config file under ~/.bird/bird.conf. You can give bird the path to the config file with -c.

dn42 has a great guide on using bird (with peers!). I'm going to copy the configuration structure from that page here and attempt to add more commentary.

The guide requires a few pieces of information that we'll need to discuss:

• <AS>: your Autonomous System Number
• <GATEWAY_IP>: your gateway ip, the internal 0200::/7 address that the router will run on.
• <SUBNET>: whatever subnet you control. Such as 0200:1234::/16.
• <PEER_IP>: the ip of a peer connected to you.
• <PEER_AS>: the AS number of the peer.
• <PEER_NAME>: what you want to call that peer.

Most of this information should be located in (and authenticated through) a git repo. If we do it with git, then updating bird should simply be a git pull and then running a cli tool to generate the config and reload bird. We need to find a balance between magic and understanding of what's happening. Maybe a tool that automates it but describes what's going on is enough?

Now for the actual configuration. We're going to be using IPv6 only. If we did v4 we would need a separate set of rules files, subnets, ips, etc.

The first file is the root bird.conf. This is what we point bird to when starting the daemon. I will try to comment every line and block with my understanding of what it means.

The bird conf file format is a structured tree of blocks with some simple syntax. A protocol is the base unit of work in bird. Protocols describe how to interact with the kernel, BGP, static route, etc.

bird.conf

# ==== CONSTANTS ====

define OWNAS = <OWNAS>; # your autonomous system number

# the first non-zero IPv4 you control. Since we're only using
# IPv6. Use the first non-zero 32 bits you control. E.g., if
# you control 255::/16, use 2.85.0.1 (255 is two hex bytes,
# 0x02, and 0x55 which is 2.85 in a v4 addr).
define OWNIP = <OWNIP>;

# the router's actual IPv6 addr that it can be reached by.
define OWNIPv6 = <OWNIPv6>;

# the subnet you control
define OWNNETv6 = <OWNNET>;

# set of all addrs you control.
define OWNNETSETv6 = [<OWNNET>+];

# ===================

router id OWNIP;

# "everything" is a protocol in bird. The `device` protocol is not really
# a protocol, it just instructs bird to ask the kernel for information
# on devices (interfaces).
protocol device {
  # scan devices every 10 seconds.
  scan time 10;
}

# ==== UTILITY FUNCTIONS ====

function is_self_net() {
  return net ~ OWNNETSETv6;
}

function is_valid_network() {
  return net ~ [
    0200::/7+
  ];
}

# Tell bird how to interact with the kernel (modify routes):
#
# Again, not really a protocol but ¯\_(ツ)_/¯
protocol kernel {
  # scan the routing table every 20 seconds (update bird state).
  scan time 20;

  # configure ipv6
  ipv6 {
    # Don't import any routes from the kernel, let bird figure out
    # what routes to add or change. `none` is a "filter" that discards
    # routes that are given to it. The kernel gives bird routes and we
    # want bird to ignore them. kernel -> bird.
    import none;

    # We write an inline filter that describes what routes to give the
    # kernel from bird. bird -> kernel.
    export filter {
      # if the source of this route is static configuration, then we
      # don't want to give it to the kernel. (only want dynamic routes).
      # I don't know why this is important? Is that right?
      if source = RTS_STATIC then reject;
      # set the `krt_prefsrc` attribute for this route.
      # see comment above above the kernel block.
      krt_prefsrc = OWNIPv6;
      # accept this route (don't filter it out).
      accept;
    };
  };
}

# Static protocol lets you define static routes that do not change.
protocol static {
  # reject any routes in our subnet (I think?)
  route OWNNETv6 reject;

  ipv6 {
    # tell bird about everything. static -> bird.
    import all;
    # bird can't configure static routes (default)?.
    # bird -> static.
    export none;
  };
}

# Create a template for the `bgp` protocol. This is where the real
# magic happens. We use a template since every peer will essentially
# have the same BGP configuration. The template name is `alapeers`.
#
# Each peer gets its own template nad file under `peers6/*``.
#
# We're using BGP in "exterior" or `eBGP` mode. This means we're using
# the protocol to define routes between Autonomous Systems, not really
# between individual IP addrs.
#
# Each instance of the `bgp` protocol corresponds with one neighboring
# router, or in our case, a peer.
template bgp alapeers {
  # our `as` number is the OWNAS constant. This is defined in the local
  # defs file that comes later.
  local as OWNAS;
  # compare path lengths to determine which route is the best one.
  path metric 1;

  ipv6 {
    # only accept routes from peers that follow this inline filter.
    # bgp -> bird.
    import filter {
      # Accept the route if its a valid network and is not our
      # own network. (fns defined in local defs file).
      if is_valid_network() && !is_self_net() then {
        accept;
      }
      reject;
    };
    # only send routes that follow this inline filter.
    # bird -> bgp (other peers).
    export filter {
      # Only send if it's a valid netowrk and the route comes from
      # static configuration or from BGP.
      if is_valid_network() && source ~ [RTS_STATIC, RTS_BGP] then {
        accept;
      }
      reject;
    };
    # Only allow for 1000 total routes to be imported. Once we hit
    # 1000, block further routes.
    import limit 1000 action block;
  };
}

# Include all of our peers!
include "peers/*";

Whew, that was a rather larger file but that's pretty much it. Now to configure a peer.

peers/*.conf

Peer configuration is super simple with the template we defined above:

protocol bgp <PEER_NAME> from alapeers {
  neighbor <PEERING_IP> as <PEER_AS>; # as == autonomous system
}

Done! Adding a new peer just means adding a new file in the peers directory.

ROA Tables

ROA == Route Origin Authorization. This is basically a central list of who owns what routes and is allowed to send changes. There's some crypto stuff you can do too, but that's only really important if we start letting arbitrary people join.

There is a concern that one of us messes up the config and starts sending out bad routes, but we'll cross that bridge when we get there (large outage lol). ROA tables might help prevent that.

What's Next

Now I need to actually test this with another router. I'll most likely setup another VPS or ask someone else to spin something up.

I need to determine a fake AS number, subnets, etc, and then I'll report back if it works or not. To test the bird configuration, I plan to connect lykos to a different router running on a different subnet, and then ssh into hyperion from lykos. The routes should "just work" (right lol).

(edit) Testing Bird with Two Routers

I have two DO droplets that will run bird. Here's how I'm setting up the network.

OG Router (OGR):

• AS: AS550001
• router id: 2.0.0.1
• GATEWAY_IP: 0200::1
• SUBNET: 0200::/16
• Internal WG Interface: ohea0

Test Router (TR):

• AS: AS550002
• router id: 2.85.0.1
• GATEWAY_IP: 0255::1
• SUBNET: 0255::/16
• Internal WG Interface: alanet0

I'm keeping hyperion connected to OGR with address 0200::3. Lykos will connect to TR with address 0255::2. I've setup wireguard (wg) on my two clients, hyperion and lykos, to allow all IPs from 0200::/7.

Things started getting a little hairy here. wg-quick automatically sets up the routing table which interferes with bird, so we need to disable it with Table = off in our WG configuration. I also enabled PersistentKeepalive with my two routers in order to confirm that a tunnel could actually be created. Since I disabled the wg routing table, I can't seem to ping one router from the other. I believe this is because there is nothing telling the kernel to route arbitrary 200::/7 packets to the wg tunnel and from the wg tunnel to the router I'm trying to ping. I'm guessing bird will set this up for me when I add a peer.

Adding bird to each router

I've put the bird configuration for both routers under the bird_example_conf dir right next to this article. Inside of that folder you'll find an ogr folder for the OG Router, tr folder for the test router, and template folder for the template files above.

Updating a server conf:

rsync -a ogr/* router:.bird

First Snag, the Router ID

The router id in local.conf needs to be an IPv4 address (ugh). I'm just going to use my router's public ipv4 addr and call it good.

Second Snag, Syntax

Syntax errors everywhere.

Third Snag, wrong bird version

FML. The guide from dn42 was for bird 1.6.3, and I'm running bird 2.0.10. dn42 has a bird2 guide which I'm using to modify the configuration that I wrote about above. This has led to more syntax errors...

Some of the errors have been rather helpful though!

bird: bird.conf:15:28 Invalid IPv6 prefix 200::1/16, maybe you wanted 200::/16

Running bird

After fixing those errors and changing the conf file, here's the command I'm using to run bird:

bird -d -c bird.conf

-d for debug messages and for running in the foreground and -c to point to my local config file, .bird/bird.conf.

Apparently ubuntu has a bird2 package that's bird version 2.0.9-3. I'll attempt to use that for the test-router.

I have bird running on both routers now. Now the real debugging begins.

Further wg configuration

After reading dn42's guide on wireguard I believe we need tell the interface that it's point-to-point and split up the config.

Here's my wireguard setup for ogr:

I now have two wireguard config files, ohea0.conf and peer-tr.conf. ohea0.conf has all of my configuration for my local devices and lets lykos ssh into hyperion through ogr. It's a standard wg configuration. The key here is that I'm listening on port 50005. peer-tr.conf is an interface just for peering with the test-router. I expect that wg configuration to be copy-pasted for each peer, albeit with modified ListenPorts.

Internal Network: ohea0.conf

[Interface]
PrivateKey = (redacted)
Address = 200::1/16
ListenPort = 50005

[Peer]
PublicKey = ZKR6n8/IersOc1SKoUyU6Z3/q/jPeZs5ljf4hg04pA0=
AllowedIPs = 0200::3/128

BGP Peer: peer-tr.conf

[Interface]
PrivateKey = 
# Different listen port for peers!
ListenPort = 50004
# Make this link a point-to-point connection.
# Note ogr's ip 200::1 and the peer's (tr's) ip 255::1 
PostUp = /sbin/ip addr add dev %i 200::1/128 peer 255::1/128
# Don't modify any routes
Table = off

[Peer]
PublicKey = C6RPH/RMqZ9u0ot6XN8ZcS+vTtU2zo9ZD7+wCfPzHxg=
AllowedIPs = 0200::/7
Endpoint = 46.101.144.104:50004
PersistentKeepalive = 25

The config for tr is essentially the same, just flipping the ips and keys.

ogr can now ping tr and we're now split on multiple interfaces.

Debugging BGP

After making the peer wg config point-to-point, ogr's and tr's birds starting speaking BGP with eachother. I used the bird client birdc to see if anything changed. birdc connects to the currently running bird process over the control socket created when bird starts.

Run show protocols to get the list of configured protocols (blocks in the config file). Here's what ogr's protocols look like:

bird> show protocols
Name       Proto      Table      State  Since         Info
device1    Device     ---        up     09:10:13.122
kernel1    Kernel     master6    up     09:10:13.122
static1    Static     master6    up     09:10:13.122
peer_test_router BGP        ---        up     09:10:22.257  Established

And show protocols peer_test_router gives much more info on BGP that's running. I saw in that output that some routes were added/seen by bird.

Here's show route stats:

bird> show route stats
Table master4:
0 of 0 routes for 0 networks in table master4

Table master6:
200::/16             unreachable [static1 09:10:13.122] * (200)
255::/16             unicast [peer_test_router 09:10:23.124] * (100) [AS550002i]
	via 255::1 on peer-tr
2 of 2 routes for 2 networks in table master6

Total: 2 of 2 routes for 2 networks in 2 tables

Seeing 255::/16 in there is promising, since that's the subnet that the test-router controls (comes from AS550002).

Let's try it out!

IT FUCKING WORKS

I ssh'ed into hyperion FROM lykos! This is pretty fuckin cool. I feel like some sort of internet god.

Here's my final topology:

┌───────────┐                ┌───────────┐
│    ogr    │peer-tr         │    tr     │
│ 200::1/16 ◄────────────────► 255::1/16 │
│  AS55001  │        peer-ogr│  AS55002  │
└────▲──────┘                └────▲──────┘
     │                            │
     │ohea0                alanet0│
     │                            │
     │                            │
┌────▼─────┐                 ┌────▼───┐
│ hyperion │                 │ lykos  │
│  200::3  │                 │ 255::2 │
└──────────┘                 └────────┘

So in order to ssh from lykos, tr needed to know that it could send 200::/16 packets to ogr. Remember that the wg link from tr <-> ogr is only point-to-point, there was no concept of a subnet. That means routing info for ogr must have been set by bird in tr's routing table, which allowed the IP/TCP packets to be forwarded to ogr and then to hyperion.

Conclusion

That was a grind. I think like 99% of it can be automated with a nice tool and config file, at least all of the bird configuration. Auto configuring WG is also pretty doable.

We still need to test this with multiple peers using an actual "network", but that's not too bad.