I was investigating using MetalLB for my Kubernetes cluster with my home network. MetalLB can use either a L2 IPs with ARP or BGP to manage connections, so began the research around using BGP with pfSense. A recommendation from r/pfsense pointed me towards a video produced by Netgate to use a package called FRR.

From so long ago, when I took the CCNA classes I didn’t cover BGP in detail or do not remember it. So this video seems to be a great refresher. Here are my notes:

BGP

The goal of BGP is to automate routing to be resilient to link failures.

BGP is built on top of TCP/IP on port 179, having both an IPv4 and IPv6 implementation. All peers are expected to be known and availble to determine the best routes.

BGP uses Autonomous System Numbers, abbrevitated to ASN or AS. These identify internal networks which each BGP peer is responsible for routing too. Generally each peer will map to a differnet site, however there might be cases such as on premise data centers which would have a different address than the offices they are associated with.

BPG can run in two modes. Either internal (iBGP) facing towards the LANs or external (eBGP) facing the general internet.

OSPF

Stands for Open Shortest Path First. Is designed to manage routes within a specific site in an automated manner, responding to availability, congestion, and possibly other cost factors an operator would like to include.

OSPF is built directly on top of the IPv4 and IPv6 datagrams as protocol 89. These datagrams communicate with adjacent nodes using multicast. According to the documentation these seems to be apart of the reasonsing why OSPF is only suitable for internal network descriptions.

OSPF will build a network map using OSPF peers. These peers will be classified as one of the following:

  • Backbone Router: The core network requiring traveral between areas of the network. Could be thought of as your core routers and switches to get traffice to move between networks.
  • Autonomous System Boundary Router: These are responsible for trasniting data to outside networks and will generally run BGP to route traffic to the internet. This is abbreviated as ASBRs.
  • Area boundary router: routers which integrate with the backbone routers, allowing transit into regions of the graph.
  • Internal router: A router which only has interfaces within a specific area of the graph, not connecting the backbone or other ASBRs.

FRR

FRRouting is a software suite implementing various autonomous networking systems such as RIP, OSPF, BGP, and LDP. The architecture has a centeral controller, Zebra, for coordination with the various protocol specific adapters and the operating system. This is based on another project called Quagga.

Overall looks like a solid product. UX is definitely not pfSense’s strength but I have also seen worse.

Conclusion

BGP is an interesting protocol, especially for dealing with the human organization boundaries in an automated way. OSPF seems like a great protocol for running internally on more complicated networks, easing some of the burden of managing complicated subnets. Unfortunately BGP doesn’t seem to be something my residential provider really supports out of the box at this time, so in terms of performance it doesn’t have much of a benefit.

As for MetalLB I am a bit puzzled by their choice to use BGP over OSPF. OSPF seems to be a bit more resilient to the changes you expect in a Kubernetes cluster, such as rotating pods out or broadcasting multiple paths to connect into the services. I hope to be missing something major as to their choice however I think I will stay with the L2 implementation for my network.