diff --git a/Makefile b/Makefile index 42fef965..2f594b3c 100644 --- a/Makefile +++ b/Makefile @@ -78,6 +78,7 @@ TARGETS += drbd TARGETS += dvb-cx23885 TARGETS += dvb-m88ds3103 TARGETS += ecr-credential-provider +TARGETS += frr TARGETS += fuse3 TARGETS += gasket-driver TARGETS += gpio-pinctrl diff --git a/Pkgfile b/Pkgfile index b5bac186..121aab52 100644 --- a/Pkgfile +++ b/Pkgfile @@ -30,6 +30,8 @@ vars: LINUX_DVB_FIRMWARE: 0.0.51 LINUX_DVB_FIRMWARE_SHA256: cef3ce537d213e020af794cecf9de207e2882c375ceda39102eb6fa2580bad8d LINUX_DVB_FIRMWARE_SHA512: 2372dba98083c76865f5f0f8101b1160888e03cdbe911dd08621e7b6f38e8a25ae5d56eefc21728bf65fd09ea613b53606df4021d3972b0bc9d2bd8b6cbe20a1 + # renovate: datasource=docker depName=quay.io/frrouting/frr + FRR_VERSION: "10.5.1" labels: org.opencontainers.image.source: https://github.com/siderolabs/extensions diff --git a/network/frr/README.md b/network/frr/README.md new file mode 100644 index 00000000..be055c69 --- /dev/null +++ b/network/frr/README.md @@ -0,0 +1,272 @@ +# FRR (Free Range Routing) Extension for Talos + +This extension provides FRR for BGP routing on Talos hosts, with built-in MetalLB VRF integration for advertising Kubernetes LoadBalancer IPs via BGP. + +## Overview + +### Purpose + +1. **FRR (Free Range Routing)** for BGP routing on Talos hosts +2. **MetalLB VRF integration** with a veth pair for Kubernetes LoadBalancer IP advertisement +3. **Private IPv6 point-to-point connection** (`fd00::/8`) between FRR and MetalLB speaker +4. **Dynamic configuration** via Jinja2 template (`frr.conf.j2`) rendered using `jinja2-cli` +5. **Interface discovery** from MAC addresses specified in `FE_MACS` or ports from `FE_PORT_NAMES` environment variable + +### Architecture + +- **FRR runs in host network namespace** and manages both: + - Fabric-facing BGP peering (eBGP with leaf switches via physical interfaces) + - MetalLB-facing BGP peering (eBGP with node local MetalLB BGP speaker in VRF `metallb`) + +- **MetalLB speaker** runs with `hostNetwork: true`, connects to FRR via a veth pair: + - `veth-metallb` interface: in **host namespace** - MetalLB speaker binds here + - `veth-frr` interface: in **VRF metallb** - FRR listens here for MetalLB connections + +- **Route flow**: MetalLB advertises LoadBalancer IPs → FRR VRF BGP → imported to default VRF → advertised to fabric + +### Components + +| Component | Description | +|-----------|-------------| +| `frr-startup.sh` | Bash script that runs before FRR to set up VRF, veth pair, IPv6 addresses, MTU, and generate `frr.conf` | +| `frr.conf.j2` | Jinja2 template for FRR configuration with variables from environment | +| `daemons` | FRR daemons configuration (zebra, bgpd, staticd, bfdd enabled) | + +### Startup Script Responsibilities + +1. **Validate mandatory environment variables** (`NODE_IP`, `ASN_LOCAL`, and either `FE_MACS` or `FE_PORT_NAMES`) +2. **Resolve interface names** from MAC addresses in `FE_MACS` or port in `FE_PORT_NAMES` (CSV format) +3. **Auto-detect MTU** from first fabric interface (if `INTERFACE_MTU` not set) +4. **Create VRF `metallb`** with configurable routing table ID (default: 88) +5. **Create veth pair**: + - `veth-frr` assigned to VRF `metallb` with `PEER_IP_FRR` (default: `fda1:b2c3:d4e5:0001::0/127`) + - `veth-metallb` in host namespace with `PEER_IP_METALLB` (default: `fda1:b2c3:d4e5:0001::1/127`) +6. **Set MTU** on veth pair from `INTERFACE_MTU` (auto-detected or user-provided) +7. **Generate `/etc/frr/frr.conf`** from Jinja2 template using `jinja2-cli` +8. **Exec to FRR's `docker-start`** script to launch FRR daemons + +## Network Topology + +``` +┌──────────────────────────────────────────────────────────────────┐ +│ Talos Host │ +│ │ +│ ┌─────────────────────────────────────────────────────────────┐ │ +│ │ Default VRF (Host) │ │ +│ │ │ │ +│ │ ┌──────────────────────────┐ │ │ +│ │ │ MetalLB BGP Speaker │ │ │ +│ │ │ (hostNetwork: true) │ │ │ +│ │ │ ASN: 4000099999 │ │ │ +│ │ │ │ │ │ +│ │ │ Binds to veth-metallb │ │ │ +│ │ │ fda1:...:0001::1/127 │ │ │ +│ │ └────────────┬─────────────┘ │ │ +│ │ │ │ │ +│ │ veth-metallb (in host ns) │ │ +│ │ fda1:...:0001::1/127 │ │ +│ │ │ veth │ │ +│ └───────────────┼──────────────────────────────────────────────┘ │ +│ │ │ +│ ┌───────────────┼──────────────────────────────────────────────┐ │ +│ │ │ VRF: metallb (table 88) │ │ +│ │ ▼ │ │ +│ │ veth-frr (in VRF) │ │ +│ │ fda1:...:0001::0/127 │ │ +│ │ │ │ │ +│ │ ┌───────────▼─────────────────────────────────────────┐ │ │ +│ │ │ FRR │ │ │ +│ │ │ router bgp 4000099998 vrf metallb │ │ │ +│ │ │ - Listens passively on veth-frr:179 │ │ │ +│ │ │ - Receives LB IP routes from MetalLB │ │ │ +│ │ │ │ │ │ +│ │ │ router bgp 65001 (default VRF) │ │ │ +│ │ │ - Peers with fabric via FE_MACS/FE_PORT_NAMES ifaces│ │ │ +│ │ │ - Imports routes from VRF metallb │ │ │ +│ │ │ - Advertises LB IPs to fabric │ │ │ +│ │ └──────────────────────────────────────────────────────┘ │ │ +│ └───────────────────────────────────────────────────────────────┘ │ +│ │ +│ Physical Interfaces (resolved from FE_MACS or from FE_PORT_NAMES)│ +│ ├── eth0 ──► Leaf Switch 1 (eBGP unnumbered, IPv6 link-local) │ +│ └── eth1 ──► Leaf Switch 2 (eBGP unnumbered, IPv6 link-local) │ +└───────────────────────────────────────────────────────────────────┘ +``` + +## Host Networking Architecture + +This extension supports advertising host IPs from loopback and dummy interfaces to enable: + +1. **Host Traffic HA/Load Balancing via ECMP** - Multiple equal-cost paths to host IPs +2. **Anycast Kubernetes API** - Control plane nodes advertise a shared VIP via `dummy1` + +### Interface IP Advertisement + +FRR advertises connected routes from these interfaces to fabric switches: + +| Interface | Purpose | +|-----------|---------| +| `lo` | Primary loopback IP (node identity) | +| `dummy0` | Additional service IPs for the host | +| `dummy1` | Anycast IPs (e.g., Kubernetes API VIP on control plane nodes) | + +This is configured via the `loopbacks` route-map which matches and redistributes connected routes from `lo`, `dummy0`, and `dummy1`. + +### ECMP Load Balancing + +When multiple paths exist to a destination (e.g., dual-homed hosts with two fabric uplinks), FRR enables ECMP via: + +``` +bgp bestpath as-path multipath-relax +``` + +This allows traffic to be load-balanced across multiple equal-cost paths, providing both redundancy and increased bandwidth. + +### Anycast Kubernetes API + +For highly available Kubernetes API access: + +1. Configure `dummy1` on all control plane nodes with the same anycast VIP (e.g., `10.0.0.100/32`) +2. FRR advertises this IP from all control plane nodes +3. Fabric switches see multiple paths to the VIP and load-balance traffic via ECMP +4. If a control plane node fails, its route is withdrawn and traffic flows to remaining nodes + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ Leaf/Spine Fabric │ +│ │ +│ Routes to 10.0.0.100/32: │ +│ ├── via Control Plane 1 (eth0) │ +│ ├── via Control Plane 2 (eth0) │ +│ └── via Control Plane 3 (eth0) │ +│ │ +│ ECMP distributes API traffic across all healthy control planes │ +└─────────────────────────────────────────────────────────────────────┘ + │ │ │ + ▼ ▼ ▼ +┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ +│ Control Plane 1 │ │ Control Plane 2 │ │ Control Plane 3 │ +│ │ │ │ │ │ +│ dummy1: │ │ dummy1: │ │ dummy1: │ +│ 10.0.0.100/32 │ │ 10.0.0.100/32 │ │ 10.0.0.100/32 │ +│ │ │ │ │ │ +│ kube-apiserver │ │ kube-apiserver │ │ kube-apiserver │ +└─────────────────┘ └─────────────────┘ └─────────────────┘ +``` + +### Configuring Dummy Interfaces on Talos + +Add dummy interfaces via Talos machine config: + +```yaml +machine: + network: + interfaces: + - interface: dummy0 + dummy: true + addresses: + - 10.0.1.1/32 # Host IP + - interface: dummy1 + dummy: true + vip: + ip: 10.0.0.100 # Anycast K8s API VIP (control plane only) +``` + +## Environment Variables + +| Variable | Required | Default | Description | +|----------|----------|---------|-------------| +| `NODE_IP` | **Yes** | - | Node's primary IPv4 address (BGP router-id) | +| `ASN_LOCAL` | **Yes** | - | Local BGP AS number for fabric peering | +| `FE_MACS` | No* | - | Comma-separated MAC addresses of fabric-facing interfaces | +| `FE_PORT_NAMES` | No* | - | Comma-separated interface names of fabric-facing interfaces | +| `ASN_VRF_METALLB` | No | `4000099998` | BGP ASN for FRR's MetalLB VRF session | +| `ASN_METALLB_SPEAKER` | No | `4000099999` | BGP ASN for MetalLB speaker | +| `VRF_METALLB` | No | `metallb` | Name of the MetalLB VRF | +| `METALLB_VRF_ROUTE_TABLE_ID` | No | `88` | Linux routing table ID for VRF | +| `PEER_IP_FRR` | No | `fda1:b2c3:d4e5:0001::0` | FRR-side IPv6 address (in VRF) | +| `PEER_IP_METALLB` | No | `fda1:b2c3:d4e5:0001::1` | MetalLB-side IPv6 address (in host ns) | +| `PEER_IP_PREFIX` | No | `127` | IPv6 prefix length for p2p link | +| `INTERFACE_MTU` | No | Auto-detect from first fabric interface | MTU for veth pair | +| `ENABLE_BFD` | No | `true` | Enable BFD for fabric BGP peers | +| `FRR_PROFILE` | No | `datacenter` | FRR defaults profile (`datacenter` or `traditional`) | + +\* Either `FE_MACS` or `FE_PORT_NAMES` is required (not both). + +## Usage + +### Extension Configuration + +Configure the FRR extension via `ExtensionServiceConfig`: + +```yaml +--- +apiVersion: v1alpha1 +kind: ExtensionServiceConfig +name: frr +environment: + - NODE_IP=10.0.0.1 + - ASN_LOCAL=65001 + - FE_MACS=00:11:22:33:44:55,00:11:22:33:44:56 +``` + +Or using interface names directly: + +```yaml +--- +apiVersion: v1alpha1 +kind: ExtensionServiceConfig +name: frr +environment: + - NODE_IP=10.0.0.1 + - ASN_LOCAL=65001 + - FE_PORT_NAMES=eth0,eth1 +``` + +### MetalLB Configuration + +Configure MetalLB to peer with FRR via the veth interface: + +```yaml +apiVersion: metallb.io/v1beta2 +kind: BGPPeer +metadata: + name: frr-peer + namespace: metallb-system +spec: + myASN: 4000099999 + peerASN: 4000099998 + peerAddress: fda1:b2c3:d4e5:0001::0 + sourceAddress: fda1:b2c3:d4e5:1::1 +``` + +### Example: Advertising LoadBalancer IPs + +1. MetalLB speaker (with `hostNetwork: true`) binds to `veth-metallb` interface +2. MetalLB initiates BGP connection to FRR at `fda1:b2c3:d4e5:0001::0:179` +3. FRR's VRF BGP process (`router bgp 4000099998 vrf metallb`) accepts the connection +4. MetalLB advertises LoadBalancer IPs to FRR +5. FRR imports routes from VRF metallb into the default VRF +6. FRR advertises the LoadBalancer IPs to fabric switches via eBGP + +## FRR Daemons + +The following FRR daemons are enabled: + +- **zebra** - Routing table manager +- **bgpd** - BGP routing daemon +- **staticd** - Static route daemon +- **bfdd** - BFD (Bidirectional Forwarding Detection) daemon + +## Building + +This extension is built using the Talos extensions build system: + +```bash +make frr +``` + +## License + +This extension is licensed under the Mozilla Public License 2.0. +FRR is licensed under GPL-2.0-or-later. diff --git a/network/frr/files/daemons b/network/frr/files/daemons new file mode 100644 index 00000000..b72addd7 --- /dev/null +++ b/network/frr/files/daemons @@ -0,0 +1,10 @@ +zebra=true +zebra_options="-A 127.0.0.1 -s 90000000" +bgpd=true +bgpd_options="-A 127.0.0.1" +staticd=true +staticd_options="-A 127.0.0.1" +bfdd=true +bfdd_options="-A 127.0.0.1" +vtysh_enable=yes +mgmtd_options="-A 127.0.0.1" diff --git a/network/frr/files/frr-startup.sh b/network/frr/files/frr-startup.sh new file mode 100644 index 00000000..2ec74820 --- /dev/null +++ b/network/frr/files/frr-startup.sh @@ -0,0 +1,210 @@ +#!/bin/bash +# FRR Startup Script for Talos +# This script sets up the MetalLB VRF, veth pair, and generates frr.conf before starting FRR. + +set -e + +log() { echo "$(date '+%Y-%m-%d %H:%M:%S') [frr-startup] $*" >&2; } +error() { echo "$(date '+%Y-%m-%d %H:%M:%S') [frr-startup] ERROR: $*" >&2; exit 1; } + +# Default values +ASN_VRF_METALLB="${ASN_VRF_METALLB:-4000099998}" +ASN_METALLB_SPEAKER="${ASN_METALLB_SPEAKER:-4000099999}" +VRF_METALLB="${VRF_METALLB:-metallb}" +METALLB_VRF_ROUTE_TABLE_ID="${METALLB_VRF_ROUTE_TABLE_ID:-88}" +PEER_IP_FRR="${PEER_IP_FRR:-fda1:b2c3:d4e5:0001::0}" +PEER_IP_METALLB="${PEER_IP_METALLB:-fda1:b2c3:d4e5:0001::1}" +PEER_IP_PREFIX="${PEER_IP_PREFIX:-127}" +ENABLE_BFD="${ENABLE_BFD:-true}" +FRR_PROFILE="${FRR_PROFILE:-datacenter}" +# INTERFACE_MTU - auto-detected from first fabric interface if not set + +# Validate mandatory variables +[ -z "$NODE_IP" ] && error "NODE_IP is required" +[ -z "$ASN_LOCAL" ] && error "ASN_LOCAL is required" + +# Validate: either FE_MACS or FE_PORT_NAlog syslogMES is required (not both) +if [ -n "$FE_MACS" ] && [ -n "$FE_PORT_NAMES" ]; then + error "Specify either FE_MACS or FE_PORT_NAMES, not both" +elif [ -z "$FE_MACS" ] && [ -z "$FE_PORT_NAMES" ]; then + error "Either FE_MACS or FE_PORT_NAMES is required" +fi + +# Validate FRR_PROFILE +case "$FRR_PROFILE" in + datacenter|traditional) ;; + *) error "FRR_PROFILE must be 'datacenter' or 'traditional', got: $FRR_PROFILE" ;; +esac + +# Resolve MAC addresses to interface names +# Input: comma-separated MAC addresses +# Output: comma-separated interface names +resolve_interfaces() { + local macs="$1" + local interfaces="" + + IFS=',' read -ra MAC_ARRAY <<< "$macs" + for mac in "${MAC_ARRAY[@]}"; do + mac=$(echo "$mac" | tr '[:upper:]' '[:lower:]' | xargs) + iface=$(/sbin/ip -o link | awk -v mac="$mac" 'tolower($0) ~ mac {print $2}' | tr -d ':') + if [ -n "$iface" ]; then + interfaces="${interfaces:+$interfaces,}$iface" + log "Resolved MAC $mac -> $iface" + else + log "WARNING: No interface found for MAC $mac" + fi + done + echo "$interfaces" +} + +# Create MetalLB VRF +create_metallb_vrf() { + log "Creating VRF $VRF_METALLB with table $METALLB_VRF_ROUTE_TABLE_ID..." + + if ! /sbin/ip link show "$VRF_METALLB" &>/dev/null; then + /sbin/ip link add "$VRF_METALLB" type vrf table "$METALLB_VRF_ROUTE_TABLE_ID" + fi + /sbin/ip link set "$VRF_METALLB" up + log "VRF $VRF_METALLB is up" +} + +# Create veth pair for MetalLB +# frr-metallb: in VRF metallb (FRR listens here) +# metallb-frr: in host namespace (MetalLB speaker connects from here) +create_metallb_veth() { + local veth_frr="veth-frr" + local veth_metallb="veth-metallb" + + log "Creating veth pair: $veth_frr <-> $veth_metallb..." + /sbin/ip link add "$veth_frr" type veth peer name "$veth_metallb" 2>/dev/null || log "Veth pair $veth_frr <-> $veth_metallb already exists" + + log "Assining $veth_frr to VRF $VRF_METALLB..." + /sbin/ip link set "$veth_frr" master "$VRF_METALLB" + + # Set MTU with explicit error handling + log "Setting MTU $INTERFACE_MTU on $veth_frr..." + /sbin/ip link set "$veth_frr" mtu "$INTERFACE_MTU" + + log "Setting MTU $INTERFACE_MTU on $veth_metallb..." + /sbin/ip link set "$veth_metallb" mtu "$INTERFACE_MTU" + + # Assign IPv6 addresses + # PEER_IP_FRR is on frr-metallb (in VRF) - FRR listens here + # PEER_IP_METALLB is on metallb-frr (in host ns) - MetalLB binds here + log "Assigning IPv6 addresses..." + /sbin/ip -6 addr add "${PEER_IP_FRR}/${PEER_IP_PREFIX}" dev "$veth_frr" 2>/dev/null || log "IPv6 $PEER_IP_FRR already assigned to $veth_frr" + /sbin/ip -6 addr add "${PEER_IP_METALLB}/${PEER_IP_PREFIX}" dev "$veth_metallb" 2>/dev/null || log "IPv6 $PEER_IP_METALLB already assigned to $veth_metallb" + + # Bring up interfaces + log "Bringing up interfaces..." + /sbin/ip link set "$veth_frr" up + /sbin/ip link set "$veth_metallb" up + + log "MetalLB veth pair configured:" + log " $veth_frr ($PEER_IP_FRR) in VRF $VRF_METALLB (FRR listens)" + log " $veth_metallb ($PEER_IP_METALLB) in host ns (MetalLB connects from)" +} + +# Configure daemons based on ENABLE_BFD +configure_daemons() { + if [ "$ENABLE_BFD" = "true" ]; then + log "BFD enabled" + else + log "BFD disabled, modifying /etc/frr/daemons..." + sed -i 's/^bfdd=yes/bfdd=no/' /etc/frr/daemons + fi +} + +# Generate frr.conf from Jinja2 template +generate_frr_conf() { + local interfaces="$1" + + log "Generating /etc/frr/frr.conf from template..." + log "FRR Profile: $FRR_PROFILE" + + # Build JSON for ports array + local ports_json="[" + if [ -n "$interfaces" ]; then + IFS=',' read -ra IFACE_ARRAY <<< "$interfaces" + for i in "${!IFACE_ARRAY[@]}"; do + [ $i -gt 0 ] && ports_json+="," + ports_json+="\"${IFACE_ARRAY[$i]}\"" + done + fi + ports_json+="]" + + # Build JSON with all template variables + cat > /frr-vars.json < /etc/frr/frr.conf + log "Generated /etc/frr/frr.conf" +} + +# Main +main() { + log "FRR Startup Script - Initializing..." + + # Get interfaces from FE_PORT_NAMES or resolve from FE_MACS + local interfaces="" + if [ -n "$FE_PORT_NAMES" ]; then + interfaces="$FE_PORT_NAMES" + log "Using fabric interfaces from FE_PORT_NAMES: $interfaces" + else + interfaces=$(resolve_interfaces "$FE_MACS") + log "Resolved fabric interfaces from FE_MACS: $interfaces" + fi + + # Use MTU from first fabric interface as default if not set + if [ -z "$INTERFACE_MTU" ]; then + local first_iface="${interfaces%%,*}" + if [ -n "$first_iface" ]; then + INTERFACE_MTU=$(cat "/sys/class/net/$first_iface/mtu" 2>/dev/null || echo "1500") + log "Using MTU $INTERFACE_MTU from $first_iface" + else + INTERFACE_MTU="1500" + log "No interfaces resolved, using default MTU 1500" + fi + else + log "Using user-provided MTU $INTERFACE_MTU" + fi + + # Create MetalLB VRF and veth pair + create_metallb_vrf + create_metallb_veth + + # Configure daemons based on ENABLE_BFD + configure_daemons + + # Generate FRR configuration + generate_frr_conf "$interfaces" + + ## create vtysh.conf + [ -r /etc/frr/vtysh.conf ] || touch /etc/frr/vtysh.conf + + # Ensure FRR runtime directories exist + log "mkdir -p /var/run/frr || true" + mkdir -p /var/run/frr || true + chown -R frr:frr /etc/frr || true + rm -rf /var/run/frr/* || true + chown -R frr:frr /var/run/frr || true + log "Initialization complete, starting FRR..." + + # Start FRR using the standard docker-start script + exec /usr/lib/frr/docker-start +} + +main "$@" diff --git a/network/frr/files/frr-wrapper.sh b/network/frr/files/frr-wrapper.sh new file mode 100644 index 00000000..47b86620 --- /dev/null +++ b/network/frr/files/frr-wrapper.sh @@ -0,0 +1,4 @@ +#!/bin/bash +# Wrapper script to invoke frr-startup.sh +# This allows overriding frr-startup.sh via extension service config for testing +exec /bin/bash /usr/local/bin/frr-startup.sh diff --git a/network/frr/files/frr.conf.j2 b/network/frr/files/frr.conf.j2 new file mode 100644 index 00000000..c74de0a1 --- /dev/null +++ b/network/frr/files/frr.conf.j2 @@ -0,0 +1,84 @@ +{# FRR Configuration Template for Talos #} +{# Variables: ports, NODE_IP, ASN_LOCAL, ASN_VRF_METALLB, #} +{# ASN_METALLB_SPEAKER, VRF_METALLB, PEER_IP_FRR, #} +{# PEER_IP_METALLB, ENABLE_BFD, FRR_PROFILE #} +frr defaults {{FRR_PROFILE}} +log syslog +! +{%- for n in ports %} +interface {{n}} + ipv6 nd ra-interval 10 + no ipv6 nd suppress-ra +! +{%- endfor %} + +ipv6 prefix-list only-host-prefixes seq 10 permit ::/0 ge 128 +! +ip prefix-list denyall seq 10 deny 0.0.0.0/0 +ip prefix-list out-filter seq 10 permit 0.0.0.0/0 ge 24 +ip prefix-list out-filter seq 11 deny 0.0.0.0/0 +! +route-map out-map permit 10 + match ip address prefix-list out-filter +exit +! +route-map denymap permit 1 + match ip address prefix-list denyall +! +route-map SETSRC permit 10 + set src {{NODE_IP}} +! +route-map loopbacks permit 10 + match interface lo +! +route-map loopbacks permit 20 + match interface dummy0 +! +route-map loopbacks permit 30 + match interface dummy1 +! + +router bgp {{ASN_VRF_METALLB}} vrf {{VRF_METALLB}} + bgp router-id {{NODE_IP}} + no bgp ebgp-requires-policy + neighbor METALLB peer-group + neighbor METALLB remote-as {{ASN_METALLB_SPEAKER}} + neighbor METALLB passive + neighbor {{PEER_IP_METALLB}} peer-group METALLB + ! + address-family ipv4 unicast + neighbor METALLB route-map denymap out + exit-address-family +! + +router bgp {{ASN_LOCAL}} + bgp router-id {{NODE_IP}} + no bgp ebgp-requires-policy + bgp bestpath as-path multipath-relax + neighbor FABRIC peer-group + neighbor FABRIC remote-as external +{%- if ENABLE_BFD == "true" %} + neighbor FABRIC bfd +{%- endif %} + neighbor FABRIC timers 1 3 +{%- for n in ports %} + neighbor {{n}} interface v6only peer-group FABRIC +{%- endfor %} + ! + address-family ipv4 unicast + redistribute connected route-map loopbacks + neighbor FABRIC soft-reconfiguration inbound + neighbor FABRIC route-map out-map out + import vrf {{VRF_METALLB}} + exit-address-family + ! + address-family ipv6 unicast + neighbor FABRIC activate + neighbor FABRIC prefix-list only-host-prefixes out + exit-address-family +! +ip protocol bgp route-map SETSRC +! +line vty +! + diff --git a/network/frr/frr.yaml b/network/frr/frr.yaml new file mode 100644 index 00000000..8dbcf602 --- /dev/null +++ b/network/frr/frr.yaml @@ -0,0 +1,37 @@ +# © 2026 Nokia +# Licensed under the Mozilla Public License 2.0 +# SPDX-License-Identifier: MPL-2.0 +name: frr +depends: + - service: cri + - network: + - addresses + - connectivity + - configuration: true +container: + entrypoint: /sbin/tini + args: + - -- + - /usr/local/bin/frr-wrapper.sh + security: + writeableRootfs: true + writeableSysfs: true + maskedPaths: [] + readonlyPaths: [] + mounts: + - source: /var/run/frr + destination: /var/run/frr + type: bind + options: + - rshared + - rbind + - rw + - source: /dev/log + destination: /dev/log + type: bind + options: + - rshared + - rbind + - ro +restart: always +logToConsole: true diff --git a/network/frr/manifest.yaml.tmpl b/network/frr/manifest.yaml.tmpl new file mode 100644 index 00000000..ae3183c9 --- /dev/null +++ b/network/frr/manifest.yaml.tmpl @@ -0,0 +1,15 @@ +# © 2026 Nokia +# Licensed under the Mozilla Public License 2.0 +# SPDX-License-Identifier: MPL-2.0 +version: v1alpha1 +metadata: + name: frr + version: "{{ .VERSION }}" + author: Kai Zhang + description: | + [{{ .TIER }}] FRR (Free Range Routing) for BGP routing on Talos hosts. + Includes MetalLB VRF integration for advertising Kubernetes LoadBalancer IPs via BGP. + Based on quay.io/frrouting/frr:{{ .FRR_VERSION }}. + compatibility: + talos: + version: ">= v1.11.0" diff --git a/network/frr/pkg.yaml b/network/frr/pkg.yaml new file mode 100644 index 00000000..63556df0 --- /dev/null +++ b/network/frr/pkg.yaml @@ -0,0 +1,58 @@ +# © 2026 Nokia +# Licensed under the Mozilla Public License 2.0 +# SPDX-License-Identifier: MPL-2.0 +name: frr +variant: scratch +shell: /bin/bash +dependencies: + - stage: base + - image: "quay.io/frrouting/frr:{{ .FRR_VERSION }}" +steps: + - env: + SOURCE_DATE_EPOCH: "{{ .BUILD_ARG_SOURCE_DATE_EPOCH }}" + network: default + prepare: + - apk add --no-cache jinja2-cli + install: + - | + # Create container rootfs structure + mkdir -p /rootfs/usr/local/{etc/containers,lib/containers/frr} + + # Copy FRR filesystem components (we're running inside FRR image context) + # Include /var and /etc to get all FRR required directories + cp -a /bin /rootfs/usr/local/lib/containers/frr/ + cp -a /lib /rootfs/usr/local/lib/containers/frr/ + cp -a /usr /rootfs/usr/local/lib/containers/frr/ + cp -a /sbin /rootfs/usr/local/lib/containers/frr/ 2>/dev/null || true + cp -a /var /rootfs/usr/local/lib/containers/frr/ + cp -a /etc /rootfs/usr/local/lib/containers/frr/ + + # Copy our files (overwrite FRR defaults) + cp /pkg/files/frr-startup.sh /rootfs/usr/local/lib/containers/frr/usr/local/bin/ + chmod +x /rootfs/usr/local/lib/containers/frr/usr/local/bin/frr-startup.sh + cp /pkg/files/frr-wrapper.sh /rootfs/usr/local/lib/containers/frr/usr/local/bin/ + chmod +x /rootfs/usr/local/lib/containers/frr/usr/local/bin/frr-wrapper.sh + + cp /pkg/files/frr.conf.j2 /rootfs/usr/local/lib/containers/frr/etc/frr/ + cp /pkg/files/daemons /rootfs/usr/local/lib/containers/frr/etc/frr/ + + # Copy service definition + cp /pkg/frr.yaml /rootfs/usr/local/etc/containers/ + test: + - | + mkdir -p /extensions-validator-rootfs + cp -r /rootfs/ /extensions-validator-rootfs/rootfs + cp /pkg/manifest.yaml /extensions-validator-rootfs/manifest.yaml + /extensions-validator validate --rootfs=/extensions-validator-rootfs --pkg-name="${PKG_NAME}" + sbom: + outputPath: /rootfs/usr/local/share/spdx/frr.spdx.json + version: "{{ .VERSION }}" + cpes: + - "cpe:2.3:a:frrouting:frr:{{ .VERSION }}:*:*:*:*:*:*:*" + licenses: + - GPL-2.0-or-later +finalize: + - from: /rootfs + to: /rootfs + - from: /pkg/manifest.yaml + to: / diff --git a/network/frr/vars.yaml b/network/frr/vars.yaml new file mode 100644 index 00000000..9f2d9d18 --- /dev/null +++ b/network/frr/vars.yaml @@ -0,0 +1,5 @@ +# © 2026 Nokia +# Licensed under the Mozilla Public License 2.0 +# SPDX-License-Identifier: MPL-2.0 +VERSION: "{{ .FRR_VERSION }}" +TIER: "contrib"