January 28, 2025 15 min to read

MetalLB - Load Balancer for Bare Metal Kubernetes Clusters

A comprehensive guide to implementing MetalLB in bare metal environments

Image Reference

Overview

In cloud environments (AWS, GCP, Azure), Kubernetes can leverage built-in cloud load balancers when you create a Service of type LoadBalancer. However, in bare metal environments, these cloud resources aren’t available, leaving the LoadBalancer service in a perpetual “pending” state.

MetalLB solves this problem by providing a network load balancer implementation for Kubernetes clusters that don’t run on a cloud provider, effectively enabling the LoadBalancer service type in any environment.

$ kubectl get svc
NAME          TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes    ClusterIP      10.96.0.1        <none>        443/TCP        10h
myapp         LoadBalancer   10.106.247.153   <pending>     80:31122/TCP   10m

MetalLB Architecture

MetalLB consists of two main components that work together to provide load balancing functionality:

1. Controller: A deployment that assigns IP addresses to LoadBalancer services
2. Speaker: A DaemonSet that advertises the services using either Layer 2 (ARP/NDP) or BGP

How It Works

1. When a LoadBalancer service is created, the MetalLB controller assigns an IP address from a configured address pool
2. The speaker pods (running on each node) advertise the IP address using either Layer 2 or BGP mode
3. External traffic can now reach the service using the assigned external IP
4. Once traffic reaches a node, standard Kubernetes service routing directs it to the appropriate pods

Operating Modes

MetalLB supports two operating modes, each with distinct characteristics and use cases:

Layer 2 Mode (ARP/NDP)

In Layer 2 mode, one speaker pod becomes the “leader” for each service and responds to ARP requests for the service’s IP address, making the node appear to be the machine that owns the service IP.

BGP Mode

In BGP mode, each speaker pod establishes BGP peering sessions with configured routers and advertises routes for service IPs.

Mode Comparison

Installation and Configuration

Prerequisites

For optimal operation, especially when using kube-proxy in IPVS mode, enable strictARP:

Installation Methods

Method 1: Install with Helm (Recommended)

# Add MetalLB helm repo
helm repo add metallb https://metallb.github.io/metallb

# Update helm repositories
helm repo update

# Install MetalLB
helm install metallb metallb/metallb -n metallb-system --create-namespace

# Verify the installation
kubectl get pods -n metallb-system

Method 2: Install with Manifests

# Install MetalLB components
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/main/config/manifests/metallb-native.yaml

# Verify the installation
kubectl get pods -n metallb-system

Configuration

MetalLB requires two key configurations:

1. IPAddressPool: Defines the IP ranges that MetalLB can assign
2. L2Advertisement or BGPAdvertisement: Defines how these IPs are advertised

Layer 2 Configuration Example

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.240-192.168.1.250
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: layer2-config
  namespace: metallb-system
spec:
  ipAddressPools:
  - first-pool

BGP Configuration Example

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: bgp-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.10.0/24
---
apiVersion: metallb.io/v1beta1
kind: BGPPeer
metadata:
  name: router-peer
  namespace: metallb-system
spec:
  myASN: 64500
  peerASN: 64501
  peerAddress: 192.168.0.1
---
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: bgp-advert
  namespace: metallb-system
spec:
  ipAddressPools:
  - bgp-pool

Complete Configuration Example

Here’s a complete example with address pools and both Layer 2 and BGP configuration:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: production-public-ips
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.240-192.168.1.250
  - 192.168.10.0/24
  autoAssign: true
  avoidBuggyIPs: true
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: layer2-prod
  namespace: metallb-system
spec:
  ipAddressPools:
  - production-public-ips
---
apiVersion: metallb.io/v1beta1
kind: BGPPeer
metadata:
  name: router1
  namespace: metallb-system
spec:
  myASN: 64500
  peerASN: 64501
  peerAddress: 192.168.0.1
  holdTime: 120s
  keepaliveTime: 30s
  routerID: 192.168.0.100
  nodeSelectors:
  - matchLabels:
      kubernetes.io/hostname: worker-1
---
apiVersion: metallb.io/v1beta1
kind: BGPPeer
metadata:
  name: router2
  namespace: metallb-system
spec:
  myASN: 64500
  peerASN: 64501
  peerAddress: 192.168.0.2
  holdTime: 120s
  keepaliveTime: 30s
  routerID: 192.168.0.100
  nodeSelectors:
  - matchLabels:
      kubernetes.io/hostname: worker-2
---
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: bgp-prod
  namespace: metallb-system
spec:
  ipAddressPools:
  - production-public-ips
  communities:
  - 64500:100
  localPref: 100
  aggregationLength: 32

Applying and Verifying Configuration

# Apply configuration
kubectl apply -f metallb-config.yaml

# Verify IPAddressPool configuration
kubectl describe ipaddresspool -n metallb-system

# Verify L2Advertisement configuration
kubectl describe l2advertisement -n metallb-system

# For BGP mode, verify BGP configuration
kubectl describe bgppeer -n metallb-system
kubectl describe bgpadvertisement -n metallb-system

Testing MetalLB

Deploy a Test Application

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test
spec:
  selector:
    matchLabels:
      app: nginx-test
  template:
    metadata:
      labels:
        app: nginx-test
    spec:
      containers:
      - name: nginx
        image: nginx:1.21
        ports:
        - name: http
          containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-test
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx-test
  type: LoadBalancer

Verify External IP Assignment

# Check if the service received an external IP
kubectl get svc nginx-test

# Expected output:
# NAME         TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)        AGE
# nginx-test   LoadBalancer   10.43.162.209   192.168.1.240    80:31621/TCP   10s

# Test connectivity
curl http://192.168.1.240

Advanced Configuration

Address Pool Options

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: advanced-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.10.0/24
  - 192.168.20.10-192.168.20.50
  autoAssign: true  # Whether to automatically assign from this pool (default: true)
  avoidBuggyIPs: true  # Avoid .0 and .255 addresses which can cause problems

BGP Options

apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: advanced-bgp
  namespace: metallb-system
spec:
  ipAddressPools:
  - advanced-pool
  aggregationLength: 32  # Advertise individual /32 routes
  localPref: 100  # BGP local preference
  communities:  # BGP communities to attach
  - 65535:65282  # no-export community
  - 64500:100    # Custom community

BFD (Bidirectional Forwarding Detection) for Faster Failover

For faster BGP failover, MetalLB supports BFD (available in newer versions):

apiVersion: metallb.io/v1beta1
kind: BFDProfile
metadata:
  name: fast-failover
  namespace: metallb-system
spec:
  receiveInterval: 300ms
  transmitInterval: 300ms
  detectMultiplier: 3
  echoInterval: 50ms
  echoMode: true
  passiveMode: false
---
apiVersion: metallb.io/v1beta1
kind: BGPPeer
metadata:
  name: router-with-bfd
  namespace: metallb-system
spec:
  myASN: 64500
  peerASN: 64501
  peerAddress: 192.168.0.1
  bfdProfile: fast-failover

Troubleshooting

Common Issues

Validating Configuration

# Check controller and speaker pod status
kubectl get pods -n metallb-system

# Check for events related to MetalLB
kubectl get events -n metallb-system

# Check controller logs
kubectl logs -n metallb-system -l app=metallb,component=controller

# Check speaker logs
kubectl logs -n metallb-system -l app=metallb,component=speaker

# View address assignments
kubectl get services --all-namespaces -o wide | grep LoadBalancer

Debugging Commands

# For Layer 2 mode, check ARP table on a client machine
arp -a

# For BGP mode, check BGP sessions on the router
# Example for a Cisco router:
# show ip bgp summary
# show ip bgp neighbors

# Test connectivity to assigned IPs
ping <external-ip>
curl http://<external-ip>

# Check MetalLB speaker's internal state (newer versions)
kubectl exec -n metallb-system -l app=metallb,component=speaker -- mirror-status

Best Practices

Security Considerations

1. Network Policies: Implement Kubernetes network policies to control traffic to your exposed services
2. Separate Network: Use a dedicated IP range for MetalLB that’s separate from your regular infrastructure
3. RBAC: Ensure proper RBAC permissions for MetalLB components
4. BGP Authentication: For BGP mode, implement MD5 authentication with your BGP peers when possible

Performance Optimization

1. Choose the Right Mode: Use BGP mode for high-traffic services that need true load balancing
2. Node Selection: For Layer 2 mode, use node annotations to influence leader election for optimal traffic distribution
3. Bandwidth Consideration: Monitor node network usage when using Layer 2 mode as traffic concentration can lead to bottlenecks
4. Resource Allocation: Ensure MetalLB components have adequate resources, especially in large clusters

Deployment Recommendations

1. Use Helm: The Helm chart provides a more manageable way to install and upgrade MetalLB
2. Multiple Address Pools: Define separate address pools for different environments or service types
3. Start Small: Begin with Layer 2 mode for simplicity, then migrate to BGP if needed
4. Regular Updates: Keep MetalLB updated to benefit from bug fixes and new features
5. Monitoring: Implement monitoring for MetalLB components using Prometheus metrics

Conclusion

MetalLB provides a robust solution for implementing LoadBalancer services in bare metal Kubernetes environments. By understanding the differences between Layer 2 and BGP modes, you can choose the approach that best fits your infrastructure requirements and operational capabilities.

For small to medium deployments, Layer 2 mode offers simplicity and ease of setup. For larger, more complex environments where true load balancing is required, BGP mode provides the necessary flexibility and scalability.

References

• MetalLB Official Documentation
• GitHub Repository
• Kubernetes Load Balancing
• MetalLB Configuration Examples
• MetalLB Troubleshooting Guide
• BGP Concepts and Configuration