Plugins

Load Balancer

Upstream routing with round-robin, weighted, least-connections, fast-response, and sticky session algorithms.

Load Balancer Plugin

The load balancer plugin routes requests to upstream backend servers using configurable algorithms. It supports multiple load balancer pools, weighted targets, health checks, and sticky sessions.

Configuration

{
  "plugins": {
    "load_balancer": {
      "enabled": true,
      "load_balancers": [
        {
          "id": "default",
          "name": "My Backend",
          "enabled": true,
          "conditions": [],
          "algorithm": {
            "type": "round_robin"
          },
          "targets": [
            {
              "id": "backend-1",
              "host": "10.0.0.1",
              "port": 8080,
              "protocol": "http",
              "weight": 100,
              "enabled": true,
              "tls": {
                "verify": false,
                "sni": null
              }
            },
            {
              "id": "backend-2",
              "host": "10.0.0.2",
              "port": 8080,
              "protocol": "http",
              "weight": 100,
              "enabled": true
            }
          ],
          "host_header": {
            "mode": "visitor"
          },
          "health_check": {
            "mode": "off"
          }
        }
      ]
    }
  }
}

Load balancer structure

Field	Type	Required	Description
`id`	string	✅	Unique load balancer identifier
`name`	string	✅	Human-readable name
`enabled`	bool	—	Enable/disable (default: `true`)
`conditions`	array	—	When to use this LB (empty = always)
`algorithm`	object	✅	Load balancing algorithm
`targets`	array	✅	Upstream backend servers (at least 1)
`host_header`	object	—	Host header forwarding mode
`health_check`	object	—	Health check configuration

Algorithms

Round Robin

Distributes requests evenly across all targets in order.

{
  "algorithm": {
    "type": "round_robin"
  }
}

Weighted Round Robin

Distributes requests based on target weights. A target with weight 200 receives twice as many requests as one with weight 100.

{
  "algorithm": {
    "type": "weighted_round_robin"
  }
}

Least Connections

Routes to the target with the fewest active connections.

{
  "algorithm": {
    "type": "least_connections"
  }
}

Fast Response

Routes to the target with the lowest EWMA (Exponentially Weighted Moving Average) latency.

{
  "algorithm": {
    "type": "fast_response"
  }
}

Sticky Session

Routes the same client to the same target based on a hash of the key.

{
  "algorithm": {
    "type": "sticky_session",
    "key": "req.client_ip"
  }
}

The key field accepts any condition path:

Key	Description
`req.client_ip`	Sticky by client IP (default)
`req.header.X-API-Key`	Sticky by API key
`req.cookie.Session-ID`	Sticky by session cookie
`req.header.X-Tenant-ID`	Sticky by tenant

Targets

Field	Type	Default	Description
`id`	string	✅	Unique target identifier
`host`	string	✅	Backend hostname or IP
`port`	number	✅	Backend port
`protocol`	string	`"http"`	`"http"` or `"https"`
`weight`	number	`100`	Weight for weighted algorithms
`enabled`	bool	`true`	Enable/disable target
`tls.verify`	bool	`false`	Verify upstream TLS certificate
`tls.sni`	string	—	Custom SNI for TLS handshake

TLS configuration

{
  "id": "secure-backend",
  "host": "backend.internal",
  "port": 443,
  "protocol": "https",
  "weight": 100,
  "enabled": true,
  "tls": {
    "verify": true,
    "sni": "api.example.com"
  }
}

When tls.sni is not set:

If target host is an IP → uses the visitor's Host header
If target host is a hostname → uses the target hostname

Host header

Controls what Host header is sent to the upstream.

Mode	Description
`visitor`	Forward the visitor's original `Host` header
`custom`	Use a custom value
`target`	Use the target's hostname

{
  "host_header": {
    "mode": "custom",
    "value": "api.internal.example.com"
  }
}

Health checks

Off mode

No health checks. All enabled targets are considered healthy.

{
  "health_check": {
    "mode": "off"
  }
}

HTTPS mode

Periodic HTTP(S) health checks — an HTTP GET to path using the target's protocol.

{
  "health_check": {
    "mode": "https",
    "path": "/health",
    "interval_seconds": 5,
    "timeout_ms": 1000,
    "expected_statuses": [200],
    "healthy_threshold": 2,
    "unhealthy_threshold": 3
  }
}

The valid health_check.mode values are https, accepts_connections, and off. There is no"http" mode — https performs an HTTP(S) GET using the target's own protocol (http or https targets both work).

Field	Type	Default	Description
`path`	string	`"/"`	Health check endpoint
`interval_seconds`	int	`5`	Check interval
`timeout_ms`	int	`1000`	Request timeout
`expected_statuses`	array	`[200]`	Healthy status codes
`healthy_threshold`	int	`2`	Consecutive successes to mark healthy
`unhealthy_threshold`	int	`3`	Consecutive failures to mark unhealthy

Accepts connections mode

Simple TCP connection check.

{
  "health_check": {
    "mode": "accepts_connections",
    "interval_seconds": 5,
    "timeout_ms": 1000
  }
}

How it works

Request arrives
    │
    ▼
1. Evaluate conditions for each load balancer
    │
    ├── No match → continue (no upstream selected)
    │
    └── Match:
        │
        ├── 2. Filter enabled targets
        │
        ├── No enabled targets → 503 "no_healthy_upstream"
        │
        └── 3. Select target using algorithm
            │
            ├── round_robin → atomic counter % len
            ├── weighted_round_robin → weight-based selection
            ├── least_connections → min active connections
            ├── fast_response → min EWMA latency
            └── sticky_session → hash(key + target_id)

            │
            ├── 4. Record selection in context
            ├── 5. Increment connection count
            │
            └── 6. Create HttpPeer → forward to upstream

Examples

Simple round-robin

{
  "id": "default",
  "name": "Backend Pool",
  "enabled": true,
  "conditions": [],
  "algorithm": { "type": "round_robin" },
  "targets": [
    { "id": "b1", "host": "10.0.0.1", "port": 8080, "protocol": "http", "weight": 100, "enabled": true },
    { "id": "b2", "host": "10.0.0.2", "port": 8080, "protocol": "http", "weight": 100, "enabled": true }
  ],
  "host_header": { "mode": "visitor" },
  "health_check": { "mode": "off" }
}

Weighted with health checks

{
  "id": "default",
  "name": "Production Pool",
  "enabled": true,
  "conditions": [],
  "algorithm": { "type": "weighted_round_robin" },
  "targets": [
    { "id": "large", "host": "10.0.0.1", "port": 8080, "protocol": "http", "weight": 200, "enabled": true },
    { "id": "small", "host": "10.0.0.2", "port": 8080, "protocol": "http", "weight": 100, "enabled": true }
  ],
  "host_header": { "mode": "visitor" },
  "health_check": {
    "mode": "https",
    "path": "/health",
    "interval_seconds": 10,
    "timeout_ms": 2000,
    "expected_statuses": [200, 204],
    "healthy_threshold": 2,
    "unhealthy_threshold": 3
  }
}

Multiple pools by path

{
  "load_balancers": [
    {
      "id": "api-pool",
      "name": "API Servers",
      "enabled": true,
      "conditions": [
        { "left": "req.path", "operator": "starts_with", "value": "/api/" }
      ],
      "algorithm": { "type": "least_connections" },
      "targets": [
        { "id": "api-1", "host": "10.0.1.1", "port": 8080, "protocol": "http", "weight": 100, "enabled": true },
        { "id": "api-2", "host": "10.0.1.2", "port": 8080, "protocol": "http", "weight": 100, "enabled": true }
      ],
      "host_header": { "mode": "visitor" },
      "health_check": { "mode": "off" }
    },
    {
      "id": "static-pool",
      "name": "Static Servers",
      "enabled": true,
      "conditions": [],
      "algorithm": { "type": "round_robin" },
      "targets": [
        { "id": "static-1", "host": "10.0.2.1", "port": 80, "protocol": "http", "weight": 100, "enabled": true }
      ],
      "host_header": { "mode": "visitor" },
      "health_check": { "mode": "off" }
    }
  ]
}

{
  "id": "session-pool",
  "name": "Session Servers",
  "enabled": true,
  "conditions": [],
  "algorithm": {
    "type": "sticky_session",
    "key": "req.cookie.Session-ID"
  },
  "targets": [
    { "id": "app-1", "host": "10.0.0.1", "port": 8080, "protocol": "http", "weight": 100, "enabled": true },
    { "id": "app-2", "host": "10.0.0.2", "port": 8080, "protocol": "http", "weight": 100, "enabled": true }
  ],
  "host_header": { "mode": "visitor" },
  "health_check": { "mode": "off" }
}

HTTPS upstream with TLS verification

{
  "id": "secure-pool",
  "name": "Secure Backend",
  "enabled": true,
  "conditions": [],
  "algorithm": { "type": "round_robin" },
  "targets": [
    {
      "id": "secure-1",
      "host": "backend.internal",
      "port": 443,
      "protocol": "https",
      "weight": 100,
      "enabled": true,
      "tls": {
        "verify": true,
        "sni": "api.example.com"
      }
    }
  ],
  "host_header": { "mode": "target" },
  "health_check": { "mode": "off" }
}

Upstream headers

The gateway adds these headers to upstream requests:

Header	Value
`X-Forwarded-Proto`	Original request scheme
`X-Forwarded-Host`	Original request host
`X-Forwarded-For`	Client IP (appended to existing)

Log output

{
  "load_balancer": {
    "lb_id": "default",
    "algorithm": "RoundRobin",
    "target_id": "backend-1",
    "target": "10.0.0.1"
  }
}

Performance

Metric	Value
Target selection	O(n) where n = targets
Connection tracking	Atomic counters
Latency tracking	EWMA (lock-free)
Sticky session hash	SHA-256

Notes

Multiple load balancers are evaluated in order — first match wins
If no load balancer matches, the request has no upstream target (will fail)
Connection counts are per-gateway-instance (not distributed)
Health checks run in a background task (when enabled)
If every target is unhealthy the gateway returns 503 with {"error": "no_healthy_upstream"}

Edit this pageorReport an issue

Cache

Edge caching with TTL, custom keys, stale-while-revalidate, stale-if-error, and singleflight revalidation.

Auto-SSL

On-demand TLS certificate provisioning via Let's Encrypt ACME (HTTP-01) with in-memory cert state.