Testing & Validation

The all-rules config and 57-assertion live test suite that proves every documented behavior.

Testing & Validation

The gateway ships a live end-to-end test harness that exercises every operator, every condition path, and every plugin feature against a running gateway — not unit mocks, but real HTTP requests with observable outcomes (status codes, response bodies, log events).

Two artifacts make this up:

Artifact	Purpose
`config.all-rules.json`	One gateway config that turns on every feature at once
`scripts/test-all-rules.sh`	Boots the stack, fires 57 assertions, prints PASS/FAIL, exits non-zero on any failure
`scripts/all-rules-echo.py`	Minimal echo origin (reports which instance answered) used as upstream

Quick start

cargo build --release
bash scripts/test-all-rules.sh

Requirements: bash, curl, python3. The script is CI-friendly — it exits 0 only when all 57 assertions pass.

To keep the stack running after the tests for manual poking:

KEEP=1 bash scripts/test-all-rules.sh
curl http://127.0.0.1:8099/lb/rr

Component	Address
Gateway (HTTP)	`http://127.0.0.1:8099`
Gateway (HTTPS)	`https://127.0.0.1:8499`
Echo origin A	`127.0.0.1:9099`
Echo origin B	`127.0.0.1:9100`
Access log	`/tmp/quayel-all-rules-access.jsonl`
Gateway log	`/tmp/quayel-all-rules-gw.log`

What `config.all-rules.json` covers

Area	Coverage
Firewall	All 15 operators, every condition path family (`req.`, `ip.`, headers incl. JWT claims, query, cookie, `http.*` aliases), pure `and`/`or`/mixed logic chains, all 5 actions
Rate Limiting	Global catch-all plus single-key, composite, JWT-claim, legacy template, and cookie+query keys; an ip-intel-conditioned rule
Cache	Custom and origin-driven edge TTLs, browser TTL modes (`custom`/`origin`/`bypass`), custom variant cache keys, `bypass` for cookied requests, SWR/SIE, operator conditions
Load Balancer	All 5 algorithms (`round_robin`, `weighted_round_robin`, `least_connections`, `fast_response`, `sticky_session`), all host-header modes, health-check modes `https`/`accepts_connections`/`off`, a TLS target, and a catch-all pool
Logging	JSONL file mode with buffered writes
Auto-SSL	Present but disabled (staging mode) so the full schema is exercised without hitting Let's Encrypt

The JWT used by claim-based conditions and keys is a self-signed token with payload {"user_id":"u-42","data":{"role":"admin"}}, minted inline by the harness — no real identity provider needed.

The 8 assertion groups

#	Group	What it proves	Count
1	Firewall: all 15 operators	Each operator matches/negates correctly on real requests	15
2	Firewall: condition paths	Every left-hand path family resolves (`req.host`, `req.ip`, `req.peer_ip`, JWT claims, `http.` aliases, all `ip.` fields)	16
3	Firewall: logic + actions	OR chains match any branch; mixed AND/OR composes; `redirect` returns the configured status + `Location`	6
4	Header-injection visibility	A firewall `set_request_header` is visible to later plugins (proven via a rate-limit key)	1
5	Rate limiting: key modes	Every key mode buckets independently and returns 429 after `limit` requests	6
6	Cache	MISS→HIT behavior across TTL modes, variant keys, and bypass	5
7	Load balancer	Round robin alternates targets, sticky sessions pin, every algorithm routes	5
8	Logging	JSONL events written for cache hits, rate limits, and every expected firewall action	3

How assertions work

The harness uses four assertion helpers, each keyed on an observable outcome:

assert_status  "name" 200        curl-args...   # exact status code
assert_block   "name" 403 rule-id curl-args...  # status + rule_id in the JSON block body
assert_action  "name" rule-id Action curl-args... # request passes (200) AND the action
                                                  # appears in the access log
rl_burst       "name" N          curl-args...   # N-th request in a burst is 429

Blocked responses carry the rule identity in the body, which is what makes assertions precise:

{"error":"blocked","status":403,"rule_id":"fw-op-contains"}

Pass-through actions (allow, set_request_header, set_response_header) don't stop the request, so assert_action asserts two things at once: the request reached the origin (status 200) and the firewall action was written to the JSONL log. A final sweep verifies that every expected (rule_id, action) pair appears in the log.

Verified behaviors

The suite pins down several semantics that are easy to get wrong:

set_request_header lives in the request context, not on the upstream request. Injected headers are visible to later plugins — condition resolution, rate-limit keys, cache keys — but are not forwarded to the upstream request. The gateway adds its own X-Forwarded-* headers upstream. Group 4 proves this: a rate-limit rule keyed on the injected header buckets per injected value.

set_response_header is currently a no-op on the wire. The rule matches, is logged, and the request continues — but no response header is set. Treat it as reserved for future use.

Rate-limiting rules may have empty conditions (a global limit applies to every request), while firewall rules must declare at least one condition — startup rejects empty firewall conditions. Use a rate-limit rule with "conditions": [] for catch-all quotas.

First match wins — even for pass-through actions. A matched allow or set_request_header rule stops evaluation of later rules. Order your rules from specific to general.

Reading the output

════════════════════════ ALL-RULES RESULTS ════════════════════════
  PASS  op:equals(req.method)                       status=421 want=421
  PASS  op:not_equals(req.scheme)                   status=200 rule=fw-op-not-equals action=SetRequestHeader
  ...
  PASS  logging:firewall action events for all rules expected=18 missing=0
────────────────────────────────────────────────────────────────────
  TOTAL: 57   PASS: 57   FAIL: 0
════════════════════════════════════════════════════════════════════

Each line shows the observed value next to the expectation, so a failure is diagnosable from the report alone. On any failure the script exits 1.

Extending the suite

Add the feature to config.all-rules.json (give the rule a descriptive id like fw-op-… / rl-… / cache-…).
Add an assertion to the matching group in scripts/test-all-rules.sh.
For pass-through firewall actions, register the expectation with expect_log <rule_id> <Action> so the final log sweep covers it.
Re-run — the report must show FAIL: 0, and the new assertion must be counted in TOTAL.

Run with KEEP=1 and poke the gateway by hand with curl while tailing /tmp/quayel-all-rules-access.jsonl — the JSONL stream shows exactly how each request was evaluated.

Conditions — operators, paths, and logic chains the suite exercises
Logging — the JSONL format assertions are matched against
Versioning — config_version conventions used by the test config
AI Agent Guide — machine-readable contract validated by this suite

Edit this pageorReport an issue