Before you read a single number: these are loopback micro-benchmarks. They measure proxy overhead — no real network, tiny payloads, one machine. Absolute figures will differ on your hardware; the ratios and the operational tradeoffs are the signal. Never capacity-plan off someone else's laptop.

What is (and isn't) compared

omlb straddles two categories, so it has two kinds of peers — and only one of them can be measured fairly here:

• As an L7 proxy, its natural rival is nginx, and that's what the throughput and latency tables below compare. HAProxy belongs in the same list but wasn't installed on the test box, so we don't pretend to have numbers for it.
• As a failover mechanism, the peer is keepalived — both speak VRRP, and detection time is the same arithmetic on both (advert interval × missed adverts). We didn't benchmark keepalived head-to-head: it needs root for the VIP and wasn't installable in this environment, and racing an L7 proxy against a failover daemon on RPS would be a category error anyway.

Reproduce

# Unified oha-driven runner — every scenario, one methodology (recommended):
python3 bench/bench.py                       # all: L7 vs nginx, plugin, cache, security
python3 bench/bench.py l7 security            # a subset
CONC=128 DUR=15 RUNS=5 python3 bench/bench.py
bash bench/failover.sh                        # omlb HA failover time (rootless, 3 trials)

# Legacy per-scenario scripts (loadgen.go driver) still work:
bash bench/run.sh ; python3 bench/plugin_tax.py ; python3 bench/cache_bench.py

The unified runner drives everything with oha — closed-loop, JSON output, a warmup pass, median of RUNS, and a cooldown between scenarios so a long session stays internally comparable.

The legacy run.sh harness keeps the contest fair by construction: backend, proxy, and load generator each get their own disjoint CPU set (on ≥16-core machines), every proxy gets an identical 4 workers on the same 4 cores, access logs are off across the board, and one keep-alive load generator (loadgen.go) drives every target. The backend itself is a deliberately trivial Go server, so the origin is never the bottleneck being measured.

Environment

AMD Ryzen 7 PRO 8840U (16 logical CPUs), Linux 6.18, loopback, 4 proxy workers pinned to 4 cores, concurrency 64, 10 s, median of 3 runs.

Results

L7 throughput & latency

target	throughput (req/s)	p50	p90	p99
direct to backend (ceiling)	~97,000	0.51 ms	1.33 ms	2.5 ms
omlb	~62,000	0.94 ms	1.51 ms	3.0 ms
nginx	~78,000	0.60 ms	1.51 ms	3.7 ms

How we read that table:

• nginx takes raw RPS, ~78k to ~62k — call it 25% — and the lower median. A C proxy with two decades of tuning should win this, and it does.
• omlb holds ~0.8× of nginx's throughput, adds well under a millisecond at the median over hitting the backend directly, and in the stable runs its p99 came out slightly tighter than nginx's.
• Expect noise from a shared laptop — one pass dipped for both proxies while the direct-to-backend run held ~97k. That's why every figure is a median of three, and why you should re-run bench/bench.py yourself.

For a proxy this young, landing within 20–25% of nginx while carrying ACME, failover, and abuse protection in the same process is the result we wanted to see.

Failover (omlb VRRP)

bench/failover.sh, advert_interval=500ms, dead_multiplier=3:

trial	kill-master → backup owns VIP
1	1463 ms
2	1480 ms
3	1971 ms

Call it ~1.5 s in the typical case. The bound is exactly dead_multiplier × advert_interval plus one tick — the same arithmetic keepalived exposes. Want sub-second detection? Drop advert_interval to 100ms and pay for it in heartbeat chatter.

Plugin overhead

bench/plugin_tax.py isolates what a single filter costs: identical route, identical load (concurrency 64, one backend, proxy pinned to 4 cores), only the filter changes:

filter	req/s	p50	p99	vs none
none (baseline)	~118,000	0.49 ms	1.6 ms	100%
native header_inject	~102,000	0.58 ms	1.9 ms	~86%
WASM add-header — per_request	~28,000	2.2 ms	3.6 ms	~24%
WASM add-header — pooled	~71,000	0.88 ms	1.8 ms	~61%
WASM jwt (HS256/req) — per_request	~23,000	2.8 ms	4.5 ms	~20%
WASM jwt (HS256/req) — pooled	~61,000	1.0 ms	2.1 ms	~52%

What this table actually says:

• Native filters barely register. They mutate the live Pingora request through zero-copy accessors — no snapshot, no header-map rebuild — and hold ~86% of baseline. That's the right tool for anything always-on. (On loopback this bench is I/O-bound, so removing per-request allocations doesn't move the throughput needle much — what it buys is tail latency against real, slower backends.)
• The WASM tax is instance bring-up, not serialization. We profiled a fresh call (~16 µs, single thread): instantiation ~2.6 µs, guest code ~2.5 µs, the guest's first allocation (dlmalloc init + memory.grow) ~5.8 µs, host serialization ~0.16 µs. That's why switching the wire format from JSON to postcard barely mattered — and why a warm instance costs ~1 µs per call.
• So the pooling knob is where the performance lives. Reusing warm instances — the proxy-wasm/Envoy model — is ~2.5× faster end to end: add-header climbs from 24% to 61% of baseline, JWT from 20% to 52%. The price is that guest heap state can survive between requests on a pooled instance. The default, per_request, builds a fully isolated Store per call (via InstancePre, a pooling allocator, and epoch interruption), so a misbehaving plugin can only ruin its own request. Pick pooled for stateless filters you trust on hot paths.

Our own rule of thumb: native for the highest-QPS always-on logic; WASM pooled for sandboxed-but-hot things like auth, WAF rules, canaries; WASM per_request when isolation is the point. And put response caching in front of any of them — a hit skips the whole chain.

Response cache

bench/cache_bench.py runs one omlb against one backend and compares two paths through it — one route cached (so after the first request, everything is a memory hit), one route proxying every request:

path	req/s	p50	p99
cache HIT	~137,000	0.38 ms	1.7 ms
upstream (no cache)	~92,000	0.61 ms	2.3 ms

The honest multiplier here is "only" ~1.5× — and that understates the feature, because our un-cached path proxies to a trivial Go server on loopback, which is nearly free already. The interesting property is that a hit's cost is decoupled from the origin: ~137k req/s whether the thing behind you is fast, slow, distant, or on fire. Against a real origin the gap widens dramatically, every hit is one request the origin never sees, and hits short-circuit ahead of the response-phase filters — a cached route with plugins skips their cost too.

Caching stays opt-in per route (cache: { ttl, max_body }), takes only unambiguously cacheable GETs, and lives in a bounded LRU (server.cache_max_entries). Watch it through omlb_cache_hits_total and omlb_cache_misses_total.

What you operate, compared

The chart that actually decides deployments isn't RPS — it's how many daemons you operate. The classic HA L7 answer is two of them, each with its own config language; omlb's answer is one process and one file.

	omlb	nginx (OSS)	HAProxy	keepalived	Envoy
L7 HTTP proxy	✅	✅	✅	❌ (L4/IPVS)	✅
Active health checks	✅	⚠️ Plus-only	✅	✅ (IPVS)	✅
Passive outlier / circuit-break	✅	⚠️ basic	✅	❌	✅
Built-in VRRP self-HA + VIP	✅	❌	❌	✅	❌
WASM plugins	✅	❌ (Lua/njs)	❌ (Lua)	❌	✅ (Proxy-Wasm)
Hot config reload	✅ (subset)	✅ (worker reload)	✅ (hitless)	✅ (SIGHUP)	✅ (xDS)
Single binary, no runtime deps	✅	✅	✅	✅	✅
L7 and HA in one process/config	✅	❌ (+keepalived)	❌ (+keepalived)	❌ (+proxy)	❌ (+keepalived)
Maturity / battle-tested	🆕 young	✅✅✅	✅✅✅	✅✅✅	✅✅✅

Read it plainly: omlb's win is integration — balancing, both kinds of health detection, retries, VRRP, and hot-swappable WASM in one binary — while the incumbents keep raw speed and years of production scar tissue. Both facts belong in the same sentence.

Do our numbers match everyone else's?

Comparing absolute numbers across other people's benchmarks is unsound — different hardware, payloads, concurrency, keep-alive settings, methodology. What is worth checking is the shape: does our omlb-vs-nginx relationship match what others independently measure for Pingora (the engine under omlb) against nginx?

• It does. Independent testing of Pingora against nginx lands on "nginx performs slightly better on throughput, while Pingora shows slightly lower latency" (pingora#372) — the same shape we see: nginx ~25% ahead on RPS, omlb holding a comparable-to-tighter tail. Whatever gap remains on top of that is our per-request overhead above Pingora, not the engine's ceiling — which is precisely what the recent hot-path work went after.
• It's also worth remembering what Cloudflare actually got out of Pingora in production: not a bigger RPS number, but ~70% less CPU and ~67% less memory than their nginx-based stack, mostly from cross-thread connection pooling — reuse hit ~99.9% for a large customer (how we built Pingora). Architectural wins, not drag-race wins — the same category omlb competes in.
• And for scale on the numbers themselves: public VPS benchmarks put nginx around ~40k+ RPS and HAProxy ~50k+ on 4 cores, with steady-state memory near 50 MB for HAProxy, 80 MB for nginx, 150 MB for Envoy (onidel 2025). Our figures run far higher because loopback and a 102-byte body remove the network from the equation — one more reason to read every number here as a ratio, never a promise.

Bottom line: this is where a young Pingora-based proxy is supposed to land — 20–25% behind nginx on synthetic throughput, competitive on latency — and its case was never going to be a loopback chart. It's the daemons you don't run.