Benchmarks · Throughput and latency

Arion, an ultra high performance data plane

Arion brings nginx-level forwarding performance to a fully programmable gateway. Compare HTTP forwarding, MCP tool calls and request processing under load.

Read about the test hardware and methodology behind these measurements.

HTTP forwarding 2.0M requests per second with HTTP/1.1 forwarding MCP gateway 6.8× agentgateway’s throughput for REST-backed tool calls External processing 3.8× Envoy’s throughput with request-header processing

01 · HTTP forwarding

Fast forwarding under load.

Small requests, persistent connections and a static upstream. Explore plain HTTP, TLS and HTTP/2, then see how throughput changes as concurrency grows.

HTTP/1.1 → HTTP/1.1 · 24 proxy CPUs · 1,024 concurrent connections · persistent connections

Throughput requests/s · higher is better

p99 request latency milliseconds · lower is better

Around 2 million requests per second: forwarding performance on par with nginx, with 5× the throughput of Envoy.

Compare the numbers

Gateway	Requests/s	p50 · ms	p90 · ms	p99 · ms
Arion	2.01M	0.497	0.67	0.849
nginx	1.97M	0.51	0.628	0.865
Envoy	398.5K	1.124	7.003	11.151
agentgateway	418.0K	2.283	3.613	4.963
HAProxy	1.22M	0.809	1.087	1.62

Read the analysis: HTTP baselines and caveats · Four-CPU deployments.

02 · MCP gateway

Fast tool calls, across backend mappings.

Serve REST APIs as MCP tools, or forward calls to an upstream MCP server with JSON or SSE responses. Compare these gateway workloads with plain MCP traffic forwarding, where no MCP-aware filter runs on the proxy.

MCP → REST · query parameter mapping · 24 proxy CPUs · concurrency 1,024

Throughput tool calls/s · higher is better

p99 tool-call latency milliseconds · lower is better

Over 1 million tool calls per second, with 6.8× the throughput of agentgateway for query-mapped REST tools.

Compare the numbers

Gateway	Calls/s	p50 · ms	p90 · ms	p99 · ms	HTTP 5xx
Arion	1.23M	0.823	1.039	1.386	0
agentgateway	180.1K	5.123	9.215	13.399	0

Read the analysis: MCP forwarding · REST-backed tool mappings · Native MCP upstreams.

Path parameters 6.6× agentgateway throughput

Query parameters 6.8× agentgateway throughput

Request body 6.4× agentgateway throughput

03 · Programmable request processing

Dynamically program the request path with custom code.

Run custom Wasm filters or call external processors. Compare throughput and latency with custom code executing on every request.

Wasm filters

Execute a filter or exercise the header-map API on each request.

External processing

Process request headers, or headers and body, over gRPC.

Wasm benchmark filter · HTTP/1.1 · 24 proxy CPUs · concurrency 1,024

Throughput requests/s · higher is better

p99 request latency milliseconds · lower is better

Arion sustains over 1 million requests per second while executing a Wasm filter on the request path.

Compare the numbers

Gateway	Requests/s	p50 · ms	p90 · ms	p99 · ms	HTTP 5xx
Arion	1.8M	0.562	0.72	0.86	858
nginx	1.3M	0.788	0.936	1.066	0
Envoy	344.9K	1.281	7.395	12.287	0

04 · Run the benchmarks

See for yourself.

Run the benchmark suite on your own hardware, explore the configurations or dig into the full results behind these comparisons.

Benchmark harness

The sweep runner, load-generator commands and utilities for comparing throughput and latency.

Run the benchmarks

Results and analysis

Workload analysis, caveats, measurement data and proxy configurations from this benchmark campaign.

Explore the results and analysis

These results are indicative of performance across the workloads shown. Throughput and latency depend on your hardware, traffic and configuration. Each comparison uses the workload and concurrency shown above its charts. Published full-scale measurements used 24 proxy CPUs; the reproduction configs now use 32. See the reproduction notes for details.