Arion is a proxy and API gateway written in Rust that reads Envoy configuration natively — use it where you would put Envoy at the edge or in front of services, and where your traffic now includes agents calling tools.
Its configuration is generated from Envoy's xDS protobuf definitions and consumed the same way Envoy consumes it: as a static bootstrap file or dynamically from the Arion control plane or another compatible delta ADS server. On top of that base it adds its own extensions for AI traffic: an LLM gateway and an MCP gateway.
One data plane, multiple gateways
LLM gateway. Arion puts one endpoint in front of several model providers, with failover between them, token budgets and guardrails, so applications talk to models through the same governed front door as everything else. Its documentation is under construction and lands with the feature.
AI inference gateway. For self-hosted model servers on Kubernetes, Arion combines Gateway API with the Inference Extension. The controller builds a pool of ready Pods from an InferencePool, and Arion uses ext_proc to ask an external endpoint picker where to send each request. Start at the AI inference gateway overview or go straight to the inference routing guide.
MCP gateway. Arion can serve tools to agents over the Model Context Protocol. An HTTP filter exposes an MCP endpoint and owns a tool registry. Tools are configured statically, expanded from OpenAPI documents, discovered from upstream MCP servers, or delivered incrementally over xDS. REST backends let existing HTTP services appear as tools without writing an MCP server, and RBAC controls who may call what. In code mode an agent sends one TypeScript script that composes many tools, and Arion runs it in a sandbox, so a whole workflow takes one exchange instead of many model turns. Start at the MCP gateway overview.
L7 and L4 proxy, and API gateway. Arion terminates HTTP and TCP connections and applies the commonly used Envoy features: routing, load balancing, TLS, JWT authentication, local and global rate limiting, circuit breaking, connection limits, health checks and external processing. You describe all of it as standard Envoy static_resources, or deliver it over xDS.
Extensible data plane. When configuration is not enough, you can program the proxy: write HTTP filters in WebAssembly with the Rust SDK and load them at runtime, or call out to an external processor over gRPC with ext_proc. See extending Arion.
When to choose Arion
You already speak Envoy. Arion accepts Envoy configuration and makes a strict promise about it: if Arion accepts a configuration meant for Envoy, it behaves essentially like Envoy. If it cannot honor a config the way Envoy would, it refuses to parse the file — or rejects the update, when configured over xDS — rather than silently ignoring any part of it. A config that starts is a config that works. The Envoy compatibility page describes the supported subset; migrating from Envoy walks through a real move.
You really care about performance and throughput. Arion achieves serious performance. It achieves multiple times the performance of other Gateways in the ecosystem that have similar feature sets. See the Benchmarks for more details.
You want a programmable proxy. The Wasm filter host and ext_proc give you a place to put custom authentication, routing decisions or body rewriting without forking the proxy or writing C++. Arion can run Wasm extensions at blazingly fast speeds (over 1M RPS).
You want a small operational surface. Arion builds into a single binary named arion, configured by one file (or one control plane connection). Being Rust, it avoids entire classes of memory-management and data-race bugs, which makes it a sound foundation for the piece of infrastructure every request passes through.
Next
- How Arion works — listener, filter chain, route and cluster in one picture.
- Install — build the binary or the Docker image.
- Quickstart — proxy your first request in five minutes.