Arion is an open-source L7 gateway written in Rust. It reads Envoy configuration natively and runs as an API gateway and proxy, an LLM gateway and an MCP gateway — one data plane for services, models and tools.
This is the first post on this blog, so before we write about internals and benchmarks, we want to start with the story: where Arion came from, and why we built it.
Where Arion came from
The project behind Arion began as a research project and was later donated to the Kmesh GitHub organisation as an open-source subproject called Orion. Arion is an independent fork of Orion, led by its original authors and long-term maintainers. Arion builds on what we learned over a number of years of developing a high-throughput data plane for cloud services.
We started this work initially while employed in Huawei's European research labs, aiming to address the many challenges that Cloud service teams and product teams had running large fleets of proxies and gateway services at scale. It was clear that the tools available to them each solved only half of the problem.
We used nginx heavily, and on raw performance it is outstanding. But its configuration model comes from a world where configuration changes rarely: pushing changes across large fleets of nginx instances, continuously and safely, is brittle and error-prone.
Envoy solves exactly that problem: it is battle-tested, and its dynamic configuration over xDS lets whole fleets reconfigure continuously without dropping traffic. But Envoy is heavy. It uses a lot of CPU and memory for the traffic it moves, and its throughput sits far below nginx's.
Neither system gave us both. We wanted the performance of nginx and the operability of Envoy in one data plane.
A better Envoy, in Rust
We set out to build a better Envoy. The requirements were written down early and never really changed:
- Dynamically reconfigurable. Configuration arrives over xDS and applies without pausing traffic.
- Stable and predictable. Every request passes through the proxy, so it has to run reliably for long periods and fail in predictable ways.
- Simple to deploy and operate. One binary, one config file or one control-plane connection.
- Lightweight. Frugal with memory and CPU, because at fleet scale resource usage translates directly into cost and capacity.
Rust made those goals compatible. It gives us memory safety without a garbage collector: whole classes of memory bugs are ruled out at compile time, with no runtime cost on the request path. And speaking Envoy's configuration natively, rather than inventing our own, meant the existing ecosystem of control planes and tooling carried over.
The performance target was clear from production experience running high-throughput services: nginx. For our scenarios it has always been the fastest option available, and that was the bar we set for Arion.
From prototype to production
The codebase behind Arion grew in stages: first a prototype, then a pre-production system, then versions of it carrying real production traffic.
Arion builds on learning and experience gained from an internal production version deployed across thousands of instances powering AI cloud services. We now carry that work forward independently through Arion.
Building for stability in production shaped the design. A few early decisions — configuration updates delivered as messages rather than guarded by locks, a data path that scales by adding whole runtimes, nothing on the hot path that can block — were made for stability and operability first. The same decisions, it turns out, are what make a proxy fast. The architecture page describes how those pieces fit together.
We kept tuning, and release after release the gap to nginx narrowed, then closed. Arion now matches nginx, and beats it in a number of scenarios. Against agentgateway, Envoy and other gateways with similar feature sets, Arion delivers multiples of their throughput. The benchmarks show the numbers and exactly how we produce them, so you can rerun them yourself.
What Arion is today
A short tour, with the details left to the docs:
- An L7 and L4 proxy and API gateway. Routing, load balancing, TLS, JWT authentication, rate limiting, circuit breaking, health checks — the most commonly used Envoy features, described as standard Envoy configuration. Start with What is Arion.
- Envoy-compatible, strictly. If Arion accepts a configuration meant for Envoy, it behaves like Envoy. If it cannot honor one, it refuses to start rather than silently ignoring part of it. The compatibility page defines the supported subset, and there is a guide for migrating from Envoy.
- A gateway for AI traffic. An LLM gateway puts one governed front door in front of model providers, and an MCP gateway serves tools to agents — including code mode, where an agent sends one program instead of many tool calls.
- Programmable. WebAssembly filters with a Rust SDK, native Rust filters, and external processing over gRPC, for the cases configuration cannot express.
What's next
A lot has changed in the last couple of years. AI and agentic workloads emerged as a new and important class of traffic. It turned out that the properties we built for scale — a fast, safe, dynamically configurable data plane that is simple to extend — are exactly what that traffic needs. Most of Arion's recent development has gone into these AI features, and most of what is coming next will too. The posts that follow this one will go deeper: how we benchmark, how the runtime is structured, and what we learned building AI features on a proxy.
If you want to try Arion, the quickstart proxies your first request in five minutes. The code is on GitHub, and the community page lists where to ask questions. We look forward to seeing what you build with Arion.