Code mode: let the model write the code

Arion MCP gateway turns REST services, OpenAPI documents and upstream MCP servers into one catalog of tools that agents can discover and call. Arion Code mode provides an alternative way for agents to discover and interact with MCP tools.

Arion MCP code mode works similarly to how code mode works in agent harnesses, but it runs in the context of the gateway. Every tool call made through Arion's code mode SDK goes through the full gateway policy stack in the same way a direct MCP tool call would.

Why support Code Mode

With conventional MCP, an agent calls one tool, waits, reads the result and decides the next call, over and over. Take a routine support request:

1. Find the customer by email.
2. Fetch their recent tickets.
3. Fetch their billing plan.
4. Open a ticket, at high priority if they are on the enterprise tier.

In many agent harnesses, this results in at least four tool calls, with model turns and eight crossings between the model and the gateway. This introduces additional latency. Worse, if the agent harness does not provide code mode natively or a similar optimization method, then all the raw responses are added to the model's context, including the ticket history it needed only long enough to count. This is both inefficient and wasteful.

A large catalog can add a lot of unncessary cost before any work starts. In many settings, adding a few MCP servers can result in tens and often hundreds of tool definitions that the model has to select from. Tool search helps, but, until MCP progressive discovery is standardized, the harness again needs custom code to effectively use tool search.

The model ends up doing coordination work that a few lines of code could do faster, more cheaply and more reliably.

The approach: one tool, one script

Code mode is enabled in the MCP gateway configuration. When code mode is enabled, it exposes exactly one tool called execute. The model, following instructions from the execute tool description generates code to call the Arion SDK and passes the program to Arion, Arion then runs the provided code in a sandbox. The code can call any MCP tool the agent is allowed to use, combine results, loop, branch and run calls in parallel, then return only the overall result of the program.

The support request becomes one script:

const customer = await tools["crm.find_customer"]({ email: "ada@example.com" });
if (!customer.ok) return { error: "unknown customer" };

const [history, plan] = await Promise.all([
  tools["support.recent_tickets"]({ customer_id: customer.data.id }),
  tools["billing.current_plan"]({ customer_id: customer.data.id }),
]);

const ticket = await tools["support.create_ticket"]({
  ticket: {
    customer_id: customer.data.id,
    priority: plan.ok && plan.data.tier === "enterprise" ? "high" : "normal",
    summary: "Refund not received",
  },
});
return { ticket: ticket.data.id, seen: history.data.items.length };

The history and plan lookups run in parallel, the script branches on the plan, and the model gets back a small object instead of four raw responses. The four calls never touch its context.

Some small design choices help make this practical.

The tool describes itself, per caller. The execute description is written fresh for each caller. It lists the toolkits and namespaces that caller is allowed to see, and how to find more. It has a 4 KiB budget, so a gateway with thousands of tools still gives the model a compact starting point instead of a wall of text.

Discovery happens inside the script. A small catalog API lets the script search, list and describe tools on demand, so the model pulls in only the definitions it needs. catalog.tools.search({ query }) uses the same BM25 or embedding ranking as tool search, but without the extra round trip.

Every call returns one predictable shape. A tool call resolves to { ok: true, data } or { ok: false, error: { code, message, retryable } }. A failure is a value, not an exception, so the script checks ok and moves on, and failures stay readable.

On top of that, operators can give the model a head start. A toolkit is a named bundle of related tools with a short "how to use them together" note, authored in configuration. Where the raw tool list tells the model what exists, a toolkit guides the model in how common tasks can be performed: find the customer, then open a ticket, then notify the owner. A toolkit appears only if the caller may use every tool in it, so partial bundles never leak, and toolkits can be added or removed live over xDS. Toolkits are somewhat similar to skills, but are more tightly coupled to describing how to use a subset of tools from the catalog to achieve a specific task.

How it runs

Code mode is just an alternative operating mode for Arion MCP gateway. It changes how tools are called, never what is allowed.

Policy is always the same for both modes. A tool call from a script travels the same path as an ordinary tools/call. It lands on the one shared tool registry, which checks that the caller is permitted, validates the arguments against the tool's schema, enforces size and time limits, and records the same metrics. A script cannot reach anything its caller could not already reach, and it cannot skip a single check. The registry was reorganized into small, focused pieces so that both modes share it with no duplicated logic to drift apart.

Sandbox runtime Agent provided code runs in WASM using Wasmtime, a mature WebAssembly runtime, on QuickJS, a small and fast JavaScript engine compiled as a WASI component. Inside, the module JS runtime is highly constrained: no files, no network, no environment. The only interaction with outside world is through a very limited set of host calls: logging, tool invocations or tool catalog search.

Fast to start, nothing running when idle. The engine is initialized ahead of time with Wizer, so each run starts from a snapshot rather than booting an interpreter. Instances come from a pool, so starting one is quick. Each run still gets a fresh, empty environment, so one caller can never see another's data. The thread that enforces the compute budget runs only while a script is executing, so an idle gateway does no background work.

Two clocks, and limits on everything. A 5-second compute budget stops runaway code. A total-time limit, 30 seconds by default, bounds the whole run. Waiting on a slow backend does not count against compute time. Memory (64 MiB), tool calls per run (100), runs at once per listener (32), parallel calls within a run (8), result size (1 MiB) and script size (256 KiB) are all bounded too. A script that hits the call limit cannot hide the error by catching it. Excess runs are turned away rather than queued. Operators also set a process-wide ceiling on sandboxes and memory, and per-listener settings cannot exceed it.

Result Composition A run returns a single object: a status, the script's return value, any log lines, and a list of every tool it called with timing and outcome. If that object is too big, the gateway keeps the actual answer, trims the extras and says exactly what it trimmed. A script error gets a plain reason and a line number for the agent to resolve.

What it changes

For the support request above, the comparison is plain:

	Tools mode	Code mode
Model turns	4	1
Crossings between model and gateway	8	2
Tool responses in the model's context	4, in full	0; one small object
Tool definitions offered up front	every listed tool	one, execute

Those are counts from the example. The backend calls take as long as they always did, and the two lookups now run in parallel. The model turns in between are gone, and those turns are usually where the time and the tokens go. Workflows with more steps or more fan-out gain more. We have not published timings for code mode yet, so we are not quoting any here.

The catalog scales the same way. tools/list returns one tool whether the gateway serves ten tools or ten thousand, and the model reads a definition only when its script asks for one.

Trade-offs

• The model has to write working code. The errors are built for a model to act on: a parse error gives a line and column, and a failed call is a value with a code. Toolkits cut the guesswork further. If it takes a model multiple attempts to generate the correct program, some benefits are lost.
• A listener is one mode or the other. A code-mode listener serves only execute. Clients that expect the classic tool list need a separate listener in tools mode. Both can front the same tools.
• Scripts keep no state. Every run starts fresh. Anything that must survive between runs belongs in your tools.
• The TypeScript is strip-only. Annotations are removed, but enum, namespace declarations and decorators are rejected. In practice, they are typically not necessary.
• It is opt-in at build time. Code mode sits behind the codemode Cargo feature, which adds the Wasmtime runtime and a sandbox component of just under 1 MiB. A build without it rejects a code-mode configuration cleanly instead of misbehaving.

Turning it on

Build with --features codemode, then pick the mode per listener:

typed_config:
  "@type": type.googleapis.com/arion.extensions.filters.http.mcp.mcp_gateway.v3.McpGateway
  server_info:
    name: support-tools
    version: "1.0.0"
  code_mode: {}

Every setting has a sensible default. Tools, upstream MCP servers, OpenAPI sources and toolkits can all be delivered and changed at runtime over xDS. It is still standard MCP, so existing clients keep working; the model just sees one tool. The sandbox itself can be swapped for a vetted custom build, which Arion checks when the listener loads.

What is next

To try it end to end, follow code mode for agents. It builds the support workflow above against three REST services. The code mode page covers when to use it, and the SDK reference lists every call, error code and limit. For a quick way to fill the catalog, pair it with OpenAPI tools: point a source at an API's spec, and every operation becomes a tool that a script can call.