---
title: "External processing"
url: "/docs/extend/ext-proc/"
kind: "feature"
summary: "Stream requests and responses to a gRPC service you run: processing modes, mutations, timeouts, failure mode, and per-route overrides."
---

> Arion site index: [/llms.txt](/llms.txt)

# External processing

Stream requests and responses to a gRPC service you run: processing modes, mutations, timeouts, failure mode, and per-route overrides.

The `ext_proc` filter streams request and response phases over a bidirectional gRPC stream to an external processor — a service you write in any language, running anywhere Arion can reach. Use it when your logic should live outside the proxy: existing code, a language other than Rust, or transformations too heavy for in-process filters, at the cost of one network hop per processed message.

## Enable it

Add the filter to `http_filters` and point `grpc_service` at a cluster for your processor (the cluster must speak HTTP/2):

```yaml,arion=fragment
http_filters:
  - name: envoy.filters.http.ext_proc
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
      failure_mode_allow: false
      message_timeout: 5s
      processing_mode:
        request_header_mode: SEND
        response_header_mode: SKIP
        request_body_mode: NONE
        response_body_mode: NONE
      grpc_service:
        envoy_grpc:
          cluster_name: ext_proc_server
```

```yaml,arion=fragment
clusters:
  - name: ext_proc_server
    type: STATIC
    typed_extension_protocol_options:
      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
        "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
        explicit_http_config:
          http2_protocol_options: {}
    load_assignment:
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address: { address: 127.0.0.1, port_value: 18080 }
```

`grpc_service` also accepts `google_grpc` with a `target_uri`, and a `timeout` for the stream.

## Processing modes

`processing_mode` decides which phases reach the processor. Headers: `DEFAULT` (send), `SEND`, `SKIP`. Bodies: `NONE` (default), `STREAMED`, `BUFFERED`, `BUFFERED_PARTIAL`, `FULL_DUPLEX_STREAMED`. Trailers: `DEFAULT` (skip), `SKIP`, `SEND`. With no `processing_mode` at all, both header phases are sent and nothing else. In `BUFFERED` mode the body is capped at 100 MiB by default; override the cap with `max_receive_message_length` on `envoy_grpc`. Oversized bodies are rejected with 413.

An inference endpoint picker may need the whole request body before answering the request headers. Use `FULL_DUPLEX_STREAMED` with `send_body_without_waiting_for_header_response: true` to let the body reach the processor while Arion waits for that answer. Response phases can also pass through the processor, including streamed inference responses.

## What a processor can return

- **Header mutations** — set and remove headers. Omitting `mutation_rules` applies no mutation-rule restrictions. A configured empty rules object protects `host`, `:authority`, `:scheme`, `:method` and `x-envoy-` headers; use fields such as `allow_all_routing`, `allow_envoy` and `disallow_system` to adjust it. `set_headers` replaces an existing value by default; `append: true` requests appending.
- **Body mutations** — replace or clear the body in body-processing modes.
- **An immediate response** — status, body, headers, and optional `grpc-status`, sent straight to the client. `disable_immediate_response: true` turns these into errors instead.
- **Route cache clearing** — after mutating routing-relevant headers, the request is re-matched against the route table; `route_cache_action` (`DEFAULT`, `CLEAR`, `RETAIN`) forces the behaviour.
- **Mode overrides** — with `allow_mode_override: true`, the processor can switch later phases on or off per request, limited to `allowed_override_modes` when set, and extend the clock with `override_message_timeout`.

## Timeouts and failure

Arion waits `message_timeout` (default 200ms) for each processor reply; `max_message_timeout` caps what `override_message_timeout` may request. What happens on timeout or stream error is `failure_mode_allow`:

- `false` (default) — fail closed: processor message timeouts return 504; processor or stream errors return 500.
- `true` — fail open: the request continues after the processor failure.

`observability_mode: true` makes the stream fire-and-forget — phases are mirrored to the processor, replies are not waited for and cannot mutate traffic.

## Per-route override

Routes and virtual hosts can disable the filter or replace its settings via `typed_per_filter_config`, keyed by the filter's chain name:

```yaml,arion=fragment
typed_per_filter_config:
  envoy.filters.http.ext_proc:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExtProcPerRoute
    overrides:
      processing_mode:
        request_body_mode: BUFFERED
      failure_mode_allow: true
```

Set `disabled: true` instead of `overrides` to skip the processor on that route. Per-route `grpc_service` lets one route talk to a different processor. These overrides work with both inline `route_config` and routes delivered over RDS.

## Endpoint feedback

Set `metadata_options.forwarding_namespaces.untyped: [envoy.lb]` on the filter to send the actual served endpoint back to the processor with the response headers. For a cluster using the `OverrideHost` load-balancing policy, Arion includes `x-gateway-destination-endpoint-served` in the `envoy.lb` namespace of `ProcessingRequest.metadata_context`. This reports the endpoint that served the response, including when endpoint selection used the fallback balancer.

This is a specific endpoint-feedback capability. Other untyped namespace names are accepted but do not enable arbitrary metadata forwarding. Typed forwarding, receiving metadata and cluster-metadata forwarding are unsupported. Per-route overrides inherit this filter-level setting; they cannot override `metadata_options`.

## Verify

With `failure_mode_allow: false` and the processor not running, requests fail closed:

```bash
curl -si http://127.0.0.1:8080/ | head -1
```

```text
HTTP/1.1 500 Internal Server Error
```

Start your processor and the same request goes through.

## Limits

- The processor must be gRPC; Envoy's `http_service` variant is not supported and refuses to start.
- The `GRPC` body send mode and attribute forwarding (`request_attributes`, `response_attributes`) are not supported. Metadata forwarding is limited to the endpoint feedback described above.
- `forward_rules` (`allowed_headers` / `disallowed_headers`) controls which headers are forwarded to the processor; everything else about the stream follows the Envoy ext_proc protocol.

The [AI Inference Gateway overview](@/docs/ai-inference-gateway/overview.md) explains how the endpoint picker and proxy work together. Follow the [inference routing guide](@/guides/inference/inference-routing.md) to configure inference traffic; field details are in the [ext_proc reference](@/reference/filters/http/ext-proc.md).