---
title: "Performance tuning"
url: "/docs/operate/performance-tuning/"
kind: "feature"
summary: "The runtime knobs: worker threads, one runtime versus per-core runtimes, CPU affinity, the jemalloc build and socket basics."
---

> Arion site index: [/llms.txt](/llms.txt)

# Performance tuning

The runtime knobs: worker threads, one runtime versus per-core runtimes, CPU affinity, the jemalloc build and socket basics.

Arion's defaults — one multi-threaded runtime sized to the CPUs it detects — are sensible for most deployments. This page explains what each knob in the top-level `runtime` section trades, so you change them for a reason. There are no magic numbers here: measure your own workload before and after.

## The knobs

```yaml,name=arion.yaml,arion=fragment
runtime:
  num_cpus: 4        # worker threads in total (default: detected CPUs, cgroup-aware)
  num_runtimes: 1    # independent proxy runtimes sharing those threads (default: 1)
  num_service_threads: 1   # threads for the services runtime (default: 1)
```

| Field | Default | Trades |
| --- | --- | --- |
| `num_cpus` | detected | Total proxy worker threads. More threads, more parallelism, more contention. |
| `num_runtimes` | 1 | How many independent Tokio runtimes split those threads. Isolation versus work stealing. |
| `num_service_threads` | 1 | The services runtime: xDS client, access loggers, admin server. |
| `affinity_strategy` | none | Pins runtimes to cores. Cache locality versus scheduler freedom. |
| `event_interval`, `global_queue_interval`, `max_io_events_per_tick` | Tokio defaults | Scheduler internals; leave alone unless profiling says otherwise. |

Every field can also be set by CLI flag (`--num-cpus`, `--num-runtimes`, ...) or environment variable (`ARION_CPU_LIMIT`, `ARION_GATEWAY_RUNTIMES`, ...); environment beats flag beats file.

## One runtime or per-core runtimes

With `num_runtimes: 1`, all workers share one Tokio runtime and steal work from each other — the forgiving choice when traffic is bursty or uneven, since an idle thread picks up another's backlog.

Raising `num_runtimes` splits the process into independent runtimes with `num_cpus / num_runtimes` threads each (`num_cpus` must divide evenly, or startup fails). Each runtime binds its own copy of every listener socket with `SO_REUSEPORT`, so the kernel spreads incoming connections across runtimes and a connection stays on the runtime that accepted it. That removes cross-runtime coordination from the hot path, at the cost of no work stealing between runtimes: a runtime that lands several heavy connections cannot shed them. The end of that spectrum, `num_runtimes` equal to `num_cpus`, gives single-threaded shared-nothing runtimes — the classic per-core model, best combined with affinity.

## CPU affinity

`affinity_strategy` pins runtime threads to cores, keeping caches warm and avoiding migrations:

```yaml,name=arion.yaml,arion=fragment
runtime:
  num_cpus: 4
  num_runtimes: 4
  affinity_strategy:
    type: auto
```

`auto` assigns cores from the machine's NUMA topology. `nodes` lets you supply the core lists per NUMA node, and `runtimes` names the exact cores per runtime for full control (`type: runtimes` with `map: [[0], [1], [2], [3]]`). A pin that cannot be satisfied logs a warning and continues unpinned.

## The jemalloc build

Building with `--features jemalloc` swaps the global allocator for jemalloc, configured for transparent huge pages, per-thread caching and many arenas. A proxy's allocation pattern — many short-lived, similarly-sized buffers across many threads — is what that configuration targets; it also changes the numbers reported by `/memory` and the `server_memory_*` metrics. It is a build-time choice, not a config key.

## System limits

Two kernel-side settings commonly bite before any Arion knob does:

- **File descriptors.** Each proxied connection holds a downstream and usually an upstream socket. Raise the limit (`LimitNOFILE=` under systemd, `ulimit -n` otherwise) well above twice your expected concurrent connections.
- **Accept backlog.** A listener's `tcp_backlog_size` (default 128) is passed to `listen(2)`, and the kernel caps it at `net.core.somaxconn` — raise both together if accept bursts overflow.

## Verify

The startup log states the layout Arion actually chose:

```bash
arion --config /etc/arion/arion.yaml 2>&1 | grep -E "Launching|pinned"
```

Expected output like `Launching 4 Proxy runtime(s) with 1 thread(s) each` and, with affinity enabled, one `runtime[N] pinned to core [...]` line per runtime.

For what runtimes and threads mean architecturally, read [runtime and threading](@/docs/concepts/runtime-and-threading.md).