Performance tuning

Arion's defaults — one multi-threaded runtime sized to the CPUs it detects — are sensible for most deployments. This page explains what each knob in the top-level runtime section trades, so you change them for a reason. There are no magic numbers here: measure your own workload before and after.

The knobs

runtime:
  num_cpus: 4        # worker threads in total (default: detected CPUs, cgroup-aware)
  num_runtimes: 1    # independent proxy runtimes sharing those threads (default: 1)
  num_service_threads: 1   # threads for the services runtime (default: 1)

Field	Default	Trades
`num_cpus`	detected	Total proxy worker threads. More threads, more parallelism, more contention.
`num_runtimes`	1	How many independent Tokio runtimes split those threads. Isolation versus work stealing.
`num_service_threads`	1	The services runtime: xDS client, access loggers, admin server.
`affinity_strategy`	none	Pins runtimes to cores. Cache locality versus scheduler freedom.
`event_interval`, `global_queue_interval`, `max_io_events_per_tick`	Tokio defaults	Scheduler internals; leave alone unless profiling says otherwise.

Every field can also be set by CLI flag (--num-cpus, --num-runtimes, ...) or environment variable (ARION_CPU_LIMIT, ARION_GATEWAY_RUNTIMES, ...); environment beats flag beats file.

One runtime or per-core runtimes

With num_runtimes: 1, all workers share one Tokio runtime and steal work from each other — the forgiving choice when traffic is bursty or uneven, since an idle thread picks up another's backlog.

Raising num_runtimes splits the process into independent runtimes with num_cpus / num_runtimes threads each (num_cpus must divide evenly, or startup fails). Each runtime binds its own copy of every listener socket with SO_REUSEPORT, so the kernel spreads incoming connections across runtimes and a connection stays on the runtime that accepted it. That removes cross-runtime coordination from the hot path, at the cost of no work stealing between runtimes: a runtime that lands several heavy connections cannot shed them. The end of that spectrum, num_runtimes equal to num_cpus, gives single-threaded shared-nothing runtimes — the classic per-core model, best combined with affinity.

CPU affinity

affinity_strategy pins runtime threads to cores, keeping caches warm and avoiding migrations:

runtime:
  num_cpus: 4
  num_runtimes: 4
  affinity_strategy:
    type: auto

auto assigns cores from the machine's NUMA topology. nodes lets you supply the core lists per NUMA node, and runtimes names the exact cores per runtime for full control (type: runtimes with map: [[0], [1], [2], [3]]). A pin that cannot be satisfied logs a warning and continues unpinned.

The jemalloc build

Building with --features jemalloc swaps the global allocator for jemalloc, configured for transparent huge pages, per-thread caching and many arenas. A proxy's allocation pattern — many short-lived, similarly-sized buffers across many threads — is what that configuration targets; it also changes the numbers reported by /memory and the server_memory_* metrics. It is a build-time choice, not a config key.

System limits

Two kernel-side settings commonly bite before any Arion knob does:

File descriptors. Each proxied connection holds a downstream and usually an upstream socket. Raise the limit (LimitNOFILE= under systemd, ulimit -n otherwise) well above twice your expected concurrent connections.
Accept backlog. A listener's tcp_backlog_size (default 128) is passed to listen(2), and the kernel caps it at net.core.somaxconn — raise both together if accept bursts overflow.

Verify

The startup log states the layout Arion actually chose:

arion --config /etc/arion/arion.yaml 2>&1 | grep -E "Launching|pinned"

Expected output like Launching 4 Proxy runtime(s) with 1 thread(s) each and, with affinity enabled, one runtime[N] pinned to core [...] line per runtime.

For what runtimes and threads mean architecturally, read runtime and threading.