Arion's threading model is a configuration choice: run the data path on one multi-threaded Tokio runtime, or split it into several independent runtimes, one per core — and this page explains both modes and the runtime block that selects them. Read it when planning deployment for a production scenario.
Why it matters
The two modes trade flexibility for isolation. A single runtime with many worker threads uses Tokio's work-stealing scheduler: any thread can pick up any connection's next piece of work, which smooths out uneven load. Per-core runtimes are share-nothing: a connection is accepted by one runtime and every byte of its processing stays there, which keeps a connection's state on one core's caches and removes cross-core coordination from the data path. The per-core model wins significantly for high demand, high throughput scenarios with massive scaling needs. Choose based on your traffic shape — switching between them is a simple two-line config change.
How it works
The runtime block sizes everything. num_cpus (default: detected) is the total core budget for the data path, num_runtimes (default: 1) divides it, and each proxy runtime gets num_cpus / num_runtimes worker threads — the division must be exact, or Arion refuses to start. A runtime that gets exactly one thread runs on Tokio's lighter current-thread scheduler. num_service_threads (default: 1) sizes the separate services runtime.
runtime:
num_cpus: 8 # default: detected, container-aware
num_runtimes: 4 # 8 / 4 = 2 worker threads per runtime
num_service_threads: 1
affinity_strategy:
type: runtimes
map: [[0, 1], [2, 3], [4, 5], [6, 7]]The default num_cpus is container-aware. On Linux, Arion reads the cgroup CPU limit — cpu.max on cgroups v2, the CFS quota and period on v1 — and rounds the quota up to whole cores; without a limit it falls back to the system count. A pod limited to cpus: "2" therefore defaults to two data-path threads without any configuration. Arion also checks the budget against the cores its process is actually allowed to run on and refuses to start if num_cpus exceeds them.
SO_REUSEPORT is what makes multiple runtimes work. Every listener socket is opened with it, so each proxy runtime binds its own socket on the same address and port, and the kernel spreads incoming connections across them. There is no accept thread handing connections around: acceptance, TLS, filters, and upstream I/O for a connection all run on the runtime that accepted it, for the connection's whole life.
affinity_strategy pins runtimes to cores. With type: auto, Arion groups the available cores by NUMA node (read from /proc/cpuinfo) and deals runtimes across the nodes round-robin, giving each a contiguous chunk. type: nodes does the same with core groups you list explicitly, and type: runtimes maps runtime i to the i-th core list, taking as many cores as that runtime has threads. Pinning keeps a runtime's threads — and with per-core runtimes, each connection — on the same core and memory node; a failed pin logs a warning and continues unpinned.
The remaining keys — global_queue_interval, event_interval, max_io_events_per_tick — pass straight through to Tokio's scheduler and are best left unset until performance tuning gives you a reason. All of the sizing keys can be overridden per environment with flags or ARION_* variables, as described in the configuration model.
Key terms
- Proxy runtime — one Tokio runtime running the data path;
num_runtimesof them exist. - Worker thread — an OS thread inside a runtime;
num_cpus / num_runtimesper runtime. SO_REUSEPORT— the socket option letting every runtime bind the same port, with the kernel balancing accepts.- Affinity strategy — the optional pinning of each runtime's threads to specific cores or NUMA nodes.
- Services runtime — the control-side runtime, sized independently by
num_service_threads.
Next
- Where the runtimes sit in the whole process: architecture.
- Deployment sizing and cgroup interplay: deploy and performance tuning.
- Every
runtimefield and default: the file format reference.