--- title: "Upstreams and load balancing" url: "/docs/traffic/upstreams-and-load-balancing/" kind: "feature" summary: "Define clusters of upstream endpoints, choose how they are discovered, and pick a load-balancing policy with weights and priorities." --- > Arion site index: [/llms.txt](/llms.txt) # Upstreams and load balancing Define clusters of upstream endpoints, choose how they are discovered, and pick a load-balancing policy with weights and priorities. A cluster names a group of upstream endpoints and tells Arion how to reach them: how endpoints are discovered, which protocol to speak, and how to spread load across them. Define one cluster per backend service, then route to it by name. ## Define a cluster ```yaml,arion=fragment clusters: - name: api type: STATIC connect_timeout: 0.25s lb_policy: ROUND_ROBIN load_assignment: endpoints: - lb_endpoints: - endpoint: address: socket_address: { address: 127.0.0.1, port_value: 5678 } - endpoint: address: socket_address: { address: 127.0.0.1, port_value: 5679 } ``` ## Discovery types | `type` | Endpoints come from | | -------------- | ---------------------------------------------------------------------- | | `STATIC` | The inline `load_assignment`; addresses must be IPs, pipes, or internal | | `STRICT_DNS` | The inline `load_assignment`, hostnames resolved at connection time | | `EDS` | The control plane over xDS; no inline `load_assignment` allowed | | `ORIGINAL_DST` | The connection's original destination, a header, or metadata | `ORIGINAL_DST` clusters must set `lb_policy: CLUSTER_PROVIDED`; the target comes from the original destination address by default, or from `original_dst_lb_config` with `use_http_header` (default header `x-envoy-original-dst-host`, or a custom `http_header_name`) or a `metadata_key`, optionally overriding the port with `upstream_port_override`. `LOGICAL_DNS` is not supported and is refused at startup. ## Load-balancing policies `lb_policy` accepts `ROUND_ROBIN` (the default), `RANDOM`, `LEAST_REQUEST`, `RING_HASH`, `MAGLEV`, and `CLUSTER_PROVIDED`. Endpoints take a `load_balancing_weight` (default 1), which every policy respects: ```yaml,arion=fragment lb_endpoints: - endpoint: address: socket_address: { address: 127.0.0.1, port_value: 5678 } load_balancing_weight: 3 - endpoint: address: socket_address: { address: 127.0.0.1, port_value: 5679 } load_balancing_weight: 1 ``` `RING_HASH` and `MAGLEV` are consistent-hashing policies: they need a `hash_policy` on the route to build the hash from a header, query parameter, or source IP: ```yaml,arion=fragment route: cluster: api hash_policy: - header: { header_name: x-session-id } - connection_properties: { source_ip: true } ``` Policies apply in order; a matching policy marked `terminal: true` stops the walk, and if nothing matches the hash is random. The `load_balancing_policy` extension point accepts the typed equivalents of the same policies plus `override_host`, which pins the upstream host to the value of a request header and falls back to a standard policy when the header is absent or invalid. ## Priorities and failover Endpoint groups carry a `priority` (0 is highest; levels must be contiguous from 0). Traffic goes to priority 0 while it is healthy; as its healthy share drops, load spills to the next level following Envoy's priority-levels algorithm with a 1.4 overprovisioning factor: ```yaml,arion=fragment load_assignment: endpoints: - priority: 0 lb_endpoints: - endpoint: address: socket_address: { address: 10.0.0.1, port_value: 5678 } - priority: 1 lb_endpoints: - endpoint: address: socket_address: { address: 10.1.0.1, port_value: 5678 } ``` Health comes from [active health checks](@/docs/traffic/health-checks-and-resilience.md) and the endpoint's configured `health_status`. ## Upstream protocol Arion speaks HTTP/1.1 to upstreams unless the cluster says otherwise through `typed_extension_protocol_options`: ```yaml,arion=fragment typed_extension_protocol_options: envoy.extensions.upstreams.http.v3.HttpProtocolOptions: "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions explicit_http_config: http2_protocol_options: {} ``` Only explicit configuration is supported (`http_protocol_options` for HTTP/1.1 with optional `enable_trailers`, or `http2_protocol_options` with `max_concurrent_streams`, initial window sizes, and `connection_keepalive`); auto-negotiation and use-downstream-protocol are not. `common_http_protocol_options.idle_timeout` closes idle upstream connections. ## Upstream TLS Add a `transport_socket` with an `UpstreamTlsContext` to encrypt traffic to the backend; `sni` is required: ```yaml,arion=fragment transport_socket: name: envoy.transport_sockets.tls typed_config: "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext sni: api.example.com common_tls_context: validation_context: trusted_ca: { filename: ./certs/ca.pem } ``` Add one `tls_certificates` entry (or an SDS reference) for mTLS toward the upstream — see [TLS and mTLS](@/docs/security/tls-and-mtls.md). ## Verify ```bash curl -s http://127.0.0.1:9901/clusters | grep "^api::" ``` ```text api::127.0.0.1:5678::health_flags::healthy api::127.0.0.1:5678::weight::3 api::127.0.0.1:5678::priority::0 ... ``` ## Limits - Endpoint `locality` is not supported — grouping is by `priority` only, and a config that sets locality is refused. - Policy tuning knobs (ring size, Maglev table size, least-request choice count, slow start) are not supported. - Outlier detection, subset load balancing, and DNS tuning fields (`dns_refresh_rate`, `respect_dns_ttl`) are not supported. - One health check and one set of protocol options per cluster. - `override_host` reads a header; metadata as a host source is not yet supported. The full cluster surface is in the [cluster reference](@/reference/resources/clusters.md), and failure handling continues in [health checks and resilience](@/docs/traffic/health-checks-and-resilience.md).