--- title: "Build an L7 load balancer" url: "/guides/proxy/l7-load-balancer/" kind: "guide" summary: "Stand up an HTTP load balancer that spreads traffic across three upstreams, health-checks them, retries failures and logs every request." --- > Arion site index: [/llms.txt](/llms.txt) # Build an L7 load balancer Stand up an HTTP load balancer that spreads traffic across three upstreams, health-checks them, retries failures and logs every request. ## Outcome This guide builds an HTTP load balancer: one listener on port 8080 that spreads every request across three upstream servers, takes unhealthy servers out of rotation automatically, retries failed requests, and writes one JSON access-log line per request to stdout. Use it whenever several copies of the same HTTP service should sit behind a single address. ## Prerequisites - An `arion` binary — see [install](@/docs/get-started/install.md). Access-log output additionally needs a build with the `access-log` feature. - Three HTTP servers to balance. The verify section starts throwaway echo servers, so nothing real is required. - `curl` and, optionally, Docker for the echo servers. ## The architecture - A listener on `0.0.0.0:8080` runs the HTTP connection manager with a single catch-all route. - The route sends everything to one cluster, `web`, which holds three static endpoints balanced with the `LEAST_REQUEST` policy. - The cluster probes each endpoint's `/healthz` every five seconds, caps concurrent requests with a circuit breaker, and the virtual host retries failures twice. ## The complete config ````yaml # arion.yaml, composed from 6 features. logging: log_level: info access_logging: blocking: false envoy_bootstrap: static_resources: listeners: - name: http_8080 address: socket_address: { address: 0.0.0.0, port_value: 8080 } filter_chains: - name: http_8080_chain filters: - name: http_gateway typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager access_log: - name: envoy.access_loggers.stdout typed_config: "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog log_format: text_format: | {"start_time":"%START_TIME%","method":"%REQ(:METHOD)%","path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","protocol":"%PROTOCOL%","response_code":"%RESPONSE_CODE%","response_flags":"%RESPONSE_FLAGS%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","duration_ms":"%DURATION%","upstream_cluster":"%UPSTREAM_CLUSTER%","upstream_host":"%UPSTREAM_HOST%","authority":"%REQ(:AUTHORITY)%","user_agent":"%REQ(USER-AGENT)%","request_id":"%REQ(X-REQUEST-ID)%"} http_filters: - name: envoy.filters.http.router typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router route_config: name: http_8080_routes virtual_hosts: - name: backend domains: ["*"] retry_policy: retry_on: 5xx,connect-failure num_retries: 2 routes: - match: prefix: / route: cluster: web clusters: - name: web connect_timeout: 5s type: STATIC lb_policy: LEAST_REQUEST load_assignment: cluster_name: web endpoints: - lb_endpoints: - endpoint: address: socket_address: { address: 10.0.1.11, port_value: 8080 } - endpoint: address: socket_address: { address: 10.0.1.12, port_value: 8080 } - endpoint: address: socket_address: { address: 10.0.1.13, port_value: 8080 } health_checks: - timeout: 1s interval: 5s healthy_threshold: 3 unhealthy_threshold: 2 http_health_check: path: /healthz circuit_breakers: thresholds: - max_requests: 1000 ```` ## Walkthrough ### Arion-native keys The two top-level keys before `envoy_bootstrap` are Arion's own: `logging.log_level` controls the process log, and `access_logging.blocking: false` keeps per-request log writes off the worker threads. Everything under `envoy_bootstrap` is standard Envoy configuration — see the [configuration model](@/docs/concepts/configuration-model.md). ### Listener and connection manager The listener `http_8080` binds `0.0.0.0:8080` with one filter chain. Its only network filter is the HTTP connection manager, which parses requests and hands them to the `http_filters` list — here just the router, the terminal filter that forwards to a cluster. ### Access log The connection manager's `access_log` writes to stdout using a `text_format` whose template is a JSON object built from command operators such as `%RESPONSE_CODE%` and `%UPSTREAM_HOST%`. Each request becomes one machine-parseable line naming the method, path, status, duration and which upstream served it. Operators and sinks are covered in [access logs](@/docs/observe/access-logs.md). ### Route and retries `route_config` declares one virtual host matching every domain (`domains: ["*"]`) with a single route: prefix `/` to cluster `web`. The virtual host also carries a `retry_policy` — `retry_on: 5xx,connect-failure` with `num_retries: 2` — which applies to every route in the host, so a request that hits a failing upstream is transparently re-sent up to twice. See [routing](@/docs/traffic/routing.md). ### The cluster `web` is a `STATIC` cluster: three fixed endpoints, a 5-second `connect_timeout`, and `lb_policy: LEAST_REQUEST`, which favors the endpoint with the fewest requests in flight — a good default when response times vary. Policies and endpoint discovery are compared in [upstreams and load balancing](@/docs/traffic/upstreams-and-load-balancing.md). ### Health checks and the circuit breaker `health_checks` sends `GET /healthz` to every endpoint each 5 seconds with a 1-second timeout. Two consecutive failures (`unhealthy_threshold: 2`) eject an endpoint from rotation; three passes (`healthy_threshold: 3`) bring it back. `circuit_breakers` caps the cluster at 1000 concurrent requests, shedding load instead of queueing without bound. Both are explained in [health checks and resilience](@/docs/traffic/health-checks-and-resilience.md). ## Run and verify Start three echo servers, each answering with its own name: ```bash docker run -d --rm -p 5678:5678 hashicorp/http-echo -text=one docker run -d --rm -p 5679:5678 hashicorp/http-echo -text=two docker run -d --rm -p 5680:5678 hashicorp/http-echo -text=three ``` Without Docker, `python3 -m http.server 5678` (one port per terminal) works too — the responses are directory listings instead of names. The recipe points at `10.0.1.11`–`13:8080`. For a local run, change the three endpoint addresses to `127.0.0.1` ports `5678`, `5679` and `5680`, then start Arion: ```bash arion --config arion.yaml ``` Send thirty requests and count who answered: ```bash for i in $(seq 1 30); do curl -s http://127.0.0.1:8080/; done | sort | uniq -c ``` ```text 11 one 10 two 9 three ``` Exact counts vary run to run, but the split stays close to even: idle echo servers answer instantly, so least-request sees no reason to favor anyone. Each request also produces one JSON line on Arion's stdout naming the `upstream_host` that served it. Now stop one upstream (`docker stop` its container). Within about ten seconds — two failed checks, five seconds apart — the endpoint leaves rotation and the loop shows only two names. Restart it and it returns after three passing checks. Notice what the loop does not show: errors. In the window between the upstream dying and its second failed health check, requests that land on the dead endpoint fail to connect — and `retry_on: connect-failure` re-sends them to a healthy one, so the client sees nothing. ## Production notes - Health-check cadence is a trade-off: shorter `interval` and a lower `unhealthy_threshold` detect failures faster but probe harder and eject flappier. Start conservative and tighten with evidence. - `LEAST_REQUEST` suits mixed workloads; `ROUND_ROBIN` is predictable for uniform ones. Changing `lb_policy` is a one-line edit — see [upstreams and load balancing](@/docs/traffic/upstreams-and-load-balancing.md). - `connect_timeout` bounds only connection establishment. Set request-level timeouts on routes so slow upstreams cannot hold requests open indefinitely. - Retrying `5xx` re-sends requests that may have partially executed. Keep it for idempotent traffic, or narrow `retry_on` to `connect-failure`. - `STATIC` endpoints are fixed at start. If upstream addresses change, use a DNS-resolved cluster type or serve endpoints from a control plane. ## Next Add authentication and rate limiting in front of the same cluster in the [API gateway guide](@/guides/proxy/api-gateway.md), or split traffic between two versions in [canary with weighted clusters](@/guides/proxy/canary-with-weighted-clusters.md). Cluster fields are listed in the [cluster reference](@/reference/resources/clusters.md).