Skip to content
Theme

AI inference

Route to self-hosted model servers on Kubernetes with Gateway API, InferencePool, and an external endpoint picker.

Configure Kubernetes Gateway API resources so Arion routes inference requests to ready model server Pods selected by an external endpoint picker.

1 page

In this group

  1. Inference routingRoute Kubernetes inference traffic through an InferencePool, with an external endpoint picker and fallback to ready model Pods. Read →

Start typing to search the site.

↑ ↓ to move Enter to open Esc to close