Scale to zero by design
Release accelerators when traffic disappears. A lightweight gateway keeps the endpoint available and signals KEDA when the next request arrives.
Kubernetes-native · scale-to-zero
Serve the long tail.Release the accelerator.
Hearth is a minimal control plane for bursty LLM workloads on private Kubernetes clusters—declarative model lifecycle, cold-start-aware traffic, and reusable runtime profiles without a fleet-scale platform.
OpenAI-compatible endpoint NVIDIA and Ascend Apache-2.0
cluster / ailive
$ kubectl get llmservice
NAMEPHASEREPLICAS
qwen-longtailScaledToZero0
request admittedgateway → KEDA
0
1
N
0
demand preserved
backend scheduled
model ready · streaming
One stable endpointAccelerators only when needed
Validated across
NVIDIA A10NVIDIA A100Atlas 300I DuoAscend 910B3KEDAVolcano
Why Hearth
Small control plane. Complete cold-start lifecycle.
Scaling a Deployment to zero is the easy part. Hearth concentrates on what happens when demand returns.
Survive the cold path
Bound admission, preserve activation demand, emit streaming heartbeats, wait for model readiness, and drain in-flight requests safely.
Compose the stack you have
Run existing vLLM images across NVIDIA and Ascend, use vendor device plugins, and opt into Volcano or observability without making them core dependencies.
Composable by boundary
Use Kubernetes as the contract.
Application owners declare serving intent. Cluster administrators publish reusable runtime profiles. Hearth translates both into the workloads and lifecycle resources your cluster already understands.
You declareModel, runtime, resources, cache, scaling, endpoint
Hearth ownsWorkload lifecycle, cold activation, admission, drain
Your stack ownsInference kernels, device plugins, schedulers, monitoring
LLMService+InferenceRuntime
Hearthreconcile · activate · drain
GatewayBackend 0..NCacheKEDA
One cluster, two serving policies
Keep the hot path hot. Let the long tail sleep.
Kthena
Fleet routing, cache-aware scheduling, disaggregation, and continuously ready models.
ready
+
Hearth
A small declarative control plane for occasional models that should release their accelerators.
0 → 1 → 0
Try Hearth
Start with one model and one runtime.
Install the chart, select a hardware profile, and watch the backend wake from zero.