KostAI · LLM Cost Observability

Last 30 minutesRefresh 5s

Production-hosted demo

This page uses the same deterministic workload and metric names as the provisioned Grafana stack. It is demo telemetry—120 baseline/optimized pairs—not customer usage. No prompt or completion bodies are shown or exported.

Scrape health

UPprometheus · kostai:3674

Recorded LLM calls

240120 baseline + optimized pairs

Observed cost

$19.469ai_cost_cost_usd_total

Avoidable cost found

$11.048ai_cost_avoidable_cost_usd_total

Waste ratio

56.7%

avoidable / observed

Throughput multiplier

1.5×wire + avoided / wire

Cost and waste over timeUSD · cumulative demo workload

Prometheus

$20$15$10$5$0

Observed cost $19.469 Avoidable cost $11.048

Token throughput24-hour trailing demo average

Prometheus

60K45K30K15K0

Tokens / hour 56.4K Avoided / hour 30.7K

Durable queue statedemo completion state

Prometheus

queued0

running0

done120

failed0

Prometheus targetscrape configuration proof

Endpoint: kostai:3674/metrics
Job: kostai
Interval: 5 seconds
Environment: demo

Tempo · trace search{resource.service.name="kostai-demo"}

1 trace

Trace IDServiceNameDurationStatus

c7487ab1c5314d33ee6ab3d8627c3692kostai-demokostai.telemetry.verify<1 ms OK

WHAT THIS DEMONSTRATES

Metrics are native.

KostAI exposes Prometheus-format cost, waste, token, queue, peer, and throughput signals from its local dashboard endpoint.

Traces stay portable.

The opt-in OpenTelemetry path exports backend-neutral OTLP traces that Grafana Tempo can search without prompt content.

The stack is reproducible.

The local demo provisions KostAI, Prometheus, Tempo, and Grafana from version-controlled configuration with one Compose command.