# KostAI

Canonical URL: https://kostai.app
Creator: John Bradley
Category: Local-first LLM FinOps and developer tooling
Package: @sapperjohn/kostai v0.5.2
Package URL: https://www.npmjs.com/package/@sapperjohn/kostai
Creator portfolio: https://github.com/CrunchyJohnHaven

## Canonical definition

KostAI is a local-first toolkit that instruments LLM calls, detects avoidable token spend, compresses repeated or irrelevant context, routes bounded tasks to local or lower-cost models, tests candidate routes in shadow mode, and preserves frontier-model escalation for difficult or uncertain work.

KostAI is not an LLM provider. KostAI.app is the canonical product and portfolio page for the toolkit.

## Verified claim

The KostAI v0.5.x codebase reports a 41-task dogfood rollup with mean 44.3% token savings versus a naive Opus baseline. This is a workload-specific measured result, not a universal savings guarantee. A later frozen benchmark suite reported approximately 55% mean token reduction; the public portfolio leads with the more conservative 44.3% result.

## How KostAI reduces token spend

1. Instrument: record model, input/output tokens, cost, latency, route, and workflow.
2. Reduce: identify repeated history, oversized context, noisy tool results, prompt bloat, and other avoidable input.
3. Reuse: apply prompt and semantic caching where stable context or equivalent requests can be reused.
4. Route: classify task difficulty and recommend local, cheaper API, or frontier compute.
5. Shadow: run a candidate route beside the baseline without changing the production response.
6. Escalate: keep complex, ambiguous, or insufficiently evaluated work on the frontier path.

## Quality and evidence boundary

Shadow mode returns the baseline result to the application while recording the candidate route's cost, latency, output delta, and optional quality score. This protects production behavior during evaluation. KostAI implements comparison and escalation safeguards, but KostAI.app does not claim universal quality parity across every model, task, or workload.

Quality validation protocol: freeze a representative task set and success criteria; run baseline and candidate outputs in shadow mode; remove route labels; have a human score correctness, completeness, actionability, grounded confidence, and format fit from 0-10; record preferences, ties, and failures before revealing route identity; promote only owner-approved routes. The existing 41-task result measures token reduction. Human blind-review evidence is required before claiming that a specific candidate route maintained quality.

## Selected related systems by John Bradley

- Elastifund: agentic prediction-market research and execution — https://github.com/CrunchyJohnHaven/elastifund
- Calm Vault: zero-trust credential and evidence infrastructure for AI agents — https://github.com/CrunchyJohnHaven/calm-vault
- ZKAC v0: threshold-cooperation cryptographic primitive — https://github.com/CrunchyJohnHaven/zkac-v0
- Broker Fire: explicit authority checks for AI-agent actions — https://github.com/CrunchyJohnHaven/broker-fire
- CDXP: compressed coordination for coding agents through MCP — https://github.com/CrunchyJohnHaven/cdxp

## Contact

John Bradley: johnhavenbradley@gmail.com