Reuse stable context
Prompt caching keeps repeated system instructions and shared context from being billed as fresh work on every call.
Independent AI engineer · available for the next hard problem
I built an AI system that cut token spend by 44.3%.
KostAI is a local-first LLM FinOps toolkit. It instruments each call, removes context that does not help, routes bounded work to cheaper compute, and keeps hard work on frontier models. I built it—and the systems below—to turn expensive AI ambiguity into measurable engineering proof.
Looking for AI platform, solutions engineering, and product engineering roles.
01LLM FinOps
02Agent systems
03Zero-trust security
04Full-stack delivery
The original pipeline, rebuilt honestly
Spend less by sending less—and choosing the right brain.
KostAI does not rely on one magic trick. It measures the call, removes avoidable input, reuses stable context, and routes only bounded work away from frontier models. The exact mix varies by workload.
Measured benchmark flow
100 baseline token units → 55.7 routed units01 · observeEvery callmodel · tokens · cost · latency
02 · reduceUseful contextdedupe · cache · compress
03 · routeRight-size computelocal · cheaper API · frontier
Naive Opus baseline100.0
KostAI measured mean55.7
44.3 saved
Cut context noise
Deterministic reducers remove duplicated history, boilerplate, filler, and other tokens that do not help answer the current task.
Shrink tool results
A local model extracts task-relevant facts while preserving exact values, identifiers, errors, and line numbers needed downstream.
Route by difficulty
Bounded, non-complex work can use a local or lower-cost model; hard or ambiguous work stays on the frontier path.
Protect the answer
Quality is a promotion gate, not a slogan.
Shadow mode tests a cheaper route beside the existing path while the production caller still receives the baseline result. That makes savings observable before a route is trusted.
Baselinereturned to app
Existing frontier path
The application behaves exactly as it did before while KostAI records cost, latency, and an output preview.
Candidateread-only test
Cheaper or local path
The proposed route runs in parallel. Its failure cannot replace or interrupt the baseline response.
Compare each route
- input and output tokens
- cost and latency
- blind human quality review
- route reason and confidence
Pass: candidate for controlled routing
Uncertain: keep or escalate to frontier
Claim boundary
The 44.3% reduction is measured on a 41-task dogfood workload. KostAI has implemented shadow comparison and escalation safeguards; this page does not claim universal quality parity across every model, task, or production workload.
Pre-registered promotion protocol
Five scores. Blinded labels. Failures stay in the record.
Baseline and candidate outputs are stripped of route labels and scored from 0–10 for correctness, completeness, actionability, grounded confidence, and format fit. Preference, ties, and “both bad” outcomes are recorded before the route identity is revealed.
- 01Freeze tasksRepresentative prompts and success criteria are agreed first.
- 02Run shadow pairsThe baseline remains the production answer.
- 03Blind the outputsThe reviewer cannot see which route produced A or B.
- 04Score five dimensionsQuality evidence sits beside tokens, cost, and latency.
- 05Promote or holdOnly owner-approved routes move beyond shadow mode.
How the result was produced
The number has a receipt—and a boundary.
The 44.3% figure is a workload-specific dogfood benchmark, not a universal promise. It compares naive Opus routing with KostAI’s routed execution across 41 tasks.
Instrument every call
Record model, route, input/output tokens, cost, latency, and workflow without exporting prompt bodies.
Score avoidable work
Detect repeated context, oversized tool results, frontier overuse, and other recoverable token patterns.
Shadow before switching
Run the proposed route beside the baseline, compare outputs and cost, and keep the production answer unchanged.
Escalate uncertainty
Keep frontier models for complex work and hold routes that lack sufficient confidence or evaluation evidence.
44.3%41-task mean savings
55%later frozen-suite mean
30boil-specific tests added
28MCP tools in the release
Canonical definition
What “KostAI.app” means.
KostAI is John Bradley’s local-first LLM FinOps toolkit and portfolio anchor—not a model provider and not a universal savings guarantee.
- Category
- LLM FinOps · developer tooling
- Core loop
- instrument → reduce → route → shadow → escalate
- Verified claim
- 44.3% mean token savings on 41 dogfood tasks
- Package
- @sapperjohn/kostai v0.5.2
- Machine-readable brief
- kostai.app/llms.txt ↗
Selected work
The other systems behind the message.
These projects connect the same operating pattern across cost, security, governance, and agent infrastructure: make the hard part observable, then automate it.
KostAI
Local-first instrumentation, routing, and shadow-mode evaluation for finding expensive context and moving safe work off frontier models.
41-task dogfood rollup · mean 44.3% token savings
Elastifund
A self-improving operating system for prediction-market research and execution across Polymarket and Kalshi.
Public Python system · live product
Calm Vault
A zero-trust credential broker and verifiable evidence vault for AI agents, built around selective disclosure and provenance.
Public security infrastructure
ZKAC v0
A threshold-cooperation primitive combining Pedersen commitments, FROST signatures, a hashchain, and an external verifier.
76-test public Rust implementation
Broker Fire
A dependency-free gateway between an AI agent’s intent and action, with explicit authority checks and synthetic safety gates.
TypeScript/Python parity · zero dependencies
CDXP
Codebase-map compression and wire-format encoding for lower-cost coordination between coding agents through MCP.
Public MCP server and proxy
What I bring
Technical depth with customer altitude.
I’m most useful where engineering, product judgment, and a live customer problem collide.
AI systems
Agent workflows, evaluation, model routing, MCP, and production guardrails.
Full-stack delivery
React, TypeScript, Python, APIs, data pipelines, testing, and deployment.
Security engineering
Zero-trust identity, verifiable credentials, policy, and cryptographic protocols.
Customer-facing execution
Turning ambiguous business pain into a working demo, proof, and implementation plan.
A no-regret first step
Prove the savings on your workload before changing production.
Start with a bounded shadow evaluation: keep every existing answer, measure avoidable spend locally, blind-review candidate outputs, and leave with a route-by-route evidence packet. No forced migration. No universal promise.
Discuss a shadow evaluation ↗The next build
Give me the problem that is expensive to misunderstand.
If you’re hiring for AI platform, solutions, or product engineering work, I’d like to show you how I move from ambiguity to measurable proof.