Hypothesis paper · Governing AI model spend
Route on evidence, not on the prompt
A proposed design for keeping everyday AI work on cheaper models without blocking anyone from frontier intelligence, and a study protocol to test it against the right controls.
Andrew Kaiserauer · September 2026 · Version 0.2 (protocol revised; not yet registered, not yet tested)
- lower cost per completed task on Sonnet than Opus in exp05, at equal completion
- 81%lower cost per completed task on Sonnet than Opus in exp05, at equal completion
- extra saving a perfect prompt router could have added on top of that
- 1.3%extra saving a perfect prompt router could have added on top of that
- times Copilot Auto upgraded the model when told a task was complex, in Michelin’s audit
- 0 / 24times Copilot Auto upgraded the model when told a task was complex, in Michelin’s audit
- tasks in exp05 that only the frontier model could solve — the number this study must first change
- 0tasks in exp05 that only the frontier model could solve — the number this study must first change
Companies are overspending on AI because people pick the most capable model for everything. Setting a cheaper default helps until users switch back, and they do. Blocking the expensive model fails the minority of tasks that need it. Asking people to choose well does not work, because they lack the time and the information.
The instinctive fix is a router: a classifier that reads the prompt and picks the model. The evidence says this is the wrong place to decide. How hard a task is often cannot be seen in the prompt, off-the-shelf routers send whole topics such as coding to the strongest model, and in the one independent production audit found, a vendor router cost 6 to 18 times more than a small model that solved the same tasks.
The hypothesis
Decide the model on evidence gathered while the work happens, not on the prompt before it starts. Every session starts on a mid-tier model that users cannot change. Frontier intelligence is never blocked; it is reached by climbing an escalation ladder. The first rungs keep the same model and its cache (clarify, think harder, consult a frontier advisor). The upper rungs hand the task to a frontier session when there is evidence it is needed: failing tests, no progress, a frustrated user, the advisor’s recommendation, or an explicit request with a reason. A budget guard downgrades instead of blocking.
Three roles get separated. The user decides the task. The working model decides how much intelligence it needs, because it is the only party that sees the evidence (the code, the errors, the user’s reactions). The platform decides the budget and enforces the starting point. Most current products mix these up.
The problem, precisely
The companion study measured 65 agentic coding tasks. A fresh session on Sonnet completed 100% of them at $0.099 per completed task; the same work on Opus completed 99% at $0.528. That is an 81% saving from the default alone. A perfect hindsight router would have saved only another 1.3%. The money is in the default, and it leaks because people override it.
| Approach | What happens |
|---|---|
| Set a cheaper default | Works for most people, most of the time, but the users who care most about quality (often engineers) switch back. A default a user can change is only a suggestion. |
| Block the expensive model | Saves money and fails the tasks that need it. People route around it with personal accounts, or lose trust in the platform. |
| Show people the cost | Helps a little. People do not bear the cost, and the effect fades. |
That leaves automation: something the user cannot override, that still gets them the stronger model when the task truly needs it.
Why a prompt router is the wrong automation
- Difficulty is invisible up front. Prompt-only routing on software tasks has a formal error floor, and in the companion study no task description predicted which tasks the cheap model would fail.
- Routers route by topic, not difficulty. A standard router sent every coding and math query to the strongest model. For a company whose heavy users are engineers, that reproduces the “everyone on Opus” problem automatically.
- In production, the one independent audit found the router cost more. Michelin audited about 700 GitHub Copilot Auto requests. On tasks a small model solved every time, Auto billed 6 to 18 times more; telling it a task was complex produced zero upgrades in 24 attempts.
- Routers can be gamed. Short appended text reliably pushes routers to the strongest model. Internal users will find “this is critical, think very hard” within a week.
- A vague prompt needs a question, not a bigger model. On ambiguous coding tasks, asking clarifying questions raised GPT-4’s pass rate from 70.96% to 80.80%, while a 2026 benchmark found a stronger model with extended thinking did slightly worse on ambiguous tasks.
The design: an escalation ladder
| Rung | Mechanism | Trigger | Cache |
|---|---|---|---|
| Start | Mid-tier model, medium effort. Enforced; the user cannot change the starting model. | Every session | kept |
| L0 Clarify | Ask one to three questions before starting | Request is ambiguous | kept |
| L1 Effort | Same model, higher reasoning effort | Model’s own judgment, or a classifier prior | kept |
| L2 Advisor | Frontier model reads the transcript and returns guidance; the working model continues | Model decides, plus a forced check before declaring a task done | kept |
| L3 Handoff | New frontier session, clean checkout, with a written brief and the failure log | Verifier still failing after K attempts; no progress in N turns; user says “that’s wrong”; advisor recommends it | new |
| L4 Request | User asks for the frontier model with a one-line reason, counted against a monthly allowance | Explicit | new |
| Budget guard | Past a per-person allowance, frontier requests downgrade to the default model with a notice instead of being blocked | Every rung |
Routing is decided per session and kept fixed within it, because changing model mid-session breaks the prompt cache. L3 is not a mid-session switch: it is a new session, where the cache is empty regardless. Enforcement sits where the user cannot change it: managed client settings, an API gateway that pins each session to its starting model, and admin-console defaults for chat products. Every response shows a small routing receipt, for example “Sonnet · Opus consulted twice”, because silent routing destroys trust.
What exp06 will test
- H0, the gate: headroom exists. At least 10 tasks (or 10% of the registered set) must be measured hard in calibration: the frontier model passes at least 2 of 3 trials and the mid-tier at most 1 of 3. If that fails after the task set has been extended once, the study stops and publishes “no frontier-only work found” as the result.
- L1, primary: the ladder earns its cost over the best fixed default. On the hard stratum the ladder completes at least 10 percentage points more tasks than always-Sonnet, and its cost per completed task over the whole set is no more than 1.5× always-Sonnet’s. Both are required, so a ladder that wins by escalating everything does not pass.
- L2 to L5, secondary. Cache-preserving rungs do most of the work, paired with measured escalation recall and precision; evidence gathered while working beats a frozen prompt classifier on cost; a handoff from a clean checkout beats one that inherits the failed attempt; clarifying first reduces handoffs on ambiguous tasks.
- F1 to F3, field pilot. Fewer complaints than a hard block, a request path that stays rare, and spend that falls by at least what a shadow-mode log predicted, measured against a baseline taken before anything changes.
How the protocol guards against a false positive
Version 0.1 stated the primary hypothesis against always-frontier. The companion study already shows always-mid-tier meets that bar with no escalation at all, so a study built on it could pass without the ladder doing anything. Version 0.2 fixes that and the other weaknesses of the exp05 process before any data exists.
- The control is the best fixed default, not the most expensive model. Always-frontier is reported as the ceiling.
- Tasks are drawn before they are calibrated. Real handoffs are sampled at random from 562 harvested task chips, plus long multi-module work and an ambiguous stratum with a scripted answer key. Adding tasks because the mid-tier model failed them is not allowed.
- Tuning and testing never share tasks. A seeded 30/70 split: every free parameter (K, N, the advisor and clarifying prompts, the comparison classifier) is frozen on the tuning split, and only the confirmatory split is scored.
- Every escalation path must fire before registration. The verifier’s false-accept rate is measured first, and each escalation path must fire on a real failure in a smoke run. exp05’s cascade was registered before its escalation path had ever fired.
- The margin is sized by a power calculation. The +10-point hard-stratum claim needs 45 to 90 hard tasks depending on discordance; v0.1’s 2-point margin would have needed 650 to 2,000 tasks, out of reach at any budget this study will have.
- Deterministic grading, blinding, intention to treat. No LLM judge; no arm sees a task’s label; every planned cell counts once; outage handling and a budget-priority order for dropping arms are fixed up front.
Ways this could be wrong
- Models may not escalate when they should. An overconfident mid-tier model delivers mediocre work without asking for help. The forced advisor check and the recall measurement exist so this shows up as a number rather than a suspicion.
- Advisor costs may add up. Each advisor call rereads the full transcript at frontier prices. On long sessions, frequent consultations could cost more than a handoff.
- Headroom may not exist for this class of work. exp05 found none. If exp06 finds none either, the practical advice is the companion study’s: set the default to the mid-tier model and stop.
- The verifier may still false-accept. Generated tests can be wrong in the same direction as the code.
- Single author, single vendor, list prices. The tasks are written by the person who designed the policies. The real-handoff stratum limits that, but does not remove it.