Tokenless Routes Agents to Cheaper Models

Tokenless (YC S26) is an API gateway from Rohit and co-founders Andrew and Kev that routes agent traffic between different AI models turn by turn. It deals with a very real agentic coding problem: frontier models are good at software work, but long coding sessions can burn through token budgets fast. The useful takeaway is simple: model routing is worth testing on separable agent turns, but it needs the same review discipline you would use to implement code review habits for ai-generated code. This is also why the story matters to Cursor, Anysphere's AI code editor, and to anyone planning ai coding training for teams without turning every prompt into a blank check.

Automatic model switching is the practice of sending different parts of an agent workflow to different models based on cost, quality, or task difficulty. Tokenless says it does this dynamically, by routing agent requests across models and deciding which model should continue. The bet is that many turns do not need the most expensive model for the whole ride.

Why Hacker News cared about the cache math

Developers cared because the problem is not theoretical. Agentic coding creates long chains of tool calls, repo reads, test runs, patches, and follow-up messages. Even a small per-turn inefficiency can become a large bill when the agent is crawling a monorepo all afternoon.

The interesting part of Tokenless is not merely choosing a cheap model. It is the claim that the gateway can inspect progress across models and route the turn dynamically. That makes the project feel closer to an execution strategy than a static model picker.

The objection was sharp: if you fan out a request to multiple models, you may pay input tokens more than once. In many coding-agent workloads, especially ones with a hot context cache, input tokens can dominate the economics. A cheaper output is not a win if you duplicated a large prompt to get there.

That is the first trap to avoid. Do not evaluate a router only on sticker price per million tokens. Evaluate the full turn: cache hit rate, duplicated input, tool-call length, answer quality, and whether the agent had to retry.

Compare routing against picking one model

The honest comparison is Tokenless-style routing versus two boring baselines: always use the frontier model, or always use the cheaper model. Boring baselines are healthy. They stop clever infrastructure from winning only in demos.

Verdict: Tokenless wins when a coding workflow has many independent or semi-independent turns where cheaper models can safely do useful work. A fixed frontier model wins when correctness is expensive to recover. A fixed smaller model wins when the task is narrow, repeatable, and easy to verify.

For Cursor users, the natural experiment is not replacing your whole workflow. It is routing low-risk subagent work first: summarize failing tests, inspect logs, draft migration notes, or propose a small refactor plan before the main agent edits code.

Try Tokenless where the turn is actually separable

A good first test is a task where the agent can be wrong without touching production code. For example, ask a subagent to classify flaky test failures from CI logs, then have the main Cursor session decide which files to inspect. The router gets a real workload, but the repo boundary stays intact.

A weaker test is a single long chat that reads half the repository, keeps a hot cache, and performs a risky edit. That is exactly where the Hacker News cache objection bites. If the session benefits from repeated cached context, switching models may save less than it appears.

Use Model Context Protocol, often called MCP, carefully here. MCP is a standard way for AI applications to connect to external tools and data sources through servers. If a routed agent can reach GitHub, Slack, Jira, or a database, the cost question becomes a permission question too.

This is where the related training topic becomes practical rather than abstract. The point is not to slow the agent down. The point is to make the expensive, risky, or irreversible actions visible.

Keep the review boundary in Cursor

The best way to implement code review habits for ai-generated code is to review the artifact, not the chat. In Cursor, that means rules, diffs, tests, and a short checklist the reviewer can apply without replaying every model decision.

Here is a small .mdc rule stub for a repo that is trying model routing without giving every agent write access everywhere:

---
description: Review boundary for routed coding-agent work
alwaysApply: true
---

Routed agent output must arrive as a diff, test result, or written note.
Do not accept changes that only cite model confidence.

Before merging AI-generated code:
- Confirm the touched files match the requested scope.
- Run the narrowest relevant test command.
- Check for deleted validation, auth, logging, or error handling.
- Ask for a human review when the change crosses a package boundary.

Routed subagents may summarize, inspect, and propose.
Only the main coding session may edit files unless the task explicitly says otherwise.

That rule is intentionally plain. It does not care whether the text came from a frontier model, a small model, or a router. It cares whether the resulting change is scoped, tested, and reviewable.

The trap is treating model choice as a substitute for review. A cheaper good-enough model can still delete an invariant. A stronger model can still make a confident mistake. Verification work has the same lesson in other domains; see how verified-3d-mesh-intersection verifies 3D mesh CSG by making the claim checkable instead of trusting the narration.

Try it safely checklist

Use this as a lightweight experiment receipt, not a committee process.

• Pick one workflow with separable turns, such as CI failure summaries, dependency update notes, or test-output triage.
• Record the baseline: fixed frontier model cost, fixed small model cost, task success, retries, and reviewer time.
• Run the same task class through Tokenless-style routing for a small batch.
• Compare total turn cost, not just output-token cost.
• Track cache behavior when the session includes long repo context.
• Keep MCP tools read-only for the first pass unless the action is easy to undo.
• Require a diff, command output, or written handoff note before a human accepts the result.
• Stop the test if routed output increases retries, hides uncertainty, or creates larger review burden.

A simple handoff receipt is enough:

Task: summarize failing checkout tests
Agent path: routed model gateway
Inputs used: CI log, package test command, changed files list
Output type: diagnosis note, no file edits
Reviewer check: reran pnpm test checkout --filter failing-case
Decision: accepted / rejected / needs frontier-model pass

This gives you the one thing most model-routing demos lack: a way to tell whether savings survived contact with review.

Common questions

• Does Tokenless always save money for coding agents?

  No, not always. The economics depend on duplicated input tokens, cache hits, retries, and whether the cheaper model produces work that reviewers can accept. Tokenless is most plausible where turns are separable and the router can avoid wasting expensive reasoning on easy steps.

• What are the best ways to implement code review habits for ai-generated code?

  Review the diff, tests, and scope instead of the model transcript. A practical habit is to require three artifacts for AI-generated code: the changed files, the narrow test command, and a short note explaining the intended behavior. This works whether the code came from one model or a router.

• Where would I try this first in Cursor?

  Start with read-only or low-risk subagent work in Cursor, such as summarizing CI failures, drafting migration notes, or classifying errors before the main agent edits files. Keep repo rules explicit in .mdc files or AGENTS.md so routed output is still judged against local project constraints.

• Is this the same as using a cheaper default model?

  No. A cheaper default model is a static choice, while Tokenless-style routing tries to decide per turn which model should continue. That flexibility is the point, but it also creates measurement work. You need to compare against both a fixed frontier model and a fixed small model.

• Does MCP make model routing riskier?

  MCP can make routing more powerful and riskier because it connects agents to real tools and data. If routed agents can call GitHub, databases, or internal systems, start with read-only permissions. Cost optimization should not silently expand what an agent is allowed to do.

Best ways to use this research

• Best for: teams already running agentic coding workflows with visible token spend, especially where subagents perform many small inspection or summarization turns.
• Best first artifact: a Cursor review checklist or .mdc rule that says what evidence must exist before AI-generated code is accepted.
• Best comparison angle: measure Tokenless-style routing against fixed frontier and fixed small-model baselines using total task cost, retries, and reviewer time.
• Best safety boundary: keep MCP servers read-only during the first experiment, then add write permissions only for narrow, reversible actions.
• Best workshop use: in an ai coding workshop, treat model routing as a measurement exercise: same task, same repo, three model strategies, one review checklist.

Further reading

• Tokenless (YC S26) — source
• Model Context Protocol — specification
• OWASP — Top 10 for Large Language Model Applications
• Cursor docs — rules

Next step

Try one routed, read-only subagent task and compare it against your normal model choice. If it saves money without adding retries or review confusion, then widen the experiment by one workflow.

One methodology lens

One useful way to read this through our methodology is the Plan step: delegate first-pass decomposition and dependency mapping, review the sequencing and assumptions, and keep ownership of scope and priorities. If that split is still fuzzy, the workflow usually is too.