Part 4 of a five-part series: From Cloud AI to Local Models.
Cloud AI pricing looks reasonable when AI is occasional.
A few prompts a day. A pull request summary. A debugging session. A design review. At that scale, paying per token feels like paying for convenience, and most teams are right not to overthink it. The model is useful, the bill is manageable, and the alternative is spending engineering time operating something you barely use.
Then AI gets good enough to put everywhere.
That is when the economics change.
Occasional tools can be rented casually
There is nothing wrong with renting intelligence for occasional work.
If a cloud model saves an engineer an hour, the cost of the call is not the interesting number. If a frontier model helps untangle a production incident, you should not be doing token math in the middle of the incident. If a hosted model makes a high-value architecture review better, pay the bill.
This is the mistake some local-first arguments make. They treat every cloud call as waste, as if the goal is purity instead of leverage.
The goal is not to minimize tokens.
The goal is to spend intelligence where it changes the outcome.
At low volume, cloud models are often the obvious answer because the operational cost of self-hosting would dwarf the usage cost. But low-volume assumptions do not survive once AI moves from "tool people sometimes use" to "substrate under the engineering workflow."
Infrastructure volume is different
Imagine the model is no longer just a chat window.
It is in the editor, suggesting small changes all day. It is in CI, explaining test failures. It is in code review, summarizing diffs and flagging risk. It is in support, classifying tickets. It is in incident response, reading logs and runbooks. It is in documentation, keeping internal pages aligned with code. It is in migration tooling, dependency upgrades, release notes, onboarding, backlog cleanup, and internal search.
Each call is small enough to justify.
The aggregate is not small.
That is the moment AI stops behaving like a subscription and starts behaving like infrastructure. The cost is no longer "what did this one useful answer cost?" The cost is "what recurring tax did we attach to every engineering workflow we want to improve?"
The per-token model is clean because it maps directly to vendor cost. It is also dangerous because it turns natural usage growth into a tax on adoption. The more teams integrate AI into the places it helps, the more the bill becomes a shadow platform cost.
If the model is part of the build system, the IDE, the review process, the incident loop, and the docs pipeline, it is not an add-on anymore.
It is a platform dependency with a meter attached.
Local changes the shape of the bill
Local and self-hosted models do not make cost disappear.
They move it.
Instead of paying a provider for every token, you pay for hardware, power, runtime operations, storage, deployment, monitoring, upgrades, and the people who keep the stack healthy. That can be worse if the usage is low, the team is small, or the workload needs frontier reasoning most of the time.
But for high-volume bounded tasks, the shape of local cost is attractive.
The marginal call gets cheaper. Failed experiments stop feeling like billable mistakes. Internal automation can run more freely. Teams can build loops that call the model many times because the cost is capacity planning, not a transaction fee on every attempt.
That matters for engineering workflows because many useful AI systems are not one-shot. They are iterative. They classify, inspect, retry, compare, verify, summarize, and refine. Charging every micro-step at cloud rates pushes teams toward fewer, bigger calls. But many reliable agent workflows want more, smaller calls with checks in between.
The economics of the model shape the architecture of the workflow.
The right answer is routing
The future is probably not pure cloud or pure local.
The practical answer is a router.
Use local models for the common, bounded, high-volume work:
- repository-aware search
- first-pass summaries
- log classification
- test scaffolding
- simple refactors
- documentation cleanup
- repetitive review assistance
- private internal context
Use frontier cloud models for the moments where stronger reasoning is worth the cost:
- ambiguous architecture questions
- security-sensitive design review
- hard debugging
- complex migrations
- unfamiliar domains
- high-impact decisions
- synthesis across messy constraints
That split is not about pride. It is about economics, latency, privacy, and capability matching the work.
The important part is that routing should be explicit. If every task defaults to the strongest model because that was easiest to wire first, the team has not chosen a strategy. It has accepted a bill.
Measure workflows, not tokens
Token cost is visible, so teams overfocus on it.
The better metric is cost per successful workflow.
How much does it cost to produce a useful review summary? To triage a failed CI run? To update docs after a code change? To classify a support ticket correctly? To run an agentic refactor until tests pass? To generate incident notes that the on-call actually trusts?
Those numbers include model cost, but they also include latency, retry rate, developer attention, privacy review, tool failures, and the cost of correcting bad output. A cheap model that produces noise is expensive. An expensive model that solves a rare hard problem can be cheap.
This is why the local-versus-cloud argument gets silly when it stays abstract. The right answer is workload-specific. Route the repetitive, measurable, context-heavy work to cheaper local capacity where it performs well. Escalate when the local system lacks confidence or the task deserves more reasoning.
That is how every mature platform handles resources.
Cost is also permission
There is a softer effect here that shows up in team behavior.
When AI calls feel expensive, people ration them. They use the model for the obvious high-value moments and avoid embedding it into every small workflow where it might help. That restraint is rational, but it also means the organization never discovers the long tail of boring automation that could have compounded.
Local capacity changes the permission structure.
If internal summarization is effectively free at the margin, more systems can summarize. If local code search can call a model repeatedly without opening a budget conversation, more tools can become model-aware. If failed attempts are capacity noise instead of invoice line items, teams experiment more.
That does not mean waste is good. It means the cost model either encourages or discourages exploration. Once AI becomes infrastructure, that matters.
The reframe
Cloud pricing is fine for occasional intelligence.
It gets more complicated when intelligence becomes part of every engineering loop.
At that point, the question is not "can we afford this one call?" The question is "what cost model do we want attached to the way engineering work happens here?"
Some calls should absolutely go to frontier models. They are worth it. The goal is not to replace every cloud call with a local one.
The goal is to stop paying frontier-model prices for every repetitive, private, bounded, high-volume task just because the first deployment model happened to be an API.
Once AI becomes part of the build system, the IDE, the review process, the incident loop, and the docs pipeline, it stops being a subscription.
It becomes infrastructure.
Infrastructure needs routing.
Previous: Part 3 — Capability Is Not a Single Number Next: Part 5 — Models That Know How Your Company Thinks