Part 3 of a five-part series: From Cloud AI to Local Models.
"Local models are less capable" is true in the way most incomplete statements are true.
Frontier cloud models are better generalists. They know more, reason across more domains, handle more ambiguous prompts, and recover from messy instructions more often. If the task is "answer any question a person might ask," the largest hosted models are usually the right bet.
But engineering work is not one task.
And capability is not a single number.
Coding assistance is not one job
When people talk about "AI for coding," they often compress a whole field of work into one vague category.
But the actual jobs are different:
- autocomplete
- explanation
- code search
- refactoring
- test generation
- code review
- migration planning
- incident analysis
- log summarization
- dependency upgrade help
- documentation updates
- architecture critique
- scaffolding a new feature
- translating intent into a patch
Those tasks do not need the same model. They do not need the same context. They do not fail in the same way.
Autocomplete needs low latency and local syntax awareness. Review needs judgment and suspicion. Migration planning needs repository context and historical patterns. Incident analysis needs logs, runbooks, topology, and current system state. Documentation updates need style consistency and the discipline not to invent behavior that does not exist.
Saying one model is "more capable" without naming the task is like saying one database is "better" without saying whether you need transactions, search, analytics, or a cache.
It might be true. It is not yet useful.
Frontier models are better generalists
The honest local-model argument starts by admitting the obvious thing.
Frontier models are better at being dropped into unknown situations. They are better when the prompt is vague, the domain is broad, the task spans several areas, and the user expects the model to fill in missing context from general knowledge. They are especially useful when the work is high-value and the cost of being wrong is not just a failed tool call but a bad decision.
There is no prize for pretending otherwise.
If I need a model to reason through an unfamiliar architecture, critique a security-sensitive design, or synthesize tradeoffs across product, operations, and implementation, I want the strongest generalist I can get.
But most internal engineering assistance is not that open-ended. It is specific. It lives inside a repository, a build system, a deployment model, a set of conventions, and a history of mistakes the company already paid to learn.
That is where specialization starts to matter.
Local models can be better specialists
A smaller local model does not need to know everything if the system around it gives it the right things.
It does not need to have memorized your codebase if it can search it.
It does not need to guess your test command if the harness supplies it.
It does not need to know your deployment rules from pretraining if your internal docs and policies are in context.
It does not need to infer ownership if the repository metadata and on-call history are available.
It does not need to hallucinate an API if it can read the generated types or the OpenAPI schema.
That is the key shift. The valuable unit is not "the model." It is:
model plus context plus tools plus evaluation.
Once you see the system that way, local models become much more interesting. They are not competing with frontier models as generic brains floating in a chatbox. They are components inside a workflow that can be supplied with specific files, symbols, logs, docs, commands, and feedback.
A mediocre prompt to a great model often loses to a decent model inside a good system.
Context beats vibes more often than people admit
The difference between useful and useless AI coding help is often not model quality. It is whether the model is operating from evidence.
Ask a model to "fix the bug" with no repository context and it will produce a plausible story. Give it the failing test, the relevant files, the recent diff, the stack trace, the dependency versions, and the command that reproduces the failure, and suddenly a less impressive model has enough structure to do real work.
This is why local models pair so naturally with engineering tooling. They can sit close to the filesystem, the index, the test runner, the logs, and the company's internal docs. They can be embedded in workflows that force them to look before they answer. They can be constrained to cite files, run commands, and treat verification as part of the loop instead of a sentence at the end.
That does not make the model magically smarter.
It makes the task less dependent on magic.
Evaluations change the argument
The weaker version of the local-model argument leans too hard on vibes: "they're getting good enough."
The stronger version asks, "good enough for which measured workflow?"
Recent model evaluations keep showing a pattern that should change how teams think about this. Local and open models can match or approach mid-tier hosted models on some structured tasks, especially when the task is narrow and the evaluation is explicit. They also still expose real weaknesses, especially in long-context reasoning, complex correction workflows, and situations where the model has to recover from its own bad intermediate assumptions.
That is the honest story.
Not "local wins everything."
Not "cloud is always necessary."
The useful claim is narrower and stronger: local models are already good enough for more engineering work than many teams route to them, as long as the task is bounded, the context is supplied, and the evaluation loop catches failures.
That should push teams toward measurement, not ideology.
Run the local model on your review summaries. Run it on your test generation. Run it on your incident notes. Run it on your code search and doc cleanup. Track where it succeeds, where it fails, and where escalation to a stronger model pays for itself.
That is how infrastructure decisions get made.
The model alone is the wrong abstraction
If the only question is "which model is best," the answer will usually be the largest frontier model you can afford.
But that question hides the architecture.
The better questions are:
- What context does the model get?
- What tools can it use?
- What can it verify?
- What is the cost of a bad answer?
- Can the task be decomposed into smaller checked steps?
- Is there a local model that handles the common case?
- When should the system escalate to a stronger model?
Those questions turn AI from a vendor comparison into a design problem.
That is where engineering judgment belongs. The model is not the whole system any more than the database is the whole product. It is a powerful component whose usefulness depends on the boundaries and feedback loops around it.
The reframe
Local models are less capable if you mean "less capable as general-purpose brains with no support."
But that is not how serious AI-assisted engineering should be designed.
A local model with repository search, structured context, reliable tool access, project-specific docs, and a tight verification loop can beat a stronger cloud model used as a vague autocomplete oracle. Not because the local model is secretly smarter, but because the system asked a better-shaped question.
Capability lives in the whole stack:
- the model
- the context
- the tools
- the workflow
- the evals
- the escalation path
Once teams start measuring that stack instead of worshiping a single leaderboard number, the deployment picture gets more interesting.
Frontier models remain valuable.
Local models become useful infrastructure.
The win is knowing which one belongs in the loop.
Previous: Part 2 — Fast at What? Next: Part 4 — The Per-Token Tax on Engineering