In short: local coding works well for bounded, verifiable tasks. For complex changes across many files, API models are often still more effective. Today, the strongest setup is a hybrid workflow.
We tested several open-weight models on work resembling real projects: understanding existing code, writing tests, debugging, refactoring and making changes across a repository. We were not looking for a leaderboard winner. We wanted to answer one practical question: when is local coding actually worth it?
Where local models work well
The best results came from tasks with a clear scope and an output that is easy to check:
- generating tests and boilerplate;
- explaining functions and modules;
- repetitive transformations;
- local refactoring with explicit constraints;
- analysing code that should not leave the organisation.
In these cases, teams can iterate without per-token costs while keeping code on their own infrastructure.
Where APIs remain stronger
The most capable API models still have an advantage on ambiguous tasks, changes spanning many parts of a repository and problems requiring sustained planning and reasoning.
A local model is not cost-effective simply because each call is free. Hardware, setup, latency and review time all count. An answer that requires several corrections can cost more than an API call that works on the first attempt.
The practical conclusion
The right model depends on the task, available hardware and minimum acceptable quality. The workflow we currently prefer is:
- local for frequent, bounded or sensitive work;
- API for complex or high-risk changes;
- evaluation on your own code, not only public benchmarks.
Our complete guide includes the models tested, configurations, evaluation criteria and a practical outline for repeating the comparison in your own environment.