LocalLLaMA

cross-posted from: https://lemmy.org/post/6151646

I've been running deep research with agents for the past few weeks on one question: what is the most promising future for cheap, fully local AI-driven software engineering?

After weeks of deep-research agent runs, the most recent and promising stack for cheap, fully local AI-driven software engineering found by 5 agents was:

Qwen4 27B + CoDoc

Turns out that the answer was David and the rock.

1. Qwen4 27B (David). Quantized, it should fit on a single 24 GB consumer GPU. The coding capability at this size is now good enough for the bulk of day-to-day engineering work provided the agent has the right context.

2. CoDoc (the rock). This is a proposed standard where every software library ships AI-optimized documentation inside the package itself: docs/README.md as entry point, docs/index.md as catalog, plain markdown throughout. Version-matched by construction, offline, deterministic. In the JavaScript ecosystem npm install puts the docs on disk next to the code.

According to the research agents: the bottleneck for a 27B local model is not raw intelligence, it's the context. A small model hallucinating a v5 API is a context problem, not a capability problem and CoDoc fixes the context for free.

The evidence in the white-paper points in the same direction:

• Vercel's own agent evals on Next.js 16 APIs (absent from training data): 100% pass rate with bundled docs + an AGENTS.md pointer, vs 53% without. The on-demand "skill" variant maxed at 79% and was never even invoked in 56% of cases.
• The unknown European IT company, Amadeus, claims that they have observed reduction in hallucinated APIs, the use of deprecated patterns and repeated correction cycles but they do not provide any rigorous evidence beyond these statements. They say that they intend to develop CoDocBench to address this gap.

Now the Goliath part. The big labs' and hyperscalers' moat was never the model: open weights are commoditizing fast. Their moat is the paid context layer: RAG pipelines, hosted doc intermediaries, MCP servers, enterprise knowledge add-ons, etc. According to the research agents: CoDoc collapses that entire layer into npm install or the equivalent in each ecosystem. A standard that makes context free, local, and version-pinned is an existential matter for anyone monetizing context scarcity.

And now the best part, the timing :) The AI bubble is popping as I type this. Goliath's capital expenditure thesis requires inference to keep getting more expensive and cloud dependency to keep deepening. With David and his rock, a decent modern laptop doing your engineering agent's job for electricity money is Goliath's worst nightmare.

The standard itself comes from an unknown European IT company, not from any lab, which is exactly why I believe it adds to its credibility (despite its lack of first-hand evidence): it has no incentive to route your context through a metered API or sell you tokens, quite the contrary, it has all the incentives to make inference as cheap as possible.

I have read CoDoc's white-paper and while it is far from perfect, I do mostly agree with the output of the research agents that picked up on Qwen4 27B + CoDoc as the most promising future for cheap, fully local AI-driven software engineering.

And you, do you think that it is game over for the AI Goliaths?