One simple rule: the LLM plays chess with its own reasoning. No scores, no centipawn values, no engine evaluation or calculation. Just cold raw facts from the board.
I've found that the moment you give an LLM an engine to evaluate with, it abandons its own reasoning and just tries to game the numbers blindly. Any "centipawn" values for exchanges or positions and it completely forgets how to reason at all.
This is my attempt: github.com/mfnbc/chessdb.nu
Current state:
Lasts 20–30 moves against Fruit (a weak chess engine). Probably playing around 800 ELO.
Plays very defensively, trying not to blunder.
Early on it complained about what it couldn't see. Now when it does blunder, it goes back and checks and sees the problem was reported.
Architecture (very simple):
Exposes Rust's Shakmaty for board facts. No scores, no evaluations (Shakmaty's scores are only hand-tuned and not battle-tested anyway).
Information is structured Nuon (like JSON).
Focus happens through Nushell's excellent structured language functions.
The rest are skills and tools the LLM developed on its own for visualization.
I coach it in between games and let it alter the skills and tools as it sees fit. That might sound lazy, but it's after many months of trying to impose my own opinion, which turned out to be the limiting factor. With my hand-written skills it only got to 10–20 moves (~400ELO; with its own skills it's doing noticeably better.
Open to:
Games by correspondence, Lichess, or ICC to test what works and what doesn't.
Forking — the architecture has some cruft from earlier work, so it might be easier to start fresh.
If you know of anyone else trying with similar constraints (raw facts, no eval), let's pit them against each other.
Feel free to read the Findings.md to see the journey.
One odd postscript, discussing chess with the computer has raised my ELO and consistency...