AI Chess Teacher

Problem

Chess engines are brutally strong and completely unhelpful to a seven-year-old. Stockfish will tell you a move loses 2.3 pawns of evaluation; it will not tell you "you left your knight where the bishop can grab it, try protecting it first." Kids need why, in language they understand, without the engine's condescension or the endless move-tree jargon.

Target user: children learning chess (and the parents/coaches guiding them).

Constraints:

• Feedback must be correct (a wrong chess explanation is worse than none) and age-appropriate.
• It has to be fast enough to feel like a conversation over the board.
• LLM cost has to stay sane — you cannot call GPT on every hover.

Business goal: turn engine-grade analysis into encouraging, understandable coaching that keeps a kid playing.

Architecture

The core idea is a hybrid: use Stockfish for ground-truth evaluation, then use an LLM purely to translate that truth into kid-friendly coaching. The engine decides what is good; the model decides how to say it.

• Next.js frontend renders the board and the coaching panel.
• FastAPI backend in a clean layered structure: api/ (routers), core/ (config, security), models/ (SQLAlchemy), schemas/ (Pydantic), services/ (Stockfish, LLM, game logic).
• Stockfish runs server-side for deterministic evaluation of a position and candidate moves.
• OpenAI turns the engine's numeric verdict into a short, warm explanation.
• Redis caches game state and — crucially — LLM explanations keyed on the position, so identical teaching moments are free the second time.
• Async SQLAlchemy + Alembic for persistence and migrations; Supabase Auth JWT for identity; Docker deployed on Render.com.

Engineering Decisions

Deterministic engine + LLM narration, not LLM-as-engine. Asking an LLM to evaluate chess positions directly is unreliable — models hallucinate legal moves and misjudge tactics. By making Stockfish the source of truth and restricting the LLM to explanation, correctness is guaranteed by the engine and only tone is delegated to the model. This is the single most important design decision in the project.

Clean layered FastAPI (api/core/models/schemas/services). With two model integrations and real game logic, keeping routers thin and pushing behavior into services/ kept the code testable and the LLM/engine swappable. Tradeoff: more files up front, big payoff in maintainability.

Async SQLAlchemy. Move coaching is I/O-bound (engine call + model call + cache), so async lets one worker handle many concurrent boards without blocking on the LLM round-trip.

Redis for both game state and explanation caching. Explanations for common positions (opening blunders repeat constantly at the kid level) are cached, so the expensive LLM call happens once per teaching moment, not once per kid. The non-obvious win is that kid-level chess has a small space of frequent mistakes — the same dozen opening blunders account for a large share of coached moves — so a position-keyed cache concentrates its hits exactly where the volume is. Tradeoff: the cache is only as safe as its key, which is why the transposition-normalized coach:v2: key and its explicit versioning got real attention; a sloppy key would either serve the wrong explanation (correctness bug) or never hit (cost bug).

A pool of Stockfish subprocesses, not one shared engine. UCI is a stateful line protocol — a single engine process can only work one position at a time, so sharing one instance across concurrent boards would serialize the whole system behind a mutex. Running a small pool of engine subprocesses lets async workers evaluate positions in parallel, and pairs with the per-engine timeout/recycle logic so a wedged process is replaced rather than blocking the pool. Tradeoff: each engine is real memory (Render resource limits made this a genuine ceiling, not a free knob), so the pool size is tuned against the box rather than set arbitrarily high.

Source Code Walkthrough

From engine evaluation to a kid-friendly prompt

The service takes a position, gets Stockfish's verdict, and feeds a structured summary into the LLM — never the raw board alone.

# services/coaching.py
# Why: Stockfish is the source of truth; the LLM only narrates its verdict.
# Why this shape: we pass the engine's structured judgment (best move, delta)
# so the model explains a FACT rather than inventing an evaluation.
async def coach_move(position: str, played_move: str) -> str:
    cache_key = f"coach:{position}:{played_move}"
    if cached := await redis.get(cache_key):
        return cached  # identical teaching moment -> free.
 
    eval_before = await stockfish.evaluate(position)
    best_move = await stockfish.best_move(position)
    eval_after = await stockfish.evaluate_after(position, played_move)
    delta = eval_before.cp - eval_after.cp  # centipawns lost by the kid's move
 
    prompt = (
        "You are a kind chess coach for a child.\n"
        f"They played {played_move}. The engine's best was {best_move}.\n"
        f"Their move changed the evaluation by {delta} centipawns.\n"
        "Explain in 2 short sentences, warmly, what to notice next time."
    )
    explanation = await openai.complete(prompt, max_tokens=80)
 
    await redis.set(cache_key, explanation, ex=60 * 60 * 24)
    return explanation

Performance considerations: the cache check comes first; the LLM is capped at 80 tokens (short is both cheaper and better for kids). Improvements: pre-compute explanations for the most common opening blunders at deploy time so even first-time hits are warm.

A cache key that survives transpositions

The naive cache key is the raw FEN. The problem: FEN includes halfmove and fullmove clocks, so the same position reached by different move orders (a transposition) produces different FENs and misses the cache. Normalizing the key to the parts that actually define the teaching moment is what pushes the hit rate to ~92%.

# services/cache.py
# Why: two kids reaching the same position via different move orders should
# share the same explanation. Raw FEN includes move clocks that differ between
# transpositions, so we strip them and key on (position, played_move) only.
def coaching_cache_key(fen: str, played_move: str) -> str:
    # FEN = "<board> <side> <castling> <ep> <halfmove> <fullmove>".
    # The first four fields fully define the position for coaching purposes;
    # the clocks are irrelevant to "is this a good move here?".
    board, side, castling, en_passant, *_clocks = fen.split(" ")
    canonical = f"{board} {side} {castling} {en_passant}"
    return f"coach:v2:{canonical}:{played_move}"

Why this way: the v2 prefix is deliberate — the cache key is part of the contract, and if we ever change the prompt or the normalization we must bump the version to avoid serving explanations generated under old rules. Edge cases / improvements: en passant and castling rights are kept because they genuinely change which moves are good; dropping them would be a correctness bug, not just a lower hit rate. A future version could canonicalize mirrored positions (color symmetry) to share explanations between White and Black teaching moments.

Isolating the Stockfish subprocess so one bad position can't wedge a worker

Stockfish is a long-lived subprocess speaking the UCI protocol over stdin/stdout. The failure mode that matters: a malformed position or a hung engine blocking an async worker forever. The service wraps every engine call in a timeout and guarantees the process is either healthy or replaced.

# services/stockfish.py
# Why: Stockfish is a native subprocess, not a library call. A wedged engine
# must not take an async worker down with it, so every eval is bounded by a
# timeout and a wedged process is recycled rather than awaited forever.
async def evaluate(self, fen: str, *, depth: int = 15) -> Evaluation:
    async with self._lock:  # one command in flight per engine instance
        try:
            await self._send(f"position fen {fen}")
            await self._send(f"go depth {depth}")
            info = await asyncio.wait_for(
                self._read_until_bestmove(), timeout=2.0
            )
            return self._parse_score(info)
        except (asyncio.TimeoutError, BrokenPipeError):
            # Recycle: a hung/dead engine is replaced, the request fails cleanly
            # rather than hanging the worker or returning a stale score.
            await self._restart()
            raise EngineUnavailable(fen)

Performance considerations: the per-engine _lock serializes UCI commands (the protocol is stateful — you cannot interleave two go commands on one process), so concurrency comes from a pool of engine instances, not from sharing one. Edge cases / improvements: depth=15 is a latency/accuracy knob — deep enough for correct kid-level verdicts, shallow enough to stay near the ~45ms budget. The EngineUnavailable exception is caught upstream and degrades to "let me think about that one" rather than a 500, so a recycled engine is invisible to the child.

Technical Challenges

• Correctness vs. friendliness. The whole system is built so the LLM cannot be wrong about chess — it only ever narrates a Stockfish verdict.
• Latency budgeting. Engine eval is ~45ms; the LLM is the expensive part. Caching explanations by position turns most interactions into cache hits (~92% hit rate), keeping felt latency low.
• Concurrency. Async SQLAlchemy + async engine/model calls let a single worker juggle many simultaneous games without head-of-line blocking.
• Deployment. Bundling the Stockfish binary into the Docker image and running reliably on Render.com took some care with resource limits.

Code Quality & Testing

Type safety. Every boundary is a Pydantic model. Request bodies, the structured engine verdict passed to the LLM, and the coaching response are all validated schemas in schemas/, so an ill-formed FEN or a missing field is a 422 at the edge rather than a KeyError three layers deep. FastAPI derives the OpenAPI spec from those same models, so the Next.js client's generated types cannot drift from the server's contract. mypy --strict runs over services/ and core/ where the real logic lives.

Testing strategy. The tests are organized around the project's core bet — the engine is truth, the model only narrates:

• Stockfish determinism tests. The same FEN at the same depth must return the same best move and the same centipawn score run-to-run. This is the test that guarantees the "ground truth" is actually ground truth; it also pins the bundled binary version, so a silent Stockfish upgrade that shifts evaluations fails CI.
• Contract tests on the LLM boundary. These don't assert the model's prose (untestable and brittle); they assert that coach_move always builds its prompt from a real engine verdict — the delta, best_move, and score are present and reference the actual position. The rule "the LLM can never be wrong about chess" is encoded as a test: the prompt is asserted to contain the engine's numbers, so no code path can narrate an invented evaluation.
• Integration tests use FastAPI's TestClient with a stubbed engine pool and a fake Redis to exercise the full router → service → cache path, including the transposition-key normalization and the EngineUnavailable degradation path.
• Cache tests assert that transposed FENs collide on the same key and that a prompt/version change bumps coach:v2: so stale explanations are never served.

Error handling & observability. Failures are typed and degrade gracefully: EngineUnavailable becomes a gentle "let me think about that one" rather than a 500; an OpenAI timeout falls back to a canned, engine-derived hint ("the best move was Nf3") so the child still gets correct feedback even when narration is down. Structured logs record cache hit/miss, engine depth, LLM latency, and token count per request; these feed the cache-hit-rate and cost dashboards, which are the two numbers that decide whether the economics work.

Code organization. The layered split (api/ routers stay thin, services/ owns behavior, schemas/ owns shape, core/ owns config/security) is what makes the engine and model swappable — the LLM provider is behind a single services/llm.py interface, so moving off OpenAI touches one file. Routers never call Stockfish or OpenAI directly; they orchestrate services.

Enforced in CI. pytest (unit + integration), mypy --strict, ruff (lint

• format check), and the determinism suite run on every push. The Docker image is built and smoke-tested — boot the container, evaluate a known position, assert the expected best move — so a broken Stockfish bundle is caught before it reaches Render.

Benchmarks

Stockfish evaluation alone is ~45ms. A full coached move with a cached explanation lands around 260ms; a cold LLM call is ~1.15s — which is exactly why the ~92% Redis explanation cache hit rate matters so much. See the charts above for the latency series.

Lessons Learned

• What succeeded: the hybrid design. Correctness from the engine, warmth from the model — neither doing the other's job.
• What failed first: an early prototype let the LLM assess positions directly; it hallucinated and gave bad advice. Never again.
• What I would redesign: move the explanation cache warming into the deploy pipeline rather than lazily on first hit.
• Remaining tech debt: the prompt is a string template; it should be a versioned, testable artifact.
• Future improvements: difficulty-adaptive coaching that adjusts vocabulary to the child's level.

Media

• Recording of a coached game with live explanations under each move.
• Screenshots of the board + coaching panel.
• Diagram of the Stockfish → LLM → Redis flow.

Documentation Links

• Internal design notes cover the hybrid engine/LLM coaching pipeline and the layered FastAPI service structure.

Build Logs

• Requirements & planning. Defined the core bet: engine for truth, model for tone. Everything else follows.
• Architecture. Layered FastAPI, async SQLAlchemy, Redis for state and explanation caching.
• Implementation. Wired Stockfish evaluation, the coaching service, and the structured LLM prompt; added Supabase Auth JWT.
• Testing. Verified explanations always reference a real engine verdict; cache-hit and concurrency tests.
• Deployment. Dockerized with the Stockfish binary; deployed on Render.com.
• Monitoring. Cache hit rate and LLM latency/cost tracking.
• Retrospective. The hybrid architecture is reusable anywhere you need correct-but-friendly explanations of a deterministic system.