Back to Blog

Keeping the LLM on a Leash

·Geoff·6 min read
devlogaigame-design

A note on timing: I started this post and its follow-up back in September, then got busy with other projects. I've dusted them off and am publishing them in their current state, because "perfection is the enemy of progress." I'm also planning to explore the Gemini decision API, and these posts should be a good starting point for that.

One way to build a game on an LLM is to hand the model the keys. It narrates, it decides who won the fight, and it keeps track of what's true. When that works it's magical.

When it doesn't, you get a sword that was destroyed two turns ago showing up again, a player who "wins" a fight they clearly lost, and a bill that grows with every clever thing the model is allowed to do.

I tried a different pattern for NextOath.

Design Contract

The game decides what happened. The model tells the story about it.

Where the model has a say, and where it doesn't

The model still influences the game. It judges the plans players write and helps decide which cards they get. But once those cards are on the table, code owns the outcome. The model can interpret what a player meant. It can't rewrite what happened.

Here's one turn, step by step:

Model: judges and narrates, from a fixed menuCode: decides what happened
  1. PlayerPlayer writes a plan

    Free text, anything they like.

  2. ModelModel grades the plan
    It may only answerpooradequategoodexcellent

    Plus one fixed choice: what kind of move it is.

  3. CodeCode turns the grade into cards

    Fixed rules: a better grade earns stronger cards.

  4. CodeCode resolves the board

    Who owns what, who moved first, who collided.

  5. Outcome locked
  6. ModelModel narrates the result

    Describes what happened. There is no field in its reply that could change any of it.

The model shows up twice, at step 2 and step 5, and both times it's boxed in.

At step 2 it can only answer with a small set of words. It can't hand out "a slightly better card" or invent a new kind of card. The code decides what a grade is worth. A wrong grade means a card is a bit too strong or a bit too weak. That's a balance problem, not a broken game.

At step 5 it's handed the finished results: who tried what, whether it worked, and roughly how big the outcome was, in words like "a foothold" or "reshaped the front." It never sees a number it could fiddle with, and there's no field in its reply that could change a result. The prompt says it plainly: "The outcome of every action this round is ALREADY DECIDED — you only narrate it."

Why the model doesn't do steps 3 and 4

I wanted the strategy game to have clear, balanced rules. Rules and balance need a real rules system and tuning. I couldn't guarantee that a small model would reliably follow a complex rule system and deliver consistent rulings on it. (I tried...)

What the leash bought me

  • A bad model response is a clumsy sentence, not a broken game. The worst it can do is tell the story badly, or grade a plan poorly.
  • The board replays exactly. Every move a player makes is recorded, and the rules system plays it out the same way every time.
  • It's cheaper. The model does less, so a smaller, faster one can handle more of the work.

The trade-off

A leashed model is less surprising. It can't invent a twist that changes who's winning, because it isn't allowed to change who's winning.

If the story is the game, keep the model in charge. NextOath is a strategy game, and in a strategy game the rules have to actually be rules. That's the tack I chose for this game.

What the model is actually good at: judging

Games have always been good at judging things you can measure. Did the unit reach the square? Is this number bigger than that one? Judging a free-form idea — was that a clever plan? — was hard to do before LLMs, and what you could do was limited.

A model can judge an idea. A player writes "let them take the granary, then spread the rumor that the grain is blighted and buy it back cheap." The model in step 2 can grade that idea. A better plan earns better cards.

There's more than one way to do that grading step, and they behave differently in small models:

  • One plan at a time. Grade each plan on its own against a short checklist. This is what NextOath does. It's steady and consistent across a whole game, and it reliably separates a weak plan from a good one.
  • Two plans head to head. I also tried grading plans against each other in small models. That sounds like the fix for ties, but it has a nasty habit. The smaller models I was using would pick whichever plan they read second. It was position, not judgment, so I didn't structure decisions like this in the game.

Separately, at step 5, the model looks at everyone's moves and writes what happened between them: who blocked whom, whose trap caught whose push. That isn't grading, and it decides nothing. But it's where the model's ability to be creative shines, and it's the part of the game that feels most alive.

Be careful what you let the model decide.

Next

The next post is the practical one: the tricks that got small models to reliably give the game the answers it needs.

I've also put these ideas in a small companion repo, llm-on-a-leash, with runnable examples and a field guide to the patterns. It's how I handled these problems in this version of NextOath, using the small models available at the time. If you've got a better pattern, tell me — I'll add it next to mine, with your numbers. These patterns may age quickly with the release of more capable small models.

P.S. "Leash" sounds harsher than it is; I picked it to grab the reader's attention. The model still gets to write every word players read. The model doesn't get to keep score. Maybe that would have been a better blog post name: "The LLM doesn't keep score."

Follow our changelog for updates, and don't hesitate to reach out via our support page with ideas, feedback or questions.