I Taught an AI to Play My Game. The Training Was the Easy Part.

by
Offline Arena: a cartoon bot in a scuffed red helmet with a glowing antenna crouches behind a parked car on a city street at dusk, a thought bubble showing a lit green medical cross, a blaster and footprints, while the game's hero watches from the park wall

In my last post I said I’d been teaching an AI to play my game, and that more training had taken it “from near the bottom to mid-table, about as good as the game’s own bots”.

That was wrong. It was worse than the bots. I’d believed a score that turned out to measure nothing, and a sample too small to tell me anything.

This is how I found out, and what fixed it.

Where we left it

Offline Arena is the phone game I built in a weekend, a last-one-standing shooter where every other player is a bot. The bots run on hand-written rules.

There’s a new type of AI called a decision model. It doesn’t chat. It answers in one of three ways: a yes or no, which it calls a boolean, a choice from a list, like A, B, C, D or E, or a score on a scale. It also tells you how sure it is. Jev is the popular one, and it runs in its maker’s cloud. I wanted a free one that runs on my own laptop, so I used Laya.

Decision models are incredibly fast. On my MacBook, Laya answers in under a tenth of a second, which is about 12 decisions a second. A chatbot would still be clearing its throat.

That speed is what a game needs. About twice a second, one bot in the match hands Laya a paragraph about what it can see: its health, its gun, the nearest enemy, the nearest medkit. Laya answers with one of seven choices: fight, flee, heal, loot, head for the safe zone, roam or search. The bot’s own code then carries that choice out. Laya decides. It doesn’t aim.

70% on paper

Out of the box Laya was hopeless. In a test of 50 real situations it chose “heal” 23 times, 12 of them at full health. It never once headed for the safe zone.

So I trained it, and it took three goes.

  • First it copied what the bots did. The flaw is obvious in hindsight. A model that copies the bots can only ever be a bot.
  • Then I had bots try things they wouldn’t normally choose, and scored each choice by where that bot finished the match. Laya learned from those scores. In play it averaged 14th out of 19.
  • Then a full retrain on the same scores. On test questions it had never seen, its score went from 45% to 70%.

70% felt like progress. I recorded 20 matches and it averaged just under 11th, with no wins. Mid-table. I updated the last post to say so.

A second opinion, from the wrong model

Claude Code with Opus 5.5 had built all of this, and it couldn’t get Laya past the bots it was learning from. Its next idea was to look ahead: at each decision, play the match forward once for every choice and take the best one. That did worse than the plain bots in 6 matches out of 7.

So I asked Anthropic’s top model for a review. I meant to use the new one, Fable 5.2. My session was stale and I got Fable 5.1 without noticing.

Its verdict was bleak. Laya only picks from seven choices and bot code does the rest, so the best it could ever be was one ordinary bot, with a 1 in 19 chance of winning. Aim and reactions, it said, were not what held it back.

I restarted, picked Fable 5.2 on purpose this time, and asked it to challenge that review instead of repeating it.

Measure the match, not the answer

The first thing Fable 5.2 did was build a faster test. 600 complete matches in about two minutes, each one repeatable, so two different players can be dropped into exactly the same match and compared. From then on everything was measured in finishing places over hundreds of matches. No more scores on test questions.

Four things fell out of that.

My “mid-table” was luck. Over 200 matches the trained Laya averaged 11.7 and won 2% of them. The plain bots, in the same 200 matches, averaged 9.8 and won 7%. A single match can move a bot’s finish by five places either way, so 20 matches can easily be a place out. Mine were. One in five of its answers was “fight” with nobody there to fight.

The 70% was measuring noise. The “right answer” for each situation was whichever choice had the best average finish. But the gap between the best choice and the second best was usually about a third of a place, in a game where luck moves you five. When Fable split the data in half, the two halves agreed on the best choice somewhere between 52% and 60% of the time. Laya had been memorising a table of near coin flips, and 70% was how well it had memorised it.

The ceiling was wrong. Fable went looking for rules that actually win, and tested each one over 600 matches. It kept three changes to the way the bots already play:

  • Hurt, with a medkit in reach? Heal first, then fight.
  • A better gun or armour in reach? Pick it up before walking to the safe zone.
  • An armed enemy in sight? Fight, don’t carry on looting.

A bot playing by its normal rules won 5% of 600 matches. With those three changes it won 14.7%. Same seven choices, same bot code doing the work. So much for one ordinary bot.

And aim does matter. The game has three grades of bot. The sharp ones win four times as often as the rookies. The earlier review had said the opposite, from a much smaller sample.

There was a fifth, smaller thing. The paragraph Laya reads left out facts it needed. It never mentioned power-ups, for one. A simple stand-in student, given only the facts in the old paragraph, could copy the winning rules 91% of the time. With five missing facts added, 98%.

It wasn’t all clean. Part way through, Fable ran a command to stop some test runs and killed Claude Code itself, taking two long runs with it. When I asked what had happened it told me the cause was unknown. When I pushed, it traced it back to its own command and said so.

The top model wrote the plan. The cheapest one built it.

In the last post I admitted that I have top models write their briefs for a smaller model, and that I should have aimed mine at Haiku and then gave the work to Opus anyway.

This time I did it properly. I told Fable: “this should be done by cheaper agents than you. create a detailed build plan that Haiku can follow.”

The plan has 18 steps. Each one gives the exact command to run and the exact output to expect, down to the last digit, which works because the test matches are repeatable. One rule sits above the rest: if the output is different, stop. Don’t fix it.

Haiku agents ran the whole thing. They stopped once, at step 5, because two percentages were out by one point. Fable checked. The plan was wrong. It had copied those two numbers from an earlier run and never refreshed them, and the files Haiku had built were right.

In the last post, one model reviewing another caught the mistakes. This time the cheapest model caught the most expensive one, just by following its instructions to the letter.

The training itself took eight hours on my MacBook.

Did it work?

The new Laya gives the same answer as the three-rule recipe 99.4% of the time, on 5,000 situations from matches it never saw.

That’s another score on paper, and I’d just learned what those are worth. So, 200 matches:

  • The old Laya: average finish 11.7, 4 wins.
  • The game’s own bots: 9.8, 14 wins.
  • The new Laya: 7.6, 20 wins.

About two places better than the bots, in exactly the same matches.

Then I put it on camera. The first 20 matches on watch.offlinearena.com had five second places and no wins. I asked whether 20 more might get it a win. Claude’s condition was that the new matches get added to the first 20 and never swapped in for them. A post about clearing noise out of the data shouldn’t end with a hand-picked set of results.

All 40 are up. It won two and came second nine times. It also came 18th twice.

What it actually learned

Here’s the catch. Laya didn’t learn to play. It learned to copy three rules that a much bigger model found by measuring. It is exactly as good as that recipe and no better. If I want more, the recipe has to improve, or Laya needs better aim. With a sharp bot’s aim, the same recipe won 29% of its test matches.

I still think that’s the real finding. Every attempt that failed was a training attempt, and training was never the problem. I didn’t know what good play looked like, and I was keeping score with a number that didn’t measure it. Once there was a measured answer worth copying, a small model learned it in a day on a laptop. It now plays from a paragraph of English, in a fraction of a second.

Training took eight hours. Finding something worth teaching it took four days and three wrong turns.

Watch it, and tell me what it gets wrong

You can watch Laya play at watch.offlinearena.com. You see every choice it makes, what it could see at the time, and how sure it was. If you spot it doing something daft, put it in the comments. That’s how the recipe gets better.

The game is still free, with no ads and no sign up: offlinearena.com.

Next time, the question I promised to answer. Did anyone who isn’t related to me come back to play? The pass mark was 1 in 5, over 20 phones. I have the 20 phones now.

I Vibe Coded a Game in a Weekend. Writing the Code Was the Easy Part.

by
Offline Arena: a cartoon soldier in an orange-striped helmet takes cover behind a parked car on a city street, with two bots approaching from the park

I was flying back from a work conference in Berlin on a budget airline. No entertainment, no Wi-Fi. Usually I’d use a flight to catch up on a film, or plug the laptop in and work. On this one I could do neither.

The colleague sitting next to me had an offline game. I had nothing. Try as I might, I couldn’t help noticing them busy playing away on an arcade puzzle the whole way home.

So, out of frustration, the first thing I did when I got home was build a game. Free to play. No ads. Works offline.

It was inspired by a top-down shooter I played years ago. I can’t remember its name, it must have been pulled from the App Store, and I can’t play it any more. But I remember the bits I liked. I wanted those, plus the best bits of all the best games I remember from the last 20 years, to make something new.

That Saturday I asked Claude to build it.

By Sunday night it had four maps, night and rain, a training mode, sound and music, unlockable skins, an offline mode, and my family playing it on their phones and a tablet. There’s a link to play it at the end.

That isn’t the interesting part. Anyone can get an AI to write a game now.

I’ve built an agentic developer harness, FADE, for exactly this kind of work. I wrote about it here. This time I deliberately didn’t use it. I wanted to vibe code it, to see what difference that makes.

What I found is that almost none of what made the game good was the code. It was the engineering discipline, and it crept in even without the framework.

Writing code took minutes. Knowing it was wrong took telemetry.

The first version worked, and it played well enough. If I’d stopped there I’d have told you AI can build a game in an afternoon, and I’d have been wrong about what I had.

On the first evening the game started sending one small record per match back to my server: how long you lasted, how you died, what gun you had, how accurate you were. A random ID per phone, no names, no location. The start screen says so.

Before anyone played, Claude ran a test match with a player who never moved and never fired. It came 5th out of 20. The bots were so busy killing each other that doing nothing was a strong tactic. No amount of reading the code would have told me that. One test did.

Then my family played 37 matches. The numbers were brutal. On one phone, 18 of 28 finished matches ended in 20th place, most of them between 4 and 16 seconds in, usually with a nearly full gun. Nobody told me that. They just kept pressing Play again. The data said it.

The cause turned out to be a bug. The bots hold fire for the first 12 seconds of a match. The player’s auto-fire didn’t. So every new player started a gunfight they couldn’t win, against a bot that had looted something better. You’d never spot that by playing.

The same thing happened again and again over the weekend:

  • My son Luca kept coming last and said he couldn’t aim. “I know I have to face straight, I don’t know exactly what angle.” I suggested a laser pointer from the gun. Later, when the data showed new players dying in the first few seconds, we added a short training level that runs the first time you press Play.
  • On the Fire tablet at home the game felt sluggish. The telemetry put a number on it: about 12 to 15 frames a second, slow enough that the whole game ran in slow motion. Now the game notices and lowers its own resolution.
  • On the Monday I complained that the forest map, meant to be the easy one, felt like the hardest. The data agreed it had the worst results. A simulation said the map wasn’t the problem: most of those matches came from our weakest player on that slow tablet.

My family tell me how it feels. The data tells me why. You need both, and the second one you have to build on purpose.

No single AI was in charge

I didn’t use one AI. I used tools from two companies, and the most useful thing any of them did was catch another one’s mistake.

It helps to keep three layers apart. There’s the company: Anthropic or OpenAI. There’s the harness, the tool you actually work in, which reads your files, runs commands and calls the model: Claude Code is Anthropic’s, Codex is OpenAI’s. And there’s the model, the brain inside, which comes in tiers. Anthropic’s Claude models run from Haiku through Sonnet and Opus up to Fable. OpenAI’s GPT-6 models run from Luna through Sol up to Astra.

Here’s who did what:

  • Claude Code with Opus 5.5, my daily driver, did the building: the engine, the bots, the maps, the deploys.
  • Claude Code with Fable 5.1, Anthropic’s top model, did the adversarial reviews. I asked it to find the bugs, the logic flaws and the design problems, and to change nothing.
  • Codex with GPT-6 Astra, OpenAI’s top model, did the artwork for the desert, city and neon menus and wired it into the game, and gave me the brutal second opinions, including a release review.

One habit I always keep. When a top model breaks work down, I have it write the brief as if a smaller model will do the work. A bigger model can infer things and figure stuff out. A smaller one won’t, and it will make mistakes, so it needs explicit paths and explicit instructions. Fable’s review came with a fix brief written for Sonnet, the middle tier. If I’m honest, I should have aimed it at Haiku. Then I gave it to Opus to build anyway. I’d been away for a week and hadn’t spent my token budget, so I used it.

The reviews earned their keep. Astra’s first review found that a win could turn into a defeat, because damage kept landing after the match had ended and two end-screen timers raced each other. Opus checked every line reference before fixing anything. All of them were right. Then, while fixing them, Opus introduced a crash of its own on loading. The test run caught it before it reached a phone.

Fable’s review found that one error anywhere would freeze the game for good, and that on older iPhones the City map threw one on its very first frame. It found that about 2 in 100 forest and desert matches ended with the final safe circle somewhere nobody could stand.

Astra later did a release review and refused to sign it off for the App Store. Fair enough. It listed the reasons, Opus checked each one against the code, and the real ones got fixed.

Then I asked Fable the question I actually cared about: would a human developer look at this and see good engineering, or vibe coded slop?

Its answer was more useful than flattery. As a two day prototype, it said, a developer would call it good work: decisions written down with dates and names, commit messages that say what changed, a deploy that checks the live copy byte for byte, telemetry with a report. “None of that is typical of slop.” As the codebase for an App Store product, it isn’t there yet: one 1,300 line file where everything shares the same 62 variables, 37 empty error handlers, and no committed tests until the end of the first day.

Both halves are fair, and I’d rather hear it now than from a developer later.

What I took from it: an AI checking its own work is weak. A different AI checking it, with instructions to find problems and change nothing, is strong. And whoever fixes the findings still has to verify them, because reviewers are wrong too. Fable criticised one bot behaviour that had already been removed.

The bit the AI can’t do for you

So what did I actually do all weekend? Mostly, I made decisions.

It had to be offline, with no servers and no ads. Both control schemes stayed, because I couldn’t tell which was better and neither could my family. The reviews happened before strangers saw it, not after. And before I spend anything on the App Store, I set a pass mark: at least 1 in 5 new players must come back the next day, measured over at least 20 phones that don’t belong to my family. If it fails, I fix the game instead of paying Apple.

Even the safety net had a hole in it. The deploy script was supposed to refuse to say “deployed” when the live copy didn’t match. A review found it printed “deployed” anyway. That’s the kind of mistake that hides for months, and it was in the very thing meant to catch mistakes.

I didn’t plan any of this. I set out to vibe code. But I couldn’t help going back to clear requirements, clear evidence from the telemetry, engineering standards and adversarial review. All of it crept in.

I’ve always thought of vibe coding as a dirty word. Asking for code and shipping whatever comes back is still dangerous. But maybe, if you bring the engineering discipline of what good looks like, vibe coding gets you somewhere real. The tools that make it safe are old and boring: measure what users actually do, get a second pair of eyes from someone who didn’t write it, write down why you decided things, and don’t believe “it works” until you’ve seen it work.

The AI wrote the code in minutes. The engineering was everything else.

Then I tried to teach an AI to play it

Once the game worked, I wondered whether an AI could play it instead of my hand-written bots.

There’s a new type of AI called a decision model. It doesn’t chat. It answers in one of three ways: a yes or no, which it calls a boolean, a choice from a list, like A, B, C, D or E, or a score on a scale. It also tells you how sure it is. Jev is the popular one, and it runs in its maker’s cloud. I wanted a free one that runs on my own laptop, so I used Laya.

Decision models are incredibly fast. On my MacBook, Laya answers in under a tenth of a second, which is about 12 decisions a second.

Out of the box it was hopeless. Given a bot’s choices (fight, run, heal, grab a better gun, head for the safe zone), it kept choosing to heal at full health. So I trained it on my laptop, using thousands of matches the bots played against each other.

That’s the next post. If you want to see where it has got to, you can watch it play at watch.offlinearena.com. One bot in each match takes its decisions from Laya, and you can see every choice it makes, what it could see at the time, and how sure it was. It’s still learning. Since I wrote this, I found out the answers I’d been training it on were mostly noise, and fixed them. In testing it now finishes about two places higher than the game’s own bots. In its first 40 matches on the site it won two and came second nine times.

Play it, and tell me what’s wrong

The game is free, with no ads and no sign up: offlinearena.com. It runs in your phone’s browser, and once you’ve opened it, it works with no signal. The other players are bots, and the game tells you so.

If you’ve got advice, or something annoys you, or you find a bug, put it in the comments on this post. I read them all, and they go straight into the next build.

The first person outside my family to play it was someone I only know from another phone game. They did the training, then won their first match. Since then eight phones I don’t know have played it, most of them from a Facebook post. Of the six that have had a day to come back, none has. Six is far too few to mean anything. The pass mark needs twenty, and I’ll tell you how it goes.

I Asked Free AI to Build a Real Game. It Failed Until I Changed One Thing

by

NVIDIA hands out free developer keys to dozens of open AI models. I pay for AI every month, so I wanted to know how far free really gets you. Not on quiz questions. On a real job, the kind I’d normally give a paid model.

So I asked three of the best free models to build a phone puzzle game from scratch. The first attempt produced nothing you could play. Neither did the second. By the end of the night one free model had built a game that works, and you can play it below. What changed wasn’t the model.

What free gets you

You sign up at build.nvidia.com, verify your account and generate a key. The page that shows the key says two things you should take seriously: “This is the only time your key will be displayed”, and that the key “is for API testing use only and is valid for 6 months.” Copy it somewhere safe straight away.

NVIDIA’s own FAQ is clear about the deal: “All models on build.nvidia.com are free to prototype with.” The limit is up to 40 requests a minute for most models, with no charge per word. And they say why it’s free: “The free tier exists so you can verify that yourself before committing to anything.” So this is a shop window, not a subscription. Keep that in mind for everything below.

The website listed 45 models. The key itself unlocked 85. Some are for driving cars, designing proteins or reading quantum computer charts. I ignored those and tested the 17 that hold a conversation.

The warm-up: 17 models

I gave every model the same two jobs.

The first was a sum with a trap in it: a train leaves at 14:37 and the journey takes 2 hours 48 minutes, what time does it arrive? Reply with just the time. (It’s 17:25.)

The second was the thing that decides whether a model can do real work: I gave it a “read file” tool and asked it to read a file and tell me what was in it. A model that can’t call a tool properly can chat, but it can’t act.

Model Sum Used the tool Speed Verdict
Nemotron 3.5 Super (NVIDIA) Right Yes About 1 second The best all-rounder I found
MiniMax M3.1 Flash Right Yes 1 to 2 seconds Strong, built for coding and agents
Laguna XS 2.1 (Poolside) Right Yes Under a second Built for coding. Often too busy, see below
Nemotron 3 Ultra (NVIDIA) Right Yes 2 to 30 seconds The big one. Slower to start, very capable
Nemotron 3.5 Lightning, gpt-oss 20B, Nemotron Nano Omni, Muse Glimmer Right Yes 1 to 6 seconds Fine, nothing special
DiffusionGemma, Nemotron Bolt Right Yes Under a second Tiny and fast. Fun, but small
Llama 3.2 11B Wrong (16:25) Yes 1.5 seconds No
Step 5 Preview Right Yes 14 to 100 seconds Too slow
GLM 5.3 Flash Right Yes About 2 minutes Too slow
GLM 5.3 Right Timed out Over 2 minutes Its page said “Deprecation in 4d”
DeepSeek V4.1 Flash, Gemma 4 31B, Llama 3.2 90B No answer No answer Nothing in 3 minutes, twice Didn’t turn up

The famous names were the problem. DeepSeek, Google’s Gemma 4 and Meta’s biggest Llama gave me nothing at all, on two separate tries, three minutes each. I don’t know why, and NVIDIA doesn’t say. My guess is that the popular models are the busiest, and on a free tier busy means you wait in a queue that never moves.

Lesson one: a free catalogue is not a free service. The list tells you what exists. Only testing tells you what answers.

The ones that did answer got everything right, fast. It looked easy. A sum and a file prove very little, though, so I gave the best three a real job.

The real test

Earlier this month I had Claude build my wife Heidi an ad-free copy of a phone puzzle she plays called Arrows. The board is packed with arrows, some straight, some bent like snakes. Tap one and it slides out the way its head points, but only if nothing is in the way. Clear them in the right order and you win. Drawing that is easy. Making a board that is full, always solvable and still makes you think is not. And I had my own version to measure against.

I took the three best models from the table. Laguna, the one built for coding, would have been a fourth, but NVIDIA’s server for it kept saying it was full: “Worker local total request limit reached (79/32)”. Each model got a one-page brief: build the game in an empty folder, no logins, no photos, 30 levels that get harder, plus a daily puzzle.

Then I marked the results with a test they never saw. It generated every level and checked each board was full, solvable and deep. Deep means how many rounds it takes if you tap every free arrow at once. My version goes from 12 rounds on level 1 to about 33 on the hardest. Chrome then opened each game at phone size, pressed Play and tapped through level 1.

Round one: nothing to play

None of them made a game you could play. Super wrote no files at all, twice. Ultra and MiniMax used up the 25 minutes I gave them. Their boards were valid but shallow, 3 to 8 rounds deep, and tapping only threw errors. I called MiniMax’s attempt “flashy junk, the arrows are just left and right, so no skill at all regardless of how many arrows”. So the test got stricter: at least a fifth of the arrows must bend, and heads must point all four ways.

Super’s silence turned out to be a setting. Pi limits each reply to 16,384 tokens unless you say otherwise, and Super spent the whole allowance thinking before it wrote a word. If a model goes quiet on you, check this first. I raised the limit to 65,536 and ran round one again, one model at a time.

It made no difference. Super’s boards were trivial, 3 to 5 rounds deep. Ultra wrote half a game with no screen to play it on. MiniMax spent 25 minutes trying ideas in throwaway snippets and left an empty folder.

Was it the brief?

The brief was written by Claude Opus, the paid model I use every day. So I asked it straight: was that prompt really aimed at these models, or at Opus? Its answer was mostly Opus. A page that says “make it hard” assumes you already know how. A mid-size model doesn’t.

Round two became “Opus plans, Nemotron builds”. Opus broke the job into five steps, each with a test the model had to pass before the next one: the rules, the board maker, the difficulty, the play screen, then the home screen and daily puzzle. Opus didn’t get its own part right first time either. Its idea for the board maker, tried before handing it over, only filled 57 to 86 percent of the board. So step two describes the method from my Arrows game in plain words instead.

That means round two isn’t a fair rematch of round one. The models got a recipe. But that’s the finding: the plan carries a lot of the weight.

Round two

Model Build time Hidden test: levels, dailies Depth, easy to hard Plays in Chrome?
Nemotron 3.5 Super 25 minutes 27 of 30, 6 of 6 15.6 to 39.8 No, Play does nothing
Nemotron 3 Ultra 16 minutes 29 of 30, 5 of 6 13.8 to 32 No, stops after one tap
MiniMax M3.1 Flash 49 minutes 30 of 30, 6 of 6 13 to 31.4 Yes
My version, for comparison 30 of 30, 6 of 6 12 to 32.8 Yes

Super made deeper boards than mine, then never let you see them. One step styled the screens to appear with one label while the code switched them with another, so Play changes nothing. Ultra’s looked exactly like the real game, bent arrows and all, and broke on the second tap. MiniMax was the slowest and the only one that worked from start to finish. Same model I’d called flashy junk two hours earlier.

I played it and spotted something the automatic tester couldn’t. Bent arrows slid off as one stiff piece, where in Heidi’s game they snake out along their own path. The plan had asked for that. I gave the fix back to MiniMax rather than let Claude do it, so the game stays free-model work. One edit later, the arrows follow their corners out.

The fixes that weren’t

Super and Ultra got one go each at fixing theirs, the same way: the tester’s raw report, the goal of clearing level 1, and no hints. Both came back sure they’d done it. Super wrote that tapping every arrow “shows Level complete”. Ultra wrote that its level 1 “now completes successfully”. Each had checked with a pretend version of the page that it wrote itself, not a real browser. In Chrome, Super’s Play button still did nothing and Ultra’s game still broke on the second tap. Both fixed something. Neither fixed the thing that mattered.

You can play it. MiniMax’s game is live for a month, until about 24 November: free-ai-arrows-159-69-195-190.sslip.io/minimax. The stiff version is still at /minimax-rigid if you want to see the difference. I honestly don’t know yet if the later levels are hard. Tell me.

What I learned

The models didn’t get cleverer between rounds. The plan did. A free model handed a vague job flounders. Handed a small job with a test it has to pass, it delivers.

The checking mattered just as much. Opus’s tests for the last two steps checked that the page loaded, not that Play worked, which is how Super’s dead button got through. Every model passed every test it was given, and two of the three games still didn’t work.

It’s free to try, not free forever. NVIDIA says so itself. The key lasts six months. The model list moves: one model I tested was due to be withdrawn four days after I signed up. Don’t build anything you depend on around a free tier.

Don’t send it anything private. I looked for a plain statement of what happens to the questions you send, and didn’t find one on the pages I read. So I treat it like a stranger’s computer. No health details, no money, nothing from work, no other people’s information. That’s a bigger question than one post, and it’s where this series is heading.

Check the work, not the report. Both failed fixes came with a confident note saying they’d worked. Open it, tap it, run it yourself.

A coding agent can read anything you can. This was a single-file game in an empty folder, so I judged the risk too small to bother with a container and ran the agents straight on my Mac. The logs changed my mind. While checking its fix, MiniMax wandered out of its folder and read the other models’ games, my hidden test, my notes and the source code of Heidi’s real game. Another of its runs had opened the hidden test earlier. Pi puts no fence round the folder you start it in. I checked: every look outside came after it had made its changes, so its work is its own. But nothing stopped it. A small experiment still runs on a computer full of things that aren’t small. Every agent run belongs in a container, however trivial the job looks.

Try it yourself

Chat

On a laptop or desktop computer, the easiest way in is your web browser. NVIDIA’s own help pages point you to each model’s playground: you type, it answers, nothing to install. That’s how I’d start, with Nemotron 3.5 Super.

If you’re happy in a terminal, the coding agent below doubles as a chat window on your computer. Start it and just talk to it. It answers questions as readily as it writes code.

On a phone or tablet you can install a chat app and point it at NVIDIA with your key. I did that on a cheap Amazon tablet, and it was a story in itself.

Coding

The game builds all ran through pi, a small open-source coding agent that runs in your terminal. An agent is different from a chat window: it can read your files, write new ones and run commands, then look at what happened and carry on. Pi does that with four tools and a short set of instructions, and it already knows about NVIDIA, so it needs nothing but your key.

These steps are for a Mac. You need Node.js installed first (the LTS download from nodejs.org).

On Windows, pi works too, but install Git for Windows first. Pi runs its commands through Bash, which Windows doesn’t have, and Git for Windows brings it. The rest is in pi’s Windows guide. I haven’t tested the Windows route myself yet.

1. Install pi. In Terminal, from any folder:

npm install -g @earendil-works/pi-coding-agent

Use that name exactly. Older guides say @mariozechner/pi-coding-agent, which still installs but is marked as replaced.

2. Give it your key for this Terminal window. Paste your own key in place of the dots:

export NVIDIA_API_KEY=nvapi-...

A new Terminal window forgets this. You’ll need to run it again each time, or add the line to your shell’s settings file.

3. Make a folder to work in, and start pi in it:

mkdir ~/free-ai-test
cd ~/free-ai-test
pi --provider nvidia --model nvidia/nemotron-3-ultra-550b-a55b

Always start pi in a new, empty folder like this one, never in your home folder, because it can read anything you can. For anything more than a test, run it in a container.

Check it works. Type: Create hello.txt containing the word hello. Mine replied “Created hello.txt with the word “hello”.” in 16 seconds, and the file was really there. Don’t trust the reply on its own. Run cat hello.txt in another window and look.

If you see this, you asked for a model pi doesn’t know about:

Warning: Model "nemotron-3.5-super-vl-120b-a12b" not found for provider "nvidia". Using custom model id.
404 404 page not found

Nemotron 3.5 Super, my favourite from the table, is one of those. Pi’s built-in list covers Ultra, Lightning, Laguna, gpt-oss and about 15 others. Run pi --list-models nvidia to see them. You can add Super with a small settings file, but Ultra works without one, so start there.

Next week: the same free models on a £100 Amazon tablet, tested on something my wife actually uses AI for.

Powered by CookieAdmin