machine-learning ↗

Replacing Jev With an Open Model in Super Mario Bros. Took Four Tries

I swapped the Jev API for Laya, an open 421M decision model, in my Mario controller. Out of the box it died at the first Goomba. After a shorter prompt, two fine-tunes and a fallback, it cleared World 1-1 with 7 Jev calls instead of 208.

Replacing Jev With an Open Model in Super Mario Bros. Took Four Tries

My Mario controller asks Jev what to do roughly every 8 frames. Each answer is an API call that takes about 600 ms, and a clear of World 1-1 takes 208 of them.

In the last post I got rid of those calls twice: once by copying Jev into a small random forest, and once by training a PPO agent from pixels. Both work, but both are narrow. The forest only knows Mario's state as numbers, and the PPO agent only knows pixels.

Then I found Laya. It is an open-weights model (Apache 2.0) built for the same job as Jev: give it a state and typed questions, and it returns an answer with probabilities in one forward pass. Its model card compares itself directly with Jev, and it ships a server that speaks Jev's API.

That looked like a drop-in replacement. It wasn't, at first. Here is every experiment, including the ones that failed. I worked through them with Claude Code.

Laya in one paragraph

Laya is a 421M-parameter encoder (ModernBERT-large plus a small decision head). It doesn't generate text. Each answer option gets a marker token, and the model scores all options at once. The model card claims about 33 ms for one question on a GPU, and it is candid about one limit: the base checkpoints are "near chance zero-shot" on new decision types and need fine-tuning to be useful.

I didn't take that seriously enough at first.

Try 1: point the controller at Laya

Because Laya's server accepts Jev's request format, the controller needed no code changes. I pointed its base URL at the local server and ran it with the exact same prompt, state and seven controller actions.

I scored it two ways: by replaying 200 of Jev's logged decisions to see how often Laya picked the same action, and by letting it play.

Checkpoint Agrees with Jev Game
English 13% Died at x 1,935
Multilingual 25% Ended at x 171

The flag is at x ≈ 3,161. Each decision took about 210 ms, not 33.

The cause showed up in the server's usage report:

state_tokens: 680   state_tokens_dropped: 364   truncated: true
state_tokens: 897   state_tokens_dropped: 581   truncated: true

The English checkpoint reads 512 tokens. Jev's prompt, written for Jev, has three questions with long instructions plus a large JSON state. Laya was dropping 40–65% of the game state on every call, including the enemy and terrain fields it needed most.

Try 2: a prompt that fits

I wrote a separate Laya policy with a compact state: only the fields a decision depends on (Mario's position, speed and jump phase, the nearest enemy and time to contact, the next obstacle or gap), and a single one-line question.

To test it fairly, I held out Jev's two most recent runs (624 decisions) and never trained on them.

Prompt Agrees with Jev Per decision
Jev's prompt 13% ~210 ms
Compact 70% 72 ms
Always right_jump 85% –

Much better, and three times faster. But still worse than a model that presses right_jump every time, because Jev presses it 85% of the time. In the game it reached x 1,216 before dying.

That's the zero-shot ceiling the model card warned about. Next step: teach it.

Try 3: fine-tune on Jev's decisions

Every Jev decision was logged with the state it saw and the probability it gave each action. That's a training set: 1,503 decisions from the older runs. I used Laya's own fine-tuning script unchanged and fed it my items, with Jev's full probabilities as soft targets rather than just its top choice.

Four epochs took about 40 minutes. The result looked great:

Model Agrees with Jev
Base, compact prompt 70.2%
Fine-tuned 93.8%

Then it played, and died at x 309. The first Goomba.

Here's what it saw on its last few decisions:

x 261  goomba 69 px ahead, contact in 23 frames    right_run 0.71   right_jump 0.15
x 285  goomba 40 px ahead, MUST JUMP NOW           right_run 0.69   right_jump 0.15
x 309  goomba 11 px ahead, contact in 4 frames     right_run 0.70   right_jump 0.15

The probabilities barely moved. At x 285 the state said the jump had to start on this decision, and it still chose to run. In Jev's logged runs, it pressed right_run_jump at exactly that point.

Why 93.8% was a lie

Most of Jev's decisions are easy: keep holding jump while in the air, keep running on open ground. The decisions that matter, taking off from the ground at the right moment, are rare. Only 117 of the 1,503 training rows (7.8%) were takeoffs.

So I added a second metric: of the 39 held-out moments where Jev jumped from the ground, how often did Laya also jump?

Model Agrees Takeoffs
Base, compact 70.2% 33/39
Fine-tuned 93.8% 15/39

Fine-tuning made it better at the easy decisions and worse at the one that keeps Mario alive. Overall accuracy hid it completely.

Try 4: make the danger readable, and make it count

Two changes.

A plain-language summary. Laya is a language model. A true buried among two dozen JSON fields is hard for it to notice; a sentence isn't. Every state now starts with one line built from the same fields:

on the ground; goomba 40 px ahead, contact in 13 frames; MUST JUMP NOW
in the air, rising; gap 7 tiles wide, 2 tiles ahead; crossing a gap, keep moving right

Weight the takeoffs. Each ground takeoff appears 8 times in the training data, so the model can't score well by ignoring them. That made 2,322 training items; I trained for two epochs.

Model Agrees Takeoffs
Fine-tuned v1 93.8% 15/39
Fine-tuned v2 99.5% 39/39

It cleared the Goombas and the pipes on Jev's route. Then it died at x 1,134, the first pit.

Dying at the pit

This time the decision wasn't wrong. It was the right decision from the wrong place.

In Jev's logged runs, Mario lands from the previous jump at x 1,048 and take off again from there. Laya's earlier jump came down a little sooner, at about x 1,020. It did what Jev does on landing, jump again, and came up short.

This is the standard weakness of copying a teacher. The student only learns the states the teacher visited. Drift a few pixels off that path and it's somewhere the teacher never showed it, and small errors compound.

The fix that worked: ask Jev when unsure

The forest from the last post has a safety net: when it isn't confident, it asks Jev. I gave Laya the same one. If Laya's confidence is below 0.5 and Mario is on the ground (a takeoff decision), Jev decides that move.

The first attempt still died at the pit, with 8 Jev calls. Allowing Jev to decide in the air as well raised it to 32 calls, and it still died at the same pit.

That was suspicious, so I diffed this run against one of Jev's clears. They were identical until decision 19, at x 373. Same position, same state. In the clearing run Jev chose right_run. Asked from the Laya run, Jev chose jump with 0.13 confidence, then noop, and the run drifted from there.

The difference wasn't in the logs. Jev's clears came after I added experience memory: a summary of earlier attempts in the same situation, sent along with each request. The logs record the game state but not that memory. My fallback was calling Jev without it.

With the fallback Jev given a copy of its memory, and every move's outcome (Laya's included) fed back into it:

Jev alone Laya + Jev
Decisions 208 208
Jev API calls 208 7
Laya decisions 0 201
Result Clear Clear

It reached the flag twice in a row, the same way both times. Jev was called at x 331, 373, 1,022, 1,432, 1,565, 2,691 and 3,143. The call at x 1,022 is the takeoff before the pit that killed every earlier version. Laya's own decisions took about 66 ms each.

What I took from this

"Drop-in compatible" covers the API, not the prompt. Laya accepts Jev's requests, but a prompt written for Jev silently overflows a 512-token encoder. Check truncation before judging any model.

Read the model card's limits. "Near chance zero-shot" was right there. The model was a starting point to fine-tune, and I treated it as a finished product.

Overall accuracy can hide the decision that matters. 93.8% looked like success and died at the first enemy. Measure the rare, high-stakes cases separately.

Write the important signal so the model can read it. A one-line summary did more than a careful JSON schema.

Log everything the teacher saw. The hardest bug was a memory the logs didn't record. Jev with memory and Jev without it are different teachers.

Where the three local approaches stand

Forest PPO Laya
Learns from Jev's logs Reward Jev's logs
Input Numbers Pixels Text
Per move <10 ms – ~66 ms
Jev calls 0 0 7

"Forest" is the random-forest copy of Jev. I didn't time the PPO agent's moves, and Laya's 7 Jev calls are per clear.

On this one level, the random forest is still the cheapest. Laya's advantage is that it reads text, so the same model could take new instructions without new features.

The next step is DAgger: let Laya play, have Jev label the states Laya actually reaches, retrain, and repeat until it no longer needs the 7 calls.