debugging ↗

Jev Kept Dying in Mario. The Controller Was Part of the Problem.

I connected Jev to Super Mario Bros. and watched it repeat the same mistakes. Fixing timing, jump inputs, and experience memory eventually produced seven recorded World 1-1 clears.

A robot plays a fantasy video game at a desk while a man relaxes on a sofa with a sleeping golden retriever.

I wanted Jev to play Super Mario Bros. I got a window with Mario moving, model probabilities updating, and an agent that kept dying in almost the same way.

Restart. Run right. Die. Repeat.

My first question was why it couldn't play correctly. My next question was why it wasn't learning from those attempts.

Working through the logs with Codex revealed several different problems. Decisions arrived too late. A jump release happened at the wrong moment. Restarting discarded the controller's recent history. The original objective also gave the agent little reason to care about collecting coins.

After changes to the controller and the addition of persistent experience, the later logs recorded seven World 1-1 clears. Each collected one coin and registered two enemy defeats. That was useful progress, with clear limits on what it proved.

Giving Jev a controller

I started with fhshaik's TypeSafe Mario project. Its existing emulator integration and dashboard provided the foundation; I worked with Codex on the local changes described here.

The controller reads game state from the emulator and builds structured observations. Jev chooses among a small set of inputs such as moving right, running, jumping, or combining movement with a jump. The game advances, and the next observation goes back to the model.

The loop looks like this:

Emulator state → structured observation → Jev decision → controller input

I used an OpenRouter key. The TypeSafe SDK supports that by setting its base URL to https://openrouter.ai/api; the SDK adds /v1/systemone. My launcher selected jev-1.13. OpenRouter documents this integration.

At that point, successful API requests looked encouraging. But receiving valid decisions was only the beginning.

Mario could die while Jev was deciding

The first run log contained seven attempts and 28 decisions. Median response latency across those decisions was about 595 milliseconds.

Meanwhile, the dashboard loop kept stepping the emulator with the previous input.

At 60 frames per second, 595 milliseconds is roughly 36 game frames. The intended action duration was eight frames. An observation could become badly outdated before the controller acted on the answer.

One terminal decision in that first log looked like this, with unrelated fields removed:

{
  "decision": 3,
  "action": "right_jump",
  "latency_ms": 575.1600330113433,
  "terminated": true
}

The jump answer arrived after the episode had already ended. Across those seven attempts, the last observations sent for decisions were around horizontal positions 210–246.

The fix was to pause emulation while a decision was pending, then apply the chosen action for eight frames. The dashboard continued handling redraws and keyboard input during the wait.

This made play visibly slower. It also made the observation and the action refer to the same game situation.

The state parser needed another correction: it had to update once per emulated frame. Updating it only once per decision made an eight-frame displacement look like a one-frame velocity. Redrawing a paused window also shouldn't increase an airborne timer or a stall counter.

An offline emulator check used the same input sequence with fast and delayed simulated decisions. After the timing change, both produced identical game states and rewards.

The next live log got much farther: two attempts reached roughly x=1400. Both still died.

Holding Jump was hiding another bug

The repeated gap failure initially looked like a bad model choice. Jev kept selecting right_jump, yet Mario fell.

A frame-by-frame replay exposed the input problem. Mario landed partway through an eight-frame action. The controller waited until the next decision boundary to release Jump and press it again. By then, Mario was walking off the edge.

The controller needed to handle the button transition when Mario landed, even if the current action still had frames remaining.

That change needed care. Testing showed a one-frame delay before the emulator reflected a newly pressed Jump button. Releasing it immediately could cut the jump short. The fix accounted for that delay and used the RAM motion state to avoid treating Mario as grounded when he had already started falling.

A controlled replay reproduced the old death around x=1411. Applying the landing fix to the same approach got Mario safely beyond that gap, reaching x=1449 on supported ground.

That was evidence for a specific controller fix. It didn't establish that the model had learned the level.

Making attempts leave something behind

Restarting the original setup returned Mario to the start and reset the parser. The next request described the current situation, without a useful record of the previous death.

To make attempts influence later decisions, we added local SQLite experience memory.

For similar situations, it records the chosen action, progress, and what happened afterward. Matching includes position, movement, grounded state, nearby hazards, and action duration. The collector objective also includes nearby collectible targets.

Jev receives a compact summary of that experience along with the current observation. A nearby failed approach can therefore appear in a later request before Mario reaches the same obstacle.

There is also a small exploration rule: after an action has been followed by death in two matching grounded situations, with no observed survival window, the controller can remove it from the next set of choices. Jev then chooses among alternatives. This rule is restricted to grounded situations to avoid interrupting an airborne jump merely to try something different.

Outcomes are associated with the preceding 16 decisions. Reward contributions are discounted by 0.9 for each earlier decision. These are initial design choices. A death after a sequence of actions doesn't prove which action caused it.

Manual restarts and decision limits aren't labeled as deaths. Memory survives pressing R and closing the program.

This is learning in the surrounding controller: stored experience changes future observations and available choices. We did not update Jev's model weights.

Teaching the controller what I wanted

I also wanted Mario to collect coins, defeat enemies, and avoid falling. That needed both an explicit objective and measurable events.

We added a collector preset with these starting weights:

Event Reward
Collect a coin +25
Observe a supported enemy defeat +100
Reach a new furthest position +0.02 per pixel
Complete the level +1,500
Lose a powerup −200
Die −2,000
Fall to death Another −1,000
Remain stalled on the ground −1 per decision

The progress reward applies only to new ground, so walking back and forth doesn't generate repeated progress credit. The death and fall penalties make survival a strong priority.

Counting enemy defeats required extra work. The emulator didn't expose a ready-made kill counter. An enemy disappearing could simply mean it had left the screen, and an increase in score could come from something else.

The detector follows supported defeat-state transitions in RAM, using the Super Mario Bros. disassembly as a reference. It counts a Goomba stomp once. A Koopa entering its shell is still a hazard and isn't counted as a permanent defeat. The detector doesn't cover every boss or special-enemy mechanic.

We also exposed nearby coin and block targets to Jev and made sure coin tiles weren't treated as solid ground.

Offline checks against actual emulator play detected a coin pickup and assigned +25, then detected a Goomba defeat and assigned +100. The local test suite reached 31 passing tests, including persistence, reward accounting, coin-counter rollover, and avoiding false defeat rewards for disappearing enemies.

The dashboard now shows collected coins, observed defeats, and learning reward. Collector experience is kept separate from the earlier progress-only objective.

What the later runs actually achieved

The collector's recorded results included one death followed by seven completed World 1-1 attempts. The successful attempts shared these metrics:

Measurement Recorded result per clear
Controller decisions 208
Final horizontal progress 3,161
Coins collected 1
Observed enemy defeats 2
Remaining game timer 318
Recorded fall deaths 0

The event logs and the SQLite episode records agreed on the clears and achievement totals.

This was a repeatable completion result for this setup on World 1-1. It was still far from my goal of collecting everything and defeating every enemy.

The results also don't isolate the benefit of experience memory. We changed timing, button handling, observations, prompts, and rewards during the experiment. A comparison with memory enabled and disabled under otherwise identical conditions is still needed before assigning the improvement to learning.

Running another batch

These commands refer to my locally modified checkout and launcher, including the experience and collector additions:

# Five attempts, at most 300 decisions each. Uses OpenRouter credits.
./play.sh --display none --episodes 5 --max-decisions 300

# Watch an attempt using the saved collector experience.
./play.sh

# Inspect saved results without calling the model.
.venv/bin/typesafe-mario memory --preset collector

For a baseline, ./play.sh --no-learning disables reading and writing experience. It keeps the same collector objective. Starting another experiment with --memory artifacts/new-experiment.sqlite3 leaves the existing database intact.

The most useful debugging habit from this project was following one decision all the way through: the state the model saw, how many game frames elapsed, the input actually applied, and the resulting state.

Before changing the model or adding another reward, I want those four things to agree. Otherwise, the controller can keep presenting the same mistake as another opportunity to learn.

Super Mario played by Jev

Follow-up: I later replaced Jev with a network that learned to clear World 1-1 from screen pixels. Read how the PPO training works.