u/AgeOfEmpires4AOE4

I trained an AI to speedrun Super Mario Bros using Reinforcement Learning — after more than 6 million deaths 😅

The agent starts completely clueless:

• running into the first Goomba
• falling into pits
• getting stuck against pipes

Over time, it slowly learns:

• movement timing
• enemy avoidance
• jump precision
• speed optimization

What’s interesting is that some “speedrunner-like” behaviors emerged naturally during training:

• maintaining momentum
• minimizing hesitation
• optimizing jump timing

The training was done using a custom RL setup with frame stacking and temporal modeling. Watching the progression from random movement to competent gameplay was honestly one of the coolest parts of the project.

I’d love feedback from people into:

• RL
• game AI
• imitation learning
• emergent behavior

Hey everyone,

I’ve been experimenting with Behavior Cloning on a classic arcade game (Final Fight), and I wanted to share the results and get some feedback from the community.

The setup is fairly simple: I trained an agent purely from demonstrations (no reward shaping initially), then evaluated how far it could go in the first stage. I also plan to extend this with GAIL + PPO to see how much performance improves beyond imitation.

A couple of interesting challenges came up:

• Action space remapping (MultiBinary → emulator input)
• Trajectory alignment issues (obs/action offset bugs 😅)
• LSTM policy behaving differently under evaluation vs manual rollout
• Managing rollouts efficiently without loading everything into memory

The agent can already make some progress, but still struggles with consistency and survival.

I’d love to hear thoughts on:

• Improving BC performance with limited trajectories
• Best practices for transitioning BC → PPO
• Handling partial observability in these environments

Here’s the code if you want to see the full process and results:
notebooks-rl/final_fight at main · paulo101977/notebooks-rl

Any feedback is very welcome!