jepa video-game-ai reinforcement-learning world-modeling

LeMario: Training a JEPA World Model on Super Mario Bros

LeMario: Super Mario Bros trained on a JEPA Model

benjamin-bai.com

July 14, 2026

11 min read

🔥🔥🔥🔥🔥

53/100

Summary

LeMario is a Joint-Embedding Predictive Architecture (JEPA) model trained on Super Mario Bros to learn world dynamics from pixels and actions. The model successfully passed all initial tests.

Key Takeaways

The LeMario model is based on a Joint-Embedding Predictive Architecture (JEPA) that learns world dynamics from pixels and actions in Super Mario Bros.
The model successfully generalized to held-out episodes and predicted five-step futures better than strong baselines, but struggled with navigating toward distant goals.
The architecture includes a vision encoder and an action encoder that compresses visual frames and button states into latent representations for prediction.
The causal predictor uses six transformer blocks and incorporates Adaptive LayerNorm Zero (AdaLN-Zero) to modulate the influence of actions on frame predictions.

Read original article

View story →

Coverage & discussions

Hacker News · benjamin-bai.com

LeMario: Training a JEPA World Model on Super Mario Bros ↗

Community discussions

Hacker News discussion ↗

Historical count: 5 · 5 saved in sample · Historical sample

Analysis ready

Community Sentiment

Mixed

Positives

This paper shines a light on JEPA's potential, especially how it can predict horizontal position effectively after minimal training — a promising step for model-based control.
The insights shared by the author about the training process are invaluable, showcasing a real-world application of AI concepts in gaming.

Concerns

Planning in latent space can be noisy and unrepresentative of actual game positions, which raises concerns about control and optimization in dynamic environments.
Chunking planning into intermediate goals feels limiting, undermining the essence of model-based control where flexibility is key.