Story
MTG Bench: Testing how well LLMs can play Magic
1 report
1 website
1 discussion
Latest report
Overview
Fable 5 successfully plays a scry land and examines the top card of the deck, while Gemini 3.5 performs a complex turn involving scry, discover, and tutor effects. The benchmark tests the capability of LLMs to play Magic: The Gathering without relying on a rules engine, suggesting that high-performing LLMs should not require one.
Summary from mtgautodeck.com.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
mtgautodeck.com
MTG Bench: Testing how well LLMs can play Magic
Read report →
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · mtgautodeck.com
Hacker News discussion ↗
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-06-12.
- Praise: This benchmark is timely and addresses the shortcomings of existing benchmarks, highlighting the complexity of Magic's mechanics and edge cases.
- Praise: Testing LLMs with maximum thinking time and web search capabilities showed they could follow rules better than average players, suggesting potential for advanced AI in gaming.
- Praise: The exploration of letting LLMs play Magic like humans could lead to more engaging gameplay experiences, emphasizing natural interaction over rigid rule enforcement.
- Concern: Concerns were raised about the reliability of LLM grading, suggesting that using a rules engine might yield more consistent results than relying on LLMs alone.
- Concern: The scoring method based on a simple prompt may not adequately capture the quality of the simulation, indicating potential biases in evaluation.