Story
Are AI Labs Pelicanmaxxing?
1 report
1 website
1 discussion
Latest report
Overview
Simon Willison has tested major LLM releases using the prompt “Generate an SVG of a pelican riding a bicycle,” creating an informal benchmark in AI. The results often receive significant engagement on platforms like Hacker News, sparking discussions about the benchmark's usefulness.
Summary from dylancastillo.co.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
dylancastillo.co
Are AI Labs Pelicanmaxxing?
Read report →
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · dylancastillo.co
Hacker News discussion ↗
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-07-23.
- Praise: Dylan's methodology here is significantly more robust than anything I was considering, showcasing a solid approach to evaluating model performance.
- Praise: I'm glad someone ran the numbers on this; it's a refreshing take on quantitative analysis of AI outputs that often get dismissed.
- Praise: The outputs of Qwen3.7 Plus have to be seen to be believed; there's clearly impressive capability on display here.
- Concern: Using a LLM to generate complex SVGs is pretty niche; this makes me question the general applicability of the benchmark.
- Concern: There's a lack of evidence that these models have trained on specific combinations like pelicans on bicycles, which raises doubts about their generalization capabilities.
- Concern: The argument that AI labs could be sucking up all the training material without real improvement is disheartening and suggests stagnation.