A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis

Commenters mostly treated the result as an overdue reality check, not a surprise: a compact vision model trained directly on tens of thousands of dermoscopic images should beat a general-purpose language model on an eleven-class image task. The dominant view is that narrow medical classifiers are the right tool when accuracy on a known metric matters, while LLMs are better deployed upstream to help non-specialists design, train, and integrate those classifiers. Several readers argued that the current AI boom has encouraged an everything-box mentality in which plausible LLM answers are mistaken for task-specific competence. The liveliest dispute concerned whether the CNN result merely games a favorable test. Most rejected that framing: held-out, in-domain evaluation and fine-tuning are standard classifier development, and a skin-cancer system need not generalize to unrelated images to be useful. Skeptics still wanted external-dataset testing before trusting claims of robustness. Readers also pushed back on treating the old CNN as the clinical endpoint. Modern ensembles of vision transformers may perform better, but larger models are harder to run locally and every current system can miss cancers, making unsupervised consumer deployment dangerous. A smaller group argued that multimodal systems could recover ground by combining images with notes, patient questions, and messy real-world context, rather than asking a bare LLM to classify a lesion from pixels alone.