Story
Study: Self-generated Agent Skills are useless
1 report
1 website
1 discussion
Latest report
Overview
SkillsBench is a benchmarking framework designed to evaluate the effectiveness of agent skills across 86 tasks in 11 domains. It includes curated skills and deterministic verifiers to assess their impact on large language model (LLM) agents during inference.
Summary from arxiv.org.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · arxiv.org
Hacker News discussion ↗
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-02-17.
- Praise: The finding that curated skills provide a significant positive benefit (+16.2pp) suggests that LLMs excel at utilizing existing procedural knowledge rather than generating new skills.
- Praise: Using LLMs to distill information from research can enhance skill creation, leading to more effective and relevant outcomes tailored to specific workflows.
- Concern: Self-generated skills have been shown to provide a negative benefit (-1.3pp), indicating that LLMs struggle to produce useful procedural knowledge independently.
- Concern: The observation that layering LLM outputs leads to diminishing returns highlights a critical limitation in the current approach to automating tasks with LLMs.