ai-agents human-in-the-loop code-generation ai-safety

Humans missed 1 in 3 threats approving AI agent commands across 40k game runs

Humans missed 1 in 3 threats approving AI agent commands across 40,000 plays

scalex.dev

August 6, 2026

6 min read

🔥🔥🔥🔥🔥

63/100

Summary

Humans missed 1 in 3 threats while approving AI agent commands in a browser game involving 40,000 plays. The game simulated a human-in-the-loop scenario where players approved or denied commands under time pressure, revealing vulnerabilities in command approval processes.

Key Takeaways

Players missed 1 in 3 threats when approving AI agent commands, achieving an average accuracy of 66.3%.
32.9% of game sessions resulted in a negative score due to penalties from approved threats and blocked safe commands.
The command "npm run analyze" was the most frequently missed threat, approved 64.7% of the time by players.
Commands categorized as exfiltration or code execution had a miss rate of 33.4%, indicating a significant risk of undetected threats.

Read original article

View story →

Coverage & discussions

Hacker News · scalex.dev

Humans missed 1 in 3 threats approving AI agent commands across 40k game runs ↗

Community discussions

Hacker News discussion ↗

Historical count: 5 · 5 saved in sample · Historical sample

Analysis ready

Community Sentiment

Mixed

Positives

The stats from the AI agent game reveal fascinating insights about human decision-making under pressure — 40k runs and still 1 in 3 threats missed is eye-opening.
The AI agent's sandboxing approach is an innovative step, eliminating permission prompts and allowing for more autonomous operations in a controlled environment.

Concerns

Relying on users to constantly approve commands is a laughable security model — history shows it just doesn't work over time.
The inconsistency in what prompts are considered dangerous makes the entire testing process fundamentally flawed and the results meaningless.
The permission prompts are a terrible model that only leads to fatigue and mistakes; we need better methods for AI decision-making.