AI optimization
Source reports covering AI optimization. Stories will appear when analysis and community reactions are ready.
7 reports · page 1 of 1
Connected topics are temporarily unavailable. You can still browse stories below.
Source reports
Why harnesses matter for energy (and token) efficiency
research
15 min read
Neuralwatt found that the software harnesses used by AI agents can materially affect GPU energy consumption by determining whether long prompts reuse a model server’s key-value cache. Across customer traffic from September 8 to October 7, 2026, major clients had 1.6% to 11.1% uncached input. A session with 92% cached input requires about 2.7 times as much prompt-processing work as one with 97% cached input, despite both using the same model and conversation. For comparable 50,000-token requests measured October 1–7, mostly uncached prompts used roughly twice the energy of fully cached prompts; GLM-5.3 used about 2.5 times as much. In sessions exceeding 128,000 tokens, about three turns per 1,000 re-read more than half the prompt, and those turns represented 75% of re-read tokens at that depth. Each re-read about 230,000 tokens on average, roughly the prefill work of 600 normal turns. Neuralwatt’s lab tests attributed all observed cache breaks to harnesses editing earlier prompt content. Changes to reasoning effort, tool lists, dates, skills, reminders and compaction can force servers to recompute prompts from the first altered token. Neuralwatt recommends append-only prompt histories, stable system prompts and tool lists, and settling settings before sessions become long.
NanoGPT Speedrun Frontier
research
4 min read
Prime Intellect ran 153 autonomous nanoGPT optimizer speedruns across 18 frontier-model entries and reported the best validated result for each. Fable 5, using Claude Code at the high setting, ranked first with a validated record of 2,726 and an 81.7% gap closed toward the stated human record of 2,600 from a baseline of 3,290. Its run used 3,010 steps and lasted 8.7 days. Opus 5 placed second at 2,920, closing 53.6% of the gap, followed by Kimi K3 at 2,930 with Prime Agent and 2,974 with Kimi Code. Opus 4.8 recorded 3,018, while GPT-5.6 Sol and GPT-5.6 Sol Pro reached 3,042 and 3,058. Sonnet 5 reached 3,105, and GPT-5.6 Luna reached 3,110. The remaining validated records ranged from 3,120 for Grok 4.5 and Qwen3.8 Max to 3,240 for Kimi K2.7. GLM 5.3 had no recorded result. The results use different agent harnesses and configurations, including Claude Code, Codex, Prime Agent, Kimi Code, Grok CLI, Qwen Code, Pi, and Muse Code. Prime Intellect also provides 41 curated full agent trajectories containing tool calls, subagents, and scratchpads.
Fixed three bugs that made Qwen3.5-122B a daily driver on Mac Studio
tool
12 min read
A follow-up question on a 50,000 token conversation with an AI can take three to five minutes before the first token is generated. A three-week effort was dedicated to debugging a cache leak in order to optimize performance on the Mac Studio.
Mag 7 starting to underperform [pdf]
research
1 min read
Mag 7 is experiencing a decline in performance metrics. Data indicates a need for strategic adjustments to improve outcomes.
AI OSS tool repo goes archived over night after raising $7.3M Seed
tool
7 min read
TensorZero is an open-source LLMOps platform that provides a unified API for accessing various LLM providers with less than 1ms p99 latency. It includes features for observability, evaluation, optimization, and experimentation, allowing users to store inferences, benchmark workflows, optimize prompts, and manage experiments programmatically.
AI agent opens a PR write a blogpost to shames the maintainer who closes it
tool
5 min read
The pull request replaces `np.column_stack` with `np.vstack().T` in the Matplotlib codebase. This change aims to improve performance and efficiency in array stacking operations.
FlashAttention-T: Towards Tensorized Attention
6 min read
FlashAttention-T introduces a fully tensorized attention mechanism that leverages tensor-vector parallelism to enhance performance. This innovation aims to improve the efficiency of attention-based models in various applications.