We gave terabytes of CI logs to an LLM
mendral.com
February 27, 2026
11 min read
Summary
An AI agent analyzed terabytes of continuous integration (CI) logs by generating SQL queries, tracing a flaky test back to a dependency change within seconds. The agent requires comprehensive context from all historical logs, processing approximately 1.5 billion log lines and 700,000 jobs weekly.
Key Takeaways
- The agent can trace flaky tests by writing SQL queries that scan hundreds of millions of log lines, completing investigations in seconds.
- The system processes approximately 1.5 billion CI log lines and 700,000 jobs weekly, storing data in ClickHouse with a compression ratio of 35:1.
- The agent utilizes a SQL interface to construct its own queries, allowing for flexible and novel questions during debugging.
- The agent typically scans 335,000 rows across about 3 queries per investigation, with some sessions scanning up to 4.3 billion rows.
Community Sentiment
MixedPositives
- Developing a tool that reduces log noise using a TF-IDF classifier and BERT model demonstrates innovative approaches to enhance LLM performance in log analysis.
- The integration of LLMs with SQL for exploratory data analysis shows potential for improving data insights, despite the need for human oversight.
Concerns
- LLMs often hallucinate reasons during log analysis, which can lead to misunderstandings in complex scenarios where multiple failures occur.
- The effectiveness of LLMs in generating SQL queries is questioned, as they may require substantial human input for critical business decisions.
Source
mendral.com
Published
February 27, 2026
Reading Time
11 minutes
Relevance Score
59/100
🔥🔥🔥🔥🔥
Why It Matters
This page is optimized for focused reading: quick context up top, a clean summary block, and a direct path to the original source when you want the full story.