llms tabular-data predictive-analytics machine-learning

Why Large Language Models Fail at Tabular Prediction

arxiv.org

August 4, 2026

2 min read

🔥🔥🔥🔥🔥

52/100

Summary

Large language models (LLMs) struggle with predictive analytics over tabular data, which is a common machine learning workload. This limitation has led to the emergence of tabular foundation models aimed at addressing this gap.

Key Takeaways

Large language models (LLMs) struggle with predictive analytics over tabular data, failing to achieve success in this common machine learning workload.
Dimensionality of input data significantly impacts LLM performance; accuracy decreases as dimensionality increases, unlike classical models which maintain or improve accuracy.
Controlled experiments indicate that LLMs cannot effectively handle noisy or non-linearly-separable data, and issues related to tokenization and data format do not explain their poor performance.
In two dimensions, LLMs predict similarly to local, distance-based methods, but in higher dimensions, no classical model can replicate their predictions.

Read original article

View story →

Coverage & discussions

Hacker News · arxiv.org

Why Large Language Models Fail at Tabular Prediction ↗

Community discussions

Hacker News discussion ↗

Historical count: 5 · 5 saved in sample · Historical sample

Analysis ready

Community Sentiment

Negative

Positives

The paper tackles a critical issue in AI, addressing why LLMs struggle with tabular data, which is vital for improving future models.
Some commenters highlight that tree boosting methods are still the reigning champs for tabular tasks, showing the need for LLMs to evolve.

Concerns

Many see this research as niche and irrelevant to real-world applications, questioning its practical value.
The consensus is that LLMs are fundamentally outclassed by older algorithms like tree boosting for tabular predictions, raising doubts about their versatility.