CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs
arxiv.org
May 22, 2026
2 min read
🔥🔥🔥🔥🔥
50/100
Summary
Computer Science > Machine Learning [Submitted on 19 May 2026 (v1), last revised 20 May 2026 (this version, v2)] Title:CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs View PDF HTML (experimental)Abstract:Transformer training systems are built around dense linear algebra, yet a nontrivial fraction of end-to-end time is spent on surrounding memory-bound operators. Normalization, activat...