Compaction in Pi occurs during long coding sessions to manage memory and context. Large language models (LLMs) have limited context windows due to the constraints of their transformer architecture, affecting the amount of input they can process.