Attention Residuals (AttnRes) serves as a drop-in replacement for standard residual connections in Transformers, allowing each layer to selectively aggregate earlier representations. It includes two variants: Full AttnRes, where each layer attends over all previous outputs, and Block AttnRes, which groups layers into blocks to reduce memory usage from O(Ld) to O(Nd).

Machine learning employs statistical techniques to automatically identify patterns in data, enabling accurate predictions. A model can be created using a dataset about homes to differentiate between homes in New York and those in San Francisco.

Billion-parameter theories aim to explain complex phenomena in the universe using concise mathematical formulations. Historical explanations of natural events transitioned from mystical interpretations to scientific inquiry with succinct equations like F=ma and E=mc².

Writing agents and tools for AI systems enables enhanced problem-solving and architecture decisions. The integration of AI allows for more efficient workflows, with AI handling heavy lifting while the user focuses on strategic thinking.

Speculative decoding accelerates autoregressive inference by using a fast draft model to predict upcoming tokens from a slower target model. It verifies predictions in parallel with a single forward pass of the target model, addressing the sequential dependency bottleneck.

Holly Herndon has developed an AI voice clone that allows users to create music using her custom models. Her journey into machine learning began in 2015, evolving from initial "scratchy" outputs to sophisticated tools for musical expression.

Decision Trees create sequential rules that split data into distinct regions for classification. Entropy is used to measure information and identify regions with significant data separation.

The course "10-202: Introduction to Modern AI" covers the workings of modern AI systems, focusing on machine learning methods and large language models (LLMs) such as ChatGPT, Gemini, and Claude. The curriculum emphasizes the contemporary understanding of AI, primarily relating to chatbot technologies used daily.

A minimal transformer model has been developed to perform 10-digit addition tasks. The model demonstrates the ability to learn and execute arithmetic operations effectively.

Data from December 2022 to December 2025 shows a steady increase in submissions, with numbers rising from 800 in 2022 to 855 in 2025. From January 1 to February 15, 2026, submissions reached 617, indicating a year-over-year growth trend.