Train Your Own LLM from Scratch
github.com
May 5, 2026
4 min read
🔥🔥🔥🔥🔥
67/100
Summary
GitHub repository angelos-p/llm-from-scratch offers a hands-on workshop for building a GPT training pipeline from scratch. The workshop focuses on reproducing the GPT-2 model with 124 million parameters using PyTorch.
Key Takeaways
- The workshop allows participants to build a GPT training pipeline from scratch, covering essential components like tokenization, model architecture, and training loops.
- The project targets a ~10M parameter model that can be trained on a laptop in under an hour, using either Apple Silicon GPU, NVIDIA GPU, or CPU.
- Participants will write code for various parts of the model, including a character-level tokenizer, transformer architecture, and text generation methods.
- The workshop is designed for individuals comfortable with Python, without requiring prior machine learning experience.
Related Articles
Microgpt explained interactively
Mar 1, 2026
Autoresearch: Agents researching on single-GPU nanochat training automatically
Mar 7, 2026
CS336: Language Modeling from Scratch
Jun 1, 2026
Mr. Chatterbox is a Victorian-era ethically trained model
Mar 31, 2026
Nano-vLLM: How a vLLM-style inference engine works
Feb 2, 2026