Train Your Own LLM from Scratch

github.com

May 5, 2026

4 min read

🔥🔥🔥🔥🔥

67/100

Summary

GitHub repository angelos-p/llm-from-scratch offers a hands-on workshop for building a GPT training pipeline from scratch. The workshop focuses on reproducing the GPT-2 model with 124 million parameters using PyTorch.

Key Takeaways

The workshop allows participants to build a GPT training pipeline from scratch, covering essential components like tokenization, model architecture, and training loops.
The project targets a ~10M parameter model that can be trained on a laptop in under an hour, using either Apple Silicon GPU, NVIDIA GPU, or CPU.
Participants will write code for various parts of the model, including a character-level tokenizer, transformer architecture, and text generation methods.
The workshop is designed for individuals comfortable with Python, without requiring prior machine learning experience.

Read original article

View story →