Andrej Karpathy developed a 200-line Python script that trains and runs a GPT model from scratch without any libraries or dependencies. The model is trained on a dataset of 32,000 human names, with each name listed on a separate line.

microgpt is a single-file Python script consisting of 200 lines that implements a GPT-like model with no dependencies. It includes components such as a dataset, tokenizer, autograd engine, neural network architecture, Adam optimizer, and both training and inference loops.