gpt-2
Latest stories covering gpt-2, with summaries and links to related coverage.
4 articles · page 1 of 1
Connected topics
Be skeptical of OpenAI's rogue hacker agent story
opinion
3 min read
OpenAI announced GPT-2 on February 14, 2019, but withheld its release due to concerns about safety and potential abuse. The decision was met with criticism from some researchers who felt it limited opportunities for study and understanding of the model.
GPT-2: Too Dangerous To Release (2019)
research
3 min read
GPT-2 is a scaled-up version of GPT-1, featuring more parameters and trained on a larger dataset. OpenAI decided not to release the full model due to concerns about potential malicious applications, opting instead to release a smaller model and a technical paper for research purposes.
Train Your Own LLM from Scratch
tool
4 min read
GitHub repository angelos-p/llm-from-scratch offers a hands-on workshop for building a GPT training pipeline from scratch. The workshop focuses on reproducing the GPT-2 model with 124 million parameters using PyTorch.
"time to GPT-2", down to 2.91 hours
2 min read
Enabled fp8 training for GPT-2 resulted in a 4.3% reduction in training time, bringing it down to 2.91 hours. Using 8XH100 spot instance prices, the cost to reproduce GPT-2 is approximately $20.