← All topics
transformer-models
Latest stories covering transformer-models, with summaries and links to related coverage.
2 stories · 2 reports · page 1 of 1
Connected topics
TokenTown: A visual way to understand how LLMs work
tool
2 min read
TokenTown visualizes a transformer language model as an isometric city, with each district representing a stage of the model. A convoy transports a hidden state through various processes, including tokenization, vector casting, and layer processing, ultimately producing a probability distribution for token selection.
Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train
research
2 min read
Training a single transformer layer can achieve performance comparable to full-parameter reinforcement learning (RL) training. This finding suggests that RL adaptation may not require uniform updates across all layers of large language models.