speculative-decoding
Latest stories covering speculative-decoding, with summaries and links to related coverage.
1 articles · page 1 of 1
Connected topics
Speculative Speculative Decoding (SSD)
research
2 min read
Speculative decoding accelerates autoregressive inference by using a fast draft model to predict upcoming tokens from a slower target model. It verifies predictions in parallel with a single forward pass of the target model, addressing the sequential dependency bottleneck.