inference-acceleration
Latest stories covering inference-acceleration, with summaries and links to related coverage.
2 articles · page 1 of 1
Connected topics
Cerebras CS-4
tool
2 min read
Cerebras introduced the CS-4, a rack-scale AI inference system built around three WSE-3 Turbo wafers per system and the company’s new Nexus Platform Architecture. Cerebras says each wafer delivers up to twice the speed of the prior generation, while the full CS-4 system provides inference up to 30 times faster than GPU systems. The company also claims CS-4 can generate more than 1,000 tokens per second on models exceeding 10 trillion parameters. CS-4 combines compute, power, cooling and I/O in modular Wafer-Scale Backpacks. Each backpack includes a wafer, power conversion, direct liquid cooling, high-speed I/O and control electronics in a compact 3D package that Cerebras says has 50% fewer components. Power delivery sits 0.5 millimeters from the processor, compared with roughly 50 mm on conventional GPU boards, enabling twice as much power to reach the WSE-3T, according to Cerebras. A programmable I/O subsystem doubles I/O bandwidth and reduces latency, with wafer-to-wafer links reaching as low as two microseconds without a switch. CS-4 separates its power, cooling and network infrastructure from modular compute units: PowerRacks can be installed and facility-qualified before compute backpacks arrive, which Cerebras says reduces deployment time from days to hours.
Speculative Speculative Decoding (SSD)
research
2 min read
Speculative decoding accelerates autoregressive inference by using a fast draft model to predict upcoming tokens from a slower target model. It verifies predictions in parallel with a single forward pass of the target model, addressing the sequential dependency bottleneck.