Trillion parameter models
Latest stories covering Trillion parameter models, with summaries and links to related coverage.
1 stories · 1 reports · page 1 of 1
1 story · 1 report in the last 7 days
Latest report · Counts as of (UTC)
Models in this coverage
No tracked models are linked to this topic’s reports in the last 30 days.
No established topics share multiple stories with this topic yet.
Latest stories
Cerebras Deploys Trillion-Parameter AI Models
research
6 min read
Cerebras says it can deploy multi-trillion-parameter AI models by distributing their layers across WSE-3T wafer-scale chips and connecting the chips as an inference pipeline. Each WSE-3T wafer contains 44 GB of on-chip SRAM, while a trillion-parameter model requires terabytes of weight storage. Cerebras says this layout can run multi-trillion-parameter models at 1,000 to more than 2,000 tokens per second. Using the open-source Kimi K3 model as an example, Cerebras maps its roughly 3 trillion parameters, 69 linear-attention layers, and 24 dense layers across 59 wafers. With 4-bit weight quantization, each layer occupies about 16 GB. Dense layers are assigned one per wafer because the remaining memory is used for the key-value cache; two linear-attention layers fit on a wafer because they use fixed-size per-request state rather than a context-length-dependent KV cache. Activations, rather than model weights, move between wafers through RDMA over Ethernet at roughly two microseconds per hop. Multiple requests remain in flight simultaneously through pipeline stages, and each stage can process a micro-batch. Cerebras says smaller micro-batches reduce individual-request latency, while larger ones raise throughput. Available KV-cache memory on dense-layer wafers limits concurrency for 256,000-token average requests using an 8-bit KV cache.