← All topics
inference-frameworks
Latest stories covering inference-frameworks, with summaries and links to related coverage.
1 stories · 1 reports · page 1 of 1
Connected topics
BitNet: 100B Param 1-Bit model for local CPUs
tool
7 min read
BitNet is an official inference framework for 1-bit LLMs, providing optimized kernels for fast and lossless inference on CPU and GPU. The initial release of bitnet.cpp shows performance improvements of 1.37x to 5.07x on ARM CPUs, particularly benefiting larger models.