← All topics
quantization
Latest stories covering quantization, with summaries and links to related coverage.
2 stories · 2 reports · page 1 of 1
Connected topics
Compression is prediction
opinion
18 min read
Compression techniques and large language models (LLMs) both aim to predict and represent data efficiently. Both fields utilize similar principles to achieve their respective goals of data reduction and understanding.
Unsloth Dynamic 2.0 GGUFs
tool
8 min read
Unsloth Dynamic v2.0 quantization significantly enhances performance over previous methods, achieving new benchmarks for Aider Polglot, 5-shot MMLU, and KL Divergence. The 2.0 GGUFs allow for running and fine-tuning quantized LLMs with minimal accuracy loss on various inference engines, including llama.cpp and LM Studio.