Breaking the 1.58-bit Barrier for Ternary LLMs

This paper shows that ternary LLMs can be stored and served more efficiently than the long-assumed 1.58-bit-per-weight floor by exploiting the fact that zero values are often much more common than ±1, cutting model footprint and improving inference throughput. For CIOs, the business impact is lower memory and compute cost, faster decode performance, and new deployment options for constrained environments such as edge, client, and on-prem systems. Strategically, it signals that AI infrastructure teams should treat weight layout and sparsity-aware encoding as a competitive lever, not just a low-level optimization, because it can materially change the economics of running LLMs at scale.