Decoded
← Feed
arXiv:2609.16338AILG14 sep 2026

Breaking the 1.58-bit Barrier for Ternary LLMs

1 min readHN ×3PDF ↗
BITCOS1.585-TRIT PER BYTE1.632-BIT PACKING202 BITS PER WEIGHT (BITNET)
fig. bits per weight (bitnet)BITCOS stores BitNet smaller than both packings

Ternary LLM weights are mostly zeros, so storing them by zero-ness beats 1.58 bits.

1.485bits per weight

Storage cost on sparsest model

Below the log2 3 ≈ 1.585 bits usually treated as the floor

How it works

  1. Packing ignores zeros

    Five-trit-per-byte packing costs 1.625 bits per weight and treats -1, 0, +1 as equally common. Yet zeros reach 51.5%.

  2. Bitmap plus signs

    Store one presence bit per weight and one sign bit only per non-zero. Cost is 2 minus zero density.

  3. Smaller, faster decode

    Beats five-trit packing on 26 of 29 models. Decode runs up to 1.18x faster on CPUs, 1.27x on GPUs.

non-zero bitssigns onlymaskscatterweightsTernarycheckpointPresencebitmapCompactedsign vectorpdep /LUT unpackMatrix-vectormultiplynon-zero bitssigns onlymaskscatterweightsTernarycheckpointPresencebitmapCompactedsign vectorpdep /LUT unpackMatrix-vectormultiply

What they found

Which comes out on top on Bits per weight (BitNet)?

More zeros, fewer bits

try it

Drag zero density up and watch bits per weight fall.

Zero densityBits per weightFew zeros: 2 bitsMany zeros: under 1.5Sparsest real model
Bits per weight
low
Cost is 2 − z; past z = 0.375 it beats the fixed 1.625-bit packing.Illustrative model, not the paper's data

Terms

6 terms

Ternary weight A weight restricted to -1, 0 or +1, times a shared group scale.

A seating chart: first mark which seats are occupied, then list only the occupants' team colors. Empty seats cost one tick, nothing more.

Where it breaks
Reading the colors back needs counting occupied seats, which costs compute.

Limits

  • Slower than 2-bit kernels on bandwidth-rich, few-core Lunar Lake CPU.
  • Gains shrink when zero density is below 0.375; 3 models lose.
  • Unpack kernels only shown for x86 and Intel Xe2 GPUs.
From the paper