Model Quantization

---

Research checked: 20 September 2026.

Quantization represents model numbers using fewer bits. A weight originally stored in a 16-bit format might be approximated with an 8-bit or 4-bit value plus information needed to interpret it. The purpose is to reduce memory use and, when the hardware and kernels support it efficiently, accelerate computation. The central challenge is preserving useful behavior while introducing numerical approximation.

Source: Figure 2 from Frantar et al., “GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers” (2022).

A simplified uniform quantizer uses an integer $q$ to represent a value $x$ through a scale $s$ and zero point $z$:

$$q = \operatorname{clip}(\operatorname{round}(x/s)+z), \qquad \hat{x}=s(q-z).$$

Rounding loses detail, and clipping loses values outside the supported range. Real implementations differ in grouping, codebooks, scaling, and arithmetic. The example explains the basic compromise rather than specifying a production format.

The storage savings can be substantial. For a hypothetical seven-billion-parameter model, 16-bit weights occupy about 14 decimal gigabytes, while four-bit weight values occupy about 3.5 gigabytes before metadata and overhead. That calculation describes weights only. A running model also needs activations, a context cache, and workspace. Weight-only quantization does not automatically reduce every one of those components.

Post-training methods start from an existing model. GPTQ uses approximate second-order information to make weight quantization more accurate, compensating for errors as weights are processed. Its contribution is a way to retain model quality under aggressive compression without performing full model retraining. Frantar et al., GPTQ.

AWQ uses activation information to identify and protect particularly important weight channels through scaling. This addresses an important asymmetry: identical numerical error in two weights need not have identical consequences for the model’s outputs. Lin et al., AWQ.

SmoothQuant tackles a related difficulty when quantizing both weights and activations. Activation outliers can make low-precision representation hard. Its equivalent rescaling shifts part of that difficulty from activations to weights, supporting efficient integer inference in the studied setting. Weight-only compression and weight-and-activation quantization therefore solve overlapping but different deployment problems. Xiao et al., SmoothQuant.

My deployment rule would be to benchmark the complete path. A smaller representation can still require unpacking, conversion, or kernels that are poorly matched to the device. Compare the same prompts, context lengths, batch sizes, and quality criteria. Report time to first output separately from sustained generation speed, and inspect peak memory rather than just file size.

Quality checks should include the behaviors that matter most to the application: exact extraction, arithmetic, rare vocabulary, code generation, or multilingual inputs. An average score can hide a concentrated regression. Calibration examples should also resemble the deployment workload. Quantization is successful when the saved resources outweigh the measured quality cost for a specific use case; the bit count alone cannot tell that story.