Which tf.lite post-training quantization modes require a representative_dataset, and why?

## The three post-training modes, in terms of the API Everything is driven off `converter.optimizations = [tf.lite.Optimize.DEFAULT]`. What you add alongside it decides the mode. **Dynamic-range** — `optimizations` alone. Weights are converted to int8 and stored that way, which cuts file size about 4x versus float32. Activations remain float32 in the graph; at inference the kernel quantizes them on the fly, does an integer matmul or convolution, and dequantizes the result. No calibration data is needed because weights are constants sitting in the file — the converter reads them and computes their ranges directly. **Float16** — `optimizations` plus `target_spec.supported_types = [tf.float16]`. Weights are cast to 16-bit float, roughly halving the file. This is a pure numeric cast with no range estimation involved, so again no calibration data. Accuracy impact is usually negligible, and float16 kernels are attractive when the target has good half-precision hardware. **Full-integer** — `optimizations` plus `representative_dataset`. Now both weights and activations become int8, and the graph can run end to end in integer arithmetic. This is the only mode that needs data. ## Why activations are the ones that need data Quantizing a tensor to int8 means choosing an affine mapping between real values and the 256 available integer levels: `real ≈ scale * (q - zero_point)`. To choose `scale` you need to know the range of values the tensor takes. For a weight tensor that is trivial: the values are baked into the model, so the converter reads the array and takes its min and max. Convolution weights are typically quantized per output channel (per-axis), which handles channels with wildly different magnitudes. An activation tensor has no values until something is fed through the network. The output range of layer 12 depends entirely on what the inputs look like. The only way to learn it is to run inference on representative samples and watch. That is exactly what the calibration pass does: it executes the float graph over your generator, records min/max statistics per tensor, and freezes them into the model as static quantization parameters. At inference, no range estimation happens at all — the scales are constants, which is precisely what makes integer-only accelerators able to run the graph. ## What the generator must yield ``` def representative_dataset(): for sample in calibration_samples: # ~100-500 samples yield [sample] # a list, one entry per model input ``` The generator yields a **list** of inputs per call, one element per model input, each shaped like a single batch. Two rules matter more than the mechanics: 1. **Use real data.** Random noise produces ranges that no real input ever produces. Typically that means implausibly wide ranges, which stretch the scale and throw away precision where the real values actually live — accuracy collapses and nothing errors. 2. **Use production preprocessing.** If your serving path normalizes to [-1, 1] but calibration feeds raw 0-255 pixels, every recorded range is wrong. This is the single most common cause of "quantization destroyed my model." Labels are irrelevant — calibration is unsupervised, it only watches tensors. A few hundred samples is the standard guidance; the returns flatten quickly, but too few samples miss the tails. ## Forcing the integer-only op set By default, if the converter meets an op with no int8 kernel it may leave that op in float, producing a mixed graph that still runs on CPU but cannot be handed wholesale to an integer accelerator. Adding ``` converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8] ``` turns that silent fallback into a conversion error naming the offending op. On any project targeting an integer-only NPU or DSP, set it — you want the failure at build time, not a mysterious performance cliff on device. ## Choosing between the modes The deciding factor is usually the target hardware, not the accuracy budget. Integer-only accelerators and microcontrollers require full-int8; there is no other option. Float16 suits targets with fast half-precision float units and is the least disruptive to accuracy. Dynamic-range is the cheap default: it gets the size win with no data pipeline and no calibration risk, but the per-inference quantize/dequantize work means it will not match full-int8 latency, and it does not unlock integer-only hardware. A sensible order of work: convert float32 and verify parity, then try dynamic-range, then full-int8 with a proper calibration set, measuring accuracy on a held-out set at each step so you know which step cost you what.