How does converting a QAT Keras model to TFLite differ from post-training int8?

## Two ways to get the same int8 model Both paths end at an int8 `.tflite` file. They differ in where the quantization parameters come from and whether the weights ever got a chance to adapt. **Post-training**: train normally, then hand the converter a `representative_dataset`. The converter runs those samples, records each activation tensor's range, and computes scales. The weights never saw rounding during training. **Quantization-aware training**: wrap the model before (or partway through) training so that the forward pass simulates int8 rounding, then fine-tune. Ranges are learned as part of training, and gradients push the weights toward values that survive rounding. ## What the wrapper actually inserts `tfmot.quantization.keras.quantize_model(model)` returns a copy of the model whose layers are wrapped with quantize-annotation wrappers. Those wrappers add *fake-quant* operations: in the forward pass a tensor is quantized to the int8 grid and immediately dequantized back to float. The value that flows onward is float but has been snapped to the values int8 can represent, so the loss the network sees is the loss it would have with real int8 arithmetic. Backpropagation passes gradients through this non-differentiable rounding using a straight-through estimator, so training proceeds normally. Each fake-quant node also owns min/max variables that track the observed range during training. Those are the numbers the converter will later use as scales — no separate calibration pass required. For partial coverage, `quantize_annotate_layer` marks individual layers and `quantize_apply` materializes the wrappers, letting you leave numerically delicate layers in float. Loading a saved QAT model needs `quantize_scope` so the custom wrapper classes resolve. ## The conversion itself ``` converter = tf.lite.TFLiteConverter.from_keras_model(q_aware_model) converter.optimizations = [tf.lite.Optimize.DEFAULT] tflite_model = converter.convert() ``` Note what is present and what is absent. `optimizations` is still required — it is what tells the converter to produce a quantized model at all. `representative_dataset` is absent, because the ranges are in the graph. The converter recognizes the fake-quant pattern, removes it, and emits genuine int8 operators with the recorded scale and zero-point attached. If you want an integer boundary for integer-only hardware, `inference_input_type` and `inference_output_type` work exactly as they do on the post-training path. ## When QAT is worth it Start with post-training quantization every time — it is a conversion-script change, not a training project. Reach for QAT when post-training quantization has been properly attempted (real calibration data, good coverage, sensitive layers identified) and still misses the accuracy bar, and when selective float or 16-bit activations are unavailable because the target hardware wants a fully integer graph. Models that tend to need it: small, already-compressed architectures with little redundancy to spare; models with wide activation ranges; and tasks where a one-point accuracy drop is genuinely unacceptable. ## The costs, honestly stated **A training cycle.** Usually a short fine-tune from the trained checkpoint rather than training from scratch, but it means data access, a training environment and a re-validation. **Layer coverage.** The toolkit supports a defined set of Keras layers. Custom layers require you to supply a quantize configuration describing which tensors to quantize, or to leave those layers unquantized. **Toolchain currency.** The toolkit's Keras quantization API was built against the pre-Keras-3 API. In an environment where Keras 3 is the default, verify the toolkit works with your setup before scheduling the work; teams commonly pin the legacy Keras path for these runs. **Pipeline complexity.** The training script and the conversion script are now coupled: someone who retrains without the wrappers produces a checkpoint that silently converts as post-training instead. Make it explicit in the pipeline and check the converted model's op types as a build assertion. ## The interview answer What separates a strong answer here is the API detail rather than the concept: knowing that QAT conversion still needs `optimizations` but drops `representative_dataset`, that the fake-quant nodes carry the ranges the converter consumes, and that QAT is the escalation after post-training quantization has been done properly — not the default starting point.