Which PyTorch API quantizes a model for on-device ExecuTorch use today?

## The question behind the question An interviewer asking this is usually checking whether your on-device knowledge is current. PyTorch's mobile story has been rewritten twice: the runtime brand changed (PyTorch Mobile → ExecuTorch) and the quantization API changed (eager-mode `torch.ao.quantization` → PT2 export quantization in `torchao`). A candidate who answers "call `quantize_dynamic` and then `optimize_for_mobile`" is describing a path the vendor has deprecated. ## The current path, step by step ExecuTorch is ahead-of-time: nothing about your Python model class survives to the phone. What ships is a `.pte` file produced from an **exported graph** — a flat, traced representation of the operations. Quantization therefore has to be a graph transformation, applied before lowering: 1. **Capture.** `torch.export.export(model, example_inputs).module()` gives you a graph module you can still run in Python. 2. **Choose a quantizer.** A quantizer is a backend-specific object that decides *which* nodes get quantized and *how*. For CPU inference that is `XNNPACKQuantizer`, configured with `get_symmetric_quantization_config(is_per_channel=True)` and attached with `set_global(...)`. 3. **Prepare.** `prepare_pt2e(graph, quantizer)` inserts observers at the annotated tensors. 4. **Calibrate or fine-tune.** Run representative inputs through the prepared module (static PTQ), or fine-tune it if you used `prepare_qat_pt2e`. 5. **Convert.** `convert_pt2e(prepared)` replaces observers with quantize/dequantize op pairs carrying concrete scales and zero-points. 6. **Lower.** Export again and lower to `.pte` for the target backend. The imports that matter: `torchao.quantization.pt2e.quantize_pt2e` for `prepare_pt2e` / `convert_pt2e` / `prepare_qat_pt2e`, and `executorch.backends.xnnpack.quantizer.xnnpack_quantizer` for `XNNPACKQuantizer` and `get_symmetric_quantization_config`. ## What the old path looked like Eager-mode quantization worked on the module tree, not a graph. You either called `torch.ao.quantization.quantize_dynamic(model, {nn.Linear}, dtype=torch.qint8)` for a one-liner dynamic conversion, or you inserted `QuantStub`/`DeQuantStub` into your `forward`, set a `qconfig`, called `prepare`/`convert`, and — for the old lite interpreter — passed the TorchScript module through `torch.utils.mobile_optimizer.optimize_for_mobile` before saving. Every one of those symbols still imports today; presence is not currency. `torch.ao.quantization` carries a deprecation notice pointing at `torchao`. ## Why the migration happened at all Eager quantization had two structural problems. First, it could only see modules, so anything expressed as a functional call in `forward` (a `+`, a `torch.matmul`, a `F.relu`) was invisible to it — hence the `QuantStub` boilerplate and the `FloatFunctional` shims. Second, its module swaps produced objects that were awkward to export; the quantized model and the deployable model were built by two different mechanisms. PT2 export quantization fixes both: the graph shows every operation, so the quantizer can annotate patterns like conv→batchnorm→relu directly, and the artifact it produces is the same exported graph the lowering pipeline already consumes. It also decouples "what to quantize" (the quantizer, owned by the backend) from "how to do it" (`prepare_pt2e`/`convert_pt2e`, owned by the framework), which is why a new accelerator can ship a quantizer without patching the core flow. ## What to say, and what not to Say: quantization for on-device PyTorch is a graph pass over an exported model, done with `prepare_pt2e`/`convert_pt2e` and a backend quantizer, before lowering to `.pte`. Mention that the eager API is the deprecated predecessor and that you would recognise it in an old repo. Do not say that quantization is a flag on the runtime or the interpreter — the `.pte` you ship is already quantized; the phone just executes int8 kernels. Do not say you quantize after lowering; by then the backend has already decided which subgraphs it can take, and that decision depends on the quantization annotations being present. And do not describe the on-device runtime as "PyTorch Mobile" — the TorchScript lite-interpreter path is the legacy alternative, not the default. ## Version caveat The exact import path for `prepare_pt2e` has moved once already (from `torch.ao.quantization.quantize_pt2e` to `torchao.quantization.pt2e.quantize_pt2e`), and `XNNPACKQuantizer` is imported from the ExecuTorch backend package. When you name these in an interview, say which release you are describing; the flow shape is stable even where the module path is not.