Why can a network with 5x fewer FLOPs still be slower than its rival on a mobile CPU?

## What a FLOP count actually is A FLOP count is an **analytic** number: you walk the network layer by layer, multiply out the arithmetic each layer performs on one input, and add it up. For a convolution with `k x k` kernels, `c_in` input channels, `c_out` output channels and an `h x w` output map, the multiply-accumulate count is `k * k * c_in * c_out * h * w`. Nothing is measured; no device is involved. That is exactly the property that makes FLOPs useful — you can compare two architectures on paper before you own a trained model — and exactly the property that makes it an unreliable predictor of time. **A naming trap first.** One multiply-accumulate (MAC) is a multiply plus an add, i.e. two floating-point operations. Many papers count MACs and print the word "FLOPs", so two published numbers can differ by 2x purely by convention. Conventions also differ on whether biases, normalization, activation functions and pooling are counted at all — they are usually dropped as "negligible", which is precisely the arithmetic that is cheap but not fast. Before comparing two published numbers, check what each one counted, or recount both yourself. ## Why the ordering flips Latency is arithmetic **plus** everything else the device must do: - **Data movement.** Every operation reads inputs and weights and writes outputs. Some operations do a lot of arithmetic per byte touched (a big dense matrix multiply); others do very little (an elementwise activation, a normalization, a concatenation, a resize). A network that shaves FLOPs by replacing dense work with many light-touch operations can spend most of its wall-clock time shuttling tensors rather than computing. - **Utilization.** Hardware reaches peak throughput only when a single operation offers enough independent work with the right shape. Thin layers, small channel counts, grouped and depthwise convolutions, and odd tensor shapes all leave lanes idle. A depthwise convolution may cost a tiny fraction of the FLOPs of the dense convolution it replaces and still take a comparable amount of time on some devices, because its effective utilization is far lower. - **Fixed per-operation cost.** Each operation has overhead that does not scale with its FLOPs. A network of 200 tiny operations pays that 200 times; a network of 40 large ones pays it 40 times. This is why the mismatch is worst at batch size one, where there is the least real work to amortize the overhead against. - **Implementation maturity.** A well-worn operation with a highly tuned implementation on that device beats an exotic operation with a generic fallback path, at equal FLOPs. - **Non-arithmetic work.** Layout changes, padding, type conversions and data-dependent control flow cost time and count zero FLOPs. Because utilization and implementation quality are properties of the **device**, the ranking is device-specific. Two models can swap places between a phone CPU, a phone's neural accelerator and a server GPU. "Model A is faster" is only a claim about the hardware you measured on. ## The worked case Candidate A has one-fifth the FLOPs of candidate B and is measurably slower on the target mobile CPU. What the FLOP count failed to price in: A's savings came from operations with low arithmetic density and low utilization, A has many more separate layers so pays overhead many more times, its normalization and activation layers contribute almost nothing to the FLOP total but a real share of the runtime, and it is being run at batch size one, the regime that most punishes small operations. B is "expensive" in a currency the device happens to spend efficiently. ## So why report FLOPs at all Because it is device-independent, computable at design time before any training run, and roughly monotone **within** one architecture family — doubling the width of the same network really does make it slower. It is a legitimate design-time proxy and a legitimate way to say "this model is in a different cost class". It is not an acceptance criterion. If the deployment constraint is stated in milliseconds, the number that closes the ticket is a measured millisecond figure on the target hardware, in the batching regime the product actually uses. ## What a strong answer sounds like Name the metrics separately — FLOPs is arithmetic, latency is time — give two concrete reasons they diverge (utilization and per-operation overhead, or data movement and operation count), note that the ranking is hardware-specific, mention the MAC-versus-FLOP convention trap, and finish with the practical rule: use FLOPs to shortlist candidates, use measured on-device latency to choose between them.