How would you specify a neural architecture search for an on-device image classifier?

## Why a specification, not an algorithm Asking "which architecture search should I use" is the wrong first question. A search is defined by three choices, and two of them are yours regardless of which published strategy you pick. ### 1. The search space The space is the set of architectures the search can ever return. Two common shapes: - **Cell-based**: search a small directed graph — a *cell* — over a fixed menu of operations, then stack copies of the cell in a hand-designed macro skeleton (stem, stages, downsampling points, classifier head). Cheap, transferable, but the skeleton is hand-designed and does a lot of the work. - **Macro / chain-structured**: fix the block *type* and search per-stage hyperparameters — kernel size, expansion ratio, channel count, number of repeats, where to downsample. This is the shape most on-device searches take, because those are the knobs that actually move latency. The critical property: **the space encodes your prior**. A space in which every reachable point is already a decent model makes the search look brilliant; a space with no good points makes the best strategy useless. When you report a result, the space is part of the result. ### 2. The objective For a deployment-constrained model this is multi-objective. You have accuracy on a validation split, and you have a cost your deployment constraint actually names. There are three standard ways to combine them: - **Hard constraint**: maximize accuracy subject to cost <= budget; candidates over budget are rejected or heavily penalized. Simple and honest when the budget is genuinely hard (a frame deadline). - **Scalarized trade-off**: a weighted product such as accuracy multiplied by `(latency / target)^w` with a negative exponent, so exceeding the target is penalized smoothly. Tune `w` to control how badly you want to sit near the target. - **Pareto front**: keep the non-dominated set and choose afterwards. Most useful when you will ship several models to several device tiers. The operational point is that a search optimizes *exactly* what you write down, with none of a human's tacit judgement. If you write down a proxy — a count of operations — you will get an architecture that is excellent at that proxy, which is not what the product needs. Hardware-aware searches therefore put **latency measured on the target device** into the reward (measuring on the phone, or via a lookup table of per-operator measurements built from that phone), rather than a device-independent stand-in. MnasNet is the canonical example of exactly this multi-objective, on-device-measured formulation. ### 3. The search strategy Three families dominate: - **Reinforcement learning**: a controller emits an architecture description, the child is evaluated, and the resulting score is the reward used to update the controller. Conceptually clean, sample-hungry. - **Evolutionary**: maintain a population, mutate and recombine architectures, discard the worst (or, in regularized evolution, the oldest). Robust, easy to parallelize, easy to add constraints to. - **Differentiable relaxation**: replace the discrete choice among candidate operations with a continuous mixture weighted by learned coefficients, optimize architecture coefficients and weights jointly, then discretize by taking the strongest operation per edge. DARTS is the reference method. Enormously cheaper; the discretization step is the known weak point, since the mixture that trained well is not always the argmax that gets shipped. ### The fourth thing everyone forgets: how a candidate gets scored Strategy choice is nearly free; **candidate evaluation is the whole bill**. Training every candidate to convergence is what made early searches cost thousands of accelerator-days. The practical options are proxy evaluation (fewer epochs, a subset of the data, a smaller image size, a shrunken model) and **weight sharing**: train one over-parameterized supernet containing every candidate as a subnetwork, then score candidates by inheriting supernet weights. Both trade fidelity for speed, and both should be checked: does the cheap score actually rank candidates the same way full training would? ### Putting it together for an on-device classifier A defensible specification: a macro space over per-stage kernel size, expansion and repeats within a fixed block family; an objective of validation accuracy under a hard latency ceiling from the product's frame budget, measured on the target device via a per-operator latency table; a weight-sharing supernet trained once, searched with evolution under the latency constraint; then the top few candidates retrained from scratch with the full recipe and re-measured on the device before anyone believes the ranking. State the total budget before starting, and state what you will compare against — otherwise you cannot tell whether the search or the space produced the win.