With 800 labelled chest radiographs and a natural-image backbone, do you freeze or fine-tune?

## The decision, framed properly A pretrained network splits into a **backbone** — everything up to the pooled feature vector — and a **head**, the small layer mapping that vector to your task's outputs. The head is always new and always trained, because a new task means a new label set. The real question is what happens to the backbone: leave every weight fixed (*feature extraction*), update every weight (*full fine-tuning*), or fix the lower part and update the top (*partial freezing*). Two variables decide it, and an interviewer wants to hear both: 1. **How many labelled target examples you have.** Every weight you unfreeze is a parameter fitted from your data. A mid-sized image backbone holds tens of millions of weights; 800 labelled radiographs cannot constrain that many degrees of freedom, so the network fits the training set and generalises badly. The working rule is that trainable parameter count should scale with labelled example count. 2. **How far the target domain is from the pretraining domain.** Early layers learn generic structure — oriented edges, corners, textures, intensity transitions — which is present in essentially any image, radiographs included. Later layers compose those into domain-specific parts: animal faces, wheels, text. On a far domain those late features describe things that are not in your images, so freezing them asks the classifier to separate your classes using coordinates that do not describe them. ## The four cells - **Large data, near domain** — 200,000 in-domain marketplace product photos against a natural-image backbone. Fine-tune everything. The data supports it, the pretrained weights act as a good initialisation rather than a constraint, and freezing leaves accuracy on the table. - **Small data, near domain** — a few thousand photos of the same kind of object. Freeze the backbone and train the head, or unfreeze only the last block. Fast, hard to overfit, and the frozen features are already close to right. - **Large data, far domain** — hundreds of thousands of radiographs. Fine-tune everything; with that much in-domain data the pretrained weights are mostly a faster starting point. - **Small data, far domain** — the 800-radiograph case, and the one interviewers pick. Keep the generic early layers, replace the head, unfreeze only the top stage so the domain-specific part can be relearned inside a parameter budget the data can support. Lean on augmentation, weight decay and early stopping, and remember that a validation split carved out of 800 examples is itself noisy — cross-validation buys you a more trustworthy comparison. ## Where to cut Cut at a structural boundary — the end of a stage or a residual block — never in the middle of one, so a block's convolutions and its normalization move together. In a five-stage convolutional backbone, *freeze stages 1-3, tune stages 4-5* is the common split, and it lands roughly where features stop being generic. You can locate the boundary empirically: pool the activations at the output of each stage, train a cheap linear classifier on each pooled representation, and see where target accuracy stops improving with depth. If accuracy peaks at stage 3 and then falls, the layers above encode the source domain — those are exactly the ones worth unfreezing. ## Run the cheap baseline first Feature extraction is the fastest experiment available: with the backbone fixed you push each image through once, store the resulting vector, and train heads on the stored vectors in seconds. That gives a number within an hour, tells you whether the pretrained representation is useful for your task at all, and sets the bar every heavier option must clear. ## Costs that follow the choice Freezing does more than skip weight updates. A frozen sub-network stores no intermediate activations for backpropagation and computes no gradients through them, so memory drops and epochs get faster, and cached embeddings can be reused across many head experiments. Full fine-tuning gives all of that up and produces a full model copy per task, which matters when you serve many tasks from one platform. ## Reading the result Compare options on the same held-out split. A wide gap between training and validation accuracy after full fine-tuning is the signature of too many trainable parameters for the data — unfreeze less. A frozen baseline that underfits *both* training and validation is the opposite signal: the representation is wrong for your domain, so unfreeze more. Two adjacent questions have their own answers elsewhere — how small the step size must become once you unfreeze, and what the model forgets about its source task. One practical warning: if you decide to freeze, make sure the backbone is actually fixed. Normalization layers left in training mode keep updating their running statistics even when no weight receives a gradient, and then your 'frozen' features are quietly moving.