After you delete an output filter from a conv layer, what else must be removed to keep the network valid?

## Pruning is a graph edit, and edits propagate The seductive mental model is that pruning is a per-layer decision: score this layer's channels, keep the top ones, move on. That model breaks the first time you try to run the pruned network, because a channel index is shared by everything that touches the tensor. **Directly attached to the filter.** Output filter `j` owns a slice of the weight tensor and one bias entry. If a normalization layer follows the convolution, it holds four per-channel vectors — learned scale, learned shift, running mean and running variance — and entry `j` of each must go. Forgetting the running statistics is a classic bug: with luck it raises a shape error, and without luck an off-by-one reindexing quietly applies channel `k`'s statistics to channel `k+1`'s activations, which shows up as an unexplained accuracy collapse that recovery training only partly hides. **Consumers.** Every layer reading this feature map has an input dimension equal to its channel count, so each consumer's kernels lose their `j`-th input slice. Multi-branch architectures have several consumers of the same tensor, and every one of them must be edited consistently. ## Coupling groups: where independence disappears **Residual sums.** An addition is elementwise. Two tensors can only be added if they have the same channel count *and* if channel `i` of one means the same position as channel `i` of the other. So if a block's output is added to a skip path, the block's final layer and whatever produced the skip tensor cannot choose different survivors — they must drop the same index set. In a deep network these constraints chain: a whole stage of blocks whose outputs are added together forms one coupled group with a single shared channel dimension. This is why practitioners usually prune the *interior* channels of a block, which nothing else consumes and which are therefore free, and leave the block's output width alone or prune the entire group with a merged score (for example the sum or maximum of each branch's per-index scores). **Concatenation.** Concatenating branches is friendlier: each branch keeps its own channels, so branches may prune independently. What you must track is the offset bookkeeping — the consumer's input slices for branch two shift once branch one gets narrower. **Flatten into a fully connected layer.** If the last feature map is `C x H x W` and is flattened, the first fully connected layer has `C * H * W` input columns. Removing one channel removes not one column but `H * W` of them — every column belonging to that channel's spatial positions. Get the index arithmetic wrong here and the model still runs, silently permuting features. ## The same constraint outside convolutions Attention heads are the transformer's version of the same story. A head is not one parameter block but four coupled slices: its share of the query projection, of the key projection, of the value projection, and the matching block of rows in the output projection that maps its result back into the residual stream. Removing a head means removing that group together; you cannot delete its value slice and keep its output rows. Pruning heads from a trained encoder with, say, twelve heads per layer is attractive precisely because published studies find that many layers keep most of their accuracy on only one or two surviving heads — the redundancy is real, but the removal is group-shaped. Removing an entire residual block is the coarsest group of all: the block's output must be shape-compatible with the identity path, which is exactly why a block can be dropped wholesale without disturbing anything around it. ## How this is done in practice Build an explicit dependency graph over the network before pruning anything. Nodes are the tensors; edges record producer, consumers, and the operations (addition, concatenation, flatten) that constrain them. Then group the parameters that must be pruned together, score at the level of the *group* rather than the individual layer, and apply one index set per group. What falls out of this analysis is a list of unconstrained dimensions — the ones you can cut freely — and constrained ones that cost you either a group-wide decision or nothing at all. ## Failure modes worth naming A shape error at load time is the good outcome; it is loud. The dangerous outcomes are silent: mis-indexed normalization statistics, an unshifted offset after a concatenation, or a flatten whose column removal was done per channel instead of per channel-position. All three produce a network that runs, produces plausible-looking outputs, and trains back to *nearly* the accuracy you expected — hiding the bug behind recovery training. Verify by checking that the pruned network and the original agree on a batch when the pruned channels are instead zeroed in the original: the outputs should match closely.