For a 3x3 convolution with 256 input and 256 output channels, how many parameters and multiply-adds does it cost on a 56x56 map?

## Two different quantities A convolution layer has two costs that people routinely conflate: how much it **stores** and how much it **computes**. The arithmetic separates them cleanly. ### Parameters One filter produces one output channel and spans the full input depth, so it is a `k_h x k_w x C_in` block of weights. With `C_out` filters and one bias per filter: ``` params = k_h * k_w * C_in * C_out + C_out ``` For the 3x3, 256-in, 256-out layer: `3 * 3 * 256 * 256 = 589,824` weights, plus 256 biases, giving **590,080**. Notice what is absent from the expression: height, width, batch size. Nothing about the input resolution appears, because the same filter is reused at every spatial position. That reuse — weight sharing — is precisely why a convolution's storage is resolution-independent. Biases are a rounding error here: 256 of 590,080 is under 0.05 percent. They are still worth naming when asked for an exact count, and worth knowing they are often omitted entirely when a normalization layer such as BatchNorm follows, because that layer supplies its own per-channel shift. ### Multiply-accumulates Every output value is a dot product over the filter block, which is `k_h * k_w * C_in` multiply-accumulates. There are `C_out * H_out * W_out` output values, so ``` MACs = k_h * k_w * C_in * C_out * H_out * W_out ``` On a 56x56 output map: `589,824 * 3,136 = 1,849,688,064`, about **1.85 billion MACs**. A MAC is one multiply plus one add, so if you report FLOPs with multiplies and adds counted separately the figure roughly doubles to about 3.7 billion. Always state which convention you are using; half the disagreements about "how many FLOPs" are convention mismatches rather than arithmetic errors. ## The resolution lever Now move the identical layer to a 28x28 map. Output positions fall from 3,136 to 784, so ``` MACs = 589,824 * 784 = 462,422,016 ``` about 0.46 billion — **exactly one quarter**, because halving both spatial axes quarters the area. The parameter count is untouched at 590,080. Nothing about the layer changed; only how many times its weights were applied. This gives a compact way to think about the two levers. A convenient identity is ``` MACs = (params - C_out) * H_out * W_out ``` so `H_out * W_out` is literally the **reuse factor**: how many times each weight is used in a forward pass. Doubling channel width roughly quadruples both parameters and MACs (both `C_in` and `C_out` grow); halving resolution quarters MACs and leaves parameters alone. ## Reading a whole stack Run the arithmetic across a typical downsampling backbone and a pattern falls out. Early layers work at high resolution with few channels: small `C_in * C_out`, huge `H_out * W_out`, so few parameters but a lot of arithmetic. Late layers work at low resolution with wide channels: the reverse — many parameters, comparatively little arithmetic. This is why an answer that says "the parameter count tells you how expensive the layer is" is wrong in both directions, and why a proposal to shrink a model must state *which* cost it is shrinking. ## Doing it under interview conditions Keep the two formulas separate and substitute carefully: 1. Kernel area times input channels gives one filter's weight count. 2. Multiply by output channels for the layer's weights; add `C_out` for the biases. 3. Multiply the weight count by `H_out * W_out` for MACs — and get `H_out`, `W_out` from the output-shape formula, not from the input size, since a strided layer has fewer output positions than input positions. Step 3 is where most candidates slip: for a stride-2 version of the same 3x3 layer on a 56x56 input, the output is 28x28, so the MACs are a quarter of the stride-1 figure even though the input map is identical. The misconceptions worth pre-empting: that parameter count grows with image size; that a filter is 2D rather than a `k x k x C_in` block; that there is one kernel per layer rather than one per output channel; and that quoting a weight count answers a question about arithmetic cost.