Convolutional Networks

Semantic segmentation labels each pixel with a class but no identity, so touching objects merge. Instance segmentation returns one mask per countable object. Panoptic gives every pixel exactly one class and, for countable classes, an instance id.

IoU is the overlap area of a predicted and a ground-truth box divided by the area they jointly cover. A prediction counts as correct when its IoU with an unclaimed ground-truth box of the same class clears a threshold, commonly 0.5.

A 3x3 filter over a 4-channel input is a 3x3x4 weight block plus one scalar bias. The kernel always spans every input channel, sums across them, and produces a single output channel.

A convolution reuses one small kernel at every position, so it learns a feature detector once instead of relearning it at each pixel. That cuts the parameter count enormously and lets the same feature be found anywhere in the input.

Max pooling keeps the largest activation in each window, so a sparse, high-contrast response survives the downsample. Average pooling keeps the window mean, so it preserves overall texture and intensity but dilutes an isolated strong response.

A residual block computes F(x) + x, so its layers only learn the change to make to the input. Behaving like an identity then just means pushing F toward zero, which plain stacked layers struggle to fit.

Drop the flatten-and-dense head and make every layer convolutional. The dense head is rewritten as an equivalent convolution, the last layer emits one score map per class, and a decoder upsamples those maps back to the input resolution.

Each spatial axis independently gives floor((n + 2p - k)/s) + 1: n is that axis's input size, p the padding per side, k the kernel size, s the stride. The floor drops any partial final window.

A 1D kernel slides only along time. Each filter carries weights for every input channel at every offset in its width, so one kernel position sums k times 6 numbers into a single output value.

Two stacked 3x3 layers see the same 5x5 input patch as one 5x5 layer, but use 18C-squared weights instead of 25C-squared and apply two nonlinearities instead of one. Same reach, cheaper, more expressive.

It factorises a dense layer into a spatial stage with one k x k filter per input channel and a pointwise stage that mixes channels. Cost falls to 1/C_out + 1/(k*k) of dense, about 1/9 at k=3.

RoI pooling rounds twice, the box onto the feature grid and then the bin edges, so features come from up to half a cell off. RoIAlign keeps float coordinates and samples bilinearly, preserving the alignment masks need.

Two disjoint boxes have IoU 0 however far apart they sit, so a 1 - IoU loss is flat there and produces no gradient. GIoU subtracts the empty fraction of the smallest box enclosing both, which keeps shrinking as they approach, restoring a direction to move.

Stride is the step between successive kernel placements. Stride 2 evaluates the filter at every other position, so each spatial dimension comes out roughly halved and compute drops about fourfold. Parameters are unchanged; spatial precision is lost.

Anchors are fixed reference boxes tiled over every feature-map location; the head predicts a class score and offsets that nudge an anchor onto an object. Anchor-free heads drop them and regress the four side distances straight from each location.

Equivariance means shifting the input shifts the feature map by a corresponding amount. Invariance means the output does not change at all. Stacked convolutions give you equivariance; invariance has to be added by the readout or learned from data.

Downsampling in the encoder destroys the precise location of edges, and upsampling cannot invent it back. U-Net concatenates the matching high-resolution encoder maps into each decoder stage, so the decoder combines deep semantics with exact boundary detail.

Causality means output at step t may use only inputs up to t. Padding (k-1)*d zeros on the left, then a valid convolution, preserves length and enforces that. Symmetric padding centres the kernel on t, so it reads the future.

Suppression assumes heavily overlapping boxes of the same class are duplicates of one object. Identical cartons packed side by side genuinely overlap above the threshold, so the lower-scoring box is deleted even though it is a real, distinct object.

Each stride-1 3x3 layer widens the receptive field by two pixels, so ten of them see just 21 input pixels, far too few for a 300-pixel lesion. Grow it by downsampling, by adding depth, or with larger kernels.

About 90% sit in the three fully connected layers at the end. The first alone, mapping the flattened 7x7x512 feature map to 4096 units, holds roughly 103 million weights; all 13 convolution layers together hold under 15 million.

Grouping partitions input and output channels into g disjoint sets, each output group seeing only its own input group. Weights and multiply-accumulates both drop by a factor of g, but no channel combines across group boundaries.

Decoupling. The classification branch already names the class, so each mask channel only has to answer whether a pixel is inside the object. A per-pixel softmax would make classes compete for pixels and tie mask quality to classification confidence.

Valid padding computes outputs only where the kernel fits inside real data, so the map shrinks at every layer and a border ring gets no output position. Same zero padding preserves size but feeds invented zeros into border windows.

Global average pooling reduces each channel's spatial map to its mean, so the classifier sees one number per channel. It adds no parameters, accepts any input size, and removes the huge flatten-to-dense layer that held most of a network's weights.

A 3x3 convolution's cost grows with input channels times output channels, so it is ruinous at wide layers. The bottleneck squeezes width down with a 1x1, runs the 3x3 cheaply, then restores width — roughly an order of magnitude fewer weights.

A ResNet shortcut can stay a plain identity only while the branch preserves spatial size and channel count, since the two are added elementwise. Where a stage strides down and widens, it becomes a 1x1 convolution.

Parameters are 33256256 + 256 = 590,080, independent of the feature-map size. Multiply-adds are 589,824 weights evaluated at every one of the 5656 output positions, about 1.85 billion.

Each layer adds (k-1)*d input steps, so the receptive field is 1 plus the sum of (k-1)*d over the layers. Width 3 with dilations 1, 2, 4, 8, 16 covers 63 steps - divide by the sampling rate for seconds.