How do two hidden units let a network express XOR over two binary flags?

## The setup Two binary sensor flags, `x1` and `x2`, each 0 or 1. The target is XOR: output 1 when exactly one flag is on, 0 when neither or both are on. Written out: ``` (0,0) -> 0 (0,1) -> 1 (1,0) -> 1 (1,1) -> 0 ``` A single weighted-sum unit computes `step(w1*x1 + w2*x2 + b)`, which turns on for every input on one side of a straight cut through the square and off on the other. XOR needs the two diagonal corners `(0,1)` and `(1,0)` on one side and `(0,0)`, `(1,1)` on the other, which no single straight cut achieves. So one unit is not enough — and the interesting question is what a second layer *adds*. ## The construction Give the network two hidden threshold units reading the same inputs: ``` h1 = step(1*x1 + 1*x2 - 0.5) fires when at least one flag is on (OR) h2 = step(1*x1 + 1*x2 - 1.5) fires only when both flags are on (AND) ``` Notice what differs between them: nothing but the bias. Same weighted sum, two different learned thresholds. That alone is enough to build two genuinely different features out of the same evidence — a direct illustration of the bias as a learned firing level rather than a bookkeeping constant. Now the output unit: ``` y = step(1*h1 - 1*h2 - 0.5) fires when h1 is on and h2 is off ``` Evaluate all four inputs: ``` (0,0): h1=0, h2=0 -> step(-0.5) = 0 (0,1): h1=1, h2=0 -> step( 0.5) = 1 (1,0): h1=1, h2=0 -> step( 0.5) = 1 (1,1): h1=1, h2=1 -> step(-0.5) = 0 ``` Exactly XOR. ## Why this works — the real lesson The hidden layer performs a *change of representation*. In the original `(x1, x2)` space the four points sit at the corners of a square with the two positive ones on opposite diagonals. In the hidden space `(h1, h2)` they land at only three distinct places: `(0,0)` for the both-off input, `(1,0)` for each of the two one-on inputs, and `(1,1)` for the both-on input. The two positive examples have been *collapsed onto the same point*, and in that new space a single straight cut separates them from the two negatives. That is the general mechanism, and it is worth stating in the interview in exactly these terms: **hidden units are learned features, and the final layer is a simple model applied to them.** The output unit in this construction is still an ordinary weighted-sum unit — it was never made more powerful. What changed is what it is looking at. ## The nonlinearity is load-bearing If the hidden units skipped their threshold and passed `x1 + x2 - 0.5` and `x1 + x2 - 1.5` straight through, both hidden values would be affine in the inputs, the output would be affine in those, and the whole network would be affine in `x1, x2` — back to one straight cut, XOR unreachable. It is the thresholding that lets two units built from the *same* weighted sum carry different information. Hidden layers only add power when something nonlinear sits between the linear maps. ## How small can it be Two hidden threshold units suffice, as constructed. One does not: in a plain 2-1-1 network the output is a monotone function of a single hidden value, which is itself a monotone function of one linear projection of the inputs — so the whole model is monotone along that projection and cannot turn on for two inputs while turning off for a third that lies between them along it. XOR needs at least two hidden units in this architecture, which is why it is the standard minimal demonstration that a hidden layer buys something. ## What training actually finds Do not claim that a trained network will recover this exact OR/AND pair. Hand construction proves *sufficiency* — it shows the architecture can express the function. Gradient training on a smooth activation will find some pair of half-planes whose combination works, and the solution it lands on may be a permutation of the two units, a sign-flipped variant, a rescaled variant, or a pair that carves the square differently but composes to the same behaviour. All are equally valid solutions, and there are many of them. The construction is an existence proof, not a prediction. ## How to present it Write the four target rows, state that one straight cut cannot produce them, then write the two hidden units and the output unit with explicit weights and biases and evaluate all four corners on the board. Finish on the representation point — the hidden layer made the problem linearly easy — because that is the sentence the interviewer is listening for.