nn.Module and Layers

## Two names for what looks like the same thing When you write a PyTorch model you subclass `nn.Module` and implement `forward(self, x)`. You then use it as `y = model(x)`. New users often ask why not `y = model.forward(x)` — the second is more explicit, and in a toy example it returns exactly the same tensor. The answer is that `nn.Module` implements `__call__`, and `__call__` does more than delegate. Roughly, it: 1. runs **forward pre-hooks** registered with `register_forward_pre_hook`, which may inspect or even replace the inputs; 2. calls your `forward()`; 3. runs **forward hooks** registered with `register_forward_hook`, which may inspect or replace the output; 4. handles bookkeeping around the whole call. Calling `forward()` yourself executes step 2 alone. ## Why the hooks matter in practice Hooks are not an exotic corner of the API — a lot of everyday tooling is built on them: - **Feature extraction.** The standard way to grab an intermediate activation without editing model code is to register a forward hook on the submodule you care about. If some code path calls `.forward()` directly, your hook never fires and your dictionary of activations is empty, with no error to explain why. - **Quantization observers.** The PT2 export quantization flow inserts observers that watch tensors as they flow through modules; bypassing `__call__` bypasses observation, and calibration produces garbage ranges. - **Profiling and debugging.** Tools that count module invocations, time each submodule, or log shapes hang off hooks. - **Wrappers.** `DistributedDataParallel` wraps your module; the gradient-synchronization bookkeeping lives in the wrapper's own call path. Calling `ddp_model.module.forward(x)` skips the wrapper entirely — the model still trains, just without the synchronization you thought you had. The common thread is that the failure is silent. Nothing raises. The tensors have the right shape and plausible values. You only discover it when a metric is wrong or an extracted activation is missing. ## The convention it enforces There is a second, softer reason. `model(x)` makes every module in the ecosystem look the same at the call site, which is what lets you swap `nn.Linear` for a custom block, or wrap a model in `nn.Sequential`, without touching the caller. `nn.Sequential.forward` itself calls each child as `module(input)`, precisely so that hooks on children still fire. ## The inverse mistake The mirror-image error is *defining* `__call__` on your subclass, or naming your compute method something other than `forward`. If you override `__call__`, you replace the hook machinery instead of extending it. If you name the method `run()` and call `model.run(x)`, hooks never fire because nothing goes through `__call__`. Both amount to opting out of the framework. Implement `forward`; let PyTorch own `__call__`. ## What about type checkers and IDEs? A practical annoyance sometimes offered as an excuse for `.forward()`: static analysis often cannot infer the signature of `model(x)` because `__call__` is typed loosely. That is a tooling complaint, not a reason to change runtime behaviour. The standard remedies are to keep a typed alias in your own code, or simply annotate the call site — not to bypass the framework's entry point. ## Summary rule Define `forward()`. Call the instance. If you find yourself typing `.forward(`, you are almost certainly in code that should say `model(x)`, and the exceptions — writing a wrapper that deliberately re-implements the call path — are rare and deliberate.