PyTorch

Register a forward hook on the submodule you want: handle = layer.register_forward_hook(fn), where fn receives (module, args, output). Detach what you store, and call handle.remove() afterwards or the hook keeps firing and holding tensors alive.

Basic indexing and slicing return views that share the original tensor's storage buffer. That buffer is freed only when the last tensor referencing it is gone, so one saved row pins the entire allocation. Call .clone() to detach the slice onto its own storage.

Call it after loss.backward() and before optimizer.step(). It computes one global L2 norm across all the gradients you pass it and, if that norm exceeds max_norm, scales every gradient by the same factor so the combined norm equals max_norm.

Let the target decide. On-device means ExecuTorch; a heterogeneous CPU or accelerator fleet with simple pre-processing favours ONNX Runtime; a GPU service with Python-shaped pre- and post-processing favours a Triton PyTorch or Python backend. TorchServe is retired and should not anchor a new design.

state_dict keys are built from the attribute path down the module tree, so self.fc produces fc.weight. Rename the attribute and every key under it changes, making a saved checkpoint's keys unrecognisable to the new class.

Subclass torch.autograd.Function with static forward and backward methods, save what backward needs via ctx.save_for_backward, and invoke it with .apply(). You need one only when autograd cannot derive the gradient itself: a non-differentiable step, a custom CUDA or C++ kernel, or a hand-written backward that is faster or more numerically stable.

Because a random generator created in the parent process is copied unchanged into every worker. PyTorch seeds each worker's torch, Python random and NumPy global generators from base_seed plus worker id, but an RNG object your Dataset holds, or a third-party library's state, is duplicated.