TensorFlow

Match the hardware and the failure model: MirroredStrategy for several GPUs in one box, MultiWorkerMirroredStrategy for several machines with a good interconnect, ParameterServerStrategy when workers are asynchronous or preemptible. All three replicate the model, so none of them fits a model too large for one device.

Decide by blast radius. TF_USE_LEGACY_KERAS=1 with the tf_keras package unblocks the TensorFlow upgrade in a day but pins you to a maintenance-only Keras 2. Treat it as a time-boxed bridge and schedule the Keras 3 migration behind it.

TF Serving is worth it when the endpoint is mostly tensor math and you want versioning, request batching, warmup and a C++ runtime for free. A custom service wins when heavy Python-side logic surrounds the model and cannot move into the graph.

Eager suits development, debugging, and code dominated by a few large ops or by dynamic Python, where graph optimization buys little. Graph execution earns its constraints where many small ops run in a hot loop, where XLA or an exported artifact is required, or where a Python interpreter cannot be in the path.

Only when the step itself is genuinely non-standard — alternating optimizers, gradient surgery, higher-order gradients. A hand-written loop is not faster; it makes you re-own metric aggregation, validation, checkpoint cadence and logging, and every one of those is a place to introduce a silent bug.