Deep Reinforcement Learning

Watch the trajectories and log physical invariants such as penetration depth, contact impulse and energy, and track a success metric defined independently of the reward. A rising return inside the same buggy simulator is not evidence.

Not as stated. Observation normalisation, reward scaling and advantage standardisation often move returns more than an algorithmic change does, so the comparison must give both arms the same details and the same tuning budget before any gain can be attributed to the algorithm.

First confirm the growth is divergence, not a legitimately large return, by comparing against the maximum possible discounted value. Then relax whichever leg your problem can afford: on-policy data, longer or full returns instead of bootstrapping, or a simpler representation.

Each extension fixes a specific symptom and adds tuning surface. Add one only when its symptom shows in the diagnostics, one at a time with matched seeds, and skip any whose failure mode your environment lacks.