Google's TrueTime API, used inside the Spanner database, doesn't return a single timestamp for 'now' - it returns an interval [earliest, latest]. Explain the mechanism behind this design, including the role GPS receivers and atomic clocks play, and why returning an interval is more useful than a system trying to report one perfectly accurate timestamp.

## Reporting the error instead of hiding it The starting insight behind TrueTime is that every physical clock has some error, and pretending otherwise by reporting a single "now" value is a form of dishonesty that pushes the error into whatever system consumes that value, usually silently. TrueTime instead reports the error explicitly: calling `TT.now()` returns a triple, or equivalently an interval `[earliest, latest]`, such that the true, real-world UTC time at the moment of the call is guaranteed to fall somewhere inside that interval. The width of the interval, commonly denoted **epsilon** (so `latest - earliest = 2*epsilon` around a midpoint estimate), is not fixed; it grows the longer it has been since the local machine last synchronized against a trusted reference, because between syncs the local oscillator's drift accumulates just as it does for any other clock. ## What keeps epsilon small What keeps epsilon small in practice is dedicated hardware. Each Google data center hosts a set of "time master" machines equipped with GPS receivers and, in many cases, atomic clocks (rubidium references) as a secondary, independent source. Using two physically different reference technologies is a deliberate defense against correlated failure: | Reference | How it fails | |---|---| | **GPS** | A GPS antenna can be knocked out by a physical fault or signal jamming. | | **Atomic clock** | An atomic clock free-runs on its own internal physics and is unaffected by that particular failure mode, and vice versa an atomic clock can drift out of calibration while GPS remains available. | A local timemaster: - **combines** readings from multiple such references, cross-checking them against each other and discarding any that disagree beyond a sanity threshold; - **serves** its combined estimate to ordinary machines in the data center over a fast, low-latency internal network path, which itself avoids much of the delay-asymmetry problem that limits plain internet NTP. Every machine's local daemon polls its timemasters at short, frequent intervals and grows epsilon linearly with time-since-last-poll, so a poll failure or a slow poll visibly and safely widens the reported uncertainty rather than silently reporting a stale, falsely-confident value. ## The trade-off it makes explicit The trade-off TrueTime makes explicit is: - **Pay:** extra hardware (GPS antennas, atomic clock modules, redundant timemasters per data center), and accept a small, bounded interval width. - **In exchange:** being able to make correctness guarantees that would otherwise require a fundamentally different, non-physical-time mechanism (like a consensus-ordered logical clock funneled through a single sequencer, which doesn't scale the same way globally). Historical published figures put typical epsilon in the low single-digit milliseconds (commonly cited around 1-7ms), which is dramatically tighter than the tens-of-milliseconds-or-worse error typical of public internet NTP, precisely because the reference hardware is local and the network hop to it is short and dedicated rather than routed across the public internet. ## Why an interval beats a best guess The reason an interval beats a single "best guess" timestamp becomes concrete in Spanner's use of it. Spanner needs to order transactions across a globally-distributed database such that the order is **externally consistent**: if transaction T1 completes (returns success to its client) before transaction T2 begins, in real wall-clock time, then T2 must be assigned a later timestamp than T1 and see T1's effects, everywhere, even if T1 and T2 touch entirely different servers with no other communication between them. A naive approach - just assign transactions timestamps from a fast local clock - can't guarantee this ordering, because two clocks that appear to agree might actually be off by more than the real time gap between the transactions (exactly the wall-clock-unreliability problem). TrueTime resolves this by giving Spanner a mechanism to know when it's safe to conclude a chosen timestamp is definitely in the past everywhere: 1. After picking timestamp `s` for a commit, Spanner waits until `TT.now().earliest` exceeds `s` (this wait is called **commit-wait**) before releasing the transaction's results. 2. Because `earliest` is a guaranteed lower bound on true time, once it exceeds `s`, true UTC time is guaranteed to be past `s` everywhere, satisfying the ordering guarantee for any subsequent transaction. Without an interval with an honest, guaranteed lower bound, there would be no safe moment to know "the true time has now definitely passed timestamp s" - a single best-guess value with hidden error offers no such safety margin to wait out. ## Provably bounded, not roughly accurate A concrete way to see why this matters: if TrueTime instead returned a single number that was, say, 90% likely to be accurate to within 2ms, Spanner would either have to 1. accept a small but nonzero probability of an external-consistency violation (unacceptable for a database marketed as strongly consistent), or 2. pad its wait time with an arbitrary, unjustified safety margin with no principled basis for how large it needs to be. The interval turns "roughly accurate" into "provably bounded," which is exactly the property a correctness-critical distributed system needs, at the cost of the specialized infrastructure required to keep that bound small enough to be commercially useful (a wide epsilon, if unaddressed, would make commit-wait latency unacceptably high for a real transactional database).