A conversion of a text column to numbers meets 40 unparseable values in 100,000 — how does coercing rather than refusing change where they surface?

## The decision a conversion cannot avoid A **re-typing step** is an operation whose only job is to hand back the same values in a different representation — text to numbers, in this case. It is one of the few operations in this whole subject that is guaranteed to meet data it cannot handle, because the values were free-form until the moment you asked them to be numbers. So every such operation carries a disposition for a value that will not convert, whether you chose it or took the default. The two dispositions in common use are: - **Coercing** — the unparseable value becomes absence, the conversion succeeds, and the program carries on. Forty rows out of a hundred thousand now hold nothing instead of a number. - **Refusing** — the conversion stops at that line, naming the value it could not handle. No table comes back. A third exists and is worth knowing about because it is the hardest to detect: a few designs substitute a **default**, typically zero, rather than absence. That is worse than either of the above, because a substituted zero is indistinguishable from a real zero and there is nothing left to count. And at the other extreme, some designs offer no coercing mode at all and force you to filter or repair before converting, which is a refusal by another route. ## What actually differs: distance, not correctness Neither disposition is right in the abstract. What separates them is **how far the discovery of a bad value travels from the line that produced it**. | | Coercing | Refusing | |---|---|---| | Where the 40 values are noticed | wherever a figure finally looks wrong | at the conversion, immediately | | What you still have when you notice | absence, with the original text gone | the offending text itself | | Who notices | often a reader of the report | the person running the code | | Cost of a false alarm | none, the run continues | a stopped run over one bad value | | Failure mode | silent, cumulative | loud, immediate, sometimes over-eager | Coercing is not a shortcut and refusing is not rigour. Coercing is a decision that the pipeline's job is to keep running and that a known, small share of unconvertible values is part of the data's nature. Refusing is a decision that every value was supposed to parse and any that does not is a defect somewhere upstream. ## When coercing is the honest call 1. **The unparseable share is expected and bounded.** A free-text field that has always carried a few dashes and blanks is not a defect; it is the field. 2. **You look at the count.** Coercing is defensible only if the number of values that became absence is something a human sees before the result is used. Coercing and not counting is how a field that quietly went 100% unparseable after an upstream change produces an empty column and no alarm at all. 3. **The alternative is worse.** A nightly run stopped by one malformed value out of a million, at three in the morning, with nobody to look at it, is a real cost and not an imaginary one. ## When refusing is the honest call 1. **Every value is supposed to parse.** A column produced by another step in your own code has no business containing text that will not convert, so if it does, something earlier is broken and you want to know now. 2. **The values matter individually.** If the 40 rows are the ones the report is about, turning them into absence has changed the answer rather than degraded it. 3. **You are still developing.** During development a refusal is free information; you can always soften it once you know what the data actually contains. ## The part candidates miss The disposition is a property of the conversion you wrote, not a global setting for the pipeline, and it is entirely reasonable for one step to refuse and the next to coerce. What is not reasonable is not knowing which one you took. Most conversions have a default disposition, and defaults differ between designs — some refuse by default and make you opt into coercing, others coerce by default and make you opt into refusing. A candidate who says "it just turns them into missing values" has described one design's default and is going to be surprised by the other. The second thing candidates miss is that a coercing conversion destroys evidence. Once the text is gone, the only fact remaining is that something was absent, and you cannot answer the question everyone asks next — *what were those forty values?* If you coerce, the time to look at the offending values is before the conversion, not after. ## What an interviewer is listening for That you name both dispositions, that you frame the choice as where the discovery happens rather than as strict against lenient, and that you say out loud which default your tools take instead of assuming everyone's is the same.