In an automated coding assessment, why can code that passes the sample cases still score low?

## Two different sets of tests Every automated assessment problem carries two test sets. The **sample cases** are printed in the statement, usually one to three of them, and they exist to disambiguate the wording: what the input looks like, what the output looks like, whether there is a trailing line, whether the answer is a count or a list. The **hidden suite** is what the grader actually runs, and it is written by whoever authored the problem with the explicit intent of separating candidates. The two sets do not overlap in purpose. A submission that satisfies the samples has proven that it reads the format correctly. It has proven almost nothing about correctness. ## What the hidden suite usually contains Across platforms the pattern is consistent enough to plan around. On the invented assessment used through this leaf — a mid-size product consultancy screening junior frontend candidates with three problems in an 82-minute window — the third problem carried 17 hidden tests behind 2 shown examples. A plausible breakdown, and a good default guess for any problem: - **Format and happy path** (the samples again, plus a few variations). - **Boundary inputs**: empty, one element, two elements, all elements identical, already-ordered input, input where the answer is the first or last item. - **Value extremes**: the smallest and largest values the stated range allows, including negatives and zero when the range permits them. - **Scale tests**: inputs built at the constraint ceiling, run under a time limit. These are the tests that fail with a timeout rather than a wrong answer. - **Output pedantry**: rounding, ordering of a returned collection, whitespace, the exact wording of a sentinel value. ## Partial credit changes the strategy Where scoring is per hidden test, the problem score is proportional to the fraction passed. On a 180-point problem with 17 hidden tests, passing 16 of them earns roughly 169 points; passing 5 earns roughly 53. Three practical consequences follow. First, **an unfinished problem is worth attempting**. A solution that handles only the simple case often passes the format and happy-path tests and several boundary tests, which is a meaningfully non-zero score against a blank editor's zero. Second, **a slow but correct solution is not a failure**. It typically clears everything except the scale tests, which is most of the suite. That is why submitting a correct brute force before optimising is the safe order of operations. Third, **the last few tests are the expensive ones**. Moving from 5 of 17 to 14 of 17 is usually a matter of boundary handling and costs minutes; moving from 16 to 17 can require a different approach entirely. Know which one you are buying with the time you have left. ## Reading the constraints as a test plan The constraints paragraph is the most under-read text on the page, and it is effectively a summary of the hidden suite. If a length may be zero, there is a test with zero. If values may be negative, there is a test with negatives. If the ceiling is large enough that a naive scan of every pair would be slow, there is a timing test built exactly there. Treat each clause as a test case and run it yourself before submitting. ## Reading the score report afterwards When the run is over, the platform's score report shows the per-problem hidden-test summary: how many passed, how many failed, and often the category or name of each failing test. That summary is the highest-value artefact of the whole exercise for your own practice, because it tells you whether you lose points to correctness, to boundaries, or to scale — three different problems with three different fixes. It also exposes the compounding version of the classic failure: attempting the problems in the order given and timing out on the hardest leaves no minutes for self-testing anywhere, so the earlier problems come back with boundary failures too.