8 min read · 15 July 2025

Your 99.9% is fifty people

Fifty people is not a rounding error. Move 50,000 records at 99.9% accuracy and about fifty of them come out wrong. Each of those fifty is somebody getting the wrong bill, the wrong grade or the wrong seat.

The percentage makes it onto the slide. The fifty never do.

Say you're moving a university's student records from an old system to a new one. Fifty thousand rows: names, IDs, courses, grades, fee balances. The cutover is Friday, and somebody has to sign a line that says the data came across correctly.

There are two ways to earn that signature.

The cheap way: pull 1,000 rows at random, open each one next to the old system and read. Suppose fifty bad rows are hiding in the table. A thousand rows is 2% of it, so on average you'll catch one. One bad row in a thousand, so you write 99.9%, and you're not even lying.

That "on average" is the trick. I ran the numbers. About 36% of the time, your thousand rows contain no bad ones at all. About 37% of the time you find one, and about 19% of the time you find two. The same table, checked the same honest way, comes back as 100%, 99.9% or 99.8% depending on which rows the dice picked. More than a third of the time, the spot check says perfect.

A hypothetical table of 50,000 rows and a random sample of 1,000, drawn as a handful of cells (not to scale). The raised cells are the rows somebody read; the orange ones are wrong, and the sample missed every one of them, which a sample this size does about one time in three.

The expensive way: compare every row in the new system against its row in the old one, field by field. That prints the same 99.9%. Then it prints fifty lines underneath it, each one a student ID, a field, the old value and the new value.

Same number. Only one of them comes with names.

A percentage with no method behind it is decoration. I call it decorative accuracy. It has no denominator you can check, no list of the rows that failed, and no definition of what "failed" means. It looks rigorous because it carries a decimal point. You'll find it on migration sign-offs, vendor data sheets, and model evaluation slides.

Round 99.9% to zero decimal places and it prints 100%.

Fifty people, rounded to nobody.

Even an honest sample certifies less than people think. Statisticians have a neat rule for the case where you look and find nothing, the rule of three: if you check n items at random and see zero problems, you can say with 95% confidence that the true problem rate is below about 3/n. It comes from a 1983 paper in JAMA with the best title in medicine, "If Nothing Goes Wrong, Is Everything All Right?". Zero bad rows in a sample of a thousand puts the error rate under about 0.3%, at that 95% confidence. On 50,000 rows, the honest reading is "no more than about 150 wrong." That is all a clean spot check can promise.

And even a big, perfect sample answers the wrong question. A sample estimates how many. Outside the rows it read, it never tells you which. That's fine when the decision is "do we ship or not." It's useless when the job is "fix every student whose grade got mangled," and for a records migration that is always the job.

You can't send an apology to a percentage.

The defense arrives as a proverb: "Perfect is the enemy of good."

Nobody asked for perfect. I'm asking for the list.

Fifty known-wrong rows on a list is a fine outcome. You fix them on Monday, or you decide three of them don't matter and say so in writing. Fifty unknown-wrong rows is a time bomb with a very boring fuse: a fee statement in March, a transcript request in June, a graduation audit two years later, each one landing on someone who has no idea a migration ever happened.

Uptime is the one place where I'll take a percentage, and 99.9% is a fair target there. It allows 8.76 hours of downtime a year, and those hours are interchangeable enough that a team can budget them and spend them on a risky deploy. Records don't average out like that. The fiftieth student's record isn't 99.9% right. It's wrong, and so is everything downstream that reads it.

Try telling the man at an empty baggage carousel that 99.9% of bags arrived.

The worst version of this has nothing to do with migrations. A computer prints a total, a court treats it as evidence, and nobody asks for the rows behind it.

For years the UK Post Office ran its branches on an accounting system called Horizon. When Horizon said a branch was short, the Post Office believed Horizon. The first volume of the public inquiry's final report, published on July 8, 2025, says that between 2000 and the autumn of 2013 the Post Office prosecuted postmasters in England and Wales "in reliance upon accounting data produced by Horizon," and asserted "that the data produced by Horizon was wholly reliable."

The chair, Sir Wyn Williams, found that a number of Post Office employees "knew or, at the very least, should have known that Legacy Horizon was capable of error" (Legacy Horizon is the system's first version), and that the Post Office "maintained the fiction that its data was always accurate." His summary of the result: "many hundreds of people have been convicted, wrongly," and "many thousands" were held responsible "for losses which were illusory, as opposed to real."

The law leaned the same way. In England and Wales, the courts work on a presumption that computers work correctly unless someone shows otherwise. In January 2025 the government asked for evidence on whether to change that rule, and pointed to Horizon as the reason.

A shortfall on a branch screen was a single number. Behind it sat a list of transactions, and somewhere in that list were the ones a bug made up. The people who could have produced the list didn't, and the people accused had no way to get it. A total is the easiest thing in the world to trust, because there's nothing in it to argue with.

Now the engineering version, where the total is a green test run. At 04:09 UTC on July 19, 2024, a CrowdStrike content update crashed about 8.5 million Windows machines, by Microsoft's estimate.

CrowdStrike's root cause analysis comes down to two numbers that nobody compared. A new template type defined 21 input fields. The sensor code that fed it supplied 20. The Content Validator, the piece that exists to stop bad content before it ships, judged the content on the assumption that 21 inputs would be there. And the test content used a wildcard, a match-anything pattern, in the 21st field, so the tests never actually asked for the value that wasn't there.

The validator said pass, which is a percentage with the decimals filed off. A pass names no fields, so nobody saw 21 on one side and 20 on the other. The tests were a sample with a blind spot exactly where the bug lived.

Among the fixes CrowdStrike listed: a check that the number of inputs matches what the content expects, and a runtime bounds check, which stops the code from reading past the end of a list. A compare, in other words.

A compare is easy code, a loop over the IDs with an equality test inside it:

diffs = []
for key, old in source.items():
    new = target.get(key)
    for field, old_value in old.items():
        new_value = new.get(field) if new else None
        if not same(field, old_value, new_value):
            diffs.append((key, field, old_value, new_value))

Run it the other way too, for rows and fields that showed up in the target from nowhere. A first-year student writes this in an afternoon. Its output is the thing the percentage threw away, and it's the same move as making a model show its work: you get something a person can check, row by row.

The hard part is same. Is "Smith " the same as "Smith"? Is 01/05/2024 in January or in May? Does 1200.5 equal "1,200.50"? Is an empty string a missing value? Every one of those is a decision, and it belongs in a meeting before the cutover, not in a hallway after a student's fee statement shows the wrong number.

That argument is the real work. The percentage is what you report when you'd rather skip it.

When the next sign-off says 99.9%, skip the congratulations.

Which fifty?