A fact register: stopping an LLM inventing your statistics

My website said I had delivered over a hundred and fifty projects. It said I had a satisfaction rate in the high nineties. It said I had over a decade of experience and had worked with `hundreds of businesses across various industries`. None of that had happened. I am one engineer with one paying client. I did not write those sentences and I did not catch them by reading. A model generated the copy, I read it, it sounded like a website, and it went live. I found them weeks later while auditing something unrelated — four separate fabrications, sitting on the page whose entire job is making a stranger trust me. ## Why reading does not catch this Because the invented numbers are the ones that read best. A model writing marketing copy is doing exactly what it was asked. Marketing copy contains confident figures — that is the genre. Asked to produce copy for a software consultancy, a plausible project count and a plausible satisfaction percentage are not errors in its output. They are the shape of the thing. And when you read it back, a number does not announce itself as unsourced. There is no visual difference between a figure you verified last month and a figure that was invented forty seconds ago. Both are just digits in a sentence you are skimming for tone. That is the whole problem. The failure is invisible at exactly the moment you are supposed to catch it. ## Why a better prompt does not fix it My first instinct was to add "do not invent statistics" to the system prompt. That is a wish. It works most of the time, which is the worst possible property — frequent enough to build confidence, unreliable enough to eventually ship. And you have no way to know which draft is the one where it did not hold, because the failure looks identical to the success. Then I tried the obvious next thing: ask a model to check the copy. *Is anything here fabricated?* This does not work either, and the reason is worth stating plainly. It is the same generator, asked a different question. It has no privileged access to whether `150 projects` is true — it never did. It will read a plausible number, find it plausible, and approve it. You have added a second unmeasured system on top of the first one and called it verification. ## What actually works: an allowlist The fix is boring and mechanical. **A register of every figure the copy is allowed to contain**, each with a source precise enough to re-verify: ```toml [[fact]] id = "haslett-invoiced" scope = "public" value = "$216K" claim = "invoiced through software I wrote and still operate" source = "Haslett Handyman master CRM, cumulative invoiced total" stale_after = "2026-12-31" [[fact]] id = "audit-sites" scope = "public" value = "456" claim = "Michigan wellness websites audited" source = "places-leads sweep + site-audit, Michigan wellness vertical, 2026-08" ``` **And a checker with no model call in it.** It extracts every number from the prose, normalises it, and fails on anything not in the register. Money, percentages, counts of three digits or more, multipliers, and written-out quantities like `a third`. ```python NUMBER = re.compile(r"""(?