The manufacturer example is the one that got me. 0.4% populated is exactly the kind of column that can quietly produce a very convincing nonsense chart.
Feels like FlightFinder could be pulling in a lot more organic traffic than it is right now. Especially with 283K occurrences and that level of provenance detail while you're preparing the public release.
Have you found any other field where the coverage story completely changed what you thought you had?
Yeah, that column is why I now gate every aggregate on a minimum coverage before it can render. At 0.4% manufacturer doesn't produce a chart at all in the current build — below the threshold it shows as a labeled gap, not a bar. Same failure mode bit me earlier on a different field: an 8%-populated column was quietly producing "trends" that were pure sampling artifact. Coverage-gating before aggregation was the fix.
The strongest part is the provenance discipline. Discovering that “populated” didn't mean “usable” turns data quality from a hidden technical issue into a product-level trust problem.
That's the shift that took me longest to internalize: "populated" and "usable" are different columns. What helped was making provenance visible per record instead of trusting the merged view — anything that can't be traced to a structured source now carries that status rather than silently averaging into a headline number. It did turn a backend hygiene problem into a user-facing trust decision, which is uncomfortable but correct.
Too early for an honest user read — traffic on those pages is small enough that anything I claimed about "how users respond" would be noise. The internal effect I can report: coverage-gating killed several charts outright, and rebuilding them as labeled gaps took longer than the original feature did. The signal I'm actually waiting on is whether anyone clicks through to the source record when a field is marked unverified. If they don't, the provenance layer is just honesty for my own benefit — fine, but not a product.
That’s a more interesting signal to wait for. The difference between a useful provenance feature and something that’s simply good data hygiene should become clearer once you have that behavior.
Agreed, and the part I want to get right is picking the threshold before the data arrives - otherwise I'll read whatever number comes back as confirmation.
Where I've landed: the provenance layer earns a place in the UI if source-record clicks on unverified fields run meaningfully above the click rate on ordinary outbound references on the same pages. Below that line it's hygiene I keep, not a feature I surface.
The trap I'm watching for in myself is using "nobody clicked" as permission to quietly drop the labels. The labels stay either way - what the behaviour decides is whether they get real estate on the page or just live in the data.
That’s a thoughtful way to frame the test. I’d be interested in continuing the conversation beyond the thread — would you be open to sharing the best email to reach you on?
Happy to keep it here. The thread is the useful part - anyone else building on messy public datasets can read it, which a DM does not give them. If there is a specific question behind it, ask it in the open and I will answer properly.
About
The safety record behind a flight sits in a dozen investigation archives that don't talk to each other. I pulled 283K occurrences from 128 official sources into one lookup, and made every number show its work.
12 Comments
The manufacturer example is the one that got me. 0.4% populated is exactly the kind of column that can quietly produce a very convincing nonsense chart.
Feels like FlightFinder could be pulling in a lot more organic traffic than it is right now. Especially with 283K occurrences and that level of provenance detail while you're preparing the public release.
Have you found any other field where the coverage story completely changed what you thought you had?
The strongest part is the provenance discipline. Discovering that “populated” didn't mean “usable” turns data quality from a hidden technical issue into a product-level trust problem.
That’s an interesting distinction. I’d be curious to see how users respond to that provenance layer as the product develops.
Too early for an honest user read — traffic on those pages is small enough that anything I claimed about "how users respond" would be noise. The internal effect I can report: coverage-gating killed several charts outright, and rebuilding them as labeled gaps took longer than the original feature did. The signal I'm actually waiting on is whether anyone clicks through to the source record when a field is marked unverified. If they don't, the provenance layer is just honesty for my own benefit — fine, but not a product.
That’s a more interesting signal to wait for. The difference between a useful provenance feature and something that’s simply good data hygiene should become clearer once you have that behavior.