I doubled my dataset. The documentation debt problem got worse, not better.

im currently working on DocsKoala. It drafts docs from merged GitHub PRs. Before pushing further on automation, i wanted a real number for something I kept assuming:

that documentation debt is mostly a lag problem, that devs get to it eventually. Ran an audit instead of trusting that assumption. This is the second pass at it, dataset roughly doubled since the first.

the first run had 15,364 merged PRs across 256 public repos, out of which 3,741 happened to be customer facing. on the second run, using same methodology just the number of repos cranked up, 29,917 merged PRs across 514 public repos among which 7,654 were customer-facing.

i classified each pr as, customer facing feature, internal, bug fix, not draftable/unclear. for the customer facing set,i tracked whether it got a docs commit, and how many days after merge.

the direction held, and even got slightly worse with more data, which is the opposite of what you'd expect from a fluke in a small sample.

The undocumented rate went from 17.8% to 20.3%. median delay went from 0.3 days to 0.5. p90 delay went from 8 days to 11. automatable share stayed roughly flat, 36.4% down to 35.2%, so the split itself is stable, it's the debt piling up around it that's growing.

so, heres the shape of it in the current run. Median delay for PRs that did get documented - 0.5 days, same day or next, basically. p90 is 11 days - one in ten documented PRs sat live and undocumented for over a week and a half, and 20.3% of customer-facing PRs never got a docs commit at all.

Of those, 6.3% of all customer-facing PRs (482 of them) are past the point I count as "never," no update in 30+ days after merge, conclusively missed, not just slow.

so it's not one problem, its a fast lane and a graveyard with nothing much in between. the PRs that get written up get written up almost immediately and The ones that dont, dont, and that gap is widening as the dataset grows, not shrinking.

i alsochecked which of the undocumented ones couldve been drafted from the diff and title alone, without extra context needed from the devs at all, 35.2% of all customer-facing PRs were automatable that way i.e. 2,694 of them,it went up from 1,362 in the first run. manually writing docs for just that automatable slice, at 20 to 40 min each, is 898 to 1,796 hours, $67,350 to $134,700 across these 514 repos, up from $34,050 to $68,100 in the first run.

And thats exactly the number that shapes where DocsKoala draws the line. The 35% with enough signal in the diff gets auto-drafted the same day, closing the gap before it gets to becomes a graveyard PR. and the rest gets flagged for a human, because guessing context that isn't in the diff produces worse docs than no docs.

For full breakdown, methodology, and live numbers: docskoala.com/research/state-of-documentation-debt