90 manual decisions in one "automated" run

Ten weeks ago, I ran the first end-to-end batch through my lead pipeline and logged what it actually cost me. I keep coming back to that log because the lesson hasn't changed since.

The initial setup yielded 175 raw leads aggregated from five sources, four filtering phases (a hard pre-filter for competing billing platforms, a scored rubric on subscription signals, a manual review tier, then an email resolution stage). The first version of the runbook estimated 30-45 minutes of my time per batch. The real first batch took about 4 hours.

Where did the time go? Not the scoring. The score ran fine, but 90 ambiguous rows still needed a personal include-or-reject decision, and another 71 auto-rejected rows got a second look because the cut felt too harsh to me (and 37 of them were worth recovering). Around 175 individual judgment calls in total.

Three things the numbers taught me:

Source quality beats volume: Persona-based sourcing ("works at a SaaS company") gave a 17% first-pass include rate. Filtering by detected Stripe billing gave 45%. Revenue-verified founder communities: 67%. The sources that expose the buying signal outperform the ones that expose the demographic. One batch of data, so early signal, not a conclusion.

Scoring shortcuts backfire quietly: I auto-granted points to rows from Stripe-verified sources. That correctly recovered 9 SPA-rendered sites my scanner had missed, and it also promoted 18 agencies and capital firms that use Stripe to bill clients. "Uses Stripe" and "is a subscription SaaS" look identical to pattern matching. Only reading the value prop separates them.

Automation moves the bottleneck, but it doesn't remove it: The bottleneck was never list-building, it's eyeballing. Ten weeks later, the per-batch eyeball cost is roughly the same. What I'm trying now is pointing deep-research agents at the shortlisted leads instead of pre-filtering harder, and honestly, it looks promising.

For anyone doing founder-led outbound, how do you build a filter that pre-qualifies for the buying signal instead of the persona that correlates with it? If you've cracked that, I want to hear about it.