What 100B+ Claude tokens actually look like inside a tiny company

The expensive part of "confidently wrong" isn't the review, it's that the next three runs already built on top of it before you woke up. Unwinding costs an hour at step one and a day at step eight, so what's worth bounding is what an overnight run is allowed to depend on, not just what it's allowed to do.

When a morning review rejects something, do you have a way to find what already consumed it?

Token volume shows activity, not leverage. I would publish the operating metrics beside it: cost per accepted deliverable, cycle time, human review minutes, defect escape rate, rollback rate, and percentage of work discarded. The system is winning only if useful throughput rises faster than review burden and correction cost. Otherwise the agents have simply created a larger queue.

The part that stands out to me is not the token count itself, but the shift in where the bottleneck moves. Once agents can keep producing work in parallel, the scarce resource becomes review quality: deciding what is safe to ship, what needs another pass, and what should be discarded entirely.

For a tiny team, I think the operating system around the agents matters as much as the agents themselves: clear task boundaries, visible state, automated checks where possible, and a hard rule that “confident output” is not the same as validated output. Otherwise the leverage turns into a larger review queue instead of actual throughput.

The "harder to switch off when work can continue without you" line is the one that actually sticks with me, more than the token count.

Most solo/small-team framing around this focuses on the productivity side more output, less grind. The always-on tension you're naming is a different, less talked-about cost. Curious how you've actually drawn the line on what's safe to leave running overnight versus what needs a check-in before it goes further. Is that a gut-feel thing at this point, or do you have anything closer to an actual rule for it?

The most valuable takeaway here isn’t the 100B+ token number—it’s the change in where human effort goes. At this level, AI becomes less about “getting an answer” and more about running an execution layer around the business.

The distinction between automation and autonomy is especially important. Agents can keep working around the clock, but humans still have to provide context, evaluate outcomes, and decide when something is actually worth shipping. The “confidently wrong” problem makes that last part impossible to ignore.

I also like the honesty around the trade-off: more leverage and flexibility, but also a real risk of creating a never-ending review queue. That seems like one of the most important operational challenges for small teams adopting agent-first workflows.

Would be interesting to see how your process evolves once the volume of agent output becomes larger than what a human can realistically review each morning.

Great insights on how automation and agent-based workflows are changing the way small teams operate. I especially found the balance between automated execution and human judgment interesting. The same approach can also be useful when managing streaming projects, where content, technical tasks, and distribution often need to work together efficiently.

I keep humans at policy boundaries and agents at event boundaries. A clear event like passing tests can move a task; deciding that the result is actually safe to ship is still a human call. The part I did not expect was needing attribution as much as permission: once several agents act, knowing exactly which one changed the shared state is what makes review possible. Do you give each workflow its own identity, or is all of that agent traffic effectively one actor?

I’m building a SaaS on my own (also a tiny team of one), and I’ve experienced a similar shift in my role — I spend far less time writing code manually and far more time reviewing what the AI produces, fixing context drift, and deciding what’s worth building next.

The most unexpected lesson was that the AI can be confidently wrong in ways that look correct at first glance, and that’s where human judgment becomes the bottleneck.

I’m curious: how do you handle error detection at scale? Do you have automated tests that run after agent code changes, or do you rely entirely on manual review?

The most interesting part isn’t the 100B tokens, but how the human role changes. Once agents can handle more of the execution, judgment and review become the real bottlenecks. The “confidently wrong” problem is especially important — more automation only helps if you have a reliable way to catch those mistakes. Really interesting look at how a tiny team operates at this scale.

The morning triage point really lands. We've seen the same shift: the bottleneck is no longer generation speed but review bandwidth. Capping overnight agent work to what one person can honestly re-read feels like the right guardrail.

great

The number is less interesting than the morning triage problem you describe. When agents work overnight, the scarce resource is no longer output, it is how many diffs one person can actually own before they start rubber stamping. We hit the same wall: once concurrent tasks exceed what fits in working memory, review turns into heuristics, and that is exactly when the confidently wrong stuff lands. Capping overnight work to what one person can honestly re-read the next morning has been a better lever than adding more agents.

The part that resonates most isn't the token count — it's the morning triage problem. When agents run overnight, the bottleneck shifts from "can we produce output" to "can we review output faster than we produce it." That's a different muscle entirely.

One thing that surprised us: the review cost isn't linear with output volume. After a certain point, you stop being able to hold enough context in your head to judge individual diffs, and you start relying on heuristics — "this file changed, skip it" or "this looks roughly right, ship it." That's when the confidently-wrong stuff slips through. We found that capping the number of concurrent overnight tasks to what one person can actually review in 30 minutes the next morning made the system more reliable than adding more agents. Counterintuitive but true.

RogierB and to21as are both aimed at the file-read share, and both at making it smaller — an index so each lookup costs less, a written record so the next session doesn't rederive what the last one already worked out. That's the size question. A breakdown answers a second one, and that one took me a while to notice.

I had a side project frozen for months and never paused its scheduled jobs. They kept running daily against a metered API. The bill that finally tripped my spend cap was about $6.50 — the total was too small to mean anything. The shape of it wasn't: my code called the cheap tier only, and the biggest line item was the most expensive model. Reading back through my own dispatcher afterwards, the best-supported explanation was that error bursts triggered its automatic fallback, and that the fallback pointed up rather than down — escalating hardest at the exact moment the API was failing.

Your table is tokens by category and mine was dollars by model, so the numbers don't transfer. The second read does: once for size, once for any line you can't account for. "Browser automation is heavy" is a cost question. "Browser automation is heavy and I don't remember writing much that browses" is a different one, and no total will ever raise it.

Two rules I kept: a fallback may go cheaper or it may stop, but it never points up. And every scheduled job gets an expiry date now — mine outlived its project by months, and the scheduler had no opinion about that.

People often talk about token volume like it's a vanity metric, but what 16.9B tokens/month really shows here is the massive ratio of background context gathering required to produce a single accurate output.

I keep the human at the point where a mistake wouldn't announce itself.

Agents write most of my code, and that's fine — wrong code fails loudly. Tests break, requests error, something goes red. But my product also depends on rules extracted from published medical papers, and I do that part by hand, every one, checked against the source PDF. A fabricated guideline reads exactly like a real one. No test catches it. There's no error message for plausible.

So the line for me isn't task type or risk level. It's whether the failure is self-announcing. Delegate everything that fails loudly. Keep everything that fails quietly and sounds right.

16.9B tokens in 30 days is a number I had to read twice. The 63,996 model turns make it concrete—you're not prompting, you're reviewing an assembly line.

The "confidently wrong implementation" detail resonates. I've had mornings where I undo what an agent built overnight. That trade-off between flexibility and the always-running system is real.

How do you keep yourself from checking in on the system during family time? Is that a discipline thing, or have you found a specific boundary that works?

The "harder to switch off" trade-off matches my experience. Once agents can run overnight, the bottleneck becomes morning triage: reading diffs to decide what to keep and what to unwind. One thing that helped us was having agents write a short decision log into the repo as they work, so the morning review is a page of notes instead of a raw transcript. How do you keep the unwinding cheap at your volume, small commits or just reading everything?

someone upthread asked whether automated checks catch the bad outputs before a human sees them. we tried exactly that and measured it, so here are the numbers instead of my impression of them.

the setup is a pipeline that reads public communities every day and pulls labelled evidence out of them. the obvious next thing to delegate was QA: second model reviews the first model's labels and flags the wrong ones. when we finally calibrated it against 42 rows a human had already ruled on, agreement was 69%. a function that ignores the input entirely and answers "correct" every time would have scored 71.4%. the reviewer was worse than a constant.

it wasn't uniformly bad though. sorted by the reviewer's own stated confidence, its top bucket was 6 for 6 and the one below it fell to roughly 30%. small n and i won't pretend otherwise, but the shape was clear enough to build on: it can raise a hand, it can never clear a row. a "looks fine" from something running at 30% is worse than no signal, because you stop looking.

the less flattering half of that: of the 13 disagreements, 8 landed on the same label, and the cause turned out to be that our own spec defined that label three different ways in three different places. the model wasn't confused, we were. i'd guess a fair share of what gets filed as agent unreliability is really that, and the only way we found it was hand-judging rows.

so on your actual question — what stays human is anything that decides. scoring and state transitions here are plain deterministic code, and the model only reads, extracts and matches. not because a model couldn't do the scoring, but because when a number looks wrong six weeks later i need a line i can point at rather than a prompt i can re-run.

Really interesting to see how a small company can use AI at this scale. Do you think this level of automation creates bigger opportunities for non-technical founders too?

The shift from manual execution to agent-driven workflows is really interesting, especially at this scale.
It shows how AI can handle context gathering and execution while humans stay focused on judgment and strategy.
A similar automation mindset can even help with tasks like a Pinterest Video Downloader, where repetitive work can be streamlined efficiently.
The key is keeping the right human-in-the-loop checkpoints to catch confidently wrong outputs.

The shift from doing every task manually to focusing more on context, judgment, and direction really resonates. I think the same approach can work beyond software development too, especially for researching and evaluating online platforms like digital earning opportunities. The key seems to be having agents handle the repetitive research while humans still verify the important decisions.

The reading-and-editing share being your biggest category is the part I'd dig into, because it's the one line item that's mostly avoidable.

Most of it isn't reading, it's re-reading. A session rediscovers the same five facts about a codebase that three previous sessions already established, because the conclusions lived in a transcript and the transcript is gone. An index makes each lookup cheaper but doesn't reduce how many happen.

Two things that actually moved it for me. Searching is delegated to sub-agents, so the file dumps and the dead ends stay in the sub-agent and only the conclusion comes back to the main thread. And every decision, gotcha and current-state note gets written to a file in the repo the moment it's reached, so the next session opens with the answer instead of deriving it. The rule I hold myself to is that the conversation is disposable and the written record is the real one, and if something only exists in the conversation it isn't finished.

On your actual question: what stayed human longest for me isn't a category of decision, it's anything where no independent check exists. Where a separate tool can say yes or no, I let it run. Where the only available verdict is the agent's own summary of its own work, I read it myself, because a confident summary and a correct one are indistinguishable from the inside.

That’s a pretty interesting shift in how a small team can operate. The part that stands out to me isn’t even the 100B+ token number, but how little of the usage is direct messaging compared with file work, browser automation, and terminal commands.

It also highlights the less glamorous side of agentic workflows. Getting an agent to complete a task is one thing; knowing when its output is actually trustworthy still requires human judgment. I imagine deciding what not to delegate becomes just as important as deciding what to delegate.

Curious how you handle review at that scale—do you have automated checks catching most of the bad outputs before they reach a human, or is the human review still fairly hands-on?

The 'confidently wrong implementation' line is the one that took me longest to handle gracefully. Agents are unusually good at being wrong with high confidence, and the failure mode isn't usually random — it's that they optimized for something that sounded like your intent but wasn't. The signal that matters isn't output quality, it's how far wrong they get before anything surfaces it.

For us, the things that stayed human-in-the-loop longest were anything where the cost of being confidently wrong was asymmetric: customer-facing messaging, any decision that required saying no to something that looked good on paper, and anything where being wrong meant spending two weeks going the right direction for the wrong reasons.

The shift you describe — direction + context + review — is right, but I'd add one more: pattern recognition on the error classes. After you've seen the same type of wrong implementation a few times, you can catch it faster at the context stage rather than the review stage. That feedback loop changes the economics.

The system looks great I really appreciate the work that's gone into it Jan.

We're entering an era where AI agents are becoming the new workforce, handling tasks autonomously while keeping humans in the loop to review, validate, and ensure quality. Businesses are rapidly adopting AI agents, specialized skills, and MCPs to automate workflows, significantly reducing manual effort and cutting task completion time by as much as 90%. This shift is transforming how work gets done, allowing teams to focus on higher-value decision making instead of repetitive operations.

The "tries to" qualifier is the part most agent demos leave out. Once agents run overnight, the bottleneck isn't producing more work. It's proving that something changed in the outside world. The most reliable pattern I've found is to define an explicit output contract, then run a separate verification step. If the evidence is missing, the run should fail. Otherwise, a confidently wrong result can sit there looking like progress until morning.

The token count is a vanity metric, the real story is the shift you describe: direction, context, review, correcting course. At SocialPost.ai our AI PM spends most of her time doing exactly that, catching wrong turns rather than writing prompts. The one place we still keep a hard human gate is anything that touches a customer's published content or billing, agents draft it, a person approves the send.

Really impressive setup, Jan. Congrats on building this with such a small team, and thanks for sharing the real trade-offs instead of just the upside.

I think the biggest shift here is that the bottleneck moves from execution to judgment. Research, drafting, data movement, and repetitive tasks can be heavily delegated, but human checkpoints still matter around irreversible actions and customer-facing decisions.

At this scale, designing the right review system probably becomes just as important as the agents themselves.

Most forget when you yourself build you do a thousand checks. Why should it be different with AI? Plan, build, check, recheck, test, rinse and repeat.

I see reading and editing is your main token usage. I build a little tool for my repo's that makes a database with all references so AI can find things quick without going through thousands of lines. It cut my token usage for reading significantly. Might be usefull.

Mostly delegated the drafting grind - first-pass research, structuring, turning it into an actual finished deliverable. Kept myself in the loop on anything that goes out publicly or touches a live account: final review before publishing, pricing/positioning calls, that kind of thing. The "confidently wrong" risk you mention is real, especially the first few times you let it run further than you're used to - worth checking its output against reality before it snowballs.

The shift from doing the work to reviewing and correcting agent output seems like the more interesting change here.

I’m curious where you’ve found the boundary is hardest to define — which decisions can safely run without you, and which still require human judgment?

The shift from doing to reviewing and correcting is the real change. Most people underestimate how much judgment is still needed — agents multiply your output but they don't replace the decisions about what's worth building.