Building in the AI reliability space — looking for feedback
Hey everyone 👋
I'm exploring a problem around AI customer support: teams can deploy an AI chatbot successfully, but understanding its quality across real customer conversations can become difficult as usage grows.
I've been talking to AI founders and collecting different perspectives on how they currently evaluate their systems.
My next step is to build a very small prototype around one specific problem I've identified from these conversations.
I'm intentionally keeping the first version simple rather than trying to build a huge platform.
I'd love to hear from other Indie Hackers:
If you were building an AI customer-support product, what would you want an evaluation tool to tell you every day?
Would really appreciate your thoughts, especially from people who have built or deployed AI products.
@Veriai The next step is turning that outline into a test you can hand to three design partners. I’ll help you define the sample, failure taxonomy, severity weighting, owner workflow, replay test, and the continue-or-kill thresholds. The focused 20-minute session is $75. Book the AI and operations advisory call here: https://calendly.com/dontae-threeum-nsuo/advisor-call-20-min
Thanks for the offer! I’m currently doing early validation through conversations with AI teams, so I’m keeping the research focused on understanding the problem from their side first.
@Veriai I would run the first week as a proof of action, not a dashboard test. Day 1: collect 100 real conversations from three design partners and agree on wrong answer, false confidence, and failed handoff, with severity. Day 2: label the baseline and attach business impact. Day 3: deliver a ranked queue with an owner and proposed fix. Day 4: have each partner change one prompt, handoff rule, or knowledge article. Day 5: replay the same set and report the change in high-severity failures and time to resolution. The success gate is that each partner reviews the queue, makes a change, verifies improvement, and agrees to continue the pilot. If you have access to real conversations, I can help turn this into a test plan and review the results with you in a focused 20-minute call.
The 5-day proof-of-action approach is interesting. Starting with real conversations, labeling failures, making a fix, and then replaying the same cases to measure improvement is very close to the kind of pilot I’m thinking about for Veri AI.
The useful daily output isn’t a score. It’s a ranked action queue: the conversation, business impact, likely root cause, owner, and whether the fix survived replay. I’d start with three buckets, wrong answer, false confidence, and failed handoff, then show expected cost per 100 conversations. I advise AI operators on turning signals like this into an execution loop. If useful, I can outline the first-week test I’d run with three design partners.
I really appreciate the offer. I'd definitely be interested in hearing your thoughts on how you'd approach the first-week test. Thanks for offering to share it!
The challenge of understanding AI quality as usage grows is definitely real. It’s easy to get buried in metrics, but the truly hard part, in my experience, is translating those into actionable improvements. When thinking about an evaluation tool, a key thing for me would be not just 'what' the AI did, but 'why' it did it - and, more importantly, 'what specifically needs to be tweaked' to get a different outcome next time. For instance, pinpointing specific conversation segments where the AI consistently misunderstood intent, or where it provided an unhelpful response, rather than just an overall "accuracy" score. That kind of granular, prescriptive feedback is gold.
Completely agree. A score without context isn't very actionable. The point about understanding the “why” is especially useful — thanks for sharing this.
The first thing I'd want is a risk-weighted failure queue, not one aggregate score. Tag each conversation by customer impact, model confidence, evidence support, and whether a human later reversed it. Then rank by expected loss: frequency multiplied by severity multiplied by detectability. A 1% hallucination that creates refunds should outrank 10% awkward phrasing.
For a small prototype, replay the same held-out conversations before and after every prompt, model, or knowledge-base change and show the regression delta. The kill condition is simple: if founders do not change a prompt, handoff rule, or knowledge article after reviewing the daily queue, the evaluation is reporting rather than a product workflow.
This is a really useful way to think about it. I especially like the distinction between reporting failures and actually helping teams prioritize what needs to be fixed. The example of a low-frequency but high-impact hallucination outranking frequent awkward phrasing makes a lot of sense. Thanks for sharing this — definitely something I'm going to keep in mind.
I’d separate the daily evaluation into three layers: outcome, safety, and drift. Outcome asks whether the customer’s issue was resolved; safety asks whether the answer should have been refused or handed to a human; drift asks whether those rates changed after a prompt, model, or knowledge-base update. The dashboard matters less than turning a bad conversation into a prioritized fix with an owner and a before/after check. A useful first prototype could sample ten conversations per day, let a reviewer label those three dimensions, and show only the regressions. Which signal are founders currently willing to review every morning: unresolved intents, unsafe confidence, or quality changes after deploys?
This is a really useful framework. I hadn't thought about outcome, safety, and drift as three separate layers. Definitely giving me something to think about — thanks for sharing it!
honestly for me it'd be less about raw metrics and more about "would this response have made me trust the company less." things like, did it actually solve the person's problem or just sound confident while dodging it, did it know when to hand off to a human instead of guessing, and consistency over time (not just one good convo, but not drifting into bad answers after a few hundred conversations). the trust piece is the hard one to quantify but it's usually what actually gets escalated to a founder's inbox lol. curious what you're hearing most from the AI founders you've talked to so far, is it mostly accuracy issues or more the "doesn't know when to stop" problem?
That's a really interesting distinction. The “knowing when to stop” point is something I hadn't thought about enough. Really appreciate you sharing this perspective.
glad that landed, it's the one that's easy to overlook because it doesn't show up in accuracy benchmarks at all. still interested to hear what you're seeing across the founders you've talked to, if it's more "the AI is wrong" or more "the AI doesn't know it's wrong and just keeps going"
That’s actually a distinction I’m starting to investigate. There’s a big difference between “the AI was wrong” and “the AI was wrong but had no reliable signal that it was wrong.” I’m asking founders about this specifically to see which pattern appears more often.
that split matches something we've run into building in FX. an AI can quote a wrong exchange rate confidently and there's nothing in the output itself that flags it, you only catch it if you're checking against an external source of truth. so we ended up building the check as a separate verification step instead of trying to get the model itself to "know" it's wrong. wondering if that pattern holds for the founders you're talking to too, external verification vs the model self-flagging
What you're describing sounds less like an AI reliability problem and more like an evaluation problem that only becomes obvious once teams have enough real conversations. I'd be curious whether the prototype is centered on finding failures, measuring quality over time, or helping teams decide what to fix first.
That's a good way to frame it. I think the difference between simply finding failures and actually making them useful is really important. Appreciate the insight!