Exploring a Problem in AI Customer Support
Hey everyone! 👋
I'm currently researching a problem in AI customer support that I keep hearing about from founders and teams building AI chatbots.
AI support can handle thousands of conversations, but one question keeps coming up:
How do you know when the AI starts giving bad answers in real customer conversations?
I'm currently talking to people in the space and trying to understand how teams monitor AI quality after deployment.
I'm also building a small early version around this problem and opening a waitlist for people who want to follow along.
I'd love to hear from other founders:
What's the biggest challenge you've faced with AI reliability or quality after deployment?
Would love to learn from the Indie Hackers community. 🚀
The post-deployment part is what makes this interesting.
In the conversations you've had so far, are teams already doing something manually to catch bad answers, or are they mostly acknowledging the risk without actively trying to solve it yet?
Good question. Right now it's mostly the second — teams are aware of the risk but don't have a structured way to catch it yet. A few do spot-checks manually, but nothing consistent or scalable. That gap is exactly what I'm trying to solve.
That gap between knowing the risk and having a consistent process for it is interesting. I’d be curious what you learn as you talk to more teams about how they handle it today.
The question that came to my mind wasn't just "How do you know when the AI gives a bad answer?" but "How do you know when people are starting to lose confidence in it?"
Those aren't always the same moment. A system can be technically correct most of the time, but if people stop trusting its reasoning, the quality problem has already become a decision-making problem.
To me, that makes observability about more than outputs. It's also about understanding when confidence begins to drift.
This is a really sharp distinction, thank you. You're right that "technically correct" and "trusted" aren't the same thing — I've been mostly focused on catching bad outputs, but the confidence-drift angle is something I hadn't framed that clearly. Curious how you'd even start measuring that — is it more about consistency over time, or how the AI handles uncertainty in its own answers?
I think it’s probably both, but I’d add a third signal: what people do when they don’t trust the answer.
Consistency over time can reveal drift, and how the system handles uncertainty tells you something about whether it knows its own limits. But I’d also watch for behavioral signals - users rephrasing the same question, asking for a human, or agents increasingly overriding the AI.
I wouldn't expect one confidence score to capture it. I'd probably look for several small signals that start moving together. That seems more useful than trying to define "trust" as a single metric.
This is a really interesting angle. I hadn’t thought enough about user behavior as a reliability signal. Rephrasing, escalation, and human overrides could potentially reveal failures that the model’s own evaluation misses. I’m going to explore this further.