Stop building AI that acts like an oracle. Build a scientific partner instead.
I got a piece of feedback today that completely changed how I am architecting the AI features in my current project.
Like a lot of us building right now, I'm working on a tool that feeds application data to an LLM to generate plain-English insights. In my case, it’s a lightweight behavioral analytics tool that watches user drop-offs and tries to explain why they happened.
Initially, I thought the goal was to make the AI sound confident.
Data goes in ➔ AI spots a 10-second idle time on a checkout button ➔ AI tells the founder, "Your button contrast is too low."
But a developer in the community pointed out a massive flaw in this UX: It masks a hypothesis as a measured fact.
When we give an AI a "confidence score" and a text box, we create an illusion of precision. The drop-off is an objective fact. The reason is just a synthesized guess. If the AI is wrong (and it will be), the founder loses trust in the entire platform.
So, I’m ripping out the standard "AI Output" text box and replacing it with a structured Scientific Method UI.
Going forward, every AI-generated insight in the dashboard will be physically separated into three distinct blocks:
-
The Evidence (Immutable)
What the DOM/database actually saw.
Example: "84 sessions idled on #password-input for >8 seconds before exiting." -
The Inference (Mutable)
What the AI suspects is the friction.
Example: "Hypothesis: Password requirements are hidden until validation fails." -
The Falsification Experiment (Actionable)
How to prove the AI wrong.
Example: "Display password rules in plain text below the input field. If idle time does not drop below 3s, this hypothesis is invalid."
We also changed how "dismissing" an alert works. Instead of just hiding a notification, dismissing an AI hypothesis now acts as a deployment state. The system logs it as: "Archived. I will silence this hypothesis, but I will continue watching this element. I will only wake this alert back up if I observe materially new evidence (e.g., a >15% shift in baseline drop-off)."
We need to stop building AI wrappers that pretend to have all the answers. The real value isn't in generating a perfect guess; it’s in helping developers separate the signal from the noise so they can run better experiments.
How are you all handling AI confidence in your own apps? Are you letting the LLM speak authoritatively, or are you fencing it in?
Separating evidence from inference is a good start, but I think the most useful next step is to turn each hypothesis into a verifiable A/B test.
The AI should define the control and variant, target audience, primary metric, test duration, minimum sample size, and success threshold. It should also state what result would disprove its hypothesis.
That way, the AI is not asking users to trust its explanation. It is giving them a practical way to test whether the explanation is actually true.
The experiment block still needs a stopping rule. Show password rules and see if idle time drops is incomplete unless it states sample size, window, and what counts as failure. Otherwise the AI has moved from confident diagnosis to an unfalsifiable suggestion. Put those three fields in the UI.