PromptProbe

Building the product isn't the hard part.

Finding people who already have the problem is.

I spent weeks thinking, "If I make PromptProbe better, users will come."

They didn't.

Then I started replying to founders on X and Indie Hackers, asking questions instead of pitching.

Those conversations taught me more than hours of building.

I'm starting to believe early-stage distribution is less about marketing and more about having useful conversations with the right people.

What's one lesson your startup taught you this week?

Over the past week I realized building the product is only half the job.

I've started spending as much time talking to builders as I do writing code. The feedback has already changed how I'm thinking about the next features.

Still early, but the conversations are proving just as valuable as the commits.

What's one change your users made you build that wasn't on your roadmap?

This week wasn't about adding more features.

It was about learning what actually matters to people.

Here's what changed based on conversations with founders:

• I stopped thinking PromptProbe was just another prompt playground.

• The focus shifted toward measuring prompt reliability, not just generating outputs.

• I realized founders don't want more dashboards—they want confidence that a prompt will behave consistently before shipping.

• Instead of chasing more features, I'm spending more time talking to builders on X, GitHub and Indie Hackers.

One lesson that surprised me:

Building is only half the job.

Explaining the problem, getting feedback, and improving from real conversations is where the product starts becoming useful.

Next week I'll focus on:

More founder interviews.

Improving the reliability reports.

Shipping changes based on feedback instead of assumptions.

If you're building with LLMs, what's the hardest part for you right now—prompt creation, evaluation, or reliability?

I'm building PromptProbe to help people test whether their prompts are actually reliable.

The idea is simple: Run the same prompt multiple times, compare the outputs, and see how consistent it really is.

My question is:

If you work with LLMs, what do you struggle to measure today that existing AI tools don't show you?

I'm trying to avoid building features nobody needs, so I'd love brutally honest feedback.

Try here - https://www.promptprobe.tech/

Building PromptProbe has changed how I think about AI.

I used to ask:
"Did the prompt work?"

Now I ask:
"Would I trust this prompt if it were running in production every day?"

Those are very different questions.

Still learning. Still talking to builders. Still shipping.

I kept running into the same problem while building AI workflows: a prompt would work once, then behave differently on the next run.

Existing evals helped, but I wanted a faster way to spot output drift before shipping changes to production.

So I built PromptProbe. It runs the same prompt multiple times, compares outputs, and highlights where responses diverge.

I'm still figuring out where it fits in the AI testing stack, but the conversations so far have been surprisingly insightful.