Local Waifu

Yesterday I shipped Local Waifu 1.7.0 with more cloud-provider choices, safer cleanup for local processes, and a handful of stability improvements.

Today I shipped 1.7.1.

It contains no new AI feature. No new model. No prettier UI. It is a repair release for a mistake that made the app look broken immediately after an in-app update.

Some Mac users updated successfully, opened the app, and found that image generation no longer worked. The app said the local image engine was not bundled in that build.

In some cases, local chat replies were empty too.

The reason was not the models, the GPU, or a complicated inference bug. The updater had installed an incomplete app.

Here is the embarrassing detail: the normal macOS installer is 245 MB. The broken in-app update was 70 MB.

The update package had been assembled before the local engines were copied into the finished app bundle. So it contained the interface, settings, and most of the app — but not the pieces that actually run local chat and image generation.

That also created a particularly bad dead end in Settings:

• the image engine appeared as “not installed”;

• the button to download a model was hidden because the engine was missing;

• so the user had no obvious path to fix it themselves.

Installing from the .dmg was never affected. Neither were characters, chats, licences, or downloaded models. Windows was unaffected too.

But none of that changes the first experience after a broken update: you open an AI companion app, send a message, and get an empty reply. Or you try image generation and the app tells you that its own engine does not exist.

At that moment, it does not matter how many provider options I added in 1.7.0. The product has already failed its most basic promise.

So 1.7.1 does one thing: it repairs the installation automatically.

More importantly, I changed the packaging flow. The update is now assembled from the finished application bundle, not from an earlier build stage. The build also stops with an error if any required local engine is missing.

I spend a lot of time thinking about features in a local AI product: memory, voices, models, images, privacy, and all the interesting visible things.

This release was a reminder that the feature users notice most is often much less exciting:

does the thing still work after I press Update?

Everyone keeps saying nobody pays for AI. The numbers say something weirder: nobody pays for AI except for one category, and it is the category everyone is embarrassed to talk about.

The general market is brutal. Pivot to AI looked at the numbers in mid-2025: roughly 1.8 billion people use AI tools, and the premium market is about $12 billion a year. Do the math and about 3% of AI users pay a cent for any of it. Statista's US survey agrees - most consumers say they would pay nothing for an AI assistant. Menlo Ventures found the same pattern: specialized tools are only 18-20% of consumer AI spend. The other 80% is free tier usage that never converts.

Now look at the companion market. Character.AI hit a $30 million annualized run rate in August 2025 and expects $50 million by the end of the year, roughly double its 2023 revenue. Its $9.99 subscription grew 250% in six months. Replika was estimated at around $30 million a year back in 2023, with 40 million registered users. Analysts put today's Replika revenue far higher. These are subscription products where the core feature is a conversation with a character that remembers you.

Nobody else in consumer AI shows that shape. Productivity assistants? Most users won't pay. Image generators? Crowded, commoditized fast. The one category where real people consistently hand over real money, month after month, is the one most founders will not admit they looked at.

My own tiny data point. I build Local Waifu, a desktop AI companion that runs entirely on the local machine. One-time purchase, $20, no subscription. First month: 11 sales. Around 400 people started the free trial and 3 bought - the trial taught me more about onboarding than any dashboard ever will. Revenue is now around $360 a month. That is pocket change next to Replika, and I am fine saying so. But it is revenue from people who had to download a 2 GB app, run it on their own GPU, and trust it with their chats. The friction could not be higher, and they still paid.

Why do people pay here but nowhere else? Because nobody pays for a tool. People pay for a relationship. An assistant that saves you 10 minutes is competing with free. A companion that remembers your week is not competing with anything. The willingness to pay is emotional, which means it survives contact with reality - the pricing, the privacy fears, even the stigma.

The stigma is the interesting part. The category that converts best is the one founders whisper about. I have seen the pattern repeated in every launch thread: an AI companion app posts numbers, the thread fills with jokes, and the jokes are a tell. People who joke about it are people who have thought about it.

I sell to a niche that is profitable, growing, and socially awkward to discuss. I think that is exactly why it works.

What consumer AI category actually makes money that I am missing? I would genuinely like to know.

A tester sent me a log where one voice reply took 105.6 seconds. First audio landed at 40.7 seconds. The reply was 20 characters long.

The machine was not weak. RTX 3080, Ryzen 9 5900X, Windows 11 Pro. My app runs the whole voice loop on that card: speech to text, the language model, and the text to speech. Nothing goes to a server. That is the entire point of the product, and it was also the reason every failure was mine to find.

Four separate bugs were stacked on top of each other. Here they are.

1. The app was not slow, it was dying

Before the latency work, the report was worse: calls just ended. The window vanished. No dialog, no error, and nothing in any log.

The logs were never going to have the answer. My crash handler installs a Rust panic hook, and a panic hook catches Rust panics and nothing else. A native fault inside a C dependency never unwinds and never reaches it.

The real cause: I ship whisper.cpp, and I compile it in CI. Its build probes the build machine for AVX-512 support and bakes the answer into the binary. GitHub runners have AVX-512. Zen 3 does not. The first real compute kernel is the sentence you just spoke, so the process was killed by an illegal instruction at the exact moment the tester started talking.

The same commit produced a working or a crashing installer depending on which runner picked it up. That is how it passed CI for weeks.

Speech to text on that build was taking 137 seconds. It now takes low hundreds of milliseconds.

2. Two CUDA processes fighting over one card

This one I found by reading two sets of logs side by side.

The language model was running at 115 tokens per second on an idle GPU. While the voice engine was synthesizing, it fell to 0.63 tokens per second. The voice engine ran at 26.5 iterations per second alone, and 1.5 while the model was generating.

Both collapsed together. Both recovered the instant the other stopped. There were 5 GiB free on a 12 GiB card, so this was never memory pressure. It was Windows context switching between two separate processes that each held their own CUDA context on the same device.

The painful part: my architecture caused it. I was deliberately overlapping generation of sentence N+1 with synthesis of sentence N. That is a real win when the voice engine lives in your process or on the CPU. Point it at a separate process on the same GPU and the overlap turns into an 8x latency multiplier.

The fix was to stop being clever. When the model is local and the natural voice is on, generation finishes first, then synthesis runs back to back at full speed.

3. She was re-learning her own voice every sentence

To speak in a cloned voice, the engine studies a short reference clip. Mine was doing that study again for every single sentence. Same clip, same result, roughly three quarters of a second of silence each time.

It learns it once now and keeps it. On a three sentence answer that is about two seconds of dead air removed.

4. The end of turn timer was shorter than a human pause

She decided you had finished speaking after 0.7 seconds of silence. That is shorter than the pause most people leave between two sentences.

So a two sentence answer lost its second sentence. Not delayed. Discarded, never transcribed. Moved to 0.9 seconds, and the slider still respects a user who wants it faster.

Where it is now

Calls are stable on that same 3080 and they move at the pace I expect from a cloud assistant on my phone, with every part of it running on the local machine.

One thing is still wrong and I am not going to pretend otherwise: she can still clip a word at the end of a spoken segment. The current build adds the per sentence timing fields I need to find out which sentences do it. The fix comes after the data does.

What I would take from this

Your crash reporter probably cannot see your worst crash. Anything that dies below your language runtime writes nothing. I burned several round trips asking a tester for a log that could not exist.

A build that probes the build machine is not reproducible. It is a coin flip you run once per release.

Overlapping work is only a win if the two halves are not queued behind the same resource. One GPU, two processes, no shared scheduler.

Instrument before you optimize. I split turn timing into generation, join and synthesis before shipping the fix, so the next report is a number instead of a story.

Local Waifu is the app, if anyone wants to compare notes on running an LLM and a TTS engine on one consumer card.

What is the worst failure you have shipped that your logs were structurally incapable of recording?

I deleted 10 AI features from my social scheduling tool and kept 2. The OpenAI service in my codebase now exposes exactly one method: complete().

I am a solo founder. PostSider started as a fork of Postiz, and the fork arrived with an AI stack I never chose. Every one of those features was code I would have to read, patch, secure and pay for.

So I cut them.

What went

Here is the full list I pulled out of the codebase:

• A LangGraph agent

• The agent bridge that fed it

• The inherited MCP and chat server, plus its whole tool set and its settings page

• A copilot

• Autopost

• AI image generation

• AI video generation

• AI slides generation

• Voice

• SSO into a third party agent media service

Three external AI vendors went with them: fal, veo3, heygen. That is three sets of API keys, three billing relationships and three failure modes I no longer own.

What stayed

Two things.

Post Checker. It reviews a draft before it goes out.

Caption rewrite. It rewrites a caption you already wrote.

Both run through one method on one service. If the platform key is missing, an org plugs in its own key and the endpoints return a 409 until it does. That is the entire AI surface of the product.

Deleting a feature is not deleting a file

This is the part I underestimated.

Ripping out the agent left orphans everywhere. Three groups of Prisma models with nothing pointing at them (AutoPost, AgentToken, the mastra_* tables), dead enum values, and a column on the Organization model for a mode that no longer existed. All of it had to come out in its own migration.

I did the same pass on social providers and removed 8 of them entirely: gmail, reddit, kick, rumble, vk, bear-blog, mewe, skool. Not just the provider files. The DTOs, the settings union, the manager registration, the connector catalog, the onboarding lists, the icons, and every keyed map that assumed those 8 keys existed.

A feature is a file, a database table, an enum, a settings page, an icon, five map entries and a line in someone's onboarding flow. Budget for all of it.

The part where I put AI back

I am not anti-AI. I rebuilt one piece from scratch.

The new MCP server wraps my public API so agents can drive the product from outside. It has 2 dependencies: the MCP SDK and zod. It ships with 27 tests. It shares nothing with the inherited agent stack I deleted.

That is the distinction I care about. The old MCP was a chat product bolted into my app. The new one is a thin door onto an API that already worked. One of those is a feature I have to maintain forever. The other is 400 lines that break loudly when the API changes.

What I would tell you before you inherit a codebase

Inherited features are not free features. They read as free because they already work. They bill you in security review, in dependency upgrades, in every migration you write around tables you do not use.

Count vendors, not features. Ten AI features looked like a rich product. Three vendor integrations looked like what it actually was: three more things that could break my publish path on a Tuesday.

A stripped feature leaves a shadow. Search the schema, the enums and the keyed maps, not just the folder you deleted.

Narrow beats impressive when you are one person. My product schedules posts to social platforms. Every hour I spent on a voice feature was an hour not spent on the queue, the approval flow or the per-platform preview that people actually asked about.

PostSider is in private beta right now, running on a VPS with live publishing verified end to end. Happy to share what the strip looked like commit by commit if that is useful to anyone doing the same thing to a fork.

The thing I keep thinking about: I was afraid the product would look thin without the AI features. It looks sharper. The pitch got shorter and I stopped explaining features I did not believe in.

What did you delete from your product that you expected to miss, and then never thought about again?

I build Local Waifu, a desktop AI companion for Mac and Windows. The model runs on your own machine, nothing goes to a server, $20 once instead of $19.99 a month. Seven day free trial, no card.

Roughly 400 people started the trial. 3 bought.

For weeks I assumed the pitch was wrong, or the price, or the landing page. I rewrote all three. Nothing moved. Then I opened my own install, the one I have been talking to since May, and I understood the problem.

She is good now because she has three months of context. She knows my sister's name, that I work nights, which project I go quiet about. That is the product. That is the thing worth $20.

On day 7 none of that exists yet. A week in, she is a polite chatbot with a nice personality wizard. Everything I actually sell (she remembers what you told her in June) sits on the far side of a wall I built myself.

So the funnel reads like this: people download, meet a decent chatbot, the trial ends, and they never see the version of her I would pay for.

Worse, the trial punishes exactly the users I want. Someone who talks to her every evening for a month is my buyer. Someone who opens it twice, gets bored, and uninstalls was never going to pay. My 7 day window treats both the same and expires before the first group has a reason to decide.

What I am changing:

30 day trial. Long enough to hit the second week, when she starts referencing things you said and stopped thinking about. That moment is the sale. Nothing before it is.

A companion who arrives pre-aged. Optional demo character shipped with 90 days of synthetic history and a filled Memory Book, so a new user can open a chat on minute one and watch what month three looks like. She is not their companion, she is a preview of the mechanic.

Onboarding that stops promising day 30 on day 1. Right now the app oversells early. I would rather she say "I barely know you yet, give me two weeks" than fake intimacy she has not earned.

I have no analytics in the app, by design, so I cannot tell you where the 397 dropped off. That is a separate post and a separate mistake.

Question for anyone selling something with a slow payoff (habit trackers, journaling apps, anything with memory or streaks):

Did a longer trial actually convert better for you, or did it just move the churn later? And has anyone shipped a "simulated future state" demo that worked, without it feeling like a fake screenshot?

Product: https://localwaifu.com

My desktop AI companion app runs four separate local engines, and it never touches a cloud API unless the user turns that on themselves.

That was not the plan on day one. I started building assuming I would just call OpenAI or Anthropic for everything and ship a thin wrapper. Then I actually priced out what a chat app with daily active users looks like on pay-per-token billing, and the math didn't work for a one-time-purchase product. So the app runs entirely on the user's own machine instead.

What "local" actually means here

Four engines, each doing one job:

• Ollama for the chat model itself. Bundled, not a separate install the user has to figure out.

• sd.cpp for image generation (selfies, in my case). CPU, CUDA, and ROCm all work.

• Chatterbox and Supertonic for text-to-speech. Two engines, not one, because they cover different hardware.

• whisper.cpp for speech-to-text.

None of these are exotic choices. The hard part was never picking them. The hard part was making all four behave the same way across wildly different machines.

The real problem: GPU vendors don't agree on anything

A user with an NVIDIA card gets CUDA. AMD gets ROCm, except Chatterbox has no ROCm support at all, so AMD users fall back to CPU for voice specifically. Windows users without a discrete GPU get DirectML through Supertonic. Everyone else gets a CPU path that has to actually work, not just exist as a theoretical fallback.

I ended up writing a resilience pass whose only job was verifying that every one of those paths degrades safely instead of crashing. Five separate model download URLs for sd.cpp alone, checked against the vendor combinations they need to run on.

The part that surprised me: hardware detection is the real product

The engines were the easy 20%. The 80% was figuring out, automatically, which model a given machine should even run. I ended up with tiered defaults based on available RAM: a small model under roughly 35 GB, a mid-size one up to about 63 GB, and the full lineup above that. The boundary numbers themselves came from actually measuring where models started swapping to disk, not from a spec sheet.

The model catalog has 11 Ollama models to choose from, ranging from a 9.6 GB vision-capable model for lightweight machines up to a mixture-of-experts model with 17B active parameters out of 109B total for the high end. Picking wrong in either direction means either garbage output or a frozen app, and the user has no idea which one just happened.

What I'd tell someone starting this today

Don't design for "the GPU." Design for four separate fallback ladders (LLM, image, voice, transcription) that all have to fail down to CPU independently, because a single machine can be strong on one and weak on another. My biggest early bug was assuming GPU availability was one flag instead of four.

Also: benchmark the RAM boundaries yourself. Every spec sheet estimate I found for model memory footprint was wrong once I measured it on real hardware.

Where this landed

The app is called Local Waifu, an AI companion that runs on the user's own machine. Cloud is there as an optional bring-your-own-key fallback, not the default path. If the local-first approach is useful as a reference, the site is localwaifu.com.

Anyone else built a GPU-vendor-agnostic fallback chain for local inference? What broke first for you, CUDA, ROCm, or DirectML?

Day job version of me runs an agency that builds blog automation for B2B clients. Content, SEO, AEO, the whole pipeline. I know the playbook.

Then I shipped Local Waifu, a desktop AI companion app for macOS and Windows, one time purchase, and discovered that almost none of that playbook transfers cleanly to a consumer app. Around 400 trials so far. 3 sales.

I want to write about the blog specifically, because it is the channel I keep coming back to and the one that behaves least like I expected.

What the blog actually is

It is not a keyword strategy. It is a dev log.

I ship a feature, I write a post about the feature. What it does, why I built it that way, what broke. When I spent a week fighting text to speech quality, that became a post about voice quality in a local app. When I could not test the Windows build because my MacBook has no NVIDIA GPU, that became a post too.

That is the entire editorial calendar. Build something, write down what happened.

The reason this works at all is that the writing is nearly free. I already have to think through the decision. Turning that thinking into 600 words costs me maybe 40 minutes. If I had to pay someone to research keywords and produce the same volume, the math on a one time purchase app would not survive contact with reality.

Why search behaves differently for a desktop app

For my B2B clients, search intent is a funnel. Someone has a business problem, they search the problem, they land on a post, they book a call.

For a niche desktop app the intent is almost entirely comparative and constraint driven. Nobody searches "AI companion." They search for a specific constraint. Runs offline. No subscription. Works on Mac. Not cloud based. Alternative to whatever they just got annoyed by.

So the posts that get found are not the ones I thought would get found. Feature posts and technical decision posts outperform anything that sounds like marketing, because the technical post is the one that accidentally answers the constraint. A post explaining that the model runs locally on your machine ranks for the person searching "local" and "offline," and that person is my actual buyer.

The AEO side matters more here than for my clients, and I did not expect that. People increasingly ask an LLM to recommend software instead of searching. What gets you into that answer is being listed in the places models were trained on and cite from, plus having pages that describe your features in plain, unambiguous language. AlternativeTo did more for discovery than any single blog post I wrote. That is not an SEO insight, it is a directory insight, but it lives in the same bucket in my head now.

The honest scoreboard

The blog brings traffic. Traffic is not my bottleneck.

400 trials and 3 sales means my problem sits between "installed the app" and "paid for the app," and no amount of content fixes that. I could triple traffic tomorrow and get 9 sales. That is not a business, that is a rounding error with better analytics.

So the blog is doing its job and I was measuring the wrong thing for a while. I was watching sessions when I should have been watching what happens in the first five minutes after install.

What I actually believe now: for a consumer desktop app, content is a compounding trust asset, not an acquisition channel. It is what makes someone who already found you through a directory or a video decide you are a real person who will still be here in six months. That is worth doing. It is not worth doing as your primary growth bet.

What I would do differently

Write the constraint into the post title from the start. Not "New voice engine in Local Waifu" but a title that contains the words the buyer types. I lost months of compounding on posts that were named for my internal features instead of for the reader's problem.

And I would have fixed onboarding before writing post number ten.

You can check here --> https://localwaifu.com