I built a long-document AI translation workflow. Here’s what one literary test taught me.
I’m building GoodTrans, an AI translation workflow for long documents.
The original problem was simple: translating a few paragraphs is easy. Translating an entire book, research paper, technical document, or long report is much harder.
Terms drift. Tone changes between chapters. Important details get lost. And after the translation is finished, there is often no practical way to review the result.
So I wanted to test GoodTrans on something more difficult than a short product description.
The test
I used Lao She’s Chinese essay “Thinking of Beiping” (《想北平》).
It is a literary essay about Beijing, memory, family, local food, and personal identity. Much of its meaning comes from tone, cultural references, and small expressions that are difficult to translate literally.
I compared three things:
- The original Chinese text
- A well-known published English translation by Zhang Peiji
- A GoodTrans translation generated through a multi-stage workflow
The GoodTrans workflow included terminology extraction, draft translation, reflection, revision, and consistency checks.
A few examples
The original contains the expression:
我的北平大概等于牛的一毛
The classic translation uses:
a mere drop in the ocean
GoodTrans used:
no more than a single hair on an ox
The first version is immediately natural to an English reader. The second keeps the concrete image from the Chinese expression.
Another example is:
让我单摆浮搁的讲一套北平，我没办法。
The classic translation is formal and polished:
I shall be at a complete loss if I should be called upon to write exclusively about Peiping.
GoodTrans produced:
But ask me to talk about Beiping straight out, just like that, and I'm at a loss.
The second version is less formal, but it tries to preserve the conversational voice of the original.
How did we evaluate it?
I asked two independent AI systems, Gemini and Grok, to evaluate the versions.
They gave the GoodTrans version scores of 95 and 92, while the classic published translation received 90.
This is not proof that AI has replaced a professional translator. It was one literary text, and AI evaluating AI has obvious limitations.
The interesting result was not the score itself. It was that the two versions made different translation choices:
- Should the translator preserve a Chinese image or use a familiar English idiom?
- Should the voice sound formal or conversational?
- Should cultural flavor be explained, adapted, or left intact?
There is rarely one automatic answer.
What I’m building
GoodTrans is designed for long, review-heavy documents.
A user can upload a PDF, EPUB, Markdown, or TXT file, add terminology guidance, and let the task run asynchronously.
The delivery package includes:
- Editable Markdown and TXT files
- A bilingual review file
- A document-level quality report
- Email delivery when the task is complete
The goal is not to claim “perfect translation.” The goal is to make long-document translation easier to inspect, revise, and deliver.
You can try the workflow here:
Try GoodTrans with 3,000 trial Credits
I also shared a shorter version of this experiment on X, where it received more than 4,000 views. That gave me some evidence that people are interested in the question, but visibility is not the same as product adoption. I’m now trying to understand how to turn useful translation research into a product people will actually use.
I’d love feedback from other indie hackers:
- How would you evaluate the quality of an AI translation?
- Would a bilingual review file and quality report be useful in your workflow?
- What kind of long document would you be willing to test?
I think your own reply narrowed the next experiment more than the benchmark did.
Contracts, academic papers and books all need “good translation,” but they have completely different costs of error and buying behavior. I wouldn’t keep validating all three at once.
I’d pick the segment where one bad sentence creates the clearest consequence, then test the full workflow rather than translation quality:
real long document → translation → bilingual review opened → risky passages inspected or routed to a human → user comes back with another document.
If that chain happens, you have evidence that reviewability changes behavior. If people like the quality report but never actually use it, it’s probably proof content rather than product value.
The one thing I’d want to know before choosing the segment: has any trial user already completed a genuinely long document and actually used the bilingual review file afterward?
That's correct — we've run the full workflow end-to-end on long documents multiple times. Before launch (May–June), we stress-tested with documents ranging from 5,000 to 36,000 characters — chapters, novel excerpts, and one PDF. All completed the full pipeline; the larger ones ran slower, which is expected — longer texts simply need more time for inference and review. The checkpoint-resume feature actually came out of this: running that 36,000-character document is what convinced us the workflow needed segment-level recovery.
What we haven't seen yet is an external user doing it — most trial users so far have run shorter documents. So the honest picture is: the pipeline is proven on long documents, but the chain you described — open the bilingual review, inspect the risky passages, come back with a second document — hasn't been observed with a real outside user yet.
On picking a segment: agreed, we shouldn't validate all three at once. Demand data pulls toward books (search volume, plus a loud Reddit thread of people wanting 800-page novels translated and saying they'd pay). Your error-cost argument points at contracts. We'd rather not settle that by theory — the first real long-document users will tell us which segment we're actually in.
The distinction between translation quality and reviewability is interesting.
Curious whether long-document users would value the bilingual review and quality report enough to change their workflow, or whether translation accuracy alone is still the deciding factor.
Honest answer: for long documents, accuracy alone is often unverifiable
by the buyer. If you're translating a 200-page contract and you don't read
the source language, "the AI was accurate" is a claim you cannot check —
you're just trusting a benchmark score you found on a vendor's homepage.
That's the gap bilingual review fills. It doesn't make the translation
better — it makes the quality inspectable. You spot-check against the
source, see where the model was uncertain, and route only the risky 5% to
a human reviewer. The quality report is the same idea: it turns "trust me"
into "here's the receipt."
You're right that for short casual texts, accuracy and price alone decide —
nobody audits a menu translation. The workflow change only shows up where
the cost of one wrong sentence is high: contracts, academic papers, books.
Which is exactly the segment we built for.
Are you building in this space, or evaluating tools? Happy to walk you
through a real bilingual review output if useful.
That’s useful context. Happy to continue the conversation privately — what’s the best email to reach you on?
support@goodtrans.app works best — that inbox comes straight to me.
Quick heads up: if you'd rather see the bilingual review + quality report on an actual document of yours, the trial credits at goodtrans.app are enough for a chapter or two. Happy to walk through the report together afterwards — spot-checking the alignment is exactly the workflow it was built for.
Thanks! I’ve just sent it over.
Looking forward to hearing your thoughts whenever you have a chance.