Author: Latha Shree
Editor: Vahe Aslanyan
A practical engineering handbook on shipping translation quality in production — based on lessons from building a production-grade LLM translation pipeline, a placeholder-protection system, and the operational infrastructure around them. Rather than presenting document translation as an API integration exercise, this handbook treats it as a quality engineering discipline, and walks through the architecture, testing philosophy, and operational practices that separate a demo from a dependable product.
Sending paragraphs to a large language model and pasting the results back into a PDF sounds simple. Anyone can wire up an API call in an afternoon and produce something that looks like a translated document. In production, however, this naive approach fails in ways that are invisible to unit tests and often invisible to the developer until a customer complains. Product codes get split across formula placeholders and come back garbled. Table-of-contents dot leaders reflow into meaningless number strings. The model's own commentary — apologies, disclaimers, explanations — gets rendered directly into form fields. Text that sits adjacent to images in the drawing layer never passes through the translator at all, and therefore never appears in any translation log you might inspect.
These failures share a common trait: they live in the gap between the text the model sees and the document the user receives. Closing that gap is the real work of document translation, and it is almost entirely engineering work rather than model work.
This handbook covers that engineering in three broad areas. The first is translation architecture: how to structure LLM calls, how to protect content that must never be translated, and how to enforce terminology consistently across hundreds of pages. The second is quality engineering: why tests written against plain strings systematically miss the bugs that matter, and what a testing strategy that actually catches them looks like. The third is production operations: how a translation job moves from upload to download, how to manage multiple model providers, and how to keep the whole pipeline observable and recoverable when things go wrong.
Throughout, the guiding principle is the same. Translation quality is validated against text extracted from the generated output document, not against string equality in a test file. Everything else in the handbook follows from taking that principle seriously.
The code examples throughout are illustrative Python sketches rather than a complete library. They show the shape of each component — its inputs, its responsibilities, and the failure modes it guards against — so that the patterns can be adapted to any codebase.
At the heart of any document translation pipeline sits a translator module that orchestrates everything between the parsed source document and the translated text that will be laid back onto the page. In a mature system this module grows to several thousand lines, and its responsibilities are worth enumerating because each one exists to solve a failure mode discovered in production.
The translator injects placeholders into the text before any LLM call, so that formulas, product codes, and numeric data are shielded from the model. It constructs the prompt for each unit of text, folding in glossary terms, surrounding context, and the system instructions appropriate to the document type. It fans translation work out across a thread pool so that pages translate in parallel rather than sequentially. When translations come back, it sanitizes and normalizes them, repairing the artifacts that models reliably introduce. Finally, it writes a detailed audit trail — a record of every paragraph's input shape, placeholder mapping, and output — which becomes the single most important debugging artifact in the entire system.
Each paragraph is treated as one translation unit. For every unit, the model receives the paragraph text with placeholders substituted in, a window of context from the adjacent paragraphs, any terminology extracted from the document or supplied by the user, and a system prompt tailored to the document's genre. None of these ingredients is optional. Remove the context and translations drift between adjacent paragraphs. Remove the terminology and a product name is rendered five different ways across five chapters. Remove the genre-specific instructions and the model starts refusing to translate perfectly ordinary forms.
In code, the translation unit and the orchestration around it look like this:
from dataclasses import dataclass
@dataclass
class TranslationUnit:
"""One paragraph — the atomic unit of translation."""
paragraph_id: str
source_text: str # text with placeholders already injected
placeholders: dict[str, str] # {"v1": "T15", "TOKEN_1": "Halcyon TX12"}
context_before: str # tail of the previous paragraph
context_after: str # head of the next paragraph
glossary_terms: dict[str, str] # enforced source -> target renderings
genre: str # "standard" | "form" | "academic"
def translate_unit(unit: TranslationUnit, provider: ModelProvider) -> str:
prompt = build_prompt(
text=unit.source_text,
context=(unit.context_before, unit.context_after),
glossary=unit.glossary_terms,
system=system_prompt_for(unit.genre),
)
raw = provider.complete(prompt)
if is_translator_commentary(raw):
raise CommentaryRejected(unit.paragraph_id) # retried upstream
restored = restore_placeholders(raw, unit.placeholders)
normalized = normalize_translation(restored, unit)
audit_trail.record(unit, raw, normalized) # translation_audit.json
return normalized
Every line of this skeleton corresponds to a production failure mode. The commentary check exists because refusals were rendered into documents. The normalization step exists because models garble reassembled placeholders. The audit record exists because, without it, no bug in the pipeline can be diagnosed to the correct layer.
The choice of translation unit deserves its own discussion, because it is one of the earliest architectural decisions and one of the hardest to reverse later.
Sentence-level translation seems attractive at first: smaller units mean cheaper calls, easier caching, and finer-grained parallelism. In practice it destroys quality. Sentences lose the cross-sentence context that resolves pronouns, maintains register, and keeps terminology stable. Worse, sentence boundaries frequently cut straight through the things you most need to keep intact. A product name may span two sentences after optical character recognition has split it awkwardly, and once the two halves travel through the model separately, no amount of post-processing will reliably reunite them.
Page-level translation fails in the opposite direction. Whole pages exceed comfortable context windows for dense documents, they prevent meaningful parallelization within a page, and a single model failure poisons an entire page rather than one paragraph. Retries become expensive, and caching becomes nearly useless because no two pages are ever identical.
Paragraph-level translation is the sweet spot. A paragraph is large enough to carry its own context and keep multi-word terms whole, yet small enough to translate quickly, retry cheaply, and cache effectively. Every production lesson in this handbook assumes paragraph-level units, and the placeholder system described next is designed around them.
Before any text reaches the model, every span that must not be translated is replaced with an opaque placeholder token. Mathematical expressions, product codes, firmware versions, dates, identifiers, and inline styling boundaries are all swapped out, translated around, and swapped back in afterward. This sounds like a minor preprocessing step. It is, in fact, the most engineering-intensive part of document translation, and more production bugs trace back to placeholder handling than to any other component.
The difficulty is that several very different kinds of content compete for protection, and each kind fails differently when mishandled. A system that treats them uniformly will corrupt some of them.
In practice, four categories of placeholder emerge. Formula placeholders protect genuine mathematical content: expressions, superscripts, and subscripts that the layout parser has identified as math. Token placeholders protect known vocabulary: brand names, protocol names, and firmware versions that appear on a curated protection list. Data placeholders protect numeric material: sequences of digits, dates, and identifiers whose exact characters must survive translation untouched. Finally, rich-text markers protect formatting rather than content — they record where bold or italic spans begin and end inside a paragraph so that styling can be reapplied to the translated text.
The categories matter because the failure modes differ. A mistranslated brand name embarrasses you; a corrupted identifier in a legal or medical document can have real consequences. Knowing which category a span belongs to determines how aggressively it is protected and how it is validated afterward.
from enum import Enum
from dataclasses import dataclass
class PlaceholderKind(Enum):
FORMULA = "formula" # {v1} — math expressions, superscripts, subscripts
TOKEN = "token" # [[TOKEN_1]] — protected vocabulary
DATA = "data" # [[data_1]] — digit sequences, dates, identifiers
STYLE = "style" # markers for bold/italic span boundaries
@dataclass
class Placeholder:
kind: PlaceholderKind
key: str # "v1", "TOKEN_1", "data_1"
original: str # the exact source span, restored verbatim after translation
The most instructive placeholder failure is worth walking through in detail, because it illustrates why this layer is so treacherous.
Consider a fictional radio controller called the Vertex T15. A PDF layout parser, seeing the short alphanumeric fragment "T15" set slightly apart from surrounding text, may classify it as a mathematical formula and replace it with a formula placeholder. The model then receives the text "Vertex" followed by an opaque token. It has no idea the token is half of a product name. Translating into Russian, it may render the visible word phonetically and mangle the reassembled result into something like "Вертекс 5T1" — the digits transposed, the product name destroyed. The bug is invisible at the API layer, because the model did nothing wrong with the text it was shown. The corruption happened before the model was ever called.
The fix is a classification step that runs before placeholder injection. Short alphanumeric fragments that look like model numbers, SKU codes, or battery specifications are inlined back into the text as ordinary words, while genuine mathematical expressions and chemical formulas remain protected. Writing this classifier is a matter of accumulating patterns from real documents: fragments like "T15", "04X", "2S", and short numeric ranges are text; anything with operators, Greek letters, or structural math markup is formula. The classifier will never be finished — every new document domain contributes new edge cases — but each pattern added prevents an entire family of garbled names.
import re
# Short fragments that look like codes, not math.
_MODEL_CODE_RE = re.compile(r"^[A-Za-z]{0,3}\d{1,4}[A-Za-z]{0,3}$") # T15, 04X
_BATTERY_SPEC_RE = re.compile(r"^\d[Ss]$") # 2S, 3S
_SHORT_RANGE_RE = re.compile(r"^\d{1,2}\s*[–-]\s*\d{1,2}$") # 1–4
# Signals of genuine math: operators, Greek letters, structural markup.
_MATH_SIGNALS_RE = re.compile(r"[=+^_\\∑∫√±×÷]|\\frac|\\sqrt|[α-ωΑ-Ω]")
def should_inline_formula_as_text(fragment: str) -> bool:
"""Return True when a 'formula' is really a product code and must stay
inline, so the model and the glossary see the complete term."""
fragment = fragment.strip()
if len(fragment) > 12: # long spans are never codes
return False
if _MATH_SIGNALS_RE.search(fragment): # real math stays protected
return False
return bool(
_MODEL_CODE_RE.match(fragment)
or _BATTERY_SPEC_RE.match(fragment)
or _SHORT_RANGE_RE.match(fragment)
)
Once codes are inlined, the glossary layer can finally see complete terms. Protection only works on whole names; half a product name is worse than none.
Beyond the general classifier, production systems maintain explicit protection patterns for vocabulary known to appear in their document domains. Product names, protocol and firmware names, and standard technical abbreviations are matched in the source text and converted to token placeholders before the model call. The model translates around them, and the original strings are restored verbatim afterward.
PROTECTED_TERMS: list[re.Pattern] = [
re.compile(r"\bHalcyon\s+TX12\b"),
re.compile(r"\bVertex\s+T15\b"),
re.compile(r"\bSkyRay\s+SR04X\s+Pro\b"),
re.compile(r"\b(?:OpenFlight|AeroLink|FlightCore)\b"),
re.compile(r"\bLi(?:Po|-ion)\b"),
]
def protect_tokens(text: str, placeholders: dict[str, str]) -> str:
"""Swap protected vocabulary for opaque tokens before the LLM call."""
def _swap(match: re.Match) -> str:
key = f"TOKEN_{len(placeholders) + 1}"
placeholders[key] = match.group(0) # restored verbatim afterward
return f"[[{key}]]"
for pattern in PROTECTED_TERMS:
text = pattern.sub(_swap, text)
return text
This list is a living artifact. Every time quality control turns up a mistranslated term, the term joins the list. Over months, the protection list becomes one of the most valuable assets the system owns, because it encodes exactly which vocabulary the target document domains care about.
Even with thorough pre-protection, models introduce artifacts, and source documents arrive with optical character recognition damage that survives into the translation. A normalization pass therefore runs on the translated text, repairing known corruptions back to canonical forms. Misread brand names are corrected to their proper spellings. Transposed product codes are restored. Battery chemistry abbreviations with the wrong capitalization are fixed. Invented bracket wrappers — models occasionally hallucinate placeholder-like markup around ordinary words — are stripped away. Table-of-contents artifacts, where dot leaders and section numbers have fused into nonsense strings, are detected and separated.
Normalization runs both before and after the model call. Running it before cleans up OCR damage so the model receives sane input; running it after catches what the model itself introduced. The two passes share pattern tables, and like the protection list, those tables grow with every quality control cycle.
# Canonical repairs, applied both before AND after the model call.
_CANONICAL_FORMS: dict[str, str] = {
"Holcyon": "Halcyon", # OCR misread of a brand name
"Vertex 5T1": "Vertex T15", # transposed product code
"Lift fO ff": "Liftoff", # OCR-split compound word
"LiPO": "LiPo", # wrong capitalization
"Li-lon": "Li-ion", # lowercase L misread as capital I
}
_HALLUCINATED_BRACKETS_RE = re.compile(r"\[\[([A-Za-zА-Яа-я]{2,20})\]\]")
def normalize_translation(text: str, unit: TranslationUnit) -> str:
for damaged, canonical in _CANONICAL_FORMS.items():
text = text.replace(damaged, canonical)
# Strip [[WORD]] wrappers the model invented around ordinary words —
# but never around keys that map to real placeholders.
text = _HALLUCINATED_BRACKETS_RE.sub(
lambda m: m.group(0) if m.group(1) in unit.placeholders else m.group(1),
text,
)
return normalize_toc_artifacts(text) # dot leaders, fused section numbers
This is the single most important quality framework in the handbook, and it exists because of a painful, repeated experience: a bug is reported, a fix is written, the test suite passes, and the regenerated PDF still contains the bug. The three-layer acceptance model prevents that cycle by defining what "fixed" actually means. A fix is complete only when all three layers pass, and each layer catches a class of failure the others cannot see.
The first layer is ordinary unit testing of the normalization functions against clean string input. Given a garbled product name, the normalizer returns the canonical form; given a corrupted code, it returns the correct one. These tests are fast, precise, and essential for pinning down the behavior of individual functions.
def test_transposed_product_code():
assert normalize("Vertex 5T1") == "Vertex T15"
def test_ocr_brand_misread():
assert normalize("Holcyon") == "Halcyon"
def test_battery_capitalization():
assert normalize("LiPO battery") == "LiPo battery"
Their limitation is fundamental, though. They test the function, not the pipeline. A normalization function can be perfectly correct and still never fire in production, because the real document path splits the term across a placeholder boundary before the normalizer ever sees it. Plain string tests pass while the shipped PDF stays broken. Treating layer one as sufficient is the single most common quality mistake in document translation systems.
The second layer tests against the actual input shapes the translator encounters, which are recoverable from the translation audit trail. Instead of testing "Vertex T15" as a clean string, the test presents "Vertex" followed by a placeholder whose hidden value is "T15", exactly as the real pipeline produced it, and asserts that the reassembled output contains the intact product name and none of the known corruptions. Similar tests cover terms split across two placeholders, duplicated fragments, and codes embedded mid-word.
def test_product_code_behind_formula_placeholder():
# Exact input shape recovered from the translation audit trail.
result = translate_with_placeholders(
"Vertex {v1}", placeholders={"v1": "T15"}, target="ru",
)
assert "T15" in result # the code survives reassembly intact
assert "5T1" not in result # the known transposition never appears
def test_sku_split_across_two_placeholders():
result = translate_with_placeholders(
"SkyRay SR{v1}SR{v2} Pro",
placeholders={"v1": "040", "v2": "04X"},
target="ru",
)
assert result.count("SR") == 1 # no duplicated fragment
Layer two is where most real translation bugs are actually reproduced. The discipline that makes it work is simple: when quality control finds a bug, the first step is to pull the paragraph's exact input shape from the audit trail and turn it into a test case. The test fails, the fix is written, the test passes — and the input shape joins the permanent regression suite.
The third layer validates the final product. Text is extracted from the generated PDF itself and checked against two marker lists. The bad-marker list contains strings that must never appear: known garblings, transposed codes, fused table-of-contents numbers, mangled firmware names. The good-marker list contains strings that must appear: the correct product names, the correct technical terms, the correct section structure. A document passes only when every bad marker is absent and every good marker is present.
BAD_MARKERS = ["372.3", "5T1", "Lift f0", "LiP /O", "OpenFliqht"]
GOOD_MARKERS = ["Vertex T15", "Halcyon", "OpenFlight", "LiPo", "FlightCore"]
def qc_check(pdf_path: str) -> QCReport:
text = extract_text(pdf_path) # any PDF text-extraction library
bad_hits = [m for m in BAD_MARKERS if m in text]
good_misses = [m for m in GOOD_MARKERS if m not in text]
return QCReport(
passed=not bad_hits and not good_misses,
bad_hits=bad_hits,
good_misses=good_misses,
)
Layer three exists because entire categories of failure happen after translation, in the rendering stage, where no translator test can reach them. It also provides the decisive diagnostic rule of the whole framework: if a bad marker appears in the extracted PDF text but does not appear anywhere in the translation audit trail, the bug is in the render layer, not the translator. Fixing the translator in that situation is wasted effort, and worse, it adds speculative code to the wrong component. Check the audit trail first, always.
A document translation pipeline has two places where text can go wrong: the translation layer, where words are converted between languages, and the render layer, where translated words are drawn back onto pages. The distinction sounds obvious, yet misdiagnosing which layer owns a bug is among the most common and expensive mistakes in practice.
The audit trail is the arbiter. A wrong product name that is visible in the audit trail is a translator bug, and belongs in the translator module. An untranslated running header that never appears in the audit trail at all is a render-layer issue — the text bypassed the translator entirely, usually because it lives adjacent to images or in the drawing layer rather than in the parsed text flow. Stray characters from the drawing layer, garbled fonts, and encoding damage are likewise render-layer problems. Table-of-contents reflow is the awkward case that genuinely spans both layers: the translator contributes by treating dot leaders as translatable text, and the renderer contributes by reflowing the result, so the fix touches both.
The render-layer answer to these problems is a cleanup stage that runs inside the PDF writer itself — after the content streams are built, but before the file is saved. It is worth emphasizing that this is not an external script or an optional post-processing step. It is a first-class stage of the normal generation pipeline, and treating it as such is what makes its behavior testable and repeatable.
The cleanup stage has grown several distinct capabilities. It reconstructs tables of contents: TOC pages are detected structurally, then redrawn with measured layout — titles aligned left, page numbers aligned right, and dotted leaders filling exactly the space between. It translates running headers that were missed upstream because they appear as image-adjacent or repeated decorative text. It repairs optical character recognition fragments that survived into the translated prose, such as mangled firmware names and broken axis labels. And its pattern maps are gated by target language, so that cleanup rules written for Russian output never fire on French or German documents, where they would do more harm than good.
# Cleanup rules are scoped per target language: a repair written for
# Russian output must never fire on a French or German document.
CLEANUP_RULES: dict[str, list[tuple[re.Pattern, str]]] = {
"ru": [
(re.compile(r"OpenFliqht"), "OpenFlight"), # q/g OCR confusion
(re.compile(r"РЕЖИМЫ\s*\|"), "РЕЖИМЫ"), # stray table pipe
],
"fr": [
(re.compile(r"\bLiftOjf\b"), "Liftoff"),
],
}
def post_render_cleanup(page_text: str, target_lang: str) -> str:
for pattern, repair in CLEANUP_RULES.get(target_lang, []):
page_text = pattern.sub(repair, page_text)
return page_text
Like the protection lists in the translation layer, these cleanup rules accumulate over time. Each new target language starts with a small rule set and grows one as its particular artifacts are discovered in quality control.
A production pipeline should never be coupled to a single model vendor. A provider abstraction maps model identifiers to provider implementations — commercial API vendors alongside self-hosted deployments — so that the rest of the pipeline is indifferent to where translations actually come from. This matters for cost management, for regional availability, and for the simple reality that the best available model changes several times a year.
Every provider implementation carries the same set of protective behaviors, because every provider eventually needs them. Rate limiting caps both queries per second and maximum concurrent workers, tuned per provider. Retries use exponential backoff, so transient failures heal without operator involvement. Repetition-loop detection watches for the failure mode where a model gets stuck emitting the same phrase indefinitely, and cuts the request off rather than paying for garbage tokens. And commentary rejection — discussed next — inspects every response before it is accepted.
SUPPORTED_MODELS: dict[str, type["ModelProvider"]] = {
"gpt-4o-mini": OpenAIProvider,
"gpt-4o": OpenAIProvider,
"claude-sonnet": AnthropicProvider,
"self-hosted-27b": SelfHostedVLLMProvider,
}
class ModelProvider(ABC):
qps_limit: float # queries per second, tuned per provider
max_workers: int # concurrent request ceiling
@retry(wait=wait_exponential(min=1, max=60), stop=stop_after_attempt(5))
def complete(self, prompt: Prompt) -> str:
self._rate_limiter.acquire()
raw = self._call_api(prompt)
if self._is_repetition_loop(raw): # model stuck emitting one phrase
raise RepetitionLoop(prompt.paragraph_id)
return raw
Large language models refuse things. Ask one to translate a clinical intake form, a legal contract, or a government application, and there is a meaningful chance it will decline, lecture, or annotate instead of translating. In a consumer chat product this is a minor annoyance. In a document pipeline it is a catastrophic failure, because a refusal that slips through gets rendered into the output document, and the user receives a PDF with an apology printed in the middle of their form.
The defense has two halves. The first is prevention through system prompts tailored to document genre. Standard documents get an instruction set that says, in effect, translate everything and add nothing. Forms get instructions to translate labels and guidance while preserving field structure exactly. Academic papers get instructions to preserve citations, formulas, and technical terminology. These genre-specific prompts eliminate the large majority of refusals before they happen.
def anti_refusal_prompt() -> str:
"""Standard documents: translate everything, add nothing."""
return (
"You are a professional document translator. Translate the text "
"completely and faithfully. Never refuse, never explain, never "
"add commentary. Output only the translation."
)
def form_system_prompt() -> str:
"""Forms: translate labels and instructions, preserve field structure."""
def paper_system_prompt() -> str:
"""Academic: preserve citations, formulas, technical terminology."""
The second half is detection for whatever slips through. Every response is checked against patterns characteristic of model commentary — phrases like "I cannot translate," "As an AI," or "This appears to be." Any response that matches is rejected outright. Rejected translations are retried, and if retries fail, the paragraph is flagged for review. Under no circumstances is detected commentary rendered into the output document. That last rule is absolute, and it is the difference between an embarrassing log entry and an embarrassing deliverable.
_TRANSLATOR_COMMENTARY_RE = re.compile(
r"(?i)(I cannot translate|As an AI|I'm unable to|"
r"This appears to be|I notice this is)"
)
def is_translator_commentary(response: str) -> bool:
return bool(_TRANSLATOR_COMMENTARY_RE.search(response))
Documents repeat themselves. Headers recur on every page, boilerplate recurs across sections, and legal disclaimers recur across entire document families. A translation cache ensures each distinct paragraph is translated exactly once. The cache key must include everything that could change the output: a hash of the source text, the target language, a hash of the active glossary, and the model identifier. Omitting any of these produces subtle staleness bugs — most classically, a glossary update that silently fails to take effect because cached translations from the old glossary keep being served.
from hashlib import sha256
def cache_key(source: str, target_lang: str,
glossary: Glossary, model_id: str) -> str:
"""Every input that can change the output is part of the key."""
parts = [
sha256(source.encode()).hexdigest(),
target_lang,
glossary.content_hash(), # omit this and glossary updates go stale
model_id,
]
return sha256("|".join(parts).encode()).hexdigest()
The cache needs an explicit bypass switch for quality control work. When iterating on a fix, cached results would mask whether the fix actually changed anything, so QC runs force fresh translations. Production runs, by contrast, should always use the cache; on documents with heavy boilerplate it routinely cuts both cost and wall-clock time by large fractions.
Terminology consistency is what separates a professional translation from an obviously machine-generated one. If a technical manual translates the same component name three different ways in three chapters, readers notice immediately. The pipeline therefore enforces terminology from two sources: glossaries supplied by the user, and terms extracted automatically from the document itself.
Enforcement requires matching every glossary term against every paragraph of source text before translation, and injecting the required renderings into the model's prompt. Done naively, this matching becomes a performance problem — a large operator manual can carry a glossary of many hundreds of terms, and scanning every paragraph against every term linearly turns preprocessing into minutes of dead time. A high-performance multi-pattern matching engine solves this, compiling the entire glossary into a single database that scans each paragraph in one pass. For heavily technical documents, this is the difference between seconds and minutes before translation even begins.
import hyperscan
def build_glossary_db(entries: list[GlossaryEntry]) -> hyperscan.Database:
"""Compile the whole glossary into one multi-pattern database.
One scan per paragraph replaces len(entries) linear scans."""
db = hyperscan.Database()
db.compile(
expressions=[e.source_pattern.encode() for e in entries],
ids=list(range(len(entries))),
flags=[hyperscan.HS_FLAG_CASELESS] * len(entries),
)
return db
def match_glossary_terms(db, entries, paragraph: str) -> dict[str, str]:
hits: dict[str, str] = {}
def on_match(entry_id, start, end, flags, ctx):
hits[entries[entry_id].source] = entries[entry_id].target
db.scan(paragraph.encode(), match_event_handler=on_match)
return hits # injected into the prompt as enforced renderings
User glossaries only cover what the user thought to include. The rest of the document's vocabulary still needs consistent treatment, and this is handled by an extraction pass that runs before the main translation. A model reads through the document and produces a structured list of term pairs: proper nouns that should pass through unchanged, and technical phrases paired with their preferred renderings in the target language. For an English manual translated into Russian, the extraction pass returns something like:
[
{"src": "Halcyon", "tgt": "Halcyon"},
{"src": "FPV goggles", "tgt": "очки FPV"},
{"src": "LiPo battery", "tgt": "аккумулятор LiPo"}
]
This extracted list then feeds the same enforcement machinery as the user glossary. The pass is not cheap — it can consume roughly a third of total pipeline time — but the payoff is dramatic. Proper nouns and technical terms come out uniform across a five-hundred-page manual, instead of varying with each paragraph's local context. For long technical documents, no other single investment improves perceived quality as much.
A brief word on multilingual expectations, because they shape how quality effort is allocated. Language support in a production system is never uniform; it forms tiers. The major world languages — the ones best represented in model training data — produce publication-ready output with the standard pipeline. A second tier produces strong output that benefits from minimal review. A long tail of less-resourced languages produces usable output that needs light proofreading, and typically needs render-layer cleanup rules added iteratively as each language's characteristic artifacts are discovered.
The practical implication is to be honest about these tiers with users, and to treat expansion into a new language as an engineering task with a quality control cycle of its own, not as flipping a configuration flag.
A translation job's life runs from upload to download, and every stage of it needs to be explicit and observable. When a request arrives, the system creates a job with a unique identifier, recorded both in a durable database and in an in-memory live cache for fast status queries. A worker thread launches the translation pipeline as a subprocess and then watches a progress file the pipeline updates continuously, polling it at sub-second intervals so the user's progress bar reflects reality rather than guesswork. When the pipeline finishes, the output documents — typically a translated-only version and a side-by-side bilingual version — are copied to the outputs location, optionally uploaded to object storage, and the job record is updated to its terminal state.
Nothing in this flow is exotic, and that is the point. Every stage writes its state somewhere inspectable, so that when a job misbehaves, an operator can determine exactly how far it got and what it was doing when it stopped.
@app.post("/api/translate")
def create_job(upload: UploadFile, target_lang: str, model: str) -> JobStatus:
job_id = str(uuid4())
db.insert_job(job_id, status="queued")
LIVE_CACHE[job_id] = {"status": "queued", "progress": 0.0}
threading.Thread(
target=run_translation_job,
args=(job_id, upload, target_lang, model),
daemon=True,
).start()
return JobStatus(id=job_id, status="queued")
def run_translation_job(job_id, upload, target_lang, model):
proc = subprocess.Popen(
build_pipeline_command(
upload.path, target_lang, model,
progress_file=progress_path(job_id),
),
env=pipeline_env(),
)
while proc.poll() is None:
LIVE_CACHE[job_id]["progress"] = read_progress(job_id)
if LIVE_CACHE[job_id]["status"] == "cancelled":
proc.kill() # decisive cancellation — no cooperative flags
return
time.sleep(0.4) # sub-second polling keeps the progress bar honest
publish_outputs(job_id) # copy .mono.pdf / .dual.pdf, optional upload
db.update_job(job_id, status="done")
One operational rule pays for itself many times over: all pipeline configuration lives in exactly one module. That module builds the command-line arguments — model selection, concurrency, rate limits, watermarking, progress reporting — and it builds the environment the pipeline runs in, covering thread caps, cache directories, and encoding settings. The API server imports from this module. The standalone command-line tool imports from this module. Nothing else constructs pipeline invocations.
# pipeline_runner.py — the ONLY module that builds pipeline invocations.
def build_pipeline_command(input_pdf, target_lang, model, *,
pages=None, ignore_cache=False,
progress_file=None) -> list[str]:
cmd = [
"translate-cli",
"--input", input_pdf,
"--target", target_lang,
"--model", model,
"--qps", str(qps_for(model)),
"--workers", str(memory_aware_worker_cap(DEFAULT_WORKERS)),
]
if pages:
cmd += ["--pages", pages] # e.g. "7-10,24-25" for partial regen
if ignore_cache:
cmd += ["--ignore-cache"]
if progress_file:
cmd += ["--progress-file", progress_file]
return cmd
def pipeline_env() -> dict[str, str]:
return {
**os.environ,
"OMP_NUM_THREADS": "2", # cap native thread pools
"CACHE_DIR": str(RUNTIME_CACHE), # translation cache location
"AUTO_FIT_FILL": "1",
"PYTHONUTF8": "1",
}
The failure mode this prevents is configuration drift, and it is worth spelling out because it is so common. The moment two callers each assemble their own flags, they begin to disagree — quietly, and then damagingly. A bug reproduces through the server but not the CLI, or a performance setting applies in one path but not the other, and engineers lose days to phantom differences that are really just two copies of the same configuration slowly diverging. One source of truth makes that entire category of bug impossible.
Long-running jobs need clean cancellation. The pipeline runs as a subprocess precisely so that cancellation can be decisive: the server polls the live cache for a cancelled status and kills the subprocess when it sees one, rather than hoping cooperative flags get checked somewhere deep in a translation loop.
Several other operational defenses earn their place in any production deployment. Provider warmup checks confirm that a model backend is actually ready before the job starts, which matters greatly for self-hosted models on GPU infrastructure where cold starts can take minutes — without the check, the first minutes of a job are spent failing against a backend that is not yet awake. Memory-aware worker caps size the concurrency to the machine, preventing out-of-memory crashes on smaller instances. And every production exception is captured with its full job context attached, so a stack trace arrives with the job identifier, document characteristics, and pipeline stage that produced it.
def memory_aware_worker_cap(requested: int) -> int:
"""Never schedule more workers than the machine can feed."""
available_gb = psutil.virtual_memory().available / 1e9
return max(1, min(requested, int(available_gb // 1.5)))
def wait_for_provider_ready(provider, timeout=300):
"""Block until a self-hosted backend answers a health probe —
GPU cold starts can take minutes."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if provider.healthcheck():
return
time.sleep(5)
raise ProviderNotReady(provider.model_id)
Finally, real-world documents demand escape hatches. Some PDFs arrive with font encodings so damaged that the normal parsing path crashes outright. For these, a safe mode exists: each page is translated independently and in parallel, results are merged into a single output document, and the sophisticated intermediate-layout machinery is bypassed entirely. Output quality is lower — that is the trade — but a lower-quality translation delivered is worth more than a perfect translation that crashed. Safe mode is a fallback, never the default, but production systems without such fallbacks simply fail on the documents that need them most.
When a user reports translation artifacts in a five-hundred-page manual, the temptation is to guess at a fix and regenerate the whole document to see if it worked. That loop takes half an hour per attempt and teaches almost nothing. The disciplined loop is faster and converges.
First, identify the affected pages, either from the user's report or by searching the extracted PDF text for the artifact. Second, inspect the translation audit trail for those paragraphs, and determine the exact input shape — the placeholder layout, the surrounding context — that produced the bad output. Third, write a placeholder-boundary test that reproduces the failure with that exact shape; watching it fail confirms the diagnosis. Fourth, fix the bug in the correct layer, translator or render, using the audit-trail visibility rule to decide which. Fifth, regenerate only the affected pages, with the cache bypassed, using the pipeline's partial-page selection. Sixth, run the extraction-based quality check against the bad and good marker lists. Only when the targeted pages pass does the seventh step, full-document regeneration, happen at all.
The regeneration step of the loop is a single command:
# Regenerate only the affected pages, bypassing the translation cache
translate-cli --input manual.pdf --target ru \
--pages "7-10,24-25" --ignore-cache
Partial-page regeneration is what makes this loop practical. It turns a thirty-minute iteration into roughly two minutes, which is the difference between trying one hypothesis per session and trying a dozen. If a pipeline supports only whole-document runs, adding page selection should be considered urgent operational work, not a nice-to-have.
The extraction-based checks should not remain a manual ritual. Spot-check scripts that pull text from generated documents and report marker hits belong in the repository alongside the pipeline itself, and the marker lists belong in version control, growing with every fixed bug. Run in continuous integration against a small suite of representative documents, they convert every past bug into a permanent regression guard. The document that surfaced a bug becomes, forever after, the document that proves the bug has not returned.
@pytest.mark.parametrize("doc", REPRESENTATIVE_DOCS)
def test_no_known_artifacts(doc):
output_pdf = translate(doc, target="ru")
report = qc_check(output_pdf)
assert report.passed, (
f"bad markers present: {report.bad_hits}, "
f"good markers missing: {report.good_misses}"
)
A single production fix cycle ties all of these threads together. The document was an English drone operator's manual, several hundred pages, translated into Russian. The user's report listed three distinct artifacts. Product names were garbled — the controller name "Vertex T15" appeared throughout as "Vertex 5T1", with the code transposed. The table of contents was broken, with page numbers fused into strings like "372.3" instead of appearing as separate right-aligned numbers. And the running headers at the top of each page were still in English, untranslated, on an otherwise fully translated document.
The three artifacts turned out to have three different root causes, in three different layers — which is precisely why the diagnostic discipline matters.
The garbled product names traced to the placeholder layer: the short code "T15" was being classified as a mathematical formula and hidden behind a placeholder, so the model never saw the full product name, and reassembly scrambled it. The audit trail showed the damage clearly, marking this as a translator-layer bug. The broken table of contents was a hybrid: dot leaders were being treated as translatable text and becoming separate paragraphs, after which the renderer reflowed the fragments into fused numbers — a fix was needed on both sides. The untranslated running headers never appeared in the audit trail at all: they were image-adjacent text that bypassed the translator completely, marking them unambiguously as render-layer work.
The repairs followed the layers. The formula-inlining classifier gained patterns for short model codes, so fragments like "T15" and "04X" stayed inline as ordinary text. Token protection patterns were added for the product names appearing in the manual. A table-of-contents normalization routine was added to the translator to repair section numbering, and a post-render cleanup stage was built into PDF generation to reconstruct TOC layout and translate the running headers the parser could not reach.
Verification ran through all three acceptance layers, ending with extraction from the regenerated PDF: every bad marker absent, every good marker present, across all checked pages. Partial-page regeneration carried the whole investigation — each hypothesis cost about two minutes to test instead of thirty — and the input shapes recovered from the audit trail became permanent regression tests.
Seven lessons summarize the handbook, and each of them was learned the expensive way.
First, unit tests on plain strings are necessary but insufficient. They validate functions, not pipelines, and the pipeline is where translation bugs live. Always test the placeholder-boundary shapes the translator actually sees, and always validate against text extracted from the final document.
Second, placeholders are the hardest problem in the system. Product codes, formulas, OCR artifacts, and numeric ranges all compete for the same protection machinery, and every one of them fails differently. Inline aggressively when fragments are really text, protect known terms explicitly, and normalize after every model call.
Third, fix bugs in the correct layer. Translator fixes do not help render-layer artifacts, and speculative fixes in the wrong layer add complexity while the real bug survives. The audit trail settles the question: if the artifact is not visible there, the translator is innocent.
Fourth, system prompts prevent refusals, and detection catches the rest. Forms, clinical documents, and legal paperwork need explicit instructions to translate everything without commentary, and every response needs to be screened before it can reach a rendered page.
Fifth, cache aggressively and invalidate surgically. Key the cache on everything that affects output — source text, target language, glossary, and model — and bypass it only during quality control iterations, never in production runs.
Sixth, a single configuration source prevents drift. One module builds the pipeline's arguments and environment for every caller, and the entire category of "works on the server but not the CLI" bugs disappears.
Seventh, partial-page regeneration is essential for iteration speed. Without it, every fix attempt costs a full pipeline run, and the quality control loop becomes so slow that fixes stop being attempted.
AI-powered document translation is not an API integration problem. It is a quality engineering problem, and the model call is genuinely the easy part. The hard parts are protecting non-translatable content through a placeholder system that respects the differences between formulas, codes, and identifiers; normalizing the artifacts that OCR and the models themselves introduce, both before and after translation; fixing the render-layer issues that bypass the translator entirely and therefore evade every translator-level test; validating every fix against text extracted from the generated document rather than against string comparisons; and operating the whole pipeline with real cancellation, real progress reporting, and real error context.
Build the three-layer acceptance model into the workflow from the first week of the project. It is the cheapest insurance available against the most demoralizing sentence in this line of work: the test passes, but the document is still wrong.
%402x.svg){kind=icon}