PDF OCR Troubleshooting for Scanned Chinese Contracts

Lesson learned: 2026-06-30 — 通州金鹰 (23MB lease + 1.4MB property contract)

Problem Pattern: Mixed OCR Results

Scanned PDFs of Chinese contracts often produce:

Some pages: clean OCR text via ocrmypdf + pdftotext/pymupdf
Other pages: completely empty output (0 chars)
Other pages: garbled character soup (\u0000 null bytes, random symbols)

Diagnosis

from PIL import Image
import numpy as np

for i in range(1, N+1):
    img = Image.open(f'pages/page-{i}.jpg').convert('L')
    arr = np.array(img)
    mean = arr.mean()  # ~160-170 for scanned contracts
    std = arr.std()    # ~30-37 for text-heavy pages
    # If tesseract returns 0 chars but mean/std look normal → needs preprocessing

Fix 1: Binary Threshold Conversion

Standard tesseract OCR produces empty output on some scanned pages. The fix:

from PIL import Image
import numpy as np

for i in empty_pages:
    img = Image.open(f'pages/page-{i}.jpg').convert('L')
    arr = np.array(img)
    threshold = 140  # Key value: too high = lose text, too low = noise
    arr = (arr < threshold).astype(np.uint8) * 255
    bw = Image.fromarray(arr)
    bw.save(f'pages/page-{i}_bw.png')
    # Then: tesseract pages/page-{i}_bw.png stdout -l chi_sim

Why this works: Some scanners produce pages where the background is very slightly off-white (mean ~168 vs pure white 255). Tesseract's adaptive thresholding fails on these. Hard binary threshold at 140 separates text (dark, <140) from background (>140) cleanly.

Fix 2: Vision Service Timeout

vision_analyze times out on large images (>200KB at 300dpi). Always compress:

from PIL import Image
img = Image.open(f'pages/page-{i}.jpg')
img = img.resize((img.width // 3, img.height // 3), Image.LANCZOS)
img.save(f'pages/page-{i}_small.jpg', quality=60)
# Target: 80-95KB per page at 850x1100 pixels

Fix 3: ocrmypdf Text Layer Extraction Failure

Even after ocrmypdf, the embedded text layer may be garbled when extracted via pdftotext or pymupdf. This is a known issue with ocrmypdf + tesseract for Chinese text.

Solution: Don't rely on ocrmypdf's text layer. Instead:

Use ocrmypdf to produce the OCR'd PDF (for archiving)
Separately extract pages as images: pdftoppm -jpeg -r 300 input.pdf pages/page
Run tesseract directly on each image
For empty pages, apply binary threshold (Fix 1) and retry

Workflow: Complete OCR Pipeline

# 1. Extract pages as images
pdftoppm -jpeg -r 300 "contract.pdf" pages/page

# 2. Standard tesseract on all pages
for f in pages/page-*.jpg; do
    tesseract "$f" "${f%.jpg}" -l chi_sim 2>/dev/null
done

# 3. Identify empty pages (0 chars)
for f in pages/page-*.txt; do
    chars=$(wc -c < "$f")
    if [ "$chars" -lt 10 ]; then
        echo "EMPTY: $f"
    fi
done

# 4. Binary threshold + retry on empty pages (Python)
# 5. Vision for remaining empty pages (compress to <100KB first)
# 6. Concatenate all page texts into final OCR file

Known OCR Garble Patterns in Chinese Contracts

OCR Output	Likely Value	Context
"于65" / "也65" / "了芋65"	765	Area in ㎡
"巧" / "葬"	15 / 30	Working days
"101"	10	Percentage
"0.1‰%o"	0.1‰	Daily penalty rate
"直通市赴"	南通市通州区	City name
"驳玉年"	2025年	Year
"瑞殉年"	2025年	Year
"嫂万元整"	肆万元整	Amount in Chinese
"103"	10	Working days

Rule: Never guess OCR values. Mark as "⚠️待核实原件" in the Excel and list in 需客户核实 section.

3.6 KiB Raw Blame History