OCR Troubleshooting — Binarization Fallback

Problem

Some scanned PDF pages return empty text when processed with standard ocrmypdf + pdftotext or tesseract. The pages appear to have content visually but the OCR engine produces zero characters.

Diagnosis

Check if tesseract output is empty:

tesseract page.jpg stdout -l chi_sim | wc -c
# If 0 chars → page needs binarization

Check image statistics to confirm it's not truly blank:

from PIL import Image
import numpy as np
arr = np.array(Image.open('page.jpg').convert('L'))
print(f'mean={arr.mean():.0f} std={arr.std():.0f}')
# If std > 25 → image has content, OCR failure is processing issue

Fix: Binarization (threshold=140)

Convert to pure black-and-white before re-running tesseract:

from PIL import Image
import numpy as np

img = Image.open('page.jpg').convert('L')
arr = np.array(img)
threshold = 140  # Adjust if needed (120-160 range)
arr = (arr < threshold).astype(np.uint8) * 255
Image.fromarray(arr).save('page_bw.png')

Then run tesseract on the binarized image:

tesseract page_bw.png stdout -l chi_sim

Vision Tool Timeout Fix

If vision_analyze times out on scanned pages:

Check ~/.hermes/config.yaml → vision.timeout (default 30s is too short for large images)
Increase to 90s: sed -i 's/timeout: 30/timeout: 90/' ~/.hermes/config.yaml
Compress images before sending: resize to 850x1100, quality=60 (~80-120KB per page)
Send one page at a time, not multiple simultaneously

Workflow for Scanned Contracts

ocrmypdf -l chi_sim --skip-text input.pdf output_ocr.pdf
pdftotext output_ocr.pdf - | wc -c — check if text layer exists
If empty pages: pdftoppm -jpeg -r 300 input.pdf pages/page → binarize → tesseract
Combine: original tesseract pages + binarized pages into single .md file
For remaining garbled fields: use vision_analyze on compressed page images

1.9 KiB Raw Blame History