54 lines
1.9 KiB
Markdown
54 lines
1.9 KiB
Markdown
# OCR Troubleshooting — Binarization Fallback
|
|
|
|
## Problem
|
|
Some scanned PDF pages return empty text when processed with standard `ocrmypdf` + `pdftotext` or `tesseract`. The pages appear to have content visually but the OCR engine produces zero characters.
|
|
|
|
## Diagnosis
|
|
Check if tesseract output is empty:
|
|
```bash
|
|
tesseract page.jpg stdout -l chi_sim | wc -c
|
|
# If 0 chars → page needs binarization
|
|
```
|
|
|
|
Check image statistics to confirm it's not truly blank:
|
|
```python
|
|
from PIL import Image
|
|
import numpy as np
|
|
arr = np.array(Image.open('page.jpg').convert('L'))
|
|
print(f'mean={arr.mean():.0f} std={arr.std():.0f}')
|
|
# If std > 25 → image has content, OCR failure is processing issue
|
|
```
|
|
|
|
## Fix: Binarization (threshold=140)
|
|
Convert to pure black-and-white before re-running tesseract:
|
|
|
|
```python
|
|
from PIL import Image
|
|
import numpy as np
|
|
|
|
img = Image.open('page.jpg').convert('L')
|
|
arr = np.array(img)
|
|
threshold = 140 # Adjust if needed (120-160 range)
|
|
arr = (arr < threshold).astype(np.uint8) * 255
|
|
Image.fromarray(arr).save('page_bw.png')
|
|
```
|
|
|
|
Then run tesseract on the binarized image:
|
|
```bash
|
|
tesseract page_bw.png stdout -l chi_sim
|
|
```
|
|
|
|
## Vision Tool Timeout Fix
|
|
If `vision_analyze` times out on scanned pages:
|
|
1. Check `~/.hermes/config.yaml` → `vision.timeout` (default 30s is too short for large images)
|
|
2. Increase to 90s: `sed -i 's/timeout: 30/timeout: 90/' ~/.hermes/config.yaml`
|
|
3. Compress images before sending: resize to 850x1100, quality=60 (~80-120KB per page)
|
|
4. Send one page at a time, not multiple simultaneously
|
|
|
|
## Workflow for Scanned Contracts
|
|
1. `ocrmypdf -l chi_sim --skip-text input.pdf output_ocr.pdf`
|
|
2. `pdftotext output_ocr.pdf - | wc -c` — check if text layer exists
|
|
3. If empty pages: `pdftoppm -jpeg -r 300 input.pdf pages/page` → binarize → tesseract
|
|
4. Combine: original tesseract pages + binarized pages into single .md file
|
|
5. For remaining garbled fields: use `vision_analyze` on compressed page images
|