55 lines
2.9 KiB
Markdown
55 lines
2.9 KiB
Markdown
# Output Style Consistency — Three-Layer Defense
|
|
|
|
## Problem (2026-06-30 Maggie discovery)
|
|
When processing multiple campuses in separate sessions, the LLM's data-extractor output drifts in style because earlier campus outputs are no longer in context. The first batch (解放中路→星月, 7 campuses) was consistent due to context anchoring — each campus saw the previous output and copied its style. When 跃龙路 was processed in a new session, the LLM had no reference and generated a different style:
|
|
- H列: bullet-list format instead of paragraph-style
|
|
- K列: statistical tag format ("10项风险（3高/5中/2低）") instead of narrative ("【整体评价】...")
|
|
- L列: categorized headers ("【核心缺失16项】") instead of numbered list
|
|
|
|
Maggie: "金额，风险，模版对比各列内容的审查和修改意见表达都不一样了"
|
|
|
|
## Root Cause
|
|
LLM context anchoring effect: style consistency comes from seeing previous outputs in context, not from prompt instructions alone. When context window resets between sessions, the anchoring is lost.
|
|
|
|
## Three-Layer Defense (implemented in nantong-lease-audit v2 workflow)
|
|
|
|
### Layer 1: Fixed Templates in Workflow Prompt
|
|
In the data-extractor role's procedure, embed concrete style examples with explicit "禁止" (prohibited) patterns:
|
|
|
|
```yaml
|
|
# H列 format: paragraph-style with 【】category headers, no bullets
|
|
# K列 format: 【整体评价】opening + ❗【需客户核实】numbered list
|
|
# L列 format: one-line relationship statement + numbered diff list
|
|
# J列: exactly "履行中"/"已到期"/"已解除", no parenthetical explanations
|
|
```
|
|
|
|
### Layer 2: Reference Loading Before Generation
|
|
Before generating data for a new campus, load the most recent completed campus's xlsx and extract H/K/L content as style anchor:
|
|
|
|
```bash
|
|
REF_XLSX=$(ls -t /path/to/房租物业合同/*/*梳理*.xlsx 2>/dev/null | head -1)
|
|
```
|
|
|
|
Read with openpyxl, inject into prompt context. LLM sees "previous campus looks like this" and naturally aligns.
|
|
|
|
### Layer 3: Post-Processing Validation
|
|
Run `output-style-check.py` after generation, before upload:
|
|
|
|
```bash
|
|
python3 ~/.hermes/scripts/output-style-check.py <campus>-row-data.json
|
|
```
|
|
|
|
Checks 8 rules:
|
|
- H列: no bullet symbols (•/-/*), no "【category·clause】" merged headers
|
|
- K列: no statistical openings ("X项风险(Y高/Z中)"), no "序号·等级·条款号" tags, no markdown tables, must have 【整体评价】
|
|
- L列: no statistical openings ("X处差异(Y缺失/Z修改)"), no "【核心缺失/修改】" sub-headers, no "vs" separators
|
|
- J列: exact match "履行中"/"已到期"/"已解除"
|
|
|
|
Exit 0 = pass, exit 1 = list violations for fix.
|
|
|
|
## Workflow Integration
|
|
In nantong-lease-audit.yaml v2 (hash C77579MQ9QPKE), the data-extractor procedure sections 5-6 implement layers 1-2. Layer 3 is run manually after workflow completes, before xlsx generation.
|
|
|
|
## Script Location
|
|
`~/.hermes/scripts/output-style-check.py` — standalone, no dependencies beyond stdlib json/re/sys.
|