feat: export core Hermes skills
This commit is contained in:
@@ -0,0 +1,91 @@
|
||||
# 合同审查完整性审计方法 (Audit Methodology)
|
||||
|
||||
## 触发条件
|
||||
|
||||
Doro说"查一查""核查""核实""是不是都审查了/pass了/登记了"→ 这是**验证指令**。
|
||||
|
||||
## 铁律
|
||||
|
||||
**回复中必须先有工具调用再有结论。context记忆≠查证,不可直接输出。**
|
||||
|
||||
## 时区转换(铁律)
|
||||
|
||||
服务器时区UTC,Maggie/Doro/邱律师北京时间(UTC+8)。当问"今天发了多少"时:
|
||||
- **北京时间7月3日** = UTC 7月2日 16:00 ~ 7月3日 16:00
|
||||
- `find` 命令用 `-newermt "2026-07-02 16:00:00" ! -newermt "2026-07-03 16:00:00"`
|
||||
- 或 `TZ='Asia/Shanghai' date` 确认当前北京时间
|
||||
|
||||
**典型错误**:用UTC当天(00:00-24:00)筛选→会把北京时间前一天下午的文件算进来、漏掉当天上午的文件。2026-07-03教训:初始查询用 `-mtime -1` 返回了UTC时间范围的文件(含前一天的6份),Maggie追问"北京时间7/3的"后改用精确UTC窗口,确认只有1份。
|
||||
|
||||
## 必须覆盖的数据源(缺一不可)
|
||||
|
||||
### 1. Tracker JSON
|
||||
```bash
|
||||
cat ~/.hermes/data/contract-tracker.json
|
||||
```
|
||||
- 按seq范围筛选
|
||||
- 逐条列出original_filename, party, status, xlsx_updated_at
|
||||
|
||||
### 2. xlsx(与tracker交叉比对)
|
||||
```bash
|
||||
sudo docker cp nextcloud-nextcloud-1:/var/www/html/data/doro/files/Doro合同审查任务/合同审查清单.xlsx /tmp/
|
||||
```
|
||||
- openpyxl读取,逐行比对tracker
|
||||
|
||||
### 3. Gateway log - 全部接收渠道
|
||||
```bash
|
||||
# QiuTing私信文件(空消息=文件附件)
|
||||
grep 'user=QiuTing.*chat=QiuTing' gateway.log | grep "msg=''"
|
||||
|
||||
# Doro私信文件
|
||||
grep 'user=doro.*chat=doro' gateway.log | grep "msg=''"
|
||||
|
||||
# Doro群文件
|
||||
grep 'user=doro.*chat=wrbAFkXAAAiWC3styKqNj0bZyH6BbJ_Q' gateway.log | grep "msg=''"
|
||||
|
||||
# Doro "待审查上传"指令(表示Doro直接往Nextcloud上传了文件)
|
||||
grep 'user=doro' gateway.log | grep -i '待审查.*上传\|上传.*新.*合同'
|
||||
|
||||
# 飞书渠道
|
||||
grep 'feishu.*ou_757f053c9d7aff6c73b18aa60c337756' gateway.log | grep 'media='
|
||||
```
|
||||
|
||||
### 4. Nextcloud目录实时状态
|
||||
```bash
|
||||
# 待审查(原文件仍在=未pass或等cleanup)
|
||||
sudo docker exec nextcloud-nextcloud-1 ls Doro合同审查任务/待审查/
|
||||
|
||||
# 任务交付(交付物)
|
||||
sudo docker exec nextcloud-nextcloud-1 ls Doro合同审查任务/任务交付/
|
||||
```
|
||||
|
||||
### 5. 交叉比对
|
||||
- 接收总数(各渠道文件消息数之和)
|
||||
- 处理总数(tracker completed + 待pass + 跳过 + 排除)
|
||||
- 差值 = 可能遗漏
|
||||
|
||||
## 汇报格式
|
||||
|
||||
```
|
||||
=== 查证方法 ===
|
||||
1. 读了什么(tracker/xlsx/gateway log哪些渠道/Nextcloud哪些目录)
|
||||
2. 每个数据源的结果数
|
||||
|
||||
=== 查证结果 ===
|
||||
- 已pass登记:X份(seq范围)
|
||||
- 已交付未pass:X份(列出文件名)
|
||||
- 已排除:X份(原因)
|
||||
- 差异/存疑:X份(说明)
|
||||
|
||||
=== 无法确认的 ===
|
||||
- 明确说"这些我查不到/确认不了"
|
||||
- 说明已尝试的搜索策略
|
||||
```
|
||||
|
||||
## 反面教材(2026-07-02)
|
||||
|
||||
❌ "查证属实,无遗漏" → 实际没跑任何工具
|
||||
❌ "找不到第2份" → 实际数据在tracker里,自己之前还列过表
|
||||
❌ "你记得叫什么名字吗?" → 把验证责任转嫁用户
|
||||
|
||||
✅ 正确做法:跑完全部5个数据源 → 列出原始数据 → 标注不确定项 → 再给结论
|
||||
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Audit 待审查 and 任务交付 directories against tracker.
|
||||
|
||||
Usage:
|
||||
python3 ~/.hermes/skills/legal/contract-pass-workflow/references/cleanup-audit.py
|
||||
|
||||
Prints a matrix showing which files are:
|
||||
- In tracker (and their status/age/cleaned flag)
|
||||
- NOT in tracker (orphans that will never be auto-cleaned)
|
||||
- Should be cleaned (>24h + completed + cleaned=false)
|
||||
|
||||
Does NOT delete anything. Pure diagnostic.
|
||||
"""
|
||||
import json, os, subprocess
|
||||
from datetime import datetime, timezone, timedelta
|
||||
|
||||
BJT = timezone(timedelta(hours=8))
|
||||
now = datetime.now(BJT)
|
||||
|
||||
TRACKER = os.path.expanduser('~/.hermes/data/contract-tracker.json')
|
||||
BASE = os.path.expanduser('~/nextcloud/data/data/doro/files/Doro合同审查任务')
|
||||
待审查 = os.path.join(BASE, '待审查')
|
||||
任务交付 = os.path.join(BASE, '任务交付')
|
||||
|
||||
def nc_ls(path):
|
||||
"""List files in a Nextcloud-managed directory (needs sudo)."""
|
||||
r = subprocess.run(['sudo', 'ls', path], capture_output=True, text=True)
|
||||
return [f for f in r.stdout.strip().split('\n') if f] if r.stdout.strip() else []
|
||||
|
||||
def file_age_hours(path):
|
||||
"""Get file age in hours from mtime."""
|
||||
r = subprocess.run(['sudo', 'stat', '-c', '%Y', path], capture_output=True, text=True)
|
||||
if r.stdout.strip():
|
||||
mtime = int(r.stdout.strip())
|
||||
return (now - datetime.fromtimestamp(mtime, tz=BJT)).total_seconds() / 3600
|
||||
return -1
|
||||
|
||||
def main():
|
||||
with open(TRACKER) as f:
|
||||
tracker = json.load(f)
|
||||
|
||||
# Build lookup: filename -> list of tracker records
|
||||
lookup = {}
|
||||
for c in tracker['contracts']:
|
||||
for key in ['delivered_filename', 'original_filename', 'converted_filename']:
|
||||
fn = c.get(key, '')
|
||||
if fn:
|
||||
lookup.setdefault(fn, []).append(c)
|
||||
|
||||
orphans = []
|
||||
|
||||
for label, directory in [('待审查', 待审查), ('任务交付', 任务交付)]:
|
||||
print(f"\n{'='*70}")
|
||||
print(f" {label} ({directory})")
|
||||
print(f"{'='*70}")
|
||||
files = nc_ls(directory)
|
||||
if not files:
|
||||
print(" (empty)")
|
||||
continue
|
||||
|
||||
for fn in sorted(files):
|
||||
records = lookup.get(fn, [])
|
||||
age = file_age_hours(os.path.join(directory, fn))
|
||||
is_companion = any(k in fn for k in ['审查意见', '合同流程单'])
|
||||
|
||||
if records:
|
||||
for r in records:
|
||||
ts = r.get('xlsx_updated_at', '')
|
||||
h = (now - datetime.fromisoformat(ts)).total_seconds() / 3600 if ts else -1
|
||||
should = r['status'] == 'completed' and h > 24
|
||||
flag = '🔴 SHOULD_CLEAN' if should else '⏳ waiting'
|
||||
print(f" {fn}")
|
||||
print(f" seq={r['seq']} | {h:.0f}h | cleaned={r.get('cleaned')} | {flag}")
|
||||
else:
|
||||
tag = '📋 COMPANION_ORPHAN' if is_companion else '⚠️ NOT_IN_TRACKER'
|
||||
print(f" {fn}")
|
||||
print(f" {tag} | age={age:.0f}h")
|
||||
orphans.append((label, fn, age))
|
||||
|
||||
if orphans:
|
||||
print(f"\n{'='*70}")
|
||||
print(f" ORPHANS SUMMARY: {len(orphans)} files not tracked")
|
||||
print(f"{'='*70}")
|
||||
for label, fn, age in orphans:
|
||||
print(f" [{label}] {fn} ({age:.0f}h old)")
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,23 @@
|
||||
# Companion 文件清理修复方案(待 WeiWei 实施)
|
||||
|
||||
## 问题
|
||||
cleanup cron (`contract-cleanup.py`) 只删 tracker 里有记录的文件。审查意见、合同流程单等 companion 文件按规则不单独写 tracker,主合同被清理后 companion 成为孤儿,永远留在 `任务交付/`。
|
||||
|
||||
## 2026-06-29 实证
|
||||
清理了 11 个 orphan companion 文件(6个合同流程单 xlsx、5个审查意见 docx),最老的残留 91 小时。
|
||||
|
||||
## 修复方案(方案 A,推荐)
|
||||
|
||||
### 1. pass workflow (skill) 改动
|
||||
步骤1写 tracker 时,扫描 `任务交付/` 目录,找到与主合同同名的 companion 文件,写入 `companion_files` 字段。
|
||||
|
||||
### 2. cleanup 脚本改动
|
||||
删主合同时,读 `companion_files` 字段,一并删除。
|
||||
|
||||
## .doc 原始文件残留修复
|
||||
|
||||
cleanup 删 `original_filename` 时,如果文件不存在,尝试同名但换扩展名(.doc 换 .docx 或反之)。
|
||||
|
||||
## 状态
|
||||
- 2026-06-29:方案已提交给 WeiWei 讨论
|
||||
- 待 WeiWei 确认后实施代码改动
|
||||
@@ -0,0 +1,36 @@
|
||||
# Companion 文件清理方案(待 WeiWei 决策)
|
||||
|
||||
## 问题
|
||||
`contract-cleanup.py` 只删 tracker 里 `original_filename` / `converted_filename` / `delivered_filename` 三个字段精确匹配的文件。companion 文件(审查意见、合同流程单)从不进 tracker → 主合同被清理后 companion 成孤儿,永远残留。
|
||||
|
||||
## 方案 A:tracker 增加 companion_files 字段(推荐)
|
||||
|
||||
**改动1:pass workflow(skill contract-pass-workflow)**
|
||||
步骤1写 tracker 时,扫描 `任务交付/` 目录,找到与主合同同目录且包含"审查意见"/"合同流程单"的文件,写入:
|
||||
```json
|
||||
"companion_files": ["【审】购销合同 审查意见.docx", "【审】合同流程单-xxx.xlsx"]
|
||||
```
|
||||
|
||||
**改动2:cleanup 脚本(~/.hermes/scripts/contract-cleanup.py)**
|
||||
删主合同时,读 `companion_files` 字段,逐个 `docker exec rm`。
|
||||
|
||||
**优点**:精确匹配,不误删。
|
||||
**缺点**:需改两处(skill + 脚本)。旧记录无此字段,但不影响——旧 companion 已手动清理或无价值。
|
||||
|
||||
## 方案 B:cleanup 按 stem 模糊匹配
|
||||
|
||||
**改动:仅 cleanup 脚本**
|
||||
删主合同时,取 `delivered_filename` 的 stem(去扩展名),在 `任务交付/` 目录找 `*{stem}*审查意见*` / `*{stem}*合同流程单*` 一并删除。
|
||||
|
||||
**优点**:只改一处。
|
||||
**缺点**:模糊匹配有误删风险(如两个合同名相近)。
|
||||
|
||||
## 附加修复:.doc 原始文件残留
|
||||
|
||||
**问题**:tracker 的 `original_filename` 有时记录了 `.docx`(转换后文件名)而非 `.doc`(真正的原始文件),cleanup 按 `.docx` 去删找不到 `.doc` → 残留。
|
||||
|
||||
**修复**:cleanup 脚本删 `original_filename` 时,如果文件不存在,尝试换扩展名(`.doc` ↔ `.docx`)再找一次。兜底逻辑,不影响正常流程。
|
||||
|
||||
## 状态
|
||||
- 2026-06-29:方案已提出,Doro 未授权执行
|
||||
- 需 WeiWei 决策后实施
|
||||
@@ -0,0 +1,36 @@
|
||||
# Contract Processing Completeness Audit
|
||||
|
||||
When asked "have all contracts been reviewed/registered" for a date range, follow this exhaustive verification method. Do NOT answer from memory — every claim must be tool-verified.
|
||||
|
||||
## Input Channels to Check (ALL of these)
|
||||
|
||||
1. **QiuTing private messages** — gateway.log `user=QiuTing chat=QiuTing msg=''` (empty msg = file)
|
||||
2. **Doro private messages** — gateway.log `user=doro chat=doro msg=''` (empty msg = file)
|
||||
3. **Doro group messages** — gateway.log `user=doro chat=wrbAFkXAAAiWC3styKqNj0bZyH6BbJ_Q msg=''`
|
||||
4. **Feishu messages** — gateway.log feishu platform entries with `media=` indicators
|
||||
5. **Direct Nextcloud uploads** — Doro uploads directly to 待审查/ without going through gateway (indicated by Doro saying "待审查里上传了新合同" without a preceding file message)
|
||||
|
||||
## Cross-Reference Procedure
|
||||
|
||||
```
|
||||
Step 1: Count ALL file-receive events per channel in date range (gateway.log grep)
|
||||
Step 2: Read tracker JSON — list all entries in date range by seq
|
||||
Step 3: Read xlsx — verify 1:1 match with tracker
|
||||
Step 4: Check 待审查/ directory for unprocessed files
|
||||
Step 5: Check 任务交付/ for delivered but un-tracked files
|
||||
Step 6: Reconcile: total received (Step 1) vs total tracked (Step 2)
|
||||
Account for: duplicates (Doro said "重复了 不用审了"), non-contracts (excluded),
|
||||
multi-file batches, files awaiting pass
|
||||
```
|
||||
|
||||
## Critical Rule
|
||||
|
||||
**Never say "查证属实无遗漏" unless ALL channels have been checked and reconciled.** If a channel cannot be fully verified (e.g., log doesn't record filenames for empty messages), state the uncertainty explicitly.
|
||||
|
||||
## Pitfalls (from 2026-06-29 incident)
|
||||
|
||||
- Doro sends files via private chat AND uploads directly to Nextcloud — both channels must be checked
|
||||
- Gateway log records `msg=''` for file messages but does NOT record the filename — you cannot map file→contract from log alone
|
||||
- When Doro says "2份 顾问单位是X", the number must be verified against tracker entries matching that party
|
||||
- A contract may be tracked under a different party name than expected (e.g., "重固卫生服务中心" contract could be filed under the actual contract title without "重固" in it)
|
||||
- Session memory is NOT verification — "I remember processing it" is not evidence
|
||||
@@ -0,0 +1,51 @@
|
||||
# Manual Review Checklist (2026-07-13 session)
|
||||
|
||||
When Doro asks to "审查workflow修改的情况" on delivered contracts, follow this checklist:
|
||||
|
||||
## Process
|
||||
1. **Read the审查规则 first** — full text of review-rules-root.md
|
||||
2. **Get both files**: original from 待审查/ + delivered from 任务交付/
|
||||
3. **Identify ALL WB modifications** — list every WB INS/DEL with paragraph number
|
||||
4. **Distinguish WB from original revisions** — other authors (WB-1, 86187, etc.) are original, don't touch
|
||||
5. **Check each WB modification against rules** — one by one
|
||||
6. **Read full accepted text** — check for语句不通顺, especially at INS boundaries
|
||||
7. **Fix problems directly** — don't ask Doro if you should fix. Just fix.
|
||||
8. **Report findings** — list what's correct and what's wrong
|
||||
9. **Wait for Doro to say pass** — never self-initiate pass
|
||||
|
||||
## Common Workflow Issues Found (2026-07-13)
|
||||
|
||||
### 1. INS rFonts多余属性
|
||||
Workflow consistently adds `hAnsi`, `cs`, `hint` to WB INS runs even when original runs only have `eastAsia` + `ascii`. Fix: strip these three attributes from all WB INS rPr/rFonts.
|
||||
|
||||
### 2. Sub-numbering not updated
|
||||
When workflow inserts a new chapter (e.g. 第七条转包), it changes chapter headings (七→八, 八→九) but does NOT change sub-clause numbering (7.1→8.1, 8.1→9.1, 9.1→10.1). Fix: add DEL old number + INS new number for each sub-clause.
|
||||
|
||||
### 3. New clause heading format mismatch
|
||||
- Wrong pStyle (e.g. Style15 instead of Heading4)
|
||||
- Extra space in heading text (e.g. "第七条 转包" vs original "第六条违约责任" no space)
|
||||
- Missing paragraph properties (spacing, ind) that originals have
|
||||
|
||||
### 4. Missing "法律顾问修订版" footer
|
||||
Workflow sometimes doesn't add the footer. Fix: add via python-docx, then wrap the run in `w:ins author=WB` (must be tracked change format).
|
||||
|
||||
### 5. Duplicate content insertion
|
||||
P32 example: original already had "保密义务不因合同解除...而免除", but WB added another "本条保密义务不因本合同的终止或解除而终止" — semantic duplicate. Fix: remove WB's duplicate.
|
||||
|
||||
### 6. Sentence flow at INS boundaries
|
||||
WB appends保密/数据归属 text directly after original sentence without transition. If the original sentence's context doesn't naturally lead into the INS content (e.g. "遵守保密规范。保密义务不因..."), add a proper subject/definition sentence as transition.
|
||||
|
||||
### 7. 赔偿上限未删
|
||||
Rule says "赔偿上限能删就删". Watch for bilateral clauses with caps (e.g. "违约金额为合同总金额的20%") — if it limits what our client can claim, delete it.
|
||||
|
||||
### 8. File naming with pre-existing【修】prefix
|
||||
If the original file already has 【修】prefix (e.g. from previous editor), the delivered should technically be 【修】【修】... per strict rules. Record as known workflow defect.
|
||||
|
||||
## Self-check before reporting "满意"
|
||||
- [ ] Every WB INS/DEL reviewed against rules
|
||||
- [ ] Full accepted text read for fluency (especially INS boundaries)
|
||||
- [ ] rFonts cleaned (no hAnsi/cs/hint extras)
|
||||
- [ ] Sub-numbering顺延 complete (not just chapter headings)
|
||||
- [ ] Footer "法律顾问修订版" present in tracked change format
|
||||
- [ ] No duplicate semantic content
|
||||
- [ ] Heading style/format matches originals
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
# 交付通知未送达诊断(2026-07-09 职业卫生+舜珙血压计)
|
||||
|
||||
## 现象
|
||||
Doro说"有几份合同没收到交付通知"。任务交付目录有文件,tracker状态=delivered。
|
||||
|
||||
## 根因
|
||||
今日凌晨01:27-02:15企微WebSocket连接中断(errcode 846609: aibot websocket not subscribed)。
|
||||
Workflow的final_review步骤内部调用`_send_wecom(extra, 'doro', msg)`发送通知,但此时WS已断,通知静默失败。
|
||||
|
||||
## 时间线
|
||||
- 01:27:23 — 首次846609错误
|
||||
- 01:28:01 — 职业卫生合同thread end(final_review完成,通知发送失败)
|
||||
- 01:30:41 — WebSocket closed (attempt 6)
|
||||
- 02:12:29 — 舜珙血压计thread end(final_review完成,通知发送失败)
|
||||
- 02:15:50 — WebSocket closed (attempt 7)
|
||||
- 02:18:30 — Doro发"hi",WS恢复
|
||||
|
||||
## 诊断命令
|
||||
```bash
|
||||
# 1. 找tracker中delivered状态的合同
|
||||
python3 -c "import json; t=json.load(open('~/.hermes/data/contract-tracker.json')); [print(c['delivered_filename']) for c in t['contracts'] if c.get('status')=='delivered']"
|
||||
|
||||
# 2. 查gateway.log确认WS断连时段
|
||||
grep '846609\|WebSocket error\|WebSocket closed' ~/.hermes/logs/gateway.log | grep '2026-07-09'
|
||||
|
||||
# 3. 确认thread完成了final_review
|
||||
uwf step list <thread_id> # 看是否有final_revi步骤且有时长
|
||||
|
||||
# 4. 确认文件在任务交付目录
|
||||
sudo docker exec nextcloud-nextcloud-1 stat "/var/www/html/data/doro/files/Doro合同审查任务/任务交付/【修】XXX.docx"
|
||||
```
|
||||
|
||||
## 补发方法
|
||||
```bash
|
||||
python3 ~/.hermes/scripts/wecom_dm.py --to doro --text "合同审查完成通知(补发):
|
||||
|
||||
1️⃣ 合同名称
|
||||
顾问单位: XXX
|
||||
修订: N处插入、M处删除
|
||||
主要修订: ...
|
||||
|
||||
(此通知因XX时段企微WebSocket连接中断未能实时送达,现补发)"
|
||||
```
|
||||
|
||||
## 修订内容提取方法(用于补发摘要)
|
||||
```python
|
||||
import zipfile
|
||||
from lxml import etree
|
||||
W = 'http://schemas.openxmlformats.org/wordprocessingml/2006/main'
|
||||
|
||||
with zipfile.ZipFile(path) as z:
|
||||
xml = z.read('word/document.xml')
|
||||
root = etree.fromstring(xml)
|
||||
|
||||
# Count WB modifications
|
||||
wb_ins = [ins for ins in root.findall(f'.//{{{W}}}ins') if ins.get(f'{{{W}}}author') == 'WB']
|
||||
wb_del = [d for d in root.findall(f'.//{{{W}}}del') if d.get(f'{{{W}}}author') == 'WB']
|
||||
print(f"WB修订: {len(wb_ins)} ins, {len(wb_del)} del")
|
||||
|
||||
# Get INS text summaries
|
||||
for ins in wb_ins[:10]:
|
||||
texts = [t.text for t in ins.iter(f'{{{W}}}t') if t.text]
|
||||
text = ''.join(texts).strip()
|
||||
if text and len(text) > 2:
|
||||
print(f" + {text[:80]}")
|
||||
```
|
||||
|
||||
## 预防措施
|
||||
- auto_notify_watchdog cron每5分钟检查WS连接
|
||||
- gateway WS重连机制(attempt 6-8自动重连)
|
||||
- 但final_review内的`_send_wecom`调用没有重试机制——WS断了就直接失败
|
||||
- **待改进**:final_review的通知步骤应增加重试逻辑或失败后写入pending_notifications队列
|
||||
@@ -0,0 +1,64 @@
|
||||
# Notification Silent Failure — WeChat 846609 WebSocket Disconnection
|
||||
|
||||
## Incident: 2026-07-09
|
||||
|
||||
### Timeline
|
||||
- 01:27 UTC — Gateway errcode 846609 first appears (WebSocket not subscribed)
|
||||
- 01:28 — 职业卫生监督 workflow final_review completes, notification fails silently
|
||||
- 01:30 — WebSocket error attempt 6
|
||||
- 02:12 — 舜珙血压计 workflow final_review completes, notification fails silently
|
||||
- 02:15 — WebSocket error attempt 7
|
||||
- 02:18 — Doro sends "hi", WebSocket recovers (inbound works before outbound stabilizes)
|
||||
|
||||
### Root Cause
|
||||
`final_review` calls `_send_wecom(extra, 'doro', msg)` in a subprocess. When the WeCom WebSocket is disconnected (846609), the send fails but:
|
||||
1. The uwf thread still ends successfully (status=end)
|
||||
2. The queue-runner marks the contract as `delivered` in tracker
|
||||
3. No retry mechanism exists for failed notifications
|
||||
4. No alarm fires for silent notification failures
|
||||
|
||||
### Diagnostic Commands
|
||||
|
||||
```bash
|
||||
# 1. Find delivered contracts that may have missed notifications
|
||||
python3 -c "
|
||||
import json
|
||||
t = json.load(open('$HOME/.hermes/data/contract-tracker.json'))
|
||||
for c in t['contracts']:
|
||||
if c.get('status') == 'delivered':
|
||||
print(f\" {c['original_filename']} delivered_at={c.get('delivered_at','?')}\")
|
||||
"
|
||||
|
||||
# 2. Check for 846609 errors in the time window
|
||||
grep '846609' ~/.hermes/logs/gateway.log | grep "$(date +%Y-%m-%d)"
|
||||
|
||||
# 3. Check WebSocket disconnection periods
|
||||
grep 'WebSocket error\|websocket closed' ~/.hermes/logs/gateway.log | grep "$(date +%Y-%m-%d)"
|
||||
|
||||
# 4. Verify if notification was actually sent (look for successful send around delivered_at)
|
||||
# A successful notification looks like:
|
||||
# INFO gateway.platforms.base: [Wecom] Sending response (XXX chars) to doro
|
||||
# WITHOUT a subsequent 846609 error in the same second
|
||||
|
||||
# 5. Check which threads completed during outage
|
||||
grep 'DONE.*status=end' /tmp/contract-queue/queue.log | grep "TIME_RANGE"
|
||||
```
|
||||
|
||||
### Remediation
|
||||
```bash
|
||||
# Manually re-send notification for affected contracts
|
||||
python3 ~/.hermes/scripts/wecom_dm.py --to doro --text "合同审查完成通知(补发):
|
||||
|
||||
文件名: 【修】XXX.docx
|
||||
顾问单位: XXX
|
||||
修订摘要: X处插入、Y处删除
|
||||
主要修订: ...
|
||||
|
||||
已上传至Nextcloud任务交付目录。
|
||||
(因企微连接中断未能实时送达,现补发)"
|
||||
```
|
||||
|
||||
### Prevention (not yet implemented)
|
||||
- `final_review` should check send result and retry 3x with backoff
|
||||
- Queue-runner should distinguish "delivered + notified" from "delivered + notification failed"
|
||||
- Watchdog could audit: for each `delivered` record older than 30 min, verify gateway.log has a matching successful send
|
||||
+70
@@ -0,0 +1,70 @@
|
||||
# 交付通知丢失:企微WS凌晨断连 + fire-and-forget架构
|
||||
|
||||
## 事件:2026-07-09 职业卫生+舜珙血压计两份合同交付无通知
|
||||
|
||||
### 时间线
|
||||
- 01:23:06 — 最后一条成功发送(gateway → doro)
|
||||
- 01:25~01:27 — WS断连(原因未知,WeCom服务端)
|
||||
- 01:27:23 — 首次846609 "aibot websocket not subscribed"
|
||||
- 01:28:01 — 职业卫生监督thread=end,final_review尝试通知→失败
|
||||
- 01:30:44 — WS reconnected
|
||||
- 02:04:16 — 成功发送(QiuTing),说明短暂恢复
|
||||
- 02:12:29 — 舜珙thread=end,通知可能在02:05-02:12期间尝试
|
||||
- 02:15:50 — WS再次断开
|
||||
- 持续不稳定直到 06:36
|
||||
|
||||
### 根因链
|
||||
1. WeCom WS凌晨不稳定(每30-60min断一次,服务端维护/长连接超时)
|
||||
2. Gateway自动重连成功但846609持续("not subscribed"是服务端状态滞后)
|
||||
3. workflow 24/7运行,final_review完成时间不可控
|
||||
4. final_review通知是fire-and-forget:`_send_wecom(extra, 'doro', msg)` 调一次,失败即丢弃
|
||||
|
||||
### 通知机制分析
|
||||
```
|
||||
final_review procedure step 3:
|
||||
cd ~/.hermes/hermes-agent && source venv/bin/activate && python -c "
|
||||
from tools.send_message_tool import _send_wecom
|
||||
...asyncio.run(_send_wecom(extra, 'doro', msg))..."
|
||||
```
|
||||
|
||||
`_send_wecom`实现:
|
||||
- 创建**新的** WeComAdapter实例
|
||||
- connect() → send() → disconnect()
|
||||
- 独立WS连接,不依赖gateway的WS
|
||||
- 但用的是同一个WeCom API,846609是服务端状态,新连接一样受影响
|
||||
- 失败返回 `{"error": "..."}` 给LLM,LLM可能仍标记notification_sent=true
|
||||
|
||||
### 7月8日也有相同模式
|
||||
- 01:13 Timeout → 03:07 reconnect失败 → 06:04 gateway重启才恢复
|
||||
- 约5小时不可用窗口
|
||||
|
||||
### 解决方案(待实施)
|
||||
|
||||
**推荐方案B:watchdog补发**
|
||||
- watchdog cron(每20min)增加逻辑:
|
||||
1. 扫描tracker中 `status=delivered` + `delivered_at > 30min前` + `notification_sent != true`
|
||||
2. 用 `wecom_dm.py --to doro` 补发通知(独立WS连接)
|
||||
3. 成功后写 `notification_sent=true` + `notification_at=timestamp`
|
||||
4. 失败则 `notification_attempts += 1`,下次tick继续重试
|
||||
5. attempts > 6(即2小时)仍失败→日志告警不再重试
|
||||
|
||||
**tracker字段扩展**:
|
||||
```json
|
||||
{
|
||||
"notification_sent": false,
|
||||
"notification_at": null,
|
||||
"notification_attempts": 0
|
||||
}
|
||||
```
|
||||
|
||||
**wecom_dm.py优势**:
|
||||
- 独立WS连接,不受gateway状态影响
|
||||
- 有明确的返回值(success/fail)
|
||||
- 凌晨WS虽然不稳定但有恢复窗口(如02:04成功发送)
|
||||
- 20min tick间隔 × 多次重试,大概率能命中一个可用窗口
|
||||
|
||||
### 临时止血
|
||||
当发现合同delivered但Doro没收到通知时:
|
||||
```bash
|
||||
python3 ~/.hermes/scripts/wecom_dm.py --to doro --text "合同审查完成通知(补发): ..."
|
||||
```
|
||||
@@ -0,0 +1,56 @@
|
||||
# Pre-Pass Audit Checklist (2026-07-13 练塘璞石+环保袋实战)
|
||||
|
||||
When Doro asks to "审查workflow修改" before pass, use this checklist.
|
||||
|
||||
## Step 1: Identify WB vs Original Revisions
|
||||
|
||||
```python
|
||||
# In delivered docx:
|
||||
for ins in root.iter(f'{{{W}}}ins'):
|
||||
author = ins.get(f'{{{W}}}author')
|
||||
# WB = workflow's modifications (audit these)
|
||||
# Others (WB-1, 86187, 杨丽, etc.) = original file revisions (leave alone)
|
||||
```
|
||||
|
||||
## Step 2: Format Audit (per WB INS run)
|
||||
|
||||
| Check | How | Common Fail |
|
||||
|-------|-----|-------------|
|
||||
| rFonts extra attrs | Compare WB INS rFonts with same-para orig run | hAnsi/cs/hint added by workflow |
|
||||
| sz mismatch | Compare sz values | Usually OK if same as orig |
|
||||
| pStyle on new headings | Compare with adjacent original headings | Style15 instead of Heading4 |
|
||||
| Heading spacing/ind | Must match original heading paragraphs | Missing before/after=0, ind |
|
||||
| Bold | If orig headings not bold, new ones shouldn't be | Usually OK |
|
||||
| Title space | "第七条转包" vs "第七条 转包" | Workflow adds space |
|
||||
|
||||
## Step 3: Content/Numbering Audit
|
||||
|
||||
| Check | How | Common Fail |
|
||||
|-------|-----|-------------|
|
||||
| Sub-numbering顺延 | If 第七条→第八条, check 7.1→8.1 etc. | Workflow only changes chapter heading, forgets sub-numbers |
|
||||
| 赔偿上限20% | Bilateral caps limit our client's recovery | Workflow misses bilateral cap deletion |
|
||||
| Content dedup | Check if INS保密存续 duplicates existing text | Original may already have "义务不因...终止而免除" |
|
||||
| 编号冲突 | Original may already have duplicate numbers | Don't fix original numbering bugs (per rules) |
|
||||
|
||||
## Step 4: Global Checks
|
||||
|
||||
- [ ] 脚注"法律顾问修订版" exists AND is in tracked change format (w:ins author=WB)
|
||||
- [ ] File naming: 【修】+ original filename unchanged
|
||||
- [ ] Read full accepted text for WB-introduced grammar issues
|
||||
- [ ] Original revisions (other authors) untouched
|
||||
|
||||
## Fix Patterns
|
||||
|
||||
### Sub-numbering (split across runs: "7" + ".1 ")
|
||||
Only need to DEL/INS the first digit run. Don't touch ".1 " run.
|
||||
|
||||
### Precise text deletion (P49 pattern)
|
||||
When deleting middle of a single large run:
|
||||
1. Split run into: before_text | DEL_text | after_text
|
||||
2. Create 3 elements: normal_run(before) + del_elem(middle) + normal_run(after)
|
||||
3. Insert at original position
|
||||
|
||||
### Footer tracked change
|
||||
1. Add footer text via python-docx
|
||||
2. Re-open with zipfile, find footer XML
|
||||
3. Wrap the text run in `<w:ins id="..." author="WB" date="...">`
|
||||
@@ -0,0 +1,91 @@
|
||||
# Queue-Runner / Watchdog 重复审查 Bug(2026-07-02 确诊)
|
||||
|
||||
## 症状
|
||||
|
||||
Doro 不断收到同一份合同的重复"审查完毕"通知。同一天内夏阳合同被审查交付2次,家庭医生签约合同被审查交付后又有一个 thread 在跑。
|
||||
|
||||
## 根因
|
||||
|
||||
`contract-queue-watchdog`(cron `3174518affda`,每20分钟)与 `contract-queue-runner.sh` 的交互存在逻辑缺陷:
|
||||
|
||||
### 时间线
|
||||
|
||||
1. Runner 启动,从 manifest.txt 读取待处理文件列表
|
||||
2. Runner 对文件A启动 `uwf thread start` + `uwf thread exec --background`
|
||||
3. 背景 worker(node进程)开始跑 workflow,耗时 1-2 小时
|
||||
4. Runner 自身在 `wait for worker PID` 循环中——但如果 runner 自己因某种原因退出(进程被杀、OOM、超时),只剩 worker 在跑
|
||||
5. **关键 bug**:workflow 跑完后 deliverer 交付文件 + final_review 通知 Doro,但 runner 已经不在了——无法把文件从 queue/ 移入 done/
|
||||
6. 20分钟后 watchdog tick:发现 `RUNNER_PID` 为空 + `DONE_CNT < TOTAL` + 无活跃 worker → **重启 runner**
|
||||
7. 新 runner 读 manifest,发现文件还在 queue/(因为没被移到 done/)→ **再次启动 workflow** → 重复审查 → 重复通知
|
||||
|
||||
### 更隐蔽的变体(本次实证)
|
||||
|
||||
即使 runner 没死,也会出问题:
|
||||
- Runner 在等 worker exit,worker 正常完成 → runner 移文件到 done/ → 进入下一份
|
||||
- 但 runner 处理完 manifest 所有文件后正常退出
|
||||
- 新文件在 runner 退出后被追加到 manifest(如 auto_notify 追加)
|
||||
- Watchdog 重启 runner → runner 从头读 manifest → 前面的文件已在 done/ 会被 SKIP
|
||||
- **但如果某份文件在 queue/ 中仍然存在**(不在 done/)→ 又跑一遍
|
||||
|
||||
### 为什么文件会在 queue/ 而不在 done/
|
||||
|
||||
1. Runner 异常退出,文件从未被移到 done/
|
||||
2. 新追加的文件,上一轮 runner 没跑到就退出了
|
||||
3. 文件被 auto_notify 或手动操作重新放回 queue/(不太可能但理论上存在)
|
||||
|
||||
## 缺失的防线
|
||||
|
||||
Runner 的 SKIP 逻辑只有一层:
|
||||
|
||||
```bash
|
||||
[ -e "$FILE" ] || { log "SKIP (not found / already done): $BASENAME"; continue; }
|
||||
```
|
||||
|
||||
只看文件是否还在 queue/ 目录。**完全不查 contract-tracker.json**。
|
||||
|
||||
## 修复方案
|
||||
|
||||
在 runner 的 `=== START:` 之前加 tracker 查重:
|
||||
|
||||
```bash
|
||||
# === BEFORE START: check tracker for already-completed ===
|
||||
if python3 -c "
|
||||
import json, sys
|
||||
t = json.load(open('$HOME/.hermes/data/contract-tracker.json'))
|
||||
completed = [c['original_filename'] for c in t['contracts'] if c['status']=='completed']
|
||||
sys.exit(0 if '$BASENAME' in completed else 1)
|
||||
" 2>/dev/null; then
|
||||
log "SKIP (already completed in tracker): $BASENAME"
|
||||
mv "$FILE" "$QUEUE_DIR/done/"
|
||||
continue
|
||||
fi
|
||||
```
|
||||
|
||||
## 止血操作(已执行)
|
||||
|
||||
1. ✅ kill 了正在重复跑的 reviewer thread (06FJ5NPHT9XX63HW0WDKXV3ZSR) 的 worker
|
||||
2. ✅ kill 了 queue-runner 进程 (PID 1907973)
|
||||
3. ✅ 将所有已处理文件移入 done/(家庭医生签约、朱家角标识标牌、计划生育协议)
|
||||
4. ✅ 验证 done/ 数量 ≥ manifest 行数 → watchdog 不会再重启 runner
|
||||
|
||||
## 今天的重复统计
|
||||
|
||||
| 合同 | 正常审查 | 重复审查 | 影响 |
|
||||
|------|----------|----------|------|
|
||||
| 恭兴 | 05:00 (end) | 02:45被cancel了不算 | 无重复 |
|
||||
| 肃言 | 06:25 (end) | — | 无重复 |
|
||||
| 卫健委 | 07:28 (end) | — | 无重复 |
|
||||
| 夏阳 | 08:42 (end) | 11:40 再跑一遍 (end) | ⚠️ 重复通知 |
|
||||
| 家庭医生签约 | 12:38→19:21交付 | 19:40又重启→21:16重复交付 | ⚠️ 重复通知 |
|
||||
|
||||
## Watchdog 今天重启 runner 的次数
|
||||
|
||||
今天 watchdog tick 64次,其中触发 runner 重启 **41次**(00:00-10:40每20分钟都重启一次!)。大多数重启只是空跑(文件都在 done/ 了),但恭兴和夏阳那两次重启时文件还没进 done/,导致重复审查。
|
||||
|
||||
## 相关组件
|
||||
|
||||
- Runner 脚本:`~/.hermes/skills/devops/uwf/scripts/contract-queue-runner.sh`
|
||||
- Watchdog 脚本:`~/.hermes/scripts/contract-queue-watchdog.sh`
|
||||
- Watchdog cron:`contract-queue-watchdog` (job_id: `3174518affda`),每20分钟
|
||||
- Queue 目录:`/tmp/contract-queue/`(manifest.txt + done/)
|
||||
- Tracker:`~/.hermes/data/contract-tracker.json`
|
||||
+85
@@ -0,0 +1,85 @@
|
||||
# Queue-Runner + Watchdog 重复审查Bug(2026-07-02 实证)
|
||||
|
||||
## 现象
|
||||
Doro反复收到同一份合同的"审查完毕"通知。今天受影响的合同:
|
||||
- 夏阳合同:被通知至少2次(可能3次)
|
||||
- 家庭医生签约合同:被通知2次(第3次在final_review被手动杀掉)
|
||||
|
||||
## Bug链条(已验证)
|
||||
|
||||
```
|
||||
1. Queue-runner 启动 → START 合同X → 创建 worker PID → "Waiting for worker..."
|
||||
2. Runner 进程异常退出(OOM/信号/shell被杀),但 worker 子进程继续运行
|
||||
3. Worker 独立完成整个 workflow(包括 final_review = 私信通知 Doro)
|
||||
4. Runner 已死 → 没有执行 "mv $FILE done/" → 文件仍在 queue 目录
|
||||
5. Watchdog(20min cron)检测到:runner不在 + done/ < manifest → 重启 runner
|
||||
6. 新 runner 看到文件还在 queue → SKIP逻辑只检查 `[ -e "$FILE" ]` → 认为未处理
|
||||
7. 新 runner 启动第二个 thread → 从头审查 → final_review 又通知 Doro
|
||||
```
|
||||
|
||||
## 额外失败模式(watchdog恢复旧thread)
|
||||
|
||||
```
|
||||
watchdog.log:
|
||||
[2026-07-02 18:40:14] suspended 06FJ42KNSK1ZA767W8AZ6MXJ6C → exec 恢复
|
||||
[2026-07-02 18:40:15] idle 06FJ3ZJXR5WHJNPDHF22YYMGCM → exec 续跑
|
||||
```
|
||||
|
||||
Watchdog 恢复处于 idle/suspended 状态的旧 thread,这些 thread 的同名合同可能已被新 runner 完成。
|
||||
旧 thread 被唤醒后接着跑完 final_review → 又一次通知。
|
||||
|
||||
## 今天的实际时间线
|
||||
|
||||
| 时间(BJT) | 事件 |
|
||||
|-----------|------|
|
||||
| 02:45 | 第一个runner启动,处理恭兴(cancelled) |
|
||||
| 05:00 | Watchdog重启runner → 恭兴(成功) |
|
||||
| 06:25 | 肃言完成 |
|
||||
| 07:28 | 卫健委完成 |
|
||||
| 08:42 | 夏阳 thread#1 启动 (06FJ3ZJXR) |
|
||||
| ~11:00 | 夏阳#1 完成(含final_review通知);但runner已死,文件没进done/ |
|
||||
| 11:40 | Watchdog重启runner → 夏阳 thread#2 启动 (06FJ58AD) |
|
||||
| 12:38 | 夏阳#2 完成(第二次通知);家庭医生签约 thread 启动 (06FJ5NPHT) |
|
||||
| 18:40 | Watchdog恢复旧idle thread 06FJ3ZJXR5WHJNPDHF22YYMGCM (夏阳#1) + 06FJ42KNSK (家庭医生#?) |
|
||||
| 19:21 | 家庭医生签约交付到Nextcloud |
|
||||
| ~21:00 | 06FJ42KNSK 完成 final_review(第2次家庭医生通知) |
|
||||
| 21:16 | Watchdog再次重启runner → 家庭医生 thread#2 (06FJ5NPHT) 进入 final_review |
|
||||
| 21:30 | 手动 kill -9 杀掉 06FJ5NPHT 的 final_review → 阻止第3次通知 |
|
||||
|
||||
## 止血SOP
|
||||
|
||||
当 Doro 报告收到重复通知时:
|
||||
|
||||
1. **找重复进程**:`ps aux | grep -E "(background-worker|uwf-hermes)" | grep -v grep`
|
||||
2. **杀掉重复进程**:`kill -9 <worker-PID> <uwf-hermes-PID>`
|
||||
3. **杀掉queue-runner**:`kill <queue-runner-PID>`
|
||||
4. **清理queue**:把所有tracker中已completed的文件移入done/
|
||||
```python
|
||||
import json, os, shutil
|
||||
tracker = json.load(open(os.path.expanduser('~/.hermes/data/contract-tracker.json')))
|
||||
completed = {c['original_filename'] for c in tracker['contracts'] if c['status'] == 'completed'}
|
||||
queue_dir = '/tmp/contract-queue'
|
||||
for f in os.listdir(queue_dir):
|
||||
if f.endswith(('.doc', '.docx', '.pdf')) and f in completed:
|
||||
shutil.move(f'{queue_dir}/{f}', f'{queue_dir}/done/{f}')
|
||||
```
|
||||
5. **验证**:`find /tmp/contract-queue/ -maxdepth 1 -name '*.doc*'` 应为空
|
||||
6. **确认watchdog不会重启**:done/ 文件数 ≥ manifest.txt 行数
|
||||
|
||||
## 缺失防线(待修复)
|
||||
|
||||
| 位置 | 应加的检查 |
|
||||
|------|-----------|
|
||||
| queue-runner START 逻辑 | 启动workflow前查tracker:已completed直接mv到done/ |
|
||||
| watchdog 恢复thread逻辑 | resume前查:同filename的tracker记录是否already completed |
|
||||
| final_review | 发通知前查:是否24h内已有同合同的通知(防御性去重) |
|
||||
|
||||
## 关键文件路径
|
||||
|
||||
- Queue runner: `/home/maggie/.hermes/skills/devops/uwf/scripts/contract-queue-runner.sh`
|
||||
- Watchdog: `~/.hermes/scripts/contract-queue-watchdog.sh`
|
||||
- Queue dir: `/tmp/contract-queue/` (manifest.txt + done/)
|
||||
- Queue log: `/tmp/contract-queue/queue.log`
|
||||
- Watchdog log: `/tmp/contract-queue/watchdog.log`
|
||||
- Tracker: `~/.hermes/data/contract-tracker.json`
|
||||
- Cron jobs: `contract-queue-watchdog` (*/20, job_id:3174518affda), `auto-notify-watchdog` (5min, job_id:63bb31d4f050)
|
||||
+110
@@ -0,0 +1,110 @@
|
||||
# Queue-Runner: Same-Name Different-Content File Silently Dropped (2026-07-08)
|
||||
|
||||
## Incident
|
||||
|
||||
邱律师 sent two versions of "2026年华新镇公立中小学生健康体检服务合同.docx" on the same day:
|
||||
- 11:05 BJT (27889 bytes): generic version without fee cap
|
||||
- 16:04 BJT (27482 bytes): specific version with 华新镇 in project name, ¥170,000 fee cap, different start date
|
||||
|
||||
Only the first was reviewed. The second was silently dropped.
|
||||
|
||||
## Root Cause Chain (3 components)
|
||||
|
||||
### 1. auto_notify manifest dedup (`grep -qFx`)
|
||||
|
||||
```bash
|
||||
# In auto_notify_new_file.sh:
|
||||
if ! grep -qFx "$orig_name" "$QUEUE_DIR/manifest.txt" 2>/dev/null; then
|
||||
echo "$orig_name" >> "$QUEUE_DIR/manifest.txt"
|
||||
fi
|
||||
```
|
||||
|
||||
The filename was already in manifest.txt from the first file → second file NOT appended → manifest has only ONE entry for this filename.
|
||||
|
||||
### 2. Runner single-pass no-backtrack
|
||||
|
||||
The runner reads manifest top-to-bottom in one pass. By 08:00:09 UTC it had already passed the 华新镇 line (first version was in done/ → "SKIP not found"). When auto_notify wrote the second file to queue/ at 08:04, the runner was already past that line processing later files. It never goes back.
|
||||
|
||||
### 3. done/ presence satisfies watchdog progress check
|
||||
|
||||
`[ -e "$QUEUE_DIR/done/$f" ]` — the first version in done/ counts as "complete" for this manifest line.
|
||||
|
||||
## Key Principle (Doro 2026-07-08 铁律)
|
||||
|
||||
**判断是否为相同文件不能只看文件名。** 必须检查合同实质内容:
|
||||
- 顾问单位(甲方)名称
|
||||
- 金额/费用上限
|
||||
- 合同期限(起止日期)
|
||||
- 项目内容描述
|
||||
- 字节大小
|
||||
|
||||
以上任何一项不同 → 视为新版本/不同合同,正常入队审查。
|
||||
全部相同 → 视为重复发送,可跳过。
|
||||
|
||||
## Fix Implemented (2026-07-09)
|
||||
|
||||
### 1. Content comparison script: `~/.hermes/scripts/contract_content_compare.py`
|
||||
|
||||
```
|
||||
Usage: python3 contract_content_compare.py <file_a> <file_b>
|
||||
Exit 0 = same content (duplicate)
|
||||
Exit 1 = different content (new version / different contract)
|
||||
Exit 2 = cannot read (treat as different, err on safe side)
|
||||
```
|
||||
|
||||
Compares:
|
||||
- File size (bytes)
|
||||
- 甲方 name (regex extraction from first 2000 chars)
|
||||
- All amounts (阿拉伯数字 ≥4 digits + 元/万, 人民币XXX, percentages)
|
||||
- All dates (YYYY年M月D日 format)
|
||||
- Project summary (first 500 chars normalized)
|
||||
|
||||
### 2. auto_notify_new_file.sh modification
|
||||
|
||||
Replaced the simple `grep -qFx` manifest dedup with content-level comparison:
|
||||
|
||||
```bash
|
||||
if grep -qFx "$orig_name" "$QUEUE_DIR/manifest.txt" 2>/dev/null; then
|
||||
# Same filename exists in manifest — compare content
|
||||
EXISTING="" # find in done/ or queue/
|
||||
COMPARE_RESULT=$(python3 contract_content_compare.py "$EXISTING" "$filepath")
|
||||
if [ $? -eq 0 ]; then
|
||||
# Content identical → true duplicate, skip
|
||||
log "SKIP duplicate (content identical): $orig_name"
|
||||
return
|
||||
else
|
||||
# Content different → new version, rename with timestamp suffix and queue
|
||||
RENAMED="${orig_name%.*}_v${TS}.${orig_name##*.}"
|
||||
cp "$filepath" "$QUEUE_DIR/$RENAMED"
|
||||
echo "$RENAMED" >> "$QUEUE_DIR/manifest.txt"
|
||||
log "SAME NAME DIFFERENT CONTENT — queued as new: $RENAMED"
|
||||
fi
|
||||
else
|
||||
# Brand new filename, normal flow
|
||||
cp "$filepath" "$QUEUE_DIR/${orig_name}"
|
||||
echo "$orig_name" >> "$QUEUE_DIR/manifest.txt"
|
||||
fi
|
||||
```
|
||||
|
||||
### 3. Verification
|
||||
|
||||
Tested with the actual 华新镇 two files:
|
||||
```
|
||||
$ python3 contract_content_compare.py file1.docx file2.docx
|
||||
DIFFERENT: 两份文件内容不同(新版本/不同合同)
|
||||
- 字节大小不同: 27889 vs 27482
|
||||
- 金额不同: A多set(), B多{'170000'}
|
||||
- 日期不同: A多{'2026年9月10日'}, B多{'2026年9月1日'}
|
||||
- 内容不同(第212字起): '...年公立中小学生健康检查...' vs '...年华新镇公立中小学生健康体检...'
|
||||
```
|
||||
|
||||
## Key Lesson
|
||||
|
||||
The initial diagnosis went through multiple wrong iterations:
|
||||
1. First said "queue-runner only checks filename" — wrong (tracker guard also exists)
|
||||
2. Then said "tracker guard is the one that skipped" — wrong (no tracker skip in logs)
|
||||
3. Then said "整个链路没有内容比对" — wrong (Doro corrected: the system should have had it)
|
||||
|
||||
**Correct diagnosis process**: trace the actual log timestamps precisely, verify each claim with tool output, don't assume mechanisms exist or don't exist without reading the actual code.
|
||||
|
||||
**Doro's requirement**: "不能用文件名判断是否为相同文件,需要查看合同内容(包括顾问单位名称、金额、日期等)以及字节大小,要全面判断。" This is a capability requirement, not just a bug fix — the system must understand what makes two contracts "the same" at a business level.
|
||||
+79
@@ -0,0 +1,79 @@
|
||||
# Watchdog "suspended + delivered" Deadlock (2026-07-08)
|
||||
|
||||
## Symptom
|
||||
- Doro reports not receiving delivery notification for completed contracts
|
||||
- `tail watchdog.log` shows repeated lines every 20 minutes:
|
||||
```
|
||||
BLOCK resume <thread_id>: tracker shows <filename> already delivered
|
||||
runner 退出但有 worker/卡住thread在处理,暂不重启(避免撞车)
|
||||
```
|
||||
- Files ARE in 任务交付/ directory (deliverer succeeded), but notification was never sent
|
||||
- Queue runner has exited, watchdog won't restart it
|
||||
|
||||
## Root Cause Chain
|
||||
1. Workflow reaches `final_review` step (responsible for quality check + notification)
|
||||
2. `final_review` encounters HTTP 500 / API failure → thread suspends
|
||||
3. But `deliverer` already succeeded earlier → tracker shows `status=delivered`
|
||||
4. Queue-runner sees `status=suspended` (not `end`) → marks as "needs inspection", does NOT archive to done/
|
||||
5. Queue-runner continues to next file, eventually exits
|
||||
6. Watchdog checks: finds suspended thread → tries to resume → checks tracker → sees "already delivered" → BLOCK
|
||||
7. File still in queue/ (not in done/) → watchdog thinks work remains → won't restart runner for other files
|
||||
8. **Infinite loop**: every 20 min watchdog ticks, hits same BLOCK, same "暂不重启"
|
||||
|
||||
## Impact Scope (2026-07-08 batch)
|
||||
All 5 contracts from the 16:00 batch were affected (all hit HTTP 500 at final_review):
|
||||
- 【修】华新慢病支持中心运维合同(1)_2.docx
|
||||
- 赵巷合同1.doc
|
||||
- 疾控中心(卫监所)实习人员宿舍改造工程.docx
|
||||
- 2026年爱在党群·沟通有方家长沟通之道青春健康教育项目合作协议.docx
|
||||
- 合同.docx
|
||||
|
||||
## Resolution (止血 SOP)
|
||||
|
||||
### Step 1: Cancel suspended threads
|
||||
```bash
|
||||
/home/maggie/.hermes/node/bin/uwf thread cancel <thread_id>
|
||||
# Repeat for all suspended threads
|
||||
```
|
||||
|
||||
### Step 2: Move stuck files to done/
|
||||
```bash
|
||||
cd /tmp/contract-queue
|
||||
mv "filename1.docx" done/
|
||||
mv "filename2.doc" done/
|
||||
# Move ALL files that are in tracker as delivered/completed
|
||||
```
|
||||
|
||||
### Step 3: Verify queue is clear
|
||||
```bash
|
||||
ls /tmp/contract-queue/*.doc* 2>/dev/null # Should be empty
|
||||
```
|
||||
|
||||
### Step 4: Do pass for delivered files
|
||||
Since files are already in 任务交付/ and tracker shows delivered, proceed with normal pass flow (update tracker to completed + write xlsx).
|
||||
|
||||
## Prevention
|
||||
- The watchdog should detect "suspended + already delivered" as a terminal state and auto-archive to done/ (currently it only BLOCKs resume without archiving)
|
||||
- The final_review step should have retry logic for transient HTTP 500 errors
|
||||
- Queue-runner should archive files to done/ even when thread status=suspended IF tracker shows delivered (the work is done, only notification failed)
|
||||
|
||||
## Diagnosis Commands
|
||||
```bash
|
||||
# Check for deadlock pattern
|
||||
tail -20 /tmp/contract-queue/watchdog.log | grep "BLOCK resume"
|
||||
|
||||
# Check queue-runner status
|
||||
ps aux | grep contract-queue-runner | grep -v grep
|
||||
|
||||
# Check what's stuck in queue
|
||||
ls /tmp/contract-queue/*.doc* 2>/dev/null
|
||||
|
||||
# Check tracker for delivered status
|
||||
python3 -c "
|
||||
import json
|
||||
t = json.load(open('$HOME/.hermes/data/contract-tracker.json'))
|
||||
for c in t['contracts']:
|
||||
if c.get('status') == 'delivered':
|
||||
print(f\" {c['original_filename']} → delivered\")
|
||||
"
|
||||
```
|
||||
@@ -0,0 +1,26 @@
|
||||
# Workflow Issues Report 2026-07-01 (Summary)
|
||||
|
||||
Full report at: Nextcloud 小Maggie协作区/workflow-issues-report-20260701.md
|
||||
|
||||
## Key Findings
|
||||
|
||||
### File Disappearance Root Cause
|
||||
Files "missing" from 待审查 were likely **never uploaded there**. auto_notify failed silently (Mode B),
|
||||
workflow read directly from cache path, audit completed, but Nextcloud 待审查 directory was never populated.
|
||||
Cleanup cron confirmed NOT responsible (all cleaned records show >24h gap).
|
||||
|
||||
### Duplicate Review (朱家角标识牌)
|
||||
Root cause: `/tmp/contract-queue/manifest` retained filename after pass+clean.
|
||||
relay-runner reads manifest without checking contract-tracker.json for completed status.
|
||||
Fix: relay-runner must `grep original_filename tracker.json` before `uwf thread start`.
|
||||
|
||||
### Companion Cleanup
|
||||
Confirmed approach (Doro 2026-07-01):
|
||||
- Naming rule: `【审】{原文件名去扩展名} 审查意见.docx`
|
||||
- Pass skill writes `companion_files` field to tracker
|
||||
- Cleanup script reads and deletes alongside main contract
|
||||
- Both sides must be modified simultaneously
|
||||
|
||||
### Private Message Routing
|
||||
`wecom_group_notify.py` (default=group) was used when `wecom_dm.py --to qiuting` (DM) was required.
|
||||
Workflow YAML line 16-17 also incorrectly references group notify script.
|
||||
@@ -0,0 +1,31 @@
|
||||
# xlsx覆盖事故 2026-07-03
|
||||
|
||||
## 事故
|
||||
|
||||
pass流程中使用了/tmp残留的旧xlsx(240行,截止6月30日),覆盖了Nextcloud上的最新版(252行,截止7月3日),导致seq 241-251共11条记录丢失。
|
||||
|
||||
## 根因
|
||||
|
||||
没有按skill规定从Nextcloud实时拉取最新xlsx,直接用了本地旧文件。
|
||||
|
||||
## 恢复
|
||||
|
||||
/tmp下恰好有另一份正确副本(今早其他session拉取的),用它恢复+追加新行。
|
||||
|
||||
## 铁律
|
||||
|
||||
1. 写xlsx前必须 `rm -f → docker cp从NC拉取 → 读max_row打印seq确认 → 追加 → 上传`
|
||||
2. 禁止使用/tmp中任何已存在的xlsx文件
|
||||
3. 上传后重新拉取验证行数
|
||||
|
||||
## Doro原话
|
||||
|
||||
"你这个错误太可怕了"
|
||||
"不仅要写入,避免下次发生,还得写入铁律"
|
||||
|
||||
## 连带错误
|
||||
|
||||
同一session中还犯了:
|
||||
- 查到邱律师新文件后没查auto_notify日志就手动启动workflow → 重复
|
||||
- 主观判断两份文件"相同" → 被Doro纠正"你也不要去判断是不是同一份文件"
|
||||
- 时间用UTC而非北京时间 → 多次纠正
|
||||
Reference in New Issue
Block a user