feat: agent adapter token usage reporting ($usage in frontmatter) #68

New Issue

2026-06-04T14:54:25Z

xiaoju commented

2026-06-04 14:54:25 +00:00

Background

Eval framework (#34) needs per-step token consumption data. Currently agent adapters only output role frontmatter — no telemetry.

Design

Extend adapter output frontmatter with $usage field:

---
$status: done
$usage:
  turns: 4
  inputTokens: 12000
  outputTokens: 3500
  duration: 45
---

Sub-issues

Phase	Issue	Description
1	#74	`Usage` type in protocol + extract pipeline in util-agent
2	#75	agent-mock emits fixed `$usage` + E2E test
3a	#76	agent-hermes collects from API response
3b	#77	agent-claude-code parses from CLI stdout

Phase 1+2 together, Phase 3a/3b independent.

— 小橘 🍊（NEKO Team）

## Background Eval framework (#34) needs per-step token consumption data. Currently agent adapters only output role frontmatter — no telemetry. ## Design Extend adapter output frontmatter with `$usage` field: ```yaml --- $status: done $usage: turns: 4 inputTokens: 12000 outputTokens: 3500 duration: 45 --- ``` ## Sub-issues | Phase | Issue | Description | |-------|-------|-------------| | 1 | #74 | `Usage` type in protocol + extract pipeline in util-agent | | 2 | #75 | agent-mock emits fixed `$usage` + E2E test | | 3a | #76 | agent-hermes collects from API response | | 3b | #77 | agent-claude-code parses from CLI stdout | Phase 1+2 together, Phase 3a/3b independent. — 小橘 🍊（NEKO Team）

xiaoju referenced this issue

2026-06-04 15:13:23 +00:00

feat: builtin judges — frontmatter, upstream, hallucination, token-stats #71

xiaoju referenced this issue

2026-06-04 15:14:04 +00:00

test: E2E eval — 真实 agent 效果评估 #34

xiaoju referenced this issue

2026-06-04 15:23:12 +00:00

feat: $usage type in protocol + extract pipeline in util-agent (#68 Phase 1) #74

xiaoju referenced this issue

2026-06-04 15:23:13 +00:00

feat: agent-mock emits $usage (#68 Phase 2) #75