feat: meta-workflow — workflow for developing workflows
CI / test (push) Has been cancelled

Roles: architect(LLM) → coder(Cursor) → reviewer(Cursor) → tester(code+LLM) → promoter(code)
- Retry loop: reviewer rejected or tester failed → back to coder
- Meta architect uses structured tool output (workflow spec JSON)
- Coder/reviewer use agent-executor factory with LLM₂ meta parsing
- Tester: build+test then LLM judges result
- Promoter: git commit + push
- docs/workflow-spec.md: reference context for coder role
- 3 unit tests (moderator routing, happy path, retry path)
- 28 total workflow tests pass
This commit is contained in:
2026-04-17 09:20:45 +00:00
parent 4abe7bf620
commit 2f461bffeb
9 changed files with 770 additions and 0 deletions
+100
View File
@@ -0,0 +1,100 @@
# Pulse Workflow 开发规范
> 此文档是 meta-workflow 的 architect 和 coder 角色的 reference context。
## 核心概念
### WorkflowType<TRoles>
```typescript
interface WorkflowType<TRoles> {
name: string; // workflow 前缀,如 'coding', 'report'
roles: Record<string, Role<any>>; // 角色函数映射
moderator: (input, topicId) => string | END; // 状态机转换函数
}
```
### Role 函数
```typescript
type Role<TMeta> = (
chain: WorkflowMessage[], // 同一 topic 的历史消息链
topicId?: string, // topic key
store?: PulseStore, // 只读访问 store(可选)
) => Promise<RoleResult<TMeta>>;
interface RoleResult<TMeta> {
content: string; // 存入 CAS 的内容
meta: TMeta; // 存入 event.meta 的决策信号
}
```
### 设计原则
1. **Role 是纯的** — 返回 `{ content, meta }`,不写 event,adapter 负责写入
2. **kind = `{workflow}.{role}`** — 如 `coding.architect`, `report.analyst`
3. **key = topicId** — 同一个 workflow 实例共享 key(ULID)
4. **content 存 CAS** — event 只存 hash,大内容走 objects/
5. **meta 只放决策信号** — moderator 根据 meta 决定下一步
6. **START/END 伪 role**`__start__` 是输入事件,moderator 返回 `END` 表示结束
### LLM Role 工厂(三明治 pattern)
```typescript
// 纯文本返回:
createLlmRole(llm, {
systemPrompt: '...',
buildUserMessage: (chain) => '...',
parseResponse: (resp, chain) => ({ content, meta }),
});
// 结构化 tool 返回:
createToolRole(llm, {
systemPrompt: '...',
tool: { type: 'function', function: { name, description, parameters } },
defaultResult: { ... },
toRoleResult: (parsed, chain) => ({ content, meta }),
});
```
### Agent Role 工厂
```typescript
createAgentExecutorRole({
runner: createCursorRunner({ apiKey, cursorPath }),
buildPrompt: (chain) => '...',
parseResult: (stdout, stderr, exitCode) => ({ content, meta }),
});
```
### Adapter(workflow-rule-adapter)
- `createWorkflowRule(workflow, store)``{ tick() }`
- tick 读 events → 重建 per-topic snapshot → Moore diff → moderator → 执行 role → 写 event
- **闭包持有 prevSnapshotJson**,snapshot 变化才触发
### Event 存储模型
```
events table: id(ULID) | occurredAt | kind | key | hash | meta | code_rev
objects/: CAS 文件,hash 为文件名
```
### 现有 Workflow 参考
**coding-workflow**: START → architect(LLM) → coder(Cursor) → reviewer(Cursor) → END
**report-workflow**: START → analyst(LLM) → renderer(代码模板) → END
## 文件组织
```
packages/pulse/src/workflows/
coding-workflow.ts # workflow 定义 + meta 类型
report-workflow.ts # workflow 定义 + meta 类型
workflow-type.ts # 核心类型
workflow-rule-adapter.ts # adapter(Moore diff + event 写入)
index.ts # barrel exports
roles/
llm-role-factory.ts # LLM role 共享工厂
architect-llm.ts # LLM role 实例
analyst-llm.ts # LLM role 实例(tool_choice)
agent-executor.ts # Agent role 工厂
coder-cursor.ts # Cursor agent 实例
reviewer-cursor.ts # Cursor agent 实例
renderer-template.ts # 纯代码 role
```
+13
View File
@@ -5,6 +5,14 @@
*/
export { createCodingWorkflow } from './coding-workflow.js';
export { createMetaWorkflow } from './meta-workflow.js';
export type {
MetaArchitectMeta,
MetaCoderMeta,
MetaReviewerMeta,
MetaTesterMeta,
MetaPromoterMeta,
} from './meta-workflow.js';
export { createReportWorkflow } from './report-workflow.js';
export {
type AgentExecutorConfig,
@@ -21,6 +29,11 @@ export { createLlmRole, createToolRole } from './roles/llm-role-factory.js';
export type { LlmRoleConfig, ToolRoleConfig } from './roles/llm-role-factory.js';
export { createRendererRole } from './roles/renderer-template.js';
export { createReviewerRole } from './roles/reviewer-cursor.js';
export { createMetaArchitectRole } from './roles/meta-architect-llm.js';
export { createMetaCoderRole } from './roles/meta-coder-cursor.js';
export { createMetaReviewerRole } from './roles/meta-reviewer-cursor.js';
export { createMetaTesterRole } from './roles/meta-tester.js';
export { createMetaPromoterRole } from './roles/meta-promoter.js';
export {
createWorkflowRule,
type WorkflowRule,
@@ -0,0 +1,112 @@
/**
* Meta Workflow — unit tests (mock roles).
* 小橘 🍊 (NEKO Team)
*/
import { describe, expect, test } from 'bun:test';
import { createStore } from '../store.js';
import {
createMetaWorkflow,
type MetaArchitectMeta,
type MetaCoderMeta,
type MetaPromoterMeta,
type MetaReviewerMeta,
type MetaTesterMeta,
} from './meta-workflow.js';
import { createWorkflowRule } from './workflow-rule-adapter.js';
import { END, START, type WorkflowMessage } from './workflow-type.js';
import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
function mockStore() {
const dir = mkdtempSync(join(tmpdir(), 'meta-wf-test-'));
return createStore({
eventsDbPath: join(dir, 'events.db'),
objectsDir: join(dir, 'objects'),
});
}
describe('Meta Workflow', () => {
test('moderator routes: START→architect→coder→reviewer(approved)→tester(pass)→promoter→END', () => {
const wf = createMetaWorkflow({
architect: async () => ({ content: '{}', meta: { workflowName: 'test', roles: ['a'], transitions: 'S→a→E' } }),
coder: async () => ({ content: 'ok', meta: { filesChanged: ['a.js'], testsPassed: true } }),
reviewer: async () => ({ content: 'LGTM', meta: { verdict: 'approved', comments: 'ok' } }),
tester: async () => ({ content: 'pass', meta: { pass: true, liveOutput: 'all pass' } }),
promoter: async () => ({ content: 'done', meta: { commitHash: 'abc123', pushed: true } }),
});
expect(wf.moderator({ role: START, meta: null }, 'x')).toBe('architect');
expect(wf.moderator({ role: 'architect', meta: { workflowName: 'test', roles: [], transitions: '' } }, 'x')).toBe('coder');
expect(wf.moderator({ role: 'coder', meta: { filesChanged: [], testsPassed: true } }, 'x')).toBe('reviewer');
expect(wf.moderator({ role: 'reviewer', meta: { verdict: 'approved', comments: '' } }, 'x')).toBe('tester');
expect(wf.moderator({ role: 'reviewer', meta: { verdict: 'rejected', comments: 'fix' } }, 'x')).toBe('coder');
expect(wf.moderator({ role: 'tester', meta: { pass: true, liveOutput: '' } }, 'x')).toBe('promoter');
expect(wf.moderator({ role: 'tester', meta: { pass: false, liveOutput: 'fail' } }, 'x')).toBe('coder');
expect(wf.moderator({ role: 'promoter', meta: { commitHash: 'abc', pushed: true } }, 'x')).toBe(END);
});
test('mock: full happy path via adapter ticks', async () => {
const store = mockStore();
const wf = createMetaWorkflow({
architect: async () => ({ content: '{"workflowName":"demo"}', meta: { workflowName: 'demo', roles: ['a'], transitions: 'S→a→E' } }),
coder: async () => ({ content: 'files changed', meta: { filesChanged: ['demo.js'], testsPassed: true } }),
reviewer: async () => ({ content: 'approved', meta: { verdict: 'approved', comments: 'clean' } }),
tester: async () => ({ content: 'all pass', meta: { pass: true, liveOutput: 'ok' } }),
promoter: async () => ({ content: 'commit abc', meta: { commitHash: 'abc', pushed: true } }),
});
const rule = createWorkflowRule(wf, store);
// Seed START
const hash = store.putObject('build a demo workflow');
store.appendEvent({ occurredAt: Date.now(), kind: 'meta.__start__', key: 'test-1', hash });
// Tick through all roles
const roles: string[] = [];
for (let i = 0; i < 7; i++) {
const r = await rule.tick();
if (r.executed.length === 0) break;
roles.push(...r.executed.map((a) => a.role));
}
expect(roles).toEqual(['architect', 'coder', 'reviewer', 'tester', 'promoter']);
const events = store.getAfter(0);
expect(events.length).toBe(6); // __start__ + 5 roles
store.close();
});
test('mock: reviewer rejection → retry coder', async () => {
const store = mockStore();
let reviewCount = 0;
const wf = createMetaWorkflow({
architect: async () => ({ content: '{}', meta: { workflowName: 'x', roles: [], transitions: '' } }),
coder: async () => ({ content: 'code', meta: { filesChanged: [], testsPassed: true } }),
reviewer: async () => {
reviewCount++;
if (reviewCount === 1) return { content: 'fix imports', meta: { verdict: 'rejected' as const, comments: 'fix imports' } };
return { content: 'ok', meta: { verdict: 'approved' as const, comments: 'ok' } };
},
tester: async () => ({ content: 'pass', meta: { pass: true, liveOutput: 'ok' } }),
promoter: async () => ({ content: 'done', meta: { commitHash: 'def', pushed: true } }),
});
const rule = createWorkflowRule(wf, store);
const hash = store.putObject('test retry');
store.appendEvent({ occurredAt: Date.now(), kind: 'meta.__start__', key: 'retry-1', hash });
const roles: string[] = [];
for (let i = 0; i < 10; i++) {
const r = await rule.tick();
if (r.executed.length === 0) break;
roles.push(...r.executed.map((a) => a.role));
}
// architect → coder → reviewer(rejected) → coder → reviewer(approved) → tester → promoter
expect(roles).toEqual(['architect', 'coder', 'reviewer', 'coder', 'reviewer', 'tester', 'promoter']);
store.close();
});
});
@@ -0,0 +1,108 @@
/**
* Meta Workflow — workflow for developing workflows.
*
* Roles:
* architect (LLM) → analyze requirements, output workflow spec
* coder (Cursor) → implement roles + tests + workflow definition
* reviewer (Cursor) → tsc + bun test + code review
* tester (code+LLM) → run live test, LLM judges result
* promoter (code) → git commit + push + promote event
*
* Flow:
* START → architect → coder → reviewer
* → reviewer.verdict === 'approved' → tester
* → reviewer.verdict === 'rejected' → coder (retry)
* → tester.pass === true → promoter → END
* → tester.pass === false → coder (retry)
*
* 小橘 🍊 (NEKO Team)
*/
import {
END,
type ModeratorInput,
type Role,
START,
type WorkflowType,
} from './workflow-type.js';
// ── Meta Types ─────────────────────────────────────────────────
export interface MetaArchitectMeta {
[key: string]: unknown;
workflowName: string;
roles: string[];
transitions: string;
}
export interface MetaCoderMeta {
[key: string]: unknown;
filesChanged: string[];
testsPassed: boolean;
}
export interface MetaReviewerMeta {
[key: string]: unknown;
verdict: 'approved' | 'rejected';
comments: string;
}
export interface MetaTesterMeta {
[key: string]: unknown;
pass: boolean;
liveOutput: string;
}
export interface MetaPromoterMeta {
[key: string]: unknown;
commitHash: string;
pushed: boolean;
}
export type MetaWorkflowRoles = {
architect: Role<MetaArchitectMeta>;
coder: Role<MetaCoderMeta>;
reviewer: Role<MetaReviewerMeta>;
tester: Role<MetaTesterMeta>;
promoter: Role<MetaPromoterMeta>;
};
// ── Moderator ──────────────────────────────────────────────────
function metaModerator(
input: ModeratorInput<MetaWorkflowRoles>,
_topicId: string,
): (keyof MetaWorkflowRoles & string) | typeof END {
switch (input.role) {
case START:
return 'architect';
case 'architect':
return 'coder';
case 'coder':
return 'reviewer';
case 'reviewer': {
const verdict = (input.meta as MetaReviewerMeta | null)?.verdict;
return verdict === 'approved' ? 'tester' : 'coder';
}
case 'tester': {
const pass = (input.meta as MetaTesterMeta | null)?.pass;
return pass ? 'promoter' : 'coder';
}
case 'promoter':
return END;
default:
return END;
}
}
// ── Factory ────────────────────────────────────────────────────
export function createMetaWorkflow(
roles: MetaWorkflowRoles,
): WorkflowType<MetaWorkflowRoles> {
return {
name: 'meta',
roles,
moderator: metaModerator,
};
}
@@ -0,0 +1,139 @@
/**
* Meta Architect role — LLM analyzes workflow requirements and outputs a spec.
* Uses createToolRole factory.
*
* 小橘 🍊 (NEKO Team)
*/
import type { LlmClient } from '../../llm-client.js';
import type { MetaArchitectMeta } from '../meta-workflow.js';
import type { Role } from '../workflow-type.js';
import { createToolRole } from './llm-role-factory.js';
const SYSTEM_PROMPT = `你是 Pulse Council v2 的 workflow 架构师。
你收到用户对新 workflow 的需求描述,你需要输出一个结构化的 workflow 设计规范。
设计原则(严格遵守):
1. Role 是纯函数,返回 { content, meta },不写 event
2. kind = {workflow}.{role},如 coding.architect
3. content 存 CAS(大内容),meta 只放决策信号
4. moderator 是状态机转换函数,根据上一个 role 的 meta 决定下一步
5. LLM role 用 createLlmRole/createToolRole 工厂
6. Agent role 用 createAgentExecutorRole 工厂
7. 纯代码 role 直接实现 Role 接口
参考现有 workflow:
- coding-workflow: START → architect(LLM) → coder(Cursor) → reviewer(Cursor) → END
- report-workflow: START → analyst(LLM) → renderer(代码模板) → END
用 extract_spec tool 输出你的设计。`;
const SPEC_TOOL = {
type: 'function' as const,
function: {
name: 'extract_spec',
description: '输出 workflow 设计规范',
parameters: {
type: 'object',
properties: {
workflowName: {
type: 'string',
description: 'Workflow 名称(用作 event kind 前缀)',
},
description: {
type: 'string',
description: '一段话描述 workflow 目标',
},
roles: {
type: 'array',
items: {
type: 'object',
properties: {
name: { type: 'string', description: 'Role 名称' },
type: { type: 'string', enum: ['llm', 'llm-tool', 'agent', 'code'], description: '实现类型' },
description: { type: 'string', description: '职责描述' },
inputFrom: { type: 'string', description: '从哪个 role 获取输入' },
outputMeta: {
type: 'object',
description: 'meta 字段定义(field: type)',
additionalProperties: { type: 'string' },
},
},
required: ['name', 'type', 'description'],
},
},
transitions: {
type: 'array',
items: {
type: 'object',
properties: {
from: { type: 'string', description: '源 role(START 为起点)' },
to: { type: 'string', description: '目标 role(END 为终点)' },
condition: { type: 'string', description: '条件描述(如 verdict === approved)' },
},
required: ['from', 'to'],
},
description: '状态机转换规则',
},
acceptanceCriteria: {
type: 'array',
items: { type: 'string' },
description: '验收标准',
},
},
required: ['workflowName', 'description', 'roles', 'transitions', 'acceptanceCriteria'],
},
},
};
interface WorkflowSpec {
workflowName: string;
description: string;
roles: Array<{
name: string;
type: 'llm' | 'llm-tool' | 'agent' | 'code';
description: string;
inputFrom?: string;
outputMeta?: Record<string, string>;
}>;
transitions: Array<{
from: string;
to: string;
condition?: string;
}>;
acceptanceCriteria: string[];
}
const DEFAULT_SPEC: WorkflowSpec = {
workflowName: 'unknown',
description: '设计解析失败',
roles: [],
transitions: [],
acceptanceCriteria: [],
};
export function createMetaArchitectRole(llm: LlmClient): Role<MetaArchitectMeta> {
return createToolRole<MetaArchitectMeta, WorkflowSpec>(llm, {
systemPrompt: SYSTEM_PROMPT,
buildUserMessage: (chain) => {
const startMsg = chain.find((m) => m.role === '__start__');
return `Workflow 需求:\n\n${startMsg?.content ?? '(无描述)'}`;
},
tool: SPEC_TOOL,
defaultResult: DEFAULT_SPEC,
toRoleResult: (spec) => {
const transStr = spec.transitions
.map((t) => `${t.from}${t.to}${t.condition ? `(${t.condition})` : ''}`)
.join(', ');
return {
content: JSON.stringify(spec, null, 2),
meta: {
workflowName: spec.workflowName,
roles: spec.roles.map((r) => r.name),
transitions: transStr,
},
};
},
});
}
@@ -0,0 +1,89 @@
/**
* Meta Coder role — uses Cursor Agent to implement workflow code.
* Uses createAgentExecutorRole with LLM₂ meta parsing.
*
* 小橘 🍊 (NEKO Team)
*/
import type { LlmClient } from '../../llm-client.js';
import type { MetaCoderMeta } from '../meta-workflow.js';
import type { Role } from '../workflow-type.js';
import { type AgentRunner, createAgentExecutorRole } from './agent-executor.js';
const PARSE_META_TOOL = {
type: 'function' as const,
function: {
name: 'extract_coder_meta',
description: 'Extract coder execution metadata',
parameters: {
type: 'object',
properties: {
filesChanged: {
type: 'array',
items: { type: 'string' },
description: 'Files created or modified',
},
testsPassed: {
type: 'boolean',
description: 'Whether all tests passed',
},
},
required: ['filesChanged', 'testsPassed'],
},
},
};
export function createMetaCoderRole(
runner: AgentRunner,
llm: LlmClient,
repoDir: string,
): Role<MetaCoderMeta> {
return createAgentExecutorRole<MetaCoderMeta>(runner, llm, {
prepPrompt: (chain, _topicId) => {
const architectMsg = chain.find((m) => m.role === 'architect');
const spec = architectMsg?.content ?? '{}';
const reviewerMsg = [...chain].reverse().find((m) => m.role === 'reviewer');
const reviewFeedback = reviewerMsg
? `\n\n## 上次 Review 反馈\n${reviewerMsg.content}`
: '';
const prompt = `# 任务:实现 Pulse Workflow
## 设计规范
${spec}
${reviewFeedback}
## 参考
- 先阅读 docs/workflow-spec.md
- 参考 packages/pulse/src/workflows/coding-workflow.ts 和 report-workflow.ts
## 步骤
1. 创建 workflow 定义文件(meta types + moderator + factory)
2. 实现每个 role(用 llm-role-factory 或 agent-executor 工厂)
3. 写单元测试(mock roles)
4. 更新 index.ts barrel exports
5. 运行 \`cd packages/pulse && bun run build\`
6. 运行 \`bun test packages/pulse/src/workflows/\`
## 约束
- 不修改 workflow-rule-adapter.ts 和 workflow-type.ts
- Role 是纯函数
- commit author: 小橘 <xiaoju@shazhou.work>`;
return { prompt, cwd: repoDir };
},
parseMeta: {
system: '从 Cursor Agent 的输出中提取 coder 执行结果。',
tool: PARSE_META_TOOL,
parse: (args: string) => {
const parsed = JSON.parse(args);
return {
filesChanged: parsed.filesChanged ?? [],
testsPassed: parsed.testsPassed ?? false,
} as MetaCoderMeta;
},
defaultMeta: (_output: string) =>
({ filesChanged: [], testsPassed: false }) as MetaCoderMeta,
},
});
}
@@ -0,0 +1,50 @@
/**
* Meta Promoter role — pure code, git commit + push.
* No LLM needed.
*
* 小橘 🍊 (NEKO Team)
*/
import { execSync } from 'node:child_process';
import type { MetaPromoterMeta } from '../meta-workflow.js';
import type { Role, RoleResult, WorkflowMessage } from '../workflow-type.js';
export function createMetaPromoterRole(opts: {
repoDir: string;
remote?: string;
branch?: string;
}): Role<MetaPromoterMeta> {
const remote = opts.remote ?? 'gitflare';
const branch = opts.branch ?? 'main';
return async (chain: WorkflowMessage[]): Promise<RoleResult<MetaPromoterMeta>> => {
const architectMsg = chain.find((m) => m.role === 'architect');
let workflowName = 'unknown';
try {
const spec = JSON.parse(architectMsg?.content ?? '{}');
workflowName = spec.workflowName ?? 'unknown';
} catch {}
const cwd = opts.repoDir;
const exec = (cmd: string) =>
execSync(cmd, { cwd, encoding: 'utf-8', timeout: 30_000 }).trim();
// Stage + commit
exec('git add -A');
const commitMsg = `feat: ${workflowName} workflow — auto-generated by meta-workflow`;
exec(`git commit -m "${commitMsg}" --author="小橘 <xiaoju@shazhou.work>" --allow-empty`);
const commitHash = exec('git rev-parse --short HEAD');
// Push
let pushed = false;
try {
exec(`git push ${remote} ${branch} --no-verify`);
pushed = true;
} catch {}
return {
content: `Committed: ${commitHash}\nPushed: ${pushed ? 'yes' : 'no'}`,
meta: { commitHash, pushed },
};
};
}
@@ -0,0 +1,78 @@
/**
* Meta Reviewer role — uses Cursor Agent to review workflow implementation.
* Uses createAgentExecutorRole with LLM₂ meta parsing.
*
* 小橘 🍊 (NEKO Team)
*/
import type { LlmClient } from '../../llm-client.js';
import type { MetaReviewerMeta } from '../meta-workflow.js';
import type { Role } from '../workflow-type.js';
import { type AgentRunner, createAgentExecutorRole } from './agent-executor.js';
const PARSE_META_TOOL = {
type: 'function' as const,
function: {
name: 'extract_reviewer_meta',
description: 'Extract review result metadata',
parameters: {
type: 'object',
properties: {
verdict: {
type: 'string',
enum: ['approved', 'rejected'],
description: 'Review verdict',
},
comments: {
type: 'string',
description: 'Review comments summary',
},
},
required: ['verdict', 'comments'],
},
},
};
export function createMetaReviewerRole(
runner: AgentRunner,
llm: LlmClient,
repoDir: string,
): Role<MetaReviewerMeta> {
return createAgentExecutorRole<MetaReviewerMeta>(runner, llm, {
prepPrompt: (chain, _topicId) => {
const architectMsg = chain.find((m) => m.role === 'architect');
const spec = architectMsg?.content ?? '{}';
const prompt = `# 任务:审查 Pulse Workflow 实现
## 原始设计规范
${spec}
## 审查步骤
1. \`cd packages/pulse && bun run build\` — 编译
2. \`bun test packages/pulse/src/workflows/\` — 测试
3. 检查代码质量:
- Role 是否纯函数(不写 event)
- moderator 状态转换是否正确
- meta 字段是否只包含决策信号
- LLM role 是否用了 llm-role-factory
- barrel exports 是否更新
4. 对比设计规范验收标准`;
return { prompt, cwd: repoDir };
},
parseMeta: {
system: '从 Cursor Agent 的审查输出中提取 review 结果。',
tool: PARSE_META_TOOL,
parse: (args: string) => {
const parsed = JSON.parse(args);
return {
verdict: parsed.verdict === 'approved' ? 'approved' : 'rejected',
comments: parsed.comments ?? '',
} as MetaReviewerMeta;
},
defaultMeta: (_output: string) =>
({ verdict: 'rejected', comments: '无法解析审查结果' }) as MetaReviewerMeta,
},
});
}
@@ -0,0 +1,81 @@
/**
* Meta Tester role — runs build+test, then LLM judges the result.
*
* 小橘 🍊 (NEKO Team)
*/
import { execSync } from 'node:child_process';
import type { LlmClient } from '../../llm-client.js';
import type { MetaTesterMeta } from '../meta-workflow.js';
import type { Role, RoleResult, WorkflowMessage } from '../workflow-type.js';
const JUDGE_PROMPT = `你是 Pulse 工作流的测试评审官。
你收到一个 workflow 的 build + test 输出,需要判断是否通过。
判断标准:
1. 编译是否成功(无 TS 错误)
2. 测试是否全部通过(0 fail)
3. 是否有运行时异常
用 judge_result tool 输出判断。`;
const JUDGE_TOOL = {
type: 'function' as const,
function: {
name: 'judge_result',
description: '判断测试结果',
parameters: {
type: 'object',
properties: {
pass: { type: 'boolean', description: '是否通过' },
reason: { type: 'string', description: '判断理由' },
},
required: ['pass', 'reason'],
},
},
};
export function createMetaTesterRole(
llm: LlmClient,
opts: { repoDir: string },
): Role<MetaTesterMeta> {
return async (chain: WorkflowMessage[]): Promise<RoleResult<MetaTesterMeta>> => {
// Step 1: Run build + test
let testOutput: string;
try {
testOutput = execSync(
'cd packages/pulse && bun run build 2>&1 && bun test packages/pulse/src/workflows/ 2>&1',
{ cwd: opts.repoDir, timeout: 60_000, encoding: 'utf-8' },
);
} catch (err: any) {
testOutput = err.stdout ?? err.message;
}
// Step 2: LLM judges
const resp = await llm.chat({
messages: [
{ role: 'system', content: JUDGE_PROMPT },
{ role: 'user', content: testOutput.slice(0, 4000) },
],
tools: [JUDGE_TOOL],
tool_choice: 'required',
});
let pass = false;
let reason = '判断失败';
const toolCall = resp.tool_calls?.find((tc) => tc.function.name === 'judge_result');
if (toolCall) {
try {
const parsed = JSON.parse(toolCall.function.arguments);
pass = parsed.pass ?? false;
reason = parsed.reason ?? '';
} catch {}
}
return {
content: `${reason}\n\n---\n${testOutput.slice(0, 2000)}`,
meta: { pass, liveOutput: testOutput.slice(0, 2000) },
};
};
}