Compare commits

..

19 Commits

Author SHA1 Message Date
xingyue 8fe26417cf feat(cli): add --latest, --debug, --role flags to live command (#37 Phase 2)
- --latest: auto-find most recent thread by start timestamp
- --debug: display .info.jsonl debug log with tags
- --role: filter output to specific role
- Add live-argv.ts for flag parsing
- Add fixtures and test coverage for all flags

Testing: #50
2026-05-07 21:44:19 +08:00
xingyue 990200230b feat(cli): add live command for real-time thread monitoring (#37 Phase 1)
- Add cmd-live.ts: tail .data.jsonl with formatted output
- Display role steps with timestamp, role name, truncated content, meta
- fs.watch for running threads, auto-exit on completion
- Write WorkflowResult to .data.jsonl in worker.ts for completion detection
- Add live.test.ts with JSONL fixtures

Testing: #49
2026-05-07 21:42:32 +08:00
xiaoju 4eaefd9974 Merge pull request 'feat: tester role + develop workflow template' (#61) from feat/58-develop-workflow into main 2026-05-07 13:42:16 +00:00
xiaoju 1a685583bd feat: tester role + develop workflow template
- New workflow-role-tester: runs tests/build/lint, reports pass/fail
- Committer: removed push, only creates branch and commits
- New workflow-template-develop: planner → coder ⟲ → reviewer ⟲ → tester → committer
- 173 tests passing

Fixes #58
2026-05-07 13:42:01 +00:00
xiaomo 19769efea6 Merge pull request 'feat(cli): init command — scaffold workflow workspace' (#56) from feat/36-init-command into main 2026-05-07 13:37:56 +00:00
xiaoju 7f64541c5b Merge pull request 'feat: ReAct ExtractFn with tool-use' (#53) from feat/44-react-extract into main 2026-05-07 13:28:22 +00:00
xiaoju 43a6600378 feat: ReAct ExtractFn with tool-use
- RoleDefinition.extractMode: "single" | "react"
- reactExtract: multi-turn LLM with cas_get tool for DAG traversal
- Max 10 tool-call rounds, schema validation on final output
- create-workflow routes to reactExtract when extractMode is "react"
- All existing roles set to "single" (no behavior change)
- 162 tests passing

Fixes #44
2026-05-07 13:28:00 +00:00
xingyue 74e3f5434c feat(cli): complete AGENTS.md generation (#36 Phase 3)
- Replace placeholder with comprehensive coding agent instructions
- Covers: project structure, core concepts, dev workflow, coding
  conventions, template reuse, build/test, common pitfalls
- Add test coverage for AGENTS.md sections and terms

Testing: #48
2026-05-07 21:23:41 +08:00
xiaoju 220c9c5224 Merge pull request 'feat: global extract provider config' (#52) from feat/43-extract-provider-config into main 2026-05-07 13:21:57 +00:00
xiaoju cae59b589e feat: global extract provider config
- workflow.yaml supports config section (maxDepth, extract provider)
- ExtractProviderConfig with env: prefix for apiKey resolution
- getExtractProvider(storageRoot) returns LlmProvider from config
- workflowAsAgent uses config maxDepth (fallback 3)
- Registry read/write preserves config
- 158 tests passing

Fixes #43
2026-05-07 13:21:38 +00:00
xingyue 703ac9dfcc feat(cli): add init template command (#36 Phase 2)
- Implement cmdInitTemplate: find workspace root, generate template package
- Generate roles.ts, moderator.ts, index.ts with hello-world boilerplate
- Detect workspace by walking up to find package.json with workspaces
- Error on existing template dir or outside workspace
- Add init-template.test.ts

Testing: #47
2026-05-07 21:21:23 +08:00
xingyue 2df8accf2f feat(cli): add init workspace command (#36 Phase 1)
- Add cmd-init.ts with cmdInitWorkspace and stub cmdInitTemplate
- Wire init subcommands into cli-dispatch.ts
- Generate monorepo skeleton: package.json (bun workspace), biome.json,
  tsconfig.json, AGENTS.md placeholder, README.md, templates/, workflows/
- Error on existing directory
- Add init-workspace.test.ts (all passing)

Testing: #46
2026-05-07 21:18:58 +08:00
xiaoju b5cc0db17e Merge pull request 'feat: thread root node + workflowAsAgent returns root hash' (#51) from feat/42-thread-root-node into main 2026-05-07 13:18:13 +00:00
xiaoju 6196e0974a feat: thread root node + workflowAsAgent returns root hash
- Engine generates step Merkle nodes (type: step) after each role step
- Engine generates thread root Merkle node (type: thread) on completion
- WorkflowResult gains rootHash field
- WorkflowFn returns WorkflowCompletion (no rootHash), engine wraps with rootHash
- workflowAsAgent returns rootHash instead of summary
- Full DAG traversal: root → steps → content
- 151 tests passing

Fixes #42
2026-05-07 13:17:44 +00:00
xiaoju 410e9e6d9b Merge pull request 'feat: Merkle node format + content → CAS' (#45) from feat/41-merkle-content-cas into main 2026-05-07 13:14:16 +00:00
xiaoju 84de74721d feat: Merkle node format + content → CAS
- MerkleNode type: { type, payload, children } serialized as YAML
- RoleOutput.content → contentHash (CAS hash of Merkle content node)
- Engine stores content in global CAS before writing to .data.jsonl
- create-workflow puts content as Merkle node, merges contentHash into refs
- fork/parse adapted for contentHash format
- buildAgentPrompt now async, reads content from CAS
- Bundle validator allows @uncaged/workflow import
- 150 tests passing

BREAKING: .data.jsonl no longer contains inline content

Fixes #41
2026-05-07 13:14:01 +00:00
xiaoju 4403532f35 Merge pull request 'feat: workflowAsAgent factory' (#39) from feat/33-workflow-as-agent into main 2026-05-07 10:52:40 +00:00
xiaoju e95e76c145 feat: workflowAsAgent factory
- workflowAsAgent(name) resolves via registry → bundle → child thread
- System-level depth limit (max 3, constant)
- Returns summary string, errors as string (no throw)
- Integration test with nested workflow execution
- 146 tests passing

Fixes #33
2026-05-07 10:52:26 +00:00
xiaoju af69e773a0 Merge pull request 'feat: CAS garbage collection' (#38) from feat/32-cas-gc into main 2026-05-07 10:48:07 +00:00
79 changed files with 4948 additions and 314 deletions
+2
View File
@@ -29,6 +29,7 @@ const greeter: RoleDefinition<Roles["greeter"]> = {
extractPrompt: "Extract the greeting string produced for the user.",
schema: greeterMetaSchema,
extractRefs: null,
extractMode: "single",
};
const extract = createExtract({
@@ -48,4 +49,5 @@ export const run = createWorkflow<Roles>(
agent: async (ctx) => `Hello, ${ctx.start.content}`,
},
extract,
null,
);
@@ -16,6 +16,9 @@ import { addCliArgs } from "./bundle-fixture.js";
const fixtureDescriptor = `export const descriptor = { description: "fixture", roles: {} };
`;
const wfPutImport = `import { putContentMerkleNode } from "@uncaged/workflow";
`;
describe("cli workflow commands", () => {
let prevEnv: string | undefined;
let storageRoot: string;
@@ -41,11 +44,13 @@ describe("cli workflow commands", () => {
const bundlePath = join(bundleDir, "demo.esm.js");
await writeFile(
bundlePath,
`${fixtureDescriptor}import fs from "node:fs";
`${fixtureDescriptor}${wfPutImport}import fs from "node:fs";
export const run = async function* (input) {
export const run = async function* (input, options) {
fs.existsSync(".");
yield { role: "noop", content: input.prompt, meta: { done: true } };
const cas = options.cas;
const h = await putContentMerkleNode(cas, input.prompt);
yield { role: "noop", contentHash: h, meta: { done: true }, refs: [h] };
return { returnCode: 0, summary: "done" };
}
`,
@@ -112,8 +117,8 @@ export const run = async function* (input) { return { returnCode: 0, summary: in
const bundlePath = join(storageRoot, "solo.esm.js");
await writeFile(
bundlePath,
`export const run = async function* (input) {
yield { role: "x", content: input.prompt, meta: {} };
`export const run = async function* () {
yield { role: "x", contentHash: "STUBHASH00000000000000001", meta: {}, refs: [] };
return { returnCode: 0, summary: "ok" };
}
`,
@@ -141,8 +146,11 @@ export const run = async function* (input) { return { returnCode: 0, summary: in
},
},
};
export const run = async function* (input) {
yield { role: "greeter", content: input.prompt, meta: { greeting: "hi" } };
${wfPutImport}
export const run = async function* (input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, input.prompt);
yield { role: "greeter", contentHash: h, meta: { greeting: "hi" }, refs: [h] };
return { returnCode: 0, summary: "ok" };
};
`,
@@ -180,8 +188,10 @@ export const run = async function* (input) {
const bundlePath = join(bundleDir, "demo.esm.js");
await writeFile(
bundlePath,
`${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "x", meta: {} };
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "x");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "x" };
}
`,
@@ -209,8 +219,10 @@ export const run = async function* (input) {
const dtsPath = join(bundleDir, "types.d.ts");
await writeFile(
bundlePath,
`${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "x", meta: {} };
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "x");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "x" };
}
`,
@@ -240,8 +252,10 @@ export const run = async function* (input) {
const bundlePath = join(bundleDir, "demo.esm.js");
await writeFile(
bundlePath,
`${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "x", meta: {} };
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "x");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "x" };
}
`,
@@ -261,13 +275,17 @@ export const run = async function* (input) {
const bundleDir = join(storageRoot, "src");
await mkdir(bundleDir, { recursive: true });
const bundlePath = join(bundleDir, "demo.esm.js");
const v1 = `${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "v1", meta: {} };
const v1 = `${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "v1");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "v1" };
}
`;
const v2 = `${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "v2", meta: {} };
const v2 = `${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "v2");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "v2" };
}
`;
@@ -299,13 +317,17 @@ export const run = async function* (input) {
const bundleDir = join(storageRoot, "src");
await mkdir(bundleDir, { recursive: true });
const bundlePath = join(bundleDir, "demo.esm.js");
const v1 = `${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "v1", meta: {} };
const v1 = `${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "v1");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "v1" };
}
`;
const v2 = `${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "v2", meta: {} };
const v2 = `${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "v2");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "v2" };
}
`;
@@ -347,8 +369,10 @@ export const run = async function* (input) {
const bundlePath = join(bundleDir, "demo.esm.js");
await writeFile(
bundlePath,
`${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "x", meta: {} };
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "x");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "x" };
}
`,
@@ -358,8 +382,10 @@ export const run = async function* (input) {
expect(add1.ok).toBe(true);
await writeFile(
bundlePath,
`${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "y", meta: {} };
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "y");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "y" };
}
`,
@@ -409,8 +435,10 @@ export const run = async function* (input) {
const bundlePath = join(bundleDir, "demo.esm.js");
await writeFile(
bundlePath,
`${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "x", meta: {} };
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "x");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "x" };
}
`,
@@ -424,8 +452,10 @@ export const run = async function* (input) {
const hash1 = add1.value.hash;
await writeFile(
bundlePath,
`${fixtureDescriptor}export const run = async function* (input) {
yield { role: "a", content: "y", meta: {} };
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "y");
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "y" };
}
`,
@@ -0,0 +1,4 @@
{"name":"demo-live","hash":"C9NMV6V2TQT81","threadId":"01LIVECMPLT01DDDDDDDDDDDDG","parameters":{"prompt":"hello","options":{"maxRounds":5,"depth":0}},"timestamp":1714963400000}
{"role":"planner","contentHash":"FF7YQ5W3S2EV6","meta":{"phase":"plan","flags":[1,2]},"refs":[],"timestamp":1714963201000}
{"role":"coder","contentHash":"EN34XX1W4WAFJ","meta":{},"refs":[],"timestamp":1714963202000}
{"returnCode":0,"summary":"fixture completed"}
@@ -0,0 +1,2 @@
{"tag":"DEBUGTAG1","content":"bundle loaded","timestamp":1714963400050}
{"tag":"DEBUGTAG2","content":"multi\nline","timestamp":1714963400500}
@@ -0,0 +1,2 @@
{"name":"demo-live","hash":"C9NMV6V2TQT81","threadId":"01LIVEINFLY01DDDDDDDDDDDDG","parameters":{"prompt":"hello","options":{"maxRounds":5,"depth":0}},"timestamp":1714963200000}
{"role":"planner","contentHash":"P6M9FHE1GSBN0","meta":{"x":1},"refs":[],"timestamp":1714963201000}
@@ -0,0 +1,2 @@
{"name":"demo-live-old","hash":"C9NMV6V2TQT81","threadId":"01LIVEOLDER01DDDDDDDDDDDDG","parameters":{"prompt":"old","options":{"maxRounds":5,"depth":0}},"timestamp":1714963000000}
{"returnCode":0,"summary":"older thread"}
@@ -2,6 +2,7 @@ import { afterEach, beforeEach, describe, expect, test } from "bun:test";
import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createCasStore, getContentMerklePayload, getGlobalCasDir } from "@uncaged/workflow";
import { cmdAdd } from "../src/cmd-add.js";
import { cmdFork } from "../src/cmd-fork.js";
import { cmdRun } from "../src/cmd-run.js";
@@ -9,7 +10,9 @@ import { pathExists } from "../src/fs-utils.js";
import { addCliArgs } from "./bundle-fixture.js";
/** Three-role workflow that respects `input.steps` for fork/resume. */
const threeRoleBundleSource = `export const descriptor = {
const threeRoleBundleSource = `import { putContentMerkleNode } from "@uncaged/workflow";
export const descriptor = {
description: "fork-cli",
roles: {
planner: { description: "planner", schema: {} },
@@ -17,20 +20,21 @@ const threeRoleBundleSource = `export const descriptor = {
reviewer: { description: "reviewer", schema: {} },
},
};
export const run = async function* (input) {
export const run = async function* (input, options) {
const cas = options.cas;
const has = (r) => input.steps.some((s) => s.role === r);
if (!has("planner")) {
yield { role: "planner", content: "p1", meta: { k: "planner" } };
const h = await putContentMerkleNode(cas, "p1");
yield { role: "planner", contentHash: h, meta: { k: "planner" }, refs: [h] };
}
if (!has("coder")) {
yield { role: "coder", content: "c1", meta: { k: "coder" } };
const h = await putContentMerkleNode(cas, "c1");
yield { role: "coder", contentHash: h, meta: { k: "coder" }, refs: [h] };
}
if (!has("reviewer")) {
yield {
role: "reviewer",
content: "rev-" + String(input.steps.length),
meta: { k: "reviewer" },
};
const body = "rev-" + String(input.steps.length);
const h = await putContentMerkleNode(cas, body);
yield { role: "reviewer", contentHash: h, meta: { k: "reviewer" }, refs: [h] };
}
return { returnCode: 0, summary: "done" };
};
@@ -107,7 +111,7 @@ describe("cli fork", () => {
const sourceData = join(storageRoot, "logs", hash, `${sourceId}.data.jsonl`);
const sourceRunning = join(storageRoot, "logs", hash, `${sourceId}.running`);
await waitUntilRunningAbsent(sourceRunning);
await waitUntilMinDataLines(sourceData, 4);
await waitUntilMinDataLines(sourceData, 5);
const forked = await cmdFork(storageRoot, sourceId, "planner");
expect(forked.ok).toBe(true);
@@ -118,21 +122,22 @@ describe("cli fork", () => {
const newData = join(storageRoot, "logs", hash, `${newId}.data.jsonl`);
const newRunning = join(storageRoot, "logs", hash, `${newId}.running`);
await waitUntilRunningAbsent(newRunning);
await waitUntilMinDataLines(newData, 4);
await waitUntilMinDataLines(newData, 5);
const text = await readFile(newData, "utf8");
const lines = text
.trim()
.split("\n")
.filter((l) => l !== "");
expect(lines.length).toBe(4);
expect(lines.length).toBe(5);
const start = JSON.parse(lines[0] ?? "{}") as Record<string, unknown>;
expect(start.threadId).toBe(newId);
expect(start.forkFrom).toEqual({ threadId: sourceId });
const last = JSON.parse(lines[lines.length - 1] ?? "{}") as Record<string, unknown>;
expect(last.role).toBe("reviewer");
expect(last.content).toBe("rev-1");
const lastRoleLine = JSON.parse(lines[lines.length - 2] ?? "{}") as Record<string, unknown>;
expect(lastRoleLine.role).toBe("reviewer");
const cas = createCasStore(getGlobalCasDir(storageRoot));
expect(await getContentMerklePayload(cas, String(lastRoleLine.contentHash))).toBe("rev-1");
});
test("fork without --from-role retries last role", async () => {
@@ -157,7 +162,7 @@ describe("cli fork", () => {
const sourceData = join(storageRoot, "logs", hash, `${sourceId}.data.jsonl`);
const sourceRunning = join(storageRoot, "logs", hash, `${sourceId}.running`);
await waitUntilRunningAbsent(sourceRunning);
await waitUntilMinDataLines(sourceData, 4);
await waitUntilMinDataLines(sourceData, 5);
const forked = await cmdFork(storageRoot, sourceId, null);
expect(forked.ok).toBe(true);
@@ -168,22 +173,23 @@ describe("cli fork", () => {
const newData = join(storageRoot, "logs", hash, `${newId}.data.jsonl`);
const newRunning = join(storageRoot, "logs", hash, `${newId}.running`);
await waitUntilRunningAbsent(newRunning);
await waitUntilMinDataLines(newData, 4);
await waitUntilMinDataLines(newData, 5);
const text = await readFile(newData, "utf8");
const lines = text
.trim()
.split("\n")
.filter((l) => l !== "");
expect(lines.length).toBe(4);
expect(lines.length).toBe(5);
const replayCoder = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
expect(replayCoder.role).toBe("coder");
expect(replayCoder.content).toBe("c1");
const cas = createCasStore(getGlobalCasDir(storageRoot));
expect(await getContentMerklePayload(cas, String(replayCoder.contentHash))).toBe("c1");
const last = JSON.parse(lines[lines.length - 1] ?? "{}") as Record<string, unknown>;
expect(last.role).toBe("reviewer");
expect(last.content).toBe("rev-2");
const lastRoleLine = JSON.parse(lines[lines.length - 2] ?? "{}") as Record<string, unknown>;
expect(lastRoleLine.role).toBe("reviewer");
expect(await getContentMerklePayload(cas, String(lastRoleLine.contentHash))).toBe("rev-2");
});
test("fork rejects unknown role with available names", async () => {
@@ -207,7 +213,7 @@ describe("cli fork", () => {
const sourceData = join(storageRoot, "logs", added.value.hash, `${sourceId}.data.jsonl`);
const sourceRunning = join(storageRoot, "logs", added.value.hash, `${sourceId}.running`);
await waitUntilRunningAbsent(sourceRunning);
await waitUntilMinDataLines(sourceData, 4);
await waitUntilMinDataLines(sourceData, 5);
const bad = await cmdFork(storageRoot, sourceId, "ghost-role");
expect(bad.ok).toBe(false);
+43 -25
View File
@@ -4,31 +4,43 @@ import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { createCasStore, garbageCollectCas, getGlobalCasDir } from "@uncaged/workflow";
import {
createCasStore,
garbageCollectCas,
getGlobalCasDir,
putContentMerkleNode,
} from "@uncaged/workflow";
import { cmdThreadRemove } from "../src/cmd-thread.js";
import { pathExists } from "../src/fs-utils.js";
const cliEntryPath = fileURLToPath(new URL("../src/cli.ts", import.meta.url));
/** Minimal valid `.data.jsonl` with one role step referencing `activeHash` in `refs`. */
function makeDataJsonl(threadId: string, bundleHash: string, activeHash: string): string {
return [
async function writeDemoDataJsonl(params: {
path: string;
threadId: string;
bundleHash: string;
cas: ReturnType<typeof createCasStore>;
activeHash: string;
}): Promise<void> {
const bodyHash = await putContentMerkleNode(params.cas, "p");
const text = [
JSON.stringify({
name: "demo",
hash: bundleHash,
threadId,
hash: params.bundleHash,
threadId: params.threadId,
parameters: { prompt: "hi", options: { maxRounds: 5 } },
timestamp: 100,
}),
JSON.stringify({
role: "planner",
content: "p",
contentHash: bodyHash,
meta: {},
refs: [activeHash],
refs: [params.activeHash, bodyHash],
timestamp: 101,
}),
"",
].join("\n");
await writeFile(params.path, text, "utf8");
}
describe("gc cli and garbageCollectCas", () => {
@@ -60,11 +72,13 @@ describe("gc cli and garbageCollectCas", () => {
const activeHash = await cas.put("active-blob");
const orphanHash = await cas.put("orphan-blob");
await writeFile(
join(logsDir, `${threadId}.data.jsonl`),
makeDataJsonl(threadId, bundleHash, activeHash),
"utf8",
);
await writeDemoDataJsonl({
path: join(logsDir, `${threadId}.data.jsonl`),
threadId,
bundleHash,
cas,
activeHash,
});
const gc = await garbageCollectCas(storageRoot);
expect(gc.ok).toBe(true);
@@ -72,7 +86,7 @@ describe("gc cli and garbageCollectCas", () => {
return;
}
expect(gc.value.scannedThreads).toBe(1);
expect(gc.value.activeRefs).toBe(1);
expect(gc.value.activeRefs).toBe(2);
expect(gc.value.deletedEntries).toBe(1);
expect(gc.value.deletedHashes).toEqual([orphanHash]);
@@ -106,16 +120,18 @@ describe("gc cli and garbageCollectCas", () => {
const activeHash = await cas.put("keep-me");
await cas.put("drop-me");
await writeFile(
join(logsDir, `${threadId}.data.jsonl`),
makeDataJsonl(threadId, bundleHash, activeHash),
"utf8",
);
await writeDemoDataJsonl({
path: join(logsDir, `${threadId}.data.jsonl`),
threadId,
bundleHash,
cas,
activeHash,
});
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const proc = spawnSync(process.execPath, [cliEntryPath, "gc"], { env, encoding: "utf8" });
expect(proc.status).toBe(0);
expect(String(proc.stdout).trim()).toBe("scanned 1 threads, 1 active refs, deleted 1 entries");
expect(String(proc.stdout).trim()).toBe("scanned 1 threads, 2 active refs, deleted 1 entries");
});
test("thread rm triggers gc so unreferenced CAS is removed", async () => {
@@ -126,11 +142,13 @@ describe("gc cli and garbageCollectCas", () => {
const cas = createCasStore(getGlobalCasDir(storageRoot));
const activeHash = await cas.put("pinned-by-ref");
await writeFile(
join(logsDir, `${threadId}.data.jsonl`),
makeDataJsonl(threadId, bundleHash, activeHash),
"utf8",
);
await writeDemoDataJsonl({
path: join(logsDir, `${threadId}.data.jsonl`),
threadId,
bundleHash,
cas,
activeHash,
});
const orphanHash = await cas.put("orphan-after-rm");
const orphanPath = join(getGlobalCasDir(storageRoot), `${orphanHash}.txt`);
@@ -0,0 +1,142 @@
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
import { mkdir, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { runCli } from "../src/cli-dispatch.js";
import { cmdInitTemplate, cmdInitWorkspace } from "../src/cmd-init.js";
import { pathExists } from "../src/fs-utils.js";
describe("init template", () => {
let parent: string;
beforeEach(async () => {
parent = join(
tmpdir(),
`wf-init-template-${Date.now()}-${Math.random().toString(36).slice(2)}`,
);
await mkdir(parent, { recursive: true });
});
afterEach(async () => {
await rm(parent, { recursive: true, force: true });
});
test("creates templates/<name> with expected files", async () => {
const ws = await cmdInitWorkspace(parent, "my-workflows");
expect(ws.ok).toBe(true);
if (!ws.ok) {
return;
}
const root = ws.value.rootPath;
const created = await cmdInitTemplate(root, "review-pr");
expect(created.ok).toBe(true);
if (!created.ok) {
return;
}
const tdir = join(root, "templates", "review-pr");
expect(created.value.templatePath).toBe(tdir);
expect(await pathExists(join(tdir, "package.json"))).toBe(true);
expect(await pathExists(join(tdir, "tsconfig.json"))).toBe(true);
expect(await pathExists(join(tdir, "src", "roles.ts"))).toBe(true);
expect(await pathExists(join(tdir, "src", "moderator.ts"))).toBe(true);
expect(await pathExists(join(tdir, "src", "index.ts"))).toBe(true);
const pkg = JSON.parse(await readFile(join(tdir, "package.json"), "utf8")) as {
name: string;
type: string;
dependencies: Record<string, string>;
};
expect(pkg.type).toBe("module");
expect(pkg.dependencies["@uncaged/workflow"]).toBeDefined();
expect(pkg.dependencies.zod).toBeDefined();
expect(pkg.name).toContain("review-pr");
const idx = await readFile(join(tdir, "src", "index.ts"), "utf8");
expect(idx).toContain("WorkflowDefinition");
const roles = await readFile(join(tdir, "src", "roles.ts"), "utf8");
expect(roles).not.toContain("interface ");
expect(roles).not.toContain("?:");
expect(roles).not.toContain("export default");
const moder = await readFile(join(tdir, "src", "moderator.ts"), "utf8");
expect(moder).not.toContain("export default");
});
test("finds workspace walking up from nested cwd", async () => {
const ws = await cmdInitWorkspace(parent, "ws");
expect(ws.ok).toBe(true);
if (!ws.ok) {
return;
}
const root = ws.value.rootPath;
const nested = join(root, "a", "b");
await mkdir(nested, { recursive: true });
const created = await cmdInitTemplate(nested, "nested-tpl");
expect(created.ok).toBe(true);
if (!created.ok) {
return;
}
expect(await pathExists(join(root, "templates", "nested-tpl", "src", "index.ts"))).toBe(true);
});
test("errors when not inside a workflow workspace", async () => {
const orphan = join(parent, "nowhere");
await mkdir(orphan, { recursive: true });
const r = await cmdInitTemplate(orphan, "x");
expect(r.ok).toBe(false);
if (!r.ok) {
expect(r.error).toContain("templates/*");
}
});
test("errors when template directory already exists", async () => {
const ws = await cmdInitWorkspace(parent, "ws");
expect(ws.ok).toBe(true);
if (!ws.ok) {
return;
}
const root = ws.value.rootPath;
const first = await cmdInitTemplate(root, "dup");
expect(first.ok).toBe(true);
const second = await cmdInitTemplate(root, "dup");
expect(second.ok).toBe(false);
if (!second.ok) {
expect(second.error).toContain("already exists");
}
});
test("errors on invalid template name", async () => {
const ws = await cmdInitWorkspace(parent, "ws");
expect(ws.ok).toBe(true);
if (!ws.ok) {
return;
}
const bad = await cmdInitTemplate(ws.value.rootPath, "a/b");
expect(bad.ok).toBe(false);
});
test.serial("runCli init template uses cwd and succeeds in workspace", async () => {
const ws = await cmdInitWorkspace(parent, "cli-ws");
expect(ws.ok).toBe(true);
if (!ws.ok) {
return;
}
const root = ws.value.rootPath;
const prev = process.cwd();
try {
process.chdir(root);
const code = await runCli(join(parent, "_storage"), ["init", "template", "from-cli"]);
expect(code).toBe(0);
expect(await pathExists(join(root, "templates", "from-cli", "package.json"))).toBe(true);
} finally {
process.chdir(prev);
}
});
});
@@ -0,0 +1,152 @@
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
import { mkdir, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { formatCliUsage, runCli } from "../src/cli-dispatch.js";
import { cmdInitWorkspace } from "../src/cmd-init.js";
import { pathExists } from "../src/fs-utils.js";
describe("init workspace", () => {
let parent: string;
beforeEach(async () => {
parent = join(tmpdir(), `wf-init-${Date.now()}-${Math.random().toString(36).slice(2)}`);
await mkdir(parent, { recursive: true });
});
afterEach(async () => {
await rm(parent, { recursive: true, force: true });
});
test("creates expected files and directories", async () => {
const created = await cmdInitWorkspace(parent, "my-workflows");
expect(created.ok).toBe(true);
if (!created.ok) {
return;
}
const root = created.value.rootPath;
expect(await pathExists(join(root, "package.json"))).toBe(true);
expect(await pathExists(join(root, "biome.json"))).toBe(true);
expect(await pathExists(join(root, "tsconfig.json"))).toBe(true);
expect(await pathExists(join(root, "AGENTS.md"))).toBe(true);
expect(await pathExists(join(root, "README.md"))).toBe(true);
expect(await pathExists(join(root, "templates"))).toBe(true);
expect(await pathExists(join(root, "templates", ".gitkeep"))).toBe(true);
expect(await pathExists(join(root, "workflows", "package.json"))).toBe(true);
const rootPkg = JSON.parse(await readFile(join(root, "package.json"), "utf8")) as {
workspaces: string[];
};
expect(rootPkg.workspaces).toEqual(["templates/*", "workflows"]);
const wfPkg = JSON.parse(await readFile(join(root, "workflows", "package.json"), "utf8")) as {
type: string;
dependencies: Record<string, string>;
};
expect(wfPkg.type).toBe("module");
expect(wfPkg.dependencies["@uncaged/workflow"]).toBeDefined();
expect(wfPkg.dependencies.zod).toBeDefined();
const tsconfig = JSON.parse(await readFile(join(root, "tsconfig.json"), "utf8")) as {
compilerOptions: { strict: boolean; module: string; target: string };
};
expect(tsconfig.compilerOptions.strict).toBe(true);
expect(tsconfig.compilerOptions.module).toBe("ESNext");
expect(tsconfig.compilerOptions.target).toBe("ESNext");
});
test("AGENTS.md contains coding agent guide sections and terms", async () => {
const created = await cmdInitWorkspace(parent, "my-workflows");
expect(created.ok).toBe(true);
if (!created.ok) {
return;
}
const agentsPath = join(created.value.rootPath, "AGENTS.md");
const body = await readFile(agentsPath, "utf8");
for (const section of [
"项目结构",
"核心概念",
"开发流程",
"编码规范",
"Template",
"Build",
"常见陷阱",
]) {
expect(body).toContain(section);
}
for (const term of [
"RoleDefinition",
"WorkflowDefinition",
"Moderator",
"AgentFn",
"ExtractFn",
"RoleMeta",
]) {
expect(body).toContain(term);
}
expect(body).toMatch(/type[\s\S]*interface/i);
expect(body).toMatch(/function[\s\S]*class/i);
expect(body).toContain("Crockford Base32");
expect(body).toMatch(/no[\s\S]*default export/i);
expect(body).toMatch(/no[\s\S]*console/i);
expect(body).toMatch(/no[\s\S]*dynamic import/i);
expect(body).toContain("bun run check");
expect(body).toContain("bun test");
expect(body).toContain("uncaged-workflow");
expect(body).toContain("bun build");
expect(body).toContain("CLAUDE.md");
expect(body).toContain("docs/architecture.md");
});
test("errors when directory already exists", async () => {
const first = await cmdInitWorkspace(parent, "dup");
expect(first.ok).toBe(true);
const second = await cmdInitWorkspace(parent, "dup");
expect(second.ok).toBe(false);
if (!second.ok) {
expect(second.error).toContain("already exists");
}
});
test("errors on invalid workspace name", async () => {
const slash = await cmdInitWorkspace(parent, "a/b");
expect(slash.ok).toBe(false);
const dots = await cmdInitWorkspace(parent, "..");
expect(dots.ok).toBe(false);
const empty = await cmdInitWorkspace(parent, "");
expect(empty.ok).toBe(false);
});
test("usage lists init subcommands", () => {
const u = formatCliUsage();
expect(u).toContain("uncaged-workflow init workspace <name>");
expect(u).toContain("uncaged-workflow init template <name>");
});
test("runCli rejects unknown init subcommand", async () => {
const code = await runCli(join(parent, "_storage"), ["init", "bogus", "name"]);
expect(code).toBe(1);
});
test.serial("runCli init workspace uses cwd", async () => {
const prev = process.cwd();
try {
process.chdir(parent);
const code = await runCli(join(parent, "_storage"), ["init", "workspace", "from-cli"]);
expect(code).toBe(0);
expect(await pathExists(join(parent, "from-cli", "workflows", "package.json"))).toBe(true);
} finally {
process.chdir(prev);
}
});
});
@@ -0,0 +1,369 @@
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
import { spawn, spawnSync } from "node:child_process";
import { cp, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { createCasStore, getGlobalCasDir, putContentMerkleNode } from "@uncaged/workflow";
import {
formatLiveDebugLine,
formatLiveTimeLabel,
LIVE_CONTENT_MAX_LINES,
type LiveRoleRow,
renderLiveRoleStepLines,
} from "../src/cmd-live.js";
import { parseLiveArgv } from "../src/live-argv.js";
const cliEntryPath = fileURLToPath(new URL("../src/cli.ts", import.meta.url));
const fixtureRoot = fileURLToPath(new URL("./fixtures/live", import.meta.url));
/** Bodies for Merkle content nodes; hashes must match `.data.jsonl` fixtures. */
const LIVE_FIXTURE_PLANNER_BODY =
"alpha\nbeta\ngamma\nLINE4\nLINE5\nLINE6\nLINE7\nLINE8\nLINE9\nLINE10\nLINE11";
describe("live helpers", () => {
test("formatLiveTimeLabel pads HH:MM:SS", () => {
const label = formatLiveTimeLabel(new Date("2024-06-01T09:08:07.000Z").getTime());
expect(label).toMatch(/^\d{2}:\d{2}:\d{2}$/);
});
test("formatLiveDebugLine flattens newlines in message", () => {
const line = formatLiveDebugLine(0, "TAG1", "a\nb");
expect(line).toContain("[TAG1]");
expect(line).toContain("a b");
expect(line).not.toContain("\n");
});
test("renderLiveRoleStepLines truncates content to LIVE_CONTENT_MAX_LINES", () => {
const lines = Array.from({ length: LIVE_CONTENT_MAX_LINES + 3 }, (_, i) => `L${i + 1}`);
const row: LiveRoleRow = {
role: "r",
content: lines.join("\n"),
meta: { k: "v" },
timestamp: 0,
};
const out = renderLiveRoleStepLines(row, "r");
const body = out.filter((l) => l.startsWith(" L"));
expect(body.length).toBe(LIVE_CONTENT_MAX_LINES);
expect(out.some((l) => l.includes("more line"))).toBe(true);
expect(out.some((l) => l.startsWith(" meta: "))).toBe(true);
});
});
describe("parseLiveArgv", () => {
test("parses thread id and flags in any order", () => {
const a = parseLiveArgv(["01ABC", "--debug", "--role", "planner"]);
expect(a.ok).toBe(true);
if (a.ok) {
expect(a.value.threadId).toBe("01ABC");
expect(a.value.latest).toBe(false);
expect(a.value.debug).toBe(true);
expect(a.value.role).toBe("planner");
}
const b = parseLiveArgv(["--latest", "--role", "x"]);
expect(b.ok).toBe(true);
if (b.ok) {
expect(b.value.latest).toBe(true);
expect(b.value.threadId).toBe(null);
expect(b.value.role).toBe("x");
}
});
test("rejects --latest with thread id", () => {
const r = parseLiveArgv(["--latest", "01ABC"]);
expect(r.ok).toBe(false);
});
});
describe("live CLI", () => {
let prevEnv: string | undefined;
let storageRoot: string;
beforeEach(async () => {
prevEnv = process.env.UNCAGED_WORKFLOW_STORAGE_ROOT;
storageRoot = await mkdtemp(join(tmpdir(), "uncaged-wf-live-"));
process.env.UNCAGED_WORKFLOW_STORAGE_ROOT = storageRoot;
await mkdir(join(storageRoot, "logs", "C9NMV6V2TQT81"), { recursive: true });
await cp(
join(fixtureRoot, "logs", "C9NMV6V2TQT81", "01LIVECMPLT01DDDDDDDDDDDDG.data.jsonl"),
join(storageRoot, "logs", "C9NMV6V2TQT81", "01LIVECMPLT01DDDDDDDDDDDDG.data.jsonl"),
);
await cp(
join(fixtureRoot, "logs", "C9NMV6V2TQT81", "01LIVECMPLT01DDDDDDDDDDDDG.info.jsonl"),
join(storageRoot, "logs", "C9NMV6V2TQT81", "01LIVECMPLT01DDDDDDDDDDDDG.info.jsonl"),
);
await cp(
join(fixtureRoot, "logs", "C9NMV6V2TQT81", "01LIVEINFLY01DDDDDDDDDDDDG.data.jsonl"),
join(storageRoot, "logs", "C9NMV6V2TQT81", "01LIVEINFLY01DDDDDDDDDDDDG.data.jsonl"),
);
await cp(
join(fixtureRoot, "logs", "C9NMV6V2TQT81", "01LIVEOLDER01DDDDDDDDDDDDG.data.jsonl"),
join(storageRoot, "logs", "C9NMV6V2TQT81", "01LIVEOLDER01DDDDDDDDDDDDG.data.jsonl"),
);
const cas = createCasStore(getGlobalCasDir(storageRoot));
await putContentMerkleNode(cas, LIVE_FIXTURE_PLANNER_BODY);
await putContentMerkleNode(cas, "patch");
await putContentMerkleNode(cas, "still running");
});
afterEach(async () => {
if (prevEnv === undefined) {
delete process.env.UNCAGED_WORKFLOW_STORAGE_ROOT;
} else {
process.env.UNCAGED_WORKFLOW_STORAGE_ROOT = prevEnv;
}
await rm(storageRoot, { recursive: true, force: true });
});
test("prints role steps and summary for a completed thread", async () => {
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const proc = spawn(process.execPath, [cliEntryPath, "live", "01LIVECMPLT01DDDDDDDDDDDDG"], {
env,
stdio: ["ignore", "pipe", "pipe"],
});
const stdout = await new Promise<string>((resolve, reject) => {
let buf = "";
proc.stdout?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.stderr?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.on("error", reject);
proc.on("exit", (code: number | null) => {
if (code === 0) {
resolve(buf);
} else {
reject(new Error(`exit ${code}: ${buf}`));
}
});
});
expect(stdout).toContain("planner");
expect(stdout).toContain("coder");
expect(stdout).toContain("meta:");
expect(stdout).toContain('"phase":"plan"');
expect(stdout).toContain("LINE10");
expect(stdout).not.toContain("LINE11");
expect(stdout).toContain("more line");
expect(stdout).toContain("completed: returnCode=0");
expect(stdout).toContain("fixture completed");
});
test("--latest tails the newest thread by start timestamp", async () => {
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const proc = spawn(process.execPath, [cliEntryPath, "live", "--latest"], {
env,
stdio: ["ignore", "pipe", "pipe"],
});
const stdout = await new Promise<string>((resolve, reject) => {
let buf = "";
proc.stdout?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.stderr?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.on("error", reject);
proc.on("exit", (code: number | null) => {
if (code === 0) {
resolve(buf);
} else {
reject(new Error(`exit ${code}: ${buf}`));
}
});
});
expect(stdout).toContain("fixture completed");
expect(stdout).not.toContain("older thread");
});
test("--debug prints .info.jsonl records after data output", async () => {
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const proc = spawn(
process.execPath,
[cliEntryPath, "live", "01LIVECMPLT01DDDDDDDDDDDDG", "--debug"],
{
env,
stdio: ["ignore", "pipe", "pipe"],
},
);
const stdout = await new Promise<string>((resolve, reject) => {
let buf = "";
proc.stdout?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.stderr?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.on("error", reject);
proc.on("exit", (code: number | null) => {
if (code === 0) {
resolve(buf);
} else {
reject(new Error(`exit ${code}: ${buf}`));
}
});
});
expect(stdout).toContain("[DEBUGTAG1]");
expect(stdout).toContain("bundle loaded");
expect(stdout).toContain("[DEBUGTAG2]");
expect(stdout).toContain("multi line");
});
test("--role filters out non-matching roles", async () => {
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const proc = spawn(
process.execPath,
[cliEntryPath, "live", "01LIVECMPLT01DDDDDDDDDDDDG", "--role", "planner"],
{
env,
stdio: ["ignore", "pipe", "pipe"],
},
);
const stdout = await new Promise<string>((resolve, reject) => {
let buf = "";
proc.stdout?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.stderr?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.on("error", reject);
proc.on("exit", (code: number | null) => {
if (code === 0) {
resolve(buf);
} else {
reject(new Error(`exit ${code}: ${buf}`));
}
});
});
expect(stdout).toContain("planner");
expect(stdout).not.toContain("patch");
expect(stdout).toContain("completed: returnCode=0");
});
test("--latest --debug --role combine", async () => {
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const proc = spawn(
process.execPath,
[cliEntryPath, "live", "--latest", "--debug", "--role", "planner"],
{
env,
stdio: ["ignore", "pipe", "pipe"],
},
);
const stdout = await new Promise<string>((resolve, reject) => {
let buf = "";
proc.stdout?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.stderr?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.on("error", reject);
proc.on("exit", (code: number | null) => {
if (code === 0) {
resolve(buf);
} else {
reject(new Error(`exit ${code}: ${buf}`));
}
});
});
expect(stdout).toContain("[DEBUGTAG1]");
expect(stdout).toContain("planner");
expect(stdout).not.toContain("patch");
expect(stdout).toContain("fixture completed");
});
test("unknown thread id exits 1", () => {
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const r = spawnSync(process.execPath, [cliEntryPath, "live", "01UNKNOWNXXXXXXXXXXXXXXXXX"], {
env,
encoding: "utf8",
});
expect(r.status).toBe(1);
expect(String(r.stderr ?? "")).toContain("thread not found");
});
test("follows file until WorkflowResult is appended", async () => {
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const dataPath = join(
storageRoot,
"logs",
"C9NMV6V2TQT81",
"01LIVEINFLY01DDDDDDDDDDDDG.data.jsonl",
);
const proc = spawn(process.execPath, [cliEntryPath, "live", "01LIVEINFLY01DDDDDDDDDDDDG"], {
env,
stdio: ["ignore", "pipe", "pipe"],
});
await new Promise((r) => setTimeout(r, 120));
const prior = await readFile(dataPath, "utf8");
await writeFile(
dataPath,
`${prior.replace(/\s*$/, "")}\n${JSON.stringify({ returnCode: 0, summary: "caught up" })}\n`,
"utf8",
);
const stdout = await new Promise<string>((resolve, reject) => {
let buf = "";
proc.stdout?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.stderr?.on("data", (c: Buffer) => {
buf += c.toString("utf8");
});
proc.on("error", reject);
proc.on("exit", (code: number | null) => {
if (code === 0) {
resolve(buf);
} else {
reject(new Error(`exit ${code}: ${buf}`));
}
});
});
expect(stdout).toContain("planner");
expect(stdout).toContain("completed: returnCode=0");
expect(stdout).toContain("caught up");
});
});
describe("live --latest with empty storage", () => {
let prevEnv: string | undefined;
let emptyRoot: string;
beforeEach(async () => {
prevEnv = process.env.UNCAGED_WORKFLOW_STORAGE_ROOT;
emptyRoot = await mkdtemp(join(tmpdir(), "uncaged-wf-live-empty-"));
process.env.UNCAGED_WORKFLOW_STORAGE_ROOT = emptyRoot;
});
afterEach(async () => {
if (prevEnv === undefined) {
delete process.env.UNCAGED_WORKFLOW_STORAGE_ROOT;
} else {
process.env.UNCAGED_WORKFLOW_STORAGE_ROOT = prevEnv;
}
await rm(emptyRoot, { recursive: true, force: true });
});
test("exits 1 when no threads exist", () => {
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: emptyRoot };
const r = spawnSync(process.execPath, [cliEntryPath, "live", "--latest"], {
env,
encoding: "utf8",
});
expect(r.status).toBe(1);
expect(String(r.stderr ?? "")).toContain("no threads");
});
});
@@ -17,6 +17,9 @@ import { cmdThreads } from "../src/cmd-threads.js";
import { pathExists, readTextFileIfExists } from "../src/fs-utils.js";
import { addCliArgs } from "./bundle-fixture.js";
const wfPutImport = `import { putContentMerkleNode } from "@uncaged/workflow";
`;
const threadFixtureDescriptor = `export const descriptor = {
description: "thread-cli",
roles: {
@@ -31,18 +34,26 @@ const threadFixtureDescriptor = `export const descriptor = {
`;
const fastBundleSource = `${threadFixtureDescriptor}
export const run = async function* (input) {
yield { role: "planner", content: "plan", meta: { plan: input.prompt } };
yield { role: "coder", content: "code", meta: { diff: "y" } };
${wfPutImport}
export const run = async function* (input, options) {
const cas = options.cas;
let h = await putContentMerkleNode(cas, "plan");
yield { role: "planner", contentHash: h, meta: { plan: input.prompt }, refs: [h] };
h = await putContentMerkleNode(cas, "code");
yield { role: "coder", contentHash: h, meta: { diff: "y" }, refs: [h] };
return { returnCode: 0, summary: "done" };
};
`;
const slowPlannerBundleSource = `${threadFixtureDescriptor}
export const run = async function* (input) {
${wfPutImport}
export const run = async function* (input, options) {
await new Promise((r) => setTimeout(r, 400));
yield { role: "planner", content: "plan", meta: { plan: input.prompt } };
yield { role: "coder", content: "code", meta: { diff: "y" } };
const cas = options.cas;
let h = await putContentMerkleNode(cas, "plan");
yield { role: "planner", contentHash: h, meta: { plan: input.prompt }, refs: [h] };
h = await putContentMerkleNode(cas, "code");
yield { role: "coder", contentHash: h, meta: { diff: "y" }, refs: [h] };
return { returnCode: 0, summary: "done" };
};
`;
@@ -50,27 +61,38 @@ export const run = async function* (input) {
const cliEntryPath = fileURLToPath(new URL("../src/cli.ts", import.meta.url));
const abortablePlannerBundleSource = `${threadFixtureDescriptor}
export const run = async function* (input) {
${wfPutImport}
export const run = async function* (input, options) {
await new Promise((r) => setTimeout(r, 600));
yield { role: "planner", content: "plan", meta: { plan: input.prompt } };
yield { role: "coder", content: "code", meta: { diff: "y" } };
const cas = options.cas;
let h = await putContentMerkleNode(cas, "plan");
yield { role: "planner", contentHash: h, meta: { plan: input.prompt }, refs: [h] };
h = await putContentMerkleNode(cas, "code");
yield { role: "coder", contentHash: h, meta: { diff: "y" }, refs: [h] };
return { returnCode: 0, summary: "done" };
};
`;
const pauseResumeBundleSource = `${threadFixtureDescriptor}
export const run = async function* (input) {
yield { role: "first", content: "f", meta: {} };
${wfPutImport}
export const run = async function* (_input, options) {
const cas = options.cas;
let h = await putContentMerkleNode(cas, "f");
yield { role: "first", contentHash: h, meta: {}, refs: [h] };
await new Promise((r) => setTimeout(r, 1500));
yield { role: "second", content: "s", meta: {} };
h = await putContentMerkleNode(cas, "s");
yield { role: "second", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "done" };
};
`;
const delayedFirstYieldBundleSource = `${threadFixtureDescriptor}
export const run = async function* (input) {
${wfPutImport}
export const run = async function* (_input, options) {
await new Promise((r) => setTimeout(r, 900));
yield { role: "only", content: "x", meta: {} };
const cas = options.cas;
const h = await putContentMerkleNode(cas, "x");
yield { role: "only", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "done" };
};
`;
@@ -301,7 +323,7 @@ describe("cli thread commands", () => {
.trim()
.split("\n")
.filter((l) => l !== "");
expect(lines.length).toBe(2);
expect(lines.length).toBe(3);
const runningPath = join(dirname(dataPath), `${threadId}.running`);
expect(await pathExists(runningPath)).toBe(false);
@@ -340,8 +362,8 @@ describe("cli thread commands", () => {
const resumed = await cmdResume(storageRoot, threadId);
expect(resumed.ok).toBe(true);
await waitUntilMinDataLines(dataPath, 3, 120);
expect(await countDataJsonlLines(dataPath)).toBe(3);
await waitUntilMinDataLines(dataPath, 4, 120);
expect(await countDataJsonlLines(dataPath)).toBe(4);
const runningPath = join(dirname(dataPath), `${threadId}.running`);
await waitUntilRunningFileAbsent(runningPath, 100);
+75 -25
View File
@@ -4,8 +4,10 @@ import { cmdCasGet, cmdCasList, cmdCasPut, cmdCasRm } from "./cmd-cas.js";
import { cmdFork, parseForkArgv } from "./cmd-fork.js";
import { cmdGc } from "./cmd-gc.js";
import { cmdHistory } from "./cmd-history.js";
import { cmdInitTemplate, cmdInitWorkspace } from "./cmd-init.js";
import { cmdKill } from "./cmd-kill.js";
import { cmdList, formatListLines } from "./cmd-list.js";
import { cmdLive } from "./cmd-live.js";
import { cmdPause } from "./cmd-pause.js";
import { cmdPs } from "./cmd-ps.js";
import { cmdRemove } from "./cmd-remove.js";
@@ -15,9 +17,10 @@ import { cmdRun } from "./cmd-run.js";
import { cmdShow, formatShowYaml } from "./cmd-show.js";
import { cmdThreadRemove, cmdThreadShow } from "./cmd-thread.js";
import { cmdThreads } from "./cmd-threads.js";
import { parseLiveArgv } from "./live-argv.js";
import { parseRunArgv } from "./run-argv.js";
function usage(): string {
export function formatCliUsage(): string {
return [
"Usage:",
" uncaged-workflow add <name> <file.esm.js> [--types <path>]",
@@ -27,6 +30,8 @@ function usage(): string {
" uncaged-workflow run <name> [--prompt <text>] [--max-rounds N]",
" uncaged-workflow ps",
" uncaged-workflow kill <thread-id>",
" uncaged-workflow live <thread-id> [--debug] [--role <name>]",
" uncaged-workflow live --latest [--debug] [--role <name>]",
" uncaged-workflow history <name>",
" uncaged-workflow rollback <name> [hash]",
" uncaged-workflow pause <thread-id>",
@@ -40,13 +45,47 @@ function usage(): string {
" uncaged-workflow cas put <thread-id> <content>",
" uncaged-workflow cas list <thread-id>",
" uncaged-workflow cas rm <thread-id> <hash>",
" uncaged-workflow init workspace <name>",
" uncaged-workflow init template <name>",
].join("\n");
}
async function dispatchInit(_storageRoot: string, argv: string[]): Promise<number> {
const sub = argv[0];
const name = argv[1];
if (sub === undefined || name === undefined || argv.length > 2) {
printCliError(`${formatCliUsage()}\n\nerror: init requires workspace|template <name>`);
return 1;
}
if (sub === "workspace") {
const result = await cmdInitWorkspace(process.cwd(), name);
if (!result.ok) {
printCliError(result.error);
return 1;
}
printCliLine(`initialized workflow workspace at ${result.value.rootPath}`);
return 0;
}
if (sub === "template") {
const result = await cmdInitTemplate(process.cwd(), name);
if (!result.ok) {
printCliError(result.error);
return 1;
}
printCliLine(`initialized template at ${result.value.templatePath}`);
return 0;
}
printCliError(`${formatCliUsage()}\n\nerror: unknown init subcommand: ${sub}`);
return 1;
}
async function dispatchAdd(storageRoot: string, argv: string[]): Promise<number> {
const parsed = parseAddArgv(argv);
if (!parsed.ok) {
printCliError(`${usage()}\n\nerror: ${parsed.error}`);
printCliError(`${formatCliUsage()}\n\nerror: ${parsed.error}`);
return 1;
}
const result = await cmdAdd(storageRoot, parsed.value);
@@ -63,7 +102,7 @@ async function dispatchAdd(storageRoot: string, argv: string[]): Promise<number>
async function dispatchList(storageRoot: string, argv: string[]): Promise<number> {
if (argv.length > 0) {
printCliError(`${usage()}\n\nerror: list takes no arguments`);
printCliError(`${formatCliUsage()}\n\nerror: list takes no arguments`);
return 1;
}
const result = await cmdList(storageRoot);
@@ -80,7 +119,7 @@ async function dispatchList(storageRoot: string, argv: string[]): Promise<number
async function dispatchShow(storageRoot: string, argv: string[]): Promise<number> {
const name = argv[0];
if (name === undefined || argv.length > 1) {
printCliError(`${usage()}\n\nerror: show requires <name>`);
printCliError(`${formatCliUsage()}\n\nerror: show requires <name>`);
return 1;
}
const result = await cmdShow(storageRoot, name);
@@ -95,7 +134,7 @@ async function dispatchShow(storageRoot: string, argv: string[]): Promise<number
async function dispatchRemove(storageRoot: string, argv: string[]): Promise<number> {
const name = argv[0];
if (name === undefined || argv.length > 1) {
printCliError(`${usage()}\n\nerror: remove requires <name>`);
printCliError(`${formatCliUsage()}\n\nerror: remove requires <name>`);
return 1;
}
const result = await cmdRemove(storageRoot, name);
@@ -110,7 +149,7 @@ async function dispatchRemove(storageRoot: string, argv: string[]): Promise<numb
async function dispatchRun(storageRoot: string, argv: string[]): Promise<number> {
const parsed = parseRunArgv(argv);
if (!parsed.ok) {
printCliError(`${usage()}\n\nerror: ${parsed.error}`);
printCliError(`${formatCliUsage()}\n\nerror: ${parsed.error}`);
return 1;
}
@@ -131,7 +170,7 @@ async function dispatchRun(storageRoot: string, argv: string[]): Promise<number>
async function dispatchPs(storageRoot: string, argv: string[]): Promise<number> {
if (argv.length > 0) {
printCliError(`${usage()}\n\nerror: ps takes no arguments`);
printCliError(`${formatCliUsage()}\n\nerror: ps takes no arguments`);
return 1;
}
for (const line of await cmdPs(storageRoot)) {
@@ -143,7 +182,7 @@ async function dispatchPs(storageRoot: string, argv: string[]): Promise<number>
async function dispatchKill(storageRoot: string, argv: string[]): Promise<number> {
const threadId = argv[0];
if (threadId === undefined || argv.length > 1) {
printCliError(`${usage()}\n\nerror: kill requires <thread-id>`);
printCliError(`${formatCliUsage()}\n\nerror: kill requires <thread-id>`);
return 1;
}
const result = await cmdKill(storageRoot, threadId);
@@ -155,10 +194,19 @@ async function dispatchKill(storageRoot: string, argv: string[]): Promise<number
return 0;
}
async function dispatchLive(storageRoot: string, argv: string[]): Promise<number> {
const parsed = parseLiveArgv(argv);
if (!parsed.ok) {
printCliError(`${formatCliUsage()}\n\nerror: ${parsed.error}`);
return 1;
}
return cmdLive(storageRoot, parsed.value);
}
async function dispatchHistory(storageRoot: string, argv: string[]): Promise<number> {
const name = argv[0];
if (name === undefined || argv.length > 1) {
printCliError(`${usage()}\n\nerror: history requires <name>`);
printCliError(`${formatCliUsage()}\n\nerror: history requires <name>`);
return 1;
}
const result = await cmdHistory(storageRoot, name);
@@ -175,7 +223,7 @@ async function dispatchHistory(storageRoot: string, argv: string[]): Promise<num
async function dispatchRollback(storageRoot: string, argv: string[]): Promise<number> {
const name = argv[0];
if (name === undefined || argv.length > 2) {
printCliError(`${usage()}\n\nerror: rollback requires <name> [hash]`);
printCliError(`${formatCliUsage()}\n\nerror: rollback requires <name> [hash]`);
return 1;
}
const hashArg = argv[1];
@@ -191,7 +239,7 @@ async function dispatchRollback(storageRoot: string, argv: string[]): Promise<nu
async function dispatchPause(storageRoot: string, argv: string[]): Promise<number> {
const threadId = argv[0];
if (threadId === undefined || argv.length > 1) {
printCliError(`${usage()}\n\nerror: pause requires <thread-id>`);
printCliError(`${formatCliUsage()}\n\nerror: pause requires <thread-id>`);
return 1;
}
const result = await cmdPause(storageRoot, threadId);
@@ -206,7 +254,7 @@ async function dispatchPause(storageRoot: string, argv: string[]): Promise<numbe
async function dispatchResume(storageRoot: string, argv: string[]): Promise<number> {
const threadId = argv[0];
if (threadId === undefined || argv.length > 1) {
printCliError(`${usage()}\n\nerror: resume requires <thread-id>`);
printCliError(`${formatCliUsage()}\n\nerror: resume requires <thread-id>`);
return 1;
}
const result = await cmdResume(storageRoot, threadId);
@@ -233,7 +281,7 @@ async function dispatchThreads(storageRoot: string, argv: string[]): Promise<num
async function dispatchThread(storageRoot: string, argv: string[]): Promise<number> {
const id = argv[0];
if (id === undefined || argv.length > 1) {
printCliError(`${usage()}\n\nerror: thread requires <id>`);
printCliError(`${formatCliUsage()}\n\nerror: thread requires <id>`);
return 1;
}
const result = await cmdThreadShow(storageRoot, id);
@@ -248,7 +296,7 @@ async function dispatchThread(storageRoot: string, argv: string[]): Promise<numb
async function dispatchThreadRm(storageRoot: string, argv: string[]): Promise<number> {
const id = argv[0];
if (id === undefined || argv.length > 1) {
printCliError(`${usage()}\n\nerror: thread rm requires <id>`);
printCliError(`${formatCliUsage()}\n\nerror: thread rm requires <id>`);
return 1;
}
const result = await cmdThreadRemove(storageRoot, id);
@@ -270,7 +318,7 @@ async function dispatchThreadBranch(storageRoot: string, rest: string[]): Promis
async function dispatchGc(storageRoot: string, argv: string[]): Promise<number> {
if (argv.length > 0) {
printCliError(`${usage()}\n\nerror: gc takes no arguments`);
printCliError(`${formatCliUsage()}\n\nerror: gc takes no arguments`);
return 1;
}
const result = await cmdGc(storageRoot);
@@ -288,7 +336,7 @@ async function dispatchGc(storageRoot: string, argv: string[]): Promise<number>
async function dispatchFork(storageRoot: string, argv: string[]): Promise<number> {
const parsed = parseForkArgv(argv);
if (!parsed.ok) {
printCliError(`${usage()}\n\nerror: ${parsed.error}`);
printCliError(`${formatCliUsage()}\n\nerror: ${parsed.error}`);
return 1;
}
const result = await cmdFork(storageRoot, parsed.value.threadId, parsed.value.fromRole);
@@ -304,7 +352,7 @@ async function dispatchCasGet(storageRoot: string, rest: string[]): Promise<numb
const threadId = rest[0];
const hash = rest[1];
if (threadId === undefined || hash === undefined || rest.length > 2) {
printCliError(`${usage()}\n\nerror: cas get requires <thread-id> <hash>`);
printCliError(`${formatCliUsage()}\n\nerror: cas get requires <thread-id> <hash>`);
return 1;
}
const result = await cmdCasGet(storageRoot, threadId, hash);
@@ -320,7 +368,7 @@ async function dispatchCasPut(storageRoot: string, rest: string[]): Promise<numb
const threadId = rest[0];
const content = rest[1];
if (threadId === undefined || content === undefined || rest.length > 2) {
printCliError(`${usage()}\n\nerror: cas put requires <thread-id> <content>`);
printCliError(`${formatCliUsage()}\n\nerror: cas put requires <thread-id> <content>`);
return 1;
}
const result = await cmdCasPut(storageRoot, threadId, content);
@@ -335,7 +383,7 @@ async function dispatchCasPut(storageRoot: string, rest: string[]): Promise<numb
async function dispatchCasList(storageRoot: string, rest: string[]): Promise<number> {
const threadId = rest[0];
if (threadId === undefined || rest.length > 1) {
printCliError(`${usage()}\n\nerror: cas list requires <thread-id>`);
printCliError(`${formatCliUsage()}\n\nerror: cas list requires <thread-id>`);
return 1;
}
const result = await cmdCasList(storageRoot, threadId);
@@ -353,7 +401,7 @@ async function dispatchCasRm(storageRoot: string, rest: string[]): Promise<numbe
const threadId = rest[0];
const hash = rest[1];
if (threadId === undefined || hash === undefined || rest.length > 2) {
printCliError(`${usage()}\n\nerror: cas rm requires <thread-id> <hash>`);
printCliError(`${formatCliUsage()}\n\nerror: cas rm requires <thread-id> <hash>`);
return 1;
}
const result = await cmdCasRm(storageRoot, threadId, hash);
@@ -378,12 +426,12 @@ const CAS_SUBCOMMAND_TABLE: Record<
async function dispatchCas(storageRoot: string, argv: string[]): Promise<number> {
const sub = argv[0];
if (sub === undefined) {
printCliError(`${usage()}\n\nerror: unknown cas subcommand: (none)`);
printCliError(`${formatCliUsage()}\n\nerror: unknown cas subcommand: (none)`);
return 1;
}
const handler = CAS_SUBCOMMAND_TABLE[sub];
if (handler === undefined) {
printCliError(`${usage()}\n\nerror: unknown cas subcommand: ${sub}`);
printCliError(`${formatCliUsage()}\n\nerror: unknown cas subcommand: ${sub}`);
return 1;
}
return handler(storageRoot, argv.slice(1));
@@ -393,12 +441,14 @@ type DispatchFn = (storageRoot: string, argv: string[]) => Promise<number>;
const COMMAND_TABLE: Record<string, DispatchFn> = {
add: dispatchAdd,
init: dispatchInit,
list: dispatchList,
show: dispatchShow,
remove: dispatchRemove,
run: dispatchRun,
ps: dispatchPs,
kill: dispatchKill,
live: dispatchLive,
history: dispatchHistory,
rollback: dispatchRollback,
pause: dispatchPause,
@@ -412,18 +462,18 @@ const COMMAND_TABLE: Record<string, DispatchFn> = {
export async function runCli(storageRoot: string, argv: string[]): Promise<number> {
if (argv.length === 0) {
printCliError(usage());
printCliError(formatCliUsage());
return 1;
}
const command = argv[0];
if (command === undefined) {
printCliError(usage());
printCliError(formatCliUsage());
return 1;
}
const rest = argv.slice(1);
const dispatch = COMMAND_TABLE[command];
if (dispatch === undefined) {
printCliError(`${usage()}\n\nerror: unknown command ${command}`);
printCliError(`${formatCliUsage()}\n\nerror: unknown command ${command}`);
return 1;
}
return dispatch(storageRoot, rest);
Regular → Executable
View File
+1 -1
View File
@@ -192,7 +192,7 @@ export async function cmdAdd(
return validated;
}
const extracted = await extractBundleExports(resolvedPath);
const extracted = await extractBundleExports(resolvedPath, { storageRoot });
if (!extracted.ok) {
return extracted;
}
+2 -1
View File
@@ -65,8 +65,9 @@ export async function cmdFork(
const newThreadId = generateUlid(Date.now());
const stepsOnWire = plan.value.historicalSteps.map((s) => ({
role: s.role,
content: s.content,
contentHash: s.contentHash,
meta: s.meta,
refs: s.refs,
timestamp: s.timestamp,
}));
+415
View File
@@ -0,0 +1,415 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { dirname, join, resolve } from "node:path";
import { err, ok, type Result } from "@uncaged/workflow";
import { pathExists } from "./fs-utils.js";
export type CmdInitWorkspaceSuccess = {
rootPath: string;
};
export type CmdInitTemplateSuccess = {
templatePath: string;
};
function validateWorkspaceSegment(name: string): Result<void, string> {
if (name.length === 0) {
return err("workspace name must not be empty");
}
if (name === "." || name === "..") {
return err("invalid workspace name");
}
if (name.includes("/") || name.includes("\\")) {
return err("workspace name must not contain path separators");
}
return ok(undefined);
}
function rootPackageJson(workspaceName: string): string {
return `${JSON.stringify(
{
name: workspaceName,
private: true,
type: "module",
workspaces: ["templates/*", "workflows"],
},
null,
2,
)}\n`;
}
function workflowsPackageJson(): string {
return `${JSON.stringify(
{
name: "workflows",
version: "0.0.0",
private: true,
type: "module",
dependencies: {
"@uncaged/workflow": "^0.1.0",
zod: "^4.0.0",
},
},
null,
2,
)}\n`;
}
function biomeJson(): string {
return `${JSON.stringify(
{
$schema: "https://biomejs.dev/schemas/2.4.14/schema.json",
files: {
includes: ["**", "!**/node_modules", "!**/dist"],
},
formatter: {
indentWidth: 2,
},
linter: {
enabled: true,
rules: {
recommended: true,
},
},
},
null,
2,
)}\n`;
}
function tsconfigJson(): string {
return `${JSON.stringify(
{
compilerOptions: {
strict: true,
target: "ESNext",
module: "ESNext",
moduleResolution: "Bundler",
skipLibCheck: true,
},
},
null,
2,
)}\n`;
}
function agentsMd(): string {
return `# AGENTS — Workflow 工作区开发指南
面向在本仓库中编写 workflow 的 coding agent。引擎层术语与架构细节与 **@uncaged/workflow** 上游文档一致,编写时可对照 \`CLAUDE.md\`\`docs/architecture.md\`
## 1. 项目结构(workspace / template / workflow instance)
| 层级 | 目录 / 产物 | 职责 |
|------|----------------|------|
| **Workspace** | 仓库根(\`package.json\`\`workspaces: ["templates/*", "workflows"]\`) | Bun monorepo:统一管理本地模板包与 workflow 实例 |
| **Template** | \`templates/<name>/\`(如 \`src/roles.ts\`\`src/moderator.ts\`\`src/index.ts\`) | 纯数据:**WorkflowDefinition**(各 **RoleDefinition** + **Moderator**),**不绑定**具体 Agent |
| **Workflow instance** | \`workflows/\`(或单独包) | 把模板与运行时 **AgentFn** / **ExtractFn** 组合,产出可注册的 **单文件 ESM bundle**(\`run\` + \`descriptor\` 命名导出) |
Init 生成的骨架:\`templates/\` 下放可复用定义,\`workflows/\` 下放绑定与打包入口。
## 2. 核心概念
- **RoleMeta**:\`Record<string, Record<string, unknown>>\`,角色名 → 该角色结构化 meta 的形状约定。
- **RoleDefinition<Meta>**:纯数据——\`description\`\`systemPrompt\`\`extractPrompt\`\`schema\`(Zod v4)。不含执行逻辑。
- **WorkflowDefinition<M extends RoleMeta>**:\`description\` + \`roles\`(各角色定义)+ **Moderator**。
- **Moderator**:\`(ctx: ModeratorContext<M>) => (角色名) | END\`。同步、纯函数,只做路由。
- **AgentFn**:\`(ctx: AgentContext) => Promise<string>\`,原始文本输出;从上下文读取当前角色的 \`systemPrompt\`
- **ExtractFn**:从上下文与 prompt 解析结构化数据(引擎与 Agent 都可使用)。
引擎循环简述:**Moderator** → 选角色 → **Agent** 产出文本 → **Extract** 写入 **meta** → 追加 step,重复直至 **END**。详见 \`docs/architecture.md\` 中的三阶段说明。
## 3. 开发流程
1. **定义 RoleMeta**:为每个角色约定 meta 的 TypeScript 类型(与 Zod schema 对齐)。
2. **编写 RoleDefinition**:为每个角色写 Zod \`schema\`,补齐 \`systemPrompt\` / \`extractPrompt\` / \`description\`
3. **编写 Moderator**:根据 \`ctx.steps\` 与业务状态返回下一个角色名或 \`END\`
4. **组装 WorkflowDefinition**:在模板 \`index\` 中导出 definition(以及必要的角色 / moderator 导出)。
5. **实例化**:在 workflow 包中使用 \`createWorkflow(def, binding, extract)\`(或项目约定的封装)绑定 **AgentFn** / **ExtractFn**。
6. **构建**:打包为单个 **.esm.js** bundle,使用 **uncaged-workflow add** 注册。
## 4. 编码规范
与 **CLAUDE.md** 对齐,摘要如下:
- **Functional-first**:优先 \`function\` + \`type\`,避免面向对象业务模型。
- **type 而非 interface**:类型别名一律用 \`type\`,不要使用 \`interface\`
- **显式可空**:不要用 \`?:\`;可空字段写成 \`T | null\`
- **function 而非 class**:不用 class(第三方库要求或 \`Error\` 子类除外)。
- **Crockford Base32**:日志 tag、bundle hash、thread id 等标识约定(引擎侧);工作区内自定义日志若沿用引擎 logger,tag 为 8 字符 Crockford Base32,且每个调用点唯一。
- **Named exports only**:不要使用 **default export**;workflow bundle 须 **export const run** 与 **export const descriptor**。
- **No console.log**:库代码用结构化 logger;CLI 用户输出可按项目 Biome 规则例外标注。
- **No dynamic import**:业务与 bundle 内禁止 \`import()\`;例外仅限「运行时路径由用户提供的 bundle 加载器」(引擎内部)。
## 5. Template 复用
- **已发布模板**:可通过 npm 依赖 \`@uncaged/workflow-template-*\` 等包,在 workflow 实例中 import 其 **WorkflowDefinition** 再绑定 Agent。
- **本地模板**:放在本仓库 \`templates/<name>/\`,由 workspace 协议引用(如 \`"template-foo": "workspace:*"\` 或相对路径),便于同源修改与版本控制。
选择模板时保持 **definition 与 agent 绑定分离**:模板只描述「做什么、顺序如何」,实例决定「谁执行、如何抽取 meta」。
## 6. Build and Test
日常命令:
\`\`\`sh
bun install
bun run check # Biome:lint + format
bun test
bun build # 若包内配置了 build 脚本则用于产出 dist / bundle
uncaged-workflow add <name> <path/to/bundle.esm.js>
\`\`\`
提交前至少运行 **bun run check** 与 **bun test**;registry 与本地运行流参见 README 与 CLI 文档。
## 7. 常见陷阱
- **No dynamic import**:bundle 须静态可分析;动态 \`import()\` 会破坏哈希与加载约束。
- **No default export**:引擎只接受命名导出 \`run\` / \`descriptor\`
- **No console.log**:避免在可被 Biome \`noConsole\` 规则覆盖的代码路径直接使用 console。
- **Single-file ESM bundle**:交付物是单一 \`.esm.js\`;静态 import 仅限 Node 内置(见 architecture 文档中的 Bundle Contract)。
---
编写新 workflow 时,先对齐 **RoleMeta → RoleDefinition(Zod)→ Moderator → 绑定 → 单文件 bundle**,再对照本节规范自检。
`;
}
function readmeMd(workspaceName: string): string {
return `# ${workspaceName}
Local workflow development workspace (Bun monorepo).
## Layout
- \`templates/\` — reusable workflow definition packages (roles + moderator), no agent binding
- \`workflows/\` — workflow instances that bind templates to agents and export \`run\` + \`descriptor\`
## Commands
\`\`\`sh
bun install
bun run check # after you add scripts / Biome
uncaged-workflow add <name> <bundle.esm.js>
uncaged-workflow run <name>
\`\`\`
Create this skeleton with:
\`\`\`sh
uncaged-workflow init workspace ${workspaceName}
\`\`\`
`;
}
export async function cmdInitWorkspace(
parentDir: string,
workspaceName: string,
): Promise<Result<CmdInitWorkspaceSuccess, string>> {
const validated = validateWorkspaceSegment(workspaceName);
if (!validated.ok) {
return validated;
}
const rootPath = join(parentDir, workspaceName);
if (await pathExists(rootPath)) {
return err(`directory already exists: ${rootPath}`);
}
await mkdir(rootPath, { recursive: false });
await mkdir(join(rootPath, "templates"), { recursive: false });
await mkdir(join(rootPath, "workflows"), { recursive: false });
await Promise.all([
writeFile(join(rootPath, "package.json"), rootPackageJson(workspaceName), "utf8"),
writeFile(join(rootPath, "biome.json"), biomeJson(), "utf8"),
writeFile(join(rootPath, "tsconfig.json"), tsconfigJson(), "utf8"),
writeFile(join(rootPath, "AGENTS.md"), agentsMd(), "utf8"),
writeFile(join(rootPath, "README.md"), readmeMd(workspaceName), "utf8"),
writeFile(join(rootPath, "templates", ".gitkeep"), "", "utf8"),
writeFile(join(rootPath, "workflows", "package.json"), workflowsPackageJson(), "utf8"),
]);
return ok({ rootPath });
}
function hasTemplatesWorkspaceGlob(workspaces: unknown): boolean {
return Array.isArray(workspaces) && workspaces.includes("templates/*");
}
async function readPackageJsonWorkspaces(dir: string): Promise<unknown | null> {
const pkgPath = join(dir, "package.json");
if (!(await pathExists(pkgPath))) {
return null;
}
let raw: string;
try {
raw = await readFile(pkgPath, "utf8");
} catch {
return null;
}
let parsed: unknown;
try {
parsed = JSON.parse(raw) as unknown;
} catch {
return null;
}
if (typeof parsed !== "object" || parsed === null || !("workspaces" in parsed)) {
return null;
}
return (parsed as { workspaces: unknown }).workspaces;
}
/** Resolve uncaged-workflow workspace root (package.json with `templates/*` in `workspaces`). */
async function findWorkflowWorkspaceRoot(startDir: string): Promise<Result<string, string>> {
let dir = resolve(startDir);
for (;;) {
const workspaces = await readPackageJsonWorkspaces(dir);
if (workspaces !== null && hasTemplatesWorkspaceGlob(workspaces)) {
return ok(dir);
}
const parent = dirname(dir);
if (parent === dir) {
return err(
'not inside a workflow workspace (no package.json with workspaces containing "templates/*")',
);
}
dir = parent;
}
}
function templatePackageJson(templateName: string): string {
return `${JSON.stringify(
{
name: `template-${templateName}`,
version: "0.0.0",
private: true,
type: "module",
dependencies: {
"@uncaged/workflow": "^0.1.0",
zod: "^4.0.0",
},
},
null,
2,
)}\n`;
}
function templateTsconfigJson(): string {
return `${JSON.stringify(
{
extends: "../../tsconfig.json",
compilerOptions: {
rootDir: "src",
outDir: "dist",
},
include: ["src/**/*.ts"],
},
null,
2,
)}\n`;
}
function templateRolesTs(): string {
return `import type { RoleDefinition } from "@uncaged/workflow";
import * as z from "zod/v4";
export const HELLO_TEMPLATE_DESCRIPTION =
"Minimal starter template: one greeter role, then END.";
export type HelloTemplateMeta = {
greeter: {
message: string;
};
};
const greeterMetaSchema = z.object({
message: z.string(),
});
export const greeterRole: RoleDefinition<HelloTemplateMeta["greeter"]> = {
description: "Says hello — replace with your first role.",
systemPrompt: "You are a helpful assistant. Reply with one short friendly sentence.",
extractPrompt: "Extract the assistant's greeting as message.",
schema: greeterMetaSchema,
extractRefs: null,
};
`;
}
function templateModeratorTs(): string {
return `import { END, type Moderator, type ModeratorContext } from "@uncaged/workflow";
import type { HelloTemplateMeta } from "./roles.js";
export const helloTemplateModerator: Moderator<HelloTemplateMeta> = (
ctx: ModeratorContext<HelloTemplateMeta>,
) => {
if (ctx.steps.length === 0) {
return "greeter";
}
return END;
};
`;
}
function templateIndexTs(): string {
return `import type { WorkflowDefinition } from "@uncaged/workflow";
import { helloTemplateModerator } from "./moderator.js";
import {
HELLO_TEMPLATE_DESCRIPTION,
type HelloTemplateMeta,
greeterRole,
} from "./roles.js";
export {
HELLO_TEMPLATE_DESCRIPTION,
type HelloTemplateMeta,
greeterRole,
} from "./roles.js";
export { helloTemplateModerator } from "./moderator.js";
export const helloTemplateWorkflowDefinition: WorkflowDefinition<HelloTemplateMeta> = {
description: HELLO_TEMPLATE_DESCRIPTION,
roles: {
greeter: greeterRole,
},
moderator: helloTemplateModerator,
};
`;
}
export async function cmdInitTemplate(
startDir: string,
templateName: string,
): Promise<Result<CmdInitTemplateSuccess, string>> {
const validated = validateWorkspaceSegment(templateName);
if (!validated.ok) {
return validated;
}
const rootResult = await findWorkflowWorkspaceRoot(startDir);
if (!rootResult.ok) {
return rootResult;
}
const workspaceRoot = rootResult.value;
const templateDir = join(workspaceRoot, "templates", templateName);
if (await pathExists(templateDir)) {
return err(`template already exists: ${templateDir}`);
}
await mkdir(join(templateDir, "src"), { recursive: true });
await Promise.all([
writeFile(join(templateDir, "package.json"), templatePackageJson(templateName), "utf8"),
writeFile(join(templateDir, "tsconfig.json"), templateTsconfigJson(), "utf8"),
writeFile(join(templateDir, "src", "roles.ts"), templateRolesTs(), "utf8"),
writeFile(join(templateDir, "src", "moderator.ts"), templateModeratorTs(), "utf8"),
writeFile(join(templateDir, "src", "index.ts"), templateIndexTs(), "utf8"),
]);
return ok({ templatePath: templateDir });
}
+463
View File
@@ -0,0 +1,463 @@
import { watch } from "node:fs";
import { readFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import {
type CasStore,
createCasStore,
getContentMerklePayload,
getGlobalCasDir,
tryParseRoleStepRecord,
tryParseWorkflowResultRecord,
type WorkflowCompletion,
} from "@uncaged/workflow";
import { printCliError, printCliLine } from "./cli-output.js";
import { pathExists } from "./fs-utils.js";
import type { ParsedLiveArgv } from "./live-argv.js";
import { findLatestThreadDataPath, resolveThreadDataPath } from "./thread-scan.js";
export const LIVE_CONTENT_MAX_LINES = 10;
export type LiveRoleRow = {
role: string;
content: string;
meta: Record<string, unknown>;
timestamp: number;
};
export function formatLiveTimeLabel(timestampMs: number): string {
const d = new Date(timestampMs);
const hh = String(d.getHours()).padStart(2, "0");
const mm = String(d.getMinutes()).padStart(2, "0");
const ss = String(d.getSeconds()).padStart(2, "0");
return `${hh}:${mm}:${ss}`;
}
function shouldUseColor(): boolean {
return process.stdout.isTTY === true && process.env.NO_COLOR === undefined;
}
function highlightLiveRole(name: string): string {
if (!shouldUseColor()) {
return name;
}
return `\x1b[1m\x1b[36m${name}\x1b[0m`;
}
function dimGreyLine(line: string): string {
if (!shouldUseColor()) {
return line;
}
return `\x1b[2m\x1b[90m${line}\x1b[0m`;
}
export function formatLiveDebugLine(timestampMs: number, tag: string, message: string): string {
const label = `[${formatLiveTimeLabel(timestampMs)}] [${tag}] ${message.replace(/\n/g, " ")}`;
return dimGreyLine(label);
}
export function renderLiveRoleStepLines(row: LiveRoleRow, roleDisplay: string): string[] {
const header = `[${formatLiveTimeLabel(row.timestamp)}] ▶ ${roleDisplay}`;
const lines: string[] = [header];
const parts = row.content.split("\n");
const shown = parts.slice(0, LIVE_CONTENT_MAX_LINES);
for (const ln of shown) {
lines.push(` ${ln}`);
}
const omitted = parts.length - shown.length;
if (omitted > 0) {
lines.push(` … (${omitted} more line${omitted === 1 ? "" : "s"})`);
}
lines.push(` meta: ${JSON.stringify(row.meta)}`);
return lines;
}
function printSummary(result: WorkflowCompletion): void {
printCliLine(`completed: returnCode=${result.returnCode}${result.summary}`);
}
type LiveSessionState = {
sawStart: boolean;
completed: boolean;
carry: string;
contentOffset: number;
};
type InfoLiveState = {
carry: string;
contentOffset: number;
};
function tryParseInfoRecord(obj: Record<string, unknown>): {
tag: string;
content: string;
timestamp: number;
} | null {
const tag = obj.tag;
const content = obj.content;
const timestamp = obj.timestamp;
if (
typeof tag !== "string" ||
typeof content !== "string" ||
typeof timestamp !== "number" ||
!Number.isFinite(timestamp)
) {
return null;
}
return { tag, content, timestamp };
}
async function handleJsonlLine(
rawLine: string,
state: LiveSessionState,
roleFilter: string | null,
cas: CasStore,
): Promise<{ parseError: string | null; workflowResult: WorkflowCompletion | null }> {
const trimmed = rawLine.trim();
if (trimmed === "") {
return { parseError: null, workflowResult: null };
}
let rec: unknown;
try {
rec = JSON.parse(trimmed) as unknown;
} catch {
return { parseError: "invalid JSON in thread data file", workflowResult: null };
}
if (rec === null || typeof rec !== "object") {
return { parseError: "invalid record in thread data file", workflowResult: null };
}
const obj = rec as Record<string, unknown>;
if (!state.sawStart) {
state.sawStart = true;
return { parseError: null, workflowResult: null };
}
const wf = tryParseWorkflowResultRecord(obj);
if (wf !== null) {
state.completed = true;
return { parseError: null, workflowResult: wf };
}
const roleRow = tryParseRoleStepRecord(obj);
if (roleRow === null) {
return {
parseError: "unrecognized record in thread data (expected role step or result)",
workflowResult: null,
};
}
if (roleFilter !== null && roleRow.role !== roleFilter) {
return { parseError: null, workflowResult: null };
}
const payload = await getContentMerklePayload(cas, roleRow.contentHash);
const content =
payload !== null ? payload : `(content not in CAS; contentHash=${roleRow.contentHash})`;
const row: LiveRoleRow = {
role: roleRow.role,
content,
meta: roleRow.meta,
timestamp: roleRow.timestamp,
};
for (const outLine of renderLiveRoleStepLines(row, highlightLiveRole(row.role))) {
printCliLine(outLine);
}
return { parseError: null, workflowResult: null };
}
async function pumpNewContent(
dataPath: string,
state: LiveSessionState,
roleFilter: string | null,
cas: CasStore,
): Promise<number | null> {
let text: string;
try {
text = await readFile(dataPath, "utf8");
} catch {
return null;
}
if (text.length < state.contentOffset) {
state.contentOffset = 0;
state.carry = "";
}
const chunk = text.slice(state.contentOffset);
state.contentOffset = text.length;
state.carry += chunk;
const parts = state.carry.split("\n");
state.carry = parts.pop() ?? "";
for (const line of parts) {
const { parseError, workflowResult } = await handleJsonlLine(line, state, roleFilter, cas);
if (parseError !== null) {
printCliError(parseError);
return 1;
}
if (workflowResult !== null) {
printSummary(workflowResult);
return 0;
}
}
return null;
}
async function pumpNewInfoContent(infoPath: string, state: InfoLiveState): Promise<void> {
let text: string;
try {
text = await readFile(infoPath, "utf8");
} catch {
return;
}
if (text.length < state.contentOffset) {
state.contentOffset = 0;
state.carry = "";
}
const chunk = text.slice(state.contentOffset);
state.contentOffset = text.length;
state.carry += chunk;
const parts = state.carry.split("\n");
state.carry = parts.pop() ?? "";
for (const line of parts) {
const trimmed = line.trim();
if (trimmed === "") {
continue;
}
let rec: unknown;
try {
rec = JSON.parse(trimmed) as unknown;
} catch {
continue;
}
if (rec === null || typeof rec !== "object") {
continue;
}
const parsed = tryParseInfoRecord(rec as Record<string, unknown>);
if (parsed === null) {
continue;
}
printCliLine(formatLiveDebugLine(parsed.timestamp, parsed.tag, parsed.content));
}
}
type WatchPumpTask = {
path: string;
pump: () => Promise<number | null>;
};
async function runWatchPumpStep(
settled: () => boolean,
pump: () => Promise<number | null>,
closeAll: () => void,
finish: (code: number) => void,
): Promise<void> {
if (settled()) {
return;
}
try {
const code = await pump();
if (code !== null) {
closeAll();
finish(code);
}
} catch (e) {
closeAll();
throw e instanceof Error ? e : new Error(String(e));
}
}
function watchLivePaths(params: { tasks: WatchPumpTask[]; signal: AbortSignal }): Promise<number> {
const { tasks, signal } = params;
return new Promise((resolve, reject) => {
let settled = false;
const finish = (code: number): void => {
if (settled) {
return;
}
settled = true;
resolve(code);
};
const pumpChains = new Map<string, Promise<void>>();
for (const t of tasks) {
pumpChains.set(t.path, Promise.resolve());
}
const watchers: ReturnType<typeof watch>[] = [];
const closeAll = (): void => {
for (const w of watchers) {
w.close();
}
};
function schedulePump(path: string, pump: () => Promise<number | null>): void {
const prev = pumpChains.get(path) ?? Promise.resolve();
const next = (async () => {
await prev;
await runWatchPumpStep(() => settled, pump, closeAll, finish);
})();
pumpChains.set(path, next);
}
for (const { path, pump } of tasks) {
const watcher = watch(path, (eventType) => {
if (eventType === "rename") {
return;
}
schedulePump(path, pump);
});
watchers.push(watcher);
watcher.on("error", (err: Error) => {
closeAll();
reject(err);
});
}
const onAbort = (): void => {
closeAll();
finish(0);
};
signal.addEventListener("abort", onAbort, { once: true });
for (const { path, pump } of tasks) {
schedulePump(path, pump);
}
});
}
type LiveThreadTarget = {
threadId: string;
dataPath: string;
};
async function resolveLiveThreadTarget(
storageRoot: string,
parsed: ParsedLiveArgv,
): Promise<LiveThreadTarget | null> {
if (parsed.latest) {
const found = await findLatestThreadDataPath(storageRoot);
if (found === null) {
printCliError("live: no threads found");
return null;
}
return found;
}
const id = parsed.threadId;
if (id === null) {
printCliError("live: internal error: missing thread id");
return null;
}
const resolved = await resolveThreadDataPath(storageRoot, id);
if (resolved === null) {
printCliError(`thread not found: ${id}`);
return null;
}
return { threadId: id, dataPath: resolved };
}
async function buildLiveWatchTasks(params: {
dataPath: string;
infoPath: string;
debug: boolean;
dataState: LiveSessionState;
infoState: InfoLiveState;
roleFilter: string | null;
cas: CasStore;
}): Promise<WatchPumpTask[]> {
const { dataPath, infoPath, debug, dataState, infoState, roleFilter, cas } = params;
const tasks: WatchPumpTask[] = [
{
path: dataPath,
pump: () => pumpNewContent(dataPath, dataState, roleFilter, cas),
},
];
if (debug && (await pathExists(infoPath))) {
tasks.push({
path: infoPath,
pump: async () => {
await pumpNewInfoContent(infoPath, infoState);
return null;
},
});
}
return tasks;
}
export async function cmdLive(storageRoot: string, parsed: ParsedLiveArgv): Promise<number> {
const target = await resolveLiveThreadTarget(storageRoot, parsed);
if (target === null) {
return 1;
}
const { threadId, dataPath } = target;
const roleFilter = parsed.role;
const infoPath = join(dirname(dataPath), `${threadId}.info.jsonl`);
const cas = createCasStore(getGlobalCasDir(storageRoot));
const dataState: LiveSessionState = {
sawStart: false,
completed: false,
carry: "",
contentOffset: 0,
};
const infoState: InfoLiveState = {
carry: "",
contentOffset: 0,
};
const controller = new AbortController();
const onSigInt = (): void => {
controller.abort();
};
process.on("SIGINT", onSigInt);
try {
const firstData = await pumpNewContent(dataPath, dataState, roleFilter, cas);
if (firstData === 1) {
return 1;
}
if (parsed.debug && (await pathExists(infoPath))) {
await pumpNewInfoContent(infoPath, infoState);
}
if (firstData === 0 || dataState.completed) {
return 0;
}
const tasks = await buildLiveWatchTasks({
dataPath,
infoPath,
debug: parsed.debug,
dataState,
infoState,
roleFilter,
cas,
});
return await watchLivePaths({ tasks, signal: controller.signal });
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
printCliError(`live: ${message}`);
return 1;
} finally {
process.off("SIGINT", onSigInt);
}
}
@@ -44,6 +44,7 @@ export async function cmdRollback(
}
const nextRegistry = {
config: reg.value.config,
workflows: { ...reg.value.workflows, [name]: rolled.value },
};
const written = await writeWorkflowRegistry(storageRoot, nextRegistry);
+1 -1
View File
@@ -46,7 +46,7 @@ export async function cmdRun(
threadId,
workflowName: name,
prompt,
options: { maxRounds },
options: { maxRounds, depth: 0 },
},
{ awaitResponseLine: false },
);
+75
View File
@@ -0,0 +1,75 @@
import { err, ok, type Result } from "@uncaged/workflow";
export type ParsedLiveArgv = {
threadId: string | null;
latest: boolean;
debug: boolean;
role: string | null;
};
type LiveArgvScan = {
latest: boolean;
debug: boolean;
role: string | null;
threadId: string | null;
};
function applyLiveArgvToken(argv: string[], i: number, s: LiveArgvScan): Result<number, string> {
const a = argv[i];
if (a === "--latest") {
s.latest = true;
return ok(i + 1);
}
if (a === "--debug") {
s.debug = true;
return ok(i + 1);
}
if (a === "--role") {
const v = argv[i + 1];
if (v === undefined || v.startsWith("--")) {
return err("missing value for --role");
}
s.role = v;
return ok(i + 2);
}
if (a.startsWith("--")) {
return err(`unknown live flag: ${a}`);
}
if (s.threadId !== null) {
return err("unexpected extra argument");
}
s.threadId = a;
return ok(i + 1);
}
export function parseLiveArgv(argv: string[]): Result<ParsedLiveArgv, string> {
const s: LiveArgvScan = {
latest: false,
debug: false,
role: null,
threadId: null,
};
let i = 0;
while (i < argv.length) {
const step = applyLiveArgvToken(argv, i, s);
if (!step.ok) {
return step;
}
i = step.value;
}
if (s.latest && s.threadId !== null) {
return err("live --latest does not take <thread-id>");
}
if (!s.latest && s.threadId === null) {
return err("live requires <thread-id> or --latest");
}
return ok({
threadId: s.threadId,
latest: s.latest,
debug: s.debug,
role: s.role,
});
}
+67 -1
View File
@@ -1,4 +1,4 @@
import { readdir } from "node:fs/promises";
import { readdir, stat } from "node:fs/promises";
import { join } from "node:path";
import { pathExists, readTextFileIfExists } from "./fs-utils.js";
@@ -15,6 +15,28 @@ export type HistoricalThreadRow = {
workflowName: string | null;
};
async function readThreadStartTimestampMs(dataPath: string): Promise<number | null> {
const text = await readTextFileIfExists(dataPath);
if (text === null) {
return null;
}
const firstLine = text.split("\n")[0];
if (firstLine === undefined || firstLine.trim() === "") {
return null;
}
let parsed: unknown;
try {
parsed = JSON.parse(firstLine) as unknown;
} catch {
return null;
}
if (parsed === null || typeof parsed !== "object") {
return null;
}
const ts = (parsed as Record<string, unknown>).timestamp;
return typeof ts === "number" && Number.isFinite(ts) ? ts : null;
}
async function readWorkflowNameFromDataJsonl(dataPath: string): Promise<string | null> {
const text = await readTextFileIfExists(dataPath);
if (text === null) {
@@ -124,6 +146,50 @@ export async function listHistoricalThreads(
return out;
}
/**
* Picks the thread whose `.data.jsonl` is newest by start-record `timestamp`,
* falling back to file `mtime` when the timestamp is missing.
* Tie-breaker: larger `mtime` wins when start timestamps are equal.
*/
export async function findLatestThreadDataPath(
storageRoot: string,
): Promise<{ threadId: string; dataPath: string } | null> {
const threads = await listHistoricalThreads(storageRoot, null);
if (threads.length === 0) {
return null;
}
let best: {
threadId: string;
dataPath: string;
primary: number;
secondary: number;
} | null = null;
for (const t of threads) {
const dataPath = join(storageRoot, "logs", t.hash, `${t.threadId}.data.jsonl`);
let mtimeMs = 0;
try {
const st = await stat(dataPath);
mtimeMs = st.mtimeMs;
} catch {
continue;
}
const startTs = await readThreadStartTimestampMs(dataPath);
const primary = startTs !== null ? startTs : mtimeMs;
const secondary = mtimeMs;
if (
best === null ||
primary > best.primary ||
(primary === best.primary && secondary > best.secondary)
) {
best = { threadId: t.threadId, dataPath, primary, secondary };
}
}
return best === null ? null : { threadId: best.threadId, dataPath: best.dataPath };
}
export async function resolveThreadDataPath(
storageRoot: string,
threadId: string,
+1 -1
View File
@@ -54,7 +54,7 @@ export function createCursorAgent(config: CursorAgentConfig): AgentFn {
"From the thread context, determine the absolute filesystem path where the project/repository is located.",
extractCtx,
);
const fullPrompt = buildAgentPrompt(ctx);
const fullPrompt = await buildAgentPrompt(ctx);
const args = [
"-p",
fullPrompt,
+1 -1
View File
@@ -35,7 +35,7 @@ export function createHermesAgent(config: HermesAgentConfig): AgentFn {
const timeoutMs = config.timeout;
return async (ctx) => {
const fullPrompt = buildAgentPrompt(ctx);
const fullPrompt = await buildAgentPrompt(ctx);
const args = [
"chat",
"-q",
@@ -1,8 +1,14 @@
import { describe, expect, test } from "bun:test";
import { START, type ThreadContext } from "@uncaged/workflow";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createCasStore, START, type ThreadContext } from "@uncaged/workflow";
import { createLlmAdapter } from "../src/create-llm-adapter.js";
const casDir = mkdtempSync(join(tmpdir(), "wf-llm-adapter-cas-"));
const testCas = createCasStore(casDir);
function makeCtx(userContent: string): ThreadContext {
return {
start: {
@@ -11,9 +17,11 @@ function makeCtx(userContent: string): ThreadContext {
meta: { maxRounds: 10 },
timestamp: 1,
},
depth: 0,
steps: [],
threadId: "01TEST000000000000000000TR",
currentRole: { name: "planner", systemPrompt: "system instructions" },
cas: testCas,
};
}
@@ -39,4 +39,5 @@ export const coderRole: RoleDefinition<CoderMeta> = {
"Extract completedPhase: the planner phase hash finished this round (exact hash string from the plan). If multiple phases were finished in one round, use the last finished phase hash. Extract filesChanged and a summary of the work.",
schema: coderMetaSchema,
extractRefs: (meta) => [meta.completedPhase],
extractMode: "single",
};
@@ -21,15 +21,16 @@ export const committerMetaSchema = z.discriminatedUnion("status", [
export type CommitterMeta = z.infer<typeof committerMetaSchema>;
const COMMITTER_SYSTEM = `You are the git committer. Create a branch, commit the changes, and push.
const COMMITTER_SYSTEM = `You are the git committer. Create a branch and commit the changes.
Report the branch name and commit SHA. On failure, classify as recoverable or unrecoverable.
Do not attempt to fix failures yourself.`;
export const committerRole: RoleDefinition<CommitterMeta> = {
description: "Creates branch, commits, and pushes when review passes.",
description: "Creates a branch and commits changes.",
systemPrompt: COMMITTER_SYSTEM,
extractPrompt:
"Extract the commit result: committed (with branch and SHA), recoverable failure, or unrecoverable failure. Include error details and log references if applicable.",
schema: committerMetaSchema,
extractRefs: null,
extractMode: "single",
};
@@ -50,4 +50,5 @@ export const plannerRole: RoleDefinition<PlannerMeta> = {
"Extract the implementation phases from the agent's output. Each phase has a hash (the CAS content-hash returned by the cas put command) and a title (one-line summary).",
schema: plannerMetaSchema,
extractRefs: (meta) => meta.phases.map((p) => p.hash),
extractMode: "single",
};
@@ -48,4 +48,5 @@ export const preparerRole: RoleDefinition<PreparerMeta> = {
"Extract repoPath (absolute path), defaultBranch, conventions (summary string or null), and toolchain (packageManager, testCommand, lintCommand, buildCommand — each string or null).",
schema: preparerMetaSchema,
extractRefs: null,
extractMode: "single",
};
@@ -22,4 +22,5 @@ export const reviewerRole: RoleDefinition<ReviewerMeta> = {
"Extract the review verdict: approved or rejected. If rejected, list the blocking issues.",
schema: reviewerMetaSchema,
extractRefs: null,
extractMode: "single",
};
@@ -0,0 +1,15 @@
{
"name": "@uncaged/workflow-role-tester",
"version": "0.1.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"scripts": {
"build": "echo 'TODO'",
"test": "bun test"
},
"dependencies": {
"@uncaged/workflow": "workspace:*",
"zod": "^4.0.0"
}
}
@@ -0,0 +1 @@
export { type TesterMeta, testerMetaSchema, testerRole } from "./tester.js";
@@ -0,0 +1,27 @@
import type { RoleDefinition } from "@uncaged/workflow";
import * as z from "zod/v4";
export const testerMetaSchema = z.discriminatedUnion("status", [
z.object({
status: z.literal("passed"),
details: z.string(),
}),
z.object({
status: z.literal("failed"),
details: z.string(),
}),
]);
export type TesterMeta = z.infer<typeof testerMetaSchema>;
const TESTER_SYSTEM = `You are a tester. Run the project's test suite, build, and lint commands. Check what commands are available from the preparer's output in the thread. Report pass/fail with details of what failed.`;
export const testerRole: RoleDefinition<TesterMeta> = {
description: "Runs test, build, and lint commands and reports pass or fail with details.",
systemPrompt: TESTER_SYSTEM,
extractPrompt:
"Extract the verification result: passed with summary details, or failed with details of what broke.",
schema: testerMetaSchema,
extractRefs: null,
extractMode: "single",
};
@@ -0,0 +1,10 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "dist",
"composite": true
},
"include": ["src/**/*.ts"],
"references": [{ "path": "../workflow" }]
}
@@ -0,0 +1,260 @@
import { describe, expect, test } from "bun:test";
import {
END,
type ModeratorContext,
type RoleStep,
START,
validateWorkflowDescriptor,
} from "@uncaged/workflow";
import type { CommitterMeta } from "@uncaged/workflow-role-committer";
import type { PlannerMeta } from "@uncaged/workflow-role-planner";
import { buildDevelopDescriptor } from "../src/descriptor.js";
import { developModerator } from "../src/index.js";
import type { DevelopMeta } from "../src/roles.js";
const DEFAULT_PHASES: PlannerMeta["phases"] = [
{
hash: "4KNMR2PX",
title: "Do the work",
},
];
function makeStart(maxRounds: number): ModeratorContext<DevelopMeta>["start"] {
return {
role: START,
content: "Implement the feature",
meta: { maxRounds },
timestamp: 0,
};
}
function makeCtx(
maxRounds: number,
steps: ModeratorContext<DevelopMeta>["steps"],
): ModeratorContext<DevelopMeta> {
return {
threadId: "01TEST000000000000000000TR",
depth: 0,
start: makeStart(maxRounds),
steps,
};
}
function plannerStep(phases: PlannerMeta["phases"] = DEFAULT_PHASES): RoleStep<DevelopMeta> {
return {
role: "planner",
contentHash: "STUBHASHPLANNER001",
meta: { phases },
refs: phases.map((p) => p.hash),
timestamp: 1,
};
}
function coderStep(completedPhase = "4KNMR2PX"): RoleStep<DevelopMeta> {
return {
role: "coder",
contentHash: "STUBHASHCODER00001",
meta: { completedPhase, filesChanged: ["a.ts"], summary: "implemented" },
refs: [completedPhase],
timestamp: 2,
};
}
function reviewerStep(approved: boolean): RoleStep<DevelopMeta> {
return {
role: "reviewer",
contentHash: "STUBHASHREVIEWER01",
meta: approved
? { status: "approved" as const }
: { status: "rejected" as const, issues: ["needs fix"] },
refs: [],
timestamp: 3,
};
}
function testerStep(passed: boolean): RoleStep<DevelopMeta> {
return {
role: "tester",
contentHash: "STUBHASHTESTER01",
meta: passed
? { status: "passed" as const, details: "all checks passed" }
: { status: "failed" as const, details: "lint failed" },
refs: [],
timestamp: 4,
};
}
function committerStep(meta: CommitterMeta): RoleStep<DevelopMeta> {
return {
role: "committer",
contentHash: "STUBHASHCOMMITTER1",
meta,
refs: [],
timestamp: 5,
};
}
describe("developModerator", () => {
test("routes initial → planner → coder → reviewer → tester → committer → END", () => {
expect(developModerator(makeCtx(20, []))).toBe("planner");
expect(developModerator(makeCtx(20, [plannerStep()]))).toBe("coder");
expect(developModerator(makeCtx(20, [plannerStep(), coderStep()]))).toBe("reviewer");
expect(developModerator(makeCtx(20, [plannerStep(), coderStep(), reviewerStep(true)]))).toBe(
"tester",
);
expect(
developModerator(
makeCtx(20, [plannerStep(), coderStep(), reviewerStep(true), testerStep(true)]),
),
).toBe("committer");
expect(
developModerator(
makeCtx(20, [
plannerStep(),
coderStep(),
reviewerStep(true),
testerStep(true),
committerStep({ status: "committed", branch: "feat/x", commitSha: "abc1234" }),
]),
),
).toBe(END);
});
test("reviewer rejects → coder retry when budget allows", () => {
const steps: ModeratorContext<DevelopMeta>["steps"] = [
plannerStep(),
coderStep(),
reviewerStep(false),
];
expect(developModerator(makeCtx(20, steps))).toBe("coder");
});
test("reviewer rejects → END when max rounds exhausted", () => {
const steps: ModeratorContext<DevelopMeta>["steps"] = [
plannerStep(),
coderStep(),
reviewerStep(false),
];
expect(developModerator(makeCtx(4, steps))).toBe(END);
});
test("tester failed → coder retry when budget allows", () => {
const steps: ModeratorContext<DevelopMeta>["steps"] = [
plannerStep(),
coderStep(),
reviewerStep(true),
testerStep(false),
];
expect(developModerator(makeCtx(20, steps))).toBe("coder");
});
test("tester failed → END when max rounds exhausted", () => {
const steps: ModeratorContext<DevelopMeta>["steps"] = [
plannerStep(),
coderStep(),
reviewerStep(true),
testerStep(false),
];
expect(developModerator(makeCtx(5, steps))).toBe(END);
});
test("multiple planner phases → coder until all complete, then reviewer", () => {
const phases: PlannerMeta["phases"] = [
{ hash: "AA000001", title: "first phase" },
{ hash: "AA000002", title: "second phase" },
];
expect(developModerator(makeCtx(20, [plannerStep(phases)]))).toBe("coder");
expect(developModerator(makeCtx(20, [plannerStep(phases), coderStep("AA000001")]))).toBe(
"coder",
);
expect(
developModerator(
makeCtx(20, [plannerStep(phases), coderStep("AA000001"), coderStep("AA000002")]),
),
).toBe("reviewer");
});
test("one-shot coder reports only last phase hash → reviewer (moderator treats as all phases done)", () => {
const phases: PlannerMeta["phases"] = [
{ hash: "BB000001", title: "setup branch" },
{ hash: "BB000002", title: "write tests" },
{ hash: "BB000003", title: "verify" },
{ hash: "BB000004", title: "polish" },
];
expect(developModerator(makeCtx(20, [plannerStep(phases), coderStep("BB000004")]))).toBe(
"reviewer",
);
});
test("unrecognised completedPhase hash → coder retry when budget allows", () => {
const phases: PlannerMeta["phases"] = [
{ hash: "CC000001", title: "first phase" },
{ hash: "CC000002", title: "second phase" },
];
expect(developModerator(makeCtx(20, [plannerStep(phases), coderStep("all-done")]))).toBe(
"coder",
);
});
test("incomplete phases → END when max rounds exhausted", () => {
const phases: PlannerMeta["phases"] = [
{ hash: "DD000001", title: "first phase" },
{ hash: "DD000002", title: "second phase" },
];
const steps: ModeratorContext<DevelopMeta>["steps"] = [
plannerStep(phases),
coderStep("DD000001"),
];
expect(developModerator(makeCtx(3, steps))).toBe(END);
});
test("committer → END for any committer meta status", () => {
const committed = committerStep({ status: "committed", branch: "f", commitSha: "x" });
const recoverable = committerStep({
status: "recoverable",
error: "merge conflict",
logRef: null,
});
const unrecoverable = committerStep({
status: "unrecoverable",
error: "repo missing",
logRef: "log1",
});
const base: ModeratorContext<DevelopMeta>["steps"] = [
plannerStep(),
coderStep(),
reviewerStep(true),
testerStep(true),
];
expect(developModerator(makeCtx(20, [...base, committed]))).toBe(END);
expect(developModerator(makeCtx(20, [...base, recoverable]))).toBe(END);
expect(developModerator(makeCtx(20, [...base, unrecoverable]))).toBe(END);
});
});
describe("buildDevelopDescriptor", () => {
test("lists all roles with schemas that validate", () => {
const descriptor = buildDevelopDescriptor();
const validated = validateWorkflowDescriptor(descriptor);
expect(validated.ok).toBe(true);
if (!validated.ok) {
throw new Error(validated.error);
}
expect(Object.keys(validated.value.roles).sort()).toEqual([
"coder",
"committer",
"planner",
"reviewer",
"tester",
]);
for (const key of ["planner", "coder", "reviewer", "tester", "committer"] as const) {
const role = validated.value.roles[key];
expect(role).toBeDefined();
expect(typeof role.schema).toBe("object");
expect(role.schema).not.toBeNull();
expect(Array.isArray(role.schema)).toBe(false);
}
});
});
@@ -0,0 +1,19 @@
{
"name": "@uncaged/workflow-template-develop",
"version": "0.1.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"scripts": {
"build": "echo 'TODO'",
"test": "bun test"
},
"dependencies": {
"@uncaged/workflow": "workspace:*",
"@uncaged/workflow-role-coder": "workspace:*",
"@uncaged/workflow-role-committer": "workspace:*",
"@uncaged/workflow-role-planner": "workspace:*",
"@uncaged/workflow-role-reviewer": "workspace:*",
"@uncaged/workflow-role-tester": "workspace:*"
}
}
@@ -0,0 +1,12 @@
import { buildDescriptor } from "@uncaged/workflow";
import { developModerator } from "./moderator.js";
import { DEVELOP_WORKFLOW_DESCRIPTION, developRoles } from "./roles.js";
export function buildDevelopDescriptor() {
return buildDescriptor({
description: DEVELOP_WORKFLOW_DESCRIPTION,
roles: developRoles,
moderator: developModerator,
});
}
@@ -0,0 +1,60 @@
import {
type AgentBinding,
createWorkflow,
type ExtractFn,
type LlmProvider,
type WorkflowDefinition,
type WorkflowFn,
} from "@uncaged/workflow";
import { developModerator } from "./moderator.js";
import { DEVELOP_WORKFLOW_DESCRIPTION, type DevelopMeta, developRoles } from "./roles.js";
export {
type CoderMeta,
coderMetaSchema,
coderRole,
} from "@uncaged/workflow-role-coder";
export {
type CommitterMeta,
committerMetaSchema,
committerRole,
} from "@uncaged/workflow-role-committer";
export {
type PlannerMeta,
phaseSchema,
plannerMetaSchema,
plannerRole,
} from "@uncaged/workflow-role-planner";
export {
type ReviewerMeta,
reviewerMetaSchema,
reviewerRole,
} from "@uncaged/workflow-role-reviewer";
export {
type TesterMeta,
testerMetaSchema,
testerRole,
} from "@uncaged/workflow-role-tester";
export { buildDevelopDescriptor } from "./descriptor.js";
export { developModerator } from "./moderator.js";
export {
DEVELOP_WORKFLOW_DESCRIPTION,
type DevelopMeta,
type DevelopRoles,
developRoles,
} from "./roles.js";
export const developWorkflowDefinition: WorkflowDefinition<DevelopMeta> = {
description: DEVELOP_WORKFLOW_DESCRIPTION,
roles: developRoles,
moderator: developModerator,
};
export function createDevelopRun(
binding: AgentBinding,
extract: ExtractFn,
llmProvider: LlmProvider | null,
): WorkflowFn {
return createWorkflow(developWorkflowDefinition, binding, extract, llmProvider);
}
@@ -0,0 +1,89 @@
import type { Moderator, ModeratorContext } from "@uncaged/workflow";
import { END } from "@uncaged/workflow";
import type { DevelopMeta } from "./roles.js";
function coderFinishedAllPlannedPhases(
phases: ReadonlyArray<{ hash: string }>,
coderCompletedPhases: ReadonlyArray<string>,
): boolean {
if (phases.length === 0) {
return true;
}
const plannedHashes = new Set(phases.map((p) => p.hash));
const lastHash = phases[phases.length - 1].hash;
const explicit = new Set(coderCompletedPhases.filter((h) => plannedHashes.has(h)));
if (phases.every((p) => explicit.has(p.hash))) {
return true;
}
if (coderCompletedPhases.some((h) => h === lastHash)) {
return true;
}
return false;
}
function nextAfterCoder(
ctx: ModeratorContext<DevelopMeta>,
maxRounds: number,
): (keyof DevelopMeta & string) | typeof END {
const plannerStep = ctx.steps.find((s) => s.role === "planner");
if (plannerStep === undefined) {
return "reviewer";
}
const phases = plannerStep.meta.phases;
const coderCompletedPhases = ctx.steps
.filter((s) => s.role === "coder")
.map((s) => s.meta.completedPhase);
const allDone = coderFinishedAllPlannedPhases(phases, coderCompletedPhases);
if (allDone) {
return "reviewer";
}
if (ctx.steps.length < maxRounds - 1) {
return "coder";
}
return END;
}
export const developModerator: Moderator<DevelopMeta> = (ctx) => {
const maxRounds = ctx.start.meta.maxRounds;
if (ctx.steps.length === 0) {
return "planner";
}
const last = ctx.steps[ctx.steps.length - 1];
if (last.role === "planner") {
return "coder";
}
if (last.role === "coder") {
return nextAfterCoder(ctx, maxRounds);
}
if (last.role === "reviewer") {
if (last.meta.status === "approved") {
return "tester";
}
if (ctx.steps.length < maxRounds - 1) {
return "coder";
}
return END;
}
if (last.role === "tester") {
if (last.meta.status === "passed") {
return "committer";
}
if (ctx.steps.length < maxRounds - 1) {
return "coder";
}
return END;
}
if (last.role === "committer") {
return END;
}
return END;
};
@@ -0,0 +1,29 @@
import type { RoleDefinition } from "@uncaged/workflow";
import { type CoderMeta, coderRole } from "@uncaged/workflow-role-coder";
import { type CommitterMeta, committerRole } from "@uncaged/workflow-role-committer";
import { type PlannerMeta, plannerRole } from "@uncaged/workflow-role-planner";
import { type ReviewerMeta, reviewerRole } from "@uncaged/workflow-role-reviewer";
import { type TesterMeta, testerRole } from "@uncaged/workflow-role-tester";
export const DEVELOP_WORKFLOW_DESCRIPTION =
"Plan phases, implement incrementally, review, verify with tests/build/lint, and commit (planner → coder [repeat per phase] → reviewer → tester → committer).";
export type DevelopMeta = {
planner: PlannerMeta;
coder: CoderMeta;
reviewer: ReviewerMeta;
tester: TesterMeta;
committer: CommitterMeta;
};
export type DevelopRoles = {
[K in keyof DevelopMeta]: RoleDefinition<DevelopMeta[K]>;
};
export const developRoles: DevelopRoles = {
planner: plannerRole,
coder: coderRole,
reviewer: reviewerRole,
tester: testerRole,
committer: committerRole,
};
@@ -0,0 +1,17 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "dist",
"composite": true
},
"include": ["src/**/*.ts"],
"references": [
{ "path": "../workflow" },
{ "path": "../workflow-role-coder" },
{ "path": "../workflow-role-committer" },
{ "path": "../workflow-role-planner" },
{ "path": "../workflow-role-reviewer" },
{ "path": "../workflow-role-tester" }
]
}
@@ -1,5 +1,9 @@
import { afterEach, describe, expect, test } from "bun:test";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import {
createCasStore,
createExtract,
END,
type ModeratorContext,
@@ -104,6 +108,7 @@ function makeCtx(
): ModeratorContext<SolveIssueMeta> {
return {
threadId: "01TEST000000000000000000TR",
depth: 0,
start: makeStart(maxRounds),
steps,
};
@@ -112,7 +117,7 @@ function makeCtx(
function preparerStep(): RoleStep<SolveIssueMeta> {
return {
role: "preparer",
content: "prepared",
contentHash: "STUBHASHPREPARER01",
meta: {
repoPath: "/home/user/repos/test",
defaultBranch: "main",
@@ -132,7 +137,7 @@ function preparerStep(): RoleStep<SolveIssueMeta> {
function plannerStep(phases: PlannerMeta["phases"] = DEFAULT_PHASES): RoleStep<SolveIssueMeta> {
return {
role: "planner",
content: "plan",
contentHash: "STUBHASHPLANNER001",
meta: { phases },
refs: phases.map((p) => p.hash),
timestamp: 1,
@@ -142,7 +147,7 @@ function plannerStep(phases: PlannerMeta["phases"] = DEFAULT_PHASES): RoleStep<S
function coderStep(completedPhase = "4KNMR2PX"): RoleStep<SolveIssueMeta> {
return {
role: "coder",
content: "code",
contentHash: "STUBHASHCODER00001",
meta: { completedPhase, filesChanged: ["a.ts"], summary: "fixed" },
refs: [completedPhase],
timestamp: 2,
@@ -152,7 +157,7 @@ function coderStep(completedPhase = "4KNMR2PX"): RoleStep<SolveIssueMeta> {
function reviewerStep(approved: boolean): RoleStep<SolveIssueMeta> {
return {
role: "reviewer",
content: "rev",
contentHash: "STUBHASHREVIEWER01",
meta: approved
? { status: "approved" as const }
: { status: "rejected" as const, issues: ["needs fix"] },
@@ -164,7 +169,7 @@ function reviewerStep(approved: boolean): RoleStep<SolveIssueMeta> {
function committerStep(): RoleStep<SolveIssueMeta> {
return {
role: "committer",
content: "commit",
contentHash: "STUBHASHCOMMITTER1",
meta: { status: "committed", branch: "feat/issue-1", commitSha: "abc1234" },
refs: [],
timestamp: 4,
@@ -280,10 +285,15 @@ describe("solveIssueModerator", () => {
describe("createSolveIssueRun", () => {
let restoreFetch: (() => void) | null = null;
let casDir: string | undefined;
afterEach(() => {
afterEach(async () => {
restoreFetch?.();
restoreFetch = null;
if (casDir !== undefined) {
await rm(casDir, { recursive: true, force: true }).catch(() => {});
casDir = undefined;
}
});
test("structured extraction yields preparer then planner meta from mocked chat completions", async () => {
@@ -300,10 +310,13 @@ describe("createSolveIssueRun", () => {
};
restoreFetch = installMockChatCompletions([EXPECT_PREPARER_META, EXPECT_PLANNER_META]);
const run = createSolveIssueRun({ agent: async () => "" }, stubExtract);
casDir = await mkdtemp(join(tmpdir(), "solve-issue-cas-"));
const cas = createCasStore(casDir);
const run = createSolveIssueRun({ agent: async () => "" }, stubExtract, null);
const gen = run(
{ prompt: "task", steps: [] },
{ threadId: "01TEST000000000000000000TR", maxRounds: 20 },
{ threadId: "01TEST000000000000000000TR", maxRounds: 20, depth: 0, cas },
);
const first = await gen.next();
expect(first.done).toBe(false);
@@ -335,6 +348,9 @@ describe("createSolveIssueRun", () => {
EXPECT_CODER_META,
]);
casDir = await mkdtemp(join(tmpdir(), "solve-issue-cas-"));
const cas = createCasStore(casDir);
const calls: string[] = [];
const run = createSolveIssueRun(
{
@@ -358,10 +374,11 @@ describe("createSolveIssueRun", () => {
},
},
stubExtract,
null,
);
const gen = run(
{ prompt: "task", steps: [] },
{ threadId: "01TEST000000000000000000TR", maxRounds: 20 },
{ threadId: "01TEST000000000000000000TR", maxRounds: 20, depth: 0, cas },
);
await gen.next();
expect(calls).toEqual(["preparer"]);
@@ -2,6 +2,7 @@ import {
type AgentBinding,
createWorkflow,
type ExtractFn,
type LlmProvider,
type WorkflowDefinition,
type WorkflowFn,
} from "@uncaged/workflow";
@@ -50,6 +51,10 @@ export const solveIssueWorkflowDefinition: WorkflowDefinition<SolveIssueMeta> =
moderator: solveIssueModerator,
};
export function createSolveIssueRun(binding: AgentBinding, extract: ExtractFn): WorkflowFn {
return createWorkflow(solveIssueWorkflowDefinition, binding, extract);
export function createSolveIssueRun(
binding: AgentBinding,
extract: ExtractFn,
llmProvider: LlmProvider | null,
): WorkflowFn {
return createWorkflow(solveIssueWorkflowDefinition, binding, extract, llmProvider);
}
@@ -1,5 +1,8 @@
import { describe, expect, test } from "bun:test";
import { START, type ThreadContext } from "@uncaged/workflow";
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createCasStore, putContentMerkleNode, START, type ThreadContext } from "@uncaged/workflow";
import { buildAgentPrompt } from "../src/index.js";
@@ -13,36 +16,53 @@ function startTask(content: string): ThreadContext["start"] {
}
describe("buildAgentPrompt", () => {
test("includes system prompt and full task; omits tools when there are no steps", () => {
let casRoot: string;
beforeEach(async () => {
casRoot = await mkdtemp(join(tmpdir(), "wf-build-prompt-cas-"));
});
afterEach(async () => {
await rm(casRoot, { recursive: true, force: true });
});
test("includes system prompt and full task; omits tools when there are no steps", async () => {
const cas = createCasStore(casRoot);
const ctx: ThreadContext = {
start: startTask("fix the bug"),
depth: 0,
steps: [],
threadId: "01TEST000000000000000000TR",
currentRole: { name: START, systemPrompt: "You are an agent." },
cas,
};
const text = buildAgentPrompt(ctx);
const text = await buildAgentPrompt(ctx);
expect(text).toContain("You are an agent.");
expect(text).toContain("## Task");
expect(text).toContain("fix the bug");
expect(text).not.toContain("## Tools");
});
test("single step shows full content and meta, and includes tools", () => {
test("single step shows full content and meta, and includes tools", async () => {
const cas = createCasStore(casRoot);
const onlyHash = await putContentMerkleNode(cas, "only step full body");
const ctx: ThreadContext = {
start: startTask("user task"),
depth: 0,
threadId: "01TEST000000000000000000TR",
currentRole: { name: "coder", systemPrompt: "Be helpful." },
cas,
steps: [
{
role: "coder",
content: "only step full body",
contentHash: onlyHash,
meta: { files: ["a.ts"] },
refs: [],
refs: [onlyHash],
timestamp: 2,
},
],
};
const text = buildAgentPrompt(ctx);
const text = await buildAgentPrompt(ctx);
expect(text).toContain("## Task");
expect(text).toContain("user task");
expect(text).toContain("## Step: coder");
@@ -52,29 +72,34 @@ describe("buildAgentPrompt", () => {
expect(text).toContain("uncaged-workflow thread 01TEST000000000000000000TR");
});
test("two or more steps: previous steps are meta-only; latest step is full", () => {
test("two or more steps: previous steps are meta-only; latest step is full", async () => {
const cas = createCasStore(casRoot);
const plannerHash = await putContentMerkleNode(cas, "PLANNER_SECRET_FULL_TEXT");
const coderHash = await putContentMerkleNode(cas, "last step full content");
const ctx: ThreadContext = {
start: startTask("first message full: task content here"),
depth: 0,
threadId: "01TEST000000000000000000TR",
currentRole: { name: "coder", systemPrompt: "System." },
cas,
steps: [
{
role: "planner",
content: "PLANNER_SECRET_FULL_TEXT",
contentHash: plannerHash,
meta: { plan: "short" },
refs: [],
refs: [plannerHash],
timestamp: 2,
},
{
role: "coder",
content: "last step full content",
contentHash: coderHash,
meta: { done: true },
refs: [],
refs: [coderHash],
timestamp: 3,
},
],
};
const text = buildAgentPrompt(ctx);
const text = await buildAgentPrompt(ctx);
expect(text).toContain("first message full: task content here");
expect(text).toContain("## Previous Steps");
expect(text).toContain("### Step 1: planner");
@@ -87,36 +112,42 @@ describe("buildAgentPrompt", () => {
expect(text).toContain("uncaged-workflow thread 01TEST000000000000000000TR");
});
test("middle steps show meta summary only, not full content", () => {
test("middle steps show meta summary only, not full content", async () => {
const cas = createCasStore(casRoot);
const ha = await putContentMerkleNode(cas, "HIDDEN_A");
const hb = await putContentMerkleNode(cas, "HIDDEN_B_MIDDLE");
const hc = await putContentMerkleNode(cas, "VISIBLE_LAST");
const ctx: ThreadContext = {
start: startTask("start"),
depth: 0,
threadId: "01TEST000000000000000000TR",
currentRole: { name: "c", systemPrompt: "S" },
cas,
steps: [
{
role: "a",
content: "HIDDEN_A",
contentHash: ha,
meta: { n: 1 },
refs: [],
refs: [ha],
timestamp: 2,
},
{
role: "b",
content: "HIDDEN_B_MIDDLE",
contentHash: hb,
meta: { n: 2 },
refs: [],
refs: [hb],
timestamp: 3,
},
{
role: "c",
content: "VISIBLE_LAST",
contentHash: hc,
meta: { n: 3 },
refs: [],
refs: [hc],
timestamp: 4,
},
],
};
const text = buildAgentPrompt(ctx);
const text = await buildAgentPrompt(ctx);
expect(text).not.toContain("HIDDEN_A");
expect(text).not.toContain("HIDDEN_B_MIDDLE");
expect(text).toContain('Summary: {"n":1}');
@@ -1,7 +1,16 @@
import type { AgentContext } from "@uncaged/workflow";
import { getContentMerklePayload } from "@uncaged/workflow";
async function resolveStepText(ctx: AgentContext, contentHash: string): Promise<string> {
const text = await getContentMerklePayload(ctx.cas, contentHash);
if (text === null) {
throw new Error(`buildAgentPrompt: missing CAS blob for ${contentHash}`);
}
return text;
}
/** Builds the full agent prompt: system instructions plus summarized thread history. */
export function buildAgentPrompt(ctx: AgentContext): string {
export async function buildAgentPrompt(ctx: AgentContext): Promise<string> {
const lines: string[] = [];
lines.push(ctx.currentRole.systemPrompt);
lines.push("");
@@ -15,10 +24,11 @@ export function buildAgentPrompt(ctx: AgentContext): string {
if (steps.length === 1) {
const s = steps[0];
const body = await resolveStepText(ctx, s.contentHash);
lines.push("");
lines.push(`## Step: ${s.role}`);
lines.push("");
lines.push(s.content);
lines.push(body);
lines.push("");
lines.push(`Meta: ${JSON.stringify(s.meta)}`);
} else {
@@ -31,10 +41,11 @@ export function buildAgentPrompt(ctx: AgentContext): string {
lines.push(`Summary: ${JSON.stringify(s.meta)}`);
}
const last = steps[steps.length - 1];
const lastBody = await resolveStepText(ctx, last.contentHash);
lines.push("");
lines.push(`## Latest Step: ${last.role}`);
lines.push("");
lines.push(last.content);
lines.push(lastBody);
lines.push("");
lines.push(`Meta: ${JSON.stringify(last.meta)}`);
}
@@ -23,6 +23,7 @@ describe("buildDescriptor", () => {
extractPrompt: "Extract title and count from the analysis.",
schema,
extractRefs: null,
extractMode: "single",
},
},
moderator: () => END,
@@ -26,6 +26,19 @@ export const run = async function* (input) {
expect(r.ok).toBe(true);
});
test("allows static import of @uncaged/workflow", () => {
const source = `${minimalDescriptor}import { putContentMerkleNode } from "@uncaged/workflow";
export const run = async function* (_input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "x");
return { returnCode: 0, summary: h };
};
`;
const r = validateWorkflowBundle({ filePath: "/tmp/w.esm.js", source });
expect(r.ok).toBe(true);
});
test("rejects wrong filename suffix", () => {
const r = validateWorkflowBundle({
filePath: "/tmp/w.js",
+308 -14
View File
@@ -4,11 +4,18 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import * as z from "zod/v4";
import { createCasStore } from "../src/cas.js";
import { createWorkflow } from "../src/create-workflow.js";
import { executeThread } from "../src/engine.js";
import { createExtract } from "../src/extract-fn.js";
import { createLogger } from "../src/logger.js";
import { END } from "../src/types.js";
import {
createContentMerkleNode,
getContentMerklePayload,
parseMerkleNode,
serializeMerkleNode,
} from "../src/merkle.js";
import { END, type LlmProvider } from "../src/types.js";
const plannerMetaSchema = z.object({
plan: z.string(),
@@ -90,6 +97,7 @@ const demoWorkflow = createWorkflow<DemoMeta>(
extractPrompt: "Extract plan text and affected files list.",
schema: plannerMetaSchema,
extractRefs: null,
extractMode: "single",
},
coder: {
description: "Demo coder",
@@ -97,6 +105,7 @@ const demoWorkflow = createWorkflow<DemoMeta>(
extractPrompt: "Extract the code diff summary.",
schema: coderMetaSchema,
extractRefs: null,
extractMode: "single",
},
},
moderator: (ctx) => {
@@ -117,6 +126,7 @@ const demoWorkflow = createWorkflow<DemoMeta>(
},
},
demoExtract,
null,
);
describe("executeThread", () => {
@@ -140,6 +150,7 @@ describe("executeThread", () => {
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
await mkdir(join(root, "logs", hash), { recursive: true });
const cas = createCasStore(join(root, "cas"));
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
const ac = new AbortController();
@@ -150,16 +161,30 @@ describe("executeThread", () => {
{ prompt: "Fix the login redirect bug in #3", steps: [] },
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
prefilledDiskSteps: null,
},
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath },
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
logger,
);
expect(result.returnCode).toBe(0);
expect(typeof result.rootHash).toBe("string");
expect(result.rootHash.length).toBeGreaterThan(0);
const rootYaml = await cas.get(result.rootHash);
expect(rootYaml).not.toBeNull();
const rootNode = parseMerkleNode(rootYaml ?? "");
expect(rootNode.type).toBe("thread");
const rootPayload = rootNode.payload as Record<string, unknown>;
expect(rootPayload.workflow).toBe("demo-flow");
expect(rootPayload.threadId).toBe(threadId);
const rootResult = rootPayload.result as Record<string, unknown>;
expect(rootResult.returnCode).toBe(0);
expect(rootNode.children.length).toBe(2);
const dataText = await readFile(dataPath, "utf8");
const lines = dataText
@@ -178,18 +203,34 @@ describe("executeThread", () => {
expect(params.prompt).toBe("Fix the login redirect bug in #3");
const opts = params.options as Record<string, unknown>;
expect(opts.maxRounds).toBe(5);
expect(Object.keys(opts).sort()).toEqual(["maxRounds"]);
expect(opts.depth).toBe(0);
expect(Object.keys(opts).sort()).toEqual(["depth", "maxRounds"]);
const role1 = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
expect(role1.role).toBe("planner");
expect(role1.content).toBe("plan-body");
expect(typeof role1.contentHash).toBe("string");
expect(await getContentMerklePayload(cas, String(role1.contentHash))).toBe("plan-body");
expect(role1.meta).toEqual({ plan: "do-it", files: ["a.ts"] });
expect(role1.refs).toEqual([]);
expect(role1.refs).toEqual([role1.contentHash]);
expect(typeof role1.timestamp).toBe("number");
const role2 = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
expect(role2.role).toBe("coder");
expect(role2.refs).toEqual([]);
expect(role2.refs).toEqual([role2.contentHash]);
const step1Yaml = await cas.get(rootNode.children[0] ?? "");
const step2Yaml = await cas.get(rootNode.children[1] ?? "");
expect(step1Yaml).not.toBeNull();
expect(step2Yaml).not.toBeNull();
const step1Node = parseMerkleNode(step1Yaml ?? "");
const step2Node = parseMerkleNode(step2Yaml ?? "");
expect(step1Node.type).toBe("step");
expect(step2Node.type).toBe("step");
expect(step1Node.children).toEqual([String(role1.contentHash)]);
expect(step2Node.children).toEqual([String(role2.contentHash)]);
const step1Payload = step1Node.payload as Record<string, unknown>;
expect(step1Payload.role).toBe("planner");
expect(step1Payload.meta).toEqual({ plan: "do-it", files: ["a.ts"] });
const infoText = await readFile(infoPath, "utf8");
const infoLines = infoText
@@ -217,11 +258,14 @@ describe("executeThread", () => {
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
await mkdir(join(root, "logs", hash), { recursive: true });
const cas = createCasStore(join(root, "cas"));
const plannerHash = await cas.put(serializeMerkleNode(createContentMerkleNode("plan-body")));
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
const ac = new AbortController();
const histTs = 9_000_000;
const mergedPlannerRefs = ["CAS111AAAAAAA", plannerHash];
const result = await executeThread(
demoWorkflow,
"demo-flow",
@@ -230,32 +274,38 @@ describe("executeThread", () => {
steps: [
{
role: "planner",
content: "plan-body",
contentHash: plannerHash,
meta: { plan: "do-it", files: ["a.ts"] },
refs: ["CAS111AAAAAAA"],
refs: mergedPlannerRefs,
},
],
},
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: "01SRC1111111111111111111",
prefilledDiskSteps: [
{
role: "planner",
content: "plan-body",
contentHash: plannerHash,
meta: { plan: "do-it", files: ["a.ts"] },
refs: ["CAS111AAAAAAA"],
refs: mergedPlannerRefs,
timestamp: histTs,
},
],
},
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath },
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
logger,
);
expect(result.returnCode).toBe(0);
expect(typeof result.rootHash).toBe("string");
const rootYaml = await cas.get(result.rootHash);
const rootNode = parseMerkleNode(rootYaml ?? "");
expect(rootNode.children.length).toBe(2);
const dataText = await readFile(dataPath, "utf8");
const lines = dataText
@@ -270,11 +320,11 @@ describe("executeThread", () => {
const role0 = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
expect(role0.role).toBe("planner");
expect(role0.timestamp).toBe(histTs);
expect(role0.refs).toEqual(["CAS111AAAAAAA"]);
expect(role0.refs).toEqual(mergedPlannerRefs);
const role1 = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
expect(role1.role).toBe("coder");
expect(role1.content).toBe("code-body");
expect(await getContentMerklePayload(cas, String(role1.contentHash))).toBe("code-body");
} finally {
await rm(root, { recursive: true, force: true });
}
@@ -288,6 +338,7 @@ describe("executeThread", () => {
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
await mkdir(join(root, "logs", hash), { recursive: true });
const cas = createCasStore(join(root, "cas"));
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
const ac = new AbortController();
@@ -298,16 +349,23 @@ describe("executeThread", () => {
{ prompt: "hello", steps: [] },
{
maxRounds: 0,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
prefilledDiskSteps: null,
},
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath },
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
logger,
);
expect(result.returnCode).toBe(0);
expect(typeof result.rootHash).toBe("string");
const rootYaml = await cas.get(result.rootHash);
const rootNode = parseMerkleNode(rootYaml ?? "");
expect(rootNode.type).toBe("thread");
expect(rootNode.children.length).toBe(0);
const dataText = await readFile(dataPath, "utf8");
const lines = dataText
@@ -319,4 +377,240 @@ describe("executeThread", () => {
await rm(root, { recursive: true, force: true });
}
});
test("Merkle DAG: root → step nodes → content for full thread traversal", async () => {
restoreFetch = installMockChatCompletions([
{ plan: "do-it", files: ["a.ts"] },
{ diff: "+ok" },
]);
const root = await mkdtemp(join(tmpdir(), "wf-engine-dag-"));
try {
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
const hash = "C9NMV6V2TQT81";
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
await mkdir(join(root, "logs", hash), { recursive: true });
const cas = createCasStore(join(root, "cas"));
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
const ac = new AbortController();
const result = await executeThread(
demoWorkflow,
"demo-flow",
{ prompt: "DAG test", steps: [] },
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
prefilledDiskSteps: null,
},
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
logger,
);
const dataText = await readFile(dataPath, "utf8");
const lines = dataText
.trim()
.split("\n")
.filter((l) => l !== "");
expect(lines.length).toBe(3);
const rolePlanner = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
const roleCoder = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
const threadYaml = await cas.get(result.rootHash);
expect(threadYaml).not.toBeNull();
const threadNode = parseMerkleNode(threadYaml ?? "");
expect(threadNode.type).toBe("thread");
const bodies: string[] = [];
for (const stepHash of threadNode.children) {
const stepYaml = await cas.get(stepHash);
expect(stepYaml).not.toBeNull();
const stepNode = parseMerkleNode(stepYaml ?? "");
expect(stepNode.type).toBe("step");
expect(stepNode.children.length).toBe(1);
const contentHash = stepNode.children[0];
expect(contentHash).toBeDefined();
const body = await getContentMerklePayload(cas, contentHash ?? "");
expect(body).not.toBeNull();
bodies.push(body ?? "");
}
expect(bodies.sort()).toEqual(["code-body", "plan-body"].sort());
expect(rolePlanner.role).toBe("planner");
expect(roleCoder.role).toBe("coder");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("extractMode react traverses CAS DAG via cas_get during extraction", async () => {
const dagMetaSchema = z.object({ leafPayload: z.string() });
type DagDemoMeta = { walker: z.infer<typeof dagMetaSchema> };
const origFetch = globalThis.fetch;
restoreFetch = () => {
globalThis.fetch = origFetch;
};
let fetchRound = 0;
const root = await mkdtemp(join(tmpdir(), "wf-engine-react-"));
try {
const cas = createCasStore(join(root, "cas"));
const leafYaml = serializeMerkleNode(createContentMerkleNode("needle-from-leaf"));
const leafHash = await cas.put(leafYaml);
const rootYaml = serializeMerkleNode({
type: "thread",
payload: {
workflow: "dag-demo",
threadId: "01DAG00000000000000000001",
result: { returnCode: 0, summary: "" },
},
children: [leafHash],
});
const dagRootHash = await cas.put(rootYaml);
globalThis.fetch = Object.assign(
async (_input: Parameters<typeof fetch>[0], _init?: RequestInit) => {
fetchRound += 1;
if (fetchRound === 1) {
return new Response(
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
id: "c1",
type: "function",
function: {
name: "cas_get",
arguments: JSON.stringify({ hash: dagRootHash }),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
}
if (fetchRound === 2) {
return new Response(
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
id: "c2",
type: "function",
function: {
name: "cas_get",
arguments: JSON.stringify({ hash: leafHash }),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
}
return new Response(
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
id: "c3",
type: "function",
function: {
name: "extract",
arguments: JSON.stringify({ leafPayload: "needle-from-leaf" }),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
},
{ preconnect: origFetch.preconnect.bind(origFetch) },
) as typeof fetch;
const llm: LlmProvider = { baseUrl: "http://127.0.0.1:9", apiKey: "test", model: "test" };
const extractFn = createExtract(llm);
const dagWorkflow = createWorkflow<DagDemoMeta>(
{
roles: {
walker: {
description: "DAG walker",
systemPrompt: "Output only the root CAS hash.",
extractPrompt:
"Set leafPayload to the string payload of the content Merkle node under the root.",
schema: dagMetaSchema,
extractRefs: null,
extractMode: "react",
},
},
moderator: (ctx) => (ctx.steps.length === 0 ? "walker" : END),
},
{ agent: async () => dagRootHash },
extractFn,
llm,
);
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
const hash = "C9NMV6V2TQT81";
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
await mkdir(join(root, "logs", hash), { recursive: true });
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
const ac = new AbortController();
const result = await executeThread(
dagWorkflow,
"dag-demo",
{ prompt: "traverse", steps: [] },
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
prefilledDiskSteps: null,
},
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
logger,
);
expect(result.returnCode).toBe(0);
expect(fetchRound).toBe(3);
const dataText = await readFile(dataPath, "utf8");
const lines = dataText
.trim()
.split("\n")
.filter((l) => l !== "");
const roleRec = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
expect(roleRec.role).toBe("walker");
expect(roleRec.meta).toEqual({ leafPayload: "needle-from-leaf" });
} finally {
globalThis.fetch = origFetch;
await rm(root, { recursive: true, force: true });
}
});
});
@@ -0,0 +1,87 @@
import { describe, expect, test } from "bun:test";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { getExtractProvider } from "../src/extract-provider.js";
describe("getExtractProvider", () => {
test("returns provider when config.extract is present", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-ext-prov-ok-"));
try {
await mkdir(root, { recursive: true });
await writeFile(
join(root, "workflow.yaml"),
`config:
maxDepth: 3
extract:
baseUrl: https://dashscope.aliyuncs.com/compatible-mode/v1
model: qwen-plus
apiKey: literal-key
workflows: {}
`,
"utf8",
);
const r = await getExtractProvider(root);
expect(r.ok).toBe(true);
if (!r.ok) {
return;
}
expect(r.value.baseUrl).toBe("https://dashscope.aliyuncs.com/compatible-mode/v1");
expect(r.value.model).toBe("qwen-plus");
expect(r.value.apiKey).toBe("literal-key");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("errs when registry has no config section", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-ext-prov-missing-"));
try {
await mkdir(root, { recursive: true });
await writeFile(join(root, "workflow.yaml"), "workflows: {}\n", "utf8");
const r = await getExtractProvider(root);
expect(r.ok).toBe(false);
if (r.ok) {
return;
}
expect(r.error).toContain("no global config");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("resolves apiKey from env at registry read time", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-ext-prov-env-"));
const prev = process.env.WF_GET_EXTRACT_PROVIDER_KEY;
process.env.WF_GET_EXTRACT_PROVIDER_KEY = "resolved-secret";
try {
await mkdir(root, { recursive: true });
await writeFile(
join(root, "workflow.yaml"),
`config:
maxDepth: 1
extract:
baseUrl: https://example.com
model: m
apiKey: env:WF_GET_EXTRACT_PROVIDER_KEY
workflows: {}
`,
"utf8",
);
const r = await getExtractProvider(root);
expect(r.ok).toBe(true);
if (!r.ok) {
return;
}
expect(r.value.apiKey).toBe("resolved-secret");
} finally {
if (prev === undefined) {
delete process.env.WF_GET_EXTRACT_PROVIDER_KEY;
} else {
process.env.WF_GET_EXTRACT_PROVIDER_KEY = prev;
}
await rm(root, { recursive: true, force: true });
}
});
});
@@ -7,9 +7,9 @@ import {
} from "../src/fork-thread.js";
const sampleDataJsonl = `{"name":"demo","hash":"C9NMV6V2TQT81","threadId":"01AAA1111111111111111111","parameters":{"prompt":"hi","options":{"maxRounds":5}},"timestamp":100}
{"role":"planner","content":"p","meta":{},"timestamp":101}
{"role":"coder","content":"c","meta":{},"timestamp":102}
{"role":"reviewer","content":"r","meta":{},"timestamp":103}
{"role":"planner","contentHash":"HP0000000000000000000001","meta":{},"refs":[],"timestamp":101}
{"role":"coder","contentHash":"HP0000000000000000000002","meta":{},"refs":[],"timestamp":102}
{"role":"reviewer","contentHash":"HP0000000000000000000003","meta":{},"refs":[],"timestamp":103}
`;
describe("fork-thread", () => {
@@ -24,6 +24,7 @@ describe("fork-thread", () => {
expect(r.value.start.threadId).toBe("01AAA1111111111111111111");
expect(r.value.start.prompt).toBe("hi");
expect(r.value.start.maxRounds).toBe(5);
expect(r.value.start.depth).toBe(0);
expect(r.value.roleSteps.length).toBe(3);
expect(r.value.roleSteps[0]?.role).toBe("planner");
});
@@ -83,6 +84,44 @@ describe("fork-thread", () => {
expect(r.value.workflowName).toBe("demo");
expect(r.value.historicalSteps.length).toBe(1);
expect(r.value.historicalSteps[0]?.timestamp).toBe(101);
expect(r.value.runOptions).toEqual({ maxRounds: 5 });
expect(r.value.runOptions).toEqual({ maxRounds: 5, depth: 0 });
});
test("parseThreadDataJsonl ignores trailing WorkflowResult line", () => {
const text = `${sampleDataJsonl.trim()}\n{"returnCode":0,"summary":"done"}\n`;
const r = parseThreadDataJsonl(text);
expect(r.ok).toBe(true);
if (!r.ok) {
return;
}
expect(r.value.roleSteps.length).toBe(3);
expect(r.value.roleSteps[2]?.role).toBe("reviewer");
});
test("parseThreadDataJsonl errors when WorkflowResult is not last", () => {
const text = `{"name":"demo","hash":"H","threadId":"01ZZZZZZZZZZZZZZZZZZZZZZ","parameters":{"prompt":"p","options":{"maxRounds":3}},"timestamp":1}
{"returnCode":0,"summary":"early"}
{"role":"planner","content":"x","meta":{},"timestamp":2}
`;
const r = parseThreadDataJsonl(text);
expect(r.ok).toBe(false);
});
test("parseThreadDataJsonl reads explicit depth from start record", () => {
const text = `{"name":"demo","hash":"H","threadId":"01ZZZZZZZZZZZZZZZZZZZZZZ","parameters":{"prompt":"p","options":{"maxRounds":3,"depth":2}},"timestamp":1}
{"role":"planner","contentHash":"HP0000000000000000000099","meta":{},"refs":[],"timestamp":2}
`;
const r = parseThreadDataJsonl(text);
expect(r.ok).toBe(true);
if (!r.ok) {
return;
}
expect(r.value.start.depth).toBe(2);
const plan = buildForkPlan(text, null);
expect(plan.ok).toBe(true);
if (!plan.ok) {
return;
}
expect(plan.value.runOptions).toEqual({ maxRounds: 3, depth: 2 });
});
});
@@ -0,0 +1,29 @@
import { describe, expect, test } from "bun:test";
import { createContentMerkleNode, parseMerkleNode, serializeMerkleNode } from "../src/merkle.js";
describe("merkle", () => {
test("content node roundtrips through YAML", () => {
const node = createContentMerkleNode("hello\nworld");
const yaml = serializeMerkleNode(node);
const back = parseMerkleNode(yaml);
expect(back).toEqual(node);
});
test("step node with object payload roundtrips", () => {
const node = {
type: "step" as const,
payload: { role: "planner", foo: 1 },
children: ["ABC123", "DEF456"],
};
const yaml = serializeMerkleNode(node);
const back = parseMerkleNode(yaml);
expect(back.type).toBe("step");
expect(back.payload).toEqual({ role: "planner", foo: 1 });
expect(back.children).toEqual(["ABC123", "DEF456"]);
});
test("parse rejects invalid YAML root", () => {
expect(() => parseMerkleNode("[]")).toThrow();
});
});
@@ -0,0 +1,209 @@
import { afterEach, describe, expect, test } from "bun:test";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import * as z from "zod/v4";
import { createCasStore } from "../src/cas.js";
import { createContentMerkleNode, serializeMerkleNode } from "../src/merkle.js";
import { reactExtract } from "../src/react-extract.js";
import type { LlmProvider } from "../src/types.js";
const metaSchema = z.object({ seen: z.string() });
const provider: LlmProvider = {
baseUrl: "http://127.0.0.1:9",
apiKey: "test",
model: "test",
};
describe("reactExtract", () => {
let restoreFetch: (() => void) | null = null;
afterEach(() => {
restoreFetch?.();
restoreFetch = null;
});
test("cas_get rounds then extract tool yields validated meta", async () => {
const casDir = await mkdtemp(join(tmpdir(), "react-extract-"));
const cas = createCasStore(casDir);
try {
const blob = serializeMerkleNode(createContentMerkleNode("needle"));
const h = await cas.put(blob);
const origFetch = globalThis.fetch;
let round = 0;
restoreFetch = () => {
globalThis.fetch = origFetch;
};
globalThis.fetch = Object.assign(
async (_input: Parameters<typeof fetch>[0], _init?: RequestInit) => {
round += 1;
if (round === 1) {
return new Response(
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
id: "t1",
type: "function",
function: {
name: "cas_get",
arguments: JSON.stringify({ hash: h }),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
}
return new Response(
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
id: "t2",
type: "function",
function: {
name: "extract",
arguments: JSON.stringify({ seen: "needle" }),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
},
{ preconnect: origFetch.preconnect.bind(origFetch) },
) as typeof fetch;
const text = `## Agent Output\n${h}\n## Extraction Instruction\nExtract seen from CAS.`;
const result = await reactExtract({
text,
schema: metaSchema,
provider,
cas,
});
expect(result.ok).toBe(true);
if (!result.ok) {
return;
}
expect(result.value).toEqual({ seen: "needle" });
expect(round).toBe(2);
} finally {
await rm(casDir, { recursive: true, force: true });
}
});
test("stops after max tool rounds when model keeps calling cas_get", async () => {
const casDir = await mkdtemp(join(tmpdir(), "react-extract-max-"));
const cas = createCasStore(casDir);
try {
const blob = serializeMerkleNode(createContentMerkleNode("x"));
const h = await cas.put(blob);
const origFetch = globalThis.fetch;
let round = 0;
restoreFetch = () => {
globalThis.fetch = origFetch;
};
globalThis.fetch = Object.assign(
async (_input: Parameters<typeof fetch>[0], _init?: RequestInit) => {
round += 1;
return new Response(
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
id: `loop-${round}`,
type: "function",
function: {
name: "cas_get",
arguments: JSON.stringify({ hash: h }),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
},
{ preconnect: origFetch.preconnect.bind(origFetch) },
) as typeof fetch;
const result = await reactExtract({
text: "## Agent Output\nnoop\n## Extraction Instruction\nExtract seen.",
schema: metaSchema,
provider,
cas,
});
expect(result.ok).toBe(false);
if (result.ok) {
return;
}
expect(result.error).toBe("max_react_rounds_exceeded");
expect(round).toBe(10);
} finally {
await rm(casDir, { recursive: true, force: true });
}
});
test("passthrough JSON assistant message without tool calls", async () => {
const casDir = await mkdtemp(join(tmpdir(), "react-extract-pass-"));
const cas = createCasStore(casDir);
try {
const origFetch = globalThis.fetch;
restoreFetch = () => {
globalThis.fetch = origFetch;
};
globalThis.fetch = Object.assign(
async (_input: Parameters<typeof fetch>[0], _init?: RequestInit) =>
new Response(
JSON.stringify({
choices: [
{
message: {
content: '{"seen":"direct"}',
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
),
{ preconnect: origFetch.preconnect.bind(origFetch) },
) as typeof fetch;
const result = await reactExtract({
text: "## Agent Output\nok\n## Extraction Instruction\nExtract.",
schema: metaSchema,
provider,
cas,
});
expect(result.ok).toBe(true);
if (!result.ok) {
return;
}
expect(result.value).toEqual({ seen: "direct" });
} finally {
await rm(casDir, { recursive: true, force: true });
}
});
});
@@ -4,6 +4,7 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import * as z from "zod/v4";
import { createCasStore } from "../src/cas.js";
import { createWorkflow } from "../src/create-workflow.js";
import { executeThread } from "../src/engine.js";
import { createExtract } from "../src/extract-fn.js";
@@ -90,6 +91,7 @@ const refsDemoWorkflow = createWorkflow<RefsDemoMeta>(
extractPrompt: "Extract phases with CAS hashes.",
schema: plannerMetaSchema,
extractRefs: (meta) => meta.phases.map((p) => p.hash),
extractMode: "single",
},
},
moderator: (ctx) => (ctx.steps.length === 0 ? "planner" : END),
@@ -98,6 +100,7 @@ const refsDemoWorkflow = createWorkflow<RefsDemoMeta>(
agent: async () => "plan-output",
},
refsDemoExtract,
null,
);
describe("RoleStep refs tracking", () => {
@@ -110,8 +113,8 @@ describe("RoleStep refs tracking", () => {
test("parseThreadDataJsonl reads refs and defaults missing refs to []", () => {
const text = `{"name":"demo","hash":"C9NMV6V2TQT81","threadId":"01AAA1111111111111111111","parameters":{"prompt":"hi","options":{"maxRounds":5}},"timestamp":100}
{"role":"planner","content":"p","meta":{},"refs":["H111AAAAAAAAA","H222AAAAAAAAA"],"timestamp":101}
{"role":"coder","content":"c","meta":{},"timestamp":102}
{"role":"planner","contentHash":"HPAYLOAD111111","meta":{},"refs":["H111AAAAAAAAA","H222AAAAAAAAA"],"timestamp":101}
{"role":"coder","contentHash":"HPAYLOAD222222","meta":{},"timestamp":102}
`;
const r = parseThreadDataJsonl(text);
expect(r.ok).toBe(true);
@@ -139,6 +142,7 @@ describe("RoleStep refs tracking", () => {
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
await mkdir(join(root, "logs", hash), { recursive: true });
const cas = createCasStore(join(root, "cas"));
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
const ac = new AbortController();
@@ -149,16 +153,19 @@ describe("RoleStep refs tracking", () => {
{ prompt: "task", steps: [] },
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
prefilledDiskSteps: null,
},
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath },
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
logger,
);
expect(result.returnCode).toBe(0);
expect(typeof result.rootHash).toBe("string");
expect(result.rootHash.length).toBeGreaterThan(0);
const dataText = await readFile(dataPath, "utf8");
const lines = dataText
@@ -169,7 +176,12 @@ describe("RoleStep refs tracking", () => {
const role1 = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
expect(role1.role).toBe("planner");
expect(role1.refs).toEqual(["C9NMV6V2TQT81", "C9NMV6V2TQT82"]);
const refs = role1.refs as string[];
expect(refs).toContain("C9NMV6V2TQT81");
expect(refs).toContain("C9NMV6V2TQT82");
expect(typeof role1.contentHash).toBe("string");
expect(refs).toContain(String(role1.contentHash));
expect(refs.length).toBe(3);
} finally {
await rm(root, { recursive: true, force: true });
}
@@ -177,8 +189,8 @@ describe("RoleStep refs tracking", () => {
test("buildForkPlan carries refs on historical steps", () => {
const text = `{"name":"demo","hash":"C9NMV6V2TQT81","threadId":"01AAA1111111111111111111","parameters":{"prompt":"hi","options":{"maxRounds":5}},"timestamp":100}
{"role":"planner","content":"p","meta":{},"refs":["KEEPREFAAAAAA"],"timestamp":101}
{"role":"coder","content":"c","meta":{},"refs":["CODERHASHAAAA"],"timestamp":102}
{"role":"planner","contentHash":"HP111111111111","meta":{},"refs":["KEEPREFAAAAAA"],"timestamp":101}
{"role":"coder","contentHash":"HP222222222222","meta":{},"refs":["CODERHASHAAAA"],"timestamp":102}
`;
const plan = buildForkPlan(text, null);
expect(plan.ok).toBe(true);
+82 -1
View File
@@ -4,6 +4,7 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import {
parseWorkflowRegistryYaml,
readWorkflowRegistry,
registerWorkflowVersion,
rollbackWorkflowToHistoryHash,
@@ -21,6 +22,7 @@ describe("workflow registry", () => {
if (!empty.ok) {
return;
}
expect(empty.value.config).toBeNull();
const r1 = registerWorkflowVersion(empty.value, "solve-issue", "AAAAAAAAAAAAA", 100);
const w1 = await writeWorkflowRegistry(dir, r1);
@@ -68,7 +70,7 @@ describe("workflow registry", () => {
});
test("rollbackWorkflowToHistoryHash swaps head with a prior version", () => {
let reg = registerWorkflowVersion({ workflows: {} }, "solve-issue", "H1", 100);
let reg = registerWorkflowVersion({ config: null, workflows: {} }, "solve-issue", "H1", 100);
reg = registerWorkflowVersion(reg, "solve-issue", "H2", 200);
reg = registerWorkflowVersion(reg, "solve-issue", "H3", 300);
const entry = reg.workflows["solve-issue"];
@@ -99,6 +101,85 @@ describe("workflow registry", () => {
expect(bad.ok).toBe(false);
});
test("parses config section and literal apiKey", () => {
const yaml = `
config:
maxDepth: 3
extract:
baseUrl: https://example.com/v1
model: qwen-plus
apiKey: secret-key
workflows:
solve-issue:
hash: SPVR4BDMSGC1W
timestamp: 1
history: []
`;
const r = parseWorkflowRegistryYaml(yaml);
expect(r.ok).toBe(true);
if (!r.ok) {
return;
}
expect(r.value.config).not.toBeNull();
if (r.value.config === null) {
return;
}
expect(r.value.config.maxDepth).toBe(3);
expect(r.value.config.extract.baseUrl).toBe("https://example.com/v1");
expect(r.value.config.extract.model).toBe("qwen-plus");
expect(r.value.config.extract.apiKey).toBe("secret-key");
});
test("parses config apiKey env: prefix from process.env", () => {
const prev = process.env.WF_REGISTRY_TEST_API_KEY;
process.env.WF_REGISTRY_TEST_API_KEY = "from-env";
try {
const yaml = `
config:
maxDepth: 1
extract:
baseUrl: https://dashscope.aliyuncs.com/compatible-mode/v1
model: qwen-plus
apiKey: env:WF_REGISTRY_TEST_API_KEY
workflows: {}
`;
const r = parseWorkflowRegistryYaml(yaml);
expect(r.ok).toBe(true);
if (!r.ok) {
return;
}
expect(r.value.config?.extract.apiKey).toBe("from-env");
} finally {
if (prev === undefined) {
delete process.env.WF_REGISTRY_TEST_API_KEY;
} else {
process.env.WF_REGISTRY_TEST_API_KEY = prev;
}
}
});
test("parse errors when env: apiKey variable is unset", () => {
const prev = process.env.WF_REGISTRY_TEST_API_KEY_UNSET;
delete process.env.WF_REGISTRY_TEST_API_KEY_UNSET;
try {
const yaml = `
config:
maxDepth: 1
extract:
baseUrl: https://example.com
model: m
apiKey: env:WF_REGISTRY_TEST_API_KEY_UNSET
workflows: {}
`;
const r = parseWorkflowRegistryYaml(yaml);
expect(r.ok).toBe(false);
} finally {
if (prev !== undefined) {
process.env.WF_REGISTRY_TEST_API_KEY_UNSET = prev;
}
}
});
test("parse errors on invalid shape", async () => {
const dir = join(tmpdir(), `wf-reg3-${process.pid}-${Date.now()}`);
await mkdir(dir, { recursive: true });
@@ -10,6 +10,7 @@ describe("RFC-001 thread JSONL shapes", () => {
prompt: "Fix the login redirect bug in #3",
options: {
maxRounds: 5,
depth: 0,
},
},
timestamp: 1714963200000,
@@ -17,7 +18,7 @@ describe("RFC-001 thread JSONL shapes", () => {
const roleRecord = {
role: "planner",
content: "Plan: modify auth middleware...",
contentHash: "CPHASH000000000000000001",
meta: { plan: "...", files: ["src/auth.ts"] },
refs: [] as string[],
timestamp: 1714963201000,
@@ -27,7 +28,7 @@ describe("RFC-001 thread JSONL shapes", () => {
["hash", "name", "parameters", "threadId", "timestamp"].sort(),
);
expect(Object.keys(roleRecord).sort()).toEqual(
["content", "meta", "refs", "role", "timestamp"].sort(),
["contentHash", "meta", "refs", "role", "timestamp"].sort(),
);
});
+24 -9
View File
@@ -5,22 +5,29 @@ import { createConnection } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createCasStore } from "../src/cas.js";
import { createContentMerkleNode, serializeMerkleNode } from "../src/merkle.js";
import { getWorkerHostScriptPath } from "../src/worker-entry-path.js";
const bundleSource = `export const descriptor = {
const bundleSource = `import { putContentMerkleNode } from "@uncaged/workflow";
export const descriptor = {
description: "worker-test",
roles: {
planner: { description: "planner", schema: {} },
coder: { description: "coder", schema: {} },
},
};
export const run = async function* (input) {
export const run = async function* (input, options) {
const cas = options.cas;
const has = (r) => input.steps.some((s) => s.role === r);
if (!has("planner")) {
yield { role: "planner", content: "p", meta: { plan: input.prompt } };
const h = await putContentMerkleNode(cas, "p");
yield { role: "planner", contentHash: h, meta: { plan: input.prompt }, refs: [h] };
}
if (!has("coder")) {
yield { role: "coder", content: "c", meta: { diff: "y" } };
const h = await putContentMerkleNode(cas, "c");
yield { role: "coder", contentHash: h, meta: { diff: "y" }, refs: [h] };
}
return { returnCode: 0, summary: "completed: moderator returned END" };
};
@@ -102,7 +109,7 @@ describe("worker process", () => {
threadId,
workflowName: "demo-flow",
prompt: "hello",
options: { maxRounds: 5 },
options: { maxRounds: 5, depth: 0 },
});
const exitCode: number = await new Promise((resolve) => {
@@ -118,7 +125,7 @@ describe("worker process", () => {
.trim()
.split("\n")
.filter((l) => l !== "").length,
).toBe(3);
).toBe(4);
} finally {
await rm(root, { recursive: true, force: true });
}
@@ -143,6 +150,11 @@ describe("worker process", () => {
const port = await readReadyPort(child);
const cas = createCasStore(join(root, "cas"));
const plannerReplayHash = await cas.put(
serializeMerkleNode(createContentMerkleNode("p-old")),
);
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
const srcId = "01SRCMMMMMMMMMMMMMMMMMMMM";
await sendJson(port, {
@@ -150,12 +162,13 @@ describe("worker process", () => {
threadId,
workflowName: "demo-flow",
prompt: "hello",
options: { maxRounds: 5 },
options: { maxRounds: 5, depth: 0 },
steps: [
{
role: "planner",
content: "p-old",
contentHash: plannerReplayHash,
meta: { plan: "z" },
refs: [plannerReplayHash],
timestamp: 555,
},
],
@@ -174,7 +187,7 @@ describe("worker process", () => {
.trim()
.split("\n")
.filter((l) => l !== "");
expect(lines.length).toBe(3);
expect(lines.length).toBe(4);
const start = JSON.parse(lines[0] ?? "{}") as Record<string, unknown>;
expect(start.forkFrom).toEqual({ threadId: srcId });
const replay = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
@@ -182,6 +195,8 @@ describe("worker process", () => {
expect(replay.timestamp).toBe(555);
const coder = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
expect(coder.role).toBe("coder");
const done = JSON.parse(lines[3] ?? "{}") as Record<string, unknown>;
expect(done.returnCode).toBe(0);
} finally {
await rm(root, { recursive: true, force: true });
}
@@ -0,0 +1,225 @@
import { afterEach, describe, expect, test } from "bun:test";
import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import * as z from "zod/v4";
import { createCasStore } from "../src/cas.js";
import { createWorkflow } from "../src/create-workflow.js";
import { executeThread } from "../src/engine.js";
import { createExtract } from "../src/extract-fn.js";
import { hashWorkflowBundleBytes } from "../src/hash.js";
import { createLogger } from "../src/logger.js";
import { getContentMerklePayload, parseMerkleNode } from "../src/merkle.js";
import {
readWorkflowRegistry,
registerWorkflowVersion,
writeWorkflowRegistry,
} from "../src/registry.js";
import { END } from "../src/types.js";
import { workflowAsAgent } from "../src/workflow-as-agent.js";
const callerMetaSchema = z.object({ done: z.literal(true) });
type ParentMeta = {
caller: z.infer<typeof callerMetaSchema>;
};
function installMockChatCompletions(sequence: ReadonlyArray<Record<string, unknown>>): () => void {
const origFetch = globalThis.fetch;
let i = 0;
const mockFetch = async (
input: Parameters<typeof fetch>[0],
init?: RequestInit,
): Promise<Response> => {
const args = sequence[i] ?? sequence[sequence.length - 1];
if (args === undefined) {
throw new Error("installMockChatCompletions: empty sequence");
}
i += 1;
void input;
const body = init?.body ? (JSON.parse(String(init.body)) as Record<string, unknown>) : {};
const tools = body.tools;
const firstTool =
Array.isArray(tools) && tools.length > 0 && tools[0] !== null && typeof tools[0] === "object"
? (tools[0] as Record<string, unknown>)
: null;
const fn =
firstTool !== null ? (firstTool.function as Record<string, unknown> | undefined) : undefined;
const toolName = typeof fn?.name === "string" ? fn.name : "extract";
return new Response(
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
type: "function",
function: {
name: toolName,
arguments: JSON.stringify(args),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
};
globalThis.fetch = Object.assign(mockFetch, {
preconnect: origFetch.preconnect.bind(origFetch),
}) as typeof fetch;
return () => {
globalThis.fetch = origFetch;
};
}
const parentExtract = createExtract({
baseUrl: "http://127.0.0.1:9",
apiKey: "test",
model: "test",
});
const childBundleSource = `import { putContentMerkleNode } from "@uncaged/workflow";
export const descriptor = {
description: "child-integration",
roles: {
agent: {
description: "agent",
schema: { type: "object", properties: {}, additionalProperties: true },
},
},
};
export async function* run(input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "child-body");
yield { role: "agent", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "child-done:" + input.prompt };
}
`;
async function installChildWorkflow(storageRoot: string): Promise<{ hash: string }> {
const bytes = new TextEncoder().encode(childBundleSource);
const hash = hashWorkflowBundleBytes(bytes);
await mkdir(join(storageRoot, "bundles"), { recursive: true });
await writeFile(join(storageRoot, "bundles", `${hash}.esm.js`), childBundleSource, "utf8");
const reg = await readWorkflowRegistry(storageRoot);
if (!reg.ok) {
throw reg.error;
}
const next = registerWorkflowVersion(reg.value, "child-wf", hash, Date.now());
const wr = await writeWorkflowRegistry(storageRoot, next);
if (!wr.ok) {
throw wr.error;
}
return { hash };
}
describe("workflowAsAgent integration", () => {
let restoreFetch: (() => void) | null = null;
afterEach(() => {
restoreFetch?.();
restoreFetch = null;
});
test("createWorkflow parent invokes nested workflow via workflowAsAgent", async () => {
restoreFetch = installMockChatCompletions([{ done: true }]);
const root = await mkdtemp(join(tmpdir(), "wf-waa-int-"));
try {
const { hash: childHash } = await installChildWorkflow(root);
const parentWorkflow = createWorkflow<ParentMeta>(
{
roles: {
caller: {
description: "delegates to child workflow",
systemPrompt: "system",
extractPrompt: "extract done flag",
schema: callerMetaSchema,
extractRefs: null,
extractMode: "single",
},
},
moderator: (ctx) => (ctx.steps.length === 0 ? "caller" : END),
},
{ agent: workflowAsAgent("child-wf", { storageRoot: root }) },
parentExtract,
null,
);
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
const parentHash = "C9NMV6V2TQT81";
const dataPath = join(root, "logs", parentHash, `${threadId}.data.jsonl`);
const infoPath = join(root, "logs", parentHash, `${threadId}.info.jsonl`);
await mkdir(join(root, "logs", parentHash), { recursive: true });
const cas = createCasStore(join(root, "cas"));
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
const ac = new AbortController();
const result = await executeThread(
parentWorkflow,
"parent-wf",
{ prompt: "from-parent", steps: [] },
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
prefilledDiskSteps: null,
},
{ threadId, hash: parentHash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
logger,
);
expect(result.returnCode).toBe(0);
expect(typeof result.rootHash).toBe("string");
const parentText = await readFile(dataPath, "utf8");
const parentLines = parentText
.trim()
.split("\n")
.filter((l) => l !== "");
expect(parentLines.length).toBe(2);
const callerLine = JSON.parse(parentLines[1] ?? "{}") as Record<string, unknown>;
expect(callerLine.role).toBe("caller");
const childRootHash = await getContentMerklePayload(cas, String(callerLine.contentHash));
expect(childRootHash).not.toBeNull();
const childThreadYaml = await cas.get(childRootHash ?? "");
expect(childThreadYaml).not.toBeNull();
const childThreadNode = parseMerkleNode(childThreadYaml ?? "");
expect(childThreadNode.type).toBe("thread");
const childPayload = childThreadNode.payload as Record<string, unknown>;
expect(childPayload.workflow).toBe("child-wf");
const childResult = childPayload.result as Record<string, unknown>;
expect(childResult.summary).toBe("child-done:from-parent");
const childDir = join(root, "logs", childHash);
const childFiles = await readdir(childDir);
const childDataName = childFiles.find((n) => n.endsWith(".data.jsonl"));
expect(childDataName).toBeDefined();
const childText = await readFile(join(childDir, childDataName ?? ""), "utf8");
const childStart = JSON.parse(
childText
.trim()
.split("\n")
.filter((l) => l !== "")[0] ?? "{}",
) as Record<string, unknown>;
expect(childStart.forkFrom).toEqual({ threadId });
const childOpts = (childStart.parameters as Record<string, unknown>).options as Record<
string,
unknown
>;
expect(childOpts.depth).toBe(1);
} finally {
await rm(root, { recursive: true, force: true });
}
});
});
@@ -0,0 +1,168 @@
import { describe, expect, test } from "bun:test";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createCasStore } from "../src/cas.js";
import { hashWorkflowBundleBytes } from "../src/hash.js";
import { parseMerkleNode } from "../src/merkle.js";
import {
readWorkflowRegistry,
registerWorkflowVersion,
writeWorkflowRegistry,
} from "../src/registry.js";
import { type AgentContext, START } from "../src/types.js";
import { workflowAsAgent } from "../src/workflow-as-agent.js";
function makeAgentCtx(params: {
storageRoot: string;
depth: number;
prompt: string;
maxRounds: number;
}): AgentContext {
const ts = Date.now();
return {
threadId: "01PARENT000000000000000001AA",
depth: params.depth,
start: {
role: START,
content: params.prompt,
meta: { maxRounds: params.maxRounds },
timestamp: ts,
},
steps: [],
currentRole: {
name: "caller",
systemPrompt: "caller",
},
cas: createCasStore(join(params.storageRoot, "agent-ctx-cas")),
};
}
const childBundleSource = `import { putContentMerkleNode } from "@uncaged/workflow";
export const descriptor = {
description: "child-test",
roles: {
agent: {
description: "agent",
schema: { type: "object", properties: {}, additionalProperties: true },
},
},
};
export async function* run(input, options) {
const cas = options.cas;
const h = await putContentMerkleNode(cas, "child-body");
yield { role: "agent", contentHash: h, meta: {}, refs: [h] };
return { returnCode: 0, summary: "child-done:" + input.prompt };
}
`;
async function installChildWorkflow(storageRoot: string): Promise<{ hash: string }> {
const bytes = new TextEncoder().encode(childBundleSource);
const hash = hashWorkflowBundleBytes(bytes);
await mkdir(join(storageRoot, "bundles"), { recursive: true });
await writeFile(join(storageRoot, "bundles", `${hash}.esm.js`), childBundleSource, "utf8");
const reg = await readWorkflowRegistry(storageRoot);
if (!reg.ok) {
throw reg.error;
}
const next = registerWorkflowVersion(reg.value, "child-wf", hash, Date.now());
const wr = await writeWorkflowRegistry(storageRoot, next);
if (!wr.ok) {
throw wr.error;
}
return { hash };
}
describe("workflowAsAgent", () => {
test("returns error when workflow name is not registered", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-waa-missing-"));
try {
const agent = workflowAsAgent("missing-wf", { storageRoot: root });
const out = await agent(
makeAgentCtx({ storageRoot: root, depth: 0, prompt: "x", maxRounds: 5 }),
);
expect(out).toContain("not found in registry");
expect(out).toContain("missing-wf");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("runs registered workflow and returns child thread root CAS hash", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-waa-ok-"));
try {
await installChildWorkflow(root);
const agent = workflowAsAgent("child-wf", { storageRoot: root });
const out = await agent(
makeAgentCtx({ storageRoot: root, depth: 0, prompt: "hello-parent", maxRounds: 5 }),
);
const cas = createCasStore(join(root, "cas"));
const threadYaml = await cas.get(out);
expect(threadYaml).not.toBeNull();
const node = parseMerkleNode(threadYaml ?? "");
expect(node.type).toBe("thread");
const payload = node.payload as Record<string, unknown>;
expect(payload.workflow).toBe("child-wf");
const resultObj = payload.result as Record<string, unknown>;
expect(resultObj.summary).toBe("child-done:hello-parent");
expect(node.children.length).toBe(1);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("enforces depth limit (returns error string, does not throw)", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-waa-depth-"));
try {
const agent = workflowAsAgent("child-wf", { storageRoot: root });
const out = await agent(
makeAgentCtx({ storageRoot: root, depth: 3, prompt: "x", maxRounds: 5 }),
);
expect(out).toContain("depth limit");
expect(out).toContain("max 3");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("uses registry config maxDepth when set", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-waa-maxdepth-cfg-"));
try {
await installChildWorkflow(root);
const reg = await readWorkflowRegistry(root);
expect(reg.ok).toBe(true);
if (!reg.ok) {
return;
}
const withCfg = {
...reg.value,
config: {
maxDepth: 2,
extract: {
baseUrl: "http://127.0.0.1:9",
model: "m",
apiKey: "k",
},
},
};
const wr = await writeWorkflowRegistry(root, withCfg);
expect(wr.ok).toBe(true);
const agent = workflowAsAgent("child-wf", { storageRoot: root });
const okOut = await agent(
makeAgentCtx({ storageRoot: root, depth: 1, prompt: "nest-once", maxRounds: 5 }),
);
expect(okOut).not.toContain("depth limit");
const badOut = await agent(
makeAgentCtx({ storageRoot: root, depth: 2, prompt: "x", maxRounds: 5 }),
);
expect(badOut).toContain("depth limit");
expect(badOut).toContain("max 2");
} finally {
await rm(root, { recursive: true, force: true });
}
});
});
@@ -0,0 +1,8 @@
import { pathToFileURL } from "node:url";
/**
* Dynamic-import a workflow bundle path (see {@link extractBundleExports} — symlink must exist first).
*/
export async function importWorkflowBundleModule(bundlePath: string): Promise<unknown> {
return import(pathToFileURL(bundlePath).href);
}
+7 -4
View File
@@ -41,9 +41,12 @@ function isAllowedImportSpecifier(spec: string): boolean {
if (spec.length === 0) {
return false;
}
if (spec.startsWith(".") || spec.startsWith("/")) {
if (spec.startsWith(".") || spec.startsWith("/") || spec.startsWith("file:")) {
return false;
}
if (spec === "@uncaged/workflow") {
return true;
}
return isBuiltin(spec);
}
@@ -297,7 +300,7 @@ function validateImportDeclaration(node: ImportDeclaration): string | null {
return "only static string import specifiers are allowed";
}
if (!isAllowedImportSpecifier(spec)) {
return `disallowed import specifier "${spec}" (only Node built-ins are allowed)`;
return `disallowed import specifier "${spec}" (only Node built-ins and "@uncaged/workflow" are allowed)`;
}
return null;
}
@@ -312,7 +315,7 @@ function validateExportSource(
return staticMessage;
}
if (!isAllowedImportSpecifier(spec)) {
return `${disallowedPrefix} "${spec}" (only Node built-ins are allowed)`;
return `${disallowedPrefix} "${spec}" (only Node built-ins and "@uncaged/workflow" are allowed)`;
}
return null;
}
@@ -365,7 +368,7 @@ function bundleConstraintViolationForNode(node: Node): string | null {
/**
* Validate RFC-001 bundle rules: single-file ESM shape, named exports `run` + `descriptor`,
* no default export, no dynamic `import()`, static imports restricted to Node builtins.
* no default export, no dynamic `import()`, static imports restricted to Node builtins plus `@uncaged/workflow`.
*/
export function validateWorkflowBundle(input: WorkflowBundleValidationInput): Result<void, string> {
if (!endsWithEsmJs(input.filePath)) {
+67 -14
View File
@@ -1,9 +1,14 @@
import type { ExtractFn } from "./extract-fn.js";
import type { CasStore } from "./cas.js";
import { buildExtractUserContent, type ExtractFn } from "./extract-fn.js";
import { putContentMerkleNode } from "./merkle.js";
import { reactExtract } from "./react-extract.js";
import { mergeRefsWithContentHash } from "./refs-field.js";
import {
type AgentBinding,
type AgentContext,
END,
type ExtractContext,
type LlmProvider,
type ModeratorContext,
type RoleDefinition,
type RoleMeta,
@@ -11,10 +16,10 @@ import {
type RoleStep,
START,
type ThreadInput,
type WorkflowCompletion,
type WorkflowDefinition,
type WorkflowFn,
type WorkflowFnOptions,
type WorkflowResult,
} from "./types.js";
function isRoleNext<M extends RoleMeta>(
@@ -34,19 +39,56 @@ function resolveExtractedRefs(
return extractRefsFn(meta as Record<string, unknown>);
}
async function resolveRoleMeta<M extends RoleMeta>(
roleDef: RoleDefinition<Record<string, unknown>>,
extractCtx: ExtractContext<M>,
extract: ExtractFn,
llmProvider: LlmProvider | null,
cas: CasStore,
): Promise<Record<string, unknown>> {
if (roleDef.extractMode === "react") {
if (llmProvider === null) {
throw new Error(
'createWorkflow: llmProvider is required when a role uses extractMode "react"',
);
}
const text = await buildExtractUserContent(
extractCtx as unknown as ExtractContext,
roleDef.extractPrompt,
);
const reactResult = await reactExtract({
text,
schema: roleDef.schema,
provider: llmProvider,
cas,
});
if (!reactResult.ok) {
throw new Error(`react extract failed: ${reactResult.error}`);
}
return reactResult.value as Record<string, unknown>;
}
return (await extract(
roleDef.schema,
roleDef.extractPrompt,
extractCtx as unknown as ExtractContext,
)) as Record<string, unknown>;
}
/**
* Binds pure role definitions + moderator to runtime agents and structured extraction.
* Assign with `export const run = createWorkflow(def, binding, extract)`.
* Assign with `export const run = createWorkflow(def, binding, extract, llmProvider)`.
* Pass the same {@link LlmProvider} as {@link createExtract} when any role uses `extractMode: "react"`.
*/
export function createWorkflow<M extends RoleMeta>(
def: Pick<WorkflowDefinition<M>, "roles" | "moderator">,
binding: AgentBinding,
extract: ExtractFn,
llmProvider: LlmProvider | null,
): WorkflowFn {
return async function* workflowLoop(
input: ThreadInput,
options: WorkflowFnOptions,
): AsyncGenerator<RoleOutput, WorkflowResult> {
): AsyncGenerator<RoleOutput, WorkflowCompletion> {
const nowMs = Date.now();
const start: ModeratorContext<M>["start"] = {
role: START,
@@ -58,7 +100,7 @@ export function createWorkflow<M extends RoleMeta>(
const baseTs = Date.now();
let steps: RoleStep<M>[] = input.steps.map((out, i) => ({
role: out.role,
content: out.content,
contentHash: out.contentHash,
meta: out.meta,
refs: out.refs,
timestamp: baseTs + i,
@@ -74,6 +116,7 @@ export function createWorkflow<M extends RoleMeta>(
const modCtx: ModeratorContext<M> = {
threadId: options.threadId,
depth: options.depth,
start,
steps,
};
@@ -92,6 +135,7 @@ export function createWorkflow<M extends RoleMeta>(
const agentCtx: AgentContext<M> = {
...modCtx,
currentRole: { name: next, systemPrompt: roleDef.systemPrompt },
cas: options.cas,
};
const agent = binding.overrides?.[next] ?? binding.agent;
@@ -103,27 +147,36 @@ export function createWorkflow<M extends RoleMeta>(
agentContent: raw,
};
const meta = await extract(
roleDef.schema,
roleDef.extractPrompt,
extractCtx as unknown as ExtractContext,
const meta = await resolveRoleMeta(
roleDef as unknown as RoleDefinition<Record<string, unknown>>,
extractCtx,
extract,
llmProvider,
options.cas,
);
const refs = resolveExtractedRefs(
roleDef as unknown as RoleDefinition<Record<string, unknown>>,
meta,
const contentHash = await putContentMerkleNode(options.cas, raw);
const refs = mergeRefsWithContentHash(
resolveExtractedRefs(roleDef as unknown as RoleDefinition<Record<string, unknown>>, meta),
contentHash,
);
const ts = Date.now();
const step = {
role: next,
content: raw,
contentHash,
meta,
refs,
timestamp: ts,
} as RoleStep<M>;
yield { role: step.role, content: step.content, meta: step.meta, refs: step.refs };
yield {
role: step.role,
contentHash: step.contentHash,
meta: step.meta,
refs: step.refs,
};
steps = [...steps, step];
}
+131 -16
View File
@@ -1,21 +1,30 @@
import { appendFile, mkdir } from "node:fs/promises";
import { dirname } from "node:path";
import type { CasStore } from "./cas.js";
import type { LogFn } from "./logger.js";
import { getContentMerklePayload, putStepMerkleNode, putThreadMerkleNode } from "./merkle.js";
import { normalizeRefsField } from "./refs-field.js";
import type { ThreadInput, WorkflowFn, WorkflowFnOptions, WorkflowResult } from "./types.js";
import type {
ThreadInput,
WorkflowCompletion,
WorkflowFn,
WorkflowFnOptions,
WorkflowResult,
} from "./types.js";
export type ExecuteThreadIo = {
threadId: string;
hash: string;
dataJsonlPath: string;
infoJsonlPath: string;
cas: CasStore;
};
/** One persisted role line in `.data.jsonl` (engine adds these for fork replay before running the generator). */
export type PrefilledDiskStep = {
role: string;
content: string;
contentHash: string;
meta: Record<string, unknown>;
refs: string[];
timestamp: number;
@@ -23,6 +32,8 @@ export type PrefilledDiskStep = {
export type ExecuteThreadOptions = {
maxRounds: number;
/** Passed to the bundle as `WorkflowFnOptions.depth`. */
depth: number;
signal: AbortSignal;
/** Invoked after each successful yield (and outer-loop checks); used for pause/resume. */
awaitAfterEachYield: () => Promise<void>;
@@ -40,51 +51,123 @@ async function appendDataLine(path: string, record: unknown): Promise<void> {
await appendFile(path, line, "utf8");
}
async function finalizeThreadResult(params: {
cas: CasStore;
workflowName: string;
threadId: string;
stepMerkleHashes: readonly string[];
completion: WorkflowCompletion;
}): Promise<WorkflowResult> {
const rootHash = await putThreadMerkleNode(
params.cas,
{
workflow: params.workflowName,
threadId: params.threadId,
result: {
returnCode: params.completion.returnCode,
summary: params.completion.summary,
},
},
params.stepMerkleHashes,
);
return {
returnCode: params.completion.returnCode,
summary: params.completion.summary,
rootHash,
};
}
async function driveWorkflowGenerator(params: {
fn: WorkflowFn;
workflowName: string;
input: ThreadInput;
bundleOptions: WorkflowFnOptions;
executeOptions: ExecuteThreadOptions;
dataJsonlPath: string;
threadId: string;
logger: LogFn;
cas: CasStore;
stepMerkleHashes: string[];
}): Promise<WorkflowResult> {
const { fn, input, bundleOptions, executeOptions, dataJsonlPath, threadId, logger } = params;
const {
fn,
workflowName,
input,
bundleOptions,
executeOptions,
dataJsonlPath,
threadId,
logger,
cas,
stepMerkleHashes,
} = params;
const gen = fn(input, bundleOptions);
let written = 0;
while (true) {
if (executeOptions.signal.aborted) {
logger("V8JX4NP2", `thread ${threadId} aborted`);
return { returnCode: 130, summary: "thread aborted" };
return await finalizeThreadResult({
cas,
workflowName,
threadId,
stepMerkleHashes,
completion: { returnCode: 130, summary: "thread aborted" },
});
}
if (written >= executeOptions.maxRounds) {
logger("R3CW7YBQ", `thread ${threadId} stopped at maxRounds=${executeOptions.maxRounds}`);
return {
returnCode: 0,
summary: `completed: reached maxRounds (${executeOptions.maxRounds})`,
};
return await finalizeThreadResult({
cas,
workflowName,
threadId,
stepMerkleHashes,
completion: {
returnCode: 0,
summary: `completed: reached maxRounds (${executeOptions.maxRounds})`,
},
});
}
const iterResult = await gen.next();
if (iterResult.done) {
logger("F3HN8QKP", `thread ${threadId} generator finished`);
return iterResult.value;
const completion = iterResult.value;
return await finalizeThreadResult({
cas,
workflowName,
threadId,
stepMerkleHashes,
completion,
});
}
written++;
const step = iterResult.value;
const resolved = await getContentMerklePayload(cas, step.contentHash);
if (resolved === null) {
throw new Error(
`role step ${step.role}: CAS blob missing for contentHash ${step.contentHash}`,
);
}
const ts = Date.now();
await appendDataLine(dataJsonlPath, {
role: step.role,
content: step.content,
contentHash: step.contentHash,
meta: step.meta,
refs: normalizeRefsField(step.refs),
timestamp: ts,
});
const stepNodeHash = await putStepMerkleNode(
cas,
{ role: step.role, meta: step.meta },
step.contentHash,
);
stepMerkleHashes.push(stepNodeHash);
logger("N7BW4YHQ", `thread ${threadId} wrote role ${step.role}`);
await Promise.race([
@@ -100,7 +183,13 @@ async function driveWorkflowGenerator(params: {
if (executeOptions.signal.aborted) {
logger("V8JX4NP4", `thread ${threadId} aborted`);
return { returnCode: 130, summary: "thread aborted" };
return await finalizeThreadResult({
cas,
workflowName,
threadId,
stepMerkleHashes,
completion: { returnCode: 130, summary: "thread aborted" },
});
}
}
}
@@ -136,6 +225,7 @@ export async function executeThread(
prompt: input.prompt,
options: {
maxRounds: options.maxRounds,
depth: options.depth,
},
},
timestamp: nowMs,
@@ -148,38 +238,63 @@ export async function executeThread(
logger("T9HQ2KHM", `thread ${io.threadId} started for workflow ${workflowName}`);
const stepMerkleHashes: string[] = [];
if (prefilled !== null) {
for (const row of prefilled) {
const prefilledPayload = await getContentMerklePayload(io.cas, row.contentHash);
if (prefilledPayload === null) {
throw new Error(
`prefilled step ${row.role}: CAS blob missing for contentHash ${row.contentHash}`,
);
}
await appendDataLine(io.dataJsonlPath, {
role: row.role,
content: row.content,
contentHash: row.contentHash,
meta: row.meta,
refs: normalizeRefsField(row.refs),
timestamp: row.timestamp,
});
const stepNodeHash = await putStepMerkleNode(
io.cas,
{ role: row.role, meta: row.meta },
row.contentHash,
);
stepMerkleHashes.push(stepNodeHash);
}
}
if (options.maxRounds <= 0) {
logger("R3CW7YBQ", `thread ${io.threadId} stopped at maxRounds=${options.maxRounds}`);
return {
returnCode: 0,
summary: `completed: reached maxRounds (${options.maxRounds})`,
};
return await finalizeThreadResult({
cas: io.cas,
workflowName,
threadId: io.threadId,
stepMerkleHashes,
completion: {
returnCode: 0,
summary: `completed: reached maxRounds (${options.maxRounds})`,
},
});
}
const bundleOptions: WorkflowFnOptions = {
threadId: io.threadId,
maxRounds: options.maxRounds,
depth: options.depth,
cas: io.cas,
};
return await driveWorkflowGenerator({
fn,
workflowName,
input,
bundleOptions,
executeOptions: options,
dataJsonlPath: io.dataJsonlPath,
threadId: io.threadId,
logger,
cas: io.cas,
stepMerkleHashes,
});
}
@@ -0,0 +1,36 @@
import { mkdir, readlink, symlink, unlink } from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
/** This module lives in `@uncaged/workflow/src`; parent dir is the package root. */
function installedWorkflowPackageDir(): string {
return fileURLToPath(new URL("..", import.meta.url));
}
/**
* Ensures `<storageRoot>/node_modules/@uncaged/workflow` points at the installed `@uncaged/workflow`
* package so workflow bundles loaded from `<storageRoot>/bundles/*.esm.js` can resolve `import "@uncaged/workflow"`.
*/
export async function ensureUncagedWorkflowSymlink(storageRoot: string): Promise<void> {
const target = installedWorkflowPackageDir();
const linkDir = path.join(storageRoot, "node_modules", "@uncaged");
const linkPath = path.join(linkDir, "workflow");
await mkdir(linkDir, { recursive: true });
try {
const existing = await readlink(linkPath);
const normalizedExisting = path.resolve(linkDir, existing);
if (normalizedExisting === target) {
return;
}
await unlink(linkPath);
} catch (e) {
const errObj = e as NodeJS.ErrnoException;
if (errObj.code !== "ENOENT" && errObj.code !== "EINVAL") {
throw e;
}
}
const linkType = process.platform === "win32" ? "junction" : "dir";
await symlink(target, linkPath, linkType);
}
@@ -1,5 +1,5 @@
import { pathToFileURL } from "node:url";
import { importWorkflowBundleModule } from "./bundle-import-env.js";
import { ensureUncagedWorkflowSymlink } from "./ensure-uncaged-workflow-symlink.js";
import { err, ok, type Result } from "./result.js";
import type { WorkflowFn } from "./types.js";
import type { WorkflowDescriptor } from "./workflow-descriptor.js";
@@ -10,14 +10,23 @@ export type ExtractedBundleExports = {
descriptor: WorkflowDescriptor;
};
export type ExtractBundleExportsOptions = {
/** When set, ensures `node_modules/@uncaged/workflow` exists under this root before import. */
storageRoot: string | null;
};
/** Load a workflow `.esm.js` bundle and read its named exports (`run`, `descriptor`). */
export async function extractBundleExports(
bundlePath: string,
options: ExtractBundleExportsOptions = { storageRoot: null },
): Promise<Result<ExtractedBundleExports, string>> {
let modUnknown: unknown;
try {
if (options.storageRoot !== null) {
await ensureUncagedWorkflowSymlink(options.storageRoot);
}
// Dynamic import required: user bundle path resolved at runtime
modUnknown = await import(pathToFileURL(bundlePath).href);
modUnknown = await importWorkflowBundleModule(bundlePath);
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
return err(`failed to import bundle: ${message}`);
+36 -23
View File
@@ -1,6 +1,7 @@
import type * as z from "zod/v4";
import { llmExtractWithRetry } from "./llm-extract.js";
import { getContentMerklePayload } from "./merkle.js";
import type { ExtractContext, LlmProvider } from "./types.js";
export type ExtractFn = <T extends Record<string, unknown>>(
@@ -9,6 +10,40 @@ export type ExtractFn = <T extends Record<string, unknown>>(
ctx: ExtractContext,
) => Promise<T>;
/** Builds the user-side extraction prompt (thread + agent output + instruction). */
export async function buildExtractUserContent(
ctx: ExtractContext,
prompt: string,
): Promise<string> {
const lines: string[] = [];
lines.push(`## Role: ${ctx.currentRole.name}`);
lines.push(ctx.currentRole.systemPrompt);
lines.push("");
lines.push("## Task");
lines.push(ctx.start.content);
lines.push("");
if (ctx.steps.length > 0) {
lines.push("## Thread History");
for (const step of ctx.steps) {
const body = await getContentMerklePayload(ctx.cas, step.contentHash);
if (body === null) {
throw new Error(`extract: missing CAS blob for step ${step.role}: ${step.contentHash}`);
}
lines.push(`### ${step.role}`);
lines.push(body);
lines.push(`Meta: ${JSON.stringify(step.meta)}`);
lines.push("");
}
}
lines.push("## Agent Output");
lines.push(ctx.agentContent);
lines.push("");
lines.push("## Extraction Instruction");
lines.push(prompt);
return lines.join("\n");
}
/**
* Create an ExtractFn backed by an LLM provider.
* Builds prompt text from {@link ExtractContext} plus `prompt` and calls structured extraction.
@@ -19,29 +54,7 @@ export function createExtract(provider: LlmProvider): ExtractFn {
prompt: string,
ctx: ExtractContext,
): Promise<T> => {
const lines: string[] = [];
lines.push(`## Role: ${ctx.currentRole.name}`);
lines.push(ctx.currentRole.systemPrompt);
lines.push("");
lines.push("## Task");
lines.push(ctx.start.content);
lines.push("");
if (ctx.steps.length > 0) {
lines.push("## Thread History");
for (const step of ctx.steps) {
lines.push(`### ${step.role}`);
lines.push(step.content);
lines.push(`Meta: ${JSON.stringify(step.meta)}`);
lines.push("");
}
}
lines.push("## Agent Output");
lines.push(ctx.agentContent);
lines.push("");
lines.push("## Extraction Instruction");
lines.push(prompt);
const text = lines.join("\n");
const text = await buildExtractUserContent(ctx, prompt);
const result = await llmExtractWithRetry({ text, schema, provider });
if (!result.ok) {
throw new Error(`extract failed: ${JSON.stringify(result.error)}`);
+35
View File
@@ -0,0 +1,35 @@
import { readWorkflowRegistry } from "./registry.js";
import type { WorkflowConfig } from "./registry-types.js";
import { err, ok, type Result } from "./result.js";
import { getDefaultWorkflowStorageRoot } from "./storage-root.js";
import type { LlmProvider } from "./types.js";
const DEFAULT_WORKFLOW_AS_AGENT_MAX_DEPTH = 3;
export function getWorkflowAsAgentMaxDepth(config: WorkflowConfig | null): number {
if (config === null) {
return DEFAULT_WORKFLOW_AS_AGENT_MAX_DEPTH;
}
return config.maxDepth;
}
/** Loads `config.extract` from workflow.yaml (apiKey already resolved at registry parse time). */
export async function getExtractProvider(
storageRoot: string | undefined,
): Promise<Result<LlmProvider, string>> {
const root = storageRoot ?? getDefaultWorkflowStorageRoot();
const regResult = await readWorkflowRegistry(root);
if (!regResult.ok) {
return err(regResult.error.message);
}
const cfg = regResult.value.config;
if (cfg === null) {
return err("workflow registry has no global config section");
}
const ex = cfg.extract;
return ok({
baseUrl: ex.baseUrl,
apiKey: ex.apiKey,
model: ex.model,
});
}
+63 -26
View File
@@ -1,6 +1,6 @@
import { normalizeRefsField } from "./refs-field.js";
import { err, ok, type Result } from "./result.js";
import type { RoleOutput } from "./types.js";
import type { RoleOutput, WorkflowCompletion } from "./types.js";
/** Role steps replayed from `.data.jsonl`, including persisted timestamps. */
export type ForkHistoricalStep = RoleOutput & { timestamp: number };
@@ -11,35 +11,59 @@ export type ParsedThreadStartRecord = {
threadId: string;
prompt: string;
maxRounds: number;
depth: number;
};
/** Recognizes a persisted workflow completion line (no `role`; has numeric `returnCode` and string `summary`). Omits `rootHash` when absent. */
export function tryParseWorkflowResultRecord(
obj: Record<string, unknown>,
): WorkflowCompletion | null {
if (obj.role !== undefined) {
return null;
}
const returnCode = obj.returnCode;
const summary = obj.summary;
if (typeof returnCode !== "number" || typeof summary !== "string") {
return null;
}
return { returnCode, summary };
}
export function tryParseRoleStepRecord(obj: Record<string, unknown>): ForkHistoricalStep | null {
const role = obj.role;
const contentHash = obj.contentHash;
const meta = obj.meta;
const timestamp = obj.timestamp;
if (typeof role !== "string") {
return null;
}
if (typeof contentHash !== "string") {
return null;
}
if (meta === null || typeof meta !== "object") {
return null;
}
if (typeof timestamp !== "number") {
return null;
}
return {
role,
contentHash,
meta: meta as Record<string, unknown>,
refs: normalizeRefsField(obj.refs),
timestamp,
};
}
function parseRoleLine(
obj: Record<string, unknown>,
lineIndex: number,
): Result<ForkHistoricalStep, string> {
const role = obj.role;
const content = obj.content;
const meta = obj.meta;
const timestamp = obj.timestamp;
if (typeof role !== "string") {
return err(`invalid role record at line ${lineIndex}: missing role`);
const parsed = tryParseRoleStepRecord(obj);
if (parsed === null) {
return err(`invalid role record at line ${lineIndex}`);
}
if (typeof content !== "string") {
return err(`invalid role record at line ${lineIndex}: missing content`);
}
if (meta === null || typeof meta !== "object") {
return err(`invalid role record at line ${lineIndex}: missing meta`);
}
if (typeof timestamp !== "number") {
return err(`invalid role record at line ${lineIndex}: missing timestamp`);
}
return ok({
role,
content,
meta: meta as Record<string, unknown>,
refs: normalizeRefsField(obj.refs),
timestamp,
});
return ok(parsed);
}
function parseStartRecordLine(firstLine: string): Result<ParsedThreadStartRecord, string> {
@@ -78,12 +102,17 @@ function parseStartRecordLine(firstLine: string): Result<ParsedThreadStartRecord
return err("start record missing parameters.options.maxRounds");
}
const depthRaw = optRec.depth;
const depth =
typeof depthRaw === "number" && Number.isFinite(depthRaw) ? Math.trunc(depthRaw) : 0;
return ok({
workflowName: name,
hash,
threadId,
prompt,
maxRounds,
depth,
});
}
@@ -103,7 +132,15 @@ function parseFollowingRoleLines(lines: string[]): Result<ForkHistoricalStep[],
if (rec === null || typeof rec !== "object") {
return err(`invalid record at line ${i + 1}`);
}
const parsed = parseRoleLine(rec as Record<string, unknown>, i + 1);
const recObj = rec as Record<string, unknown>;
const wf = tryParseWorkflowResultRecord(recObj);
if (wf !== null) {
if (i !== lines.length - 1) {
return err("WorkflowResult record must be the final line in `.data.jsonl`");
}
break;
}
const parsed = parseRoleLine(recObj, i + 1);
if (!parsed.ok) {
return parsed;
}
@@ -196,7 +233,7 @@ export type ForkPlan = {
hash: string;
sourceThreadId: string;
prompt: string;
runOptions: { maxRounds: number };
runOptions: { maxRounds: number; depth: number };
historicalSteps: ForkHistoricalStep[];
};
@@ -221,7 +258,7 @@ export function buildForkPlan(
hash: start.hash,
sourceThreadId: start.threadId,
prompt: start.prompt,
runOptions: { maxRounds: start.maxRounds },
runOptions: { maxRounds: start.maxRounds, depth: start.depth },
historicalSteps: selected.value,
});
}
+22
View File
@@ -17,6 +17,7 @@ export {
} from "./engine.js";
export { type ExtractedBundleExports, extractBundleExports } from "./extract-bundle-exports.js";
export { createExtract, type ExtractFn } from "./extract-fn.js";
export { getExtractProvider } from "./extract-provider.js";
export {
buildForkPlan,
type ForkHistoricalStep,
@@ -24,6 +25,8 @@ export {
type ParsedThreadStartRecord,
parseThreadDataJsonl,
selectForkHistoricalSteps,
tryParseRoleStepRecord,
tryParseWorkflowResultRecord,
} from "./fork-thread.js";
export { type GcResult, garbageCollectCas } from "./gc.js";
export { stringifyWorkflowDescriptor } from "./generate-descriptor.js";
@@ -41,6 +44,21 @@ export {
type LoggerSink,
} from "./logger.js";
export {
createContentMerkleNode,
getContentMerklePayload,
type MerkleNode,
type MerkleNodeType,
parseMerkleNode,
putContentMerkleNode,
putStepMerkleNode,
putThreadMerkleNode,
type StepMerklePayload,
serializeMerkleNode,
type ThreadMerklePayload,
} from "./merkle.js";
export { type ReactExtractArgs, reactExtract } from "./react-extract.js";
export {
type ExtractProviderConfig,
getRegisteredWorkflow,
listRegisteredWorkflowNames,
parseWorkflowRegistryYaml,
@@ -49,6 +67,7 @@ export {
rollbackWorkflowToHistoryHash,
stringifyWorkflowRegistryYaml,
unregisterWorkflow,
type WorkflowConfig,
type WorkflowHistoryEntry,
type WorkflowRegistryEntry,
type WorkflowRegistryFile,
@@ -64,6 +83,7 @@ export {
type AgentFn,
END,
type ExtractContext,
type ExtractMode,
type LlmProvider,
type Moderator,
type ModeratorContext,
@@ -75,6 +95,7 @@ export {
type StartStep,
type ThreadContext,
type ThreadInput,
type WorkflowCompletion,
type WorkflowDefinition,
type WorkflowFn,
type WorkflowFnOptions,
@@ -82,6 +103,7 @@ export {
} from "./types.js";
export { generateUlid } from "./ulid.js";
export { getWorkerHostScriptPath } from "./worker-entry-path.js";
export { type WorkflowAsAgentOptions, workflowAsAgent } from "./workflow-as-agent.js";
export {
validateWorkflowDescriptor,
type WorkflowDescriptor,
+20 -8
View File
@@ -47,6 +47,21 @@ function readToolDescription(parametersSchema: Record<string, unknown>): string
return "Extract structured data from the input text.";
}
/** Builds OpenAI function-tool metadata from a Zod meta schema (same naming rules as single-shot extract). */
export function extractFunctionToolFromZodSchema(schema: z.ZodType<unknown>): {
name: string;
description: string;
parameters: Record<string, unknown>;
} {
const rawJsonSchema = z.toJSONSchema(schema) as Record<string, unknown>;
const parameters = stripJsonSchemaMeta(rawJsonSchema);
return {
name: readToolName(parameters),
description: readToolDescription(parameters),
parameters,
};
}
function readToolArgumentsJson(parsed: unknown, previewSource: string): Result<string, LlmError> {
if (!isRecord(parsed)) {
return err({ kind: "invalid_response_json", message: "Top-level JSON is not an object" });
@@ -124,10 +139,7 @@ export function llmErrorToCause(error: LlmError): Error {
async function performLlmExtract<T>(
options: LlmExtractArgs<T> & { userContent: string },
): Promise<Result<T, LlmError>> {
const rawJsonSchema = z.toJSONSchema(options.schema) as Record<string, unknown>;
const parameters = stripJsonSchemaMeta(rawJsonSchema);
const toolName = readToolName(parameters);
const toolDescription = readToolDescription(parameters);
const extractTool = extractFunctionToolFromZodSchema(options.schema);
const body = {
model: options.provider.model,
@@ -142,13 +154,13 @@ async function performLlmExtract<T>(
{
type: "function" as const,
function: {
name: toolName,
description: toolDescription,
parameters,
name: extractTool.name,
description: extractTool.description,
parameters: extractTool.parameters,
},
},
],
tool_choice: { type: "function" as const, function: { name: toolName } },
tool_choice: { type: "function" as const, function: { name: extractTool.name } },
};
let response: Response;
+122
View File
@@ -0,0 +1,122 @@
import { parse, stringify } from "yaml";
import type { CasStore } from "./cas.js";
export type MerkleNodeType = "content" | "step" | "thread";
export type MerkleNode = {
type: MerkleNodeType;
payload: string | Record<string, unknown>;
children: string[];
};
export function serializeMerkleNode(node: MerkleNode): string {
return stringify(
{ type: node.type, payload: node.payload, children: node.children },
{ indent: 2 },
);
}
export function parseMerkleNode(yamlText: string): MerkleNode {
const raw = parse(yamlText) as unknown;
if (raw === null || typeof raw !== "object") {
throw new Error("merkle: YAML root must be an object");
}
const rec = raw as Record<string, unknown>;
const type = rec.type;
const payload = rec.payload;
const children = rec.children;
if (type !== "content" && type !== "step" && type !== "thread") {
throw new Error("merkle: invalid or missing type");
}
if (typeof payload !== "string" && (payload === null || typeof payload !== "object")) {
throw new Error("merkle: payload must be a string or object");
}
if (!Array.isArray(children)) {
throw new Error("merkle: children must be an array");
}
const childHashes: string[] = [];
for (const c of children) {
if (typeof c !== "string") {
throw new Error("merkle: child hash must be a string");
}
childHashes.push(c);
}
return {
type,
payload: typeof payload === "string" ? payload : (payload as Record<string, unknown>),
children: childHashes,
};
}
export function createContentMerkleNode(payload: string): MerkleNode {
return { type: "content", payload, children: [] };
}
export type StepMerklePayload = {
role: string;
meta: Record<string, unknown>;
};
export type ThreadMerklePayload = {
workflow: string;
threadId: string;
result: {
returnCode: number;
summary: string;
};
};
/** Serializes a step Merkle node (role + meta + content child) and stores it in CAS. */
export async function putStepMerkleNode(
store: CasStore,
payload: StepMerklePayload,
contentHash: string,
): Promise<string> {
const node: MerkleNode = {
type: "step",
payload: { role: payload.role, meta: payload.meta },
children: [contentHash],
};
return store.put(serializeMerkleNode(node));
}
/** Serializes the thread root Merkle node and stores it in CAS. */
export async function putThreadMerkleNode(
store: CasStore,
payload: ThreadMerklePayload,
stepHashes: readonly string[],
): Promise<string> {
const node: MerkleNode = {
type: "thread",
payload: {
workflow: payload.workflow,
threadId: payload.threadId,
result: payload.result,
},
children: [...stepHashes],
};
return store.put(serializeMerkleNode(node));
}
/** Serializes a content Merkle node and stores it in CAS; returns its hash. */
export async function putContentMerkleNode(store: CasStore, content: string): Promise<string> {
const yamlText = serializeMerkleNode(createContentMerkleNode(content));
return store.put(yamlText);
}
/** Loads a CAS blob and returns the payload string for a `content` Merkle node. */
export async function getContentMerklePayload(
store: CasStore,
hash: string,
): Promise<string | null> {
const yamlText = await store.get(hash);
if (yamlText === null) {
return null;
}
const node = parseMerkleNode(yamlText);
if (node.type !== "content" || typeof node.payload !== "string") {
return null;
}
return node.payload;
}
+330
View File
@@ -0,0 +1,330 @@
import type * as z from "zod/v4";
import type { CasStore } from "./cas.js";
import { extractFunctionToolFromZodSchema } from "./llm-extract.js";
import { err, ok, type Result } from "./result.js";
import type { LlmProvider } from "./types.js";
export type ReactExtractArgs<T extends Record<string, unknown>> = {
text: string;
schema: z.ZodType<T>;
provider: LlmProvider;
cas: CasStore;
};
const MAX_REACT_ROUNDS = 10;
const CAS_GET_TOOL_DEFINITION = {
type: "function" as const,
function: {
name: "cas_get",
description:
"Read a Merkle DAG node from content-addressed storage by its hash. Returns YAML-formatted node with type, payload, and children fields.",
parameters: {
type: "object",
properties: {
hash: { type: "string", description: "The CAS hash to retrieve" },
},
required: ["hash"],
},
},
};
function chatCompletionsUrl(baseUrl: string): string {
const trimmed = baseUrl.replace(/\/+$/, "");
return `${trimmed}/chat/completions`;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
function tryParseJsonContent(content: string): unknown | null {
const trimmed = content.trim();
const fenceMatch = /^```(?:json)?\s*([\s\S]*?)```$/m.exec(trimmed);
const payload = fenceMatch !== null ? fenceMatch[1].trim() : trimmed;
try {
return JSON.parse(payload) as unknown;
} catch {
return null;
}
}
type ToolCall = {
id: string;
type: "function";
function: { name: string; arguments: string };
};
type ChatMessage =
| { role: "system"; content: string }
| { role: "user"; content: string }
| {
role: "assistant";
content: string | null;
tool_calls: ToolCall[];
}
| { role: "tool"; tool_call_id: string; content: string };
type AssistantTurn<T> =
| { kind: "plain_json"; value: T }
| { kind: "tool_calls"; calls: ToolCall[]; assistantContent: string | null };
function firstAssistantMessage(responseText: string): Result<Record<string, unknown>, string> {
let parsed: unknown;
try {
parsed = JSON.parse(responseText) as unknown;
} catch (cause) {
const message = cause instanceof Error ? cause.message : String(cause);
return err(`invalid_response_json:${message}`);
}
if (!isRecord(parsed)) {
return err("invalid_response_top_level");
}
const choices = parsed.choices;
if (!Array.isArray(choices) || choices.length === 0) {
return err("no_choices_in_response");
}
const firstChoice = choices[0];
if (!isRecord(firstChoice)) {
return err("invalid_choice");
}
const messageObj = firstChoice.message;
if (!isRecord(messageObj)) {
return err("invalid_message");
}
return ok(messageObj);
}
function normalizeToolCalls(toolCallsRaw: unknown[]): Result<ToolCall[], string> {
const toolCalls: ToolCall[] = [];
for (const tc of toolCallsRaw) {
if (!isRecord(tc)) {
return err("invalid_tool_call");
}
const id = tc.id;
const tcType = tc.type;
const fn = tc.function;
if (typeof id !== "string" || tcType !== "function" || !isRecord(fn)) {
return err("invalid_tool_call_shape");
}
const name = fn.name;
const argumentsStr = fn.arguments;
if (typeof name !== "string" || typeof argumentsStr !== "string") {
return err("invalid_tool_call_function");
}
toolCalls.push({ id, type: "function", function: { name, arguments: argumentsStr } });
}
return ok(toolCalls);
}
function classifyAssistantTurn<T extends Record<string, unknown>>(
messageObj: Record<string, unknown>,
schema: z.ZodType<T>,
): Result<AssistantTurn<T>, string> {
const toolCallsRaw = messageObj.tool_calls;
if (!Array.isArray(toolCallsRaw) || toolCallsRaw.length === 0) {
const content = messageObj.content;
if (typeof content !== "string") {
return err("no_tool_calls_and_no_string_content");
}
const jsonParsed = tryParseJsonContent(content);
if (jsonParsed === null) {
return err("no_tool_calls_and_content_not_json");
}
const validated = schema.safeParse(jsonParsed);
if (!validated.success) {
return err(`schema_validation_failed:${validated.error.message}`);
}
return ok({ kind: "plain_json", value: validated.data });
}
const callsResult = normalizeToolCalls(toolCallsRaw);
if (!callsResult.ok) {
return err(callsResult.error);
}
const assistantContent = messageObj.content;
return ok({
kind: "tool_calls",
calls: callsResult.value,
assistantContent: typeof assistantContent === "string" ? assistantContent : null,
});
}
async function appendCasGetToolResult(
tc: ToolCall,
cas: CasStore,
messages: ChatMessage[],
): Promise<Result<null, string>> {
let hash: string;
try {
const ta = JSON.parse(tc.function.arguments) as unknown;
if (!isRecord(ta) || typeof ta.hash !== "string") {
return err("cas_get_invalid_arguments");
}
hash = ta.hash;
} catch {
return err("cas_get_arguments_not_json");
}
const blob = await cas.get(hash);
const toolContent = blob === null ? "null" : blob;
messages.push({
role: "tool",
tool_call_id: tc.id,
content: toolContent,
});
return ok(null);
}
async function appendExtractToolResult<T extends Record<string, unknown>>(
tc: ToolCall,
schema: z.ZodType<T>,
messages: ChatMessage[],
): Promise<Result<T, string>> {
let parsedArgs: unknown;
try {
parsedArgs = JSON.parse(tc.function.arguments) as unknown;
} catch {
return err("extract_tool_arguments_not_json");
}
const validated = schema.safeParse(parsedArgs);
if (!validated.success) {
return err(`schema_validation_failed:${validated.error.message}`);
}
messages.push({
role: "tool",
tool_call_id: tc.id,
content: '{"ok":true}',
});
return ok(validated.data);
}
async function appendToolResults<T extends Record<string, unknown>>(
toolCalls: ToolCall[],
extractToolName: string,
schema: z.ZodType<T>,
cas: CasStore,
messages: ChatMessage[],
): Promise<Result<T | null, string>> {
let extracted: T | null = null;
for (const tc of toolCalls) {
if (tc.function.name === "cas_get") {
const casRes = await appendCasGetToolResult(tc, cas, messages);
if (!casRes.ok) {
return casRes;
}
continue;
}
if (tc.function.name === extractToolName) {
const exRes = await appendExtractToolResult(tc, schema, messages);
if (!exRes.ok) {
return exRes;
}
extracted = exRes.value;
continue;
}
return err(`unknown_tool:${tc.function.name}`);
}
return ok(extracted);
}
async function postChatCompletion(
provider: LlmProvider,
messages: ChatMessage[],
tools: readonly Record<string, unknown>[],
): Promise<Result<string, string>> {
try {
const response = await fetch(chatCompletionsUrl(provider.baseUrl), {
method: "POST",
headers: {
Authorization: `Bearer ${provider.apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: provider.model,
messages,
tools,
tool_choice: "auto",
}),
});
const responseText = await response.text();
if (!response.ok) {
return err(`http_error:${String(response.status)}:${responseText.slice(0, 4000)}`);
}
return ok(responseText);
} catch (cause) {
const message = cause instanceof Error ? cause.message : String(cause);
return err(`network_error:${message}`);
}
}
/**
* Multi-turn ReAct extraction with `cas_get` plus a schema-shaped extract tool (OpenAI-compatible).
* Final meta comes from a successful extract tool call or from plain JSON in the assistant message.
*/
export async function reactExtract<T extends Record<string, unknown>>(
args: ReactExtractArgs<T>,
): Promise<Result<T, string>> {
const extractTool = extractFunctionToolFromZodSchema(args.schema);
const tools = [
CAS_GET_TOOL_DEFINITION,
{
type: "function" as const,
function: {
name: extractTool.name,
description: extractTool.description,
parameters: extractTool.parameters,
},
},
];
const systemContent = `You extract structured metadata from the agent output below. Use cas_get to read Merkle DAG nodes from CAS (YAML: type, payload, children) when the agent output references hashes you must traverse. When you have the complete structured object, call the ${extractTool.name} tool with JSON arguments matching the schema. You may instead reply with only a JSON object (no prose) when no tools are needed.`;
const messages: ChatMessage[] = [
{ role: "system", content: systemContent },
{ role: "user", content: args.text },
];
for (let round = 0; round < MAX_REACT_ROUNDS; round++) {
const bodyResult = await postChatCompletion(args.provider, messages, tools);
if (!bodyResult.ok) {
return bodyResult;
}
const msgResult = firstAssistantMessage(bodyResult.value);
if (!msgResult.ok) {
return msgResult;
}
const classified = classifyAssistantTurn(msgResult.value, args.schema);
if (!classified.ok) {
return classified;
}
const turn = classified.value;
if (turn.kind === "plain_json") {
return ok(turn.value);
}
messages.push({
role: "assistant",
content: turn.assistantContent,
tool_calls: turn.calls,
});
const toolsRound = await appendToolResults(
turn.calls,
extractTool.name,
args.schema,
args.cas,
messages,
);
if (!toolsRound.ok) {
return toolsRound;
}
if (toolsRound.value !== null) {
return ok(toolsRound.value);
}
}
return err("max_react_rounds_exceeded");
}
+9
View File
@@ -1,3 +1,12 @@
/** Append `contentHash` to `refs` when not already present (dedupe by first occurrence order). */
export function mergeRefsWithContentHash(refs: string[], contentHash: string): string[] {
const out = [...refs];
if (!out.includes(contentHash)) {
out.push(contentHash);
}
return out;
}
/** Normalize `refs` from persisted JSONL or IPC payloads (missing or invalid → []). */
export function normalizeRefsField(value: unknown): string[] {
if (!Array.isArray(value)) {
+68 -1
View File
@@ -1,10 +1,68 @@
import type {
ExtractProviderConfig,
WorkflowConfig,
WorkflowHistoryEntry,
WorkflowRegistryEntry,
WorkflowRegistryFile,
} from "./registry-types.js";
import { err, ok, type Result } from "./result.js";
function resolveRegistryApiKey(raw: string): Result<string, Error> {
if (raw.startsWith("env:")) {
const name = raw.slice("env:".length);
if (name === "") {
return err(new Error('config.extract.apiKey "env:" reference must name a variable'));
}
const value = process.env[name];
if (value === undefined) {
return err(new Error(`config.extract.apiKey: environment variable "${name}" is not set`));
}
return ok(value);
}
return ok(raw);
}
function normalizeExtractProviderConfig(raw: unknown): Result<ExtractProviderConfig, Error> {
if (raw === null || typeof raw !== "object") {
return err(new Error('registry config must contain an "extract" mapping'));
}
const e = raw as Record<string, unknown>;
const baseUrl = e.baseUrl;
const model = e.model;
const apiKeyRaw = e.apiKey;
if (typeof baseUrl !== "string" || baseUrl === "") {
return err(new Error("config.extract.baseUrl must be a non-empty string"));
}
if (typeof model !== "string" || model === "") {
return err(new Error("config.extract.model must be a non-empty string"));
}
if (typeof apiKeyRaw !== "string" || apiKeyRaw === "") {
return err(new Error("config.extract.apiKey must be a non-empty string"));
}
const apiKeyResult = resolveRegistryApiKey(apiKeyRaw);
if (!apiKeyResult.ok) {
return apiKeyResult;
}
return ok({ baseUrl, model, apiKey: apiKeyResult.value });
}
function normalizeWorkflowConfig(raw: unknown): Result<WorkflowConfig, Error> {
if (raw === null || typeof raw !== "object") {
return err(new Error('registry "config" must be a mapping'));
}
const c = raw as Record<string, unknown>;
const maxDepth = c.maxDepth;
const extractRaw = c.extract;
if (typeof maxDepth !== "number" || !Number.isInteger(maxDepth) || maxDepth < 0) {
return err(new Error("config.maxDepth must be a non-negative integer"));
}
const extractResult = normalizeExtractProviderConfig(extractRaw);
if (!extractResult.ok) {
return extractResult;
}
return ok({ maxDepth, extract: extractResult.value });
}
export function normalizeWorkflowHistoryEntry(
workflowName: string,
index: number,
@@ -61,6 +119,15 @@ export function normalizeWorkflowRegistryRoot(raw: unknown): Result<WorkflowRegi
return err(new Error("registry root must be a mapping"));
}
const root = raw as Record<string, unknown>;
const configRaw = root.config;
let config: WorkflowConfig | null = null;
if (configRaw !== undefined && configRaw !== null) {
const configResult = normalizeWorkflowConfig(configRaw);
if (!configResult.ok) {
return configResult;
}
config = configResult.value;
}
const workflowsRaw = root.workflows;
if (workflowsRaw === null || workflowsRaw === undefined || typeof workflowsRaw !== "object") {
return err(new Error('registry must contain a "workflows" mapping'));
@@ -73,5 +140,5 @@ export function normalizeWorkflowRegistryRoot(raw: unknown): Result<WorkflowRegi
}
workflows[name] = entryResult.value;
}
return ok({ workflows });
return ok({ config, workflows });
}
+13
View File
@@ -9,6 +9,19 @@ export type WorkflowRegistryEntry = {
history: WorkflowHistoryEntry[];
};
/** LLM provider settings under `config.extract` in workflow.yaml (apiKey resolved after parse). */
export type ExtractProviderConfig = {
baseUrl: string;
model: string;
apiKey: string;
};
export type WorkflowConfig = {
maxDepth: number;
extract: ExtractProviderConfig;
};
export type WorkflowRegistryFile = {
config: WorkflowConfig | null;
workflows: Record<string, WorkflowRegistryEntry>;
};
+5 -2
View File
@@ -12,6 +12,8 @@ import type {
import { err, ok, type Result } from "./result.js";
export type {
ExtractProviderConfig,
WorkflowConfig,
WorkflowHistoryEntry,
WorkflowRegistryEntry,
WorkflowRegistryFile,
@@ -22,7 +24,7 @@ export function workflowRegistryPath(storageRoot: string): string {
}
function emptyRegistry(): WorkflowRegistryFile {
return { workflows: {} };
return { config: null, workflows: {} };
}
export function parseWorkflowRegistryYaml(text: string): Result<WorkflowRegistryFile, Error> {
@@ -103,6 +105,7 @@ export function registerWorkflowVersion(
: [{ hash: prev.hash, timestamp: prev.timestamp }, ...baseHistory];
const next: WorkflowRegistryEntry = { hash, timestamp, history };
return {
config: registry.config,
workflows: { ...registry.workflows, [name]: next },
};
}
@@ -150,5 +153,5 @@ export function unregisterWorkflow(
return err(new Error(`workflow not registered: ${name}`));
}
const { [name]: _removed, ...rest } = registry.workflows;
return ok({ workflows: rest });
return ok({ config: registry.config, workflows: rest });
}
+24 -5
View File
@@ -1,5 +1,7 @@
import type * as z from "zod/v4";
import type { CasStore } from "./cas.js";
/** Sentinel values for automaton control flow. */
export const START = "__start__" as const;
export const END = "__end__" as const;
@@ -14,21 +16,30 @@ export type LlmProvider = {
model: string;
};
/** How the engine runs meta extraction for a role after the agent phase. */
export type ExtractMode = "single" | "react";
/** What each generator yield produces — one role's output (engine adds `timestamp` when persisting). */
export type RoleOutput = {
role: string;
content: string;
/** CAS hash of the serialized Merkle content node for this step's body text. */
contentHash: string;
meta: Record<string, unknown>;
/** CAS hashes produced or consumed by this step (for GC traceability). */
refs: string[];
};
/** What the workflow AsyncGenerator returns when done. */
export type WorkflowResult = {
/** Generator completion value from a workflow bundle (`run` export). Root hash is added by the engine. */
export type WorkflowCompletion = {
returnCode: number;
summary: string;
};
/** Final thread outcome from {@link executeThread}, including Merkle thread root CAS hash. */
export type WorkflowResult = WorkflowCompletion & {
rootHash: string;
};
/** Input to a workflow — prompt plus optional historical steps for fork/resume. */
export type ThreadInput = {
prompt: string;
@@ -39,13 +50,17 @@ export type ThreadInput = {
export type WorkflowFnOptions = {
threadId: string;
maxRounds: number;
/** Nesting depth for workflow-as-agent chains; root threads use `0`. */
depth: number;
/** Global CAS store for Merkle content blobs (role step bodies). */
cas: CasStore;
};
/** Bundle contract — named export `run` is a function returning an AsyncGenerator. */
export type WorkflowFn = (
input: ThreadInput,
options: WorkflowFnOptions,
) => AsyncGenerator<RoleOutput, WorkflowResult>;
) => AsyncGenerator<RoleOutput, WorkflowCompletion>;
/** Engine start frame: initial prompt + thread identity. */
export type StartStep = {
@@ -60,7 +75,7 @@ export type RoleStep<M extends RoleMeta> = {
[K in keyof M & string]: {
role: K;
meta: M[K];
content: string;
contentHash: string;
refs: string[];
timestamp: number;
};
@@ -69,6 +84,8 @@ export type RoleStep<M extends RoleMeta> = {
/** Phase 1: Moderator decides next role. */
export type ModeratorContext<M extends RoleMeta = RoleMeta> = {
threadId: string;
/** Same as `WorkflowFnOptions.depth` for the active thread. */
depth: number;
start: StartStep;
steps: RoleStep<M>[];
};
@@ -79,6 +96,7 @@ export type AgentContext<M extends RoleMeta = RoleMeta> = ModeratorContext<M> &
name: string;
systemPrompt: string;
};
cas: CasStore;
};
/** Phase 3: Extractor runs — has agent output; the extraction instruction is a separate argument to the extract function. */
@@ -106,6 +124,7 @@ export type RoleDefinition<Meta extends Record<string, unknown>> = {
schema: z.ZodType<Meta>;
/** When non-null, produces CAS hashes to persist on this role's steps (see `RoleOutput.refs`). */
extractRefs: ((meta: Meta) => string[]) | null;
extractMode: ExtractMode;
};
/**
+24 -11
View File
@@ -1,14 +1,17 @@
import { mkdir, unlink, writeFile } from "node:fs/promises";
import { appendFile, mkdir, unlink, writeFile } from "node:fs/promises";
import { createServer, type Socket } from "node:net";
import { dirname, join } from "node:path";
import { pathToFileURL } from "node:url";
import { importWorkflowBundleModule } from "./bundle-import-env.js";
import { createCasStore } from "./cas.js";
import type { PrefilledDiskStep } from "./engine.js";
import { type ExecuteThreadIo, executeThread } from "./engine.js";
import { ensureUncagedWorkflowSymlink } from "./ensure-uncaged-workflow-symlink.js";
import { createLogger } from "./logger.js";
import { normalizeRefsField } from "./refs-field.js";
import { err, ok, type Result } from "./result.js";
import { getGlobalCasDir } from "./storage-root.js";
import { createThreadPauseGate, type ThreadPauseGate } from "./thread-pause-gate.js";
import type { RoleOutput, WorkflowFn } from "./types.js";
import type { RoleOutput, WorkflowFn, WorkflowResult } from "./types.js";
const bootLog = createLogger({ sink: { kind: "stderr" } });
@@ -17,7 +20,7 @@ type RunCommand = {
threadId: string;
workflowName: string;
prompt: string;
options: { maxRounds: number };
options: { maxRounds: number; depth: number };
steps: RoleOutput[];
/** Timestamps aligned with `steps` for `.data.jsonl` replay; length must match `steps` when non-null. */
stepTimestamps: number[] | null;
@@ -48,9 +51,9 @@ type ThreadHandle = {
function parseRoleOutputRecord(obj: Record<string, unknown>): RoleOutput | null {
const role = obj.role;
const content = obj.content;
const contentHash = obj.contentHash;
const meta = obj.meta;
if (typeof role !== "string" || typeof content !== "string") {
if (typeof role !== "string" || typeof contentHash !== "string") {
return null;
}
if (meta === null || typeof meta !== "object") {
@@ -58,7 +61,7 @@ function parseRoleOutputRecord(obj: Record<string, unknown>): RoleOutput | null
}
return {
role,
content,
contentHash,
meta: meta as Record<string, unknown>,
refs: normalizeRefsField(obj.refs),
};
@@ -124,6 +127,9 @@ function parseRunControlPayload(rec: Record<string, unknown>): RunCommand | null
if (typeof maxRounds !== "number") {
return null;
}
const depthRaw = optRec.depth;
const depth =
typeof depthRaw === "number" && Number.isFinite(depthRaw) ? Math.trunc(depthRaw) : 0;
const parsedSteps = parseRunStepsPayload(rec);
if (parsedSteps === null) {
return null;
@@ -141,7 +147,7 @@ function parseRunControlPayload(rec: Record<string, unknown>): RunCommand | null
threadId,
workflowName,
prompt,
options: { maxRounds },
options: { maxRounds, depth },
steps: parsedSteps.steps,
stepTimestamps: parsedSteps.stepTimestamps,
forkSourceThreadId,
@@ -297,8 +303,9 @@ async function main(): Promise<void> {
return;
}
await ensureUncagedWorkflowSymlink(storageRoot);
// Dynamic import required: user bundle path resolved at runtime
const modUnknown: unknown = await import(pathToFileURL(bundlePath).href);
const modUnknown: unknown = await importWorkflowBundleModule(bundlePath);
const modRec = modUnknown as Record<string, unknown>;
const runExport = modRec.run;
if (!isWorkflowFnLike(runExport)) {
@@ -312,6 +319,8 @@ async function main(): Promise<void> {
let activeThreads = 0;
let shutdownTimer: ReturnType<typeof setTimeout> | null = null;
const cas = createCasStore(getGlobalCasDir(storageRoot));
const workerCtlPath = join(storageRoot, "workers", `${hash}.json`);
function cancelShutdownTimer(): void {
@@ -360,6 +369,7 @@ async function main(): Promise<void> {
hash,
dataJsonlPath,
infoJsonlPath,
cas,
};
const existing = threads.get(threadId);
@@ -386,7 +396,7 @@ async function main(): Promise<void> {
const ts = cmd.stepTimestamps?.[i];
return {
role: step.role,
content: step.content,
contentHash: step.contentHash,
meta: step.meta,
refs: normalizeRefsField(step.refs),
timestamp: typeof ts === "number" && ts > 0 ? ts : baseTs + i,
@@ -394,7 +404,7 @@ async function main(): Promise<void> {
});
}
await executeThread(
const runResult = await executeThread(
workflowFn,
cmd.workflowName,
{ prompt: cmd.prompt, steps: cmd.steps },
@@ -408,9 +418,12 @@ async function main(): Promise<void> {
io,
logger,
);
await appendFile(dataJsonlPath, `${JSON.stringify(runResult)}\n`, "utf8");
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
bootLog("Q3MN8YKW", `thread ${threadId} failed: ${message}`);
const failure: WorkflowResult = { returnCode: 1, summary: message, rootHash: "" };
await appendFile(dataJsonlPath, `${JSON.stringify(failure)}\n`, "utf8").catch(() => {});
} finally {
threads.delete(threadId);
await unlink(runningPath).catch(() => {});
+101
View File
@@ -0,0 +1,101 @@
import { join } from "node:path";
import { createCasStore } from "./cas.js";
import { type ExecuteThreadIo, executeThread } from "./engine.js";
import { extractBundleExports } from "./extract-bundle-exports.js";
import { getWorkflowAsAgentMaxDepth } from "./extract-provider.js";
import { createLogger } from "./logger.js";
import { getRegisteredWorkflow, readWorkflowRegistry } from "./registry.js";
import { getDefaultWorkflowStorageRoot, getGlobalCasDir } from "./storage-root.js";
import type { AgentContext, AgentFn, ThreadInput } from "./types.js";
import { generateUlid } from "./ulid.js";
export type WorkflowAsAgentOptions = {
/** When `null`, uses `getDefaultWorkflowStorageRoot()`. */
storageRoot: string | null;
};
function resolveWorkflowAsAgentStorageRoot(options: WorkflowAsAgentOptions | null): string {
if (options !== null && options.storageRoot !== null) {
return options.storageRoot;
}
return getDefaultWorkflowStorageRoot();
}
/**
* Returns an {@link AgentFn} that runs another registered workflow in a new thread,
* using the parent thread's initial prompt (`ctx.start.content`) as the child {@link ThreadInput.prompt}.
*/
export function workflowAsAgent(
workflowName: string,
options: WorkflowAsAgentOptions | null = null,
): AgentFn {
return async (ctx: AgentContext): Promise<string> => {
const nextDepth = ctx.depth + 1;
const storageRoot = resolveWorkflowAsAgentStorageRoot(options);
const registryResult = await readWorkflowRegistry(storageRoot);
if (!registryResult.ok) {
return `ERROR: failed to read workflow registry: ${registryResult.error.message}`;
}
const maxDepth = getWorkflowAsAgentMaxDepth(registryResult.value.config);
if (nextDepth > maxDepth) {
return `ERROR: workflow-as-agent depth limit exceeded (max ${maxDepth})`;
}
const entry = getRegisteredWorkflow(registryResult.value, workflowName);
if (entry === null) {
return `ERROR: workflow "${workflowName}" not found in registry`;
}
const bundlePath = join(storageRoot, "bundles", `${entry.hash}.esm.js`);
const bundleExportsResult = await extractBundleExports(bundlePath, { storageRoot });
if (!bundleExportsResult.ok) {
return `ERROR: ${bundleExportsResult.error}`;
}
const input: ThreadInput = {
prompt: ctx.start.content,
steps: [],
};
const childThreadId = generateUlid(Date.now());
const dataJsonlPath = join(storageRoot, "logs", entry.hash, `${childThreadId}.data.jsonl`);
const infoJsonlPath = join(storageRoot, "logs", entry.hash, `${childThreadId}.info.jsonl`);
const io: ExecuteThreadIo = {
threadId: childThreadId,
hash: entry.hash,
dataJsonlPath,
infoJsonlPath,
cas: createCasStore(getGlobalCasDir(storageRoot)),
};
const logger = createLogger({ sink: { kind: "file", path: infoJsonlPath } });
const signalNever = new AbortController();
try {
const result = await executeThread(
bundleExportsResult.value.run,
workflowName,
input,
{
maxRounds: ctx.start.meta.maxRounds,
depth: nextDepth,
signal: signalNever.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: ctx.threadId,
prefilledDiskSteps: null,
},
io,
logger,
);
return result.rootHash;
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
return `ERROR: ${message}`;
}
};
}
+3 -1
View File
@@ -23,10 +23,12 @@
{ "path": "packages/workflow-role-coder" },
{ "path": "packages/workflow-role-planner" },
{ "path": "packages/workflow-role-reviewer" },
{ "path": "packages/workflow-role-tester" },
{ "path": "packages/workflow-agent-cursor" },
{ "path": "packages/workflow-agent-hermes" },
{ "path": "packages/workflow-util-agent" },
{ "path": "packages/cli-workflow" },
{ "path": "packages/workflow-template-solve-issue" }
{ "path": "packages/workflow-template-solve-issue" },
{ "path": "packages/workflow-template-develop" }
]
}