Compare commits

...

15 Commits

Author SHA1 Message Date
xiaoju e95e76c145 feat: workflowAsAgent factory
- workflowAsAgent(name) resolves via registry → bundle → child thread
- System-level depth limit (max 3, constant)
- Returns summary string, errors as string (no throw)
- Integration test with nested workflow execution
- 146 tests passing

Fixes #33
2026-05-07 10:52:26 +00:00
xiaoju af69e773a0 Merge pull request 'feat: CAS garbage collection' (#38) from feat/32-cas-gc into main 2026-05-07 10:48:07 +00:00
xiaoju 6488b7bbb4 feat: CAS garbage collection
- garbageCollectCas() mark-and-sweep: scan .data.jsonl refs, delete orphans
- 'uncaged-workflow gc' CLI command
- thread rm triggers GC automatically
- 141 tests passing

Fixes #32
2026-05-07 10:47:52 +00:00
xiaoju 15d39c96a7 Merge pull request 'feat: add refs tracking to RoleStep' (#35) from feat/31-refs-tracking into main 2026-05-07 10:44:44 +00:00
xiaoju 30e4e99908 feat: add refs tracking to RoleStep
- RoleOutput gains refs: string[] for CAS reference tracking
- RoleDefinition gains extractRefs: ((meta) => string[]) | null
- planner: phases.map(p => p.hash), coder: [completedPhase]
- Engine persists refs, fork preserves refs
- Backward compat: missing refs normalized to []
- 137 tests passing

Fixes #31
2026-05-07 10:44:25 +00:00
xiaoju a3c70a5041 Merge pull request 'feat: migrate CAS to global storage' (#34) from feat/30-global-cas into main 2026-05-07 10:40:41 +00:00
xiaoju 12d58a8206 feat: migrate CAS to global storage
- Add getGlobalCasDir() to storage-root.ts
- cmd-cas.ts uses global CAS dir, threadId kept for CLI compat
- thread rm no longer deletes .cas/ directories
- Rename createThreadCas → createCasStore (deprecated alias kept)
- 134 tests passing

BREAKING: CAS moves from <thread>.cas/ to <storageRoot>/cas/

Fixes #30
2026-05-07 10:40:14 +00:00
xiaoju c096f4d94e docs: add workflow-as-agent implementation plan
Covers 4 phases:
1. Global CAS migration
2. RoleStep refs tracking
3. CAS garbage collection
4. workflowAsAgent(name) factory

Refs #25
2026-05-07 10:35:26 +00:00
xiaoju 500401d93c feat(workflow): add preparer role to solve-issue workflow (#29, closes #28) 2026-05-07 10:18:05 +00:00
xiaoju 43f466eb67 style(solve-issue): fix biome formatting in test file
Refs #28
2026-05-07 10:15:10 +00:00
xiaoju fe829d9ae6 feat(workflow): add preparer role as first step in solve-issue workflow
- New package: @uncaged/workflow-role-preparer with PreparerMeta type,
  schema, system prompt, and extract prompt
- Preparer locates/clones repo, detects toolchain and conventions
- Moderator updated: preparer → planner → coder → reviewer → committer
- solve-issue template re-exports preparer types
- Tests updated for new flow (129 pass, 0 fail)

Fixes #28
2026-05-07 10:12:59 +00:00
xiaoju f80535d742 fix(planner,coder): clarify CAS CLI usage in agent prompts (#27) 2026-05-07 09:43:21 +00:00
xiaoju 0eab3b7001 chore: remove temporary bundle entry file 2026-05-07 09:43:00 +00:00
xiaoju 37c5b89c98 fix(planner,coder): clarify CAS CLI usage in agent prompts
- Planner: mandatory CAS put with complete command template, thread ID guidance
- Coder: CAS get command template with thread ID guidance
- Forbid inventing storage paths — must use uncaged-workflow cas CLI

Fixes #26
2026-05-07 09:42:52 +00:00
xiaoju 0fdf19879a Merge pull request 'test(workflow): add unit tests for validateWorkflowDescriptor' (#20) from test/19-validate-workflow-descriptor into main 2026-05-07 09:38:54 +00:00
45 changed files with 1715 additions and 88 deletions
+315
View File
@@ -0,0 +1,315 @@
# Workflow-as-Agent Implementation Plan
> **For Hermes:** Use subagent-driven-development skill to implement this plan task-by-task.
**Goal:** Enable workflows to invoke other workflows as agents, backed by global CAS and refs tracking.
**Architecture:** Migrate CAS from thread-local to global (`~/.uncaged/workflow/cas/`), add `refs` to RoleStep for GC traceability, then build `workflowAsAgent(name)` factory that resolves workflow name → bundle via registry and spawns a child thread.
**Tech Stack:** TypeScript, Bun, Zod v4, monorepo with `packages/`
**Issue:** https://git.shazhou.work/uncaged/workflow/issues/25
---
## Phase 1: Global CAS Migration
Move CAS storage from `<threadDir>/<threadId>.cas/` to `~/.uncaged/workflow/cas/` (global, content-addressed, immutable). This is a **breaking change** — thread-local `.cas/` directories are abandoned.
### Task 1.1: Add `globalCasDir` helper to `storage-root.ts`
**Objective:** Provide a single function that returns the global CAS directory path.
**Files:**
- Modify: `packages/workflow/src/storage-root.ts`
- Test: `packages/workflow/__tests__/storage-root.test.ts`
**Implementation:**
```typescript
// storage-root.ts — add export
export function getGlobalCasDir(storageRoot?: string): string {
const root = storageRoot ?? getDefaultWorkflowStorageRoot();
return join(root, "cas");
}
```
Export from `packages/workflow/src/index.ts`.
### Task 1.2: Update `cmd-cas.ts` to use global CAS
**Objective:** CLI `cas get/put/list/rm` no longer needs threadId for storage location — CAS is global. But keep threadId in CLI for backward compat of planner/coder prompts (they pass threadId).
**Files:**
- Modify: `packages/cli-workflow/src/cmd-cas.ts`
**Changes:**
- `resolveCasDir` → use `getGlobalCasDir(storageRoot)` instead of deriving from thread data path
- `cmdCasPut` / `cmdCasGet` / `cmdCasList` / `cmdCasRm`: threadId is still accepted (prompts pass it) but storage goes to global dir
- Remove the `resolveThreadDataPath` dependency for CAS operations — thread doesn't need to exist to read CAS
```typescript
import { createThreadCas, getGlobalCasDir } from "@uncaged/workflow";
export async function cmdCasGet(
storageRoot: string,
_threadId: string, // kept for CLI compat, not used for path
hash: string,
): Promise<Result<string, string>> {
const cas = createThreadCas(getGlobalCasDir(storageRoot));
const content = await cas.get(hash);
if (content === null) {
return err(`cas entry not found: ${hash}`);
}
return ok(content);
}
// ... same pattern for put/list/rm
```
### Task 1.3: Update `cmd-thread.ts` — thread rm no longer deletes `.cas/`
**Objective:** Since CAS is global, `thread rm` should NOT delete CAS entries. CAS cleanup is GC's job.
**Files:**
- Modify: `packages/cli-workflow/src/cmd-thread.ts`
- Check: remove any `rmdir` / `unlink` of `<threadId>.cas/` directory
### Task 1.4: Rename `createThreadCas` → `createCasStore`
**Objective:** The name `createThreadCas` is misleading now. Rename to `createCasStore`.
**Files:**
- Modify: `packages/workflow/src/cas.ts` — rename function
- Modify: `packages/workflow/src/index.ts` — update export (keep `createThreadCas` as deprecated alias for one release)
- Modify: all consumers (`cmd-cas.ts`)
### Task 1.5: Update tests
**Objective:** All CAS-related tests use global dir instead of thread-local.
**Files:**
- Modify: `packages/cli-workflow/__tests__/commands.test.ts`
- Verify: `bun test` passes
### Task 1.6: Clean up old thread-local `.cas/` references
**Objective:** Remove dead code that creates/reads thread-local `.cas/` directories.
**Files:**
- Search all `*.ts` for `.cas` path construction patterns
- Remove orphaned helpers
---
## Phase 2: RoleStep `refs` Tracking
Add `refs: string[]` to persisted role steps so GC can trace which CAS entries are alive.
### Task 2.1: Add `refs` to `RoleOutput` and engine persistence
**Objective:** Every role step can declare which CAS hashes it produced or consumed.
**Files:**
- Modify: `packages/workflow/src/types.ts`
- Modify: `packages/workflow/src/engine.ts`
**Changes to `types.ts`:**
```typescript
export type RoleOutput = {
role: string;
content: string;
meta: Record<string, unknown>;
refs: string[]; // CAS hashes produced/consumed by this step
};
```
**Changes to `engine.ts`:**
- `appendDataLine` for role steps: include `refs` field (default `[]` if not provided)
### Task 2.2: Auto-populate refs from meta hashes
**Objective:** The engine should automatically extract CAS hashes from `meta` to populate `refs`, so roles don't need to manually track them.
**Strategy:** After meta extraction, walk the meta object and collect any string that looks like a CAS hash (Crockford Base32, 13 chars). This is a heuristic but works because CAS hashes are distinctive.
Alternative (simpler): Let each `RoleDefinition` optionally declare a `extractRefs(meta: M) => string[]` function. For planner, this returns `meta.phases.map(p => p.hash)`. For coder, `[meta.completedPhase]`.
**Recommended:** The explicit `extractRefs` approach — no magic, no false positives.
**Files:**
- Modify: `packages/workflow/src/types.ts` — add optional `extractRefs` to `RoleDefinition`
- Modify: `packages/workflow/src/create-workflow.ts` — call `extractRefs` after meta extraction, set on `RoleOutput.refs`
- Modify: `packages/workflow-role-planner/src/planner.ts` — implement `extractRefs`
- Modify: `packages/workflow-role-coder/src/coder.ts` — implement `extractRefs`
```typescript
// types.ts — RoleDefinition addition
export type RoleDefinition<Meta extends Record<string, unknown>> = {
description: string;
systemPrompt: string;
extractPrompt: string;
schema: z.ZodType<Meta>;
extractRefs?: (meta: Meta) => string[]; // CAS hashes to track
};
// planner.ts
extractRefs: (meta) => meta.phases.map(p => p.hash),
// coder.ts
extractRefs: (meta) => [meta.completedPhase],
```
### Task 2.3: Update fork logic to preserve refs
**Objective:** When forking a thread, `refs` from historical steps must be carried over.
**Files:**
- Modify: `packages/workflow/src/fork-thread.ts`
- Verify: `ForkHistoricalStep` / `PrefilledDiskStep` include `refs`
### Task 2.4: Tests for refs tracking
**Files:**
- Add: `packages/workflow/__tests__/refs-tracking.test.ts`
- Verify: refs appear in `.data.jsonl` output
---
## Phase 3: CAS Garbage Collection
### Task 3.1: Implement `gc.ts` in `@uncaged/workflow`
**Objective:** Mark-and-sweep GC — scan all thread `.data.jsonl` files, collect `refs`, delete orphaned CAS entries.
**Files:**
- Create: `packages/workflow/src/gc.ts`
- Export from: `packages/workflow/src/index.ts`
```typescript
export type GcResult = {
scannedThreads: number;
activeRefs: number;
deletedEntries: number;
deletedHashes: string[];
};
export async function garbageCollectCas(storageRoot: string): Promise<GcResult> {
// 1. Find all .data.jsonl files under storageRoot
// 2. Parse each, flatMap step.refs → Set<string>
// 3. List all CAS entries via createCasStore(globalCasDir).list()
// 4. Delete entries not in active set
// 5. Return stats
}
```
### Task 3.2: Add `uncaged-workflow gc` CLI command
**Files:**
- Create: `packages/cli-workflow/src/cmd-gc.ts`
- Modify: `packages/cli-workflow/src/cli-dispatch.ts` — add `gc` subcommand
### Task 3.3: Run GC on `thread rm`
**Files:**
- Modify: `packages/cli-workflow/src/cmd-thread.ts` — after deleting thread data, optionally run GC
### Task 3.4: Tests for GC
**Files:**
- Create: `packages/cli-workflow/__tests__/gc-cli.test.ts`
---
## Phase 4: `workflowAsAgent` Factory
### Task 4.1: Create `workflowAsAgent` in `@uncaged/workflow`
**Objective:** Factory function that takes a workflow name, resolves to bundle, returns an `AgentFn`.
**Files:**
- Create: `packages/workflow/src/workflow-as-agent.ts`
- Export from: `packages/workflow/src/index.ts`
```typescript
import type { AgentFn } from "./types.js";
export type WorkflowAsAgentOptions = {
storageRoot?: string;
};
export function workflowAsAgent(
workflowName: string,
options?: WorkflowAsAgentOptions,
): AgentFn {
return async (ctx) => {
const storageRoot = options?.storageRoot ?? getDefaultWorkflowStorageRoot();
// 1. Read registry → resolve name to bundle hash + path
const registry = await readWorkflowRegistry(storageRoot);
const entry = getRegisteredWorkflow(registry, workflowName);
if (entry === null) {
return `ERROR: workflow "${workflowName}" not found in registry`;
}
// 2. Load bundle
const bundlePath = join(storageRoot, "bundles", `${entry.hash}.esm.js`);
const bundleExports = await extractBundleExports(bundlePath);
// 3. Create child thread input from ctx.start.content (parent prompt)
const input: ThreadInput = {
prompt: ctx.start.content,
steps: [],
};
// 4. Generate child threadId
const childThreadId = generateUlid();
// 5. Execute — collect all yields, return final content
const io: ExecuteThreadIo = { ... };
const result = await executeThread(bundleExports.run, workflowName, input, ...);
// 6. Return summary as agent content
return result.summary;
};
}
```
### Task 4.2: System-level depth limit
**Objective:** Prevent infinite recursion. Track depth via thread metadata, enforce a global max (default 3, configurable in `workflow.yaml`).
**Files:**
- Modify: `packages/workflow/src/types.ts` — add `depth` to `WorkflowFnOptions`
- Modify: `packages/workflow/src/workflow-as-agent.ts` — increment depth, check limit
- Modify: registry or config types for `maxDepth` setting
### Task 4.3: Tests for workflowAsAgent
**Files:**
- Create: `packages/workflow/__tests__/workflow-as-agent.test.ts`
- Test: name resolution, depth limit, child thread execution
### Task 4.4: Integration test — nested workflow
**Objective:** Create a minimal test workflow that calls another workflow via `workflowAsAgent`.
**Files:**
- Create: `packages/workflow/__tests__/workflow-as-agent-integration.test.ts`
---
## Execution Order
```
Phase 1 (Global CAS) → Phase 2 (refs) → Phase 3 (GC) → Phase 4 (workflowAsAgent)
```
Each phase is independently mergeable. Phase 3 depends on Phase 2 (needs refs to know what's alive). Phase 4 depends on Phase 1 (global CAS for cross-thread sharing).
## Breaking Changes
- CAS storage location moves from `<thread>.cas/` to `~/.uncaged/workflow/cas/`
- `RoleOutput` gains required `refs: string[]` field
- Existing threads with thread-local CAS will lose access to old CAS data (acceptable — those are short-lived workflow artifacts)
- `createThreadCas` renamed to `createCasStore` (alias kept temporarily)
+1
View File
@@ -28,6 +28,7 @@ const greeter: RoleDefinition<Roles["greeter"]> = {
systemPrompt: "You greet the user briefly.",
extractPrompt: "Extract the greeting string produced for the user.",
schema: greeterMetaSchema,
extractRefs: null,
};
const extract = createExtract({
@@ -3,8 +3,9 @@ import { mkdir, mkdtemp, readFile, rm, unlink, writeFile } from "node:fs/promise
import { tmpdir } from "node:os";
import { join } from "node:path";
import { getRegisteredWorkflow, readWorkflowRegistry } from "@uncaged/workflow";
import { getGlobalCasDir, getRegisteredWorkflow, readWorkflowRegistry } from "@uncaged/workflow";
import { cmdAdd } from "../src/cmd-add.js";
import { cmdCasGet, cmdCasList, cmdCasPut, cmdCasRm } from "../src/cmd-cas.js";
import { cmdHistory } from "../src/cmd-history.js";
import { cmdList, formatListLines } from "../src/cmd-list.js";
import { cmdRemove } from "../src/cmd-remove.js";
@@ -371,6 +372,37 @@ export const run = async function* (input) {
expect(bad.ok).toBe(false);
});
test("cas put/get/list/rm use global cas dir (thread id not required for storage)", async () => {
const put = await cmdCasPut(storageRoot, "nonexistent-thread-id", "phase doc");
expect(put.ok).toBe(true);
if (!put.ok) {
return;
}
const hash = put.value;
const blobPath = join(getGlobalCasDir(storageRoot), `${hash}.txt`);
expect(await readFile(blobPath, "utf8")).toBe("phase doc");
const got = await cmdCasGet(storageRoot, "other-thread", hash);
expect(got.ok).toBe(true);
if (!got.ok) {
return;
}
expect(got.value).toBe("phase doc");
const listed = await cmdCasList(storageRoot, "another-thread");
expect(listed.ok).toBe(true);
if (!listed.ok) {
return;
}
expect(listed.value).toContain(hash);
const removed = await cmdCasRm(storageRoot, "rm-thread", hash);
expect(removed.ok).toBe(true);
const missing = await cmdCasGet(storageRoot, "after-rm", hash);
expect(missing.ok).toBe(false);
});
test("rollback rejects missing bundle file for target hash", async () => {
const bundleDir = join(storageRoot, "src");
await mkdir(bundleDir, { recursive: true });
@@ -0,0 +1,144 @@
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
import { spawnSync } from "node:child_process";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { createCasStore, garbageCollectCas, getGlobalCasDir } from "@uncaged/workflow";
import { cmdThreadRemove } from "../src/cmd-thread.js";
import { pathExists } from "../src/fs-utils.js";
const cliEntryPath = fileURLToPath(new URL("../src/cli.ts", import.meta.url));
/** Minimal valid `.data.jsonl` with one role step referencing `activeHash` in `refs`. */
function makeDataJsonl(threadId: string, bundleHash: string, activeHash: string): string {
return [
JSON.stringify({
name: "demo",
hash: bundleHash,
threadId,
parameters: { prompt: "hi", options: { maxRounds: 5 } },
timestamp: 100,
}),
JSON.stringify({
role: "planner",
content: "p",
meta: {},
refs: [activeHash],
timestamp: 101,
}),
"",
].join("\n");
}
describe("gc cli and garbageCollectCas", () => {
let prevEnv: string | undefined;
let storageRoot: string;
beforeEach(async () => {
prevEnv = process.env.UNCAGED_WORKFLOW_STORAGE_ROOT;
storageRoot = await mkdtemp(join(tmpdir(), "uncaged-wf-gc-"));
process.env.UNCAGED_WORKFLOW_STORAGE_ROOT = storageRoot;
});
afterEach(async () => {
if (prevEnv === undefined) {
delete process.env.UNCAGED_WORKFLOW_STORAGE_ROOT;
} else {
process.env.UNCAGED_WORKFLOW_STORAGE_ROOT = prevEnv;
}
await rm(storageRoot, { recursive: true, force: true });
});
test("garbageCollectCas keeps CAS entries referenced by thread refs", async () => {
const bundleHash = "C9NMV6V2TQT81";
const threadId = "01AAA1111111111111111111";
const logsDir = join(storageRoot, "logs", bundleHash);
await mkdir(logsDir, { recursive: true });
const cas = createCasStore(getGlobalCasDir(storageRoot));
const activeHash = await cas.put("active-blob");
const orphanHash = await cas.put("orphan-blob");
await writeFile(
join(logsDir, `${threadId}.data.jsonl`),
makeDataJsonl(threadId, bundleHash, activeHash),
"utf8",
);
const gc = await garbageCollectCas(storageRoot);
expect(gc.ok).toBe(true);
if (!gc.ok) {
return;
}
expect(gc.value.scannedThreads).toBe(1);
expect(gc.value.activeRefs).toBe(1);
expect(gc.value.deletedEntries).toBe(1);
expect(gc.value.deletedHashes).toEqual([orphanHash]);
expect(await pathExists(join(getGlobalCasDir(storageRoot), `${activeHash}.txt`))).toBe(true);
expect(await pathExists(join(getGlobalCasDir(storageRoot), `${orphanHash}.txt`))).toBe(false);
});
test("garbageCollectCas deletes orphaned CAS when no threads reference them", async () => {
const cas = createCasStore(getGlobalCasDir(storageRoot));
const orphanHash = await cas.put("lonely");
const gc = await garbageCollectCas(storageRoot);
expect(gc.ok).toBe(true);
if (!gc.ok) {
return;
}
expect(gc.value.scannedThreads).toBe(0);
expect(gc.value.activeRefs).toBe(0);
expect(gc.value.deletedEntries).toBe(1);
expect(gc.value.deletedHashes).toEqual([orphanHash]);
expect(await pathExists(join(getGlobalCasDir(storageRoot), `${orphanHash}.txt`))).toBe(false);
});
test("cli gc prints stats", async () => {
const bundleHash = "C9NMV6V2TQT81";
const threadId = "01BBB2222222222222222222";
const logsDir = join(storageRoot, "logs", bundleHash);
await mkdir(logsDir, { recursive: true });
const cas = createCasStore(getGlobalCasDir(storageRoot));
const activeHash = await cas.put("keep-me");
await cas.put("drop-me");
await writeFile(
join(logsDir, `${threadId}.data.jsonl`),
makeDataJsonl(threadId, bundleHash, activeHash),
"utf8",
);
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const proc = spawnSync(process.execPath, [cliEntryPath, "gc"], { env, encoding: "utf8" });
expect(proc.status).toBe(0);
expect(String(proc.stdout).trim()).toBe("scanned 1 threads, 1 active refs, deleted 1 entries");
});
test("thread rm triggers gc so unreferenced CAS is removed", async () => {
const bundleHash = "C9NMV6V2TQT81";
const threadId = "01CCC3333333333333333333";
const logsDir = join(storageRoot, "logs", bundleHash);
await mkdir(logsDir, { recursive: true });
const cas = createCasStore(getGlobalCasDir(storageRoot));
const activeHash = await cas.put("pinned-by-ref");
await writeFile(
join(logsDir, `${threadId}.data.jsonl`),
makeDataJsonl(threadId, bundleHash, activeHash),
"utf8",
);
const orphanHash = await cas.put("orphan-after-rm");
const orphanPath = join(getGlobalCasDir(storageRoot), `${orphanHash}.txt`);
const removed = await cmdThreadRemove(storageRoot, threadId);
expect(removed.ok).toBe(true);
expect(await pathExists(orphanPath)).toBe(false);
expect(await pathExists(join(getGlobalCasDir(storageRoot), `${activeHash}.txt`))).toBe(false);
});
});
@@ -4,7 +4,9 @@ import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { getGlobalCasDir } from "@uncaged/workflow";
import { cmdAdd } from "../src/cmd-add.js";
import { cmdCasPut } from "../src/cmd-cas.js";
import { cmdKill } from "../src/cmd-kill.js";
import { cmdPause } from "../src/cmd-pause.js";
import { cmdPs } from "../src/cmd-ps.js";
@@ -12,7 +14,7 @@ import { cmdResume } from "../src/cmd-resume.js";
import { cmdRun } from "../src/cmd-run.js";
import { cmdThreadRemove, cmdThreadShow } from "../src/cmd-thread.js";
import { cmdThreads } from "../src/cmd-threads.js";
import { pathExists } from "../src/fs-utils.js";
import { pathExists, readTextFileIfExists } from "../src/fs-utils.js";
import { addCliArgs } from "./bundle-fixture.js";
const threadFixtureDescriptor = `export const descriptor = {
@@ -175,6 +177,55 @@ describe("cli thread commands", () => {
expect(await pathExists(dataPath)).toBe(false);
});
test("thread rm runs GC and removes CAS blobs not referenced by any remaining thread", async () => {
const bundleDir = join(storageRoot, "src");
await mkdir(bundleDir, { recursive: true });
const bundlePath = join(bundleDir, "demo.esm.js");
await writeFile(bundlePath, fastBundleSource, "utf8");
const added = await cmdAdd(storageRoot, addCliArgs("solve-issue", bundlePath));
expect(added.ok).toBe(true);
if (!added.ok) {
return;
}
const ran = await cmdRun(storageRoot, "solve-issue", "hello", 5);
expect(ran.ok).toBe(true);
if (!ran.ok) {
return;
}
const threadId = ran.value.threadId;
let threads = await cmdThreads(storageRoot, []);
for (
let attempt = 0;
attempt < 50 && threads.ok && !threads.value.some((l) => l.includes(threadId));
attempt++
) {
await new Promise((r) => setTimeout(r, 20));
threads = await cmdThreads(storageRoot, []);
}
const dataPath = join(storageRoot, "logs", added.value.hash, `${threadId}.data.jsonl`);
const runningPath = join(dirname(dataPath), `${threadId}.running`);
await waitUntilRunningFileAbsent(runningPath, 120);
const put = await cmdCasPut(storageRoot, threadId, "keep-after-thread-rm");
expect(put.ok).toBe(true);
if (!put.ok) {
return;
}
const hash = put.value;
const casBlob = join(getGlobalCasDir(storageRoot), `${hash}.txt`);
const removed = await cmdThreadRemove(storageRoot, threadId);
expect(removed.ok).toBe(true);
const stillThere = await readTextFileIfExists(casBlob);
expect(stillThere).toBeNull();
});
test("cli entrypoint dispatches threads / ps (spawn)", () => {
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
const threads = spawnSync(process.execPath, [cliEntryPath, "threads"], {
+20
View File
@@ -2,6 +2,7 @@ import { printCliError, printCliLine, printCliWarn } from "./cli-output.js";
import { cmdAdd, formatAddSuccess, parseAddArgv } from "./cmd-add.js";
import { cmdCasGet, cmdCasList, cmdCasPut, cmdCasRm } from "./cmd-cas.js";
import { cmdFork, parseForkArgv } from "./cmd-fork.js";
import { cmdGc } from "./cmd-gc.js";
import { cmdHistory } from "./cmd-history.js";
import { cmdKill } from "./cmd-kill.js";
import { cmdList, formatListLines } from "./cmd-list.js";
@@ -34,6 +35,7 @@ function usage(): string {
" uncaged-workflow thread <id>",
" uncaged-workflow thread rm <id>",
" uncaged-workflow fork <thread-id> [--from-role <role>]",
" uncaged-workflow gc",
" uncaged-workflow cas get <thread-id> <hash>",
" uncaged-workflow cas put <thread-id> <content>",
" uncaged-workflow cas list <thread-id>",
@@ -266,6 +268,23 @@ async function dispatchThreadBranch(storageRoot: string, rest: string[]): Promis
return dispatchThread(storageRoot, rest);
}
async function dispatchGc(storageRoot: string, argv: string[]): Promise<number> {
if (argv.length > 0) {
printCliError(`${usage()}\n\nerror: gc takes no arguments`);
return 1;
}
const result = await cmdGc(storageRoot);
if (!result.ok) {
printCliError(result.error);
return 1;
}
const stats = result.value;
printCliLine(
`scanned ${stats.scannedThreads} threads, ${stats.activeRefs} active refs, deleted ${stats.deletedEntries} entries`,
);
return 0;
}
async function dispatchFork(storageRoot: string, argv: string[]): Promise<number> {
const parsed = parseForkArgv(argv);
if (!parsed.ok) {
@@ -387,6 +406,7 @@ const COMMAND_TABLE: Record<string, DispatchFn> = {
threads: dispatchThreads,
thread: dispatchThreadBranch,
fork: dispatchFork,
gc: dispatchGc,
cas: dispatchCas,
};
+9 -33
View File
@@ -1,23 +1,11 @@
import { dirname, join } from "node:path";
import { createThreadCas, err, ok, type Result } from "@uncaged/workflow";
import { resolveThreadDataPath } from "./thread-scan.js";
function resolveCasDir(threadDataPath: string, threadId: string): string {
return join(dirname(threadDataPath), `${threadId}.cas`);
}
import { createCasStore, err, getGlobalCasDir, ok, type Result } from "@uncaged/workflow";
export async function cmdCasGet(
storageRoot: string,
threadId: string,
_threadId: string,
hash: string,
): Promise<Result<string, string>> {
const dataPath = await resolveThreadDataPath(storageRoot, threadId);
if (dataPath === null) {
return err(`thread not found: ${threadId}`);
}
const cas = createThreadCas(resolveCasDir(dataPath, threadId));
const cas = createCasStore(getGlobalCasDir(storageRoot));
const content = await cas.get(hash);
if (content === null) {
return err(`cas entry not found: ${hash}`);
@@ -27,41 +15,29 @@ export async function cmdCasGet(
export async function cmdCasPut(
storageRoot: string,
threadId: string,
_threadId: string,
content: string,
): Promise<Result<string, string>> {
const dataPath = await resolveThreadDataPath(storageRoot, threadId);
if (dataPath === null) {
return err(`thread not found: ${threadId}`);
}
const cas = createThreadCas(resolveCasDir(dataPath, threadId));
const cas = createCasStore(getGlobalCasDir(storageRoot));
const hash = await cas.put(content);
return ok(hash);
}
export async function cmdCasList(
storageRoot: string,
threadId: string,
_threadId: string,
): Promise<Result<string[], string>> {
const dataPath = await resolveThreadDataPath(storageRoot, threadId);
if (dataPath === null) {
return err(`thread not found: ${threadId}`);
}
const cas = createThreadCas(resolveCasDir(dataPath, threadId));
const cas = createCasStore(getGlobalCasDir(storageRoot));
const hashes = await cas.list();
return ok(hashes);
}
export async function cmdCasRm(
storageRoot: string,
threadId: string,
_threadId: string,
hash: string,
): Promise<Result<void, string>> {
const dataPath = await resolveThreadDataPath(storageRoot, threadId);
if (dataPath === null) {
return err(`thread not found: ${threadId}`);
}
const cas = createThreadCas(resolveCasDir(dataPath, threadId));
const cas = createCasStore(getGlobalCasDir(storageRoot));
await cas.delete(hash);
return ok(undefined);
}
+5
View File
@@ -0,0 +1,5 @@
import { type GcResult, garbageCollectCas, type Result } from "@uncaged/workflow";
export async function cmdGc(storageRoot: string): Promise<Result<GcResult, string>> {
return garbageCollectCas(storageRoot);
}
+1 -1
View File
@@ -46,7 +46,7 @@ export async function cmdRun(
threadId,
workflowName: name,
prompt,
options: { maxRounds },
options: { maxRounds, depth: 0 },
},
{ awaitResponseLine: false },
);
+4 -4
View File
@@ -1,7 +1,7 @@
import { rm, unlink } from "node:fs/promises";
import { unlink } from "node:fs/promises";
import { dirname, join } from "node:path";
import { err, ok, type Result } from "@uncaged/workflow";
import { err, garbageCollectCas, ok, type Result } from "@uncaged/workflow";
import { readTextFileIfExists } from "./fs-utils.js";
import { resolveThreadDataPath } from "./thread-scan.js";
@@ -33,12 +33,12 @@ export async function cmdThreadRemove(
const dir = dirname(dataPath);
const infoPath = join(dir, `${threadId}.info.jsonl`);
const runningPath = join(dir, `${threadId}.running`);
const casPath = join(dir, `${threadId}.cas`);
await unlink(dataPath);
await unlink(infoPath).catch(() => {});
await unlink(runningPath).catch(() => {});
await rm(casPath, { recursive: true, force: true });
await garbageCollectCas(storageRoot);
return ok(undefined);
}
@@ -11,6 +11,7 @@ function makeCtx(userContent: string): ThreadContext {
meta: { maxRounds: 10 },
timestamp: 1,
},
depth: 0,
steps: [],
threadId: "01TEST000000000000000000TR",
currentRole: { name: "planner", systemPrompt: "system instructions" },
+17 -1
View File
@@ -10,9 +10,24 @@ export const coderMetaSchema = z.object({
export type CoderMeta = z.infer<typeof coderMetaSchema>;
const CODER_SYSTEM = `You are a **coder**. Read the thread for the plan and work on the NEXT incomplete phase only.
## Finding the current thread ID
The thread ID is a 26-character Crockford Base32 string (e.g. \`06F03H5V6JTMDST6P3TVH42RWM\`). It appears in the first message of this conversation. If you are unsure, run:
uncaged-workflow threads
and use the ID of the active thread.
## Reading phase details
Each planner phase is identified by a content-hash and a title. To read a phase's full details (name, description, acceptance criteria), run:
uncaged-workflow cas get <thread-id> <hash>
uncaged-workflow cas get <THREAD_ID> <HASH>
Replace \`<THREAD_ID>\` with the actual thread ID and \`<HASH>\` with the phase hash from the plan.
## Completing a phase
Report which phase you completed using the phase **hash** (not the title). If you legitimately finish every remaining phase in this single turn, set completedPhase to the **last** phase hash in the plan (the workflow treats that as full completion). List the files you changed and summarize what you did.`;
@@ -23,4 +38,5 @@ export const coderRole: RoleDefinition<CoderMeta> = {
extractPrompt:
"Extract completedPhase: the planner phase hash finished this round (exact hash string from the plan). If multiple phases were finished in one round, use the last finished phase hash. Extract filesChanged and a summary of the work.",
schema: coderMetaSchema,
extractRefs: (meta) => [meta.completedPhase],
};
@@ -31,4 +31,5 @@ export const committerRole: RoleDefinition<CommitterMeta> = {
extractPrompt:
"Extract the commit result: committed (with branch and SHA), recoverable failure, or unrecoverable failure. Include error details and log references if applicable.",
schema: committerMetaSchema,
extractRefs: null,
};
+24 -5
View File
@@ -14,16 +14,34 @@ export type PlannerMeta = z.infer<typeof plannerMetaSchema>;
const PLANNER_SYSTEM = `You are a **planner** for a software task. Break the work into **sequential phases** the coder will execute one at a time.
For each phase, decide on a name, detailed description, and acceptance criteria. Then store the full detail text in CAS so the coder can retrieve it later:
## Finding the current thread ID
uncaged-workflow cas put <thread-id> "# <name>\n\nDescription: <description>\n\nAcceptance: <acceptance>"
The thread ID is a 26-character Crockford Base32 string (e.g. \`06F03H5V6JTMDST6P3TVH42RWM\`). It appears in the first message of this conversation. If you are unsure, run:
The command prints a content-hash to stdout. Use that hash as the phase identifier.
uncaged-workflow threads
Your final structured output must contain compact phases only:
and use the ID of the active thread.
## Storing phase details — MANDATORY
For each phase you MUST store its full detail text in CAS using this exact CLI command:
uncaged-workflow cas put <THREAD_ID> '# <name>
Description: <description>
Acceptance: <acceptance>'
Replace \`<THREAD_ID>\` with the actual thread ID you found above. The command prints a content-hash to stdout — use that hash as the phase identifier.
**Do NOT store phase details in any other way** (no temp files, no invented paths). The CLI command is the only supported storage mechanism.
## Output format
After storing all phases via the CLI, output compact JSON only:
{ "phases": [{ "hash": "<hash-from-cas-put>", "title": "<one-line-summary>" }] }
The current thread ID is provided in the thread context. Order phases so earlier steps unblock later ones. Cover root cause, edge cases, and verification across the phases.`;
Order phases so earlier steps unblock later ones. Cover root cause, edge cases, and verification across the phases.`;
export const plannerRole: RoleDefinition<PlannerMeta> = {
description: "Breaks the task into sequential phases for the coder.",
@@ -31,4 +49,5 @@ export const plannerRole: RoleDefinition<PlannerMeta> = {
extractPrompt:
"Extract the implementation phases from the agent's output. Each phase has a hash (the CAS content-hash returned by the cas put command) and a title (one-line summary).",
schema: plannerMetaSchema,
extractRefs: (meta) => meta.phases.map((p) => p.hash),
};
@@ -0,0 +1,15 @@
{
"name": "@uncaged/workflow-role-preparer",
"version": "0.1.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"scripts": {
"build": "echo 'TODO'",
"test": "echo no tests"
},
"dependencies": {
"@uncaged/workflow": "workspace:*",
"zod": "^4.0.0"
}
}
@@ -0,0 +1,5 @@
export {
type PreparerMeta,
preparerMetaSchema,
preparerRole,
} from "./preparer.js";
@@ -0,0 +1,51 @@
import type { RoleDefinition } from "@uncaged/workflow";
import * as z from "zod/v4";
const toolchainSchema = z.object({
packageManager: z.union([z.string(), z.null()]),
testCommand: z.union([z.string(), z.null()]),
lintCommand: z.union([z.string(), z.null()]),
buildCommand: z.union([z.string(), z.null()]),
});
export const preparerMetaSchema = z.object({
repoPath: z.string(),
defaultBranch: z.string(),
conventions: z.union([z.string(), z.null()]),
toolchain: toolchainSchema,
});
export type PreparerMeta = z.infer<typeof preparerMetaSchema>;
const PREPARER_SYSTEM = `You are a **preparer** for a software task. Your job is to locate (or clone) the target repository locally, ensure it is up to date, and gather project context before work begins.
## Responsibilities
1. Parse the issue/task prompt to identify the target repository (URL, org/repo, or name).
2. Search for an existing local clone in these locations (in order):
- ~/Code/<repo-name>/
- ~/repos/<repo-name>/
- ~/Code/<org>/<repo-name>/
- ~/repos/<org>/<repo-name>/
3. If not found locally, \`git clone\` it into ~/repos/<repo-name>/.
4. \`git checkout main && git pull\` (or the default branch) to ensure latest.
5. Read project conventions: \`CLAUDE.md\`, \`CONTRIBUTING.md\`, \`.cursor/rules/*.mdc\`, \`CONVENTIONS.md\`.
6. Detect toolchain: package manager, test runner, linter, build system.
## Output
Report your findings as structured data:
- **repoPath**: absolute path to the local repo
- **defaultBranch**: the default branch name (e.g. "main")
- **conventions**: a summary of project conventions found, or null if none
- **toolchain**: detected commands for packageManager, testCommand, lintCommand, buildCommand (null if not detected)`;
export const preparerRole: RoleDefinition<PreparerMeta> = {
description:
"Locates or clones the target repository, ensures it is up to date, and gathers project context (conventions, toolchain).",
systemPrompt: PREPARER_SYSTEM,
extractPrompt:
"Extract repoPath (absolute path), defaultBranch, conventions (summary string or null), and toolchain (packageManager, testCommand, lintCommand, buildCommand — each string or null).",
schema: preparerMetaSchema,
extractRefs: null,
};
@@ -0,0 +1,8 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "dist"
},
"include": ["src"]
}
@@ -21,4 +21,5 @@ export const reviewerRole: RoleDefinition<ReviewerMeta> = {
extractPrompt:
"Extract the review verdict: approved or rejected. If rejected, list the blocking issues.",
schema: reviewerMetaSchema,
extractRefs: null,
};
@@ -10,6 +10,7 @@ import {
import type { CoderMeta } from "@uncaged/workflow-role-coder";
import type { PlannerMeta } from "@uncaged/workflow-role-planner";
import type { PreparerMeta } from "@uncaged/workflow-role-preparer";
import { buildSolveIssueDescriptor } from "../src/descriptor.js";
import { createSolveIssueRun, solveIssueModerator } from "../src/index.js";
@@ -103,16 +104,38 @@ function makeCtx(
): ModeratorContext<SolveIssueMeta> {
return {
threadId: "01TEST000000000000000000TR",
depth: 0,
start: makeStart(maxRounds),
steps,
};
}
function preparerStep(): RoleStep<SolveIssueMeta> {
return {
role: "preparer",
content: "prepared",
meta: {
repoPath: "/home/user/repos/test",
defaultBranch: "main",
conventions: null,
toolchain: {
packageManager: "bun",
testCommand: "bun test",
lintCommand: null,
buildCommand: "bun run build",
},
},
refs: [],
timestamp: 0,
};
}
function plannerStep(phases: PlannerMeta["phases"] = DEFAULT_PHASES): RoleStep<SolveIssueMeta> {
return {
role: "planner",
content: "plan",
meta: { phases },
refs: phases.map((p) => p.hash),
timestamp: 1,
};
}
@@ -122,6 +145,7 @@ function coderStep(completedPhase = "4KNMR2PX"): RoleStep<SolveIssueMeta> {
role: "coder",
content: "code",
meta: { completedPhase, filesChanged: ["a.ts"], summary: "fixed" },
refs: [completedPhase],
timestamp: 2,
};
}
@@ -133,6 +157,7 @@ function reviewerStep(approved: boolean): RoleStep<SolveIssueMeta> {
meta: approved
? { status: "approved" as const }
: { status: "rejected" as const, issues: ["needs fix"] },
refs: [],
timestamp: 3,
};
}
@@ -142,6 +167,7 @@ function committerStep(): RoleStep<SolveIssueMeta> {
role: "committer",
content: "commit",
meta: { status: "committed", branch: "feat/issue-1", commitSha: "abc1234" },
refs: [],
timestamp: 4,
};
}
@@ -153,16 +179,27 @@ const stubExtract = createExtract({
});
describe("solveIssueModerator", () => {
test("routes planner → coder → reviewer → committer → END", () => {
expect(solveIssueModerator(makeCtx(20, []))).toBe("planner");
expect(solveIssueModerator(makeCtx(20, [plannerStep()]))).toBe("coder");
expect(solveIssueModerator(makeCtx(20, [plannerStep(), coderStep()]))).toBe("reviewer");
expect(solveIssueModerator(makeCtx(20, [plannerStep(), coderStep(), reviewerStep(true)]))).toBe(
"committer",
test("routes preparer → planner → coder → reviewer → committer → END", () => {
expect(solveIssueModerator(makeCtx(20, []))).toBe("preparer");
expect(solveIssueModerator(makeCtx(20, [preparerStep()]))).toBe("planner");
expect(solveIssueModerator(makeCtx(20, [preparerStep(), plannerStep()]))).toBe("coder");
expect(solveIssueModerator(makeCtx(20, [preparerStep(), plannerStep(), coderStep()]))).toBe(
"reviewer",
);
expect(
solveIssueModerator(
makeCtx(20, [plannerStep(), coderStep(), reviewerStep(true), committerStep()]),
makeCtx(20, [preparerStep(), plannerStep(), coderStep(), reviewerStep(true)]),
),
).toBe("committer");
expect(
solveIssueModerator(
makeCtx(20, [
preparerStep(),
plannerStep(),
coderStep(),
reviewerStep(true),
committerStep(),
]),
),
).toBe(END);
});
@@ -250,25 +287,54 @@ describe("createSolveIssueRun", () => {
restoreFetch = null;
});
test("structured extraction yields planner meta from mocked chat completions", async () => {
restoreFetch = installMockChatCompletions([EXPECT_PLANNER_META]);
test("structured extraction yields preparer then planner meta from mocked chat completions", async () => {
const EXPECT_PREPARER_META: PreparerMeta = {
repoPath: "/home/user/repos/test",
defaultBranch: "main",
conventions: null,
toolchain: {
packageManager: "bun",
testCommand: "bun test",
lintCommand: null,
buildCommand: "bun run build",
},
};
restoreFetch = installMockChatCompletions([EXPECT_PREPARER_META, EXPECT_PLANNER_META]);
const run = createSolveIssueRun({ agent: async () => "" }, stubExtract);
const gen = run(
{ prompt: "task", steps: [] },
{ threadId: "01TEST000000000000000000TR", maxRounds: 20 },
{ threadId: "01TEST000000000000000000TR", maxRounds: 20, depth: 0 },
);
const first = await gen.next();
expect(first.done).toBe(false);
if (first.done) {
throw new Error("expected yield");
}
expect(first.value.role).toBe("planner");
expect(first.value.meta).toEqual(EXPECT_PLANNER_META);
expect(first.value.role).toBe("preparer");
expect(first.value.meta).toEqual(EXPECT_PREPARER_META);
const second = await gen.next();
expect(second.done).toBe(false);
if (second.done) {
throw new Error("expected yield");
}
expect(second.value.role).toBe("planner");
expect(second.value.meta).toEqual(EXPECT_PLANNER_META);
});
test("per-role agent overrides default", async () => {
restoreFetch = installMockChatCompletions([EXPECT_PLANNER_META, EXPECT_CODER_META]);
const PREPARER_META: PreparerMeta = {
repoPath: "/tmp/r",
defaultBranch: "main",
conventions: null,
toolchain: { packageManager: null, testCommand: null, lintCommand: null, buildCommand: null },
};
restoreFetch = installMockChatCompletions([
PREPARER_META,
EXPECT_PLANNER_META,
EXPECT_CODER_META,
]);
const calls: string[] = [];
const run = createSolveIssueRun(
@@ -278,6 +344,10 @@ describe("createSolveIssueRun", () => {
return "";
},
overrides: {
preparer: async () => {
calls.push("preparer");
return "";
},
planner: async () => {
calls.push("planner");
return "";
@@ -292,9 +362,13 @@ describe("createSolveIssueRun", () => {
);
const gen = run(
{ prompt: "task", steps: [] },
{ threadId: "01TEST000000000000000000TR", maxRounds: 20 },
{ threadId: "01TEST000000000000000000TR", maxRounds: 20, depth: 0 },
);
await gen.next();
expect(calls).toEqual(["preparer"]);
calls.length = 0;
await gen.next();
expect(calls).toEqual(["planner"]);
calls.length = 0;
@@ -315,9 +389,10 @@ describe("buildSolveIssueDescriptor", () => {
"coder",
"committer",
"planner",
"preparer",
"reviewer",
]);
for (const key of ["planner", "coder", "reviewer", "committer"] as const) {
for (const key of ["preparer", "planner", "coder", "reviewer", "committer"] as const) {
const role = validated.value.roles[key];
expect(role).toBeDefined();
expect(typeof role.schema).toBe("object");
@@ -13,6 +13,7 @@
"@uncaged/workflow-role-committer": "workspace:*",
"@uncaged/workflow-role-coder": "workspace:*",
"@uncaged/workflow-role-planner": "workspace:*",
"@uncaged/workflow-role-preparer": "workspace:*",
"@uncaged/workflow-role-reviewer": "workspace:*"
}
}
@@ -25,6 +25,11 @@ export {
plannerMetaSchema,
plannerRole,
} from "@uncaged/workflow-role-planner";
export {
type PreparerMeta,
preparerMetaSchema,
preparerRole,
} from "@uncaged/workflow-role-preparer";
export {
type ReviewerMeta,
reviewerMetaSchema,
@@ -48,11 +48,15 @@ export const solveIssueModerator: Moderator<SolveIssueMeta> = (ctx) => {
const maxRounds = ctx.start.meta.maxRounds;
if (ctx.steps.length === 0) {
return "planner";
return "preparer";
}
const last = ctx.steps[ctx.steps.length - 1];
if (last.role === "preparer") {
return "planner";
}
if (last.role === "planner") {
return "coder";
}
@@ -2,12 +2,14 @@ import type { RoleDefinition } from "@uncaged/workflow";
import { type CoderMeta, coderRole } from "@uncaged/workflow-role-coder";
import { type CommitterMeta, committerRole } from "@uncaged/workflow-role-committer";
import { type PlannerMeta, plannerRole } from "@uncaged/workflow-role-planner";
import { type PreparerMeta, preparerRole } from "@uncaged/workflow-role-preparer";
import { type ReviewerMeta, reviewerRole } from "@uncaged/workflow-role-reviewer";
export const SOLVE_ISSUE_WORKFLOW_DESCRIPTION =
"Phased plan, incremental implementation per phase, review, and commit to resolve an issue end-to-end (planner → coder [repeat per phase] → reviewer → committer).";
"Prepare repo context, plan phases, implement incrementally, review, and commit to resolve an issue end-to-end (preparer → planner → coder [repeat per phase] → reviewer → committer).";
export type SolveIssueMeta = {
preparer: PreparerMeta;
planner: PlannerMeta;
coder: CoderMeta;
reviewer: ReviewerMeta;
@@ -19,6 +21,7 @@ export type SolveIssueRoles = {
};
export const solveIssueRoles: SolveIssueRoles = {
preparer: preparerRole,
planner: plannerRole,
coder: coderRole,
reviewer: reviewerRole,
@@ -16,6 +16,7 @@ describe("buildAgentPrompt", () => {
test("includes system prompt and full task; omits tools when there are no steps", () => {
const ctx: ThreadContext = {
start: startTask("fix the bug"),
depth: 0,
steps: [],
threadId: "01TEST000000000000000000TR",
currentRole: { name: START, systemPrompt: "You are an agent." },
@@ -30,6 +31,7 @@ describe("buildAgentPrompt", () => {
test("single step shows full content and meta, and includes tools", () => {
const ctx: ThreadContext = {
start: startTask("user task"),
depth: 0,
threadId: "01TEST000000000000000000TR",
currentRole: { name: "coder", systemPrompt: "Be helpful." },
steps: [
@@ -37,6 +39,7 @@ describe("buildAgentPrompt", () => {
role: "coder",
content: "only step full body",
meta: { files: ["a.ts"] },
refs: [],
timestamp: 2,
},
],
@@ -54,6 +57,7 @@ describe("buildAgentPrompt", () => {
test("two or more steps: previous steps are meta-only; latest step is full", () => {
const ctx: ThreadContext = {
start: startTask("first message full: task content here"),
depth: 0,
threadId: "01TEST000000000000000000TR",
currentRole: { name: "coder", systemPrompt: "System." },
steps: [
@@ -61,12 +65,14 @@ describe("buildAgentPrompt", () => {
role: "planner",
content: "PLANNER_SECRET_FULL_TEXT",
meta: { plan: "short" },
refs: [],
timestamp: 2,
},
{
role: "coder",
content: "last step full content",
meta: { done: true },
refs: [],
timestamp: 3,
},
],
@@ -87,6 +93,7 @@ describe("buildAgentPrompt", () => {
test("middle steps show meta summary only, not full content", () => {
const ctx: ThreadContext = {
start: startTask("start"),
depth: 0,
threadId: "01TEST000000000000000000TR",
currentRole: { name: "c", systemPrompt: "S" },
steps: [
@@ -94,18 +101,21 @@ describe("buildAgentPrompt", () => {
role: "a",
content: "HIDDEN_A",
meta: { n: 1 },
refs: [],
timestamp: 2,
},
{
role: "b",
content: "HIDDEN_B_MIDDLE",
meta: { n: 2 },
refs: [],
timestamp: 3,
},
{
role: "c",
content: "VISIBLE_LAST",
meta: { n: 3 },
refs: [],
timestamp: 4,
},
],
@@ -22,6 +22,7 @@ describe("buildDescriptor", () => {
systemPrompt: "You are an analyst.",
extractPrompt: "Extract title and count from the analysis.",
schema,
extractRefs: null,
},
},
moderator: () => END,
+18 -12
View File
@@ -3,10 +3,16 @@ import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createThreadCas } from "../src/cas.js";
import { createCasStore, createThreadCas } from "../src/cas.js";
import { hashString } from "../src/hash.js";
describe("createThreadCas", () => {
describe("cas module exports", () => {
test("createThreadCas is a deprecated alias of createCasStore", () => {
expect(createThreadCas).toBe(createCasStore);
});
});
describe("createCasStore", () => {
let casDir: string;
beforeEach(async () => {
@@ -18,7 +24,7 @@ describe("createThreadCas", () => {
});
test("put returns consistent hash for same content", async () => {
const cas = createThreadCas(casDir);
const cas = createCasStore(casDir);
const h1 = await cas.put("hello world");
const h2 = await cas.put("hello world");
expect(h1).toBe(h2);
@@ -26,14 +32,14 @@ describe("createThreadCas", () => {
});
test("put returns hash matching hashString", async () => {
const cas = createThreadCas(casDir);
const cas = createCasStore(casDir);
const content = "some content to store";
const h = await cas.put(content);
expect(h).toBe(hashString(content));
});
test("get returns stored content", async () => {
const cas = createThreadCas(casDir);
const cas = createCasStore(casDir);
const content = "line1\nline2\nline3";
const h = await cas.put(content);
const retrieved = await cas.get(h);
@@ -41,13 +47,13 @@ describe("createThreadCas", () => {
});
test("get returns null for missing hash", async () => {
const cas = createThreadCas(casDir);
const cas = createCasStore(casDir);
const result = await cas.get("0000000000000");
expect(result).toBeNull();
});
test("delete removes entry", async () => {
const cas = createThreadCas(casDir);
const cas = createCasStore(casDir);
const h = await cas.put("to be deleted");
await cas.delete(h);
const result = await cas.get(h);
@@ -55,12 +61,12 @@ describe("createThreadCas", () => {
});
test("delete on missing hash does not throw", async () => {
const cas = createThreadCas(casDir);
const cas = createCasStore(casDir);
await cas.delete("0000000000000");
});
test("list returns all stored hashes", async () => {
const cas = createThreadCas(casDir);
const cas = createCasStore(casDir);
const h1 = await cas.put("aaa");
const h2 = await cas.put("bbb");
const h3 = await cas.put("ccc");
@@ -69,13 +75,13 @@ describe("createThreadCas", () => {
});
test("list returns empty array when cas dir does not exist", async () => {
const cas = createThreadCas(join(casDir, "nonexistent"));
const cas = createCasStore(join(casDir, "nonexistent"));
const hashes = await cas.list();
expect(hashes).toEqual([]);
});
test("put is idempotent — same content written twice causes no error", async () => {
const cas = createThreadCas(casDir);
const cas = createCasStore(casDir);
const h1 = await cas.put("idempotent");
const h2 = await cas.put("idempotent");
expect(h1).toBe(h2);
@@ -84,7 +90,7 @@ describe("createThreadCas", () => {
});
test("different content produces different hashes", async () => {
const cas = createThreadCas(casDir);
const cas = createCasStore(casDir);
const h1 = await cas.put("alpha");
const h2 = await cas.put("beta");
expect(h1).not.toBe(h2);
+12 -1
View File
@@ -89,12 +89,14 @@ const demoWorkflow = createWorkflow<DemoMeta>(
systemPrompt: "You are a planner.",
extractPrompt: "Extract plan text and affected files list.",
schema: plannerMetaSchema,
extractRefs: null,
},
coder: {
description: "Demo coder",
systemPrompt: "You are a coder.",
extractPrompt: "Extract the code diff summary.",
schema: coderMetaSchema,
extractRefs: null,
},
},
moderator: (ctx) => {
@@ -148,6 +150,7 @@ describe("executeThread", () => {
{ prompt: "Fix the login redirect bug in #3", steps: [] },
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
@@ -176,16 +179,19 @@ describe("executeThread", () => {
expect(params.prompt).toBe("Fix the login redirect bug in #3");
const opts = params.options as Record<string, unknown>;
expect(opts.maxRounds).toBe(5);
expect(Object.keys(opts).sort()).toEqual(["maxRounds"]);
expect(opts.depth).toBe(0);
expect(Object.keys(opts).sort()).toEqual(["depth", "maxRounds"]);
const role1 = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
expect(role1.role).toBe("planner");
expect(role1.content).toBe("plan-body");
expect(role1.meta).toEqual({ plan: "do-it", files: ["a.ts"] });
expect(role1.refs).toEqual([]);
expect(typeof role1.timestamp).toBe("number");
const role2 = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
expect(role2.role).toBe("coder");
expect(role2.refs).toEqual([]);
const infoText = await readFile(infoPath, "utf8");
const infoLines = infoText
@@ -228,11 +234,13 @@ describe("executeThread", () => {
role: "planner",
content: "plan-body",
meta: { plan: "do-it", files: ["a.ts"] },
refs: ["CAS111AAAAAAA"],
},
],
},
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: "01SRC1111111111111111111",
@@ -241,6 +249,7 @@ describe("executeThread", () => {
role: "planner",
content: "plan-body",
meta: { plan: "do-it", files: ["a.ts"] },
refs: ["CAS111AAAAAAA"],
timestamp: histTs,
},
],
@@ -264,6 +273,7 @@ describe("executeThread", () => {
const role0 = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
expect(role0.role).toBe("planner");
expect(role0.timestamp).toBe(histTs);
expect(role0.refs).toEqual(["CAS111AAAAAAA"]);
const role1 = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
expect(role1.role).toBe("coder");
@@ -291,6 +301,7 @@ describe("executeThread", () => {
{ prompt: "hello", steps: [] },
{
maxRounds: 0,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
@@ -24,6 +24,7 @@ describe("fork-thread", () => {
expect(r.value.start.threadId).toBe("01AAA1111111111111111111");
expect(r.value.start.prompt).toBe("hi");
expect(r.value.start.maxRounds).toBe(5);
expect(r.value.start.depth).toBe(0);
expect(r.value.roleSteps.length).toBe(3);
expect(r.value.roleSteps[0]?.role).toBe("planner");
});
@@ -83,6 +84,24 @@ describe("fork-thread", () => {
expect(r.value.workflowName).toBe("demo");
expect(r.value.historicalSteps.length).toBe(1);
expect(r.value.historicalSteps[0]?.timestamp).toBe(101);
expect(r.value.runOptions).toEqual({ maxRounds: 5 });
expect(r.value.runOptions).toEqual({ maxRounds: 5, depth: 0 });
});
test("parseThreadDataJsonl reads explicit depth from start record", () => {
const text = `{"name":"demo","hash":"H","threadId":"01ZZZZZZZZZZZZZZZZZZZZZZ","parameters":{"prompt":"p","options":{"maxRounds":3,"depth":2}},"timestamp":1}
{"role":"planner","content":"x","meta":{},"timestamp":2}
`;
const r = parseThreadDataJsonl(text);
expect(r.ok).toBe(true);
if (!r.ok) {
return;
}
expect(r.value.start.depth).toBe(2);
const plan = buildForkPlan(text, null);
expect(plan.ok).toBe(true);
if (!plan.ok) {
return;
}
expect(plan.value.runOptions).toEqual({ maxRounds: 3, depth: 2 });
});
});
@@ -0,0 +1,192 @@
import { afterEach, describe, expect, test } from "bun:test";
import { mkdir, mkdtemp, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import * as z from "zod/v4";
import { createWorkflow } from "../src/create-workflow.js";
import { executeThread } from "../src/engine.js";
import { createExtract } from "../src/extract-fn.js";
import { buildForkPlan, parseThreadDataJsonl } from "../src/fork-thread.js";
import { createLogger } from "../src/logger.js";
import { END } from "../src/types.js";
const phaseSchema = z.object({
hash: z.string(),
title: z.string(),
});
const plannerMetaSchema = z.object({
phases: z.array(phaseSchema),
});
type RefsDemoMeta = {
planner: z.infer<typeof plannerMetaSchema>;
};
function installMockChatCompletions(sequence: ReadonlyArray<Record<string, unknown>>): () => void {
const origFetch = globalThis.fetch;
let i = 0;
const mockFetch = async (
input: Parameters<typeof fetch>[0],
init?: RequestInit,
): Promise<Response> => {
const args = sequence[i] ?? sequence[sequence.length - 1];
if (args === undefined) {
throw new Error("installMockChatCompletions: empty sequence");
}
i += 1;
void input;
const body = init?.body ? (JSON.parse(String(init.body)) as Record<string, unknown>) : {};
const tools = body.tools;
const firstTool =
Array.isArray(tools) && tools.length > 0 && tools[0] !== null && typeof tools[0] === "object"
? (tools[0] as Record<string, unknown>)
: null;
const fn =
firstTool !== null ? (firstTool.function as Record<string, unknown> | undefined) : undefined;
const toolName = typeof fn?.name === "string" ? fn.name : "extract";
return new Response(
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
type: "function",
function: {
name: toolName,
arguments: JSON.stringify(args),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
};
globalThis.fetch = Object.assign(mockFetch, {
preconnect: origFetch.preconnect.bind(origFetch),
}) as typeof fetch;
return () => {
globalThis.fetch = origFetch;
};
}
const refsDemoExtract = createExtract({
baseUrl: "http://127.0.0.1:9",
apiKey: "test",
model: "test",
});
const refsDemoWorkflow = createWorkflow<RefsDemoMeta>(
{
roles: {
planner: {
description: "Planner with phase hashes",
systemPrompt: "Plan.",
extractPrompt: "Extract phases with CAS hashes.",
schema: plannerMetaSchema,
extractRefs: (meta) => meta.phases.map((p) => p.hash),
},
},
moderator: (ctx) => (ctx.steps.length === 0 ? "planner" : END),
},
{
agent: async () => "plan-output",
},
refsDemoExtract,
);
describe("RoleStep refs tracking", () => {
let restoreFetch: (() => void) | null = null;
afterEach(() => {
restoreFetch?.();
restoreFetch = null;
});
test("parseThreadDataJsonl reads refs and defaults missing refs to []", () => {
const text = `{"name":"demo","hash":"C9NMV6V2TQT81","threadId":"01AAA1111111111111111111","parameters":{"prompt":"hi","options":{"maxRounds":5}},"timestamp":100}
{"role":"planner","content":"p","meta":{},"refs":["H111AAAAAAAAA","H222AAAAAAAAA"],"timestamp":101}
{"role":"coder","content":"c","meta":{},"timestamp":102}
`;
const r = parseThreadDataJsonl(text);
expect(r.ok).toBe(true);
if (!r.ok) {
return;
}
expect(r.value.roleSteps[0]?.refs).toEqual(["H111AAAAAAAAA", "H222AAAAAAAAA"]);
expect(r.value.roleSteps[1]?.refs).toEqual([]);
});
test("executeThread persists refs from extractRefs on role yields", async () => {
restoreFetch = installMockChatCompletions([
{
phases: [
{ hash: "C9NMV6V2TQT81", title: "phase-a" },
{ hash: "C9NMV6V2TQT82", title: "phase-b" },
],
},
]);
const root = await mkdtemp(join(tmpdir(), "wf-refs-"));
try {
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
const hash = "C9NMV6V2TQT81";
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
await mkdir(join(root, "logs", hash), { recursive: true });
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
const ac = new AbortController();
const result = await executeThread(
refsDemoWorkflow,
"refs-demo",
{ prompt: "task", steps: [] },
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
prefilledDiskSteps: null,
},
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath },
logger,
);
expect(result.returnCode).toBe(0);
const dataText = await readFile(dataPath, "utf8");
const lines = dataText
.trim()
.split("\n")
.filter((l) => l !== "");
expect(lines.length).toBe(2);
const role1 = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
expect(role1.role).toBe("planner");
expect(role1.refs).toEqual(["C9NMV6V2TQT81", "C9NMV6V2TQT82"]);
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("buildForkPlan carries refs on historical steps", () => {
const text = `{"name":"demo","hash":"C9NMV6V2TQT81","threadId":"01AAA1111111111111111111","parameters":{"prompt":"hi","options":{"maxRounds":5}},"timestamp":100}
{"role":"planner","content":"p","meta":{},"refs":["KEEPREFAAAAAA"],"timestamp":101}
{"role":"coder","content":"c","meta":{},"refs":["CODERHASHAAAA"],"timestamp":102}
`;
const plan = buildForkPlan(text, null);
expect(plan.ok).toBe(true);
if (!plan.ok) {
return;
}
expect(plan.value.historicalSteps.length).toBe(1);
expect(plan.value.historicalSteps[0]?.refs).toEqual(["KEEPREFAAAAAA"]);
});
});
@@ -0,0 +1,14 @@
import { describe, expect, test } from "bun:test";
import { join } from "node:path";
import { getDefaultWorkflowStorageRoot, getGlobalCasDir } from "../src/storage-root.js";
describe("getGlobalCasDir", () => {
test("joins cas segment under explicit storage root", () => {
expect(getGlobalCasDir("/tmp/wf-root")).toBe(join("/tmp/wf-root", "cas"));
});
test("defaults to default workflow root when storage root is undefined", () => {
expect(getGlobalCasDir(undefined)).toBe(join(getDefaultWorkflowStorageRoot(), "cas"));
});
});
@@ -10,6 +10,7 @@ describe("RFC-001 thread JSONL shapes", () => {
prompt: "Fix the login redirect bug in #3",
options: {
maxRounds: 5,
depth: 0,
},
},
timestamp: 1714963200000,
@@ -19,13 +20,16 @@ describe("RFC-001 thread JSONL shapes", () => {
role: "planner",
content: "Plan: modify auth middleware...",
meta: { plan: "...", files: ["src/auth.ts"] },
refs: [] as string[],
timestamp: 1714963201000,
};
expect(Object.keys(startRecord).sort()).toEqual(
["hash", "name", "parameters", "threadId", "timestamp"].sort(),
);
expect(Object.keys(roleRecord).sort()).toEqual(["content", "meta", "role", "timestamp"].sort());
expect(Object.keys(roleRecord).sort()).toEqual(
["content", "meta", "refs", "role", "timestamp"].sort(),
);
});
test("documents the `.info.jsonl` debug record keys", () => {
@@ -0,0 +1,206 @@
import { afterEach, describe, expect, test } from "bun:test";
import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import * as z from "zod/v4";
import { createWorkflow } from "../src/create-workflow.js";
import { executeThread } from "../src/engine.js";
import { createExtract } from "../src/extract-fn.js";
import { hashWorkflowBundleBytes } from "../src/hash.js";
import { createLogger } from "../src/logger.js";
import {
readWorkflowRegistry,
registerWorkflowVersion,
writeWorkflowRegistry,
} from "../src/registry.js";
import { END } from "../src/types.js";
import { workflowAsAgent } from "../src/workflow-as-agent.js";
const callerMetaSchema = z.object({ done: z.literal(true) });
type ParentMeta = {
caller: z.infer<typeof callerMetaSchema>;
};
function installMockChatCompletions(sequence: ReadonlyArray<Record<string, unknown>>): () => void {
const origFetch = globalThis.fetch;
let i = 0;
const mockFetch = async (
input: Parameters<typeof fetch>[0],
init?: RequestInit,
): Promise<Response> => {
const args = sequence[i] ?? sequence[sequence.length - 1];
if (args === undefined) {
throw new Error("installMockChatCompletions: empty sequence");
}
i += 1;
void input;
const body = init?.body ? (JSON.parse(String(init.body)) as Record<string, unknown>) : {};
const tools = body.tools;
const firstTool =
Array.isArray(tools) && tools.length > 0 && tools[0] !== null && typeof tools[0] === "object"
? (tools[0] as Record<string, unknown>)
: null;
const fn =
firstTool !== null ? (firstTool.function as Record<string, unknown> | undefined) : undefined;
const toolName = typeof fn?.name === "string" ? fn.name : "extract";
return new Response(
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
type: "function",
function: {
name: toolName,
arguments: JSON.stringify(args),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
};
globalThis.fetch = Object.assign(mockFetch, {
preconnect: origFetch.preconnect.bind(origFetch),
}) as typeof fetch;
return () => {
globalThis.fetch = origFetch;
};
}
const parentExtract = createExtract({
baseUrl: "http://127.0.0.1:9",
apiKey: "test",
model: "test",
});
const childBundleSource = `export const descriptor = {
description: "child-integration",
roles: {
agent: {
description: "agent",
schema: { type: "object", properties: {}, additionalProperties: true },
},
},
};
export async function* run(input) {
yield { role: "agent", content: "child-body", meta: {}, refs: [] };
return { returnCode: 0, summary: "child-done:" + input.prompt };
}
`;
async function installChildWorkflow(storageRoot: string): Promise<{ hash: string }> {
const bytes = new TextEncoder().encode(childBundleSource);
const hash = hashWorkflowBundleBytes(bytes);
await mkdir(join(storageRoot, "bundles"), { recursive: true });
await writeFile(join(storageRoot, "bundles", `${hash}.esm.js`), childBundleSource, "utf8");
const reg = await readWorkflowRegistry(storageRoot);
if (!reg.ok) {
throw reg.error;
}
const next = registerWorkflowVersion(reg.value, "child-wf", hash, Date.now());
const wr = await writeWorkflowRegistry(storageRoot, next);
if (!wr.ok) {
throw wr.error;
}
return { hash };
}
describe("workflowAsAgent integration", () => {
let restoreFetch: (() => void) | null = null;
afterEach(() => {
restoreFetch?.();
restoreFetch = null;
});
test("createWorkflow parent invokes nested workflow via workflowAsAgent", async () => {
restoreFetch = installMockChatCompletions([{ done: true }]);
const root = await mkdtemp(join(tmpdir(), "wf-waa-int-"));
try {
const { hash: childHash } = await installChildWorkflow(root);
const parentWorkflow = createWorkflow<ParentMeta>(
{
roles: {
caller: {
description: "delegates to child workflow",
systemPrompt: "system",
extractPrompt: "extract done flag",
schema: callerMetaSchema,
extractRefs: null,
},
},
moderator: (ctx) => (ctx.steps.length === 0 ? "caller" : END),
},
{ agent: workflowAsAgent("child-wf", { storageRoot: root }) },
parentExtract,
);
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
const parentHash = "C9NMV6V2TQT81";
const dataPath = join(root, "logs", parentHash, `${threadId}.data.jsonl`);
const infoPath = join(root, "logs", parentHash, `${threadId}.info.jsonl`);
await mkdir(join(root, "logs", parentHash), { recursive: true });
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
const ac = new AbortController();
const result = await executeThread(
parentWorkflow,
"parent-wf",
{ prompt: "from-parent", steps: [] },
{
maxRounds: 5,
depth: 0,
signal: ac.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: null,
prefilledDiskSteps: null,
},
{ threadId, hash: parentHash, dataJsonlPath: dataPath, infoJsonlPath: infoPath },
logger,
);
expect(result.returnCode).toBe(0);
const parentText = await readFile(dataPath, "utf8");
const parentLines = parentText
.trim()
.split("\n")
.filter((l) => l !== "");
expect(parentLines.length).toBe(2);
const callerLine = JSON.parse(parentLines[1] ?? "{}") as Record<string, unknown>;
expect(callerLine.role).toBe("caller");
expect(callerLine.content).toBe("child-done:from-parent");
const childDir = join(root, "logs", childHash);
const childFiles = await readdir(childDir);
const childDataName = childFiles.find((n) => n.endsWith(".data.jsonl"));
expect(childDataName).toBeDefined();
const childText = await readFile(join(childDir, childDataName ?? ""), "utf8");
const childStart = JSON.parse(
childText
.trim()
.split("\n")
.filter((l) => l !== "")[0] ?? "{}",
) as Record<string, unknown>;
expect(childStart.forkFrom).toEqual({ threadId });
const childOpts = (childStart.parameters as Record<string, unknown>).options as Record<
string,
unknown
>;
expect(childOpts.depth).toBe(1);
} finally {
await rm(root, { recursive: true, force: true });
}
});
});
@@ -0,0 +1,101 @@
import { describe, expect, test } from "bun:test";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { hashWorkflowBundleBytes } from "../src/hash.js";
import {
readWorkflowRegistry,
registerWorkflowVersion,
writeWorkflowRegistry,
} from "../src/registry.js";
import { type AgentContext, START } from "../src/types.js";
import { workflowAsAgent } from "../src/workflow-as-agent.js";
function makeAgentCtx(params: { depth: number; prompt: string; maxRounds: number }): AgentContext {
const ts = Date.now();
return {
threadId: "01PARENT000000000000000001AA",
depth: params.depth,
start: {
role: START,
content: params.prompt,
meta: { maxRounds: params.maxRounds },
timestamp: ts,
},
steps: [],
currentRole: {
name: "caller",
systemPrompt: "caller",
},
};
}
const childBundleSource = `export const descriptor = {
description: "child-test",
roles: {
agent: {
description: "agent",
schema: { type: "object", properties: {}, additionalProperties: true },
},
},
};
export async function* run(input) {
yield { role: "agent", content: "child-body", meta: {}, refs: [] };
return { returnCode: 0, summary: "child-done:" + input.prompt };
}
`;
async function installChildWorkflow(storageRoot: string): Promise<{ hash: string }> {
const bytes = new TextEncoder().encode(childBundleSource);
const hash = hashWorkflowBundleBytes(bytes);
await mkdir(join(storageRoot, "bundles"), { recursive: true });
await writeFile(join(storageRoot, "bundles", `${hash}.esm.js`), childBundleSource, "utf8");
const reg = await readWorkflowRegistry(storageRoot);
if (!reg.ok) {
throw reg.error;
}
const next = registerWorkflowVersion(reg.value, "child-wf", hash, Date.now());
const wr = await writeWorkflowRegistry(storageRoot, next);
if (!wr.ok) {
throw wr.error;
}
return { hash };
}
describe("workflowAsAgent", () => {
test("returns error when workflow name is not registered", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-waa-missing-"));
try {
const agent = workflowAsAgent("missing-wf", { storageRoot: root });
const out = await agent(makeAgentCtx({ depth: 0, prompt: "x", maxRounds: 5 }));
expect(out).toContain("not found in registry");
expect(out).toContain("missing-wf");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("runs registered workflow and returns child summary string", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-waa-ok-"));
try {
await installChildWorkflow(root);
const agent = workflowAsAgent("child-wf", { storageRoot: root });
const out = await agent(makeAgentCtx({ depth: 0, prompt: "hello-parent", maxRounds: 5 }));
expect(out).toBe("child-done:hello-parent");
} finally {
await rm(root, { recursive: true, force: true });
}
});
test("enforces depth limit (returns error string, does not throw)", async () => {
const root = await mkdtemp(join(tmpdir(), "wf-waa-depth-"));
try {
const agent = workflowAsAgent("child-wf", { storageRoot: root });
const out = await agent(makeAgentCtx({ depth: 3, prompt: "x", maxRounds: 5 }));
expect(out).toContain("depth limit");
} finally {
await rm(root, { recursive: true, force: true });
}
});
});
+4 -1
View File
@@ -10,7 +10,7 @@ export type CasStore = {
list(): Promise<string[]>;
};
export function createThreadCas(casDir: string): CasStore {
export function createCasStore(casDir: string): CasStore {
async function ensureDir(): Promise<void> {
await mkdir(casDir, { recursive: true });
}
@@ -68,3 +68,6 @@ export function createThreadCas(casDir: string): CasStore {
},
};
}
/** @deprecated Use {@link createCasStore} — CAS is global, not per-thread. */
export const createThreadCas = createCasStore;
+21 -1
View File
@@ -5,6 +5,7 @@ import {
END,
type ExtractContext,
type ModeratorContext,
type RoleDefinition,
type RoleMeta,
type RoleOutput,
type RoleStep,
@@ -22,6 +23,17 @@ function isRoleNext<M extends RoleMeta>(
return next !== END;
}
function resolveExtractedRefs(
roleDef: RoleDefinition<Record<string, unknown>>,
meta: unknown,
): string[] {
const extractRefsFn = roleDef.extractRefs;
if (extractRefsFn === null || typeof extractRefsFn !== "function") {
return [];
}
return extractRefsFn(meta as Record<string, unknown>);
}
/**
* Binds pure role definitions + moderator to runtime agents and structured extraction.
* Assign with `export const run = createWorkflow(def, binding, extract)`.
@@ -48,6 +60,7 @@ export function createWorkflow<M extends RoleMeta>(
role: out.role,
content: out.content,
meta: out.meta,
refs: out.refs,
timestamp: baseTs + i,
})) as RoleStep<M>[];
@@ -61,6 +74,7 @@ export function createWorkflow<M extends RoleMeta>(
const modCtx: ModeratorContext<M> = {
threadId: options.threadId,
depth: options.depth,
start,
steps,
};
@@ -96,15 +110,21 @@ export function createWorkflow<M extends RoleMeta>(
extractCtx as unknown as ExtractContext,
);
const refs = resolveExtractedRefs(
roleDef as unknown as RoleDefinition<Record<string, unknown>>,
meta,
);
const ts = Date.now();
const step = {
role: next,
content: raw,
meta,
refs,
timestamp: ts,
} as RoleStep<M>;
yield { role: step.role, content: step.content, meta: step.meta };
yield { role: step.role, content: step.content, meta: step.meta, refs: step.refs };
steps = [...steps, step];
}
+8
View File
@@ -2,6 +2,7 @@ import { appendFile, mkdir } from "node:fs/promises";
import { dirname } from "node:path";
import type { LogFn } from "./logger.js";
import { normalizeRefsField } from "./refs-field.js";
import type { ThreadInput, WorkflowFn, WorkflowFnOptions, WorkflowResult } from "./types.js";
export type ExecuteThreadIo = {
@@ -16,11 +17,14 @@ export type PrefilledDiskStep = {
role: string;
content: string;
meta: Record<string, unknown>;
refs: string[];
timestamp: number;
};
export type ExecuteThreadOptions = {
maxRounds: number;
/** Passed to the bundle as `WorkflowFnOptions.depth`. */
depth: number;
signal: AbortSignal;
/** Invoked after each successful yield (and outer-loop checks); used for pause/resume. */
awaitAfterEachYield: () => Promise<void>;
@@ -79,6 +83,7 @@ async function driveWorkflowGenerator(params: {
role: step.role,
content: step.content,
meta: step.meta,
refs: normalizeRefsField(step.refs),
timestamp: ts,
});
@@ -133,6 +138,7 @@ export async function executeThread(
prompt: input.prompt,
options: {
maxRounds: options.maxRounds,
depth: options.depth,
},
},
timestamp: nowMs,
@@ -151,6 +157,7 @@ export async function executeThread(
role: row.role,
content: row.content,
meta: row.meta,
refs: normalizeRefsField(row.refs),
timestamp: row.timestamp,
});
}
@@ -167,6 +174,7 @@ export async function executeThread(
const bundleOptions: WorkflowFnOptions = {
threadId: io.threadId,
maxRounds: options.maxRounds,
depth: options.depth,
};
return await driveWorkflowGenerator({
+10 -2
View File
@@ -1,3 +1,4 @@
import { normalizeRefsField } from "./refs-field.js";
import { err, ok, type Result } from "./result.js";
import type { RoleOutput } from "./types.js";
@@ -10,6 +11,7 @@ export type ParsedThreadStartRecord = {
threadId: string;
prompt: string;
maxRounds: number;
depth: number;
};
function parseRoleLine(
@@ -36,6 +38,7 @@ function parseRoleLine(
role,
content,
meta: meta as Record<string, unknown>,
refs: normalizeRefsField(obj.refs),
timestamp,
});
}
@@ -76,12 +79,17 @@ function parseStartRecordLine(firstLine: string): Result<ParsedThreadStartRecord
return err("start record missing parameters.options.maxRounds");
}
const depthRaw = optRec.depth;
const depth =
typeof depthRaw === "number" && Number.isFinite(depthRaw) ? Math.trunc(depthRaw) : 0;
return ok({
workflowName: name,
hash,
threadId,
prompt,
maxRounds,
depth,
});
}
@@ -194,7 +202,7 @@ export type ForkPlan = {
hash: string;
sourceThreadId: string;
prompt: string;
runOptions: { maxRounds: number };
runOptions: { maxRounds: number; depth: number };
historicalSteps: ForkHistoricalStep[];
};
@@ -219,7 +227,7 @@ export function buildForkPlan(
hash: start.hash,
sourceThreadId: start.threadId,
prompt: start.prompt,
runOptions: { maxRounds: start.maxRounds },
runOptions: { maxRounds: start.maxRounds, depth: start.depth },
historicalSteps: selected.value,
});
}
+131
View File
@@ -0,0 +1,131 @@
import { readdir, readFile } from "node:fs/promises";
import { join } from "node:path";
import { type CasStore, createCasStore } from "./cas.js";
import { parseThreadDataJsonl } from "./fork-thread.js";
import { err, ok, type Result } from "./result.js";
import { getGlobalCasDir } from "./storage-root.js";
export type GcResult = {
scannedThreads: number;
activeRefs: number;
deletedEntries: number;
deletedHashes: string[];
};
async function listThreadDataJsonlPaths(storageRoot: string): Promise<Result<string[], string>> {
const logsRoot = join(storageRoot, "logs");
const paths: string[] = [];
let hashes: string[];
try {
hashes = await readdir(logsRoot);
} catch (e) {
const errObj = e as NodeJS.ErrnoException;
if (errObj.code === "ENOENT") {
return ok([]);
}
return err(`failed to read logs directory: ${String(e)}`);
}
for (const hash of hashes) {
const dir = join(logsRoot, hash);
let entries: string[];
try {
entries = await readdir(dir);
} catch {
continue;
}
for (const fileName of entries) {
if (fileName.endsWith(".data.jsonl")) {
paths.push(join(dir, fileName));
}
}
}
paths.sort();
return ok(paths);
}
async function collectActiveRefsFromDataPaths(
dataPaths: string[],
): Promise<Result<Set<string>, string>> {
const activeRefs = new Set<string>();
for (const dataPath of dataPaths) {
let text: string;
try {
text = await readFile(dataPath, "utf8");
} catch (e) {
return err(`failed to read ${dataPath}: ${String(e)}`);
}
const parsed = parseThreadDataJsonl(text);
if (!parsed.ok) {
return err(`${dataPath}: ${parsed.error}`);
}
for (const step of parsed.value.roleSteps) {
for (const ref of step.refs) {
activeRefs.add(ref);
}
}
}
return ok(activeRefs);
}
async function deleteCasNotInSet(
cas: CasStore,
activeRefs: Set<string>,
): Promise<Result<string[], string>> {
let listed: string[];
try {
listed = await cas.list();
} catch (e) {
return err(`failed to list cas entries: ${String(e)}`);
}
const deletedHashes: string[] = [];
for (const hash of listed) {
if (activeRefs.has(hash)) {
continue;
}
try {
await cas.delete(hash);
} catch (e) {
return err(`failed to delete cas ${hash}: ${String(e)}`);
}
deletedHashes.push(hash);
}
deletedHashes.sort();
return ok(deletedHashes);
}
/**
* Mark-and-sweep CAS GC: collect `refs` from all thread `.data.jsonl` files under `storageRoot`,
* then delete CAS blobs not referenced by any surviving thread data.
*/
export async function garbageCollectCas(storageRoot: string): Promise<Result<GcResult, string>> {
const pathsResult = await listThreadDataJsonlPaths(storageRoot);
if (!pathsResult.ok) {
return pathsResult;
}
const paths = pathsResult.value;
const refsResult = await collectActiveRefsFromDataPaths(paths);
if (!refsResult.ok) {
return refsResult;
}
const activeRefs = refsResult.value;
const cas = createCasStore(getGlobalCasDir(storageRoot));
const deletedResult = await deleteCasNotInSet(cas, activeRefs);
if (!deletedResult.ok) {
return deletedResult;
}
const deletedHashes = deletedResult.value;
return ok({
scannedThreads: paths.length,
activeRefs: activeRefs.size,
deletedEntries: deletedHashes.length,
deletedHashes,
});
}
+4 -2
View File
@@ -7,7 +7,7 @@ export {
} from "./base32.js";
export { buildDescriptor } from "./build-descriptor.js";
export { validateWorkflowBundle, type WorkflowBundleValidationInput } from "./bundle-validator.js";
export { type CasStore, createThreadCas } from "./cas.js";
export { type CasStore, createCasStore, createThreadCas } from "./cas.js";
export { createWorkflow } from "./create-workflow.js";
export {
type ExecuteThreadIo,
@@ -25,6 +25,7 @@ export {
parseThreadDataJsonl,
selectForkHistoricalSteps,
} from "./fork-thread.js";
export { type GcResult, garbageCollectCas } from "./gc.js";
export { stringifyWorkflowDescriptor } from "./generate-descriptor.js";
export { hashString, hashWorkflowBundleBytes } from "./hash.js";
export {
@@ -55,7 +56,7 @@ export {
writeWorkflowRegistry,
} from "./registry.js";
export { err, ok, type Result } from "./result.js";
export { getDefaultWorkflowStorageRoot } from "./storage-root.js";
export { getDefaultWorkflowStorageRoot, getGlobalCasDir } from "./storage-root.js";
export { createThreadPauseGate, type ThreadPauseGate } from "./thread-pause-gate.js";
export {
type AgentBinding,
@@ -81,6 +82,7 @@ export {
} from "./types.js";
export { generateUlid } from "./ulid.js";
export { getWorkerHostScriptPath } from "./worker-entry-path.js";
export { type WorkflowAsAgentOptions, workflowAsAgent } from "./workflow-as-agent.js";
export {
validateWorkflowDescriptor,
type WorkflowDescriptor,
+13
View File
@@ -0,0 +1,13 @@
/** Normalize `refs` from persisted JSONL or IPC payloads (missing or invalid → []). */
export function normalizeRefsField(value: unknown): string[] {
if (!Array.isArray(value)) {
return [];
}
const out: string[] = [];
for (const x of value) {
if (typeof x === "string") {
out.push(x);
}
}
return out;
}
+6
View File
@@ -5,3 +5,9 @@ import { join } from "node:path";
export function getDefaultWorkflowStorageRoot(): string {
return join(homedir(), ".uncaged", "workflow");
}
/** Global content-addressed store directory under the workflow storage root (`<root>/cas`). */
export function getGlobalCasDir(storageRoot: string | undefined): string {
const root = storageRoot ?? getDefaultWorkflowStorageRoot();
return join(root, "cas");
}
+15 -1
View File
@@ -19,6 +19,8 @@ export type RoleOutput = {
role: string;
content: string;
meta: Record<string, unknown>;
/** CAS hashes produced or consumed by this step (for GC traceability). */
refs: string[];
};
/** What the workflow AsyncGenerator returns when done. */
@@ -37,6 +39,8 @@ export type ThreadInput = {
export type WorkflowFnOptions = {
threadId: string;
maxRounds: number;
/** Nesting depth for workflow-as-agent chains; root threads use `0`. */
depth: number;
};
/** Bundle contract — named export `run` is a function returning an AsyncGenerator. */
@@ -55,12 +59,20 @@ export type StartStep = {
/** A completed role step in the thread. */
export type RoleStep<M extends RoleMeta> = {
[K in keyof M & string]: { role: K; meta: M[K]; content: string; timestamp: number };
[K in keyof M & string]: {
role: K;
meta: M[K];
content: string;
refs: string[];
timestamp: number;
};
}[keyof M & string];
/** Phase 1: Moderator decides next role. */
export type ModeratorContext<M extends RoleMeta = RoleMeta> = {
threadId: string;
/** Same as `WorkflowFnOptions.depth` for the active thread. */
depth: number;
start: StartStep;
steps: RoleStep<M>[];
};
@@ -96,6 +108,8 @@ export type RoleDefinition<Meta extends Record<string, unknown>> = {
systemPrompt: string;
extractPrompt: string;
schema: z.ZodType<Meta>;
/** When non-null, produces CAS hashes to persist on this role's steps (see `RoleOutput.refs`). */
extractRefs: ((meta: Meta) => string[]) | null;
};
/**
+13 -3
View File
@@ -5,6 +5,7 @@ import { pathToFileURL } from "node:url";
import type { PrefilledDiskStep } from "./engine.js";
import { type ExecuteThreadIo, executeThread } from "./engine.js";
import { createLogger } from "./logger.js";
import { normalizeRefsField } from "./refs-field.js";
import { err, ok, type Result } from "./result.js";
import { createThreadPauseGate, type ThreadPauseGate } from "./thread-pause-gate.js";
import type { RoleOutput, WorkflowFn } from "./types.js";
@@ -16,7 +17,7 @@ type RunCommand = {
threadId: string;
workflowName: string;
prompt: string;
options: { maxRounds: number };
options: { maxRounds: number; depth: number };
steps: RoleOutput[];
/** Timestamps aligned with `steps` for `.data.jsonl` replay; length must match `steps` when non-null. */
stepTimestamps: number[] | null;
@@ -55,7 +56,12 @@ function parseRoleOutputRecord(obj: Record<string, unknown>): RoleOutput | null
if (meta === null || typeof meta !== "object") {
return null;
}
return { role, content, meta: meta as Record<string, unknown> };
return {
role,
content,
meta: meta as Record<string, unknown>,
refs: normalizeRefsField(obj.refs),
};
}
function parseRunStepsPayload(rec: Record<string, unknown>): {
@@ -118,6 +124,9 @@ function parseRunControlPayload(rec: Record<string, unknown>): RunCommand | null
if (typeof maxRounds !== "number") {
return null;
}
const depthRaw = optRec.depth;
const depth =
typeof depthRaw === "number" && Number.isFinite(depthRaw) ? Math.trunc(depthRaw) : 0;
const parsedSteps = parseRunStepsPayload(rec);
if (parsedSteps === null) {
return null;
@@ -135,7 +144,7 @@ function parseRunControlPayload(rec: Record<string, unknown>): RunCommand | null
threadId,
workflowName,
prompt,
options: { maxRounds },
options: { maxRounds, depth },
steps: parsedSteps.steps,
stepTimestamps: parsedSteps.stepTimestamps,
forkSourceThreadId,
@@ -382,6 +391,7 @@ async function main(): Promise<void> {
role: step.role,
content: step.content,
meta: step.meta,
refs: normalizeRefsField(step.refs),
timestamp: typeof ts === "number" && ts > 0 ? ts : baseTs + i,
};
});
@@ -0,0 +1,99 @@
import { join } from "node:path";
import { type ExecuteThreadIo, executeThread } from "./engine.js";
import { extractBundleExports } from "./extract-bundle-exports.js";
import { createLogger } from "./logger.js";
import { getRegisteredWorkflow, readWorkflowRegistry } from "./registry.js";
import { getDefaultWorkflowStorageRoot } from "./storage-root.js";
import type { AgentContext, AgentFn, ThreadInput } from "./types.js";
import { generateUlid } from "./ulid.js";
/** Maximum `WorkflowFnOptions.depth` allowed for a child spawned via `workflowAsAgent`. */
const WORKFLOW_AS_AGENT_MAX_DEPTH = 3;
export type WorkflowAsAgentOptions = {
/** When `null`, uses `getDefaultWorkflowStorageRoot()`. */
storageRoot: string | null;
};
function resolveWorkflowAsAgentStorageRoot(options: WorkflowAsAgentOptions | null): string {
if (options !== null && options.storageRoot !== null) {
return options.storageRoot;
}
return getDefaultWorkflowStorageRoot();
}
/**
* Returns an {@link AgentFn} that runs another registered workflow in a new thread,
* using the parent thread's initial prompt (`ctx.start.content`) as the child {@link ThreadInput.prompt}.
*/
export function workflowAsAgent(
workflowName: string,
options: WorkflowAsAgentOptions | null = null,
): AgentFn {
return async (ctx: AgentContext): Promise<string> => {
const nextDepth = ctx.depth + 1;
if (nextDepth > WORKFLOW_AS_AGENT_MAX_DEPTH) {
return `ERROR: workflow-as-agent depth limit exceeded (max ${WORKFLOW_AS_AGENT_MAX_DEPTH})`;
}
const storageRoot = resolveWorkflowAsAgentStorageRoot(options);
const registryResult = await readWorkflowRegistry(storageRoot);
if (!registryResult.ok) {
return `ERROR: failed to read workflow registry: ${registryResult.error.message}`;
}
const entry = getRegisteredWorkflow(registryResult.value, workflowName);
if (entry === null) {
return `ERROR: workflow "${workflowName}" not found in registry`;
}
const bundlePath = join(storageRoot, "bundles", `${entry.hash}.esm.js`);
const bundleExportsResult = await extractBundleExports(bundlePath);
if (!bundleExportsResult.ok) {
return `ERROR: ${bundleExportsResult.error}`;
}
const input: ThreadInput = {
prompt: ctx.start.content,
steps: [],
};
const childThreadId = generateUlid(Date.now());
const dataJsonlPath = join(storageRoot, "logs", entry.hash, `${childThreadId}.data.jsonl`);
const infoJsonlPath = join(storageRoot, "logs", entry.hash, `${childThreadId}.info.jsonl`);
const io: ExecuteThreadIo = {
threadId: childThreadId,
hash: entry.hash,
dataJsonlPath,
infoJsonlPath,
};
const logger = createLogger({ sink: { kind: "file", path: infoJsonlPath } });
const signalNever = new AbortController();
try {
const result = await executeThread(
bundleExportsResult.value.run,
workflowName,
input,
{
maxRounds: ctx.start.meta.maxRounds,
depth: nextDepth,
signal: signalNever.signal,
awaitAfterEachYield: async () => {},
forkSourceThreadId: ctx.threadId,
prefilledDiskSteps: null,
},
io,
logger,
);
return result.summary;
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
return `ERROR: ${message}`;
}
};
}