Compare commits
29 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 92d1e95de2 | |||
| c1597d6efa | |||
| 9066322f19 | |||
| 7f64541c5b | |||
| 43a6600378 | |||
| 220c9c5224 | |||
| cae59b589e | |||
| b5cc0db17e | |||
| 6196e0974a | |||
| 410e9e6d9b | |||
| 84de74721d | |||
| 4403532f35 | |||
| e95e76c145 | |||
| af69e773a0 | |||
| 6488b7bbb4 | |||
| 15d39c96a7 | |||
| 30e4e99908 | |||
| a3c70a5041 | |||
| 12d58a8206 | |||
| c096f4d94e | |||
| 500401d93c | |||
| 43f466eb67 | |||
| fe829d9ae6 | |||
| f80535d742 | |||
| 0eab3b7001 | |||
| 37c5b89c98 | |||
| 0fdf19879a | |||
| f73bf1e313 | |||
| 8c4441bf6b |
@@ -0,0 +1,315 @@
|
||||
# Workflow-as-Agent Implementation Plan
|
||||
|
||||
> **For Hermes:** Use subagent-driven-development skill to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Enable workflows to invoke other workflows as agents, backed by global CAS and refs tracking.
|
||||
|
||||
**Architecture:** Migrate CAS from thread-local to global (`~/.uncaged/workflow/cas/`), add `refs` to RoleStep for GC traceability, then build `workflowAsAgent(name)` factory that resolves workflow name → bundle via registry and spawns a child thread.
|
||||
|
||||
**Tech Stack:** TypeScript, Bun, Zod v4, monorepo with `packages/`
|
||||
|
||||
**Issue:** https://git.shazhou.work/uncaged/workflow/issues/25
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Global CAS Migration
|
||||
|
||||
Move CAS storage from `<threadDir>/<threadId>.cas/` to `~/.uncaged/workflow/cas/` (global, content-addressed, immutable). This is a **breaking change** — thread-local `.cas/` directories are abandoned.
|
||||
|
||||
### Task 1.1: Add `globalCasDir` helper to `storage-root.ts`
|
||||
|
||||
**Objective:** Provide a single function that returns the global CAS directory path.
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/workflow/src/storage-root.ts`
|
||||
- Test: `packages/workflow/__tests__/storage-root.test.ts`
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```typescript
|
||||
// storage-root.ts — add export
|
||||
export function getGlobalCasDir(storageRoot?: string): string {
|
||||
const root = storageRoot ?? getDefaultWorkflowStorageRoot();
|
||||
return join(root, "cas");
|
||||
}
|
||||
```
|
||||
|
||||
Export from `packages/workflow/src/index.ts`.
|
||||
|
||||
### Task 1.2: Update `cmd-cas.ts` to use global CAS
|
||||
|
||||
**Objective:** CLI `cas get/put/list/rm` no longer needs threadId for storage location — CAS is global. But keep threadId in CLI for backward compat of planner/coder prompts (they pass threadId).
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/cli-workflow/src/cmd-cas.ts`
|
||||
|
||||
**Changes:**
|
||||
- `resolveCasDir` → use `getGlobalCasDir(storageRoot)` instead of deriving from thread data path
|
||||
- `cmdCasPut` / `cmdCasGet` / `cmdCasList` / `cmdCasRm`: threadId is still accepted (prompts pass it) but storage goes to global dir
|
||||
- Remove the `resolveThreadDataPath` dependency for CAS operations — thread doesn't need to exist to read CAS
|
||||
|
||||
```typescript
|
||||
import { createThreadCas, getGlobalCasDir } from "@uncaged/workflow";
|
||||
|
||||
export async function cmdCasGet(
|
||||
storageRoot: string,
|
||||
_threadId: string, // kept for CLI compat, not used for path
|
||||
hash: string,
|
||||
): Promise<Result<string, string>> {
|
||||
const cas = createThreadCas(getGlobalCasDir(storageRoot));
|
||||
const content = await cas.get(hash);
|
||||
if (content === null) {
|
||||
return err(`cas entry not found: ${hash}`);
|
||||
}
|
||||
return ok(content);
|
||||
}
|
||||
// ... same pattern for put/list/rm
|
||||
```
|
||||
|
||||
### Task 1.3: Update `cmd-thread.ts` — thread rm no longer deletes `.cas/`
|
||||
|
||||
**Objective:** Since CAS is global, `thread rm` should NOT delete CAS entries. CAS cleanup is GC's job.
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/cli-workflow/src/cmd-thread.ts`
|
||||
- Check: remove any `rmdir` / `unlink` of `<threadId>.cas/` directory
|
||||
|
||||
### Task 1.4: Rename `createThreadCas` → `createCasStore`
|
||||
|
||||
**Objective:** The name `createThreadCas` is misleading now. Rename to `createCasStore`.
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/workflow/src/cas.ts` — rename function
|
||||
- Modify: `packages/workflow/src/index.ts` — update export (keep `createThreadCas` as deprecated alias for one release)
|
||||
- Modify: all consumers (`cmd-cas.ts`)
|
||||
|
||||
### Task 1.5: Update tests
|
||||
|
||||
**Objective:** All CAS-related tests use global dir instead of thread-local.
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/cli-workflow/__tests__/commands.test.ts`
|
||||
- Verify: `bun test` passes
|
||||
|
||||
### Task 1.6: Clean up old thread-local `.cas/` references
|
||||
|
||||
**Objective:** Remove dead code that creates/reads thread-local `.cas/` directories.
|
||||
|
||||
**Files:**
|
||||
- Search all `*.ts` for `.cas` path construction patterns
|
||||
- Remove orphaned helpers
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: RoleStep `refs` Tracking
|
||||
|
||||
Add `refs: string[]` to persisted role steps so GC can trace which CAS entries are alive.
|
||||
|
||||
### Task 2.1: Add `refs` to `RoleOutput` and engine persistence
|
||||
|
||||
**Objective:** Every role step can declare which CAS hashes it produced or consumed.
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/workflow/src/types.ts`
|
||||
- Modify: `packages/workflow/src/engine.ts`
|
||||
|
||||
**Changes to `types.ts`:**
|
||||
|
||||
```typescript
|
||||
export type RoleOutput = {
|
||||
role: string;
|
||||
content: string;
|
||||
meta: Record<string, unknown>;
|
||||
refs: string[]; // CAS hashes produced/consumed by this step
|
||||
};
|
||||
```
|
||||
|
||||
**Changes to `engine.ts`:**
|
||||
- `appendDataLine` for role steps: include `refs` field (default `[]` if not provided)
|
||||
|
||||
### Task 2.2: Auto-populate refs from meta hashes
|
||||
|
||||
**Objective:** The engine should automatically extract CAS hashes from `meta` to populate `refs`, so roles don't need to manually track them.
|
||||
|
||||
**Strategy:** After meta extraction, walk the meta object and collect any string that looks like a CAS hash (Crockford Base32, 13 chars). This is a heuristic but works because CAS hashes are distinctive.
|
||||
|
||||
Alternative (simpler): Let each `RoleDefinition` optionally declare a `extractRefs(meta: M) => string[]` function. For planner, this returns `meta.phases.map(p => p.hash)`. For coder, `[meta.completedPhase]`.
|
||||
|
||||
**Recommended:** The explicit `extractRefs` approach — no magic, no false positives.
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/workflow/src/types.ts` — add optional `extractRefs` to `RoleDefinition`
|
||||
- Modify: `packages/workflow/src/create-workflow.ts` — call `extractRefs` after meta extraction, set on `RoleOutput.refs`
|
||||
- Modify: `packages/workflow-role-planner/src/planner.ts` — implement `extractRefs`
|
||||
- Modify: `packages/workflow-role-coder/src/coder.ts` — implement `extractRefs`
|
||||
|
||||
```typescript
|
||||
// types.ts — RoleDefinition addition
|
||||
export type RoleDefinition<Meta extends Record<string, unknown>> = {
|
||||
description: string;
|
||||
systemPrompt: string;
|
||||
extractPrompt: string;
|
||||
schema: z.ZodType<Meta>;
|
||||
extractRefs?: (meta: Meta) => string[]; // CAS hashes to track
|
||||
};
|
||||
|
||||
// planner.ts
|
||||
extractRefs: (meta) => meta.phases.map(p => p.hash),
|
||||
|
||||
// coder.ts
|
||||
extractRefs: (meta) => [meta.completedPhase],
|
||||
```
|
||||
|
||||
### Task 2.3: Update fork logic to preserve refs
|
||||
|
||||
**Objective:** When forking a thread, `refs` from historical steps must be carried over.
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/workflow/src/fork-thread.ts`
|
||||
- Verify: `ForkHistoricalStep` / `PrefilledDiskStep` include `refs`
|
||||
|
||||
### Task 2.4: Tests for refs tracking
|
||||
|
||||
**Files:**
|
||||
- Add: `packages/workflow/__tests__/refs-tracking.test.ts`
|
||||
- Verify: refs appear in `.data.jsonl` output
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: CAS Garbage Collection
|
||||
|
||||
### Task 3.1: Implement `gc.ts` in `@uncaged/workflow`
|
||||
|
||||
**Objective:** Mark-and-sweep GC — scan all thread `.data.jsonl` files, collect `refs`, delete orphaned CAS entries.
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/workflow/src/gc.ts`
|
||||
- Export from: `packages/workflow/src/index.ts`
|
||||
|
||||
```typescript
|
||||
export type GcResult = {
|
||||
scannedThreads: number;
|
||||
activeRefs: number;
|
||||
deletedEntries: number;
|
||||
deletedHashes: string[];
|
||||
};
|
||||
|
||||
export async function garbageCollectCas(storageRoot: string): Promise<GcResult> {
|
||||
// 1. Find all .data.jsonl files under storageRoot
|
||||
// 2. Parse each, flatMap step.refs → Set<string>
|
||||
// 3. List all CAS entries via createCasStore(globalCasDir).list()
|
||||
// 4. Delete entries not in active set
|
||||
// 5. Return stats
|
||||
}
|
||||
```
|
||||
|
||||
### Task 3.2: Add `uncaged-workflow gc` CLI command
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/cli-workflow/src/cmd-gc.ts`
|
||||
- Modify: `packages/cli-workflow/src/cli-dispatch.ts` — add `gc` subcommand
|
||||
|
||||
### Task 3.3: Run GC on `thread rm`
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/cli-workflow/src/cmd-thread.ts` — after deleting thread data, optionally run GC
|
||||
|
||||
### Task 3.4: Tests for GC
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/cli-workflow/__tests__/gc-cli.test.ts`
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: `workflowAsAgent` Factory
|
||||
|
||||
### Task 4.1: Create `workflowAsAgent` in `@uncaged/workflow`
|
||||
|
||||
**Objective:** Factory function that takes a workflow name, resolves to bundle, returns an `AgentFn`.
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/workflow/src/workflow-as-agent.ts`
|
||||
- Export from: `packages/workflow/src/index.ts`
|
||||
|
||||
```typescript
|
||||
import type { AgentFn } from "./types.js";
|
||||
|
||||
export type WorkflowAsAgentOptions = {
|
||||
storageRoot?: string;
|
||||
};
|
||||
|
||||
export function workflowAsAgent(
|
||||
workflowName: string,
|
||||
options?: WorkflowAsAgentOptions,
|
||||
): AgentFn {
|
||||
return async (ctx) => {
|
||||
const storageRoot = options?.storageRoot ?? getDefaultWorkflowStorageRoot();
|
||||
|
||||
// 1. Read registry → resolve name to bundle hash + path
|
||||
const registry = await readWorkflowRegistry(storageRoot);
|
||||
const entry = getRegisteredWorkflow(registry, workflowName);
|
||||
if (entry === null) {
|
||||
return `ERROR: workflow "${workflowName}" not found in registry`;
|
||||
}
|
||||
|
||||
// 2. Load bundle
|
||||
const bundlePath = join(storageRoot, "bundles", `${entry.hash}.esm.js`);
|
||||
const bundleExports = await extractBundleExports(bundlePath);
|
||||
|
||||
// 3. Create child thread input from ctx.start.content (parent prompt)
|
||||
const input: ThreadInput = {
|
||||
prompt: ctx.start.content,
|
||||
steps: [],
|
||||
};
|
||||
|
||||
// 4. Generate child threadId
|
||||
const childThreadId = generateUlid();
|
||||
|
||||
// 5. Execute — collect all yields, return final content
|
||||
const io: ExecuteThreadIo = { ... };
|
||||
const result = await executeThread(bundleExports.run, workflowName, input, ...);
|
||||
|
||||
// 6. Return summary as agent content
|
||||
return result.summary;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Task 4.2: System-level depth limit
|
||||
|
||||
**Objective:** Prevent infinite recursion. Track depth via thread metadata, enforce a global max (default 3, configurable in `workflow.yaml`).
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/workflow/src/types.ts` — add `depth` to `WorkflowFnOptions`
|
||||
- Modify: `packages/workflow/src/workflow-as-agent.ts` — increment depth, check limit
|
||||
- Modify: registry or config types for `maxDepth` setting
|
||||
|
||||
### Task 4.3: Tests for workflowAsAgent
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/workflow/__tests__/workflow-as-agent.test.ts`
|
||||
- Test: name resolution, depth limit, child thread execution
|
||||
|
||||
### Task 4.4: Integration test — nested workflow
|
||||
|
||||
**Objective:** Create a minimal test workflow that calls another workflow via `workflowAsAgent`.
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/workflow/__tests__/workflow-as-agent-integration.test.ts`
|
||||
|
||||
---
|
||||
|
||||
## Execution Order
|
||||
|
||||
```
|
||||
Phase 1 (Global CAS) → Phase 2 (refs) → Phase 3 (GC) → Phase 4 (workflowAsAgent)
|
||||
```
|
||||
|
||||
Each phase is independently mergeable. Phase 3 depends on Phase 2 (needs refs to know what's alive). Phase 4 depends on Phase 1 (global CAS for cross-thread sharing).
|
||||
|
||||
## Breaking Changes
|
||||
|
||||
- CAS storage location moves from `<thread>.cas/` to `~/.uncaged/workflow/cas/`
|
||||
- `RoleOutput` gains required `refs: string[]` field
|
||||
- Existing threads with thread-local CAS will lose access to old CAS data (acceptable — those are short-lived workflow artifacts)
|
||||
- `createThreadCas` renamed to `createCasStore` (alias kept temporarily)
|
||||
@@ -28,6 +28,8 @@ const greeter: RoleDefinition<Roles["greeter"]> = {
|
||||
systemPrompt: "You greet the user briefly.",
|
||||
extractPrompt: "Extract the greeting string produced for the user.",
|
||||
schema: greeterMetaSchema,
|
||||
extractRefs: null,
|
||||
extractMode: "single",
|
||||
};
|
||||
|
||||
const extract = createExtract({
|
||||
@@ -47,4 +49,5 @@ export const run = createWorkflow<Roles>(
|
||||
agent: async (ctx) => `Hello, ${ctx.start.content}`,
|
||||
},
|
||||
extract,
|
||||
null,
|
||||
);
|
||||
|
||||
@@ -3,8 +3,9 @@ import { mkdir, mkdtemp, readFile, rm, unlink, writeFile } from "node:fs/promise
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { getRegisteredWorkflow, readWorkflowRegistry } from "@uncaged/workflow";
|
||||
import { getGlobalCasDir, getRegisteredWorkflow, readWorkflowRegistry } from "@uncaged/workflow";
|
||||
import { cmdAdd } from "../src/cmd-add.js";
|
||||
import { cmdCasGet, cmdCasList, cmdCasPut, cmdCasRm } from "../src/cmd-cas.js";
|
||||
import { cmdHistory } from "../src/cmd-history.js";
|
||||
import { cmdList, formatListLines } from "../src/cmd-list.js";
|
||||
import { cmdRemove } from "../src/cmd-remove.js";
|
||||
@@ -15,6 +16,9 @@ import { addCliArgs } from "./bundle-fixture.js";
|
||||
const fixtureDescriptor = `export const descriptor = { description: "fixture", roles: {} };
|
||||
`;
|
||||
|
||||
const wfPutImport = `import { putContentMerkleNode } from "@uncaged/workflow";
|
||||
`;
|
||||
|
||||
describe("cli workflow commands", () => {
|
||||
let prevEnv: string | undefined;
|
||||
let storageRoot: string;
|
||||
@@ -40,11 +44,13 @@ describe("cli workflow commands", () => {
|
||||
const bundlePath = join(bundleDir, "demo.esm.js");
|
||||
await writeFile(
|
||||
bundlePath,
|
||||
`${fixtureDescriptor}import fs from "node:fs";
|
||||
`${fixtureDescriptor}${wfPutImport}import fs from "node:fs";
|
||||
|
||||
export const run = async function* (input) {
|
||||
export const run = async function* (input, options) {
|
||||
fs.existsSync(".");
|
||||
yield { role: "noop", content: input.prompt, meta: { done: true } };
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, input.prompt);
|
||||
yield { role: "noop", contentHash: h, meta: { done: true }, refs: [h] };
|
||||
return { returnCode: 0, summary: "done" };
|
||||
}
|
||||
`,
|
||||
@@ -111,8 +117,8 @@ export const run = async function* (input) { return { returnCode: 0, summary: in
|
||||
const bundlePath = join(storageRoot, "solo.esm.js");
|
||||
await writeFile(
|
||||
bundlePath,
|
||||
`export const run = async function* (input) {
|
||||
yield { role: "x", content: input.prompt, meta: {} };
|
||||
`export const run = async function* () {
|
||||
yield { role: "x", contentHash: "STUBHASH00000000000000001", meta: {}, refs: [] };
|
||||
return { returnCode: 0, summary: "ok" };
|
||||
}
|
||||
`,
|
||||
@@ -140,8 +146,11 @@ export const run = async function* (input) { return { returnCode: 0, summary: in
|
||||
},
|
||||
},
|
||||
};
|
||||
export const run = async function* (input) {
|
||||
yield { role: "greeter", content: input.prompt, meta: { greeting: "hi" } };
|
||||
${wfPutImport}
|
||||
export const run = async function* (input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, input.prompt);
|
||||
yield { role: "greeter", contentHash: h, meta: { greeting: "hi" }, refs: [h] };
|
||||
return { returnCode: 0, summary: "ok" };
|
||||
};
|
||||
`,
|
||||
@@ -179,8 +188,10 @@ export const run = async function* (input) {
|
||||
const bundlePath = join(bundleDir, "demo.esm.js");
|
||||
await writeFile(
|
||||
bundlePath,
|
||||
`${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "x", meta: {} };
|
||||
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "x");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "x" };
|
||||
}
|
||||
`,
|
||||
@@ -208,8 +219,10 @@ export const run = async function* (input) {
|
||||
const dtsPath = join(bundleDir, "types.d.ts");
|
||||
await writeFile(
|
||||
bundlePath,
|
||||
`${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "x", meta: {} };
|
||||
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "x");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "x" };
|
||||
}
|
||||
`,
|
||||
@@ -239,8 +252,10 @@ export const run = async function* (input) {
|
||||
const bundlePath = join(bundleDir, "demo.esm.js");
|
||||
await writeFile(
|
||||
bundlePath,
|
||||
`${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "x", meta: {} };
|
||||
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "x");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "x" };
|
||||
}
|
||||
`,
|
||||
@@ -260,13 +275,17 @@ export const run = async function* (input) {
|
||||
const bundleDir = join(storageRoot, "src");
|
||||
await mkdir(bundleDir, { recursive: true });
|
||||
const bundlePath = join(bundleDir, "demo.esm.js");
|
||||
const v1 = `${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "v1", meta: {} };
|
||||
const v1 = `${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "v1");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "v1" };
|
||||
}
|
||||
`;
|
||||
const v2 = `${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "v2", meta: {} };
|
||||
const v2 = `${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "v2");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "v2" };
|
||||
}
|
||||
`;
|
||||
@@ -298,13 +317,17 @@ export const run = async function* (input) {
|
||||
const bundleDir = join(storageRoot, "src");
|
||||
await mkdir(bundleDir, { recursive: true });
|
||||
const bundlePath = join(bundleDir, "demo.esm.js");
|
||||
const v1 = `${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "v1", meta: {} };
|
||||
const v1 = `${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "v1");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "v1" };
|
||||
}
|
||||
`;
|
||||
const v2 = `${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "v2", meta: {} };
|
||||
const v2 = `${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "v2");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "v2" };
|
||||
}
|
||||
`;
|
||||
@@ -346,8 +369,10 @@ export const run = async function* (input) {
|
||||
const bundlePath = join(bundleDir, "demo.esm.js");
|
||||
await writeFile(
|
||||
bundlePath,
|
||||
`${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "x", meta: {} };
|
||||
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "x");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "x" };
|
||||
}
|
||||
`,
|
||||
@@ -357,8 +382,10 @@ export const run = async function* (input) {
|
||||
expect(add1.ok).toBe(true);
|
||||
await writeFile(
|
||||
bundlePath,
|
||||
`${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "y", meta: {} };
|
||||
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "y");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "y" };
|
||||
}
|
||||
`,
|
||||
@@ -371,14 +398,47 @@ export const run = async function* (input) {
|
||||
expect(bad.ok).toBe(false);
|
||||
});
|
||||
|
||||
test("cas put/get/list/rm use global cas dir (thread id not required for storage)", async () => {
|
||||
const put = await cmdCasPut(storageRoot, "nonexistent-thread-id", "phase doc");
|
||||
expect(put.ok).toBe(true);
|
||||
if (!put.ok) {
|
||||
return;
|
||||
}
|
||||
const hash = put.value;
|
||||
const blobPath = join(getGlobalCasDir(storageRoot), `${hash}.txt`);
|
||||
expect(await readFile(blobPath, "utf8")).toBe("phase doc");
|
||||
|
||||
const got = await cmdCasGet(storageRoot, "other-thread", hash);
|
||||
expect(got.ok).toBe(true);
|
||||
if (!got.ok) {
|
||||
return;
|
||||
}
|
||||
expect(got.value).toBe("phase doc");
|
||||
|
||||
const listed = await cmdCasList(storageRoot, "another-thread");
|
||||
expect(listed.ok).toBe(true);
|
||||
if (!listed.ok) {
|
||||
return;
|
||||
}
|
||||
expect(listed.value).toContain(hash);
|
||||
|
||||
const removed = await cmdCasRm(storageRoot, "rm-thread", hash);
|
||||
expect(removed.ok).toBe(true);
|
||||
|
||||
const missing = await cmdCasGet(storageRoot, "after-rm", hash);
|
||||
expect(missing.ok).toBe(false);
|
||||
});
|
||||
|
||||
test("rollback rejects missing bundle file for target hash", async () => {
|
||||
const bundleDir = join(storageRoot, "src");
|
||||
await mkdir(bundleDir, { recursive: true });
|
||||
const bundlePath = join(bundleDir, "demo.esm.js");
|
||||
await writeFile(
|
||||
bundlePath,
|
||||
`${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "x", meta: {} };
|
||||
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "x");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "x" };
|
||||
}
|
||||
`,
|
||||
@@ -392,8 +452,10 @@ export const run = async function* (input) {
|
||||
const hash1 = add1.value.hash;
|
||||
await writeFile(
|
||||
bundlePath,
|
||||
`${fixtureDescriptor}export const run = async function* (input) {
|
||||
yield { role: "a", content: "y", meta: {} };
|
||||
`${fixtureDescriptor}${wfPutImport}export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "y");
|
||||
yield { role: "a", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "y" };
|
||||
}
|
||||
`,
|
||||
|
||||
@@ -2,6 +2,7 @@ import { afterEach, beforeEach, describe, expect, test } from "bun:test";
|
||||
import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { createCasStore, getContentMerklePayload, getGlobalCasDir } from "@uncaged/workflow";
|
||||
import { cmdAdd } from "../src/cmd-add.js";
|
||||
import { cmdFork } from "../src/cmd-fork.js";
|
||||
import { cmdRun } from "../src/cmd-run.js";
|
||||
@@ -9,7 +10,9 @@ import { pathExists } from "../src/fs-utils.js";
|
||||
import { addCliArgs } from "./bundle-fixture.js";
|
||||
|
||||
/** Three-role workflow that respects `input.steps` for fork/resume. */
|
||||
const threeRoleBundleSource = `export const descriptor = {
|
||||
const threeRoleBundleSource = `import { putContentMerkleNode } from "@uncaged/workflow";
|
||||
|
||||
export const descriptor = {
|
||||
description: "fork-cli",
|
||||
roles: {
|
||||
planner: { description: "planner", schema: {} },
|
||||
@@ -17,20 +20,21 @@ const threeRoleBundleSource = `export const descriptor = {
|
||||
reviewer: { description: "reviewer", schema: {} },
|
||||
},
|
||||
};
|
||||
export const run = async function* (input) {
|
||||
export const run = async function* (input, options) {
|
||||
const cas = options.cas;
|
||||
const has = (r) => input.steps.some((s) => s.role === r);
|
||||
if (!has("planner")) {
|
||||
yield { role: "planner", content: "p1", meta: { k: "planner" } };
|
||||
const h = await putContentMerkleNode(cas, "p1");
|
||||
yield { role: "planner", contentHash: h, meta: { k: "planner" }, refs: [h] };
|
||||
}
|
||||
if (!has("coder")) {
|
||||
yield { role: "coder", content: "c1", meta: { k: "coder" } };
|
||||
const h = await putContentMerkleNode(cas, "c1");
|
||||
yield { role: "coder", contentHash: h, meta: { k: "coder" }, refs: [h] };
|
||||
}
|
||||
if (!has("reviewer")) {
|
||||
yield {
|
||||
role: "reviewer",
|
||||
content: "rev-" + String(input.steps.length),
|
||||
meta: { k: "reviewer" },
|
||||
};
|
||||
const body = "rev-" + String(input.steps.length);
|
||||
const h = await putContentMerkleNode(cas, body);
|
||||
yield { role: "reviewer", contentHash: h, meta: { k: "reviewer" }, refs: [h] };
|
||||
}
|
||||
return { returnCode: 0, summary: "done" };
|
||||
};
|
||||
@@ -132,7 +136,8 @@ describe("cli fork", () => {
|
||||
|
||||
const last = JSON.parse(lines[lines.length - 1] ?? "{}") as Record<string, unknown>;
|
||||
expect(last.role).toBe("reviewer");
|
||||
expect(last.content).toBe("rev-1");
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
expect(await getContentMerklePayload(cas, String(last.contentHash))).toBe("rev-1");
|
||||
});
|
||||
|
||||
test("fork without --from-role retries last role", async () => {
|
||||
@@ -179,11 +184,12 @@ describe("cli fork", () => {
|
||||
|
||||
const replayCoder = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
|
||||
expect(replayCoder.role).toBe("coder");
|
||||
expect(replayCoder.content).toBe("c1");
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
expect(await getContentMerklePayload(cas, String(replayCoder.contentHash))).toBe("c1");
|
||||
|
||||
const last = JSON.parse(lines[lines.length - 1] ?? "{}") as Record<string, unknown>;
|
||||
expect(last.role).toBe("reviewer");
|
||||
expect(last.content).toBe("rev-2");
|
||||
expect(await getContentMerklePayload(cas, String(last.contentHash))).toBe("rev-2");
|
||||
});
|
||||
|
||||
test("fork rejects unknown role with available names", async () => {
|
||||
|
||||
@@ -0,0 +1,162 @@
|
||||
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
createCasStore,
|
||||
garbageCollectCas,
|
||||
getGlobalCasDir,
|
||||
putContentMerkleNode,
|
||||
} from "@uncaged/workflow";
|
||||
import { cmdThreadRemove } from "../src/cmd-thread.js";
|
||||
import { pathExists } from "../src/fs-utils.js";
|
||||
|
||||
const cliEntryPath = fileURLToPath(new URL("../src/cli.ts", import.meta.url));
|
||||
|
||||
async function writeDemoDataJsonl(params: {
|
||||
path: string;
|
||||
threadId: string;
|
||||
bundleHash: string;
|
||||
cas: ReturnType<typeof createCasStore>;
|
||||
activeHash: string;
|
||||
}): Promise<void> {
|
||||
const bodyHash = await putContentMerkleNode(params.cas, "p");
|
||||
const text = [
|
||||
JSON.stringify({
|
||||
name: "demo",
|
||||
hash: params.bundleHash,
|
||||
threadId: params.threadId,
|
||||
parameters: { prompt: "hi", options: { maxRounds: 5 } },
|
||||
timestamp: 100,
|
||||
}),
|
||||
JSON.stringify({
|
||||
role: "planner",
|
||||
contentHash: bodyHash,
|
||||
meta: {},
|
||||
refs: [params.activeHash, bodyHash],
|
||||
timestamp: 101,
|
||||
}),
|
||||
"",
|
||||
].join("\n");
|
||||
await writeFile(params.path, text, "utf8");
|
||||
}
|
||||
|
||||
describe("gc cli and garbageCollectCas", () => {
|
||||
let prevEnv: string | undefined;
|
||||
let storageRoot: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
prevEnv = process.env.UNCAGED_WORKFLOW_STORAGE_ROOT;
|
||||
storageRoot = await mkdtemp(join(tmpdir(), "uncaged-wf-gc-"));
|
||||
process.env.UNCAGED_WORKFLOW_STORAGE_ROOT = storageRoot;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
if (prevEnv === undefined) {
|
||||
delete process.env.UNCAGED_WORKFLOW_STORAGE_ROOT;
|
||||
} else {
|
||||
process.env.UNCAGED_WORKFLOW_STORAGE_ROOT = prevEnv;
|
||||
}
|
||||
await rm(storageRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("garbageCollectCas keeps CAS entries referenced by thread refs", async () => {
|
||||
const bundleHash = "C9NMV6V2TQT81";
|
||||
const threadId = "01AAA1111111111111111111";
|
||||
const logsDir = join(storageRoot, "logs", bundleHash);
|
||||
await mkdir(logsDir, { recursive: true });
|
||||
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
const activeHash = await cas.put("active-blob");
|
||||
const orphanHash = await cas.put("orphan-blob");
|
||||
|
||||
await writeDemoDataJsonl({
|
||||
path: join(logsDir, `${threadId}.data.jsonl`),
|
||||
threadId,
|
||||
bundleHash,
|
||||
cas,
|
||||
activeHash,
|
||||
});
|
||||
|
||||
const gc = await garbageCollectCas(storageRoot);
|
||||
expect(gc.ok).toBe(true);
|
||||
if (!gc.ok) {
|
||||
return;
|
||||
}
|
||||
expect(gc.value.scannedThreads).toBe(1);
|
||||
expect(gc.value.activeRefs).toBe(2);
|
||||
expect(gc.value.deletedEntries).toBe(1);
|
||||
expect(gc.value.deletedHashes).toEqual([orphanHash]);
|
||||
|
||||
expect(await pathExists(join(getGlobalCasDir(storageRoot), `${activeHash}.txt`))).toBe(true);
|
||||
expect(await pathExists(join(getGlobalCasDir(storageRoot), `${orphanHash}.txt`))).toBe(false);
|
||||
});
|
||||
|
||||
test("garbageCollectCas deletes orphaned CAS when no threads reference them", async () => {
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
const orphanHash = await cas.put("lonely");
|
||||
|
||||
const gc = await garbageCollectCas(storageRoot);
|
||||
expect(gc.ok).toBe(true);
|
||||
if (!gc.ok) {
|
||||
return;
|
||||
}
|
||||
expect(gc.value.scannedThreads).toBe(0);
|
||||
expect(gc.value.activeRefs).toBe(0);
|
||||
expect(gc.value.deletedEntries).toBe(1);
|
||||
expect(gc.value.deletedHashes).toEqual([orphanHash]);
|
||||
expect(await pathExists(join(getGlobalCasDir(storageRoot), `${orphanHash}.txt`))).toBe(false);
|
||||
});
|
||||
|
||||
test("cli gc prints stats", async () => {
|
||||
const bundleHash = "C9NMV6V2TQT81";
|
||||
const threadId = "01BBB2222222222222222222";
|
||||
const logsDir = join(storageRoot, "logs", bundleHash);
|
||||
await mkdir(logsDir, { recursive: true });
|
||||
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
const activeHash = await cas.put("keep-me");
|
||||
await cas.put("drop-me");
|
||||
|
||||
await writeDemoDataJsonl({
|
||||
path: join(logsDir, `${threadId}.data.jsonl`),
|
||||
threadId,
|
||||
bundleHash,
|
||||
cas,
|
||||
activeHash,
|
||||
});
|
||||
|
||||
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
|
||||
const proc = spawnSync(process.execPath, [cliEntryPath, "gc"], { env, encoding: "utf8" });
|
||||
expect(proc.status).toBe(0);
|
||||
expect(String(proc.stdout).trim()).toBe("scanned 1 threads, 2 active refs, deleted 1 entries");
|
||||
});
|
||||
|
||||
test("thread rm triggers gc so unreferenced CAS is removed", async () => {
|
||||
const bundleHash = "C9NMV6V2TQT81";
|
||||
const threadId = "01CCC3333333333333333333";
|
||||
const logsDir = join(storageRoot, "logs", bundleHash);
|
||||
await mkdir(logsDir, { recursive: true });
|
||||
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
const activeHash = await cas.put("pinned-by-ref");
|
||||
await writeDemoDataJsonl({
|
||||
path: join(logsDir, `${threadId}.data.jsonl`),
|
||||
threadId,
|
||||
bundleHash,
|
||||
cas,
|
||||
activeHash,
|
||||
});
|
||||
|
||||
const orphanHash = await cas.put("orphan-after-rm");
|
||||
const orphanPath = join(getGlobalCasDir(storageRoot), `${orphanHash}.txt`);
|
||||
|
||||
const removed = await cmdThreadRemove(storageRoot, threadId);
|
||||
expect(removed.ok).toBe(true);
|
||||
|
||||
expect(await pathExists(orphanPath)).toBe(false);
|
||||
expect(await pathExists(join(getGlobalCasDir(storageRoot), `${activeHash}.txt`))).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,142 @@
|
||||
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
|
||||
import { mkdir, readFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { runCli } from "../src/cli-dispatch.js";
|
||||
import { cmdInitTemplate, cmdInitWorkspace } from "../src/cmd-init.js";
|
||||
import { pathExists } from "../src/fs-utils.js";
|
||||
|
||||
describe("init template", () => {
|
||||
let parent: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
parent = join(
|
||||
tmpdir(),
|
||||
`wf-init-template-${Date.now()}-${Math.random().toString(36).slice(2)}`,
|
||||
);
|
||||
await mkdir(parent, { recursive: true });
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await rm(parent, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("creates templates/<name> with expected files", async () => {
|
||||
const ws = await cmdInitWorkspace(parent, "my-workflows");
|
||||
expect(ws.ok).toBe(true);
|
||||
if (!ws.ok) {
|
||||
return;
|
||||
}
|
||||
const root = ws.value.rootPath;
|
||||
|
||||
const created = await cmdInitTemplate(root, "review-pr");
|
||||
expect(created.ok).toBe(true);
|
||||
if (!created.ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
const tdir = join(root, "templates", "review-pr");
|
||||
expect(created.value.templatePath).toBe(tdir);
|
||||
expect(await pathExists(join(tdir, "package.json"))).toBe(true);
|
||||
expect(await pathExists(join(tdir, "tsconfig.json"))).toBe(true);
|
||||
expect(await pathExists(join(tdir, "src", "roles.ts"))).toBe(true);
|
||||
expect(await pathExists(join(tdir, "src", "moderator.ts"))).toBe(true);
|
||||
expect(await pathExists(join(tdir, "src", "index.ts"))).toBe(true);
|
||||
|
||||
const pkg = JSON.parse(await readFile(join(tdir, "package.json"), "utf8")) as {
|
||||
name: string;
|
||||
type: string;
|
||||
dependencies: Record<string, string>;
|
||||
};
|
||||
expect(pkg.type).toBe("module");
|
||||
expect(pkg.dependencies["@uncaged/workflow"]).toBeDefined();
|
||||
expect(pkg.dependencies.zod).toBeDefined();
|
||||
expect(pkg.name).toContain("review-pr");
|
||||
|
||||
const idx = await readFile(join(tdir, "src", "index.ts"), "utf8");
|
||||
expect(idx).toContain("WorkflowDefinition");
|
||||
|
||||
const roles = await readFile(join(tdir, "src", "roles.ts"), "utf8");
|
||||
expect(roles).not.toContain("interface ");
|
||||
expect(roles).not.toContain("?:");
|
||||
expect(roles).not.toContain("export default");
|
||||
|
||||
const moder = await readFile(join(tdir, "src", "moderator.ts"), "utf8");
|
||||
expect(moder).not.toContain("export default");
|
||||
});
|
||||
|
||||
test("finds workspace walking up from nested cwd", async () => {
|
||||
const ws = await cmdInitWorkspace(parent, "ws");
|
||||
expect(ws.ok).toBe(true);
|
||||
if (!ws.ok) {
|
||||
return;
|
||||
}
|
||||
const root = ws.value.rootPath;
|
||||
const nested = join(root, "a", "b");
|
||||
await mkdir(nested, { recursive: true });
|
||||
|
||||
const created = await cmdInitTemplate(nested, "nested-tpl");
|
||||
expect(created.ok).toBe(true);
|
||||
if (!created.ok) {
|
||||
return;
|
||||
}
|
||||
expect(await pathExists(join(root, "templates", "nested-tpl", "src", "index.ts"))).toBe(true);
|
||||
});
|
||||
|
||||
test("errors when not inside a workflow workspace", async () => {
|
||||
const orphan = join(parent, "nowhere");
|
||||
await mkdir(orphan, { recursive: true });
|
||||
const r = await cmdInitTemplate(orphan, "x");
|
||||
expect(r.ok).toBe(false);
|
||||
if (!r.ok) {
|
||||
expect(r.error).toContain("templates/*");
|
||||
}
|
||||
});
|
||||
|
||||
test("errors when template directory already exists", async () => {
|
||||
const ws = await cmdInitWorkspace(parent, "ws");
|
||||
expect(ws.ok).toBe(true);
|
||||
if (!ws.ok) {
|
||||
return;
|
||||
}
|
||||
const root = ws.value.rootPath;
|
||||
|
||||
const first = await cmdInitTemplate(root, "dup");
|
||||
expect(first.ok).toBe(true);
|
||||
|
||||
const second = await cmdInitTemplate(root, "dup");
|
||||
expect(second.ok).toBe(false);
|
||||
if (!second.ok) {
|
||||
expect(second.error).toContain("already exists");
|
||||
}
|
||||
});
|
||||
|
||||
test("errors on invalid template name", async () => {
|
||||
const ws = await cmdInitWorkspace(parent, "ws");
|
||||
expect(ws.ok).toBe(true);
|
||||
if (!ws.ok) {
|
||||
return;
|
||||
}
|
||||
const bad = await cmdInitTemplate(ws.value.rootPath, "a/b");
|
||||
expect(bad.ok).toBe(false);
|
||||
});
|
||||
|
||||
test.serial("runCli init template uses cwd and succeeds in workspace", async () => {
|
||||
const ws = await cmdInitWorkspace(parent, "cli-ws");
|
||||
expect(ws.ok).toBe(true);
|
||||
if (!ws.ok) {
|
||||
return;
|
||||
}
|
||||
const root = ws.value.rootPath;
|
||||
const prev = process.cwd();
|
||||
try {
|
||||
process.chdir(root);
|
||||
const code = await runCli(join(parent, "_storage"), ["init", "template", "from-cli"]);
|
||||
expect(code).toBe(0);
|
||||
expect(await pathExists(join(root, "templates", "from-cli", "package.json"))).toBe(true);
|
||||
} finally {
|
||||
process.chdir(prev);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,152 @@
|
||||
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
|
||||
import { mkdir, readFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { formatCliUsage, runCli } from "../src/cli-dispatch.js";
|
||||
import { cmdInitWorkspace } from "../src/cmd-init.js";
|
||||
import { pathExists } from "../src/fs-utils.js";
|
||||
|
||||
describe("init workspace", () => {
|
||||
let parent: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
parent = join(tmpdir(), `wf-init-${Date.now()}-${Math.random().toString(36).slice(2)}`);
|
||||
await mkdir(parent, { recursive: true });
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await rm(parent, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("creates expected files and directories", async () => {
|
||||
const created = await cmdInitWorkspace(parent, "my-workflows");
|
||||
expect(created.ok).toBe(true);
|
||||
if (!created.ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
const root = created.value.rootPath;
|
||||
expect(await pathExists(join(root, "package.json"))).toBe(true);
|
||||
expect(await pathExists(join(root, "biome.json"))).toBe(true);
|
||||
expect(await pathExists(join(root, "tsconfig.json"))).toBe(true);
|
||||
expect(await pathExists(join(root, "AGENTS.md"))).toBe(true);
|
||||
expect(await pathExists(join(root, "README.md"))).toBe(true);
|
||||
expect(await pathExists(join(root, "templates"))).toBe(true);
|
||||
expect(await pathExists(join(root, "templates", ".gitkeep"))).toBe(true);
|
||||
expect(await pathExists(join(root, "workflows", "package.json"))).toBe(true);
|
||||
|
||||
const rootPkg = JSON.parse(await readFile(join(root, "package.json"), "utf8")) as {
|
||||
workspaces: string[];
|
||||
};
|
||||
expect(rootPkg.workspaces).toEqual(["templates/*", "workflows"]);
|
||||
|
||||
const wfPkg = JSON.parse(await readFile(join(root, "workflows", "package.json"), "utf8")) as {
|
||||
type: string;
|
||||
dependencies: Record<string, string>;
|
||||
};
|
||||
expect(wfPkg.type).toBe("module");
|
||||
expect(wfPkg.dependencies["@uncaged/workflow"]).toBeDefined();
|
||||
expect(wfPkg.dependencies.zod).toBeDefined();
|
||||
|
||||
const tsconfig = JSON.parse(await readFile(join(root, "tsconfig.json"), "utf8")) as {
|
||||
compilerOptions: { strict: boolean; module: string; target: string };
|
||||
};
|
||||
expect(tsconfig.compilerOptions.strict).toBe(true);
|
||||
expect(tsconfig.compilerOptions.module).toBe("ESNext");
|
||||
expect(tsconfig.compilerOptions.target).toBe("ESNext");
|
||||
});
|
||||
|
||||
test("AGENTS.md contains coding agent guide sections and terms", async () => {
|
||||
const created = await cmdInitWorkspace(parent, "my-workflows");
|
||||
expect(created.ok).toBe(true);
|
||||
if (!created.ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
const agentsPath = join(created.value.rootPath, "AGENTS.md");
|
||||
const body = await readFile(agentsPath, "utf8");
|
||||
|
||||
for (const section of [
|
||||
"项目结构",
|
||||
"核心概念",
|
||||
"开发流程",
|
||||
"编码规范",
|
||||
"Template",
|
||||
"Build",
|
||||
"常见陷阱",
|
||||
]) {
|
||||
expect(body).toContain(section);
|
||||
}
|
||||
|
||||
for (const term of [
|
||||
"RoleDefinition",
|
||||
"WorkflowDefinition",
|
||||
"Moderator",
|
||||
"AgentFn",
|
||||
"ExtractFn",
|
||||
"RoleMeta",
|
||||
]) {
|
||||
expect(body).toContain(term);
|
||||
}
|
||||
|
||||
expect(body).toMatch(/type[\s\S]*interface/i);
|
||||
expect(body).toMatch(/function[\s\S]*class/i);
|
||||
expect(body).toContain("Crockford Base32");
|
||||
expect(body).toMatch(/no[\s\S]*default export/i);
|
||||
expect(body).toMatch(/no[\s\S]*console/i);
|
||||
expect(body).toMatch(/no[\s\S]*dynamic import/i);
|
||||
|
||||
expect(body).toContain("bun run check");
|
||||
expect(body).toContain("bun test");
|
||||
expect(body).toContain("uncaged-workflow");
|
||||
expect(body).toContain("bun build");
|
||||
expect(body).toContain("CLAUDE.md");
|
||||
expect(body).toContain("docs/architecture.md");
|
||||
});
|
||||
|
||||
test("errors when directory already exists", async () => {
|
||||
const first = await cmdInitWorkspace(parent, "dup");
|
||||
expect(first.ok).toBe(true);
|
||||
|
||||
const second = await cmdInitWorkspace(parent, "dup");
|
||||
expect(second.ok).toBe(false);
|
||||
if (!second.ok) {
|
||||
expect(second.error).toContain("already exists");
|
||||
}
|
||||
});
|
||||
|
||||
test("errors on invalid workspace name", async () => {
|
||||
const slash = await cmdInitWorkspace(parent, "a/b");
|
||||
expect(slash.ok).toBe(false);
|
||||
|
||||
const dots = await cmdInitWorkspace(parent, "..");
|
||||
expect(dots.ok).toBe(false);
|
||||
|
||||
const empty = await cmdInitWorkspace(parent, "");
|
||||
expect(empty.ok).toBe(false);
|
||||
});
|
||||
|
||||
test("usage lists init subcommands", () => {
|
||||
const u = formatCliUsage();
|
||||
expect(u).toContain("uncaged-workflow init workspace <name>");
|
||||
expect(u).toContain("uncaged-workflow init template <name>");
|
||||
});
|
||||
|
||||
test("runCli rejects unknown init subcommand", async () => {
|
||||
const code = await runCli(join(parent, "_storage"), ["init", "bogus", "name"]);
|
||||
expect(code).toBe(1);
|
||||
});
|
||||
|
||||
test.serial("runCli init workspace uses cwd", async () => {
|
||||
const prev = process.cwd();
|
||||
try {
|
||||
process.chdir(parent);
|
||||
const code = await runCli(join(parent, "_storage"), ["init", "workspace", "from-cli"]);
|
||||
expect(code).toBe(0);
|
||||
expect(await pathExists(join(parent, "from-cli", "workflows", "package.json"))).toBe(true);
|
||||
} finally {
|
||||
process.chdir(prev);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -4,7 +4,9 @@ import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { getGlobalCasDir } from "@uncaged/workflow";
|
||||
import { cmdAdd } from "../src/cmd-add.js";
|
||||
import { cmdCasPut } from "../src/cmd-cas.js";
|
||||
import { cmdKill } from "../src/cmd-kill.js";
|
||||
import { cmdPause } from "../src/cmd-pause.js";
|
||||
import { cmdPs } from "../src/cmd-ps.js";
|
||||
@@ -12,9 +14,12 @@ import { cmdResume } from "../src/cmd-resume.js";
|
||||
import { cmdRun } from "../src/cmd-run.js";
|
||||
import { cmdThreadRemove, cmdThreadShow } from "../src/cmd-thread.js";
|
||||
import { cmdThreads } from "../src/cmd-threads.js";
|
||||
import { pathExists } from "../src/fs-utils.js";
|
||||
import { pathExists, readTextFileIfExists } from "../src/fs-utils.js";
|
||||
import { addCliArgs } from "./bundle-fixture.js";
|
||||
|
||||
const wfPutImport = `import { putContentMerkleNode } from "@uncaged/workflow";
|
||||
`;
|
||||
|
||||
const threadFixtureDescriptor = `export const descriptor = {
|
||||
description: "thread-cli",
|
||||
roles: {
|
||||
@@ -29,18 +34,26 @@ const threadFixtureDescriptor = `export const descriptor = {
|
||||
`;
|
||||
|
||||
const fastBundleSource = `${threadFixtureDescriptor}
|
||||
export const run = async function* (input) {
|
||||
yield { role: "planner", content: "plan", meta: { plan: input.prompt } };
|
||||
yield { role: "coder", content: "code", meta: { diff: "y" } };
|
||||
${wfPutImport}
|
||||
export const run = async function* (input, options) {
|
||||
const cas = options.cas;
|
||||
let h = await putContentMerkleNode(cas, "plan");
|
||||
yield { role: "planner", contentHash: h, meta: { plan: input.prompt }, refs: [h] };
|
||||
h = await putContentMerkleNode(cas, "code");
|
||||
yield { role: "coder", contentHash: h, meta: { diff: "y" }, refs: [h] };
|
||||
return { returnCode: 0, summary: "done" };
|
||||
};
|
||||
`;
|
||||
|
||||
const slowPlannerBundleSource = `${threadFixtureDescriptor}
|
||||
export const run = async function* (input) {
|
||||
${wfPutImport}
|
||||
export const run = async function* (input, options) {
|
||||
await new Promise((r) => setTimeout(r, 400));
|
||||
yield { role: "planner", content: "plan", meta: { plan: input.prompt } };
|
||||
yield { role: "coder", content: "code", meta: { diff: "y" } };
|
||||
const cas = options.cas;
|
||||
let h = await putContentMerkleNode(cas, "plan");
|
||||
yield { role: "planner", contentHash: h, meta: { plan: input.prompt }, refs: [h] };
|
||||
h = await putContentMerkleNode(cas, "code");
|
||||
yield { role: "coder", contentHash: h, meta: { diff: "y" }, refs: [h] };
|
||||
return { returnCode: 0, summary: "done" };
|
||||
};
|
||||
`;
|
||||
@@ -48,27 +61,38 @@ export const run = async function* (input) {
|
||||
const cliEntryPath = fileURLToPath(new URL("../src/cli.ts", import.meta.url));
|
||||
|
||||
const abortablePlannerBundleSource = `${threadFixtureDescriptor}
|
||||
export const run = async function* (input) {
|
||||
${wfPutImport}
|
||||
export const run = async function* (input, options) {
|
||||
await new Promise((r) => setTimeout(r, 600));
|
||||
yield { role: "planner", content: "plan", meta: { plan: input.prompt } };
|
||||
yield { role: "coder", content: "code", meta: { diff: "y" } };
|
||||
const cas = options.cas;
|
||||
let h = await putContentMerkleNode(cas, "plan");
|
||||
yield { role: "planner", contentHash: h, meta: { plan: input.prompt }, refs: [h] };
|
||||
h = await putContentMerkleNode(cas, "code");
|
||||
yield { role: "coder", contentHash: h, meta: { diff: "y" }, refs: [h] };
|
||||
return { returnCode: 0, summary: "done" };
|
||||
};
|
||||
`;
|
||||
|
||||
const pauseResumeBundleSource = `${threadFixtureDescriptor}
|
||||
export const run = async function* (input) {
|
||||
yield { role: "first", content: "f", meta: {} };
|
||||
${wfPutImport}
|
||||
export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
let h = await putContentMerkleNode(cas, "f");
|
||||
yield { role: "first", contentHash: h, meta: {}, refs: [h] };
|
||||
await new Promise((r) => setTimeout(r, 1500));
|
||||
yield { role: "second", content: "s", meta: {} };
|
||||
h = await putContentMerkleNode(cas, "s");
|
||||
yield { role: "second", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "done" };
|
||||
};
|
||||
`;
|
||||
|
||||
const delayedFirstYieldBundleSource = `${threadFixtureDescriptor}
|
||||
export const run = async function* (input) {
|
||||
${wfPutImport}
|
||||
export const run = async function* (_input, options) {
|
||||
await new Promise((r) => setTimeout(r, 900));
|
||||
yield { role: "only", content: "x", meta: {} };
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "x");
|
||||
yield { role: "only", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "done" };
|
||||
};
|
||||
`;
|
||||
@@ -175,6 +199,55 @@ describe("cli thread commands", () => {
|
||||
expect(await pathExists(dataPath)).toBe(false);
|
||||
});
|
||||
|
||||
test("thread rm runs GC and removes CAS blobs not referenced by any remaining thread", async () => {
|
||||
const bundleDir = join(storageRoot, "src");
|
||||
await mkdir(bundleDir, { recursive: true });
|
||||
const bundlePath = join(bundleDir, "demo.esm.js");
|
||||
await writeFile(bundlePath, fastBundleSource, "utf8");
|
||||
|
||||
const added = await cmdAdd(storageRoot, addCliArgs("solve-issue", bundlePath));
|
||||
expect(added.ok).toBe(true);
|
||||
if (!added.ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
const ran = await cmdRun(storageRoot, "solve-issue", "hello", 5);
|
||||
expect(ran.ok).toBe(true);
|
||||
if (!ran.ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
const threadId = ran.value.threadId;
|
||||
|
||||
let threads = await cmdThreads(storageRoot, []);
|
||||
for (
|
||||
let attempt = 0;
|
||||
attempt < 50 && threads.ok && !threads.value.some((l) => l.includes(threadId));
|
||||
attempt++
|
||||
) {
|
||||
await new Promise((r) => setTimeout(r, 20));
|
||||
threads = await cmdThreads(storageRoot, []);
|
||||
}
|
||||
|
||||
const dataPath = join(storageRoot, "logs", added.value.hash, `${threadId}.data.jsonl`);
|
||||
const runningPath = join(dirname(dataPath), `${threadId}.running`);
|
||||
await waitUntilRunningFileAbsent(runningPath, 120);
|
||||
|
||||
const put = await cmdCasPut(storageRoot, threadId, "keep-after-thread-rm");
|
||||
expect(put.ok).toBe(true);
|
||||
if (!put.ok) {
|
||||
return;
|
||||
}
|
||||
const hash = put.value;
|
||||
const casBlob = join(getGlobalCasDir(storageRoot), `${hash}.txt`);
|
||||
|
||||
const removed = await cmdThreadRemove(storageRoot, threadId);
|
||||
expect(removed.ok).toBe(true);
|
||||
|
||||
const stillThere = await readTextFileIfExists(casBlob);
|
||||
expect(stillThere).toBeNull();
|
||||
});
|
||||
|
||||
test("cli entrypoint dispatches threads / ps (spawn)", () => {
|
||||
const env = { ...process.env, UNCAGED_WORKFLOW_STORAGE_ROOT: storageRoot };
|
||||
const threads = spawnSync(process.execPath, [cliEntryPath, "threads"], {
|
||||
|
||||
@@ -2,7 +2,9 @@ import { printCliError, printCliLine, printCliWarn } from "./cli-output.js";
|
||||
import { cmdAdd, formatAddSuccess, parseAddArgv } from "./cmd-add.js";
|
||||
import { cmdCasGet, cmdCasList, cmdCasPut, cmdCasRm } from "./cmd-cas.js";
|
||||
import { cmdFork, parseForkArgv } from "./cmd-fork.js";
|
||||
import { cmdGc } from "./cmd-gc.js";
|
||||
import { cmdHistory } from "./cmd-history.js";
|
||||
import { cmdInitTemplate, cmdInitWorkspace } from "./cmd-init.js";
|
||||
import { cmdKill } from "./cmd-kill.js";
|
||||
import { cmdList, formatListLines } from "./cmd-list.js";
|
||||
import { cmdPause } from "./cmd-pause.js";
|
||||
@@ -16,7 +18,7 @@ import { cmdThreadRemove, cmdThreadShow } from "./cmd-thread.js";
|
||||
import { cmdThreads } from "./cmd-threads.js";
|
||||
import { parseRunArgv } from "./run-argv.js";
|
||||
|
||||
function usage(): string {
|
||||
export function formatCliUsage(): string {
|
||||
return [
|
||||
"Usage:",
|
||||
" uncaged-workflow add <name> <file.esm.js> [--types <path>]",
|
||||
@@ -34,17 +36,52 @@ function usage(): string {
|
||||
" uncaged-workflow thread <id>",
|
||||
" uncaged-workflow thread rm <id>",
|
||||
" uncaged-workflow fork <thread-id> [--from-role <role>]",
|
||||
" uncaged-workflow gc",
|
||||
" uncaged-workflow cas get <thread-id> <hash>",
|
||||
" uncaged-workflow cas put <thread-id> <content>",
|
||||
" uncaged-workflow cas list <thread-id>",
|
||||
" uncaged-workflow cas rm <thread-id> <hash>",
|
||||
" uncaged-workflow init workspace <name>",
|
||||
" uncaged-workflow init template <name>",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
async function dispatchInit(_storageRoot: string, argv: string[]): Promise<number> {
|
||||
const sub = argv[0];
|
||||
const name = argv[1];
|
||||
if (sub === undefined || name === undefined || argv.length > 2) {
|
||||
printCliError(`${formatCliUsage()}\n\nerror: init requires workspace|template <name>`);
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (sub === "workspace") {
|
||||
const result = await cmdInitWorkspace(process.cwd(), name);
|
||||
if (!result.ok) {
|
||||
printCliError(result.error);
|
||||
return 1;
|
||||
}
|
||||
printCliLine(`initialized workflow workspace at ${result.value.rootPath}`);
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (sub === "template") {
|
||||
const result = await cmdInitTemplate(process.cwd(), name);
|
||||
if (!result.ok) {
|
||||
printCliError(result.error);
|
||||
return 1;
|
||||
}
|
||||
printCliLine(`initialized template at ${result.value.templatePath}`);
|
||||
return 0;
|
||||
}
|
||||
|
||||
printCliError(`${formatCliUsage()}\n\nerror: unknown init subcommand: ${sub}`);
|
||||
return 1;
|
||||
}
|
||||
|
||||
async function dispatchAdd(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const parsed = parseAddArgv(argv);
|
||||
if (!parsed.ok) {
|
||||
printCliError(`${usage()}\n\nerror: ${parsed.error}`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: ${parsed.error}`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdAdd(storageRoot, parsed.value);
|
||||
@@ -61,7 +98,7 @@ async function dispatchAdd(storageRoot: string, argv: string[]): Promise<number>
|
||||
|
||||
async function dispatchList(storageRoot: string, argv: string[]): Promise<number> {
|
||||
if (argv.length > 0) {
|
||||
printCliError(`${usage()}\n\nerror: list takes no arguments`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: list takes no arguments`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdList(storageRoot);
|
||||
@@ -78,7 +115,7 @@ async function dispatchList(storageRoot: string, argv: string[]): Promise<number
|
||||
async function dispatchShow(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const name = argv[0];
|
||||
if (name === undefined || argv.length > 1) {
|
||||
printCliError(`${usage()}\n\nerror: show requires <name>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: show requires <name>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdShow(storageRoot, name);
|
||||
@@ -93,7 +130,7 @@ async function dispatchShow(storageRoot: string, argv: string[]): Promise<number
|
||||
async function dispatchRemove(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const name = argv[0];
|
||||
if (name === undefined || argv.length > 1) {
|
||||
printCliError(`${usage()}\n\nerror: remove requires <name>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: remove requires <name>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdRemove(storageRoot, name);
|
||||
@@ -108,7 +145,7 @@ async function dispatchRemove(storageRoot: string, argv: string[]): Promise<numb
|
||||
async function dispatchRun(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const parsed = parseRunArgv(argv);
|
||||
if (!parsed.ok) {
|
||||
printCliError(`${usage()}\n\nerror: ${parsed.error}`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: ${parsed.error}`);
|
||||
return 1;
|
||||
}
|
||||
|
||||
@@ -129,7 +166,7 @@ async function dispatchRun(storageRoot: string, argv: string[]): Promise<number>
|
||||
|
||||
async function dispatchPs(storageRoot: string, argv: string[]): Promise<number> {
|
||||
if (argv.length > 0) {
|
||||
printCliError(`${usage()}\n\nerror: ps takes no arguments`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: ps takes no arguments`);
|
||||
return 1;
|
||||
}
|
||||
for (const line of await cmdPs(storageRoot)) {
|
||||
@@ -141,7 +178,7 @@ async function dispatchPs(storageRoot: string, argv: string[]): Promise<number>
|
||||
async function dispatchKill(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const threadId = argv[0];
|
||||
if (threadId === undefined || argv.length > 1) {
|
||||
printCliError(`${usage()}\n\nerror: kill requires <thread-id>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: kill requires <thread-id>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdKill(storageRoot, threadId);
|
||||
@@ -156,7 +193,7 @@ async function dispatchKill(storageRoot: string, argv: string[]): Promise<number
|
||||
async function dispatchHistory(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const name = argv[0];
|
||||
if (name === undefined || argv.length > 1) {
|
||||
printCliError(`${usage()}\n\nerror: history requires <name>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: history requires <name>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdHistory(storageRoot, name);
|
||||
@@ -173,7 +210,7 @@ async function dispatchHistory(storageRoot: string, argv: string[]): Promise<num
|
||||
async function dispatchRollback(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const name = argv[0];
|
||||
if (name === undefined || argv.length > 2) {
|
||||
printCliError(`${usage()}\n\nerror: rollback requires <name> [hash]`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: rollback requires <name> [hash]`);
|
||||
return 1;
|
||||
}
|
||||
const hashArg = argv[1];
|
||||
@@ -189,7 +226,7 @@ async function dispatchRollback(storageRoot: string, argv: string[]): Promise<nu
|
||||
async function dispatchPause(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const threadId = argv[0];
|
||||
if (threadId === undefined || argv.length > 1) {
|
||||
printCliError(`${usage()}\n\nerror: pause requires <thread-id>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: pause requires <thread-id>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdPause(storageRoot, threadId);
|
||||
@@ -204,7 +241,7 @@ async function dispatchPause(storageRoot: string, argv: string[]): Promise<numbe
|
||||
async function dispatchResume(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const threadId = argv[0];
|
||||
if (threadId === undefined || argv.length > 1) {
|
||||
printCliError(`${usage()}\n\nerror: resume requires <thread-id>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: resume requires <thread-id>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdResume(storageRoot, threadId);
|
||||
@@ -231,7 +268,7 @@ async function dispatchThreads(storageRoot: string, argv: string[]): Promise<num
|
||||
async function dispatchThread(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const id = argv[0];
|
||||
if (id === undefined || argv.length > 1) {
|
||||
printCliError(`${usage()}\n\nerror: thread requires <id>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: thread requires <id>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdThreadShow(storageRoot, id);
|
||||
@@ -246,7 +283,7 @@ async function dispatchThread(storageRoot: string, argv: string[]): Promise<numb
|
||||
async function dispatchThreadRm(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const id = argv[0];
|
||||
if (id === undefined || argv.length > 1) {
|
||||
printCliError(`${usage()}\n\nerror: thread rm requires <id>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: thread rm requires <id>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdThreadRemove(storageRoot, id);
|
||||
@@ -266,10 +303,27 @@ async function dispatchThreadBranch(storageRoot: string, rest: string[]): Promis
|
||||
return dispatchThread(storageRoot, rest);
|
||||
}
|
||||
|
||||
async function dispatchGc(storageRoot: string, argv: string[]): Promise<number> {
|
||||
if (argv.length > 0) {
|
||||
printCliError(`${formatCliUsage()}\n\nerror: gc takes no arguments`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdGc(storageRoot);
|
||||
if (!result.ok) {
|
||||
printCliError(result.error);
|
||||
return 1;
|
||||
}
|
||||
const stats = result.value;
|
||||
printCliLine(
|
||||
`scanned ${stats.scannedThreads} threads, ${stats.activeRefs} active refs, deleted ${stats.deletedEntries} entries`,
|
||||
);
|
||||
return 0;
|
||||
}
|
||||
|
||||
async function dispatchFork(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const parsed = parseForkArgv(argv);
|
||||
if (!parsed.ok) {
|
||||
printCliError(`${usage()}\n\nerror: ${parsed.error}`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: ${parsed.error}`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdFork(storageRoot, parsed.value.threadId, parsed.value.fromRole);
|
||||
@@ -285,7 +339,7 @@ async function dispatchCasGet(storageRoot: string, rest: string[]): Promise<numb
|
||||
const threadId = rest[0];
|
||||
const hash = rest[1];
|
||||
if (threadId === undefined || hash === undefined || rest.length > 2) {
|
||||
printCliError(`${usage()}\n\nerror: cas get requires <thread-id> <hash>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: cas get requires <thread-id> <hash>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdCasGet(storageRoot, threadId, hash);
|
||||
@@ -301,7 +355,7 @@ async function dispatchCasPut(storageRoot: string, rest: string[]): Promise<numb
|
||||
const threadId = rest[0];
|
||||
const content = rest[1];
|
||||
if (threadId === undefined || content === undefined || rest.length > 2) {
|
||||
printCliError(`${usage()}\n\nerror: cas put requires <thread-id> <content>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: cas put requires <thread-id> <content>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdCasPut(storageRoot, threadId, content);
|
||||
@@ -316,7 +370,7 @@ async function dispatchCasPut(storageRoot: string, rest: string[]): Promise<numb
|
||||
async function dispatchCasList(storageRoot: string, rest: string[]): Promise<number> {
|
||||
const threadId = rest[0];
|
||||
if (threadId === undefined || rest.length > 1) {
|
||||
printCliError(`${usage()}\n\nerror: cas list requires <thread-id>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: cas list requires <thread-id>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdCasList(storageRoot, threadId);
|
||||
@@ -334,7 +388,7 @@ async function dispatchCasRm(storageRoot: string, rest: string[]): Promise<numbe
|
||||
const threadId = rest[0];
|
||||
const hash = rest[1];
|
||||
if (threadId === undefined || hash === undefined || rest.length > 2) {
|
||||
printCliError(`${usage()}\n\nerror: cas rm requires <thread-id> <hash>`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: cas rm requires <thread-id> <hash>`);
|
||||
return 1;
|
||||
}
|
||||
const result = await cmdCasRm(storageRoot, threadId, hash);
|
||||
@@ -359,12 +413,12 @@ const CAS_SUBCOMMAND_TABLE: Record<
|
||||
async function dispatchCas(storageRoot: string, argv: string[]): Promise<number> {
|
||||
const sub = argv[0];
|
||||
if (sub === undefined) {
|
||||
printCliError(`${usage()}\n\nerror: unknown cas subcommand: (none)`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: unknown cas subcommand: (none)`);
|
||||
return 1;
|
||||
}
|
||||
const handler = CAS_SUBCOMMAND_TABLE[sub];
|
||||
if (handler === undefined) {
|
||||
printCliError(`${usage()}\n\nerror: unknown cas subcommand: ${sub}`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: unknown cas subcommand: ${sub}`);
|
||||
return 1;
|
||||
}
|
||||
return handler(storageRoot, argv.slice(1));
|
||||
@@ -374,6 +428,7 @@ type DispatchFn = (storageRoot: string, argv: string[]) => Promise<number>;
|
||||
|
||||
const COMMAND_TABLE: Record<string, DispatchFn> = {
|
||||
add: dispatchAdd,
|
||||
init: dispatchInit,
|
||||
list: dispatchList,
|
||||
show: dispatchShow,
|
||||
remove: dispatchRemove,
|
||||
@@ -387,23 +442,24 @@ const COMMAND_TABLE: Record<string, DispatchFn> = {
|
||||
threads: dispatchThreads,
|
||||
thread: dispatchThreadBranch,
|
||||
fork: dispatchFork,
|
||||
gc: dispatchGc,
|
||||
cas: dispatchCas,
|
||||
};
|
||||
|
||||
export async function runCli(storageRoot: string, argv: string[]): Promise<number> {
|
||||
if (argv.length === 0) {
|
||||
printCliError(usage());
|
||||
printCliError(formatCliUsage());
|
||||
return 1;
|
||||
}
|
||||
const command = argv[0];
|
||||
if (command === undefined) {
|
||||
printCliError(usage());
|
||||
printCliError(formatCliUsage());
|
||||
return 1;
|
||||
}
|
||||
const rest = argv.slice(1);
|
||||
const dispatch = COMMAND_TABLE[command];
|
||||
if (dispatch === undefined) {
|
||||
printCliError(`${usage()}\n\nerror: unknown command ${command}`);
|
||||
printCliError(`${formatCliUsage()}\n\nerror: unknown command ${command}`);
|
||||
return 1;
|
||||
}
|
||||
return dispatch(storageRoot, rest);
|
||||
|
||||
Regular → Executable
@@ -192,7 +192,7 @@ export async function cmdAdd(
|
||||
return validated;
|
||||
}
|
||||
|
||||
const extracted = await extractBundleExports(resolvedPath);
|
||||
const extracted = await extractBundleExports(resolvedPath, { storageRoot });
|
||||
if (!extracted.ok) {
|
||||
return extracted;
|
||||
}
|
||||
|
||||
@@ -1,23 +1,11 @@
|
||||
import { dirname, join } from "node:path";
|
||||
|
||||
import { createThreadCas, err, ok, type Result } from "@uncaged/workflow";
|
||||
|
||||
import { resolveThreadDataPath } from "./thread-scan.js";
|
||||
|
||||
function resolveCasDir(threadDataPath: string, threadId: string): string {
|
||||
return join(dirname(threadDataPath), `${threadId}.cas`);
|
||||
}
|
||||
import { createCasStore, err, getGlobalCasDir, ok, type Result } from "@uncaged/workflow";
|
||||
|
||||
export async function cmdCasGet(
|
||||
storageRoot: string,
|
||||
threadId: string,
|
||||
_threadId: string,
|
||||
hash: string,
|
||||
): Promise<Result<string, string>> {
|
||||
const dataPath = await resolveThreadDataPath(storageRoot, threadId);
|
||||
if (dataPath === null) {
|
||||
return err(`thread not found: ${threadId}`);
|
||||
}
|
||||
const cas = createThreadCas(resolveCasDir(dataPath, threadId));
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
const content = await cas.get(hash);
|
||||
if (content === null) {
|
||||
return err(`cas entry not found: ${hash}`);
|
||||
@@ -27,41 +15,29 @@ export async function cmdCasGet(
|
||||
|
||||
export async function cmdCasPut(
|
||||
storageRoot: string,
|
||||
threadId: string,
|
||||
_threadId: string,
|
||||
content: string,
|
||||
): Promise<Result<string, string>> {
|
||||
const dataPath = await resolveThreadDataPath(storageRoot, threadId);
|
||||
if (dataPath === null) {
|
||||
return err(`thread not found: ${threadId}`);
|
||||
}
|
||||
const cas = createThreadCas(resolveCasDir(dataPath, threadId));
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
const hash = await cas.put(content);
|
||||
return ok(hash);
|
||||
}
|
||||
|
||||
export async function cmdCasList(
|
||||
storageRoot: string,
|
||||
threadId: string,
|
||||
_threadId: string,
|
||||
): Promise<Result<string[], string>> {
|
||||
const dataPath = await resolveThreadDataPath(storageRoot, threadId);
|
||||
if (dataPath === null) {
|
||||
return err(`thread not found: ${threadId}`);
|
||||
}
|
||||
const cas = createThreadCas(resolveCasDir(dataPath, threadId));
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
const hashes = await cas.list();
|
||||
return ok(hashes);
|
||||
}
|
||||
|
||||
export async function cmdCasRm(
|
||||
storageRoot: string,
|
||||
threadId: string,
|
||||
_threadId: string,
|
||||
hash: string,
|
||||
): Promise<Result<void, string>> {
|
||||
const dataPath = await resolveThreadDataPath(storageRoot, threadId);
|
||||
if (dataPath === null) {
|
||||
return err(`thread not found: ${threadId}`);
|
||||
}
|
||||
const cas = createThreadCas(resolveCasDir(dataPath, threadId));
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
await cas.delete(hash);
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
@@ -65,8 +65,9 @@ export async function cmdFork(
|
||||
const newThreadId = generateUlid(Date.now());
|
||||
const stepsOnWire = plan.value.historicalSteps.map((s) => ({
|
||||
role: s.role,
|
||||
content: s.content,
|
||||
contentHash: s.contentHash,
|
||||
meta: s.meta,
|
||||
refs: s.refs,
|
||||
timestamp: s.timestamp,
|
||||
}));
|
||||
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
import { type GcResult, garbageCollectCas, type Result } from "@uncaged/workflow";
|
||||
|
||||
export async function cmdGc(storageRoot: string): Promise<Result<GcResult, string>> {
|
||||
return garbageCollectCas(storageRoot);
|
||||
}
|
||||
@@ -0,0 +1,415 @@
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
|
||||
import { err, ok, type Result } from "@uncaged/workflow";
|
||||
|
||||
import { pathExists } from "./fs-utils.js";
|
||||
|
||||
export type CmdInitWorkspaceSuccess = {
|
||||
rootPath: string;
|
||||
};
|
||||
|
||||
export type CmdInitTemplateSuccess = {
|
||||
templatePath: string;
|
||||
};
|
||||
|
||||
function validateWorkspaceSegment(name: string): Result<void, string> {
|
||||
if (name.length === 0) {
|
||||
return err("workspace name must not be empty");
|
||||
}
|
||||
if (name === "." || name === "..") {
|
||||
return err("invalid workspace name");
|
||||
}
|
||||
if (name.includes("/") || name.includes("\\")) {
|
||||
return err("workspace name must not contain path separators");
|
||||
}
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
function rootPackageJson(workspaceName: string): string {
|
||||
return `${JSON.stringify(
|
||||
{
|
||||
name: workspaceName,
|
||||
private: true,
|
||||
type: "module",
|
||||
workspaces: ["templates/*", "workflows"],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`;
|
||||
}
|
||||
|
||||
function workflowsPackageJson(): string {
|
||||
return `${JSON.stringify(
|
||||
{
|
||||
name: "workflows",
|
||||
version: "0.0.0",
|
||||
private: true,
|
||||
type: "module",
|
||||
dependencies: {
|
||||
"@uncaged/workflow": "^0.1.0",
|
||||
zod: "^4.0.0",
|
||||
},
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`;
|
||||
}
|
||||
|
||||
function biomeJson(): string {
|
||||
return `${JSON.stringify(
|
||||
{
|
||||
$schema: "https://biomejs.dev/schemas/2.4.14/schema.json",
|
||||
files: {
|
||||
includes: ["**", "!**/node_modules", "!**/dist"],
|
||||
},
|
||||
formatter: {
|
||||
indentWidth: 2,
|
||||
},
|
||||
linter: {
|
||||
enabled: true,
|
||||
rules: {
|
||||
recommended: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`;
|
||||
}
|
||||
|
||||
function tsconfigJson(): string {
|
||||
return `${JSON.stringify(
|
||||
{
|
||||
compilerOptions: {
|
||||
strict: true,
|
||||
target: "ESNext",
|
||||
module: "ESNext",
|
||||
moduleResolution: "Bundler",
|
||||
skipLibCheck: true,
|
||||
},
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`;
|
||||
}
|
||||
|
||||
function agentsMd(): string {
|
||||
return `# AGENTS — Workflow 工作区开发指南
|
||||
|
||||
面向在本仓库中编写 workflow 的 coding agent。引擎层术语与架构细节与 **@uncaged/workflow** 上游文档一致,编写时可对照 \`CLAUDE.md\` 与 \`docs/architecture.md\`。
|
||||
|
||||
## 1. 项目结构(workspace / template / workflow instance)
|
||||
|
||||
| 层级 | 目录 / 产物 | 职责 |
|
||||
|------|----------------|------|
|
||||
| **Workspace** | 仓库根(\`package.json\` 含 \`workspaces: ["templates/*", "workflows"]\`) | Bun monorepo:统一管理本地模板包与 workflow 实例 |
|
||||
| **Template** | \`templates/<name>/\`(如 \`src/roles.ts\`、\`src/moderator.ts\`、\`src/index.ts\`) | 纯数据:**WorkflowDefinition**(各 **RoleDefinition** + **Moderator**),**不绑定**具体 Agent |
|
||||
| **Workflow instance** | \`workflows/\`(或单独包) | 把模板与运行时 **AgentFn** / **ExtractFn** 组合,产出可注册的 **单文件 ESM bundle**(\`run\` + \`descriptor\` 命名导出) |
|
||||
|
||||
Init 生成的骨架:\`templates/\` 下放可复用定义,\`workflows/\` 下放绑定与打包入口。
|
||||
|
||||
## 2. 核心概念
|
||||
|
||||
- **RoleMeta**:\`Record<string, Record<string, unknown>>\`,角色名 → 该角色结构化 meta 的形状约定。
|
||||
- **RoleDefinition<Meta>**:纯数据——\`description\`、\`systemPrompt\`、\`extractPrompt\`、\`schema\`(Zod v4)。不含执行逻辑。
|
||||
- **WorkflowDefinition<M extends RoleMeta>**:\`description\` + \`roles\`(各角色定义)+ **Moderator**。
|
||||
- **Moderator**:\`(ctx: ModeratorContext<M>) => (角色名) | END\`。同步、纯函数,只做路由。
|
||||
- **AgentFn**:\`(ctx: AgentContext) => Promise<string>\`,原始文本输出;从上下文读取当前角色的 \`systemPrompt\`。
|
||||
- **ExtractFn**:从上下文与 prompt 解析结构化数据(引擎与 Agent 都可使用)。
|
||||
|
||||
引擎循环简述:**Moderator** → 选角色 → **Agent** 产出文本 → **Extract** 写入 **meta** → 追加 step,重复直至 **END**。详见 \`docs/architecture.md\` 中的三阶段说明。
|
||||
|
||||
## 3. 开发流程
|
||||
|
||||
1. **定义 RoleMeta**:为每个角色约定 meta 的 TypeScript 类型(与 Zod schema 对齐)。
|
||||
2. **编写 RoleDefinition**:为每个角色写 Zod \`schema\`,补齐 \`systemPrompt\` / \`extractPrompt\` / \`description\`。
|
||||
3. **编写 Moderator**:根据 \`ctx.steps\` 与业务状态返回下一个角色名或 \`END\`。
|
||||
4. **组装 WorkflowDefinition**:在模板 \`index\` 中导出 definition(以及必要的角色 / moderator 导出)。
|
||||
5. **实例化**:在 workflow 包中使用 \`createWorkflow(def, binding, extract)\`(或项目约定的封装)绑定 **AgentFn** / **ExtractFn**。
|
||||
6. **构建**:打包为单个 **.esm.js** bundle,使用 **uncaged-workflow add** 注册。
|
||||
|
||||
## 4. 编码规范
|
||||
|
||||
与 **CLAUDE.md** 对齐,摘要如下:
|
||||
|
||||
- **Functional-first**:优先 \`function\` + \`type\`,避免面向对象业务模型。
|
||||
- **type 而非 interface**:类型别名一律用 \`type\`,不要使用 \`interface\`。
|
||||
- **显式可空**:不要用 \`?:\`;可空字段写成 \`T | null\`。
|
||||
- **function 而非 class**:不用 class(第三方库要求或 \`Error\` 子类除外)。
|
||||
- **Crockford Base32**:日志 tag、bundle hash、thread id 等标识约定(引擎侧);工作区内自定义日志若沿用引擎 logger,tag 为 8 字符 Crockford Base32,且每个调用点唯一。
|
||||
- **Named exports only**:不要使用 **default export**;workflow bundle 须 **export const run** 与 **export const descriptor**。
|
||||
- **No console.log**:库代码用结构化 logger;CLI 用户输出可按项目 Biome 规则例外标注。
|
||||
- **No dynamic import**:业务与 bundle 内禁止 \`import()\`;例外仅限「运行时路径由用户提供的 bundle 加载器」(引擎内部)。
|
||||
|
||||
## 5. Template 复用
|
||||
|
||||
- **已发布模板**:可通过 npm 依赖 \`@uncaged/workflow-template-*\` 等包,在 workflow 实例中 import 其 **WorkflowDefinition** 再绑定 Agent。
|
||||
- **本地模板**:放在本仓库 \`templates/<name>/\`,由 workspace 协议引用(如 \`"template-foo": "workspace:*"\` 或相对路径),便于同源修改与版本控制。
|
||||
|
||||
选择模板时保持 **definition 与 agent 绑定分离**:模板只描述「做什么、顺序如何」,实例决定「谁执行、如何抽取 meta」。
|
||||
|
||||
## 6. Build and Test
|
||||
|
||||
日常命令:
|
||||
|
||||
\`\`\`sh
|
||||
bun install
|
||||
bun run check # Biome:lint + format
|
||||
bun test
|
||||
bun build # 若包内配置了 build 脚本则用于产出 dist / bundle
|
||||
uncaged-workflow add <name> <path/to/bundle.esm.js>
|
||||
\`\`\`
|
||||
|
||||
提交前至少运行 **bun run check** 与 **bun test**;registry 与本地运行流参见 README 与 CLI 文档。
|
||||
|
||||
## 7. 常见陷阱
|
||||
|
||||
- **No dynamic import**:bundle 须静态可分析;动态 \`import()\` 会破坏哈希与加载约束。
|
||||
- **No default export**:引擎只接受命名导出 \`run\` / \`descriptor\`。
|
||||
- **No console.log**:避免在可被 Biome \`noConsole\` 规则覆盖的代码路径直接使用 console。
|
||||
- **Single-file ESM bundle**:交付物是单一 \`.esm.js\`;静态 import 仅限 Node 内置(见 architecture 文档中的 Bundle Contract)。
|
||||
|
||||
---
|
||||
|
||||
编写新 workflow 时,先对齐 **RoleMeta → RoleDefinition(Zod)→ Moderator → 绑定 → 单文件 bundle**,再对照本节规范自检。
|
||||
`;
|
||||
}
|
||||
|
||||
function readmeMd(workspaceName: string): string {
|
||||
return `# ${workspaceName}
|
||||
|
||||
Local workflow development workspace (Bun monorepo).
|
||||
|
||||
## Layout
|
||||
|
||||
- \`templates/\` — reusable workflow definition packages (roles + moderator), no agent binding
|
||||
- \`workflows/\` — workflow instances that bind templates to agents and export \`run\` + \`descriptor\`
|
||||
|
||||
## Commands
|
||||
|
||||
\`\`\`sh
|
||||
bun install
|
||||
bun run check # after you add scripts / Biome
|
||||
uncaged-workflow add <name> <bundle.esm.js>
|
||||
uncaged-workflow run <name>
|
||||
\`\`\`
|
||||
|
||||
Create this skeleton with:
|
||||
|
||||
\`\`\`sh
|
||||
uncaged-workflow init workspace ${workspaceName}
|
||||
\`\`\`
|
||||
`;
|
||||
}
|
||||
|
||||
export async function cmdInitWorkspace(
|
||||
parentDir: string,
|
||||
workspaceName: string,
|
||||
): Promise<Result<CmdInitWorkspaceSuccess, string>> {
|
||||
const validated = validateWorkspaceSegment(workspaceName);
|
||||
if (!validated.ok) {
|
||||
return validated;
|
||||
}
|
||||
|
||||
const rootPath = join(parentDir, workspaceName);
|
||||
if (await pathExists(rootPath)) {
|
||||
return err(`directory already exists: ${rootPath}`);
|
||||
}
|
||||
|
||||
await mkdir(rootPath, { recursive: false });
|
||||
await mkdir(join(rootPath, "templates"), { recursive: false });
|
||||
await mkdir(join(rootPath, "workflows"), { recursive: false });
|
||||
|
||||
await Promise.all([
|
||||
writeFile(join(rootPath, "package.json"), rootPackageJson(workspaceName), "utf8"),
|
||||
writeFile(join(rootPath, "biome.json"), biomeJson(), "utf8"),
|
||||
writeFile(join(rootPath, "tsconfig.json"), tsconfigJson(), "utf8"),
|
||||
writeFile(join(rootPath, "AGENTS.md"), agentsMd(), "utf8"),
|
||||
writeFile(join(rootPath, "README.md"), readmeMd(workspaceName), "utf8"),
|
||||
writeFile(join(rootPath, "templates", ".gitkeep"), "", "utf8"),
|
||||
writeFile(join(rootPath, "workflows", "package.json"), workflowsPackageJson(), "utf8"),
|
||||
]);
|
||||
|
||||
return ok({ rootPath });
|
||||
}
|
||||
|
||||
function hasTemplatesWorkspaceGlob(workspaces: unknown): boolean {
|
||||
return Array.isArray(workspaces) && workspaces.includes("templates/*");
|
||||
}
|
||||
|
||||
async function readPackageJsonWorkspaces(dir: string): Promise<unknown | null> {
|
||||
const pkgPath = join(dir, "package.json");
|
||||
if (!(await pathExists(pkgPath))) {
|
||||
return null;
|
||||
}
|
||||
let raw: string;
|
||||
try {
|
||||
raw = await readFile(pkgPath, "utf8");
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(raw) as unknown;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (typeof parsed !== "object" || parsed === null || !("workspaces" in parsed)) {
|
||||
return null;
|
||||
}
|
||||
return (parsed as { workspaces: unknown }).workspaces;
|
||||
}
|
||||
|
||||
/** Resolve uncaged-workflow workspace root (package.json with `templates/*` in `workspaces`). */
|
||||
async function findWorkflowWorkspaceRoot(startDir: string): Promise<Result<string, string>> {
|
||||
let dir = resolve(startDir);
|
||||
for (;;) {
|
||||
const workspaces = await readPackageJsonWorkspaces(dir);
|
||||
if (workspaces !== null && hasTemplatesWorkspaceGlob(workspaces)) {
|
||||
return ok(dir);
|
||||
}
|
||||
const parent = dirname(dir);
|
||||
if (parent === dir) {
|
||||
return err(
|
||||
'not inside a workflow workspace (no package.json with workspaces containing "templates/*")',
|
||||
);
|
||||
}
|
||||
dir = parent;
|
||||
}
|
||||
}
|
||||
|
||||
function templatePackageJson(templateName: string): string {
|
||||
return `${JSON.stringify(
|
||||
{
|
||||
name: `template-${templateName}`,
|
||||
version: "0.0.0",
|
||||
private: true,
|
||||
type: "module",
|
||||
dependencies: {
|
||||
"@uncaged/workflow": "^0.1.0",
|
||||
zod: "^4.0.0",
|
||||
},
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`;
|
||||
}
|
||||
|
||||
function templateTsconfigJson(): string {
|
||||
return `${JSON.stringify(
|
||||
{
|
||||
extends: "../../tsconfig.json",
|
||||
compilerOptions: {
|
||||
rootDir: "src",
|
||||
outDir: "dist",
|
||||
},
|
||||
include: ["src/**/*.ts"],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`;
|
||||
}
|
||||
|
||||
function templateRolesTs(): string {
|
||||
return `import type { RoleDefinition } from "@uncaged/workflow";
|
||||
import * as z from "zod/v4";
|
||||
|
||||
export const HELLO_TEMPLATE_DESCRIPTION =
|
||||
"Minimal starter template: one greeter role, then END.";
|
||||
|
||||
export type HelloTemplateMeta = {
|
||||
greeter: {
|
||||
message: string;
|
||||
};
|
||||
};
|
||||
|
||||
const greeterMetaSchema = z.object({
|
||||
message: z.string(),
|
||||
});
|
||||
|
||||
export const greeterRole: RoleDefinition<HelloTemplateMeta["greeter"]> = {
|
||||
description: "Says hello — replace with your first role.",
|
||||
systemPrompt: "You are a helpful assistant. Reply with one short friendly sentence.",
|
||||
extractPrompt: "Extract the assistant's greeting as message.",
|
||||
schema: greeterMetaSchema,
|
||||
extractRefs: null,
|
||||
};
|
||||
`;
|
||||
}
|
||||
|
||||
function templateModeratorTs(): string {
|
||||
return `import { END, type Moderator, type ModeratorContext } from "@uncaged/workflow";
|
||||
|
||||
import type { HelloTemplateMeta } from "./roles.js";
|
||||
|
||||
export const helloTemplateModerator: Moderator<HelloTemplateMeta> = (
|
||||
ctx: ModeratorContext<HelloTemplateMeta>,
|
||||
) => {
|
||||
if (ctx.steps.length === 0) {
|
||||
return "greeter";
|
||||
}
|
||||
return END;
|
||||
};
|
||||
`;
|
||||
}
|
||||
|
||||
function templateIndexTs(): string {
|
||||
return `import type { WorkflowDefinition } from "@uncaged/workflow";
|
||||
|
||||
import { helloTemplateModerator } from "./moderator.js";
|
||||
import {
|
||||
HELLO_TEMPLATE_DESCRIPTION,
|
||||
type HelloTemplateMeta,
|
||||
greeterRole,
|
||||
} from "./roles.js";
|
||||
|
||||
export {
|
||||
HELLO_TEMPLATE_DESCRIPTION,
|
||||
type HelloTemplateMeta,
|
||||
greeterRole,
|
||||
} from "./roles.js";
|
||||
export { helloTemplateModerator } from "./moderator.js";
|
||||
|
||||
export const helloTemplateWorkflowDefinition: WorkflowDefinition<HelloTemplateMeta> = {
|
||||
description: HELLO_TEMPLATE_DESCRIPTION,
|
||||
roles: {
|
||||
greeter: greeterRole,
|
||||
},
|
||||
moderator: helloTemplateModerator,
|
||||
};
|
||||
`;
|
||||
}
|
||||
|
||||
export async function cmdInitTemplate(
|
||||
startDir: string,
|
||||
templateName: string,
|
||||
): Promise<Result<CmdInitTemplateSuccess, string>> {
|
||||
const validated = validateWorkspaceSegment(templateName);
|
||||
if (!validated.ok) {
|
||||
return validated;
|
||||
}
|
||||
|
||||
const rootResult = await findWorkflowWorkspaceRoot(startDir);
|
||||
if (!rootResult.ok) {
|
||||
return rootResult;
|
||||
}
|
||||
|
||||
const workspaceRoot = rootResult.value;
|
||||
const templateDir = join(workspaceRoot, "templates", templateName);
|
||||
if (await pathExists(templateDir)) {
|
||||
return err(`template already exists: ${templateDir}`);
|
||||
}
|
||||
|
||||
await mkdir(join(templateDir, "src"), { recursive: true });
|
||||
|
||||
await Promise.all([
|
||||
writeFile(join(templateDir, "package.json"), templatePackageJson(templateName), "utf8"),
|
||||
writeFile(join(templateDir, "tsconfig.json"), templateTsconfigJson(), "utf8"),
|
||||
writeFile(join(templateDir, "src", "roles.ts"), templateRolesTs(), "utf8"),
|
||||
writeFile(join(templateDir, "src", "moderator.ts"), templateModeratorTs(), "utf8"),
|
||||
writeFile(join(templateDir, "src", "index.ts"), templateIndexTs(), "utf8"),
|
||||
]);
|
||||
|
||||
return ok({ templatePath: templateDir });
|
||||
}
|
||||
@@ -44,6 +44,7 @@ export async function cmdRollback(
|
||||
}
|
||||
|
||||
const nextRegistry = {
|
||||
config: reg.value.config,
|
||||
workflows: { ...reg.value.workflows, [name]: rolled.value },
|
||||
};
|
||||
const written = await writeWorkflowRegistry(storageRoot, nextRegistry);
|
||||
|
||||
@@ -46,7 +46,7 @@ export async function cmdRun(
|
||||
threadId,
|
||||
workflowName: name,
|
||||
prompt,
|
||||
options: { maxRounds },
|
||||
options: { maxRounds, depth: 0 },
|
||||
},
|
||||
{ awaitResponseLine: false },
|
||||
);
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { rm, unlink } from "node:fs/promises";
|
||||
import { unlink } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
|
||||
import { err, ok, type Result } from "@uncaged/workflow";
|
||||
import { err, garbageCollectCas, ok, type Result } from "@uncaged/workflow";
|
||||
|
||||
import { readTextFileIfExists } from "./fs-utils.js";
|
||||
import { resolveThreadDataPath } from "./thread-scan.js";
|
||||
@@ -33,12 +33,12 @@ export async function cmdThreadRemove(
|
||||
const dir = dirname(dataPath);
|
||||
const infoPath = join(dir, `${threadId}.info.jsonl`);
|
||||
const runningPath = join(dir, `${threadId}.running`);
|
||||
const casPath = join(dir, `${threadId}.cas`);
|
||||
|
||||
await unlink(dataPath);
|
||||
await unlink(infoPath).catch(() => {});
|
||||
await unlink(runningPath).catch(() => {});
|
||||
await rm(casPath, { recursive: true, force: true });
|
||||
|
||||
await garbageCollectCas(storageRoot);
|
||||
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
@@ -54,7 +54,7 @@ export function createCursorAgent(config: CursorAgentConfig): AgentFn {
|
||||
"From the thread context, determine the absolute filesystem path where the project/repository is located.",
|
||||
extractCtx,
|
||||
);
|
||||
const fullPrompt = buildAgentPrompt(ctx);
|
||||
const fullPrompt = await buildAgentPrompt(ctx);
|
||||
const args = [
|
||||
"-p",
|
||||
fullPrompt,
|
||||
|
||||
@@ -35,7 +35,7 @@ export function createHermesAgent(config: HermesAgentConfig): AgentFn {
|
||||
const timeoutMs = config.timeout;
|
||||
|
||||
return async (ctx) => {
|
||||
const fullPrompt = buildAgentPrompt(ctx);
|
||||
const fullPrompt = await buildAgentPrompt(ctx);
|
||||
const args = [
|
||||
"chat",
|
||||
"-q",
|
||||
|
||||
@@ -1,8 +1,14 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { START, type ThreadContext } from "@uncaged/workflow";
|
||||
import { mkdtempSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { createCasStore, START, type ThreadContext } from "@uncaged/workflow";
|
||||
|
||||
import { createLlmAdapter } from "../src/create-llm-adapter.js";
|
||||
|
||||
const casDir = mkdtempSync(join(tmpdir(), "wf-llm-adapter-cas-"));
|
||||
const testCas = createCasStore(casDir);
|
||||
|
||||
function makeCtx(userContent: string): ThreadContext {
|
||||
return {
|
||||
start: {
|
||||
@@ -11,9 +17,11 @@ function makeCtx(userContent: string): ThreadContext {
|
||||
meta: { maxRounds: 10 },
|
||||
timestamp: 1,
|
||||
},
|
||||
depth: 0,
|
||||
steps: [],
|
||||
threadId: "01TEST000000000000000000TR",
|
||||
currentRole: { name: "planner", systemPrompt: "system instructions" },
|
||||
cas: testCas,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -10,9 +10,24 @@ export const coderMetaSchema = z.object({
|
||||
export type CoderMeta = z.infer<typeof coderMetaSchema>;
|
||||
|
||||
const CODER_SYSTEM = `You are a **coder**. Read the thread for the plan and work on the NEXT incomplete phase only.
|
||||
|
||||
## Finding the current thread ID
|
||||
|
||||
The thread ID is a 26-character Crockford Base32 string (e.g. \`06F03H5V6JTMDST6P3TVH42RWM\`). It appears in the first message of this conversation. If you are unsure, run:
|
||||
|
||||
uncaged-workflow threads
|
||||
|
||||
and use the ID of the active thread.
|
||||
|
||||
## Reading phase details
|
||||
|
||||
Each planner phase is identified by a content-hash and a title. To read a phase's full details (name, description, acceptance criteria), run:
|
||||
|
||||
uncaged-workflow cas get <thread-id> <hash>
|
||||
uncaged-workflow cas get <THREAD_ID> <HASH>
|
||||
|
||||
Replace \`<THREAD_ID>\` with the actual thread ID and \`<HASH>\` with the phase hash from the plan.
|
||||
|
||||
## Completing a phase
|
||||
|
||||
Report which phase you completed using the phase **hash** (not the title). If you legitimately finish every remaining phase in this single turn, set completedPhase to the **last** phase hash in the plan (the workflow treats that as full completion). List the files you changed and summarize what you did.`;
|
||||
|
||||
@@ -23,4 +38,6 @@ export const coderRole: RoleDefinition<CoderMeta> = {
|
||||
extractPrompt:
|
||||
"Extract completedPhase: the planner phase hash finished this round (exact hash string from the plan). If multiple phases were finished in one round, use the last finished phase hash. Extract filesChanged and a summary of the work.",
|
||||
schema: coderMetaSchema,
|
||||
extractRefs: (meta) => [meta.completedPhase],
|
||||
extractMode: "single",
|
||||
};
|
||||
|
||||
@@ -31,4 +31,6 @@ export const committerRole: RoleDefinition<CommitterMeta> = {
|
||||
extractPrompt:
|
||||
"Extract the commit result: committed (with branch and SHA), recoverable failure, or unrecoverable failure. Include error details and log references if applicable.",
|
||||
schema: committerMetaSchema,
|
||||
extractRefs: null,
|
||||
extractMode: "single",
|
||||
};
|
||||
|
||||
@@ -14,16 +14,34 @@ export type PlannerMeta = z.infer<typeof plannerMetaSchema>;
|
||||
|
||||
const PLANNER_SYSTEM = `You are a **planner** for a software task. Break the work into **sequential phases** the coder will execute one at a time.
|
||||
|
||||
For each phase, decide on a name, detailed description, and acceptance criteria. Then store the full detail text in CAS so the coder can retrieve it later:
|
||||
## Finding the current thread ID
|
||||
|
||||
uncaged-workflow cas put <thread-id> "# <name>\n\nDescription: <description>\n\nAcceptance: <acceptance>"
|
||||
The thread ID is a 26-character Crockford Base32 string (e.g. \`06F03H5V6JTMDST6P3TVH42RWM\`). It appears in the first message of this conversation. If you are unsure, run:
|
||||
|
||||
The command prints a content-hash to stdout. Use that hash as the phase identifier.
|
||||
uncaged-workflow threads
|
||||
|
||||
Your final structured output must contain compact phases only:
|
||||
and use the ID of the active thread.
|
||||
|
||||
## Storing phase details — MANDATORY
|
||||
|
||||
For each phase you MUST store its full detail text in CAS using this exact CLI command:
|
||||
|
||||
uncaged-workflow cas put <THREAD_ID> '# <name>
|
||||
|
||||
Description: <description>
|
||||
|
||||
Acceptance: <acceptance>'
|
||||
|
||||
Replace \`<THREAD_ID>\` with the actual thread ID you found above. The command prints a content-hash to stdout — use that hash as the phase identifier.
|
||||
|
||||
**Do NOT store phase details in any other way** (no temp files, no invented paths). The CLI command is the only supported storage mechanism.
|
||||
|
||||
## Output format
|
||||
|
||||
After storing all phases via the CLI, output compact JSON only:
|
||||
{ "phases": [{ "hash": "<hash-from-cas-put>", "title": "<one-line-summary>" }] }
|
||||
|
||||
The current thread ID is provided in the thread context. Order phases so earlier steps unblock later ones. Cover root cause, edge cases, and verification across the phases.`;
|
||||
Order phases so earlier steps unblock later ones. Cover root cause, edge cases, and verification across the phases.`;
|
||||
|
||||
export const plannerRole: RoleDefinition<PlannerMeta> = {
|
||||
description: "Breaks the task into sequential phases for the coder.",
|
||||
@@ -31,4 +49,6 @@ export const plannerRole: RoleDefinition<PlannerMeta> = {
|
||||
extractPrompt:
|
||||
"Extract the implementation phases from the agent's output. Each phase has a hash (the CAS content-hash returned by the cas put command) and a title (one-line summary).",
|
||||
schema: plannerMetaSchema,
|
||||
extractRefs: (meta) => meta.phases.map((p) => p.hash),
|
||||
extractMode: "single",
|
||||
};
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"name": "@uncaged/workflow-role-preparer",
|
||||
"version": "0.1.0",
|
||||
"type": "module",
|
||||
"main": "src/index.ts",
|
||||
"types": "src/index.ts",
|
||||
"scripts": {
|
||||
"build": "echo 'TODO'",
|
||||
"test": "echo no tests"
|
||||
},
|
||||
"dependencies": {
|
||||
"@uncaged/workflow": "workspace:*",
|
||||
"zod": "^4.0.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
export {
|
||||
type PreparerMeta,
|
||||
preparerMetaSchema,
|
||||
preparerRole,
|
||||
} from "./preparer.js";
|
||||
@@ -0,0 +1,52 @@
|
||||
import type { RoleDefinition } from "@uncaged/workflow";
|
||||
import * as z from "zod/v4";
|
||||
|
||||
const toolchainSchema = z.object({
|
||||
packageManager: z.union([z.string(), z.null()]),
|
||||
testCommand: z.union([z.string(), z.null()]),
|
||||
lintCommand: z.union([z.string(), z.null()]),
|
||||
buildCommand: z.union([z.string(), z.null()]),
|
||||
});
|
||||
|
||||
export const preparerMetaSchema = z.object({
|
||||
repoPath: z.string(),
|
||||
defaultBranch: z.string(),
|
||||
conventions: z.union([z.string(), z.null()]),
|
||||
toolchain: toolchainSchema,
|
||||
});
|
||||
|
||||
export type PreparerMeta = z.infer<typeof preparerMetaSchema>;
|
||||
|
||||
const PREPARER_SYSTEM = `You are a **preparer** for a software task. Your job is to locate (or clone) the target repository locally, ensure it is up to date, and gather project context before work begins.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
1. Parse the issue/task prompt to identify the target repository (URL, org/repo, or name).
|
||||
2. Search for an existing local clone in these locations (in order):
|
||||
- ~/Code/<repo-name>/
|
||||
- ~/repos/<repo-name>/
|
||||
- ~/Code/<org>/<repo-name>/
|
||||
- ~/repos/<org>/<repo-name>/
|
||||
3. If not found locally, \`git clone\` it into ~/repos/<repo-name>/.
|
||||
4. \`git checkout main && git pull\` (or the default branch) to ensure latest.
|
||||
5. Read project conventions: \`CLAUDE.md\`, \`CONTRIBUTING.md\`, \`.cursor/rules/*.mdc\`, \`CONVENTIONS.md\`.
|
||||
6. Detect toolchain: package manager, test runner, linter, build system.
|
||||
|
||||
## Output
|
||||
|
||||
Report your findings as structured data:
|
||||
- **repoPath**: absolute path to the local repo
|
||||
- **defaultBranch**: the default branch name (e.g. "main")
|
||||
- **conventions**: a summary of project conventions found, or null if none
|
||||
- **toolchain**: detected commands for packageManager, testCommand, lintCommand, buildCommand (null if not detected)`;
|
||||
|
||||
export const preparerRole: RoleDefinition<PreparerMeta> = {
|
||||
description:
|
||||
"Locates or clones the target repository, ensures it is up to date, and gathers project context (conventions, toolchain).",
|
||||
systemPrompt: PREPARER_SYSTEM,
|
||||
extractPrompt:
|
||||
"Extract repoPath (absolute path), defaultBranch, conventions (summary string or null), and toolchain (packageManager, testCommand, lintCommand, buildCommand — each string or null).",
|
||||
schema: preparerMetaSchema,
|
||||
extractRefs: null,
|
||||
extractMode: "single",
|
||||
};
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"extends": "../../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "dist"
|
||||
},
|
||||
"include": ["src"]
|
||||
}
|
||||
@@ -21,4 +21,6 @@ export const reviewerRole: RoleDefinition<ReviewerMeta> = {
|
||||
extractPrompt:
|
||||
"Extract the review verdict: approved or rejected. If rejected, list the blocking issues.",
|
||||
schema: reviewerMetaSchema,
|
||||
extractRefs: null,
|
||||
extractMode: "single",
|
||||
};
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
import { afterEach, describe, expect, test } from "bun:test";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import {
|
||||
createCasStore,
|
||||
createExtract,
|
||||
END,
|
||||
type ModeratorContext,
|
||||
@@ -10,6 +14,7 @@ import {
|
||||
|
||||
import type { CoderMeta } from "@uncaged/workflow-role-coder";
|
||||
import type { PlannerMeta } from "@uncaged/workflow-role-planner";
|
||||
import type { PreparerMeta } from "@uncaged/workflow-role-preparer";
|
||||
|
||||
import { buildSolveIssueDescriptor } from "../src/descriptor.js";
|
||||
import { createSolveIssueRun, solveIssueModerator } from "../src/index.js";
|
||||
@@ -103,16 +108,38 @@ function makeCtx(
|
||||
): ModeratorContext<SolveIssueMeta> {
|
||||
return {
|
||||
threadId: "01TEST000000000000000000TR",
|
||||
depth: 0,
|
||||
start: makeStart(maxRounds),
|
||||
steps,
|
||||
};
|
||||
}
|
||||
|
||||
function preparerStep(): RoleStep<SolveIssueMeta> {
|
||||
return {
|
||||
role: "preparer",
|
||||
contentHash: "STUBHASHPREPARER01",
|
||||
meta: {
|
||||
repoPath: "/home/user/repos/test",
|
||||
defaultBranch: "main",
|
||||
conventions: null,
|
||||
toolchain: {
|
||||
packageManager: "bun",
|
||||
testCommand: "bun test",
|
||||
lintCommand: null,
|
||||
buildCommand: "bun run build",
|
||||
},
|
||||
},
|
||||
refs: [],
|
||||
timestamp: 0,
|
||||
};
|
||||
}
|
||||
|
||||
function plannerStep(phases: PlannerMeta["phases"] = DEFAULT_PHASES): RoleStep<SolveIssueMeta> {
|
||||
return {
|
||||
role: "planner",
|
||||
content: "plan",
|
||||
contentHash: "STUBHASHPLANNER001",
|
||||
meta: { phases },
|
||||
refs: phases.map((p) => p.hash),
|
||||
timestamp: 1,
|
||||
};
|
||||
}
|
||||
@@ -120,8 +147,9 @@ function plannerStep(phases: PlannerMeta["phases"] = DEFAULT_PHASES): RoleStep<S
|
||||
function coderStep(completedPhase = "4KNMR2PX"): RoleStep<SolveIssueMeta> {
|
||||
return {
|
||||
role: "coder",
|
||||
content: "code",
|
||||
contentHash: "STUBHASHCODER00001",
|
||||
meta: { completedPhase, filesChanged: ["a.ts"], summary: "fixed" },
|
||||
refs: [completedPhase],
|
||||
timestamp: 2,
|
||||
};
|
||||
}
|
||||
@@ -129,10 +157,11 @@ function coderStep(completedPhase = "4KNMR2PX"): RoleStep<SolveIssueMeta> {
|
||||
function reviewerStep(approved: boolean): RoleStep<SolveIssueMeta> {
|
||||
return {
|
||||
role: "reviewer",
|
||||
content: "rev",
|
||||
contentHash: "STUBHASHREVIEWER01",
|
||||
meta: approved
|
||||
? { status: "approved" as const }
|
||||
: { status: "rejected" as const, issues: ["needs fix"] },
|
||||
refs: [],
|
||||
timestamp: 3,
|
||||
};
|
||||
}
|
||||
@@ -140,8 +169,9 @@ function reviewerStep(approved: boolean): RoleStep<SolveIssueMeta> {
|
||||
function committerStep(): RoleStep<SolveIssueMeta> {
|
||||
return {
|
||||
role: "committer",
|
||||
content: "commit",
|
||||
contentHash: "STUBHASHCOMMITTER1",
|
||||
meta: { status: "committed", branch: "feat/issue-1", commitSha: "abc1234" },
|
||||
refs: [],
|
||||
timestamp: 4,
|
||||
};
|
||||
}
|
||||
@@ -153,16 +183,27 @@ const stubExtract = createExtract({
|
||||
});
|
||||
|
||||
describe("solveIssueModerator", () => {
|
||||
test("routes planner → coder → reviewer → committer → END", () => {
|
||||
expect(solveIssueModerator(makeCtx(20, []))).toBe("planner");
|
||||
expect(solveIssueModerator(makeCtx(20, [plannerStep()]))).toBe("coder");
|
||||
expect(solveIssueModerator(makeCtx(20, [plannerStep(), coderStep()]))).toBe("reviewer");
|
||||
expect(solveIssueModerator(makeCtx(20, [plannerStep(), coderStep(), reviewerStep(true)]))).toBe(
|
||||
"committer",
|
||||
test("routes preparer → planner → coder → reviewer → committer → END", () => {
|
||||
expect(solveIssueModerator(makeCtx(20, []))).toBe("preparer");
|
||||
expect(solveIssueModerator(makeCtx(20, [preparerStep()]))).toBe("planner");
|
||||
expect(solveIssueModerator(makeCtx(20, [preparerStep(), plannerStep()]))).toBe("coder");
|
||||
expect(solveIssueModerator(makeCtx(20, [preparerStep(), plannerStep(), coderStep()]))).toBe(
|
||||
"reviewer",
|
||||
);
|
||||
expect(
|
||||
solveIssueModerator(
|
||||
makeCtx(20, [plannerStep(), coderStep(), reviewerStep(true), committerStep()]),
|
||||
makeCtx(20, [preparerStep(), plannerStep(), coderStep(), reviewerStep(true)]),
|
||||
),
|
||||
).toBe("committer");
|
||||
expect(
|
||||
solveIssueModerator(
|
||||
makeCtx(20, [
|
||||
preparerStep(),
|
||||
plannerStep(),
|
||||
coderStep(),
|
||||
reviewerStep(true),
|
||||
committerStep(),
|
||||
]),
|
||||
),
|
||||
).toBe(END);
|
||||
});
|
||||
@@ -244,31 +285,71 @@ describe("solveIssueModerator", () => {
|
||||
|
||||
describe("createSolveIssueRun", () => {
|
||||
let restoreFetch: (() => void) | null = null;
|
||||
let casDir: string | undefined;
|
||||
|
||||
afterEach(() => {
|
||||
afterEach(async () => {
|
||||
restoreFetch?.();
|
||||
restoreFetch = null;
|
||||
if (casDir !== undefined) {
|
||||
await rm(casDir, { recursive: true, force: true }).catch(() => {});
|
||||
casDir = undefined;
|
||||
}
|
||||
});
|
||||
|
||||
test("structured extraction yields planner meta from mocked chat completions", async () => {
|
||||
restoreFetch = installMockChatCompletions([EXPECT_PLANNER_META]);
|
||||
test("structured extraction yields preparer then planner meta from mocked chat completions", async () => {
|
||||
const EXPECT_PREPARER_META: PreparerMeta = {
|
||||
repoPath: "/home/user/repos/test",
|
||||
defaultBranch: "main",
|
||||
conventions: null,
|
||||
toolchain: {
|
||||
packageManager: "bun",
|
||||
testCommand: "bun test",
|
||||
lintCommand: null,
|
||||
buildCommand: "bun run build",
|
||||
},
|
||||
};
|
||||
restoreFetch = installMockChatCompletions([EXPECT_PREPARER_META, EXPECT_PLANNER_META]);
|
||||
|
||||
const run = createSolveIssueRun({ agent: async () => "" }, stubExtract);
|
||||
casDir = await mkdtemp(join(tmpdir(), "solve-issue-cas-"));
|
||||
const cas = createCasStore(casDir);
|
||||
|
||||
const run = createSolveIssueRun({ agent: async () => "" }, stubExtract, null);
|
||||
const gen = run(
|
||||
{ prompt: "task", steps: [] },
|
||||
{ threadId: "01TEST000000000000000000TR", maxRounds: 20 },
|
||||
{ threadId: "01TEST000000000000000000TR", maxRounds: 20, depth: 0, cas },
|
||||
);
|
||||
const first = await gen.next();
|
||||
expect(first.done).toBe(false);
|
||||
if (first.done) {
|
||||
throw new Error("expected yield");
|
||||
}
|
||||
expect(first.value.role).toBe("planner");
|
||||
expect(first.value.meta).toEqual(EXPECT_PLANNER_META);
|
||||
expect(first.value.role).toBe("preparer");
|
||||
expect(first.value.meta).toEqual(EXPECT_PREPARER_META);
|
||||
|
||||
const second = await gen.next();
|
||||
expect(second.done).toBe(false);
|
||||
if (second.done) {
|
||||
throw new Error("expected yield");
|
||||
}
|
||||
expect(second.value.role).toBe("planner");
|
||||
expect(second.value.meta).toEqual(EXPECT_PLANNER_META);
|
||||
});
|
||||
|
||||
test("per-role agent overrides default", async () => {
|
||||
restoreFetch = installMockChatCompletions([EXPECT_PLANNER_META, EXPECT_CODER_META]);
|
||||
const PREPARER_META: PreparerMeta = {
|
||||
repoPath: "/tmp/r",
|
||||
defaultBranch: "main",
|
||||
conventions: null,
|
||||
toolchain: { packageManager: null, testCommand: null, lintCommand: null, buildCommand: null },
|
||||
};
|
||||
restoreFetch = installMockChatCompletions([
|
||||
PREPARER_META,
|
||||
EXPECT_PLANNER_META,
|
||||
EXPECT_CODER_META,
|
||||
]);
|
||||
|
||||
casDir = await mkdtemp(join(tmpdir(), "solve-issue-cas-"));
|
||||
const cas = createCasStore(casDir);
|
||||
|
||||
const calls: string[] = [];
|
||||
const run = createSolveIssueRun(
|
||||
@@ -278,6 +359,10 @@ describe("createSolveIssueRun", () => {
|
||||
return "";
|
||||
},
|
||||
overrides: {
|
||||
preparer: async () => {
|
||||
calls.push("preparer");
|
||||
return "";
|
||||
},
|
||||
planner: async () => {
|
||||
calls.push("planner");
|
||||
return "";
|
||||
@@ -289,12 +374,17 @@ describe("createSolveIssueRun", () => {
|
||||
},
|
||||
},
|
||||
stubExtract,
|
||||
null,
|
||||
);
|
||||
const gen = run(
|
||||
{ prompt: "task", steps: [] },
|
||||
{ threadId: "01TEST000000000000000000TR", maxRounds: 20 },
|
||||
{ threadId: "01TEST000000000000000000TR", maxRounds: 20, depth: 0, cas },
|
||||
);
|
||||
await gen.next();
|
||||
expect(calls).toEqual(["preparer"]);
|
||||
|
||||
calls.length = 0;
|
||||
await gen.next();
|
||||
expect(calls).toEqual(["planner"]);
|
||||
|
||||
calls.length = 0;
|
||||
@@ -315,9 +405,10 @@ describe("buildSolveIssueDescriptor", () => {
|
||||
"coder",
|
||||
"committer",
|
||||
"planner",
|
||||
"preparer",
|
||||
"reviewer",
|
||||
]);
|
||||
for (const key of ["planner", "coder", "reviewer", "committer"] as const) {
|
||||
for (const key of ["preparer", "planner", "coder", "reviewer", "committer"] as const) {
|
||||
const role = validated.value.roles[key];
|
||||
expect(role).toBeDefined();
|
||||
expect(typeof role.schema).toBe("object");
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
"@uncaged/workflow-role-committer": "workspace:*",
|
||||
"@uncaged/workflow-role-coder": "workspace:*",
|
||||
"@uncaged/workflow-role-planner": "workspace:*",
|
||||
"@uncaged/workflow-role-preparer": "workspace:*",
|
||||
"@uncaged/workflow-role-reviewer": "workspace:*"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ import {
|
||||
type AgentBinding,
|
||||
createWorkflow,
|
||||
type ExtractFn,
|
||||
type LlmProvider,
|
||||
type WorkflowDefinition,
|
||||
type WorkflowFn,
|
||||
} from "@uncaged/workflow";
|
||||
@@ -25,6 +26,11 @@ export {
|
||||
plannerMetaSchema,
|
||||
plannerRole,
|
||||
} from "@uncaged/workflow-role-planner";
|
||||
export {
|
||||
type PreparerMeta,
|
||||
preparerMetaSchema,
|
||||
preparerRole,
|
||||
} from "@uncaged/workflow-role-preparer";
|
||||
export {
|
||||
type ReviewerMeta,
|
||||
reviewerMetaSchema,
|
||||
@@ -45,6 +51,10 @@ export const solveIssueWorkflowDefinition: WorkflowDefinition<SolveIssueMeta> =
|
||||
moderator: solveIssueModerator,
|
||||
};
|
||||
|
||||
export function createSolveIssueRun(binding: AgentBinding, extract: ExtractFn): WorkflowFn {
|
||||
return createWorkflow(solveIssueWorkflowDefinition, binding, extract);
|
||||
export function createSolveIssueRun(
|
||||
binding: AgentBinding,
|
||||
extract: ExtractFn,
|
||||
llmProvider: LlmProvider | null,
|
||||
): WorkflowFn {
|
||||
return createWorkflow(solveIssueWorkflowDefinition, binding, extract, llmProvider);
|
||||
}
|
||||
|
||||
@@ -48,11 +48,15 @@ export const solveIssueModerator: Moderator<SolveIssueMeta> = (ctx) => {
|
||||
const maxRounds = ctx.start.meta.maxRounds;
|
||||
|
||||
if (ctx.steps.length === 0) {
|
||||
return "planner";
|
||||
return "preparer";
|
||||
}
|
||||
|
||||
const last = ctx.steps[ctx.steps.length - 1];
|
||||
|
||||
if (last.role === "preparer") {
|
||||
return "planner";
|
||||
}
|
||||
|
||||
if (last.role === "planner") {
|
||||
return "coder";
|
||||
}
|
||||
|
||||
@@ -2,12 +2,14 @@ import type { RoleDefinition } from "@uncaged/workflow";
|
||||
import { type CoderMeta, coderRole } from "@uncaged/workflow-role-coder";
|
||||
import { type CommitterMeta, committerRole } from "@uncaged/workflow-role-committer";
|
||||
import { type PlannerMeta, plannerRole } from "@uncaged/workflow-role-planner";
|
||||
import { type PreparerMeta, preparerRole } from "@uncaged/workflow-role-preparer";
|
||||
import { type ReviewerMeta, reviewerRole } from "@uncaged/workflow-role-reviewer";
|
||||
|
||||
export const SOLVE_ISSUE_WORKFLOW_DESCRIPTION =
|
||||
"Phased plan, incremental implementation per phase, review, and commit to resolve an issue end-to-end (planner → coder [repeat per phase] → reviewer → committer).";
|
||||
"Prepare repo context, plan phases, implement incrementally, review, and commit to resolve an issue end-to-end (preparer → planner → coder [repeat per phase] → reviewer → committer).";
|
||||
|
||||
export type SolveIssueMeta = {
|
||||
preparer: PreparerMeta;
|
||||
planner: PlannerMeta;
|
||||
coder: CoderMeta;
|
||||
reviewer: ReviewerMeta;
|
||||
@@ -19,6 +21,7 @@ export type SolveIssueRoles = {
|
||||
};
|
||||
|
||||
export const solveIssueRoles: SolveIssueRoles = {
|
||||
preparer: preparerRole,
|
||||
planner: plannerRole,
|
||||
coder: coderRole,
|
||||
reviewer: reviewerRole,
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { START, type ThreadContext } from "@uncaged/workflow";
|
||||
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { createCasStore, putContentMerkleNode, START, type ThreadContext } from "@uncaged/workflow";
|
||||
|
||||
import { buildAgentPrompt } from "../src/index.js";
|
||||
|
||||
@@ -13,35 +16,53 @@ function startTask(content: string): ThreadContext["start"] {
|
||||
}
|
||||
|
||||
describe("buildAgentPrompt", () => {
|
||||
test("includes system prompt and full task; omits tools when there are no steps", () => {
|
||||
let casRoot: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
casRoot = await mkdtemp(join(tmpdir(), "wf-build-prompt-cas-"));
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await rm(casRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("includes system prompt and full task; omits tools when there are no steps", async () => {
|
||||
const cas = createCasStore(casRoot);
|
||||
const ctx: ThreadContext = {
|
||||
start: startTask("fix the bug"),
|
||||
depth: 0,
|
||||
steps: [],
|
||||
threadId: "01TEST000000000000000000TR",
|
||||
currentRole: { name: START, systemPrompt: "You are an agent." },
|
||||
cas,
|
||||
};
|
||||
const text = buildAgentPrompt(ctx);
|
||||
const text = await buildAgentPrompt(ctx);
|
||||
expect(text).toContain("You are an agent.");
|
||||
expect(text).toContain("## Task");
|
||||
expect(text).toContain("fix the bug");
|
||||
expect(text).not.toContain("## Tools");
|
||||
});
|
||||
|
||||
test("single step shows full content and meta, and includes tools", () => {
|
||||
test("single step shows full content and meta, and includes tools", async () => {
|
||||
const cas = createCasStore(casRoot);
|
||||
const onlyHash = await putContentMerkleNode(cas, "only step full body");
|
||||
const ctx: ThreadContext = {
|
||||
start: startTask("user task"),
|
||||
depth: 0,
|
||||
threadId: "01TEST000000000000000000TR",
|
||||
currentRole: { name: "coder", systemPrompt: "Be helpful." },
|
||||
cas,
|
||||
steps: [
|
||||
{
|
||||
role: "coder",
|
||||
content: "only step full body",
|
||||
contentHash: onlyHash,
|
||||
meta: { files: ["a.ts"] },
|
||||
refs: [onlyHash],
|
||||
timestamp: 2,
|
||||
},
|
||||
],
|
||||
};
|
||||
const text = buildAgentPrompt(ctx);
|
||||
const text = await buildAgentPrompt(ctx);
|
||||
expect(text).toContain("## Task");
|
||||
expect(text).toContain("user task");
|
||||
expect(text).toContain("## Step: coder");
|
||||
@@ -51,27 +72,34 @@ describe("buildAgentPrompt", () => {
|
||||
expect(text).toContain("uncaged-workflow thread 01TEST000000000000000000TR");
|
||||
});
|
||||
|
||||
test("two or more steps: previous steps are meta-only; latest step is full", () => {
|
||||
test("two or more steps: previous steps are meta-only; latest step is full", async () => {
|
||||
const cas = createCasStore(casRoot);
|
||||
const plannerHash = await putContentMerkleNode(cas, "PLANNER_SECRET_FULL_TEXT");
|
||||
const coderHash = await putContentMerkleNode(cas, "last step full content");
|
||||
const ctx: ThreadContext = {
|
||||
start: startTask("first message full: task content here"),
|
||||
depth: 0,
|
||||
threadId: "01TEST000000000000000000TR",
|
||||
currentRole: { name: "coder", systemPrompt: "System." },
|
||||
cas,
|
||||
steps: [
|
||||
{
|
||||
role: "planner",
|
||||
content: "PLANNER_SECRET_FULL_TEXT",
|
||||
contentHash: plannerHash,
|
||||
meta: { plan: "short" },
|
||||
refs: [plannerHash],
|
||||
timestamp: 2,
|
||||
},
|
||||
{
|
||||
role: "coder",
|
||||
content: "last step full content",
|
||||
contentHash: coderHash,
|
||||
meta: { done: true },
|
||||
refs: [coderHash],
|
||||
timestamp: 3,
|
||||
},
|
||||
],
|
||||
};
|
||||
const text = buildAgentPrompt(ctx);
|
||||
const text = await buildAgentPrompt(ctx);
|
||||
expect(text).toContain("first message full: task content here");
|
||||
expect(text).toContain("## Previous Steps");
|
||||
expect(text).toContain("### Step 1: planner");
|
||||
@@ -84,33 +112,42 @@ describe("buildAgentPrompt", () => {
|
||||
expect(text).toContain("uncaged-workflow thread 01TEST000000000000000000TR");
|
||||
});
|
||||
|
||||
test("middle steps show meta summary only, not full content", () => {
|
||||
test("middle steps show meta summary only, not full content", async () => {
|
||||
const cas = createCasStore(casRoot);
|
||||
const ha = await putContentMerkleNode(cas, "HIDDEN_A");
|
||||
const hb = await putContentMerkleNode(cas, "HIDDEN_B_MIDDLE");
|
||||
const hc = await putContentMerkleNode(cas, "VISIBLE_LAST");
|
||||
const ctx: ThreadContext = {
|
||||
start: startTask("start"),
|
||||
depth: 0,
|
||||
threadId: "01TEST000000000000000000TR",
|
||||
currentRole: { name: "c", systemPrompt: "S" },
|
||||
cas,
|
||||
steps: [
|
||||
{
|
||||
role: "a",
|
||||
content: "HIDDEN_A",
|
||||
contentHash: ha,
|
||||
meta: { n: 1 },
|
||||
refs: [ha],
|
||||
timestamp: 2,
|
||||
},
|
||||
{
|
||||
role: "b",
|
||||
content: "HIDDEN_B_MIDDLE",
|
||||
contentHash: hb,
|
||||
meta: { n: 2 },
|
||||
refs: [hb],
|
||||
timestamp: 3,
|
||||
},
|
||||
{
|
||||
role: "c",
|
||||
content: "VISIBLE_LAST",
|
||||
contentHash: hc,
|
||||
meta: { n: 3 },
|
||||
refs: [hc],
|
||||
timestamp: 4,
|
||||
},
|
||||
],
|
||||
};
|
||||
const text = buildAgentPrompt(ctx);
|
||||
const text = await buildAgentPrompt(ctx);
|
||||
expect(text).not.toContain("HIDDEN_A");
|
||||
expect(text).not.toContain("HIDDEN_B_MIDDLE");
|
||||
expect(text).toContain('Summary: {"n":1}');
|
||||
|
||||
@@ -1,7 +1,16 @@
|
||||
import type { AgentContext } from "@uncaged/workflow";
|
||||
import { getContentMerklePayload } from "@uncaged/workflow";
|
||||
|
||||
async function resolveStepText(ctx: AgentContext, contentHash: string): Promise<string> {
|
||||
const text = await getContentMerklePayload(ctx.cas, contentHash);
|
||||
if (text === null) {
|
||||
throw new Error(`buildAgentPrompt: missing CAS blob for ${contentHash}`);
|
||||
}
|
||||
return text;
|
||||
}
|
||||
|
||||
/** Builds the full agent prompt: system instructions plus summarized thread history. */
|
||||
export function buildAgentPrompt(ctx: AgentContext): string {
|
||||
export async function buildAgentPrompt(ctx: AgentContext): Promise<string> {
|
||||
const lines: string[] = [];
|
||||
lines.push(ctx.currentRole.systemPrompt);
|
||||
lines.push("");
|
||||
@@ -15,10 +24,11 @@ export function buildAgentPrompt(ctx: AgentContext): string {
|
||||
|
||||
if (steps.length === 1) {
|
||||
const s = steps[0];
|
||||
const body = await resolveStepText(ctx, s.contentHash);
|
||||
lines.push("");
|
||||
lines.push(`## Step: ${s.role}`);
|
||||
lines.push("");
|
||||
lines.push(s.content);
|
||||
lines.push(body);
|
||||
lines.push("");
|
||||
lines.push(`Meta: ${JSON.stringify(s.meta)}`);
|
||||
} else {
|
||||
@@ -31,10 +41,11 @@ export function buildAgentPrompt(ctx: AgentContext): string {
|
||||
lines.push(`Summary: ${JSON.stringify(s.meta)}`);
|
||||
}
|
||||
const last = steps[steps.length - 1];
|
||||
const lastBody = await resolveStepText(ctx, last.contentHash);
|
||||
lines.push("");
|
||||
lines.push(`## Latest Step: ${last.role}`);
|
||||
lines.push("");
|
||||
lines.push(last.content);
|
||||
lines.push(lastBody);
|
||||
lines.push("");
|
||||
lines.push(`Meta: ${JSON.stringify(last.meta)}`);
|
||||
}
|
||||
|
||||
@@ -22,6 +22,8 @@ describe("buildDescriptor", () => {
|
||||
systemPrompt: "You are an analyst.",
|
||||
extractPrompt: "Extract title and count from the analysis.",
|
||||
schema,
|
||||
extractRefs: null,
|
||||
extractMode: "single",
|
||||
},
|
||||
},
|
||||
moderator: () => END,
|
||||
|
||||
@@ -26,6 +26,19 @@ export const run = async function* (input) {
|
||||
expect(r.ok).toBe(true);
|
||||
});
|
||||
|
||||
test("allows static import of @uncaged/workflow", () => {
|
||||
const source = `${minimalDescriptor}import { putContentMerkleNode } from "@uncaged/workflow";
|
||||
|
||||
export const run = async function* (_input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "x");
|
||||
return { returnCode: 0, summary: h };
|
||||
};
|
||||
`;
|
||||
const r = validateWorkflowBundle({ filePath: "/tmp/w.esm.js", source });
|
||||
expect(r.ok).toBe(true);
|
||||
});
|
||||
|
||||
test("rejects wrong filename suffix", () => {
|
||||
const r = validateWorkflowBundle({
|
||||
filePath: "/tmp/w.js",
|
||||
|
||||
@@ -3,10 +3,16 @@ import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { createThreadCas } from "../src/cas.js";
|
||||
import { createCasStore, createThreadCas } from "../src/cas.js";
|
||||
import { hashString } from "../src/hash.js";
|
||||
|
||||
describe("createThreadCas", () => {
|
||||
describe("cas module exports", () => {
|
||||
test("createThreadCas is a deprecated alias of createCasStore", () => {
|
||||
expect(createThreadCas).toBe(createCasStore);
|
||||
});
|
||||
});
|
||||
|
||||
describe("createCasStore", () => {
|
||||
let casDir: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
@@ -18,7 +24,7 @@ describe("createThreadCas", () => {
|
||||
});
|
||||
|
||||
test("put returns consistent hash for same content", async () => {
|
||||
const cas = createThreadCas(casDir);
|
||||
const cas = createCasStore(casDir);
|
||||
const h1 = await cas.put("hello world");
|
||||
const h2 = await cas.put("hello world");
|
||||
expect(h1).toBe(h2);
|
||||
@@ -26,14 +32,14 @@ describe("createThreadCas", () => {
|
||||
});
|
||||
|
||||
test("put returns hash matching hashString", async () => {
|
||||
const cas = createThreadCas(casDir);
|
||||
const cas = createCasStore(casDir);
|
||||
const content = "some content to store";
|
||||
const h = await cas.put(content);
|
||||
expect(h).toBe(hashString(content));
|
||||
});
|
||||
|
||||
test("get returns stored content", async () => {
|
||||
const cas = createThreadCas(casDir);
|
||||
const cas = createCasStore(casDir);
|
||||
const content = "line1\nline2\nline3";
|
||||
const h = await cas.put(content);
|
||||
const retrieved = await cas.get(h);
|
||||
@@ -41,13 +47,13 @@ describe("createThreadCas", () => {
|
||||
});
|
||||
|
||||
test("get returns null for missing hash", async () => {
|
||||
const cas = createThreadCas(casDir);
|
||||
const cas = createCasStore(casDir);
|
||||
const result = await cas.get("0000000000000");
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
|
||||
test("delete removes entry", async () => {
|
||||
const cas = createThreadCas(casDir);
|
||||
const cas = createCasStore(casDir);
|
||||
const h = await cas.put("to be deleted");
|
||||
await cas.delete(h);
|
||||
const result = await cas.get(h);
|
||||
@@ -55,12 +61,12 @@ describe("createThreadCas", () => {
|
||||
});
|
||||
|
||||
test("delete on missing hash does not throw", async () => {
|
||||
const cas = createThreadCas(casDir);
|
||||
const cas = createCasStore(casDir);
|
||||
await cas.delete("0000000000000");
|
||||
});
|
||||
|
||||
test("list returns all stored hashes", async () => {
|
||||
const cas = createThreadCas(casDir);
|
||||
const cas = createCasStore(casDir);
|
||||
const h1 = await cas.put("aaa");
|
||||
const h2 = await cas.put("bbb");
|
||||
const h3 = await cas.put("ccc");
|
||||
@@ -69,13 +75,13 @@ describe("createThreadCas", () => {
|
||||
});
|
||||
|
||||
test("list returns empty array when cas dir does not exist", async () => {
|
||||
const cas = createThreadCas(join(casDir, "nonexistent"));
|
||||
const cas = createCasStore(join(casDir, "nonexistent"));
|
||||
const hashes = await cas.list();
|
||||
expect(hashes).toEqual([]);
|
||||
});
|
||||
|
||||
test("put is idempotent — same content written twice causes no error", async () => {
|
||||
const cas = createThreadCas(casDir);
|
||||
const cas = createCasStore(casDir);
|
||||
const h1 = await cas.put("idempotent");
|
||||
const h2 = await cas.put("idempotent");
|
||||
expect(h1).toBe(h2);
|
||||
@@ -84,7 +90,7 @@ describe("createThreadCas", () => {
|
||||
});
|
||||
|
||||
test("different content produces different hashes", async () => {
|
||||
const cas = createThreadCas(casDir);
|
||||
const cas = createCasStore(casDir);
|
||||
const h1 = await cas.put("alpha");
|
||||
const h2 = await cas.put("beta");
|
||||
expect(h1).not.toBe(h2);
|
||||
|
||||
@@ -4,11 +4,18 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import * as z from "zod/v4";
|
||||
|
||||
import { createCasStore } from "../src/cas.js";
|
||||
import { createWorkflow } from "../src/create-workflow.js";
|
||||
import { executeThread } from "../src/engine.js";
|
||||
import { createExtract } from "../src/extract-fn.js";
|
||||
import { createLogger } from "../src/logger.js";
|
||||
import { END } from "../src/types.js";
|
||||
import {
|
||||
createContentMerkleNode,
|
||||
getContentMerklePayload,
|
||||
parseMerkleNode,
|
||||
serializeMerkleNode,
|
||||
} from "../src/merkle.js";
|
||||
import { END, type LlmProvider } from "../src/types.js";
|
||||
|
||||
const plannerMetaSchema = z.object({
|
||||
plan: z.string(),
|
||||
@@ -89,12 +96,16 @@ const demoWorkflow = createWorkflow<DemoMeta>(
|
||||
systemPrompt: "You are a planner.",
|
||||
extractPrompt: "Extract plan text and affected files list.",
|
||||
schema: plannerMetaSchema,
|
||||
extractRefs: null,
|
||||
extractMode: "single",
|
||||
},
|
||||
coder: {
|
||||
description: "Demo coder",
|
||||
systemPrompt: "You are a coder.",
|
||||
extractPrompt: "Extract the code diff summary.",
|
||||
schema: coderMetaSchema,
|
||||
extractRefs: null,
|
||||
extractMode: "single",
|
||||
},
|
||||
},
|
||||
moderator: (ctx) => {
|
||||
@@ -115,6 +126,7 @@ const demoWorkflow = createWorkflow<DemoMeta>(
|
||||
},
|
||||
},
|
||||
demoExtract,
|
||||
null,
|
||||
);
|
||||
|
||||
describe("executeThread", () => {
|
||||
@@ -138,6 +150,7 @@ describe("executeThread", () => {
|
||||
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
|
||||
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
|
||||
await mkdir(join(root, "logs", hash), { recursive: true });
|
||||
const cas = createCasStore(join(root, "cas"));
|
||||
|
||||
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
|
||||
const ac = new AbortController();
|
||||
@@ -148,16 +161,30 @@ describe("executeThread", () => {
|
||||
{ prompt: "Fix the login redirect bug in #3", steps: [] },
|
||||
{
|
||||
maxRounds: 5,
|
||||
depth: 0,
|
||||
signal: ac.signal,
|
||||
awaitAfterEachYield: async () => {},
|
||||
forkSourceThreadId: null,
|
||||
prefilledDiskSteps: null,
|
||||
},
|
||||
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath },
|
||||
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
|
||||
logger,
|
||||
);
|
||||
|
||||
expect(result.returnCode).toBe(0);
|
||||
expect(typeof result.rootHash).toBe("string");
|
||||
expect(result.rootHash.length).toBeGreaterThan(0);
|
||||
|
||||
const rootYaml = await cas.get(result.rootHash);
|
||||
expect(rootYaml).not.toBeNull();
|
||||
const rootNode = parseMerkleNode(rootYaml ?? "");
|
||||
expect(rootNode.type).toBe("thread");
|
||||
const rootPayload = rootNode.payload as Record<string, unknown>;
|
||||
expect(rootPayload.workflow).toBe("demo-flow");
|
||||
expect(rootPayload.threadId).toBe(threadId);
|
||||
const rootResult = rootPayload.result as Record<string, unknown>;
|
||||
expect(rootResult.returnCode).toBe(0);
|
||||
expect(rootNode.children.length).toBe(2);
|
||||
|
||||
const dataText = await readFile(dataPath, "utf8");
|
||||
const lines = dataText
|
||||
@@ -176,16 +203,34 @@ describe("executeThread", () => {
|
||||
expect(params.prompt).toBe("Fix the login redirect bug in #3");
|
||||
const opts = params.options as Record<string, unknown>;
|
||||
expect(opts.maxRounds).toBe(5);
|
||||
expect(Object.keys(opts).sort()).toEqual(["maxRounds"]);
|
||||
expect(opts.depth).toBe(0);
|
||||
expect(Object.keys(opts).sort()).toEqual(["depth", "maxRounds"]);
|
||||
|
||||
const role1 = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
|
||||
expect(role1.role).toBe("planner");
|
||||
expect(role1.content).toBe("plan-body");
|
||||
expect(typeof role1.contentHash).toBe("string");
|
||||
expect(await getContentMerklePayload(cas, String(role1.contentHash))).toBe("plan-body");
|
||||
expect(role1.meta).toEqual({ plan: "do-it", files: ["a.ts"] });
|
||||
expect(role1.refs).toEqual([role1.contentHash]);
|
||||
expect(typeof role1.timestamp).toBe("number");
|
||||
|
||||
const role2 = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
|
||||
expect(role2.role).toBe("coder");
|
||||
expect(role2.refs).toEqual([role2.contentHash]);
|
||||
|
||||
const step1Yaml = await cas.get(rootNode.children[0] ?? "");
|
||||
const step2Yaml = await cas.get(rootNode.children[1] ?? "");
|
||||
expect(step1Yaml).not.toBeNull();
|
||||
expect(step2Yaml).not.toBeNull();
|
||||
const step1Node = parseMerkleNode(step1Yaml ?? "");
|
||||
const step2Node = parseMerkleNode(step2Yaml ?? "");
|
||||
expect(step1Node.type).toBe("step");
|
||||
expect(step2Node.type).toBe("step");
|
||||
expect(step1Node.children).toEqual([String(role1.contentHash)]);
|
||||
expect(step2Node.children).toEqual([String(role2.contentHash)]);
|
||||
const step1Payload = step1Node.payload as Record<string, unknown>;
|
||||
expect(step1Payload.role).toBe("planner");
|
||||
expect(step1Payload.meta).toEqual({ plan: "do-it", files: ["a.ts"] });
|
||||
|
||||
const infoText = await readFile(infoPath, "utf8");
|
||||
const infoLines = infoText
|
||||
@@ -213,11 +258,14 @@ describe("executeThread", () => {
|
||||
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
|
||||
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
|
||||
await mkdir(join(root, "logs", hash), { recursive: true });
|
||||
const cas = createCasStore(join(root, "cas"));
|
||||
const plannerHash = await cas.put(serializeMerkleNode(createContentMerkleNode("plan-body")));
|
||||
|
||||
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
|
||||
const ac = new AbortController();
|
||||
|
||||
const histTs = 9_000_000;
|
||||
const mergedPlannerRefs = ["CAS111AAAAAAA", plannerHash];
|
||||
const result = await executeThread(
|
||||
demoWorkflow,
|
||||
"demo-flow",
|
||||
@@ -226,30 +274,38 @@ describe("executeThread", () => {
|
||||
steps: [
|
||||
{
|
||||
role: "planner",
|
||||
content: "plan-body",
|
||||
contentHash: plannerHash,
|
||||
meta: { plan: "do-it", files: ["a.ts"] },
|
||||
refs: mergedPlannerRefs,
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
maxRounds: 5,
|
||||
depth: 0,
|
||||
signal: ac.signal,
|
||||
awaitAfterEachYield: async () => {},
|
||||
forkSourceThreadId: "01SRC1111111111111111111",
|
||||
prefilledDiskSteps: [
|
||||
{
|
||||
role: "planner",
|
||||
content: "plan-body",
|
||||
contentHash: plannerHash,
|
||||
meta: { plan: "do-it", files: ["a.ts"] },
|
||||
refs: mergedPlannerRefs,
|
||||
timestamp: histTs,
|
||||
},
|
||||
],
|
||||
},
|
||||
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath },
|
||||
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
|
||||
logger,
|
||||
);
|
||||
|
||||
expect(result.returnCode).toBe(0);
|
||||
expect(typeof result.rootHash).toBe("string");
|
||||
|
||||
const rootYaml = await cas.get(result.rootHash);
|
||||
const rootNode = parseMerkleNode(rootYaml ?? "");
|
||||
expect(rootNode.children.length).toBe(2);
|
||||
|
||||
const dataText = await readFile(dataPath, "utf8");
|
||||
const lines = dataText
|
||||
@@ -264,10 +320,11 @@ describe("executeThread", () => {
|
||||
const role0 = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
|
||||
expect(role0.role).toBe("planner");
|
||||
expect(role0.timestamp).toBe(histTs);
|
||||
expect(role0.refs).toEqual(mergedPlannerRefs);
|
||||
|
||||
const role1 = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
|
||||
expect(role1.role).toBe("coder");
|
||||
expect(role1.content).toBe("code-body");
|
||||
expect(await getContentMerklePayload(cas, String(role1.contentHash))).toBe("code-body");
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
@@ -281,6 +338,7 @@ describe("executeThread", () => {
|
||||
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
|
||||
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
|
||||
await mkdir(join(root, "logs", hash), { recursive: true });
|
||||
const cas = createCasStore(join(root, "cas"));
|
||||
|
||||
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
|
||||
const ac = new AbortController();
|
||||
@@ -291,16 +349,23 @@ describe("executeThread", () => {
|
||||
{ prompt: "hello", steps: [] },
|
||||
{
|
||||
maxRounds: 0,
|
||||
depth: 0,
|
||||
signal: ac.signal,
|
||||
awaitAfterEachYield: async () => {},
|
||||
forkSourceThreadId: null,
|
||||
prefilledDiskSteps: null,
|
||||
},
|
||||
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath },
|
||||
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
|
||||
logger,
|
||||
);
|
||||
|
||||
expect(result.returnCode).toBe(0);
|
||||
expect(typeof result.rootHash).toBe("string");
|
||||
|
||||
const rootYaml = await cas.get(result.rootHash);
|
||||
const rootNode = parseMerkleNode(rootYaml ?? "");
|
||||
expect(rootNode.type).toBe("thread");
|
||||
expect(rootNode.children.length).toBe(0);
|
||||
|
||||
const dataText = await readFile(dataPath, "utf8");
|
||||
const lines = dataText
|
||||
@@ -312,4 +377,240 @@ describe("executeThread", () => {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("Merkle DAG: root → step nodes → content for full thread traversal", async () => {
|
||||
restoreFetch = installMockChatCompletions([
|
||||
{ plan: "do-it", files: ["a.ts"] },
|
||||
{ diff: "+ok" },
|
||||
]);
|
||||
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-engine-dag-"));
|
||||
try {
|
||||
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
|
||||
const hash = "C9NMV6V2TQT81";
|
||||
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
|
||||
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
|
||||
await mkdir(join(root, "logs", hash), { recursive: true });
|
||||
const cas = createCasStore(join(root, "cas"));
|
||||
|
||||
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
|
||||
const ac = new AbortController();
|
||||
|
||||
const result = await executeThread(
|
||||
demoWorkflow,
|
||||
"demo-flow",
|
||||
{ prompt: "DAG test", steps: [] },
|
||||
{
|
||||
maxRounds: 5,
|
||||
depth: 0,
|
||||
signal: ac.signal,
|
||||
awaitAfterEachYield: async () => {},
|
||||
forkSourceThreadId: null,
|
||||
prefilledDiskSteps: null,
|
||||
},
|
||||
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
|
||||
logger,
|
||||
);
|
||||
|
||||
const dataText = await readFile(dataPath, "utf8");
|
||||
const lines = dataText
|
||||
.trim()
|
||||
.split("\n")
|
||||
.filter((l) => l !== "");
|
||||
expect(lines.length).toBe(3);
|
||||
|
||||
const rolePlanner = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
|
||||
const roleCoder = JSON.parse(lines[2] ?? "{}") as Record<string, unknown>;
|
||||
|
||||
const threadYaml = await cas.get(result.rootHash);
|
||||
expect(threadYaml).not.toBeNull();
|
||||
const threadNode = parseMerkleNode(threadYaml ?? "");
|
||||
expect(threadNode.type).toBe("thread");
|
||||
|
||||
const bodies: string[] = [];
|
||||
for (const stepHash of threadNode.children) {
|
||||
const stepYaml = await cas.get(stepHash);
|
||||
expect(stepYaml).not.toBeNull();
|
||||
const stepNode = parseMerkleNode(stepYaml ?? "");
|
||||
expect(stepNode.type).toBe("step");
|
||||
expect(stepNode.children.length).toBe(1);
|
||||
const contentHash = stepNode.children[0];
|
||||
expect(contentHash).toBeDefined();
|
||||
const body = await getContentMerklePayload(cas, contentHash ?? "");
|
||||
expect(body).not.toBeNull();
|
||||
bodies.push(body ?? "");
|
||||
}
|
||||
|
||||
expect(bodies.sort()).toEqual(["code-body", "plan-body"].sort());
|
||||
expect(rolePlanner.role).toBe("planner");
|
||||
expect(roleCoder.role).toBe("coder");
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("extractMode react traverses CAS DAG via cas_get during extraction", async () => {
|
||||
const dagMetaSchema = z.object({ leafPayload: z.string() });
|
||||
type DagDemoMeta = { walker: z.infer<typeof dagMetaSchema> };
|
||||
|
||||
const origFetch = globalThis.fetch;
|
||||
restoreFetch = () => {
|
||||
globalThis.fetch = origFetch;
|
||||
};
|
||||
let fetchRound = 0;
|
||||
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-engine-react-"));
|
||||
try {
|
||||
const cas = createCasStore(join(root, "cas"));
|
||||
const leafYaml = serializeMerkleNode(createContentMerkleNode("needle-from-leaf"));
|
||||
const leafHash = await cas.put(leafYaml);
|
||||
const rootYaml = serializeMerkleNode({
|
||||
type: "thread",
|
||||
payload: {
|
||||
workflow: "dag-demo",
|
||||
threadId: "01DAG00000000000000000001",
|
||||
result: { returnCode: 0, summary: "" },
|
||||
},
|
||||
children: [leafHash],
|
||||
});
|
||||
const dagRootHash = await cas.put(rootYaml);
|
||||
|
||||
globalThis.fetch = Object.assign(
|
||||
async (_input: Parameters<typeof fetch>[0], _init?: RequestInit) => {
|
||||
fetchRound += 1;
|
||||
if (fetchRound === 1) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
tool_calls: [
|
||||
{
|
||||
id: "c1",
|
||||
type: "function",
|
||||
function: {
|
||||
name: "cas_get",
|
||||
arguments: JSON.stringify({ hash: dagRootHash }),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
}
|
||||
if (fetchRound === 2) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
tool_calls: [
|
||||
{
|
||||
id: "c2",
|
||||
type: "function",
|
||||
function: {
|
||||
name: "cas_get",
|
||||
arguments: JSON.stringify({ hash: leafHash }),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
}
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
tool_calls: [
|
||||
{
|
||||
id: "c3",
|
||||
type: "function",
|
||||
function: {
|
||||
name: "extract",
|
||||
arguments: JSON.stringify({ leafPayload: "needle-from-leaf" }),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
},
|
||||
{ preconnect: origFetch.preconnect.bind(origFetch) },
|
||||
) as typeof fetch;
|
||||
|
||||
const llm: LlmProvider = { baseUrl: "http://127.0.0.1:9", apiKey: "test", model: "test" };
|
||||
const extractFn = createExtract(llm);
|
||||
|
||||
const dagWorkflow = createWorkflow<DagDemoMeta>(
|
||||
{
|
||||
roles: {
|
||||
walker: {
|
||||
description: "DAG walker",
|
||||
systemPrompt: "Output only the root CAS hash.",
|
||||
extractPrompt:
|
||||
"Set leafPayload to the string payload of the content Merkle node under the root.",
|
||||
schema: dagMetaSchema,
|
||||
extractRefs: null,
|
||||
extractMode: "react",
|
||||
},
|
||||
},
|
||||
moderator: (ctx) => (ctx.steps.length === 0 ? "walker" : END),
|
||||
},
|
||||
{ agent: async () => dagRootHash },
|
||||
extractFn,
|
||||
llm,
|
||||
);
|
||||
|
||||
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
|
||||
const hash = "C9NMV6V2TQT81";
|
||||
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
|
||||
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
|
||||
await mkdir(join(root, "logs", hash), { recursive: true });
|
||||
|
||||
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
|
||||
const ac = new AbortController();
|
||||
|
||||
const result = await executeThread(
|
||||
dagWorkflow,
|
||||
"dag-demo",
|
||||
{ prompt: "traverse", steps: [] },
|
||||
{
|
||||
maxRounds: 5,
|
||||
depth: 0,
|
||||
signal: ac.signal,
|
||||
awaitAfterEachYield: async () => {},
|
||||
forkSourceThreadId: null,
|
||||
prefilledDiskSteps: null,
|
||||
},
|
||||
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
|
||||
logger,
|
||||
);
|
||||
|
||||
expect(result.returnCode).toBe(0);
|
||||
expect(fetchRound).toBe(3);
|
||||
|
||||
const dataText = await readFile(dataPath, "utf8");
|
||||
const lines = dataText
|
||||
.trim()
|
||||
.split("\n")
|
||||
.filter((l) => l !== "");
|
||||
const roleRec = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
|
||||
expect(roleRec.role).toBe("walker");
|
||||
expect(roleRec.meta).toEqual({ leafPayload: "needle-from-leaf" });
|
||||
} finally {
|
||||
globalThis.fetch = origFetch;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { getExtractProvider } from "../src/extract-provider.js";
|
||||
|
||||
describe("getExtractProvider", () => {
|
||||
test("returns provider when config.extract is present", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-ext-prov-ok-"));
|
||||
try {
|
||||
await mkdir(root, { recursive: true });
|
||||
await writeFile(
|
||||
join(root, "workflow.yaml"),
|
||||
`config:
|
||||
maxDepth: 3
|
||||
extract:
|
||||
baseUrl: https://dashscope.aliyuncs.com/compatible-mode/v1
|
||||
model: qwen-plus
|
||||
apiKey: literal-key
|
||||
workflows: {}
|
||||
`,
|
||||
"utf8",
|
||||
);
|
||||
const r = await getExtractProvider(root);
|
||||
expect(r.ok).toBe(true);
|
||||
if (!r.ok) {
|
||||
return;
|
||||
}
|
||||
expect(r.value.baseUrl).toBe("https://dashscope.aliyuncs.com/compatible-mode/v1");
|
||||
expect(r.value.model).toBe("qwen-plus");
|
||||
expect(r.value.apiKey).toBe("literal-key");
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("errs when registry has no config section", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-ext-prov-missing-"));
|
||||
try {
|
||||
await mkdir(root, { recursive: true });
|
||||
await writeFile(join(root, "workflow.yaml"), "workflows: {}\n", "utf8");
|
||||
const r = await getExtractProvider(root);
|
||||
expect(r.ok).toBe(false);
|
||||
if (r.ok) {
|
||||
return;
|
||||
}
|
||||
expect(r.error).toContain("no global config");
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("resolves apiKey from env at registry read time", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-ext-prov-env-"));
|
||||
const prev = process.env.WF_GET_EXTRACT_PROVIDER_KEY;
|
||||
process.env.WF_GET_EXTRACT_PROVIDER_KEY = "resolved-secret";
|
||||
try {
|
||||
await mkdir(root, { recursive: true });
|
||||
await writeFile(
|
||||
join(root, "workflow.yaml"),
|
||||
`config:
|
||||
maxDepth: 1
|
||||
extract:
|
||||
baseUrl: https://example.com
|
||||
model: m
|
||||
apiKey: env:WF_GET_EXTRACT_PROVIDER_KEY
|
||||
workflows: {}
|
||||
`,
|
||||
"utf8",
|
||||
);
|
||||
const r = await getExtractProvider(root);
|
||||
expect(r.ok).toBe(true);
|
||||
if (!r.ok) {
|
||||
return;
|
||||
}
|
||||
expect(r.value.apiKey).toBe("resolved-secret");
|
||||
} finally {
|
||||
if (prev === undefined) {
|
||||
delete process.env.WF_GET_EXTRACT_PROVIDER_KEY;
|
||||
} else {
|
||||
process.env.WF_GET_EXTRACT_PROVIDER_KEY = prev;
|
||||
}
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -7,9 +7,9 @@ import {
|
||||
} from "../src/fork-thread.js";
|
||||
|
||||
const sampleDataJsonl = `{"name":"demo","hash":"C9NMV6V2TQT81","threadId":"01AAA1111111111111111111","parameters":{"prompt":"hi","options":{"maxRounds":5}},"timestamp":100}
|
||||
{"role":"planner","content":"p","meta":{},"timestamp":101}
|
||||
{"role":"coder","content":"c","meta":{},"timestamp":102}
|
||||
{"role":"reviewer","content":"r","meta":{},"timestamp":103}
|
||||
{"role":"planner","contentHash":"HP0000000000000000000001","meta":{},"refs":[],"timestamp":101}
|
||||
{"role":"coder","contentHash":"HP0000000000000000000002","meta":{},"refs":[],"timestamp":102}
|
||||
{"role":"reviewer","contentHash":"HP0000000000000000000003","meta":{},"refs":[],"timestamp":103}
|
||||
`;
|
||||
|
||||
describe("fork-thread", () => {
|
||||
@@ -24,6 +24,7 @@ describe("fork-thread", () => {
|
||||
expect(r.value.start.threadId).toBe("01AAA1111111111111111111");
|
||||
expect(r.value.start.prompt).toBe("hi");
|
||||
expect(r.value.start.maxRounds).toBe(5);
|
||||
expect(r.value.start.depth).toBe(0);
|
||||
expect(r.value.roleSteps.length).toBe(3);
|
||||
expect(r.value.roleSteps[0]?.role).toBe("planner");
|
||||
});
|
||||
@@ -83,6 +84,24 @@ describe("fork-thread", () => {
|
||||
expect(r.value.workflowName).toBe("demo");
|
||||
expect(r.value.historicalSteps.length).toBe(1);
|
||||
expect(r.value.historicalSteps[0]?.timestamp).toBe(101);
|
||||
expect(r.value.runOptions).toEqual({ maxRounds: 5 });
|
||||
expect(r.value.runOptions).toEqual({ maxRounds: 5, depth: 0 });
|
||||
});
|
||||
|
||||
test("parseThreadDataJsonl reads explicit depth from start record", () => {
|
||||
const text = `{"name":"demo","hash":"H","threadId":"01ZZZZZZZZZZZZZZZZZZZZZZ","parameters":{"prompt":"p","options":{"maxRounds":3,"depth":2}},"timestamp":1}
|
||||
{"role":"planner","contentHash":"HP0000000000000000000099","meta":{},"refs":[],"timestamp":2}
|
||||
`;
|
||||
const r = parseThreadDataJsonl(text);
|
||||
expect(r.ok).toBe(true);
|
||||
if (!r.ok) {
|
||||
return;
|
||||
}
|
||||
expect(r.value.start.depth).toBe(2);
|
||||
const plan = buildForkPlan(text, null);
|
||||
expect(plan.ok).toBe(true);
|
||||
if (!plan.ok) {
|
||||
return;
|
||||
}
|
||||
expect(plan.value.runOptions).toEqual({ maxRounds: 3, depth: 2 });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { createContentMerkleNode, parseMerkleNode, serializeMerkleNode } from "../src/merkle.js";
|
||||
|
||||
describe("merkle", () => {
|
||||
test("content node roundtrips through YAML", () => {
|
||||
const node = createContentMerkleNode("hello\nworld");
|
||||
const yaml = serializeMerkleNode(node);
|
||||
const back = parseMerkleNode(yaml);
|
||||
expect(back).toEqual(node);
|
||||
});
|
||||
|
||||
test("step node with object payload roundtrips", () => {
|
||||
const node = {
|
||||
type: "step" as const,
|
||||
payload: { role: "planner", foo: 1 },
|
||||
children: ["ABC123", "DEF456"],
|
||||
};
|
||||
const yaml = serializeMerkleNode(node);
|
||||
const back = parseMerkleNode(yaml);
|
||||
expect(back.type).toBe("step");
|
||||
expect(back.payload).toEqual({ role: "planner", foo: 1 });
|
||||
expect(back.children).toEqual(["ABC123", "DEF456"]);
|
||||
});
|
||||
|
||||
test("parse rejects invalid YAML root", () => {
|
||||
expect(() => parseMerkleNode("[]")).toThrow();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,209 @@
|
||||
import { afterEach, describe, expect, test } from "bun:test";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import * as z from "zod/v4";
|
||||
|
||||
import { createCasStore } from "../src/cas.js";
|
||||
import { createContentMerkleNode, serializeMerkleNode } from "../src/merkle.js";
|
||||
import { reactExtract } from "../src/react-extract.js";
|
||||
import type { LlmProvider } from "../src/types.js";
|
||||
|
||||
const metaSchema = z.object({ seen: z.string() });
|
||||
|
||||
const provider: LlmProvider = {
|
||||
baseUrl: "http://127.0.0.1:9",
|
||||
apiKey: "test",
|
||||
model: "test",
|
||||
};
|
||||
|
||||
describe("reactExtract", () => {
|
||||
let restoreFetch: (() => void) | null = null;
|
||||
|
||||
afterEach(() => {
|
||||
restoreFetch?.();
|
||||
restoreFetch = null;
|
||||
});
|
||||
|
||||
test("cas_get rounds then extract tool yields validated meta", async () => {
|
||||
const casDir = await mkdtemp(join(tmpdir(), "react-extract-"));
|
||||
const cas = createCasStore(casDir);
|
||||
try {
|
||||
const blob = serializeMerkleNode(createContentMerkleNode("needle"));
|
||||
const h = await cas.put(blob);
|
||||
|
||||
const origFetch = globalThis.fetch;
|
||||
let round = 0;
|
||||
restoreFetch = () => {
|
||||
globalThis.fetch = origFetch;
|
||||
};
|
||||
globalThis.fetch = Object.assign(
|
||||
async (_input: Parameters<typeof fetch>[0], _init?: RequestInit) => {
|
||||
round += 1;
|
||||
if (round === 1) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
tool_calls: [
|
||||
{
|
||||
id: "t1",
|
||||
type: "function",
|
||||
function: {
|
||||
name: "cas_get",
|
||||
arguments: JSON.stringify({ hash: h }),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
}
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
tool_calls: [
|
||||
{
|
||||
id: "t2",
|
||||
type: "function",
|
||||
function: {
|
||||
name: "extract",
|
||||
arguments: JSON.stringify({ seen: "needle" }),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
},
|
||||
{ preconnect: origFetch.preconnect.bind(origFetch) },
|
||||
) as typeof fetch;
|
||||
|
||||
const text = `## Agent Output\n${h}\n## Extraction Instruction\nExtract seen from CAS.`;
|
||||
const result = await reactExtract({
|
||||
text,
|
||||
schema: metaSchema,
|
||||
provider,
|
||||
cas,
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
if (!result.ok) {
|
||||
return;
|
||||
}
|
||||
expect(result.value).toEqual({ seen: "needle" });
|
||||
expect(round).toBe(2);
|
||||
} finally {
|
||||
await rm(casDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("stops after max tool rounds when model keeps calling cas_get", async () => {
|
||||
const casDir = await mkdtemp(join(tmpdir(), "react-extract-max-"));
|
||||
const cas = createCasStore(casDir);
|
||||
try {
|
||||
const blob = serializeMerkleNode(createContentMerkleNode("x"));
|
||||
const h = await cas.put(blob);
|
||||
|
||||
const origFetch = globalThis.fetch;
|
||||
let round = 0;
|
||||
restoreFetch = () => {
|
||||
globalThis.fetch = origFetch;
|
||||
};
|
||||
globalThis.fetch = Object.assign(
|
||||
async (_input: Parameters<typeof fetch>[0], _init?: RequestInit) => {
|
||||
round += 1;
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
tool_calls: [
|
||||
{
|
||||
id: `loop-${round}`,
|
||||
type: "function",
|
||||
function: {
|
||||
name: "cas_get",
|
||||
arguments: JSON.stringify({ hash: h }),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
},
|
||||
{ preconnect: origFetch.preconnect.bind(origFetch) },
|
||||
) as typeof fetch;
|
||||
|
||||
const result = await reactExtract({
|
||||
text: "## Agent Output\nnoop\n## Extraction Instruction\nExtract seen.",
|
||||
schema: metaSchema,
|
||||
provider,
|
||||
cas,
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
if (result.ok) {
|
||||
return;
|
||||
}
|
||||
expect(result.error).toBe("max_react_rounds_exceeded");
|
||||
expect(round).toBe(10);
|
||||
} finally {
|
||||
await rm(casDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("passthrough JSON assistant message without tool calls", async () => {
|
||||
const casDir = await mkdtemp(join(tmpdir(), "react-extract-pass-"));
|
||||
const cas = createCasStore(casDir);
|
||||
try {
|
||||
const origFetch = globalThis.fetch;
|
||||
restoreFetch = () => {
|
||||
globalThis.fetch = origFetch;
|
||||
};
|
||||
globalThis.fetch = Object.assign(
|
||||
async (_input: Parameters<typeof fetch>[0], _init?: RequestInit) =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
content: '{"seen":"direct"}',
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
),
|
||||
{ preconnect: origFetch.preconnect.bind(origFetch) },
|
||||
) as typeof fetch;
|
||||
|
||||
const result = await reactExtract({
|
||||
text: "## Agent Output\nok\n## Extraction Instruction\nExtract.",
|
||||
schema: metaSchema,
|
||||
provider,
|
||||
cas,
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
if (!result.ok) {
|
||||
return;
|
||||
}
|
||||
expect(result.value).toEqual({ seen: "direct" });
|
||||
} finally {
|
||||
await rm(casDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,203 @@
|
||||
import { afterEach, describe, expect, test } from "bun:test";
|
||||
import { mkdir, mkdtemp, readFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import * as z from "zod/v4";
|
||||
|
||||
import { createCasStore } from "../src/cas.js";
|
||||
import { createWorkflow } from "../src/create-workflow.js";
|
||||
import { executeThread } from "../src/engine.js";
|
||||
import { createExtract } from "../src/extract-fn.js";
|
||||
import { buildForkPlan, parseThreadDataJsonl } from "../src/fork-thread.js";
|
||||
import { createLogger } from "../src/logger.js";
|
||||
import { END } from "../src/types.js";
|
||||
|
||||
const phaseSchema = z.object({
|
||||
hash: z.string(),
|
||||
title: z.string(),
|
||||
});
|
||||
|
||||
const plannerMetaSchema = z.object({
|
||||
phases: z.array(phaseSchema),
|
||||
});
|
||||
|
||||
type RefsDemoMeta = {
|
||||
planner: z.infer<typeof plannerMetaSchema>;
|
||||
};
|
||||
|
||||
function installMockChatCompletions(sequence: ReadonlyArray<Record<string, unknown>>): () => void {
|
||||
const origFetch = globalThis.fetch;
|
||||
let i = 0;
|
||||
const mockFetch = async (
|
||||
input: Parameters<typeof fetch>[0],
|
||||
init?: RequestInit,
|
||||
): Promise<Response> => {
|
||||
const args = sequence[i] ?? sequence[sequence.length - 1];
|
||||
if (args === undefined) {
|
||||
throw new Error("installMockChatCompletions: empty sequence");
|
||||
}
|
||||
i += 1;
|
||||
void input;
|
||||
const body = init?.body ? (JSON.parse(String(init.body)) as Record<string, unknown>) : {};
|
||||
const tools = body.tools;
|
||||
const firstTool =
|
||||
Array.isArray(tools) && tools.length > 0 && tools[0] !== null && typeof tools[0] === "object"
|
||||
? (tools[0] as Record<string, unknown>)
|
||||
: null;
|
||||
const fn =
|
||||
firstTool !== null ? (firstTool.function as Record<string, unknown> | undefined) : undefined;
|
||||
const toolName = typeof fn?.name === "string" ? fn.name : "extract";
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
tool_calls: [
|
||||
{
|
||||
type: "function",
|
||||
function: {
|
||||
name: toolName,
|
||||
arguments: JSON.stringify(args),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
};
|
||||
globalThis.fetch = Object.assign(mockFetch, {
|
||||
preconnect: origFetch.preconnect.bind(origFetch),
|
||||
}) as typeof fetch;
|
||||
return () => {
|
||||
globalThis.fetch = origFetch;
|
||||
};
|
||||
}
|
||||
|
||||
const refsDemoExtract = createExtract({
|
||||
baseUrl: "http://127.0.0.1:9",
|
||||
apiKey: "test",
|
||||
model: "test",
|
||||
});
|
||||
|
||||
const refsDemoWorkflow = createWorkflow<RefsDemoMeta>(
|
||||
{
|
||||
roles: {
|
||||
planner: {
|
||||
description: "Planner with phase hashes",
|
||||
systemPrompt: "Plan.",
|
||||
extractPrompt: "Extract phases with CAS hashes.",
|
||||
schema: plannerMetaSchema,
|
||||
extractRefs: (meta) => meta.phases.map((p) => p.hash),
|
||||
extractMode: "single",
|
||||
},
|
||||
},
|
||||
moderator: (ctx) => (ctx.steps.length === 0 ? "planner" : END),
|
||||
},
|
||||
{
|
||||
agent: async () => "plan-output",
|
||||
},
|
||||
refsDemoExtract,
|
||||
null,
|
||||
);
|
||||
|
||||
describe("RoleStep refs tracking", () => {
|
||||
let restoreFetch: (() => void) | null = null;
|
||||
|
||||
afterEach(() => {
|
||||
restoreFetch?.();
|
||||
restoreFetch = null;
|
||||
});
|
||||
|
||||
test("parseThreadDataJsonl reads refs and defaults missing refs to []", () => {
|
||||
const text = `{"name":"demo","hash":"C9NMV6V2TQT81","threadId":"01AAA1111111111111111111","parameters":{"prompt":"hi","options":{"maxRounds":5}},"timestamp":100}
|
||||
{"role":"planner","contentHash":"HPAYLOAD111111","meta":{},"refs":["H111AAAAAAAAA","H222AAAAAAAAA"],"timestamp":101}
|
||||
{"role":"coder","contentHash":"HPAYLOAD222222","meta":{},"timestamp":102}
|
||||
`;
|
||||
const r = parseThreadDataJsonl(text);
|
||||
expect(r.ok).toBe(true);
|
||||
if (!r.ok) {
|
||||
return;
|
||||
}
|
||||
expect(r.value.roleSteps[0]?.refs).toEqual(["H111AAAAAAAAA", "H222AAAAAAAAA"]);
|
||||
expect(r.value.roleSteps[1]?.refs).toEqual([]);
|
||||
});
|
||||
|
||||
test("executeThread persists refs from extractRefs on role yields", async () => {
|
||||
restoreFetch = installMockChatCompletions([
|
||||
{
|
||||
phases: [
|
||||
{ hash: "C9NMV6V2TQT81", title: "phase-a" },
|
||||
{ hash: "C9NMV6V2TQT82", title: "phase-b" },
|
||||
],
|
||||
},
|
||||
]);
|
||||
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-refs-"));
|
||||
try {
|
||||
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
|
||||
const hash = "C9NMV6V2TQT81";
|
||||
const dataPath = join(root, "logs", hash, `${threadId}.data.jsonl`);
|
||||
const infoPath = join(root, "logs", hash, `${threadId}.info.jsonl`);
|
||||
await mkdir(join(root, "logs", hash), { recursive: true });
|
||||
const cas = createCasStore(join(root, "cas"));
|
||||
|
||||
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
|
||||
const ac = new AbortController();
|
||||
|
||||
const result = await executeThread(
|
||||
refsDemoWorkflow,
|
||||
"refs-demo",
|
||||
{ prompt: "task", steps: [] },
|
||||
{
|
||||
maxRounds: 5,
|
||||
depth: 0,
|
||||
signal: ac.signal,
|
||||
awaitAfterEachYield: async () => {},
|
||||
forkSourceThreadId: null,
|
||||
prefilledDiskSteps: null,
|
||||
},
|
||||
{ threadId, hash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
|
||||
logger,
|
||||
);
|
||||
|
||||
expect(result.returnCode).toBe(0);
|
||||
expect(typeof result.rootHash).toBe("string");
|
||||
expect(result.rootHash.length).toBeGreaterThan(0);
|
||||
|
||||
const dataText = await readFile(dataPath, "utf8");
|
||||
const lines = dataText
|
||||
.trim()
|
||||
.split("\n")
|
||||
.filter((l) => l !== "");
|
||||
expect(lines.length).toBe(2);
|
||||
|
||||
const role1 = JSON.parse(lines[1] ?? "{}") as Record<string, unknown>;
|
||||
expect(role1.role).toBe("planner");
|
||||
const refs = role1.refs as string[];
|
||||
expect(refs).toContain("C9NMV6V2TQT81");
|
||||
expect(refs).toContain("C9NMV6V2TQT82");
|
||||
expect(typeof role1.contentHash).toBe("string");
|
||||
expect(refs).toContain(String(role1.contentHash));
|
||||
expect(refs.length).toBe(3);
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("buildForkPlan carries refs on historical steps", () => {
|
||||
const text = `{"name":"demo","hash":"C9NMV6V2TQT81","threadId":"01AAA1111111111111111111","parameters":{"prompt":"hi","options":{"maxRounds":5}},"timestamp":100}
|
||||
{"role":"planner","contentHash":"HP111111111111","meta":{},"refs":["KEEPREFAAAAAA"],"timestamp":101}
|
||||
{"role":"coder","contentHash":"HP222222222222","meta":{},"refs":["CODERHASHAAAA"],"timestamp":102}
|
||||
`;
|
||||
const plan = buildForkPlan(text, null);
|
||||
expect(plan.ok).toBe(true);
|
||||
if (!plan.ok) {
|
||||
return;
|
||||
}
|
||||
expect(plan.value.historicalSteps.length).toBe(1);
|
||||
expect(plan.value.historicalSteps[0]?.refs).toEqual(["KEEPREFAAAAAA"]);
|
||||
});
|
||||
});
|
||||
@@ -4,6 +4,7 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import {
|
||||
parseWorkflowRegistryYaml,
|
||||
readWorkflowRegistry,
|
||||
registerWorkflowVersion,
|
||||
rollbackWorkflowToHistoryHash,
|
||||
@@ -21,6 +22,7 @@ describe("workflow registry", () => {
|
||||
if (!empty.ok) {
|
||||
return;
|
||||
}
|
||||
expect(empty.value.config).toBeNull();
|
||||
|
||||
const r1 = registerWorkflowVersion(empty.value, "solve-issue", "AAAAAAAAAAAAA", 100);
|
||||
const w1 = await writeWorkflowRegistry(dir, r1);
|
||||
@@ -68,7 +70,7 @@ describe("workflow registry", () => {
|
||||
});
|
||||
|
||||
test("rollbackWorkflowToHistoryHash swaps head with a prior version", () => {
|
||||
let reg = registerWorkflowVersion({ workflows: {} }, "solve-issue", "H1", 100);
|
||||
let reg = registerWorkflowVersion({ config: null, workflows: {} }, "solve-issue", "H1", 100);
|
||||
reg = registerWorkflowVersion(reg, "solve-issue", "H2", 200);
|
||||
reg = registerWorkflowVersion(reg, "solve-issue", "H3", 300);
|
||||
const entry = reg.workflows["solve-issue"];
|
||||
@@ -99,6 +101,85 @@ describe("workflow registry", () => {
|
||||
expect(bad.ok).toBe(false);
|
||||
});
|
||||
|
||||
test("parses config section and literal apiKey", () => {
|
||||
const yaml = `
|
||||
config:
|
||||
maxDepth: 3
|
||||
extract:
|
||||
baseUrl: https://example.com/v1
|
||||
model: qwen-plus
|
||||
apiKey: secret-key
|
||||
workflows:
|
||||
solve-issue:
|
||||
hash: SPVR4BDMSGC1W
|
||||
timestamp: 1
|
||||
history: []
|
||||
`;
|
||||
const r = parseWorkflowRegistryYaml(yaml);
|
||||
expect(r.ok).toBe(true);
|
||||
if (!r.ok) {
|
||||
return;
|
||||
}
|
||||
expect(r.value.config).not.toBeNull();
|
||||
if (r.value.config === null) {
|
||||
return;
|
||||
}
|
||||
expect(r.value.config.maxDepth).toBe(3);
|
||||
expect(r.value.config.extract.baseUrl).toBe("https://example.com/v1");
|
||||
expect(r.value.config.extract.model).toBe("qwen-plus");
|
||||
expect(r.value.config.extract.apiKey).toBe("secret-key");
|
||||
});
|
||||
|
||||
test("parses config apiKey env: prefix from process.env", () => {
|
||||
const prev = process.env.WF_REGISTRY_TEST_API_KEY;
|
||||
process.env.WF_REGISTRY_TEST_API_KEY = "from-env";
|
||||
try {
|
||||
const yaml = `
|
||||
config:
|
||||
maxDepth: 1
|
||||
extract:
|
||||
baseUrl: https://dashscope.aliyuncs.com/compatible-mode/v1
|
||||
model: qwen-plus
|
||||
apiKey: env:WF_REGISTRY_TEST_API_KEY
|
||||
workflows: {}
|
||||
`;
|
||||
const r = parseWorkflowRegistryYaml(yaml);
|
||||
expect(r.ok).toBe(true);
|
||||
if (!r.ok) {
|
||||
return;
|
||||
}
|
||||
expect(r.value.config?.extract.apiKey).toBe("from-env");
|
||||
} finally {
|
||||
if (prev === undefined) {
|
||||
delete process.env.WF_REGISTRY_TEST_API_KEY;
|
||||
} else {
|
||||
process.env.WF_REGISTRY_TEST_API_KEY = prev;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("parse errors when env: apiKey variable is unset", () => {
|
||||
const prev = process.env.WF_REGISTRY_TEST_API_KEY_UNSET;
|
||||
delete process.env.WF_REGISTRY_TEST_API_KEY_UNSET;
|
||||
try {
|
||||
const yaml = `
|
||||
config:
|
||||
maxDepth: 1
|
||||
extract:
|
||||
baseUrl: https://example.com
|
||||
model: m
|
||||
apiKey: env:WF_REGISTRY_TEST_API_KEY_UNSET
|
||||
workflows: {}
|
||||
`;
|
||||
const r = parseWorkflowRegistryYaml(yaml);
|
||||
expect(r.ok).toBe(false);
|
||||
} finally {
|
||||
if (prev !== undefined) {
|
||||
process.env.WF_REGISTRY_TEST_API_KEY_UNSET = prev;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("parse errors on invalid shape", async () => {
|
||||
const dir = join(tmpdir(), `wf-reg3-${process.pid}-${Date.now()}`);
|
||||
await mkdir(dir, { recursive: true });
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { getDefaultWorkflowStorageRoot, getGlobalCasDir } from "../src/storage-root.js";
|
||||
|
||||
describe("getGlobalCasDir", () => {
|
||||
test("joins cas segment under explicit storage root", () => {
|
||||
expect(getGlobalCasDir("/tmp/wf-root")).toBe(join("/tmp/wf-root", "cas"));
|
||||
});
|
||||
|
||||
test("defaults to default workflow root when storage root is undefined", () => {
|
||||
expect(getGlobalCasDir(undefined)).toBe(join(getDefaultWorkflowStorageRoot(), "cas"));
|
||||
});
|
||||
});
|
||||
@@ -10,6 +10,7 @@ describe("RFC-001 thread JSONL shapes", () => {
|
||||
prompt: "Fix the login redirect bug in #3",
|
||||
options: {
|
||||
maxRounds: 5,
|
||||
depth: 0,
|
||||
},
|
||||
},
|
||||
timestamp: 1714963200000,
|
||||
@@ -17,15 +18,18 @@ describe("RFC-001 thread JSONL shapes", () => {
|
||||
|
||||
const roleRecord = {
|
||||
role: "planner",
|
||||
content: "Plan: modify auth middleware...",
|
||||
contentHash: "CPHASH000000000000000001",
|
||||
meta: { plan: "...", files: ["src/auth.ts"] },
|
||||
refs: [] as string[],
|
||||
timestamp: 1714963201000,
|
||||
};
|
||||
|
||||
expect(Object.keys(startRecord).sort()).toEqual(
|
||||
["hash", "name", "parameters", "threadId", "timestamp"].sort(),
|
||||
);
|
||||
expect(Object.keys(roleRecord).sort()).toEqual(["content", "meta", "role", "timestamp"].sort());
|
||||
expect(Object.keys(roleRecord).sort()).toEqual(
|
||||
["contentHash", "meta", "refs", "role", "timestamp"].sort(),
|
||||
);
|
||||
});
|
||||
|
||||
test("documents the `.info.jsonl` debug record keys", () => {
|
||||
|
||||
@@ -5,22 +5,29 @@ import { createConnection } from "node:net";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { createCasStore } from "../src/cas.js";
|
||||
import { createContentMerkleNode, serializeMerkleNode } from "../src/merkle.js";
|
||||
import { getWorkerHostScriptPath } from "../src/worker-entry-path.js";
|
||||
|
||||
const bundleSource = `export const descriptor = {
|
||||
const bundleSource = `import { putContentMerkleNode } from "@uncaged/workflow";
|
||||
|
||||
export const descriptor = {
|
||||
description: "worker-test",
|
||||
roles: {
|
||||
planner: { description: "planner", schema: {} },
|
||||
coder: { description: "coder", schema: {} },
|
||||
},
|
||||
};
|
||||
export const run = async function* (input) {
|
||||
export const run = async function* (input, options) {
|
||||
const cas = options.cas;
|
||||
const has = (r) => input.steps.some((s) => s.role === r);
|
||||
if (!has("planner")) {
|
||||
yield { role: "planner", content: "p", meta: { plan: input.prompt } };
|
||||
const h = await putContentMerkleNode(cas, "p");
|
||||
yield { role: "planner", contentHash: h, meta: { plan: input.prompt }, refs: [h] };
|
||||
}
|
||||
if (!has("coder")) {
|
||||
yield { role: "coder", content: "c", meta: { diff: "y" } };
|
||||
const h = await putContentMerkleNode(cas, "c");
|
||||
yield { role: "coder", contentHash: h, meta: { diff: "y" }, refs: [h] };
|
||||
}
|
||||
return { returnCode: 0, summary: "completed: moderator returned END" };
|
||||
};
|
||||
@@ -102,7 +109,7 @@ describe("worker process", () => {
|
||||
threadId,
|
||||
workflowName: "demo-flow",
|
||||
prompt: "hello",
|
||||
options: { maxRounds: 5 },
|
||||
options: { maxRounds: 5, depth: 0 },
|
||||
});
|
||||
|
||||
const exitCode: number = await new Promise((resolve) => {
|
||||
@@ -143,6 +150,11 @@ describe("worker process", () => {
|
||||
|
||||
const port = await readReadyPort(child);
|
||||
|
||||
const cas = createCasStore(join(root, "cas"));
|
||||
const plannerReplayHash = await cas.put(
|
||||
serializeMerkleNode(createContentMerkleNode("p-old")),
|
||||
);
|
||||
|
||||
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
|
||||
const srcId = "01SRCMMMMMMMMMMMMMMMMMMMM";
|
||||
await sendJson(port, {
|
||||
@@ -150,12 +162,13 @@ describe("worker process", () => {
|
||||
threadId,
|
||||
workflowName: "demo-flow",
|
||||
prompt: "hello",
|
||||
options: { maxRounds: 5 },
|
||||
options: { maxRounds: 5, depth: 0 },
|
||||
steps: [
|
||||
{
|
||||
role: "planner",
|
||||
content: "p-old",
|
||||
contentHash: plannerReplayHash,
|
||||
meta: { plan: "z" },
|
||||
refs: [plannerReplayHash],
|
||||
timestamp: 555,
|
||||
},
|
||||
],
|
||||
|
||||
@@ -0,0 +1,225 @@
|
||||
import { afterEach, describe, expect, test } from "bun:test";
|
||||
import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import * as z from "zod/v4";
|
||||
|
||||
import { createCasStore } from "../src/cas.js";
|
||||
import { createWorkflow } from "../src/create-workflow.js";
|
||||
import { executeThread } from "../src/engine.js";
|
||||
import { createExtract } from "../src/extract-fn.js";
|
||||
import { hashWorkflowBundleBytes } from "../src/hash.js";
|
||||
import { createLogger } from "../src/logger.js";
|
||||
import { getContentMerklePayload, parseMerkleNode } from "../src/merkle.js";
|
||||
import {
|
||||
readWorkflowRegistry,
|
||||
registerWorkflowVersion,
|
||||
writeWorkflowRegistry,
|
||||
} from "../src/registry.js";
|
||||
import { END } from "../src/types.js";
|
||||
import { workflowAsAgent } from "../src/workflow-as-agent.js";
|
||||
|
||||
const callerMetaSchema = z.object({ done: z.literal(true) });
|
||||
|
||||
type ParentMeta = {
|
||||
caller: z.infer<typeof callerMetaSchema>;
|
||||
};
|
||||
|
||||
function installMockChatCompletions(sequence: ReadonlyArray<Record<string, unknown>>): () => void {
|
||||
const origFetch = globalThis.fetch;
|
||||
let i = 0;
|
||||
const mockFetch = async (
|
||||
input: Parameters<typeof fetch>[0],
|
||||
init?: RequestInit,
|
||||
): Promise<Response> => {
|
||||
const args = sequence[i] ?? sequence[sequence.length - 1];
|
||||
if (args === undefined) {
|
||||
throw new Error("installMockChatCompletions: empty sequence");
|
||||
}
|
||||
i += 1;
|
||||
void input;
|
||||
const body = init?.body ? (JSON.parse(String(init.body)) as Record<string, unknown>) : {};
|
||||
const tools = body.tools;
|
||||
const firstTool =
|
||||
Array.isArray(tools) && tools.length > 0 && tools[0] !== null && typeof tools[0] === "object"
|
||||
? (tools[0] as Record<string, unknown>)
|
||||
: null;
|
||||
const fn =
|
||||
firstTool !== null ? (firstTool.function as Record<string, unknown> | undefined) : undefined;
|
||||
const toolName = typeof fn?.name === "string" ? fn.name : "extract";
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
tool_calls: [
|
||||
{
|
||||
type: "function",
|
||||
function: {
|
||||
name: toolName,
|
||||
arguments: JSON.stringify(args),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
};
|
||||
globalThis.fetch = Object.assign(mockFetch, {
|
||||
preconnect: origFetch.preconnect.bind(origFetch),
|
||||
}) as typeof fetch;
|
||||
return () => {
|
||||
globalThis.fetch = origFetch;
|
||||
};
|
||||
}
|
||||
|
||||
const parentExtract = createExtract({
|
||||
baseUrl: "http://127.0.0.1:9",
|
||||
apiKey: "test",
|
||||
model: "test",
|
||||
});
|
||||
|
||||
const childBundleSource = `import { putContentMerkleNode } from "@uncaged/workflow";
|
||||
|
||||
export const descriptor = {
|
||||
description: "child-integration",
|
||||
roles: {
|
||||
agent: {
|
||||
description: "agent",
|
||||
schema: { type: "object", properties: {}, additionalProperties: true },
|
||||
},
|
||||
},
|
||||
};
|
||||
export async function* run(input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "child-body");
|
||||
yield { role: "agent", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "child-done:" + input.prompt };
|
||||
}
|
||||
`;
|
||||
|
||||
async function installChildWorkflow(storageRoot: string): Promise<{ hash: string }> {
|
||||
const bytes = new TextEncoder().encode(childBundleSource);
|
||||
const hash = hashWorkflowBundleBytes(bytes);
|
||||
await mkdir(join(storageRoot, "bundles"), { recursive: true });
|
||||
await writeFile(join(storageRoot, "bundles", `${hash}.esm.js`), childBundleSource, "utf8");
|
||||
const reg = await readWorkflowRegistry(storageRoot);
|
||||
if (!reg.ok) {
|
||||
throw reg.error;
|
||||
}
|
||||
const next = registerWorkflowVersion(reg.value, "child-wf", hash, Date.now());
|
||||
const wr = await writeWorkflowRegistry(storageRoot, next);
|
||||
if (!wr.ok) {
|
||||
throw wr.error;
|
||||
}
|
||||
return { hash };
|
||||
}
|
||||
|
||||
describe("workflowAsAgent integration", () => {
|
||||
let restoreFetch: (() => void) | null = null;
|
||||
|
||||
afterEach(() => {
|
||||
restoreFetch?.();
|
||||
restoreFetch = null;
|
||||
});
|
||||
|
||||
test("createWorkflow parent invokes nested workflow via workflowAsAgent", async () => {
|
||||
restoreFetch = installMockChatCompletions([{ done: true }]);
|
||||
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-waa-int-"));
|
||||
try {
|
||||
const { hash: childHash } = await installChildWorkflow(root);
|
||||
|
||||
const parentWorkflow = createWorkflow<ParentMeta>(
|
||||
{
|
||||
roles: {
|
||||
caller: {
|
||||
description: "delegates to child workflow",
|
||||
systemPrompt: "system",
|
||||
extractPrompt: "extract done flag",
|
||||
schema: callerMetaSchema,
|
||||
extractRefs: null,
|
||||
extractMode: "single",
|
||||
},
|
||||
},
|
||||
moderator: (ctx) => (ctx.steps.length === 0 ? "caller" : END),
|
||||
},
|
||||
{ agent: workflowAsAgent("child-wf", { storageRoot: root }) },
|
||||
parentExtract,
|
||||
null,
|
||||
);
|
||||
|
||||
const threadId = "01KQXKW18CT8G75T53R8F4G7YG";
|
||||
const parentHash = "C9NMV6V2TQT81";
|
||||
const dataPath = join(root, "logs", parentHash, `${threadId}.data.jsonl`);
|
||||
const infoPath = join(root, "logs", parentHash, `${threadId}.info.jsonl`);
|
||||
await mkdir(join(root, "logs", parentHash), { recursive: true });
|
||||
const cas = createCasStore(join(root, "cas"));
|
||||
|
||||
const logger = createLogger({ sink: { kind: "file", path: infoPath } });
|
||||
const ac = new AbortController();
|
||||
|
||||
const result = await executeThread(
|
||||
parentWorkflow,
|
||||
"parent-wf",
|
||||
{ prompt: "from-parent", steps: [] },
|
||||
{
|
||||
maxRounds: 5,
|
||||
depth: 0,
|
||||
signal: ac.signal,
|
||||
awaitAfterEachYield: async () => {},
|
||||
forkSourceThreadId: null,
|
||||
prefilledDiskSteps: null,
|
||||
},
|
||||
{ threadId, hash: parentHash, dataJsonlPath: dataPath, infoJsonlPath: infoPath, cas },
|
||||
logger,
|
||||
);
|
||||
|
||||
expect(result.returnCode).toBe(0);
|
||||
expect(typeof result.rootHash).toBe("string");
|
||||
|
||||
const parentText = await readFile(dataPath, "utf8");
|
||||
const parentLines = parentText
|
||||
.trim()
|
||||
.split("\n")
|
||||
.filter((l) => l !== "");
|
||||
expect(parentLines.length).toBe(2);
|
||||
const callerLine = JSON.parse(parentLines[1] ?? "{}") as Record<string, unknown>;
|
||||
expect(callerLine.role).toBe("caller");
|
||||
const childRootHash = await getContentMerklePayload(cas, String(callerLine.contentHash));
|
||||
expect(childRootHash).not.toBeNull();
|
||||
const childThreadYaml = await cas.get(childRootHash ?? "");
|
||||
expect(childThreadYaml).not.toBeNull();
|
||||
const childThreadNode = parseMerkleNode(childThreadYaml ?? "");
|
||||
expect(childThreadNode.type).toBe("thread");
|
||||
const childPayload = childThreadNode.payload as Record<string, unknown>;
|
||||
expect(childPayload.workflow).toBe("child-wf");
|
||||
const childResult = childPayload.result as Record<string, unknown>;
|
||||
expect(childResult.summary).toBe("child-done:from-parent");
|
||||
|
||||
const childDir = join(root, "logs", childHash);
|
||||
const childFiles = await readdir(childDir);
|
||||
const childDataName = childFiles.find((n) => n.endsWith(".data.jsonl"));
|
||||
expect(childDataName).toBeDefined();
|
||||
|
||||
const childText = await readFile(join(childDir, childDataName ?? ""), "utf8");
|
||||
const childStart = JSON.parse(
|
||||
childText
|
||||
.trim()
|
||||
.split("\n")
|
||||
.filter((l) => l !== "")[0] ?? "{}",
|
||||
) as Record<string, unknown>;
|
||||
expect(childStart.forkFrom).toEqual({ threadId });
|
||||
const childOpts = (childStart.parameters as Record<string, unknown>).options as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
expect(childOpts.depth).toBe(1);
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,168 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { createCasStore } from "../src/cas.js";
|
||||
import { hashWorkflowBundleBytes } from "../src/hash.js";
|
||||
import { parseMerkleNode } from "../src/merkle.js";
|
||||
import {
|
||||
readWorkflowRegistry,
|
||||
registerWorkflowVersion,
|
||||
writeWorkflowRegistry,
|
||||
} from "../src/registry.js";
|
||||
import { type AgentContext, START } from "../src/types.js";
|
||||
import { workflowAsAgent } from "../src/workflow-as-agent.js";
|
||||
|
||||
function makeAgentCtx(params: {
|
||||
storageRoot: string;
|
||||
depth: number;
|
||||
prompt: string;
|
||||
maxRounds: number;
|
||||
}): AgentContext {
|
||||
const ts = Date.now();
|
||||
return {
|
||||
threadId: "01PARENT000000000000000001AA",
|
||||
depth: params.depth,
|
||||
start: {
|
||||
role: START,
|
||||
content: params.prompt,
|
||||
meta: { maxRounds: params.maxRounds },
|
||||
timestamp: ts,
|
||||
},
|
||||
steps: [],
|
||||
currentRole: {
|
||||
name: "caller",
|
||||
systemPrompt: "caller",
|
||||
},
|
||||
cas: createCasStore(join(params.storageRoot, "agent-ctx-cas")),
|
||||
};
|
||||
}
|
||||
|
||||
const childBundleSource = `import { putContentMerkleNode } from "@uncaged/workflow";
|
||||
|
||||
export const descriptor = {
|
||||
description: "child-test",
|
||||
roles: {
|
||||
agent: {
|
||||
description: "agent",
|
||||
schema: { type: "object", properties: {}, additionalProperties: true },
|
||||
},
|
||||
},
|
||||
};
|
||||
export async function* run(input, options) {
|
||||
const cas = options.cas;
|
||||
const h = await putContentMerkleNode(cas, "child-body");
|
||||
yield { role: "agent", contentHash: h, meta: {}, refs: [h] };
|
||||
return { returnCode: 0, summary: "child-done:" + input.prompt };
|
||||
}
|
||||
`;
|
||||
|
||||
async function installChildWorkflow(storageRoot: string): Promise<{ hash: string }> {
|
||||
const bytes = new TextEncoder().encode(childBundleSource);
|
||||
const hash = hashWorkflowBundleBytes(bytes);
|
||||
await mkdir(join(storageRoot, "bundles"), { recursive: true });
|
||||
await writeFile(join(storageRoot, "bundles", `${hash}.esm.js`), childBundleSource, "utf8");
|
||||
const reg = await readWorkflowRegistry(storageRoot);
|
||||
if (!reg.ok) {
|
||||
throw reg.error;
|
||||
}
|
||||
const next = registerWorkflowVersion(reg.value, "child-wf", hash, Date.now());
|
||||
const wr = await writeWorkflowRegistry(storageRoot, next);
|
||||
if (!wr.ok) {
|
||||
throw wr.error;
|
||||
}
|
||||
return { hash };
|
||||
}
|
||||
|
||||
describe("workflowAsAgent", () => {
|
||||
test("returns error when workflow name is not registered", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-waa-missing-"));
|
||||
try {
|
||||
const agent = workflowAsAgent("missing-wf", { storageRoot: root });
|
||||
const out = await agent(
|
||||
makeAgentCtx({ storageRoot: root, depth: 0, prompt: "x", maxRounds: 5 }),
|
||||
);
|
||||
expect(out).toContain("not found in registry");
|
||||
expect(out).toContain("missing-wf");
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("runs registered workflow and returns child thread root CAS hash", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-waa-ok-"));
|
||||
try {
|
||||
await installChildWorkflow(root);
|
||||
const agent = workflowAsAgent("child-wf", { storageRoot: root });
|
||||
const out = await agent(
|
||||
makeAgentCtx({ storageRoot: root, depth: 0, prompt: "hello-parent", maxRounds: 5 }),
|
||||
);
|
||||
const cas = createCasStore(join(root, "cas"));
|
||||
const threadYaml = await cas.get(out);
|
||||
expect(threadYaml).not.toBeNull();
|
||||
const node = parseMerkleNode(threadYaml ?? "");
|
||||
expect(node.type).toBe("thread");
|
||||
const payload = node.payload as Record<string, unknown>;
|
||||
expect(payload.workflow).toBe("child-wf");
|
||||
const resultObj = payload.result as Record<string, unknown>;
|
||||
expect(resultObj.summary).toBe("child-done:hello-parent");
|
||||
expect(node.children.length).toBe(1);
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("enforces depth limit (returns error string, does not throw)", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-waa-depth-"));
|
||||
try {
|
||||
const agent = workflowAsAgent("child-wf", { storageRoot: root });
|
||||
const out = await agent(
|
||||
makeAgentCtx({ storageRoot: root, depth: 3, prompt: "x", maxRounds: 5 }),
|
||||
);
|
||||
expect(out).toContain("depth limit");
|
||||
expect(out).toContain("max 3");
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("uses registry config maxDepth when set", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "wf-waa-maxdepth-cfg-"));
|
||||
try {
|
||||
await installChildWorkflow(root);
|
||||
const reg = await readWorkflowRegistry(root);
|
||||
expect(reg.ok).toBe(true);
|
||||
if (!reg.ok) {
|
||||
return;
|
||||
}
|
||||
const withCfg = {
|
||||
...reg.value,
|
||||
config: {
|
||||
maxDepth: 2,
|
||||
extract: {
|
||||
baseUrl: "http://127.0.0.1:9",
|
||||
model: "m",
|
||||
apiKey: "k",
|
||||
},
|
||||
},
|
||||
};
|
||||
const wr = await writeWorkflowRegistry(root, withCfg);
|
||||
expect(wr.ok).toBe(true);
|
||||
|
||||
const agent = workflowAsAgent("child-wf", { storageRoot: root });
|
||||
const okOut = await agent(
|
||||
makeAgentCtx({ storageRoot: root, depth: 1, prompt: "nest-once", maxRounds: 5 }),
|
||||
);
|
||||
expect(okOut).not.toContain("depth limit");
|
||||
|
||||
const badOut = await agent(
|
||||
makeAgentCtx({ storageRoot: root, depth: 2, prompt: "x", maxRounds: 5 }),
|
||||
);
|
||||
expect(badOut).toContain("depth limit");
|
||||
expect(badOut).toContain("max 2");
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,196 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { validateWorkflowDescriptor } from "../src/workflow-descriptor.js";
|
||||
|
||||
describe("validateWorkflowDescriptor", () => {
|
||||
// 1. Valid minimal descriptor
|
||||
test("accepts a minimal descriptor with empty roles", () => {
|
||||
const result = validateWorkflowDescriptor({ description: "x", roles: {} });
|
||||
expect(result.ok).toBe(true);
|
||||
if (result.ok) {
|
||||
expect(result.value.description).toBe("x");
|
||||
expect(result.value.roles).toEqual({});
|
||||
}
|
||||
});
|
||||
|
||||
// 2. Valid descriptor with one role
|
||||
test("accepts a descriptor with one role", () => {
|
||||
const result = validateWorkflowDescriptor({
|
||||
description: "workflow",
|
||||
roles: {
|
||||
solver: { description: "solves things", schema: { type: "object" } },
|
||||
},
|
||||
});
|
||||
expect(result.ok).toBe(true);
|
||||
if (result.ok) {
|
||||
expect(result.value.description).toBe("workflow");
|
||||
expect(result.value.roles.solver.description).toBe("solves things");
|
||||
expect(result.value.roles.solver.schema).toEqual({ type: "object" });
|
||||
}
|
||||
});
|
||||
|
||||
// 3. Valid descriptor with multiple roles
|
||||
test("accepts a descriptor with multiple roles", () => {
|
||||
const result = validateWorkflowDescriptor({
|
||||
description: "multi",
|
||||
roles: {
|
||||
a: { description: "role a", schema: {} },
|
||||
b: { description: "role b", schema: { type: "string" } },
|
||||
},
|
||||
});
|
||||
expect(result.ok).toBe(true);
|
||||
if (result.ok) {
|
||||
expect(Object.keys(result.value.roles)).toEqual(["a", "b"]);
|
||||
}
|
||||
});
|
||||
|
||||
// 4-6. Root is null / array / string / number / undefined
|
||||
test("rejects null", () => {
|
||||
const result = validateWorkflowDescriptor(null);
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor must be a non-array object");
|
||||
});
|
||||
|
||||
test("rejects an array", () => {
|
||||
const result = validateWorkflowDescriptor([]);
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor must be a non-array object");
|
||||
});
|
||||
|
||||
test("rejects a string", () => {
|
||||
const result = validateWorkflowDescriptor("hello");
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor must be a non-array object");
|
||||
});
|
||||
|
||||
test("rejects a number", () => {
|
||||
const result = validateWorkflowDescriptor(42);
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor must be a non-array object");
|
||||
});
|
||||
|
||||
test("rejects undefined", () => {
|
||||
const result = validateWorkflowDescriptor(undefined);
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor must be a non-array object");
|
||||
});
|
||||
|
||||
// 7-8. Missing or non-string description
|
||||
test("rejects missing description", () => {
|
||||
const result = validateWorkflowDescriptor({ roles: {} });
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.description must be a string");
|
||||
});
|
||||
|
||||
test("rejects numeric description", () => {
|
||||
const result = validateWorkflowDescriptor({ description: 123, roles: {} });
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.description must be a string");
|
||||
});
|
||||
|
||||
test("rejects null description", () => {
|
||||
const result = validateWorkflowDescriptor({ description: null, roles: {} });
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.description must be a string");
|
||||
});
|
||||
|
||||
test("rejects boolean description", () => {
|
||||
const result = validateWorkflowDescriptor({ description: true, roles: {} });
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.description must be a string");
|
||||
});
|
||||
|
||||
// 9-11. Missing / null / array roles
|
||||
test("rejects missing roles", () => {
|
||||
const result = validateWorkflowDescriptor({ description: "x" });
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.roles must be a non-array object");
|
||||
});
|
||||
|
||||
test("rejects null roles", () => {
|
||||
const result = validateWorkflowDescriptor({ description: "x", roles: null });
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.roles must be a non-array object");
|
||||
});
|
||||
|
||||
test("rejects array roles", () => {
|
||||
const result = validateWorkflowDescriptor({ description: "x", roles: [] });
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.roles must be a non-array object");
|
||||
});
|
||||
|
||||
// 12-13. Role entry is null / array
|
||||
test("rejects null role entry", () => {
|
||||
const result = validateWorkflowDescriptor({ description: "x", roles: { bad: null } });
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.roles.bad must be a non-array object");
|
||||
});
|
||||
|
||||
test("rejects array role entry", () => {
|
||||
const result = validateWorkflowDescriptor({ description: "x", roles: { bad: [] } });
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.roles.bad must be a non-array object");
|
||||
});
|
||||
|
||||
// 14-15. Role missing description / non-string description
|
||||
test("rejects role with missing description", () => {
|
||||
const result = validateWorkflowDescriptor({
|
||||
description: "x",
|
||||
roles: { r: { schema: {} } },
|
||||
});
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.roles.r.description must be a string");
|
||||
});
|
||||
|
||||
test("rejects role with non-string description", () => {
|
||||
const result = validateWorkflowDescriptor({
|
||||
description: "x",
|
||||
roles: { r: { description: 99, schema: {} } },
|
||||
});
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.roles.r.description must be a string");
|
||||
});
|
||||
|
||||
// 16-18. Role schema null / array / missing
|
||||
test("rejects role with null schema", () => {
|
||||
const result = validateWorkflowDescriptor({
|
||||
description: "x",
|
||||
roles: { r: { description: "d", schema: null } },
|
||||
});
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok)
|
||||
expect(result.error).toBe("descriptor.roles.r.schema must be a non-array object");
|
||||
});
|
||||
|
||||
test("rejects role with array schema", () => {
|
||||
const result = validateWorkflowDescriptor({
|
||||
description: "x",
|
||||
roles: { r: { description: "d", schema: [] } },
|
||||
});
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok)
|
||||
expect(result.error).toBe("descriptor.roles.r.schema must be a non-array object");
|
||||
});
|
||||
|
||||
test("rejects role with missing schema", () => {
|
||||
const result = validateWorkflowDescriptor({
|
||||
description: "x",
|
||||
roles: { r: { description: "d" } },
|
||||
});
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok)
|
||||
expect(result.error).toBe("descriptor.roles.r.schema must be a non-array object");
|
||||
});
|
||||
|
||||
// 19. First role valid, second role invalid
|
||||
test("rejects at first invalid role when earlier roles are valid", () => {
|
||||
const result = validateWorkflowDescriptor({
|
||||
description: "x",
|
||||
roles: {
|
||||
good: { description: "ok", schema: {} },
|
||||
bad: { description: 123, schema: {} },
|
||||
},
|
||||
});
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBe("descriptor.roles.bad.description must be a string");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,8 @@
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
/**
|
||||
* Dynamic-import a workflow bundle path (see {@link extractBundleExports} — symlink must exist first).
|
||||
*/
|
||||
export async function importWorkflowBundleModule(bundlePath: string): Promise<unknown> {
|
||||
return import(pathToFileURL(bundlePath).href);
|
||||
}
|
||||
@@ -41,9 +41,12 @@ function isAllowedImportSpecifier(spec: string): boolean {
|
||||
if (spec.length === 0) {
|
||||
return false;
|
||||
}
|
||||
if (spec.startsWith(".") || spec.startsWith("/")) {
|
||||
if (spec.startsWith(".") || spec.startsWith("/") || spec.startsWith("file:")) {
|
||||
return false;
|
||||
}
|
||||
if (spec === "@uncaged/workflow") {
|
||||
return true;
|
||||
}
|
||||
return isBuiltin(spec);
|
||||
}
|
||||
|
||||
@@ -297,7 +300,7 @@ function validateImportDeclaration(node: ImportDeclaration): string | null {
|
||||
return "only static string import specifiers are allowed";
|
||||
}
|
||||
if (!isAllowedImportSpecifier(spec)) {
|
||||
return `disallowed import specifier "${spec}" (only Node built-ins are allowed)`;
|
||||
return `disallowed import specifier "${spec}" (only Node built-ins and "@uncaged/workflow" are allowed)`;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -312,7 +315,7 @@ function validateExportSource(
|
||||
return staticMessage;
|
||||
}
|
||||
if (!isAllowedImportSpecifier(spec)) {
|
||||
return `${disallowedPrefix} "${spec}" (only Node built-ins are allowed)`;
|
||||
return `${disallowedPrefix} "${spec}" (only Node built-ins and "@uncaged/workflow" are allowed)`;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -365,7 +368,7 @@ function bundleConstraintViolationForNode(node: Node): string | null {
|
||||
|
||||
/**
|
||||
* Validate RFC-001 bundle rules: single-file ESM shape, named exports `run` + `descriptor`,
|
||||
* no default export, no dynamic `import()`, static imports restricted to Node builtins.
|
||||
* no default export, no dynamic `import()`, static imports restricted to Node builtins plus `@uncaged/workflow`.
|
||||
*/
|
||||
export function validateWorkflowBundle(input: WorkflowBundleValidationInput): Result<void, string> {
|
||||
if (!endsWithEsmJs(input.filePath)) {
|
||||
|
||||
@@ -10,7 +10,7 @@ export type CasStore = {
|
||||
list(): Promise<string[]>;
|
||||
};
|
||||
|
||||
export function createThreadCas(casDir: string): CasStore {
|
||||
export function createCasStore(casDir: string): CasStore {
|
||||
async function ensureDir(): Promise<void> {
|
||||
await mkdir(casDir, { recursive: true });
|
||||
}
|
||||
@@ -68,3 +68,6 @@ export function createThreadCas(casDir: string): CasStore {
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** @deprecated Use {@link createCasStore} — CAS is global, not per-thread. */
|
||||
export const createThreadCas = createCasStore;
|
||||
|
||||
@@ -1,19 +1,25 @@
|
||||
import type { ExtractFn } from "./extract-fn.js";
|
||||
import type { CasStore } from "./cas.js";
|
||||
import { buildExtractUserContent, type ExtractFn } from "./extract-fn.js";
|
||||
import { putContentMerkleNode } from "./merkle.js";
|
||||
import { reactExtract } from "./react-extract.js";
|
||||
import { mergeRefsWithContentHash } from "./refs-field.js";
|
||||
import {
|
||||
type AgentBinding,
|
||||
type AgentContext,
|
||||
END,
|
||||
type ExtractContext,
|
||||
type LlmProvider,
|
||||
type ModeratorContext,
|
||||
type RoleDefinition,
|
||||
type RoleMeta,
|
||||
type RoleOutput,
|
||||
type RoleStep,
|
||||
START,
|
||||
type ThreadInput,
|
||||
type WorkflowCompletion,
|
||||
type WorkflowDefinition,
|
||||
type WorkflowFn,
|
||||
type WorkflowFnOptions,
|
||||
type WorkflowResult,
|
||||
} from "./types.js";
|
||||
|
||||
function isRoleNext<M extends RoleMeta>(
|
||||
@@ -22,19 +28,67 @@ function isRoleNext<M extends RoleMeta>(
|
||||
return next !== END;
|
||||
}
|
||||
|
||||
function resolveExtractedRefs(
|
||||
roleDef: RoleDefinition<Record<string, unknown>>,
|
||||
meta: unknown,
|
||||
): string[] {
|
||||
const extractRefsFn = roleDef.extractRefs;
|
||||
if (extractRefsFn === null || typeof extractRefsFn !== "function") {
|
||||
return [];
|
||||
}
|
||||
return extractRefsFn(meta as Record<string, unknown>);
|
||||
}
|
||||
|
||||
async function resolveRoleMeta<M extends RoleMeta>(
|
||||
roleDef: RoleDefinition<Record<string, unknown>>,
|
||||
extractCtx: ExtractContext<M>,
|
||||
extract: ExtractFn,
|
||||
llmProvider: LlmProvider | null,
|
||||
cas: CasStore,
|
||||
): Promise<Record<string, unknown>> {
|
||||
if (roleDef.extractMode === "react") {
|
||||
if (llmProvider === null) {
|
||||
throw new Error(
|
||||
'createWorkflow: llmProvider is required when a role uses extractMode "react"',
|
||||
);
|
||||
}
|
||||
const text = await buildExtractUserContent(
|
||||
extractCtx as unknown as ExtractContext,
|
||||
roleDef.extractPrompt,
|
||||
);
|
||||
const reactResult = await reactExtract({
|
||||
text,
|
||||
schema: roleDef.schema,
|
||||
provider: llmProvider,
|
||||
cas,
|
||||
});
|
||||
if (!reactResult.ok) {
|
||||
throw new Error(`react extract failed: ${reactResult.error}`);
|
||||
}
|
||||
return reactResult.value as Record<string, unknown>;
|
||||
}
|
||||
return (await extract(
|
||||
roleDef.schema,
|
||||
roleDef.extractPrompt,
|
||||
extractCtx as unknown as ExtractContext,
|
||||
)) as Record<string, unknown>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Binds pure role definitions + moderator to runtime agents and structured extraction.
|
||||
* Assign with `export const run = createWorkflow(def, binding, extract)`.
|
||||
* Assign with `export const run = createWorkflow(def, binding, extract, llmProvider)`.
|
||||
* Pass the same {@link LlmProvider} as {@link createExtract} when any role uses `extractMode: "react"`.
|
||||
*/
|
||||
export function createWorkflow<M extends RoleMeta>(
|
||||
def: Pick<WorkflowDefinition<M>, "roles" | "moderator">,
|
||||
binding: AgentBinding,
|
||||
extract: ExtractFn,
|
||||
llmProvider: LlmProvider | null,
|
||||
): WorkflowFn {
|
||||
return async function* workflowLoop(
|
||||
input: ThreadInput,
|
||||
options: WorkflowFnOptions,
|
||||
): AsyncGenerator<RoleOutput, WorkflowResult> {
|
||||
): AsyncGenerator<RoleOutput, WorkflowCompletion> {
|
||||
const nowMs = Date.now();
|
||||
const start: ModeratorContext<M>["start"] = {
|
||||
role: START,
|
||||
@@ -46,8 +100,9 @@ export function createWorkflow<M extends RoleMeta>(
|
||||
const baseTs = Date.now();
|
||||
let steps: RoleStep<M>[] = input.steps.map((out, i) => ({
|
||||
role: out.role,
|
||||
content: out.content,
|
||||
contentHash: out.contentHash,
|
||||
meta: out.meta,
|
||||
refs: out.refs,
|
||||
timestamp: baseTs + i,
|
||||
})) as RoleStep<M>[];
|
||||
|
||||
@@ -61,6 +116,7 @@ export function createWorkflow<M extends RoleMeta>(
|
||||
|
||||
const modCtx: ModeratorContext<M> = {
|
||||
threadId: options.threadId,
|
||||
depth: options.depth,
|
||||
start,
|
||||
steps,
|
||||
};
|
||||
@@ -79,6 +135,7 @@ export function createWorkflow<M extends RoleMeta>(
|
||||
const agentCtx: AgentContext<M> = {
|
||||
...modCtx,
|
||||
currentRole: { name: next, systemPrompt: roleDef.systemPrompt },
|
||||
cas: options.cas,
|
||||
};
|
||||
|
||||
const agent = binding.overrides?.[next] ?? binding.agent;
|
||||
@@ -90,21 +147,36 @@ export function createWorkflow<M extends RoleMeta>(
|
||||
agentContent: raw,
|
||||
};
|
||||
|
||||
const meta = await extract(
|
||||
roleDef.schema,
|
||||
roleDef.extractPrompt,
|
||||
extractCtx as unknown as ExtractContext,
|
||||
const meta = await resolveRoleMeta(
|
||||
roleDef as unknown as RoleDefinition<Record<string, unknown>>,
|
||||
extractCtx,
|
||||
extract,
|
||||
llmProvider,
|
||||
options.cas,
|
||||
);
|
||||
|
||||
const contentHash = await putContentMerkleNode(options.cas, raw);
|
||||
|
||||
const refs = mergeRefsWithContentHash(
|
||||
resolveExtractedRefs(roleDef as unknown as RoleDefinition<Record<string, unknown>>, meta),
|
||||
contentHash,
|
||||
);
|
||||
|
||||
const ts = Date.now();
|
||||
const step = {
|
||||
role: next,
|
||||
content: raw,
|
||||
contentHash,
|
||||
meta,
|
||||
refs,
|
||||
timestamp: ts,
|
||||
} as RoleStep<M>;
|
||||
|
||||
yield { role: step.role, content: step.content, meta: step.meta };
|
||||
yield {
|
||||
role: step.role,
|
||||
contentHash: step.contentHash,
|
||||
meta: step.meta,
|
||||
refs: step.refs,
|
||||
};
|
||||
|
||||
steps = [...steps, step];
|
||||
}
|
||||
|
||||
+135
-16
@@ -1,26 +1,39 @@
|
||||
import { appendFile, mkdir } from "node:fs/promises";
|
||||
import { dirname } from "node:path";
|
||||
|
||||
import type { CasStore } from "./cas.js";
|
||||
import type { LogFn } from "./logger.js";
|
||||
import type { ThreadInput, WorkflowFn, WorkflowFnOptions, WorkflowResult } from "./types.js";
|
||||
import { getContentMerklePayload, putStepMerkleNode, putThreadMerkleNode } from "./merkle.js";
|
||||
import { normalizeRefsField } from "./refs-field.js";
|
||||
import type {
|
||||
ThreadInput,
|
||||
WorkflowCompletion,
|
||||
WorkflowFn,
|
||||
WorkflowFnOptions,
|
||||
WorkflowResult,
|
||||
} from "./types.js";
|
||||
|
||||
export type ExecuteThreadIo = {
|
||||
threadId: string;
|
||||
hash: string;
|
||||
dataJsonlPath: string;
|
||||
infoJsonlPath: string;
|
||||
cas: CasStore;
|
||||
};
|
||||
|
||||
/** One persisted role line in `.data.jsonl` (engine adds these for fork replay before running the generator). */
|
||||
export type PrefilledDiskStep = {
|
||||
role: string;
|
||||
content: string;
|
||||
contentHash: string;
|
||||
meta: Record<string, unknown>;
|
||||
refs: string[];
|
||||
timestamp: number;
|
||||
};
|
||||
|
||||
export type ExecuteThreadOptions = {
|
||||
maxRounds: number;
|
||||
/** Passed to the bundle as `WorkflowFnOptions.depth`. */
|
||||
depth: number;
|
||||
signal: AbortSignal;
|
||||
/** Invoked after each successful yield (and outer-loop checks); used for pause/resume. */
|
||||
awaitAfterEachYield: () => Promise<void>;
|
||||
@@ -38,50 +51,123 @@ async function appendDataLine(path: string, record: unknown): Promise<void> {
|
||||
await appendFile(path, line, "utf8");
|
||||
}
|
||||
|
||||
async function finalizeThreadResult(params: {
|
||||
cas: CasStore;
|
||||
workflowName: string;
|
||||
threadId: string;
|
||||
stepMerkleHashes: readonly string[];
|
||||
completion: WorkflowCompletion;
|
||||
}): Promise<WorkflowResult> {
|
||||
const rootHash = await putThreadMerkleNode(
|
||||
params.cas,
|
||||
{
|
||||
workflow: params.workflowName,
|
||||
threadId: params.threadId,
|
||||
result: {
|
||||
returnCode: params.completion.returnCode,
|
||||
summary: params.completion.summary,
|
||||
},
|
||||
},
|
||||
params.stepMerkleHashes,
|
||||
);
|
||||
return {
|
||||
returnCode: params.completion.returnCode,
|
||||
summary: params.completion.summary,
|
||||
rootHash,
|
||||
};
|
||||
}
|
||||
|
||||
async function driveWorkflowGenerator(params: {
|
||||
fn: WorkflowFn;
|
||||
workflowName: string;
|
||||
input: ThreadInput;
|
||||
bundleOptions: WorkflowFnOptions;
|
||||
executeOptions: ExecuteThreadOptions;
|
||||
dataJsonlPath: string;
|
||||
threadId: string;
|
||||
logger: LogFn;
|
||||
cas: CasStore;
|
||||
stepMerkleHashes: string[];
|
||||
}): Promise<WorkflowResult> {
|
||||
const { fn, input, bundleOptions, executeOptions, dataJsonlPath, threadId, logger } = params;
|
||||
const {
|
||||
fn,
|
||||
workflowName,
|
||||
input,
|
||||
bundleOptions,
|
||||
executeOptions,
|
||||
dataJsonlPath,
|
||||
threadId,
|
||||
logger,
|
||||
cas,
|
||||
stepMerkleHashes,
|
||||
} = params;
|
||||
const gen = fn(input, bundleOptions);
|
||||
let written = 0;
|
||||
|
||||
while (true) {
|
||||
if (executeOptions.signal.aborted) {
|
||||
logger("V8JX4NP2", `thread ${threadId} aborted`);
|
||||
return { returnCode: 130, summary: "thread aborted" };
|
||||
return await finalizeThreadResult({
|
||||
cas,
|
||||
workflowName,
|
||||
threadId,
|
||||
stepMerkleHashes,
|
||||
completion: { returnCode: 130, summary: "thread aborted" },
|
||||
});
|
||||
}
|
||||
|
||||
if (written >= executeOptions.maxRounds) {
|
||||
logger("R3CW7YBQ", `thread ${threadId} stopped at maxRounds=${executeOptions.maxRounds}`);
|
||||
return {
|
||||
returnCode: 0,
|
||||
summary: `completed: reached maxRounds (${executeOptions.maxRounds})`,
|
||||
};
|
||||
return await finalizeThreadResult({
|
||||
cas,
|
||||
workflowName,
|
||||
threadId,
|
||||
stepMerkleHashes,
|
||||
completion: {
|
||||
returnCode: 0,
|
||||
summary: `completed: reached maxRounds (${executeOptions.maxRounds})`,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const iterResult = await gen.next();
|
||||
|
||||
if (iterResult.done) {
|
||||
logger("F3HN8QKP", `thread ${threadId} generator finished`);
|
||||
return iterResult.value;
|
||||
const completion = iterResult.value;
|
||||
return await finalizeThreadResult({
|
||||
cas,
|
||||
workflowName,
|
||||
threadId,
|
||||
stepMerkleHashes,
|
||||
completion,
|
||||
});
|
||||
}
|
||||
|
||||
written++;
|
||||
const step = iterResult.value;
|
||||
const resolved = await getContentMerklePayload(cas, step.contentHash);
|
||||
if (resolved === null) {
|
||||
throw new Error(
|
||||
`role step ${step.role}: CAS blob missing for contentHash ${step.contentHash}`,
|
||||
);
|
||||
}
|
||||
const ts = Date.now();
|
||||
await appendDataLine(dataJsonlPath, {
|
||||
role: step.role,
|
||||
content: step.content,
|
||||
contentHash: step.contentHash,
|
||||
meta: step.meta,
|
||||
refs: normalizeRefsField(step.refs),
|
||||
timestamp: ts,
|
||||
});
|
||||
|
||||
const stepNodeHash = await putStepMerkleNode(
|
||||
cas,
|
||||
{ role: step.role, meta: step.meta },
|
||||
step.contentHash,
|
||||
);
|
||||
stepMerkleHashes.push(stepNodeHash);
|
||||
|
||||
logger("N7BW4YHQ", `thread ${threadId} wrote role ${step.role}`);
|
||||
|
||||
await Promise.race([
|
||||
@@ -97,7 +183,13 @@ async function driveWorkflowGenerator(params: {
|
||||
|
||||
if (executeOptions.signal.aborted) {
|
||||
logger("V8JX4NP4", `thread ${threadId} aborted`);
|
||||
return { returnCode: 130, summary: "thread aborted" };
|
||||
return await finalizeThreadResult({
|
||||
cas,
|
||||
workflowName,
|
||||
threadId,
|
||||
stepMerkleHashes,
|
||||
completion: { returnCode: 130, summary: "thread aborted" },
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -133,6 +225,7 @@ export async function executeThread(
|
||||
prompt: input.prompt,
|
||||
options: {
|
||||
maxRounds: options.maxRounds,
|
||||
depth: options.depth,
|
||||
},
|
||||
},
|
||||
timestamp: nowMs,
|
||||
@@ -145,37 +238,63 @@ export async function executeThread(
|
||||
|
||||
logger("T9HQ2KHM", `thread ${io.threadId} started for workflow ${workflowName}`);
|
||||
|
||||
const stepMerkleHashes: string[] = [];
|
||||
|
||||
if (prefilled !== null) {
|
||||
for (const row of prefilled) {
|
||||
const prefilledPayload = await getContentMerklePayload(io.cas, row.contentHash);
|
||||
if (prefilledPayload === null) {
|
||||
throw new Error(
|
||||
`prefilled step ${row.role}: CAS blob missing for contentHash ${row.contentHash}`,
|
||||
);
|
||||
}
|
||||
await appendDataLine(io.dataJsonlPath, {
|
||||
role: row.role,
|
||||
content: row.content,
|
||||
contentHash: row.contentHash,
|
||||
meta: row.meta,
|
||||
refs: normalizeRefsField(row.refs),
|
||||
timestamp: row.timestamp,
|
||||
});
|
||||
const stepNodeHash = await putStepMerkleNode(
|
||||
io.cas,
|
||||
{ role: row.role, meta: row.meta },
|
||||
row.contentHash,
|
||||
);
|
||||
stepMerkleHashes.push(stepNodeHash);
|
||||
}
|
||||
}
|
||||
|
||||
if (options.maxRounds <= 0) {
|
||||
logger("R3CW7YBQ", `thread ${io.threadId} stopped at maxRounds=${options.maxRounds}`);
|
||||
return {
|
||||
returnCode: 0,
|
||||
summary: `completed: reached maxRounds (${options.maxRounds})`,
|
||||
};
|
||||
return await finalizeThreadResult({
|
||||
cas: io.cas,
|
||||
workflowName,
|
||||
threadId: io.threadId,
|
||||
stepMerkleHashes,
|
||||
completion: {
|
||||
returnCode: 0,
|
||||
summary: `completed: reached maxRounds (${options.maxRounds})`,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const bundleOptions: WorkflowFnOptions = {
|
||||
threadId: io.threadId,
|
||||
maxRounds: options.maxRounds,
|
||||
depth: options.depth,
|
||||
cas: io.cas,
|
||||
};
|
||||
|
||||
return await driveWorkflowGenerator({
|
||||
fn,
|
||||
workflowName,
|
||||
input,
|
||||
bundleOptions,
|
||||
executeOptions: options,
|
||||
dataJsonlPath: io.dataJsonlPath,
|
||||
threadId: io.threadId,
|
||||
logger,
|
||||
cas: io.cas,
|
||||
stepMerkleHashes,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
import { mkdir, readlink, symlink, unlink } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
/** This module lives in `@uncaged/workflow/src`; parent dir is the package root. */
|
||||
function installedWorkflowPackageDir(): string {
|
||||
return fileURLToPath(new URL("..", import.meta.url));
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensures `<storageRoot>/node_modules/@uncaged/workflow` points at the installed `@uncaged/workflow`
|
||||
* package so workflow bundles loaded from `<storageRoot>/bundles/*.esm.js` can resolve `import "@uncaged/workflow"`.
|
||||
*/
|
||||
export async function ensureUncagedWorkflowSymlink(storageRoot: string): Promise<void> {
|
||||
const target = installedWorkflowPackageDir();
|
||||
const linkDir = path.join(storageRoot, "node_modules", "@uncaged");
|
||||
const linkPath = path.join(linkDir, "workflow");
|
||||
await mkdir(linkDir, { recursive: true });
|
||||
|
||||
try {
|
||||
const existing = await readlink(linkPath);
|
||||
const normalizedExisting = path.resolve(linkDir, existing);
|
||||
if (normalizedExisting === target) {
|
||||
return;
|
||||
}
|
||||
await unlink(linkPath);
|
||||
} catch (e) {
|
||||
const errObj = e as NodeJS.ErrnoException;
|
||||
if (errObj.code !== "ENOENT" && errObj.code !== "EINVAL") {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
const linkType = process.platform === "win32" ? "junction" : "dir";
|
||||
await symlink(target, linkPath, linkType);
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
import { importWorkflowBundleModule } from "./bundle-import-env.js";
|
||||
import { ensureUncagedWorkflowSymlink } from "./ensure-uncaged-workflow-symlink.js";
|
||||
import { err, ok, type Result } from "./result.js";
|
||||
import type { WorkflowFn } from "./types.js";
|
||||
import type { WorkflowDescriptor } from "./workflow-descriptor.js";
|
||||
@@ -10,14 +10,23 @@ export type ExtractedBundleExports = {
|
||||
descriptor: WorkflowDescriptor;
|
||||
};
|
||||
|
||||
export type ExtractBundleExportsOptions = {
|
||||
/** When set, ensures `node_modules/@uncaged/workflow` exists under this root before import. */
|
||||
storageRoot: string | null;
|
||||
};
|
||||
|
||||
/** Load a workflow `.esm.js` bundle and read its named exports (`run`, `descriptor`). */
|
||||
export async function extractBundleExports(
|
||||
bundlePath: string,
|
||||
options: ExtractBundleExportsOptions = { storageRoot: null },
|
||||
): Promise<Result<ExtractedBundleExports, string>> {
|
||||
let modUnknown: unknown;
|
||||
try {
|
||||
if (options.storageRoot !== null) {
|
||||
await ensureUncagedWorkflowSymlink(options.storageRoot);
|
||||
}
|
||||
// Dynamic import required: user bundle path resolved at runtime
|
||||
modUnknown = await import(pathToFileURL(bundlePath).href);
|
||||
modUnknown = await importWorkflowBundleModule(bundlePath);
|
||||
} catch (e) {
|
||||
const message = e instanceof Error ? e.message : String(e);
|
||||
return err(`failed to import bundle: ${message}`);
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import type * as z from "zod/v4";
|
||||
|
||||
import { llmExtractWithRetry } from "./llm-extract.js";
|
||||
import { getContentMerklePayload } from "./merkle.js";
|
||||
import type { ExtractContext, LlmProvider } from "./types.js";
|
||||
|
||||
export type ExtractFn = <T extends Record<string, unknown>>(
|
||||
@@ -9,6 +10,40 @@ export type ExtractFn = <T extends Record<string, unknown>>(
|
||||
ctx: ExtractContext,
|
||||
) => Promise<T>;
|
||||
|
||||
/** Builds the user-side extraction prompt (thread + agent output + instruction). */
|
||||
export async function buildExtractUserContent(
|
||||
ctx: ExtractContext,
|
||||
prompt: string,
|
||||
): Promise<string> {
|
||||
const lines: string[] = [];
|
||||
lines.push(`## Role: ${ctx.currentRole.name}`);
|
||||
lines.push(ctx.currentRole.systemPrompt);
|
||||
lines.push("");
|
||||
lines.push("## Task");
|
||||
lines.push(ctx.start.content);
|
||||
lines.push("");
|
||||
if (ctx.steps.length > 0) {
|
||||
lines.push("## Thread History");
|
||||
for (const step of ctx.steps) {
|
||||
const body = await getContentMerklePayload(ctx.cas, step.contentHash);
|
||||
if (body === null) {
|
||||
throw new Error(`extract: missing CAS blob for step ${step.role}: ${step.contentHash}`);
|
||||
}
|
||||
lines.push(`### ${step.role}`);
|
||||
lines.push(body);
|
||||
lines.push(`Meta: ${JSON.stringify(step.meta)}`);
|
||||
lines.push("");
|
||||
}
|
||||
}
|
||||
lines.push("## Agent Output");
|
||||
lines.push(ctx.agentContent);
|
||||
lines.push("");
|
||||
lines.push("## Extraction Instruction");
|
||||
lines.push(prompt);
|
||||
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Create an ExtractFn backed by an LLM provider.
|
||||
* Builds prompt text from {@link ExtractContext} plus `prompt` and calls structured extraction.
|
||||
@@ -19,29 +54,7 @@ export function createExtract(provider: LlmProvider): ExtractFn {
|
||||
prompt: string,
|
||||
ctx: ExtractContext,
|
||||
): Promise<T> => {
|
||||
const lines: string[] = [];
|
||||
lines.push(`## Role: ${ctx.currentRole.name}`);
|
||||
lines.push(ctx.currentRole.systemPrompt);
|
||||
lines.push("");
|
||||
lines.push("## Task");
|
||||
lines.push(ctx.start.content);
|
||||
lines.push("");
|
||||
if (ctx.steps.length > 0) {
|
||||
lines.push("## Thread History");
|
||||
for (const step of ctx.steps) {
|
||||
lines.push(`### ${step.role}`);
|
||||
lines.push(step.content);
|
||||
lines.push(`Meta: ${JSON.stringify(step.meta)}`);
|
||||
lines.push("");
|
||||
}
|
||||
}
|
||||
lines.push("## Agent Output");
|
||||
lines.push(ctx.agentContent);
|
||||
lines.push("");
|
||||
lines.push("## Extraction Instruction");
|
||||
lines.push(prompt);
|
||||
|
||||
const text = lines.join("\n");
|
||||
const text = await buildExtractUserContent(ctx, prompt);
|
||||
const result = await llmExtractWithRetry({ text, schema, provider });
|
||||
if (!result.ok) {
|
||||
throw new Error(`extract failed: ${JSON.stringify(result.error)}`);
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
import { readWorkflowRegistry } from "./registry.js";
|
||||
import type { WorkflowConfig } from "./registry-types.js";
|
||||
import { err, ok, type Result } from "./result.js";
|
||||
import { getDefaultWorkflowStorageRoot } from "./storage-root.js";
|
||||
import type { LlmProvider } from "./types.js";
|
||||
|
||||
const DEFAULT_WORKFLOW_AS_AGENT_MAX_DEPTH = 3;
|
||||
|
||||
export function getWorkflowAsAgentMaxDepth(config: WorkflowConfig | null): number {
|
||||
if (config === null) {
|
||||
return DEFAULT_WORKFLOW_AS_AGENT_MAX_DEPTH;
|
||||
}
|
||||
return config.maxDepth;
|
||||
}
|
||||
|
||||
/** Loads `config.extract` from workflow.yaml (apiKey already resolved at registry parse time). */
|
||||
export async function getExtractProvider(
|
||||
storageRoot: string | undefined,
|
||||
): Promise<Result<LlmProvider, string>> {
|
||||
const root = storageRoot ?? getDefaultWorkflowStorageRoot();
|
||||
const regResult = await readWorkflowRegistry(root);
|
||||
if (!regResult.ok) {
|
||||
return err(regResult.error.message);
|
||||
}
|
||||
const cfg = regResult.value.config;
|
||||
if (cfg === null) {
|
||||
return err("workflow registry has no global config section");
|
||||
}
|
||||
const ex = cfg.extract;
|
||||
return ok({
|
||||
baseUrl: ex.baseUrl,
|
||||
apiKey: ex.apiKey,
|
||||
model: ex.model,
|
||||
});
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import { normalizeRefsField } from "./refs-field.js";
|
||||
import { err, ok, type Result } from "./result.js";
|
||||
import type { RoleOutput } from "./types.js";
|
||||
|
||||
@@ -10,6 +11,7 @@ export type ParsedThreadStartRecord = {
|
||||
threadId: string;
|
||||
prompt: string;
|
||||
maxRounds: number;
|
||||
depth: number;
|
||||
};
|
||||
|
||||
function parseRoleLine(
|
||||
@@ -17,14 +19,14 @@ function parseRoleLine(
|
||||
lineIndex: number,
|
||||
): Result<ForkHistoricalStep, string> {
|
||||
const role = obj.role;
|
||||
const content = obj.content;
|
||||
const contentHash = obj.contentHash;
|
||||
const meta = obj.meta;
|
||||
const timestamp = obj.timestamp;
|
||||
if (typeof role !== "string") {
|
||||
return err(`invalid role record at line ${lineIndex}: missing role`);
|
||||
}
|
||||
if (typeof content !== "string") {
|
||||
return err(`invalid role record at line ${lineIndex}: missing content`);
|
||||
if (typeof contentHash !== "string") {
|
||||
return err(`invalid role record at line ${lineIndex}: missing contentHash`);
|
||||
}
|
||||
if (meta === null || typeof meta !== "object") {
|
||||
return err(`invalid role record at line ${lineIndex}: missing meta`);
|
||||
@@ -34,8 +36,9 @@ function parseRoleLine(
|
||||
}
|
||||
return ok({
|
||||
role,
|
||||
content,
|
||||
contentHash,
|
||||
meta: meta as Record<string, unknown>,
|
||||
refs: normalizeRefsField(obj.refs),
|
||||
timestamp,
|
||||
});
|
||||
}
|
||||
@@ -76,12 +79,17 @@ function parseStartRecordLine(firstLine: string): Result<ParsedThreadStartRecord
|
||||
return err("start record missing parameters.options.maxRounds");
|
||||
}
|
||||
|
||||
const depthRaw = optRec.depth;
|
||||
const depth =
|
||||
typeof depthRaw === "number" && Number.isFinite(depthRaw) ? Math.trunc(depthRaw) : 0;
|
||||
|
||||
return ok({
|
||||
workflowName: name,
|
||||
hash,
|
||||
threadId,
|
||||
prompt,
|
||||
maxRounds,
|
||||
depth,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -194,7 +202,7 @@ export type ForkPlan = {
|
||||
hash: string;
|
||||
sourceThreadId: string;
|
||||
prompt: string;
|
||||
runOptions: { maxRounds: number };
|
||||
runOptions: { maxRounds: number; depth: number };
|
||||
historicalSteps: ForkHistoricalStep[];
|
||||
};
|
||||
|
||||
@@ -219,7 +227,7 @@ export function buildForkPlan(
|
||||
hash: start.hash,
|
||||
sourceThreadId: start.threadId,
|
||||
prompt: start.prompt,
|
||||
runOptions: { maxRounds: start.maxRounds },
|
||||
runOptions: { maxRounds: start.maxRounds, depth: start.depth },
|
||||
historicalSteps: selected.value,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
import { readdir, readFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { type CasStore, createCasStore } from "./cas.js";
|
||||
import { parseThreadDataJsonl } from "./fork-thread.js";
|
||||
import { err, ok, type Result } from "./result.js";
|
||||
import { getGlobalCasDir } from "./storage-root.js";
|
||||
|
||||
export type GcResult = {
|
||||
scannedThreads: number;
|
||||
activeRefs: number;
|
||||
deletedEntries: number;
|
||||
deletedHashes: string[];
|
||||
};
|
||||
|
||||
async function listThreadDataJsonlPaths(storageRoot: string): Promise<Result<string[], string>> {
|
||||
const logsRoot = join(storageRoot, "logs");
|
||||
const paths: string[] = [];
|
||||
let hashes: string[];
|
||||
try {
|
||||
hashes = await readdir(logsRoot);
|
||||
} catch (e) {
|
||||
const errObj = e as NodeJS.ErrnoException;
|
||||
if (errObj.code === "ENOENT") {
|
||||
return ok([]);
|
||||
}
|
||||
return err(`failed to read logs directory: ${String(e)}`);
|
||||
}
|
||||
|
||||
for (const hash of hashes) {
|
||||
const dir = join(logsRoot, hash);
|
||||
let entries: string[];
|
||||
try {
|
||||
entries = await readdir(dir);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
for (const fileName of entries) {
|
||||
if (fileName.endsWith(".data.jsonl")) {
|
||||
paths.push(join(dir, fileName));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
paths.sort();
|
||||
return ok(paths);
|
||||
}
|
||||
|
||||
async function collectActiveRefsFromDataPaths(
|
||||
dataPaths: string[],
|
||||
): Promise<Result<Set<string>, string>> {
|
||||
const activeRefs = new Set<string>();
|
||||
for (const dataPath of dataPaths) {
|
||||
let text: string;
|
||||
try {
|
||||
text = await readFile(dataPath, "utf8");
|
||||
} catch (e) {
|
||||
return err(`failed to read ${dataPath}: ${String(e)}`);
|
||||
}
|
||||
const parsed = parseThreadDataJsonl(text);
|
||||
if (!parsed.ok) {
|
||||
return err(`${dataPath}: ${parsed.error}`);
|
||||
}
|
||||
for (const step of parsed.value.roleSteps) {
|
||||
for (const ref of step.refs) {
|
||||
activeRefs.add(ref);
|
||||
}
|
||||
}
|
||||
}
|
||||
return ok(activeRefs);
|
||||
}
|
||||
|
||||
async function deleteCasNotInSet(
|
||||
cas: CasStore,
|
||||
activeRefs: Set<string>,
|
||||
): Promise<Result<string[], string>> {
|
||||
let listed: string[];
|
||||
try {
|
||||
listed = await cas.list();
|
||||
} catch (e) {
|
||||
return err(`failed to list cas entries: ${String(e)}`);
|
||||
}
|
||||
|
||||
const deletedHashes: string[] = [];
|
||||
for (const hash of listed) {
|
||||
if (activeRefs.has(hash)) {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
await cas.delete(hash);
|
||||
} catch (e) {
|
||||
return err(`failed to delete cas ${hash}: ${String(e)}`);
|
||||
}
|
||||
deletedHashes.push(hash);
|
||||
}
|
||||
|
||||
deletedHashes.sort();
|
||||
return ok(deletedHashes);
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark-and-sweep CAS GC: collect `refs` from all thread `.data.jsonl` files under `storageRoot`,
|
||||
* then delete CAS blobs not referenced by any surviving thread data.
|
||||
*/
|
||||
export async function garbageCollectCas(storageRoot: string): Promise<Result<GcResult, string>> {
|
||||
const pathsResult = await listThreadDataJsonlPaths(storageRoot);
|
||||
if (!pathsResult.ok) {
|
||||
return pathsResult;
|
||||
}
|
||||
const paths = pathsResult.value;
|
||||
|
||||
const refsResult = await collectActiveRefsFromDataPaths(paths);
|
||||
if (!refsResult.ok) {
|
||||
return refsResult;
|
||||
}
|
||||
const activeRefs = refsResult.value;
|
||||
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
const deletedResult = await deleteCasNotInSet(cas, activeRefs);
|
||||
if (!deletedResult.ok) {
|
||||
return deletedResult;
|
||||
}
|
||||
const deletedHashes = deletedResult.value;
|
||||
|
||||
return ok({
|
||||
scannedThreads: paths.length,
|
||||
activeRefs: activeRefs.size,
|
||||
deletedEntries: deletedHashes.length,
|
||||
deletedHashes,
|
||||
});
|
||||
}
|
||||
@@ -7,7 +7,7 @@ export {
|
||||
} from "./base32.js";
|
||||
export { buildDescriptor } from "./build-descriptor.js";
|
||||
export { validateWorkflowBundle, type WorkflowBundleValidationInput } from "./bundle-validator.js";
|
||||
export { type CasStore, createThreadCas } from "./cas.js";
|
||||
export { type CasStore, createCasStore, createThreadCas } from "./cas.js";
|
||||
export { createWorkflow } from "./create-workflow.js";
|
||||
export {
|
||||
type ExecuteThreadIo,
|
||||
@@ -17,6 +17,7 @@ export {
|
||||
} from "./engine.js";
|
||||
export { type ExtractedBundleExports, extractBundleExports } from "./extract-bundle-exports.js";
|
||||
export { createExtract, type ExtractFn } from "./extract-fn.js";
|
||||
export { getExtractProvider } from "./extract-provider.js";
|
||||
export {
|
||||
buildForkPlan,
|
||||
type ForkHistoricalStep,
|
||||
@@ -25,6 +26,7 @@ export {
|
||||
parseThreadDataJsonl,
|
||||
selectForkHistoricalSteps,
|
||||
} from "./fork-thread.js";
|
||||
export { type GcResult, garbageCollectCas } from "./gc.js";
|
||||
export { stringifyWorkflowDescriptor } from "./generate-descriptor.js";
|
||||
export { hashString, hashWorkflowBundleBytes } from "./hash.js";
|
||||
export {
|
||||
@@ -40,6 +42,21 @@ export {
|
||||
type LoggerSink,
|
||||
} from "./logger.js";
|
||||
export {
|
||||
createContentMerkleNode,
|
||||
getContentMerklePayload,
|
||||
type MerkleNode,
|
||||
type MerkleNodeType,
|
||||
parseMerkleNode,
|
||||
putContentMerkleNode,
|
||||
putStepMerkleNode,
|
||||
putThreadMerkleNode,
|
||||
type StepMerklePayload,
|
||||
serializeMerkleNode,
|
||||
type ThreadMerklePayload,
|
||||
} from "./merkle.js";
|
||||
export { type ReactExtractArgs, reactExtract } from "./react-extract.js";
|
||||
export {
|
||||
type ExtractProviderConfig,
|
||||
getRegisteredWorkflow,
|
||||
listRegisteredWorkflowNames,
|
||||
parseWorkflowRegistryYaml,
|
||||
@@ -48,6 +65,7 @@ export {
|
||||
rollbackWorkflowToHistoryHash,
|
||||
stringifyWorkflowRegistryYaml,
|
||||
unregisterWorkflow,
|
||||
type WorkflowConfig,
|
||||
type WorkflowHistoryEntry,
|
||||
type WorkflowRegistryEntry,
|
||||
type WorkflowRegistryFile,
|
||||
@@ -55,7 +73,7 @@ export {
|
||||
writeWorkflowRegistry,
|
||||
} from "./registry.js";
|
||||
export { err, ok, type Result } from "./result.js";
|
||||
export { getDefaultWorkflowStorageRoot } from "./storage-root.js";
|
||||
export { getDefaultWorkflowStorageRoot, getGlobalCasDir } from "./storage-root.js";
|
||||
export { createThreadPauseGate, type ThreadPauseGate } from "./thread-pause-gate.js";
|
||||
export {
|
||||
type AgentBinding,
|
||||
@@ -63,6 +81,7 @@ export {
|
||||
type AgentFn,
|
||||
END,
|
||||
type ExtractContext,
|
||||
type ExtractMode,
|
||||
type LlmProvider,
|
||||
type Moderator,
|
||||
type ModeratorContext,
|
||||
@@ -74,6 +93,7 @@ export {
|
||||
type StartStep,
|
||||
type ThreadContext,
|
||||
type ThreadInput,
|
||||
type WorkflowCompletion,
|
||||
type WorkflowDefinition,
|
||||
type WorkflowFn,
|
||||
type WorkflowFnOptions,
|
||||
@@ -81,6 +101,7 @@ export {
|
||||
} from "./types.js";
|
||||
export { generateUlid } from "./ulid.js";
|
||||
export { getWorkerHostScriptPath } from "./worker-entry-path.js";
|
||||
export { type WorkflowAsAgentOptions, workflowAsAgent } from "./workflow-as-agent.js";
|
||||
export {
|
||||
validateWorkflowDescriptor,
|
||||
type WorkflowDescriptor,
|
||||
|
||||
@@ -47,6 +47,21 @@ function readToolDescription(parametersSchema: Record<string, unknown>): string
|
||||
return "Extract structured data from the input text.";
|
||||
}
|
||||
|
||||
/** Builds OpenAI function-tool metadata from a Zod meta schema (same naming rules as single-shot extract). */
|
||||
export function extractFunctionToolFromZodSchema(schema: z.ZodType<unknown>): {
|
||||
name: string;
|
||||
description: string;
|
||||
parameters: Record<string, unknown>;
|
||||
} {
|
||||
const rawJsonSchema = z.toJSONSchema(schema) as Record<string, unknown>;
|
||||
const parameters = stripJsonSchemaMeta(rawJsonSchema);
|
||||
return {
|
||||
name: readToolName(parameters),
|
||||
description: readToolDescription(parameters),
|
||||
parameters,
|
||||
};
|
||||
}
|
||||
|
||||
function readToolArgumentsJson(parsed: unknown, previewSource: string): Result<string, LlmError> {
|
||||
if (!isRecord(parsed)) {
|
||||
return err({ kind: "invalid_response_json", message: "Top-level JSON is not an object" });
|
||||
@@ -124,10 +139,7 @@ export function llmErrorToCause(error: LlmError): Error {
|
||||
async function performLlmExtract<T>(
|
||||
options: LlmExtractArgs<T> & { userContent: string },
|
||||
): Promise<Result<T, LlmError>> {
|
||||
const rawJsonSchema = z.toJSONSchema(options.schema) as Record<string, unknown>;
|
||||
const parameters = stripJsonSchemaMeta(rawJsonSchema);
|
||||
const toolName = readToolName(parameters);
|
||||
const toolDescription = readToolDescription(parameters);
|
||||
const extractTool = extractFunctionToolFromZodSchema(options.schema);
|
||||
|
||||
const body = {
|
||||
model: options.provider.model,
|
||||
@@ -142,13 +154,13 @@ async function performLlmExtract<T>(
|
||||
{
|
||||
type: "function" as const,
|
||||
function: {
|
||||
name: toolName,
|
||||
description: toolDescription,
|
||||
parameters,
|
||||
name: extractTool.name,
|
||||
description: extractTool.description,
|
||||
parameters: extractTool.parameters,
|
||||
},
|
||||
},
|
||||
],
|
||||
tool_choice: { type: "function" as const, function: { name: toolName } },
|
||||
tool_choice: { type: "function" as const, function: { name: extractTool.name } },
|
||||
};
|
||||
|
||||
let response: Response;
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
import { parse, stringify } from "yaml";
|
||||
|
||||
import type { CasStore } from "./cas.js";
|
||||
|
||||
export type MerkleNodeType = "content" | "step" | "thread";
|
||||
|
||||
export type MerkleNode = {
|
||||
type: MerkleNodeType;
|
||||
payload: string | Record<string, unknown>;
|
||||
children: string[];
|
||||
};
|
||||
|
||||
export function serializeMerkleNode(node: MerkleNode): string {
|
||||
return stringify(
|
||||
{ type: node.type, payload: node.payload, children: node.children },
|
||||
{ indent: 2 },
|
||||
);
|
||||
}
|
||||
|
||||
export function parseMerkleNode(yamlText: string): MerkleNode {
|
||||
const raw = parse(yamlText) as unknown;
|
||||
if (raw === null || typeof raw !== "object") {
|
||||
throw new Error("merkle: YAML root must be an object");
|
||||
}
|
||||
const rec = raw as Record<string, unknown>;
|
||||
const type = rec.type;
|
||||
const payload = rec.payload;
|
||||
const children = rec.children;
|
||||
if (type !== "content" && type !== "step" && type !== "thread") {
|
||||
throw new Error("merkle: invalid or missing type");
|
||||
}
|
||||
if (typeof payload !== "string" && (payload === null || typeof payload !== "object")) {
|
||||
throw new Error("merkle: payload must be a string or object");
|
||||
}
|
||||
if (!Array.isArray(children)) {
|
||||
throw new Error("merkle: children must be an array");
|
||||
}
|
||||
const childHashes: string[] = [];
|
||||
for (const c of children) {
|
||||
if (typeof c !== "string") {
|
||||
throw new Error("merkle: child hash must be a string");
|
||||
}
|
||||
childHashes.push(c);
|
||||
}
|
||||
return {
|
||||
type,
|
||||
payload: typeof payload === "string" ? payload : (payload as Record<string, unknown>),
|
||||
children: childHashes,
|
||||
};
|
||||
}
|
||||
|
||||
export function createContentMerkleNode(payload: string): MerkleNode {
|
||||
return { type: "content", payload, children: [] };
|
||||
}
|
||||
|
||||
export type StepMerklePayload = {
|
||||
role: string;
|
||||
meta: Record<string, unknown>;
|
||||
};
|
||||
|
||||
export type ThreadMerklePayload = {
|
||||
workflow: string;
|
||||
threadId: string;
|
||||
result: {
|
||||
returnCode: number;
|
||||
summary: string;
|
||||
};
|
||||
};
|
||||
|
||||
/** Serializes a step Merkle node (role + meta + content child) and stores it in CAS. */
|
||||
export async function putStepMerkleNode(
|
||||
store: CasStore,
|
||||
payload: StepMerklePayload,
|
||||
contentHash: string,
|
||||
): Promise<string> {
|
||||
const node: MerkleNode = {
|
||||
type: "step",
|
||||
payload: { role: payload.role, meta: payload.meta },
|
||||
children: [contentHash],
|
||||
};
|
||||
return store.put(serializeMerkleNode(node));
|
||||
}
|
||||
|
||||
/** Serializes the thread root Merkle node and stores it in CAS. */
|
||||
export async function putThreadMerkleNode(
|
||||
store: CasStore,
|
||||
payload: ThreadMerklePayload,
|
||||
stepHashes: readonly string[],
|
||||
): Promise<string> {
|
||||
const node: MerkleNode = {
|
||||
type: "thread",
|
||||
payload: {
|
||||
workflow: payload.workflow,
|
||||
threadId: payload.threadId,
|
||||
result: payload.result,
|
||||
},
|
||||
children: [...stepHashes],
|
||||
};
|
||||
return store.put(serializeMerkleNode(node));
|
||||
}
|
||||
|
||||
/** Serializes a content Merkle node and stores it in CAS; returns its hash. */
|
||||
export async function putContentMerkleNode(store: CasStore, content: string): Promise<string> {
|
||||
const yamlText = serializeMerkleNode(createContentMerkleNode(content));
|
||||
return store.put(yamlText);
|
||||
}
|
||||
|
||||
/** Loads a CAS blob and returns the payload string for a `content` Merkle node. */
|
||||
export async function getContentMerklePayload(
|
||||
store: CasStore,
|
||||
hash: string,
|
||||
): Promise<string | null> {
|
||||
const yamlText = await store.get(hash);
|
||||
if (yamlText === null) {
|
||||
return null;
|
||||
}
|
||||
const node = parseMerkleNode(yamlText);
|
||||
if (node.type !== "content" || typeof node.payload !== "string") {
|
||||
return null;
|
||||
}
|
||||
return node.payload;
|
||||
}
|
||||
@@ -0,0 +1,330 @@
|
||||
import type * as z from "zod/v4";
|
||||
|
||||
import type { CasStore } from "./cas.js";
|
||||
import { extractFunctionToolFromZodSchema } from "./llm-extract.js";
|
||||
import { err, ok, type Result } from "./result.js";
|
||||
import type { LlmProvider } from "./types.js";
|
||||
|
||||
export type ReactExtractArgs<T extends Record<string, unknown>> = {
|
||||
text: string;
|
||||
schema: z.ZodType<T>;
|
||||
provider: LlmProvider;
|
||||
cas: CasStore;
|
||||
};
|
||||
|
||||
const MAX_REACT_ROUNDS = 10;
|
||||
|
||||
const CAS_GET_TOOL_DEFINITION = {
|
||||
type: "function" as const,
|
||||
function: {
|
||||
name: "cas_get",
|
||||
description:
|
||||
"Read a Merkle DAG node from content-addressed storage by its hash. Returns YAML-formatted node with type, payload, and children fields.",
|
||||
parameters: {
|
||||
type: "object",
|
||||
properties: {
|
||||
hash: { type: "string", description: "The CAS hash to retrieve" },
|
||||
},
|
||||
required: ["hash"],
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
function chatCompletionsUrl(baseUrl: string): string {
|
||||
const trimmed = baseUrl.replace(/\/+$/, "");
|
||||
return `${trimmed}/chat/completions`;
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function tryParseJsonContent(content: string): unknown | null {
|
||||
const trimmed = content.trim();
|
||||
const fenceMatch = /^```(?:json)?\s*([\s\S]*?)```$/m.exec(trimmed);
|
||||
const payload = fenceMatch !== null ? fenceMatch[1].trim() : trimmed;
|
||||
try {
|
||||
return JSON.parse(payload) as unknown;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
type ToolCall = {
|
||||
id: string;
|
||||
type: "function";
|
||||
function: { name: string; arguments: string };
|
||||
};
|
||||
|
||||
type ChatMessage =
|
||||
| { role: "system"; content: string }
|
||||
| { role: "user"; content: string }
|
||||
| {
|
||||
role: "assistant";
|
||||
content: string | null;
|
||||
tool_calls: ToolCall[];
|
||||
}
|
||||
| { role: "tool"; tool_call_id: string; content: string };
|
||||
|
||||
type AssistantTurn<T> =
|
||||
| { kind: "plain_json"; value: T }
|
||||
| { kind: "tool_calls"; calls: ToolCall[]; assistantContent: string | null };
|
||||
|
||||
function firstAssistantMessage(responseText: string): Result<Record<string, unknown>, string> {
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(responseText) as unknown;
|
||||
} catch (cause) {
|
||||
const message = cause instanceof Error ? cause.message : String(cause);
|
||||
return err(`invalid_response_json:${message}`);
|
||||
}
|
||||
if (!isRecord(parsed)) {
|
||||
return err("invalid_response_top_level");
|
||||
}
|
||||
const choices = parsed.choices;
|
||||
if (!Array.isArray(choices) || choices.length === 0) {
|
||||
return err("no_choices_in_response");
|
||||
}
|
||||
const firstChoice = choices[0];
|
||||
if (!isRecord(firstChoice)) {
|
||||
return err("invalid_choice");
|
||||
}
|
||||
const messageObj = firstChoice.message;
|
||||
if (!isRecord(messageObj)) {
|
||||
return err("invalid_message");
|
||||
}
|
||||
return ok(messageObj);
|
||||
}
|
||||
|
||||
function normalizeToolCalls(toolCallsRaw: unknown[]): Result<ToolCall[], string> {
|
||||
const toolCalls: ToolCall[] = [];
|
||||
for (const tc of toolCallsRaw) {
|
||||
if (!isRecord(tc)) {
|
||||
return err("invalid_tool_call");
|
||||
}
|
||||
const id = tc.id;
|
||||
const tcType = tc.type;
|
||||
const fn = tc.function;
|
||||
if (typeof id !== "string" || tcType !== "function" || !isRecord(fn)) {
|
||||
return err("invalid_tool_call_shape");
|
||||
}
|
||||
const name = fn.name;
|
||||
const argumentsStr = fn.arguments;
|
||||
if (typeof name !== "string" || typeof argumentsStr !== "string") {
|
||||
return err("invalid_tool_call_function");
|
||||
}
|
||||
toolCalls.push({ id, type: "function", function: { name, arguments: argumentsStr } });
|
||||
}
|
||||
return ok(toolCalls);
|
||||
}
|
||||
|
||||
function classifyAssistantTurn<T extends Record<string, unknown>>(
|
||||
messageObj: Record<string, unknown>,
|
||||
schema: z.ZodType<T>,
|
||||
): Result<AssistantTurn<T>, string> {
|
||||
const toolCallsRaw = messageObj.tool_calls;
|
||||
if (!Array.isArray(toolCallsRaw) || toolCallsRaw.length === 0) {
|
||||
const content = messageObj.content;
|
||||
if (typeof content !== "string") {
|
||||
return err("no_tool_calls_and_no_string_content");
|
||||
}
|
||||
const jsonParsed = tryParseJsonContent(content);
|
||||
if (jsonParsed === null) {
|
||||
return err("no_tool_calls_and_content_not_json");
|
||||
}
|
||||
const validated = schema.safeParse(jsonParsed);
|
||||
if (!validated.success) {
|
||||
return err(`schema_validation_failed:${validated.error.message}`);
|
||||
}
|
||||
return ok({ kind: "plain_json", value: validated.data });
|
||||
}
|
||||
const callsResult = normalizeToolCalls(toolCallsRaw);
|
||||
if (!callsResult.ok) {
|
||||
return err(callsResult.error);
|
||||
}
|
||||
const assistantContent = messageObj.content;
|
||||
return ok({
|
||||
kind: "tool_calls",
|
||||
calls: callsResult.value,
|
||||
assistantContent: typeof assistantContent === "string" ? assistantContent : null,
|
||||
});
|
||||
}
|
||||
|
||||
async function appendCasGetToolResult(
|
||||
tc: ToolCall,
|
||||
cas: CasStore,
|
||||
messages: ChatMessage[],
|
||||
): Promise<Result<null, string>> {
|
||||
let hash: string;
|
||||
try {
|
||||
const ta = JSON.parse(tc.function.arguments) as unknown;
|
||||
if (!isRecord(ta) || typeof ta.hash !== "string") {
|
||||
return err("cas_get_invalid_arguments");
|
||||
}
|
||||
hash = ta.hash;
|
||||
} catch {
|
||||
return err("cas_get_arguments_not_json");
|
||||
}
|
||||
const blob = await cas.get(hash);
|
||||
const toolContent = blob === null ? "null" : blob;
|
||||
messages.push({
|
||||
role: "tool",
|
||||
tool_call_id: tc.id,
|
||||
content: toolContent,
|
||||
});
|
||||
return ok(null);
|
||||
}
|
||||
|
||||
async function appendExtractToolResult<T extends Record<string, unknown>>(
|
||||
tc: ToolCall,
|
||||
schema: z.ZodType<T>,
|
||||
messages: ChatMessage[],
|
||||
): Promise<Result<T, string>> {
|
||||
let parsedArgs: unknown;
|
||||
try {
|
||||
parsedArgs = JSON.parse(tc.function.arguments) as unknown;
|
||||
} catch {
|
||||
return err("extract_tool_arguments_not_json");
|
||||
}
|
||||
const validated = schema.safeParse(parsedArgs);
|
||||
if (!validated.success) {
|
||||
return err(`schema_validation_failed:${validated.error.message}`);
|
||||
}
|
||||
messages.push({
|
||||
role: "tool",
|
||||
tool_call_id: tc.id,
|
||||
content: '{"ok":true}',
|
||||
});
|
||||
return ok(validated.data);
|
||||
}
|
||||
|
||||
async function appendToolResults<T extends Record<string, unknown>>(
|
||||
toolCalls: ToolCall[],
|
||||
extractToolName: string,
|
||||
schema: z.ZodType<T>,
|
||||
cas: CasStore,
|
||||
messages: ChatMessage[],
|
||||
): Promise<Result<T | null, string>> {
|
||||
let extracted: T | null = null;
|
||||
for (const tc of toolCalls) {
|
||||
if (tc.function.name === "cas_get") {
|
||||
const casRes = await appendCasGetToolResult(tc, cas, messages);
|
||||
if (!casRes.ok) {
|
||||
return casRes;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (tc.function.name === extractToolName) {
|
||||
const exRes = await appendExtractToolResult(tc, schema, messages);
|
||||
if (!exRes.ok) {
|
||||
return exRes;
|
||||
}
|
||||
extracted = exRes.value;
|
||||
continue;
|
||||
}
|
||||
return err(`unknown_tool:${tc.function.name}`);
|
||||
}
|
||||
return ok(extracted);
|
||||
}
|
||||
|
||||
async function postChatCompletion(
|
||||
provider: LlmProvider,
|
||||
messages: ChatMessage[],
|
||||
tools: readonly Record<string, unknown>[],
|
||||
): Promise<Result<string, string>> {
|
||||
try {
|
||||
const response = await fetch(chatCompletionsUrl(provider.baseUrl), {
|
||||
method: "POST",
|
||||
headers: {
|
||||
Authorization: `Bearer ${provider.apiKey}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: provider.model,
|
||||
messages,
|
||||
tools,
|
||||
tool_choice: "auto",
|
||||
}),
|
||||
});
|
||||
const responseText = await response.text();
|
||||
if (!response.ok) {
|
||||
return err(`http_error:${String(response.status)}:${responseText.slice(0, 4000)}`);
|
||||
}
|
||||
return ok(responseText);
|
||||
} catch (cause) {
|
||||
const message = cause instanceof Error ? cause.message : String(cause);
|
||||
return err(`network_error:${message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Multi-turn ReAct extraction with `cas_get` plus a schema-shaped extract tool (OpenAI-compatible).
|
||||
* Final meta comes from a successful extract tool call or from plain JSON in the assistant message.
|
||||
*/
|
||||
export async function reactExtract<T extends Record<string, unknown>>(
|
||||
args: ReactExtractArgs<T>,
|
||||
): Promise<Result<T, string>> {
|
||||
const extractTool = extractFunctionToolFromZodSchema(args.schema);
|
||||
const tools = [
|
||||
CAS_GET_TOOL_DEFINITION,
|
||||
{
|
||||
type: "function" as const,
|
||||
function: {
|
||||
name: extractTool.name,
|
||||
description: extractTool.description,
|
||||
parameters: extractTool.parameters,
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
const systemContent = `You extract structured metadata from the agent output below. Use cas_get to read Merkle DAG nodes from CAS (YAML: type, payload, children) when the agent output references hashes you must traverse. When you have the complete structured object, call the ${extractTool.name} tool with JSON arguments matching the schema. You may instead reply with only a JSON object (no prose) when no tools are needed.`;
|
||||
|
||||
const messages: ChatMessage[] = [
|
||||
{ role: "system", content: systemContent },
|
||||
{ role: "user", content: args.text },
|
||||
];
|
||||
|
||||
for (let round = 0; round < MAX_REACT_ROUNDS; round++) {
|
||||
const bodyResult = await postChatCompletion(args.provider, messages, tools);
|
||||
if (!bodyResult.ok) {
|
||||
return bodyResult;
|
||||
}
|
||||
|
||||
const msgResult = firstAssistantMessage(bodyResult.value);
|
||||
if (!msgResult.ok) {
|
||||
return msgResult;
|
||||
}
|
||||
|
||||
const classified = classifyAssistantTurn(msgResult.value, args.schema);
|
||||
if (!classified.ok) {
|
||||
return classified;
|
||||
}
|
||||
|
||||
const turn = classified.value;
|
||||
if (turn.kind === "plain_json") {
|
||||
return ok(turn.value);
|
||||
}
|
||||
|
||||
messages.push({
|
||||
role: "assistant",
|
||||
content: turn.assistantContent,
|
||||
tool_calls: turn.calls,
|
||||
});
|
||||
|
||||
const toolsRound = await appendToolResults(
|
||||
turn.calls,
|
||||
extractTool.name,
|
||||
args.schema,
|
||||
args.cas,
|
||||
messages,
|
||||
);
|
||||
if (!toolsRound.ok) {
|
||||
return toolsRound;
|
||||
}
|
||||
if (toolsRound.value !== null) {
|
||||
return ok(toolsRound.value);
|
||||
}
|
||||
}
|
||||
|
||||
return err("max_react_rounds_exceeded");
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
/** Append `contentHash` to `refs` when not already present (dedupe by first occurrence order). */
|
||||
export function mergeRefsWithContentHash(refs: string[], contentHash: string): string[] {
|
||||
const out = [...refs];
|
||||
if (!out.includes(contentHash)) {
|
||||
out.push(contentHash);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Normalize `refs` from persisted JSONL or IPC payloads (missing or invalid → []). */
|
||||
export function normalizeRefsField(value: unknown): string[] {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
const out: string[] = [];
|
||||
for (const x of value) {
|
||||
if (typeof x === "string") {
|
||||
out.push(x);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
@@ -1,10 +1,68 @@
|
||||
import type {
|
||||
ExtractProviderConfig,
|
||||
WorkflowConfig,
|
||||
WorkflowHistoryEntry,
|
||||
WorkflowRegistryEntry,
|
||||
WorkflowRegistryFile,
|
||||
} from "./registry-types.js";
|
||||
import { err, ok, type Result } from "./result.js";
|
||||
|
||||
function resolveRegistryApiKey(raw: string): Result<string, Error> {
|
||||
if (raw.startsWith("env:")) {
|
||||
const name = raw.slice("env:".length);
|
||||
if (name === "") {
|
||||
return err(new Error('config.extract.apiKey "env:" reference must name a variable'));
|
||||
}
|
||||
const value = process.env[name];
|
||||
if (value === undefined) {
|
||||
return err(new Error(`config.extract.apiKey: environment variable "${name}" is not set`));
|
||||
}
|
||||
return ok(value);
|
||||
}
|
||||
return ok(raw);
|
||||
}
|
||||
|
||||
function normalizeExtractProviderConfig(raw: unknown): Result<ExtractProviderConfig, Error> {
|
||||
if (raw === null || typeof raw !== "object") {
|
||||
return err(new Error('registry config must contain an "extract" mapping'));
|
||||
}
|
||||
const e = raw as Record<string, unknown>;
|
||||
const baseUrl = e.baseUrl;
|
||||
const model = e.model;
|
||||
const apiKeyRaw = e.apiKey;
|
||||
if (typeof baseUrl !== "string" || baseUrl === "") {
|
||||
return err(new Error("config.extract.baseUrl must be a non-empty string"));
|
||||
}
|
||||
if (typeof model !== "string" || model === "") {
|
||||
return err(new Error("config.extract.model must be a non-empty string"));
|
||||
}
|
||||
if (typeof apiKeyRaw !== "string" || apiKeyRaw === "") {
|
||||
return err(new Error("config.extract.apiKey must be a non-empty string"));
|
||||
}
|
||||
const apiKeyResult = resolveRegistryApiKey(apiKeyRaw);
|
||||
if (!apiKeyResult.ok) {
|
||||
return apiKeyResult;
|
||||
}
|
||||
return ok({ baseUrl, model, apiKey: apiKeyResult.value });
|
||||
}
|
||||
|
||||
function normalizeWorkflowConfig(raw: unknown): Result<WorkflowConfig, Error> {
|
||||
if (raw === null || typeof raw !== "object") {
|
||||
return err(new Error('registry "config" must be a mapping'));
|
||||
}
|
||||
const c = raw as Record<string, unknown>;
|
||||
const maxDepth = c.maxDepth;
|
||||
const extractRaw = c.extract;
|
||||
if (typeof maxDepth !== "number" || !Number.isInteger(maxDepth) || maxDepth < 0) {
|
||||
return err(new Error("config.maxDepth must be a non-negative integer"));
|
||||
}
|
||||
const extractResult = normalizeExtractProviderConfig(extractRaw);
|
||||
if (!extractResult.ok) {
|
||||
return extractResult;
|
||||
}
|
||||
return ok({ maxDepth, extract: extractResult.value });
|
||||
}
|
||||
|
||||
export function normalizeWorkflowHistoryEntry(
|
||||
workflowName: string,
|
||||
index: number,
|
||||
@@ -61,6 +119,15 @@ export function normalizeWorkflowRegistryRoot(raw: unknown): Result<WorkflowRegi
|
||||
return err(new Error("registry root must be a mapping"));
|
||||
}
|
||||
const root = raw as Record<string, unknown>;
|
||||
const configRaw = root.config;
|
||||
let config: WorkflowConfig | null = null;
|
||||
if (configRaw !== undefined && configRaw !== null) {
|
||||
const configResult = normalizeWorkflowConfig(configRaw);
|
||||
if (!configResult.ok) {
|
||||
return configResult;
|
||||
}
|
||||
config = configResult.value;
|
||||
}
|
||||
const workflowsRaw = root.workflows;
|
||||
if (workflowsRaw === null || workflowsRaw === undefined || typeof workflowsRaw !== "object") {
|
||||
return err(new Error('registry must contain a "workflows" mapping'));
|
||||
@@ -73,5 +140,5 @@ export function normalizeWorkflowRegistryRoot(raw: unknown): Result<WorkflowRegi
|
||||
}
|
||||
workflows[name] = entryResult.value;
|
||||
}
|
||||
return ok({ workflows });
|
||||
return ok({ config, workflows });
|
||||
}
|
||||
|
||||
@@ -9,6 +9,19 @@ export type WorkflowRegistryEntry = {
|
||||
history: WorkflowHistoryEntry[];
|
||||
};
|
||||
|
||||
/** LLM provider settings under `config.extract` in workflow.yaml (apiKey resolved after parse). */
|
||||
export type ExtractProviderConfig = {
|
||||
baseUrl: string;
|
||||
model: string;
|
||||
apiKey: string;
|
||||
};
|
||||
|
||||
export type WorkflowConfig = {
|
||||
maxDepth: number;
|
||||
extract: ExtractProviderConfig;
|
||||
};
|
||||
|
||||
export type WorkflowRegistryFile = {
|
||||
config: WorkflowConfig | null;
|
||||
workflows: Record<string, WorkflowRegistryEntry>;
|
||||
};
|
||||
|
||||
@@ -12,6 +12,8 @@ import type {
|
||||
import { err, ok, type Result } from "./result.js";
|
||||
|
||||
export type {
|
||||
ExtractProviderConfig,
|
||||
WorkflowConfig,
|
||||
WorkflowHistoryEntry,
|
||||
WorkflowRegistryEntry,
|
||||
WorkflowRegistryFile,
|
||||
@@ -22,7 +24,7 @@ export function workflowRegistryPath(storageRoot: string): string {
|
||||
}
|
||||
|
||||
function emptyRegistry(): WorkflowRegistryFile {
|
||||
return { workflows: {} };
|
||||
return { config: null, workflows: {} };
|
||||
}
|
||||
|
||||
export function parseWorkflowRegistryYaml(text: string): Result<WorkflowRegistryFile, Error> {
|
||||
@@ -103,6 +105,7 @@ export function registerWorkflowVersion(
|
||||
: [{ hash: prev.hash, timestamp: prev.timestamp }, ...baseHistory];
|
||||
const next: WorkflowRegistryEntry = { hash, timestamp, history };
|
||||
return {
|
||||
config: registry.config,
|
||||
workflows: { ...registry.workflows, [name]: next },
|
||||
};
|
||||
}
|
||||
@@ -150,5 +153,5 @@ export function unregisterWorkflow(
|
||||
return err(new Error(`workflow not registered: ${name}`));
|
||||
}
|
||||
const { [name]: _removed, ...rest } = registry.workflows;
|
||||
return ok({ workflows: rest });
|
||||
return ok({ config: registry.config, workflows: rest });
|
||||
}
|
||||
|
||||
@@ -5,3 +5,9 @@ import { join } from "node:path";
|
||||
export function getDefaultWorkflowStorageRoot(): string {
|
||||
return join(homedir(), ".uncaged", "workflow");
|
||||
}
|
||||
|
||||
/** Global content-addressed store directory under the workflow storage root (`<root>/cas`). */
|
||||
export function getGlobalCasDir(storageRoot: string | undefined): string {
|
||||
const root = storageRoot ?? getDefaultWorkflowStorageRoot();
|
||||
return join(root, "cas");
|
||||
}
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
import type * as z from "zod/v4";
|
||||
|
||||
import type { CasStore } from "./cas.js";
|
||||
|
||||
/** Sentinel values for automaton control flow. */
|
||||
export const START = "__start__" as const;
|
||||
export const END = "__end__" as const;
|
||||
@@ -14,19 +16,30 @@ export type LlmProvider = {
|
||||
model: string;
|
||||
};
|
||||
|
||||
/** How the engine runs meta extraction for a role after the agent phase. */
|
||||
export type ExtractMode = "single" | "react";
|
||||
|
||||
/** What each generator yield produces — one role's output (engine adds `timestamp` when persisting). */
|
||||
export type RoleOutput = {
|
||||
role: string;
|
||||
content: string;
|
||||
/** CAS hash of the serialized Merkle content node for this step's body text. */
|
||||
contentHash: string;
|
||||
meta: Record<string, unknown>;
|
||||
/** CAS hashes produced or consumed by this step (for GC traceability). */
|
||||
refs: string[];
|
||||
};
|
||||
|
||||
/** What the workflow AsyncGenerator returns when done. */
|
||||
export type WorkflowResult = {
|
||||
/** Generator completion value from a workflow bundle (`run` export). Root hash is added by the engine. */
|
||||
export type WorkflowCompletion = {
|
||||
returnCode: number;
|
||||
summary: string;
|
||||
};
|
||||
|
||||
/** Final thread outcome from {@link executeThread}, including Merkle thread root CAS hash. */
|
||||
export type WorkflowResult = WorkflowCompletion & {
|
||||
rootHash: string;
|
||||
};
|
||||
|
||||
/** Input to a workflow — prompt plus optional historical steps for fork/resume. */
|
||||
export type ThreadInput = {
|
||||
prompt: string;
|
||||
@@ -37,13 +50,17 @@ export type ThreadInput = {
|
||||
export type WorkflowFnOptions = {
|
||||
threadId: string;
|
||||
maxRounds: number;
|
||||
/** Nesting depth for workflow-as-agent chains; root threads use `0`. */
|
||||
depth: number;
|
||||
/** Global CAS store for Merkle content blobs (role step bodies). */
|
||||
cas: CasStore;
|
||||
};
|
||||
|
||||
/** Bundle contract — named export `run` is a function returning an AsyncGenerator. */
|
||||
export type WorkflowFn = (
|
||||
input: ThreadInput,
|
||||
options: WorkflowFnOptions,
|
||||
) => AsyncGenerator<RoleOutput, WorkflowResult>;
|
||||
) => AsyncGenerator<RoleOutput, WorkflowCompletion>;
|
||||
|
||||
/** Engine start frame: initial prompt + thread identity. */
|
||||
export type StartStep = {
|
||||
@@ -55,12 +72,20 @@ export type StartStep = {
|
||||
|
||||
/** A completed role step in the thread. */
|
||||
export type RoleStep<M extends RoleMeta> = {
|
||||
[K in keyof M & string]: { role: K; meta: M[K]; content: string; timestamp: number };
|
||||
[K in keyof M & string]: {
|
||||
role: K;
|
||||
meta: M[K];
|
||||
contentHash: string;
|
||||
refs: string[];
|
||||
timestamp: number;
|
||||
};
|
||||
}[keyof M & string];
|
||||
|
||||
/** Phase 1: Moderator decides next role. */
|
||||
export type ModeratorContext<M extends RoleMeta = RoleMeta> = {
|
||||
threadId: string;
|
||||
/** Same as `WorkflowFnOptions.depth` for the active thread. */
|
||||
depth: number;
|
||||
start: StartStep;
|
||||
steps: RoleStep<M>[];
|
||||
};
|
||||
@@ -71,6 +96,7 @@ export type AgentContext<M extends RoleMeta = RoleMeta> = ModeratorContext<M> &
|
||||
name: string;
|
||||
systemPrompt: string;
|
||||
};
|
||||
cas: CasStore;
|
||||
};
|
||||
|
||||
/** Phase 3: Extractor runs — has agent output; the extraction instruction is a separate argument to the extract function. */
|
||||
@@ -96,6 +122,9 @@ export type RoleDefinition<Meta extends Record<string, unknown>> = {
|
||||
systemPrompt: string;
|
||||
extractPrompt: string;
|
||||
schema: z.ZodType<Meta>;
|
||||
/** When non-null, produces CAS hashes to persist on this role's steps (see `RoleOutput.refs`). */
|
||||
extractRefs: ((meta: Meta) => string[]) | null;
|
||||
extractMode: ExtractMode;
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,11 +1,15 @@
|
||||
import { mkdir, unlink, writeFile } from "node:fs/promises";
|
||||
import { createServer, type Socket } from "node:net";
|
||||
import { dirname, join } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { importWorkflowBundleModule } from "./bundle-import-env.js";
|
||||
import { createCasStore } from "./cas.js";
|
||||
import type { PrefilledDiskStep } from "./engine.js";
|
||||
import { type ExecuteThreadIo, executeThread } from "./engine.js";
|
||||
import { ensureUncagedWorkflowSymlink } from "./ensure-uncaged-workflow-symlink.js";
|
||||
import { createLogger } from "./logger.js";
|
||||
import { normalizeRefsField } from "./refs-field.js";
|
||||
import { err, ok, type Result } from "./result.js";
|
||||
import { getGlobalCasDir } from "./storage-root.js";
|
||||
import { createThreadPauseGate, type ThreadPauseGate } from "./thread-pause-gate.js";
|
||||
import type { RoleOutput, WorkflowFn } from "./types.js";
|
||||
|
||||
@@ -16,7 +20,7 @@ type RunCommand = {
|
||||
threadId: string;
|
||||
workflowName: string;
|
||||
prompt: string;
|
||||
options: { maxRounds: number };
|
||||
options: { maxRounds: number; depth: number };
|
||||
steps: RoleOutput[];
|
||||
/** Timestamps aligned with `steps` for `.data.jsonl` replay; length must match `steps` when non-null. */
|
||||
stepTimestamps: number[] | null;
|
||||
@@ -47,15 +51,20 @@ type ThreadHandle = {
|
||||
|
||||
function parseRoleOutputRecord(obj: Record<string, unknown>): RoleOutput | null {
|
||||
const role = obj.role;
|
||||
const content = obj.content;
|
||||
const contentHash = obj.contentHash;
|
||||
const meta = obj.meta;
|
||||
if (typeof role !== "string" || typeof content !== "string") {
|
||||
if (typeof role !== "string" || typeof contentHash !== "string") {
|
||||
return null;
|
||||
}
|
||||
if (meta === null || typeof meta !== "object") {
|
||||
return null;
|
||||
}
|
||||
return { role, content, meta: meta as Record<string, unknown> };
|
||||
return {
|
||||
role,
|
||||
contentHash,
|
||||
meta: meta as Record<string, unknown>,
|
||||
refs: normalizeRefsField(obj.refs),
|
||||
};
|
||||
}
|
||||
|
||||
function parseRunStepsPayload(rec: Record<string, unknown>): {
|
||||
@@ -118,6 +127,9 @@ function parseRunControlPayload(rec: Record<string, unknown>): RunCommand | null
|
||||
if (typeof maxRounds !== "number") {
|
||||
return null;
|
||||
}
|
||||
const depthRaw = optRec.depth;
|
||||
const depth =
|
||||
typeof depthRaw === "number" && Number.isFinite(depthRaw) ? Math.trunc(depthRaw) : 0;
|
||||
const parsedSteps = parseRunStepsPayload(rec);
|
||||
if (parsedSteps === null) {
|
||||
return null;
|
||||
@@ -135,7 +147,7 @@ function parseRunControlPayload(rec: Record<string, unknown>): RunCommand | null
|
||||
threadId,
|
||||
workflowName,
|
||||
prompt,
|
||||
options: { maxRounds },
|
||||
options: { maxRounds, depth },
|
||||
steps: parsedSteps.steps,
|
||||
stepTimestamps: parsedSteps.stepTimestamps,
|
||||
forkSourceThreadId,
|
||||
@@ -291,8 +303,9 @@ async function main(): Promise<void> {
|
||||
return;
|
||||
}
|
||||
|
||||
await ensureUncagedWorkflowSymlink(storageRoot);
|
||||
// Dynamic import required: user bundle path resolved at runtime
|
||||
const modUnknown: unknown = await import(pathToFileURL(bundlePath).href);
|
||||
const modUnknown: unknown = await importWorkflowBundleModule(bundlePath);
|
||||
const modRec = modUnknown as Record<string, unknown>;
|
||||
const runExport = modRec.run;
|
||||
if (!isWorkflowFnLike(runExport)) {
|
||||
@@ -306,6 +319,8 @@ async function main(): Promise<void> {
|
||||
let activeThreads = 0;
|
||||
let shutdownTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
|
||||
const cas = createCasStore(getGlobalCasDir(storageRoot));
|
||||
|
||||
const workerCtlPath = join(storageRoot, "workers", `${hash}.json`);
|
||||
|
||||
function cancelShutdownTimer(): void {
|
||||
@@ -354,6 +369,7 @@ async function main(): Promise<void> {
|
||||
hash,
|
||||
dataJsonlPath,
|
||||
infoJsonlPath,
|
||||
cas,
|
||||
};
|
||||
|
||||
const existing = threads.get(threadId);
|
||||
@@ -380,8 +396,9 @@ async function main(): Promise<void> {
|
||||
const ts = cmd.stepTimestamps?.[i];
|
||||
return {
|
||||
role: step.role,
|
||||
content: step.content,
|
||||
contentHash: step.contentHash,
|
||||
meta: step.meta,
|
||||
refs: normalizeRefsField(step.refs),
|
||||
timestamp: typeof ts === "number" && ts > 0 ? ts : baseTs + i,
|
||||
};
|
||||
});
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
import { join } from "node:path";
|
||||
|
||||
import { createCasStore } from "./cas.js";
|
||||
import { type ExecuteThreadIo, executeThread } from "./engine.js";
|
||||
import { extractBundleExports } from "./extract-bundle-exports.js";
|
||||
import { getWorkflowAsAgentMaxDepth } from "./extract-provider.js";
|
||||
import { createLogger } from "./logger.js";
|
||||
import { getRegisteredWorkflow, readWorkflowRegistry } from "./registry.js";
|
||||
import { getDefaultWorkflowStorageRoot, getGlobalCasDir } from "./storage-root.js";
|
||||
import type { AgentContext, AgentFn, ThreadInput } from "./types.js";
|
||||
import { generateUlid } from "./ulid.js";
|
||||
|
||||
export type WorkflowAsAgentOptions = {
|
||||
/** When `null`, uses `getDefaultWorkflowStorageRoot()`. */
|
||||
storageRoot: string | null;
|
||||
};
|
||||
|
||||
function resolveWorkflowAsAgentStorageRoot(options: WorkflowAsAgentOptions | null): string {
|
||||
if (options !== null && options.storageRoot !== null) {
|
||||
return options.storageRoot;
|
||||
}
|
||||
return getDefaultWorkflowStorageRoot();
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns an {@link AgentFn} that runs another registered workflow in a new thread,
|
||||
* using the parent thread's initial prompt (`ctx.start.content`) as the child {@link ThreadInput.prompt}.
|
||||
*/
|
||||
export function workflowAsAgent(
|
||||
workflowName: string,
|
||||
options: WorkflowAsAgentOptions | null = null,
|
||||
): AgentFn {
|
||||
return async (ctx: AgentContext): Promise<string> => {
|
||||
const nextDepth = ctx.depth + 1;
|
||||
|
||||
const storageRoot = resolveWorkflowAsAgentStorageRoot(options);
|
||||
|
||||
const registryResult = await readWorkflowRegistry(storageRoot);
|
||||
if (!registryResult.ok) {
|
||||
return `ERROR: failed to read workflow registry: ${registryResult.error.message}`;
|
||||
}
|
||||
|
||||
const maxDepth = getWorkflowAsAgentMaxDepth(registryResult.value.config);
|
||||
if (nextDepth > maxDepth) {
|
||||
return `ERROR: workflow-as-agent depth limit exceeded (max ${maxDepth})`;
|
||||
}
|
||||
|
||||
const entry = getRegisteredWorkflow(registryResult.value, workflowName);
|
||||
if (entry === null) {
|
||||
return `ERROR: workflow "${workflowName}" not found in registry`;
|
||||
}
|
||||
|
||||
const bundlePath = join(storageRoot, "bundles", `${entry.hash}.esm.js`);
|
||||
const bundleExportsResult = await extractBundleExports(bundlePath, { storageRoot });
|
||||
if (!bundleExportsResult.ok) {
|
||||
return `ERROR: ${bundleExportsResult.error}`;
|
||||
}
|
||||
|
||||
const input: ThreadInput = {
|
||||
prompt: ctx.start.content,
|
||||
steps: [],
|
||||
};
|
||||
|
||||
const childThreadId = generateUlid(Date.now());
|
||||
const dataJsonlPath = join(storageRoot, "logs", entry.hash, `${childThreadId}.data.jsonl`);
|
||||
const infoJsonlPath = join(storageRoot, "logs", entry.hash, `${childThreadId}.info.jsonl`);
|
||||
|
||||
const io: ExecuteThreadIo = {
|
||||
threadId: childThreadId,
|
||||
hash: entry.hash,
|
||||
dataJsonlPath,
|
||||
infoJsonlPath,
|
||||
cas: createCasStore(getGlobalCasDir(storageRoot)),
|
||||
};
|
||||
|
||||
const logger = createLogger({ sink: { kind: "file", path: infoJsonlPath } });
|
||||
const signalNever = new AbortController();
|
||||
|
||||
try {
|
||||
const result = await executeThread(
|
||||
bundleExportsResult.value.run,
|
||||
workflowName,
|
||||
input,
|
||||
{
|
||||
maxRounds: ctx.start.meta.maxRounds,
|
||||
depth: nextDepth,
|
||||
signal: signalNever.signal,
|
||||
awaitAfterEachYield: async () => {},
|
||||
forkSourceThreadId: ctx.threadId,
|
||||
prefilledDiskSteps: null,
|
||||
},
|
||||
io,
|
||||
logger,
|
||||
);
|
||||
return result.rootHash;
|
||||
} catch (e) {
|
||||
const message = e instanceof Error ? e.message : String(e);
|
||||
return `ERROR: ${message}`;
|
||||
}
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user