united-workforce/docs/rfc-001-workflow-engine.md

RFC-001: Workflow Engine Design

Author: 小橘 🍊（NEKO Team） Date: 2026-05-06 Status: Draft

---

1. Package Structure

Package	npm Name	Binary
Core lib	@uncaged/workflow	—
CLI	@uncaged/cli-workflow	uncaged-workflow

Future: @uncaged/cli umbrella, invoke via uncaged workflow <subcommand>.

Monorepo uses bun workspace.

2. Workflow Physical Implementation

A Workflow is a single-file ESM module that named-exports an AsyncGenerator function as run and workflow metadata as descriptor:

/** What each yield produces — one role's output. */
type RoleOutput = {
  role: string;
  content: string;
  meta: Record<string, unknown>;
};

/** What the generator returns when done. */
type WorkflowResult = {
  returnCode: number;
  summary: string;
};

/** Input to a workflow — prompt + optional historical steps for fork/resume. */
type ThreadInput = {
  prompt: string;
  steps: RoleOutput[];   // [] for new thread, pre-filled for fork/resume
};

/** The bundle contract — an AsyncGenerator, not a Promise. */
type WorkflowFn = (
  input: ThreadInput,
  options: { threadId: string; maxRounds: number }
) => AsyncGenerator<RoleOutput, WorkflowResult>;

Why AsyncGenerator?

The workflow yields each role output instead of writing to an injected writer or exporting a framework-specific shape:

// Example bundle — zero framework dependency (named exports only)
export const descriptor = {
  description: "Fix auth bug",
  roles: {
    planner: {
      description: "Plans the fix",
      schema: { type: "object", properties: { files: { type: "array", items: { type: "string" } } } },
    },
    coder: {
      description: "Implements the plan",
      schema: { type: "object", properties: { diff: { type: "string" } } },
    },
  },
};

export const run = async function* (input, options) {
  const plan = await callLLM("plan: " + input.prompt);
  yield { role: "planner", content: plan, meta: { files: ["src/auth.ts"] } };

  const code = await callLLM("implement: " + plan);
  yield { role: "coder", content: code, meta: { diff: "..." } };

  return { returnCode: 0, summary: "Fixed auth bug" };
};

Engine controls the loop, not the bundle:

• Each yield → engine writes to .data.jsonl, checks AbortSignal, handles pause/resume
• return → engine writes the final result, marks thread complete
• Fork = read historical steps from .data.jsonl, pass as input.steps to a new generator
• Zero injection — the bundle doesn't import or receive anything from the engine

Fork/Resume via ThreadInput

When using the createRoleModerator helper, fork is naturally handled:

// The moderator receives ThreadContext with historical steps
// It sees planner already ran → routes to coder automatically
const gen = workflow(
  { prompt: "fix bug #3", steps: [{ role: "planner", content: "...", meta: {} }] },
  { threadId: "01KQXKW18CT8G75T53R8F4G7YG", maxRounds: 10 }
);
// First yield will be coder's output, not planner's

No special replay logic needed — the moderator/role pattern inherently supports resuming from any snapshot, because moderator routing is a pure function of the accumulated steps.

This follows the Dependency Inversion Principle: the engine depends on the generator protocol (a language primitive), not on a framework-specific WorkflowDefinition. Bundles remain pure functions with no coupling to @uncaged/workflow.

Relationship to Role/Moderator Pattern

The Role + Moderator pattern from Section 8 is one implementation strategy inside a bundle, not the bundle contract itself. A helper like createRoleModerator(roles, moderator) can produce the AsyncGenerator internally, but simple workflows can yield directly without any framework types.

Constraints

• Single .esm.js file
• Named exports run (callable AsyncGenerator workflow) and descriptor (metadata object)
• No default export
• No dynamic import()
• All static imports must be Node built-in modules only

This guarantees the file is self-contained, and its XXH64 hash (encoded as Crockford Base32) serves as a globally unique version identifier.

Role Descriptor (export const descriptor)

The bundle must export a descriptor object describing roles for tooling/agent consumption.

Shape: { description: string, roles: Record<string, { description: string, schema: JSONSchema }> }

When you register a bundle via uncaged-workflow add, the engine imports the module, validates descriptor, and writes {hash}.yaml next to {hash}.esm.js under bundles/ (same serialized shape as below):

description: "Workflow brief introduction"
roles:
  planner:
    description: "Analyzes the issue and creates a plan"
    schema:
      type: object
      properties:
        plan:
          type: string
        files:
          type: array
          items:
            type: string
  coder:
    description: "Implements the plan"
    schema:
      type: object
      properties:
        diff:
          type: string

Execution uses run only; YAML is for tooling and introspection.

3. Storage Layout

All data lives under ~/.uncaged/workflow/:

~/.uncaged/workflow/
├── bundles/                              # ESM bundles
│   ├── C9NMV6V2TQT81.esm.js             # Crockford Base32 of XXH64 hash
│   └── C9NMV6V2TQT81.yaml               # Role descriptor (from bundle export, at register time)
├── logs/                                 # Thread data, one folder per bundle hash
│   └── C9NMV6V2TQT81/
│       ├── 01KQXKW18CT8G75T53R8F4G7YG.data.jsonl
│       └── 01KQXKW18CT8G75T53R8F4G7YG.info.jsonl
└── workflow.yaml                         # Registry

Not a git repo. Not an npm package. Bundles are self-contained single files.

ID Encoding

All IDs use Crockford Base32:

• Better readability than Base64
• Higher density than hex (shorter filenames)
• ULID: 10 chars timestamp (high 2 bits zero-padded for future use) + 16 chars random

4. Registry (workflow.yaml)

workflows:
  solve-issue:
    hash: "C9NMV6V2TQT81"
    timestamp: 1714963200000
    history:
      - hash: "A7BKR3M1NPQ40"
        timestamp: 1714876800000
      - hash: "X2FGH8J4KLM56"
        timestamp: 1714790400000

Type:

{
  workflows: Record<string, {
    hash: string;           // Crockford Base32 of current XXH64
    timestamp: number;
    history: { hash: string; timestamp: number }[];
  }>
}

No concurrency control or timeout settings in the registry — those belong to each workflow/role/adapter.

5. Thread JSONL Format

.data.jsonl — Thread State

Line 1: Start record

{
  "name": "solve-issue",
  "hash": "C9NMV6V2TQT81",
  "threadId": "01KQXKW18CT8G75T53R8F4G7YG",
  "parameters": {
    "prompt": "Fix the login redirect bug in #3",
    "options": {
      "maxRounds": 5
    }
  },
  "timestamp": 1714963200000
}

Line 2+: Role outputs

{
  "role": "planner",
  "content": "Plan: modify auth middleware...",
  "meta": { "plan": "...", "files": ["src/auth.ts"] },
  "timestamp": 1714963201000
}

.info.jsonl — Debug Log

{
  "tag": "4KNMR2PX",       // 40-bit random, Crockford Base32 (8 chars)
  "content": "Loading workflow bundle...",
  "timestamp": 1714963200500
}

6. Execution Model

• No daemon. uncaged-workflow run <name> starts a worker process.
• Same bundle's threads share one process (memory efficiency).
• Process exits automatically when all threads complete.
• Thread termination requires IPC within the process (not just kill PID).

7. CLI Requirements

P1 (Must Have)

Command	Description
uncaged-workflow add <name> <file.esm.js> [--types <path>]	Register a compiled .esm.js bundle (descriptor extracted from export const descriptor)
uncaged-workflow list	List registered workflows
uncaged-workflow show <name>	Show workflow details
uncaged-workflow remove <name>	Remove a workflow
uncaged-workflow run <name> [--prompt] [--max-rounds]	Start a thread
uncaged-workflow threads [name]	List threads (optionally filter by workflow)
uncaged-workflow thread <id>	Show thread state
uncaged-workflow thread rm <id>	Delete a thread
uncaged-workflow ps	List running threads
uncaged-workflow kill <thread-id>	Terminate a running thread (via IPC)

P2 (Should Have)

Command	Description
uncaged-workflow history <name>	Show version history
uncaged-workflow rollback <name> [hash]	Switch to a previous version
uncaged-workflow pause <thread-id>	Pause a running thread
uncaged-workflow resume <thread-id>	Resume a paused thread

P3 (Nice to Have)

Command	Description
uncaged-workflow fork <thread-id> [--from-role <role>]	Fork from a historical thread state

8. Role/Moderator Pattern (Helper, Not Contract)

The bundle contract is the AsyncGenerator from Section 2. The Role + Moderator pattern below is a convenience helper for the common case of multi-role workflows with a routing function. It lives in @uncaged/workflow as an optional utility.

Helper Function

function createRoleModerator<M extends RoleMeta>(
  def: { roles: { [K in keyof M & string]: Role<M[K]> }; moderator: Moderator<M> }
): WorkflowFn;  // returns (input: ThreadInput, options) => AsyncGenerator

Usage in a bundle:

import { createRoleModerator, END } from "@uncaged/workflow";

export const descriptor = {
  description: "Example multi-role workflow",
  roles: {
    planner: { description: "Plans work", schema: {} },
    coder: { description: "Writes code", schema: {} },
  },
};

export const run = createRoleModerator({
  roles: { planner, coder },
  moderator(ctx) { return ctx.steps.length === 0 ? "planner" : END; },
});
// Accepts ThreadInput — fork with pre-filled steps works automatically

Supporting Types

/** Sentinel values for automaton control flow. */
const START = "__start__" as const;
const END = "__end__" as const;

/** Maps role names → their meta types. Single generic drives all inference. */
type RoleMeta = Record<string, Record<string, unknown>>;

/** Typed output of a Role execution. */
type RoleResult<Meta> = { content: string; meta: Meta };

/** Engine start frame: initial prompt + thread identity. */
type StartStep = {
  role: START;
  content: string;              // the user prompt
  meta: { maxRounds: number; threadId: string };
  timestamp: number;
};

/** A completed role step in the thread. */
type RoleStep<M extends RoleMeta> = {
  [K in keyof M & string]: { role: K; meta: M[K]; content: string; timestamp: number };
}[keyof M & string];

/** Thread-scoped context passed to roles and moderator. */
type ThreadContext<M extends RoleMeta = RoleMeta> = {
  threadId: string;
  start: StartStep;
  steps: RoleStep<M>[];
};

/**
 * A Role — receives full thread context, returns typed content + meta.
 * Implementation can be an agent, LLM call, script, HTTP request, etc.
 */
type Role<Meta> = (ctx: ThreadContext) => Promise<RoleResult<Meta>>;

/**
 * An Agent — raw string output interface for LLM/CLI adapters.
 * Structured meta is extracted by the role's extract layer.
 */
type AgentFn = (ctx: ThreadContext, systemPrompt: string) => Promise<string>;

/**
 * The Moderator — a pure routing function.
 * Receives the full thread context (start + all prior steps).
 * On initial call, `steps` is empty.
 * Returns the next role name or END to terminate.
 */
type Moderator<M extends RoleMeta> = (ctx: ThreadContext<M>) => (keyof M & string) | END;

/** Complete workflow definition as authored by users. */
type WorkflowDefinition<M extends RoleMeta> = {
  name: string;
  roles: { [K in keyof M & string]: Role<M[K]> };
  moderator: Moderator<M>;
};

Execution Flow (when using createRoleModerator)

START (prompt) → Moderator → Role A → Moderator → Role B → ... → Moderator → END

1. Engine creates a StartStep with the user prompt and maxRounds
2. Moderator is called with steps = [], returns the first role name
3. Role executes, appends a RoleStep to the thread
4. Moderator is called again with updated steps, returns next role or END
5. Repeat until END or maxRounds reached

Responsibilities

Component	Responsibility	Purity
Moderator	Route to next role based on thread state	Pure function, no side effects
Role	Execute a step (call LLM, run script, etc.)	Async, may have side effects
AgentFn	Low-level LLM/CLI invocation adapter	Async, side effects

Key Constraints

• Moderator is synchronous and pure — no I/O, no state mutation
• Roles receive the full thread context (not just the last message)
• Round count = steps.length; max rounds in start.meta.maxRounds
• The meta field on each step is typed per role via the RoleMeta generic

9. Design Decisions & Rationale

Why single-file ESM?

• Hash = version. No ambiguity.
• No dependency hell. Self-contained.
• Simple to distribute, store, and verify.

Why no daemon?

• Unnecessary complexity for process-per-bundle model.
• OS process management (systemd, etc.) handles restarts.
• IPC within process handles thread lifecycle.

Why Crockford Base32?

• Case-insensitive, filesystem-safe.
• No ambiguous characters (0/O, 1/I/L).
• More compact than hex (13 chars for 64-bit vs 16).

Why not control concurrency in registry?

• Different workflows have different constraints.
• Same workflow may allow cross-project concurrency but not intra-project.
• Concurrency belongs at workflow/role/adapter level.