uncaged/workflow
10
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

RFC: Thread-scoped CAS (Content-Addressable Storage) for workflow metadata #23

New Issue
Closed
opened 2026-05-07 03:53:08 +00:00 by xiaoju · 3 comments
xiaoju commented 2026-05-07 03:53:08 +00:00
Owner
Copy Link
Copy Source

Summary

Add a thread-scoped Content-Addressable Storage (CAS) to @uncaged/workflow core. Large text blobs (phase descriptions, acceptance criteria, etc.) are stored by XXH64 hash, and only the hash is kept in role meta. This keeps meta compact and gives moderator tamper-proof IDs to track.

Motivation

Problem 1: Phase tracking is fragile

The coder role reports completedPhase: string as free text. The extract LLM can hallucinate phase names ("all-done", "complete") that don't match any planned phase. The moderator can't reliably determine progress. (See #21)

Problem 2: Meta bloat

Planner meta currently stores full description and acceptance text for every phase. As plans grow, this bloats the meta that gets serialized into .data.jsonl and injected into every subsequent agent context.

Design

CAS Core API (@uncaged/workflow)

type CasStore = {
  put(content: string): Promise<string>;  // returns XXH64 Crockford Base32 hash
  get(hash: string): Promise<string | null>;
  delete(hash: string): Promise<void>;
  list(): Promise<string[]>;
};

function createThreadCas(storageRoot: string, threadId: string): CasStore;
  • Hash algorithm: XXH64 → 13-char Crockford Base32 (same as bundle hashes)
  • Content-addressable: same content → same hash, naturally deduped
  • Thread-scoped: each thread has its own CAS namespace

Storage Layout

~/.uncaged/workflow/
├── logs/
│   └── <bundle-hash>/
│       ├── <thread-id>.data.jsonl
│       ├── <thread-id>.info.jsonl
│       └── <thread-id>.cas/         # CAS lives alongside thread data
│           ├── 4KNMR2PX5G1AB.txt
│           └── 7BQST3VW0M2NK.txt

When a thread is deleted (uncaged-workflow thread rm <id>), the .cas/ directory is deleted with it.

CLI Subcommand

uncaged-workflow cas get <thread-id> <hash>     # read content by hash
uncaged-workflow cas put <thread-id> <content>   # store content, print hash
uncaged-workflow cas list <thread-id>            # list all hashes
uncaged-workflow cas rm <thread-id> <hash>       # delete entry

Agents (hermes/cursor) can shell out to these commands to read phase details.

Planner Meta (before → after)

Before:

{
  "phases": [
    { "name": "setup-branch", "description": "Create branch from main...", "acceptance": "On the new branch, clean working tree." },
    { "name": "write-tests", "description": "Create test file with 19 cases...", "acceptance": "File exists, all branches covered." }
  ]
}

After:

{
  "phases": ["4KNMR2PX5G1AB", "7BQST3VW0M2NK"]
}

Each hash points to a CAS entry containing the full phase spec (name + description + acceptance as structured text).

Coder Meta (before → after)

Before: { "completedPhase": "setup-branch", ... }
After: { "completedPhase": "4KNMR2PX5G1AB", ... }

The coder must report one of the exact hashes from the planner. The moderator does a simple set intersection — no fuzzy matching, no sentinel hacks.

Agent Access Pattern

  1. Planner runs → stores each phase detail via CAS → meta contains hash array
  2. Coder receives context with phase hashes → reads detail via uncaged-workflow cas get <thread-id> <hash> → works on it → reports the hash as completedPhase
  3. Moderator checks: completedPhases ⊇ planned phases (pure hash comparison)

Scope

Phase 1: Core + CLI

  • createThreadCas() in @uncaged/workflow
  • uncaged-workflow cas subcommands
  • Thread deletion cleans up CAS

Phase 2: Integration

  • Update planner role to use CAS for phase storage
  • Update coder role to read from CAS and report hash IDs
  • Update moderator to compare hashes instead of names
  • Remove the sentinel workaround from #21

Open Questions

  1. Should CAS entries have a format field (text vs JSON) or always plain text?
  2. Should the engine auto-inject CAS into AgentContext, or leave it to agent code to shell out?

—— 小橘 🍊（NEKO Team）

## Summary Add a thread-scoped Content-Addressable Storage (CAS) to `@uncaged/workflow` core. Large text blobs (phase descriptions, acceptance criteria, etc.) are stored by XXH64 hash, and only the hash is kept in role meta. This keeps meta compact and gives moderator tamper-proof IDs to track. ## Motivation ### Problem 1: Phase tracking is fragile The coder role reports `completedPhase: string` as free text. The extract LLM can hallucinate phase names (`"all-done"`, `"complete"`) that don't match any planned phase. The moderator can't reliably determine progress. (See #21) ### Problem 2: Meta bloat Planner meta currently stores full `description` and `acceptance` text for every phase. As plans grow, this bloats the meta that gets serialized into `.data.jsonl` and injected into every subsequent agent context. ## Design ### CAS Core API (`@uncaged/workflow`) ```typescript type CasStore = { put(content: string): Promise<string>; // returns XXH64 Crockford Base32 hash get(hash: string): Promise<string | null>; delete(hash: string): Promise<void>; list(): Promise<string[]>; }; function createThreadCas(storageRoot: string, threadId: string): CasStore; ``` - Hash algorithm: **XXH64 → 13-char Crockford Base32** (same as bundle hashes) - Content-addressable: same content → same hash, naturally deduped - **Thread-scoped**: each thread has its own CAS namespace ### Storage Layout ``` ~/.uncaged/workflow/ ├── logs/ │ └── <bundle-hash>/ │ ├── <thread-id>.data.jsonl │ ├── <thread-id>.info.jsonl │ └── <thread-id>.cas/ # CAS lives alongside thread data │ ├── 4KNMR2PX5G1AB.txt │ └── 7BQST3VW0M2NK.txt ``` When a thread is deleted (`uncaged-workflow thread rm <id>`), the `.cas/` directory is deleted with it. ### CLI Subcommand ```bash uncaged-workflow cas get <thread-id> <hash> # read content by hash uncaged-workflow cas put <thread-id> <content> # store content, print hash uncaged-workflow cas list <thread-id> # list all hashes uncaged-workflow cas rm <thread-id> <hash> # delete entry ``` Agents (hermes/cursor) can shell out to these commands to read phase details. ### Planner Meta (before → after) **Before:** ```json { "phases": [ { "name": "setup-branch", "description": "Create branch from main...", "acceptance": "On the new branch, clean working tree." }, { "name": "write-tests", "description": "Create test file with 19 cases...", "acceptance": "File exists, all branches covered." } ] } ``` **After:** ```json { "phases": ["4KNMR2PX5G1AB", "7BQST3VW0M2NK"] } ``` Each hash points to a CAS entry containing the full phase spec (name + description + acceptance as structured text). ### Coder Meta (before → after) **Before:** `{ "completedPhase": "setup-branch", ... }` **After:** `{ "completedPhase": "4KNMR2PX5G1AB", ... }` The coder must report one of the exact hashes from the planner. The moderator does a simple set intersection — no fuzzy matching, no sentinel hacks. ### Agent Access Pattern 1. Planner runs → stores each phase detail via CAS → meta contains hash array 2. Coder receives context with phase hashes → reads detail via `uncaged-workflow cas get <thread-id> <hash>` → works on it → reports the hash as `completedPhase` 3. Moderator checks: `completedPhases ⊇ planned phases` (pure hash comparison) ## Scope ### Phase 1: Core + CLI - `createThreadCas()` in `@uncaged/workflow` - `uncaged-workflow cas` subcommands - Thread deletion cleans up CAS ### Phase 2: Integration - Update planner role to use CAS for phase storage - Update coder role to read from CAS and report hash IDs - Update moderator to compare hashes instead of names - Remove the sentinel workaround from #21 ## Open Questions 1. Should CAS entries have a format field (text vs JSON) or always plain text? 2. Should the engine auto-inject CAS into AgentContext, or leave it to agent code to shell out? —— 小橘 🍊（NEKO Team）
xiaoju commented 2026-05-07 03:59:17 +00:00
Author
Owner
Copy Link
Copy Source

Amendment: Thread directory restructure

While adding CAS, consolidate all thread-related files into a single directory.

Before

logs/<bundle-hash>/
  <thread-id>.data.jsonl
  <thread-id>.info.jsonl
  <thread-id>.running
  <thread-id>.cas/

After

logs/<bundle-hash>/
  <thread-id>/
    data.jsonl
    info.jsonl
    running
    cas/
      <hash>.txt

This is a breaking change to storage layout. All thread file path references in the codebase (worker.ts, engine.ts, cmd-thread.ts, etc.) need updating.

Benefit: thread rm <id> becomes a single rm -rf on the directory. No more glob patterns to find related files.

—— 小橘 🍊（NEKO Team）

## Amendment: Thread directory restructure While adding CAS, consolidate all thread-related files into a single directory. ### Before ``` logs/<bundle-hash>/ <thread-id>.data.jsonl <thread-id>.info.jsonl <thread-id>.running <thread-id>.cas/ ``` ### After ``` logs/<bundle-hash>/ <thread-id>/ data.jsonl info.jsonl running cas/ <hash>.txt ``` This is a **breaking change** to storage layout. All thread file path references in the codebase (`worker.ts`, `engine.ts`, `cmd-thread.ts`, etc.) need updating. Benefit: `thread rm <id>` becomes a single `rm -rf` on the directory. No more glob patterns to find related files. —— 小橘 🍊（NEKO Team）
xiaoju commented 2026-05-07 04:13:27 +00:00
Author
Owner
Copy Link
Copy Source

Phase entry 增加 title 字段

Planner meta 的 phases 不应该是纯 hash 数组，每个 entry 应该带一个一句话 title。理由：meta 会注入 agent prompt，纯 hash 对 agent 没有语义，每次都要 cas get 才能理解计划全貌。

Before (纯 hash)

{ "phases": ["4KNMR2PX5G1AB", "7BQST3VW0M2NK"] }

After (hash + title)

{
  "phases": [
    { "hash": "4KNMR2PX5G1AB", "title": "Create feature branch from main" },
    { "hash": "7BQST3VW0M2NK", "title": "Write unit tests for all edge cases" }
  ]
}

Agent 看 title 就能理解计划概要和当前进度，需要细节（description、acceptance）时才用 hash 去 CAS 拉全文。

同步更新 plannerMetaSchema：

export const phaseSchema = z.object({
  hash: z.string(),
  title: z.string(),
});

export const plannerMetaSchema = z.object({
  phases: z.array(phaseSchema),
});

coderMetaSchema 的 completedPhase 仍然用 hash（精确匹配），不需要改。

—— 小橘 🍊（NEKO Team）

## Phase entry 增加 title 字段 Planner meta 的 phases 不应该是纯 hash 数组，每个 entry 应该带一个一句话 title。理由：meta 会注入 agent prompt，纯 hash 对 agent 没有语义，每次都要 `cas get` 才能理解计划全貌。 ### Before (纯 hash) ```json { "phases": ["4KNMR2PX5G1AB", "7BQST3VW0M2NK"] } ``` ### After (hash + title) ```json { "phases": [ { "hash": "4KNMR2PX5G1AB", "title": "Create feature branch from main" }, { "hash": "7BQST3VW0M2NK", "title": "Write unit tests for all edge cases" } ] } ``` Agent 看 title 就能理解计划概要和当前进度，需要细节（description、acceptance）时才用 hash 去 CAS 拉全文。 同步更新 `plannerMetaSchema`： ```typescript export const phaseSchema = z.object({ hash: z.string(), title: z.string(), }); export const plannerMetaSchema = z.object({ phases: z.array(phaseSchema), }); ``` `coderMetaSchema` 的 `completedPhase` 仍然用 hash（精确匹配），不需要改。 —— 小橘 🍊（NEKO Team）
xiaoju commented 2026-05-07 05:01:35 +00:00
Author
Owner
Copy Link
Copy Source

验证结果汇总

  • ✅ Phase 1: CAS Core + CLI（createThreadCas, hashString, CLI subcommands, thread rm cleanup）
  • ✅ Phase 2: Planner/Coder/Moderator 集成（compact {hash, title} meta, sentinel 移除）

106 tests 全绿，已 merge to main (8c4441b)

Close RFC-023.

—— 小橘 🍊（NEKO Team）

## 验证结果汇总 - ✅ Phase 1: CAS Core + CLI（`createThreadCas`, `hashString`, CLI subcommands, thread rm cleanup） - ✅ Phase 2: Planner/Coder/Moderator 集成（compact {hash, title} meta, sentinel 移除） 106 tests 全绿，已 merge to main (8c4441b) Close RFC-023. —— 小橘 🍊（NEKO Team）
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
aobing jiashuang (Jia Shuang) jiayi (Jiayi) luming scottwei tuanzi xiaoju xiaomo xiaonuo xingyue
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: uncaged/workflow#23
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?