Compare commits

..

53 Commits

Author SHA1 Message Date
xiaomo 0455f928f5 fix: address review feedback (星月) for Phase 3
1. sendToWorker: IPC send failure now marks thread as failed + dequeues next
2. crashLimitBlocked Set: prevents new startWorkflow from bypassing crash limit
3. "respawning" log skipped when crash limit is active
4. logWorkflowEvent payload: unknown | null (project convention, not ?:)
2026-04-30 14:10:06 +00:00
xiaomo dc4454d23e refactor: migrate workflow-manager process logic to WorkerRuntime (RFC-006 Phase 3)
- workflow-manager.ts: 792 → 498 lines, no more fork/ChildProcess/crash counting
- Extract pure functions to workflow-manager-support.ts (256 lines)
- WorkerRuntime gains: onCrashLimitReached callback, WorkerDrainOpts for per-call timeout
- worker-pool.ts updated for new evict/drain signature (null opts)
- All 167 daemon tests pass

Closes #282
2026-04-30 13:56:43 +00:00
xingyue 082d2e72f2 Merge pull request 'refactor: align develop prompts and .knowledge with flat workspace' (#288) from refactor/287-align-prompts-knowledge into main 2026-04-30 13:46:33 +00:00
xingyue fbf63e0266 Merge pull request 'RFC-006 Phase 2: Migrate SenseWorkerPool to WorkerRuntime' (#292) from refactor/rfc-006-worker-runtime into main 2026-04-30 13:44:56 +00:00
xingyue 7d89e8ab61 Merge pull request 'feat(cli): add hermes nerve skill — Phase 1 of RFC #289' (#291) from feat/agent-inject-phase1 into main 2026-04-30 13:42:05 +00:00
xiaomo e67ddc58d8 fix: address review feedback (星月)
1. trySendSync: wrap child.send in try/catch — IPC race between connected check and send
2. gracefulStop: same try/catch for shutdown send
3. Remove crashTimestamps reset on ready — crash window detection was being bypassed
2026-04-30 13:41:31 +00:00
xiaoju 06b1e3d785 refactor(cli,workflow-meta): scaffold AGENT.md on init; align develop prompts
Generate AGENT.md at ~/.uncaged-nerve root during nerve init (layout, verb-first workflows, createRole four-tuple, root build, coding style). Role prompts instruct agents to use cat AGENT.md instead of node_modules nerve-skills paths.

E2E init test asserts AGENT.md. Retain .knowledge workflow/adapter updates and flat single-file roles guidance from the branch.

Fixes #287

Made-with: Cursor
2026-04-30 13:41:24 +00:00
tuanzi f828ebc28b fix: align testing issue commands + add moderator sync pitfall
Address review from 星月:
- Update testing issue #290 to match actual CLI commands
- Add pitfall: moderator must be sync (not async)
- knowledge commands confirmed real (exist in codebase)

Ref: #290
2026-04-30 13:38:56 +00:00
tuanzi 809a11afe3 feat(cli): add hermes nerve skill (Phase 1 of #289)
Add SKILL.md for Hermes Agent covering:
- Core concepts (Sense → Signal → Workflow → Log)
- Complete CLI reference
- Sense development guide with examples
- Workflow development guide with examples
- Daily operation patterns
- nerve.yaml config reference

Ref: #289, #290
2026-04-30 13:36:03 +00:00
xiaomo 4dffcb636b fix: resolve 2 failing tests after WorkerRuntime migration
- Add trySendSync() for synchronous send when worker is ready+connected
- sendCompute uses sync path first, async fallback for cold start
- Add forwardStderr, allowRespawn, hasDisconnectedChild, onReady(key,msg)
- Tests: add connected:true to mocks, flush async fork microtasks
- All 167 daemon tests pass
2026-04-30 13:34:10 +00:00
xiaomo c34ec46416 feat(daemon): WorkerRuntime — generic message-routed process manager (closes #280)
RFC-006 Phase 1: ManagedWorker state machine + WorkerRuntime<K> with
cold start, crash respawn, drain/evict, graceful shutdown.
8 test cases covering all lifecycle scenarios.
2026-04-30 13:09:19 +00:00
xingyue d2bb0275dc Merge pull request 'feat(workflow-utils): add createLlmAdapter AgentFn factory' (#278) from refactor/277-llm-adapter-four-tuple into main 2026-04-30 12:51:29 +00:00
xiaoju 005739f6bc chore(workflow-utils): remove deprecated role factory exports
Remove createCursorRole, createHermesRole, createLlmRole, createReActRole
from public API — all superseded by createRole(adapter, prompt, schema, extract).
Source files retained as internal implementation.

Also remove unused type exports: CliPromptFn, CursorRole*, HermesRole*,
LlmPromptFn, LlmRole*, ReActRole*, ReActTool. Keep LlmMessage and
MetaExtractConfig (used internally).

Refs #277
2026-04-30 12:44:30 +00:00
xiaoju fbe1cc8eba feat(workflow-utils): add createLlmAdapter AgentFn factory
Single-turn chat via chatCompletionText: system from createRole prompt, user from ctx.start.content.

Fixes #277

Made-with: Cursor
2026-04-30 12:38:00 +00:00
xiaomo ba286a2f27 Merge pull request 'refactor(cli): single-package workspace init and root dist build' (#276) from refactor/274-single-package-workspace into main 2026-04-30 11:24:19 +00:00
xiaoju c98e14e9e6 refactor(cli): single-package workspace init and root dist build (#274)
Init templates match ~/.uncaged-nerve: scripts/build.mjs writes dist/senses/*/index.js and dist/workflows/*/index.js; drop @uncaged/nerve-skills from generated package.json; refresh Cursor skills rule copy.

Sense worker sends full compute result on signal IPC so the kernel can route workflow triggers; update e2e harness paths (migrations under senses/, noop under dist/workflows).

Fixes #274

Made-with: Cursor
2026-04-30 10:17:44 +00:00
xiaomo 011345e114 Merge pull request 'refactor(core): consolidate file structure — 22 files → 6' (#275) from refactor/core-file-consolidation into main 2026-04-30 09:21:01 +00:00
xiaoju d9c86c49ae refactor(daemon): load sense/workflow bundles from dist/ directory
Workspace build output moved from senses/<name>/index.js and
workflows/<name>/dist/index.js to dist/senses/<name>/index.js
and dist/workflows/<name>/index.js.

Refs #274
小橘 <xiaoju@shazhou.work>
2026-04-30 09:16:25 +00:00
xiaomo 0d78df89b1 refactor(core): consolidate file structure — 22 files → 6 (closes #273) 2026-04-30 09:15:18 +00:00
xiaomo 0140cdd952 Merge pull request 'refactor: RFC-005 — Separate Agent and Role types' (#272) from refactor/rfc-005-phase-1 into main 2026-04-30 08:29:12 +00:00
xiaomo bfadfffd40 fix: move isDryRun to value export (not type-only) 2026-04-30 08:27:07 +00:00
xiaomo e6093c35db docs: update knowledge cards for RFC-005 (ThreadContext, AgentFn) 2026-04-30 08:09:05 +00:00
xiaomo de8c7c5150 fix: address review — revert unrelated sense-worker change, restore isDryRun as deprecated 2026-04-30 08:00:46 +00:00
xiaomo f799cee51f refactor(cli,docs): RFC-005 Phase 4 — update templates, tests, docs (closes #271) 2026-04-30 07:24:11 +00:00
xiaomo d13b59e787 refactor(daemon): RFC-005 Phase 3 — workflow-worker uses ThreadContext (closes #270) 2026-04-30 07:10:58 +00:00
xiaomo 975f15c66d refactor(workflow-utils): RFC-005 Phase 2 — adapt to ThreadContext, new AgentFn signature (closes #269) 2026-04-30 06:59:15 +00:00
xiaomo 3e51335d91 refactor(core): RFC-005 Phase 1 — ThreadContext, AgentFn, Role signature (closes #268) 2026-04-30 06:54:03 +00:00
xiaoju 9c832b0e21 docs(knowledge): update cards via knowledge-extraction workflow (5q/round)
7 cards updated, 4 new cards added. Topics: signal-routing,
worker-isolation, storage-layer, adapter-isolation, sense contracts,
workflow runtime enforcement, coding conventions details.

小橘 <xiaoju@shazhou.work>
2026-04-30 05:56:29 +00:00
xiaoju 2387b73141 fix(daemon): remove stale exports openPeerDb, loadComputeFn from index
These functions were renamed/removed from sense-runtime.ts but index.ts
still re-exported them, causing rslib build to fail (no JS output).

小橘 <xiaoju@shazhou.work>
2026-04-30 05:56:24 +00:00
xiaoju 8824421f26 docs: remove Reflex concept from architecture docs and CLAUDE.md
Reflex was folded into Sense config (interval/on) and ComputeResult
(workflow trigger). Two extension points now: Sense + Workflow.

— 小橘 🍊(NEKO Team)
2026-04-30 00:43:04 +00:00
xiaoju b27a6aced8 feat: sense compute returns ComputeResult<T> with workflow trigger support
- SenseComputeFn returns ComputeResult<T> = null | { signal, workflow }
- sense-runtime persists result.signal, not result itself
- sense-worker sends workflow trigger message when workflow is non-null
- New SenseWorkflowTriggerMessage in IPC protocol
- Knowledge card updated to match

— 小橘 🍊(NEKO Team)
2026-04-30 00:37:16 +00:00
xiaoju bfd8fe729a docs: update sense knowledge card to match pure compute API
- No args (no db, no peers, no signal)
- Runtime handles db.insert
- Export { compute, table }

— 小橘 🍊(NEKO Team)
2026-04-30 00:28:23 +00:00
xiaoju 748df10e6a fix: remove AbortSignal from SenseComputeFn
Compute is truly zero-arg now: () => Promise<T | null>.
Runtime handles timeout via Promise.race, sense doesn't need signal.

— 小橘 🍊(NEKO Team)
2026-04-30 00:22:57 +00:00
xiaoju 3ef9cfcb27 Merge pull request 'refactor: pure sense compute — no db, no peers' (#265) from refactor/pure-sense-compute into main 2026-04-30 00:11:57 +00:00
xiaoju 8c9adf08c5 refactor: pure sense compute — no db, no peers
SenseComputeFn is now (signal: AbortSignal) => Promise<T | null>.
sense-runtime handles db.insert when compute returns non-null.
Senses export { compute, table } — SenseModule type added to core.

Closes #264
Refs #260

— 小橘 🍊(NEKO Team)
2026-04-30 00:07:49 +00:00
scottwei 08e8020cb6 Merge pull request 'feat: add sense contract types to nerve-core' (#263) from feat/sense-contract into main
Reviewed-on: #263
2026-04-29 23:44:01 +00:00
xiaoju e287e07dab feat: add sense contract types to @uncaged/nerve-core
Export SenseComputeFn, SenseComputeOptions, SenseBlobStore,
SensePeerMap — formalizing the compute function signature that
senses must implement.

Closes #262
Refs #260

— 小橘 🍊(NEKO Team)
2026-04-29 15:11:49 +00:00
xiaomo 239dfffb28 Merge pull request 'feat: add @uncaged/nerve-workflow-meta package' (#259) from feat/workflow-meta-package into main 2026-04-29 14:48:57 +00:00
xiaoju 6ccb33bf40 feat: add @uncaged/nerve-workflow-meta package
Extract develop-sense and develop-workflow meta workflows into a
shared package. Reviewer and committer roles imported from their
respective packages.

Refs RFC-004 Phase 2

— 小橘 🍊(NEKO Team)
2026-04-29 14:47:12 +00:00
xiaomo 0c95a9d716 Merge pull request 'feat: add @uncaged/nerve-role-reviewer package' (#258) from feat/role-reviewer-package into main 2026-04-29 14:33:45 +00:00
xiaoju aa64ea86ca feat: add @uncaged/nerve-role-reviewer package
Extract shared reviewer role (diff analysis, convention checking) into
a reusable package. Identical logic currently duplicated across
develop-sense and develop-workflow workspaces.

Refs RFC-004 Phase 1
2026-04-29 14:31:36 +00:00
xiaomo 431627019a Merge pull request 'feat: add @uncaged/nerve-role-committer package (RFC-004 Phase 1)' (#257) from feat/rfc004-role-committer into main 2026-04-29 14:16:11 +00:00
xiaoju 915584ff11 feat: add @uncaged/nerve-role-committer package (RFC-004 Phase 1)
First shared role package. Extracts workspace committer into a
reusable package with decorator chain (withDryRun + onFail).

Also fixes workflow-utils test types (StartStep shape, TestMeta constraint).
2026-04-29 14:07:26 +00:00
xiaoju 3644cce2c8 Merge pull request 'docs: RFC-004 package architecture' (#256) from docs/rfc-004-package-architecture into main 2026-04-29 14:03:02 +00:00
xiaoju 9855eba894 docs: RFC-004 package architecture — shareable workflows, roles & senses 2026-04-29 13:29:53 +00:00
scottwei 554933d43b Merge pull request 'feat(workflow-utils): add withDryRun role wrapper' (#255) from feat/254-with-dry-run into main
Reviewed-on: #255
2026-04-29 13:28:22 +00:00
xiaoju 9c11cd53e6 rename with-dry-run → role-decorators, remove RFC-004 from this PR 2026-04-29 13:27:08 +00:00
xiaoju c08e7f085d refactor(workflow-utils): split withDryRun into decorator chain
- withDryRun(opts) — only handles dry-run skip
- onFail(opts) — only handles try/catch error wrapping
- decorateRole(role, [...]) — composes decorators left-to-right
- RoleDecorator<M> type for custom decorators

Addresses review feedback on #255
2026-04-29 13:23:40 +00:00
xiaoju 4a4a03a2bc feat(workflow-utils): add withDryRun role wrapper
Extracts repeated dry-run skip + try/catch error handling into a
reusable wrapper. Includes tests.

Closes #254
2026-04-29 12:57:33 +00:00
xiaoju bfb5b9b17d Merge pull request 'chore: add .knowledge/ curated cards + knowledge.yaml' (#251) from chore/knowledge-cards into main 2026-04-29 10:01:22 +00:00
xiaomo a566cdabf8 Merge pull request 'refactor: replace WorkflowSpec with createRole helper' (#253) from refactor/252-create-role into main 2026-04-29 09:55:58 +00:00
xiaoju 9b4ab6225a refactor(core): remove WorkflowSpec and compileWorkflowSpec
Add createRole to workflow-utils wrapping AgentFn, Zod meta, and extractMetaOrThrow.

Refs #252

Signed-off-by: 小橘 🍊(NEKO Team) <dev@uncaged.ai>
Made-with: Cursor
2026-04-29 09:54:13 +00:00
小橘 🍊(NEKO Team) 77500ee6dd feat(daemon): pass dryRun into compileWorkflowSpec extractFn
Adds optional CursorAdapter.mode for ask/plan. Export zodMeta from workflow-utils for RoleSpec meta schemas.

Made-with: Cursor
2026-04-29 09:35:12 +00:00
115 changed files with 5532 additions and 2236 deletions
+1
View File
@@ -3,3 +3,4 @@ dist
.turbo
*.tsbuildinfo
*.tgz
knowledge.db
+171
View File
@@ -0,0 +1,171 @@
# Adapter Process Isolation
Describes sandboxing, process isolation, resource limits, and timeout enforcement for adapter invocations in the Nerve workflow system.
## Process Isolation Model
Adapters run in a **two-tier isolation** model:
1. **Workflow Worker Process** — Each workflow runs in a dedicated Node.js worker process (`workflow-worker.ts`) forked from the main daemon
2. **Adapter Child Process** — Each adapter spawns CLI tools as child processes via `spawnSafe()` with `shell: false`
## Resource Limits & Timeouts
### Adapter-Level Timeouts
- **Default timeout**: 300 seconds (300,000ms) for both cursor and hermes adapters
- **Configurable** via `AgentConfig.timeout` in adapter factory functions
- **Wall-clock enforcement** using `setTimeout()` — kills child process with `SIGTERM` on timeout
- **AbortSignal support** — external cancellation triggers immediate `SIGTERM`
### Timeout Behavior
```ts
// Timeout resolution priority (packages/core/src/spawn-safe.ts):
// 1. Explicit timeoutMs value
// 2. AbortSignal presence → no internal timer (relies on external abort)
// 3. DEFAULT_TIMEOUT_MS (300_000) fallback
```
- Child process terminated with `SIGTERM` on timeout/abort
- Returns `{ kind: "timeout", stdout, stderr }` error result
- **No grace period** — immediate kill
- **No SIGKILL escalation** — relies entirely on `SIGTERM` effectiveness
#### SIGTERM Limitations
If a child process **ignores or blocks `SIGTERM`** (e.g., signal handlers, blocked delivery):
- **No fallback to `SIGKILL`** — process may remain alive indefinitely
- **No escalation timer** — spawnSafe() does not implement progressive signal escalation
- **Potential zombie/orphan risk** — unresponsive processes continue consuming resources
- **OS-level cleanup only** — relies on parent process death or OS reaping mechanisms
## Sandboxing Characteristics
### What's Isolated
- **File system**: Child process runs in specified `cwd` (workflow working directory)
- **Environment**: Controlled env vars via `nerveCommandEnv()` + optional overrides
- **Network**: No explicit restrictions (inherits parent process network access)
- **Process tree**: Child processes are direct children, not containerized
### What's NOT Sandboxed
- **No resource quotas** (CPU, memory, disk I/O limits)
- **No filesystem chroot/containers** — full filesystem access within user permissions
- **No network isolation** — can make arbitrary network calls
- **No syscall filtering** — no seccomp or similar restrictions
#### Runtime Resource Enforcement
**No active resource monitoring or constraints**:
- **No cgroups** (Linux) — no CPU, memory, or I/O limits enforced
- **No job objects** (Windows) — no resource quotas or process tree limits
- **No worker_threads resource tracking** — Node.js worker processes run unrestricted
- **Pure timeout-based enforcement** — only wall-clock time limits via `setTimeout()`
- **OS-scheduled resource sharing** — relies entirely on operating system process scheduling
Adapters can consume unlimited:
- **CPU time** (until timeout)
- **Memory** (until OOM)
- **Disk I/O** (no quotas)
- **Network bandwidth** (no throttling)
- **File descriptors** (until ulimit)
#### Environment Variable Security
The `nerveCommandEnv()` function provides **minimal sanitization**:
```ts
// spawn-safe.ts lines 47-55
export function nerveCommandEnv(): SpawnEnv {
const home = homedir();
const pnpmHome = join(home, ".local/share/pnpm");
return {
...process.env, // ← Full parent environment inherited
PNPM_HOME: pnpmHome,
PATH: `${pnpmHome}:${process.env.PATH ?? ""}`,
};
}
```
- **No filtering of sensitive keys** — `NODE_OPTIONS`, `LD_PRELOAD`, `PYTHONPATH` passed through unchanged
- **Full environment inheritance** — all parent process environment variables copied
- **Injection risk** — malicious env vars (e.g., `NODE_OPTIONS=--require=evil.js`) affect Node.js child processes
- **Path manipulation** — sensitive PATH entries remain accessible to adapters
## Security Model
### Execution Context
- Uses `shell: false` to prevent shell injection attacks
- Arguments passed as separate array elements (not shell-parsed)
- PATH includes `~/.local/share/pnpm` for tool discovery
- Inherits parent process user/group permissions
#### File Descriptor Management
```ts
// spawn-safe.ts line 122
stdio: ["ignore", "pipe", "pipe"]
```
- **stdin closed**: Child receives no input (`stdio[0]: "ignore"`)
- **stdout/stderr captured**: Piped to parent for collection (`stdio[1,2]: "pipe"`)
- **No explicit fd closing**: Node.js default behavior — inherits other file descriptors
- **Parent sockets/pipes accessible**: Child can access parent's open network connections, database handles, etc.
- **Security risk**: Adapter processes may access unintended parent file descriptors
### Attack Surface
- CLI tools have **full user-level filesystem access**
- Can spawn additional processes (not tracked/limited)
- Network requests unrestricted
- Resource consumption relies on OS-level limits
## Worker Process Management
### Workflow Isolation
- Each workflow type gets dedicated worker process
- Worker processes handle multiple concurrent threads (runIds)
- Kill flags enable per-thread cancellation without killing worker
- Graceful shutdown waits up to 10 seconds for in-flight operations
#### Cross-RunId Contamination Risks
**Shared mutable state** poses contamination risks between concurrent runIds:
- **`process.env` mutations**: Environment changes affect all subsequent runIds in same worker
- **`require.cache` pollution**: Module cache shared across all runIds — side effects persist
- **Global variables**: Any global state mutations from one runId visible to others
- **`process.cwd()` changes**: Working directory changes affect entire worker process
- **File descriptors**: Open files/sockets shared between runId executions
**No runId-specific scoping** implemented:
- Worker reuses single Node.js process for efficiency
- Each role execution sees cumulative environment from previous runIds
- **Mitigation relies on adapter discipline** — clean implementations avoid global mutations
### Error Handling
- Adapter failures don't crash the worker process
- Timeout/abort errors are isolated to specific role execution
- Worker process survives adapter failures and continues serving other threads
## Configuration
```yaml
# Example nerve.yaml configuration for timeout overrides
workflows:
my-workflow:
roles:
coder:
adapter:
type: cursor
timeout: 600000 # 10 minutes in milliseconds
```
Timeout configuration happens at the adapter creation level, not as a system-wide sandbox policy.
+46 -4
View File
@@ -5,21 +5,52 @@ Adapter = capability. Role = scenario. Workflows declare adapters directly via i
## AgentFn Protocol
```ts
type AgentFn = (prompt: string, context: WorkflowContext) => Promise<string>
type AgentFn = (ctx: ThreadContext, systemPrompt: string) => Promise<string>
```
- Input: prompt + context (start frame, messages, workdir, AbortSignal)
- Output: raw string — structured extraction is separate
- Input: thread context (`{ threadId, start, steps }`) + system prompt (role identity)
- Output: **single-shot `Promise<string>`** — no streaming support
- Adapter handles tool-specific details internally
### Streaming Limitations
The `AgentFn` protocol does **not** support streaming responses (`AsyncIterable<string>` or `ReadableStream`). It's strictly limited to single-shot `Promise<string>` returns.
For long-running or incremental agent outputs:
- CLI tools buffer full output until completion
- Timeout enforcement via `timeoutMs` (default 300s)
- No intermediate results exposed to workflow logic
- Progress indication happens at the CLI tool level only
## Available Adapters
| Package | Adapter | Tool |
|---------|---------|------|
| `@uncaged/nerve-adapter-cursor` | `cursorAdapter` / `createCursorAdapter()` | cursor-agent CLI |
| `@uncaged/nerve-adapter-hermes` | `hermesAdapter` / `createHermesAdapter()` | hermes chat CLI |
| `@uncaged/nerve-workflow-utils` | `createLlmAdapter(provider)` | OpenAI-compatible HTTP chat (single-turn) |
Each exports a **default instance** (sensible defaults) and a **factory** for custom config.
The Cursor and Hermes adapter packages each export a **default instance** (sensible defaults) and a **factory** for custom config. `createLlmAdapter` is a factory on `@uncaged/nerve-workflow-utils` only.
## createLlmAdapter
`createLlmAdapter` builds an `AgentFn` from an `LlmProvider` (`baseUrl`, `apiKey`, `model`). One chat completion per role step: **system** = the string passed by `createRole` (your prompt); **user** = `ctx.start.content` (the thread’s start frame). On failure it throws with a formatted LLM error.
```ts
import { createLlmAdapter, createRole } from "@uncaged/nerve-workflow-utils";
import { z } from "zod";
const metaSchema = z.object({ ok: z.boolean() });
const planner = createRole(
createLlmAdapter({ baseUrl: "https://api.example.com/v1", apiKey: "…", model: "gpt-4o-mini" }),
"You are a planner…",
metaSchema,
extractConfig,
);
```
Use this when you want a role backed by an HTTP LLM instead of a subprocess CLI adapter.
## Usage in Workflows
@@ -45,3 +76,14 @@ extract:
```
Two-level merge: global → role override. Retry once on parse failure (feeds error back to LLM), then throw `ExtractError`.
## Error Handling
When adapters' underlying CLI tools (e.g., `cursor-agent` or `hermes`) fail, errors are surfaced **synchronously via rejection** with no fallback/retry logic:
- **Missing/unavailable tool**: `spawn_failed` error when CLI binary not found in `$PATH`
- **Non-zero exit code**: `non_zero_exit` error with captured stdout/stderr
- **Timeout**: `timeout` error when execution exceeds configured `timeoutMs`
- **Abort signal**: `aborted` error when `AbortSignal` triggers cancellation
All errors are immediately thrown as `Error` instances with descriptive messages (e.g., `"cursor-agent: exitCode=7 stdout=... stderr=..."`). No automatic retries or fallback adapters.
+19 -6
View File
@@ -5,20 +5,22 @@ Observation engine for autonomous agents — sense the world, react to changes,
## Core Pipeline
```
External World → Sense → Signal → Reflex → Workflow → Log
External World → Sense → Signal → Workflow → Log
compute() returns
{ signal, workflow }
```
Causality is **one-directional**. Logs are the end of the chain — they cannot trigger Reflexes (prevents feedback loops).
Causality is **one-directional**. Logs are the end of the chain — they cannot trigger further Senses (prevents feedback loops).
## Three Orthogonal Extension Points
## Two Extension Points
| Extension | Question | Nature |
|-----------|----------|--------|
| **Sense** | What to compute | `compute()` function |
| **Reflex** | When to compute | Declarative YAML (interval / on) |
| **Sense** | What to observe & when to react | `compute()` pure function + YAML config (interval / on) |
| **Workflow** | What to do | Roles + Moderator |
Each is independent. Reflex doesn't know compute internals, Sense doesn't know when it's triggered, Workflow doesn't know why it was started.
Senses own both the "what" (compute logic) and the "when" (config-driven scheduling). A Sense can trigger a Workflow directly by returning `{ signal, workflow: { name, prompt } }`.
## Two Event Types
@@ -31,3 +33,14 @@ Each is independent. Reflex doesn't know compute internals, Sense doesn't know w
- One worker per Workflow type (on-demand)
- Workers never talk to each other
- All user code runs in isolated Workers; kernel never loads user code directly
## Storage Systems
- **Log Store** — SQLite with WAL mode for audit trails and workflow state
- **Sense Databases** — Isolated SQLite per sense group for private data
- **Knowledge Store** — Vector search index for project context
- **Blob Store** — Content-addressable storage for large artifacts
## Signal Flow
Sense compute outputs are routed through signal routing logic that determines whether to emit a signal or trigger a workflow—never both simultaneously.
+26 -56
View File
@@ -5,54 +5,43 @@
## Workspace Lifecycle
```bash
nerve init # scaffold workspace (nerve.yaml, senses/, workflows/)
nerve init --from <git-url> # clone existing workspace from git
nerve init --force # reinitialize (preserves data/)
nerve init # scaffold a new workspace (nerve.yaml, senses/, workflows/)
nerve init --force # reinitialize workspace even if ~/.uncaged-nerve/ exists (preserves data/)
nerve init --from <git-url> # clone existing workspace from git repository
nerve validate # validate nerve.yaml config
nerve dev # run kernel foreground (development, Ctrl+C to stop)
nerve dev --port 3000 # with HTTP API on specific port
nerve start # start daemon (background)
nerve start --port 3000 # with HTTP API port
nerve stop # stop daemon
nerve status # check daemon health (uptime, senses, workflows)
nerve daemon restart # stop + start
nerve daemon logs # alias for nerve logs
nerve daemon # restart daemon (stop + start)
```
### Init Behavior
**Default `nerve init`**: Creates workspace at `~/.uncaged-nerve/`. If this directory already exists and is non-empty, **exits with error** requiring `--force` flag. No merge/overwrite logic — prevents accidental workspace destruction.
**Force mode `nerve init --force`**: Reinitializes workspace even if `~/.uncaged-nerve/` exists. **Preserves `data/` directory** (containing sense SQLite databases and logs) but overwrites all config files (`nerve.yaml`, `package.json`, etc.) and example senses.
**Git clone `nerve init --from <url>`**: Clones existing repository to `~/.uncaged-nerve/`. Requires empty target directory — fails if workspace already exists and is non-empty.
## Sense Management
```bash
nerve create sense <name> # scaffold a new sense (compute.ts + schema.ts)
nerve create sense <name> --force # overwrite existing
nerve sense list # list configured senses and status
nerve sense list # list configured senses
nerve sense trigger <name> # manually trigger a sense compute
nerve sense schema <name> # print CREATE TABLE statements from sense SQLite
nerve sense schema <name> --json # as JSON array
nerve sense query <name> # inspect sense SQLite database (preview rows)
nerve sense query <name> --json # rows as JSON
nerve sense schema <name> # show sense Drizzle schema
nerve sense query <name> # inspect sense SQLite database
nerve sense query <name> --sql "SELECT * FROM samples LIMIT 5"
```
## Workflow Management
```bash
nerve create workflow <name> # scaffold a new workflow
nerve create workflow <name> --force # overwrite existing
nerve workflow list # list workflow definitions from nerve.yaml
nerve workflow status # show live status (concurrency, active, queued)
nerve workflow trigger <name> --prompt "..." [--max-rounds N] [--dry-run]
```
## Thread Management
```bash
nerve thread list # list active (queued/started) workflow runs
nerve thread list --all # include completed/failed/crashed
nerve thread list --workflow <name> # filter by workflow name
nerve thread show <runId> # print role rounds for a run (agent-oriented)
nerve thread show <runId> --before N # limit rounds (pagination)
nerve thread inspect <runId> # show details and thread events
nerve thread inspect <runId> --offset N --limit N # paginate events
nerve thread kill <runId> # kill a running or queued thread
nerve workflow list # list configured workflows
nerve thread # list active (queued/started) workflow threads
```
## Knowledge
@@ -62,30 +51,21 @@ nerve knowledge sync # chunk files per knowledge.yaml, compute embeddin
nerve knowledge query "text" # search indexed knowledge (cosine similarity)
nerve knowledge query -g "text" # global search across all indexed repos
nerve knowledge query --repo /path "text" # search specific repo
nerve knowledge query "text" --limit 20 # max hits (default 10)
```
## Logs & Store
```bash
nerve logs # show daemon log output (last 50 lines)
nerve logs -n 200 # last N lines
nerve logs --offset 100 # start from line N (pagination)
nerve logs # view daemon logs (last 50 lines)
nerve logs -f # follow logs (tail -f style)
nerve store archive # archive logs older than 30 days to JSONL
nerve store archive --vacuum # also run SQLite VACUUM after archiving
nerve logs -n 200 # last N lines
nerve store archive # archive old log entries to JSONL
```
## Remote Management
## Remote
```bash
nerve remote add <name> <host:port> [--token <token>] # add remote daemon
nerve remote list # list all remotes
nerve remote show <name> # show remote details
nerve remote set-url <name> <host:port> # update remote host
nerve remote set-token <name> <token> # update remote token
nerve remote remove <name> # remove a remote
nerve remote default [<name>] # set or show default remote
nerve remote add <name> <url> # add a remote daemon endpoint
nerve status --remote <name> # check remote daemon health
```
@@ -97,22 +77,12 @@ my-agent/
knowledge.yaml # knowledge index config (optional)
senses/
cpu-usage/
src/
index.ts # sense compute implementation
schema.ts # Drizzle schema (single source of truth)
migrations/ # auto-generated by drizzle-kit
package.json # with esbuild build script
index.js # bundled output (generated by pnpm build)
compute.ts # sense implementation
schema.ts # Drizzle schema
migrations/ # auto-generated
workflows/
cleanup/
src/index.ts # workflow definition
build.ts # factory function
moderator.ts # moderator + meta types
roles/ # one file per role
package.json
data/
senses/ # per-sense SQLite databases
archive/ # archived logs (JSONL)
knowledge.db # generated by nerve knowledge sync
.knowledge/ # curated knowledge cards
```
+31
View File
@@ -24,6 +24,21 @@ type Config = { throttle?: string }
- `throw` only for programmer errors (bugs)
- No try-catch for flow control
### Result<T, E> Type
Defined in `@uncaged/nerve-core` (`packages/core/src/result.ts`):
```ts
export type Result<T, E = Error> = { ok: true; value: T } | { ok: false; error: E };
```
**Discriminated union** with tagged `ok` field. Helper functions:
- `ok(value)``{ ok: true, value }`
- `err(error)``{ ok: false, error }`
**Exhaustive handling**: Pattern is `if (!result.ok) { handle error }` then access `result.value`.
No compiler enforcement - relies on manual discipline and TypeScript's flow control analysis.
## Naming
| Type | Style |
@@ -38,9 +53,25 @@ type Config = { throttle?: string }
- Always named exports, never default
- One module = one responsibility
### Module Naming Conventions
**Primary exports** use descriptive, unambiguous names:
- Functions: `createXxx()`, `parseXxx()`, `xxxAgent()` (e.g., `createCursorAdapter`, `cursorAgent`)
- Types: Domain-specific prefixes (e.g., `CursorAgentOptions`, `SenseComputeFn`, `ThreadContext`)
- Constants: `UPPER_SNAKE_CASE` with context (e.g., `DEFAULT_SENSE_SIGNAL_RETENTION`, `CURSOR_ADAPTER_DEFAULT_MS`)
**Avoiding ambiguity**:
- Package-scoped naming: `@uncaged/nerve-adapter-cursor` exports `cursorAgent`, `createCursorAdapter`
- Factory pattern: `createXxxAdapter()` for configurable instances, `xxxAdapter` for defaults
- Descriptive type prefixes prevent collision (e.g., `CursorAgentOptions` vs `HermesAgentOptions`)
## Async
- Always `async/await`, never `.then()` chains
- Use `AbortSignal` for cancellation: `AbortController` to create signals, pass to long-running operations
- `spawn-safe.ts` and adapter functions accept `abortSignal: AbortSignal | null` parameter
- On abort: child processes receive `SIGTERM`, async operations should check `signal.aborted`
- No enforced Biome/Vitest rules for AbortSignal usage (manual discipline required)
## No Dynamic Import
+14 -4
View File
@@ -19,10 +19,20 @@ nerve knowledge query --repo /path "query" # search specific repo
## Embedding
- Remote service: configured via `EMBED_SERVICE_URL` env var (self-hosted Cloudflare Worker + KV cache)
- Model: Dashscope text-embedding-v3 (1024 dims)
- Cache: content-addressable (sha256 of model+text), never expires
- Fallback: word-overlap scoring when embed service not configured
- **Default model**: Dashscope text-embedding-v3 (1024 dimensions)
- **Remote service**: configured via `EMBED_SERVICE_URL` env var (self-hosted Cloudflare Worker + KV cache)
- **Model configuration**: No mechanism to specify alternate models — hardcoded to text-embedding-v3 in remote service
- **Vector dimensions**: Fixed at 1024 (Float32Array, stored as 4096-byte Buffer blobs in SQLite)
- **Cache**: content-addressable (sha256 of model+text), never expires
- **Fallback**: word-overlap scoring when embed service not configured
### Configuration
The embedding model is **not configurable** through `knowledge.yaml` or other config files. The remote service at `embed.shazhou.workers.dev` uses Dashscope text-embedding-v3 exclusively. To use different models, you would need to:
1. Deploy your own embedding service compatible with the same API
2. Point `EMBED_SERVICE_URL` to your service
3. Ensure vector dimensions match (1024) or modify knowledge database schema
## Chunking
+40 -8
View File
@@ -2,18 +2,39 @@
A `compute()` function that samples or derives external data. The only first-class citizen in nerve.
## Behavior
## Contract
- Returns `T | null` — non-null emits a Signal, null is silent (no storage write, no signal, no downstream trigger)
- Each Sense has its own **independent SQLite database**
- Cross-sense reads are read-only via `peers` parameter
- Schema defined with Drizzle ORM (`schema.ts` is single source of truth)
Each sense module (`src/index.ts`) must export:
## Sense → Workflow
```ts
export { snapshots as table } from "./schema.ts"; // drizzle table for runtime to insert into
If `compute()` returns an object with `workflow: "name|maxRounds|prompt"`, the engine starts that workflow and does **not** emit a Signal. `workflow: null` or `""` means emit signal normally.
export async function compute(): Promise<ComputeResult<T>> { ... } // pure, no args
```
See `routeSenseComputeOutput` / `parseSenseWorkflowDirective` in `@uncaged/nerve-core`.
**Function Signature & Input Schema:**
- `compute()` is **parameterless** — no direct inputs, environment variables available
- No database access within compute — runtime provides isolated execution context
- Must be pure function (no side effects, no external API calls)
**Return Value Contract:**
- `ComputeResult<T>` = `null | { signal: T; workflow: WorkflowTrigger | null }`
- `null` → silent, no storage, no signal
- `{ signal: data, workflow: null }` → persist + emit signal
- `{ signal, workflow: WorkflowTrigger }` → persist + emit signal + trigger workflow
- Any other value → treated as `{ signal: value, workflow: null }`
**Error Handling & Serialization:**
- Exceptions caught by worker, logged as errors (no signal emitted)
- Signal payload must be JSON-serializable (passed via IPC)
- Invalid workflow triggers silently dropped (signal still emitted)
**Timeout & Scheduling Semantics:**
- Timeout priority: explicit config → AbortSignal → DEFAULT_TIMEOUT_MS (30s)
- Enforced via `Promise.race()` with timeout promise
- Grace period can trigger `process.exit(1)` after timeout (kills worker group)
- Interval translation: YAML config values used directly as milliseconds in `setInterval()`
- Jitter control: throttle mechanism prevents rapid-fire, single deferred trigger per throttle window
## Config (nerve.yaml)
@@ -27,3 +48,14 @@ senses:
interval: 30s # periodic trigger (optional)
on: [disk-pressure] # trigger on signals from other senses (optional)
```
## Manual Trigger Context
**`nerve sense trigger <name>`** sends IPC message to running daemon. The compute context is initialized as follows:
- **SQLite Database**: Opened in **read-write mode** at `data/senses/<name>.db`
- **Migrations**: All `*.sql` files in `senses/<name>/migrations/` applied in lexicographic order
- **Environment**: Inherits daemon process environment (no special secrets injection)
- **Arguments**: No runtime arguments or mock inputs supported — `compute()` is always pure function with no parameters
- **Isolation**: Runs in forked child process (worker) with full filesystem access within user permissions
- **Persistence**: Runtime automatically calls `db.insert(table).values(result.signal)` if compute returns non-null signal
+91
View File
@@ -0,0 +1,91 @@
# Signal Routing
Signal routing is the core mechanism that determines how Sense outputs flow through the Nerve system.
## Routing Logic
When a Sense `compute()` function returns non-null, the output goes through `routeSenseComputeOutput()` in `packages/core/src/sense-workflow-directive.ts`:
```
Sense compute() → non-null → routeSenseComputeOutput() → { signal, workflow }
kernel.ts → signal ALWAYS emitted + optional workflow start
```
## Two Output Formats
### 1. Explicit Format
```typescript
{
signal: any, // emitted as signal
workflow: { // optional workflow trigger
name: string,
maxRounds: number,
prompt: string,
dryRun: boolean
} | null
}
```
### 2. Shorthand Format
Any other value is treated as:
```typescript
{ signal: payload, workflow: null }
```
## Workflow Directive Parsing
## Concrete Routing Predicates
The routing decision is implemented in `routeSenseComputeOutput()` using these exact matching criteria:
### 1. Explicit Format Detection
```typescript
if (isPlainRecord(payload) && Object.hasOwn(payload, "signal"))
```
- Payload must be a plain object
- Must have `signal` property (any value)
- Workflow extracted from `workflow` property or defaults to null
### 2. Workflow Validation
When workflow is non-null, it's validated via `parseWorkflowTrigger()`:
- `name`: non-empty string (trimmed)
- `maxRounds`: positive integer >= 1
- `prompt`: string
- `dryRun`: boolean
**Critical behavior**: Invalid workflows are silently dropped (become null) but signal emission continues. This prevents malformed workflow config from blocking signals.
### 3. Fallback to Shorthand
Any value that doesn't match explicit format becomes:
```typescript
{ signal: payload, workflow: null }
```
## Processing Flow
```typescript
// In kernel.ts handleSenseWorkerSignal()
const { signal: signalPayload, workflow } = routeResult.value;
// Signal is ALWAYS emitted when compute returns non-null
bus.emit({ id, senseId, payload: signalPayload, timestamp });
// Workflow is started ONLY if workflow is non-null
if (workflow !== null) {
workflowManager.startWorkflow(workflow.name, { ... });
}
```
## Legacy String Format (Deprecated)
The old `"name|maxRounds|prompt"` string format is converted to the structured format internally but should not be used in new code.
## Key Behaviors
1. **Signal priority**: Every non-null compute result emits a signal, regardless of workflow
2. **Additive behavior**: Valid workflow triggers are executed in addition to signal emission
3. **Failure tolerance**: Invalid workflow directives are silently ignored, signal still emits
4. **Structure-based routing**: No complex predicates - simply checks object structure and property existence
This routing mechanism ensures clean separation between perception (signals) and action (workflows) while maintaining backward compatibility.
+132
View File
@@ -0,0 +1,132 @@
# Storage Layer
Nerve uses multiple storage systems designed for different data types and access patterns.
## Core Storage Components
### 1. Log Store (`logs.db`)
Append-only audit trail implemented in SQLite with WAL mode.
**Schema:**
- `logs` — all system events (signals, workflow transitions, sense outputs)
- `meta` — key-value store for system metadata
- `workflow_runs` — materialized view of workflow execution state
**Key Features:**
- Atomic workflow state updates via transactions
- Thread message persistence for crash recovery
- Configurable log archival to JSONL files
- Full-text search across log entries
### 2. Sense Databases
Each sense group gets its own SQLite database for private state.
**Characteristics:**
- Isolated per sense group (e.g., `system-senses.db`)
- Managed by individual sense compute functions
- Drizzle ORM integration for schema management
- No cross-sense data sharing
### 3. Knowledge Store (`knowledge.db`)
Vector-enabled search index for project context.
**Contents:**
- Chunked source files with embeddings
- Curated knowledge cards from `.knowledge/`
- Semantic search capabilities
- Global vs. repo-scoped search modes
### 4. Blob Store (CAS)
Content-addressable storage for large artifacts.
**Design:**
- SHA-256 based file naming
- Automatic deduplication
- Used for workflow artifacts and large payloads
## Consistency & Isolation Mechanisms
### SQLite WAL Mode
All SQLite databases use `PRAGMA journal_mode=WAL` for:
- **Writer-reader concurrency** — readers don't block writers
- **Atomic writes** — each transaction is fully applied or rolled back
- **Crash recovery** — WAL provides consistent state after crashes
### Transaction Management
#### Log Store Transactions
Uses `BEGIN IMMEDIATE` transactions (`packages/store/src/log-store.ts`):
```typescript
function runInTransaction<T>(db: DatabaseSync, fn: () => T): T {
db.exec("BEGIN IMMEDIATE"); // Exclusive write lock
try {
const result = fn();
db.exec("COMMIT");
return result;
} catch (e) {
db.exec("ROLLBACK");
throw e;
}
}
```
**Key Operations:**
- `upsertWorkflowRun()` — atomically writes log entry + workflow state
- `archiveLogs()` — transactional export + delete + watermark update
#### Sense Database Isolation
- Each sense group has its own SQLite file (e.g., `system-senses.db`)
- No cross-sense transactions or coordination required
- Independent schema migrations per sense
- Private `_signals` table for signal history retention
### Process-Level Isolation
#### Worker Process Architecture
- **One worker per sense group** — prevents data races within group
- **One worker per workflow type** — isolated execution contexts
- **No shared memory** — all communication via IPC messages
#### Concurrency Control
Workflow manager enforces limits per workflow:
```yaml
workflows:
my-workflow:
concurrency: 2 # Max parallel threads
overflow: "queue" # or "drop"
maxQueue: 10 # Queue depth limit
```
### Consistency Guarantees & Failure Modes
**Strong Consistency (Single Database)**:
1. **Within Log Store** — ACID transactions with immediate consistency
2. **Within Sense DB** — WAL mode ensures atomic commits per database
3. **Workflow State**`upsertWorkflowRun()` atomically updates log + materialized view
**No Cross-Database Consistency**:
- No distributed transactions across multiple SQLite files
- Log Store and Sense Databases can temporarily diverge during failures
- Signal emission and workflow triggering are separate, non-atomic operations
**Failure Recovery Mechanisms**:
- **Sense worker crash**: State rebuilt from sense SQLite database on respawn
- **Workflow worker crash**: Thread state recovered from log store message history
- **Kernel crash**: All workers respawned, state recovered from persistent stores
- **Log Store corruption**: WAL recovery on database open
- **Sense DB corruption**: Migrations re-run, `_signals` table rebuilt if needed
**Rollback Scenarios**:
- **Log write failure**: Transaction rolled back, no state changes persisted
- **Sense compute failure**: Error logged, no signal/workflow emitted
- **Workflow failure**: Thread marked as failed in materialized view
- **IPC failure**: Worker respawned, pending operations lost (not rolled back)
## Archive Strategy
Logs older than retention window (default 30 days) are:
1. Exported to `data/archive/logs/YYYY-MM-DD.jsonl`
2. Deleted from active database
3. Watermark updated to prevent re-processing
This keeps the active database size bounded while preserving audit trails.
+152
View File
@@ -0,0 +1,152 @@
# Worker Isolation
Nerve's worker architecture ensures complete isolation between different types of user code while maintaining system stability.
## Process Architecture
```
Kernel (Main Process)
├── Sense Worker (Group A) ── sense-1, sense-2
├── Sense Worker (Group B) ── sense-3, sense-4
├── Workflow Worker (cleanup) ── cleanup workflow instances
└── Workflow Worker (review) ── review workflow instances
```
## Isolation Boundaries
### 1. Sense Workers
- **One worker per sense group** (configured in `nerve.yaml`)
- Groups share a child process but have isolated execution contexts
- Crash in one sense doesn't affect other groups
- Each group has its own SQLite database
### 2. Workflow Workers
- **One worker per workflow type** (spawned on-demand)
- Multiple threads of the same workflow share a worker process
- Concurrency limits enforced at the workflow level
- Workers terminate when no active threads remain
### 3. Kernel Protection
- **User code never runs in kernel process**
- All `compute()` and workflow role functions run in workers
- Kernel only handles IPC, scheduling, and coordination
- System remains stable even with infinite loops or crashes in user code
## Worker Lifecycle
### Sense Workers
```
nerve daemon start → spawn worker per group → long-lived process
→ hot reload on file changes
→ respawn on crash
```
### Workflow Workers
```
workflow trigger → check existing worker → reuse or spawn
→ execute thread
→ terminate when idle
```
## Communication Patterns
### Kernel ↔ Sense Worker
- IPC via child process stdio
- JSON-formatted messages
- Worker reports signals back to kernel
- Bidirectional: kernel can request immediate computes
### Kernel ↔ Workflow Worker
- Similar IPC protocol
- Workflow definition loaded in worker
- Role execution results streamed back
- Thread state managed in kernel
## Resource Limits & Control
### Timeout Enforcement
Configurable timeouts per sense (in `nerve.yaml`):
```yaml
senses:
my-sense:
timeout: 30000 # Execution timeout (ms)
gracePeriod: 5000 # Grace period before hard kill
```
**Timeout Implementation:**
- `AbortController` for async operations
- `Promise.race()` between compute and timeout
- Grace period triggers `process.exit(1)` to kill entire worker group
### Memory & CPU Limits
**No Application-Level Resource Quotas**:
- No memory caps, CPU throttling, or disk I/O limits enforced by Nerve
- Workers can consume arbitrary system resources until OS limits
- No cgroup/container isolation — full filesystem access within user permissions
- No syscall filtering (no seccomp restrictions)
**OS-Level Constraints Only**:
- Process memory limited by system `ulimit -m`
- CPU usage bounded by scheduler only
- Network requests unrestricted
- Can spawn additional processes (not tracked by Nerve)
### Concurrency Control
#### Sense Workers
- One active compute per sense at a time (serialized via promise chains)
- No memory sharing between sense groups
- Crash isolation: one sense crash doesn't affect other groups
#### Workflow Workers
Per-workflow limits configured in `nerve.yaml`:
```yaml
workflows:
my-workflow:
concurrency: 2 # Max parallel threads
overflow: "drop" # or "queue"
maxQueue: 10 # Queue size limit
```
### Process Management
#### Signal Handling
Workers ignore session broadcast signals (SIGINT/SIGTERM):
```typescript
// Workers ignore terminal signals; kernel coordinates shutdown
process.on("SIGINT", () => {});
process.on("SIGTERM", () => {});
```
#### Graceful Shutdown & State Handoff
**Sense Workers**:
- IPC `shutdown` message → `process.exit(0)` (immediate)
- No graceful termination period for senses
- State rebuilt from SQLite on respawn (no handoff needed)
**Workflow Workers**:
- IPC `shutdown` → wait for in-flight threads to complete
- Drain timeout: `WORKER_SHUTDOWN_TIMEOUT_MS` (10s)
- If threads don't complete → `SIGKILL` force termination
- Thread state preserved in log store for crash recovery
**State Handoff Mechanism**:
- No explicit state transfer between old/new workers
- Sense workers: SQLite database contains full state
- Workflow workers: Log store contains thread message history
- Kernel coordinates recovery via `recoverThreadsForWorker()`
## Failure Handling
### Worker Crashes
- **Sense workers**: Automatic respawn after 1s delay, state rebuilt from DB
- **Workflow workers**: Crash recovery from log store thread messages
- **Kernel protection**: Main process continues, marks affected runs as crashed
- **Crash limits**: Max 5 crashes per workflow in 60s window (prevents infinite respawn)
### Resource Exhaustion
- **Memory**: Worker process killed by OS, kernel respawns automatically
- **Compute timeout**: Grace period → hard kill → respawn
- **Infinite loops**: Timeout enforcement prevents hanging indefinitely
This architecture allows Nerve to run untrusted or experimental code safely while maintaining system availability.
+80 -3
View File
@@ -2,12 +2,26 @@
Stateful multi-step execution driven by Roles and a Moderator.
## Workspace Layout (authoring)
User Nerve workspaces use a **flat** build: one root `package.json`, one root bundle script (typically `scripts/build.mjs` wired from `scripts.build`), and **no** per-workflow `package.json` or `tsconfig.json`.
| Location | Purpose |
|----------|---------|
| `workflows/<name>/index.ts` | Default export: `WorkflowDefinition` (moderator + role map). |
| `workflows/<name>/roles/<role>.ts` | One module per role — schemas, prompts, `createRole` factories, or hand-written async role functions. |
| `dist/workflows/<name>/index.js` | Emit of the root build; this is what the daemon loads. |
**Naming:** Workflow ids should be **verb-first** kebab-case phrases (e.g. `deploy-staging`, `scan-dependencies`), not opaque nouns alone.
Senses follow the same flat pattern: `senses/<name>/src/*.ts`, `migrations/`, root build → `dist/senses/<name>/index.js`. See `.knowledge/sense.md`.
## Core Concepts
- **Workflow** — definition with concurrency strategy
- **Thread** — one execution instance, unique `runId`
- **Role** — executes actions (has side effects). `(start, messages) → { content, meta }`
- **Moderator** — pure routing function. `(context) → next role | END`
- **Role** — executes actions (has side effects). `(ctx: ThreadContext) → Promise<RoleResult<M>>`
- **Moderator** — pure routing function. `(ctx: ThreadContext) → next role | END`
## Thread Lifecycle
@@ -54,6 +68,69 @@ const workflow: WorkflowDefinition<MyMeta> = {
```
- `adapter: AgentFn` — direct function reference
- `prompt: string | ((start, messages) => Promise<string>)` — static or dynamic
- `prompt: string | ((ctx: ThreadContext) => Promise<string>)` — static or dynamic
- `meta: z.ZodType<M>` — Zod schema, directly (no wrapper needed)
- `extract: LlmExtractorConfig` — provider for structured extraction
## Runtime Enforcement Mechanisms
### Role Authority & Validation
**Role Function Lookup**:
- Roles accessed via `def.roles[nextRole]` dictionary lookup
- Unknown roles trigger immediate workflow error (`Unknown role: ${nextRole}`)
- No dynamic role registration during execution
**Result Validation** (`validateRoleResult()`):
```typescript
// Required return shape from every role function
{ content: string, meta: Record<string, unknown> }
```
- `content` must be string (non-string → workflow error)
- `meta` must be plain object (array/null/primitive → workflow error)
- Validation failure terminates thread immediately
### Moderator Authority & Routing Control
**Next Role Selection**:
- Moderator must return role name from `roles` keys OR `END` symbol
- Called after every role completion (receives full context)
- No validation of role name until execution attempt
- Pure function constraint: cannot perform side effects
**Causal Chain Integrity**:
- Moderator receives immutable **ThreadContext**: `{ threadId, start, steps }`
- Steps array contains ALL role outputs in chronological order
- No role can modify prior steps or start metadata
- Thread context built from log store on crash recovery
### Unauthorized Command Event Prevention
**Message Flow Control**:
- Role functions have NO direct access to kernel IPC
- All outputs flow through `sendWorkflowMessage()` wrapper
- Worker process validates messages before kernel transmission
- No direct log store database access from roles
**Process Isolation**:
- Roles execute in forked worker processes (not kernel)
- File system access limited to user permissions
- No network isolation (roles can make arbitrary HTTP calls)
- Worker has read/write access to workflow workspace only
### Concurrent Thread Management
**Kill Flag Implementation**:
```typescript
type KillFlag = { value: boolean };
// Checked before role execution and after completion
if (killFlag.value) {
sendThreadEvent(runId, "killed", { exitCode: 137 });
return;
}
```
**Concurrency Enforcement**:
- Workflow manager enforces per-workflow limits in kernel
- Excess threads queued/dropped per overflow policy
- No role can spawn additional threads (no access to workflow manager)
+34 -10
View File
@@ -3,7 +3,7 @@
## Core Concepts
```
External World → Sense → Signal → Reflex → Workflow → Log
External World → Sense → Signal → Workflow → Log
↑ ↑
"what to observe" "what to do"
```
@@ -14,19 +14,18 @@ External World → Sense → Signal → Reflex → Workflow → Log
| Concept | What it is |
|---------|-----------|
| **Sense** | A `compute()` function that samples or derives data. Returns `T \| null` — non-null emits a Signal, null is silent. Each Sense has its own SQLite database. |
| **Sense** | A `compute()` function that samples or derives data. Returns `ComputeResult<T>` — non-null emits a Signal (and optionally triggers a Workflow), null is silent. Each Sense has its own SQLite database. Scheduling (interval, on) is configured in nerve.yaml. |
| **Signal** | A notification emitted when a Sense returns non-null. Pure fact, no intent. Distributed via an in-memory Signal Bus. Not persisted. |
| **Reflex** | A declarative trigger (YAML) connecting Senses to actions. Trigger types: `interval` (periodic), `on` (react to Signals). Action types: trigger a Sense, or start a Workflow. |
| **Workflow** | A stateful multi-step execution. Contains **Roles** (actors with side effects) and a **Moderator** (pure router). Each instance is a **Thread** with a unique `runId`. |
| **Log** | Immutable audit trail. Records executions, state transitions, errors. **Cannot trigger Reflexes** — prevents feedback loops. |
| **Engine** | The kernel orchestrating everything. Holds Signal Bus, Reflex Scheduler, Process Manager, Workflow Manager. Never loads user code directly — all user code runs in isolated Workers. |
| **Log** | Immutable audit trail. Records executions, state transitions, errors. Cannot trigger Senses — prevents feedback loops. |
| **Engine** | The kernel orchestrating everything. Holds Signal Bus, Process Manager, Workflow Manager. Never loads user code directly — all user code runs in isolated Workers. |
| **Daemon** | The `nerve-daemon` package — engine runtime. Runs as a background process. |
### Architecture Rules
- **Three orthogonal extension points**: Sense (what to compute), Reflex (when to compute), Workflow (what to do)
- **Two extension points**: Sense (what to observe + when), Workflow (what to do)
- **Process isolation**: One worker per Sense group (long-lived), one per Workflow type (on-demand). Workers never talk to each other.
- **Causality is one-directional**: External world → Sense → Signal → Reflex → Action + Log. Logs are the end of the chain.
- **Causality is one-directional**: External world → Sense → Signal → Workflow + Log. Logs are the end of the chain.
@@ -94,9 +93,34 @@ For mutually exclusive fields, use discriminated unions:
```typescript
// ✅ Good
type ReflexConfig =
| { kind: "sense"; sense: string; interval: string | null; on: string[] | null }
| { kind: "workflow"; workflow: string; on: string[] | null };
type ComputeResult<T> =
| null
| { signal: T; workflow: WorkflowTrigger | null };
```
### Workflow authoring (user modules)
Roles and moderators take **ThreadContext** (`threadId`, `start`, `steps`) — not separate `StartStep` / message arrays.
```typescript
import type { RoleResult, ThreadContext, WorkflowDefinition } from "@uncaged/nerve-core";
import { END } from "@uncaged/nerve-core";
type MyMeta = { round: number };
async function planner(ctx: ThreadContext): Promise<RoleResult<MyMeta>> {
void ctx.start;
void ctx.steps;
return { content: "plan", meta: { round: ctx.steps.length } };
}
const workflow: WorkflowDefinition<Record<"planner", MyMeta>> = {
name: "example",
roles: { planner },
moderator(ctx: ThreadContext<Record<"planner", MyMeta>>) {
return ctx.steps.length === 0 ? "planner" : END;
},
};
```
## Modules & Exports
+187
View File
@@ -0,0 +1,187 @@
# RFC-004: Package Architecture — Shareable Workflows, Roles & Senses
**Author:** 小橘 🍊(NEKO Team)
**Status:** Draft
**Created:** 2026-04-29
## Summary
Make workflows, roles, and senses publishable as lightweight npm packages. Workspaces become pure configuration — selecting packages, wiring adapters, and providing credentials. No builtin workflows in the nerve core.
## Motivation
Currently, workflows like `develop-sense` and `develop-workflow` live inside the workspace (`~/.uncaged-nerve/workflows/`). This creates problems:
1. **No sharing** — every workspace duplicates the same workflow code
2. **No versioning** — upgrading a workflow means manual file edits
3. **Builtin is a trap** — if we bake workflows into nerve core, they require adapters and LLM providers that may not be installed. A fresh `nerve` install on a bare machine would fail to load builtins.
4. **Roles are already shared**`_shared/workspace-committer.ts` proves the pattern works; we just need to formalize it as packages
The adapter pattern (`@uncaged/nerve-adapter-hermes`, `@uncaged/nerve-adapter-cursor`) already established the precedent: infrastructure as packages, workspace as wiring.
## Design
### Package Taxonomy
```
@uncaged/nerve-core # types, engine
@uncaged/nerve-daemon # runtime
@uncaged/nerve-workflow-utils # createRole, decorateRole, withDryRun, onFail, etc.
# Adapters (existing)
@uncaged/nerve-adapter-hermes
@uncaged/nerve-adapter-cursor
# Workflows (new)
@uncaged/nerve-workflow-solve-issue
@uncaged/nerve-workflow-develop-sense
@uncaged/nerve-workflow-develop-workflow
# Shared Roles (new)
@uncaged/nerve-role-committer # workspace committer (branch, commit, push)
@uncaged/nerve-role-reviewer # code review role
@uncaged/nerve-role-publisher # PR creation role
# Senses (existing pattern, formalized)
@uncaged/nerve-sense-cpu-usage
@uncaged/nerve-sense-disk-usage
```
### Package Contract
Each package type exports a factory function:
#### Workflow Package
```ts
// @uncaged/nerve-workflow-develop-sense
import type { AgentFn, WorkflowDefinition } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
export type SenseMeta = { /* ... */ };
export type CreateDevelopSenseDeps = {
defaultAdapter: AgentFn;
adapters?: Partial<Record<keyof SenseMeta, AgentFn>>;
extract: LlmExtractorConfig;
cwd: string;
};
export function createDevelopSenseWorkflow(deps: CreateDevelopSenseDeps): WorkflowDefinition<SenseMeta>;
```
Key design decisions:
- `defaultAdapter` + optional `adapters` override per role — via `Partial<Record<keyof Meta, AgentFn>>`
- Adapter keys are derived from `Meta` type — adding/removing a role automatically updates the adapter map
- Roles that don't need an agent simply don't appear in `adapters` (the `Partial` allows this)
#### Role Package
```ts
// @uncaged/nerve-role-committer
import type { AgentFn, Role } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { createRole, decorateRole, withDryRun, onFail } from "@uncaged/nerve-workflow-utils";
export type CommitterMeta = { committed: boolean };
export function createCommitterRole(adapter: AgentFn, extract: LlmExtractorConfig): Role<CommitterMeta> {
const inner = createRole(adapter, prompt, committerMetaSchema, extract);
return decorateRole(inner, [
withDryRun({ label: "committer", meta: { committed: true } }),
onFail({ label: "committer", meta: { committed: false } }),
]);
}
```
Roles compose with the decorator chain from `workflow-utils`:
- `withDryRun` — skip execution in dry-run mode
- `onFail` — catch errors into structured failure results
- `decorateRole(role, [...])` — apply decorators left-to-right
- Custom `RoleDecorator<M>` can be created for project-specific needs
#### Sense Package
```ts
// @uncaged/nerve-sense-cpu-usage
export const senseName = "cpu-usage";
export const schema = { /* drizzle schema */ };
export async function compute(ctx: SenseContext): Promise<SenseResult>;
```
### Workspace as Configuration
The workspace becomes a thin wiring layer:
```
~/.uncaged-nerve/
nerve.yaml # senses, extract config
package.json # depends on workflow/role/adapter packages
workflows/
develop-sense/
index.ts # ~10 lines: import package, wire adapters, export
solve-issue/
index.ts # same pattern
```
A typical `index.ts`:
```ts
import { createDevelopSenseWorkflow } from "@uncaged/nerve-workflow-develop-sense";
import { hermesAdapter } from "@uncaged/nerve-adapter-hermes";
import { cursorAdapter } from "@uncaged/nerve-adapter-cursor";
export default createDevelopSenseWorkflow({
defaultAdapter: hermesAdapter,
adapters: { planner: cursorAdapter, coder: cursorAdapter },
extract: { provider: { apiKey, baseUrl, model } },
cwd: nerveRoot,
});
```
### What Stays in Workspace
- **Custom workflows** — project-specific workflows that aren't general enough to share
- **Custom senses** — project-specific metrics
- **Configuration** — adapter selection, credentials, `nerve.yaml`
- **Overrides** — a workspace can always write its own role/workflow instead of using a package
### Dependency Rules
```
nerve-core ← no deps on other nerve packages
nerve-workflow-utils ← depends on nerve-core
nerve-adapter-* ← depends on nerve-core
nerve-role-* ← depends on nerve-core, nerve-workflow-utils
nerve-workflow-* ← depends on nerve-core, nerve-workflow-utils, may depend on nerve-role-*
nerve-sense-* ← depends on nerve-core
nerve-daemon ← depends on nerve-core, nerve-store
```
Workflow packages depend on role packages (not adapters). Adapters are injected at the workspace level.
### Migration Path
1. **Phase 1: Extract role packages** — Start with `@uncaged/nerve-role-committer` (already `_shared/workspace-committer.ts`). Publish, update workspace to import from package.
2. **Phase 2: Extract workflow packages** — Move `develop-sense` and `develop-workflow` to packages. Workspace `index.ts` becomes pure wiring.
3. **Phase 3: Sense packages** — Formalize sense packaging (lower priority, senses are already self-contained directories).
4. **Phase 4: Community** — Document the package contract so others can publish workflows/roles/senses.
### Not in Scope
- **No builtin workflows** — nerve core ships zero workflows. All workflows are packages installed by the workspace.
- **No workflow marketplace/registry** — just npm packages. `pnpm add @uncaged/nerve-workflow-solve-issue`.
- **No nerve.yaml workflow declaration** — workflows are still TypeScript entry points. The daemon discovers them the same way it does today.
## Open Questions
1. **Monorepo vs separate repos?** — Should workflow/role packages live in the nerve monorepo or separate repos? Monorepo is easier for coordinated releases; separate repos allow independent versioning.
2. **Sense package format** — Senses currently bundle with esbuild. Should sense packages ship pre-bundled or as TypeScript source?
3. **Version coupling** — How tightly should workflow packages pin `nerve-core`? Peer deps with semver range?
## Prior Art
- Adapter packages (`@uncaged/nerve-adapter-*`) — established the factory + injection pattern
- `_shared/workspace-committer.ts` — proved roles can be shared across workflows
- `createRole` / `decorateRole` / `withDryRun` / `onFail` in `workflow-utils` — building blocks that role packages compose
- `defaultAdapter` + `Partial<Record<keyof Meta, AgentFn>>` pattern — adapter injection without coupling
+13 -7
View File
@@ -1,4 +1,4 @@
import type { AgentConfig, AgentFn, WorkflowContext } from "@uncaged/nerve-core";
import type { AgentConfig, AgentFn, ThreadContext } from "@uncaged/nerve-core";
import { type Result, type SpawnEnv, type SpawnError, ok, spawnSafe } from "@uncaged/nerve-core";
export type CursorAgentMode = "plan" | "ask" | "default";
@@ -84,22 +84,28 @@ function throwCursorSpawnError(error: SpawnError): never {
/** Default adapter config: model auto-selection and 300s wall-clock cap (milliseconds). */
const CURSOR_ADAPTER_DEFAULT_MS = 300_000;
export type CursorAdapterConfig = AgentConfig & {
/** When set, passes `--mode=ask` or `--mode=plan` to `cursor-agent` (default runs without extra mode). */
mode?: CursorAgentMode;
};
/**
* Builds a Cursor CLI `AgentFn` from adapter config (model, timeout).
*/
export function createCursorAdapter(config: AgentConfig): AgentFn {
export function createCursorAdapter(config: CursorAdapterConfig): AgentFn {
const timeoutMs = config.timeout;
const mode = config.mode ?? "default";
return async (prompt: string, context: WorkflowContext): Promise<string> => {
return async (_ctx: ThreadContext, prompt: string): Promise<string> => {
const run = await cursorAgent({
prompt,
mode: "default",
mode,
model: config.model,
cwd: context.workdir,
cwd: process.cwd(),
env: null,
timeoutMs,
dryRun: context.start.meta.dryRun,
abortSignal: context.signal,
dryRun: false,
abortSignal: null,
});
if (!run.ok) {
throwCursorSpawnError(run.error);
+4 -4
View File
@@ -1,4 +1,4 @@
import type { AgentConfig, AgentFn, WorkflowContext } from "@uncaged/nerve-core";
import type { AgentConfig, AgentFn, ThreadContext } from "@uncaged/nerve-core";
import { type Result, type SpawnEnv, type SpawnError, ok, spawnSafe } from "@uncaged/nerve-core";
/**
@@ -96,7 +96,7 @@ export function createHermesAdapter(config: AgentConfig): AgentFn {
const modelFromConfig = config.model === "auto" ? null : config.model;
const timeoutMs = config.timeout;
return async (prompt: string, context: WorkflowContext): Promise<string> => {
return async (_ctx: ThreadContext, prompt: string): Promise<string> => {
const run = await hermesAgent({
prompt,
model: modelFromConfig,
@@ -106,8 +106,8 @@ export function createHermesAdapter(config: AgentConfig): AgentFn {
maxTurns: HERMES_ADAPTER_DEFAULT_MAX_TURNS,
env: null,
timeoutMs,
dryRun: context.start.meta.dryRun,
abortSignal: context.signal,
dryRun: false,
abortSignal: null,
});
if (!run.ok) {
throwHermesSpawnError(run.error);
+1
View File
@@ -17,6 +17,7 @@
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "rslib build",
"pretest": "pnpm --filter @uncaged/nerve-core run build && pnpm --filter @uncaged/nerve-daemon run build",
"test": "vitest run"
},
"dependencies": {
+505
View File
@@ -0,0 +1,505 @@
---
name: nerve
version: 0.5.0
description: >
Nerve — AI agent 观测引擎。掌握 nerve 的核心概念、CLI 操作、sense/workflow 开发。
加载此 skill 后你可以:查看系统状态、监控 sense、触发 workflow、开发新 sense 和 workflow。
metadata:
hermes:
tags: [nerve, sense, workflow, monitoring, agent-kernel]
homepage: https://git.shazhou.work/uncaged/nerve
---
# Nerve — AI Agent 观测引擎
Nerve 是一个轻量级观测引擎守护进程。它持续观测外部状态,通过声明式规则响应变化,编排多步骤工作流。
## 核心架构
```
External World → Sense → Signal → Workflow → Log
```
| 概念 | 说明 |
|------|------|
| **Sense** | 观测函数,`compute()` 采样或推导数据。返回非 null 则发出 Signal,可选触发 Workflow。每个 Sense 有独立 SQLite 数据库。 |
| **Signal** | Sense 返回非 null 时发出的通知。纯事实,无意图。通过内存 Signal Bus 分发,不持久化。 |
| **Workflow** | 有状态的多步骤执行。包含 Role(有副作用的执行者)和 Moderator(纯路由器)。每个实例是一个 Thread,有唯一 runId。 |
| **Log** | 不可变审计日志。记录执行、状态转换、错误。不能触发 Sense(防止反馈循环)。 |
| **Engine** | 内核,持有 Signal Bus、Process Manager、Workflow Manager。不直接加载用户代码。 |
| **Daemon** | 引擎运行时,作为后台进程运行。 |
**关键规则:**
- 因果链单向:External → Sense → Signal → Workflow + Log
- 进程隔离:每个 Sense group 一个 worker(长期),每个 Workflow 类型一个 worker(按需)
- 两个扩展点:Sense(观测什么 + 何时)、Workflow(做什么)
## 工作区结构
```
~/.uncaged-nerve/ # 默认工作区(nerve init 创建)
├── nerve.yaml # 核心配置
├── senses/
│ └── <name>/
│ ├── src/index.ts # exports compute() + table
│ ├── src/schema.ts # drizzle 表定义
│ └── migrations/ # SQL 迁移
├── workflows/
│ └── <name>/
│ ├── index.ts # exports WorkflowDefinition
│ └── roles/<role>/
│ ├── index.ts # role 实现
│ └── prompt.md # 可选 system prompt
└── data/ # 运行时数据(SQLite、blobs)
```
---
## CLI 完整参考
全局选项:`--host <host:port>`(连接远程 daemon)、`--api-token <secret>`(Bearer 认证)
### 初始化与脚手架
```bash
nerve init # 初始化工作区
nerve init --from <git-url> # 从 git 仓库克隆工作区
nerve init workspace # 只初始化工作区结构
nerve create sense <name> # 创建 sense 脚手架
nerve create sense <name> --force # 覆盖已有
nerve create workflow <name> # 创建 workflow 脚手架
nerve create workflow <name> --force
nerve validate # 验证 nerve.yaml 配置
```
### Daemon 管理
```bash
nerve daemon start # 启动后台 daemon
nerve daemon start --port 3000 # 指定 HTTP API 端口
nerve daemon stop # 停止 daemon
nerve daemon restart # 重启
nerve daemon status # 查看状态
nerve daemon logs # 查看日志
nerve daemon logs --follow # 实时日志
nerve daemon logs --n 50 # 最近 50 行
nerve dev # 前台开发模式(不 fork daemon)
nerve dev --port 3000 # 指定端口
```
### Sense 操作
```bash
nerve sense list # 列出所有注册的 sense
nerve sense trigger <name> # 手动触发 sense 计算
nerve sense schema <name> # 查看 sense 数据库表结构
nerve sense schema <name> --json # JSON 格式
nerve sense query <name> <sql> # 对 sense 数据库执行只读 SQL
nerve sense query <name> "SELECT * FROM samples ORDER BY ts DESC LIMIT 10" --json
```
### Workflow 操作
```bash
nerve workflow list # 列出 nerve.yaml 中定义的 workflow
nerve workflow status # 查看运行中的 workflow 状态
nerve workflow trigger <name> # 触发 workflow
nerve workflow trigger <name> --prompt "检查生产环境"
nerve workflow trigger <name> --maxRounds 50
nerve workflow trigger <name> --dryRun # 干跑模式
```
### Thread(Workflow 执行记录)
```bash
nerve thread list # 列出最近的 workflow 执行
nerve thread list --all # 包含已完成/失败的
nerve thread list --workflow <name> # 按 workflow 过滤
nerve thread list --limit 50 # 最多 50 条
nerve thread show <runId> # 查看 role 对话轮次
nerve thread show <runId> --budget 16000 # 增大输出预算(默认 8000 字符)
nerve thread inspect <runId> # 查看详情和事件
nerve thread kill <runId> # 终止运行中/排队中的 thread
```
### Store(日志归档)
```bash
nerve store archive # 导出旧日志到 JSONL 归档
nerve store archive --vacuum # 归档后 VACUUM 数据库
```
### Knowledge(知识库)
```bash
nerve knowledge sync # 从 knowledge.yaml 重建索引
nerve knowledge query "搜索内容" # 搜索知识库
nerve knowledge query "内容" --limit 5
nerve knowledge query "内容" -g # 搜索所有注册仓库
```
### Remote(远程 daemon)
```bash
nerve remote add <name> <host:port> --token <secret>
nerve remote list
nerve remote show <name>
nerve remote set-url <name> <host>
nerve remote set-token <name> <token>
nerve remote remove <name>
nerve remote default <name> # 设为默认远程
```
---
## nerve.yaml 配置参考
```yaml
# 引擎全局配置
max_rounds: 100 # moderator 最大轮次(默认 100)
# Sense 配置
senses:
cpu-usage:
group: system # 必填,同 group 的 sense 共享 worker
interval: 10s # 轮询间隔(duration: 5s, 10m, 1h)
throttle: 5s # 最小计算间隔
timeout: 10s # compute 超时
grace_period: null # 优雅关闭等待
retention: 10000 # _signals 表最大行数(默认 10000)
system-health:
group: derived
on: [cpu-usage, disk-usage] # 响应式:被列出的 sense 发出 signal 时触发
throttle: null
timeout: null
# Workflow 配置
workflows:
my-workflow:
concurrency: 1 # 必填,并发数
overflow: drop # 必填,超并发时处理:drop | queue
max_queue: 100 # overflow=queue 时的队列上限(默认 100)
# HTTP API
api:
port: 3000 # null = 不启用 HTTP
host: "127.0.0.1" # 监听地址
token: null # 非 loopback 时必填
# LLM Extract(可选)
extract:
provider: anthropic
model: claude-sonnet-4-20250514
```
---
## Sense 开发指南
### compute 函数签名
```typescript
import type { LibSQLDatabase } from "drizzle-orm/libsql";
import type { ComputeResult, WorkflowTrigger } from "@uncaged/nerve-core";
export async function compute(
db: LibSQLDatabase, // 此 sense 的 Drizzle ORM 数据库
peers: Record<string, LibSQLDatabase>, // 其他 sense 的数据库(只读)
options: { signal: AbortSignal }, // 超时 abort signal
): Promise<ComputeResult<T>>
```
### 返回值
```typescript
// 返回 null = 静默,不发 signal
// 返回非 null = 发出 signal,可选触发 workflow
type ComputeResult<T> =
| null
| { signal: T; workflow: WorkflowTrigger | null };
type WorkflowTrigger = {
name: string; // workflow 名称(对应 nerve.yaml 中的 key)
maxRounds: number; // moderator 最大轮次
prompt: string; // 初始 prompt
dryRun: boolean; // 干跑模式
};
```
### Sense 模块导出
```typescript
// senses/<name>/src/index.ts
import type { SenseModule, ComputeResult } from "@uncaged/nerve-core";
import { table } from "./schema.js";
export async function compute(
db: LibSQLDatabase,
_peers: Record<string, LibSQLDatabase>,
_options: { signal: AbortSignal },
): Promise<ComputeResult<number>> {
const value = Math.random(); // 替换为真实观测逻辑
await db.insert(table).values({ ts: Date.now(), value });
return { signal: value, workflow: null };
}
export { table };
```
### Schema 定义
```typescript
// senses/<name>/src/schema.ts
import { sqliteTable, integer, real } from "drizzle-orm/sqlite-core";
export const table = sqliteTable("samples", {
ts: integer("ts").notNull(),
value: real("value").notNull(),
});
```
### 调度方式
1. **interval 轮询**`interval: 10s` — 每 10 秒执行一次
2. **响应式触发**`on: [cpu-usage]` — 当 cpu-usage 发出 signal 时触发
3. 两者可以组合
### 调试
```bash
nerve dev # 前台运行,看实时输出
nerve sense trigger <name> # 手动触发一次
nerve sense query <name> "SELECT * FROM samples ORDER BY ts DESC LIMIT 5"
```
### 完整示例:CPU 监控
```typescript
// senses/cpu-usage/src/schema.ts
import { sqliteTable, integer, real } from "drizzle-orm/sqlite-core";
export const table = sqliteTable("samples", {
ts: integer("ts").notNull(),
value: real("value").notNull(),
});
// senses/cpu-usage/src/index.ts
import os from "node:os";
import type { LibSQLDatabase } from "drizzle-orm/libsql";
import type { ComputeResult } from "@uncaged/nerve-core";
import { table } from "./schema.js";
export async function compute(
db: LibSQLDatabase,
_peers: Record<string, LibSQLDatabase>,
_options: { signal: AbortSignal },
): Promise<ComputeResult<number>> {
const oneMin = os.loadavg()[0];
await db.insert(table).values({ ts: Date.now(), value: oneMin });
return { signal: oneMin, workflow: null };
}
export { table };
```
nerve.yaml:
```yaml
senses:
cpu-usage:
group: system
interval: 10s
throttle: 5s
timeout: 10s
retention: 10000
```
---
## Workflow 开发指南
### 核心类型
```typescript
import type {
WorkflowDefinition,
RoleResult,
ThreadContext,
StartStep,
RoleStep,
} from "@uncaged/nerve-core";
import { END } from "@uncaged/nerve-core";
// Role:执行者,接收上下文返回结果
type Role<Meta> = (ctx: ThreadContext) => Promise<RoleResult<Meta>>;
type RoleResult<Meta> = { content: string; meta: Meta };
// Moderator:路由器,决定下一个 role 或结束
type Moderator<M> = (ctx: ThreadContext<M>) => (keyof M & string) | typeof END;
// ThreadContext:对话上下文
type ThreadContext<M = RoleMeta> = {
threadId: string;
start: StartStep; // 初始 prompt(role: "__start__")
steps: RoleStep<M>[]; // 所有 role 的执行记录
};
// WorkflowDefinition:完整定义
type WorkflowDefinition<M> = {
name: string;
roles: { [K in keyof M & string]: Role<M[K]> };
moderator: Moderator<M>;
};
```
### 基本 Workflow 示例
```typescript
// workflows/example/index.ts
import type { RoleResult, ThreadContext, WorkflowDefinition } from "@uncaged/nerve-core";
import { END } from "@uncaged/nerve-core";
type Meta = Record<"main", { round: number }>;
async function main(ctx: ThreadContext): Promise<RoleResult<{ round: number }>> {
const prompt = ctx.start.content;
return {
content: `处理完成: ${prompt}`,
meta: { round: ctx.steps.length },
};
}
const workflow: WorkflowDefinition<Meta> = {
name: "example",
roles: { main },
moderator(ctx: ThreadContext<Meta>) {
// 执行一次 main 就结束
return ctx.steps.length === 0 ? "main" : END;
},
};
export default workflow;
```
### 多 Role Workflow 示例
```typescript
import type { WorkflowDefinition, RoleResult, ThreadContext } from "@uncaged/nerve-core";
import { END } from "@uncaged/nerve-core";
type Roles = Record<"planner" | "executor" | "reviewer", { status: string }>;
async function planner(ctx: ThreadContext): Promise<RoleResult<{ status: string }>> {
return { content: "计划: ...", meta: { status: "planned" } };
}
async function executor(ctx: ThreadContext): Promise<RoleResult<{ status: string }>> {
return { content: "执行: ...", meta: { status: "executed" } };
}
async function reviewer(ctx: ThreadContext): Promise<RoleResult<{ status: string }>> {
return { content: "审核通过", meta: { status: "approved" } };
}
const workflow: WorkflowDefinition<Roles> = {
name: "plan-execute-review",
roles: { planner, executor, reviewer },
moderator(ctx: ThreadContext<Roles>) {
if (ctx.steps.length === 0) return "planner";
const last = ctx.steps[ctx.steps.length - 1];
if (last.role === "planner") return "executor";
if (last.role === "executor") return "reviewer";
return END;
},
};
export default workflow;
```
### Agent 适配器
Workflow role 可以集成 AI agent。已知适配器 ID:`echo``cursor``hermes``codex`
```typescript
type AgentFn = (ctx: ThreadContext, systemPrompt: string) => Promise<string>;
```
### Workflow 运行状态
`queued``started``completed` | `failed` | `crashed` | `killed` | `interrupted` | `dropped`
---
## 日常操作 Pattern
### 查看系统整体状态
```bash
nerve daemon status # daemon 是否在运行
nerve sense list # 所有 sense 及其调度配置
nerve workflow status # 运行中的 workflow
nerve thread list # 最近的 workflow 执行记录
```
### 检查某个 sense 的历史数据
```bash
nerve sense query cpu-usage "SELECT * FROM samples ORDER BY ts DESC LIMIT 10" --json
nerve sense schema cpu-usage # 查看表结构
```
### 手动触发 workflow
```bash
nerve workflow trigger my-workflow --prompt "手动检查"
nerve thread list --workflow my-workflow # 查看执行状态
nerve thread show <runId> # 查看对话详情
```
### 排查 sense 报错
```bash
nerve daemon logs --follow # 查看实时日志
nerve sense trigger <name> # 手动触发看报错
nerve dev # 前台模式,更详细的输出
```
### 开发新 sense
```bash
nerve create sense my-sensor # 脚手架
# 编辑 senses/my-sensor/src/index.ts 和 schema.ts
nerve validate # 验证配置
nerve dev # 前台测试
nerve sense trigger my-sensor # 单次触发验证
nerve sense query my-sensor "SELECT * FROM ..." # 检查数据
```
### 开发新 workflow
```bash
nerve create workflow my-flow # 脚手架
# 编辑 workflows/my-flow/index.ts 和 roles/
nerve validate # 验证配置
nerve workflow trigger my-flow --prompt "测试" --dryRun # 干跑
nerve thread show <runId> # 查看执行轨迹
```
---
## Pitfalls
- **Sense 返回值**:返回 `null` 表示静默(不发 signal);返回 `{ signal, workflow }` 才发 signal。不要返回 undefined。
- **no optional properties**:nerve 代码规范禁止 `?:`,用 `T | null` 代替。
- **函数式风格**:用 `function` + `type`,不用 `class` + `interface`
- **workflow 用 default export**:这是唯一允许 default export 的场景。
- **_signals 表**:每个 sense 自动有 `_signals` 表记录 signal 历史,受 `retention` 配置限制。
- **peers 只读**:sense 的 `peers` 参数提供其他 sense 数据库的只读访问,不要写入。
- **concurrency + overflow**:workflow 必须配置并发策略,否则验证失败。
- **moderator 是同步函数**:不要加 async,moderator 是纯路由逻辑,不能有副作用。
@@ -7,7 +7,6 @@ import { describe, expect, it } from "vitest";
import {
buildSenseIndexTs,
buildSenseMigrationSql,
buildSensePackageJson,
buildSenseSchemaTs,
validateResourceName,
} from "../commands/create.js";
@@ -46,20 +45,11 @@ describe("buildSenseMigrationSql", () => {
});
});
describe("buildSensePackageJson", () => {
it("includes esbuild script and sense name", () => {
const pkg = JSON.parse(buildSensePackageJson("my-sense"));
expect(pkg.name).toBe("nerve-sense-my-sense");
expect(pkg.scripts.build).toContain("esbuild");
expect(pkg.scripts.build).toContain("src/index.ts");
expect(pkg.devDependencies.esbuild).toBeTruthy();
});
});
describe("buildSenseIndexTs", () => {
it("embeds sense id in stub with TypeScript types", () => {
const ts = buildSenseIndexTs("my-sense");
expect(ts).toContain("my-sense");
expect(ts).toContain("export { mySense as table }");
expect(ts).toContain("export async function compute");
expect(ts).toContain("LibSQLDatabase");
expect(ts).toContain("Promise<SenseResult>");
@@ -9,7 +9,7 @@ import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "nod
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { buildWorkflowPackageJson, buildWorkflowScaffold } from "../commands/create.js";
import { buildWorkflowScaffold } from "../commands/create.js";
let tmpDir: string;
@@ -33,9 +33,11 @@ describe("buildWorkflowScaffold", () => {
expect(indexTs).toContain("@uncaged/nerve-core");
});
it("root index wires moderator and END", () => {
it("root index wires moderator with ThreadContext and END", () => {
const { indexTs } = buildWorkflowScaffold("test");
expect(indexTs).toContain("moderator");
expect(indexTs).toContain("ThreadContext");
expect(indexTs).toContain("ctx.steps.length");
expect(indexTs).toContain("END");
});
@@ -46,9 +48,13 @@ describe("buildWorkflowScaffold", () => {
expect(indexTs).toContain("./roles/main/index.js");
});
it("main role module exports mainRole function", () => {
it("main role module exports mainRole with ThreadContext", () => {
const { roleMainIndexTs } = buildWorkflowScaffold("test");
expect(roleMainIndexTs).toContain("export async function mainRole");
expect(roleMainIndexTs).toContain("ThreadContext");
expect(roleMainIndexTs).toContain("RoleResult");
expect(roleMainIndexTs).not.toContain("StartStep");
expect(roleMainIndexTs).not.toContain("WorkflowMessage");
});
it("uses different names per call", () => {
@@ -75,21 +81,6 @@ describe("buildWorkflowScaffold", () => {
const { roleMainPromptMd } = buildWorkflowScaffold("my-flow");
expect(roleMainPromptMd).toContain("# my-flow — main role");
});
it("package.json defines esbuild bundling to dist/", () => {
const pkg = JSON.parse(buildWorkflowPackageJson("my-flow")) as {
scripts: { build: string };
devDependencies: { esbuild: string };
};
expect(pkg.scripts.build).toContain("esbuild");
expect(pkg.scripts.build).toContain("--outdir=dist");
expect(pkg.devDependencies.esbuild).toBeTruthy();
});
it("buildWorkflowScaffold includes package.json body", () => {
const { packageJson } = buildWorkflowScaffold("wf");
expect(JSON.parse(packageJson).scripts.build).toContain("esbuild");
});
});
describe("workflow scaffold file writing (simulated)", () => {
+13 -18
View File
@@ -122,54 +122,49 @@ describe("e2e create", () => {
});
it(
"create workflow scaffolds sources and package.json with esbuild build",
{ timeout: 10_000 },
"create workflow scaffolds sources and root build emits dist/workflows/<name>/index.js",
{ timeout: 120_000 },
async () => {
fakeHome = mkdtempSync(join(tmpdir(), "nerve-create-e2e-"));
const nerveRoot = join(fakeHome, ".uncaged-nerve");
await runTestCli(fakeHome, ["init", "--force", "--skip-install"]);
await runTestCli(fakeHome, ["init", "--force"]);
const wf = await runTestCli(fakeHome, ["create", "workflow", "e2e-flow"]);
expect(wf.exitCode).toBe(0);
expect(wf.stdout).toContain("✅");
const pkgPath = join(nerveRoot, "workflows", "e2e-flow", "package.json");
const indexPath = join(nerveRoot, "workflows", "e2e-flow", "index.ts");
const mainRolePath = join(nerveRoot, "workflows", "e2e-flow", "roles", "main", "index.ts");
expect(existsSync(pkgPath)).toBe(true);
expect(JSON.parse(readFileSync(pkgPath, "utf8")).scripts.build).toContain("esbuild");
const wfDir = join(nerveRoot, "workflows", "e2e-flow");
const indexPath = join(wfDir, "index.ts");
const mainRolePath = join(wfDir, "roles", "main", "index.ts");
expect(existsSync(join(wfDir, "package.json"))).toBe(false);
expect(existsSync(indexPath)).toBe(true);
expect(existsSync(mainRolePath)).toBe(true);
expect(readFileSync(indexPath, "utf8")).toContain('name: "e2e-flow"');
expect(readFileSync(mainRolePath, "utf8")).toContain("e2e-flow started");
expect(existsSync(join(nerveRoot, "dist", "workflows", "e2e-flow", "index.js"))).toBe(true);
},
);
it(
"create sense scaffolds src/index.ts, src/schema.ts, package.json and migration",
{ timeout: 60_000 },
"create sense scaffolds src/, migration, and root build emits dist/senses/<name>/index.js",
{ timeout: 120_000 },
async () => {
fakeHome = mkdtempSync(join(tmpdir(), "nerve-create-e2e-"));
const nerveRoot = join(fakeHome, ".uncaged-nerve");
await runTestCli(fakeHome, ["init", "--force", "--skip-install"]);
await runTestCli(fakeHome, ["init", "--force"]);
const sense = await runTestCli(fakeHome, ["create", "sense", "e2e-sense"]);
expect(sense.exitCode).toBe(0);
expect(sense.stdout).toContain("✅");
const base = join(nerveRoot, "senses", "e2e-sense");
expect(existsSync(join(base, "package.json"))).toBe(true);
expect(existsSync(join(base, "package.json"))).toBe(false);
expect(existsSync(join(base, "src", "index.ts"))).toBe(true);
expect(existsSync(join(base, "src", "schema.ts"))).toBe(true);
expect(existsSync(join(base, "migrations", "0001_init.sql"))).toBe(true);
const pkg = JSON.parse(readFileSync(join(base, "package.json"), "utf8"));
expect(pkg.scripts.build).toContain("esbuild");
// pnpm install + build should produce index.js
expect(existsSync(join(base, "index.js"))).toBe(true);
expect(existsSync(join(nerveRoot, "dist", "senses", "e2e-sense", "index.js"))).toBe(true);
},
);
+79 -22
View File
@@ -37,7 +37,15 @@
* ```
*/
import { existsSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
import {
existsSync,
mkdirSync,
mkdtempSync,
readFileSync,
rmSync,
symlinkSync,
writeFileSync,
} from "node:fs";
import { createRequire } from "node:module";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
@@ -61,6 +69,27 @@ const nerveDaemonRoot = dirname(require.resolve("@uncaged/nerve-daemon/package.j
const senseWorkerScript = join(nerveDaemonRoot, "dist", "sense-worker.js");
const workflowWorkerScript = join(nerveDaemonRoot, "dist", "workflow-worker.js");
function resolveDrizzleOrmPackageRoot(): string {
const requireFromDaemon = createRequire(join(nerveDaemonRoot, "package.json"));
const entry = requireFromDaemon.resolve("drizzle-orm");
let dir = dirname(entry);
for (let i = 0; i < 12; i += 1) {
const pkgPath = join(dir, "package.json");
if (existsSync(pkgPath)) {
try {
const name = (JSON.parse(readFileSync(pkgPath, "utf8")) as { name: string }).name;
if (name === "drizzle-orm") return dir;
} catch {
// keep walking
}
}
const parent = dirname(dir);
if (parent === dir) break;
dir = parent;
}
throw new Error("Could not resolve drizzle-orm package root for e2e harness");
}
const nerveYamlTemplate = `senses:
counter:
group: e2e
@@ -88,9 +117,9 @@ const echoWorkflowIndexJs = `const END = "__end__";
export default {
name: "echo",
roles: {
echo: async (start, _messages) => {
echo: async (ctx) => {
await new Promise((r) => setTimeout(r, 350));
const p = typeof start.content === "string" ? start.content : "";
const p = typeof ctx.start.content === "string" ? ctx.start.content : "";
return {
content: p.length > 0 ? "echo:" + p : "echo:empty",
meta: {},
@@ -121,17 +150,30 @@ api:
host: 127.0.0.1
`;
/** Empty migration — counter sense uses only `_signals` (auto-created by daemon). */
const counterMigration = `-- no-op migration for e2e counter sense
SELECT 1;
/** Schema for sense signal rows persisted via \`db.insert(table)\` (see sense-runtime). */
const counterMigration = `CREATE TABLE IF NOT EXISTS counter_signals (
id INTEGER PRIMARY KEY AUTOINCREMENT,
count INTEGER,
launched INTEGER,
idle INTEGER
);
`;
/**
* Minimal counter sense — each compute returns an incrementing count.
* Does NOT touch the DB directly; signal persistence is handled by the daemon
* (`runtime.persistSignal`) which writes to `_signals` automatically.
* Does NOT touch the DB directly in compute(); the daemon inserts into \`table\`
* and persistSignal handles \`_signals\`.
*/
const counterIndexJs = `let _count = 0;
const counterIndexJs = `import { integer, sqliteTable } from "drizzle-orm/sqlite-core";
export const table = sqliteTable("counter_signals", {
id: integer("id").primaryKey({ autoIncrement: true }),
count: integer("count"),
launched: integer("launched"),
idle: integer("idle"),
});
let _count = 0;
export async function compute(_db, _peers, _options) {
_count += 1;
return { signal: { count: _count }, workflow: null };
@@ -139,12 +181,21 @@ export async function compute(_db, _peers, _options) {
`;
/** First trigger launches local noop workflow; later triggers emit a plain signal. */
const counterIndexJsWithNoopWorkflow = `let _launched = false;
const counterIndexJsWithNoopWorkflow = `import { integer, sqliteTable } from "drizzle-orm/sqlite-core";
export const table = sqliteTable("counter_signals", {
id: integer("id").primaryKey({ autoIncrement: true }),
count: integer("count"),
launched: integer("launched"),
idle: integer("idle"),
});
let _launched = false;
export async function compute(_db, _peers, _options) {
if (!_launched) {
_launched = true;
return {
signal: { launched: true },
signal: { launched: 1 },
workflow: {
name: "noop",
maxRounds: 3,
@@ -153,7 +204,7 @@ export async function compute(_db, _peers, _options) {
},
};
}
return { signal: { idle: true }, workflow: null };
return { signal: { idle: 1 }, workflow: null };
}
`;
@@ -209,7 +260,8 @@ function writeWorkspaceLayout(nerveRoot: string, withNoopWorkflow: boolean): voi
mkdirSync(join(nerveRoot, "data", "senses"), { recursive: true });
mkdirSync(join(nerveRoot, "data", "blobs"), { recursive: true });
mkdirSync(join(nerveRoot, "senses", "counter", "migrations"), { recursive: true });
mkdirSync(join(nerveRoot, "workflows", "echo", "dist"), { recursive: true });
mkdirSync(join(nerveRoot, "dist", "senses", "counter"), { recursive: true });
mkdirSync(join(nerveRoot, "dist", "workflows", "echo"), { recursive: true });
writeFileSync(
join(nerveRoot, "nerve.yaml"),
withNoopWorkflow ? nerveYamlWithNoopWorkflow : nerveYamlTemplate,
@@ -221,20 +273,19 @@ function writeWorkspaceLayout(nerveRoot: string, withNoopWorkflow: boolean): voi
"utf8",
);
writeFileSync(
join(nerveRoot, "senses", "counter", "index.js"),
join(nerveRoot, "dist", "senses", "counter", "index.js"),
withNoopWorkflow ? counterIndexJsWithNoopWorkflow : counterIndexJs,
"utf8",
);
writeFileSync(
join(nerveRoot, "workflows", "echo", "dist", "index.js"),
join(nerveRoot, "dist", "workflows", "echo", "index.js"),
echoWorkflowIndexJs,
"utf8",
);
if (withNoopWorkflow) {
mkdirSync(join(nerveRoot, "workflows", "noop", "dist"), { recursive: true });
mkdirSync(join(nerveRoot, "workflows", "noop", "migrations"), { recursive: true });
mkdirSync(join(nerveRoot, "dist", "workflows", "noop"), { recursive: true });
writeFileSync(
join(nerveRoot, "workflows", "noop", "dist", "index.js"),
join(nerveRoot, "dist", "workflows", "noop", "index.js"),
noopWorkflowIndexJs,
"utf8",
);
@@ -267,11 +318,17 @@ function useNoopWorkflow(opts: StartTestDaemonOpts): boolean {
*/
export function linkWorkspaceDaemonIntoNerveRoot(nerveRoot: string): void {
const daemonPkgRoot = dirname(require.resolve("@uncaged/nerve-daemon/package.json"));
const linkDir = join(nerveRoot, "node_modules", "@uncaged");
const linkPath = join(linkDir, "nerve-daemon");
const nm = join(nerveRoot, "node_modules");
mkdirSync(nm, { recursive: true });
const linkDir = join(nm, "@uncaged");
mkdirSync(linkDir, { recursive: true });
if (existsSync(linkPath)) return;
symlinkSync(daemonPkgRoot, linkPath);
const linkPath = join(linkDir, "nerve-daemon");
if (!existsSync(linkPath)) symlinkSync(daemonPkgRoot, linkPath);
const drizzlePkgRoot = resolveDrizzleOrmPackageRoot();
const drizzleLink = join(nm, "drizzle-orm");
if (!existsSync(drizzleLink)) symlinkSync(drizzlePkgRoot, drizzleLink);
}
/**
@@ -202,10 +202,13 @@ describe("e2e init", () => {
// Verify key files exist
expect(existsSync(join(nerveRoot, "nerve.yaml"))).toBe(true);
expect(existsSync(join(nerveRoot, "package.json"))).toBe(true);
expect(existsSync(join(nerveRoot, "pnpm-workspace.yaml"))).toBe(true);
expect(existsSync(join(nerveRoot, "scripts", "build.mjs"))).toBe(true);
expect(existsSync(join(nerveRoot, "biome.json"))).toBe(true);
expect(existsSync(join(nerveRoot, ".gitignore"))).toBe(true);
expect(existsSync(join(nerveRoot, "senses", "cpu-usage", "package.json"))).toBe(true);
expect(existsSync(join(nerveRoot, "AGENT.md"))).toBe(true);
const agentMd = readFileSync(join(nerveRoot, "AGENT.md"), "utf8");
expect(agentMd).toContain("verb-first");
expect(agentMd).toContain("createRole");
expect(existsSync(join(nerveRoot, "senses", "cpu-usage", "src", "index.ts"))).toBe(true);
expect(existsSync(join(nerveRoot, "senses", "cpu-usage", "src", "schema.ts"))).toBe(true);
expect(existsSync(join(nerveRoot, "senses", "cpu-usage", "migrations", "0001_init.sql"))).toBe(
@@ -214,19 +217,14 @@ describe("e2e init", () => {
expect(existsSync(join(nerveRoot, ".cursor", "rules", "nerve-skills.mdc"))).toBe(true);
const pkgJson = readFileSync(join(nerveRoot, "package.json"), "utf8");
expect(pkgJson).toContain('"@uncaged/nerve-skills": "latest"');
expect(pkgJson).toContain('"build": "pnpm -r build"');
expect(pkgJson).not.toContain("nerve-skills");
expect(pkgJson).toContain('"build": "node scripts/build.mjs"');
expect(pkgJson).toContain('"esbuild": "^0.27.0"');
const workspaceYaml = readFileSync(join(nerveRoot, "pnpm-workspace.yaml"), "utf8");
expect(workspaceYaml).toContain("workflows/*");
expect(workspaceYaml).toContain("senses/*");
const sensePkgJson = readFileSync(
join(nerveRoot, "senses", "cpu-usage", "package.json"),
"utf8",
);
expect(sensePkgJson).toContain("nerve-sense-cpu-usage");
expect(sensePkgJson).toContain("esbuild");
const buildScript = readFileSync(join(nerveRoot, "scripts", "build.mjs"), "utf8");
expect(buildScript).toContain('path.join(root, "senses")');
expect(buildScript).toContain('path.join(root, "workflows")');
expect(buildScript).toContain("dist");
});
it("generated nerve.yaml passes validate", { timeout: 10_000 }, async () => {
@@ -1,5 +1,5 @@
/**
* RFC-003 Phase 5: nerve validate — WorkflowSpec adapter usage and extract.
* RFC-003 Phase 5: nerve validate — workflow adapter usage and extract.
*/
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
@@ -38,9 +38,8 @@ describe("validateAgentConfigurationLayer", () => {
writeFileSync(
join(nerveRoot, "workflows", "demo", "src", "index.ts"),
`
import type { WorkflowSpec } from "@uncaged/nerve-core";
const adapter = async () => "";
const spec: WorkflowSpec<{ r: { x: number } }> = {
const spec = {
name: "demo",
roles: {
r: { adapter: adapter, prompt: "p", meta: {} as never },
+33 -66
View File
@@ -20,39 +20,18 @@ export type WorkflowScaffoldFiles = {
indexTs: string;
roleMainIndexTs: string;
roleMainPromptMd: string;
packageJson: string;
};
export function buildWorkflowPackageJson(name: string): string {
return `${JSON.stringify(
{
name: `nerve-workflow-${name}`,
private: true,
type: "module",
scripts: {
build:
"esbuild index.ts --bundle --platform=node --format=esm --outdir=dist --packages=external",
},
devDependencies: {
esbuild: "^0.27.0",
},
},
null,
2,
)}\n`;
}
export function buildWorkflowScaffold(name: string): WorkflowScaffoldFiles {
return {
indexTs: buildWorkflowIndexTs(name),
roleMainIndexTs: buildWorkflowMainRoleIndexTs(name),
roleMainPromptMd: buildWorkflowMainRolePromptMd(name),
packageJson: buildWorkflowPackageJson(name),
};
}
function buildWorkflowIndexTs(name: string): string {
return `import type { WorkflowDefinition } from "@uncaged/nerve-core";
return `import type { ThreadContext, WorkflowDefinition } from "@uncaged/nerve-core";
import { END } from "@uncaged/nerve-core";
import { mainRole } from "./roles/main/index.js";
@@ -64,8 +43,8 @@ const workflow: WorkflowDefinition<Record<"main", MainMeta>> = {
roles: {
main: mainRole,
},
moderator({ steps }) {
if (steps.length === 0) {
moderator(ctx: ThreadContext<Record<"main", MainMeta>>) {
if (ctx.steps.length === 0) {
return "main";
}
return END;
@@ -77,18 +56,16 @@ export default workflow;
}
function buildWorkflowMainRoleIndexTs(name: string): string {
return `import type { RoleResult, StartStep, WorkflowMessage } from "@uncaged/nerve-core";
return `import type { RoleResult, ThreadContext } from "@uncaged/nerve-core";
/**
* Main role — implement LLM calls, scripts, HTTP, etc.
* Optional: align behavior with \`prompt.md\` in this directory.
*/
export async function mainRole(
start: StartStep,
messages: WorkflowMessage[],
ctx: ThreadContext,
): Promise<RoleResult<Record<string, unknown>>> {
void start;
void messages;
void ctx;
// TODO: implement your role logic here
return {
content: "${name} started",
@@ -134,32 +111,14 @@ export const ${exportName} = sqliteTable("${table}", {
`;
}
export function buildSensePackageJson(name: string): string {
return `${JSON.stringify(
{
name: `nerve-sense-${name}`,
private: true,
type: "module",
scripts: {
build:
"esbuild src/index.ts --bundle --platform=node --format=esm --outdir=. --out-extension:.js=.js --packages=external",
},
devDependencies: {
esbuild: "^0.27.0",
"drizzle-orm": "*",
},
},
null,
2,
)}\n`;
}
export function buildSenseIndexTs(senseId: string): string {
const exportName = senseIdToSchemaExportName(senseId);
return `import type { LibSQLDatabase } from "drizzle-orm/libsql";
import { ${exportName} } from "./schema.js";
export { ${exportName} as table } from "./schema.js";
type SenseResult = {
signal: { label: string; ts: number };
workflow: null;
@@ -247,30 +206,39 @@ const createWorkflowCommand = defineCommand({
mkdirSync(workflowDir, { recursive: true });
const scaffold = buildWorkflowScaffold(args.name);
writeFile(join(workflowDir, "package.json"), scaffold.packageJson);
writeFile(join(workflowDir, "index.ts"), scaffold.indexTs);
writeFile(join(workflowDir, "roles", "main", "index.ts"), scaffold.roleMainIndexTs);
writeFile(join(workflowDir, "roles", "main", "prompt.md"), scaffold.roleMainPromptMd);
process.stdout.write("✅ Workflow scaffolded:\n");
process.stdout.write(` ${join(workflowDir, "package.json")}\n`);
process.stdout.write(` ${join(workflowDir, "index.ts")}\n`);
process.stdout.write(` ${join(workflowDir, "roles", "main", "index.ts")}\n`);
process.stdout.write(` ${join(workflowDir, "roles", "main", "prompt.md")}\n`);
process.stdout.write("\nBuilding workspace (workflows + senses)…\n");
try {
await spawnAsync("pnpm", ["run", "build"], nerveRoot);
process.stdout.write(
`✅ Build complete — ${join("dist", "workflows", args.name, "index.js")} ready.\n`,
);
} catch {
process.stdout.write(`⚠️ Build failed. Run manually:\n cd ${nerveRoot} && pnpm run build\n`);
}
process.stdout.write("\n💡 Next steps:\n");
process.stdout.write(
` 1. In ${workflowDir}, run \`npm install\` then \`npm run build\` (bundles to dist/index.js).\n`,
);
process.stdout.write(" 2. Add to nerve.yaml:\n");
process.stdout.write(" 1. Add to nerve.yaml:\n");
process.stdout.write(" workflows:\n");
process.stdout.write(` ${args.name}:\n`);
process.stdout.write(" concurrency: 1\n");
process.stdout.write(" overflow: drop\n");
process.stdout.write(
` 3. Edit ${join(workflowDir, "roles", "main", "index.ts")} (and optional prompt.md).\n`,
` 2. Edit ${join(workflowDir, "roles", "main", "index.ts")} (and optional prompt.md).\n`,
);
process.stdout.write(
` 4. Adjust moderator routing in ${join(workflowDir, "index.ts")} if you add roles.\n`,
` 3. Adjust moderator routing in ${join(workflowDir, "index.ts")} if you add roles.\n`,
);
process.stdout.write(
` 4. After edits, run \`pnpm run build\` from the workspace root (${nerveRoot}); output is dist/workflows/<name>/index.js.\n`,
);
process.stdout.write(" 5. Run `nerve start` to launch the daemon.\n");
},
@@ -311,26 +279,23 @@ const createSenseCommand = defineCommand({
mkdirSync(join(senseDir, "src"), { recursive: true });
mkdirSync(join(senseDir, "migrations"), { recursive: true });
writeFile(join(senseDir, "package.json"), buildSensePackageJson(args.name));
writeFile(join(senseDir, "src", "index.ts"), buildSenseIndexTs(args.name));
writeFile(join(senseDir, "src", "schema.ts"), buildSenseSchemaTs(args.name));
writeFile(join(senseDir, "migrations", "0001_init.sql"), buildSenseMigrationSql(args.name));
process.stdout.write("✅ Sense scaffolded:\n");
process.stdout.write(` ${join(senseDir, "package.json")}\n`);
process.stdout.write(` ${join(senseDir, "src", "index.ts")}\n`);
process.stdout.write(` ${join(senseDir, "src", "schema.ts")}\n`);
process.stdout.write(` ${join(senseDir, "migrations", "0001_init.sql")}\n`);
process.stdout.write("\nInstalling sense dependencies and building…\n");
process.stdout.write("\nBuilding workspace (senses + workflows)…\n");
try {
await spawnAsync("pnpm", ["install", "--no-cache", "--ignore-workspace"], senseDir);
await spawnAsync("pnpm", ["run", "build"], senseDir);
process.stdout.write("✅ Build complete — index.js ready.\n");
} catch {
await spawnAsync("pnpm", ["run", "build"], nerveRoot);
process.stdout.write(
`⚠️ Build failed. Run manually:\n cd ${senseDir} && pnpm install --no-cache --ignore-workspace && pnpm run build\n`,
` Build complete — ${join("dist", "senses", args.name, "index.js")} ready.\n`,
);
} catch {
process.stdout.write(`⚠️ Build failed. Run manually:\n cd ${nerveRoot} && pnpm run build\n`);
}
process.stdout.write("\n💡 Next steps:\n");
@@ -343,7 +308,9 @@ const createSenseCommand = defineCommand({
process.stdout.write(
` 2. Edit ${join(senseDir, "src", "index.ts")} to implement ${args.name}.\n`,
);
process.stdout.write(` 3. Re-run \`pnpm run build\` in ${senseDir} after edits.\n`);
process.stdout.write(
` 3. Re-run \`pnpm run build\` from the workspace root (${nerveRoot}) after edits.\n`,
);
process.stdout.write(" 4. Run `nerve start` to launch the daemon.\n");
},
});
+131 -42
View File
@@ -17,11 +17,6 @@ senses:
interval: 10s
`;
const PNPM_WORKSPACE_YAML = `packages:
- 'workflows/*'
- 'senses/*'
`;
const BIOME_JSON = `{
"$schema": "https://biomejs.dev/schemas/1.9.0/schema.json",
"formatter": {
@@ -54,17 +49,20 @@ const PACKAGE_JSON = `${JSON.stringify(
private: true,
type: "module",
scripts: {
build: "pnpm -r build",
build: "node scripts/build.mjs",
},
dependencies: {
"@uncaged/nerve-core": "latest",
"@uncaged/nerve-daemon": "latest",
"@uncaged/nerve-skills": "latest",
"drizzle-orm": "latest",
zod: "^4.3.6",
},
devDependencies: {
"@biomejs/biome": "latest",
"@types/node": "^22.0.0",
"drizzle-kit": "latest",
esbuild: "^0.27.0",
typescript: "^5.7.0",
},
pnpm: {
onlyBuiltDependencies: ["esbuild"],
@@ -74,6 +72,54 @@ const PACKAGE_JSON = `${JSON.stringify(
2,
)}\n`;
const BUILD_MJS = `import * as esbuild from "esbuild";
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), "..");
const dist = path.join(root, "dist");
const opts = {
bundle: true,
platform: "node",
format: "esm",
packages: "external",
};
function listDirs(dir) {
if (!fs.existsSync(dir)) return [];
return fs
.readdirSync(dir)
.filter((name) => !name.startsWith(".") && !name.startsWith("_"))
.map((name) => ({ name, full: path.join(dir, name) }))
.filter(({ full }) => fs.statSync(full).isDirectory());
}
async function main() {
// Clean dist/
fs.rmSync(dist, { recursive: true, force: true });
for (const { name, full } of listDirs(path.join(root, "senses"))) {
const entry = path.join(full, "src", "index.ts");
if (!fs.existsSync(entry)) continue;
const outfile = path.join(dist, "senses", name, "index.js");
fs.mkdirSync(path.dirname(outfile), { recursive: true });
await esbuild.build({ ...opts, entryPoints: [entry], outfile });
}
for (const { name, full } of listDirs(path.join(root, "workflows"))) {
const entry = path.join(full, "index.ts");
if (!fs.existsSync(entry)) continue;
const outfile = path.join(dist, "workflows", name, "index.js");
fs.mkdirSync(path.dirname(outfile), { recursive: true });
await esbuild.build({ ...opts, entryPoints: [entry], outfile });
}
}
await main();
`;
const GITIGNORE = `data/
logs/
nerve.pid
@@ -81,33 +127,92 @@ node_modules/
knowledge.db
`;
/** Generated at workspace root so agents can \`cat AGENT.md\` instead of npm skill paths. */
const AGENT_MD = `# Nerve workspace — agent guide
This file is created by \`nerve init\`. Read it before implementing senses or workflows.
## Directory layout
| Path | Purpose |
|------|---------|
| \`nerve.yaml\` | Senses, workflows, intervals, groups |
| \`package.json\` | Single root package — no per-sense/per-workflow packages |
| \`scripts/build.mjs\` | Root esbuild step; output under \`dist/\` |
| \`senses/<name>/src/index.ts\` | Sense \`compute()\` entry |
| \`senses/<name>/src/schema.ts\` | Drizzle SQLite schema (TypeScript) |
| \`senses/<name>/migrations/*.sql\` | SQL migrations (next to \`src/\`, not inside it) |
| \`workflows/<name>/index.ts\` | Default export: \`WorkflowDefinition\` |
| \`workflows/<name>/roles/<role>.ts\` | One TypeScript file per role |
| \`dist/senses/<name>/index.js\` | Bundled sense (after build) |
| \`dist/workflows/<name>/index.js\` | Bundled workflow (after build) |
There is **no** \`package.json\` or \`tsconfig.json\` inside individual senses or workflows.
## Naming
- **Workflows:** verb-first kebab-case (e.g. \`review-pull-request\`, \`deploy-staging\`). Avoid bare nouns like \`notifications\`.
- **Senses:** kebab-case descriptive nouns (e.g. \`cpu-usage\`).
## Workflow roles — four-tuple pattern
Wire each role with \`createRole\` from \`@uncaged/nerve-workflow-utils\`:
1. **Adapter** — \`AgentFn\` (LLM call)
2. **Prompt builder** — \`async (ctx: ThreadContext) => string\`
3. **Meta schema** — Zod object (routing / structured output from the model)
4. **Extractor config** — how JSON meta is parsed from replies
Keep meta small (often one boolean per role). The **moderator** in \`WorkflowDefinition\` routes between role names.
## Build commands
Always run from the **workspace root**:
\`\`\`bash
pnpm run build
# or: npm run build
\`\`\`
Fix errors until this succeeds. New workflows must appear under \`workflows/<name>/\` and be registered in \`nerve.yaml\`; new senses under \`senses/<name>/\` with matching \`nerve.yaml\` entries.
## Coding style (Nerve conventions)
- Use \`type\`, not \`interface\`; prefer \`function\` over classes (except errors / library requirements).
- **Named exports only** — no \`export default\` (exception: \`workflows/<name>/index.ts\` uses default export for the daemon loader).
- Nullable fields: \`T | null\`, not TypeScript optional \`?:\`.
- No dynamic \`import()\` in workspace code (bundling and tooling assume static imports).
- Use \`async\`/\`await\`; use a \`Result\` type for expected failures instead of control-flow try/catch.
## Extra references (optional)
- \`CONVENTIONS.md\` — project-specific overrides at repo root.
- \`.knowledge/*.md\` — deeper docs when working inside the Nerve monorepo.
- \`.cursor/skills/\` — Cursor Agent Skills (\`SKILL.md\` per skill).
`;
const NERVE_SKILLS_MDC = `---
description: >-
Nerve skills package — where bundled Agent Skills live in this workspace and how to use them
Where Agent Skills live in this Nerve workspace and how to use them with Cursor
alwaysApply: true
---
# Nerve skills (\`@uncaged/nerve-skills\`)
# Nerve Agent Skills
This workspace lists **@uncaged/nerve-skills** in \`package.json\`. It ships **Agent Skills** (one directory per skill, each with a \`SKILL.md\`) for Nerve development and related tasks.
**Agent Skills** are directories that contain a \`SKILL.md\` (with YAML frontmatter). Cursor loads them from **Project Skills** paths (for example \`.cursor/skills/\` or your global skills directory).
## After install
## Getting Nerve-oriented skills
Run your package manager in this workspace (e.g. \`pnpm install\`, \`npm install\` — whatever \`nerve init\` used). Then skills are on disk at:
There is no separate npm package for skills in the default workspace. To align with Nerve CLI, daemon, and monorepo conventions:
- \`node_modules/@uncaged/nerve-skills/<skill-id>/SKILL.md\`
Example (current catalog):
- **nerve-dev** — Nerve architecture, CLI, sense/workflow patterns, \`nerve.yaml\`, and conventions: read \`node_modules/@uncaged/nerve-skills/nerve-dev/SKILL.md\`.
1. Copy or symlink skill folders from the **Nerve** repository (e.g. \`packages/skills/*/\`) into \`.cursor/skills/\`, **or**
2. Follow project documentation and \`CLAUDE.md\` / \`.cursor/rules/\` in this repo.
## How to use in an agent
1. For tasks that match a skill’s **description** (in the \`SKILL.md\` frontmatter), open that \`SKILL.md\` and follow its structure and checklists.
2. Prefer the skill as the **source of truth** for Nerve-specific conventions over generic assumptions.
3. If the catalog grows, new skills appear as new sibling directories under \`node_modules/@uncaged/nerve-skills/\`.
Do not commit \`node_modules\`; the dependency is the supported way to get and update skills to match \`@uncaged/nerve-skills\` on npm.
1. When a task matches a skill’s **description** (in \`SKILL.md\` frontmatter), open that file and follow its steps.
2. Prefer those conventions for sense/workflow layout, \`nerve.yaml\`, and tooling over generic guesses.
3. Keep skills versioned with your dotfiles or project; update them when you upgrade Nerve.
`;
const execFileAsync = promisify(execFile);
@@ -124,6 +229,8 @@ export const cpuUsage = sqliteTable("cpu_usage", {
const CPU_INDEX_TS = `import { cpus } from "node:os";
export { cpuUsage as table } from "./schema.js";
type SenseResult = {
signal: { model: string; loadPercent: number; ts: number };
workflow: null;
@@ -154,24 +261,6 @@ export async function compute(): Promise<SenseResult> {
}
`;
const CPU_SENSE_PACKAGE_JSON = `${JSON.stringify(
{
name: "nerve-sense-cpu-usage",
private: true,
type: "module",
scripts: {
build:
"esbuild src/index.ts --bundle --platform=node --format=esm --outdir=. --out-extension:.js=.js --packages=external",
},
devDependencies: {
esbuild: "^0.27.0",
"drizzle-orm": "*",
},
},
null,
2,
)}\n`;
const CPU_MIGRATION_SQL = `CREATE TABLE IF NOT EXISTS cpu_usage (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts INTEGER NOT NULL,
@@ -334,10 +423,10 @@ async function runInitWorkspace(force: boolean, skipInstall = false): Promise<vo
writeFile(join(nerveRoot, "nerve.yaml"), NERVE_YAML);
writeFile(join(nerveRoot, "package.json"), PACKAGE_JSON);
writeFile(join(nerveRoot, "pnpm-workspace.yaml"), PNPM_WORKSPACE_YAML);
writeFile(join(nerveRoot, "scripts", "build.mjs"), BUILD_MJS);
writeFile(join(nerveRoot, "biome.json"), BIOME_JSON);
writeFile(join(nerveRoot, ".gitignore"), GITIGNORE);
writeFile(join(nerveRoot, "senses", "cpu-usage", "package.json"), CPU_SENSE_PACKAGE_JSON);
writeFile(join(nerveRoot, "AGENT.md"), AGENT_MD);
writeFile(join(nerveRoot, "senses", "cpu-usage", "src", "index.ts"), CPU_INDEX_TS);
writeFile(join(nerveRoot, "senses", "cpu-usage", "src", "schema.ts"), CPU_SCHEMA_TS);
writeFile(
@@ -8,7 +8,7 @@ import { join } from "node:path";
import type { NerveConfig } from "@uncaged/nerve-core";
/**
* Detects RoleSpec `adapter:` usage in workflow TypeScript sources.
* Detects `adapter:` usage in workflow TypeScript sources (e.g. createRole wiring).
* NOTE: This regex can match occurrences inside comments.
*/
const WORKFLOW_SPEC_ADAPTER_PATTERN = /adapter:\s*[a-zA-Z_$]/;
@@ -26,7 +26,7 @@ function collectTsSourceFiles(dir: string, acc: string[]): void {
}
/**
* Returns true when any workflow `src` tree appears to use WorkflowSpec roles with adapters.
* Returns true when any workflow `src` tree appears to use roles with adapters.
*/
export function workflowSourcesDeclareAdapterRoles(nerveRoot: string): boolean {
const workflowsRoot = join(nerveRoot, "workflows");
@@ -66,7 +66,7 @@ export function validateAgentConfigurationLayer(
return {
ok: false,
message:
"extract: required when WorkflowSpec roles use adapters (configure extract.provider and extract.model)",
"extract: required when workflow roles use adapters (configure extract.provider and extract.model)",
};
}
+1
View File
@@ -20,6 +20,7 @@
"test": "vitest run"
},
"dependencies": {
"drizzle-orm": "1.0.0-beta.23-c10d10c",
"yaml": "^2.8.3"
},
"devDependencies": {
+1 -1
View File
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { parseNerveConfig } from "../parse-nerve-config.js";
import { parseNerveConfig } from "../config.js";
const VALID_CONFIG = `
senses:
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { parseDaemonIpcRequest } from "../daemon-ipc-protocol.js";
import { parseDaemonIpcRequest } from "../daemon.js";
describe("parseDaemonIpcRequest", () => {
it("parses trigger-workflow", () => {
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { parseKnowledgeYaml } from "../knowledge-config.js";
import { parseKnowledgeYaml } from "../config.js";
describe("parseKnowledgeYaml", () => {
it("parses include and exclude glob lists", () => {
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { parseWorkflowTrigger, routeSenseComputeOutput } from "../sense-workflow-directive.js";
import { parseWorkflowTrigger, routeSenseComputeOutput } from "../sense.js";
describe("parseWorkflowTrigger", () => {
it("accepts a valid trigger object", () => {
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { spawnSafe } from "../spawn-safe.js";
import { spawnSafe } from "../util.js";
describe("spawnSafe", () => {
it("passes argv literally without shell interpretation (injection-safe)", async () => {
-5
View File
@@ -1,5 +0,0 @@
/**
* Agent adapter ids referenced by tooling / docs (RFC-003).
* Workflows import adapter packages directly; echo may be used in tests via a small factory.
*/
export const KNOWN_AGENT_ADAPTER_IDS = ["echo", "cursor", "hermes", "codex"] as const;
@@ -21,3 +21,9 @@ export class ExtractError extends Error {
Object.setPrototypeOf(this, new.target.prototype);
}
}
/**
* Agent adapter ids referenced by tooling / docs (RFC-003).
* Workflows import adapter packages directly; echo may be used in tests via a small factory.
*/
export const KNOWN_AGENT_ADAPTER_IDS = ["echo", "cursor", "hermes", "codex"] as const;
+399 -2
View File
@@ -1,5 +1,7 @@
/** Default max rows kept in each sense's `_signals` SQLite table (see `retention` on `SenseConfig`). */
export const DEFAULT_SENSE_SIGNAL_RETENTION = 10_000;
import { parse } from "yaml";
import { type Result, err, isPlainRecord, ok, parseDurationStringToMs } from "./util.js";
import { DEFAULT_ENGINE_MAX_ROUNDS } from "./workflow.js";
export type SenseConfig = {
group: string;
@@ -75,3 +77,398 @@ export type NerveConfig = {
/** Global extract defaults; `null` when the section is omitted. */
extract: ExtractConfig | null;
};
export type KnowledgeConfig = {
include: ReadonlyArray<string>;
exclude: ReadonlyArray<string>;
};
/** Default max rows kept in each sense's `_signals` SQLite table (see `retention` on `SenseConfig`). */
export const DEFAULT_SENSE_SIGNAL_RETENTION = 10_000;
function isValidGroupName(value: string): boolean {
return /^[a-zA-Z0-9_-]+$/.test(value);
}
function parseRetentionField(name: string, field: unknown): Result<number> {
if (field === undefined || field === null) {
return ok(DEFAULT_SENSE_SIGNAL_RETENTION);
}
if (typeof field !== "number" || !Number.isInteger(field) || field < 1) {
return err(new Error(`senses.${name}.retention: must be a positive integer`));
}
return ok(field);
}
function parseDurationField(field: unknown, label: string): Result<number | null> {
if (field === undefined || field === null) return ok(null);
if (typeof field !== "string") {
return err(
new Error(`${label}: invalid duration "${field}" (expected e.g. "5s", "10m", "1h")`),
);
}
const msResult = parseDurationStringToMs(field);
if (!msResult.ok) {
return err(new Error(`${label}: ${msResult.error.message}`));
}
return ok(msResult.value);
}
function validateSenseConfig(name: string, raw: unknown): Result<SenseConfig> {
if (!isPlainRecord(raw)) {
return err(new Error(`senses.${name}: must be an object`));
}
const obj = raw;
if (typeof obj.group !== "string" || obj.group.trim() === "") {
return err(new Error(`senses.${name}.group: required string`));
}
if (!isValidGroupName(obj.group)) {
return err(
new Error(
`senses.${name}.group: invalid name "${obj.group}" (only alphanumeric, underscore, hyphen allowed)`,
),
);
}
const throttleResult = parseDurationField(obj.throttle, `senses.${name}.throttle`);
if (!throttleResult.ok) return throttleResult;
const timeoutResult = parseDurationField(obj.timeout, `senses.${name}.timeout`);
if (!timeoutResult.ok) return timeoutResult;
const graceResult = parseDurationField(obj.grace_period, `senses.${name}.grace_period`);
if (!graceResult.ok) return graceResult;
const retentionResult = parseRetentionField(name, obj.retention);
if (!retentionResult.ok) return retentionResult;
const intervalResult = parseDurationField(obj.interval, `senses.${name}.interval`);
if (!intervalResult.ok) return intervalResult;
let on: string[] = [];
if (obj.on !== undefined && obj.on !== null) {
if (
!Array.isArray(obj.on) ||
!obj.on.every((item: unknown): item is string => typeof item === "string")
) {
return err(new Error(`senses.${name}.on: must be an array of strings`));
}
on = obj.on;
}
return ok({
group: obj.group,
throttle: throttleResult.value,
timeout: timeoutResult.value,
gracePeriod: graceResult.value,
retention: retentionResult.value,
interval: intervalResult.value,
on,
});
}
function parseEngineMaxRounds(obj: Record<string, unknown>): Result<number> {
if (obj.max_rounds === undefined || obj.max_rounds === null) {
return ok(DEFAULT_ENGINE_MAX_ROUNDS);
}
if (
typeof obj.max_rounds !== "number" ||
!Number.isInteger(obj.max_rounds) ||
obj.max_rounds < 1
) {
return err(new Error("max_rounds: must be a positive integer"));
}
return ok(obj.max_rounds);
}
function validateWorkflowConfig(name: string, raw: unknown): Result<WorkflowConfig> {
if (!isPlainRecord(raw)) {
return err(new Error(`workflows.${name}: must be an object`));
}
const obj = raw;
if (
typeof obj.concurrency !== "number" ||
!Number.isInteger(obj.concurrency) ||
obj.concurrency < 1
) {
return err(new Error(`workflows.${name}.concurrency: must be a positive integer`));
}
if (obj.overflow !== "drop" && obj.overflow !== "queue") {
return err(new Error(`workflows.${name}.overflow: must be "drop" or "queue"`));
}
if (obj.overflow === "drop") {
if (obj.max_queue !== undefined && obj.max_queue !== null) {
return err(new Error(`workflows.${name}: max_queue is not allowed with overflow "drop"`));
}
return ok({
concurrency: obj.concurrency,
overflow: "drop" as const,
});
}
// overflow: "queue"
let maxQueue = 100; // default
if (obj.max_queue !== undefined && obj.max_queue !== null) {
if (
typeof obj.max_queue !== "number" ||
!Number.isInteger(obj.max_queue) ||
obj.max_queue < 1
) {
return err(new Error(`workflows.${name}.max_queue: must be a positive integer`));
}
maxQueue = obj.max_queue;
}
return ok({
concurrency: obj.concurrency,
overflow: "queue" as const,
maxQueue,
});
}
function parseSenses(
obj: Record<string, unknown>,
): Result<{ senses: Record<string, SenseConfig> }> {
if (!isPlainRecord(obj.senses)) {
return err(new Error("senses: required object"));
}
const sensesRaw = obj.senses;
const senses: Record<string, SenseConfig> = {};
for (const [name, senseRaw] of Object.entries(sensesRaw)) {
const result = validateSenseConfig(name, senseRaw);
if (!result.ok) return result;
senses[name] = result.value;
}
return ok({ senses });
}
const DEFAULT_API_BIND_HOST = "127.0.0.1";
/** Hosts that may bind the HTTP API without `api.token` (loopback-only). */
function isLoopbackOnlyApiHost(host: string): boolean {
const h = host.trim();
return h === "127.0.0.1" || h.toLowerCase() === "localhost";
}
function parseApiTokenField(api: Record<string, unknown>): Result<string | null> {
if (api.token === undefined || api.token === null) {
return ok(null);
}
if (typeof api.token !== "string") {
return err(new Error("api.token: must be a string when provided"));
}
if (api.token.length === 0) {
return err(new Error("api.token: must not be empty when provided"));
}
return ok(api.token);
}
function parseApiHostField(api: Record<string, unknown>): Result<string> {
if (api.host === undefined || api.host === null) {
return ok(DEFAULT_API_BIND_HOST);
}
if (typeof api.host !== "string") {
return err(new Error("api.host: must be a string when provided"));
}
if (api.host.length === 0) {
return err(new Error("api.host: must not be empty when provided"));
}
return ok(api.host);
}
function parseApiConfig(obj: Record<string, unknown>): Result<NerveApiConfig> {
if (obj.api === undefined || obj.api === null) {
return ok({ port: null, token: null, host: DEFAULT_API_BIND_HOST });
}
if (!isPlainRecord(obj.api)) {
return err(new Error("api: must be an object if provided"));
}
const api = obj.api;
if (api.port === undefined || api.port === null) {
return ok({ port: null, token: null, host: DEFAULT_API_BIND_HOST });
}
if (
typeof api.port !== "number" ||
!Number.isInteger(api.port) ||
api.port < 1 ||
api.port > 65_535
) {
return err(new Error("api.port: must be an integer between 1 and 65535 if provided"));
}
const tokenResult = parseApiTokenField(api);
if (!tokenResult.ok) return tokenResult;
const hostResult = parseApiHostField(api);
if (!hostResult.ok) return hostResult;
if (!isLoopbackOnlyApiHost(hostResult.value) && tokenResult.value === null) {
return err(
new Error("api.host binds to non-loopback address, api.token is required for security"),
);
}
return ok({ port: api.port, token: tokenResult.value, host: hostResult.value });
}
function parseWorkflows(obj: Record<string, unknown>): Result<Record<string, WorkflowConfig>> {
if (obj.workflows === undefined || obj.workflows === null) return ok({});
if (!isPlainRecord(obj.workflows)) {
return err(new Error("workflows: must be an object if provided"));
}
const workflowsRaw = obj.workflows;
const workflows: Record<string, WorkflowConfig> = {};
for (const [name, wfRaw] of Object.entries(workflowsRaw)) {
const result = validateWorkflowConfig(name, wfRaw);
if (!result.ok) return result;
workflows[name] = result.value;
}
return ok(workflows);
}
function parseExtract(obj: Record<string, unknown>): Result<ExtractConfig | null> {
if (obj.extract === undefined || obj.extract === null) {
return ok(null);
}
if (!isPlainRecord(obj.extract)) {
return err(new Error("extract: must be an object if provided"));
}
const ext = obj.extract;
if (typeof ext.provider !== "string" || ext.provider.trim() === "") {
return err(new Error("extract.provider: required non-empty string"));
}
if (typeof ext.model !== "string" || ext.model.trim() === "") {
return err(new Error("extract.model: required non-empty string"));
}
return ok({ provider: ext.provider, model: ext.model });
}
export function parseNerveConfig(raw: string): Result<NerveConfig> {
let parsed: unknown;
try {
parsed = parse(raw);
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
return err(new Error(`YAML parse error: ${message}`));
}
if (!isPlainRecord(parsed)) {
return err(new Error("Config must be a YAML object"));
}
const obj = parsed;
const sensesResult = parseSenses(obj);
if (!sensesResult.ok) return sensesResult;
const { senses } = sensesResult.value;
// Legacy top-level `reflexes` is rejected; each sense carries `interval` / `on` for the sense scheduler.
if (Object.hasOwn(obj, "reflexes")) {
return err(
new Error(
"reflexes: top-level key is no longer supported; set `interval` and `on` on each sense under `senses.<name>`",
),
);
}
const workflowsResult = parseWorkflows(obj);
if (!workflowsResult.ok) return workflowsResult;
const maxRoundsResult = parseEngineMaxRounds(obj);
if (!maxRoundsResult.ok) return maxRoundsResult;
const apiResult = parseApiConfig(obj);
if (!apiResult.ok) return apiResult;
if (Object.hasOwn(obj, "agents")) {
return err(
new Error(
"agents: key is no longer supported — declare adapters on workflow roles (RFC-003)",
),
);
}
const extractResult = parseExtract(obj);
if (!extractResult.ok) return extractResult;
return ok({
maxRounds: maxRoundsResult.value,
senses,
workflows: workflowsResult.value,
api: apiResult.value,
extract: extractResult.value,
});
}
function parseStringList(field: unknown, label: string): Result<ReadonlyArray<string>> {
if (field === undefined || field === null) {
return ok([]);
}
if (!Array.isArray(field)) {
return err(new Error(`${label}: must be an array of strings`));
}
const out: string[] = [];
for (let i = 0; i < field.length; i++) {
const item = field[i];
if (typeof item !== "string" || item.length === 0) {
return err(new Error(`${label}[${String(i)}]: must be a non-empty string`));
}
out.push(item);
}
return ok(out);
}
/**
* Parse `knowledge.yaml` at the repo root (RFC-003 Knowledge Layer).
* `include` / `exclude` entries are glob patterns resolved against the repo root.
*/
export function parseKnowledgeYaml(raw: string): Result<KnowledgeConfig> {
let parsed: unknown;
try {
parsed = parse(raw);
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
return err(new Error(`YAML parse error: ${message}`));
}
if (parsed === undefined || parsed === null) {
return ok({ include: [], exclude: [] });
}
if (!isPlainRecord(parsed)) {
return err(new Error("knowledge.yaml: root must be a mapping"));
}
const includeResult = parseStringList(parsed.include, "include");
if (!includeResult.ok) {
return includeResult;
}
const excludeResult = parseStringList(parsed.exclude, "exclude");
if (!excludeResult.ok) {
return excludeResult;
}
return ok({
include: includeResult.value,
exclude: excludeResult.value,
});
}
@@ -1,33 +0,0 @@
import type { WorkflowStatus } from "./daemon-ipc-protocol.js";
import { isPlainRecord } from "./is-plain-record.js";
import type { SenseInfo } from "./sense.js";
/** Type guard for JSON {@link SenseInfo} payloads from daemon HTTP/IPC. */
export function isSenseInfo(value: unknown): value is SenseInfo {
if (!isPlainRecord(value)) return false;
return (
typeof value.name === "string" &&
typeof value.group === "string" &&
(value.throttle === null || typeof value.throttle === "number") &&
(value.timeout === null || typeof value.timeout === "number") &&
Array.isArray(value.triggers) &&
value.triggers.every((t: unknown) => typeof t === "string") &&
(value.lastSignalTimestamp === null || typeof value.lastSignalTimestamp === "number")
);
}
/** Type guard for JSON {@link WorkflowStatus} payloads from daemon HTTP/IPC. */
export function isWorkflowStatus(value: unknown): value is WorkflowStatus {
if (!isPlainRecord(value)) return false;
const cfg = value.config;
if (!isPlainRecord(cfg)) return false;
return (
typeof value.name === "string" &&
typeof value.activeThreads === "number" &&
Array.isArray(value.activeRunIds) &&
value.activeRunIds.every((id: unknown) => typeof id === "string") &&
typeof value.queuedThreads === "number" &&
typeof cfg.concurrency === "number" &&
typeof cfg.overflow === "string"
);
}
-28
View File
@@ -1,28 +0,0 @@
import type { HealthInfo, WorkflowStatus } from "./daemon-ipc-protocol.js";
import type { SenseInfo } from "./sense.js";
export type DaemonTransportTriggerResult = { ok: true } | { ok: false; error: string };
export type DaemonTransportWorkflowLaunch = {
prompt: string;
maxRounds: number;
dryRun: boolean;
};
/**
* Abstraction over daemon control plane (Unix socket IPC today, HTTP in Phase 2).
* Implementations live in CLI / tools; the daemon kernel uses shared handler logic.
*/
export type DaemonTransport = {
health(): Promise<HealthInfo>;
listSenses(): Promise<SenseInfo[]>;
listWorkflows(): Promise<WorkflowStatus[]>;
triggerSense(name: string): Promise<DaemonTransportTriggerResult>;
/** When `launch` is null, implementations use engine defaults (empty prompt, default max rounds, dryRun false). */
triggerWorkflow(
name: string,
launch: DaemonTransportWorkflowLaunch | null,
): Promise<DaemonTransportTriggerResult>;
/** Kill a running or queued workflow thread by `runId` (same field as IPC `kill-workflow`). */
killWorkflow(runId: string): Promise<DaemonTransportTriggerResult>;
};
@@ -4,8 +4,8 @@
* one response object per line from the daemon.
*/
import { isPlainRecord } from "./is-plain-record.js";
import type { SenseInfo } from "./sense.js";
import { isPlainRecord } from "./util.js";
/** Runtime status of a registered workflow (for listing / observability). */
export type WorkflowStatus = {
@@ -100,6 +100,32 @@ export type DaemonIpcResponse =
| DaemonIpcListWorkflowsResponse
| DaemonIpcHealthResponse;
export type DaemonTransportTriggerResult = { ok: true } | { ok: false; error: string };
export type DaemonTransportWorkflowLaunch = {
prompt: string;
maxRounds: number;
dryRun: boolean;
};
/**
* Abstraction over daemon control plane (Unix socket IPC today, HTTP in Phase 2).
* Implementations live in CLI / tools; the daemon kernel uses shared handler logic.
*/
export type DaemonTransport = {
health(): Promise<HealthInfo>;
listSenses(): Promise<SenseInfo[]>;
listWorkflows(): Promise<WorkflowStatus[]>;
triggerSense(name: string): Promise<DaemonTransportTriggerResult>;
/** When `launch` is null, implementations use engine defaults (empty prompt, default max rounds, dryRun false). */
triggerWorkflow(
name: string,
launch: DaemonTransportWorkflowLaunch | null,
): Promise<DaemonTransportTriggerResult>;
/** Kill a running or queued workflow thread by `runId` (same field as IPC `kill-workflow`). */
killWorkflow(runId: string): Promise<DaemonTransportTriggerResult>;
};
function parseTriggerWorkflowFields(
req: Record<string, unknown>,
): DaemonIpcTriggerWorkflowRequest | null {
@@ -150,3 +176,33 @@ export function parseDaemonIpcRequest(line: string): DaemonIpcRequest | null {
return null;
}
}
/** Type guard for JSON {@link SenseInfo} payloads from daemon HTTP/IPC. */
export function isSenseInfo(value: unknown): value is SenseInfo {
if (!isPlainRecord(value)) return false;
return (
typeof value.name === "string" &&
typeof value.group === "string" &&
(value.throttle === null || typeof value.throttle === "number") &&
(value.timeout === null || typeof value.timeout === "number") &&
Array.isArray(value.triggers) &&
value.triggers.every((t: unknown) => typeof t === "string") &&
(value.lastSignalTimestamp === null || typeof value.lastSignalTimestamp === "number")
);
}
/** Type guard for JSON {@link WorkflowStatus} payloads from daemon HTTP/IPC. */
export function isWorkflowStatus(value: unknown): value is WorkflowStatus {
if (!isPlainRecord(value)) return false;
const cfg = value.config;
if (!isPlainRecord(cfg)) return false;
return (
typeof value.name === "string" &&
typeof value.activeThreads === "number" &&
Array.isArray(value.activeRunIds) &&
value.activeRunIds.every((id: unknown) => typeof id === "string") &&
typeof value.queuedThreads === "number" &&
typeof cfg.concurrency === "number" &&
typeof cfg.overflow === "string"
);
}
-22
View File
@@ -1,22 +0,0 @@
import type { Result } from "./result.js";
import { err, ok } from "./result.js";
const DURATION_RE = /^(\d+)([smh])$/;
const DURATION_MULTIPLIERS: Record<string, number> = {
s: 1_000,
m: 60_000,
h: 3_600_000,
};
/**
* Parse a duration string such as `5s`, `10m`, `1h` to milliseconds.
* Used by `parseNerveConfig` sense/workflow duration fields.
*/
export function parseDurationStringToMs(value: string): Result<number> {
const match = DURATION_RE.exec(value);
if (!match) {
return err(new Error(`invalid duration "${value}" (expected e.g. "5s", "10m", "1h")`));
}
return ok(Number(match[1]) * DURATION_MULTIPLIERS[match[2]]);
}
+20 -19
View File
@@ -12,13 +12,15 @@ export type {
ComputeResult,
} from "./config.js";
export type { Signal, SenseInfo } from "./sense.js";
export { labelSenseTrigger, senseTriggerLabels } from "./sense-trigger-labels.js";
export type { SenseComputeFn, SenseModule } from "./sense.js";
export { labelSenseTrigger, senseTriggerLabels } from "./sense.js";
export type {
WorkflowMessage,
RoleResult,
Role,
RoleMeta,
StartStep,
ThreadContext,
WorkflowContext,
AgentFn,
RoleStep,
@@ -27,12 +29,11 @@ export type {
WorkflowDefinition,
} from "./workflow.js";
export { START, END, DEFAULT_ENGINE_MAX_ROUNDS } from "./workflow.js";
export type { PromptInput, RoleSpec, WorkflowSpec } from "./workflow-spec.js";
export { parseDurationStringToMs } from "./duration.js";
export type { Schema, ExtractFn } from "./extract-layer.js";
export { ExtractError } from "./extract-layer.js";
export type { Result } from "./result.js";
export { ok, err } from "./result.js";
export { parseDurationStringToMs } from "./util.js";
export type { Schema, ExtractFn } from "./agent.js";
export { ExtractError } from "./agent.js";
export type { Result } from "./util.js";
export { ok, err } from "./util.js";
export {
nerveCommandEnv,
spawnSafe,
@@ -40,17 +41,17 @@ export {
type SpawnError,
type SpawnResult,
type SpawnSafeOptions,
} from "./spawn-safe.js";
export { parseNerveConfig } from "./parse-nerve-config.js";
export type { KnowledgeConfig } from "./knowledge-config.js";
export { parseKnowledgeYaml } from "./knowledge-config.js";
export { isPlainRecord } from "./is-plain-record.js";
export { KNOWN_AGENT_ADAPTER_IDS } from "./agent-adapter-ids.js";
} from "./util.js";
export { parseNerveConfig } from "./config.js";
export type { KnowledgeConfig } from "./config.js";
export { parseKnowledgeYaml } from "./config.js";
export { isPlainRecord } from "./util.js";
export { KNOWN_AGENT_ADAPTER_IDS } from "./agent.js";
export type { RoutedSenseOutput } from "./sense-workflow-directive.js";
export { parseWorkflowTrigger, routeSenseComputeOutput } from "./sense-workflow-directive.js";
export type { RoutedSenseOutput } from "./sense.js";
export { parseWorkflowTrigger, routeSenseComputeOutput } from "./sense.js";
export { isSenseInfo, isWorkflowStatus } from "./daemon-payload-guards.js";
export { isSenseInfo, isWorkflowStatus } from "./daemon.js";
export type {
WorkflowStatus,
HealthInfo,
@@ -68,10 +69,10 @@ export type {
DaemonIpcListWorkflowsResponse,
DaemonIpcHealthResponse,
DaemonIpcResponse,
} from "./daemon-ipc-protocol.js";
export { parseDaemonIpcRequest } from "./daemon-ipc-protocol.js";
} from "./daemon.js";
export { parseDaemonIpcRequest } from "./daemon.js";
export type {
DaemonTransport,
DaemonTransportTriggerResult,
DaemonTransportWorkflowLaunch,
} from "./daemon-transport.js";
} from "./daemon.js";
-7
View File
@@ -1,7 +0,0 @@
/**
* Narrows `unknown` to a plain JSON-style object (not null, not array).
* Use after `JSON.parse` / YAML / IPC when validating structure field-by-field.
*/
export function isPlainRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
-64
View File
@@ -1,64 +0,0 @@
import { parse } from "yaml";
import { isPlainRecord } from "./is-plain-record.js";
import type { Result } from "./result.js";
import { err, ok } from "./result.js";
export type KnowledgeConfig = {
include: ReadonlyArray<string>;
exclude: ReadonlyArray<string>;
};
function parseStringList(field: unknown, label: string): Result<ReadonlyArray<string>> {
if (field === undefined || field === null) {
return ok([]);
}
if (!Array.isArray(field)) {
return err(new Error(`${label}: must be an array of strings`));
}
const out: string[] = [];
for (let i = 0; i < field.length; i++) {
const item = field[i];
if (typeof item !== "string" || item.length === 0) {
return err(new Error(`${label}[${String(i)}]: must be a non-empty string`));
}
out.push(item);
}
return ok(out);
}
/**
* Parse `knowledge.yaml` at the repo root (RFC-003 Knowledge Layer).
* `include` / `exclude` entries are glob patterns resolved against the repo root.
*/
export function parseKnowledgeYaml(raw: string): Result<KnowledgeConfig> {
let parsed: unknown;
try {
parsed = parse(raw);
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
return err(new Error(`YAML parse error: ${message}`));
}
if (parsed === undefined || parsed === null) {
return ok({ include: [], exclude: [] });
}
if (!isPlainRecord(parsed)) {
return err(new Error("knowledge.yaml: root must be a mapping"));
}
const includeResult = parseStringList(parsed.include, "include");
if (!includeResult.ok) {
return includeResult;
}
const excludeResult = parseStringList(parsed.exclude, "exclude");
if (!excludeResult.ok) {
return excludeResult;
}
return ok({
include: includeResult.value,
exclude: excludeResult.value,
});
}
-348
View File
@@ -1,348 +0,0 @@
import { parse } from "yaml";
import {
DEFAULT_SENSE_SIGNAL_RETENTION,
type ExtractConfig,
type NerveApiConfig,
type NerveConfig,
type SenseConfig,
type WorkflowConfig,
} from "./config.js";
import { parseDurationStringToMs } from "./duration.js";
import { isPlainRecord } from "./is-plain-record.js";
import type { Result } from "./result.js";
import { err, ok } from "./result.js";
import { DEFAULT_ENGINE_MAX_ROUNDS } from "./workflow.js";
function isValidGroupName(value: string): boolean {
return /^[a-zA-Z0-9_-]+$/.test(value);
}
function parseRetentionField(name: string, field: unknown): Result<number> {
if (field === undefined || field === null) {
return ok(DEFAULT_SENSE_SIGNAL_RETENTION);
}
if (typeof field !== "number" || !Number.isInteger(field) || field < 1) {
return err(new Error(`senses.${name}.retention: must be a positive integer`));
}
return ok(field);
}
function parseDurationField(field: unknown, label: string): Result<number | null> {
if (field === undefined || field === null) return ok(null);
if (typeof field !== "string") {
return err(
new Error(`${label}: invalid duration "${field}" (expected e.g. "5s", "10m", "1h")`),
);
}
const msResult = parseDurationStringToMs(field);
if (!msResult.ok) {
return err(new Error(`${label}: ${msResult.error.message}`));
}
return ok(msResult.value);
}
function validateSenseConfig(name: string, raw: unknown): Result<SenseConfig> {
if (!isPlainRecord(raw)) {
return err(new Error(`senses.${name}: must be an object`));
}
const obj = raw;
if (typeof obj.group !== "string" || obj.group.trim() === "") {
return err(new Error(`senses.${name}.group: required string`));
}
if (!isValidGroupName(obj.group)) {
return err(
new Error(
`senses.${name}.group: invalid name "${obj.group}" (only alphanumeric, underscore, hyphen allowed)`,
),
);
}
const throttleResult = parseDurationField(obj.throttle, `senses.${name}.throttle`);
if (!throttleResult.ok) return throttleResult;
const timeoutResult = parseDurationField(obj.timeout, `senses.${name}.timeout`);
if (!timeoutResult.ok) return timeoutResult;
const graceResult = parseDurationField(obj.grace_period, `senses.${name}.grace_period`);
if (!graceResult.ok) return graceResult;
const retentionResult = parseRetentionField(name, obj.retention);
if (!retentionResult.ok) return retentionResult;
const intervalResult = parseDurationField(obj.interval, `senses.${name}.interval`);
if (!intervalResult.ok) return intervalResult;
let on: string[] = [];
if (obj.on !== undefined && obj.on !== null) {
if (
!Array.isArray(obj.on) ||
!obj.on.every((item: unknown): item is string => typeof item === "string")
) {
return err(new Error(`senses.${name}.on: must be an array of strings`));
}
on = obj.on;
}
return ok({
group: obj.group,
throttle: throttleResult.value,
timeout: timeoutResult.value,
gracePeriod: graceResult.value,
retention: retentionResult.value,
interval: intervalResult.value,
on,
});
}
function parseEngineMaxRounds(obj: Record<string, unknown>): Result<number> {
if (obj.max_rounds === undefined || obj.max_rounds === null) {
return ok(DEFAULT_ENGINE_MAX_ROUNDS);
}
if (
typeof obj.max_rounds !== "number" ||
!Number.isInteger(obj.max_rounds) ||
obj.max_rounds < 1
) {
return err(new Error("max_rounds: must be a positive integer"));
}
return ok(obj.max_rounds);
}
function validateWorkflowConfig(name: string, raw: unknown): Result<WorkflowConfig> {
if (!isPlainRecord(raw)) {
return err(new Error(`workflows.${name}: must be an object`));
}
const obj = raw;
if (
typeof obj.concurrency !== "number" ||
!Number.isInteger(obj.concurrency) ||
obj.concurrency < 1
) {
return err(new Error(`workflows.${name}.concurrency: must be a positive integer`));
}
if (obj.overflow !== "drop" && obj.overflow !== "queue") {
return err(new Error(`workflows.${name}.overflow: must be "drop" or "queue"`));
}
if (obj.overflow === "drop") {
if (obj.max_queue !== undefined && obj.max_queue !== null) {
return err(new Error(`workflows.${name}: max_queue is not allowed with overflow "drop"`));
}
return ok({
concurrency: obj.concurrency,
overflow: "drop" as const,
});
}
// overflow: "queue"
let maxQueue = 100; // default
if (obj.max_queue !== undefined && obj.max_queue !== null) {
if (
typeof obj.max_queue !== "number" ||
!Number.isInteger(obj.max_queue) ||
obj.max_queue < 1
) {
return err(new Error(`workflows.${name}.max_queue: must be a positive integer`));
}
maxQueue = obj.max_queue;
}
return ok({
concurrency: obj.concurrency,
overflow: "queue" as const,
maxQueue,
});
}
function parseSenses(
obj: Record<string, unknown>,
): Result<{ senses: Record<string, SenseConfig> }> {
if (!isPlainRecord(obj.senses)) {
return err(new Error("senses: required object"));
}
const sensesRaw = obj.senses;
const senses: Record<string, SenseConfig> = {};
for (const [name, senseRaw] of Object.entries(sensesRaw)) {
const result = validateSenseConfig(name, senseRaw);
if (!result.ok) return result;
senses[name] = result.value;
}
return ok({ senses });
}
const DEFAULT_API_BIND_HOST = "127.0.0.1";
/** Hosts that may bind the HTTP API without `api.token` (loopback-only). */
function isLoopbackOnlyApiHost(host: string): boolean {
const h = host.trim();
return h === "127.0.0.1" || h.toLowerCase() === "localhost";
}
function parseApiTokenField(api: Record<string, unknown>): Result<string | null> {
if (api.token === undefined || api.token === null) {
return ok(null);
}
if (typeof api.token !== "string") {
return err(new Error("api.token: must be a string when provided"));
}
if (api.token.length === 0) {
return err(new Error("api.token: must not be empty when provided"));
}
return ok(api.token);
}
function parseApiHostField(api: Record<string, unknown>): Result<string> {
if (api.host === undefined || api.host === null) {
return ok(DEFAULT_API_BIND_HOST);
}
if (typeof api.host !== "string") {
return err(new Error("api.host: must be a string when provided"));
}
if (api.host.length === 0) {
return err(new Error("api.host: must not be empty when provided"));
}
return ok(api.host);
}
function parseApiConfig(obj: Record<string, unknown>): Result<NerveApiConfig> {
if (obj.api === undefined || obj.api === null) {
return ok({ port: null, token: null, host: DEFAULT_API_BIND_HOST });
}
if (!isPlainRecord(obj.api)) {
return err(new Error("api: must be an object if provided"));
}
const api = obj.api;
if (api.port === undefined || api.port === null) {
return ok({ port: null, token: null, host: DEFAULT_API_BIND_HOST });
}
if (
typeof api.port !== "number" ||
!Number.isInteger(api.port) ||
api.port < 1 ||
api.port > 65_535
) {
return err(new Error("api.port: must be an integer between 1 and 65535 if provided"));
}
const tokenResult = parseApiTokenField(api);
if (!tokenResult.ok) return tokenResult;
const hostResult = parseApiHostField(api);
if (!hostResult.ok) return hostResult;
if (!isLoopbackOnlyApiHost(hostResult.value) && tokenResult.value === null) {
return err(
new Error("api.host binds to non-loopback address, api.token is required for security"),
);
}
return ok({ port: api.port, token: tokenResult.value, host: hostResult.value });
}
function parseWorkflows(obj: Record<string, unknown>): Result<Record<string, WorkflowConfig>> {
if (obj.workflows === undefined || obj.workflows === null) return ok({});
if (!isPlainRecord(obj.workflows)) {
return err(new Error("workflows: must be an object if provided"));
}
const workflowsRaw = obj.workflows;
const workflows: Record<string, WorkflowConfig> = {};
for (const [name, wfRaw] of Object.entries(workflowsRaw)) {
const result = validateWorkflowConfig(name, wfRaw);
if (!result.ok) return result;
workflows[name] = result.value;
}
return ok(workflows);
}
function parseExtract(obj: Record<string, unknown>): Result<ExtractConfig | null> {
if (obj.extract === undefined || obj.extract === null) {
return ok(null);
}
if (!isPlainRecord(obj.extract)) {
return err(new Error("extract: must be an object if provided"));
}
const ext = obj.extract;
if (typeof ext.provider !== "string" || ext.provider.trim() === "") {
return err(new Error("extract.provider: required non-empty string"));
}
if (typeof ext.model !== "string" || ext.model.trim() === "") {
return err(new Error("extract.model: required non-empty string"));
}
return ok({ provider: ext.provider, model: ext.model });
}
export function parseNerveConfig(raw: string): Result<NerveConfig> {
let parsed: unknown;
try {
parsed = parse(raw);
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
return err(new Error(`YAML parse error: ${message}`));
}
if (!isPlainRecord(parsed)) {
return err(new Error("Config must be a YAML object"));
}
const obj = parsed;
const sensesResult = parseSenses(obj);
if (!sensesResult.ok) return sensesResult;
const { senses } = sensesResult.value;
// Legacy top-level `reflexes` is rejected; each sense carries `interval` / `on` for the sense scheduler.
if (Object.hasOwn(obj, "reflexes")) {
return err(
new Error(
"reflexes: top-level key is no longer supported; set `interval` and `on` on each sense under `senses.<name>`",
),
);
}
const workflowsResult = parseWorkflows(obj);
if (!workflowsResult.ok) return workflowsResult;
const maxRoundsResult = parseEngineMaxRounds(obj);
if (!maxRoundsResult.ok) return maxRoundsResult;
const apiResult = parseApiConfig(obj);
if (!apiResult.ok) return apiResult;
if (Object.hasOwn(obj, "agents")) {
return err(
new Error(
"agents: key is no longer supported — declare adapters on WorkflowSpec roles (RFC-003)",
),
);
}
const extractResult = parseExtract(obj);
if (!extractResult.ok) return extractResult;
return ok({
maxRounds: maxRoundsResult.value,
senses,
workflows: workflowsResult.value,
api: apiResult.value,
extract: extractResult.value,
});
}
-9
View File
@@ -1,9 +0,0 @@
export type Result<T, E = Error> = { ok: true; value: T } | { ok: false; error: E };
export function ok<T>(value: T): Result<T, never> {
return { ok: true, value };
}
export function err<E = Error>(error: E): Result<never, E> {
return { ok: false, error };
}
-40
View File
@@ -1,40 +0,0 @@
import type { SenseConfig } from "./config.js";
function formatIntervalMs(ms: number): string {
const totalSeconds = Math.floor(ms / 1000);
if (totalSeconds < 60) return `${totalSeconds}s`;
const minutes = Math.floor(totalSeconds / 60);
if (minutes < 60) return `${minutes}m`;
const hours = Math.floor(minutes / 60);
const remainingMinutes = minutes % 60;
return `${hours}h ${remainingMinutes}m`;
}
/** Human-readable label for a sense schedule (`interval` and/or `on`). */
export function labelSenseTrigger(slice: Pick<SenseConfig, "interval" | "on">): string {
const parts: string[] = [];
if (slice.interval !== null) {
parts.push(`every ${formatIntervalMs(slice.interval)}`);
}
if (slice.on.length > 0) {
parts.push(`on: ${slice.on.join(", ")}`);
}
if (parts.length === 0) {
return "trigger (no interval or on)";
}
return parts.join(" · ");
}
/**
* Human-readable trigger labels for a sense from its `SenseConfig.interval` / `.on`.
* Returns an empty array when the sense is missing or has no schedule.
*/
export function senseTriggerLabels(
senseName: string,
senses: Record<string, SenseConfig>,
): string[] {
const sc = senses[senseName];
if (sc === undefined) return [];
if (sc.interval === null && sc.on.length === 0) return [];
return [labelSenseTrigger({ interval: sc.interval, on: sc.on })];
}
@@ -1,56 +0,0 @@
import type { WorkflowTrigger } from "./config.js";
import { isPlainRecord } from "./is-plain-record.js";
import type { Result } from "./result.js";
import { err, ok } from "./result.js";
/** Normalized non-null compute output for the kernel (unknown signal payload). */
export type RoutedSenseOutput = {
signal: unknown;
workflow: WorkflowTrigger | null;
};
/**
* Validates a structured workflow trigger object from Sense compute or IPC.
*/
export function parseWorkflowTrigger(value: unknown): Result<WorkflowTrigger> {
if (!isPlainRecord(value)) {
return err(new Error("workflow trigger must be a plain object"));
}
const nameRaw = value.name;
if (typeof nameRaw !== "string" || nameRaw.trim().length === 0) {
return err(new Error('workflow trigger: "name" must be a non-empty string'));
}
const maxRounds = value.maxRounds;
if (typeof maxRounds !== "number" || !Number.isInteger(maxRounds) || maxRounds < 1) {
return err(new Error('workflow trigger: "maxRounds" must be an integer >= 1'));
}
const prompt = value.prompt;
if (typeof prompt !== "string") {
return err(new Error('workflow trigger: "prompt" must be a string'));
}
const dryRun = value.dryRun;
if (typeof dryRun !== "boolean") {
return err(new Error('workflow trigger: "dryRun" must be a boolean'));
}
return ok({ name: nameRaw.trim(), maxRounds, prompt, dryRun });
}
/**
* Interprets a Sense compute non-null return value for the engine.
* - Explicit `{ signal, workflow }` (workflow may be null): validates `workflow` when non-null.
* - Any other value: treated as `{ signal: payload, workflow: null }` (shorthand).
*/
export function routeSenseComputeOutput(payload: unknown): Result<RoutedSenseOutput> {
if (isPlainRecord(payload) && Object.hasOwn(payload, "signal")) {
const wfRaw = Object.hasOwn(payload, "workflow") ? payload.workflow : null;
if (wfRaw === null) {
return ok({ signal: payload.signal, workflow: null });
}
const parsed = parseWorkflowTrigger(wfRaw);
if (!parsed.ok) {
return ok({ signal: payload.signal, workflow: null });
}
return ok({ signal: payload.signal, workflow: parsed.value });
}
return ok({ signal: payload, workflow: null });
}
+116
View File
@@ -1,3 +1,8 @@
import type { SQLiteTable } from "drizzle-orm/sqlite-core";
import type { ComputeResult, SenseConfig, WorkflowTrigger } from "./config.js";
import { type Result, err, isPlainRecord, ok } from "./util.js";
export type Signal = {
id: number;
senseId: string;
@@ -15,3 +20,114 @@ export type SenseInfo = {
triggers: string[];
lastSignalTimestamp: number | null;
};
/**
* The function signature every sense `src/index.ts` must export as a named
* `compute` export.
*
* Pure: no DB, no peers.
* Return `null` to stay silent, or `{ signal, workflow }` to emit a Signal
* (and optionally trigger a Workflow).
* The runtime handles persistence via `db.insert(table).values(result.signal)`.
*/
export type SenseComputeFn<T = unknown> = () => Promise<ComputeResult<T>>;
/**
* The full shape a sense module (`src/index.ts`) must export.
* `compute` provides the data; `table` tells the runtime where to persist it.
*/
export type SenseModule<T = unknown> = {
compute: SenseComputeFn<T>;
table: SQLiteTable;
};
/** Normalized non-null compute output for the kernel (unknown signal payload). */
export type RoutedSenseOutput = {
signal: unknown;
workflow: WorkflowTrigger | null;
};
function formatIntervalMs(ms: number): string {
const totalSeconds = Math.floor(ms / 1000);
if (totalSeconds < 60) return `${totalSeconds}s`;
const minutes = Math.floor(totalSeconds / 60);
if (minutes < 60) return `${minutes}m`;
const hours = Math.floor(minutes / 60);
const remainingMinutes = minutes % 60;
return `${hours}h ${remainingMinutes}m`;
}
/** Human-readable label for a sense schedule (`interval` and/or `on`). */
export function labelSenseTrigger(slice: Pick<SenseConfig, "interval" | "on">): string {
const parts: string[] = [];
if (slice.interval !== null) {
parts.push(`every ${formatIntervalMs(slice.interval)}`);
}
if (slice.on.length > 0) {
parts.push(`on: ${slice.on.join(", ")}`);
}
if (parts.length === 0) {
return "trigger (no interval or on)";
}
return parts.join(" · ");
}
/**
* Human-readable trigger labels for a sense from its `SenseConfig.interval` / `.on`.
* Returns an empty array when the sense is missing or has no schedule.
*/
export function senseTriggerLabels(
senseName: string,
senses: Record<string, SenseConfig>,
): string[] {
const sc = senses[senseName];
if (sc === undefined) return [];
if (sc.interval === null && sc.on.length === 0) return [];
return [labelSenseTrigger({ interval: sc.interval, on: sc.on })];
}
/**
* Validates a structured workflow trigger object from Sense compute or IPC.
*/
export function parseWorkflowTrigger(value: unknown): Result<WorkflowTrigger> {
if (!isPlainRecord(value)) {
return err(new Error("workflow trigger must be a plain object"));
}
const nameRaw = value.name;
if (typeof nameRaw !== "string" || nameRaw.trim().length === 0) {
return err(new Error('workflow trigger: "name" must be a non-empty string'));
}
const maxRounds = value.maxRounds;
if (typeof maxRounds !== "number" || !Number.isInteger(maxRounds) || maxRounds < 1) {
return err(new Error('workflow trigger: "maxRounds" must be an integer >= 1'));
}
const prompt = value.prompt;
if (typeof prompt !== "string") {
return err(new Error('workflow trigger: "prompt" must be a string'));
}
const dryRun = value.dryRun;
if (typeof dryRun !== "boolean") {
return err(new Error('workflow trigger: "dryRun" must be a boolean'));
}
return ok({ name: nameRaw.trim(), maxRounds, prompt, dryRun });
}
/**
* Interprets a Sense compute non-null return value for the engine.
* - Explicit `{ signal, workflow }` (workflow may be null): validates `workflow` when non-null.
* - Any other value: treated as `{ signal: payload, workflow: null }` (shorthand).
*/
export function routeSenseComputeOutput(payload: unknown): Result<RoutedSenseOutput> {
if (isPlainRecord(payload) && Object.hasOwn(payload, "signal")) {
const wfRaw = Object.hasOwn(payload, "workflow") ? payload.workflow : null;
if (wfRaw === null) {
return ok({ signal: payload.signal, workflow: null });
}
const parsed = parseWorkflowTrigger(wfRaw);
if (!parsed.ok) {
return ok({ signal: payload.signal, workflow: null });
}
return ok({ signal: payload.signal, workflow: parsed.value });
}
return ok({ signal: payload, workflow: null });
}
@@ -1,7 +1,8 @@
import { spawn } from "node:child_process";
import { homedir } from "node:os";
import { join } from "node:path";
import { type Result, err, ok } from "./result.js";
export type Result<T, E = Error> = { ok: true; value: T } | { ok: false; error: E };
/** Compatible with `process.env` for `child_process.spawn`. */
export type SpawnEnv = Record<string, string | undefined>;
@@ -40,6 +41,42 @@ type SpawnSafeOptionsInput = SpawnSafeOptions | Omit<SpawnSafeOptions, "dryRun">
const DEFAULT_TIMEOUT_MS = 300_000;
export function ok<T>(value: T): Result<T, never> {
return { ok: true, value };
}
export function err<E = Error>(error: E): Result<never, E> {
return { ok: false, error };
}
/**
* Narrows `unknown` to a plain JSON-style object (not null, not array).
* Use after `JSON.parse` / YAML / IPC when validating structure field-by-field.
*/
export function isPlainRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
const DURATION_RE = /^(\d+)([smh])$/;
const DURATION_MULTIPLIERS: Record<string, number> = {
s: 1_000,
m: 60_000,
h: 3_600_000,
};
/**
* Parse a duration string such as `5s`, `10m`, `1h` to milliseconds.
* Used by `parseNerveConfig` sense/workflow duration fields.
*/
export function parseDurationStringToMs(value: string): Result<number> {
const match = DURATION_RE.exec(value);
if (!match) {
return err(new Error(`invalid duration "${value}" (expected e.g. "5s", "10m", "1h")`));
}
return ok(Number(match[1]) * DURATION_MULTIPLIERS[match[2]]);
}
/**
* PATH and PNPM_HOME for running `pnpm` and `nerve` from workflow roles.
* Uses the pnpm store home only (no npm user bin); binaries must resolve via PATH.
-24
View File
@@ -1,24 +0,0 @@
import type { Schema } from "./extract-layer.js";
import type { AgentFn, Moderator, RoleMeta, StartStep, WorkflowMessage } from "./workflow.js";
/** Static string or async prompt built from thread context (RFC-003 dynamic prompts). */
export type PromptInput =
| string
| ((start: StartStep, messages: WorkflowMessage[]) => Promise<string>);
/**
* Authoring-time role: adapter function, prompt, extract schema (RFC-003).
* Compiles to runtime `Role<Meta>` via `compileWorkflowSpec`.
*/
export type RoleSpec<Meta extends Record<string, unknown>> = {
adapter: AgentFn;
prompt: PromptInput;
meta: Schema<Meta>;
};
/** User-facing workflow authoring shape; compiles to `WorkflowDefinition`. */
export type WorkflowSpec<M extends RoleMeta> = {
name: string;
roles: { [K in keyof M]: RoleSpec<M[K]> };
moderator: Moderator<M>;
};
+28 -32
View File
@@ -21,39 +21,41 @@ export type WorkflowMessage = {
/** The typed output of a Role execution. */
export type RoleResult<Meta> = { content: string; meta: Meta };
/**
* A Role is a pure async function: receives the engine start frame plus prior
* role messages only (the start frame is not included in `messages`).
* Returns typed content + meta. Implementation can be an agent, LLM call,
* script, HTTP request, etc.
*/
export type Role<Meta> = (
start: StartStep,
messages: WorkflowMessage[],
) => Promise<RoleResult<Meta>>;
/** Maps role names to their meta types — the single generic that drives all inference. */
export type RoleMeta = Record<string, Record<string, unknown>>;
/** Engine start frame: prompt, max rounds cap, dry-run flag, and timestamps for the thread. */
/** Engine start frame: initial prompt, max rounds cap, and thread identity. */
export type StartStep = {
role: START;
content: string;
/** Thread identity (same as workflow `runId`); for role prompts and CLI `nerve thread` context. */
meta: { maxRounds: number; dryRun: boolean; threadId: string };
meta: { maxRounds: number; threadId: string };
timestamp: number;
};
/** Thread context passed to agent adapters (RFC-003): conversation frame, repo root, cancellation. */
export type WorkflowContext = {
/**
* Thread-scoped context for roles, moderator, and agent adapters (RFC-005).
* `workdir` and `signal` are adapter/engine config — not part of this shape.
*/
export type ThreadContext<M extends RoleMeta = RoleMeta> = {
threadId: string;
start: StartStep;
messages: WorkflowMessage[];
workdir: string;
signal: AbortSignal;
steps: RoleStep<M>[];
};
/**
* @deprecated Use {@link ThreadContext}.
*/
export type WorkflowContext = ThreadContext;
/**
* A Role receives the full thread context (start frame + prior role steps) and returns
* typed content + meta. Implementation can be an agent, LLM call, script, HTTP request, etc.
*/
export type Role<Meta> = (ctx: ThreadContext) => Promise<RoleResult<Meta>>;
/** Unified agent invocation — raw string output; structured meta uses the extract layer. */
export type AgentFn = (prompt: string, context: WorkflowContext) => Promise<string>;
export type AgentFn = (ctx: ThreadContext, systemPrompt: string) => Promise<string>;
/** A discriminated union of role steps after each execution, aligned with `StartStep` shape. */
export type RoleStep<M extends RoleMeta> = {
@@ -61,23 +63,17 @@ export type RoleStep<M extends RoleMeta> = {
}[keyof M & string];
/**
* Moderator input: the complete workflow history.
* Contains the start frame and all role steps so far.
* On initial call, `steps` is empty — moderator can check `steps.length === 0`.
* Round count is `steps.length`; maxRounds is in `start.meta.maxRounds`.
* @deprecated Use {@link ThreadContext}.
*/
export type ModeratorContext<M extends RoleMeta> = {
start: StartStep;
steps: RoleStep<M>[];
};
export type ModeratorContext<M extends RoleMeta> = ThreadContext<M>;
/**
* The moderator — a pure routing function. Receives the full workflow context
* (start frame + all prior steps). Returns the next role name or END.
* The moderator — a pure routing function. Receives the full thread context
* (start frame + all prior steps). On initial call, `steps` is empty.
* Round count is `steps.length`; maxRounds is in `start.meta.maxRounds`.
* Returns the next role name or END.
*/
export type Moderator<M extends RoleMeta> = (
context: ModeratorContext<M>,
) => (keyof M & string) | END;
export type Moderator<M extends RoleMeta> = (ctx: ThreadContext<M>) => (keyof M & string) | END;
/** The complete definition of a workflow, as authored by users. */
export type WorkflowDefinition<M extends RoleMeta> = {
@@ -1,188 +0,0 @@
import { describe, expect, it } from "vitest";
import type {
AgentFn,
ModeratorContext,
RoleMeta,
Schema,
StartStep,
WorkflowContext,
WorkflowDefinition,
WorkflowMessage,
WorkflowSpec,
} from "@uncaged/nerve-core";
import { END, START } from "@uncaged/nerve-core";
import { compileWorkflowSpec } from "../compile-workflow-spec.js";
type DemoMeta = { n: number };
function echoAdapter(): AgentFn {
return async (prompt: string, _ctx: WorkflowContext) => prompt;
}
function makeStart(threadId = "t1"): StartStep {
return {
role: START,
content: "",
meta: { maxRounds: 10, dryRun: false, threadId },
timestamp: Date.now(),
};
}
function makeContext(start: StartStep, messages: WorkflowMessage[]): WorkflowContext {
return {
start,
messages,
workdir: "/tmp/repo",
signal: new AbortController().signal,
};
}
describe("compileWorkflowSpec", () => {
it("compiles WorkflowSpec to WorkflowDefinition shape", () => {
const witness: DemoMeta | null = null;
const schema: Schema<DemoMeta> = { witness };
const spec: WorkflowSpec<{ main: DemoMeta }> = {
name: "demo",
roles: {
main: {
adapter: echoAdapter(),
prompt: "hello",
meta: schema,
},
},
moderator: (_ctx: ModeratorContext<{ main: DemoMeta }>) => END,
};
const def = compileWorkflowSpec(spec, {
extractFn: async <T>(raw: string, _s: Schema<T>) => ({ n: raw.length }) as T,
createContext: makeContext,
});
expect(def.name).toBe("demo");
expect(typeof def.roles.main).toBe("function");
expect(def.moderator).toBe(spec.moderator);
});
it("runs AgentFn then ExtractFn in order", async () => {
const witness: DemoMeta | null = null;
const schema: Schema<DemoMeta> = { witness };
const order: string[] = [];
const baseEcho = echoAdapter();
const spyAgent: AgentFn = async (prompt, ctx) => {
order.push("agent");
return baseEcho(prompt, ctx);
};
const spec: WorkflowSpec<{ main: DemoMeta }> = {
name: "order-test",
roles: {
main: {
adapter: spyAgent,
prompt: "ping",
meta: schema,
},
},
moderator: () => END,
};
const def = compileWorkflowSpec(spec, {
extractFn: async <T>(raw: string, _sch: Schema<T>) => {
order.push("extract");
return { n: raw.length } as T;
},
createContext: makeContext,
});
const start = makeStart();
await def.roles.main(start, []);
expect(order).toEqual(["agent", "extract"]);
});
it("passes WorkflowContext from createContext to AgentFn (adapter owns timeout)", async () => {
const witness: DemoMeta | null = null;
const schema: Schema<DemoMeta> = { witness };
const seenCtx: WorkflowContext[] = [];
const adapter: AgentFn = async (_prompt, ctx) => {
seenCtx.push(ctx);
return "x";
};
const spec: WorkflowSpec<{ main: DemoMeta }> = {
name: "ctx",
roles: {
main: {
adapter,
prompt: "x",
meta: schema,
},
},
moderator: () => END,
};
await compileWorkflowSpec(spec, {
extractFn: async <T>(_raw: string, _s: Schema<T>) => ({ n: 0 }) as T,
createContext: makeContext,
}).roles.main(makeStart(), []);
expect(seenCtx).toHaveLength(1);
expect(seenCtx[0].workdir).toBe("/tmp/repo");
});
it("resolves dynamic prompt functions before AgentFn", async () => {
const witness: DemoMeta | null = null;
const schema: Schema<DemoMeta> = { witness };
const spec: WorkflowSpec<{ main: DemoMeta }> = {
name: "dyn",
roles: {
main: {
adapter: echoAdapter(),
prompt: async (start, messages) => `tid=${start.meta.threadId} n=${messages.length}`,
meta: schema,
},
},
moderator: () => END,
};
const def = compileWorkflowSpec(spec, {
extractFn: async <T>(raw: string, _s: Schema<T>) => ({ n: raw.length }) as T,
createContext: makeContext,
});
const start = makeStart("thread-x");
const msgs: WorkflowMessage[] = [{ role: "a", content: "m", meta: {}, timestamp: 1 }];
const out = await def.roles.main(start, msgs);
expect(out.content).toBe("tid=thread-x n=1");
expect(out.meta.n).toBe(out.content.length);
});
});
describe("backward compatibility", () => {
it("hand-written Role-based WorkflowDefinition remains valid", async () => {
type M = RoleMeta & { legacy: { id: string } };
const manual: WorkflowDefinition<M> = {
name: "legacy",
roles: {
legacy: async (_start, _messages) => ({
content: "hi",
meta: { id: "a" },
}),
},
moderator: (_ctx: ModeratorContext<M>) => END,
};
const start = makeStart();
const out = await manual.roles.legacy(start, []);
expect(out.content).toBe("hi");
expect(out.meta.id).toBe("a");
});
});
@@ -28,6 +28,11 @@ function makeMockChild(pid = 1): MockChild {
child.connected = true;
child.exitCode = null;
child.pid = pid;
setImmediate(() => {
if (child.connected) {
child.emit("message", { type: "ready" });
}
});
child.send = vi.fn((msg: unknown) => {
if (
msg !== null &&
@@ -132,6 +137,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
mgr.startWorkflow("my-wf", { prompt: "test 1", maxRounds: 10, dryRun: false });
mgr.startWorkflow("my-wf", { prompt: "test 2", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mgr.activeCount("my-wf")).toBe(2);
// Simulate unexpected exit (not shutdown)
@@ -159,6 +165,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mgr.activeCount("my-wf")).toBe(2);
const child = mockChildren[0];
@@ -183,6 +190,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mockChildren).toHaveLength(1);
const child = mockChildren[0];
@@ -216,6 +224,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const firstChild = mockChildren[0];
firstChild.exitCode = 1;
firstChild.connected = false;
@@ -260,6 +269,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
// Start one thread to fill the concurrency slot (so queued run stays queued on respawn)
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const firstChild = mockChildren[0];
firstChild.exitCode = 1;
firstChild.connected = false;
@@ -285,6 +295,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const child = mockChildren[0];
const startCall = (child.send as ReturnType<typeof vi.fn>).mock.calls[0];
@@ -322,6 +333,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
const launch = { prompt: "build-docker for myrepo", maxRounds: 10, dryRun: false };
mgr.startWorkflow("my-wf", launch);
await vi.runAllTimersAsync();
const startedCall = logStore.upsertWorkflowRun.mock.calls.find(
(args: any[]) => (args[0] as { type: string }).type === "started",
@@ -357,6 +369,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
// Start one thread to fill the concurrency slot
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const firstChild = mockChildren[0];
// Crash once → respawn → crash again → second respawn
@@ -398,6 +411,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const firstChild = mockChildren[0];
firstChild.exitCode = 1;
firstChild.connected = false;
@@ -428,6 +442,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("crash-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
// Crash the worker 6 times in rapid succession (within CRASH_WINDOW_MS = 60s)
for (let i = 0; i < 6; i++) {
@@ -0,0 +1,9 @@
// Ready then crashes on a timer; still echoes IPC so parent tests can send after respawn
process.on("message", (msg) => {
if (msg && msg.type === "shutdown") {
process.exit(0);
}
process.send({ type: "echo", payload: msg });
});
process.send({ type: "ready" });
setTimeout(() => process.exit(1), 50);
@@ -0,0 +1,9 @@
// Simple test worker: sends ready, echoes messages, handles shutdown
process.on("message", (msg) => {
if (msg && msg.type === "shutdown") {
process.exit(0);
}
// Echo back with 'echo' type
process.send({ type: "echo", payload: msg });
});
process.send({ type: "ready" });
@@ -0,0 +1,9 @@
// Like echo-worker but writes stderr for tail diagnostics
console.error("stderr-marker");
process.on("message", (msg) => {
if (msg && msg.type === "shutdown") {
process.exit(0);
}
process.send({ type: "echo", payload: msg });
});
process.send({ type: "ready" });
@@ -33,6 +33,11 @@ function makeMockChild(pid = 1): MockChild {
child.connected = true;
child.exitCode = null;
child.pid = pid;
setImmediate(() => {
if (child.connected) {
child.emit("message", { type: "ready" });
}
});
child.send = vi.fn((msg: unknown) => {
if (
msg !== null &&
@@ -114,6 +119,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mockChildren).toHaveLength(1);
// Remove workflow from config before drain completes
@@ -134,6 +140,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mgr.activeCount("my-wf")).toBe(2);
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
@@ -165,6 +172,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mockChildren).toHaveLength(1);
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
@@ -181,6 +189,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mockChildren).toHaveLength(1);
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
@@ -198,6 +207,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
await vi.runAllTimersAsync();
@@ -223,6 +233,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "first", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
await vi.runAllTimersAsync();
@@ -230,6 +241,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
// Start a new thread on the fresh worker
mgr.startWorkflow("my-wf", { prompt: "second", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const newChild = mockChildren[1];
const startCalls = (newChild.send as ReturnType<typeof vi.fn>).mock.calls.filter(
@@ -257,12 +269,13 @@ describe("WorkflowManager — drainWhenIdle (hot reload without interrupting in-
vi.clearAllMocks();
});
it("does not send shutdown while a thread is still active", () => {
it("does not send shutdown while a thread is still active", async () => {
const logStore = makeLogStore();
const config = makeWfConfig({ "my-wf": { concurrency: 1, overflow: "drop" } });
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const child = mockChildren[0];
mgr.drainWhenIdle("my-wf");
@@ -282,6 +295,7 @@ describe("WorkflowManager — drainWhenIdle (hot reload without interrupting in-
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const child = mockChildren[0];
const runId = (child.send as ReturnType<typeof vi.fn>).mock.calls[0][0] as { runId: string };
@@ -311,6 +325,7 @@ describe("WorkflowManager — drainWhenIdle (hot reload without interrupting in-
mgr.startWorkflow("my-wf", { prompt: "a", maxRounds: 10, dryRun: false });
mgr.startWorkflow("my-wf", { prompt: "b", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const child = mockChildren[0];
const sendMock = child.send as ReturnType<typeof vi.fn>;
const runIdA = (sendMock.mock.calls[0][0] as { runId: string }).runId;
@@ -355,6 +370,7 @@ describe("WorkflowManager — drainWhenIdle (hot reload without interrupting in-
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const child = mockChildren[0];
const runId = (child.send as ReturnType<typeof vi.fn>).mock.calls[0][0] as { runId: string };
@@ -388,6 +404,7 @@ describe("WorkflowManager — drainWhenIdle (hot reload without interrupting in-
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const child = mockChildren[0];
const runId = (child.send as ReturnType<typeof vi.fn>).mock.calls[0][0] as { runId: string };
@@ -414,6 +431,7 @@ describe("WorkflowManager — drainWhenIdle (hot reload without interrupting in-
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { prompt: "once", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const firstChild = mockChildren[0];
const runId = (firstChild.send as ReturnType<typeof vi.fn>).mock.calls[0][0] as {
runId: string;
@@ -471,6 +489,7 @@ describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
// Trigger a workflow thread so a worker is spawned
kernel.workflowManager.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
// Manually call drainAndRespawn (simulating what kernel does on workflow file change)
const drainPromise = kernel.workflowManager.drainAndRespawn("my-wf", 1000);
@@ -511,6 +530,7 @@ describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
maxRounds: 10,
dryRun: false,
});
await vi.runAllTimersAsync();
expect(mockChildren).toHaveLength(1);
// Reload config without old-wf
@@ -551,6 +571,7 @@ describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
});
kernel.workflowManager.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const workersBefore = mockChildren.length;
// Reload with updated concurrency — should NOT spawn a new workflow worker
@@ -573,6 +594,7 @@ describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
// Can now start up to 5 concurrent threads (previously only 1)
kernel.workflowManager.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
kernel.workflowManager.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(kernel.workflowManager.activeCount("my-wf")).toBe(3);
const stopPromise = kernel.stop();
@@ -70,6 +70,14 @@ const { createKernel } = await import("../kernel.js");
// Helpers
// ---------------------------------------------------------------------------
/** Sense worker `fork` runs on the next microtask per scheduled `start`. */
async function flushSenseWorkerForkMicrotasks(kernel: { groups: Set<string> }): Promise<void> {
const n = kernel.groups.size;
for (let i = 0; i < n; i++) {
await Promise.resolve();
}
}
function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
return {
senses: {
@@ -142,6 +150,8 @@ describe("kernel — getHealth", () => {
},
});
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
const health = kernel.getHealth();
expect(health.activeSenses).toBe(3);
@@ -171,6 +181,8 @@ describe("kernel — restartGroup", () => {
it("sends shutdown to old worker and spawns new one", async () => {
const config = makeConfig();
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
expect(mockChildren.length).toBe(1);
const oldChild = mockChildren[0];
@@ -178,6 +190,7 @@ describe("kernel — restartGroup", () => {
const restartPromise = kernel.restartGroup("system");
// The shutdown message triggers exit in the mock
await restartPromise;
await vi.runAllTimersAsync();
// A new child should have been spawned
expect(mockChildren.length).toBe(2);
@@ -191,6 +204,8 @@ describe("kernel — restartGroup", () => {
it("restartGroup on unknown group does nothing", async () => {
const config = makeConfig();
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
expect(mockChildren.length).toBe(1);
await kernel.restartGroup("nonexistent");
@@ -218,6 +233,8 @@ describe("kernel — reloadConfig", () => {
it("adds new group worker when new sense group appears", async () => {
const config = makeConfig();
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
expect(mockChildren.length).toBe(1); // only system group
expect(kernel.groups.has("network")).toBe(false);
@@ -249,6 +266,9 @@ describe("kernel — reloadConfig", () => {
api: { port: null, token: null, host: "127.0.0.1" },
});
await Promise.resolve();
await vi.runAllTimersAsync();
expect(kernel.groups.has("network")).toBe(true);
expect(mockChildren.length).toBe(2); // system + network
@@ -283,6 +303,8 @@ describe("kernel — reloadConfig", () => {
api: { port: null, token: null, host: "127.0.0.1" },
};
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
expect(mockChildren.length).toBe(2);
expect(kernel.groups.has("network")).toBe(true);
@@ -308,6 +330,7 @@ describe("kernel — reloadConfig", () => {
});
expect(kernel.groups.has("network")).toBe(false);
await vi.runAllTimersAsync();
// Network child should have received shutdown
expect(networkChild.send).toHaveBeenCalledWith(expect.objectContaining({ type: "shutdown" }));
@@ -317,6 +340,8 @@ describe("kernel — reloadConfig", () => {
it("health reflects updated sense count after reloadConfig", async () => {
const config = makeConfig();
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
expect(kernel.getHealth().activeSenses).toBe(1);
@@ -29,6 +29,9 @@ type MockChild = EventEmitter & {
function makeMockChild(pid = 1): MockChild {
const child = new EventEmitter() as MockChild;
child.connected = true;
setImmediate(() => {
child.emit("message", { type: "ready" });
});
child.send = vi.fn((msg: unknown) => {
if (
msg !== null &&
@@ -136,6 +139,7 @@ describe("kernel.triggerSense()", () => {
logStore: makeMockLogStore() as never,
});
await vi.runAllTimersAsync();
expect(() => kernel.triggerSense("no-such-sense")).toThrow(/Unknown sense/);
await kernel.stop();
@@ -169,6 +173,7 @@ describe("kernel.triggerSense()", () => {
logStore: makeMockLogStore() as never,
});
await vi.runAllTimersAsync();
// Two groups → two workers
expect(mockChildren.length).toBe(2);
@@ -214,6 +219,7 @@ describe("kernel.triggerSense()", () => {
logStore: makeMockLogStore() as never,
});
await vi.runAllTimersAsync();
// Both senses share the "system" group → one worker only
expect(mockChildren.length).toBe(1);
const worker = mockChildren[0];
@@ -237,6 +243,7 @@ describe("kernel.triggerSense()", () => {
logStore: makeMockLogStore() as never,
});
await new Promise<void>((resolve) => setImmediate(resolve));
const worker = mockChildren[0];
worker.connected = false;
@@ -102,6 +102,13 @@ function makeLogStore() {
};
}
async function flushSenseWorkerForkMicrotasks(kernel: { groups: Set<string> }): Promise<void> {
const n = kernel.groups.size;
for (let i = 0; i < n; i++) {
await Promise.resolve();
}
}
function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
return {
senses: {
@@ -164,6 +171,8 @@ describe("kernel + workflowManager integration", () => {
workerScript: "fake-worker.js",
logStore,
});
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
// Simulate a sense worker sending a signal with workflow launch payload
// The kernel's handleWorkerMessage processes "signal" type messages
@@ -185,6 +194,8 @@ describe("kernel + workflowManager integration", () => {
});
}
await vi.runAllTimersAsync();
// A workflow worker should be spawned and a start-thread message sent
const workflowWorker = mockChildren.find((c) =>
(c.send as ReturnType<typeof vi.fn>).mock.calls.some(
@@ -222,6 +233,8 @@ describe("kernel + workflowManager integration", () => {
workerScript: "fake-worker.js",
logStore,
});
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
// Simulate sense worker returning a signal plus workflow launch
const workerPool = mockChildren[0];
@@ -241,6 +254,8 @@ describe("kernel + workflowManager integration", () => {
});
}
await vi.runAllTimersAsync();
// Find the start-thread call and verify triggerPayload
const startThreadCall = mockChildren
.flatMap((c) => (c.send as ReturnType<typeof vi.fn>).mock.calls as [unknown][])
@@ -275,6 +290,8 @@ describe("kernel + workflowManager integration", () => {
workerScript: "fake-worker.js",
logStore,
});
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
const workerPool = mockChildren[0];
if (workerPool) {
@@ -293,6 +310,8 @@ describe("kernel + workflowManager integration", () => {
});
}
await vi.runAllTimersAsync();
const senseEntries = logStore.append.mock.calls
.map((c) => c[0] as { source: string; type: string; refId: string | null })
.filter((e) => e.source === "sense" && e.refId === "cpu-usage");
@@ -337,6 +356,8 @@ describe("kernel + workflowManager integration", () => {
workerScript: "fake-worker.js",
logStore,
});
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
// Emit a regular signal (shorthand payload) — should NOT trigger any workflow
const workerPool = mockChildren[0];
@@ -387,6 +408,8 @@ describe("kernel + workflowManager integration", () => {
workerScript: "fake-worker.js",
logStore,
});
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
// Simulate sense compute returning a signal plus workflow launch
const workerPool = mockChildren[0];
@@ -406,6 +429,8 @@ describe("kernel + workflowManager integration", () => {
});
}
await vi.runAllTimersAsync();
expect(logStore.upsertWorkflowRun).toHaveBeenCalledWith(
expect.objectContaining({ source: "workflow", type: "started" }),
expect.objectContaining({ workflow: "log-test-workflow", status: "started" }),
@@ -440,6 +465,8 @@ describe("kernel + workflowManager integration", () => {
workerScript: "fake-worker.js",
logStore,
});
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
// Reload with a workflow added
const newConfig: NerveConfig = {
@@ -479,6 +506,8 @@ describe("kernel + workflowManager integration", () => {
});
}
await vi.runAllTimersAsync();
const startThreadCall = mockChildren
.flatMap((c) => (c.send as ReturnType<typeof vi.fn>).mock.calls as [unknown][])
.find(
@@ -517,6 +546,8 @@ describe("kernel + workflowManager integration", () => {
workerScript: "fake-worker.js",
logStore,
});
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
// Reload with the workflow removed
const newConfig: NerveConfig = {
@@ -561,6 +592,8 @@ describe("kernel + workflowManager integration", () => {
});
}
await vi.runAllTimersAsync();
const startThreadCall = mockChildren
.flatMap((c) => (c.send as ReturnType<typeof vi.fn>).mock.calls as [unknown][])
.find(
@@ -600,6 +633,8 @@ describe("kernel + workflowManager integration", () => {
workerScript: "fake-worker.js",
logStore,
});
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
// Trigger a workflow via sense compute return value
const workerPool = mockChildren[0];
@@ -619,6 +654,8 @@ describe("kernel + workflowManager integration", () => {
});
}
await vi.runAllTimersAsync();
const stopPromise = kernel.stop();
await vi.runAllTimersAsync();
await expect(stopPromise).resolves.toBeUndefined();
@@ -664,6 +701,8 @@ describe("kernel + workflowManager integration", () => {
workerScript: "fake-worker.js",
logStore,
});
await flushSenseWorkerForkMicrotasks(kernel);
await vi.runAllTimersAsync();
const health = kernel.getHealth();
expect(health).toHaveProperty("activeWorkflows");
+22 -2
View File
@@ -16,10 +16,12 @@ type MockChild = EventEmitter & {
send: ReturnType<typeof vi.fn>;
kill: ReturnType<typeof vi.fn>;
pid: number;
connected: boolean;
};
function makeMockChild(pid = 1): MockChild {
const child = new EventEmitter() as MockChild;
child.connected = true;
setImmediate(() => {
child.emit("message", { type: "ready" });
});
@@ -27,7 +29,10 @@ function makeMockChild(pid = 1): MockChild {
if (msg === null || typeof msg !== "object") return;
const m = msg as Record<string, unknown>;
if (m.type === "shutdown") {
setImmediate(() => child.emit("exit", 0, null));
setImmediate(() => {
child.connected = false;
child.emit("exit", 0, null);
});
return;
}
if (m.type === "compute" && typeof m.sense === "string") {
@@ -37,6 +42,7 @@ function makeMockChild(pid = 1): MockChild {
}
});
child.kill = vi.fn((_signal?: string) => {
child.connected = false;
child.emit("exit", null, _signal ?? "SIGKILL");
});
child.pid = pid;
@@ -59,6 +65,14 @@ const { createLogStore } = await import("@uncaged/nerve-store");
// Helpers
// ---------------------------------------------------------------------------
/** `WorkerRuntime.start` schedules `fork` on the next microtask — flush one tick per initial group. */
async function flushSenseWorkerForkMicrotasks(kernel: { groups: Set<string> }): Promise<void> {
const n = kernel.groups.size;
for (let i = 0; i < n; i++) {
await Promise.resolve();
}
}
function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
return {
senses: {
@@ -173,6 +187,7 @@ describe("kernel — message routing", () => {
},
});
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
const child = mockChildren[0];
child.emit("message", { type: "error", sense: "cpu-usage", error: "compute failed" });
@@ -201,6 +216,7 @@ describe("kernel — message routing", () => {
},
});
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
const child = mockChildren[0];
const callsBefore = stderrSpy.mock.calls.length;
@@ -228,6 +244,7 @@ describe("kernel — message routing", () => {
},
});
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
const child = mockChildren[0];
expect(() => child.emit("message", { type: "unknown-type" })).not.toThrow();
@@ -290,6 +307,7 @@ describe("kernel — groupForSense mapping", () => {
api: { port: null, token: null, host: "127.0.0.1" },
};
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
// system and network = 2 unique groups
expect(mockChildren.length).toBe(2);
@@ -311,8 +329,10 @@ describe("kernel — groupForSense mapping", () => {
},
});
const kernel = createKernel(config, nerveRoot);
await flushSenseWorkerForkMicrotasks(kernel);
const child = mockChildren[0];
child.emit("message", { type: "ready" });
vi.advanceTimersByTime(500);
expect(child.send).toHaveBeenCalledWith(
@@ -7,10 +7,9 @@ import { drizzle } from "drizzle-orm/node-sqlite";
import { integer, real, sqliteTable } from "drizzle-orm/sqlite-core";
import { describe, expect, it } from "vitest";
import { createBlobStore } from "@uncaged/nerve-store";
import { parseParentMessage } from "../ipc.js";
import { executeCompute, openPeerDb, openSenseDb, runMigrations } from "../sense-runtime.js";
import type { ComputeFn, DrizzleDB, PeerMap, SenseRuntime } from "../sense-runtime.js";
import { executeCompute, openSenseDb, runMigrations } from "../sense-runtime.js";
import type { DrizzleDB, SenseRuntime } from "../sense-runtime.js";
// ---------------------------------------------------------------------------
// Helpers
@@ -151,76 +150,50 @@ describe("openSenseDb", () => {
});
});
// ---------------------------------------------------------------------------
// openPeerDb
// ---------------------------------------------------------------------------
describe("openPeerDb", () => {
it("opens an existing db in read-only mode", () => {
// Create a writable db first
const dbPath = makeTempDbPath();
const sqlite = new DatabaseSync(dbPath);
sqlite.exec(INIT_SQL);
sqlite.prepare("INSERT INTO samples (ts, value) VALUES (1, 42.0)").run();
sqlite.close();
const result = openPeerDb(dbPath);
expect(result.ok).toBe(true);
if (!result.ok) return;
// Should be able to read
const peerDb = result.value;
const rows = peerDb.select().from(samples).all();
expect(rows).toHaveLength(1);
expect(rows[0].value).toBe(42.0);
});
it("returns err when db path does not exist", () => {
const result = openPeerDb("/nonexistent/path/to/peer.db");
expect(result.ok).toBe(false);
});
});
// ---------------------------------------------------------------------------
// executeCompute
// ---------------------------------------------------------------------------
describe("executeCompute", () => {
function makeRuntime(computeFn: ComputeFn): {
runtime: SenseRuntime;
sqlite: DatabaseSync;
} {
const sqlite = new DatabaseSync(":memory:");
sqlite.exec(INIT_SQL);
const db = drizzle({ client: sqlite }) as DrizzleDB;
function makeRuntime(
computeFn: SenseRuntime["compute"],
sqlite?: DatabaseSync,
): { runtime: SenseRuntime; sqlite: DatabaseSync } {
const db_sqlite = sqlite ?? new DatabaseSync(":memory:");
if (!sqlite) db_sqlite.exec(INIT_SQL);
const db = drizzle({ client: db_sqlite }) as DrizzleDB;
return {
runtime: { name: "test-sense", db, compute: computeFn, persistSignal: () => {} },
sqlite,
runtime: {
name: "test-sense",
db,
compute: computeFn,
table: samples,
persistSignal: () => {},
},
sqlite: db_sqlite,
};
}
const emptyPeers: PeerMap = {};
it("returns non-null and inserts into table when compute returns data", async () => {
const { runtime, sqlite } = makeRuntime(async () => ({
signal: { ts: 1000, value: 0.5 },
workflow: null,
}));
it("returns the compute result when compute returns a non-null value", async () => {
const { runtime, sqlite } = makeRuntime(async (db) => {
await db.insert(samples).values({ ts: Date.now(), value: 0.5 });
return { signal: 0.5, workflow: null };
});
const result = await executeCompute(runtime, emptyPeers);
const result = await executeCompute(runtime);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.value).toEqual({ signal: 0.5, workflow: null });
expect(result.value).toEqual({ signal: { ts: 1000, value: 0.5 }, workflow: null });
const rows = sqlite.prepare("SELECT * FROM samples").all();
expect(rows).toHaveLength(1);
sqlite.close();
});
it("returns null (no signal) when compute returns null", async () => {
it("returns null and does not insert when compute returns null", async () => {
const { runtime, sqlite } = makeRuntime(async () => null);
const result = await executeCompute(runtime, emptyPeers);
const result = await executeCompute(runtime);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.value).toBeNull();
@@ -235,60 +208,14 @@ describe("executeCompute", () => {
throw new Error("something went wrong");
});
const result = await executeCompute(runtime, emptyPeers);
const result = await executeCompute(runtime);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toContain("something went wrong");
sqlite.close();
});
it("compute can read from peers", async () => {
// Set up a peer db with data
const peerSqlite = new DatabaseSync(":memory:");
peerSqlite.exec(INIT_SQL);
peerSqlite.prepare("INSERT INTO samples (ts, value) VALUES (100, 3.14)").run();
const peerDb = drizzle({ client: peerSqlite }) as DrizzleDB;
const peers: PeerMap = { "other-sense": peerDb };
const { runtime, sqlite } = makeRuntime(async (_db, p) => {
const rows = await p["other-sense"].select().from(samples).all();
return rows.length > 0 ? { signal: rows[0].value, workflow: null } : null;
});
const result = await executeCompute(runtime, peers);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.value).toEqual({ signal: 3.14, workflow: null });
peerSqlite.close();
sqlite.close();
});
it("write to own db does not affect peer db (isolation)", async () => {
const peerSqlite = new DatabaseSync(":memory:");
peerSqlite.exec(INIT_SQL);
const peerDb = drizzle({ client: peerSqlite }) as DrizzleDB;
const peers: PeerMap = { "peer-sense": peerDb };
const { runtime, sqlite } = makeRuntime(async (db) => {
await db.insert(samples).values({ ts: 999, value: 9.9 });
return { signal: 9.9, workflow: null };
});
await executeCompute(runtime, peers);
const peerRows = peerSqlite.prepare("SELECT * FROM samples").all();
expect(peerRows).toHaveLength(0);
const ownRows = sqlite.prepare("SELECT * FROM samples").all();
expect(ownRows).toHaveLength(1);
peerSqlite.close();
sqlite.close();
});
it("inserts correctly into the sense db directory path", async () => {
it("inserts correctly into the sense db from openSenseDb", async () => {
const dbPath = makeTempDbPath();
const migrationsDir = makeTempMigrationsDir(INIT_SQL);
const dbResult = openSenseDb(dbPath, migrationsDir);
@@ -301,14 +228,12 @@ describe("executeCompute", () => {
const runtime: SenseRuntime = {
name: "cpu-usage",
db,
compute: async (d) => {
await d.insert(samples).values({ ts: 1000, value: 1.23 });
return { signal: 1.23, workflow: null };
},
compute: async () => ({ signal: { ts: 1000, value: 1.23 }, workflow: null }),
table: samples,
persistSignal: () => {},
};
const result = await executeCompute(runtime, {});
const result = await executeCompute(runtime);
expect(result.ok).toBe(true);
const rows = dbSqlite.prepare("SELECT * FROM samples").all() as Array<{
@@ -323,17 +248,10 @@ describe("executeCompute", () => {
it("returns err when compute exceeds timeoutMs", async () => {
const { runtime, sqlite } = makeRuntime(
(_db, _peers, options) =>
new Promise<null>((resolve, reject) => {
const t = setTimeout(() => resolve(null), 5_000);
options?.signal.addEventListener("abort", () => {
clearTimeout(t);
reject(new Error("aborted"));
});
}),
() => new Promise<null>((resolve) => setTimeout(() => resolve(null), 5_000)),
);
const result = await executeCompute(runtime, emptyPeers, 50);
const result = await executeCompute(runtime, 50);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toMatch(/timed out/i);
@@ -341,37 +259,14 @@ describe("executeCompute", () => {
});
it("completes within timeout when compute is fast", async () => {
const { runtime, sqlite } = makeRuntime(async () => ({ signal: 42, workflow: null }));
const result = await executeCompute(runtime, emptyPeers, 5_000);
const { runtime, sqlite } = makeRuntime(async () => ({
signal: { ts: 1, value: 42 },
workflow: null,
}));
const result = await executeCompute(runtime, 5_000);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.value).toEqual({ signal: 42, workflow: null });
sqlite.close();
});
it("passes AbortSignal to compute fn", async () => {
let capturedSignal: AbortSignal | undefined;
const { runtime, sqlite } = makeRuntime(async (_db, _peers, options) => {
capturedSignal = options?.signal;
return null;
});
await executeCompute(runtime, emptyPeers, 1_000);
expect(capturedSignal).toBeInstanceOf(AbortSignal);
sqlite.close();
});
it("passes BlobStore as options.blobs when blobStore argument is provided", async () => {
const blobsRoot = mkdtempSync(join(tmpdir(), "nerve-blobs-"));
const blobStore = createBlobStore(blobsRoot);
let seen: ReturnType<typeof createBlobStore> | undefined;
const { runtime, sqlite } = makeRuntime(async (_db, _peers, options) => {
seen = options?.blobs;
return null;
});
await executeCompute(runtime, emptyPeers, undefined, blobStore);
expect(seen).toBe(blobStore);
expect(result.value).toEqual({ signal: { ts: 1, value: 42 }, workflow: null });
sqlite.close();
});
});
@@ -429,19 +324,14 @@ describe("runMigrations journal", () => {
const first = runMigrations(sqlite, dir);
expect(first.ok).toBe(true);
// Insert a row so we can verify second run doesn't fail on CREATE TABLE
sqlite.exec("INSERT INTO samples (ts, value) VALUES (1, 1.0)");
// Run again — migration must NOT re-run (would fail without IF NOT EXISTS but
// the journal prevents it even for non-idempotent SQL)
const nonIdempotentSql = "CREATE TABLE samples2 (id INTEGER PRIMARY KEY)";
writeFileSync(join(dir, "0002_samples2.sql"), nonIdempotentSql);
// First time: creates samples2
const second = runMigrations(sqlite, dir);
expect(second.ok).toBe(true);
// Second time: 0002 already in journal, must not re-run
const third = runMigrations(sqlite, dir);
expect(third.ok).toBe(true);
@@ -50,6 +50,7 @@ async function startWorkerWithReady(
group: string,
): Promise<void> {
const pr = pool.startWorker(group);
await Promise.resolve();
const child = mockChildren[mockChildren.length - 1];
child.emit("message", { type: "ready" });
await pr;
@@ -137,6 +138,7 @@ describe("createSenseWorkerPool", () => {
expect(pool.activeGroupCount()).toBe(1);
pool.evictGroup("x");
expect(pool.hasWorkerForGroup("x")).toBe(false);
await Promise.resolve();
expect(mockChildren[0].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
@@ -159,6 +161,7 @@ describe("createSenseWorkerPool", () => {
const p = pool.restartGroup("g");
expect(onBeforeGroupRestart).toHaveBeenCalledWith("g");
await Promise.resolve();
expect(mockChildren[0].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
@@ -171,7 +174,7 @@ describe("createSenseWorkerPool", () => {
});
it("onWorkerCrashed runs and schedules respawn after non-zero exit", async () => {
vi.useFakeTimers({ shouldAdvanceTime: true });
vi.useFakeTimers();
const onWorkerCrashed = vi.fn();
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
@@ -0,0 +1,181 @@
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { afterEach, describe, expect, it, vi } from "vitest";
import { createWorkerRuntime } from "../worker-runtime.js";
const fixturesDir = join(dirname(fileURLToPath(import.meta.url)), "fixtures");
const echoWorkerPath = join(fixturesDir, "echo-worker.js");
const crashWorkerPath = join(fixturesDir, "crash-worker.js");
const stderrWorkerPath = join(fixturesDir, "stderr-worker.js");
function baseConfig(script: string) {
return {
script,
argsForKey: () => [],
forwardStderr: true,
onMessage: vi.fn(),
onReady: vi.fn(),
onExit: vi.fn(),
onCrashLimitReached: null,
respawn: {
enabled: true,
maxCrashes: 6,
windowMs: 60_000,
delayMs: 80,
allowRespawn: null,
},
shutdownTimeoutMs: 5000,
};
}
describe("createWorkerRuntime", () => {
const runtimes: Array<{ shutdown: () => Promise<void> }> = [];
afterEach(async () => {
await Promise.all(runtimes.splice(0).map((r) => r.shutdown()));
});
function track<R extends { shutdown: () => Promise<void> }>(r: R): R {
runtimes.push(r);
return r;
}
it("start + send message + receive echo", async () => {
const incoming: unknown[] = [];
const rt = track(
createWorkerRuntime({
...baseConfig(echoWorkerPath),
onMessage: (_key, msg) => {
incoming.push(msg);
},
}),
);
await rt.start("a");
expect(rt.has("a")).toBe(true);
await rt.send("a", { type: "ping", n: 1 });
await vi.waitFor(() => {
expect(incoming.some((m) => isEchoOf(m, { type: "ping", n: 1 }))).toBe(true);
});
await rt.shutdown();
});
it("cold start on send (no explicit start)", async () => {
const incoming: unknown[] = [];
const rt = track(
createWorkerRuntime({
...baseConfig(echoWorkerPath),
onMessage: (_key, msg) => {
incoming.push(msg);
},
}),
);
expect(rt.has("x")).toBe(false);
await rt.send("x", { type: "hi" });
await vi.waitFor(() => {
expect(rt.has("x")).toBe(true);
expect(incoming.some((m) => isEchoOf(m, { type: "hi" }))).toBe(true);
});
await rt.shutdown();
});
it("evict stops worker; has() is false", async () => {
const rt = track(createWorkerRuntime(baseConfig(echoWorkerPath)));
await rt.start("k");
expect(rt.has("k")).toBe(true);
await rt.evict("k", null);
expect(rt.has("k")).toBe(false);
await rt.shutdown();
});
it("drain stops and respawns (new pid)", async () => {
const rt = track(createWorkerRuntime(baseConfig(echoWorkerPath)));
await rt.start("k");
const before = rt.pid("k");
expect(before).not.toBeNull();
await rt.drain("k", null);
const after = rt.pid("k");
expect(after).not.toBeNull();
expect(after).not.toBe(before);
await rt.shutdown();
});
it("crash triggers auto-respawn", async () => {
const incoming: unknown[] = [];
const onExit = vi.fn();
const rt = track(
createWorkerRuntime({
...baseConfig(crashWorkerPath),
onExit,
onMessage: (_key, msg) => {
incoming.push(msg);
},
}),
);
await rt.start("c");
await vi.waitFor(() => expect(onExit.mock.calls.length).toBeGreaterThanOrEqual(1), {
timeout: 3000,
});
await vi.waitFor(() => expect(rt.has("c")).toBe(true), { timeout: 3000 });
await rt.send("c", { type: "after-crash" });
await vi.waitFor(() => {
expect(incoming.some((m) => isEchoOf(m, { type: "after-crash" }))).toBe(true);
});
await rt.shutdown();
});
it("crash limit reached → no more automatic respawns", async () => {
const rt = track(
createWorkerRuntime({
...baseConfig(crashWorkerPath),
respawn: {
enabled: true,
maxCrashes: 2,
windowMs: 60_000,
delayMs: 50,
allowRespawn: null,
},
}),
);
await rt.start("z");
await vi.waitFor(() => expect(rt.has("z")).toBe(false), { timeout: 8000 });
await rt.shutdown();
});
it("shutdown stops all workers", async () => {
const rt = track(createWorkerRuntime(baseConfig(echoWorkerPath)));
await rt.start("a");
await rt.start("b");
expect(rt.keys().sort()).toEqual(["a", "b"].sort());
await rt.shutdown();
expect(rt.keys()).toEqual([]);
expect(rt.has("a")).toBe(false);
expect(rt.has("b")).toBe(false);
});
it("stderrTail captures stderr output", async () => {
const rt = track(createWorkerRuntime(baseConfig(stderrWorkerPath)));
await rt.start("s");
await vi.waitFor(() => {
expect(rt.stderrTail("s")).toContain("stderr-marker");
});
await rt.shutdown();
});
});
function isEchoOf(msg: unknown, payload: unknown): boolean {
return (
typeof msg === "object" &&
msg !== null &&
(msg as Record<string, unknown>).type === "echo" &&
JSON.stringify((msg as Record<string, unknown>).payload) === JSON.stringify(payload)
);
}
@@ -26,6 +26,11 @@ function makeMockChild(pid = 1): MockChild {
child.connected = true;
child.exitCode = null;
child.pid = pid;
setImmediate(() => {
if (child.connected) {
child.emit("message", { type: "ready" });
}
});
child.send = vi.fn((msg: unknown) => {
if (
msg !== null &&
@@ -110,7 +115,7 @@ describe("WorkflowManager", () => {
});
describe("startWorkflow under concurrency limit dispatches thread", () => {
it("forks a worker and sends start-thread when active < concurrency", () => {
it("forks a worker and sends start-thread when active < concurrency", async () => {
const logStore = makeLogStore();
const config = makeConfig({
"my-workflow": { concurrency: 2, overflow: "drop" },
@@ -118,6 +123,7 @@ describe("WorkflowManager", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-workflow", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mockChildren).toHaveLength(1);
expect(mockChildren[0].send).toHaveBeenCalledWith(
@@ -126,7 +132,7 @@ describe("WorkflowManager", () => {
expect(mgr.activeCount("my-workflow")).toBe(1);
});
it("reuses the same worker for a second thread under the limit", () => {
it("reuses the same worker for a second thread under the limit", async () => {
const logStore = makeLogStore();
const config = makeConfig({
"my-workflow": { concurrency: 3, overflow: "drop" },
@@ -135,6 +141,7 @@ describe("WorkflowManager", () => {
mgr.startWorkflow("my-workflow", { prompt: "test 1", maxRounds: 10, dryRun: false });
mgr.startWorkflow("my-workflow", { prompt: "test 2", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
// Only one forked child — worker is reused
expect(mockChildren).toHaveLength(1);
@@ -142,7 +149,7 @@ describe("WorkflowManager", () => {
expect(mgr.activeCount("my-workflow")).toBe(2);
});
it("logs a 'started' event for each dispatched thread", () => {
it("logs a 'started' event for each dispatched thread", async () => {
const logStore = makeLogStore();
const config = makeConfig({
"my-workflow": { concurrency: 2, overflow: "drop" },
@@ -150,6 +157,7 @@ describe("WorkflowManager", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-workflow", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(logStore.upsertWorkflowRun).toHaveBeenCalledWith(
expect.objectContaining({ source: "workflow", type: "started" }),
@@ -159,7 +167,7 @@ describe("WorkflowManager", () => {
});
describe("startWorkflow at limit with drop overflow drops the request", () => {
it("does NOT send start-thread when at concurrency limit with overflow=drop", () => {
it("does NOT send start-thread when at concurrency limit with overflow=drop", async () => {
const logStore = makeLogStore();
const config = makeConfig({
"drop-wf": { concurrency: 1, overflow: "drop" },
@@ -169,6 +177,7 @@ describe("WorkflowManager", () => {
mgr.startWorkflow("drop-wf", { prompt: "first", maxRounds: 10, dryRun: false });
// now at limit — second call should be dropped
mgr.startWorkflow("drop-wf", { prompt: "second", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mgr.activeCount("drop-wf")).toBe(1);
expect(mgr.queueLength("drop-wf")).toBe(0);
@@ -254,7 +263,7 @@ describe("WorkflowManager", () => {
});
describe("completing a thread dequeues the next one", () => {
it("dispatches the next queued thread when the active thread sends completed", () => {
it("dispatches the next queued thread when the active thread sends completed", async () => {
const logStore = makeLogStore();
const config = makeConfig({
"queue-wf": { concurrency: 1, overflow: "queue", maxQueue: 5 },
@@ -263,6 +272,7 @@ describe("WorkflowManager", () => {
mgr.startWorkflow("queue-wf", { prompt: "first", maxRounds: 10, dryRun: false });
mgr.startWorkflow("queue-wf", { prompt: "second", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
expect(mgr.activeCount("queue-wf")).toBe(1);
expect(mgr.queueLength("queue-wf")).toBe(1);
@@ -289,7 +299,7 @@ describe("WorkflowManager", () => {
);
});
it("dispatches next queued thread when active thread sends failed", () => {
it("dispatches next queued thread when active thread sends failed", async () => {
const logStore = makeLogStore();
const config = makeConfig({
"queue-wf": { concurrency: 1, overflow: "queue", maxQueue: 5 },
@@ -298,6 +308,7 @@ describe("WorkflowManager", () => {
mgr.startWorkflow("queue-wf", { prompt: "first", maxRounds: 10, dryRun: false });
mgr.startWorkflow("queue-wf", { prompt: "second", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
const child = mockChildren[0];
const firstRunId = (child.send as ReturnType<typeof vi.fn>).mock.calls[0][0].runId as string;
@@ -325,6 +336,7 @@ describe("WorkflowManager", () => {
mgr.startWorkflow("wf-a", { prompt: "test", maxRounds: 10, dryRun: false });
mgr.startWorkflow("wf-b", { prompt: "test", maxRounds: 10, dryRun: false });
await vi.runAllTimersAsync();
// Two distinct workers should have been forked
expect(mockChildren).toHaveLength(2);
+2 -2
View File
@@ -1,9 +1,9 @@
import type { AgentConfig, AgentFn, WorkflowContext } from "@uncaged/nerve-core";
import type { AgentConfig, AgentFn, ThreadContext } from "@uncaged/nerve-core";
/**
* Echo adapter (`type: "echo"`) returns the assembled prompt unchanged.
* Used for tests and dry-run wiring before real adapters exist.
*/
export function createEchoAgent(_config: AgentConfig): AgentFn {
return async (prompt: string, _context: WorkflowContext) => prompt;
return async (_ctx: ThreadContext, prompt: string) => prompt;
}
@@ -1,59 +0,0 @@
import type {
Role,
RoleMeta,
RoleSpec,
Schema,
StartStep,
WorkflowContext,
WorkflowDefinition,
WorkflowMessage,
WorkflowSpec,
} from "@uncaged/nerve-core";
export type CompileWorkflowSpecDeps = {
/**
* Typed extraction for agent raw output (global/role merge applied before compile).
*/
extractFn: <T>(raw: string, schema: Schema<T>) => Promise<T>;
/** Builds thread context for each role invocation (workdir, cancellation, etc.). */
createContext: (start: StartStep, messages: WorkflowMessage[]) => WorkflowContext;
};
function compileRoleForSpec<Meta extends Record<string, unknown>>(
roleSpec: RoleSpec<Meta>,
deps: CompileWorkflowSpecDeps,
): Role<Meta> {
return async (start: StartStep, messages: WorkflowMessage[]) => {
const ctx = deps.createContext(start, messages);
const promptText =
typeof roleSpec.prompt === "string"
? roleSpec.prompt
: await roleSpec.prompt(start, messages);
const raw = await roleSpec.adapter(promptText, ctx);
const meta = await deps.extractFn(raw, roleSpec.meta);
return { content: raw, meta };
};
}
/**
* Turns RFC-003 `WorkflowSpec` into engine `WorkflowDefinition`: wires adapters and extract per role.
*/
export function compileWorkflowSpec<M extends RoleMeta>(
spec: WorkflowSpec<M>,
deps: CompileWorkflowSpecDeps,
): WorkflowDefinition<M> {
const roleKeys = Object.keys(spec.roles) as Array<keyof M & string>;
const roles = {} as WorkflowDefinition<M>["roles"];
for (const key of roleKeys) {
roles[key] = compileRoleForSpec(spec.roles[key], deps);
}
return {
name: spec.name,
roles,
moderator: spec.moderator,
};
}
+2 -5
View File
@@ -17,13 +17,12 @@ export type {
export type { SignalBus, SignalHandler, Unsubscribe } from "./signal-bus.js";
export type { ComputeFn, DrizzleDB, PeerMap, SenseRuntime } from "./sense-runtime.js";
export type { DrizzleDB, SenseRuntime } from "./sense-runtime.js";
export {
runMigrations,
openSenseDb,
openPeerDb,
loadComputeFn,
loadSenseModule,
executeCompute,
} from "./sense-runtime.js";
@@ -58,6 +57,4 @@ export type {
export { createWorkflowManager } from "./workflow-manager.js";
export type { WorkflowManager } from "./workflow-manager.js";
export { compileWorkflowSpec } from "./compile-workflow-spec.js";
export type { CompileWorkflowSpecDeps } from "./compile-workflow-spec.js";
export { createEchoAgent } from "./agent-adapters/echo.js";
+42 -2
View File
@@ -3,7 +3,7 @@
* Protocol per RFC §5.2: hub-and-spoke, all messages through engine.
*/
import type { Result } from "@uncaged/nerve-core";
import type { Result, WorkflowTrigger } from "@uncaged/nerve-core";
import { err, isPlainRecord, ok } from "@uncaged/nerve-core";
/** Parent → Worker: trigger one compute cycle for a sense */
@@ -72,6 +72,13 @@ export type SignalMessage = {
payload: unknown;
};
/** Worker → Parent: sense compute result includes a workflow to start */
export type SenseWorkflowTriggerMessage = {
type: "sense-workflow-trigger";
sense: string;
workflow: WorkflowTrigger;
};
/** Worker → Parent: compute threw or returned an unexpected error */
export type ErrorMessage = {
type: "error";
@@ -139,7 +146,8 @@ export type WorkerToParentMessage =
| HealthResponseMessage
| ThreadEventMessage
| WorkflowErrorMessage
| ThreadWorkflowMessageMessage;
| ThreadWorkflowMessageMessage
| SenseWorkflowTriggerMessage;
const PARENT_MSG_TYPES = new Set([
"compute",
@@ -340,6 +348,7 @@ const WORKER_MSG_TYPES = new Set([
"thread-event",
"workflow-error",
"thread-workflow-message",
"sense-workflow-trigger",
]);
function parseThreadWorkflowMessageMsg(
@@ -377,6 +386,36 @@ function parseThreadWorkflowMessageMsg(
});
}
function parseSenseWorkflowTriggerMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (typeof obj.sense !== "string") {
return err(new Error("Worker 'sense-workflow-trigger' message missing string 'sense' field"));
}
if (!isPlainRecord(obj.workflow)) {
return err(
new Error("Worker 'sense-workflow-trigger' message missing object 'workflow' field"),
);
}
const wf = obj.workflow;
if (typeof wf.name !== "string")
return err(new Error("Worker 'sense-workflow-trigger' workflow missing string 'name'"));
if (typeof wf.maxRounds !== "number")
return err(new Error("Worker 'sense-workflow-trigger' workflow missing number 'maxRounds'"));
if (typeof wf.prompt !== "string")
return err(new Error("Worker 'sense-workflow-trigger' workflow missing string 'prompt'"));
if (typeof wf.dryRun !== "boolean")
return err(new Error("Worker 'sense-workflow-trigger' workflow missing boolean 'dryRun'"));
return ok({
type: "sense-workflow-trigger",
sense: obj.sense,
workflow: {
name: wf.name,
maxRounds: wf.maxRounds,
prompt: wf.prompt,
dryRun: wf.dryRun,
},
});
}
/** Validate and parse an unknown IPC message received from a worker process. */
export function parseWorkerMessage(raw: unknown): Result<WorkerToParentMessage> {
if (!isPlainRecord(raw)) {
@@ -395,5 +434,6 @@ export function parseWorkerMessage(raw: unknown): Result<WorkerToParentMessage>
if (obj.type === "thread-event") return parseThreadEventMsg(obj);
if (obj.type === "workflow-error") return parseWorkflowErrorMsg(obj);
if (obj.type === "thread-workflow-message") return parseThreadWorkflowMessageMsg(obj);
if (obj.type === "sense-workflow-trigger") return parseSenseWorkflowTriggerMsg(obj);
return ok({ type: "ready" });
}
+36 -59
View File
@@ -4,43 +4,20 @@ import { DatabaseSync } from "node:sqlite";
import { drizzle } from "drizzle-orm/node-sqlite";
import type { NodeSQLiteDatabase } from "drizzle-orm/node-sqlite";
import type { SQLiteTable } from "drizzle-orm/sqlite-core";
import type { ComputeResult, Result } from "@uncaged/nerve-core";
import type { ComputeResult, Result, SenseComputeFn } from "@uncaged/nerve-core";
import { DEFAULT_SENSE_SIGNAL_RETENTION, err, isPlainRecord, ok } from "@uncaged/nerve-core";
import type { BlobStore } from "@uncaged/nerve-store";
/** A Drizzle DB instance (schema-generic) */
export type DrizzleDB = NodeSQLiteDatabase<Record<string, never>>;
/** Read-only map of peer sense name → their Drizzle DB */
export type PeerMap = Readonly<Record<string, DrizzleDB>>;
/** Options passed to a compute function */
export type ComputeOptions = {
signal: AbortSignal;
/** CAS under `data/blobs/`; injected by the sense worker when available. */
blobs?: BlobStore;
};
/**
* The shape every sense's index.ts must export.
* Engine injects `db` (read-write), `peers` (read-only), and `options`
* (`signal`, and `blobs` when running in the sense worker RFC-001 §8 CAS).
* Returns a structured result when a signal should be emitted (and optionally a workflow),
* or null for silence.
*/
export type ComputeFn<T = unknown> = (
db: DrizzleDB,
peers: PeerMap,
options?: ComputeOptions,
) => Promise<ComputeResult<T>>;
/** All state held for one sense inside a worker */
export type SenseRuntime = {
name: string;
db: DrizzleDB;
compute: ComputeFn;
compute: SenseComputeFn;
table: SQLiteTable;
persistSignal: (payload: unknown) => void;
};
@@ -126,13 +103,13 @@ export function runMigrations(sqlite: DatabaseSync, migrationsDir: string): Resu
return ok(undefined);
}
/** Run `_signals` row prune after this many inserts (amortize DELETE cost). */
const SIGNAL_INSERTS_PER_PRUNE = 100;
/**
* Open (or create) the SQLite file at `dbPath`, run all migrations in
* `migrationsDir`, and wrap with Drizzle ORM.
*/
/** Run `_signals` row prune after this many inserts (amortize DELETE cost). */
const SIGNAL_INSERTS_PER_PRUNE = 100;
export function openSenseDb(
dbPath: string,
migrationsDir: string,
@@ -184,27 +161,12 @@ export function openSenseDb(
}
/**
* Open a peer sense DB in read-only mode (no migrations).
* Dynamically import the compute function and table from a sense's index.ts/js.
* The module must export a named `compute` function and a named `table` (SQLiteTable).
*/
export function openPeerDb(dbPath: string): Result<DrizzleDB> {
let sqlite: DatabaseSync;
try {
sqlite = new DatabaseSync(dbPath, { readOnly: true });
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
return err(new Error(`Failed to open peer database "${dbPath}" (readonly): ${msg}`));
}
// Same schema-agnostic Drizzle wrapper as openSenseDb.
return ok(drizzle({ client: sqlite }) as DrizzleDB);
}
/**
* Dynamically import the compute function from a sense's index.ts/js.
* The module must export a named `compute` function.
*/
export async function loadComputeFn(senseIndexPath: string): Promise<Result<ComputeFn>> {
export async function loadSenseModule(
senseIndexPath: string,
): Promise<Result<{ compute: SenseComputeFn; table: SQLiteTable }>> {
let mod: unknown;
try {
@@ -221,26 +183,34 @@ export async function loadComputeFn(senseIndexPath: string): Promise<Result<Comp
);
}
return ok(mod.compute as ComputeFn);
if (!("table" in mod) || mod.table === null || typeof mod.table !== "object") {
return err(
new Error(
`Sense module "${senseIndexPath}" must export a named "table" (drizzle SQLiteTable)`,
),
);
}
return ok({
compute: mod.compute as SenseComputeFn,
table: mod.table as SQLiteTable,
});
}
/**
* Execute a sense's compute function with an optional soft timeout.
* If timeoutMs is provided and compute takes longer, the AbortSignal is
* triggered and an error Result is returned.
* When `blobStore` is set, it is exposed as `options.blobs` (see RFC-001 §8).
* When compute returns non-null, `result.signal` is persisted to the sense's
* table via `db.insert(table).values(result.signal)` and `persistSignal` is
* called with `result.signal`. Returns the full `ComputeResult` so callers
* can inspect the `workflow` field.
*/
export async function executeCompute(
runtime: SenseRuntime,
peers: PeerMap,
timeoutMs?: number,
blobStore?: BlobStore,
): Promise<Result<ComputeResult<unknown>>> {
const controller = new AbortController();
const options: ComputeOptions =
blobStore !== undefined
? { signal: controller.signal, blobs: blobStore }
: { signal: controller.signal };
let timer: ReturnType<typeof setTimeout> | undefined;
const timeoutPromise =
@@ -254,10 +224,17 @@ export async function executeCompute(
: null;
try {
const computePromise = runtime.compute(runtime.db, peers, options);
const computePromise = runtime.compute();
const result = timeoutPromise
? await Promise.race([computePromise, timeoutPromise])
: await computePromise;
if (result !== null) {
// Cast required: DrizzleDB is schema-agnostic; the sense module guarantees shape compatibility.
await runtime.db.insert(runtime.table).values(result.signal as Record<string, unknown>);
runtime.persistSignal(result.signal);
}
return ok(result);
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
+22 -77
View File
@@ -3,14 +3,13 @@
*
* Entry point for `nerve worker sense --group <name>`.
* Receives the group name via CLI args, reads nerve.yaml, initialises one
* SenseRuntime per sense in the group, builds peer read-only connections,
* then signals ready and enters the IPC event loop.
* SenseRuntime per sense in the group, then signals ready and enters the
* IPC event loop.
*
* Layout assumptions (nerve user config at `~/.uncaged-nerve/`):
* senses/<name>/index.js compiled compute
* dist/senses/<name>/index.js bundled compute (esbuild)
* senses/<name>/migrations/ SQL migration files
* data/senses/<name>.db SQLite data file
* data/blobs/<aa>/<hashrest> CAS (sha256), via options.blobs in compute
* nerve.yaml config
*/
@@ -19,14 +18,13 @@ import "./experimental-warning-suppression.js";
import { readFileSync } from "node:fs";
import { join, resolve } from "node:path";
import { parseNerveConfig, routeSenseComputeOutput } from "@uncaged/nerve-core";
import { parseNerveConfig } from "@uncaged/nerve-core";
import type { NerveConfig } from "@uncaged/nerve-core";
import { createBlobStore } from "@uncaged/nerve-store";
import type { WorkerToParentMessage } from "./ipc.js";
import { parseParentMessage } from "./ipc.js";
import { executeCompute, loadComputeFn, openPeerDb, openSenseDb } from "./sense-runtime.js";
import type { DrizzleDB, PeerMap, SenseRuntime } from "./sense-runtime.js";
import { executeCompute, loadSenseModule, openSenseDb } from "./sense-runtime.js";
import type { SenseRuntime } from "./sense-runtime.js";
import { ignoreSessionBroadcastSignals } from "./worker-fork-support.js";
// ---------------------------------------------------------------------------
@@ -52,7 +50,7 @@ function sendError(sense: string, error: string): void {
}
// ---------------------------------------------------------------------------
// Initialisation helpers (each extracted to keep bootstrap complexity low)
// Initialisation helpers
// ---------------------------------------------------------------------------
function readConfig(nerveRoot: string): NerveConfig {
@@ -78,65 +76,30 @@ async function initSense(
nerveRoot: string,
senseName: string,
retention: number,
): Promise<{ db: DrizzleDB; runtime: SenseRuntime }> {
): Promise<SenseRuntime> {
const dbPath = join(nerveRoot, "data", "senses", `${senseName}.db`);
const migrationsDir = join(nerveRoot, "senses", senseName, "migrations");
const senseIndexPath = resolve(join(nerveRoot, "senses", senseName, "index.js"));
const senseIndexPath = resolve(join(nerveRoot, "dist", "senses", senseName, "index.js"));
const dbResult = openSenseDb(dbPath, migrationsDir, retention);
if (!dbResult.ok) {
throw new Error(`Failed to init DB for "${senseName}": ${dbResult.error.message}`);
}
const computeResult = await loadComputeFn(senseIndexPath);
if (!computeResult.ok) {
throw new Error(`Failed to load compute for "${senseName}": ${computeResult.error.message}`);
const moduleResult = await loadSenseModule(senseIndexPath);
if (!moduleResult.ok) {
throw new Error(`Failed to load module for "${senseName}": ${moduleResult.error.message}`);
}
const { db } = dbResult.value;
return {
db,
runtime: {
name: senseName,
db,
compute: computeResult.value,
persistSignal: dbResult.value.persistSignal,
},
name: senseName,
db: dbResult.value.db,
compute: moduleResult.value.compute,
table: moduleResult.value.table,
persistSignal: dbResult.value.persistSignal,
};
}
function buildPeers(
nerveRoot: string,
allSenseNames: string[],
ownDbs: Map<string, DrizzleDB>,
groupSenseNames: Set<string>,
): PeerMap {
const entries: [string, DrizzleDB][] = [];
for (const peerName of allSenseNames) {
// Exclude senses that belong to this worker's own group — they are not peers
if (groupSenseNames.has(peerName)) continue;
const own = ownDbs.get(peerName);
if (own !== undefined) {
entries.push([peerName, own]);
continue;
}
const peerDbPath = join(nerveRoot, "data", "senses", `${peerName}.db`);
const peerResult = openPeerDb(peerDbPath);
if (!peerResult.ok) {
process.stderr.write(
`[sense-worker] Warning: could not open peer DB for "${peerName}": ${peerResult.error.message}\n`,
);
continue;
}
entries.push([peerName, peerResult.value]);
}
return Object.fromEntries(entries);
}
// ---------------------------------------------------------------------------
// Grace period: hard kill after soft timeout
//
@@ -173,13 +136,11 @@ function clearGracePeriodTimer(senseName: string): void {
async function runCompute(
senseName: string,
runtime: SenseRuntime,
peers: PeerMap,
timeoutMs: number,
gracePeriodMs: number | null,
blobStore: ReturnType<typeof createBlobStore>,
): Promise<void> {
try {
const result = await executeCompute(runtime, peers, timeoutMs, blobStore);
const result = await executeCompute(runtime, timeoutMs);
if (!result.ok) {
sendError(senseName, result.error.message);
if (gracePeriodMs !== null && result.error.message.includes("timed out")) {
@@ -189,12 +150,7 @@ async function runCompute(
}
clearGracePeriodTimer(senseName);
if (result.value != null) {
const routeResult = routeSenseComputeOutput(result.value);
if (!routeResult.ok) {
sendError(senseName, routeResult.error.message);
return;
}
runtime.persistSignal(routeResult.value.signal);
// Single IPC message: kernel uses routeSenseComputeOutput(payload) for signal + optional workflow.
sendSignal(senseName, result.value);
}
} catch (e: unknown) {
@@ -210,11 +166,9 @@ async function runCompute(
function handleMessage(
raw: unknown,
runtimes: Map<string, SenseRuntime>,
peers: PeerMap,
group: string,
senseConfigs: Map<string, { timeout: number | null; gracePeriod: number | null }>,
inFlight: Map<string, Promise<void>>,
blobStore: ReturnType<typeof createBlobStore>,
): void {
const parseResult = parseParentMessage(raw);
if (!parseResult.ok) {
@@ -245,14 +199,13 @@ function handleMessage(
return;
}
// Look up timeout/gracePeriod per-sense at compute time (RFC §5.3: these are per-sense)
const sc = senseConfigs.get(msg.sense);
const timeoutMs = sc?.timeout ?? DEFAULT_TIMEOUT_MS;
const gracePeriodMs = sc?.gracePeriod ?? null;
const previous = inFlight.get(msg.sense) ?? Promise.resolve();
const next = previous
.then(() => runCompute(msg.sense, runtime, peers, timeoutMs, gracePeriodMs, blobStore))
.then(() => runCompute(msg.sense, runtime, timeoutMs, gracePeriodMs))
.catch((e: unknown) => {
const errMsg = e instanceof Error ? e.message : String(e);
sendError(msg.sense, errMsg);
@@ -279,14 +232,12 @@ async function bootstrap(nerveRoot: string, group: string): Promise<void> {
}
const runtimes = new Map<string, SenseRuntime>();
const ownDbs = new Map<string, DrizzleDB>();
const failedSenses: string[] = [];
for (const senseName of groupSenses) {
try {
const retention = config.senses[senseName].retention;
const { db, runtime } = await initSense(nerveRoot, senseName, retention);
ownDbs.set(senseName, db);
const runtime = await initSense(nerveRoot, senseName, retention);
runtimes.set(senseName, runtime);
} catch (e: unknown) {
const eMsg = e instanceof Error ? e.message : String(e);
@@ -297,16 +248,11 @@ async function bootstrap(nerveRoot: string, group: string): Promise<void> {
}
}
// If ALL senses failed, exit with error so kernel respawns
if (runtimes.size === 0) {
process.stderr.write(`[sense-worker] All senses in group "${group}" failed to load, exiting\n`);
process.exit(1);
}
const groupSenseNames = new Set(groupSenses);
const peers = buildPeers(nerveRoot, Object.keys(config.senses), ownDbs, groupSenseNames);
// Build per-sense timeout/gracePeriod map (RFC §5.3: these are per-sense, not per-group)
const senseConfigs = new Map<string, { timeout: number | null; gracePeriod: number | null }>();
for (const senseName of groupSenses) {
const sc = config.senses[senseName];
@@ -317,12 +263,11 @@ async function bootstrap(nerveRoot: string, group: string): Promise<void> {
}
const inFlight = new Map<string, Promise<void>>();
const blobStore = createBlobStore(join(nerveRoot, "data", "blobs"));
sendReady();
process.on("message", (raw: unknown) => {
handleMessage(raw, runtimes, peers, group, senseConfigs, inFlight, blobStore);
handleMessage(raw, runtimes, group, senseConfigs, inFlight);
});
}
+74 -122
View File
@@ -1,19 +1,13 @@
/**
* Sense worker pool forked child processes per sense group (IPC lifecycle).
* Sense worker pool thin wrapper around WorkerRuntime (RFC-006): one fork per sense group.
*/
import { fork } from "node:child_process";
import type { ChildProcess } from "node:child_process";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { ComputeMessage, ShutdownMessage } from "./ipc.js";
import { parseWorkerMessage } from "./ipc.js";
import {
formatCapturedStderrTail,
formatChildExitSummary,
teeCapturedStderr,
} from "./worker-fork-support.js";
import type { ComputeMessage } from "./ipc.js";
import { formatCapturedStderrTail, formatChildExitSummary } from "./worker-fork-support.js";
import { createWorkerRuntime } from "./worker-runtime.js";
export function resolveWorkerScript(): string {
const __filename = fileURLToPath(import.meta.url);
@@ -21,17 +15,12 @@ export function resolveWorkerScript(): string {
return join(__dir, "sense-worker.js");
}
type WorkerEntry = {
group: string;
process: ChildProcess;
};
export type SenseWorkerPoolOptions = {
nerveRoot: string;
workerScript: string;
/** Invoked for every IPC message from a worker (including ready / signal / error). */
onWorkerMessage: (raw: unknown) => void;
/** Sense names in a group — used when clearing scheduler state on crash or restart. */
/** Sense names in a group — reserved for scheduler-aligned cleanup (kernel passes current config). */
sensesForGroup: (group: string) => string[];
/**
* Called when a worker exits with non-zero code before scheduling a respawn
@@ -58,144 +47,107 @@ export type SenseWorkerPool = {
activeGroupCount: () => number;
};
function spawnWorker(
nerveRoot: string,
group: string,
workerScript: string,
stderrTail: { value: string },
): ChildProcess {
const child = fork(workerScript, ["--group", group, "--root", nerveRoot], {
stdio: ["ignore", "inherit", "pipe", "ipc"],
});
teeCapturedStderr(child, stderrTail);
child.on("error", (err) => {
if ((err as NodeJS.ErrnoException).code !== "EPIPE") {
console.error("[worker] error:", err.message);
}
});
return child;
}
function sendComputeToProcess(worker: ChildProcess, senseName: string): void {
if (worker.connected === false) return;
const msg: ComputeMessage = { type: "compute", sense: senseName };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function sendShutdownToProcess(worker: ChildProcess): void {
if (worker.connected === false) return;
const msg: ShutdownMessage = { type: "shutdown" };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function waitForExit(child: ChildProcess, timeoutMs: number): Promise<void> {
return new Promise((resolve) => {
const timer = setTimeout(() => {
child.kill("SIGKILL");
resolve();
}, timeoutMs);
child.once("exit", () => {
clearTimeout(timer);
resolve();
});
});
}
/** Matches legacy pool: long crash window, 1s respawn delay, practical unlimited respawns. */
const SENSE_WORKER_RESPAWN = {
enabled: true,
maxCrashes: 100_000,
windowMs: 86_400_000,
delayMs: 1000,
} as const;
export function createSenseWorkerPool(options: SenseWorkerPoolOptions): SenseWorkerPool {
const workers = new Map<string, WorkerEntry>();
function startWorker(group: string): Promise<void> {
const stderrTail = { value: "" };
const child = spawnWorker(options.nerveRoot, group, options.workerScript, stderrTail);
let workerReadyResolve: (() => void) | undefined;
const workerReady = new Promise<void>((resolve) => {
workerReadyResolve = resolve;
});
child.on("message", (raw: unknown) => {
const result = parseWorkerMessage(raw);
if (result.ok && result.value.type === "ready") {
workerReadyResolve?.();
}
const runtime = createWorkerRuntime<string>({
script: options.workerScript,
argsForKey: (group) => ["--group", group, "--root", options.nerveRoot],
forwardStderr: true,
onMessage: (_key, raw) => {
options.onWorkerMessage(raw);
});
child.on("exit", (code, signal) => {
const summary = formatChildExitSummary(code, signal ?? null);
},
onReady: (_key, msg) => {
options.onWorkerMessage(msg);
},
onCrashLimitReached: null,
onExit: (group, code, signal) => {
const sig =
signal === null || signal === undefined || signal === ""
? null
: (signal as NodeJS.Signals);
const summary = formatChildExitSummary(code, sig);
process.stderr.write(
`[kernel] worker for group "${group}" exited (${summary})${formatCapturedStderrTail(stderrTail.value)}\n`,
`[kernel] worker for group "${group}" exited (${summary})${formatCapturedStderrTail(runtime.stderrTail(group))}\n`,
);
workerReadyResolve?.();
if (!options.isStopped() && code !== 0) {
process.stderr.write(`[kernel] respawning worker for group "${group}" in 1s\n`);
options.onWorkerCrashed(group);
setTimeout(() => {
if (!options.isStopped()) {
startWorker(group);
}
}, 1000);
}
});
},
respawn: {
...SENSE_WORKER_RESPAWN,
allowRespawn: (_group) => !options.isStopped(),
},
shutdownTimeoutMs: 5000,
});
workers.set(group, { group, process: child });
return workerReady;
/** Groups we have ever started — mirrors legacy Map presence for `restartGroup` no-op when unknown. */
const trackedGroups = new Set<string>();
/** Marks groups mid-evict so `hasWorkerForGroup` drops immediately (legacy synchronous eviction). */
const evicting = new Set<string>();
async function startWorker(group: string): Promise<void> {
trackedGroups.add(group);
await runtime.start(group);
}
async function restartGroup(group: string): Promise<void> {
const entry = workers.get(group);
if (entry === undefined) return;
options.onBeforeGroupRestart(group);
sendShutdownToProcess(entry.process);
await waitForExit(entry.process, 5000);
if (!options.isStopped()) {
await startWorker(group);
if (!trackedGroups.has(group)) {
return;
}
options.onBeforeGroupRestart(group);
await runtime.drain(group, null);
}
function evictGroup(group: string): void {
const entry = workers.get(group);
if (entry === undefined) return;
sendShutdownToProcess(entry.process);
workers.delete(group);
trackedGroups.delete(group);
evicting.add(group);
void runtime.evict(group, null).finally(() => {
evicting.delete(group);
});
}
async function shutdownAll(): Promise<void> {
const exitPromises: Promise<void>[] = [];
for (const entry of workers.values()) {
sendShutdownToProcess(entry.process);
exitPromises.push(waitForExit(entry.process, 5000));
}
await Promise.all(exitPromises);
await runtime.shutdown();
trackedGroups.clear();
evicting.clear();
}
function sendCompute(group: string, senseName: string): void {
const entry = workers.get(group);
if (entry === undefined) return;
sendComputeToProcess(entry.process, senseName);
if (!trackedGroups.has(group) || evicting.has(group)) {
return;
}
// Legacy pool: `child.send` no-op when IPC is closed (still allow cold start: child === null).
if (runtime.hasDisconnectedChild(group)) {
return;
}
const msg: ComputeMessage = { type: "compute", sense: senseName };
if (!runtime.trySendSync(group, msg)) {
void runtime.send(group, msg).catch(() => {
// IPC channel may close between scheduling and send — same as legacy try/catch on child.send
});
}
}
function getWorkerPid(group: string): number | null {
return workers.get(group)?.process.pid ?? null;
return runtime.pid(group);
}
/** True once `startWorker` has been called for the group and it is not mid-evict (matches legacy Map key). */
function hasWorkerForGroup(group: string): boolean {
return workers.has(group);
return trackedGroups.has(group) && !evicting.has(group);
}
/** Count of sense groups with a worker slot (includes not-yet-ready), excluding evicted keys. */
function activeGroupCount(): number {
return workers.size;
return trackedGroups.size;
}
return {
+422
View File
@@ -0,0 +1,422 @@
/**
* Generic message-routed worker process manager (RFC-006).
* One forked Node child per key; cold start, crash respawn, drain/evict, shutdown.
*/
import { type ChildProcess, type Serializable, fork } from "node:child_process";
import { isPlainRecord } from "@uncaged/nerve-core";
const STDERR_TAIL_MAX_CHARS = 2048;
export type WorkerDrainOpts = {
shutdownTimeoutMs: number | null;
};
export type WorkerRuntimeConfig<K extends string> = {
script: string;
argsForKey: (key: K) => string[];
/** When false, stderr is not captured into `stderrTail` (e.g. tests without a pipe). */
forwardStderr: boolean;
onMessage: (key: K, msg: unknown) => void;
onReady: (key: K, msg: unknown) => void;
onExit: (key: K, code: number | null, signal: string | null) => void;
/** Invoked when automatic respawn is skipped because `maxCrashes` was exceeded in `windowMs`. */
onCrashLimitReached: ((key: K) => void) | null;
respawn: {
enabled: boolean;
maxCrashes: number;
windowMs: number;
delayMs: number;
/** When non-null, return false to skip automatic respawn after an unexpected exit. */
allowRespawn: ((key: K) => boolean) | null;
};
shutdownTimeoutMs: number;
};
export type WorkerRuntime<K extends string> = {
send: (key: K, msg: unknown) => Promise<void>;
/** When the worker is already ready and IPC-connected, sends synchronously (returns true). Otherwise false — caller may fall back to `send`. */
trySendSync: (key: K, msg: unknown) => boolean;
start: (key: K) => Promise<void>;
evict: (key: K, opts: WorkerDrainOpts | null) => Promise<void>;
drain: (key: K, opts: WorkerDrainOpts | null) => Promise<void>;
shutdown: () => Promise<void>;
has: (key: K) => boolean;
/** True when a child exists but IPC is disconnected (legacy pool skipped sends in this case). */
hasDisconnectedChild: (key: K) => boolean;
pid: (key: K) => number | null;
keys: () => K[];
stderrTail: (key: K) => string;
};
type WorkerMachineState = "stopped" | "starting" | "ready" | "draining";
type ReadyWaiter = {
resolve: () => void;
reject: (err: Error) => void;
};
/** Internal: one forked process slot (ManagedWorker). */
type WorkerSlot<K extends string> = {
key: K;
state: WorkerMachineState;
child: ChildProcess | null;
pid: number | null;
stderrTail: string;
crashTimestamps: number[];
expectExit: boolean;
readyWaiters: ReadyWaiter[];
opChain: Promise<void>;
};
function isReadyIpcMessage(raw: unknown): boolean {
return isPlainRecord(raw) && raw.type === "ready";
}
function signalToString(signal: NodeJS.Signals | null): string | null {
if (signal === null) {
return null;
}
return String(signal);
}
function attachStderrTail<K extends string>(child: ChildProcess, slot: WorkerSlot<K>): void {
const stream = child.stderr;
if (stream == null) {
return;
}
stream.setEncoding("utf8");
stream.on("data", (chunk: string | Buffer) => {
const text = typeof chunk === "string" ? chunk : chunk.toString("utf8");
slot.stderrTail = (slot.stderrTail + text).slice(-STDERR_TAIL_MAX_CHARS);
});
}
function enqueueOp<K extends string>(slot: WorkerSlot<K>, fn: () => Promise<void>): Promise<void> {
const run = slot.opChain.then(fn, fn);
slot.opChain = run.then(
() => {},
() => {},
);
return run;
}
function resolveReadyWaiters<K extends string>(slot: WorkerSlot<K>): void {
const waiters = slot.readyWaiters;
slot.readyWaiters = [];
for (const w of waiters) {
w.resolve();
}
}
function rejectReadyWaiters<K extends string>(slot: WorkerSlot<K>, err: Error): void {
const waiters = slot.readyWaiters;
slot.readyWaiters = [];
for (const w of waiters) {
w.reject(err);
}
}
function waitForReady<K extends string>(
slot: WorkerSlot<K>,
shutdownTimeoutMs: number,
): Promise<void> {
if (slot.state === "ready" && slot.child !== null && slot.child.connected) {
return Promise.resolve();
}
return new Promise((resolve, reject) => {
let settled = false;
const timer = setTimeout(() => {
if (!settled) {
settled = true;
reject(new Error(`Worker "${String(slot.key)}" ready timeout`));
}
}, shutdownTimeoutMs);
slot.readyWaiters.push({
resolve: () => {
if (settled) {
return;
}
settled = true;
clearTimeout(timer);
resolve();
},
reject: (err: Error) => {
if (settled) {
return;
}
settled = true;
clearTimeout(timer);
reject(err);
},
});
});
}
async function waitForChildExit(child: ChildProcess, timeoutMs: number): Promise<void> {
await new Promise<void>((resolve) => {
const timer = setTimeout(() => {
child.kill("SIGKILL");
}, timeoutMs);
child.once("exit", () => {
clearTimeout(timer);
resolve();
});
});
}
export function createWorkerRuntime<K extends string>(
config: WorkerRuntimeConfig<K>,
): WorkerRuntime<K> {
const workers = new Map<K, WorkerSlot<K>>();
function getOrCreateSlot(key: K): WorkerSlot<K> {
let slot = workers.get(key);
if (slot === undefined) {
slot = {
key,
state: "stopped",
child: null,
pid: null,
stderrTail: "",
crashTimestamps: [],
expectExit: false,
readyWaiters: [],
opChain: Promise.resolve(),
};
workers.set(key, slot);
}
return slot;
}
function handleWorkerMessage(slot: WorkerSlot<K>, msg: unknown): void {
if (isReadyIpcMessage(msg)) {
if (slot.state === "starting") {
slot.state = "ready";
config.onReady(slot.key, msg);
resolveReadyWaiters(slot);
}
return;
}
config.onMessage(slot.key, msg);
}
function onChildExit(
slot: WorkerSlot<K>,
code: number | null,
signal: NodeJS.Signals | null,
): void {
config.onExit(slot.key, code, signalToString(signal));
if (slot.child !== null) {
slot.child.removeAllListeners("message");
slot.child.removeAllListeners("exit");
}
const wasExpect = slot.expectExit;
slot.expectExit = false;
slot.child = null;
slot.pid = null;
if (wasExpect) {
slot.state = "stopped";
return;
}
rejectReadyWaiters(slot, new Error(`Worker "${String(slot.key)}" exited unexpectedly`));
slot.state = "stopped";
void enqueueOp(slot, async () => {
await handleUnexpectedCrashRecovery(slot);
});
}
function registerChild(slot: WorkerSlot<K>, child: ChildProcess): void {
slot.child = child;
slot.pid = child.pid ?? null;
if (config.forwardStderr) {
attachStderrTail(child, slot);
}
child.on("message", (msg: unknown) => {
handleWorkerMessage(slot, msg);
});
child.on("exit", (code, sig) => {
onChildExit(slot, code, sig ?? null);
});
}
async function forkAndWaitReady(slot: WorkerSlot<K>): Promise<void> {
if (slot.state === "ready" && slot.child !== null && slot.child.connected) {
return;
}
slot.state = "starting";
let child: ChildProcess;
try {
child = fork(config.script, config.argsForKey(slot.key), {
stdio: ["ignore", "inherit", "pipe", "ipc"],
env: process.env,
});
} catch (e) {
slot.state = "stopped";
const err = e instanceof Error ? e : new Error(String(e));
rejectReadyWaiters(slot, err);
throw err;
}
registerChild(slot, child);
await waitForReady(slot, config.shutdownTimeoutMs);
}
function resolveShutdownTimeoutMs(opts: WorkerDrainOpts | null): number {
if (opts !== null && opts.shutdownTimeoutMs !== null) {
return opts.shutdownTimeoutMs;
}
return config.shutdownTimeoutMs;
}
async function gracefulStop(slot: WorkerSlot<K>, shutdownTimeoutMs: number): Promise<void> {
if (slot.child === null) {
return;
}
slot.expectExit = true;
slot.state = "draining";
const child = slot.child;
try {
child.send({ type: "shutdown" });
} catch {
// IPC channel may have closed between null-check and send
}
await waitForChildExit(child, shutdownTimeoutMs);
}
async function handleUnexpectedCrashRecovery(slot: WorkerSlot<K>): Promise<void> {
if (!config.respawn.enabled) {
return;
}
if (config.respawn.allowRespawn !== null && !config.respawn.allowRespawn(slot.key)) {
return;
}
const now = Date.now();
slot.crashTimestamps.push(now);
slot.crashTimestamps = slot.crashTimestamps.filter((t) => now - t <= config.respawn.windowMs);
if (slot.crashTimestamps.length >= config.respawn.maxCrashes) {
console.error(
`[WorkerRuntime] worker "${String(slot.key)}" exceeded crash limit (${String(config.respawn.maxCrashes)} in ${String(config.respawn.windowMs)}ms); not respawning`,
);
if (config.onCrashLimitReached !== null) {
config.onCrashLimitReached(slot.key);
}
return;
}
await new Promise<void>((resolve) => setTimeout(resolve, config.respawn.delayMs));
await forkAndWaitReady(slot);
}
async function shutdownWorker(slot: WorkerSlot<K>): Promise<void> {
await gracefulStop(slot, config.shutdownTimeoutMs);
workers.delete(slot.key);
}
function isActive(slot: WorkerSlot<K>): boolean {
return slot.state === "ready" && slot.child !== null && slot.child.connected;
}
return {
send: async (key: K, msg: unknown) => {
const slot = getOrCreateSlot(key);
await enqueueOp(slot, async () => {
await forkAndWaitReady(slot);
const child = slot.child;
if (child === null || !child.connected) {
throw new Error(`Worker "${String(key)}" is not connected`);
}
child.send(msg as Serializable);
});
},
trySendSync: (key: K, msg: unknown): boolean => {
const slot = workers.get(key);
if (slot === undefined || !isActive(slot)) {
return false;
}
const child = slot.child;
if (child === null || !child.connected) {
return false;
}
try {
child.send(msg as Serializable);
return true;
} catch {
return false;
}
},
start: async (key: K) => {
const slot = getOrCreateSlot(key);
await enqueueOp(slot, async () => {
await forkAndWaitReady(slot);
});
},
evict: async (key: K, opts: WorkerDrainOpts | null) => {
const slot = getOrCreateSlot(key);
const shutdownMs = resolveShutdownTimeoutMs(opts);
await enqueueOp(slot, async () => {
await gracefulStop(slot, shutdownMs);
workers.delete(key);
});
},
drain: async (key: K, opts: WorkerDrainOpts | null) => {
const slot = getOrCreateSlot(key);
const shutdownMs = resolveShutdownTimeoutMs(opts);
await enqueueOp(slot, async () => {
if (slot.child === null) {
await forkAndWaitReady(slot);
return;
}
await gracefulStop(slot, shutdownMs);
await forkAndWaitReady(slot);
});
},
shutdown: async () => {
const snapshot = [...workers.values()];
await Promise.all(snapshot.map((slot) => enqueueOp(slot, () => shutdownWorker(slot))));
},
has: (key: K) => {
const slot = workers.get(key);
return slot !== undefined && isActive(slot);
},
hasDisconnectedChild: (key: K): boolean => {
const slot = workers.get(key);
if (slot === undefined || slot.child === null) {
return false;
}
return !slot.child.connected;
},
pid: (key: K) => {
const slot = workers.get(key);
if (slot === undefined || !isActive(slot) || slot.pid === null) {
return null;
}
return slot.pid;
},
keys: () => [...workers.values()].filter((slot) => isActive(slot)).map((slot) => slot.key),
stderrTail: (key: K) => {
const slot = workers.get(key);
return slot === undefined ? "" : slot.stderrTail;
},
};
}
@@ -0,0 +1,256 @@
/**
* Pure helpers and IPC branching for workflow-manager (keeps workflow-manager.ts lean).
*/
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { WorkflowMessage } from "@uncaged/nerve-core";
import { START, isPlainRecord } from "@uncaged/nerve-core";
import type { LogStore, WorkflowRunStatus } from "@uncaged/nerve-store";
import type { ResumeThreadMessage, ThreadEventMessage } from "./ipc.js";
import type { WorkerToParentMessage } from "./ipc.js";
export type PendingThread = {
runId: string;
prompt: string;
maxRounds: number;
dryRun: boolean;
};
export type WorkflowState = {
active: Set<string>;
queue: PendingThread[];
};
/** Matches legacy manager: 6 crashes within 60s stops respawn (was `length > 5`). */
export const WORKFLOW_WORKER_RESPAWN = {
enabled: true,
maxCrashes: 6,
windowMs: 60_000,
delayMs: 0,
} as const;
/**
* Worker shutdown timeout must stay in sync with SHUTDOWN_TIMEOUT_MS in workflow-worker.ts.
* The drain timeout passed to drainAndRespawn must be >= this value so the worker has
* enough time to finish in-flight threads before the parent force-kills it.
*/
export const WORKER_SHUTDOWN_TIMEOUT_MS = 10_000;
export const DEFAULT_MAX_QUEUE = 100;
export function readLaunchFromTriggerPayload(
raw: unknown,
engineDefaultMaxRounds: number,
): { prompt: string; maxRounds: number; dryRun: boolean } {
if (isPlainRecord(raw)) {
const o = raw;
if (typeof o.prompt === "string" && typeof o.maxRounds === "number") {
const dryRun = typeof o.dryRun === "boolean" ? o.dryRun : false;
return { prompt: o.prompt, maxRounds: o.maxRounds, dryRun };
}
}
return { prompt: "", maxRounds: engineDefaultMaxRounds, dryRun: false };
}
export function ensureThreadMessagesWithStart(
messages: Array<{ role: string; content: string; meta: unknown; timestamp: number }>,
threadId: string,
fallbackPrompt: string,
fallbackMaxRounds: number,
): WorkflowMessage[] {
const mapped: WorkflowMessage[] = messages.map((m) => ({
role: m.role,
content: m.content,
meta: m.meta,
timestamp: m.timestamp,
}));
if (mapped.length > 0 && mapped[0].role === START) {
return mapped;
}
const start: WorkflowMessage = {
role: START,
content: fallbackPrompt,
meta: { maxRounds: fallbackMaxRounds, threadId },
timestamp: Date.now(),
};
return [start, ...mapped];
}
export function resolveWorkflowWorkerScript(): string {
const __filename = fileURLToPath(import.meta.url);
const __dir = dirname(__filename);
return join(__dir, "workflow-worker.js");
}
export function mapWorkflowRunStatus(eventType: string): WorkflowRunStatus | null {
const map: Record<string, WorkflowRunStatus> = {
started: "started",
queued: "queued",
completed: "completed",
failed: "failed",
crashed: "crashed",
dropped: "dropped",
interrupted: "interrupted",
killed: "killed",
};
return map[eventType] ?? null;
}
export function extractExitCodeFromPayload(payload: unknown): number | null {
if (isPlainRecord(payload) && typeof payload.exitCode === "number") {
return payload.exitCode;
}
return null;
}
export function appendWorkflowRunLog(
logStore: LogStore,
workflowName: string,
runId: string,
eventType: string,
payload: unknown | undefined,
exitCode: number | null,
): void {
const timestamp = Date.now();
const serialised = payload !== undefined ? JSON.stringify(payload) : null;
const status = mapWorkflowRunStatus(eventType);
if (status !== null) {
logStore.upsertWorkflowRun(
{
source: "workflow",
type: eventType,
refId: runId,
payload: serialised,
timestamp,
},
{ runId, workflow: workflowName, status, timestamp, exitCode },
);
} else {
logStore.append({
source: "workflow",
type: eventType,
refId: runId,
payload: serialised,
timestamp,
});
}
}
export function recoverQueuedRun(
workflowName: string,
runId: string,
state: WorkflowState,
logStore: LogStore,
engineMaxRounds: number,
): void {
if (state.queue.some((q) => q.runId === runId)) return;
const launch = readLaunchFromTriggerPayload(logStore.getTriggerPayload(runId), engineMaxRounds);
state.queue.push({
runId,
prompt: launch.prompt,
maxRounds: launch.maxRounds,
dryRun: launch.dryRun,
});
process.stderr.write(
`[workflow-manager] crash-recovery: re-queued thread "${runId}" for "${workflowName}"\n`,
);
}
export function recoverStartedRun(
workflowName: string,
runId: string,
state: WorkflowState,
logStore: LogStore,
engineMaxRounds: number,
sendResume: (wf: string, msg: ResumeThreadMessage) => void,
): void {
if (state.active.has(runId)) return;
const rawMessages = logStore.getThreadMessages(runId);
const launch = readLaunchFromTriggerPayload(logStore.getTriggerPayload(runId), engineMaxRounds);
const messages = ensureThreadMessagesWithStart(
rawMessages,
runId,
launch.prompt,
launch.maxRounds,
);
state.active.add(runId);
const msg: ResumeThreadMessage = {
type: "resume-thread",
runId,
messages,
maxRounds: launch.maxRounds,
dryRun: launch.dryRun,
};
sendResume(workflowName, msg);
process.stderr.write(
`[workflow-manager] crash-recovery: resuming thread "${runId}" for "${workflowName}" (${String(messages.length)} messages)\n`,
);
}
export function recoverThreadsFromStore(
workflowName: string,
logStore: LogStore,
engineMaxRounds: number,
getOrCreateState: (name: string) => WorkflowState,
sendResume: (wf: string, msg: ResumeThreadMessage) => void,
): void {
const activeRuns = logStore.getActiveWorkflowRuns(workflowName);
const state = getOrCreateState(workflowName);
for (const run of activeRuns) {
if (run.status === "queued") {
recoverQueuedRun(workflowName, run.runId, state, logStore, engineMaxRounds);
} else if (run.status === "started") {
recoverStartedRun(workflowName, run.runId, state, logStore, engineMaxRounds, sendResume);
}
}
}
export type WorkflowManagerMessageDeps = {
logStore: LogStore;
handleThreadEvent: (workflowName: string, msg: ThreadEventMessage) => void;
onWorkflowRoleError: (
workflowName: string,
runId: string,
error: string,
exitCode: number,
) => void;
};
export function dispatchWorkflowWorkerMessage(
workflowName: string,
msg: WorkerToParentMessage,
deps: WorkflowManagerMessageDeps,
): void {
if (msg.type === "thread-event") {
deps.handleThreadEvent(workflowName, msg);
return;
}
if (msg.type === "thread-workflow-message") {
deps.logStore.append({
source: "workflow",
type: "thread_workflow_message",
refId: msg.runId,
payload: JSON.stringify(msg.message),
timestamp: Date.now(),
});
return;
}
if (msg.type === "workflow-error") {
process.stderr.write(
`[workflow-manager] workflow-error for runId "${msg.runId}" in "${workflowName}": ${msg.error}\n`,
);
deps.onWorkflowRoleError(workflowName, msg.runId, msg.error, msg.exitCode);
return;
}
if (msg.type === "error") {
process.stderr.write(`[workflow-manager] error from "${workflowName}" worker: ${msg.error}\n`);
}
}
+175 -450
View File
@@ -6,33 +6,24 @@
* Concurrency and overflow (drop/queue) are enforced here in the parent process.
*/
import { fork } from "node:child_process";
import type { ChildProcess } from "node:child_process";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { NerveConfig, WorkflowConfig, WorkflowStatus } from "@uncaged/nerve-core";
import type {
NerveConfig,
WorkflowConfig,
WorkflowMessage,
WorkflowStatus,
} from "@uncaged/nerve-core";
import { START, isPlainRecord } from "@uncaged/nerve-core";
import type { LogStore, WorkflowRunStatus } from "@uncaged/nerve-store";
import type {
KillThreadMessage,
ResumeThreadMessage,
ShutdownMessage,
StartThreadMessage,
ThreadEventMessage,
} from "./ipc.js";
import type { LogStore } from "@uncaged/nerve-store";
import type { KillThreadMessage, StartThreadMessage, ThreadEventMessage } from "./ipc.js";
import { parseWorkerMessage } from "./ipc.js";
import { formatCapturedStderrTail, formatChildExitSummary } from "./worker-fork-support.js";
import { createWorkerRuntime } from "./worker-runtime.js";
import {
formatCapturedStderrTail,
formatChildExitSummary,
teeCapturedStderr,
} from "./worker-fork-support.js";
DEFAULT_MAX_QUEUE,
WORKER_SHUTDOWN_TIMEOUT_MS,
WORKFLOW_WORKER_RESPAWN,
type WorkflowState,
appendWorkflowRunLog,
dispatchWorkflowWorkerMessage,
extractExitCodeFromPayload,
recoverThreadsFromStore,
resolveWorkflowWorkerScript,
} from "./workflow-manager-support.js";
export type WorkflowLaunchParams = {
prompt: string;
@@ -60,7 +51,7 @@ export type WorkflowManager = {
updateConfig: (newConfig: NerveConfig) => void;
/**
* Drain active threads for a workflow, then respawn its worker process.
* Used for hot reload when bundled workflow output under workflows/<name>/dist/ changes.
* Used for hot reload when bundled workflow output under dist/workflows/<name>/ changes.
* Waits up to `drainTimeoutMs` for threads to complete before force-killing.
*/
drainAndRespawn: (workflowName: string, drainTimeoutMs?: number) => Promise<void>;
@@ -74,170 +65,109 @@ export type WorkflowManager = {
stop: () => Promise<void>;
};
type PendingThread = {
runId: string;
prompt: string;
maxRounds: number;
dryRun: boolean;
};
type WorkflowState = {
active: Set<string>;
queue: PendingThread[];
};
type WorkerEntry = {
workflowName: string;
process: ChildProcess;
stopping: boolean;
/** When set, the worker is draining before a hot-reload respawn. */
draining: boolean;
stderrTail: { value: string };
};
// Crash respawn backoff: track crash timestamps per workflow.
const MAX_CRASHES_IN_WINDOW = 5;
const CRASH_WINDOW_MS = 60_000;
/**
* Worker shutdown timeout must stay in sync with SHUTDOWN_TIMEOUT_MS in workflow-worker.ts.
* The drain timeout passed to drainAndRespawn must be >= this value so the worker has
* enough time to finish in-flight threads before the parent force-kills it.
*/
const WORKER_SHUTDOWN_TIMEOUT_MS = 10_000;
const DEFAULT_MAX_QUEUE = 100;
function readLaunchFromTriggerPayload(
raw: unknown,
engineDefaultMaxRounds: number,
): { prompt: string; maxRounds: number; dryRun: boolean } {
if (isPlainRecord(raw)) {
const o = raw;
if (typeof o.prompt === "string" && typeof o.maxRounds === "number") {
const dryRun = typeof o.dryRun === "boolean" ? o.dryRun : false;
return { prompt: o.prompt, maxRounds: o.maxRounds, dryRun };
}
}
return { prompt: "", maxRounds: engineDefaultMaxRounds, dryRun: false };
}
function ensureThreadMessagesWithStart(
messages: Array<{ role: string; content: string; meta: unknown; timestamp: number }>,
threadId: string,
fallbackPrompt: string,
fallbackMaxRounds: number,
fallbackDryRun: boolean,
): WorkflowMessage[] {
const mapped: WorkflowMessage[] = messages.map((m) => ({
role: m.role,
content: m.content,
meta: m.meta,
timestamp: m.timestamp,
}));
if (mapped.length > 0 && mapped[0].role === START) {
return mapped;
}
const start: WorkflowMessage = {
role: START,
content: fallbackPrompt,
meta: { maxRounds: fallbackMaxRounds, dryRun: fallbackDryRun, threadId },
timestamp: Date.now(),
};
return [start, ...mapped];
}
function resolveWorkerScript(): string {
const __filename = fileURLToPath(import.meta.url);
const __dir = dirname(__filename);
return join(__dir, "workflow-worker.js");
}
function spawnWorkflowWorker(
nerveRoot: string,
workflowName: string,
workerScript: string,
stderrTail: { value: string },
): ChildProcess {
const child = fork(workerScript, ["--workflow", workflowName, "--root", nerveRoot], {
stdio: ["ignore", "inherit", "pipe", "ipc"],
});
teeCapturedStderr(child, stderrTail);
// Prevent unhandled EPIPE when writing to a child whose IPC channel closed
child.on("error", (err) => {
if ((err as NodeJS.ErrnoException).code !== "EPIPE") {
console.error("[worker] error:", err.message);
}
});
return child;
}
function sendStartThread(worker: ChildProcess, msg: StartThreadMessage): void {
if (worker.connected === false) return;
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function sendShutdown(worker: ChildProcess, entry: WorkerEntry): void {
entry.stopping = true;
if (worker.connected === false) return;
const msg: ShutdownMessage = { type: "shutdown" };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function sendResumeThread(worker: ChildProcess, msg: ResumeThreadMessage): void {
if (worker.connected === false) return;
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function sendKillThread(worker: ChildProcess, runId: string): void {
if (worker.connected === false) return;
const msg: KillThreadMessage = { type: "kill-thread", runId };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function waitForExit(child: ChildProcess, timeoutMs: number): Promise<void> {
return new Promise((resolve) => {
const timer = setTimeout(() => {
child.kill("SIGKILL");
resolve();
}, timeoutMs);
child.once("exit", () => {
clearTimeout(timer);
resolve();
});
});
}
export function createWorkflowManager(
nerveRoot: string,
initialConfig: NerveConfig,
logStore: LogStore,
): WorkflowManager {
const workerScript = resolveWorkerScript();
const workerScript = resolveWorkflowWorkerScript();
/**
* Default drain timeout must be at least WORKER_SHUTDOWN_TIMEOUT_MS so the worker
* has enough time to finish in-flight threads before the parent force-kills it.
*/
const DEFAULT_DRAIN_TIMEOUT_MS = Math.max(30_000, WORKER_SHUTDOWN_TIMEOUT_MS + 5_000);
const states = new Map<string, WorkflowState>();
const workers = new Map<string, WorkerEntry>();
const crashTimestamps = new Map<string, number[]>();
const trackedWorkflows = new Set<string>();
const hotReloadEvicting = new Set<string>();
const crashRecoveryPending = new Set<string>();
const crashLimitBlocked = new Set<string>();
let stopped = false;
let config = initialConfig;
const pendingDrains = new Set<string>();
function logWorkflowEvent(
workflowName: string,
runId: string,
eventType: string,
payload: unknown | null = null,
exitCode: number | null = null,
): void {
appendWorkflowRunLog(logStore, workflowName, runId, eventType, payload, exitCode);
}
const runtime = createWorkerRuntime<string>({
script: workerScript,
argsForKey: (workflowName) => ["--workflow", workflowName, "--root", nerveRoot],
forwardStderr: true,
onMessage: (workflowName, raw) => {
handleWorkerMessage(workflowName, raw);
},
onReady: (workflowName, _msg) => {
if (crashRecoveryPending.has(workflowName)) {
crashRecoveryPending.delete(workflowName);
recoverThreadsFromStore(
workflowName,
logStore,
config.maxRounds,
getOrCreateState,
(wf, msg) => {
sendToWorker(wf, msg);
},
);
}
},
onExit: (workflowName, code, signalStr) => {
const sig =
signalStr === null || signalStr === undefined || signalStr === ""
? null
: (signalStr as NodeJS.Signals);
if (hotReloadEvicting.has(workflowName)) {
hotReloadEvicting.delete(workflowName);
markActiveRunsInterrupted(workflowName);
if (!stopped && workflowConfig(workflowName) !== null) {
process.stderr.write(
`[workflow-manager] worker for "${workflowName}" drained, respawning\n`,
);
}
return;
}
if (stopped) {
const state = states.get(workflowName);
if (state !== undefined) {
state.active.clear();
}
crashRecoveryPending.delete(workflowName);
return;
}
const summary = formatChildExitSummary(code, sig);
const stderrExtra = formatCapturedStderrTail(runtime.stderrTail(workflowName));
process.stderr.write(
`[workflow-manager] worker for "${workflowName}" exited (${summary})${stderrExtra}\n`,
);
cleanupAfterUnexpectedWorkerExit(workflowName);
crashRecoveryPending.add(workflowName);
},
onCrashLimitReached: (workflowName) => {
crashRecoveryPending.delete(workflowName);
trackedWorkflows.delete(workflowName);
crashLimitBlocked.add(workflowName);
process.stderr.write(
`[workflow-manager] worker for "${workflowName}" exceeded crash limit (${String(WORKFLOW_WORKER_RESPAWN.maxCrashes)} in ${String(WORKFLOW_WORKER_RESPAWN.windowMs)}ms) — stopping respawn\n`,
);
},
respawn: {
...WORKFLOW_WORKER_RESPAWN,
allowRespawn: (_wf) => !stopped,
},
shutdownTimeoutMs: DEFAULT_DRAIN_TIMEOUT_MS,
});
function getOrCreateState(workflowName: string): WorkflowState {
let state = states.get(workflowName);
if (state === undefined) {
@@ -251,60 +181,38 @@ export function createWorkflowManager(
return config.workflows[workflowName] ?? null;
}
function toWorkflowRunStatus(eventType: string): WorkflowRunStatus | null {
const map: Record<string, WorkflowRunStatus> = {
started: "started",
queued: "queued",
completed: "completed",
failed: "failed",
crashed: "crashed",
dropped: "dropped",
interrupted: "interrupted",
killed: "killed",
};
return map[eventType] ?? null;
}
function extractExitCode(payload: unknown): number | null {
if (isPlainRecord(payload) && typeof payload.exitCode === "number") {
return payload.exitCode;
/** IPC send — matches legacy pool: no-op when IPC is disconnected; cold-start via WorkerRuntime.send. */
function sendToWorker(workflowName: string, msg: unknown): void {
if (crashLimitBlocked.has(workflowName)) {
return;
}
return null;
}
function logWorkflowEvent(
workflowName: string,
runId: string,
eventType: string,
payload?: unknown,
exitCode: number | null = null,
): void {
const timestamp = Date.now();
const serialised = payload !== undefined ? JSON.stringify(payload) : null;
const status = toWorkflowRunStatus(eventType);
if (status !== null) {
logStore.upsertWorkflowRun(
{
source: "workflow",
type: eventType,
refId: runId,
payload: serialised,
timestamp,
},
{ runId, workflow: workflowName, status, timestamp, exitCode },
);
} else {
logStore.append({
source: "workflow",
type: eventType,
refId: runId,
payload: serialised,
timestamp,
trackedWorkflows.add(workflowName);
if (runtime.hasDisconnectedChild(workflowName)) {
return;
}
if (!runtime.trySendSync(workflowName, msg)) {
void runtime.send(workflowName, msg).catch(() => {
// IPC channel closed — mark any thread from this message as failed
if (isStartThreadMsg(msg)) {
const state = states.get(workflowName);
if (state?.active.has(msg.runId)) {
state.active.delete(msg.runId);
logWorkflowEvent(workflowName, msg.runId, "failed", { error: "IPC channel closed" }, 1);
dequeueNext(workflowName);
}
}
});
}
}
function isStartThreadMsg(msg: unknown): msg is StartThreadMessage {
return (
msg !== null &&
typeof msg === "object" &&
(msg as Record<string, unknown>).type === "start-thread"
);
}
function dispatchThread(
workflowName: string,
runId: string,
@@ -315,7 +223,6 @@ export function createWorkflowManager(
const state = getOrCreateState(workflowName);
state.active.add(runId);
const worker = getOrSpawnWorker(workflowName);
const msg: StartThreadMessage = {
type: "start-thread",
runId,
@@ -324,7 +231,7 @@ export function createWorkflowManager(
maxRounds,
dryRun,
};
sendStartThread(worker.process, msg);
sendToWorker(workflowName, msg);
logWorkflowEvent(workflowName, runId, "started", { prompt, maxRounds, dryRun });
}
@@ -368,93 +275,20 @@ export function createWorkflowManager(
if (msg.eventType === "completed" || msg.eventType === "failed" || msg.eventType === "killed") {
state.active.delete(msg.runId);
dequeueNext(workflowName);
const exitCode = extractExitCode(msg.payload);
const exitCode = extractExitCodeFromPayload(msg.payload);
logWorkflowEvent(workflowName, msg.runId, msg.eventType, msg.payload, exitCode);
maybeDeferredHotReloadDrain(workflowName);
}
}
function recoverQueuedRun(workflowName: string, runId: string, state: WorkflowState): void {
if (state.queue.some((q) => q.runId === runId)) return;
const launch = readLaunchFromTriggerPayload(
logStore.getTriggerPayload(runId),
config.maxRounds,
);
state.queue.push({
runId,
prompt: launch.prompt,
maxRounds: launch.maxRounds,
dryRun: launch.dryRun,
});
process.stderr.write(
`[workflow-manager] crash-recovery: re-queued thread "${runId}" for "${workflowName}"\n`,
);
}
function recoverStartedRun(
workflowName: string,
runId: string,
state: WorkflowState,
worker: WorkerEntry,
): void {
if (state.active.has(runId)) return;
const rawMessages = logStore.getThreadMessages(runId);
const launch = readLaunchFromTriggerPayload(
logStore.getTriggerPayload(runId),
config.maxRounds,
);
const messages = ensureThreadMessagesWithStart(
rawMessages,
runId,
launch.prompt,
launch.maxRounds,
launch.dryRun,
);
state.active.add(runId);
const msg: ResumeThreadMessage = {
type: "resume-thread",
runId,
messages,
maxRounds: launch.maxRounds,
dryRun: launch.dryRun,
};
sendResumeThread(worker.process, msg);
process.stderr.write(
`[workflow-manager] crash-recovery: resuming thread "${runId}" for "${workflowName}" (${messages.length} messages)\n`,
);
}
function recoverThreadsForWorker(workflowName: string, worker: WorkerEntry): void {
const activeRuns = logStore.getActiveWorkflowRuns(workflowName);
const state = getOrCreateState(workflowName);
for (const run of activeRuns) {
if (run.status === "queued") {
recoverQueuedRun(workflowName, run.runId, state);
} else if (run.status === "started") {
recoverStartedRun(workflowName, run.runId, state, worker);
}
}
}
function recordCrashAndCheckLimit(workflowName: string): boolean {
const now = Date.now();
const timestamps = (crashTimestamps.get(workflowName) ?? []).filter(
(t) => now - t < CRASH_WINDOW_MS,
);
timestamps.push(now);
crashTimestamps.set(workflowName, timestamps);
return timestamps.length > MAX_CRASHES_IN_WINDOW;
}
function handleWorkerCrash(workflowName: string): void {
function cleanupAfterUnexpectedWorkerExit(workflowName: string): void {
const state = states.get(workflowName);
if (state === undefined) return;
const crashedCount = state.active.size;
if (crashedCount > 0) {
process.stderr.write(
`[workflow-manager] worker for "${workflowName}" crashed with ${crashedCount} active thread(s)\n`,
`[workflow-manager] worker for "${workflowName}" crashed with ${String(crashedCount)} active thread(s)\n`,
);
for (const runId of state.active) {
logWorkflowEvent(workflowName, runId, "crashed", undefined, 255);
@@ -462,26 +296,13 @@ export function createWorkflowManager(
}
state.active.clear();
workers.delete(workflowName);
pendingDrains.delete(workflowName);
if (stopped || workflowConfig(workflowName) === null) return;
if (recordCrashAndCheckLimit(workflowName)) {
const count = crashTimestamps.get(workflowName)?.length ?? 0;
if (!stopped && !crashLimitBlocked.has(workflowName) && workflowConfig(workflowName) !== null) {
process.stderr.write(
`[workflow-manager] worker for "${workflowName}" crashed ${count} times in ${CRASH_WINDOW_MS}ms — stopping respawn\n`,
`[workflow-manager] respawning worker for "${workflowName}" after crash\n`,
);
return;
}
process.stderr.write(
`[workflow-manager] respawning worker for "${workflowName}" after crash\n`,
);
const newWorker = getOrSpawnWorker(workflowName);
setImmediate(() => {
recoverThreadsForWorker(workflowName, newWorker);
});
}
function handleWorkerMessage(workflowName: string, raw: unknown): void {
@@ -492,43 +313,19 @@ export function createWorkflowManager(
);
return;
}
const msg = result.value;
if (msg.type === "thread-event") {
handleThreadEvent(workflowName, msg);
return;
}
if (msg.type === "thread-workflow-message") {
logStore.append({
source: "workflow",
type: "thread_workflow_message",
refId: msg.runId,
payload: JSON.stringify(msg.message),
timestamp: Date.now(),
});
return;
}
if (msg.type === "workflow-error") {
process.stderr.write(
`[workflow-manager] workflow-error for runId "${msg.runId}" in "${workflowName}": ${msg.error}\n`,
);
const state = states.get(workflowName);
if (state !== undefined) {
state.active.delete(msg.runId);
dequeueNext(workflowName);
}
logWorkflowEvent(workflowName, msg.runId, "failed", { error: msg.error }, msg.exitCode);
maybeDeferredHotReloadDrain(workflowName);
return;
}
if (msg.type === "error") {
process.stderr.write(
`[workflow-manager] error from "${workflowName}" worker: ${msg.error}\n`,
);
}
dispatchWorkflowWorkerMessage(workflowName, result.value, {
logStore,
handleThreadEvent,
onWorkflowRoleError: (wf, runId, error, exitCode) => {
const state = states.get(wf);
if (state !== undefined) {
state.active.delete(runId);
dequeueNext(wf);
}
logWorkflowEvent(wf, runId, "failed", { error }, exitCode);
maybeDeferredHotReloadDrain(wf);
},
});
}
function markActiveRunsInterrupted(workflowName: string): void {
@@ -540,67 +337,6 @@ export function createWorkflowManager(
state.active.clear();
}
function handleWorkerExit(
workflowName: string,
code: number | null,
signal: NodeJS.Signals | null,
): void {
const entry = workers.get(workflowName);
if (entry?.draining) {
workers.delete(workflowName);
markActiveRunsInterrupted(workflowName);
if (!stopped && workflowConfig(workflowName) !== null) {
process.stderr.write(
`[workflow-manager] worker for "${workflowName}" drained, respawning\n`,
);
getOrSpawnWorker(workflowName);
}
return;
}
if (entry?.stopping) {
workers.delete(workflowName);
const state = states.get(workflowName);
if (state !== undefined) {
state.active.clear();
}
return;
}
const summary = formatChildExitSummary(code, signal);
const stderrExtra = entry !== undefined ? formatCapturedStderrTail(entry.stderrTail.value) : "";
process.stderr.write(
`[workflow-manager] worker for "${workflowName}" exited (${summary})${stderrExtra}\n`,
);
handleWorkerCrash(workflowName);
}
function getOrSpawnWorker(workflowName: string): WorkerEntry {
const existing = workers.get(workflowName);
if (existing !== undefined && existing.process.exitCode === null) {
return existing;
}
const stderrTail = { value: "" };
const child = spawnWorkflowWorker(nerveRoot, workflowName, workerScript, stderrTail);
child.on("message", (raw: unknown) => {
handleWorkerMessage(workflowName, raw);
});
child.on("exit", (code, signal) => {
handleWorkerExit(workflowName, code, signal ?? null);
});
const entry: WorkerEntry = {
workflowName,
process: child,
stopping: false,
draining: false,
stderrTail,
};
workers.set(workflowName, entry);
return entry;
}
function killThread(runId: string): boolean {
for (const [workflowName, state] of states) {
const queueIdx = state.queue.findIndex((q) => q.runId === runId);
@@ -611,10 +347,8 @@ export function createWorkflowManager(
}
if (state.active.has(runId)) {
const workerEntry = workers.get(workflowName);
if (workerEntry !== undefined) {
sendKillThread(workerEntry.process, runId);
}
const msg: KillThreadMessage = { type: "kill-thread", runId };
sendToWorker(workflowName, msg);
return true;
}
}
@@ -665,7 +399,7 @@ export function createWorkflowManager(
state.queue.push({ runId, prompt, maxRounds, dryRun });
logWorkflowEvent(workflowName, runId, "queued");
process.stderr.write(
`[workflow-manager] queued thread for "${workflowName}" runId "${runId}" (queue length: ${state.queue.length})\n`,
`[workflow-manager] queued thread for "${workflowName}" runId "${runId}" (queue length: ${String(state.queue.length)})\n`,
);
}
@@ -709,35 +443,29 @@ export function createWorkflowManager(
config = newConfig;
}
/**
* Default drain timeout must be at least WORKER_SHUTDOWN_TIMEOUT_MS so the worker
* has enough time to finish in-flight threads before the parent force-kills it.
*/
const DEFAULT_DRAIN_TIMEOUT_MS = Math.max(30_000, WORKER_SHUTDOWN_TIMEOUT_MS + 5_000);
async function drainAndRespawn(
workflowName: string,
drainTimeoutMs: number = DEFAULT_DRAIN_TIMEOUT_MS,
): Promise<void> {
const entry = workers.get(workflowName);
if (entry === undefined) {
// No active worker — nothing to drain
if (!trackedWorkflows.has(workflowName)) {
return;
}
entry.draining = true;
// Send shutdown without setting stopping=true (so the exit handler uses the draining branch)
if (entry.process.connected) {
const msg: ShutdownMessage = { type: "shutdown" };
try {
entry.process.send(msg);
} catch {
// IPC closed
const shutdownMs = Math.max(drainTimeoutMs, WORKER_SHUTDOWN_TIMEOUT_MS);
hotReloadEvicting.add(workflowName);
try {
await runtime.evict(workflowName, { shutdownTimeoutMs: shutdownMs });
trackedWorkflows.delete(workflowName);
if (!stopped && workflowConfig(workflowName) !== null) {
trackedWorkflows.add(workflowName);
await runtime.start(workflowName);
}
} finally {
hotReloadEvicting.delete(workflowName);
}
await waitForExit(entry.process, drainTimeoutMs);
// The exit handler (draining branch) will respawn the worker automatically
}
function drainWhenIdle(workflowName: string): void {
const state = states.get(workflowName);
const hasActiveRuns = state !== undefined && state.active.size > 0;
@@ -763,20 +491,17 @@ export function createWorkflowManager(
pendingDrains.add(workflowName);
process.stderr.write(
`[workflow-manager] deferring hot-reload for "${workflowName}" until ${state.active.size} active run(s) complete\n`,
`[workflow-manager] deferring hot-reload for "${workflowName}" until ${String(state.active.size)} active run(s) complete\n`,
);
}
async function stop(): Promise<void> {
stopped = true;
pendingDrains.clear();
const exitPromises: Promise<void>[] = [];
for (const entry of workers.values()) {
sendShutdown(entry.process, entry);
exitPromises.push(waitForExit(entry.process, 5000));
}
await Promise.all(exitPromises);
workers.clear();
hotReloadEvicting.clear();
crashRecoveryPending.clear();
await runtime.shutdown();
trackedWorkflows.clear();
}
return {
+45 -40
View File
@@ -14,7 +14,14 @@ import "./experimental-warning-suppression.js";
import { existsSync } from "node:fs";
import { join, resolve } from "node:path";
import type { RoleMeta, StartStep, WorkflowDefinition, WorkflowMessage } from "@uncaged/nerve-core";
import type {
RoleMeta,
RoleStep,
StartStep,
ThreadContext,
WorkflowDefinition,
WorkflowMessage,
} from "@uncaged/nerve-core";
import { END, START, isPlainRecord } from "@uncaged/nerve-core";
import type {
@@ -87,15 +94,14 @@ function normalizeStartMeta(
threadIdFallback: string,
): StartStep["meta"] {
if (!isPlainRecord(meta)) {
return { maxRounds: maxRoundsFallback, dryRun: false, threadId: threadIdFallback };
return { maxRounds: maxRoundsFallback, threadId: threadIdFallback };
}
const maxRounds = typeof meta.maxRounds === "number" ? meta.maxRounds : maxRoundsFallback;
const dryRun = typeof meta.dryRun === "boolean" ? meta.dryRun : false;
const threadId =
typeof meta.threadId === "string" && meta.threadId.length > 0
? meta.threadId
: threadIdFallback;
return { maxRounds, dryRun, threadId };
return { maxRounds, threadId };
}
function startStepFromWorkflowMessage(
@@ -107,7 +113,7 @@ function startStepFromWorkflowMessage(
return {
role: START,
content: "",
meta: { maxRounds: maxRoundsFallback, dryRun: false, threadId: threadIdFallback },
meta: { maxRounds: maxRoundsFallback, threadId: threadIdFallback },
timestamp: Date.now(),
};
}
@@ -124,8 +130,30 @@ type ThreadMessagesState = {
start: StartStep;
/** Role outputs only; never includes the `__start__` frame. */
messages: WorkflowMessage[];
/** From IPC (`start-thread` / `resume-thread`); not part of `StartStep.meta`. */
dryRun: boolean;
};
function workflowMessagesToRoleSteps(messages: WorkflowMessage[]): RoleStep<RoleMeta>[] {
return messages.map((m) => ({
role: m.role,
meta: m.meta as Record<string, unknown>,
content: m.content,
timestamp: m.timestamp,
})) as RoleStep<RoleMeta>[];
}
function buildThreadContext(
start: StartStep,
roleMessages: WorkflowMessage[],
): ThreadContext<RoleMeta> {
return {
threadId: start.meta.threadId,
start,
steps: workflowMessagesToRoleSteps(roleMessages),
};
}
function initThreadMessages(
runId: string,
resumeMessages: WorkflowMessage[],
@@ -139,6 +167,7 @@ function initThreadMessages(
return {
start: startStepFromWorkflowMessage(first, maxRounds, runId),
messages: [...rest],
dryRun,
};
}
const prompt = freshPrompt ?? "";
@@ -146,17 +175,18 @@ function initThreadMessages(
start: {
role: START,
content: prompt,
meta: { maxRounds, dryRun, threadId: runId },
meta: { maxRounds, threadId: runId },
timestamp: Date.now(),
},
messages: [...resumeMessages],
dryRun,
};
}
const prompt = freshPrompt ?? "";
const start: StartStep = {
role: START,
content: prompt,
meta: { maxRounds, dryRun, threadId: runId },
meta: { maxRounds, threadId: runId },
timestamp: Date.now(),
};
sendWorkflowMessage(runId, {
@@ -165,14 +195,13 @@ function initThreadMessages(
meta: start.meta,
timestamp: start.timestamp,
});
return { start, messages: [] };
return { start, messages: [], dryRun };
}
async function executeRole(
def: WorkflowDefinition<RoleMeta>,
nextRole: string,
start: StartStep,
messages: WorkflowMessage[],
ctx: ThreadContext<RoleMeta>,
runId: string,
): Promise<{ content: string; meta: Record<string, unknown> } | null> {
const role = def.roles[nextRole];
@@ -183,7 +212,7 @@ async function executeRole(
let result: { content: string; meta: Record<string, unknown> };
try {
result = await role(start, messages);
result = await role(ctx);
} catch (e: unknown) {
const errMsg = e instanceof Error ? e.message : String(e);
sendThreadEvent(runId, "failed", { error: errMsg, exitCode: 1 });
@@ -213,37 +242,20 @@ async function runThread(
dryRun,
);
const steps: Array<{
role: string;
meta: Record<string, unknown>;
content: string;
timestamp: number;
}> = [];
// Rebuild steps from any resumed messages
for (const msg of roleMessages) {
steps.push({
role: msg.role,
meta: msg.meta as Record<string, unknown>,
content: msg.content,
timestamp: msg.timestamp,
});
}
if (killFlag.value) {
sendThreadEvent(runId, "killed", { exitCode: 137 });
return;
}
let nextRole = def.moderator({ start, steps });
let nextRole = def.moderator(buildThreadContext(start, roleMessages));
if (nextRole === END) {
sendThreadEvent(runId, "completed", { exitCode: 0 });
return;
}
while (steps.length < maxRounds) {
const result = await executeRole(def, nextRole, start, roleMessages, runId);
while (roleMessages.length < maxRounds) {
const result = await executeRole(def, nextRole, buildThreadContext(start, roleMessages), runId);
if (killFlag.value) {
sendThreadEvent(runId, "killed", { exitCode: 137 });
@@ -261,14 +273,7 @@ async function runThread(
roleMessages.push(message);
sendWorkflowMessage(runId, message);
steps.push({
role: nextRole,
meta: result.meta,
content: result.content,
timestamp: message.timestamp,
});
nextRole = def.moderator({ start, steps });
nextRole = def.moderator(buildThreadContext(start, roleMessages));
if (nextRole === END) {
sendThreadEvent(runId, "completed", { exitCode: 0 });
@@ -298,7 +303,7 @@ async function loadWorkflowDefinition(
nerveRoot: string,
workflowName: string,
): Promise<WorkflowDefinition<RoleMeta>> {
const indexPath = resolve(join(nerveRoot, "workflows", workflowName, "dist", "index.js"));
const indexPath = resolve(join(nerveRoot, "dist", "workflows", workflowName, "index.js"));
if (!existsSync(indexPath)) {
throw new Error(
`Workflow definition not found for "${workflowName}". Expected:\n ${indexPath}`,
+26
View File
@@ -0,0 +1,26 @@
{
"name": "@uncaged/nerve-role-committer",
"version": "0.5.0",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": ["dist"],
"publishConfig": {
"access": "public"
},
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "rslib build",
"test": "vitest run --passWithNoTests"
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*",
"@uncaged/nerve-workflow-utils": "workspace:*",
"zod": "^4.3.6"
},
"devDependencies": {
"@rslib/core": "^0.21.3",
"typescript": "^5.8.3",
"vitest": "^4.1.5"
}
}
+19
View File
@@ -0,0 +1,19 @@
import { defineConfig } from "@rslib/core";
export default defineConfig({
lib: [
{
format: "esm",
dts: true,
},
],
source: {
entry: {
index: "src/index.ts",
},
},
output: {
target: "node",
cleanDistPath: true,
},
});
+59
View File
@@ -0,0 +1,59 @@
import type { AgentFn, Role, ThreadContext } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { createRole, decorateRole, onFail, withDryRun } from "@uncaged/nerve-workflow-utils";
import { z } from "zod";
export const committerMetaSchema = z.object({
committed: z
.boolean()
.describe("true if branch created, changes committed, and pushed successfully"),
});
export type CommitterMeta = z.infer<typeof committerMetaSchema>;
function committerPrompt(threadId: string): string {
return `You are the committer agent. The coder finished with a passing build; your job is to branch, commit, and push.
1. Read the workflow thread: \`nerve thread show ${threadId}\` — understand what was planned, coded, and reviewed.
2. Run \`git status\`. If nothing to commit, set committed=false.
3. Create a feature branch: infer a good \`fix/<slug>\` or \`feat/<slug>\` name from the thread context.
4. \`git add -A\`
5. Write a conventional commit message based on the thread context.
6. \`git commit -m "<message>"\` — do NOT pass \`--author\`, use repo git config.
7. \`git push -u origin <branch>\`
**committed=true** only if branch was created, commit succeeded, and **push** succeeded.
End your reply with a JSON line:
\`\`\`json
{ "committed": true }
\`\`\`
or
\`\`\`json
{ "committed": false }
\`\`\``;
}
/**
* Creates a committer role that branches, commits, and pushes changes.
* The agent reads the workflow thread to infer branch name, scope, and commit message.
*/
export function createCommitterRole(
adapter: AgentFn,
extract: LlmExtractorConfig,
): Role<CommitterMeta> {
const inner = createRole(
adapter,
async (ctx: ThreadContext) => committerPrompt(ctx.threadId),
committerMetaSchema,
extract,
);
return decorateRole(inner, [
withDryRun({
label: "committer",
meta: { committed: true } as CommitterMeta,
dryRun: extract.dryRun === true,
}),
onFail({ label: "committer", meta: { committed: false } as CommitterMeta }),
]) as Role<CommitterMeta>;
}
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src",
"composite": false
},
"include": ["src"]
}
+26
View File
@@ -0,0 +1,26 @@
{
"name": "@uncaged/nerve-role-reviewer",
"version": "0.5.0",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": ["dist"],
"publishConfig": {
"access": "public"
},
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "rslib build",
"test": "vitest run --passWithNoTests"
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*",
"@uncaged/nerve-workflow-utils": "workspace:*",
"zod": "^4.3.6"
},
"devDependencies": {
"@rslib/core": "^0.21.3",
"typescript": "^5.8.3",
"vitest": "^4.1.5"
}
}
+19
View File
@@ -0,0 +1,19 @@
import { defineConfig } from "@rslib/core";
export default defineConfig({
lib: [
{
format: "esm",
dts: true,
},
],
source: {
entry: {
index: "src/index.ts",
},
},
output: {
target: "node",
cleanDistPath: true,
},
});
+2
View File
@@ -0,0 +1,2 @@
export { createReviewerRole, reviewerMetaSchema } from "./reviewer.js";
export type { ReviewerMeta, ReviewerConfig } from "./reviewer.js";
+91
View File
@@ -0,0 +1,91 @@
import type { AgentFn, Role, ThreadContext } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { createRole } from "@uncaged/nerve-workflow-utils";
import { z } from "zod";
export const reviewerMetaSchema = z.object({
approved: z.boolean().describe("true if the diff is clean and ready to merge"),
});
export type ReviewerMeta = z.infer<typeof reviewerMetaSchema>;
export type ReviewerConfig = {
/** Working directory of the project */
cwd: string;
/** Path to conventions/standards file, relative to cwd. Default: "CONVENTIONS.md" */
conventionsPath: string | null;
/** Extra checklist items appended to the review criteria */
extraChecks: ReadonlyArray<string>;
};
const defaults: ReviewerConfig = {
cwd: ".",
conventionsPath: "CONVENTIONS.md",
extraChecks: [],
};
function reviewerPrompt({
threadId,
config,
}: { threadId: string; config: ReviewerConfig }): string {
const { cwd, conventionsPath, extraChecks } = config;
const conventionsBlock = conventionsPath
? `Read project conventions: \`cat ${cwd}/${conventionsPath}\`\n`
: "";
const extraBlock =
extraChecks.length > 0
? `\n### 📋 Project-specific checks\n${extraChecks.map((c) => `- ${c}`).join("\n")}\n`
: "";
return `You are a **code reviewer**. You run after the coder and before the tester.
**IMPORTANT: The project is at \`${cwd}\`. Always \`cd ${cwd}\` first.**
Read the workflow thread for context: \`nerve thread ${threadId}\`
${conventionsBlock}
## Your job static analysis of the git diff
Run these commands and analyze the output:
1. **\`cd ${cwd} && git diff --stat\`** — see what files changed
2. **\`cd ${cwd} && git diff\`** — read the actual diff
3. **\`cd ${cwd} && git status --short\`** — check for untracked files
## Checklist
### 🔴 Reject (approved: false) tell coder exactly what to fix
- **Garbage files**: build artifacts, lockfiles, IDE config that shouldn't be committed
- **Secrets/credentials**: API keys, tokens, passwords hardcoded in the diff
- **Unrelated changes**: files modified outside the scope of the task
${conventionsPath ? `- **Convention violations**: patterns that contradict ${conventionsPath}\n` : ""}${extraBlock}
### Approve (approved: true) no comment needed
- Diff is clean, focused, follows project standards
End with:
\`\`\`json
{ "approved": true }
\`\`\`
or
\`\`\`json
{ "approved": false }
\`\`\``;
}
/**
* Creates a reviewer role that performs static analysis on git diffs.
* Checks for garbage files, secrets, unrelated changes, and project conventions.
*/
export function createReviewerRole(
adapter: AgentFn,
extract: LlmExtractorConfig,
config: Partial<ReviewerConfig> = {},
): Role<ReviewerMeta> {
const resolved: ReviewerConfig = { ...defaults, ...config };
return createRole(
adapter,
async (ctx: ThreadContext) => reviewerPrompt({ threadId: ctx.threadId, config: resolved }),
reviewerMetaSchema,
extract,
);
}
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src",
"composite": false
},
"include": ["src"]
}
+28
View File
@@ -0,0 +1,28 @@
{
"name": "@uncaged/nerve-workflow-meta",
"version": "0.5.0",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": ["dist"],
"publishConfig": {
"access": "public"
},
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "rslib build",
"test": "vitest run --passWithNoTests"
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*",
"@uncaged/nerve-role-committer": "workspace:*",
"@uncaged/nerve-role-reviewer": "workspace:*",
"@uncaged/nerve-workflow-utils": "workspace:*",
"zod": "^4.3.6"
},
"devDependencies": {
"@rslib/core": "^0.21.3",
"typescript": "^5.8.3",
"vitest": "^4.1.5"
}
}
+19
View File
@@ -0,0 +1,19 @@
import { defineConfig } from "@rslib/core";
export default defineConfig({
lib: [
{
format: "esm",
dts: true,
},
],
source: {
entry: {
index: "src/index.ts",
},
},
output: {
target: "node",
cleanDistPath: true,
},
});
@@ -0,0 +1,42 @@
import type { AgentFn, WorkflowDefinition } from "@uncaged/nerve-core";
import { createCommitterRole } from "@uncaged/nerve-role-committer";
import { createReviewerRole } from "@uncaged/nerve-role-reviewer";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { moderator } from "./moderator.js";
import type { SenseMeta } from "./moderator.js";
import { createCoderRole } from "./roles/coder.js";
import { createPlannerRole } from "./roles/planner.js";
import { createTesterRole } from "./roles/tester.js";
export type CreateDevelopSenseDeps = {
defaultAdapter: AgentFn;
adapters: Partial<Record<keyof SenseMeta, AgentFn>> | null;
extract: LlmExtractorConfig;
cwd: string;
};
export function createDevelopSenseWorkflow({
defaultAdapter,
adapters,
extract,
cwd,
}: CreateDevelopSenseDeps): WorkflowDefinition<SenseMeta> {
const a = (role: keyof SenseMeta) => adapters?.[role] ?? defaultAdapter;
const roles = {
planner: createPlannerRole(a("planner"), extract),
coder: createCoderRole(a("coder"), extract),
reviewer: createReviewerRole(a("reviewer"), extract, {
cwd,
conventionsPath: "CONVENTIONS.md",
}),
tester: createTesterRole(a("tester"), extract, cwd),
committer: createCommitterRole(a("committer"), extract),
};
return {
name: "develop-sense",
roles,
moderator,
};
}
@@ -0,0 +1,65 @@
import { END } from "@uncaged/nerve-core";
import type { Moderator } from "@uncaged/nerve-core";
import type { CommitterMeta } from "@uncaged/nerve-role-committer";
import type { ReviewerMeta } from "@uncaged/nerve-role-reviewer";
import type { CoderMeta } from "./roles/coder.js";
import type { PlannerMeta } from "./roles/planner.js";
import type { TesterMeta } from "./roles/tester.js";
export type SenseMeta = {
planner: PlannerMeta;
coder: CoderMeta;
reviewer: ReviewerMeta;
tester: TesterMeta;
committer: CommitterMeta;
};
const MAX_CODER_ROUNDS = 20;
const MAX_TOTAL_REJECTIONS = 10;
function coderRounds(steps: { role: string }[]): number {
return steps.filter((s) => s.role === "coder").length;
}
function totalRejections(steps: { role: string; meta: unknown }[]): number {
return steps.filter((s) => {
if (s.role === "reviewer") return !(s.meta as Record<string, boolean>).approved;
if (s.role === "tester") return !(s.meta as Record<string, boolean>).passed;
if (s.role === "committer") return !(s.meta as Record<string, boolean>).committed;
return false;
}).length;
}
function canRetryCoder(steps: { role: string; meta: unknown }[]): boolean {
return coderRounds(steps) < MAX_CODER_ROUNDS && totalRejections(steps) < MAX_TOTAL_REJECTIONS;
}
export const moderator: Moderator<SenseMeta> = (context) => {
if (context.steps.length === 0) return "planner";
const last = context.steps[context.steps.length - 1];
if (last.role === "planner") return "coder";
if (last.role === "coder") {
if (last.meta.filesCreated) return "reviewer";
return canRetryCoder(context.steps) ? "coder" : END;
}
if (last.role === "reviewer") {
if (last.meta.approved) return "tester";
return canRetryCoder(context.steps) ? "coder" : END;
}
if (last.role === "tester") {
if (last.meta.passed) return "committer";
return canRetryCoder(context.steps) ? "coder" : END;
}
if (last.role === "committer") {
if (last.meta.committed) return END;
return canRetryCoder(context.steps) ? "coder" : END;
}
return END;
};
@@ -0,0 +1,50 @@
import type { AgentFn, Role, ThreadContext } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { createRole } from "@uncaged/nerve-workflow-utils";
import { z } from "zod";
export const coderMetaSchema = z.object({
filesCreated: z.boolean().describe("true if the sense files were created"),
});
export type CoderMeta = z.infer<typeof coderMetaSchema>;
export function coderPrompt({ threadId }: { threadId: string }): string {
return `Read the workflow thread for the planner's sense design and any tester feedback: \`nerve thread ${threadId}\`
Read \`cat AGENT.md\` from the repository root, then \`CONVENTIONS.md\` and \`.knowledge/sense.md\` if present.
## Your task
Implement (or fix) the sense the planner designed. If there is tester feedback in the thread, fix the issues it identified.
## Multi-step approach
You do NOT need to finish everything in one pass. You may return \`done: false\` to continue in the next iteration.
## File structure for each sense (flat workspace)
The workspace has **one root** \`package.json\` and root \`scripts/build.mjs\` (or equivalent) that bundles all senses. There is **no** per-sense \`package.json\`. Bundled output is \`dist/senses/<name>/index.js\` after a root build.
- \`senses/<name>/src/index.ts\` — compute entry; import schema as \`./schema.ts\`
- \`senses/<name>/src/schema.ts\` — Drizzle schema (TypeScript)
- \`senses/<name>/migrations/\` — SQL migration files (at sense root, not inside \`src/\`)
Look at existing senses for patterns.
## When to return done: true
Return \`done: true\` ONLY when ALL of the following are true:
- All required files are created
- From the **workspace root**, \`pnpm run build\` or \`npm run build\` succeeds (run it!) and \`dist/senses/<name>/index.js\` exists
- \`nerve.yaml\` is updated with the sense config
Return \`done: false\` if you made progress but there is still work to do.`;
}
export function createCoderRole(adapter: AgentFn, extract: LlmExtractorConfig): Role<CoderMeta> {
return createRole(
adapter,
async (ctx: ThreadContext) => coderPrompt({ threadId: ctx.threadId }),
coderMetaSchema,
extract,
);
}
@@ -0,0 +1,39 @@
import type { AgentFn, Role, ThreadContext } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { createRole } from "@uncaged/nerve-workflow-utils";
import { z } from "zod";
export const plannerMetaSchema = z.object({
senseName: z.string().describe("kebab-case sense name from the plan"),
});
export type PlannerMeta = z.infer<typeof plannerMetaSchema>;
export function plannerPrompt({ threadId }: { threadId: string }): string {
return `You are planning a new Nerve sense.
Read the workflow thread for the user's request: \`nerve thread ${threadId}\`
Read the workspace guide: \`cat AGENT.md\` from the repository root (created by \`nerve init\`). Also read \`CONVENTIONS.md\` and \`.knowledge/sense.md\` if present. Optional skills live under \`.cursor/skills/\`.
Also look at existing senses in the \`senses/\` directory for patterns.
Pick a good kebab-case name for this sense. Produce a PLAN (not code) in markdown:
## Sense Design
### Name kebab-case
### Fields name, type (integer/real/text), description
### Compute Logic step-by-step, specific Node.js APIs or shell commands
### Trigger Config group, interval, throttle, timeout
Output ONLY the plan. Be precise and implementation-ready.`;
}
export function createPlannerRole(
adapter: AgentFn,
extract: LlmExtractorConfig,
): Role<PlannerMeta> {
return createRole(
adapter,
async (ctx: ThreadContext) => plannerPrompt({ threadId: ctx.threadId }),
plannerMetaSchema,
extract,
);
}
@@ -0,0 +1,59 @@
import type { AgentFn, Role, ThreadContext } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { createRole } from "@uncaged/nerve-workflow-utils";
import { z } from "zod";
export const testerMetaSchema = z.object({
passed: z.boolean().describe("true if all e2e checks passed"),
});
export type TesterMeta = z.infer<typeof testerMetaSchema>;
export function testerPrompt({
threadId,
nerveRoot,
}: { threadId: string; nerveRoot: string }): string {
return `You are testing a newly created Nerve sense end-to-end.
**IMPORTANT: The Nerve workspace is at \`${nerveRoot}\`. All paths below are relative to this directory. Always \`cd ${nerveRoot}\` first.**
Read the workflow thread for context: \`nerve thread ${threadId}\`
Read \`cat ${nerveRoot}/AGENT.md\`, then \`${nerveRoot}/CONVENTIONS.md\` and \`${nerveRoot}/.knowledge/sense.md\` if they exist.
Verify the full lifecycle in this order:
1. **File check** all required sense files exist (no per-sense \`package.json\`):
- \`senses/<name>/src/index.ts\`
- \`senses/<name>/src/schema.ts\`
- \`senses/<name>/migrations/\`
2. **Build** from the workspace root:
\`\`\`
cd ${nerveRoot} && pnpm run build
\`\`\`
(or \`npm run build\` per root \`package.json\`.) Must produce \`${nerveRoot}/dist/senses/<name>/index.js\` without errors.
3. **Config check** \`nerve validate\` passes, confirming nerve.yaml is valid.
4. **Sense list** \`nerve sense list\` shows the sense.
5. **Trigger** \`nerve sense trigger <name>\` completes without error.
6. **Query** \`nerve sense query <name>\` — retry up to 20s until rows appear.
If any step fails, include the relevant error output.
Output a clear summary: what you checked, what passed, what failed, and why.`;
}
export function createTesterRole(
adapter: AgentFn,
extract: LlmExtractorConfig,
nerveRoot: string,
): Role<TesterMeta> {
return createRole(
adapter,
async (ctx: ThreadContext) => testerPrompt({ threadId: ctx.threadId, nerveRoot }),
testerMetaSchema,
extract,
);
}
@@ -0,0 +1,42 @@
import type { AgentFn, WorkflowDefinition } from "@uncaged/nerve-core";
import { createCommitterRole } from "@uncaged/nerve-role-committer";
import { createReviewerRole } from "@uncaged/nerve-role-reviewer";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { moderator } from "./moderator.js";
import type { WorkflowMeta } from "./moderator.js";
import { createCoderRole } from "./roles/coder.js";
import { createPlannerRole } from "./roles/planner.js";
import { createTesterRole } from "./roles/tester.js";
export type CreateDevelopWorkflowDeps = {
defaultAdapter: AgentFn;
adapters: Partial<Record<keyof WorkflowMeta, AgentFn>> | null;
extract: LlmExtractorConfig;
nerveRoot: string;
};
export function createDevelopWorkflowWorkflow({
defaultAdapter,
adapters,
extract,
nerveRoot,
}: CreateDevelopWorkflowDeps): WorkflowDefinition<WorkflowMeta> {
const a = (role: keyof WorkflowMeta) => adapters?.[role] ?? defaultAdapter;
const roles = {
planner: createPlannerRole(a("planner"), extract),
coder: createCoderRole(a("coder"), extract),
reviewer: createReviewerRole(a("reviewer"), extract, {
cwd: nerveRoot,
conventionsPath: "CONVENTIONS.md",
}),
tester: createTesterRole(a("tester"), extract, nerveRoot),
committer: createCommitterRole(a("committer"), extract),
};
return {
name: "develop-workflow",
roles,
moderator,
};
}
@@ -0,0 +1,67 @@
import { END } from "@uncaged/nerve-core";
import type { Moderator } from "@uncaged/nerve-core";
import type { CommitterMeta } from "@uncaged/nerve-role-committer";
import type { ReviewerMeta } from "@uncaged/nerve-role-reviewer";
import type { CoderMeta } from "./roles/coder.js";
import type { PlannerMeta } from "./roles/planner.js";
import type { TesterMeta } from "./roles/tester.js";
export type WorkflowMeta = {
planner: PlannerMeta;
coder: CoderMeta;
reviewer: ReviewerMeta;
tester: TesterMeta;
committer: CommitterMeta;
};
const MAX_CODER_ROUNDS = 20;
const MAX_TOTAL_REJECTIONS = 10;
function coderRounds(steps: { role: string }[]): number {
return steps.filter((s) => s.role === "coder").length;
}
function totalRejections(steps: { role: string; meta: unknown }[]): number {
return steps.filter((s) => {
if (s.role === "reviewer") return !(s.meta as Record<string, boolean>).approved;
if (s.role === "tester") return !(s.meta as Record<string, boolean>).passed;
if (s.role === "committer") return !(s.meta as Record<string, boolean>).committed;
return false;
}).length;
}
function canRetryCoder(steps: { role: string; meta: unknown }[]): boolean {
return coderRounds(steps) < MAX_CODER_ROUNDS && totalRejections(steps) < MAX_TOTAL_REJECTIONS;
}
export const moderator: Moderator<WorkflowMeta> = (context) => {
if (context.steps.length === 0) return "planner";
const last = context.steps[context.steps.length - 1];
if (last.role === "planner") {
return last.meta.ready ? "coder" : END;
}
if (last.role === "coder") {
if (last.meta.done) return "reviewer";
return canRetryCoder(context.steps) ? "coder" : END;
}
if (last.role === "reviewer") {
if (last.meta.approved) return "tester";
return canRetryCoder(context.steps) ? "coder" : END;
}
if (last.role === "tester") {
if (last.meta.passed) return "committer";
return canRetryCoder(context.steps) ? "coder" : END;
}
if (last.role === "committer") {
if (last.meta.committed) return END;
return canRetryCoder(context.steps) ? "coder" : END;
}
return END;
};
@@ -0,0 +1,67 @@
import type { AgentFn, Role, ThreadContext } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { createRole } from "@uncaged/nerve-workflow-utils";
import { z } from "zod";
export const coderMetaSchema = z.object({
done: z.boolean().describe("true if the workflow files were created and build passes"),
});
export type CoderMeta = z.infer<typeof coderMetaSchema>;
export function coderPrompt({ threadId }: { threadId: string }): string {
return `Read the workflow thread to get the planner's design and any reviewer/tester/committer feedback: \`nerve thread ${threadId}\`
Read \`cat AGENT.md\` from the repository root, then \`CONVENTIONS.md\` and \`.knowledge/workflow.md\` if present. Optional skills live under \`.cursor/skills/\`.
Also look at existing workflows in the \`workflows/\` directory for patterns.
## Your task
Implement the planner's design. This may be **creating a new workflow** or **modifying an existing one**. If there is reviewer, tester, or committer feedback in the thread, fix the issues they identified.
**IMPORTANT:** The thread contains both the **initial user prompt** (the first message) and the **planner's design**. Read both carefully:
- The **initial prompt** contains the user's specific requirements for role behavior, tools to use, and acceptance criteria
- The **planner's design** contains the architecture, file structure, and routing logic
- When writing role prompts, follow the user's behavioral requirements from the initial prompt do not invent your own interpretation
## Multi-step approach
You do NOT need to finish everything in one pass. You may return \`done: false\` to continue in the next iteration. For example:
1. First pass: scaffold files / make structural changes
2. Second pass: implement role logic
3. Third pass: fix build/lint errors
## Workflow file structure (flat workspace)
The workspace has **one root** \`package.json\` and **one** root build (\`pnpm run build\` or \`npm run build\`), implemented by \`scripts/build.mjs\`, which emits bundles under \`dist/workflows/<name>/index.js\`. There is **no** per-workflow \`package.json\` or \`tsconfig.json\`.
Each workflow must have:
- \`workflows/<name>/index.ts\` — default export \`WorkflowDefinition\` (moderator and meta types typically live here or are imported from co-located modules)
- \`workflows/<name>/roles/<role>.ts\` — one TypeScript file per role (schemas, prompts, \`createRole\` wiring, or plain async role functions)
For **new workflows**, also update \`nerve.yaml\` with \`workflows.<name>\`.
## Rules
- Keep the WorkflowDefinition<WorkflowMeta> pattern
- No dynamic import()
- Use types (not interfaces)
- Meta should be simple routing signals (single boolean per role)
- Write compile-ready TypeScript
## When to return done: true
Return \`done: true\` ONLY when ALL of the following are true:
- All changes from the plan are implemented
- From the **workspace root**, \`pnpm run build\` or \`npm run build\` succeeds (run it!) so \`dist/workflows/<name>/index.js\` is produced
- No lint or type errors remain
Return \`done: false\` if you made progress but there is still work to do, or if build/lint has errors you plan to fix in the next iteration.`;
}
export function createCoderRole(adapter: AgentFn, extract: LlmExtractorConfig): Role<CoderMeta> {
return createRole(
adapter,
async (ctx: ThreadContext) => coderPrompt({ threadId: ctx.threadId }),
coderMetaSchema,
extract,
);
}
@@ -0,0 +1,68 @@
import type { AgentFn, Role, ThreadContext } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { createRole } from "@uncaged/nerve-workflow-utils";
import { z } from "zod";
export const plannerMetaSchema = z.object({
ready: z.boolean().describe("true if requirements are clear and a workflow can be implemented"),
});
export type PlannerMeta = z.infer<typeof plannerMetaSchema>;
export function plannerPrompt({ threadId }: { threadId: string }): string {
return `You are a Nerve workflow planner. You can **create new workflows** or **modify existing ones**.
Read the workflow thread for the user's request: \`nerve thread ${threadId}\`
Read the workspace guide: \`cat AGENT.md\` from the repository root (created by \`nerve init\`). Also read \`CONVENTIONS.md\` if it exists; if \`.knowledge/workflow.md\` exists (e.g. Nerve monorepo), read it for layout and engine behavior. Optional Cursor skills live under \`.cursor/skills/\`.
List existing workflows: \`ls workflows/\`
## Determine the task type
1. If the user wants to **modify an existing workflow** read its current code (\`cat workflows/<name>/index.ts\`, \`ls workflows/<name>/roles/\`, \`cat workflows/<name>/roles/<role>.ts\`, etc.) and understand its current structure before planning changes.
2. If the user wants to **create a new workflow** look at existing workflows in \`workflows/\` for patterns to follow.
## Produce a PLAN (not code) in markdown
For **new workflows**:
- Workflow name **verb-first** kebab-case phrase (e.g. \`review-pull-request\`, \`deploy-staging\`), not a bare noun
- Roles list (name, purpose, tool)
- Flow transitions / moderator routing logic
- Validation loops design
- External dependencies
- Data flow between roles
For **modifications to existing workflows**:
- Workflow name (existing)
- What changes are needed and why
- Files to add/modify/delete
- Impact on moderator routing logic (this workflow's typical order is planner coder reviewer tester committer)
- Backward compatibility considerations (if any)
**For every role (new or modified)**, include a **Role Behavior** section that describes:
- What the role should do, check, or produce
- What tools or commands it should use
- What criteria determine its meta output (e.g. approved/passed/done)
- Preserve the user's specific requirements verbatim do NOT summarize away details
If requirements are NOT clear, describe what is missing or ambiguous.
End your response with a JSON block:
\`\`\`json
{ "ready": true }
\`\`\`
or
\`\`\`json
{ "ready": false }
\`\`\``;
}
export function createPlannerRole(
adapter: AgentFn,
extract: LlmExtractorConfig,
): Role<PlannerMeta> {
return createRole(
adapter,
async (ctx: ThreadContext) => plannerPrompt({ threadId: ctx.threadId }),
plannerMetaSchema,
extract,
);
}
@@ -0,0 +1,59 @@
import type { AgentFn, Role, ThreadContext } from "@uncaged/nerve-core";
import type { LlmExtractorConfig } from "@uncaged/nerve-workflow-utils";
import { createRole } from "@uncaged/nerve-workflow-utils";
import { z } from "zod";
export const testerMetaSchema = z.object({
passed: z.boolean().describe("true if all validation checks passed"),
});
export type TesterMeta = z.infer<typeof testerMetaSchema>;
export function testerPrompt({
threadId,
nerveRoot,
}: { threadId: string; nerveRoot: string }): string {
return `You are testing a Nerve workflow — either newly created or recently modified.
**IMPORTANT: The Nerve workspace is at \`${nerveRoot}\`. All paths below are relative to this directory. Always \`cd ${nerveRoot}\` first.**
Read the workflow thread for context: \`nerve thread ${threadId}\`
Read \`cat ${nerveRoot}/AGENT.md\`, then \`${nerveRoot}/CONVENTIONS.md\` and \`${nerveRoot}/.knowledge/workflow.md\` if they exist.
Get the workflow name from the thread (the planner's output).
Verify the full lifecycle in this order:
1. **File check** all required workflow sources exist (under \`${nerveRoot}/\`):
- \`workflows/<name>/index.ts\`
- \`workflows/<name>/roles/\` with one \`.ts\` file per role (flat files, not per-role packages)
- **No** \`workflows/<name>/package.json\` or \`tsconfig.json\` expected
2. **Build** from the workspace root:
\`\`\`
cd ${nerveRoot} && pnpm run build
\`\`\`
(or \`npm run build\` if that is what the root \`package.json\` defines.) Must produce \`${nerveRoot}/dist/workflows/<name>/index.js\` without errors.
3. **Config check** \`cd ${nerveRoot} && nerve validate\` passes, confirming nerve.yaml is valid.
4. **Workflow list** \`nerve workflow list\` shows the workflow.
5. **Trigger test** \`nerve workflow trigger <name> --dry-run\` if available, otherwise just confirm the workflow appears in \`nerve workflow status\`.
If any step fails, include the relevant error output.
Output a clear summary: what you checked, what passed, what failed, and why.`;
}
export function createTesterRole(
adapter: AgentFn,
extract: LlmExtractorConfig,
nerveRoot: string,
): Role<TesterMeta> {
return createRole(
adapter,
async (ctx: ThreadContext) => testerPrompt({ threadId: ctx.threadId, nerveRoot }),
testerMetaSchema,
extract,
);
}
+4
View File
@@ -0,0 +1,4 @@
export { createDevelopSenseWorkflow } from "./develop-sense/build.js";
export type { CreateDevelopSenseDeps } from "./develop-sense/build.js";
export { createDevelopWorkflowWorkflow } from "./develop-workflow/build.js";
export type { CreateDevelopWorkflowDeps } from "./develop-workflow/build.js";
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src",
"composite": false
},
"include": ["src"]
}
@@ -0,0 +1,54 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { START, type ThreadContext } from "@uncaged/nerve-core";
import { createLlmAdapter } from "../create-llm-adapter.js";
function makeCtx(threadId: string, userContent: string): ThreadContext {
return {
threadId,
start: {
role: START,
content: userContent,
meta: { maxRounds: 10, threadId },
timestamp: 1,
},
steps: [],
};
}
describe("createLlmAdapter", () => {
afterEach(() => {
vi.unstubAllGlobals();
vi.restoreAllMocks();
});
it("posts system + user (start.content) and returns assistant text", async () => {
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
status: 200,
text: async () =>
JSON.stringify({
choices: [{ message: { content: "model reply" } }],
}),
});
vi.stubGlobal("fetch", fetchMock);
const provider = { baseUrl: "https://api.example/v1", apiKey: "k", model: "m" };
const adapter = createLlmAdapter(provider);
const out = await adapter(makeCtx("t1", "trigger text"), "system instructions");
expect(out).toBe("model reply");
expect(fetchMock).toHaveBeenCalledTimes(1);
const [, init] = fetchMock.mock.calls[0] as [string, RequestInit];
const body = JSON.parse(init.body as string) as {
model: string;
messages: Array<{ role: string; content: string }>;
};
expect(body.model).toBe("m");
expect(body.messages).toEqual([
{ role: "system", content: "system instructions" },
{ role: "user", content: "trigger text" },
]);
});
});

Some files were not shown because too many files have changed in this diff Show More