Compare commits

...

21 Commits

Author SHA1 Message Date
xiaoju a7e6caf6e7 docs: update all README files to match actual code
Rewrite documentation across all packages to reflect current
architecture, APIs, and CLI commands.

- README.md: fix reflex examples, add store package, update config
- core/README.md: add Sense→workflow routing, IPC types
- daemon/README.md: complete module table, crash recovery, createKernel
- cli/README.md: add workflow/sense/store subcommands
- store/README.md: new file documenting LogStore/BlobStore

Fixes #95
2026-04-24 21:47:37 +00:00
xiaomo d4dcd9722f Merge pull request 'refactor: share IPC message types between CLI and daemon' (#94) from refactor/93-shared-ipc-types into main 2026-04-24 15:14:50 +00:00
xiaoju 3082568b85 refactor(daemon): exhaustive IPC request dispatch
Ensure new DaemonIpcRequest variants require an explicit handler branch.

Made-with: Cursor
2026-04-24 15:11:58 +00:00
xiaoju 830b0aa762 refactor(core): shared daemon IPC request/response types
Move wire protocol types and parseDaemonIpcRequest into @uncaged/nerve-core so CLI and daemon share one definition. Type sendAndReceive message as DaemonIpcRequest. Align workflow trigger CLI with daemon (prompt, maxRounds from --payload JSON).

Made-with: Cursor
2026-04-24 15:10:00 +00:00
xiaoju 777d51cc73 chore: bump version to 0.4.0
小橘 🍊(NEKO Team)
2026-04-24 13:22:30 +00:00
xiaomo 06a957d62a Merge pull request 'chore: add pre-push hook to run tests before push' (#92) from chore/add-pre-push-hook into main 2026-04-24 13:19:10 +00:00
xiaoju b2c379cbfd refactor: reduce cognitive complexity in 3 functions
Extract helpers to bring all functions below biome's complexity threshold (15):
- store/log-store.ts: extract recordToRoundMessage() from parseRoundPayload()
- cli/commands/workflow.ts: extract buildTruncatedSingleRound() from buildThreadCommandOutput()
- daemon/workflow-worker.ts: extract validateRoleResult(), buildInitialLastSignal(),
  initChain(), executeRole() from runThread()

小橘 🍊(NEKO Team)
2026-04-24 12:44:39 +00:00
xiaoju 7cb7112ed6 chore: fix biome lint errors and tune overrides
- Remove duplicate 'prepare' key in package.json
- Allow default exports in rslib.config.ts
- Relax noExplicitAny and noNonNullAssertion in test files
- Auto-fix 17 files (imports, formatting)

小橘 🍊(NEKO Team)
2026-04-24 12:36:57 +00:00
xiaoju 48c81c2e19 chore: add biome lint check to pre-push hook
小橘 🍊(NEKO Team)
2026-04-24 12:32:41 +00:00
xiaoju dd3d4315c4 chore: add pre-push hook to run tests before push
Adds husky with a pre-push hook that runs `pnpm -r test` to catch
test failures before they reach the remote.

小橘 🍊(NEKO Team)
2026-04-24 12:28:47 +00:00
xingyue 788ebc6779 Merge pull request 'fix(test): align tests with type-safety refactor' (#91) from fix/test-failures-after-type-safety-refactor into main 2026-04-24 12:24:50 +00:00
xiaoju 8807b0ac6a fix(test): align tests with type-safety refactor
Update test expectations after workflow reflexes were removed from
YAML config and type signatures were tightened:

- core/config: workflow reflex tests now expect 'not supported' error
- cli/workflow: partitionWorkflowMessage test uses strict typed params
- daemon/crash-recovery: remove triggerPayload from resume-thread assertion
- daemon/daemon-ipc: trigger-workflow sends prompt+maxRounds
- daemon/kernel-workflow: use Sense-driven workflow trigger pattern

Fixes 12 test failures across core, cli, and daemon packages.

Refs #88, #89
2026-04-24 12:23:21 +00:00
xiaomo 5b65afdc4b Merge pull request 'refactor: improve type safety across codebase' (#90) from refactor/type-safety into main 2026-04-24 12:09:36 +00:00
xingyue f5cb72db50 refactor: improve type safety across codebase
- Add isPlainRecord() type guard to eliminate 'as Record<string, unknown>' casts
- Replace 'as any' with properly typed assertions in start.ts
- Remove 'null as unknown as' pattern in kernel.ts
- Add type predicates for array narrowing (item is string)
- Improve IPC message type narrowing in daemon-client.ts and ipc.ts
- Type better-sqlite3 and drizzle return values properly

No runtime behavior changes.
2026-04-24 20:07:58 +08:00
xiaomo e433e7c2a9 Merge pull request 'refactor(daemon): split kernel.ts into focused modules' (#89) from refactor/split-kernel into main 2026-04-24 11:41:44 +00:00
xingyue 47cc49eab4 refactor(daemon): split kernel.ts into focused modules (#86)
- Extract worker-pool.ts (211 LOC): sense worker fork/shutdown/restart/crash recovery
- Extract kernel-file-watch.ts (92 LOC): file change handlers for hot reload
- Extract kernel-sense-groups.ts (29 LOC): group lookup utilities
- kernel.ts reduced from 617 → 380 LOC (thin orchestrator)
- Add worker-pool.test.ts with 8 test cases
- No behavior changes, all existing tests unchanged
2026-04-24 19:39:10 +08:00
xiaomo 65012fbb53 Merge pull request 'refactor(store): extract @uncaged/nerve-store from daemon' (#88) from refactor/extract-nerve-store into main 2026-04-24 11:29:13 +00:00
xingyue 8d00f9cba1 refactor(store): extract @uncaged/nerve-store from daemon (#85)
- Create packages/store/ with log-store, log-archive, blob-store (~900 LOC)
- daemon depends on @uncaged/nerve-store (workspace:*)
- CLI depends on @uncaged/nerve-store, delete daemon-types.ts
- Move store-related tests to packages/store/src/__tests__/
- All store tests pass (73/73), no new regressions
2026-04-24 19:26:46 +08:00
xiaomo ef38b121f7 Merge pull request 'fix: PR #81 review follow-ups (closes #83)' (#84) from fix/pr81-review-followups into main 2026-04-24 11:10:45 +00:00
xiaoju 9bf0b2abb8 fix: PR #81 review follow-ups (closes #83)
- Filter __start__ messages in getThreadRoundCount SQL to fix round offset
- Remove duplicate parseWorkflowField, use parseSenseWorkflowDirective
- Remove unnecessary double casts in workflow CLI
- Add runtime validation for Role meta in workflow-worker
- Export DEFAULT_ENGINE_MAX_ROUNDS from types.ts

小橘 🍊(NEKO Team)
2026-04-24 11:09:31 +00:00
xiaomo d93f5c8fa2 Merge pull request 'refactor(core): restore type-safe workflow automaton from Pulse design' (#81) from refactor/workflow-type-safety into main 2026-04-24 11:02:23 +00:00
68 changed files with 1735 additions and 980 deletions
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
pnpm check
pnpm -r test
+68 -44
View File
@@ -7,28 +7,31 @@ Nerve is a lightweight daemon that continuously observes external state through
## Core Concepts
```
External World → Sense → Signal → Reflex → Workflow → Log
↑ ↑
"what to observe" "what to do"
External World → Sense ─┬→ Signal → Reflex → Sense (scheduled compute)
└→ Workflow (Sense return with workflow directive) → Log
```
| Concept | Metaphor | Role |
|---------|----------|------|
| **Sense** | 👁️ Perception | A `compute()` function that samples or derives data. Each sense has its own SQLite database. |
| **Reflex** | ⚡ Reaction | Declarative trigger — interval-based, event-driven, or both. Connects senses to actions. |
| **Signal** | 📡 Notification | Emitted when a sense returns non-null. Other reflexes can listen for signals. |
| **Workflow** | 🔧 Action | Stateful multi-step execution with Roles (actors) and a Moderator (coordinator). |
| **Reflex** | ⚡ Reaction | Declarative rules that **only schedule Sense computes** (interval and/or `on` signal names). Reflex YAML cannot reference workflows. |
| **Signal** | 📡 Notification | Emitted when a sense returns a non-null value that is routed as a normal signal (see Sense → Workflow below). Other reflexes can listen via `on`. |
| **Workflow** | 🔧 Action | Stateful multi-step execution with Roles and a Moderator. Started from a Sense return value or from CLI/daemon IPC—not from reflex YAML. |
| **Log** | 📝 Record | Immutable audit trail. Queryable by senses, but **cannot** trigger reflexes (prevents feedback loops). |
Three extension points, fully orthogonal — a Sense doesn't know when it runs, a Reflex doesn't know what it computes, a Workflow doesn't know why it was triggered.
**Sense → Workflow:** if `compute()` returns a plain object with a string field `workflow` in the form `name|maxRounds|prompt` (only the first two `|` delimit name and rounds; the rest is the prompt), the engine starts that workflow and **does not** emit a Signal for that return. `workflow: null` or `""` means “emit a signal” and strip the key from the payload. Invalid `workflow` strings are treated like a normal signal (directive stripped). See `@uncaged/nerve-core` `routeSenseComputeOutput` / `parseSenseWorkflowDirective`.
Three extension points for **what / when / multi-step action** — reflexes never replace Sense-driven workflow launches.
## Packages
| Package | Description |
|---------|-------------|
| [`@uncaged/nerve-core`](./packages/core) | Shared types and config parser |
| [`@uncaged/nerve-daemon`](./packages/daemon) | The observation engine — kernel, sense runtime, reflex scheduler, workflow manager |
| [`@uncaged/nerve-cli`](./packages/cli) | CLI tool (`nerve`) — init, start, stop, logs, query |
| [`@uncaged/nerve-core`](./packages/core) | Shared types, config parser, Sense→workflow routing, daemon IPC protocol |
| [`@uncaged/nerve-store`](./packages/store) | Append-only log SQLite, JSONL archive, CAS blob store, workflow run rows |
| [`@uncaged/nerve-daemon`](./packages/daemon) | Kernel, workers, signal bus, reflex scheduler, workflow manager, file watcher, IPC |
| [`@uncaged/nerve-cli`](./packages/cli) | CLI (`nerve`) — init, validate, daemon, dev, logs, sense, store, workflow |
## Quick Start
@@ -70,15 +73,17 @@ nerve logs # view logs
## Configuration
`nerve.yaml` declares senses, reflexes, and workflows:
`nerve.yaml` declares senses, reflexes (sense-only), optional workflows (concurrency), and optional engine `max_rounds`:
```yaml
max_rounds: 100 # default moderator cap (e.g. CLI workflow trigger)
senses:
cpu-usage:
group: system # senses in the same group share a worker process
throttle: 10s # min interval between computes
timeout: 30s # max compute duration
gracePeriod: 5s # wait before first compute after startup
grace_period: 5s # wait before first compute after startup
reflexes:
- kind: sense
@@ -86,10 +91,6 @@ reflexes:
interval: 30s # periodic trigger
on: [disk-pressure] # also trigger on signals from other senses
- kind: workflow
workflow: cleanup
on: [disk-pressure] # start a workflow when signal fires
workflows:
cleanup:
concurrency: 1
@@ -97,43 +98,66 @@ workflows:
code-review:
concurrency: 3
overflow: queue
maxQueue: 20
max_queue: 20
```
YAML must **not** include `workflow:` under `reflexes` — the parser rejects it. Declare workflows under `workflows:` and start them from Sense `compute()` or `nerve workflow trigger`.
**Example — Sense starts a workflow** (`senses/disk-pressure/compute.ts`):
```typescript
export async function compute() {
const full = await diskNearlyFull();
if (!full) return null;
return {
path: "/data",
workflow: "cleanup|10|Disk partition nearly full", // name|maxRounds|prompt
};
}
```
## Architecture
```
┌─────────────────────────────────────────────────────────┐
│ Kernel │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ │ Worker │ │ Worker │ │ Worker (1 per
│ (group A)│ │ (group B)│ │ (group C)│ group)
│ sense-1 │ │ sense-3 │ │ sense-5 │ │
│ │ sense-2 │ │ sense-4 │ │ │ │
│ └─────────┘ └────┬─────┘ └────┬─────┘
└──────────────┼──────────────┘
┌──────────────┐
│ Signal Bus │
│ └─────────────┘
──────────────────
│ Reflex Scheduler │
└────────┬─────────
┌───────────────────
Workflow Manager │──→ Log Store (SQLite)
└───────────────────
└─────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────
│ Kernel
│ ┌──────────────┐ watches nerve.yaml / senses / workflows
│ │ File Watcher ├──────────────────────────────────────────┐
└──────────────┘ │
┌──────────────┐ CLI ↔ newline JSON (trigger-workflow, │ │
│ │ Daemon IPC │ trigger-sense, list-senses) │ │
│ └──────┬───────┘
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ │ Worker │ │ Worker │ │ Worker │ (1 per
│ (group A)│ │ (group B)│ │ (group C)│ group)
│ sense-1 │ sense-3 │ sense-5 │
│ │ sense-2 │ │ sense-4 │ │
└────┬─────┘ └─────────└────┬─────┘
│ │
└────────────────────────────┘
│ ┌──────────────┐
│ Signal Bus │
│ └─────────────
│ │
│ ┌──────────────────
│ │ │ Reflex Scheduler│ │
│ │ └────────┬─────────┘ │
│ │ ▼ │
│ │ ┌───────────────────┐ │
│ └───────────────────►│ Workflow Manager │──→ @uncaged/nerve-store │
│ └───────────────────┘ (logs.db, …) │
└────────────────────────────────────────────────────────────────────────┘
```
- **Worker processes** — one per sense group, forked by the kernel. Isolated compute execution.
- **Worker pool** — one child process per sense group; isolation between groups.
- **Signal Bus** — in-memory pub/sub for signal distribution.
- **Reflex Scheduler** — interval timers + signal subscriptions, with throttle/coalesce.
- **Workflow Manager** — concurrency control (drop/queue), thread lifecycle tracking.
- **Log Store** — WAL-mode SQLite via `node:sqlite`, with archival and retention policies.
- **Workflow Manager** — concurrency (drop/queue), per-workflow workers, crash recovery.
- **File watcher** — hot reload for config, sense modules, and workflow modules.
- **Daemon IPC** — Unix domain socket; used by the CLI when the daemon is running.
- **Log / blob storage** — implemented in `@uncaged/nerve-store` (WAL SQLite, JSONL archive, CAS blobs).
## Tech Stack
+14 -1
View File
@@ -19,7 +19,7 @@
},
"overrides": [
{
"include": ["tsup.config.ts"],
"include": ["tsup.config.ts", "*/rslib.config.ts"],
"linter": {
"rules": {
"style": {
@@ -27,6 +27,19 @@
}
}
}
},
{
"include": ["**/__tests__/**"],
"linter": {
"rules": {
"suspicious": {
"noExplicitAny": "off"
},
"style": {
"noNonNullAssertion": "off"
}
}
}
}
],
"linter": {
+2
View File
@@ -5,6 +5,7 @@
"node": ">=22.5.0"
},
"scripts": {
"prepare": "husky",
"build": "pnpm -r run build",
"check": "biome check .",
"format": "biome format --write ."
@@ -12,6 +13,7 @@
"devDependencies": {
"@biomejs/biome": "^1.9.0",
"@rslib/core": "^0.21.3",
"husky": "^9.1.7",
"typescript": "^5.5.0"
}
}
+32 -14
View File
@@ -21,41 +21,59 @@ nerve init # Initialize a nerve workspace (installs deps, scaff
nerve validate # Validate nerve.yaml configuration
```
### Daemon Management
### Daemon management
```bash
nerve daemon start # Start the daemon (background)
nerve daemon stop # Stop the daemon
nerve daemon status # Check daemon health
nerve daemon restart # Restart the daemon
nerve daemon logs # Tail daemon logs
nerve daemon status # Show pid, uptime, sense names from nerve.yaml (process must exist)
nerve daemon restart # Stop then start
nerve daemon logs # Tail daemon process logs (file under workspace logs/)
```
### Development
```bash
nerve dev # Run in foreground mode (no daemon, Ctrl+C to stop)
nerve dev # Foreground kernel with file watcher + IPC (Ctrl+C stops)
```
### Querying
### Querying & status
```bash
nerve logs # View structured logs
nerve sense query <name> # Query a sense's SQLite database
nerve sense schema <name> # Show a sense's database schema
nerve status # Daemon health summary
nerve logs # Tail or page the daemon text log file (path in footer; default ~/.uncaged-nerve/logs/nerve.log)
nerve status # Short daemon health summary (aliases daemon status)
```
Structured rows in `data/logs.db` are surfaced via **`nerve workflow inspect`** / **`nerve workflow list`** (and `LogStore` in code), not via `nerve logs`.
### Sense
```bash
nerve sense list # List senses (live fields from daemon IPC when running)
nerve sense trigger <name> # IPC trigger-sense — queue a compute for that sense
nerve sense query <name> # Read-only SQL on data/senses/<name>.db (optional SQL args)
nerve sense schema <name> # Print CREATE TABLE statements for that sense DB
```
### Store maintenance
```bash
nerve store archive # Move old log rows to JSONL under data/archive/logs/… (optional --vacuum)
```
### Workflows
```bash
nerve workflow list # List workflow runs
nerve workflow show <runId> # Show workflow run details
nerve workflow list # Queued/started runs (add --all for terminal states; --workflow, --limit, --offset)
nerve workflow inspect <runId> # Run metadata + paginated workflow log lines
nerve workflow thread <runId> # Role rounds from persisted messages (--before, --budget)
nerve workflow trigger <name> # IPC trigger-workflow (daemon must be running)
# Optional JSON: --payload '{"prompt":"…","maxRounds":50}'
```
### Top-level Aliases
`nerve workflow trigger` sends a `trigger-workflow` line on the daemon Unix socket (same protocol as `@uncaged/nerve-core` / `parseDaemonIpcRequest`). It does not read `nerve.yaml` workflow definitions beyond what the running daemon already loaded.
For convenience, these aliases are available:
### Top-level aliases
```bash
nerve start → nerve daemon start
+2 -2
View File
@@ -3,7 +3,7 @@
"engines": {
"node": ">=22.5.0"
},
"version": "0.3.0",
"version": "0.4.0",
"type": "module",
"bin": {
"nerve": "dist/cli.js"
@@ -21,13 +21,13 @@
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*",
"@uncaged/nerve-store": "workspace:*",
"citty": "^0.1.6",
"yaml": "^2.8.3"
},
"devDependencies": {
"@rslib/core": "^0.21.3",
"@types/node": "^22.0.0",
"@uncaged/nerve-daemon": "workspace:*",
"vitest": "^4.1.5"
}
}
+1 -1
View File
@@ -20,6 +20,6 @@ export default defineConfig({
output: {
target: "node",
cleanDistPath: true,
externals: ["@uncaged/nerve-daemon"],
externals: ["@uncaged/nerve-daemon", "@uncaged/nerve-store"],
},
});
@@ -1,80 +0,0 @@
/**
* Compile-time check: daemon-types.ts stays in sync with @uncaged/nerve-daemon exports.
* If the daemon package changes its public API, this file will fail to compile.
*/
import type { SenseInfo } from "@uncaged/nerve-core";
import type {
ArchiveLogsDayResult as DaemonArchiveLogsDayResult,
ArchiveLogsOptions as DaemonArchiveLogsOptions,
ArchiveLogsResult as DaemonArchiveLogsResult,
LogEntry as DaemonLogEntry,
LogQuery as DaemonLogQuery,
LogStore as DaemonLogStore,
SenseInfo as DaemonSenseInfo,
WorkflowRun as DaemonWorkflowRun,
WorkflowRunStatus as DaemonWorkflowRunStatus,
} from "@uncaged/nerve-daemon";
import { describe, expectTypeOf, it } from "vitest";
import type {
ArchiveLogsDayResult,
ArchiveLogsOptions,
ArchiveLogsResult,
LogEntry,
LogQuery,
LogStore,
WorkflowRun,
WorkflowRunStatus,
} from "../daemon-types.js";
describe("daemon-types drift guard", () => {
it("SenseInfo matches daemon package export (list-senses IPC)", () => {
expectTypeOf<SenseInfo>().toMatchTypeOf<DaemonSenseInfo>();
expectTypeOf<DaemonSenseInfo>().toMatchTypeOf<SenseInfo>();
});
it("WorkflowRunStatus is assignable both ways", () => {
expectTypeOf<WorkflowRunStatus>().toMatchTypeOf<DaemonWorkflowRunStatus>();
expectTypeOf<DaemonWorkflowRunStatus>().toMatchTypeOf<WorkflowRunStatus>();
});
it("WorkflowRun is assignable both ways", () => {
expectTypeOf<WorkflowRun>().toMatchTypeOf<DaemonWorkflowRun>();
expectTypeOf<DaemonWorkflowRun>().toMatchTypeOf<WorkflowRun>();
});
it("LogEntry is assignable both ways", () => {
expectTypeOf<LogEntry>().toMatchTypeOf<DaemonLogEntry>();
expectTypeOf<DaemonLogEntry>().toMatchTypeOf<LogEntry>();
});
it("LogQuery is assignable both ways", () => {
expectTypeOf<LogQuery>().toMatchTypeOf<DaemonLogQuery>();
expectTypeOf<DaemonLogQuery>().toMatchTypeOf<LogQuery>();
});
it("LogStore has all required methods", () => {
expectTypeOf<LogStore>().toMatchTypeOf<
Pick<
DaemonLogStore,
| "query"
| "getWorkflowRun"
| "getActiveWorkflowRuns"
| "getAllWorkflowRuns"
| "upsertWorkflowRun"
| "archiveLogs"
| "close"
>
>();
});
it("ArchiveLogs types match daemon", () => {
expectTypeOf<ArchiveLogsOptions>().toMatchTypeOf<DaemonArchiveLogsOptions>();
expectTypeOf<DaemonArchiveLogsOptions>().toMatchTypeOf<ArchiveLogsOptions>();
expectTypeOf<ArchiveLogsResult>().toMatchTypeOf<DaemonArchiveLogsResult>();
expectTypeOf<DaemonArchiveLogsResult>().toMatchTypeOf<ArchiveLogsResult>();
expectTypeOf<ArchiveLogsDayResult>().toMatchTypeOf<DaemonArchiveLogsDayResult>();
expectTypeOf<DaemonArchiveLogsDayResult>().toMatchTypeOf<ArchiveLogsDayResult>();
});
});
+2 -2
View File
@@ -234,7 +234,7 @@ describe("logsCommand negative offset", () => {
it("exits with code 1 and writes to stderr when offset is negative", async () => {
await expect(
logsCommand.run!({
logsCommand.run?.({
args: { n: "50", offset: "-5", follow: false },
rawArgs: [],
cmd: logsCommand as never,
@@ -247,7 +247,7 @@ describe("logsCommand negative offset", () => {
it("exits with code 1 for offset=-1", async () => {
await expect(
logsCommand.run!({
logsCommand.run?.({
args: { n: "10", offset: "-1", follow: false },
rawArgs: [],
cmd: logsCommand as never,
+13 -8
View File
@@ -12,9 +12,10 @@ import { createServer } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createLogStore } from "@uncaged/nerve-daemon";
import { createLogStore } from "@uncaged/nerve-store";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import type { LogStore, ThreadRoundRow, WorkflowRun } from "@uncaged/nerve-store";
import {
DEFAULT_THREAD_BUDGET_CHARS,
buildInspectOutput,
@@ -28,7 +29,6 @@ import {
statusIcon,
} from "../commands/workflow.js";
import { triggerWorkflowViaDaemon } from "../daemon-client.js";
import type { LogStore, ThreadRoundRow, WorkflowRun } from "../daemon-types.js";
// ---------------------------------------------------------------------------
// Test helpers
@@ -342,9 +342,14 @@ describe("partitionWorkflowMessage", () => {
expect(p.meta).toEqual({ items: [1, 2] });
});
it("uses fallback role and stringifies non-string content", () => {
const p = partitionWorkflowMessage({ content: { n: 1 } });
expect(p.roleStr).toBe("?");
it("passes through role and content as-is", () => {
const p = partitionWorkflowMessage({
role: "unknown",
content: '{"n":1}',
meta: null,
timestamp: 0,
});
expect(p.roleStr).toBe("unknown");
expect(p.contentBody).toBe('{"n":1}');
});
});
@@ -509,7 +514,7 @@ describe("triggerWorkflowViaDaemon", () => {
await new Promise<void>((r) => server.listen(sockPath, r));
try {
const result = await triggerWorkflowViaDaemon(sockPath, "my-workflow", {});
const result = await triggerWorkflowViaDaemon(sockPath, "my-workflow", "", 100);
expect(result).toEqual({ ok: true });
} finally {
await new Promise<void>((r) => server.close(() => r()));
@@ -525,7 +530,7 @@ describe("triggerWorkflowViaDaemon", () => {
await new Promise<void>((r) => server.listen(sockPath, r));
try {
const result = await triggerWorkflowViaDaemon(sockPath, "missing", {});
const result = await triggerWorkflowViaDaemon(sockPath, "missing", "", 100);
expect(result).toEqual({ ok: false, error: "unknown workflow" });
} finally {
await new Promise<void>((r) => server.close(() => r()));
@@ -533,7 +538,7 @@ describe("triggerWorkflowViaDaemon", () => {
});
it("rejects when no daemon is listening on the socket", async () => {
await expect(triggerWorkflowViaDaemon(sockPath, "my-workflow", {})).rejects.toThrow(
await expect(triggerWorkflowViaDaemon(sockPath, "my-workflow", "", 100)).rejects.toThrow(
/Cannot connect to daemon/,
);
});
+1 -1
View File
@@ -85,7 +85,7 @@ export function buildLogFooter(slice: LogSlice, nArg: number, logPath: string):
let footer = `\n📄 ${rangeStr} | ${logPath}\n`;
if (slice.nextOffset !== null) {
footer += `⏩ Earlier lines available. Fetch previous page:\n`;
footer += "⏩ Earlier lines available. Fetch previous page:\n";
footer += ` nerve logs --offset ${slice.nextOffset} -n ${nArg}\n`;
}
+4 -4
View File
@@ -1,13 +1,12 @@
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { DatabaseSync } from "node:sqlite";
import type { DatabaseSync } from "node:sqlite";
import { type SenseInfo, parseNerveConfig } from "@uncaged/nerve-core";
import { type SenseInfo, isPlainRecord, parseNerveConfig } from "@uncaged/nerve-core";
import { defineCommand } from "citty";
import { listSensesViaDaemon, triggerSenseViaDaemon } from "../daemon-client.js";
import {
assertSenseDbExists,
defaultPreviewSql,
formatRowsAsAlignedTable,
listTableSqlStatements,
@@ -240,7 +239,8 @@ const senseQueryCommand = defineCommand({
}
}
const rows = db.prepare(sql).all() as Record<string, unknown>[];
const rawRows: unknown[] = db.prepare(sql).all();
const rows: Record<string, unknown>[] = rawRows.filter(isPlainRecord);
if (args.json) {
process.stdout.write(`${JSON.stringify(rows, null, 2)}\n`);
+3 -1
View File
@@ -74,9 +74,11 @@ async function runDaemon(nerveRoot: string): Promise<void> {
const bootstrapPath = daemonBootstrapScript();
// After `open`, file-backed WriteStream has a numeric OS fd for spawn stdio; `@types/node` omits `fd` on this WriteStream alias.
const logFd = (logStream as unknown as { fd: number }).fd;
const child = spawn(process.execPath, [bootstrapPath], {
detached: true,
stdio: ["ignore", (logStream as any).fd, (logStream as any).fd],
stdio: ["ignore", logFd, logFd],
env: { ...process.env, NERVE_ROOT: nerveRoot },
cwd: nerveRoot,
});
+5 -1
View File
@@ -47,7 +47,11 @@ export const statusCommand = defineCommand({
return;
}
const pid = readPidFile() as number;
const pid = readPidFile();
if (pid === null) {
process.stdout.write("😴 Nerve daemon is not running.\n");
return;
}
const configPath = join(getNerveRoot(), "nerve.yaml");
let senseList: string[] = [];
+59 -53
View File
@@ -1,11 +1,13 @@
import { existsSync } from "node:fs";
import { join } from "node:path";
import type { DaemonIpcTriggerResponse } from "@uncaged/nerve-core";
import { DEFAULT_ENGINE_MAX_ROUNDS, isPlainRecord } from "@uncaged/nerve-core";
import { defineCommand } from "citty";
import { stringify } from "yaml";
import type { LogStore, ThreadRoundRow, WorkflowRun } from "@uncaged/nerve-store";
import { triggerWorkflowViaDaemon } from "../daemon-client.js";
import type { LogStore, ThreadRoundRow, WorkflowRun } from "../daemon-types.js";
import { loadDaemonModule } from "../workspace-daemon.js";
import { getNerveRoot, getSocketPath, isRunning } from "../workspace.js";
@@ -193,18 +195,19 @@ export type PartitionedMessage = {
* Extract display fields from a WorkflowMessage-shaped object.
* `role` and `content` are used for header/body; `meta` is serialized as YAML frontmatter.
*/
export function partitionWorkflowMessage(msg: Record<string, unknown>): PartitionedMessage {
const roleStr = typeof msg.role === "string" ? msg.role : "?";
const contentRaw = msg.content;
const contentBody =
contentRaw === undefined || contentRaw === null
? ""
: typeof contentRaw === "string"
? contentRaw
: JSON.stringify(contentRaw);
export function partitionWorkflowMessage(msg: {
role: string;
content: string;
meta: unknown;
timestamp: number;
}): PartitionedMessage {
const roleStr = msg.role;
const contentBody = msg.content;
const meta: Record<string, unknown> =
msg.meta !== null && msg.meta !== undefined && typeof msg.meta === "object"
? (msg.meta as Record<string, unknown>)
? isPlainRecord(msg.meta)
? msg.meta
: (msg.meta as Record<string, unknown>)
: {};
return { roleStr, contentBody, meta };
}
@@ -213,18 +216,10 @@ export function partitionWorkflowMessage(msg: Record<string, unknown>): Partitio
* One role round as plain text: header line, YAML frontmatter (meta only), body (content).
*/
export function formatThreadRoundBlock(row: ThreadRoundRow): string {
const { roleStr, contentBody, meta } = partitionWorkflowMessage(
row.message as unknown as Record<string, unknown>,
);
const { roleStr, contentBody, meta } = partitionWorkflowMessage(row.message);
const yamlBlock =
Object.keys(meta).length === 0 ? "{}\n" : `${stringify(meta, { lineWidth: 100 })}\n`;
return (
`[#${row.round} ${roleStr}] ${formatTs(row.ts)}\n` +
`---\n` +
yamlBlock +
`---\n` +
`${contentBody}\n\n`
);
return `[#${row.round} ${roleStr}] ${formatTs(row.ts)}\n---\n${yamlBlock}---\n${contentBody}\n\n`;
}
export type ThreadCommandOutput = {
@@ -232,6 +227,33 @@ export type ThreadCommandOutput = {
paginationHint: string | null;
};
function buildTruncatedSingleRound(
row: ThreadRoundRow,
remaining: number,
prefixLines: string[],
runId: string,
budgetFlag: string,
): ThreadCommandOutput {
const { roleStr, contentBody, meta } = partitionWorkflowMessage(row.message);
const yamlBlock =
Object.keys(meta).length === 0 ? "{}\n" : `${stringify(meta, { lineWidth: 100 })}\n`;
const header = `[#${row.round} ${roleStr}] ${formatTs(row.ts)}\n---\n${yamlBlock}---\n`;
const maxBody = Math.max(0, remaining - header.length - "[truncated]\n".length);
const truncated =
maxBody > 0 && contentBody.length > maxBody
? `${contentBody.slice(0, maxBody)}\n[truncated]\n`
: `${contentBody}\n[truncated]\n`;
const single = `${header + truncated}\n`;
const hintRound = row.round;
return {
lines: [...prefixLines, single],
paginationHint:
hintRound > 1
? `\n⏩ Older rounds exist. Fetch with:\n nerve workflow thread ${runId} --before ${String(hintRound)}${budgetFlag}\n`
: null,
};
}
/**
* Build stdout lines for `nerve workflow thread`: newest-first selection from
* `descRows` until `budgetChars` (including `prefixLines`), then chronological order.
@@ -257,27 +279,7 @@ export function buildThreadCommandOutput(
continue;
}
if (picked.length === 0) {
const { roleStr, contentBody, meta } = partitionWorkflowMessage(
row.message as unknown as Record<string, unknown>,
);
const yamlBlock =
Object.keys(meta).length === 0 ? "{}\n" : `${stringify(meta, { lineWidth: 100 })}\n`;
const header =
`[#${row.round} ${roleStr}] ${formatTs(row.ts)}\n` + `---\n` + yamlBlock + `---\n`;
const maxBody = Math.max(0, remaining - header.length - `[truncated]\n`.length);
const truncated =
maxBody > 0 && contentBody.length > maxBody
? `${contentBody.slice(0, maxBody)}\n[truncated]\n`
: `${contentBody}\n[truncated]\n`;
const single = header + truncated + "\n";
const hintRound = row.round;
return {
lines: [...prefixLines, single],
paginationHint:
hintRound > 1
? `\n⏩ Older rounds exist. Fetch with:\n nerve workflow thread ${runId} --before ${String(hintRound)}${budgetFlag}\n`
: null,
};
return buildTruncatedSingleRound(row, remaining, prefixLines, runId, budgetFlag);
}
break;
}
@@ -286,9 +288,7 @@ export function buildThreadCommandOutput(
const shownMinRound = picked.length === 0 ? null : Math.min(...picked.map((r) => r.round));
let paginationHint: string | null = null;
if (shownMinRound !== null && shownMinRound > 1) {
paginationHint =
`\n⏩ Older rounds not shown. Fetch with:\n` +
` nerve workflow thread ${runId} --before ${String(shownMinRound)}${budgetFlag}\n`;
paginationHint = `\n⏩ Older rounds not shown. Fetch with:\n nerve workflow thread ${runId} --before ${String(shownMinRound)}${budgetFlag}\n`;
}
return { lines: [...prefixLines, ...blocksAsc], paginationHint };
@@ -457,10 +457,7 @@ const workflowThreadCommand = defineCommand({
const totalRoleRounds = store.getThreadRoundCount(args.runId);
if (totalRoleRounds === 0) {
process.stdout.write(
`🧵 Workflow thread: ${run.runId}\n` +
` workflow: ${run.workflow}\n` +
` status: ${run.status}\n\n` +
`📭 No role rounds recorded for this run.\n`,
`🧵 Workflow thread: ${run.runId}\n workflow: ${run.workflow}\n status: ${run.status}\n\n📭 No role rounds recorded for this run.\n`,
);
return;
}
@@ -471,7 +468,7 @@ const workflowThreadCommand = defineCommand({
});
const prefixLines = [
`🧵 Role rounds (workflow thread)\n`,
"🧵 Role rounds (workflow thread)\n",
` runId: ${run.runId}\n`,
` workflow: ${run.workflow}\n`,
` status: ${run.status}\n`,
@@ -517,7 +514,8 @@ const workflowTriggerCommand = defineCommand({
},
payload: {
type: "string",
description: "JSON payload to pass as trigger payload (default: {})",
description:
'JSON with optional "prompt" (string) and "maxRounds" (number) for the workflow run (default: {})',
default: "{}",
},
},
@@ -530,15 +528,23 @@ const workflowTriggerCommand = defineCommand({
process.exit(1);
}
let prompt = "";
let maxRounds = DEFAULT_ENGINE_MAX_ROUNDS;
if (isPlainRecord(triggerPayload)) {
const p = triggerPayload;
if (typeof p.prompt === "string") prompt = p.prompt;
if (typeof p.maxRounds === "number") maxRounds = p.maxRounds;
}
if (!isRunning()) {
process.stderr.write("❌ Nerve daemon is not running. Start it with `nerve start`.\n");
process.exit(1);
}
const socketPath = getSocketPath();
let response: { ok: true } | { ok: false; error: string };
let response: DaemonIpcTriggerResponse;
try {
response = await triggerWorkflowViaDaemon(socketPath, args.name, triggerPayload);
response = await triggerWorkflowViaDaemon(socketPath, args.name, prompt, maxRounds);
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`❌ Failed to contact daemon: ${msg}\n`);
+48 -26
View File
@@ -8,22 +8,35 @@
import { connect } from "node:net";
import type { Socket } from "node:net";
import type { SenseInfo } from "@uncaged/nerve-core";
import type {
DaemonIpcListSensesResponse,
DaemonIpcRequest,
DaemonIpcTriggerResponse,
SenseInfo,
} from "@uncaged/nerve-core";
import { isPlainRecord } from "@uncaged/nerve-core";
const CONNECT_TIMEOUT_MS = 3_000;
const RESPONSE_TIMEOUT_MS = 5_000;
export type { SenseInfo };
type TriggerResponse = { ok: true } | { ok: false; error: string };
function isSenseInfo(value: unknown): value is SenseInfo {
if (!isPlainRecord(value)) return false;
return (
typeof value.name === "string" &&
typeof value.group === "string" &&
(value.throttle === null || typeof value.throttle === "number") &&
(value.timeout === null || typeof value.timeout === "number") &&
(value.lastSignalTs === null || typeof value.lastSignalTs === "number")
);
}
type ListSensesResponse = { ok: true; senses: SenseInfo[] } | { ok: false; error: string };
function parseDaemonResponse(line: string): TriggerResponse {
function parseDaemonResponse(line: string): DaemonIpcTriggerResponse {
try {
const obj = JSON.parse(line) as unknown;
if (obj !== null && typeof obj === "object") {
const r = obj as Record<string, unknown>;
const obj: unknown = JSON.parse(line);
if (isPlainRecord(obj)) {
const r = obj;
if (r.ok === true) return { ok: true };
if (r.ok === false && typeof r.error === "string") return { ok: false, error: r.error };
}
@@ -33,14 +46,15 @@ function parseDaemonResponse(line: string): TriggerResponse {
return { ok: false, error: `Unexpected daemon response: ${line}` };
}
function parseListSensesResponse(line: string): ListSensesResponse {
function parseListSensesResponse(line: string): DaemonIpcListSensesResponse {
try {
const obj = JSON.parse(line) as unknown;
if (obj !== null && typeof obj === "object") {
const r = obj as Record<string, unknown>;
const obj: unknown = JSON.parse(line);
if (isPlainRecord(obj)) {
const r = obj;
if (r.ok === false && typeof r.error === "string") return { ok: false, error: r.error };
if (r.ok === true && Array.isArray(r.senses))
return { ok: true, senses: r.senses as SenseInfo[] };
if (r.ok === true && Array.isArray(r.senses) && r.senses.every(isSenseInfo)) {
return { ok: true, senses: r.senses };
}
}
} catch {
// fall through
@@ -54,7 +68,7 @@ function parseListSensesResponse(line: string): ListSensesResponse {
*/
function sendAndReceive<T>(
socketPath: string,
message: object,
message: DaemonIpcRequest,
parseFirstLine: (trimmed: string) => T,
responseTimeoutMs: number = RESPONSE_TIMEOUT_MS,
): Promise<T> {
@@ -119,27 +133,35 @@ function sendAndReceive<T>(
export function triggerWorkflowViaDaemon(
socketPath: string,
workflow: string,
payload: unknown,
): Promise<TriggerResponse> {
return sendAndReceive(
socketPath,
{ type: "trigger-workflow", workflow, payload },
parseDaemonResponse,
);
prompt: string,
maxRounds: number,
): Promise<DaemonIpcTriggerResponse> {
const message: DaemonIpcRequest = {
type: "trigger-workflow",
workflow,
prompt,
maxRounds,
};
return sendAndReceive(socketPath, message, parseDaemonResponse);
}
/**
* Send a trigger-sense message to the running daemon via its Unix socket.
* Resolves with the daemon's response or rejects on connection/timeout errors.
*/
export function triggerSenseViaDaemon(socketPath: string, sense: string): Promise<TriggerResponse> {
return sendAndReceive(socketPath, { type: "trigger-sense", sense }, parseDaemonResponse);
export function triggerSenseViaDaemon(
socketPath: string,
sense: string,
): Promise<DaemonIpcTriggerResponse> {
const message: DaemonIpcRequest = { type: "trigger-sense", sense };
return sendAndReceive(socketPath, message, parseDaemonResponse);
}
/**
* Send a list-senses message to the running daemon via its Unix socket.
* Resolves with the list of registered senses or rejects on connection/timeout errors.
*/
export function listSensesViaDaemon(socketPath: string): Promise<ListSensesResponse> {
return sendAndReceive(socketPath, { type: "list-senses" }, parseListSensesResponse);
export function listSensesViaDaemon(socketPath: string): Promise<DaemonIpcListSensesResponse> {
const message: DaemonIpcRequest = { type: "list-senses" };
return sendAndReceive(socketPath, message, parseListSensesResponse);
}
-86
View File
@@ -1,86 +0,0 @@
/**
* Structural types for workflow CLI — mirrors @uncaged/nerve-daemon log-store
* public API so the CLI runtime does not statically depend on the daemon package.
*
* ⚠️ Keep in sync with @uncaged/nerve-daemon exports.
* Run `pnpm --filter @uncaged/nerve-cli test` to catch drift via satisfies assertions.
*/
export type WorkflowRunStatus =
| "queued"
| "started"
| "completed"
| "failed"
| "crashed"
| "dropped"
| "interrupted";
export type WorkflowRun = {
runId: string;
workflow: string;
status: WorkflowRunStatus;
ts: number;
};
export type LogEntry = {
id?: number;
source: string;
type: string;
refId: string | null;
payload: string | null;
ts: number;
};
export type LogQuery = {
source?: string;
type?: string;
refId?: string;
since?: number;
until?: number;
limit?: number;
};
export type ArchiveLogsOptions = {
now?: number;
vacuum?: boolean;
maxDays?: number;
retentionMs?: number;
};
export type ArchiveLogsDayResult = {
day: string;
rowCount: number;
filePath: string;
};
export type ArchiveLogsResult = {
days: ArchiveLogsDayResult[];
vacuumed: boolean;
};
/** One role round row — keep in sync with daemon `log-store` `ThreadRoundRow`. */
export type ThreadRoundRow = {
round: number;
logId: number;
ts: number;
message: { role: string; content: string; meta: unknown; timestamp: number };
};
/** Keep in sync with daemon `log-store` `GetThreadRoundsParams`. */
export type GetThreadRoundsParams = {
before: number;
limit: number;
};
/** Subset of daemon LogStore used by the CLI workflow commands. */
export type LogStore = {
query: (filter?: LogQuery) => LogEntry[];
getWorkflowRun: (runId: string) => WorkflowRun | null;
getActiveWorkflowRuns: (workflowName?: string) => WorkflowRun[];
getAllWorkflowRuns: (workflowName: string | null) => WorkflowRun[];
upsertWorkflowRun: (entry: Omit<LogEntry, "id">, run: WorkflowRun) => LogEntry;
getThreadRoundCount: (runId: string) => number;
getThreadRounds: (runId: string, params: GetThreadRoundsParams) => ThreadRoundRow[];
archiveLogs: (options?: ArchiveLogsOptions) => ArchiveLogsResult;
close: () => void;
};
+3 -2
View File
@@ -5,7 +5,7 @@ import { pathToFileURL } from "node:url";
import type { NerveConfig } from "@uncaged/nerve-core";
import type { LogStore } from "./daemon-types.js";
import type { LogStore } from "@uncaged/nerve-store";
export function getDaemonEntryPath(nerveRoot: string): string | undefined {
const pkgPath = join(nerveRoot, "node_modules", "@uncaged", "nerve-daemon", "package.json");
@@ -29,7 +29,7 @@ export function assertWorkspaceDaemonInstalled(nerveRoot: string): string {
return entry;
}
/** Loaded from ~/.uncaged-nerve/node_modules at runtime — keep types structural only. */
/** Loaded from ~/.uncaged-nerve/node_modules at runtime. */
export type DaemonModule = {
createKernel: (
config: NerveConfig,
@@ -46,5 +46,6 @@ export type DaemonModule = {
export async function loadDaemonModule(nerveRoot: string): Promise<DaemonModule> {
const entry = assertWorkspaceDaemonInstalled(nerveRoot);
const url = pathToFileURL(entry).href;
// Dynamic import return type is module-specific; narrow at this workspace boundary.
return import(url) as Promise<DaemonModule>;
}
+29 -3
View File
@@ -4,9 +4,12 @@ Shared types and configuration parser for the [nerve](../../README.md) observati
## What's Inside
- **Type definitions** — `Signal`, `SenseConfig`, `ReflexConfig`, `WorkflowConfig`, `NerveConfig`, and all related types
- **Config parser** — `parseNerveConfig(yaml)` validates and parses `nerve.yaml` into a typed `NerveConfig`
- **Result type** — `Result<T>` with `ok()` / `err()` helpers for explicit error handling (no thrown exceptions)
- **Type definitions** — `Signal`, `SenseConfig`, `SenseInfo`, `SenseReflexConfig`, `ReflexConfig` (sense-only), `WorkflowConfig`, `NerveConfig`, and related types
- **Config parser** — `parseNerveConfig(yaml)` validates and parses `nerve.yaml` into `NerveConfig` (rejects reflex entries that declare a `workflow` key; reflexes only schedule senses)
- **Sense → workflow routing** — `parseSenseWorkflowDirective`, `routeSenseComputeOutput`, and types `ParsedSenseWorkflowDirective`, `SenseComputeRoute`
- **Daemon IPC protocol** — request/response types (`DaemonIpcRequest`, `DaemonIpcResponse`, …) and `parseDaemonIpcRequest` for newline-delimited JSON on the CLI ↔ daemon socket
- **Workflow automaton types** — `START` / `END` sentinel constants, `WorkflowMessage`, `StartSignal`, `RoleSignal`, `Moderator`, `WorkflowDefinition`, `Role`, `SenseResult`, plus `DEFAULT_ENGINE_MAX_ROUNDS`
- **Result type** — `Result<T>` with `ok()` / `err()` helpers for explicit error handling (no thrown exceptions for parse paths)
## Usage
@@ -20,6 +23,29 @@ if (result.ok) {
}
```
### Sense return → signal vs workflow
```typescript
import { parseSenseWorkflowDirective, routeSenseComputeOutput } from "@uncaged/nerve-core";
const directive = parseSenseWorkflowDirective("my-workflow|8|Hello from sense");
if (directive.ok) {
console.log(directive.value.workflowName, directive.value.maxRounds, directive.value.prompt);
}
const route = routeSenseComputeOutput({
metric: 42,
workflow: "my-workflow|8|Run now",
});
if (route.kind === "launch") {
// engine starts workflow; no Signal to the bus for this return
console.log(route.launch);
} else {
// normal signal with payload
console.log(route.payload);
}
```
## Duration Format
Config fields like `throttle`, `timeout`, and `interval` accept human-readable durations:
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@uncaged/nerve-core",
"version": "0.3.0",
"version": "0.4.0",
"type": "module",
"main": "dist/index.js",
"files": ["dist"],
+6 -6
View File
@@ -193,7 +193,7 @@ reflexes:
expect(result.error.message).toMatch(/disk.*not found in senses/);
});
it("returns error when workflow reflex references a non-existent workflow", () => {
it("returns error when reflex uses unsupported workflow field", () => {
const yaml = `
senses:
cpu:
@@ -206,10 +206,10 @@ reflexes:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toMatch(/missing_wf.*not found in workflows/);
expect(result.error.message).toMatch(/workflow.*not supported/);
});
it("returns error when workflow reflex references non-existent workflow (with workflows defined)", () => {
it("returns error when reflex uses unsupported workflow field (with workflows defined)", () => {
const yaml = `
senses:
cpu:
@@ -226,7 +226,7 @@ workflows:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toMatch(/unknown.*not found in workflows/);
expect(result.error.message).toMatch(/workflow.*not supported/);
});
it("returns error for invalid throttle format", () => {
@@ -354,7 +354,7 @@ reflexes:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toMatch(/cannot have both/);
expect(result.error.message).toMatch(/workflow.*not supported/);
});
it("returns error when reflex has neither sense nor workflow", () => {
@@ -368,7 +368,7 @@ reflexes:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toMatch(/must have either/);
expect(result.error.message).toMatch(/must include "sense"/);
});
});
});
@@ -0,0 +1,51 @@
import { describe, expect, it } from "vitest";
import { parseDaemonIpcRequest } from "../daemon-ipc-protocol.js";
describe("parseDaemonIpcRequest", () => {
it("parses trigger-workflow", () => {
expect(
parseDaemonIpcRequest(
JSON.stringify({
type: "trigger-workflow",
workflow: "wf",
prompt: "go",
maxRounds: 3,
}),
),
).toEqual({
type: "trigger-workflow",
workflow: "wf",
prompt: "go",
maxRounds: 3,
});
});
it("rejects trigger-workflow with empty workflow", () => {
expect(
parseDaemonIpcRequest(
JSON.stringify({
type: "trigger-workflow",
workflow: "",
prompt: "",
maxRounds: 1,
}),
),
).toBeNull();
});
it("parses trigger-sense and list-senses", () => {
expect(parseDaemonIpcRequest(JSON.stringify({ type: "trigger-sense", sense: "x" }))).toEqual({
type: "trigger-sense",
sense: "x",
});
expect(parseDaemonIpcRequest(JSON.stringify({ type: "list-senses" }))).toEqual({
type: "list-senses",
});
});
it("returns null for invalid JSON or unknown type", () => {
expect(parseDaemonIpcRequest("not json")).toBeNull();
expect(parseDaemonIpcRequest(JSON.stringify({ type: "nope" }))).toBeNull();
});
});
+16 -16
View File
@@ -1,10 +1,10 @@
import { parse } from "yaml";
import { isPlainRecord } from "./is-plain-record.js";
import type { Result } from "./result.js";
import { err, ok } from "./result.js";
import type { NerveConfig, ReflexConfig, SenseConfig, WorkflowConfig } from "./types.js";
const DEFAULT_ENGINE_MAX_ROUNDS = 100;
import { DEFAULT_ENGINE_MAX_ROUNDS } from "./types.js";
const DURATION_RE = /^(\d+)([smh])$/;
@@ -41,11 +41,11 @@ function parseDurationField(field: unknown, label: string): Result<number | null
}
function validateSenseConfig(name: string, raw: unknown): Result<SenseConfig> {
if (raw === null || typeof raw !== "object" || Array.isArray(raw)) {
if (!isPlainRecord(raw)) {
return err(new Error(`senses.${name}: must be an object`));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
if (typeof obj.group !== "string" || obj.group.trim() === "") {
return err(new Error(`senses.${name}.group: required string`));
@@ -78,10 +78,10 @@ function validateSenseConfig(name: string, raw: unknown): Result<SenseConfig> {
function parseOnField(index: number, obj: Record<string, unknown>): Result<string[] | null> {
if (obj.on === undefined || obj.on === null) return ok(null);
if (!Array.isArray(obj.on) || !obj.on.every((item) => typeof item === "string")) {
if (!Array.isArray(obj.on) || !obj.on.every((item): item is string => typeof item === "string")) {
return err(new Error(`reflexes[${index}].on: must be an array of strings`));
}
return ok(obj.on as string[]);
return ok(obj.on);
}
function parseSenseReflex(
@@ -119,11 +119,11 @@ function validateReflexConfig(
raw: unknown,
senseNames: Set<string>,
): Result<ReflexConfig> {
if (raw === null || typeof raw !== "object" || Array.isArray(raw)) {
if (!isPlainRecord(raw)) {
return err(new Error(`reflexes[${index}]: must be an object`));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
const hasSense = obj.sense !== undefined;
const hasWorkflowKey = Object.hasOwn(obj, "workflow");
@@ -159,11 +159,11 @@ function parseEngineMaxRounds(obj: Record<string, unknown>): Result<number> {
}
function validateWorkflowConfig(name: string, raw: unknown): Result<WorkflowConfig> {
if (raw === null || typeof raw !== "object" || Array.isArray(raw)) {
if (!isPlainRecord(raw)) {
return err(new Error(`workflows.${name}: must be an object`));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
if (
typeof obj.concurrency !== "number" ||
@@ -210,11 +210,11 @@ function validateWorkflowConfig(name: string, raw: unknown): Result<WorkflowConf
function parseSenses(
obj: Record<string, unknown>,
): Result<{ senses: Record<string, SenseConfig>; senseNames: Set<string> }> {
if (obj.senses === null || typeof obj.senses !== "object" || Array.isArray(obj.senses)) {
if (!isPlainRecord(obj.senses)) {
return err(new Error("senses: required object"));
}
const sensesRaw = obj.senses as Record<string, unknown>;
const sensesRaw = obj.senses;
const senses: Record<string, SenseConfig> = {};
const senseNames = new Set(Object.keys(sensesRaw));
@@ -250,11 +250,11 @@ function parseWorkflows(
): Result<Record<string, WorkflowConfig> | null> {
if (obj.workflows === undefined || obj.workflows === null) return ok(null);
if (typeof obj.workflows !== "object" || Array.isArray(obj.workflows)) {
if (!isPlainRecord(obj.workflows)) {
return err(new Error("workflows: must be an object if provided"));
}
const workflowsRaw = obj.workflows as Record<string, unknown>;
const workflowsRaw = obj.workflows;
const workflows: Record<string, WorkflowConfig> = {};
for (const [name, wfRaw] of Object.entries(workflowsRaw)) {
@@ -276,11 +276,11 @@ export function parseNerveConfig(raw: string): Result<NerveConfig> {
return err(new Error(`YAML parse error: ${message}`));
}
if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
if (!isPlainRecord(parsed)) {
return err(new Error("Config must be a YAML object"));
}
const obj = parsed as Record<string, unknown>;
const obj = parsed;
const sensesResult = parseSenses(obj);
if (!sensesResult.ok) return sensesResult;
+85
View File
@@ -0,0 +1,85 @@
/**
* Daemon Unix-socket IPC protocol (CLI → daemon).
* Newline-delimited JSON: one request object per line from the client,
* one response object per line from the daemon.
*/
import { isPlainRecord } from "./is-plain-record.js";
import type { SenseInfo } from "./types.js";
/** Client → daemon: start a workflow run. */
export type DaemonIpcTriggerWorkflowRequest = {
type: "trigger-workflow";
workflow: string;
prompt: string;
maxRounds: number;
};
/** Client → daemon: run a sense compute on demand. */
export type DaemonIpcTriggerSenseRequest = {
type: "trigger-sense";
sense: string;
};
/** Client → daemon: list registered senses. */
export type DaemonIpcListSensesRequest = {
type: "list-senses";
};
/** Union of all JSON requests the daemon IPC server accepts. */
export type DaemonIpcRequest =
| DaemonIpcTriggerWorkflowRequest
| DaemonIpcTriggerSenseRequest
| DaemonIpcListSensesRequest;
/** Successful trigger / trigger-sense reply (no body). */
export type DaemonIpcTriggerOkResponse = { ok: true };
export type DaemonIpcErrorResponse = { ok: false; error: string };
/** Replies for trigger-workflow and trigger-sense. */
export type DaemonIpcTriggerResponse = DaemonIpcTriggerOkResponse | DaemonIpcErrorResponse;
/** Reply for list-senses. */
export type DaemonIpcListSensesResponse =
| { ok: true; senses: SenseInfo[] }
| DaemonIpcErrorResponse;
/** Any JSON response the daemon may write on the IPC socket. */
export type DaemonIpcResponse =
| DaemonIpcTriggerOkResponse
| DaemonIpcErrorResponse
| { ok: true; senses: SenseInfo[] };
/**
* Parse a single line of JSON into a {@link DaemonIpcRequest}, or null if invalid.
* Kept in core with the request types so CLI and daemon stay aligned at compile time.
*/
export function parseDaemonIpcRequest(line: string): DaemonIpcRequest | null {
try {
const obj: unknown = JSON.parse(line);
if (!isPlainRecord(obj)) return null;
const req = obj;
if (req.type === "trigger-workflow") {
if (typeof req.workflow !== "string" || req.workflow.length === 0) return null;
if (typeof req.prompt !== "string") return null;
if (typeof req.maxRounds !== "number") return null;
return {
type: "trigger-workflow",
workflow: req.workflow,
prompt: req.prompt,
maxRounds: req.maxRounds,
};
}
if (req.type === "trigger-sense") {
if (typeof req.sense !== "string" || req.sense.length === 0) return null;
return { type: "trigger-sense", sense: req.sense };
}
if (req.type === "list-senses") {
return { type: "list-senses" };
}
return null;
} catch {
return null;
}
}
+22 -9
View File
@@ -18,17 +18,30 @@ export type {
WorkflowDefinition,
SenseResult,
} from "./types.js";
export { START, END } from "./types.js";
export { START, END, DEFAULT_ENGINE_MAX_ROUNDS } from "./types.js";
export type { Result } from "./result.js";
export { ok, err } from "./result.js";
export { parseNerveConfig } from "./config.js";
export { isPlainRecord } from "./is-plain-record.js";
export function parseWorkflowField(field: string): { name: string; maxRounds: number; prompt: string } {
const [name, rounds, ...rest] = field.split("|");
const prompt = rest.join("|");
const maxRounds = parseInt(rounds, 10);
return { name: name ?? "", maxRounds, prompt };
}
export type {
ParsedSenseWorkflowDirective,
SenseComputeRoute,
} from "./sense-workflow-directive.js";
export {
parseSenseWorkflowDirective,
routeSenseComputeOutput,
} from "./sense-workflow-directive.js";
export type { ParsedSenseWorkflowDirective, SenseComputeRoute } from "./sense-workflow-directive.js";
export { parseSenseWorkflowDirective, routeSenseComputeOutput } from "./sense-workflow-directive.js";
export type {
DaemonIpcTriggerWorkflowRequest,
DaemonIpcTriggerSenseRequest,
DaemonIpcListSensesRequest,
DaemonIpcRequest,
DaemonIpcTriggerOkResponse,
DaemonIpcErrorResponse,
DaemonIpcTriggerResponse,
DaemonIpcListSensesResponse,
DaemonIpcResponse,
} from "./daemon-ipc-protocol.js";
export { parseDaemonIpcRequest } from "./daemon-ipc-protocol.js";
+7
View File
@@ -0,0 +1,7 @@
/**
* Narrows `unknown` to a plain JSON-style object (not null, not array).
* Use after `JSON.parse` / YAML / IPC when validating structure field-by-field.
*/
export function isPlainRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
@@ -1,3 +1,4 @@
import { isPlainRecord } from "./is-plain-record.js";
import type { Result } from "./result.js";
import { err, ok } from "./result.js";
@@ -54,10 +55,10 @@ function stripWorkflowKey(payload: Record<string, unknown>): Record<string, unkn
* - `workflow: "name|n|prompt"` → launch workflow; no Signal is emitted to the bus
*/
export function routeSenseComputeOutput(payload: unknown): SenseComputeRoute {
if (payload === null || typeof payload !== "object" || Array.isArray(payload)) {
if (!isPlainRecord(payload)) {
return { kind: "signal", payload };
}
const obj = payload as Record<string, unknown>;
const obj = payload;
if (!Object.hasOwn(obj, "workflow")) {
return { kind: "signal", payload };
}
+3
View File
@@ -61,6 +61,9 @@ export const END = "__end__" as const;
export type START = typeof START;
export type END = typeof END;
/** Engine-wide fallback for max moderator rounds when not specified in config. */
export const DEFAULT_ENGINE_MAX_ROUNDS = 100;
/** A single message in the workflow conversation chain (runtime, type-erased). */
export type WorkflowMessage = {
role: string;
+53 -18
View File
@@ -4,18 +4,33 @@ The observation engine runtime for [nerve](../../README.md) — runs senses, rou
## Architecture
| Module | Responsibility |
|--------|---------------|
| **Kernel** | Top-level orchestrator — spawns workers, wires up signal bus, scheduler, and workflow manager. Supports hot reload and graceful shutdown. |
| **Sense Runtime** | Per-sense SQLite database (via `node:sqlite` + Drizzle ORM), migration runner, peer DB read access. |
| **Sense Worker** | Forked child process — one per sense group. Runs compute functions in isolation. |
| **Signal Bus** | In-memory pub/sub. Sense computes emit signals; reflexes and workflows subscribe. |
| **Reflex Scheduler** | Drives compute triggers — interval timers, signal-based events, throttle/coalesce logic. |
| **Workflow Manager** | Concurrency control (drop/queue), thread lifecycle, worker process management (RFC-002). |
| **Log Store** | Structured log storage in WAL-mode SQLite. Supports retention policies, archival to JSONL, and workflow run tracking. |
| **Blob Store** | Binary artifact storage for workflow outputs. |
| **File Watcher** | Watches `nerve.yaml` and sense files for hot reload. |
| **Daemon IPC** | Unix socket server for CLI ↔ daemon communication. |
| Module | Source (indicative) | Responsibility |
|--------|---------------------|----------------|
| **Kernel** | `kernel.ts` | Orchestrator — worker pool, signal bus, reflex scheduler, workflow manager, optional file watcher and daemon IPC, config reload hooks |
| **Worker pool** | `worker-pool.ts` | Fork and supervise one child process per sense group; restart/shutdown; crash cleanup hooks for scheduler state |
| **Kernel sense groups** | `kernel-sense-groups.ts` | Derive sense groups from config; list senses per group for scheduling |
| **Sense runtime** | sense worker + Drizzle | Per-sense SQLite (`node:sqlite`), migrations, peer DB reads |
| **Sense worker** | `sense-worker.ts` (fork target) | Child process entry — runs `compute()` per sense in a group |
| **Signal bus** | `signal-bus.ts` | In-memory pub/sub for sense signals |
| **Reflex scheduler** | `reflex-scheduler.ts` | Interval + `on` subscriptions, throttle/coalesce |
| **Workflow manager** | `workflow-manager.ts` | One worker per workflow name, concurrency (drop/queue), queue caps |
| **Workflow worker** | `workflow-worker.ts` | Child process — runs RFC-002 threads (`start-thread`, `resume-thread` IPC) |
| **IPC (parent ↔ workers)** | `ipc.ts` | Typed messages for sense and workflow workers (includes `resume-thread` for recovery) |
| **Log / workflow persistence** | via `@uncaged/nerve-store` | Structured logs, `workflow_runs`, thread messages (used for recovery) |
| **Blob store** | `@uncaged/nerve-store` | CAS under `data/blobs/` — sense workers construct `createBlobStore(join(nerveRoot, "data", "blobs"))` for artifact writes |
| **File watcher** | `file-watcher.ts` | Watches workspace paths for config / sense / workflow file changes |
| **Kernel file watch** | `kernel-file-watch.ts` | Maps watcher events to `reloadConfig`, sense group restart, workflow `drainAndRespawn` |
| **Daemon IPC** | `daemon-ipc.ts` | Unix socket server — parses `@uncaged/nerve-core` `DaemonIpcRequest`, dispatches trigger-workflow / trigger-sense / list-senses |
## Crash recovery (workflow workers)
If a workflow worker exits unexpectedly while threads are active:
- In-flight runs are marked **`crashed`** in the log store; the manager respawns a fresh worker.
- Runs still in **`started`** state can be **`resume-thread`**’d: the manager rebuilds the message chain from persisted workflow log rows and sends `resume-thread` to the new worker.
- **Crash-loop backoff:** repeated crashes for the same workflow name are counted in a sliding window (`60s`); after **`5`** crashes in that window, the manager **stops respawning** that worker and logs the condition (avoids tight crash loops).
Hot reload (`drainAndRespawn`) uses a controlled drain: in-flight runs may be marked **`interrupted`** when the old worker is torn down after a timeout — that path is distinct from unexpected crash recovery.
## Key Design Decisions
@@ -26,24 +41,44 @@ The observation engine runtime for [nerve](../../README.md) — runs senses, rou
## Usage
The daemon is typically started via the CLI (`nerve daemon start`), but can be used programmatically:
The daemon is typically started via the CLI (`nerve daemon start` / `nerve dev`), but you can embed the kernel:
```typescript
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { parseNerveConfig } from "@uncaged/nerve-core";
import { createKernel } from "@uncaged/nerve-daemon";
const kernel = await createKernel(nerveRoot);
const nerveRoot = "/path/to/workspace";
const yamlPath = join(nerveRoot, "nerve.yaml");
const parsed = parseNerveConfig(readFileSync(yamlPath, "utf8"));
if (!parsed.ok) {
throw parsed.error;
}
const kernel = createKernel(parsed.value, nerveRoot, {
enableFileWatcher: true,
ipcSocketPath: join(nerveRoot, "nerve.sock"),
});
await kernel.ready;
// Trigger a sense manually
kernel.triggerSense("cpu-usage");
// Check health
const health = kernel.getHealth();
// Graceful shutdown
await kernel.stop();
```
`createKernel(config, nerveRoot, options?)``config` is a parsed `NerveConfig`; `nerveRoot` is the workspace root (contains `nerve.yaml`, `data/`, etc.). Optional `KernelOptions`:
| Field | Meaning |
|-------|---------|
| `workerScript` | Override path to the sense worker entry script (defaults to the package’s resolved worker) |
| `enableFileWatcher` | Watch config / senses / workflows for hot reload |
| `logStore` | Inject a `LogStore` instance (defaults to `createLogStore(join(nerveRoot, "data", "logs.db"))`) |
| `ipcSocketPath` | When non-null, listen for daemon IPC on this Unix socket path |
## Install
```bash
+2 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@uncaged/nerve-daemon",
"version": "0.3.0",
"version": "0.4.0",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
@@ -15,6 +15,7 @@
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*",
"@uncaged/nerve-store": "workspace:*",
"drizzle-orm": "1.0.0-beta.23-c10d10c",
"yaml": "^2.8.3"
},
@@ -235,7 +235,6 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
expect(resumeCalls[0][0]).toMatchObject({
type: "resume-thread",
runId: "run-started-1",
triggerPayload: { trigger: "initial" },
});
expect(Array.isArray((resumeCalls[0][0] as Record<string, unknown>).messages)).toBe(true);
@@ -318,8 +317,8 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
const payload = { prompt: "build-docker for myrepo", maxRounds: 10 };
mgr.startWorkflow("my-wf", payload);
const launch = { prompt: "build-docker for myrepo", maxRounds: 10 };
mgr.startWorkflow("my-wf", launch);
const startedCall = logStore.upsertWorkflowRun.mock.calls.find(
(args: any[]) => (args[0] as { type: string }).type === "started",
@@ -328,7 +327,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
const logEntry = startedCall?.[0] as { payload: string | null };
expect(logEntry.payload).not.toBeNull();
const parsed = JSON.parse(logEntry.payload as string) as Record<string, unknown>;
expect(parsed.triggerPayload).toMatchObject(payload);
expect(parsed).toMatchObject({ prompt: "build-docker for myrepo", maxRounds: 10 });
const stopPromise = mgr.stop();
await vi.runAllTimersAsync();
@@ -2,7 +2,7 @@
* Unit + integration tests for daemon-ipc.ts — trigger-sense request type.
*
* Tests cover:
* - parseRequest correctly accepts/rejects trigger-sense messages
* - parseDaemonIpcRequest (core) / server correctly accept or reject trigger-sense messages
* - createDaemonIpcServer routes trigger-sense to opts.triggerSense
* - Error response when triggerSense throws (unknown sense)
* - Success response on valid sense trigger
@@ -152,12 +152,16 @@ describe("daemon-ipc — trigger-sense", () => {
const resp = await sendRaw(sockPath, {
type: "trigger-workflow",
workflow: "my-workflow",
payload: {},
prompt: "test prompt",
maxRounds: 10,
});
expect(resp).toEqual({ ok: true });
expect(triggerSense).not.toHaveBeenCalled();
expect(wfManager.startWorkflow).toHaveBeenCalledWith("my-workflow", {});
expect(wfManager.startWorkflow).toHaveBeenCalledWith("my-workflow", {
prompt: "test prompt",
maxRounds: 10,
});
});
it("responds ok:false for completely unknown request type", async () => {
@@ -304,7 +304,7 @@ describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
senses: {},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -181,7 +181,7 @@ describe("kernel — reloadConfig", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
});
expect(kernel.groups.has("network")).toBe(true);
@@ -198,7 +198,7 @@ describe("kernel — reloadConfig", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
const kernel = createKernel(config, "/tmp/nerve-test");
@@ -213,7 +213,7 @@ describe("kernel — reloadConfig", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
});
expect(kernel.groups.has("network")).toBe(false);
@@ -236,7 +236,7 @@ describe("kernel — reloadConfig", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
});
expect(kernel.getHealth().activeSenses).toBe(2);
@@ -116,14 +116,14 @@ describe("kernel + workflowManager integration", () => {
vi.clearAllMocks();
});
describe("sense signal triggers workflow via reflex", () => {
it("calls workflowManager.startWorkflow when a sense signal fires on a workflow reflex", async () => {
describe("sense compute triggers workflow via return value", () => {
it("calls workflowManager.startWorkflow when a sense compute returns a workflow launch", async () => {
const logStore = makeLogStore();
const config = makeConfig({
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "my-workflow", on: ["cpu-usage"] } as any],
reflexes: [],
workflows: { "my-workflow": { concurrency: 2, overflow: "drop" } },
});
@@ -132,14 +132,20 @@ describe("kernel + workflowManager integration", () => {
logStore,
});
// Emit a signal from "cpu-usage" on the bus
const { createSignalBus } = await import("../signal-bus.js");
void createSignalBus; // ensure import resolves
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload: { value: 80 }, ts: Date.now() });
// Simulate a sense worker sending a signal with workflow launch payload
// The kernel's handleWorkerMessage processes "signal" type messages
// and uses routeSenseComputeOutput to detect workflow launches
const workerPool = mockChildren[0];
if (workerPool) {
// Simulate the worker sending a signal message with workflow field
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "my-workflow|10|run this workflow" },
});
}
// The workflow worker should be spawned (one for the sense group, one for workflow)
// The sense group worker is mockChildren[0]; the workflow worker is mockChildren[1]
// We need to check that a start-thread message was sent to the workflow worker
// A workflow worker should be spawned and a start-thread message sent
const workflowWorker = mockChildren.find((c) =>
(c.send as ReturnType<typeof vi.fn>).mock.calls.some(
(args: unknown[]) =>
@@ -155,13 +161,13 @@ describe("kernel + workflowManager integration", () => {
await stopPromise;
});
it("passes the signal payload as triggerPayload to the workflow", async () => {
it("passes prompt and maxRounds from the workflow field to the workflow", async () => {
const logStore = makeLogStore();
const config = makeConfig({
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "alert-workflow", on: ["cpu-usage"] } as any],
reflexes: [],
workflows: { "alert-workflow": { concurrency: 1, overflow: "drop" } },
});
@@ -170,8 +176,15 @@ describe("kernel + workflowManager integration", () => {
logStore,
});
const payload = { level: "critical", value: 99 };
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload, ts: Date.now() });
// Simulate sense worker returning a workflow launch
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "alert-workflow|5|handle critical alert" },
});
}
// Find the start-thread call and verify triggerPayload
const startThreadCall = mockChildren
@@ -187,7 +200,8 @@ describe("kernel + workflowManager integration", () => {
expect(startThreadCall?.[0]).toMatchObject({
type: "start-thread",
workflow: "alert-workflow",
triggerPayload: payload,
prompt: "handle critical alert",
maxRounds: 5,
});
const stopPromise = kernel.stop();
@@ -202,7 +216,7 @@ describe("kernel + workflowManager integration", () => {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
"disk-io": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "my-workflow", on: ["disk-io"] } as any],
reflexes: [],
workflows: { "my-workflow": { concurrency: 1, overflow: "drop" } },
});
@@ -211,10 +225,17 @@ describe("kernel + workflowManager integration", () => {
logStore,
});
// Emit signal from cpu-usage — NOT in the workflow's "on" list
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload: 50, ts: Date.now() });
// Emit a regular signal (no workflow field) — should NOT trigger any workflow
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: 50,
});
}
// No workflow worker should have been spawned (only the sense group worker)
// No workflow should have been started
const workflowWorkerSpawned = mockChildren.some((c) =>
(c.send as ReturnType<typeof vi.fn>).mock.calls.some(
(args: unknown[]) =>
@@ -232,13 +253,13 @@ describe("kernel + workflowManager integration", () => {
});
describe("workflow events are logged", () => {
it("logs a 'started' event when workflow thread is triggered", async () => {
it("logs a 'started' event when workflow thread is triggered via sense compute", async () => {
const logStore = makeLogStore();
const config = makeConfig({
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "log-test-workflow", on: ["cpu-usage"] } as any],
reflexes: [],
workflows: { "log-test-workflow": { concurrency: 2, overflow: "drop" } },
});
@@ -247,7 +268,15 @@ describe("kernel + workflowManager integration", () => {
logStore,
});
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload: null, ts: Date.now() });
// Simulate sense compute returning a workflow launch
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "log-test-workflow|10|test prompt" },
});
}
expect(logStore.upsertWorkflowRun).toHaveBeenCalledWith(
expect.objectContaining({ source: "workflow", type: "started" }),
@@ -261,7 +290,7 @@ describe("kernel + workflowManager integration", () => {
});
describe("reloadConfig handles workflow changes", () => {
it("new workflow reflexes are active after reloadConfig", async () => {
it("new workflows are available after reloadConfig", async () => {
const logStore = makeLogStore();
const initialConfig = makeConfig({
senses: {
@@ -269,7 +298,7 @@ describe("kernel + workflowManager integration", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
});
const kernel = createKernel(initialConfig, "/tmp/nerve-test", {
@@ -277,19 +306,26 @@ describe("kernel + workflowManager integration", () => {
logStore,
});
// Reload with a workflow reflex added
// Reload with a workflow added
const newConfig: NerveConfig = {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "new-workflow", on: ["cpu-usage"] } as any],
reflexes: [],
workflows: { "new-workflow": { concurrency: 1, overflow: "drop" } },
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
// Now emit a signal — should trigger the new workflow
kernel.bus.emit({ id: 2, senseId: "cpu-usage", payload: "reload-test", ts: Date.now() });
// Simulate sense compute returning a workflow launch for the new workflow
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "new-workflow|10|reload test" },
});
}
const startThreadCall = mockChildren
.flatMap((c) => (c.send as ReturnType<typeof vi.fn>).mock.calls as [unknown][])
@@ -308,13 +344,13 @@ describe("kernel + workflowManager integration", () => {
await stopPromise;
});
it("old workflow reflexes are removed after reloadConfig", async () => {
it("old workflows are removed after reloadConfig", async () => {
const logStore = makeLogStore();
const initialConfig = makeConfig({
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "old-workflow", on: ["cpu-usage"] } as any],
reflexes: [],
workflows: { "old-workflow": { concurrency: 1, overflow: "drop" } },
});
@@ -323,14 +359,14 @@ describe("kernel + workflowManager integration", () => {
logStore,
});
// Reload with the workflow reflex removed
// Reload with the workflow removed
const newConfig: NerveConfig = {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -339,8 +375,15 @@ describe("kernel + workflowManager integration", () => {
(c.send as ReturnType<typeof vi.fn>).mockClear();
}
// Emit a signal — old-workflow should NOT be triggered
kernel.bus.emit({ id: 3, senseId: "cpu-usage", payload: "after-reload", ts: Date.now() });
// Simulate sense compute trying to launch the old workflow — it should still not start
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "old-workflow|10|should not work" },
});
}
const startThreadCall = mockChildren
.flatMap((c) => (c.send as ReturnType<typeof vi.fn>).mock.calls as [unknown][])
@@ -366,7 +409,7 @@ describe("kernel + workflowManager integration", () => {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "shutdown-test", on: ["cpu-usage"] } as any],
reflexes: [],
workflows: { "shutdown-test": { concurrency: 1, overflow: "drop" } },
});
@@ -375,8 +418,15 @@ describe("kernel + workflowManager integration", () => {
logStore,
});
// Trigger a workflow so a worker is spawned
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload: null, ts: Date.now() });
// Trigger a workflow via sense compute return value
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "shutdown-test|10|test" },
});
}
const stopPromise = kernel.stop();
await vi.runAllTimersAsync();
@@ -408,7 +458,7 @@ describe("kernel + workflowManager integration", () => {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "health-wf", on: ["cpu-usage"] } as any],
reflexes: [],
workflows: { "health-wf": { concurrency: 2, overflow: "drop" } },
});
+2 -2
View File
@@ -47,7 +47,7 @@ vi.mock("node:child_process", () => ({
// Import after mock is set up
const { createKernel } = await import("../kernel.js");
const { createLogStore } = await import("../log-store.js");
const { createLogStore } = await import("@uncaged/nerve-store");
// ---------------------------------------------------------------------------
// Helpers
@@ -201,7 +201,7 @@ describe("kernel — groupForSense mapping", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
const kernel = createKernel(config, "/tmp/nerve-test");
@@ -4,8 +4,8 @@ import { join } from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { NerveConfig, Signal } from "@uncaged/nerve-core";
import { createLogStore } from "../log-store.js";
import type { LogStore } from "../log-store.js";
import { createLogStore } from "@uncaged/nerve-store";
import type { LogStore } from "@uncaged/nerve-store";
import { createReflexScheduler } from "../reflex-scheduler.js";
import { createSignalBus } from "../signal-bus.js";
@@ -30,7 +30,7 @@ describe("LogStore + ReflexScheduler integration", () => {
},
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: null, on: ["cpu-usage"] }],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
const bus = createSignalBus();
const triggered: string[] = [];
@@ -58,7 +58,7 @@ describe("LogStore + ReflexScheduler integration", () => {
},
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 1000, on: null }],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
const bus = createSignalBus();
const ref: { scheduler: ReturnType<typeof createReflexScheduler> | null } = { scheduler: null };
@@ -89,7 +89,7 @@ describe("LogStore + ReflexScheduler integration", () => {
},
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: null, on: ["cpu-usage"] }],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
const bus = createSignalBus();
const triggered: string[] = [];
@@ -137,7 +137,7 @@ describe("phase6 — reloadConfig", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -157,7 +157,7 @@ describe("phase6 — reloadConfig", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
workerScript: MOCK_WORKER,
@@ -172,7 +172,7 @@ describe("phase6 — reloadConfig", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -203,7 +203,7 @@ describe("phase6 — error isolation", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
@@ -307,7 +307,7 @@ describe("phase6 — getHealth", () => {
},
reflexes: [],
workflows: null,
maxRounds: 10,
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -7,7 +7,7 @@ import { drizzle } from "drizzle-orm/node-sqlite";
import { integer, real, sqliteTable } from "drizzle-orm/sqlite-core";
import { describe, expect, it } from "vitest";
import { createBlobStore } from "../blob-store.js";
import { createBlobStore } from "@uncaged/nerve-store";
import { parseParentMessage } from "../ipc.js";
import { executeCompute, openPeerDb, openSenseDb, runMigrations } from "../sense-runtime.js";
import type { ComputeFn, DrizzleDB, PeerMap, SenseRuntime } from "../sense-runtime.js";
@@ -0,0 +1,235 @@
import { EventEmitter } from "node:events";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const mockChildren: MockChild[] = [];
type MockChild = EventEmitter & {
send: ReturnType<typeof vi.fn>;
kill: ReturnType<typeof vi.fn>;
pid: number;
connected: boolean;
};
function makeMockChild(pid = 1): MockChild {
const child = new EventEmitter() as MockChild;
child.connected = true;
child.send = vi.fn((msg: unknown) => {
if (
msg !== null &&
typeof msg === "object" &&
(msg as Record<string, unknown>).type === "shutdown"
) {
child.connected = false;
setImmediate(() => child.emit("exit", 0, null));
}
});
child.kill = vi.fn((_signal?: string) => {
child.connected = false;
child.emit("exit", null, _signal ?? "SIGKILL");
});
child.pid = pid;
return child;
}
vi.mock("node:child_process", () => ({
fork: vi.fn((_script: string, _args: string[], _opts: unknown) => {
const child = makeMockChild(mockChildren.length + 1);
mockChildren.push(child);
return child;
}),
}));
const { createSenseWorkerPool } = await import("../worker-pool.js");
async function flushSetImmediate(): Promise<void> {
await new Promise<void>((resolve) => setImmediate(resolve));
}
async function startWorkerWithReady(
pool: ReturnType<typeof createSenseWorkerPool>,
group: string,
): Promise<void> {
const pr = pool.startWorker(group);
const child = mockChildren[mockChildren.length - 1];
child.emit("message", { type: "ready" });
await pr;
}
describe("createSenseWorkerPool", () => {
beforeEach(() => {
mockChildren.length = 0;
});
afterEach(() => {
vi.useRealTimers();
});
it("forks one child per startWorker and routes IPC to onWorkerMessage", async () => {
const onWorkerMessage = vi.fn();
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage,
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "g1");
expect(mockChildren).toHaveLength(1);
const child = mockChildren[0];
child.emit("message", { type: "signal", sense: "s", payload: 1 });
expect(onWorkerMessage).toHaveBeenCalledWith({ type: "signal", sense: "s", payload: 1 });
});
it("sendCompute delivers to the worker for that group", async () => {
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "sys");
const child = mockChildren[0];
pool.sendCompute("sys", "cpu");
expect(child.send).toHaveBeenCalledWith(
expect.objectContaining({ type: "compute", sense: "cpu" }),
);
});
it("hasWorkerForGroup and getWorkerPid reflect running workers", async () => {
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
expect(pool.hasWorkerForGroup("a")).toBe(false);
expect(pool.getWorkerPid("a")).toBeNull();
await startWorkerWithReady(pool, "a");
expect(pool.hasWorkerForGroup("a")).toBe(true);
expect(pool.getWorkerPid("a")).toBe(1);
expect(pool.activeGroupCount()).toBe(1);
});
it("evictGroup sends shutdown and removes the entry without waiting", async () => {
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "x");
expect(pool.activeGroupCount()).toBe(1);
pool.evictGroup("x");
expect(pool.hasWorkerForGroup("x")).toBe(false);
expect(mockChildren[0].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
});
it("restartGroup invokes onBeforeGroupRestart then respawns", async () => {
const onBeforeGroupRestart = vi.fn();
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => ["s1"],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart,
isStopped: () => false,
});
await startWorkerWithReady(pool, "g");
expect(mockChildren).toHaveLength(1);
const p = pool.restartGroup("g");
expect(onBeforeGroupRestart).toHaveBeenCalledWith("g");
expect(mockChildren[0].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
await flushSetImmediate();
expect(mockChildren).toHaveLength(2);
mockChildren[1].emit("message", { type: "ready" });
await p;
expect(pool.hasWorkerForGroup("g")).toBe(true);
});
it("onWorkerCrashed runs and schedules respawn after non-zero exit", async () => {
vi.useFakeTimers({ shouldAdvanceTime: true });
const onWorkerCrashed = vi.fn();
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: (g) => (g === "g" ? ["a", "b"] : []),
onWorkerCrashed,
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "g");
expect(mockChildren).toHaveLength(1);
mockChildren[0].emit("exit", 1, null);
expect(onWorkerCrashed).toHaveBeenCalledWith("g");
await vi.advanceTimersByTimeAsync(1000);
expect(mockChildren).toHaveLength(2);
});
it("shutdownAll sends shutdown to every worker", async () => {
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "a");
await startWorkerWithReady(pool, "b");
await pool.shutdownAll();
expect(mockChildren[0].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
expect(mockChildren[1].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
});
it("does not respawn after crash when isStopped is true", async () => {
vi.useFakeTimers({ shouldAdvanceTime: true });
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => true,
});
await startWorkerWithReady(pool, "g");
const n = mockChildren.length;
mockChildren[0].emit("exit", 1, null);
await vi.advanceTimersByTimeAsync(1000);
expect(mockChildren.length).toBe(n);
});
});
+17 -64
View File
@@ -2,77 +2,24 @@
* Daemon IPC server — listens on a Unix domain socket so that the CLI
* can send commands (e.g. trigger-workflow, trigger-sense) to the running daemon process.
*
* Protocol: newline-delimited JSON messages.
* Each request: { type: "trigger-workflow"; workflow: string; payload: unknown }
* | { type: "trigger-sense"; sense: string }
* | { type: "list-senses" }
* Each response: { ok: true } | { ok: false; error: string }
* | { ok: true; senses: SenseInfo[] } (for list-senses)
* Protocol: newline-delimited JSON — request/response types and
* `parseDaemonIpcRequest` live in `@uncaged/nerve-core`.
*/
import { rmSync } from "node:fs";
import { type Server, type Socket, createServer } from "node:net";
import type { SenseInfo } from "@uncaged/nerve-core";
import type { DaemonIpcResponse, SenseInfo } from "@uncaged/nerve-core";
import { parseDaemonIpcRequest } from "@uncaged/nerve-core";
import type { WorkflowManager } from "./workflow-manager.js";
export type { SenseInfo };
/** JSON message sent by the CLI to trigger a workflow. */
export type TriggerWorkflowRequest = {
type: "trigger-workflow";
workflow: string;
prompt: string;
maxRounds: number;
};
/** JSON message sent by the CLI to trigger a sense compute on-demand. */
export type TriggerSenseRequest = {
type: "trigger-sense";
sense: string;
};
/** JSON message sent by the CLI to list registered senses. */
export type ListSensesRequest = {
type: "list-senses";
};
type DaemonRequest = TriggerWorkflowRequest | TriggerSenseRequest | ListSensesRequest;
type DaemonResponse =
| { ok: true }
| { ok: false; error: string }
| { ok: true; senses: SenseInfo[] };
export type DaemonIpcServer = {
close: () => Promise<void>;
};
function parseRequest(line: string): DaemonRequest | null {
try {
const obj = JSON.parse(line) as unknown;
if (obj === null || typeof obj !== "object") return null;
const req = obj as Record<string, unknown>;
if (req.type === "trigger-workflow") {
if (typeof req.workflow !== "string" || req.workflow.length === 0) return null;
if (typeof req.prompt !== "string") return null;
if (typeof req.maxRounds !== "number") return null;
return { type: "trigger-workflow", workflow: req.workflow, prompt: req.prompt, maxRounds: req.maxRounds as number };
}
if (req.type === "trigger-sense") {
if (typeof req.sense !== "string" || req.sense.length === 0) return null;
return { type: "trigger-sense", sense: req.sense };
}
if (req.type === "list-senses") {
return { type: "list-senses" };
}
return null;
} catch {
return null;
}
}
export type DaemonIpcServerOptions = {
/** Called when a trigger-sense request arrives. Should throw if the sense is unknown. */
triggerSense: (senseName: string) => void;
@@ -96,30 +43,36 @@ export function createDaemonIpcServer(
const trimmed = line.trim();
if (trimmed.length === 0) return;
const req = parseRequest(trimmed);
const req = parseDaemonIpcRequest(trimmed);
if (req === null) {
const resp: DaemonResponse = { ok: false, error: "Invalid request" };
const resp: DaemonIpcResponse = { ok: false, error: "Invalid request" };
socket.write(`${JSON.stringify(resp)}\n`);
return;
}
try {
if (req.type === "trigger-workflow") {
workflowManager.startWorkflow(req.workflow, { prompt: req.prompt, maxRounds: req.maxRounds });
const resp: DaemonResponse = { ok: true };
workflowManager.startWorkflow(req.workflow, {
prompt: req.prompt,
maxRounds: req.maxRounds,
});
const resp: DaemonIpcResponse = { ok: true };
socket.write(`${JSON.stringify(resp)}\n`);
} else if (req.type === "trigger-sense") {
opts.triggerSense(req.sense);
const resp: DaemonResponse = { ok: true };
const resp: DaemonIpcResponse = { ok: true };
socket.write(`${JSON.stringify(resp)}\n`);
} else if (req.type === "list-senses") {
const senses = opts.listSenses();
const resp: DaemonResponse = { ok: true, senses };
const resp: DaemonIpcResponse = { ok: true, senses };
socket.write(`${JSON.stringify(resp)}\n`);
} else {
const _exhaustive: never = req;
void _exhaustive;
}
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
const resp: DaemonResponse = { ok: false, error: msg };
const resp: DaemonIpcResponse = { ok: false, error: msg };
socket.write(`${JSON.stringify(resp)}\n`);
}
}
+15 -12
View File
@@ -30,27 +30,30 @@ export {
export { createKernel } from "./kernel.js";
export type { Kernel, KernelOptions, KernelHealth } from "./kernel.js";
export type { SenseInfo } from "./daemon-ipc.js";
export type { SenseInfo } from "@uncaged/nerve-core";
export { createFileWatcher } from "./file-watcher.js";
export type { FileWatcher, FileChange, FileChangeHandler } from "./file-watcher.js";
export { createBlobStore, normalizeBlobHash } from "./blob-store.js";
export type { BlobStore } from "./blob-store.js";
export { createLogStore, LOG_ARCHIVE_META_KEY } from "./log-store.js";
export {
createBlobStore,
createLogStore,
LOG_ARCHIVE_META_KEY,
normalizeBlobHash,
} from "@uncaged/nerve-store";
export type {
LogStore,
LogEntry,
LogQuery,
WorkflowRun,
WorkflowRunStatus,
ArchiveLogsDayResult,
ArchiveLogsOptions,
ArchiveLogsResult,
ThreadRoundRow,
BlobStore,
GetThreadRoundsParams,
} from "./log-store.js";
LogEntry,
LogQuery,
LogStore,
ThreadRoundRow,
WorkflowRun,
WorkflowRunStatus,
} from "@uncaged/nerve-store";
export { createWorkflowManager } from "./workflow-manager.js";
export type { WorkflowManager } from "./workflow-manager.js";
+96 -43
View File
@@ -4,7 +4,7 @@
*/
import type { Result } from "@uncaged/nerve-core";
import { err, ok } from "@uncaged/nerve-core";
import { err, isPlainRecord, ok } from "@uncaged/nerve-core";
/** Parent → Worker: trigger one compute cycle for a sense */
export type ComputeMessage = {
@@ -148,76 +148,115 @@ function validateResumeThreadMsg(obj: Record<string, unknown>): string | null {
/** Validate and parse an unknown IPC message received from the parent process. */
export function parseParentMessage(raw: unknown): Result<ParentToWorkerMessage> {
if (raw === null || typeof raw !== "object") {
if (!isPlainRecord(raw)) {
return err(new Error("IPC message is not an object"));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
if (typeof obj.type !== "string") {
return err(new Error("IPC message missing string 'type' field"));
}
if (!PARENT_MSG_TYPES.has(obj.type)) {
return err(new Error(`Unknown IPC message type: "${obj.type}"`));
}
if (obj.type === "compute") {
if (typeof obj.sense !== "string") {
return err(new Error("IPC 'compute' message missing string 'sense' field"));
}
return ok({ type: "compute", sense: obj.sense });
}
if (obj.type === "shutdown") {
return ok({ type: "shutdown" });
}
if (obj.type === "health-request") {
return ok({ type: "health-request" });
}
if (obj.type === "start-thread") {
const errMsg = validateStartThreadMsg(obj);
if (errMsg !== null) return err(new Error(errMsg));
// Field types are validated above; `Record<string, unknown>` values stay `unknown` to TypeScript.
return ok({
type: "start-thread",
runId: obj.runId,
workflow: obj.workflow,
prompt: obj.prompt,
maxRounds: obj.maxRounds,
} as StartThreadMessage);
}
if (obj.type === "resume-thread") {
const errMsg = validateResumeThreadMsg(obj);
if (errMsg !== null) return err(new Error(errMsg));
// Elements are validated as plain objects by the kernel; trust the wire shape here.
return ok({
type: "resume-thread",
runId: obj.runId,
messages: obj.messages as ResumeThreadMessage["messages"],
maxRounds: obj.maxRounds,
} as ResumeThreadMessage);
}
return ok(raw as ParentToWorkerMessage);
return err(new Error(`Unhandled IPC message type: "${obj.type}"`));
}
function parseSignalMsg(obj: Record<string, unknown>, raw: unknown): Result<WorkerToParentMessage> {
function parseSignalMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (typeof obj.sense !== "string") {
return err(new Error("Worker 'signal' message missing string 'sense' field"));
}
if (!("payload" in obj)) {
return err(new Error("Worker 'signal' message missing 'payload' field"));
}
return ok(raw as SignalMessage);
return ok({
type: "signal",
sense: obj.sense,
payload: obj.payload,
});
}
function parseErrorMsg(obj: Record<string, unknown>, raw: unknown): Result<WorkerToParentMessage> {
function parseErrorMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (typeof obj.sense !== "string") {
return err(new Error("Worker 'error' message missing string 'sense' field"));
}
if (typeof obj.error !== "string") {
return err(new Error("Worker 'error' message missing string 'error' field"));
}
return ok(raw as ErrorMessage);
return ok({
type: "error",
sense: obj.sense,
error: obj.error,
});
}
function parseHealthResponseMsg(
obj: Record<string, unknown>,
raw: unknown,
): Result<WorkerToParentMessage> {
function parseHealthResponseMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (!Array.isArray(obj.senses)) {
return err(new Error("Worker 'health-response' message missing 'senses' array"));
}
if (typeof obj.inFlightCount !== "number") {
return err(new Error("Worker 'health-response' message missing 'inFlightCount' number"));
}
return ok(raw as HealthResponseMessage);
return ok({
type: "health-response",
// Kernel only sends string[] today; keep accepting any array elements without filtering.
senses: obj.senses as string[],
inFlightCount: obj.inFlightCount,
});
}
const THREAD_EVENT_TYPES = new Set<string>([
"queued",
"started",
"step_complete",
"completed",
"failed",
]);
function isThreadEventType(value: string): value is ThreadEventType {
switch (value) {
case "queued":
case "started":
case "step_complete":
case "completed":
case "failed":
return true;
default:
return false;
}
}
function parseThreadEventMsg(
obj: Record<string, unknown>,
raw: unknown,
): Result<WorkerToParentMessage> {
function parseThreadEventMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (typeof obj.runId !== "string") {
return err(new Error("Worker 'thread-event' message missing string 'runId' field"));
}
if (typeof obj.eventType !== "string" || !THREAD_EVENT_TYPES.has(obj.eventType)) {
if (typeof obj.eventType !== "string" || !isThreadEventType(obj.eventType)) {
return err(
new Error(`Worker 'thread-event' message has invalid 'eventType': "${obj.eventType}"`),
);
@@ -225,20 +264,26 @@ function parseThreadEventMsg(
if (!("payload" in obj)) {
return err(new Error("Worker 'thread-event' message missing 'payload' field"));
}
return ok(raw as ThreadEventMessage);
return ok({
type: "thread-event",
runId: obj.runId,
eventType: obj.eventType,
payload: obj.payload,
});
}
function parseWorkflowErrorMsg(
obj: Record<string, unknown>,
raw: unknown,
): Result<WorkerToParentMessage> {
function parseWorkflowErrorMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (typeof obj.runId !== "string") {
return err(new Error("Worker 'workflow-error' message missing string 'runId' field"));
}
if (typeof obj.error !== "string") {
return err(new Error("Worker 'workflow-error' message missing string 'error' field"));
}
return ok(raw as WorkflowErrorMessage);
return ok({
type: "workflow-error",
runId: obj.runId,
error: obj.error,
});
}
const WORKER_MSG_TYPES = new Set([
@@ -253,15 +298,14 @@ const WORKER_MSG_TYPES = new Set([
function parseThreadWorkflowMessageMsg(
obj: Record<string, unknown>,
raw: unknown,
): Result<WorkerToParentMessage> {
if (typeof obj.runId !== "string") {
return err(new Error("Worker 'thread-workflow-message' missing string 'runId' field"));
}
if (obj.message === null || typeof obj.message !== "object") {
if (!isPlainRecord(obj.message)) {
return err(new Error("Worker 'thread-workflow-message' missing object 'message' field"));
}
const msg = obj.message as Record<string, unknown>;
const msg = obj.message;
if (typeof msg.role !== "string") {
return err(new Error("Worker 'thread-workflow-message' message missing string 'role' field"));
}
@@ -275,26 +319,35 @@ function parseThreadWorkflowMessageMsg(
new Error("Worker 'thread-workflow-message' message missing number 'timestamp' field"),
);
}
return ok(raw as ThreadWorkflowMessageMessage);
return ok({
type: "thread-workflow-message",
runId: obj.runId,
message: {
role: msg.role,
content: msg.content,
meta: "meta" in msg ? msg.meta : undefined,
timestamp: msg.timestamp,
},
});
}
/** Validate and parse an unknown IPC message received from a worker process. */
export function parseWorkerMessage(raw: unknown): Result<WorkerToParentMessage> {
if (raw === null || typeof raw !== "object") {
if (!isPlainRecord(raw)) {
return err(new Error("Worker IPC message is not an object"));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
if (typeof obj.type !== "string") {
return err(new Error("Worker IPC message missing string 'type' field"));
}
if (!WORKER_MSG_TYPES.has(obj.type)) {
return err(new Error(`Unknown worker IPC message type: "${obj.type}"`));
}
if (obj.type === "signal") return parseSignalMsg(obj, raw);
if (obj.type === "error") return parseErrorMsg(obj, raw);
if (obj.type === "health-response") return parseHealthResponseMsg(obj, raw);
if (obj.type === "thread-event") return parseThreadEventMsg(obj, raw);
if (obj.type === "workflow-error") return parseWorkflowErrorMsg(obj, raw);
if (obj.type === "thread-workflow-message") return parseThreadWorkflowMessageMsg(obj, raw);
if (obj.type === "signal") return parseSignalMsg(obj);
if (obj.type === "error") return parseErrorMsg(obj);
if (obj.type === "health-response") return parseHealthResponseMsg(obj);
if (obj.type === "thread-event") return parseThreadEventMsg(obj);
if (obj.type === "workflow-error") return parseWorkflowErrorMsg(obj);
if (obj.type === "thread-workflow-message") return parseThreadWorkflowMessageMsg(obj);
return ok({ type: "ready" });
}
+92
View File
@@ -0,0 +1,92 @@
/**
* File-watcher callbacks for nerve.yaml / sense / workflow sources (hot reload wiring).
*/
import { readFileSync } from "node:fs";
import { join } from "node:path";
import type { NerveConfig } from "@uncaged/nerve-core";
import { parseNerveConfig } from "@uncaged/nerve-core";
import type { LogStore } from "@uncaged/nerve-store";
import type { WorkflowManager } from "./workflow-manager.js";
export type KernelFileWatchDeps = {
nerveRoot: string;
getConfig: () => NerveConfig;
logStore: LogStore;
workflowManager: WorkflowManager;
restartGroup: (group: string) => Promise<void>;
reloadConfig: (newConfig: NerveConfig) => void;
};
export type KernelFileWatchHandlers = {
onSenseFileChange: (senseName: string) => void;
onWorkflowFileChange: (workflowName: string) => void;
onConfigFileChange: () => void;
};
export function createKernelFileWatchHandlers(deps: KernelFileWatchDeps): KernelFileWatchHandlers {
function onSenseFileChange(senseName: string): void {
const sc = deps.getConfig().senses[senseName];
if (sc === undefined) return;
process.stderr.write(
`[kernel] sense file changed: "${senseName}", restarting group "${sc.group}"\n`,
);
deps.logStore.append({
source: "system",
type: "sense_reload",
refId: senseName,
payload: null,
ts: Date.now(),
});
deps.restartGroup(sc.group).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] restartGroup error: ${msg}\n`);
});
}
function onWorkflowFileChange(workflowName: string): void {
process.stderr.write(
`[kernel] workflow file changed: "${workflowName}", draining and respawning worker\n`,
);
deps.logStore.append({
source: "system",
type: "workflow_reload",
refId: workflowName,
payload: null,
ts: Date.now(),
});
deps.workflowManager.drainAndRespawn(workflowName).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] drainAndRespawn error for "${workflowName}": ${msg}\n`);
});
}
function onConfigFileChange(): void {
process.stderr.write("[kernel] nerve.yaml changed, reloading config\n");
deps.logStore.append({
source: "system",
type: "config_reload",
refId: null,
payload: null,
ts: Date.now(),
});
try {
const raw = readFileSync(join(deps.nerveRoot, "nerve.yaml"), "utf8");
const parseResult = parseNerveConfig(raw);
if (!parseResult.ok) {
process.stderr.write(
`[kernel] config parse error, keeping current config: ${parseResult.error.message}\n`,
);
return;
}
deps.reloadConfig(parseResult.value);
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] failed to read nerve.yaml, keeping current config: ${msg}\n`);
}
}
return { onSenseFileChange, onWorkflowFileChange, onConfigFileChange };
}
@@ -0,0 +1,29 @@
import type { NerveConfig } from "@uncaged/nerve-core";
export function groupForSense(config: NerveConfig, senseName: string): string | null {
const senseConfig = config.senses[senseName];
if (senseConfig === undefined) return null;
return senseConfig.group;
}
export function senseNamesInGroup(config: NerveConfig, group: string): string[] {
return Object.entries(config.senses)
.filter(([, sc]) => sc.group === group)
.map(([name]) => name);
}
export function collectSenseGroups(cfg: NerveConfig): Set<string> {
const result = new Set<string>();
for (const sc of Object.values(cfg.senses)) {
result.add(sc.group);
}
return result;
}
export function senseNamesInGroupAsSet(cfg: NerveConfig, group: string): Set<string> {
const result = new Set<string>();
for (const [name, sc] of Object.entries(cfg.senses)) {
if (sc.group === group) result.add(name);
}
return result;
}
+58 -294
View File
@@ -1,43 +1,32 @@
/**
* Kernel — the main orchestrator that ties sense workers, signal bus, and
* reflex scheduler together.
*
* Responsibilities:
* - Spawn one child process per sense group (via fork)
* - Route SignalMessage from workers → SignalBus
* - Route ErrorMessage from workers → stderr log
* - Drive compute triggers via ReflexScheduler
* - Graceful shutdown: stop scheduler, send shutdown to all workers
* - Hot reload: restartGroup, reloadConfig, file watcher integration
* - Health reporting: getHealth
* Kernel — ties sense workers, signal bus, reflex scheduler, workflow manager,
* optional file watcher, and daemon IPC.
*/
import { fork } from "node:child_process";
import type { ChildProcess } from "node:child_process";
import { readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { join } from "node:path";
import type { NerveConfig, SenseInfo, Signal } from "@uncaged/nerve-core";
import { parseNerveConfig, routeSenseComputeOutput } from "@uncaged/nerve-core";
import { routeSenseComputeOutput } from "@uncaged/nerve-core";
import { createLogStore } from "@uncaged/nerve-store";
import type { LogStore } from "@uncaged/nerve-store";
import { createDaemonIpcServer } from "./daemon-ipc.js";
import type { DaemonIpcServer } from "./daemon-ipc.js";
import { createFileWatcher } from "./file-watcher.js";
import type { FileWatcher } from "./file-watcher.js";
import type { ComputeMessage, ShutdownMessage } from "./ipc.js";
import { parseWorkerMessage } from "./ipc.js";
import { createLogStore } from "./log-store.js";
import type { LogStore } from "./log-store.js";
import { createKernelFileWatchHandlers } from "./kernel-file-watch.js";
import {
collectSenseGroups,
groupForSense,
senseNamesInGroup,
senseNamesInGroupAsSet,
} from "./kernel-sense-groups.js";
import { createReflexScheduler } from "./reflex-scheduler.js";
import type { ReflexScheduler } from "./reflex-scheduler.js";
import { createSignalBus } from "./signal-bus.js";
import type { SignalBus } from "./signal-bus.js";
import {
formatCapturedStderrTail,
formatChildExitSummary,
teeCapturedStderr,
} from "./worker-fork-support.js";
import { createSenseWorkerPool, resolveWorkerScript } from "./worker-pool.js";
import { createWorkflowManager } from "./workflow-manager.js";
import type { WorkflowManager } from "./workflow-manager.js";
@@ -57,93 +46,19 @@ export type Kernel = {
bus: SignalBus;
logStore: LogStore;
workflowManager: WorkflowManager;
/** Resolves when all workers have sent their initial "ready" message. */
ready: Promise<void>;
/** Returns the PID of the worker process for a given group, or null if not found. */
getWorkerPid: (group: string) => number | null;
/** Sends a compute message to the worker responsible for the given sense. */
triggerCompute: (senseName: string) => void;
/**
* On-demand sense trigger — looks up the group for `senseName`, finds its worker,
* and sends a compute message. Throws if the sense is unknown.
*/
triggerSense: (senseName: string) => void;
/** Gracefully restart a group worker (wait for exit, then respawn). */
restartGroup: (group: string) => Promise<void>;
/** Reload config from a new NerveConfig, incrementally updating scheduler and workers.
* Note: any pending/throttled computes in the old scheduler are silently dropped on reload.
* In-flight state is not preserved across reloadConfig. */
reloadConfig: (newConfig: NerveConfig) => void;
/** Return daemon health info. */
getHealth: () => KernelHealth;
};
type WorkerEntry = {
group: string;
process: ChildProcess;
};
function resolveWorkerScript(): string {
const __filename = fileURLToPath(import.meta.url);
const __dir = dirname(__filename);
return join(__dir, "sense-worker.js");
}
function spawnWorker(
nerveRoot: string,
group: string,
workerScript: string,
stderrTail: { value: string },
): ChildProcess {
const child = fork(workerScript, ["--group", group, "--root", nerveRoot], {
stdio: ["ignore", "inherit", "pipe", "ipc"],
});
teeCapturedStderr(child, stderrTail);
// Prevent unhandled EPIPE when writing to a child whose IPC channel closed
child.on("error", (err) => {
if ((err as NodeJS.ErrnoException).code !== "EPIPE") {
console.error("[worker] error:", err.message);
}
});
return child;
}
function sendCompute(worker: ChildProcess, senseName: string): void {
// worker.connected is false when the IPC channel has been closed (e.g. worker crashed)
if (worker.connected === false) return;
const msg: ComputeMessage = { type: "compute", sense: senseName };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function sendShutdown(worker: ChildProcess): void {
if (worker.connected === false) return;
const msg: ShutdownMessage = { type: "shutdown" };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function groupForSense(config: NerveConfig, senseName: string): string | null {
const senseConfig = config.senses[senseName];
if (senseConfig === undefined) return null;
return senseConfig.group;
}
export type KernelOptions = {
workerScript?: string | null;
enableFileWatcher?: boolean;
/** Override the LogStore instance (useful for testing). */
logStore?: LogStore;
/**
* Unix socket path for the daemon IPC server (used by CLI to send trigger-workflow).
* When null, the IPC server is not started (e.g. during tests).
*/
ipcSocketPath?: string | null;
};
@@ -184,9 +99,9 @@ export function createKernel(
groups.add(senseConfig.group);
}
const workers = new Map<string, WorkerEntry>();
let stopped = false;
let scheduler: ReflexScheduler = null as unknown as ReflexScheduler;
/** Assigned before workers start; `handleWorkerMessage` only runs after this is set. */
let scheduler!: ReflexScheduler;
let readyResolve: (() => void) | undefined;
const ready = new Promise<void>((resolve) => {
@@ -194,10 +109,10 @@ export function createKernel(
});
let pendingReadyCount = groups.size > 0 ? groups.size : 0;
function sensesForGroup(group: string): string[] {
return Object.entries(config.senses)
.filter(([, sc]) => sc.group === group)
.map(([name]) => name);
function clearSchedulerForGroup(group: string): void {
for (const senseName of senseNamesInGroup(config, group)) {
scheduler.onComputeComplete(senseName);
}
}
function handleWorkerMessage(raw: unknown): void {
@@ -259,50 +174,17 @@ export function createKernel(
}
scheduler.onComputeComplete(msg.sense);
}
// health-response is handled externally by the caller; no action needed here
}
function startWorker(group: string): Promise<void> {
const stderrTail = { value: "" };
const child = spawnWorker(nerveRoot, group, workerScript, stderrTail);
let workerReadyResolve: (() => void) | undefined;
const workerReady = new Promise<void>((resolve) => {
workerReadyResolve = resolve;
});
child.on("message", (raw: unknown) => {
const result = parseWorkerMessage(raw);
if (result.ok && result.value.type === "ready") {
workerReadyResolve?.();
}
handleWorkerMessage(raw);
});
child.on("exit", (code, signal) => {
const summary = formatChildExitSummary(code, signal ?? null);
process.stderr.write(
`[kernel] worker for group "${group}" exited (${summary})${formatCapturedStderrTail(stderrTail.value)}\n`,
);
// Resolve ready in case the worker exits before sending ready (prevents hangs)
workerReadyResolve?.();
if (!stopped && code !== 0) {
process.stderr.write(`[kernel] respawning worker for group "${group}" in 1s\n`);
for (const senseName of sensesForGroup(group)) {
scheduler.onComputeComplete(senseName);
}
setTimeout(() => {
if (!stopped) {
startWorker(group);
}
}, 1000);
}
});
workers.set(group, { group, process: child });
return workerReady;
}
const senseWorkerPool = createSenseWorkerPool({
nerveRoot,
workerScript,
onWorkerMessage: handleWorkerMessage,
sensesForGroup: (group) => senseNamesInGroup(config, group),
onWorkerCrashed: clearSchedulerForGroup,
onBeforeGroupRestart: clearSchedulerForGroup,
isStopped: () => stopped,
});
function triggerFn(senseName: string): void {
const group = groupForSense(config, senseName);
@@ -310,12 +192,7 @@ export function createKernel(
process.stderr.write(`[kernel] triggerFn: unknown sense "${senseName}"\n`);
return;
}
const entry = workers.get(group);
if (entry === undefined) {
process.stderr.write(`[kernel] triggerFn: no worker for group "${group}"\n`);
return;
}
sendCompute(entry.process, senseName);
senseWorkerPool.sendCompute(group, senseName);
}
function triggerSense(senseName: string): void {
@@ -323,11 +200,10 @@ export function createKernel(
if (group === null) {
throw new Error(`Unknown sense: "${senseName}"`);
}
const entry = workers.get(group);
if (entry === undefined) {
if (!senseWorkerPool.hasWorkerForGroup(group)) {
throw new Error(`No worker running for group "${group}" (sense: "${senseName}")`);
}
sendCompute(entry.process, senseName);
senseWorkerPool.sendCompute(group, senseName);
}
scheduler = createReflexScheduler(config, bus, triggerFn, {
@@ -339,63 +215,13 @@ export function createKernel(
}
for (const group of groups) {
startWorker(group);
}
function waitForExit(child: ChildProcess, timeoutMs: number): Promise<void> {
return new Promise((resolve) => {
const timer = setTimeout(() => {
child.kill("SIGKILL");
resolve();
}, timeoutMs);
child.once("exit", () => {
clearTimeout(timer);
resolve();
});
});
}
// --- restartGroup: gracefully stop worker, then respawn and await ready ---
async function restartGroup(group: string): Promise<void> {
const entry = workers.get(group);
if (entry === undefined) return;
for (const senseName of sensesForGroup(group)) {
scheduler.onComputeComplete(senseName);
}
sendShutdown(entry.process);
await waitForExit(entry.process, 5000);
if (!stopped) {
await startWorker(group);
}
}
function collectGroups(cfg: NerveConfig): Set<string> {
const result = new Set<string>();
for (const sc of Object.values(cfg.senses)) {
result.add(sc.group);
}
return result;
}
function sensesForGroupInConfig(cfg: NerveConfig, group: string): Set<string> {
const result = new Set<string>();
for (const [name, sc] of Object.entries(cfg.senses)) {
if (sc.group === group) result.add(name);
}
return result;
senseWorkerPool.startWorker(group);
}
function removeStaleGroups(oldGroups: Set<string>, newGroups: Set<string>): void {
for (const g of oldGroups) {
if (newGroups.has(g)) continue;
const entry = workers.get(g);
if (entry !== undefined) {
sendShutdown(entry.process);
workers.delete(g);
}
senseWorkerPool.evictGroup(g);
groups.delete(g);
}
}
@@ -404,27 +230,25 @@ export function createKernel(
for (const g of newGroups) {
if (oldGroups.has(g)) continue;
groups.add(g);
if (!stopped) startWorker(g);
if (!stopped) {
senseWorkerPool.startWorker(g);
}
}
}
function reloadConfig(newConfig: NerveConfig): void {
const oldGroups = collectGroups(config);
const oldGroups = collectSenseGroups(config);
const oldConfig = config;
const oldWorkflows = config.workflows ?? {};
config = newConfig;
// Note: pending/throttled computes in the old scheduler are silently dropped here.
// In-flight state is not preserved across reloadConfig.
scheduler.stop();
scheduler = createReflexScheduler(config, bus, triggerFn, {
logStore,
});
// Update workflow concurrency/overflow config incrementally — no restart needed
workflowManager.updateConfig(newConfig);
const newWorkflows = newConfig.workflows ?? {};
// Drain + remove workers for deleted workflows
for (const workflowName of Object.keys(oldWorkflows)) {
if (!(workflowName in newWorkflows)) {
process.stderr.write(
@@ -439,20 +263,17 @@ export function createKernel(
}
}
const newGroups = collectGroups(newConfig);
const newGroups = collectSenseGroups(newConfig);
removeStaleGroups(oldGroups, newGroups);
addNewGroups(oldGroups, newGroups);
// Restart existing groups that gained new senses — the running worker process
// was spawned with the old config and will report "Unknown sense" for any newly
// added sense until it is restarted.
for (const g of newGroups) {
if (!oldGroups.has(g)) continue; // already handled by addNewGroups
const oldSenses = sensesForGroupInConfig(oldConfig, g);
const newSenses = sensesForGroupInConfig(newConfig, g);
if (!oldGroups.has(g)) continue;
const oldSenses = senseNamesInGroupAsSet(oldConfig, g);
const newSenses = senseNamesInGroupAsSet(newConfig, g);
const gained = [...newSenses].some((s) => !oldSenses.has(s));
if (gained) {
restartGroup(g).catch((e) => {
senseWorkerPool.restartGroup(g).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] reloadConfig restartGroup error for "${g}": ${msg}\n`);
});
@@ -464,80 +285,28 @@ export function createKernel(
return {
uptime: Date.now() - startTime,
activeSenses: Object.keys(config.senses).length,
activeGroups: workers.size,
activeGroups: senseWorkerPool.activeGroupCount(),
pendingComputes: 0,
activeWorkflows: workflowManager.totalActiveCount(),
memoryUsage: process.memoryUsage(),
};
}
function handleSenseFileChange(senseName: string): void {
const sc = config.senses[senseName];
if (sc === undefined) return;
process.stderr.write(
`[kernel] sense file changed: "${senseName}", restarting group "${sc.group}"\n`,
);
logStore.append({
source: "system",
type: "sense_reload",
refId: senseName,
payload: null,
ts: Date.now(),
});
restartGroup(sc.group).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] restartGroup error: ${msg}\n`);
});
}
function handleWorkflowFileChange(workflowName: string): void {
process.stderr.write(
`[kernel] workflow file changed: "${workflowName}", draining and respawning worker\n`,
);
logStore.append({
source: "system",
type: "workflow_reload",
refId: workflowName,
payload: null,
ts: Date.now(),
});
workflowManager.drainAndRespawn(workflowName).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] drainAndRespawn error for "${workflowName}": ${msg}\n`);
});
}
function handleConfigFileChange(): void {
process.stderr.write("[kernel] nerve.yaml changed, reloading config\n");
logStore.append({
source: "system",
type: "config_reload",
refId: null,
payload: null,
ts: Date.now(),
});
try {
const raw = readFileSync(join(nerveRoot, "nerve.yaml"), "utf8");
const parseResult = parseNerveConfig(raw);
if (!parseResult.ok) {
process.stderr.write(
`[kernel] config parse error, keeping current config: ${parseResult.error.message}\n`,
);
return;
}
reloadConfig(parseResult.value);
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] failed to read nerve.yaml, keeping current config: ${msg}\n`);
}
}
const fileWatchHandlers = createKernelFileWatchHandlers({
nerveRoot,
getConfig: () => config,
logStore,
workflowManager,
restartGroup: (group) => senseWorkerPool.restartGroup(group),
reloadConfig,
});
let fileWatcher: FileWatcher | null = null;
if (options.enableFileWatcher) {
fileWatcher = createFileWatcher(nerveRoot, (change) => {
if (change.kind === "sense") handleSenseFileChange(change.senseName);
if (change.kind === "config") handleConfigFileChange();
if (change.kind === "workflow") handleWorkflowFileChange(change.workflowName);
if (change.kind === "sense") fileWatchHandlers.onSenseFileChange(change.senseName);
if (change.kind === "config") fileWatchHandlers.onConfigFileChange();
if (change.kind === "workflow") fileWatchHandlers.onWorkflowFileChange(change.workflowName);
});
}
@@ -577,12 +346,7 @@ export function createKernel(
}
scheduler.stop();
await workflowManager.stop();
const exitPromises: Promise<void>[] = [];
for (const entry of workers.values()) {
sendShutdown(entry.process);
exitPromises.push(waitForExit(entry.process, 5000));
}
await Promise.all(exitPromises);
await senseWorkerPool.shutdownAll();
logStore.append({
source: "system",
type: "stop",
@@ -594,7 +358,7 @@ export function createKernel(
}
function getWorkerPid(group: string): number | null {
return workers.get(group)?.process.pid ?? null;
return senseWorkerPool.getWorkerPid(group);
}
const senseCount = Object.keys(config.senses).length;
@@ -610,7 +374,7 @@ export function createKernel(
getWorkerPid,
triggerCompute: triggerFn,
triggerSense,
restartGroup,
restartGroup: (group) => senseWorkerPool.restartGroup(group),
reloadConfig,
getHealth,
};
+1 -1
View File
@@ -10,7 +10,7 @@
*/
import type { NerveConfig } from "@uncaged/nerve-core";
import type { LogStore } from "./log-store.js";
import type { LogStore } from "@uncaged/nerve-store";
import type { SignalBus, Unsubscribe } from "./signal-bus.js";
/** Sends a compute message to the worker responsible for the given sense. */
+13 -13
View File
@@ -6,9 +6,9 @@ import { drizzle } from "drizzle-orm/node-sqlite";
import type { NodeSQLiteDatabase } from "drizzle-orm/node-sqlite";
import type { Result } from "@uncaged/nerve-core";
import { err, ok } from "@uncaged/nerve-core";
import { err, isPlainRecord, ok } from "@uncaged/nerve-core";
import type { BlobStore } from "./blob-store.js";
import type { BlobStore } from "@uncaged/nerve-store";
/** A Drizzle DB instance (schema-generic) */
export type DrizzleDB = NodeSQLiteDatabase<Record<string, never>>;
@@ -108,10 +108,11 @@ export function runMigrations(sqlite: DatabaseSync, migrationsDir: string): Resu
const filesResult = listMigrationFiles(migrationsDir);
if (!filesResult.ok) return filesResult;
const migrationRows = sqlite.prepare("SELECT name FROM _migrations").all();
const applied = new Set<string>(
(sqlite.prepare("SELECT name FROM _migrations").all() as Array<{ name: string }>).map(
(r) => r.name,
),
migrationRows
.filter((r): r is { name: string } => isPlainRecord(r) && typeof r.name === "string")
.map((r) => r.name),
);
for (const file of filesResult.value) {
@@ -145,6 +146,7 @@ export function openSenseDb(
const migResult = runMigrations(sqlite, migrationsDir);
if (!migResult.ok) return migResult;
// Drizzle infers a schema-specific DB type; senses are schema-agnostic at this layer.
const db = drizzle({ client: sqlite }) as DrizzleDB;
return ok({ sqlite, db });
}
@@ -162,6 +164,7 @@ export function openPeerDb(dbPath: string): Result<DrizzleDB> {
return err(new Error(`Failed to open peer database "${dbPath}" (readonly): ${msg}`));
}
// Same schema-agnostic Drizzle wrapper as openSenseDb.
return ok(drizzle({ client: sqlite }) as DrizzleDB);
}
@@ -180,18 +183,13 @@ export async function loadComputeFn(senseIndexPath: string): Promise<Result<Comp
return err(new Error(`Failed to import sense module "${senseIndexPath}": ${msg}`));
}
if (
mod === null ||
typeof mod !== "object" ||
!("compute" in mod) ||
typeof (mod as Record<string, unknown>).compute !== "function"
) {
if (!isPlainRecord(mod) || !("compute" in mod) || typeof mod.compute !== "function") {
return err(
new Error(`Sense module "${senseIndexPath}" must export a named "compute" function`),
);
}
return ok((mod as { compute: ComputeFn }).compute);
return ok(mod.compute as ComputeFn);
}
/**
@@ -232,7 +230,9 @@ export async function executeCompute(
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
if (controller.signal.aborted) {
return err(new Error(`compute("${runtime.name}") timed out after ${timeoutMs as number}ms`));
return err(
new Error(`compute("${runtime.name}") timed out after ${String(timeoutMs ?? "?")}ms`),
);
}
return err(new Error(`compute("${runtime.name}") threw: ${msg}`));
} finally {
+1 -1
View File
@@ -20,7 +20,7 @@ import { join, resolve } from "node:path";
import { parseNerveConfig } from "@uncaged/nerve-core";
import type { NerveConfig } from "@uncaged/nerve-core";
import { createBlobStore } from "./blob-store.js";
import { createBlobStore } from "@uncaged/nerve-store";
import type { WorkerToParentMessage } from "./ipc.js";
import { parseParentMessage } from "./ipc.js";
import { executeCompute, loadComputeFn, openPeerDb, openSenseDb } from "./sense-runtime.js";
+211
View File
@@ -0,0 +1,211 @@
/**
* Sense worker pool — forked child processes per sense group (IPC lifecycle).
*/
import { fork } from "node:child_process";
import type { ChildProcess } from "node:child_process";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { ComputeMessage, ShutdownMessage } from "./ipc.js";
import { parseWorkerMessage } from "./ipc.js";
import {
formatCapturedStderrTail,
formatChildExitSummary,
teeCapturedStderr,
} from "./worker-fork-support.js";
export function resolveWorkerScript(): string {
const __filename = fileURLToPath(import.meta.url);
const __dir = dirname(__filename);
return join(__dir, "sense-worker.js");
}
type WorkerEntry = {
group: string;
process: ChildProcess;
};
export type SenseWorkerPoolOptions = {
nerveRoot: string;
workerScript: string;
/** Invoked for every IPC message from a worker (including ready / signal / error). */
onWorkerMessage: (raw: unknown) => void;
/** Sense names in a group — used when clearing scheduler state on crash or restart. */
sensesForGroup: (group: string) => string[];
/**
* Called when a worker exits with non-zero code before scheduling a respawn
* (scheduler should release pending computes for senses in that group).
*/
onWorkerCrashed: (group: string) => void;
/**
* Called at the beginning of `restartGroup` before shutdown
* (same scheduler cleanup as crash path).
*/
onBeforeGroupRestart: (group: string) => void;
isStopped: () => boolean;
};
export type SenseWorkerPool = {
startWorker: (group: string) => Promise<void>;
restartGroup: (group: string) => Promise<void>;
/** Send shutdown and drop the entry without waiting (matches reloadConfig stale-group removal). */
evictGroup: (group: string) => void;
shutdownAll: () => Promise<void>;
sendCompute: (group: string, senseName: string) => void;
getWorkerPid: (group: string) => number | null;
hasWorkerForGroup: (group: string) => boolean;
activeGroupCount: () => number;
};
function spawnWorker(
nerveRoot: string,
group: string,
workerScript: string,
stderrTail: { value: string },
): ChildProcess {
const child = fork(workerScript, ["--group", group, "--root", nerveRoot], {
stdio: ["ignore", "inherit", "pipe", "ipc"],
});
teeCapturedStderr(child, stderrTail);
child.on("error", (err) => {
if ((err as NodeJS.ErrnoException).code !== "EPIPE") {
console.error("[worker] error:", err.message);
}
});
return child;
}
function sendComputeToProcess(worker: ChildProcess, senseName: string): void {
if (worker.connected === false) return;
const msg: ComputeMessage = { type: "compute", sense: senseName };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function sendShutdownToProcess(worker: ChildProcess): void {
if (worker.connected === false) return;
const msg: ShutdownMessage = { type: "shutdown" };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function waitForExit(child: ChildProcess, timeoutMs: number): Promise<void> {
return new Promise((resolve) => {
const timer = setTimeout(() => {
child.kill("SIGKILL");
resolve();
}, timeoutMs);
child.once("exit", () => {
clearTimeout(timer);
resolve();
});
});
}
export function createSenseWorkerPool(options: SenseWorkerPoolOptions): SenseWorkerPool {
const workers = new Map<string, WorkerEntry>();
function startWorker(group: string): Promise<void> {
const stderrTail = { value: "" };
const child = spawnWorker(options.nerveRoot, group, options.workerScript, stderrTail);
let workerReadyResolve: (() => void) | undefined;
const workerReady = new Promise<void>((resolve) => {
workerReadyResolve = resolve;
});
child.on("message", (raw: unknown) => {
const result = parseWorkerMessage(raw);
if (result.ok && result.value.type === "ready") {
workerReadyResolve?.();
}
options.onWorkerMessage(raw);
});
child.on("exit", (code, signal) => {
const summary = formatChildExitSummary(code, signal ?? null);
process.stderr.write(
`[kernel] worker for group "${group}" exited (${summary})${formatCapturedStderrTail(stderrTail.value)}\n`,
);
workerReadyResolve?.();
if (!options.isStopped() && code !== 0) {
process.stderr.write(`[kernel] respawning worker for group "${group}" in 1s\n`);
options.onWorkerCrashed(group);
setTimeout(() => {
if (!options.isStopped()) {
startWorker(group);
}
}, 1000);
}
});
workers.set(group, { group, process: child });
return workerReady;
}
async function restartGroup(group: string): Promise<void> {
const entry = workers.get(group);
if (entry === undefined) return;
options.onBeforeGroupRestart(group);
sendShutdownToProcess(entry.process);
await waitForExit(entry.process, 5000);
if (!options.isStopped()) {
await startWorker(group);
}
}
function evictGroup(group: string): void {
const entry = workers.get(group);
if (entry === undefined) return;
sendShutdownToProcess(entry.process);
workers.delete(group);
}
async function shutdownAll(): Promise<void> {
const exitPromises: Promise<void>[] = [];
for (const entry of workers.values()) {
sendShutdownToProcess(entry.process);
exitPromises.push(waitForExit(entry.process, 5000));
}
await Promise.all(exitPromises);
}
function sendCompute(group: string, senseName: string): void {
const entry = workers.get(group);
if (entry === undefined) return;
sendComputeToProcess(entry.process, senseName);
}
function getWorkerPid(group: string): number | null {
return workers.get(group)?.process.pid ?? null;
}
function hasWorkerForGroup(group: string): boolean {
return workers.has(group);
}
function activeGroupCount(): number {
return workers.size;
}
return {
startWorker,
restartGroup,
evictGroup,
shutdownAll,
sendCompute,
getWorkerPid,
hasWorkerForGroup,
activeGroupCount,
};
}
+12 -7
View File
@@ -12,8 +12,9 @@ import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { NerveConfig, WorkflowConfig, WorkflowMessage } from "@uncaged/nerve-core";
import { START } from "@uncaged/nerve-core";
import { START, isPlainRecord } from "@uncaged/nerve-core";
import type { LogStore, WorkflowRunStatus } from "@uncaged/nerve-store";
import type {
ResumeThreadMessage,
ShutdownMessage,
@@ -21,8 +22,6 @@ import type {
ThreadEventMessage,
} from "./ipc.js";
import { parseWorkerMessage } from "./ipc.js";
import type { LogStore } from "./log-store.js";
import type { WorkflowRunStatus } from "./log-store.js";
import {
formatCapturedStderrTail,
formatChildExitSummary,
@@ -92,8 +91,8 @@ function readLaunchFromTriggerPayload(
raw: unknown,
engineDefaultMaxRounds: number,
): { prompt: string; maxRounds: number } {
if (raw !== null && typeof raw === "object" && !Array.isArray(raw)) {
const o = raw as Record<string, unknown>;
if (isPlainRecord(raw)) {
const o = raw;
if (typeof o.prompt === "string" && typeof o.maxRounds === "number") {
return { prompt: o.prompt, maxRounds: o.maxRounds };
}
@@ -308,7 +307,10 @@ export function createWorkflowManager(
function recoverQueuedRun(workflowName: string, runId: string, state: WorkflowState): void {
if (state.queue.some((q) => q.runId === runId)) return;
const launch = readLaunchFromTriggerPayload(logStore.getTriggerPayload(runId), config.maxRounds);
const launch = readLaunchFromTriggerPayload(
logStore.getTriggerPayload(runId),
config.maxRounds,
);
state.queue.push({ runId, prompt: launch.prompt, maxRounds: launch.maxRounds });
process.stderr.write(
`[workflow-manager] crash-recovery: re-queued thread "${runId}" for "${workflowName}"\n`,
@@ -323,7 +325,10 @@ export function createWorkflowManager(
): void {
if (state.active.has(runId)) return;
const rawMessages = logStore.getThreadMessages(runId);
const launch = readLaunchFromTriggerPayload(logStore.getTriggerPayload(runId), config.maxRounds);
const launch = readLaunchFromTriggerPayload(
logStore.getTriggerPayload(runId),
config.maxRounds,
);
const messages = ensureThreadMessagesWithStart(rawMessages, launch.prompt, launch.maxRounds);
state.active.add(runId);
const msg: ResumeThreadMessage = {
+103 -57
View File
@@ -12,8 +12,14 @@
import { existsSync } from "node:fs";
import { join, resolve } from "node:path";
import type { RoleMeta, WorkflowDefinition, WorkflowMessage } from "@uncaged/nerve-core";
import { END, START } from "@uncaged/nerve-core";
import type {
Moderator,
RoleMeta,
StartSignal,
WorkflowDefinition,
WorkflowMessage,
} from "@uncaged/nerve-core";
import { END, START, isPlainRecord } from "@uncaged/nerve-core";
import type {
ThreadEventType,
@@ -23,6 +29,8 @@ import type {
import { parseParentMessage } from "./ipc.js";
import { ignoreSessionBroadcastSignals } from "./worker-fork-support.js";
type ModeratorInput = Parameters<Moderator<RoleMeta>>[0];
// ---------------------------------------------------------------------------
// IPC helpers
// ---------------------------------------------------------------------------
@@ -63,6 +71,79 @@ function sendWorkflowMessage(runId: string, message: WorkflowMessage): void {
// Thread loop (signal-driven automaton, issue #80)
// ---------------------------------------------------------------------------
function validateRoleResult(
result: { content: string; meta: Record<string, unknown> },
roleName: string,
runId: string,
): boolean {
if (typeof result.content !== "string") {
sendWorkflowError(runId, `Role "${roleName}" returned non-string content`);
return false;
}
if (result.meta === null || typeof result.meta !== "object" || Array.isArray(result.meta)) {
sendWorkflowError(runId, `Role "${roleName}" returned invalid meta (must be a plain object)`);
return false;
}
return true;
}
function buildInitialLastSignal(lastMsg: WorkflowMessage): ModeratorInput {
if (lastMsg.role === START) {
return {
role: START,
content: lastMsg.content,
meta: lastMsg.meta as StartSignal["meta"],
timestamp: lastMsg.timestamp,
};
}
return { role: lastMsg.role, meta: lastMsg.meta as Record<string, unknown> };
}
function initChain(
runId: string,
resumeMessages: WorkflowMessage[],
freshPrompt: string | null,
maxRounds: number,
): WorkflowMessage[] {
if (resumeMessages.length > 0) {
return [...resumeMessages];
}
const prompt = freshPrompt ?? "";
const startMsg: WorkflowMessage = {
role: START,
content: prompt,
meta: { maxRounds },
timestamp: Date.now(),
};
sendWorkflowMessage(runId, startMsg);
return [startMsg];
}
async function executeRole(
def: WorkflowDefinition<RoleMeta>,
nextRole: string,
chain: WorkflowMessage[],
runId: string,
): Promise<{ content: string; meta: Record<string, unknown> } | null> {
const role = def.roles[nextRole];
if (!role) {
sendWorkflowError(runId, `Unknown role: ${nextRole}`);
return null;
}
let result: { content: string; meta: Record<string, unknown> };
try {
result = await role(chain);
} catch (e: unknown) {
const errMsg = e instanceof Error ? e.message : String(e);
sendThreadEvent(runId, "failed", { error: errMsg });
return null;
}
if (!validateRoleResult(result, nextRole, runId)) return null;
return result;
}
async function runThread(
def: WorkflowDefinition<RoleMeta>,
runId: string,
@@ -70,21 +151,7 @@ async function runThread(
resumeMessages: WorkflowMessage[] = [],
freshPrompt: string | null = null,
): Promise<void> {
let chain: WorkflowMessage[];
if (resumeMessages.length > 0) {
chain = [...resumeMessages];
} else {
const prompt = freshPrompt ?? "";
const startMsg: WorkflowMessage = {
role: START,
content: prompt,
meta: { maxRounds },
timestamp: Date.now(),
};
chain = [startMsg];
sendWorkflowMessage(runId, startMsg);
}
const chain = initChain(runId, resumeMessages, freshPrompt, maxRounds);
let roleRound = chain.filter((m) => m.role !== START).length;
const lastMsg = chain[chain.length - 1];
@@ -93,21 +160,7 @@ async function runThread(
return;
}
const lastSignal =
lastMsg.role === START
? {
role: START,
content: lastMsg.content,
meta: lastMsg.meta as { maxRounds: number },
timestamp: lastMsg.timestamp,
}
: { role: lastMsg.role, meta: lastMsg.meta as Record<string, unknown> };
let nextRole = def.moderator(
lastSignal as Parameters<typeof def.moderator>[0],
roleRound,
maxRounds,
);
let nextRole = def.moderator(buildInitialLastSignal(lastMsg), roleRound, maxRounds);
if (nextRole === END) {
sendThreadEvent(runId, "completed", null);
@@ -115,20 +168,8 @@ async function runThread(
}
while (roleRound < maxRounds) {
const role = def.roles[nextRole];
if (!role) {
sendWorkflowError(runId, `Unknown role: ${nextRole}`);
return;
}
let result: { content: string; meta: Record<string, unknown> };
try {
result = await role(chain);
} catch (e: unknown) {
const errMsg = e instanceof Error ? e.message : String(e);
sendThreadEvent(runId, "failed", { error: errMsg });
return;
}
const result = await executeRole(def, nextRole, chain, runId);
if (result === null) return;
const message: WorkflowMessage = {
role: nextRole,
@@ -141,8 +182,8 @@ async function runThread(
roleRound += 1;
const signal = { role: nextRole, meta: result.meta };
nextRole = def.moderator(signal as Parameters<typeof def.moderator>[0], roleRound, maxRounds);
const signal: ModeratorInput = { role: nextRole, meta: result.meta };
nextRole = def.moderator(signal, roleRound, maxRounds);
if (nextRole === END) {
sendThreadEvent(runId, "completed", null);
@@ -157,6 +198,17 @@ async function runThread(
// Workflow definition loader
// ---------------------------------------------------------------------------
function isWorkflowDefinitionShape(def: unknown): def is WorkflowDefinition<RoleMeta> {
if (!isPlainRecord(def)) return false;
return (
typeof def.moderator === "function" &&
typeof def.roles === "object" &&
def.roles !== null &&
!Array.isArray(def.roles) &&
typeof def.name === "string"
);
}
async function loadWorkflowDefinition(
nerveRoot: string,
workflowName: string,
@@ -177,19 +229,13 @@ async function loadWorkflowDefinition(
const mod = await import(indexPath);
const def: unknown = mod.default ?? mod;
if (
def === null ||
typeof def !== "object" ||
typeof (def as WorkflowDefinition<RoleMeta>).moderator !== "function" ||
typeof (def as WorkflowDefinition<RoleMeta>).roles !== "object" ||
typeof (def as WorkflowDefinition<RoleMeta>).name !== "string"
) {
if (!isWorkflowDefinitionShape(def)) {
throw new Error(
`Workflow "${workflowName}" must export a WorkflowDefinition with "name", "roles", and "moderator".`,
);
}
return def as WorkflowDefinition<RoleMeta>;
return def;
}
// ---------------------------------------------------------------------------
@@ -244,7 +290,7 @@ function handleMessage(
const previous = inFlight.get(runId) ?? Promise.resolve();
const next = previous
.then(() => runThread(def, runId, maxRounds, messages as WorkflowMessage[], null))
.then(() => runThread(def, runId, maxRounds, messages, null))
.catch((e: unknown) => {
const errMsg = e instanceof Error ? e.message : String(e);
sendWorkflowError(runId, errMsg);
+48
View File
@@ -0,0 +1,48 @@
# @uncaged/nerve-store
Persistent storage for the [nerve](../../README.md) daemon — append-only structured logs, optional JSONL cold archive, and content-addressable blobs.
## LogStore (`createLogStore`, `log-store.ts`)
- **Append-only log table** — rows with `source`, `type`, `refId`, `payload`, `ts` (string payloads for ad hoc fields)
- **SQLite WAL** — `DatabaseSync` from `node:sqlite`
- **Workflow run tracking** — materialized `workflow_runs` table plus helpers to list active runs, upsert status transitions, and read **thread messages** / **role rounds** for CLI and crash recovery
- **Meta key-value** — small `meta` table (e.g. archive watermarks)
Public exports include `LogStore`, `LogEntry`, `LogQuery`, `WorkflowRun`, `WorkflowRunStatus`, `ThreadRoundRow`, `GetThreadRoundsParams`, and archive-related types re-exported from `log-archive`.
## WorkflowRunStatus
Runs progress through a small state machine. Typical paths:
1. **`queued`** → **`started`** when a worker picks up the thread
2. **`started`** → **`completed`** | **`failed`** | **`crashed`** | **`interrupted`** | **`dropped`**
Semantics in the daemon/store layer:
- **`completed` / `failed`** — normal terminal outcomes from the workflow worker
- **`crashed`** — worker exited unexpectedly; manager may respawn and **`resume-thread`** eligible **`started`** runs
- **`interrupted`** — e.g. hot-reload drain killed an in-flight thread after timeout
- **`dropped`** — concurrency **`overflow: drop`** rejected a new run, or **`overflow: queue`** evicted an queued item when the queue was full
## LogArchive (`log-archive.ts`)
- **`archiveLogs`** / helpers — export eligible UTC days of old rows to **`data/archive/logs/YYYY-MM-DD.jsonl`**, delete archived rows from SQLite, optional **`VACUUM`**
- Used by **`nerve store archive`** in `@uncaged/nerve-cli`
## BlobStore (`createBlobStore`, `blob-store.ts`)
- **Content-addressable storage** — `write` returns lowercase **sha256** hex; files live under **`data/blobs/<2-hex>/<62-hex>`**
- **`read` / `exists`** — path must match digest on disk (tamper detection)
## Install
```bash
pnpm add @uncaged/nerve-store
```
Requires Node.js ≥ 22.5 (same as the rest of the stack).
## License
MIT
+24
View File
@@ -0,0 +1,24 @@
{
"name": "@uncaged/nerve-store",
"version": "0.4.0",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": ["dist"],
"publishConfig": {
"access": "public"
},
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "rslib build",
"test": "vitest run"
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*"
},
"devDependencies": {
"@rslib/core": "^0.21.3",
"@types/node": "^22.0.0",
"vitest": "^4.1.5"
}
}
+19
View File
@@ -0,0 +1,19 @@
import { defineConfig } from "@rslib/core";
export default defineConfig({
lib: [
{
format: "esm",
dts: true,
},
],
source: {
entry: {
index: "src/index.ts",
},
},
output: {
target: "node",
cleanDistPath: true,
},
});
+16
View File
@@ -0,0 +1,16 @@
/**
* @uncaged/nerve-store — append-only log storage, cold-archive helpers, CAS blob store.
*/
export * from "./blob-store.js";
export * from "./log-archive.js";
export { createLogStore } from "./log-store.js";
export type {
GetThreadRoundsParams,
LogEntry,
LogQuery,
LogStore,
ThreadRoundRow,
WorkflowRun,
WorkflowRunStatus,
} from "./log-store.js";
@@ -11,6 +11,8 @@ import { mkdirSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { DatabaseSync, type StatementSync } from "node:sqlite";
import { isPlainRecord } from "@uncaged/nerve-core";
import {
DEFAULT_LOG_RETENTION_MS,
LOG_ARCHIVE_META_KEY,
@@ -68,11 +70,15 @@ const VALID_WORKFLOW_STATUSES = new Set<string>([
"interrupted",
]);
function isWorkflowRunStatus(value: string): value is WorkflowRunStatus {
return VALID_WORKFLOW_STATUSES.has(value);
}
function validateWorkflowRunStatus(status: string): WorkflowRunStatus {
if (!VALID_WORKFLOW_STATUSES.has(status)) {
if (!isWorkflowRunStatus(status)) {
throw new Error(`Invalid workflow run status from DB: "${status}"`);
}
return status as WorkflowRunStatus;
return status;
}
/** One row in the workflow_runs materialized table. */
@@ -147,14 +153,14 @@ export type LogStore = {
runId: string,
) => Array<{ role: string; content: string; meta: unknown; timestamp: number }>;
/**
* Count role command events for a run (excludes `thread_start` and invalid payloads).
* Count role command events for a run (excludes `thread_start`/`__start__` messages and invalid payloads).
* Round indices for {@link getThreadRounds} are 1..count in chronological order.
*/
getThreadRoundCount: (runId: string) => number;
/**
* Role rounds for agent-oriented retrieval: each row is one `thread_command_event`
* whose JSON `type` is not `thread_start`, with `round` from ROW_NUMBER() OVER (ORDER BY id ASC).
* No schema migration numbering is computed in SQL.
* Role rounds for agent-oriented retrieval: each row is one `thread_command_event` or
* `thread_workflow_message` whose JSON `type` is not `thread_start` and `role` is not `__start__`,
* with `round` from ROW_NUMBER() OVER (ORDER BY id ASC). No schema migration numbering is computed in SQL.
*/
getThreadRounds: (runId: string, params: GetThreadRoundsParams) => ThreadRoundRow[];
/**
@@ -324,7 +330,8 @@ export function createLogStore(dbPath: string): LogStore {
`SELECT COUNT(*) AS c FROM logs
WHERE source = 'workflow' AND type IN ('thread_command_event', 'thread_workflow_message') AND ref_id = ?
AND payload IS NOT NULL AND json_valid(payload) = 1
AND COALESCE(json_extract(payload, '$.type'), '') != 'thread_start'`,
AND COALESCE(json_extract(payload, '$.type'), '') != 'thread_start'
AND COALESCE(json_extract(payload, '$.role'), '') != '__start__'`,
);
const getThreadRoundsStmt = sqlite.prepare(
@@ -335,6 +342,7 @@ export function createLogStore(dbPath: string): LogStore {
WHERE source = 'workflow' AND type IN ('thread_command_event', 'thread_workflow_message') AND ref_id = @runId
AND payload IS NOT NULL AND json_valid(payload) = 1
AND COALESCE(json_extract(payload, '$.type'), '') != 'thread_start'
AND COALESCE(json_extract(payload, '$.role'), '') != '__start__'
)
SELECT id, ts, payload, rn FROM numbered
WHERE (@before = 0 OR rn < @before)
@@ -506,10 +514,9 @@ export function createLogStore(dbPath: string): LogStore {
const row = getTriggerPayloadStmt.get(runId) as { payload: string | null } | undefined;
if (row === undefined || row.payload === null) return null;
try {
const parsed = JSON.parse(row.payload) as unknown;
if (parsed !== null && typeof parsed === "object") {
const obj = parsed as Record<string, unknown>;
return obj.triggerPayload ?? null;
const parsed: unknown = JSON.parse(row.payload);
if (isPlainRecord(parsed)) {
return parsed.triggerPayload ?? null;
}
} catch {
// malformed
@@ -523,12 +530,8 @@ export function createLogStore(dbPath: string): LogStore {
for (const row of rows) {
if (row.payload === null) continue;
try {
const parsed = JSON.parse(row.payload) as unknown;
if (
parsed !== null &&
typeof parsed === "object" &&
typeof (parsed as Record<string, unknown>).type === "string"
) {
const parsed: unknown = JSON.parse(row.payload);
if (isPlainRecord(parsed) && typeof parsed.type === "string") {
result.push(parsed as { type: string; [key: string]: unknown });
}
} catch {
@@ -542,9 +545,9 @@ export function createLogStore(dbPath: string): LogStore {
payload: string,
): { role: string; content: string; meta: unknown; timestamp: number } | null {
try {
const parsed = JSON.parse(payload) as unknown;
if (parsed === null || typeof parsed !== "object") return null;
const obj = parsed as Record<string, unknown>;
const parsed: unknown = JSON.parse(payload);
if (!isPlainRecord(parsed)) return null;
const obj = parsed;
if (typeof obj.role !== "string" || typeof obj.content !== "string") return null;
return {
role: obj.role,
@@ -577,31 +580,37 @@ export function createLogStore(dbPath: string): LogStore {
return Number(c);
}
function recordToRoundMessage(
obj: Record<string, unknown>,
fallbackTs: number,
): { role: string; content: string; meta: unknown; timestamp: number } | null {
if (typeof obj.role === "string" && typeof obj.content === "string") {
return {
role: obj.role,
content: obj.content,
meta: obj.meta,
timestamp: typeof obj.timestamp === "number" ? obj.timestamp : 0,
};
}
if (typeof obj.type === "string") {
return {
role: typeof obj.role === "string" ? obj.role : obj.type,
content: typeof obj.content === "string" ? obj.content : JSON.stringify(obj),
meta: obj,
timestamp: fallbackTs,
};
}
return null;
}
function parseRoundPayload(
payload: string,
fallbackTs: number,
): { role: string; content: string; meta: unknown; timestamp: number } | null {
try {
const parsed = JSON.parse(payload) as unknown;
if (parsed === null || typeof parsed !== "object") return null;
const obj = parsed as Record<string, unknown>;
if (typeof obj.role === "string" && typeof obj.content === "string") {
return {
role: obj.role,
content: obj.content,
meta: obj.meta,
timestamp: typeof obj.timestamp === "number" ? obj.timestamp : 0,
};
}
if (typeof obj.type === "string") {
return {
role: typeof obj.role === "string" ? obj.role : obj.type,
content: typeof obj.content === "string" ? obj.content : JSON.stringify(obj),
meta: obj,
timestamp: fallbackTs,
};
}
return null;
const parsed: unknown = JSON.parse(payload);
if (!isPlainRecord(parsed)) return null;
return recordToRoundMessage(parsed, fallbackTs);
} catch {
return null;
}
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src",
"composite": false
},
"include": ["src"]
}
+32 -3
View File
@@ -14,6 +14,9 @@ importers:
'@rslib/core':
specifier: ^0.21.3
version: 0.21.3(typescript@5.9.3)
husky:
specifier: ^9.1.7
version: 9.1.7
typescript:
specifier: ^5.5.0
version: 5.9.3
@@ -23,6 +26,9 @@ importers:
'@uncaged/nerve-core':
specifier: workspace:*
version: link:../core
'@uncaged/nerve-store':
specifier: workspace:*
version: link:../store
citty:
specifier: ^0.1.6
version: 0.1.6
@@ -36,9 +42,6 @@ importers:
'@types/node':
specifier: ^22.0.0
version: 22.19.17
'@uncaged/nerve-daemon':
specifier: workspace:*
version: link:../daemon
vitest:
specifier: ^4.1.5
version: 4.1.5(@types/node@22.19.17)(vite@8.0.9(@types/node@22.19.17)(esbuild@0.27.7)(yaml@2.8.3))
@@ -61,6 +64,9 @@ importers:
'@uncaged/nerve-core':
specifier: workspace:*
version: link:../core
'@uncaged/nerve-store':
specifier: workspace:*
version: link:../store
drizzle-orm:
specifier: 1.0.0-beta.23-c10d10c
version: 1.0.0-beta.23-c10d10c(@types/better-sqlite3@7.6.13)(@types/mssql@9.1.11(@azure/core-client@1.10.1))(better-sqlite3@11.10.0)(mssql@11.0.1(@azure/core-client@1.10.1))(sql.js@1.14.1)
@@ -78,6 +84,22 @@ importers:
specifier: ^4.1.5
version: 4.1.5(@types/node@22.19.17)(vite@8.0.9(@types/node@22.19.17)(esbuild@0.27.7)(yaml@2.8.3))
packages/store:
dependencies:
'@uncaged/nerve-core':
specifier: workspace:*
version: link:../core
devDependencies:
'@rslib/core':
specifier: ^0.21.3
version: 0.21.3(typescript@5.9.3)
'@types/node':
specifier: ^22.0.0
version: 22.19.17
vitest:
specifier: ^4.1.5
version: 4.1.5(@types/node@22.19.17)(vite@8.0.9(@types/node@22.19.17)(esbuild@0.27.7)(yaml@2.8.3))
packages:
'@ast-grep/napi-darwin-arm64@0.37.0':
@@ -991,6 +1013,11 @@ packages:
resolution: {integrity: sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==}
engines: {node: '>= 14'}
husky@9.1.7:
resolution: {integrity: sha512-5gs5ytaNjBrh5Ow3zrvdUUY+0VxIuWVL4i9irt6friV+BqdCfmV11CQTWMiBYWHbXhco+J1kHfTOUkePhCDvMA==}
engines: {node: '>=18'}
hasBin: true
iconv-lite@0.6.3:
resolution: {integrity: sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==}
engines: {node: '>=0.10.0'}
@@ -2239,6 +2266,8 @@ snapshots:
- supports-color
optional: true
husky@9.1.7: {}
iconv-lite@0.6.3:
dependencies:
safer-buffer: 2.1.2
+1
View File
@@ -17,6 +17,7 @@
},
"references": [
{ "path": "packages/core" },
{ "path": "packages/store" },
{ "path": "packages/cli" },
{ "path": "packages/daemon" }
]