xiaoju
fc7fc9158c
Phase 4 of RFC #308: Stateful Sense refactor. - CLAUDE.md: updated diagram, tables, examples (no more Signal) - Cleaned stale Signal Bus / DrizzleDB / _signals / retention refs across READMEs, .cursor rules, copilot instructions, .knowledge - Removed drizzle-orm from core package.json (no longer used) - Updated pnpm-lock.yaml Refs #308
92 lines
5.0 KiB
Markdown
92 lines
5.0 KiB
Markdown
# @uncaged/nerve-daemon
|
|
|
|
The observation engine runtime for [nerve](../../README.md) — runs senses, persists JSON state, runs the sense scheduler, and manages workflows.
|
|
|
|
## Architecture
|
|
|
|
| Module | Source (indicative) | Responsibility |
|
|
|--------|---------------------|----------------|
|
|
| **Kernel** | `kernel.ts` | Orchestrator — worker pool, sense scheduler, workflow manager, optional file watcher and daemon IPC, config reload hooks |
|
|
| **Worker pool** | `worker-pool.ts` | Fork and supervise one child process per sense group; restart/shutdown; crash cleanup hooks for scheduler state |
|
|
| **Kernel sense groups** | `kernel-sense-groups.ts` | Derive sense groups from config; list senses per group for scheduling |
|
|
| **Sense runtime** | `sense-runtime.ts` + sense worker | Loads user modules (`compute`, `initialState`), reads/writes `data/senses/<name>.json` |
|
|
| **Sense worker** | `sense-worker.ts` (fork target) | Child process entry — runs `compute(state)` per sense in a group |
|
|
| **Sense scheduler** | `sense-scheduler.ts` | Interval + `on` subscriptions (reverse-index by upstream sense), throttle/coalesce |
|
|
| **Workflow manager** | `workflow-manager.ts` | One worker per workflow name, concurrency (drop/queue), queue caps |
|
|
| **Workflow worker** | `workflow-worker.ts` | Child process — runs RFC-002 threads (`start-thread`, `resume-thread` IPC) |
|
|
| **IPC (parent ↔ workers)** | `ipc.ts` | Typed messages for sense and workflow workers (includes `resume-thread` for recovery) |
|
|
| **Log / workflow persistence** | via `@uncaged/nerve-store` | Structured logs, `workflow_runs`, thread messages (used for recovery) |
|
|
| **Blob store** | `@uncaged/nerve-store` | CAS under `data/blobs/` — workflows use blob storage for artifacts as configured |
|
|
| **File watcher** | `file-watcher.ts` | Watches workspace paths for config / sense / workflow file changes |
|
|
| **Kernel file watch** | `kernel-file-watch.ts` | Maps watcher events to `reloadConfig`, sense group restart, workflow `drainAndRespawn` |
|
|
| **Daemon IPC** | `daemon-ipc.ts` | Unix socket server — parses `@uncaged/nerve-core` `DaemonIpcRequest`, dispatches trigger-workflow / trigger-sense / list-senses |
|
|
|
|
## Crash recovery (workflow workers)
|
|
|
|
If a workflow worker exits unexpectedly while threads are active:
|
|
|
|
- In-flight runs are marked **`crashed`** in the log store; the manager respawns a fresh worker.
|
|
- Runs still in **`started`** state can be **`resume-thread`**’d: the manager rebuilds the message chain from persisted workflow log rows and sends `resume-thread` to the new worker.
|
|
- **Crash-loop backoff:** repeated crashes for the same workflow name are counted in a sliding window (`60s`); after **`5`** crashes in that window, the manager **stops respawning** that worker and logs the condition (avoids tight crash loops).
|
|
|
|
Hot reload (`drainAndRespawn`) uses a controlled drain: in-flight runs may be marked **`interrupted`** when the old worker is torn down after a timeout — that path is distinct from unexpected crash recovery.
|
|
|
|
## Key Design Decisions
|
|
|
|
- **One worker process per sense group** — isolation between groups, shared compute within a group
|
|
- **Sense state as JSON** — `data/senses/<name>.json`, updated after each successful compute in the worker
|
|
- **Throttle + coalesce** — if compute is in-flight, at most one pending trigger is queued (no unbounded accumulation)
|
|
- **Log ≠ Sense trigger** — logs are queryable data assets but cannot schedule sense computes or workflows (prevents feedback loops)
|
|
|
|
## Usage
|
|
|
|
The daemon is typically started via the CLI (`nerve daemon start` / `nerve dev`), but you can embed the kernel:
|
|
|
|
```typescript
|
|
import { readFileSync } from "node:fs";
|
|
import { join } from "node:path";
|
|
|
|
import { parseNerveConfig } from "@uncaged/nerve-core";
|
|
import { createKernel } from "@uncaged/nerve-daemon";
|
|
|
|
const nerveRoot = "/path/to/workspace";
|
|
const yamlPath = join(nerveRoot, "nerve.yaml");
|
|
const parsed = parseNerveConfig(readFileSync(yamlPath, "utf8"));
|
|
if (!parsed.ok) {
|
|
throw parsed.error;
|
|
}
|
|
|
|
const kernel = createKernel(parsed.value, nerveRoot, {
|
|
enableFileWatcher: true,
|
|
ipcSocketPath: join(nerveRoot, "nerve.sock"),
|
|
});
|
|
|
|
await kernel.ready;
|
|
|
|
kernel.triggerSense("cpu-usage");
|
|
const health = kernel.getHealth();
|
|
|
|
await kernel.stop();
|
|
```
|
|
|
|
`createKernel(config, nerveRoot, options?)` — `config` is a parsed `NerveConfig`; `nerveRoot` is the workspace root (contains `nerve.yaml`, `data/`, etc.). Optional `KernelOptions`:
|
|
|
|
| Field | Meaning |
|
|
|-------|---------|
|
|
| `workerScript` | Override path to the sense worker entry script (defaults to the package’s resolved worker) |
|
|
| `enableFileWatcher` | Watch config / senses / workflows for hot reload |
|
|
| `logStore` | Inject a `LogStore` instance (defaults to `createLogStore(join(nerveRoot, "data", "logs.db"))`) |
|
|
| `ipcSocketPath` | When non-null, listen for daemon IPC on this Unix socket path |
|
|
|
|
## Install
|
|
|
|
```bash
|
|
pnpm add @uncaged/nerve-daemon
|
|
```
|
|
|
|
Requires Node.js ≥ 22.5 (for `node:sqlite` in the log store and related persistence).
|
|
|
|
## License
|
|
|
|
MIT
|