docs: update all docs/conventions for stateful sense, remove stale refs
Phase 4 of RFC #308: Stateful Sense refactor. - CLAUDE.md: updated diagram, tables, examples (no more Signal) - Cleaned stale Signal Bus / DrizzleDB / _signals / retention refs across READMEs, .cursor rules, copilot instructions, .knowledge - Removed drizzle-orm from core package.json (no longer used) - Updated pnpm-lock.yaml Refs #308
This commit is contained in:
@@ -51,10 +51,10 @@ Structured rows in `data/logs.db` are surfaced via **`nerve workflow inspect`**
|
||||
```bash
|
||||
nerve sense list # List senses (live fields from daemon IPC when running)
|
||||
nerve sense trigger <name> # IPC trigger-sense — queue a compute for that sense
|
||||
nerve sense query <name> # Read-only SQL on data/senses/<name>.db (optional SQL args)
|
||||
nerve sense schema <name> # Print CREATE TABLE statements for that sense DB
|
||||
```
|
||||
|
||||
Sense state is persisted as JSON under `data/senses/<name>.json` by the sense worker after each successful compute.
|
||||
|
||||
### Store maintenance
|
||||
|
||||
```bash
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## sense list
|
||||
|
||||
- ✅ prints sense list with name, group, throttle, triggers, and last signal time
|
||||
- ✅ prints sense list with name, group, throttle, triggers
|
||||
- 🔲 empty state — no senses registered, prints empty message
|
||||
- 🔲 `--json` — outputs valid JSON array
|
||||
|
||||
@@ -11,19 +11,3 @@
|
||||
- ✅ trigger known sense exits 0, stdout contains "Triggered"
|
||||
- ✅ trigger non-existent sense writes error to stderr and exits 1
|
||||
- ✅ sends correct IPC message `{ type: trigger-sense, sense: <name> }` to daemon
|
||||
|
||||
## sense query
|
||||
|
||||
- ✅ after trigger, persisted `_signals` table has at least one row
|
||||
- ✅ default output lists payload column and counter count
|
||||
- ✅ `--json` prints valid JSON array with payload on each row
|
||||
- ✅ `--sql` runs custom read-only SQL and prints result
|
||||
- 🔲 query non-existent sense — error message
|
||||
- 🔲 `--limit` / `--offset` pagination
|
||||
|
||||
## sense schema
|
||||
|
||||
- ✅ prints CREATE TABLE statements for the sense database
|
||||
- ✅ includes `_signals` table in output
|
||||
- ✅ `--json` prints valid JSON array of SQL strings
|
||||
- 🔲 schema for non-existent sense — error message
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
|
||||
Full round-trip integration tests that exercise multiple subcommands together.
|
||||
|
||||
- ✅ sense list + sense query after trigger — registers sense, triggers, verifies persisted signal and query output
|
||||
- ✅ sense list after trigger — daemon lists configured senses; trigger queues a compute (state persisted under `data/senses/` by the worker)
|
||||
- 🔲 init → dev → trigger workflow → thread inspect round-trip
|
||||
|
||||
@@ -72,8 +72,8 @@ describe("queryKnowledgeRepo (word overlap fallback)", () => {
|
||||
path: "a.md",
|
||||
slug: "a.md#0",
|
||||
chunkIndex: 0,
|
||||
text: "the signal bus emits notifications",
|
||||
contentHash: contentHash("the signal bus emits notifications"),
|
||||
text: "the sense scheduler triggers computes",
|
||||
contentHash: contentHash("the sense scheduler triggers computes"),
|
||||
embedding: fakeEmbeddingBytes("a"),
|
||||
},
|
||||
{
|
||||
@@ -89,7 +89,7 @@ describe("queryKnowledgeRepo (word overlap fallback)", () => {
|
||||
db.close();
|
||||
}
|
||||
|
||||
const ranked = await queryKnowledgeRepo(root, dbPath, "signal bus", 10);
|
||||
const ranked = await queryKnowledgeRepo(root, dbPath, "sense scheduler", 10);
|
||||
expect(ranked.length).toBe(2);
|
||||
expect(ranked[0]?.path).toBe("a.md");
|
||||
expect(ranked[1]?.path).toBe("b.md");
|
||||
|
||||
+8
-21
@@ -4,9 +4,9 @@ Shared types and configuration parser for the [nerve](../../README.md) observati
|
||||
|
||||
## What's Inside
|
||||
|
||||
- **Type definitions** — `Signal`, `SenseConfig`, `SenseInfo`, `WorkflowConfig`, `NerveConfig`, and related types
|
||||
- **Config parser** — `parseNerveConfig(yaml)` validates and parses `nerve.yaml` into `NerveConfig` (rejects reflex entries that declare a `workflow` key; reflexes only schedule senses)
|
||||
- **Sense → workflow routing** — `parseWorkflowTrigger`, `routeSenseComputeOutput`, and types `WorkflowTrigger`, `RoutedSenseOutput`
|
||||
- **Type definitions** — `SenseConfig`, `SenseInfo`, `SenseComputeFn`, `SenseModule`, `WorkflowConfig`, `NerveConfig`, `WorkflowTrigger`, and related types
|
||||
- **Config parser** — `parseNerveConfig(yaml)` validates and parses `nerve.yaml` into `NerveConfig` (top-level `reflexes` is rejected; use `interval` / `on` on each sense)
|
||||
- **Workflow triggers** — `parseWorkflowTrigger` validates structured workflow launch objects from Sense compute results or IPC
|
||||
- **Daemon IPC protocol** — request/response types (`DaemonIpcRequest`, `DaemonIpcResponse`, …) and `parseDaemonIpcRequest` for newline-delimited JSON on the CLI ↔ daemon socket
|
||||
- **Workflow automaton types** — `START` / `END` sentinel constants, `WorkflowMessage`, `StartStep`, `RoleStep`, `ModeratorContext` (`start` + `steps`; empty `steps` on first moderator call), `Moderator` (single `context` argument), `WorkflowDefinition`, `Role`, `RoleResult`, plus `DEFAULT_ENGINE_MAX_ROUNDS`
|
||||
- **Result type** — `Result<T>` with `ok()` / `err()` helpers for explicit error handling (no thrown exceptions for parse paths)
|
||||
@@ -15,7 +15,7 @@ Shared types and configuration parser for the [nerve](../../README.md) observati
|
||||
|
||||
```typescript
|
||||
import { parseNerveConfig, ok, err } from "@uncaged/nerve-core";
|
||||
import type { NerveConfig, Signal, Result } from "@uncaged/nerve-core";
|
||||
import type { NerveConfig, Result } from "@uncaged/nerve-core";
|
||||
|
||||
const result: Result<NerveConfig> = parseNerveConfig(yamlString);
|
||||
if (result.ok) {
|
||||
@@ -23,10 +23,10 @@ if (result.ok) {
|
||||
}
|
||||
```
|
||||
|
||||
### Sense return → signal vs workflow
|
||||
### Workflow trigger validation
|
||||
|
||||
```typescript
|
||||
import { parseWorkflowTrigger, routeSenseComputeOutput } from "@uncaged/nerve-core";
|
||||
import { parseWorkflowTrigger } from "@uncaged/nerve-core";
|
||||
|
||||
const directive = parseWorkflowTrigger({
|
||||
name: "my-workflow",
|
||||
@@ -37,23 +37,10 @@ const directive = parseWorkflowTrigger({
|
||||
if (directive.ok) {
|
||||
console.log(directive.value.name, directive.value.maxRounds, directive.value.prompt);
|
||||
}
|
||||
|
||||
const route = routeSenseComputeOutput({
|
||||
signal: { metric: 42 },
|
||||
workflow: {
|
||||
name: "my-workflow",
|
||||
maxRounds: 8,
|
||||
prompt: "Run now",
|
||||
dryRun: false,
|
||||
},
|
||||
});
|
||||
if (route.ok && route.value.workflow !== null) {
|
||||
console.log(route.value.workflow);
|
||||
} else if (route.ok) {
|
||||
console.log(route.value.signal);
|
||||
}
|
||||
```
|
||||
|
||||
Sense modules return `{ state, workflow }` from `compute(state)`; when `workflow` is non-null it must satisfy the shape validated by `parseWorkflowTrigger` (the daemon validates before starting a run).
|
||||
|
||||
## Duration Format
|
||||
|
||||
Config fields like `throttle`, `timeout`, and `interval` accept human-readable durations:
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
"test": "vitest run"
|
||||
},
|
||||
"dependencies": {
|
||||
"drizzle-orm": "1.0.0-beta.23-c10d10c",
|
||||
"yaml": "^2.8.3"
|
||||
},
|
||||
"devDependencies": {
|
||||
|
||||
@@ -10,7 +10,7 @@ export type SenseConfig = {
|
||||
gracePeriod: number | null;
|
||||
/** Polling interval (ms). When set, the sense is triggered periodically. */
|
||||
interval: number | null;
|
||||
/** Other sense names whose signals trigger this sense. */
|
||||
/** Other sense names whose successful computes schedule this sense (kernel reverse-index). */
|
||||
on: string[];
|
||||
};
|
||||
|
||||
|
||||
@@ -1,23 +1,22 @@
|
||||
# @uncaged/nerve-daemon
|
||||
|
||||
The observation engine runtime for [nerve](../../README.md) — runs senses, routes signals, runs the sense scheduler, and manages workflows.
|
||||
The observation engine runtime for [nerve](../../README.md) — runs senses, persists JSON state, runs the sense scheduler, and manages workflows.
|
||||
|
||||
## Architecture
|
||||
|
||||
| Module | Source (indicative) | Responsibility |
|
||||
|--------|---------------------|----------------|
|
||||
| **Kernel** | `kernel.ts` | Orchestrator — worker pool, signal bus, sense scheduler, workflow manager, optional file watcher and daemon IPC, config reload hooks |
|
||||
| **Kernel** | `kernel.ts` | Orchestrator — worker pool, sense scheduler, workflow manager, optional file watcher and daemon IPC, config reload hooks |
|
||||
| **Worker pool** | `worker-pool.ts` | Fork and supervise one child process per sense group; restart/shutdown; crash cleanup hooks for scheduler state |
|
||||
| **Kernel sense groups** | `kernel-sense-groups.ts` | Derive sense groups from config; list senses per group for scheduling |
|
||||
| **Sense runtime** | sense worker + Drizzle | Per-sense SQLite (`node:sqlite`), migrations, peer DB reads |
|
||||
| **Sense worker** | `sense-worker.ts` (fork target) | Child process entry — runs `compute()` per sense in a group |
|
||||
| **Signal bus** | `signal-bus.ts` | In-memory pub/sub for sense signals |
|
||||
| **Sense scheduler** | `sense-scheduler.ts` | Interval + `on` subscriptions, throttle/coalesce |
|
||||
| **Sense runtime** | `sense-runtime.ts` + sense worker | Loads user modules (`compute`, `initialState`), reads/writes `data/senses/<name>.json` |
|
||||
| **Sense worker** | `sense-worker.ts` (fork target) | Child process entry — runs `compute(state)` per sense in a group |
|
||||
| **Sense scheduler** | `sense-scheduler.ts` | Interval + `on` subscriptions (reverse-index by upstream sense), throttle/coalesce |
|
||||
| **Workflow manager** | `workflow-manager.ts` | One worker per workflow name, concurrency (drop/queue), queue caps |
|
||||
| **Workflow worker** | `workflow-worker.ts` | Child process — runs RFC-002 threads (`start-thread`, `resume-thread` IPC) |
|
||||
| **IPC (parent ↔ workers)** | `ipc.ts` | Typed messages for sense and workflow workers (includes `resume-thread` for recovery) |
|
||||
| **Log / workflow persistence** | via `@uncaged/nerve-store` | Structured logs, `workflow_runs`, thread messages (used for recovery) |
|
||||
| **Blob store** | `@uncaged/nerve-store` | CAS under `data/blobs/` — sense workers construct `createBlobStore(join(nerveRoot, "data", "blobs"))` for artifact writes |
|
||||
| **Blob store** | `@uncaged/nerve-store` | CAS under `data/blobs/` — workflows use blob storage for artifacts as configured |
|
||||
| **File watcher** | `file-watcher.ts` | Watches workspace paths for config / sense / workflow file changes |
|
||||
| **Kernel file watch** | `kernel-file-watch.ts` | Maps watcher events to `reloadConfig`, sense group restart, workflow `drainAndRespawn` |
|
||||
| **Daemon IPC** | `daemon-ipc.ts` | Unix socket server — parses `@uncaged/nerve-core` `DaemonIpcRequest`, dispatches trigger-workflow / trigger-sense / list-senses |
|
||||
@@ -35,9 +34,9 @@ Hot reload (`drainAndRespawn`) uses a controlled drain: in-flight runs may be ma
|
||||
## Key Design Decisions
|
||||
|
||||
- **One worker process per sense group** — isolation between groups, shared compute within a group
|
||||
- **`node:sqlite` (DatabaseSync)** — zero native addons, WAL mode, built into Node.js ≥ 22.5
|
||||
- **Sense state as JSON** — `data/senses/<name>.json`, updated after each successful compute in the worker
|
||||
- **Throttle + coalesce** — if compute is in-flight, at most one pending trigger is queued (no unbounded accumulation)
|
||||
- **Log ≠ Signal** — logs are queryable data assets but cannot trigger the sense scheduler or workflows (prevents feedback loops)
|
||||
- **Log ≠ Sense trigger** — logs are queryable data assets but cannot schedule sense computes or workflows (prevents feedback loops)
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -85,7 +84,7 @@ await kernel.stop();
|
||||
pnpm add @uncaged/nerve-daemon
|
||||
```
|
||||
|
||||
Requires Node.js ≥ 22.5 (for `node:sqlite`).
|
||||
Requires Node.js ≥ 22.5 (for `node:sqlite` in the log store and related persistence).
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -79,7 +79,7 @@ describe("createFileWatcher", () => {
|
||||
await new Promise((r) => setTimeout(r, 100));
|
||||
writeFileSync(
|
||||
join(root, "senses", "cpu-usage", "index.js"),
|
||||
"export async function compute() { return { signal: 42, workflow: null }; }",
|
||||
"export const initialState = {}; export async function compute(state) { return { state, workflow: null }; }",
|
||||
);
|
||||
|
||||
await waitFor(() => changes.length > 0, 3000);
|
||||
|
||||
@@ -21,7 +21,7 @@ export function resolveWorkerScript(): string {
|
||||
export type SenseWorkerPoolOptions = {
|
||||
nerveRoot: string;
|
||||
workerScript: string;
|
||||
/** Invoked for every IPC message from a worker (including ready / signal / error). */
|
||||
/** Invoked for every IPC message from a worker (including ready / compute-result / error). */
|
||||
onWorkerMessage: (raw: unknown) => void;
|
||||
/** Sense names in a group — reserved for scheduler-aligned cleanup (kernel passes current config). */
|
||||
sensesForGroup: (group: string) => string[];
|
||||
|
||||
@@ -41,7 +41,7 @@ export function readNerveYaml(options: ReadNerveYamlOptions): Result<string, Ner
|
||||
* Shared context for workflow agents: how Nerve fits together and common CLI verbs.
|
||||
*/
|
||||
export const nerveAgentContext = `
|
||||
Nerve observes the world through **Senses** (each has its own SQLite DB and a \`compute()\` function).
|
||||
**Reflexes** (YAML) schedule sense runs or start **Workflows** on intervals or signals.
|
||||
The \`nerve\` CLI manages config, triggers, and queries; keep paths and commands aligned with the host nerve.yaml and senses directory.
|
||||
Nerve observes the world through **Senses**: each exports \`compute(state)\`, \`initialState\`, and persists JSON state under \`data/senses/\`.
|
||||
Per-sense \`interval\` and \`on\` in \`nerve.yaml\` schedule computes; a non-null \`workflow\` in the return starts a **Workflow**.
|
||||
The \`nerve\` CLI manages config and triggers; keep paths aligned with the workspace \`nerve.yaml\` and \`senses/\` directory.
|
||||
`.trim();
|
||||
|
||||
Reference in New Issue
Block a user