Compare commits

...

69 Commits

Author SHA1 Message Date
xiaoju 3ce9e3a846 refactor(core): restructure ModeratorContext to { start, steps }
- ModeratorContext: discriminated union → { start: StartStep; steps: RoleStep<M>[] }
- Moderator signature: (context, round, maxRounds) → (context)
- round derivable from steps.length, maxRounds from start.meta.maxRounds
- workflow-worker.ts: build steps array, pass full context to moderator
- Remove unused ModeratorContext import from workflow-worker
- Update README.md

Refs #110
2026-04-25 02:48:28 +00:00
xiaoju 0fff8ef954 Merge pull request 'refactor(core): rename RoleSignal → RoleStep, StartSignal → StartStep' (#116) from refactor/109-role-step into main 2026-04-25 02:37:03 +00:00
xiaoju beada2ae09 refactor(core): rename RoleSignal → RoleStep, StartSignal → StartStep
- RoleStep now includes content and timestamp fields (aligned with StartStep)
- ModeratorContext.signal → ModeratorContext.step
- workflow-utils: start-signal.ts → start-step.ts, isDryRun updated

Fixes #109
2026-04-25 02:34:33 +00:00
xiaoju 47d23bc1a7 Merge pull request 'refactor(store): rename LogEntry.ts → LogEntry.timestamp' (#114) from refactor/113-logentry-timestamp into main 2026-04-25 02:28:38 +00:00
xiaoju 3dc835e1de refactor(store): rename LogEntry/WorkflowRun/ThreadRoundRow ts → timestamp
- Rename logs & workflow_runs table column ts → timestamp (breaking, no migration)
- Update all SQL, types, mocks, CLI output, and tests
- Integration tests use mkdtempSync to avoid stale DB conflicts

Fixes #113
2026-04-25 02:24:39 +00:00
xiaoju 4da2c87a77 refactor(store): rename LogEntry.ts → LogEntry.timestamp
- Rename logs table column ts → timestamp (no migration, breaking)
- Update all SQL, type definitions, and consumers
- Integration tests use mkdtempSync to avoid stale DB conflicts

Fixes #113
2026-04-25 02:08:57 +00:00
xiaoju 529cceba06 Merge pull request 'refactor(core): remove unnecessary | null, unify timestamp naming' (#112) from refactor/108-remove-null-unify-ts into main 2026-04-25 01:57:48 +00:00
xiaoju 020a1bfe85 refactor(core): remove unnecessary | null, unify timestamp naming
- SenseReflexConfig.on: string[] | null → string[] (empty = no conditions)
- NerveConfig.workflows: Record | null → Record (empty = no workflows)
- Signal.ts → Signal.timestamp
- SenseInfo.lastSignalTs → SenseInfo.lastSignalTimestamp
- All consumers across daemon/cli/store updated
- parseNerveConfig: on defaults to [], workflows defaults to {}

Fixes #108
2026-04-25 01:52:58 +00:00
xiaomo 7ce3970027 Merge pull request 'feat(cli): workspace biome.json with noConsole, remove dryRun console.log' (#107) from feat/106-workspace-biome into main 2026-04-25 01:15:53 +00:00
xiaoju fcde29ed1c feat(cli): add biome.json to workspace init, remove dryRun console.log
- init.ts: scaffold biome.json with noConsole: error for workflows
- package.json template: add @biomejs/biome to devDependencies
- workflow-utils: remove console.log from dryRun paths (stub returns
  are captured by log-store via role results)

Fixes #106
2026-04-25 01:00:41 +00:00
xiaomo 611bc48751 Merge pull request 'feat(workflow-utils): dryRun support for spawnSafe, cursorAgent, llmExtract' (#105) from feat/104-dryrun-utils into main 2026-04-25 00:26:27 +00:00
xiaoju 70bea92133 feat(workflow-utils): dryRun support for spawnSafe, cursorAgent, llmExtract
When dryRun=true, each function logs its parameters and returns a stub
result without executing any subprocess or network call. Log output is
captured by log-store for analysis.

- spawnSafe: returns { exitCode: 0, stdout: '[dryRun] skipped' }
- cursorAgent: short-circuits before spawnSafe, returns ok('[dryRun] skipped')
- llmExtract: skips fetch, returns ok({} as T)
- Tests added for spawnSafe and llmExtract dryRun paths

Fixes #104
2026-04-25 00:23:43 +00:00
xiaomo 6f2cddd695 Merge pull request 'feat(core,daemon,cli): add dryRun thread-level parameter to StartSignal' (#103) from feat/101-dry-run into main 2026-04-24 23:50:55 +00:00
xiaoju c4dc707eb0 feat(core,daemon,cli): add dryRun thread-level parameter to StartSignal
- StartSignal.meta gains dryRun: boolean (alongside maxRounds)
- DaemonIpcTriggerWorkflowRequest includes dryRun, parsed with default false
- CLI parses dryRun from --payload JSON, passes through daemon client
- workflow-worker/workflow-manager propagate dryRun through full IPC chain
- Sense-triggered workflows default to dryRun: false
- workflow-utils exports isDryRun(start) helper
- All tests updated, 376 pass

Fixes #101
2026-04-24 23:45:29 +00:00
xiaomo a7ce8401ce Merge pull request 'refactor(core,daemon): extract StartSignal as independent Role parameter' (#102) from refactor/100-extract-start-signal into main 2026-04-24 23:35:09 +00:00
xiaoju e9e6df2f5a refactor(core,daemon): extract StartSignal as independent Role parameter
- Role<Meta> now takes (start: StartSignal, messages: WorkflowMessage[])
- messages no longer contains the __start__ frame
- Add ModeratorContext<M> discriminated union (kind: start | step)
- Moderator receives typed context instead of raw StartSignal | RoleSignal union
- workflow-worker separates start from role messages throughout

Refs #100
2026-04-24 23:14:45 +00:00
xingyue b3b0dad2bb Merge pull request 'feat: add workflow-utils package' (#98) from feat/97-workflow-utils into main 2026-04-24 22:43:07 +00:00
xiaoju e0ce1d995c fix: readNerveYaml returns Result + path traversal guard
Address review feedback:
- Return Result<string, NerveYamlError> instead of throwing
- Add path traversal protection via resolve + startsWith check
- Export NerveYamlError type
- Update sense-generator to handle Result
2026-04-24 22:41:27 +00:00
xiaoju 0a4a2330dc feat: add workflow-utils package
Closes #97
2026-04-24 22:32:29 +00:00
xiaomo d3088c623b Merge pull request 'docs: update all README files to match actual code' (#96) from docs/95-update-readme-to-match-code into main 2026-04-24 21:49:33 +00:00
xiaoju a7e6caf6e7 docs: update all README files to match actual code
Rewrite documentation across all packages to reflect current
architecture, APIs, and CLI commands.

- README.md: fix reflex examples, add store package, update config
- core/README.md: add Sense→workflow routing, IPC types
- daemon/README.md: complete module table, crash recovery, createKernel
- cli/README.md: add workflow/sense/store subcommands
- store/README.md: new file documenting LogStore/BlobStore

Fixes #95
2026-04-24 21:47:37 +00:00
xiaomo d4dcd9722f Merge pull request 'refactor: share IPC message types between CLI and daemon' (#94) from refactor/93-shared-ipc-types into main 2026-04-24 15:14:50 +00:00
xiaoju 3082568b85 refactor(daemon): exhaustive IPC request dispatch
Ensure new DaemonIpcRequest variants require an explicit handler branch.

Made-with: Cursor
2026-04-24 15:11:58 +00:00
xiaoju 830b0aa762 refactor(core): shared daemon IPC request/response types
Move wire protocol types and parseDaemonIpcRequest into @uncaged/nerve-core so CLI and daemon share one definition. Type sendAndReceive message as DaemonIpcRequest. Align workflow trigger CLI with daemon (prompt, maxRounds from --payload JSON).

Made-with: Cursor
2026-04-24 15:10:00 +00:00
xiaoju 777d51cc73 chore: bump version to 0.4.0
小橘 🍊(NEKO Team)
2026-04-24 13:22:30 +00:00
xiaomo 06a957d62a Merge pull request 'chore: add pre-push hook to run tests before push' (#92) from chore/add-pre-push-hook into main 2026-04-24 13:19:10 +00:00
xiaoju b2c379cbfd refactor: reduce cognitive complexity in 3 functions
Extract helpers to bring all functions below biome's complexity threshold (15):
- store/log-store.ts: extract recordToRoundMessage() from parseRoundPayload()
- cli/commands/workflow.ts: extract buildTruncatedSingleRound() from buildThreadCommandOutput()
- daemon/workflow-worker.ts: extract validateRoleResult(), buildInitialLastSignal(),
  initChain(), executeRole() from runThread()

小橘 🍊(NEKO Team)
2026-04-24 12:44:39 +00:00
xiaoju 7cb7112ed6 chore: fix biome lint errors and tune overrides
- Remove duplicate 'prepare' key in package.json
- Allow default exports in rslib.config.ts
- Relax noExplicitAny and noNonNullAssertion in test files
- Auto-fix 17 files (imports, formatting)

小橘 🍊(NEKO Team)
2026-04-24 12:36:57 +00:00
xiaoju 48c81c2e19 chore: add biome lint check to pre-push hook
小橘 🍊(NEKO Team)
2026-04-24 12:32:41 +00:00
xiaoju dd3d4315c4 chore: add pre-push hook to run tests before push
Adds husky with a pre-push hook that runs `pnpm -r test` to catch
test failures before they reach the remote.

小橘 🍊(NEKO Team)
2026-04-24 12:28:47 +00:00
xingyue 788ebc6779 Merge pull request 'fix(test): align tests with type-safety refactor' (#91) from fix/test-failures-after-type-safety-refactor into main 2026-04-24 12:24:50 +00:00
xiaoju 8807b0ac6a fix(test): align tests with type-safety refactor
Update test expectations after workflow reflexes were removed from
YAML config and type signatures were tightened:

- core/config: workflow reflex tests now expect 'not supported' error
- cli/workflow: partitionWorkflowMessage test uses strict typed params
- daemon/crash-recovery: remove triggerPayload from resume-thread assertion
- daemon/daemon-ipc: trigger-workflow sends prompt+maxRounds
- daemon/kernel-workflow: use Sense-driven workflow trigger pattern

Fixes 12 test failures across core, cli, and daemon packages.

Refs #88, #89
2026-04-24 12:23:21 +00:00
xiaomo 5b65afdc4b Merge pull request 'refactor: improve type safety across codebase' (#90) from refactor/type-safety into main 2026-04-24 12:09:36 +00:00
xingyue f5cb72db50 refactor: improve type safety across codebase
- Add isPlainRecord() type guard to eliminate 'as Record<string, unknown>' casts
- Replace 'as any' with properly typed assertions in start.ts
- Remove 'null as unknown as' pattern in kernel.ts
- Add type predicates for array narrowing (item is string)
- Improve IPC message type narrowing in daemon-client.ts and ipc.ts
- Type better-sqlite3 and drizzle return values properly

No runtime behavior changes.
2026-04-24 20:07:58 +08:00
xiaomo e433e7c2a9 Merge pull request 'refactor(daemon): split kernel.ts into focused modules' (#89) from refactor/split-kernel into main 2026-04-24 11:41:44 +00:00
xingyue 47cc49eab4 refactor(daemon): split kernel.ts into focused modules (#86)
- Extract worker-pool.ts (211 LOC): sense worker fork/shutdown/restart/crash recovery
- Extract kernel-file-watch.ts (92 LOC): file change handlers for hot reload
- Extract kernel-sense-groups.ts (29 LOC): group lookup utilities
- kernel.ts reduced from 617 → 380 LOC (thin orchestrator)
- Add worker-pool.test.ts with 8 test cases
- No behavior changes, all existing tests unchanged
2026-04-24 19:39:10 +08:00
xiaomo 65012fbb53 Merge pull request 'refactor(store): extract @uncaged/nerve-store from daemon' (#88) from refactor/extract-nerve-store into main 2026-04-24 11:29:13 +00:00
xingyue 8d00f9cba1 refactor(store): extract @uncaged/nerve-store from daemon (#85)
- Create packages/store/ with log-store, log-archive, blob-store (~900 LOC)
- daemon depends on @uncaged/nerve-store (workspace:*)
- CLI depends on @uncaged/nerve-store, delete daemon-types.ts
- Move store-related tests to packages/store/src/__tests__/
- All store tests pass (73/73), no new regressions
2026-04-24 19:26:46 +08:00
xiaomo ef38b121f7 Merge pull request 'fix: PR #81 review follow-ups (closes #83)' (#84) from fix/pr81-review-followups into main 2026-04-24 11:10:45 +00:00
xiaoju 9bf0b2abb8 fix: PR #81 review follow-ups (closes #83)
- Filter __start__ messages in getThreadRoundCount SQL to fix round offset
- Remove duplicate parseWorkflowField, use parseSenseWorkflowDirective
- Remove unnecessary double casts in workflow CLI
- Add runtime validation for Role meta in workflow-worker
- Export DEFAULT_ENGINE_MAX_ROUNDS from types.ts

小橘 🍊(NEKO Team)
2026-04-24 11:09:31 +00:00
xiaomo d93f5c8fa2 Merge pull request 'refactor(core): restore type-safe workflow automaton from Pulse design' (#81) from refactor/workflow-type-safety into main 2026-04-24 11:02:23 +00:00
xiaoju fa210ec3e0 refactor: restore Pulse-style workflow type safety
- Replace loose payload types with WorkflowLaunchParams { prompt, maxRounds }
- Add SenseResult.workflow field with pipe-separated format (name|rounds|prompt)
- Add parseWorkflowField utility and routeSenseComputeOutput in @nerve/core
- Integrate sense→workflow routing in kernel
- Remove deprecated workflow reflex kind from ReflexScheduler
- Update all test files to use new type-safe interfaces

小橘 🍊(NEKO Team)
2026-04-24 10:58:48 +00:00
xiaoju f72b64d481 refactor(core): restore type-safe workflow automaton from Pulse design (closes #80) 2026-04-24 09:50:28 +00:00
xiaoju b033a98553 chore: bump version to 0.3.0
小橘 <xiaoju@shazhou.work>
2026-04-24 06:08:11 +00:00
xiaomo 68071ffa1e Merge pull request 'feat(cli): add nerve workflow thread <runId> command — closes #77' (#78) from feat/workflow-thread-77 into main 2026-04-24 06:04:32 +00:00
xiaoju f08ad802b0 fix: remove accidentally committed tgz and add to .gitignore
小橘 <xiaoju@shazhou.work>
2026-04-24 06:03:00 +00:00
xiaoju dcfb00128d feat(cli): add nerve workflow thread <runId> command — closes #77
Implements the workflow thread CLI command that retrieves
workflow execution context (logs, events, state) for a given run.

- Add 'nerve workflow thread <runId>' subcommand
- Add log-store query API in daemon
- Add tests for CLI and log-store
- Export new daemon types for thread data

小橘 <xiaoju@shazhou.work>
2026-04-24 05:59:53 +00:00
xiaomo 9cdac05f2c Merge pull request 'docs: add coding agent rules (.cursor/rules + CLAUDE.md)' (#76) from chore/cursor-rules-from-conventions into main 2026-04-23 12:12:08 +00:00
xiaomo 24a8ec927d docs: add core concepts (sense, signal, reflex, workflow) to agent rules 2026-04-23 12:05:31 +00:00
xiaomo 554a79775c docs: add .github/copilot-instructions.md for GitHub Copilot 2026-04-23 12:03:15 +00:00
xiaomo ceb5998fa3 docs: add cursor rules and CLAUDE.md from coding conventions
- .cursor/rules/global.mdc: coding conventions as Cursor agent rules
- CLAUDE.md: same conventions for Claude Code / Hermes agents
- Content derived from docs/coding-conventions.md
- Includes no-dynamic-import rule in CLAUDE.md for completeness
2026-04-23 12:01:03 +00:00
xiaomo 49b5099065 Merge pull request 'fix(daemon): accept string triggerPayload in workflow thread' (#75) from fix/trigger-payload-string-support into main 2026-04-23 11:50:48 +00:00
xiaoju 01d2185495 fix(daemon): accept string triggerPayload in workflow thread
The original code only accepted object-type triggerPayload, silently
discarding string values by replacing them with {}. This meant
`nerve workflow trigger <name> --payload '"some string"'` would
lose the payload entirely.

Changed to `triggerPayload ?? {}` so strings (and other non-null
values) pass through correctly.

小橘 🍊(NEKO Team)
2026-04-23 11:48:05 +00:00
xiaoju 5cedc6a33d release: v0.2.0 — core, daemon, cli 2026-04-23 10:58:49 +00:00
xiaomo c291d3a69a Merge pull request 'feat(cli): add nerve init --from to clone workspace from git' (#74) from feat/init-from-git into main 2026-04-23 10:56:17 +00:00
xiaomo 7960f5af8b Merge pull request 'docs: add comprehensive README for root and all packages' (#73) from docs/readme-update into main 2026-04-23 10:54:43 +00:00
xiaomo 5be14d0d8b docs: add comprehensive README for root and all packages 2026-04-23 10:53:45 +00:00
xiaoju 0e0eb4eec6 feat(cli): add nerve init --from to clone workspace from git
Made-with: Cursor
2026-04-23 10:53:06 +00:00
xiaomo cf2b0ac223 Merge pull request 'build: migrate from tsup to rslib' (#71) from build/tsup-to-rslib into main 2026-04-23 09:50:55 +00:00
xiaoju 1b5a52ea4d build: migrate from tsup to rslib
Replace tsup (esbuild-based) with rslib (rspack-based) across all packages.

tsup's built-in nodeProtocolPlugin strips the 'node:' prefix from all
Node.js builtins. Unlike node:fs etc., node:sqlite has no unprefixed
form, causing ERR_MODULE_NOT_FOUND at runtime. rslib handles node:
imports correctly without any workarounds.

Changes:
- Replace tsup.config.ts with rslib.config.ts in core, daemon, cli
- Swap tsup → @rslib/core in devDependencies
- Fix log-store.ts params type (Record<string, unknown> → Record<string, string | number>)
- Fix logStream.fd type cast in start.ts
- Exclude __tests__ from CLI tsconfig to avoid DTS errors
- All 356 tests pass, nerve init works correctly

Closes #70

小橘 🍊(NEKO Team)
2026-04-23 09:48:45 +00:00
xiaoju a084205b47 Revert "fix: restore node:sqlite prefix stripped by tsup bundler"
This reverts commit 57550ccfdb.
2026-04-23 09:41:28 +00:00
xiaoju 57550ccfdb fix: restore node:sqlite prefix stripped by tsup bundler
tsup's built-in node-protocol-plugin strips the 'node:' prefix from
all builtins. Unlike node:fs etc., node:sqlite has no unprefixed form,
causing ERR_MODULE_NOT_FOUND at runtime.

- Add onSuccess hook to both cli and daemon tsup configs to restore
  'node:sqlite' imports in bundled output
- Fix log-store params type to Record<string, string | number>

小橘 🍊(NEKO Team)
2026-04-23 09:32:20 +00:00
xiaomo 37588df402 Merge pull request 'refactor(daemon): upgrade Drizzle v1.0-beta + migrate better-sqlite3 → node:sqlite' (#69) from refactor/drizzle-v1-node-sqlite into main 2026-04-23 09:20:15 +00:00
xiaoju 85dd11c84d refactor(daemon): upgrade Drizzle v1.0-beta + migrate better-sqlite3 → node:sqlite
- Upgrade drizzle-orm from 0.43.1 to 1.0.0-beta.23
- Replace better-sqlite3 with node:sqlite (DatabaseSync) in:
  - sense-runtime.ts (Drizzle driver)
  - log-store.ts (raw SQL)
  - all test files
- Replace sqlite.pragma() with sqlite.exec('PRAGMA ...')
- Replace sqlite.transaction() with manual BEGIN/COMMIT/ROLLBACK
- Update CLI init command to verify node:sqlite instead of better-sqlite3
- Remove better-sqlite3 and @types/better-sqlite3 from dependencies
- Zero native addons remaining in the monorepo 🎉

Closes #67

小橘 <xiaoju@shazhou.work>
2026-04-23 09:18:44 +00:00
xiaomo d80a414530 Merge pull request 'chore: walkthrough cleanup — engines, types, mock fixes' (#68) from fix/walkthrough-cleanup into main 2026-04-23 09:10:09 +00:00
xiaoju 7f780f0642 chore: walkthrough cleanup — engines, types, mock fixes
- Add engines >= 22.5.0 to root and cli package.json (node:sqlite requirement)
- Remove unused @types/better-sqlite3 from cli devDeps (leftover from sql.js migration)
- Add files/publishConfig to core package.json (parity with other packages)
- Fix daemon test type errors: add getAllWorkflowRuns to mock LogStore,
  fix array destructuring on mock.calls, fix sense-runtime callback signatures

All 356 tests pass across all packages.

小橘 🍊(NEKO Team)
2026-04-23 09:08:24 +00:00
xiaomo 33e0d9a705 Merge pull request 'refactor(cli): replace sql.js with node:sqlite' (#66) from refactor/node-sqlite into main 2026-04-23 08:51:01 +00:00
xiaoju 418d8ee0c8 refactor(cli): replace sql.js with node:sqlite
Drop the sql.js WASM dependency in favour of Node 22's built-in
node:sqlite (DatabaseSync). This eliminates the ~2 MB WASM binary,
removes the async init ceremony, and lets us open databases in
readonly mode directly on disk instead of loading them into memory.

Breaking: requires Node >= 22.5.0 (sqlite support).

- Remove sql.js from cli dependencies
- Rewrite sense-sqlite.ts to use DatabaseSync
- Update sense command (schema/query) — sync API, no more queryAsObjects
- Update tests to use node:sqlite directly
- Remove sql.js from tsup externals

小橘 🍊(NEKO Team)
2026-04-23 08:43:39 +00:00
xiaomo 719c4c1449 Merge pull request 'refactor(cli): replace better-sqlite3 with sql.js (pure WASM) — implements RFC #63' (#64) from refactor/sql-js-migration into main 2026-04-23 07:32:38 +00:00
103 changed files with 6239 additions and 2128 deletions
+189
View File
@@ -0,0 +1,189 @@
---
description: Nerve project coding conventions — style, patterns, and toolchain
globs: packages/*/src/**/*.ts
alwaysApply: true
---
# Nerve Coding Conventions
## Core Concepts
```
External World → Sense → Signal → Reflex → Workflow → Log
↑ ↑
"what to observe" "what to do"
```
**Nerve** is a lightweight observation engine daemon for autonomous agents. It continuously observes external state, reacts to changes via declarative rules, and orchestrates multi-step workflows.
### Key Terms
| Concept | What it is |
|---------|-----------|
| **Sense** | A `compute()` function that samples or derives data. Returns `T \| null` — non-null emits a Signal, null is silent. Each Sense has its own SQLite database. |
| **Signal** | A notification emitted when a Sense returns non-null. Pure fact, no intent. Distributed via an in-memory Signal Bus. Not persisted. |
| **Reflex** | A declarative trigger (YAML) connecting Senses to actions. Trigger types: `interval` (periodic), `on` (react to Signals). Action types: trigger a Sense, or start a Workflow. |
| **Workflow** | A stateful multi-step execution. Contains **Roles** (actors with side effects) and a **Moderator** (pure router). Each instance is a **Thread** with a unique `runId`. |
| **Log** | Immutable audit trail. Records executions, state transitions, errors. **Cannot trigger Reflexes** — prevents feedback loops. |
| **Engine** | The kernel orchestrating everything. Holds Signal Bus, Reflex Scheduler, Process Manager, Workflow Manager. Never loads user code directly — all user code runs in isolated Workers. |
| **Daemon** | The `nerve-daemon` package — engine runtime. Runs as a background process. |
### Architecture Rules
- **Three orthogonal extension points**: Sense (what to compute), Reflex (when to compute), Workflow (what to do)
- **Process isolation**: One worker per Sense group (long-lived), one per Workflow type (on-demand). Workers never talk to each other.
- **Causality is one-directional**: External world → Sense → Signal → Reflex → Action + Log. Logs are the end of the chain.
## Language & Paradigm
### Functional-first
Use `function` + `type`, not `class` + `interface`.
```typescript
// ✅ Good
type Signal = {
senseId: string;
value: unknown;
ts: number;
};
function createSignal(senseId: string, value: unknown): Signal {
return { senseId, value, ts: Date.now() };
}
// ❌ Bad — no class, no interface
class Signal implements ISignal { ... }
```
### Rules
| Rule | Description |
|------|-------------|
| `type` over `interface` | All type definitions use `type` |
| `function` over `class` | Pure functions + closures, no class |
| No `this` | Functions must not depend on `this` context |
| No inheritance | No `extends`, `implements`, `abstract` |
| Composition over inheritance | Use function composition |
| Immutability first | Use `Readonly<T>`, `as const`, avoid mutation |
| No optional properties | Use `T \| null` instead of `?:` — see below |
### Exceptions
Classes are allowed when:
- Required by a third-party library (e.g. Drizzle's `sqliteTable`)
- Error subclasses (`class NerveError extends Error`)
### No Optional Properties
Never use `?:`. All nullable fields must be explicit `T | null`.
```typescript
// ✅ Good
type SenseConfig = {
group: string;
throttle: string | null;
timeout: string | null;
};
// ❌ Bad
type SenseConfig = {
group: string;
throttle?: string;
timeout?: string;
};
```
For mutually exclusive fields, use discriminated unions:
```typescript
// ✅ Good
type ReflexConfig =
| { kind: "sense"; sense: string; interval: string | null; on: string[] | null }
| { kind: "workflow"; workflow: string; on: string[] | null };
```
## Modules & Exports
- Always named exports, never default exports
- One module = one responsibility, filename = purpose
```typescript
// ✅ Named exports only
export function startEngine(config: EngineConfig): Engine { ... }
export type EngineConfig = { ... };
// ❌ No default exports
export default function startEngine() { ... }
```
## Naming
| Type | Style | Example |
|------|-------|---------|
| Files | kebab-case | `signal-bus.ts` |
| Types | PascalCase | `SignalBus` |
| Functions/variables | camelCase | `createSignalBus` |
| Constants | UPPER_SNAKE | `MAX_RETRY_COUNT` |
| Generics | Single letter or descriptive | `T`, `TValue` |
## Error Handling
- Use `Result` type for expected failures
- `throw` only for unrecoverable bugs (programmer errors)
- No try-catch for flow control
```typescript
type Result<T, E = Error> = { ok: true; value: T } | { ok: false; error: E };
function parseSenseConfig(raw: unknown): Result<SenseConfig> { ... }
```
## Async
- Always `async/await`, never `.then()` chains
## Toolchain
| Tool | Purpose |
|------|---------|
| **pnpm** | Package manager |
| **TypeScript** | Type checking (strict mode) |
| **Biome** | Lint + format (replaces ESLint + Prettier) |
| **tsup** | Bundling |
### Commands
```bash
pnpm run check # biome check (lint + format)
pnpm run format # biome format --write
pnpm run build # full build
pnpm test # run tests
```
## Monorepo Structure
```
nerve/
packages/
core/ # @nerve/core — shared types and utils
cli/ # @nerve/cli — CLI entry point
daemon/ # @nerve/daemon — engine runtime
docs/ # RFCs, conventions
biome.json # root Biome config
tsconfig.json # root TypeScript config (composite project references)
```
- `core` is the shared layer; `cli` and `daemon` both depend on it
- `cli` and `daemon` must NOT depend on each other
## Commit Convention
```
<type>(<scope>): <description>
type: feat | fix | refactor | docs | chore | test
scope: core | cli | daemon | rfc-001 | ...
```
+180
View File
@@ -0,0 +1,180 @@
# Nerve Coding Conventions
## Core Concepts
```
External World → Sense → Signal → Reflex → Workflow → Log
↑ ↑
"what to observe" "what to do"
```
**Nerve** is a lightweight observation engine daemon for autonomous agents. It continuously observes external state, reacts to changes via declarative rules, and orchestrates multi-step workflows.
### Key Terms
| Concept | What it is |
|---------|-----------|
| **Sense** | A `compute()` function that samples or derives data. Returns `T \| null` — non-null emits a Signal, null is silent. Each Sense has its own SQLite database. |
| **Signal** | A notification emitted when a Sense returns non-null. Pure fact, no intent. Distributed via an in-memory Signal Bus. Not persisted. |
| **Reflex** | A declarative trigger (YAML) connecting Senses to actions. Trigger types: `interval` (periodic), `on` (react to Signals). Action types: trigger a Sense, or start a Workflow. |
| **Workflow** | A stateful multi-step execution. Contains **Roles** (actors with side effects) and a **Moderator** (pure router). Each instance is a **Thread** with a unique `runId`. |
| **Log** | Immutable audit trail. Records executions, state transitions, errors. **Cannot trigger Reflexes** — prevents feedback loops. |
| **Engine** | The kernel orchestrating everything. Holds Signal Bus, Reflex Scheduler, Process Manager, Workflow Manager. Never loads user code directly — all user code runs in isolated Workers. |
| **Daemon** | The `nerve-daemon` package — engine runtime. Runs as a background process. |
### Architecture Rules
- **Three orthogonal extension points**: Sense (what to compute), Reflex (when to compute), Workflow (what to do)
- **Process isolation**: One worker per Sense group (long-lived), one per Workflow type (on-demand). Workers never talk to each other.
- **Causality is one-directional**: External world → Sense → Signal → Reflex → Action + Log. Logs are the end of the chain.
## Language & Paradigm
### Functional-first
Use `function` + `type`, not `class` + `interface`.
```typescript
// ✅ Good
type Signal = {
senseId: string;
value: unknown;
ts: number;
};
function createSignal(senseId: string, value: unknown): Signal {
return { senseId, value, ts: Date.now() };
}
// ❌ Bad — no class, no interface
class Signal implements ISignal { ... }
```
### Rules
| Rule | Description |
|------|-------------|
| `type` over `interface` | All type definitions use `type` |
| `function` over `class` | Pure functions + closures, no class |
| No `this` | Functions must not depend on `this` context |
| No inheritance | No `extends`, `implements`, `abstract` |
| Composition over inheritance | Use function composition |
| Immutability first | Use `Readonly<T>`, `as const`, avoid mutation |
| No optional properties | Use `T \| null` instead of `?:` — see below |
### Exceptions
Classes are allowed when:
- Required by a third-party library (e.g. Drizzle's `sqliteTable`)
- Error subclasses (`class NerveError extends Error`)
### No Optional Properties
Never use `?:`. All nullable fields must be explicit `T | null`.
```typescript
// ✅ Good
type SenseConfig = {
group: string;
throttle: string | null;
timeout: string | null;
};
// ❌ Bad
type SenseConfig = {
group: string;
throttle?: string;
timeout?: string;
};
```
For mutually exclusive fields, use discriminated unions:
```typescript
// ✅ Good
type ReflexConfig =
| { kind: "sense"; sense: string; interval: string | null; on: string[] | null }
| { kind: "workflow"; workflow: string; on: string[] | null };
```
## Modules & Exports
- Always named exports, never default exports
- One module = one responsibility, filename = purpose
## Naming
| Type | Style | Example |
|------|-------|---------|
| Files | kebab-case | `signal-bus.ts` |
| Types | PascalCase | `SignalBus` |
| Functions/variables | camelCase | `createSignalBus` |
| Constants | UPPER_SNAKE | `MAX_RETRY_COUNT` |
| Generics | Single letter or descriptive | `T`, `TValue` |
## Error Handling
- Use `Result` type for expected failures
- `throw` only for unrecoverable bugs (programmer errors)
- No try-catch for flow control
```typescript
type Result<T, E = Error> = { ok: true; value: T } | { ok: false; error: E };
```
## Async
- Always `async/await`, never `.then()` chains
## No Dynamic Import
Do NOT use `await import()` in production code. Always use static top-level `import`.
Exceptions (must include a comment):
1. `sense-runtime.ts` — user module paths known only at runtime
2. `workflow-worker.ts` — user module paths known only at runtime
Test files (`__tests__/**`) are exempt.
## Toolchain
| Tool | Purpose |
|------|---------|
| **pnpm** | Package manager |
| **TypeScript** | Type checking (strict mode) |
| **Biome** | Lint + format (replaces ESLint + Prettier) |
| **tsup** | Bundling |
### Commands
```bash
pnpm run check # biome check (lint + format)
pnpm run format # biome format --write
pnpm run build # full build
pnpm test # run tests
```
## Monorepo Structure
```
nerve/
packages/
core/ # @nerve/core — shared types and utils
cli/ # @nerve/cli — CLI entry point
daemon/ # @nerve/daemon — engine runtime
docs/ # RFCs, conventions
```
- `core` is the shared layer; `cli` and `daemon` both depend on it
- `cli` and `daemon` must NOT depend on each other
## Commit Convention
```
<type>(<scope>): <description>
type: feat | fix | refactor | docs | chore | test
scope: core | cli | daemon | rfc-001 | ...
```
+1
View File
@@ -2,3 +2,4 @@ node_modules
dist
.turbo
*.tsbuildinfo
*.tgz
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
pnpm check
pnpm -r test
+180
View File
@@ -0,0 +1,180 @@
# Nerve Coding Conventions
## Core Concepts
```
External World → Sense → Signal → Reflex → Workflow → Log
↑ ↑
"what to observe" "what to do"
```
**Nerve** is a lightweight observation engine daemon for autonomous agents. It continuously observes external state, reacts to changes via declarative rules, and orchestrates multi-step workflows.
### Key Terms
| Concept | What it is |
|---------|-----------|
| **Sense** | A `compute()` function that samples or derives data. Returns `T \| null` — non-null emits a Signal, null is silent. Each Sense has its own SQLite database. |
| **Signal** | A notification emitted when a Sense returns non-null. Pure fact, no intent. Distributed via an in-memory Signal Bus. Not persisted. |
| **Reflex** | A declarative trigger (YAML) connecting Senses to actions. Trigger types: `interval` (periodic), `on` (react to Signals). Action types: trigger a Sense, or start a Workflow. |
| **Workflow** | A stateful multi-step execution. Contains **Roles** (actors with side effects) and a **Moderator** (pure router). Each instance is a **Thread** with a unique `runId`. |
| **Log** | Immutable audit trail. Records executions, state transitions, errors. **Cannot trigger Reflexes** — prevents feedback loops. |
| **Engine** | The kernel orchestrating everything. Holds Signal Bus, Reflex Scheduler, Process Manager, Workflow Manager. Never loads user code directly — all user code runs in isolated Workers. |
| **Daemon** | The `nerve-daemon` package — engine runtime. Runs as a background process. |
### Architecture Rules
- **Three orthogonal extension points**: Sense (what to compute), Reflex (when to compute), Workflow (what to do)
- **Process isolation**: One worker per Sense group (long-lived), one per Workflow type (on-demand). Workers never talk to each other.
- **Causality is one-directional**: External world → Sense → Signal → Reflex → Action + Log. Logs are the end of the chain.
## Language & Paradigm
### Functional-first
Use `function` + `type`, not `class` + `interface`.
```typescript
// ✅ Good
type Signal = {
senseId: string;
value: unknown;
ts: number;
};
function createSignal(senseId: string, value: unknown): Signal {
return { senseId, value, ts: Date.now() };
}
// ❌ Bad — no class, no interface
class Signal implements ISignal { ... }
```
### Rules
| Rule | Description |
|------|-------------|
| `type` over `interface` | All type definitions use `type` |
| `function` over `class` | Pure functions + closures, no class |
| No `this` | Functions must not depend on `this` context |
| No inheritance | No `extends`, `implements`, `abstract` |
| Composition over inheritance | Use function composition |
| Immutability first | Use `Readonly<T>`, `as const`, avoid mutation |
| No optional properties | Use `T \| null` instead of `?:` — see below |
### Exceptions
Classes are allowed when:
- Required by a third-party library (e.g. Drizzle's `sqliteTable`)
- Error subclasses (`class NerveError extends Error`)
### No Optional Properties
Never use `?:`. All nullable fields must be explicit `T | null`.
```typescript
// ✅ Good
type SenseConfig = {
group: string;
throttle: string | null;
timeout: string | null;
};
// ❌ Bad
type SenseConfig = {
group: string;
throttle?: string;
timeout?: string;
};
```
For mutually exclusive fields, use discriminated unions:
```typescript
// ✅ Good
type ReflexConfig =
| { kind: "sense"; sense: string; interval: string | null; on: string[] | null }
| { kind: "workflow"; workflow: string; on: string[] | null };
```
## Modules & Exports
- Always named exports, never default exports
- One module = one responsibility, filename = purpose
## Naming
| Type | Style | Example |
|------|-------|---------|
| Files | kebab-case | `signal-bus.ts` |
| Types | PascalCase | `SignalBus` |
| Functions/variables | camelCase | `createSignalBus` |
| Constants | UPPER_SNAKE | `MAX_RETRY_COUNT` |
| Generics | Single letter or descriptive | `T`, `TValue` |
## Error Handling
- Use `Result` type for expected failures
- `throw` only for unrecoverable bugs (programmer errors)
- No try-catch for flow control
```typescript
type Result<T, E = Error> = { ok: true; value: T } | { ok: false; error: E };
```
## Async
- Always `async/await`, never `.then()` chains
## No Dynamic Import
Do NOT use `await import()` in production code. Always use static top-level `import`.
Exceptions (must include a comment):
1. `sense-runtime.ts` — user module paths known only at runtime
2. `workflow-worker.ts` — user module paths known only at runtime
Test files (`__tests__/**`) are exempt.
## Toolchain
| Tool | Purpose |
|------|---------|
| **pnpm** | Package manager |
| **TypeScript** | Type checking (strict mode) |
| **Biome** | Lint + format (replaces ESLint + Prettier) |
| **tsup** | Bundling |
### Commands
```bash
pnpm run check # biome check (lint + format)
pnpm run format # biome format --write
pnpm run build # full build
pnpm test # run tests
```
## Monorepo Structure
```
nerve/
packages/
core/ # @nerve/core — shared types and utils
cli/ # @nerve/cli — CLI entry point
daemon/ # @nerve/daemon — engine runtime
docs/ # RFCs, conventions
```
- `core` is the shared layer; `cli` and `daemon` both depend on it
- `cli` and `daemon` must NOT depend on each other
## Commit Convention
```
<type>(<scope>): <description>
type: feat | fix | refactor | docs | chore | test
scope: core | cli | daemon | rfc-001 | ...
```
+187 -1
View File
@@ -1,3 +1,189 @@
# nerve
Observation engine — Sense, Reflex, Workflow
**Observation engine for autonomous agents**sense the world, react to changes, run workflows.
Nerve is a lightweight daemon that continuously observes external state through **Senses**, reacts via declarative **Reflexes**, and orchestrates multi-step **Workflows**. Built for the [Uncaged](https://github.com/uncaged) agent framework.
## Core Concepts
```
External World → Sense ─┬→ Signal → Reflex → Sense (scheduled compute)
└→ Workflow (Sense return with workflow directive) → Log
```
| Concept | Metaphor | Role |
|---------|----------|------|
| **Sense** | 👁️ Perception | A `compute()` function that samples or derives data. Each sense has its own SQLite database. |
| **Reflex** | ⚡ Reaction | Declarative rules that **only schedule Sense computes** (interval and/or `on` signal names). Reflex YAML cannot reference workflows. |
| **Signal** | 📡 Notification | Emitted when a sense returns a non-null value that is routed as a normal signal (see Sense → Workflow below). Other reflexes can listen via `on`. |
| **Workflow** | 🔧 Action | Stateful multi-step execution with Roles and a Moderator. Started from a Sense return value or from CLI/daemon IPC—not from reflex YAML. |
| **Log** | 📝 Record | Immutable audit trail. Queryable by senses, but **cannot** trigger reflexes (prevents feedback loops). |
**Sense → Workflow:** if `compute()` returns a plain object with a string field `workflow` in the form `name|maxRounds|prompt` (only the first two `|` delimit name and rounds; the rest is the prompt), the engine starts that workflow and **does not** emit a Signal for that return. `workflow: null` or `""` means “emit a signal” and strip the key from the payload. Invalid `workflow` strings are treated like a normal signal (directive stripped). See `@uncaged/nerve-core` `routeSenseComputeOutput` / `parseSenseWorkflowDirective`.
Three extension points for **what / when / multi-step action** — reflexes never replace Sense-driven workflow launches.
## Packages
| Package | Description |
|---------|-------------|
| [`@uncaged/nerve-core`](./packages/core) | Shared types, config parser, Sense→workflow routing, daemon IPC protocol |
| [`@uncaged/nerve-store`](./packages/store) | Append-only log SQLite, JSONL archive, CAS blob store, workflow run rows |
| [`@uncaged/nerve-daemon`](./packages/daemon) | Kernel, workers, signal bus, reflex scheduler, workflow manager, file watcher, IPC |
| [`@uncaged/nerve-cli`](./packages/cli) | CLI (`nerve`) — init, validate, daemon, dev, logs, sense, store, workflow |
## Quick Start
```bash
# Requirements: Node.js ≥ 22.5, pnpm
pnpm add -g @uncaged/nerve-cli
# Initialize a workspace
mkdir my-agent && cd my-agent
nerve init
# Write a sense
cat > senses/cpu-usage/compute.ts << 'EOF'
export async function compute() {
const [load] = (await import("node:os")).loadavg();
return load > 2.0 ? { load } : null; // signal only when load is high
}
EOF
# Configure reflexes in nerve.yaml
cat > nerve.yaml << 'EOF'
senses:
cpu-usage:
group: system
throttle: 10s
reflexes:
- kind: sense
sense: cpu-usage
interval: 30s
EOF
# Run
nerve dev # foreground (development)
nerve daemon start # background (production)
nerve status # check health
nerve logs # view logs
```
## Configuration
`nerve.yaml` declares senses, reflexes (sense-only), optional workflows (concurrency), and optional engine `max_rounds`:
```yaml
max_rounds: 100 # default moderator cap (e.g. CLI workflow trigger)
senses:
cpu-usage:
group: system # senses in the same group share a worker process
throttle: 10s # min interval between computes
timeout: 30s # max compute duration
grace_period: 5s # wait before first compute after startup
reflexes:
- kind: sense
sense: cpu-usage
interval: 30s # periodic trigger
on: [disk-pressure] # also trigger on signals from other senses
workflows:
cleanup:
concurrency: 1
overflow: drop # discard if already running
code-review:
concurrency: 3
overflow: queue
max_queue: 20
```
YAML must **not** include `workflow:` under `reflexes` — the parser rejects it. Declare workflows under `workflows:` and start them from Sense `compute()` or `nerve workflow trigger`.
**Example — Sense starts a workflow** (`senses/disk-pressure/compute.ts`):
```typescript
export async function compute() {
const full = await diskNearlyFull();
if (!full) return null;
return {
path: "/data",
workflow: "cleanup|10|Disk partition nearly full", // name|maxRounds|prompt
};
}
```
## Architecture
```
┌────────────────────────────────────────────────────────────────────────┐
│ Kernel │
│ │
│ ┌──────────────┐ watches nerve.yaml / senses / workflows │
│ │ File Watcher ├──────────────────────────────────────────┐ │
│ └──────────────┘ │ │
│ ┌──────────────┐ CLI ↔ newline JSON (trigger-workflow, │ │
│ │ Daemon IPC │ trigger-sense, list-senses) │ │
│ └──────┬───────┘ ▼ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ │ Worker │ │ Worker │ │ Worker │ (1 per│
│ │ │ (group A)│ │ (group B)│ │ (group C)│ group) │
│ │ │ sense-1 │ │ sense-3 │ │ sense-5 │ │
│ │ │ sense-2 │ │ sense-4 │ │ │ │
│ │ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │ │
│ │ └──────────────┼──────────────┘ │
│ │ ▼ │
│ │ ┌──────────────┐ │
│ │ │ Signal Bus │ │
│ │ └──────┬───────┘ │
│ │ ▼ │
│ │ ┌──────────────────┐ │
│ │ │ Reflex Scheduler│ │
│ │ └────────┬─────────┘ │
│ │ ▼ │
│ │ ┌───────────────────┐ │
│ └───────────────────►│ Workflow Manager │──→ @uncaged/nerve-store │
│ └───────────────────┘ (logs.db, …) │
└────────────────────────────────────────────────────────────────────────┘
```
- **Worker pool** — one child process per sense group; isolation between groups.
- **Signal Bus** — in-memory pub/sub for signal distribution.
- **Reflex Scheduler** — interval timers + signal subscriptions, with throttle/coalesce.
- **Workflow Manager** — concurrency (drop/queue), per-workflow workers, crash recovery.
- **File watcher** — hot reload for config, sense modules, and workflow modules.
- **Daemon IPC** — Unix domain socket; used by the CLI when the daemon is running.
- **Log / blob storage** — implemented in `@uncaged/nerve-store` (WAL SQLite, JSONL archive, CAS blobs).
## Tech Stack
- **Zero native addons** — uses Node.js built-in `node:sqlite` (DatabaseSync)
- **Drizzle ORM** v1.0 for sense databases
- **rslib** (rspack) for building
- **Biome** for formatting/linting
- **Vitest** for testing
- **pnpm** workspaces for monorepo management
## Development
```bash
git clone https://git.shazhou.work/uncaged/nerve.git
cd nerve
pnpm install
pnpm build
pnpm -r test # run all tests
```
## Design Documents
- [RFC-001: Observation Engine](./docs/rfc-001-observation-engine.md) — Sense, Signal, Reflex model
- [RFC-002: Workflow Engine](./docs/rfc-002-workflow-engine.md) — Stateful workflow execution
- [Coding Conventions](./docs/coding-conventions.md)
## License
MIT
+14 -1
View File
@@ -19,7 +19,7 @@
},
"overrides": [
{
"include": ["tsup.config.ts"],
"include": ["tsup.config.ts", "*/rslib.config.ts"],
"linter": {
"rules": {
"style": {
@@ -27,6 +27,19 @@
}
}
}
},
{
"include": ["**/__tests__/**"],
"linter": {
"rules": {
"suspicious": {
"noExplicitAny": "off"
},
"style": {
"noNonNullAssertion": "off"
}
}
}
}
],
"linter": {
+6 -1
View File
@@ -1,14 +1,19 @@
{
"name": "nerve",
"private": true,
"engines": {
"node": ">=22.5.0"
},
"scripts": {
"prepare": "husky",
"build": "pnpm -r run build",
"check": "biome check .",
"format": "biome format --write ."
},
"devDependencies": {
"@biomejs/biome": "^1.9.0",
"tsup": "^8.0.0",
"@rslib/core": "^0.21.3",
"husky": "^9.1.7",
"typescript": "^5.5.0"
}
}
+87
View File
@@ -0,0 +1,87 @@
# @uncaged/nerve-cli
Command-line interface for the [nerve](../../README.md) observation engine.
## Install
```bash
pnpm add -g @uncaged/nerve-cli
# or
npx @uncaged/nerve-cli
```
Requires Node.js ≥ 22.5.
## Commands
### Workspace
```bash
nerve init # Initialize a nerve workspace (installs deps, scaffolds config)
nerve validate # Validate nerve.yaml configuration
```
### Daemon management
```bash
nerve daemon start # Start the daemon (background)
nerve daemon stop # Stop the daemon
nerve daemon status # Show pid, uptime, sense names from nerve.yaml (process must exist)
nerve daemon restart # Stop then start
nerve daemon logs # Tail daemon process logs (file under workspace logs/)
```
### Development
```bash
nerve dev # Foreground kernel with file watcher + IPC (Ctrl+C stops)
```
### Querying & status
```bash
nerve logs # Tail or page the daemon text log file (path in footer; default ~/.uncaged-nerve/logs/nerve.log)
nerve status # Short daemon health summary (aliases daemon status)
```
Structured rows in `data/logs.db` are surfaced via **`nerve workflow inspect`** / **`nerve workflow list`** (and `LogStore` in code), not via `nerve logs`.
### Sense
```bash
nerve sense list # List senses (live fields from daemon IPC when running)
nerve sense trigger <name> # IPC trigger-sense — queue a compute for that sense
nerve sense query <name> # Read-only SQL on data/senses/<name>.db (optional SQL args)
nerve sense schema <name> # Print CREATE TABLE statements for that sense DB
```
### Store maintenance
```bash
nerve store archive # Move old log rows to JSONL under data/archive/logs/… (optional --vacuum)
```
### Workflows
```bash
nerve workflow list # Queued/started runs (add --all for terminal states; --workflow, --limit, --offset)
nerve workflow inspect <runId> # Run metadata + paginated workflow log lines
nerve workflow thread <runId> # Role rounds from persisted messages (--before, --budget)
nerve workflow trigger <name> # IPC trigger-workflow (daemon must be running)
# Optional JSON: --payload '{"prompt":"…","maxRounds":50}'
```
`nerve workflow trigger` sends a `trigger-workflow` line on the daemon Unix socket (same protocol as `@uncaged/nerve-core` / `parseDaemonIpcRequest`). It does not read `nerve.yaml` workflow definitions beyond what the running daemon already loaded.
### Top-level aliases
```bash
nerve start → nerve daemon start
nerve stop → nerve daemon stop
nerve status → nerve daemon status
nerve logs → nerve daemon logs
```
## License
MIT
+9 -8
View File
@@ -1,32 +1,33 @@
{
"name": "@uncaged/nerve-cli",
"version": "0.1.8",
"engines": {
"node": ">=22.5.0"
},
"version": "0.4.0",
"type": "module",
"bin": {
"nerve": "dist/cli.js"
},
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": [
"dist"
],
"files": ["dist"],
"publishConfig": {
"access": "public"
},
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "tsup",
"build": "rslib build",
"test": "vitest run"
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*",
"@uncaged/nerve-store": "workspace:*",
"citty": "^0.1.6",
"sql.js": "^1.14.1"
"yaml": "^2.8.3"
},
"devDependencies": {
"@types/better-sqlite3": "^7.6.13",
"@rslib/core": "^0.21.3",
"@types/node": "^22.0.0",
"@uncaged/nerve-daemon": "workspace:*",
"vitest": "^4.1.5"
}
}
+25
View File
@@ -0,0 +1,25 @@
import { defineConfig } from "@rslib/core";
export default defineConfig({
lib: [
{
format: "esm",
dts: true,
banner: {
js: "#!/usr/bin/env node",
},
},
],
source: {
entry: {
index: "src/index.ts",
cli: "src/cli.ts",
"daemon-bootstrap": "src/daemon-bootstrap.ts",
},
},
output: {
target: "node",
cleanDistPath: true,
externals: ["@uncaged/nerve-daemon", "@uncaged/nerve-store"],
},
});
@@ -1,80 +0,0 @@
/**
* Compile-time check: daemon-types.ts stays in sync with @uncaged/nerve-daemon exports.
* If the daemon package changes its public API, this file will fail to compile.
*/
import type { SenseInfo } from "@uncaged/nerve-core";
import type {
ArchiveLogsDayResult as DaemonArchiveLogsDayResult,
ArchiveLogsOptions as DaemonArchiveLogsOptions,
ArchiveLogsResult as DaemonArchiveLogsResult,
LogEntry as DaemonLogEntry,
LogQuery as DaemonLogQuery,
LogStore as DaemonLogStore,
SenseInfo as DaemonSenseInfo,
WorkflowRun as DaemonWorkflowRun,
WorkflowRunStatus as DaemonWorkflowRunStatus,
} from "@uncaged/nerve-daemon";
import { describe, expectTypeOf, it } from "vitest";
import type {
ArchiveLogsDayResult,
ArchiveLogsOptions,
ArchiveLogsResult,
LogEntry,
LogQuery,
LogStore,
WorkflowRun,
WorkflowRunStatus,
} from "../daemon-types.js";
describe("daemon-types drift guard", () => {
it("SenseInfo matches daemon package export (list-senses IPC)", () => {
expectTypeOf<SenseInfo>().toMatchTypeOf<DaemonSenseInfo>();
expectTypeOf<DaemonSenseInfo>().toMatchTypeOf<SenseInfo>();
});
it("WorkflowRunStatus is assignable both ways", () => {
expectTypeOf<WorkflowRunStatus>().toMatchTypeOf<DaemonWorkflowRunStatus>();
expectTypeOf<DaemonWorkflowRunStatus>().toMatchTypeOf<WorkflowRunStatus>();
});
it("WorkflowRun is assignable both ways", () => {
expectTypeOf<WorkflowRun>().toMatchTypeOf<DaemonWorkflowRun>();
expectTypeOf<DaemonWorkflowRun>().toMatchTypeOf<WorkflowRun>();
});
it("LogEntry is assignable both ways", () => {
expectTypeOf<LogEntry>().toMatchTypeOf<DaemonLogEntry>();
expectTypeOf<DaemonLogEntry>().toMatchTypeOf<LogEntry>();
});
it("LogQuery is assignable both ways", () => {
expectTypeOf<LogQuery>().toMatchTypeOf<DaemonLogQuery>();
expectTypeOf<DaemonLogQuery>().toMatchTypeOf<LogQuery>();
});
it("LogStore has all required methods", () => {
expectTypeOf<LogStore>().toMatchTypeOf<
Pick<
DaemonLogStore,
| "query"
| "getWorkflowRun"
| "getActiveWorkflowRuns"
| "getAllWorkflowRuns"
| "upsertWorkflowRun"
| "archiveLogs"
| "close"
>
>();
});
it("ArchiveLogs types match daemon", () => {
expectTypeOf<ArchiveLogsOptions>().toMatchTypeOf<DaemonArchiveLogsOptions>();
expectTypeOf<DaemonArchiveLogsOptions>().toMatchTypeOf<ArchiveLogsOptions>();
expectTypeOf<ArchiveLogsResult>().toMatchTypeOf<DaemonArchiveLogsResult>();
expectTypeOf<DaemonArchiveLogsResult>().toMatchTypeOf<ArchiveLogsResult>();
expectTypeOf<ArchiveLogsDayResult>().toMatchTypeOf<DaemonArchiveLogsDayResult>();
expectTypeOf<DaemonArchiveLogsDayResult>().toMatchTypeOf<ArchiveLogsDayResult>();
});
});
+10 -2
View File
@@ -234,7 +234,11 @@ describe("logsCommand negative offset", () => {
it("exits with code 1 and writes to stderr when offset is negative", async () => {
await expect(
logsCommand.run!({ args: { n: "50", offset: "-5", follow: false }, rawArgs: [], cmd: logsCommand as never }),
logsCommand.run?.({
args: { n: "50", offset: "-5", follow: false },
rawArgs: [],
cmd: logsCommand as never,
}),
).rejects.toThrow("process.exit(1)");
expect(exitCode).toBe(1);
expect(stderrOutput).toContain("--offset must be a non-negative integer");
@@ -243,7 +247,11 @@ describe("logsCommand negative offset", () => {
it("exits with code 1 for offset=-1", async () => {
await expect(
logsCommand.run!({ args: { n: "10", offset: "-1", follow: false }, rawArgs: [], cmd: logsCommand as never }),
logsCommand.run?.({
args: { n: "10", offset: "-1", follow: false },
rawArgs: [],
cmd: logsCommand as never,
}),
).rejects.toThrow("process.exit(1)");
expect(exitCode).toBe(1);
});
+37 -11
View File
@@ -29,10 +29,22 @@ const SAMPLE_SENSES: SenseInfo[] = [
group: "system",
throttle: 5000,
timeout: 3000,
lastSignalTs: 1_700_000_000_000,
lastSignalTimestamp: 1_700_000_000_000,
},
{
name: "disk-usage",
group: "system",
throttle: 30000,
timeout: null,
lastSignalTimestamp: null,
},
{
name: "active-tasks",
group: "tasks",
throttle: 10000,
timeout: 30000,
lastSignalTimestamp: null,
},
{ name: "disk-usage", group: "system", throttle: 30000, timeout: null, lastSignalTs: null },
{ name: "active-tasks", group: "tasks", throttle: 10000, timeout: 30000, lastSignalTs: null },
];
// ---------------------------------------------------------------------------
@@ -100,14 +112,14 @@ describe("formatSenseList", () => {
expect(output).toContain("—");
});
it("shows '(never)' when lastSignalTs is null", () => {
it("shows '(never)' when lastSignalTimestamp is null", () => {
const output = formatSenseList(SAMPLE_SENSES);
expect(output).toContain("(never)");
});
it("shows ISO timestamp when lastSignalTs is set", () => {
it("shows ISO timestamp when lastSignalTimestamp is set", () => {
const output = formatSenseList(SAMPLE_SENSES);
// cpu-usage has lastSignalTs = 1_700_000_000_000
// cpu-usage has lastSignalTimestamp = 1_700_000_000_000
expect(output).toContain(new Date(1_700_000_000_000).toISOString());
});
});
@@ -157,11 +169,19 @@ reflexes: []
);
const result = sensesFromConfig(path);
expect(result).toHaveLength(2);
expect(result[0]).toMatchObject({ name: "cpu-usage", group: "system", lastSignalTs: null });
expect(result[1]).toMatchObject({ name: "disk-usage", group: "system", lastSignalTs: null });
expect(result[0]).toMatchObject({
name: "cpu-usage",
group: "system",
lastSignalTimestamp: null,
});
expect(result[1]).toMatchObject({
name: "disk-usage",
group: "system",
lastSignalTimestamp: null,
});
});
it("always sets lastSignalTs to null (static fallback)", () => {
it("always sets lastSignalTimestamp to null (static fallback)", () => {
const path = join(tmpDir, "nerve.yaml");
writeFileSync(
path,
@@ -173,7 +193,7 @@ reflexes: []
`.trim(),
);
const result = sensesFromConfig(path);
expect(result[0].lastSignalTs).toBeNull();
expect(result[0].lastSignalTimestamp).toBeNull();
});
it("populates throttle and timeout from config", () => {
@@ -238,7 +258,13 @@ describe("listSensesViaDaemon", () => {
it("resolves with populated senses array", async () => {
const senses: SenseInfo[] = [
{ name: "cpu-usage", group: "system", throttle: 5000, timeout: 3000, lastSignalTs: 12345 },
{
name: "cpu-usage",
group: "system",
throttle: 5000,
timeout: 3000,
lastSignalTimestamp: 12345,
},
];
const server = createServer((s) => {
s.on("data", () => {
+28 -51
View File
@@ -2,12 +2,12 @@
* Tests for sense SQLite helpers used by `nerve sense schema` / `nerve sense query`.
*/
import { mkdirSync, rmSync, writeFileSync } from "node:fs";
import { mkdirSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { DatabaseSync } from "node:sqlite";
import initSqlJs, { type Database } from "sql.js";
import { afterEach, beforeAll, beforeEach, describe, expect, it } from "vitest";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import {
assertSenseDbExists,
@@ -17,17 +17,11 @@ import {
listTableSqlStatements,
parseSenseQueryArgs,
pickDefaultPreviewTable,
queryAsObjects,
senseDbPath,
} from "../sense-sqlite.js";
let SQL: Awaited<ReturnType<typeof initSqlJs>>;
let tmpDir: string;
beforeAll(async () => {
SQL = await initSqlJs();
});
beforeEach(() => {
tmpDir = join(
tmpdir(),
@@ -40,22 +34,6 @@ afterEach(() => {
rmSync(tmpDir, { recursive: true, force: true });
});
/** Helper: create a SQLite db file with the given setup SQL. */
function createDb(name: string, setupSql: string): void {
const db = new SQL.Database();
db.run(setupSql);
const data = db.export();
db.close();
writeFileSync(join(tmpDir, "data", "senses", `${name}.db`), Buffer.from(data));
}
/** Helper: open an in-memory db with setup SQL for unit tests. */
function memDb(setupSql?: string): Database {
const db = new SQL.Database();
if (setupSql) db.run(setupSql);
return db;
}
describe("senseDbPath", () => {
it("points at data/senses/<name>.db under the given root", () => {
expect(senseDbPath("/root", "cpu-usage")).toBe(join("/root", "data", "senses", "cpu-usage.db"));
@@ -68,14 +46,18 @@ describe("assertSenseDbExists", () => {
});
it("returns the path when the file exists", () => {
createDb("x", "SELECT 1");
expect(assertSenseDbExists(tmpDir, "x")).toBe(join(tmpDir, "data", "senses", "x.db"));
const p = join(tmpDir, "data", "senses", "x.db");
new DatabaseSync(p).close();
expect(assertSenseDbExists(tmpDir, "x")).toBe(p);
});
});
describe("listTableSqlStatements", () => {
it("returns CREATE statements ordered by tbl_name", () => {
const db = memDb("CREATE TABLE zebra (id INTEGER); CREATE TABLE alpha (id INTEGER);");
const p = join(tmpDir, "data", "senses", "t.db");
const db = new DatabaseSync(p);
db.exec("CREATE TABLE zebra (id INTEGER)");
db.exec("CREATE TABLE alpha (id INTEGER)");
const stmts = listTableSqlStatements(db);
db.close();
expect(stmts).toHaveLength(2);
@@ -86,16 +68,18 @@ describe("listTableSqlStatements", () => {
describe("pickDefaultPreviewTable", () => {
it("prefers non-_migrations tables when both exist", () => {
const db = memDb(
`CREATE TABLE _migrations (name TEXT PRIMARY KEY);
CREATE TABLE readings (id INTEGER);`,
);
const p = join(tmpDir, "data", "senses", "t.db");
const db = new DatabaseSync(p);
db.exec("CREATE TABLE _migrations (name TEXT PRIMARY KEY)");
db.exec("CREATE TABLE readings (id INTEGER)");
expect(pickDefaultPreviewTable(db)).toBe("readings");
db.close();
});
it("uses _migrations when it is the only table", () => {
const db = memDb("CREATE TABLE _migrations (name TEXT PRIMARY KEY);");
const p = join(tmpDir, "data", "senses", "t.db");
const db = new DatabaseSync(p);
db.exec("CREATE TABLE _migrations (name TEXT PRIMARY KEY)");
expect(pickDefaultPreviewTable(db)).toBe("_migrations");
db.close();
});
@@ -153,29 +137,22 @@ describe("collectColumnKeys", () => {
});
});
describe("queryAsObjects", () => {
it("converts columnar sql.js results to row objects", () => {
const db = memDb("CREATE TABLE t (x INTEGER, y TEXT); INSERT INTO t VALUES (1, 'a'), (2, 'b');");
const rows = queryAsObjects(db, "SELECT * FROM t ORDER BY x");
db.close();
expect(rows).toEqual([
{ x: 1, y: "a" },
{ x: 2, y: "b" },
]);
});
});
describe("readonly query integration", () => {
it("runs default preview SQL on a real db file", () => {
createDb("demo", "CREATE TABLE items (id INTEGER PRIMARY KEY, v TEXT); INSERT INTO items (v) VALUES ('a'), ('b');");
it("runs default preview SQL on a real db", () => {
const p = join(tmpDir, "data", "senses", "demo.db");
const rw = new DatabaseSync(p);
rw.exec("CREATE TABLE items (id INTEGER PRIMARY KEY, v TEXT)");
rw.exec("INSERT INTO items (v) VALUES ('a'), ('b')");
rw.close();
const buffer = require("node:fs").readFileSync(join(tmpDir, "data", "senses", "demo.db"));
const db = new SQL.Database(buffer);
const db = new DatabaseSync(p, { readOnly: true });
const table = pickDefaultPreviewTable(db);
expect(table).toBe("items");
if (table === null) throw new Error("expected items table");
if (table === null) {
throw new Error("expected items table");
}
const sql = defaultPreviewSql(table);
const rows = queryAsObjects(db, sql);
const rows = db.prepare(sql).all() as Record<string, unknown>[];
db.close();
expect(rows.length).toBeGreaterThanOrEqual(1);
});
+122 -26
View File
@@ -12,19 +12,23 @@ import { createServer } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createLogStore } from "@uncaged/nerve-daemon";
import { createLogStore } from "@uncaged/nerve-store";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import type { LogStore, ThreadRoundRow, WorkflowRun } from "@uncaged/nerve-store";
import {
DEFAULT_THREAD_BUDGET_CHARS,
buildInspectOutput,
buildListOutput,
buildThreadCommandOutput,
formatThreadRoundBlock,
formatTs,
getAllWorkflowRuns,
parseIntArg,
partitionWorkflowMessage,
statusIcon,
} from "../commands/workflow.js";
import { triggerWorkflowViaDaemon } from "../daemon-client.js";
import type { LogStore, WorkflowRun } from "../daemon-types.js";
// ---------------------------------------------------------------------------
// Test helpers
@@ -37,11 +41,11 @@ function upsertRun(
runId: string,
workflow: string,
status: WorkflowRun["status"],
ts: number,
timestampMs: number,
): void {
store.upsertWorkflowRun(
{ source: "workflow", type: status, refId: runId, payload: null, ts },
{ runId, workflow, status, ts },
{ source: "workflow", type: status, refId: runId, payload: null, timestamp: timestampMs },
{ runId, workflow, status, timestamp: timestampMs },
);
}
@@ -61,8 +65,8 @@ afterEach(() => {
describe("formatTs", () => {
it("returns ISO 8601 string", () => {
const ts = new Date("2026-01-01T00:00:00.000Z").getTime();
expect(formatTs(ts)).toBe("2026-01-01T00:00:00.000Z");
const timestampMs = new Date("2026-01-01T00:00:00.000Z").getTime();
expect(formatTs(timestampMs)).toBe("2026-01-01T00:00:00.000Z");
});
});
@@ -123,14 +127,14 @@ describe("getAllWorkflowRuns", () => {
}
});
it("sorts by ts descending (newest first)", () => {
it("sorts by timestamp descending (newest first)", () => {
upsertRun("r1", "cleanup", "completed", 1000);
upsertRun("r2", "cleanup", "started", 3000);
upsertRun("r3", "cleanup", "failed", 2000);
const runs = getAllWorkflowRuns(store, null);
expect(runs[0].ts).toBeGreaterThan(runs[1].ts);
expect(runs[1].ts).toBeGreaterThan(runs[2].ts);
expect(runs[0].timestamp).toBeGreaterThan(runs[1].timestamp);
expect(runs[1].timestamp).toBeGreaterThan(runs[2].timestamp);
});
});
@@ -143,9 +147,9 @@ describe("buildListOutput", () => {
runId: string,
workflow: string,
status: WorkflowRun["status"],
ts: number,
timestampMs: number,
): WorkflowRun {
return { runId, workflow, status, ts };
return { runId, workflow, status, timestamp: timestampMs };
}
it("returns empty message when no runs and --all=false", () => {
@@ -231,7 +235,7 @@ describe("buildInspectOutput", () => {
runId: "run-xyz",
workflow: "cleanup",
status: "completed",
ts: 1_700_000_000_000,
timestamp: 1_700_000_000_000,
};
it("shows header with run details", () => {
@@ -247,8 +251,8 @@ describe("buildInspectOutput", () => {
expect(eventLines.join("")).toContain("no events recorded");
});
it("shows event lines with type and ts", () => {
const logs = [{ ts: 1_700_000_001_000, type: "started", payload: null }];
it("shows event lines with type and timestamp", () => {
const logs = [{ timestamp: 1_700_000_001_000, type: "started", payload: null }];
const { eventLines } = buildInspectOutput(baseRun, logs, 0, 20);
const text = eventLines.join("");
expect(text).toContain("type=started");
@@ -256,7 +260,7 @@ describe("buildInspectOutput", () => {
it("truncates long payloads to 200 chars with ellipsis", () => {
const longPayload = "x".repeat(250);
const logs = [{ ts: 1000, type: "step_complete", payload: longPayload }];
const logs = [{ timestamp: 1000, type: "step_complete", payload: longPayload }];
const { eventLines } = buildInspectOutput(baseRun, logs, 0, 20);
const text = eventLines.join("");
expect(text).toContain("…");
@@ -264,14 +268,14 @@ describe("buildInspectOutput", () => {
});
it("shows short payloads in full", () => {
const logs = [{ ts: 1000, type: "step_complete", payload: '{"count":5}' }];
const logs = [{ timestamp: 1000, type: "step_complete", payload: '{"count":5}' }];
const { eventLines } = buildInspectOutput(baseRun, logs, 0, 20);
expect(eventLines.join("")).toContain('{"count":5}');
});
it("paginates events with a hint", () => {
const logs = Array.from({ length: 5 }, (_, i) => ({
ts: 1000 + i,
timestamp: 1000 + i,
type: "step_complete",
payload: null,
}));
@@ -283,7 +287,7 @@ describe("buildInspectOutput", () => {
});
it("no pagination hint when all events fit on one page", () => {
const logs = [{ ts: 1000, type: "started", payload: null }];
const logs = [{ timestamp: 1000, type: "started", payload: null }];
const { paginationHint } = buildInspectOutput(baseRun, logs, 0, 20);
expect(paginationHint).toBeNull();
});
@@ -322,6 +326,98 @@ describe("workflow list — integration with real store", () => {
});
});
// ---------------------------------------------------------------------------
// nerve workflow thread — formatting helpers
// ---------------------------------------------------------------------------
describe("partitionWorkflowMessage", () => {
it("extracts role, content, and meta", () => {
const p = partitionWorkflowMessage({
role: "scanner",
content: "ok",
meta: { items: [1, 2] },
});
expect(p.roleStr).toBe("scanner");
expect(p.contentBody).toBe("ok");
expect(p.meta).toEqual({ items: [1, 2] });
});
it("passes through role and content as-is", () => {
const p = partitionWorkflowMessage({
role: "unknown",
content: '{"n":1}',
meta: null,
timestamp: 0,
});
expect(p.roleStr).toBe("unknown");
expect(p.contentBody).toBe('{"n":1}');
});
});
describe("formatThreadRoundBlock", () => {
const row: ThreadRoundRow = {
round: 2,
logId: 99,
timestamp: new Date("2026-01-02T03:04:05.006Z").getTime(),
message: { role: "bot", content: "hi", meta: { score: 0.5 }, timestamp: 1735783445006 },
};
it("includes header, YAML frontmatter for meta, and body", () => {
const text = formatThreadRoundBlock(row);
expect(text).toContain("[#2 bot]");
expect(text).toContain("---\n");
expect(text).toContain("score: 0.5");
expect(text).toContain("hi");
});
});
describe("buildThreadCommandOutput", () => {
function row(n: number, content: string): ThreadRoundRow {
return {
round: n,
logId: 10 + n,
timestamp: 1000 + n,
message: { role: "r", content, meta: { extra: n }, timestamp: 1000 + n },
};
}
it("orders rounds chronologically (oldest first in output)", () => {
const desc = [row(3, "ccc"), row(2, "bbb"), row(1, "aaa")];
const prefix = ["HEADER\n"];
const { lines, paginationHint } = buildThreadCommandOutput(prefix, desc, 50_000, "run-x");
const text = lines.join("");
const idxA = text.indexOf("\naaa\n");
const idxB = text.indexOf("\nbbb\n");
const idxC = text.indexOf("\nccc\n");
expect(idxA).toBeGreaterThan(-1);
expect(idxB).toBeGreaterThan(idxA);
expect(idxC).toBeGreaterThan(idxB);
expect(paginationHint).toBeNull();
});
it("emits pagination hint with --before when oldest shown round is still > 1", () => {
const desc = [row(4, "d"), row(3, "c")];
const { paginationHint } = buildThreadCommandOutput([], desc, 50_000, "run-y");
expect(paginationHint).toContain("--before 3");
expect(paginationHint).toContain("run-y");
});
it("respects budget and hints with non-default --budget in command", () => {
const big = "y".repeat(500);
const desc = [row(2, big), row(1, "a")];
const { lines, paginationHint } = buildThreadCommandOutput([], desc, 400, "run-z");
const text = lines.join("");
expect(text).toContain("[#2");
expect(text).not.toContain("[#1");
expect(paginationHint).toContain("--before 2");
expect(paginationHint).toContain("--budget 400");
});
it("default budget constant matches workflow command default", () => {
expect(DEFAULT_THREAD_BUDGET_CHARS).toBe(8000);
});
});
// ---------------------------------------------------------------------------
// parseIntArg
// ---------------------------------------------------------------------------
@@ -366,15 +462,15 @@ describe("getAllWorkflowRuns — uses store.getAllWorkflowRuns SQL path", () =>
expect(runs).toHaveLength(7);
});
it("returns runs sorted by ts descending (newest first)", () => {
it("returns runs sorted by timestamp descending (newest first)", () => {
upsertRun("r1", "deploy", "completed", 1000);
upsertRun("r2", "deploy", "completed", 3000);
upsertRun("r3", "deploy", "completed", 2000);
const runs = getAllWorkflowRuns(store, null);
expect(runs[0].ts).toBe(3000);
expect(runs[1].ts).toBe(2000);
expect(runs[2].ts).toBe(1000);
expect(runs[0].timestamp).toBe(3000);
expect(runs[1].timestamp).toBe(2000);
expect(runs[2].timestamp).toBe(1000);
});
it("filters by workflow name", () => {
@@ -418,7 +514,7 @@ describe("triggerWorkflowViaDaemon", () => {
await new Promise<void>((r) => server.listen(sockPath, r));
try {
const result = await triggerWorkflowViaDaemon(sockPath, "my-workflow", {});
const result = await triggerWorkflowViaDaemon(sockPath, "my-workflow", "", 100);
expect(result).toEqual({ ok: true });
} finally {
await new Promise<void>((r) => server.close(() => r()));
@@ -434,7 +530,7 @@ describe("triggerWorkflowViaDaemon", () => {
await new Promise<void>((r) => server.listen(sockPath, r));
try {
const result = await triggerWorkflowViaDaemon(sockPath, "missing", {});
const result = await triggerWorkflowViaDaemon(sockPath, "missing", "", 100);
expect(result).toEqual({ ok: false, error: "unknown workflow" });
} finally {
await new Promise<void>((r) => server.close(() => r()));
@@ -442,7 +538,7 @@ describe("triggerWorkflowViaDaemon", () => {
});
it("rejects when no daemon is listening on the socket", async () => {
await expect(triggerWorkflowViaDaemon(sockPath, "my-workflow", {})).rejects.toThrow(
await expect(triggerWorkflowViaDaemon(sockPath, "my-workflow", "", 100)).rejects.toThrow(
/Cannot connect to daemon/,
);
});
+21 -1
View File
@@ -12,6 +12,26 @@ import { storeCommand } from "./commands/store.js";
import { validateCommand } from "./commands/validate.js";
import { workflowCommand } from "./commands/workflow.js";
/**
* Citty picks the first non-flag token as a subcommand name. Rewrite
* `nerve init --from <url>` so the URL is not mistaken for `workflow`/`workspace`.
*/
function normalizeNerveArgv(argv: string[]): string[] {
const initIdx = argv.indexOf("init");
if (initIdx === -1) return argv;
const tail = argv.slice(initIdx + 1);
const fromAt = tail.indexOf("--from");
if (fromAt === -1) return argv;
const beforeFrom = tail.slice(0, fromAt);
if (beforeFrom.some((a) => !a.startsWith("-"))) return argv;
const next = tail[fromAt + 1];
if (next === undefined || next.startsWith("-")) return argv;
const reserved = new Set(["workflow", "workspace"]);
if (reserved.has(next)) return argv;
const mergedTail = [...tail.slice(0, fromAt), `--from=${next}`, ...tail.slice(fromAt + 2)];
return [...argv.slice(0, initIdx + 1), ...mergedTail];
}
const main = defineCommand({
meta: {
name: "nerve",
@@ -32,4 +52,4 @@ const main = defineCommand({
},
});
runMain(main);
runMain(main, { rawArgs: normalizeNerveArgv(process.argv.slice(2)) });
+126 -32
View File
@@ -1,5 +1,5 @@
import { spawn, execFile } from "node:child_process";
import { existsSync, mkdirSync, writeFileSync } from "node:fs";
import { execFile, spawn } from "node:child_process";
import { existsSync, mkdirSync, readdirSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { promisify } from "node:util";
@@ -21,6 +21,31 @@ reflexes:
interval: 10s
`;
const BIOME_JSON = `{
"$schema": "https://biomejs.dev/schemas/1.9.0/schema.json",
"formatter": {
"indentStyle": "space",
"indentWidth": 2,
"lineWidth": 100
},
"javascript": {
"formatter": {
"quoteStyle": "double",
"semicolons": "always"
}
},
"linter": {
"enabled": true,
"rules": {
"recommended": true,
"suspicious": {
"noConsole": "error"
}
}
}
}
`;
const PACKAGE_JSON = `{
"name": "my-nerve-workspace",
"version": "0.0.1",
@@ -32,10 +57,11 @@ const PACKAGE_JSON = `{
"drizzle-orm": "latest"
},
"devDependencies": {
"@biomejs/biome": "latest",
"drizzle-kit": "latest"
},
"pnpm": {
"onlyBuiltDependencies": ["better-sqlite3", "esbuild"]
"onlyBuiltDependencies": ["esbuild"]
}
}
`;
@@ -218,20 +244,94 @@ const initWorkspaceCommand = defineCommand({
},
});
async function tryRequireSqlite(nerveRoot: string): Promise<boolean> {
/** Verify built-in `node:sqlite` (Node.js ≥22.5) loads in a child process. */
async function verifyNodeSqlite(): Promise<boolean> {
try {
const modulePath = join(nerveRoot, "node_modules", "better-sqlite3");
// Use a child process to test if the native module loads
await execFileAsync("node", ["-e", `require(${JSON.stringify(modulePath)})`], {
cwd: nerveRoot,
timeout: 10_000,
});
await execFileAsync(
"node",
[
"--input-type=module",
"-e",
"import { DatabaseSync } from 'node:sqlite'; new DatabaseSync(':memory:').exec('SELECT 1');",
],
{ timeout: 10_000 },
);
return true;
} catch {
return false;
}
}
function isNerveRootNonEmpty(nerveRoot: string): boolean {
if (!existsSync(nerveRoot)) return false;
return readdirSync(nerveRoot).length > 0;
}
async function runInitFromGit(url: string): Promise<void> {
const trimmed = url.trim();
if (trimmed.length === 0) {
process.stderr.write("❌ --from requires a non-empty git URL.\n");
process.exit(1);
}
const nerveRoot = getNerveRoot();
if (isNerveRootNonEmpty(nerveRoot)) {
process.stderr.write(
`${nerveRoot} already exists and is not empty. Remove it (or empty it) before using --from.\n`,
);
process.exit(1);
}
try {
await execFileAsync("git", ["--version"]);
} catch {
process.stderr.write("❌ git is not available. Install git and retry.\n");
process.exit(1);
}
try {
await execFileAsync("pnpm", ["--version"]);
} catch {
process.stderr.write("❌ pnpm is not available. Install pnpm and retry.\n");
process.exit(1);
}
process.stdout.write(`Cloning ${trimmed}${nerveRoot}\n`);
try {
await runCommand("git", ["clone", trimmed, nerveRoot], process.cwd());
} catch {
process.stderr.write("❌ git clone failed.\n");
process.exit(1);
}
if (!existsSync(join(nerveRoot, "nerve.yaml"))) {
process.stdout.write(`⚠️ ${join(nerveRoot, "nerve.yaml")} not found after clone.\n`);
}
if (!existsSync(join(nerveRoot, "package.json"))) {
process.stdout.write(`⚠️ ${join(nerveRoot, "package.json")} not found after clone.\n`);
}
process.stdout.write("Installing dependencies with pnpm …\n");
try {
await runCommand("pnpm", ["install", "--no-cache"], nerveRoot);
} catch {
process.stdout.write(
`⚠️ pnpm install failed. Try manually:\n cd ${nerveRoot} && pnpm install --no-cache\n`,
);
}
if (!(await verifyNodeSqlite())) {
process.stdout.write(
"⚠️ Built-in SQLite (node:sqlite) is not available in this Node.js build. " +
"The daemon requires Node.js 22.5 or newer with SQLite enabled.\n",
);
}
process.stdout.write(
`✅ Workspace cloned to ${nerveRoot}\n\n💡 Next steps:\n 1. Review nerve.yaml and install any missing tooling.\n 2. Run \`nerve start\` to launch the daemon.\n`,
);
}
async function runInitWorkspace(force: boolean): Promise<void> {
const nerveRoot = getNerveRoot();
@@ -246,6 +346,7 @@ async function runInitWorkspace(force: boolean): Promise<void> {
writeFile(join(nerveRoot, "nerve.yaml"), NERVE_YAML);
writeFile(join(nerveRoot, "package.json"), PACKAGE_JSON);
writeFile(join(nerveRoot, "biome.json"), BIOME_JSON);
writeFile(join(nerveRoot, ".gitignore"), GITIGNORE);
writeFile(join(nerveRoot, "senses", "cpu-usage", "schema.ts"), CPU_SCHEMA_TS);
writeFile(join(nerveRoot, "senses", "cpu-usage", "index.js"), CPU_INDEX_JS);
@@ -264,27 +365,11 @@ async function runInitWorkspace(force: boolean): Promise<void> {
);
}
// Verify better-sqlite3 native module — rebuild up to 2 times if broken
const sqlitePath = join(nerveRoot, "node_modules", "better-sqlite3");
if (existsSync(sqlitePath)) {
for (let attempt = 1; attempt <= 2; attempt++) {
if (await tryRequireSqlite(nerveRoot)) break;
process.stdout.write(
`${attempt === 1 ? "Building" : "Retrying build of"} native module better-sqlite3 (attempt ${attempt}/2)…\n`,
);
try {
await runCommand(cmd, ["rebuild", "better-sqlite3"], nerveRoot);
} catch {
// will be caught by the verify below
}
}
if (!(await tryRequireSqlite(nerveRoot))) {
process.stdout.write(
`⚠️ better-sqlite3 native module is not working. The daemon will fail to start.\n` +
` Fix: cd ${nerveRoot} && ${cmd} rebuild better-sqlite3\n` +
` Or: npm install --build-from-source better-sqlite3\n`,
);
}
if (!(await verifyNodeSqlite())) {
process.stdout.write(
"⚠️ Built-in SQLite (node:sqlite) is not available in this Node.js build. " +
"The daemon requires Node.js 22.5 or newer with SQLite enabled.\n",
);
}
if (!existsSync(join(nerveRoot, ".git"))) {
@@ -306,7 +391,7 @@ export const initCommand = defineCommand({
meta: {
name: "init",
description:
"Initialize workspace (nerve init) or scaffold templates (nerve init workflow <name>)",
"Initialize workspace (nerve init), clone from git (nerve init --from <url>), or scaffold templates (nerve init workflow <name>)",
},
args: {
force: {
@@ -314,12 +399,21 @@ export const initCommand = defineCommand({
description: "Reinitialize even if workspace already exists (preserves data/)",
default: false,
},
from: {
type: "string",
description: "Clone an existing git repo into ~/.uncaged-nerve instead of scaffolding",
required: false,
},
},
subCommands: {
workflow: initWorkflowCommand,
workspace: initWorkspaceCommand,
},
async run({ args }) {
if (args.from !== undefined) {
await runInitFromGit(String(args.from));
return;
}
await runInitWorkspace(args.force);
},
});
+1 -1
View File
@@ -85,7 +85,7 @@ export function buildLogFooter(slice: LogSlice, nArg: number, logPath: string):
let footer = `\n📄 ${rangeStr} | ${logPath}\n`;
if (slice.nextOffset !== null) {
footer += `⏩ Earlier lines available. Fetch previous page:\n`;
footer += "⏩ Earlier lines available. Fetch previous page:\n";
footer += ` nerve logs --offset ${slice.nextOffset} -n ${nArg}\n`;
}
+11 -11
View File
@@ -1,19 +1,18 @@
import { readFileSync } from "node:fs";
import { join } from "node:path";
import type { DatabaseSync } from "node:sqlite";
import { type SenseInfo, parseNerveConfig } from "@uncaged/nerve-core";
import { type SenseInfo, isPlainRecord, parseNerveConfig } from "@uncaged/nerve-core";
import { defineCommand } from "citty";
import { listSensesViaDaemon, triggerSenseViaDaemon } from "../daemon-client.js";
import {
assertSenseDbExists,
defaultPreviewSql,
formatRowsAsAlignedTable,
listTableSqlStatements,
openSenseDb,
parseSenseQueryArgs,
pickDefaultPreviewTable,
queryAsObjects,
} from "../sense-sqlite.js";
import { getNerveRoot, getSocketPath, isRunning } from "../workspace.js";
@@ -44,7 +43,8 @@ export function formatSenseList(senses: SenseInfo[]): string {
lines.push(` group: ${s.group}\n`);
lines.push(` throttle: ${formatDuration(s.throttle)}\n`);
lines.push(` timeout: ${formatDuration(s.timeout)}\n`);
const lastSignal = s.lastSignalTs !== null ? new Date(s.lastSignalTs).toISOString() : "(never)";
const lastSignal =
s.lastSignalTimestamp !== null ? new Date(s.lastSignalTimestamp).toISOString() : "(never)";
lines.push(` last signal: ${lastSignal}\n`);
}
return lines.join("");
@@ -65,7 +65,7 @@ export function sensesFromConfig(configPath: string): SenseInfo[] {
group: cfg.group,
throttle: cfg.throttle,
timeout: cfg.timeout,
lastSignalTs: null,
lastSignalTimestamp: null,
}));
}
@@ -80,7 +80,6 @@ const senseListCommand = defineCommand({
},
async run() {
if (!isRunning()) {
// Daemon not running — show static info from nerve.yaml
process.stderr.write(
"⚠️ Daemon is not running — showing static config only (no last signal time).\n\n",
);
@@ -171,9 +170,9 @@ const senseSchemaCommand = defineCommand({
},
async run({ args }) {
const nerveRoot = getNerveRoot();
let db: ReturnType<Awaited<ReturnType<typeof import("sql.js")>>["Database"]> | undefined;
let db: DatabaseSync | undefined;
try {
db = await openSenseDb(nerveRoot, args.name);
db = openSenseDb(nerveRoot, args.name);
const statements = listTableSqlStatements(db);
if (args.json) {
process.stdout.write(`${JSON.stringify(statements, null, 2)}\n`);
@@ -217,7 +216,7 @@ const senseQueryCommand = defineCommand({
},
async run({ args, rawArgs }) {
const nerveRoot = getNerveRoot();
let db: ReturnType<Awaited<ReturnType<typeof import("sql.js")>>["Database"]> | undefined;
let db: DatabaseSync | undefined;
try {
let parsed: { name: string; sql: string | undefined };
try {
@@ -228,7 +227,7 @@ const senseQueryCommand = defineCommand({
process.exit(1);
}
db = await openSenseDb(nerveRoot, args.name);
db = openSenseDb(nerveRoot, args.name);
let sql = parsed.sql?.trim();
if (!sql) {
@@ -241,7 +240,8 @@ const senseQueryCommand = defineCommand({
}
}
const rows = queryAsObjects(db, sql);
const rawRows: unknown[] = db.prepare(sql).all();
const rows: Record<string, unknown>[] = rawRows.filter(isPlainRecord);
if (args.json) {
process.stdout.write(`${JSON.stringify(rows, null, 2)}\n`);
+3 -1
View File
@@ -74,9 +74,11 @@ async function runDaemon(nerveRoot: string): Promise<void> {
const bootstrapPath = daemonBootstrapScript();
// After `open`, file-backed WriteStream has a numeric OS fd for spawn stdio; `@types/node` omits `fd` on this WriteStream alias.
const logFd = (logStream as unknown as { fd: number }).fd;
const child = spawn(process.execPath, [bootstrapPath], {
detached: true,
stdio: ["ignore", logStream.fd, logStream.fd],
stdio: ["ignore", logFd, logFd],
env: { ...process.env, NERVE_ROOT: nerveRoot },
cwd: nerveRoot,
});
+5 -1
View File
@@ -47,7 +47,11 @@ export const statusCommand = defineCommand({
return;
}
const pid = readPidFile() as number;
const pid = readPidFile();
if (pid === null) {
process.stdout.write("😴 Nerve daemon is not running.\n");
return;
}
const configPath = join(getNerveRoot(), "nerve.yaml");
let senseList: string[] = [];
+1 -1
View File
@@ -31,7 +31,7 @@ export const validateCommand = defineCommand({
const config = result.value;
const senseCount = Object.keys(config.senses).length;
const reflexCount = config.reflexes.length;
const workflowCount = config.workflows ? Object.keys(config.workflows).length : 0;
const workflowCount = Object.keys(config.workflows).length;
process.stdout.write(
`✅ nerve.yaml is valid — ${senseCount} sense(s), ${reflexCount} reflex(es), ${workflowCount} workflow(s)\n`,
+228 -11
View File
@@ -1,15 +1,24 @@
import { existsSync } from "node:fs";
import { join } from "node:path";
import type { DaemonIpcTriggerResponse } from "@uncaged/nerve-core";
import { DEFAULT_ENGINE_MAX_ROUNDS, isPlainRecord } from "@uncaged/nerve-core";
import { defineCommand } from "citty";
import { stringify } from "yaml";
import type { LogStore, ThreadRoundRow, WorkflowRun } from "@uncaged/nerve-store";
import { triggerWorkflowViaDaemon } from "../daemon-client.js";
import type { LogStore, WorkflowRun } from "../daemon-types.js";
import { loadDaemonModule } from "../workspace-daemon.js";
import { getNerveRoot, getSocketPath, isRunning } from "../workspace.js";
export const DEFAULT_PAGE_SIZE = 20;
/** Default max characters for `nerve workflow thread` output (including run header). */
export const DEFAULT_THREAD_BUDGET_CHARS = 8000;
/** Max role-round rows read from SQLite per invocation (DESC by round). */
export const THREAD_ROUNDS_FETCH_LIMIT = 8192;
export function parseIntArg(raw: string, fallback: number): number {
const v = Number.parseInt(raw, 10);
return Number.isNaN(v) ? fallback : v;
@@ -19,8 +28,8 @@ export function getDbPath(): string {
return join(getNerveRoot(), "data", "logs.db");
}
export function formatTs(ts: number): string {
return new Date(ts).toISOString();
export function formatTs(timestampMs: number): string {
return new Date(timestampMs).toISOString();
}
async function openStore(): Promise<LogStore> {
@@ -58,7 +67,7 @@ export function statusIcon(status: WorkflowRun["status"]): string {
}
/**
* Retrieve all workflow runs from the store, sorted by ts descending (newest first).
* Retrieve all workflow runs from the store, sorted by timestamp descending (newest first).
* Delegates to the store's efficient SQL query on the workflow_runs table.
*/
export function getAllWorkflowRuns(store: LogStore, filterWorkflow: string | null): WorkflowRun[] {
@@ -70,7 +79,7 @@ export function getAllWorkflowRuns(store: LogStore, filterWorkflow: string | nul
*/
export function formatRunLine(run: WorkflowRun): string {
const icon = statusIcon(run.status);
return ` ${icon} ${run.runId} workflow=${run.workflow} status=${run.status} ts=${formatTs(run.ts)}\n`;
return ` ${icon} ${run.runId} workflow=${run.workflow} status=${run.status} timestamp=${formatTs(run.timestamp)}\n`;
}
/**
@@ -130,7 +139,7 @@ export type InspectOutput = {
export function buildInspectOutput(
run: WorkflowRun,
allLogs: Array<{ ts: number; type: string; payload: string | null }>,
allLogs: Array<{ timestamp: number; type: string; payload: string | null }>,
offset: number,
limit: number,
): InspectOutput {
@@ -143,7 +152,7 @@ export function buildInspectOutput(
`🔍 Workflow run: ${run.runId}\n`,
` workflow: ${run.workflow}\n`,
` status: ${run.status}\n`,
` ts: ${formatTs(run.ts)}\n`,
` timestamp: ${formatTs(run.timestamp)}\n`,
`\n📜 Thread events (${shown} of ${total}):\n`,
];
@@ -158,7 +167,7 @@ export function buildInspectOutput(
: entry.payload.length <= 200
? ` payload=${entry.payload}`
: ` payload=${entry.payload.slice(0, 200)}`;
eventLines.push(` [${formatTs(entry.ts)}] type=${entry.type}${payloadStr}\n`);
eventLines.push(` [${formatTs(entry.timestamp)}] type=${entry.type}${payloadStr}\n`);
}
}
@@ -172,6 +181,119 @@ export function buildInspectOutput(
return { header, eventLines, paginationHint };
}
// ---------------------------------------------------------------------------
// nerve workflow thread <runId> — agent-oriented role rounds
// ---------------------------------------------------------------------------
export type PartitionedMessage = {
roleStr: string;
contentBody: string;
meta: Record<string, unknown>;
};
/**
* Extract display fields from a WorkflowMessage-shaped object.
* `role` and `content` are used for header/body; `meta` is serialized as YAML frontmatter.
*/
export function partitionWorkflowMessage(msg: {
role: string;
content: string;
meta: unknown;
timestamp: number;
}): PartitionedMessage {
const roleStr = msg.role;
const contentBody = msg.content;
const meta: Record<string, unknown> =
msg.meta !== null && msg.meta !== undefined && typeof msg.meta === "object"
? isPlainRecord(msg.meta)
? msg.meta
: (msg.meta as Record<string, unknown>)
: {};
return { roleStr, contentBody, meta };
}
/**
* One role round as plain text: header line, YAML frontmatter (meta only), body (content).
*/
export function formatThreadRoundBlock(row: ThreadRoundRow): string {
const { roleStr, contentBody, meta } = partitionWorkflowMessage(row.message);
const yamlBlock =
Object.keys(meta).length === 0 ? "{}\n" : `${stringify(meta, { lineWidth: 100 })}\n`;
return `[#${row.round} ${roleStr}] ${formatTs(row.timestamp)}\n---\n${yamlBlock}---\n${contentBody}\n\n`;
}
export type ThreadCommandOutput = {
lines: string[];
paginationHint: string | null;
};
function buildTruncatedSingleRound(
row: ThreadRoundRow,
remaining: number,
prefixLines: string[],
runId: string,
budgetFlag: string,
): ThreadCommandOutput {
const { roleStr, contentBody, meta } = partitionWorkflowMessage(row.message);
const yamlBlock =
Object.keys(meta).length === 0 ? "{}\n" : `${stringify(meta, { lineWidth: 100 })}\n`;
const header = `[#${row.round} ${roleStr}] ${formatTs(row.timestamp)}\n---\n${yamlBlock}---\n`;
const maxBody = Math.max(0, remaining - header.length - "[truncated]\n".length);
const truncated =
maxBody > 0 && contentBody.length > maxBody
? `${contentBody.slice(0, maxBody)}\n[truncated]\n`
: `${contentBody}\n[truncated]\n`;
const single = `${header + truncated}\n`;
const hintRound = row.round;
return {
lines: [...prefixLines, single],
paginationHint:
hintRound > 1
? `\n⏩ Older rounds exist. Fetch with:\n nerve workflow thread ${runId} --before ${String(hintRound)}${budgetFlag}\n`
: null,
};
}
/**
* Build stdout lines for `nerve workflow thread`: newest-first selection from
* `descRows` until `budgetChars` (including `prefixLines`), then chronological order.
*/
export function buildThreadCommandOutput(
prefixLines: string[],
descRows: ThreadRoundRow[],
budgetChars: number,
runId: string,
): ThreadCommandOutput {
const prefixText = prefixLines.join("");
let remaining = Math.max(0, budgetChars - prefixText.length);
const picked: ThreadRoundRow[] = [];
const budgetFlag =
budgetChars === DEFAULT_THREAD_BUDGET_CHARS ? "" : ` --budget ${String(budgetChars)}`;
for (const row of descRows) {
const block = formatThreadRoundBlock(row);
if (block.length <= remaining) {
picked.push(row);
remaining -= block.length;
continue;
}
if (picked.length === 0) {
return buildTruncatedSingleRound(row, remaining, prefixLines, runId, budgetFlag);
}
break;
}
const blocksAsc = picked.map(formatThreadRoundBlock).reverse();
const shownMinRound = picked.length === 0 ? null : Math.min(...picked.map((r) => r.round));
let paginationHint: string | null = null;
if (shownMinRound !== null && shownMinRound > 1) {
paginationHint = `\n⏩ Older rounds not shown. Fetch with:\n nerve workflow thread ${runId} --before ${String(shownMinRound)}${budgetFlag}\n`;
}
return { lines: [...prefixLines, ...blocksAsc], paginationHint };
}
// ---------------------------------------------------------------------------
// nerve workflow list
// ---------------------------------------------------------------------------
@@ -293,6 +415,89 @@ const workflowInspectCommand = defineCommand({
},
});
// ---------------------------------------------------------------------------
// nerve workflow thread <runId>
// ---------------------------------------------------------------------------
const workflowThreadCommand = defineCommand({
meta: {
name: "thread",
description: "Print role rounds for a workflow run (agent-oriented, budget-limited)",
},
args: {
runId: {
type: "positional",
description: "The run ID to dump role rounds for",
},
before: {
type: "string",
description:
"Exclusive upper bound on 1-based round index (use with hint from prior output to load older rounds)",
default: "0",
},
budget: {
type: "string",
description: `Max output characters including header (default: ${String(DEFAULT_THREAD_BUDGET_CHARS)})`,
default: String(DEFAULT_THREAD_BUDGET_CHARS),
},
},
async run({ args }) {
const store = await openStore();
try {
const before = Math.max(0, parseIntArg(args.before, 0));
const budgetChars = Math.max(1, parseIntArg(args.budget, DEFAULT_THREAD_BUDGET_CHARS));
const run = store.getWorkflowRun(args.runId);
if (run === null) {
process.stderr.write(`❌ No workflow run found with runId: ${args.runId}\n`);
process.exit(1);
}
const totalRoleRounds = store.getThreadRoundCount(args.runId);
if (totalRoleRounds === 0) {
process.stdout.write(
`🧵 Workflow thread: ${run.runId}\n workflow: ${run.workflow}\n status: ${run.status}\n\n📭 No role rounds recorded for this run.\n`,
);
return;
}
const descRows = store.getThreadRounds(args.runId, {
before,
limit: THREAD_ROUNDS_FETCH_LIMIT,
});
const prefixLines = [
"🧵 Role rounds (workflow thread)\n",
` runId: ${run.runId}\n`,
` workflow: ${run.workflow}\n`,
` status: ${run.status}\n`,
` rounds: ${String(totalRoleRounds)} role event(s) total\n\n`,
];
const { lines, paginationHint } = buildThreadCommandOutput(
prefixLines,
descRows,
budgetChars,
args.runId,
);
for (const line of lines) {
process.stdout.write(line);
}
if (paginationHint !== null) {
process.stdout.write(paginationHint);
}
if (descRows.length === 0 && before > 0) {
process.stdout.write(`\n📭 No rounds with index < ${String(before)}.\n`);
}
} finally {
store.close();
}
},
});
// ---------------------------------------------------------------------------
// nerve workflow trigger <name>
// ---------------------------------------------------------------------------
@@ -309,7 +514,8 @@ const workflowTriggerCommand = defineCommand({
},
payload: {
type: "string",
description: "JSON payload to pass as trigger payload (default: {})",
description:
'JSON with optional "prompt" (string), "maxRounds" (number), and "dryRun" (boolean) for the workflow run (default: {})',
default: "{}",
},
},
@@ -322,15 +528,25 @@ const workflowTriggerCommand = defineCommand({
process.exit(1);
}
let prompt = "";
let maxRounds = DEFAULT_ENGINE_MAX_ROUNDS;
let dryRun = false;
if (isPlainRecord(triggerPayload)) {
const p = triggerPayload;
if (typeof p.prompt === "string") prompt = p.prompt;
if (typeof p.maxRounds === "number") maxRounds = p.maxRounds;
if (typeof p.dryRun === "boolean") dryRun = p.dryRun;
}
if (!isRunning()) {
process.stderr.write("❌ Nerve daemon is not running. Start it with `nerve start`.\n");
process.exit(1);
}
const socketPath = getSocketPath();
let response: { ok: true } | { ok: false; error: string };
let response: DaemonIpcTriggerResponse;
try {
response = await triggerWorkflowViaDaemon(socketPath, args.name, triggerPayload);
response = await triggerWorkflowViaDaemon(socketPath, args.name, prompt, maxRounds, dryRun);
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`❌ Failed to contact daemon: ${msg}\n`);
@@ -359,6 +575,7 @@ export const workflowCommand = defineCommand({
subCommands: {
list: workflowListCommand,
inspect: workflowInspectCommand,
thread: workflowThreadCommand,
trigger: workflowTriggerCommand,
},
});
+50 -26
View File
@@ -8,22 +8,35 @@
import { connect } from "node:net";
import type { Socket } from "node:net";
import type { SenseInfo } from "@uncaged/nerve-core";
import type {
DaemonIpcListSensesResponse,
DaemonIpcRequest,
DaemonIpcTriggerResponse,
SenseInfo,
} from "@uncaged/nerve-core";
import { isPlainRecord } from "@uncaged/nerve-core";
const CONNECT_TIMEOUT_MS = 3_000;
const RESPONSE_TIMEOUT_MS = 5_000;
export type { SenseInfo };
type TriggerResponse = { ok: true } | { ok: false; error: string };
function isSenseInfo(value: unknown): value is SenseInfo {
if (!isPlainRecord(value)) return false;
return (
typeof value.name === "string" &&
typeof value.group === "string" &&
(value.throttle === null || typeof value.throttle === "number") &&
(value.timeout === null || typeof value.timeout === "number") &&
(value.lastSignalTimestamp === null || typeof value.lastSignalTimestamp === "number")
);
}
type ListSensesResponse = { ok: true; senses: SenseInfo[] } | { ok: false; error: string };
function parseDaemonResponse(line: string): TriggerResponse {
function parseDaemonResponse(line: string): DaemonIpcTriggerResponse {
try {
const obj = JSON.parse(line) as unknown;
if (obj !== null && typeof obj === "object") {
const r = obj as Record<string, unknown>;
const obj: unknown = JSON.parse(line);
if (isPlainRecord(obj)) {
const r = obj;
if (r.ok === true) return { ok: true };
if (r.ok === false && typeof r.error === "string") return { ok: false, error: r.error };
}
@@ -33,14 +46,15 @@ function parseDaemonResponse(line: string): TriggerResponse {
return { ok: false, error: `Unexpected daemon response: ${line}` };
}
function parseListSensesResponse(line: string): ListSensesResponse {
function parseListSensesResponse(line: string): DaemonIpcListSensesResponse {
try {
const obj = JSON.parse(line) as unknown;
if (obj !== null && typeof obj === "object") {
const r = obj as Record<string, unknown>;
const obj: unknown = JSON.parse(line);
if (isPlainRecord(obj)) {
const r = obj;
if (r.ok === false && typeof r.error === "string") return { ok: false, error: r.error };
if (r.ok === true && Array.isArray(r.senses))
return { ok: true, senses: r.senses as SenseInfo[] };
if (r.ok === true && Array.isArray(r.senses) && r.senses.every(isSenseInfo)) {
return { ok: true, senses: r.senses };
}
}
} catch {
// fall through
@@ -54,7 +68,7 @@ function parseListSensesResponse(line: string): ListSensesResponse {
*/
function sendAndReceive<T>(
socketPath: string,
message: object,
message: DaemonIpcRequest,
parseFirstLine: (trimmed: string) => T,
responseTimeoutMs: number = RESPONSE_TIMEOUT_MS,
): Promise<T> {
@@ -119,27 +133,37 @@ function sendAndReceive<T>(
export function triggerWorkflowViaDaemon(
socketPath: string,
workflow: string,
payload: unknown,
): Promise<TriggerResponse> {
return sendAndReceive(
socketPath,
{ type: "trigger-workflow", workflow, payload },
parseDaemonResponse,
);
prompt: string,
maxRounds: number,
dryRun = false,
): Promise<DaemonIpcTriggerResponse> {
const message: DaemonIpcRequest = {
type: "trigger-workflow",
workflow,
prompt,
maxRounds,
dryRun,
};
return sendAndReceive(socketPath, message, parseDaemonResponse);
}
/**
* Send a trigger-sense message to the running daemon via its Unix socket.
* Resolves with the daemon's response or rejects on connection/timeout errors.
*/
export function triggerSenseViaDaemon(socketPath: string, sense: string): Promise<TriggerResponse> {
return sendAndReceive(socketPath, { type: "trigger-sense", sense }, parseDaemonResponse);
export function triggerSenseViaDaemon(
socketPath: string,
sense: string,
): Promise<DaemonIpcTriggerResponse> {
const message: DaemonIpcRequest = { type: "trigger-sense", sense };
return sendAndReceive(socketPath, message, parseDaemonResponse);
}
/**
* Send a list-senses message to the running daemon via its Unix socket.
* Resolves with the list of registered senses or rejects on connection/timeout errors.
*/
export function listSensesViaDaemon(socketPath: string): Promise<ListSensesResponse> {
return sendAndReceive(socketPath, { type: "list-senses" }, parseListSensesResponse);
export function listSensesViaDaemon(socketPath: string): Promise<DaemonIpcListSensesResponse> {
const message: DaemonIpcRequest = { type: "list-senses" };
return sendAndReceive(socketPath, message, parseListSensesResponse);
}
-70
View File
@@ -1,70 +0,0 @@
/**
* Structural types for workflow CLI — mirrors @uncaged/nerve-daemon log-store
* public API so the CLI runtime does not statically depend on the daemon package.
*
* ⚠️ Keep in sync with @uncaged/nerve-daemon exports.
* Run `pnpm --filter @uncaged/nerve-cli test` to catch drift via satisfies assertions.
*/
export type WorkflowRunStatus =
| "queued"
| "started"
| "completed"
| "failed"
| "crashed"
| "dropped"
| "interrupted";
export type WorkflowRun = {
runId: string;
workflow: string;
status: WorkflowRunStatus;
ts: number;
};
export type LogEntry = {
id?: number;
source: string;
type: string;
refId: string | null;
payload: string | null;
ts: number;
};
export type LogQuery = {
source?: string;
type?: string;
refId?: string;
since?: number;
until?: number;
limit?: number;
};
export type ArchiveLogsOptions = {
now?: number;
vacuum?: boolean;
maxDays?: number;
retentionMs?: number;
};
export type ArchiveLogsDayResult = {
day: string;
rowCount: number;
filePath: string;
};
export type ArchiveLogsResult = {
days: ArchiveLogsDayResult[];
vacuumed: boolean;
};
/** Subset of daemon LogStore used by the CLI workflow commands. */
export type LogStore = {
query: (filter?: LogQuery) => LogEntry[];
getWorkflowRun: (runId: string) => WorkflowRun | null;
getActiveWorkflowRuns: (workflowName?: string) => WorkflowRun[];
getAllWorkflowRuns: (workflowName: string | null) => WorkflowRun[];
upsertWorkflowRun: (entry: Omit<LogEntry, "id">, run: WorkflowRun) => LogEntry;
archiveLogs: (options?: ArchiveLogsOptions) => ArchiveLogsResult;
close: () => void;
};
+29 -57
View File
@@ -1,25 +1,6 @@
import { existsSync, readFileSync } from "node:fs";
import { existsSync } from "node:fs";
import { join } from "node:path";
import initSqlJs, { type Database } from "sql.js";
// ── WASM singleton ──────────────────────────────────────────────────────────
let _SQL: Awaited<ReturnType<typeof initSqlJs>> | null = null;
async function getSQL() {
if (!_SQL) {
_SQL = await initSqlJs();
}
return _SQL;
}
/** Open a sense SQLite database (readonly, loaded into memory via sql.js). */
export async function openSenseDb(nerveRoot: string, senseName: string): Promise<Database> {
const path = assertSenseDbExists(nerveRoot, senseName);
const SQL = await getSQL();
const buffer = readFileSync(path);
return new SQL.Database(buffer);
}
import { DatabaseSync } from "node:sqlite";
/** SQLite path for a sense under the nerve workspace root. */
export function senseDbPath(nerveRoot: string, senseName: string): string {
@@ -34,31 +15,39 @@ export function assertSenseDbExists(nerveRoot: string, senseName: string): strin
return path;
}
/** Open a sense SQLite database in readonly mode using node:sqlite. */
export function openSenseDb(nerveRoot: string, senseName: string): DatabaseSync {
const path = assertSenseDbExists(nerveRoot, senseName);
return new DatabaseSync(path, { readOnly: true });
}
/** `SELECT sql FROM sqlite_master WHERE type='table'` (non-null sql only). */
export function listTableSqlStatements(db: Database): string[] {
const results = db.exec(
`SELECT sql FROM sqlite_master WHERE type = 'table' AND sql IS NOT NULL ORDER BY tbl_name`,
);
if (results.length === 0) return [];
return results[0].values.map((row) => row[0] as string);
export function listTableSqlStatements(db: DatabaseSync): string[] {
const rows = db
.prepare(
`SELECT sql FROM sqlite_master WHERE type = 'table' AND sql IS NOT NULL ORDER BY tbl_name`,
)
.all() as { sql: string }[];
return rows.map((r) => r.sql);
}
/**
* Table used for `nerve sense query <name>` with no SQL.
* Prefers real data tables over `_migrations`, then lexicographic by name.
*/
export function pickDefaultPreviewTable(db: Database): string | null {
const results = db.exec(
`SELECT name FROM sqlite_master
WHERE type = 'table' AND sql IS NOT NULL
AND name NOT LIKE 'sqlite\\_%' ESCAPE '\\'
ORDER BY
CASE WHEN name = '_migrations' THEN 1 ELSE 0 END,
name
LIMIT 1`,
);
if (results.length === 0 || results[0].values.length === 0) return null;
return results[0].values[0][0] as string;
export function pickDefaultPreviewTable(db: DatabaseSync): string | null {
const row = db
.prepare(
`SELECT name FROM sqlite_master
WHERE type = 'table' AND sql IS NOT NULL
AND name NOT LIKE 'sqlite\\_%' ESCAPE '\\'
ORDER BY
CASE WHEN name = '_migrations' THEN 1 ELSE 0 END,
name
LIMIT 1`,
)
.get() as { name: string } | undefined;
return row?.name ?? null;
}
export function defaultPreviewSql(table: string): string {
@@ -93,7 +82,7 @@ function stringifyCell(value: unknown): string {
if (typeof value === "bigint") return value.toString();
if (typeof value === "number" || typeof value === "boolean") return String(value);
if (typeof value === "string") return value;
if (value instanceof Uint8Array) return Buffer.from(value).toString("hex");
if (Buffer.isBuffer(value)) return value.toString("hex");
try {
return JSON.stringify(value);
} catch {
@@ -136,20 +125,3 @@ export function formatRowsAsAlignedTable(rows: Record<string, unknown>[]): strin
const body = cells.map((r) => r.map((cell, j) => cell.padEnd(widths[j])).join(" | ")).join("\n");
return `${header}\n${sep}\n${body}\n`;
}
/**
* Run a SQL query via sql.js and return rows as key-value objects.
* sql.js returns columnar data; this converts to the familiar row format.
*/
export function queryAsObjects(db: Database, sql: string): Record<string, unknown>[] {
const results = db.exec(sql);
if (results.length === 0) return [];
const { columns, values } = results[0];
return values.map((row) => {
const obj: Record<string, unknown> = {};
for (let i = 0; i < columns.length; i++) {
obj[columns[i]] = row[i];
}
return obj;
});
}
+3 -2
View File
@@ -5,7 +5,7 @@ import { pathToFileURL } from "node:url";
import type { NerveConfig } from "@uncaged/nerve-core";
import type { LogStore } from "./daemon-types.js";
import type { LogStore } from "@uncaged/nerve-store";
export function getDaemonEntryPath(nerveRoot: string): string | undefined {
const pkgPath = join(nerveRoot, "node_modules", "@uncaged", "nerve-daemon", "package.json");
@@ -29,7 +29,7 @@ export function assertWorkspaceDaemonInstalled(nerveRoot: string): string {
return entry;
}
/** Loaded from ~/.uncaged-nerve/node_modules at runtime — keep types structural only. */
/** Loaded from ~/.uncaged-nerve/node_modules at runtime. */
export type DaemonModule = {
createKernel: (
config: NerveConfig,
@@ -46,5 +46,6 @@ export type DaemonModule = {
export async function loadDaemonModule(nerveRoot: string): Promise<DaemonModule> {
const entry = assertWorkspaceDaemonInstalled(nerveRoot);
const url = pathToFileURL(entry).href;
// Dynamic import return type is module-specific; narrow at this workspace boundary.
return import(url) as Promise<DaemonModule>;
}
+2 -1
View File
@@ -6,5 +6,6 @@
"composite": false,
"types": ["node"]
},
"include": ["src"]
"include": ["src"],
"exclude": ["src/__tests__"]
}
-13
View File
@@ -1,13 +0,0 @@
import { defineConfig } from "tsup";
export default defineConfig({
entry: ["src/index.ts", "src/cli.ts", "src/daemon-bootstrap.ts"],
format: ["esm"],
dts: true,
clean: true,
banner: {
js: "#!/usr/bin/env node",
},
/** Daemon is loaded from workspace node_modules at runtime — never bundle it. */
external: ["@uncaged/nerve-daemon", "sql.js"],
});
+65
View File
@@ -0,0 +1,65 @@
# @uncaged/nerve-core
Shared types and configuration parser for the [nerve](../../README.md) observation engine.
## What's Inside
- **Type definitions** — `Signal`, `SenseConfig`, `SenseInfo`, `SenseReflexConfig`, `ReflexConfig` (sense-only), `WorkflowConfig`, `NerveConfig`, and related types
- **Config parser** — `parseNerveConfig(yaml)` validates and parses `nerve.yaml` into `NerveConfig` (rejects reflex entries that declare a `workflow` key; reflexes only schedule senses)
- **Sense → workflow routing** — `parseSenseWorkflowDirective`, `routeSenseComputeOutput`, and types `ParsedSenseWorkflowDirective`, `SenseComputeRoute`
- **Daemon IPC protocol** — request/response types (`DaemonIpcRequest`, `DaemonIpcResponse`, …) and `parseDaemonIpcRequest` for newline-delimited JSON on the CLI ↔ daemon socket
- **Workflow automaton types** — `START` / `END` sentinel constants, `WorkflowMessage`, `StartStep`, `RoleStep`, `ModeratorContext` (`start` + `steps`; empty `steps` on first moderator call), `Moderator` (single `context` argument), `WorkflowDefinition`, `Role`, `SenseResult`, plus `DEFAULT_ENGINE_MAX_ROUNDS`
- **Result type** — `Result<T>` with `ok()` / `err()` helpers for explicit error handling (no thrown exceptions for parse paths)
## Usage
```typescript
import { parseNerveConfig, ok, err } from "@uncaged/nerve-core";
import type { NerveConfig, Signal, Result } from "@uncaged/nerve-core";
const result: Result<NerveConfig> = parseNerveConfig(yamlString);
if (result.ok) {
console.log(result.value.senses);
}
```
### Sense return → signal vs workflow
```typescript
import { parseSenseWorkflowDirective, routeSenseComputeOutput } from "@uncaged/nerve-core";
const directive = parseSenseWorkflowDirective("my-workflow|8|Hello from sense");
if (directive.ok) {
console.log(directive.value.workflowName, directive.value.maxRounds, directive.value.prompt);
}
const route = routeSenseComputeOutput({
metric: 42,
workflow: "my-workflow|8|Run now",
});
if (route.kind === "launch") {
// engine starts workflow; no Signal to the bus for this return
console.log(route.launch);
} else {
// normal signal with payload
console.log(route.payload);
}
```
## Duration Format
Config fields like `throttle`, `timeout`, and `interval` accept human-readable durations:
- `5s` — 5 seconds
- `10m` — 10 minutes
- `1h` — 1 hour
## Install
```bash
pnpm add @uncaged/nerve-core
```
## License
MIT
+7 -2
View File
@@ -1,18 +1,23 @@
{
"name": "@uncaged/nerve-core",
"version": "0.1.4",
"version": "0.4.0",
"type": "module",
"main": "dist/index.js",
"files": ["dist"],
"publishConfig": {
"access": "public"
},
"types": "dist/index.d.ts",
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "tsup",
"build": "rslib build",
"test": "vitest run"
},
"dependencies": {
"yaml": "^2.8.3"
},
"devDependencies": {
"@rslib/core": "^0.21.3",
"vitest": "^4.1.5"
}
}
+19
View File
@@ -0,0 +1,19 @@
import { defineConfig } from "@rslib/core";
export default defineConfig({
lib: [
{
format: "esm",
dts: true,
},
],
source: {
entry: {
index: "src/index.ts",
},
},
output: {
target: "node",
cleanDistPath: true,
},
});
+14 -21
View File
@@ -18,9 +18,6 @@ reflexes:
- sense: memory
on:
- high_usage
- workflow: alert
on:
- cpu
workflows:
alert:
@@ -48,12 +45,12 @@ describe("parseNerveConfig", () => {
timeout: 10_000,
gracePeriod: 3000,
});
expect(result.value.reflexes).toHaveLength(3);
expect(result.value.reflexes).toHaveLength(2);
expect(result.value.reflexes[0]).toEqual({
kind: "sense",
sense: "cpu",
interval: 30_000,
on: null,
on: [],
});
expect(result.value.reflexes[1]).toEqual({
kind: "sense",
@@ -61,12 +58,7 @@ describe("parseNerveConfig", () => {
interval: null,
on: ["high_usage"],
});
expect(result.value.reflexes[2]).toEqual({
kind: "workflow",
workflow: "alert",
on: ["cpu"],
});
expect(result.value.workflows?.alert).toEqual({
expect(result.value.workflows.alert).toEqual({
concurrency: 2,
overflow: "queue",
maxQueue: 10,
@@ -93,11 +85,12 @@ senses:
group: system
reflexes:
- sense: cpu
interval: 1s
`;
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.value.workflows).toBeNull();
expect(result.value.workflows).toEqual({});
});
it("sense config has null for omitted throttle/timeout/gracePeriod", () => {
@@ -150,11 +143,11 @@ workflows:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.value.workflows?.alert).toEqual({
expect(result.value.workflows.alert).toEqual({
concurrency: 1,
overflow: "drop",
});
expect("maxQueue" in (result.value.workflows?.alert ?? {})).toBe(false);
expect("maxQueue" in result.value.workflows.alert).toBe(false);
});
it("overflow: queue defaults maxQueue to 100", () => {
@@ -171,7 +164,7 @@ workflows:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.value.workflows?.alert).toEqual({
expect(result.value.workflows.alert).toEqual({
concurrency: 1,
overflow: "queue",
maxQueue: 100,
@@ -201,7 +194,7 @@ reflexes:
expect(result.error.message).toMatch(/disk.*not found in senses/);
});
it("returns error when workflow reflex references a non-existent workflow", () => {
it("returns error when reflex uses unsupported workflow field", () => {
const yaml = `
senses:
cpu:
@@ -214,10 +207,10 @@ reflexes:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toMatch(/missing_wf.*not found in workflows/);
expect(result.error.message).toMatch(/workflow.*not supported/);
});
it("returns error when workflow reflex references non-existent workflow (with workflows defined)", () => {
it("returns error when reflex uses unsupported workflow field (with workflows defined)", () => {
const yaml = `
senses:
cpu:
@@ -234,7 +227,7 @@ workflows:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toMatch(/unknown.*not found in workflows/);
expect(result.error.message).toMatch(/workflow.*not supported/);
});
it("returns error for invalid throttle format", () => {
@@ -362,7 +355,7 @@ reflexes:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toMatch(/cannot have both/);
expect(result.error.message).toMatch(/workflow.*not supported/);
});
it("returns error when reflex has neither sense nor workflow", () => {
@@ -376,7 +369,7 @@ reflexes:
const result = parseNerveConfig(yaml);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toMatch(/must have either/);
expect(result.error.message).toMatch(/must include "sense"/);
});
});
});
@@ -0,0 +1,72 @@
import { describe, expect, it } from "vitest";
import { parseDaemonIpcRequest } from "../daemon-ipc-protocol.js";
describe("parseDaemonIpcRequest", () => {
it("parses trigger-workflow", () => {
expect(
parseDaemonIpcRequest(
JSON.stringify({
type: "trigger-workflow",
workflow: "wf",
prompt: "go",
maxRounds: 3,
}),
),
).toEqual({
type: "trigger-workflow",
workflow: "wf",
prompt: "go",
maxRounds: 3,
dryRun: false,
});
});
it("parses trigger-workflow with dryRun true", () => {
expect(
parseDaemonIpcRequest(
JSON.stringify({
type: "trigger-workflow",
workflow: "wf",
prompt: "go",
maxRounds: 3,
dryRun: true,
}),
),
).toEqual({
type: "trigger-workflow",
workflow: "wf",
prompt: "go",
maxRounds: 3,
dryRun: true,
});
});
it("rejects trigger-workflow with empty workflow", () => {
expect(
parseDaemonIpcRequest(
JSON.stringify({
type: "trigger-workflow",
workflow: "",
prompt: "",
maxRounds: 1,
}),
),
).toBeNull();
});
it("parses trigger-sense and list-senses", () => {
expect(parseDaemonIpcRequest(JSON.stringify({ type: "trigger-sense", sense: "x" }))).toEqual({
type: "trigger-sense",
sense: "x",
});
expect(parseDaemonIpcRequest(JSON.stringify({ type: "list-senses" }))).toEqual({
type: "list-senses",
});
});
it("returns null for invalid JSON or unknown type", () => {
expect(parseDaemonIpcRequest("not json")).toBeNull();
expect(parseDaemonIpcRequest(JSON.stringify({ type: "nope" }))).toBeNull();
});
});
+48 -58
View File
@@ -1,8 +1,10 @@
import { parse } from "yaml";
import { isPlainRecord } from "./is-plain-record.js";
import type { Result } from "./result.js";
import { err, ok } from "./result.js";
import type { NerveConfig, ReflexConfig, SenseConfig, WorkflowConfig } from "./types.js";
import { DEFAULT_ENGINE_MAX_ROUNDS } from "./types.js";
const DURATION_RE = /^(\d+)([smh])$/;
@@ -39,11 +41,11 @@ function parseDurationField(field: unknown, label: string): Result<number | null
}
function validateSenseConfig(name: string, raw: unknown): Result<SenseConfig> {
if (raw === null || typeof raw !== "object" || Array.isArray(raw)) {
if (!isPlainRecord(raw)) {
return err(new Error(`senses.${name}: must be an object`));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
if (typeof obj.group !== "string" || obj.group.trim() === "") {
return err(new Error(`senses.${name}.group: required string`));
@@ -74,19 +76,19 @@ function validateSenseConfig(name: string, raw: unknown): Result<SenseConfig> {
});
}
function parseOnField(index: number, obj: Record<string, unknown>): Result<string[] | null> {
if (obj.on === undefined || obj.on === null) return ok(null);
if (!Array.isArray(obj.on) || !obj.on.every((item) => typeof item === "string")) {
function parseOnField(index: number, obj: Record<string, unknown>): Result<string[]> {
if (obj.on === undefined || obj.on === null) return ok([]);
if (!Array.isArray(obj.on) || !obj.on.every((item): item is string => typeof item === "string")) {
return err(new Error(`reflexes[${index}].on: must be an array of strings`));
}
return ok(obj.on as string[]);
return ok(obj.on);
}
function parseSenseReflex(
index: number,
obj: Record<string, unknown>,
senseNames: Set<string>,
on: string[] | null,
on: string[],
): Result<ReflexConfig> {
if (typeof obj.sense !== "string") {
return err(new Error(`reflexes[${index}].sense: must be a string`));
@@ -98,7 +100,7 @@ function parseSenseReflex(
const intervalResult = parseDurationField(obj.interval, `reflexes[${index}].interval`);
if (!intervalResult.ok) return intervalResult;
if (intervalResult.value === null && on !== null && on.length === 0) {
if (intervalResult.value === null && on.length === 0) {
return err(
new Error(`reflexes[${index}]: sense reflex must have at least one of "interval" or "on"`),
);
@@ -112,61 +114,56 @@ function parseSenseReflex(
});
}
function parseWorkflowReflex(
index: number,
obj: Record<string, unknown>,
on: string[] | null,
): Result<ReflexConfig> {
if (typeof obj.workflow !== "string") {
return err(new Error(`reflexes[${index}].workflow: must be a string`));
}
if (obj.interval !== undefined) {
return err(
new Error(`reflexes[${index}]: workflow reflex does not support "interval" (use "on")`),
);
}
return ok({
kind: "workflow" as const,
workflow: obj.workflow,
on,
});
}
function validateReflexConfig(
index: number,
raw: unknown,
senseNames: Set<string>,
): Result<ReflexConfig> {
if (raw === null || typeof raw !== "object" || Array.isArray(raw)) {
if (!isPlainRecord(raw)) {
return err(new Error(`reflexes[${index}]: must be an object`));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
const hasSense = obj.sense !== undefined;
const hasWorkflow = obj.workflow !== undefined;
const hasWorkflowKey = Object.hasOwn(obj, "workflow");
if (hasSense && hasWorkflow) {
return err(new Error(`reflexes[${index}]: cannot have both "sense" and "workflow"`));
if (hasWorkflowKey) {
return err(
new Error(
`reflexes[${index}]: YAML "workflow" entries are not supported — start workflows from a Sense compute return value using a "workflow" string field (format: name|maxRounds|prompt)`,
),
);
}
if (!hasSense && !hasWorkflow) {
return err(new Error(`reflexes[${index}]: must have either "sense" or "workflow"`));
if (!hasSense) {
return err(new Error(`reflexes[${index}]: must include "sense"`));
}
const onResult = parseOnField(index, obj);
if (!onResult.ok) return onResult;
if (hasSense) {
return parseSenseReflex(index, obj, senseNames, onResult.value);
return parseSenseReflex(index, obj, senseNames, onResult.value);
}
function parseEngineMaxRounds(obj: Record<string, unknown>): Result<number> {
if (obj.max_rounds === undefined || obj.max_rounds === null) {
return ok(DEFAULT_ENGINE_MAX_ROUNDS);
}
return parseWorkflowReflex(index, obj, onResult.value);
if (
typeof obj.max_rounds !== "number" ||
!Number.isInteger(obj.max_rounds) ||
obj.max_rounds < 1
) {
return err(new Error("max_rounds: must be a positive integer"));
}
return ok(obj.max_rounds);
}
function validateWorkflowConfig(name: string, raw: unknown): Result<WorkflowConfig> {
if (raw === null || typeof raw !== "object" || Array.isArray(raw)) {
if (!isPlainRecord(raw)) {
return err(new Error(`workflows.${name}: must be an object`));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
if (
typeof obj.concurrency !== "number" ||
@@ -213,11 +210,11 @@ function validateWorkflowConfig(name: string, raw: unknown): Result<WorkflowConf
function parseSenses(
obj: Record<string, unknown>,
): Result<{ senses: Record<string, SenseConfig>; senseNames: Set<string> }> {
if (obj.senses === null || typeof obj.senses !== "object" || Array.isArray(obj.senses)) {
if (!isPlainRecord(obj.senses)) {
return err(new Error("senses: required object"));
}
const sensesRaw = obj.senses as Record<string, unknown>;
const sensesRaw = obj.senses;
const senses: Record<string, SenseConfig> = {};
const senseNames = new Set(Object.keys(sensesRaw));
@@ -248,16 +245,14 @@ function parseReflexes(
return ok(reflexes);
}
function parseWorkflows(
obj: Record<string, unknown>,
): Result<Record<string, WorkflowConfig> | null> {
if (obj.workflows === undefined || obj.workflows === null) return ok(null);
function parseWorkflows(obj: Record<string, unknown>): Result<Record<string, WorkflowConfig>> {
if (obj.workflows === undefined || obj.workflows === null) return ok({});
if (typeof obj.workflows !== "object" || Array.isArray(obj.workflows)) {
if (!isPlainRecord(obj.workflows)) {
return err(new Error("workflows: must be an object if provided"));
}
const workflowsRaw = obj.workflows as Record<string, unknown>;
const workflowsRaw = obj.workflows;
const workflows: Record<string, WorkflowConfig> = {};
for (const [name, wfRaw] of Object.entries(workflowsRaw)) {
@@ -279,11 +274,11 @@ export function parseNerveConfig(raw: string): Result<NerveConfig> {
return err(new Error(`YAML parse error: ${message}`));
}
if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
if (!isPlainRecord(parsed)) {
return err(new Error("Config must be a YAML object"));
}
const obj = parsed as Record<string, unknown>;
const obj = parsed;
const sensesResult = parseSenses(obj);
if (!sensesResult.ok) return sensesResult;
@@ -295,16 +290,11 @@ export function parseNerveConfig(raw: string): Result<NerveConfig> {
const workflowsResult = parseWorkflows(obj);
if (!workflowsResult.ok) return workflowsResult;
// Cross-validate: workflow reflexes must reference defined workflows
const workflowNames = new Set(workflowsResult.value ? Object.keys(workflowsResult.value) : []);
for (let i = 0; i < reflexesResult.value.length; i++) {
const reflex = reflexesResult.value[i];
if (reflex.kind === "workflow" && !workflowNames.has(reflex.workflow)) {
return err(new Error(`reflexes[${i}].workflow: "${reflex.workflow}" not found in workflows`));
}
}
const maxRoundsResult = parseEngineMaxRounds(obj);
if (!maxRoundsResult.ok) return maxRoundsResult;
return ok({
maxRounds: maxRoundsResult.value,
senses,
reflexes: reflexesResult.value,
workflows: workflowsResult.value,
+94
View File
@@ -0,0 +1,94 @@
/**
* Daemon Unix-socket IPC protocol (CLI → daemon).
* Newline-delimited JSON: one request object per line from the client,
* one response object per line from the daemon.
*/
import { isPlainRecord } from "./is-plain-record.js";
import type { SenseInfo } from "./types.js";
/** Client → daemon: start a workflow run. */
export type DaemonIpcTriggerWorkflowRequest = {
type: "trigger-workflow";
workflow: string;
prompt: string;
maxRounds: number;
dryRun: boolean;
};
/** Client → daemon: run a sense compute on demand. */
export type DaemonIpcTriggerSenseRequest = {
type: "trigger-sense";
sense: string;
};
/** Client → daemon: list registered senses. */
export type DaemonIpcListSensesRequest = {
type: "list-senses";
};
/** Union of all JSON requests the daemon IPC server accepts. */
export type DaemonIpcRequest =
| DaemonIpcTriggerWorkflowRequest
| DaemonIpcTriggerSenseRequest
| DaemonIpcListSensesRequest;
/** Successful trigger / trigger-sense reply (no body). */
export type DaemonIpcTriggerOkResponse = { ok: true };
export type DaemonIpcErrorResponse = { ok: false; error: string };
/** Replies for trigger-workflow and trigger-sense. */
export type DaemonIpcTriggerResponse = DaemonIpcTriggerOkResponse | DaemonIpcErrorResponse;
/** Reply for list-senses. */
export type DaemonIpcListSensesResponse =
| { ok: true; senses: SenseInfo[] }
| DaemonIpcErrorResponse;
/** Any JSON response the daemon may write on the IPC socket. */
export type DaemonIpcResponse =
| DaemonIpcTriggerOkResponse
| DaemonIpcErrorResponse
| { ok: true; senses: SenseInfo[] };
function parseTriggerWorkflowFields(
req: Record<string, unknown>,
): DaemonIpcTriggerWorkflowRequest | null {
if (typeof req.workflow !== "string" || req.workflow.length === 0) return null;
if (typeof req.prompt !== "string") return null;
if (typeof req.maxRounds !== "number") return null;
const dryRun = typeof req.dryRun === "boolean" ? req.dryRun : false;
return {
type: "trigger-workflow",
workflow: req.workflow,
prompt: req.prompt,
maxRounds: req.maxRounds,
dryRun,
};
}
/**
* Parse a single line of JSON into a {@link DaemonIpcRequest}, or null if invalid.
* Kept in core with the request types so CLI and daemon stay aligned at compile time.
*/
export function parseDaemonIpcRequest(line: string): DaemonIpcRequest | null {
try {
const obj: unknown = JSON.parse(line);
if (!isPlainRecord(obj)) return null;
const req = obj;
if (req.type === "trigger-workflow") {
return parseTriggerWorkflowFields(req);
}
if (req.type === "trigger-sense") {
if (typeof req.sense !== "string" || req.sense.length === 0) return null;
return { type: "trigger-sense", sense: req.sense };
}
if (req.type === "list-senses") {
return { type: "list-senses" };
}
return null;
} catch {
return null;
}
}
+32 -7
View File
@@ -3,21 +3,46 @@ export type {
SenseConfig,
SenseInfo,
SenseReflexConfig,
WorkflowReflexConfig,
ReflexConfig,
DropOverflowConfig,
QueueOverflowConfig,
WorkflowConfig,
NerveConfig,
CommandEvent,
ThreadState,
ModerateResult,
WorkflowContext,
RoleExecuteFn,
WorkflowMessage,
RoleResult,
Role,
ModerateFn,
RoleMeta,
StartStep,
RoleStep,
ModeratorContext,
Moderator,
WorkflowDefinition,
SenseResult,
} from "./types.js";
export { START, END, DEFAULT_ENGINE_MAX_ROUNDS } from "./types.js";
export type { Result } from "./result.js";
export { ok, err } from "./result.js";
export { parseNerveConfig } from "./config.js";
export { isPlainRecord } from "./is-plain-record.js";
export type {
ParsedSenseWorkflowDirective,
SenseComputeRoute,
} from "./sense-workflow-directive.js";
export {
parseSenseWorkflowDirective,
routeSenseComputeOutput,
} from "./sense-workflow-directive.js";
export type {
DaemonIpcTriggerWorkflowRequest,
DaemonIpcTriggerSenseRequest,
DaemonIpcListSensesRequest,
DaemonIpcRequest,
DaemonIpcTriggerOkResponse,
DaemonIpcErrorResponse,
DaemonIpcTriggerResponse,
DaemonIpcListSensesResponse,
DaemonIpcResponse,
} from "./daemon-ipc-protocol.js";
export { parseDaemonIpcRequest } from "./daemon-ipc-protocol.js";
+7
View File
@@ -0,0 +1,7 @@
/**
* Narrows `unknown` to a plain JSON-style object (not null, not array).
* Use after `JSON.parse` / YAML / IPC when validating structure field-by-field.
*/
export function isPlainRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
@@ -0,0 +1,77 @@
import { isPlainRecord } from "./is-plain-record.js";
import type { Result } from "./result.js";
import { err, ok } from "./result.js";
/** Parsed `workflow-name|maxRounds|prompt` from a Sense compute return value. */
export type ParsedSenseWorkflowDirective = {
workflowName: string;
maxRounds: number;
prompt: string;
};
/**
* Parses the pipe-separated `workflow` field from a Sense compute result.
* `prompt` may contain `|` — only the first two pipes delimit name and rounds.
*/
export function parseSenseWorkflowDirective(field: string): Result<ParsedSenseWorkflowDirective> {
const trimmed = field.trim();
if (trimmed.length === 0) {
return err(new Error("workflow directive is empty"));
}
const parts = trimmed.split("|");
if (parts.length < 3) {
return err(
new Error(
`workflow directive must be "name|maxRounds|prompt" (got ${String(parts.length)} segment(s))`,
),
);
}
const workflowName = (parts[0] ?? "").trim();
if (workflowName.length === 0) {
return err(new Error("workflow directive: empty workflow name"));
}
const roundsRaw = (parts[1] ?? "").trim();
const maxRounds = Number.parseInt(roundsRaw, 10);
if (!Number.isInteger(maxRounds) || maxRounds < 1) {
return err(new Error(`workflow directive: invalid maxRounds "${roundsRaw}"`));
}
const prompt = parts.slice(2).join("|");
return ok({ workflowName, maxRounds, prompt });
}
export type SenseComputeRoute =
| { kind: "launch"; launch: ParsedSenseWorkflowDirective }
| { kind: "signal"; payload: unknown };
function stripWorkflowKey(payload: Record<string, unknown>): Record<string, unknown> {
const { workflow: _drop, ...rest } = payload;
return rest;
}
/**
* Interprets a Sense compute non-null return value for the engine:
* - `workflow` missing → normal signal with full payload
* - `workflow: null` or `""` → normal signal; `workflow` key stripped from emitted payload
* - `workflow: "name|n|prompt"` → launch workflow; no Signal is emitted to the bus
*/
export function routeSenseComputeOutput(payload: unknown): SenseComputeRoute {
if (!isPlainRecord(payload)) {
return { kind: "signal", payload };
}
const obj = payload;
if (!Object.hasOwn(obj, "workflow")) {
return { kind: "signal", payload };
}
const w = obj.workflow;
if (w === null || w === "") {
return { kind: "signal", payload: stripWorkflowKey(obj) };
}
if (typeof w !== "string") {
return { kind: "signal", payload };
}
const parsed = parseSenseWorkflowDirective(w);
if (!parsed.ok) {
return { kind: "signal", payload: stripWorkflowKey(obj) };
}
return { kind: "launch", launch: parsed.value };
}
+74 -50
View File
@@ -2,7 +2,7 @@ export type Signal = {
id: number;
senseId: string;
payload: unknown;
ts: number;
timestamp: number;
};
export type SenseConfig = {
@@ -18,23 +18,18 @@ export type SenseInfo = {
group: string;
throttle: number | null;
timeout: number | null;
lastSignalTs: number | null;
lastSignalTimestamp: number | null;
};
export type SenseReflexConfig = {
kind: "sense";
sense: string;
interval: number | null;
on: string[] | null;
on: string[];
};
export type WorkflowReflexConfig = {
kind: "workflow";
workflow: string;
on: string[] | null;
};
export type ReflexConfig = SenseReflexConfig | WorkflowReflexConfig;
/** Reflexes only schedule Senses; workflow launches come from Sense return values. */
export type ReflexConfig = SenseReflexConfig;
export type DropOverflowConfig = {
concurrency: number;
@@ -50,62 +45,91 @@ export type QueueOverflowConfig = {
export type WorkflowConfig = DropOverflowConfig | QueueOverflowConfig;
export type NerveConfig = {
/** Engine-wide default max moderator rounds (e.g. CLI workflow trigger when omitted). */
maxRounds: number;
senses: Record<string, SenseConfig>;
reflexes: ReflexConfig[];
workflows: Record<string, WorkflowConfig> | null;
workflows: Record<string, WorkflowConfig>;
};
// ---------------------------------------------------------------------------
// Workflow Engine types (RFC-002)
// Workflow Automaton types (issue #80)
// ---------------------------------------------------------------------------
/** A single event in the command event stream that drives a workflow thread. */
export type CommandEvent = {
type: string;
[key: string]: unknown;
};
export const START = "__start__" as const;
export const END = "__end__" as const;
export type START = typeof START;
export type END = typeof END;
/** Accumulated state of a running thread — the event history for moderate(). */
export type ThreadState = {
runId: string;
/** All events so far, including the initial thread_start event. */
events: CommandEvent[];
};
/** Engine-wide fallback for max moderator rounds when not specified in config. */
export const DEFAULT_ENGINE_MAX_ROUNDS = 100;
/** The result of moderate() — which role to hand to next, and what prompt to pass. */
export type ModerateResult = {
/** A single message in the workflow conversation chain (runtime, type-erased). */
export type WorkflowMessage = {
role: string;
prompt: unknown;
content: string;
meta: unknown;
timestamp: number;
};
/** Context injected into every role execute() call. */
export type WorkflowContext = {
runId: string;
workflowName: string;
/** Emit a log message back to the parent process. */
log: (message: string) => void;
/** The typed output of a Role execution. */
export type RoleResult<Meta> = { content: string; meta: Meta };
/**
* A Role is a pure async function: receives the engine start frame plus prior
* role messages only (the start frame is not included in `messages`).
* Returns typed content + meta. Implementation can be an agent, LLM call,
* script, HTTP request, etc.
*/
export type Role<Meta> = (
start: StartStep,
messages: WorkflowMessage[],
) => Promise<RoleResult<Meta>>;
/** Maps role names to their meta types — the single generic that drives all inference. */
export type RoleMeta = Record<string, Record<string, unknown>>;
/** Engine start frame: prompt, max rounds cap, dry-run flag, and timestamps for the thread. */
export type StartStep = {
role: START;
content: string;
meta: { maxRounds: number; dryRun: boolean };
timestamp: number;
};
/** A discriminated union of role steps after each execution, aligned with `StartStep` shape. */
export type RoleStep<M extends RoleMeta> = {
[K in keyof M & string]: { role: K; meta: M[K]; content: string; timestamp: number };
}[keyof M & string];
/**
* Moderator input: the complete workflow history.
* Contains the start frame and all role steps so far.
* On initial call, `steps` is empty — moderator can check `steps.length === 0`.
* Round count is `steps.length`; maxRounds is in `start.meta.maxRounds`.
*/
export type ModeratorContext<M extends RoleMeta> = {
start: StartStep;
steps: RoleStep<M>[];
};
/**
* A role's execute function. Has side effects (API calls, file I/O, etc.).
* Returns a CommandEvent that is fed back into moderate().
* The moderator — a pure routing function. Receives the full workflow context
* (start frame + all prior steps). Returns the next role name or END.
*/
export type RoleExecuteFn = (prompt: unknown, ctx: WorkflowContext) => Promise<CommandEvent>;
/** A role in a workflow — a named unit of execution with side effects. */
export type Role = {
execute: RoleExecuteFn;
};
/**
* The moderator function — pure, no side effects.
* Decides which role to pass control to next.
* Returns null to signal thread completion.
*/
export type ModerateFn = (thread: ThreadState, event: CommandEvent) => ModerateResult | null;
export type Moderator<M extends RoleMeta> = (
context: ModeratorContext<M>,
) => (keyof M & string) | END;
/** The complete definition of a workflow, as authored by users. */
export type WorkflowDefinition = {
roles: Record<string, Role>;
moderate: ModerateFn;
export type WorkflowDefinition<M extends RoleMeta> = {
name: string;
roles: { [K in keyof M & string]: Role<M[K]> };
moderator: Moderator<M>;
};
/** The result of a Sense compute — payload plus optional workflow directive. */
export type SenseResult = {
payload: unknown;
workflow: string | null;
};
-8
View File
@@ -1,8 +0,0 @@
import { defineConfig } from "tsup";
export default defineConfig({
entry: ["src/index.ts"],
format: ["esm"],
dts: true,
clean: true,
});
+92
View File
@@ -0,0 +1,92 @@
# @uncaged/nerve-daemon
The observation engine runtime for [nerve](../../README.md) — runs senses, routes signals, schedules reflexes, and manages workflows.
## Architecture
| Module | Source (indicative) | Responsibility |
|--------|---------------------|----------------|
| **Kernel** | `kernel.ts` | Orchestrator — worker pool, signal bus, reflex scheduler, workflow manager, optional file watcher and daemon IPC, config reload hooks |
| **Worker pool** | `worker-pool.ts` | Fork and supervise one child process per sense group; restart/shutdown; crash cleanup hooks for scheduler state |
| **Kernel sense groups** | `kernel-sense-groups.ts` | Derive sense groups from config; list senses per group for scheduling |
| **Sense runtime** | sense worker + Drizzle | Per-sense SQLite (`node:sqlite`), migrations, peer DB reads |
| **Sense worker** | `sense-worker.ts` (fork target) | Child process entry — runs `compute()` per sense in a group |
| **Signal bus** | `signal-bus.ts` | In-memory pub/sub for sense signals |
| **Reflex scheduler** | `reflex-scheduler.ts` | Interval + `on` subscriptions, throttle/coalesce |
| **Workflow manager** | `workflow-manager.ts` | One worker per workflow name, concurrency (drop/queue), queue caps |
| **Workflow worker** | `workflow-worker.ts` | Child process — runs RFC-002 threads (`start-thread`, `resume-thread` IPC) |
| **IPC (parent ↔ workers)** | `ipc.ts` | Typed messages for sense and workflow workers (includes `resume-thread` for recovery) |
| **Log / workflow persistence** | via `@uncaged/nerve-store` | Structured logs, `workflow_runs`, thread messages (used for recovery) |
| **Blob store** | `@uncaged/nerve-store` | CAS under `data/blobs/` — sense workers construct `createBlobStore(join(nerveRoot, "data", "blobs"))` for artifact writes |
| **File watcher** | `file-watcher.ts` | Watches workspace paths for config / sense / workflow file changes |
| **Kernel file watch** | `kernel-file-watch.ts` | Maps watcher events to `reloadConfig`, sense group restart, workflow `drainAndRespawn` |
| **Daemon IPC** | `daemon-ipc.ts` | Unix socket server — parses `@uncaged/nerve-core` `DaemonIpcRequest`, dispatches trigger-workflow / trigger-sense / list-senses |
## Crash recovery (workflow workers)
If a workflow worker exits unexpectedly while threads are active:
- In-flight runs are marked **`crashed`** in the log store; the manager respawns a fresh worker.
- Runs still in **`started`** state can be **`resume-thread`**’d: the manager rebuilds the message chain from persisted workflow log rows and sends `resume-thread` to the new worker.
- **Crash-loop backoff:** repeated crashes for the same workflow name are counted in a sliding window (`60s`); after **`5`** crashes in that window, the manager **stops respawning** that worker and logs the condition (avoids tight crash loops).
Hot reload (`drainAndRespawn`) uses a controlled drain: in-flight runs may be marked **`interrupted`** when the old worker is torn down after a timeout — that path is distinct from unexpected crash recovery.
## Key Design Decisions
- **One worker process per sense group** — isolation between groups, shared compute within a group
- **`node:sqlite` (DatabaseSync)** — zero native addons, WAL mode, built into Node.js ≥ 22.5
- **Throttle + coalesce** — if compute is in-flight, at most one pending trigger is queued (no unbounded accumulation)
- **Log ≠ Signal** — logs are queryable data assets but cannot trigger reflexes (prevents feedback loops)
## Usage
The daemon is typically started via the CLI (`nerve daemon start` / `nerve dev`), but you can embed the kernel:
```typescript
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { parseNerveConfig } from "@uncaged/nerve-core";
import { createKernel } from "@uncaged/nerve-daemon";
const nerveRoot = "/path/to/workspace";
const yamlPath = join(nerveRoot, "nerve.yaml");
const parsed = parseNerveConfig(readFileSync(yamlPath, "utf8"));
if (!parsed.ok) {
throw parsed.error;
}
const kernel = createKernel(parsed.value, nerveRoot, {
enableFileWatcher: true,
ipcSocketPath: join(nerveRoot, "nerve.sock"),
});
await kernel.ready;
kernel.triggerSense("cpu-usage");
const health = kernel.getHealth();
await kernel.stop();
```
`createKernel(config, nerveRoot, options?)``config` is a parsed `NerveConfig`; `nerveRoot` is the workspace root (contains `nerve.yaml`, `data/`, etc.). Optional `KernelOptions`:
| Field | Meaning |
|-------|---------|
| `workerScript` | Override path to the sense worker entry script (defaults to the package’s resolved worker) |
| `enableFileWatcher` | Watch config / senses / workflows for hot reload |
| `logStore` | Inject a `LogStore` instance (defaults to `createLogStore(join(nerveRoot, "data", "logs.db"))`) |
| `ipcSocketPath` | When non-null, listen for daemon IPC on this Unix socket path |
## Install
```bash
pnpm add @uncaged/nerve-daemon
```
Requires Node.js ≥ 22.5 (for `node:sqlite`).
## License
MIT
+7 -8
View File
@@ -1,28 +1,27 @@
{
"name": "@uncaged/nerve-daemon",
"version": "0.1.5",
"version": "0.4.0",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": [
"dist"
],
"files": ["dist"],
"publishConfig": {
"access": "public"
},
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "tsup",
"build": "rslib build",
"test": "vitest run"
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*",
"better-sqlite3": "^11.10.0",
"drizzle-orm": "^0.43.1",
"@uncaged/nerve-store": "workspace:*",
"drizzle-orm": "1.0.0-beta.23-c10d10c",
"yaml": "^2.8.3"
},
"devDependencies": {
"@types/better-sqlite3": "^7.6.13",
"@rslib/core": "^0.21.3",
"@types/node": "^22.0.0",
"vitest": "^4.1.5"
}
}
+21
View File
@@ -0,0 +1,21 @@
import { defineConfig } from "@rslib/core";
export default defineConfig({
lib: [
{
format: "esm",
dts: true,
},
],
source: {
entry: {
index: "src/index.ts",
"sense-worker": "src/sense-worker.ts",
"workflow-worker": "src/workflow-worker.ts",
},
},
output: {
target: "node",
cleanDistPath: true,
},
});
@@ -64,6 +64,7 @@ function makeConfig(workflows: Record<string, WorkflowConfig> = {}): NerveConfig
senses: {},
reflexes: [],
workflows,
maxRounds: 10,
};
}
@@ -72,7 +73,7 @@ function makeLogStore(
runId: string;
workflow: string;
status: "queued" | "started";
ts: number;
timestamp: number;
}> = [],
) {
const store = {
@@ -89,10 +90,20 @@ function makeLogStore(
}
return activeRuns;
}),
getTriggerPayload: vi.fn(() => ({ value: 42 })),
getThreadEvents: vi.fn(() => [{ type: "thread_start", triggerPayload: {} }]),
getTriggerPayload: vi.fn((): unknown => ({ value: 42 })),
getThreadEvents: vi.fn(
(): Array<{ type: string; [key: string]: unknown }> => [
{ type: "thread_start", triggerPayload: {} },
],
),
getThreadMessages: vi.fn(
(): Array<{ role: string; content: string; meta: unknown; timestamp: number }> => [],
),
getThreadRoundCount: vi.fn(() => 0),
getThreadRounds: vi.fn(() => []),
archiveLogs: vi.fn(() => ({ days: [], vacuumed: false })),
close: vi.fn(),
getAllWorkflowRuns: vi.fn(() => []),
};
return store;
}
@@ -116,8 +127,8 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { n: 1 });
mgr.startWorkflow("my-wf", { n: 2 });
mgr.startWorkflow("my-wf", { prompt: "test 1", maxRounds: 10, dryRun: false });
mgr.startWorkflow("my-wf", { prompt: "test 2", maxRounds: 10, dryRun: false });
expect(mgr.activeCount("my-wf")).toBe(2);
// Simulate unexpected exit (not shutdown)
@@ -127,7 +138,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
child.emit("exit", 1, null);
const crashedCalls = logStore.upsertWorkflowRun.mock.calls.filter(
([entry]: [{ type: string }]) => entry.type === "crashed",
(args: any[]) => (args[0] as { type: string }).type === "crashed",
);
expect(crashedCalls).toHaveLength(2);
@@ -143,8 +154,8 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
expect(mgr.activeCount("my-wf")).toBe(2);
const child = mockChildren[0];
@@ -168,7 +179,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
expect(mockChildren).toHaveLength(1);
const child = mockChildren[0];
@@ -188,12 +199,11 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
it("sends resume-thread for 'started' runs from DB after respawn", async () => {
const activeRuns = [
{ runId: "run-started-1", workflow: "my-wf", status: "started" as const, ts: 1000 },
{ runId: "run-started-1", workflow: "my-wf", status: "started" as const, timestamp: 1000 },
];
const logStore = makeLogStore(activeRuns);
logStore.getThreadEvents.mockReturnValue([
{ type: "thread_start", triggerPayload: {} },
{ type: "scan_complete", items: ["a"] },
logStore.getThreadMessages.mockReturnValue([
{ role: "scanner", content: "done", meta: { items: ["a"] }, timestamp: 1000 },
]);
logStore.getTriggerPayload.mockReturnValue({ trigger: "initial" });
@@ -202,7 +212,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
const firstChild = mockChildren[0];
firstChild.exitCode = 1;
firstChild.connected = false;
@@ -216,18 +226,17 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
// resume-thread should have been sent
const resumeCalls = (secondChild.send as ReturnType<typeof vi.fn>).mock.calls.filter(
([msg]: [unknown]) =>
msg !== null &&
typeof msg === "object" &&
(msg as Record<string, unknown>).type === "resume-thread",
(args: any[]) =>
args[0] !== null &&
typeof args[0] === "object" &&
(args[0] as Record<string, unknown>).type === "resume-thread",
);
expect(resumeCalls).toHaveLength(1);
expect(resumeCalls[0][0]).toMatchObject({
type: "resume-thread",
runId: "run-started-1",
triggerPayload: { trigger: "initial" },
});
expect(Array.isArray((resumeCalls[0][0] as Record<string, unknown>).events)).toBe(true);
expect(Array.isArray((resumeCalls[0][0] as Record<string, unknown>).messages)).toBe(true);
const stopPromise = mgr.stop();
await vi.runAllTimersAsync();
@@ -236,7 +245,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
it("re-queues 'queued' runs from DB after respawn", async () => {
const activeRuns = [
{ runId: "run-queued-1", workflow: "my-wf", status: "queued" as const, ts: 900 },
{ runId: "run-queued-1", workflow: "my-wf", status: "queued" as const, timestamp: 900 },
];
const logStore = makeLogStore(activeRuns);
logStore.getTriggerPayload.mockReturnValue({ queued: "payload" });
@@ -247,7 +256,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
// Start one thread to fill the concurrency slot (so queued run stays queued on respawn)
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
const firstChild = mockChildren[0];
firstChild.exitCode = 1;
firstChild.connected = false;
@@ -264,34 +273,33 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
});
});
describe("command events are persisted (for crash recovery replay)", () => {
it("persists thread_command_event when worker sends thread-command-event IPC", async () => {
describe("workflow messages are persisted (for crash recovery replay)", () => {
it("persists thread_workflow_message when worker sends thread-workflow-message IPC", async () => {
const logStore = makeLogStore();
const config = makeConfig({
"my-wf": { concurrency: 1, overflow: "drop" },
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { x: 1 });
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
const child = mockChildren[0];
const startCall = (child.send as ReturnType<typeof vi.fn>).mock.calls[0];
const runId = (startCall[0] as Record<string, unknown>).runId as string;
// Simulate worker sending a command event back
child.emit("message", {
type: "thread-command-event",
type: "thread-workflow-message",
runId,
event: { type: "scan_complete", items: ["a", "b"] },
message: { role: "scanner", content: "done", meta: { items: ["a", "b"] }, timestamp: 1000 },
});
const appendCalls = logStore.append.mock.calls.filter(
([entry]: [{ type: string }]) => entry.type === "thread_command_event",
(args: any[]) => (args[0] as { type: string }).type === "thread_workflow_message",
);
expect(appendCalls).toHaveLength(1);
expect(appendCalls[0][0]).toMatchObject({
source: "workflow",
type: "thread_command_event",
type: "thread_workflow_message",
refId: runId,
});
@@ -309,17 +317,21 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
const payload = { task: "build-docker", repo: "myrepo" };
mgr.startWorkflow("my-wf", payload);
const launch = { prompt: "build-docker for myrepo", maxRounds: 10, dryRun: false };
mgr.startWorkflow("my-wf", launch);
const startedCall = logStore.upsertWorkflowRun.mock.calls.find(
([entry]: [{ type: string }]) => entry.type === "started",
(args: any[]) => (args[0] as { type: string }).type === "started",
);
expect(startedCall).toBeDefined();
const logEntry = startedCall?.[0] as { payload: string | null };
expect(logEntry.payload).not.toBeNull();
const parsed = JSON.parse(logEntry.payload as string) as Record<string, unknown>;
expect(parsed.triggerPayload).toMatchObject(payload);
expect(parsed).toMatchObject({
prompt: "build-docker for myrepo",
maxRounds: 10,
dryRun: false,
});
const stopPromise = mgr.stop();
await vi.runAllTimersAsync();
@@ -330,7 +342,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
describe("runId deduplication in crash recovery", () => {
it("does not push duplicate runIds into the queue during crash recovery", async () => {
const activeRuns = [
{ runId: "run-queued-dup", workflow: "my-wf", status: "queued" as const, ts: 900 },
{ runId: "run-queued-dup", workflow: "my-wf", status: "queued" as const, timestamp: 900 },
];
const logStore = makeLogStore(activeRuns);
logStore.getTriggerPayload.mockReturnValue({ q: 1 });
@@ -341,7 +353,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
// Start one thread to fill the concurrency slot
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
const firstChild = mockChildren[0];
// Crash once → respawn → crash again → second respawn
@@ -366,10 +378,15 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
it("does not add duplicate active runIds during crash recovery", async () => {
const activeRuns = [
{ runId: "run-started-dup", workflow: "my-wf", status: "started" as const, ts: 1000 },
{
runId: "run-started-dup",
workflow: "my-wf",
status: "started" as const,
timestamp: 1000,
},
];
const logStore = makeLogStore(activeRuns);
logStore.getThreadEvents.mockReturnValue([{ type: "thread_start", triggerPayload: {} }]);
logStore.getThreadMessages.mockReturnValue([]);
logStore.getTriggerPayload.mockReturnValue({ s: 1 });
const config = makeConfig({
@@ -377,7 +394,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
const firstChild = mockChildren[0];
firstChild.exitCode = 1;
firstChild.connected = false;
@@ -407,7 +424,7 @@ describe("WorkflowManager — crash recovery (Phase 3)", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("crash-wf", {});
mgr.startWorkflow("crash-wf", { prompt: "test", maxRounds: 10, dryRun: false });
// Crash the worker 6 times in rapid succession (within CRASH_WINDOW_MS = 60s)
for (let i = 0; i < 6; i++) {
@@ -2,7 +2,7 @@
* Unit + integration tests for daemon-ipc.ts — trigger-sense request type.
*
* Tests cover:
* - parseRequest correctly accepts/rejects trigger-sense messages
* - parseDaemonIpcRequest (core) / server correctly accept or reject trigger-sense messages
* - createDaemonIpcServer routes trigger-sense to opts.triggerSense
* - Error response when triggerSense throws (unknown sense)
* - Success response on valid sense trigger
@@ -63,7 +63,10 @@ function sendRaw(path: string, message: object): Promise<object> {
}
beforeEach(() => {
sockPath = join(tmpdir(), `nerve-ipc-test-${Date.now()}-${Math.random().toString(36).slice(2)}.sock`);
sockPath = join(
tmpdir(),
`nerve-ipc-test-${Date.now()}-${Math.random().toString(36).slice(2)}.sock`,
);
});
afterEach(async () => {
@@ -149,12 +152,18 @@ describe("daemon-ipc — trigger-sense", () => {
const resp = await sendRaw(sockPath, {
type: "trigger-workflow",
workflow: "my-workflow",
payload: {},
prompt: "test prompt",
maxRounds: 10,
dryRun: false,
});
expect(resp).toEqual({ ok: true });
expect(triggerSense).not.toHaveBeenCalled();
expect(wfManager.startWorkflow).toHaveBeenCalledWith("my-workflow", {});
expect(wfManager.startWorkflow).toHaveBeenCalledWith("my-workflow", {
prompt: "test prompt",
maxRounds: 10,
dryRun: false,
});
});
it("responds ok:false for completely unknown request type", async () => {
@@ -191,8 +200,20 @@ describe("daemon-ipc — list-senses", () => {
it("responds ok:true with senses populated from listSenses", async () => {
const sensesData = [
{ name: "cpu-usage", group: "system", throttle: 5000, timeout: 3000, lastSignalTs: 1000 },
{ name: "disk-usage", group: "system", throttle: 30000, timeout: null, lastSignalTs: null },
{
name: "cpu-usage",
group: "system",
throttle: 5000,
timeout: 3000,
lastSignalTimestamp: 1000,
},
{
name: "disk-usage",
group: "system",
throttle: 30000,
timeout: null,
lastSignalTimestamp: null,
},
];
const listSenses = vi.fn(() => sensesData);
server = createDaemonIpcServer(sockPath, makeMockWorkflowManager() as never, {
@@ -10,6 +10,9 @@
*/
import { EventEmitter } from "node:events";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { NerveConfig, WorkflowConfig } from "@uncaged/nerve-core";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
@@ -62,7 +65,7 @@ const { createWorkflowManager } = await import("../workflow-manager.js");
const { createKernel } = await import("../kernel.js");
function makeWfConfig(workflows: Record<string, WorkflowConfig> = {}): NerveConfig {
return { senses: {}, reflexes: [], workflows };
return { senses: {}, reflexes: [], workflows, maxRounds: 10 };
}
function makeLogStore() {
@@ -77,8 +80,12 @@ function makeLogStore() {
getActiveWorkflowRuns: vi.fn(() => []),
getTriggerPayload: vi.fn(() => null),
getThreadEvents: vi.fn(() => []),
getThreadMessages: vi.fn(() => []),
getThreadRoundCount: vi.fn(() => 0),
getThreadRounds: vi.fn(() => []),
archiveLogs: vi.fn(() => ({ days: [], vacuumed: false })),
close: vi.fn(),
getAllWorkflowRuns: vi.fn(() => []),
};
}
@@ -98,7 +105,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const config = makeWfConfig({ "my-wf": { concurrency: 1, overflow: "drop" } });
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
expect(mockChildren).toHaveLength(1);
// Remove workflow from config before drain completes
@@ -117,8 +124,8 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const config = makeWfConfig({ "my-wf": { concurrency: 2, overflow: "drop" } });
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { n: 1 });
mgr.startWorkflow("my-wf", { n: 2 });
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
expect(mgr.activeCount("my-wf")).toBe(2);
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
@@ -126,7 +133,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
await drainPromise;
const interruptedCalls = logStore.upsertWorkflowRun.mock.calls.filter(
([entry]: [{ type: string }]) => entry.type === "interrupted",
(args: any[]) => (args[0] as { type: string }).type === "interrupted",
);
expect(interruptedCalls).toHaveLength(2);
@@ -149,7 +156,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const config = makeWfConfig({ "my-wf": { concurrency: 1, overflow: "drop" } });
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
expect(mockChildren).toHaveLength(1);
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
@@ -165,7 +172,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const config = makeWfConfig({ "my-wf": { concurrency: 1, overflow: "drop" } });
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
expect(mockChildren).toHaveLength(1);
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
@@ -182,7 +189,7 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const config = makeWfConfig({ "my-wf": { concurrency: 1, overflow: "drop" } });
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", {});
mgr.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
await vi.runAllTimersAsync();
@@ -190,10 +197,10 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const newChild = mockChildren[1];
const resumeCalls = (newChild.send as ReturnType<typeof vi.fn>).mock.calls.filter(
([msg]: [unknown]) =>
msg !== null &&
typeof msg === "object" &&
(msg as Record<string, unknown>).type === "resume-thread",
(args: any[]) =>
args[0] !== null &&
typeof args[0] === "object" &&
(args[0] as Record<string, unknown>).type === "resume-thread",
);
expect(resumeCalls).toHaveLength(0);
@@ -207,21 +214,21 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
const config = makeWfConfig({ "my-wf": { concurrency: 1, overflow: "drop" } });
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-wf", { first: true });
mgr.startWorkflow("my-wf", { prompt: "first", maxRounds: 10, dryRun: false });
const drainPromise = mgr.drainAndRespawn("my-wf", 5000);
await vi.runAllTimersAsync();
await drainPromise;
// Start a new thread on the fresh worker
mgr.startWorkflow("my-wf", { second: true });
mgr.startWorkflow("my-wf", { prompt: "second", maxRounds: 10, dryRun: false });
const newChild = mockChildren[1];
const startCalls = (newChild.send as ReturnType<typeof vi.fn>).mock.calls.filter(
([msg]: [unknown]) =>
msg !== null &&
typeof msg === "object" &&
(msg as Record<string, unknown>).type === "start-thread",
(args: any[]) =>
args[0] !== null &&
typeof args[0] === "object" &&
(args[0] as Record<string, unknown>).type === "start-thread",
);
expect(startCalls).toHaveLength(1);
@@ -232,31 +239,36 @@ describe("WorkflowManager — drainAndRespawn (Phase 3 hot reload)", () => {
});
describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
let nerveRoot: string;
beforeEach(() => {
mockChildren.length = 0;
vi.useFakeTimers({ shouldAdvanceTime: true });
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-hot-reload-"));
});
afterEach(async () => {
vi.useRealTimers();
vi.clearAllMocks();
rmSync(nerveRoot, { recursive: true, force: true });
});
it("handleWorkflowFileChange logs workflow_reload system event", async () => {
const logStore = makeLogStore();
const config: NerveConfig = {
senses: {},
reflexes: [{ kind: "workflow", workflow: "my-wf", on: null }],
reflexes: [],
workflows: { "my-wf": { concurrency: 1, overflow: "drop" } },
maxRounds: 10,
};
const kernel = createKernel(config, "/tmp/nerve-hot-reload-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
// Trigger a workflow thread so a worker is spawned
kernel.workflowManager.startWorkflow("my-wf", {});
kernel.workflowManager.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
// Manually call drainAndRespawn (simulating what kernel does on workflow file change)
const drainPromise = kernel.workflowManager.drainAndRespawn("my-wf", 1000);
@@ -266,7 +278,9 @@ describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
// Kernel's handleWorkflowFileChange should log a workflow_reload event
// We test this via the kernel itself
const appendCalls = logStore.append.mock.calls;
const startCall = appendCalls.find(([e]: [{ type: string }]) => e.type === "start");
const startCall = appendCalls.find(
(args: any[]) => (args[0] as { type: string }).type === "start",
);
expect(startCall).toBeDefined();
const stopPromise = kernel.stop();
@@ -278,24 +292,30 @@ describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
const logStore = makeLogStore();
const initialConfig: NerveConfig = {
senses: {},
reflexes: [{ kind: "workflow", workflow: "old-wf", on: null }],
reflexes: [],
workflows: { "old-wf": { concurrency: 1, overflow: "drop" } },
maxRounds: 10,
};
const kernel = createKernel(initialConfig, "/tmp/nerve-hot-reload-test", {
const kernel = createKernel(initialConfig, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
// Spawn a worker for old-wf
kernel.workflowManager.startWorkflow("old-wf", {});
kernel.workflowManager.startWorkflow("old-wf", {
prompt: "test",
maxRounds: 10,
dryRun: false,
});
expect(mockChildren).toHaveLength(1);
// Reload config without old-wf
const newConfig: NerveConfig = {
senses: {},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -315,23 +335,25 @@ describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
const logStore = makeLogStore();
const initialConfig: NerveConfig = {
senses: {},
reflexes: [{ kind: "workflow", workflow: "my-wf", on: null }],
reflexes: [],
workflows: { "my-wf": { concurrency: 1, overflow: "drop" } },
maxRounds: 10,
};
const kernel = createKernel(initialConfig, "/tmp/nerve-hot-reload-test", {
const kernel = createKernel(initialConfig, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
kernel.workflowManager.startWorkflow("my-wf", {});
kernel.workflowManager.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
const workersBefore = mockChildren.length;
// Reload with updated concurrency — should NOT spawn a new workflow worker
const newConfig: NerveConfig = {
senses: {},
reflexes: [{ kind: "workflow", workflow: "my-wf", on: null }],
reflexes: [],
workflows: { "my-wf": { concurrency: 5, overflow: "queue", maxQueue: 50 } },
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -343,8 +365,8 @@ describe("Kernel — workflow hot reload via file-watcher (Phase 3)", () => {
expect(kernel.workflowManager.activeCount("my-wf")).toBe(1);
// Can now start up to 5 concurrent threads (previously only 1)
kernel.workflowManager.startWorkflow("my-wf", { n: 2 });
kernel.workflowManager.startWorkflow("my-wf", { n: 3 });
kernel.workflowManager.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
kernel.workflowManager.startWorkflow("my-wf", { prompt: "test", maxRounds: 10, dryRun: false });
expect(kernel.workflowManager.activeCount("my-wf")).toBe(3);
const stopPromise = kernel.stop();
@@ -6,12 +6,14 @@
* artifacts are required.
*/
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { Signal } from "@uncaged/nerve-core";
import type { NerveConfig } from "@uncaged/nerve-core";
import { afterEach, describe, expect, it } from "vitest";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { createKernel } from "../kernel.js";
import type { Kernel } from "../kernel.js";
@@ -26,7 +28,8 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
...overrides,
};
}
@@ -54,12 +57,18 @@ async function pollUntil(
describe("kernel integration — real child processes", () => {
let kernel: Kernel | null = null;
let nerveRoot: string;
beforeEach(() => {
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-integration-"));
});
afterEach(async () => {
if (kernel !== null) {
await kernel.stop();
kernel = null;
}
rmSync(nerveRoot, { recursive: true, force: true });
});
it("returns correct groups and senseCount", () => {
@@ -70,7 +79,7 @@ describe("kernel integration — real child processes", () => {
"net-rx": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
});
kernel = createKernel(config, "/tmp/nerve-integration-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
@@ -82,7 +91,7 @@ describe("kernel integration — real child processes", () => {
it("workers start and respond to compute messages with signals", async () => {
const config = makeConfig();
kernel = createKernel(config, "/tmp/nerve-integration-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
@@ -114,7 +123,7 @@ describe("kernel integration — real child processes", () => {
"net-rx": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
});
kernel = createKernel(config, "/tmp/nerve-integration-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
@@ -130,7 +139,7 @@ describe("kernel integration — real child processes", () => {
it("compute round-trip: worker receives compute and sends signal back through bus", async () => {
const config = makeConfig();
kernel = createKernel(config, "/tmp/nerve-integration-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
@@ -157,7 +166,7 @@ describe("kernel integration — real child processes", () => {
it("crash recovery: kernel respawns worker after unexpected exit and new worker is functional", async () => {
const config = makeConfig();
kernel = createKernel(config, "/tmp/nerve-integration-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
@@ -4,6 +4,9 @@
*/
import { EventEmitter } from "node:events";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { NerveConfig } from "@uncaged/nerve-core";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
@@ -73,7 +76,8 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
...overrides,
};
}
@@ -83,13 +87,17 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
// ---------------------------------------------------------------------------
describe("kernel — getHealth", () => {
let nerveRoot: string;
beforeEach(() => {
mockChildren.length = 0;
vi.useFakeTimers({ shouldAdvanceTime: true });
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-kernel-p6-health-"));
});
afterEach(() => {
vi.useRealTimers();
rmSync(nerveRoot, { recursive: true, force: true });
});
it("returns correct health shape", async () => {
@@ -100,7 +108,7 @@ describe("kernel — getHealth", () => {
"net-rx": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
});
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
const health = kernel.getHealth();
expect(health.activeSenses).toBe(3);
@@ -114,18 +122,22 @@ describe("kernel — getHealth", () => {
});
describe("kernel — restartGroup", () => {
let nerveRoot: string;
beforeEach(() => {
mockChildren.length = 0;
vi.useFakeTimers({ shouldAdvanceTime: true });
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-kernel-p6-restart-"));
});
afterEach(() => {
vi.useRealTimers();
rmSync(nerveRoot, { recursive: true, force: true });
});
it("sends shutdown to old worker and spawns new one", async () => {
const config = makeConfig();
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
expect(mockChildren.length).toBe(1);
const oldChild = mockChildren[0];
@@ -145,7 +157,7 @@ describe("kernel — restartGroup", () => {
it("restartGroup on unknown group does nothing", async () => {
const config = makeConfig();
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
expect(mockChildren.length).toBe(1);
await kernel.restartGroup("nonexistent");
@@ -157,18 +169,22 @@ describe("kernel — restartGroup", () => {
});
describe("kernel — reloadConfig", () => {
let nerveRoot: string;
beforeEach(() => {
mockChildren.length = 0;
vi.useFakeTimers({ shouldAdvanceTime: true });
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-kernel-p6-reload-"));
});
afterEach(() => {
vi.useRealTimers();
rmSync(nerveRoot, { recursive: true, force: true });
});
it("adds new group worker when new sense group appears", async () => {
const config = makeConfig();
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
expect(mockChildren.length).toBe(1); // only system group
expect(kernel.groups.has("network")).toBe(false);
@@ -179,7 +195,8 @@ describe("kernel — reloadConfig", () => {
"net-rx": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
});
expect(kernel.groups.has("network")).toBe(true);
@@ -195,9 +212,10 @@ describe("kernel — reloadConfig", () => {
"net-rx": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
};
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
expect(mockChildren.length).toBe(2);
expect(kernel.groups.has("network")).toBe(true);
@@ -209,7 +227,8 @@ describe("kernel — reloadConfig", () => {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
});
expect(kernel.groups.has("network")).toBe(false);
@@ -221,7 +240,7 @@ describe("kernel — reloadConfig", () => {
it("health reflects updated sense count after reloadConfig", async () => {
const config = makeConfig();
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
expect(kernel.getHealth().activeSenses).toBe(1);
@@ -231,7 +250,8 @@ describe("kernel — reloadConfig", () => {
"disk-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
});
expect(kernel.getHealth().activeSenses).toBe(2);
@@ -2,10 +2,13 @@
* Unit tests for kernel.triggerSense() — IPC issue #36.
*
* These tests use a mock child_process and a mock LogStore so they do NOT
* require better-sqlite3 to be present in the test environment.
* require a real LogStore (node:sqlite) in integration tests.
*/
import { EventEmitter } from "node:events";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { NerveConfig } from "@uncaged/nerve-core";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
@@ -58,7 +61,7 @@ vi.mock("node:child_process", () => ({
const { createKernel } = await import("../kernel.js");
// ---------------------------------------------------------------------------
// Mock LogStore factory (avoids better-sqlite3 dependency)
// Mock LogStore factory (avoids SQLite I/O in this unit test)
// ---------------------------------------------------------------------------
function makeMockLogStore() {
@@ -74,6 +77,9 @@ function makeMockLogStore() {
getAllWorkflowRuns: vi.fn(() => []),
getTriggerPayload: vi.fn(() => null),
getThreadEvents: vi.fn(() => []),
getThreadMessages: vi.fn(() => []),
getThreadRoundCount: vi.fn(() => 0),
getThreadRounds: vi.fn(() => []),
archiveLogs: vi.fn(() => ({ days: [], vacuumed: false })),
close: vi.fn(),
};
@@ -89,7 +95,8 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
...overrides,
};
}
@@ -99,18 +106,22 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
// ---------------------------------------------------------------------------
describe("kernel.triggerSense()", () => {
let nerveRoot: string;
beforeEach(() => {
mockChildren.length = 0;
vi.useFakeTimers({ shouldAdvanceTime: true });
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-trigger-sense-"));
});
afterEach(() => {
vi.useRealTimers();
rmSync(nerveRoot, { recursive: true, force: true });
});
it("throws for an unknown sense name", async () => {
const config = makeConfig();
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: null,
logStore: makeMockLogStore() as never,
});
@@ -128,7 +139,7 @@ describe("kernel.triggerSense()", () => {
},
reflexes: [],
});
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: null,
logStore: makeMockLogStore() as never,
});
@@ -158,7 +169,7 @@ describe("kernel.triggerSense()", () => {
},
reflexes: [],
});
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: null,
logStore: makeMockLogStore() as never,
});
@@ -181,7 +192,7 @@ describe("kernel.triggerSense()", () => {
vi.useRealTimers();
const config = makeConfig();
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: null,
logStore: makeMockLogStore() as never,
});
@@ -191,9 +202,7 @@ describe("kernel.triggerSense()", () => {
// Should not throw even when the worker is disconnected
expect(() => kernel.triggerSense("cpu-usage")).not.toThrow();
expect(worker.send).not.toHaveBeenCalledWith(
expect.objectContaining({ type: "compute" }),
);
expect(worker.send).not.toHaveBeenCalledWith(expect.objectContaining({ type: "compute" }));
await kernel.stop();
}, 10_000);
@@ -9,6 +9,9 @@
*/
import { EventEmitter } from "node:events";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { NerveConfig } from "@uncaged/nerve-core";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
@@ -78,8 +81,12 @@ function makeLogStore() {
appendWithWorkflowUpdate: vi.fn(),
getWorkflowRun: vi.fn(() => null),
getActiveWorkflowRuns: vi.fn(() => []),
getAllWorkflowRuns: vi.fn(() => []),
getTriggerPayload: vi.fn(() => null),
getThreadEvents: vi.fn(() => []),
getThreadMessages: vi.fn(() => []),
getThreadRoundCount: vi.fn(() => 0),
getThreadRounds: vi.fn(() => []),
archiveLogs: vi.fn(() => ({ days: [], vacuumed: false })),
close: vi.fn(),
};
@@ -91,7 +98,8 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
...overrides,
};
}
@@ -101,46 +109,56 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
// ---------------------------------------------------------------------------
describe("kernel + workflowManager integration", () => {
let nerveRoot: string;
beforeEach(() => {
mockChildren.length = 0;
vi.useFakeTimers({ shouldAdvanceTime: true });
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-kernel-wf-"));
});
afterEach(async () => {
vi.useRealTimers();
vi.clearAllMocks();
rmSync(nerveRoot, { recursive: true, force: true });
});
describe("sense signal triggers workflow via reflex", () => {
it("calls workflowManager.startWorkflow when a sense signal fires on a workflow reflex", async () => {
describe("sense compute triggers workflow via return value", () => {
it("calls workflowManager.startWorkflow when a sense compute returns a workflow launch", async () => {
const logStore = makeLogStore();
const config = makeConfig({
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "my-workflow", on: ["cpu-usage"] }],
reflexes: [],
workflows: { "my-workflow": { concurrency: 2, overflow: "drop" } },
});
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
// Emit a signal from "cpu-usage" on the bus
const { createSignalBus } = await import("../signal-bus.js");
void createSignalBus; // ensure import resolves
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload: { value: 80 }, ts: Date.now() });
// Simulate a sense worker sending a signal with workflow launch payload
// The kernel's handleWorkerMessage processes "signal" type messages
// and uses routeSenseComputeOutput to detect workflow launches
const workerPool = mockChildren[0];
if (workerPool) {
// Simulate the worker sending a signal message with workflow field
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "my-workflow|10|run this workflow" },
});
}
// The workflow worker should be spawned (one for the sense group, one for workflow)
// The sense group worker is mockChildren[0]; the workflow worker is mockChildren[1]
// We need to check that a start-thread message was sent to the workflow worker
// A workflow worker should be spawned and a start-thread message sent
const workflowWorker = mockChildren.find((c) =>
(c.send as ReturnType<typeof vi.fn>).mock.calls.some(
([msg]: [unknown]) =>
msg !== null &&
typeof msg === "object" &&
(msg as Record<string, unknown>).type === "start-thread",
(args: unknown[]) =>
args[0] !== null &&
typeof args[0] === "object" &&
(args[0] as Record<string, unknown>).type === "start-thread",
),
);
expect(workflowWorker).toBeDefined();
@@ -150,23 +168,30 @@ describe("kernel + workflowManager integration", () => {
await stopPromise;
});
it("passes the signal payload as triggerPayload to the workflow", async () => {
it("passes prompt and maxRounds from the workflow field to the workflow", async () => {
const logStore = makeLogStore();
const config = makeConfig({
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "alert-workflow", on: ["cpu-usage"] }],
reflexes: [],
workflows: { "alert-workflow": { concurrency: 1, overflow: "drop" } },
});
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
const payload = { level: "critical", value: 99 };
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload, ts: Date.now() });
// Simulate sense worker returning a workflow launch
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "alert-workflow|5|handle critical alert" },
});
}
// Find the start-thread call and verify triggerPayload
const startThreadCall = mockChildren
@@ -182,7 +207,9 @@ describe("kernel + workflowManager integration", () => {
expect(startThreadCall?.[0]).toMatchObject({
type: "start-thread",
workflow: "alert-workflow",
triggerPayload: payload,
prompt: "handle critical alert",
maxRounds: 5,
dryRun: false,
});
const stopPromise = kernel.stop();
@@ -197,25 +224,32 @@ describe("kernel + workflowManager integration", () => {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
"disk-io": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "my-workflow", on: ["disk-io"] }],
reflexes: [],
workflows: { "my-workflow": { concurrency: 1, overflow: "drop" } },
});
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
// Emit signal from cpu-usage — NOT in the workflow's "on" list
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload: 50, ts: Date.now() });
// Emit a regular signal (no workflow field) — should NOT trigger any workflow
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: 50,
});
}
// No workflow worker should have been spawned (only the sense group worker)
// No workflow should have been started
const workflowWorkerSpawned = mockChildren.some((c) =>
(c.send as ReturnType<typeof vi.fn>).mock.calls.some(
([msg]: [unknown]) =>
msg !== null &&
typeof msg === "object" &&
(msg as Record<string, unknown>).type === "start-thread",
(args: unknown[]) =>
args[0] !== null &&
typeof args[0] === "object" &&
(args[0] as Record<string, unknown>).type === "start-thread",
),
);
expect(workflowWorkerSpawned).toBe(false);
@@ -227,22 +261,30 @@ describe("kernel + workflowManager integration", () => {
});
describe("workflow events are logged", () => {
it("logs a 'started' event when workflow thread is triggered", async () => {
it("logs a 'started' event when workflow thread is triggered via sense compute", async () => {
const logStore = makeLogStore();
const config = makeConfig({
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "log-test-workflow", on: ["cpu-usage"] }],
reflexes: [],
workflows: { "log-test-workflow": { concurrency: 2, overflow: "drop" } },
});
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload: null, ts: Date.now() });
// Simulate sense compute returning a workflow launch
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "log-test-workflow|10|test prompt" },
});
}
expect(logStore.upsertWorkflowRun).toHaveBeenCalledWith(
expect.objectContaining({ source: "workflow", type: "started" }),
@@ -256,33 +298,42 @@ describe("kernel + workflowManager integration", () => {
});
describe("reloadConfig handles workflow changes", () => {
it("new workflow reflexes are active after reloadConfig", async () => {
it("new workflows are available after reloadConfig", async () => {
const logStore = makeLogStore();
const initialConfig = makeConfig({
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
});
const kernel = createKernel(initialConfig, "/tmp/nerve-test", {
const kernel = createKernel(initialConfig, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
// Reload with a workflow reflex added
// Reload with a workflow added
const newConfig: NerveConfig = {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "new-workflow", on: ["cpu-usage"] }],
reflexes: [],
workflows: { "new-workflow": { concurrency: 1, overflow: "drop" } },
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
// Now emit a signal — should trigger the new workflow
kernel.bus.emit({ id: 2, senseId: "cpu-usage", payload: "reload-test", ts: Date.now() });
// Simulate sense compute returning a workflow launch for the new workflow
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "new-workflow|10|reload test" },
});
}
const startThreadCall = mockChildren
.flatMap((c) => (c.send as ReturnType<typeof vi.fn>).mock.calls as [unknown][])
@@ -301,28 +352,29 @@ describe("kernel + workflowManager integration", () => {
await stopPromise;
});
it("old workflow reflexes are removed after reloadConfig", async () => {
it("old workflows are removed after reloadConfig", async () => {
const logStore = makeLogStore();
const initialConfig = makeConfig({
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "old-workflow", on: ["cpu-usage"] }],
reflexes: [],
workflows: { "old-workflow": { concurrency: 1, overflow: "drop" } },
});
const kernel = createKernel(initialConfig, "/tmp/nerve-test", {
const kernel = createKernel(initialConfig, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
// Reload with the workflow reflex removed
// Reload with the workflow removed
const newConfig: NerveConfig = {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -331,8 +383,15 @@ describe("kernel + workflowManager integration", () => {
(c.send as ReturnType<typeof vi.fn>).mockClear();
}
// Emit a signal — old-workflow should NOT be triggered
kernel.bus.emit({ id: 3, senseId: "cpu-usage", payload: "after-reload", ts: Date.now() });
// Simulate sense compute trying to launch the old workflow — it should still not start
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "old-workflow|10|should not work" },
});
}
const startThreadCall = mockChildren
.flatMap((c) => (c.send as ReturnType<typeof vi.fn>).mock.calls as [unknown][])
@@ -358,17 +417,24 @@ describe("kernel + workflowManager integration", () => {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "shutdown-test", on: ["cpu-usage"] }],
reflexes: [],
workflows: { "shutdown-test": { concurrency: 1, overflow: "drop" } },
});
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
// Trigger a workflow so a worker is spawned
kernel.bus.emit({ id: 1, senseId: "cpu-usage", payload: null, ts: Date.now() });
// Trigger a workflow via sense compute return value
const workerPool = mockChildren[0];
if (workerPool) {
workerPool.emit("message", {
type: "signal",
sense: "cpu-usage",
payload: { workflow: "shutdown-test|10|test" },
});
}
const stopPromise = kernel.stop();
await vi.runAllTimersAsync();
@@ -381,7 +447,7 @@ describe("kernel + workflowManager integration", () => {
workflows: { "my-wf": { concurrency: 1, overflow: "drop" } },
});
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
@@ -400,11 +466,11 @@ describe("kernel + workflowManager integration", () => {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "health-wf", on: ["cpu-usage"] }],
reflexes: [],
workflows: { "health-wf": { concurrency: 2, overflow: "drop" } },
});
const kernel = createKernel(config, "/tmp/nerve-test", {
const kernel = createKernel(config, nerveRoot, {
workerScript: "fake-worker.js",
logStore,
});
+22 -10
View File
@@ -47,7 +47,7 @@ vi.mock("node:child_process", () => ({
// Import after mock is set up
const { createKernel } = await import("../kernel.js");
const { createLogStore } = await import("../log-store.js");
const { createLogStore } = await import("@uncaged/nerve-store");
// ---------------------------------------------------------------------------
// Helpers
@@ -59,7 +59,8 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
...overrides,
};
}
@@ -69,13 +70,17 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
// ---------------------------------------------------------------------------
describe("kernel — message routing", () => {
let nerveRoot: string;
beforeEach(() => {
mockChildren.length = 0;
vi.useFakeTimers({ shouldAdvanceTime: true });
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-kernel-msg-"));
});
afterEach(() => {
vi.useRealTimers();
rmSync(nerveRoot, { recursive: true, force: true });
});
it("routes signal message to bus without throwing", async () => {
@@ -85,7 +90,7 @@ describe("kernel — message routing", () => {
},
reflexes: [],
});
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
expect(mockChildren.length).toBe(1);
const child = mockChildren[0];
@@ -128,7 +133,7 @@ describe("kernel — message routing", () => {
},
reflexes: [],
});
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
const child = mockChildren[0];
child.emit("message", { type: "error", sense: "cpu-usage", error: "compute failed" });
@@ -149,7 +154,7 @@ describe("kernel — message routing", () => {
},
reflexes: [],
});
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
const child = mockChildren[0];
const callsBefore = stderrSpy.mock.calls.length;
@@ -169,7 +174,7 @@ describe("kernel — message routing", () => {
},
reflexes: [],
});
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
const child = mockChildren[0];
expect(() => child.emit("message", { type: "unknown-type" })).not.toThrow();
@@ -182,13 +187,17 @@ describe("kernel — message routing", () => {
});
describe("kernel — groupForSense mapping", () => {
let nerveRoot: string;
beforeEach(() => {
mockChildren.length = 0;
vi.useFakeTimers({ shouldAdvanceTime: true });
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-kernel-groups-"));
});
afterEach(() => {
vi.useRealTimers();
rmSync(nerveRoot, { recursive: true, force: true });
});
it("spawns one worker per unique group", async () => {
@@ -199,9 +208,10 @@ describe("kernel — groupForSense mapping", () => {
"net-usage": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
};
const kernel = createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
// system and network = 2 unique groups
expect(mockChildren.length).toBe(2);
@@ -213,9 +223,9 @@ describe("kernel — groupForSense mapping", () => {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 500, on: null }],
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 500, on: [] }],
});
createKernel(config, "/tmp/nerve-test");
const kernel = createKernel(config, nerveRoot);
const child = mockChildren[0];
vi.advanceTimersByTime(500);
@@ -223,5 +233,7 @@ describe("kernel — groupForSense mapping", () => {
expect(child.send).toHaveBeenCalledWith(
expect.objectContaining({ type: "compute", sense: "cpu-usage" }),
);
await kernel.stop();
});
});
@@ -4,8 +4,8 @@ import { join } from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { NerveConfig, Signal } from "@uncaged/nerve-core";
import { createLogStore } from "../log-store.js";
import type { LogStore } from "../log-store.js";
import { createLogStore } from "@uncaged/nerve-store";
import type { LogStore } from "@uncaged/nerve-store";
import { createReflexScheduler } from "../reflex-scheduler.js";
import { createSignalBus } from "../signal-bus.js";
@@ -29,7 +29,8 @@ describe("LogStore + ReflexScheduler integration", () => {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: null, on: ["cpu-usage"] }],
workflows: null,
workflows: {},
maxRounds: 10,
};
const bus = createSignalBus();
const triggered: string[] = [];
@@ -37,7 +38,7 @@ describe("LogStore + ReflexScheduler integration", () => {
logStore,
});
const signal: Signal = { id: 1, senseId: "cpu-usage", payload: 42, ts: Date.now() };
const signal: Signal = { id: 1, senseId: "cpu-usage", payload: 42, timestamp: Date.now() };
bus.emit(signal);
const logs = logStore.query({ source: "reflex", type: "run_start" });
@@ -55,8 +56,9 @@ describe("LogStore + ReflexScheduler integration", () => {
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 1000, on: null }],
workflows: null,
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 1000, on: [] }],
workflows: {},
maxRounds: 10,
};
const bus = createSignalBus();
const ref: { scheduler: ReturnType<typeof createReflexScheduler> | null } = { scheduler: null };
@@ -86,7 +88,8 @@ describe("LogStore + ReflexScheduler integration", () => {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: null, on: ["cpu-usage"] }],
workflows: null,
workflows: {},
maxRounds: 10,
};
const bus = createSignalBus();
const triggered: string[] = [];
@@ -105,7 +108,7 @@ describe("LogStore + ReflexScheduler integration", () => {
type: "run_complete",
refId: "cpu-usage",
payload: '{"v":99}',
ts: Date.now(),
timestamp: Date.now(),
});
// Writing to the log store should NOT trigger any reflex.
@@ -2,12 +2,14 @@
* Phase 6 integration tests — hot reload, error isolation, grace period, health.
*/
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { Signal } from "@uncaged/nerve-core";
import type { NerveConfig } from "@uncaged/nerve-core";
import { afterEach, describe, expect, it } from "vitest";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { createKernel } from "../kernel.js";
import type { Kernel } from "../kernel.js";
@@ -23,7 +25,8 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
...overrides,
};
}
@@ -54,17 +57,23 @@ async function pollUntil(
describe("phase6 — restartGroup", () => {
let kernel: Kernel | null = null;
let nerveRoot: string;
beforeEach(() => {
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-phase6-restart-"));
});
afterEach(async () => {
if (kernel !== null) {
await kernel.stop();
kernel = null;
}
rmSync(nerveRoot, { recursive: true, force: true });
});
it("restartGroup stops old worker and spawns a new one", async () => {
const config = makeConfig();
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
@@ -96,7 +105,7 @@ describe("phase6 — restartGroup", () => {
it("restartGroup on nonexistent group does nothing", async () => {
const config = makeConfig();
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
await kernel.ready;
@@ -112,17 +121,23 @@ describe("phase6 — restartGroup", () => {
describe("phase6 — reloadConfig", () => {
let kernel: Kernel | null = null;
let nerveRoot: string;
beforeEach(() => {
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-phase6-reload-"));
});
afterEach(async () => {
if (kernel !== null) {
await kernel.stop();
kernel = null;
}
rmSync(nerveRoot, { recursive: true, force: true });
});
it("adds new group when new sense group is introduced", async () => {
const config = makeConfig();
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
await kernel.ready;
@@ -135,7 +150,8 @@ describe("phase6 — reloadConfig", () => {
"net-rx": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -154,9 +170,10 @@ describe("phase6 — reloadConfig", () => {
"net-rx": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
};
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
await kernel.ready;
@@ -168,7 +185,8 @@ describe("phase6 — reloadConfig", () => {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -183,12 +201,18 @@ describe("phase6 — reloadConfig", () => {
describe("phase6 — error isolation", () => {
let kernel: Kernel | null = null;
let nerveRoot: string;
beforeEach(() => {
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-phase6-err-"));
});
afterEach(async () => {
if (kernel !== null) {
await kernel.stop();
kernel = null;
}
rmSync(nerveRoot, { recursive: true, force: true });
});
it("error from one sense does not crash the worker — other senses still work", async () => {
@@ -198,10 +222,11 @@ describe("phase6 — error isolation", () => {
"bad-sense": { group: "mixed", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
};
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
await kernel.ready;
@@ -233,7 +258,7 @@ describe("phase6 — error isolation", () => {
process.stderr.write = stderrSpy as typeof process.stderr.write;
const config = makeConfig();
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: ERROR_WORKER,
});
await kernel.ready;
@@ -256,12 +281,18 @@ describe("phase6 — error isolation", () => {
describe("phase6 — getHealth", () => {
let kernel: Kernel | null = null;
let nerveRoot: string;
beforeEach(() => {
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-phase6-health-"));
});
afterEach(async () => {
if (kernel !== null) {
await kernel.stop();
kernel = null;
}
rmSync(nerveRoot, { recursive: true, force: true });
});
it("returns health snapshot with correct shape", async () => {
@@ -272,7 +303,7 @@ describe("phase6 — getHealth", () => {
"net-rx": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
});
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
await kernel.ready;
@@ -288,7 +319,7 @@ describe("phase6 — getHealth", () => {
it("health reflects config changes after reloadConfig", async () => {
const config = makeConfig();
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
await kernel.ready;
@@ -301,7 +332,8 @@ describe("phase6 — getHealth", () => {
"net-rx": { group: "network", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
};
kernel.reloadConfig(newConfig);
@@ -316,17 +348,23 @@ describe("phase6 — getHealth", () => {
describe("phase6 — auto-respawn on worker crash", () => {
let kernel: Kernel | null = null;
let nerveRoot: string;
beforeEach(() => {
nerveRoot = mkdtempSync(join(tmpdir(), "nerve-phase6-crash-"));
});
afterEach(async () => {
if (kernel !== null) {
await kernel.stop();
kernel = null;
}
rmSync(nerveRoot, { recursive: true, force: true });
});
it("kernel auto-respawns worker and new worker is functional", async () => {
const config = makeConfig();
kernel = createKernel(config, "/tmp/nerve-phase6-test", {
kernel = createKernel(config, nerveRoot, {
workerScript: MOCK_WORKER,
});
await kernel.ready;
@@ -10,13 +10,14 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
...overrides,
};
}
function makeSignal(senseId: string, payload: unknown = 1): Signal {
return { id: 1, senseId, payload, ts: Date.now() };
return { id: 1, senseId, payload, timestamp: Date.now() };
}
describe("ReflexScheduler — throttle + pending deferred trigger", () => {
@@ -16,13 +16,14 @@ function makeConfig(overrides: Partial<NerveConfig> = {}): NerveConfig {
"system-health": { group: "derived", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [],
workflows: null,
workflows: {},
maxRounds: 10,
...overrides,
};
}
function makeSignal(senseId: string, payload: unknown = 1): Signal {
return { id: 1, senseId, payload, ts: Date.now() };
return { id: 1, senseId, payload, timestamp: Date.now() };
}
// ---------------------------------------------------------------------------
@@ -40,7 +41,7 @@ describe("ReflexScheduler — interval reflex", () => {
it("fires triggerFn on schedule", () => {
const triggered: string[] = [];
const config = makeConfig({
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 1000, on: null }],
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 1000, on: [] }],
});
const bus = createSignalBus();
// Use a ref so the triggerFn can call back into the scheduler
@@ -65,7 +66,7 @@ describe("ReflexScheduler — interval reflex", () => {
it("stops firing after stop() is called", () => {
const triggered: string[] = [];
const config = makeConfig({
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 500, on: null }],
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 500, on: [] }],
});
const bus = createSignalBus();
const ref: { scheduler: ReturnType<typeof createReflexScheduler> | null } = {
@@ -88,7 +89,7 @@ describe("ReflexScheduler — interval reflex", () => {
it("starts from current time — does not compensate for past intervals", () => {
const triggered: string[] = [];
const config = makeConfig({
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 1000, on: null }],
reflexes: [{ kind: "sense", sense: "cpu-usage", interval: 1000, on: [] }],
});
const bus = createSignalBus();
const scheduler = createReflexScheduler(config, bus, (name) => triggered.push(name));
@@ -290,10 +291,11 @@ describe("ReflexScheduler — workflow reflexes ignored", () => {
it("does not set up any scheduling for workflow kind reflexes", () => {
const triggered: string[] = [];
const config: NerveConfig = {
maxRounds: 10,
senses: {
"cpu-usage": { group: "system", throttle: null, timeout: null, gracePeriod: null },
},
reflexes: [{ kind: "workflow", workflow: "my-workflow", on: ["cpu-usage"] }],
reflexes: [{ kind: "workflow", workflow: "my-workflow", on: ["cpu-usage"] } as any],
workflows: {
"my-workflow": { concurrency: 1, overflow: "drop" },
},
@@ -2,15 +2,15 @@ import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import Database from "better-sqlite3";
import { drizzle } from "drizzle-orm/better-sqlite3";
import { DatabaseSync } from "node:sqlite";
import { drizzle } from "drizzle-orm/node-sqlite";
import { integer, real, sqliteTable } from "drizzle-orm/sqlite-core";
import { describe, expect, it } from "vitest";
import { createBlobStore } from "../blob-store.js";
import { createBlobStore } from "@uncaged/nerve-store";
import { parseParentMessage } from "../ipc.js";
import { executeCompute, openPeerDb, openSenseDb, runMigrations } from "../sense-runtime.js";
import type { DrizzleDB, PeerMap, SenseRuntime } from "../sense-runtime.js";
import type { ComputeFn, DrizzleDB, PeerMap, SenseRuntime } from "../sense-runtime.js";
// ---------------------------------------------------------------------------
// Helpers
@@ -49,7 +49,7 @@ const samples = sqliteTable("samples", {
describe("runMigrations", () => {
it("creates table via SQL migration file", () => {
const sqlite = new Database(":memory:");
const sqlite = new DatabaseSync(":memory:");
const migrationsDir = makeTempMigrationsDir(INIT_SQL);
const result = runMigrations(sqlite, migrationsDir);
@@ -64,7 +64,7 @@ describe("runMigrations", () => {
});
it("runs multiple migrations in lexicographic order", () => {
const sqlite = new Database(":memory:");
const sqlite = new DatabaseSync(":memory:");
const dir = mkdtempSync(join(tmpdir(), "nerve-multi-"));
writeFileSync(join(dir, "0001_init.sql"), INIT_SQL);
@@ -81,7 +81,7 @@ describe("runMigrations", () => {
});
it("returns ok when migrations directory is empty", () => {
const sqlite = new Database(":memory:");
const sqlite = new DatabaseSync(":memory:");
const dir = makeTempMigrationsDirEmpty();
const result = runMigrations(sqlite, dir);
expect(result.ok).toBe(true);
@@ -89,14 +89,14 @@ describe("runMigrations", () => {
});
it("returns err when migrations directory does not exist", () => {
const sqlite = new Database(":memory:");
const sqlite = new DatabaseSync(":memory:");
const result = runMigrations(sqlite, "/nonexistent/path/migrations");
expect(result.ok).toBe(false);
sqlite.close();
});
it("returns err when a migration SQL is invalid", () => {
const sqlite = new Database(":memory:");
const sqlite = new DatabaseSync(":memory:");
const dir = mkdtempSync(join(tmpdir(), "nerve-bad-sql-"));
writeFileSync(join(dir, "0001_bad.sql"), "NOT VALID SQL !!!;");
const result = runMigrations(sqlite, dir);
@@ -141,7 +141,7 @@ describe("openPeerDb", () => {
it("opens an existing db in read-only mode", () => {
// Create a writable db first
const dbPath = makeTempDbPath();
const sqlite = new Database(dbPath);
const sqlite = new DatabaseSync(dbPath);
sqlite.exec(INIT_SQL);
sqlite.prepare("INSERT INTO samples (ts, value) VALUES (1, 42.0)").run();
sqlite.close();
@@ -168,13 +168,13 @@ describe("openPeerDb", () => {
// ---------------------------------------------------------------------------
describe("executeCompute", () => {
function makeRuntime(computeFn: (db: DrizzleDB, peers: PeerMap) => Promise<unknown | null>): {
function makeRuntime(computeFn: ComputeFn): {
runtime: SenseRuntime;
sqlite: Database.Database;
sqlite: DatabaseSync;
} {
const sqlite = new Database(":memory:");
const sqlite = new DatabaseSync(":memory:");
sqlite.exec(INIT_SQL);
const db = drizzle(sqlite) as DrizzleDB;
const db = drizzle({ client: sqlite }) as DrizzleDB;
return {
runtime: { name: "test-sense", db, compute: computeFn },
sqlite,
@@ -226,10 +226,10 @@ describe("executeCompute", () => {
it("compute can read from peers", async () => {
// Set up a peer db with data
const peerSqlite = new Database(":memory:");
const peerSqlite = new DatabaseSync(":memory:");
peerSqlite.exec(INIT_SQL);
peerSqlite.prepare("INSERT INTO samples (ts, value) VALUES (100, 3.14)").run();
const peerDb = drizzle(peerSqlite) as DrizzleDB;
const peerDb = drizzle({ client: peerSqlite }) as DrizzleDB;
const peers: PeerMap = { "other-sense": peerDb };
@@ -248,9 +248,9 @@ describe("executeCompute", () => {
});
it("write to own db does not affect peer db (isolation)", async () => {
const peerSqlite = new Database(":memory:");
const peerSqlite = new DatabaseSync(":memory:");
peerSqlite.exec(INIT_SQL);
const peerDb = drizzle(peerSqlite) as DrizzleDB;
const peerDb = drizzle({ client: peerSqlite }) as DrizzleDB;
const peers: PeerMap = { "peer-sense": peerDb };
const { runtime, sqlite } = makeRuntime(async (db) => {
@@ -403,7 +403,7 @@ describe("parseParentMessage", () => {
describe("runMigrations journal", () => {
it("does not re-run an already-applied migration", () => {
const sqlite = new Database(":memory:");
const sqlite = new DatabaseSync(":memory:");
const dir = mkdtempSync(join(tmpdir(), "nerve-journal-"));
writeFileSync(join(dir, "0001_init.sql"), INIT_SQL);
@@ -430,7 +430,7 @@ describe("runMigrations journal", () => {
});
it("tracks migrations in _migrations table", () => {
const sqlite = new Database(":memory:");
const sqlite = new DatabaseSync(":memory:");
const dir = mkdtempSync(join(tmpdir(), "nerve-journal2-"));
writeFileSync(join(dir, "0001_init.sql"), INIT_SQL);
@@ -4,7 +4,7 @@ import type { Signal } from "@uncaged/nerve-core";
import { createSignalBus } from "../signal-bus.js";
function makeSignal(senseId: string, payload: unknown = 1): Signal {
return { id: 1, senseId, payload, ts: Date.now() };
return { id: 1, senseId, payload, timestamp: Date.now() };
}
describe("createSignalBus", () => {
@@ -0,0 +1,235 @@
import { EventEmitter } from "node:events";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const mockChildren: MockChild[] = [];
type MockChild = EventEmitter & {
send: ReturnType<typeof vi.fn>;
kill: ReturnType<typeof vi.fn>;
pid: number;
connected: boolean;
};
function makeMockChild(pid = 1): MockChild {
const child = new EventEmitter() as MockChild;
child.connected = true;
child.send = vi.fn((msg: unknown) => {
if (
msg !== null &&
typeof msg === "object" &&
(msg as Record<string, unknown>).type === "shutdown"
) {
child.connected = false;
setImmediate(() => child.emit("exit", 0, null));
}
});
child.kill = vi.fn((_signal?: string) => {
child.connected = false;
child.emit("exit", null, _signal ?? "SIGKILL");
});
child.pid = pid;
return child;
}
vi.mock("node:child_process", () => ({
fork: vi.fn((_script: string, _args: string[], _opts: unknown) => {
const child = makeMockChild(mockChildren.length + 1);
mockChildren.push(child);
return child;
}),
}));
const { createSenseWorkerPool } = await import("../worker-pool.js");
async function flushSetImmediate(): Promise<void> {
await new Promise<void>((resolve) => setImmediate(resolve));
}
async function startWorkerWithReady(
pool: ReturnType<typeof createSenseWorkerPool>,
group: string,
): Promise<void> {
const pr = pool.startWorker(group);
const child = mockChildren[mockChildren.length - 1];
child.emit("message", { type: "ready" });
await pr;
}
describe("createSenseWorkerPool", () => {
beforeEach(() => {
mockChildren.length = 0;
});
afterEach(() => {
vi.useRealTimers();
});
it("forks one child per startWorker and routes IPC to onWorkerMessage", async () => {
const onWorkerMessage = vi.fn();
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage,
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "g1");
expect(mockChildren).toHaveLength(1);
const child = mockChildren[0];
child.emit("message", { type: "signal", sense: "s", payload: 1 });
expect(onWorkerMessage).toHaveBeenCalledWith({ type: "signal", sense: "s", payload: 1 });
});
it("sendCompute delivers to the worker for that group", async () => {
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "sys");
const child = mockChildren[0];
pool.sendCompute("sys", "cpu");
expect(child.send).toHaveBeenCalledWith(
expect.objectContaining({ type: "compute", sense: "cpu" }),
);
});
it("hasWorkerForGroup and getWorkerPid reflect running workers", async () => {
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
expect(pool.hasWorkerForGroup("a")).toBe(false);
expect(pool.getWorkerPid("a")).toBeNull();
await startWorkerWithReady(pool, "a");
expect(pool.hasWorkerForGroup("a")).toBe(true);
expect(pool.getWorkerPid("a")).toBe(1);
expect(pool.activeGroupCount()).toBe(1);
});
it("evictGroup sends shutdown and removes the entry without waiting", async () => {
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "x");
expect(pool.activeGroupCount()).toBe(1);
pool.evictGroup("x");
expect(pool.hasWorkerForGroup("x")).toBe(false);
expect(mockChildren[0].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
});
it("restartGroup invokes onBeforeGroupRestart then respawns", async () => {
const onBeforeGroupRestart = vi.fn();
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => ["s1"],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart,
isStopped: () => false,
});
await startWorkerWithReady(pool, "g");
expect(mockChildren).toHaveLength(1);
const p = pool.restartGroup("g");
expect(onBeforeGroupRestart).toHaveBeenCalledWith("g");
expect(mockChildren[0].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
await flushSetImmediate();
expect(mockChildren).toHaveLength(2);
mockChildren[1].emit("message", { type: "ready" });
await p;
expect(pool.hasWorkerForGroup("g")).toBe(true);
});
it("onWorkerCrashed runs and schedules respawn after non-zero exit", async () => {
vi.useFakeTimers({ shouldAdvanceTime: true });
const onWorkerCrashed = vi.fn();
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: (g) => (g === "g" ? ["a", "b"] : []),
onWorkerCrashed,
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "g");
expect(mockChildren).toHaveLength(1);
mockChildren[0].emit("exit", 1, null);
expect(onWorkerCrashed).toHaveBeenCalledWith("g");
await vi.advanceTimersByTimeAsync(1000);
expect(mockChildren).toHaveLength(2);
});
it("shutdownAll sends shutdown to every worker", async () => {
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => false,
});
await startWorkerWithReady(pool, "a");
await startWorkerWithReady(pool, "b");
await pool.shutdownAll();
expect(mockChildren[0].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
expect(mockChildren[1].send).toHaveBeenCalledWith(
expect.objectContaining({ type: "shutdown" }),
);
});
it("does not respawn after crash when isStopped is true", async () => {
vi.useFakeTimers({ shouldAdvanceTime: true });
const pool = createSenseWorkerPool({
nerveRoot: "/tmp/n",
workerScript: "/fake/sense-worker.js",
onWorkerMessage: vi.fn(),
sensesForGroup: () => [],
onWorkerCrashed: vi.fn(),
onBeforeGroupRestart: vi.fn(),
isStopped: () => true,
});
await startWorkerWithReady(pool, "g");
const n = mockChildren.length;
mockChildren[0].emit("exit", 1, null);
await vi.advanceTimersByTimeAsync(1000);
expect(mockChildren.length).toBe(n);
});
});
@@ -74,13 +74,18 @@ function makeLogStore() {
getActiveWorkflowRuns: vi.fn(() => []),
getTriggerPayload: vi.fn(() => null),
getThreadEvents: vi.fn(() => []),
getThreadMessages: vi.fn(() => []),
getThreadRoundCount: vi.fn(() => 0),
getThreadRounds: vi.fn(() => []),
archiveLogs: vi.fn(() => ({ days: [], vacuumed: false })),
close: vi.fn(),
getAllWorkflowRuns: vi.fn(() => []),
};
}
function makeConfig(overrides: Partial<NerveConfig["workflows"]> = {}): NerveConfig {
return {
maxRounds: 10,
senses: {},
reflexes: [],
workflows: overrides as NerveConfig["workflows"],
@@ -110,7 +115,7 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-workflow", { event: "test" });
mgr.startWorkflow("my-workflow", { prompt: "test", maxRounds: 10, dryRun: false });
expect(mockChildren).toHaveLength(1);
expect(mockChildren[0].send).toHaveBeenCalledWith(
@@ -126,8 +131,8 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-workflow", { n: 1 });
mgr.startWorkflow("my-workflow", { n: 2 });
mgr.startWorkflow("my-workflow", { prompt: "test 1", maxRounds: 10, dryRun: false });
mgr.startWorkflow("my-workflow", { prompt: "test 2", maxRounds: 10, dryRun: false });
// Only one forked child — worker is reused
expect(mockChildren).toHaveLength(1);
@@ -142,7 +147,7 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("my-workflow", { x: 1 });
mgr.startWorkflow("my-workflow", { prompt: "test", maxRounds: 10, dryRun: false });
expect(logStore.upsertWorkflowRun).toHaveBeenCalledWith(
expect.objectContaining({ source: "workflow", type: "started" }),
@@ -159,9 +164,9 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("drop-wf", { first: true });
mgr.startWorkflow("drop-wf", { prompt: "first", maxRounds: 10, dryRun: false });
// now at limit — second call should be dropped
mgr.startWorkflow("drop-wf", { second: true });
mgr.startWorkflow("drop-wf", { prompt: "second", maxRounds: 10, dryRun: false });
expect(mgr.activeCount("drop-wf")).toBe(1);
expect(mgr.queueLength("drop-wf")).toBe(0);
@@ -176,8 +181,8 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("drop-wf", {});
mgr.startWorkflow("drop-wf", {});
mgr.startWorkflow("drop-wf", { prompt: "test", maxRounds: 10, dryRun: false });
mgr.startWorkflow("drop-wf", { prompt: "test", maxRounds: 10, dryRun: false });
const droppedCall = logStore.upsertWorkflowRun.mock.calls.find(
([entry]) => entry.type === "dropped",
@@ -194,8 +199,8 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("queue-wf", { first: true });
mgr.startWorkflow("queue-wf", { second: true });
mgr.startWorkflow("queue-wf", { prompt: "first", maxRounds: 10, dryRun: false });
mgr.startWorkflow("queue-wf", { prompt: "second", maxRounds: 10, dryRun: false });
expect(mgr.activeCount("queue-wf")).toBe(1);
expect(mgr.queueLength("queue-wf")).toBe(1);
@@ -208,8 +213,8 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("queue-wf", {});
mgr.startWorkflow("queue-wf", {});
mgr.startWorkflow("queue-wf", { prompt: "test", maxRounds: 10, dryRun: false });
mgr.startWorkflow("queue-wf", { prompt: "test", maxRounds: 10, dryRun: false });
const queuedCall = logStore.upsertWorkflowRun.mock.calls.find(
([entry]) => entry.type === "queued",
@@ -228,12 +233,12 @@ describe("WorkflowManager", () => {
const mgr = createWorkflowManager("/nerve-root", config, logStore);
// Fill the concurrency slot
mgr.startWorkflow("queue-wf", { n: 0 });
mgr.startWorkflow("queue-wf", { prompt: "test 0", maxRounds: 10, dryRun: false });
// Fill the queue to maxQueue
mgr.startWorkflow("queue-wf", { n: 1 });
mgr.startWorkflow("queue-wf", { n: 2 });
// This one should push out { n: 1 }
mgr.startWorkflow("queue-wf", { n: 3 });
mgr.startWorkflow("queue-wf", { prompt: "test 1", maxRounds: 10, dryRun: false });
mgr.startWorkflow("queue-wf", { prompt: "test 2", maxRounds: 10, dryRun: false });
// This one should push out the oldest
mgr.startWorkflow("queue-wf", { prompt: "test 3", maxRounds: 10, dryRun: false });
// Queue should still be at maxQueue (2)
expect(mgr.queueLength("queue-wf")).toBe(2);
@@ -254,8 +259,8 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("queue-wf", { first: true });
mgr.startWorkflow("queue-wf", { second: true });
mgr.startWorkflow("queue-wf", { prompt: "first", maxRounds: 10, dryRun: false });
mgr.startWorkflow("queue-wf", { prompt: "second", maxRounds: 10, dryRun: false });
expect(mgr.activeCount("queue-wf")).toBe(1);
expect(mgr.queueLength("queue-wf")).toBe(1);
@@ -289,8 +294,8 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("queue-wf", { first: true });
mgr.startWorkflow("queue-wf", { second: true });
mgr.startWorkflow("queue-wf", { prompt: "first", maxRounds: 10, dryRun: false });
mgr.startWorkflow("queue-wf", { prompt: "second", maxRounds: 10, dryRun: false });
const child = mockChildren[0];
const firstRunId = (child.send as ReturnType<typeof vi.fn>).mock.calls[0][0].runId as string;
@@ -316,8 +321,8 @@ describe("WorkflowManager", () => {
});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("wf-a", {});
mgr.startWorkflow("wf-b", {});
mgr.startWorkflow("wf-a", { prompt: "test", maxRounds: 10, dryRun: false });
mgr.startWorkflow("wf-b", { prompt: "test", maxRounds: 10, dryRun: false });
// Two distinct workers should have been forked
expect(mockChildren).toHaveLength(2);
@@ -343,7 +348,7 @@ describe("WorkflowManager", () => {
await vi.runAllTimersAsync();
await stopPromise;
mgr.startWorkflow("wf-a", {});
mgr.startWorkflow("wf-a", { prompt: "test", maxRounds: 10, dryRun: false });
// No worker should have been spawned
expect(mockChildren).toHaveLength(0);
@@ -356,7 +361,7 @@ describe("WorkflowManager", () => {
const config = makeConfig({});
const mgr = createWorkflowManager("/nerve-root", config, logStore);
mgr.startWorkflow("no-such-workflow", {});
mgr.startWorkflow("no-such-workflow", { prompt: "test", maxRounds: 10, dryRun: false });
expect(mockChildren).toHaveLength(0);
expect(logStore.upsertWorkflowRun).not.toHaveBeenCalled();
+18 -61
View File
@@ -2,74 +2,24 @@
* Daemon IPC server listens on a Unix domain socket so that the CLI
* can send commands (e.g. trigger-workflow, trigger-sense) to the running daemon process.
*
* Protocol: newline-delimited JSON messages.
* Each request: { type: "trigger-workflow"; workflow: string; payload: unknown }
* | { type: "trigger-sense"; sense: string }
* | { type: "list-senses" }
* Each response: { ok: true } | { ok: false; error: string }
* | { ok: true; senses: SenseInfo[] } (for list-senses)
* Protocol: newline-delimited JSON request/response types and
* `parseDaemonIpcRequest` live in `@uncaged/nerve-core`.
*/
import { rmSync } from "node:fs";
import { type Server, type Socket, createServer } from "node:net";
import type { SenseInfo } from "@uncaged/nerve-core";
import type { DaemonIpcResponse, SenseInfo } from "@uncaged/nerve-core";
import { parseDaemonIpcRequest } from "@uncaged/nerve-core";
import type { WorkflowManager } from "./workflow-manager.js";
export type { SenseInfo };
/** JSON message sent by the CLI to trigger a workflow. */
export type TriggerWorkflowRequest = {
type: "trigger-workflow";
workflow: string;
payload: unknown;
};
/** JSON message sent by the CLI to trigger a sense compute on-demand. */
export type TriggerSenseRequest = {
type: "trigger-sense";
sense: string;
};
/** JSON message sent by the CLI to list registered senses. */
export type ListSensesRequest = {
type: "list-senses";
};
type DaemonRequest = TriggerWorkflowRequest | TriggerSenseRequest | ListSensesRequest;
type DaemonResponse =
| { ok: true }
| { ok: false; error: string }
| { ok: true; senses: SenseInfo[] };
export type DaemonIpcServer = {
close: () => Promise<void>;
};
function parseRequest(line: string): DaemonRequest | null {
try {
const obj = JSON.parse(line) as unknown;
if (obj === null || typeof obj !== "object") return null;
const req = obj as Record<string, unknown>;
if (req.type === "trigger-workflow") {
if (typeof req.workflow !== "string" || req.workflow.length === 0) return null;
return { type: "trigger-workflow", workflow: req.workflow, payload: req.payload ?? {} };
}
if (req.type === "trigger-sense") {
if (typeof req.sense !== "string" || req.sense.length === 0) return null;
return { type: "trigger-sense", sense: req.sense };
}
if (req.type === "list-senses") {
return { type: "list-senses" };
}
return null;
} catch {
return null;
}
}
export type DaemonIpcServerOptions = {
/** Called when a trigger-sense request arrives. Should throw if the sense is unknown. */
triggerSense: (senseName: string) => void;
@@ -93,30 +43,37 @@ export function createDaemonIpcServer(
const trimmed = line.trim();
if (trimmed.length === 0) return;
const req = parseRequest(trimmed);
const req = parseDaemonIpcRequest(trimmed);
if (req === null) {
const resp: DaemonResponse = { ok: false, error: "Invalid request" };
const resp: DaemonIpcResponse = { ok: false, error: "Invalid request" };
socket.write(`${JSON.stringify(resp)}\n`);
return;
}
try {
if (req.type === "trigger-workflow") {
workflowManager.startWorkflow(req.workflow, req.payload);
const resp: DaemonResponse = { ok: true };
workflowManager.startWorkflow(req.workflow, {
prompt: req.prompt,
maxRounds: req.maxRounds,
dryRun: req.dryRun,
});
const resp: DaemonIpcResponse = { ok: true };
socket.write(`${JSON.stringify(resp)}\n`);
} else if (req.type === "trigger-sense") {
opts.triggerSense(req.sense);
const resp: DaemonResponse = { ok: true };
const resp: DaemonIpcResponse = { ok: true };
socket.write(`${JSON.stringify(resp)}\n`);
} else if (req.type === "list-senses") {
const senses = opts.listSenses();
const resp: DaemonResponse = { ok: true, senses };
const resp: DaemonIpcResponse = { ok: true, senses };
socket.write(`${JSON.stringify(resp)}\n`);
} else {
const _exhaustive: never = req;
void _exhaustive;
}
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
const resp: DaemonResponse = { ok: false, error: msg };
const resp: DaemonIpcResponse = { ok: false, error: msg };
socket.write(`${JSON.stringify(resp)}\n`);
}
}
+17 -11
View File
@@ -12,6 +12,7 @@ export type {
ResumeThreadMessage,
ThreadEventMessage,
WorkflowErrorMessage,
ThreadWorkflowMessageMessage,
} from "./ipc.js";
export type { SignalBus, SignalHandler, Unsubscribe } from "./signal-bus.js";
@@ -29,25 +30,30 @@ export {
export { createKernel } from "./kernel.js";
export type { Kernel, KernelOptions, KernelHealth } from "./kernel.js";
export type { SenseInfo } from "./daemon-ipc.js";
export type { SenseInfo } from "@uncaged/nerve-core";
export { createFileWatcher } from "./file-watcher.js";
export type { FileWatcher, FileChange, FileChangeHandler } from "./file-watcher.js";
export { createBlobStore, normalizeBlobHash } from "./blob-store.js";
export type { BlobStore } from "./blob-store.js";
export { createLogStore, LOG_ARCHIVE_META_KEY } from "./log-store.js";
export {
createBlobStore,
createLogStore,
LOG_ARCHIVE_META_KEY,
normalizeBlobHash,
} from "@uncaged/nerve-store";
export type {
LogStore,
LogEntry,
LogQuery,
WorkflowRun,
WorkflowRunStatus,
ArchiveLogsDayResult,
ArchiveLogsOptions,
ArchiveLogsResult,
} from "./log-store.js";
BlobStore,
GetThreadRoundsParams,
LogEntry,
LogQuery,
LogStore,
ThreadRoundRow,
WorkflowRun,
WorkflowRunStatus,
} from "@uncaged/nerve-store";
export { createWorkflowManager } from "./workflow-manager.js";
export type { WorkflowManager } from "./workflow-manager.js";
+138 -64
View File
@@ -4,7 +4,7 @@
*/
import type { Result } from "@uncaged/nerve-core";
import { err, ok } from "@uncaged/nerve-core";
import { err, isPlainRecord, ok } from "@uncaged/nerve-core";
/** Parent → Worker: trigger one compute cycle for a sense */
export type ComputeMessage = {
@@ -31,18 +31,23 @@ export type StartThreadMessage = {
type: "start-thread";
runId: string;
workflow: string;
/** The trigger payload from the Reflex that initiated this thread. */
triggerPayload: unknown;
prompt: string;
/** Safety-valve: max moderator rounds for this thread (engine launch parameter). */
maxRounds: number;
/** When true, roles may skip side effects (thread-level hint on the start frame). */
dryRun: boolean;
};
/** Parent → Workflow Worker: resume an existing thread after crash recovery */
export type ResumeThreadMessage = {
type: "resume-thread";
runId: string;
/** Serialised CommandEvent history to rebuild ThreadState. */
events: Array<{ type: string; [key: string]: unknown }>;
/** Serialised trigger payload (the same value as in the original start-thread). */
triggerPayload: unknown;
/** Serialised WorkflowMessage history to rebuild chain (must begin with `__start__`). */
messages: Array<{ role: string; content: string; meta: unknown; timestamp: number }>;
/** Safety-valve: max moderator rounds for this thread. */
maxRounds: number;
/** Thread-level dry-run hint (aligns with persisted `__start__` meta when replaying). */
dryRun: boolean;
};
/** Union of all messages the parent sends to a worker */
@@ -103,12 +108,12 @@ export type WorkflowErrorMessage = {
error: string;
};
/** Workflow Worker → Parent: a thread CommandEvent produced by a role (for crash recovery). */
export type ThreadCommandEventMessage = {
type: "thread-command-event";
/** Workflow Worker → Parent: a WorkflowMessage produced by a role (for crash recovery). */
export type ThreadWorkflowMessageMessage = {
type: "thread-workflow-message";
runId: string;
/** The CommandEvent returned by role.execute() — will be persisted for crash recovery. */
event: { type: string; [key: string]: unknown };
/** The WorkflowMessage produced by the role persisted for crash recovery. */
message: { role: string; content: string; meta: unknown; timestamp: number };
};
/** Union of all messages a worker sends to the parent */
@@ -119,7 +124,7 @@ export type WorkerToParentMessage =
| HealthResponseMessage
| ThreadEventMessage
| WorkflowErrorMessage
| ThreadCommandEventMessage;
| ThreadWorkflowMessageMessage;
const PARENT_MSG_TYPES = new Set([
"compute",
@@ -132,89 +137,134 @@ const PARENT_MSG_TYPES = new Set([
function validateStartThreadMsg(obj: Record<string, unknown>): string | null {
if (typeof obj.runId !== "string") return "'start-thread' message missing string 'runId'";
if (typeof obj.workflow !== "string") return "'start-thread' message missing string 'workflow'";
if (!("triggerPayload" in obj)) return "'start-thread' message missing 'triggerPayload'";
if (typeof obj.prompt !== "string") return "'start-thread' message missing string 'prompt'";
if (typeof obj.maxRounds !== "number") return "'start-thread' message missing number 'maxRounds'";
if (typeof obj.dryRun !== "boolean") return "'start-thread' message missing boolean 'dryRun'";
return null;
}
function validateResumeThreadMsg(obj: Record<string, unknown>): string | null {
if (typeof obj.runId !== "string") return "'resume-thread' message missing string 'runId'";
if (!Array.isArray(obj.events)) return "'resume-thread' message missing 'events' array";
if (!("triggerPayload" in obj)) return "'resume-thread' message missing 'triggerPayload'";
if (!Array.isArray(obj.messages)) return "'resume-thread' message missing 'messages' array";
if (typeof obj.maxRounds !== "number")
return "'resume-thread' message missing number 'maxRounds'";
if (typeof obj.dryRun !== "boolean") return "'resume-thread' message missing boolean 'dryRun'";
return null;
}
/** Validate and parse an unknown IPC message received from the parent process. */
export function parseParentMessage(raw: unknown): Result<ParentToWorkerMessage> {
if (raw === null || typeof raw !== "object") {
if (!isPlainRecord(raw)) {
return err(new Error("IPC message is not an object"));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
if (typeof obj.type !== "string") {
return err(new Error("IPC message missing string 'type' field"));
}
if (!PARENT_MSG_TYPES.has(obj.type)) {
return err(new Error(`Unknown IPC message type: "${obj.type}"`));
}
if (obj.type === "compute") {
if (typeof obj.sense !== "string") {
return err(new Error("IPC 'compute' message missing string 'sense' field"));
}
return ok({ type: "compute", sense: obj.sense });
}
if (obj.type === "shutdown") {
return ok({ type: "shutdown" });
}
if (obj.type === "health-request") {
return ok({ type: "health-request" });
}
if (obj.type === "start-thread") {
const errMsg = validateStartThreadMsg(obj);
if (errMsg !== null) return err(new Error(errMsg));
// Field types are validated above; `Record<string, unknown>` values stay `unknown` to TypeScript.
return ok({
type: "start-thread",
runId: obj.runId,
workflow: obj.workflow,
prompt: obj.prompt,
maxRounds: obj.maxRounds,
dryRun: obj.dryRun,
} as StartThreadMessage);
}
if (obj.type === "resume-thread") {
const errMsg = validateResumeThreadMsg(obj);
if (errMsg !== null) return err(new Error(errMsg));
// Elements are validated as plain objects by the kernel; trust the wire shape here.
return ok({
type: "resume-thread",
runId: obj.runId,
messages: obj.messages as ResumeThreadMessage["messages"],
maxRounds: obj.maxRounds,
dryRun: obj.dryRun,
} as ResumeThreadMessage);
}
return ok(raw as ParentToWorkerMessage);
return err(new Error(`Unhandled IPC message type: "${obj.type}"`));
}
function parseSignalMsg(obj: Record<string, unknown>, raw: unknown): Result<WorkerToParentMessage> {
function parseSignalMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (typeof obj.sense !== "string") {
return err(new Error("Worker 'signal' message missing string 'sense' field"));
}
if (!("payload" in obj)) {
return err(new Error("Worker 'signal' message missing 'payload' field"));
}
return ok(raw as SignalMessage);
return ok({
type: "signal",
sense: obj.sense,
payload: obj.payload,
});
}
function parseErrorMsg(obj: Record<string, unknown>, raw: unknown): Result<WorkerToParentMessage> {
function parseErrorMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (typeof obj.sense !== "string") {
return err(new Error("Worker 'error' message missing string 'sense' field"));
}
if (typeof obj.error !== "string") {
return err(new Error("Worker 'error' message missing string 'error' field"));
}
return ok(raw as ErrorMessage);
return ok({
type: "error",
sense: obj.sense,
error: obj.error,
});
}
function parseHealthResponseMsg(
obj: Record<string, unknown>,
raw: unknown,
): Result<WorkerToParentMessage> {
function parseHealthResponseMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (!Array.isArray(obj.senses)) {
return err(new Error("Worker 'health-response' message missing 'senses' array"));
}
if (typeof obj.inFlightCount !== "number") {
return err(new Error("Worker 'health-response' message missing 'inFlightCount' number"));
}
return ok(raw as HealthResponseMessage);
return ok({
type: "health-response",
// Kernel only sends string[] today; keep accepting any array elements without filtering.
senses: obj.senses as string[],
inFlightCount: obj.inFlightCount,
});
}
const THREAD_EVENT_TYPES = new Set<string>([
"queued",
"started",
"step_complete",
"completed",
"failed",
]);
function isThreadEventType(value: string): value is ThreadEventType {
switch (value) {
case "queued":
case "started":
case "step_complete":
case "completed":
case "failed":
return true;
default:
return false;
}
}
function parseThreadEventMsg(
obj: Record<string, unknown>,
raw: unknown,
): Result<WorkerToParentMessage> {
function parseThreadEventMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (typeof obj.runId !== "string") {
return err(new Error("Worker 'thread-event' message missing string 'runId' field"));
}
if (typeof obj.eventType !== "string" || !THREAD_EVENT_TYPES.has(obj.eventType)) {
if (typeof obj.eventType !== "string" || !isThreadEventType(obj.eventType)) {
return err(
new Error(`Worker 'thread-event' message has invalid 'eventType': "${obj.eventType}"`),
);
@@ -222,20 +272,26 @@ function parseThreadEventMsg(
if (!("payload" in obj)) {
return err(new Error("Worker 'thread-event' message missing 'payload' field"));
}
return ok(raw as ThreadEventMessage);
return ok({
type: "thread-event",
runId: obj.runId,
eventType: obj.eventType,
payload: obj.payload,
});
}
function parseWorkflowErrorMsg(
obj: Record<string, unknown>,
raw: unknown,
): Result<WorkerToParentMessage> {
function parseWorkflowErrorMsg(obj: Record<string, unknown>): Result<WorkerToParentMessage> {
if (typeof obj.runId !== "string") {
return err(new Error("Worker 'workflow-error' message missing string 'runId' field"));
}
if (typeof obj.error !== "string") {
return err(new Error("Worker 'workflow-error' message missing string 'error' field"));
}
return ok(raw as WorkflowErrorMessage);
return ok({
type: "workflow-error",
runId: obj.runId,
error: obj.error,
});
}
const WORKER_MSG_TYPES = new Set([
@@ -245,43 +301,61 @@ const WORKER_MSG_TYPES = new Set([
"health-response",
"thread-event",
"workflow-error",
"thread-command-event",
"thread-workflow-message",
]);
function parseThreadCommandEventMsg(
function parseThreadWorkflowMessageMsg(
obj: Record<string, unknown>,
raw: unknown,
): Result<WorkerToParentMessage> {
if (typeof obj.runId !== "string") {
return err(new Error("Worker 'thread-command-event' message missing string 'runId' field"));
return err(new Error("Worker 'thread-workflow-message' missing string 'runId' field"));
}
if (obj.event === null || typeof obj.event !== "object") {
return err(new Error("Worker 'thread-command-event' message missing object 'event' field"));
if (!isPlainRecord(obj.message)) {
return err(new Error("Worker 'thread-workflow-message' missing object 'message' field"));
}
const event = obj.event as Record<string, unknown>;
if (typeof event.type !== "string") {
return err(new Error("Worker 'thread-command-event' event missing string 'type' field"));
const msg = obj.message;
if (typeof msg.role !== "string") {
return err(new Error("Worker 'thread-workflow-message' message missing string 'role' field"));
}
return ok(raw as ThreadCommandEventMessage);
if (typeof msg.content !== "string") {
return err(
new Error("Worker 'thread-workflow-message' message missing string 'content' field"),
);
}
if (typeof msg.timestamp !== "number") {
return err(
new Error("Worker 'thread-workflow-message' message missing number 'timestamp' field"),
);
}
return ok({
type: "thread-workflow-message",
runId: obj.runId,
message: {
role: msg.role,
content: msg.content,
meta: "meta" in msg ? msg.meta : undefined,
timestamp: msg.timestamp,
},
});
}
/** Validate and parse an unknown IPC message received from a worker process. */
export function parseWorkerMessage(raw: unknown): Result<WorkerToParentMessage> {
if (raw === null || typeof raw !== "object") {
if (!isPlainRecord(raw)) {
return err(new Error("Worker IPC message is not an object"));
}
const obj = raw as Record<string, unknown>;
const obj = raw;
if (typeof obj.type !== "string") {
return err(new Error("Worker IPC message missing string 'type' field"));
}
if (!WORKER_MSG_TYPES.has(obj.type)) {
return err(new Error(`Unknown worker IPC message type: "${obj.type}"`));
}
if (obj.type === "signal") return parseSignalMsg(obj, raw);
if (obj.type === "error") return parseErrorMsg(obj, raw);
if (obj.type === "health-response") return parseHealthResponseMsg(obj, raw);
if (obj.type === "thread-event") return parseThreadEventMsg(obj, raw);
if (obj.type === "workflow-error") return parseWorkflowErrorMsg(obj, raw);
if (obj.type === "thread-command-event") return parseThreadCommandEventMsg(obj, raw);
if (obj.type === "signal") return parseSignalMsg(obj);
if (obj.type === "error") return parseErrorMsg(obj);
if (obj.type === "health-response") return parseHealthResponseMsg(obj);
if (obj.type === "thread-event") return parseThreadEventMsg(obj);
if (obj.type === "workflow-error") return parseWorkflowErrorMsg(obj);
if (obj.type === "thread-workflow-message") return parseThreadWorkflowMessageMsg(obj);
return ok({ type: "ready" });
}
+92
View File
@@ -0,0 +1,92 @@
/**
* File-watcher callbacks for nerve.yaml / sense / workflow sources (hot reload wiring).
*/
import { readFileSync } from "node:fs";
import { join } from "node:path";
import type { NerveConfig } from "@uncaged/nerve-core";
import { parseNerveConfig } from "@uncaged/nerve-core";
import type { LogStore } from "@uncaged/nerve-store";
import type { WorkflowManager } from "./workflow-manager.js";
export type KernelFileWatchDeps = {
nerveRoot: string;
getConfig: () => NerveConfig;
logStore: LogStore;
workflowManager: WorkflowManager;
restartGroup: (group: string) => Promise<void>;
reloadConfig: (newConfig: NerveConfig) => void;
};
export type KernelFileWatchHandlers = {
onSenseFileChange: (senseName: string) => void;
onWorkflowFileChange: (workflowName: string) => void;
onConfigFileChange: () => void;
};
export function createKernelFileWatchHandlers(deps: KernelFileWatchDeps): KernelFileWatchHandlers {
function onSenseFileChange(senseName: string): void {
const sc = deps.getConfig().senses[senseName];
if (sc === undefined) return;
process.stderr.write(
`[kernel] sense file changed: "${senseName}", restarting group "${sc.group}"\n`,
);
deps.logStore.append({
source: "system",
type: "sense_reload",
refId: senseName,
payload: null,
timestamp: Date.now(),
});
deps.restartGroup(sc.group).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] restartGroup error: ${msg}\n`);
});
}
function onWorkflowFileChange(workflowName: string): void {
process.stderr.write(
`[kernel] workflow file changed: "${workflowName}", draining and respawning worker\n`,
);
deps.logStore.append({
source: "system",
type: "workflow_reload",
refId: workflowName,
payload: null,
timestamp: Date.now(),
});
deps.workflowManager.drainAndRespawn(workflowName).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] drainAndRespawn error for "${workflowName}": ${msg}\n`);
});
}
function onConfigFileChange(): void {
process.stderr.write("[kernel] nerve.yaml changed, reloading config\n");
deps.logStore.append({
source: "system",
type: "config_reload",
refId: null,
payload: null,
timestamp: Date.now(),
});
try {
const raw = readFileSync(join(deps.nerveRoot, "nerve.yaml"), "utf8");
const parseResult = parseNerveConfig(raw);
if (!parseResult.ok) {
process.stderr.write(
`[kernel] config parse error, keeping current config: ${parseResult.error.message}\n`,
);
return;
}
deps.reloadConfig(parseResult.value);
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] failed to read nerve.yaml, keeping current config: ${msg}\n`);
}
}
return { onSenseFileChange, onWorkflowFileChange, onConfigFileChange };
}
@@ -0,0 +1,29 @@
import type { NerveConfig } from "@uncaged/nerve-core";
export function groupForSense(config: NerveConfig, senseName: string): string | null {
const senseConfig = config.senses[senseName];
if (senseConfig === undefined) return null;
return senseConfig.group;
}
export function senseNamesInGroup(config: NerveConfig, group: string): string[] {
return Object.entries(config.senses)
.filter(([, sc]) => sc.group === group)
.map(([name]) => name);
}
export function collectSenseGroups(cfg: NerveConfig): Set<string> {
const result = new Set<string>();
for (const sc of Object.values(cfg.senses)) {
result.add(sc.group);
}
return result;
}
export function senseNamesInGroupAsSet(cfg: NerveConfig, group: string): Set<string> {
const result = new Set<string>();
for (const [name, sc] of Object.entries(cfg.senses)) {
if (sc.group === group) result.add(name);
}
return result;
}
+91 -320
View File
@@ -1,43 +1,32 @@
/**
* Kernel the main orchestrator that ties sense workers, signal bus, and
* reflex scheduler together.
*
* Responsibilities:
* - Spawn one child process per sense group (via fork)
* - Route SignalMessage from workers SignalBus
* - Route ErrorMessage from workers stderr log
* - Drive compute triggers via ReflexScheduler
* - Graceful shutdown: stop scheduler, send shutdown to all workers
* - Hot reload: restartGroup, reloadConfig, file watcher integration
* - Health reporting: getHealth
* Kernel ties sense workers, signal bus, reflex scheduler, workflow manager,
* optional file watcher, and daemon IPC.
*/
import { fork } from "node:child_process";
import type { ChildProcess } from "node:child_process";
import { readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { join } from "node:path";
import type { NerveConfig, SenseInfo, Signal } from "@uncaged/nerve-core";
import { parseNerveConfig } from "@uncaged/nerve-core";
import { routeSenseComputeOutput } from "@uncaged/nerve-core";
import { createLogStore } from "@uncaged/nerve-store";
import type { LogStore } from "@uncaged/nerve-store";
import { createDaemonIpcServer } from "./daemon-ipc.js";
import type { DaemonIpcServer } from "./daemon-ipc.js";
import { createFileWatcher } from "./file-watcher.js";
import type { FileWatcher } from "./file-watcher.js";
import type { ComputeMessage, ShutdownMessage } from "./ipc.js";
import { parseWorkerMessage } from "./ipc.js";
import { createLogStore } from "./log-store.js";
import type { LogStore } from "./log-store.js";
import { createKernelFileWatchHandlers } from "./kernel-file-watch.js";
import {
collectSenseGroups,
groupForSense,
senseNamesInGroup,
senseNamesInGroupAsSet,
} from "./kernel-sense-groups.js";
import { createReflexScheduler } from "./reflex-scheduler.js";
import type { ReflexScheduler } from "./reflex-scheduler.js";
import { createSignalBus } from "./signal-bus.js";
import type { SignalBus } from "./signal-bus.js";
import {
formatCapturedStderrTail,
formatChildExitSummary,
teeCapturedStderr,
} from "./worker-fork-support.js";
import { createSenseWorkerPool, resolveWorkerScript } from "./worker-pool.js";
import { createWorkflowManager } from "./workflow-manager.js";
import type { WorkflowManager } from "./workflow-manager.js";
@@ -57,93 +46,19 @@ export type Kernel = {
bus: SignalBus;
logStore: LogStore;
workflowManager: WorkflowManager;
/** Resolves when all workers have sent their initial "ready" message. */
ready: Promise<void>;
/** Returns the PID of the worker process for a given group, or null if not found. */
getWorkerPid: (group: string) => number | null;
/** Sends a compute message to the worker responsible for the given sense. */
triggerCompute: (senseName: string) => void;
/**
* On-demand sense trigger looks up the group for `senseName`, finds its worker,
* and sends a compute message. Throws if the sense is unknown.
*/
triggerSense: (senseName: string) => void;
/** Gracefully restart a group worker (wait for exit, then respawn). */
restartGroup: (group: string) => Promise<void>;
/** Reload config from a new NerveConfig, incrementally updating scheduler and workers.
* Note: any pending/throttled computes in the old scheduler are silently dropped on reload.
* In-flight state is not preserved across reloadConfig. */
reloadConfig: (newConfig: NerveConfig) => void;
/** Return daemon health info. */
getHealth: () => KernelHealth;
};
type WorkerEntry = {
group: string;
process: ChildProcess;
};
function resolveWorkerScript(): string {
const __filename = fileURLToPath(import.meta.url);
const __dir = dirname(__filename);
return join(__dir, "sense-worker.js");
}
function spawnWorker(
nerveRoot: string,
group: string,
workerScript: string,
stderrTail: { value: string },
): ChildProcess {
const child = fork(workerScript, ["--group", group, "--root", nerveRoot], {
stdio: ["ignore", "inherit", "pipe", "ipc"],
});
teeCapturedStderr(child, stderrTail);
// Prevent unhandled EPIPE when writing to a child whose IPC channel closed
child.on("error", (err) => {
if ((err as NodeJS.ErrnoException).code !== "EPIPE") {
console.error("[worker] error:", err.message);
}
});
return child;
}
function sendCompute(worker: ChildProcess, senseName: string): void {
// worker.connected is false when the IPC channel has been closed (e.g. worker crashed)
if (worker.connected === false) return;
const msg: ComputeMessage = { type: "compute", sense: senseName };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function sendShutdown(worker: ChildProcess): void {
if (worker.connected === false) return;
const msg: ShutdownMessage = { type: "shutdown" };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function groupForSense(config: NerveConfig, senseName: string): string | null {
const senseConfig = config.senses[senseName];
if (senseConfig === undefined) return null;
return senseConfig.group;
}
export type KernelOptions = {
workerScript?: string | null;
enableFileWatcher?: boolean;
/** Override the LogStore instance (useful for testing). */
logStore?: LogStore;
/**
* Unix socket path for the daemon IPC server (used by CLI to send trigger-workflow).
* When null, the IPC server is not started (e.g. during tests).
*/
ipcSocketPath?: string | null;
};
@@ -166,7 +81,7 @@ export function createKernel(
type: "start",
refId: null,
payload: null,
ts: startTime,
timestamp: startTime,
});
let config = initialConfig;
@@ -184,9 +99,9 @@ export function createKernel(
groups.add(senseConfig.group);
}
const workers = new Map<string, WorkerEntry>();
let stopped = false;
let scheduler: ReflexScheduler = null as unknown as ReflexScheduler;
/** Assigned before workers start; `handleWorkerMessage` only runs after this is set. */
let scheduler!: ReflexScheduler;
let readyResolve: (() => void) | undefined;
const ready = new Promise<void>((resolve) => {
@@ -194,10 +109,10 @@ export function createKernel(
});
let pendingReadyCount = groups.size > 0 ? groups.size : 0;
function sensesForGroup(group: string): string[] {
return Object.entries(config.senses)
.filter(([, sc]) => sc.group === group)
.map(([name]) => name);
function clearSchedulerForGroup(group: string): void {
for (const senseName of senseNamesInGroup(config, group)) {
scheduler.onComputeComplete(senseName);
}
}
function handleWorkerMessage(raw: unknown): void {
@@ -223,73 +138,53 @@ export function createKernel(
type: "error",
refId: msg.sense,
payload: JSON.stringify({ error: msg.error }),
ts: Date.now(),
timestamp: Date.now(),
});
scheduler.onComputeComplete(msg.sense);
return;
}
if (msg.type === "signal") {
const signal: Signal = {
id: nextSignalId(),
senseId: msg.sense,
payload: msg.payload,
ts: Date.now(),
};
logStore.append({
source: "sense",
type: "signal",
refId: msg.sense,
payload: JSON.stringify(msg.payload),
ts: signal.ts,
});
bus.emit(signal);
const route = routeSenseComputeOutput(msg.payload);
if (route.kind === "launch") {
const { workflowName, maxRounds, prompt } = route.launch;
workflowManager.startWorkflow(workflowName, { prompt, maxRounds, dryRun: false });
logStore.append({
source: "sense",
type: "workflow-launch",
refId: msg.sense,
payload: JSON.stringify(route.launch),
timestamp: Date.now(),
});
} else {
const signal: Signal = {
id: nextSignalId(),
senseId: msg.sense,
payload: route.payload,
timestamp: Date.now(),
};
logStore.append({
source: "sense",
type: "signal",
refId: msg.sense,
payload: JSON.stringify(route.payload),
timestamp: signal.timestamp,
});
bus.emit(signal);
}
scheduler.onComputeComplete(msg.sense);
}
// health-response is handled externally by the caller; no action needed here
}
function startWorker(group: string): Promise<void> {
const stderrTail = { value: "" };
const child = spawnWorker(nerveRoot, group, workerScript, stderrTail);
let workerReadyResolve: (() => void) | undefined;
const workerReady = new Promise<void>((resolve) => {
workerReadyResolve = resolve;
});
child.on("message", (raw: unknown) => {
const result = parseWorkerMessage(raw);
if (result.ok && result.value.type === "ready") {
workerReadyResolve?.();
}
handleWorkerMessage(raw);
});
child.on("exit", (code, signal) => {
const summary = formatChildExitSummary(code, signal ?? null);
process.stderr.write(
`[kernel] worker for group "${group}" exited (${summary})${formatCapturedStderrTail(stderrTail.value)}\n`,
);
// Resolve ready in case the worker exits before sending ready (prevents hangs)
workerReadyResolve?.();
if (!stopped && code !== 0) {
process.stderr.write(`[kernel] respawning worker for group "${group}" in 1s\n`);
for (const senseName of sensesForGroup(group)) {
scheduler.onComputeComplete(senseName);
}
setTimeout(() => {
if (!stopped) {
startWorker(group);
}
}, 1000);
}
});
workers.set(group, { group, process: child });
return workerReady;
}
const senseWorkerPool = createSenseWorkerPool({
nerveRoot,
workerScript,
onWorkerMessage: handleWorkerMessage,
sensesForGroup: (group) => senseNamesInGroup(config, group),
onWorkerCrashed: clearSchedulerForGroup,
onBeforeGroupRestart: clearSchedulerForGroup,
isStopped: () => stopped,
});
function triggerFn(senseName: string): void {
const group = groupForSense(config, senseName);
@@ -297,12 +192,7 @@ export function createKernel(
process.stderr.write(`[kernel] triggerFn: unknown sense "${senseName}"\n`);
return;
}
const entry = workers.get(group);
if (entry === undefined) {
process.stderr.write(`[kernel] triggerFn: no worker for group "${group}"\n`);
return;
}
sendCompute(entry.process, senseName);
senseWorkerPool.sendCompute(group, senseName);
}
function triggerSense(senseName: string): void {
@@ -310,18 +200,14 @@ export function createKernel(
if (group === null) {
throw new Error(`Unknown sense: "${senseName}"`);
}
const entry = workers.get(group);
if (entry === undefined) {
if (!senseWorkerPool.hasWorkerForGroup(group)) {
throw new Error(`No worker running for group "${group}" (sense: "${senseName}")`);
}
sendCompute(entry.process, senseName);
senseWorkerPool.sendCompute(group, senseName);
}
scheduler = createReflexScheduler(config, bus, triggerFn, {
logStore,
workflowTriggerFn: (workflowName, payload) => {
workflowManager.startWorkflow(workflowName, payload);
},
});
if (groups.size === 0) {
@@ -329,63 +215,13 @@ export function createKernel(
}
for (const group of groups) {
startWorker(group);
}
function waitForExit(child: ChildProcess, timeoutMs: number): Promise<void> {
return new Promise((resolve) => {
const timer = setTimeout(() => {
child.kill("SIGKILL");
resolve();
}, timeoutMs);
child.once("exit", () => {
clearTimeout(timer);
resolve();
});
});
}
// --- restartGroup: gracefully stop worker, then respawn and await ready ---
async function restartGroup(group: string): Promise<void> {
const entry = workers.get(group);
if (entry === undefined) return;
for (const senseName of sensesForGroup(group)) {
scheduler.onComputeComplete(senseName);
}
sendShutdown(entry.process);
await waitForExit(entry.process, 5000);
if (!stopped) {
await startWorker(group);
}
}
function collectGroups(cfg: NerveConfig): Set<string> {
const result = new Set<string>();
for (const sc of Object.values(cfg.senses)) {
result.add(sc.group);
}
return result;
}
function sensesForGroupInConfig(cfg: NerveConfig, group: string): Set<string> {
const result = new Set<string>();
for (const [name, sc] of Object.entries(cfg.senses)) {
if (sc.group === group) result.add(name);
}
return result;
senseWorkerPool.startWorker(group);
}
function removeStaleGroups(oldGroups: Set<string>, newGroups: Set<string>): void {
for (const g of oldGroups) {
if (newGroups.has(g)) continue;
const entry = workers.get(g);
if (entry !== undefined) {
sendShutdown(entry.process);
workers.delete(g);
}
senseWorkerPool.evictGroup(g);
groups.delete(g);
}
}
@@ -394,30 +230,25 @@ export function createKernel(
for (const g of newGroups) {
if (oldGroups.has(g)) continue;
groups.add(g);
if (!stopped) startWorker(g);
if (!stopped) {
senseWorkerPool.startWorker(g);
}
}
}
function reloadConfig(newConfig: NerveConfig): void {
const oldGroups = collectGroups(config);
const oldGroups = collectSenseGroups(config);
const oldConfig = config;
const oldWorkflows = config.workflows ?? {};
const oldWorkflows = config.workflows;
config = newConfig;
// Note: pending/throttled computes in the old scheduler are silently dropped here.
// In-flight state is not preserved across reloadConfig.
scheduler.stop();
scheduler = createReflexScheduler(config, bus, triggerFn, {
logStore,
workflowTriggerFn: (workflowName, payload) => {
workflowManager.startWorkflow(workflowName, payload);
},
});
// Update workflow concurrency/overflow config incrementally — no restart needed
workflowManager.updateConfig(newConfig);
const newWorkflows = newConfig.workflows ?? {};
const newWorkflows = newConfig.workflows;
// Drain + remove workers for deleted workflows
for (const workflowName of Object.keys(oldWorkflows)) {
if (!(workflowName in newWorkflows)) {
process.stderr.write(
@@ -432,20 +263,17 @@ export function createKernel(
}
}
const newGroups = collectGroups(newConfig);
const newGroups = collectSenseGroups(newConfig);
removeStaleGroups(oldGroups, newGroups);
addNewGroups(oldGroups, newGroups);
// Restart existing groups that gained new senses — the running worker process
// was spawned with the old config and will report "Unknown sense" for any newly
// added sense until it is restarted.
for (const g of newGroups) {
if (!oldGroups.has(g)) continue; // already handled by addNewGroups
const oldSenses = sensesForGroupInConfig(oldConfig, g);
const newSenses = sensesForGroupInConfig(newConfig, g);
if (!oldGroups.has(g)) continue;
const oldSenses = senseNamesInGroupAsSet(oldConfig, g);
const newSenses = senseNamesInGroupAsSet(newConfig, g);
const gained = [...newSenses].some((s) => !oldSenses.has(s));
if (gained) {
restartGroup(g).catch((e) => {
senseWorkerPool.restartGroup(g).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] reloadConfig restartGroup error for "${g}": ${msg}\n`);
});
@@ -457,80 +285,28 @@ export function createKernel(
return {
uptime: Date.now() - startTime,
activeSenses: Object.keys(config.senses).length,
activeGroups: workers.size,
activeGroups: senseWorkerPool.activeGroupCount(),
pendingComputes: 0,
activeWorkflows: workflowManager.totalActiveCount(),
memoryUsage: process.memoryUsage(),
};
}
function handleSenseFileChange(senseName: string): void {
const sc = config.senses[senseName];
if (sc === undefined) return;
process.stderr.write(
`[kernel] sense file changed: "${senseName}", restarting group "${sc.group}"\n`,
);
logStore.append({
source: "system",
type: "sense_reload",
refId: senseName,
payload: null,
ts: Date.now(),
});
restartGroup(sc.group).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] restartGroup error: ${msg}\n`);
});
}
function handleWorkflowFileChange(workflowName: string): void {
process.stderr.write(
`[kernel] workflow file changed: "${workflowName}", draining and respawning worker\n`,
);
logStore.append({
source: "system",
type: "workflow_reload",
refId: workflowName,
payload: null,
ts: Date.now(),
});
workflowManager.drainAndRespawn(workflowName).catch((e) => {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] drainAndRespawn error for "${workflowName}": ${msg}\n`);
});
}
function handleConfigFileChange(): void {
process.stderr.write("[kernel] nerve.yaml changed, reloading config\n");
logStore.append({
source: "system",
type: "config_reload",
refId: null,
payload: null,
ts: Date.now(),
});
try {
const raw = readFileSync(join(nerveRoot, "nerve.yaml"), "utf8");
const parseResult = parseNerveConfig(raw);
if (!parseResult.ok) {
process.stderr.write(
`[kernel] config parse error, keeping current config: ${parseResult.error.message}\n`,
);
return;
}
reloadConfig(parseResult.value);
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[kernel] failed to read nerve.yaml, keeping current config: ${msg}\n`);
}
}
const fileWatchHandlers = createKernelFileWatchHandlers({
nerveRoot,
getConfig: () => config,
logStore,
workflowManager,
restartGroup: (group) => senseWorkerPool.restartGroup(group),
reloadConfig,
});
let fileWatcher: FileWatcher | null = null;
if (options.enableFileWatcher) {
fileWatcher = createFileWatcher(nerveRoot, (change) => {
if (change.kind === "sense") handleSenseFileChange(change.senseName);
if (change.kind === "config") handleConfigFileChange();
if (change.kind === "workflow") handleWorkflowFileChange(change.workflowName);
if (change.kind === "sense") fileWatchHandlers.onSenseFileChange(change.senseName);
if (change.kind === "config") fileWatchHandlers.onConfigFileChange();
if (change.kind === "workflow") fileWatchHandlers.onWorkflowFileChange(change.workflowName);
});
}
@@ -551,7 +327,7 @@ export function createKernel(
group: senseConfig.group,
throttle: senseConfig.throttle,
timeout: senseConfig.timeout,
lastSignalTs: lastEntry !== null ? lastEntry.ts : null,
lastSignalTimestamp: lastEntry !== null ? lastEntry.timestamp : null,
};
});
},
@@ -570,24 +346,19 @@ export function createKernel(
}
scheduler.stop();
await workflowManager.stop();
const exitPromises: Promise<void>[] = [];
for (const entry of workers.values()) {
sendShutdown(entry.process);
exitPromises.push(waitForExit(entry.process, 5000));
}
await Promise.all(exitPromises);
await senseWorkerPool.shutdownAll();
logStore.append({
source: "system",
type: "stop",
refId: null,
payload: null,
ts: Date.now(),
timestamp: Date.now(),
});
logStore.close();
}
function getWorkerPid(group: string): number | null {
return workers.get(group)?.process.pid ?? null;
return senseWorkerPool.getWorkerPid(group);
}
const senseCount = Object.keys(config.senses).length;
@@ -603,7 +374,7 @@ export function createKernel(
getWorkerPid,
triggerCompute: triggerFn,
triggerSense,
restartGroup,
restartGroup: (group) => senseWorkerPool.restartGroup(group),
reloadConfig,
getHealth,
};
+3 -22
View File
@@ -10,15 +10,12 @@
*/
import type { NerveConfig } from "@uncaged/nerve-core";
import type { LogStore } from "./log-store.js";
import type { LogStore } from "@uncaged/nerve-store";
import type { SignalBus, Unsubscribe } from "./signal-bus.js";
/** Sends a compute message to the worker responsible for the given sense. */
export type TriggerFn = (senseName: string) => void;
/** Triggers a workflow run in response to a signal. */
export type WorkflowTriggerFn = (workflowName: string, payload: unknown) => void;
/** Per-sense mutable state tracked by the scheduler. */
type SenseState = {
lastComputeAt: number;
@@ -40,7 +37,6 @@ function makeSenseState(): SenseState {
export type ReflexSchedulerOptions = {
logStore?: LogStore;
workflowTriggerFn?: WorkflowTriggerFn;
};
/**
@@ -82,7 +78,7 @@ export function createReflexScheduler(
type: "run_start",
refId: senseName,
payload: null,
ts: Date.now(),
timestamp: Date.now(),
});
triggerFn(senseName);
}
@@ -157,21 +153,6 @@ export function createReflexScheduler(
}
for (const reflex of config.reflexes) {
if (reflex.kind === "workflow") {
if (opts?.workflowTriggerFn !== undefined && reflex.on !== null && reflex.on.length > 0) {
const workflowTriggerFn = opts.workflowTriggerFn;
const workflowName = reflex.workflow;
const watchedSenses = new Set(reflex.on);
const unsub = bus.subscribe((signal) => {
if (watchedSenses.has(signal.senseId)) {
workflowTriggerFn(workflowName, signal.payload);
}
});
unsubscribers.push(unsub);
}
continue;
}
if (reflex.kind !== "sense") continue;
const senseReflex = reflex;
const senseName = senseReflex.sense;
@@ -183,7 +164,7 @@ export function createReflexScheduler(
intervals.push(id);
}
if (senseReflex.on !== null && senseReflex.on.length > 0) {
if (senseReflex.on.length > 0) {
const watchedSenses = new Set(senseReflex.on);
const unsub = bus.subscribe((signal) => {
if (watchedSenses.has(signal.senseId)) {
+38 -38
View File
@@ -1,17 +1,17 @@
import { mkdirSync, readFileSync, readdirSync } from "node:fs";
import { dirname, join } from "node:path";
import { DatabaseSync } from "node:sqlite";
import Database from "better-sqlite3";
import { drizzle } from "drizzle-orm/better-sqlite3";
import type { BetterSQLite3Database } from "drizzle-orm/better-sqlite3";
import { drizzle } from "drizzle-orm/node-sqlite";
import type { NodeSQLiteDatabase } from "drizzle-orm/node-sqlite";
import type { Result } from "@uncaged/nerve-core";
import { err, ok } from "@uncaged/nerve-core";
import { err, isPlainRecord, ok } from "@uncaged/nerve-core";
import type { BlobStore } from "./blob-store.js";
import type { BlobStore } from "@uncaged/nerve-store";
/** A Drizzle DB instance (schema-generic) */
export type DrizzleDB = BetterSQLite3Database<Record<string, never>>;
export type DrizzleDB = NodeSQLiteDatabase<Record<string, never>>;
/** Read-only map of peer sense name → their Drizzle DB */
export type PeerMap = Readonly<Record<string, DrizzleDB>>;
@@ -42,7 +42,7 @@ export type SenseRuntime = {
compute: ComputeFn;
};
function ensureMigrationsTable(sqlite: Database.Database): Result<void> {
function ensureMigrationsTable(sqlite: DatabaseSync): Result<void> {
try {
sqlite.exec(
`CREATE TABLE IF NOT EXISTS _migrations (
@@ -69,11 +69,7 @@ function listMigrationFiles(migrationsDir: string): Result<string[]> {
}
}
function applyMigrationFile(
sqlite: Database.Database,
file: string,
filePath: string,
): Result<void> {
function applyMigrationFile(sqlite: DatabaseSync, file: string, filePath: string): Result<void> {
let sql: string;
try {
sql = readFileSync(filePath, "utf8");
@@ -83,13 +79,18 @@ function applyMigrationFile(
}
const insertJournal = sqlite.prepare("INSERT INTO _migrations (name, applied_at) VALUES (?, ?)");
sqlite.exec("BEGIN IMMEDIATE");
try {
sqlite.transaction(() => {
sqlite.exec(sql);
insertJournal.run(file, Date.now());
})();
sqlite.exec(sql);
insertJournal.run(file, Date.now());
sqlite.exec("COMMIT");
return ok(undefined);
} catch (e) {
try {
sqlite.exec("ROLLBACK");
} catch {
// ignore secondary errors during rollback
}
const msg = e instanceof Error ? e.message : String(e);
return err(new Error(`Migration "${file}" failed: ${msg}`));
}
@@ -97,20 +98,21 @@ function applyMigrationFile(
/**
* Run all *.sql migration files in the given directory against a
* better-sqlite3 Database, in lexicographic order.
* `node:sqlite` DatabaseSync, in lexicographic order.
* Tracks applied migrations in _migrations table to avoid re-running.
*/
export function runMigrations(sqlite: Database.Database, migrationsDir: string): Result<void> {
export function runMigrations(sqlite: DatabaseSync, migrationsDir: string): Result<void> {
const tableResult = ensureMigrationsTable(sqlite);
if (!tableResult.ok) return tableResult;
const filesResult = listMigrationFiles(migrationsDir);
if (!filesResult.ok) return filesResult;
const migrationRows = sqlite.prepare("SELECT name FROM _migrations").all();
const applied = new Set<string>(
(sqlite.prepare("SELECT name FROM _migrations").all() as Array<{ name: string }>).map(
(r) => r.name,
),
migrationRows
.filter((r): r is { name: string } => isPlainRecord(r) && typeof r.name === "string")
.map((r) => r.name),
);
for (const file of filesResult.value) {
@@ -129,14 +131,13 @@ export function runMigrations(sqlite: Database.Database, migrationsDir: string):
export function openSenseDb(
dbPath: string,
migrationsDir: string,
): Result<{ sqlite: Database.Database; db: DrizzleDB }> {
let sqlite: Database.Database;
): Result<{ sqlite: DatabaseSync; db: DrizzleDB }> {
let sqlite: DatabaseSync;
try {
mkdirSync(dirname(dbPath), { recursive: true });
sqlite = new Database(dbPath);
// WAL mode for better concurrent read performance
sqlite.pragma("journal_mode = WAL");
sqlite = new DatabaseSync(dbPath);
sqlite.exec("PRAGMA journal_mode=WAL");
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
return err(new Error(`Failed to open database "${dbPath}": ${msg}`));
@@ -145,7 +146,8 @@ export function openSenseDb(
const migResult = runMigrations(sqlite, migrationsDir);
if (!migResult.ok) return migResult;
const db = drizzle(sqlite) as DrizzleDB;
// Drizzle infers a schema-specific DB type; senses are schema-agnostic at this layer.
const db = drizzle({ client: sqlite }) as DrizzleDB;
return ok({ sqlite, db });
}
@@ -153,16 +155,17 @@ export function openSenseDb(
* Open a peer sense DB in read-only mode (no migrations).
*/
export function openPeerDb(dbPath: string): Result<DrizzleDB> {
let sqlite: Database.Database;
let sqlite: DatabaseSync;
try {
sqlite = new Database(dbPath, { readonly: true });
sqlite = new DatabaseSync(dbPath, { readOnly: true });
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
return err(new Error(`Failed to open peer database "${dbPath}" (readonly): ${msg}`));
}
return ok(drizzle(sqlite) as DrizzleDB);
// Same schema-agnostic Drizzle wrapper as openSenseDb.
return ok(drizzle({ client: sqlite }) as DrizzleDB);
}
/**
@@ -180,18 +183,13 @@ export async function loadComputeFn(senseIndexPath: string): Promise<Result<Comp
return err(new Error(`Failed to import sense module "${senseIndexPath}": ${msg}`));
}
if (
mod === null ||
typeof mod !== "object" ||
!("compute" in mod) ||
typeof (mod as Record<string, unknown>).compute !== "function"
) {
if (!isPlainRecord(mod) || !("compute" in mod) || typeof mod.compute !== "function") {
return err(
new Error(`Sense module "${senseIndexPath}" must export a named "compute" function`),
);
}
return ok((mod as { compute: ComputeFn }).compute);
return ok(mod.compute as ComputeFn);
}
/**
@@ -232,7 +230,9 @@ export async function executeCompute(
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
if (controller.signal.aborted) {
return err(new Error(`compute("${runtime.name}") timed out after ${timeoutMs as number}ms`));
return err(
new Error(`compute("${runtime.name}") timed out after ${String(timeoutMs ?? "?")}ms`),
);
}
return err(new Error(`compute("${runtime.name}") threw: ${msg}`));
} finally {
+1 -1
View File
@@ -20,7 +20,7 @@ import { join, resolve } from "node:path";
import { parseNerveConfig } from "@uncaged/nerve-core";
import type { NerveConfig } from "@uncaged/nerve-core";
import { createBlobStore } from "./blob-store.js";
import { createBlobStore } from "@uncaged/nerve-store";
import type { WorkerToParentMessage } from "./ipc.js";
import { parseParentMessage } from "./ipc.js";
import { executeCompute, loadComputeFn, openPeerDb, openSenseDb } from "./sense-runtime.js";
+1 -4
View File
@@ -26,10 +26,7 @@ export function teeCapturedStderr(child: ChildProcess, tail: { value: string }):
});
}
export function formatChildExitSummary(
code: number | null,
signal: NodeJS.Signals | null,
): string {
export function formatChildExitSummary(code: number | null, signal: NodeJS.Signals | null): string {
const codeStr = code === null || code === undefined ? "null" : String(code);
if (signal) {
return `code=${codeStr} signal=${signal}`;
+211
View File
@@ -0,0 +1,211 @@
/**
* Sense worker pool forked child processes per sense group (IPC lifecycle).
*/
import { fork } from "node:child_process";
import type { ChildProcess } from "node:child_process";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { ComputeMessage, ShutdownMessage } from "./ipc.js";
import { parseWorkerMessage } from "./ipc.js";
import {
formatCapturedStderrTail,
formatChildExitSummary,
teeCapturedStderr,
} from "./worker-fork-support.js";
export function resolveWorkerScript(): string {
const __filename = fileURLToPath(import.meta.url);
const __dir = dirname(__filename);
return join(__dir, "sense-worker.js");
}
type WorkerEntry = {
group: string;
process: ChildProcess;
};
export type SenseWorkerPoolOptions = {
nerveRoot: string;
workerScript: string;
/** Invoked for every IPC message from a worker (including ready / signal / error). */
onWorkerMessage: (raw: unknown) => void;
/** Sense names in a group — used when clearing scheduler state on crash or restart. */
sensesForGroup: (group: string) => string[];
/**
* Called when a worker exits with non-zero code before scheduling a respawn
* (scheduler should release pending computes for senses in that group).
*/
onWorkerCrashed: (group: string) => void;
/**
* Called at the beginning of `restartGroup` before shutdown
* (same scheduler cleanup as crash path).
*/
onBeforeGroupRestart: (group: string) => void;
isStopped: () => boolean;
};
export type SenseWorkerPool = {
startWorker: (group: string) => Promise<void>;
restartGroup: (group: string) => Promise<void>;
/** Send shutdown and drop the entry without waiting (matches reloadConfig stale-group removal). */
evictGroup: (group: string) => void;
shutdownAll: () => Promise<void>;
sendCompute: (group: string, senseName: string) => void;
getWorkerPid: (group: string) => number | null;
hasWorkerForGroup: (group: string) => boolean;
activeGroupCount: () => number;
};
function spawnWorker(
nerveRoot: string,
group: string,
workerScript: string,
stderrTail: { value: string },
): ChildProcess {
const child = fork(workerScript, ["--group", group, "--root", nerveRoot], {
stdio: ["ignore", "inherit", "pipe", "ipc"],
});
teeCapturedStderr(child, stderrTail);
child.on("error", (err) => {
if ((err as NodeJS.ErrnoException).code !== "EPIPE") {
console.error("[worker] error:", err.message);
}
});
return child;
}
function sendComputeToProcess(worker: ChildProcess, senseName: string): void {
if (worker.connected === false) return;
const msg: ComputeMessage = { type: "compute", sense: senseName };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function sendShutdownToProcess(worker: ChildProcess): void {
if (worker.connected === false) return;
const msg: ShutdownMessage = { type: "shutdown" };
try {
worker.send(msg);
} catch {
// IPC channel closed between connected check and send
}
}
function waitForExit(child: ChildProcess, timeoutMs: number): Promise<void> {
return new Promise((resolve) => {
const timer = setTimeout(() => {
child.kill("SIGKILL");
resolve();
}, timeoutMs);
child.once("exit", () => {
clearTimeout(timer);
resolve();
});
});
}
export function createSenseWorkerPool(options: SenseWorkerPoolOptions): SenseWorkerPool {
const workers = new Map<string, WorkerEntry>();
function startWorker(group: string): Promise<void> {
const stderrTail = { value: "" };
const child = spawnWorker(options.nerveRoot, group, options.workerScript, stderrTail);
let workerReadyResolve: (() => void) | undefined;
const workerReady = new Promise<void>((resolve) => {
workerReadyResolve = resolve;
});
child.on("message", (raw: unknown) => {
const result = parseWorkerMessage(raw);
if (result.ok && result.value.type === "ready") {
workerReadyResolve?.();
}
options.onWorkerMessage(raw);
});
child.on("exit", (code, signal) => {
const summary = formatChildExitSummary(code, signal ?? null);
process.stderr.write(
`[kernel] worker for group "${group}" exited (${summary})${formatCapturedStderrTail(stderrTail.value)}\n`,
);
workerReadyResolve?.();
if (!options.isStopped() && code !== 0) {
process.stderr.write(`[kernel] respawning worker for group "${group}" in 1s\n`);
options.onWorkerCrashed(group);
setTimeout(() => {
if (!options.isStopped()) {
startWorker(group);
}
}, 1000);
}
});
workers.set(group, { group, process: child });
return workerReady;
}
async function restartGroup(group: string): Promise<void> {
const entry = workers.get(group);
if (entry === undefined) return;
options.onBeforeGroupRestart(group);
sendShutdownToProcess(entry.process);
await waitForExit(entry.process, 5000);
if (!options.isStopped()) {
await startWorker(group);
}
}
function evictGroup(group: string): void {
const entry = workers.get(group);
if (entry === undefined) return;
sendShutdownToProcess(entry.process);
workers.delete(group);
}
async function shutdownAll(): Promise<void> {
const exitPromises: Promise<void>[] = [];
for (const entry of workers.values()) {
sendShutdownToProcess(entry.process);
exitPromises.push(waitForExit(entry.process, 5000));
}
await Promise.all(exitPromises);
}
function sendCompute(group: string, senseName: string): void {
const entry = workers.get(group);
if (entry === undefined) return;
sendComputeToProcess(entry.process, senseName);
}
function getWorkerPid(group: string): number | null {
return workers.get(group)?.process.pid ?? null;
}
function hasWorkerForGroup(group: string): boolean {
return workers.has(group);
}
function activeGroupCount(): number {
return workers.size;
}
return {
startWorker,
restartGroup,
evictGroup,
shutdownAll,
sendCompute,
getWorkerPid,
hasWorkerForGroup,
activeGroupCount,
};
}
+108 -30
View File
@@ -11,8 +11,10 @@ import type { ChildProcess } from "node:child_process";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import type { NerveConfig, WorkflowConfig } from "@uncaged/nerve-core";
import type { NerveConfig, WorkflowConfig, WorkflowMessage } from "@uncaged/nerve-core";
import { START, isPlainRecord } from "@uncaged/nerve-core";
import type { LogStore, WorkflowRunStatus } from "@uncaged/nerve-store";
import type {
ResumeThreadMessage,
ShutdownMessage,
@@ -20,17 +22,21 @@ import type {
ThreadEventMessage,
} from "./ipc.js";
import { parseWorkerMessage } from "./ipc.js";
import type { LogStore } from "./log-store.js";
import type { WorkflowRunStatus } from "./log-store.js";
import {
formatCapturedStderrTail,
formatChildExitSummary,
teeCapturedStderr,
} from "./worker-fork-support.js";
export type WorkflowLaunchParams = {
prompt: string;
maxRounds: number;
dryRun: boolean;
};
export type WorkflowManager = {
/** Trigger a new workflow thread (called by Reflex scheduler). */
startWorkflow: (workflowName: string, payload: unknown) => void;
/** Trigger a new workflow thread (Sense-driven launch or CLI / IPC). */
startWorkflow: (workflowName: string, launch: WorkflowLaunchParams) => void;
/** Number of currently active (running) threads for a workflow. */
activeCount: (workflowName: string) => number;
/** Number of pending queued threads waiting to run for a workflow. */
@@ -51,7 +57,9 @@ export type WorkflowManager = {
type PendingThread = {
runId: string;
payload: unknown;
prompt: string;
maxRounds: number;
dryRun: boolean;
};
type WorkflowState = {
@@ -81,6 +89,44 @@ const WORKER_SHUTDOWN_TIMEOUT_MS = 10_000;
const DEFAULT_MAX_QUEUE = 100;
function readLaunchFromTriggerPayload(
raw: unknown,
engineDefaultMaxRounds: number,
): { prompt: string; maxRounds: number; dryRun: boolean } {
if (isPlainRecord(raw)) {
const o = raw;
if (typeof o.prompt === "string" && typeof o.maxRounds === "number") {
const dryRun = typeof o.dryRun === "boolean" ? o.dryRun : false;
return { prompt: o.prompt, maxRounds: o.maxRounds, dryRun };
}
}
return { prompt: "", maxRounds: engineDefaultMaxRounds, dryRun: false };
}
function ensureThreadMessagesWithStart(
messages: Array<{ role: string; content: string; meta: unknown; timestamp: number }>,
fallbackPrompt: string,
fallbackMaxRounds: number,
fallbackDryRun: boolean,
): WorkflowMessage[] {
const mapped: WorkflowMessage[] = messages.map((m) => ({
role: m.role,
content: m.content,
meta: m.meta,
timestamp: m.timestamp,
}));
if (mapped.length > 0 && mapped[0].role === START) {
return mapped;
}
const start: WorkflowMessage = {
role: START,
content: fallbackPrompt,
meta: { maxRounds: fallbackMaxRounds, dryRun: fallbackDryRun },
timestamp: Date.now(),
};
return [start, ...mapped];
}
function resolveWorkerScript(): string {
const __filename = fileURLToPath(import.meta.url);
const __dir = dirname(__filename);
@@ -171,7 +217,7 @@ export function createWorkflowManager(
}
function workflowConfig(workflowName: string): WorkflowConfig | null {
return config.workflows?.[workflowName] ?? null;
return config.workflows[workflowName] ?? null;
}
function toWorkflowRunStatus(eventType: string): WorkflowRunStatus | null {
@@ -193,14 +239,20 @@ export function createWorkflowManager(
eventType: string,
payload?: unknown,
): void {
const ts = Date.now();
const timestamp = Date.now();
const serialised = payload !== undefined ? JSON.stringify(payload) : null;
const status = toWorkflowRunStatus(eventType);
if (status !== null) {
logStore.upsertWorkflowRun(
{ source: "workflow", type: eventType, refId: runId, payload: serialised, ts },
{ runId, workflow: workflowName, status, ts },
{
source: "workflow",
type: eventType,
refId: runId,
payload: serialised,
timestamp,
},
{ runId, workflow: workflowName, status, timestamp },
);
} else {
logStore.append({
@@ -208,12 +260,18 @@ export function createWorkflowManager(
type: eventType,
refId: runId,
payload: serialised,
ts,
timestamp,
});
}
}
function dispatchThread(workflowName: string, runId: string, payload: unknown): void {
function dispatchThread(
workflowName: string,
runId: string,
prompt: string,
maxRounds: number,
dryRun: boolean,
): void {
const state = getOrCreateState(workflowName);
state.active.add(runId);
@@ -222,11 +280,12 @@ export function createWorkflowManager(
type: "start-thread",
runId,
workflow: workflowName,
triggerPayload: payload,
prompt,
maxRounds,
dryRun,
};
sendStartThread(worker.process, msg);
// Store triggerPayload in the log so it can be recovered after a crash
logWorkflowEvent(workflowName, runId, "started", { triggerPayload: payload });
logWorkflowEvent(workflowName, runId, "started", { prompt, maxRounds, dryRun });
}
function dequeueNext(workflowName: string): void {
@@ -239,7 +298,7 @@ export function createWorkflowManager(
if (state.active.size < concurrency) {
const next = state.queue.shift();
if (next !== undefined) {
dispatchThread(workflowName, next.runId, next.payload);
dispatchThread(workflowName, next.runId, next.prompt, next.maxRounds, next.dryRun);
}
}
}
@@ -260,8 +319,16 @@ export function createWorkflowManager(
function recoverQueuedRun(workflowName: string, runId: string, state: WorkflowState): void {
if (state.queue.some((q) => q.runId === runId)) return;
const triggerPayload = logStore.getTriggerPayload(runId);
state.queue.push({ runId, payload: triggerPayload });
const launch = readLaunchFromTriggerPayload(
logStore.getTriggerPayload(runId),
config.maxRounds,
);
state.queue.push({
runId,
prompt: launch.prompt,
maxRounds: launch.maxRounds,
dryRun: launch.dryRun,
});
process.stderr.write(
`[workflow-manager] crash-recovery: re-queued thread "${runId}" for "${workflowName}"\n`,
);
@@ -274,18 +341,28 @@ export function createWorkflowManager(
worker: WorkerEntry,
): void {
if (state.active.has(runId)) return;
const events = logStore.getThreadEvents(runId);
const triggerPayload = logStore.getTriggerPayload(runId);
const rawMessages = logStore.getThreadMessages(runId);
const launch = readLaunchFromTriggerPayload(
logStore.getTriggerPayload(runId),
config.maxRounds,
);
const messages = ensureThreadMessagesWithStart(
rawMessages,
launch.prompt,
launch.maxRounds,
launch.dryRun,
);
state.active.add(runId);
const msg: ResumeThreadMessage = {
type: "resume-thread",
runId,
events,
triggerPayload,
messages,
maxRounds: launch.maxRounds,
dryRun: launch.dryRun,
};
sendResumeThread(worker.process, msg);
process.stderr.write(
`[workflow-manager] crash-recovery: resuming thread "${runId}" for "${workflowName}" (${events.length} events)\n`,
`[workflow-manager] crash-recovery: resuming thread "${runId}" for "${workflowName}" (${messages.length} messages)\n`,
);
}
@@ -363,13 +440,13 @@ export function createWorkflowManager(
return;
}
if (msg.type === "thread-command-event") {
if (msg.type === "thread-workflow-message") {
logStore.append({
source: "workflow",
type: "thread_command_event",
type: "thread_workflow_message",
refId: msg.runId,
payload: JSON.stringify(msg.event),
ts: Date.now(),
payload: JSON.stringify(msg.message),
timestamp: Date.now(),
});
return;
}
@@ -464,7 +541,7 @@ export function createWorkflowManager(
return entry;
}
function startWorkflow(workflowName: string, payload: unknown): void {
function startWorkflow(workflowName: string, launch: WorkflowLaunchParams): void {
if (stopped) return;
const wfConfig = workflowConfig(workflowName);
@@ -477,9 +554,10 @@ export function createWorkflowManager(
const state = getOrCreateState(workflowName);
const runId = crypto.randomUUID();
const { prompt, maxRounds, dryRun } = launch;
if (state.active.size < wfConfig.concurrency) {
dispatchThread(workflowName, runId, payload);
dispatchThread(workflowName, runId, prompt, maxRounds, dryRun);
return;
}
@@ -504,7 +582,7 @@ export function createWorkflowManager(
}
}
state.queue.push({ runId, payload });
state.queue.push({ runId, prompt, maxRounds, dryRun });
logWorkflowEvent(workflowName, runId, "queued");
process.stderr.write(
`[workflow-manager] queued thread for "${workflowName}" runId "${runId}" (queue length: ${state.queue.length})\n`,
+199 -115
View File
@@ -12,14 +12,14 @@
import { existsSync } from "node:fs";
import { join, resolve } from "node:path";
import type {
CommandEvent,
ThreadState,
WorkflowContext,
WorkflowDefinition,
} from "@uncaged/nerve-core";
import type { RoleMeta, StartStep, WorkflowDefinition, WorkflowMessage } from "@uncaged/nerve-core";
import { END, START, isPlainRecord } from "@uncaged/nerve-core";
import type { ThreadCommandEventMessage, ThreadEventType, WorkerToParentMessage } from "./ipc.js";
import type {
ThreadEventType,
ThreadWorkflowMessageMessage,
WorkerToParentMessage,
} from "./ipc.js";
import { parseParentMessage } from "./ipc.js";
import { ignoreSessionBroadcastSignals } from "./worker-fork-support.js";
@@ -45,147 +45,237 @@ function sendWorkflowError(runId: string, error: string): void {
send({ type: "workflow-error", runId, error });
}
function sendCommandEvent(runId: string, event: CommandEvent): void {
const msg: ThreadCommandEventMessage = {
type: "thread-command-event",
function sendWorkflowMessage(runId: string, message: WorkflowMessage): void {
const msg: ThreadWorkflowMessageMessage = {
type: "thread-workflow-message",
runId,
event: event as { type: string; [key: string]: unknown },
message: {
role: message.role,
content: message.content,
meta: message.meta,
timestamp: message.timestamp,
},
};
send(msg);
}
// ---------------------------------------------------------------------------
// Thread loop (RFC-002 §5.4)
// Thread loop (signal-driven automaton, issue #80)
// ---------------------------------------------------------------------------
/**
* Replay persisted events through moderate() to reconstruct ThreadState,
* then execute the next role and return the resulting CommandEvent.
* Returns null if the thread is already complete (moderate returned null).
*/
async function replayAndResume(
def: WorkflowDefinition,
function validateRoleResult(
result: { content: string; meta: Record<string, unknown> },
roleName: string,
runId: string,
ctx: WorkflowContext,
state: ThreadState,
resumeEvents: CommandEvent[],
): Promise<CommandEvent | null> {
let lastNext: ReturnType<typeof def.moderate> = null;
for (const ev of resumeEvents) {
state.events.push(ev);
lastNext = def.moderate(state, ev);
if (lastNext === null) {
sendThreadEvent(runId, "completed", null);
return null;
): boolean {
if (typeof result.content !== "string") {
sendWorkflowError(runId, `Role "${roleName}" returned non-string content`);
return false;
}
if (result.meta === null || typeof result.meta !== "object" || Array.isArray(result.meta)) {
sendWorkflowError(runId, `Role "${roleName}" returned invalid meta (must be a plain object)`);
return false;
}
return true;
}
function isStartMeta(meta: unknown): meta is StartStep["meta"] {
return (
isPlainRecord(meta) && typeof meta.maxRounds === "number" && typeof meta.dryRun === "boolean"
);
}
function normalizeStartMeta(meta: unknown, maxRoundsFallback: number): StartStep["meta"] {
if (!isPlainRecord(meta)) {
return { maxRounds: maxRoundsFallback, dryRun: false };
}
const maxRounds = typeof meta.maxRounds === "number" ? meta.maxRounds : maxRoundsFallback;
const dryRun = typeof meta.dryRun === "boolean" ? meta.dryRun : false;
return { maxRounds, dryRun };
}
function startStepFromWorkflowMessage(msg: WorkflowMessage, maxRoundsFallback: number): StartStep {
if (msg.role !== START) {
return {
role: START,
content: "",
meta: { maxRounds: maxRoundsFallback, dryRun: false },
timestamp: Date.now(),
};
}
const meta = isStartMeta(msg.meta) ? msg.meta : normalizeStartMeta(msg.meta, maxRoundsFallback);
return {
role: START,
content: msg.content,
meta,
timestamp: msg.timestamp,
};
}
type ThreadMessagesState = {
start: StartStep;
/** Role outputs only; never includes the `__start__` frame. */
messages: WorkflowMessage[];
};
function initThreadMessages(
runId: string,
resumeMessages: WorkflowMessage[],
freshPrompt: string | null,
maxRounds: number,
dryRun: boolean,
): ThreadMessagesState {
if (resumeMessages.length > 0) {
const [first, ...rest] = resumeMessages;
if (first.role === START) {
return {
start: startStepFromWorkflowMessage(first, maxRounds),
messages: [...rest],
};
}
const prompt = freshPrompt ?? "";
return {
start: {
role: START,
content: prompt,
meta: { maxRounds, dryRun },
timestamp: Date.now(),
},
messages: [...resumeMessages],
};
}
const prompt = freshPrompt ?? "";
const start: StartStep = {
role: START,
content: prompt,
meta: { maxRounds, dryRun },
timestamp: Date.now(),
};
sendWorkflowMessage(runId, {
role: start.role,
content: start.content,
meta: start.meta,
timestamp: start.timestamp,
});
return { start, messages: [] };
}
const next = lastNext;
if (next === null) {
sendThreadEvent(runId, "completed", null);
return null;
}
const role = def.roles[next.role];
async function executeRole(
def: WorkflowDefinition<RoleMeta>,
nextRole: string,
start: StartStep,
messages: WorkflowMessage[],
runId: string,
): Promise<{ content: string; meta: Record<string, unknown> } | null> {
const role = def.roles[nextRole];
if (!role) {
sendWorkflowError(runId, `Unknown role: ${next.role}`);
sendWorkflowError(runId, `Unknown role: ${nextRole}`);
return null;
}
let result: { content: string; meta: Record<string, unknown> };
try {
const event = await role.execute(next.prompt, ctx);
sendCommandEvent(runId, event);
return event;
result = await role(start, messages);
} catch (e: unknown) {
const errMsg = e instanceof Error ? e.message : String(e);
sendThreadEvent(runId, "failed", { error: errMsg });
return null;
}
if (!validateRoleResult(result, nextRole, runId)) return null;
return result;
}
async function runThread(
def: WorkflowDefinition,
workflowName: string,
def: WorkflowDefinition<RoleMeta>,
runId: string,
triggerPayload: unknown,
/** Pre-existing event history for crash-recovery resume. Empty for a fresh thread. */
resumeEvents: CommandEvent[] = [],
maxRounds: number,
resumeMessages: WorkflowMessage[] = [],
freshPrompt: string | null = null,
dryRun = false,
): Promise<void> {
const state: ThreadState = { runId, events: [] };
const ctx: WorkflowContext = {
const { start, messages: roleMessages } = initThreadMessages(
runId,
workflowName,
log: (msg) => sendThreadEvent(runId, "step_complete", { message: msg }),
};
resumeMessages,
freshPrompt,
maxRounds,
dryRun,
);
const initialEvent: CommandEvent = {
type: "thread_start",
triggerPayload:
triggerPayload != null && typeof triggerPayload === "object" ? triggerPayload : {},
};
const steps: Array<{
role: string;
meta: Record<string, unknown>;
content: string;
timestamp: number;
}> = [];
// On resume: replay persisted events, run the next un-executed role, then continue.
if (resumeEvents.length > 0) {
const nextEvent = await replayAndResume(def, runId, ctx, state, resumeEvents);
if (nextEvent === null) return;
await continueThread(def, runId, ctx, state, nextEvent);
// Rebuild steps from any resumed messages
for (const msg of roleMessages) {
steps.push({
role: msg.role,
meta: msg.meta as Record<string, unknown>,
content: msg.content,
timestamp: msg.timestamp,
});
}
let nextRole = def.moderator({ start, steps });
if (nextRole === END) {
sendThreadEvent(runId, "completed", null);
return;
}
// Fresh thread — send the initial command event and enter the loop.
sendCommandEvent(runId, initialEvent);
await continueThread(def, runId, ctx, state, initialEvent);
}
while (steps.length < maxRounds) {
const result = await executeRole(def, nextRole, start, roleMessages, runId);
if (result === null) return;
async function continueThread(
def: WorkflowDefinition,
runId: string,
ctx: WorkflowContext,
state: ThreadState,
firstEvent: CommandEvent,
): Promise<void> {
let event = firstEvent;
const message: WorkflowMessage = {
role: nextRole,
content: result.content,
meta: result.meta,
timestamp: Date.now(),
};
roleMessages.push(message);
sendWorkflowMessage(runId, message);
const MAX_STEPS = 1000;
let step = 0;
while (step < MAX_STEPS) {
step++;
state.events.push(event);
const next = def.moderate(state, event);
steps.push({
role: nextRole,
meta: result.meta,
content: result.content,
timestamp: message.timestamp,
});
if (next === null) {
nextRole = def.moderator({ start, steps });
if (nextRole === END) {
sendThreadEvent(runId, "completed", null);
return;
}
const role = def.roles[next.role];
if (!role) {
sendWorkflowError(runId, `Unknown role: ${next.role}`);
return;
}
try {
event = await role.execute(next.prompt, ctx);
} catch (e: unknown) {
const errMsg = e instanceof Error ? e.message : String(e);
sendThreadEvent(runId, "failed", { error: errMsg });
return;
}
sendCommandEvent(runId, event);
}
if (step >= MAX_STEPS) {
sendWorkflowError(runId, `Thread exceeded maximum steps (${MAX_STEPS})`);
}
sendWorkflowError(runId, `Thread exceeded maximum rounds (${maxRounds})`);
}
// ---------------------------------------------------------------------------
// Workflow definition loader
// ---------------------------------------------------------------------------
function isWorkflowDefinitionShape(def: unknown): def is WorkflowDefinition<RoleMeta> {
if (!isPlainRecord(def)) return false;
return (
typeof def.moderator === "function" &&
typeof def.roles === "object" &&
def.roles !== null &&
!Array.isArray(def.roles) &&
typeof def.name === "string"
);
}
async function loadWorkflowDefinition(
nerveRoot: string,
workflowName: string,
): Promise<WorkflowDefinition> {
): Promise<WorkflowDefinition<RoleMeta>> {
const candidates = [
resolve(join(nerveRoot, "workflows", workflowName, "index.ts")),
resolve(join(nerveRoot, "workflows", workflowName, "index.js")),
@@ -202,18 +292,13 @@ async function loadWorkflowDefinition(
const mod = await import(indexPath);
const def: unknown = mod.default ?? mod;
if (
def === null ||
typeof def !== "object" ||
typeof (def as WorkflowDefinition).moderate !== "function" ||
typeof (def as WorkflowDefinition).roles !== "object"
) {
if (!isWorkflowDefinitionShape(def)) {
throw new Error(
`Workflow "${workflowName}" must export a WorkflowDefinition with "roles" and "moderate".`,
`Workflow "${workflowName}" must export a WorkflowDefinition with "name", "roles", and "moderator".`,
);
}
return def as WorkflowDefinition;
return def;
}
// ---------------------------------------------------------------------------
@@ -222,8 +307,7 @@ async function loadWorkflowDefinition(
function handleMessage(
raw: unknown,
def: WorkflowDefinition,
workflowName: string,
def: WorkflowDefinition<RoleMeta>,
inFlight: Map<string, Promise<void>>,
shuttingDown: { value: boolean },
): void {
@@ -246,11 +330,11 @@ function handleMessage(
if (msg.type === "start-thread") {
if (shuttingDown.value) return;
const { runId, triggerPayload } = msg;
const { runId, prompt, maxRounds, dryRun } = msg;
const previous = inFlight.get(runId) ?? Promise.resolve();
const next = previous
.then(() => runThread(def, workflowName, runId, triggerPayload))
.then(() => runThread(def, runId, maxRounds, [], prompt, dryRun))
.catch((e: unknown) => {
const errMsg = e instanceof Error ? e.message : String(e);
sendWorkflowError(runId, errMsg);
@@ -265,11 +349,11 @@ function handleMessage(
if (msg.type === "resume-thread") {
if (shuttingDown.value) return;
const { runId, events, triggerPayload } = msg;
const { runId, messages, maxRounds, dryRun } = msg;
const previous = inFlight.get(runId) ?? Promise.resolve();
const next = previous
.then(() => runThread(def, workflowName, runId, triggerPayload, events as CommandEvent[]))
.then(() => runThread(def, runId, maxRounds, messages, null, dryRun))
.catch((e: unknown) => {
const errMsg = e instanceof Error ? e.message : String(e);
sendWorkflowError(runId, errMsg);
@@ -288,7 +372,7 @@ function handleMessage(
// ---------------------------------------------------------------------------
async function bootstrap(nerveRoot: string, workflowName: string): Promise<void> {
let def: WorkflowDefinition;
let def: WorkflowDefinition<RoleMeta>;
try {
def = await loadWorkflowDefinition(nerveRoot, workflowName);
} catch (e: unknown) {
@@ -303,7 +387,7 @@ async function bootstrap(nerveRoot: string, workflowName: string): Promise<void>
sendReady();
process.on("message", (raw: unknown) => {
handleMessage(raw, def, workflowName, inFlight, shuttingDown);
handleMessage(raw, def, inFlight, shuttingDown);
});
}
-8
View File
@@ -1,8 +0,0 @@
import { defineConfig } from "tsup";
export default defineConfig({
entry: ["src/index.ts", "src/sense-worker.ts"],
format: ["esm"],
dts: true,
clean: true,
});
+48
View File
@@ -0,0 +1,48 @@
# @uncaged/nerve-store
Persistent storage for the [nerve](../../README.md) daemon — append-only structured logs, optional JSONL cold archive, and content-addressable blobs.
## LogStore (`createLogStore`, `log-store.ts`)
- **Append-only log table** — rows with `source`, `type`, `refId`, `payload`, `ts` (string payloads for ad hoc fields)
- **SQLite WAL**`DatabaseSync` from `node:sqlite`
- **Workflow run tracking** — materialized `workflow_runs` table plus helpers to list active runs, upsert status transitions, and read **thread messages** / **role rounds** for CLI and crash recovery
- **Meta key-value** — small `meta` table (e.g. archive watermarks)
Public exports include `LogStore`, `LogEntry`, `LogQuery`, `WorkflowRun`, `WorkflowRunStatus`, `ThreadRoundRow`, `GetThreadRoundsParams`, and archive-related types re-exported from `log-archive`.
## WorkflowRunStatus
Runs progress through a small state machine. Typical paths:
1. **`queued`** → **`started`** when a worker picks up the thread
2. **`started`** → **`completed`** | **`failed`** | **`crashed`** | **`interrupted`** | **`dropped`**
Semantics in the daemon/store layer:
- **`completed` / `failed`** — normal terminal outcomes from the workflow worker
- **`crashed`** — worker exited unexpectedly; manager may respawn and **`resume-thread`** eligible **`started`** runs
- **`interrupted`** — e.g. hot-reload drain killed an in-flight thread after timeout
- **`dropped`** — concurrency **`overflow: drop`** rejected a new run, or **`overflow: queue`** evicted an queued item when the queue was full
## LogArchive (`log-archive.ts`)
- **`archiveLogs`** / helpers — export eligible UTC days of old rows to **`data/archive/logs/YYYY-MM-DD.jsonl`**, delete archived rows from SQLite, optional **`VACUUM`**
- Used by **`nerve store archive`** in `@uncaged/nerve-cli`
## BlobStore (`createBlobStore`, `blob-store.ts`)
- **Content-addressable storage**`write` returns lowercase **sha256** hex; files live under **`data/blobs/<2-hex>/<62-hex>`**
- **`read` / `exists`** — path must match digest on disk (tamper detection)
## Install
```bash
pnpm add @uncaged/nerve-store
```
Requires Node.js ≥ 22.5 (same as the rest of the stack).
## License
MIT
+24
View File
@@ -0,0 +1,24 @@
{
"name": "@uncaged/nerve-store",
"version": "0.4.0",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": ["dist"],
"publishConfig": {
"access": "public"
},
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "rslib build",
"test": "vitest run"
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*"
},
"devDependencies": {
"@rslib/core": "^0.21.3",
"@types/node": "^22.0.0",
"vitest": "^4.1.5"
}
}
+19
View File
@@ -0,0 +1,19 @@
import { defineConfig } from "@rslib/core";
export default defineConfig({
lib: [
{
format: "esm",
dts: true,
},
],
source: {
entry: {
index: "src/index.ts",
},
},
output: {
target: "node",
cleanDistPath: true,
},
});
@@ -30,8 +30,8 @@ describe("LogStore — cold archive (RFC-001 §5.4)", () => {
it("exports one UTC day to JSONL, deletes rows, advances archived_up_to", () => {
const ts = Date.UTC(2026, 1, 1, 10, 0, 0);
store.append({ source: "system", type: "x", refId: null, payload: '{"a":1}', ts });
store.append({ source: "reflex", type: "y", refId: "z", payload: null, ts: ts + 1 });
store.append({ source: "system", type: "x", refId: null, payload: '{"a":1}', timestamp: ts });
store.append({ source: "reflex", type: "y", refId: "z", payload: null, timestamp: ts + 1 });
const now = nowForLastArchivableFeb1();
const result = store.archiveLogs({ now, retentionMs: 30 * DAY_MS });
@@ -61,7 +61,7 @@ describe("LogStore — cold archive (RFC-001 §5.4)", () => {
it("does nothing when all logs are inside the hot window", () => {
const now = Date.UTC(2026, 3, 23, 12, 0, 0);
const ts = now - 5 * DAY_MS;
store.append({ source: "system", type: "warm", refId: null, payload: null, ts });
store.append({ source: "system", type: "warm", refId: null, payload: null, timestamp: ts });
const r = store.archiveLogs({ now, retentionMs: 30 * DAY_MS });
expect(r.days).toHaveLength(0);
expect(store.query()).toHaveLength(1);
@@ -69,7 +69,7 @@ describe("LogStore — cold archive (RFC-001 §5.4)", () => {
it("second archive with same clock is a no-op (watermark already caught up)", () => {
const ts = Date.UTC(2026, 1, 1, 10, 0, 0);
store.append({ source: "system", type: "x", refId: null, payload: null, ts });
store.append({ source: "system", type: "x", refId: null, payload: null, timestamp: ts });
const now = nowForLastArchivableFeb1();
store.archiveLogs({ now, retentionMs: 30 * DAY_MS });
const path = join(tmpDir, "data", "archive", "logs", "2026-02-01.jsonl");
@@ -82,11 +82,11 @@ describe("LogStore — cold archive (RFC-001 §5.4)", () => {
it("overwrites JSONL when the same UTC day is archived again after watermark rewind", () => {
const ts = Date.UTC(2026, 1, 1, 10, 0, 0);
store.append({ source: "a", type: "1", refId: null, payload: null, ts });
store.append({ source: "a", type: "1", refId: null, payload: null, timestamp: ts });
const now = nowForLastArchivableFeb1();
store.archiveLogs({ now, retentionMs: 30 * DAY_MS });
store.setMeta(LOG_ARCHIVE_META_KEY, "2026-01-31");
store.append({ source: "b", type: "2", refId: null, payload: null, ts: ts + 100 });
store.append({ source: "b", type: "2", refId: null, payload: null, timestamp: ts + 100 });
store.archiveLogs({ now, retentionMs: 30 * DAY_MS });
const path = join(tmpDir, "data", "archive", "logs", "2026-02-01.jsonl");
@@ -98,8 +98,8 @@ describe("LogStore — cold archive (RFC-001 §5.4)", () => {
it("respects maxDays across invocations", () => {
const t1 = Date.UTC(2026, 1, 1, 10, 0, 0);
const t2 = Date.UTC(2026, 1, 2, 10, 0, 0);
store.append({ source: "system", type: "a", refId: null, payload: null, ts: t1 });
store.append({ source: "system", type: "b", refId: null, payload: null, ts: t2 });
store.append({ source: "system", type: "a", refId: null, payload: null, timestamp: t1 });
store.append({ source: "system", type: "b", refId: null, payload: null, timestamp: t2 });
const now = Date.UTC(2027, 0, 1, 12, 0, 0);
const r1 = store.archiveLogs({ now, retentionMs: 30 * DAY_MS, maxDays: 1 });
@@ -116,7 +116,7 @@ describe("LogStore — cold archive (RFC-001 §5.4)", () => {
it("starts from earliest log day when it is before watermark+1", () => {
store.setMeta(LOG_ARCHIVE_META_KEY, "2026-01-10");
const ts = Date.UTC(2026, 1, 1, 10, 0, 0);
store.append({ source: "x", type: "p", refId: null, payload: null, ts });
store.append({ source: "x", type: "p", refId: null, payload: null, timestamp: ts });
const result = store.archiveLogs({ now: nowForLastArchivableFeb1(), retentionMs: 30 * DAY_MS });
expect(result.days.map((d) => d.day)).toContain("2026-02-01");
});
@@ -128,7 +128,7 @@ describe("LogStore — cold archive (RFC-001 §5.4)", () => {
it("runs VACUUM when vacuum: true", () => {
const ts = Date.UTC(2026, 1, 1, 10, 0, 0);
store.append({ source: "system", type: "x", refId: null, payload: null, ts });
store.append({ source: "system", type: "x", refId: null, payload: null, timestamp: ts });
const r = store.archiveLogs({
now: nowForLastArchivableFeb1(),
retentionMs: 30 * DAY_MS,
@@ -39,9 +39,9 @@ describe("LogStore — crash recovery helpers (Phase 3)", () => {
type: "started",
refId: "run-1",
payload: JSON.stringify({ triggerPayload: payload }),
ts: 1000,
timestamp: 1000,
},
{ runId: "run-1", workflow: "my-wf", status: "started", ts: 1000 },
{ runId: "run-1", workflow: "my-wf", status: "started", timestamp: 1000 },
);
const result = store.getTriggerPayload("run-1");
@@ -55,9 +55,9 @@ describe("LogStore — crash recovery helpers (Phase 3)", () => {
type: "started",
refId: "run-2",
payload: null,
ts: 1000,
timestamp: 1000,
},
{ runId: "run-2", workflow: "my-wf", status: "started", ts: 1000 },
{ runId: "run-2", workflow: "my-wf", status: "started", timestamp: 1000 },
);
expect(store.getTriggerPayload("run-2")).toBeNull();
@@ -72,14 +72,14 @@ describe("LogStore — crash recovery helpers (Phase 3)", () => {
type: "started",
refId: "run-3",
payload: JSON.stringify({ triggerPayload: payloadA }),
ts: 100,
timestamp: 100,
});
store.append({
source: "workflow",
type: "started",
refId: "run-3",
payload: JSON.stringify({ triggerPayload: payloadB }),
ts: 200,
timestamp: 200,
});
const result = store.getTriggerPayload("run-3");
@@ -106,7 +106,7 @@ describe("LogStore — crash recovery helpers (Phase 3)", () => {
type: "thread_command_event",
refId: "run-4",
payload: JSON.stringify(event),
ts: Date.now(),
timestamp: Date.now(),
});
}
@@ -123,14 +123,14 @@ describe("LogStore — crash recovery helpers (Phase 3)", () => {
type: "thread_command_event",
refId: "run-5",
payload: null,
ts: 1000,
timestamp: 1000,
});
store.append({
source: "workflow",
type: "thread_command_event",
refId: "run-5",
payload: JSON.stringify({ type: "valid_event" }),
ts: 1001,
timestamp: 1001,
});
const result = store.getThreadEvents("run-5");
@@ -146,23 +146,23 @@ describe("LogStore — crash recovery helpers (Phase 3)", () => {
type: "started",
refId: "run-6",
payload: JSON.stringify({ triggerPayload: {} }),
ts: 1000,
timestamp: 1000,
},
{ runId: "run-6", workflow: "my-wf", status: "started", ts: 1000 },
{ runId: "run-6", workflow: "my-wf", status: "started", timestamp: 1000 },
);
store.append({
source: "workflow",
type: "thread_command_event",
refId: "run-6",
payload: JSON.stringify({ type: "step_one" }),
ts: 1001,
timestamp: 1001,
});
store.append({
source: "workflow",
type: "step_complete",
refId: "run-6",
payload: JSON.stringify({ message: "done step" }),
ts: 1002,
timestamp: 1002,
});
const result = store.getThreadEvents("run-6");
@@ -176,14 +176,14 @@ describe("LogStore — crash recovery helpers (Phase 3)", () => {
type: "thread_command_event",
refId: "run-7",
payload: JSON.stringify({ type: "event_for_7" }),
ts: 1000,
timestamp: 1000,
});
store.append({
source: "workflow",
type: "thread_command_event",
refId: "run-8",
payload: JSON.stringify({ type: "event_for_8" }),
ts: 1001,
timestamp: 1001,
});
const result7 = store.getThreadEvents("run-7");
@@ -195,4 +195,65 @@ describe("LogStore — crash recovery helpers (Phase 3)", () => {
expect(result8[0].type).toBe("event_for_8");
});
});
describe("getThreadRoundCount / getThreadRounds", () => {
it("excludes thread_start from rounds and assigns ROW_NUMBER in chronological order", () => {
store.append({
source: "workflow",
type: "thread_command_event",
refId: "run-tr",
payload: JSON.stringify({ type: "thread_start", triggerPayload: { x: 1 } }),
timestamp: 100,
});
store.append({
source: "workflow",
type: "thread_command_event",
refId: "run-tr",
payload: JSON.stringify({
type: "step_a",
role: "alpha",
content: "hello",
meta: 1,
}),
timestamp: 101,
});
store.append({
source: "workflow",
type: "thread_command_event",
refId: "run-tr",
payload: JSON.stringify({ type: "step_b", role: "beta", content: "world" }),
timestamp: 102,
});
expect(store.getThreadRoundCount("run-tr")).toBe(2);
const all = store.getThreadRounds("run-tr", { before: 0, limit: 50 });
expect(all).toHaveLength(2);
expect(all.map((r) => r.round)).toEqual([2, 1]);
expect(all[0].message.role).toBe("beta");
expect(all[1].message.role).toBe("alpha");
});
it("getThreadRounds respects exclusive before bound", () => {
for (let i = 0; i < 3; i++) {
store.append({
source: "workflow",
type: "thread_command_event",
refId: "run-b4",
payload: JSON.stringify({ type: `ev_${i}`, role: "r", content: String(i) }),
timestamp: 200 + i,
});
}
expect(store.getThreadRoundCount("run-b4")).toBe(3);
const page = store.getThreadRounds("run-b4", { before: 3, limit: 50 });
expect(page.map((r) => r.round)).toEqual([2, 1]);
});
it("returns empty when no role rounds for runId", () => {
expect(store.getThreadRoundCount("missing")).toBe(0);
expect(store.getThreadRounds("missing", { before: 0, limit: 10 })).toHaveLength(0);
});
});
});
@@ -30,11 +30,11 @@ describe("LogStore — workflow_runs", () => {
runId: "run-1",
workflow: "cleanup",
status: "started",
ts: 1000,
timestamp: 1000,
};
const entry = store.upsertWorkflowRun(
{ source: "workflow", type: "started", refId: "run-1", payload: null, ts: 1000 },
{ source: "workflow", type: "started", refId: "run-1", payload: null, timestamp: 1000 },
run,
);
@@ -47,23 +47,23 @@ describe("LogStore — workflow_runs", () => {
expect(stored?.runId).toBe("run-1");
expect(stored?.workflow).toBe("cleanup");
expect(stored?.status).toBe("started");
expect(stored?.ts).toBe(1000);
expect(stored?.timestamp).toBe(1000);
});
it("updates existing workflow_runs row on upsert (status transition)", () => {
store.upsertWorkflowRun(
{ source: "workflow", type: "started", refId: "run-2", payload: null, ts: 1000 },
{ runId: "run-2", workflow: "cleanup", status: "started", ts: 1000 },
{ source: "workflow", type: "started", refId: "run-2", payload: null, timestamp: 1000 },
{ runId: "run-2", workflow: "cleanup", status: "started", timestamp: 1000 },
);
store.upsertWorkflowRun(
{ source: "workflow", type: "completed", refId: "run-2", payload: null, ts: 2000 },
{ runId: "run-2", workflow: "cleanup", status: "completed", ts: 2000 },
{ source: "workflow", type: "completed", refId: "run-2", payload: null, timestamp: 2000 },
{ runId: "run-2", workflow: "cleanup", status: "completed", timestamp: 2000 },
);
const stored = store.getWorkflowRun("run-2");
expect(stored?.status).toBe("completed");
expect(stored?.ts).toBe(2000);
expect(stored?.timestamp).toBe(2000);
// Both log entries should be present (event sourcing)
const logs = store.query({ refId: "run-2" });
@@ -71,15 +71,15 @@ describe("LogStore — workflow_runs", () => {
});
it("the log entries act as source of truth for event history", () => {
for (const [type, status, ts] of [
for (const [type, status, timestamp] of [
["queued", "queued", 1000],
["started", "started", 1001],
["step_complete", "started", 1002],
["completed", "completed", 1005],
] as const) {
store.upsertWorkflowRun(
{ source: "workflow", type, refId: "run-3", payload: null, ts },
{ runId: "run-3", workflow: "cleanup", status, ts },
{ source: "workflow", type, refId: "run-3", payload: null, timestamp },
{ runId: "run-3", workflow: "cleanup", status, timestamp },
);
}
@@ -97,37 +97,37 @@ describe("LogStore — workflow_runs", () => {
it("returns the latest state after multiple upserts", () => {
store.upsertWorkflowRun(
{ source: "workflow", type: "queued", refId: "run-4", payload: null, ts: 100 },
{ runId: "run-4", workflow: "code-review", status: "queued", ts: 100 },
{ source: "workflow", type: "queued", refId: "run-4", payload: null, timestamp: 100 },
{ runId: "run-4", workflow: "code-review", status: "queued", timestamp: 100 },
);
store.upsertWorkflowRun(
{ source: "workflow", type: "started", refId: "run-4", payload: null, ts: 200 },
{ runId: "run-4", workflow: "code-review", status: "started", ts: 200 },
{ source: "workflow", type: "started", refId: "run-4", payload: null, timestamp: 200 },
{ runId: "run-4", workflow: "code-review", status: "started", timestamp: 200 },
);
const run = store.getWorkflowRun("run-4");
expect(run?.status).toBe("started");
expect(run?.ts).toBe(200);
expect(run?.timestamp).toBe(200);
});
});
describe("getActiveWorkflowRuns", () => {
beforeEach(() => {
store.upsertWorkflowRun(
{ source: "workflow", type: "queued", refId: "r1", payload: null, ts: 100 },
{ runId: "r1", workflow: "cleanup", status: "queued", ts: 100 },
{ source: "workflow", type: "queued", refId: "r1", payload: null, timestamp: 100 },
{ runId: "r1", workflow: "cleanup", status: "queued", timestamp: 100 },
);
store.upsertWorkflowRun(
{ source: "workflow", type: "started", refId: "r2", payload: null, ts: 200 },
{ runId: "r2", workflow: "cleanup", status: "started", ts: 200 },
{ source: "workflow", type: "started", refId: "r2", payload: null, timestamp: 200 },
{ runId: "r2", workflow: "cleanup", status: "started", timestamp: 200 },
);
store.upsertWorkflowRun(
{ source: "workflow", type: "completed", refId: "r3", payload: null, ts: 300 },
{ runId: "r3", workflow: "cleanup", status: "completed", ts: 300 },
{ source: "workflow", type: "completed", refId: "r3", payload: null, timestamp: 300 },
{ runId: "r3", workflow: "cleanup", status: "completed", timestamp: 300 },
);
store.upsertWorkflowRun(
{ source: "workflow", type: "failed", refId: "r4", payload: null, ts: 400 },
{ runId: "r4", workflow: "deploy", status: "queued", ts: 400 },
{ source: "workflow", type: "failed", refId: "r4", payload: null, timestamp: 400 },
{ runId: "r4", workflow: "deploy", status: "queued", timestamp: 400 },
);
});
@@ -164,9 +164,9 @@ describe("LogStore — workflow_runs", () => {
expect(store.getActiveWorkflowRuns("nonexistent")).toHaveLength(0);
});
it("returns runs ordered by ts ascending", () => {
it("returns runs ordered by timestamp ascending", () => {
const active = store.getActiveWorkflowRuns();
expect(active[0].ts).toBeLessThan(active[1].ts);
expect(active[0].timestamp).toBeLessThan(active[1].timestamp);
});
});
@@ -176,8 +176,8 @@ describe("LogStore — workflow_runs", () => {
(status) => {
const runId = `run-${status}`;
store.upsertWorkflowRun(
{ source: "workflow", type: status, refId: runId, payload: null, ts: 1 },
{ runId, workflow: "test", status, ts: 1 },
{ source: "workflow", type: status, refId: runId, payload: null, timestamp: 1 },
{ runId, workflow: "test", status, timestamp: 1 },
);
expect(store.getWorkflowRun(runId)?.status).toBe(status);
},
@@ -27,7 +27,7 @@ describe("LogStore", () => {
type: "start",
refId: null,
payload: null,
ts: 1000,
timestamp: 1000,
});
expect(entry.id).toBe(1);
@@ -41,28 +41,40 @@ describe("LogStore", () => {
type: "start",
refId: null,
payload: null,
ts: 1000,
timestamp: 1000,
});
const e2 = store.append({
source: "system",
type: "stop",
refId: null,
payload: null,
ts: 2000,
timestamp: 2000,
});
expect(e2.id).toBe((e1.id ?? 0) + 1);
});
it("returns all entries when queried with no filter", () => {
store.append({ source: "system", type: "start", refId: null, payload: null, ts: 1000 });
store.append({ source: "reflex", type: "run_start", refId: "cpu", payload: null, ts: 2000 });
store.append({
source: "system",
type: "start",
refId: null,
payload: null,
timestamp: 1000,
});
store.append({
source: "reflex",
type: "run_start",
refId: "cpu",
payload: null,
timestamp: 2000,
});
store.append({
source: "reflex",
type: "run_complete",
refId: "cpu",
payload: '{"v":42}',
ts: 3000,
timestamp: 3000,
});
const all = store.query();
@@ -72,23 +84,35 @@ describe("LogStore", () => {
describe("query filters", () => {
beforeEach(() => {
store.append({ source: "system", type: "start", refId: null, payload: null, ts: 1000 });
store.append({ source: "reflex", type: "run_start", refId: "cpu", payload: null, ts: 2000 });
store.append({
source: "system",
type: "start",
refId: null,
payload: null,
timestamp: 1000,
});
store.append({
source: "reflex",
type: "run_start",
refId: "cpu",
payload: null,
timestamp: 2000,
});
store.append({
source: "reflex",
type: "run_complete",
refId: "cpu",
payload: '{"v":42}',
ts: 3000,
timestamp: 3000,
});
store.append({
source: "system",
type: "error",
refId: "disk",
payload: '{"error":"fail"}',
ts: 4000,
timestamp: 4000,
});
store.append({ source: "system", type: "stop", refId: null, payload: null, ts: 5000 });
store.append({ source: "system", type: "stop", refId: null, payload: null, timestamp: 5000 });
});
it("filters by source", () => {
@@ -111,7 +135,7 @@ describe("LogStore", () => {
it("filters by since (inclusive)", () => {
const results = store.query({ since: 3000 });
expect(results).toHaveLength(3);
expect(results[0].ts).toBe(3000);
expect(results[0].timestamp).toBe(3000);
});
it("filters by until (inclusive)", () => {
@@ -146,12 +170,24 @@ describe("LogStore", () => {
describe("query ordering", () => {
it("returns entries in insertion order (ascending id)", () => {
store.append({ source: "system", type: "start", refId: null, payload: null, ts: 5000 });
store.append({ source: "reflex", type: "run_start", refId: "a", payload: null, ts: 1000 });
store.append({
source: "system",
type: "start",
refId: null,
payload: null,
timestamp: 5000,
});
store.append({
source: "reflex",
type: "run_start",
refId: "a",
payload: null,
timestamp: 1000,
});
const all = store.query();
expect(all[0].ts).toBe(5000);
expect(all[1].ts).toBe(1000);
expect(all[0].timestamp).toBe(5000);
expect(all[1].timestamp).toBe(1000);
});
});
@@ -182,7 +218,7 @@ describe("LogStore", () => {
describe("append-only semantics", () => {
it("ids are always increasing", () => {
const entries = Array.from({ length: 10 }, (_, i) =>
store.append({ source: "system", type: "test", refId: null, payload: null, ts: i }),
store.append({ source: "system", type: "test", refId: null, payload: null, timestamp: i }),
);
for (let i = 1; i < entries.length; i++) {
@@ -194,7 +230,13 @@ describe("LogStore", () => {
describe("payload JSON round-trip", () => {
it("preserves JSON payload", () => {
const payload = JSON.stringify({ cpu: 95, host: "node-1" });
store.append({ source: "reflex", type: "run_complete", refId: "cpu", payload, ts: 1000 });
store.append({
source: "reflex",
type: "run_complete",
refId: "cpu",
payload,
timestamp: 1000,
});
const results = store.query({ refId: "cpu" });
expect(results).toHaveLength(1);
@@ -206,7 +248,13 @@ describe("LogStore", () => {
it("creates nested directory structure for db path", () => {
const deepPath = join(tmpDir, "a", "b", "c", "test.db");
const deepStore = createLogStore(deepPath);
deepStore.append({ source: "system", type: "start", refId: null, payload: null, ts: 1000 });
deepStore.append({
source: "system",
type: "start",
refId: null,
payload: null,
timestamp: 1000,
});
expect(deepStore.query()).toHaveLength(1);
deepStore.close();
});
+16
View File
@@ -0,0 +1,16 @@
/**
* @uncaged/nerve-store append-only log storage, cold-archive helpers, CAS blob store.
*/
export * from "./blob-store.js";
export * from "./log-archive.js";
export { createLogStore } from "./log-store.js";
export type {
GetThreadRoundsParams,
LogEntry,
LogQuery,
LogStore,
ThreadRoundRow,
WorkflowRun,
WorkflowRunStatus,
} from "./log-store.js";
@@ -9,8 +9,9 @@
import { mkdirSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import Database from "better-sqlite3";
import type BetterSqlite3 from "better-sqlite3";
import { DatabaseSync, type StatementSync } from "node:sqlite";
import { isPlainRecord } from "@uncaged/nerve-core";
import {
DEFAULT_LOG_RETENTION_MS,
@@ -34,7 +35,7 @@ export type LogEntry = {
type: string;
refId: string | null;
payload: string | null;
ts: number;
timestamp: number;
};
export type LogQuery = {
@@ -69,11 +70,15 @@ const VALID_WORKFLOW_STATUSES = new Set<string>([
"interrupted",
]);
function isWorkflowRunStatus(value: string): value is WorkflowRunStatus {
return VALID_WORKFLOW_STATUSES.has(value);
}
function validateWorkflowRunStatus(status: string): WorkflowRunStatus {
if (!VALID_WORKFLOW_STATUSES.has(status)) {
if (!isWorkflowRunStatus(status)) {
throw new Error(`Invalid workflow run status from DB: "${status}"`);
}
return status as WorkflowRunStatus;
return status;
}
/** One row in the workflow_runs materialized table. */
@@ -81,7 +86,26 @@ export type WorkflowRun = {
runId: string;
workflow: string;
status: WorkflowRunStatus;
ts: number;
timestamp: number;
};
/** One role-produced workflow-message row with 1-based round index (ROW_NUMBER over role messages only). */
export type ThreadRoundRow = {
round: number;
logId: number;
timestamp: number;
message: { role: string; content: string; meta: unknown; timestamp: number };
};
/** Parameters for {@link LogStore.getThreadRounds}. */
export type GetThreadRoundsParams = {
/**
* Exclusive upper bound on round index (1-based among role events).
* Use `0` to include all rounds (subject to `limit`).
*/
before: number;
/** Maximum rows returned from the DB (DESC by round). */
limit: number;
};
export type LogStore = {
@@ -107,7 +131,7 @@ export type LogStore = {
*/
getActiveWorkflowRuns: (workflowName?: string) => WorkflowRun[];
/**
* Get all workflow runs regardless of status, sorted by ts descending.
* Get all workflow runs regardless of status, sorted by timestamp descending.
* Optionally filter by workflow name.
*/
getAllWorkflowRuns: (workflowName: string | null) => WorkflowRun[];
@@ -118,9 +142,27 @@ export type LogStore = {
getTriggerPayload: (runId: string) => unknown;
/**
* Get all workflow CommandEvents for a specific run, ordered by id ASC.
* Used for crash recovery to rebuild ThreadState.
* @deprecated Use getThreadMessages for the new WorkflowMessage format.
*/
getThreadEvents: (runId: string) => Array<{ type: string; [key: string]: unknown }>;
/**
* Get all WorkflowMessages for a specific run, ordered by id ASC.
* Used for crash recovery to rebuild the message chain.
*/
getThreadMessages: (
runId: string,
) => Array<{ role: string; content: string; meta: unknown; timestamp: number }>;
/**
* Count role command events for a run (excludes `thread_start`/`__start__` messages and invalid payloads).
* Round indices for {@link getThreadRounds} are 1..count in chronological order.
*/
getThreadRoundCount: (runId: string) => number;
/**
* Role rounds for agent-oriented retrieval: each row is one `thread_command_event` or
* `thread_workflow_message` whose JSON `type` is not `thread_start` and `role` is not `__start__`,
* with `round` from ROW_NUMBER() OVER (ORDER BY id ASC). No schema migration numbering is computed in SQL.
*/
getThreadRounds: (runId: string, params: GetThreadRoundsParams) => ThreadRoundRow[];
/**
* Export logs older than the retention window to `data/archive/logs/YYYY-MM-DD.jsonl`,
* then delete those rows and advance `meta.archived_up_to` in one transaction per day
@@ -132,16 +174,16 @@ export type LogStore = {
const SCHEMA_SQL = `
CREATE TABLE IF NOT EXISTS logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source TEXT NOT NULL,
type TEXT NOT NULL,
ref_id TEXT,
payload TEXT,
ts INTEGER NOT NULL
id INTEGER PRIMARY KEY AUTOINCREMENT,
source TEXT NOT NULL,
type TEXT NOT NULL,
ref_id TEXT,
payload TEXT,
timestamp INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_logs_source_type ON logs(source, type);
CREATE INDEX IF NOT EXISTS idx_logs_ts ON logs(ts);
CREATE INDEX IF NOT EXISTS idx_logs_timestamp ON logs(timestamp);
CREATE INDEX IF NOT EXISTS idx_logs_ref_id ON logs(ref_id);
CREATE TABLE IF NOT EXISTS meta (
@@ -153,7 +195,7 @@ CREATE TABLE IF NOT EXISTS workflow_runs (
run_id TEXT PRIMARY KEY,
workflow TEXT NOT NULL,
status TEXT NOT NULL,
ts INTEGER NOT NULL
timestamp INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_workflow_runs_status ON workflow_runs(status);
@@ -166,7 +208,7 @@ type SqlLogRow = {
type: string;
ref_id: string | null;
payload: string | null;
ts: number;
timestamp: number;
};
function buildJsonlBody(rows: SqlLogRow[]): string {
@@ -178,13 +220,64 @@ function buildJsonlBody(rows: SqlLogRow[]): string {
type: r.type,
refId: r.ref_id,
payload: r.payload,
ts: r.ts,
timestamp: r.timestamp,
}),
);
return `${lines.join("\n")}\n`;
}
function runOptionalVacuum(sqlite: BetterSqlite3.Database, vacuum?: boolean): boolean {
function runInTransaction<T>(db: DatabaseSync, fn: () => T): T {
db.exec("BEGIN IMMEDIATE");
try {
const out = fn();
db.exec("COMMIT");
return out;
} catch (e) {
try {
db.exec("ROLLBACK");
} catch {
// ignore rollback errors
}
throw e;
}
}
function launchShapeFromRecord(rec: Record<string, unknown>): {
prompt: string;
maxRounds: number;
dryRun: boolean;
} | null {
if (typeof rec.prompt !== "string" || typeof rec.maxRounds !== "number") return null;
return {
prompt: rec.prompt,
maxRounds: rec.maxRounds,
dryRun: typeof rec.dryRun === "boolean" ? rec.dryRun : false,
};
}
/** Parse JSON from a workflow `started` log row into a trigger / launch payload for crash recovery. */
function triggerPayloadFromStartedLogJson(payload: string): unknown | null {
let parsed: unknown;
try {
parsed = JSON.parse(payload);
} catch {
return null;
}
if (!isPlainRecord(parsed)) return null;
const direct = launchShapeFromRecord(parsed);
if (direct !== null) return direct;
const inner = parsed.triggerPayload;
if (inner !== null && isPlainRecord(inner)) {
const fromInner = launchShapeFromRecord(inner);
if (fromInner !== null) return fromInner;
return inner;
}
return null;
}
function runOptionalVacuum(sqlite: DatabaseSync, vacuum?: boolean): boolean {
if (vacuum !== true) return false;
sqlite.exec("VACUUM");
return true;
@@ -199,7 +292,7 @@ function resolveArchiveStartDay(watermark: string | null, minDay: string): strin
function runArchiveDayLoop(
dbPath: string,
options: ArchiveLogsOptions,
selectLogsForDayStmt: BetterSqlite3.Statement,
selectLogsForDayStmt: StatementSync,
archiveDayTx: (day: string, start: number, endExclusive: number) => void,
startDay: string,
lastDay: string,
@@ -235,12 +328,12 @@ function runArchiveDayLoop(
export function createLogStore(dbPath: string): LogStore {
mkdirSync(dirname(dbPath), { recursive: true });
const sqlite: BetterSqlite3.Database = new Database(dbPath);
sqlite.pragma("journal_mode = WAL");
const sqlite = new DatabaseSync(dbPath);
sqlite.exec("PRAGMA journal_mode=WAL");
sqlite.exec(SCHEMA_SQL);
const insertStmt = sqlite.prepare(
"INSERT INTO logs (source, type, ref_id, payload, ts) VALUES (@source, @type, @refId, @payload, @ts)",
"INSERT INTO logs (source, type, ref_id, payload, timestamp) VALUES (@source, @type, @refId, @payload, @timestamp)",
);
const getMetaStmt = sqlite.prepare("SELECT value FROM meta WHERE key = ?");
@@ -249,11 +342,11 @@ export function createLogStore(dbPath: string): LogStore {
);
const upsertWorkflowRunStmt = sqlite.prepare(
"INSERT OR REPLACE INTO workflow_runs (run_id, workflow, status, ts) VALUES (@runId, @workflow, @status, @ts)",
"INSERT OR REPLACE INTO workflow_runs (run_id, workflow, status, timestamp) VALUES (@runId, @workflow, @status, @timestamp)",
);
const getWorkflowRunStmt = sqlite.prepare(
"SELECT run_id, workflow, status, ts FROM workflow_runs WHERE run_id = ?",
"SELECT run_id, workflow, status, timestamp FROM workflow_runs WHERE run_id = ?",
);
const getTriggerPayloadStmt = sqlite.prepare(
@@ -264,48 +357,76 @@ export function createLogStore(dbPath: string): LogStore {
"SELECT payload FROM logs WHERE source = 'workflow' AND type = 'thread_command_event' AND ref_id = ? ORDER BY id ASC",
);
const getThreadMessagesStmt = sqlite.prepare(
"SELECT payload FROM logs WHERE source = 'workflow' AND type = 'thread_workflow_message' AND ref_id = ? ORDER BY id ASC",
);
const getThreadRoundCountStmt = sqlite.prepare(
`SELECT COUNT(*) AS c FROM logs
WHERE source = 'workflow' AND type IN ('thread_command_event', 'thread_workflow_message') AND ref_id = ?
AND payload IS NOT NULL AND json_valid(payload) = 1
AND COALESCE(json_extract(payload, '$.type'), '') != 'thread_start'
AND COALESCE(json_extract(payload, '$.role'), '') != '__start__'`,
);
const getThreadRoundsStmt = sqlite.prepare(
`WITH numbered AS (
SELECT id, timestamp, payload,
ROW_NUMBER() OVER (ORDER BY id ASC) AS rn
FROM logs
WHERE source = 'workflow' AND type IN ('thread_command_event', 'thread_workflow_message') AND ref_id = @runId
AND payload IS NOT NULL AND json_valid(payload) = 1
AND COALESCE(json_extract(payload, '$.type'), '') != 'thread_start'
AND COALESCE(json_extract(payload, '$.role'), '') != '__start__'
)
SELECT id, timestamp, payload, rn FROM numbered
WHERE (@before = 0 OR rn < @before)
ORDER BY rn DESC
LIMIT @lim`,
);
const getActiveWorkflowRunsStmt = sqlite.prepare(
"SELECT run_id, workflow, status, ts FROM workflow_runs WHERE status IN ('queued', 'started') ORDER BY ts ASC",
"SELECT run_id, workflow, status, timestamp FROM workflow_runs WHERE status IN ('queued', 'started') ORDER BY timestamp ASC",
);
const getActiveWorkflowRunsByNameStmt = sqlite.prepare(
"SELECT run_id, workflow, status, ts FROM workflow_runs WHERE status IN ('queued', 'started') AND workflow = ? ORDER BY ts ASC",
"SELECT run_id, workflow, status, timestamp FROM workflow_runs WHERE status IN ('queued', 'started') AND workflow = ? ORDER BY timestamp ASC",
);
const getAllWorkflowRunsStmt = sqlite.prepare(
"SELECT run_id, workflow, status, ts FROM workflow_runs ORDER BY ts DESC",
"SELECT run_id, workflow, status, timestamp FROM workflow_runs ORDER BY timestamp DESC",
);
const getAllWorkflowRunsByNameStmt = sqlite.prepare(
"SELECT run_id, workflow, status, ts FROM workflow_runs WHERE workflow = ? ORDER BY ts DESC",
"SELECT run_id, workflow, status, timestamp FROM workflow_runs WHERE workflow = ? ORDER BY timestamp DESC",
);
const minLogTsStmt = sqlite.prepare("SELECT MIN(ts) AS m FROM logs");
const minLogTsStmt = sqlite.prepare("SELECT MIN(timestamp) AS m FROM logs");
const selectLogsForDayStmt = sqlite.prepare(
"SELECT id, source, type, ref_id, payload, ts FROM logs WHERE ts >= @start AND ts < @endExclusive ORDER BY id ASC",
"SELECT id, source, type, ref_id, payload, timestamp FROM logs WHERE timestamp >= @start AND timestamp < @endExclusive ORDER BY id ASC",
);
const deleteLogsForDayStmt = sqlite.prepare(
"DELETE FROM logs WHERE ts >= @start AND ts < @endExclusive",
"DELETE FROM logs WHERE timestamp >= @start AND timestamp < @endExclusive",
);
const upsertWorkflowRunTx = sqlite.transaction(
(entry: Omit<LogEntry, "id">, run: WorkflowRun) => {
function upsertWorkflowRunTx(entry: Omit<LogEntry, "id">, run: WorkflowRun): LogEntry {
return runInTransaction(sqlite, () => {
const info = insertStmt.run({
source: entry.source,
type: entry.type,
refId: entry.refId,
payload: entry.payload,
ts: entry.ts,
timestamp: entry.timestamp,
});
upsertWorkflowRunStmt.run({
runId: run.runId,
workflow: run.workflow,
status: run.status,
ts: run.ts,
timestamp: run.timestamp,
});
return { ...entry, id: Number(info.lastInsertRowid) };
},
);
});
}
function append(entry: Omit<LogEntry, "id">): LogEntry {
const info = insertStmt.run({
@@ -313,14 +434,14 @@ export function createLogStore(dbPath: string): LogStore {
type: entry.type,
refId: entry.refId,
payload: entry.payload,
ts: entry.ts,
timestamp: entry.timestamp,
});
return { ...entry, id: Number(info.lastInsertRowid) };
}
function query(filter: LogQuery = {}): LogEntry[] {
const conditions: string[] = [];
const params: Record<string, unknown> = {};
const params: Record<string, string | number> = {};
if (filter.source !== undefined) {
conditions.push("source = @source");
@@ -335,17 +456,17 @@ export function createLogStore(dbPath: string): LogStore {
params.refId = filter.refId;
}
if (filter.since !== undefined) {
conditions.push("ts >= @since");
conditions.push("timestamp >= @since");
params.since = filter.since;
}
if (filter.until !== undefined) {
conditions.push("ts <= @until");
conditions.push("timestamp <= @until");
params.until = filter.until;
}
const where = conditions.length > 0 ? `WHERE ${conditions.join(" AND ")}` : "";
const limit = filter.limit !== undefined ? `LIMIT ${filter.limit}` : "";
const sql = `SELECT id, source, type, ref_id, payload, ts FROM logs ${where} ORDER BY id ASC ${limit}`;
const sql = `SELECT id, source, type, ref_id, payload, timestamp FROM logs ${where} ORDER BY id ASC ${limit}`;
const rows = sqlite.prepare(sql).all(params) as Array<{
id: number;
@@ -353,7 +474,7 @@ export function createLogStore(dbPath: string): LogStore {
type: string;
ref_id: string | null;
payload: string | null;
ts: number;
timestamp: number;
}>;
return rows.map((r) => ({
@@ -362,7 +483,7 @@ export function createLogStore(dbPath: string): LogStore {
type: r.type,
refId: r.ref_id,
payload: r.payload,
ts: r.ts,
timestamp: r.timestamp,
}));
}
@@ -376,23 +497,23 @@ export function createLogStore(dbPath: string): LogStore {
}
function upsertWorkflowRun(entry: Omit<LogEntry, "id">, run: WorkflowRun): LogEntry {
return upsertWorkflowRunTx(entry, run) as LogEntry;
return upsertWorkflowRunTx(entry, run);
}
function appendWithWorkflowUpdate(entry: Omit<LogEntry, "id">, run: WorkflowRun): LogEntry {
return upsertWorkflowRunTx(entry, run) as LogEntry;
return upsertWorkflowRunTx(entry, run);
}
function getWorkflowRun(runId: string): WorkflowRun | null {
const row = getWorkflowRunStmt.get(runId) as
| { run_id: string; workflow: string; status: string; ts: number }
| { run_id: string; workflow: string; status: string; timestamp: number }
| undefined;
if (row === undefined) return null;
return {
runId: row.run_id,
workflow: row.workflow,
status: validateWorkflowRunStatus(row.status),
ts: row.ts,
timestamp: row.timestamp,
};
}
@@ -401,12 +522,12 @@ export function createLogStore(dbPath: string): LogStore {
workflowName !== undefined
? getActiveWorkflowRunsByNameStmt.all(workflowName)
: getActiveWorkflowRunsStmt.all()
) as Array<{ run_id: string; workflow: string; status: string; ts: number }>;
) as Array<{ run_id: string; workflow: string; status: string; timestamp: number }>;
return rows.map((r) => ({
runId: r.run_id,
workflow: r.workflow,
status: validateWorkflowRunStatus(r.status),
ts: r.ts,
timestamp: r.timestamp,
}));
}
@@ -415,28 +536,19 @@ export function createLogStore(dbPath: string): LogStore {
workflowName !== null
? getAllWorkflowRunsByNameStmt.all(workflowName)
: getAllWorkflowRunsStmt.all()
) as Array<{ run_id: string; workflow: string; status: string; ts: number }>;
) as Array<{ run_id: string; workflow: string; status: string; timestamp: number }>;
return rows.map((r) => ({
runId: r.run_id,
workflow: r.workflow,
status: validateWorkflowRunStatus(r.status),
ts: r.ts,
timestamp: r.timestamp,
}));
}
function getTriggerPayload(runId: string): unknown {
const row = getTriggerPayloadStmt.get(runId) as { payload: string | null } | undefined;
if (row === undefined || row.payload === null) return null;
try {
const parsed = JSON.parse(row.payload) as unknown;
if (parsed !== null && typeof parsed === "object") {
const obj = parsed as Record<string, unknown>;
return obj.triggerPayload ?? null;
}
} catch {
// malformed
}
return null;
return triggerPayloadFromStartedLogJson(row.payload);
}
function getThreadEvents(runId: string): Array<{ type: string; [key: string]: unknown }> {
@@ -445,12 +557,8 @@ export function createLogStore(dbPath: string): LogStore {
for (const row of rows) {
if (row.payload === null) continue;
try {
const parsed = JSON.parse(row.payload) as unknown;
if (
parsed !== null &&
typeof parsed === "object" &&
typeof (parsed as Record<string, unknown>).type === "string"
) {
const parsed: unknown = JSON.parse(row.payload);
if (isPlainRecord(parsed) && typeof parsed.type === "string") {
result.push(parsed as { type: string; [key: string]: unknown });
}
} catch {
@@ -460,10 +568,107 @@ export function createLogStore(dbPath: string): LogStore {
return result;
}
const archiveDayTx = sqlite.transaction((day: string, start: number, endExclusive: number) => {
deleteLogsForDayStmt.run({ start, endExclusive });
setMetaStmt.run({ key: LOG_ARCHIVE_META_KEY, value: day });
});
function tryParseWorkflowMessage(
payload: string,
): { role: string; content: string; meta: unknown; timestamp: number } | null {
try {
const parsed: unknown = JSON.parse(payload);
if (!isPlainRecord(parsed)) return null;
const obj = parsed;
if (typeof obj.role !== "string" || typeof obj.content !== "string") return null;
return {
role: obj.role,
content: obj.content,
meta: obj.meta,
timestamp: typeof obj.timestamp === "number" ? obj.timestamp : 0,
};
} catch {
return null;
}
}
function getThreadMessages(
runId: string,
): Array<{ role: string; content: string; meta: unknown; timestamp: number }> {
const rows = getThreadMessagesStmt.all(runId) as Array<{ payload: string | null }>;
const result: Array<{ role: string; content: string; meta: unknown; timestamp: number }> = [];
for (const row of rows) {
if (row.payload === null) continue;
const msg = tryParseWorkflowMessage(row.payload);
if (msg !== null) result.push(msg);
}
return result;
}
function getThreadRoundCount(runId: string): number {
const row = getThreadRoundCountStmt.get(runId) as { c: number } | undefined;
const c = row?.c;
if (c === null || c === undefined) return 0;
return Number(c);
}
function recordToRoundMessage(
obj: Record<string, unknown>,
fallbackTs: number,
): { role: string; content: string; meta: unknown; timestamp: number } | null {
if (typeof obj.role === "string" && typeof obj.content === "string") {
return {
role: obj.role,
content: obj.content,
meta: obj.meta,
timestamp: typeof obj.timestamp === "number" ? obj.timestamp : 0,
};
}
if (typeof obj.type === "string") {
return {
role: typeof obj.role === "string" ? obj.role : obj.type,
content: typeof obj.content === "string" ? obj.content : JSON.stringify(obj),
meta: obj,
timestamp: fallbackTs,
};
}
return null;
}
function parseRoundPayload(
payload: string,
fallbackTs: number,
): { role: string; content: string; meta: unknown; timestamp: number } | null {
try {
const parsed: unknown = JSON.parse(payload);
if (!isPlainRecord(parsed)) return null;
return recordToRoundMessage(parsed, fallbackTs);
} catch {
return null;
}
}
function getThreadRounds(runId: string, params: GetThreadRoundsParams): ThreadRoundRow[] {
if (params.limit < 1) return [];
const rows = getThreadRoundsStmt.all({
runId,
before: params.before,
lim: params.limit,
}) as Array<{ id: number; timestamp: number; payload: string | null; rn: number }>;
const out: ThreadRoundRow[] = [];
for (const row of rows) {
if (row.payload === null) continue;
const message = parseRoundPayload(row.payload, row.timestamp);
if (message !== null) {
out.push({ round: row.rn, logId: row.id, timestamp: row.timestamp, message });
}
}
return out;
}
function archiveDayTx(day: string, start: number, endExclusive: number): void {
runInTransaction(sqlite, () => {
deleteLogsForDayStmt.run({ start, endExclusive });
setMetaStmt.run({ key: LOG_ARCHIVE_META_KEY, value: day });
});
}
function readWatermark(): string | null {
const raw = getMeta(LOG_ARCHIVE_META_KEY);
@@ -522,6 +727,9 @@ export function createLogStore(dbPath: string): LogStore {
getAllWorkflowRuns,
getTriggerPayload,
getThreadEvents,
getThreadMessages,
getThreadRoundCount,
getThreadRounds,
archiveLogs,
close,
};
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src",
"composite": false
},
"include": ["src"]
}
+25
View File
@@ -0,0 +1,25 @@
{
"name": "@uncaged/nerve-workflow-utils",
"version": "0.4.0",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": ["dist"],
"publishConfig": {
"access": "public"
},
"scripts": {
"prepublishOnly": "bash ../../scripts/prepublish-check.sh",
"build": "rslib build",
"test": "vitest run"
},
"dependencies": {
"@uncaged/nerve-core": "workspace:*",
"zod": "^4.3.6"
},
"devDependencies": {
"@rslib/core": "^0.21.3",
"@types/node": "^22.0.0",
"vitest": "^4.1.5"
}
}
+19
View File
@@ -0,0 +1,19 @@
import { defineConfig } from "@rslib/core";
export default defineConfig({
lib: [
{
format: "esm",
dts: true,
},
],
source: {
entry: {
index: "src/index.ts",
},
},
output: {
target: "node",
cleanDistPath: true,
},
});
@@ -0,0 +1,130 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { z } from "zod";
import { llmExtract } from "../llm-extract.js";
describe("llmExtract", () => {
afterEach(() => {
vi.unstubAllGlobals();
vi.restoreAllMocks();
});
it("parses tool call arguments and validates with the zod schema", async () => {
const schema = z
.object({
name: z.string(),
description: z.string(),
})
.describe("Extract sense metadata from plan");
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
status: 200,
text: async () =>
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{
function: {
name: "extract",
arguments: JSON.stringify({ name: "cpu-usage", description: "CPU load" }),
},
},
],
},
},
],
}),
});
vi.stubGlobal("fetch", fetchMock);
const result = await llmExtract({
text: "some plan",
schema,
provider: {
baseUrl: "https://example.com/v1",
apiKey: "k",
model: "m",
},
});
expect(result.ok).toBe(true);
if (!result.ok) {
return;
}
expect(result.value).toEqual({ name: "cpu-usage", description: "CPU load" });
expect(fetchMock).toHaveBeenCalledTimes(1);
const [, init] = fetchMock.mock.calls[0] as [string, RequestInit];
expect(init.method).toBe("POST");
expect(init.headers).toMatchObject({
Authorization: "Bearer k",
"Content-Type": "application/json",
});
const body = JSON.parse(init.body as string) as {
model: string;
tool_choice: { function: { name: string } };
};
expect(body.model).toBe("m");
expect(body.tool_choice.function.name).toBeDefined();
});
it("returns schema_validation_failed when arguments do not match the schema", async () => {
const schema = z.object({ n: z.number() });
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue({
ok: true,
status: 200,
text: async () =>
JSON.stringify({
choices: [
{
message: {
tool_calls: [
{ function: { name: "extract", arguments: JSON.stringify({ n: "oops" }) } },
],
},
},
],
}),
}),
);
const result = await llmExtract({
text: "x",
schema,
provider: { baseUrl: "https://example.com", apiKey: "k", model: "m" },
});
expect(result.ok).toBe(false);
if (result.ok) {
return;
}
expect(result.error.kind).toBe("schema_validation_failed");
});
it("dryRun skips fetch and returns an empty stub value", async () => {
const fetchMock = vi.fn();
vi.stubGlobal("fetch", fetchMock);
const schema = z.object({ n: z.number() });
const result = await llmExtract({
text: "ignored",
schema,
provider: { baseUrl: "https://example.com", apiKey: "k", model: "m" },
dryRun: true,
});
expect(fetchMock).not.toHaveBeenCalled();
expect(result.ok).toBe(true);
if (!result.ok) {
return;
}
expect(result.value).toEqual({});
});
});
@@ -0,0 +1,59 @@
import { describe, expect, it } from "vitest";
import { spawnSafe } from "../spawn-safe.js";
describe("spawnSafe", () => {
it("passes argv literally without shell interpretation (injection-safe)", async () => {
const injection = "$(echo BAD)";
const result = await spawnSafe(
process.execPath,
["-e", "process.stdout.write(process.argv[1] ?? '')", injection],
{ cwd: null, env: null, timeoutMs: 10_000 },
);
expect(result.ok).toBe(true);
if (!result.ok) {
return;
}
expect(result.value.stdout).toBe(injection);
expect(result.value.exitCode).toBe(0);
});
it("returns err on non-zero exit", async () => {
const result = await spawnSafe(process.execPath, ["-e", "process.exit(7)"], {
cwd: null,
env: null,
timeoutMs: 10_000,
});
expect(result.ok).toBe(false);
if (result.ok) {
return;
}
expect(result.error.kind).toBe("non_zero_exit");
if (result.error.kind !== "non_zero_exit") {
return;
}
expect(result.error.exitCode).toBe(7);
});
it("dryRun skips spawn and returns a zero-exit stub", async () => {
const result = await spawnSafe(process.execPath, ["-e", "process.exit(1)"], {
cwd: null,
env: null,
timeoutMs: 10_000,
dryRun: true,
});
expect(result.ok).toBe(true);
if (!result.ok) {
return;
}
expect(result.value).toEqual({
stdout: "[dryRun] skipped",
stderr: "",
exitCode: 0,
signal: null,
});
});
});
+47
View File
@@ -0,0 +1,47 @@
import { readFileSync } from "node:fs";
import { resolve } from "node:path";
import { type Result, err, ok } from "@uncaged/nerve-core";
export type ReadNerveYamlOptions = {
nerveRoot: string;
};
export type NerveYamlError = {
code: "PATH_TRAVERSAL" | "READ_FAILED";
message: string;
};
/**
* Reads `nerve.yaml` from a Nerve data directory (typically `~/.uncaged-nerve`).
* Returns Result to avoid throwing on expected failures (missing file, bad perms).
* Validates that the resolved path stays within nerveRoot to prevent path traversal.
*/
export function readNerveYaml(options: ReadNerveYamlOptions): Result<string, NerveYamlError> {
const root = resolve(options.nerveRoot);
const target = resolve(root, "nerve.yaml");
if (!target.startsWith(root)) {
return err({
code: "PATH_TRAVERSAL",
message: `Resolved path "${target}" escapes nerveRoot "${root}"`,
});
}
try {
return ok(readFileSync(target, "utf-8"));
} catch (e) {
return err({
code: "READ_FAILED",
message: e instanceof Error ? e.message : String(e),
});
}
}
/**
* Shared context for workflow agents: how Nerve fits together and common CLI verbs.
*/
export const nerveAgentContext = `
Nerve observes the world through **Senses** (each has its own SQLite DB and a \`compute()\` function).
**Reflexes** (YAML) schedule sense runs or start **Workflows** on intervals or signals.
The \`nerve\` CLI manages config, triggers, and queries; keep paths and commands aligned with the host nerve.yaml and senses directory.
`.trim();
@@ -0,0 +1,61 @@
import { type Result, ok } from "@uncaged/nerve-core";
import { type SpawnEnv, type SpawnError, spawnSafe } from "./spawn-safe.js";
export type CursorAgentMode = "plan" | "ask" | "default";
export type CursorAgentOptions = {
prompt: string;
mode: CursorAgentMode;
cwd: string;
env: SpawnEnv | null;
timeoutMs: number | null;
dryRun: boolean;
};
type CursorAgentOptionsInput = CursorAgentOptions | Omit<CursorAgentOptions, "dryRun">;
function resolveCursorAgentDryRun(options: CursorAgentOptionsInput): boolean {
return "dryRun" in options ? options.dryRun : false;
}
/**
* Invokes `cursor-agent` with the prompt passed as a single argv slot (`shell: false`).
*/
export async function cursorAgent(
options: CursorAgentOptionsInput,
): Promise<Result<string, SpawnError>> {
const dryRun = resolveCursorAgentDryRun(options);
if (dryRun) {
return ok("[dryRun] skipped");
}
const args: string[] = [
"-p",
options.prompt,
"--model",
"auto",
"--output-format",
"text",
"--trust",
"--force",
];
if (options.mode === "plan") {
args.push("--mode=plan");
} else if (options.mode === "ask") {
args.push("--mode=ask");
}
const run = await spawnSafe("cursor-agent", args, {
cwd: options.cwd,
env: options.env,
timeoutMs: options.timeoutMs,
dryRun: false,
});
if (!run.ok) {
return run;
}
return ok(run.value.stdout);
}
+22
View File
@@ -0,0 +1,22 @@
export { cursorAgent, type CursorAgentMode, type CursorAgentOptions } from "./cursor-agent.js";
export {
nerveAgentContext,
readNerveYaml,
type NerveYamlError,
type ReadNerveYamlOptions,
} from "./context.js";
export {
llmExtract,
type LlmError,
type LlmExtractOptions,
type LlmProvider,
} from "./llm-extract.js";
export {
nerveCommandEnv,
spawnSafe,
type SpawnEnv,
type SpawnError,
type SpawnResult,
type SpawnSafeOptions,
} from "./spawn-safe.js";
export { isDryRun } from "./start-step.js";
+188
View File
@@ -0,0 +1,188 @@
import { type Result, err, ok } from "@uncaged/nerve-core";
import { toJSONSchema, type z } from "zod";
export type LlmProvider = {
baseUrl: string;
apiKey: string;
model: string;
};
export type LlmExtractOptions<T> = {
text: string;
schema: z.ZodType<T>;
provider: LlmProvider;
dryRun: boolean;
};
type LlmExtractOptionsInput<T> = LlmExtractOptions<T> | Omit<LlmExtractOptions<T>, "dryRun">;
function resolveLlmExtractDryRun<T>(options: LlmExtractOptionsInput<T>): boolean {
return "dryRun" in options ? options.dryRun : false;
}
export type LlmError =
| { kind: "http_error"; status: number; body: string }
| { kind: "invalid_response_json"; message: string }
| { kind: "no_tool_call"; preview: string }
| { kind: "tool_arguments_invalid_json"; message: string }
| { kind: "schema_validation_failed"; message: string }
| { kind: "network_error"; message: string };
function chatCompletionsUrl(baseUrl: string): string {
const trimmed = baseUrl.replace(/\/+$/, "");
return `${trimmed}/chat/completions`;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
function stripJsonSchemaMeta(json: Record<string, unknown>): Record<string, unknown> {
const { $schema: _drop, ...rest } = json;
return rest;
}
function readToolName(parametersSchema: Record<string, unknown>): string {
const title = parametersSchema.title;
if (typeof title === "string" && title.trim().length > 0) {
return title.trim();
}
return "extract";
}
function readToolArgumentsJson(parsed: unknown, previewSource: string): Result<string, LlmError> {
if (!isRecord(parsed)) {
return err({ kind: "invalid_response_json", message: "Top-level JSON is not an object" });
}
const choices = parsed.choices;
if (!Array.isArray(choices) || choices.length === 0) {
return err({ kind: "no_tool_call", preview: previewSource.slice(0, 500) });
}
const first = choices[0];
if (!isRecord(first)) {
return err({ kind: "no_tool_call", preview: previewSource.slice(0, 500) });
}
const messageObj = first.message;
if (!isRecord(messageObj)) {
return err({ kind: "no_tool_call", preview: previewSource.slice(0, 500) });
}
const toolCalls = messageObj.tool_calls;
if (!Array.isArray(toolCalls) || toolCalls.length === 0) {
return err({ kind: "no_tool_call", preview: previewSource.slice(0, 500) });
}
const call0 = toolCalls[0];
if (!isRecord(call0)) {
return err({ kind: "no_tool_call", preview: previewSource.slice(0, 500) });
}
const fn = call0.function;
if (!isRecord(fn)) {
return err({ kind: "no_tool_call", preview: previewSource.slice(0, 500) });
}
const argsRaw = fn.arguments;
if (typeof argsRaw !== "string") {
return err({ kind: "no_tool_call", preview: previewSource.slice(0, 500) });
}
return ok(argsRaw);
}
/**
* Calls an OpenAI-compatible chat completions API with `tool_choice` forced to a single function
* derived from a Zod v4 schema (`toJSONSchema`). Uses `fetch()` only (no shell).
*/
export async function llmExtract<T>(
options: LlmExtractOptionsInput<T>,
): Promise<Result<T, LlmError>> {
const dryRun = resolveLlmExtractDryRun(options);
if (dryRun) {
return ok({} as T);
}
const rawJsonSchema = toJSONSchema(options.schema) as Record<string, unknown>;
const parameters = stripJsonSchemaMeta(rawJsonSchema);
const toolName = readToolName(parameters);
const toolDescription =
typeof options.schema.description === "string" && options.schema.description.trim().length > 0
? options.schema.description.trim()
: "Extract structured data from the input text.";
const body = {
model: options.provider.model,
messages: [
{
role: "system" as const,
content: "Extract the requested information from the provided text. Be precise.",
},
{ role: "user" as const, content: options.text },
],
tools: [
{
type: "function" as const,
function: {
name: toolName,
description: toolDescription,
parameters,
},
},
],
tool_choice: { type: "function" as const, function: { name: toolName } },
};
let response: Response;
try {
response = await fetch(chatCompletionsUrl(options.provider.baseUrl), {
method: "POST",
headers: {
Authorization: `Bearer ${options.provider.apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
});
} catch (cause) {
const message = cause instanceof Error ? cause.message : String(cause);
return err({ kind: "network_error", message });
}
const responseText = await response.text();
if (!response.ok) {
return err({ kind: "http_error", status: response.status, body: responseText.slice(0, 4000) });
}
let parsed: unknown;
try {
parsed = JSON.parse(responseText) as unknown;
} catch (cause) {
const message = cause instanceof Error ? cause.message : String(cause);
return err({ kind: "invalid_response_json", message });
}
const argsJson = readToolArgumentsJson(parsed, responseText);
if (!argsJson.ok) {
return argsJson;
}
let argsParsed: unknown;
try {
argsParsed = JSON.parse(argsJson.value) as unknown;
} catch (cause) {
const message = cause instanceof Error ? cause.message : String(cause);
return err({ kind: "tool_arguments_invalid_json", message });
}
const validated = options.schema.safeParse(argsParsed);
if (!validated.success) {
return err({
kind: "schema_validation_failed",
message: validated.error.message,
});
}
return ok(validated.data);
}
+159
View File
@@ -0,0 +1,159 @@
import { spawn } from "node:child_process";
import { homedir } from "node:os";
import { join } from "node:path";
import { type Result, err, ok } from "@uncaged/nerve-core";
/** Compatible with `process.env` for `child_process.spawn`. */
export type SpawnEnv = Record<string, string | undefined>;
export type SpawnResult = {
stdout: string;
stderr: string;
exitCode: number;
/** OS signal name (e.g. `"SIGTERM"`) when terminated by signal; otherwise `null`. */
signal: string | null;
};
export type SpawnError =
| {
kind: "non_zero_exit";
stdout: string;
stderr: string;
exitCode: number;
signal: string | null;
}
| { kind: "timeout"; stdout: string; stderr: string }
| { kind: "spawn_failed"; message: string };
export type SpawnSafeOptions = {
cwd: string | null;
/** When null, merges {@link nerveCommandEnv} over `process.env`. When set, merges over that default. */
env: SpawnEnv | null;
timeoutMs: number | null;
dryRun: boolean;
};
type SpawnSafeOptionsInput = SpawnSafeOptions | Omit<SpawnSafeOptions, "dryRun">;
const DEFAULT_TIMEOUT_MS = 300_000;
/**
* PATH and PNPM_HOME for running `pnpm` and `nerve` from workflow roles.
* Uses the pnpm store home only (no npm user bin); binaries must resolve via PATH.
*/
export function nerveCommandEnv(): SpawnEnv {
const home = homedir();
const pnpmHome = join(home, ".local/share/pnpm");
return {
...process.env,
PNPM_HOME: pnpmHome,
PATH: `${pnpmHome}:${process.env.PATH ?? ""}`,
};
}
function mergeEnv(user: SpawnEnv | null): SpawnEnv {
const base = nerveCommandEnv();
if (user === null) {
return base;
}
return { ...base, ...user };
}
function resolveTimeout(timeoutMs: number | null): number {
if (timeoutMs === null) {
return DEFAULT_TIMEOUT_MS;
}
return timeoutMs;
}
function resolveDryRun(options: SpawnSafeOptionsInput): boolean {
return "dryRun" in options ? options.dryRun : false;
}
/**
* Spawn a process with `shell: false` (argv only), default {@link nerveCommandEnv}, and optional timeout.
* Returns `ok` only when the process exits with code 0.
*/
export function spawnSafe(
command: string,
args: ReadonlyArray<string>,
options: SpawnSafeOptionsInput,
): Promise<Result<SpawnResult, SpawnError>> {
const dryRun = resolveDryRun(options);
if (dryRun) {
return Promise.resolve(
ok({
stdout: "[dryRun] skipped",
stderr: "",
exitCode: 0,
signal: null,
}),
);
}
return new Promise((resolve) => {
const cwd = options.cwd === null ? process.cwd() : options.cwd;
const env = mergeEnv(options.env);
const timeoutMs = resolveTimeout(options.timeoutMs);
const child = spawn(command, args, {
cwd,
env,
shell: false,
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
let settled = false;
const finish = (outcome: Result<SpawnResult, SpawnError>) => {
if (settled) {
return;
}
settled = true;
clearTimeout(timer);
resolve(outcome);
};
const timer = setTimeout(() => {
child.kill("SIGTERM");
finish(err({ kind: "timeout", stdout, stderr }));
}, timeoutMs);
child.stdout?.on("data", (chunk: Buffer | string) => {
stdout += typeof chunk === "string" ? chunk : chunk.toString("utf-8");
});
child.stderr?.on("data", (chunk: Buffer | string) => {
stderr += typeof chunk === "string" ? chunk : chunk.toString("utf-8");
});
child.on("error", (cause: Error) => {
finish(err({ kind: "spawn_failed", message: cause.message }));
});
child.on("close", (code, signal) => {
const exitCode = code ?? 1;
const sig = signal === undefined || signal === null ? null : String(signal);
const result: SpawnResult = {
stdout: stdout.trimEnd(),
stderr: stderr.trimEnd(),
exitCode,
signal: sig,
};
if (exitCode !== 0) {
finish(
err({
kind: "non_zero_exit",
stdout: result.stdout,
stderr: result.stderr,
exitCode,
signal: sig,
}),
);
return;
}
finish(ok(result));
});
});
}
@@ -0,0 +1,6 @@
import type { StartStep } from "@uncaged/nerve-core";
/** Returns the thread-level dry-run flag from the workflow start frame. */
export function isDryRun(start: StartStep): boolean {
return start.meta.dryRun;
}
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src",
"composite": false
},
"include": ["src"]
}

Some files were not shown because too many files have changed in this diff Show More