Compare commits

...

8 Commits

Author SHA1 Message Date
xiaoju 6d94be34a9 feat: validate model connectivity during uwf setup
Send a test completion request after configuration to verify the model
is reachable. If validation fails, warn the user and suggest trying a
different model or checking their settings.

Fixes #335
2026-05-22 10:30:39 +00:00
xiaoju b9252b5ce2 fix: dynamic frontmatter instruction from role schema (closes #389) 2026-05-22 10:03:56 +00:00
xiaoju 4d47effd39 fix: generate frontmatter instruction dynamically from role schema
Replace hardcoded 5-field example with schema-driven generation.
Now shows actual enum values, types, and required markers for
each role's frontmatter schema.

Fixes #389

小橘 <xiaoju@shazhou.work>
2026-05-22 10:03:45 +00:00
xiaoju 7b93ce8f3e fix: dynamic frontmatter field extraction from role schema (closes #388) 2026-05-22 09:57:45 +00:00
xiaoju 67870392ab fix: dynamic frontmatter field extraction from role schema
Replace hardcoded 5-field candidate with schema-driven extraction.
Now reads outputSchema properties and picks matching fields from
parsed frontmatter, supporting role-specific fields like plan,
approved, success.

Falls back to standard 5 fields when schema has no properties.

Fixes #388

小橘 <xiaoju@shazhou.work>
2026-05-22 09:57:30 +00:00
xiaomo 6b9ff9781d Merge pull request 'fix: revert unnecessary output protocol changes from #385' (#386) from fix/385-revert-output-protocol into main 2026-05-22 09:40:33 +00:00
xiaoju 487c48effa fix: revert output protocol changes from #385
Agent CLI outputs plain CAS hash (not JSON), engine parses plain hash.
StepOutput no longer carries sessionId — session info is already in CAS detail.
Keeps the valuable parts of #385: sessionId in AgentRunResult (process-internal),
continue support, and frontmatter retry loop.
2026-05-22 09:39:36 +00:00
xiaomo 4eca2d533c Merge pull request 'feat: agent session protocol — sessionId, continue, frontmatter retry' (#385) from feat/384-agent-session-protocol into main 2026-05-22 09:20:35 +00:00
10 changed files with 714 additions and 87 deletions
+83
View File
@@ -0,0 +1,83 @@
# Test Spec: uwf setup model connectivity validation (#335)
## Context
File: `packages/cli-workflow/src/commands/setup.ts`
Test file: `packages/cli-workflow/src/__tests__/setup-validate.test.ts`
After `cmdSetup` writes config, it should send a test chat completion request to verify the configured model is reachable. If validation fails, warn the user (don't abort — config is already saved).
## Implementation Notes
- Add a `validateModel(baseUrl, apiKey, model)` function that sends a minimal chat completion request (`POST /chat/completions` with `messages: [{role:"user",content:"hi"}]`, `max_tokens: 1`)
- Returns `Result<void, string>` — ok if 2xx response, error with reason string otherwise
- Use `AbortSignal.timeout(15_000)` for the request
- Both `cmdSetup` and `cmdSetupInteractive` should call it after saving config
- `cmdSetup` returns validation result in its return object: `{ ...existing, validation: { ok: true } | { ok: false, error: string } }`
- `cmdSetupInteractive` prints a warning to console if validation fails, success message if it passes
- Use the project logger (`createLogger`) — no raw `console.log` except in interactive CLI output (per CLAUDE.md)
## Test Cases (vitest)
### 1. `validateModel` — success path
- Mock `fetch` to return `{ status: 200, ok: true, json: () => ({}) }`
- Call `validateModel(baseUrl, apiKey, model)`
- Assert returns `{ ok: true, value: undefined }`
- Assert fetch was called with correct URL (`${baseUrl}/chat/completions`), correct headers (`Authorization: Bearer ${apiKey}`), correct body (model, messages, max_tokens: 1)
### 2. `validateModel` — HTTP error (401 unauthorized)
- Mock `fetch` to return `{ status: 401, ok: false, statusText: "Unauthorized" }`
- Call `validateModel(baseUrl, apiKey, model)`
- Assert returns `{ ok: false, error: <string containing "401"> }`
### 3. `validateModel` — HTTP error (404 model not found)
- Mock `fetch` to return `{ status: 404, ok: false, statusText: "Not Found" }`
- Assert returns `{ ok: false, error: <string containing "404"> }`
### 4. `validateModel` — network timeout
- Mock `fetch` to throw `DOMException` with name `AbortError`
- Assert returns `{ ok: false, error: <string containing "timeout" or "unreachable"> }`
### 5. `validateModel` — network error (DNS failure, connection refused)
- Mock `fetch` to throw `TypeError("fetch failed")`
- Assert returns `{ ok: false, error: <string mentioning connectivity> }`
### 6. `cmdSetup` — includes validation result on success
- Mock global `fetch` for `/chat/completions` to succeed
- Call `cmdSetup({ provider, baseUrl, apiKey, model, storageRoot })`
- Assert returned object has `validation: { ok: true, value: undefined }`
- Assert config files are still written (existing behavior preserved)
### 7. `cmdSetup` — includes validation result on failure (config still saved)
- Mock global `fetch` for `/chat/completions` to return 401
- Call `cmdSetup({ ... })`
- Assert returned object has `validation: { ok: false, error: ... }`
- Assert `config.yaml` and `.env` are still written (validation failure doesn't prevent saving)
### 8. `cmdSetupInteractive` — prints success message on validation pass
- Mock `fetch` for both `/models` and `/chat/completions` to succeed
- Mock stdin to provide valid selections
- Capture console output
- Assert output contains a success message like "Model verified" or "✓"
### 9. `cmdSetupInteractive` — prints warning on validation failure
- Mock `fetch`: `/models` succeeds, `/chat/completions` returns 401
- Mock stdin for valid selections
- Capture console output
- Assert output contains a warning about model not being reachable and suggests trying a different model
### 10. `validateModel` — request body correctness
- Mock `fetch` to capture the request body
- Call `validateModel(baseUrl, apiKey, "test-model")`
- Assert body is `{ model: "test-model", messages: [{role: "user", content: "hi"}], max_tokens: 1 }`
## Export Requirements
- `validateModel` must be exported (for direct unit testing)
- Signature: `async function validateModel(baseUrl: string, apiKey: string, model: string): Promise<Result<void, string>>`
- `Result` type: `{ ok: true; value: T } | { ok: false; error: E }` (project convention)
## Files to Create/Modify
- **New**: `packages/cli-workflow/src/__tests__/setup-validate.test.ts` — all test cases above
- **Modify**: `packages/cli-workflow/src/commands/setup.ts` — add `validateModel`, integrate into `cmdSetup` and `cmdSetupInteractive`
@@ -0,0 +1,150 @@
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, beforeEach, describe, expect, test, vi } from "vitest";
import { cmdSetup, validateModel } from "../commands/setup.js";
describe("validateModel", () => {
const BASE_URL = "https://api.example.com/v1";
const API_KEY = "sk-test-key";
const MODEL = "test-model";
afterEach(() => {
vi.restoreAllMocks();
});
test("success path — returns ok on 200", async () => {
const mockFetch = vi
.spyOn(globalThis, "fetch")
.mockResolvedValue(new Response(JSON.stringify({}), { status: 200 }));
const result = await validateModel(BASE_URL, API_KEY, MODEL);
expect(result).toEqual({ ok: true, value: undefined });
expect(mockFetch).toHaveBeenCalledOnce();
const [url, opts] = mockFetch.mock.calls[0]!;
expect(url).toBe(`${BASE_URL}/chat/completions`);
expect((opts as RequestInit).headers).toEqual(
expect.objectContaining({ Authorization: `Bearer ${API_KEY}` }),
);
const body = JSON.parse((opts as RequestInit).body as string);
expect(body).toEqual({
model: MODEL,
messages: [{ role: "user", content: "hi" }],
max_tokens: 1,
});
});
test("HTTP 401 — returns error containing 401", async () => {
vi.spyOn(globalThis, "fetch").mockResolvedValue(
new Response("Unauthorized", { status: 401, statusText: "Unauthorized" }),
);
const result = await validateModel(BASE_URL, API_KEY, MODEL);
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.error).toContain("401");
}
});
test("HTTP 404 — returns error containing 404", async () => {
vi.spyOn(globalThis, "fetch").mockResolvedValue(
new Response("Not Found", { status: 404, statusText: "Not Found" }),
);
const result = await validateModel(BASE_URL, API_KEY, MODEL);
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.error).toContain("404");
}
});
test("network timeout — returns error mentioning timeout", async () => {
const err = new DOMException("signal timed out", "AbortError");
vi.spyOn(globalThis, "fetch").mockRejectedValue(err);
const result = await validateModel(BASE_URL, API_KEY, MODEL);
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.error.toLowerCase()).toMatch(/timeout|timed out/);
}
});
test("network error (DNS/connection) — returns error mentioning connectivity", async () => {
vi.spyOn(globalThis, "fetch").mockRejectedValue(new TypeError("fetch failed"));
const result = await validateModel(BASE_URL, API_KEY, MODEL);
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.error.toLowerCase()).toMatch(/connect|reach|network/);
}
});
test("request body correctness", async () => {
const mockFetch = vi
.spyOn(globalThis, "fetch")
.mockResolvedValue(new Response(JSON.stringify({}), { status: 200 }));
await validateModel(BASE_URL, API_KEY, "my-special-model");
const body = JSON.parse((mockFetch.mock.calls[0]![1] as RequestInit).body as string);
expect(body).toEqual({
model: "my-special-model",
messages: [{ role: "user", content: "hi" }],
max_tokens: 1,
});
});
});
describe("cmdSetup with validation", () => {
let storageRoot: string;
beforeEach(async () => {
storageRoot = await mkdtemp(join(tmpdir(), "uwf-setup-validate-"));
});
afterEach(async () => {
vi.restoreAllMocks();
await rm(storageRoot, { recursive: true, force: true });
});
const setupArgs = () => ({
provider: "testprovider",
baseUrl: "https://api.test.com/v1",
apiKey: "sk-test",
model: "test-model",
storageRoot,
});
test("includes validation result on success", async () => {
vi.spyOn(globalThis, "fetch").mockResolvedValue(
new Response(JSON.stringify({}), { status: 200 }),
);
const result = await cmdSetup(setupArgs());
expect(result.validation).toEqual({ ok: true, value: undefined });
// Config files should still be written
expect(result.configPath).toBeTruthy();
expect(result.envPath).toBeTruthy();
});
test("includes validation failure — config still saved", async () => {
vi.spyOn(globalThis, "fetch").mockResolvedValue(
new Response("Unauthorized", { status: 401, statusText: "Unauthorized" }),
);
const result = await cmdSetup(setupArgs());
expect(result.validation).toBeDefined();
expect((result.validation as { ok: boolean }).ok).toBe(false);
// Config files should still be written despite validation failure
expect(result.configPath).toBeTruthy();
expect(result.envPath).toBeTruthy();
});
});
+55 -2
View File
@@ -2,9 +2,45 @@ import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { stdin as input, stdout as output } from "node:process";
import { createInterface } from "node:readline/promises";
import type { Result } from "@uncaged/workflow-util";
import { parse, stringify } from "yaml";
/**
* Send a minimal chat completion request to verify the model is reachable.
* Returns ok on 2xx, error with reason string otherwise.
*/
export async function validateModel(
baseUrl: string,
apiKey: string,
model: string,
): Promise<Result<void, string>> {
try {
const url = `${baseUrl.replace(/\/+$/, "")}/chat/completions`;
const res = await fetch(url, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model,
messages: [{ role: "user", content: "hi" }],
max_tokens: 1,
}),
signal: AbortSignal.timeout(15_000),
});
if (!res.ok) {
return { ok: false, error: `HTTP ${res.status} ${res.statusText}` };
}
return { ok: true, value: undefined };
} catch (err: unknown) {
if (err instanceof DOMException && err.name === "AbortError") {
return { ok: false, error: "Request timed out — model endpoint unreachable" };
}
return { ok: false, error: `Network error — could not reach endpoint (${String(err)})` };
}
}
/**
* Preset provider list — embedded to avoid runtime YAML loading dependency.
* Keep in sync with providers.yaml in cli-workflow.
@@ -163,12 +199,16 @@ export async function cmdSetup(args: SetupArgs): Promise<Record<string, unknown>
envData[envName] = args.apiKey;
saveEnvFile(envPath, envData);
// Validate model connectivity
const validation = await validateModel(args.baseUrl, args.apiKey, args.model);
return {
configPath,
envPath,
provider: args.provider,
model: args.model,
defaultAgent: merged.defaultAgent,
validation,
};
}
@@ -328,7 +368,7 @@ export async function cmdSetupInteractive(storageRoot: string): Promise<Record<s
console.log(`${providerName}/${model}\n`);
await cmdSetup({
const setupResult = await cmdSetup({
provider: providerName,
baseUrl,
apiKey,
@@ -336,6 +376,19 @@ export async function cmdSetupInteractive(storageRoot: string): Promise<Record<s
storageRoot,
});
// Show validation result
if (setupResult.validation && typeof setupResult.validation === "object") {
const v = setupResult.validation as { ok: boolean; error?: string };
if (v.ok) {
console.log("✓ Model verified — connection successful.\n");
} else {
console.log(`\n⚠ Warning: Could not reach model — ${v.error}`);
console.log(
" Config saved, but you may want to try a different model or check your API key.\n",
);
}
}
console.log("Setup complete! Get started:\n");
console.log(" uwf workflow put <workflow.yaml> Register a workflow");
console.log(' uwf thread start <name> -p "..." Start a thread');
+4 -27
View File
@@ -200,7 +200,6 @@ export async function cmdThreadShow(storageRoot: string, threadId: ThreadId): Pr
thread: threadId,
head: activeHead,
done: false,
sessionId: null,
};
}
@@ -211,7 +210,6 @@ export async function cmdThreadShow(storageRoot: string, threadId: ThreadId): Pr
thread: threadId,
head: hist.head,
done: true,
sessionId: null,
};
}
@@ -626,12 +624,7 @@ function resolveAgentConfig(
return agentConfig;
}
type SpawnAgentResult = {
stepHash: CasRef;
sessionId: string | null;
};
function spawnAgent(agent: AgentConfig, threadId: ThreadId, role: string): SpawnAgentResult {
function spawnAgent(agent: AgentConfig, threadId: ThreadId, role: string): CasRef {
const argv = [...agent.args, threadId, role];
let stdout: string;
try {
@@ -653,24 +646,10 @@ function spawnAgent(agent: AgentConfig, threadId: ThreadId, role: string): Spawn
}
const line = stdout.trim().split("\n").pop()?.trim() ?? "";
// Try JSON output first (new protocol)
try {
const parsed = JSON.parse(line) as Record<string, unknown>;
const stepHash = parsed.stepHash;
const sessionId = parsed.sessionId;
if (typeof stepHash === "string" && isCasRef(stepHash) && typeof sessionId === "string") {
return { stepHash, sessionId };
}
} catch {
// Not JSON — fall through to legacy CAS hash parsing
}
// Legacy: plain CAS hash on stdout
if (!isCasRef(line)) {
fail(`agent stdout is not a valid CAS hash or JSON: ${line || "(empty)"}`);
fail(`agent stdout is not a valid CAS hash: ${line || "(empty)"}`);
}
return { stepHash: line, sessionId: null };
return line;
}
async function archiveThread(
@@ -719,7 +698,6 @@ export async function cmdThreadStep(
thread: threadId,
head: headHash,
done: true,
sessionId: null,
};
}
@@ -728,7 +706,7 @@ export async function cmdThreadStep(
const agent = resolveAgentConfig(config, workflow, role, agentOverride);
loadDotenv({ path: getEnvPath(storageRoot) });
const { stepHash: newHead, sessionId } = spawnAgent(agent, threadId, role);
const newHead = spawnAgent(agent, threadId, role);
// Re-create store to pick up nodes written by the agent subprocess
const uwfAfter = await createUwfStore(storageRoot);
@@ -759,7 +737,6 @@ export async function cmdThreadStep(
thread: threadId,
head: newHead,
done,
sessionId,
};
}
@@ -2,13 +2,32 @@ import { describe, expect, test } from "vitest";
import { buildOutputFormatInstruction } from "../src/build-output-format-instruction.js";
const PLANNER_SCHEMA = {
type: "object",
properties: {
status: { type: "string", enum: ["ready", "insufficient_info"] },
plan: { type: "string" },
},
required: ["status"],
additionalProperties: false,
};
const REVIEWER_SCHEMA = {
type: "object",
properties: {
approved: { type: "boolean" },
},
required: ["approved"],
additionalProperties: false,
};
describe("buildOutputFormatInstruction", () => {
test("always includes the frontmatter example block", () => {
const result = buildOutputFormatInstruction({});
expect(result).toContain("---");
expect(result).toContain("status: done");
expect(result).toContain("confidence:");
expect(result).toContain("scope: role");
expect(result).not.toContain("status: done");
expect(result).not.toContain("confidence:");
expect(result).not.toContain("scope: role");
});
test("always marks frontmatter as the primary deliverable", () => {
@@ -16,17 +35,36 @@ describe("buildOutputFormatInstruction", () => {
expect(result).toContain("primary deliverable");
});
test("lists fields from a flat object schema", () => {
test("generates planner-specific YAML example from schema", () => {
const result = buildOutputFormatInstruction(PLANNER_SCHEMA);
expect(result).toContain("status: ready # required | ready | insufficient_info");
expect(result).toContain("plan: <string>");
expect(result).not.toContain("status: done");
expect(result).not.toContain("confidence:");
expect(result).not.toContain("artifacts:");
});
test("generates reviewer-specific YAML example from schema", () => {
const result = buildOutputFormatInstruction(REVIEWER_SCHEMA);
expect(result).toContain("approved: true # required | true | false");
expect(result).not.toContain("status:");
});
test("lists fields from a flat object schema with required marker", () => {
const schema = {
type: "object",
properties: {
status: { type: "string" },
confidence: { type: "number" },
},
required: ["status"],
};
const result = buildOutputFormatInstruction(schema);
expect(result).toContain("`status`");
expect(result).toContain("`status` (required)");
expect(result).toContain("`confidence`");
expect(result).not.toContain("`confidence` (required)");
expect(result).toContain("status: <string> # required");
expect(result).toContain("confidence: <number>");
});
test("lists union of fields from an anyOf schema", () => {
@@ -45,6 +83,8 @@ describe("buildOutputFormatInstruction", () => {
const result = buildOutputFormatInstruction(schema);
expect(result).toContain("`alpha`");
expect(result).toContain("`beta`");
expect(result).toContain("alpha: <string>");
expect(result).toContain("beta: <number>");
});
test("lists union of fields from a oneOf schema", () => {
@@ -63,6 +103,8 @@ describe("buildOutputFormatInstruction", () => {
const result = buildOutputFormatInstruction(schema);
expect(result).toContain("`foo`");
expect(result).toContain("`bar`");
expect(result).toContain("foo: <string>");
expect(result).toContain("bar: true # true | false");
});
test("falls back gracefully for a non-object schema with no properties", () => {
@@ -80,6 +122,23 @@ describe("buildOutputFormatInstruction", () => {
const result = buildOutputFormatInstruction(schema);
const matches = [...result.matchAll(/`shared`/g)];
expect(matches.length).toBe(1);
expect(result).toContain("shared: <string>");
});
test("marks required when any union variant requires the field", () => {
const schema = {
anyOf: [
{
type: "object",
properties: { shared: { type: "string" } },
required: ["shared"],
},
{ type: "object", properties: { shared: { type: "number" } } },
],
};
const result = buildOutputFormatInstruction(schema);
expect(result).toContain("`shared` (required)");
expect(result).toContain("shared: <string> # required");
});
test("includes focus reminder about role scope", () => {
@@ -29,6 +29,27 @@ const STRICT_SCHEMA = {
additionalProperties: false,
};
/** Role-specific schema (reviewer) — only approved, no standard agent fields. */
const REVIEWER_SCHEMA = {
type: "object",
properties: {
approved: { type: "boolean" },
},
required: ["approved"],
additionalProperties: false,
};
/** Role-specific schema (planner) — custom status enum + plan hash. */
const PLANNER_SCHEMA = {
type: "object",
properties: {
status: { type: "string", enum: ["ready", "insufficient_info"] },
plan: { type: "string" },
},
required: ["status"],
additionalProperties: false,
};
async function makeStoreWithSchema(schema: Record<string, unknown>) {
const store = createMemoryStore();
const schemaHash = await putSchema(store, schema);
@@ -134,3 +155,48 @@ describe("tryFrontmatterFastPath — fallback: schema mismatch", () => {
expect(result).toBeNull();
});
});
// ── Role-specific schema fields ───────────────────────────────────────────────
describe("tryFrontmatterFastPath — role-specific fields", () => {
test("extracts approved only for reviewer schema (no extra standard fields)", async () => {
const { store, schemaHash } = await makeStoreWithSchema(REVIEWER_SCHEMA);
const raw = "---\napproved: true\n---\n\nReview passed.";
const result = await tryFrontmatterFastPath(raw, schemaHash, store);
expect(result).not.toBeNull();
const node = store.get(result!.outputHash);
expect(node).not.toBeNull();
const payload = node!.payload as Record<string, unknown>;
expect(payload).toEqual({ approved: true });
expect(payload.status).toBeUndefined();
expect(payload.scope).toBeUndefined();
});
test("extracts plan and role-specific status for planner schema", async () => {
const { store, schemaHash } = await makeStoreWithSchema(PLANNER_SCHEMA);
const raw = "---\nstatus: ready\nplan: 01HASHPLANNER0001\n---\n\nSpec summary.";
const result = await tryFrontmatterFastPath(raw, schemaHash, store);
expect(result).not.toBeNull();
const node = store.get(result!.outputHash);
expect(node).not.toBeNull();
const payload = node!.payload as Record<string, unknown>;
expect(payload.status).toBe("ready");
expect(payload.plan).toBe("01HASHPLANNER0001");
expect(payload.scope).toBeUndefined();
});
test("returns null when required role-specific field is missing", async () => {
const { store, schemaHash } = await makeStoreWithSchema(REVIEWER_SCHEMA);
const raw = "---\nstatus: done\nscope: role\n---\n\nBody.";
const result = await tryFrontmatterFastPath(raw, schemaHash, store);
expect(result).toBeNull();
});
});
@@ -1,5 +1,11 @@
import type { JSONSchema } from "@uncaged/json-cas";
type SchemaProperty = {
name: string;
schema: JSONSchema;
required: boolean;
};
/**
* Extract top-level property names from a JSON Schema object.
*
@@ -9,9 +15,44 @@ import type { JSONSchema } from "@uncaged/json-cas";
*
* Returns an empty array for schemas with no inspectable property definitions.
*/
function extractSchemaFields(schema: JSONSchema): string[] {
export function extractSchemaFields(schema: JSONSchema): string[] {
return extractSchemaProperties(schema).map((p) => p.name);
}
function extractSchemaProperties(schema: JSONSchema): SchemaProperty[] {
const objectSchemas = collectObjectSchemas(schema);
if (objectSchemas.length === 0) {
return [];
}
const byName = new Map<string, SchemaProperty>();
for (const objectSchema of objectSchemas) {
const requiredSet = new Set(
Array.isArray(objectSchema.required) ? (objectSchema.required as string[]) : [],
);
const properties = objectSchema.properties as Record<string, JSONSchema> | null | undefined;
if (typeof properties !== "object" || properties === null) {
continue;
}
for (const [name, propSchema] of Object.entries(properties)) {
const required = requiredSet.has(name);
const existing = byName.get(name);
if (existing === undefined) {
byName.set(name, { name, schema: propSchema, required });
} else if (required) {
byName.set(name, { ...existing, required: true });
}
}
}
return [...byName.values()];
}
function collectObjectSchemas(schema: JSONSchema): JSONSchema[] {
if (typeof schema.properties === "object" && schema.properties !== null) {
return Object.keys(schema.properties as Record<string, unknown>);
return [schema];
}
const unionKey = Array.isArray(schema.anyOf)
@@ -20,18 +61,109 @@ function extractSchemaFields(schema: JSONSchema): string[] {
? "oneOf"
: null;
if (unionKey !== null) {
const variants = schema[unionKey] as JSONSchema[];
const fieldSet = new Set<string>();
for (const variant of variants) {
for (const field of extractSchemaFields(variant)) {
fieldSet.add(field);
}
}
return [...fieldSet];
if (unionKey === null) {
return [];
}
return [];
const variants = schema[unionKey] as JSONSchema[];
const result: JSONSchema[] = [];
for (const variant of variants) {
result.push(...collectObjectSchemas(variant));
}
return result;
}
function resolvePropertySchema(prop: JSONSchema): JSONSchema {
if (Array.isArray(prop.enum) && prop.enum.length > 0) {
return prop;
}
const unionKey = Array.isArray(prop.anyOf) ? "anyOf" : Array.isArray(prop.oneOf) ? "oneOf" : null;
if (unionKey !== null) {
const variants = prop[unionKey] as JSONSchema[];
const nonNull = variants.filter((v) => v.type !== "null");
if (nonNull.length === 1) {
return nonNull[0];
}
}
return prop;
}
function formatYamlScalar(value: unknown): string {
if (typeof value === "boolean") {
return String(value);
}
if (typeof value === "number") {
return String(value);
}
return String(value);
}
function buildPropertyComment(parts: string[]): string {
const filtered = parts.filter((p) => p.length > 0);
return filtered.length > 0 ? ` # ${filtered.join(" | ")}` : "";
}
function buildPropertyExampleLine(prop: SchemaProperty): string {
const resolved = resolvePropertySchema(prop.schema);
const commentParts: string[] = [];
if (prop.required) {
commentParts.push("required");
}
if (Array.isArray(resolved.enum) && resolved.enum.length > 0) {
const enumValues = resolved.enum.map((v) => String(v));
commentParts.push(...enumValues);
const first = resolved.enum[0];
return `${prop.name}: ${formatYamlScalar(first)}${buildPropertyComment(commentParts)}`;
}
if (resolved.type === "boolean") {
commentParts.push("true", "false");
return `${prop.name}: true${buildPropertyComment(commentParts)}`;
}
if (resolved.type === "string") {
return `${prop.name}: <string>${buildPropertyComment(commentParts)}`;
}
if (resolved.type === "number" || resolved.type === "integer") {
return `${prop.name}: <number>${buildPropertyComment(commentParts)}`;
}
if (resolved.type === "array") {
return `${prop.name}:\n - <item>${buildPropertyComment(commentParts)}`;
}
if (resolved.type === "object") {
return `${prop.name}: <object>${buildPropertyComment(commentParts)}`;
}
return `${prop.name}: <value>${buildPropertyComment(commentParts)}`;
}
function buildYamlExampleBlock(properties: SchemaProperty[]): string {
if (properties.length === 0) {
return "---\n\n... your markdown work here ...";
}
const lines = properties.map((p) => buildPropertyExampleLine(p));
return `---\n${lines.join("\n")}\n---\n\n... your markdown work here ...`;
}
function buildFieldList(properties: SchemaProperty[]): string {
if (properties.length === 0) {
return " (schema fields will be extracted automatically)";
}
return properties
.map((p) => {
const suffix = p.required ? " (required)" : "";
return ` - \`${p.name}\`${suffix}`;
})
.join("\n");
}
/**
@@ -42,28 +174,16 @@ function extractSchemaFields(schema: JSONSchema): string[] {
* system prompt so the deliverable format is the first thing the agent sees.
*/
export function buildOutputFormatInstruction(schema: JSONSchema): string {
const fields = extractSchemaFields(schema);
const fieldList =
fields.length > 0
? fields.map((f) => ` - \`${f}\``).join("\n")
: " (schema fields will be extracted automatically)";
const properties = extractSchemaProperties(schema);
const yamlExample = buildYamlExampleBlock(properties);
const fieldList = buildFieldList(properties);
return `## Deliverable Format
Your response MUST begin with a YAML frontmatter block followed by your markdown work:
\`\`\`
---
status: done # done | needs_input | in_progress | failed
next: <role-name> # suggested next role, or omit
confidence: 0.9 # 0.0–1.0, your self-assessed confidence
artifacts: # list of file paths or CAS hashes you produced
- path/to/file.ts
scope: role # role | thread
---
... your markdown work here ...
${yamlExample}
\`\`\`
The frontmatter is the **primary deliverable** — the engine reads it directly.
+143 -9
View File
@@ -1,13 +1,139 @@
import type { Store } from "@uncaged/json-cas";
import { validate } from "@uncaged/json-cas";
import { getSchema, validate } from "@uncaged/json-cas";
import type { CasRef } from "@uncaged/workflow-protocol";
import { parseFrontmatterMarkdown, validateFrontmatter } from "@uncaged/workflow-util";
import {
type AgentFrontmatter,
createLogger,
parseFrontmatterMarkdown,
validateFrontmatter,
} from "@uncaged/workflow-util";
import { parse as parseYaml } from "yaml";
import { extractSchemaFields } from "./build-output-format-instruction.js";
const log = createLogger({ sink: { kind: "stderr" } });
const STANDARD_KEYS = ["status", "next", "confidence", "artifacts", "scope"] as const;
type StandardKey = (typeof STANDARD_KEYS)[number];
export type FrontmatterFastPathResult = {
body: string;
outputHash: CasRef;
};
function extractYamlBlock(raw: string): string | null {
const fence = "---";
if (!raw.startsWith(fence)) {
return null;
}
const rest = raw.slice(fence.length);
if (rest.length > 0 && rest[0] !== "\n" && rest[0] !== "\r") {
return null;
}
const afterOpen = rest.startsWith("\n") ? rest.slice(1) : rest;
const closeIndex = afterOpen.indexOf(`\n${fence}`);
if (closeIndex === -1) {
return null;
}
return afterOpen.slice(0, closeIndex);
}
function parseRawFrontmatterFields(raw: string): Record<string, unknown> {
const yamlText = extractYamlBlock(raw);
if (yamlText === null) {
return {};
}
try {
const parsed = parseYaml(yamlText);
if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) {
return {};
}
return parsed as Record<string, unknown>;
} catch {
return {};
}
}
function defaultCandidate(frontmatter: AgentFrontmatter): Record<string, unknown> {
return {
status: frontmatter.status,
next: frontmatter.next,
confidence: frontmatter.confidence,
artifacts: [...frontmatter.artifacts],
scope: frontmatter.scope,
};
}
function pickStandardField(frontmatter: AgentFrontmatter, key: StandardKey): unknown {
switch (key) {
case "status":
return frontmatter.status;
case "next":
return frontmatter.next;
case "confidence":
return frontmatter.confidence;
case "artifacts":
return [...frontmatter.artifacts];
case "scope":
return frontmatter.scope;
}
}
function isStandardKey(key: string): key is StandardKey {
return (STANDARD_KEYS as readonly string[]).includes(key);
}
function pickFieldValue(
field: string,
frontmatter: AgentFrontmatter,
rawFields: Record<string, unknown>,
): unknown | undefined {
if (!isStandardKey(field)) {
return Object.hasOwn(rawFields, field) ? rawFields[field] : undefined;
}
const coerced = pickStandardField(frontmatter, field);
if (field === "artifacts" || field === "scope") {
return coerced;
}
if (coerced !== null) {
return coerced;
}
return Object.hasOwn(rawFields, field) ? rawFields[field] : coerced;
}
/**
* Build a CAS candidate object from schema property keys and parsed frontmatter.
*
* When the schema has no inspectable properties, falls back to the five standard
* agent frontmatter fields for backward compatibility.
*/
function buildCandidate(
frontmatter: AgentFrontmatter,
rawFields: Record<string, unknown>,
schemaFields: string[],
): Record<string, unknown> {
if (schemaFields.length === 0) {
return defaultCandidate(frontmatter);
}
const candidate: Record<string, unknown> = {};
for (const field of schemaFields) {
const value = pickFieldValue(field, frontmatter, rawFields);
if (value !== undefined) {
candidate[field] = value;
}
}
return candidate;
}
/**
* Try to satisfy `outputSchema` from frontmatter fields alone.
*
@@ -32,16 +158,22 @@ export async function tryFrontmatterFastPath(
const validationErrors = validateFrontmatter(frontmatter);
if (validationErrors.length > 0) {
log(
"9GNPS4WY",
`frontmatter validation errors: ${validationErrors.map((e) => e.message).join("; ")}`,
);
return null;
}
const candidate: Record<string, unknown> = {
status: frontmatter.status,
next: frontmatter.next,
confidence: frontmatter.confidence,
artifacts: [...frontmatter.artifacts],
scope: frontmatter.scope,
};
const schema = getSchema(store, outputSchema);
if (schema === null) {
log("8FHMR2QX", `output schema not found in CAS: ${outputSchema}`);
return null;
}
const schemaFields = extractSchemaFields(schema);
const rawFields = parseRawFrontmatterFields(raw);
const candidate = buildCandidate(frontmatter, rawFields, schemaFields);
let outputHash: CasRef;
let node: ReturnType<Store["get"]>;
@@ -50,10 +182,12 @@ export async function tryFrontmatterFastPath(
outputHash = await store.put(outputSchema, candidate);
node = store.get(outputHash);
} catch {
log("2KMQT7NR", "failed to store frontmatter candidate in CAS");
return null;
}
if (node === null || !validate(store, node)) {
log("2KMQT7NR", "stored frontmatter candidate failed schema validation");
return null;
}
+1 -15
View File
@@ -98,19 +98,6 @@ async function persistStep(options: {
});
}
export type AgentCliOutput = {
stepHash: CasRef;
sessionId: string;
};
/**
* Create an agent CLI entrypoint.
* Parses argv (`<thread-id> <role>`), runs the agent, extracts structured output,
* writes StepNode to CAS, and prints JSON result to stdout.
*
* If frontmatter extraction fails, retries up to MAX_FRONTMATTER_RETRIES times
* by calling agent.continue() with a correction message.
*/
export function createAgent(options: AgentOptions): () => Promise<void> {
return async function main(): Promise<void> {
const { threadId, role } = parseArgv(process.argv);
@@ -161,7 +148,6 @@ export function createAgent(options: AgentOptions): () => Promise<void> {
agentName: agentLabel(options.name),
});
const result: AgentCliOutput = { stepHash, sessionId: agentResult.sessionId };
process.stdout.write(`${JSON.stringify(result)}\n`);
process.stdout.write(`${stepHash}\n`);
};
}
-1
View File
@@ -81,7 +81,6 @@ export type StepOutput = {
thread: ThreadId;
head: CasRef;
done: boolean;
sessionId: string | null;
};
/** uwf thread steps — single step entry */