@iowarp/clio-coder 0.4.0 → 0.4.2

package/skills/meta/clio-test/references/harness.md

@@ -1,126 +0,0 @@
-# Clio test harness reference
-
-How to drive the real Clio binary, a mock provider, the ACP surface, and a
-real pseudo-terminal in tests. Every model here is a stub; these are machinery
-tests. A run against a real model is `benchmarks/internal/SKILL.md`.
-
-## Contents
-- The spawn harness (`runCli`, `makeScratchHome`)
-- Mocking a provider (OpenAI-compatible SSE fixture)
-- ACP over JSON-RPC/stdio
-- The PTY (`openPty`, `runInPty`)
-- One-off probes (no test file)
-
-## The spawn harness
-
-`tests/harness/spawn.ts` spawns `node dist/cli/index.js` with piped stdio, so
-**build first** (or keep `npm run dev` running) before running smoke.
-
-```ts
-import { makeScratchHome, runCli } from "../harness/spawn.js";
-
-const scratch = makeScratchHome();
-try {
-  await runCli(["doctor", "--fix"], { env: scratch.env });   // bootstrap config
-  const res = await runCli(["targets", "--json"], { env: scratch.env, timeoutMs: 20_000 });
-  // res = { code, signal, stdout, stderr }
-  const parsed = JSON.parse(res.stdout) as { targets: unknown[] };
-} finally {
-  scratch.cleanup();
-}
-```
-
-- `runCli(args, { env, cwd, timeoutMs, input })` → `Promise<{ code, signal, stdout, stderr }>`.
-  Default `timeoutMs` is 15_000; raise it for `run`/`acp`.
-- `makeScratchHome()` → `{ dir, env, cleanup }`. The `env` sets `CLIO_CODER_HOME`,
-  `CLIO_CODER_DATA_DIR`, `CLIO_CODER_CONFIG_DIR`, `CLIO_CODER_CACHE_DIR`, and
-  `CLIO_CODER_REQUIRE_HOME_PREFIX=1`. **Always pass `env: scratch.env`** so the test
-  never touches the developer's real config, and always `cleanup()` in `finally`.
-- Bootstrap a scratch home with `runCli(["doctor", "--fix"], …)` before commands
-  that need settings.
-
-Useful flags seen in smoke tests: `--no-context-files`, `--no-skills`,
-`--skill <path>`, `--json`. For the live CLI surface, run `clio-coder --help` and
-`clio-coder <command> --help` rather than hardcoding a command list here.
-
-## Mocking a provider
-
-To exercise an agent `run` without a real model, stand up an in-process
-OpenAI-compatible server that streams one SSE chunk, then point a target at it by
-patching `settings.yaml`. This is the pattern in `tests/smoke/cli.test.ts`:
-
-```ts
-// 1. Start a fixture that replies with a fixed string over text/event-stream.
-const fixture = await startOpenAICompatFixture("mock reply");
-// 2. Patch the scratch settings.yaml: add an `openai-compat` target whose
-//    url is fixture.url, set orchestrator.target/model to it, and supply the
-//    apiKeyEnvVar it expects.
-seedOpenAICompatOrchestrator(join(scratch.dir, "config"), fixture.url);
-// 3. Run, providing the key env var the target references.
-const res = await runCli(["--no-context-files", "run", "hello"], {
-  env: { ...scratch.env, CLIO_CODER_TEST_OPENAI_KEY: "sk-test" },
-  timeoutMs: 20_000,
-});
-// res.stdout === "mock reply\n"
-await closeServer(fixture.server);
-```
-
-The fixture also records `fixture.requests`, so you can assert what Clio sent
-(e.g. that an explicit `--skill` was injected into the prompt).
-
-## ACP over JSON-RPC/stdio
-
-`clio-coder acp` speaks ACP v1 over stdio. Drive it with a line-delimited JSON-RPC
-client (see `createJsonRpcProcessClient` in the smoke test): `initialize` →
-`session/new` → `session/prompt` → `session/close`. Streaming arrives as
-`session/update` notifications whose `update.sessionUpdate` must be a v1 variant
-(`agent_message_chunk`, `tool_call`, `plan`, `current_mode_update`, …). A
-non-spec discriminator breaks strict clients like Zed, so the smoke test asserts
-every emitted variant is in the v1 set.
-
-## The PTY
-
-Piped stdio reports no terminal width and no TTY, so the TUI refuses to start
-and every width-sensitive path collapses to 80 columns. `tests/harness/pty.ts`
-opens a real pseudo-terminal through `node-pty` (a devDependency, never
-shipped):
-
-```ts
-import { openPty, runInPty, stripAnsi, visibleLines } from "../harness/pty.js";
-
-// Scripted: type on a schedule, stop when the output matches, bounded by a timeout.
-const run = await runInPty(process.execPath, [CLI], { cols: 120, rows: 40, cwd, env,
-  readyWhen: /ctx /, input: [{ afterMs: 200, data: "/quit\r" }], until: /bye/, timeoutMs: 20_000 });
-
-// Controllable: write, resize, pause output, wait for a matcher, wait for exit.
-const session = await openPty(process.execPath, [CLI], { cols: 140, rows: 44, cwd, env });
-await session.waitForOutput((out) => /ctx /.test(stripAnsi(out)), 30_000);
-session.write("/quit\r");
-await session.waitForExit(10_000);
-```
-
-Use it only for what a pipe cannot show: width, raw mode, SIGINT through a
-terminal, the alternate-screen and keyboard-protocol teardown. The three
-suites that need it are `tests/smoke/tui-width-matrix.test.ts`,
-`instant-shell-pty.test.ts`, and `render-trace-pty.test.ts`. Anything else
-belongs on `runCli`.
-
-## One-off probes (no test file)
-
-To poke at Clio without writing a permanent test, drop a throwaway script in
-your scratch directory and run it with tsx. Delete it when done — never leave
-probes under `tests/`, `scripts/`, or `benchmarks/`.
-
-```ts
-// /tmp/probe.ts
-import { makeScratchHome, runCli } from "/abs/path/to/repo/tests/harness/spawn.js";
-const scratch = makeScratchHome();
-await runCli(["doctor", "--fix"], { env: scratch.env });
-const out = await runCli(["skills", "list", "--json", "--all"], { env: scratch.env, cwd: process.cwd() });
-console.log(out.stdout.slice(0, 400));
-scratch.cleanup();
-```
-
-```bash
-npx tsx /tmp/probe.ts
-```

package/skills/meta/clio-test/references/test-map.md

@@ -1,69 +0,0 @@
-# Where Clio's tests live
-
-Three layers under `tests/`. Add a new test next to the closest existing file;
-create a new file only for a genuinely new domain cluster.
-
-## Layout
-
-| Layer | Path | Runner | Build needed |
-|---|---|---|---|
-| contracts | `tests/contracts/*.test.ts` | `npm run test:file -- <glob>` (tsx + scratch root) | no (imports `src`) |
-| smoke | `tests/smoke/*.test.ts` | `npm run test:file -- <glob>` | **yes** (spawns `dist/`) |
-| boundaries | `tests/boundaries/check-boundaries.ts` | `npm run lint` (hygiene) | no |
-| harness (not tests) | `tests/harness/*.ts` | imported by contracts and smoke | — |
-
-The harness modules: `spawn.ts` (run the built CLI with pipes), `scratch-env.ts`
-(isolated Clio home), `pty.ts` (a real pseudo-terminal), `openai-compat-fixture.ts`
-and `fake-lmstudio-server.ts` (stub providers), `fake-ssh.ts` (stub fleet node),
-`clock.ts` (steppable clock), plus dispatch, receipt, and module-graph helpers.
-Everything under `tests/` stubs the model. Real-model runs are
-`benchmarks/internal/` and never run under `npm test`.
-
-## Contract test files
-
-| Area | File |
-|---|---|
-| ACP contract | `tests/contracts/acp.test.ts` |
-| context bootstrap / CLIO-CODER.md parse+render | `tests/contracts/bootstrap.test.ts` |
-| config schema + hot-reload classification | `tests/contracts/config.test.ts` |
-| dispatch (validation / admission / ledger) | `tests/contracts/dispatch.test.ts` |
-| session / memory / evidence persistence | `tests/contracts/persistence.test.ts` |
-| prompt fragments + hashing | `tests/contracts/prompts.test.ts` |
-| provider catalog / matcher / resolver | `tests/contracts/providers.test.ts` |
-| safety classification | `tests/contracts/safety.test.ts` |
-| skills loader / collisions / provenance | `tests/contracts/skills.test.ts` |
-| skill activation + compaction interplay | `tests/contracts/skill-activation-compaction.test.ts` |
-| tool registry / names / profiles | `tests/contracts/tools.test.ts` |
-
-## Smoke + boundaries
-
-| Area | File |
-|---|---|
-| non-interactive CLI + ACP-over-stdio end-to-end | `tests/smoke/cli.test.ts` |
-| the package as installed from `npm pack` | `tests/smoke/pack-install.test.ts` |
-| TUI at real terminal sizes, NO_COLOR, Ctrl-C teardown (PTY) | `tests/smoke/tui-width-matrix.test.ts` |
-| instant shell before hydration, SIGTERM through the lease (PTY) | `tests/smoke/instant-shell-pty.test.ts` |
-| committed-frame render trace under PTY backpressure (PTY) | `tests/smoke/render-trace-pty.test.ts` |
-| import boundary rules (rule1/2/3), run under `npm run lint` | `tests/boundaries/check-boundaries.ts` |
-
-## Running a subset
-
-```bash
-# all contracts
-npm run test:file -- 'tests/contracts/**/*.test.ts'
-# one file
-npm run test:file -- tests/contracts/skills.test.ts
-# only it.only / describe.only within a file
-npm run test:file -- --test-only tests/contracts/skills.test.ts
-```
-
-## Writing tests
-
-- `node:test` + `node:assert/strict`. Group with `describe` / `it`.
-- Local imports end in `.js` (NodeNext), e.g. `from "../../src/domains/x/y.js"`.
-- `tsconfig.tests.json` is strict with `noUncheckedIndexedAccess` and
-  `exactOptionalPropertyTypes`; narrow array access before use.
-- Biome rejects `delete obj.key`; use `Reflect.deleteProperty(obj, "key")` when
-  cleaning env maps or object keys.
-- Filesystem tests use a scratch home via `makeScratchHome()` (smoke) or the
-  `CLIO_CODER_HOME` / `CLIO_CODER_*_DIR` env overrides (contracts); clean up in `finally`.

package/src/engine/apis/degraded-inference.ts

@@ -1,95 +0,0 @@
-/**
- * Degraded-inference watchdog for local runtimes.
- *
- * A local server whose weights or KV cache spilled to CPU does not fail: it
- * answers, at roughly a hundredth of the speed, and the operator sees only a
- * spinner. The observed case was 2m18s of streaming for 25 tokens with no
- * indication that anything was wrong. This watchdog turns that silence into a
- * notice: once a prediction has run past a grace period, a sustained token rate
- * below the floor is reported once per turn, along with what is resident on the
- * target so the cause is visible in the same line.
- *
- * The watchdog observes; it never cancels a turn. A slow model is still a
- * working model, and the operator decides whether to abort.
- */
-
-/** Wall-clock a prediction gets before its rate is judged. */
-export const DEGRADED_GRACE_MS = 30_000;
-/** Sustained output tokens per second below which inference is reported as degraded. */
-export const DEGRADED_FLOOR_TOKENS_PER_SECOND = 2;
-
-export interface DegradedInferenceReport {
-	elapsedMs: number;
-	tokens: number;
-	tokensPerSecond: number;
-}
-
-export interface DegradedInferenceOptions {
-	/** Called at most once, when the observed rate stays under the floor past the grace period. */
-	onDegraded: (report: DegradedInferenceReport) => void;
-	graceMs?: number;
-	floorTokensPerSecond?: number;
-	/** Sampling interval; the watchdog checks the rate this often. */
-	pollMs?: number;
-	now?: () => number;
-	setTimer?: (fn: () => void, ms: number) => { cancel: () => void };
-}
-
-export interface DegradedInferenceWatchdog {
-	/** Record generated tokens (output plus reasoning; both are work the server did). */
-	addTokens: (count: number) => void;
-	/** Test seam: evaluate the rate now instead of waiting for the next poll. */
-	check: () => void;
-	stop: () => void;
-}
-
-function defaultTimer(fn: () => void, ms: number): { cancel: () => void } {
-	const handle = setInterval(fn, ms);
-	handle.unref?.();
-	return { cancel: () => clearInterval(handle) };
-}
-
-/**
- * Start watching a prediction's token rate. Call {@link
- * DegradedInferenceWatchdog.addTokens} as fragments arrive and `stop()` when the
- * turn ends, in a `finally`, so a failed turn never leaves a timer behind.
- */
-export function startDegradedInferenceWatchdog(options: DegradedInferenceOptions): DegradedInferenceWatchdog {
-	const now = options.now ?? Date.now;
-	const graceMs = options.graceMs ?? DEGRADED_GRACE_MS;
-	const floor = options.floorTokensPerSecond ?? DEGRADED_FLOOR_TOKENS_PER_SECOND;
-	const pollMs = options.pollMs ?? 5_000;
-	const startedAt = now();
-	let tokens = 0;
-	let reported = false;
-	let stopped = false;
-
-	const check = (): void => {
-		if (reported || stopped) return;
-		const elapsedMs = now() - startedAt;
-		if (elapsedMs < graceMs) return;
-		const tokensPerSecond = tokens / (elapsedMs / 1000);
-		if (tokensPerSecond >= floor) return;
-		reported = true;
-		timer.cancel();
-		try {
-			options.onDegraded({ elapsedMs, tokens, tokensPerSecond });
-		} catch {
-			// A notice sink failure must never escape into the turn it describes.
-		}
-	};
-
-	const timer = (options.setTimer ?? defaultTimer)(check, pollMs);
-
-	return {
-		addTokens: (count) => {
-			if (Number.isFinite(count) && count > 0) tokens += count;
-		},
-		check,
-		stop: () => {
-			if (stopped) return;
-			stopped = true;
-			timer.cancel();
-		},
-	};
-}