@skill-map/spec 0.1.0

package/job-events.md

@@ -0,0 +1,322 @@
+# Job events
+
+Canonical event stream emitted during job execution. Every implementation MUST emit these events in the order described, with the shapes defined below. Consumers include the CLI pretty printer, the `--json` ndjson output, the Server's WebSocket broadcaster, and any third-party integration.
+
+This document is **normative**. The set of event types, their payload shapes, and their ordering rules are stable contracts.
+
+---
+
+## Transport
+
+Events are records produced by the kernel through `ProgressEmitterPort` (see `architecture.md`). An implementation MUST provide three output adapters:
+
+| Adapter | Purpose | Format |
+|---|---|---|
+| `pretty` | Default TTY output. Human-readable, colored, line-based progress. | Free-form; not normative. |
+| `stream-output` | Pretty + model tokens inline. Debugging mode. | Free-form; not normative. |
+| `json` | Machine-readable ndjson. One event per line; each line is a complete JSON object. | **Normative.** Matches the shapes below. |
+
+The Server exposes the same events over WebSocket (`/ws`) using the same JSON shapes; each event is a single WebSocket text frame.
+
+---
+
+## Common envelope
+
+Every event is a JSON object with this envelope:
+
+```json
+{
+  "type": "<event-type>",
+  "timestamp": <unix-ms>,
+  "runId": "<run-id>",
+  "jobId": "<job-id> | null",
+  "data": { ... }
+}
+```
+
+| Field | Required | Meaning |
+|---|---|---|
+| `type` | always | One of the canonical event types below. |
+| `timestamp` | always | Unix milliseconds when the event was emitted. |
+| `runId` | always | Identifier of the `sm job run` invocation. One run emits many events. Format: `r-YYYYMMDD-HHMMSS-XXXX`. |
+| `jobId` | when job-scoped | The job the event refers to. Null for run-level events (`run.*`). |
+| `data` | per-event | Event-specific payload, shape defined below. |
+
+Implementations MUST include every envelope field in every event, even if `jobId` is null. This simplifies consumers.
+
+Unknown fields in `data` MUST be ignored by consumers (forward compatibility).
+
+---
+
+## Event catalog
+
+Emitted in roughly this order during a `sm job run --all` invocation. The exact sequence may interleave for parallel runs (post-MVP).
+
+### `run.started`
+
+Emitted once at the start of every `sm job run` invocation.
+
+```json
+{
+  "type": "run.started",
+  "timestamp": 1745159455123,
+  "runId": "r-20260420-143055-a3f2",
+  "jobId": null,
+  "data": {
+    "mode": "single | all | max",
+    "maxJobs": 10,
+    "filter": { "action": "skill-summarizer" }
+  }
+}
+```
+
+- `mode`: what the runner was asked to do.
+- `maxJobs`: cap on concurrent drain (`--max N` or null).
+- `filter`: resolved filter predicate, free-form object.
+
+### `run.reap.started`
+
+Emitted before auto-reap scans for expired jobs.
+
+```json
+{
+  "type": "run.reap.started",
+  "timestamp": 1745159455200,
+  "runId": "...",
+  "jobId": null,
+  "data": {}
+}
+```
+
+### `run.reap.completed`
+
+Emitted after auto-reap finishes.
+
+```json
+{
+  "type": "run.reap.completed",
+  "timestamp": 1745159455201,
+  "runId": "...",
+  "jobId": null,
+  "data": {
+    "reapedCount": 0,
+    "reapedIds": []
+  }
+}
+```
+
+- `reapedIds` lists the jobs transitioned from `running` to `failed`. May be empty.
+
+### `job.claimed`
+
+Emitted when the runner successfully claims a job.
+
+```json
+{
+  "type": "job.claimed",
+  "timestamp": 1745159455300,
+  "runId": "...",
+  "jobId": "d-20260420-143055-b001",
+  "data": {
+    "actionId": "skill-summarizer",
+    "actionVersion": "1.2.0",
+    "nodeId": "skills/my-skill.md",
+    "ttlSeconds": 180,
+    "priority": 0
+  }
+}
+```
+
+### `job.skipped`
+
+Emitted when a drain attempts to claim but finds no eligible job.
+
+```json
+{
+  "type": "job.skipped",
+  "timestamp": 1745159455400,
+  "runId": "...",
+  "jobId": null,
+  "data": {
+    "reason": "queue-empty | filter-excluded-all"
+  }
+}
+```
+
+### `job.spawning`
+
+Emitted when the runner is about to execute the job file.
+
+```json
+{
+  "type": "job.spawning",
+  "timestamp": 1745159455500,
+  "runId": "...",
+  "jobId": "...",
+  "data": {
+    "runner": "cli | skill | in-process",
+    "command": "claude -p",
+    "jobFilePath": ".skill-map/jobs/d-20260420-143055-b001.md"
+  }
+}
+```
+
+`command` is implementation-defined free-form; it is descriptive, not invokable.
+
+### `model.delta`
+
+Emitted in `stream-output` mode only. Carries incremental model output.
+
+```json
+{
+  "type": "model.delta",
+  "timestamp": 1745159456000,
+  "runId": "...",
+  "jobId": "...",
+  "data": {
+    "text": "Analyzing the skill...",
+    "channel": "assistant | thinking | tool-use"
+  }
+}
+```
+
+Consumers of the canonical `json` output MAY receive these events if the runner chose to emit them. `pretty` and `json` adapters MAY drop `model.delta` events for brevity.
+
+### `job.callback.received`
+
+Emitted inside `sm record` when the callback arrives and passes nonce validation.
+
+```json
+{
+  "type": "job.callback.received",
+  "timestamp": 1745159465000,
+  "runId": "...",
+  "jobId": "...",
+  "data": {
+    "status": "completed | failed",
+    "model": "claude-opus-4-7",
+    "reportPath": ".skill-map/reports/d-20260420-143055-b001.json"
+  }
+}
+```
+
+`runId` on this event is the run that originally claimed the job. If the record is called from outside a run (interactive skill), `runId` is a synthetic id prefixed `r-ext-`.
+
+### `job.completed`
+
+Emitted when a job transitions to `completed`.
+
+```json
+{
+  "type": "job.completed",
+  "timestamp": 1745159465100,
+  "runId": "...",
+  "jobId": "...",
+  "data": {
+    "durationMs": 9700,
+    "tokensIn": 2431,
+    "tokensOut": 1072,
+    "model": "claude-opus-4-7",
+    "reportPath": ".skill-map/reports/d-20260420-143055-b001.json"
+  }
+}
+```
+
+### `job.failed`
+
+Emitted when a job transitions to `failed` by any path.
+
+```json
+{
+  "type": "job.failed",
+  "timestamp": 1745159465200,
+  "runId": "...",
+  "jobId": "...",
+  "data": {
+    "reason": "runner-error | report-invalid | timeout | abandoned | job-file-missing | user-cancelled",
+    "message": "Subprocess exited with code 127",
+    "exitCode": 127,
+    "durationMs": 180000
+  }
+}
+```
+
+`reason` enum matches `execution-record.failureReason`. `message` is human-readable free-form; MAY be truncated for display.
+
+### `run.summary`
+
+Emitted once at the end of `sm job run`, after the last job event.
+
+```json
+{
+  "type": "run.summary",
+  "timestamp": 1745159475000,
+  "runId": "...",
+  "jobId": null,
+  "data": {
+    "jobsAttempted": 5,
+    "jobsCompleted": 4,
+    "jobsFailed": 1,
+    "totalDurationMs": 20000,
+    "totalTokensIn": 12500,
+    "totalTokensOut": 5300
+  }
+}
+```
+
+`jobsAttempted = jobsCompleted + jobsFailed` always.
+
+---
+
+## Ordering rules
+
+For each job, the normative order is:
+
+```
+job.claimed → job.spawning → (model.delta)* → job.callback.received → (job.completed | job.failed)
+```
+
+For a run:
+
+```
+run.started
+  → run.reap.started → run.reap.completed
+  → (per-job sequence above)*
+  → run.summary
+```
+
+A parallel implementation MAY interleave per-job sequences across different `jobId` values, but MUST preserve ordering within a single `jobId`.
+
+`job.failed` with reason `abandoned` MAY appear without a matching `job.claimed` in the current run — it refers to a job claimed in a previous run that expired before the next reap.
+
+---
+
+## Error handling
+
+If an event payload cannot be serialized (internal bug), the implementation MUST emit a synthetic event:
+
+```json
+{
+  "type": "emitter.error",
+  "timestamp": <now>,
+  "runId": "<runId>",
+  "jobId": null,
+  "data": {
+    "message": "failed to emit event of type '<type>': <reason>"
+  }
+}
+```
+
+Consumers MAY treat `emitter.error` as a soft failure (log and continue). Implementations MUST NOT crash the run because of a serialization failure.
+
+---
+
+## Stability
+
+The **event type list** above is stable as of spec v1.0.0. Adding a new event type is a minor bump. Removing or renaming one is a major bump.
+
+**Adding** fields to `data` is a minor bump. Changing a field's type or removing a field is a major bump.
+
+Consumers MUST ignore unknown fields (forward compatibility).
+
+The envelope (`type`, `timestamp`, `runId`, `jobId`, `data`) is stable. Adding an envelope field is a major bump because every consumer would need to handle it.

package/package.json

@@ -0,0 +1,49 @@
+{
+  "name": "@skill-map/spec",
+  "version": "0.1.0",
+  "description": "JSON Schemas, prose contracts, and conformance suite for the skill-map specification.",
+  "license": "MIT",
+  "type": "module",
+  "homepage": "https://skill-map.dev",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/crystian/skill-map.git",
+    "directory": "spec"
+  },
+  "bugs": {
+    "url": "https://github.com/crystian/skill-map/issues"
+  },
+  "keywords": [
+    "skill-map",
+    "spec",
+    "specification",
+    "json-schema",
+    "ai-agents",
+    "markdown",
+    "claude-code"
+  ],
+  "exports": {
+    ".": "./index.json",
+    "./index.json": "./index.json",
+    "./schemas/*.json": "./schemas/*.json"
+  },
+  "files": [
+    "README.md",
+    "CHANGELOG.md",
+    "versioning.md",
+    "architecture.md",
+    "cli-contract.md",
+    "dispatch-lifecycle.md",
+    "job-events.md",
+    "prompt-preamble.md",
+    "db-schema.md",
+    "plugin-kv-api.md",
+    "interfaces/",
+    "schemas/",
+    "conformance/",
+    "index.json"
+  ],
+  "publishConfig": {
+    "access": "public"
+  }
+}

package/plugin-kv-api.md

@@ -0,0 +1,208 @@
+# Plugin KV API
+
+Normative contract for plugin-accessible persistence. Two modes exist (see `db-schema.md` for the catalog entries):
+
+- **Mode A — KV**: plugin uses the kernel-provided `ctx.store.*` accessor. Backed by the shared `state_plugin_kv` table.
+- **Mode B — Dedicated**: plugin owns its own tables with the `plugin_<normalizedId>_` prefix, migrated by the kernel.
+
+This document defines mode A in full and clarifies the boundary with mode B. Implementations MUST expose this API to every plugin that declares `"storage": { "mode": "kv" }` in its manifest.
+
+---
+
+## Overview
+
+A plugin extension receives a `ctx` object at construction time. `ctx.store` is present if and only if the plugin declared storage. Its shape depends on the mode:
+
+| Mode | `ctx.store` shape |
+|---|---|
+| No storage declared | `undefined`. |
+| `mode: "kv"` | `KvStore` (this document). |
+| `mode: "dedicated"` | `DedicatedStore` (scoped Database wrapper). See mode B below. |
+
+Plugins SHOULD pick the minimum mode they need. Mode A is simpler, deployed across every scope from day zero, and requires no migrations. Mode B is for plugins that need relational shape, indexes, or cross-row queries.
+
+---
+
+## Mode A: `ctx.store` KV accessor
+
+### Interface
+
+```typescript
+interface KvStore {
+  get<T = unknown>(key: string, options?: { nodePath?: string }): Promise<T | null>;
+  set<T = unknown>(key: string, value: T, options?: { nodePath?: string }): Promise<void>;
+  delete(key: string, options?: { nodePath?: string }): Promise<boolean>;
+  list(options?: { nodePath?: string; prefix?: string }): Promise<KvEntry[]>;
+}
+
+interface KvEntry {
+  key: string;
+  value: unknown;
+  nodePath: string | null;
+  updatedAt: number;
+}
+```
+
+Implementations in other languages MUST expose the same semantic surface.
+
+### Scoping
+
+Every operation is scoped by the caller's `pluginId`. The plugin cannot specify, override, or observe another plugin's `pluginId`. This is enforced by the kernel when constructing the `ctx.store` — the `pluginId` is captured at registration time and is not an argument.
+
+Operations MAY be additionally scoped by `nodePath`:
+
+- **Global KV (no `nodePath`)**: `{pluginId, nodePath: null, key}`. One row per plugin + key.
+- **Node-scoped KV (with `nodePath`)**: `{pluginId, nodePath: "<path>", key}`. One row per plugin + node + key.
+
+Both scopes share the same underlying `state_plugin_kv` table (see `db-schema.md`). The `nodePath` column is nullable; implementations MUST use a sentinel empty string internally when the backing engine rejects NULL in composite primary keys.
+
+### Semantics
+
+| Operation | Behaviour |
+|---|---|
+| `get(key, { nodePath })` | Returns the stored value (JSON-decoded) or `null` if no row exists. Never throws for "missing". |
+| `set(key, value, { nodePath })` | Upsert. Replaces any existing value. Updates `updatedAt`. The value is JSON-encoded by the kernel; it MUST be JSON-serializable. Cyclic or non-serializable values MUST be rejected with a typed error. |
+| `delete(key, { nodePath })` | Deletes the row if present. Returns `true` if a row was deleted, `false` otherwise. Idempotent. |
+| `list({ nodePath, prefix })` | Returns all entries matching the scope. `nodePath` omitted: returns global entries (`nodePath IS NULL`). `nodePath: null` (explicit): same as omitted. `nodePath: "<path>"`: returns entries for that node. `prefix`: filters keys starting with the given string. |
+
+Return order of `list` is NOT specified by this spec; consumers MUST NOT rely on ordering. Implementations SHOULD order by `key ASC` for developer ergonomics.
+
+### Key constraints
+
+- `key` MUST be a non-empty string, length ≤ 256 bytes (UTF-8).
+- `key` SHOULD be dot-separated namespaces (`foo.bar.baz`) for discoverability, but this is not enforced.
+- The kernel MAY log a warning when `key` exceeds a reasonable length (e.g. 128), but MUST NOT reject below 256.
+
+### Value constraints
+
+- Value MUST be JSON-serializable (plain objects, arrays, strings, numbers, booleans, null).
+- Values containing `undefined` or functions MUST be rejected with a typed error before writing.
+- The kernel MAY impose a per-value size limit (reference impl: 1 MiB). Exceeding it is a typed error, not a silent truncation.
+
+### Transactions
+
+The `KvStore` operations are individually atomic. There is NO multi-operation transaction in mode A — plugins that need transactional semantics across several rows MUST use mode B.
+
+Implementations MUST NOT expose a `transaction()` method on `KvStore` in mode A. The shape is intentionally minimal to keep the backing table simple.
+
+### Errors
+
+All errors are typed. An implementation MUST expose these error classes (or language equivalents):
+
+| Error | Cause |
+|---|---|
+| `KvKeyInvalidError` | Key is empty, non-string, or too long. |
+| `KvValueNotSerializableError` | Value cannot be JSON-encoded. |
+| `KvValueTooLargeError` | Encoded value exceeds the size limit. |
+| `KvOperationFailedError` | Unexpected backend failure (e.g., DB full, IO error). Wraps the underlying cause. |
+
+Errors MUST NOT leak backend-specific details (SQL strings, file paths) to plugin code unless wrapped in `KvOperationFailedError.cause`.
+
+---
+
+## Mode B: dedicated tables
+
+Mode B is governed by `db-schema.md` (catalog rules + triple protection). This section restates the API surface.
+
+### Declaration
+
+```json
+{
+  "storage": {
+    "mode": "dedicated",
+    "tables": ["rule_exceptions", "cache_entries"],
+    "migrations": ["migrations/001_initial.sql"]
+  }
+}
+```
+
+The `tables` array lists logical table names **without** the `plugin_<id>_` prefix. The kernel prepends the prefix when applying migrations and when routing queries.
+
+### Accessor
+
+```typescript
+interface DedicatedStore {
+  db: Database;   // scoped wrapper, see below
+}
+```
+
+`DedicatedStore.db` is a wrapper — NOT a raw handle. Every query passes through a validator that rejects:
+
+- References to tables whose name doesn't start with this plugin's prefix.
+- DDL statements (`CREATE`, `ALTER`, `DROP`, `TRUNCATE`). Mode B DDL is runtime-immutable after migrations; plugins change shape via a new migration, not at runtime.
+- `ATTACH DATABASE` statements.
+- `PRAGMA` statements that aren't scoped to the plugin's own tables.
+
+A query that fails validation raises `ScopedDbViolationError`. The plugin continues to run; only the offending query is rejected.
+
+### Transaction support
+
+Mode B plugins MAY call `db.transaction(async (tx) => { ... })`. The kernel provides transaction isolation consistent with the backing engine. Nested transactions are NOT supported; the kernel MUST reject a nested `transaction()` call with a typed error.
+
+### Migrations
+
+- Location: `<plugin-dir>/migrations/NNN_snake_case.sql`.
+- Applied in order after kernel migrations on boot.
+- Prefix injection: the kernel rewrites `CREATE TABLE <name>` into `CREATE TABLE plugin_<id>_<name>` if the prefix is missing.
+- Index and constraint prefixes are similarly injected.
+- A failing plugin migration disables only that plugin (`status: load-error`); other plugins and the kernel continue.
+
+See `db-schema.md` for the normative migration rules.
+
+---
+
+## Mode selection guidance
+
+Non-normative; descriptive guidance for plugin authors.
+
+**Prefer mode A when**:
+
+- Each value is a small JSON blob (preferences, per-node flags, hash pins).
+- Queries are "get by key" or "list under a prefix".
+- You need to ship without asking the user to run a migration.
+
+**Prefer mode B when**:
+
+- You need indexes beyond `(pluginId, nodePath, key)`.
+- You need to `JOIN` rows, aggregate, or do relational queries.
+- Your data model is actually tabular (cache with TTL, observation log, provider registry).
+- You are willing to own migrations forever.
+
+A plugin MUST declare **exactly one** storage mode. Mixing modes in the same plugin is forbidden. The `plugins-registry.schema.json` enforces this at the manifest level (`storage` is a `oneOf` between `kv` and `dedicated`), and at runtime `ctx.store` exposes either the `KvStore` or the `DedicatedStore` shape — never both. A plugin that needs both KV-like and relational access MUST use mode B and implement KV-style rows as a dedicated table.
+
+---
+
+## Visibility rules
+
+- A plugin MUST NOT read or write rows outside its scope. Mode A: the accessor is scoped. Mode B: the validator enforces the prefix.
+- The kernel MAY expose read-only introspection for diagnostics (e.g., `sm plugins show <id> --storage` lists key counts). This is authoritative, not a plugin-level API.
+- `sm db shell` can read any table. This is an operator-level escape hatch; plugins MUST NOT rely on it.
+
+---
+
+## Backup and retention
+
+- Mode A rows are stored in `state_plugin_kv` and are backed up with `sm db backup`.
+- Mode B rows live in the plugin's dedicated tables, prefixed `plugin_<id>_`, and are likewise backed up.
+- `sm plugins disable <id>` does NOT drop the plugin's data — disabled plugins keep their KV rows and dedicated tables. `sm plugins forget <id>` (post-MVP) is the verb that wipes.
+- `sm db reset` drops `scan_*` + `state_*`. This includes `state_plugin_kv` (mode A) AND every `plugin_<id>_*` table (mode B). Users MUST be warned by the CLI before `reset` proceeds.
+
+---
+
+## Honest note on isolation
+
+Mode A is perfectly isolated at the row level: the accessor physically cannot see another plugin's rows.
+
+Mode B is **isolated against accidents, not hostile code**. The scoped `Database` wrapper rejects cross-namespace queries at runtime. But a malicious plugin running in the same JavaScript process can bypass the wrapper by importing raw engine bindings directly. Plugins are user-placed code; the kernel trusts the user's judgement at install time.
+
+Post-v1.0 work: signed manifest, sandboxed worker-thread isolation, per-plugin DB file. None of these land before cut 1.
+
+---
+
+## Stability
+
+- The `KvStore` interface (method names, options, return shapes) is **stable** as of spec v1.0.0.
+- Adding a method to `KvStore` is a minor bump; removing or changing signature is a major bump.
+- Mode names (`kv`, `dedicated`) are **stable**. Adding a third mode is a minor bump.
+- Key and value size limits are implementation-defined and MAY change without a spec bump; implementations MUST document their limits in their own changelog.
+- Error class names are **stable**; adding a new error class is a minor bump.