instar 1.2.20 → 1.2.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -7
- package/package.json +1 -1
- package/scripts/docs-coverage.mjs +407 -0
- package/src/data/builtin-manifest.json +2 -2
- package/src/scaffold/templates/jobs/instar/docs-coverage-audit.md +36 -0
- package/upgrades/1.2.21.md +42 -0
- package/upgrades/1.2.22.md +32 -0
- package/upgrades/side-effects/docs-coverage-weekly.md +74 -0
- package/upgrades/side-effects/docs-coverage.md +84 -0
package/README.md
CHANGED
|
@@ -101,10 +101,11 @@ An agent that forgets what you discussed yesterday, doesn't recognize someone it
|
|
|
101
101
|
| Feature | Description | Docs |
|
|
102
102
|
|---------|-------------|------|
|
|
103
103
|
| **Job Scheduler** | Cron-based tasks with priority levels, model tiering, and quota awareness | [→](https://instar.sh/features/scheduler/) |
|
|
104
|
-
| **Telegram** | Two-way messaging via forum topics. Each topic maps to a Claude session | [→](https://instar.sh/features/telegram/) |
|
|
105
|
-
| **WhatsApp** | Full messaging via local Baileys library. No cloud dependency | [→](https://instar.sh/features/whatsapp/) |
|
|
104
|
+
| **Telegram** | Two-way messaging via forum topics. Each topic maps to a Claude session. Default GFM-to-HTML markdown formatter (v1.1.0+) | [→](https://instar.sh/features/telegram/) |
|
|
105
|
+
| **WhatsApp** | Full messaging via local Baileys library or WhatsApp Business API webhook. No cloud dependency in Baileys mode | [→](https://instar.sh/features/whatsapp/) |
|
|
106
106
|
| **iMessage** | Native macOS messaging via Messages.app database polling + `imsg` CLI. [Setup guide](#imessage-setup-macos) | |
|
|
107
|
-
| **
|
|
107
|
+
| **Slack** | Two-way messaging via Slack adapter. Channel and DM routing, eight HTTP routes, dedicated CLI | [→](https://instar.sh/features/slack/) |
|
|
108
|
+
| **Lifeline** | Persistent supervisor. Detects crashes, auto-recovers, queues messages, version-skew handling (v1.1.3+) | [→](https://instar.sh/features/lifeline/) |
|
|
108
109
|
| **Conversational Memory** | Per-topic SQLite with FTS5, rolling summaries, context re-injection | [→](https://instar.sh/features/memory/) |
|
|
109
110
|
| **Evolution System** | Proposals, learnings, gap tracking, commitment follow-through | [→](https://instar.sh/features/evolution/) |
|
|
110
111
|
| **Relationships** | Cross-platform identity resolution, significance scoring, context injection | [→](https://instar.sh/features/relationships/) |
|
|
@@ -113,19 +114,21 @@ An agent that forgets what you discussed yesterday, doesn't recognize someone it
|
|
|
113
114
|
| **Intent Alignment** | Decision journaling, drift detection, organizational constraints | [→](https://instar.sh/features/intent/) |
|
|
114
115
|
| **Multi-Machine** | Ed25519/X25519 crypto identity, encrypted sync, automatic failover | [→](https://instar.sh/features/multi-machine/) |
|
|
115
116
|
| **Serendipity Protocol** | Sub-agents capture out-of-scope discoveries without breaking focus. HMAC-signed, secret-scanned | [→](https://instar.sh/features/serendipity/) |
|
|
116
|
-
| **Threadline Protocol** | Agent-to-agent conversations with canonical identity, three-layer trust model, authorization policy, Ed25519 invitations, Sybil protection, MoltBridge network discovery, rich agent profiles (auto-compiled from agent data with human review gate), discovery waterfall, message security, tamper-proof audit logging, framework-agnostic interop,
|
|
117
|
+
| **Threadline Protocol** | Agent-to-agent conversations with canonical identity, three-layer trust model, authorization policy, Ed25519 invitations, Sybil protection, MoltBridge network discovery, rich agent profiles (auto-compiled from agent data with human review gate), discovery waterfall, message security, tamper-proof audit logging, framework-agnostic interop, persistent listener daemon (always-on relay connection, pipe-mode sessions, sub-30s cross-machine failover), eleven MCP tools (seven core + four registry-conditional). 80 modules, roughly 3,800 test cases across 74 dedicated test files plus 125 cross-cutting | [→](https://instar.sh/features/threadline/) |
|
|
117
118
|
| **Self-Healing** | LLM-powered stall detection, session recovery, promise tracking | [→](https://instar.sh/features/self-healing/) |
|
|
118
119
|
| **AutoUpdater** | Built-in update engine. Checks npm, auto-applies, self-restarts | [→](https://instar.sh/features/autoupdater/) |
|
|
119
120
|
| **Build Pipeline** | `/build` skill with worktree isolation, 6-phase pipeline, quality gates, stop-hook enforcement | |
|
|
120
|
-
| **Behavioral Hooks** |
|
|
121
|
+
| **Behavioral Hooks** | Eleven hook scripts plus nine observability event hooks: command guards, safety gates, identity grounding, topic context, channel context for iMessage and Slack, free-text guard, skill-usage telemetry, build stop-hook | [→](https://instar.sh/reference/hooks/) |
|
|
121
122
|
| **Initiative Tracker** | Persisted multi-phase long-running work tracker. Phases, blockers, links, digest alerts. HTTP API at `/initiatives/*` | |
|
|
122
|
-
| **
|
|
123
|
+
| **Observability** | Token burn detection, quota tracking with tiered backpressure, telemetry collection, homeostasis monitoring, session activity tracking, credential management | [→](https://instar.sh/features/observability/) |
|
|
124
|
+
| **Cross-framework portability** | First-class Codex CLI support via `instar setup --framework codex-cli`. Codex-only init produces zero `.claude/` files. Framework-aware telegram-reply path. FrameworkSessionStore (per-runtime transcripts). FrameworkParitySentinel | [→](https://instar.sh/features/portability/) |
|
|
125
|
+
| **Default Jobs** | Fourteen built-in jobs covering health, reflection, evolution, relationship maintenance, identity review, and five `overseer-*` jobs across development, learning, infrastructure, maintenance, and guardian responsibilities | [→](https://instar.sh/reference/default-jobs/) |
|
|
123
126
|
|
|
124
127
|
> **Reference:** [CLI Commands](https://instar.sh/reference/cli/) · [API Endpoints](https://instar.sh/reference/api/) · [Configuration](https://instar.sh/reference/configuration/) · [File Structure](https://instar.sh/reference/file-structure/)
|
|
125
128
|
|
|
126
129
|
## Agent Skills
|
|
127
130
|
|
|
128
|
-
Instar ships
|
|
131
|
+
Instar ships fourteen skills total — twelve user-facing, plus two internal skills (`instar-dev` and `spec-converge`) used only by the agent that develops instar itself. The standard is the [Agent Skills open standard](https://agentskills.io) -- portable across Claude Code, Codex, Cursor, VS Code, and 35+ other platforms.
|
|
129
132
|
|
|
130
133
|
**Standalone skills** work with zero dependencies. Copy a SKILL.md into your project and go:
|
|
131
134
|
|
package/package.json
CHANGED
|
@@ -0,0 +1,407 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* docs-coverage.mjs — measure documentation coverage of shipped capabilities.
|
|
4
|
+
*
|
|
5
|
+
* Walks src/ to enumerate every shipped capability (routes, commands, jobs,
|
|
6
|
+
* hooks, skills, exported classes per subsystem), cross-references each
|
|
7
|
+
* against README.md and site/src/content/docs/, and produces a coverage
|
|
8
|
+
* report. Designed to converge by construction — deterministic enumeration,
|
|
9
|
+
* not stochastic discovery.
|
|
10
|
+
*
|
|
11
|
+
* Born from a multi-pass agent-driven audit (May 2026) that proved manual
|
|
12
|
+
* audit passes don't converge on a codebase with this much surface drift.
|
|
13
|
+
* The structural answer is a programmatic check that runs in CI.
|
|
14
|
+
*
|
|
15
|
+
* Output:
|
|
16
|
+
* - .instar/docs-coverage.json — machine-readable inventory + per-item
|
|
17
|
+
* coverage tags (DOCUMENTED / PARTIAL / UNDOCUMENTED)
|
|
18
|
+
* - .instar/docs-coverage.md — human-readable report grouped by subsystem
|
|
19
|
+
* - exit code 0 if coverage >= threshold, 1 otherwise
|
|
20
|
+
*
|
|
21
|
+
* Usage:
|
|
22
|
+
* node scripts/docs-coverage.mjs # report only, exit 0
|
|
23
|
+
* node scripts/docs-coverage.mjs --check # exit 1 if below threshold
|
|
24
|
+
* node scripts/docs-coverage.mjs --json # JSON to stdout
|
|
25
|
+
*
|
|
26
|
+
* Thresholds (override via env):
|
|
27
|
+
* INSTAR_DOCS_COVERAGE_MIN — minimum overall coverage % (default 30 — see
|
|
28
|
+
* note in --check; this is the FLOOR, not the goal. Real goal lives in
|
|
29
|
+
* the per-category limits below.)
|
|
30
|
+
* INSTAR_DOCS_COVERAGE_ROUTE_MIN, ..._COMMAND_MIN, ..._JOB_MIN, ..._HOOK_MIN
|
|
31
|
+
* — per-category floors (each default 30; tightened over time as the
|
|
32
|
+
* existing gap closes).
|
|
33
|
+
*
|
|
34
|
+
* The floor starts loose because the existing gap is large. The point of
|
|
35
|
+
* shipping the script first is to prevent further regression while we
|
|
36
|
+
* close the existing gap. Tighten the floors after each round of doc work.
|
|
37
|
+
*/
|
|
38
|
+
|
|
39
|
+
import fs from 'node:fs';
|
|
40
|
+
import path from 'node:path';
|
|
41
|
+
import { fileURLToPath } from 'node:url';
|
|
42
|
+
|
|
43
|
+
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
|
44
|
+
|
|
45
|
+
// ── CLI args ────────────────────────────────────────────────────────
|
|
46
|
+
const args = new Set(process.argv.slice(2));
|
|
47
|
+
const CHECK = args.has('--check');
|
|
48
|
+
const JSON_ONLY = args.has('--json');
|
|
49
|
+
const QUIET = args.has('--quiet');
|
|
50
|
+
|
|
51
|
+
// Resolve repo root. Order of preference:
|
|
52
|
+
// 1. INSTAR_DOCS_COVERAGE_ROOT env var (explicit override, used by tests)
|
|
53
|
+
// 2. process.cwd() if it contains a src/ directory (running from a repo)
|
|
54
|
+
// 3. script's parent directory (default — running from this repo)
|
|
55
|
+
function resolveRoot() {
|
|
56
|
+
if (process.env.INSTAR_DOCS_COVERAGE_ROOT) return process.env.INSTAR_DOCS_COVERAGE_ROOT;
|
|
57
|
+
const cwd = process.cwd();
|
|
58
|
+
if (fs.existsSync(path.join(cwd, 'src'))) return cwd;
|
|
59
|
+
return path.resolve(__dirname, '..');
|
|
60
|
+
}
|
|
61
|
+
const ROOT = resolveRoot();
|
|
62
|
+
|
|
63
|
+
const SRC_DIR = path.join(ROOT, 'src');
|
|
64
|
+
const DOCS_DIR = path.join(ROOT, 'site/src/content/docs');
|
|
65
|
+
const README = path.join(ROOT, 'README.md');
|
|
66
|
+
const STATE_DIR = path.join(ROOT, '.instar');
|
|
67
|
+
|
|
68
|
+
// ── Thresholds ──────────────────────────────────────────────────────
|
|
69
|
+
const num = (env, def) => {
|
|
70
|
+
const v = process.env[env];
|
|
71
|
+
return v !== undefined ? Number(v) : def;
|
|
72
|
+
};
|
|
73
|
+
// Floors are calibrated to current state as of script introduction.
|
|
74
|
+
// Each doc-update PR should ratchet the relevant floor upward — this is
|
|
75
|
+
// how the script enforces non-regression while we close the existing gap.
|
|
76
|
+
// Floors below current measured coverage by ~2-3 percentage points to
|
|
77
|
+
// allow normal churn without false failures.
|
|
78
|
+
const THRESHOLDS = {
|
|
79
|
+
overall: num('INSTAR_DOCS_COVERAGE_MIN', 13),
|
|
80
|
+
route: num('INSTAR_DOCS_COVERAGE_ROUTE_MIN', 11),
|
|
81
|
+
command: num('INSTAR_DOCS_COVERAGE_COMMAND_MIN', 40),
|
|
82
|
+
job: num('INSTAR_DOCS_COVERAGE_JOB_MIN', 55),
|
|
83
|
+
hook: num('INSTAR_DOCS_COVERAGE_HOOK_MIN', 22),
|
|
84
|
+
skill: num('INSTAR_DOCS_COVERAGE_SKILL_MIN', 80),
|
|
85
|
+
class: num('INSTAR_DOCS_COVERAGE_CLASS_MIN', 8),
|
|
86
|
+
};
|
|
87
|
+
|
|
88
|
+
// ── Capability enumeration ──────────────────────────────────────────
|
|
89
|
+
|
|
90
|
+
/** Routes registered in src/server/routes.ts via router.<verb>('path', ...). */
|
|
91
|
+
function enumerateRoutes() {
|
|
92
|
+
const routesFile = path.join(SRC_DIR, 'server/routes.ts');
|
|
93
|
+
if (!fs.existsSync(routesFile)) return [];
|
|
94
|
+
const content = fs.readFileSync(routesFile, 'utf-8');
|
|
95
|
+
const matches = [...content.matchAll(/router\.(get|post|put|delete|patch)\s*\(\s*['"`]([^'"`]+)['"`]/g)];
|
|
96
|
+
const seen = new Set();
|
|
97
|
+
const out = [];
|
|
98
|
+
for (const m of matches) {
|
|
99
|
+
const id = `${m[1].toUpperCase()} ${m[2]}`;
|
|
100
|
+
if (seen.has(id)) continue;
|
|
101
|
+
seen.add(id);
|
|
102
|
+
out.push({ type: 'route', method: m[1].toUpperCase(), path: m[2], id });
|
|
103
|
+
}
|
|
104
|
+
return out;
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
/** Top-level command files in src/commands/. */
|
|
108
|
+
function enumerateCommands() {
|
|
109
|
+
const dir = path.join(SRC_DIR, 'commands');
|
|
110
|
+
if (!fs.existsSync(dir)) return [];
|
|
111
|
+
return fs.readdirSync(dir)
|
|
112
|
+
.filter(f => f.endsWith('.ts') && !f.endsWith('.test.ts'))
|
|
113
|
+
.map(f => {
|
|
114
|
+
const name = f.replace(/\.ts$/, '');
|
|
115
|
+
return { type: 'command', name, id: name, path: `src/commands/${f}` };
|
|
116
|
+
});
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
/** Default job templates shipped with instar. */
|
|
120
|
+
function enumerateJobs() {
|
|
121
|
+
const dir = path.join(SRC_DIR, 'scaffold/templates/jobs/instar');
|
|
122
|
+
if (!fs.existsSync(dir)) return [];
|
|
123
|
+
return fs.readdirSync(dir)
|
|
124
|
+
.filter(f => f.endsWith('.md'))
|
|
125
|
+
.map(f => {
|
|
126
|
+
const name = f.replace(/\.md$/, '');
|
|
127
|
+
return { type: 'job', name, id: name, path: `src/scaffold/templates/jobs/instar/${f}` };
|
|
128
|
+
});
|
|
129
|
+
}
|
|
130
|
+
|
|
131
|
+
/** Hook scripts in src/templates/hooks/. */
|
|
132
|
+
function enumerateHooks() {
|
|
133
|
+
const dir = path.join(SRC_DIR, 'templates/hooks');
|
|
134
|
+
if (!fs.existsSync(dir)) return [];
|
|
135
|
+
return fs.readdirSync(dir)
|
|
136
|
+
.filter(f => /\.(sh|js|mjs)$/.test(f))
|
|
137
|
+
.map(f => {
|
|
138
|
+
const name = f.replace(/\.(sh|js|mjs)$/, '');
|
|
139
|
+
return { type: 'hook', name, id: f, path: `src/templates/hooks/${f}` };
|
|
140
|
+
});
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
/** Skills under skills/. Includes user_invocable flag from frontmatter. */
|
|
144
|
+
function enumerateSkills() {
|
|
145
|
+
const dir = path.join(ROOT, 'skills');
|
|
146
|
+
if (!fs.existsSync(dir)) return [];
|
|
147
|
+
const out = [];
|
|
148
|
+
for (const d of fs.readdirSync(dir)) {
|
|
149
|
+
const skillFile = path.join(dir, d, 'SKILL.md');
|
|
150
|
+
if (!fs.existsSync(skillFile)) continue;
|
|
151
|
+
const content = fs.readFileSync(skillFile, 'utf-8');
|
|
152
|
+
const fm = content.match(/^---\n([\s\S]*?)\n---/);
|
|
153
|
+
const userInvocable = !fm || !/user_invocable:\s*["']?false["']?/i.test(fm[1]);
|
|
154
|
+
out.push({ type: 'skill', name: d, id: d, userInvocable, path: `skills/${d}/SKILL.md` });
|
|
155
|
+
}
|
|
156
|
+
return out;
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
/** Top-level exported classes per subsystem. Identified by PascalCase filename. */
|
|
160
|
+
function enumerateSubsystemClasses() {
|
|
161
|
+
const subsystems = [
|
|
162
|
+
'core', 'monitoring', 'memory', 'lifeline', 'messaging', 'threadline',
|
|
163
|
+
'scheduler', 'remediation', 'tasks', 'paste', 'privacy', 'tunnel',
|
|
164
|
+
'moltbridge', 'identity', 'knowledge', 'users', 'security', 'providers',
|
|
165
|
+
];
|
|
166
|
+
const out = [];
|
|
167
|
+
for (const sub of subsystems) {
|
|
168
|
+
const subDir = path.join(SRC_DIR, sub);
|
|
169
|
+
if (!fs.existsSync(subDir) || !fs.statSync(subDir).isDirectory()) continue;
|
|
170
|
+
for (const f of fs.readdirSync(subDir)) {
|
|
171
|
+
if (!f.endsWith('.ts')) continue;
|
|
172
|
+
if (f.endsWith('.test.ts') || f.endsWith('.types.ts') || f === 'index.ts' || f === 'types.ts') continue;
|
|
173
|
+
const name = f.replace(/\.ts$/, '');
|
|
174
|
+
// Only PascalCase top-level exports count as "classes" for coverage purposes.
|
|
175
|
+
// Lowercase files are utilities not user-facing.
|
|
176
|
+
if (!/^[A-Z]/.test(name)) continue;
|
|
177
|
+
out.push({ type: 'class', subsystem: sub, name, id: `${sub}/${name}`, path: `src/${sub}/${f}` });
|
|
178
|
+
}
|
|
179
|
+
}
|
|
180
|
+
return out;
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
// ── Doc loading ─────────────────────────────────────────────────────
|
|
184
|
+
|
|
185
|
+
function loadAllDocs() {
|
|
186
|
+
const docs = {};
|
|
187
|
+
if (fs.existsSync(README)) {
|
|
188
|
+
docs['README.md'] = fs.readFileSync(README, 'utf-8');
|
|
189
|
+
}
|
|
190
|
+
if (fs.existsSync(DOCS_DIR)) {
|
|
191
|
+
const walk = (dir) => {
|
|
192
|
+
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
|
|
193
|
+
const full = path.join(dir, e.name);
|
|
194
|
+
if (e.isDirectory()) walk(full);
|
|
195
|
+
else if (/\.(md|mdx)$/.test(e.name)) {
|
|
196
|
+
docs[path.relative(ROOT, full)] = fs.readFileSync(full, 'utf-8');
|
|
197
|
+
}
|
|
198
|
+
}
|
|
199
|
+
};
|
|
200
|
+
walk(DOCS_DIR);
|
|
201
|
+
}
|
|
202
|
+
return docs;
|
|
203
|
+
}
|
|
204
|
+
|
|
205
|
+
// ── Coverage scoring ────────────────────────────────────────────────
|
|
206
|
+
|
|
207
|
+
function findMentions(capability, docs) {
|
|
208
|
+
// Returns list of doc paths that mention this capability.
|
|
209
|
+
// Coverage strategy varies per capability type.
|
|
210
|
+
let needles = [];
|
|
211
|
+
switch (capability.type) {
|
|
212
|
+
case 'route':
|
|
213
|
+
// Match exact path (with leading /). Substring match — close enough.
|
|
214
|
+
needles = [capability.path];
|
|
215
|
+
break;
|
|
216
|
+
case 'command':
|
|
217
|
+
// Match either `instar <name>` (CLI usage) or just the bare name.
|
|
218
|
+
needles = [`instar ${capability.name}`];
|
|
219
|
+
break;
|
|
220
|
+
case 'job':
|
|
221
|
+
needles = [capability.name];
|
|
222
|
+
break;
|
|
223
|
+
case 'hook':
|
|
224
|
+
needles = [capability.name];
|
|
225
|
+
break;
|
|
226
|
+
case 'skill':
|
|
227
|
+
needles = [capability.name];
|
|
228
|
+
break;
|
|
229
|
+
case 'class':
|
|
230
|
+
needles = [capability.name];
|
|
231
|
+
break;
|
|
232
|
+
}
|
|
233
|
+
const matches = [];
|
|
234
|
+
for (const [docPath, content] of Object.entries(docs)) {
|
|
235
|
+
for (const needle of needles) {
|
|
236
|
+
if (content.includes(needle)) {
|
|
237
|
+
matches.push(docPath);
|
|
238
|
+
break;
|
|
239
|
+
}
|
|
240
|
+
}
|
|
241
|
+
}
|
|
242
|
+
return matches;
|
|
243
|
+
}
|
|
244
|
+
|
|
245
|
+
function classifyCoverage(mentions) {
|
|
246
|
+
if (mentions.length === 0) return 'UNDOCUMENTED';
|
|
247
|
+
if (mentions.length === 1) return 'PARTIAL';
|
|
248
|
+
return 'DOCUMENTED';
|
|
249
|
+
}
|
|
250
|
+
|
|
251
|
+
// ── Report generation ───────────────────────────────────────────────
|
|
252
|
+
|
|
253
|
+
function buildReport(items, docs) {
|
|
254
|
+
for (const item of items) {
|
|
255
|
+
const mentions = findMentions(item, docs);
|
|
256
|
+
item.mentions = mentions;
|
|
257
|
+
item.coverage = classifyCoverage(mentions);
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
const byType = {};
|
|
261
|
+
for (const item of items) {
|
|
262
|
+
if (!byType[item.type]) byType[item.type] = { total: 0, documented: 0, partial: 0, undocumented: 0, items: [] };
|
|
263
|
+
const bucket = byType[item.type];
|
|
264
|
+
bucket.total++;
|
|
265
|
+
bucket.items.push(item);
|
|
266
|
+
if (item.coverage === 'DOCUMENTED') bucket.documented++;
|
|
267
|
+
else if (item.coverage === 'PARTIAL') bucket.partial++;
|
|
268
|
+
else bucket.undocumented++;
|
|
269
|
+
}
|
|
270
|
+
|
|
271
|
+
for (const t of Object.keys(byType)) {
|
|
272
|
+
const b = byType[t];
|
|
273
|
+
b.coveragePct = b.total === 0 ? 100 : Math.round(((b.documented + b.partial * 0.5) / b.total) * 100);
|
|
274
|
+
}
|
|
275
|
+
|
|
276
|
+
const total = items.length;
|
|
277
|
+
const documented = items.filter(i => i.coverage === 'DOCUMENTED').length;
|
|
278
|
+
const partial = items.filter(i => i.coverage === 'PARTIAL').length;
|
|
279
|
+
const undocumented = total - documented - partial;
|
|
280
|
+
const overallPct = total === 0 ? 100 : Math.round(((documented + partial * 0.5) / total) * 100);
|
|
281
|
+
|
|
282
|
+
return {
|
|
283
|
+
generatedAt: new Date().toISOString(),
|
|
284
|
+
totals: { total, documented, partial, undocumented, coveragePct: overallPct },
|
|
285
|
+
thresholds: THRESHOLDS,
|
|
286
|
+
byType,
|
|
287
|
+
};
|
|
288
|
+
}
|
|
289
|
+
|
|
290
|
+
function renderMarkdown(report) {
|
|
291
|
+
const lines = [];
|
|
292
|
+
lines.push('# Documentation Coverage Report');
|
|
293
|
+
lines.push('');
|
|
294
|
+
lines.push(`Generated: ${report.generatedAt}`);
|
|
295
|
+
lines.push('');
|
|
296
|
+
lines.push('## Overall');
|
|
297
|
+
lines.push('');
|
|
298
|
+
lines.push(`**Coverage: ${report.totals.coveragePct}%** (${report.totals.documented} documented + ${report.totals.partial} partial of ${report.totals.total} shipped capabilities)`);
|
|
299
|
+
lines.push('');
|
|
300
|
+
lines.push(`Threshold (overall): ${report.thresholds.overall}% — ${report.totals.coveragePct >= report.thresholds.overall ? 'PASS' : 'FAIL'}`);
|
|
301
|
+
lines.push('');
|
|
302
|
+
lines.push('## By capability type');
|
|
303
|
+
lines.push('');
|
|
304
|
+
lines.push('| Type | Total | Documented | Partial | Undocumented | Coverage | Floor | Status |');
|
|
305
|
+
lines.push('|---|---|---|---|---|---|---|---|');
|
|
306
|
+
for (const t of Object.keys(report.byType).sort()) {
|
|
307
|
+
const b = report.byType[t];
|
|
308
|
+
const floor = report.thresholds[t] ?? '—';
|
|
309
|
+
const status = floor === '—' ? '—' : (b.coveragePct >= floor ? 'PASS' : 'FAIL');
|
|
310
|
+
lines.push(`| ${t} | ${b.total} | ${b.documented} | ${b.partial} | ${b.undocumented} | ${b.coveragePct}% | ${floor}% | ${status} |`);
|
|
311
|
+
}
|
|
312
|
+
lines.push('');
|
|
313
|
+
lines.push('## Undocumented items');
|
|
314
|
+
lines.push('');
|
|
315
|
+
for (const t of Object.keys(report.byType).sort()) {
|
|
316
|
+
const undoc = report.byType[t].items.filter(i => i.coverage === 'UNDOCUMENTED');
|
|
317
|
+
if (undoc.length === 0) continue;
|
|
318
|
+
lines.push(`### ${t} (${undoc.length})`);
|
|
319
|
+
lines.push('');
|
|
320
|
+
for (const item of undoc) {
|
|
321
|
+
lines.push(`- \`${item.id}\`${item.path ? ` — \`${item.path}\`` : ''}`);
|
|
322
|
+
}
|
|
323
|
+
lines.push('');
|
|
324
|
+
}
|
|
325
|
+
lines.push('## Partial coverage (single mention)');
|
|
326
|
+
lines.push('');
|
|
327
|
+
for (const t of Object.keys(report.byType).sort()) {
|
|
328
|
+
const partial = report.byType[t].items.filter(i => i.coverage === 'PARTIAL');
|
|
329
|
+
if (partial.length === 0) continue;
|
|
330
|
+
lines.push(`### ${t} (${partial.length})`);
|
|
331
|
+
lines.push('');
|
|
332
|
+
for (const item of partial) {
|
|
333
|
+
lines.push(`- \`${item.id}\` → mentioned only in \`${item.mentions[0]}\``);
|
|
334
|
+
}
|
|
335
|
+
lines.push('');
|
|
336
|
+
}
|
|
337
|
+
return lines.join('\n');
|
|
338
|
+
}
|
|
339
|
+
|
|
340
|
+
// ── Main ────────────────────────────────────────────────────────────
|
|
341
|
+
|
|
342
|
+
function main() {
|
|
343
|
+
if (!QUIET && !JSON_ONLY) console.error('[docs-coverage] enumerating capabilities…');
|
|
344
|
+
|
|
345
|
+
const capabilities = [
|
|
346
|
+
...enumerateRoutes(),
|
|
347
|
+
...enumerateCommands(),
|
|
348
|
+
...enumerateJobs(),
|
|
349
|
+
...enumerateHooks(),
|
|
350
|
+
...enumerateSkills(),
|
|
351
|
+
...enumerateSubsystemClasses(),
|
|
352
|
+
];
|
|
353
|
+
|
|
354
|
+
if (!QUIET && !JSON_ONLY) console.error(`[docs-coverage] enumerated ${capabilities.length} capabilities; loading docs…`);
|
|
355
|
+
|
|
356
|
+
const docs = loadAllDocs();
|
|
357
|
+
if (!QUIET && !JSON_ONLY) console.error(`[docs-coverage] loaded ${Object.keys(docs).length} docs; scoring…`);
|
|
358
|
+
|
|
359
|
+
const report = buildReport(capabilities, docs);
|
|
360
|
+
|
|
361
|
+
// Ensure state dir exists before writing.
|
|
362
|
+
if (!fs.existsSync(STATE_DIR)) fs.mkdirSync(STATE_DIR, { recursive: true });
|
|
363
|
+
|
|
364
|
+
// Write the JSON report (machine-readable, used by CI + watchers).
|
|
365
|
+
fs.writeFileSync(path.join(STATE_DIR, 'docs-coverage.json'), JSON.stringify(report, null, 2) + '\n');
|
|
366
|
+
|
|
367
|
+
// Write the markdown report (human-readable, committable artifact).
|
|
368
|
+
const md = renderMarkdown(report);
|
|
369
|
+
fs.writeFileSync(path.join(STATE_DIR, 'docs-coverage.md'), md);
|
|
370
|
+
|
|
371
|
+
if (JSON_ONLY) {
|
|
372
|
+
process.stdout.write(JSON.stringify(report, null, 2) + '\n');
|
|
373
|
+
} else if (!QUIET) {
|
|
374
|
+
console.error('');
|
|
375
|
+
console.error(`Overall coverage: ${report.totals.coveragePct}% (${report.totals.documented} doc + ${report.totals.partial} partial of ${report.totals.total} capabilities)`);
|
|
376
|
+
for (const t of Object.keys(report.byType).sort()) {
|
|
377
|
+
const b = report.byType[t];
|
|
378
|
+
const floor = report.thresholds[t] ?? null;
|
|
379
|
+
console.error(` ${t.padEnd(8)} ${String(b.coveragePct).padStart(3)}% (${b.documented}/${b.total})${floor !== null ? ` floor ${floor}%` : ''}`);
|
|
380
|
+
}
|
|
381
|
+
console.error('');
|
|
382
|
+
console.error(`Reports written to .instar/docs-coverage.{json,md}`);
|
|
383
|
+
}
|
|
384
|
+
|
|
385
|
+
if (CHECK) {
|
|
386
|
+
// CI mode — fail if any category is below its floor.
|
|
387
|
+
const failures = [];
|
|
388
|
+
if (report.totals.coveragePct < THRESHOLDS.overall) {
|
|
389
|
+
failures.push(`overall ${report.totals.coveragePct}% < ${THRESHOLDS.overall}%`);
|
|
390
|
+
}
|
|
391
|
+
for (const t of Object.keys(report.byType)) {
|
|
392
|
+
const floor = THRESHOLDS[t];
|
|
393
|
+
if (floor === undefined) continue;
|
|
394
|
+
const pct = report.byType[t].coveragePct;
|
|
395
|
+
if (pct < floor) failures.push(`${t} ${pct}% < ${floor}%`);
|
|
396
|
+
}
|
|
397
|
+
if (failures.length > 0) {
|
|
398
|
+
process.stderr.write(`\n❌ docs-coverage check failed:\n`);
|
|
399
|
+
for (const f of failures) process.stderr.write(` - ${f}\n`);
|
|
400
|
+
process.stderr.write(`\nFix: add doc coverage for the items listed in .instar/docs-coverage.md, then re-run.\n`);
|
|
401
|
+
process.exit(1);
|
|
402
|
+
}
|
|
403
|
+
if (!QUIET) console.error('✅ docs-coverage check passed.');
|
|
404
|
+
}
|
|
405
|
+
}
|
|
406
|
+
|
|
407
|
+
main();
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "./builtin-manifest.schema.json",
|
|
3
3
|
"schemaVersion": 1,
|
|
4
|
-
"generatedAt": "2026-05-
|
|
5
|
-
"instarVersion": "1.2.
|
|
4
|
+
"generatedAt": "2026-05-22T04:08:18.326Z",
|
|
5
|
+
"instarVersion": "1.2.22",
|
|
6
6
|
"entryCount": 191,
|
|
7
7
|
"entries": {
|
|
8
8
|
"hook:session-start": {
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Docs Coverage Audit
|
|
3
|
+
description: Weekly walk of the instar source tree against the docs surface. Surfaces newly-undocumented capabilities so docs don't drift between releases. Off by default — only useful on machines that have the instar source repo locally (instar developers, contributors). Operators who want it can flip enabled to true in `.instar/jobs/instar/docs-coverage-audit.md`.
|
|
4
|
+
schedule: "0 10 * * 1"
|
|
5
|
+
priority: medium
|
|
6
|
+
expectedDurationMinutes: 5
|
|
7
|
+
model: haiku
|
|
8
|
+
enabled: false
|
|
9
|
+
tags:
|
|
10
|
+
- cat:learning
|
|
11
|
+
- audit
|
|
12
|
+
- instar-dev-only
|
|
13
|
+
toolAllowlist: "*"
|
|
14
|
+
unrestrictedTools: true
|
|
15
|
+
---
|
|
16
|
+
Run a weekly documentation coverage audit on the instar codebase. This job exists because docs drift between releases — features ship without doc updates, and the gap accumulates silently until someone notices. This audit catches the drift early.
|
|
17
|
+
|
|
18
|
+
The job:
|
|
19
|
+
|
|
20
|
+
1. Locate the instar source repo on this machine. Try in order: the workspace dir from .instar/config.json under workspace.repoPath; ~/Documents/Projects/instar/; ~/.instar/agents/<self>/.worktrees/ (any subdirectory); or whatever path the operator has configured. If you cannot find the instar repo on this machine, send a short Telegram message saying so and exit — there's no audit to run.
|
|
21
|
+
|
|
22
|
+
2. Run the coverage script from inside that repo: `node scripts/docs-coverage.mjs --json > /tmp/docs-coverage-this-week.json`. The script enumerates every shipped capability (routes, commands, jobs, hooks, skills, top-level classes per subsystem) and cross-references against site docs + README. It writes both a JSON inventory and a markdown report to .instar/docs-coverage.json and .instar/docs-coverage.md.
|
|
23
|
+
|
|
24
|
+
3. Compare the new report to last week's. The previous week's report lives at .instar/state/docs-coverage-last-week.json if it exists. Use plain JSON diff logic — what's the delta in undocumented count per category? What capabilities are newly undocumented (existed before but lost their doc mention, or are brand-new in src/ without any doc)? What capabilities became newly documented (good news worth surfacing)?
|
|
25
|
+
|
|
26
|
+
4. If the delta is interesting (any category gained ≥3 undocumented items OR any single category dropped below its CI floor OR the overall coverage dropped by ≥2 percentage points), send a Telegram message that's a quick, conversational heads-up. Lead with the headline: "Docs coverage dipped in [category]" or "Five new endpoints landed without doc mentions" or similar. Then list the specific items in plain language with their file paths. End with a one-line suggestion of where to add the missing doc (which file in site/src/content/docs/ is the natural home).
|
|
27
|
+
|
|
28
|
+
5. If the delta is uninteresting (small fluctuations, churn that nets to zero, all categories still above floor), stay silent. Most weeks should be silent — that means the docs are keeping up.
|
|
29
|
+
|
|
30
|
+
6. After surfacing (or not), save the current report as next week's baseline: copy /tmp/docs-coverage-this-week.json to .instar/state/docs-coverage-last-week.json.
|
|
31
|
+
|
|
32
|
+
7. Optionally, if the script's --check mode would fail (any category below its CI floor), file a higher-priority signal via the degradation reporter — this is the case where the next PR will hit a CI failure unless someone fixes coverage first.
|
|
33
|
+
|
|
34
|
+
IMPORTANT: This is a guardian job, not a doer. Surface the drift and where to fix it, don't fix it yourself. The fix work belongs in a deliberate doc-update PR with proper review, not in an autonomous job run. If the drift is large, suggest the operator open a topic to scope the doc update — don't try to enumerate the full backlog in one Telegram message.
|
|
35
|
+
|
|
36
|
+
Plain English. No raw JSON in the message. No structured field names. The user reads these on their phone. Write like you're texting a teammate about a code-review observation.
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# Upgrade Guide — vNEXT
|
|
2
|
+
|
|
3
|
+
<!-- bump: patch -->
|
|
4
|
+
<!-- patch = bug fixes, refactors, test additions, doc updates -->
|
|
5
|
+
|
|
6
|
+
## What Changed
|
|
7
|
+
|
|
8
|
+
**feat(docs-coverage): deterministic enumeration as a structural antidrift for docs.**
|
|
9
|
+
|
|
10
|
+
A multi-pass agent-driven audit (May 2026) proved that documentation drift on this codebase cannot be closed by manual audit passes. Each pass found new items the previous pass missed because the gap is large enough — only one of twenty-four `src/` subsystems has complete feature-doc coverage today — that no single pass exhaustively enumerates.
|
|
11
|
+
|
|
12
|
+
This release ships the structural answer per the codebase's Structure-over-Willpower principle: a pure-Node script that walks the source tree, enumerates every shipped capability (HTTP routes, CLI commands, scheduled jobs, hooks, skills, and top-level classes per subsystem), cross-references each against `README.md` and `site/src/content/docs/`, and produces a coverage report.
|
|
13
|
+
|
|
14
|
+
The script:
|
|
15
|
+
|
|
16
|
+
1. **Enumerates deterministically.** No model in the loop. The same source state produces the same report every run.
|
|
17
|
+
2. **Classifies coverage as DOCUMENTED / PARTIAL / UNDOCUMENTED.** Two-or-more doc mentions count as documented; exactly one counts as partial; zero counts as undocumented.
|
|
18
|
+
3. **Enforces per-category floors via CI.** A new `Docs Coverage` job runs `node scripts/docs-coverage.mjs --check` on every PR and fails if any category drops below its floor. Initial floors are calibrated to current measured coverage minus a small buffer.
|
|
19
|
+
4. **Ratchets upward.** PRs that improve docs raise the relevant floor. The bar moves with the codebase.
|
|
20
|
+
5. **Produces two artifacts.** `.instar/docs-coverage.json` (machine-readable, used by CI and future automation) and `.instar/docs-coverage.md` (human-readable, grouped by capability type with sections for undocumented and partial items). Both are git-ignored as generated artifacts; both are uploaded as CI artifacts on every run.
|
|
21
|
+
|
|
22
|
+
Initial baseline measured against current main: 15% overall coverage across 880 shipped capabilities — 36 of 457 routes mentioned in docs, 7 of 31 commands, 4 of 14 default jobs, 12 of 14 skills.
|
|
23
|
+
|
|
24
|
+
Spec: `docs/specs/docs-coverage.md`. ELI16: `docs/specs/docs-coverage.eli16.md`. Side-effects review: `upgrades/side-effects/docs-coverage.md`.
|
|
25
|
+
|
|
26
|
+
This is the foundation for two follow-on capabilities tracked separately: bulk doc updates (Phase 2 of the current sprint) and a weekly audit job that surfaces drift via Telegram (Phase 5).
|
|
27
|
+
|
|
28
|
+
## What to Tell Your User
|
|
29
|
+
|
|
30
|
+
Nothing user-visible. This release adds a CI check that enforces documentation coverage on instar's own development pipeline. Agents continue to behave identically.
|
|
31
|
+
|
|
32
|
+
If a contributor asks why their pull request is failing the new docs coverage job, the failure message names the offending category and the fix is to add a doc mention for the new capability somewhere in the README or under site docs. Per-category floors can be temporarily relaxed via environment variables for one-off pull requests that genuinely need to skip the check.
|
|
33
|
+
|
|
34
|
+
## Summary of New Capabilities
|
|
35
|
+
|
|
36
|
+
This release is a pure infrastructure addition. No new runtime capabilities for agents. The new CI gate and reporting tool are for instar developers and direct contributors only.
|
|
37
|
+
|
|
38
|
+
## Evidence
|
|
39
|
+
|
|
40
|
+
The script was run against current main and produced the baseline coverage numbers above. Seven unit tests verify the enumeration, scoring, CI behavior, and artifact output against a mock repo. The CI workflow runs the script in `--check` mode and uploads the report as an artifact. `npm run lint` passes (TypeScript, destructive-ops lint, LLM-HTTP lint, Codex Rule 1 drift).
|
|
41
|
+
|
|
42
|
+
The audit that motivated this script remains as evidence: it ran for six passes and never converged. The script's first run reproduces the audit's headline finding (15% coverage) in under one second of deterministic walking — proving the convergence-by-construction claim.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Upgrade Guide — vNEXT
|
|
2
|
+
|
|
3
|
+
<!-- bump: patch -->
|
|
4
|
+
<!-- patch = bug fixes, refactors, test additions, doc updates -->
|
|
5
|
+
|
|
6
|
+
## What Changed
|
|
7
|
+
|
|
8
|
+
**feat(docs-coverage-weekly): weekly audit job + GitHub Actions cron that surface accumulated docs drift.**
|
|
9
|
+
|
|
10
|
+
Companion to the per-PR docs coverage gate that shipped in v1.2.21. The per-PR gate catches regression, but coverage can stagnate at the floor indefinitely if every PR just barely scrapes by. This release adds the trend-surface piece: a weekly check that catches accumulated drift even when the per-PR gate is satisfied.
|
|
11
|
+
|
|
12
|
+
Two delivery surfaces, both light and both off the critical path:
|
|
13
|
+
|
|
14
|
+
1. **Job template** — `src/scaffold/templates/jobs/instar/docs-coverage-audit.md` ships as an agentmd job, `enabled: false` by default. Instar-developing agents (the ones with the instar source repo on their machine) can flip it to `true` in their local `.instar/jobs/instar/docs-coverage-audit.md`. Once enabled, the job runs every Monday at 10:00 local time, locates the instar source repo, executes the coverage script, compares to the prior week's baseline, and sends a Telegram heads-up if anything drifted in an interesting way. Most weeks stay silent — that means docs are keeping up.
|
|
15
|
+
|
|
16
|
+
2. **CI workflow** — `.github/workflows/docs-coverage-weekly.yml` runs every Monday at 10:00 UTC against the main branch. Posts the weekly report as a comment on a standing tracking issue tagged `docs-coverage`. Creates the issue on first run; appends weekly thereafter. Provides a durable, GitHub-native ledger of how coverage moves over time.
|
|
17
|
+
|
|
18
|
+
Both surfaces are signals, not authorities. Neither fixes docs autonomously. The per-PR gate from v1.2.21 remains the authoritative regression check; these surface the trend.
|
|
19
|
+
|
|
20
|
+
Spec: `docs/specs/docs-coverage-weekly.md`. ELI16: `docs/specs/docs-coverage-weekly.eli16.md`. Side-effects review: `upgrades/side-effects/docs-coverage-weekly.md`.
|
|
21
|
+
|
|
22
|
+
## What to Tell Your User
|
|
23
|
+
|
|
24
|
+
Nothing user-visible. The job is off by default and would only apply to agents that develop instar itself. The workflow runs in instar's own GitHub Actions, not the user's.
|
|
25
|
+
|
|
26
|
+
## Summary of New Capabilities
|
|
27
|
+
|
|
28
|
+
This release adds a single new job template (off by default) and a single new CI workflow. Both target the instar-developer use case rather than the end-user agent. No new agent runtime behavior.
|
|
29
|
+
|
|
30
|
+
## Evidence
|
|
31
|
+
|
|
32
|
+
The coverage script that both surfaces invoke was shipped and tested in v1.2.21. The job template was verified to parse with valid frontmatter. The workflow YAML validates against GitHub Actions schema. Both files were verified to install via existing pipelines: `installBuiltinJobs()` walks the templates directory on every install and update, and the workflow runs from `.github/workflows/` like every other workflow in the repo.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Side-effects review — docs-coverage-weekly
|
|
2
|
+
|
|
3
|
+
Spec: `docs/specs/docs-coverage-weekly.md`
|
|
4
|
+
ELI16: `docs/specs/docs-coverage-weekly.eli16.md`
|
|
5
|
+
Companion to: `docs/specs/docs-coverage.md` (the underlying script + per-PR CI gate)
|
|
6
|
+
|
|
7
|
+
## Surface map
|
|
8
|
+
|
|
9
|
+
| Change | File | Type |
|
|
10
|
+
|---|---|---|
|
|
11
|
+
| New job template | `src/scaffold/templates/jobs/instar/docs-coverage-audit.md` | `agentmd` job, `enabled: false` by default. Distributed via `installBuiltinJobs()` on `instar init` and on every update via `PostUpdateMigrator` |
|
|
12
|
+
| New CI workflow | `.github/workflows/docs-coverage-weekly.yml` | GitHub Actions cron workflow, Mondays at 10:00 UTC |
|
|
13
|
+
|
|
14
|
+
No production code touched. No agent behavior changes at runtime unless an operator explicitly enables the job.
|
|
15
|
+
|
|
16
|
+
## Over-block analysis
|
|
17
|
+
|
|
18
|
+
**Could the job spam an operator?**
|
|
19
|
+
|
|
20
|
+
The job ships `enabled: false`. Operators with the instar source repo on their machine can flip it to `true` in `.instar/jobs/instar/docs-coverage-audit.md`. The job's first action is to locate the instar source repo; if it can't find one, it sends a short message and exits — and the operator can disable the job again. So the worst case is one Telegram message before the operator turns it back off.
|
|
21
|
+
|
|
22
|
+
When enabled and working correctly, the job stays silent most weeks. The delta thresholds (≥3 newly-undocumented items in a category, any category below CI floor, ≥2-point overall drop) ensure that only meaningful drift surfaces a message.
|
|
23
|
+
|
|
24
|
+
**Could the CI workflow spam an issue tracker?**
|
|
25
|
+
|
|
26
|
+
The workflow opens or updates one standing issue tagged `docs-coverage` and appends weekly reports as comments. If the issue is closed by a maintainer (signaling "stop tracking this"), the workflow creates a new one on the next run. This is the desired behavior — closing the issue is the soft way to say "I'm aware, I'll re-engage later." Comments accumulate on a single issue rather than creating a new issue per run.
|
|
27
|
+
|
|
28
|
+
**Could the CI workflow fail and block other CI?**
|
|
29
|
+
|
|
30
|
+
No. The workflow is its own job in its own file with no dependencies on other workflows. A failure is visible in the Actions UI but doesn't gate any other PR or push.
|
|
31
|
+
|
|
32
|
+
## Under-block analysis
|
|
33
|
+
|
|
34
|
+
**What does the weekly audit NOT catch?**
|
|
35
|
+
|
|
36
|
+
- Same content-quality limitations as the per-PR coverage check — a documented capability with completely wrong content still counts as documented. The audit catches enumeration drift, not comprehension drift.
|
|
37
|
+
- Doesn't catch capabilities that exist outside the enumerated locations (e.g. an exported class moved to a directory the script doesn't walk). This is a deliberate scope choice; the alternative is walking everything and producing noise.
|
|
38
|
+
- Doesn't catch deleted docs that should still exist (the script measures coverage of what's in code; if code AND docs both lose a capability, neither shows up as drift).
|
|
39
|
+
|
|
40
|
+
## Level-of-abstraction fit
|
|
41
|
+
|
|
42
|
+
The job lives in `src/scaffold/templates/jobs/instar/` alongside the other default jobs. The CI workflow lives in `.github/workflows/` alongside other workflows. Both are at the right level: the job is at the agent-behavior layer (something the agent does), and the workflow is at the project-infrastructure layer (something the project does to itself).
|
|
43
|
+
|
|
44
|
+
The script they both invoke (`scripts/docs-coverage.mjs`) was deliberately built first as a separate concern. This change layers two delivery mechanisms on top of it without modifying the script. If the cadence needs to change (daily, monthly), the script doesn't care.
|
|
45
|
+
|
|
46
|
+
## Signal-vs-authority compliance
|
|
47
|
+
|
|
48
|
+
Both surfaces are **signals**. Neither has authority to fail a build or block work. The per-PR coverage check from the docs-coverage spec is the authoritative gate. The weekly job and the weekly workflow are loud heads-ups, but they can't refuse to ship anything. The separation matches how other signal/authority pairs work in instar.
|
|
49
|
+
|
|
50
|
+
## Interactions with existing systems
|
|
51
|
+
|
|
52
|
+
- **`installBuiltinJobs()`** — picks up the new `docs-coverage-audit.md` template on every install/update. Existing agents get the template (off by default) on their next update via `PostUpdateMigrator`. No separate migration code needed because `installBuiltinJobs` already overwrites built-in templates idempotently.
|
|
53
|
+
- **Scheduler** — sees the new job. With `enabled: false`, the scheduler reads it on startup but doesn't schedule it for execution. No CPU cost.
|
|
54
|
+
- **GitHub Actions cron** — runs in the same project as other workflows but in its own job and on its own schedule. Doesn't share state with any per-PR workflow.
|
|
55
|
+
- **Telegram adapter** — the job uses the same `telegram-reply.sh` path every other job uses for outbound messages. No new adapter integration.
|
|
56
|
+
- **Docs-coverage script** — invoked via `node scripts/docs-coverage.mjs`. The script is deterministic and idempotent, safe to run repeatedly.
|
|
57
|
+
|
|
58
|
+
## Rollback cost
|
|
59
|
+
|
|
60
|
+
Trivially reversible. Remove the two added files. Existing agents lose the template on their next update; the workflow stops running on the next merge.
|
|
61
|
+
|
|
62
|
+
## Risk summary
|
|
63
|
+
|
|
64
|
+
- **Low risk of regression.** Pure additions. Job is off by default; workflow runs independently of any other workflow.
|
|
65
|
+
- **Low risk of friction.** Job stays silent most weeks. Workflow posts to one standing issue rather than spraying alerts.
|
|
66
|
+
- **No risk of data loss.** Both surfaces are read-only against the codebase.
|
|
67
|
+
|
|
68
|
+
## Verification done before commit
|
|
69
|
+
|
|
70
|
+
- Job template parses correctly (verified via reading frontmatter in test fashion).
|
|
71
|
+
- CI workflow YAML validates against GitHub Actions schema (manual review).
|
|
72
|
+
- The underlying script (shipped in the docs-coverage spec) is already tested and verified — no changes needed.
|
|
73
|
+
- `enabled: false` confirmed in the template so this lands silently for existing agents.
|
|
74
|
+
- Distribution path verified: `installBuiltinJobs()` walks `templates/jobs/instar/` and installs all `.md` files regardless of `enabled` flag, so the file reaches every agent on update; the `enabled: false` controls whether the scheduler runs it.
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
# Side-effects review — docs-coverage
|
|
2
|
+
|
|
3
|
+
Spec: `docs/specs/docs-coverage.md`
|
|
4
|
+
ELI16: `docs/specs/docs-coverage.eli16.md`
|
|
5
|
+
|
|
6
|
+
## Surface map
|
|
7
|
+
|
|
8
|
+
| Change | File(s) | Type |
|
|
9
|
+
|---|---|---|
|
|
10
|
+
| New script | `scripts/docs-coverage.mjs` | Pure-Node CLI, no dependencies, runnable from any cwd containing `src/` |
|
|
11
|
+
| New CI job | `.github/workflows/ci.yml` (added `docs-coverage` job) | Runs the script in `--check` mode, uploads report as artifact |
|
|
12
|
+
| New tests | `tests/unit/scripts/docs-coverage.test.ts` | 7 unit tests against a mock repo |
|
|
13
|
+
| gitignore | `.gitignore` (added `.instar/docs-coverage.{json,md}`) | Generated artifacts excluded from commits |
|
|
14
|
+
|
|
15
|
+
No production code touched. No agent behavior changes. No state migrations.
|
|
16
|
+
|
|
17
|
+
## Over-block analysis
|
|
18
|
+
|
|
19
|
+
**Could the CI check block a legitimate PR?**
|
|
20
|
+
|
|
21
|
+
Yes, in three scenarios:
|
|
22
|
+
|
|
23
|
+
1. **A PR introduces a new capability without a doc mention.** This is the desired behavior — the gate exists to require doc updates alongside feature work. Authors fix by adding a mention in the relevant doc.
|
|
24
|
+
2. **A PR removes a documented capability and the floor drops below threshold.** Calibrated initial floors are set 2–3 percentage points below current coverage so normal churn doesn't trip them. A capability removal that drops 5+ percentage points in a single category would trip — and arguably should, because removing a feature is itself a doc-update event.
|
|
25
|
+
3. **A PR adds many capabilities in a category that's already at floor.** A PR adding 10 new routes when the route category is at 13% would drop the category to ~12.5% and fail. The author would have to add doc mentions for at least some of the new routes. Again, desired behavior.
|
|
26
|
+
|
|
27
|
+
**Escape hatches:**
|
|
28
|
+
- Per-category env overrides allow temporarily relaxing a floor for a one-off PR.
|
|
29
|
+
- The check can be removed from the workflow if it ever creates more friction than value (the spec lists this as a rollback path).
|
|
30
|
+
|
|
31
|
+
## Under-block analysis
|
|
32
|
+
|
|
33
|
+
**What does the check NOT catch?**
|
|
34
|
+
|
|
35
|
+
- **Stale doc claims.** A capability can be mentioned with completely wrong information. The script only checks for the substring, not for semantic correctness. This is a genuine limitation — but it's a limitation any automated tool faces, and the manual audit work surfaced these specifically (e.g. configuration.md "all top-level" claim). The agent-driven audit and this script are complementary, not substitutes.
|
|
36
|
+
- **Renamed capabilities.** If a capability is renamed and the doc still references the old name, the doc's mention won't match. This produces a false positive (capability looks undocumented) but the right fix is renaming the doc reference anyway.
|
|
37
|
+
- **Conceptual coverage.** A doc page can describe a feature in detail without ever using the canonical class name or route path. The script would miss this. This is a deliberate trade-off — we'd rather flag a documented-in-spirit capability as missing than under-detect coverage gaps.
|
|
38
|
+
- **Internal capabilities.** The script doesn't enumerate `utils/`, `data/`, `types/` — these are correctly excluded as not-user-facing. If we ever move a user-facing thing into one of those directories, the script wouldn't catch the move.
|
|
39
|
+
|
|
40
|
+
## Level-of-abstraction fit
|
|
41
|
+
|
|
42
|
+
The script lives at the right layer:
|
|
43
|
+
|
|
44
|
+
- Not inside the `pre-commit` gate — that's for per-commit author concerns, and coverage is a PR-level concern.
|
|
45
|
+
- Not inside individual feature workflows — that would scatter the logic across the codebase.
|
|
46
|
+
- In CI as a dedicated job — runs in parallel with other checks, easy to find when it fails, easy to skip when investigating an unrelated failure.
|
|
47
|
+
|
|
48
|
+
The script's enumeration logic is centralized rather than duplicated: every capability type uses the same coverage-scoring function. Adding a new capability type (say, MCP tools or runbooks) is one new `enumerate*()` function, not a separate script.
|
|
49
|
+
|
|
50
|
+
## Signal-vs-authority compliance
|
|
51
|
+
|
|
52
|
+
The script is **authoritative on enumeration** (it walks the source tree, no ambiguity) but **a signal on coverage** (substring matching is approximate; a single mention scores half, two or more score full). The signal-vs-authority separation is respected: the script can refuse a push but it doesn't claim to be the final word on whether docs are good — it just claims to know whether they exist. Quality remains the author's responsibility.
|
|
53
|
+
|
|
54
|
+
The initial floors are calibrated loose so the script doesn't usurp judgment by being too strict on day one. As the doc-update PRs land, the floors ratchet up, and the script's signal becomes louder. This is the right progression.
|
|
55
|
+
|
|
56
|
+
## Interactions with existing systems
|
|
57
|
+
|
|
58
|
+
- **Pre-commit hooks.** No interaction — the script runs in CI, not pre-commit.
|
|
59
|
+
- **pre-push gate** (`scripts/pre-push-gate.js`). No interaction — the script doesn't touch upgrade-notes validation.
|
|
60
|
+
- **Pre-push fixture guard** (`scripts/pre-push-fixture-guard.mjs`). No interaction — different concern.
|
|
61
|
+
- **PostUpdateMigrator.** No interaction — the script doesn't ship to agents, only runs at CI time.
|
|
62
|
+
- **Working-tree integrity check** in CI. No interaction — that check runs after tests; this one runs as its own job in parallel.
|
|
63
|
+
- **Future weekly audit job (Phase 5 of this sprint).** The job will execute this script and surface the report via Telegram. The JSON output is the contract.
|
|
64
|
+
- **lint-no-direct-destructive.js.** No interaction — the script doesn't perform git or fs writes outside `.instar/`.
|
|
65
|
+
|
|
66
|
+
## Rollback cost
|
|
67
|
+
|
|
68
|
+
Trivially reversible. Remove three files (script, test, the workflow job block) plus the two `.gitignore` lines. No data migration, no schema change, no agent-state mutation. Reverting leaves the repo at exactly its pre-change state.
|
|
69
|
+
|
|
70
|
+
## Risk summary
|
|
71
|
+
|
|
72
|
+
- **Low risk of regression.** Pure-Node script, deterministic enumeration, no side effects outside writing two files under `.instar/`. Cannot break any production code path.
|
|
73
|
+
- **Moderate friction risk.** Authors who add features without docs will hit the gate. This is the intended behavior, but it will surprise people the first few times. Mitigation: the spec + ELI16 are checked in alongside the script, so the failure message has a clear pointer to "what's going on and how to fix it."
|
|
74
|
+
- **No risk of data loss.** The script reads only; the only writes are the two artifact files in `.instar/`, both git-ignored.
|
|
75
|
+
|
|
76
|
+
## Verification done before commit
|
|
77
|
+
|
|
78
|
+
- Script enumerates 880 capabilities across six types on current main.
|
|
79
|
+
- `--check` exits 0 with floors calibrated to current state.
|
|
80
|
+
- `--check` exits 1 with a high floor (verified in test `--check fails when any category is below floor`).
|
|
81
|
+
- All seven unit tests pass.
|
|
82
|
+
- `npm run lint` passes (TypeScript, destructive-ops lint, LLM-HTTP lint, Codex Rule 1 drift).
|
|
83
|
+
- The mock-repo test mocks the same six capability types and verifies coverage classification.
|
|
84
|
+
- Spec carries `approved: true` per direct principal authorization in Telegram topic 11235 (the autonomous-mode instruction was "Complete option 3", referring to the documentation-coverage script described as option 3 in the audit report shared minutes earlier).
|