@onlooker-community/ecosystem 0.28.0 → 0.29.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +13 -0
- package/.claude-plugin/plugin.json +1 -1
- package/.release-please-manifest.json +2 -2
- package/CHANGELOG.md +14 -0
- package/CLAUDE.md +2 -0
- package/docs/plugin-catalog.md +125 -0
- package/package.json +3 -3
- package/plugins/compass/.claude-plugin/plugin.json +1 -1
- package/plugins/compass/CHANGELOG.md +14 -0
- package/plugins/compass/README.md +1 -3
- package/plugins/compass/config.json +1 -2
- package/plugins/compass/docs/design.md +1 -2
- package/plugins/compass/scripts/hooks/compass-bash-gate.sh +8 -1
- package/plugins/compass/scripts/hooks/compass-pre-tool-use.sh +17 -5
- package/plugins/compass/scripts/hooks/compass-record-write.sh +5 -0
- package/plugins/compass/scripts/hooks/compass-session-start.sh +0 -8
- package/plugins/compass/scripts/lib/compass-evaluator.sh +58 -98
- package/plugins/compass/scripts/lib/compass-gate.sh +15 -18
- package/plugins/compass/scripts/lib/compass-sanitizer.sh +4 -4
- package/plugins/compass/scripts/lib/compass-transcript.sh +79 -112
- package/plugins/inspector/.claude-plugin/plugin.json +14 -0
- package/plugins/inspector/README.md +155 -0
- package/plugins/inspector/config.json +25 -0
- package/plugins/inspector/docs/design.md +286 -0
- package/plugins/inspector/hooks/hooks.json +33 -0
- package/plugins/inspector/scripts/hooks/inspector-post-write.sh +124 -0
- package/plugins/inspector/scripts/lib/inspector-config.sh +108 -0
- package/plugins/inspector/scripts/lib/inspector-events.sh +82 -0
- package/plugins/inspector/scripts/lib/inspector-project-key.sh +55 -0
- package/plugins/inspector/scripts/lib/inspector-run.sh +305 -0
- package/plugins/inspector/scripts/lib/inspector-ulid.sh +45 -0
- package/test/bats/archivist-project-key.bats +79 -0
- package/test/bats/archivist-storage.bats +79 -0
- package/test/bats/compact-tracker.bats +125 -0
- package/test/bats/compass-config.bats +65 -0
- package/test/bats/compass-gate.bats +129 -0
- package/test/bats/compass-sanitizer.bats +69 -0
- package/test/bats/compass-symbolic-skip.bats +88 -0
- package/test/bats/compass-transcript.bats +80 -0
- package/test/bats/inspector-config.bats +118 -0
- package/test/bats/inspector-events.bats +156 -0
- package/test/bats/inspector-post-write-hook.bats +164 -0
- package/test/bats/inspector-project-key.bats +68 -0
- package/test/bats/inspector-ulid.bats +34 -0
- package/test/bats/onlooker-schema.bats +111 -0
- package/test/bats/prompt-rules.bats +98 -0
- package/test/bats/session-tracker.bats +260 -0
- package/test/bats/skill-usage-tracker.bats +63 -0
- package/test/bats/task-tracker.bats +102 -0
- package/test/bats/turn-tracker.bats +180 -0
- package/test/bats/validate-path.bats +125 -0
- package/test/bats/worktree-tracker.bats +167 -0
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
# Inspector
|
|
2
|
+
|
|
3
|
+
Per-edit lint and typecheck gate for the Onlooker ecosystem — runs the project's
|
|
4
|
+
configured checks on **just the touched file** after every `Write`, `Edit`, and
|
|
5
|
+
`MultiEdit`, so the agent sees its own lint and type errors before it claims
|
|
6
|
+
success.
|
|
7
|
+
|
|
8
|
+
Inspector is a sibling plugin to [`ecosystem`](../../) and assumes the Onlooker
|
|
9
|
+
observability substrate (`~/.onlooker/`) is present.
|
|
10
|
+
|
|
11
|
+
## Why it exists
|
|
12
|
+
|
|
13
|
+
The ecosystem already has plugins that judge agent output after the fact and
|
|
14
|
+
plugins that gate ambiguous writes before they happen. What it didn't have until
|
|
15
|
+
now is a fast feedback loop that runs after every edit and tells the agent
|
|
16
|
+
*whether the code it just wrote actually compiles*. Inspector is that loop.
|
|
17
|
+
|
|
18
|
+
- **Not [proctor]** (planned) — proctor runs the project's full verification
|
|
19
|
+
command at `Stop`. Inspector runs only on touched files, only on `PostToolUse`.
|
|
20
|
+
Cheaper, fires far more often, narrower scope.
|
|
21
|
+
- **Not [assayer]** — assayer parses the agent's final message for testable
|
|
22
|
+
claims and cross-checks them against actual exit codes in the transcript.
|
|
23
|
+
Assayer catches the agent *lying* about claims. Inspector ensures the agent
|
|
24
|
+
has *accurate ground truth* to make claims from. They compose: inspector
|
|
25
|
+
emits real pass/fail signals; assayer can later confirm the agent's claims
|
|
26
|
+
line up with those signals.
|
|
27
|
+
- **Not a build system** — inspector runs the configured command, captures the
|
|
28
|
+
result, emits an event, exits. No cross-file caching, no dependency graphs.
|
|
29
|
+
|
|
30
|
+
## How it works
|
|
31
|
+
|
|
32
|
+
| Hook | Matcher | What Inspector does |
|
|
33
|
+
|------|---------|---------------------|
|
|
34
|
+
| `PostToolUse` | `Edit`, `Write`, `MultiEdit` | Resolves the touched file from `tool_input.file_path`, looks up the configured checks for the file's extension, runs each check with a per-check timeout, and emits `inspector.check.passed` / `.failed` / `.skipped` plus a single `inspector.run.completed` summary. Bounded by `total_timeout_seconds`. Always exits 0 — inspector is advisory and never blocks the tool call. |
|
|
35
|
+
|
|
36
|
+
The hook's stdout (the additional-context channel for `PostToolUse` hooks) is
|
|
37
|
+
the agent-facing summary. By default it's quiet on clean runs:
|
|
38
|
+
|
|
39
|
+
```
|
|
40
|
+
inspector: src/cart.ts
|
|
41
|
+
✗ biome (3 issues, exit 1)
|
|
42
|
+
src/cart.ts:42:5 — Unused variable 'subtotal'
|
|
43
|
+
src/cart.ts:51:3 — Missing return type annotation
|
|
44
|
+
src/cart.ts:64:9 — Unreachable code
|
|
45
|
+
✗ tsc (1 issue(s), exit 2)
|
|
46
|
+
src/cart.ts:42:5 — Type 'string | undefined' is not assignable to 'string'
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Set `inspector.show_clean_runs: true` to surface the file header on passing
|
|
50
|
+
checks too. The agent sees this on its next turn.
|
|
51
|
+
|
|
52
|
+
## Activation
|
|
53
|
+
|
|
54
|
+
Inspector ships disabled. Opt in per project (or globally) by adding the
|
|
55
|
+
`inspector` block to `.claude/settings.json`:
|
|
56
|
+
|
|
57
|
+
```jsonc
|
|
58
|
+
{
|
|
59
|
+
"inspector": {
|
|
60
|
+
"enabled": true,
|
|
61
|
+
"checks": {
|
|
62
|
+
".ts": [{ "name": "biome", "kind": "lint", "argv": ["biome", "check", "${file}"] },
|
|
63
|
+
{ "name": "tsc", "kind": "typecheck", "argv": ["tsc", "--noEmit"] }],
|
|
64
|
+
".tsx": [{ "name": "biome", "kind": "lint", "argv": ["biome", "check", "${file}"] },
|
|
65
|
+
{ "name": "tsc", "kind": "typecheck", "argv": ["tsc", "--noEmit"] }],
|
|
66
|
+
".py": [{ "name": "ruff", "kind": "lint", "argv": ["ruff", "check", "${file}"] }],
|
|
67
|
+
".sh": [{ "name": "shellcheck", "kind": "lint", "argv": ["shellcheck", "${file}"] }]
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Each check is an `{ name, kind, argv }` object. `kind` is one of `lint` or
|
|
74
|
+
`typecheck` (used for downstream grouping). `argv` is the literal argv array;
|
|
75
|
+
the following placeholders are expanded before exec:
|
|
76
|
+
|
|
77
|
+
| Placeholder | Resolves to |
|
|
78
|
+
|-------------------|--------------------------------------------|
|
|
79
|
+
| `${file}` | absolute path to the touched file |
|
|
80
|
+
| `${file_relative}`| path relative to the repo root |
|
|
81
|
+
| `${repo_root}` | the repo's `git rev-parse --show-toplevel` |
|
|
82
|
+
|
|
83
|
+
A bare argv array (`["shellcheck", "${file}"]`) is also accepted as a shorthand
|
|
84
|
+
— inspector treats the first entry as the check name and the kind as `lint`.
|
|
85
|
+
|
|
86
|
+
## Config
|
|
87
|
+
|
|
88
|
+
| Field | Default | Meaning |
|
|
89
|
+
|---|---|---|
|
|
90
|
+
| `enabled` | `false` | Master switch. |
|
|
91
|
+
| `timeout_seconds_per_check` | `10` | Wall-clock cap per check. Exceeded → `inspector.check.skipped` with `reason: "timeout"`. |
|
|
92
|
+
| `total_timeout_seconds` | `30` | Wall-clock cap for the whole run. Remaining checks emit `.skipped` with `reason: "total_budget_exhausted"`. |
|
|
93
|
+
| `output_excerpt_max_bytes` | `4096` | Cap on captured output, both in the event and shown to the agent. Excess is replaced with `…[truncated]`. |
|
|
94
|
+
| `show_clean_runs` | `false` | If `true`, the agent-facing summary includes passing checks too. Off by default to keep token usage low. |
|
|
95
|
+
| `exclude_paths` | `["node_modules", ".git", "vendor", ".venv", "dist", ".next", ".nuxt", "build", "__pycache__", "target", "coverage"]` | Containment match against the file's path relative to the repo root. A match emits `inspector.check.skipped` with `reason: "excluded_path"` and runs no checks. |
|
|
96
|
+
| `checks` | `{}` | Map of file extension (with leading dot) to an array of check definitions. Empty by default — opt in per project. |
|
|
97
|
+
|
|
98
|
+
Config precedence: plugin defaults < `~/.claude/settings.json` (`.inspector`) <
|
|
99
|
+
`<repo>/.claude/settings.json` (`.inspector`). Each layer fully replaces the
|
|
100
|
+
`checks` array for a given extension; per-entry deep-merge is intentionally not
|
|
101
|
+
supported because the override behavior would be unpredictable.
|
|
102
|
+
|
|
103
|
+
## Events
|
|
104
|
+
|
|
105
|
+
All events are registered in `@onlooker-community/schema`.
|
|
106
|
+
|
|
107
|
+
| Event | When | Notable payload fields |
|
|
108
|
+
|---|---|---|
|
|
109
|
+
| `inspector.check.passed` | A check returned exit 0 | `file_path`, `tool_name`, `check_name`, `check_kind`, `argv`, `duration_ms` |
|
|
110
|
+
| `inspector.check.failed` | A check returned non-zero | `exit_code`, `issue_count` (best-effort, may be `null`), `output_excerpt`, `output_truncated` |
|
|
111
|
+
| `inspector.check.skipped` | A check or whole file was not run | `reason`: one of `disabled`, `excluded_path`, `no_extension_match`, `not_in_repo`, `tool_missing`, `timeout`, `total_budget_exhausted` |
|
|
112
|
+
| `inspector.run.completed` | Once per hook fire after all checks | `checks_run`, `checks_passed`, `checks_failed`, `checks_skipped`, `duration_ms` |
|
|
113
|
+
|
|
114
|
+
Downstream consumers that just want "did this edit produce broken code?" read
|
|
115
|
+
`inspector.run.completed` and check `checks_failed > 0`. Consumers that want
|
|
116
|
+
per-tool detail subscribe to `inspector.check.*`.
|
|
117
|
+
|
|
118
|
+
## Whole-project checks (tsc, mypy, …)
|
|
119
|
+
|
|
120
|
+
TypeScript's typecheck is project-scoped: `tsc --noEmit` checks every file, not
|
|
121
|
+
just the touched one. There is no meaningful "tsc on one file." The supported
|
|
122
|
+
pattern is to run the full project tsc and rely on its incremental cache
|
|
123
|
+
(`tsBuildInfoFile`) to keep latency down. Same applies to mypy, cargo check,
|
|
124
|
+
and golangci-lint.
|
|
125
|
+
|
|
126
|
+
The downside is that `tsc --noEmit` reports errors in *every* file. Inspector's
|
|
127
|
+
v1 surfaces all of them to the agent. A follow-up will add an opt-in filter
|
|
128
|
+
that shows only errors mentioning the touched file plus a collateral-error
|
|
129
|
+
count.
|
|
130
|
+
|
|
131
|
+
## Failure modes
|
|
132
|
+
|
|
133
|
+
Inspector is advisory — it never blocks the tool call. Specifically:
|
|
134
|
+
|
|
135
|
+
- Missing tool on PATH → `.skipped` event, no agent-facing output
|
|
136
|
+
- Timeout → `.skipped` event, one-line note to the agent
|
|
137
|
+
- Hook script error → exit 0 with no event (last-resort path)
|
|
138
|
+
- Schema validation error → logged to stderr, hook continues
|
|
139
|
+
|
|
140
|
+
## Compatibility
|
|
141
|
+
|
|
142
|
+
- bash 3.2+ (the macOS system bash is supported; no `mapfile` / `readarray` /
|
|
143
|
+
associative arrays)
|
|
144
|
+
- `jq` is required (already a hard requirement for the ecosystem substrate)
|
|
145
|
+
- `timeout` from coreutils is used when present; falls back to no-timeout mode
|
|
146
|
+
when absent (and emits a warning to the inspector hook log)
|
|
147
|
+
|
|
148
|
+
## Design
|
|
149
|
+
|
|
150
|
+
See [`docs/design.md`](docs/design.md) for the full design record, including
|
|
151
|
+
the rationale for project-wide check semantics, output filtering, and open
|
|
152
|
+
questions.
|
|
153
|
+
|
|
154
|
+
[proctor]: https://github.com/onlooker-community/ecosystem#planned
|
|
155
|
+
[assayer]: ../assayer
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
{
|
|
2
|
+
"plugin_name": "inspector",
|
|
3
|
+
"storage_path": "~/.onlooker",
|
|
4
|
+
"inspector": {
|
|
5
|
+
"enabled": false,
|
|
6
|
+
"timeout_seconds_per_check": 10,
|
|
7
|
+
"total_timeout_seconds": 30,
|
|
8
|
+
"output_excerpt_max_bytes": 4096,
|
|
9
|
+
"show_clean_runs": false,
|
|
10
|
+
"exclude_paths": [
|
|
11
|
+
"node_modules",
|
|
12
|
+
".git",
|
|
13
|
+
"vendor",
|
|
14
|
+
".venv",
|
|
15
|
+
"dist",
|
|
16
|
+
".next",
|
|
17
|
+
".nuxt",
|
|
18
|
+
"build",
|
|
19
|
+
"__pycache__",
|
|
20
|
+
"target",
|
|
21
|
+
"coverage"
|
|
22
|
+
],
|
|
23
|
+
"checks": {}
|
|
24
|
+
}
|
|
25
|
+
}
|
|
@@ -0,0 +1,286 @@
|
|
|
1
|
+
# Inspector — Design
|
|
2
|
+
|
|
3
|
+
**Layer:** verification / execution
|
|
4
|
+
**Hook surface:** `PostToolUse` for `Write`, `Edit`, `MultiEdit`
|
|
5
|
+
**Status:** initial design — first implementation under this doc
|
|
6
|
+
|
|
7
|
+
Inspector is the per-edit lint and typecheck gate. Every time the agent writes
|
|
8
|
+
to a file, inspector runs the project's configured checks on **just that file**
|
|
9
|
+
and surfaces the result back to the agent before its next turn. The agent sees
|
|
10
|
+
its own type errors immediately and self-corrects, instead of claiming success
|
|
11
|
+
on broken code and getting caught later by [assayer] or a human reviewer.
|
|
12
|
+
|
|
13
|
+
## What inspector is — and isn't
|
|
14
|
+
|
|
15
|
+
- **Inspector is not proctor.** Proctor (planned) runs the project's full
|
|
16
|
+
verification command (`npm test`, `mise run check`, …) at `Stop`. Inspector
|
|
17
|
+
runs only on touched files, only on `PostToolUse`. Cheaper, fires far more
|
|
18
|
+
often, narrower scope.
|
|
19
|
+
- **Inspector is not assayer.** Assayer (shipped) parses the agent's final
|
|
20
|
+
message for testable claims ("the build passes", "I ran the tests") and
|
|
21
|
+
cross-checks them against actual exit codes in the transcript. Assayer catches
|
|
22
|
+
the agent *lying* about claims. Inspector ensures the agent has *accurate
|
|
23
|
+
ground truth* to make claims from. They compose: inspector emits real
|
|
24
|
+
pass/fail signals; assayer can later confirm the agent's claims line up with
|
|
25
|
+
those signals.
|
|
26
|
+
- **Inspector is not a build system.** It does not chain dependencies, cache
|
|
27
|
+
intermediate results, or share state across invocations beyond timeouts. It
|
|
28
|
+
runs the configured check, captures the result, emits an event, exits. If
|
|
29
|
+
configuration says "run tsc on a single file," it does that — even though
|
|
30
|
+
tsc's per-file mode loses project context. Choosing the right command for a
|
|
31
|
+
given language is the user's job; inspector executes what it is told.
|
|
32
|
+
|
|
33
|
+
## Hook flow
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
PostToolUse(Write|Edit|MultiEdit)
|
|
37
|
+
→ inspector-post-write.sh
|
|
38
|
+
→ resolve touched file from tool_input.file_path
|
|
39
|
+
→ load merged config (plugin defaults < home < repo)
|
|
40
|
+
→ if disabled OR excluded path OR no extension match → emit .skipped, exit 0
|
|
41
|
+
→ for each configured check matching the extension:
|
|
42
|
+
→ expand ${file}, ${file_relative}, ${repo_root} in the command
|
|
43
|
+
→ run with per-check timeout
|
|
44
|
+
→ emit inspector.check.passed / .failed / .skipped
|
|
45
|
+
→ exit 0 (never blocks the tool call)
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
The hook always exits 0. Inspector is advisory — it never blocks the agent's
|
|
49
|
+
write. It surfaces what it found in the additional-context channel of the
|
|
50
|
+
PostToolUse hook reply so the agent sees the result on its next turn.
|
|
51
|
+
|
|
52
|
+
## Configuration
|
|
53
|
+
|
|
54
|
+
The minimum useful configuration is a map from file extension to a list of
|
|
55
|
+
commands to run. Each command is an argv array; `${file}` substitutes the
|
|
56
|
+
canonical absolute path to the touched file.
|
|
57
|
+
|
|
58
|
+
```jsonc
|
|
59
|
+
{
|
|
60
|
+
"inspector": {
|
|
61
|
+
"enabled": false,
|
|
62
|
+
"timeout_seconds_per_check": 10,
|
|
63
|
+
"total_timeout_seconds": 30,
|
|
64
|
+
"exclude_paths": ["node_modules", ".git", "vendor", "dist", "build",
|
|
65
|
+
".next", "__pycache__", ".venv"],
|
|
66
|
+
"checks": {
|
|
67
|
+
".ts": [{ "name": "biome", "argv": ["biome", "check", "${file}"] },
|
|
68
|
+
{ "name": "tsc", "argv": ["tsc", "--noEmit"] }],
|
|
69
|
+
".tsx": [{ "name": "biome", "argv": ["biome", "check", "${file}"] },
|
|
70
|
+
{ "name": "tsc", "argv": ["tsc", "--noEmit"] }],
|
|
71
|
+
".py": [{ "name": "ruff", "argv": ["ruff", "check", "${file}"] }],
|
|
72
|
+
".sh": [{ "name": "shellcheck", "argv": ["shellcheck", "${file}"] }]
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Merging follows the standard ecosystem precedence (plugin defaults < home <
|
|
79
|
+
repo). Each layer can fully replace `checks` for a given extension by setting
|
|
80
|
+
the array; deep-merge of individual entries within an extension is intentionally
|
|
81
|
+
not supported — it makes the override behavior unpredictable.
|
|
82
|
+
|
|
83
|
+
`config.json` ships with `enabled: false` to match the rest of the ecosystem.
|
|
84
|
+
The first PR that ships inspector also adds an opt-in path in the README.
|
|
85
|
+
|
|
86
|
+
## Path handling
|
|
87
|
+
|
|
88
|
+
- `file_path` is resolved via `realpath` where available, falling back to
|
|
89
|
+
`readlink -f`, falling back to the raw input.
|
|
90
|
+
- The file must be inside the current repo root (`git rev-parse --show-toplevel`
|
|
91
|
+
from `cwd`). Out-of-tree writes are skipped — inspector is per-project.
|
|
92
|
+
- `exclude_paths` is matched against the file's path relative to the repo root.
|
|
93
|
+
Match semantics are *containment* (`vendor/foo.ts` matches `vendor`), not
|
|
94
|
+
glob. Compass uses globs; inspector uses containment because the use case is
|
|
95
|
+
"skip this whole directory tree."
|
|
96
|
+
|
|
97
|
+
## Per-check execution
|
|
98
|
+
|
|
99
|
+
Each check runs in the repo root (`cwd`) with the following environment:
|
|
100
|
+
|
|
101
|
+
- inherited PATH plus any `mise`-shimmed bins (already set up at session start)
|
|
102
|
+
- `INSPECTOR_FILE` — absolute path to the touched file
|
|
103
|
+
- `INSPECTOR_FILE_RELATIVE` — relative to repo root
|
|
104
|
+
- `INSPECTOR_REPO_ROOT` — the repo root
|
|
105
|
+
- `INSPECTOR_PROJECT_KEY` — the project key
|
|
106
|
+
|
|
107
|
+
`timeout_seconds_per_check` is enforced via the `timeout` command (or a
|
|
108
|
+
fallback bash trap on systems that lack it). On timeout, inspector emits
|
|
109
|
+
`inspector.check.skipped` with `reason: "timeout"`.
|
|
110
|
+
|
|
111
|
+
If the command's first argv entry is not on PATH, inspector emits
|
|
112
|
+
`inspector.check.skipped` with `reason: "tool_missing"`. This is the dominant
|
|
113
|
+
"new project, lint not installed yet" case and must be quiet — no error in the
|
|
114
|
+
hook output, just the skipped event in the log.
|
|
115
|
+
|
|
116
|
+
## What the agent sees
|
|
117
|
+
|
|
118
|
+
The hook's stdout (the additional-context channel for PostToolUse hooks)
|
|
119
|
+
contains a compact, one-line-per-finding summary:
|
|
120
|
+
|
|
121
|
+
```
|
|
122
|
+
inspector: src/cart.ts
|
|
123
|
+
✗ biome (3 issues)
|
|
124
|
+
src/cart.ts:42:5 — Unused variable 'subtotal'
|
|
125
|
+
src/cart.ts:51:3 — Missing return type annotation
|
|
126
|
+
src/cart.ts:64:9 — Unreachable code
|
|
127
|
+
✗ tsc (1 error)
|
|
128
|
+
src/cart.ts:42:5 — Type 'string | undefined' is not assignable to 'string'
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
On clean runs, inspector either emits nothing to stdout (the silent case) or a
|
|
132
|
+
single confirmation line, controlled by `inspector.show_clean_runs`
|
|
133
|
+
(default `false`). Default silence avoids token spam on every edit.
|
|
134
|
+
|
|
135
|
+
Output capture per check is bounded by `inspector.output_excerpt_max_bytes`
|
|
136
|
+
(default 4096) — beyond that, inspector truncates with a `…[truncated]` marker
|
|
137
|
+
both in the agent-facing output and in the event payload.
|
|
138
|
+
|
|
139
|
+
## tsc and other whole-project checks
|
|
140
|
+
|
|
141
|
+
TypeScript's typecheck is project-scoped: `tsc --noEmit` checks every file in
|
|
142
|
+
the project, not just the touched one. Inspector cannot meaningfully run "tsc
|
|
143
|
+
on a single file" — `tsc --noEmit src/foo.ts` runs in a degraded mode that
|
|
144
|
+
loses project context (imports, references, lib types).
|
|
145
|
+
|
|
146
|
+
The right behavior is to run `tsc --noEmit -p tsconfig.json` (full project)
|
|
147
|
+
and filter the output to errors that mention the touched file. The first
|
|
148
|
+
implementation runs the full project tsc and post-filters. This is more
|
|
149
|
+
expensive than the lint case but cached by tsc's incremental compilation
|
|
150
|
+
(`tsBuildInfoFile`). Users who care about latency can disable the tsc check
|
|
151
|
+
per-project.
|
|
152
|
+
|
|
153
|
+
Other languages with similar whole-project semantics (mypy, cargo check,
|
|
154
|
+
golangci-lint) follow the same pattern: run the project-wide command, filter
|
|
155
|
+
results to the touched file. Users opt into these in `config.json` because the
|
|
156
|
+
cost is higher than per-file lint.
|
|
157
|
+
|
|
158
|
+
## Events
|
|
159
|
+
|
|
160
|
+
All four events are registered in `@onlooker-community/schema` under
|
|
161
|
+
`plugins-verification.json`.
|
|
162
|
+
|
|
163
|
+
### `inspector.check.passed`
|
|
164
|
+
|
|
165
|
+
Emitted once per check that passed cleanly.
|
|
166
|
+
|
|
167
|
+
```jsonc
|
|
168
|
+
{
|
|
169
|
+
"file_path": "/abs/path/to/src/cart.ts",
|
|
170
|
+
"file_path_relative": "src/cart.ts",
|
|
171
|
+
"tool_name": "Edit", // Write | Edit | MultiEdit
|
|
172
|
+
"check_name": "biome",
|
|
173
|
+
"check_kind": "lint", // lint | typecheck
|
|
174
|
+
"argv": ["biome", "check", "/abs/path/to/src/cart.ts"],
|
|
175
|
+
"duration_ms": 124
|
|
176
|
+
}
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
### `inspector.check.failed`
|
|
180
|
+
|
|
181
|
+
Emitted once per check that returned a non-zero exit code.
|
|
182
|
+
|
|
183
|
+
```jsonc
|
|
184
|
+
{
|
|
185
|
+
"file_path": "/abs/path/to/src/cart.ts",
|
|
186
|
+
"file_path_relative": "src/cart.ts",
|
|
187
|
+
"tool_name": "Edit",
|
|
188
|
+
"check_name": "tsc",
|
|
189
|
+
"check_kind": "typecheck",
|
|
190
|
+
"argv": ["tsc", "--noEmit"],
|
|
191
|
+
"duration_ms": 980,
|
|
192
|
+
"exit_code": 2,
|
|
193
|
+
"issue_count": 3,
|
|
194
|
+
"output_excerpt": "src/cart.ts:42:5 — Type 'string | undefined' is not assignable to 'string'\n…"
|
|
195
|
+
}
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
`issue_count` is best-effort: inspector parses common output formats (one
|
|
199
|
+
issue per non-empty line, ignoring obvious headers/footers) where possible
|
|
200
|
+
and falls back to `null` when the format is unknown.
|
|
201
|
+
|
|
202
|
+
### `inspector.check.skipped`
|
|
203
|
+
|
|
204
|
+
Emitted when a check did not run.
|
|
205
|
+
|
|
206
|
+
```jsonc
|
|
207
|
+
{
|
|
208
|
+
"file_path": "/abs/path/to/src/cart.ts",
|
|
209
|
+
"file_path_relative": "src/cart.ts",
|
|
210
|
+
"tool_name": "Edit",
|
|
211
|
+
"check_name": "tsc", // optional — absent for whole-file skips
|
|
212
|
+
"reason": "tool_missing" // tool_missing | disabled | excluded_path
|
|
213
|
+
// | no_extension_match | timeout
|
|
214
|
+
// | not_in_repo | total_budget_exhausted
|
|
215
|
+
}
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
### `inspector.run.completed`
|
|
219
|
+
|
|
220
|
+
Emitted once per hook invocation, after all per-check events for that file.
|
|
221
|
+
|
|
222
|
+
```jsonc
|
|
223
|
+
{
|
|
224
|
+
"file_path": "/abs/path/to/src/cart.ts",
|
|
225
|
+
"file_path_relative": "src/cart.ts",
|
|
226
|
+
"tool_name": "Edit",
|
|
227
|
+
"checks_run": 2,
|
|
228
|
+
"checks_passed": 0,
|
|
229
|
+
"checks_failed": 2,
|
|
230
|
+
"checks_skipped": 0,
|
|
231
|
+
"duration_ms": 1104
|
|
232
|
+
}
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
Downstream consumers that just want "did this edit produce broken code?" read
|
|
236
|
+
`inspector.run.completed` and check `checks_failed > 0`. Consumers that want
|
|
237
|
+
per-tool detail subscribe to `inspector.check.*`.
|
|
238
|
+
|
|
239
|
+
## Project-key derivation
|
|
240
|
+
|
|
241
|
+
Same algorithm as the rest of the ecosystem:
|
|
242
|
+
|
|
243
|
+
```
|
|
244
|
+
SHA256("remote:" + git remote get-url origin) → first 12 hex chars
|
|
245
|
+
→ fall back to SHA256("root:" + git rev-parse --show-toplevel)[:12]
|
|
246
|
+
→ fall back to SHA256("cwd:" + pwd)[:12]
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
Implemented in `plugins/inspector/scripts/lib/inspector-project-key.sh`,
|
|
250
|
+
mirroring the equivalent helper in cartographer.
|
|
251
|
+
|
|
252
|
+
## Failure modes and fail-soft behavior
|
|
253
|
+
|
|
254
|
+
Inspector is advisory. It never blocks the tool call. Specifically:
|
|
255
|
+
|
|
256
|
+
- Missing tool on PATH → `.skipped` event, no agent-facing output
|
|
257
|
+
- Timeout → `.skipped` event, agent sees `inspector: src/foo.ts — biome timed out`
|
|
258
|
+
- Hook script error → exit 0 with no event (last-resort; logged to
|
|
259
|
+
`~/.onlooker/inspector/<project>/hook.log`)
|
|
260
|
+
- Schema validation error → logged to stderr, hook continues
|
|
261
|
+
- Concurrent invocation → no lock; each check runs independently. tsc's own
|
|
262
|
+
incremental cache handles concurrent invocations safely
|
|
263
|
+
|
|
264
|
+
The "exit 0 with no event" path is the only case where inspector goes silent.
|
|
265
|
+
This is deliberate: inspector is a noticeably new behavior surface; bugs in the
|
|
266
|
+
hook script must not block writes.
|
|
267
|
+
|
|
268
|
+
## Open questions
|
|
269
|
+
|
|
270
|
+
These are deferred to a follow-up ADR after first real-world use.
|
|
271
|
+
|
|
272
|
+
1. **Output filtering for whole-project checks.** Filtering tsc/mypy output to
|
|
273
|
+
the touched file is necessary for "only what you broke just now" UX. But
|
|
274
|
+
compile errors in unrelated files often *are* caused by this edit (a removed
|
|
275
|
+
export breaks five importers). Showing only the touched-file errors hides
|
|
276
|
+
real damage; showing all errors floods the channel. Tentative: show only
|
|
277
|
+
touched-file errors by default; add `inspector.show_collateral_errors`
|
|
278
|
+
config knob if requests come in.
|
|
279
|
+
2. **Caching.** Repeated edits to the same file within seconds will re-run all
|
|
280
|
+
checks. Worth caching at the level of "skip identical file content"? Not
|
|
281
|
+
for v1.
|
|
282
|
+
3. **Parallel checks.** Checks for the same file run sequentially today. Worth
|
|
283
|
+
parallelizing? Not for v1 — most lints finish in <500ms and bash-level
|
|
284
|
+
parallelism with timeouts is fiddly.
|
|
285
|
+
|
|
286
|
+
[assayer]: ../../assayer/docs/design.md
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
{
|
|
2
|
+
"hooks": {
|
|
3
|
+
"PostToolUse": [
|
|
4
|
+
{
|
|
5
|
+
"matcher": "Write",
|
|
6
|
+
"hooks": [
|
|
7
|
+
{
|
|
8
|
+
"type": "command",
|
|
9
|
+
"command": "\"$CLAUDE_PLUGIN_ROOT\"/scripts/hooks/inspector-post-write.sh"
|
|
10
|
+
}
|
|
11
|
+
]
|
|
12
|
+
},
|
|
13
|
+
{
|
|
14
|
+
"matcher": "Edit",
|
|
15
|
+
"hooks": [
|
|
16
|
+
{
|
|
17
|
+
"type": "command",
|
|
18
|
+
"command": "\"$CLAUDE_PLUGIN_ROOT\"/scripts/hooks/inspector-post-write.sh"
|
|
19
|
+
}
|
|
20
|
+
]
|
|
21
|
+
},
|
|
22
|
+
{
|
|
23
|
+
"matcher": "MultiEdit",
|
|
24
|
+
"hooks": [
|
|
25
|
+
{
|
|
26
|
+
"type": "command",
|
|
27
|
+
"command": "\"$CLAUDE_PLUGIN_ROOT\"/scripts/hooks/inspector-post-write.sh"
|
|
28
|
+
}
|
|
29
|
+
]
|
|
30
|
+
}
|
|
31
|
+
]
|
|
32
|
+
}
|
|
33
|
+
}
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# inspector-post-write.sh — PostToolUse hook for Write / Edit / MultiEdit.
|
|
3
|
+
#
|
|
4
|
+
# Runs the project's configured lint and typecheck commands on just the
|
|
5
|
+
# touched file. Emits inspector.check.* and inspector.run.completed events.
|
|
6
|
+
# Surfaces a compact summary on stdout for the agent's next turn.
|
|
7
|
+
# Always exits 0 — inspector is advisory.
|
|
8
|
+
|
|
9
|
+
set -uo pipefail
|
|
10
|
+
|
|
11
|
+
# Recursion guard — prevents inspector from re-triggering itself if a check
|
|
12
|
+
# command happens to write to a watched file via its own tooling.
|
|
13
|
+
[[ "${INSPECTOR_NESTED:-0}" == "1" ]] && exit 0
|
|
14
|
+
export INSPECTOR_NESTED=1
|
|
15
|
+
|
|
16
|
+
PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(cd "$(dirname "$0")/../.." && pwd)}"
|
|
17
|
+
export CLAUDE_PLUGIN_ROOT="$PLUGIN_ROOT"
|
|
18
|
+
|
|
19
|
+
source "$PLUGIN_ROOT/scripts/lib/inspector-config.sh"
|
|
20
|
+
source "$PLUGIN_ROOT/scripts/lib/inspector-project-key.sh"
|
|
21
|
+
source "$PLUGIN_ROOT/scripts/lib/inspector-events.sh"
|
|
22
|
+
source "$PLUGIN_ROOT/scripts/lib/inspector-run.sh"
|
|
23
|
+
|
|
24
|
+
HOOK_INPUT=$(cat)
|
|
25
|
+
CWD=$(printf '%s' "$HOOK_INPUT" | jq -r '.cwd // empty' 2>/dev/null)
|
|
26
|
+
_HOOK_SESSION_ID=$(printf '%s' "$HOOK_INPUT" | jq -r '.session_id // empty' 2>/dev/null)
|
|
27
|
+
TOOL_NAME=$(printf '%s' "$HOOK_INPUT" | jq -r '.tool_name // empty' 2>/dev/null)
|
|
28
|
+
export _HOOK_SESSION_ID
|
|
29
|
+
|
|
30
|
+
# Bail on missing input — never block the tool call.
|
|
31
|
+
[[ -z "$CWD" ]] && exit 0
|
|
32
|
+
case "$TOOL_NAME" in
|
|
33
|
+
Write|Edit|MultiEdit) ;;
|
|
34
|
+
*) exit 0 ;;
|
|
35
|
+
esac
|
|
36
|
+
export INSPECTOR_TOOL_NAME="$TOOL_NAME"
|
|
37
|
+
|
|
38
|
+
# Resolve touched file from tool input.
|
|
39
|
+
TOOL_TARGET=$(printf '%s' "$HOOK_INPUT" \
|
|
40
|
+
| jq -r '.tool_input.file_path // .tool_input.path // empty' 2>/dev/null)
|
|
41
|
+
[[ -z "$TOOL_TARGET" ]] && exit 0
|
|
42
|
+
|
|
43
|
+
# Canonicalize.
|
|
44
|
+
if command -v realpath &>/dev/null; then
|
|
45
|
+
CANONICAL=$(realpath "$TOOL_TARGET" 2>/dev/null) || CANONICAL="$TOOL_TARGET"
|
|
46
|
+
elif command -v readlink &>/dev/null; then
|
|
47
|
+
CANONICAL=$(readlink -f "$TOOL_TARGET" 2>/dev/null) || CANONICAL="$TOOL_TARGET"
|
|
48
|
+
else
|
|
49
|
+
CANONICAL="$TOOL_TARGET"
|
|
50
|
+
fi
|
|
51
|
+
export INSPECTOR_FILE="$CANONICAL"
|
|
52
|
+
|
|
53
|
+
REPO_ROOT=$(inspector_project_repo_root "$CWD")
|
|
54
|
+
export INSPECTOR_REPO_ROOT="$REPO_ROOT"
|
|
55
|
+
|
|
56
|
+
# Project-key derivation — always succeeds (falls back to cwd hash).
|
|
57
|
+
PROJECT_KEY=$(inspector_project_key "$CWD")
|
|
58
|
+
export INSPECTOR_PROJECT_KEY="$PROJECT_KEY"
|
|
59
|
+
|
|
60
|
+
# File must live under repo root.
|
|
61
|
+
if [[ "$CANONICAL" != "$REPO_ROOT"/* && "$CANONICAL" != "$REPO_ROOT" ]]; then
|
|
62
|
+
export INSPECTOR_FILE_RELATIVE="$CANONICAL"
|
|
63
|
+
inspector_config_load "$REPO_ROOT"
|
|
64
|
+
inspector_config_enabled || exit 0
|
|
65
|
+
inspector_emit_whole_file_skipped "not_in_repo"
|
|
66
|
+
exit 0
|
|
67
|
+
fi
|
|
68
|
+
export INSPECTOR_FILE_RELATIVE="${CANONICAL#"$REPO_ROOT"/}"
|
|
69
|
+
|
|
70
|
+
inspector_config_load "$REPO_ROOT"
|
|
71
|
+
|
|
72
|
+
if ! inspector_config_enabled; then
|
|
73
|
+
# Silent skip — do not even emit an event when disabled. Disabled means
|
|
74
|
+
# "this plugin is dormant," not "this file was uninteresting."
|
|
75
|
+
exit 0
|
|
76
|
+
fi
|
|
77
|
+
|
|
78
|
+
# Excluded path containment check.
|
|
79
|
+
EXCLUDES=$(inspector_config_exclude_paths)
|
|
80
|
+
if [[ -n "$EXCLUDES" && "$EXCLUDES" != "null" && "$EXCLUDES" != "[]" ]]; then
|
|
81
|
+
if jq -e --arg rel "$INSPECTOR_FILE_RELATIVE" \
|
|
82
|
+
'any(.[]; . as $p | $rel | startswith($p + "/") or . == $p or (("/" + $rel) | contains("/" + $p + "/")))' \
|
|
83
|
+
<<<"$EXCLUDES" >/dev/null 2>&1; then
|
|
84
|
+
inspector_emit_whole_file_skipped "excluded_path"
|
|
85
|
+
exit 0
|
|
86
|
+
fi
|
|
87
|
+
fi
|
|
88
|
+
|
|
89
|
+
# Look up checks for this file's extension. Use the *longest* matching suffix
|
|
90
|
+
# so `.test.ts` matches before `.ts`. For now this is a simple two-step:
|
|
91
|
+
# first the multi-dot suffix, then the simple extension.
|
|
92
|
+
FILE_BASE=$(basename "$CANONICAL")
|
|
93
|
+
EXT_LONG=""
|
|
94
|
+
EXT_SHORT=""
|
|
95
|
+
if [[ "$FILE_BASE" == *.*.* ]]; then
|
|
96
|
+
EXT_LONG=".${FILE_BASE#*.}"
|
|
97
|
+
fi
|
|
98
|
+
if [[ "$FILE_BASE" == *.* ]]; then
|
|
99
|
+
EXT_SHORT=".${FILE_BASE##*.}"
|
|
100
|
+
fi
|
|
101
|
+
|
|
102
|
+
CHECKS="[]"
|
|
103
|
+
if [[ -n "$EXT_LONG" ]]; then
|
|
104
|
+
CANDIDATE=$(inspector_config_checks_for_extension "$EXT_LONG")
|
|
105
|
+
if [[ -n "$CANDIDATE" && "$CANDIDATE" != "[]" ]]; then
|
|
106
|
+
CHECKS="$CANDIDATE"
|
|
107
|
+
fi
|
|
108
|
+
fi
|
|
109
|
+
if [[ "$CHECKS" == "[]" && -n "$EXT_SHORT" ]]; then
|
|
110
|
+
CANDIDATE=$(inspector_config_checks_for_extension "$EXT_SHORT")
|
|
111
|
+
if [[ -n "$CANDIDATE" && "$CANDIDATE" != "[]" ]]; then
|
|
112
|
+
CHECKS="$CANDIDATE"
|
|
113
|
+
fi
|
|
114
|
+
fi
|
|
115
|
+
|
|
116
|
+
if [[ "$CHECKS" == "[]" ]]; then
|
|
117
|
+
inspector_emit_whole_file_skipped "no_extension_match"
|
|
118
|
+
exit 0
|
|
119
|
+
fi
|
|
120
|
+
|
|
121
|
+
# Execute. Always exit 0 regardless of check outcomes.
|
|
122
|
+
inspector_run "$CHECKS" || true
|
|
123
|
+
|
|
124
|
+
exit 0
|