@profoundry-us/highball 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ONBOARDING.md +29 -23
- package/README.md +96 -24
- package/assets/dashboard.html +8 -1
- package/bin/highball.js +7 -11
- package/lib/config.js +29 -15
- package/lib/git.js +12 -0
- package/lib/init.js +16 -8
- package/lib/mcp.js +111 -41
- package/lib/posthog.js +149 -0
- package/lib/run.js +32 -8
- package/lib/stamp.js +65 -0
- package/package.json +7 -6
- package/lib/login.js +0 -68
- package/lib/report.js +0 -75
package/ONBOARDING.md
CHANGED
|
@@ -3,10 +3,10 @@
|
|
|
3
3
|
You are setting up Highball in this repository. Highball gives you (the
|
|
4
4
|
agent) guardrails while you work — a checks runner your Claude Code hooks
|
|
5
5
|
fire on every edit and turn end, whose exit code 2 blocks you until the
|
|
6
|
-
repo's rules pass
|
|
7
|
-
|
|
8
|
-
work: set it up so the rules reflect what this repo already trusts,
|
|
9
|
-
what you wish it did.
|
|
6
|
+
repo's rules pass. Runs can optionally be reported to your human's own
|
|
7
|
+
PostHog project. You are configuring the tool that will check your own
|
|
8
|
+
future work: set it up so the rules reflect what this repo already trusts,
|
|
9
|
+
not what you wish it did.
|
|
10
10
|
|
|
11
11
|
Work through the steps in order. Steps marked **human** need your human's
|
|
12
12
|
input or action — ask, don't guess, and never handle credential values
|
|
@@ -100,8 +100,11 @@ like this:
|
|
|
100
100
|
version: 1
|
|
101
101
|
project: my-mud # ask your human if their org has slug conventions
|
|
102
102
|
|
|
103
|
-
|
|
104
|
-
|
|
103
|
+
# Optional — see step 4. Skip unless your human asks for telemetry.
|
|
104
|
+
# reporting:
|
|
105
|
+
# posthog:
|
|
106
|
+
# host: https://us.i.posthog.com # human provides; committed, not secret
|
|
107
|
+
# project_key: phc_xxx
|
|
105
108
|
|
|
106
109
|
# Toolchain in Docker: declare the wrapper once. -T is mandatory (hook
|
|
107
110
|
# shells have no TTY); --workdir should be the container's repo mount.
|
|
@@ -143,17 +146,18 @@ Decision rules:
|
|
|
143
146
|
repo-relative) to every rule — scripts that want changed-only behavior
|
|
144
147
|
can read it instead of shelling out to git.
|
|
145
148
|
|
|
146
|
-
## 4.
|
|
149
|
+
## 4. Telemetry — **human**, and optional
|
|
147
150
|
|
|
148
|
-
|
|
151
|
+
Highball enforces with no account and no network. Reporting is a separate,
|
|
152
|
+
optional decision, and it is your human's to make — ask, do not assume.
|
|
149
153
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
154
|
+
If they want it, they add a `reporting.posthog` block to `checks.yml` with
|
|
155
|
+
their PostHog host and project key. That key is write-only by design (it is
|
|
156
|
+
the same one that ships in client-side web bundles), so it is committed
|
|
157
|
+
config, not a secret — there is no login step and no credentials file. If
|
|
158
|
+
they would rather not send anything anywhere, skip the block: the local
|
|
159
|
+
journal (`highball runs`) already records every run in more detail than
|
|
160
|
+
PostHog receives.
|
|
157
161
|
|
|
158
162
|
## 5. Verify — all four proofs, not just the happy path
|
|
159
163
|
|
|
@@ -165,17 +169,19 @@ vars instead. The repo tree never contains a secret.
|
|
|
165
169
|
failing file (e.g. a syntax error in a `tmp_highball_plant.*` file),
|
|
166
170
|
run the fast path, confirm `FAILED` plus exit code 2 plus the failure
|
|
167
171
|
text on stderr — then delete the plant and confirm green again.
|
|
168
|
-
4. **The
|
|
169
|
-
|
|
170
|
-
`
|
|
171
|
-
`
|
|
172
|
-
project
|
|
172
|
+
4. **The record:** `npx @profoundry-us/highball runs` lists the runs you
|
|
173
|
+
just made, with per-rule status. If your human opted into telemetry,
|
|
174
|
+
each run also prints `reported N events to <host>`; if you see
|
|
175
|
+
`highball posthog reporting skipped: …` instead, check the host and
|
|
176
|
+
project key in `reporting.posthog` and whether this machine can reach
|
|
177
|
+
the host. Reporting never blocks a run, so this is the last thing to
|
|
178
|
+
fix, not the first.
|
|
173
179
|
|
|
174
180
|
## 6. Report back — **human**
|
|
175
181
|
|
|
176
182
|
Tell your human, concretely: which rules you wired and why each is
|
|
177
183
|
fast/turn-end/todo; what you deliberately did NOT gate (slow suites,
|
|
178
184
|
live-server tests); that hooks now block your turns on failures and how
|
|
179
|
-
to remove them (`.claude/settings.json`) if they ever need to; and
|
|
180
|
-
|
|
181
|
-
files — do not commit without being asked.
|
|
185
|
+
to remove them (`.claude/settings.json`) if they ever need to; and whether
|
|
186
|
+
you wired telemetry or left it off. Follow this repo's own norms about
|
|
187
|
+
committing the new files — do not commit without being asked.
|
package/README.md
CHANGED
|
@@ -1,13 +1,16 @@
|
|
|
1
1
|
# @profoundry-us/highball
|
|
2
2
|
|
|
3
|
-
The Highball runner: executes a repo's `.highball/checks.yml` rules
|
|
4
|
-
coding agents on failure (exit 2, the Claude Code hook contract)
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
3
|
+
The Highball runner: executes a repo's `.highball/checks.yml` rules and blocks
|
|
4
|
+
AI coding agents on failure (exit 2, the Claude Code hook contract) — "local
|
|
5
|
+
CI for AI agents". Enforcement is entirely local and needs no account, no
|
|
6
|
+
network, and no configuration beyond `checks.yml`.
|
|
7
|
+
|
|
8
|
+
Runs can optionally be reported to PostHog for team-wide visibility. That is
|
|
9
|
+
the witness half, and it is deliberately somebody else's server: you point at
|
|
10
|
+
your own PostHog project, so Highball never takes custody of your data. The
|
|
11
|
+
runner never uploads code — only rule ids, pass/fail, durations, and a
|
|
12
|
+
one-line summary of a failure. Reporting is always best-effort: an unreachable
|
|
13
|
+
endpoint means checks still run and still block, they just aren't recorded.
|
|
11
14
|
|
|
12
15
|
## Install
|
|
13
16
|
|
|
@@ -22,7 +25,7 @@ From a local tarball (pre-release):
|
|
|
22
25
|
|
|
23
26
|
```bash
|
|
24
27
|
npm pack # in this repo → profoundry-us-highball-<v>.tgz
|
|
25
|
-
npm install --save-dev ../highball
|
|
28
|
+
npm install --save-dev ../highball/profoundry-us-highball-<v>.tgz
|
|
26
29
|
```
|
|
27
30
|
|
|
28
31
|
## Setup: let the repo's own agent do it
|
|
@@ -34,30 +37,41 @@ installing the package, tell the repo's Claude Code agent:
|
|
|
34
37
|
|
|
35
38
|
[ONBOARDING.md](ONBOARDING.md) (which that command prints) walks the agent
|
|
36
39
|
through surveying the repo's real toolchain, scaffolding, writing rules that
|
|
37
|
-
reflect what the repo already trusts,
|
|
38
|
-
|
|
40
|
+
reflect what the repo already trusts, and verifying all four proofs —
|
|
41
|
+
including that exit 2 actually blocks.
|
|
39
42
|
|
|
40
43
|
The pieces, for reference or manual setup:
|
|
41
44
|
|
|
42
45
|
```bash
|
|
43
46
|
npx @profoundry-us/highball init # scaffolds checks.yml + Claude Code hooks
|
|
44
|
-
npx @profoundry-us/highball login # stores a project token (once per machine)
|
|
45
47
|
```
|
|
46
48
|
|
|
47
49
|
`init` never overwrites an existing `checks.yml` and never edits an existing
|
|
48
|
-
`.claude/settings.json` (it prints the hook snippet to merge by hand).
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
50
|
+
`.claude/settings.json` (it prints the hook snippet to merge by hand). There
|
|
51
|
+
is no login step: the only credential Highball takes is a PostHog project
|
|
52
|
+
key, which is write-only by design and lives in committed config.
|
|
53
|
+
|
|
54
|
+
The fast hook matches `Write|Edit|Bash` and runs `--fast --if-changed`.
|
|
55
|
+
Agents in auto mode edit through Bash — `sed`, heredocs, scripts — so a hook
|
|
56
|
+
on the edit tools alone never fires for them (one repo here went two weeks
|
|
57
|
+
with zero fast runs that way while full runs kept landing). Matching Bash
|
|
58
|
+
too would run the fast rules after every command, most of them reads, so
|
|
59
|
+
`--if-changed` fingerprints the working tree (HEAD plus each dirty path's
|
|
60
|
+
size and mtime) and exits at once when nothing moved since the last run.
|
|
61
|
+
Older installs: change the matcher and add the flag by hand.
|
|
52
62
|
|
|
53
63
|
## checks.yml
|
|
54
64
|
|
|
55
65
|
```yaml
|
|
56
66
|
version: 1
|
|
57
67
|
project: my-app
|
|
68
|
+
# runs_limit: 25 # rows the MCP widget / list_runs return; HIGHBALL_RUNS_LIMIT overrides
|
|
58
69
|
|
|
70
|
+
# Optional. Omit the block entirely to keep runs in the local journal only.
|
|
59
71
|
reporting:
|
|
60
|
-
|
|
72
|
+
posthog:
|
|
73
|
+
host: https://us.i.posthog.com # EU cloud or self-hosted also fine
|
|
74
|
+
project_key: phc_xxx # write-only by design — committed
|
|
61
75
|
|
|
62
76
|
# Containerized toolchain? Declare the wrapper once and every rule runs
|
|
63
77
|
# through it; rules opt out with `exec: host`. Rule definitions stay
|
|
@@ -121,6 +135,51 @@ Three properties are enforced by the runner rather than left to each repo:
|
|
|
121
135
|
Rubrics live with the opinions they express: a framework pack such as
|
|
122
136
|
`@profoundry-us/highball-rails` ships them, and the runner supplies the engine.
|
|
123
137
|
|
|
138
|
+
## Reporting to PostHog (optional)
|
|
139
|
+
|
|
140
|
+
`reporting.posthog` sends runs to PostHog — the runner's only telemetry path.
|
|
141
|
+
A team that already runs PostHog needs no server for this, and a team that
|
|
142
|
+
doesn't can skip the block entirely and use the local journal. The project key
|
|
143
|
+
is write-only by design, so it is committed config: no login step, no
|
|
144
|
+
credentials file. `HIGHBALL_POSTHOG_KEY` / `HIGHBALL_POSTHOG_HOST` (or
|
|
145
|
+
`POSTHOG_API_KEY` / `POSTHOG_HOST`) override it for CI.
|
|
146
|
+
|
|
147
|
+
The whole run leaves in ONE request to `/batch/`. PostHog events are
|
|
148
|
+
immutable, which suits a runner that already defers reporting to after the
|
|
149
|
+
checks finish.
|
|
150
|
+
|
|
151
|
+
Two event types per run:
|
|
152
|
+
|
|
153
|
+
| event | one per | key properties |
|
|
154
|
+
| --- | --- | --- |
|
|
155
|
+
| `highball_run` | run | `status`, `duration_ms`, `rules_passed/failed/todo`, `rules_run[]`, `failed_rules[]` |
|
|
156
|
+
| `highball_check` | rule result | `rule_id`, `rule_name`, `status`, `duration_ms`, `summary`, `command` |
|
|
157
|
+
|
|
158
|
+
Run context (`project`, `branch`, `commit`, `trigger`, `session_key`,
|
|
159
|
+
`runner_version`) is repeated on every event rather than joined at query time,
|
|
160
|
+
because PostHog has no join back to a run: a breakdown like "failure rate by
|
|
161
|
+
rule, on this branch only" needs `branch` on the check event itself.
|
|
162
|
+
`rules_run[]` is the denominator for any per-rule rate that spans runs whose
|
|
163
|
+
rulesets differ.
|
|
164
|
+
|
|
165
|
+
Log tails are never sent — multi-kilobyte blobs in event properties bloat the
|
|
166
|
+
column store and slow every query that touches it. Failures carry a one-line
|
|
167
|
+
`summary`; the full output stays in the local journal (below).
|
|
168
|
+
|
|
169
|
+
Volume is smaller than "runs on every edit" suggests, because agent edits
|
|
170
|
+
batch into turns. Measured across 842 real runs on one developer's machine
|
|
171
|
+
over 18 active days and 9 projects: a median of 22 runs/day, `edit` and `stop`
|
|
172
|
+
runs at close to 1.3:1, and ~15k events/month/dev at the model above.
|
|
173
|
+
|
|
174
|
+
Dashboard queries live in [docs/posthog-queries.sql](docs/posthog-queries.sql)
|
|
175
|
+
— rule cost vs. benefit, failure rate by week, fast-path latency, repo health,
|
|
176
|
+
and todo debt. They are versioned rather than left as PostHog UI state, which
|
|
177
|
+
drifts and cannot be reviewed.
|
|
178
|
+
|
|
179
|
+
Events are attributed to `git config user.email`, falling back to
|
|
180
|
+
`user@hostname` when git has no identity (CI images, fresh containers).
|
|
181
|
+
`HIGHBALL_POSTHOG_DISTINCT_ID` overrides it.
|
|
182
|
+
|
|
124
183
|
## The MCP dashboard widget
|
|
125
184
|
|
|
126
185
|
`highball mcp` serves the journal over MCP (stdio) with three tools —
|
|
@@ -142,6 +201,18 @@ The journal it reads is machine-global (`~/.highball/runs/`), so one
|
|
|
142
201
|
registration covers every repo on that machine — there is no per-repo MCP
|
|
143
202
|
setup.
|
|
144
203
|
|
|
204
|
+
Which project the widget shows is never guessed. Hosts spawn the server with
|
|
205
|
+
no useful working directory (Claude Desktop and Claude Code both use `/`), so
|
|
206
|
+
`list_runs` resolves the project from its `project` or `dir` argument, a
|
|
207
|
+
`.highball/checks.yml` at the server's cwd, or the client's MCP roots. When
|
|
208
|
+
none of those names a repo it returns the journaled projects for the widget
|
|
209
|
+
to offer as a picker, rather than showing whichever repo happened to run most
|
|
210
|
+
recently. An agent calling from inside a repo should pass `dir`.
|
|
211
|
+
|
|
212
|
+
`list_runs` returns the newest 25 runs, and the widget says so at the top.
|
|
213
|
+
Change it per call with `limit`, per machine with `HIGHBALL_RUNS_LIMIT`, or
|
|
214
|
+
per repo with `runs_limit:` in checks.yml.
|
|
215
|
+
|
|
145
216
|
The split is capability-driven, not guesswork: the server reads the
|
|
146
217
|
client's initialize capabilities (`io.modelcontextprotocol/ui`) — hosts
|
|
147
218
|
that render MCP Apps get a short text summary plus the widget; everything
|
|
@@ -151,17 +222,18 @@ plays the host role against the real `assets/dashboard.html` and live
|
|
|
151
222
|
journal data, so widget edits are a reload away instead of a Claude
|
|
152
223
|
Desktop restart.
|
|
153
224
|
|
|
154
|
-
## Run history
|
|
225
|
+
## Run history, with no telemetry at all
|
|
155
226
|
|
|
156
|
-
Every run
|
|
227
|
+
Every run appends to a local journal (`~/.highball/runs/<project>.jsonl`,
|
|
157
228
|
pruned to the last 200) — unconditionally, whether or not reporting is
|
|
158
229
|
configured. `npx @profoundry-us/highball runs` lists recent runs; adding a
|
|
159
230
|
number shows one run's detail with failure output, and `--logs` prints every
|
|
160
|
-
rule's captured output, GitHub-Actions-style
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
231
|
+
rule's captured output, GitHub-Actions-style.
|
|
232
|
+
|
|
233
|
+
The journal is the richer of the two records: it keeps the last 10KB per rule
|
|
234
|
+
pass or fail, while PostHog gets a one-line summary and no logs at all. So the
|
|
235
|
+
runner is fully self-sufficient with no `reporting:` block — PostHog adds
|
|
236
|
+
cross-developer trends, not visibility you'd otherwise lack.
|
|
165
237
|
|
|
166
238
|
## Roadmap
|
|
167
239
|
|
package/assets/dashboard.html
CHANGED
|
@@ -194,6 +194,7 @@
|
|
|
194
194
|
}
|
|
195
195
|
app.innerHTML =
|
|
196
196
|
header("Highball — pick a project", "") +
|
|
197
|
+
`<div class="group">no .highball/checks.yml in the current repo, so nothing is shown by default</div>` +
|
|
197
198
|
`<div class="empty" style="padding:10px 6px">` +
|
|
198
199
|
projects.map((slug) => `<button data-project="${esc(slug)}" style="margin:0 4px">${esc(slug)}</button>`).join("") +
|
|
199
200
|
`</div>`;
|
|
@@ -225,8 +226,14 @@
|
|
|
225
226
|
<span class="meta">${esc(tally(run.results))}</span>
|
|
226
227
|
</div>`;
|
|
227
228
|
}).join("");
|
|
229
|
+
// Say up front when the list is cut short — the host's own history
|
|
230
|
+
// may be much longer than what the widget shows.
|
|
231
|
+
const scope = sc.total > sc.runs.length
|
|
232
|
+
? `<div class="group" style="font-style:normal">showing the last ${sc.runs.length} of ${sc.total} runs` +
|
|
233
|
+
` · raise with <span class="mono">runs_limit</span> in checks.yml or <span class="mono">HIGHBALL_RUNS_LIMIT</span></div>`
|
|
234
|
+
: "";
|
|
228
235
|
app.innerHTML =
|
|
229
|
-
header(`Highball · ${esc(sc.project)}`, rerunButtons()) +
|
|
236
|
+
header(`Highball · ${esc(sc.project)}`, rerunButtons()) + scope +
|
|
230
237
|
(rows || `<div class="empty">No runs recorded yet.</div>`);
|
|
231
238
|
app.querySelectorAll(".row").forEach((el) =>
|
|
232
239
|
el.addEventListener("click", () =>
|
package/bin/highball.js
CHANGED
|
@@ -5,7 +5,6 @@
|
|
|
5
5
|
// the failure output back".
|
|
6
6
|
import { run } from "../lib/run.js";
|
|
7
7
|
import { init } from "../lib/init.js";
|
|
8
|
-
import { login } from "../lib/login.js";
|
|
9
8
|
import { onboard } from "../lib/onboard.js";
|
|
10
9
|
import { runs } from "../lib/runs.js";
|
|
11
10
|
import { mcp } from "../lib/mcp.js";
|
|
@@ -18,18 +17,18 @@ Usage:
|
|
|
18
17
|
highball run [--fast] Run this repo's .highball/checks.yml rules.
|
|
19
18
|
Exits 2 on failure (blocks Claude Code hooks).
|
|
20
19
|
--fast runs only rules marked fast: true.
|
|
21
|
-
|
|
20
|
+
--if-changed skips when the working tree is
|
|
21
|
+
unchanged since the last run (for hooks that
|
|
22
|
+
also match Bash).
|
|
23
|
+
highball init Scaffold .highball/checks.yml and Claude Code
|
|
22
24
|
hooks in the current repo.
|
|
23
|
-
highball login Store a project token in
|
|
24
|
-
~/.highball/credentials.json. Reads the token
|
|
25
|
-
from stdin with --token-stdin (recommended).
|
|
26
25
|
highball onboard Print the setup guide written for this repo's
|
|
27
26
|
AI agent — tell your agent to run this and
|
|
28
27
|
follow it.
|
|
29
28
|
highball runs [n] Local run history (newest first) from
|
|
30
|
-
~/.highball/runs
|
|
31
|
-
|
|
32
|
-
|
|
29
|
+
~/.highball/runs. With a number, that run's
|
|
30
|
+
detail; add --logs for every rule's captured
|
|
31
|
+
output.
|
|
33
32
|
highball mcp Serve run history over MCP (stdio), with an
|
|
34
33
|
MCP Apps dashboard widget for hosts that
|
|
35
34
|
render them (e.g. Claude Desktop).
|
|
@@ -42,9 +41,6 @@ switch (command) {
|
|
|
42
41
|
case "init":
|
|
43
42
|
process.exit(await init(args));
|
|
44
43
|
break;
|
|
45
|
-
case "login":
|
|
46
|
-
process.exit(await login(args));
|
|
47
|
-
break;
|
|
48
44
|
case "onboard":
|
|
49
45
|
process.exit(await onboard(args));
|
|
50
46
|
break;
|
package/lib/config.js
CHANGED
|
@@ -1,12 +1,10 @@
|
|
|
1
1
|
// Config resolution, kept pure where possible so tests can exercise the
|
|
2
2
|
// decision logic without a filesystem.
|
|
3
3
|
import { readFileSync, existsSync } from "node:fs";
|
|
4
|
-
import { homedir } from "node:os";
|
|
5
4
|
import { join } from "node:path";
|
|
6
5
|
import YAML from "yaml";
|
|
7
6
|
|
|
8
7
|
export const CONFIG_PATH = ".highball/checks.yml";
|
|
9
|
-
export const CREDENTIALS_PATH = join(homedir(), ".highball", "credentials.json");
|
|
10
8
|
|
|
11
9
|
// Loads .highball/checks.yml from the given repo root. Throws with a
|
|
12
10
|
// friendly message — `highball run` outside a configured repo should read
|
|
@@ -29,23 +27,39 @@ export function loadConfig(root = process.cwd()) {
|
|
|
29
27
|
);
|
|
30
28
|
}
|
|
31
29
|
}
|
|
30
|
+
// The dashboard sink is gone; PostHog is the only telemetry path. A repo
|
|
31
|
+
// carrying the old block would otherwise report nowhere and say nothing
|
|
32
|
+
// about it — the same silent-pass shape that makes stale config dangerous.
|
|
33
|
+
if (config.reporting?.url) {
|
|
34
|
+
console.error(
|
|
35
|
+
`highball: ${CONFIG_PATH} has \`reporting.url\`, which is no longer ` +
|
|
36
|
+
"supported. Runs report to PostHog via `reporting.posthog`, or " +
|
|
37
|
+
"nowhere at all. Delete the key to silence this."
|
|
38
|
+
);
|
|
39
|
+
}
|
|
40
|
+
|
|
32
41
|
return config;
|
|
33
42
|
}
|
|
34
43
|
|
|
35
|
-
//
|
|
36
|
-
//
|
|
37
|
-
//
|
|
38
|
-
//
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
44
|
+
// PostHog is the runner's telemetry sink. There is no secret to resolve: a
|
|
45
|
+
// PostHog project key is write-only by design, so it lives in committed
|
|
46
|
+
// config right next to the host — no login step, no credentials file. Env
|
|
47
|
+
// vars still win so CI can point a run somewhere else without editing the
|
|
48
|
+
// repo.
|
|
49
|
+
//
|
|
50
|
+
// The host defaults to PostHog's US cloud; EU and self-hosted installs set
|
|
51
|
+
// it explicitly.
|
|
52
|
+
const POSTHOG_DEFAULT_HOST = "https://us.i.posthog.com";
|
|
53
|
+
|
|
54
|
+
export function resolvePosthog(config, env = process.env) {
|
|
55
|
+
const posthog = config.reporting?.posthog;
|
|
56
|
+
const key =
|
|
57
|
+
env.HIGHBALL_POSTHOG_KEY || env.POSTHOG_API_KEY || posthog?.project_key || null;
|
|
58
|
+
if (!key) return { host: null, key: null };
|
|
45
59
|
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
return
|
|
60
|
+
const host =
|
|
61
|
+
env.HIGHBALL_POSTHOG_HOST || env.POSTHOG_HOST || posthog?.host || POSTHOG_DEFAULT_HOST;
|
|
62
|
+
return { host, key };
|
|
49
63
|
}
|
|
50
64
|
|
|
51
65
|
// The execution-context decision (ADR 202608): rule definitions stay
|
package/lib/git.js
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
// Git facts about the current checkout, for stamping onto reported runs.
|
|
2
|
+
// Never throws: a run outside a repo, or with no commits yet, reports empty
|
|
3
|
+
// strings rather than failing the checks.
|
|
4
|
+
import { execSync } from "node:child_process";
|
|
5
|
+
|
|
6
|
+
export function git(command) {
|
|
7
|
+
try {
|
|
8
|
+
return execSync(`${command} 2>/dev/null`, { encoding: "utf8" }).trim();
|
|
9
|
+
} catch {
|
|
10
|
+
return "";
|
|
11
|
+
}
|
|
12
|
+
}
|
package/lib/init.js
CHANGED
|
@@ -13,11 +13,12 @@ const CHECKS_TEMPLATE = (project) => `# ${project}'s Highball rules — the file
|
|
|
13
13
|
version: 1
|
|
14
14
|
project: ${project}
|
|
15
15
|
|
|
16
|
-
#
|
|
17
|
-
#
|
|
18
|
-
# \`highball login\` once per machine (or set HIGHBALL_TOKEN in CI).
|
|
16
|
+
# Optional telemetry. The PostHog project key is write-only by design, so
|
|
17
|
+
# it is committed config — there is no login step and no credentials file.
|
|
19
18
|
# reporting:
|
|
20
|
-
#
|
|
19
|
+
# posthog:
|
|
20
|
+
# host: https://us.i.posthog.com
|
|
21
|
+
# project_key: phc_your_key
|
|
21
22
|
|
|
22
23
|
# If this repo's toolchain lives in a container, declare the wrapper once
|
|
23
24
|
# and every rule runs through it; rules that belong on the host opt out
|
|
@@ -39,12 +40,19 @@ checks:
|
|
|
39
40
|
// Always the SCOPED command. The unscoped npm name belongs to an unrelated
|
|
40
41
|
// package, so a bare `npx highball` in a committed hook is one uninstalled
|
|
41
42
|
// checkout away from fetching a stranger's code and running it on every edit.
|
|
43
|
+
//
|
|
44
|
+
// The fast hook matches Bash as well as the edit tools: agents in auto mode
|
|
45
|
+
// edit through Bash, and a hook on Write|Edit alone never fires for them.
|
|
46
|
+
// --if-changed keeps the Bash firings free when the tree hasn't moved.
|
|
42
47
|
const HOOKS_JSON = {
|
|
43
48
|
hooks: {
|
|
44
49
|
PostToolUse: [
|
|
45
50
|
{
|
|
46
|
-
matcher: "Write|Edit",
|
|
47
|
-
hooks: [{
|
|
51
|
+
matcher: "Write|Edit|Bash",
|
|
52
|
+
hooks: [{
|
|
53
|
+
type: "command",
|
|
54
|
+
command: "npx @profoundry-us/highball run --fast --if-changed"
|
|
55
|
+
}]
|
|
48
56
|
}
|
|
49
57
|
],
|
|
50
58
|
Stop: [
|
|
@@ -82,8 +90,8 @@ export async function init() {
|
|
|
82
90
|
}
|
|
83
91
|
|
|
84
92
|
console.log(
|
|
85
|
-
"\nnext: fill in .highball/checks.yml,
|
|
86
|
-
"\
|
|
93
|
+
"\nnext: fill in .highball/checks.yml, and optionally uncomment the" +
|
|
94
|
+
"\nreporting.posthog block to send runs to PostHog."
|
|
87
95
|
);
|
|
88
96
|
return 0;
|
|
89
97
|
}
|
package/lib/mcp.js
CHANGED
|
@@ -21,34 +21,61 @@ const VERSION = JSON.parse(
|
|
|
21
21
|
readFileSync(new URL("../package.json", import.meta.url), "utf8")
|
|
22
22
|
).version;
|
|
23
23
|
|
|
24
|
-
// The
|
|
25
|
-
//
|
|
26
|
-
//
|
|
27
|
-
//
|
|
28
|
-
//
|
|
29
|
-
//
|
|
30
|
-
|
|
31
|
-
|
|
24
|
+
// The host decides the server's working directory, and real hosts give it
|
|
25
|
+
// nothing useful — Claude Desktop and Claude Code both spawn it at `/`. So
|
|
26
|
+
// "the current repo" has to arrive some other way: an explicit `project`
|
|
27
|
+
// or `dir` argument, a checks.yml at cwd, or the client's MCP roots.
|
|
28
|
+
// Nothing else is guessed. In particular the journal is NOT a fallback:
|
|
29
|
+
// "whichever project ran most recently on this machine" is usually some
|
|
30
|
+
// other repo, and a widget that silently shows another project's runs
|
|
31
|
+
// reads as this project's. Unresolved means the widget offers a picker.
|
|
32
|
+
//
|
|
33
|
+
// Journal records carry the repo dir (runs record process.cwd()), so an
|
|
34
|
+
// explicit project still grounds the widget's re-run buttons. The loaded
|
|
35
|
+
// config rides along so per-repo settings (runs_limit) can apply.
|
|
36
|
+
export function resolveProject({
|
|
37
|
+
project, dir, cwd = process.cwd(), roots = [], journalDir
|
|
38
|
+
} = {}) {
|
|
39
|
+
if (project) {
|
|
40
|
+
const home = dir ?? latestDirFor(project, journalDir);
|
|
41
|
+
return { project, dir: home, config: tryLoad(home) };
|
|
42
|
+
}
|
|
43
|
+
for (const candidate of [ dir, cwd, ...roots ].filter(Boolean)) {
|
|
44
|
+
const config = tryLoad(candidate);
|
|
45
|
+
if (config) return { project: config.project, dir: candidate, config };
|
|
46
|
+
}
|
|
47
|
+
return { project: null, dir: null, config: null };
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
function tryLoad(root) {
|
|
51
|
+
if (!root) return null;
|
|
32
52
|
try {
|
|
33
|
-
return
|
|
53
|
+
return loadConfig(root);
|
|
34
54
|
} catch {
|
|
35
|
-
|
|
36
|
-
for (const project of journaledProjects()) {
|
|
37
|
-
const newest = readRuns(project)[0];
|
|
38
|
-
if (!newest) continue;
|
|
39
|
-
if (!current || newest.started_at > current.started_at) {
|
|
40
|
-
current = { project, started_at: newest.started_at };
|
|
41
|
-
}
|
|
42
|
-
}
|
|
43
|
-
if (!current) return { project: null, dir: null };
|
|
44
|
-
return { project: current.project, dir: latestDirFor(current.project) };
|
|
55
|
+
return null;
|
|
45
56
|
}
|
|
46
57
|
}
|
|
47
58
|
|
|
48
59
|
// Newest journal record that knows its repo dir (older records predate
|
|
49
60
|
// the field).
|
|
50
|
-
function latestDirFor(project) {
|
|
51
|
-
return readRuns(project).find((run) => run.dir)?.dir ?? null;
|
|
61
|
+
function latestDirFor(project, journalDir) {
|
|
62
|
+
return readRuns(project, journalDir).find((run) => run.dir)?.dir ?? null;
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
// How many runs list_runs returns. 200 journaled runs is a fine history and
|
|
66
|
+
// a terrible tool result — the structured payload alone reaches ~300KB,
|
|
67
|
+
// which text-only hosts hand straight to the model. Resolution: the call's
|
|
68
|
+
// `limit`, then HIGHBALL_RUNS_LIMIT, then `runs_limit:` in checks.yml (only
|
|
69
|
+
// present when the project resolved through a repo), then the default.
|
|
70
|
+
export const DEFAULT_RUNS_LIMIT = 25;
|
|
71
|
+
|
|
72
|
+
export function resolveRunsLimit({ limit, env = process.env, config } = {}) {
|
|
73
|
+
for (const candidate of [ limit, env.HIGHBALL_RUNS_LIMIT, config?.runs_limit ]) {
|
|
74
|
+
if (candidate == null || candidate === "") continue;
|
|
75
|
+
const n = Number(candidate);
|
|
76
|
+
if (Number.isInteger(n) && n > 0) return n;
|
|
77
|
+
}
|
|
78
|
+
return DEFAULT_RUNS_LIMIT;
|
|
52
79
|
}
|
|
53
80
|
|
|
54
81
|
// List payloads stay lean — output tails ride only on get_run.
|
|
@@ -82,8 +109,12 @@ function reply(text, structuredContent) {
|
|
|
82
109
|
const glyphFor = (status) =>
|
|
83
110
|
status === "passed" ? "✓" : status === "todo" ? "•" : "✗";
|
|
84
111
|
|
|
85
|
-
export function listText(project, runs) {
|
|
112
|
+
export function listText(project, runs, total = runs.length) {
|
|
86
113
|
if (runs.length === 0) return `No runs recorded for ${project} yet.`;
|
|
114
|
+
const scope = total > runs.length
|
|
115
|
+
? `, last ${runs.length} of ${total} — raise with limit, HIGHBALL_RUNS_LIMIT, ` +
|
|
116
|
+
"or runs_limit in checks.yml"
|
|
117
|
+
: "";
|
|
87
118
|
const rows = runs.map((run) => [
|
|
88
119
|
`#${run.index}`,
|
|
89
120
|
run.status === "passed" ? "✓ passed" : "✗ FAILED",
|
|
@@ -95,7 +126,7 @@ export function listText(project, runs) {
|
|
|
95
126
|
]);
|
|
96
127
|
const widths = rows[0].map((_, col) =>
|
|
97
128
|
Math.max(...rows.map((row) => row[col].length)));
|
|
98
|
-
const lines = [ `Runs for ${project} (newest first):` ];
|
|
129
|
+
const lines = [ `Runs for ${project} (newest first${scope}):` ];
|
|
99
130
|
let lastGroup;
|
|
100
131
|
runs.forEach((run, i) => {
|
|
101
132
|
// Group header whenever the work (or its session) changes between
|
|
@@ -139,31 +170,68 @@ export async function mcp() {
|
|
|
139
170
|
(!capability.mimeTypes || capability.mimeTypes.includes(RESOURCE_MIME_TYPE));
|
|
140
171
|
};
|
|
141
172
|
|
|
173
|
+
// The client's MCP roots are the one thing a host can tell us about
|
|
174
|
+
// which repo is open. Best-effort: only asked of clients that advertise
|
|
175
|
+
// the capability, and a slow or failing answer resolves to nothing.
|
|
176
|
+
const clientRoots = async () => {
|
|
177
|
+
try {
|
|
178
|
+
if (!server.server.getClientCapabilities()?.roots) return [];
|
|
179
|
+
const { roots } = await server.server.listRoots(undefined, { timeout: 3000 });
|
|
180
|
+
return roots
|
|
181
|
+
.map((root) => root.uri)
|
|
182
|
+
.filter((uri) => uri.startsWith("file:"))
|
|
183
|
+
.map((uri) => fileURLToPath(uri));
|
|
184
|
+
} catch {
|
|
185
|
+
return [];
|
|
186
|
+
}
|
|
187
|
+
};
|
|
188
|
+
|
|
189
|
+
const projectArg = z.string().optional().describe(
|
|
190
|
+
"Project slug. Defaults to the repo at `dir`, at the server's cwd, or " +
|
|
191
|
+
"at the client's MCP roots; when none has a .highball/checks.yml the " +
|
|
192
|
+
"result lists journaled projects to choose from instead of guessing."
|
|
193
|
+
);
|
|
194
|
+
const dirArg = z.string().optional().describe(
|
|
195
|
+
"Repo root containing .highball/checks.yml. Pass the current working " +
|
|
196
|
+
"directory when calling from inside a repo."
|
|
197
|
+
);
|
|
198
|
+
|
|
142
199
|
registerAppTool(server, "list_runs", {
|
|
143
200
|
title: "Highball runs",
|
|
144
201
|
description:
|
|
145
202
|
"Recent Highball check runs for a project, from the machine-local " +
|
|
146
|
-
|
|
203
|
+
`journal (~/.highball/runs). Newest ${DEFAULT_RUNS_LIMIT} by default. ` +
|
|
204
|
+
"Renders the runs dashboard widget.",
|
|
147
205
|
inputSchema: {
|
|
148
|
-
project:
|
|
149
|
-
|
|
206
|
+
project: projectArg,
|
|
207
|
+
dir: dirArg,
|
|
208
|
+
limit: z.number().int().min(1).optional().describe(
|
|
209
|
+
`Max runs to return, newest first (default ${DEFAULT_RUNS_LIMIT}; ` +
|
|
210
|
+
"HIGHBALL_RUNS_LIMIT or runs_limit in checks.yml also override)"
|
|
211
|
+
)
|
|
150
212
|
},
|
|
151
213
|
_meta: { ui: { resourceUri: DASHBOARD_URI } }
|
|
152
|
-
}, async ({ project: explicit }) => {
|
|
153
|
-
const { project, dir } =
|
|
214
|
+
}, async ({ project: explicit, dir: explicitDir, limit }) => {
|
|
215
|
+
const { project, dir, config } =
|
|
216
|
+
resolveProject({ project: explicit, dir: explicitDir, roots: await clientRoots() });
|
|
154
217
|
if (!project) {
|
|
155
218
|
const known = journaledProjects();
|
|
156
219
|
return reply(
|
|
157
|
-
|
|
220
|
+
"No project resolved: no .highball/checks.yml at the working " +
|
|
221
|
+
"directory or the client's roots. Pass `project` or `dir`. " +
|
|
222
|
+
`Journaled projects: ${known.join(", ") || "(none)"}`,
|
|
158
223
|
{ projects: known }
|
|
159
224
|
);
|
|
160
225
|
}
|
|
161
|
-
const
|
|
226
|
+
const history = readRuns(project);
|
|
227
|
+
const max = resolveRunsLimit({ limit, config });
|
|
228
|
+
const runs = history.slice(0, max).map(summarize);
|
|
162
229
|
return reply(
|
|
163
230
|
uiHost()
|
|
164
|
-
? `${runs.length} runs for ${project} —
|
|
165
|
-
|
|
166
|
-
|
|
231
|
+
? `${runs.length} of ${history.length} runs for ${project} — ` +
|
|
232
|
+
"rendered in the dashboard widget."
|
|
233
|
+
: listText(project, runs, history.length),
|
|
234
|
+
{ project, dir, runs, total: history.length, limit: max }
|
|
167
235
|
);
|
|
168
236
|
});
|
|
169
237
|
|
|
@@ -174,13 +242,14 @@ export async function mcp() {
|
|
|
174
242
|
"captured command output. index counts from 1, newest first.",
|
|
175
243
|
inputSchema: {
|
|
176
244
|
index: z.number().int().min(1).describe("1-based index, newest first"),
|
|
177
|
-
project:
|
|
178
|
-
|
|
245
|
+
project: projectArg,
|
|
246
|
+
dir: dirArg
|
|
179
247
|
},
|
|
180
248
|
_meta: { ui: { resourceUri: DASHBOARD_URI } }
|
|
181
|
-
}, async ({ index, project: explicit }) => {
|
|
182
|
-
const { project, dir } =
|
|
183
|
-
|
|
249
|
+
}, async ({ index, project: explicit, dir: explicitDir }) => {
|
|
250
|
+
const { project, dir } =
|
|
251
|
+
resolveProject({ project: explicit, dir: explicitDir, roots: await clientRoots() });
|
|
252
|
+
if (!project) return reply("No project resolved — pass `project` or `dir`.", {});
|
|
184
253
|
const history = readRuns(project);
|
|
185
254
|
const run = history[index - 1];
|
|
186
255
|
if (!run) return reply(`No run #${index} (${history.length} recorded).`, {});
|
|
@@ -205,9 +274,10 @@ export async function mcp() {
|
|
|
205
274
|
},
|
|
206
275
|
_meta: { ui: { resourceUri: DASHBOARD_URI } }
|
|
207
276
|
}, async ({ dir, fast }) => {
|
|
208
|
-
//
|
|
209
|
-
//
|
|
210
|
-
|
|
277
|
+
// The widget passes the dir it was grounded with, so re-runs work from
|
|
278
|
+
// hosts with no useful cwd; otherwise resolve the same way list_runs
|
|
279
|
+
// does and let the child's own error explain a missing checks.yml.
|
|
280
|
+
const cwd = dir || resolveProject({ roots: await clientRoots() }).dir || process.cwd();
|
|
211
281
|
const child = spawnSync(
|
|
212
282
|
process.execPath,
|
|
213
283
|
[ BIN_PATH, "run", ...(fast ? [ "--fast" ] : []) ],
|
package/lib/posthog.js
ADDED
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
// The runner's telemetry sink. Enforcement is local and always works;
|
|
2
|
+
// this is the witness half, and it is deliberately somebody else's
|
|
3
|
+
// server. An earlier iteration shipped a bespoke Rails dashboard, which
|
|
4
|
+
// meant hosting, auth, and data custody for what is ultimately a
|
|
5
|
+
// warehouse query — so a team points at their own PostHog instead.
|
|
6
|
+
//
|
|
7
|
+
// The whole run leaves in ONE request. The old protocol opened a run,
|
|
8
|
+
// POSTed each result, then PATCHed the status: twenty round trips for an
|
|
9
|
+
// eighteen-rule run. PostHog events are immutable and append-only, which
|
|
10
|
+
// suits a runner that already defers all reporting to after the checks
|
|
11
|
+
// finish — nothing to open, nothing to finalize.
|
|
12
|
+
//
|
|
13
|
+
// Best-effort, always: failures warn and return. A dead analytics
|
|
14
|
+
// endpoint must never block an agent.
|
|
15
|
+
import { hostname, userInfo } from "node:os";
|
|
16
|
+
import { git } from "./git.js";
|
|
17
|
+
|
|
18
|
+
const TIMEOUT_MS = 15_000;
|
|
19
|
+
|
|
20
|
+
// Two event types, and deliberately no third. `highball_run` answers
|
|
21
|
+
// "how are runs doing" and `highball_check` answers "which rules are
|
|
22
|
+
// earning their keep"; every question we set out to ask breaks down
|
|
23
|
+
// from one of those.
|
|
24
|
+
export const RUN_EVENT = "highball_run";
|
|
25
|
+
export const CHECK_EVENT = "highball_check";
|
|
26
|
+
|
|
27
|
+
// Returns true when the batch was accepted, false when reporting was
|
|
28
|
+
// skipped or failed — the caller journals either way.
|
|
29
|
+
export async function reportPosthog({
|
|
30
|
+
host, key, project, results, hook, fastOnly, startedAt, durationMs, branch,
|
|
31
|
+
commitSha, version
|
|
32
|
+
}) {
|
|
33
|
+
try {
|
|
34
|
+
const batch = buildEvents({
|
|
35
|
+
project, results, hook, fastOnly, startedAt, durationMs, branch, commitSha,
|
|
36
|
+
version, distinctId: distinctId()
|
|
37
|
+
});
|
|
38
|
+
|
|
39
|
+
const response = await fetch(new URL("/batch/", host), {
|
|
40
|
+
method: "POST",
|
|
41
|
+
headers: { "Content-Type": "application/json" },
|
|
42
|
+
body: JSON.stringify({ api_key: key, batch }),
|
|
43
|
+
signal: AbortSignal.timeout(TIMEOUT_MS)
|
|
44
|
+
});
|
|
45
|
+
if (response.status >= 300) {
|
|
46
|
+
throw new Error(`${response.status}: ${(await response.text()).slice(0, 200)}`);
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
console.log(`reported ${batch.length} events to ${new URL(host).host}`);
|
|
50
|
+
return true;
|
|
51
|
+
} catch (error) {
|
|
52
|
+
console.error(`highball posthog reporting skipped: ${error.message}`);
|
|
53
|
+
return false;
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
// Pure, so the event shape can be tested without a network or a clock.
|
|
58
|
+
export function buildEvents({
|
|
59
|
+
project, results, hook, fastOnly, startedAt, durationMs, branch, commitSha,
|
|
60
|
+
version, distinctId
|
|
61
|
+
}) {
|
|
62
|
+
const timestamp = startedAt.toISOString();
|
|
63
|
+
const trigger = fastOnly ? "edit" : "stop";
|
|
64
|
+
const sessionKey =
|
|
65
|
+
hook?.session_id || process.env.HIGHBALL_SESSION_KEY || `manual-${hostname()}`;
|
|
66
|
+
|
|
67
|
+
// Repeated on every event rather than joined at query time. PostHog has
|
|
68
|
+
// no joins back to a "run" table, so a breakdown like "failure rate by
|
|
69
|
+
// rule, on this branch only" needs branch to sit on the check event
|
|
70
|
+
// itself.
|
|
71
|
+
const shared = {
|
|
72
|
+
project,
|
|
73
|
+
branch,
|
|
74
|
+
commit: commitSha,
|
|
75
|
+
trigger,
|
|
76
|
+
session_key: sessionKey,
|
|
77
|
+
agent: hook?.session_id ? "claude-code" : "manual",
|
|
78
|
+
runner_version: version,
|
|
79
|
+
// PostHog surfaces these in its UI as the sending client.
|
|
80
|
+
$lib: "highball",
|
|
81
|
+
$lib_version: version
|
|
82
|
+
};
|
|
83
|
+
|
|
84
|
+
const failed = results.filter((result) => !result.passed);
|
|
85
|
+
const todo = results.filter((result) => result.todo);
|
|
86
|
+
|
|
87
|
+
const events = [ {
|
|
88
|
+
event: RUN_EVENT,
|
|
89
|
+
distinct_id: distinctId,
|
|
90
|
+
timestamp,
|
|
91
|
+
properties: {
|
|
92
|
+
...shared,
|
|
93
|
+
status: failed.length === 0 ? "passed" : "failed",
|
|
94
|
+
duration_ms: durationMs,
|
|
95
|
+
rules_total: results.length,
|
|
96
|
+
rules_passed: results.length - failed.length - todo.length,
|
|
97
|
+
rules_failed: failed.length,
|
|
98
|
+
rules_todo: todo.length,
|
|
99
|
+
// The denominator for any "how often does rule X fail" question that
|
|
100
|
+
// spans runs whose rulesets differ.
|
|
101
|
+
rules_run: results.map((result) => result.rule.id),
|
|
102
|
+
failed_rules: failed.map((result) => result.rule.id)
|
|
103
|
+
}
|
|
104
|
+
} ];
|
|
105
|
+
|
|
106
|
+
for (const result of results) {
|
|
107
|
+
events.push({
|
|
108
|
+
event: CHECK_EVENT,
|
|
109
|
+
distinct_id: distinctId,
|
|
110
|
+
timestamp,
|
|
111
|
+
properties: {
|
|
112
|
+
...shared,
|
|
113
|
+
rule_id: result.rule.id,
|
|
114
|
+
rule_name: result.rule.name,
|
|
115
|
+
status: result.todo ? "todo" : result.passed ? "passed" : "failed",
|
|
116
|
+
duration_ms: result.durationMs,
|
|
117
|
+
// A one-line summary, never the log tail. Multi-kilobyte blobs in
|
|
118
|
+
// event properties bloat the column store and slow every query that
|
|
119
|
+
// touches it; the full output is already on disk in the journal.
|
|
120
|
+
summary: summarize(result),
|
|
121
|
+
command:
|
|
122
|
+
result.rule.run ?? (result.rule.rubric ? `judge ${result.rule.rubric}` : null)
|
|
123
|
+
}
|
|
124
|
+
});
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
return events;
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
function summarize(result) {
|
|
131
|
+
if (result.todo) return "planned — not implemented yet";
|
|
132
|
+
if (result.passed) return null;
|
|
133
|
+
return result.output.split("\n").find((line) => line.trim())?.trim().slice(0, 120) ?? null;
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
// Who the run belongs to. The git identity is the one that means anything
|
|
137
|
+
// across machines — the same person on a laptop and a devcontainer should
|
|
138
|
+
// be one person — with the machine as the fallback when git has no
|
|
139
|
+
// identity configured (CI images, fresh containers).
|
|
140
|
+
export function distinctId(env = process.env) {
|
|
141
|
+
if (env.HIGHBALL_POSTHOG_DISTINCT_ID) return env.HIGHBALL_POSTHOG_DISTINCT_ID;
|
|
142
|
+
const email = git("git config user.email");
|
|
143
|
+
if (email) return email;
|
|
144
|
+
try {
|
|
145
|
+
return `${userInfo().username}@${hostname()}`;
|
|
146
|
+
} catch {
|
|
147
|
+
return `host-${hostname()}`;
|
|
148
|
+
}
|
|
149
|
+
}
|
package/lib/run.js
CHANGED
|
@@ -4,12 +4,21 @@
|
|
|
4
4
|
// output back). Reporting is the witness half and is best-effort: a dead
|
|
5
5
|
// dashboard must never block the agent.
|
|
6
6
|
import { execSync, spawnSync } from "node:child_process";
|
|
7
|
-
import {
|
|
7
|
+
import { readFileSync } from "node:fs";
|
|
8
|
+
import { loadConfig, resolvePosthog, commandFor } from "./config.js";
|
|
8
9
|
import { appendRun } from "./journal.js";
|
|
9
|
-
import {
|
|
10
|
+
import { readStamp, treeFingerprint, writeStamp } from "./stamp.js";
|
|
11
|
+
import { git } from "./git.js";
|
|
12
|
+
import { reportPosthog } from "./posthog.js";
|
|
10
13
|
import { judge } from "./judge.js";
|
|
11
14
|
import { latestUserPrompt } from "./transcript.js";
|
|
12
15
|
|
|
16
|
+
// Stamped onto reported events so a query can tell which runner
|
|
17
|
+
// produced them — rule semantics change between releases.
|
|
18
|
+
const VERSION = JSON.parse(
|
|
19
|
+
readFileSync(new URL("../package.json", import.meta.url), "utf8")
|
|
20
|
+
).version;
|
|
21
|
+
|
|
13
22
|
export async function run(args) {
|
|
14
23
|
// When an AI-judged rule spawns a judge session inside this repo, the
|
|
15
24
|
// judge inherits the repo's hooks — and its Stop hook would re-enter
|
|
@@ -26,6 +35,18 @@ export async function run(args) {
|
|
|
26
35
|
return 1;
|
|
27
36
|
}
|
|
28
37
|
|
|
38
|
+
// --if-changed: a fast hook that also matches Bash fires after every
|
|
39
|
+
// command, and most commands are reads. Skip outright when the working
|
|
40
|
+
// tree is exactly where the last run left it. The fingerprint is taken
|
|
41
|
+
// now and stamped after the checks, so a formatter that rewrites files
|
|
42
|
+
// mid-run makes the next call run again rather than trust a stale pass.
|
|
43
|
+
const fingerprint = treeFingerprint();
|
|
44
|
+
if (args.includes("--if-changed") && fingerprint &&
|
|
45
|
+
fingerprint === readStamp(config.project)) {
|
|
46
|
+
console.log("highball: no changes since the last run — skipped");
|
|
47
|
+
return 0;
|
|
48
|
+
}
|
|
49
|
+
|
|
29
50
|
// Rubric rules never join a fast run, even if a config marks one `fast`:
|
|
30
51
|
// LLM latency and cost would be paid on every edit. That belongs at turn
|
|
31
52
|
// end, and the invariant is enforced here rather than left to each repo.
|
|
@@ -86,16 +107,20 @@ export async function run(args) {
|
|
|
86
107
|
results.push({ rule, passed, todo: false, durationMs, output });
|
|
87
108
|
}
|
|
88
109
|
|
|
110
|
+
// Stamped pass or fail: after a failure the agent's next reads must not
|
|
111
|
+
// re-run and re-block; its next edit moves the tree and runs again.
|
|
112
|
+
if (fingerprint) writeStamp(config.project, fingerprint);
|
|
113
|
+
|
|
89
114
|
const failures = results.filter((result) => !result.passed);
|
|
90
115
|
const durationMs = Date.now() - startedAt.getTime();
|
|
91
116
|
const branch = git("git branch --show-current");
|
|
92
117
|
const commitSha = git("git rev-parse HEAD");
|
|
93
118
|
|
|
94
|
-
const {
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
119
|
+
const { host, key } = resolvePosthog(config);
|
|
120
|
+
if (host && key) {
|
|
121
|
+
await reportPosthog({
|
|
122
|
+
host, key, project: config.project, results, hook, fastOnly, startedAt,
|
|
123
|
+
durationMs, branch, commitSha, version: VERSION
|
|
99
124
|
});
|
|
100
125
|
}
|
|
101
126
|
|
|
@@ -118,7 +143,6 @@ export async function run(args) {
|
|
|
118
143
|
branch,
|
|
119
144
|
commit: commitSha,
|
|
120
145
|
status: failures.length === 0 ? "passed" : "failed",
|
|
121
|
-
reported_run_id: reportedRunId,
|
|
122
146
|
results: results.map((result) => ({
|
|
123
147
|
id: result.rule.id,
|
|
124
148
|
name: result.rule.name,
|
package/lib/stamp.js
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
// Working-tree fingerprint behind `run --fast --if-changed`.
|
|
2
|
+
//
|
|
3
|
+
// Agents in auto mode edit through Bash — sed, heredocs, scripts — so a
|
|
4
|
+
// fast hook matched only on the edit tools never fires for them. Matching
|
|
5
|
+
// Bash too means firing after every command, and most commands are reads.
|
|
6
|
+
// The fingerprint makes those free: HEAD plus every dirty path with its
|
|
7
|
+
// size and mtime, hashed. A further edit to an already-dirty file moves
|
|
8
|
+
// its mtime, so the stamp tracks edits rather than just the set of dirty
|
|
9
|
+
// paths. Stamps live per project outside the repo, like the journal.
|
|
10
|
+
import { createHash } from "node:crypto";
|
|
11
|
+
import { execSync } from "node:child_process";
|
|
12
|
+
import { mkdirSync, readFileSync, statSync, writeFileSync } from "node:fs";
|
|
13
|
+
import { homedir } from "node:os";
|
|
14
|
+
import { join } from "node:path";
|
|
15
|
+
|
|
16
|
+
export function stampDir() {
|
|
17
|
+
return join(homedir(), ".highball", "stamps");
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
// null outside a git repo: with no way to tell whether anything changed,
|
|
21
|
+
// --if-changed must never skip.
|
|
22
|
+
export function treeFingerprint(cwd = process.cwd()) {
|
|
23
|
+
const sh = (command) => {
|
|
24
|
+
try {
|
|
25
|
+
return execSync(command, { cwd, encoding: "utf8", stdio: [ "ignore", "pipe", "ignore" ] }).trim();
|
|
26
|
+
} catch {
|
|
27
|
+
return "";
|
|
28
|
+
}
|
|
29
|
+
};
|
|
30
|
+
const head = sh("git rev-parse HEAD");
|
|
31
|
+
if (!head) return null;
|
|
32
|
+
|
|
33
|
+
const hash = createHash("sha256").update(head).update("\0");
|
|
34
|
+
const status = sh("git status --porcelain=v1 -z --untracked-files=all");
|
|
35
|
+
for (const entry of status.split("\0").filter(Boolean)) {
|
|
36
|
+
hash.update(entry).update("\0");
|
|
37
|
+
try {
|
|
38
|
+
const stat = statSync(join(cwd, entry.slice(3)));
|
|
39
|
+
hash.update(`${stat.size}:${stat.mtimeMs}`);
|
|
40
|
+
} catch {
|
|
41
|
+
hash.update("gone");
|
|
42
|
+
}
|
|
43
|
+
hash.update("\0");
|
|
44
|
+
}
|
|
45
|
+
return hash.digest("hex");
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
export function readStamp(project, dir = stampDir()) {
|
|
49
|
+
try {
|
|
50
|
+
return readFileSync(join(dir, project), "utf8").trim();
|
|
51
|
+
} catch {
|
|
52
|
+
return null;
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
// Best-effort, like the journal: a stamp that fails to write costs one
|
|
57
|
+
// extra fast run, never a failed one.
|
|
58
|
+
export function writeStamp(project, fingerprint, dir = stampDir()) {
|
|
59
|
+
try {
|
|
60
|
+
mkdirSync(dir, { recursive: true, mode: 0o700 });
|
|
61
|
+
writeFileSync(join(dir, project), `${fingerprint}\n`);
|
|
62
|
+
} catch {
|
|
63
|
+
// ignore
|
|
64
|
+
}
|
|
65
|
+
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@profoundry-us/highball",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Highball runner — local CI for AI coding agents: runs a repo's .highball/checks.yml rules, blocks the agent on failure, and reports runs to
|
|
3
|
+
"version": "0.5.0",
|
|
4
|
+
"description": "Highball runner — local CI for AI coding agents: runs a repo's .highball/checks.yml rules, blocks the agent on failure, and optionally reports runs to PostHog.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ai",
|
|
7
7
|
"claude-code",
|
|
@@ -12,11 +12,11 @@
|
|
|
12
12
|
],
|
|
13
13
|
"repository": {
|
|
14
14
|
"type": "git",
|
|
15
|
-
"url": "git+https://github.com/profoundry-us/highball
|
|
15
|
+
"url": "git+https://github.com/profoundry-us/highball.git"
|
|
16
16
|
},
|
|
17
|
-
"homepage": "https://github.com/profoundry-us/highball
|
|
17
|
+
"homepage": "https://github.com/profoundry-us/highball#readme",
|
|
18
18
|
"bugs": {
|
|
19
|
-
"url": "https://github.com/profoundry-us/highball
|
|
19
|
+
"url": "https://github.com/profoundry-us/highball/issues"
|
|
20
20
|
},
|
|
21
21
|
"type": "module",
|
|
22
22
|
"bin": {
|
|
@@ -39,7 +39,8 @@
|
|
|
39
39
|
"dependencies": {
|
|
40
40
|
"@modelcontextprotocol/ext-apps": "^1.7.5",
|
|
41
41
|
"@modelcontextprotocol/sdk": "^1.30.0",
|
|
42
|
-
"yaml": "^2.5.0"
|
|
42
|
+
"yaml": "^2.5.0",
|
|
43
|
+
"zod": "^3.25 || ^4.0"
|
|
43
44
|
},
|
|
44
45
|
"license": "MIT"
|
|
45
46
|
}
|
package/lib/login.js
DELETED
|
@@ -1,68 +0,0 @@
|
|
|
1
|
-
// `highball login` — stores a project token in the machine-local
|
|
2
|
-
// credentials file (~/.highball/credentials.json, host → project →
|
|
3
|
-
// token). The token is read from stdin with --token-stdin (recommended:
|
|
4
|
-
// no shell history, no process listing) or prompted interactively.
|
|
5
|
-
// Tokens never touch the repo tree and are never echoed back.
|
|
6
|
-
import { chmodSync, mkdirSync, writeFileSync } from "node:fs";
|
|
7
|
-
import { dirname } from "node:path";
|
|
8
|
-
import { createInterface } from "node:readline/promises";
|
|
9
|
-
import { CREDENTIALS_PATH, loadConfig, readCredentials } from "./config.js";
|
|
10
|
-
|
|
11
|
-
export async function login(args) {
|
|
12
|
-
const options = parse(args);
|
|
13
|
-
|
|
14
|
-
// The local checks.yml, when present, already knows the url + project —
|
|
15
|
-
// don't make the user repeat what the repo declares.
|
|
16
|
-
let config = null;
|
|
17
|
-
try {
|
|
18
|
-
config = loadConfig();
|
|
19
|
-
} catch {
|
|
20
|
-
// Not in a configured repo; url/project must come from flags/prompts.
|
|
21
|
-
}
|
|
22
|
-
|
|
23
|
-
const rl = createInterface({ input: process.stdin, output: process.stdout });
|
|
24
|
-
try {
|
|
25
|
-
const url =
|
|
26
|
-
options.url ||
|
|
27
|
-
config?.reporting?.url ||
|
|
28
|
-
(await rl.question("Highball URL (e.g. https://highball.example.com): "));
|
|
29
|
-
const project =
|
|
30
|
-
options.project || config?.project || (await rl.question("Project slug: "));
|
|
31
|
-
const token = options.tokenStdin
|
|
32
|
-
? (await readAllStdin()).trim()
|
|
33
|
-
: (await rl.question("Project token (input is visible — prefer --token-stdin): ")).trim();
|
|
34
|
-
|
|
35
|
-
if (!url || !project || !token) {
|
|
36
|
-
console.error("highball: url, project, and token are all required.");
|
|
37
|
-
return 1;
|
|
38
|
-
}
|
|
39
|
-
|
|
40
|
-
const credentials = readCredentials();
|
|
41
|
-
(credentials[url] ??= {})[project] = token;
|
|
42
|
-
|
|
43
|
-
mkdirSync(dirname(CREDENTIALS_PATH), { recursive: true, mode: 0o700 });
|
|
44
|
-
writeFileSync(CREDENTIALS_PATH, JSON.stringify(credentials, null, 2) + "\n");
|
|
45
|
-
chmodSync(CREDENTIALS_PATH, 0o600);
|
|
46
|
-
|
|
47
|
-
console.log(`stored token for ${project} @ ${url} in ${CREDENTIALS_PATH}`);
|
|
48
|
-
return 0;
|
|
49
|
-
} finally {
|
|
50
|
-
rl.close();
|
|
51
|
-
}
|
|
52
|
-
}
|
|
53
|
-
|
|
54
|
-
function parse(args) {
|
|
55
|
-
const options = { tokenStdin: false };
|
|
56
|
-
for (let i = 0; i < args.length; i++) {
|
|
57
|
-
if (args[i] === "--url") options.url = args[++i];
|
|
58
|
-
else if (args[i] === "--project") options.project = args[++i];
|
|
59
|
-
else if (args[i] === "--token-stdin") options.tokenStdin = true;
|
|
60
|
-
}
|
|
61
|
-
return options;
|
|
62
|
-
}
|
|
63
|
-
|
|
64
|
-
async function readAllStdin() {
|
|
65
|
-
let text = "";
|
|
66
|
-
for await (const chunk of process.stdin) text += chunk;
|
|
67
|
-
return text;
|
|
68
|
-
}
|
package/lib/report.js
DELETED
|
@@ -1,75 +0,0 @@
|
|
|
1
|
-
// The witness half: best-effort reporting to the Highball app — open
|
|
2
|
-
// run, per-check results, finalize. Payloads mirror the Ruby
|
|
3
|
-
// proto-runner exactly so the ingestion API sees one dialect. Failures
|
|
4
|
-
// here warn and return; they NEVER fail the checks.
|
|
5
|
-
import { execSync } from "node:child_process";
|
|
6
|
-
import { hostname } from "node:os";
|
|
7
|
-
|
|
8
|
-
// Returns the server's run id on success, null when reporting was
|
|
9
|
-
// skipped or failed — the caller journals it either way.
|
|
10
|
-
export async function report({ url, token, rules, results, hook, fastOnly, startedAt, branch, commitSha }) {
|
|
11
|
-
try {
|
|
12
|
-
const base = new URL(url);
|
|
13
|
-
const request = async (method, path, body) => {
|
|
14
|
-
const response = await fetch(new URL(path, base), {
|
|
15
|
-
method,
|
|
16
|
-
headers: {
|
|
17
|
-
Authorization: `Bearer ${token}`,
|
|
18
|
-
"Content-Type": "application/json"
|
|
19
|
-
},
|
|
20
|
-
body: JSON.stringify(body),
|
|
21
|
-
signal: AbortSignal.timeout(15_000)
|
|
22
|
-
});
|
|
23
|
-
if (response.status >= 300) {
|
|
24
|
-
throw new Error(`${path} -> ${response.status}: ${await response.text()}`);
|
|
25
|
-
}
|
|
26
|
-
return response.json();
|
|
27
|
-
};
|
|
28
|
-
|
|
29
|
-
const opened = await request("POST", "/api/v1/runs", {
|
|
30
|
-
agent: hook.session_id ? "claude-code" : "manual",
|
|
31
|
-
session_key:
|
|
32
|
-
hook.session_id || process.env.HIGHBALL_SESSION_KEY || `manual-${hostname()}`,
|
|
33
|
-
trigger: fastOnly ? "edit" : "stop",
|
|
34
|
-
branch,
|
|
35
|
-
commit_sha: commitSha,
|
|
36
|
-
rules_snapshot: rules,
|
|
37
|
-
started_at: startedAt.toISOString()
|
|
38
|
-
});
|
|
39
|
-
const runId = opened.run_id;
|
|
40
|
-
|
|
41
|
-
for (const result of results) {
|
|
42
|
-
await request("POST", `/api/v1/runs/${runId}/results`, {
|
|
43
|
-
check_key: result.rule.id,
|
|
44
|
-
name: result.rule.name,
|
|
45
|
-
status: result.todo ? "todo" : result.passed ? "passed" : "failed",
|
|
46
|
-
duration_ms: result.durationMs,
|
|
47
|
-
// Full logs stay local; the tail is enough to read a failure on
|
|
48
|
-
// the dashboard without shipping megabytes per keystroke.
|
|
49
|
-
log_tail: result.passed ? null : result.output.slice(-4000),
|
|
50
|
-
summary: result.todo
|
|
51
|
-
? "planned — not implemented yet"
|
|
52
|
-
: result.passed
|
|
53
|
-
? null
|
|
54
|
-
: result.output.split("\n")[0]?.trim().slice(0, 120)
|
|
55
|
-
});
|
|
56
|
-
}
|
|
57
|
-
|
|
58
|
-
await request("PATCH", `/api/v1/runs/${runId}`, {
|
|
59
|
-
status: results.every((result) => result.passed) ? "passed" : "failed"
|
|
60
|
-
});
|
|
61
|
-
console.log(`reported to ${base.host} (run ${runId})`);
|
|
62
|
-
return runId;
|
|
63
|
-
} catch (error) {
|
|
64
|
-
console.error(`highball reporting skipped: ${error.message}`);
|
|
65
|
-
return null;
|
|
66
|
-
}
|
|
67
|
-
}
|
|
68
|
-
|
|
69
|
-
export function git(command) {
|
|
70
|
-
try {
|
|
71
|
-
return execSync(`${command} 2>/dev/null`, { encoding: "utf8" }).trim();
|
|
72
|
-
} catch {
|
|
73
|
-
return "";
|
|
74
|
-
}
|
|
75
|
-
}
|