tracehub 0.5.3 → 0.5.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -78,7 +78,7 @@ Everything is computed on the developer's machine, redacted there, checked again
78
78
  Set it up once with `setup` (the other commands are under [Commands](#commands)). Run it in your terminal:
79
79
 
80
80
  ```sh
81
- npx --yes tracehub@0.5.3 setup
81
+ npx --yes tracehub@0.5.4 setup
82
82
  ```
83
83
 
84
84
  Setup is a form of three steps. Each step saves what you chose before the next one starts; cancelling (Ctrl-C) keeps the steps already done.
@@ -89,7 +89,7 @@ Setup is a form of three steps. Each step saves what you chose before the next o
89
89
 
90
90
  It ends with a summary of the account, the allowed rules and the hooks, and what is left to do. Setup needs an interactive terminal; without one it exits 1.
91
91
 
92
- To skip npx on every commit, install it globally instead: `npm install -g tracehub@0.5.3`, then `tracehub setup`.
92
+ To skip npx on every commit, install it globally instead: `npm install -g tracehub@0.5.4`, then `tracehub setup`.
93
93
 
94
94
  ### Git hooks
95
95
 
@@ -110,7 +110,7 @@ Setup installs global Git hooks: it points the global `core.hooksPath` at `~/.co
110
110
  ### Uninstall
111
111
 
112
112
  ```sh
113
- npx --yes tracehub@0.5.3 uninstall
113
+ npx --yes tracehub@0.5.4 uninstall
114
114
  ```
115
115
 
116
116
  Lists what it removes and asks first (default no); with nothing set up it says so and asks nothing. Then it removes the global Git hooks and restores your previous global `core.hooksPath` setting, deletes the permission rules (`consent.json`) and signs you out (`auth.json`). It asks before deleting the deny list and the current repository's `.git/tracehub` (its markers, including ones not uploaded yet). Other repositories keep their `.git/tracehub`; delete it by hand if you want. Before signing out it asks TraceHub to forget this machine's rules (see [Consent registry](#consent-registry)); when TraceHub can't be reached it says that TraceHub still lists them, and goes on. If the global hooks setting was changed by another tool since setup, uninstall leaves the hooks alone, still removes the rest, and exits 1.
@@ -120,8 +120,8 @@ Lists what it removes and asks first (default no); with nothing set up it says s
120
120
  Uploads are restricted to repositories verified as owned by the GitHub organization **`assistant-ui`** (CLI 0.4.0 and later).
121
121
 
122
122
  - The repository is `origin`'s: all of its effective fetch and push URLs must name the same repository on `github.com`, with the exact owner `assistant-ui` (case insensitive). HTTPS, GitHub SSH URLs and `git@github.com:…` are accepted. A missing `origin`, or one with unknown or conflicting URLs, blocks uploads. A personal fork as `origin` with an organization upstream is blocked. Other remotes, such as contributors' forks kept for reviewing their pull requests, play no part.
123
- - Git's URL rewrites are resolved. A push uploads only when it goes to `origin`'s repository: the destination URL Git passes to the hook, and the pushed remote's own URLs, must name it. A push to a fork remote uploads nothing and the traces stay pending. The hooks forward `"$@"` and Git's stdin ref updates; missing arguments or ref input block upload, even if the configured origin is eligible. The commit hook skips trace markers for ineligible remotes; it makes no network request. When `TRACEHUB_ID` names a session but `origin` is not eligible, the commit hook says so in one line instead of skipping silently.
124
- - Before sending any trace or marker, the upload run asks GitHub for the canonical repository owner using the signed-in user's token. The returned owner must be the `assistant-ui` organization and the canonical repository name must match the remotes. Redirects, transfers, renames, missing fields, access failures and network failures block uploads. Update remotes after a rename. An organization-owned fork remains eligible; a fork owned elsewhere does not.
123
+ - Git's URL rewrites are resolved. A push uploads when it goes to `origin`'s repository or to a fork of it, such as a contributor's pull request branch a maintainer pushes to: the destination URL Git passes to the hook, and the pushed remote's own URLs, must name one repository on `github.com`. A push anywhere else uploads nothing and the traces stay pending. The hooks forward `"$@"` and Git's stdin ref updates; missing arguments or ref input block upload, even if the configured origin is eligible. The commit hook skips trace markers for ineligible remotes; it makes no network request. When `TRACEHUB_ID` names a session but `origin` is not eligible, the commit hook says so in one line instead of skipping silently.
124
+ - Before sending any trace or marker, the upload run asks GitHub for the canonical repository owner using the signed-in user's token. The returned owner must be the `assistant-ui` organization and the canonical repository name must match the remotes. Redirects, transfers, renames, missing fields, access failures and network failures block uploads. Update remotes after a rename. An organization-owned fork remains eligible as `origin`; a fork owned elsewhere does not. A push to a fork also asks GitHub for the pushed repository, which must be a fork whose parent is `origin`'s repository; anything else blocks the upload.
125
125
  - This applies to every upload of the pre-push hook, including previously pending markers. Upload methods also verify ownership themselves. A refused hook still exits 0, preserving pending work for a later push.
126
126
  - A permission rule and a sign-in remain required. This check does not grant access, change trace visibility, or send traces to GitHub; it runs fresh at upload time.
127
127
 
@@ -166,7 +166,7 @@ TraceHub keeps a copy of your rules (`_consent/users/<your id>`).
166
166
 
167
167
  ## How it works
168
168
 
169
- 1. **Commit.** `prepare-commit-msg` finds the sessions of the commit (see Discovery). If the message has no `Assisted-by` trailer yet, it adds one with a new 16-hex trace id. For each session it appends a marker to `.git/tracehub/markers.jsonl` (in the repository's common git directory, so a linked worktree's markers outlive the worktree): `{traceId, session, line, ts, context?, uploaded}`, where `line` is the number of complete lines the session file had and `context` the references of `TRACEHUB_CONTEXT`. Merge and squash commits are skipped.
169
+ 1. **Commit.** `prepare-commit-msg` finds the sessions of the commit (see Discovery). If the message has no `Assisted-by` trailer yet, it uses a new 16-hex trace id and adds the trailer once at least one session's marker is recorded; a commit whose sessions are all missing from this machine gets no trailer. For each session found it appends a marker to `.git/tracehub/markers.jsonl` (in the repository's common git directory, so a linked worktree's markers outlive the worktree): `{traceId, session, line, ts, context?, uploaded}`, where `line` is the number of complete lines the session file had and `context` the references of `TRACEHUB_CONTEXT`. Merge and squash commits are skipped.
170
170
  2. **Amend, rebase, squash.** The message keeps its trailer, so the commit keeps its trace id. A new marker for the same trace id and session replaces the old one.
171
171
  3. **Push.** `pre-push` reads Git's ref updates and uploads only the traces of commits being pushed. New branches exclude commits already known on that remote. Deletions upload nothing; an unavailable remote base leaves traces pending with a recovery warning. For each pending marker the CLI builds the trace (below), fetches its context conversations from their providers and `PUT`s it; each success is recorded before the next. A later push also retries the pending traces of earlier commits in its history. When the 15 s run out, the push says so and a detached `tracehub hook upload` goes on with the same traces for up to 10 minutes, writing to `.git/tracehub/upload.log`.
172
172
  4. **Pull request.** The GitHub App receives the `pull_request` event, reads the trace ids from the commit messages, and writes a link at the bottom of the description, between `<!-- tracehub:start -->` and `<!-- tracehub:end -->`: an "Agent traces" badge, next to Rupic's "Track in Rupic" badge when there is one. A pull request with attached context gets the link too.
@@ -222,9 +222,48 @@ Where the sessions are:
222
222
  | `providers remove <name>` | removes a provider |
223
223
  | `context add --pr <n> (<provider>:<id> … \| --from <file\|->) [--dry-run]` | attaches context conversations to a pull request |
224
224
  | `context remove --pr <n> <provider>:<id>…` | detaches context conversations from a pull request |
225
+ | `backfill --pr <n> [--repo <owner/repo>] [--session <agent:id>,…] [--dry-run]` | finds and uploads the agent sessions behind a pull request's untraced commits |
226
+ | `backfill <commit>… [--session <agent:id>,…] [--dry-run]` | the same for commits named |
225
227
 
226
228
  The hooks run `tracehub hook prepare-commit-msg` and `tracehub hook pre-push`, an internal entry point that `--help` leaves out. `docs/pages/cli-reference.mdx` renders the reference from `src/cli/usage.ts`, as `--help` does.
227
229
 
230
+ ## Backfill
231
+
232
+ Commits made before `tracehub setup`, or whose trace never reached TraceHub (a lost push), have no trace, and their pull request's check fails. `tracehub backfill` finds the agent sessions behind them on this machine and uploads them, so the check passes:
233
+
234
+ ```sh
235
+ npx --yes tracehub backfill --pr 386 --dry-run # print the evidence, upload nothing
236
+ npx --yes tracehub backfill --pr 386 # upload the sure matches
237
+ ```
238
+
239
+ Run it in a clone of the pull request's repository, on the machine where the agent ran; `--repo <owner/repo>` only confirms the clone. It needs a sign-in and a permission rule, as uploads do, and only works on pull requests you opened. Commits whose trace TraceHub already has, and merges, are skipped; a commit whose message names a trace TraceHub never got is backfilled under that trace id.
240
+
241
+ For each commit it scores the Claude Code sessions and subagents (`~/.claude/projects`), Codex rollouts (`~/.codex/sessions`) and OpenCode sessions that were written to after the commit's time and name one of its short shas or paths:
242
+
243
+ | evidence | points |
244
+ | --- | --- |
245
+ | `git commit` output (`[branch abc1234] …`) naming the commit | 6, ties |
246
+ | a `git commit` call carrying the commit's subject | 3 |
247
+ | its cwd, an edited file, or a `cd <dir> && git commit` in a checkout of the repository (this clone's worktrees, or any directory whose origin is the repository) | 2, ties |
248
+ | the pull request's URL in it (`gh pr create` prints it) | 3, ties |
249
+ | else the repository's GitHub URL in it (a push) | 1, ties |
250
+ | on the pull request's branch (Claude Code's `gitBranch`) | 2 |
251
+ | edits of the commit's files, up to 3 | 1 each |
252
+ | of the commit's added lines, the share its edits wrote: ≥50%, ≥20%, any | 3, 2, 1 |
253
+
254
+ Only what the session did before the commit counts (author time for a rebased or amended commit). A session with no tie to the repository is never uploaded. Confidence is **high** at 7 points with a tie, **medium** at 4. It prints, per commit, the likely sessions with their evidence, and uploads the high ones. `--session <agent>:<id>[,…]` considers only those sessions and uploads them at medium too: for an agent that has checked the evidence itself. A subagent whose parent session is uploaded is a run of that session's trace instead.
255
+
256
+ An upload is a normal one: the session cut where the commit was made, redacted on this machine (the detectors, the deny list, the home directory), checked again by the API, with the context its briefs name. The trace is then linked to the commit (`PUT /repos/:owner/:repo/commits/:sha/traces/:trace`), and the pull request's check is run again. Running it again reuses each commit's trace id, so it replaces rather than adds.
257
+
258
+ ### For coding agents
259
+
260
+ Asked to "upload traces for these PRs":
261
+
262
+ 1. In a clone of the repository, run `npx --yes tracehub backfill --pr <n> --dry-run` for each pull request.
263
+ 2. Read the evidence. High matches upload as they are. For a commit with only medium or low matches, look at the sessions yourself (their cwd, their `git commit` calls, the files they edited) and, when one clearly made the commit, name it with `--session`.
264
+ 3. Run it without `--dry-run`, then check the pull request's **TraceHub traces** check.
265
+ 4. Never name a session from another repository or unrelated work, and never backfill someone else's pull request.
266
+
228
267
  ## Context conversations
229
268
 
230
269
  A context conversation is what asked for the work. TraceHub gets it from a provider, by reference: `<provider>:<id>`, for example `nomos:t_16da1c327635`.
@@ -331,6 +370,7 @@ Literals in `~/.config/tracehub/deny.txt`, one per line, are always redacted. Ed
331
370
  | `PUT /repos/:owner/:repo/traces/:trace/sessions/:name` | stores a session's trace at a commit, replacing the one there |
332
371
  | `PUT /repos/:owner/:repo/pulls/:number/context` | attaches context conversations to a pull request (`{context: [...]}`), merged into the uploader's by conversation and message id |
333
372
  | `DELETE /repos/:owner/:repo/pulls/:number/context?ref=<provider>:<id>` | detaches the conversations (`ref` repeated) from every uploader's document; answers `{removed}`, 404 when none was attached |
373
+ | `PUT /repos/:owner/:repo/commits/:sha/traces/:trace` | links an uploaded trace to a commit whose message does not name it (a backfill); only who uploaded one of its sessions may; answers `{sha, traceId, pulls}`, 404 without a session, 503 when the GitHub App is not configured |
334
374
  | `POST /repos/:owner/:repo/pulls/:number/check` | recomputes the pull request's head check; answers `{submitted: true}` (bearer, push access), 503 when the GitHub App is not configured |
335
375
  | `PUT /consent` | registers a machine's permission rules for the signed-in user; `rules: []` forgets the machine |
336
376
  | `GET /login`, `/logout`, `/api/auth/callback` | browser sign-in (through the web app) |
@@ -349,7 +389,7 @@ The App of the deployed Tracehub API exists; the Tracehub API reads its key from
349
389
 
350
390
  For another deployment, create an App in the GitHub organization:
351
391
 
352
- - Permissions: Pull requests: write, Checks: write (the TraceHub traces check), Contents: read, Metadata: read, Merge queues: read. Events: Pull request, Push, Merge group. Webhook `<tracehub>/github/webhook`.
392
+ - Permissions: Pull requests: write, Checks: write (the TraceHub traces check), Contents: read, Metadata: read, Merge queues: read. Events: Pull request, Push, Merge group, Check run, Check suite. Webhook `<tracehub>/github/webhook`.
353
393
  - Callback URL `<tracehub>/api/auth/callback`, for the browser sign-in.
354
394
  - Device flow enabled, for the sign-in of `tracehub setup`.
355
395
  - A client secret, for the browser sign-in.
@@ -366,17 +406,19 @@ printf %s "<webhook secret>" | npx wrangler secret put GITHUB_WEBHOOK_SECRET
366
406
 
367
407
  ### The TraceHub traces check
368
408
 
369
- The App reports a check run named **TraceHub traces** on the head of every pull request of `assistant-ui` repositories, and on commits that have a known trace (an uploaded session marked by a trace id the commit message names).
409
+ The App reports a check run named **TraceHub traces** on the head of every pull request of `assistant-ui` repositories, and on commits that have a known trace (an uploaded session marked by a trace id the commit message names, or linked to the commit by `tracehub backfill`).
370
410
 
371
411
  - First party: the pull request's author is an organization member or collaborator, or has write access. Bots (`[bot]` logins, `type: Bot`) and outside contributors are exempt.
372
- - **Success** when at least one of the pull request's current commits has a known trace. The output lists each trace with a link to its TraceHub page, its agents and its number of sessions; the details link goes to the pull request's TraceHub page.
373
- - **Success** also when no commit has one but the pull request has context attached (`tracehub context add --pr`): at least one conversation with at least one message, from any uploader; a withheld message counts. The output says it passed on the attached context, with the number of conversations and a link to the pull request's TraceHub page, which the details link also opens.
412
+ - **Success** when at least one of the pull request's current commits has a known trace with content: a session that shows at least one prompt or reply with text, in its turns or a subagent's. A session whose messages are all empty or withheld is empty, and a trace of only empty sessions does not count. The output lists each trace with content with a link to its TraceHub page, its agents and its number of sessions with content; the details link goes to the pull request's TraceHub page.
413
+ - **Success** also when no commit has one but the pull request has context fetched from a provider attached (`tracehub context add --pr <n> <provider>:<id>`): at least one conversation with at least one message with text, from any uploader; withheld messages do not count. Conversations pasted from a file (`--from`, ref `manual:…`) stay on the page but never pass the check alone. The output says it passed on the attached context, with the number of conversations and a link to the pull request's TraceHub page, which the details link also opens.
374
414
  - **Skipped** when every pull request headed at the commit is exempt and has neither a trace nor context. GitHub treats this as passing for a required check.
375
- - **Failure** when a first-party pull request has neither. The output links to the public npm package and says how to fix it: run `npx --yes tracehub setup`, then push a new commit made in an agent session (Claude Code, Codex, OpenCode). A pull request without agent work passes once context is attached with `npx --yes tracehub context add --pr <n> …`. No comment is posted.
376
- - Any other commit gets a success when it has a known trace itself: on a push, and when an upload attaches a trace to a commit TraceHub has seen.
415
+ - **Failure** when a first-party pull request has neither. The title says what was found: `No trace found`, `<n> traces found, all empty` (the summary lists them), or `Only pasted context found`. The output links to the public npm package and says how to fix it: run `npx --yes tracehub setup`, then push a new commit made in an agent session (Claude Code, Codex, OpenCode), or, for commits made before that, run `npx --yes tracehub backfill --pr <n>` (see Backfill). A pull request without agent work passes once the conversation that asked for it is attached from its provider with `npx --yes tracehub context add --pr <n> <provider>:<id>`. No comment is posted.
416
+ - Any other commit gets a success when it has a known trace with content itself: on a push, and when an upload attaches a trace to a commit TraceHub has seen.
377
417
  - A trace uploaded later updates the check on the pull request heads and commits the index knows for the trace, so a failure turns into a success.
378
418
  - Attaching context (`PUT …/pulls/<n>/context`) or detaching it (`DELETE`) runs the check on the pull request's head again right away: a failure turns into a success, and detaching the last conversation can turn it back into a failure.
379
419
  - An unchanged check is not written again; a changed one updates the same check run.
420
+ - **Re-run** on GitHub (`check_run` or `check_suite` with action `rerequested`) lists the pull request's commits again and writes a new check run on the head, changed or not.
421
+ - R2 calls it turns away for a moment (10058 too many reads or writes of one object, 10001, 10043) are tried again a few times with a short backoff, so a busy object does not drop the job that updates the check.
380
422
  - On `merge_group` with action `checks_requested`, the App refreshes the pull request named by the queue ref and mirrors its stored head check onto the merge-group commit, including its conclusion, title, summary and details link. Queue branch pushes are ignored. A missing stored output is logged and no queue check is written.
381
423
  - `POST /repos/:owner/:repo/pulls/:number/check` recomputes the head check for a pull request opened before the App handled it. Push access is required.
382
424
 
@@ -388,7 +430,7 @@ Do not try it with the session you are working in: what is uploaded stays in the
388
430
 
389
431
  ```sh
390
432
  gh repo create assistant-ui/tracehub-playground --private --clone && cd tracehub-playground
391
- npx --yes tracehub@0.5.3 setup # allow this repository, install the hooks
433
+ npx --yes tracehub@0.5.4 setup # allow this repository, install the hooks
392
434
 
393
435
  # open a fresh agent session in this directory, ask it for something harmless
394
436
  export TRACEHUB_ID=claude:<throwaway session id>
package/dist/cli/args.js CHANGED
@@ -1,5 +1,5 @@
1
1
  // Flags that take a value: `--pr 12` or `--pr=12`.
2
- const VALUE_FLAGS = new Set(["pr", "from", "timeout"]);
2
+ const VALUE_FLAGS = new Set(["pr", "from", "timeout", "repo", "session"]);
3
3
  export function parseArgs(argv) {
4
4
  const positional = [];
5
5
  const flags = {};
@@ -0,0 +1,320 @@
1
+ import { existsSync, readFileSync } from "node:fs";
2
+ import { basename, dirname, join } from "node:path";
3
+ import { freshAuth } from "../../client/auth.js";
4
+ import { pullCommits } from "../../client/github.js";
5
+ import { traceUploaded, tracehubUrl, } from "../../client/tracehub.js";
6
+ import { uploadMarkers } from "../../client/upload.js";
7
+ import { sameTarget, targetOfUploadRemotes, UPLOAD_POLICY_ERROR, } from "../../client/uploadPolicy.js";
8
+ import { addedLines, git, remoteUrls, repoRoot, tryGit, } from "../../git/git.js";
9
+ import { appendMarkers, readMarkers, } from "../../git/markers.js";
10
+ import { newTraceId, traceIds } from "../../git/trailers.js";
11
+ import { candidateOf, chosen, commitTimeMs, rankMatches, } from "../../local/backfill.js";
12
+ import { claudeIdFromFile, ranInside, walkCodexFiles, } from "../../local/discover.js";
13
+ import { fileEntry, firstLineOf, listDir } from "../../local/files.js";
14
+ import { completeLines } from "../../local/lines.js";
15
+ import { listOpencodeSessions, opencodeLines } from "../../local/opencode.js";
16
+ import { claudeProjectsDir, home, opencodeDb } from "../../local/paths.js";
17
+ import { formatSessionRef, parseSessionRef, } from "../../protocol/sessionRef.js";
18
+ import { codexSessionMeta } from "../../transcript/parseCodex.js";
19
+ import { flagValue } from "../args.js";
20
+ import { out, warn } from "../io.js";
21
+ import { clientFor, redactOptions, reportUpload, usableAuth, } from "../remote.js";
22
+ import { consentFor, stateDir, uploadTargetFor } from "../repo.js";
23
+ import { commandUsage } from "../usage.js";
24
+ const HOUR_MS = 3600_000;
25
+ function prOf(args) {
26
+ const raw = flagValue(args, "pr");
27
+ if (raw === undefined)
28
+ return undefined;
29
+ const pr = Number(raw);
30
+ if (!Number.isSafeInteger(pr) || pr < 1)
31
+ throw new Error("--pr <number> must be a pull request number");
32
+ return pr;
33
+ }
34
+ // The commit as git has it; undefined when this clone does not have it.
35
+ function commitFacts(cwd, sha) {
36
+ const head = tryGit(["show", "-s", "--format=%H%x00%P%x00%s%x00%at%x00%ct", sha], { cwd });
37
+ if (!head)
38
+ return undefined;
39
+ const [full, parents, subject, at, ct] = head.split("\0");
40
+ // A merge's changes are its other parent's, not a session's.
41
+ if ((parents ?? "").trim().split(/\s+/).length > 1)
42
+ return undefined;
43
+ const files = git([
44
+ "-c",
45
+ "core.quotePath=false",
46
+ "show",
47
+ "--first-parent",
48
+ "--format=",
49
+ "--name-only",
50
+ full,
51
+ ], { cwd })
52
+ .split("\n")
53
+ .filter(Boolean);
54
+ return {
55
+ sha: full,
56
+ subject: subject ?? "",
57
+ authorMs: Number(at) * 1000,
58
+ committerMs: Number(ct) * 1000,
59
+ files,
60
+ added: addedLines(cwd, full),
61
+ };
62
+ }
63
+ // Every checkout of the repository: this clone's worktrees, and any other
64
+ // directory whose origin is the same GitHub repository.
65
+ function repoOf(cwd, target, branch, pull) {
66
+ const checkouts = (tryGit(["worktree", "list", "--porcelain"], { cwd }) ?? "")
67
+ .split("\n")
68
+ .filter((l) => l.startsWith("worktree "))
69
+ .map((l) => l.slice("worktree ".length));
70
+ const roots = new Map();
71
+ const nearestDir = (path) => {
72
+ for (let at = path;; at = dirname(at)) {
73
+ if (existsSync(at))
74
+ return at;
75
+ if (dirname(at) === at)
76
+ return undefined;
77
+ }
78
+ };
79
+ return {
80
+ branch,
81
+ home: home(),
82
+ slug: `${target.owner}/${target.repo}`,
83
+ pull,
84
+ contains(path) {
85
+ if (checkouts.some((c) => ranInside(path, c)))
86
+ return true;
87
+ const dir = nearestDir(path);
88
+ if (!dir)
89
+ return false;
90
+ const root = repoRoot(dir);
91
+ if (!root)
92
+ return false;
93
+ if (!roots.has(root)) {
94
+ const other = targetOfUploadRemotes(remoteUrls(root, "origin"));
95
+ roots.set(root, !!other && sameTarget(other, target));
96
+ }
97
+ return roots.get(root);
98
+ },
99
+ };
100
+ }
101
+ // The sessions written to since `sinceMs` whose text names one of the
102
+ // commits or one of their files: Claude Code sessions and subagents, Codex
103
+ // rollouts, OpenCode sessions.
104
+ function candidates(sinceMs, hints) {
105
+ const found = [];
106
+ const mentions = (text) => hints.some((h) => text.includes(h));
107
+ const read = (file) => {
108
+ try {
109
+ return readFileSync(file, "utf8");
110
+ }
111
+ catch {
112
+ return "";
113
+ }
114
+ };
115
+ const base = claudeProjectsDir();
116
+ for (const project of listDir(base)) {
117
+ for (const name of listDir(join(base, project))) {
118
+ const path = join(base, project, name);
119
+ if (name.endsWith(".jsonl")) {
120
+ const id = claudeIdFromFile(name);
121
+ const entry = fileEntry(path);
122
+ if (!id || !entry || entry.mtimeMs < sinceMs)
123
+ continue;
124
+ const text = read(path);
125
+ if (!mentions(text))
126
+ continue;
127
+ found.push(candidateOf({ agent: "claude", id }, path, completeLines(text)));
128
+ continue;
129
+ }
130
+ const dir = join(path, "subagents");
131
+ for (const sub of listDir(dir)) {
132
+ if (!/^agent-[A-Za-z0-9_-]+\.jsonl$/.test(sub))
133
+ continue;
134
+ const file = join(dir, sub);
135
+ const entry = fileEntry(file);
136
+ if (!entry || entry.mtimeMs < sinceMs)
137
+ continue;
138
+ const text = read(file);
139
+ if (!mentions(text))
140
+ continue;
141
+ found.push(candidateOf({ agent: "claude", id: basename(sub, ".jsonl") }, file, completeLines(text), { sidechain: true, parent: `claude:${name}` }));
142
+ }
143
+ }
144
+ }
145
+ for (const { file } of walkCodexFiles(sinceMs)) {
146
+ const meta = codexSessionMeta(firstLineOf(file));
147
+ if (!meta)
148
+ continue;
149
+ const text = read(file);
150
+ if (!mentions(text))
151
+ continue;
152
+ found.push(candidateOf({ agent: "codex", id: meta.id }, file, completeLines(text), {
153
+ ...(meta.parentId ? { parent: `codex:${meta.parentId}` } : {}),
154
+ }));
155
+ }
156
+ for (const { id } of listOpencodeSessions(sinceMs)) {
157
+ const lines = opencodeLines(id);
158
+ if (!mentions(lines.join("\n")))
159
+ continue;
160
+ found.push(candidateOf({ agent: "opencode", id }, opencodeDb(), lines));
161
+ }
162
+ return found;
163
+ }
164
+ const day = (ms) => new Date(ms).toISOString().slice(0, 16).replace("T", " ");
165
+ // Finds the sessions that made commits traced nowhere, prints the evidence,
166
+ // and uploads the ones it is sure of through the normal upload path, linked
167
+ // to their commits.
168
+ async function backfill(args, cwd) {
169
+ const pr = prOf(args);
170
+ const shas = args.positional.slice(1);
171
+ if (!pr && !shas.length)
172
+ throw new Error('name a pull request with --pr <n>, or commits; see "tracehub backfill --help"');
173
+ const root = repoRoot(cwd);
174
+ if (!root)
175
+ throw new Error("not a git repository; run it in a clone of the repository");
176
+ const target = uploadTargetFor(cwd);
177
+ if (!target)
178
+ throw new Error(UPLOAD_POLICY_ERROR);
179
+ const named = flagValue(args, "repo");
180
+ if (named && named.toLowerCase() !== `${target.owner}/${target.repo}`)
181
+ throw new Error(`this clone's origin is ${target.owner}/${target.repo}, not ${named}; run it in a clone of ${named}`);
182
+ const dryRun = !!args.flags["dry-run"];
183
+ const only = (flagValue(args, "session") ?? "")
184
+ .split(",")
185
+ .filter(Boolean)
186
+ .map((s) => {
187
+ const ref = parseSessionRef(s);
188
+ if (!ref)
189
+ throw new Error(`--session: "${s}" is not <claude|codex|opencode>:<id>`);
190
+ return formatSessionRef(ref);
191
+ });
192
+ if (!dryRun && consentFor(cwd) !== true)
193
+ throw new Error(`no permission rule allows uploads for ${target.owner}/${target.repo}; run "tracehub setup"`);
194
+ const stored = usableAuth();
195
+ if (!stored)
196
+ throw new Error('not signed in; run "tracehub setup"');
197
+ const auth = await freshAuth(stored);
198
+ // The commits and their messages, and the pull request's head branch.
199
+ let branch;
200
+ let listed;
201
+ if (pr) {
202
+ const pull = await pullCommits(target, pr, auth.accessToken);
203
+ if (pull.author.toLowerCase() !== auth.login.toLowerCase())
204
+ throw new Error(`${target.owner}/${target.repo}#${pr} was opened by ${pull.author}, not ${auth.login}; backfill only uploads to your own pull requests`);
205
+ branch = pull.head;
206
+ listed = pull.commits;
207
+ if (listed.some(({ sha }) => !tryGit(["cat-file", "-e", `${sha}^{commit}`], { cwd })))
208
+ tryGit(["fetch", "--quiet", "origin", `pull/${pr}/head`], { cwd });
209
+ }
210
+ else
211
+ listed = shas.map((sha) => ({
212
+ sha,
213
+ message: tryGit(["log", "-1", "--format=%B", sha], { cwd }) ?? "",
214
+ }));
215
+ // A commit whose message names a trace that was uploaded is traced; one
216
+ // whose named trace never reached TraceHub (its push was lost) is
217
+ // backfilled under that trace id.
218
+ const commits = [];
219
+ const traceOf = new Map();
220
+ for (const { sha, message } of listed) {
221
+ const named = traceIds(message);
222
+ let uploaded = false;
223
+ for (const id of named) {
224
+ try {
225
+ uploaded ||= await traceUploaded(target, id, auth.accessToken);
226
+ }
227
+ catch (err) {
228
+ warn(`trace ${id}: ${err.message}; counted as uploaded`);
229
+ uploaded = true;
230
+ }
231
+ }
232
+ if (uploaded) {
233
+ out(`${sha.slice(0, 7)} already has trace ${named.join(", ")}; skipped`);
234
+ continue;
235
+ }
236
+ const facts = commitFacts(cwd, sha);
237
+ if (!facts) {
238
+ warn(`commit ${sha.slice(0, 7)} is a merge or not in this clone; skipped`);
239
+ continue;
240
+ }
241
+ commits.push(facts);
242
+ if (named[0])
243
+ traceOf.set(facts.sha, named[0]);
244
+ }
245
+ if (!commits.length) {
246
+ out("nothing to backfill");
247
+ return;
248
+ }
249
+ const since = Math.min(...commits.map(commitTimeMs)) - HOUR_MS;
250
+ const hints = [
251
+ ...commits.map((c) => c.sha.slice(0, 7)),
252
+ ...commits.flatMap((c) => c.files),
253
+ ];
254
+ const repo = repoOf(cwd, target, branch, pr);
255
+ let pool = candidates(since, hints);
256
+ if (only.length)
257
+ pool = pool.filter((c) => only.includes(formatSessionRef(c.ref)));
258
+ // A commit's earlier backfill keeps its trace id.
259
+ const dir = stateDir(cwd);
260
+ const earlier = new Map(readMarkers(dir)
261
+ .filter((m) => m.commit)
262
+ .map((m) => [m.commit, m.traceId]));
263
+ const markers = [];
264
+ for (const commit of commits) {
265
+ out(`${commit.sha.slice(0, 7)} ${commit.subject} (${day(commitTimeMs(commit))} UTC)`);
266
+ // The likely ones, or the best of the unlikely.
267
+ const all = rankMatches(commit, pool, repo);
268
+ const likely = all.filter((r) => r.match.confidence !== "low");
269
+ const ranked = (likely.length ? likely : all.slice(0, 1)).slice(0, 5);
270
+ if (!ranked.length)
271
+ out(" no session found");
272
+ const traceId = traceOf.get(commit.sha) ?? earlier.get(commit.sha) ?? newTraceId();
273
+ for (const { match } of ranked) {
274
+ const take = chosen(match, only.includes(match.session));
275
+ out(` ${take ? "upload" : "skip "} ${match.confidence.padEnd(6)} ${String(match.score).padStart(2)} ${match.session}${match.tied ? "" : " (not tied to this repository)"}`);
276
+ out(` ${match.evidence.join("; ") || "no evidence"}`);
277
+ if (take)
278
+ markers.push({
279
+ traceId,
280
+ session: match.session,
281
+ line: match.cut,
282
+ ts: new Date(commitTimeMs(commit)).toISOString(),
283
+ commit: commit.sha,
284
+ uploaded: false,
285
+ });
286
+ }
287
+ }
288
+ const traced = new Set(markers.map((m) => m.commit)).size;
289
+ out(`${traced} of ${commits.length} commit(s) have a session to upload`);
290
+ if (dryRun || !markers.length) {
291
+ if (dryRun)
292
+ out("dry run: nothing uploaded");
293
+ return;
294
+ }
295
+ appendMarkers(dir, markers);
296
+ const client = clientFor(auth);
297
+ const result = await uploadMarkers(client, target, dir, cwd, root, markers, redactOptions(), true);
298
+ reportUpload(result, target, out);
299
+ const linked = new Map();
300
+ for (const m of result.uploaded)
301
+ linked.set(m.commit, m.traceId);
302
+ for (const [sha, traceId] of linked) {
303
+ await client.linkCommit(target, sha, traceId);
304
+ out(`linked trace ${traceId} to ${sha.slice(0, 7)}`);
305
+ }
306
+ if (pr && linked.size) {
307
+ await client.checkPull(target, pr);
308
+ out(`asked TraceHub to check ${target.owner}/${target.repo}#${pr} again`);
309
+ out(`${tracehubUrl()}/pr/${target.owner}/${target.repo}/${pr}`);
310
+ }
311
+ if (result.failed)
312
+ throw new Error(`upload incomplete: ${result.failed.error}`);
313
+ }
314
+ export async function cmdBackfill(args, cwd) {
315
+ if (args.flags["help"] || args.flags["h"]) {
316
+ out(commandUsage("backfill"));
317
+ return;
318
+ }
319
+ await backfill(args, cwd);
320
+ }
@@ -17,7 +17,7 @@ import { formatSessionRef } from "../../protocol/sessionRef.js";
17
17
  import { ENV_CONTEXT, ENV_SESSION_ID } from "../../local/paths.js";
18
18
  import { warn } from "../io.js";
19
19
  import { BACKGROUND_BUDGET_MS, clientFor, commitCount, pushBudgetMs, ranOutOfTime, reportUpload, redactOptions, usableAuth, } from "../remote.js";
20
- import { consentFor, hooksDisabled, resolveSessions, stateDir, uploadTargetFor, } from "../repo.js";
20
+ import { consentFor, hooksDisabled, pushTargetFor, resolveSessions, stateDir, uploadTargetFor, } from "../repo.js";
21
21
  // The trailer names the commit's sessions on tracehub, so a message that
22
22
  // already carries one (amend, rebase, cherry-pick, a squash folding earlier
23
23
  // messages) keeps it, and the markers recorded under it stay with the commit.
@@ -44,14 +44,18 @@ function hookPrepareCommitMsg(args, cwd) {
44
44
  warn(`${refs.length} agent session(s) could have been attached to this commit; run "tracehub setup" to allow uploads for this repository or its organization`);
45
45
  return;
46
46
  }
47
- let id = traceIds(readFileSync(messageFile, "utf8")).at(-1);
48
- if (!id) {
49
- id = newTraceId();
50
- git(interpretTrailersArgs(target, id, messageFile), { cwd });
51
- }
52
- for (const ref of recordMarkers(stateDir(cwd), id, refs, new Date(), envContextRefs())) {
47
+ const kept = traceIds(readFileSync(messageFile, "utf8")).at(-1);
48
+ const id = kept ?? newTraceId();
49
+ const missing = recordMarkers(stateDir(cwd), id, refs, new Date(), envContextRefs());
50
+ for (const ref of missing)
53
51
  warn(`session ${formatSessionRef(ref)} not found on this machine; no marker recorded`);
54
- }
52
+ if (kept)
53
+ return;
54
+ // A trailer without a marker would link to a trace nothing ever uploads.
55
+ if (missing.length < refs.length)
56
+ git(interpretTrailersArgs(target, id, messageFile), { cwd });
57
+ else
58
+ warn("no trace attached to this commit");
55
59
  }
56
60
  // The context refs of TRACEHUB_CONTEXT; one that is not a ref is reported.
57
61
  function envContextRefs() {
@@ -75,14 +79,15 @@ async function hookPrePush(args, cwd) {
75
79
  warn('pre-push requires Git\'s destination arguments; forward "$@" from the hook; nothing was uploaded');
76
80
  return;
77
81
  }
78
- const target = uploadTargetFor(cwd, {
82
+ const push = pushTargetFor(cwd, {
79
83
  remote: args.positional[2],
80
84
  url: pushedUrl,
81
85
  });
82
- if (!target) {
86
+ if (!push) {
83
87
  warn(UPLOAD_POLICY_ERROR);
84
88
  return;
85
89
  }
90
+ const { target } = push;
86
91
  const dir = stateDir(cwd);
87
92
  const input = process.stdin.isTTY ? "" : readFileSync(0, "utf8");
88
93
  if (!input.trim()) {
@@ -115,7 +120,7 @@ async function hookPrePush(args, cwd) {
115
120
  const auth = await freshAuth(stored);
116
121
  await retryRegistration(auth.accessToken);
117
122
  const client = clientFor(auth, pushBudgetMs());
118
- await client.verifyTarget(target);
123
+ await client.verifyPush(push);
119
124
  const result = await uploadMarkers(client, target, dir, cwd, repoRoot(cwd) ?? cwd, markers, redactOptions(), consentFor(cwd) === true);
120
125
  // A big session can take longer than a push waits. The rest goes on in
121
126
  // the background, so a commit whose branch is never pushed again (a
@@ -154,11 +159,12 @@ async function hookUpload(args, cwd) {
154
159
  const [pushedUrl, ...ids] = args.positional.slice(2);
155
160
  if (!pushedUrl || !ids.length)
156
161
  return;
157
- const target = uploadTargetFor(cwd, { url: pushedUrl });
158
- if (!target) {
162
+ const push = pushTargetFor(cwd, { url: pushedUrl });
163
+ if (!push) {
159
164
  warn(UPLOAD_POLICY_ERROR);
160
165
  return;
161
166
  }
167
+ const { target } = push;
162
168
  const dir = stateDir(cwd);
163
169
  const markers = readMarkers(dir).filter((marker) => ids.includes(marker.traceId));
164
170
  if (!pendingMarkers(markers).length)
@@ -171,7 +177,7 @@ async function hookUpload(args, cwd) {
171
177
  try {
172
178
  const auth = await freshAuth(stored);
173
179
  const client = clientFor(auth, BACKGROUND_BUDGET_MS);
174
- await client.verifyTarget(target);
180
+ await client.verifyPush(push);
175
181
  const result = await uploadMarkers(client, target, dir, cwd, repoRoot(cwd) ?? cwd, markers, redactOptions(), consentFor(cwd) === true);
176
182
  reportUpload(result, target, warn);
177
183
  }
package/dist/cli/repo.js CHANGED
@@ -2,7 +2,7 @@ import { mkdirSync } from "node:fs";
2
2
  import { join } from "node:path";
3
3
  import { getConsent, readConsent } from "../client/consent.js";
4
4
  import { gitDir, headTimestampMs, originUrl, remoteUrls } from "../git/git.js";
5
- import { sameTarget, targetOfUploadRemotes } from "../client/uploadPolicy.js";
5
+ import { repositoryOfRemotes, sameTarget, targetOfUploadRemotes, } from "../client/uploadPolicy.js";
6
6
  import { pendingCommitCall } from "../local/commitBoundary.js";
7
7
  import { sessionLines } from "../local/lines.js";
8
8
  import { opencodePendingCommitCall } from "../local/opencode.js";
@@ -55,16 +55,25 @@ export function consentFor(cwd) {
55
55
  }
56
56
  // The repository traces upload to is origin's: every effective fetch/push
57
57
  // URL of origin must name it. Other remotes (contributors' forks kept for
58
- // review) play no part. A push must go to that same repository: the URLs of
59
- // the remote being pushed to, when it is a configured remote, and the URL Git
60
- // passes to the hook.
61
- export function uploadTargetFor(cwd, push) {
62
- const target = targetOfUploadRemotes(remoteUrls(cwd, "origin"));
63
- if (!target || !push)
64
- return target;
58
+ // review) play no part.
59
+ export function uploadTargetFor(cwd) {
60
+ return targetOfUploadRemotes(remoteUrls(cwd, "origin"));
61
+ }
62
+ // A push uploads when it goes to origin's repository or to a fork of it,
63
+ // which the upload run confirms with GitHub. The destination is the URL Git
64
+ // passes to the hook and, when it is a configured remote, the pushed remote's
65
+ // own URLs, all naming one GitHub repository.
66
+ export function pushTargetFor(cwd, push) {
67
+ const target = uploadTargetFor(cwd);
68
+ if (!target)
69
+ return undefined;
65
70
  const pushed = [push.url];
66
71
  if (push.remote && push.remote !== push.url)
67
72
  pushed.push(...remoteUrls(cwd, push.remote));
68
- const destination = targetOfUploadRemotes(pushed);
69
- return destination && sameTarget(destination, target) ? target : undefined;
73
+ const destination = repositoryOfRemotes(pushed);
74
+ if (!destination)
75
+ return undefined;
76
+ return sameTarget(destination, target)
77
+ ? { target }
78
+ : { target, fork: destination };
70
79
  }
package/dist/cli/usage.js CHANGED
@@ -56,6 +56,18 @@ export const COMMANDS = [
56
56
  "tracehub context remove --pr 12 manual:5f2de08703b94d2b",
57
57
  ],
58
58
  },
59
+ {
60
+ name: "backfill",
61
+ usage: "backfill (--pr <n> | <commit>…) [--dry-run]",
62
+ summary: "find and upload the agent sessions behind commits made before tracing",
63
+ about: "For each commit of your pull request (or each commit named) that has no trace on TraceHub, merges aside, scores the Claude Code, Codex and OpenCode sessions on this machine by what they left: git commit output naming the commit, a git commit call with its subject, a cwd or edit in a checkout of the repository, the pull request's URL, its branch, edits of the commit's files and of the lines it adds, all before the commit's time. It prints the evidence per commit and uploads, through the normal upload path (redacted, cut at the commit), each session of high confidence that is tied to this repository, links its trace to the commit and asks TraceHub to check the pull request again. --session <agent:id>[,…] only considers those sessions and also uploads them at medium confidence. --dry-run prints the matches and uploads nothing. Run it in a clone of the repository, on the machine where the agent ran. It needs a sign-in, a permission rule for the repository and, with --pr, a pull request you opened. --repo <owner/repo> confirms the clone.",
64
+ examples: [
65
+ "tracehub backfill --pr 386 --dry-run",
66
+ "tracehub backfill --pr 386",
67
+ "tracehub backfill --pr 386 --session claude:agent-a27592ec8a99c1afd",
68
+ "tracehub backfill 1a2b3c4d",
69
+ ],
70
+ },
59
71
  ];
60
72
  export const ENV = [
61
73
  {
package/dist/cli.js CHANGED
@@ -1,5 +1,6 @@
1
1
  #!/usr/bin/env node
2
2
  import { parseArgs } from "./cli/args.js";
3
+ import { cmdBackfill } from "./cli/commands/backfill.js";
3
4
  import { cmdHook } from "./cli/commands/hook.js";
4
5
  import { cmdContext } from "./cli/commands/context.js";
5
6
  import { cmdProviders } from "./cli/commands/providers.js";
@@ -12,6 +13,7 @@ const COMMAND_HANDLERS = {
12
13
  uninstall: cmdUninstall,
13
14
  providers: cmdProviders,
14
15
  context: cmdContext,
16
+ backfill: cmdBackfill,
15
17
  };
16
18
  async function main() {
17
19
  const args = parseArgs(process.argv.slice(2));
@@ -127,3 +127,39 @@ export async function loginOf(client, token) {
127
127
  throw new Error("GitHub named no user");
128
128
  return login;
129
129
  }
130
+ async function githubGet(path, token) {
131
+ const res = await fetch(`${GITHUB_API}${path}`, {
132
+ headers: {
133
+ accept: "application/vnd.github+json",
134
+ authorization: `Bearer ${token}`,
135
+ "user-agent": "tracehub-cli",
136
+ "x-github-api-version": "2022-11-28",
137
+ },
138
+ signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
139
+ });
140
+ if (!res.ok)
141
+ throw new Error(`GitHub answered ${res.status} for ${path}`);
142
+ return res.json();
143
+ }
144
+ export async function pullCommits(repo, number, token) {
145
+ const base = `/repos/${repo.owner}/${repo.repo}/pulls/${number}`;
146
+ const pull = (await githubGet(base, token));
147
+ const commits = [];
148
+ for (let page = 1; page <= 3; page++) {
149
+ const listed = (await githubGet(`${base}/commits?per_page=100&page=${page}`, token));
150
+ for (const c of listed)
151
+ commits.push({
152
+ sha: c.sha,
153
+ message: c.commit?.message ?? "",
154
+ authorAt: c.commit?.author?.date ?? "",
155
+ committedAt: c.commit?.committer?.date ?? "",
156
+ });
157
+ if (listed.length < 100)
158
+ break;
159
+ }
160
+ return {
161
+ author: pull.user?.login ?? "",
162
+ head: pull.head?.ref ?? "",
163
+ commits,
164
+ };
165
+ }
@@ -1,4 +1,5 @@
1
- import { verifyUploadOwnership } from "./uploadPolicy.js";
1
+ import { DASHBOARD_URL } from "../git/trailers.js";
2
+ import { verifyFork, verifyUploadOwnership, } from "./uploadPolicy.js";
2
3
  export const DEFAULT_TRACEHUB_URL = "https://assisted-by.aui.dev";
3
4
  export const ENV_TRACEHUB = "TRACEHUB_URL";
4
5
  const REQUEST_TIMEOUT_MS = 15_000;
@@ -51,6 +52,11 @@ export class TracehubClient {
51
52
  }
52
53
  return verified;
53
54
  }
55
+ async verifyPush(push) {
56
+ await this.verifyTarget(push.target);
57
+ if (push.fork)
58
+ await verifyFork(push.fork, push.target, this.token, this.timeLeft());
59
+ }
54
60
  async request(method, path, body) {
55
61
  const timeLeft = this.timeLeft();
56
62
  if (timeLeft <= 0)
@@ -76,6 +82,20 @@ export class TracehubClient {
76
82
  const res = await this.request("PUT", `${repo}/traces/${traceId}/sessions/${name}`, upload);
77
83
  return (await res.json());
78
84
  }
85
+ // Links a trace to a commit whose message does not name it (a backfill).
86
+ // Answers the pull requests tracehub knows the commit in.
87
+ async linkCommit(target, sha, traceId) {
88
+ await this.verifyTarget(target);
89
+ const repo = `/repos/${target.owner}/${target.repo}`;
90
+ const res = await this.request("PUT", `${repo}/commits/${sha}/traces/${traceId}`);
91
+ return (await res.json());
92
+ }
93
+ // Recomputes the "TraceHub traces" check on the pull request's head.
94
+ async checkPull(target, number) {
95
+ await this.verifyTarget(target);
96
+ const repo = `/repos/${target.owner}/${target.repo}`;
97
+ await this.request("POST", `${repo}/pulls/${number}/check`);
98
+ }
79
99
  // Adds context conversations to the pull request, merged by conversation
80
100
  // and message id.
81
101
  async putContext(target, number, context) {
@@ -96,3 +116,20 @@ export class TracehubClient {
96
116
  return (await res.json());
97
117
  }
98
118
  }
119
+ // Whether TraceHub has any session of the trace, asked of the dashboard's
120
+ // `.json` page with the user's GitHub token.
121
+ export async function traceUploaded(target, traceId, token) {
122
+ const res = await fetch(`${DASHBOARD_URL}/t/${target.owner}/${target.repo}/${traceId}.json`, {
123
+ headers: {
124
+ authorization: `Bearer ${token}`,
125
+ "user-agent": "tracehub-cli",
126
+ },
127
+ redirect: "manual",
128
+ signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
129
+ });
130
+ if (res.status === 404)
131
+ return false;
132
+ if (res.ok)
133
+ return true;
134
+ throw new Error(`TraceHub answered ${res.status}`);
135
+ }
@@ -53,7 +53,7 @@ export async function uploadMarkers(client, target, stateDir, cwd, root, markers
53
53
  continue;
54
54
  }
55
55
  try {
56
- const sha = commitOfTrace(cwd, marker.traceId);
56
+ const sha = marker.commit ?? commitOfTrace(cwd, marker.traceId);
57
57
  const built = buildTrace({
58
58
  agent: ref.agent,
59
59
  id: ref.id,
@@ -2,40 +2,41 @@ export const UPLOAD_OWNER = "assistant-ui";
2
2
  export const UPLOAD_POLICY_ERROR = "trace uploads require one unambiguous GitHub repository owned by assistant-ui; nothing was uploaded";
3
3
  // Only the real GitHub host and conventional Git transports are accepted.
4
4
  // Do not use consent's permissive URL normalization as a security boundary.
5
- export function parseUploadRemote(remote) {
5
+ function parseGitHubRemote(remote) {
6
6
  const match = /^(?:git@github\.com:|https:\/\/github\.com\/|ssh:\/\/git@github\.com\/)([A-Za-z0-9-]+)\/([A-Za-z0-9_.-]+?)(?:\.git)?\/?$/i.exec(remote);
7
- if (!match || match[1].toLowerCase() !== UPLOAD_OWNER)
7
+ if (!match)
8
8
  return undefined;
9
9
  const repo = match[2].toLowerCase();
10
10
  if (repo === "." || repo === "..")
11
11
  return undefined;
12
- return { owner: UPLOAD_OWNER, repo };
12
+ return { owner: match[1].toLowerCase(), repo };
13
+ }
14
+ export function parseUploadRemote(remote) {
15
+ const target = parseGitHubRemote(remote);
16
+ return target?.owner === UPLOAD_OWNER ? target : undefined;
13
17
  }
14
18
  export function sameTarget(a, b) {
15
19
  return a.owner === b.owner && a.repo === b.repo;
16
20
  }
17
- // Every given URL (one remote's effective fetch/push URLs, a push
18
- // destination) must identify the same org-owned repository.
19
- export function targetOfUploadRemotes(remotes) {
20
- if (!remotes.length)
21
- return undefined;
22
- const targets = remotes.map(parseUploadRemote);
23
- const first = targets[0];
24
- return first && targets.every((target) => target && sameTarget(first, target))
21
+ // The one GitHub repository every given URL (one remote's effective
22
+ // fetch/push URLs, a push destination) names.
23
+ export function repositoryOfRemotes(remotes) {
24
+ const repos = remotes.map(parseGitHubRemote);
25
+ const first = repos[0];
26
+ return first && repos.every((repo) => repo && sameTarget(first, repo))
25
27
  ? first
26
28
  : undefined;
27
29
  }
28
- // No trace contents are sent to GitHub. Verify fresh on every upload run,
29
- // without following redirects after a transfer or rename. No positive disk cache.
30
- export async function verifyUploadOwnership(target, token, timeoutMs, fetchFn = fetch) {
31
- const parsed = parseUploadRemote(`https://github.com/${target.owner}/${target.repo}`);
32
- if (!parsed || !sameTarget(parsed, target))
33
- throw new Error(UPLOAD_POLICY_ERROR);
30
+ export function targetOfUploadRemotes(remotes) {
31
+ const repo = repositoryOfRemotes(remotes);
32
+ return repo?.owner === UPLOAD_OWNER ? repo : undefined;
33
+ }
34
+ async function githubRepository(repo, token, timeoutMs, fetchFn, check) {
34
35
  if (timeoutMs <= 0)
35
36
  throw new Error("upload budget exhausted; nothing was uploaded");
36
37
  let response;
37
38
  try {
38
- response = await fetchFn(`https://api.github.com/repos/${target.owner}/${target.repo}`, {
39
+ response = await fetchFn(`https://api.github.com/repos/${repo.owner}/${repo.repo}`, {
39
40
  headers: {
40
41
  accept: "application/vnd.github+json",
41
42
  authorization: `Bearer ${token}`,
@@ -47,14 +48,32 @@ export async function verifyUploadOwnership(target, token, timeoutMs, fetchFn =
47
48
  });
48
49
  }
49
50
  catch {
50
- throw new Error("GitHub could not verify repository ownership; nothing was uploaded");
51
+ throw new Error(`GitHub could not verify ${check}; nothing was uploaded`);
51
52
  }
52
53
  if (!response.ok) {
53
- throw new Error(`GitHub ownership check answered ${response.status}; nothing was uploaded`);
54
+ throw new Error(`GitHub answered ${response.status} when verifying ${check}; nothing was uploaded`);
54
55
  }
55
- const data = (await response.json());
56
+ return response.json();
57
+ }
58
+ // No trace contents are sent to GitHub. Verify fresh on every upload run,
59
+ // without following redirects after a transfer or rename. No positive disk cache.
60
+ export async function verifyUploadOwnership(target, token, timeoutMs, fetchFn = fetch) {
61
+ const parsed = parseUploadRemote(`https://github.com/${target.owner}/${target.repo}`);
62
+ if (!parsed || !sameTarget(parsed, target))
63
+ throw new Error(UPLOAD_POLICY_ERROR);
64
+ const data = (await githubRepository(target, token, timeoutMs, fetchFn, "repository ownership"));
56
65
  if (data?.owner?.type !== "Organization" ||
57
66
  data.owner.login?.toLowerCase() !== UPLOAD_OWNER ||
58
67
  data.full_name?.toLowerCase() !== `${target.owner}/${target.repo}`)
59
68
  throw new Error(UPLOAD_POLICY_ERROR);
60
69
  }
70
+ // A push to a fork of the target uploads too: a maintainer's commits on a
71
+ // contributor's pull request branch. GitHub must name the target as the
72
+ // fork's parent.
73
+ export async function verifyFork(fork, target, token, timeoutMs, fetchFn = fetch) {
74
+ const data = (await githubRepository(fork, token, timeoutMs, fetchFn, "the fork's parent"));
75
+ if (data?.fork !== true ||
76
+ data.full_name?.toLowerCase() !== `${fork.owner}/${fork.repo}` ||
77
+ data.parent?.full_name?.toLowerCase() !== `${target.owner}/${target.repo}`)
78
+ throw new Error(`${fork.owner}/${fork.repo} is not a fork of ${target.owner}/${target.repo}; nothing was uploaded`);
79
+ }
@@ -30,6 +30,9 @@ function parseMarker(line) {
30
30
  line: at,
31
31
  ts,
32
32
  ...(context.length ? { context } : {}),
33
+ ...(typeof m.commit === "string" && /^[0-9a-f]{40}$/.test(m.commit)
34
+ ? { commit: m.commit }
35
+ : {}),
33
36
  uploaded: m.uploaded === true,
34
37
  };
35
38
  }
@@ -91,3 +94,10 @@ export function recordMarkers(stateDir, traceId, sessions, now = new Date(), con
91
94
  }
92
95
  return missing;
93
96
  }
97
+ // Appends markers as they are, for a backfill that knows its commits.
98
+ export function appendMarkers(stateDir, markers) {
99
+ if (!markers.length)
100
+ return;
101
+ mkdirSync(stateDir, { recursive: true });
102
+ appendFileSync(markersFile(stateDir), serializeMarkers(markers));
103
+ }
@@ -1,6 +1,6 @@
1
1
  const TRAILER_KEY = "Assisted-by";
2
2
  // The dashboard, where a trailer links to its trace.
3
- const DASHBOARD_URL = "https://tracehub.aui.dev";
3
+ export const DASHBOARD_URL = "https://tracehub.aui.dev";
4
4
  const ID = /^[0-9a-f]{16}$/;
5
5
  // The trailer's value is the trace's URL, or the bare trace id.
6
6
  const LINE = /^[ \t]*Assisted-by:[ \t]*(?:\S*\/)?([0-9a-f]{16})[ \t]*$/gm;
@@ -0,0 +1,226 @@
1
+ import { isAbsolute, resolve } from "node:path";
2
+ import { isCommitCommand } from "./commitBoundary.js";
3
+ import { parseSessionLines } from "../trace/build.js";
4
+ import { formatSessionRef } from "../protocol/sessionRef.js";
5
+ // The directories a command changes to: `cd <dir>`, `pushd <dir>`, `-C <dir>`.
6
+ export function checkoutsNamed(command, home = "") {
7
+ const dirs = [];
8
+ for (const m of command.matchAll(/(?:^|[\s;&|(])(?:cd|pushd|-C)\s+("[^"]+"|'[^']+'|[^\s;&|)]+)/g)) {
9
+ let dir = m[1].replace(/^["']|["']$/g, "");
10
+ if (home && (dir === "~" || dir.startsWith("~/")))
11
+ dir = home + dir.slice(1);
12
+ if (isAbsolute(dir))
13
+ dirs.push(dir.replace(/\/+$/, ""));
14
+ }
15
+ return dirs;
16
+ }
17
+ // How far before a commit's author time a session may have started on it.
18
+ const LOOKBACK_MS = 7 * 24 * 3600_000;
19
+ // The commit's time is in whole seconds, its session's in milliseconds.
20
+ const SLACK_MS = 60_000;
21
+ export const HIGH = 7;
22
+ export const MEDIUM = 4;
23
+ // The session's events with the line each one came from; parsed one line at
24
+ // a time, as commitBoundary reads them.
25
+ export function candidateOf(ref, file, lines, opts = {}) {
26
+ const sidechain = opts.sidechain ?? false;
27
+ const events = [];
28
+ const branches = new Set();
29
+ let cwd = "";
30
+ lines.forEach((line, index) => {
31
+ if (!line.trim())
32
+ return;
33
+ const parsed = parseSessionLines(ref.agent, ref.id, [line], sidechain);
34
+ if (!cwd && parsed.cwd)
35
+ cwd = parsed.cwd;
36
+ for (const event of parsed.events)
37
+ events.push({ event, line: index });
38
+ const branch = /"gitBranch":"([^"]+)"/.exec(line)?.[1];
39
+ if (branch)
40
+ branches.add(branch);
41
+ });
42
+ return {
43
+ ref,
44
+ file,
45
+ lines,
46
+ sidechain,
47
+ parent: opts.parent,
48
+ cwd,
49
+ branches: [...branches],
50
+ events,
51
+ };
52
+ }
53
+ // When the commit was made: its committer time, unless that is a rebase or
54
+ // amend long after it was first written, whose author time stays.
55
+ export function commitTimeMs(commit) {
56
+ const late = commit.committerMs - commit.authorMs > 3600_000;
57
+ return late ? commit.authorMs : commit.committerMs;
58
+ }
59
+ // "[branch abc1234] subject": git commit's own output, naming the commit.
60
+ const COMMIT_OUTPUT = /\[[^\]\n]*?\s([0-9a-f]{7,40})\]/g;
61
+ function commitOutputNames(text, sha) {
62
+ for (const m of text.matchAll(COMMIT_OUTPUT))
63
+ if (sha.startsWith(m[1]))
64
+ return m[1];
65
+ return undefined;
66
+ }
67
+ // Lines that say something: not braces, blank or a word or two.
68
+ function telling(line) {
69
+ const t = line.trim();
70
+ return t.length >= 10 && /[A-Za-z]/.test(t);
71
+ }
72
+ // The commit's path an edit wrote, if any: an absolute path ends with it, a
73
+ // relative one does once resolved against the session's cwd.
74
+ function editedPath(path, cwd, files) {
75
+ const full = isAbsolute(path) ? path : resolve(cwd || "/", path);
76
+ return files.find((f) => full === f || full.endsWith(`/${f}`));
77
+ }
78
+ const TRUNK = new Set(["main", "master", "HEAD", "develop"]);
79
+ export function confidenceOf(score, tied) {
80
+ if (tied && score >= HIGH)
81
+ return "high";
82
+ if (score >= MEDIUM)
83
+ return "medium";
84
+ return "low";
85
+ }
86
+ // Scores one session against one commit. Undefined when the session cannot
87
+ // have made it: nothing of it was written in the week before the commit.
88
+ export function scoreMatch(commit, candidate, repo) {
89
+ const atMs = commitTimeMs(commit);
90
+ const ts = (e) => Date.parse(e.ts);
91
+ const before = candidate.events.filter(({ event }) => {
92
+ const t = ts(event);
93
+ return t <= atMs + SLACK_MS && t >= commit.authorMs - LOOKBACK_MS;
94
+ });
95
+ if (!before.length)
96
+ return undefined;
97
+ const evidence = [];
98
+ let score = 0;
99
+ let tied = false;
100
+ const short = commit.sha.slice(0, 7);
101
+ // The commit's own output, wherever it was made from.
102
+ let cut;
103
+ for (const { event, line } of candidate.events) {
104
+ if (event.kind !== "tool_result" && event.kind !== "assistant")
105
+ continue;
106
+ const named = commitOutputNames(event.text, commit.sha);
107
+ if (named) {
108
+ cut = line + 1;
109
+ break;
110
+ }
111
+ }
112
+ if (cut !== undefined) {
113
+ score += 6;
114
+ tied = true;
115
+ evidence.push(`git commit output names ${short}`);
116
+ }
117
+ const subject = commit.subject.trim();
118
+ const commitCalls = before.filter(({ event }) => event.kind === "tool_call" && isCommitCommand(event.arg));
119
+ const committed = commitCalls.some(({ event }) => subject.length >= 10 &&
120
+ event.arg.includes(subject.slice(0, 60)));
121
+ if (committed) {
122
+ score += 3;
123
+ evidence.push("a git commit call carries its subject");
124
+ }
125
+ // `cd <checkout> && git commit`, `git -C <checkout> commit`.
126
+ const committedHere = commitCalls.some(({ event }) => checkoutsNamed(event.arg, repo.home).some((dir) => repo.contains(dir)));
127
+ const edits = before.flatMap(({ event }) => event.kind === "tool_call" ? (event.edits ?? []) : []);
128
+ const inRepo = committedHere ||
129
+ (!!candidate.cwd && repo.contains(candidate.cwd)) ||
130
+ edits.some((e) => isAbsolute(e.path) && repo.contains(e.path));
131
+ if (inRepo) {
132
+ score += 2;
133
+ tied = true;
134
+ evidence.push("ran in a checkout of the repository");
135
+ }
136
+ // A push to the repository or the pull request's URL (`gh pr create`
137
+ // prints it), for a checkout removed since.
138
+ if (repo.slug) {
139
+ const slug = repo.slug.toLowerCase().replace(/[.]/g, "\\.");
140
+ const pullUrl = repo.pull
141
+ ? new RegExp(`github\\.com/${slug}/pull/${repo.pull}(?![0-9])`, "i")
142
+ : undefined;
143
+ const repoUrl = new RegExp(`github\\.com[/:]${slug}(?:\\.git)?(?![\\w.-])`, "i");
144
+ const texts = candidate.events.map(({ event }) => event.kind === "tool_call" ? event.arg : event.text);
145
+ if (pullUrl && texts.some((t) => pullUrl.test(t))) {
146
+ score += 3;
147
+ tied = true;
148
+ evidence.push(`names pull request #${repo.pull}`);
149
+ }
150
+ else if (!inRepo && texts.some((t) => repoUrl.test(t))) {
151
+ score += 1;
152
+ tied = true;
153
+ evidence.push(`names ${repo.slug} on GitHub`);
154
+ }
155
+ }
156
+ if (repo.branch &&
157
+ !TRUNK.has(repo.branch) &&
158
+ candidate.branches.includes(repo.branch)) {
159
+ score += 2;
160
+ evidence.push(`on branch ${repo.branch}`);
161
+ }
162
+ // The commit's files the session edited, and how many of the lines the
163
+ // commit adds those edits wrote.
164
+ const touched = new Set();
165
+ const written = new Set();
166
+ for (const edit of edits) {
167
+ const path = editedPath(edit.path, candidate.cwd, commit.files);
168
+ if (!path)
169
+ continue;
170
+ touched.add(path);
171
+ for (const line of edit.added)
172
+ written.add(line.trim());
173
+ }
174
+ if (touched.size) {
175
+ score += Math.min(3, touched.size);
176
+ evidence.push(`edited ${touched.size}/${commit.files.length} of its files`);
177
+ }
178
+ const wanted = new Set([...commit.added.values()]
179
+ .flat()
180
+ .filter(telling)
181
+ .map((l) => l.trim()));
182
+ let share = 0;
183
+ if (wanted.size && written.size) {
184
+ const hit = [...wanted].filter((l) => written.has(l)).length;
185
+ share = hit / wanted.size;
186
+ if (hit) {
187
+ score += share >= 0.5 ? 3 : share >= 0.2 ? 2 : 1;
188
+ evidence.push(`wrote ${Math.round(share * 100)}% of its added lines`);
189
+ }
190
+ }
191
+ if (!tied && touched.size && (share >= 0.5 || (committed && share > 0))) {
192
+ tied = true;
193
+ }
194
+ if (cut === undefined) {
195
+ const last = before.at(-1);
196
+ cut = last.line + 1;
197
+ }
198
+ return {
199
+ session: formatSessionRef(candidate.ref),
200
+ score,
201
+ confidence: confidenceOf(score, tied),
202
+ tied,
203
+ evidence,
204
+ cut,
205
+ };
206
+ }
207
+ // Whether a match is uploaded: a tied session of high confidence, or of
208
+ // medium when the caller named the session.
209
+ export function chosen(match, named = false) {
210
+ if (!match.tied)
211
+ return false;
212
+ return (match.confidence === "high" || (named && match.confidence === "medium"));
213
+ }
214
+ // The matches of a commit, best first, without a subagent whose session is
215
+ // chosen too (its run is in that session's trace).
216
+ export function rankMatches(commit, candidates, repo) {
217
+ const scored = candidates
218
+ .map((candidate) => ({
219
+ candidate,
220
+ match: scoreMatch(commit, candidate, repo),
221
+ }))
222
+ .filter((s) => s.match !== undefined && s.match.score > 0)
223
+ .sort((a, b) => b.match.score - a.match.score);
224
+ const parents = new Set(scored.filter((s) => chosen(s.match)).map((s) => s.match.session));
225
+ return scored.filter((s) => !s.candidate.parent || !parents.has(s.candidate.parent));
226
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "tracehub",
3
- "version": "0.5.3",
3
+ "version": "0.5.4",
4
4
  "description": "Attach coding-agent sessions (Claude Code, Codex, OpenCode) to git commits and pull requests, redacted.",
5
5
  "type": "module",
6
6
  "license": "MIT",