claude-attribution 1.8.0 → 1.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +67 -37
- package/package.json +2 -2
- package/src/__tests__/claude-projects.test.ts +42 -0
- package/src/__tests__/copilot-session.test.ts +141 -0
- package/src/__tests__/differ.test.ts +14 -2
- package/src/__tests__/git-notes.test.ts +28 -0
- package/src/__tests__/installed-repos.test.ts +2 -1
- package/src/__tests__/integration-helpers.ts +12 -1
- package/src/__tests__/integration.test.ts +140 -1
- package/src/__tests__/runtime.test.ts +12 -1
- package/src/__tests__/transcript.test.ts +61 -0
- package/src/attribution/commit.ts +68 -39
- package/src/attribution/differ.ts +76 -20
- package/src/attribution/git-notes.ts +26 -0
- package/src/attribution/runtime.ts +8 -7
- package/src/metrics/claude-projects.ts +48 -0
- package/src/metrics/collect.ts +113 -57
- package/src/metrics/copilot-session.ts +383 -0
- package/src/metrics/local-session.ts +12 -0
- package/src/metrics/session-metrics.ts +22 -0
- package/src/metrics/transcript.ts +56 -17
- package/src/setup/installed-repos.ts +4 -4
- package/src/setup/uninstall.ts +7 -2
package/README.md
CHANGED
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
# claude-attribution
|
|
2
2
|
|
|
3
|
-
AI code attribution for Claude Code. After every `git commit`, a one-line summary appears in your terminal:
|
|
3
|
+
AI code attribution for Claude Code and GitHub Copilot. After every `git commit`, a one-line summary appears in your terminal:
|
|
4
4
|
|
|
5
5
|
```
|
|
6
6
|
[claude-attribution] a3f1b2c — 142 AI / 38 human / 4 mixed lines (77% AI)
|
|
7
7
|
```
|
|
8
8
|
|
|
9
|
-
When a PR is opened, full metrics — model usage, token counts, tool calls, and attribution percentages — are injected automatically into the PR body. No copy-paste, no manual tracking.
|
|
9
|
+
When a PR is opened, full metrics — model usage, token counts when available, tool calls, and attribution percentages — are injected automatically into the PR body. No copy-paste, no manual tracking.
|
|
10
10
|
|
|
11
11
|
**Quick start:**
|
|
12
12
|
|
|
@@ -19,13 +19,13 @@ git commit -m "chore: install claude-attribution hooks"
|
|
|
19
19
|
git push
|
|
20
20
|
```
|
|
21
21
|
|
|
22
|
-
**Using Copilot or `@claude` (claude-code-action)?**
|
|
22
|
+
**Using Copilot or `@claude` (claude-code-action)?** Local Copilot CLI sessions are parsed from `~/.copilot/session-state/...`, and bot commits are auto-detected as 100% AI when no local session data exists. See [Copilot CLI Support](#copilot-cli-support) and [AI Actor Attribution](#ai-actor-attribution-copilot-bot-claude-gha).
|
|
23
23
|
|
|
24
24
|
**Requirements:** [Bun](https://bun.sh) (preferred) or Node 18+, and `gh` (GitHub CLI) authenticated for the `/pr` command.
|
|
25
25
|
|
|
26
26
|
---
|
|
27
27
|
|
|
28
|
-
Measures which lines in a commit were written by
|
|
28
|
+
Measures which lines in a commit were written by an AI assistant vs. a human — using checkpoint snapshots and line-level SHA-256 comparison when available, with explicit fallbacks for Copilot CLI sessions and hosted AI bot commits.
|
|
29
29
|
|
|
30
30
|
---
|
|
31
31
|
|
|
@@ -188,7 +188,7 @@ If you reinstall `claude-attribution` globally (e.g. after upgrading), you can n
|
|
|
188
188
|
claude-attribution update
|
|
189
189
|
```
|
|
190
190
|
|
|
191
|
-
The installer records each repo in a per-user registry at `~/.claude/claude-attribution/installed-repos.json`. `update` re-runs the installer for every still-valid tracked repo, skips repos already on the current CLI version, and prunes paths that no longer exist or are no longer git repos.
|
|
191
|
+
The installer records each repo installed with `claude-attribution` v1.8.0 or later in a per-user registry at `~/.claude/claude-attribution/installed-repos.json`. `update` re-runs the installer for every still-valid tracked repo in that registry, skips repos already on the current CLI version, and prunes paths that no longer exist or are no longer git repos. Repos installed before v1.8.0 will not appear there until you run `claude-attribution install` for them once on the newer CLI.
|
|
192
192
|
|
|
193
193
|
To force a reinstall even when the recorded version already matches, use:
|
|
194
194
|
|
|
@@ -242,33 +242,41 @@ Metrics are injected automatically — no command needed.
|
|
|
242
242
|
|
|
243
243
|
**When Claude creates the PR** (asks Claude to open a PR, uses `/pr`): the `post-bash` hook fires immediately after `gh pr create` succeeds, injects full local metrics (token usage, tool counts, attribution) into the PR body before Claude continues.
|
|
244
244
|
|
|
245
|
-
**When you create the PR yourself** (`gh pr create`, GitHub UI, Copilot): the `claude-attribution-pr.yml` GitHub Actions workflow fires on `opened` and
|
|
245
|
+
**When you create the PR yourself** (`gh pr create`, GitHub UI, Copilot): the `claude-attribution-pr.yml` GitHub Actions workflow fires on `opened` and rebuilds metrics from durable git notes when available. If only CI-visible data exists, it falls back to attribution-only metrics because local session logs are not available in CI.
|
|
246
246
|
|
|
247
247
|
**On every new push to an open PR**: the workflow fires on `synchronize` and updates the attribution percentages to reflect new commits.
|
|
248
248
|
|
|
249
249
|
The metrics block injected into the PR body looks like (when the cumulative minimap exists):
|
|
250
250
|
|
|
251
|
-
> ##
|
|
251
|
+
> ## AI Coding Metrics
|
|
252
252
|
>
|
|
253
253
|
> **Codebase: ~77% AI** (3200 / 4150 lines)
|
|
254
|
-
> **This PR:** 184 lines changed (4% of codebase) · 77%
|
|
254
|
+
> **This PR:** 184 lines changed (4% of codebase) · 77% AI edits · 142 AI lines
|
|
255
|
+
> **Session:** 12 prompts · 24m total (18m AI · 6m human)
|
|
256
|
+
> **Assistant runtime:** Claude Code (claude-sonnet-4-6)
|
|
255
257
|
>
|
|
256
258
|
> | Model | Calls | Input | Output | Cache |
|
|
257
259
|
> |-------|-------|-------|--------|-------|
|
|
258
260
|
> | Sonnet | 45 | 120K | 35K | 10K |
|
|
259
261
|
> | **Total** | 45 | 120K | 35K | 10K |
|
|
260
262
|
>
|
|
261
|
-
> **
|
|
263
|
+
> **Estimated cost:** ~$1.23
|
|
262
264
|
>
|
|
263
265
|
> <details><summary>Tools · Agents · Files</summary>
|
|
264
266
|
>
|
|
265
|
-
> **
|
|
267
|
+
> **External tools:** WebSearch ×2
|
|
266
268
|
>
|
|
267
269
|
> </details>
|
|
268
270
|
|
|
269
271
|
Before running `init --ai` (or on a fresh install with no minimap), the headline falls back to the session-only view:
|
|
270
272
|
|
|
271
|
-
> **AI contribution: ~77%** (142 of 184 committed lines)
|
|
273
|
+
> **AI contribution: ~77%** (142 of 184 committed lines)
|
|
274
|
+
|
|
275
|
+
For **Copilot CLI** sessions, the same block is rendered with provider-aware differences:
|
|
276
|
+
|
|
277
|
+
- assistant runtime shows as `GitHub Copilot CLI`
|
|
278
|
+
- model usage shows **Known Tokens** instead of Claude-style input/output/cache columns
|
|
279
|
+
- cost is shown as **unavailable** unless durable local billing data exists
|
|
272
280
|
|
|
273
281
|
The block is wrapped in HTML comments for idempotent updates — re-running replaces the existing block rather than appending:
|
|
274
282
|
|
|
@@ -301,31 +309,19 @@ claude-attribution metrics
|
|
|
301
309
|
The output is markdown you paste into your PR description:
|
|
302
310
|
|
|
303
311
|
```markdown
|
|
304
|
-
##
|
|
305
|
-
|
|
306
|
-
**Session ID:** `abc-123-...`
|
|
307
|
-
|
|
308
|
-
### Tools Used
|
|
309
|
-
- **Edit:** 47 calls
|
|
310
|
-
- **Read:** 31 calls
|
|
311
|
-
...
|
|
312
|
+
## AI Coding Metrics
|
|
312
313
|
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
314
|
+
**Codebase: ~77% AI** (3200 / 4150 lines)
|
|
315
|
+
**This PR:** 184 lines changed (4% of codebase) · 77% AI edits · 142 AI lines
|
|
316
|
+
**Session:** 12 prompts · 24m total (18m AI · 6m human)
|
|
317
|
+
**Assistant runtime:** GitHub Copilot CLI (v1.0.15 · gpt-5.4)
|
|
316
318
|
|
|
317
|
-
|
|
319
|
+
| Model | Calls | Known Tokens |
|
|
320
|
+
|-------|-------|--------------|
|
|
321
|
+
| gpt-5.4 | 12 | 48K |
|
|
322
|
+
| **Total** | 12 | 48K |
|
|
318
323
|
|
|
319
|
-
|
|
320
|
-
- **AI-authored lines:** 142
|
|
321
|
-
- **Human-authored lines:** 38
|
|
322
|
-
- **Mixed lines (AI wrote, human modified):** 4
|
|
323
|
-
- **Total committed lines:** 184
|
|
324
|
-
- **AI contribution:** ~77%
|
|
325
|
-
|
|
326
|
-
#### Files Modified by Claude
|
|
327
|
-
- `src/components/Foo.tsx` — 89% AI (82 lines)
|
|
328
|
-
- `src/hooks/useBar.ts` — 61% AI (44 lines)
|
|
324
|
+
**Estimated cost:** unavailable — Copilot session data does not expose enough local billing data to estimate spend reliably.
|
|
329
325
|
```
|
|
330
326
|
|
|
331
327
|
### Running with a specific session ID
|
|
@@ -336,7 +332,35 @@ If you have multiple sessions and want metrics for a specific one:
|
|
|
336
332
|
claude-attribution metrics <session-id>
|
|
337
333
|
```
|
|
338
334
|
|
|
339
|
-
|
|
335
|
+
Claude session IDs are shown in `.claude/logs/tool-usage.jsonl`. Copilot CLI session IDs are the directory names under `~/.copilot/session-state/`.
|
|
336
|
+
|
|
337
|
+
### Copilot CLI support
|
|
338
|
+
|
|
339
|
+
`claude-attribution` now supports **local GitHub Copilot CLI sessions** without adding a separate Copilot-specific install flow.
|
|
340
|
+
|
|
341
|
+
How it works:
|
|
342
|
+
|
|
343
|
+
1. The existing repo install still provides the post-commit hook and PR workflow.
|
|
344
|
+
2. On commit or `/metrics`, the tool first tries the normal Claude-local data sources.
|
|
345
|
+
3. If Claude session data is unavailable, it looks for a matching Copilot CLI session under `~/.copilot/session-state/<session-id>/events.jsonl`.
|
|
346
|
+
4. It matches sessions by repo path, branch, and recency, then extracts:
|
|
347
|
+
- prompt count
|
|
348
|
+
- AI vs human active time
|
|
349
|
+
- tool usage
|
|
350
|
+
- skill usage
|
|
351
|
+
- agent usage
|
|
352
|
+
- dominant model / known token counts
|
|
353
|
+
5. That normalized session data is attached to the git note so PR metrics can still be rebuilt later from durable note metadata.
|
|
354
|
+
|
|
355
|
+
Optional prompt markers:
|
|
356
|
+
|
|
357
|
+
- If your Copilot CLI workflow includes prompts beginning with `start work` and `create pr`, those are used as a tighter timing window.
|
|
358
|
+
- If not, the tool falls back to the broader repo/branch session window automatically.
|
|
359
|
+
|
|
360
|
+
Limitations:
|
|
361
|
+
|
|
362
|
+
- Copilot CLI local session-state currently does **not** expose enough durable billing data to estimate cost reliably, so cost is rendered as **unavailable**.
|
|
363
|
+
- Copilot CLI does not provide Claude-style file checkpoints, so attribution may fall back to commit/diff heuristics when no checkpoints exist.
|
|
340
364
|
|
|
341
365
|
### Checking raw attribution data
|
|
342
366
|
|
|
@@ -528,7 +552,7 @@ When no destination is configured, the workflow runs in **dry-run mode** — it
|
|
|
528
552
|
|
|
529
553
|
Set secrets at the org level so they apply to all repos that use this tool. Multiple export destinations can be active simultaneously (e.g. both `OTEL_EXPORTER_OTLP_ENDPOINT` and `METRICS_WEBHOOK_URL`).
|
|
530
554
|
|
|
531
|
-
**Model pricing (org variables — update when Anthropic changes pricing):**
|
|
555
|
+
**Model pricing (Claude-only org variables — update when Anthropic changes pricing):**
|
|
532
556
|
|
|
533
557
|
| Variable | Default | Description |
|
|
534
558
|
|----------|---------|-------------|
|
|
@@ -541,7 +565,7 @@ Set secrets at the org level so they apply to all repos that use this tool. Mult
|
|
|
541
565
|
| `CLAUDE_PRICE_CACHE_READ_MULT` | `0.1` | Fraction of input price for cache reads |
|
|
542
566
|
| `CLAUDE_PRICE_CACHE_WRITE_MULT` | `1.25` | Fraction of input price for cache writes |
|
|
543
567
|
|
|
544
|
-
|
|
568
|
+
These pricing variables apply to **Claude token-based cost estimation only**. Copilot metrics currently report cost as **unavailable** unless a future durable billing/token source is added. Unrecognized Claude model names fall back to Opus pricing. Set these as org-level **variables** (not secrets) — they're not sensitive.
|
|
545
569
|
|
|
546
570
|
|
|
547
571
|
---
|
|
@@ -562,7 +586,13 @@ When generating PR metrics, branch commits that have no git note are checked for
|
|
|
562
586
|
| `Co-authored-by:` trailer | `Co-authored-by: Claude <...>` |
|
|
563
587
|
| `Co-authored-by:` trailer | `Co-authored-by: GitHub Copilot <...>` |
|
|
564
588
|
|
|
565
|
-
If detected, all non-blank committed lines are counted as AI in the metrics output. No git note is written — this is a metrics-time synthesis only.
|
|
589
|
+
If detected, all non-blank committed lines are counted as AI in the metrics output and the assistant runtime is labeled as Copilot or Claude when that can be inferred. No git note is written — this is a metrics-time synthesis only.
|
|
590
|
+
|
|
591
|
+
This hosted/bot path is intentionally conservative:
|
|
592
|
+
|
|
593
|
+
- attribution: **heuristic but durable enough for PR metrics**
|
|
594
|
+
- assistant runtime: **best effort**
|
|
595
|
+
- time/cost: **not available** from current hosted signals
|
|
566
596
|
|
|
567
597
|
**Not detectable:** Copilot "Commit suggestion" (the button on PR review comments). Those commits are authored as the human who clicked the button — there is no metadata distinguishing them from a human edit. They will count as HUMAN.
|
|
568
598
|
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "claude-attribution",
|
|
3
|
-
"version": "1.
|
|
4
|
-
"description": "AI code attribution tracking for Claude Code
|
|
3
|
+
"version": "1.9.0",
|
|
4
|
+
"description": "AI code attribution tracking for Claude Code and GitHub Copilot sessions",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
7
7
|
"claude-attribution": "bin/claude-attribution"
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
import { describe, expect, test } from "bun:test";
|
|
2
|
+
import { mkdir } from "fs/promises";
|
|
3
|
+
import { join } from "path";
|
|
4
|
+
import { createTempContext } from "./integration-helpers.ts";
|
|
5
|
+
import {
|
|
6
|
+
claudeProjectKey,
|
|
7
|
+
resolveClaudeProjectDir,
|
|
8
|
+
} from "../metrics/claude-projects.ts";
|
|
9
|
+
|
|
10
|
+
describe("claude project directory resolution", () => {
|
|
11
|
+
test("sanitizes dotted repo paths the same way Claude transcripts do", () => {
|
|
12
|
+
expect(claudeProjectKey("/Users/alice.smith/Code/my-repo")).toBe(
|
|
13
|
+
"-Users-alice-smith-Code-my-repo",
|
|
14
|
+
);
|
|
15
|
+
});
|
|
16
|
+
|
|
17
|
+
test("resolves sanitized transcript directories", async () => {
|
|
18
|
+
const ctx = await createTempContext("claude-projects");
|
|
19
|
+
try {
|
|
20
|
+
const repo = "/Users/alice.smith/Code/my-repo";
|
|
21
|
+
const projectsRoot = join(ctx.home, ".claude", "projects");
|
|
22
|
+
const dir = join(projectsRoot, "-Users-alice-smith-Code-my-repo");
|
|
23
|
+
await mkdir(dir, { recursive: true });
|
|
24
|
+
expect(await resolveClaudeProjectDir(repo, projectsRoot)).toBe(dir);
|
|
25
|
+
} finally {
|
|
26
|
+
await ctx.cleanup();
|
|
27
|
+
}
|
|
28
|
+
});
|
|
29
|
+
|
|
30
|
+
test("falls back to legacy slash-only keys for backward compatibility", async () => {
|
|
31
|
+
const ctx = await createTempContext("claude-projects-legacy");
|
|
32
|
+
try {
|
|
33
|
+
const repo = "/Users/alice/Code/my-repo";
|
|
34
|
+
const projectsRoot = join(ctx.home, ".claude", "projects");
|
|
35
|
+
const dir = join(projectsRoot, "-Users-alice-Code-my-repo");
|
|
36
|
+
await mkdir(dir, { recursive: true });
|
|
37
|
+
expect(await resolveClaudeProjectDir(repo, projectsRoot)).toBe(dir);
|
|
38
|
+
} finally {
|
|
39
|
+
await ctx.cleanup();
|
|
40
|
+
}
|
|
41
|
+
});
|
|
42
|
+
});
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
import { describe, expect, test } from "bun:test";
|
|
2
|
+
import { parseCopilotSession, resolveCopilotSessionId } from "../metrics/copilot-session.ts";
|
|
3
|
+
import { createTempContext, writeCopilotSession } from "./integration-helpers.ts";
|
|
4
|
+
|
|
5
|
+
describe("copilot-session", () => {
|
|
6
|
+
test("parses local Copilot session metrics", async () => {
|
|
7
|
+
const ctx = await createTempContext("copilot-session");
|
|
8
|
+
const originalHome = process.env.HOME;
|
|
9
|
+
try {
|
|
10
|
+
process.env.HOME = ctx.home;
|
|
11
|
+
const sessionId = "copilot-session-1";
|
|
12
|
+
await writeCopilotSession(ctx.home, sessionId, [
|
|
13
|
+
{
|
|
14
|
+
type: "session.start",
|
|
15
|
+
timestamp: "2026-04-01T10:00:00.000Z",
|
|
16
|
+
data: {
|
|
17
|
+
context: {
|
|
18
|
+
cwd: ctx.repo,
|
|
19
|
+
gitRoot: ctx.repo,
|
|
20
|
+
branch: "feature/copilot",
|
|
21
|
+
},
|
|
22
|
+
},
|
|
23
|
+
},
|
|
24
|
+
{
|
|
25
|
+
type: "user.message",
|
|
26
|
+
timestamp: "2026-04-01T10:00:00.000Z",
|
|
27
|
+
data: { content: "Investigate parser" },
|
|
28
|
+
},
|
|
29
|
+
{
|
|
30
|
+
type: "tool.execution_start",
|
|
31
|
+
timestamp: "2026-04-01T10:00:05.000Z",
|
|
32
|
+
data: { toolName: "web_search" },
|
|
33
|
+
},
|
|
34
|
+
{
|
|
35
|
+
type: "tool.execution_start",
|
|
36
|
+
timestamp: "2026-04-01T10:00:06.000Z",
|
|
37
|
+
data: { toolName: "skill", arguments: { skill: "pr" } },
|
|
38
|
+
},
|
|
39
|
+
{
|
|
40
|
+
type: "tool.execution_complete",
|
|
41
|
+
timestamp: "2026-04-01T10:01:00.000Z",
|
|
42
|
+
data: { model: "gpt-5.4" },
|
|
43
|
+
},
|
|
44
|
+
{
|
|
45
|
+
type: "assistant.message",
|
|
46
|
+
timestamp: "2026-04-01T10:01:00.000Z",
|
|
47
|
+
data: { outputTokens: 320 },
|
|
48
|
+
},
|
|
49
|
+
{
|
|
50
|
+
type: "assistant.turn_end",
|
|
51
|
+
timestamp: "2026-04-01T10:01:00.000Z",
|
|
52
|
+
},
|
|
53
|
+
{
|
|
54
|
+
type: "subagent.started",
|
|
55
|
+
timestamp: "2026-04-01T10:01:10.000Z",
|
|
56
|
+
data: { agentName: "ux-reviewer" },
|
|
57
|
+
},
|
|
58
|
+
{
|
|
59
|
+
type: "subagent.completed",
|
|
60
|
+
timestamp: "2026-04-01T10:01:30.000Z",
|
|
61
|
+
data: { model: "gpt-5-mini", totalTokens: 80 },
|
|
62
|
+
},
|
|
63
|
+
{
|
|
64
|
+
type: "user.message",
|
|
65
|
+
timestamp: "2026-04-01T10:02:00.000Z",
|
|
66
|
+
data: { content: "Create the PR" },
|
|
67
|
+
},
|
|
68
|
+
{
|
|
69
|
+
type: "assistant.turn_end",
|
|
70
|
+
timestamp: "2026-04-01T10:03:00.000Z",
|
|
71
|
+
},
|
|
72
|
+
]);
|
|
73
|
+
|
|
74
|
+
const parsed = await parseCopilotSession(sessionId, ctx.repo);
|
|
75
|
+
|
|
76
|
+
expect(parsed).not.toBeNull();
|
|
77
|
+
expect(parsed?.provider).toBe("copilot");
|
|
78
|
+
expect(parsed?.humanPromptCount).toBe(2);
|
|
79
|
+
expect(parsed?.toolCounts).toEqual({ web_search: 1, skill: 1 });
|
|
80
|
+
expect(parsed?.skillNames).toEqual(["pr"]);
|
|
81
|
+
expect(parsed?.agentCounts).toEqual({ "ux-reviewer": 1 });
|
|
82
|
+
expect(parsed?.byModel).toEqual([
|
|
83
|
+
{
|
|
84
|
+
modelShort: "Unknown",
|
|
85
|
+
modelFull: "gpt-5-mini",
|
|
86
|
+
calls: 1,
|
|
87
|
+
inputTokens: 0,
|
|
88
|
+
outputTokens: 80,
|
|
89
|
+
cacheCreationTokens: 0,
|
|
90
|
+
cacheReadTokens: 0,
|
|
91
|
+
},
|
|
92
|
+
{
|
|
93
|
+
modelShort: "Unknown",
|
|
94
|
+
modelFull: "gpt-5.4",
|
|
95
|
+
calls: 1,
|
|
96
|
+
inputTokens: 0,
|
|
97
|
+
outputTokens: 320,
|
|
98
|
+
cacheCreationTokens: 0,
|
|
99
|
+
cacheReadTokens: 0,
|
|
100
|
+
},
|
|
101
|
+
]);
|
|
102
|
+
expect(parsed?.activeMinutes).toBe(3);
|
|
103
|
+
expect(parsed?.aiMinutes).toBe(2);
|
|
104
|
+
expect(parsed?.humanMinutes).toBe(1);
|
|
105
|
+
expect(parsed?.costMode).toBe("unavailable");
|
|
106
|
+
} finally {
|
|
107
|
+
process.env.HOME = originalHome;
|
|
108
|
+
await ctx.cleanup();
|
|
109
|
+
}
|
|
110
|
+
});
|
|
111
|
+
|
|
112
|
+
test("resolves the best matching session for a repo and branch", async () => {
|
|
113
|
+
const ctx = await createTempContext("copilot-session-resolve");
|
|
114
|
+
const originalHome = process.env.HOME;
|
|
115
|
+
try {
|
|
116
|
+
process.env.HOME = ctx.home;
|
|
117
|
+
await writeCopilotSession(ctx.home, "older-session", [
|
|
118
|
+
{
|
|
119
|
+
type: "session.start",
|
|
120
|
+
timestamp: "2026-04-01T09:00:00.000Z",
|
|
121
|
+
data: { context: { gitRoot: ctx.repo, branch: "main" } },
|
|
122
|
+
},
|
|
123
|
+
]);
|
|
124
|
+
await writeCopilotSession(ctx.home, "best-session", [
|
|
125
|
+
{
|
|
126
|
+
type: "session.start",
|
|
127
|
+
timestamp: "2026-04-01T10:00:00.000Z",
|
|
128
|
+
data: {
|
|
129
|
+
context: { gitRoot: ctx.repo, cwd: ctx.repo, branch: "feature/copilot" },
|
|
130
|
+
},
|
|
131
|
+
},
|
|
132
|
+
]);
|
|
133
|
+
|
|
134
|
+
const resolved = await resolveCopilotSessionId(ctx.repo, "feature/copilot");
|
|
135
|
+
expect(resolved).toBe("best-session");
|
|
136
|
+
} finally {
|
|
137
|
+
process.env.HOME = originalHome;
|
|
138
|
+
await ctx.cleanup();
|
|
139
|
+
}
|
|
140
|
+
});
|
|
141
|
+
});
|
|
@@ -75,8 +75,8 @@ describe("attributeLines — basic classification", () => {
|
|
|
75
75
|
});
|
|
76
76
|
|
|
77
77
|
test("line not in after, committed file longer than after-snapshot → HUMAN", () => {
|
|
78
|
-
//
|
|
79
|
-
//
|
|
78
|
+
// Insertions are treated as HUMAN unless we can anchor them as one-for-one
|
|
79
|
+
// replacements of Claude-authored lines.
|
|
80
80
|
const before: string[] = [];
|
|
81
81
|
const after = ["const x = 1;"]; // only 1 line
|
|
82
82
|
// Human committed 2 lines — position 1 is beyond after-snapshot length
|
|
@@ -86,6 +86,18 @@ describe("attributeLines — basic classification", () => {
|
|
|
86
86
|
expect(attribution[1]).toBe("HUMAN"); // position 1 is beyond after-snapshot — no MIXED
|
|
87
87
|
});
|
|
88
88
|
|
|
89
|
+
test("human insertion above AI lines does not create false MIXED classifications", () => {
|
|
90
|
+
const before: string[] = [];
|
|
91
|
+
const after = ["const a = 1;", "const b = 2;"];
|
|
92
|
+
const committed = ["// intro", "const a = 1;", "const b = 2;"];
|
|
93
|
+
const { attribution, stats } = attributeLines(before, after, committed);
|
|
94
|
+
expect(attribution).toEqual(["HUMAN", "AI", "AI"]);
|
|
95
|
+
expect(stats.ai).toBe(2);
|
|
96
|
+
expect(stats.human).toBe(1);
|
|
97
|
+
expect(stats.mixed).toBe(0);
|
|
98
|
+
expect(stats.pctAi).toBe(67);
|
|
99
|
+
});
|
|
100
|
+
|
|
89
101
|
test("empty lines always → HUMAN regardless of snapshot", () => {
|
|
90
102
|
const before: string[] = [];
|
|
91
103
|
const after = ["", "const x = 1;", ""];
|
|
@@ -137,4 +137,32 @@ timestamp: new Date().toISOString(),
|
|
|
137
137
|
}),
|
|
138
138
|
).toBe(false);
|
|
139
139
|
});
|
|
140
|
+
|
|
141
|
+
test("buildAllAiResult tags hosted Copilot commits with assistant runtime", async () => {
|
|
142
|
+
const ctx = await createTempContext("claude-attribution-copilot-bot");
|
|
143
|
+
try {
|
|
144
|
+
await initGitRepo(ctx.repo);
|
|
145
|
+
await runCommand("git", ["config", "user.name", "copilot[bot]"], { cwd: ctx.repo });
|
|
146
|
+
await runCommand(
|
|
147
|
+
"git",
|
|
148
|
+
["config", "user.email", "000000+copilot[bot]@users.noreply.github.com"],
|
|
149
|
+
{ cwd: ctx.repo },
|
|
150
|
+
);
|
|
151
|
+
await writeFile(join(ctx.repo, "agent.ts"), "export const generated = true;\n");
|
|
152
|
+
await commitAll(
|
|
153
|
+
ctx.repo,
|
|
154
|
+
"feat: add hosted change\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>",
|
|
155
|
+
);
|
|
156
|
+
const sha = await currentSha(ctx.repo);
|
|
157
|
+
|
|
158
|
+
const result = await buildAllAiResult(ctx.repo, sha);
|
|
159
|
+
|
|
160
|
+
expect(result.assistantRuntime).toEqual({
|
|
161
|
+
vendor: "copilot",
|
|
162
|
+
client: "GitHub Copilot",
|
|
163
|
+
});
|
|
164
|
+
} finally {
|
|
165
|
+
await ctx.cleanup();
|
|
166
|
+
}
|
|
167
|
+
});
|
|
140
168
|
});
|
|
@@ -34,8 +34,9 @@ describe("installed repo registry", () => {
|
|
|
34
34
|
"1.7.0",
|
|
35
35
|
);
|
|
36
36
|
|
|
37
|
-
await unregisterInstalledRepo(repo2, ctx.home);
|
|
37
|
+
expect(await unregisterInstalledRepo(repo2, ctx.home)).toBe(true);
|
|
38
38
|
expect(await readInstalledRepoRegistry(ctx.home)).toHaveLength(1);
|
|
39
|
+
expect(await unregisterInstalledRepo(repo2, ctx.home)).toBe(false);
|
|
39
40
|
|
|
40
41
|
await writeInstalledRepoRegistry(
|
|
41
42
|
[
|
|
@@ -10,6 +10,7 @@ import {
|
|
|
10
10
|
import { tmpdir } from "os";
|
|
11
11
|
import { dirname, join, resolve } from "path";
|
|
12
12
|
import { promisify } from "util";
|
|
13
|
+
import { claudeProjectKey } from "../metrics/claude-projects.ts";
|
|
13
14
|
|
|
14
15
|
const execFileAsync = promisify(execFile);
|
|
15
16
|
|
|
@@ -113,7 +114,7 @@ export async function writeJsonl(
|
|
|
113
114
|
}
|
|
114
115
|
|
|
115
116
|
export function transcriptDirFor(home: string, repo: string): string {
|
|
116
|
-
return join(home, ".claude", "projects", repo
|
|
117
|
+
return join(home, ".claude", "projects", claudeProjectKey(repo));
|
|
117
118
|
}
|
|
118
119
|
|
|
119
120
|
export async function writeTranscript(
|
|
@@ -127,6 +128,16 @@ export async function writeTranscript(
|
|
|
127
128
|
await writeJsonl(join(dir, `${sessionId}.jsonl`), entries);
|
|
128
129
|
}
|
|
129
130
|
|
|
131
|
+
export async function writeCopilotSession(
|
|
132
|
+
home: string,
|
|
133
|
+
sessionId: string,
|
|
134
|
+
entries: unknown[],
|
|
135
|
+
): Promise<void> {
|
|
136
|
+
const dir = join(home, ".copilot", "session-state", sessionId);
|
|
137
|
+
await mkdir(dir, { recursive: true });
|
|
138
|
+
await writeJsonl(join(dir, "events.jsonl"), entries);
|
|
139
|
+
}
|
|
140
|
+
|
|
130
141
|
export async function readJson(path: string): Promise<unknown> {
|
|
131
142
|
return JSON.parse(await readFile(path, "utf8"));
|
|
132
143
|
}
|
|
@@ -17,6 +17,7 @@ import {
|
|
|
17
17
|
initGitRepo,
|
|
18
18
|
readJson,
|
|
19
19
|
runCommand,
|
|
20
|
+
writeCopilotSession,
|
|
20
21
|
writeJsonl,
|
|
21
22
|
writeTranscript,
|
|
22
23
|
} from "./integration-helpers.ts";
|
|
@@ -207,7 +208,145 @@ describe("integration", () => {
|
|
|
207
208
|
expect(metrics.stdout).toContain("**Skills:** /pr");
|
|
208
209
|
expect(metrics.stdout).toContain("**Agents:** code-review");
|
|
209
210
|
expect(metrics.stdout).toContain("**External tools:** WebSearch ×2");
|
|
210
|
-
expect(metrics.stdout).toContain("**Estimated cost:**");
|
|
211
|
+
expect(metrics.stdout).toContain("**Estimated cost:** unavailable");
|
|
212
|
+
} finally {
|
|
213
|
+
await ctx.cleanup();
|
|
214
|
+
}
|
|
215
|
+
});
|
|
216
|
+
|
|
217
|
+
test("post-commit falls back to Copilot session-state metadata", async () => {
|
|
218
|
+
const ctx = await createTempContext("copilot-attribution-commit");
|
|
219
|
+
try {
|
|
220
|
+
await initGitRepo(ctx.repo);
|
|
221
|
+
const filePath = join(ctx.repo, "src.ts");
|
|
222
|
+
await writeFile(filePath, "const value = 1;\n");
|
|
223
|
+
await commitAll(ctx.repo, "initial");
|
|
224
|
+
|
|
225
|
+
await writeFile(filePath, "const value = 1;\nconst nextValue = 2;\n");
|
|
226
|
+
await commitAll(ctx.repo, "feature work");
|
|
227
|
+
|
|
228
|
+
const branch = (
|
|
229
|
+
await runCommand("git", ["rev-parse", "--abbrev-ref", "HEAD"], {
|
|
230
|
+
cwd: ctx.repo,
|
|
231
|
+
})
|
|
232
|
+
).stdout.trim();
|
|
233
|
+
const sessionId = "copilot-session-commit-1";
|
|
234
|
+
await writeCopilotSession(ctx.home, sessionId, [
|
|
235
|
+
{
|
|
236
|
+
type: "session.start",
|
|
237
|
+
timestamp: "2026-04-01T10:00:00.000Z",
|
|
238
|
+
data: {
|
|
239
|
+
context: {
|
|
240
|
+
cwd: ctx.repo,
|
|
241
|
+
gitRoot: ctx.repo,
|
|
242
|
+
branch,
|
|
243
|
+
},
|
|
244
|
+
},
|
|
245
|
+
},
|
|
246
|
+
{
|
|
247
|
+
type: "user.message",
|
|
248
|
+
timestamp: "2026-04-01T10:00:00.000Z",
|
|
249
|
+
data: { content: "start work" },
|
|
250
|
+
},
|
|
251
|
+
{
|
|
252
|
+
type: "tool.execution_start",
|
|
253
|
+
timestamp: "2026-04-01T10:00:05.000Z",
|
|
254
|
+
data: { toolName: "web_search" },
|
|
255
|
+
},
|
|
256
|
+
{
|
|
257
|
+
type: "tool.execution_complete",
|
|
258
|
+
timestamp: "2026-04-01T10:01:00.000Z",
|
|
259
|
+
data: { model: "gpt-5.4" },
|
|
260
|
+
},
|
|
261
|
+
{
|
|
262
|
+
type: "assistant.message",
|
|
263
|
+
timestamp: "2026-04-01T10:01:00.000Z",
|
|
264
|
+
data: { outputTokens: 240 },
|
|
265
|
+
},
|
|
266
|
+
{
|
|
267
|
+
type: "assistant.turn_end",
|
|
268
|
+
timestamp: "2026-04-01T10:01:00.000Z",
|
|
269
|
+
},
|
|
270
|
+
]);
|
|
271
|
+
|
|
272
|
+
await runCommand(CLI_BIN, ["hook", "post-commit"], {
|
|
273
|
+
cwd: ctx.repo,
|
|
274
|
+
env: {
|
|
275
|
+
HOME: ctx.home,
|
|
276
|
+
COPILOT_CLI: "1",
|
|
277
|
+
COPILOT_CLI_BINARY_VERSION: "1.0.15",
|
|
278
|
+
},
|
|
279
|
+
});
|
|
280
|
+
|
|
281
|
+
const note = JSON.parse(
|
|
282
|
+
(
|
|
283
|
+
await runCommand(
|
|
284
|
+
"git",
|
|
285
|
+
["notes", "--ref", "claude-attribution", "show", "HEAD"],
|
|
286
|
+
{ cwd: ctx.repo },
|
|
287
|
+
)
|
|
288
|
+
).stdout,
|
|
289
|
+
) as AttributionResult;
|
|
290
|
+
|
|
291
|
+
expect(note.session).toBe(sessionId);
|
|
292
|
+
expect(note.totals.ai).toBeGreaterThan(0);
|
|
293
|
+
expect(note.modelUsage).toEqual([
|
|
294
|
+
{
|
|
295
|
+
modelFull: "gpt-5.4",
|
|
296
|
+
modelShort: "Unknown",
|
|
297
|
+
calls: 1,
|
|
298
|
+
inputTokens: 0,
|
|
299
|
+
outputTokens: 240,
|
|
300
|
+
cacheCreationTokens: 0,
|
|
301
|
+
cacheReadTokens: 0,
|
|
302
|
+
},
|
|
303
|
+
]);
|
|
304
|
+
expect(note.sessionMetrics?.toolCounts).toEqual({ web_search: 1 });
|
|
305
|
+
expect(note.sessionMetrics?.humanPromptCount).toBe(1);
|
|
306
|
+
expect(note.assistantRuntime).toEqual({
|
|
307
|
+
vendor: "copilot",
|
|
308
|
+
client: "GitHub Copilot CLI",
|
|
309
|
+
clientVersion: "1.0.15",
|
|
310
|
+
modelFamily: "gpt-5.4",
|
|
311
|
+
});
|
|
312
|
+
} finally {
|
|
313
|
+
await ctx.cleanup();
|
|
314
|
+
}
|
|
315
|
+
});
|
|
316
|
+
|
|
317
|
+
test("metrics synthesizes hosted Copilot bot commits without local notes", async () => {
|
|
318
|
+
const ctx = await createTempContext("copilot-hosted-metrics");
|
|
319
|
+
try {
|
|
320
|
+
await initGitRepo(ctx.repo);
|
|
321
|
+
await writeFile(join(ctx.repo, "baseline.ts"), "export const baseline = true;\n");
|
|
322
|
+
await commitAll(ctx.repo, "initial");
|
|
323
|
+
await runCommand("git", ["config", "user.name", "copilot[bot]"], {
|
|
324
|
+
cwd: ctx.repo,
|
|
325
|
+
});
|
|
326
|
+
await runCommand(
|
|
327
|
+
"git",
|
|
328
|
+
["config", "user.email", "000000+copilot[bot]@users.noreply.github.com"],
|
|
329
|
+
{ cwd: ctx.repo },
|
|
330
|
+
);
|
|
331
|
+
await writeFile(
|
|
332
|
+
join(ctx.repo, "agent.ts"),
|
|
333
|
+
"export const generated = true;\n",
|
|
334
|
+
);
|
|
335
|
+
await commitAll(
|
|
336
|
+
ctx.repo,
|
|
337
|
+
"feat: hosted change\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>",
|
|
338
|
+
);
|
|
339
|
+
|
|
340
|
+
const metrics = await runCommand(CLI_BIN, ["metrics"], {
|
|
341
|
+
cwd: ctx.repo,
|
|
342
|
+
env: { HOME: ctx.home },
|
|
343
|
+
});
|
|
344
|
+
|
|
345
|
+
expect(metrics.stdout).toContain("## AI Coding Metrics");
|
|
346
|
+
expect(metrics.stdout).toContain("**AI contribution: ~50%**");
|
|
347
|
+
expect(metrics.stdout).toContain("**Assistant runtime:** GitHub Copilot");
|
|
348
|
+
expect(metrics.stdout).toContain("`agent.ts` — 50% AI (1 lines)");
|
|
349
|
+
expect(metrics.stdout).not.toContain("**Estimated cost:**");
|
|
211
350
|
} finally {
|
|
212
351
|
await ctx.cleanup();
|
|
213
352
|
}
|