agent-council 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,346 @@
1
+ # Test Coverage Analysis by Logical Area Prompt
2
+
3
+ Give this prompt to an AI coding agent from the root of your Ruby project. It
4
+ uses cov-loupe to produce a report of line coverage organized by the logical
5
+ parts of your system, down to individual classes and methods, and ranks the
6
+ gaps by risk. It never asks for input, so it also works headless (e.g.
7
+ `claude -p`, `codex exec`, CI).
8
+
9
+ **Requirement:** run your full test suite (every suite that feeds SimpleCov)
10
+ first, make sure it passes, and don't change source or test files until the
11
+ analysis finishes. The prompt does not run tests; it reads the existing
12
+ `coverage.json` and refuses to start if that file looks out of date. This lets
13
+ several agents run it concurrently against the same coverage data.
14
+
15
+ ---
16
+
17
+ ### Unattended Operation
18
+
19
+ This prompt never asks the user anything, so it runs the same interactively or
20
+ headless. Never stop to wait for input: when a precondition fails, stop with an
21
+ error message instead. The invocation may name a different report location
22
+ (e.g. "write the report to `tmp/reports`") or coverage file location.
23
+
24
+ ### Preconditions
25
+
26
+ Before you begin the analysis:
27
+
28
+ 1. If your current working directory is not the project root (the output of
29
+ `git rev-parse --show-toplevel`), stop with an error that states your
30
+ current working directory and the expected root. Do not run the analysis.
31
+ 2. Always open the report by citing the most recent git commit at the time you
32
+ begin writing.
33
+ 3. If `git status` shows uncommitted changes, include them in the analysis:
34
+ analyze the working tree as it is, and list the `git status` details
35
+ immediately after the commit citation.
36
+ 4. Analyze the files git would consider part of the project: tracked files
37
+ plus new files not yet added, excluding anything git ignores
38
+ (`git ls-files --cached --others --exclude-standard`). Mark the untracked
39
+ files in the `git status` details.
40
+
41
+ ----
42
+
43
+ - You are a senior engineer assessing the test suite of this project.
44
+ - Your task is to produce a report of line coverage **organized by the logical
45
+ parts of the system**, not just a per-file list, and to judge where the gaps
46
+ actually matter.
47
+ - Get every coverage number from cov-loupe. Do not read `coverage.json`
48
+ directly, and do not estimate coverage by reading tests. Aggregating
49
+ cov-loupe's per-line output into area totals is expected (see Step 4).
50
+ - Prefer the cov-loupe MCP server. If it is unavailable, use the CLI
51
+ (`cov-loupe`, or `bundle exec cov-loupe` if it is in the Gemfile; run
52
+ `cov-loupe --help` for usage), adding `-m cli -fJ` for JSON output. Say in
53
+ the report which one you used.
54
+ - Use source and test files only to define the logical areas and to judge the
55
+ importance of uncovered code.
56
+ - Do not run the test suite, and do not modify source files, test files, or
57
+ coverage output. Other agents may be running this analysis concurrently
58
+ against the same data. This is a read-only analysis; the only file you
59
+ write is the report.
60
+ - Before recommending tests for uncovered code, check whether it is
61
+ intentionally excluded or untested: SimpleCov filters and `# :nocov:`
62
+ markers, and any project documentation or code comments explaining why.
63
+
64
+ ## Step 1: Verify the coverage data is current
65
+
66
+ The user must run the full, passing test suite before this analysis. Verify
67
+ what you can, and if any check below fails, stop with an error that names the
68
+ failed check, gives the coverage file's path, size, and modification time (if
69
+ it exists), and tells the user to rerun the full suite. Do not write a report.
70
+
71
+ 1. Find the SimpleCov configuration (`.simplecov`, `spec/spec_helper.rb`,
72
+ `test/test_helper.rb`, or wherever the project starts SimpleCov) and the
73
+ coverage file (`coverage/coverage.json` unless the configuration or the
74
+ invocation says otherwise). The file must exist and be written by
75
+ SimpleCov 1.x. Record its path, size in bytes, and modification time (in
76
+ UTC) from the file system (e.g. `ls -l` or `stat`); this metadata is not
77
+ coverage data, so reading it is allowed.
78
+ 2. Run the project totals with the tracked globs from Step 2
79
+ (`project_coverage_totals` / `cov-loupe -m cli -fJ -g '<globs>' totals`)
80
+ and the file list with the same globs (`project_coverage` /
81
+ `cov-loupe -m cli -fJ -g '<globs>' list`). `timestamp_status` must be `ok`,
82
+ and `newer_files`, `deleted_files`, `length_mismatch_files`,
83
+ `unreadable_files`, and `skipped_files` must all be empty. These show
84
+ whether source files changed after the run.
85
+ 3. No test, fixture, or configuration file may be newer than the coverage
86
+ file, since cov-loupe checks only source files. For example:
87
+ `find spec test features Gemfile.lock .rspec .simplecov -newer coverage/coverage.json`
88
+ (adjusted to the paths that exist). Include directories in the search,
89
+ not just files, so that deleted test files show up too.
90
+ 4. If the project records test results, they must show no failures: for
91
+ RSpec, the file named by `example_status_persistence_file_path`
92
+ (commonly `.rspec_status`) must have no `failed` entries.
93
+
94
+ These checks cannot prove that the last run was the complete suite. Say in
95
+ the report that the analysis relies on the user's assurance that it was.
96
+
97
+ `missing_tracked_files` (tracked files the suite never loaded) is not a
98
+ staleness failure; it is a finding for Step 2.
99
+
100
+ ## Step 2: Establish the project baseline
101
+
102
+ 1. Get overall totals with no globs, then with tracked globs matching the
103
+ code SimpleCov is meant to measure (from its `track_files` and filter
104
+ settings; typically `lib/**/*.rb`, or `app/**/*.rb,lib/**/*.rb` for Rails).
105
+ Passing `tracked_globs` makes cov-loupe report tracked files that have
106
+ **no** coverage data at all (`missing_tracked_files`, `without_coverage`);
107
+ these are files never loaded by the suite and are often the biggest real
108
+ gap.
109
+ 2. Get the full file list as JSON, sorted ascending (worst first).
110
+ 3. Note what the numbers cover: cov-loupe reports **line** coverage only.
111
+ If SimpleCov also collects branch coverage, say that branch coverage is out
112
+ of scope for this report.
113
+
114
+ ## Step 3: Identify the logical areas
115
+
116
+ Before computing any rollups, decide how to carve up the code. The goal is the
117
+ most precise, targeted analysis, so draw boundaries by what the code does, not
118
+ by where it lives. File and directory boundaries are not area boundaries: one
119
+ file may hold a class or module with a critical mission next to code that
120
+ matters little, and those belong in different areas. This takes more reading
121
+ and more tokens than a file-level rollup; that is the point.
122
+
123
+ 1. Read the code to understand its structure: entry points, `require` and
124
+ autoload structure, namespaces, classes, modules, and methods, and how
125
+ responsibilities flow between them. Identify the kind of project (Rails
126
+ app, gem, CLI tool, service) and its conventions.
127
+ 2. Define **logical areas at whatever granularity is meaningful**. An area is
128
+ a set of code units (classes, modules, methods, or line ranges) wherever
129
+ they live. It can be:
130
+ - a broad grouping spanning several directories (e.g. "everything in the
131
+ checkout flow", "the public API"),
132
+ - a single namespace or class,
133
+ - a single method or a few methods within a class (e.g. the price
134
+ calculation inside an order model, or the argument parsing in a CLI
135
+ command),
136
+ - or a cross-cutting concern spanning many places (e.g. authorization,
137
+ error handling, input validation).
138
+ 3. Arrange the areas in a hierarchy where that helps (a broad area containing
139
+ narrower ones), and nest as deep as it is useful. Cross-cutting areas may
140
+ overlap others; say so where they do.
141
+ 4. Record each area's definition precisely as file paths with line ranges, and
142
+ name the classes, modules, or methods those ranges cover so a reader can
143
+ follow them. When an area is made of whole files, a glob is a fine
144
+ shorthand.
145
+ 5. Make sure at least one level of the hierarchy is a **complete partition**:
146
+ every coverable line in the source files under analysis belongs to exactly
147
+ one area at that level, with an explicit "Other" area for code that fits
148
+ nowhere. This level is what the totals are reconciled against.
149
+ 6. Give each area a **criticality** judgment with a short reason. Treat it as a
150
+ continuum, not a fixed set of buckets: place each area relative to the
151
+ others and explain what drives the placement. Weigh user-visible behavior,
152
+ data correctness and integrity, money and security (authentication,
153
+ authorization, code evaluation, file system access), error paths, public
154
+ API, and how often the code runs. Use the framework's usual danger spots as
155
+ prompts, not as fixed categories: in a Rails app, validations, callbacks,
156
+ and scopes that guard data; authorization in controllers; background-job
157
+ retries and failure handling; mailers; CSRF and parameter filtering. In
158
+ other projects, look at domain rules and calculations, state transitions,
159
+ external API clients, file I/O and parsing, serialization, and concurrency.
160
+
161
+ ## Step 4: Compute coverage per area
162
+
163
+ 1. For each file touched by an area, get per-line data with
164
+ `file_coverage_detailed` / `cov-loupe -m cli -fJ detailed <path>`. Each row
165
+ gives a line number, its hit count, and whether it is covered; lines that
166
+ are not coverable (comments, blank lines, `end`) do not appear.
167
+ 2. For each area, sum the covered and coverable lines that fall in its line
168
+ ranges, and compute the percentage. Also record the number of code units
169
+ in the area and its weakest units (e.g. the least-covered methods).
170
+ 3. Get a broader area's numbers by summing its children's line counts, never
171
+ by averaging child percentages.
172
+ 4. For an area made of whole files, cross-check your sums against
173
+ `project_coverage_totals` with `tracked_globs` /
174
+ `cov-loupe -m cli -fJ -g '<globs>' totals`; they must match exactly.
175
+ 5. Confirm that the line totals for the partition level from Step 3 add up to
176
+ the project's covered and total lines for the same overall globs. If they
177
+ don't, find the overlap or orphaned lines and fix the map; state any
178
+ remaining discrepancy.
179
+ 6. Keep the aggregation mechanical and reproducible. If you write a helper
180
+ script to sum the per-line rows, put it in a temporary directory of your
181
+ own outside the project (e.g. from `mktemp -d`, so concurrent agents don't
182
+ collide), feed it cov-loupe output only, and include it in the appendix.
183
+
184
+ ## Step 5: Examine the gaps
185
+
186
+ Decide for yourself which files and areas deserve a close look. Do not apply a
187
+ fixed percentage cutoff: weigh each area's coverage against its criticality,
188
+ so a critical area at a high percentage can still warrant scrutiny while a
189
+ low-stakes area at a lower percentage may not. Always include every tracked
190
+ file with no coverage data. State the reasoning behind your selection in the
191
+ report.
192
+
193
+ For each selected file or area:
194
+
195
+ 1. Get the uncovered lines (`file_uncovered_lines` /
196
+ `cov-loupe -m cli -fJ uncovered <path>`); use `file_coverage_detailed` or
197
+ `cov-loupe -m cli -s uncovered -n 2 uncovered <path>` to see the source in
198
+ context.
199
+ 2. Characterize each cluster of uncovered lines: error/rescue path, edge-case
200
+ branch, defensive guard, platform-specific code, dead code, debug/logging
201
+ code, or core logic.
202
+ 3. Find the tests for that code, following the project's conventions (e.g.
203
+ `spec/` mirroring `lib/` or `app/`, or `test/`). Note whether the gap is
204
+ "no tests at all", "tests exist but miss these paths", or "covered only
205
+ indirectly by integration tests".
206
+ 4. Judge the **risk** of each gap: what goes wrong for a user if this code is
207
+ broken and no test notices? Be concrete.
208
+ 5. Judge, separately from risk, how **hard the gap is to test**: dependencies
209
+ to set up, I/O, external services, time, concurrency, or missing seams
210
+ (code that can't be exercised without refactoring).
211
+ 6. Flag likely dead code separately: it should be deleted, not tested.
212
+
213
+ Do not pad the report: a high percentage with an untested critical error path
214
+ is worth more attention than modest coverage of formatting helpers.
215
+
216
+ ## Step 6: Look beyond line percentages
217
+
218
+ Briefly assess, per major area, using the tests themselves:
219
+
220
+ - Whether coverage comes from focused unit tests or only from broad
221
+ integration/end-to-end tests (high coverage via integration alone can hide
222
+ weak assertions).
223
+ - Whether tests assert on behavior or merely execute code (spot-check a few
224
+ tests per area; cite them).
225
+ - Consistency across entry points: behavior reachable through more than one
226
+ interface (e.g. web UI and API, CLI and library) should be tested through
227
+ each.
228
+ - Testing anti-patterns. Report only those you actually observe, such as:
229
+ over-mocking that makes tests brittle or hides integration bugs;
230
+ near-duplicate test code that should use shared examples, helpers, or
231
+ iteration over input arrays; untested error handling and failure
232
+ scenarios; missing integration tests for critical user flows; untested
233
+ authorization logic; controller tests where request specs would serve
234
+ better (Rails); missing tests for jobs, mailers, external integrations,
235
+ CLI argument parsing, or data transformation pipelines.
236
+ - Testing infrastructure problems: inadequate fixtures or factories, missing
237
+ helpers, or code structure that makes tests hard to write.
238
+ - Well-tested areas: where coverage and test quality are both strong, and
239
+ the testing patterns there that are worth repeating elsewhere.
240
+
241
+ Keep this qualitative and evidence-based; cite test files and line numbers.
242
+
243
+ ## Report
244
+
245
+ Write your analysis in the project root (or in the directory the invocation
246
+ names) as a Markdown file whose name is:
247
+ - today's date in UTC `%Y-%m-%d-%H-%M` format +
248
+ - '-test-coverage-analysis-' +
249
+ - your name (e.g. 'codex', 'claude', 'gemini') +
250
+ - the `.md` extension.
251
+
252
+ If a file with that name already exists (another agent finished in the same
253
+ minute), append `-2`, `-3`, and so on before the extension. Never overwrite an
254
+ existing report.
255
+
256
+ The file should have the following structure:
257
+
258
+ ---
259
+
260
+ ### Executive Summary
261
+ - Commit cited, cov-loupe interface used (MCP or CLI), and the results of the
262
+ Step 1 checks.
263
+ - The coverage file's path (relative to the project root), size, and
264
+ modification time (UTC), so readers can tell which test run the report
265
+ describes.
266
+ - Overall line coverage (with and without tracked globs).
267
+ - The best- and worst-covered areas, and the top risk areas.
268
+ - A **one-line verdict** (e.g. *"Strong domain coverage; payment error
269
+ handling is the main gap"*).
270
+
271
+ ### Area Map
272
+ The hierarchy of logical areas from Step 3, showing for each area its parent,
273
+ its definition (file paths with line ranges and the code units they cover, or
274
+ globs for whole files), and criticality with reason. Mark the partition level
275
+ used for reconciliation and any overlapping cross-cutting areas. Explain why
276
+ you carved the code this way.
277
+
278
+ ### Coverage by Area
279
+ One table per hierarchy level that is worth showing, each sorted ascending by
280
+ percentage:
281
+
282
+ | Area | Criticality | Code units | Covered | Total | % | Weakest unit (%) |
283
+ |------|-------------|------------|---------|-------|---|------------------|
284
+
285
+ List tracked files with no coverage data separately, with the areas they
286
+ belong to.
287
+
288
+ Follow with a short interpretation of each area that stands out, positively or
289
+ negatively, relative to its criticality.
290
+
291
+ ### Gap Analysis
292
+ State how you chose what to examine. Then, for each gap from Step 5, grouped by
293
+ area and ordered by risk:
294
+ - File and uncovered line numbers (as reported by cov-loupe).
295
+ - What the code does and the gap's category.
296
+ - Test status (none / partial / indirect only).
297
+ - Risk, with a concrete failure scenario.
298
+ - Difficulty to test, with what makes it hard.
299
+ - Recommendation: test to add (describe the scenario, not the code), code to
300
+ delete, or "acceptable as is" with the reason.
301
+
302
+ ### Well-Tested Areas
303
+ Areas where coverage and test quality are both strong, and the patterns
304
+ worth repeating, with citations.
305
+
306
+ ### Test Quality Observations
307
+ The remaining findings from Step 6 (unit vs. integration balance, assertion
308
+ strength, entry-point consistency, anti-patterns, infrastructure), with
309
+ citations.
310
+
311
+ ### Data Caveats
312
+ Reliance on the user's assurance that the last run was the complete, passing
313
+ suite; tracked files missing from the coverage data; branch coverage being out
314
+ of scope; ambiguous line assignments at area boundaries; anything else that
315
+ limits confidence in the numbers.
316
+
317
+ ### Prioritized Recommendations
318
+ A table of tasks, sorted by priority (highest first):
319
+
320
+ | # | Task | Area | Importance (1–10) | Effort (S/M/L) | Priority | Expected improvement |
321
+ |---|------|------|-------------------|----------------|----------|----------------------|
322
+
323
+ - **Task**: one concrete action (tests to add, code to delete, or an
324
+ infrastructure fix).
325
+ - **Importance**: the risk the task removes, from the area's criticality and
326
+ the gap's risk.
327
+ - **Effort**: from the gap's difficulty to test (Step 5).
328
+ - **Priority**: rank by importance relative to effort. Among tasks of similar
329
+ importance, put the cheaper one first: quick wins are easy to clear, and each
330
+ one cleared drops out of the next report, so later reports concentrate on
331
+ what remains. Place infrastructure fixes ahead of the tasks they unblock.
332
+
333
+ Briefly state the rule you used to combine importance and effort. After the
334
+ table, give each task a ready-to-use prompt the user can give an AI assistant
335
+ to implement it, keyed by its number.
336
+
337
+ ### Appendix: Commands Run
338
+ Every cov-loupe call (MCP tool name and arguments, or CLI command) and
339
+ freshness check used to produce the numbers, plus any aggregation script, so the
340
+ report can be reproduced.
341
+
342
+ ---
343
+
344
+ End your final message with the report's path and the one-line verdict, or,
345
+ if you stopped early, with the error. A headless caller can then find the
346
+ outcome without parsing the rest of your output.
data/exe/agent-council ADDED
@@ -0,0 +1,12 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ # agent-council — executable entry point. All code lives in lib/:
5
+ # lib/agent_council.rb orchestration (AgentCouncil)
6
+ # lib/agent_council/harnesses/pi_harness.rb pi-specific behavior (AgentCouncil::Harnesses::PiHarness)
7
+ #
8
+ # Usage, options, and AGENT_COUNCIL_OPTS: agent-council -h, and the README.
9
+
10
+ require_relative '../lib/agent_council'
11
+
12
+ AgentCouncil.main(ARGV)
data/exe/pi-models ADDED
@@ -0,0 +1,39 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ require_relative '../lib/agent_council/platform'
5
+
6
+ AgentCouncil.abort_if_native_windows('pi-models')
7
+
8
+ require_relative '../lib/agent_council/pi_models'
9
+
10
+ help_output = ARGV.delete('-h') || ARGV.delete('--help')
11
+ json_output = ARGV.delete('--json') || ARGV.delete('-j')
12
+ ids_output = ARGV.delete('--ids') || ARGV.delete('-i')
13
+ ARGV.delete('--table') || ARGV.delete('-t') # table is the default format; accepted for explicitness
14
+
15
+ if help_output
16
+ puts AgentCouncil::PiModels.help_text
17
+ exit
18
+ end
19
+
20
+ unless ARGV.empty?
21
+ warn "pi-models: unknown option(s): #{ARGV.join(' ')}"
22
+ warn "Try 'pi-models --help' for usage."
23
+ exit 1
24
+ end
25
+
26
+ begin
27
+ models = AgentCouncil::PiModels.fetch_models
28
+ rescue AgentCouncil::PiModels::Error => e
29
+ warn "pi-models: #{e.message}"
30
+ exit 1
31
+ end
32
+
33
+ if json_output
34
+ puts AgentCouncil::PiModels.format_as_json(models)
35
+ elsif ids_output
36
+ print AgentCouncil::PiModels.format_as_ids(models)
37
+ else
38
+ puts AgentCouncil::PiModels.format_as_table(models)
39
+ end
@@ -0,0 +1,169 @@
1
+ # frozen_string_literal: true
2
+
3
+ # AgentCouncil::Harnesses::PiHarness — all knowledge specific to the pi
4
+ # coding-agent harness: how to invoke the pi CLI for one model run, the
5
+ # tool allowlist, and read-only scaffolding delivered with it, and how to
6
+ # read pi's JSONL event stream afterward.
7
+ #
8
+ # AgentCouncil is harness-agnostic and refers to this module only via
9
+ # AgentCouncil::HARNESS. A second harness (claude, codex, ...) would be
10
+ # another module under AgentCouncil::Harnesses shaped like this one —
11
+ # this is deliberately a plain namespace, not an adapter framework.
12
+
13
+ require 'json'
14
+
15
+ module AgentCouncil
16
+ module Harnesses
17
+ module PiHarness
18
+ NAME = 'pi'
19
+
20
+ # Analysis toolset (pi's built-in tool names): edit/write (and the
21
+ # Windows-only powershell) are withheld. bash, a full shell, stays for
22
+ # investigation (git log/status, dependency inspection, ...); only the
23
+ # scaffolding text asks the model to use it read-only.
24
+ TOOLS = 'read,grep,find,ls,bash'
25
+
26
+ # Invocation for one model run. The substantive prompt is not part of argv;
27
+ # it is piped to stdin (byte-identical for every analyst; the
28
+ # consolidator gets its own).
29
+ module_function def argv(provider, model, scratch_dir)
30
+ [
31
+ NAME,
32
+ '-p', # non-interactive: process prompt and exit
33
+ '--mode', 'json', # JSONL event stream on stdout
34
+ '--no-session', # ephemeral: no session state between runs
35
+ '--tools', TOOLS,
36
+ '--provider', provider,
37
+ '--model', model,
38
+ '--append-system-prompt', scaffold_text(scratch_dir)
39
+ ]
40
+ end
41
+
42
+ # Scaffolding appended to pi's system prompt. Identical for every agent
43
+ # except the scratch dir path.
44
+ module_function def scaffold_text(scratch_dir)
45
+ <<~TEXT
46
+ You are one of the independent agents agent-council runs on this
47
+ repository. This is a strictly read-only analysis task.
48
+
49
+ Rules:
50
+ - Do not modify, create, rename, or delete any project source or
51
+ configuration file. Do not change git state, install or remove
52
+ software, modify user or system configuration, or start persistent
53
+ services.
54
+ - Do not run test suites, builds, coverage tools, application servers,
55
+ or database tasks unless the user's prompt explicitly permits it:
56
+ other agents may be running concurrently and such commands can collide
57
+ through shared state (databases, ports, Docker, fixed temp paths).
58
+ Where the prompt permits them, direct their temporary output to the
59
+ scratch directory below, and still skip any that need shared
60
+ services (databases, servers, fixed ports, Docker). Otherwise, if
61
+ running such a command would materially improve your analysis,
62
+ describe what you would run and why instead of running it.
63
+ - You may run clearly read-only commands: reading, searching, and
64
+ listing files; inspecting git history and status; inspecting
65
+ dependency and configuration information.
66
+ - If you need scratch space for temporary artifacts, use this
67
+ disposable directory that is unique to your run: #{scratch_dir}
68
+ - Deliver your complete analysis as your final message text. Do not
69
+ write the report to a file; the orchestrator captures your final
70
+ message directly.
71
+ TEXT
72
+ end
73
+
74
+ # --- event stream reading ---
75
+
76
+ # The assistant message from a single pi JSONL line, or nil if the line
77
+ # is not an assistant message_end event.
78
+ module_function def assistant_message_from_line(line)
79
+ event =
80
+ begin
81
+ JSON.parse(line)
82
+ rescue JSON::ParserError
83
+ nil
84
+ end
85
+
86
+ return nil unless event.is_a?(Hash)
87
+ return nil unless event['type'] == 'message_end'
88
+
89
+ message = event['message']
90
+ return nil unless message && message['role'] == 'assistant'
91
+
92
+ message
93
+ end
94
+
95
+ # Assistant messages from all message_end events in a pi JSONL stream,
96
+ # in order. Unparseable lines are skipped.
97
+ module_function def assistant_messages(jsonl)
98
+ jsonl.each_line.filter_map { |line| assistant_message_from_line(line) }
99
+ end
100
+
101
+ module_function def response_text(message)
102
+ Array(message['content'])
103
+ .select { |block| block['type'] == 'text' }
104
+ .map { |block| block['text'] }
105
+ .join("\n")
106
+ end
107
+
108
+ # The final assistant response: per pi's docs, message_end holds the
109
+ # authoritative final message of each turn; the last assistant message_end
110
+ # with actual text is the report (earlier ones belong to tool-use turns).
111
+ module_function def final_response(jsonl)
112
+ message = saved_message(assistant_messages(jsonl))
113
+ message && response_text(message)
114
+ end
115
+
116
+ # The message final_response saves: the last one with text.
117
+ module_function def saved_message(messages)
118
+ messages.reverse_each.detect { |m| !response_text(m).empty? }
119
+ end
120
+
121
+ # Our usage keys, each mapped to pi's name for it. pi normalizes usage
122
+ # across providers; its input excludes cache reads and writes.
123
+ USAGE_KEYS = { 'input' => 'input', 'output' => 'output',
124
+ 'cache_read' => 'cacheRead', 'cache_write' => 'cacheWrite' }.freeze
125
+
126
+ # Token usage summed over all assistant messages (each is a separate
127
+ # model request), as AgentCouncil expects it: integer 'input', 'output',
128
+ # 'cache_read', and 'cache_write', plus 'cost' in US dollars when pi
129
+ # priced the run. nil when the stream reports no usage, as some
130
+ # providers don't.
131
+ module_function def usage(jsonl)
132
+ usages = assistant_messages(jsonl).map { |m| m['usage'] }.grep(Hash)
133
+ totals = USAGE_KEYS.transform_values { |key| usages.sum { |u| count(u[key]) } }
134
+ return nil if totals.values.all?(&:zero?)
135
+
136
+ cost = usages.sum { |u| u['cost'].is_a?(Hash) ? count(u['cost']['total']) : 0 }
137
+ cost.positive? ? totals.merge('cost' => cost.round(6)) : totals
138
+ end
139
+
140
+ module_function def count(value)
141
+ value.is_a?(Numeric) && value.positive? ? value : 0
142
+ end
143
+
144
+ # Stop reasons meaning a message was cut off, cancelled, or failed, not
145
+ # completed. error normally comes with an errorMessage, which is reported
146
+ # instead; this catches one that is missing or blank.
147
+ UNUSABLE_STOP_REASONS = %w[length aborted error].freeze
148
+
149
+ # Why the run produced no usable report, or nil. Pi exits 0 even when the
150
+ # provider rejects the request or the output hits its token limit, so this
151
+ # is how such failures are surfaced. Two messages are checked: the final
152
+ # one (an error in an earlier turn that a later turn recovered from is not
153
+ # a failure, while an error on the last turn means any earlier text is a
154
+ # partial fragment) and the one final_response would save (a cut-off
155
+ # message must not be saved just because a text-less message followed it).
156
+ # A non-blank string errorMessage on either wins over a cut-off stopReason on
157
+ # the other, since it says more about what went wrong.
158
+ module_function def error_message(jsonl)
159
+ messages = assistant_messages(jsonl)
160
+ candidates = [messages.last, saved_message(messages)].compact.uniq
161
+ message = candidates.detect { |m| m['errorMessage'].is_a?(String) && !m['errorMessage'].strip.empty? }
162
+ return message['errorMessage'] if message
163
+
164
+ message = candidates.detect { |m| UNUSABLE_STOP_REASONS.include?(m['stopReason']) }
165
+ message && "stopped early (stopReason: #{message['stopReason']})"
166
+ end
167
+ end
168
+ end
169
+ end