ace-review 0.54.2 → 0.55.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.ace-defaults/review/presets/integration.yml +11 -0
- data/CHANGELOG.md +13 -0
- data/handbook/guides/code-review-process.g.md +35 -0
- data/handbook/prompts/focus/phase/integration.md +3 -0
- data/handbook/prompts/synthesize-feedback.system.md +2 -1
- data/handbook/skills/as-review-pr/SKILL.md +3 -6
- data/handbook/workflow-instructions/review/pr.wf.md +27 -261
- data/lib/ace/review/atoms/diff_boundary_finder.rb +47 -7
- data/lib/ace/review/atoms/prompt_budget.rb +79 -0
- data/lib/ace/review/atoms/token_estimator.rb +9 -3
- data/lib/ace/review/cli/commands/review.rb +23 -5
- data/lib/ace/review/cli.rb +17 -1
- data/lib/ace/review/models/feedback_item.rb +2 -2
- data/lib/ace/review/models/review_options.rb +2 -0
- data/lib/ace/review/molecules/diff_scope.rb +92 -0
- data/lib/ace/review/molecules/feedback_synthesizer.rb +40 -31
- data/lib/ace/review/molecules/gh_comment_poster.rb +13 -12
- data/lib/ace/review/molecules/gh_comment_resolver.rb +7 -7
- data/lib/ace/review/molecules/gh_pr_comment_fetcher.rb +7 -6
- data/lib/ace/review/molecules/gh_pr_fetcher.rb +88 -23
- data/lib/ace/review/molecules/goals_brief.rb +236 -0
- data/lib/ace/review/molecules/llm_executor.rb +22 -0
- data/lib/ace/review/molecules/multi_model_executor.rb +1 -0
- data/lib/ace/review/molecules/preset_manager.rb +18 -2
- data/lib/ace/review/molecules/review_evidence.rb +52 -0
- data/lib/ace/review/molecules/subject_extractor.rb +2 -1
- data/lib/ace/review/molecules/subject_filter.rb +9 -2
- data/lib/ace/review/organisms/feedback_manager.rb +55 -25
- data/lib/ace/review/organisms/review_manager.rb +568 -76
- data/lib/ace/review/version.rb +1 -1
- data/lib/ace/review.rb +2 -1
- metadata +44 -11
- data/lib/ace/review/molecules/context_extractor.rb +0 -159
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: bccdbe28f93f46e3fb13b54474acd6e3f10b4b6b01e0b1259c1f9b2e3395f184
|
|
4
|
+
data.tar.gz: 6b948eea0b8a90354d71c5c233d3e2674fd550687f76d477374218d3e8abc820
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d4a3e6588a9e1808b086aa729062f407a4f89ff7d64269e0e4af57f8cec29933da7170388ad0ed998dbe3a504c96039d6bc4a2b9800f95638822eb3838ad78b3
|
|
7
|
+
data.tar.gz: 72153ae476b7416db5a286384c8fd2c023490ba1d3cb8a67a3b3d1f20dffc5c6ab46caa403348b54d6c84ee7511d3199436ae24870f23c4308621ea63c1f9587
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
presets:
|
|
2
|
+
- code-valid
|
|
3
|
+
description: "Cross-module contracts and regression risks within a review round"
|
|
4
|
+
review_role: integration
|
|
5
|
+
instructions:
|
|
6
|
+
bundle:
|
|
7
|
+
sections:
|
|
8
|
+
integration_focus:
|
|
9
|
+
title: "Integration Review Focus"
|
|
10
|
+
files:
|
|
11
|
+
- "prompt://focus/phase/integration"
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.55.0] - 2026-09-24
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- Scope complete PR diff blocks by module/lens and enforce 30k context, 30k instructions and 128k complete-input budgets without silent truncation.
|
|
15
|
+
- Cache a shared goals brief by source content and instructions; include selected prior findings and resolutions as optional context.
|
|
16
|
+
|
|
17
|
+
### Fixed
|
|
18
|
+
|
|
19
|
+
- Preserve PR identity, task requirements, complete sources and the selected diff in the actual prompt; reject command sources in PR review context.
|
|
20
|
+
- Resolve Pi model/reasoning and record actual reviewer execution identity; incomplete or empty reports do not count as completed reviews.
|
|
21
|
+
- Use one agent-led workflow: at least three PR rounds, ending after two consecutive rounds without confirmed Critical/High findings. Commits do not reset review progress; there is no separate final certification.
|
|
22
|
+
|
|
10
23
|
## [0.54.2] - 2026-09-13
|
|
11
24
|
|
|
12
25
|
### Technical
|
|
@@ -126,6 +126,41 @@ The project uses a structured review system that supports:
|
|
|
126
126
|
|
|
127
127
|
Reviews are executed through standardized commands that create organized sessions with input files, prompts, model reports, and synthesis results. The system handles both general code reviews and specialized handbook reviews with appropriate templates and context.
|
|
128
128
|
|
|
129
|
+
For a large monorepo PR, compose a review preset from high-level module and functional-lens presets. `file_pattern_groups` intersects its groups: a changed path must match every group's include patterns, and any matching exclusion removes it. Composition concatenates groups, while the older single `file_patterns` remains available for simple scopes. For example:
|
|
130
|
+
|
|
131
|
+
```yaml
|
|
132
|
+
presets:
|
|
133
|
+
admin:
|
|
134
|
+
file_pattern_groups:
|
|
135
|
+
- include: ["apps/admin/**"]
|
|
136
|
+
ui:
|
|
137
|
+
file_pattern_groups:
|
|
138
|
+
- include: ["**/*.tsx", "**/*.css"]
|
|
139
|
+
exclude: ["**/*.test.tsx"]
|
|
140
|
+
admin-ui:
|
|
141
|
+
presets: [admin, ui]
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Select only the scopes needed for the PR. A code-only pass does not account for omitted tests; include needed code/test evidence and cross-module integration within the round. Scope manifests show selected and omitted files. ACE refuses an unparsed diff, missing sources or an oversized prompt rather than silently truncating it.
|
|
145
|
+
|
|
146
|
+
The agent follows `wfi://review/pr`: minimum three completed PR rounds, ending after the last two consecutive rounds have no confirmed P0/P1 (Critical/High). Provider failures and incomplete reports do not count. Earlier open blockers remain visible; P2 and lower findings do not independently extend the loop. Changes of SHA do not reset counters and there is no additional final review or coverage certificate. Use `--evidence-session` to include selected earlier reports and finding dispositions, including reports from earlier commits. Formal merge checks remain at merge time.
|
|
147
|
+
|
|
148
|
+
Presets can opt into one shared, content-addressed goals brief. Sources name an exact PR ref and authority. Only base sources may be marked accepted; head sources remain proposals. The brief includes source links and hashes, and its cache key includes source content, summary instructions and the resolved model. Prepare it once before dry runs; subsequent scoped reviews reuse the same artifact. A dry run with a cache miss fails rather than unexpectedly calling a model.
|
|
149
|
+
|
|
150
|
+
```yaml
|
|
151
|
+
goals_brief:
|
|
152
|
+
sources:
|
|
153
|
+
- path: docs/requirements.md
|
|
154
|
+
ref: base
|
|
155
|
+
authority: accepted
|
|
156
|
+
- path: task
|
|
157
|
+
ref: head
|
|
158
|
+
authority: proposed
|
|
159
|
+
models: ["pi:glmflash", "gemini:flash-latest"]
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
`task` resolves the single applicable PR task spec. Run `ace-review --pr 123 --preset admin-ui --prepare-goals-brief` before `--dry-run`. Model preference is configurable and unavailable providers can be replaced; the cache records which model actually generated the brief. Changing the source content or summarization contract invalidates it.
|
|
163
|
+
|
|
129
164
|
## Integration with Development Workflow
|
|
130
165
|
|
|
131
166
|
### Development Phases
|
|
@@ -0,0 +1,3 @@
|
|
|
1
|
+
# Cross-module integration
|
|
2
|
+
|
|
3
|
+
Inspect the contracts between changed modules, including data production, publication and consumption. Check fixes and their possible regressions using the relevant code, tests and supplied prior findings. Report concrete defects with impact and evidence. Closed findings require new evidence to reopen. Integration is part of the current review round, not a separate final certification.
|
|
@@ -12,6 +12,7 @@ Read all provided review reports and identify **unique actionable findings**.
|
|
|
12
12
|
|
|
13
13
|
- **Single report**: Extract each finding with the reviewer tracked
|
|
14
14
|
- **Multiple reports**: When reviewers identify the same issue, merge them into one finding and track all reviewers
|
|
15
|
+
- Preserve every distinct finding, even when only one reviewer identified it. Consensus is metadata, not a filter.
|
|
15
16
|
|
|
16
17
|
When merging findings from multiple reviewers:
|
|
17
18
|
1. Merge them into a single finding
|
|
@@ -144,4 +145,4 @@ User input is directly interpolated into SQL.
|
|
|
144
145
|
- **For single reports**: Extract each finding with the reviewer in the `reviewers` array
|
|
145
146
|
- **For multiple reports**: Never duplicate findings - if multiple reviewers found the same issue, merge them
|
|
146
147
|
- Process ALL reports thoroughly to capture ALL unique findings
|
|
147
|
-
- When uncertain
|
|
148
|
+
- When uncertain whether two findings describe the same defect, keep them separate for verification.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: as-review-pr
|
|
3
|
-
description: Review PR and
|
|
3
|
+
description: Review PR through verified findings and converging rounds
|
|
4
4
|
# bundle: wfi://review/pr
|
|
5
5
|
# context: no-fork
|
|
6
6
|
# agent: general-purpose
|
|
@@ -24,7 +24,7 @@ integration:
|
|
|
24
24
|
assign:
|
|
25
25
|
steps:
|
|
26
26
|
- name: review-pr
|
|
27
|
-
description: Review
|
|
27
|
+
description: Review and resolve PR feedback through converging rounds
|
|
28
28
|
prerequisites:
|
|
29
29
|
- name: create-pr
|
|
30
30
|
strength: required
|
|
@@ -32,12 +32,9 @@ assign:
|
|
|
32
32
|
produces: [review-feedback]
|
|
33
33
|
consumes: [pull-request]
|
|
34
34
|
when_to_skip:
|
|
35
|
-
- "No
|
|
36
|
-
- "Changes are trivial (typo fix, config update)"
|
|
35
|
+
- "No PR exists yet"
|
|
37
36
|
effort: medium
|
|
38
37
|
tags: [review, quality]
|
|
39
|
-
context:
|
|
40
|
-
default: fork
|
|
41
38
|
skill:
|
|
42
39
|
kind: workflow
|
|
43
40
|
execution:
|
|
@@ -1,284 +1,50 @@
|
|
|
1
1
|
---
|
|
2
2
|
doc-type: workflow
|
|
3
|
-
title: Review PR
|
|
4
|
-
purpose: Review PR and
|
|
3
|
+
title: Review PR in converging rounds
|
|
4
|
+
purpose: Review a PR, verify concrete findings and finish after two clean rounds
|
|
5
5
|
ace-docs:
|
|
6
|
-
last-updated: 2026-
|
|
7
|
-
last-checked: 2026-
|
|
6
|
+
last-updated: 2026-09-24
|
|
7
|
+
last-checked: 2026-09-24
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
-
# Review PR
|
|
10
|
+
# Review PR
|
|
11
11
|
|
|
12
|
-
## Goal
|
|
12
|
+
## Goal and completion rule
|
|
13
13
|
|
|
14
|
-
|
|
14
|
+
The agent runs this workflow; ACE executes individual reviews. Complete at least **3 rounds for the PR**, with the **last 2 consecutive rounds free of confirmed P0/P1 (Critical/High)**. An unresolved earlier P0/P1 remains blocking even if another reviewer does not mention it. A confirmed P0/P1 resets the clean-round streak. There is no additional final review after this rule is satisfied.
|
|
15
15
|
|
|
16
|
-
|
|
16
|
+
One round covers the PR's needed scope, including cross-module integration. A large round can use several coherent module sessions; modules do not have separate round counters. A round counts only after the needed reports have completed and the agent has verified their findings. Failed providers and incomplete reports do not count as clean rounds.
|
|
17
17
|
|
|
18
|
-
|
|
18
|
+
Use at least one available reviewer per needed scope. Prefer the project's configured models, but preferences are not an allowlist. Honor the user's current provider/model restrictions. Record actual provider, model, reasoning and completion status. A zero process exit code alone is not a completed review.
|
|
19
19
|
|
|
20
|
-
##
|
|
20
|
+
## 1. Choose and prepare the scope
|
|
21
21
|
|
|
22
|
-
|
|
22
|
+
Identify the PR from the argument or `gh pr view`. Read its goal, current requirements, relevant ADRs and changed-file list. Use the smallest set of presets covering the change. Small coherent PRs can be reviewed whole; split large inputs by high-level module or functional lens, not arbitrary file chunks. Include needed contracts and code when reviewing tests and include integration behavior when modules interact.
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
Use `ace-review --pr <number> --preset <preset> --dry-run` to inspect the selected/omitted files, complete prompt and budget. Context and instructions each have a 30k ceiling; total input has a 128k ceiling. Change scope if too large; do not silently truncate or claim omitted code was reviewed. Generated files may use an appropriate deterministic check, recorded in the round summary, without a separate certification system.
|
|
25
25
|
|
|
26
|
-
|
|
27
|
-
gh pr view --json number -q '.number'
|
|
28
|
-
```
|
|
26
|
+
For presets with a goals brief, `--prepare-goals-brief` generates or reuses the shared summary. Its cache follows source content and instructions, not review-round state. Architectural foundations should receive design review before implementation.
|
|
29
27
|
|
|
30
|
-
|
|
28
|
+
## 2. Run a round and verify findings
|
|
31
29
|
|
|
32
|
-
|
|
30
|
+
Run `ace-review --pr <number> --preset <preset> --auto-execute` for the needed scopes. Pass selected previous sessions with `--evidence-session <path>` when useful. Earlier commit SHAs are expected: prior reports provide context and dispositions, not current-code certificates. Supply a short summary of fixes and outstanding issues through the preset context when it is more useful than full reports.
|
|
33
31
|
|
|
34
|
-
|
|
32
|
+
First-round review looks for concrete defects against the agreed requirements. Subsequent rounds check fixes, regressions, unresolved issues and the integration affected by changes. They may report newly evidenced defects, but should not redesign the solution or reopen a closed finding without new evidence.
|
|
35
33
|
|
|
36
|
-
|
|
37
|
-
# If PR number provided in arguments:
|
|
38
|
-
ace-review --pr <pr-number> [additional-flags]
|
|
34
|
+
Verify reports against code and requirements. Use `ace-review-feedback list --session <path>`, `show`, `verify` and `resolve` to preserve findings and dispositions. Distinguish defects, architectural decisions and optional improvements. A single reviewer's valid finding matters; do not discard it for lack of consensus. Severity is verified by the agent, not accepted just because the model assigned it.
|
|
39
35
|
|
|
40
|
-
|
|
41
|
-
ace-review --pr $(gh pr view --json number -q '.number') [additional-flags]
|
|
42
|
-
```
|
|
36
|
+
Fix confirmed P0/P1 and run appropriate tests. Verify P2 and lower findings, fix those within the task or defer with rationale; they do not independently require another round. Do not expand the PR to implement speculative improvements. If the same blocker recurs without an effective fix, diagnose the cause or request the needed product/architecture decision instead of repeating the same review.
|
|
43
37
|
|
|
44
|
-
|
|
45
|
-
- `ace-bundle wfi://review/pr` with `123` → `ace-review --pr 123`
|
|
46
|
-
- `ace-bundle wfi://review/pr` with `--preset code-deep` → `ace-review --pr <current-pr> --preset code-deep`
|
|
47
|
-
- `ace-bundle wfi://review/pr` with `123 --preset security` → `ace-review --pr 123 --preset security`
|
|
38
|
+
## 3. Record progress and finish
|
|
48
39
|
|
|
49
|
-
|
|
40
|
+
Keep a short `rounds.md` in the existing PR review artifacts under `.ace-local/review/`: completed round number, report paths and scopes, confirmed P0/P1, dispositions, tests, open blockers and consecutive clean rounds. Preserve this record across agent restarts. Commits, squash and rebase do not reset the total or clean-round counters. Report SHAs are diagnostic; materially changed code receives the relevant review, not an automatic restart of every scope.
|
|
50
41
|
|
|
51
|
-
|
|
42
|
+
Examples:
|
|
52
43
|
|
|
53
|
-
-
|
|
54
|
-
-
|
|
55
|
-
-
|
|
56
|
-
-
|
|
44
|
+
- P1 → clean → clean: finish after round 3.
|
|
45
|
+
- clean → clean → clean: finish after round 3.
|
|
46
|
+
- P1 → clean → P1 → clean → clean: finish after round 5.
|
|
47
|
+
- An incomplete attempt does not increment either counter and cannot hide an outstanding blocker.
|
|
48
|
+
- A round that finds a confirmed P1 is not clean even if it is fixed immediately afterward.
|
|
57
49
|
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
The review includes:
|
|
61
|
-
- LLM model reviews (e.g., `review-gemini.md`, `review-gpt4.md`)
|
|
62
|
-
- Developer feedback from PR comments (`review-dev-feedback.md`) - if the PR has comments
|
|
63
|
-
- Feedback items in `feedback/` directory
|
|
64
|
-
|
|
65
|
-
### Step 3: List Feedback Items
|
|
66
|
-
|
|
67
|
-
List the feedback items extracted from the review:
|
|
68
|
-
|
|
69
|
-
```bash
|
|
70
|
-
ace-review-feedback list --status draft
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
**Precondition**: Run this step only after Step 2 process exit is confirmed.
|
|
74
|
-
|
|
75
|
-
When session ambiguity is possible, use explicit session path from Step 2:
|
|
76
|
-
|
|
77
|
-
```bash
|
|
78
|
-
ace-review-feedback list --status draft --session <session-dir-from-step-2>
|
|
79
|
-
```
|
|
80
|
-
|
|
81
|
-
This shows all draft feedback items with their IDs, severity, summaries, and sources (LLM or Developer).
|
|
82
|
-
|
|
83
|
-
#### Understanding Feedback Context
|
|
84
|
-
|
|
85
|
-
Feedback items are **session-scoped**. The `feedback list` command discovers items based on:
|
|
86
|
-
1. Explicit `--session <path>` flag (if provided)
|
|
87
|
-
2. `.ace-review-session` cache file in current directory (auto-created after reviews)
|
|
88
|
-
|
|
89
|
-
If `feedback list` returns empty after a review, first verify Step 2 process completion.
|
|
90
|
-
The session may not be linked to current context.
|
|
91
|
-
Use `ace-review-feedback list --session <session-dir>` to list from a specific session:
|
|
92
|
-
|
|
93
|
-
```bash
|
|
94
|
-
# List feedback from a specific session (path shown in review output)
|
|
95
|
-
ace-review-feedback list --session .ace-local/review/sessions/review-8p2pk3
|
|
96
|
-
```
|
|
97
|
-
|
|
98
|
-
### Step 4: Verify Each Feedback Item
|
|
99
|
-
|
|
100
|
-
For each feedback item (prioritize Critical and High severity):
|
|
101
|
-
|
|
102
|
-
```bash
|
|
103
|
-
# Read the finding details
|
|
104
|
-
ace-review-feedback show {id}
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
**Note which items come from Developer Feedback** - these are from PR comments and should be resolved after implementation.
|
|
108
|
-
|
|
109
|
-
**Then verify in the codebase:**
|
|
110
|
-
|
|
111
|
-
1. **Check the claim** - Use grep/read to verify the issue exists:
|
|
112
|
-
- If claim is "X doesn't exist" → `grep -rn "class X" lib/`
|
|
113
|
-
- If claim is "method missing" → check the actual file
|
|
114
|
-
- If claim is "file not deleted" → `ls path/to/file`
|
|
115
|
-
|
|
116
|
-
2. **Mark the verification result:**
|
|
117
|
-
|
|
118
|
-
```bash
|
|
119
|
-
# If issue is confirmed (valid finding)
|
|
120
|
-
ace-review-feedback verify {id} --valid --research "Confirmed: issue exists at line X"
|
|
121
|
-
|
|
122
|
-
# If issue is not real (false positive)
|
|
123
|
-
ace-review-feedback verify {id} --invalid --research "False positive: handled by Y"
|
|
124
|
-
```
|
|
125
|
-
|
|
126
|
-
3. **Categorization guide:**
|
|
127
|
-
|
|
128
|
-
| Result | Command | When |
|
|
129
|
-
|--------|---------|------|
|
|
130
|
-
| ✅ VALID | `--valid` | Issue confirmed in code |
|
|
131
|
-
| ❌ INVALID | `--invalid` | **False positive** - claim is factually incorrect |
|
|
132
|
-
| ✅ DONE | `feedback resolve {id}` | Already fixed in this PR |
|
|
133
|
-
| ⏭️ SKIP | `verify --skip --research "Design: ..."` | **Correct finding**, but not being fixed |
|
|
134
|
-
| 📋 DEFER | `verify --skip --research "Tracked in task XXX"` | Important, but not this PR (create task first) |
|
|
135
|
-
|
|
136
|
-
**Key distinction:**
|
|
137
|
-
- `verify --invalid`: The finding is **wrong** (false positive)
|
|
138
|
-
- "Code doesn't exist" → Actually exists
|
|
139
|
-
- "Missing validation" → Exists elsewhere
|
|
140
|
-
- "Fails in CI" → Doesn't run in CI
|
|
141
|
-
- `verify --skip`: The finding is **correct**, but you're not fixing it
|
|
142
|
-
- Design decision: Intentional choice
|
|
143
|
-
- Deferred: Tracking in separate task
|
|
144
|
-
|
|
145
|
-
**Before skipping**: Always verify the issue still exists by reading the code. Never skip with "out of scope" - either it's a design decision or it should be tracked in a task.
|
|
146
|
-
|
|
147
|
-
**Example verification:**
|
|
148
|
-
```bash
|
|
149
|
-
# Claim: "TaskPatternExtractor is undefined"
|
|
150
|
-
grep -rn "class TaskPatternExtractor" ace-git/lib/
|
|
151
|
-
# Result: Found at ace-git/lib/ace/git/atoms/task_pattern_extractor.rb:10
|
|
152
|
-
ace-review-feedback verify {id} --invalid --research "Class exists at ace-git/lib/ace/git/atoms/task_pattern_extractor.rb:10"
|
|
153
|
-
```
|
|
154
|
-
|
|
155
|
-
**Skip verification for:**
|
|
156
|
-
- Low priority items (verify only if time permits)
|
|
157
|
-
- Documentation-only suggestions
|
|
158
|
-
- Style/formatting recommendations
|
|
159
|
-
- Developer Feedback items (these are human-verified)
|
|
160
|
-
|
|
161
|
-
### Step 5: List Pending Items
|
|
162
|
-
|
|
163
|
-
After verification, list items ready to work on:
|
|
164
|
-
|
|
165
|
-
```bash
|
|
166
|
-
ace-review-feedback list --status pending
|
|
167
|
-
```
|
|
168
|
-
|
|
169
|
-
This shows only verified valid items that need fixing.
|
|
170
|
-
|
|
171
|
-
### Step 6: Apply Priority Threshold
|
|
172
|
-
|
|
173
|
-
**Default behavior**: Implement **Medium and higher** severity items (skip Low).
|
|
174
|
-
|
|
175
|
-
This means:
|
|
176
|
-
- 🔴 Critical → Implement
|
|
177
|
-
- 🟡 High → Implement
|
|
178
|
-
- 🟢 Medium → Implement
|
|
179
|
-
- 🔵 Low → Skip (unless explicitly requested)
|
|
180
|
-
|
|
181
|
-
### Step 7: Implement Fixes
|
|
182
|
-
|
|
183
|
-
**CRITICAL: Every fix MUST be marked as resolved before moving to next item.**
|
|
184
|
-
|
|
185
|
-
For each pending item:
|
|
186
|
-
|
|
187
|
-
1. **Read the full details:**
|
|
188
|
-
```bash
|
|
189
|
-
ace-review-feedback show {id}
|
|
190
|
-
```
|
|
191
|
-
|
|
192
|
-
2. **Implement the fix** based on the recommendation
|
|
193
|
-
|
|
194
|
-
3. **IMMEDIATELY mark as resolved** (do NOT skip this step):
|
|
195
|
-
```bash
|
|
196
|
-
ace-review-feedback resolve {id} --resolution "Fixed: <description>"
|
|
197
|
-
```
|
|
198
|
-
|
|
199
|
-
4. **Commit the fix** with a clear message referencing the feedback item
|
|
200
|
-
|
|
201
|
-
5. **Note the commit SHA** for PR comment resolution (if from Developer Feedback)
|
|
202
|
-
|
|
203
|
-
⚠️ **Never leave a fixed item as "pending"** - this breaks feedback tracking.
|
|
204
|
-
|
|
205
|
-
### Step 8: Handle Not-Applicable Items
|
|
206
|
-
|
|
207
|
-
Before skipping, complete the verification checklist:
|
|
208
|
-
|
|
209
|
-
1. **Read the code** - Does the issue actually still exist?
|
|
210
|
-
2. **Check current changes** - Was this already fixed in this PR?
|
|
211
|
-
3. **Consider effort** - Is this a quick win (< 5 min) that should just be done?
|
|
212
|
-
|
|
213
|
-
For items that won't be fixed in this PR:
|
|
214
|
-
|
|
215
|
-
```bash
|
|
216
|
-
# Design decision - intentionally this way
|
|
217
|
-
ace-review-feedback verify {id} --skip --research "Design: uses polling for simplicity"
|
|
218
|
-
|
|
219
|
-
# Important but deferred - ALWAYS create/reference a task
|
|
220
|
-
ace-review-feedback verify {id} --skip --research "Tracked in task 253"
|
|
221
|
-
```
|
|
222
|
-
|
|
223
|
-
**Never skip with "out of scope"** - either:
|
|
224
|
-
- It's a design decision (explain why)
|
|
225
|
-
- It needs a follow-up task (create one and reference it)
|
|
226
|
-
|
|
227
|
-
### Step 9: Resolve PR Comments (for Developer Feedback items)
|
|
228
|
-
|
|
229
|
-
After implementing fixes for items sourced from **Developer Feedback**:
|
|
230
|
-
|
|
231
|
-
1. **Reply to the PR with commit reference**:
|
|
232
|
-
```bash
|
|
233
|
-
gh pr comment $ARGUMENTS --body "Fixed in $(git rev-parse --short HEAD)"
|
|
234
|
-
```
|
|
235
|
-
|
|
236
|
-
2. **Resolve review threads** (if feedback was from a review thread with a thread ID):
|
|
237
|
-
```bash
|
|
238
|
-
# For review threads (PRRT_xxx IDs from review-dev-feedback.md)
|
|
239
|
-
gh api graphql -f query='
|
|
240
|
-
mutation {
|
|
241
|
-
resolveReviewThread(input: {threadId: "PRRT_xxx"}) {
|
|
242
|
-
thread { isResolved }
|
|
243
|
-
}
|
|
244
|
-
}
|
|
245
|
-
'
|
|
246
|
-
```
|
|
247
|
-
|
|
248
|
-
Note: The thread IDs are included in `review-dev-feedback.md` in the format `(thread: PRRT_xxx)` or `(comment: IC_xxx)`.
|
|
249
|
-
|
|
250
|
-
### Comment Resolution Summary
|
|
251
|
-
|
|
252
|
-
| Feedback Source | Resolution Action |
|
|
253
|
-
|-----------------|-------------------|
|
|
254
|
-
| LLM Review | No action needed (not a PR comment) |
|
|
255
|
-
| Developer Feedback (issue comment) | Reply with commit reference |
|
|
256
|
-
| Developer Feedback (review thread) | Reply with commit reference AND resolve thread |
|
|
257
|
-
|
|
258
|
-
## Quick Reference
|
|
259
|
-
|
|
260
|
-
```bash
|
|
261
|
-
# Feedback commands
|
|
262
|
-
ace-review-feedback list # All feedback
|
|
263
|
-
ace-review-feedback list --status draft # Unverified items
|
|
264
|
-
ace-review-feedback list --status pending # Verified valid items
|
|
265
|
-
ace-review-feedback list --session <path> # From specific session
|
|
266
|
-
ace-review-feedback show {id} # Full item details
|
|
267
|
-
ace-review-feedback verify {id} --valid # Mark as valid
|
|
268
|
-
ace-review-feedback verify {id} --invalid # Mark as false positive
|
|
269
|
-
ace-review-feedback verify {id} --skip # Mark as skipped
|
|
270
|
-
ace-review-feedback resolve {id} # Mark as fixed
|
|
271
|
-
|
|
272
|
-
# PR comment resolution
|
|
273
|
-
gh pr comment $PR --body "Fixed in $(git rev-parse --short HEAD)"
|
|
274
|
-
```
|
|
275
|
-
|
|
276
|
-
## Output / Success Criteria
|
|
277
|
-
|
|
278
|
-
- [ ] PR review completed with feedback items
|
|
279
|
-
- [ ] Feedback items verified (Critical/High priority)
|
|
280
|
-
- [ ] False positives marked as invalid
|
|
281
|
-
- [ ] Confirmed items implemented
|
|
282
|
-
- [ ] PR comments addressed with commit references
|
|
283
|
-
- [ ] Review threads resolved (if applicable)
|
|
284
|
-
- [ ] All items marked as resolved or skipped
|
|
50
|
+
After the completion rule is met and changes have appropriate test results, update the PR description with actual reviewers, scopes, fixes, deferred findings and limitations, then mark the draft ready. Do not run a separate integration or SHA gate. Keep formal merge checks at merge time. Do not merge without explicit user authorization, post replies to people or resolve their threads without authorization. Do not make optional code changes after completion that would unnecessarily reopen review.
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "strscan"
|
|
4
|
+
|
|
3
5
|
module Ace
|
|
4
6
|
module Review
|
|
5
7
|
module Atoms
|
|
@@ -20,9 +22,7 @@ module Ace
|
|
|
20
22
|
# # { path: "test/foo_test.rb", content: "diff --git...", lines: 30, change_type: :modified }
|
|
21
23
|
# # ]
|
|
22
24
|
module DiffBoundaryFinder
|
|
23
|
-
|
|
24
|
-
# Matches: diff --git a/path/to/file b/path/to/file
|
|
25
|
-
DIFF_HEADER_PATTERN = /^diff --git a\/(.+?) b\/(.+?)$/
|
|
25
|
+
DIFF_HEADER_PREFIX = "diff --git "
|
|
26
26
|
|
|
27
27
|
# Pattern to detect new file mode
|
|
28
28
|
NEW_FILE_PATTERN = /^new file mode/
|
|
@@ -50,7 +50,8 @@ module Ace
|
|
|
50
50
|
current_lines = []
|
|
51
51
|
|
|
52
52
|
diff_text.each_line do |line|
|
|
53
|
-
if
|
|
53
|
+
if line.start_with?(DIFF_HEADER_PREFIX)
|
|
54
|
+
old_path, new_path = parse_header(line)
|
|
54
55
|
# Save the previous block if exists
|
|
55
56
|
if current_block
|
|
56
57
|
current_block[:content] = current_lines.join
|
|
@@ -60,10 +61,11 @@ module Ace
|
|
|
60
61
|
|
|
61
62
|
# Start a new block
|
|
62
63
|
current_block = {
|
|
63
|
-
path:
|
|
64
|
+
path: new_path,
|
|
65
|
+
old_path: old_path,
|
|
64
66
|
content: "",
|
|
65
67
|
lines: 0,
|
|
66
|
-
change_type:
|
|
68
|
+
change_type: (old_path == new_path) ? :modified : :renamed
|
|
67
69
|
}
|
|
68
70
|
current_lines = [line]
|
|
69
71
|
elsif current_block
|
|
@@ -88,6 +90,44 @@ module Ace
|
|
|
88
90
|
blocks
|
|
89
91
|
end
|
|
90
92
|
|
|
93
|
+
def self.parse_header(line)
|
|
94
|
+
paths = line.delete_prefix(DIFF_HEADER_PREFIX).strip
|
|
95
|
+
if !paths.start_with?('"') && (match = /\Aa\/(.+?) b\/(.+)\z/.match(paths))
|
|
96
|
+
return [match[1], match[2]]
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
scanner = StringScanner.new(paths)
|
|
100
|
+
old_path = scan_git_path(scanner)
|
|
101
|
+
raise ArgumentError, "Unparseable git diff header" unless scanner.skip(/\s+/)
|
|
102
|
+
|
|
103
|
+
new_path = scan_git_path(scanner)
|
|
104
|
+
unless scanner.eos? && old_path&.start_with?("a/") && new_path&.start_with?("b/")
|
|
105
|
+
raise ArgumentError, "Unparseable git diff header"
|
|
106
|
+
end
|
|
107
|
+
|
|
108
|
+
[old_path.delete_prefix("a/"), new_path.delete_prefix("b/")]
|
|
109
|
+
end
|
|
110
|
+
private_class_method :parse_header
|
|
111
|
+
|
|
112
|
+
def self.scan_git_path(scanner)
|
|
113
|
+
if (quoted = scanner.scan(/"(?:\\.|[^"])*"/))
|
|
114
|
+
decode_git_quoted(quoted[1...-1])
|
|
115
|
+
else
|
|
116
|
+
scanner.scan(/\S+/)
|
|
117
|
+
end
|
|
118
|
+
end
|
|
119
|
+
private_class_method :scan_git_path
|
|
120
|
+
|
|
121
|
+
def self.decode_git_quoted(value)
|
|
122
|
+
escapes = {"n" => "\n", "t" => "\t", "r" => "\r", "b" => "\b", "f" => "\f",
|
|
123
|
+
"v" => "\v", "a" => "\a", "\\" => "\\", '"' => '"'}
|
|
124
|
+
value.b.gsub(/\\([0-7]{3}|.)/m) do
|
|
125
|
+
code = Regexp.last_match(1)
|
|
126
|
+
code.match?(/\A[0-7]{3}\z/) ? [code.to_i(8)].pack("C") : escapes.fetch(code, code)
|
|
127
|
+
end.force_encoding(Encoding::UTF_8)
|
|
128
|
+
end
|
|
129
|
+
private_class_method :decode_git_quoted
|
|
130
|
+
|
|
91
131
|
# Parse and return just the file paths from a diff
|
|
92
132
|
#
|
|
93
133
|
# @param diff_text [String, nil] The unified diff text to parse
|
|
@@ -111,7 +151,7 @@ module Ace
|
|
|
111
151
|
def self.file_count(diff_text)
|
|
112
152
|
return 0 if diff_text.nil? || diff_text.empty?
|
|
113
153
|
|
|
114
|
-
diff_text.
|
|
154
|
+
diff_text.each_line.count { |line| line.start_with?(DIFF_HEADER_PREFIX) }
|
|
115
155
|
end
|
|
116
156
|
|
|
117
157
|
# Group file blocks by directory
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "token_estimator"
|
|
4
|
+
require_relative "context_limit_resolver"
|
|
5
|
+
|
|
6
|
+
module Ace
|
|
7
|
+
module Review
|
|
8
|
+
module Atoms
|
|
9
|
+
# Checks the complete rendered prompt before any model receives it.
|
|
10
|
+
# An over-budget packet must be respecified as a coherent review scope;
|
|
11
|
+
# silently truncating a diff cannot produce a complete review.
|
|
12
|
+
module PromptBudget
|
|
13
|
+
DEFAULT_INPUT_LIMIT = 128_000
|
|
14
|
+
DEFAULT_SYSTEM_LIMIT = 30_000
|
|
15
|
+
DEFAULT_CONTEXT_LIMIT = 30_000
|
|
16
|
+
OUTPUT_RESERVE = 8_192
|
|
17
|
+
ESTIMATE_SAFETY_FACTOR = 1.25
|
|
18
|
+
|
|
19
|
+
def self.check(system_prompt:, user_prompt:, models:, config: {}, subject: nil, instruction_tokens: nil)
|
|
20
|
+
settings = config || {}
|
|
21
|
+
input_limit = integer_limit(settings, "input_max_tokens", DEFAULT_INPUT_LIMIT)
|
|
22
|
+
raise ArgumentError, "input_max_tokens cannot exceed 128000" if input_limit > DEFAULT_INPUT_LIMIT
|
|
23
|
+
system_limit = integer_limit(settings, "system_max_tokens", DEFAULT_SYSTEM_LIMIT)
|
|
24
|
+
raise ArgumentError, "system_max_tokens cannot exceed 30000" if system_limit > DEFAULT_SYSTEM_LIMIT
|
|
25
|
+
context_limit = integer_limit(settings, "context_max_tokens", DEFAULT_CONTEXT_LIMIT)
|
|
26
|
+
raise ArgumentError, "context_max_tokens cannot exceed 30000" if context_limit > DEFAULT_CONTEXT_LIMIT
|
|
27
|
+
system_tokens = conservative_estimate(system_prompt)
|
|
28
|
+
user_tokens = conservative_estimate(user_prompt)
|
|
29
|
+
total_tokens = system_tokens + user_tokens
|
|
30
|
+
# The context ceiling covers all non-diff user input, including
|
|
31
|
+
# instructions that also have their own ceiling. A proposed preset
|
|
32
|
+
# cannot relabel project documents as instructions to evade it.
|
|
33
|
+
context_tokens = subject ? [user_tokens - conservative_estimate(subject), 0].max : nil
|
|
34
|
+
|
|
35
|
+
errors = []
|
|
36
|
+
all_instruction_tokens = system_tokens + instruction_tokens.to_i
|
|
37
|
+
errors << "review instructions ~#{all_instruction_tokens} tokens exceeds #{system_limit}" if all_instruction_tokens > system_limit
|
|
38
|
+
errors << "non-diff review context ~#{context_tokens} tokens exceeds #{context_limit}" if context_tokens && context_tokens > context_limit
|
|
39
|
+
errors << "complete prompt ~#{total_tokens} tokens exceeds #{input_limit}" if total_tokens > input_limit
|
|
40
|
+
|
|
41
|
+
eligible_models = []
|
|
42
|
+
ineligible_models = {}
|
|
43
|
+
Array(models).each do |model|
|
|
44
|
+
limits = ContextLimitResolver.resolve_details(model)
|
|
45
|
+
reserve = [limits.respond_to?(:output_limit) ? limits.output_limit.to_i : 0, OUTPUT_RESERVE].max
|
|
46
|
+
available = limits.context_limit - reserve
|
|
47
|
+
if total_tokens > available
|
|
48
|
+
ineligible_models[model] = available
|
|
49
|
+
else
|
|
50
|
+
eligible_models << model
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
if Array(models).any? && eligible_models.empty?
|
|
54
|
+
errors << "complete prompt ~#{total_tokens} tokens exceeds every configured model input allowance: #{ineligible_models.map { |model, allowance| "#{model}=#{allowance}" }.join(", ")}"
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
{success: errors.empty?, errors: errors, system_tokens: system_tokens,
|
|
58
|
+
user_tokens: user_tokens, total_tokens: total_tokens, input_limit: input_limit,
|
|
59
|
+
context_tokens: context_tokens, context_limit: context_limit,
|
|
60
|
+
instruction_tokens: all_instruction_tokens,
|
|
61
|
+
eligible_models: eligible_models, ineligible_models: ineligible_models}
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def self.conservative_estimate(text)
|
|
65
|
+
(TokenEstimator.estimate(text) * ESTIMATE_SAFETY_FACTOR).ceil
|
|
66
|
+
end
|
|
67
|
+
private_class_method :conservative_estimate
|
|
68
|
+
|
|
69
|
+
def self.integer_limit(settings, key, default)
|
|
70
|
+
value = settings[key] || settings[key.to_sym] || default
|
|
71
|
+
raise ArgumentError, "#{key} must be a positive integer" unless value.is_a?(Integer) && value.positive?
|
|
72
|
+
|
|
73
|
+
value
|
|
74
|
+
end
|
|
75
|
+
private_class_method :integer_limit
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
end
|
|
79
|
+
end
|
|
@@ -5,8 +5,9 @@ module Ace
|
|
|
5
5
|
module Atoms
|
|
6
6
|
# Pure function for estimating token counts from text
|
|
7
7
|
#
|
|
8
|
-
# Uses
|
|
9
|
-
#
|
|
8
|
+
# Uses chars/4 for ASCII and counts each non-ASCII UTF-8 byte as one
|
|
9
|
+
# potential token. The latter deliberately errs high for multilingual
|
|
10
|
+
# review packets rather than letting a Unicode-heavy prompt exceed its cap.
|
|
10
11
|
# This is intentionally simple and fast - actual tokenization would
|
|
11
12
|
# require model-specific tokenizers which adds complexity and latency.
|
|
12
13
|
#
|
|
@@ -34,7 +35,12 @@ module Ace
|
|
|
34
35
|
def self.estimate(text)
|
|
35
36
|
return 0 if text.nil? || text.empty?
|
|
36
37
|
|
|
37
|
-
(text.length.to_f / CHARS_PER_TOKEN).ceil
|
|
38
|
+
return (text.length.to_f / CHARS_PER_TOKEN).ceil if text.ascii_only?
|
|
39
|
+
return text.bytesize unless text.valid_encoding?
|
|
40
|
+
|
|
41
|
+
ascii_chars = text.each_codepoint.count { |codepoint| codepoint < 128 }
|
|
42
|
+
non_ascii_bytes = text.bytesize - ascii_chars
|
|
43
|
+
(ascii_chars.to_f / CHARS_PER_TOKEN).ceil + non_ascii_bytes
|
|
38
44
|
end
|
|
39
45
|
|
|
40
46
|
# Estimate token count from a file
|