@fro.bot/systematic 3.2.2 → 3.2.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agents/document-review/adversarial-document-reviewer.md +5 -3
- package/agents/document-review/coherence-reviewer.md +6 -4
- package/agents/document-review/design-lens-reviewer.md +5 -3
- package/agents/document-review/feasibility-reviewer.md +5 -3
- package/agents/document-review/product-lens-reviewer.md +5 -3
- package/agents/document-review/scope-guardian-reviewer.md +5 -3
- package/agents/document-review/security-lens-reviewer.md +5 -3
- package/dist/pi.js +2840 -2415
- package/package.json +4 -4
- package/skills/document-review/SKILL.md +7 -6
- package/skills/document-review/references/findings-schema.json +5 -6
- package/skills/document-review/references/review-output-template.md +48 -34
- package/skills/document-review/references/subagent-template.md +11 -11
- package/skills/document-review/references/synthesis-and-presentation.md +3 -3
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@fro.bot/systematic",
|
|
3
|
-
"version": "3.2.
|
|
3
|
+
"version": "3.2.4",
|
|
4
4
|
"description": "Structured engineering workflows for OpenCode",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"homepage": "https://fro.bot/systematic",
|
|
@@ -90,8 +90,8 @@
|
|
|
90
90
|
"devDependencies": {
|
|
91
91
|
"@biomejs/biome": "2.5.4",
|
|
92
92
|
"@earendil-works/pi-coding-agent": "0.80.7",
|
|
93
|
-
"@opencode-ai/plugin": "1.18.
|
|
94
|
-
"@opencode-ai/sdk": "1.18.
|
|
93
|
+
"@opencode-ai/plugin": "1.18.3",
|
|
94
|
+
"@opencode-ai/sdk": "1.18.3",
|
|
95
95
|
"@semantic-release/exec": "7.1.0",
|
|
96
96
|
"@types/bun": "latest",
|
|
97
97
|
"@types/js-yaml": "4.0.9",
|
|
@@ -101,7 +101,7 @@
|
|
|
101
101
|
"conventional-changelog-conventionalcommits": "9.3.1",
|
|
102
102
|
"markdownlint-cli": "0.48.0",
|
|
103
103
|
"rimraf": "6.1.3",
|
|
104
|
-
"semantic-release": "25.0.
|
|
104
|
+
"semantic-release": "25.0.8",
|
|
105
105
|
"semantic-release-export-data": "1.2.0",
|
|
106
106
|
"typebox": "1.3.6",
|
|
107
107
|
"typescript": "6.0.3"
|
|
@@ -6,7 +6,7 @@ argument-hint: "[mode:headless] [path/to/document.md]"
|
|
|
6
6
|
|
|
7
7
|
# Document Review
|
|
8
8
|
|
|
9
|
-
Review requirements or plan documents through multi-persona analysis. Dispatches specialized reviewer agents in parallel,
|
|
9
|
+
Review requirements or plan documents through multi-persona analysis. Dispatches specialized reviewer agents in parallel, applies clear fixes, and presents proposed fixes and decisions for user input.
|
|
10
10
|
|
|
11
11
|
## Phase 0: Detect Mode
|
|
12
12
|
|
|
@@ -14,12 +14,13 @@ Check the skill arguments for `mode:headless`. Arguments may contain a document
|
|
|
14
14
|
|
|
15
15
|
If `mode:headless` is present, set **headless mode** for the rest of the workflow.
|
|
16
16
|
|
|
17
|
-
**Headless mode** changes the interaction model, not the classification boundaries. Document-review still applies the same judgment about what has one clear correct fix vs. what needs user judgment. The only difference is how
|
|
18
|
-
- `
|
|
19
|
-
- `
|
|
17
|
+
**Headless mode** changes the interaction model, not the classification boundaries. Document-review still applies the same judgment about what has one clear correct fix vs. what needs user judgment. The only difference is how findings are delivered:
|
|
18
|
+
- `safe_auto` fixes at confidence anchor `100` are applied silently (same as interactive)
|
|
19
|
+
- `gated_auto` proposed fixes and `manual` decisions are returned as structured text for the caller to handle -- no question prompts, no interactive approval
|
|
20
|
+
- Findings at confidence anchor `50` are returned as FYI observations only -- no interaction or action
|
|
20
21
|
- Phase 5 returns immediately with "Review complete" (no refine/complete question)
|
|
21
22
|
|
|
22
|
-
The caller receives
|
|
23
|
+
The caller receives applied fixes, proposed fixes, decisions, and FYI observations with their original classifications intact; proposed fixes and decisions remain for the caller to handle.
|
|
23
24
|
|
|
24
25
|
Callers invoke headless mode by including `mode:headless` in the skill arguments, e.g.:
|
|
25
26
|
```
|
|
@@ -133,7 +134,7 @@ Pass each agent the **full document** -- do not split into sections.
|
|
|
133
134
|
|
|
134
135
|
## Phases 3-5: Synthesis, Presentation, and Next Action
|
|
135
136
|
|
|
136
|
-
After all dispatched agents return, read `references/synthesis-and-presentation.md` for the synthesis pipeline (validate, gate, dedup, promote, resolve contradictions, route by autofix class),
|
|
137
|
+
After all dispatched agents return, read `references/synthesis-and-presentation.md` for the synthesis pipeline (validate, gate, dedup, promote, resolve contradictions, route by autofix class), fix application, finding presentation, and next-action menu. Do not load this file before agent dispatch completes.
|
|
137
138
|
|
|
138
139
|
---
|
|
139
140
|
|
|
@@ -45,8 +45,8 @@
|
|
|
45
45
|
},
|
|
46
46
|
"autofix_class": {
|
|
47
47
|
"type": "string",
|
|
48
|
-
"enum": ["
|
|
49
|
-
"description": "How this issue should be handled.
|
|
48
|
+
"enum": ["safe_auto", "gated_auto", "manual"],
|
|
49
|
+
"description": "How this issue should be handled. safe_auto = one mechanical, unambiguous fix suitable for silent application only at confidence 100; gated_auto = a codebase-pattern-resolved, factually incorrect, standard security or reliability, framework-native, or substantive mechanically implied fix that needs user confirmation; manual = multiple reasonable choices requiring user judgment."
|
|
50
50
|
},
|
|
51
51
|
"finding_type": {
|
|
52
52
|
"type": "string",
|
|
@@ -58,10 +58,9 @@
|
|
|
58
58
|
"description": "Concrete fix text. Omit or null if no good fix is obvious -- a bad suggestion is worse than none."
|
|
59
59
|
},
|
|
60
60
|
"confidence": {
|
|
61
|
-
"type": "
|
|
62
|
-
"
|
|
63
|
-
"
|
|
64
|
-
"maximum": 1.0
|
|
61
|
+
"type": "integer",
|
|
62
|
+
"enum": [0, 25, 50, 75, 100],
|
|
63
|
+
"description": "Reviewer confidence anchor: 0 = false positive or pre-existing issue; 25 = might be real but could not verify; 50 = verified real but nitpick, advisory, or not very important; 75 = double-checked, will hit in practice, and directly impacts correctness; 100 = evidence directly confirms and the issue will happen frequently."
|
|
65
64
|
},
|
|
66
65
|
"evidence": {
|
|
67
66
|
"type": "array",
|
|
@@ -12,56 +12,67 @@ Use this **exact format** when presenting synthesized review findings. Findings
|
|
|
12
12
|
**Document:** docs/plans/2026-03-15-feat-user-auth-plan.md
|
|
13
13
|
**Type:** plan
|
|
14
14
|
**Reviewers:** coherence, feasibility, security-lens, scope-guardian
|
|
15
|
-
- security-lens -- plan adds public API endpoint with auth flow
|
|
15
|
+
- security-lens -- plan adds a public API endpoint with an auth flow
|
|
16
16
|
- scope-guardian -- plan has 15 requirements across 3 priority levels
|
|
17
17
|
|
|
18
|
-
Applied
|
|
18
|
+
Applied 2 fixes. 4 items need attention (2 errors, 2 omissions). 1 FYI observation.
|
|
19
19
|
|
|
20
|
-
###
|
|
20
|
+
### Applied fixes
|
|
21
21
|
|
|
22
|
-
-
|
|
23
|
-
-
|
|
24
|
-
- Updated unit count from "6 units" to "7 units" to match listed units (coherence)
|
|
25
|
-
- Added "update API rate-limit config" step to Unit 4 -- implied by Unit 3's rate-limit introduction (feasibility)
|
|
26
|
-
- Added auth token refresh to test scenarios -- required by Unit 2's token expiry handling (security-lens)
|
|
22
|
+
- Corrected the stale cross-reference from Section 3.2 to Section 3.1 (coherence, confidence 100)
|
|
23
|
+
- Updated the unit count from 6 to 7 to match the listed units (feasibility, confidence 100)
|
|
27
24
|
|
|
28
25
|
### P0 -- Must Fix
|
|
29
26
|
|
|
30
27
|
#### Errors
|
|
31
28
|
|
|
29
|
+
**Decisions**
|
|
30
|
+
|
|
32
31
|
| # | Section | Issue | Reviewer | Confidence |
|
|
33
32
|
|---|---------|-------|----------|------------|
|
|
34
|
-
| 1 | Requirements Trace |
|
|
33
|
+
| 1 | Requirements Trace | The goal requires offline support, but the approach assumes persistent connectivity | coherence | 75 |
|
|
35
34
|
|
|
36
35
|
### P1 -- Should Fix
|
|
37
36
|
|
|
38
37
|
#### Errors
|
|
39
38
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
|
39
|
+
**Proposed fixes**
|
|
40
|
+
|
|
41
|
+
| # | Section | Issue | Suggested fix | Reviewer | Confidence |
|
|
42
|
+
|---|---------|-------|---------------|----------|------------|
|
|
43
|
+
| 2 | API Design | The plan claims Rails lacks native JSON error responses and proposes a custom serializer | Use Rails' existing `render json:` response path and remove the custom serializer step | feasibility | 75 |
|
|
43
44
|
|
|
44
45
|
#### Omissions
|
|
45
46
|
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
|
47
|
+
**Proposed fixes**
|
|
48
|
+
|
|
49
|
+
| # | Section | Issue | Suggested fix | Reviewer | Confidence |
|
|
50
|
+
|---|---------|-------|---------------|----------|------------|
|
|
51
|
+
| 3 | Implementation Unit 3 | The custom auth plan omits the existing Devise setup and migration path | Extend the existing Devise flow and document its migration path instead of introducing a parallel auth setup | feasibility | 100 |
|
|
49
52
|
|
|
50
53
|
### P2 -- Consider Fixing
|
|
51
54
|
|
|
52
55
|
#### Omissions
|
|
53
56
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
|
57
|
+
**Proposed fixes**
|
|
58
|
+
|
|
59
|
+
| # | Section | Issue | Suggested fix | Reviewer | Confidence |
|
|
60
|
+
|---|---------|-------|---------------|----------|------------|
|
|
61
|
+
| 4 | API Design | The public webhook endpoint has no rate limiting plan | Add the established request-throttling middleware to the webhook route and document its limit | security-lens | 75 |
|
|
62
|
+
|
|
63
|
+
### FYI observations
|
|
64
|
+
|
|
65
|
+
| # | Section | Observation | Reviewer | Confidence |
|
|
66
|
+
|---|---------|-------------|----------|------------|
|
|
67
|
+
| 5 | Error Handling | The plan does not say whether rate-limit responses include retry guidance | security-lens | 50 |
|
|
57
68
|
|
|
58
|
-
### Residual
|
|
69
|
+
### Residual concerns
|
|
59
70
|
|
|
60
71
|
| # | Concern | Source |
|
|
61
72
|
|---|---------|--------|
|
|
62
|
-
| 1 | Migration rollback strategy not addressed for Phase 2 data changes | feasibility |
|
|
73
|
+
| 1 | Migration rollback strategy is not addressed for Phase 2 data changes | feasibility |
|
|
63
74
|
|
|
64
|
-
### Deferred
|
|
75
|
+
### Deferred questions
|
|
65
76
|
|
|
66
77
|
| # | Question | Source |
|
|
67
78
|
|---|---------|--------|
|
|
@@ -69,21 +80,24 @@ Applied 5 auto-fixes. 4 findings to consider (2 errors, 2 omissions).
|
|
|
69
80
|
|
|
70
81
|
### Coverage
|
|
71
82
|
|
|
72
|
-
| Persona | Status | Findings |
|
|
73
|
-
|
|
74
|
-
| coherence | completed |
|
|
75
|
-
| feasibility | completed |
|
|
76
|
-
| security-lens | completed | 2 | 1 | 1 | 0 |
|
|
77
|
-
| scope-guardian | completed |
|
|
78
|
-
| product-lens | not activated | -- | -- | -- | -- |
|
|
79
|
-
| design-lens | not activated | -- | -- | -- | -- |
|
|
83
|
+
| Persona | Status | Findings | Fixes | Proposed fixes | Decisions | FYI observations | Residual |
|
|
84
|
+
|---------|--------|----------|-------|----------------|-----------|------------------|----------|
|
|
85
|
+
| coherence | completed | 2 | 1 | 0 | 1 | 0 | 0 |
|
|
86
|
+
| feasibility | completed | 3 | 1 | 2 | 0 | 0 | 1 |
|
|
87
|
+
| security-lens | completed | 2 | 0 | 1 | 0 | 1 | 0 |
|
|
88
|
+
| scope-guardian | completed | 0 | 0 | 0 | 0 | 0 | 1 |
|
|
89
|
+
| product-lens | not activated | -- | -- | -- | -- | -- | -- |
|
|
90
|
+
| design-lens | not activated | -- | -- | -- | -- | -- | -- |
|
|
80
91
|
```
|
|
81
92
|
|
|
82
93
|
## Section Rules
|
|
83
94
|
|
|
84
|
-
- **Summary line**: Always present after the reviewer list. Format: "Applied N
|
|
85
|
-
- **
|
|
95
|
+
- **Summary line**: Always present after the reviewer list. Format: "Applied N fixes. K items need attention (X errors, Y omissions). Z FYI observations." Omit any zero clause. `K` counts actionable proposed fixes and decisions; FYI observations are counted separately.
|
|
96
|
+
- **Applied fixes**: List all fixes applied silently (`safe_auto` at confidence `100`). Include enough detail per fix to convey the substance -- especially for fixes that add content or touch document meaning. A `safe_auto` finding at confidence `75` is demoted before routing and is not silently applied. Omit section if none.
|
|
86
97
|
- **P0-P3 sections**: Only include sections that have findings. Omit empty severity levels. Within each severity, separate into **Errors** and **Omissions** sub-headers. Omit a sub-header if that severity has none of that type.
|
|
87
|
-
- **
|
|
88
|
-
- **
|
|
89
|
-
- **
|
|
98
|
+
- **Proposed fixes**: Findings with `gated_auto` at confidence `75` or `100`. Include the concrete suggested fix and require user confirmation. Omit if none.
|
|
99
|
+
- **Decisions**: Findings with `manual` at confidence `75` or `100`. Include the suggested fix when one exists; otherwise present the judgment call without inventing a fix. Omit if none.
|
|
100
|
+
- **FYI observations**: Findings at confidence `50`, regardless of autofix class. They require no decision or action. Omit if none.
|
|
101
|
+
- **Residual concerns**: Unresolved residual risks that remain after restatement suppression, plus any residual items explicitly retained by synthesis. Omit if none.
|
|
102
|
+
- **Deferred questions**: Questions for later workflow stages. Omit if none.
|
|
103
|
+
- **Coverage**: Always include. All finding and route counts are **post-synthesis**. For each persona, **Findings** equals **Fixes + Proposed fixes + Decisions + FYI observations**. If deduplication merges a finding across personas, attribute it to the persona with the highest confidence anchor; if anchors tie, use document order, and reduce the other persona's finding and route counts. **Residual** remains the count of `residual_risks` from that persona's raw output, not the promoted or suppressed subset shown in Residual concerns. Failed or malformed reviewers are marked in Status and do not contribute findings.
|
|
@@ -26,18 +26,18 @@ Rules:
|
|
|
26
26
|
- Set `finding_type` for every finding:
|
|
27
27
|
- `error`: Something the document says that is wrong -- contradictions, incorrect statements, design tensions, incoherent tradeoffs.
|
|
28
28
|
- `omission`: Something the document forgot to say -- missing mechanical steps, absent list entries, undefined thresholds, forgotten cross-references.
|
|
29
|
+
- Set `confidence` to exactly one anchor from the schema, based on the evidence available:
|
|
30
|
+
- `0`: False positive or pre-existing issue. Suppress the finding.
|
|
31
|
+
- `25`: Might be real but could not verify. Suppress the finding.
|
|
32
|
+
- `50`: Verified real but nitpick, advisory, or not very important. This becomes an FYI observation.
|
|
33
|
+
- `75`: Double-checked, will hit in practice, and directly impacts correctness. This is actionable.
|
|
34
|
+
- `100`: Evidence directly confirms the issue and it will happen frequently. This is actionable and is the only anchor eligible for silent fixes.
|
|
29
35
|
- Set `autofix_class` based on whether there is one clear correct fix, not on severity or importance:
|
|
30
|
-
- `
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
- Missing standard security measures -- HTTPS enforcement, checksum verification, input sanitization, private IP rejection, or other controls with known implementations where omission is clearly a bug
|
|
36
|
-
- Incomplete technical descriptions -- the accurate/complete version is directly derivable from the codebase
|
|
37
|
-
- Missing requirements that follow mechanically from the document's own explicit, concrete decisions (not high-level goals -- a goal can be satisfied by multiple valid requirements)
|
|
38
|
-
The test is not "is this fix important?" but "is there more than one reasonable way to fix this?" If a competent implementer would arrive at the same fix independently, it is auto -- even if the fix is substantive. Always include `suggested_fix`. NOT auto if more than one reasonable fix exists or if scope/priority judgment is involved.
|
|
39
|
-
- `present`: Requires user judgment -- genuinely multiple valid approaches where the right choice depends on priorities, tradeoffs, or context the reviewer does not have. Examples: architectural choices with real tradeoffs, scope decisions, feature prioritization, UX design choices.
|
|
40
|
-
- `suggested_fix` is required for `auto` findings. For `present` findings, include only when the fix is obvious.
|
|
36
|
+
- `safe_auto`: A truly mechanical one-correct-fix case suitable for silent application only at anchor `100`: summary/detail mismatches, wrong counts, stale internal references, terminology drift, or additions mechanically implied by explicit content. Do not use this for codebase-pattern, factual, security/reliability, framework-native, or substantive completeness cases.
|
|
37
|
+
- `gated_auto`: A concrete fix resolved by an existing codebase pattern, factually incorrect behavior, a missing standard security or reliability control, a framework-native substitution, or a substantive mechanically implied completeness addition. The user confirms before applying it.
|
|
38
|
+
- `manual`: Multiple reasonable choices require user judgment, such as architectural tradeoffs, scope or priority decisions, feature prioritization, or UX choices.
|
|
39
|
+
The test is not "is this fix important?" but "is there more than one reasonable way to fix this?" If a competent implementer would arrive at the same fix independently, use `safe_auto` only for the truly mechanical cases above; use `gated_auto` when codebase or factual evidence resolves the choice but the fix is substantive. Do not classify a judgment call as automatic.
|
|
40
|
+
- `suggested_fix` is required for `safe_auto` and `gated_auto` findings. For `manual` findings, include it only when the fix is obvious.
|
|
41
41
|
- If you find no issues, return an empty findings array. Still populate residual_risks and deferred_questions if applicable.
|
|
42
42
|
- Use your suppress conditions. Do not flag issues that belong to other personas.
|
|
43
43
|
</output-contract>
|
|
@@ -9,14 +9,14 @@ Process findings from all agents through this pipeline. Order matters — each s
|
|
|
9
9
|
Check each agent's returned JSON against the findings schema:
|
|
10
10
|
|
|
11
11
|
- Drop findings missing any required field defined in the schema
|
|
12
|
-
- Drop findings with invalid enum values
|
|
12
|
+
- Drop findings with invalid enum values. Legacy `auto` / `present` values from older or in-flight dispatches are malformed; reject them without remapping rather than treating them as an alternate contract.
|
|
13
13
|
- Note the agent name for any malformed output in the Coverage section
|
|
14
14
|
|
|
15
15
|
**Do not narrate remap / validation diagnostics to the user.** Schema-drift notes ("persona X returned unknown enum Y, remapped to Z"), persona-prompt-drift commentary, and other validator-internal diagnostics are maintainer-facing information. They do not belong in the Phase 4 output the user reads. If a persona's output is malformed, the only user-visible consequence is a Coverage-row annotation (e.g., the persona shows fewer findings or a `malformed` marker). Everything else stays internal.
|
|
16
16
|
|
|
17
17
|
### 3.2 Confidence Gate (Anchor-Based)
|
|
18
18
|
|
|
19
|
-
Gate findings by their `confidence` anchor value. Anchors are discrete integers (`0`, `25`, `50`, `75`, `100`) with behavioral definitions documented in `references/findings-schema.json` and embedded in the persona rubric (`references/subagent-template.md`).
|
|
19
|
+
Gate findings by their `confidence` anchor value. Anchors are discrete integers (`0`, `25`, `50`, `75`, `100`) with behavioral definitions documented in `references/findings-schema.json` and embedded in the persona rubric (`references/subagent-template.md`). The anchors provide per-severity gates — doc-review economics do not warrant threshold gradation by severity, and coarse anchors prevent false-precision gaming. Continuous values from an older contract are malformed, not remapped.
|
|
20
20
|
|
|
21
21
|
| Anchor | Meaning | Route |
|
|
22
22
|
|--------|---------|-------|
|
|
@@ -174,7 +174,7 @@ Do NOT reclassify, re-route, or change the confidence anchor of any finding in t
|
|
|
174
174
|
- P1 manual "Migration lacks rollback strategy" — migration needs rollback regardless of scope. NOT linked (independence safeguard).
|
|
175
175
|
- P0 gated_auto "Deployment-ordering between migration and code" — concrete fix user confirms regardless. NOT linked (safeguard: gated_auto with own resolution path).
|
|
176
176
|
|
|
177
|
-
Result: 1 root + 4 dependents. User sees the root first; rejecting it cascades the 4 dependents
|
|
177
|
+
Result: 1 root + 4 dependents. User sees the root first; rejecting it cascades the 4 dependents out of the active decision set. Manual engagement drops from 11 → 7 (6 unlinked + 1 visible root).
|
|
178
178
|
|
|
179
179
|
**Worked example B (auth-shape).** Review of a plan to introduce a new session-management middleware. One finding is P1 manual "Middleware rewrite premise unsupported — existing session handling has no reported reliability issues" in Problem Frame. Scanning the other findings:
|
|
180
180
|
|