@yemi33/minions 0.1.2423 → 0.1.2425
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/install-internal-minions.js +1209 -0
- package/bin/minions.js +75 -9
- package/dashboard/docs/typography.md +27 -12
- package/dashboard/js/command-center.js +13 -11
- package/dashboard/js/live-stream.js +1 -1
- package/dashboard/js/memory-search.js +388 -43
- package/dashboard/js/modal-qa.js +5 -5
- package/dashboard/js/qa.js +39 -26
- package/dashboard/js/refresh.js +37 -18
- package/dashboard/js/render-agents.js +14 -18
- package/dashboard/js/render-dispatch.js +14 -74
- package/dashboard/js/render-inbox.js +3 -3
- package/dashboard/js/render-meetings.js +8 -8
- package/dashboard/js/render-other.js +45 -20
- package/dashboard/js/render-pipelines.js +6 -6
- package/dashboard/js/render-plans.js +14 -14
- package/dashboard/js/render-prd.js +46 -46
- package/dashboard/js/render-prs.js +9 -65
- package/dashboard/js/render-schedules.js +5 -5
- package/dashboard/js/render-skills.js +6 -6
- package/dashboard/js/render-utils.js +4 -4
- package/dashboard/js/render-watches.js +4 -16
- package/dashboard/js/render-work-items.js +95 -107
- package/dashboard/js/settings.js +175 -74
- package/dashboard/layout.html +3 -2
- package/dashboard/pages/inbox.html +1 -1
- package/dashboard/pages/work.html +1 -1
- package/dashboard/shared/model-display.js +9 -0
- package/dashboard/shared/watches-source.js +43 -0
- package/dashboard/shared/welcome-popup.js +259 -0
- package/dashboard/slim/body.html +6 -6
- package/dashboard/slim/js/chat.js +4 -2
- package/dashboard/slim/js/history.js +10 -3
- package/dashboard/slim/js/modals-tiles.js +85 -16
- package/dashboard/slim/js/status.js +12 -16
- package/dashboard/slim/styles.css +24 -4
- package/dashboard/styles.css +62 -27
- package/dashboard-build.js +8 -2
- package/dashboard.js +3854 -2514
- package/docs/README.md +6 -4
- package/docs/api-errors.md +144 -0
- package/docs/auto-discovery.md +83 -61
- package/docs/capture-demos.js +173 -30
- package/docs/command-center.md +5 -1
- package/docs/completion-reports.md +127 -11
- package/docs/constants.md +15 -3
- package/docs/constellation-style-telemetry.md +2 -2
- package/docs/copilot-cli-schema.md +32 -22
- package/docs/cross-repo-plans.md +30 -30
- package/docs/demo/memory-system.html +1 -1
- package/docs/deprecated.json +132 -18
- package/docs/diagnostics-memory.md +8 -4
- package/docs/human-vs-automated.md +2 -2
- package/docs/internal-install.md +190 -0
- package/docs/kb-pr3223-cascade-archiving.md +16 -0
- package/docs/kb-pr696-merge-conflict-docs.md +23 -0
- package/docs/kb-sweep.md +2 -2
- package/docs/keep-processes.md +8 -1
- package/docs/live-checkout-mode.md +12 -7
- package/docs/managed-spawn.md +21 -3
- package/docs/named-agents.md +3 -2
- package/docs/plan-lifecycle.md +32 -32
- package/docs/pr-auto-fix-dispatch.md +3 -3
- package/docs/pr-review-fix-loop.md +6 -6
- package/docs/pr-screenshots/pr-899/worker-pool-worktrees-AFTER.png +0 -0
- package/docs/pr-screenshots/pr-899/worker-pool-worktrees-BEFORE.png +0 -0
- package/docs/pr-screenshots/pr-979/auto-fix-pane-AFTER.png +0 -0
- package/docs/pr-screenshots/pr-979/auto-fix-pane-BEFORE.png +0 -0
- package/docs/pr-screenshots/pr-985/pr-column-em-dash-AFTER.png +0 -0
- package/docs/pr-screenshots/pr-985/pr-column-em-dash-BEFORE.png +0 -0
- package/docs/qa-runbook-lifecycle.md +67 -51
- package/docs/qa-runbooks.md +2 -2
- package/docs/runtime-adapters.md +101 -50
- package/docs/security.md +28 -0
- package/docs/self-improvement.md +47 -13
- package/docs/skills.md +29 -0
- package/docs/slim-ux/concepts.md +2 -2
- package/docs/specs/agent-configurability.md +43 -41
- package/docs/specs/agent-rename.md +13 -13
- package/docs/team-memory.md +283 -14
- package/docs/tutorials/03-plan-driven-feature.md +3 -3
- package/docs/tutorials/04-runtimes-and-harness.md +18 -21
- package/docs/tutorials/08-operations-and-recovery.md +3 -2
- package/docs/tutorials/README.md +4 -3
- package/docs/visual-evidence-ci.md +103 -0
- package/docs/watches.md +35 -30
- package/docs/worktree-lifecycle.md +151 -13
- package/engine/acp-transport.js +1 -0
- package/engine/ado.js +80 -33
- package/engine/agent-api-validation.js +571 -0
- package/engine/agent-worker-pool.js +7 -6
- package/engine/api-contracts/agent-content.js +761 -0
- package/engine/api-contracts/cc-ops.js +219 -0
- package/engine/api-contracts/config-runtime.js +419 -0
- package/engine/api-contracts/core.js +138 -0
- package/engine/api-contracts/index.js +506 -0
- package/engine/api-contracts/orchestration.js +602 -0
- package/engine/api-contracts/pull-requests.js +323 -0
- package/engine/api-contracts/qa-process.js +764 -0
- package/engine/api-contracts/work-plan-prd.js +494 -0
- package/engine/api-validation.js +543 -0
- package/engine/cc-api-validation.js +586 -0
- package/engine/cc-worker-pool.js +9 -1
- package/engine/cleanup.js +393 -271
- package/engine/cli.js +65 -7
- package/engine/comment-classifier.js +70 -5
- package/engine/consolidation.js +123 -5
- package/engine/content-api-validation.js +606 -0
- package/engine/create-pr-worktree.js +22 -1
- package/engine/db/migrations/027-review-learning-lifecycle.js +162 -0
- package/engine/dispatch.js +33 -0
- package/engine/distribution.js +189 -0
- package/engine/execution-model.js +59 -0
- package/engine/features.js +12 -0
- package/engine/github.js +67 -25
- package/engine/harness-context.js +313 -0
- package/engine/inbox-store.js +1 -1
- package/engine/kb-sweep.js +161 -38
- package/engine/keep-process-sweep.js +43 -3
- package/engine/lifecycle.js +1066 -165
- package/engine/llm.js +4 -0
- package/engine/managed-spawn.js +463 -48
- package/engine/meeting.js +197 -34
- package/engine/memory-retrieval.js +280 -0
- package/engine/memory-store.js +821 -38
- package/engine/model-discovery.js +17 -3
- package/engine/pipeline.js +802 -14
- package/engine/plan-prd-validation.js +745 -0
- package/engine/playbook.js +206 -17
- package/engine/pooled-agent-process.js +6 -1
- package/engine/pr-action.js +1 -1
- package/engine/pr-issue-validation.js +653 -0
- package/engine/pr-resolve.js +75 -13
- package/engine/prd-store.js +39 -6
- package/engine/preflight.js +7 -2
- package/engine/process-utils.js +79 -31
- package/engine/projects.js +198 -12
- package/engine/promotion.js +371 -0
- package/engine/qa-from-prd.js +3 -0
- package/engine/qa-process-validation.js +572 -0
- package/engine/qa-runbooks.js +41 -37
- package/engine/qa-runners/maestro.js +11 -7
- package/engine/qa-runners/playwright.js +37 -14
- package/engine/qa-runners.js +9 -10
- package/engine/qa-runs.js +124 -11
- package/engine/qa-sessions.js +139 -40
- package/engine/quarantine-refs.js +81 -11
- package/engine/queries.js +163 -9
- package/engine/review-learning-backfill.js +482 -0
- package/engine/review-learning.js +1236 -0
- package/engine/runtimes/claude.js +60 -2
- package/engine/runtimes/codex.js +57 -16
- package/engine/runtimes/contract.js +21 -0
- package/engine/runtimes/copilot.js +51 -3
- package/engine/runtimes/index.js +1 -0
- package/engine/scheduler.js +201 -32
- package/engine/settings-validation.js +909 -0
- package/engine/shared.js +705 -208
- package/engine/spawn-agent.js +48 -21
- package/engine/steering-constraints.js +31 -0
- package/engine/steering-store.js +10 -2
- package/engine/steering.js +14 -2
- package/engine/timeout.js +404 -111
- package/engine/untrusted-fence.js +24 -10
- package/engine/watch-actions.js +7 -2
- package/engine/watches.js +353 -51
- package/engine/work-item-validation.js +561 -0
- package/engine/work-items-store.js +61 -1
- package/engine/worktree-gc.js +656 -37
- package/engine/worktree-preflight.js +154 -0
- package/engine.js +2068 -342
- package/package.json +1 -1
- package/playbooks/_pr-description-audit.md +88 -41
- package/playbooks/build-fix-complex.md +3 -3
- package/playbooks/docs.md +2 -2
- package/playbooks/fix.md +18 -18
- package/playbooks/implement.md +18 -18
- package/playbooks/qa-session-draft.md +10 -12
- package/playbooks/qa-session-execute.md +16 -12
- package/playbooks/qa-validate.md +8 -7
- package/playbooks/shared-rules.md +45 -0
- package/playbooks/test.md +4 -0
- package/prompts/cc-system.md +1 -1
- package/skills/check-self-authored-review-comment/SKILL.md +2 -2
package/docs/team-memory.md
CHANGED
|
@@ -58,7 +58,7 @@ Authority is divided by responsibility:
|
|
|
58
58
|
|
|
59
59
|
| Store | Authoritative for | Notes |
|
|
60
60
|
|-------|-------------------|-------|
|
|
61
|
-
| `engine/state.db` tables `memory_records` and `
|
|
61
|
+
| `engine/state.db` tables `memory_records`, `memory_retrieval_runs`, and `memory_lifecycle_events` | Structured record bodies and metadata used by active retrieval; record lifecycle (`active`, `candidate`, `superseded`, `retracted`, `expired`); bounded retrieval and review-learning lifecycle telemetry | `memory_records_fts` is a derived external-content FTS5 index maintained by SQLite triggers. |
|
|
62
62
|
| `pinned.md` | Operator-authored pinned context | Read directly for every rendered agent prompt. It is not a structured-memory pin. |
|
|
63
63
|
| `notes/inbox/*.md` | Pending human, agent, and engine-authored findings awaiting consolidation | Successful agent tasks write one note to the assigned path; failures do not. |
|
|
64
64
|
| `notes.md` | Operator-readable team digest and legacy team-notes prompt input | Consolidation appends here under a file lock. |
|
|
@@ -71,6 +71,23 @@ SQLite is the sole authority for runtime state generally; see
|
|
|
71
71
|
[design-state-storage.md](design-state-storage.md). The Markdown paths above
|
|
72
72
|
remain intentionally file-backed documents and coordination artifacts.
|
|
73
73
|
|
|
74
|
+
### Offline review-learning backfill
|
|
75
|
+
|
|
76
|
+
`minions memory backfill-review-learnings [--dry-run] [--limit <N>] [--project <p>]`
|
|
77
|
+
(module `engine/review-learning-backfill.js`) is a strict, offline recovery
|
|
78
|
+
command that reconstructs durable review-learning lessons from legacy artifacts
|
|
79
|
+
(trusted completion artifacts + their dispatch context, tracked PR records,
|
|
80
|
+
legacy feedback inbox/archive files, and migration-027 quarantined candidates).
|
|
81
|
+
It imports through the exact live stable-key + upsert path
|
|
82
|
+
(`reviewLearning.buildFindingKey` → `memoryStore.upsertReviewLearningLesson`), so
|
|
83
|
+
a `--dry-run` classifies identically to a real run and a second real run imports
|
|
84
|
+
zero duplicates. A record is imported only when project, canonical PR, author, a
|
|
85
|
+
discrete generalized rule, exact current-head resolution evidence, source
|
|
86
|
+
consistency, and clean security provenance already exist — it never asks an LLM
|
|
87
|
+
to infer historical rules. Everything else stays non-recallable with an explicit
|
|
88
|
+
skip reason (see `REASON_BUCKETS`). A real run refuses to run while the engine is
|
|
89
|
+
live (memory-record race); `--dry-run` is read-only and always allowed.
|
|
90
|
+
|
|
74
91
|
### Synchronization is directional, not a two-way mirror
|
|
75
92
|
|
|
76
93
|
The normal ingestion paths often write both layers, but they are not one
|
|
@@ -78,6 +95,9 @@ cross-store transaction:
|
|
|
78
95
|
|
|
79
96
|
- Migration 017 performed a one-time backfill from existing knowledge files,
|
|
80
97
|
team-note sections, and per-agent sections into SQL.
|
|
98
|
+
- Migration 027 extended those tables in place. It did not rebuild migration 017
|
|
99
|
+
records; exact engine-generated legacy review-feedback records were retained
|
|
100
|
+
with `status: candidate` and `metadata.legacyUnverified: true`.
|
|
81
101
|
- Inbox consolidation writes knowledge and per-agent files and then upserts
|
|
82
102
|
corresponding semantic records.
|
|
83
103
|
- `POST /api/knowledge` writes both the knowledge file and a human-trusted SQL
|
|
@@ -187,7 +207,7 @@ separate default-off summary pass can fold older sections into a fenced
|
|
|
187
207
|
concurrent append cannot be overwritten.
|
|
188
208
|
|
|
189
209
|
The heavier KB sweep is a separate process. `engine.autoConsolidateMemory`
|
|
190
|
-
controls its four-hour automatic cadence; it does not control inbox-to-notes
|
|
210
|
+
defaults on and controls its four-hour automatic cadence; it does not control inbox-to-notes
|
|
191
211
|
consolidation. See [kb-sweep.md](kb-sweep.md) for deduplication, TTL, rewrite,
|
|
192
212
|
and archive behavior.
|
|
193
213
|
|
|
@@ -208,7 +228,7 @@ Supported record values are:
|
|
|
208
228
|
| Type | `semantic`, `episodic`, `procedural` |
|
|
209
229
|
| Scope | `global`, `project:<key>`, `agent:<key>` |
|
|
210
230
|
| Trust | `human`, `system`, `agent`, `external` |
|
|
211
|
-
| Status | `active`, `superseded`, `retracted`, `expired` |
|
|
231
|
+
| Status | `active`, `candidate`, `superseded`, `retracted`, `expired` |
|
|
212
232
|
|
|
213
233
|
Built-in file ingestion creates semantic records. Completion capture creates
|
|
214
234
|
episodic records. The schema supports procedural records, but recall excludes
|
|
@@ -219,6 +239,110 @@ hash. Re-upserting identical content is idempotent. New content for the same
|
|
|
219
239
|
source type/reference automatically supersedes the previous active row unless
|
|
220
240
|
the caller supplies an explicit supersession.
|
|
221
241
|
|
|
242
|
+
### Review-learning lifecycle storage
|
|
243
|
+
|
|
244
|
+
Migration 027 adds the storage foundation for review-derived procedural memory:
|
|
245
|
+
|
|
246
|
+
- A stable finding key has the form
|
|
247
|
+
`<project>|<canonical-pr-id>|<host-namespaced-source-key>`, for example
|
|
248
|
+
`minions-opg|github:opg-microsoft/minions#42|github:review-comment:991`.
|
|
249
|
+
Project, PR, and source host must agree; case and area facets are normalized
|
|
250
|
+
before persistence.
|
|
251
|
+
- An unresolved finding is a project-scoped procedural `memory_records` row with
|
|
252
|
+
`source_type: review-learning`, system trust, and `status: candidate`.
|
|
253
|
+
Its body is generated by the engine and contains no reviewer body. Candidates
|
|
254
|
+
remain queryable for diagnostics via `listReviewLearningCandidates` or the
|
|
255
|
+
explicit `includeReviewLearning` opt-in, but review-learning records are
|
|
256
|
+
globally excluded from the generic FTS pack, so they are never recallable
|
|
257
|
+
through the standard `Relevant Memory` path.
|
|
258
|
+
- A verified `accepted-fixed` or `satisfied-elsewhere` disposition may create an
|
|
259
|
+
active procedural lesson. Replaying unchanged content returns the same record;
|
|
260
|
+
changed rule content supersedes every older active row for the stable finding
|
|
261
|
+
key, leaving at most one active version.
|
|
262
|
+
- `invalid` closes a candidate as `retracted`, `stale` closes it as `expired`,
|
|
263
|
+
and `deferred` or `out-of-scope-follow-up` keeps it in `candidate` with a
|
|
264
|
+
bounded state marker. Positive transitions preserve the candidate and point
|
|
265
|
+
`superseded_by` at the active lesson.
|
|
266
|
+
- Both PR comment pollers retain at most 64 prompt-source records under
|
|
267
|
+
`humanFeedback.commentSources`. GitHub keys are
|
|
268
|
+
`github:<issue|review>-comment:<id>`; ADO keys are
|
|
269
|
+
`ado:thread:<thread-id>:comment:<comment-id>`. Each record stores bounded
|
|
270
|
+
author/timestamps and a SHA-256 content hash, while the same exact key is
|
|
271
|
+
included in that comment's untrusted-input fence. Edits preserve the key and
|
|
272
|
+
replace the hash.
|
|
273
|
+
- Human-feedback fix dispatches snapshot that bounded source list in
|
|
274
|
+
`meta.commentSources`, including contexts coalesced during cooldown.
|
|
275
|
+
- When `review-learning-capture` is enabled, only a completed Minions
|
|
276
|
+
`changes-requested` verdict creates an unresolved candidate. Review identity,
|
|
277
|
+
project, canonical PR, configured author, reviewer, dispatch, source item,
|
|
278
|
+
and a source hash are persisted; the raw review note remains only in its
|
|
279
|
+
existing fenced prompt context. Approval and non-blocking completions create
|
|
280
|
+
no candidate.
|
|
281
|
+
- New review completions no longer create `feedback-<author>-from-...` inbox
|
|
282
|
+
notes. Residual generated files are backfilled to non-recallable legacy
|
|
283
|
+
candidates when provenance is sufficient, then archived before LLM, team
|
|
284
|
+
note, knowledge-base, or personal-memory consolidation.
|
|
285
|
+
|
|
286
|
+
`memory_retrieval_runs` now also stores `dispatch_id`, `retrieval_kind`,
|
|
287
|
+
`selected_ids`, and a structured `decision_trace`. `memory_lifecycle_events`
|
|
288
|
+
stores a unique deterministic `event_key`, event/memory/finding identity,
|
|
289
|
+
project, agent, normalized area, disposition, work-item/dispatch references,
|
|
290
|
+
compact metadata, and timestamp. Event replay is idempotent. Event pruning
|
|
291
|
+
removes only old event rows and never deletes a memory record.
|
|
292
|
+
|
|
293
|
+
The boundary in `engine/review-learning.js` redacts secret-shaped values and
|
|
294
|
+
enforces these UTF-8/count limits:
|
|
295
|
+
|
|
296
|
+
| Value | Limit |
|
|
297
|
+
|-------|-------|
|
|
298
|
+
| Candidate or lesson body | 4,096 bytes |
|
|
299
|
+
| Generalized rules | 8 entries, 1,024 bytes each |
|
|
300
|
+
| Metadata | 4,096 bytes, 32 keys/items per container, depth 3 |
|
|
301
|
+
| Retrieval query | 2,048 bytes |
|
|
302
|
+
| Selected IDs | 32 entries / 4,096 bytes |
|
|
303
|
+
| Decision trace | 64 structured entries / 8,192 bytes |
|
|
304
|
+
| Applicable review-lesson recall | 4 records / 4,096 bytes per dispatch |
|
|
305
|
+
| Lifecycle event key | 64 bytes |
|
|
306
|
+
| PR comment source list / author | 64 entries / 160 bytes |
|
|
307
|
+
| Retained retrieval runs / lifecycle events | 5,000 / 50,000 rows |
|
|
308
|
+
| Candidate expiry / lifecycle-event max age | 180 days / 1 year |
|
|
309
|
+
|
|
310
|
+
Raw prompt, comment/reviewer-body, transcript, message, stdout/stderr, reasoning,
|
|
311
|
+
and chain-of-thought fields are rejected. Telemetry accepts IDs, hashes, bounded
|
|
312
|
+
evidence references, scores, dispositions, and reason codes instead of narrative
|
|
313
|
+
execution traces.
|
|
314
|
+
|
|
315
|
+
Rollout counters group by project and normalized area. The denominator is the
|
|
316
|
+
distinct finding keys with a `created` event and a positive disposition; the
|
|
317
|
+
numerator is distinct keys with `repeated-finding`. Aggregates also report the
|
|
318
|
+
distinct `retrieved`, `applied`, and `retracted` finding-key counts. Aggregates
|
|
319
|
+
return `insufficient-sample` below the default minimum of 20 created findings
|
|
320
|
+
rather than claiming a rate improvement.
|
|
321
|
+
|
|
322
|
+
### Diagnostics and retention (P-1ed2fb5f)
|
|
323
|
+
|
|
324
|
+
Operator diagnostics endpoints layer on top of the same SQL store, exposing only
|
|
325
|
+
IDs, hashes, reason codes, and bounded traces (never raw prompt/reviewer bodies):
|
|
326
|
+
|
|
327
|
+
- `GET /api/memory-records/search` accepts optional validated `project`,
|
|
328
|
+
`author`, `audience`, `sourceType`, and `findingKey` filters. Malformed values
|
|
329
|
+
return an actionable `400`; a `project` filter narrows scope to that project
|
|
330
|
+
plus global so results never cross into another project.
|
|
331
|
+
- `GET /api/memory-records/retrieval-decision?dispatchId=…|workItemId=…` returns
|
|
332
|
+
the latest bounded retrieval decision (why each lesson was selected/excluded)
|
|
333
|
+
by stable id and reason code. One of the two anchor ids is required.
|
|
334
|
+
- `GET /api/memory-records/lifecycle` returns a bounded (≤200) lifecycle timeline
|
|
335
|
+
plus per-event-type counters for a finding/memory/project.
|
|
336
|
+
- `GET /api/memory-records/rollout` returns the bounded rollout aggregates above.
|
|
337
|
+
|
|
338
|
+
Retention is folded into the periodic `engine/cleanup.js` cadence and is
|
|
339
|
+
deliberately decoupled from KB file TTL: unresolved review-learning candidates
|
|
340
|
+
expire (`status: expired`) after 180 days; compact lifecycle events are pruned
|
|
341
|
+
past one year and above the 50,000-row cap; active procedural lessons persist
|
|
342
|
+
until superseded or retracted. Reversible `retract`/`restore` of a review-learning
|
|
343
|
+
record emits a `retracted`/`restored` lifecycle event for the operator timeline.
|
|
344
|
+
|
|
345
|
+
|
|
222
346
|
### Query construction and selection
|
|
223
347
|
|
|
224
348
|
`engine/memory-retrieval.js` builds a task query from the work-item title,
|
|
@@ -246,6 +370,39 @@ Every packed record includes:
|
|
|
246
370
|
That provenance is visible both in the `Relevant Memory` appendix and the
|
|
247
371
|
dashboard's structured-memory search.
|
|
248
372
|
|
|
373
|
+
### Reserved deterministic review-lesson recall
|
|
374
|
+
|
|
375
|
+
Review-learning records are recalled through a dedicated path that never depends
|
|
376
|
+
on an FTS hit (`engine/memory-retrieval.js#retrieveApplicableReviewLearnings`),
|
|
377
|
+
and they are **globally excluded** from the generic FTS pack
|
|
378
|
+
(`searchMemoryRecords` drops `source_type='review-learning'` unless a caller
|
|
379
|
+
passes `includeReviewLearning`, which only the operator dashboard search does).
|
|
380
|
+
|
|
381
|
+
When `features.review-learning-recall` is on and a dispatch has a resolved
|
|
382
|
+
project context, the path:
|
|
383
|
+
|
|
384
|
+
1. lists active project-scoped lessons straight from the review-learning index
|
|
385
|
+
(`listActiveReviewLearnings`), so a lexically unrelated later task still
|
|
386
|
+
recalls a lesson touching the same file or symbol;
|
|
387
|
+
2. applies the audience gate — `project-team` lessons reach any agent in the
|
|
388
|
+
project; `author` lessons require the matching author agent — with project
|
|
389
|
+
scoping preventing any cross-project leak;
|
|
390
|
+
3. derives normalized affected-file, symbol, and tag hints from the same task
|
|
391
|
+
context (`deriveReviewFacetHints`, mirroring the capture-side normalization);
|
|
392
|
+
4. ranks exact **symbol → file → tag** overlap first, then lexical relevance,
|
|
393
|
+
confidence, and recency, keeping only applicable lessons; and
|
|
394
|
+
5. injects at most **4 records or 4 KiB** (independent of the generic memory
|
|
395
|
+
budget) as a separately fenced `## Applicable Prior Review Lessons` section,
|
|
396
|
+
wrapped with `engine/untrusted-fence.js`.
|
|
397
|
+
|
|
398
|
+
Each run records bounded telemetry through `recordRetrievalRun` with
|
|
399
|
+
`retrieval_kind='review-learning'`, the dispatch and work-item ids, selected
|
|
400
|
+
record ids, per-record scores, and reason codes (`exact-symbol`, `exact-file`,
|
|
401
|
+
`exact-tag`, `lexical`, `not-applicable`, `record-cap`, `byte-cap`) — never any
|
|
402
|
+
prompt or comment body. With recall off (capture-only), the rendered prompt is
|
|
403
|
+
unchanged with respect to review-learning records, and projectless Command
|
|
404
|
+
Center / unrelated central work receive no section.
|
|
405
|
+
|
|
249
406
|
## Prompt assembly and fallback
|
|
250
407
|
|
|
251
408
|
`engine/playbook.js` renders the requested playbook, adds shared rules, then
|
|
@@ -255,7 +412,10 @@ appends memory context in this order:
|
|
|
255
412
|
2. Either **Relevant Memory** or the legacy pair:
|
|
256
413
|
- **Team Notes** from `notes.md`;
|
|
257
414
|
- **Personal Memory** from `knowledge/agents/<agentId>.md`.
|
|
258
|
-
3.
|
|
415
|
+
3. **Applicable Prior Review Lessons**, when `review-learning-recall` is on, the
|
|
416
|
+
dispatch has a project, and at least one applicable lesson exists (bounded and
|
|
417
|
+
fenced independently of the generic memory budget).
|
|
418
|
+
4. Later non-memory appendices such as propagated project instructions.
|
|
259
419
|
|
|
260
420
|
Pinned context is independent of retrieval and remains present in every mode.
|
|
261
421
|
|
|
@@ -338,25 +498,123 @@ completion result.
|
|
|
338
498
|
| Key | Default | Purpose |
|
|
339
499
|
|-----|---------|---------|
|
|
340
500
|
| `features.memoryRetrieval` | `true` | Enables structured retrieval. Feature resolution is environment override, then `config.features`, then registry default. |
|
|
501
|
+
| `features.review-learning-capture` | `false` | Temporary gate for capturing verified review-learning candidates and lessons. |
|
|
502
|
+
| `features.review-learning-recall` | `false` | Temporary independent gate for the dedicated review-learning recall path. |
|
|
341
503
|
| `engine.memoryRetrievalShadowMode` | `false` | Measure retrieval without replacing legacy prompt memory. |
|
|
342
504
|
| `engine.memoryRetrievalTopK` | `8` | Maximum selected records. |
|
|
343
505
|
| `engine.memoryRetrievalMaxBytes` | `8192` | UTF-8 byte budget for selected record blocks. |
|
|
344
506
|
| `engine.memoryRetrievalCandidateLimit` | `50` | FTS5 candidates before reranking. |
|
|
345
507
|
| `engine.memoryEpisodicCapture` | `false` | Store compact completion episodes. |
|
|
346
508
|
| `engine.inboxConsolidateThreshold` | `5` | Eligible inbox files required before consolidation starts. |
|
|
347
|
-
| `engine.autoConsolidateMemory` | `
|
|
509
|
+
| `engine.autoConsolidateMemory` | `true` | Automatically run the separate KB sweep every four hours. |
|
|
348
510
|
| `engine.agentMemoryMaxEntries` | `300` | Non-summary section window in a personal file. |
|
|
349
511
|
| `engine.agentMemorySummaryEnabled` | `false` | Enable LLM compression of older personal sections. |
|
|
350
512
|
| `engine.agentMemorySummaryThreshold` | `30` | Oldest-section batch size for a summary. |
|
|
351
513
|
| `engine.agentMemorySummaryDays` | `30` | Age trigger for summary eligibility. |
|
|
352
514
|
|
|
353
|
-
|
|
354
|
-
surface.
|
|
355
|
-
the
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
`MINIONS_FEATURE_MEMORYRETRIEVAL=1|0
|
|
515
|
+
Feature flags are exposed by the generic Dashboard **Settings > Features**
|
|
516
|
+
surface. The two review-learning flags expire on 2027-01-31 and default off, so
|
|
517
|
+
the migration alone changes no prompt. Settings also exposes shadow mode,
|
|
518
|
+
episodic capture, retrieval bounds, the inbox threshold, and automatic KB sweep.
|
|
519
|
+
The personal-file window and summary keys are configuration-only today.
|
|
520
|
+
|
|
521
|
+
Environment overrides are `MINIONS_FEATURE_MEMORYRETRIEVAL=1|0`,
|
|
522
|
+
`MINIONS_FEATURE_REVIEW_LEARNING_CAPTURE=1|0`, and
|
|
523
|
+
`MINIONS_FEATURE_REVIEW_LEARNING_RECALL=1|0`.
|
|
524
|
+
|
|
525
|
+
## Staged rollout and acceptance
|
|
526
|
+
|
|
527
|
+
Review-learning ships behind two independent, default-off flags so the schema,
|
|
528
|
+
capture, recall, promotion, contradiction, and diagnostics code can all land on
|
|
529
|
+
`main` without changing a single rendered prompt. The rollout is deliberately
|
|
530
|
+
staged and measured; do not skip stages.
|
|
531
|
+
|
|
532
|
+
### Stage 0 — flags off (baseline behaviour)
|
|
533
|
+
|
|
534
|
+
With both `review-learning-capture` and `review-learning-recall` off:
|
|
535
|
+
|
|
536
|
+
- No candidate or lesson is captured; a `changes-requested` review completion is
|
|
537
|
+
handled exactly as before.
|
|
538
|
+
- No `## Applicable Prior Review Lessons` section is ever rendered, so the review,
|
|
539
|
+
fix, re-review, memory, comment-poller, and dashboard flows are byte-for-byte
|
|
540
|
+
identical to the pre-feature engine.
|
|
541
|
+
- `processReviewLearningApplications` short-circuits (recall gate), so an agent's
|
|
542
|
+
`reviewLearningApplications` payload is ignored and no lesson is mutated.
|
|
543
|
+
- Any records that already exist in SQL (e.g. from a prior enabled window or the
|
|
544
|
+
offline backfill) stay fully auditable through the diagnostics endpoints; only
|
|
545
|
+
prompt injection is gated, never storage or inspection.
|
|
546
|
+
|
|
547
|
+
### Stage 1 — capture-only for 30 days
|
|
548
|
+
|
|
549
|
+
Enable `review-learning-capture` only (recall stays off). This is the measurement
|
|
550
|
+
baseline: verified findings become candidates and lessons and accrue `created`,
|
|
551
|
+
`retrieved`-less, and `repeated-finding` lifecycle events, but **no lesson is ever
|
|
552
|
+
injected into a prompt**. Capture-only changes no rendered prompt. Run this for at
|
|
553
|
+
least 30 days to collect a per-project, per-normalized-area baseline recurrence
|
|
554
|
+
rate (the share of resolved valid findings that recurred).
|
|
555
|
+
|
|
556
|
+
### Stage 2 — one-project recall canary
|
|
557
|
+
|
|
558
|
+
Enable `review-learning-recall` for a **single project** (per-project
|
|
559
|
+
`config.features` override or the `MINIONS_FEATURE_REVIEW_LEARNING_RECALL` env
|
|
560
|
+
override on that project's engine). Recall now injects at most 4 records / 4 KiB of
|
|
561
|
+
bounded, fenced lessons on dispatches in that project. Watch the canary for:
|
|
562
|
+
|
|
563
|
+
- contradiction alerts (`review-learning-contradiction` inbox notes) — an
|
|
564
|
+
evidence-backed contradiction quarantines the offending lesson automatically;
|
|
565
|
+
- the retrieval decision traces (`/api/memory-records/retrieval-decision`) to
|
|
566
|
+
confirm lessons are selected for the right file/symbol reasons; and
|
|
567
|
+
- the recurrence rate for the canary project's first 30 active days.
|
|
568
|
+
|
|
569
|
+
### Expansion criteria (the success gate)
|
|
570
|
+
|
|
571
|
+
`GET /api/memory-records/rollout-comparison`
|
|
572
|
+
(`memoryStore.getReviewLearningRolloutComparison`) compares the **30-day
|
|
573
|
+
capture-only baseline window** against the **first 30 days of active recall**,
|
|
574
|
+
grouped by project and normalized area. It only reports `status: "success"` for a
|
|
575
|
+
(project, area) when **both** windows clear the resolved-valid-finding floor
|
|
576
|
+
(`minimumSample`, default 20 — otherwise `status: "insufficient-sample"`) **and**
|
|
577
|
+
the repeated valid-finding rate fell by at least `minReduction` (default `0.25` =
|
|
578
|
+
25%). A zero baseline rate is reported as `no-improvement`, never a divide-by-zero
|
|
579
|
+
success. Do not broaden the recall canary to another project until its comparison
|
|
580
|
+
row reports `success`.
|
|
581
|
+
|
|
582
|
+
### Immediate prompt rollback
|
|
583
|
+
|
|
584
|
+
Recall is a pure prompt-injection layer, so rolling it back is instant and total:
|
|
585
|
+
|
|
586
|
+
1. Set `config.features["review-learning-recall"] = false` (or
|
|
587
|
+
`MINIONS_FEATURE_REVIEW_LEARNING_RECALL=0`). The very next dispatch renders no
|
|
588
|
+
`## Applicable Prior Review Lessons` section — disabling recall is a complete
|
|
589
|
+
prompt rollback.
|
|
590
|
+
2. Optionally set `config.features["review-learning-capture"] = false` to stop
|
|
591
|
+
accruing new candidates/lessons. Existing records remain in SQL and stay
|
|
592
|
+
auditable through the diagnostics endpoints; nothing is deleted.
|
|
593
|
+
|
|
594
|
+
No migration, restart-ordering, or data cleanup is required to roll back — the flags
|
|
595
|
+
gate prompt behaviour, not storage.
|
|
596
|
+
|
|
597
|
+
### Promotion into repository guidance
|
|
598
|
+
|
|
599
|
+
A verified, **repeatedly observed** lesson can be promoted (human-approved) into
|
|
600
|
+
guidance that vanilla runtimes discover natively (`engine/promotion.js`,
|
|
601
|
+
`POST /api/memory-records/<id>/promote`). Promotion is eligible only at confidence
|
|
602
|
+
≥ `0.8` and ≥ 2 distinct valid findings (the rule recurred at least once). It never
|
|
603
|
+
edits an operator checkout: every promotion queues a normal project-bound work item
|
|
604
|
+
that follows the standard branch → PR → review → merge lifecycle, landing the rule
|
|
605
|
+
in an allowlisted instruction file (`CLAUDE.md` / `AGENTS.md` at any depth, or the
|
|
606
|
+
GitHub Copilot instruction file) or a project skill. The guidance is only seen by
|
|
607
|
+
vanilla Copilot/Claude once that PR merges.
|
|
608
|
+
|
|
609
|
+
### Contradiction quarantine
|
|
610
|
+
|
|
611
|
+
When a recall-enabled dispatch reports a lesson `outcome: "contradicted"` with
|
|
612
|
+
exact current-code evidence — and only for a lesson its own trusted retrieval trace
|
|
613
|
+
actually showed it, in its own project — the engine quarantines that lesson through
|
|
614
|
+
the reversible `retracted` lifecycle (provenance retained) and raises a
|
|
615
|
+
`review-learning-contradiction` operator inbox alert. `applied` and
|
|
616
|
+
`considered-not-applicable` are task-local and never mutate the lesson. A
|
|
617
|
+
quarantined lesson stops being recalled but can be `restore`d by an operator.
|
|
360
618
|
|
|
361
619
|
## Dashboard and API surfaces
|
|
362
620
|
|
|
@@ -376,8 +634,12 @@ inspect each lifecycle status.
|
|
|
376
634
|
| `GET` | `/api/knowledge/<category>/<file>` | Read one KB file. |
|
|
377
635
|
| `POST` | `/api/knowledge/sweep` | Start the asynchronous KB sweep. |
|
|
378
636
|
| `GET` | `/api/knowledge/sweep/status` | Poll sweep status. |
|
|
379
|
-
| `GET` | `/api/memory-records/search?q=...&status=active&limit=20` | Search structured records with provenance. |
|
|
637
|
+
| `GET` | `/api/memory-records/search?q=...&status=active&limit=20` | Search structured records with provenance; optional `project`/`author`/`audience`/`sourceType`/`findingKey` filters. |
|
|
380
638
|
| `POST` | `/api/memory-records/<id>/action` | Apply `pin`, `retract`, or `restore`. |
|
|
639
|
+
| `GET` | `/api/memory-records/retrieval-decision?dispatchId=...` | Latest bounded retrieval decision (selection/exclusion reasons) for a dispatch or work item. |
|
|
640
|
+
| `GET` | `/api/memory-records/lifecycle?findingKey=...` | Bounded lifecycle timeline + event-type counters. |
|
|
641
|
+
| `GET` | `/api/memory-records/rollout?project=...` | Bounded rollout aggregates (created/retrieved/applied/retracted/repeated + recurrence rate). |
|
|
642
|
+
| `GET` | `/api/memory-records/rollout-comparison?baselineFrom=...&baselineTo=...&activeFrom=...&activeTo=...` | Staged-rollout success gate: capture-only baseline window vs active-recall window per project/area (see **Staged rollout and acceptance**). |
|
|
381
643
|
| `POST` | `/api/inbox/persist` | Copy an inbox excerpt into `notes.md`, then archive the source. |
|
|
382
644
|
| `POST` | `/api/inbox/promote-kb` | Move an inbox item into a selected KB category. |
|
|
383
645
|
|
|
@@ -460,8 +722,15 @@ Markdown tiers are automatically copied into the prompt.
|
|
|
460
722
|
classification, personal routing, pruning, reconciliation, and summary.
|
|
461
723
|
- [`engine/db/migrations/017-agent-memory.js`](../engine/db/migrations/017-agent-memory.js)
|
|
462
724
|
- schema and one-time Markdown backfill.
|
|
725
|
+
- [`engine/db/migrations/027-review-learning-lifecycle.js`](../engine/db/migrations/027-review-learning-lifecycle.js)
|
|
726
|
+
- in-place retrieval/lifecycle schema extension and exact legacy-feedback
|
|
727
|
+
quarantine.
|
|
728
|
+
- [`engine/review-learning.js`](../engine/review-learning.js) - dispositions,
|
|
729
|
+
stable finding/area identities, redaction, bounds, record validation, event
|
|
730
|
+
keys, and candidate transitions.
|
|
463
731
|
- [`engine/memory-store.js`](../engine/memory-store.js) - record validation,
|
|
464
|
-
lifecycle, FTS5 query,
|
|
732
|
+
lifecycle, FTS5 query, candidate/lesson helpers, bounded telemetry, and
|
|
733
|
+
rollout aggregates.
|
|
465
734
|
- [`engine/memory-retrieval.js`](../engine/memory-retrieval.js) - task query,
|
|
466
735
|
reranking, deduplication, provenance formatting, and byte packing.
|
|
467
736
|
- [`engine/playbook.js`](../engine/playbook.js) - pinned/retrieved/legacy prompt
|
|
@@ -14,9 +14,9 @@ tests. Keep implementation, tests, and documentation as separate items with
|
|
|
14
14
|
explicit dependencies.
|
|
15
15
|
```
|
|
16
16
|
|
|
17
|
-
The plan agent writes Markdown under `plans/`. The plan-to-PRD
|
|
18
|
-
|
|
19
|
-
relationships.
|
|
17
|
+
The plan agent writes Markdown under `plans/`. The plan-to-PRD agent writes a
|
|
18
|
+
one-shot result sidecar; the engine validates it and imports structured items
|
|
19
|
+
with acceptance criteria and `depends_on` relationships into SQLite.
|
|
20
20
|
|
|
21
21
|
You can also provide plan text from the CLI:
|
|
22
22
|
|
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# Tutorial 4: Choose Runtimes and
|
|
1
|
+
# Tutorial 4: Choose Runtimes and Use Native Project Tools
|
|
2
2
|
|
|
3
3
|
Minions can dispatch with Copilot CLI, Claude Code, or Codex. This tutorial
|
|
4
4
|
switches the fleet runtime, separates Command Center from agent settings, and
|
|
5
|
-
checks
|
|
5
|
+
checks the selected runtime's native skill, command, and MCP discovery surface.
|
|
6
6
|
|
|
7
7
|
## 1. Inspect the current fleet
|
|
8
8
|
|
|
@@ -40,38 +40,35 @@ Changing CC does not silently switch dispatched agents.
|
|
|
40
40
|
|
|
41
41
|
## 3. Add a project-local skill
|
|
42
42
|
|
|
43
|
-
|
|
43
|
+
Use the selected runtime's native project skill root:
|
|
44
44
|
|
|
45
45
|
```text
|
|
46
|
-
.claude/skills/<skill-name>/SKILL.md
|
|
46
|
+
Claude: .claude/skills/<skill-name>/SKILL.md
|
|
47
|
+
Copilot: .github/skills/<skill-name>/SKILL.md
|
|
48
|
+
Codex: .agents/skills/<skill-name>/SKILL.md
|
|
47
49
|
```
|
|
48
50
|
|
|
49
51
|
Describe when the skill applies and provide the exact existing project command
|
|
50
|
-
to run. Commit the skill
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
52
|
+
to run. Commit the skill so worktree-mode dispatches receive it. Copilot also
|
|
53
|
+
reads `.claude/skills` and `.agents/skills`, but Claude does not natively discover
|
|
54
|
+
`.agents/skills`. Uncommitted assets are visible only when the dispatch actually
|
|
55
|
+
runs in that checkout, such as live checkout mode.
|
|
54
56
|
|
|
55
|
-
## 4. Verify
|
|
57
|
+
## 4. Verify native discovery
|
|
56
58
|
|
|
57
59
|
```bash
|
|
58
60
|
minions doctor --harness
|
|
59
61
|
```
|
|
60
62
|
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
For a known-empty execution environment, enable `hermeticHarness` for an agent
|
|
66
|
-
or the fleet. This removes user asset directories and project harness
|
|
67
|
-
propagation; use it for reproducibility tests, not as an MCP process-launch
|
|
68
|
-
control.
|
|
63
|
+
The report inventories adapter-declared roots and whether they exist. It does
|
|
64
|
+
not inject assets or attest that the runtime loaded or used them. Dispatch a
|
|
65
|
+
small task that clearly matches the skill and inspect the runtime output for
|
|
66
|
+
the observable behavior you requested.
|
|
69
67
|
|
|
70
68
|
## Checkpoint
|
|
71
69
|
|
|
72
|
-
The selected runtime passes doctor,
|
|
73
|
-
|
|
70
|
+
The selected runtime passes doctor, the committed skill is present in the
|
|
71
|
+
dispatch checkout, and a matching task exhibits the skill's requested behavior.
|
|
74
72
|
|
|
75
73
|
Reference: [Runtime Adapters](../runtime-adapters.md),
|
|
76
|
-
[Harness
|
|
77
|
-
[Harness Transparency](../harness-transparency.md).
|
|
74
|
+
[Native Repository Harness](../harness-propagation.md).
|
|
@@ -12,8 +12,9 @@ minions sources
|
|
|
12
12
|
```
|
|
13
13
|
|
|
14
14
|
Use the dashboard **Engine** page for dispatch logs, metrics, managed processes,
|
|
15
|
-
and worktree state.
|
|
16
|
-
|
|
15
|
+
and worktree state. Runtime state is canonical in `engine/state.db`; remaining
|
|
16
|
+
JSON files have explicit configuration, definition, cache, artifact, or
|
|
17
|
+
coordination contracts and are not state mirrors.
|
|
17
18
|
|
|
18
19
|
## 2. Classify before acting
|
|
19
20
|
|
package/docs/tutorials/README.md
CHANGED
|
@@ -9,7 +9,7 @@ only describing configuration.
|
|
|
9
9
|
| [1. Install and connect a project](01-install-and-connect.md) | A healthy engine, dashboard, and linked repository | 15 min |
|
|
10
10
|
| [2. Ship your first task](02-first-task.md) | One scoped work item dispatched and tracked to completion | 15 min |
|
|
11
11
|
| [3. Build a feature from a plan](03-plan-driven-feature.md) | An approved PRD with dependency-aware implementation and verification | 25 min |
|
|
12
|
-
| [4. Choose runtimes and
|
|
12
|
+
| [4. Choose runtimes and use native project tools](04-runtimes-and-harness.md) | A configured fleet with verified native skill, command, and MCP discovery | 20 min |
|
|
13
13
|
| [5. Automate recurring work and follow-ups](05-schedules-and-watches.md) | A schedule and a condition-based watch | 25 min |
|
|
14
14
|
| [6. Keep a development service running](06-managed-services.md) | An engine-owned, health-checked development service | 25 min |
|
|
15
15
|
| [7. Coordinate a cross-repository change](07-cross-repo-plan.md) | One plan fanned out safely across multiple projects | 30 min |
|
|
@@ -21,8 +21,9 @@ only describing configuration.
|
|
|
21
21
|
open a real pull request.
|
|
22
22
|
- Keep the dashboard open while following CLI steps. It is the easiest place to
|
|
23
23
|
see routing, pending reasons, live output, and artifacts together.
|
|
24
|
-
- Treat
|
|
25
|
-
|
|
24
|
+
- Treat `engine/state.db` as the sole runtime-state store. Remaining JSON files
|
|
25
|
+
are explicit configuration, definitions, caches, artifacts, or coordination
|
|
26
|
+
sidecars — not runtime mirrors.
|
|
26
27
|
- Use the linked reference docs when you need the full schema or lifecycle.
|
|
27
28
|
Tutorials intentionally focus on a successful path.
|
|
28
29
|
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# PR visual evidence CI
|
|
2
|
+
|
|
3
|
+
The OPG GitHub workflow captures matched dashboard screenshots for pull requests that change visual dashboard inputs. It compares live base and head checkouts; it never compares a committed demo image with a live PR.
|
|
4
|
+
|
|
5
|
+
## Triggering
|
|
6
|
+
|
|
7
|
+
`Visual evidence capture` runs automatically when a PR changes:
|
|
8
|
+
|
|
9
|
+
- `dashboard/**`
|
|
10
|
+
- `dashboard-build.js`
|
|
11
|
+
- `docs/capture-demos.js`
|
|
12
|
+
- `test/seed-demo-data.js`
|
|
13
|
+
|
|
14
|
+
Add the `visual-evidence` label to opt in a visual change outside that list. Removing the label removes the opt-in. Backend, test-only, prose-only, and documentation-only PRs do not start browsers unless explicitly labeled.
|
|
15
|
+
|
|
16
|
+
## Capture contract
|
|
17
|
+
|
|
18
|
+
The capture workflow:
|
|
19
|
+
|
|
20
|
+
1. Resolves the exact PR, repository, base, and head tuple from the trusted `pull_request_target` event and records it with the workflow run ID and attempt. The request builder comes from the exact `github.workflow_sha` revision, so an older PR base that predates the controller records a `controller-not-on-base` opt-out instead of failing the plan.
|
|
21
|
+
2. Captures and uploads BEFORE in a base-only job.
|
|
22
|
+
3. Starts the separate head job only after BEFORE has been uploaded, then runs the untrusted dashboard in a capability-dropped container with a read-only root/source/controller and one writable marker-owned state mount. The trusted host owns the browser and AFTER output.
|
|
23
|
+
4. Seeds marker-owned state directories with the same deterministic demo fixture.
|
|
24
|
+
5. Starts each checkout on the same port and captures it with the base branch's canonical [`capture-demos.js`](capture-demos.js).
|
|
25
|
+
6. Uses Chromium at 1400x900, dark theme, `en-US`, UTC, reduced motion, and a fixed browser time.
|
|
26
|
+
7. Requires the same ten non-empty PNG names on both sides.
|
|
27
|
+
|
|
28
|
+
Startup, fixture, and capture logs are retained with the images. Partial output is never presented as a successful comparison: a failed AFTER side fails the run, and a failed BEFORE side degrades the run to an explicitly-labelled AFTER-only publication (see [Degraded AFTER-only runs](#degraded-after-only-runs)).
|
|
29
|
+
|
|
30
|
+
## Artifacts and PR summary
|
|
31
|
+
|
|
32
|
+
GitHub Actions stores five artifacts keyed by PR number and full head SHA:
|
|
33
|
+
|
|
34
|
+
- capture-request metadata
|
|
35
|
+
- BEFORE PNG set
|
|
36
|
+
- AFTER PNG set
|
|
37
|
+
- BEFORE logs and `result.json`
|
|
38
|
+
- AFTER logs and `result.json`
|
|
39
|
+
|
|
40
|
+
Artifacts expire after **14 days**. Each upload replaces its same-run name on a rerun, so a later attempt refreshes all five artifacts instead of colliding with the prior attempt. The trusted publisher creates or updates one `github-actions[bot]` comment with BEFORE/AFTER artifact links. All publisher jobs serialize globally through GitHub Actions' multi-entry concurrency queue before mutating PR comments, so a newly queued publisher does not replace an already-pending publication. Each publisher also checks the authoritative workflow-run order before changing comments. Older runs and attempts are suppressed even when a newer opt-out run left no summary. Results for an obsolete head are suppressed when a current result exists; an already-posted result is marked stale when its head no longer matches. If the current PR diff no longer requests visual evidence, the old bot summary is removed.
|
|
41
|
+
|
|
42
|
+
## Degraded AFTER-only runs
|
|
43
|
+
|
|
44
|
+
`capture-before` checks out `github.event.pull_request.base.sha` and runs the **base branch's** seed and capture code, not the pull request's. A defect already merged to `main` therefore breaks the BEFORE side of every PR opened against that base — including the PR that repairs the base, which could otherwise never show a green check.
|
|
45
|
+
|
|
46
|
+
The BEFORE side is consequently non-blocking:
|
|
47
|
+
|
|
48
|
+
- `capture-before` carries `continue-on-error: true` and is named `Capture BEFORE (base, non-blocking)`, the same convention `pr-tests.yml` uses for its non-blocking `perf-tests` job. The failure surfaces as a warning annotation plus a job summary that names the base SHA and states the run is AFTER-only; it does not present a blocking red check on the PR.
|
|
49
|
+
- `Upload BEFORE screenshots` uses `if-no-files-found: warn`, so a crashed capture that wrote zero PNGs is reported once by the capture step instead of raising a second hard error. `Upload BEFORE capture logs` deliberately stays `if-no-files-found: error` — the logs directory is always written and is what makes the failure diagnosable.
|
|
50
|
+
- `capture-after` is gated on `!cancelled() && needs.plan.outputs.capture == 'true'` instead of a successful `capture-before`, so the head-side capture still runs and uploads. The plan gate is re-asserted directly because it is no longer inherited through a green BEFORE job, and `!cancelled()` keeps a run superseded by `cancel-in-progress` from spending a full Docker and Playwright capture.
|
|
51
|
+
|
|
52
|
+
The publisher degrades in step. When the run concludes successfully and the BEFORE PNG set is the *only* missing artifact, `assessArtifacts` returns `status: 'after-only'` and the bot comment is titled **"Visual evidence: AFTER only (BEFORE capture failed)"**. It states that this is not a BEFORE/AFTER comparison, renders the BEFORE cell as `capture failed - no screenshots`, and links the BEFORE logs for diagnosis. Any other gap — a missing AFTER set, missing BEFORE logs, duplicate artifacts, or a genuinely failed run — still reports the full failure summary. A reviewer is never handed a half-capture presented as a matched pair.
|
|
53
|
+
|
|
54
|
+
Nothing about the security posture changes: the controller and base checkout still run from the trusted base ref with `persist-credentials: false`, `Verify exact base checkout` is unchanged, and untrusted head code gains no execution privileges.
|
|
55
|
+
|
|
56
|
+
## Security boundary
|
|
57
|
+
|
|
58
|
+
The workflow definition and capture-request builder run from the exact trusted `github.workflow_sha` revision through `pull_request_target`; detection and capture execution remain pinned to the PR's trusted base SHA. If that historical base does not contain the controller, detection suppresses both browser jobs while the trusted workflow revision still records the opt-out request artifact. Neither planning path loads controller code from the untrusted head. The workflow has only `contents: read` and `pull-requests: read`, checkout credentials are not persisted, and no write-capable token is available. BEFORE and AFTER use separate runners, and AFTER cannot start until the immutable BEFORE artifact has uploaded. The head dashboard runs without the Actions process environment, host process namespace, controller checkout, screenshot output, or Docker socket; its source/controller mounts and container root are read-only. The sole writable bind mount is marker-owned `/state`, and the container runs as the host UID:GID that owns it. Secret-shaped variables and GitHub Actions control-file variables are removed from host capture child processes.
|
|
59
|
+
|
|
60
|
+
Comment publication happens later in `visual-evidence-publish.yml`, triggered by `workflow_run`. That job checks out only the repository default branch and requires a `pull_request_target` run from the trusted base repository. It downloads and parses only the trusted plan job's capture-request JSON, then binds its repository, workflow, stable run ID, PR, base, head repository, base branch, and full head SHA to the live run before writing. A request retained from an earlier attempt is valid for a selective rerun of that same run ID, while metadata claiming a future attempt is rejected. The run's `head_sha` is the captured PR head; publication does not depend on the optional `workflow_run.pull_requests` association. Screenshot and log artifacts remain link-only and are never downloaded or executed by the publisher.
|
|
61
|
+
|
|
62
|
+
## Base-controller failure mode
|
|
63
|
+
|
|
64
|
+
Because the seeder (`test/seed-demo-data.js`) and Playwright controller (`docs/capture-demos.js`) always execute from the base checkout, a break in either one is only observable once it is on `main`: every subsequent BEFORE capture fails, for every PR, and no change on a PR branch can repair it — the PR's own copy is never executed. Both files are also visual paths, so even a PR that fixes them triggers the broken capture. Since `capture-before` is non-blocking, the visible symptom is that every PR degrades to an AFTER-only publication (see [Degraded AFTER-only runs](#degraded-after-only-runs)) until the fix lands. Recovery is to land the fix on `main` (admin bypass) and then push to the affected PRs so a fresh `pull_request_target` event picks up a base SHA that contains it. `test/unit/visual-evidence-ci.test.js` runs the seeder's full seeding body so this class of break fails PR Tests before it can reach `main`.
|
|
65
|
+
|
|
66
|
+
## Local reproduction
|
|
67
|
+
|
|
68
|
+
Start from a trusted checkout of the base branch, never the pull request/head checkout. Resolve the exact full base and head SHAs independently, then use two disposable worktrees and keep output outside both:
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
BASE_WORKTREE=../visual-base
|
|
72
|
+
HEAD_WORKTREE=../visual-head
|
|
73
|
+
OUTPUT_ROOT=../visual-output
|
|
74
|
+
git worktree add --detach "$BASE_WORKTREE" <full-base-sha>
|
|
75
|
+
git worktree add --detach "$HEAD_WORKTREE" <full-head-sha>
|
|
76
|
+
test "$(git -C "$BASE_WORKTREE" rev-parse HEAD)" = "<full-base-sha>"
|
|
77
|
+
test "$(git -C "$HEAD_WORKTREE" rev-parse HEAD)" = "<full-head-sha>"
|
|
78
|
+
npm --prefix "$BASE_WORKTREE" ci --ignore-scripts
|
|
79
|
+
npm --prefix "$BASE_WORKTREE" exec -- playwright install chromium
|
|
80
|
+
node "${BASE_WORKTREE}/.github/scripts/run-visual-evidence.js" \
|
|
81
|
+
--dashboard-runtime docker \
|
|
82
|
+
--controller-root "$BASE_WORKTREE" \
|
|
83
|
+
--base-root "$BASE_WORKTREE" \
|
|
84
|
+
--head-root "$HEAD_WORKTREE" \
|
|
85
|
+
--base-sha <full-base-sha> \
|
|
86
|
+
--head-sha <full-head-sha> \
|
|
87
|
+
--output-root "$OUTPUT_ROOT"
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
The PNGs land in `../visual-output/{before,after}` and logs in `../visual-output/logs`. Docker is the default dashboard runtime so neither dashboard checkout executes in the host Node process. The host still executes the runner, fixture seeder, and Playwright controller, all from the exact base worktree; the runner verifies that controller checkout against `--base-sha` before mutating output. `--dashboard-runtime process` is an explicit opt-in for code you trust. The output root must be new, empty, or carry the runner's `.minions-visual-evidence-output` ownership marker, and it must not overlap either checkout or the controller in either direction. Reruns clear only the known generated subdirectories beneath a valid marker-owned root. Remove the disposable worktrees with the normal `git worktree remove` command after inspection.
|
|
91
|
+
|
|
92
|
+
## Complementary in-description embeds (agent-authored)
|
|
93
|
+
|
|
94
|
+
The CI workflow above is one of **two** complementary visual-evidence surfaces; it is not the only one:
|
|
95
|
+
|
|
96
|
+
- **`Visual evidence capture` (this workflow)** — an automated, sandboxed BEFORE/AFTER comparison posted by `github-actions[bot]`. It is untouched by the agent flow and remains the trusted comparison of record.
|
|
97
|
+
- **Agent-embedded screenshots (additive, W-mrz1yc79)** — when a Minions dispatch makes a meaningful **visual/UI change** (dashboard pages, slim-ux, Settings, work-item/PR/plan views, badges, layout/CSS), the agent proactively captures screenshots and embeds them directly in the PR **description** (``), rather than relying only on the bot. Backend-only, engine-logic, text/prose-only, analysis, and docs changes get no screenshots.
|
|
98
|
+
|
|
99
|
+
The agent embed path does **not** replace or modify the CI workflow — the two coexist. Mechanics the agent uses (full recipe in `playbooks/_pr-description-audit.md` → "Screenshot handling"):
|
|
100
|
+
|
|
101
|
+
- **GitHub:** raw PNGs are saved under `agents/<id>/screenshots/` (never `git add`ed), published as assets on a dedicated per-PR prerelease (`gh release create pr-<num>-visual … --prerelease`, or `gh release upload … --clobber` on re-dispatch), and embedded via each asset's `browser_download_url`. This works on private repos and never commits PNGs to the source branch. The upload uses a per-command `GH_TOKEN` for the repo owner's account (`opg-microsoft` → `yemishin_microsoft`, `yemi33` → `yemi33`) and never runs `gh auth switch`.
|
|
102
|
+
- **Azure DevOps:** `PUT /pullRequests/<id>/attachments/<filename>` with the PNG bytes, then reference the returned URL.
|
|
103
|
+
- **Degrade gracefully:** if the dev server won't start, the route 404s, Playwright is unavailable, or the upload fails, the agent skips without failing the work item and records the reason under `meta.descriptionAudit`.
|