ultra-builder-pro-cli 0.26.1 → 3.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.ultra-template/contexts/TEMPLATE.md +64 -29
- package/.ultra-template/north-star.md +36 -30
- package/.ultra-template/project-brief.md +28 -0
- package/.ultra-template/specs/product.md +4 -0
- package/.ultra-template/specs/research-distillate.md +8 -0
- package/.ultra-template/tasks.json +1 -0
- package/.ultra-template/test-report.json +18 -1
- package/CHANGELOG.md +227 -1
- package/README.md +161 -59
- package/adapters/_shared/host-profile.cjs +68 -12
- package/adapters/_shared/plugin-core.cjs +4 -5
- package/adapters/_shared/runtime-assets.cjs +11 -2
- package/adapters/zcode.js +375 -0
- package/bin/delegate-worker.cjs +20 -0
- package/bin/delegate.cjs +94 -25
- package/bin/install.js +11 -6
- package/docs/ARCHITECTURE.md +109 -25
- package/docs/ARTIFACT-AUTHORITY.md +204 -30
- package/docs/DECISIONS.md +141 -10
- package/docs/PHILOSOPHY.md +61 -8
- package/docs/PLUGIN-ISOLATION-CONTRACT.md +6 -0
- package/docs/RUNTIME-COMPAT-MATRIX.md +106 -13
- package/docs/SKILL-AUTHORING.md +14 -9
- package/docs/WORKFLOW-LIFECYCLE.md +211 -53
- package/docs/evals/adversarial-review-2026-08-14.md +84 -0
- package/docs/evals/zcode-automation-2026-08-14.md +103 -0
- package/hooks/README.md +31 -3
- package/hooks/_common.py +1699 -33
- package/hooks/adapters/zcode.py +227 -0
- package/hooks/compact_context.py +176 -17
- package/hooks/mid_workflow_recall.py +27 -5
- package/hooks/post_edit_guard.py +133 -23
- package/hooks/session_context.py +722 -11
- package/package.json +6 -3
- package/skills/ultra-change/SKILL.md +84 -64
- package/skills/ultra-change/references/change-contract.md +130 -32
- package/skills/ultra-change/references/execution-grant.md +109 -0
- package/skills/ultra-change/references/primary-transfer.md +273 -0
- package/skills/ultra-change/scripts/validate_primary_transfer.cjs +1928 -0
- package/skills/ultra-delegate/SKILL.md +38 -5
- package/skills/ultra-delegate/references/delegation-contract.md +21 -1
- package/skills/ultra-deliver/SKILL.md +123 -44
- package/skills/ultra-deliver/references/baseline-reconciliation.md +5 -3
- package/skills/ultra-dev/SKILL.md +173 -43
- package/skills/ultra-grilling/SKILL.md +19 -16
- package/skills/ultra-grilling/references/reframing.md +7 -3
- package/skills/ultra-init/SKILL.md +99 -87
- package/skills/ultra-init/references/brownfield-adoption.md +23 -40
- package/skills/ultra-init/scripts/init_project.cjs +968 -21
- package/skills/ultra-plan/SKILL.md +83 -46
- package/skills/ultra-plan/references/task-evidence-v2.md +300 -0
- package/skills/ultra-plan/scripts/validate_task_evidence.cjs +859 -0
- package/skills/ultra-research/SKILL.md +119 -68
- package/skills/ultra-research/references/{03-competitive-landscape.md → 03-alternatives.md} +6 -6
- package/skills/ultra-research/references/04-product-strategy.md +11 -2
- package/skills/ultra-research/references/21-features-scope.md +8 -0
- package/skills/ultra-research/references/22-success-metrics.md +9 -4
- package/skills/ultra-research/references/99-synthesis.md +25 -1
- package/skills/ultra-research/references/north-star-v2.md +168 -0
- package/skills/ultra-research/references/wayfinding.md +50 -0
- package/skills/ultra-research/scripts/validate_north_star.cjs +1174 -0
- package/skills/ultra-review/SKILL.md +201 -24
- package/skills/ultra-review/references/adversarial-evaluation.md +31 -0
- package/skills/ultra-review/references/spec.md +5 -0
- package/skills/ultra-review/references/unified-schema.md +146 -11
- package/skills/ultra-review/references/worker-packet.md +151 -8
- package/skills/ultra-review/scripts/review_wait.py +2653 -63
- package/skills/ultra-status/SKILL.md +146 -32
- package/skills/ultra-tdd/SKILL.md +14 -4
- package/skills/ultra-tdd/references/test-execution.md +6 -1
- package/skills/ultra-test/SKILL.md +157 -15
- package/skills/ultra-test/scripts/validate_review_transport.cjs +549 -0
- package/skills/ultra-test/scripts/worktree_digest.cjs +551 -29
- package/skills/ultra-think/SKILL.md +5 -10
- package/skills/ultra-think/references/interaction-boundary.md +2 -1
|
@@ -1,65 +1,179 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ultra-status
|
|
3
|
-
description: Infer the current Ultra
|
|
3
|
+
description: Infer the current Ultra route, baseline gaps, stale evidence, task risk, and installation health from repository files and host manifests. Use when the owner asks what is complete, blocked, stale, unhealthy, or useful to do next.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Route from artifacts without changing them
|
|
7
7
|
|
|
8
|
-
This router is read-only. It
|
|
9
|
-
evidence
|
|
8
|
+
This router is read-only. It separates verified files, model inference, and unavailable
|
|
9
|
+
evidence, then recommends one explicit public capability without invoking it. It never
|
|
10
|
+
persists a workflow-position field.
|
|
10
11
|
|
|
11
12
|
## Before you start
|
|
12
13
|
|
|
13
|
-
1. Read
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
14
|
+
1. Read `../ultra-change/references/change-contract.md` and apply its **Active Change
|
|
15
|
+
authority resolution** before reading any active `intent.md` or relying on a current
|
|
16
|
+
`change_id`. A positively observed absent `.ultra` uses the existing `ultra-init`
|
|
17
|
+
route. Otherwise continue only from the rule's stable zero-or-one result; any typed
|
|
18
|
+
diagnostic stops routing until its stated repair or retry succeeds. With one active
|
|
19
|
+
Change, read `.ultra/tasks.json`, then tasks whose `change_id` matches it and the
|
|
20
|
+
frontier task's `context_file` and `## Resume Note`. `.ultra/tasks.json` is the sole
|
|
21
|
+
task-status authority. A Status or Complexity header in a legacy context is a
|
|
22
|
+
migration diagnostic; the ledger wins. Historical and abandoned tasks never define
|
|
23
|
+
the frontier.
|
|
24
|
+
2. Read `.ultra/project-brief.md`, `.ultra/north-star.md`, `CONTEXT.md`, relevant
|
|
25
|
+
`.ultra/decisions/`, specifications, active, archived, or abandoned Changes, and
|
|
26
|
+
evidence. An abandoned Change supplies historical boundary and recovery context;
|
|
27
|
+
it never defines the current route.
|
|
28
|
+
3. Inspect Git `HEAD`, worktree status, and installed host manifests without repair.
|
|
29
|
+
4. When the active Change records a `durable work-package` Execution Grant, report it
|
|
30
|
+
as recorded authorization data — grant id, cited decision, and its recorded
|
|
31
|
+
subject, scope, topology, allowed effects, budgets, and invalidation — and state
|
|
32
|
+
plainly that stored grant text is inactive until a consuming Agent stably
|
|
33
|
+
verifies it per `../ultra-change/references/execution-grant.md`. Status itself
|
|
34
|
+
neither activates nor continues the grant; a `session-local` grant is reported as
|
|
35
|
+
requiring fresh owner activation.
|
|
18
36
|
|
|
19
37
|
## Definition of done
|
|
20
38
|
|
|
21
|
-
-
|
|
22
|
-
-
|
|
23
|
-
|
|
24
|
-
-
|
|
39
|
+
- Evaluate the artifact route in order and name the evidence supporting the match.
|
|
40
|
+
- Keep stale test evidence, dirty worktree state, baseline gaps, and task graph risks
|
|
41
|
+
visible without converting them into semantic failure.
|
|
42
|
+
- Report exact installation drift and a reachable repair without mutating it.
|
|
43
|
+
- Recommend one route and why; never launch it.
|
|
25
44
|
|
|
26
45
|
## Infer the route
|
|
27
46
|
|
|
28
|
-
| Observable files | Recommendation |
|
|
47
|
+
| Observable files and current need | Recommendation |
|
|
29
48
|
|---|---|
|
|
30
49
|
| No `.ultra/` | `ultra-init` |
|
|
31
|
-
|
|
|
50
|
+
| Project Brief has no usable one-line and no legacy `## One-line` exists | `ultra-init` |
|
|
51
|
+
| Project Brief or legacy seed exists and North Star status is `unresearched`, legacy, invalid, or not accepted for the current boundary | `ultra-research` |
|
|
32
52
|
| A new request exists and no active Change exists | `ultra-change` |
|
|
33
|
-
|
|
|
34
|
-
| A
|
|
35
|
-
|
|
|
36
|
-
| `
|
|
37
|
-
|
|
|
53
|
+
| More than one active Change exists | Diagnose the conflicting directories and ask the owner which one remains active; do not route or move either automatically |
|
|
54
|
+
| A primary-transfer handoff directory exists under `.ultra/.runtime/handoffs/` | Report its protocol state read-only (`offered`, `blocked`, `active`, or a terminal state) from `OFFER.json`/`ACK.json`/`RESULT.json`, verified with `node <ultra-change-skill-dir>/scripts/validate_primary_transfer.cjs <repository-root>`; receipts are derived observations, never authority |
|
|
55
|
+
| An ACK-ready handoff has no terminal RESULT | The accepted receiver is the sole canonical writer for that work package; report the in-flight transfer and never route canonical writes to another Agent |
|
|
56
|
+
| A handoff is `blocked` or its live re-verification is stale | Report the mismatched observation and the recovery path (fresh handoff id after owner re-confirmation); do not repair receipts |
|
|
57
|
+
| The active intent is `draft` or has a blocking unresolved decision | `ultra-change` to finish and accept the contract |
|
|
58
|
+
| The active `Research Disposition` names required exit evidence that is not satisfied | `ultra-research` for the named question and selected lenses |
|
|
59
|
+
| Research evidence exists but the accepted intent has not reconciled it | `ultra-change` to update the bounded contract |
|
|
60
|
+
| The accepted North Star revision or digest differs from the active intent trace | `ultra-change`; report the stale observation, preserve old evidence, and reconcile IDs before execution |
|
|
61
|
+
| An active Change exists and `.ultra/tasks.json` has no tasks whose `change_id` matches it | `ultra-plan` |
|
|
62
|
+
| The Change has exactly one `in_progress` task | `ultra-dev`, naming that task |
|
|
63
|
+
| Only `pending` tasks remain (the frontier) | Report them as frontier candidates with their unresolved dependencies and recommend one exact task id for an explicit owner invocation of `ultra-dev`; a pending row is not active work and never auto-selects a route |
|
|
64
|
+
| A ledger row says `completed` but its current `ultra-task-evidence-v2` record or task-review binding is missing, stale, or structurally invalid | `ultra-dev` to repair evidence and completion provenance; do not infer a context status or silently demote the row |
|
|
65
|
+
| All matching tasks are completed, each has current `ultra-task-evidence-v2` plus a consumed task review, and no current report matches this Change | `ultra-test` |
|
|
66
|
+
| A current false report has implementation, test, or wiring findings selected for repair | `ultra-dev` after owner disposition |
|
|
67
|
+
| A current false report has missing external or product evidence selected for collection | `ultra-research` after owner disposition |
|
|
68
|
+
| A current false report has an acceptance, scope, or existing-promise change selected for reconciliation | `ultra-change` after owner disposition; use `ultra-think` internally for one consequential trade-off |
|
|
69
|
+
| Every current finding is explicitly accepted, deferred, or otherwise dispositioned | `ultra-deliver`, subject to its one orphan-export gate |
|
|
70
|
+
| Report passed and all report identities match current files | `ultra-deliver` |
|
|
38
71
|
| No active Change and no new request | Report the idle state; do not manufacture work |
|
|
39
72
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
`
|
|
43
|
-
|
|
73
|
+
An explicit `[NEEDS CLARIFICATION]` is evidence of an unresolved field, not proof that
|
|
74
|
+
the entire project requires Research. Judge whether it can change the current boundary.
|
|
75
|
+
Do not add `research_complete`, a score, or another state bit to replace that judgment.
|
|
76
|
+
|
|
77
|
+
A test-report is stale when its `change_id` is not the active Change, its `task_ids` are not
|
|
78
|
+
the exact ordered ids of tasks whose `change_id` matches that Change, `git_commit`
|
|
79
|
+
differs from `HEAD`, or its `worktree.diff_digest` or `intent_digest` differs from
|
|
80
|
+
running
|
|
81
|
+
`node <ultra-test-skill-dir>/scripts/worktree_digest.cjs --project <repository-root> --change-id <change_id>`.
|
|
82
|
+
A dirty worktree is a warning, not evidence of failure.
|
|
83
|
+
The product-worktree digest has one fixed boundary: it excludes `.ultra/evidence/**`,
|
|
84
|
+
whose command and external-observation entries bind raw bytes as
|
|
85
|
+
`raw_evidence_sha256` before stable exact record bytes are bound as `evidence_digest`.
|
|
86
|
+
Status must therefore use the exact ordered `task_evidence` projection and strict receipt
|
|
87
|
+
consumers below for evidence provenance; it never adds another exclusion or treats
|
|
88
|
+
excluded publication bytes as absent evidence.
|
|
89
|
+
|
|
90
|
+
Freshness also requires the exact ordered `task_evidence` projection. Its
|
|
91
|
+
`evidence_digest`, `task_review_session`, and `task_review_summary_digest` must match the
|
|
92
|
+
current record and retained review — or, for an `external-manual` record, its
|
|
93
|
+
`task_review_receipt_ref` and `task_review_receipt_digest` must match the current
|
|
94
|
+
record and the bounded stable receipt bytes reverified through the sensor's
|
|
95
|
+
`--verify-external-receipt` mode, and take its `--projection` output verbatim.
|
|
96
|
+
Recheck whichever branch the record declares and never rewrite one branch into the
|
|
97
|
+
other. For each item, take every command and
|
|
98
|
+
external-observation `raw_evidence_ref` to bounded stable
|
|
99
|
+
repository-contained bytes from an ordinary regular non-symlink file opened nonblocking
|
|
100
|
+
and no-follow, with an 8 MiB ceiling and path/descriptor identity checks around the read.
|
|
101
|
+
Recompute `raw_evidence_sha256` from that one snapshot and require an exact match, then
|
|
102
|
+
recompute the stable exact `evidence.json` bytes as `evidence_digest` before comparing
|
|
103
|
+
the projection. A raw or record mismatch routes to the owning task's explicit Dev repair.
|
|
104
|
+
Run
|
|
105
|
+
`node <ultra-plan-skill-dir>/scripts/validate_task_evidence.cjs <evidence-ref>`, and
|
|
106
|
+
compare `task_id`, `schema`, `evidence_ref`, `task_review_session`, and
|
|
107
|
+
`task_review_summary_digest` with the current ledger row, final evidence record, and
|
|
108
|
+
retained strict task-review summary. The task evidence `subject` is an independently
|
|
109
|
+
captured completion-snapshot freshness observation made after validated task Review and
|
|
110
|
+
immediately before evidence publication. Its `task_review` separately binds the retained strict summary; neither Worker Packet v1 nor its nullable summary worktree digest proves
|
|
111
|
+
the subject. Do not require that earlier observation to equal the report's later
|
|
112
|
+
aggregate worktree digest. Recheck current Acceptance, criterion and verification type,
|
|
113
|
+
owner record, review summary, and cited affected artifacts independently.
|
|
114
|
+
|
|
115
|
+
Replay the report's strict aggregate review summary with the existing Review summary
|
|
116
|
+
waiter, then run
|
|
117
|
+
`node <ultra-test-skill-dir>/scripts/validate_review_transport.cjs --summary <summary-path> --report .ultra/test-report.json`.
|
|
118
|
+
Require the exact `packet_digest`, `admission_digest`, `subject_digest`, and `summary_digest` bindings, plus session, summary path, execution mode, ordered findings, and report root subject.
|
|
119
|
+
Each structural validator or sensor reports identity and provenance observations and never decides semantic acceptance, review quality, or `passed`.
|
|
120
|
+
Any mismatch makes the report stale and routes repair to the responsible workflow.
|
|
121
|
+
|
|
122
|
+
`passed: false` does not itself select a repair. Preserve each finding, ask for owner
|
|
123
|
+
disposition where scope or risk changes, then route by the accepted response above.
|
|
124
|
+
For a current task-review `REQUEST_CHANGES`, consume the exact validated current
|
|
125
|
+
`SUMMARY.json.findings` array and preserve every finding id. Route every current P0/P1
|
|
126
|
+
to resolution or authoritative disposition and refresh its affected evidence; never
|
|
127
|
+
reuse a finding count or list from an older summary. P2 and P3 findings remain report
|
|
128
|
+
entries or owner-selected backlog and never re-open a task on their own.
|
|
129
|
+
Likewise, ledger completion and structurally valid v2 evidence are necessary for a
|
|
130
|
+
current Test claim but never sufficient to manufacture `passed: true`. A validator
|
|
131
|
+
reports schema, identity, provenance, and freshness observations only. It cannot decide
|
|
132
|
+
whether an inspection supports a criterion, supply an `owner-judgment`, or turn an
|
|
133
|
+
external receipt into product truth.
|
|
134
|
+
|
|
135
|
+
Status remains read-only when later work invalidates a completed task. It names the
|
|
136
|
+
affected criterion IDs and reason and recommends Dev. Dev records both in the context
|
|
137
|
+
Change Log and Resume Note before the ledger-authoritative `completed` to `in_progress`
|
|
138
|
+
transition; Status never silently demotes a task.
|
|
139
|
+
|
|
140
|
+
A budget stop or `owner checkpoint` is displayed as-is: report the exhausted budget,
|
|
141
|
+
the in-progress task it stopped, and the reachable owner decision (extend, retry,
|
|
142
|
+
cancel, or abandon). Status never infers continuation, extension, or any semantic
|
|
143
|
+
verdict from a budget stop, and never treats a pending frontier row as the active
|
|
144
|
+
task behind one.
|
|
145
|
+
|
|
146
|
+
The repository has one primary writer for canonical `.ultra` files. Native review
|
|
147
|
+
workers and delegated CLIs may inspect or write isolated source roots, but their results
|
|
148
|
+
return to that writer. Concurrent canonical writes require separate worktrees and
|
|
149
|
+
explicit integration; this router does not invent locks or silently merge ledgers.
|
|
150
|
+
|
|
151
|
+
Report the accepted North Star revision and digest, every active Change trace mismatch,
|
|
152
|
+
and unresolved `FP-*`/`NS-*`/`HC-*` reference. A mismatch is stale evidence, never a
|
|
153
|
+
mechanical verdict about truth or permission; supersession does not delete prior task,
|
|
154
|
+
review, Test, or delivery evidence.
|
|
44
155
|
|
|
45
156
|
## Surface risk and installation health
|
|
46
157
|
|
|
47
|
-
List unmet task dependencies,
|
|
48
|
-
|
|
158
|
+
List unmet task dependencies, more than one matching `in_progress` task under the
|
|
159
|
+
single-writer contract, broken `trace_to` or `context_file` paths, stale subject or
|
|
160
|
+
acceptance identities, unresolved task-review findings, and concrete evidence that a
|
|
161
|
+
task can no longer resume reliably. A timestamp, complexity value, file count, or task
|
|
162
|
+
count is context for the model, never a staleness or quality verdict.
|
|
49
163
|
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
repair; this read-only skill never performs it.
|
|
164
|
+
Compare the selected host's plugin path, skill inventory, version, provenance, and hook
|
|
165
|
+
manifest with installed files. Recommend `ubp install` or `ubp update` as the repair;
|
|
166
|
+
this read-only Skill never performs it.
|
|
54
167
|
|
|
55
168
|
## When the owner decides
|
|
56
169
|
|
|
57
|
-
The owner chooses the next workflow, risk disposition and any reinstall.
|
|
58
|
-
|
|
170
|
+
The owner chooses the next public workflow, risk disposition, and any reinstall.
|
|
171
|
+
Contradictory facts remain visible instead of being resolved by routing heuristics.
|
|
59
172
|
|
|
60
173
|
## References
|
|
61
174
|
|
|
62
|
-
- `../ultra-
|
|
175
|
+
- `../ultra-change/references/change-contract.md` — canonical Active Change authority resolution.
|
|
176
|
+
- `../ultra-think/SKILL.md` — read only when contradictory evidence creates one real
|
|
63
177
|
decision rather than a mechanical repair.
|
|
64
178
|
- `../ultra-think/references/autonomy-boundary.md` — read before recommending a route
|
|
65
179
|
that would reduce an accepted commitment.
|
|
@@ -10,8 +10,17 @@ the loop that supplies it, and it serves Goal 3, Production-Ready.
|
|
|
10
10
|
|
|
11
11
|
## Before you start
|
|
12
12
|
|
|
13
|
-
1.
|
|
14
|
-
|
|
13
|
+
1. When called from a planned Change, read
|
|
14
|
+
`../ultra-change/references/change-contract.md` and apply its **Active Change
|
|
15
|
+
authority resolution** before reading any active `intent.md`, resolving the current
|
|
16
|
+
`change_id` or task, reading its context, running the baseline, or executing red-green
|
|
17
|
+
product edits. A stable zero result stops with no current task; any typed diagnostic,
|
|
18
|
+
including non-unique authority, stops with its repair before the baseline, red-green,
|
|
19
|
+
or any product write. After one valid authority, read its intent, `.ultra/tasks.json`,
|
|
20
|
+
and the matching current task context for the seams confirmed during planning. For a
|
|
21
|
+
micro edit outside the Ultra lifecycle, no active task context is required; use the
|
|
22
|
+
accepted request, maintained specification, and an existing repository seam, and
|
|
23
|
+
create no Ultra task or delivery claim.
|
|
15
24
|
2. Run the repository's existing test command once, unchanged, to see what
|
|
16
25
|
passing currently looks like. Read `references/test-execution.md` before recording
|
|
17
26
|
that run or any red/green evidence.
|
|
@@ -40,8 +49,8 @@ Write the failing test first and run it, so that red is observed rather than
|
|
|
40
49
|
assumed. Then write the smallest code that turns it green. Predict no later
|
|
41
50
|
tests and add no speculative capability. Then take the next slice.
|
|
42
51
|
|
|
43
|
-
Refactoring is not part of this loop; `ultra-review` owns it
|
|
44
|
-
|
|
52
|
+
Refactoring is not part of this loop; `ultra-review` owns it when observed duplication,
|
|
53
|
+
coupling, or a boundary defect supplies evidence that restructuring is worthwhile.
|
|
45
54
|
|
|
46
55
|
## Three patterns that produce tests worth deleting
|
|
47
56
|
|
|
@@ -74,6 +83,7 @@ goes to the owner as one.
|
|
|
74
83
|
|
|
75
84
|
## References
|
|
76
85
|
|
|
86
|
+
- `../ultra-change/references/change-contract.md` — canonical Active Change authority resolution.
|
|
77
87
|
- `../ultra-think/references/autonomy-boundary.md` — read before replacing a real
|
|
78
88
|
dependency with a fake, or when a test stands between the work and green.
|
|
79
89
|
- `references/templates/README.md` — read when choosing which enabling template
|
|
@@ -11,8 +11,13 @@ For every baseline, red, and green run record:
|
|
|
11
11
|
- runner-reported pass, fail, skip, and coverage counts when available;
|
|
12
12
|
- first useful failure with tight source or test location;
|
|
13
13
|
- whether it reproduces and whether it is product, test, environment, or flaky evidence;
|
|
14
|
-
- raw output path under `.ultra/evidence/<task-id>/`
|
|
14
|
+
- raw output path under `.ultra/evidence/<task-id>/` and the lowercase SHA-256 of its
|
|
15
|
+
exact bounded stable bytes when the output matters.
|
|
15
16
|
|
|
16
17
|
Do not weaken flags, skip a relevant failure, edit test configuration to obtain green,
|
|
17
18
|
or claim success when the command did not finish with exit zero. A concise evidence
|
|
18
19
|
record may cite the raw log; it must not copy an entire transcript into task context.
|
|
20
|
+
Before citing one, require a repository-contained ordinary regular non-symlink file,
|
|
21
|
+
opened nonblocking and no-follow with an 8 MiB ceiling and unchanged path/descriptor
|
|
22
|
+
identity around the read. Hash that one snapshot; a missing, escaped, special, symlinked,
|
|
23
|
+
oversized, or replaced receipt is a typed evidence gap, not a successful run record.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ultra-test
|
|
3
|
-
description: Audit whole-system integrity after the
|
|
3
|
+
description: Audit whole-system integrity after tasks for the active Change are complete by finding anti-patterns, coverage gaps, broken wiring, E2E failures, performance risk, and security risk. Use when preparing delivery or when locally green tasks may not form a working product.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# NOT for running unit tests (that's `ultra-dev`). This audits whole-system integrity before delivery.
|
|
@@ -11,23 +11,58 @@ the owner decides what to fix or accept.
|
|
|
11
11
|
|
|
12
12
|
## Before you start
|
|
13
13
|
|
|
14
|
-
1.
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
14
|
+
1. If model-selected, verify the live execution grant in `../ultra-change/references/execution-grant.md` — a current session-local activation or a stably verified durable work-package grant; without either, stop.
|
|
15
|
+
2. Read `../ultra-change/references/change-contract.md` and apply its **Active Change
|
|
16
|
+
authority resolution** before reading any active `intent.md` or resolving a current
|
|
17
|
+
`change_id`. This workflow requires its one valid active authority; any zero result
|
|
18
|
+
or typed diagnostic stops before audit input is read until the stated repair or retry
|
|
19
|
+
succeeds. Read `.ultra/tasks.json` and only tasks whose `change_id` matches the active
|
|
20
|
+
Change. Every matching task must be `completed` in that sole task-status authority.
|
|
21
|
+
Read each task's `context_file` Acceptance, Completion, Task Review and closing
|
|
22
|
+
`## Resume Note`.
|
|
23
|
+
Historical tasks are not part of this audit snapshot. For every current Change task,
|
|
24
|
+
require a current `ultra-task-evidence-v2` record whose context, subject, acceptance,
|
|
25
|
+
and task-review bindings satisfy `../ultra-plan/references/task-evidence-v2.md`.
|
|
26
|
+
A legacy context Status or Complexity header and v1 evidence remain migration
|
|
27
|
+
diagnostics only; recommend Dev repair rather than treating them as current Test input.
|
|
28
|
+
3. Read `CONTEXT.md` for vocabulary and the relevant `.ultra/decisions/` entries.
|
|
29
|
+
4. Read the active Change intent, the task evidence directories and the current Git
|
|
18
30
|
`HEAD`; old output is evidence only for the commit it names.
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
31
|
+
Resolve Change and task `FP-*`, `NS-*`, and `HC-*` IDs against the accepted North
|
|
32
|
+
Star revision. A stale trace is an explicit finding requiring reconciliation, not a
|
|
33
|
+
silent rewrite or automatic failure verdict.
|
|
34
|
+
5. Resolve this loaded Skill's directory and run
|
|
35
|
+
`node <ultra-test-skill-dir>/scripts/worktree_digest.cjs --project <repository-root> --change-id <change_id>`
|
|
36
|
+
before and after the audit. It returns the stable Change id, exact intent SHA-256,
|
|
37
|
+
HEAD, and product-worktree digest. The product-worktree digest has one fixed boundary:
|
|
38
|
+
it excludes `.ultra/evidence/**`, whose command and external-observation entries bind
|
|
39
|
+
raw bytes as `raw_evidence_sha256` before stable exact record bytes are bound as
|
|
40
|
+
`evidence_digest`. It also excludes the report and
|
|
41
|
+
all active/archive/abandoned Change-directory metadata. Intent freshness is checked
|
|
42
|
+
separately, so publishing or refreshing evidence, writing delivery metadata, or moving
|
|
43
|
+
the same Change cannot self-invalidate its tested product snapshot. Exact task-evidence
|
|
44
|
+
and review-receipt digests remain mandatory through their existing consumers; this is
|
|
45
|
+
not a caller-configurable exclusion. This digest defines the aggregate Change subject
|
|
46
|
+
for the current whole Change; it is distinct from each task's earlier
|
|
47
|
+
completion-snapshot freshness observation.
|
|
23
48
|
|
|
24
49
|
## Definition of done
|
|
25
50
|
|
|
26
51
|
- All six audit areas have an evidence-backed result or an explicit material omission.
|
|
27
52
|
- Wiring lists each changed export and its non-test consumers; zero matches are visible
|
|
28
53
|
as orphan findings.
|
|
29
|
-
- `.ultra/test-report.json` records `
|
|
30
|
-
|
|
54
|
+
- `.ultra/test-report.json` records `change_id`, the exact current-Change `task_ids`,
|
|
55
|
+
`intent_digest`, `git_commit`, commands, findings, omissions and the owner's
|
|
56
|
+
disposition without rewriting a finding into a pass.
|
|
57
|
+
- Every current task's v2 evidence and retained task-review summary is consumed; blocking
|
|
58
|
+
findings, resolutions, disagreements, and refresh references remain attributable.
|
|
59
|
+
- One current aggregate Change review is referenced by session, packet digest, exact
|
|
60
|
+
admission and subject digests, execution mode, coverage, summary path, exact summary
|
|
61
|
+
SHA-256, and verdict.
|
|
62
|
+
- The report names `ultra-review-findings-v4` and copies every review finding unchanged,
|
|
63
|
+
including `north_star_trace.first_principles`, `.serves`, and `.touches`, so each
|
|
64
|
+
finding carries its resolving `FP-*`/`NS-*`/`HC-*` IDs and Deliver can
|
|
65
|
+
test evidence against outcome without copying the North Star prose.
|
|
31
66
|
|
|
32
67
|
## Audit six areas
|
|
33
68
|
|
|
@@ -53,6 +88,95 @@ file is how a report loses findings.
|
|
|
53
88
|
Read `references/export-syntax.md` before collecting exports. The table finds
|
|
54
89
|
candidates; repository conventions and real consumers decide what is public.
|
|
55
90
|
|
|
91
|
+
After the six areas, follow `../ultra-review/SKILL.md` for an aggregate Change review.
|
|
92
|
+
Read only its validated summary, preserve its findings, and carry the review session,
|
|
93
|
+
packet digest, admission digest, subject digest, execution mode, coverage refs, summary
|
|
94
|
+
ref, verdict, and limitations into the report. Record
|
|
95
|
+
`review.finding_schema: "ultra-review-findings-v4"`; copy each finding
|
|
96
|
+
object unchanged from the v4 summary, including `north_star_trace`. An incomplete review
|
|
97
|
+
stays an omission; it never becomes a silent pass. Historical v3 review is evidence only:
|
|
98
|
+
run a fresh v4 aggregate review before a current Test claim.
|
|
99
|
+
|
|
100
|
+
Before accepting copied review metadata or findings, replay the exact frozen
|
|
101
|
+
`SUMMARY.json` through the sibling `ultra-review/scripts/review_wait.py` v4 summary
|
|
102
|
+
validator. Strict v4 replay is the default and requires the same session's valid
|
|
103
|
+
`ADMISSION.json`; `--legacy-v4` is an explicit read mode only for immutable historical
|
|
104
|
+
pre-admission evidence and cannot support a current Test claim. Then bind its
|
|
105
|
+
`change_id`, ordered `task_ids`, `head`, `admission_digest`, `subject_digest`,
|
|
106
|
+
`context_digest`, and
|
|
107
|
+
honest nullable `worktree_digest` to the Test report's root identity and `review`
|
|
108
|
+
provenance fields. The transport sensor accepts only canonical
|
|
109
|
+
`.ultra/test-report.json` and the normalized repository-relative
|
|
110
|
+
`.ultra/reviews/<session>/SUMMARY.json` it names. Both use the same bounded stable
|
|
111
|
+
raw-byte snapshot rule: every managed path component below the real repository root is
|
|
112
|
+
non-symlink, the final entry is a repository-contained regular file, and an 8 MiB + 1
|
|
113
|
+
byte observation returns a typed oversize diagnostic before JSON parsing or hashing.
|
|
114
|
+
The final file is opened with `O_NONBLOCK` and `O_NOFOLLOW`; its descriptor is
|
|
115
|
+
immediately required to be a regular file with the inspected path identity. A type or
|
|
116
|
+
identity replacement returns the existing retryable snapshot diagnostic instead of
|
|
117
|
+
waiting on a special file.
|
|
118
|
+
The sensor uses one report snapshot for report parsing and subject comparisons, and one
|
|
119
|
+
summary snapshot for summary parsing, SHA-256, comparisons, and the waiter's stdin
|
|
120
|
+
validation. After writing those exact values and findings, run
|
|
121
|
+
`node <ultra-test-skill-dir>/scripts/validate_review_transport.cjs --summary <validated-summary-path> --report .ultra/test-report.json`.
|
|
122
|
+
Record that exact command, exit code, and evidence reference in `commands`. A nonzero
|
|
123
|
+
result is a typed transport observation: repair the report copy and rerun it; never drop,
|
|
124
|
+
merge, inject, or rewrite a finding to obtain a pass. This sensor compares exact summary
|
|
125
|
+
bytes, review metadata, and finding objects only. It does not reinterpret a finding or
|
|
126
|
+
decide the audit verdict.
|
|
127
|
+
|
|
128
|
+
Validate each task evidence record with the shipped v2 structural sensor before using
|
|
129
|
+
it. For every record whose `task_review.review_mode` is `external-manual`, rerun the
|
|
130
|
+
sensor with `--verify-external-receipt` and recheck the declared receipt branch — the
|
|
131
|
+
bounded stable receipt bytes, its recorded SHA-256, and its task/change/subject
|
|
132
|
+
identity — instead of replaying a strict summary; strict-v4 records keep their exact
|
|
133
|
+
waiter and transport replay. A task's declared branch is preserved as-is in the
|
|
134
|
+
ordered `task_evidence` projection, consuming each item from the sensor's
|
|
135
|
+
`--projection` output verbatim: an external-manual item carries `task_review_mode`,
|
|
136
|
+
`task_review_receipt_ref`, and `task_review_receipt_digest` where a strict item
|
|
137
|
+
carries its session and summary digest. Never rewrite one branch into the other. The CLI establishes only JSON shape, token syntax, and
|
|
138
|
+
authority designation; it does not prove current identity, provenance, freshness,
|
|
139
|
+
or semantic acceptance. Test
|
|
140
|
+
takes each command and external-observation `raw_evidence_ref` to bounded stable
|
|
141
|
+
repository-contained bytes from an ordinary regular non-symlink file opened nonblocking
|
|
142
|
+
and no-follow, with an 8 MiB ceiling and path/descriptor identity checks around the read.
|
|
143
|
+
Recompute `raw_evidence_sha256` from that one snapshot and require an exact match. Then
|
|
144
|
+
recompute the stable exact `evidence.json` bytes as `evidence_digest` before copying the
|
|
145
|
+
ordered `task_evidence` projection; a raw or record mismatch is an evidence gap returned
|
|
146
|
+
to Dev.
|
|
147
|
+
Test
|
|
148
|
+
takes stable bytes independently, then rechecks and recomputes the current Acceptance-section SHA-256, aligns exact criterion IDs and verification types with the current context, and aligns the task-review session and summary digest with the retained strict summary. For `owner-judgment`, require that the durable owner record exists with its cited statement and disposition readable before rechecking all cited affected artifacts.
|
|
149
|
+
The `subject` is an independently captured completion-snapshot freshness observation made after the final strict task Review validates and immediately before evidence publication; `task_review` separately binds the retained strict summary. Worker Packet v1 carries no task worktree digest and its summary records `worktree_digest: null`, so Test never claims that those review artifacts prove or cryptographically bind `subject.worktree_digest`. It also never replaces that earlier observation with the later aggregate Change digest.
|
|
150
|
+
The owner alone supplies an owner-judgment result. A later task may change the aggregate
|
|
151
|
+
worktree without invalidating an earlier task, but any changed acceptance, owner record,
|
|
152
|
+
review binding, or cited artifact that no longer supports its criterion is a mismatch.
|
|
153
|
+
Any mismatch is an evidence gap: return the affected task and criterion IDs to Dev, or
|
|
154
|
+
return a missing owner judgment to the owner. Test never edits task status. Dev records the reason and performs
|
|
155
|
+
the explicit ledger `completed` to `in_progress` reopen; no silent demotion is allowed.
|
|
156
|
+
Never turn structural sensor success into any of those consumer conclusions.
|
|
157
|
+
|
|
158
|
+
This is a cooperative-workspace drift sensor, not an operating-system isolation
|
|
159
|
+
primitive. Use one fixed closing protocol: one complete primary observation followed by
|
|
160
|
+
one terminal seal. A persistent mismatch observed inside or before that terminal seal
|
|
161
|
+
returns typed `ULTRA_SNAPSHOT_CHANGED_DURING_OBSERVATION` recovery instead of publishing
|
|
162
|
+
the tuple. A cooperative write after the completed terminal seal belongs to the next
|
|
163
|
+
consumer, which must perform a fresh recapture before using the tuple. Hostile writers or
|
|
164
|
+
interleavings require a native Host sandbox or an isolated worktree. Never add an
|
|
165
|
+
unbounded success-seeking replay.
|
|
166
|
+
|
|
167
|
+
Aggregate finding disposition is bounded: P2 and P3 findings are recorded as backlog or
|
|
168
|
+
residual risk and never automatically returned to Dev, and this audit writes
|
|
169
|
+
non-blocking refactor observations into the report instead of reopening a completed
|
|
170
|
+
task. An implementation P0/P1 found here returns to Dev as at most one new
|
|
171
|
+
owner-approved repair task; there is no unbounded Test ↔ Deliver bounce, and any
|
|
172
|
+
re-entry through Test after a report is consumed requires an explicit owner
|
|
173
|
+
disposition.
|
|
174
|
+
|
|
175
|
+
The receipt is non-authoritative derived evidence, but retain it until both Test and
|
|
176
|
+
Deliver have consumed the review. If it is missing, the current session and Test claim
|
|
177
|
+
are `INCOMPLETE`; start a fresh review session. Do not reconstruct it or use
|
|
178
|
+
`--legacy-v4` to bypass loss from a current strict session.
|
|
179
|
+
|
|
56
180
|
## Detect substantive stubs
|
|
57
181
|
|
|
58
182
|
Report an empty return with no IO, a log-only function, a handler that only prevents
|
|
@@ -61,14 +185,28 @@ the smallest real boundary or implementation that would make it observable.
|
|
|
61
185
|
|
|
62
186
|
## Write the report
|
|
63
187
|
|
|
64
|
-
Store the exact `
|
|
65
|
-
|
|
66
|
-
|
|
188
|
+
Store the exact `change_id`, ordered current-Change `task_ids`, `intent_digest`,
|
|
189
|
+
`git_commit`, timestamp, run count, commands and exit codes, the six area results,
|
|
190
|
+
findings with paths, verified seams, material omissions, residual risk, and `passed` as
|
|
191
|
+
the model's evidence-derived summary in the one canonical
|
|
67
192
|
`.ultra/test-report.json`. Each command records `command`, `exit_code`, and an
|
|
68
193
|
`evidence_ref`; each area records `status`, `evidence_refs`, and `omissions`. Copy the
|
|
69
|
-
script's `head`, `dirty`,
|
|
194
|
+
worktree script's `head`, `dirty`, `diff_digest`, and `intent_digest` into the report.
|
|
195
|
+
That worktree identity binds the current whole Change, while ordered `task_evidence`
|
|
196
|
+
items preserve each task's distinct completion-snapshot freshness observation and
|
|
197
|
+
separate task-review identity.
|
|
198
|
+
Copy the validated review summary's exact SHA-256 into `review.summary_digest`; the
|
|
199
|
+
transport validator recomputes it before the report can be current evidence. A stale
|
|
200
|
+
commit, product digest, intent digest, Change id, or task-id snapshot is labelled stale;
|
|
70
201
|
it is never reused as current proof.
|
|
71
202
|
|
|
203
|
+
`passed` is a one-way claim: `true` requires every current-Change task completed in the
|
|
204
|
+
ledger, current v2 task evidence and task-review provenance, and no required area left
|
|
205
|
+
`not_run`. Completed rows do not force `passed: true`; findings, omitted evidence, or a
|
|
206
|
+
failed command may keep it false, while an explicitly dispositioned gap remains visible
|
|
207
|
+
even if the model's evidence summary passes. Preserve the finding and the owner's
|
|
208
|
+
disposition as separate fields. No validator result is itself semantic acceptance.
|
|
209
|
+
|
|
72
210
|
Present the findings by consequence and recommend the highest-leverage response. The
|
|
73
211
|
owner may fix all, fix selected findings, accept recorded risk, or reduce scope through
|
|
74
212
|
the normal REDUCTION decision. Recommend the next capability; do not invoke it.
|
|
@@ -80,8 +218,12 @@ an observation, not permission to auto-fix, weaken a test or silently change sco
|
|
|
80
218
|
|
|
81
219
|
## References
|
|
82
220
|
|
|
221
|
+
- `../ultra-change/references/change-contract.md` — canonical Active Change authority resolution.
|
|
83
222
|
- `references/export-syntax.md` — read during Wiring Verification to find exported
|
|
84
223
|
symbol candidates in TypeScript/JavaScript, Python, Go, Rust and Java.
|
|
85
224
|
- `scripts/worktree_digest.cjs` — deterministic HEAD plus worktree evidence identity.
|
|
225
|
+
- `../ultra-review/SKILL.md` — read for aggregate Change review after the six-area audit.
|
|
226
|
+
- `../ultra-plan/references/task-evidence-v2.md` — canonical current task evidence and task-review contract.
|
|
227
|
+
- `../ultra-change/references/execution-grant.md` — read only for grant-activated continuation.
|
|
86
228
|
- `../ultra-think/references/autonomy-boundary.md` — read when a proposed response
|
|
87
229
|
would make an existing specification commitment stop holding.
|