@spences10/pi-harness 0.1.1 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +116 -175
- package/dist/assessment.d.ts +4 -12
- package/dist/assessment.js +19 -192
- package/dist/assessment.js.map +1 -1
- package/dist/enforcement/policy.d.ts +2 -0
- package/dist/enforcement/policy.js +47 -73
- package/dist/enforcement/policy.js.map +1 -1
- package/dist/extension.d.ts +1 -1
- package/dist/extension.js +178 -412
- package/dist/extension.js.map +1 -1
- package/dist/index.d.ts +5 -4
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/dist/prompt.d.ts +1 -1
- package/dist/prompt.js +1 -7
- package/dist/prompt.js.map +1 -1
- package/dist/runtime/baseline.d.ts +6 -0
- package/dist/runtime/baseline.js +145 -0
- package/dist/runtime/baseline.js.map +1 -0
- package/dist/runtime/index.d.ts +2 -1
- package/dist/runtime/index.js +72 -29
- package/dist/runtime/index.js.map +1 -1
- package/dist/runtime/outcome.d.ts +1 -2
- package/dist/runtime/outcome.js +21 -35
- package/dist/runtime/outcome.js.map +1 -1
- package/dist/runtime/renderers.js +34 -70
- package/dist/runtime/renderers.js.map +1 -1
- package/dist/runtime/verification.d.ts +15 -0
- package/dist/runtime/verification.js +97 -0
- package/dist/runtime/verification.js.map +1 -0
- package/dist/schema.d.ts +12 -2
- package/dist/schema.js +6 -8
- package/dist/schema.js.map +1 -1
- package/package.json +6 -6
- package/skills/create-harness/SKILL.md +19 -62
- package/skills/execute-harness/SKILL.md +27 -66
- package/skills/review-harness/SKILL.md +21 -54
|
@@ -1,71 +1,28 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: create-harness
|
|
3
3
|
# prettier-ignore
|
|
4
|
-
description: Use when
|
|
5
|
-
compatibility:
|
|
6
|
-
Requires my-pi or Pi with the pi-harness extension tools enabled.
|
|
4
|
+
description: Use when the user requests a task harness or a concrete execution contract adds value for risky work. Creates task scaffolding directly; assessment is optional.
|
|
5
|
+
compatibility: Requires Pi with the pi-harness extension enabled.
|
|
7
6
|
---
|
|
8
7
|
|
|
9
|
-
#
|
|
8
|
+
# Create a Task Harness
|
|
10
9
|
|
|
11
|
-
|
|
12
|
-
candidate
|
|
10
|
+
Use direct work for routine changes. Do not introduce a harness just
|
|
11
|
+
to satisfy process, or expand a candidate idea into a larger project.
|
|
13
12
|
|
|
14
|
-
|
|
13
|
+
1. Recover the user's task, repository conventions and acceptance
|
|
14
|
+
checks.
|
|
15
|
+
2. Call `harness_create` with the task and non-empty validation
|
|
16
|
+
commands. Add `allowed_paths` when a scope narrower than the
|
|
17
|
+
project is useful. Normal source and test development are allowed
|
|
18
|
+
by default.
|
|
19
|
+
3. Read the resulting contract and use the `execute-harness` skill.
|
|
15
20
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
side effects, and explicit user requests.
|
|
21
|
+
No separate assessment or approval roundtrip is required to create the
|
|
22
|
+
scaffold. If the need or architecture is genuinely uncertain, optional
|
|
23
|
+
`harness_assess` can record investigation and a recommendation without
|
|
24
|
+
locking tools. It is not a read-only security mode.
|
|
21
25
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
## Workflow
|
|
26
|
-
|
|
27
|
-
1. Call `harness_assess` before repository mutation. Call it by
|
|
28
|
-
itself.
|
|
29
|
-
2. Recover context with the remaining read-only tools.
|
|
30
|
-
3. Establish source of truth from concrete source, tests, docs,
|
|
31
|
-
history, and observed behavior.
|
|
32
|
-
4. Find existing Pi, package, project, or platform primitives before
|
|
33
|
-
proposing a new capability.
|
|
34
|
-
5. Challenge assumptions. Record rejected options and why the evidence
|
|
35
|
-
rejects them.
|
|
36
|
-
6. Define the smallest useful vertical slice or falsifiable
|
|
37
|
-
experiment.
|
|
38
|
-
7. Call `harness_assessment_submit` with one recommendation:
|
|
39
|
-
- `direct` for bounded work;
|
|
40
|
-
- `harness` when an execution contract adds value;
|
|
41
|
-
- `reject` when the work has not earned adoption.
|
|
42
|
-
8. For a harness recommendation, include explicit allowed paths and
|
|
43
|
-
validation commands in the proposed contract.
|
|
44
|
-
9. Wait for direct user approval. Do not infer approval from agent
|
|
45
|
-
text or continue implementation while approval is pending.
|
|
46
|
-
10. The extension creates and activates an approved harness
|
|
47
|
-
atomically. Run it with the `execute-harness` skill. Harness
|
|
48
|
-
approval does not authorize Factory; use Factory only after a
|
|
49
|
-
direct user request to evaluate it. Do not call `harness_create`
|
|
50
|
-
to bypass assessment.
|
|
51
|
-
|
|
52
|
-
## Scope changes
|
|
53
|
-
|
|
54
|
-
Use `harness_amend` for bounded changes inside an approved capability.
|
|
55
|
-
A new capability, architecture decision, or outer-policy expansion
|
|
56
|
-
requires another assessment.
|
|
57
|
-
|
|
58
|
-
## Rules
|
|
59
|
-
|
|
60
|
-
- Assessment state is session control state, not project source.
|
|
61
|
-
- Read-only mode forbids project edits, writes, commits, package
|
|
62
|
-
installation, delegation, and other mutating tools.
|
|
63
|
-
- A rejected capability is a valid outcome.
|
|
64
|
-
- Do not create a planner, task graph, or backlog from rejected
|
|
65
|
-
candidates.
|
|
66
|
-
- Do not implement during assessment.
|
|
67
|
-
|
|
68
|
-
## Output
|
|
69
|
-
|
|
70
|
-
Submit only the structured assessment record. The extension presents
|
|
71
|
-
it for user approval and handles the chosen exit.
|
|
26
|
+
Creation does not authorize commits, pushes, releases, destructive
|
|
27
|
+
operations or scope outside the user's request. A broader accepted
|
|
28
|
+
contract needs user authorization, not an executor's self-amendment.
|
|
@@ -1,74 +1,35 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: execute-harness
|
|
3
3
|
# prettier-ignore
|
|
4
|
-
description: Use when
|
|
5
|
-
compatibility:
|
|
6
|
-
Requires my-pi or Pi with the pi-harness extension tools enabled.
|
|
4
|
+
description: Use when executing an existing Pi task harness. Works inside accepted scope, records progress, and requests evidence-backed completion with normal fix-and-retry loops.
|
|
5
|
+
compatibility: Requires Pi with the pi-harness extension enabled.
|
|
7
6
|
---
|
|
8
7
|
|
|
9
|
-
# Execute Harness
|
|
8
|
+
# Execute a Task Harness
|
|
10
9
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
10
|
+
1. Read the harness contract and task. Recover project context.
|
|
11
|
+
2. Record progress with `harness_update`. Work directly unless the
|
|
12
|
+
user has authorized collaboration with existing Team Mode peers.
|
|
13
|
+
3. Implement inside the accepted maximum scope. Normal project test
|
|
14
|
+
development is allowed unless explicitly excluded. Do not weaken
|
|
15
|
+
tests or fake outputs to manufacture success.
|
|
16
|
+
4. Use `harness_amend` for strategy changes. Accepted checks cannot be
|
|
17
|
+
removed, and scaffold scope never grants a larger accepted
|
|
18
|
+
boundary.
|
|
19
|
+
5. Inspect and run checks as useful during development. Fix ordinary
|
|
20
|
+
failures and retry; no arbitrary single-fix escalation applies.
|
|
21
|
+
6. Request `harness_update` with `status: completed`. It reruns
|
|
22
|
+
accepted checks and drift review through normal tool hooks, stores
|
|
23
|
+
outputs, and only records completion for a stable passing artifact.
|
|
24
|
+
7. Read `OUTCOME.md` or `outcome.json`, inspect the diff and report
|
|
25
|
+
actual validation, unresolved risks and any review still needed.
|
|
15
26
|
|
|
16
|
-
|
|
27
|
+
A failed or interrupted completion attempt is not success. Fix the
|
|
28
|
+
cause or ask for genuinely missing authority. A changed artifact must
|
|
29
|
+
be checked again. Manual evidence strings are notes, not proof.
|
|
17
30
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
4. If the `team` tool is available and you are the team lead, create
|
|
24
|
-
or reuse a team, create/claim a task for the harness, and use one
|
|
25
|
-
`member_spawn` executor with `mutating: true` for file-editing
|
|
26
|
-
work. Default to the contract `cwd`; use `workspace_mode: worktree`
|
|
27
|
-
only when the user asks for isolation or the plan covers setup,
|
|
28
|
-
runtime state, merge, and cleanup ownership.
|
|
29
|
-
5. The teammate prompt must tell it to read `harness.json`,
|
|
30
|
-
`SYSTEM.md`, `TASK.md`, and `status.json`, execute only inside the
|
|
31
|
-
contract, run `validate.sh` and `review.sh`, update harness
|
|
32
|
-
evidence, and report changed files plus risks. If using a worktree,
|
|
33
|
-
set `HARNESS_CWD` to that worktree for validation and review.
|
|
34
|
-
6. Recover any remaining source-of-truth context needed to act safely.
|
|
35
|
-
7. Execute surgically inside `scaffold.allowed_paths`. Use
|
|
36
|
-
`harness_amend` for a bounded change inside the approved
|
|
37
|
-
capability. If user direction or source evidence introduces a new
|
|
38
|
-
capability, architecture decision, or outer-policy expansion, stop
|
|
39
|
-
execution and enter assessment instead of silently expanding the
|
|
40
|
-
harness. An approved harness recommendation may then amend the
|
|
41
|
-
active scaffold.
|
|
42
|
-
8. Record decisions, phase changes, and validation evidence with
|
|
43
|
-
`harness_update`, including team status and remaining risks when
|
|
44
|
-
known.
|
|
45
|
-
9. Run `<harness_dir>/validate.sh`.
|
|
46
|
-
10. Run `<harness_dir>/review.sh` from the execution workspace. If
|
|
47
|
-
using a linked worktree, run it from that worktree or set
|
|
48
|
-
`HARNESS_CWD` to the worktree path.
|
|
49
|
-
11. Review `<harness_dir>/OUTCOME.md` or `<harness_dir>/outcome.json`
|
|
50
|
-
for changed files, baseline ignored files, validation evidence,
|
|
51
|
-
team status, and risks.
|
|
52
|
-
12. Mark `completed` only when validation passes and no escalation
|
|
53
|
-
rule applies; otherwise mark `failed` with evidence. A terminal
|
|
54
|
-
status seals the run and keeps its guard active for the remainder
|
|
55
|
-
of the current turn. The extension deactivates it on the next
|
|
56
|
-
direct user turn or session startup.
|
|
57
|
-
|
|
58
|
-
## Rules
|
|
59
|
-
|
|
60
|
-
- Do not bypass the outer policy. Do not edit outside the active
|
|
61
|
-
scaffold without first recording an authorized amendment.
|
|
62
|
-
- Do not create a replacement harness merely to commit or push
|
|
63
|
-
already-reviewed changes or to escape a completed run's guard. The
|
|
64
|
-
harness is control state, not a worktree. Use `/harness clear` only
|
|
65
|
-
to abandon a non-terminal run.
|
|
66
|
-
- Do not weaken tests, fake outputs, or bypass validation.
|
|
67
|
-
- An enforcement block is a harness mismatch, not a platform-policy
|
|
68
|
-
refusal. Amend the scaffold when authorized; otherwise ask the user.
|
|
69
|
-
Mark failed only when the task cannot proceed.
|
|
70
|
-
- If validation fails after one focused fix attempt, escalate.
|
|
71
|
-
- If team mode is unavailable, run the harness directly in the current
|
|
72
|
-
session and say that no teammate was spawned.
|
|
73
|
-
- Keep the final report to changed files, validation evidence, team
|
|
74
|
-
status/results, and remaining risks.
|
|
31
|
+
The harness is temporary workflow state, not an OS sandbox, worktree,
|
|
32
|
+
worker supervisor or a substitute for independent review. Team Mode
|
|
33
|
+
cannot spawn workers. Commits and external effects still require their
|
|
34
|
+
own user authorization. Completed runs are sealed; do not create a new
|
|
35
|
+
harness merely to commit their reviewed changes.
|
|
@@ -1,59 +1,26 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: review-harness
|
|
3
3
|
# prettier-ignore
|
|
4
|
-
description: Use when reviewing
|
|
5
|
-
compatibility:
|
|
6
|
-
Requires my-pi or Pi with the pi-harness extension tools enabled.
|
|
4
|
+
description: Use when reviewing work performed through a Pi task harness. Checks accepted scope, executed validation records, final artifacts and remaining risks without trusting self-reported completion notes.
|
|
5
|
+
compatibility: Requires Pi with the pi-harness extension enabled.
|
|
7
6
|
---
|
|
8
7
|
|
|
9
|
-
# Review Harness
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
the
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
## Review Checks
|
|
29
|
-
|
|
30
|
-
- Did work stay inside the outer policy and active scaffold?
|
|
31
|
-
- Were scaffold changes authorized, versioned, and honestly recorded?
|
|
32
|
-
- Did any amendment attempt to weaken the outer trust boundary?
|
|
33
|
-
- Were tests changed when `allow_test_changes` is false?
|
|
34
|
-
- Were validators weakened, deleted, skipped, or bypassed?
|
|
35
|
-
- Were outputs faked or hardcoded to satisfy checks?
|
|
36
|
-
- Are status and evidence entries honest and complete?
|
|
37
|
-
- Does the outcome artifact include changed files, validation
|
|
38
|
-
evidence, team status, and remaining risks?
|
|
39
|
-
- Did the executor make unapproved architecture decisions?
|
|
40
|
-
|
|
41
|
-
## Output Shape
|
|
42
|
-
|
|
43
|
-
```md
|
|
44
|
-
## Verdict
|
|
45
|
-
|
|
46
|
-
Pass | Needs executor fix | Escalate
|
|
47
|
-
|
|
48
|
-
## Findings
|
|
49
|
-
|
|
50
|
-
- P0/P1/P2: <issue> — `<path>` — <required action>
|
|
51
|
-
|
|
52
|
-
## Validation Evidence
|
|
53
|
-
|
|
54
|
-
- `<command>` — pass/fail/not run — <evidence>
|
|
55
|
-
|
|
56
|
-
## Required Next Step
|
|
57
|
-
|
|
58
|
-
- <one action>
|
|
59
|
-
```
|
|
8
|
+
# Review Harness Work
|
|
9
|
+
|
|
10
|
+
1. Read `harness.json`, task, status, amendment history and outcome.
|
|
11
|
+
2. Inspect the actual diff in the execution workspace. Run `review.sh`
|
|
12
|
+
there or set `HARNESS_CWD` for a linked worktree.
|
|
13
|
+
3. Confirm the final changes respect accepted maximum scope and
|
|
14
|
+
explicit exclusions. Strategy amendments must not grant wider
|
|
15
|
+
authority.
|
|
16
|
+
4. Inspect executed validation records and referenced output, not just
|
|
17
|
+
manual evidence strings. Check that accepted validators were
|
|
18
|
+
retained, failures were handled honestly, and the reviewed artifact
|
|
19
|
+
is current.
|
|
20
|
+
5. Check for weakened tests, fake outputs, missed user-facing
|
|
21
|
+
behaviour, and approvals still required for external effects.
|
|
22
|
+
|
|
23
|
+
Report pass, needs fix, or needs a user decision, with concrete
|
|
24
|
+
findings, validation run or reused, and remaining risks. A passing
|
|
25
|
+
runtime check is not an independent semantic review or an OS security
|
|
26
|
+
guarantee.
|