@hassangameryt/freeflow 0.4.0 → 0.6.0

package/skills/design-for-depth/references/design-pressure-signals.md

@@ -1,6 +1,6 @@
 # Design Pressure Signals
 
-Use this reference when reviewing code, artifacts, or work for shallow modules and complexity spread.
+Read this when evidence suggests caller knowledge or coordination is spreading and the next route must be classified.
 
 The goal is not to find every smell. The goal is to notice pressure that changes the next route.
 

package/skills/design-for-depth/references/interface-design-loop.md

@@ -1,6 +1,6 @@
 # Interface Design Loop
 
-Use this when structural pressure makes the current interface suspect. The goal is to choose an outcome-level module shape, not add detail to the first plausible design.
+Read this before comparing materially different interfaces when structural pressure or an important pre-implementation correctness boundary makes their ownership consequential, especially when evidence cannot choose or authority, canonical state, atomic visibility, replay, cancellation, or post-commit recovery affects correctness. The goal is to choose an outcome-level module shape, not add detail to the first plausible design.
 
 ## 1. Frame The Outcome
 

package/skills/design-for-depth/references/software-design-philosophy.md

@@ -1,6 +1,6 @@
 # Software Design Philosophy
 
-Use this reference when `design-for-depth` needs deeper reasoning than the active skill can carry.
+Read this when structural pressure is supported but the likely-changing decision or missing information-hiding boundary cannot yet be named.
 
 This is not a checklist to run every time. Pull the idea that changes the next route.
 

package/skills/diagnose-failure/SKILL.md

@@ -11,17 +11,23 @@ A **diagnostic loop** is a repeatable check or observation that can reproduce, s
 
 A requested patch, plausible code path, reviewer theory, or one favorable rerun is not a supported cause.
 
+Diagnosis inherits the current authority envelope. Selecting this method authorizes neither active evidence generation nor correction.
+
+When diagnosis applies, before choosing its loop, read [the Diagnostic Loop Catalog](references/diagnostic-loop-catalog.md). When timing, randomness, environment variance, or resource behavior shapes the failure, also read [Flaky and Performance Diagnosis](references/flaky-and-performance-diagnosis.md) before choosing or running the observation.
+
 ## Follow The Diagnostic Route
 
 ```text
 [Failure without a supported cause]
--> [Establish expected and observed behavior]
--> [Choose the smallest diagnostic loop]
--> [Reproduce or capture the failure]
+-> [Inspect existing evidence; establish expected and observed behavior]
 -> [State a falsifiable hypothesis]
--> [Run one distinguishing observation]
+-> [Choose the smallest distinguishing observation]
+-> [Authority envelope covers its effects?]
+   -> no -> [Propose purpose, expected evidence, and stop condition]
+         -> [Ask once and wait]
+   -> yes -> [Run the observation]
 -> [Route from evidence]
-   -> hypothesis contradicted -> refine or replace it -> observe again
+   -> hypothesis contradicted -> refine or replace it -> choose again
    -> evidence unavailable -> stop with the smallest missing evidence
    -> cause supported -> return cause and correction boundary to Workflow
    -> user-owned behavior or source conflict -> Decision Gate
@@ -29,7 +35,7 @@ A requested patch, plausible code path, reviewer theory, or one favorable rerun
    -> active harm before cause -> propose bounded containment
 ```
 
-Diagnosis may recur inside one Track Work slice. A new hypothesis, observation, or owning-skill route does not create a new slice by itself.
+Diagnosis may recur inside one Slice. A new hypothesis, observation, or owning-skill route does not create a new Slice by itself. When durable task memory exists, [Track Work](../track-work/SKILL.md) preserves diagnosis inside the same Current Slice.
 
 ## Enter Diagnosis Deliberately
 
@@ -41,18 +47,18 @@ If fresh direct evidence already establishes one clear local defect and intended
 
 ## Establish The Failure And Loop
 
-Before changing behavior:
+Start by inspecting the report, existing code, tests, logs, configuration, and artifacts. Before generating new evidence:
 
-1. establish expected behavior from accepted intent and source truth;
-2. reproduce or capture the reported failure at its required boundary;
-3. state a falsifiable hypothesis;
-4. change or observe one distinguishing variable at a time;
+1. establish expected and observed behavior from accepted intent, source truth, and existing evidence;
+2. state a falsifiable hypothesis and the alternatives the next observation must distinguish;
+3. choose the smallest diagnostic loop that can disagree with the hypothesis at the required boundary;
+4. run one covered observation and change one distinguishing variable at a time;
 5. trace the causal chain from trigger and state to the observed result;
 6. preserve evidence that contradicts the leading hypothesis.
 
-Prefer the reported path, input, environment, and observer. A nearby failure may suggest a hypothesis but does not prove the reported failure. Reduced-fidelity reproduction is diagnostic evidence only; name what it cannot establish.
+Diagnosis owns the hypothesis and distinguishing question. When several observers or tools can test that same question, or the likely diagnostic loop is broad, use [Action Selection](../action-selection/SKILL.md) to choose and bound one environment interaction. It returns the observation without redefining the hypothesis or cause. Skip it when the diagnostic loop and observer are already exact.
 
-Read [the diagnostic-loop catalog](references/diagnostic-loop-catalog.md) when the best loop is unclear. Read [flaky and performance diagnosis](references/flaky-and-performance.md) when timing, randomness, environment variance, or resources shape the failure.
+Prefer the reported path, input, environment, and observer. A nearby failure may suggest a hypothesis but does not prove the reported failure. Reduced-fidelity reproduction is diagnostic evidence only; name what it cannot establish.
 
 Allowed behavior is not a reproduction. A cache hit does not prove a stale-read bug when caching is permitted. A possible race in source does not explain a report until timing, traces, steps, logs, or an existing expectation connect it.
 
@@ -96,7 +102,7 @@ A requested patch may instead be an authorized learning action. Its result can s
 
 ## Return The Supported Result
 
-When evidence supports a cause, return the failure boundary, diagnostic loop, causal explanation, regression signal, and smallest coherent correction to Workflow. Return an authorized implementation correction to [Execute Work](../execute-work/SKILL.md); use [TDD](../tdd/SKILL.md) when the diagnostic loop provides a failing behavior check that should guide it. Use [Simplify Code](../simplify-code/SKILL.md) when the supported cause shows that obsolete, duplicated, or workaround machinery can be removed while preserving accepted behavior. Mess left by unsuccessful attempts is not enough: settle any behavior or contract change through Workflow first, and do not use cleanup in place of correcting the supported failure or while its evidence remains unresolved. The correction may remain in the current Track Work slice while that slice still has one coherent result.
+When evidence supports a cause, return the failure boundary, diagnostic loop, causal explanation, regression signal, and smallest coherent correction to Workflow. The evidence supports the conclusion and route; it does not by itself authorize the correction. Return an authorized implementation correction to [Execute Work](../execute-work/SKILL.md); use [TDD](../tdd/SKILL.md) when the diagnostic loop provides a failing behavior check that should guide it. Use [Simplify Code](../simplify-code/SKILL.md) when the supported cause shows that obsolete, duplicated, or workaround machinery can be removed while preserving accepted behavior. Mess left by unsuccessful attempts is not enough: settle any behavior or contract change through Workflow first, and do not use cleanup in place of correcting the supported failure or while its evidence remains unresolved. The correction may remain in the same Slice while its result stays coherent; Track Work preserves that continuity when a Working Record exists.
 
 After correction, use [Verify Work](../verify-work/SKILL.md) to rerun both the minimized regression signal and the original reported path or strongest available observer. If correction fails or exposes related shared-state consequences, re-enter diagnosis before another patch.
 

package/skills/diagnose-failure/references/diagnostic-loop-catalog.md

@@ -1,6 +1,6 @@
 # Diagnostic Loop Catalog
 
-Read this when the smallest useful diagnostic loop for a reported failure or current hypothesis is unclear.
+Read this before choosing a diagnostic loop for a reported failure or current hypothesis. Use it to select the smallest observation that can distinguish meaningful alternatives at the required boundary.
 
 ## Common Loops
 
@@ -32,5 +32,3 @@ An observer must sit on the real path and cover the complete operation. A manual
 - A registered callback allegedly never runs → invoke or observe the registered entrypoint; calling its helper directly proves only helper behavior.
 - A failure appears only in CI → compare the same input across local and CI configuration before changing timing or adding retries.
 - Two causes fit one symptom → choose an observation whose outcomes differ between them; another generic failure is not distinguishing evidence.
-
-If no safe useful loop is possible, identify the smallest missing artifact or access: command, logs, trace, failing input, timestamped screenshot, environment boundary, or permission for isolated temporary instrumentation. Leave the cause unresolved rather than substituting a weaker claim.

package/skills/diagnose-failure/references/{flaky-and-performance.md → flaky-and-performance-diagnosis.md}

@@ -1,6 +1,6 @@
 # Flaky And Performance Diagnosis
 
-Read this when timing, randomness, environment variance, or resource behavior makes the failure difficult to reproduce or explain.
+Read this before choosing or running a diagnostic observation when timing, randomness, environment variance, or resource behavior shapes the failure.
 
 ## Flaky Failures
 

package/skills/discuss/SKILL.md

@@ -7,7 +7,7 @@ description: Use when discussing, exploring, shaping, challenging, or revisiting
 
 Build enough shared understanding to choose the next sound action.
 
-Discussion may shape understanding, recommendations, proposed work, and checkpoints. It does not authorize mutation, settle a user-owned choice implicitly, or turn exploratory output into production behavior.
+Discussion may shape understanding, recommendations, proposed work, and checkpoints. It inherits the current authority envelope: selecting Discuss does not authorize active evidence generation, mutation or delivery, settle a user-owned choice implicitly, or turn exploratory output into production behavior.
 
 ## Follow The Discussion Route
 
@@ -20,13 +20,16 @@ Use this as a directed map, not a checklist:
 -> [Inspect facts / ask / compare meaningful paths]
 -> [Route from what is now supported]
    -> enough understanding
-      -> task-shaped and continuity may matter
-         -> Track Work decides whether to recommend a Working Record
-      -> otherwise -> return direction to Workflow
+      -> decide continuity
+         -> no record -> return direction to Workflow
+         -> record needed and mutation covered -> Track Work -> Discuss -> Workflow
+         -> record needed but mutation uncovered -> propose exact mutation -> wait
    -> one user-owned choice or source conflict -> Decision Gate
    -> evidence needed -> propose a bounded learning action
-                         -> when authorized -> Track Work
-                            -> Execute Work or wait from its result
+                         -> apply the continuity decision before returning the proposal
+                         -> Workflow confirms authority and routes
+                            -> Track Work when needed -> Workflow -> Execute Work or wait
+                            -> no record needed -> Execute Work or wait
                          -> re-enter Discuss with evidence
    -> unexplained or repeated failure -> Diagnose Failure
    -> stable accepted content or strategy -> Write Spec / Write Plan
@@ -56,7 +59,7 @@ Identify only what the next decision needs:
 - tentative assumptions or viable alternatives;
 - evidence and uncertainty that could change the route.
 
-When a Working Record exists, orient from its current context and check important claims against live evidence. Read older history only when the present direction or rationale is unclear. Update only what new evidence affects.
+When a Working Record exists, orient from its current context and check important claims against live evidence. Read older history only when the present direction or rationale is unclear. Update only what new evidence affects. When a material record update is needed, use Track Work's replacement summaries and one-owner storage rather than appending discussion prose. Apply the update only when the authority envelope covers its mutation. If the mutation is uncovered, return it to Workflow and wait. After any covered update, return to Discuss—a record update does not start execution or change the owning route.
 
 ## Focus The Discussion
 
@@ -64,6 +67,8 @@ Stay with the highest unresolved question that could change the outcome, boundar
 
 Inspect code, tests, docs, policies, artifacts, repository state, supplied material, or current primary sources when they can answer factual questions. Ask the user about intent, priorities, constraints, and tradeoffs that evidence cannot decide.
 
+When several inspections, searches, or other observers could answer the same factual question, or the likely probe is broad, use [Action Selection](../action-selection/SKILL.md) to choose and bound one environment interaction. Discuss keeps ownership of the question and interpretation; Action Selection returns the observation. Skip it for an obvious focused read.
+
 When materially different paths remain viable:
 
 1. compare only the few that matter, including the current path or waiting when real;
@@ -75,6 +80,27 @@ Ask in natural prose, usually about one main topic at a time. Use a menu only fo
 
 When architecture, interfaces, ownership, state, failure contracts, or spreading complexity shape the direction, use [Design for Depth](../design-for-depth/SKILL.md) as a lens and retain it while the boundary remains design-bearing.
 
+## Preserve Only Useful State
+
+Keep discussion legible as:
+
+- **Settled:** supported fact or explicit decision.
+- **Tentative:** current hypothesis or provisional direction.
+- **Open:** unresolved and capable of changing the next action.
+
+Before routing to Track Work, ask: **Would losing current task state risk later misalignment?** Use a Working Record when:
+
+- the task must survive compaction, pause, session navigation, or ownership transfer;
+- decisions, evidence, blockers, authority, or the next action need durable recovery;
+- one outcome is likely to span several implementation, verification, review, or correction iterations;
+- proposed outcomes must remain ordered and visible without becoming authorized work.
+
+Keep state in conversation when context is sufficient and the discussion or next action is short and disposable. Use [Write Spec](../write-spec/SKILL.md) for stable accepted content and [Write Plan](../write-plan/SKILL.md) for a stable ordered strategy. Do not create an artifact merely because discussion occurred.
+
+Recommend independent review, local commit, user decision, or continuity checkpoints only when they protect dependent work or reduce material risk. Discussion may shape and preserve an approved checkpoint, but does not execute it. Normal verification and silent self-review provide the ordinary self-check; Workflow establishes the slice outcome.
+
+Read [Discussion Continuity](references/discussion-continuity.md) when preserving a compact discussion state or re-entry summary, or when carrying a selected checkpoint from discussion.
+
 ## Learn Through Bounded Action
 
 When discussion alone cannot answer a material question, propose the smallest useful learning or delivery action:
@@ -88,24 +114,10 @@ Useful checkpoint, if any:
 Stop, discard, revise, or promote when:
 ```
 
-A proposal is not authorization. Before execution, confirm that the action is requested or approved. When the action is authorized, use [Track Work](../track-work/SKILL.md) to decide whether durable task memory is needed, then follow its result to [Execute Work](../execute-work/SKILL.md) or wait.
+Classifying an action as Learning, Delivery, or Deepening does not authorize it. Before returning an uncovered proposal to Workflow, apply the continuity decision above and include any required [Track Work](../track-work/SKILL.md) mutation. Once Workflow confirms that the authority envelope covers the action and any record mutation, use Track Work when needed, return to Workflow, then route to [Execute Work](../execute-work/SKILL.md) or wait.
 
 Return with the observed result. Treat prototypes, tests, benchmarks, sketches, and working behavior as evidence, not automatic approval of their design or promotion. Preserve what still holds and re-enter only the affected question.
 
-## Preserve Only Useful State
-
-Keep discussion legible as:
-
-- **Settled:** supported fact or explicit decision.
-- **Tentative:** current hypothesis or provisional direction.
-- **Open:** unresolved and capable of changing the next action.
-
-Keep this state in conversation while context is sufficient. Use Track Work when evolving context must survive compaction, pause, or another session. Use [Write Spec](../write-spec/SKILL.md) for stable accepted content and [Write Plan](../write-plan/SKILL.md) for a stable ordered strategy. Do not create an artifact merely because discussion occurred.
-
-Recommend independent review, local commit, user decision, or continuity checkpoints only when they protect dependent work or reduce material risk. Discussion may shape and preserve an approved checkpoint, but does not execute it. Normal verification and silent self-review provide the ordinary self-check; Workflow establishes the slice outcome.
-
-Read [discussion checkpoints](references/checkpoints.md) when selecting or preserving a compact discussion checkpoint or re-entry state.
-
 ## Return Supported Direction
 
 Converge when the next sound action no longer depends on unresolved direction, not when every future question is answered.

package/skills/discuss/references/discussion-continuity.md

@@ -0,0 +1,71 @@
+# Discussion Continuity
+
+Read this when preserving a compact discussion state or re-entry summary, or when carrying a selected checkpoint from discussion.
+
+A **discussion state summary** records enough shared understanding to continue or choose the next action. It is not a Checkpoint unless Workflow deliberately selects a boundary that dependent work must not cross unresolved.
+
+## Keep It In Chat When Possible
+
+Keep the summary in conversation when the current context can safely continue. Use only fields that help:
+
+```text
+Goal or question:
+Current understanding:
+Settled:
+Tentative:
+Open:
+Evidence or alternatives that matter:
+Selected checkpoints, if any:
+Recommended or accepted next action:
+Authority source or approval needed:
+```
+
+Do not turn a summary into a transcript, questionnaire, frozen architecture, automatic Spec, Plan, Working Record, or checkpoint.
+
+## Preserve Selected Checkpoints Separately
+
+When discussion proposes a possible checkpoint, record it as proposed rather than selected. Workflow selects it only when its additional boundary materially protects dependent work and its authority is covered.
+
+For a selected checkpoint preserve only what later action needs:
+
+```text
+Status: Selected
+Type:
+Purpose:
+Due boundary:
+Condition:
+Selection source:
+Authority scope:
+```
+
+Do not infer authorization from a recommendation. Keep local commit authority separate from push or integration, and keep migration, deprecation, release, and launch separately controlled.
+
+When durable task memory is needed, [Track Work](../../track-work/SKILL.md) owns the pending checkpoint and terminal result. The discussion summary remains conversational state rather than duplicate checkpoint history.
+
+## Preserve A Durable Artifact Only When Needed
+
+Choose the artifact by the information that must survive:
+
+- **Working Record:** evolving task state, selected checkpoints, Slices, Decisions, evidence, History, and next action. Read [Track Work](../../track-work/SKILL.md).
+- **Spec:** stable accepted content needs a durable source. Read [Write Spec](../../write-spec/SKILL.md).
+- **Plan:** a stable ordered strategy and its selected checkpoints need an execution artifact. Read [Write Plan](../../write-plan/SKILL.md).
+- **ADR:** a surprising, hard-to-reverse repository decision needs durable rationale.
+- **Handoff:** one point-in-time continuation package must transfer to another context or owner.
+- **Domain documentation:** stable terminology or domain meaning belongs in an existing glossary or domain source.
+
+Do not create an artifact merely because discussion occurred. Prefer an existing repository convention; ask before inventing a destination whose authority, durability, or ownership matters.
+
+## Preserve Re-Entry State
+
+When new evidence reopens discussion, preserve only the affected change:
+
+```text
+New evidence:
+Invalidated assumption:
+Still valid:
+Decision, Slice, or artifact affected:
+Question or learning action now needed:
+Authority or user decision needed:
+```
+
+Do not restart from zero or rewrite unaffected decisions, evidence, artifacts, or work.

package/skills/evaluate-skill/SKILL.md

@@ -65,9 +65,9 @@ If both variants pass, the pressure may be weak or the baseline sufficient. If b
 
 ## Declare The Definition
 
-Use the exact [definition schema](references/definition-schema.md) for group, environment, expectation, suite, and selector shapes.
+Use the exact [Evaluation Definition Schema](references/evaluation-definition-schema.md) for group, environment, expectation, suite, and selector shapes.
 
-Declare the prompt or turns, fixture, exact ordered skills and target, working-tree or Git source, UTF-8 context, tools, model and thinking, deterministic expectations, comparison IDs, and review questions. Snapshot every declared subject resource. Ambient installation is not declared composition.
+Declare the prompt or turns, fixture, exact ordered skills and target, working-tree or Git source, runtime host/session mode and ordered extension bundles when needed, literal/inherited environment sources, UTF-8 context, tools, model and thinking, deterministic expectations, comparison IDs, and review questions. Snapshot every declared subject resource. Ambient installation is not declared composition.
 
 ## Run And View
 
@@ -82,7 +82,7 @@ Run from the definition root. Definition targets resolve from the current workin
 
 With no selectors, every suite group and both variants are selected. `--group` is invalid for a direct group. Run `--help` before relying on an operation.
 
-Pi is the direct subject process. One-shot descriptions use fresh JSON-mode subjects. Ordered description turns and body groups use one isolated RPC process per variant. Body groups explicitly deliver one target on turn one. `bash`, definition-supplied command execution, and end-to-end execution are unsupported.
+Pi is the direct subject process; a runtime profile may select the compatible `piflow` command as an explicit host variant. One-shot descriptions use fresh JSON-mode subjects. Ordered description turns and body groups use one isolated RPC process per variant. Body groups explicitly deliver one target on turn one. Declared extension bundles may provide non-native custom tools in body groups and context injections; description groups remain limited to no tools or `read`. Native `bash`, `powershell`, `grep`, `find`, and `ls` tools, definition-supplied command execution, and end-to-end execution are unsupported.
 
 Read [execution and evidence](references/execution-and-evidence.md) when operating commands, resolving paths, interpreting states, or reasoning about Pi isolation, persistence, views, cancellation, cleanup, and safeguards.
 
@@ -90,9 +90,9 @@ Read [execution and evidence](references/execution-and-evidence.md) when operati
 
 Append deterministic grades only after canonical run evidence exists. A grade is derived evidence and never mutates or invalidates a run.
 
-Supported checks cover exact skill/resource reads, typed paths, changed paths, file/response text, explicit JSON states, valid turn scope, and factual baseline-to-candidate transitions. A failed check remains ordinary behavioral evidence. Missing evidence stays unavailable. Malformed checks or grading failures produce separate `grade-error` evidence. Transitions carry no quality or readiness meaning.
+Supported checks cover exact skill/resource reads, typed paths, changed paths, file/response text, explicit JSON states, tool-call outcomes and argument predicates, system-prompt or provider-context text by selected request and turn, valid turn scope, and factual baseline-to-candidate transitions. A failed check remains ordinary behavioral evidence. Missing evidence stays unavailable. Malformed checks or grading failures produce separate `grade-error` evidence. Transitions carry no quality or readiness meaning.
 
-Prefer generated grade-first views for routine inspection. Use ordinary file tools for raw run, event, transcript, response, stderr, workspace, definition, grade, and group artifacts. A view may remove repeated transport structure; it must not hide evidence required by the question.
+Prefer generated grade-first views for routine inspection. Use ordinary file tools for raw run, event, transcript, response, stderr, workspace, definition, grade, context-observation, and group artifacts. A view may remove repeated transport structure; it must not hide evidence required by the question.
 
 Read [review and revision](references/review-and-revision.md) after evidence exists, when surfaces conflict, meaning still needs judgment, or a revision/rerun boundary must be chosen.
 

package/skills/evaluate-skill/references/{definition-schema.md → evaluation-definition-schema.md}

@@ -68,6 +68,7 @@ Rules:
 - `expectations` is an ordered array with unique expectation IDs.
 - `review_questions` is an ordered string array. Questions are rendered for the reviewer and never sent to the subject.
 - `model` is optional. `model.model` names the provider/model; optional `model.thinking` selects its thinking level. The CLI has no model override.
+- `runtime` is optional. When omitted, the group runs under a plain `pi` host with no declared extensions. When present, it declares the host, ordered extension bundles, and string environment overrides.
 
 ## Description Group Example
 
@@ -203,10 +204,45 @@ A body group explicitly delivers each target on turn one and observes first-read
 }
 ```
 
-Body groups allow no tools or path-guarded `read`, `write`, and `edit`. A no-target baseline uses `skills: []` and `target: null`; the candidate must always identify a target.
+Description groups allow no tools or only built-in `read`. Body groups allow built-in `read`, `write`, and `edit`, plus custom tools declared by a runtime extension bundle. Native `bash`, `powershell`, `grep`, `find`, and `ls` tools are unsupported. A no-target baseline or runtime-only group may use `skills: []` and `target: null`; a candidate that declares skills must identify its target.
 
 The examples are complete definition shapes. Replace fixture, resource, model, and Git-ref values with paths and identities that exist under your definition root.
 
+## Runtime Profile
+
+A runtime profile is shared by both baseline and candidate variants. It changes the host environment, not the skill target:
+
+```json
+{
+  "runtime": {
+    "host": "pi",
+    "session": false,
+    "extensions": [
+      {
+        "entry": "extensions/probe.mjs",
+        "resources": ["extensions"]
+      }
+    ],
+    "environment": {
+      "literal": {
+        "PROBE_MODE": "injection-test"
+      },
+      "inherit": []
+    }
+  }
+}
+```
+
+`host` is `pi` or `piflow`; the evaluator maps it to the corresponding installed host command and records the selected host. `session` controls whether the variant receives an isolated persistent session directory; it defaults to `false` in omitted runtime profiles. A runtime may declare zero or more bundles. Each bundle has an entry file and one or more directory resources copied under their original relative paths, so sibling imports and multi-tree extensions remain intact. Bundle bytes are snapshotted once and copied identically to both variants.
+
+`environment.literal` contains non-secret string configuration and is recorded in run evidence. `environment.inherit` names parent-process variables whose values are passed to the child but never persisted; other parent variables are not passed. Missing inherited variables invalidate the variant. The evaluator supplies only a small host-runtime baseline such as `PATH` and `HOME`. Launch-control and loader keys are rejected from inherited values, literal and inherited names must not overlap, and secret-like literal keys are rejected.
+
+Extensions are trusted executable inputs, not OS-sandboxed code. The evaluator proves declared identity and post-run immutability, but extension code retains its host permissions.
+
+When a group declares `context-text` expectations, the evaluator loads its base isolation guard first, then declared bundles, then a non-mutating final observer. The observer records two per-turn surfaces: the chained `before_agent_start` system prompt and each provider-neutral `context` message projection. `context-text` expectations select `surface: "system-prompt"` or `surface: "provider-context"` and may select `request: "first"`, `"last"`, or a positive request number. Groups without context assertions do not create context-observation artifacts.
+
+A custom tool name is allowed in `group.tools` only when the runtime declares an extension bundle. The evaluator remains extension-neutral: it does not interpret feature names or configuration schemas. Put feature configuration in the ordinary group fixture or runtime environment, and test combinations as separate suite groups.
+
 ## Variant Environment
 
 Each environment has exactly:
@@ -216,7 +252,7 @@ Each environment has exactly:
   "source": { "kind": "git", "ref": "v1.2.3" },
   "skills": ["skills/target", "skills/support"],
   "target": 0,
-  "context": ["runtime/interaction-contract.md"]
+  "context": ["runtime/prompts/interaction-contract.md"]
 }
 ```
 
@@ -255,6 +291,10 @@ Without `turn`, response checks use the final response and workspace checks use
 | `file-text` | workspace `path`, string `value`, optional body `turn` | `contains`, `not-contains`, `equals` |
 | `json` | workspace `path`, optional body `turn`, conditional `pointer` and `value` | `available`, `missing`, `valid`, `malformed`, `field-present`, `field-absent`, `field-equals` |
 | `response-text` | string `value`, optional body `turn` | `contains`, `not-contains`, `equals` |
+| `tool-call` | string `tool`, optional body `turn`, optional `argumentContains`/`argumentNotContains` arrays | `called`, `succeeded`, `failed`, `not-called` |
+| `context-text` | string `value`, optional `surface` (`system-prompt` or `provider-context`), optional `request`, optional body `turn` | `contains`, `not-contains`, `equals` |
+
+For `tool-call` with `expect: "not-called"`, omitted argument filters require zero calls to the named tool. Supplied filters scope the absence check to calls matching those argument predicates.
 
 ### Resource paths
 

package/skills/evaluate-skill/references/execution-and-evidence.md

@@ -33,7 +33,9 @@ With no selectors, `run` or `view` selects every suite group and both variants.
 
 ## Current Execution Boundary
 
-`run` executes description and body groups. `end-to-end` is a reserved definition type and is rejected before subject execution. Description groups allow no tools or `read`; body groups allow no tools or path-guarded `read`, `write`, and `edit`. `bash` and definition-supplied commands are unsupported.
+`run` executes description and body groups. `end-to-end` is a reserved definition type and is rejected before subject execution. Description groups allow no tools or `read`; body groups allow no tools or path-guarded `read`, `write`, and `edit`, plus non-native custom tools supplied by a declared runtime extension bundle. Native `bash`, `powershell`, `grep`, `find`, and `ls` tools and definition-supplied commands are unsupported.
+
+A runtime profile selects the installed `pi` or `piflow` host, whether the variant uses an isolated persistent session directory, zero or more ordered extension bundles, and declarative environment sources. Literal values are non-secret configuration; only explicitly inherited parent variables are passed to the child without being persisted, alongside a small host-runtime baseline. Missing inherited variables invalidate the variant. Launch-control and loader keys are rejected from inherited values, and literal and inherited names must not overlap. For body runs, delivery remains `unavailable` until target setup succeeds; `natural-prompt` or `explicit-skill-command` records a prompt attempt. When a group declares context assertions, the evaluator launches the base guard first, declared extensions second, and a final non-mutating observer last. The observer records both the effective `before_agent_start` system prompt and each provider-neutral `context` message projection; it does not replace or sanitize extension injections after the base guard has established evaluator-owned isolation.
 
 Use fresh JSON-mode execution for one-shot descriptions. Use one persistent RPC process per selected variant for ordered description turns and all body groups. RPC correlates responses, waits for `agent_settled`, disables automatic retry and compaction, preserves directly observed partial-turn evidence, and cleans the process tree.
 
@@ -43,9 +45,12 @@ A body target is matched to the exact snapshotted `SKILL.md` and explicitly deli
 
 For each selected variant:
 
-- snapshot exact declared skills and context;
+- snapshot exact declared skills, context, and the shared runtime bundle resources;
+- copy the shared runtime bundles identically into the variant and verify their fingerprints before and after execution;
 - snapshot the group fixture once from the definition-root working tree—never from a variant Git source—then create an independent writable copy;
 - preserve skill order and target index;
+- preserve runtime host, session mode, and extension order;
+- preserve runtime bundle and provider-context observation evidence when requested;
 - replace ambient instructions with evaluator-owned declared context;
 - keep criteria and review questions outside the subject prompt;
 - allow reads only from workspace and declared immutable resources;
@@ -76,7 +81,7 @@ Views can render a complete result, all baseline or candidate variants, one grou
 
 Path cells escape literal backslashes before tab, carriage return, and newline. Variant views exclude expectation-owned errors from the other variant while retaining comparison, group, and system errors.
 
-Use ordinary file tools for raw `run.json`, events, transcript, final response, stderr, workspace, definition, grade, and group artifacts. Views remove transport noise; they do not replace canonical evidence.
+Use ordinary file tools for raw `run.json`, events, transcript, final response, stderr, workspace, definition, grade, context observations, and group artifacts. Views remove transport noise; they do not replace canonical evidence. Tool-call grades distinguish attempted, succeeded, failed, and not-called outcomes.
 
 ## Safeguards And Limits
 

package/skills/evaluate-skill/scripts/lib/definitions.mjs

@@ -4,6 +4,20 @@ import path from "node:path";
 const ID = /^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$/;
 const EVALUATION_TYPES = new Set(["description", "body", "end-to-end"]);
 const VARIANTS = ["baseline", "candidate"];
+const EVALUATION_HOSTS = new Set(["pi", "piflow"]);
+const RESERVED_ENVIRONMENT_KEYS = new Set([
+  "PATH",
+  "HOME",
+  "PWD",
+  "OLDPWD",
+  "SHELL",
+  "BASH_ENV",
+  "NODE_OPTIONS",
+  "NODE_PATH",
+  "PI_PACKAGE_DIR",
+  "PIFLOW_CODING_AGENT_DIR",
+  "PIFLOW_CODING_AGENT_SESSION_DIR",
+]);
 
 export class DefinitionError extends Error {
   constructor(code, message, definitionPath = null) {
@@ -129,6 +143,102 @@ function validateEnvironment(value, label, definitionPath) {
   }
 }
 
+function validateRuntime(value, definitionPath) {
+  const runtime = requireExactKeys(
+    value,
+    ["host", "extensions", "environment", "session"],
+    [],
+    "group.runtime",
+    definitionPath,
+  );
+  if (!EVALUATION_HOSTS.has(runtime.host)) {
+    fail("invalid-definition", "group.runtime.host must be pi or piflow", definitionPath);
+  }
+  if (!Array.isArray(runtime.extensions)) {
+    fail("invalid-definition", "group.runtime.extensions must be an array", definitionPath);
+  }
+  const bundleIds = new Set();
+  for (const [index, entry] of runtime.extensions.entries()) {
+    const bundle = requireExactKeys(
+      entry,
+      ["entry", "resources"],
+      [],
+      `group.runtime.extensions[${index}]`,
+      definitionPath,
+    );
+    requireSafeRelativePath(bundle.entry, `group.runtime.extensions[${index}].entry`, definitionPath);
+    const resources = requireStringArray(
+      bundle.resources,
+      `group.runtime.extensions[${index}].resources`,
+      definitionPath,
+      { paths: true, unique: true },
+    );
+    if (resources.length === 0) {
+      fail("invalid-definition", `group.runtime.extensions[${index}].resources must not be empty`, definitionPath);
+    }
+    const bundleId = `${bundle.entry}\u0000${resources.join("\u0000")}`;
+    if (bundleIds.has(bundleId)) {
+      fail("invalid-definition", "group.runtime.extensions contains duplicate bundles", definitionPath);
+    }
+    bundleIds.add(bundleId);
+  }
+  if (typeof runtime.session !== "boolean") {
+    fail("invalid-definition", "group.runtime.session must be a boolean", definitionPath);
+  }
+  const environment = requireExactKeys(
+    runtime.environment,
+    ["literal", "inherit"],
+    [],
+    "group.runtime.environment",
+    definitionPath,
+  );
+  const literal = requireObject(environment.literal, "group.runtime.environment.literal", definitionPath);
+  const inherited = requireStringArray(environment.inherit, "group.runtime.environment.inherit", definitionPath, {
+    unique: true,
+  });
+  const literalKeys = new Set(Object.keys(literal).map((key) => key.toUpperCase()));
+  for (const [key, value] of Object.entries(literal)) {
+    const normalizedKey = key.toUpperCase();
+    if (
+      !/^[A-Za-z_][A-Za-z0-9_]*$/.test(key) ||
+      isBlockedEnvironmentKey(normalizedKey) ||
+      /TOKEN|SECRET|PASSWORD|API[_-]?KEY|PRIVATE[_-]?KEY|CREDENTIAL/i.test(key)
+    ) {
+      fail(
+        "invalid-definition",
+        `group.runtime.environment.literal contains a reserved or secret-like key: ${key}`,
+        definitionPath,
+      );
+    }
+    if (typeof value !== "string") {
+      fail("invalid-definition", `group.runtime.environment.literal.${key} must be a string`, definitionPath);
+    }
+  }
+  for (const key of inherited) {
+    const normalizedKey = key.toUpperCase();
+    if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(key) || isBlockedEnvironmentKey(normalizedKey)) {
+      fail("invalid-definition", `group.runtime.environment.inherit contains an invalid key: ${key}`, definitionPath);
+    }
+    if (literalKeys.has(normalizedKey)) {
+      fail(
+        "invalid-definition",
+        `group.runtime.environment key is declared as both literal and inherited: ${key}`,
+        definitionPath,
+      );
+    }
+  }
+}
+
+function isBlockedEnvironmentKey(normalizedKey) {
+  return (
+    RESERVED_ENVIRONMENT_KEYS.has(normalizedKey) ||
+    normalizedKey.startsWith("SKILL_EVAL_") ||
+    normalizedKey.startsWith("PI_") ||
+    normalizedKey.startsWith("LD_") ||
+    normalizedKey.startsWith("DYLD_")
+  );
+}
+
 function validateExpectations(value, definitionPath) {
   if (!Array.isArray(value)) fail("invalid-definition", "group.expectations must be an array", definitionPath);
   const ids = new Set();
@@ -164,7 +274,7 @@ function validateGroup(value, definitionPath) {
       "expectations",
       "review_questions",
     ],
-    ["model"],
+    ["model", "runtime"],
     "group",
     definitionPath,
   );
@@ -177,12 +287,13 @@ function validateGroup(value, definitionPath) {
   validateInput(group.input, definitionPath);
   if (group.fixture !== null) requireSafeRelativePath(group.fixture, "group.fixture", definitionPath);
   requireStringArray(group.tools, "group.tools", definitionPath, { unique: true });
+  if (Object.hasOwn(group, "runtime")) validateRuntime(group.runtime, definitionPath);
   const variants = requireExactKeys(group.variants, VARIANTS, [], "group.variants", definitionPath);
   for (const variant of VARIANTS) validateEnvironment(variants[variant], `group.variants.${variant}`, definitionPath);
-  if (variants.candidate.target === null) {
+  if (variants.candidate.target === null && variants.candidate.skills.length > 0) {
     fail(
       "invalid-definition",
-      "group.variants.candidate.target must identify the candidate target skill",
+      "group.variants.candidate.target must identify the candidate target skill when candidate skills are declared",
       definitionPath,
     );
   }