@hassangameryt/freeflow 0.5.0 → 0.7.0

package/skills/write-skill/references/progressive-disclosure.md

@@ -1,63 +1,127 @@
 # Progressive Disclosure
 
-Read this before adding or reorganizing references and scripts.
+Read this before adding or reorganizing references, assets, or scripts.
 
-Make the first-read route understandable, then move only separately owned or conditional depth out of the active body.
+Progressive disclosure should change when content enters context, preserve one canonical owner, or isolate a real lifecycle boundary. Splitting files without one of those benefits adds dependency cost without reducing context.
 
-## Dependency Direction
+## Preserve Dependency Direction
 
 ```text
 description
 -> SKILL.md
--> required or conditional reference or script
+-> required or conditional resource
 ```
 
-The description routes the first useful read. `SKILL.md` establishes the job, normal method, necessary terms, and stop conditions. A required resource must be linked before the body relies on it. A conditional resource may rely on language already established by the body.
+`SKILL.md` establishes:
 
-Do not hide a definition, rule, or step needed to recognize the normal route in an unannounced reference. That is progressive reconstruction, not disclosure.
+- the job;
+- normal route;
+- necessary terms;
+- stop and return conditions;
+- observable resource read points.
 
-## Add A Required Reference When
+A reference cannot activate itself. Do not hide the rule needed to recognize a reference inside that reference.
 
-- normal execution genuinely needs separately owned depth;
-- the body can identify and require the read before using that depth;
-- keeping the material inline would obscure the skill’s controlling rules;
-- the dependency is delivered and validated with the skill package.
+That is progressive reconstruction, not disclosure.
 
-Required does not mean implicit. The body must say when to read the resource and must not act as if unread content were already known.
+## Classify The Read Point
 
-## Add A Conditional Reference When
+### Entry-required
 
-- the material is needed only for a distinct branch of work;
-- loading it every time would obscure higher-priority behavior;
-- the active body can name an observable condition for reading it;
-- the reference has one clear job and does not restate the full skill or Workflow.
+Read before the skill can follow or reject its normal route.
 
-Keep compact examples that teach the normal boundary in `SKILL.md`. Move a larger example set only when it serves a conditional branch and would obscure the normal route.
+Entry-required references are exceptional. Use one only when separation provides independent value, such as:
+
+- canonical reuse across packages;
+- separate ownership or lifecycle;
+- generated schemas or contracts;
+- compatibility or host constraints;
+- independently versioned external policy.
+
+If every useful activation reads the resource and it has no separate ownership or lifecycle value, inline it even when `SKILL.md` becomes substantially longer.
+
+### Activity-required
+
+Read before performing one named normal activity.
+
+The skill may first:
+
+- exit;
+- defer;
+- choose another route;
+- determine that the activity does not apply.
+
+Activity-required references provide real loading value even when the activity is common.
+
+### Conditional
+
+Read only when an observable branch occurs.
+
+The body must teach the agent enough to recognize the branch before loading the reference.
+
+Do not use “when unclear” if the reference contains the method for recognizing the underlying condition.
+
+## Keep The Normal Route Contiguous
+
+Start with one `SKILL.md`.
+
+Keep inline:
+
+- the first-read job;
+- normal route;
+- controlling definitions;
+- rules needed to select resources;
+- stop and return behavior.
+
+Move depth only when keeping it inline would obscure the normal route and the read point is executable.
+
+Do not split a file to satisfy a line count.
+
+## Add A Reference When
+
+- one named normal activity needs separately owned depth;
+- one observable conditional branch needs depth;
+- one canonical resource is shared without duplication;
+- one resource has a real independent lifecycle;
+- the body can name and require the read before use.
+
+The body instruction and reference introduction must state the same read condition.
+
+Required means necessary. Do not call a required reference recommended or optional.
+
+If skipping the reference preserves correct execution, either make its concrete optional value explicit or remove it.
 
 ## Add A Script When
 
-- the skill owns a deterministic feature or repeated operation;
-- retyping it is error-prone, unsafe, or materially wasteful;
-- its interface and failure behavior can be tested.
+- the skill owns a deterministic repeated operation;
+- model-generated execution would be error-prone, unsafe, or materially wasteful;
+- the interface and failure behavior can be tested;
+- the script’s output has a clear evidence boundary.
 
-A script does not require an earlier failure. Do not script ordinary shell commands merely to look complete.
+Do not script ordinary commands merely to make the package appear complete.
 
 ## Link At The Decision Point
 
-A resource link should tell the agent when it becomes useful:
+Link every agent-facing reference, asset, package dependency, and executable entrypoint from `SKILL.md` at the first useful choice.
+
+Avoid vague resource lists such as:
+
+> See references for more information.
 
-> Read the integration-evidence reference when the claim depends on host dispatch or an installed artifact.
+Prefer:
 
-Avoid vague resource lists such as “See references for more information.”
+> Read Test Design before designing or materially changing the behavior check.
 
-## Resource Test
+## Apply The Resource Test
 
-For every extra file, answer:
+For every extra file, ask:
 
-1. What normal dependency or conditional branch requires it?
-2. When can the agent recognize that read or run condition?
-3. What prior language does the resource assume, and where is it established?
-4. Why cannot the active body or a direct command do the job clearly?
-5. How will the resource be validated and kept consistent?
+1. What activity, conditional branch, shared ownership, or lifecycle requires it?
+2. When can the agent recognize the read or run condition?
+3. What prior language does it assume?
+4. Where is that language guaranteed?
+5. What context or ownership value does separation provide?
+6. Why is inline content worse?
+7. How will links, containment, and behavior be verified?
 
-If those answers are weak, remove the resource.
+If separation changes neither loading nor ownership, inline the content or remove it.

package/skills/write-skill/references/skill-dependency-graphs.md

@@ -0,0 +1,190 @@
+# Skill Dependency Graphs
+
+Read this when creating or revising multiple related skills, shared terminology, cross-skill links, capability activation, direct or routed entry paths, or context-loss recovery.
+
+A multi-skill system is a stateful, partially loaded prompt graph. It is not a mandatory sequence.
+
+## Identify The Edge Types
+
+Do not treat every link as execution.
+
+### Activation edge
+
+Causes a skill to be read or made effective:
+
+```text
+natural prompt -> description -> skill
+always-loaded system cue -> recovery skill
+capability cue -> capability skill
+```
+
+### Owner route
+
+Changes which activity controls the current bounded result.
+
+A plain-text return may express control flow without creating a Markdown package dependency.
+
+### Composition edge
+
+Adds a method, lens, reviewer contract, or domain guide without changing the current owner.
+
+### Reference edge
+
+Loads activity-required or conditional depth inside a package.
+
+### Environment-interaction edge
+
+Temporarily selects one observation or effect and returns to the same current owner.
+
+### Persistence edge
+
+Earlier skill guidance remains in context and may affect later behavior without another read.
+
+### Recovery edge
+
+Reintroduces required guidance after context loss through an always-loaded upstream cue.
+
+## Enumerate Every Valid Entry Path
+
+For each skill, identify:
+
+- natural description activation;
+- orchestrator or lifecycle routes;
+- sibling routes;
+- direct user commands;
+- capability cues;
+- context-loss recovery;
+- host-specific activation;
+- artifact or checkpoint routes.
+
+Do not optimize only for the most common path.
+
+A skill may rely only on context guaranteed across every valid activation path.
+
+If a skill can activate directly, it cannot depend on vocabulary introduced only by a commonly preceding skill.
+
+If a skill can activate through an independent recovery cue, it cannot assume that an earlier discussion or planning skill was read first.
+
+## Place Terminology Upstream
+
+For every shared term:
+
+1. list all consumers;
+2. list every valid path to those consumers;
+3. find the nearest upstream context guaranteed on all paths;
+4. define the term there;
+5. keep specialized detail with its owning skill.
+
+Place shared vocabulary in the nearest always-available context only when independently activated consumers require it.
+
+Keep local terms local when other skills should route to the owner rather than implement its method.
+
+Do not duplicate full definitions merely to compensate for a broken activation graph.
+
+## Prevent Self-Activation
+
+An unread skill cannot require its own first read.
+
+Critical activation must come from:
+
+- guaranteed base or system guidance;
+- a skill description;
+- an already-loaded owner;
+- an effective capability cue;
+- another declared upstream path.
+
+References follow the same rule: the body must name the read condition before using reference content.
+
+## Check Direct And Routed Paths
+
+For every related skill, test:
+
+```text
+direct natural activation
+routed activation from expected parent
+nearby safe-read activation
+retained use after another skill loads
+recovery after context loss
+```
+
+A path that works only because another skill happened to be read is not valid dependency composition.
+
+## Check Redundancy And Inconsistency
+
+Across all changed skills, inspect:
+
+- duplicate canonical definitions;
+- divergent wording for the same invariant;
+- early commands contradicted by later caveats;
+- two skills claiming the same current ownership;
+- methods or lenses accidentally becoming owners;
+- links with no activation, route, composition, or reference job;
+- terms used before guaranteed definition;
+- body and reference read-condition mismatch;
+- stale recovery cues;
+- direct routes bypassing required context;
+- retained instructions hijacking later owners.
+
+Distinguish semantic duplication from intentional local guards.
+
+## Repeat Across Skills Deliberately
+
+Repeat a compact invariant when a distinct activation path or decision point requires local protection.
+
+Keep:
+
+```text
+canonical owner
+-> complete definition or method
+
+consumer
+-> compact local guard
+-> route to canonical owner when needed
+```
+
+Do not copy complete sections across skills. Repetition should preserve meaning without creating competing ownership.
+
+## Keep Routing And Terminology Graphs Distinct
+
+Terminology and dependency order should normally flow from guaranteed upstream context toward consumers.
+
+Runtime routing may be cyclic:
+
+```text
+For example:
+clarification -> task memory -> clarification
+execution -> verification -> review -> execution
+```
+
+A cyclic routing graph does not justify cyclic terminology dependencies.
+
+## Validate Facts And Evaluate Behavior
+
+Deterministic checks can establish:
+
+- links;
+- containment;
+- dependency cycles;
+- resource inventory;
+- routing-map parity;
+- exact duplicate text.
+
+They cannot establish:
+
+- correct activation;
+- safe retained influence;
+- semantic consistency;
+- useful repetition;
+- behavior preservation;
+- readiness.
+
+Evaluate the exact graph claim:
+
+- activation;
+- first-read body behavior;
+- dependency composition;
+- retained use;
+- recovery;
+- artifact outcome.
+
+Do not require a graph artifact for every small skill edit. Use this method when several nodes, paths, terms, or owners change together.

package/capabilities/output-router/SKILL.md

@@ -1,206 +0,0 @@
----
-name: output-router
-description: Use after the Workflow, Decision Gate, or Discuss route is clear when choosing between native tools and Freeflow routed tools, retrieving repo/local/vault evidence, transforming bounded evidence, handling unknown-size or broad output, running likely-large/noisy commands, recovering vaulted output, tuning outputRouter/observedRouting/scriptTransform after setup, or handling optional native read/bash safety-net routing.
----
-
-# Output Router
-
-Choose how evidence moves into context.
-
-Freeflow tools are the safe first choice for unknown-size, exploratory, repo-wide, generated/log-adjacent, structured, or likely noisy output, including broad `rg`/`find`/`git`/help/docs scans. Native tools stay direct for known-small, exact, intentionally raw, or mutating work. A post-hoc cap such as `head`, `sed -n`, `tail`, or `wc` does not make a broad producer known-small.
-
-Output Router does not classify the task. Workflow, Decision Gate, or Discuss decides whether to answer, ask, explore, plan, or stop. Output Router starts after that route is clear.
-
-## First Decision
-
-Ask what kind of data you need:
-
-- Existing repo, explicit local, or vault data: use `freeflow_search`.
-- New command or script-produced output: use `freeflow_run`.
-- Several independent Freeflow operations: use `freeflow_batch`.
-- Config/status/vault/script-adapter facts: use `freeflow_status`.
-- Enabled MCP/web/fetch/code-search output: call the host tool directly; Pi observed routing runs after the host result when configured.
-- Known whole file, intentionally exact shell behavior, cmux/Pi control probes, explicit fallback after a routed-source limitation, or edits: use native `read`, `bash`, `edit`, or `write`.
-
-When unsure how much output will return, use Freeflow first. Bound the producer before using native tools; do not rely on trimming the output after a broad command runs.
-
-## `freeflow_search`: Existing Data
-
-Use `freeflow_search` for repo files, explicit external local files, and vaulted output that already exists.
-
-Actions:
-
-- `locate`: find likely paths/output ids first. Use before reading when you only need candidates.
-- `query`: return the best focused evidence packets for a search question.
-- `get`: user gave exact-ish text/code/snippet or asks “where is this?” Find the best match and return source coordinates plus matched content.
-- `retrieve`: you already know the path/outputId and line range. Return those exact lines. Do not use this as search.
-- `expand`: widen a previous evidence packet.
-- `explain`: explain a previous routed decision or vault output/recoverability.
-- `transform`: compute facts or deterministic subsets from repo/local/vault/file/output data.
-
-Use `source.kind="repo"` for the current checkout. Use `source.kind="local"` for external local docs/files with an explicit absolute `root` and optional relative `path`; do not put local roots in shared config. Use `source.kind="vault"` for previous Freeflow outputs.
-
-Vault search patterns:
-
-- Unknown output id: `source.kind="vault"` without `outputId`, plus `query` and optional `filters`.
-- Known output id and unknown line: `action="query"` or `get` with `source.outputId`.
-- Known output id and known line range: `action="retrieve"` with `lineRange`.
-- Metadata-only result: use `action="explain"`; do not claim raw recovery.
-
-Use `preserve="full"` only when exact fidelity matters. It still means exact-under-caps, not unlimited context injection.
-
-## `freeflow_search action=transform`: Processing Existing Data
-
-Use transform when raw data is too large/structured and you need computed facts.
-
-Good transform cases:
-
-- logs: status counts, error rates, slow examples,
-- JSON/CSV: counts, groupings, top values, extracted fields,
-- test/build/typecheck output already in the vault,
-- MCP/web/fetch/code-search payloads already routed into the vault,
-- deterministic regex filtering, match counts, grouping, dedupe, topN, URL/citation extraction, line/size stats.
-
-Prefer deterministic operations when enough:
-
-- `regexFilter`, `countMatches`, `jsonExtract`, `groupByRegex`, `dedupe`, `topN`, `extractUrls`, `extractCitations`, `lineStats`, `sizeStats`.
-
-Use scripts only when deterministic operations cannot express the fact:
-
-- `operation.kind="script"` runs a sandboxed transform over vault sources.
-- `script` without `operation` runs the processing-engine script path over repo/local/vault sources.
-- Sandboxed scripts require `scriptTransform.enabled=true` and a proof-backed adapter.
-- Raw script code is not persisted; metadata stores hashes/labels/limits.
-- No sandboxed script path may silently fall back to host shell execution.
-
-Unsafe unsandboxed processing is local-only, explicit per call, and only through the processing script policy. It must say `unsafe/unsandboxed`. Never put that opt-in in shared config.
-
-## `freeflow_run`: New Output
-
-Use `freeflow_run` when the producer creates new output and routed evidence is enough.
-
-Base producers are mutually exclusive:
-
-- `command`: shell command through the host-approved runner.
-- `script`: sandboxed code-as-producer with no repo/home/env/network/vault access.
-
-Use `command` for:
-
-- tests, builds, lint, typecheck, CI-like checks,
-- broad shell search with unknown output size,
-- commands likely to print logs, JSON, status, or diagnostics,
-- commands whose exact raw output may need recovery.
-
-Use `script` for:
-
-- generating data with code when no repo/home/env/network access is needed,
-- safe code-as-producer execution through the same sandbox engine as transforms,
-- stdout/stderr capture that should route like command output.
-
-Do not use `freeflow_run script` when the code needs repo files, environment variables, network, package imports, or host filesystem access. Use a host command intentionally, or transform an explicitly captured source.
-
-Run modifiers:
-
-- `goal`: tell the router why the run exists, e.g. `verification`, `diagnosis`, `build`, `log analysis`.
-- `filters`: deterministic line filtering over captured stdout/stderr/combined.
-- `scriptFilter`: sandboxed script over already captured stdout/stderr/combined. It never reruns the base producer.
-- `preserve="full"`: request exact output under caps with vault recovery over cap.
-
-Run storage:
-
-- Default `hybrid-dedupe` stores exact output when exactness-sensitive and metadata-only for small non-sensitive successes.
-- Failures, verification/test/build/typecheck/diagnosis output, `preserve="full"`, filters, script filters, reducers, script producers, and large/noisy output are exactness-sensitive.
-- Exact duplicates may store current metadata pointing to a previous exact `outputId`.
-- Plain metadata-only records get rerun guidance, not exact recovery claims.
-
-## `freeflow_batch`: Independent Freeflow Work
-
-Use `freeflow_batch` when several Freeflow-owned `run` or `search` steps can happen independently.
-
-Good batch cases:
-
-- run several bounded searches in parallel,
-- run independent checks and ask `queries[]` to extract facts from child evidence,
-- gather multiple repo/local/vault facts without injecting every child result.
-
-Do not use batch for:
-
-- sequenced workflows,
-- arbitrary external tool orchestration,
-- mutating work,
-- steps where one child needs another child’s output.
-
-Child outputs remain in `details.result.steps`; exact child run output remains recoverable by child `outputId`.
-
-## Native Tools
-
-Use native tools when direct behavior is the point:
-
-- `read`: known whole file or exact direct file content is intended.
-- `bash`: expected-small exact shell behavior, host access, cmux/Pi control probes, or bounded producer behavior is intentionally needed.
-- `edit` / `write`: mutations.
-
-Native `bash`/`read` remain valid for known-small exact checks, cmux/Pi control probes, exact file reads, or explicit fallback after a routed-source limitation; name the limitation and keep the fallback narrow.
-
-Do not run broad native commands just to see what comes back. `head`, `sed -n`, `tail`, `wc`, or similar output caps bound visible text, not producer scope or relevance. Likely-large native commands include repo-wide `rg`, `grep -R`, `find`, package scans, generated-artifact scans, docs/help scans, logs, broad `git diff/log`, full tests/builds/lints/typechecks, and one-off scripts with unknown output size.
-
-If native read/bash safety net is enabled, large/noisy native output may be replaced with labeled Freeflow-routed output. It must include an `outputId` and recovery instructions. If safety-net routing fails, native output passes through with a warning.
-
-## Observed Routing
-
-For enabled Pi MCP/web/fetch/code-search producers, call the host tool directly. Freeflow observes the completed result after host execution.
-
-Observed routing:
-
-- is off by default,
-- is explicit per producer/server,
-- does not grant permissions or block host tools,
-- can persist exact output, metadata-only, or nothing depending on config,
-- fails open with original host output if routing fails.
-
-Mutating MCP/tool calls still require normal workflow/interview judgment before the host call. Observed routing is not permission approval.
-
-## Hard Rules
-
-- Do not pretend routed output is native output.
-- Do not silently shorten native output.
-- Capture raw output before transforming it.
-- Keep raw source recovery separate from transformed/reduced output recovery.
-- Treat `preserve="full"` as exact fidelity under caps, not unlimited context injection.
-- Preserve exact failure and verification evidence lines.
-- Keep `toolStatus`, `execution.status`, and `routing.status` separate.
-- Metadata-only records must never claim exact raw recovery.
-- Freeflow mode changes guidance strength only. It must not enable `postToolRouting`.
-- Do not write removed `capture` or `providers` config.
-- Do not offer `redacted`; it is future-only.
-
-## Recovery Pattern
-
-For routed run/native/observed/transform output, recover exact text with:
-
-```json
-{
-  "action": "retrieve",
-  "source": { "kind": "vault", "outputId": "ffout_...", "stream": "stdout|stderr|combined|raw" },
-  "lineRange": { "start": 1, "end": 20 },
-  "preserve": "full"
-}
-```
-
-When line numbers are unknown, use `query` or `get` first. Use `expand` when a returned evidence packet is too narrow. Use `explain` when recoverability is unclear.
-
-## Config
-
-Use `freeflow_status` to inspect effective config, vault writability/index state, script adapters, observed routing, and migration recommendations.
-
-Use Pi's `/output-router enable` or repository settings surface for initial enablement. Read [setup guidance](references/setup.md) before repairing capability config or changing optional routing features. After valid setup, Output Router owns explicit operational tuning requested through its capability surface.
-
-Persist operational config only after an explicit tuning request or a previously accepted setup branch:
-
-- `outputRouter`: `enabled`, `profile`, `postToolRouting`, `storagePolicy`, thresholds, vault root/retention, generated paths, noisy command hints.
-- `outputRouter.observedRouting`: explicit MCP servers and web/fetch/codeSearch producer persistence.
-- `outputRouter.scriptTransform`: enabled flag, languages, sandbox/network/limits/raw-script persistence.
-
-Minimal setup stays only `defaultMode`. Missing optional sections mean built-in defaults.
-
-Read [the safety policy](references/safety-policy.md) before changing routing policy, reviewing router behavior, or handling exactness-sensitive output.

package/capabilities/output-router/references/safety-policy.md

@@ -1,40 +0,0 @@
-# Output Router Safety Policy
-
-Reference-only policy for exactness-sensitive output routing. Pi runtime context does not load this whole file by default; the active `output-router` skill carries the compact rules. Read this file before changing routing policy, reviewing router behavior, or handling a new exactness-sensitive output class.
-
-## Main Rule
-
-Do not silently summarize, compress, or replace exactness-sensitive output.
-
-Exactness-sensitive output includes:
-
-- user-requested exact/full output,
-- small native outputs where direct output is expected,
-- verification output needed for completion claims,
-- failure evidence needed for diagnosis,
-- source-truth conflict evidence,
-- security, privacy, billing, data-loss, permissions, or public API evidence,
-- failed, timed-out, or cancelled run output,
-- script-producer output,
-- filtered, script-filtered, reduced, or transformed output where raw source matters,
-- anything marked `preserve: full`.
-
-## Required Behavior
-
-- Capture raw evidence before transformation whenever output is routed.
-- Label every transformed native or observed result as Freeflow-routed output.
-- Include an `outputId` and exact recovery instructions when exact recovery exists.
-- Preserve exact failure and verification evidence lines; do not paraphrase them.
-- Pass small native outputs through unchanged unless explicit config and thresholds route them.
-- For huge exactness-sensitive output, use vaulting and exact chunk retrieval instead of lossy summarization.
-- If post-tool native safety-net routing fails, fail open: return the native output unchanged when possible and include a warning.
-- Keep raw source recovery separate from transformed/reduced output recovery.
-- Metadata-only records must never claim exact raw recovery. Duplicate metadata may point to a prior exact `outputId`; plain metadata-only records get rerun or explanation guidance.
-
-## Non-Goals
-
-- This policy does not enable post-tool routing by itself.
-- This policy does not authorize model-assisted summarization inside the router runtime.
-- This policy does not authorize unsandboxed script execution.
-- This policy does not require Pi runtime context to inject the full reference every turn.
-- This policy does not replace host permissions, sandboxing, approvals, or workflow/decision gates.