@haaaiawd/loom 2.0.0 → 2.1.0

package/CHANGELOG.md

@@ -1,5 +1,32 @@
 # Changelog
 
+## Unreleased
+
+- Added environment-aware clarification through `loom context --human-channel available|unavailable`. Unattended Agents inspect first, research only permitted objective facts, and then use bounded assumptions or blocks rather than fabricating users or substituting web search for intent.
+- Added `--state-dir <outside-workspace-dir>` sidecars for isolated benchmark and sandbox runs. Sidecars preserve virtual `.loom/` Task references without mutating the scored workspace or its `AGENTS.md`.
+- Added equal human-channel controls and unattended condition prompts to Evil Eval scaffolding.
+
+## 2.1.0
+
+- Added a current-state and recommended-action header to `loom context` output. Agents entering a project or resuming after compression now see project status, active task, work-map counts, and a recommended next action before the full protocol. The recommendation is advisory, not a script.
+- Rewrote `AGENTS.md` with explicit LOOM trigger conditions (project entry, context reset, before substantial work, between Tasks) and a standard work rhythm that keeps Agent judgment while making the next step obvious.
+- Integrated Crux integrity-chain review into the Keeper handoff. The Keeper now checks that responsible intent, project promises, design and capability decisions, Work Map, executable behavior, and human feedback loop form an intact chain, and looks for omission, substitution, drift, unsupported leaps, blindness, and ownerless gaps.
+- Added staged visibility and review guidance to the project template and active-Task execution protocol. Tasks should prefer human-visible acceptance evidence, the Agent should show real working things at milestones, run `loom check` and project tests together before declaring a batch done, and prioritize reaching an exciting surface early to sustain the human's momentum.
+- Strengthened capability source validation: `loom capability synthesize` now requires each decision-tree node's `source:` citation to reference a real `.md` file in the dossier's `research/` directory.
+- Added `research/_guide.md` generation in `loom capability research` so Agents know what good research material looks like, what to avoid, and how to cite sources.
+- Added `.loom/STRUCTURE.md` as a fifth project-truth layer declaring where source code, tests, docs, assets, and configuration files live. `loom check` warns when it is missing or still templated.
+- Added `loom decision --json-file` for recording consequential superseding decisions with affected files and tasks. `loom check` warns when a done Task is marked affected by a later decision.
+- Added `acceptance[]` as the primary Task completion structure, pairing `criterion`, `verify_by`, and `evidence`. Legacy `done_when[]` remains supported.
+- Added Keeper auto-pass: when all Keeper gaps are minor and 3 or fewer, fixing them and running `loom project ready` auto-passes without another Keeper round.
+- Added deliverable coverage tracking through `loom deliverable add` and Task `covers` field. `loom check` reports uncovered delivery units.
+- Added `loom capability research → synthesize → confirm` lifecycle with status tracking per dossier.
+- Updated help text with a typical first-pass flow and capability lifecycle sequence.
+
+## 2.0.1
+
+- Added an active-Task execution protocol that restores workspace-aware coding guidance, risk-based testing, restartable progress, criterion-level proof, and conditional commit/PR boundaries after context resets.
+- Strengthened the generated `AGENTS.md` anchor so Agents reload LOOM at project entry, after context loss, and before resumed editing without invoking it before every tool call.
+
 ## 2.0.0
 
 - Rebuilt LOOM as invisible Agent infrastructure around project understanding, a scalable design-document graph, separate professional-field dossiers, a broad Work Map, and one active Task.

package/CONTRIBUTING.md

@@ -1,37 +1,37 @@
-# Contributing to LOOM
-
-LOOM should become simpler when it changes. A contribution is strongest when it preserves the natural
-human–Agent conversation, keeps project truth inspectable on disk, and proves its effect without adding
-another ritual.
-
-## Before changing code
-
-1. Read [the system design](design.md) and [the UX loop](docs/UX_FLOW.md).
-2. Run the current suite and CLI locally:
-
-   ```bash
-   npm test
-   node cli/bin/loom.js --help
-   node cli/bin/loom.js prompts
-   ```
-
-3. Keep a change inside one observable problem. If a contract or stored shape changes, update its design,
-   prompt, help text, and tests together.
-
-## Pull requests
-
-A useful pull request explains:
-
-- the user or Agent failure it fixes;
-- the smallest complete change that fixes it;
-- the exact command or artifact that proves the result;
-- any compatibility, migration, or unverified boundary.
-
-Prompt changes must remain visible through `loom prompts` and be reflected in
-[`docs/PROMPT_CATALOG.md`](docs/PROMPT_CATALOG.md). Changes that claim better quality need a baseline-relative
-proof; a nicer-looking output alone is not evidence that LOOM caused the improvement.
-
-## Release checks
-
-Run `npm test` and `npm pack --dry-run` before requesting release. See the
-[release checklist](docs/RELEASE_CHECKLIST.md) for the maintainer flow.
+# Contributing to LOOM
+
+LOOM should become simpler when it changes. A contribution is strongest when it preserves the natural
+human–Agent conversation, keeps project truth inspectable on disk, and proves its effect without adding
+another ritual.
+
+## Before changing code
+
+1. Read [the system design](design.md) and [the UX loop](docs/UX_FLOW.md).
+2. Run the current suite and CLI locally:
+
+   ```bash
+   npm test
+   node cli/bin/loom.js --help
+   node cli/bin/loom.js prompts
+   ```
+
+3. Keep a change inside one observable problem. If a contract or stored shape changes, update its design,
+   prompt, help text, and tests together.
+
+## Pull requests
+
+A useful pull request explains:
+
+- the user or Agent failure it fixes;
+- the smallest complete change that fixes it;
+- the exact command or artifact that proves the result;
+- any compatibility, migration, or unverified boundary.
+
+Prompt changes must remain visible through `loom prompts` and be reflected in
+[`docs/PROMPT_CATALOG.md`](docs/PROMPT_CATALOG.md). Changes that claim better quality need a baseline-relative
+proof; a nicer-looking output alone is not evidence that LOOM caused the improvement.
+
+## Release checks
+
+Run `npm test` and `npm pack --dry-run` before requesting release. See the
+[release checklist](docs/RELEASE_CHECKLIST.md) for the maintainer flow.

package/EVIL_EVAL.md

@@ -1,112 +1,125 @@
-# Evil Eval
-
-Evil Eval asks a hostile question: does LOOM improve real Agent work, or does it merely generate more process
-that resembles rigor?
-
-## Core comparison
-
-Each scenario runs two conditions:
-
-- Baseline: a normal capable Agent receives the raw brief and all ordinary tools, but no LOOM.
-- LOOM: the same model receives the same raw brief, workspace, tools, user answers, and budget, with LOOM available.
-
-The baseline is not weakened. It may inspect files, browse, ask questions, plan, use memory supplied by the host,
-and create any artifacts it normally would. The intended experimental variable is LOOM's protocol and persistent
-state, not permission, information, or model quality.
-
-This is the primary causal comparison: **no framework versus LOOM 2**. LOOM 1 may be added as a third regression
-condition when we want to know whether the redesign improved on its ancestor, but it cannot replace the no-framework
-baseline and does not answer whether LOOM itself adds value.
-
-## Scenario shape
-
-A scenario records:
-
+# Evil Eval
+
+Evil Eval asks a hostile question: does LOOM improve real Agent work, or does it merely generate more process
+that resembles rigor?
+
+## Core comparison
+
+Each scenario runs two conditions:
+
+- Baseline: a normal capable Agent receives the raw brief and all ordinary tools, but no LOOM.
+- LOOM: the same model receives the same raw brief, workspace, tools, user answers, and budget, with LOOM available.
+
+The baseline is not weakened. It may inspect files, browse, ask questions, plan, use memory supplied by the host,
+and create any artifacts it normally would. The intended experimental variable is LOOM's protocol and persistent
+state, not permission, information, or model quality.
+
+This is the primary causal comparison: **no framework versus LOOM 2**. LOOM 1 may be added as a third regression
+condition when we want to know whether the redesign improved on its ancestor, but it cannot replace the no-framework
+baseline and does not answer whether LOOM itself adds value.
+
+## Scenario shape
+
+A scenario records:
+
 - the identical raw brief;
 - workspace setup and immutable snapshot;
 - hidden user facts and a deterministic answer script;
+- human-channel availability (`available` or `unavailable`), identical across conditions;
 - success criteria that judges can observe;
-- context-reset points;
-- equal token, time, and tool budgets;
-- at least three repetitions per condition.
-
-High-signal scenarios include ambiguous greenfield work, changes to an existing project, multidisciplinary
-personalized products, operational command work, and a mid-project user correction that supersedes an earlier
-decision.
-
-## The evil parts
-
-1. Force a context reset after clarification and again mid-Task.
-2. Include one attractive but wrong implementation assumption.
-3. Include a professional domain where generic advice is plausible but harmful.
-4. Change one consequential user answer after the first plan is formed.
-5. Give both conditions equal access to discoverable repository facts.
-6. Penalize unnecessary questions, ceremony, document volume, time, and token cost.
+- context-reset points;
+- equal token, time, and tool budgets;
+- at least three repetitions per condition.
+
+High-signal scenarios include ambiguous greenfield work, changes to an existing project, multidisciplinary
+personalized products, operational command work, and a mid-project user correction that supersedes an earlier
+decision.
+
+## The evil parts
+
+1. Force a context reset after clarification and again mid-Task.
+2. Include one attractive but wrong implementation assumption.
+3. Include a professional domain where generic advice is plausible but harmful.
+4. Change one consequential user answer after the first plan is formed.
+5. Give both conditions equal access to discoverable repository facts.
+6. Penalize unnecessary questions, ceremony, document volume, time, and token cost.
 7. Require an implementation attempt or operational transcript; documents alone cannot win.
 
-## Measures
-
-Every score cites observable evidence:
-
-| Measure | What it asks |
-|---|---|
-| Intent fidelity | Did the result solve the user's actual problem without unauthorized expansion? |
-| Question value | Did questions change decisions, and did the Agent avoid asking discoverable facts? |
-| Whole-project coverage | Were important behaviors, boundaries, dependencies, and failure paths understood? |
-| Capability depth | Did specialist knowledge produce a distinctive project decision rather than a summary? |
-| Buildability | Could a fresh Agent select and begin the right work without guessing? |
-| Reset continuity | What important truth survived forced context loss? |
-| User burden | How much unnecessary explanation, questioning, and framework operation reached the human? |
-| Cost and time | Was any quality gain worth its added tokens, latency, and artifacts? |
-
-## Blinding and repetition
-
-Runs are anonymized. A fresh judge receives A and B in randomized order without framework names. The judge scores
-both, then receives the order swapped and judges again. Order-sensitive conclusions are marked unstable. Each
-condition runs at least three times because a single model trajectory cannot distinguish a system effect from luck.
-
-Anonymization may replace condition labels and framework metadata, but it must preserve each runnable artifact's
-internal directory names and relative paths. Before the judge sees a packet, the harness records a source digest/file
-manifest and reruns the condition's declared tests and smoke commands from the anonymized location. A missing file or
-path failure introduced by copying is a harness failure, not a condition failure; repair the packet and rerun the judge
-while preserving the erroneous first judgment as audit evidence.
-
-Human judgment should be used for product taste or domain harm when an automated judge cannot observe it. An LLM
-judge may organize evidence but does not become ground truth by being verbose.
-
-## Ablations
-
-If full LOOM wins, run smaller comparisons to find the causal mechanism:
-
-- Task persistence only;
-- project whole plus Task persistence;
-- capability dossiers without the clarification protocol;
-- full LOOM without forced resets.
-
-LOOM should keep only mechanisms whose benefit survives ablation. If the same result comes from a smaller subset,
-delete the rest.
-
-## CLI scaffold
-
-Create a scenario description:
-
-```json
-{
-  "id": "EVAL-001",
+## Unattended external benchmarks
+
+An external benchmark such as a terminal-use suite may provide no human response channel. In that condition,
+set `human_channel` to `unavailable` for both arms. Neither arm may invent a user answer or treat web search as
+a substitute for user intent, preference, or authority. Both may inspect the workspace and use only the research
+tools the benchmark makes available; unresolved consequential choices require a bounded reversible assumption or
+a block.
+
+Run LOOM with `--state-dir` pointing outside the scored workspace. The sidecar must be fresh per run and passed
+to every LOOM command, so `.loom/` state cannot alter task files or hidden-test behavior.
+
+## Measures
+
+Every score cites observable evidence:
+
+| Measure | What it asks |
+|---|---|
+| Intent fidelity | Did the result solve the user's actual problem without unauthorized expansion? |
+| Question value | Did questions change decisions, and did the Agent avoid asking discoverable facts? |
+| Whole-project coverage | Were important behaviors, boundaries, dependencies, and failure paths understood? |
+| Capability depth | Did specialist knowledge produce a distinctive project decision rather than a summary? |
+| Buildability | Could a fresh Agent select and begin the right work without guessing? |
+| Reset continuity | What important truth survived forced context loss? |
+| User burden | How much unnecessary explanation, questioning, and framework operation reached the human? |
+| Cost and time | Was any quality gain worth its added tokens, latency, and artifacts? |
+
+## Blinding and repetition
+
+Runs are anonymized. A fresh judge receives A and B in randomized order without framework names. The judge scores
+both, then receives the order swapped and judges again. Order-sensitive conclusions are marked unstable. Each
+condition runs at least three times because a single model trajectory cannot distinguish a system effect from luck.
+
+Anonymization may replace condition labels and framework metadata, but it must preserve each runnable artifact's
+internal directory names and relative paths. Before the judge sees a packet, the harness records a source digest/file
+manifest and reruns the condition's declared tests and smoke commands from the anonymized location. A missing file or
+path failure introduced by copying is a harness failure, not a condition failure; repair the packet and rerun the judge
+while preserving the erroneous first judgment as audit evidence.
+
+Human judgment should be used for product taste or domain harm when an automated judge cannot observe it. An LLM
+judge may organize evidence but does not become ground truth by being verbose.
+
+## Ablations
+
+If full LOOM wins, run smaller comparisons to find the causal mechanism:
+
+- Task persistence only;
+- project whole plus Task persistence;
+- capability dossiers without the clarification protocol;
+- full LOOM without forced resets.
+
+LOOM should keep only mechanisms whose benefit survives ablation. If the same result comes from a smaller subset,
+delete the rest.
+
+## CLI scaffold
+
+Create a scenario description:
+
+```json
+{
+  "id": "EVAL-001",
   "title": "Ambiguous multidisciplinary build",
   "brief": "Build a calming recovery tool from this repository.",
   "hidden_user_facts": ["The user must retain manual control."],
+  "human_channel": "available",
   "success_criteria": ["A fresh Agent resumes correctly after a forced reset."],
-  "context_reset_points": ["after-shaping", "mid-task"],
-  "repetitions": 3
-}
-```
-
-Then run:
-
-```bash
-loom eval scaffold --json-file scenario.json
-```
-
-LOOM writes a manifest, equalized condition prompts, and a blind judge prompt under `.loom/eval/<scenario>/`.
-Launching isolated Agents remains host-specific; do not claim the evaluation ran merely because the scaffold exists.
+  "context_reset_points": ["after-shaping", "mid-task"],
+  "repetitions": 3
+}
+```
+
+Then run:
+
+```bash
+loom eval scaffold --json-file scenario.json
+```
+
+LOOM writes a manifest, equalized condition prompts, and a blind judge prompt under `.loom/eval/<scenario>/`.
+Launching isolated Agents remains host-specific; do not claim the evaluation ran merely because the scaffold exists.