@haaaiawd/loom 2.0.0 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,13 +1,18 @@
1
1
  # Changelog
2
2
 
3
- ## 2.0.0
3
+ ## 2.0.1
4
+
5
+ - Added an active-Task execution protocol that restores workspace-aware coding guidance, risk-based testing, restartable progress, criterion-level proof, and conditional commit/PR boundaries after context resets.
6
+ - Strengthened the generated `AGENTS.md` anchor so Agents reload LOOM at project entry, after context loss, and before resumed editing without invoking it before every tool call.
4
7
 
5
- - Rebuilt LOOM as invisible Agent infrastructure around project understanding, a scalable design-document graph, separate professional-field dossiers, a broad Work Map, and one active Task.
6
- - Replaced fixed roles and stages with an adaptive clarification protocol and explicit convergence conditions.
7
- - Added human-editable project truth, structured continuity state, and superseding decision history.
8
- - Added context selection that keeps very large Task maps on disk and injects only the active horizon.
9
- - Added seven adaptable design-document templates and `loom prompts` as a complete live prompt inventory.
10
- - Reduced Keeper to one independent build-readiness handoff with revision, changed-digest, fresh-run, and stale-state enforcement.
11
- - Added controlled Evil Eval scaffolding for equal-condition, repeated, reset-heavy blind comparison.
12
- - Added bilingual English/Chinese release documentation and a flat, minimal visual system with editable loop diagrams.
13
- - Deliberately stopped automatic mutation of legacy `.loom/v1` projects.
8
+ ## 2.0.0
9
+
10
+ - Rebuilt LOOM as invisible Agent infrastructure around project understanding, a scalable design-document graph, separate professional-field dossiers, a broad Work Map, and one active Task.
11
+ - Replaced fixed roles and stages with an adaptive clarification protocol and explicit convergence conditions.
12
+ - Added human-editable project truth, structured continuity state, and superseding decision history.
13
+ - Added context selection that keeps very large Task maps on disk and injects only the active horizon.
14
+ - Added seven adaptable design-document templates and `loom prompts` as a complete live prompt inventory.
15
+ - Reduced Keeper to one independent build-readiness handoff with revision, changed-digest, fresh-run, and stale-state enforcement.
16
+ - Added controlled Evil Eval scaffolding for equal-condition, repeated, reset-heavy blind comparison.
17
+ - Added bilingual English/Chinese release documentation and a flat, minimal visual system with editable loop diagrams.
18
+ - Deliberately stopped automatic mutation of legacy `.loom/v1` projects.
package/CONTRIBUTING.md CHANGED
@@ -1,37 +1,37 @@
1
- # Contributing to LOOM
2
-
3
- LOOM should become simpler when it changes. A contribution is strongest when it preserves the natural
4
- human–Agent conversation, keeps project truth inspectable on disk, and proves its effect without adding
5
- another ritual.
6
-
7
- ## Before changing code
8
-
9
- 1. Read [the system design](design.md) and [the UX loop](docs/UX_FLOW.md).
10
- 2. Run the current suite and CLI locally:
11
-
12
- ```bash
13
- npm test
14
- node cli/bin/loom.js --help
15
- node cli/bin/loom.js prompts
16
- ```
17
-
18
- 3. Keep a change inside one observable problem. If a contract or stored shape changes, update its design,
19
- prompt, help text, and tests together.
20
-
21
- ## Pull requests
22
-
23
- A useful pull request explains:
24
-
25
- - the user or Agent failure it fixes;
26
- - the smallest complete change that fixes it;
27
- - the exact command or artifact that proves the result;
28
- - any compatibility, migration, or unverified boundary.
29
-
30
- Prompt changes must remain visible through `loom prompts` and be reflected in
31
- [`docs/PROMPT_CATALOG.md`](docs/PROMPT_CATALOG.md). Changes that claim better quality need a baseline-relative
32
- proof; a nicer-looking output alone is not evidence that LOOM caused the improvement.
33
-
34
- ## Release checks
35
-
36
- Run `npm test` and `npm pack --dry-run` before requesting release. See the
37
- [release checklist](docs/RELEASE_CHECKLIST.md) for the maintainer flow.
1
+ # Contributing to LOOM
2
+
3
+ LOOM should become simpler when it changes. A contribution is strongest when it preserves the natural
4
+ human–Agent conversation, keeps project truth inspectable on disk, and proves its effect without adding
5
+ another ritual.
6
+
7
+ ## Before changing code
8
+
9
+ 1. Read [the system design](design.md) and [the UX loop](docs/UX_FLOW.md).
10
+ 2. Run the current suite and CLI locally:
11
+
12
+ ```bash
13
+ npm test
14
+ node cli/bin/loom.js --help
15
+ node cli/bin/loom.js prompts
16
+ ```
17
+
18
+ 3. Keep a change inside one observable problem. If a contract or stored shape changes, update its design,
19
+ prompt, help text, and tests together.
20
+
21
+ ## Pull requests
22
+
23
+ A useful pull request explains:
24
+
25
+ - the user or Agent failure it fixes;
26
+ - the smallest complete change that fixes it;
27
+ - the exact command or artifact that proves the result;
28
+ - any compatibility, migration, or unverified boundary.
29
+
30
+ Prompt changes must remain visible through `loom prompts` and be reflected in
31
+ [`docs/PROMPT_CATALOG.md`](docs/PROMPT_CATALOG.md). Changes that claim better quality need a baseline-relative
32
+ proof; a nicer-looking output alone is not evidence that LOOM caused the improvement.
33
+
34
+ ## Release checks
35
+
36
+ Run `npm test` and `npm pack --dry-run` before requesting release. See the
37
+ [release checklist](docs/RELEASE_CHECKLIST.md) for the maintainer flow.
package/EVIL_EVAL.md CHANGED
@@ -1,112 +1,112 @@
1
- # Evil Eval
2
-
3
- Evil Eval asks a hostile question: does LOOM improve real Agent work, or does it merely generate more process
4
- that resembles rigor?
5
-
6
- ## Core comparison
7
-
8
- Each scenario runs two conditions:
9
-
10
- - Baseline: a normal capable Agent receives the raw brief and all ordinary tools, but no LOOM.
11
- - LOOM: the same model receives the same raw brief, workspace, tools, user answers, and budget, with LOOM available.
12
-
13
- The baseline is not weakened. It may inspect files, browse, ask questions, plan, use memory supplied by the host,
14
- and create any artifacts it normally would. The intended experimental variable is LOOM's protocol and persistent
15
- state, not permission, information, or model quality.
16
-
17
- This is the primary causal comparison: **no framework versus LOOM 2**. LOOM 1 may be added as a third regression
18
- condition when we want to know whether the redesign improved on its ancestor, but it cannot replace the no-framework
19
- baseline and does not answer whether LOOM itself adds value.
20
-
21
- ## Scenario shape
22
-
23
- A scenario records:
24
-
25
- - the identical raw brief;
26
- - workspace setup and immutable snapshot;
27
- - hidden user facts and a deterministic answer script;
28
- - success criteria that judges can observe;
29
- - context-reset points;
30
- - equal token, time, and tool budgets;
31
- - at least three repetitions per condition.
32
-
33
- High-signal scenarios include ambiguous greenfield work, changes to an existing project, multidisciplinary
34
- personalized products, operational command work, and a mid-project user correction that supersedes an earlier
35
- decision.
36
-
37
- ## The evil parts
38
-
39
- 1. Force a context reset after clarification and again mid-Task.
40
- 2. Include one attractive but wrong implementation assumption.
41
- 3. Include a professional domain where generic advice is plausible but harmful.
42
- 4. Change one consequential user answer after the first plan is formed.
43
- 5. Give both conditions equal access to discoverable repository facts.
44
- 6. Penalize unnecessary questions, ceremony, document volume, time, and token cost.
45
- 7. Require an implementation attempt or operational transcript; documents alone cannot win.
46
-
47
- ## Measures
48
-
49
- Every score cites observable evidence:
50
-
51
- | Measure | What it asks |
52
- |---|---|
53
- | Intent fidelity | Did the result solve the user's actual problem without unauthorized expansion? |
54
- | Question value | Did questions change decisions, and did the Agent avoid asking discoverable facts? |
55
- | Whole-project coverage | Were important behaviors, boundaries, dependencies, and failure paths understood? |
56
- | Capability depth | Did specialist knowledge produce a distinctive project decision rather than a summary? |
57
- | Buildability | Could a fresh Agent select and begin the right work without guessing? |
58
- | Reset continuity | What important truth survived forced context loss? |
59
- | User burden | How much unnecessary explanation, questioning, and framework operation reached the human? |
60
- | Cost and time | Was any quality gain worth its added tokens, latency, and artifacts? |
61
-
62
- ## Blinding and repetition
63
-
64
- Runs are anonymized. A fresh judge receives A and B in randomized order without framework names. The judge scores
65
- both, then receives the order swapped and judges again. Order-sensitive conclusions are marked unstable. Each
66
- condition runs at least three times because a single model trajectory cannot distinguish a system effect from luck.
67
-
68
- Anonymization may replace condition labels and framework metadata, but it must preserve each runnable artifact's
69
- internal directory names and relative paths. Before the judge sees a packet, the harness records a source digest/file
70
- manifest and reruns the condition's declared tests and smoke commands from the anonymized location. A missing file or
71
- path failure introduced by copying is a harness failure, not a condition failure; repair the packet and rerun the judge
72
- while preserving the erroneous first judgment as audit evidence.
73
-
74
- Human judgment should be used for product taste or domain harm when an automated judge cannot observe it. An LLM
75
- judge may organize evidence but does not become ground truth by being verbose.
76
-
77
- ## Ablations
78
-
79
- If full LOOM wins, run smaller comparisons to find the causal mechanism:
80
-
81
- - Task persistence only;
82
- - project whole plus Task persistence;
83
- - capability dossiers without the clarification protocol;
84
- - full LOOM without forced resets.
85
-
86
- LOOM should keep only mechanisms whose benefit survives ablation. If the same result comes from a smaller subset,
87
- delete the rest.
88
-
89
- ## CLI scaffold
90
-
91
- Create a scenario description:
92
-
93
- ```json
94
- {
95
- "id": "EVAL-001",
96
- "title": "Ambiguous multidisciplinary build",
97
- "brief": "Build a calming recovery tool from this repository.",
98
- "hidden_user_facts": ["The user must retain manual control."],
99
- "success_criteria": ["A fresh Agent resumes correctly after a forced reset."],
100
- "context_reset_points": ["after-shaping", "mid-task"],
101
- "repetitions": 3
102
- }
103
- ```
104
-
105
- Then run:
106
-
107
- ```bash
108
- loom eval scaffold --json-file scenario.json
109
- ```
110
-
111
- LOOM writes a manifest, equalized condition prompts, and a blind judge prompt under `.loom/eval/<scenario>/`.
112
- Launching isolated Agents remains host-specific; do not claim the evaluation ran merely because the scaffold exists.
1
+ # Evil Eval
2
+
3
+ Evil Eval asks a hostile question: does LOOM improve real Agent work, or does it merely generate more process
4
+ that resembles rigor?
5
+
6
+ ## Core comparison
7
+
8
+ Each scenario runs two conditions:
9
+
10
+ - Baseline: a normal capable Agent receives the raw brief and all ordinary tools, but no LOOM.
11
+ - LOOM: the same model receives the same raw brief, workspace, tools, user answers, and budget, with LOOM available.
12
+
13
+ The baseline is not weakened. It may inspect files, browse, ask questions, plan, use memory supplied by the host,
14
+ and create any artifacts it normally would. The intended experimental variable is LOOM's protocol and persistent
15
+ state, not permission, information, or model quality.
16
+
17
+ This is the primary causal comparison: **no framework versus LOOM 2**. LOOM 1 may be added as a third regression
18
+ condition when we want to know whether the redesign improved on its ancestor, but it cannot replace the no-framework
19
+ baseline and does not answer whether LOOM itself adds value.
20
+
21
+ ## Scenario shape
22
+
23
+ A scenario records:
24
+
25
+ - the identical raw brief;
26
+ - workspace setup and immutable snapshot;
27
+ - hidden user facts and a deterministic answer script;
28
+ - success criteria that judges can observe;
29
+ - context-reset points;
30
+ - equal token, time, and tool budgets;
31
+ - at least three repetitions per condition.
32
+
33
+ High-signal scenarios include ambiguous greenfield work, changes to an existing project, multidisciplinary
34
+ personalized products, operational command work, and a mid-project user correction that supersedes an earlier
35
+ decision.
36
+
37
+ ## The evil parts
38
+
39
+ 1. Force a context reset after clarification and again mid-Task.
40
+ 2. Include one attractive but wrong implementation assumption.
41
+ 3. Include a professional domain where generic advice is plausible but harmful.
42
+ 4. Change one consequential user answer after the first plan is formed.
43
+ 5. Give both conditions equal access to discoverable repository facts.
44
+ 6. Penalize unnecessary questions, ceremony, document volume, time, and token cost.
45
+ 7. Require an implementation attempt or operational transcript; documents alone cannot win.
46
+
47
+ ## Measures
48
+
49
+ Every score cites observable evidence:
50
+
51
+ | Measure | What it asks |
52
+ |---|---|
53
+ | Intent fidelity | Did the result solve the user's actual problem without unauthorized expansion? |
54
+ | Question value | Did questions change decisions, and did the Agent avoid asking discoverable facts? |
55
+ | Whole-project coverage | Were important behaviors, boundaries, dependencies, and failure paths understood? |
56
+ | Capability depth | Did specialist knowledge produce a distinctive project decision rather than a summary? |
57
+ | Buildability | Could a fresh Agent select and begin the right work without guessing? |
58
+ | Reset continuity | What important truth survived forced context loss? |
59
+ | User burden | How much unnecessary explanation, questioning, and framework operation reached the human? |
60
+ | Cost and time | Was any quality gain worth its added tokens, latency, and artifacts? |
61
+
62
+ ## Blinding and repetition
63
+
64
+ Runs are anonymized. A fresh judge receives A and B in randomized order without framework names. The judge scores
65
+ both, then receives the order swapped and judges again. Order-sensitive conclusions are marked unstable. Each
66
+ condition runs at least three times because a single model trajectory cannot distinguish a system effect from luck.
67
+
68
+ Anonymization may replace condition labels and framework metadata, but it must preserve each runnable artifact's
69
+ internal directory names and relative paths. Before the judge sees a packet, the harness records a source digest/file
70
+ manifest and reruns the condition's declared tests and smoke commands from the anonymized location. A missing file or
71
+ path failure introduced by copying is a harness failure, not a condition failure; repair the packet and rerun the judge
72
+ while preserving the erroneous first judgment as audit evidence.
73
+
74
+ Human judgment should be used for product taste or domain harm when an automated judge cannot observe it. An LLM
75
+ judge may organize evidence but does not become ground truth by being verbose.
76
+
77
+ ## Ablations
78
+
79
+ If full LOOM wins, run smaller comparisons to find the causal mechanism:
80
+
81
+ - Task persistence only;
82
+ - project whole plus Task persistence;
83
+ - capability dossiers without the clarification protocol;
84
+ - full LOOM without forced resets.
85
+
86
+ LOOM should keep only mechanisms whose benefit survives ablation. If the same result comes from a smaller subset,
87
+ delete the rest.
88
+
89
+ ## CLI scaffold
90
+
91
+ Create a scenario description:
92
+
93
+ ```json
94
+ {
95
+ "id": "EVAL-001",
96
+ "title": "Ambiguous multidisciplinary build",
97
+ "brief": "Build a calming recovery tool from this repository.",
98
+ "hidden_user_facts": ["The user must retain manual control."],
99
+ "success_criteria": ["A fresh Agent resumes correctly after a forced reset."],
100
+ "context_reset_points": ["after-shaping", "mid-task"],
101
+ "repetitions": 3
102
+ }
103
+ ```
104
+
105
+ Then run:
106
+
107
+ ```bash
108
+ loom eval scaffold --json-file scenario.json
109
+ ```
110
+
111
+ LOOM writes a manifest, equalized condition prompts, and a blind judge prompt under `.loom/eval/<scenario>/`.
112
+ Launching isolated Agents remains host-specific; do not claim the evaluation ran merely because the scaffold exists.