@hassangameryt/freeflow 0.6.0 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +3 -2
- package/.codex-plugin/plugin.json +5 -6
- package/.cursor-plugin/plugin.json +25 -0
- package/README.md +66 -56
- package/capabilities/cognitive-routing/SKILL.md +288 -130
- package/com.github.copilot/hooks/hooks.json +11 -0
- package/command-surface.json +0 -24
- package/gemini-extension.json +5 -0
- package/hooks/adapters/claude-session-start.mjs +16 -0
- package/hooks/adapters/codex-session-start.mjs +16 -0
- package/hooks/adapters/copilot-session-start.mjs +13 -0
- package/hooks/adapters/cursor-session-start.mjs +13 -0
- package/hooks/adapters/gemini-session-start.mjs +15 -0
- package/hooks/claude/hooks.json +18 -0
- package/hooks/codex/hooks.json +18 -0
- package/hooks/cursor/hooks.json +9 -0
- package/hooks/freeflow-runtime-context.mjs +3 -650
- package/hooks/hooks.json +4 -30
- package/hooks/shared/runtime-context.mjs +210 -0
- package/hooks/tests/check-host-adapters.test.mjs +82 -0
- package/hooks/tests/check-runtime-context-hook.sh +140 -320
- package/package.json +20 -4
- package/pi-extension/dist/cognitive-routing/commands.js +3 -3
- package/pi-extension/dist/cognitive-routing/config.js +16 -20
- package/pi-extension/dist/cognitive-routing/controller.js +478 -48
- package/pi-extension/dist/cognitive-routing/history.js +12 -0
- package/pi-extension/dist/cognitive-routing/host.js +74 -12
- package/pi-extension/dist/cognitive-routing/pi-routing-session.js +414 -0
- package/pi-extension/dist/cognitive-routing/pi-session-control.js +320 -0
- package/pi-extension/dist/cognitive-routing/tool.js +1 -0
- package/pi-extension/dist/cognitive-routing/types.js +1 -1
- package/pi-extension/dist/index.js +236 -219
- package/pi-extension/dist/runtime/runtime-context.js +126 -286
- package/pi-extension/dist/settings/settings-ui.js +52 -336
- package/pi-extension/src/cognitive-routing/commands.ts +15 -5
- package/pi-extension/src/cognitive-routing/config.ts +16 -20
- package/pi-extension/src/cognitive-routing/controller.ts +561 -51
- package/pi-extension/src/cognitive-routing/history.ts +12 -0
- package/pi-extension/src/cognitive-routing/host.ts +110 -16
- package/pi-extension/src/cognitive-routing/pi-routing-session.ts +538 -0
- package/pi-extension/src/cognitive-routing/pi-session-control.ts +444 -0
- package/pi-extension/src/cognitive-routing/tool.ts +1 -0
- package/pi-extension/src/cognitive-routing/types.ts +1 -1
- package/pi-extension/src/index.ts +249 -239
- package/pi-extension/src/runtime/runtime-context.ts +129 -305
- package/pi-extension/src/settings/settings-ui.ts +53 -371
- package/pi-extension/tests/cognitive-routing/commands.test.js +1 -1
- package/pi-extension/tests/cognitive-routing/controller.test.js +69 -40
- package/pi-extension/tests/cognitive-routing/extension-lifecycle.test.js +128 -57
- package/pi-extension/tests/cognitive-routing/pi-normal-lifecycle.test.js +434 -0
- package/pi-extension/tests/cognitive-routing/pi-session-control.test.js +174 -0
- package/pi-extension/tests/cognitive-routing/pi-session-history.test.js +91 -0
- package/pi-extension/tests/cognitive-routing/profile-resolution.test.js +47 -4
- package/pi-extension/tests/integration/mode-free-core.test.js +218 -0
- package/pi-extension/tests/integration/pi-extension.test.js +430 -2478
- package/pi-extension/tests/integration/prompt-architecture.test.js +124 -79
- package/pi-extension/tests/integration/unaffected-regressions.test.js +411 -0
- package/plugin.json +33 -0
- package/runtime/prompts/cognitive-routing.md +5 -4
- package/runtime/prompts/conversation-history.md +1 -1
- package/runtime/prompts/core.md +77 -6
- package/runtime/prompts/interaction-contract.md +6 -2
- package/skills/action-selection/SKILL.md +93 -44
- package/skills/action-selection/references/trajectory-stalls.md +71 -0
- package/skills/bypass/SKILL.md +109 -39
- package/skills/commit-work/SKILL.md +94 -55
- package/skills/commit-work/references/git-state-edges.md +94 -0
- package/skills/commit-work/references/simple-push.md +92 -0
- package/skills/commit-work/references/staging-decisions.md +79 -46
- package/skills/decision-gate/SKILL.md +80 -22
- package/skills/design-for-depth/SKILL.md +208 -43
- package/skills/design-for-depth/references/design-pressure-signals.md +108 -240
- package/skills/design-for-depth/references/interface-design-loop.md +77 -97
- package/skills/design-for-depth/references/module-and-dependency-design.md +147 -0
- package/skills/design-for-depth/references/software-design-philosophy.md +127 -173
- package/skills/design-for-depth/references/state-and-failure-boundaries.md +136 -0
- package/skills/diagnose-failure/SKILL.md +198 -52
- package/skills/diagnose-failure/references/diagnostic-loop-catalog.md +19 -5
- package/skills/diagnose-failure/references/flaky-and-performance-diagnosis.md +13 -5
- package/skills/discuss/SKILL.md +118 -76
- package/skills/discuss/references/discussion-continuity.md +84 -32
- package/skills/evaluate-skill/SKILL.md +166 -52
- package/skills/evaluate-skill/references/evaluation-definition-schema.md +2 -2
- package/skills/evaluate-skill/references/evaluation-design.md +56 -30
- package/skills/evaluate-skill/references/execution-and-evidence.md +1 -1
- package/skills/evaluate-skill/references/review-and-revision.md +62 -21
- package/skills/execute-work/SKILL.md +98 -59
- package/skills/execute-work/references/code-practices.md +4 -4
- package/skills/execute-work/references/execute-work-edges.md +42 -22
- package/skills/review-artifact/SKILL.md +132 -71
- package/skills/review-artifact/references/adjudicate-artifact-review.md +101 -0
- package/skills/review-artifact/references/independent-artifact-reviewer-contract.md +52 -41
- package/skills/review-work/SKILL.md +115 -72
- package/skills/review-work/references/adjudicate-work-review.md +97 -0
- package/skills/review-work/references/independent-work-reviewer-contract.md +40 -35
- package/skills/review-work/references/security-risk-lens.md +17 -15
- package/skills/setup-freeflow/SKILL.md +14 -17
- package/skills/setup-freeflow/references/activation-contract.md +24 -30
- package/skills/setup-freeflow/references/host-setup.md +6 -7
- package/skills/simplify-code/SKILL.md +97 -39
- package/skills/simplify-code/references/simplification-patterns.md +1 -1
- package/skills/tdd/SKILL.md +108 -41
- package/skills/tdd/references/test-design.md +1 -1
- package/skills/track-work/SKILL.md +133 -300
- package/skills/track-work/references/working-record-format.md +515 -0
- package/skills/track-work/scripts/lib/document.mjs +713 -0
- package/skills/track-work/scripts/lib/format.mjs +166 -0
- package/skills/track-work/scripts/lib/store.mjs +233 -0
- package/skills/track-work/scripts/lib/transitions.mjs +790 -0
- package/skills/track-work/scripts/lib/views.mjs +27 -430
- package/skills/track-work/scripts/working-record.mjs +292 -98
- package/skills/verify-work/SKILL.md +46 -20
- package/skills/verify-work/references/browser-runtime-evidence.md +1 -1
- package/skills/verify-work/references/integration-evidence.md +1 -7
- package/skills/verify-work/references/performance-evidence.md +1 -1
- package/skills/workflow/SKILL.md +110 -80
- package/skills/workflow/references/domain-skill-composition.md +24 -18
- package/skills/write-skill/SKILL.md +218 -62
- package/skills/write-skill/references/activation-boundaries.md +63 -21
- package/skills/write-skill/references/agent-first-instructions.md +61 -27
- package/skills/write-skill/references/progressive-disclosure.md +91 -41
- package/skills/write-skill/references/skill-dependency-graphs.md +190 -0
- package/runtime/prompts/skills.md +0 -65
- package/skills/mode-contract/SKILL.md +0 -104
- package/skills/track-work/scripts/lib/codec.mjs +0 -570
- package/skills/track-work/scripts/lib/commands.mjs +0 -568
- package/skills/track-work/scripts/lib/contract.mjs +0 -303
- package/skills/track-work/scripts/lib/model.mjs +0 -506
- package/skills/track-work/scripts/lib/operations.mjs +0 -965
- package/skills/track-work/scripts/lib/record-store.mjs +0 -264
- package/skills/track-work/scripts/lib/result.mjs +0 -37
- package/skills/track-work/scripts/lib/rewrites.mjs +0 -740
- package/skills/track-work/scripts/lib/schema.mjs +0 -277
- package/skills/track-work/scripts/lib/source-inventory.mjs +0 -162
- package/skills/track-work/scripts/lib/workspace.mjs +0 -134
|
@@ -1,78 +1,168 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: diagnose-failure
|
|
3
|
-
description: Use when a bug, failure, regression, performance problem, or repeated unsuccessful
|
|
3
|
+
description: Use when a bug, failure, regression, performance problem, or repeated unsuccessful correction lacks a supported cause.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Diagnose Failure
|
|
7
7
|
|
|
8
8
|
Establish what is failing and why before selecting a correction.
|
|
9
9
|
|
|
10
|
-
A **diagnostic loop** is a repeatable
|
|
10
|
+
A **diagnostic loop** is a repeatable observation that can support, contradict, or leave unresolved a hypothesis at the boundary where the failure matters.
|
|
11
11
|
|
|
12
|
-
A
|
|
12
|
+
A **supported cause** is a causal explanation strong enough to distinguish meaningful alternatives and bound the next action. It does not require impossible certainty.
|
|
13
13
|
|
|
14
|
-
|
|
14
|
+
A requested patch, plausible code path, reviewer theory, observation consistent with one hypothesis, or one favorable rerun is not a supported cause.
|
|
15
15
|
|
|
16
|
-
|
|
16
|
+
Diagnosis owns the hypothesis, causal interpretation, and correction boundary. It does not authorize active evidence generation or correction.
|
|
17
17
|
|
|
18
|
-
##
|
|
18
|
+
## Enter Only When Cause Is Unsettled
|
|
19
|
+
|
|
20
|
+
Use diagnosis for:
|
|
21
|
+
|
|
22
|
+
- an unexplained bug, failed test, regression, or runtime failure;
|
|
23
|
+
- flaky, timing-dependent, environmental, or performance behavior;
|
|
24
|
+
- contradictory evidence;
|
|
25
|
+
- related corrections that keep failing;
|
|
26
|
+
- patches that repeatedly expose additional state or coordination;
|
|
27
|
+
- review findings whose shared cause remains unsupported.
|
|
28
|
+
|
|
29
|
+
Do not diagnose an ordinary mistake when fresh evidence already establishes one clear local defect and accepted behavior is settled. Return that correction to [Workflow](../../skills/workflow/SKILL.md).
|
|
30
|
+
|
|
31
|
+
Review count, finding count, or several bugs in one area do not by themselves prove a shared or structural cause.
|
|
32
|
+
|
|
33
|
+
Diagnosis may recur inside one Slice. A new hypothesis or observation does not create a new Slice by itself. When a Working Record exists, use [Track Work](../../skills/track-work/SKILL.md) to preserve the current hypothesis, evidence, blocker, and next distinguishing action.
|
|
34
|
+
|
|
35
|
+
## Establish The Failure Boundary
|
|
36
|
+
|
|
37
|
+
Before generating new evidence, identify:
|
|
38
|
+
|
|
39
|
+
- the failure claim;
|
|
40
|
+
- accepted expected behavior and its source;
|
|
41
|
+
- observed behavior and existing evidence;
|
|
42
|
+
- reported input, state, environment, and reproduction path;
|
|
43
|
+
- the strongest boundary currently observed;
|
|
44
|
+
- differences between the reported path and available reproduction;
|
|
45
|
+
- evidence that already contradicts possible explanations.
|
|
46
|
+
|
|
47
|
+
Prefer the reported path, environment, and observer. A nearby failure may suggest a hypothesis but does not establish the reported failure.
|
|
48
|
+
|
|
49
|
+
Reduced-fidelity reproduction remains diagnostic evidence only. State what it cannot prove.
|
|
50
|
+
|
|
51
|
+
Allowed behavior is not reproduction. A cache hit does not prove a stale-read defect when caching is permitted. A possible race in source does not explain a report until timing, traces, state, or accepted behavior connects it.
|
|
52
|
+
|
|
53
|
+
## State Competing Hypotheses
|
|
54
|
+
|
|
55
|
+
State one falsifiable leading hypothesis and only the viable alternatives the next observation must distinguish.
|
|
56
|
+
|
|
57
|
+
For each, identify:
|
|
19
58
|
|
|
20
59
|
```text
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
-> [Authority envelope covers its effects?]
|
|
26
|
-
-> no -> [Propose purpose, expected evidence, and stop condition]
|
|
27
|
-
-> [Ask once and wait]
|
|
28
|
-
-> yes -> [Run the observation]
|
|
29
|
-
-> [Route from evidence]
|
|
30
|
-
-> hypothesis contradicted -> refine or replace it -> choose again
|
|
31
|
-
-> evidence unavailable -> stop with the smallest missing evidence
|
|
32
|
-
-> cause supported -> return cause and correction boundary to Workflow
|
|
33
|
-
-> user-owned behavior or source conflict -> Decision Gate
|
|
34
|
-
-> structural cause -> Design for Depth
|
|
35
|
-
-> active harm before cause -> propose bounded containment
|
|
60
|
+
Hypothesis:
|
|
61
|
+
Predicted observation:
|
|
62
|
+
Observation that would contradict it:
|
|
63
|
+
Meaningful alternative:
|
|
36
64
|
```
|
|
37
65
|
|
|
38
|
-
|
|
66
|
+
Do not create a hypothesis around the requested patch and then use that hypothesis to prove the patch is necessary.
|
|
67
|
+
|
|
68
|
+
When accepted behavior, source truth, or a user-owned choice is unsettled, use [Decision Gate](../../skills/decision-gate/SKILL.md) rather than diagnosing an invented expectation.
|
|
69
|
+
|
|
70
|
+
## Choose One Distinguishing Loop
|
|
71
|
+
|
|
72
|
+
Choose the smallest observation whose possible outcomes differ between the meaningful hypotheses at the required boundary.
|
|
73
|
+
|
|
74
|
+
When the exact loop is not obvious, or several loop shapes remain plausible, read the [Diagnostic Loop Catalog](references/diagnostic-loop-catalog.md).
|
|
75
|
+
|
|
76
|
+
When timing, randomness, environmental variance, concurrency, or resource behavior shapes the failure, also read [Flaky and Performance Diagnosis](references/flaky-and-performance-diagnosis.md) before selecting or running the loop.
|
|
77
|
+
|
|
78
|
+
When several tools or observers could execute the same selected loop, or the likely interaction is broad, use [Action Selection](../../skills/action-selection/SKILL.md). It selects the interaction; Diagnose Failure retains the hypothesis and causal question.
|
|
79
|
+
|
|
80
|
+
Before an active observation, confirm that the authority envelope covers:
|
|
81
|
+
|
|
82
|
+
- the exact path and environment;
|
|
83
|
+
- mutation, repetition, instrumentation, and cost;
|
|
84
|
+
- evidence collection and sensitive-data handling;
|
|
85
|
+
- cleanup, rollback, or recovery.
|
|
86
|
+
|
|
87
|
+
Otherwise return the loop's purpose, expected distinguishing evidence, and stop condition to Workflow and wait.
|
|
88
|
+
|
|
89
|
+
When no safe useful loop exists, stop with the smallest missing evidence: failing input, command, trace, log window, seed, timestamp, environment, permission, or isolated instrumentation.
|
|
90
|
+
|
|
91
|
+
## Run And Interpret One Iteration
|
|
39
92
|
|
|
40
|
-
|
|
93
|
+
Run one covered observation and change one distinguishing variable at a time.
|
|
41
94
|
|
|
42
|
-
|
|
95
|
+
Apply [Verify Work](../../skills/verify-work/SKILL.md) when the observation requires its full claim-and-boundary method. Diagnosis retains ownership of causal inference after verification returns the evidence.
|
|
43
96
|
|
|
44
|
-
|
|
97
|
+
Keep three results separate.
|
|
45
98
|
|
|
46
|
-
|
|
99
|
+
### Check Result
|
|
47
100
|
|
|
48
|
-
|
|
101
|
+
How did the selected observation's defined assertion execute?
|
|
49
102
|
|
|
50
|
-
|
|
103
|
+
Use Verify Work's check-result semantics:
|
|
51
104
|
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
5. trace the causal chain from trigger and state to the observed result;
|
|
57
|
-
6. preserve evidence that contradicts the leading hypothesis.
|
|
105
|
+
- **Passed:** the observation executed validly and its assertion held.
|
|
106
|
+
- **Failed:** the observation executed validly and its assertion did not hold.
|
|
107
|
+
- **Error:** the observer or execution was invalid.
|
|
108
|
+
- **Unavailable:** the observation could not be attempted safely or reliably.
|
|
58
109
|
|
|
59
|
-
|
|
110
|
+
A target path may intentionally fail while the diagnostic check Passes. An Error or Unavailable observer says nothing about the production cause.
|
|
60
111
|
|
|
61
|
-
|
|
112
|
+
### Hypothesis Result
|
|
62
113
|
|
|
63
|
-
|
|
114
|
+
What does the evidence establish about the tested hypothesis?
|
|
64
115
|
|
|
65
|
-
|
|
116
|
+
- **Supported:** direct evidence establishes the hypothesis at its stated boundary.
|
|
117
|
+
- **Contradicted:** evidence shows the hypothesis is false.
|
|
118
|
+
- **Inconclusive:** the observation cannot distinguish it from meaningful alternatives.
|
|
119
|
+
- **Unavailable:** the required evidence cannot currently be obtained.
|
|
66
120
|
|
|
67
|
-
|
|
121
|
+
### Cause Status
|
|
122
|
+
|
|
123
|
+
- **Supported:** the evidence establishes a useful causal explanation.
|
|
124
|
+
- **Unresolved:** one or more causal links or meaningful alternatives remain open.
|
|
125
|
+
|
|
126
|
+
A Supported hypothesis does not automatically establish a Supported cause.
|
|
127
|
+
|
|
128
|
+
A cause is Supported only when the evidence:
|
|
129
|
+
|
|
130
|
+
1. establishes the relevant failure or strongest available observer;
|
|
131
|
+
2. behaves as the causal explanation predicts;
|
|
132
|
+
3. materially weakens meaningful alternatives;
|
|
133
|
+
4. connects trigger and state through mechanism to the observed symptom;
|
|
134
|
+
5. is sufficient to bound a correction or other next action without guessing.
|
|
135
|
+
|
|
136
|
+
If evidence supports only the strongest remaining hypothesis, report it as such and identify the observation that would confirm or refute the missing causal link.
|
|
137
|
+
|
|
138
|
+
## Route From The Iteration
|
|
139
|
+
|
|
140
|
+
- **Check Error:** repair or replace the observer; do not change production code.
|
|
141
|
+
- **Check Unavailable:** stop with the missing evidence or authority.
|
|
142
|
+
- **Hypothesis Contradicted:** preserve the evidence, then refine or replace the hypothesis.
|
|
143
|
+
- **Hypothesis Inconclusive:** choose a sharper distinguishing loop.
|
|
144
|
+
- **Hypothesis Supported, cause Unresolved:** test the remaining causal link or alternative.
|
|
145
|
+
- **Cause Supported:** return the causal explanation and correction boundary to Workflow.
|
|
146
|
+
- **Behavior or source conflict:** use Decision Gate.
|
|
147
|
+
- **Supported structural pressure:** apply [Design for Depth](../../skills/design-for-depth/SKILL.md) under the current owner.
|
|
148
|
+
- **Active harm before cause:** return one bounded containment option to Workflow.
|
|
149
|
+
|
|
150
|
+
Do not continue because diagnosis started. Stop when no safe observation can materially improve the route.
|
|
68
151
|
|
|
69
152
|
## Protect Diagnostic Evidence
|
|
70
153
|
|
|
71
|
-
Treat logs, traces, payloads, screenshots, dumps,
|
|
154
|
+
Treat logs, traces, payloads, screenshots, dumps, production samples, and browser state as potentially sensitive. Minimize and sanitize them. Do not expose credentials, tokens, unrestricted personal data, or private payloads.
|
|
155
|
+
|
|
156
|
+
Bound repetition by time, cost, side effects, and environment safety. Do not repeatedly exercise mutating production behavior without explicit authority and an understood recovery boundary.
|
|
157
|
+
|
|
158
|
+
Temporary instrumentation must distinguish hypotheses. Keep it isolated and state:
|
|
72
159
|
|
|
73
|
-
|
|
160
|
+
- what it observes;
|
|
161
|
+
- mutation or performance impact;
|
|
162
|
+
- cleanup and removal condition;
|
|
163
|
+
- whether it remains after diagnosis.
|
|
74
164
|
|
|
75
|
-
|
|
165
|
+
Preserve contradictory and unfavorable evidence. Do not rerun until it disappears.
|
|
76
166
|
|
|
77
167
|
## Identify The Owning Cause
|
|
78
168
|
|
|
@@ -86,26 +176,82 @@ Classify only what evidence supports:
|
|
|
86
176
|
- wrong result, scope, order, or execution strategy;
|
|
87
177
|
- structural ownership, interface, state, or failure-unit pressure.
|
|
88
178
|
|
|
89
|
-
|
|
179
|
+
Diagnosis determines whether observed pressure is causal.
|
|
90
180
|
|
|
91
|
-
Use
|
|
181
|
+
Use Design for Depth only after diagnosis establishes structural pressure, or when direct design evidence independently establishes caller coordination, distributed policy, unowned state, interface leakage, or failure-unit pressure. Design for Depth shapes the supported ownership boundary; it does not re-prove the cause.
|
|
92
182
|
|
|
93
|
-
If
|
|
183
|
+
If design work exposes an unsupported causal assumption that changes the boundary choice, re-enter diagnosis with that exact assumption and the alternatives it must distinguish.
|
|
94
184
|
|
|
95
185
|
## Contain Harm Without Claiming A Fix
|
|
96
186
|
|
|
97
|
-
When harm must be limited before
|
|
187
|
+
When immediate harm must be limited before cause is Supported, return one bounded containment option to Workflow.
|
|
98
188
|
|
|
99
|
-
Containment must
|
|
189
|
+
Containment must:
|
|
190
|
+
|
|
191
|
+
- reduce immediate harm;
|
|
192
|
+
- remain narrower and more reversible than a guessed correction;
|
|
193
|
+
- preserve diagnostic evidence;
|
|
194
|
+
- define verification and recovery;
|
|
195
|
+
- avoid settling unresolved behavior;
|
|
196
|
+
- be reported as mitigation, not resolution.
|
|
100
197
|
|
|
101
198
|
A requested patch may instead be an authorized learning action. Its result can strengthen or weaken a hypothesis; it does not become selected production behavior automatically.
|
|
102
199
|
|
|
103
|
-
## Return The
|
|
200
|
+
## Return The Correction Boundary
|
|
201
|
+
|
|
202
|
+
When cause is Supported, return:
|
|
203
|
+
|
|
204
|
+
- failure claim and required boundary;
|
|
205
|
+
- diagnostic loop and observation;
|
|
206
|
+
- check result and hypothesis result;
|
|
207
|
+
- causal chain;
|
|
208
|
+
- alternatives weakened and uncertainty remaining;
|
|
209
|
+
- owning cause;
|
|
210
|
+
- smallest coherent correction boundary;
|
|
211
|
+
- minimized regression signal;
|
|
212
|
+
- original reported path or strongest available observer.
|
|
104
213
|
|
|
105
|
-
|
|
214
|
+
The evidence supports the route but does not authorize correction.
|
|
106
215
|
|
|
107
|
-
|
|
216
|
+
Return an authorized correction to [Execute Work](../../skills/execute-work/SKILL.md). Use [TDD](../../skills/tdd/SKILL.md) when the diagnostic loop supplies a stable failing behavior check. Use [Simplify Code](../../skills/simplify-code/SKILL.md) only when the supported cause shows that obsolete or workaround machinery can be removed without changing accepted behavior. Do not use cleanup in place of correcting the supported failure or while its evidence remains unresolved.
|
|
108
217
|
|
|
109
|
-
|
|
218
|
+
After correction, use Verify Work to check both:
|
|
219
|
+
|
|
220
|
+
1. the minimized regression signal;
|
|
221
|
+
2. the original reported path or strongest available observer.
|
|
222
|
+
|
|
223
|
+
If correction fails, or exposes related shared-state consequences, re-enter diagnosis before another patch.
|
|
224
|
+
|
|
225
|
+
Do not claim fixed when only containment succeeded, a nearby check passed, or the original boundary remains unavailable.
|
|
226
|
+
|
|
227
|
+
## Report Proportionately
|
|
228
|
+
|
|
229
|
+
For one unresolved iteration:
|
|
230
|
+
|
|
231
|
+
```text
|
|
232
|
+
Failure @ boundary:
|
|
233
|
+
Diagnostic loop:
|
|
234
|
+
Check result:
|
|
235
|
+
Hypothesis result:
|
|
236
|
+
Cause status: Unresolved
|
|
237
|
+
Next distinguishing evidence:
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
For a supported cause, containment, conflicting evidence, or a route handoff:
|
|
241
|
+
|
|
242
|
+
```text
|
|
243
|
+
Failure claim:
|
|
244
|
+
Required boundary:
|
|
245
|
+
Expected / observed:
|
|
246
|
+
Diagnostic loop and evidence:
|
|
247
|
+
Check result:
|
|
248
|
+
Hypothesis result:
|
|
249
|
+
Cause status:
|
|
250
|
+
Causal explanation or strongest remaining hypothesis:
|
|
251
|
+
Alternatives weakened / unresolved:
|
|
252
|
+
Containment or instrumentation:
|
|
253
|
+
Correction boundary or next evidence:
|
|
254
|
+
Post-correction verification:
|
|
255
|
+
```
|
|
110
256
|
|
|
111
|
-
|
|
257
|
+
Omit fields that do not apply. Stop when the supported route, missing evidence, containment boundary, or unresolved decision is explicit.
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
# Diagnostic Loop Catalog
|
|
2
2
|
|
|
3
|
-
Read this
|
|
3
|
+
Read this when the exact distinguishing loop is not obvious or several loop shapes remain plausible.
|
|
4
|
+
|
|
5
|
+
Use it to select the smallest observation whose possible outcomes differ between meaningful hypotheses at the required boundary.
|
|
4
6
|
|
|
5
7
|
## Common Loops
|
|
6
8
|
|
|
@@ -13,6 +15,16 @@ Read this before choosing a diagnostic loop for a reported failure or current hy
|
|
|
13
15
|
- **Differential loop:** the same input through old and new code, configuration, dataset, dependency, or environment.
|
|
14
16
|
- **Instrumented run:** targeted probes at boundaries that distinguish competing hypotheses.
|
|
15
17
|
|
|
18
|
+
## Predict Before Running
|
|
19
|
+
|
|
20
|
+
Record only outcomes capable of distinguishing the viable hypotheses:
|
|
21
|
+
|
|
22
|
+
```text
|
|
23
|
+
Observation outcome | Effect on leading hypothesis | Effect on alternative
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
If every likely outcome leaves the same hypotheses plausible, the loop is not distinguishing enough. Sharpen the observer or choose another loop before generating evidence.
|
|
27
|
+
|
|
16
28
|
## Improve The Loop
|
|
17
29
|
|
|
18
30
|
Prefer a loop that is:
|
|
@@ -28,7 +40,9 @@ An observer must sit on the real path and cover the complete operation. A manual
|
|
|
28
40
|
|
|
29
41
|
## Boundary Examples
|
|
30
42
|
|
|
31
|
-
- A UI save remains pending
|
|
32
|
-
- A registered callback allegedly never runs
|
|
33
|
-
- A failure appears only in CI
|
|
34
|
-
- Two causes fit one symptom
|
|
43
|
+
- A UI save remains pending: exercise the browser path and inspect the request, console, and rendered state; a reducer unit test does not prove the reported boundary.
|
|
44
|
+
- A registered callback allegedly never runs: invoke or observe the registered entrypoint; calling its helper directly proves only helper behavior.
|
|
45
|
+
- A failure appears only in CI: compare the same input across local and CI configuration before changing timing or adding retries.
|
|
46
|
+
- Two causes fit one symptom: choose an observation whose outcomes differ between them; another generic failure is not distinguishing evidence.
|
|
47
|
+
|
|
48
|
+
Stop once one loop is selected with predicted distinguishing outcomes. Return it to the main skill; do not execute it from this reference.
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
# Flaky And Performance Diagnosis
|
|
2
2
|
|
|
3
|
-
Read this before
|
|
3
|
+
Read this before selecting or running a diagnostic observation when timing, randomness, environmental variance, concurrency, or resource behavior shapes the failure.
|
|
4
|
+
|
|
5
|
+
Diagnostic measurement explains why variance or resource behavior occurs. After correction, Verify Work establishes whether the representative behavior or performance claim is supported.
|
|
4
6
|
|
|
5
7
|
## Flaky Failures
|
|
6
8
|
|
|
@@ -14,7 +16,11 @@ Use, when safe and representative:
|
|
|
14
16
|
- console, network, event-order, scheduler, screenshot, or video traces;
|
|
15
17
|
- old-versus-new, local-versus-CI, or headed-versus-headless differential runs.
|
|
16
18
|
|
|
17
|
-
Choose the repetition count from runtime, cost, side effects, and the failure rate.
|
|
19
|
+
Choose the repetition count from runtime, cost, side effects, and the estimated failure rate. State the planned stop condition before running. Preserve every unfavorable result.
|
|
20
|
+
|
|
21
|
+
One green rerun does not contradict an intermittent-failure hypothesis. One red rerun may establish the symptom but not its cause. Use the aggregate pattern or captured distinguishing event to classify the hypothesis.
|
|
22
|
+
|
|
23
|
+
Do not loop mutating production behavior merely to improve reproduction.
|
|
18
24
|
|
|
19
25
|
Do not stabilize a flake with arbitrary sleeps, retries, wider timeouts, swallowed errors, or disabled checks unless evidence shows that mechanism addresses the causal timing or availability contract. Suppressing observation is not correction.
|
|
20
26
|
|
|
@@ -36,8 +42,10 @@ Sanitize production traces and payloads. When safe production evidence is unavai
|
|
|
36
42
|
|
|
37
43
|
## Boundary Examples
|
|
38
44
|
|
|
39
|
-
- Twenty green reruns after widening a timeout
|
|
40
|
-
- A parser microbenchmark improves while the representative trace is dominated by database waiting
|
|
41
|
-
- Median latency improves while tail latency or memory crosses the accepted boundary
|
|
45
|
+
- Twenty green reruns after widening a timeout: the failure was suppressed or became rarer; its cause remains unresolved.
|
|
46
|
+
- A parser microbenchmark improves while the representative trace is dominated by database waiting: the benchmark does not explain the reported slowdown.
|
|
47
|
+
- Median latency improves while tail latency or memory crosses the accepted boundary: the performance claim remains contradicted or incomplete.
|
|
42
48
|
|
|
43
49
|
After an authorized correction, verify that the representative baseline improves and required behavior remains correct. Report variance and relevant resource tradeoffs rather than one favorable run.
|
|
50
|
+
|
|
51
|
+
Stop when the loop has produced a distinguishing pattern or event, the planned repetition or cost boundary is reached, or the required representative evidence is unavailable. Return the evidence and limit to the main skill.
|