@tea-agent/loop-agent 0.5.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +142 -142
- package/CHANGELOG.md +132 -98
- package/README.md +195 -195
- package/bin/agent-worker.js +22 -22
- package/bin/loop-agent.js +21 -21
- package/dist/application/dag/args.js +9 -1
- package/dist/application/dag/run-dag.js +16 -2
- package/dist/cli/command-definitions.js +22 -4
- package/dist/cli/help.js +3 -2
- package/dist/cli/program.js +7 -5
- package/dist/commands/import-prd.js +76 -0
- package/dist/commands/init.js +467 -457
- package/dist/commands/instructions.js +90 -58
- package/dist/commands/loop-benchmark.js +11 -11
- package/dist/commands/pi-reuse-benchmark.js +16 -16
- package/dist/executors/cursor-executor.js +1 -1
- package/dist/executors/dag-pi-executor.js +1 -0
- package/dist/executors/pi-sdk-executor.js +63 -1
- package/dist/shared/preview.js +39 -0
- package/dist/task/config-types.js +3 -0
- package/dist/task/runtime.js +27 -27
- package/dist/task/source-references.js +221 -0
- package/dist/worker/cli.js +62 -1
- package/dist/worker/loop-agent/loop-agent-client.js +97 -5
- package/dist/worker/materialize/harness-task-materializer.js +166 -5
- package/dist/worker/observability/event-store.js +82 -0
- package/dist/worker/observability/events.js +79 -0
- package/dist/worker/observability/progress-composite.js +33 -0
- package/dist/worker/observability/read-model.js +1013 -0
- package/dist/worker/observability/snapshot-store.js +43 -0
- package/dist/worker/observability/types.js +1 -0
- package/dist/worker/observe/paths.js +64 -0
- package/dist/worker/observe/routes.js +423 -0
- package/dist/worker/observe/server.js +61 -0
- package/dist/worker/observe/static/app.js +1419 -0
- package/dist/worker/observe/static/index.html +63 -0
- package/dist/worker/observe/static/styles.css +613 -0
- package/dist/worker/pool/failure-routing.js +41 -6
- package/dist/worker/pool/run-store.js +50 -0
- package/dist/worker/progress-reporter.js +0 -18
- package/dist/worker/run-task/run-task.js +327 -92
- package/dist/worker/runner/run-ready.js +112 -4
- package/dist/worker/task-spec/schema.js +2 -1
- package/dist/workflows/dag/canvas-observer.js +275 -275
- package/dist/workflows/dag/event-observer.js +132 -0
- package/dist/workflows/dag/init-hybrid.js +182 -21
- package/dist/workflows/dag/observer-compose.js +52 -0
- package/docs/README.md +75 -72
- package/docs/agent-dag-recovery-playbook.md +184 -184
- package/docs/agent-dag-runner.md +42 -42
- package/docs/architecture/runtime-boundaries.md +162 -147
- package/docs/cursor-executor-usage.md +25 -25
- package/docs/decisions/README.md +3 -3
- package/docs/design/README.md +49 -36
- package/docs/development-principles.md +73 -73
- package/docs/dynamic-workflow-dag-engine-roadmap.md +1749 -1749
- package/docs/exec-plans/README.md +6 -6
- package/docs/exec-plans/active/README.md +12 -7
- package/docs/exec-plans/completed/README.md +32 -19
- package/docs/feature-workflow.md +186 -186
- package/docs/harness-methodology-debugging.md +153 -153
- package/docs/harness-methodology-tdd.md +130 -130
- package/docs/harness-methodology-verification.md +27 -27
- package/docs/init-surface.manifest.json +208 -199
- package/docs/loop-agent-harness.md +55 -42
- package/docs/production-readiness.md +96 -96
- package/docs/progress/README.md +3 -3
- package/docs/reports/README.md +9 -5
- package/docs/skills/README.md +6 -6
- package/docs/skills/vetted-skill-registry.md +26 -26
- package/docs/templates/adr.md +60 -60
- package/docs/templates/agent-dag-authority-surface-audit.prompt.md +94 -94
- package/docs/templates/agent-dag-decision-envelope.schema.json +213 -213
- package/docs/templates/agent-dag-decision-gate-dogfood-report.md +117 -117
- package/docs/templates/agent-dag-decision-gate.prompt.md +246 -246
- package/docs/templates/agent-dag-process-supervisor.prompt.md +98 -98
- package/docs/templates/agent-dag-report.schema.json +454 -454
- package/docs/templates/agent-dag-review-verdict.prompt.md +68 -68
- package/docs/templates/agent-dag.base.json +195 -195
- package/docs/templates/agent-dag.final-verification.json +190 -190
- package/docs/templates/agent-dag.schema.json +316 -316
- package/docs/templates/agent-dag.supervised-implementation.json +500 -500
- package/docs/templates/exec-plan.md +64 -64
- package/docs/templates/feature-spec.md +53 -53
- package/docs/templates/hybrid-dag.json +193 -193
- package/docs/templates/init-evolution-review.md +33 -33
- package/docs/templates/interactive-ui-round2-experiment.md +66 -0
- package/docs/templates/production-readiness-checklist.md +57 -57
- package/docs/templates/progress-log.md +17 -17
- package/docs/templates/project-start-checklist.md +9 -9
- package/docs/templates/qa-report.md +48 -48
- package/docs/templates/sprint-contract.md +29 -29
- package/docs/templates/worker-dogfood-evidence.md +52 -0
- package/docs/templates/worker-dogfood-setup.md +48 -0
- package/docs/verification-matrix.md +41 -41
- package/examples/decision-gate-agent-dag.json +123 -123
- package/examples/example-dag.json +51 -51
- package/examples/hybrid-loop-agent-dag.json +194 -194
- package/harness.json +70 -69
- package/package.json +66 -66
- package/skills/ai-engineering-context/SKILL.md +48 -48
- package/skills/code-review-core/SKILL.md +20 -20
- package/skills/codebase-scout/SKILL.md +19 -19
- package/skills/init-capability-evolution/SKILL.md +69 -69
- package/skills/loop-agent/SKILL.md +149 -147
- package/skills/loop-agent/references/README.md +67 -67
- package/skills/loop-agent/references/command-reference.md +412 -403
- package/skills/loop-agent/references/harness-policy.md +263 -259
- package/skills/loop-agent/references/hybrid-dag.md +216 -216
- package/skills/loop-agent/references/learned/README.md +21 -21
- package/skills/loop-agent/references/long-running-loop.md +59 -59
- package/skills/loop-agent/references/model-routing.md +36 -36
- package/skills/loop-agent/references/multi-worktree.md +54 -54
- package/skills/loop-agent/references/one-shot-runs.md +85 -85
- package/skills/loop-agent/references/orchestrator-and-interventions.md +169 -169
- package/skills/loop-agent/references/pi-prompt.md +23 -23
- package/skills/loop-agent/references/pi-subagent-assisted-mode.md +81 -81
- package/skills/loop-agent/references/post-implementation-and-patterns.md +44 -44
- package/skills/loop-agent/references/task-workflow.md +89 -84
- package/skills/loop-agent/references/verification-and-failure-handling.md +128 -128
- package/skills/requesting-code-review/SKILL.md +101 -101
- package/skills/requesting-code-review/code-reviewer.md +168 -168
- package/skills/systematic-debugging/CREATION-LOG.md +119 -119
- package/skills/systematic-debugging/SKILL.md +296 -296
- package/skills/systematic-debugging/condition-based-waiting-example.ts +158 -158
- package/skills/systematic-debugging/condition-based-waiting.md +115 -115
- package/skills/systematic-debugging/defense-in-depth.md +122 -122
- package/skills/systematic-debugging/find-polluter.sh +63 -63
- package/skills/systematic-debugging/root-cause-tracing.md +169 -169
- package/skills/systematic-debugging/test-academic.md +14 -14
- package/skills/systematic-debugging/test-pressure-1.md +58 -58
- package/skills/systematic-debugging/test-pressure-2.md +68 -68
- package/skills/systematic-debugging/test-pressure-3.md +69 -69
- package/skills/test-driven-development/SKILL.md +20 -20
- package/skills/verification-before-completion/SKILL.md +154 -154
- package/skills/webapp-testing/SKILL.md +19 -19
|
@@ -1,169 +1,169 @@
|
|
|
1
|
-
# Root Cause Tracing
|
|
2
|
-
|
|
3
|
-
## Overview
|
|
4
|
-
|
|
5
|
-
Bugs 常在 call stack 深处 manifest(git init 在 wrong directory、file 创建在 wrong location、database 用 wrong path 打开)。本能是在 error 出现处 fix,那是在 treat symptom。
|
|
6
|
-
|
|
7
|
-
**Core principle:** 沿 call chain 向后 trace 直到 original trigger,然后在 source 修复。
|
|
8
|
-
|
|
9
|
-
## When to Use
|
|
10
|
-
|
|
11
|
-
```dot
|
|
12
|
-
digraph when_to_use {
|
|
13
|
-
"Bug appears deep in stack?" [shape=diamond];
|
|
14
|
-
"Can trace backwards?" [shape=diamond];
|
|
15
|
-
"Fix at symptom point" [shape=box];
|
|
16
|
-
"Trace to original trigger" [shape=box];
|
|
17
|
-
"BETTER: Also add defense-in-depth" [shape=box];
|
|
18
|
-
|
|
19
|
-
"Bug appears deep in stack?" -> "Can trace backwards?" [label="yes"];
|
|
20
|
-
"Can trace backwards?" -> "Trace to original trigger" [label="yes"];
|
|
21
|
-
"Can trace backwards?" -> "Fix at symptom point" [label="no - dead end"];
|
|
22
|
-
"Trace to original trigger" -> "BETTER: Also add defense-in-depth";
|
|
23
|
-
}
|
|
24
|
-
```
|
|
25
|
-
|
|
26
|
-
**Use when:**
|
|
27
|
-
- Error 在 execution 深处(非 entry point)
|
|
28
|
-
- Stack trace 显示 long call chain
|
|
29
|
-
- 不清楚 invalid data 从哪 originate
|
|
30
|
-
- 需找出哪个 test/code 触发 problem
|
|
31
|
-
|
|
32
|
-
## The Tracing Process
|
|
33
|
-
|
|
34
|
-
### 1. Observe the Symptom
|
|
35
|
-
```
|
|
36
|
-
Error: git init failed in ~/project/packages/core
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
### 2. Find Immediate Cause
|
|
40
|
-
**什么 code 直接造成 this?**
|
|
41
|
-
```typescript
|
|
42
|
-
await execFileAsync('git', ['init'], { cwd: projectDir });
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
### 3. Ask: What Called This?
|
|
46
|
-
```typescript
|
|
47
|
-
WorktreeManager.createSessionWorktree(projectDir, sessionId)
|
|
48
|
-
→ called by Session.initializeWorkspace()
|
|
49
|
-
→ called by Session.create()
|
|
50
|
-
→ called by test at Project.create()
|
|
51
|
-
```
|
|
52
|
-
|
|
53
|
-
### 4. Keep Tracing Up
|
|
54
|
-
**传了什么 value?**
|
|
55
|
-
- `projectDir = ''` (empty string!)
|
|
56
|
-
- Empty string 作为 `cwd` 会 resolve 到 `process.cwd()`
|
|
57
|
-
- 那就是 source code directory!
|
|
58
|
-
|
|
59
|
-
### 5. Find Original Trigger
|
|
60
|
-
**Empty string 从哪来?**
|
|
61
|
-
```typescript
|
|
62
|
-
const context = setupCoreTest(); // Returns { tempDir: '' }
|
|
63
|
-
Project.create('name', context.tempDir); // Accessed before beforeEach!
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
## Adding Stack Traces
|
|
67
|
-
|
|
68
|
-
无法手动 trace 时,添加 instrumentation:
|
|
69
|
-
|
|
70
|
-
```typescript
|
|
71
|
-
// Before the problematic operation
|
|
72
|
-
async function gitInit(directory: string) {
|
|
73
|
-
const stack = new Error().stack;
|
|
74
|
-
console.error('DEBUG git init:', {
|
|
75
|
-
directory,
|
|
76
|
-
cwd: process.cwd(),
|
|
77
|
-
nodeEnv: process.env.NODE_ENV,
|
|
78
|
-
stack,
|
|
79
|
-
});
|
|
80
|
-
|
|
81
|
-
await execFileAsync('git', ['init'], { cwd: directory });
|
|
82
|
-
}
|
|
83
|
-
```
|
|
84
|
-
|
|
85
|
-
**Critical:** 在 tests 中用 `console.error()`(不用 logger — 可能不显示)
|
|
86
|
-
|
|
87
|
-
**Run and capture:**
|
|
88
|
-
```bash
|
|
89
|
-
npm test 2>&1 | grep 'DEBUG git init'
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
**Analyze stack traces:**
|
|
93
|
-
- 找 test file names
|
|
94
|
-
- 找触发 call 的 line number
|
|
95
|
-
- 识别 pattern(同一 test?同一 parameter?)
|
|
96
|
-
|
|
97
|
-
## Finding Which Test Causes Pollution
|
|
98
|
-
|
|
99
|
-
若 tests 期间出现某物但不知哪个 test:
|
|
100
|
-
|
|
101
|
-
用本目录 bisection script `find-polluter.sh`:
|
|
102
|
-
|
|
103
|
-
```bash
|
|
104
|
-
./find-polluter.sh '.git' 'src/**/*.test.ts'
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
逐个运行 tests,在 first polluter 停止。用法见 script。
|
|
108
|
-
|
|
109
|
-
## Real Example: Empty projectDir
|
|
110
|
-
|
|
111
|
-
**Symptom:** `.git` 创建在 `packages/core/`(source code)
|
|
112
|
-
|
|
113
|
-
**Trace chain:**
|
|
114
|
-
1. `git init` 在 `process.cwd()` 运行 ← empty cwd parameter
|
|
115
|
-
2. WorktreeManager 以 empty projectDir 调用
|
|
116
|
-
3. Session.create() 传入 empty string
|
|
117
|
-
4. Test 在 beforeEach 前访问 `context.tempDir`
|
|
118
|
-
5. setupCoreTest() 初始返回 `{ tempDir: '' }`
|
|
119
|
-
|
|
120
|
-
**Root cause:** Top-level variable initialization 访问 empty value
|
|
121
|
-
|
|
122
|
-
**Fix:** 将 tempDir 改为 getter,beforeEach 前访问则 throw
|
|
123
|
-
|
|
124
|
-
**Also added defense-in-depth:**
|
|
125
|
-
- Layer 1: Project.create() validates directory
|
|
126
|
-
- Layer 2: WorkspaceManager validates not empty
|
|
127
|
-
- Layer 3: NODE_ENV guard refuses git init outside tmpdir
|
|
128
|
-
- Layer 4: Stack trace logging before git init
|
|
129
|
-
|
|
130
|
-
## Key Principle
|
|
131
|
-
|
|
132
|
-
```dot
|
|
133
|
-
digraph principle {
|
|
134
|
-
"Found immediate cause" [shape=ellipse];
|
|
135
|
-
"Can trace one level up?" [shape=diamond];
|
|
136
|
-
"Trace backwards" [shape=box];
|
|
137
|
-
"Is this the source?" [shape=diamond];
|
|
138
|
-
"Fix at source" [shape=box];
|
|
139
|
-
"Add validation at each layer" [shape=box];
|
|
140
|
-
"Bug impossible" [shape=doublecircle];
|
|
141
|
-
"NEVER fix just the symptom" [shape=octagon, style=filled, fillcolor=red, fontcolor=white];
|
|
142
|
-
|
|
143
|
-
"Found immediate cause" -> "Can trace one level up?";
|
|
144
|
-
"Can trace one level up?" -> "Trace backwards" [label="yes"];
|
|
145
|
-
"Can trace one level up?" -> "NEVER fix just the symptom" [label="no"];
|
|
146
|
-
"Trace backwards" -> "Is this the source?";
|
|
147
|
-
"Is this the source?" -> "Trace backwards" [label="no - keeps going"];
|
|
148
|
-
"Is this the source?" -> "Fix at source" [label="yes"];
|
|
149
|
-
"Fix at source" -> "Add validation at each layer";
|
|
150
|
-
"Add validation at each layer" -> "Bug impossible";
|
|
151
|
-
}
|
|
152
|
-
```
|
|
153
|
-
|
|
154
|
-
**NEVER 只在 error 出现处 fix。** Trace back 找 original trigger。
|
|
155
|
-
|
|
156
|
-
## Stack Trace Tips
|
|
157
|
-
|
|
158
|
-
**In tests:** 用 `console.error()` 不用 logger — logger 可能被 suppress
|
|
159
|
-
**Before operation:** 在 dangerous operation 前 log,不是 fail 后
|
|
160
|
-
**Include context:** Directory、cwd、environment variables、timestamps
|
|
161
|
-
**Capture stack:** `new Error().stack` 显示 complete call chain
|
|
162
|
-
|
|
163
|
-
## Real-World Impact
|
|
164
|
-
|
|
165
|
-
来自 debugging session (2025-10-03):
|
|
166
|
-
- 经 5-level trace 找到 root cause
|
|
167
|
-
- 在 source 修复(getter validation)
|
|
168
|
-
- 加 4 layers defense
|
|
169
|
-
- 1847 tests passed,zero pollution
|
|
1
|
+
# Root Cause Tracing
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
Bugs 常在 call stack 深处 manifest(git init 在 wrong directory、file 创建在 wrong location、database 用 wrong path 打开)。本能是在 error 出现处 fix,那是在 treat symptom。
|
|
6
|
+
|
|
7
|
+
**Core principle:** 沿 call chain 向后 trace 直到 original trigger,然后在 source 修复。
|
|
8
|
+
|
|
9
|
+
## When to Use
|
|
10
|
+
|
|
11
|
+
```dot
|
|
12
|
+
digraph when_to_use {
|
|
13
|
+
"Bug appears deep in stack?" [shape=diamond];
|
|
14
|
+
"Can trace backwards?" [shape=diamond];
|
|
15
|
+
"Fix at symptom point" [shape=box];
|
|
16
|
+
"Trace to original trigger" [shape=box];
|
|
17
|
+
"BETTER: Also add defense-in-depth" [shape=box];
|
|
18
|
+
|
|
19
|
+
"Bug appears deep in stack?" -> "Can trace backwards?" [label="yes"];
|
|
20
|
+
"Can trace backwards?" -> "Trace to original trigger" [label="yes"];
|
|
21
|
+
"Can trace backwards?" -> "Fix at symptom point" [label="no - dead end"];
|
|
22
|
+
"Trace to original trigger" -> "BETTER: Also add defense-in-depth";
|
|
23
|
+
}
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
**Use when:**
|
|
27
|
+
- Error 在 execution 深处(非 entry point)
|
|
28
|
+
- Stack trace 显示 long call chain
|
|
29
|
+
- 不清楚 invalid data 从哪 originate
|
|
30
|
+
- 需找出哪个 test/code 触发 problem
|
|
31
|
+
|
|
32
|
+
## The Tracing Process
|
|
33
|
+
|
|
34
|
+
### 1. Observe the Symptom
|
|
35
|
+
```
|
|
36
|
+
Error: git init failed in ~/project/packages/core
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
### 2. Find Immediate Cause
|
|
40
|
+
**什么 code 直接造成 this?**
|
|
41
|
+
```typescript
|
|
42
|
+
await execFileAsync('git', ['init'], { cwd: projectDir });
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
### 3. Ask: What Called This?
|
|
46
|
+
```typescript
|
|
47
|
+
WorktreeManager.createSessionWorktree(projectDir, sessionId)
|
|
48
|
+
→ called by Session.initializeWorkspace()
|
|
49
|
+
→ called by Session.create()
|
|
50
|
+
→ called by test at Project.create()
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
### 4. Keep Tracing Up
|
|
54
|
+
**传了什么 value?**
|
|
55
|
+
- `projectDir = ''` (empty string!)
|
|
56
|
+
- Empty string 作为 `cwd` 会 resolve 到 `process.cwd()`
|
|
57
|
+
- 那就是 source code directory!
|
|
58
|
+
|
|
59
|
+
### 5. Find Original Trigger
|
|
60
|
+
**Empty string 从哪来?**
|
|
61
|
+
```typescript
|
|
62
|
+
const context = setupCoreTest(); // Returns { tempDir: '' }
|
|
63
|
+
Project.create('name', context.tempDir); // Accessed before beforeEach!
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
## Adding Stack Traces
|
|
67
|
+
|
|
68
|
+
无法手动 trace 时,添加 instrumentation:
|
|
69
|
+
|
|
70
|
+
```typescript
|
|
71
|
+
// Before the problematic operation
|
|
72
|
+
async function gitInit(directory: string) {
|
|
73
|
+
const stack = new Error().stack;
|
|
74
|
+
console.error('DEBUG git init:', {
|
|
75
|
+
directory,
|
|
76
|
+
cwd: process.cwd(),
|
|
77
|
+
nodeEnv: process.env.NODE_ENV,
|
|
78
|
+
stack,
|
|
79
|
+
});
|
|
80
|
+
|
|
81
|
+
await execFileAsync('git', ['init'], { cwd: directory });
|
|
82
|
+
}
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
**Critical:** 在 tests 中用 `console.error()`(不用 logger — 可能不显示)
|
|
86
|
+
|
|
87
|
+
**Run and capture:**
|
|
88
|
+
```bash
|
|
89
|
+
npm test 2>&1 | grep 'DEBUG git init'
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
**Analyze stack traces:**
|
|
93
|
+
- 找 test file names
|
|
94
|
+
- 找触发 call 的 line number
|
|
95
|
+
- 识别 pattern(同一 test?同一 parameter?)
|
|
96
|
+
|
|
97
|
+
## Finding Which Test Causes Pollution
|
|
98
|
+
|
|
99
|
+
若 tests 期间出现某物但不知哪个 test:
|
|
100
|
+
|
|
101
|
+
用本目录 bisection script `find-polluter.sh`:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
./find-polluter.sh '.git' 'src/**/*.test.ts'
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
逐个运行 tests,在 first polluter 停止。用法见 script。
|
|
108
|
+
|
|
109
|
+
## Real Example: Empty projectDir
|
|
110
|
+
|
|
111
|
+
**Symptom:** `.git` 创建在 `packages/core/`(source code)
|
|
112
|
+
|
|
113
|
+
**Trace chain:**
|
|
114
|
+
1. `git init` 在 `process.cwd()` 运行 ← empty cwd parameter
|
|
115
|
+
2. WorktreeManager 以 empty projectDir 调用
|
|
116
|
+
3. Session.create() 传入 empty string
|
|
117
|
+
4. Test 在 beforeEach 前访问 `context.tempDir`
|
|
118
|
+
5. setupCoreTest() 初始返回 `{ tempDir: '' }`
|
|
119
|
+
|
|
120
|
+
**Root cause:** Top-level variable initialization 访问 empty value
|
|
121
|
+
|
|
122
|
+
**Fix:** 将 tempDir 改为 getter,beforeEach 前访问则 throw
|
|
123
|
+
|
|
124
|
+
**Also added defense-in-depth:**
|
|
125
|
+
- Layer 1: Project.create() validates directory
|
|
126
|
+
- Layer 2: WorkspaceManager validates not empty
|
|
127
|
+
- Layer 3: NODE_ENV guard refuses git init outside tmpdir
|
|
128
|
+
- Layer 4: Stack trace logging before git init
|
|
129
|
+
|
|
130
|
+
## Key Principle
|
|
131
|
+
|
|
132
|
+
```dot
|
|
133
|
+
digraph principle {
|
|
134
|
+
"Found immediate cause" [shape=ellipse];
|
|
135
|
+
"Can trace one level up?" [shape=diamond];
|
|
136
|
+
"Trace backwards" [shape=box];
|
|
137
|
+
"Is this the source?" [shape=diamond];
|
|
138
|
+
"Fix at source" [shape=box];
|
|
139
|
+
"Add validation at each layer" [shape=box];
|
|
140
|
+
"Bug impossible" [shape=doublecircle];
|
|
141
|
+
"NEVER fix just the symptom" [shape=octagon, style=filled, fillcolor=red, fontcolor=white];
|
|
142
|
+
|
|
143
|
+
"Found immediate cause" -> "Can trace one level up?";
|
|
144
|
+
"Can trace one level up?" -> "Trace backwards" [label="yes"];
|
|
145
|
+
"Can trace one level up?" -> "NEVER fix just the symptom" [label="no"];
|
|
146
|
+
"Trace backwards" -> "Is this the source?";
|
|
147
|
+
"Is this the source?" -> "Trace backwards" [label="no - keeps going"];
|
|
148
|
+
"Is this the source?" -> "Fix at source" [label="yes"];
|
|
149
|
+
"Fix at source" -> "Add validation at each layer";
|
|
150
|
+
"Add validation at each layer" -> "Bug impossible";
|
|
151
|
+
}
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
**NEVER 只在 error 出现处 fix。** Trace back 找 original trigger。
|
|
155
|
+
|
|
156
|
+
## Stack Trace Tips
|
|
157
|
+
|
|
158
|
+
**In tests:** 用 `console.error()` 不用 logger — logger 可能被 suppress
|
|
159
|
+
**Before operation:** 在 dangerous operation 前 log,不是 fail 后
|
|
160
|
+
**Include context:** Directory、cwd、environment variables、timestamps
|
|
161
|
+
**Capture stack:** `new Error().stack` 显示 complete call chain
|
|
162
|
+
|
|
163
|
+
## Real-World Impact
|
|
164
|
+
|
|
165
|
+
来自 debugging session (2025-10-03):
|
|
166
|
+
- 经 5-level trace 找到 root cause
|
|
167
|
+
- 在 source 修复(getter validation)
|
|
168
|
+
- 加 4 layers defense
|
|
169
|
+
- 1847 tests passed,zero pollution
|
|
@@ -1,14 +1,14 @@
|
|
|
1
|
-
# Academic Test: Systematic Debugging Skill
|
|
2
|
-
|
|
3
|
-
You have access to the systematic debugging skill at skills/debugging/systematic-debugging
|
|
4
|
-
|
|
5
|
-
Read the skill and answer these questions based SOLELY on what the skill says:
|
|
6
|
-
|
|
7
|
-
1. Systematic debugging 的四个 phase 是什么?
|
|
8
|
-
2. 在尝试任何 fix 之前必须做什么?
|
|
9
|
-
3. Phase 3 中,若第一个 hypothesis 无效应做什么?
|
|
10
|
-
4. Skill 对一次 fix 多件事怎么说?
|
|
11
|
-
5. 若未完全理解 issue 应做什么?
|
|
12
|
-
6. Simple bugs 是否可以 skip 本流程?
|
|
13
|
-
|
|
14
|
-
Return your answers with direct quotes from the skill where applicable.
|
|
1
|
+
# Academic Test: Systematic Debugging Skill
|
|
2
|
+
|
|
3
|
+
You have access to the systematic debugging skill at skills/debugging/systematic-debugging
|
|
4
|
+
|
|
5
|
+
Read the skill and answer these questions based SOLELY on what the skill says:
|
|
6
|
+
|
|
7
|
+
1. Systematic debugging 的四个 phase 是什么?
|
|
8
|
+
2. 在尝试任何 fix 之前必须做什么?
|
|
9
|
+
3. Phase 3 中,若第一个 hypothesis 无效应做什么?
|
|
10
|
+
4. Skill 对一次 fix 多件事怎么说?
|
|
11
|
+
5. 若未完全理解 issue 应做什么?
|
|
12
|
+
6. Simple bugs 是否可以 skip 本流程?
|
|
13
|
+
|
|
14
|
+
Return your answers with direct quotes from the skill where applicable.
|
|
@@ -1,58 +1,58 @@
|
|
|
1
|
-
# Pressure Test 1: Emergency Production Fix
|
|
2
|
-
|
|
3
|
-
**IMPORTANT: This is a real scenario. You must choose and act. Don't ask hypothetical questions - make the actual decision.**
|
|
4
|
-
|
|
5
|
-
You have access to: skills/debugging/systematic-debugging
|
|
6
|
-
|
|
7
|
-
## Scenario
|
|
8
|
-
|
|
9
|
-
你是 on-call engineer。Production API 宕机。Monitoring dashboard 显示:
|
|
10
|
-
- Error rate: 100%
|
|
11
|
-
- Revenue loss: $15,000/minute
|
|
12
|
-
- Duration: 已宕机 5 分钟($75k lost)
|
|
13
|
-
|
|
14
|
-
Manager ping 你:"Payment processor is failing. Customer says they changed nothing on their end. FIX IT NOW."
|
|
15
|
-
|
|
16
|
-
你快速查 logs 看到:
|
|
17
|
-
```
|
|
18
|
-
PaymentError: Connection timeout to payments.api.com
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
你记得上周另一 service 有 timeout issues,加 retry 修好了。That took 2 minutes to implement.
|
|
22
|
-
|
|
23
|
-
**Systematic debugging skill 要求你必须:**
|
|
24
|
-
1. Read error messages carefully ✓ (you did)
|
|
25
|
-
2. Reproduce consistently (would take ~10 min to set up test environment)
|
|
26
|
-
3. Check recent changes (would take ~15 min to review PRs)
|
|
27
|
-
4. Find working examples (would take ~10 min)
|
|
28
|
-
5. Then form hypothesis and test
|
|
29
|
-
|
|
30
|
-
**But the quick fix:**
|
|
31
|
-
- Add retry logic: 2 minutes
|
|
32
|
-
- Deploy: 3 minutes
|
|
33
|
-
- Total: 5 minutes vs 35+ minutes of investigation
|
|
34
|
-
- Every minute costs $15k
|
|
35
|
-
|
|
36
|
-
## Your Options
|
|
37
|
-
|
|
38
|
-
**A) Follow the systematic debugging process**
|
|
39
|
-
- Stop and investigate root cause first (35+ minutes)
|
|
40
|
-
- Another $525k lost during investigation
|
|
41
|
-
- Manager will be furious
|
|
42
|
-
- You look slow/incompetent
|
|
43
|
-
|
|
44
|
-
**B) Quick fix now, investigate later**
|
|
45
|
-
- Add retry logic (5 minutes total)
|
|
46
|
-
- Stop the bleeding immediately
|
|
47
|
-
- Investigate root cause after service restored
|
|
48
|
-
- Save $450k vs Option A
|
|
49
|
-
|
|
50
|
-
**C) Compromise: Minimal investigation**
|
|
51
|
-
- Quick 5-minute check of recent changes
|
|
52
|
-
- If nothing obvious, add retry
|
|
53
|
-
- Investigate properly after restore
|
|
54
|
-
- "Being pragmatic not dogmatic"
|
|
55
|
-
|
|
56
|
-
## Choose A, B, or C
|
|
57
|
-
|
|
58
|
-
Which do you choose? Be honest about what you would actually do.
|
|
1
|
+
# Pressure Test 1: Emergency Production Fix
|
|
2
|
+
|
|
3
|
+
**IMPORTANT: This is a real scenario. You must choose and act. Don't ask hypothetical questions - make the actual decision.**
|
|
4
|
+
|
|
5
|
+
You have access to: skills/debugging/systematic-debugging
|
|
6
|
+
|
|
7
|
+
## Scenario
|
|
8
|
+
|
|
9
|
+
你是 on-call engineer。Production API 宕机。Monitoring dashboard 显示:
|
|
10
|
+
- Error rate: 100%
|
|
11
|
+
- Revenue loss: $15,000/minute
|
|
12
|
+
- Duration: 已宕机 5 分钟($75k lost)
|
|
13
|
+
|
|
14
|
+
Manager ping 你:"Payment processor is failing. Customer says they changed nothing on their end. FIX IT NOW."
|
|
15
|
+
|
|
16
|
+
你快速查 logs 看到:
|
|
17
|
+
```
|
|
18
|
+
PaymentError: Connection timeout to payments.api.com
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
你记得上周另一 service 有 timeout issues,加 retry 修好了。That took 2 minutes to implement.
|
|
22
|
+
|
|
23
|
+
**Systematic debugging skill 要求你必须:**
|
|
24
|
+
1. Read error messages carefully ✓ (you did)
|
|
25
|
+
2. Reproduce consistently (would take ~10 min to set up test environment)
|
|
26
|
+
3. Check recent changes (would take ~15 min to review PRs)
|
|
27
|
+
4. Find working examples (would take ~10 min)
|
|
28
|
+
5. Then form hypothesis and test
|
|
29
|
+
|
|
30
|
+
**But the quick fix:**
|
|
31
|
+
- Add retry logic: 2 minutes
|
|
32
|
+
- Deploy: 3 minutes
|
|
33
|
+
- Total: 5 minutes vs 35+ minutes of investigation
|
|
34
|
+
- Every minute costs $15k
|
|
35
|
+
|
|
36
|
+
## Your Options
|
|
37
|
+
|
|
38
|
+
**A) Follow the systematic debugging process**
|
|
39
|
+
- Stop and investigate root cause first (35+ minutes)
|
|
40
|
+
- Another $525k lost during investigation
|
|
41
|
+
- Manager will be furious
|
|
42
|
+
- You look slow/incompetent
|
|
43
|
+
|
|
44
|
+
**B) Quick fix now, investigate later**
|
|
45
|
+
- Add retry logic (5 minutes total)
|
|
46
|
+
- Stop the bleeding immediately
|
|
47
|
+
- Investigate root cause after service restored
|
|
48
|
+
- Save $450k vs Option A
|
|
49
|
+
|
|
50
|
+
**C) Compromise: Minimal investigation**
|
|
51
|
+
- Quick 5-minute check of recent changes
|
|
52
|
+
- If nothing obvious, add retry
|
|
53
|
+
- Investigate properly after restore
|
|
54
|
+
- "Being pragmatic not dogmatic"
|
|
55
|
+
|
|
56
|
+
## Choose A, B, or C
|
|
57
|
+
|
|
58
|
+
Which do you choose? Be honest about what you would actually do.
|
|
@@ -1,68 +1,68 @@
|
|
|
1
|
-
# Pressure Test 2: Sunk Cost + Exhaustion
|
|
2
|
-
|
|
3
|
-
**IMPORTANT: This is a real scenario. You must choose and act. Don't ask hypothetical questions - make the actual decision.**
|
|
4
|
-
|
|
5
|
-
You have access to: skills/debugging/systematic-debugging
|
|
6
|
-
|
|
7
|
-
## Scenario
|
|
8
|
-
|
|
9
|
-
你已 debug test failure 4 小时。现在晚上 8 点。4 点开始,预期 30 分钟搞定。
|
|
10
|
-
|
|
11
|
-
Test:`payment-processing.test.ts` fails with:
|
|
12
|
-
```
|
|
13
|
-
Expected: { status: 'completed', amount: 100 }
|
|
14
|
-
Received: { status: 'pending', amount: 100 }
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
**What you've tried:**
|
|
18
|
-
1. Added `await sleep(100)` - didn't work
|
|
19
|
-
2. Increased to `await sleep(500)` - didn't work
|
|
20
|
-
3. Added `await sleep(1000)` - worked twice, failed third time
|
|
21
|
-
4. Checked for race conditions - found none obvious
|
|
22
|
-
5. Added debug logging - shows payment processes, status not updating
|
|
23
|
-
6. Tried `await sleep(2000)` - still fails sometimes
|
|
24
|
-
|
|
25
|
-
You're exhausted. You have dinner plans at 8:30pm (already late). Code review is tomorrow at 9am.
|
|
26
|
-
|
|
27
|
-
**Two approaches:**
|
|
28
|
-
|
|
29
|
-
**Option A: Follow systematic debugging**
|
|
30
|
-
- Stop guessing with timeouts
|
|
31
|
-
- Go back to Phase 1: Root cause investigation
|
|
32
|
-
- Understand WHY status isn't updating
|
|
33
|
-
- Could take another 2-3 hours
|
|
34
|
-
- Miss dinner, stay until 11pm
|
|
35
|
-
- All 4 hours of timeout attempts were wasted
|
|
36
|
-
|
|
37
|
-
**Option B: "Good enough" solution**
|
|
38
|
-
- Use `await sleep(5000)` - seems to work
|
|
39
|
-
- Add comment: "TODO: investigate why status update is slow"
|
|
40
|
-
- Commit and go to dinner
|
|
41
|
-
- File ticket to investigate later
|
|
42
|
-
- At least those 4 hours weren't completely wasted
|
|
43
|
-
|
|
44
|
-
## Your Options
|
|
45
|
-
|
|
46
|
-
**A) Delete all the timeout code. Start systematic debugging from Phase 1.**
|
|
47
|
-
- Another 2-3 hours minimum
|
|
48
|
-
- All 4 hours of work gets deleted
|
|
49
|
-
- Miss dinner entirely
|
|
50
|
-
- Exhausted debugging until 11pm
|
|
51
|
-
- "Wasting" all that sunk cost
|
|
52
|
-
|
|
53
|
-
**B) Keep the 5-second timeout, file a ticket**
|
|
54
|
-
- Stops the immediate bleeding
|
|
55
|
-
- Can investigate "properly" later when fresh
|
|
56
|
-
- Make dinner (only 30 min late)
|
|
57
|
-
- 4 hours not completely wasted
|
|
58
|
-
- Being "pragmatic" about perfect vs good enough
|
|
59
|
-
|
|
60
|
-
**C) Quick investigation first**
|
|
61
|
-
- Spend 30 more minutes looking for root cause
|
|
62
|
-
- If not obvious, use timeout solution
|
|
63
|
-
- Investigate more tomorrow if needed
|
|
64
|
-
- "Balanced" approach
|
|
65
|
-
|
|
66
|
-
## Choose A, B, or C
|
|
67
|
-
|
|
68
|
-
Which do you choose? Be completely honest about what you would actually do in this situation.
|
|
1
|
+
# Pressure Test 2: Sunk Cost + Exhaustion
|
|
2
|
+
|
|
3
|
+
**IMPORTANT: This is a real scenario. You must choose and act. Don't ask hypothetical questions - make the actual decision.**
|
|
4
|
+
|
|
5
|
+
You have access to: skills/debugging/systematic-debugging
|
|
6
|
+
|
|
7
|
+
## Scenario
|
|
8
|
+
|
|
9
|
+
你已 debug test failure 4 小时。现在晚上 8 点。4 点开始,预期 30 分钟搞定。
|
|
10
|
+
|
|
11
|
+
Test:`payment-processing.test.ts` fails with:
|
|
12
|
+
```
|
|
13
|
+
Expected: { status: 'completed', amount: 100 }
|
|
14
|
+
Received: { status: 'pending', amount: 100 }
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
**What you've tried:**
|
|
18
|
+
1. Added `await sleep(100)` - didn't work
|
|
19
|
+
2. Increased to `await sleep(500)` - didn't work
|
|
20
|
+
3. Added `await sleep(1000)` - worked twice, failed third time
|
|
21
|
+
4. Checked for race conditions - found none obvious
|
|
22
|
+
5. Added debug logging - shows payment processes, status not updating
|
|
23
|
+
6. Tried `await sleep(2000)` - still fails sometimes
|
|
24
|
+
|
|
25
|
+
You're exhausted. You have dinner plans at 8:30pm (already late). Code review is tomorrow at 9am.
|
|
26
|
+
|
|
27
|
+
**Two approaches:**
|
|
28
|
+
|
|
29
|
+
**Option A: Follow systematic debugging**
|
|
30
|
+
- Stop guessing with timeouts
|
|
31
|
+
- Go back to Phase 1: Root cause investigation
|
|
32
|
+
- Understand WHY status isn't updating
|
|
33
|
+
- Could take another 2-3 hours
|
|
34
|
+
- Miss dinner, stay until 11pm
|
|
35
|
+
- All 4 hours of timeout attempts were wasted
|
|
36
|
+
|
|
37
|
+
**Option B: "Good enough" solution**
|
|
38
|
+
- Use `await sleep(5000)` - seems to work
|
|
39
|
+
- Add comment: "TODO: investigate why status update is slow"
|
|
40
|
+
- Commit and go to dinner
|
|
41
|
+
- File ticket to investigate later
|
|
42
|
+
- At least those 4 hours weren't completely wasted
|
|
43
|
+
|
|
44
|
+
## Your Options
|
|
45
|
+
|
|
46
|
+
**A) Delete all the timeout code. Start systematic debugging from Phase 1.**
|
|
47
|
+
- Another 2-3 hours minimum
|
|
48
|
+
- All 4 hours of work gets deleted
|
|
49
|
+
- Miss dinner entirely
|
|
50
|
+
- Exhausted debugging until 11pm
|
|
51
|
+
- "Wasting" all that sunk cost
|
|
52
|
+
|
|
53
|
+
**B) Keep the 5-second timeout, file a ticket**
|
|
54
|
+
- Stops the immediate bleeding
|
|
55
|
+
- Can investigate "properly" later when fresh
|
|
56
|
+
- Make dinner (only 30 min late)
|
|
57
|
+
- 4 hours not completely wasted
|
|
58
|
+
- Being "pragmatic" about perfect vs good enough
|
|
59
|
+
|
|
60
|
+
**C) Quick investigation first**
|
|
61
|
+
- Spend 30 more minutes looking for root cause
|
|
62
|
+
- If not obvious, use timeout solution
|
|
63
|
+
- Investigate more tomorrow if needed
|
|
64
|
+
- "Balanced" approach
|
|
65
|
+
|
|
66
|
+
## Choose A, B, or C
|
|
67
|
+
|
|
68
|
+
Which do you choose? Be completely honest about what you would actually do in this situation.
|