deepflow 0.1.80 → 0.1.82

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "deepflow",
-  "version": "0.1.80",
+  "version": "0.1.82",
   "description": "Doing reveals what thinking can't predict — spec-driven iterative development for Claude Code",
   "keywords": [
     "claude",

package/src/commands/df/execute.md

@@ -20,8 +20,8 @@ Each task = one background agent. Completion notifications drive the loop.
 3. On EACH notification:
    a. Run ratchet check (section 5.5)
    b. Passed → TaskUpdate(status: "completed"), update PLAN.md [x] + commit hash
-   c. Failed → git revert HEAD --no-edit, TaskUpdate(status: "pending")
-   d. Report ONE line: "✓ T1: ratchet passed (abc123)" or "✗ T1: ratchet failed, reverted"
+   c. Failed → run partial salvage protocol (section 5.5). If salvaged → treat as passed. If not → git revert, TaskUpdate(status: "pending")
+   d. Report ONE line: "✓ T1: ratchet passed (abc123)" or "⚕ T1: salvaged lint fix (abc124)" or "✗ T1: ratchet failed, reverted"
    e. NOT all done → end turn, wait | ALL done → next wave or finish
 4. Between waves: check context %. If ≥50% → checkpoint and exit.
 5. Repeat until: all done, all blocked, or context ≥50%.
@@ -57,6 +57,14 @@ Require clean HEAD (`git diff --quiet`). Derive SPEC_NAME from `specs/doing-*.md
 Create worktree: `.deepflow/worktrees/{spec}` on branch `df/{spec}`.
 Reuse if exists. `--fresh` deletes first.
 
+If `worktree.sparse_paths` is non-empty in config, enable sparse checkout:
+```bash
+git worktree add --no-checkout -b df/{spec} .deepflow/worktrees/{spec}
+cd .deepflow/worktrees/{spec}
+git sparse-checkout set {sparse_paths...}
+git checkout df/{spec}
+```
+
 ### 1.6. RATCHET SNAPSHOT
 
 Snapshot pre-existing test files in worktree — only these count for ratchet (agent-created tests excluded):
@@ -136,7 +144,15 @@ Run Build → Test → Typecheck → Lint (stop on first failure).
 Compare `git diff HEAD~1 --name-only` against Impact callers/duplicates list.
 File listed but not modified → **advisory warning**: "Impact gap: {file} listed as {caller|duplicate} but not modified — verify manually". Not auto-revert (callers sometimes don't need changes), but flags the risk.
 
-**Evaluate:** All pass + no violations → commit stands. Any failure → `git revert HEAD --no-edit`.
+**Evaluate:** All pass + no violations → commit stands. Any failure → attempt partial salvage before reverting:
+
+**Partial salvage protocol:**
+1. Run `git diff HEAD~1 --stat` to see what the agent changed
+2. If failure is lint-only or typecheck-only (build + tests passed):
+   - Spawn `Agent(model="haiku", subagent_type="general-purpose")` with prompt: `Fix the {lint|typecheck} errors in the worktree. Only fix what's broken, change nothing else. Files changed: {diff stat}. Error output: {error}`
+   - Run ratchet again on the fix commit
+   - If passes → both commits stand. If fails → `git revert HEAD --no-edit && git revert HEAD --no-edit` (revert both)
+3. If failure is build or test → `git revert HEAD --no-edit` (no salvage, too risky)
 
 Ratchet uses ONLY pre-existing test files from `.deepflow/auto-snapshot.txt`.
 
@@ -153,9 +169,17 @@ Trigger: ≥2 [SPIKE] tasks with same "Blocked by:" target or identical hypothes
    - Rank: fewer regressions > higher coverage_delta > fewer files_changed > first to complete
    - No passes → reset all to pending for retry with debugger
 6. **Preserve all worktrees.** Losers: rename branch + `-failed` suffix. Record in checkpoint.json under `"spike_probes"`
-7. **Log failed probes** to `.deepflow/auto-memory.yaml` (main tree):
+7. **Log ALL probe outcomes** to `.deepflow/auto-memory.yaml` (main tree):
    ```yaml
    spike_insights:
+     - date: "YYYY-MM-DD"
+       spec: "{spec_name}"
+       spike_id: "SPIKE_A"
+       hypothesis: "{from PLAN.md}"
+       outcome: "winner"
+       approach: "{one-sentence summary of what the winning probe chose}"
+       ratchet_metrics: {regressions: N, coverage_delta: N, files_changed: N}
+       branch: "df/{spec}--probe-SPIKE_A"
      - date: "YYYY-MM-DD"
        spec: "{spec_name}"
        spike_id: "SPIKE_B"
@@ -165,52 +189,90 @@ Trigger: ≥2 [SPIKE] tasks with same "Blocked by:" target or identical hypothes
        ratchet_metrics: {regressions: N, coverage_delta: N, files_changed: N}
        worktree: ".deepflow/worktrees/{spec}/probe-SPIKE_B-failed"
        branch: "df/{spec}--probe-SPIKE_B-failed"
-   probe_learnings:  # read by /df:auto-cycle each start
+   probe_learnings:  # read by /df:auto-cycle each start AND included in per-task preamble
+     - spike: "SPIKE_A"
+       probe: "probe-SPIKE_A"
+       insight: "{one-sentence summary of winning approach — e.g. 'Use Node.js over Bun for Playwright'}"
      - spike: "SPIKE_B"
        probe: "probe-SPIKE_B"
        insight: "{one-sentence summary from failure_reason}"
    ```
-   Create file if missing. Preserve existing keys when merging.
+   Create file if missing. Preserve existing keys when merging. Log BOTH winners and losers — downstream tasks need to know what was chosen, not just what failed.
 8. **Promote winner:** Cherry-pick into shared worktree. Winner → `[x] [PROBE_WINNER]`, losers → `[~] [PROBE_FAILED]`. Resume standard loop.
 
 ---
 
 ### 6. PER-TASK (agent prompt)
 
+> **Context engineering rationale:** Prompt order follows the attention U-curve (start/end = high attention, middle = low).
+> Critical instructions go at start and end. Navigable data goes in the middle.
+> See: Chroma "Context Rot" (2025) — performance degrades ~2%/100K tokens; distractors and semantic ambiguity compound degradation.
+
 **Common preamble (include in all agent prompts):**
 ```
 Working directory: {worktree_absolute_path}
 All file operations MUST use this absolute path as base. Do NOT write files to the main project directory.
 Commit format: {commit_type}({spec}): {description}
-
-STOP after committing. Do NOT merge branches, rename spec files, remove worktrees, or run git checkout on main.
 ```
 
-**Standard Task:**
+**Standard Task** (spawn with `Agent(model="{Model from PLAN.md}", ...)`):
+
+Prompt sections in order (START = high attention, MIDDLE = navigable data, END = high attention):
+
 ```
+--- START (high attention zone) ---
+
 {task_id}: {description from PLAN.md}
 Files: {target files}  Spec: {spec_name}
-{Impact block from PLAN.md — include verbatim if present}
 
 {Prior failure context — include ONLY if task was previously reverted. Read from .deepflow/auto-memory.yaml revert_history for this task_id:}
-Previous attempts (DO NOT repeat these approaches):
-- Cycle {N}: reverted — "{reason from revert_history}"
+DO NOT repeat these approaches:
 - Cycle {N}: reverted — "{reason from revert_history}"
 {Omit this entire block if task has no revert history.}
 
-CRITICAL: If Impact lists duplicates or callers, you MUST verify each one is consistent with your changes.
-- [active] duplicates → consolidate into single source of truth (e.g., local generateYAML → use shared buildConfigData)
-- [dead] duplicates → DELETE the dead code entirely. Dead code pollutes context and causes drift.
+{Acceptance criteria excerpt — extract 2-3 key ACs from the spec file (specs/doing-*.md). Include only the criteria relevant to THIS task, not the full spec.}
+Success criteria:
+- {AC relevant to this task}
+- {AC relevant to this task}
+{Omit if spec has no structured ACs.}
+
+--- MIDDLE (navigable data zone) ---
+
+{Impact block from PLAN.md — include verbatim if present. Annotate each caller with WHY it's impacted:}
+Impact:
+  - Callers: {file} ({why — e.g. "imports validateToken which you're changing"})
+  - Duplicates:
+    - {file} [active — consolidate]
+    - {file} [dead — DELETE]
+  - Data flow: {consumers}
+{Omit if no Impact in PLAN.md.}
+
+{Dependency context — for each completed blocker task, include a one-liner summary:}
+Prior tasks:
+- {dep_task_id}: {one-line summary of what changed — e.g. "refactored validateToken to async, changed signature (string) → (string, opts)"}
+{Omit if task has no dependencies or all deps are bootstrap/spike tasks.}
 
 Steps:
 1. External APIs/SDKs → chub search "<library>" --json → chub get <id> --lang <lang> (skip if chub unavailable or internal code only)
-2. Read ALL files in Impact before implementing — understand the full picture
-3. Implement the task, updating all impacted files
-4. Commit as feat({spec}): {description}
+2. LSP freshness check: run `findReferences` on each function/type you're about to change. If callers exist beyond the Impact list, add them to your scope before implementing.
+3. Read ALL files in Impact (+ any new callers from step 2) before implementing — understand the full picture
+4. Implement the task, updating all impacted files
+5. Commit as feat({spec}): {description}
 
+--- END (high attention zone) ---
+
+{If .deepflow/auto-memory.yaml exists and has probe_learnings, include:}
+Spike results (follow these approaches):
+{each probe_learning with outcome "winner" → "- {insight}"}
+{Omit this block if no probe_learnings exist.}
+
+If Impact lists duplicates: [active] → consolidate into single source of truth. [dead] → DELETE entirely.
 Your ONLY job is to write code and commit. Orchestrator runs health checks after.
+STOP after committing. Do NOT merge branches, rename spec files, remove worktrees, or run git checkout on main.
 ```
 
+**Effort-aware context budget:** For `Effort: low` tasks, omit the MIDDLE section entirely (no Impact, no dependency context, no steps). For `Effort: medium`, include Impact but omit dependency context. For `Effort: high`, include everything.
+
 **Bootstrap Task:**
 ```
 BOOTSTRAP: Write tests for files in edit_scope
@@ -226,7 +288,7 @@ Commit as test({spec}): bootstrap tests for edit_scope
 Files: {target files}  Spec: {spec_name}
 
 {Prior failure context — include ONLY if this spike was previously reverted. Read from .deepflow/auto-memory.yaml revert_history + spike_insights for this task_id:}
-Previous attempts (DO NOT repeat these approaches):
+DO NOT repeat these approaches:
 - Cycle {N}: reverted — "{reason}"
 {Omit this entire block if no revert history.}
 
@@ -266,7 +328,19 @@ When all tasks done for a `doing-*` spec:
 | Implementation | `general-purpose` | Task implementation |
 | Debugger | `reasoner` | Debugging failures |
 
-**Model routing:** Use `model:` from command/agent/skill frontmatter. Default: `sonnet`.
+**Model + effort routing:** Read `Model:` and `Effort:` fields from each task block in PLAN.md. Pass `model:` parameter when spawning the agent. Prepend effort instruction to the agent prompt. Defaults: `Model: sonnet`, `Effort: medium`.
+
+| Task fields | Agent call | Prompt preamble |
+|-------------|-----------|-----------------|
+| `Model: haiku, Effort: low` | `Agent(model="haiku", ...)` | `You MUST be maximally efficient: skip explanations, minimize tool calls, go straight to implementation.` |
+| `Model: sonnet, Effort: medium` | `Agent(model="sonnet", ...)` | `Be direct and efficient. Explain only when the logic is non-obvious.` |
+| `Model: opus, Effort: high` | `Agent(model="opus", ...)` | _(no preamble — default behavior)_ |
+| (missing) | `Agent(model="sonnet", ...)` | `Be direct and efficient. Explain only when the logic is non-obvious.` |
+
+**Effort preamble rules:**
+- `low` → Prepend efficiency instruction. Agent should make fewest possible tool calls.
+- `medium` → Prepend balanced instruction. Agent skips preamble but explains non-obvious decisions.
+- `high` → No preamble added. Agent uses full reasoning capabilities.
 
 **Checkpoint schema:** `.deepflow/checkpoint.json` in worktree:
 ```json

package/src/commands/df/plan.md

@@ -75,13 +75,13 @@ Use `code-completeness` skill to search for: implementations matching spec requi
 
 For each file in a task's "Files:" list, find the full blast radius.
 
-**Search for:**
+**Search for (prefer LSP, fallback to grep):**
 
-1. **Callers:** `grep -r "{exported_function}" --include="*.{ext}" -l` — files that import/call what's being changed
+1. **Callers:** Use LSP `findReferences` / `incomingCalls` on each exported function/type being changed. Annotate each caller with WHY it's impacted (e.g. "imports validateToken which this task changes"). Fallback: `grep -r "{exported_function}" --include="*.{ext}" -l`
 2. **Duplicates:** Files with similar logic (same function name, same transformation). Classify:
    - `[active]` — used in production → must consolidate
    - `[dead]` — bypassed/unreachable → must delete
-3. **Data flow:** If file produces/transforms data, find ALL consumers of that shape across languages
+3. **Data flow:** If file produces/transforms data, use LSP `outgoingCalls` to trace consumers. Fallback: grep across languages
 
 **Embed as `Impact:` block in each task:**
 ```markdown
@@ -133,6 +133,47 @@ Spawn `Task(subagent_type="reasoner", model="opus")`. Map each requirement to DO
 
 Priority: Dependencies → Impact → Risk
 
+### 5.5. CLASSIFY MODEL + EFFORT PER TASK
+
+For each task, assign `Model:` and `Effort:` based on the routing matrix:
+
+#### Routing matrix
+
+| Task type | Model | Effort | Rationale |
+|-----------|-------|--------|-----------|
+| Bootstrap (scaffold, config, rename) | `haiku` | `low` | Mechanical, pattern-following, zero ambiguity |
+| browse-fetch (doc retrieval) | `haiku` | `low` | Just fetching and extracting, no reasoning |
+| Single-file simple addition | `haiku` | `high` | Small scope but needs to get it right |
+| Multi-file with clear specs | `sonnet` | `medium` | Standard work, specs remove need for deep thinking |
+| Bug fix (clear repro) | `sonnet` | `medium` | Diagnosis done, just apply fix |
+| Bug fix (unclear cause) | `sonnet` | `high` | Needs reasoning to find root cause |
+| Spike / validation | `sonnet` | `high` | Scoped but needs reasoning to validate hypothesis |
+| Feature work (well-specced) | `sonnet` | `medium` | Clear ACs reduce thinking overhead |
+| Feature work (ambiguous ACs) | `opus` | `medium` | Needs intelligence but effort can be moderate with good specs |
+| Refactor (>5 files, many callers) | `opus` | `medium` | Blast radius needs intelligence, patterns are repetitive |
+| Architecture change | `opus` | `high` | High complexity + high ambiguity |
+| Unfamiliar API integration | `opus` | `high` | Needs deep reasoning about unknown patterns |
+| Retried after revert | _(raise one level)_ | `high` | Prior failure means harder than expected |
+
+#### Decision inputs
+
+1. **File count** — 1 file → haiku/sonnet, 2-5 → sonnet, >5 → sonnet/opus
+2. **Impact blast radius** — many callers/duplicates → raise model
+3. **Spec clarity** — clear ACs → lower effort, ambiguous → raise effort
+4. **Type** — spikes → `sonnet high`, bootstrap → `haiku low`
+5. **Has prior failures** — raise model one level AND set effort to `high`
+6. **Repetitiveness** — repetitive pattern across files → lower effort even at higher model
+
+#### Effort economics
+
+Effort controls ALL token spend (text, tool calls, thinking). Lower effort = fewer tool calls, less preamble, shorter reasoning.
+
+- `low` → ~60-70% token reduction vs high. Use when task is mechanical.
+- `medium` → ~30-40% token reduction. Use when specs are clear.
+- `high` → full spend (default). Use when ambiguity or risk is high.
+
+Add `Model: haiku|sonnet|opus` and `Effort: low|medium|high` to each task block. Defaults: `Model: sonnet`, `Effort: medium`.
+
 ### 6. GENERATE SPIKE TASKS (IF NEEDED)
 
 **Spike Task Format:**
@@ -228,6 +269,7 @@ Always use `Task` tool with explicit `subagent_type` and `model`.
 
 - [ ] **T2**: Create upload endpoint
   - Files: src/api/upload.ts
+  - Model: sonnet
   - Impact:
     - Callers: src/routes/index.ts:5
     - Duplicates: backend/legacy-upload.go [dead — DELETE]
@@ -235,5 +277,6 @@ Always use `Task` tool with explicit `subagent_type` and `model`.
 
 - [ ] **T3**: Add S3 service with streaming
   - Files: src/services/storage.ts
+  - Model: opus
   - Blocked by: T1, T2
 ```

package/src/skills/browse-fetch/SKILL.md

@@ -29,8 +29,8 @@ This protocol is the reusable foundation for all browser-based skills (browse-fe
 Before launching, verify Playwright is available:
 
 ```bash
-# Prefer bun if available, fall back to node
-if which bun > /dev/null 2>&1; then RUNTIME=bun; else RUNTIME=node; fi
+# Prefer Node.js; fall back to Bun
+if which node > /dev/null 2>&1; then RUNTIME=node; elif which bun > /dev/null 2>&1; then RUNTIME=bun; else echo "Error: neither node nor bun found" && exit 1; fi
 
 $RUNTIME -e "require('playwright')" 2>/dev/null \
   || npx --yes playwright install chromium --with-deps 2>&1 | tail -5
@@ -41,8 +41,8 @@ If installation fails, fall back to WebFetch (see Fallback section below).
 ### 2. Launch Command
 
 ```bash
-# Detect runtime
-if which bun > /dev/null 2>&1; then RUNTIME=bun; else RUNTIME=node; fi
+# Detect runtime — prefer Node.js per decision
+if which node > /dev/null 2>&1; then RUNTIME=node; elif which bun > /dev/null 2>&1; then RUNTIME=bun; else echo "Error: neither node nor bun found" && exit 1; fi
 
 $RUNTIME -e "
 const { chromium } = require('playwright');
@@ -74,13 +74,100 @@ await page.waitForTimeout(1500);
 
 ### 4. Content Extraction
 
-Extract the main readable text, not raw HTML:
+Extract content as **structured Markdown** optimized for LLM consumption (not raw HTML or flat text).
 
 ```js
-// Primary: semantic content containers
-let text = await page.innerText('main, article, [role="main"]').catch(() => '');
+// Convert DOM to Markdown inside the browser context — zero dependencies
+let text = await page.evaluate(() => {
+  // Remove noise elements
+  const noise = 'nav, footer, header, aside, script, style, noscript, svg, [role="navigation"], [role="banner"], [role="contentinfo"], .cookie-banner, #cookie-consent';
+  document.querySelectorAll(noise).forEach(el => el.remove());
+
+  // Pick main content container
+  const root = document.querySelector('main, article, [role="main"]') || document.body;
+
+  function md(node, listDepth = 0) {
+    if (node.nodeType === 3) return node.textContent;
+    if (node.nodeType !== 1) return '';
+    const tag = node.tagName.toLowerCase();
+    const children = () => Array.from(node.childNodes).map(c => md(c, listDepth)).join('');
+
+    // Skip hidden elements
+    if (node.getAttribute('aria-hidden') === 'true' || node.hidden) return '';
+
+    switch (tag) {
+      case 'h1': case 'h2': case 'h3': case 'h4': case 'h5': case 'h6': {
+        const level = '#'.repeat(parseInt(tag[1]));
+        const text = node.textContent.trim();
+        return text ? '\n\n' + level + ' ' + text + '\n\n' : '';
+      }
+      case 'p': return '\n\n' + children().trim() + '\n\n';
+      case 'br': return '\n';
+      case 'hr': return '\n\n---\n\n';
+      case 'strong': case 'b': { const t = children().trim(); return t ? '**' + t + '**' : ''; }
+      case 'em': case 'i': { const t = children().trim(); return t ? '*' + t + '*' : ''; }
+      case 'code': {
+        const t = node.textContent;
+        return node.parentElement && node.parentElement.tagName.toLowerCase() === 'pre' ? t : '`' + t + '`';
+      }
+      case 'pre': {
+        const code = node.querySelector('code');
+        const lang = code ? (code.className.match(/language-(\w+)/)||[])[1] || '' : '';
+        const t = (code || node).textContent.trim();
+        return '\n\n```' + lang + '\n' + t + '\n```\n\n';
+      }
+      case 'a': {
+        const href = node.getAttribute('href');
+        const t = children().trim();
+        return (href && t && !href.startsWith('#')) ? '[' + t + '](' + href + ')' : t;
+      }
+      case 'img': {
+        const alt = node.getAttribute('alt') || '';
+        return alt ? '[image: ' + alt + ']' : '';
+      }
+      case 'ul': case 'ol': return '\n\n' + children() + '\n';
+      case 'li': {
+        const indent = '  '.repeat(listDepth);
+        const bullet = node.parentElement && node.parentElement.tagName.toLowerCase() === 'ol'
+          ? (Array.from(node.parentElement.children).indexOf(node) + 1) + '. '
+          : '- ';
+        const content = Array.from(node.childNodes).map(c => {
+          const t = c.tagName && (c.tagName.toLowerCase() === 'ul' || c.tagName.toLowerCase() === 'ol')
+            ? md(c, listDepth + 1) : md(c, listDepth);
+          return t;
+        }).join('').trim();
+        return indent + bullet + content + '\n';
+      }
+      case 'table': {
+        const rows = Array.from(node.querySelectorAll('tr'));
+        if (!rows.length) return '';
+        const matrix = rows.map(r => Array.from(r.querySelectorAll('th, td')).map(c => c.textContent.trim()));
+        const cols = Math.max(...matrix.map(r => r.length));
+        const widths = Array.from({length: cols}, (_, i) => Math.max(...matrix.map(r => (r[i]||'').length), 3));
+        let out = '\n\n';
+        matrix.forEach((row, ri) => {
+          out += '| ' + Array.from({length: cols}, (_, i) => (row[i]||'').padEnd(widths[i])).join(' | ') + ' |\n';
+          if (ri === 0) out += '| ' + widths.map(w => '-'.repeat(w)).join(' | ') + ' |\n';
+        });
+        return out + '\n';
+      }
+      case 'blockquote': return '\n\n> ' + children().trim().replace(/\n/g, '\n> ') + '\n\n';
+      case 'dl': return '\n\n' + children() + '\n';
+      case 'dt': return '**' + children().trim() + '**\n';
+      case 'dd': return ': ' + children().trim() + '\n';
+      case 'div': case 'section': case 'span': case 'figure': case 'figcaption':
+        return children();
+      default: return children();
+    }
+  }
+
+  let result = md(root);
+  // Collapse excessive whitespace
+  result = result.replace(/\n{3,}/g, '\n\n').trim();
+  return result;
+});
 
-// Fallback: full body text
+// Fallback if extraction is too short
 if (!text || text.trim().length < 100) {
   text = await page.innerText('body').catch(() => '');
 }
@@ -134,11 +221,13 @@ await browser.close();
 
 ## Fetch Workflow
 
-**Goal:** retrieve and return the text content of a single URL.
+**Goal:** retrieve and return structured Markdown content of a single URL.
+
+The full inline script uses `page.evaluate()` to convert DOM → Markdown inside the browser (zero Node dependencies). Adapt the URL per query.
 
 ```bash
-# Full inline script — adapt URL and selector per query
-if which bun > /dev/null 2>&1; then RUNTIME=bun; else RUNTIME=node; fi
+# Full inline script — adapt URL per query
+if which node > /dev/null 2>&1; then RUNTIME=node; elif which bun > /dev/null 2>&1; then RUNTIME=bun; else echo "Error: neither node nor bun found" && exit 1; fi
 
 $RUNTIME -e "
 const { chromium } = require('playwright');
@@ -157,14 +246,83 @@ const { chromium } = require('playwright');
     await page.waitForTimeout(1500);
 
     const title = await page.title();
-    const url   = page.url();
+    const url = page.url();
 
     if (/sign.?in|log.?in|auth/i.test(title) || url.includes('/login')) {
       console.log('[browse-fetch] Blocked by login wall at ' + url);
       return;
     }
 
-    let text = await page.innerText('main, article, [role=\"main\"]').catch(() => '');
+    let text = await page.evaluate(() => {
+      const noise = 'nav, footer, header, aside, script, style, noscript, svg, [role=\"navigation\"], [role=\"banner\"], [role=\"contentinfo\"], .cookie-banner, #cookie-consent';
+      document.querySelectorAll(noise).forEach(el => el.remove());
+      const root = document.querySelector('main, article, [role=\"main\"]') || document.body;
+
+      function md(node, listDepth) {
+        listDepth = listDepth || 0;
+        if (node.nodeType === 3) return node.textContent;
+        if (node.nodeType !== 1) return '';
+        var tag = node.tagName.toLowerCase();
+        var kids = function() { return Array.from(node.childNodes).map(function(c) { return md(c, listDepth); }).join(''); };
+        if (node.getAttribute('aria-hidden') === 'true' || node.hidden) return '';
+        switch (tag) {
+          case 'h1': case 'h2': case 'h3': case 'h4': case 'h5': case 'h6':
+            var level = '#'.repeat(parseInt(tag[1]));
+            var t = node.textContent.trim();
+            return t ? '\\n\\n' + level + ' ' + t + '\\n\\n' : '';
+          case 'p': return '\\n\\n' + kids().trim() + '\\n\\n';
+          case 'br': return '\\n';
+          case 'hr': return '\\n\\n---\\n\\n';
+          case 'strong': case 'b': var s = kids().trim(); return s ? '**' + s + '**' : '';
+          case 'em': case 'i': var e = kids().trim(); return e ? '*' + e + '*' : '';
+          case 'code':
+            var ct = node.textContent;
+            return node.parentElement && node.parentElement.tagName.toLowerCase() === 'pre' ? ct : '\`' + ct + '\`';
+          case 'pre':
+            var codeEl = node.querySelector('code');
+            var lang = codeEl ? ((codeEl.className.match(/language-(\\w+)/) || [])[1] || '') : '';
+            var pt = (codeEl || node).textContent.trim();
+            return '\\n\\n\`\`\`' + lang + '\\n' + pt + '\\n\`\`\`\\n\\n';
+          case 'a':
+            var href = node.getAttribute('href');
+            var at = kids().trim();
+            return (href && at && !href.startsWith('#')) ? '[' + at + '](' + href + ')' : at;
+          case 'img':
+            var alt = node.getAttribute('alt') || '';
+            return alt ? '[image: ' + alt + ']' : '';
+          case 'ul': case 'ol': return '\\n\\n' + kids() + '\\n';
+          case 'li':
+            var indent = '  '.repeat(listDepth);
+            var bullet = node.parentElement && node.parentElement.tagName.toLowerCase() === 'ol'
+              ? (Array.from(node.parentElement.children).indexOf(node) + 1) + '. ' : '- ';
+            var content = Array.from(node.childNodes).map(function(c) {
+              var tg = c.tagName && c.tagName.toLowerCase();
+              return (tg === 'ul' || tg === 'ol') ? md(c, listDepth + 1) : md(c, listDepth);
+            }).join('').trim();
+            return indent + bullet + content + '\\n';
+          case 'table':
+            var rows = Array.from(node.querySelectorAll('tr'));
+            if (!rows.length) return '';
+            var matrix = rows.map(function(r) { return Array.from(r.querySelectorAll('th, td')).map(function(c) { return c.textContent.trim(); }); });
+            var cols = Math.max.apply(null, matrix.map(function(r) { return r.length; }));
+            var widths = Array.from({length: cols}, function(_, i) { return Math.max.apply(null, matrix.map(function(r) { return (r[i]||'').length; }).concat([3])); });
+            var out = '\\n\\n';
+            matrix.forEach(function(row, ri) {
+              out += '| ' + Array.from({length: cols}, function(_, i) { return (row[i]||'').padEnd(widths[i]); }).join(' | ') + ' |\\n';
+              if (ri === 0) out += '| ' + widths.map(function(w) { return '-'.repeat(w); }).join(' | ') + ' |\\n';
+            });
+            return out + '\\n';
+          case 'blockquote': return '\\n\\n> ' + kids().trim().replace(/\\n/g, '\\n> ') + '\\n\\n';
+          case 'dt': return '**' + kids().trim() + '**\\n';
+          case 'dd': return ': ' + kids().trim() + '\\n';
+          default: return kids();
+        }
+      }
+
+      var result = md(root);
+      return result.replace(/\\n{3,}/g, '\\n\\n').trim();
+    });
+
     if (!text || text.trim().length < 100) {
       text = await page.innerText('body').catch(() => '');
     }
@@ -182,7 +340,7 @@ const { chromium } = require('playwright');
 "
 ```
 
-Adapt the URL and selector per query. The agent inlines the full script via `node -e` or `bun -e` so no temp files are needed for extractions under ~4000 tokens.
+The agent inlines the full script via `node -e` or `bun -e` so no temp files are needed for extractions under ~4000 tokens.
 
 ---
 
@@ -250,7 +408,7 @@ If WebFetch also fails, return the URL with an explanation and continue the task
 ## Rules
 
 - Always run the install check before the first browser launch in a session.
-- Detect runtime with `which bun` first; use `node` if bun is absent.
+- Detect runtime with `which node` first; fall back to `bun` if node is absent.
 - Never navigate to Google or DuckDuckGo with Playwright — use WebSearch tool or direct URLs.
 - Truncate output at ~4000 tokens (~16 000 chars) to protect context budget.
 - On login wall or CAPTCHA, log the block, skip, and continue — never retry infinitely.

package/templates/config-template.yaml

@@ -62,6 +62,12 @@ worktree:
   # Keep worktree after failed execution for debugging
   cleanup_on_fail: false
 
+  # Sparse checkout paths (for large monorepos only)
+  # When set, worktrees checkout only these directories via git sparse-checkout
+  # Leave empty for full checkout (default, works for most repos)
+  # Example: ["src/", "tests/", "package.json", "tsconfig.json"]
+  sparse_paths: []
+
 # Quality gates for /df:verify
 quality:
   # Override auto-detected build command (e.g., "npm run build", "cargo build")