@mccune1224/pi-pstack 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +52 -100
- package/agents/comment-sicko.md +1 -1
- package/agents/poteto-agent.md +2 -2
- package/docs/compare.md +122 -0
- package/docs/guide/01-setup.md +37 -12
- package/docs/guide/02-poteto-mode.md +1 -1
- package/docs/guide/03-understand.md +0 -8
- package/docs/guide/05-build-and-clean.md +1 -1
- package/docs/guide/06-verify-and-ship.md +1 -1
- package/docs/guide/07-overnight.md +2 -2
- package/docs/guide/08-principles.md +4 -2
- package/docs/guide/10-recipes-and-pitfalls.md +2 -2
- package/docs/guide/README.md +2 -2
- package/extensions/compaction.test.ts +132 -0
- package/extensions/compaction.ts +183 -0
- package/extensions/context-guard.test.ts +53 -0
- package/extensions/context-guard.ts +65 -0
- package/extensions/doctor.test.ts +85 -0
- package/extensions/doctor.ts +100 -0
- package/extensions/index.ts +289 -26
- package/extensions/settings.test.ts +71 -16
- package/extensions/settings.ts +57 -9
- package/extensions/skill-budget.test.ts +110 -0
- package/extensions/skill-budget.ts +101 -0
- package/package.json +9 -4
- package/scripts/check-pi-isms.mjs +106 -0
- package/scripts/lib/pi-delta-rules.mjs +110 -0
- package/scripts/lib/pi-delta-rules.test.mjs +100 -0
- package/scripts/lib/upstream-vendor.mjs +32 -0
- package/scripts/pi-deltas.json +754 -0
- package/scripts/sync-upstream.mjs +117 -0
- package/scripts/verify-guards.mjs +132 -0
- package/scripts/verify-sync.mjs +206 -0
- package/skills/architect/SKILL.md +10 -10
- package/skills/architect/references/rationale-template.md +2 -2
- package/skills/architect/references/runner-prompt.md +7 -7
- package/skills/arena/SKILL.md +11 -11
- package/skills/automate-me/SKILL.md +21 -26
- package/skills/blast-radius/SKILL.md +5 -5
- package/skills/create-verification-skill/SKILL.md +5 -5
- package/skills/figure-it-out/SKILL.md +11 -13
- package/skills/how/SKILL.md +30 -104
- package/skills/how/references/explainer-prompt.md +6 -6
- package/skills/how/references/explorer-prompt.md +1 -1
- package/skills/interrogate/SKILL.md +13 -14
- package/skills/interrogate/references/code-quality-review.md +1 -1
- package/skills/interrogate/references/lead-judgment.md +1 -1
- package/skills/interrogate/references/rubric.md +2 -2
- package/skills/maintain-verification-skill/SKILL.md +5 -5
- package/skills/no-comments/SKILL.md +3 -3
- package/skills/poteto-mode/SKILL.md +9 -8
- package/skills/poteto-mode/playbooks/authoring-a-skill.md +4 -4
- package/skills/poteto-mode/playbooks/autonomous-run.md +4 -4
- package/skills/poteto-mode/playbooks/autopilot-full.md +9 -9
- package/skills/poteto-mode/playbooks/autopilot-stack.md +8 -8
- package/skills/poteto-mode/playbooks/babysit.md +17 -17
- package/skills/poteto-mode/playbooks/bug-fix.md +6 -6
- package/skills/poteto-mode/playbooks/eval.md +6 -8
- package/skills/poteto-mode/playbooks/feature.md +7 -7
- package/skills/poteto-mode/playbooks/hillclimb.md +10 -10
- package/skills/poteto-mode/playbooks/investigation.md +3 -3
- package/skills/poteto-mode/playbooks/multi-phase-plan.md +26 -25
- package/skills/poteto-mode/playbooks/opening-a-pr.md +17 -13
- package/skills/poteto-mode/playbooks/orchestrate.md +37 -39
- package/skills/poteto-mode/playbooks/pause-safely.md +4 -4
- package/skills/poteto-mode/playbooks/perf-issue.md +9 -9
- package/skills/poteto-mode/playbooks/prototype.md +5 -5
- package/skills/poteto-mode/playbooks/refactoring.md +9 -9
- package/skills/poteto-mode/playbooks/runtime-forensics.md +3 -3
- package/skills/poteto-mode/playbooks/session-pickup.md +4 -6
- package/skills/poteto-mode/playbooks/shipping.md +11 -14
- package/skills/poteto-mode/playbooks/trace-forensics.md +4 -4
- package/skills/poteto-mode/playbooks/visual-parity.md +3 -3
- package/skills/poteto-mode/playbooks/worktree-cleanup.md +4 -4
- package/skills/poteto-mode/references/bugbot-triage.md +3 -3
- package/skills/poteto-mode/scripts/worktree-audit.mjs +0 -7
- package/skills/principle-attack-the-premise/SKILL.md +23 -0
- package/skills/principle-boundary-discipline/SKILL.md +2 -2
- package/skills/principle-build-the-lever/SKILL.md +5 -5
- package/skills/principle-encode-lessons-in-structure/SKILL.md +3 -3
- package/skills/principle-experience-first/SKILL.md +5 -5
- package/skills/principle-fix-root-causes/SKILL.md +4 -4
- package/skills/principle-foundational-thinking/SKILL.md +3 -3
- package/skills/principle-guard-the-context-window/SKILL.md +2 -2
- package/skills/principle-laziness-protocol/SKILL.md +2 -2
- package/skills/principle-migrate-callers-then-delete-legacy-apis/SKILL.md +1 -1
- package/skills/principle-minimize-reader-load/SKILL.md +2 -2
- package/skills/principle-model-the-domain/SKILL.md +3 -3
- package/skills/principle-never-block-on-the-human/SKILL.md +4 -5
- package/skills/principle-prove-it-works/SKILL.md +3 -3
- package/skills/principle-redesign-from-first-principles/SKILL.md +3 -3
- package/skills/principle-separate-before-serializing-shared-state/SKILL.md +3 -3
- package/skills/principle-sequence-verifiable-units/SKILL.md +3 -3
- package/skills/principle-subtract-before-you-add/SKILL.md +3 -4
- package/skills/principle-test-behavior-not-implementation/SKILL.md +25 -0
- package/skills/principle-type-system-discipline/SKILL.md +5 -5
- package/skills/recall/SKILL.md +7 -7
- package/skills/reflect/SKILL.md +14 -20
- package/skills/reflect/references/divergent-reviewer.md +3 -3
- package/skills/reflect/references/judgment-reviewer.md +3 -3
- package/skills/reflect/references/synthesizer.md +2 -2
- package/skills/reflect/references/tooling-reviewer.md +4 -6
- package/skills/setup-pstack/SKILL.md +61 -49
- package/skills/show-me-your-work/SKILL.md +17 -17
- package/skills/show-me-your-work/scripts/log.mjs +2 -8
- package/skills/swarm/SKILL.md +6 -6
- package/skills/tdd/SKILL.md +2 -2
- package/skills/teach/SKILL.md +7 -7
- package/skills/technical-writing/SKILL.md +5 -8
- package/skills/typescript-best-practices/SKILL.md +6 -3
- package/skills/typescript-best-practices/references/patterns.md +35 -14
- package/skills/unslop/SKILL.md +8 -20
- package/skills/why/SKILL.md +27 -100
- package/skills/why/references/epistemics.md +6 -6
- package/skills/why/references/investigator-prompt.md +5 -5
- package/skills/why/references/source-playbook.md +1 -1
- package/skills/why/references/sources/databricks.md +7 -7
- package/skills/why/references/sources/datadog.md +5 -5
- package/skills/why/references/sources/incident-postmortem.md +2 -2
- package/skills/why/references/sources/linear.md +2 -2
- package/skills/why/references/sources/notion.md +3 -3
- package/skills/why/references/sources/sentry.md +4 -4
- package/skills/why/references/sources/slack.md +2 -2
- package/skills/why/references/synthesizer-prompt.md +3 -3
- package/skills/how/references/critic-prompt.md +0 -59
- package/skills/how/references/critique-rubric.md +0 -58
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Investigator Prompt Template
|
|
2
2
|
|
|
3
|
-
Build each investigator's prompt from this template
|
|
3
|
+
Build each investigator's prompt from this template. Fill in the placeholders. Append the single category playbook `sources/<source>.md` matching this investigator's evidence category (see `source-playbook.md` for the index). If the target code looks defensive (null checks, retry logic, timeout handling, rate limiting, feature flags, egress guards, OOM handlers), also append `sources/incident-postmortem.md` for the incident-flavored queries to run inside its own source.
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -10,7 +10,7 @@ Other investigators search different sources in parallel. Don't try to cover eve
|
|
|
10
10
|
|
|
11
11
|
## Operating Posture
|
|
12
12
|
|
|
13
|
-
Work like a careful, cautious, precise investigator. Don't produce a narrative
|
|
13
|
+
Work like a careful, cautious, precise investigator. Don't produce a narrative. Surface evidence and describe it accurately, including the parts that don't fit a tidy story. The more boring and exact your output, the more useful it is. A single verbatim quote with a precise citation beats a paragraph of plausible-sounding summary.
|
|
14
14
|
|
|
15
15
|
- **Quote, don't paraphrase** when the exact wording matters. Citations should let the reader jump to the source and confirm the claim in seconds.
|
|
16
16
|
- **Go wide before going deep.** Cast a broad first net so you don't miss related context. Only then narrow in.
|
|
@@ -44,16 +44,16 @@ Work like a careful, cautious, precise investigator. Don't produce a narrative;
|
|
|
44
44
|
|
|
45
45
|
## Investigation Instructions
|
|
46
46
|
|
|
47
|
-
Gather **evidence
|
|
47
|
+
Gather **evidence**. Don't answer the question directly. The synthesizer weighs the evidence and forms conclusions. Follow this loop:
|
|
48
48
|
|
|
49
49
|
1. **Cast a wide net first.** Start broad so you don't miss related context, then narrow in on specific items.
|
|
50
50
|
2. **Read the whole thing.** Read any PR, ticket, doc, or thread fully, not just the title or summary. The key evidence is often buried in a comment, a subtask, or a follow-up.
|
|
51
|
-
3. **Follow links within your assigned source.** If a PR references another PR or commit, pull it. If a ticket links a parent or sibling, pull it. If a doc links another doc, pull it. Stay inside your assigned source. When you spot a cross-source reference, do NOT chase it yourself. Record it under "Additional Leads" so the investigator assigned to that source can pick it up. The one-investigator-per-category design depends on this
|
|
51
|
+
3. **Follow links within your assigned source.** If a PR references another PR or commit, pull it. If a ticket links a parent or sibling, pull it. If a doc links another doc, pull it. Stay inside your assigned source. When you spot a cross-source reference, do NOT chase it yourself. Record it under "Additional Leads" so the investigator assigned to that source can pick it up. The one-investigator-per-category design depends on this. Chasing cross-source links duplicates work and confuses scope.
|
|
52
52
|
4. **Capture quotes verbatim** with their location (PR number, ticket ID, URL, commit hash, file:line). The synthesizer needs to cite this precisely.
|
|
53
53
|
5. **Note absences.** If you searched for something and came up empty, that's also a finding. Record what you searched for and what you didn't find.
|
|
54
54
|
6. **Watch for contradictions.** If two items in your source disagree, record both. Don't suppress the inconvenient one.
|
|
55
55
|
|
|
56
|
-
Don't synthesize or form a final opinion on "the why." Collect the raw material honestly and completely
|
|
56
|
+
Don't synthesize or form a final opinion on "the why." Collect the raw material honestly and completely. The synthesizer does the reasoning.
|
|
57
57
|
|
|
58
58
|
## Epistemic Discipline
|
|
59
59
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Source playbooks
|
|
2
2
|
|
|
3
|
-
The why skill spawns one investigator per available evidence category, each reading a single source-specific playbook below. The playbooks are concrete examples for common MCPs
|
|
3
|
+
The why skill spawns one investigator per available evidence category, each reading a single source-specific playbook below. The playbooks are concrete examples for common MCPs. Adapt them for a different MCP in the same category.
|
|
4
4
|
|
|
5
5
|
| Category | Playbook | Example MCP it documents |
|
|
6
6
|
|---|---|---|
|
|
@@ -2,20 +2,20 @@
|
|
|
2
2
|
|
|
3
3
|
## What this source contains
|
|
4
4
|
|
|
5
|
-
Databricks is the product-analytics, data-pipeline, and warehouse-telemetry layer. It complements Datadog
|
|
5
|
+
Databricks is the product-analytics, data-pipeline, and warehouse-telemetry layer. It complements Datadog. Datadog is the *infra/runtime* view, Databricks is the *product/data* view (what users did, which experiments ran, how feature usage evolved, where a threshold constant came from).
|
|
6
6
|
|
|
7
7
|
- **Product analytics events.** `your_warehouse.events.analytics_track_event` (raw) and typed, deduplicated per-event dbt models in `<your_analytics_db>.<schema>.<table>`. User behavior: feature invocations, clicks, accepts/rejects, submissions, client-reported errors.
|
|
8
|
-
- **Usage & billing events.** `your_warehouse.events.usage_event` / `<your_analytics_db>.<schema>.stg_usage_events
|
|
8
|
+
- **Usage & billing events.** `your_warehouse.events.usage_event` / `<your_analytics_db>.<schema>.stg_usage_events`, `your_warehouse.events.raw_model_event` / `<your_analytics_db>.<schema>.stg_raw_model_events`. For cost- or volume-driven decisions.
|
|
9
9
|
- **Experiment / feature-flag data.** Exposure and outcome tables. **Schema is company-specific.** Probe with `SHOW TABLES` before assuming names.
|
|
10
10
|
- **System tables.** `system.query.history`, `system.compute.warehouses`, `system.billing.*`, `system.access.audit`. Answer "was this query expensive?", "how often did anyone run this?", "when did warehouse load spike?"
|
|
11
|
-
- **dbt lineage.** Models in `<your_analytics_db>.<schema>` reveal what pipelines depend on a table/field
|
|
11
|
+
- **dbt lineage.** Models in `<your_analytics_db>.<schema>` reveal what pipelines depend on a table/field. Upstream changes frequently motivate consumer-code changes.
|
|
12
12
|
- **Databricks notebooks.** Exploratory analyses engineers wrote before code changes. **Not queryable via the SQL MCP.** If you suspect the rationale lives in a notebook, name it as a gap.
|
|
13
13
|
|
|
14
14
|
## How to search it
|
|
15
15
|
|
|
16
16
|
Use the Databricks SQL MCP. Primary tool: `execute_sql_read_only`. If it returns a `statement_id`, poll with `poll_sql_result` rather than re-running.
|
|
17
17
|
|
|
18
|
-
**Orient before querying.** Schemas are company-specific
|
|
18
|
+
**Orient before querying.** Schemas are company-specific. Probe before trusting a table name:
|
|
19
19
|
|
|
20
20
|
```sql
|
|
21
21
|
SHOW TABLES IN <your_analytics_db>.<schema> LIKE '*<keyword>*';
|
|
@@ -24,7 +24,7 @@ DESCRIBE TABLE <your_analytics_db>.<schema>.stg_<event>;
|
|
|
24
24
|
|
|
25
25
|
**Time-bound every query.** These tables are huge and unconstrained scans time out. Filter on `_timestamp` (events) or `start_time` (`system.query.history`) with a window bracketing the ship date, typically ~30 days before and after, wider only for strong reason.
|
|
26
26
|
|
|
27
|
-
**Prefer typed dbt models over the raw table.** `<your_analytics_db>.<schema>.<table>` is deduplicated, typed, and liquid-clustered
|
|
27
|
+
**Prefer typed dbt models over the raw table.** `<your_analytics_db>.<schema>.<table>` is deduplicated, typed, and liquid-clustered. `your_warehouse.events.analytics_track_event` has duplicates and untyped `properties_json`. Model-name pattern: `stg_<source>_<event_name_with_underscores>`, where `<source>` is `app`, `backend`, `website`, or `cli`. Confirm the exact model name with `SHOW TABLES` when the pattern alone doesn't resolve it. Drop to the raw table only when there's no dbt model yet, or you need events from inside the dbt refresh lag.
|
|
28
28
|
|
|
29
29
|
**Column conventions on the typed dbt models** (knowing these avoids a `DESCRIBE` round-trip):
|
|
30
30
|
|
|
@@ -53,7 +53,7 @@ Beyond the pattern shapes above:
|
|
|
53
53
|
|
|
54
54
|
- **Instrumented ≠ caused.** An event's existence means someone cared enough to log it, not that the target code exists *because* of it. Pair with a PR/commit citation from the git investigator before claiming causation.
|
|
55
55
|
- **Silent instrumentation changes.** A step function in event volume may mean a new event started being logged, not that user behavior changed. Check for instrumentation PRs in the same window before reading the ramp as a feature-launch signal.
|
|
56
|
-
- **Schema drift.** Event properties evolve
|
|
56
|
+
- **Schema drift.** Event properties evolve. A column on the typed dbt model today may not have existed when the target was written. Older data may carry the property only inside raw `properties_json`.
|
|
57
57
|
- **dbt refresh lag.** `<your_analytics_db>.<schema>.*` is rebuilt on a schedule (often hourly/daily). For events from the last few hours, fall back to `your_warehouse.events.*` and deduplicate by `_id`.
|
|
58
58
|
- **Company-specific tables.** Experiment, feature-flag, billing, and usage tables vary. Reporting a result from a table whose existence you never confirmed is a classic failure mode. Probe with `SHOW TABLES` / `DESCRIBE TABLE` first.
|
|
59
59
|
- **Retention cliff.** If the relevant window predates the table's retention or the dbt model's creation date, that's a *gap*, not a null result. Name it explicitly so the synthesizer doesn't read "no results" as "no activity."
|
|
@@ -66,5 +66,5 @@ For each relevant finding:
|
|
|
66
66
|
- Fully-qualified table name and the exact query you ran
|
|
67
67
|
- Time window queried
|
|
68
68
|
- Compact numeric summary (counts, percentiles, first/last-seen timestamps). **Don't dump raw rows.**
|
|
69
|
-
- Temporal correlation with the target's ship date (e.g., "first row 2024-08-15
|
|
69
|
+
- Temporal correlation with the target's ship date (e.g., "first row 2024-08-15, PR #49074 merged 2024-08-14")
|
|
70
70
|
- Relevance + strength: direct / circumstantial / weak
|
|
@@ -2,15 +2,15 @@
|
|
|
2
2
|
|
|
3
3
|
## What this source contains
|
|
4
4
|
|
|
5
|
-
Datadog holds the runtime record
|
|
5
|
+
Datadog holds the runtime record, what actually happened in production, as opposed to what was planned or discussed.
|
|
6
6
|
|
|
7
|
-
- **Metrics.** Counters, gauges, histograms instrumented by the team. A metric's *presence* is itself evidence
|
|
7
|
+
- **Metrics.** Counters, gauges, histograms instrumented by the team. A metric's *presence* is itself evidence. Someone thought this number worth watching.
|
|
8
8
|
- **Monitors & alerts.** Conditions the team decided warranted waking someone up. A monitor firing on `rate_limit_hit > 10/min` is direct evidence the team worried about that threshold.
|
|
9
9
|
- **Dashboards.** Curated views. The charts tell you what the team considers important for a subsystem.
|
|
10
10
|
- **APM traces & spans.** Request-level runtime data. Useful for "why is this slow" / "why is there a timeout here" questions.
|
|
11
11
|
- **Logs.** High-volume event records. Often contain the error conditions that motivated defensive code.
|
|
12
12
|
- **Incidents.** Formal incident records with timelines and linked postmortems.
|
|
13
|
-
- **Notebooks.** Exploratory investigations
|
|
13
|
+
- **Notebooks.** Exploratory investigations. Often contain hypotheses and analyses.
|
|
14
14
|
|
|
15
15
|
Datadog answers "what was the production reality around the time this code was written?", which often explains the code's shape.
|
|
16
16
|
|
|
@@ -51,7 +51,7 @@ Use the Datadog MCP. Start broad, then narrow.
|
|
|
51
51
|
analyze_datadog_logs (SQL-style aggregations, only when you need counts)
|
|
52
52
|
```
|
|
53
53
|
|
|
54
|
-
Search with symbols, error strings, or feature names. **Strongly prefer time-bounded queries** (e.g., 30 days before/after the change). Log volume is huge
|
|
54
|
+
Search with symbols, error strings, or feature names. **Strongly prefer time-bounded queries** (e.g., 30 days before/after the change). Log volume is huge. Unconstrained searches waste time and may time out.
|
|
55
55
|
|
|
56
56
|
5. **APM spans and traces.**
|
|
57
57
|
|
|
@@ -74,7 +74,7 @@ Use the Datadog MCP. Start broad, then narrow.
|
|
|
74
74
|
|
|
75
75
|
## What good evidence looks like here
|
|
76
76
|
|
|
77
|
-
- A monitor whose query and threshold match the constraint the code enforces (code clamps to 100
|
|
77
|
+
- A monitor whose query and threshold match the constraint the code enforces (code clamps to 100, monitor alerts when requests exceed 100/min)
|
|
78
78
|
- A dashboard created by the target's author, with widgets that correspond to what the code measures or guards against
|
|
79
79
|
- A metric showing a production spike immediately before the code was merged, and stable values after
|
|
80
80
|
- An incident record referencing the target code, the same symbols, or the same error strings
|
|
@@ -6,8 +6,8 @@ Not a separate source, a **cross-cutting angle**. Incidents often motivate defen
|
|
|
6
6
|
- **Linear**: look for tickets labeled `incident`, `sev-*`, `postmortem-action-item`, `reliability`
|
|
7
7
|
- **Slack**: search `#sev-*` and `#incident-*` channels around the dates the target code was added
|
|
8
8
|
- **Git**: commits with messages like "fix for incident", "add defensive check", "revert" followed by "re-apply with..." are strong signals
|
|
9
|
-
- **Datadog**: `search_datadog_incidents` for formal incident records with timelines
|
|
10
|
-
- **Sentry**: issues whose first-seen/last-seen window aligns with the target's PR ship date
|
|
9
|
+
- **Datadog**: `search_datadog_incidents` for formal incident records with timelines, dashboards and monitors created as postmortem action items
|
|
10
|
+
- **Sentry**: issues whose first-seen/last-seen window aligns with the target's PR ship date, stack traces through the target
|
|
11
11
|
- **Databricks**: product-analytics events that classify an error condition (client-reported failures, user-visible retry events, etc.) often spike during an incident window. A drop in that event count after the target PR ships is circumstantial support that the target code resolved the user-visible symptom, even when Datadog/Sentry signal is noisy.
|
|
12
12
|
|
|
13
13
|
If you find an incident link, fetch the full postmortem. Postmortems typically have an "Action Items" section that ties directly to code changes. When multiple sources corroborate (a Datadog incident ID appears in a Linear ticket, which appears in a Notion postmortem, which appears in a Slack thread that links to the target PR, and the Databricks error-event count drops after the fix), the evidence is especially strong.
|
|
@@ -18,7 +18,7 @@ Use the Linear MCP.
|
|
|
18
18
|
|
|
19
19
|
1. **Start with linked tickets.** If the seed commits or PRs reference ticket IDs (e.g., `ENG-1234`, `[BUG-567]`), fetch those first with `get_issue`. Read the full issue including comments.
|
|
20
20
|
2. **List related issues by keyword.** Use `list_issues` with text search for the feature name, key symbol, or business term. Try multiple phrasings.
|
|
21
|
-
3. **Walk the issue tree.** If you land on a sub-issue, fetch its parent. Sub-issues are tactical
|
|
21
|
+
3. **Walk the issue tree.** If you land on a sub-issue, fetch its parent. Sub-issues are tactical. Parents often carry the "why."
|
|
22
22
|
4. **Read project docs.** If the issue belongs to a project, use `get_project` and check attached docs. Project-level documents are where specs and rationale are most often captured.
|
|
23
23
|
5. **Check labels and milestones.** Labels hint at the category of motivation (customer-request, incident-followup, compliance). Milestones tie work to deadlines, which often reveal motivation.
|
|
24
24
|
|
|
@@ -42,7 +42,7 @@ Use the Linear MCP.
|
|
|
42
42
|
|
|
43
43
|
For each relevant ticket:
|
|
44
44
|
- Ticket ID and title
|
|
45
|
-
- The problem/motivation quoted from the description or comments (not paraphrased
|
|
45
|
+
- The problem/motivation quoted from the description or comments (not paraphrased. The synthesizer needs the exact text to cite)
|
|
46
46
|
- Labels, parent issue, project
|
|
47
47
|
- Author, created date, closed date
|
|
48
48
|
- Link to the ticket if available
|
|
@@ -23,7 +23,7 @@ Use the Notion MCP.
|
|
|
23
23
|
- Author handles (design docs are often authored before the code lands)
|
|
24
24
|
- Error strings or user-visible terms
|
|
25
25
|
- Time-bounded queries if you know when the code shipped
|
|
26
|
-
2. **Fetch candidate pages with `notion-fetch`.** Read the full content, not the preview
|
|
26
|
+
2. **Fetch candidate pages with `notion-fetch`.** Read the full content, not the preview. Rationale is often buried mid-document.
|
|
27
27
|
3. **Follow backlinks and child pages.** Design docs often have sub-pages for alternatives considered, appendices, or implementation notes.
|
|
28
28
|
4. **Check related databases.** `notion-query-data-sources` and `notion-query-meeting-notes` can surface meeting notes that discussed the decision.
|
|
29
29
|
5. **Search author-specific spaces.** If the PR author has a personal notebook (common at some companies), it may hold exploratory thinking that preceded the code.
|
|
@@ -38,8 +38,8 @@ Use the Notion MCP.
|
|
|
38
38
|
|
|
39
39
|
## Common pitfalls
|
|
40
40
|
|
|
41
|
-
- **Outdated docs.** Specs are often written before implementation and not updated
|
|
42
|
-
- **Doc vs. reality drift.** A spec may say "we'll do X" but the code actually does Y. Flag the divergence
|
|
41
|
+
- **Outdated docs.** Specs are often written before implementation and not updated. The doc may describe a plan that changed. Cross-check against the actual PR.
|
|
42
|
+
- **Doc vs. reality drift.** A spec may say "we'll do X" but the code actually does Y. Flag the divergence. The synthesizer will surface the contradiction.
|
|
43
43
|
- **Boilerplate templates.** Some orgs require a "Why" section that gets filled with fluff. Look for specificity.
|
|
44
44
|
- **Unlinked docs.** The most relevant doc may not be linked from anywhere. Broad keyword searches help.
|
|
45
45
|
- **Multiple drafts.** If a topic has multiple docs, find the one that was finalized or most recently updated. Check dates.
|
|
@@ -8,7 +8,7 @@ Sentry is the archive of things that went wrong. For defensive, corrective, or e
|
|
|
8
8
|
- **Events.** Individual error instances within an issue (stack traces, tags, user context)
|
|
9
9
|
- **Releases.** Deployment records with associated issues (useful for "which version fixed this?")
|
|
10
10
|
- **Replays.** Session recordings of user-facing errors (if enabled)
|
|
11
|
-
- **Profiles.** Performance profiling data (less useful for "why"
|
|
11
|
+
- **Profiles.** Performance profiling data (less useful for "why", more for "how slow")
|
|
12
12
|
- **Issue comments & assignments.** Sometimes contain engineer notes on root cause
|
|
13
13
|
|
|
14
14
|
The most valuable thing Sentry provides is **temporal correlation**: "issue X was created 2024-01-02, peaked at 500 events/day, stopped appearing after release v2.14.0 on 2024-01-15, the release that shipped the defensive check."
|
|
@@ -67,7 +67,7 @@ Use the Sentry MCP.
|
|
|
67
67
|
analyze_issue_with_seer
|
|
68
68
|
```
|
|
69
69
|
|
|
70
|
-
Seer produces AI root-cause analyses. Useful as a hypothesis generator, but treat them as inference, not authoritative. The actual events and stack traces are the primary evidence
|
|
70
|
+
Seer produces AI root-cause analyses. Useful as a hypothesis generator, but treat them as inference, not authoritative. The actual events and stack traces are the primary evidence. Seer's narrative is secondary.
|
|
71
71
|
|
|
72
72
|
## What good evidence looks like here
|
|
73
73
|
|
|
@@ -80,8 +80,8 @@ Use the Sentry MCP.
|
|
|
80
80
|
## Common pitfalls
|
|
81
81
|
|
|
82
82
|
- **Grouping drift.** Sentry groups errors by fingerprint. Refactors or renames can track the "same" error under a new issue ID. If an issue ends abruptly, the error may have just been regrouped. Check for new issues immediately after.
|
|
83
|
-
- **Release correlation is noisy.** A release contains many commits. An issue stopping at v2.14.0 doesn't prove the target fixed it
|
|
84
|
-
- **Silent fixes.** Sometimes the error stops because upstream changed, not because of the defensive code. The correlation suggests the fix
|
|
83
|
+
- **Release correlation is noisy.** A release contains many commits. An issue stopping at v2.14.0 doesn't prove the target fixed it. Another change in the same release might have. Cross-reference with the target's exact commit.
|
|
84
|
+
- **Silent fixes.** Sometimes the error stops because upstream changed, not because of the defensive code. The correlation suggests the fix. It doesn't prove authorship.
|
|
85
85
|
- **Resolved != fixed.** Issues can be marked "resolved" manually without any code change. Treat `resolved` as a human marker, not evidence that code fixed it.
|
|
86
86
|
- **Seer hallucinations.** Seer can generate confident-sounding explanations that aren't right. Fall back to the actual events, stack traces, and timestamps when making claims.
|
|
87
87
|
- **Sampling.** Some projects sample events aggressively. A low event count may just mean high sampling, not a rare error. If in doubt, note the gap.
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
- Post-merge discussions that explain why something was revisited
|
|
10
10
|
- DMs (usually not searchable, scope accordingly)
|
|
11
11
|
|
|
12
|
-
Slack is frequently where the *real* decisions got made, especially for smaller changes that didn't warrant a doc. It's also the most ephemeral source
|
|
12
|
+
Slack is frequently where the *real* decisions got made, especially for smaller changes that didn't warrant a doc. It's also the most ephemeral source. Threads get deleted, channels get archived, and search quality degrades over time.
|
|
13
13
|
|
|
14
14
|
## How to search it
|
|
15
15
|
|
|
@@ -38,7 +38,7 @@ Slack MCP tools vary. Check which Slack MCP is available and inspect its tool sc
|
|
|
38
38
|
## Common pitfalls
|
|
39
39
|
|
|
40
40
|
- **Channel archaeology limits.** Very old messages may be gone due to retention policies. If you can't find anything before a certain date, note the retention cliff.
|
|
41
|
-
- **Unsearched DMs.** Many decisions happen in DMs that aren't searchable. You'll miss them
|
|
41
|
+
- **Unsearched DMs.** Many decisions happen in DMs that aren't searchable. You'll miss them. That's a known limitation.
|
|
42
42
|
- **Speculative jokes as "decisions."** Slack is casual. "Lol just do the thing" isn't a decision, even if it preceded the commit. Look for considered discussion.
|
|
43
43
|
- **Context collapse in single messages.** Without the thread, a single message often reads differently than in context. Always fetch threads.
|
|
44
44
|
- **Auth failures.** If the MCP isn't authenticated, stop. Don't make up findings. Report that Slack wasn't searchable.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Synthesizer Prompt Template
|
|
2
2
|
|
|
3
|
-
Build the synthesizer's prompt from this template
|
|
3
|
+
Build the synthesizer's prompt from this template. Fill in the placeholders.
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -41,7 +41,7 @@ You MUST follow the framework in `references/epistemics.md`. Read it in full bef
|
|
|
41
41
|
2. **Reconcile overlapping findings.** Multiple investigators may have cited the same PR, ticket, or doc. Merge into a single, authoritative reference.
|
|
42
42
|
3. **Identify contradictions.** If two items of evidence disagree, don't pick one. Surface both.
|
|
43
43
|
4. **Calibrate confidence.** For each claim, identify the evidence and the tier. State Direct claims plainly with a citation. Hedge Inferred claims and explain the inference. Mark Speculative claims explicitly. Put claims with no evidence in the gaps section.
|
|
44
|
-
5. **Verify citations by spot-checking.** You can read the codebase and call MCP tools to verify citations
|
|
44
|
+
5. **Verify citations by spot-checking.** You can read the codebase and call MCP tools to verify citations. Do not write files, commit, or modify external state. If you're uncertain a cited item exists or says what's claimed, check it. Don't propagate errors.
|
|
45
45
|
6. **Don't overreach.** The user will act on your output. Better to leave an open question open than to fill it with a confident-sounding guess.
|
|
46
46
|
|
|
47
47
|
## Output Format
|
|
@@ -121,7 +121,7 @@ One or two sentences summarizing your overall confidence. E.g.:
|
|
|
121
121
|
Before finalizing, review your output against this checklist:
|
|
122
122
|
|
|
123
123
|
1. Does every claim in "What We Found" have a citation? If not, add one or move the claim to "Inferred" or "Hypotheses."
|
|
124
|
-
2. Is the phrasing tier-appropriate? (Direct claims can use "because"
|
|
124
|
+
2. Is the phrasing tier-appropriate? (Direct claims can use "because". Inferred claims cannot.)
|
|
125
125
|
3. Did you surface any contradictions you noticed, or did you quietly pick one?
|
|
126
126
|
4. Does the "What We Don't Know" section exist and name specific gaps? If it's empty or missing, be suspicious. Historical investigations almost always have gaps.
|
|
127
127
|
5. If the user embedded a hypothesis in their question, did you check it against the evidence rather than rubber-stamping it?
|
|
@@ -1,59 +0,0 @@
|
|
|
1
|
-
# Critic Prompt Template
|
|
2
|
-
|
|
3
|
-
Build each critic subagent's prompt from this template. Fill in the placeholders.
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
You are reviewing the architecture of a codebase subsystem. An explanation of how it works has already been written. Read it to orient yourself, then read the actual code to form your own judgment.
|
|
8
|
-
|
|
9
|
-
## Architectural Explanation
|
|
10
|
-
|
|
11
|
-
{EXPLANATION}
|
|
12
|
-
|
|
13
|
-
## Relevant Files
|
|
14
|
-
|
|
15
|
-
{FILE_PATHS}
|
|
16
|
-
|
|
17
|
-
## Critique Rubric
|
|
18
|
-
|
|
19
|
-
{CRITIQUE_RUBRIC_CONTENTS}
|
|
20
|
-
|
|
21
|
-
## Instructions
|
|
22
|
-
|
|
23
|
-
Read the files listed above. Use the explanation as a map, but form your own opinions from the code itself. The explanation might miss things or frame them charitably.
|
|
24
|
-
|
|
25
|
-
Find architectural problems, not line-level bugs or style issues. Ask whether this subsystem is built well for what it needs to do and how it will need to evolve.
|
|
26
|
-
|
|
27
|
-
For each finding:
|
|
28
|
-
|
|
29
|
-
1. **Severity**: `structural` | `concern` | `observation`
|
|
30
|
-
- `structural`: a fundamental architectural problem. Wrong abstraction boundary, broken data model, coupling that will block future work
|
|
31
|
-
- `concern`: a real issue that makes the system harder to work with or reason about, but not fundamentally broken
|
|
32
|
-
- `observation`: worth noting. A tradeoff that might not age well, a pattern inconsistent with the rest of the codebase, technical debt
|
|
33
|
-
2. **Finding**: the architectural issue. Be specific. Name the components, the boundary, the coupling.
|
|
34
|
-
3. **Evidence**: concrete code that demonstrates the problem. Don't just assert that "this is too coupled". Show the dependency chain.
|
|
35
|
-
4. **Impact**: what the issue costs. Harder to test? Harder to change? Performance cliff at scale? Be concrete about the consequence.
|
|
36
|
-
|
|
37
|
-
## What to Avoid
|
|
38
|
-
|
|
39
|
-
- Line-level code review (not your job here)
|
|
40
|
-
- Suggesting rewrites without demonstrating a problem with the current approach
|
|
41
|
-
- "This could use more abstraction" without showing what the abstraction would actually solve
|
|
42
|
-
- Flagging intentional tradeoffs with clear benefits as issues
|
|
43
|
-
|
|
44
|
-
If the architecture is sound, say so. An empty critique is a valid outcome.
|
|
45
|
-
|
|
46
|
-
## Output
|
|
47
|
-
|
|
48
|
-
```
|
|
49
|
-
## Findings
|
|
50
|
-
|
|
51
|
-
### 1. [Severity] Short title
|
|
52
|
-
**Components**: Which parts of the system are involved
|
|
53
|
-
**Finding**: What's wrong architecturally
|
|
54
|
-
**Evidence**: Concrete code references
|
|
55
|
-
**Impact**: What this costs in practice
|
|
56
|
-
|
|
57
|
-
### 2. [Severity] Short title
|
|
58
|
-
...
|
|
59
|
-
```
|
|
@@ -1,58 +0,0 @@
|
|
|
1
|
-
# Architectural Critique Rubric
|
|
2
|
-
|
|
3
|
-
Review through whichever of these lenses are relevant. Not every lens applies to every subsystem.
|
|
4
|
-
|
|
5
|
-
## Abstraction Fit
|
|
6
|
-
|
|
7
|
-
Are the abstractions pulling their weight?
|
|
8
|
-
|
|
9
|
-
- Does each abstraction represent a real concept, or is it an indirection layer "in case we need it"?
|
|
10
|
-
- Are the boundaries in the right place? Do they separate things that change independently?
|
|
11
|
-
- Is there accidental coupling where components share implementation details they shouldn't need to know about?
|
|
12
|
-
- Is business logic entangled with framework wiring, or cleanly separated?
|
|
13
|
-
|
|
14
|
-
Over-abstraction is as much a problem as under-abstraction. A flat, simple design is fine when the domain is simple.
|
|
15
|
-
|
|
16
|
-
## Data Model
|
|
17
|
-
|
|
18
|
-
Do the data structures fit the actual usage patterns?
|
|
19
|
-
|
|
20
|
-
- Are the data models designed for how data is actually accessed, or for how it was conceptually modeled?
|
|
21
|
-
- Are there impedance mismatches, places where code constantly reshapes data because the model doesn't match the access pattern?
|
|
22
|
-
- Are types honest? Do they represent what data actually looks like at runtime, or claim more structure than exists?
|
|
23
|
-
|
|
24
|
-
## Boundary Discipline
|
|
25
|
-
|
|
26
|
-
Are system boundaries clean and well-placed?
|
|
27
|
-
|
|
28
|
-
- Is validation concentrated at entry points, or scattered through internal code?
|
|
29
|
-
- Are errors handled at boundaries and propagated cleanly, or caught and re-thrown at every layer?
|
|
30
|
-
- Does data cross boundaries in well-typed shapes, or as bags of optional fields?
|
|
31
|
-
- Could this subsystem be tested in isolation, or does it require the entire system to be running?
|
|
32
|
-
|
|
33
|
-
## Evolution Readiness
|
|
34
|
-
|
|
35
|
-
How well will this architecture handle likely changes?
|
|
36
|
-
|
|
37
|
-
- If the most probable next requirement landed tomorrow, how much would change? "One file" or "everything"?
|
|
38
|
-
- Are there hardcoded assumptions that would need to be relaxed?
|
|
39
|
-
- Is the design bolted-on (integrated as an afterthought) or integrated (looks like it was always part of the plan)?
|
|
40
|
-
- Are legacy paths preserved for compatibility that no one depends on?
|
|
41
|
-
|
|
42
|
-
Don't penalize for not handling hypothetical changes. Focus on changes plausible given the codebase's trajectory.
|
|
43
|
-
|
|
44
|
-
## Complexity vs. Value
|
|
45
|
-
|
|
46
|
-
Is the complexity budget spent wisely?
|
|
47
|
-
|
|
48
|
-
- Is complexity concentrated in the parts that need it (core logic, tricky invariants) or in accidental places (boilerplate, unnecessary indirection, configuration)?
|
|
49
|
-
- Are there simpler ways to achieve the same behavior?
|
|
50
|
-
- Does every component earn its existence, or are there vestigial pieces from an earlier design?
|
|
51
|
-
|
|
52
|
-
## Consistency
|
|
53
|
-
|
|
54
|
-
Does this subsystem follow the patterns established elsewhere in the codebase?
|
|
55
|
-
|
|
56
|
-
- Are similar problems solved the same way here as elsewhere, or does this area invent its own patterns?
|
|
57
|
-
- If the patterns differ, is there a good reason, or did it just evolve independently?
|
|
58
|
-
- Inconsistency isn't automatically bad. But unexplained inconsistency is a maintenance burden.
|