@jiroamato/pstack 0.0.0-stage → 0.16.0

package/pstack/skills/why/references/sources/sentry.md

@@ -0,0 +1,100 @@
+# Sentry Error History
+
+## What this source contains
+
+Sentry is the archive of things that went wrong. For defensive, corrective, or error-handling code, it often holds the direct motivation: the specific exceptions, stack traces, and frequencies that pushed someone to add a check, catch, retry, or fallback.
+
+- **Issues.** Grouped errors with counts, first/last seen timestamps, affected releases, and comments
+- **Events.** Individual error instances within an issue (stack traces, tags, user context)
+- **Releases.** Deployment records with associated issues (useful for "which version fixed this?")
+- **Replays.** Session recordings of user-facing errors (if enabled)
+- **Profiles.** Performance profiling data (less useful for "why", more for "how slow")
+- **Issue comments & assignments.** Sometimes contain engineer notes on root cause
+
+The most valuable thing Sentry provides is **temporal correlation**: "issue X was created 2024-01-02, peaked at 500 events/day, stopped appearing after release v2.14.0 on 2024-01-15, the release that shipped the defensive check."
+
+## How to search it
+
+Use the Sentry MCP.
+
+1. **Orient.** If you don't know the project slug and organization:
+
+   ```
+   find_organizations
+   find_projects
+   ```
+
+2. **Search for issues related to the target.**
+
+   ```
+   search_issues (natural language, e.g., "errors in PaymentService timeout", "unhandled exceptions in uploadFile")
+   ```
+
+   Good query components: exception class names the target handles, the function or class name of the target, error message strings the target checks for, the file path of the target.
+
+3. **Narrow by release and time window.**
+
+   ```
+   search_issue_events (filter by release, time, environment, trace ID, tags)
+   get_issue_tag_values (for an issue, see distribution across versions, users, environments)
+   ```
+
+   For a suspected issue, check:
+   - **First seen.** When did the error start appearing?
+   - **Last seen.** When did it stop? Does it line up with the target's ship date?
+   - **Affected releases.** Which versions saw it? Which was the fix?
+   - **Frequency trajectory.** Did it spike, then get resolved?
+
+4. **Pull the full event for context.**
+
+   ```
+   get_sentry_resource (pass a Sentry URL or type+ID)
+   ```
+
+   Does the stack trace pass through the target code? Do the tags and breadcrumbs match the conditions the target defends against?
+
+5. **Check releases that landed near the target.**
+
+   ```
+   find_releases (around the commit date of the target)
+   ```
+
+   Cross-reference release version with the PR's merge date.
+
+6. **Use Seer sparingly.**
+
+   ```
+   analyze_issue_with_seer
+   ```
+
+   Seer produces AI root-cause analyses. Useful as a hypothesis generator, but treat them as inference, not authoritative. The actual events and stack traces are the primary evidence. Seer's narrative is secondary.
+
+## What good evidence looks like here
+
+- An issue whose **first seen** is shortly before the target's PR and **last seen** shortly after, suggesting the target addressed this error
+- Stack traces that pass through or land on the target function, showing the exact failure mode being defended against
+- A comment on the issue from the PR author describing the fix
+- The target's PR description or commit message referencing a Sentry issue URL or ID
+- An issue with high event counts that stops after the release containing the target
+
+## Common pitfalls
+
+- **Grouping drift.** Sentry groups errors by fingerprint. Refactors or renames can track the "same" error under a new issue ID. If an issue ends abruptly, the error may have just been regrouped. Check for new issues immediately after.
+- **Release correlation is noisy.** A release contains many commits. An issue stopping at v2.14.0 doesn't prove the target fixed it. Another change in the same release might have. Cross-reference with the target's exact commit.
+- **Silent fixes.** Sometimes the error stops because upstream changed, not because of the defensive code. The correlation suggests the fix. It doesn't prove authorship.
+- **Resolved != fixed.** Issues can be marked "resolved" manually without any code change. Treat `resolved` as a human marker, not evidence that code fixed it.
+- **Seer hallucinations.** Seer can generate confident-sounding explanations that aren't right. Fall back to the actual events, stack traces, and timestamps when making claims.
+- **Sampling.** Some projects sample events aggressively. A low event count may just mean high sampling, not a rare error. If in doubt, note the gap.
+
+## What to return
+
+For each relevant issue:
+- Issue ID and title
+- Project and organization
+- First seen / last seen timestamps
+- Event count (and sampling rate if known)
+- Affected releases
+- A representative stack trace snippet showing relevance to the target (verbatim excerpt, not summary)
+- First/last-seen correlation with the target's ship date
+- Link to the issue
+- Any author comments or resolution notes

package/pstack/skills/why/references/sources/slack.md

@@ -0,0 +1,54 @@
+# Slack Conversations
+
+## What this source contains
+
+- Real-time discussions of problems and decisions
+- Incident channels where fire-drill decisions were made
+- Design discussion threads where tradeoffs were debated
+- Questions answered by senior engineers that didn't make it into docs
+- Post-merge discussions that explain why something was revisited
+- DMs (usually not searchable, scope accordingly)
+
+Slack is frequently where the *real* decisions got made, especially for smaller changes that didn't warrant a doc. It's also the most ephemeral source. Threads get deleted, channels get archived, and search quality degrades over time.
+
+## How to search it
+
+Slack MCP tools vary. Check which Slack MCP is available and inspect its tool schema first. It may require `mcp_auth`. If authentication fails, stop and report the gap.
+
+1. **Author-bounded search.** Messages from the PR author around the PR merge date. Limits scope dramatically and often hits gold.
+2. **Keyword search for the feature name and key symbols.** Include misspellings and casual phrasings.
+3. **PR URL search.** Slack often links PRs when they're reviewed or discussed. Search for the PR URL (or just `/pull/<number>`).
+4. **Error string search.** If the code handles a specific error, search for the error string. Incident threads often surface.
+5. **Channel-scoped search.** Narrow to likely channels:
+   - `#eng-*`. Engineering discussions
+   - `#proj-*`. Project channels
+   - `#incident-*` / `#sev-*`. Incident channels
+   - Team-specific channels for the owning team
+   - Design review channels
+6. **Thread traversal.** When you find a relevant message, fetch the whole thread. The decision often lives in the replies.
+
+## What good evidence looks like here
+
+- A thread where tradeoffs were explicitly debated ("I was going to use A but B is better because...")
+- An incident channel message describing the bug the code prevents
+- A question from a reviewer and an authoritative answer from the author or lead
+- A reference to a meeting where a decision was made
+- A message from a product manager or customer-facing engineer explaining a customer ask
+
+## Common pitfalls
+
+- **Channel archaeology limits.** Very old messages may be gone due to retention policies. If you can't find anything before a certain date, note the retention cliff.
+- **Unsearched DMs.** Many decisions happen in DMs that aren't searchable. You'll miss them. That's a known limitation.
+- **Speculative jokes as "decisions."** Slack is casual. "Lol just do the thing" isn't a decision, even if it preceded the commit. Look for considered discussion.
+- **Context collapse in single messages.** Without the thread, a single message often reads differently than in context. Always fetch threads.
+- **Auth failures.** If the MCP isn't authenticated, stop. Don't make up findings. Report that Slack wasn't searchable.
+
+## What to return
+
+For each relevant thread:
+- Channel name
+- Permalink or thread ID
+- Participants
+- Date range of the discussion
+- The key quotes (verbatim) with attribution
+- Context: what thread/incident/discussion this was part of

package/pstack/skills/why/references/synthesizer-prompt.md

@@ -0,0 +1,135 @@
+# Synthesizer Prompt Template
+
+Build the synthesizer's prompt from this template. Fill in the placeholders.
+
+---
+
+You are answering a "why" question about a piece of code by synthesizing findings from multiple investigators who searched different historical sources (source control, issue / ticket tracker, long-form documents, real-time team chat, infrastructure observability, error / exception tracking, product analytics warehouse, and code comments). Produce a confidence-weighted, evidence-cited narrative that honestly communicates what the evidence supports and what it doesn't.
+
+## The Question
+
+> {QUESTION}
+
+## The Code Anchor
+
+**Target files:** {FILES_WITH_LINE_RANGES}
+
+**Key symbols:** {SYMBOLS}
+
+## Investigator Findings
+
+{ALL_INVESTIGATOR_FINDINGS}
+
+## Sources That Weren't Searched
+
+{SKIPPED_SOURCES_WITH_REASONS}
+
+## Epistemics Framework
+
+You MUST follow the framework in `references/epistemics.md`. Read it in full before writing the output. The key rules:
+
+1. Every claim sits in one of these tiers: **Direct**, **Supported**, **Inferred**, **Speculative**, **Unknown**. The tier determines what section the claim goes in and how it's phrased.
+2. Every Direct/Supported claim must have a citation (PR #, ticket ID, doc URL, chat permalink, commit hash, or file:line).
+3. Inferred and Speculative claims must use hedged language ("appears to", "likely", "suggests", "one possibility is").
+4. Never cite code as evidence for its own intent.
+5. Gaps in the evidence must be documented. Don't fill them with plausible-sounding guesses.
+6. If the user's question embedded a hypothesis, treat it as a candidate, not a conclusion. Check the evidence independently.
+
+## Instructions
+
+1. **Read all investigator findings.** They gathered raw evidence, not conclusions. You weigh it.
+2. **Reconcile overlapping findings.** Multiple investigators may have cited the same PR, ticket, or doc. Merge into a single, authoritative reference.
+3. **Identify contradictions.** If two items of evidence disagree, don't pick one. Surface both.
+4. **Calibrate confidence.** For each claim, identify the evidence and the tier. State Direct claims plainly with a citation. Hedge Inferred claims and explain the inference. Mark Speculative claims explicitly. Put claims with no evidence in the gaps section.
+5. **Verify citations by spot-checking.** You can read the codebase and call MCP tools to verify citations. Do not write files, commit, or modify external state. If you're uncertain a cited item exists or says what's claimed, check it. Don't propagate errors.
+6. **Don't overreach.** The user will act on your output. Better to leave an open question open than to fill it with a confident-sounding guess.
+
+## Output Format
+
+Write the output for the user. Use this exact structure:
+
+---
+
+### The Question
+
+Restate the user's question in one or two sentences so the answer is anchored.
+
+### The Code in Question
+
+File paths, line ranges, key symbols. Two or three lines to orient a reader who lands here cold.
+
+### What We Found
+
+**Claims with direct evidence**, one per bullet. Quote or paraphrase the source and cite precisely. Format each finding like:
+
+- **[Direct]** {Claim}. Source: [PR #123](url) / ticket ID / file:line. {Brief quote or paraphrase.}
+- **[Supported]** {Claim}. Evidence: {list of items and what each contributes}.
+
+Use `[Direct]` for single-source, explicit evidence. Use `[Supported]` when multiple indirect items converge on a conclusion.
+
+### What We Can Reasonably Infer
+
+**Claims that aren't explicitly stated anywhere but are well-supported by indirect evidence.** Make the inference chain visible: "Given A and B, it's likely that C." Use hedged language ("appears to", "likely", "suggests", "is consistent with"). Format:
+
+- **[Inferred]** {Hedged claim}. Reasoning: {the specific evidence and the inference step}.
+
+If there's nothing to infer, skip this section.
+
+### Competing Hypotheses
+
+**If the evidence fits multiple stories, present them.** Don't force a winner when the record doesn't support one. For each hypothesis:
+
+- **Hypothesis:** {one-sentence statement}
+- **Evidence for:** {specific items}
+- **Evidence against or missing:** {what would need to be true but isn't, or what counter-signals exist}
+
+Skip this section if there's a single clear answer.
+
+### What We Don't Know
+
+**Explicit gaps.** Things the user asked that the evidence didn't answer. Sources searched that came up empty. Sources that weren't searchable at all, such as a missing real-time team chat MCP.
+
+Be specific. "We searched the issue tracker for [query1], [query2], [query3] and found no issue discussing the rate-limit threshold" is useful. "We don't know why" is not. Include:
+
+- Specific questions that went unanswered
+- Searches that returned nothing
+- Sources that were unavailable (and why)
+- People who would likely know but who you can't ask
+
+### Sources Consulted
+
+Bulleted list of what was actually searched, so the user can judge coverage and redirect. Format:
+
+- **Source control history**: {file paths}, {number of commits reviewed}, PRs #{numbers}, and code comments searched. Or "Not searched. This should not happen because git and `gh` are always expected."
+- **Issue / ticket tracker**: {ticket IDs and keyword searches}. Or "Not searched. No matching MCP available in this environment."
+- **Long-form documents**: {page titles and search queries}. Or "Not searched. No matching MCP available in this environment."
+- **Real-time team chat**: {channels searched, date ranges, queries}. Or "Not searched. No matching MCP available in this environment."
+- **Infrastructure observability**: {dashboards, monitors, metrics, logs, traces, or incidents searched}. Or "Not searched. No matching MCP available in this environment."
+- **Error / exception tracking**: {issues, events, or releases searched}. Or "Not searched. No matching MCP available in this environment."
+- **Product analytics warehouse**: {fully-qualified tables queried, the time windows, and the numeric summaries (counts, percentiles, first/last-seen timestamps) that bore on the question}. Or "Not searched. No matching MCP available in this environment."
+
+### Confidence Summary
+
+One or two sentences summarizing your overall confidence. E.g.:
+
+> "The core rationale (A) is well-supported by direct PR and ticket evidence. The specific threshold value (100) is inferred from the surrounding context but not explicitly documented. The question of whether this was driven by a customer request could not be answered. No relevant issue tracker or long-form doc content surfaced, and real-time team chat search was unavailable."
+
+---
+
+## Quality Check Before Returning
+
+Before finalizing, review your output against this checklist:
+
+1. Does every claim in "What We Found" have a citation? If not, add one or move the claim to "Inferred" or "Hypotheses."
+2. Is the phrasing tier-appropriate? (Direct claims can use "because". Inferred claims cannot.)
+3. Did you surface any contradictions you noticed, or did you quietly pick one?
+4. Does the "What We Don't Know" section exist and name specific gaps? If it's empty or missing, be suspicious. Historical investigations almost always have gaps.
+5. If the user embedded a hypothesis in their question, did you check it against the evidence rather than rubber-stamping it?
+6. Did you cite any code as evidence for its own intent? Remove those. Code is mechanics, not motivation.
+7. Is the overall tone calibrated? A confident-sounding answer with weak evidence is the exact failure mode this skill exists to prevent.
+
+If any item fails, revise before returning.
+
+## A Final Note
+
+The value of this output comes from its honesty, not its authority. A reader who takes your answer to the original author, an engineering lead, or a product manager should be well-positioned to ask the right follow-up questions. Be clear about what's known, what's inferred, and what's missing. Don't optimize for looking decisive. Optimize for being useful.