novahiz 0.1.0

package/bundled-skills/customer-research/evals/evals.json

@@ -0,0 +1,176 @@
+{
+  "skill_name": "customer-research",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "I have 20 customer interview transcripts. Help me analyze them.",
+      "expected_output": "Should check for product-marketing.md first. Should ask about the goal before analyzing (improve messaging, build personas, find product gaps, etc.). Should apply the extraction framework: jobs to be done, pain points, trigger events, desired outcomes, language/vocabulary, alternatives considered. Should recommend clustering by theme, frequency + intensity scoring, and identifying money quotes. Should ask which deliverable is needed.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Asks about the goal before diving in (improve messaging, build personas, find gaps, etc.)",
+        "Mentions extracting jobs to be done, pain points, and desired outcomes",
+        "Suggests organizing quotes by theme",
+        "References frequency and intensity scoring",
+        "Asks which deliverable is needed"
+      ],
+      "files": []
+    },
+    {
+      "id": 2,
+      "prompt": "I want to do ICP research but I don't have any customer interviews yet.",
+      "expected_output": "Should check for product-marketing.md first. Should recommend digital watering hole research as a starting point. Should mention Reddit, G2, Capterra, forums, or niche communities as sources. Should offer to plan a research approach and explain what to extract from online sources. Should note this is Mode 2 and ask what product/category to research.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Recommends digital watering hole research as an alternative",
+        "Mentions Reddit, G2, or review sites as starting points",
+        "Asks what product or category to research",
+        "Offers to help extract insights from online sources"
+      ],
+      "files": []
+    },
+    {
+      "id": 3,
+      "prompt": "Mine Reddit and G2 to understand what people hate about project management software.",
+      "expected_output": "Should check for product-marketing.md first. Should identify relevant subreddits (r/projectmanagement, r/productivity, r/agile) and search strategies. Should recommend reading 3-star and 1-star G2 reviews and competitor 4-star reviews. Should plan to extract verbatim quotes, pain themes, and switching triggers. Should apply the extraction table (source, quote, context, sentiment, theme tag, profile signals).",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Identifies relevant subreddits or search strategies for project management",
+        "Suggests reading 3-star and 1-star G2 reviews",
+        "Recommends competitor 4-star reviews for buried complaints",
+        "Plans to extract verbatim quotes and pain themes",
+        "Mentions what to look for: complaints, workarounds, switching triggers"
+      ],
+      "files": []
+    },
+    {
+      "id": 4,
+      "prompt": "Build me a customer persona for a marketing manager at a B2B SaaS company.",
+      "expected_output": "Should check for product-marketing.md first. Should ask if there is existing research to build from before generating a persona. Should warn against inventing details without data. Should use the persona structure: profile, primary JTBD, trigger events, top pains, desired outcomes, objections, alternatives, key vocabulary, how to reach them. Should note that personas should be built from at least 5-10 data points.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Asks if there is existing research to build from before inventing details",
+        "Warns against creating personas without data",
+        "Includes jobs to be done, pains, triggers, and desired outcomes in persona structure",
+        "Mentions the need to capture actual customer vocabulary",
+        "Notes minimum data threshold (5-10 data points)"
+      ],
+      "files": []
+    },
+    {
+      "id": 5,
+      "prompt": "I have 6 months of customer support tickets. What insights can I pull from them?",
+      "expected_output": "Should check for product-marketing.md first. Should recommend categorizing tickets before analyzing (bugs vs. confusion vs. feature requests vs. expectation mismatches). Should warn against treating all tickets as equal signal. Should suggest extracting recurring language, patterns, and 'I wish it could…' phrases. Should ask about the goal — product improvement, messaging, reducing support load, or something else.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Recommends categorizing tickets before analyzing (bugs vs confusion vs feature requests)",
+        "Warns against treating all tickets as equal signal",
+        "Mentions extracting recurring language and patterns",
+        "Asks about the goal — product improvement, messaging, or something else"
+      ],
+      "files": []
+    },
+    {
+      "id": 6,
+      "prompt": "What are customers saying about my competitors on review sites?",
+      "expected_output": "Should check for product-marketing.md first. Should ask which competitors to research. Should recommend G2 and Capterra as primary sources. Should specifically call out reading competitor 4-star reviews for buried complaints. Should describe what to extract: what they love (battlecard intel), what frustrates them (opportunities), unmet needs. Should use the review mining template.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Recommends reading competitor 4-star reviews specifically for buried complaints",
+        "Mentions G2 or Capterra as sources",
+        "Describes what to extract: what they love, what frustrates them, unmet needs",
+        "Frames as competitive intelligence input"
+      ],
+      "files": []
+    },
+    {
+      "id": 7,
+      "prompt": "Help me do voice of customer research for a new SaaS in the HR space.",
+      "expected_output": "Should check for product-marketing.md first. Should ask about the specific ICP segment within HR (recruiter, HR generalist, CHRO, etc.). Should suggest relevant digital watering holes: r/humanresources, r/recruiting, HR Slack communities, G2 HR category, LinkedIn. Should plan to extract verbatim language for copy use. Should offer to produce a VOC quote bank as a deliverable.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Asks about target ICP segment within HR",
+        "Suggests relevant digital watering holes (subreddits, G2 categories, communities)",
+        "Plans to extract verbatim language for copy use",
+        "Mentions organizing findings into a VOC quote bank"
+      ],
+      "files": []
+    },
+    {
+      "id": 8,
+      "prompt": "I want to understand why customers churn. I have exit survey results.",
+      "expected_output": "Should check for product-marketing.md first. Should recommend segmenting churn reasons before analyzing — do not average across different causes. Should suggest pairing open-ended responses with quantitative data. Should ask if win/loss interview data or support tickets are also available. Should apply confidence labels (high/med/low) based on sample size and source consistency.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Recommends segmenting churn reasons before analyzing",
+        "Warns against averaging across different churn causes",
+        "Suggests pairing open-ended responses with quantitative data",
+        "Asks if win/loss interview data is also available"
+      ],
+      "files": []
+    },
+    {
+      "id": 9,
+      "prompt": "Find the digital watering holes where DevOps engineers talk shop.",
+      "expected_output": "Should check for product-marketing.md first. Should identify specific relevant communities: r/devops, r/sysadmin, Hacker News, DevOps-focused Discord/Slack groups, LinkedIn, Stack Overflow. Should suggest what to search for in those communities. Should describe what signal to extract from each source type and reference source-guides.md for detailed playbooks.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Mentions specific relevant communities (r/devops, Hacker News, LinkedIn, Discord)",
+        "Suggests what to search for in those communities",
+        "Describes what signal to extract from each source type"
+      ],
+      "files": []
+    },
+    {
+      "id": 10,
+      "prompt": "Turn my customer research into messaging I can use on my homepage.",
+      "expected_output": "Should check for product-marketing.md first. Should extract VOC language and top themes before moving to copy. Should identify the highest-signal quotes and language patterns. Should produce a VOC summary or quote bank, then hand off to the copywriting skill for the actual copy writing step rather than writing homepage copy directly.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Extracts the VOC language and themes first before jumping to copy",
+        "Identifies the highest-signal quotes for messaging",
+        "References the copywriting skill for the actual copy writing step"
+      ],
+      "files": []
+    },
+    {
+      "id": 11,
+      "prompt": "I run a mobile fitness app and want to understand why users drop off after week 2.",
+      "expected_output": "Should check for product-marketing.md first. Should recognize this as a B2C research scenario. Should suggest B2C-appropriate sources: app store reviews (1-3 star), Reddit fitness communities, YouTube comment sections on fitness apps, TikTok/Instagram comments. Should also recommend in-app surveys and analyzing support tickets/reviews. Should frame around activation and habit formation research.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Recognizes this as a B2C research scenario",
+        "Suggests app store reviews as a primary source",
+        "Mentions Reddit or community sources relevant to fitness/consumer apps",
+        "Frames around understanding drop-off triggers and desired outcomes"
+      ],
+      "files": []
+    },
+    {
+      "id": 12,
+      "prompt": "I have no existing research and don't know who my best customers are yet.",
+      "expected_output": "Should check for product-marketing.md first. Should treat this as a bootstrap research scenario. Should recommend starting with hypothesis formation before gathering data. Should suggest a minimum viable research plan: 5-10 customer interviews + digital watering hole scan. Should provide interview recruiting tips and what questions to ask. Should warn against building personas before collecting any data.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Recognizes this as a zero-research bootstrap scenario",
+        "Recommends forming hypotheses before gathering data",
+        "Suggests a minimum viable research plan (interviews + online sources)",
+        "Warns against building personas without any data"
+      ],
+      "files": []
+    },
+    {
+      "id": 13,
+      "prompt": "I want to interview and survey my customers to understand product/market fit. How should I run this?",
+      "expected_output": "Should check for product-marketing.md first. Should route to the primary-research playbook (references/interviews-and-surveys.md). Should recommend the Sean Ellis / Superhuman PMF survey question ('How would you feel if you could no longer use [product]?') and cite the 40% 'very disappointed' benchmark (Superhuman reached 58%). Should recommend recruiting best customers (high deal size, short sales cycle, low churn) and closing every call with 'who else should we talk to?'. Should mention incentives ($50/call, $5/survey; aim for 10 calls, be happy with 5). Should keep it casual ('you do not talk about customer research') and aim to prove yourself wrong, not right. Should mention 5-why laddering (Keep Asking Why) and provide an outreach email approach.",
+      "assertions": [
+        "Checks for product-marketing.md",
+        "Recommends the PMF survey question and cites the 40% 'very disappointed' benchmark (Superhuman 58%)",
+        "Recommends recruiting best customers by deal size / short sales cycle / low churn and asking 'who else should we talk to?'",
+        "Mentions incentives ($50/call, $5/survey) and aiming for 10 calls / happy with 5",
+        "Frames research as casual and disconfirming (prove yourself wrong, not right)",
+        "Mentions 5-why laddering (Keep Asking Why) or an outreach email template"
+      ],
+      "files": []
+    }
+  ]
+}

package/bundled-skills/customer-research/references/interviews-and-surveys.md

@@ -0,0 +1,155 @@
+# Customer Research — Interviews & Surveys (Primary Research)
+
+Going to the source. Mode 2 mines what customers already said in public; this is Mode 3 — you *ask*. Customer research is your marketing cheat code, and the highest-signal version is talking to customers directly.
+
+Three primary-research pillars, best used together:
+1. **Video calls** — deep, unstructured, follow-the-thread (this file)
+2. **Surveys** — broad, quantified, benchmarkable (this file)
+3. **Online sleuthing** — Sales Safari and watering-hole mining (see `references/source-guides.md`)
+
+---
+
+## The First Rule of Customer Research
+
+> The first rule of customer research: you do not talk about customer research.
+
+Keep it casual. The moment a customer thinks they're in "a research study" they perform — they give you the polished, socially-acceptable answer instead of the real one. Frame calls as a chat, not an interview. Don't lead. Don't pitch. Don't defend the product. You're there to listen and learn how they actually think, talk, and decide.
+
+**Prove yourself wrong, not right.** The point of research is not validation — it's disconfirmation. Go in trying to *break* your assumptions, not confirm them. If you only look for evidence you're right, you'll find it, and it'll be worthless.
+
+- **Dropbox example**: the team assumed users would care most about sync *speed*. Research aimed at disproving the assumption revealed users cared more that files were *reliably there and safe* than about raw speed. Chasing the confirmation would have optimized the wrong thing.
+- Ask questions that could return an answer you don't want to hear. If none of your questions can prove you wrong, rewrite them.
+
+---
+
+## Sales Safari (Amy Hoy)
+
+Amy Hoy's **Sales Safari**: go where your audience already congregates and observe them in the wild, without interrupting. It's structured online sleuthing — read threads, reviews, comments, and forum posts to mine four things:
+
+| Mine for | What you're capturing |
+|----------|-----------------------|
+| **Pains** | The problems, frustrations, and workarounds they describe unprompted |
+| **Jargon** | The exact words, phrases, and shorthand they use — copy gold |
+| **Recommendations** | What they tell each other to buy, try, or avoid |
+| **Worldview** | Their beliefs, biases, and how they see themselves and the problem |
+
+Safari is passive (you observe) where interviews are active (you ask). Run it first: it tells you what to ask about, and in whose words. For per-platform search operators and extraction tips, see `references/source-guides.md`.
+
+---
+
+## Customer Interviews (Video Calls)
+
+### Recruit your best customers
+
+Don't interview whoever answers first. Interview the customers you want *more of*. Segment your CRM and prioritize by:
+
+- **High deal size** — the accounts worth the most
+- **Short sales cycle** — they "got it" fast; their language converts fast
+- **Low churn / high retention** — they got real, lasting value
+
+Recruitment methods, in order of leverage:
+1. **Segment the CRM** by the three signals above and pull a shortlist
+2. **Ask sales and CS for referrals** — they know who loves the product and who articulates why
+3. **Always close every call with**: *"Who else should we talk to?"* — the single most reliable way to compound your interview pipeline
+
+### Incentives
+
+- **$50 per call** (~30 min); **$5 per survey response**
+- Aim for **10 calls, be happy with 5.** Signal saturates fast — by call 5-6 you'll hear the same themes repeat. Don't stall the project waiting for a perfect sample.
+- Offer the incentive up front; it dramatically lifts response rate and shows you value their time. Gift cards work fine.
+
+### Outreach email template
+
+Keep it short, casual, specific, and low-commitment. Not a "research study."
+
+```
+Subject: Quick favor — 30 min, on us
+
+Hi [First name],
+
+I'm [name] from [company]. I'm trying to get better at helping customers
+like you, and I'd love to steal 30 minutes to hear how [product area] is
+actually working for you — what's good, what's annoying, what you wish
+were different. No pitch, no agenda.
+
+As a thank you I'll send you a $50 [Amazon/Visa] gift card.
+
+Are you free [day] or [day] this week? Here's my calendar: [link]
+
+Thanks either way,
+[Name]
+```
+
+Notes:
+- "No pitch, no agenda" and "what's annoying" signal you actually want the truth.
+- One clear ask, two concrete time options, a booking link. Remove friction.
+- Never say "customer research study."
+
+---
+
+## Keep Asking Why (5-Why Laddering)
+
+The first answer is never the real answer. **Keep Asking Why** — ladder each response down 3-5 levels until you hit the root motivation, the business outcome, or the emotional driver. Surface answers are features; the bottom of the ladder is why they pay and why they stay.
+
+**Worked example** — laddering a churn signal to NRR:
+
+- **Q: Why did you downgrade your plan last quarter?**
+  - "We weren't using the advanced reports."
+- **Why weren't you using them?**
+  - "Nobody on the team knew how to build one."
+- **Why didn't anyone learn?**
+  - "The person who set us up left, and onboarding never got re-run for the new hires."
+- **Why did that matter enough to downgrade?**
+  - "Without the reports, my boss couldn't see the ROI, so at renewal it looked like an easy cost to cut."
+- **Why is that the real risk?**
+  - "If leadership can't see value, we churn — and if we *had* seen it, we'd probably have added seats, not cut them."
+
+The surface answer was "we don't use reports." The root is an **onboarding gap that quietly converts an expansion (NRR up) into a contraction or churn (NRR down)**. You can't fix "they don't use reports." You can fix re-onboarding new hires and surfacing ROI to the buyer — which is the difference between contraction and net revenue retention.
+
+**Pain points vs. passion points.** Ladder for both. Pain points are what's broken and what they'll pay to escape. Passion points are what they love, brag about, and would be "very disappointed" to lose. Passion points drive retention and referrals; pains drive acquisition. Capture both in their words.
+
+---
+
+## Surveys
+
+### The PMF Survey (Sean Ellis / Superhuman)
+
+The single most useful survey question, from Sean Ellis and popularized by Superhuman's Rahul Vohra:
+
+> **"How would you feel if you could no longer use [product]?"**
+> - Very disappointed
+> - Somewhat disappointed
+> - Not disappointed
+> - N/A — I no longer use it
+
+**The 40% benchmark**: if **40% or more** of users answer **"very disappointed,"** you likely have product/market fit. Below 40%, keep iterating. **Superhuman reached 58%** by engineering their roadmap around this metric — segmenting on the "very disappointed" cohort, doubling down on what that cohort loved, and converting the "somewhat disappointed" fence-sitters.
+
+Run it as a recurring pulse, not once. Follow the core question with:
+- *"What type of person do you think would most benefit from [product]?"* (sharpens ICP)
+- *"What is the main benefit you receive from [product]?"* (your positioning, in their words)
+- *"How can we improve [product] for you?"* (roadmap fuel from fence-sitters)
+
+Segment every answer by the "very disappointed" cohort vs. the rest — that cohort is your true market.
+
+### Survey design guardrails
+
+- Keep it short — every extra question drops completion.
+- Prefer open-ended for language mining; multiple-choice answers are artifacts of the options you gave.
+- Don't lead. A question that telegraphs the answer you want returns the answer you want, not the truth.
+- $5/response incentive lifts completion; deliver it on submit.
+
+---
+
+## Case Anchors
+
+- **Airbnb (host photography)**: research revealed listings failed because the *photos* were bad, not the pricing or copy. Airbnb sent photographers to shoot host homes — a fix nobody would have guessed without talking to the market. Research points at problems you can't see from inside.
+- **Dropbox (confirmation bias)**: assumed sync speed mattered most; disconfirming research showed reliability/safety of files mattered more. Prove yourself wrong.
+- **Superhuman (PMF survey)**: engineered the roadmap around the "very disappointed" metric, 40% → 58%.
+
+---
+
+## Where This Fits
+
+- **Analyze what you gather** with the Mode 1 extraction framework in `SKILL.md` (jobs to be done, pains, triggers, outcomes, language, alternatives) and the confidence guardrails.
+- **Mine public sources** (the passive Safari half) via `references/source-guides.md`.
+- Interview + survey signal is **first-party and high-confidence** — weight it above scraped online sources when they conflict.