niahere 0.2.52 → 0.2.56

package/skills/customer-research/evals/evals.json

@@ -0,0 +1,162 @@
+{
+  "skill_name": "customer-research",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "I have 20 customer interview transcripts. Help me analyze them.",
+      "expected_output": "Should check for product-marketing-context.md first. Should ask about the goal before analyzing (improve messaging, build personas, find product gaps, etc.). Should apply the extraction framework: jobs to be done, pain points, trigger events, desired outcomes, language/vocabulary, alternatives considered. Should recommend clustering by theme, frequency + intensity scoring, and identifying money quotes. Should ask which deliverable is needed.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Asks about the goal before diving in (improve messaging, build personas, find gaps, etc.)",
+        "Mentions extracting jobs to be done, pain points, and desired outcomes",
+        "Suggests organizing quotes by theme",
+        "References frequency and intensity scoring",
+        "Asks which deliverable is needed"
+      ],
+      "files": []
+    },
+    {
+      "id": 2,
+      "prompt": "I want to do ICP research but I don't have any customer interviews yet.",
+      "expected_output": "Should check for product-marketing-context.md first. Should recommend digital watering hole research as a starting point. Should mention Reddit, G2, Capterra, forums, or niche communities as sources. Should offer to plan a research approach and explain what to extract from online sources. Should note this is Mode 2 and ask what product/category to research.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Recommends digital watering hole research as an alternative",
+        "Mentions Reddit, G2, or review sites as starting points",
+        "Asks what product or category to research",
+        "Offers to help extract insights from online sources"
+      ],
+      "files": []
+    },
+    {
+      "id": 3,
+      "prompt": "Mine Reddit and G2 to understand what people hate about project management software.",
+      "expected_output": "Should check for product-marketing-context.md first. Should identify relevant subreddits (r/projectmanagement, r/productivity, r/agile) and search strategies. Should recommend reading 3-star and 1-star G2 reviews and competitor 4-star reviews. Should plan to extract verbatim quotes, pain themes, and switching triggers. Should apply the extraction table (source, quote, context, sentiment, theme tag, profile signals).",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Identifies relevant subreddits or search strategies for project management",
+        "Suggests reading 3-star and 1-star G2 reviews",
+        "Recommends competitor 4-star reviews for buried complaints",
+        "Plans to extract verbatim quotes and pain themes",
+        "Mentions what to look for: complaints, workarounds, switching triggers"
+      ],
+      "files": []
+    },
+    {
+      "id": 4,
+      "prompt": "Build me a customer persona for a marketing manager at a B2B SaaS company.",
+      "expected_output": "Should check for product-marketing-context.md first. Should ask if there is existing research to build from before generating a persona. Should warn against inventing details without data. Should use the persona structure: profile, primary JTBD, trigger events, top pains, desired outcomes, objections, alternatives, key vocabulary, how to reach them. Should note that personas should be built from at least 5-10 data points.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Asks if there is existing research to build from before inventing details",
+        "Warns against creating personas without data",
+        "Includes jobs to be done, pains, triggers, and desired outcomes in persona structure",
+        "Mentions the need to capture actual customer vocabulary",
+        "Notes minimum data threshold (5-10 data points)"
+      ],
+      "files": []
+    },
+    {
+      "id": 5,
+      "prompt": "I have 6 months of customer support tickets. What insights can I pull from them?",
+      "expected_output": "Should check for product-marketing-context.md first. Should recommend categorizing tickets before analyzing (bugs vs. confusion vs. feature requests vs. expectation mismatches). Should warn against treating all tickets as equal signal. Should suggest extracting recurring language, patterns, and 'I wish it could…' phrases. Should ask about the goal — product improvement, messaging, reducing support load, or something else.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Recommends categorizing tickets before analyzing (bugs vs confusion vs feature requests)",
+        "Warns against treating all tickets as equal signal",
+        "Mentions extracting recurring language and patterns",
+        "Asks about the goal — product improvement, messaging, or something else"
+      ],
+      "files": []
+    },
+    {
+      "id": 6,
+      "prompt": "What are customers saying about my competitors on review sites?",
+      "expected_output": "Should check for product-marketing-context.md first. Should ask which competitors to research. Should recommend G2 and Capterra as primary sources. Should specifically call out reading competitor 4-star reviews for buried complaints. Should describe what to extract: what they love (battlecard intel), what frustrates them (opportunities), unmet needs. Should use the review mining template.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Recommends reading competitor 4-star reviews specifically for buried complaints",
+        "Mentions G2 or Capterra as sources",
+        "Describes what to extract: what they love, what frustrates them, unmet needs",
+        "Frames as competitive intelligence input"
+      ],
+      "files": []
+    },
+    {
+      "id": 7,
+      "prompt": "Help me do voice of customer research for a new SaaS in the HR space.",
+      "expected_output": "Should check for product-marketing-context.md first. Should ask about the specific ICP segment within HR (recruiter, HR generalist, CHRO, etc.). Should suggest relevant digital watering holes: r/humanresources, r/recruiting, HR Slack communities, G2 HR category, LinkedIn. Should plan to extract verbatim language for copy use. Should offer to produce a VOC quote bank as a deliverable.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Asks about target ICP segment within HR",
+        "Suggests relevant digital watering holes (subreddits, G2 categories, communities)",
+        "Plans to extract verbatim language for copy use",
+        "Mentions organizing findings into a VOC quote bank"
+      ],
+      "files": []
+    },
+    {
+      "id": 8,
+      "prompt": "I want to understand why customers churn. I have exit survey results.",
+      "expected_output": "Should check for product-marketing-context.md first. Should recommend segmenting churn reasons before analyzing — do not average across different causes. Should suggest pairing open-ended responses with quantitative data. Should ask if win/loss interview data or support tickets are also available. Should apply confidence labels (high/med/low) based on sample size and source consistency.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Recommends segmenting churn reasons before analyzing",
+        "Warns against averaging across different churn causes",
+        "Suggests pairing open-ended responses with quantitative data",
+        "Asks if win/loss interview data is also available"
+      ],
+      "files": []
+    },
+    {
+      "id": 9,
+      "prompt": "Find the digital watering holes where DevOps engineers talk shop.",
+      "expected_output": "Should check for product-marketing-context.md first. Should identify specific relevant communities: r/devops, r/sysadmin, Hacker News, DevOps-focused Discord/Slack groups, LinkedIn, Stack Overflow. Should suggest what to search for in those communities. Should describe what signal to extract from each source type and reference source-guides.md for detailed playbooks.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Mentions specific relevant communities (r/devops, Hacker News, LinkedIn, Discord)",
+        "Suggests what to search for in those communities",
+        "Describes what signal to extract from each source type"
+      ],
+      "files": []
+    },
+    {
+      "id": 10,
+      "prompt": "Turn my customer research into messaging I can use on my homepage.",
+      "expected_output": "Should check for product-marketing-context.md first. Should extract VOC language and top themes before moving to copy. Should identify the highest-signal quotes and language patterns. Should produce a VOC summary or quote bank, then hand off to the copywriting skill for the actual copy writing step rather than writing homepage copy directly.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Extracts the VOC language and themes first before jumping to copy",
+        "Identifies the highest-signal quotes for messaging",
+        "References the copywriting skill for the actual copy writing step"
+      ],
+      "files": []
+    },
+    {
+      "id": 11,
+      "prompt": "I run a mobile fitness app and want to understand why users drop off after week 2.",
+      "expected_output": "Should check for product-marketing-context.md first. Should recognize this as a B2C research scenario. Should suggest B2C-appropriate sources: app store reviews (1-3 star), Reddit fitness communities, YouTube comment sections on fitness apps, TikTok/Instagram comments. Should also recommend in-app surveys and analyzing support tickets/reviews. Should frame around activation and habit formation research.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Recognizes this as a B2C research scenario",
+        "Suggests app store reviews as a primary source",
+        "Mentions Reddit or community sources relevant to fitness/consumer apps",
+        "Frames around understanding drop-off triggers and desired outcomes"
+      ],
+      "files": []
+    },
+    {
+      "id": 12,
+      "prompt": "I have no existing research and don't know who my best customers are yet.",
+      "expected_output": "Should check for product-marketing-context.md first. Should treat this as a bootstrap research scenario. Should recommend starting with hypothesis formation before gathering data. Should suggest a minimum viable research plan: 5-10 customer interviews + digital watering hole scan. Should provide interview recruiting tips and what questions to ask. Should warn against building personas before collecting any data.",
+      "assertions": [
+        "Checks for product-marketing-context.md",
+        "Recognizes this as a zero-research bootstrap scenario",
+        "Recommends forming hypotheses before gathering data",
+        "Suggests a minimum viable research plan (interviews + online sources)",
+        "Warns against building personas without any data"
+      ],
+      "files": []
+    }
+  ]
+}

package/skills/customer-research/references/source-guides.md

@@ -0,0 +1,345 @@
+# Customer Research — Source Guides
+
+Detailed, source-by-source playbooks for gathering customer intelligence from online watering holes.
+
+---
+
+## Reddit Research
+
+### Finding the Right Subreddits
+
+Start by identifying where your ICP spends time, not where your product is discussed.
+
+**Discovery methods:**
+- Search `site:reddit.com "[job title] tools"` or `site:reddit.com "[problem category] software"`
+- Use [subreddit search tools](https://www.reddit.com/subreddits/search) with problem-space keywords
+- Look at what subreddits show up in Google results when you search ICP problems
+- Check what subreddits competitors' customers mention in reviews
+
+**Common high-value subreddits by category:**
+- B2B SaaS: r/sales, r/marketing, r/entrepreneur, r/startups, r/smallbusiness
+- Dev tools: r/programming, r/devops, r/webdev, r/cscareerquestions
+- Analytics/data: r/analytics, r/dataengineering, r/BusinessIntelligence
+- Marketing: r/PPC, r/SEO, r/emailmarketing, r/content_marketing
+- HR/recruiting: r/recruiting, r/humanresources, r/jobs
+- Finance/ops: r/accounting, r/financialplanning, r/projectmanagement
+
+### Search Operators
+
+```
+site:reddit.com/r/[subreddit] "[keyword]"
+site:reddit.com "[problem]" "recommend" OR "suggestion" OR "alternative"
+site:reddit.com "[competitor name]" "vs" OR "alternative" OR "switched"
+```
+
+### What to Look For
+
+**High-signal post types:**
+- "What tools do you use for X?" → reveals alternatives and vocab
+- "Frustrated with [competitor], looking for alternatives" → reveals pain and switching triggers
+- "How do you handle X?" → reveals workflow and workarounds
+- "Is [your category] worth it?" → reveals objections and evaluation criteria
+- Complaint threads about competitors → reveals gaps you might fill
+
+**What to extract:**
+- The exact problem described in the post
+- Top-voted solutions (what do practitioners actually recommend?)
+- Complaints about existing solutions in comments
+- The language used — note specific words and phrases
+- Upvote patterns — consensus vs. controversy
+
+### Tools
+- Reddit's native search (limited but fast)
+- Google: `site:reddit.com [query]` (better results)
+- Pullpush.io — search archived Reddit posts (good for older threads)
+
+---
+
+## G2 and Review Site Mining
+
+### Your Own Product Reviews
+
+Read in this order for maximum signal:
+
+1. **3-star reviews** — these are the most honest. Customer liked it enough to stay but felt something was missing.
+2. **1-star reviews** — understand the failure modes. Separate product issues from support/onboarding issues.
+3. **5-star reviews** — extract the "what they love" language. These are your proof points.
+4. **4-star reviews** — often contain "the only thing I wish…" buried in praise.
+
+**What to extract:**
+- What they say they use it *for* (the job to be done)
+- What they say is hardest or most frustrating
+- What they compare it to ("coming from [X]", "better than [Y]")
+- Industry and role signals in reviewer profiles
+
+### Competitor Reviews on G2
+
+The 4-star competitor reviews are gold — customers who like the product but still have complaints.
+
+**G2 structure to exploit:**
+- "What do you like best?" → their strengths (your battlecard intel)
+- "What do you dislike?" → their weaknesses (your opportunities)
+- "What problems are you solving?" → the job to be done
+
+**Capterra** has similar structure. **Trustpilot** skews B2C. **AppSumo** reviews are useful for SMB/prosumer SaaS.
+
+### Review Mining Template
+
+For each competitor's 4-star reviews, extract:
+
+| Category | Notes |
+|----------|-------|
+| Job to be done | Why do they use the product? |
+| Top praise | What do they love (and might be hard for you to match)? |
+| Top complaint | What frustrates them? |
+| Switching context | Did they mention switching from something else? |
+| Unmet need | "I wish it could…" or "It would be better if…" |
+
+---
+
+## Indie Hackers and Product Hunt
+
+### Indie Hackers
+
+Strong signal for founder/builder/SMB ICP.
+
+**Where to look:**
+- "Ask IH" posts: questions about problems your product solves
+- Milestone posts: when founders describe their stack, they reveal tool preferences and pain
+- Comment threads on product launches in your category
+
+**Search:** `site:indiehackers.com "[problem]"` or use IH's native search.
+
+### Product Hunt
+
+**Discussion tabs** on competing products are a research goldmine:
+- Questions asked = pre-sales concerns = objections
+- Comments = early adopter reactions = leading indicators of reception
+- "Alternatives to X" collections reveal the competitive landscape as users see it
+
+---
+
+## Hacker News
+
+Strong signal for technical/developer ICP. Skews toward builders and skeptics.
+
+**High-value searches:**
+- `site:news.ycombinator.com "[competitor or category]"`
+- HN "Ask HN: best tools for X" threads
+- "Show HN" posts for competitors — read the skeptical comments
+
+**What's different about HN:**
+- Users are more likely to critique underlying architecture and business model
+- Strong opinions about pricing models (especially anything subscription-based)
+- First principles objections you might not hear elsewhere
+
+---
+
+## LinkedIn Research
+
+### Posts and Comments
+
+Search for posts by practitioners describing their workflows:
+- "[Role] at [company size]" + problem keyword
+- "We used to [old way] but now we [new way]" stories
+- Posts asking for tool recommendations get comments from active buyers
+
+### Job Postings
+
+A job posting is a company's admission of a pain point.
+
+**What to look for:**
+- What tools are listed as "nice to have" vs. "required"? (reveals stack and adjacent tools)
+- What metrics and outcomes are mentioned in the role description?
+- What does the role spend most of its time doing? (reveals the job to be done)
+
+**Search:** `site:linkedin.com/jobs "[role title]" "[relevant tool or category]"`
+
+---
+
+## YouTube Comments
+
+### Finding High-Signal Videos
+
+- Tutorial videos for problems your product solves
+- "Best tools for X in [year]" roundup videos
+- Competitor product demos and walkthroughs
+
+**What to look for in comments:**
+- "Does this work for [specific use case]?" → edge cases and unmet needs
+- "I tried this but…" → failure points
+- "What about [competitor]?" → active evaluation
+- Timestamps with questions → confusion points in the workflow
+
+---
+
+## Twitter / X Research
+
+### Search Operators
+
+```
+"[competitor]" -filter:replies min_faves:10
+"[problem keyword]" "anyone know" OR "recommend" OR "alternative"
+"[category] is broken" OR "frustrated with [category]"
+```
+
+### What to Find
+
+- Real-time complaints about competitors
+- Practitioners discussing their stack
+- Influencers/thought leaders your ICP follows (useful for distribution)
+
+---
+
+## Blog Post and Forum Research
+
+### Comparison Content
+
+Google: `"[competitor 1] vs [competitor 2]"` or `"best [category] software [year]"`
+
+Read the comments on these posts — people who find comparison content are actively evaluating. Their comments are questions your sales process should answer.
+
+### Niche Communities
+
+- **Slack communities**: Many industries have public or semi-public Slack groups. Search "[industry] Slack community".
+- **Discord servers**: Growing for developer and creator communities.
+- **Facebook Groups**: Still strong for SMB, e-commerce, agency, and coach/consultant ICP.
+- **Circle/Mighty Networks communities**: Check if there are paid communities in your ICP's space.
+
+---
+
+## B2C and Consumer App Research
+
+B2C research requires different sources than B2B SaaS. Consumer buyers don't congregate on LinkedIn or G2 — they leave traces in app stores, social media, and communities built around the activity your product serves.
+
+### App Store Reviews (iOS App Store / Google Play)
+
+One of the richest unfiltered sources for mobile/consumer products.
+
+**Read in this order:**
+1. **1-2 star reviews** — failure modes, unmet expectations, frustration peaks
+2. **3-star reviews** — honest tradeoffs and "it's good but…" feedback
+3. **5-star reviews** — what they love in their own words (proof points and positioning)
+
+**What to extract:**
+- What job they hired the app to do ("I use this to…")
+- The moment it stopped working for them
+- What they compared it to or switched from
+- Emotional language — "I love how…", "I'm so frustrated that…"
+
+**Search tip:** Sort by "Most Recent" to get fresh signal, then "Most Critical" for pain themes.
+
+### Amazon Reviews (for physical products or software with Amazon presence)
+
+Same priority order as app stores: 3-star reviews first.
+
+**G2 analog for consumer SaaS**: Trustpilot, Sitejabber, and product-specific review aggregators.
+
+### Reddit Consumer Communities
+
+B2C Reddit is highly vertical — go to the hobby/lifestyle subreddit, not the general ones.
+
+**Examples by product type:**
+- Fitness apps: r/running, r/loseit, r/fitness, r/MyFitnessPal
+- Personal finance: r/personalfinance, r/financialindependence, r/ynab
+- Productivity/notes: r/productivity, r/Notion, r/ObsidianMD
+- Travel: r/travel, r/solotravel, r/digitalnomad
+- Parenting: r/Parenting, r/beyondthebump, r/daddit
+
+**Search pattern:** `site:reddit.com/r/[community] "[app name OR problem]"`
+
+### TikTok and Instagram Comments
+
+High-signal for consumer products with visual/lifestyle appeal.
+
+**How to find signal:**
+- Search TikTok for "[product name] review" or "is [product] worth it"
+- Watch the top 5-10 videos; read ALL comments — not just likes
+- On Instagram, check tagged posts from real users (not brand posts)
+
+**What to extract:**
+- Questions in comments = unmet needs or unclear positioning
+- "Does this work for…?" = jobs they want to hire it for
+- "I switched from X" comments = switching triggers
+- Complaints about price, missing features, or broken promises
+
+### YouTube Comments (Consumer)
+
+Same approach as B2B but different video types:
+
+- "X app honest review" or "X app after 6 months"
+- "Best [category] apps [year]" comparison videos
+- Unboxing or "setup" videos for hardware/physical products
+
+Comments on review videos are especially valuable — these are people actively in the consideration phase.
+
+### Consumer Community Platforms
+
+- **Facebook Groups**: Still dominant for many consumer verticals (parenting, fitness, local services, hobbies)
+- **Discord servers**: Growing for gaming, creator tools, productivity, crypto, lifestyle communities
+- **Nextdoor**: Useful for local service businesses
+- **Quora**: Long-form questions reveal decision anxiety and evaluation criteria
+
+---
+
+## Organizing Your Research
+
+Use a simple tagging system across all sources:
+
+| Tag | Meaning |
+|-----|---------|
+| `#pain` | A problem or frustration |
+| `#trigger` | An event that prompted the search |
+| `#outcome` | What success looks like |
+| `#language` | Exact phrases worth using in copy |
+| `#alternative` | Another solution they considered or use |
+| `#objection` | Reason to hesitate or not buy |
+| `#competitor` | Anything about a competing product |
+
+Keep a running doc with columns: Source | Date | Quote | Tags | Notes
+
+After 20-30 entries, patterns will emerge. Look for quotes that appear in multiple unrelated sources — those are your highest-confidence insights.
+
+---
+
+## Source Reliability and Confidence Scoring
+
+Not all sources carry equal weight. Use this guide when assigning confidence labels.
+
+### Source Weighting
+
+| Source | Signal Strength | Bias to Note |
+|--------|----------------|--------------|
+| Customer interviews (unprompted) | Very high | Small sample; selection bias toward engaged customers |
+| Win/loss interviews | High | Recent memory only; rationalization common |
+| App store / G2 reviews | High | Skews toward strong opinions (love or hate) |
+| Reddit / community posts | Medium-high | Skews technical, skeptical, vocal minorities |
+| Support tickets | Medium | Skews toward problems; silent majority not represented |
+| Survey (open-ended) | Medium | Primed by question framing |
+| Survey (multiple choice) | Low-medium | Artifacts of the options you provided |
+| NPS verbatims | Medium | Correlates with score; prompted by the survey moment |
+| YouTube/TikTok comments | Medium | Skews toward engaged viewers; social performance |
+| Job postings | Low-medium | Aspirational, not necessarily reflective of current pain |
+
+### Confidence Labels in Practice
+
+When presenting insights, lead with confidence:
+
+```
+[HIGH CONFIDENCE] Customers feel overwhelmed by manual reporting — appears in 12 of 20 interviews,
+4 Reddit threads, and is the #1 complaint in 3-star G2 reviews. Consistent across SMB and mid-market.
+
+[MEDIUM CONFIDENCE] Customers compare us to spreadsheets more than to direct competitors —
+mentioned in 6 interviews and 3 Reddit threads, but not yet seen in review data.
+
+[LOW CONFIDENCE] Enterprise buyers may have procurement concerns — mentioned by 2 interviewees
+from companies 500+. Needs more signal before acting on it.
+```
+
+### Recency Window
+
+- **Use as primary source**: Data from the last 12 months
+- **Use with caution**: 12-24 months (product and market may have shifted)
+- **Use only for baseline context**: 2+ years old
+
+When a theme appears consistently across old and new data, that's a durable signal worth acting on.