codymaster 4.1.4 → 4.4.1

package/skills/mom-test/references/question-patterns.md

@@ -0,0 +1,198 @@
+# Question Patterns: The Three Rules in Depth
+
+Mastering The Mom Test comes down to asking better questions. This reference covers the three core rules, provides extensive examples of good vs bad questions, and includes exercises for building the skill.
+
+## The Three Rules
+
+### Rule 1: Talk About Their Life, Not Your Idea
+
+The moment you mention your idea, the conversation shifts from honest sharing to polite performance. People don't want to hurt your feelings, so they'll nod and smile no matter what you describe. The fix: keep the entire conversation about them.
+
+**What this looks like in practice:**
+
+| Bad (About Your Idea) | Good (About Their Life) |
+|------------------------|------------------------|
+| "I'm building an app that helps people track expenses. Would you use it?" | "How do you currently keep track of your expenses?" |
+| "We're thinking of adding a social feature. What do you think?" | "When you find a good deal, what do you do next?" |
+| "Our tool uses AI to schedule meetings. Interested?" | "Walk me through how you scheduled your last three meetings." |
+| "Would a dashboard that shows all your metrics be helpful?" | "How do you currently check how your business is doing?" |
+| "We have this new feature that automates reporting. Does that sound useful?" | "Tell me about the last report you had to create. What was involved?" |
+
+**Key principle:** If you can replace your product name with any competitor's name and the question still works, it's a bad question. Good questions are product-agnostic.
+
+### Rule 2: Ask About Specifics in the Past
+
+Generics ("I usually...") and hypotheticals ("I would...") are unreliable. People reconstruct memories to fit narratives and predict futures based on ideal-self behavior, not actual behavior. Specific past events are facts -- they happened, and the details reveal real patterns.
+
+**What this looks like in practice:**
+
+| Bad (Generic/Hypothetical) | Good (Specific Past Event) |
+|----------------------------|---------------------------|
+| "How often do you exercise?" | "When was the last time you went to the gym? What happened before that?" |
+| "Would you pay for a premium version?" | "What's the most you've paid for a tool like this? What made it worth it?" |
+| "Do you have trouble finding restaurants?" | "Tell me about the last time you tried to find a restaurant. What did you do?" |
+| "How do you usually handle complaints?" | "Walk me through the last customer complaint you dealt with." |
+| "Do you care about data privacy?" | "Have you ever switched away from a tool because of a privacy concern? What happened?" |
+
+**Anchor phrases that force specificity:**
+- "Tell me about the last time..."
+- "Walk me through..."
+- "Can you give me a specific example of..."
+- "When did that last happen?"
+- "What exactly did you do?"
+
+### Rule 3: Talk Less, Listen More
+
+If you're talking more than 20% of the time, you're doing it wrong. Every minute you spend talking is a minute you're not learning. Silence is a powerful tool -- when you stop talking, people fill the gap with details they wouldn't have volunteered.
+
+**Practical techniques for talking less:**
+- Ask the question, then wait. Count to ten silently if you must.
+- When they finish an answer, don't immediately respond. Wait 3-5 seconds. They'll often add the most valuable insight as an afterthought.
+- Use minimal encouragers: "Mm-hmm," "Go on," "Tell me more about that"
+- Avoid the urge to relate with your own experience ("Oh yeah, I do that too!")
+- Never finish their sentences, even if you think you know what they'll say
+- If you catch yourself about to pitch, bite your tongue and ask another question
+
+## The Question Hierarchy
+
+Not all good questions are equal. Here's a hierarchy from most to least informative:
+
+### Tier 1: Behavior Questions (Most Reliable)
+These reveal what people actually do. Behavior doesn't lie.
+
+- "Walk me through what you did the last time this came up."
+- "What did you try first? And after that?"
+- "How much time did you spend on it?"
+- "What did you end up doing?"
+- "Did you pay for anything? How much?"
+
+### Tier 2: Problem Questions (Very Reliable)
+These reveal pain points tied to real events.
+
+- "What's the hardest part about [doing this thing]?"
+- "Why was that hard?"
+- "What didn't work about the solutions you tried?"
+- "When you couldn't solve it, what happened? What was the consequence?"
+- "How often does this actually come up?"
+
+### Tier 3: Motivation Questions (Reliable with Care)
+These reveal why people do what they do, but require anchoring to specific events.
+
+- "Why did you choose that particular tool?"
+- "What made you switch from [old thing] to [new thing]?"
+- "What were you hoping would happen?"
+- "What finally pushed you to look for a solution?"
+
+### Tier 4: Worldview Questions (Handle with Care)
+These reveal beliefs and values, but are easily influenced by how you ask.
+
+- "How do you think about [problem area] in general?"
+- "What matters most to you when choosing a tool for this?"
+- "What would have to be true for you to change how you do this?"
+
+### Tier 5: Hypothetical Questions (Least Reliable -- Avoid)
+These predict future behavior and are almost always wrong.
+
+- "Would you use..." -- AVOID
+- "Would you pay..." -- AVOID
+- "How much would you pay..." -- AVOID
+- "Do you think you'd..." -- AVOID
+
+## Domain-Specific Question Banks
+
+### For SaaS Products
+
+**Understanding current workflow:**
+- "Walk me through how your team handles [process] from start to finish."
+- "What tools are involved? How do they connect?"
+- "Where does information get lost or duplicated?"
+- "How long does the whole process take? Where are the bottlenecks?"
+
+**Understanding buying behavior:**
+- "Who approved the budget for your current tool?"
+- "How long did it take from 'we need something' to 'we're paying for it'?"
+- "What would make you switch away from what you're using now?"
+- "Have you tried switching tools before? What happened?"
+
+### For Consumer Apps
+
+**Understanding behavior patterns:**
+- "What's the first app you open in the morning? Why?"
+- "When you're bored on the train, what do you do?"
+- "The last time you had [problem], what did you pull out your phone to do?"
+- "What apps have you deleted recently? Why?"
+
+**Understanding willingness to pay:**
+- "What subscription apps do you currently pay for?"
+- "What made those worth paying for?"
+- "Have you ever started paying for an app you previously used for free? What changed?"
+
+### For Marketplaces
+
+**Supply side:**
+- "How do you currently find customers/clients?"
+- "What's the most effective channel? Why?"
+- "How much do you spend on acquiring a new customer?"
+- "What's the most frustrating part of finding new business?"
+
+**Demand side:**
+- "How did you find your current [service provider]?"
+- "What made you choose them over others?"
+- "What would make you switch to someone new?"
+- "When was the last time you needed this and couldn't find it?"
+
+## Question Formulation Exercises
+
+### Exercise 1: The Idea Purge
+Take your product idea and write down 10 questions you want to ask potential customers. Now go through each one and remove any mention of your product, solution, or idea. Reframe each question to be about their life and past behavior.
+
+**Example:**
+- Before: "Would you use an AI tool that writes emails for you?"
+- After: "Walk me through how you handled your email this morning. How long did it take? What was annoying about it?"
+
+### Exercise 2: The Specificity Drill
+Take any generic question and add specificity anchors until it references a concrete past event.
+
+**Example progression:**
+1. "Do you have trouble managing your finances?" (generic -- bad)
+2. "How do you manage your finances?" (better, but still generic)
+3. "Walk me through how you paid your bills last month." (specific past event -- good)
+4. "Last month, was there a bill that surprised you? What happened?" (specific + emotional anchor -- great)
+
+### Exercise 3: The Deflection Practice
+Have a partner pitch you ideas and practice deflecting to useful questions.
+
+**Partner says:** "I'm building an app that reminds you to drink water."
+**You respond:** "Interesting. Do you personally have trouble staying hydrated? When was the last time you felt dehydrated? What happened? Have you tried any solutions? What worked and what didn't?"
+
+### Exercise 4: The Scary Question List
+Write down the five questions you're most afraid to ask because the answers might invalidate your idea. These are exactly the questions you must ask first.
+
+**Examples of scary questions:**
+- "Have you actually tried to solve this, or do you just complain about it?"
+- "You said you'd pay for this -- can I take a deposit right now?"
+- "Why haven't you already bought one of the existing solutions?"
+- "Is this actually a top-three priority for you, or more of a nice-to-have?"
+- "If I gave you this for free right now, would you actually set it up this week?"
+
+## Common Mistakes in Questioning
+
+| Mistake | Example | Fix |
+|---------|---------|-----|
+| Leading questions | "Don't you think it would be better if..." | Remove the lead-in: "How do you currently handle..." |
+| Double-barreled questions | "Do you like the design and would you pay for it?" | Ask one question at a time |
+| Assuming the problem exists | "How painful is your expense tracking?" | "How do you track expenses? Is there anything frustrating about it?" |
+| Asking for solutions | "What features would you want?" | "What's the hardest part of your current workflow?" |
+| Anchoring with numbers | "Would you pay $50/month for this?" | "What do you currently spend on tools for this?" |
+| Too many questions at once | Rapid-fire interrogation | Ask one question, wait for the full answer, then follow up |
+
+## Signal vs Noise Cheat Sheet
+
+| Signal (Reliable) | Noise (Unreliable) |
+|--------------------|--------------------|
+| "I spent 3 hours last week doing this manually" | "I would definitely use that" |
+| "We're paying $500/month for a tool that barely works" | "That's a great idea" |
+| "I tried [competitor] but quit because of X" | "I think a lot of people would want this" |
+| "My boss just asked me to fix this problem" | "You should add a feature that does Y" |
+| "I already built a spreadsheet to track this" | "I could see myself using that every day" |
+| "Can I get early access? Here's my email." | "Sure, send me an email about it sometime" |

package/skills/pandasai-analytics/SKILL.md

@@ -0,0 +1,251 @@
+---
+name: pandasai-analytics
+description: Use when analyzing DataFrames with natural language, generating visual reports from backtest/forecast results, exploring data patterns, or building conversational analytics on top of existing pandas data. Especially relevant for demand forecasting, workforce planning, and time-series evaluation projects.
+---
+
+# PandasAI Analytics
+
+## Overview
+
+PandasAI lets you **chat with your DataFrames** using natural language. It generates pandas code + matplotlib charts behind the scenes via an LLM. Use it as a supplementary analytics layer — NOT as core logic.
+
+**Core principle:** PandasAI is a report/insight tool, not a computation engine. Always keep it separate from production pipelines.
+
+## When to Use
+
+- Generating visual reports from backtest/evaluation results
+- Exploring data patterns interactively ("which SKUs have highest error?")
+- Creating AI-narrated analytics summaries
+- Building ad-hoc dashboards from forecast output
+- Comparing model performance across segments/folds
+- Quick data profiling before modeling
+
+**When NOT to use:**
+- Production inference pipelines (non-deterministic)
+- Core model training logic
+- Real-time API responses (LLM latency)
+- Security-sensitive data without Docker sandbox
+
+## Quick Setup
+
+```python
+import pandasai as pai
+from pandasai_litellm.litellm import LiteLLM
+
+# Option 1: Gemini (recommended for cost)
+llm = LiteLLM(model="gemini/gemini-2.0-flash", api_key=GEMINI_API_KEY)
+
+# Option 2: OpenRouter 
+llm = LiteLLM(
+    model="openrouter/google/gemini-2.0-flash-001",
+    api_key=OPENROUTER_KEY,
+    api_base="https://openrouter.ai/api/v1",
+)
+
+# Option 3: OpenAI
+llm = LiteLLM(model="gpt-4.1-mini", api_key=OPENAI_KEY)
+
+pai.config.set({"llm": llm})
+```
+
+## Use Cases for Forecast & Planner Projects
+
+### 1. Backtest Result Analysis
+
+```python
+import json, pandasai as pai
+
+# Load backtest JSON → DataFrame
+with open("output/backtest_results.json") as f:
+    data = json.load(f)
+
+rows = []
+for fold in data["fold_results"]:
+    om = fold.get("order_metrics", {})
+    rows.append({
+        "fold": fold["fold"],
+        "train_window": fold["train_window"],
+        "wmape": om.get("wmape"),
+        "bias": om.get("bias"),
+        "fnr": om.get("fnr"),
+        "mae": om.get("mae"),
+    })
+
+df = pai.DataFrame(rows)
+
+# Natural language queries
+df.chat("Which fold has the lowest wMAPE? Show a bar chart comparing all folds")
+df.chat("Is there a correlation between training window size and accuracy?")
+df.chat("Summarize the bias trend — are we consistently over or under-predicting?")
+```
+
+### 2. Model Comparison (LightGBM vs AutoTS)
+
+```python
+# Build comparison DataFrame from backtest
+comparison = pai.DataFrame([
+    {"model": "LightGBM", "wmape": 45.2, "mae": 1.23, "fnr": 12.5, "bias": -0.15},
+    {"model": "AutoTS",   "wmape": 168.4, "mae": 2.89, "fnr": 16.1, "bias": 2.5},
+])
+
+comparison.chat("Create a grouped bar chart comparing all metrics between the two models")
+comparison.chat("Which model is better for this use case and why? Consider all metrics.")
+```
+
+### 3. Segment Performance Deep-Dive
+
+```python
+segments = pai.DataFrame([
+    {"segment": "bau",     "wmape": 48.0, "bias": -0.12, "fnr": 10.0, "volume": 280},
+    {"segment": "sale",    "wmape": 35.0, "bias":  0.25, "fnr":  5.0, "volume": 150},
+    {"segment": "minor",   "wmape": 55.0, "bias": -0.30, "fnr": 18.0, "volume":  25},
+    {"segment": "holiday", "wmape": 62.0, "bias": -0.45, "fnr": 22.0, "volume":  15},
+])
+
+segments.chat("Which segment needs the most improvement? Consider volume impact.")
+segments.chat("Plot bias by segment. Highlight negative bias segments in red.")
+segments.chat("What's the volume-weighted average wMAPE across all segments?")
+```
+
+### 4. SKU Type Profiling
+
+```python
+# After data_profiler.run_profile() or from backtest sku_type_metrics
+sku_df = pai.DataFrame(sku_type_data)
+
+sku_df.chat("What percentage of total volume comes from each SKU type?")
+sku_df.chat("Plot a pie chart of SKU type distribution by volume")
+sku_df.chat("Which SKU type has the worst MAPE but highest volume? That's our priority.")
+```
+
+### 5. Event Impact Analysis
+
+```python
+events = pai.DataFrame(event_tier_data)
+
+events.chat("Plot MAPE by event tier. Which events are hardest to forecast?")
+events.chat("Calculate the uplift ratio: event-day demand vs BAU average")
+events.chat("Create a heatmap of error rate by event tier and day-of-week")
+```
+
+### 6. Time-Series Pattern Discovery
+
+```python
+# Daily aggregated demand
+daily = pai.DataFrame(daily_demand_data)
+
+daily.chat("Plot the demand trend over the last 90 days with a 7-day moving average")
+daily.chat("Are there any anomalous spikes? List dates with demand > 3x average")
+daily.chat("What's the day-of-week seasonality pattern? Show as boxplot.")
+daily.chat("Detect any trend change points in the last 6 months")
+```
+
+### 7. Workforce Planning Report
+
+```python
+# staffing.py output → DataFrame
+staffing = pai.DataFrame(staffing_data)
+
+staffing.chat("Plot predicted headcount by warehouse for next 14 days")
+staffing.chat("Which warehouses are understaffed vs overstaffed?")
+staffing.chat("Calculate the cost impact of over-prediction vs under-prediction")
+```
+
+### 8. Feature Importance Analysis
+
+```python
+# From model.get_feature_importance()
+importance = pai.DataFrame([
+    {"feature": name, "gain": value, "segment": seg}
+    for seg, feats in model.get_feature_importance().items()
+    for name, value in feats.items()
+])
+
+importance.chat("What are the top 10 features by gain for the 'sale' segment?")
+importance.chat("Plot feature importance comparison between bau and sale segments")
+importance.chat("Which features are important for sale but NOT for bau?")
+```
+
+## General Analytics Use Cases
+
+### Data Exploration
+```python
+df.chat("Show basic statistics for all numeric columns")
+df.chat("What's the distribution of quantity? Show histogram with outlier markers")
+df.chat("How many missing values per column? Show as percentage bar chart")
+```
+
+### Cross-DataFrame Analysis
+```python
+pai.chat("Join these datasets and find which products have orders but no forecast",
+         orders_df, forecast_df)
+```
+
+### Chart Customization
+```python
+df.chat(
+    "Plot monthly revenue trend as a line chart. "
+    "Use dark theme, add grid lines, format y-axis as currency. "
+    "Title: 'Monthly Revenue Trend 2025-2026'"
+)
+```
+
+## Integration Pattern (Standalone Add-on)
+
+```
+┌─────────────────────────────────────────┐
+│         Core Pipeline (untouched)        │
+│  pipeline.py → model.py → backtester.py │
+│              ↓ JSON output               │
+├─────────────────────────────────────────┤
+│       Analytics Add-on (separate)        │
+│  pandasai_config.py → report_generator   │
+│              ↓ HTML report               │
+└─────────────────────────────────────────┘
+```
+
+**Key rules:**
+1. **Never import PandasAI in core modules** — keep it in `src/analytics/`
+2. **Read-only** — analytics module only reads existing output files
+3. **Graceful degradation** — works without API key (static charts only)
+4. **CLI entry point** — `scripts/run_analytics_report.py`
+
+## Existing Implementation
+
+The Boxme Auto Forecast project already has this integration:
+
+| File | Purpose |
+|------|---------|
+| `src/analytics/pandasai_config.py` | Auto-detects Gemini/OpenRouter/OpenAI |
+| `src/analytics/report_generator.py` | 6 chart types + AI narrative + HTML |
+| `scripts/run_analytics_report.py` | CLI: `--input`, `--all`, `--no-ai` |
+| `tests/test_analytics_report.py` | 17 unit tests |
+
+```bash
+# Generate report (static charts, no API key needed)
+python3 scripts/run_analytics_report.py --input output/backtest.json --no-ai
+
+# Generate with AI narrative
+GEMINI_API_KEY=xxx python3 scripts/run_analytics_report.py --input output/backtest.json
+```
+
+## Common Mistakes
+
+| Mistake | Fix |
+|---------|-----|
+| Using `pd.DataFrame` instead of `pai.DataFrame` | PandasAI needs its own DataFrame wrapper |
+| Embedding PandasAI in production pipeline | Keep it as a separate analytics step |
+| Asking vague questions | Be specific: "Plot wMAPE by segment as bar chart" |
+| Not handling missing API keys | Always check `configure_pandasai()` return value |
+| Charts not rendering in scripts | Set `matplotlib.use("Agg")` before importing pyplot |
+| Large DataFrames causing token overflow | Aggregate data before passing to PandasAI |
+
+## Dependencies
+
+```
+pandasai>=3.0.0
+pandasai-litellm>=0.0.1  # For Gemini/OpenRouter/OpenAI
+matplotlib>=3.7.0         # Chart rendering
+```
+
+Python 3.8-3.11 required (PandasAI constraint).