sc-research 1.0.11 → 1.0.12

package/dist/cli.js

@@ -523,11 +523,12 @@ function printHelp() {
 sc-research - Social Communities Research bootstrap CLI
 
 Usage:
-  sc-research init --ai <target[,target...]> [options]
+    sc-research <command> [options]
 
 Commands:
   init         Initialize SC-Research support files for a project
   research     Run research engine (same as project "research" script)
+    research:deep Run deep research engine
   visualize    Launch visualization app
 
 Options:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "sc-research",
-  "version": "1.0.11",
+  "version": "1.0.12",
   "description": "Headless Social Media Research Data Provider for AI Agents",
   "type": "module",
   "main": "dist/index.js",

package/templates/base/commands/quick.md

@@ -3,11 +3,13 @@ description: Quick research a topic and get a fast text answer (Reddit only, ski
 ---
 
 1. Run the quick research fetcher (Reddit only)
-   > `sc-research research "ARGUMENTS"`
+
+   > `sc-research research "ARGUMENTS" --source=reddit`
    >
-   > This fetches Reddit data only and skips X for speed. Use `/research` if you need both sources.
+   > This fetches Reddit data only and skips X for speed. Use `/deep-research` if you need deeper multi-source analysis.
 
 2. Read the raw data
+
    > Read `reddit_data.json` only. Ignore `x_data.json` even if it exists from a prior run.
 
 3. Provide a concise answer

package/templates/base/skills/social_media_controversy.md

@@ -16,6 +16,22 @@ Use existing raw files only:
 
 At least one valid source file must exist.
 
+## Command Execution Flow
+
+Use this sequence for controversy analysis:
+
+1. Fetch or refresh raw data (outside this worker):
+
+- `sc-research research:deep "TOPIC"`
+- Optional filters: `--source=reddit|x|both --from=YYYY-MM-DD --to=YYYY-MM-DD`
+
+2. Run this `social_media_controversy` worker to generate `classified_controversy.json`.
+3. Optional visualization:
+
+- `sc-research visualize`
+
+If raw files are missing, stale, or mismatched for the requested topic/date range, run step 1 first.
+
 ## Step 1: Preflight Validation
 
 1. Parse each available source file.

package/templates/base/skills/social_media_discovery.md

@@ -16,6 +16,22 @@ Use existing raw files only:
 
 At least one valid source file must exist.
 
+## Command Execution Flow
+
+Use this sequence when running discovery end-to-end:
+
+1. Fetch or refresh discovery raw data (outside this worker):
+
+- `sc-research research:deep "TOPIC" --mode=discovery`
+- Optional filters: `--source=reddit|x|both --from=YYYY-MM-DD --to=YYYY-MM-DD`
+
+2. Run this `social_media_discovery` worker to analyze existing raw files and produce `classified_discovery.json`.
+3. Optional visualization step:
+
+- `sc-research visualize`
+
+If raw files are missing, stale, or mismatched for the requested topic/date range, instruct the caller to run step 1 first.
+
 ## Step 1: Preflight Validation
 
 1. Parse each available source file.

package/templates/base/skills/social_media_fetch.md

@@ -16,6 +16,16 @@ This worker is the data-ingestion step for the pipeline. It fetches raw social d
 
 At least one output file must be produced for a successful fetch.
 
+## Command Execution Flow
+
+Run one of these commands based on requested depth:
+
+- Quick fetch: `sc-research research "TOPIC"`
+- Deep fetch: `sc-research research:deep "TOPIC"`
+- Discovery fetch: `sc-research research:deep "TOPIC" --mode=discovery`
+
+Then validate outputs (`reddit_data.json` / `x_data.json`) before handing off to analysis workers.
+
 ## Step 1: Choose Fetch Mode
 
 - **Quick mode** (faster, lighter coverage):
@@ -73,10 +83,10 @@ Return:
 
 ## Error Handling
 
-| Scenario | Symptom | Action |
-|---|---|---|
-| Missing `OPENAI_API_KEY` | Auth failure on Reddit fetch | Set valid `OPENAI_API_KEY` in `.sc-research` |
-| Missing `XAI_API_KEY` | X file missing/empty while Reddit succeeds | Set `XAI_API_KEY` in `.sc-research` to enable X fetching |
-| No relevant results | `items` is empty | Broaden topic keywords and retry |
-| Rate limit / transient API failure | Timeout or provider error | Wait, then retry once with same parameters |
-| Malformed output | JSON parse failure or missing `items` | Re-run fetch; if repeated, report failure explicitly |
+| Scenario                           | Symptom                                    | Action                                                   |
+| ---------------------------------- | ------------------------------------------ | -------------------------------------------------------- |
+| Missing `OPENAI_API_KEY`           | Auth failure on Reddit fetch               | Set valid `OPENAI_API_KEY` in `.sc-research`             |
+| Missing `XAI_API_KEY`              | X file missing/empty while Reddit succeeds | Set `XAI_API_KEY` in `.sc-research` to enable X fetching |
+| No relevant results                | `items` is empty                           | Broaden topic keywords and retry                         |
+| Rate limit / transient API failure | Timeout or provider error                  | Wait, then retry once with same parameters               |
+| Malformed output                   | JSON parse failure or missing `items`      | Re-run fetch; if repeated, report failure explicitly     |

package/templates/base/skills/social_media_rank.md

@@ -16,6 +16,19 @@ Use existing files only:
 
 At least one valid source file must exist.
 
+## Command Execution Flow
+
+Use this sequence for ranking:
+
+1. Fetch or refresh raw data (outside this worker):
+   - `sc-research research:deep "TOPIC"`
+   - Optional filters: `--source=reddit|x|both --from=YYYY-MM-DD --to=YYYY-MM-DD`
+2. Run this `social_media_rank` worker to generate `classified_rank.json`.
+3. Optional visualization:
+   - `sc-research visualize`
+
+If raw files are missing, stale, or mismatched for the requested topic/date range, run step 1 first.
+
 ## Step 1: Preflight Validation
 
 Before analysis:

package/templates/base/skills/social_media_schema.md

@@ -15,153 +15,160 @@ Use this file as the canonical schema source for all classified outputs:
 
 If another skill instruction conflicts with this file, this file wins.
 
+## Command Execution Note
+
+This is a reference-only skill. There is no direct CLI command to run this skill.
+
+- Use it by reading this schema before writing any `classified_*.json` output.
+- Runnable commands belong to fetch/analysis/visualize skills (for example `sc-research research:deep "TOPIC"` and `sc-research visualize`).
+
 ## Canonical Type Definitions
 
 ```typescript
 export interface ClassifiedData {
-    topic: string;
-    source_file?: string;
-    products: Product[];
-    key_insights: string[];
+  topic: string;
+  source_file?: string;
+  products: Product[];
+  key_insights: string[];
 }
 
 export interface Product {
-    rank: number;
-    name: string;
-    sentiment: SentimentLabel;
-    mentions: number;
-    estimated_engagement_score: number;
-    consensus: string;
-    pros: string[];
-    cons: string[];
-    highlight_quotes: Array<{
-        text: string;
-        author: string;
-        link: string;
-        context?: "pro" | "con" | "general";
-    }>;
+  rank: number;
+  name: string;
+  sentiment: SentimentLabel;
+  mentions: number;
+  estimated_engagement_score: number;
+  consensus: string;
+  pros: string[];
+  cons: string[];
+  highlight_quotes: Array<{
+    text: string;
+    author: string;
+    link: string;
+    context?: "pro" | "con" | "general";
+  }>;
 }
 
 export type SentimentLabel =
-    | "Positive"
-    | "Negative"
-    | "Mixed"
-    | "Very Positive";
+  | "Positive"
+  | "Negative"
+  | "Mixed"
+  | "Very Positive";
 
 export interface SentimentData {
-    topic: string;
-    overall_mood: SentimentLabel;
-    distribution: {
-        very_positive: number;
-        positive: number;
-        mixed: number;
-        negative: number;
-    };
-    by_source: {
-        reddit: SourceSentiment;
-        x: SourceSentiment;
-    };
-    product_sentiments: ProductSentiment[];
-}
-
-export interface SourceSentiment {
+  topic: string;
+  overall_mood: SentimentLabel;
+  distribution: {
     very_positive: number;
     positive: number;
     mixed: number;
     negative: number;
+  };
+  by_source: {
+    reddit: SourceSentiment;
+    x: SourceSentiment;
+  };
+  product_sentiments: ProductSentiment[];
+}
+
+export interface SourceSentiment {
+  very_positive: number;
+  positive: number;
+  mixed: number;
+  negative: number;
 }
 
 export interface ProductSentiment {
-    name: string;
-    overall: SentimentLabel;
-    reddit_sentiment: SentimentLabel | null;
-    x_sentiment: SentimentLabel | null;
-    evidence_quotes: Array<{
-        text: string;
-        author: string;
-        link: string;
-        sentiment: SentimentLabel;
-    }>;
+  name: string;
+  overall: SentimentLabel;
+  reddit_sentiment: SentimentLabel | null;
+  x_sentiment: SentimentLabel | null;
+  evidence_quotes: Array<{
+    text: string;
+    author: string;
+    link: string;
+    sentiment: SentimentLabel;
+  }>;
 }
 
 export interface TrendData {
-    topic: string;
-    date_range: {
-        from: string;
-        to: string;
-    };
-    granularity?: "day" | "week" | "month";
-    timeline: TimelinePoint[];
-    key_moments: KeyMoment[];
+  topic: string;
+  date_range: {
+    from: string;
+    to: string;
+  };
+  granularity?: "day" | "week" | "month";
+  timeline: TimelinePoint[];
+  key_moments: KeyMoment[];
 }
 
 export interface TimelinePoint {
-    period: string;
-    post_count: number;
-    total_engagement: number;
-    reddit_posts: number;
-    x_posts: number;
+  period: string;
+  post_count: number;
+  total_engagement: number;
+  reddit_posts: number;
+  x_posts: number;
 }
 
 export interface KeyMoment {
-    date: string;
-    event: string;
-    significance: "high" | "medium" | "low";
-    url?: string;
+  date: string;
+  event: string;
+  significance: "high" | "medium" | "low";
+  url?: string;
 }
 
 export interface ControversyData {
-    topic: string;
-    overall_divisiveness: "Low" | "Medium" | "High";
-    controversies: Controversy[];
+  topic: string;
+  overall_divisiveness: "Low" | "Medium" | "High";
+  controversies: Controversy[];
 }
 
 export interface Controversy {
-    topic: string;
-    heat_score: number;
-    divisiveness: "Low" | "Medium" | "High";
-    side_a: ControversySide;
-    side_b: ControversySide;
+  topic: string;
+  heat_score: number;
+  divisiveness: "Low" | "Medium" | "High";
+  side_a: ControversySide;
+  side_b: ControversySide;
 }
 
 export interface ControversySide {
-    position: string;
-    supporter_count: number;
-    sample_quotes: Array<{
-        text: string;
-        author: string;
-        link: string;
-    }>;
+  position: string;
+  supporter_count: number;
+  sample_quotes: Array<{
+    text: string;
+    author: string;
+    link: string;
+  }>;
 }
 
 export interface DiscoveryData {
-    topic: string;
-    period: string;
-    total_posts_analyzed: number;
-    trending_topics: DiscoveryTopic[];
+  topic: string;
+  period: string;
+  total_posts_analyzed: number;
+  trending_topics: DiscoveryTopic[];
 }
 
 export interface DiscoveryTopic {
-    id: string;
-    topic_name: string;
-    description: string;
-    category: string;
-    engagement_score: number;
-    sentiment: "positive" | "negative" | "neutral" | "mixed";
-    key_posts: KeyPost[];
-    highlight_comments: Array<{
-        text: string;
-        author: string;
-        link: string;
-        platform: "reddit" | "x";
-    }>;
+  id: string;
+  topic_name: string;
+  description: string;
+  category: string;
+  engagement_score: number;
+  sentiment: "positive" | "negative" | "neutral" | "mixed";
+  key_posts: KeyPost[];
+  highlight_comments: Array<{
+    text: string;
+    author: string;
+    link: string;
+    platform: "reddit" | "x";
+  }>;
 }
 
 export interface KeyPost {
-    title: string;
-    url: string;
-    platform: "reddit" | "x";
-    engagement: number;
-    thumbnail?: string;
+  title: string;
+  url: string;
+  platform: "reddit" | "x";
+  engagement: number;
+  thumbnail?: string;
 }
 ```

package/templates/base/skills/social_media_sentiment.md

@@ -16,6 +16,22 @@ Use existing raw files only:
 
 At least one valid source file must exist.
 
+## Command Execution Flow
+
+Use this sequence for sentiment analysis:
+
+1. Fetch or refresh raw data (outside this worker):
+
+- `sc-research research:deep "TOPIC"`
+- Optional filters: `--source=reddit|x|both --from=YYYY-MM-DD --to=YYYY-MM-DD`
+
+2. Run this `social_media_sentiment` worker to generate `classified_sentiment.json`.
+3. Optional visualization:
+
+- `sc-research visualize`
+
+If raw files are missing, stale, or mismatched for the requested topic/date range, run step 1 first.
+
 ## Step 1: Preflight Validation
 
 1. Parse each available source file.

package/templates/base/skills/social_media_trend.md

@@ -16,6 +16,19 @@ Use existing files only:
 
 At least one valid source file must exist.
 
+## Command Execution Flow
+
+Use this sequence for trend analysis:
+
+1. Fetch or refresh raw data (outside this worker):
+   - `sc-research research:deep "TOPIC"`
+   - Optional filters: `--source=reddit|x|both --from=YYYY-MM-DD --to=YYYY-MM-DD`
+2. Run this `social_media_trend` worker to generate `classified_trend.json`.
+3. Optional visualization:
+   - `sc-research visualize`
+
+If raw files are missing, stale, or mismatched for the requested topic/date range, run step 1 first.
+
 ## Step 1: Preflight and Date Parsing
 
 1. Parse each available source file and validate `items` arrays.

package/templates/base/skills/social_media_visualize.md

@@ -19,6 +19,16 @@ Before launch:
    - `classified_discovery.json`
 2. If none exist, report that analysis must run first.
 
+## Command Execution Flow
+
+Use this sequence for visualization:
+
+1. Ensure at least one classified output exists (for example by running `social_media_rank`, `social_media_sentiment`, `social_media_trend`, `social_media_controversy`, or `social_media_discovery`).
+2. Launch dashboard:
+   - `sc-research visualize`
+3. Optional single-file mode:
+   - `sc-research visualize path/to/classified_rank.json`
+
 ## Usage
 
 Default auto-detect mode:
@@ -56,8 +66,8 @@ Return:
 
 ## Troubleshooting
 
-| Issue | Symptom | Action |
-|---|---|---|
+| Issue               | Symptom                                | Action                                                                |
+| ------------------- | -------------------------------------- | --------------------------------------------------------------------- |
 | No classified files | command exits early or empty dashboard | run a worker skill first (rank/sentiment/trend/controversy/discovery) |
-| Port already in use | server starts on another port | report actual URL shown by runtime |
-| Malformed JSON file | tab fails to render or command errors | regenerate that specific `classified_*.json` file |
+| Port already in use | server starts on another port          | report actual URL shown by runtime                                    |
+| Malformed JSON file | tab fails to render or command errors  | regenerate that specific `classified_*.json` file                     |

package/templates/base/skills/using_social_media_research.md

@@ -24,20 +24,36 @@ Treat this as the entrypoint skill when user requests map to social-media resear
 - For analysis intents, run exactly one worker by default.
 - Run multiple workers only when the user explicitly asks for multi-view output (for example: "full analysis", "all views", "run everything").
 - Use deep fetch for every worker analysis.
-- Use quick fetch for explicit quick-answer requests and ambiguity fallback.
-- If intent is still ambiguous after routing rules, use quick-answer mode (no classified file).
+- Use quick fetch for explicit quick-answer requests.
+- If intent is still ambiguous after routing rules, use rank as the default overview route.
 - Reuse existing raw data only if it still matches topic, source, and date range; otherwise refetch.
 
+## Command Execution Summary
+
+After route selection, execute commands as follows:
+
+- Rank / Sentiment / Trend / Controversy routes:
+  - `sc-research research:deep "TOPIC" [--source=...] [--from=YYYY-MM-DD --to=YYYY-MM-DD]`
+  - then run the selected worker skill to produce the matching `classified_*.json`.
+- Discovery route:
+  - `sc-research research:deep "TOPIC" --mode=discovery [--source=...] [--from=... --to=...]`
+  - then run `social_media_discovery`.
+- Quick answer route:
+  - `sc-research research "TOPIC" --source=reddit [--from=... --to=...]`
+  - then synthesize a short text answer (no classified file).
+- Optional dashboard view after any classified output:
+  - `sc-research visualize`
+
 ## Step 1: Resolve Intent (Strict Precedence)
 
 Apply rules top-to-bottom:
 
 1. **Explicit multi-analysis request**
    - If user asks for "full analysis", "all views", or equivalent, run:
-     1) `social_media_rank`
-     2) `social_media_sentiment`
-     3) `social_media_trend`
-     4) `social_media_controversy`
+     1. `social_media_rank`
+     2. `social_media_sentiment`
+     3. `social_media_trend`
+     4. `social_media_controversy`
    - Include `social_media_discovery` only if the user also asks for emerging/viral topic discovery.
 2. **Explicit template request**
    - If the user names a template ("sentiment", "trend", "controversy", "discovery", "rank"), that route wins.
@@ -46,28 +62,29 @@ Apply rules top-to-bottom:
 4. **Inferred strongest intent**
    - Map by primary question intent (keywords table below).
 5. **Fallback**
-   - If still ambiguous, default to quick-answer mode (no classified file).
+
+- If still ambiguous, default to `social_media_rank`.
 
 ## Step 2: Map Intent to Route
 
-| Route | Typical trigger phrases | Worker | Output |
-|---|---|---|---|
-| Rank | best, top, compare, recommendation, which one | `social_media_rank` | `classified_rank.json` |
-| Sentiment | feel, sentiment, opinion, positive/negative | `social_media_sentiment` | `classified_sentiment.json` |
-| Trend | timeline, over time, peak, growth, decline | `social_media_trend` | `classified_trend.json` |
-| Controversy | debate, divisive, disagreement, polarizing, vs | `social_media_controversy` | `classified_controversy.json` |
-| Discovery | trending topics, viral, discover themes, clusters | `social_media_discovery` | `classified_discovery.json` |
-| Quick Answer | quick answer, short summary, brief | none | direct text answer |
+| Route        | Typical trigger phrases                           | Worker                     | Output                        |
+| ------------ | ------------------------------------------------- | -------------------------- | ----------------------------- |
+| Rank         | best, top, compare, recommendation, which one     | `social_media_rank`        | `classified_rank.json`        |
+| Sentiment    | feel, sentiment, opinion, positive/negative       | `social_media_sentiment`   | `classified_sentiment.json`   |
+| Trend        | timeline, over time, peak, growth, decline        | `social_media_trend`       | `classified_trend.json`       |
+| Controversy  | debate, divisive, disagreement, polarizing, vs    | `social_media_controversy` | `classified_controversy.json` |
+| Discovery    | trending topics, viral, discover themes, clusters | `social_media_discovery`   | `classified_discovery.json`   |
+| Quick Answer | quick answer, short summary, brief                | none                       | direct text answer            |
 
 ## Step 3: Detect Source and Fetch Depth
 
 ### Source Detection
 
-| User wording | Source flag |
-|---|---|
-| "on Reddit", "subreddit", "Redditors" | `--source=reddit` |
-| "on X", "on Twitter", "tweets" | `--source=x` |
-| no explicit source | no source flag (default behavior) |
+| User wording                          | Source flag                       |
+| ------------------------------------- | --------------------------------- |
+| "on Reddit", "subreddit", "Redditors" | `--source=reddit`                 |
+| "on X", "on Twitter", "tweets"        | `--source=x`                      |
+| no explicit source                    | no source flag (default behavior) |
 
 ### Fetch Strategy
 
@@ -107,10 +124,10 @@ If any check fails, run a fresh fetch.
 
 ## File Map
 
-| File | Producer |
-|---|---|
-| `classified_rank.json` | `social_media_rank` |
-| `classified_sentiment.json` | `social_media_sentiment` |
-| `classified_trend.json` | `social_media_trend` |
+| File                          | Producer                   |
+| ----------------------------- | -------------------------- |
+| `classified_rank.json`        | `social_media_rank`        |
+| `classified_sentiment.json`   | `social_media_sentiment`   |
+| `classified_trend.json`       | `social_media_trend`       |
 | `classified_controversy.json` | `social_media_controversy` |
-| `classified_discovery.json` | `social_media_discovery` |
+| `classified_discovery.json`   | `social_media_discovery`   |