@aksp/opencrew 1.2.2 → 1.3.0

package/templates/_opencrew/core/prompts/sherlock-trends.md

@@ -0,0 +1,238 @@
+# Sherlock — Trends Extractor
+
+Load `sherlock-shared.md` before using this extractor.
+
+This file contains the trending topics and cultural signals extraction process. The Architect loads this file (alongside `sherlock-shared.md`) when the investigation requires understanding what's trending now — what people are talking about, what's going viral, and what cultural moments the crew can tap into.
+
+This extractor uses `web_search` and `web_fetch` native tools — no browser automation needed.
+
+---
+
+## When to Use
+
+The Architect dispatches Sherlock-Trends when:
+
+- The crew creates content that needs to feel current and culturally relevant
+- The briefing requires "what's trending" or "what's viral right now"
+- The user wants content that rides cultural waves (newsjacking, trendjacking)
+- A content crew needs angle ideas grounded in current conversations
+- The crew's audience is on platforms where trend velocity matters (Twitter/X, TikTok, Instagram)
+
+---
+
+## Trend Sources
+
+Sherlock-Trends searches across multiple trend surfaces:
+
+| Source | Method | What It Captures |
+|--------|--------|-----------------|
+| Twitter/X trending | `web_search "trending twitter {date}"` or `"{domain} twitter discussion"` | Real-time conversation spikes |
+| GitHub trending | `web_search "github trending {domain}"` or `web_fetch github.com/trending` | Developer/tech trending repos and topics |
+| Product Hunt | `web_search "product hunt trending {domain}"` or `web_fetch producthunt.com` | New product launches, tech trends |
+| Hacker News | `web_search "hacker news {domain}"` or `web_fetch news.ycombinator.com` | Tech community discussions |
+| Reddit | `web_search "site:reddit.com {domain} trending"` | Community discussions by subreddit |
+| Google Discover / News | `web_search "{domain} news today"` | News cycle peaks |
+| YouTube trending | `web_search "{domain} viral video"` or `"{domain} trending youtube"` | Video content trends |
+| Industry newsletters | `web_search "{domain} newsletter roundup 2026"` | Curated weekly trends |
+| Conference talks | `web_search "{domain} conference 2026 talks"` | What experts are presenting now |
+
+---
+
+## Extraction Process
+
+### Step 1: Broad Trend Scan
+
+Run parallel searches across 3-4 trend surfaces most relevant to the crew's domain:
+
+1. **News/current events**: `"{domain}" news {current month} 2026`
+2. **Social conversation**: `"{domain}" twitter discussion` or `"people talking about {domain}"`
+3. **Community pulse**: `"site:reddit.com {domain}"` or `"site:news.ycombinator.com {domain}"`
+4. **Product/tool trends**: `"new {domain} tools 2026"` or `"best {domain} software 2026"`
+
+Collect the top 5-8 signals from each surface. A "signal" is:
+- A topic mentioned by 3+ independent sources in the last 30 days
+- A term or phrase appearing in multiple headlines
+- A tool/product/method getting multiple mentions
+- A debate or controversy with active discussion
+
+### Step 2: Signal Validation
+
+Not every trending topic is worth acting on. Validate each signal:
+
+1. **Velocity**: Is interest accelerating or already fading?
+   - Search for the topic + date range: `"{topic}" after:2026-07-01`
+   - Check if mentions are increasing week-over-week
+
+2. **Relevance**: Does this trend connect to the crew's domain?
+   - Direct hit: trend IS about the domain → high priority
+   - Adjacent: trend touches the domain tangentially → creative angle needed
+   - Noise: trend is unrelated → skip
+
+3. **Longevity**: Is this a flash in the pan or a lasting shift?
+   - News cycle (24-48h): act fast or skip
+   - Seasonal (repeats annually): plan ahead
+   - Structural (permanent change): highest value — build strategy around it
+
+4. **Audience overlap**: Would the crew's target audience care about this?
+   - Direct: audience is actively discussing this → high priority
+   - Peripheral: audience might find it interesting → medium
+   - Irrelevant: audience doesn't care → skip
+
+### Step 3: Angle Generation
+
+For validated trends, brainstorm content angles:
+
+1. **Newsjacking**: How can the crew add its perspective to a breaking story?
+   - "X just happened — here's what it means for {audience}"
+   - "The {domain} perspective on {trend}"
+
+2. **Educational**: How can the crew explain the trend to its audience?
+   - "What {trend} means for {audience} in 2026"
+   - "5 things {audience} needs to know about {trend}"
+
+3. **Contrarian**: What's the opposite take?
+   - "Why {trend} won't matter for {audience}"
+   - "The {trend} hype — what nobody is talking about"
+
+4. **Tactical**: How can the audience act on this trend?
+   - "How to use {trend} for {outcome}"
+   - "{Steps} to take advantage of {trend} today"
+
+### Step 4: Trend Timeline
+
+Map the trend lifecycle to help the crew time its content:
+
+```
+Trend: {trend name}
+├── Emerging (now-2 weeks ago): first mentions appearing
+├── Rising (now): accelerating discussion, media picking it up ← BEST TIME TO ACT
+├── Peak (1-2 weeks from now): maximum attention, saturated coverage
+├── Declining (3-4 weeks): interest fading, late movers
+└── Evergreen (ongoing): settles into background knowledge
+```
+
+For each trend, recommend TIMING: should the crew act now (news cycle), plan for next week (rising trend), or incorporate into long-term strategy (structural shift)?
+
+---
+
+## Output
+
+### `raw-content.md`
+
+```markdown
+# Raw Content: Trend Research — {domain}
+
+Investigated: {YYYY-MM-DD}
+Trend surfaces scanned: {list — Twitter/X, Reddit, GitHub, Product Hunt, Hacker News, etc.}
+Total signals detected: {N}
+Validated trends: {N}
+
+---
+
+## Trend 1: "{Trend Name}"
+
+**Source:** First detected on {platform/url}
+**Velocity:** {accelerating / stable / declining}
+**Relevance:** {direct hit / adjacent / noise}
+**Longevity:** {news cycle / seasonal / structural}
+
+### Signal Evidence
+- {Source 1}: "{headline or excerpt}" — {date}
+- {Source 2}: "{headline or excerpt}" — {date}
+- {Source 3}: "{headline or excerpt}" — {date}
+
+### Audience Connection
+{Why the crew's audience would care about this — or why they wouldn't}
+
+### Suggested Angles
+1. **{Angle type}**: "{draft hook or headline}"
+2. **{Angle type}**: "{draft hook or headline}"
+3. **{Angle type}**: "{draft hook or headline}"
+
+---
+
+## Trend 2: "{Trend Name}"
+...
+```
+
+### `pattern-analysis.md`
+
+```markdown
+# Pattern Analysis: Trend Research — {domain}
+
+Analyzed: {YYYY-MM-DD}
+Trends detected: {N}
+Validated: {N} | Noise filtered: {N}
+Trend lifecycle distribution: {N} emerging, {N} rising, {N} at peak, {N} declining
+
+## Executive Summary
+{3-5 sentences on the cultural moment — what's happening in {domain} right now,
+where attention is flowing, and what the crew should act on immediately vs.
+incorporate long-term}
+
+## Trend Map
+
+### Act Now (rising, high relevance, <1 week window)
+| Trend | Angle | Content Urgency | Expected Shelf Life |
+|-------|-------|----------------|---------------------|
+| {trend} | {suggested angle} | 🔴 Immediate (24-48h) | 1 week |
+
+### Plan This Week (rising, medium-high relevance)
+| Trend | Angle | Content Urgency | Expected Shelf Life |
+|-------|-------|----------------|---------------------|
+| {trend} | {suggested angle} | 🟡 This week | 2-4 weeks |
+
+### Build Into Strategy (structural, evergreen relevance)
+| Trend | Strategic Implication | Timeline |
+|-------|---------------------|----------|
+| {trend} | {how this changes the crew's long-term approach} | Ongoing |
+
+## Trending Vocabulary
+
+Words and phrases gaining traction in {domain} discourse:
+
+| Term | Meaning | Trend Direction | Adopt? |
+|------|---------|----------------|--------|
+| "{term}" | {definition in context} | ↗️ | Yes — audience expects it |
+| "{term}" | {definition} | ↗️ | Cautiously — still niche |
+| "{term}" | {definition} | ↘️ | No — already dated |
+
+## Cultural Moments to Watch
+
+Upcoming events, dates, and cultural moments the crew can plan content around:
+
+| Date | Event / Moment | Relevance | Content Opportunity |
+|------|---------------|-----------|-------------------|
+| {date} | {event} | {why it matters for this audience} | {content idea} |
+
+## Recommendations for Crew
+
+1. **Act now — {trend}**: {specific content recommendation with angle and format}
+2. **Plan ahead — {trend}**: {specific content recommendation}
+3. **Ignore — {trend}**: {why this trending topic is a trap for this audience}
+4. **Monitor — {trend}**: {keep watching but don't act yet}
+5. **Build strategy — {trend}**: {long-term structural recommendation}
+```
+
+---
+
+## Smart Recommendations
+
+- **News/current events crews**: Full extraction — all 9 trend surfaces. Act on breaking trends. Emphasis on velocity and timing.
+- **Content marketing crews**: Focus on structural trends + upcoming cultural moments. Less emphasis on real-time news.
+- **Social media crews**: Focus on Twitter/X trends + Reddit communities + viral content patterns. Emphasis on angle generation.
+- **Product/tech crews**: Focus on GitHub trending + Product Hunt + Hacker News. Emphasis on tool/methodology shifts.
+- **General crews**: Top 5 trends across 3 surfaces. Quick validation pass. Actionable angles only.
+
+## Limitations
+
+- This extractor does not have API access to Twitter/X trending endpoints or TikTok's algorithm. Trend signals are from public search results and may lag real-time by hours.
+- Virality prediction is inherently uncertain — this extractor provides directional signals, not guarantees.
+- Regional trends may not surface in English-language searches. For geo-specific trends, add location qualifiers to search queries.
+
+## Timeout and Error Handling
+
+- Maximum time: 15 minutes
+- If a trend surface returns no useful signals, skip it — not every domain has active trending conversations
+- If trend data is thin, reduce scope: report what IS findable, note what isn't
+- **Never fabricate trends or engagement numbers. "Insufficient data" is a valid finding.**

package/templates/_opencrew/core/prompts/sherlock-web.md

@@ -0,0 +1,220 @@
+# Sherlock — Web Extractor
+
+Load `sherlock-shared.md` before using this extractor.
+
+This file contains the web research extraction process. The Architect loads this file (alongside `sherlock-shared.md`) when the investigation requires researching public websites, blogs, news portals, or any non-social web source.
+
+Unlike social extractors, this extractor does NOT use browser automation — it uses `web_search` and `web_fetch` native tools for content extraction.
+
+---
+
+## When to Use
+
+The Architect dispatches Sherlock-Web when:
+
+- The crew needs competitive analysis from public websites
+- The briefing requires industry research from blogs and news portals
+- The user wants to analyze a specific company's public-facing content strategy
+- The crew domain involves market research, industry trends, or technical documentation
+- Reference URLs point to non-social domains (`.com`, `.org`, `.blog`, `.dev`, etc.)
+
+---
+
+## Source Coverage
+
+Sherlock-Web can extract content from:
+
+| Source Type | Method | Output |
+|---|---|---|
+| Company websites | `web_fetch` homepage + key pages | Content strategy, messaging patterns, product positioning |
+| Industry blogs | `web_search` + `web_fetch` top results | Topic trends, writing style, audience engagement signals |
+| News portals | `web_search` + `web_fetch` articles | Headline patterns, editorial angle, sourcing habits |
+| Documentation / technical sites | `web_fetch` specific pages | Structure patterns, depth of coverage, tone |
+| Landing pages / funnels | `web_fetch` key URLs | Copy patterns, CTA strategies, value proposition framing |
+| Forums / communities (Reddit, Stack Overflow, etc.) | `web_search site:domain` + `web_fetch` | Discussion patterns, pain points, language used by audience |
+
+---
+
+## Extraction Process
+
+### Step 1: Search Strategy
+
+Based on the crew briefing and the domains identified in `discovery.yaml`, formulate search queries that target the specific knowledge needed.
+
+**Query formulation rules:**
+- Be specific — "SaaS onboarding best practices 2026" not "SaaS"
+- Use domain filters when relevant: `site:company.com strategy`
+- Search for examples: `"{domain}" examples` and `"best {content type}" examples`
+- Search for anti-patterns: `"{domain}" mistakes` and `"{domain}" pitfalls`
+- Search for frameworks: `"{domain}" framework` and `"{domain}" methodology`
+
+Run all searches using `web_search`. Collect 3-5 high-signal results per query — skip results that are:
+- Paywalled or login-gated (can't `web_fetch`)
+- Obviously low-quality (thin content, spam)
+- Duplicates of already-collected sources
+- Older than 2 years unless specifically researching historical context
+
+### Step 2: Deep Extraction
+
+For each selected result, use `web_fetch` to retrieve full content. Extract:
+
+1. **Key arguments / theses**: What is the main point? What claims are made?
+2. **Evidence cited**: What data, examples, or sources are referenced?
+3. **Structure pattern**: How is the content organized? (list, narrative, framework, comparison, etc.)
+4. **Language patterns**: Vocabulary choices, tone markers, sentence style
+5. **CTA / engagement hooks**: How does the content invite action or further reading?
+6. **Unique insights**: What does this source say that others don't?
+
+### Step 3: Competitive Analysis (when applicable)
+
+When the crew needs competitive intelligence:
+
+1. Search for competitor names + content keywords: `"{competitor}" "{topic}"`
+2. Fetch their most visible content (blog posts, landing pages, case studies)
+3. Map their content strategy: what topics do they cover? what do they avoid? what's their publishing cadence?
+4. Identify gaps: topics competitors cover weakly or not at all → these become opportunities for the crew
+
+---
+
+## Output
+
+Sherlock-Web produces the same two output files as social extractors:
+
+### `raw-content.md`
+
+```markdown
+# Raw Content: Web Research — {topic/domain}
+
+Investigated: {YYYY-MM-DD}
+Total sources analyzed: {N}
+Source types: {comma-separated: blogs, news, company sites, forums}
+
+---
+
+## Source 1: "{Title or Headline}"
+
+**URL:** {url}
+**Domain:** {domain.com}
+**Type:** {blog post | news article | landing page | documentation | forum thread}
+**Date:** {publication date or "unknown"}
+
+### Summary
+A 2-3 sentence objective summary of the source content.
+
+### Key Arguments
+- {Claim or finding 1}
+- {Claim or finding 2}
+- {Claim or finding 3}
+
+### Evidence Cited
+- {Source or data point 1}
+- {Source or data point 2}
+
+### Structure
+{Description of content structure — e.g., "listicle with 7 items", "narrative case study", "framework with 4 pillars"}
+
+### Language & Tone
+- Tone: {formal/casual/authoritative/conversational/etc.}
+- Notable vocabulary: {words or phrases characteristic of this source}
+- Sentence style: {short punchy / long elaborate / mixed}
+
+### Unique Insights
+{What this specific source contributes to the research — something not found elsewhere}
+
+---
+
+## Source 2: "{Title or Headline}"
+...
+```
+
+### `pattern-analysis.md`
+
+```markdown
+# Pattern Analysis: Web Research — {topic/domain}
+
+Analyzed: {YYYY-MM-DD}
+Sample size: {N} sources across {N} domains
+Period covered: {earliest date} to {latest date}
+
+## Executive Summary
+{3-5 sentences on the dominant patterns, contradictions, and insights found}
+
+## Content Patterns
+
+### Topic Clusters
+| Topic | Sources Covering It | Consensus or Debate? |
+|-------|-------------------|----------------------|
+| {topic 1} | 7 of 10 | Strong consensus — all say X |
+| {topic 2} | 4 of 10 | Debate — 2 say X, 2 say Y |
+
+### Structural Patterns
+- Most common content format: {listicle / framework / narrative / how-to / opinion}
+- Average depth: {superficial — 500 words / medium — 1500 words / deep — 3000+ words}
+- Evidence quality: {data-rich with citations / anecdotal / mixed}
+
+### Argument Patterns
+- Claims everyone agrees on (safe territory):
+  1. {claim}
+  2. {claim}
+- Claims with disagreement (opportunity for unique angle):
+  1. {claim} — Side A says X, Side B says Y
+  2. {claim} — Side A says X, Side B says Y
+
+## Language Patterns
+
+### Vocabulary Across Sources
+Words and phrases that appear across multiple high-quality sources:
+- "{phrase}" — used in {N} sources, signals {what}
+- "{phrase}" — used in {N} sources, signals {what}
+
+### Tone Distribution
+- Authoritative/Expert: {N} of {total} sources
+- Conversational/Accessible: {N} of {total} sources
+- Data-Driven/Analytical: {N} of {total} sources
+
+## Gaps and Opportunities
+
+Topics that are under-covered or absent from top sources:
+1. **{gap}**: Why it matters and how the crew could own this space
+2. **{gap}**: Why it matters and how the crew could own this space
+
+---
+
+## Source Quality Assessment
+
+| Source | Depth | Evidence Quality | Originality | Overall |
+|--------|-------|-----------------|-------------|---------|
+| {source 1} | High | Data-rich | High | ★★★★★ |
+| {source 2} | Medium | Anecdotal | Medium | ★★★ |
+| {source 3} | Low | None | Low | ★ |
+
+---
+
+## Recommendations for Crew
+
+Five actionable recommendations based on web research patterns:
+
+1. **[Recommendation]**: {Details with source references}
+2. **[Recommendation]**: {Details with source references}
+3. **[Recommendation]**: {Details with source references}
+4. **[Recommendation]**: {Details with source references}
+5. **[Recommendation]**: {Details with source references}
+```
+
+---
+
+## Smart Recommendations
+
+Sherlock-Web recommends extraction depth based on crew type:
+
+- **Content creation crews**: Focus on topic clusters + language patterns. Extract hooks, CTAs, and structural patterns from top-performing content in the niche. 3-5 sources per query.
+- **Strategy/analysis crews**: Deep extraction — 5-8 sources per query. Focus on competitive gaps, argument patterns, and evidence quality.
+- **Technical/research crews**: Prioritize documentation sites and authoritative sources. Extract frameworks, methodologies, and technical vocabulary.
+- **General crews**: Balanced — 3-5 sources, mix of breadth and depth.
+
+## Timeout and Error Handling
+
+- Maximum time per web research session: 15 minutes. Web search + fetch is faster than browser automation.
+- If `web_fetch` fails for a URL (paywall, 403, timeout), skip and note in raw-content.md: "[Skipped: URL inaccessible — {reason}]"
+- If search returns no results, broaden the query once. If still empty, report the gap — don't fabricate.
+- **Never fabricate data. Never declare success over empty results.**