flonat-research 0.1.0

package/skills/shared/engagement-stratified-sampling.md

@@ -0,0 +1,218 @@
+# Engagement-Stratified Sampling
+
+> Shared reference for social media and digital trace research. Ensures representative sampling across the engagement distribution. Prevents viral-content bias. Adapted from CommDAAF (Xu 2026).
+
+## Principle
+
+**Convenience samples over-represent viral content.** Most social media datasets are skewed — a small fraction of posts drive most engagement. Sampling without stratification produces findings that describe viral content, not typical content. Engagement-stratified sampling ensures coverage across the full distribution.
+
+---
+
+## Standard Engagement Tiers
+
+| Tier | Percentile | Purpose | Typical content |
+|------|------------|---------|-----------------|
+| **Viral** | Top 5% | What makes content break out | Influencer posts, breakthrough moments |
+| **High** | 75th–95th | Successful content | Engaged audiences, topical resonance |
+| **Medium** | 25th–75th | Baseline performance | Typical posts, average engagement |
+| **Low** | Bottom 25% | Why content fails / background noise | Low visibility, potential bot content |
+| **Zero** | Engagement = 0 | No-spread baseline | Posts that never circulated |
+
+---
+
+## Engagement Metric Construction
+
+### Standard composite (social media platforms)
+
+```python
+import numpy as np
+
+# Log-transform to handle skewness, +1 to handle zeros
+data['engagement'] = (
+    np.log(data['retweet_count'] + 1) +
+    np.log(data['like_count'] + 1) +
+    np.log(data['quote_count'].fillna(0) + 1)
+)
+```
+
+**Document any modification** to this formula. Platform-specific variants:
+
+| Platform | Available metrics | Notes |
+|----------|------------------|-------|
+| X/Twitter | Retweets, likes, quotes, replies | Quote count often missing in older data |
+| Reddit | Upvotes, comments, awards | Score = upvotes - downvotes |
+| YouTube | Views, likes, comments | Views dominate; consider likes/views ratio |
+| Bluesky | Likes, reposts, replies | Open API, no auth required |
+| Instagram | Likes, comments, shares, saves | Shares/saves often not available |
+
+### R equivalent
+
+```r
+data$engagement <- log(data$retweet_count + 1) +
+                   log(data$like_count + 1) +
+                   log(replace_na(data$quote_count, 0) + 1)
+```
+
+---
+
+## Stratified Sampling Implementation
+
+### Python
+
+```python
+def engagement_stratified_sample(data, engagement_col='engagement',
+                                  n_per_tier=100, seed=42):
+    """Sample equally from each engagement tier."""
+    p95 = data[engagement_col].quantile(0.95)
+    p75 = data[engagement_col].quantile(0.75)
+    p25 = data[engagement_col].quantile(0.25)
+
+    def assign_tier(val):
+        if val >= p95: return 'viral'
+        elif val >= p75: return 'high'
+        elif val >= p25: return 'medium'
+        else: return 'low'
+
+    data = data.copy()
+    data['engagement_tier'] = data[engagement_col].apply(assign_tier)
+
+    sample = (data.groupby('engagement_tier')
+              .apply(lambda x: x.sample(min(len(x), n_per_tier),
+                                        random_state=seed))
+              .reset_index(drop=True))
+
+    return sample
+```
+
+### R
+
+```r
+engagement_stratified_sample <- function(data, engagement_col = "engagement",
+                                          n_per_tier = 100, seed = 42) {
+  set.seed(seed)
+  q <- quantile(data[[engagement_col]], probs = c(0.25, 0.75, 0.95))
+
+  data$engagement_tier <- cut(data[[engagement_col]],
+    breaks = c(-Inf, q[1], q[2], q[3], Inf),
+    labels = c("low", "medium", "high", "viral"))
+
+  data %>%
+    group_by(engagement_tier) %>%
+    slice_sample(n = min(n(), n_per_tier)) %>%
+    ungroup()
+}
+```
+
+---
+
+## Multi-Criteria Sampling
+
+When engagement is one of several strata:
+
+```python
+def multi_criteria_sample(data, strata, total_n=500, seed=42):
+    """
+    strata = {
+        'engagement_tier': {'allocation': 'equal'},
+        'language': {'allocation': 'proportional'},
+        'date': {'allocation': 'coverage'}  # at least 1 per unique value
+    }
+    """
+    # Implementation depends on specific constraints
+    # Key principle: engagement strata are equal, others proportional or coverage
+    pass
+```
+
+**Common multi-criteria designs:**
+
+| Criterion | Allocation | Rationale |
+|-----------|-----------|-----------|
+| Engagement tier | Equal | Prevent viral bias |
+| Language | Proportional | Reflect population distribution |
+| Time period | Coverage (≥1 per day) | Prevent temporal blind spots |
+| Account type | Proportional or capped | Prevent influencer domination |
+| Verified status | Capped (≤30%) | Verified accounts are over-studied |
+
+---
+
+## Power Analysis for Stratified Designs
+
+| Design | Min n/group | Detects | Total for 4 tiers |
+|--------|------------|---------|-------------------|
+| 2-group comparison, d=0.5 | 64 | Medium effect | 128 |
+| 2-group comparison, d=0.3 | 175 | Small effect | 350 |
+| 7 frames × 4 tiers, d=0.3 | 50/cell | Small effect | 1,400 |
+| Regression, 5 predictors | 100 total | Medium R² | 100 |
+
+**Rule:** Calculate power before committing to sample size. Under-powered stratified samples are worse than well-powered random samples.
+
+---
+
+## Saturation Detection
+
+For qualitative or exploratory coding:
+
+```python
+def check_saturation(coded_data, code_var='frame', window=50, threshold=0.05):
+    """Check if last `window` items added < `threshold` proportion of new codes."""
+    all_codes = set()
+    new_code_positions = []
+
+    for i, item in enumerate(coded_data):
+        code = item[code_var]
+        if code not in all_codes:
+            all_codes.add(code)
+            new_code_positions.append(i)
+
+    if not new_code_positions:
+        return {'saturated': True, 'n': len(coded_data), 'last_new': 0}
+
+    last_new = new_code_positions[-1]
+    items_since = len(coded_data) - last_new
+    return {
+        'saturated': items_since >= window,
+        'n': len(coded_data),
+        'last_new': last_new,
+        'items_since_new': items_since
+    }
+```
+
+**Saturation guideline:** If the last 50 coded items produced no new codes/themes, consider stopping.
+
+---
+
+## Integration
+
+### In `/data-analysis` Phase 1
+
+When data includes engagement metrics (likes, shares, retweets, etc.), automatically:
+1. Compute composite engagement score
+2. Assign tiers
+3. Report tier distribution in EDA output
+4. Flag if analysis sample is not engagement-stratified
+
+### In `/experiment-design`
+
+When designing content analysis studies:
+1. Include engagement stratification in sampling plan
+2. Calculate power per tier
+3. Document tier boundaries in pre-analysis plan
+
+### In review agents
+
+Check whether social media studies:
+- Report their sampling strategy
+- Account for engagement distribution
+- Avoid over-representing viral content
+- Flag as Major if unstratified convenience sample is used for causal claims
+
+---
+
+## Anti-Patterns
+
+| Anti-Pattern | Problem | Fix |
+|-------------|---------|-----|
+| Sampling by keyword only | Over-represents viral posts with the keyword | Stratify by engagement after keyword filter |
+| Using "top tweets" API endpoint | Only returns high-engagement content | Use full archive search, then stratify |
+| Treating retweet count as continuous DV | Highly skewed, zero-inflated | Use engagement tiers as strata or log-transform |
+| Equal allocation when one tier is tiny | Viral tier (5%) may have < n_per_tier items | Sample min(available, target), report actual n |

package/skills/shared/escalation-protocol.md

@@ -0,0 +1,74 @@
+# Escalation Protocol: Methodological Pushback
+
+> Shared protocol for review agents and skills. Defines how to push back when methodological answers are vague, designs are unsound, or critical details are missing. Adapted from CommScribe/CommDAAF (Xu 2026).
+
+## Principle
+
+**Helping produce invalid research helps no one.** Review agents have permission — and obligation — to escalate when methodology is inadequate. Escalation is not hostility; it's rigour.
+
+## When to Escalate
+
+- Research design lacks a specified estimand or identification strategy
+- Analysis plan uses defaults without justification
+- Causal claims lack a credible identification argument
+- Sample or data limitations are hand-waved
+- Robustness checks are missing or post-hoc
+- Statistical methods are misapplied (e.g., TWFE with staggered treatment)
+- Results interpretation overstates what the evidence supports
+
+## The Four Levels
+
+| Level | Trigger | Response | Tone |
+|-------|---------|----------|------|
+| **1. Probe** | Vague or missing detail | Ask a specific clarifying question. "What is the estimand?" / "How was K selected?" | Neutral, curious |
+| **2. Explain stakes** | Still vague after probing | State why the detail matters. "Without this, the results are not interpretable because..." | Direct, educational |
+| **3. Challenge** | Pushback or deflection | Name the specific threat to validity. "This design cannot distinguish X from Y. The coefficient captures both effects." | Firm, specific |
+| **4. Flag and stop** | User insists on unsound approach | State clearly that the approach will produce invalid results. Mark the issue as a **Blocker** in the report. Do not proceed with the flawed analysis — offer alternatives instead. | Non-negotiable, constructive |
+
+## Level 4: Flag and Offer Alternatives
+
+Never just block — always offer a path forward. When an approach fails at Level 4:
+
+```
+BLOCKER: [What's wrong and why it invalidates results]
+
+ALTERNATIVES:
+1. [Modified design that addresses the threat]
+2. [Narrower claim that the current design can support]
+3. [Additional data or test that would resolve the issue]
+4. [Descriptive analysis as honest fallback]
+```
+
+## How Agents Use This
+
+### In scored reviews (paper-critic, referee2-reviewer, domain-reviewer)
+
+- Level 1-2 issues: flag in report, deduct per quality-scoring severity
+- Level 3 issues: flag as **Critical** (-15 to -25)
+- Level 4 issues: flag as **Blocker** (-100, automatic 0)
+
+### In interactive work (causal-design, experiment-design, data-analysis)
+
+- Start at Level 1 for any underspecified element
+- Escalate through levels within the conversation
+- At Level 4, refuse to run the analysis and present alternatives
+- User can override with explicit acknowledgment: "I understand the limitation, proceed anyway" — in which case, proceed but add a prominently displayed caveat to any output
+
+
+- Flag claims that exceed what the methodology supports (Level 2-3)
+- Suggest hedging language that accurately reflects the evidence
+
+## What This Is NOT
+
+- **Not a licence to block all work.** Most research involves trade-offs. The protocol targets genuine threats to validity, not stylistic preferences or minor issues.
+- **Not adversarial for its own sake.** Each escalation level includes a constructive element (question, explanation, challenge with specifics, alternatives).
+- **Not a substitute for domain expertise.** When unsure whether something is methodologically unsound, say so: "I'm uncertain whether this is valid — here's my concern: [X]. Can you confirm?"
+
+## Integration with Existing Rules
+
+| Rule | How escalation interacts |
+|------|------------------------|
+| `design-before-results` | Level 1 probe: "Has the analysis plan been specified?" before running anything |
+| `severity-gradient` | Escalation levels map to severity tiers; Phase Detection determines how strictly to apply them |
+| `scope-discipline` | Escalation is in-scope when reviewing methodology; out-of-scope for unrelated issues |
+| `no-hardcoded-results` | Level 2: explain why hard-coded results undermine reproducibility |

package/skills/shared/external-audit-protocol.md

@@ -0,0 +1,205 @@
+# External Audit Protocol
+
+> Shared workflow for all `external-*-audit` skills. Each skill references this file and provides scope-specific configuration (CWD, checklist, report path).
+
+## What This Is
+
+Uses an external LLM CLI (Codex or Gemini) to get a **fresh, independent perspective** from a competing model on Claude Code infrastructure. This extends the agents-vs-skills principle: not just fresh Claude context, but an entirely different model reviewing the work.
+
+**Key difference from `/system-audit`:** System audit uses Claude sub-agents for mechanical checks (counts, symlinks, broken links). External audits are **qualitative** — architecture coherence, design quality, redundancy, missing capabilities, improvement suggestions.
+
+**Model selection:** Pass `--model codex` or `--model gemini` (default: gemini). Running both triangulates — different training and reasoning patterns produce more reliable findings than either alone.
+
+## Model Commands
+
+| Model | CLI | Execute command | Install |
+|-------|-----|----------------|---------|
+| Codex | `codex` | `codex exec --full-auto "{prompt}"` | `npm install -g @openai/codex` + `codex login` |
+| Gemini | `gemini` | `gemini -p "{prompt}" --yolo` | `npm install -g @google/gemini-cli` |
+
+## Pre-Flight
+
+Before anything else:
+
+1. **Verify the CLI is installed:**
+   ```bash
+   which {model} && {model} --version
+   ```
+   If not found, stop and tell the user with install instructions (see table above).
+
+2. **Confirm the target directory exists** (CWD from the skill config).
+
+3. **Check git status** in the target directory to establish a clean baseline:
+   ```bash
+   cd <CWD> && git status --short
+   ```
+   Note any pre-existing uncommitted changes so we can distinguish them from model modifications later.
+
+## Phase 1: Generate Manifest
+
+Claude generates a concise ecosystem manifest describing what the model will be auditing. This gives it the map before it explores the territory.
+
+Write to `/tmp/external-audit-manifest-{scope}.md`:
+
+```markdown
+# Ecosystem Manifest: {scope}
+
+## Root Directory
+{absolute path}
+
+## Key Paths
+{list of important directories and files, relative to root}
+
+## Counts
+{relevant metrics: number of skills, hooks, files, etc.}
+
+## Architecture Summary
+{2-3 sentence description of how the system is organized}
+
+## Tech Stack
+{languages, frameworks, key dependencies}
+```
+
+**Keep it under 200 lines.** The model needs context, not a full directory listing.
+
+## Phase 2: Build Prompt
+
+Combine the manifest with the skill-specific audit checklist into a single prompt file.
+
+Write to `/tmp/external-audit-prompt-{scope}.md`:
+
+```markdown
+# External Audit: {scope}
+
+## CRITICAL INSTRUCTIONS
+
+You are conducting a **READ-ONLY architecture audit**. You must:
+- **NEVER create, modify, or delete any files** — not even "cleanup" or "improvement" edits
+- **NEVER run git commands that change state** (no commit, add, push, checkout, rm)
+- **NEVER install packages or modify dependencies**
+- **NEVER append to, rename, or reorganize existing files**
+- Only read files and output your findings as markdown to stdout
+- If you feel compelled to fix something, describe the fix in your output instead — do NOT apply it
+
+## Ecosystem Manifest
+
+{contents of manifest file}
+
+## Audit Checklist
+
+{contents of skill-specific references/audit-checklist.md}
+
+## Output Format
+
+Structure your response as a markdown report with these sections:
+
+### Executive Summary
+2-3 sentence overall assessment.
+
+### Scored Sections
+For each checklist category, provide:
+- **Score:** A/B/C/D/F
+- **Strengths:** What's done well
+- **Issues:** Problems found (severity: Critical/Major/Minor)
+- **Recommendations:** Specific, actionable improvements
+
+### Architecture Diagram
+If helpful, describe the system architecture in text form.
+
+### Top 5 Recommendations
+Prioritized list of the most impactful improvements, with effort estimates (Quick/Medium/Large).
+
+### Fresh Eyes
+Things that seem odd, redundant, or unnecessarily complex to someone seeing this for the first time. This is the most valuable section — it's what insiders miss.
+```
+
+## Phase 3: Execute
+
+Run from the target CWD with a 10-minute timeout:
+
+**Codex:**
+```bash
+cd <CWD> && timeout 600 codex exec --full-auto "$(cat /tmp/external-audit-prompt-{scope}.md)" 2>&1
+```
+
+**Gemini:**
+```bash
+cd <CWD> && timeout 600 gemini -p "$(cat /tmp/external-audit-prompt-{scope}.md)" --yolo 2>&1
+```
+
+**Capture the full output** — both stdout and stderr.
+
+**If the model fails or times out:**
+- Log the error
+- Report to the user: "{Model} execution failed: {error}. The prompt is saved at `/tmp/external-audit-prompt-{scope}.md` if you want to retry manually."
+- Do not retry automatically
+
+## Phase 4: Safety Check
+
+> **Critical warning:** Both Codex and Gemini are agentic coding tools designed to make changes. Despite read-only instructions, they may create, modify, or delete files. Gemini has a documented tendency to do this. **Always assume the model touched something** and verify thoroughly. This check is not optional.
+
+**Immediately after the model returns**, verify nothing was modified:
+
+```bash
+cd <CWD> && git diff --stat
+cd <CWD> && git status --short
+```
+
+Run **both** commands — `git diff` catches modifications to tracked files, `git status` catches new untracked files or deletions. Compare against the baseline from pre-flight.
+
+If there are **new changes not present before**:
+
+1. **Alert the user immediately:** "{Model} modified files despite read-only instructions. Changed: {list}"
+2. **Revert immediately:** `git checkout -- .` (for modifications/deletions). For new untracked files, list them and delete manually.
+3. **Do not proceed** with report generation until the working tree matches the pre-flight baseline
+4. **Note the violation** in the report header (replace "No files modified" with a description of what was changed and reverted)
+
+## Phase 5: Write Report
+
+Write the model output to the skill-specified report path (typically `audits/{model}-audit-{scope}-YYYY-MM-DD.md`).
+
+Add a header:
+
+```markdown
+# {Model} Audit: {Scope Title} — YYYY-MM-DD
+
+> Generated by {Model CLI description}.
+> Read-only audit | No files modified.
+
+---
+
+{Model output}
+```
+
+## Phase 6: Present
+
+Show the user:
+
+1. **Executive summary** from the report
+2. **Any Critical/Major issues** found
+3. **Top 5 recommendations** with effort estimates
+4. **Fresh Eyes section** — the outsider perspective
+5. Link to the full report
+
+Ask if he wants to address any findings now.
+
+## Error Handling
+
+| Situation | Action |
+|-----------|--------|
+| CLI not installed | Stop, show install instructions |
+| Target directory doesn't exist | Stop, ask the user for correct path |
+| Model times out (>10 min) | Save partial output if any, report failure |
+| Model modifies files | Alert, offer revert, do not continue |
+| Model returns empty output | Report failure, save prompt for manual retry |
+| Git not initialized in target | Skip git safety checks, warn the user |
+| Authentication error | Stop, tell the user to authenticate |
+
+## Integration with Other Skills
+
+| Skill | Relationship |
+|-------|-------------|
+| `/system-audit` | Mechanical checks (counts, symlinks) — complementary, not overlapping |
+| `/audit-project-research` | Per-project structural audit — external audits are cross-cutting |
+| `/lessons-learned` | Critical findings can feed into post-mortems |
+| `/ideas` | Recommendations can be captured as improvement ideas |