flonat-research 0.1.0

package/.context/preferences/task-naming.md

@@ -0,0 +1,28 @@
+# Task Naming Conventions
+
+## Structure
+
+`[Action verb] [Object] - [Brief context]`
+
+## Action Verbs
+
+| Verb | When to use |
+|------|-------------|
+| Write | Creating new content from scratch |
+| Draft | First version, expecting revisions |
+| Edit | Modifying existing content |
+| Review | Reading and providing feedback |
+| Read | Reading for understanding |
+| Send | Communication tasks |
+| Submit | Final submission |
+| Prepare | Getting something ready |
+| Research | Investigating a topic |
+| Decide | Making a choice |
+| Follow up | Checking on something sent |
+
+## Examples
+
+- "Draft methodology section - uncertainty paper"
+- "Review co-author feedback - nudge-exp intro"
+- "Submit ethics application - experiment 1"
+- "Read Smith (2024) - lit review"

package/.context/profile.md

@@ -0,0 +1,29 @@
+# Profile
+
+<!-- Customise this with your own details. This file helps Claude understand
+     your background and calibrate its responses. -->
+
+## Identity
+
+- **Name:** [Your Name]
+- **Role:** PhD Researcher
+- **Institution:** UK University
+
+## Supervisors
+
+| Name | Institution | Role | Focus |
+|------|-------------|------|-------|
+| [Supervisor 1] | [University] | Primary | [Their focus] |
+| [Supervisor 2] | [University] | Secondary | [Their focus] |
+
+## Teaching
+
+- Graduate TA for Research Methods (Autumn term)
+- Workshop facilitator for Data Analysis in R
+
+## Tools & Preferences
+
+- **Writing:** LaTeX (papers), Markdown (notes)
+- **Data analysis:** R (primary), Python (secondary)
+- **Version control:** Git
+- **Reference management:** Paperpile (synced to `paperpile.bib`)

package/.context/projects/_index.md

@@ -0,0 +1,41 @@
+# Projects Index
+
+> Overview of all active research projects. Updated as projects progress.
+
+## Active Projects
+
+| # | Short name | Title | Stage | Target | Co-authors |
+|---|-----------|-------|-------|--------|------------|
+| 1 | uncertainty | Decision Making Under Uncertainty | Drafting | Management Science | J. Smith |
+| 2 | nudge-exp | Nudge Effectiveness in Org Settings | Literature Review | JDM | T. Brown |
+| 3 | survey-methods | Survey Design for Behavioural Research | Idea | OBHDP | — |
+
+## Project Details
+
+### 1. uncertainty
+
+- **Directory:** `~/Research/uncertainty/`
+- **Paper:** `paper/` (Overleaf symlink)
+- **Status:** Methodology and results drafted, introduction needs revision
+- **Next:** Finish results tables, send to co-author for review
+- **Pipeline stage:** Drafting
+
+### 2. nudge-exp
+
+- **Directory:** `~/Research/nudge-exp/`
+- **Paper:** `paper/` (Overleaf symlink)
+- **Status:** Literature review in progress, 40+ papers collected
+- **Next:** Synthesise literature, draft theoretical framework
+- **Pipeline stage:** Literature Review
+
+### 3. survey-methods
+
+- **Directory:** `~/Research/survey-methods/`
+- **Paper:** Not started
+- **Status:** Idea phase — exploring feasibility
+- **Next:** Read 5 key papers, write 1-page research question memo
+- **Pipeline stage:** Idea
+
+## Archived Projects
+
+None yet.

package/.context/projects/papers/nudge-exp.md

@@ -0,0 +1,22 @@
+# Paper: Nudge Effectiveness in Organisational Settings
+
+## Metadata
+
+| Field | Value |
+|-------|-------|
+| Short name | nudge-exp |
+| Title | When Do Nudges Work? Context Effects in Organisational Decision Making |
+| Authors | Alex Chen, Tom Brown |
+| Target journal | Journal of Decision Making |
+| Stage | Literature Review |
+| Directory | `~/Research/nudge-exp/` |
+
+## Research Question
+
+Under what conditions do behavioural nudges improve decision quality in organisational contexts?
+
+## Current Status
+
+- Literature review in progress (40+ papers collected)
+- Theoretical framework emerging
+- Experiment not yet designed

package/.context/projects/papers/uncertainty.md

@@ -0,0 +1,31 @@
+# Paper: Decision Making Under Uncertainty
+
+## Metadata
+
+| Field | Value |
+|-------|-------|
+| Short name | uncertainty |
+| Title | Decision Making Under Uncertainty: A Behavioural Approach |
+| Authors | Alex Chen, Jane Smith |
+| Target journal | Management Science |
+| Stage | Drafting |
+| Directory | `~/Research/uncertainty/` |
+
+## Research Question
+
+How do individuals' decision-making strategies change when uncertainty increases in organisational settings?
+
+## Key Contributions
+
+1. Novel experimental design for measuring decision quality under varying uncertainty
+2. Theoretical framework linking uncertainty perception to strategy selection
+3. Empirical evidence from a lab experiment with N=200 participants
+
+## Current Status
+
+- Introduction: First draft (needs revision based on co-author feedback)
+- Literature review: Complete
+- Methodology: Complete
+- Results: In progress (Table 2 pending)
+- Discussion: Not started
+- Conclusion: Not started

package/.context/resources/claude-scientific-writer-review.md

@@ -0,0 +1,48 @@
+# Review: K-Dense-AI/claude-scientific-writer
+
+> Reviewed 2026-02-24. Repo: https://github.com/K-Dense-AI/claude-scientific-writer (856 stars)
+
+## What It Is
+
+A monolithic scientific document generation tool combining Claude with real-time literature search (Perplexity via OpenRouter) and AI diagram generation (Nano Banana Pro). Available as Claude Code plugin, CLI, or Python API. MIT-licensed.
+
+## Architecture
+
+- **24 skills**: writing, research-lookup, peer-review, citation-management, clinical-reports, research-grants, latex-posters, scientific-slides, hypothesis-generation, market-research-reports, scientific-schematics, infographics, literature-review, venue-templates, etc.
+- **External APIs**: Perplexity Sonar Pro Search (literature), Nano Banana Pro (diagrams), Parallel Web API (general web search/extraction)
+- **LaTeX-first** output with BibTeX citations by default
+- **Multi-pass writing**: skeleton → research per section → write → verify citations → compile → PDF review via image conversion
+- **Version management**: v1_draft.tex, v2_draft.tex, etc. — never overwrites previous versions
+- All research results saved to `sources/` folder for auditability and context recovery
+
+## Key Strengths
+
+- **Citation verification loop**: research before writing, verify every BibTeX entry has complete metadata (DOI, volume, pages), web-search for missing fields
+- **Peer review**: quantitative ScholarEval framework (8-dimension scoring)
+- **Figure generation**: minimum counts per document type, mandatory graphical abstract
+- **Full lifecycle**: hypothesis → writing → review → revision
+- **Structured output pipeline** with progress tracking and summary reports
+
+## Overlap with Our Setup
+
+| Their Feature | Our Equivalent |
+|---------------|---------------|
+| `research-lookup` + `citation-management` | `/literature` skill + `packages/mcp-bibliography` |
+| `peer-review` / `scholar-evaluation` | `referee2-reviewer`, `paper-critic`, `domain-reviewer` agents |
+| `venue-templates` | Scout `venues.json` + framing workflow |
+| `latex-posters` / `scientific-slides` | `/beamer-deck`, `/quarto-deck` |
+
+## Key Differences from Our Approach
+
+- **Monolithic vs modular**: one tool does everything; ours is skills + agents + standalone apps
+- **External API-heavy**: requires Perplexity, Nano Banana Pro, Parallel Web API keys (more cost)
+- **"Never stop" philosophy**: their CLAUDE.md says "NEVER ask 'would you like me to continue?'"; ours has plan-first discipline and phase boundaries
+- **No novelty scoring or venue-specific framing**: that's unique to our Scout
+- **Heavy CLAUDE.md**: ~500 lines of instructions; ours follows lean-claude-md rule
+- **No design-before-results discipline**: no equivalent to our research design rules
+
+## Ideas Worth Borrowing
+
+1. **Citation metadata verification loop** — after BibTeX creation, systematically search for missing DOI/volume/pages. Could enhance `/bib-validate`.
+4. **PDF review via image conversion** — convert PDF to images and visually inspect each page for formatting issues. Could add as a step in `/latex-autofix`.
+5. **Progress logging per section** — timestamped logs of word count and citation count per section during writing. Lighter version could fit our session logging.

package/.context/resources/cunningham-multi-analyst-agents.md

@@ -0,0 +1,104 @@
+# Scott Cunningham — Multi-Analyst Agent Designs & Non-Standard Errors
+
+> Source: Substack posts 21, 25, 26, 27 (Feb–Mar 2026). Series on Claude Code for causal inference.
+
+## Post 26: Computational Many-Analysts Design (DiD Part 2)
+
+### Core Idea
+
+Run N independent AI agents on the same dataset + estimator, isolating each in a temp directory with no shared memory. This approximates a many-analysts design (Silberzahn et al. 2018, Huntington-Klein et al. 2021, Menkveld et al. 2024 "non-standard errors") at near-zero marginal cost.
+
+### Experiment
+
+- 15 agents × 5 packages (csdid, csdid2, did, differences, diff-diff) × 3 languages
+- Each launched via `claude -p` in isolated temp dirs, no shared history
+- Instructions: Callaway & Sant'Anna estimator, universal base period, not-yet-treated controls
+- **Primary discretionary node:** covariate selection for conditional parallel trends
+
+### Key Findings
+
+- **Structural decisions unanimous (15/15):** control group, base period, balanced cohorts, trimming
+- **All variation in covariate selection:** log GDP (14/15), population (12/15), poverty (10/15), health spending (7/15), Bolsa Familia (2/15), geographic vars (1/15)
+- Agents drew the confounder/mediator boundary differently — same reasoning, different thresholds
+- All 15 chose doubly robust; Stata agents split on DRIPW vs DRIMP variant
+
+### Relevance
+
+- Demonstrates that agent-based many-analysts designs are feasible and cheap
+- Covariate selection as the key discretionary node in DiD — directly measurable
+- Forest plots of agent estimates could become a standard robustness diagnostic
+- The spread across agents quantifies researcher degrees of freedom computationally
+
+## Post 21: Attention, Verification, and Convex Costs
+
+### Framework
+
+Isoquants for cognitive work have flattened → human time ≈ machine time for many tasks → rational substitution toward cheaper machine time → reduced human attention despite more output.
+
+### Key Claims
+
+1. **5x productivity, >5x mess.** "Stock pollutants" (excess files, duplicate outputs, hard-coded results, branching pipelines) grow convex in productivity, not linearly.
+2. **Three binding constraints in human-AI research:**
+   - Human verification (Karpathy: "the new skill is verification")
+   - Sustained attention (resist automation of the learning process itself)
+   - Congestion management (finding things in your own output)
+3. **Legacy projects are harder.** New projects scaffold cleanly; revived R&Rs become "Frankenstein hodge-podge" of old and new organisation.
+
+### Relevance
+
+- Empirical account of human-AI collaboration friction from a power user
+- The convex cost function is testable: does error rate grow faster than output rate?
+- "Beautiful decks" as attention maintenance strategy — works at low volume, breaks at scale
+- Directly maps to the user's research themes (human-AI collaboration, org behaviour)
+
+## Post 25: OpenClaw Security & Anthropic's Response
+
+### What Happened
+
+OpenClaw (always-on WhatsApp AI agent, 230K GitHub stars) had critical security failures:
+- No authentication by default; ~1000 open installations found via Shodan
+- Prompt injection: embedded instructions in emails forwarded data to attackers
+- 230+ malicious plugins in one week on unmoderated skill marketplace
+- Cisco found a #1-ranked skill that was literal malware (curl exfiltration)
+
+### Anthropic's Response
+
+- Cowork (scheduled tasks) and Remote Control (phone → local machine)
+- Key architectural differences: no inbound ports, short-lived credentials, sandboxed, TLS encryption
+- Trade-off: more constrained but safer; computer must be awake for Cowork
+
+### Relevance
+
+- Case study in safety norms emerging in AI agent ecosystems
+- The semantic attack surface (prompt = attack vector) vs traditional code exploits
+- Anthropic's "safety as brand equity" thesis — early safety investment paying off in agent era
+
+## Post 27: Research vs Publishing Economics
+
+### Core Argument
+
+AI collapses the cost of producing submission-quality manuscripts → 5x submissions → acceptance rates crater → journals earn windfall fees → referee system breaks → prisoner's dilemma.
+
+### Numbers
+
+- ~12,000 research-active economists, ~39,000 submissions/year currently
+- At 5x: top-5 acceptance drops from 5% to 1%; 87 journals go from $6.2M to $31M in fees
+- Referee need: 146K reports/year against ~54K realistic supply
+- Individual cost of 3x scaling: ~$3,200/year (fees + Claude Max)
+
+### Project APE (Zurich Social Catalyst Lab)
+
+- 204 fully automated econ papers; 60 added in one week
+- 4.7% win rate vs AER-equivalent in head-to-head, improving to 7.6% in latest cohort
+- Goal: 1,000 papers
+
+### Cunningham's Own Test
+
+Fully automated a paper end-to-end: idea generation → shift-share IV → web data → analysis → writing → refine.ink review ($40-50) → revision → referee2 audit → cross-language code audit. Total: ~$100, few hours.
+
+### Relevance
+
+- The "binding constraint shifts from production to evaluation" thesis
+- 75 working papers on a website = "lottery player" signal, not "serious researcher"
+- Refine.ink as a verification service that gets paid multiple times per paper (polishing → submission → desk screen → R&R)
+- For the user: the institutional response to AI-generated research volume is a research topic in itself (org behaviour, mechanism design)

package/.context/resources/cunningham-multilang-code-audit.md

@@ -0,0 +1,62 @@
+# Scott Cunningham — Multi-Language Code Audits
+
+> Source: Substack (Feb 2026). First in a series on Claude Code for causal inference pipelines.
+
+## Core Idea
+
+Frame LLM coding errors as **measurement error**: random, language-specific syntax mistakes that are independent across languages. If errors in R, Python, and Stata are independent:
+
+```
+P(all three wrong) = ε_R × ε_P × ε_S ≈ very small
+```
+
+Therefore: replicate your entire data pipeline in 2-3 languages and compare outputs numerically (coefficients, test statistics, table values) to catch implementation bugs.
+
+## Key Claims
+
+1. **Hallucination as measurement error.** LLM code errors are random draws from a language-specific error distribution — analogous to classical measurement error (variable = true value + noise).
+2. **Independence across languages.** Syntax-specific errors (Stata's missing-value trap, R's factor ordering, Python's 0-indexing) are plausibly independent because they stem from different language grammars.
+3. **Full pipeline replication, not just code review.** Replicate cleaning, merging, and estimation end-to-end in 2+ languages. Compare outputs table-by-table, coefficient-by-coefficient.
+
+## When It Works
+
+- Deterministic computations: OLS, DiD, IV, F-tests, analytical standard errors, R-squared
+- Data processing: cleaning, merging, variable construction
+- Any pipeline where the same inputs should produce identical numerical outputs
+
+## When It Doesn't Work
+
+- Bootstrap (language-specific seeds)
+- Simulation-based estimators (simulated MLE, method of simulated moments)
+- Bayesian MCMC (Gibbs, HMC)
+- EM algorithms with random starting points
+- Machine learning (SGD, random forests, neural net initialisation)
+
+## Limitation: Independence Breaks Down for Conceptual Errors
+
+The independence assumption holds for *syntax-specific* bugs but NOT for *design errors* (wrong estimand, wrong merge logic, wrong identification strategy). Conceptual errors replicate across languages because they're language-agnostic. This approach catches implementation bugs, not design bugs.
+
+## Illustrative Example: Stata Missing-Value Trap
+
+```stata
+* WRONG — also replaces missing values to 10
+replace olddog = 10 if olddog > 10
+
+* CORRECT — excludes missing values
+replace olddog = 10 if olddog > 10 & olddog ~= .
+```
+
+Claude Code knows the correct version (trained on Stata manuals + Nick Cox's listserv posts) but may randomly omit the missing-value guard. If you only run Stata, this propagates silently. If you also run R and Python (where `NA` handling differs), the discrepancy surfaces in the output comparison.
+
+## Case Study
+
+Callaway & Sant'Anna DiD applied to Brazilian deinstitutionalisation (CAPS) and homicides. Packages used:
+- **Stata:** `csdid`, `csdid2`
+- **R:** `did`
+- **Python:** `differences` (Dionisi), `diff-diff` (Gerber)
+
+## Relevance
+
+- Complements existing `/code-review` skill (which already has cross-language verification as category 7/11) — but Cunningham's version is more aggressive: full pipeline replication, not just spot-checking
+- Does NOT replace design-level audits (`/devils-advocate`, `design-before-results` rule) — those catch the conceptual errors this approach misses
+- Most applicable to empirical papers with deterministic estimation pipelines

package/.context/resources/google-ai-co-scientist-review.md

@@ -0,0 +1,72 @@
+# Paper Note: Towards an AI Co-Scientist (Google, Feb 2025)
+
+> arXiv 2502.18864 | 81 pages | Saved at `to-sort/downloads/2502.18864-ai-co-scientist.pdf`
+
+## Summary
+
+Multi-agent system built on Gemini 2.0 for scientific hypothesis generation and refinement. Six specialized agents (Generation, Reflection, Ranking, Evolution, Proximity, Meta-review) orchestrated by a Supervisor agent. Applied to three biomedical domains with wet-lab validation.
+
+## Architecture
+
+- **Generation agent**: Literature search, simulated scientific debates (self-play), iterative assumption identification, research expansion
+- **Reflection agent**: Multi-tier review (initial safety screen, full literature-grounded review, deep verification, observational review, recontextualized tournament review)
+- **Ranking agent**: Pairwise tournament using Elo ratings. Evaluates on alignment, plausibility, novelty, testability, safety
+- **Proximity agent**: Builds similarity graph over hypotheses for clustering and gap identification
+- **Evolution agent**: Refines top-ranked hypotheses via enhancement, feasibility improvement, combination, simplification, out-of-box thinking
+- **Meta-review agent**: Synthesizes patterns across all reviews, provides feedback to other agents' prompts (learning without backpropagation)
+
+Key design: tournament-based evolution with Elo ratings replaces "pick best from list" with a scalable ranking mechanism. More compute = monotonically better results (test-time compute scaling).
+
+## Key Results
+
+| Metric | Result |
+|--------|--------|
+| GPQA diamond accuracy (top-1) | 78.4% |
+| Expert preference ranking | 2.36/4 (best among all systems) |
+| Expert novelty rating | 3.64/5 |
+| Expert impact rating | 3.09/5 |
+| Drug repurposing (AML) | 3/6 expert-selected drugs confirmed in vitro; KIRA6 is a genuine novel discovery |
+| Liver fibrosis targets | 2/3 epigenetic modifiers showed anti-fibrotic activity |
+| AMR (cf-PICIs) | Independently recapitulated an unpublished finding in 2 days vs. decade-long conventional programme |
+
+Expert-in-the-loop results (Fig 6): AI-augmented expert hypotheses eventually surpass both pure-AI and pure-expert baselines.
+
+## Limitations (Acknowledged)
+
+- Open-access literature only (misses paywalled papers)
+- No negative results data (published literature skews positive)
+- No multimodal reasoning (can't read figures, charts, omics data)
+- Elo is self-evaluated, not grounded in external truth
+- Biomedical only (no social science, economics, or humanities validation)
+- Small expert evaluation (11 goals, 7 experts)
+
+## Relevance to the user's Research
+
+### For human-AI collaboration research
+- Concrete case study of human-AI complementarity (citable: Fig 6 shows AI+expert > either alone)
+- Expert-in-the-loop design: scientists refine goals, provide manual reviews, contribute hypotheses that compete alongside AI-generated ones
+- Gap in their framework: no MCDM perspective on how scientists should *decide* which AI-generated hypotheses to pursue (novelty vs. feasibility vs. resource cost vs. alignment)
+
+### For multi-agent systems research
+- Well-documented architecture with emergent capabilities through orchestration
+- Tournament + meta-review loop is a novel coordination mechanism distinct from sequential pipelines or debate protocols
+- Proximity graph for idea deduplication and exploration-space mapping
+
+### Borrowed for Claude Code infrastructure
+- **Multi-turn debate** added to `/devils-advocate` (3-round: critic -> defense -> adjudication)
+
+### Not worth implementing
+- Full tournament/Elo infrastructure (overkill for 6 projects vs. hundreds of hypotheses)
+- Proximity graphs (not enough parallel outputs to need clustering)
+- Evolution agent (existing review cycles already iterate)
+
+## Citation
+
+```bibtex
+@article{gottweis2025towards,
+  title={Towards an AI Co-Scientist},
+  author={Gottweis, Juraj and others},
+  year={2025},
+  journal={arXiv preprint arXiv:2502.18864}
+}
+```

package/.context/resources/karpathy-llm-council-review.md

@@ -0,0 +1,58 @@
+# karpathy/llm-council — Review
+
+> Reviewed 2026-02-25. Repo: https://github.com/karpathy/llm-council (Karpathy "Saturday hack")
+
+## What It Does
+
+Multi-model council that answers questions in 3 stages:
+1. **Stage 1 — Individual responses**: Query all council members (4 models) in parallel via OpenRouter
+2. **Stage 2 — Peer review**: Each model ranks all other responses (anonymised as "Response A/B/C/D"), parses `FINAL RANKING:` section
+3. **Stage 3 — Chairman synthesis**: A designated chairman model synthesises all responses + rankings into a final answer
+
+## Architecture
+
+- **Backend**: FastAPI + httpx (async OpenRouter calls) + JSON file storage
+- **Frontend**: React + Vite (separate app)
+- **API**: OpenRouter only (same as our Scout migration)
+- **Config**: `COUNCIL_MODELS` list + `CHAIRMAN_MODEL` in `config.py`
+- **Streaming**: SSE endpoint streams stage completions to frontend
+- Dependencies: `fastapi`, `httpx`, `pydantic`, `uvicorn`, `python-dotenv`
+
+## Key Design Choices
+
+- Anonymous peer review: models see "Response A/B/C" not model names — prevents brand-bias
+- Parallel queries via `asyncio.gather` — all models queried simultaneously
+- Aggregate rankings: Borda-count-style average position across all reviewers
+- Chairman can be any model (default: Gemini 3 Pro)
+- Regex parsing of `FINAL RANKING:` section — fragile but works
+
+## What's Good
+
+- Clean separation: `openrouter.py` (API client) → `council.py` (orchestration) → `main.py` (routes)
+- Parallel execution throughout — fast despite 4+ model calls per stage
+- SSE streaming so frontend shows progressive results
+- Anonymisation prevents model-name bias in peer review
+
+## Weaknesses
+
+- No system prompts — all queries are bare user messages
+- JSON file storage — no DB, no caching
+- `FINAL RANKING:` parsing is fragile (regex on free-form text)
+- No retry logic on model failures
+- No cost tracking or token counting
+- Fixed council membership — no per-query model selection
+- No conversation context (each turn is independent, no history passed)
+
+## Relevance for Scout
+
+Core 3-stage pattern is directly reusable:
+1. Query N models with the same prompt (research idea to evaluate)
+2. Each model reviews/ranks the other responses (anonymised)
+3. Chairman synthesises into a consensus evaluation
+
+Key adaptations needed:
+- Domain-specific system prompts (research novelty/framing context, not general Q&A)
+- Use structured JSON output (Pydantic models) instead of free-form markdown
+- Integrate with existing OpenRouter client (`LLMService`) rather than raw httpx
+- Store results in SQLite (existing pattern) not JSON files
+- HTMX partial rendering, not React SPA

package/.context/resources/multi-coder-reliability-protocol.md

@@ -0,0 +1,175 @@
+# Multi-Coder Reliability Protocol
+
+> Source: weiai-wayne-xu/commDAAF — Nonprofit Mission Framing Study (March 2026)
+> Problem solved: Establishing inter-coder reliability for AI-assisted content analysis without human validation
+
+## Overview
+
+Traditional content analysis requires human coders for reliability. This protocol uses **three independent AI models** to establish reliability without human intervention while avoiding "single-coder circularity."
+
+## The 3-Model Protocol
+
+```
+┌─────────────┐    ┌─────────────┐
+│  Coder A    │    │  Coder B    │
+│  (Codex)    │    │  (Gemini)   │
+│  Batch 1    │    │  Batch 2    │
+└──────┬──────┘    └──────┬──────┘
+       │                  │
+       └────────┬─────────┘
+                ▼
+        ┌─────────────┐
+        │  Coder C    │
+        │  (Claude)   │
+        │ 30-item     │
+        │ reliability │
+        │ sample      │
+        └─────────────┘
+```
+
+## Protocol Steps
+
+### Step 1: Split Primary Coding
+```python
+# Divide dataset between two coders
+batch_1 = missions[:len(missions)//2]  # → Codex
+batch_2 = missions[len(missions)//2:]  # → Gemini
+```
+
+### Step 2: Code Independently
+Each coder receives identical instructions but codes independently:
+```python
+codex_results = code_with_codex(batch_1, instructions)
+gemini_results = code_with_gemini(batch_2, instructions)
+```
+
+### Step 3: Sample for Reliability
+```python
+import random
+random.seed(42)  # Reproducible
+
+# Sample from BOTH batches
+combined = codex_results + gemini_results
+reliability_sample = random.sample(combined, 30)
+```
+
+### Step 4: Third-Model Validation
+```python
+# Claude codes the 30-item sample independently
+claude_results = code_with_claude(reliability_sample, instructions)
+
+# Compare against original coders
+kappa_vs_codex = cohens_kappa(
+    [r for r in claude_results if r["original_coder"] == "codex"],
+    [r for r in codex_results if r["id"] in sample_ids]
+)
+kappa_vs_gemini = cohens_kappa(
+    [r for r in claude_results if r["original_coder"] == "gemini"],
+    [r for r in gemini_results if r["id"] in sample_ids]
+)
+```
+
+### Step 5: Report Both kappa Values
+```markdown
+### Inter-Coder Reliability
+
+| Comparison | kappa | Agreement |
+|------------|-------|-----------|
+| Claude vs Codex | 0.94 | 97% |
+| Claude vs Gemini | 0.92 | 96% |
+| **Overall** | **0.935** | **96.7%** |
+```
+
+## Why This Works
+
+| Concern | How Protocol Addresses It |
+|---------|---------------------------|
+| Single-model bias | Three different model families |
+| Circular validation | Validator (Claude) never saw training data |
+| Provider collusion | Codex (OpenAI), Gemini (Google), Claude (Anthropic) |
+| Replicability | Fixed random seed, identical instructions |
+
+## Minimum Requirements
+
+- **3 distinct models** from different providers
+- **30+ items** in reliability sample
+- **Fixed random seed** for reproducibility
+- **Identical instructions** to all coders
+- **kappa > 0.80** for acceptable reliability (Landis & Koch)
+
+## Landis & Koch kappa Interpretation
+
+| kappa | Interpretation |
+|-------|----------------|
+| < 0.00 | Poor |
+| 0.00-0.20 | Slight |
+| 0.21-0.40 | Fair |
+| 0.41-0.60 | Moderate |
+| 0.61-0.80 | Substantial |
+| **0.81-1.00** | **Almost perfect** |
+
+## Code Template
+
+```python
+from sklearn.metrics import cohen_kappa_score
+import random
+
+def compute_reliability(primary_results: list, validator_results: list) -> dict:
+    """Compute inter-coder reliability between primary coder and validator."""
+
+    # Match items by ID
+    matched = []
+    for v in validator_results:
+        for p in primary_results:
+            if p["id"] == v["id"]:
+                matched.append((p["code"], v["code"]))
+                break
+
+    primary_codes = [m[0] for m in matched]
+    validator_codes = [m[1] for m in matched]
+
+    # Cohen's kappa
+    kappa = cohen_kappa_score(primary_codes, validator_codes)
+
+    # Simple agreement
+    agreement = sum(p == v for p, v in matched) / len(matched)
+
+    return {
+        "n_sample": len(matched),
+        "cohens_kappa": round(kappa, 3),
+        "simple_agreement": round(agreement * 100, 1),
+        "interpretation": interpret_kappa(kappa),
+    }
+
+def interpret_kappa(k: float) -> str:
+    if k >= 0.81: return "almost perfect"
+    if k >= 0.61: return "substantial"
+    if k >= 0.41: return "moderate"
+    if k >= 0.21: return "fair"
+    if k >= 0.00: return "slight"
+    return "poor"
+```
+
+## Reporting Disagreements
+
+Always report the nature of disagreements:
+
+```markdown
+### Disagreement Analysis
+
+Single disagreement (1/30): County fair organization
+- Codex: SERVICE (provides entertainment to community)
+- Claude: FELLOWSHIP (exists for member exhibitors)
+
+**Interpretation:** Borderline case illustrating reasonable ambiguity
+in the scheme, not systematic divergence.
+```
+
+## Anti-Patterns to Avoid
+
+| Anti-Pattern | Problem | Fix |
+|--------------|---------|-----|
+| Same model for coding + validation | Circular | Use different model families |
+| No reliability sample | Unverifiable | Always sample 30+ items |
+| Reconciliation before kappa | Inflated agreement | Compute kappa on independent codes |
+| Reporting only agreement % | Ignores chance | Always report Cohen's kappa |