flonat-research 0.1.0

package/skills/shared/paid-api-safety.md

@@ -0,0 +1,174 @@
+# Paid API Safety Patterns
+
+> Defensive conventions for any skill or code that spends money on external APIs. Reference this document when generating scripts that call OpenAI, Anthropic, or other metered services.
+
+## Mandatory Safeguards
+
+Every script that makes paid API calls **must** include all four:
+
+| Safeguard | What it does | When it runs |
+|-----------|-------------|--------------|
+| **Cost estimation** | Calculate estimated cost from token counts and rates, display before proceeding | Before any API call |
+| **`--dry-run` mode** | Simulate the full pipeline (file prep, chunking, validation) without submitting | On `--dry-run` flag |
+| **Explicit confirmation** | Require the user to type `yes` (not just press Enter) after seeing the estimate | After cost display, before submission |
+| **Call logging** | Record every API call with timestamp, endpoint, token count, and cost to a log file | On every call |
+
+### Cost Estimation Pattern
+
+```python
+def estimate_cost(records, model, input_tpm, output_tpm):
+    """Estimate cost BEFORE spending money."""
+    est_input_tokens = sum(len(r["content"]) // 4 for r in records)  # rough: 1 token ~ 4 chars
+    est_output_tokens = len(records) * output_tpm  # expected output per record
+
+    cost = (est_input_tokens / 1_000_000) * RATES[model]["input"] \
+         + (est_output_tokens / 1_000_000) * RATES[model]["output"]
+
+    print(f"Records:          {len(records):,}")
+    print(f"Est. input tokens:  {est_input_tokens:,}")
+    print(f"Est. output tokens: {est_output_tokens:,}")
+    print(f"Model:            {model}")
+    print(f"Estimated cost:   ${cost:.2f}")
+    return cost
+```
+
+### Dry-Run Pattern
+
+```python
+import argparse
+
+parser = argparse.ArgumentParser()
+parser.add_argument("--dry-run", action="store_true",
+                    help="Show what would be submitted without calling the API")
+args = parser.parse_args()
+
+if args.dry_run:
+    print(f"[DRY RUN] Would submit {len(batches)} batches ({total_records:,} records)")
+    print(f"[DRY RUN] Estimated cost: ${estimated_cost:.2f}")
+    sys.exit(0)
+```
+
+### Confirmation Pattern
+
+```python
+def confirm_spend(estimated_cost):
+    """Require explicit 'yes' before spending money."""
+    response = input(f"\nThis will cost approximately ${estimated_cost:.2f}. Type 'yes' to proceed: ")
+    if response.strip().lower() != "yes":
+        print("Aborted.")
+        sys.exit(0)
+```
+
+## Recommended Safeguards (Batch/Bulk Operations)
+
+For jobs processing 100+ records or costing more than $1:
+
+| Safeguard | Details |
+|-----------|---------|
+| **Retry with backoff** | Start at 1s, double each retry, max 5 attempts. Respect `Retry-After` headers. |
+| **Job ID tracking** | Write batch/job IDs to a tracking file (JSON or CSV) so status can be checked later. |
+| **Chunking** | Split large jobs into chunks (e.g., 5,000-10,000 records per batch file). |
+| **Resume capability** | Track completed chunks in a state file. On restart, skip already-completed work. |
+
+### Retry Pattern
+
+```python
+import time
+
+def call_with_backoff(fn, *args, max_retries=5, base_delay=1.0):
+    """Retry with exponential backoff."""
+    for attempt in range(max_retries):
+        try:
+            return fn(*args)
+        except (RateLimitError, APIConnectionError, ssl.SSLError) as e:
+            if attempt == max_retries - 1:
+                raise
+            delay = base_delay * (2 ** attempt)
+            print(f"[RETRY] Attempt {attempt + 1}/{max_retries} failed: {e}. "
+                  f"Retrying in {delay:.0f}s...")
+            time.sleep(delay)
+```
+
+### Job Tracking Pattern
+
+```python
+import json
+from datetime import datetime
+
+TRACKING_FILE = "batch_tracking.json"
+
+def save_batch_id(batch_id, record_count, status="submitted"):
+    """Save batch ID for later status checks."""
+    tracking = json.loads(Path(TRACKING_FILE).read_text()) if Path(TRACKING_FILE).exists() else []
+    tracking.append({
+        "batch_id": batch_id,
+        "records": record_count,
+        "status": status,
+        "submitted_at": datetime.now().isoformat()
+    })
+    Path(TRACKING_FILE).write_text(json.dumps(tracking, indent=2))
+```
+
+### Resume Pattern
+
+```python
+STATE_FILE = "processing_state.json"
+
+def get_completed_chunks():
+    """Load set of already-completed chunk IDs."""
+    if Path(STATE_FILE).exists():
+        return set(json.loads(Path(STATE_FILE).read_text()).get("completed", []))
+    return set()
+
+def mark_chunk_complete(chunk_id):
+    """Record a chunk as done so it won't be re-processed."""
+    state = json.loads(Path(STATE_FILE).read_text()) if Path(STATE_FILE).exists() else {"completed": []}
+    state["completed"].append(chunk_id)
+    Path(STATE_FILE).write_text(json.dumps(state, indent=2))
+```
+
+## Cost Estimation Rates
+
+Reference rates for common APIs. **Always verify current pricing before submitting large jobs.**
+
+| Provider | Model | Input (per 1M tokens) | Output (per 1M tokens) | Notes |
+|----------|-------|----------------------|------------------------|-------|
+| OpenAI | gpt-4o | $2.50 | $10.00 | Batch API is 50% off |
+| OpenAI | gpt-4o-mini | $0.15 | $0.60 | Batch API is 50% off |
+| OpenAI | gpt-4.1 | $2.00 | $8.00 | Batch API is 50% off |
+| OpenAI | gpt-4.1-mini | $0.40 | $1.60 | Batch API is 50% off |
+| OpenAI | gpt-4.1-nano | $0.10 | $0.40 | Batch API is 50% off |
+| Anthropic | claude-sonnet-4 | $3.00 | $15.00 | Batches: 50% off, prompt caching available |
+| Anthropic | claude-haiku-3.5 | $0.80 | $4.00 | Batches: 50% off |
+
+**Formula:** Always show the user the calculation, not just the result:
+
+```
+Cost = (input_tokens / 1M) x input_rate + (output_tokens / 1M) x output_rate
+     = (2,500,000 / 1M) x $2.50 + (500,000 / 1M) x $10.00
+     = $6.25 + $5.00
+     = $11.25
+```
+
+## Error Handling Checklist
+
+| Failure mode | Required behaviour |
+|-------------|-------------------|
+| Network/SSL error | Retry with backoff (do not fail immediately) |
+| Rate limit (429) | Respect `Retry-After` header, then retry |
+| Partial batch failure | Log which batches succeeded vs. failed; save partial results |
+| Auth error (401/403) | Fail immediately with clear message (do not retry) |
+| Invalid input | Validate before submission, not after spending money |
+| Interrupted run | State file enables resume from last completed chunk |
+
+## When This Applies
+
+- Any skill that generates code calling a metered API (OpenAI, Anthropic, Google, etc.)
+- Batch processing scripts (Batch API, bulk embeddings, large-scale classification)
+- One-off scripts that could accidentally run up a bill (e.g., processing an entire dataset)
+
+## When to Skip
+
+- Free-tier APIs or APIs with no per-call cost
+- Local model inference (Ollama, vLLM, etc.)
+- Single-call interactions where cost is negligible (< $0.01)

package/skills/shared/palettes.md

@@ -0,0 +1,90 @@
+# Reference Colour Palettes
+
+> Shared reference for `/beamer-deck` and `/quarto-deck`. Starting points drawn from Scott Cunningham's example decks. Always create an original palette.
+
+## Professional (Deep Blues)
+
+| Name | Hex | Use |
+|------|-----|-----|
+| Midnight | `#1A1A2E` | Headings |
+| DeepBlue | `#16213E` | Body text |
+| RoyalBlue | `#0F3460` | Secondary |
+| Coral | `#E94560` | Accent |
+| SoftGray | `#BDC3C7` | Muted elements |
+| CloudWhite | `#FAFBFC` | Background |
+
+### Beamer
+```latex
+\definecolor{Midnight}{HTML}{1A1A2E}
+\definecolor{DeepBlue}{HTML}{16213E}
+\definecolor{RoyalBlue}{HTML}{0F3460}
+\definecolor{Coral}{HTML}{E94560}
+\definecolor{SoftGray}{HTML}{BDC3C7}
+\definecolor{CloudWhite}{HTML}{FAFBFC}
+```
+
+### CSS
+```css
+--r-background-color: #FAFBFC;
+--r-main-color: #16213E;
+--r-heading-color: #1A1A2E;
+--r-link-color: #E94560;
+```
+
+## Energetic (Warm Tones)
+
+| Name | Hex | Use |
+|------|-----|-----|
+| DarkSlate | `#2C3E50` | Headings, body |
+| Crimson | `#C0392B` | Accent |
+| Sunset | `#F39C12` | Highlights |
+| Teal | `#1ABC9C` | Secondary |
+| Lavender | `#9B59B6` | Tertiary |
+| LightGray | `#ECF0F1` | Background |
+
+### Beamer
+```latex
+\definecolor{DarkSlate}{HTML}{2C3E50}
+\definecolor{Crimson}{HTML}{C0392B}
+\definecolor{Sunset}{HTML}{F39C12}
+\definecolor{Teal}{HTML}{1ABC9C}
+\definecolor{Lavender}{HTML}{9B59B6}
+\definecolor{LightGray}{HTML}{ECF0F1}
+```
+
+### CSS
+```css
+--r-background-color: #ECF0F1;
+--r-main-color: #2C3E50;
+--r-heading-color: #2C3E50;
+--r-link-color: #C0392B;
+```
+
+## Academic (Muted Earth)
+
+| Name | Hex | Use |
+|------|-----|-----|
+| NavyBlue | `#003366` | Headings |
+| ForestGreen | `#2E7D32` | Secondary |
+| BurntOrange | `#D84315` | Accent |
+| WarmGray | `#757575` | Muted |
+| Cream | `#FFF8E1` | Background |
+| Charcoal | `#37474F` | Body text |
+
+### Beamer
+```latex
+\definecolor{NavyBlue}{HTML}{003366}
+\definecolor{ForestGreen}{HTML}{2E7D32}
+\definecolor{BurntOrange}{HTML}{D84315}
+\definecolor{WarmGray}{HTML}{757575}
+\definecolor{Cream}{HTML}{FFF8E1}
+\definecolor{Charcoal}{HTML}{37474F}
+```
+
+### CSS
+```css
+--r-background-color: #FFF8E1;
+--r-main-color: #37474F;
+--r-heading-color: #003366;
+--r-link-color: #D84315;
+```

package/skills/shared/progressive-disclosure.md

@@ -0,0 +1,92 @@
+# Progressive Disclosure for Skills
+
+> Pattern for structuring large skills so that Claude reads only the sections it needs, rather than loading everything into context upfront.
+
+## The Problem
+
+When a skill is invoked, its full `SKILL.md` is loaded into context. For large skills (200+ lines), most of the content may be irrelevant to the specific task. This wastes tokens and can push sessions toward compression earlier than necessary. Anthropic's official recommendation is to keep SKILL.md under 500 lines; our stricter threshold (200 lines to trigger splitting) ensures leaner context usage.
+
+## The Pattern
+
+Split a skill into a **lean core** (the main `SKILL.md`) and **on-demand sections** (files in `references/`). The core contains workflow logic, decision points, and a section index. Reference files contain detailed instructions that Claude reads only when the workflow reaches that branch.
+
+### Structure
+
+```
+skills/my-skill/
+  SKILL.md              # Lean core: workflow + section index (< 100 lines ideal)
+  references/
+    phase-1-details.md  # Detailed instructions for phase 1
+    phase-2-details.md  # Detailed instructions for phase 2
+    edge-cases.md       # Handling for uncommon scenarios
+    report-template.md  # Output format template
+```
+
+### Section Index in SKILL.md
+
+Include a section index that tells Claude what exists and when to read it:
+
+```markdown
+## Reference Sections
+
+Read these **only when needed** — do not pre-load all of them.
+
+| Section | When to read | File |
+|---------|-------------|------|
+| Phase 1 details | When starting Phase 1 | [`references/phase-1-details.md`](references/phase-1-details.md) |
+| Phase 2 details | When starting Phase 2 | [`references/phase-2-details.md`](references/phase-2-details.md) |
+| Edge cases | When encountering an unusual input | [`references/edge-cases.md`](references/edge-cases.md) |
+| Report template | When writing the output report | [`references/report-template.md`](references/report-template.md) |
+```
+
+### What Goes in the Core vs. References
+
+**Keep in SKILL.md (always loaded):**
+- YAML frontmatter (name, description, allowed-tools)
+- When to use / when not to use
+- High-level workflow (numbered phases)
+- Decision logic (if X then do Y)
+- Section index table
+- Cross-references to other skills
+
+**Move to references/ (loaded on demand):**
+- Detailed instructions for individual phases (> 15 lines)
+- Report/output templates
+- Lookup tables, checklists, scoring rubrics
+- Domain-specific knowledge (e.g., journal formatting rules)
+- Edge case handling
+- Examples and sample outputs
+
+### Threshold Rule
+
+Apply progressive disclosure when a skill meets **any** of:
+- Total `SKILL.md` exceeds 200 lines
+- Any single section exceeds 40 lines of reference material (not workflow logic)
+- The skill has 3+ distinct phases where only 1-2 run in a typical invocation
+- The skill serves multiple domains with domain-specific branches
+
+### Skills Already Using This Pattern
+
+These skills already follow progressive disclosure:
+- `/bib-validate` — preprint check logic in `references/preprint-check.md`, report template in `references/report-template.md`
+- `/code-review` — quality rubric in `references/quality-rubric.md`
+- `/devils-advocate` — competing hypotheses framework in `references/competing-hypotheses.md`
+
+### Reference Depth
+
+Keep references **one level deep** from SKILL.md. All reference files should link directly from SKILL.md — never chain references (SKILL.md → file A → file B). Claude may partially read deeply nested files (e.g., `head -100`), resulting in incomplete information. Per [Anthropic's official best practices](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices).
+
+### When NOT to Apply
+
+- Skills under 100 lines — overhead isn't worth it
+- Skills where every section is always needed (e.g., `/session-log`)
+- Skills where the workflow is linear and every step always runs
+
+## For Existing Skills
+
+When refactoring an existing skill to use progressive disclosure:
+1. Identify sections that are > 40 lines of reference material
+2. Extract to `references/` with a descriptive filename
+3. Replace in `SKILL.md` with a one-line summary + link
+4. Add a section index table if one doesn't exist
+5. Test by invoking the skill — verify Claude reads the right sections at the right time