atris 3.1.0 → 3.5.0

package/atris/skills/endgame/SKILL.md

@@ -75,6 +75,15 @@ After running the three moves, write the result to `atris/TODO.md`:
 
 The tag must be exactly `[endgame]` (parser only matches `\w+`, no colons or hyphens). The slug lives in the section header.
 
+3. **Always append an RSI audit as the final task:**
+
+```markdown
+- **TN:** RSI audit: read this endgame's halts, verify failures, and lessons. If the loop itself broke during this endgame (parser, reward, scorecard, verify wiring), fix it. If nothing broke, no-op. [endgame]
+  **Verify:** npm test
+```
+
+This is non-negotiable. Every endgame ends by pointing the loop inward. The loop improves what it ships (RL) AND improves itself (RSI). Same chain, last task, always.
+
 3. **Each task must include a `Verify:` line** with a deterministic check:
    - **Test command:** `npm test` or `npm run test:feature`
    - **Grep pattern:** `grep -q "pattern" file.js`

package/atris/skills/improve/SKILL.md

@@ -23,7 +23,7 @@ This is the product. The thing the user pays for. One call, one verifiable resul
   → POST /api/improve { workspace: ".", mode: "full" }
   → backend picks a task, plans, builds, reviews, verifies
   → returns { task, reward, files_changed, verify_pass, summary }
-  → CLI writes scorecard to atris/scorecards.md
+  → CLI writes scorecard to .atris/presidio/scorecards.md
   → CLI reports result to user
 ```
 
@@ -45,7 +45,7 @@ The inference is Claude Code (or whatever model the backend uses). The environme
 5. On success:
    - Show what shipped (task name, files changed, verify result)
    - Show the reward score
-   - Write scorecard to `atris/scorecards.md`
+   - Write scorecard to `.atris/presidio/scorecards.md`
    - Append tick to today's journal
 6. On failure:
    - Show the error

package/atris/skills/research-search/SKILL.md

@@ -0,0 +1,167 @@
+---
+name: research-search
+description: "Fast research sweep — arxiv, semantic scholar, github, web. Finds papers, scores relevance, extracts actionable insights, stores to wiki. Triggers on: research search, find papers, latest research, arxiv, what's new in, sweep papers, research sweep."
+version: 1.0.0
+tags:
+  - research
+  - arxiv
+  - papers
+  - knowledge
+  - ingestion
+---
+
+# /research — Fast Research Sweep
+
+Find the latest research on a topic, score it for relevance, extract what you can BUILD with it, store the best finds.
+
+## Usage
+
+```
+/research <topic>                     # Sweep a topic, show top results
+/research <topic> --ingest            # Sweep + store best finds to wiki
+/research <topic> --deep <arxiv-url>  # Deep-read a specific paper
+/research --sweep                     # Run all topics from program.md
+/research --trending                  # What's hot this week in your areas
+```
+
+## On invoke
+
+### Step 0: Load the research program
+
+Read `atris/skills/research/program.md` for:
+- Active research topics (what to search for)
+- Scoring criteria (what makes a paper relevant)
+- Date window (default: last 6 months)
+- Prior results from `atris/skills/research/results.tsv`
+
+### Step 1: Multi-source search
+
+For the given topic, search ALL of these sources in parallel (use Agent tool for parallelism):
+
+**Source A — arxiv API**
+Run via Bash:
+```bash
+python3 atris/skills/research/arxiv_search.py "<topic>" --after 2025-10-01 --limit 20
+```
+Returns JSON array of papers with title, authors, abstract, date, url, categories.
+
+**Source B — Semantic Scholar API**
+Run via Bash:
+```bash
+python3 atris/skills/research/scholar_search.py "<topic>" --after 2025-10-01 --limit 20
+```
+Returns JSON array with title, authors, abstract, date, url, citation count, venue.
+
+**Source C — Web search**
+Use WebSearch tool: `"<topic>" site:arxiv.org OR site:github.com 2025..2026`
+
+**Source D — GitHub**
+Use WebSearch tool: `"<topic>" site:github.com stars:>100 pushed:>2025-10-01`
+
+### Step 2: Deduplicate and rank
+
+Merge results from all sources. Deduplicate by title similarity.
+
+For each paper, score 1-10 on:
+- **Relevance**: Does this directly apply to our research program?
+- **Recency**: Published in the target date window?
+- **Actionability**: Can we BUILD something with this? Not just theory?
+- **Novelty**: Is this a new technique, or incremental on known work?
+
+Compute total = (relevance * 3 + actionability * 3 + recency * 2 + novelty * 2) / 10
+
+### Step 3: Present results
+
+Show a ranked table:
+
+```
+# Research Sweep: <topic>
+## Date: YYYY-MM-DD | Sources: arxiv, scholar, web, github | Papers found: N
+
+| # | Score | Title | Date | Key Insight | Source |
+|---|-------|-------|------|-------------|--------|
+| 1 | 9.2   | ...   | ...  | ...         | arxiv  |
+| 2 | 8.5   | ...   | ...  | ...         | scholar|
+```
+
+For the top 5, show:
+- **One-line insight**: What's the actionable takeaway
+- **Applies to**: Which of our projects/experiments this helps
+- **Build it**: What we'd actually implement
+
+### Step 4: Deep read (optional, on request or --ingest)
+
+For papers the user selects (or top 3 if --ingest):
+
+1. Use WebFetch to read the full arxiv abstract page
+2. If PDF: note the URL for manual reading, extract what you can from abstract + related work
+3. Extract:
+   - Core technique (one paragraph)
+   - Key results (numbers, benchmarks)
+   - How to implement at inference time (if applicable)
+   - Dependencies (what you need: fine-tuning? API access? special hardware?)
+   - Limitations the authors acknowledge
+
+### Step 5: Store (if --ingest)
+
+Write each top paper to `atris/wiki/research/<slug>.md`:
+
+```markdown
+---
+title: <paper title>
+source: <arxiv/scholar/github url>
+date: <publication date>
+relevance_score: <1-10>
+last_compiled: <today>
+tags: [<topic tags>]
+---
+
+# <Paper Title>
+
+**Authors:** ...
+**Published:** ...
+**URL:** ...
+
+## Core Technique
+<one paragraph>
+
+## Key Results
+<bullet points with numbers>
+
+## How to Use (Inference-Time)
+<practical implementation notes>
+
+## Applies To
+<which of our projects benefit>
+
+## Limitations
+<what the authors say doesn't work>
+```
+
+Update `atris/wiki/index.md` with the new pages.
+
+### Step 6: Log
+
+Append to `atris/skills/research/results.tsv`:
+```
+timestamp  topic  papers_found  top_score  top_paper  source_breakdown
+```
+
+Over time, this log shows which topics are producing the best finds and which sources are most useful.
+
+## RL Integration
+
+The research program evolves:
+1. After each sweep, note which papers scored highest and from which source
+2. If a paper leads to a successful implementation (tracked via /storysim or /autoresearch), boost that topic's weight
+3. If a sweep produces nothing actionable, refine the search queries
+4. The program.md file is the "policy" — update it as you learn what works
+
+## Rules
+
+- Date filter is HARD. Do not include papers outside the configured window.
+- Actionability > novelty. A mediocre paper you can build with beats a brilliant paper you can't.
+- No summaries without sources. Every claim needs a URL.
+- Prefer papers with code (GitHub links, "code available at...").
+- Don't deep-read everything. Score first, read the top 3-5.
+- If a paper requires fine-tuning and the user only has API access, flag it clearly.

package/atris/skills/research-search/arxiv_search.py

@@ -0,0 +1,157 @@
+#!/usr/bin/env python3
+"""
+arxiv API search — returns structured JSON for papers matching a query.
+
+Uses the arxiv Atom API (no key required, free, no rate limit for reasonable use).
+
+Usage:
+    python3 arxiv_search.py "RL creative writing" --after 2025-10-01 --limit 20
+    python3 arxiv_search.py "multi-agent debate" --categories cs.AI cs.CL --limit 10
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import urllib.parse
+import urllib.request
+import xml.etree.ElementTree as ET
+from datetime import datetime
+
+
+ARXIV_API = "http://export.arxiv.org/api/query"
+ATOM_NS = "{http://www.w3.org/2005/Atom}"
+ARXIV_NS = "{http://arxiv.org/schemas/atom}"
+
+
+def search_arxiv(
+    query: str,
+    after: str | None = None,
+    categories: list[str] | None = None,
+    limit: int = 20,
+) -> list[dict]:
+    """Search arxiv API and return structured results."""
+
+    # Build search query — use AND between words for broader matching
+    # Quoting the whole phrase is too strict; split into AND-ed terms
+    terms = query.strip().split()
+    if len(terms) <= 3:
+        term_query = " AND ".join(f"all:{t}" for t in terms)
+    else:
+        # For longer queries, group into bigrams + individual key terms
+        term_query = " AND ".join(f"all:{t}" for t in terms)
+
+    search_parts = [term_query]
+    if categories:
+        cat_query = " OR ".join(f"cat:{c}" for c in categories)
+        search_parts.append(f"({cat_query})")
+
+    search_query = " AND ".join(search_parts)
+
+    params = {
+        "search_query": search_query,
+        "start": 0,
+        "max_results": min(limit, 50),  # arxiv caps at 50 per request
+        "sortBy": "submittedDate",
+        "sortOrder": "descending",
+    }
+
+    url = f"{ARXIV_API}?{urllib.parse.urlencode(params)}"
+
+    try:
+        req = urllib.request.Request(url, headers={"User-Agent": "AtrisResearch/1.0"})
+        with urllib.request.urlopen(req, timeout=30) as resp:
+            xml_data = resp.read().decode("utf-8")
+    except Exception as e:
+        print(json.dumps({"error": str(e), "papers": []}))
+        sys.exit(1)
+
+    # Parse Atom XML
+    root = ET.fromstring(xml_data)
+    entries = root.findall(f"{ATOM_NS}entry")
+
+    papers = []
+    for entry in entries:
+        # Extract fields
+        title = entry.findtext(f"{ATOM_NS}title", "").strip().replace("\n", " ")
+        abstract = entry.findtext(f"{ATOM_NS}summary", "").strip().replace("\n", " ")
+        published = entry.findtext(f"{ATOM_NS}published", "")
+        updated = entry.findtext(f"{ATOM_NS}updated", "")
+
+        # Authors
+        authors = []
+        for author in entry.findall(f"{ATOM_NS}author"):
+            name = author.findtext(f"{ATOM_NS}name", "")
+            if name:
+                authors.append(name)
+
+        # Links
+        arxiv_url = ""
+        pdf_url = ""
+        for link in entry.findall(f"{ATOM_NS}link"):
+            href = link.get("href", "")
+            link_type = link.get("type", "")
+            link_title = link.get("title", "")
+            if link_title == "pdf":
+                pdf_url = href
+            elif link_type == "text/html" or (not arxiv_url and "abs" in href):
+                arxiv_url = href
+
+        if not arxiv_url:
+            id_elem = entry.findtext(f"{ATOM_NS}id", "")
+            arxiv_url = id_elem
+
+        # Categories
+        cats = []
+        for cat in entry.findall(f"{ARXIV_NS}primary_category"):
+            term = cat.get("term", "")
+            if term:
+                cats.append(term)
+        for cat in entry.findall(f"{ATOM_NS}category"):
+            term = cat.get("term", "")
+            if term and term not in cats:
+                cats.append(term)
+
+        # Parse date
+        pub_date = published[:10] if published else ""
+
+        # Date filter
+        if after and pub_date < after:
+            continue
+
+        papers.append({
+            "title": title,
+            "authors": authors[:5],  # Cap at 5 authors
+            "abstract": abstract[:500],  # Cap abstract length
+            "date": pub_date,
+            "url": arxiv_url,
+            "pdf": pdf_url,
+            "categories": cats[:5],
+            "source": "arxiv",
+        })
+
+    return papers
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Search arxiv for papers")
+    parser.add_argument("query", help="Search query")
+    parser.add_argument("--after", help="Only papers after this date (YYYY-MM-DD)")
+    parser.add_argument("--categories", nargs="*", help="arxiv categories (e.g. cs.AI cs.CL)")
+    parser.add_argument("--limit", type=int, default=20, help="Max results")
+    args = parser.parse_args()
+
+    papers = search_arxiv(
+        query=args.query,
+        after=args.after,
+        categories=args.categories,
+        limit=args.limit,
+    )
+
+    print(json.dumps({"papers": papers, "count": len(papers), "query": args.query}))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

package/atris/skills/research-search/program.md

@@ -0,0 +1,48 @@
+# Research Program
+
+> Customize this file for your project. Add your topics, adjust the date window, define what "actionable" means for you.
+
+## Date Window
+**After:** 2025-10-01
+**Before:** 2026-12-31
+
+## Active Topics
+
+> Replace these with your own research interests. Each topic should be specific enough to produce useful search results.
+
+### 1. Example: Inference-time compute scaling
+- Best-of-N rejection sampling
+- Tree of thought / MCTS for LLMs
+- Compute-optimal allocation
+- Extended thinking for complex tasks
+
+### 2. Example: LLM-as-Judge calibration
+- Scoring bias in LLM judges
+- Pairwise vs absolute scoring reliability
+- Multi-criteria rubric design
+- Position bias, length bias, verbosity bias
+
+### 3. Example: Self-improving AI systems
+- Curiosity-driven RL (anti-mode-collapse)
+- Verbalized sampling for diversity
+- Agent self-reflection and metacognition
+- Keep/revert experiment loops
+
+## Scoring Criteria
+
+| Criterion | Weight | What it means |
+|-----------|--------|---------------|
+| Relevance | 3x | Directly applies to one of your active topics |
+| Actionability | 3x | Can you BUILD something with this using API access only (no fine-tuning)? |
+| Recency | 2x | Published within your date window |
+| Novelty | 2x | New technique, not incremental on known work |
+
+**Total = (relevance * 3 + actionability * 3 + recency * 2 + novelty * 2) / 10**
+
+## Preferences
+
+- Papers with code > papers without
+- Inference-time techniques > training-required techniques
+- Applied results > theoretical frameworks
+- Concrete numbers > vague claims
+- Short papers that say one thing well > long surveys

package/atris/skills/research-search/results.tsv

@@ -0,0 +1,6 @@
+timestamp	topic	papers_found	top_score	top_paper	source_breakdown
+2026-04-13T11:50:00Z	rl-creative-writing+self-improvement+story-coherence	30	9.2	R2-Write (arxiv:2604.03004)	arxiv:30 scholar:0 web:0
+2026-04-13T17:50:00Z	rubric-refinement+scene-rewriting	20	9.4	RRD Rubric Refinement (arxiv:2602.05125)	arxiv:10 web:10
+2026-04-13T23:10:00Z	sensory-language+embodiment	12	8.6	Zero Body Problem (arxiv:2504.06393)	web:12
+2026-04-14T04:10:00Z	scorer-variance+judge-consistency	20	9.0	Efficient Noisy LLM Judge (arxiv:2601.05420)	web:20
+2026-04-14T05:00:00Z	micro-gesture+embodied-fiction	10	6.0	none-actionable	web:10

package/atris/skills/research-search/scholar_search.py

@@ -0,0 +1,154 @@
+#!/usr/bin/env python3
+"""
+Semantic Scholar API search — returns structured JSON for papers.
+
+Uses the Semantic Scholar Academic Graph API (free, no key required for basic use,
+rate limited to 100 requests/5 min without key).
+
+Usage:
+    python3 scholar_search.py "reinforcement learning creative writing" --after 2025-10-01 --limit 20
+    python3 scholar_search.py "LLM self-play" --min-citations 5
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import urllib.parse
+import urllib.request
+import time
+
+
+S2_API = "https://api.semanticscholar.org/graph/v1/paper/search"
+S2_FIELDS = "title,authors,abstract,year,publicationDate,externalIds,citationCount,venue,openAccessPdf,url"
+
+
+def search_scholar(
+    query: str,
+    after: str | None = None,
+    limit: int = 20,
+    min_citations: int = 0,
+) -> list[dict]:
+    """Search Semantic Scholar and return structured results."""
+
+    # Build year filter
+    year_filter = ""
+    if after:
+        start_year = after[:4]
+        year_filter = f"{start_year}-"
+
+    params = {
+        "query": query,
+        "limit": min(limit, 100),
+        "fields": S2_FIELDS,
+    }
+    if year_filter:
+        params["year"] = year_filter
+
+    url = f"{S2_API}?{urllib.parse.urlencode(params)}"
+
+    try:
+        req = urllib.request.Request(url, headers={
+            "User-Agent": "AtrisResearch/1.0",
+            "Accept": "application/json",
+        })
+        with urllib.request.urlopen(req, timeout=30) as resp:
+            data = json.loads(resp.read().decode("utf-8"))
+    except urllib.error.HTTPError as e:
+        if e.code == 429:
+            # Rate limited — wait and retry once
+            time.sleep(5)
+            try:
+                with urllib.request.urlopen(req, timeout=30) as resp:
+                    data = json.loads(resp.read().decode("utf-8"))
+            except Exception as e2:
+                print(json.dumps({"error": f"Rate limited: {e2}", "papers": []}))
+                sys.exit(1)
+        else:
+            print(json.dumps({"error": f"HTTP {e.code}: {e.reason}", "papers": []}))
+            sys.exit(1)
+    except Exception as e:
+        print(json.dumps({"error": str(e), "papers": []}))
+        sys.exit(1)
+
+    results = data.get("data", [])
+    papers = []
+
+    for item in results:
+        if not item:
+            continue
+
+        title = (item.get("title") or "").strip()
+        if not title:
+            continue
+
+        # Authors
+        authors = []
+        for author in (item.get("authors") or [])[:5]:
+            name = author.get("name", "")
+            if name:
+                authors.append(name)
+
+        abstract = (item.get("abstract") or "")[:500]
+        pub_date = item.get("publicationDate") or ""
+        year = item.get("year") or ""
+        citations = item.get("citationCount") or 0
+        venue = item.get("venue") or ""
+
+        # URL
+        paper_url = item.get("url") or ""
+        external_ids = item.get("externalIds") or {}
+        arxiv_id = external_ids.get("ArXiv")
+        if arxiv_id:
+            paper_url = f"https://arxiv.org/abs/{arxiv_id}"
+
+        # PDF
+        pdf_info = item.get("openAccessPdf") or {}
+        pdf_url = pdf_info.get("url") or ""
+
+        # Date filter
+        date_str = pub_date[:10] if pub_date else (str(year) if year else "")
+        if after and date_str and date_str < after:
+            continue
+
+        # Citation filter
+        if citations < min_citations:
+            continue
+
+        papers.append({
+            "title": title,
+            "authors": authors,
+            "abstract": abstract,
+            "date": date_str,
+            "url": paper_url,
+            "pdf": pdf_url,
+            "citations": citations,
+            "venue": venue,
+            "source": "semantic_scholar",
+        })
+
+    return papers
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Search Semantic Scholar for papers")
+    parser.add_argument("query", help="Search query")
+    parser.add_argument("--after", help="Only papers after this date (YYYY-MM-DD)")
+    parser.add_argument("--limit", type=int, default=20, help="Max results")
+    parser.add_argument("--min-citations", type=int, default=0, help="Minimum citation count")
+    args = parser.parse_args()
+
+    papers = search_scholar(
+        query=args.query,
+        after=args.after,
+        limit=args.limit,
+        min_citations=args.min_citations,
+    )
+
+    print(json.dumps({"papers": papers, "count": len(papers), "query": args.query}))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())