@diottodev/runic 1.0.0 → 1.1.0

package/engineering/llm-wiki/skills/llm-wiki/references/lint-workflow.md

@@ -0,0 +1,91 @@
+# Lint Workflow
+
+Periodic health-check the LLM runs when the user runs `/wiki-lint` or dispatches the `wiki-linter` sub-agent. Run this at least weekly, and always after a batch ingest.
+
+## Goal
+
+Keep the wiki healthy as it grows. Surface problems for the user to review.
+
+## Pass 1 — mechanical checks (script)
+
+Run `python scripts/lint_wiki.py --vault .` to get a report on:
+
+- **Orphans** — pages with zero inbound `[[wikilinks]]`
+- **Broken links** — wikilinks pointing to non-existent pages
+- **Stale pages** — pages whose `updated:` frontmatter is older than 90 days (tune via `--stale-days`)
+- **Missing frontmatter** — pages lacking `title`, `category`, or `summary`
+- **Duplicate titles** — two or more pages sharing the same title
+- **Log gap** — no log entry in the last 14 days (tune via `--log-gap-days`)
+
+Run `python scripts/graph_analyzer.py --vault .` for structural stats:
+
+- Hubs (inbound/outbound) — likely well-placed
+- Sinks — pages that don't link out; may need cross-referencing
+- Connected components — if > 1, parts of the wiki are disconnected islands
+
+## Pass 2 — semantic checks (LLM)
+
+The script can't catch these. The LLM must read and think.
+
+### A. Contradictions
+
+Scan pages whose `updated:` is recent. For each, check whether it contradicts any existing page. If so:
+- Add a `> ⚠️ Contradiction:` callout to both pages
+- Log with `op: note`
+- Surface to user: "I found a potential contradiction between X and Y. Want me to investigate?"
+
+### B. Stale claims
+
+For each flagged stale page, ask:
+- Does a newer source now contradict this?
+- Is a "Key facts" bullet likely to be outdated (person changed role, company pivoted, etc.)?
+- If yes, suggest to user: "Page X says Y. This may be outdated — do you want me to search for newer sources?"
+
+### C. Concepts mentioned but without their own page
+
+Grep for common patterns: `[[concept:xxx]]`, phrases like "see also", concept-shaped nouns mentioned across 3+ pages but with no dedicated page.
+
+Suggest new pages to create.
+
+### D. Cross-reference gaps
+
+For each page, check: do all entities and concepts mentioned have wikilinks? If a concept is referenced as plain text in 3+ places, promote it to a wikilink (and create a stub page if needed).
+
+### E. Index drift
+
+Compare `index.md` against actual `wiki/` contents. If out of sync, either regenerate (`update_index.py`) or patch inline.
+
+## Pass 3 — report
+
+Present findings to the user as a single markdown report:
+
+```markdown
+# Wiki lint — 2026-04-10
+
+**Total pages:** 87  **Components:** 1  **Last log:** 2026-04-09
+
+## Found
+- ⚠️ 3 contradictions (wiki/concepts/x, wiki/sources/y, wiki/sources/z)
+- 12 orphan pages (mostly new entities)
+- 2 broken links (wiki/concepts/x → [[foo-bar]] no such page)
+- 4 stale pages (>90 days, no re-ingest)
+- 5 concepts mentioned across 3+ pages without their own page
+
+## Suggested actions
+1. Investigate contradiction between [[sources/a]] and [[sources/b]]
+2. Create concept page for "attention masking" (mentioned in 4 sources)
+3. Re-ingest [[sources/c]] — stale and contradicted by newer sources
+4. Fix broken link in [[concepts/x]]
+5. Cross-reference the 12 orphans (most belong under [[synthesis/overview]])
+
+Want me to run these in order, or pick specific ones?
+```
+
+Append a `lint` entry to `log.md` summarizing what was found and what was fixed.
+
+## Frequency
+
+- **Weekly** — light pass, script-only (`lint_wiki.py` + quick review)
+- **After batch ingests** — always
+- **Monthly** — full pass including semantic checks
+- **Before sharing the wiki** — full pass plus an extra review

package/engineering/llm-wiki/skills/llm-wiki/references/memex-principles.md

@@ -0,0 +1,78 @@
+# Memex Principles
+
+Why the LLM Wiki pattern works, and why it failed for humans until LLMs.
+
+## Vannevar Bush's Memex (1945)
+
+In "As We May Think", Bush described a personal knowledge store where:
+
+- Documents are curated, not just searched
+- Users build **associative trails** — named, reusable paths through the material
+- The trails are as valuable as the documents
+- The system is private and personal, not a public reference
+
+This is almost exactly the LLM Wiki pattern. The difference: Bush had no one to do the bookkeeping.
+
+## Why humans abandon wikis
+
+The value of a wiki grows linearly with its size. The maintenance burden grows faster. At some inflection point — usually around 50-100 pages — maintenance starts to feel like chores and the wiki goes stale.
+
+Specific tasks that die first:
+
+- Updating cross-references when a new page is added
+- Keeping summary pages current
+- Noticing when new data contradicts old claims
+- Consolidating pages that have drifted apart
+- Filing explorations back into the knowledge base
+- Keeping the index current
+
+Humans are great at reading, curating, and thinking about what things mean. They're bad at the bookkeeping. The bookkeeping is 80% of the work.
+
+## What changed with LLMs
+
+LLMs don't get bored. They don't forget to update a cross-reference. They can touch 15 files in one pass without losing track. They cost near-zero per maintenance operation.
+
+This changes the economics. The wiki stays maintained because maintenance is now free (or nearly). The human's job collapses to:
+
+- **Source curation** — deciding what's worth reading
+- **Direction** — asking good questions, steering analysis
+- **Judgment** — deciding when a contradiction matters
+- **Taste** — knowing when the synthesis is wrong
+
+Everything else — the 80% that killed human wikis — is delegated.
+
+## Why not just RAG?
+
+RAG retrieves fragments at query time and synthesizes from scratch every query. It works, but:
+
+- **No accumulation.** Every subtle question re-derives the same synthesis.
+- **Cross-references are computed on demand.** If the cross-reference needs 5 sources to be visible, you'd better hope all 5 are in the retrieval window.
+- **Contradictions are invisible.** They surface only if you explicitly ask "is there a contradiction?"
+- **Explorations disappear.** A comparison you worked out yesterday has to be re-derived tomorrow.
+
+The wiki fixes all of these by **compiling the knowledge once**. The cross-references are already there. The contradictions have been flagged. The synthesis has absorbed everything read so far.
+
+RAG is retrieve-then-think. The wiki is think-once-retrieve-many.
+
+## When the wiki stops being enough
+
+At ~500-1000 pages, the index approach starts to creak. Options:
+
+1. **Layer on search.** Add `wiki_search.py` (BM25) or an external tool like [qmd](https://github.com/tobi/qmd) (hybrid BM25 + vector). Both work alongside the index.
+2. **Shard by topic.** Split into multiple vaults by domain.
+3. **Add an MCP retrieval layer.** Expose the wiki as a tool so agents can query it structurally.
+
+The wiki and RAG are not opposites. The wiki is a **compiled layer above RAG**. You can run RAG on top of the wiki (indexing `wiki/`) and you'll get the benefits of both: pre-synthesized knowledge + scalable retrieval.
+
+## The human role
+
+A common failure mode: users delegate curation to the LLM ("just ingest all my Pocket articles"). Don't. Curation is where human judgment lives. The LLM can help you decide *whether* to read a source, but you pick what makes it into `raw/`.
+
+If you let the LLM ingest everything, the wiki fills with low-signal summaries and the synthesis becomes meaningless. The wiki's value is a direct function of the quality of `raw/`.
+
+## Reading recommendations
+
+- Vannevar Bush, "As We May Think" (Atlantic, 1945)
+- Andrej Karpathy's original LLM Wiki gist (linked from SKILL.md)
+- Ousterhout's *A Philosophy of Software Design* — for why "deep modules" (well-summarized pages) beat shallow ones
+- Niklas Luhmann's Zettelkasten — an earlier manual version of the same pattern

package/engineering/llm-wiki/skills/llm-wiki/references/obsidian-setup.md

@@ -0,0 +1,85 @@
+# Obsidian Setup
+
+Recommended Obsidian configuration for an LLM Wiki vault. None of this is strictly required — the wiki is just markdown files — but these settings remove friction.
+
+## Open the vault
+
+1. Obsidian → "Open folder as vault" → pick your initialized vault
+2. The vault already has `wiki/`, `raw/`, `CLAUDE.md`, `AGENTS.md`
+
+## Settings → Files and Links
+
+- **Default location for new notes:** `wiki/`
+- **New link format:** `Shortest path when possible` (keeps wikilinks clean)
+- **Use `[[Wikilinks]]`:** ON
+- **Attachment folder path:** `raw/assets/` (so clipped images land in `raw/`, not `wiki/`)
+- **Automatically update internal links:** ON
+
+## Settings → Hotkeys
+
+Search for and bind:
+- **"Download attachments for current file"** → `Ctrl/Cmd + Shift + D`
+- **"Open graph view"** → `Ctrl/Cmd + G`
+
+## Core plugins to enable
+
+- **Graph view** — see the shape of your wiki. Hubs, orphans, clusters.
+- **Backlinks** — pane showing who links to the current page. Critical for browsing.
+- **Outgoing links** — complementary pane.
+- **Templates** — enable and set the template folder to `wiki/.templates`
+- **Tag pane** — tag-driven navigation
+- **Search** — obviously
+- **Page preview** — hover a wikilink to preview
+- **Canvas** — visual exploration, useful for synthesis work
+
+## Recommended community plugins
+
+- **Obsidian Web Clipper** (browser extension, not a plugin) — clip articles to `raw/articles/` as markdown
+- **Dataview** — query over frontmatter. Dynamic tables of "all concept pages touched by 3+ sources".
+- **Marp for Obsidian** — render any markdown with `marp: true` frontmatter as a slide deck inside Obsidian. Pairs with `scripts/export_marp.py`.
+- **Templater** — dynamic templates (optional, you can use the LLM for this)
+- **Advanced Tables** — easier markdown table editing
+- **Git** — commit on save, or hook into system git
+
+## Dataview examples
+
+Pages with 3+ sources:
+```dataview
+table updated, sources
+from "wiki/concepts"
+where sources >= 3
+sort updated desc
+```
+
+Recently updated synthesis pages:
+```dataview
+list
+from "wiki/synthesis"
+sort updated desc
+limit 10
+```
+
+Orphans (Dataview can't see inbound links — use the lint script for this).
+
+## Git workflow
+
+```bash
+cd <vault>
+git init
+git add .
+git commit -m "init wiki"
+
+# After every session:
+git add wiki/ log.md index.md
+git commit -m "ingest: <source>"
+```
+
+The vault is a plain markdown repo. Version history, branching, collaboration — free.
+
+## Tips
+
+- **Use the graph view daily** — it's the fastest way to see structural drift
+- **Pin `index.md`, `log.md`, and the active `synthesis/` page** to the sidebar tabs
+- **Split view** — wiki on the left, chat/CLI on the right. You browse while the LLM edits.
+- **Enable "strict line breaks"** so your LLM's markdown renders the way the LLM expects
+- **Use images aggressively** — download them locally, reference from pages. The LLM can read them with its vision tool when needed.

package/engineering/llm-wiki/skills/llm-wiki/references/page-formats.md

@@ -0,0 +1,194 @@
+# Page Formats
+
+Every wiki page has the same skeleton: YAML frontmatter + a section structure that matches its category. Below are the five canonical formats. Templates live in `assets/page-templates/`.
+
+## 1. Entity page
+
+For a person, organization, place, product, or dataset.
+
+```markdown
+---
+title: Anthropic
+category: entity
+summary: AI safety company, developer of Claude; major contributor to interpretability research
+tags: [company, ai-safety, anthropic]
+sources: 4
+updated: 2026-04-10
+---
+
+# Anthropic
+
+## What it is
+One-paragraph definition. What kind of entity, founded when, by whom, active in what.
+
+## Why it matters (to this wiki)
+Why this entity shows up across sources. What role does it play in the narrative?
+
+## Key facts
+- Founded YYYY by [[people]]
+- Known for [[concepts]]
+- Related [[entities]]
+
+## Appears in
+- [[sources/monosemanticity]] — primary work on sparse autoencoders
+- [[sources/constitutional-ai]] — alignment methodology
+- [[concepts/rlhf]] — contributor to training method
+
+## Open questions
+- Questions the sources don't yet answer; good prompts for new source hunts.
+```
+
+## 2. Concept page
+
+For an idea, theory, method, framework.
+
+```markdown
+---
+title: Sparse Autoencoder
+category: concept
+summary: Dictionary-learning method for decomposing polysemantic neurons into monosemantic features
+tags: [interpretability, sparse-autoencoders, dictionary-learning]
+sources: 3
+updated: 2026-04-10
+---
+
+# Sparse Autoencoder
+
+## Definition
+Precise, one-paragraph definition. The canonical form used across your sources.
+
+## Origin
+Who proposed it, when, in what paper/context. Link [[entities]] and [[sources]].
+
+## Key claims
+- Claim 1 — cited from [[sources/xxx]]
+- Claim 2 — cited from [[sources/yyy]]
+
+## Contrasts with
+- [[concepts/probing]] — see [[comparisons/sae-vs-probing]]
+
+## Open questions / disagreements
+- Unresolved questions across sources.
+- ⚠️ Contradiction: [[sources/a]] claims X but [[sources/b]] claims ~X.
+
+## Used in
+- [[synthesis/interpretability-overview]]
+```
+
+## 3. Source summary page
+
+One per ingested source. This is the **single place the raw source's content is summarized**; other pages cite it.
+
+```markdown
+---
+title: "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning"
+category: source
+summary: Anthropic 2024 paper using sparse autoencoders to extract interpretable features from a one-layer transformer
+tags: [interpretability, sparse-autoencoders, anthropic]
+source_path: raw/papers/monosemanticity.pdf
+source_date: 2024-10
+authors: [Bricken et al.]
+ingested: 2026-04-10
+updated: 2026-04-10
+---
+
+# Towards Monosemanticity
+
+## TL;DR
+Two sentences max. What did they do, what did they find.
+
+## Key claims
+1. Claim, with a page/section pointer if available
+2. ...
+
+## Methods
+How the work was done. Data, model, training, evaluation.
+
+## Evidence cited
+- Figure 3 shows ...
+- Table 1 ...
+
+## Surprises / contradictions
+- Where this source conflicts with [[sources/other]] or [[concepts/xxx]].
+
+## Connections
+- Extends [[concepts/sparse-autoencoder]]
+- Builds on [[entities/anthropic-interpretability-team]]'s prior work
+- Related: [[sources/superposition-2022]]
+
+## Where it's cited
+Pages in this wiki that cite this source:
+- [[concepts/sparse-autoencoder]]
+- [[entities/anthropic]]
+- [[synthesis/interpretability-overview]]
+```
+
+## 4. Comparison page
+
+For explicit cross-source or cross-concept analysis.
+
+```markdown
+---
+title: "SAE vs Probing"
+category: comparison
+summary: How sparse autoencoders differ from linear probing as interpretability methods
+tags: [interpretability, comparison]
+sources: 4
+updated: 2026-04-10
+---
+
+# Sparse Autoencoders vs Linear Probes
+
+## What they share
+Both look for human-interpretable structure inside trained models.
+
+## Where they diverge
+| Dimension | SAE | Probe |
+|---|---|---|
+| Supervision | unsupervised | supervised |
+| Output | dictionary of features | single-label classifier |
+| Scalability | model-dependent | cheap |
+| Typical use | feature discovery | feature verification |
+
+## Which sources take which side
+- [[sources/monosemanticity]] — pro-SAE
+- [[sources/probing-survey]] — pro-probes
+
+## Open questions
+- When should you prefer one over the other?
+```
+
+## 5. Synthesis page
+
+High-level views that draw on many sources and concepts.
+
+```markdown
+---
+title: Interpretability Overview
+category: synthesis
+summary: The field of interpretability research — goals, methods, open problems, key players
+tags: [interpretability, overview]
+sources: 12
+updated: 2026-04-10
+---
+
+# Interpretability Overview
+
+## Thesis
+Two or three sentences capturing the current synthesis across all sources read so far. Revised as new sources come in.
+
+## The landscape
+- Sub-area A — pursued by [[entities]], papers [[sources]]
+- Sub-area B — ...
+
+## Current open problems
+Short list with [[concepts]] and [[sources]] pointers.
+
+## How this synthesis has changed
+- **2026-04-10** — added [[sources/monosemanticity]]; shifted emphasis toward SAE.
+- **2026-03-28** — initial synthesis after first 5 sources.
+
+## Related
+- [[synthesis/alignment-overview]]
+- [[comparisons/mechinterp-vs-behavioral-interp]]
+```

package/engineering/llm-wiki/skills/llm-wiki/references/query-workflow.md

@@ -0,0 +1,76 @@
+# Query Workflow
+
+The flow the LLM follows when the user runs `/wiki-query <question>` or dispatches the `wiki-librarian` sub-agent.
+
+## Core principle
+
+**Read `index.md` first, then drill in.** Do NOT grep the entire wiki on every query — the index is there precisely so you don't have to.
+
+## Step-by-step
+
+### 1. Read `index.md`
+
+The index is the catalog. Scan it and pick the 3-10 pages most likely to contain the answer. Pick across categories: a good query usually pulls from `synthesis/` for the big picture, `concepts/` for definitions, `sources/` for evidence, and `entities/` for context.
+
+### 2. Read the picked pages
+
+Read them in full. These are short, curated, and already cross-referenced. The wiki has done the hard work for you.
+
+### 3. Follow wikilinks opportunistically
+
+If a read page points to another page that's clearly relevant, follow it. Don't follow blindly — stop when you have enough.
+
+### 4. Fall back to search if needed
+
+If the index doesn't surface the right page, use:
+
+```bash
+python scripts/wiki_search.py --vault . --query "<terms>" --limit 5
+```
+
+BM25 search over wiki pages. Standard library only. Use when:
+- The index is stale (flag this to the user — it means lint time)
+- The user asks about something niche that doesn't have its own page yet
+- You're doing a sweeping search across many pages
+
+### 5. Synthesize the answer
+
+Compose the answer as:
+- A direct answer in 1-3 sentences
+- Supporting detail, organized thematically
+- **Inline citations** using wikilinks to source pages: `[[sources/monosemanticity]]`
+- **A "Related pages" section** at the end with 3-5 wikilinks
+
+### 6. Offer to re-file
+
+**Every good answer is a candidate wiki page.** At the end of the answer, ask:
+
+> _Should I file this as a new page in the wiki? Suggested location:
+> `wiki/comparisons/sae-vs-probing.md` — or I can append it to an existing page._
+
+If the user says yes:
+- Pick the right category (most often `comparisons/` or `synthesis/`)
+- Use the appropriate template
+- Add frontmatter with `category`, `summary`, `sources` (count of cited sources), `updated`
+- Update `index.md`
+- Append to `log.md` with `op: create` and the question as the title
+
+This is how the wiki compounds — explorations don't disappear into chat history.
+
+## Output formats
+
+Not every query wants a markdown answer. Offer the user:
+
+- **Markdown page** (default) — filed back as a wiki page
+- **Comparison table** — for "A vs B" questions
+- **Marp slide deck** — via `python scripts/export_marp.py` on the synthesis page
+- **Chart (matplotlib)** — for data-driven questions; save to `wiki/assets/charts/`
+- **Obsidian Canvas** — for visual exploration (JSON format, stored at `wiki/canvases/`)
+
+## Anti-patterns
+
+- ❌ Read every page in the wiki on every query → use the index
+- ❌ Answer without citations → every claim must link to a page
+- ❌ Create a new page for a one-off trivial question → only file back answers worth keeping
+- ❌ Invent content not in the wiki → if you don't know, say so and suggest a new source to ingest
+- ❌ Skip the `log.md` entry when filing an answer back

package/engineering/llm-wiki/skills/llm-wiki/references/wiki-schema.md

@@ -0,0 +1,98 @@
+# Wiki Schema
+
+The vault has three layers. The LLM must respect the boundaries.
+
+## Layout
+
+```
+<vault>/
+├── raw/                        # IMMUTABLE sources (you own)
+│   ├── articles/*.md           # Obsidian Web Clipper output
+│   ├── papers/*.pdf
+│   ├── notes/*.md              # your own notes, journal entries
+│   └── assets/                 # images downloaded by Obsidian
+├── wiki/                       # LLM-owned knowledge base
+│   ├── index.md                # content catalog — updated every ingest
+│   ├── log.md                  # append-only timeline
+│   ├── entities/               # people, orgs, places, products
+│   ├── concepts/               # ideas, theories, frameworks, methods
+│   ├── sources/                # one summary page per ingested source
+│   ├── comparisons/            # cross-source analysis / contrasts
+│   ├── synthesis/              # high-level theses, overviews
+│   └── .templates/             # page templates (reference only, not indexed)
+├── CLAUDE.md                   # schema file for Claude Code
+├── AGENTS.md                   # same schema for Codex/Cursor/Antigravity
+└── .cursorrules                # (optional) Cursor legacy
+```
+
+## Iron rules
+
+1. **`raw/` is immutable.** The LLM reads from `raw/` but never writes to it. Never rename, never delete, never edit. If a source is wrong, the user edits it.
+2. **All LLM writes go to `wiki/`.** No exceptions.
+3. **Every ingest updates 5 files minimum:** the new source summary, the relevant entity/concept pages, `index.md`, `log.md`. A rich ingest touches 10-15.
+4. **Every wiki page carries YAML frontmatter.** Without frontmatter, `update_index.py` and `lint_wiki.py` can't see it.
+
+## Required page frontmatter
+
+```yaml
+---
+title: Mechanistic Interpretability
+category: concept            # entity | concept | source | comparison | synthesis
+summary: Reverse-engineering neural networks into human-understandable circuits
+tags: [interpretability, circuits, anthropic]
+sources: 3                   # optional — number of sources touching this page
+updated: 2026-04-10          # LLM updates this on every edit
+---
+```
+
+Allowed `category` values: `entity`, `concept`, `source`, `comparison`, `synthesis`.
+
+## Naming conventions
+
+- **Filenames:** `kebab-case.md` — lowercase, hyphens, no spaces
+- **Entities:** `entities/<kebab-case-name>.md` — e.g. `entities/chris-olah.md`
+- **Concepts:** `concepts/<kebab-case-name>.md` — e.g. `concepts/sparse-autoencoder.md`
+- **Sources:** `sources/<short-slug>.md` — e.g. `sources/monosemanticity.md`
+- **Comparisons:** `comparisons/<topic-a>-vs-<topic-b>.md`
+- **Synthesis:** `synthesis/<topic>-overview.md` or `synthesis/<topic>-thesis.md`
+
+## Linking
+
+Use Obsidian wikilinks. Three forms:
+
+```
+[[concepts/sparse-autoencoder]]                           # full path
+[[concepts/sparse-autoencoder|sparse autoencoders]]       # custom display text
+[[sparse-autoencoder]]                                    # stem — resolves if unique
+```
+
+The linter resolves stem links by matching against filenames. Prefer full paths when ambiguous.
+
+## Cross-reference rules
+
+- **Every entity mentioned in a concept/source page must be a wikilink.** If the entity page doesn't exist yet, create it.
+- **Every concept mentioned in a source summary must be a wikilink.** Same rule.
+- **Contradictions get flagged inline** with a `> ⚠️ Contradiction:` callout, and the source pages that disagree are linked from the callout.
+- **Synthesis pages link back to every concept and source they draw on.**
+
+## Index discipline
+
+`wiki/index.md` is regenerated, not hand-edited. Either:
+
+- Run `python scripts/update_index.py --vault .` after every ingest, OR
+- Have the LLM rewrite the relevant section inline.
+
+The index groups pages by `category`, alphabetized by title. Each entry is one line with a wikilink, summary, and optional metadata.
+
+## Log discipline
+
+`wiki/log.md` is append-only. Every entry starts with a standardized header so `grep "^## \[" log.md | tail -5` returns the last 5 entries.
+
+```
+## [2026-04-10] ingest | Anthropic Monosemanticity
+Added sources/monosemanticity.md. Updated concepts/sparse-autoencoder,
+concepts/polysemanticity, entities/anthropic-interpretability-team. Flagged
+contradiction with sources/distributed-representations on feature basis claim.
+```
+
+Valid ops: `ingest`, `query`, `lint`, `create`, `update`, `delete`, `note`.