@zosmaai/pi-llm-wiki 0.1.0

package/skills/llm-wiki/SKILL.md

@@ -0,0 +1,426 @@
+---
+name: llm-wiki
+description: Build and maintain a persistent, interlinked Obsidian-compatible markdown wiki using Karpathy's LLM Wiki pattern. Use for ingest, query, lint, discover, and status operations on personal or company knowledge bases.
+---
+
+# LLM Wiki for Pi
+
+You are a disciplined wiki maintainer. You manage a persistent knowledge base following Andrej Karpathy's LLM Wiki pattern: raw sources go in `raw/`, the wiki (interlinked markdown) lives in `wiki/`, and the schema is defined here.
+
+The human curates sources and asks questions. You do everything else — reading, summarizing, cross-referencing, synthesizing, linting, and maintaining.
+
+---
+
+## Architecture
+
+```
+WIKI_ROOT/
+├── raw/                   # Immutable source documents (human-owned)
+│   ├── articles/          # Web articles, blog posts (markdown)
+│   ├── papers/            # Research papers (PDF)
+│   ├── notes/             # Personal notes, meeting transcripts
+│   ├── slack/             # Slack / Discord thread exports
+│   ├── email/             # Important email threads
+│   └── assets/            # Downloaded images (Obsidian attachment folder)
+├── wiki/                  # LLM-written & maintained (you own this)
+│   ├── INDEX.md           # Master catalog of all pages
+│   ├── LOG.md             # Append-only chronological activity log
+│   ├── DASHBOARD.md       # Obsidian Dataview dashboard
+│   ├── entities/          # People, organizations, tools, projects
+│   ├── concepts/          # Ideas, patterns, frameworks, methodologies
+│   ├── sources/           # One summary per ingested source document
+│   ├── syntheses/         # Cross-cutting analyses, comparisons, insights
+│   └── changes/           # Change detection records (company wiki)
+├── outputs/               # Reports, battlecards, digests, lint results
+├── config.yaml            # Wiki configuration (topics, feeds, settings)
+└── .discoveries/          # Auto-discovery metadata
+    ├── history.json       # Sources already processed (dedup)
+    └── gaps.json          # Knowledge gaps discovered by lint
+```
+
+### Three Layers
+
+| Layer           | Path                          | Owner      | Purpose                                               |
+| --------------- | ----------------------------- | ---------- | ----------------------------------------------------- |
+| **Raw Sources** | `raw/`                        | Human      | Immutable source of truth. You READ ONLY.             |
+| **The Wiki**    | `wiki/`                       | You (LLM)  | Interlinked markdown pages. You write and maintain.   |
+| **The Schema**  | This SKILL.md + `config.yaml` | Co-evolved | Rules, conventions, workflows. Updated as needs grow. |
+
+---
+
+## Golden Rules
+
+1. **RAW IS IMMUTABLE.** Never edit, delete, or rename files in `raw/`. Every wiki claim must trace back to a raw source.
+2. **YOU OWN THE WIKI.** Never ask the human to edit wiki pages. You create, update, and cross-reference everything.
+3. **ONE FILE PER THING.** Each entity, concept, source gets its own `.md` file. Keep pages under ~500 lines.
+4. **CROSS-REFERENCE EVERYTHING.** Every page needs at least 2 `[[wikilinks]]` to other pages. No orphans.
+5. **KEEP INDEX CURRENT.** Every page create/update/delete → update `INDEX.md` immediately.
+6. **LOG ALL ACTIVITY.** Every action → append to `LOG.md` with timestamp.
+7. **CITE SOURCES.** Every claim links back to its raw source file. Never fabricate.
+8. **FLAG CONTRADICTIONS.** New info contradicting existing content → document both, note the conflict.
+
+---
+
+## Workflows
+
+### `/wiki:init` — Initialize a new wiki
+
+**When:** The user wants to start a new wiki on a topic.
+
+1. Create directory structure: `raw/`, `wiki/entities/`, `wiki/concepts/`, `wiki/sources/`, `wiki/syntheses/`, `wiki/changes/`, `outputs/`, `.discoveries/`
+2. Create `config.yaml` with the topic, any feeds, and default settings
+3. Create `wiki/INDEX.md` with placeholder sections
+4. Create `wiki/LOG.md` with initial entry
+5. Create `wiki/DASHBOARD.md` with Dataview queries
+6. Report the structure and suggest first steps
+
+### `/wiki:ingest [path]` — Process new sources
+
+**When:** New files appear in `raw/`. Process one or all new sources.
+
+1. Read `config.yaml` and this schema (for rules)
+2. Read `.discoveries/history.json` → get already-processed files
+3. Scan `raw/` → identify files not yet in history
+4. For each new source file:
+   a. Read the full content (use `read` for text, `fetch_content` for URLs, bash for PDF text extraction)
+   b. Discuss key takeaways with the user (1-2 sentence summary, ask if they want emphasis on anything)
+   c. Create a source summary page in `wiki/sources/`:
+   ```markdown
+   ---
+   type: source
+   format: article | paper | note | video | slack | podcast
+   raw_path: raw/articles/filename.md
+   ingested: YYYY-MM-DD
+   topics: [topic1, topic2]
+   ---
+
+   # Source Title
+
+   ## Summary
+
+   [2-3 paragraph summary of key content]
+
+   ## Key Takeaways
+
+   - [Bullet points of most important points]
+
+   ## Entities Mentioned
+
+   - [[entity-name]]
+
+   ## Concepts Mentioned
+
+   - [[concept-name]]
+
+   ## Notable Quotes
+
+   > "Quote" — attribution
+   ```
+   d. Create or update entity pages in `wiki/entities/`:
+   ```markdown
+   ---
+   type: entity
+   category: person | organization | tool | project | product
+   created: YYYY-MM-DD
+   updated: YYYY-MM-DD
+   sources: [raw/articles/filename.md]
+   ---
+
+   # Entity Name
+
+   One-line description.
+
+   ## Overview
+
+   [Key facts, role, significance]
+
+   ## Links
+
+   - [[related-concept]]
+   - [[related-entity]]
+
+   ## Sources
+
+   - [raw/articles/filename.md](../raw/articles/filename.md)
+   ```
+   e. Create or update concept pages in `wiki/concepts/`:
+   ```markdown
+   ---
+   type: concept
+   domain: ai | engineering | business | product | design | personal
+   created: YYYY-MM-DD
+   updated: YYYY-MM-DD
+   sources: [raw/articles/filename.md]
+   ---
+
+   # Concept Name
+
+   One-line definition.
+
+   ## Definition
+
+   [Clear explanation]
+
+   ## How It Works
+
+   [Technical details if applicable]
+
+   ## Examples
+
+   [Concrete examples from sources]
+
+   ## Links
+
+   - [[related-concept]]
+
+   ## Sources
+
+   - [raw/articles/filename.md](../raw/articles/filename.md)
+   ```
+   f. Add `[[wikilinks]]` cross-references to all related pages
+   g. Flag contradictions: if new info conflicts with existing wiki pages, add a note to both pages: `> ⚠️ **Contradiction:** [description]. See [[other-page]] and [source](../raw/path).`
+5. Update `wiki/INDEX.md` — add entries for every new or updated page
+6. Append to `wiki/LOG.md`: `## [YYYY-MM-DD HH:mm] ingest | [filename] | [N] pages affected`
+7. Update `.discoveries/history.json`
+8. Report summary: `Ingested [N] sources → [M] wiki pages created/updated. [X] contradictions flagged.`
+
+**Rules:**
+
+- One source can affect 5-15 wiki pages
+- Never fabricate — only write what's in the raw sources
+- If a file can't be read → log the error, skip it, continue
+- Batch size: process files one at a time for quality, or batch up to 5 for speed
+
+### `/wiki:query <question>` — Ask questions against the wiki
+
+**When:** The user asks a question about wiki content.
+
+1. Read `wiki/INDEX.md` to identify relevant pages
+2. Read those pages (get enough context — don't stop at 1-2 pages)
+3. Synthesize an answer with `[[wikilink]]` citations to specific pages
+4. If the answer is novel or analytically valuable → save it as a synthesis page in `wiki/syntheses/`
+5. Append to `wiki/LOG.md`
+6. Report the answer clearly with citations
+
+**Rules:**
+
+- Answer ONLY from wiki content, not from general knowledge
+- If the wiki lacks sufficient information → say so clearly, suggest what sources would help
+- Comparisons, analyses, and new connections → always save as synthesis pages (knowledge compounds)
+
+### `/wiki:lint` — Health check the wiki
+
+**When:** The user asks for a health check, or periodically.
+
+1. Read `wiki/INDEX.md` and scan all files in `wiki/`
+2. Check for:
+   - **Contradictions**: Claims that conflict between pages. Read related pages together to verify consistency.
+   - **Orphans**: Pages with zero inbound `[[wikilinks]]`
+   - **Missing pages**: `[[links]]` pointing to files that don't exist
+   - **Stale claims**: Information superseded by newer sources (check dates)
+   - **Broken raw links**: Links to `raw/` files that no longer exist
+   - **Knowledge gaps**: Important concepts mentioned but lacking their own page
+   - **Quality issues**: Pages under 3 lines, pages with no sources, pages with no cross-references
+3. Auto-fix: fix broken links, create missing pages for frequently-linked concepts, add cross-refs to orphans
+4. If contradictions found → flag them, don't silently resolve (ask the human which version is correct)
+5. Save report → `outputs/lint-YYYY-MM-DD.md`
+6. Update `.discoveries/gaps.json` with discovered gaps
+7. Append to `wiki/LOG.md`
+8. Report key findings
+
+### `/wiki:discover` — Auto-discover new sources
+
+**When:** The user wants to find new content for the wiki.
+
+1. Read `config.yaml` → extract topics, keywords, feeds, subreddits
+2. Read `.discoveries/gaps.json` → knowledge gaps identified by lint
+3. Read `.discoveries/history.json` → URLs already fetched
+4. Search for new sources using:
+   - **Web search**: Search topics + keywords from config
+   - **Gap filling**: Search for gaps identified in `.discoveries/gaps.json`
+   - **Trending**: Search for latest content on each topic
+5. For each promising source found:
+   a. Fetch the full content (use `fetch_content` or `web_search`)
+   b. Save to `raw/articles/YYYY-MM-DD-slug.md` with frontmatter:
+   ```markdown
+   ---
+   title: "Original Title"
+   url: "https://..."
+   discovered: YYYY-MM-DD
+   topic: "topic-name"
+   ---
+   ```
+6. Update `.discoveries/history.json`
+7. Report: `Discovered [N] new sources. Run /wiki:ingest to process them.`
+
+**Rules:**
+
+- Max 5-10 new sources per discover cycle
+- Skip: ads, shallow listicles, duplicates, paywalled content
+- Prefer: in-depth articles, papers, guides, high-quality analysis
+
+### `/wiki:run` — Full cycle: discover → ingest → lint
+
+**When:** The user wants a complete refresh.
+
+1. Run `discover` → find new sources
+2. Run `ingest` → process all new files in `raw/`
+3. Run `lint` → health check
+4. If lint found critical gaps → optionally run one more discover+ingest cycle
+5. Save summary report → `outputs/run-YYYY-MM-DD.md`
+6. Report final summary
+
+### `/wiki:status` — Show wiki health
+
+**When:** The user wants a quick overview.
+
+Report:
+
+```
+📊 LLM Wiki Status
+══════════════════
+Wiki Roots: [topic1], [topic2]
+Sources: [N] files
+Wiki Pages: [N] total ([E] entities, [C] concepts, [S] sources, [Y] syntheses)
+Last Ingest: YYYY-MM-DD
+Last Lint: YYYY-MM-DD
+Orphans: [N]
+Knowledge Gaps: [N]
+Health: ✅ Good | ⚠️ Warning | 🔴 Needs Attention
+```
+
+### `/wiki:digest` — Daily/Weekly digest
+
+**When:** The user wants a summary of recent changes.
+
+1. Read `wiki/LOG.md` — filter entries since last digest
+2. Summarize: new sources ingested, pages created/updated, insights discovered
+3. Report in a concise digest format
+4. Save → `outputs/digest-YYYY-MM-DD.md`
+
+### `/wiki:watch <interval>` — Auto-run on schedule
+
+**When:** The user wants the wiki to stay current automatically.
+
+Set up a recurring schedule using pi's schedule system:
+
+```
+/wiki:watch daily    → schedule_prompt action=add schedule="0 0 8 * * *" prompt="Run `/wiki:run` for the LLM Wiki"
+/wiki:watch weekly   → schedule_prompt action=add schedule="0 0 9 * * 1" prompt="Run `/wiki:run` for the LLM Wiki"
+/wiki:watch hourly   → schedule_prompt action=add schedule="0 0 * * * *" prompt="Run `/wiki:run` for the LLM Wiki"
+/wiki:watch stop     → list and remove scheduled wiki jobs
+```
+
+---
+
+## Page Conventions
+
+### Naming
+
+- File names: `kebab-case.md` (e.g., `attention-mechanism.md`, `elon-musk.md`)
+- Entity names: full name lowercased, hyphenated
+- Concept names: brief descriptive slug
+
+### Frontmatter
+
+Every page MUST have YAML frontmatter:
+
+```yaml
+---
+type: entity | concept | source | synthesis | change
+created: YYYY-MM-DD
+updated: YYYY-MM-DD
+sources: [raw/path1.md, raw/path2.md]
+---
+```
+
+Entity pages also need: `category: person | organization | tool | project | product`
+Concept pages also need: `domain: ai | engineering | business | product | design | personal`
+
+### Cross-references
+
+- Internal links: `[[kebab-case-file-name]]`
+- External links: `[text](url)`
+- Source citations: `[Source: filename](../raw/path/file.md)`
+- Every page needs ≥2 `[[wikilinks]]`
+
+### Contradictions
+
+When sources disagree, document both sides:
+
+```markdown
+> ⚠️ **Contradiction:** Source A claims X, but Source B claims Y.
+> See [[page-a]] and [[page-b]] for details.
+>
+> - X: [Source A](../raw/path/a.md)
+> - Y: [Source B](../raw/path/b.md)
+```
+
+---
+
+## Variants
+
+### Personal Wiki (`config.yaml → wiki_mode: personal`)
+
+Track: learning, journaling, book notes, health, goals, reflections.
+
+- Extra folders: `wiki/journal/`, `wiki/goals/`
+- Personal entities: people you meet, books you read, courses you take
+- Daily journal entries can be ingested as raw notes
+
+### Company Wiki (`config.yaml → wiki_mode: company`)
+
+Track: competitors, market research, customer calls, internal docs, strategy.
+
+- Extra folders: `wiki/changes/`, `wiki/decisions/`
+- Change detection: re-check competitor sources for pricing/feature changes
+- Slack/email threads as sources
+- Confidence levels in frontmatter: `confidence: high | medium | low`
+
+---
+
+## Obsidian Integration
+
+This wiki is designed to be opened as an Obsidian vault (`wiki/` directory).
+
+**Recommended Obsidian plugins:**
+
+- **Dataview** — Query pages by frontmatter (type, confidence, domain)
+- **Graph View** — Visualize `[[wikilink]]` connections
+- **Backlinks** — See what links to each page
+- **Spaced Repetition** — Flashcards from `wiki/flashcards.md`
+- **Charts View** — Dashboard analytics
+- **Marp** — Markdown slide decks
+
+**The `wiki/DASHBOARD.md`** page includes Dataview queries that surface:
+
+- Low-confidence pages needing review
+- Recent updates
+- Concepts by domain
+- Pages with the most sources
+
+---
+
+## Scheduling (Auto-Update)
+
+To keep the wiki always fresh, use pi's scheduling:
+
+```bash
+# Daily at 8 AM — discover + ingest + lint
+/wiki:watch daily
+
+# Or manually:
+/wiki:run
+```
+
+The schedule checks your configured topics, discovers new sources, ingests them, and lints for health — all automatically.
+
+---
+
+## Tips
+
+- **Obsidian Web Clipper** — Browser extension that saves articles as markdown. Clip directly into `raw/articles/`.
+- **Download images**: In Obsidian Settings → Files & Links → set "Attachment folder path" to `raw/assets/`. After clipping, use "Download attachments" (Ctrl+Shift+D).
+- **Git versioning**: The wiki is a git repo. Every ingest is a commit. Revert bad ingests, review evolution.
+- **Start small**: 3-5 sources. Let the wiki grow organically. Don't try to ingest everything at once.
+- **Evolve the schema**: As you use the wiki, you'll find what works. Update rules here as you go.

package/skills/llm-wiki/templates/DASHBOARD.md

@@ -0,0 +1,50 @@
+# Wiki Dashboard
+
+> Live views powered by Obsidian Dataview. Install the [Dataview plugin](https://github.com/blacksmithgu/obsidian-dataview) for this to work.
+
+## Recent Updates
+
+```dataview
+TABLE updated, type AS "Type", file.folder AS "Folder"
+FROM "wiki"
+SORT updated DESC
+LIMIT 20
+```
+
+## By Type
+
+```dataview
+TABLE rows.file.link AS "Pages"
+FROM "wiki"
+GROUP BY type
+```
+
+## Low Confidence Pages
+
+```dataview
+TABLE confidence, updated, file.folder AS "Folder"
+FROM "wiki"
+WHERE confidence = "low"
+SORT updated ASC
+```
+
+## Orphan Pages
+
+_Pages with no inbound [[wikilinks]] — run `/wiki:lint` to detect and fix._
+
+## Most-Connected Concepts
+
+_Pages with the most inbound [[wikilinks]] — these are the knowledge hubs._
+
+## Knowledge Gaps
+
+_Topics mentioned but lacking their own page — run `/wiki:lint` to detect._
+
+## Recent Activity
+
+```dataview
+LIST
+FROM "wiki/LOG.md"
+SORT file.mtime DESC
+LIMIT 10
+```

package/skills/llm-wiki/templates/INDEX.md

@@ -0,0 +1,33 @@
+# Wiki Index
+
+> Master catalog of all wiki pages. Auto-generated — updated on every ingest, query, or lint operation.
+
+## Entities
+
+_People, organizations, tools, projects, and products._
+
+<!-- ENTITIES will be listed here -->
+
+## Concepts
+
+_Ideas, patterns, frameworks, and methodologies._
+
+<!-- CONCEPTS will be listed here -->
+
+## Sources
+
+_One summary page per ingested source document._
+
+<!-- SOURCES will be listed here -->
+
+## Syntheses
+
+_Cross-cutting analyses, comparisons, and novel insights._
+
+<!-- SYNTHESES will be listed here -->
+
+## Changes
+
+_Change detection records (primarily for company wiki mode)._
+
+<!-- CHANGES will be listed here -->

package/skills/llm-wiki/templates/LOG.md

@@ -0,0 +1,9 @@
+# Activity Log
+
+> Append-only chronological record of all wiki operations.
+
+## [YYYY-MM-DD HH:mm] init | Wiki created
+
+- Structure: `raw/`, `wiki/`, `outputs/`, `.discoveries/`
+- Mode: personal | company
+- Topics: [topic list]

package/skills/llm-wiki/templates/config.yaml

@@ -0,0 +1,38 @@
+# LLM Wiki Configuration
+
+wiki:
+  mode: personal # personal | company
+  topic: "" # Main topic of this wiki
+  description: "" # Brief description
+
+topics:
+  - name: "" # Topic name
+    keywords: [] # Search keywords for auto-discovery
+    feeds: [] # RSS/Atom feed URLs
+    subreddits: [] # Subreddits to monitor
+    github_orgs: [] # GitHub orgs/users to watch
+
+discovery:
+  enabled: true
+  max_sources_per_cycle: 8 # Max sources to fetch per discover run
+  prefer_gap_filling: true # Prioritize known knowledge gaps
+  sources:
+    - web_search # Web search by topics + keywords
+    - reddit # Reddit monitoring
+    - github # GitHub trending / org releases
+    - feeds # RSS feeds
+
+ingest:
+  batch_size: 1 # Sources per ingest pass (1 = one at a time for quality)
+  discuss_with_user: true # Discuss key takeaways before writing wiki pages
+
+lint:
+  auto_fix: true # Auto-fix orphans, missing pages, broken links
+  report_contradictions: true # Flag contradictions for human review
+
+change_detection: false # Enable for company wiki mode — track pricing/feature changes
+
+obsidian:
+  vault_path: "" # Optional: path to Obsidian vault (if different from WIKI_ROOT)
+  dataview_enabled: true # Enable Dataview dashboard
+  attachment_folder: raw/assets # Obsidian attachment folder path

package/skills/llm-wiki/templates/pages/concept.md

@@ -0,0 +1,34 @@
+---
+type: concept
+domain: ai # ai | engineering | business | product | design | personal
+created: YYYY-MM-DD
+updated: YYYY-MM-DD
+sources:
+  - raw/articles/filename.md
+---
+
+# Concept Name
+
+One-line definition of this concept.
+
+## Definition
+
+[Clear, concise explanation of what this concept is.]
+
+## How It Works
+
+[Explanation of mechanism, principles, or methodology.]
+
+## Examples
+
+- **Example 1**: [Concrete example from sources]
+- **Example 2**: [Another example]
+
+## Related Concepts
+
+- [[related-concept-1]]
+- [[related-concept-2]]
+
+## Sources
+
+- [raw/articles/filename.md](../raw/articles/filename.md)

package/skills/llm-wiki/templates/pages/entity.md

@@ -0,0 +1,31 @@
+---
+type: entity
+category: person # person | organization | tool | project | product
+created: YYYY-MM-DD
+updated: YYYY-MM-DD
+sources:
+  - raw/articles/filename.md
+---
+
+# Entity Name
+
+One-line description of who/what this is and why they matter.
+
+## Overview
+
+[2-3 paragraphs covering key facts, role, significance, and context.]
+
+## Key Facts
+
+- **Role**: [what they do]
+- **Known for**: [key contributions]
+- **Related**: [connections]
+
+## Links
+
+- [[related-concept]]
+- [[related-entity]]
+
+## Sources
+
+- [raw/articles/filename.md](../raw/articles/filename.md)

package/skills/llm-wiki/templates/pages/source.md

@@ -0,0 +1,45 @@
+---
+type: source
+format: article # article | paper | note | video | podcast | slack | email
+raw_path: raw/articles/filename.md
+ingested: YYYY-MM-DD
+topics:
+  - topic-name
+---
+
+# Source Title
+
+> _Original: [Link to source](https://...)_
+
+## Summary
+
+[2-3 paragraph summary of the source's key content, arguments, and conclusions.]
+
+## Key Takeaways
+
+- [Most important point from this source]
+- [Second important point]
+- [Third important point]
+
+## Entities Mentioned
+
+- [[entity-1]]
+- [[entity-2]]
+
+## Concepts Mentioned
+
+- [[concept-1]]
+- [[concept-2]]
+
+## Notable Quotes
+
+> "Important quote from the source" — Attribution
+
+## Connections
+
+- This source relates to [[existing-page]] because...
+- Contrasts with [[other-source]] on the topic of...
+
+## Source
+
+- [raw/articles/filename.md](../raw/articles/filename.md)