arscontexta 0.6.0

package/reference/kernel.yaml

@@ -0,0 +1,242 @@
+# Universal Kernel — Fifteen Universal Primitives
+# Every system generated by arscontexta includes these.
+# The kernel is constant. The derivation engine varies everything above it.
+# Full automation is the default — users may opt down.
+
+schema_version: 3
+
+primitives:
+  - id: markdown-yaml
+    name: Markdown files with YAML frontmatter
+    description: Notes are plain text files with structured metadata. The file IS the complete artifact.
+    layer: foundation
+    requires: filesystem
+    enforcement: invariant
+    validation:
+      check: Every .md file has valid YAML frontmatter (--- delimited block)
+      threshold: ">95% of note files"
+    cognitive_grounding: "Cognitive offloading (Clark & Chalmers Extended Mind) — external structures extend thinking beyond context window limits"
+
+  - id: wiki-links
+    name: Wiki links as graph edges
+    description: "[[note title]] creates navigable relationships. Filenames are unique; links resolve by name."
+    layer: foundation
+    requires: filesystem, unique-filenames
+    enforcement: invariant
+    validation:
+      check: All wiki links resolve to existing files
+      threshold: ">90% resolution (allows planned/example links)"
+    cognitive_grounding: "Spreading activation — reading one note primes related notes through explicit edges"
+
+  - id: moc-hierarchy
+    name: MOC hierarchy for attention management
+    description: Maps of Content organize notes into navigable topic areas. Hub → domain → topic → notes.
+    layer: convention
+    requires: wiki-links
+    enforcement: invariant
+    validation:
+      check: At least 3 MOCs exist. Every note appears in at least one MOC.
+      threshold: "<5% orphan notes after initial creation window"
+    cognitive_grounding: "Context-switching cost (Leroy 2009) — MOCs reduce 23-minute reorientation penalty by presenting topic state immediately"
+
+  - id: tree-injection
+    name: Tree injection at session start
+    description: Agent sees full file structure immediately. Provides orientation before action.
+    layer: convention
+    requires: filesystem
+    enforcement: invariant
+    implementation:
+      claude-code: SessionStart hook injects tree output
+      openclaw: Load WORKSPACE-MAP.md at session start
+      minimal: Context file instructs agent to ls at start
+    validation:
+      check: Session start procedure loads file structure
+    cognitive_grounding: "Orientation before action — prevents wasting context on discovery"
+
+  - id: description-field
+    name: Description field for progressive disclosure
+    description: "YAML description (~150 chars) adds information beyond the title. Enables filter-before-read."
+    layer: convention
+    requires: markdown-yaml
+    enforcement: invariant
+    validation:
+      check: Every note has a description field that differs from the title
+      threshold: ">95% of notes"
+    cognitive_grounding: "Progressive disclosure — each layer reveals more but costs more tokens. Stop at the level you need."
+
+  - id: topics-footer
+    name: Topics footer linking notes to MOCs
+    description: "Every note declares which MOC(s) it belongs to via topics field."
+    layer: convention
+    requires: moc-hierarchy, markdown-yaml
+    enforcement: invariant
+    validation:
+      check: "topics field present on every non-MOC note with at least one wiki link"
+      threshold: ">95% of notes"
+    cognitive_grounding: "Bidirectional navigation — MOCs link down to notes, notes link up to MOCs"
+
+  - id: schema-enforcement
+    name: Schema enforcement via validation
+    description: Templates define required fields and enums. Validation checks notes against templates.
+    layer: convention
+    requires: markdown-yaml
+    enforcement: invariant
+    implementation:
+      tier-1: Hook-based validation (PostToolUse event)
+      tier-2: Batch validation script
+      tier-3: Context file instructions for manual checking
+    validation:
+      check: Templates exist as single source of truth. Validation mechanism catches missing required fields.
+    cognitive_grounding: "Instruction compliance degrades as context fills — deterministic validation catches what instruction-following misses"
+
+  - id: self-space
+    name: Self space for agent persistent memory
+    description: "Dedicated directory for identity, methodology, goals. Read at session start, updated at session end."
+    layer: convention
+    requires: markdown-yaml, moc-hierarchy
+    enforcement: configurable
+    structure:
+      required:
+        - identity.md
+        - methodology.md
+        - goals.md
+      optional:
+        - relationships.md
+        - memory/
+        - sessions/
+        - journal/
+    validation:
+      check: "self/ exists with identity.md, methodology.md, goals.md populated"
+    cognitive_grounding: "Session handoff — externalized state gives each fresh session a briefing from the previous one"
+
+  - id: session-rhythm
+    name: "Session rhythm: orient, work, persist"
+    description: "Three-phase cycle encoded in context file. Orient (read state), work (execute + capture), persist (update + push)."
+    layer: convention
+    requires: markdown-yaml
+    enforcement: invariant
+    validation:
+      check: "Context file documents orient/work/persist cycle. Session start loads orientation state."
+    cognitive_grounding: "Closure rituals (Newport) prevent attention residue. Orientation prevents cold starts. Together they bracket productive sessions."
+
+  - id: semantic-search
+    name: Semantic search capability
+    description: Meaning-based discovery across vocabularies. Complements structural navigation.
+    layer: automation
+    requires: wiki-links
+    enforcement: configurable
+    implementation:
+      tier-1: "Embedding tool (qmd, MCP server)"
+      tier-2: "LLM-assisted similarity search"
+      tier-3: "Periodic manual review guided by topic adjacency"
+    validation:
+      check: "Given a note, the mechanism returns related notes that share no significant keywords"
+    cognitive_grounding: "Vocabulary divergence — same concept, different words. Structural graph traversal misses these connections."
+
+  - id: unique-addresses
+    name: Filesystem graph database
+    description: "The vault is a graph database on the filesystem. Markdown files are nodes, wiki links are edges, YAML frontmatter is the property store, ripgrep is the query engine. Three query levels: field-level (rg on YAML), node-level (backlinks, link extraction), graph-level (scripts combining traversal + analysis)."
+    layer: foundation
+    requires: wiki-links, markdown-yaml
+    enforcement: invariant
+    validation:
+      check: "ops/scripts/graph/ directory exists with core analysis scripts. /graph skill generated. At minimum: orphan detection, dangling link detection, backlink counting, link density measurement, forward/backward traversal, synthesis opportunity detection."
+    cognitive_grounding: "Structured data enables queries that unstructured files cannot answer. The graph structure emerges from kernel invariants without additional infrastructure."
+
+  - id: discovery-first
+    name: Discovery-first quality gate
+    description: "Everything created must be optimized for future agent discovery. If an agent can't find a note, the note doesn't exist."
+    layer: convention
+    requires: description-field, topics-footer
+    enforcement: invariant
+    enforcement_detail:
+      context-file: "Discovery-First Design section — BEFORE creating any note, ask how will a future session find this?"
+      skills: "Every skill that creates notes includes a discovery check before writing"
+      hooks: "PostToolUse hook on Write validates discovery criteria (when platform supports hooks)"
+      learning: "When a note IS hard to find, system captures observation about why"
+    validation:
+      check: "Generated context file contains Discovery-First Design section. Generated skills include discovery checks. If hooks supported, PostToolUse validation hook exists."
+    cognitive_grounding: "Discoverability compounds — the system's value grows through findability. Every note that can't be found is negative value."
+
+  - id: operational-learning-loop
+    name: Operational learning loop
+    description: "The system accumulates friction signals (observations, tensions) and periodically reviews them to evolve itself."
+    layer: convention
+    requires: schema-enforcement
+    enforcement: invariant
+    components:
+      observation-capture:
+        description: "After every significant action, the agent asks: what did I learn? Creates atomic note in ops/observations/ with prose-sentence title and structured YAML."
+        schema:
+          required: [description, category, observed, status]
+          category_values: [methodology, process, friction, surprise, quality]
+          status_values: [pending, promoted, implemented, archived]
+      tension-capture:
+        description: "When any skill discovers content that contradicts existing notes or system assumptions, IMMEDIATELY creates atomic note in ops/tensions/."
+        schema:
+          required: [description, observed, involves, status]
+          status_values: [pending, resolved, dissolved]
+      threshold-review:
+        description: "When pending observations exceed 10 or pending tensions exceed 5, session-start orient surfaces suggestion to run /{DOMAIN:rethink}."
+        thresholds:
+          observations: 10
+          tensions: 5
+    implementation:
+      tier-1: "PostToolUse hook logs friction automatically. Skill-integrated tension detection. /{DOMAIN:rethink} command with session-start threshold check."
+      tier-2: "Context file instructs agent to log after significant actions. Context file instructs immediate tension capture. Context file documents threshold + manual review trigger."
+      tier-3: "Agent instructed to note friction in ops/. Agent instructed to note contradictions. Periodic human-triggered review."
+    validation:
+      check: "ops/observations/ and ops/tensions/ directories exist. Review trigger (threshold or condition) documented in context file. /{DOMAIN:rethink} command or equivalent exists."
+    cognitive_grounding: "Scientific method applied to knowledge systems — hypothesize, implement, observe, revise. Systems that cannot observe their own friction cannot evolve."
+
+  - id: task-stack
+    name: Task stack
+    description: "The lifecycle backbone. Every note flows through a task state. ops/tasks.md is the human-readable view; ops/queue/ holds the unified queue with both pipeline tasks (claims, enrichments) and maintenance tasks (auto-generated from condition evaluation, auto-closed when satisfied)."
+    layer: convention
+    requires: markdown-yaml, session-rhythm
+    enforcement: invariant
+    validation:
+      check: "ops/tasks.md exists. Queue file exists with schema_version >= 3. Context file references both in session-orient phase. /next reads task stack AND reconciles maintenance conditions in the queue."
+    cognitive_grounding: "Without task tracking, the agent has no lifecycle visibility. Cannot answer 'what should I work on?' or 'what processing is pending?'"
+
+  - id: methodology-folder
+    name: Methodology folder
+    description: "The vault's self-knowledge. ops/methodology/ contains linked notes explaining why the vault is configured the way it is, what processing pipeline it uses, and how it has evolved. Content types: derivation-rationale, kernel-state, pipeline-config, maintenance-conditions, vocabulary-map, configuration-state."
+    layer: convention
+    requires: markdown-yaml, moc-hierarchy, operational-learning-loop
+    enforcement: invariant
+    validation:
+      check: "ops/methodology/ directory exists with methodology.md MOC. At least one derivation-rationale note exists. Context file references ops/methodology/ for meta-skill context. /remember writes to ops/methodology/. Meta-skills read from it."
+    cognitive_grounding: "Without self-knowledge, meta-skills operate blind. /ask cannot explain config choices. /architect cannot reason about changes. /remember has nowhere to capture corrections."
+
+  - id: session-capture
+    name: Session capture
+    description: "Every session transcript is saved automatically. Stop hooks save transcripts to ops/sessions/. Auto-creates mining tasks. Friction detection runs on transcripts — the system catches what was missed during the session."
+    layer: automation
+    requires: session-rhythm, operational-learning-loop
+    enforcement: invariant
+    validation:
+      check: "ops/sessions/ directory exists. Session-end hook saves transcripts. Condition-based mining trigger exists for unprocessed sessions."
+    cognitive_grounding: "The operational learning loop depends on evidence. Session transcripts contain every redirection and correction — mining them catches what explicit /remember misses."
+
+# Dependency graph (DAG):
+# markdown-yaml → description-field
+# markdown-yaml → topics-footer
+# markdown-yaml → schema-enforcement
+# markdown-yaml → self-space
+# markdown-yaml → session-rhythm
+# wiki-links → moc-hierarchy → topics-footer
+# wiki-links → semantic-search
+# wiki-links + markdown-yaml → unique-addresses
+# description-field → discovery-first
+# topics-footer → discovery-first
+# schema-enforcement → operational-learning-loop
+# markdown-yaml + session-rhythm → task-stack
+# markdown-yaml + moc-hierarchy + operational-learning-loop → methodology-folder
+# session-rhythm + operational-learning-loop → session-capture
+# tree-injection (standalone)
+
+# Layer mapping:
+# foundation: markdown-yaml, wiki-links, unique-addresses
+# convention: moc-hierarchy, tree-injection, description-field, topics-footer, schema-enforcement, self-space, session-rhythm, discovery-first, operational-learning-loop, task-stack, methodology-folder
+# automation: semantic-search, session-capture

package/reference/methodology.md

@@ -0,0 +1,283 @@
+# Ars Contexta Methodology Reference
+
+A portable, domain-agnostic distillation of the Tools for Thought for Agents research. This document powers design decisions across all commands and skills.
+
+---
+
+## Core Philosophy
+
+**If it won't exist next session, write it down now.** Agents lose everything between sessions unless it's externalized. The vault is not storage — it's an external thinking structure that extends cognition across sessions.
+
+**Depth over breadth. Quality over speed.** When you pick a task, commit to understanding it completely. The constraint is never cost — it's whether you nailed it.
+
+**The vault is the argument.** Philosophy with proof of work. Every note, link, and MOC demonstrates the methodology it describes. If the vault fails to be navigable, the philosophy fails.
+
+---
+
+## Why This Works: Cognitive Science Foundations
+
+The methodology transfers across domains because it encodes cognitive science, not domain specifics:
+
+### Working Memory Limits (Cowan's 4±1)
+Humans hold ~4 items in working memory. Agents have context windows that degrade as they fill. Both need external structures to think beyond their native capacity. The vault IS the extended working memory.
+
+### Extended Mind (Clark & Chalmers)
+Cognition doesn't stop at the skull. When an agent reads self/ to remember who it is, that's not retrieval — it's thinking. The vault participates in the cognitive process, not just records its outputs.
+
+### Spreading Activation
+In human memory, activating one concept primes related concepts. Wiki-links implement this for agents: reading one note activates related notes through explicit edges, enabling multi-hop reasoning without entity extraction pipelines.
+
+### Small-World Topology
+Efficient knowledge networks have dense local clusters (topic areas) connected by long-range bridges (cross-topic synthesis). This enables short paths between any two concepts. MOCs provide the hubs; wiki-links provide the edges.
+
+### Context-Switching Cost
+Humans need ~23 minutes to fully re-engage after interruption. Agents suffer attention degradation when context fills with irrelevant material. MOCs solve this for both: they front-load relevant context, reducing the cost of entering a topic.
+
+---
+
+## Universal Note Pattern
+
+Every note follows the same structure regardless of domain:
+
+### 1. Prose-as-Title
+The title IS the insight, expressed as a proposition that works as prose when linked:
+- "Mom prefers phone calls on Sunday mornings"
+- "The anxiety usually starts when I skip morning routine"
+- "Spaced repetition works better when I study after exercise"
+
+**The composability test:** Can you write `since [[title]]` and have it read naturally? If yes, the title works.
+
+### 2. YAML Frontmatter
+Structured metadata makes notes queryable. Base fields are universal; domain fields extend them:
+
+```yaml
+---
+description: One sentence adding context beyond the title (~150 chars)
+type: insight | pattern | preference | fact | decision | question
+created: YYYY-MM-DD
+---
+```
+
+**The description must add NEW information.** Title gives the claim; description gives scope, mechanism, or implication the title doesn't cover.
+
+### 3. Body
+The reasoning that supports the title claim. Show the path to conclusions. Use connective words: because, therefore, this suggests, however. Acknowledge uncertainty when appropriate.
+
+### 4. Wiki-Links
+`[[note title]]` creates edges in the knowledge graph. Inline links with context are more valuable than footer links:
+
+Good: "Since [[spaced repetition works better after exercise]], I schedule gym before study blocks"
+Bad: "See also: [[spaced repetition works better after exercise]]"
+
+### 5. Topics Footer
+Every note connects to at least one MOC:
+
+```markdown
+---
+
+Topics:
+- [[learning]]
+```
+
+---
+
+## Universal Operations
+
+All knowledge systems perform these operations, regardless of methodology:
+
+| Operation | Purpose | Example |
+|-----------|---------|---------|
+| **Capture** | Get information in with zero friction | Drop URL in inbox/ |
+| **Process** | Transform raw input to structured knowledge | Extract insights from daily log |
+| **Connect** | Link related knowledge | Find wiki-link connections |
+| **Navigate** | Manage attention, find content | MOCs, semantic search |
+| **Maintain** | Keep system healthy | Orphan detection, schema validation |
+| **Synthesize** | Find patterns across notes | Cross-topic synthesis notes |
+| **Retrieve** | Find the right information when needed | Descriptions, progressive disclosure |
+| **Evolve** | Improve the system itself | Challenge assumptions, update methodology |
+
+---
+
+## Universal Components
+
+Every knowledge system is built from these structural primitives:
+
+| Component | What It Does | Implementation |
+|-----------|-------------|----------------|
+| **Notes** | Atomic units of knowledge | Prose-titled markdown files |
+| **Schema** | Structured metadata | YAML frontmatter |
+| **Links** | Graph edges | Wiki-links `[[note]]` |
+| **Navigation** | Attention management | MOCs (Maps of Content) |
+| **Folders** | Physical organization | inbox → working → archive |
+| **Templates** | Reusable structures | Per-domain note templates |
+| **Hooks** | Event-driven automation | Session orient, validation |
+| **Search** | Retrieval infrastructure | Semantic + full-text |
+| **Health** | Maintenance operations | Validation, orphan detection |
+
+---
+
+## MOC Architecture
+
+MOCs (Maps of Content) are attention management devices, not just organizational tools. They reduce context-switching cost by presenting topic state immediately.
+
+### Three-Tier Hierarchy
+
+```
+identity.md (Hub)           → links to domain MOCs
+├── methodology.md (Domain)  → links to topic areas
+├── goals.md (Domain)        → links to active threads
+├── relationships.md (Domain)→ links to people topics
+└── [topic MOCs]             → links to atomic notes
+```
+
+### MOC Structure
+
+```markdown
+# topic-name
+
+Brief orientation: what this topic covers.
+
+## Core Ideas
+- [[note]] — context explaining relevance
+
+## Tensions
+Unresolved conflicts within this topic.
+
+## Open Questions
+What's unexplored.
+
+---
+
+Topics:
+- [[parent-moc]]
+```
+
+### When MOCs Matter
+
+- **For humans:** Reduces 23-minute context-switching penalty
+- **For agents:** Prevents context window attention degradation (the "smart zone" problem)
+- **For both:** Front-loads relevant context so thinking starts immediately
+
+---
+
+## The Processing Gap
+
+Raw capture (daily logs, inbox items, session notes) is write-only memory without processing. The gap between capture and useful knowledge requires transformation:
+
+1. **Extract** — Mine raw material for atomic insights
+2. **Connect** — Compare new insights against existing notes
+3. **Navigate** — Link new notes into MOCs
+4. **Maintain** — Detect orphans, validate schema, resurface old notes
+
+This is why temporal separation of capture and processing matters: capture during active hours (zero friction), process in dedicated sessions (fresh context, quality focus).
+
+---
+
+## Session Rhythm
+
+Every session follows: **Orient → Work → Persist**
+
+### Orient
+Read self/ at session start. Remember who you are, what you're working on, what's in progress. MOCs provide topic state. This is not optional — without orientation, every session starts cold.
+
+### Work
+Do the actual task. Surface connections as you go. If you discover something worth keeping, write it down immediately — it won't exist next session otherwise.
+
+### Persist
+Write insights before session ends. Update MOCs with new notes. Capture observations. Push changes. Nothing persists without explicit externalization.
+
+---
+
+## Domain-Specific Schema Extensions
+
+The universal structure stays the same. Only YAML field names change:
+
+### Research
+```yaml
+methodology: Zettelkasten | Evergreen | Cornell | Original
+source: [[source-name]]
+classification: claim | methodology | tension
+```
+
+### Relationships
+```yaml
+person: Name
+category: preference | pattern | important-date | interaction | care-task
+last_confirmed: YYYY-MM-DD
+follow_up: true | false
+```
+
+### Learning
+```yaml
+domain: subject area
+mastery: new | developing | solid | expert
+prerequisites: ["[[concept]]"]
+```
+
+### Therapy/Reflection
+```yaml
+category: pattern | trigger | coping-strategy | insight | growth-goal
+confidence: observed | hypothesized | verified
+frequency: once | occasional | regular | constant
+```
+
+### Life Management
+```yaml
+area: health | finance | home | social | career
+priority: low | medium | high | urgent
+deadline: YYYY-MM-DD
+```
+
+### Creative Work
+```yaml
+stage: idea | draft | revision | complete
+medium: writing | visual | music | code
+inspiration: ["[[reference]]"]
+```
+
+---
+
+## Quality Standards
+
+### 1. Specificity
+Claims must be specific enough to be wrong. "Quality matters" fails. "Quality matters more at scale because small differences compound through selection" passes.
+
+### 2. Visible Reasoning
+Show the path to conclusions. The reader should be able to follow logic and disagree at each step.
+
+### 3. Acknowledged Uncertainty
+"I think this is true because X, though Y might complicate it" — confidence should match evidence.
+
+### 4. Explicit Connections
+Not "related to [[X]]" but "extends [[X]] by adding Y." Articulate WHY notes connect.
+
+### 5. Followed Implications
+Push ideas to second and third order effects. What does this mean? If true, what else must be true?
+
+---
+
+## Self-Extension Principle
+
+After scaffolding, the agent should be able to:
+1. Build its own hooks by following blueprints
+2. Build its own processing skills
+3. Build ingestion connectors for new source types
+4. Extend the schema for new domains
+5. Modify the context file as understanding deepens
+
+Init scaffolds the minimum. The methodology teaches the maximum. The agent grows the rest.
+
+---
+
+## Key Research Claims
+
+These claims from the TFT research graph ground the methodology:
+
+- **Wiki links implement GraphRAG without the infrastructure** — explicit curated edges enable multi-hop reasoning
+- **Each new note compounds value by creating traversal paths** — link density matters more than note count
+- **Descriptions are retrieval filters not summaries** — lossy compression optimized for decision-making
+- **Fresh context per task preserves quality better than chaining phases** — session isolation keeps each phase in the smart zone
+- **Skills encode methodology so manual execution bypasses quality gates** — skills contain selectivity gates
+- **Throughput matters more than accumulation** — processing velocity, not archive size
+- **The generation effect requires active transformation not just storage** — moving files is not processing
+- **Backward maintenance asks what would be different if written today** — living documents, not finished artifacts