arscontexta 0.6.0

package/reference/failure-modes.md

@@ -0,0 +1,235 @@
+# Knowledge System Failure Modes
+
+How knowledge systems die. The init wizard uses this to include domain-specific warnings in generated context files, preventing the most common decay patterns before they start.
+
+---
+
+## The 10 Failure Modes
+
+### 1. Collector's Fallacy
+
+**What:** Accumulating without processing. Inbox grows indefinitely, nothing gets synthesized.
+
+**Why it happens:** Capture is easy and feels productive. Processing is hard and feels slow. The gap widens until the inbox becomes psychologically overwhelming.
+
+**Vulnerable domains:** All (universal). Especially dangerous for Research and Learning where source material is abundant.
+
+**Prevention:**
+- Processing pipeline with explicit reduce phase
+- WIP limits on inbox (process before adding more)
+- Condition-based inbox review (trigger when inbox exceeds threshold)
+- Visible inbox count as friction signal
+
+**Warning signs:** Inbox > 20 items. New captures outnumber processed items 3:1.
+
+---
+
+### 2. Orphan Drift
+
+**What:** Notes created but never connected to the graph. The system fragments into disconnected islands.
+
+**Why it happens:** Creation feels complete without connection. The reflect phase gets skipped "just this once" which becomes always.
+
+**Vulnerable domains:** Research, Learning — high creation volume without mandatory connection.
+
+**Prevention:**
+- Mandatory reflect phase after every note creation
+- MOC maintenance as part of the creation workflow
+- Periodic orphan detection (health check)
+- Topics footer as minimum connection requirement
+
+**Warning signs:** >10% of notes have no incoming wiki-links. MOCs not updated in 30+ days.
+
+---
+
+### 3. Link Rot
+
+**What:** Wiki-links pointing to notes that were renamed or deleted. The graph develops dead ends.
+
+**Why it happens:** Manual renames don't update references. Archiving notes breaks incoming links.
+
+**Vulnerable domains:** All systems with heavy linking. Worse at scale.
+
+**Prevention:**
+- Rename scripts that update all references
+- Periodic link health checks
+- Never delete — archive instead (preserves link targets)
+- Dangling link detection in health checks
+
+**Warning signs:** >5% of wiki-links resolve to nothing. Rename operations done without reference updates.
+
+---
+
+### 4. Schema Erosion
+
+**What:** YAML fields inconsistently used, enum values drift from template, validation ignored.
+
+**Why it happens:** Schemas add friction at capture time. Without enforcement, the path of least resistance is skipping fields.
+
+**Vulnerable domains:** Systems with moderate+ schema density. Research and Learning systems with many required fields.
+
+**Prevention:**
+- Templates as single source of truth (never define schema in two places)
+- Validation automation (hooks or batch scripts)
+- Minimal viable schema — only require fields that are actively queried
+- Schema evolution: add fields when querying need emerges, remove fields nobody uses
+
+**Warning signs:** >20% of notes missing required fields. Enum values appearing that aren't in the template.
+
+---
+
+### 5. MOC Sprawl
+
+**What:** Too many MOCs created, none properly maintained. Navigation overhead exceeds navigation value.
+
+**Why it happens:** Creating a MOC feels like organizing. But an unmaintained MOC is worse than no MOC — it provides false confidence about topic coverage.
+
+**Vulnerable domains:** Research, Creative — domains where topics proliferate.
+
+**Prevention:**
+- Create MOCs only at 20+ related notes (not speculatively)
+- Merge small MOCs back when they have <10 notes
+- MOC health checks (size, freshness, coverage)
+- Max 4 tiers of hierarchy
+
+**Warning signs:** MOCs with <5 links. MOCs not updated in 60+ days. >50% of MOCs with no recent additions.
+
+---
+
+### 6. Verbatim Risk
+
+**What:** Copying content instead of transforming it. Storage without the generation effect.
+
+**Why it happens:** Copy-paste is faster than reformulation. The agent (or human) takes the shortcut, producing notes that are summaries rather than claims.
+
+**Vulnerable domains:** Research, Learning — where source material is dense and tempting to reproduce.
+
+**Prevention:**
+- Generation effect gate: every note must transform, not just store
+- The composability test: "This note argues that [title]" — if it doesn't work, it's not a claim
+- Source attribution separates the original from the transformation
+- Review for paraphrase vs genuine reformulation
+
+**Warning signs:** Notes that read like summaries. Titles that are topic labels, not claims. Notes with no original reasoning.
+
+---
+
+### 7. Cognitive Outsourcing
+
+**What:** Delegating judgment entirely to the system until the human can no longer evaluate quality.
+
+**Why it happens:** The system gets good enough that the human stops checking. Quality drifts without the human noticing.
+
+**Vulnerable domains:** Research (unchecked claims compound), Therapy (pattern detection without human validation).
+
+**Prevention:**
+- Periodic human review of recent notes
+- Never auto-implement system changes (propose, don't execute)
+- Maintain human judgment checkpoints in the pipeline
+- Flag confidence levels: the system should express uncertainty
+
+**Warning signs:** Human hasn't reviewed notes in 30+ days. System changes implemented without review. No recent human corrections.
+
+---
+
+### 8. Over-Automation
+
+**What:** Hooks encoding judgment rather than verification, corrupting quality silently.
+
+**Why it happens:** Automation feels like progress. The boundary between "check if description exists" (verification) and "write a good description" (judgment) blurs.
+
+**Vulnerable domains:** Any system at automation level. Especially dangerous when moving from convention to full automation.
+
+**Prevention:**
+- Determinism boundary: hooks for verification (does X exist?), skills for judgment (is X good?)
+- Automation should fail loudly, not fix silently
+- Review hook behavior when false positives or silent failures are suspected — are they catching errors or hiding them?
+- Graduated enforcement: warn before block
+
+**Warning signs:** Hooks auto-fixing content. Quality metrics look perfect but notes feel shallow. No validation failures (suspicious — real systems produce errors).
+
+---
+
+### 9. Productivity Porn
+
+**What:** Building the system instead of using it. Meta-work displaces the actual knowledge work.
+
+**Why it happens:** System design is interesting and feels productive. The actual work (processing, connecting, synthesizing) is harder and less visible.
+
+**Vulnerable domains:** All (universal). Especially dangerous for system builders and meta-researchers.
+
+**Prevention:**
+- Gall's Law: complex systems evolve from simple ones that work. Ship the minimum viable system.
+- Time-box system improvement to <20% of total work time
+- Track content creation vs system modification ratio
+- "Would I rather be right about methodology or have a working knowledge base?"
+
+**Warning signs:** More time on CLAUDE.md than on notes. System redesigns before 100 notes. Template modifications outnumber note creations.
+
+---
+
+### 10. Temporal Staleness
+
+**What:** Content becomes outdated but isn't flagged. Old notes presented as current.
+
+**Why it happens:** Knowledge systems don't inherently track temporal validity. A note written 6 months ago appears identical to one written yesterday.
+
+**Vulnerable domains:** Gaming strategy (meta shifts), PM (project status changes), any time-sensitive domain.
+
+**Prevention:**
+- `meta_state` or staleness fields in schema (current/outdated/speculative)
+- Maintenance condition thresholds matched to domain's rate of change
+- Periodic staleness sweeps (flag notes older than N days for review)
+- Date-aware health checks
+
+**Warning signs:** Notes referencing outdated information. No maintenance activity in domains with high change rates.
+
+---
+
+## Domain Vulnerability Matrix
+
+Which failure modes are highest risk per use case. Select the top 3-4 for inclusion in the generated context file's "Common Pitfalls" section.
+
+| Failure Mode | Research | Learning | Therapy | Relationships | Creative | PM | Companion |
+|-------------|----------|----------|---------|---------------|----------|-----|-----------|
+| Collector's Fallacy | HIGH | HIGH | medium | low | medium | medium | low |
+| Orphan Drift | HIGH | HIGH | medium | low | medium | low | low |
+| Link Rot | medium | medium | low | low | medium | low | low |
+| Schema Erosion | medium | medium | medium | medium | low | medium | low |
+| MOC Sprawl | HIGH | medium | low | low | medium | low | low |
+| Verbatim Risk | HIGH | HIGH | low | low | low | low | low |
+| Cognitive Outsourcing | HIGH | medium | HIGH | low | medium | low | medium |
+| Over-Automation | medium | medium | medium | low | low | medium | low |
+| Productivity Porn | HIGH | medium | low | low | medium | HIGH | low |
+| Temporal Staleness | low | medium | low | low | low | HIGH | low |
+
+**Reading the matrix:** Include all HIGH-risk modes in the generated context file. Mention medium-risk modes briefly. Omit low-risk modes.
+
+---
+
+## Integration with Init
+
+When generating a context file in Step 5b, include a "Common Pitfalls" section:
+
+1. Look up the user's use case in the domain vulnerability matrix
+2. Select all HIGH-risk failure modes (typically 3-4)
+3. Write each warning in domain-native vocabulary
+4. Include the prevention pattern, not just the warning
+5. Place after Self-Extension Blueprints, before System Evolution
+
+Example for Research:
+```markdown
+## Common Pitfalls
+
+### The Collector's Fallacy
+Saving sources feels productive but isn't. If your inbox grows faster than you process it, stop capturing and start extracting. WIP limit: process what you have before adding more.
+
+### Orphan Notes
+A note without connections is a note that will never be found again. Every note needs at least one MOC link (Topics footer) and ideally inline connections to related notes. Run health checks to catch orphans.
+
+### Verbatim Copying
+Summarizing a source is not the same as extracting a claim. Each note must transform the material — your words, your framing, your argument. If the title doesn't pass "This note argues that [title]", it's not a claim yet.
+
+### Productivity Porn
+It's tempting to keep perfecting the system instead of using it. The vault serves the research, not the other way around. If you're spending more time on methodology than on claims, recalibrate.
+```

package/reference/interaction-constraints.md

@@ -0,0 +1,204 @@
+# Dimension Interaction Constraints
+
+How choices in one dimension create pressure on others. The derivation engine uses this to detect incoherent configurations and warn the user before generating.
+
+The valid configuration space is much smaller than the combinatorial product. Eight dimensions with three positions each produces 6,561 theoretical combinations. Most are incoherent.
+
+---
+
+## Primary Cascades
+
+### Granularity Cascade (strongest coupling)
+
+Atomic granularity creates the widest cascade:
+
+```
+atomic granularity
+  → forces explicit linking (notes lack internal context)
+  → forces deep navigation (thousands of small nodes need MOC hierarchy)
+  → forces heavy processing (extraction, reflection, reweaving to maintain links)
+  → pressures toward semantic search (keyword search fails across vocabularies)
+```
+
+Coarse granularity creates the inverse:
+
+```
+coarse granularity
+  → permits lightweight linking (internal proximity provides context)
+  → permits shallow navigation (fewer nodes to organize)
+  → permits light processing (each note is more self-contained)
+```
+
+**Incoherent combinations to flag:**
+- Atomic granularity + shallow navigation (2-tier) → navigational vertigo
+- Atomic granularity + light processing → notes created but never connected
+- Coarse granularity + heavy processing → processing cost exceeds synthesis value
+
+### Automation Cascade
+
+```
+full automation (hooks + skills + pipelines)
+  → enables dense schemas (validation catches errors)
+  → enables heavy processing (pipelines handle volume)
+  → enables condition-based maintenance (automated trigger evaluation)
+```
+
+```
+manual operation (convention only)
+  → pressures toward minimal schemas (less to remember/validate by hand)
+  → pressures toward light processing (each step costs attention)
+  → pressures toward lax maintenance conditions (manual burden)
+```
+
+**Incoherent combinations to flag:**
+- Manual operation + dense schema → maintenance burden collapses system
+- Full automation + minimal schema → wastes enforcement capacity
+- Manual operation + heavy processing → unsustainable without pipelines
+
+### Volume Cascade
+
+```
+high volume (>200 notes)
+  → requires deep navigation (shallow structures become unnavigable)
+  → requires semantic search (grep alone misses vocabulary divergence)
+  → requires automated maintenance (manual review at scale is impractical)
+```
+
+```
+low volume (<50 notes)
+  → permits shallow navigation (agent can hold full structure in context)
+  → permits keyword search (grep works when vocabulary is consistent)
+  → permits manual maintenance (small enough to review fully)
+```
+
+**Incoherent combinations to flag:**
+- High volume + no semantic search + shallow navigation → retrieval failure
+- Low volume + full automation + deep navigation → over-engineering
+
+---
+
+## Cross-Dimension Interaction Matrix
+
+Each cell describes the pressure that the row dimension's pole creates on the column dimension.
+
+| Row → Col | Granularity | Organization | Linking | Processing | Nav Depth | Maintenance | Schema | Automation |
+|-----------|-------------|-------------|---------|------------|-----------|-------------|--------|------------|
+| **Atomic granularity** | — | pressures flat | forces explicit | forces heavy | forces deep | pressures active conditions | neutral | pressures automation |
+| **Coarse granularity** | — | permits hierarchical | permits light | permits light | permits shallow | permits lax conditions | neutral | permits manual |
+| **Flat organization** | neutral | — | requires explicit links | neutral | requires MOC overlay | neutral | neutral | neutral |
+| **Hierarchical org** | neutral | — | folder membership as linking | neutral | folder browsing sufficient | neutral | neutral | neutral |
+| **Explicit+implicit linking** | neutral | neutral | — | neutral | neutral | neutral | neutral | requires semantic search tool |
+| **Heavy processing** | neutral | neutral | enables dense links | — | produces material for deep nav | generates maintenance targets | enables field discovery | benefits from automation |
+| **Light processing** | neutral | neutral | produces few links | — | needs little nav | generates few targets | minimal fields emerge | works without automation |
+| **Full automation** | neutral | neutral | neutral | enables heavy | neutral | enables active conditions | enables dense | — |
+| **Manual operation** | neutral | neutral | neutral | pressures light | neutral | pressures infrequent | pressures minimal | — |
+
+---
+
+## Coherence Rules for Init
+
+When the user selects dimension values, check these rules. WARN on soft violations, BLOCK on hard violations.
+
+### Hard Constraints (BLOCK)
+
+These produce systems that will fail:
+
+1. `atomic + navigation_depth == "2-tier" + volume > 100`
+   → "Atomic notes at this volume need at least 3-tier navigation. 2-tier creates navigational vertigo."
+
+2. `automation == "full" + platform_lacks_hooks`
+   → "Full automation requires hooks and skills. Your platform supports convention only."
+
+3. `processing == "heavy" + automation == "manual" + no_pipeline_skills`
+   → "Heavy processing without pipeline skills is unsustainable. Either add automation or reduce processing intensity."
+
+### Soft Constraints (WARN)
+
+These produce friction but can work with compensating mechanisms:
+
+1. `atomic + processing == "light"`
+   → "Atomic notes need processing to recreate decomposed context. Light processing may leave notes disconnected. Consider medium processing."
+
+2. `schema == "dense" + automation == "convention"`
+   → "Dense schemas without automated validation create maintenance burden. Consider adding validation hooks or reducing schema density."
+
+3. `linking == "explicit+implicit" + no_semantic_search`
+   → "Implicit linking (semantic search) is enabled but no search tool is configured. The system will work with explicit links only."
+
+4. `volume > 200 + maintenance_conditions_disabled`
+   → "Large vaults need condition-based maintenance to prevent link rot and orphan accumulation. Disabling maintenance conditions at this volume risks drift."
+
+5. `processing == "heavy" + maintenance_conditions_too_lax`
+   → "Heavy processing generates maintenance targets faster than lax thresholds can catch. Consider lowering condition thresholds (e.g., orphan count, stale node percentage)."
+
+6. `coarse + processing == "heavy"`
+   → "Coarse notes are self-contained enough that heavy processing provides diminishing returns. Consider medium processing."
+
+7. `flat + navigation_depth == "2-tier" + volume > 50`
+   → "Flat organization with only 2 tiers gets crowded as notes accumulate. Consider adding topic-level MOCs (3-tier)."
+
+### Kernel Primitive Constraints
+
+These constraints apply to the 15 kernel primitives and their INVARIANT/CONFIGURABLE status:
+
+**INVARIANT primitives (always present, cannot be disabled):**
+
+1. `session_capture == false` (Primitive 15)
+   → BLOCK: "Session capture is INVARIANT. Every vault saves session transcripts. The operational learning loop depends on this evidence."
+
+2. `methodology_folder == false` (Primitive 14)
+   → BLOCK: "The methodology folder is INVARIANT. Meta-skills (/ask, /architect, /rethink) require ops/methodology/ to reason about system state."
+
+3. `schema_enforcement == false` (Primitive 7)
+   → BLOCK: "Schema enforcement is INVARIANT. Without validation, metadata drift corrupts retrieval within weeks."
+
+4. `wiki_links == false` (Primitive 3)
+   → BLOCK: "Wiki links are INVARIANT. They are the universal reference form and the foundation of the graph database."
+
+**CONFIGURABLE primitives (can be toggled):**
+
+5. `self_space == true + preset == "research"`
+   → WARN: "Self space is OFF by default for Research presets. The knowledge graph is the focus, not agent identity. Enable only if persistent agent memory across sessions is needed."
+
+6. `self_space == false + preset == "personal_assistant"`
+   → WARN: "Self space is ON by default for Personal Assistant presets. Agent identity and persistent memory are central to the experience. Disable only if the agent's sense of self is not needed."
+
+7. `semantic_search == false + linking == "explicit+implicit"`
+   → WARN: "Semantic search is configured as opt-in. Without qmd, implicit linking falls back to keyword overlap and MOC traversal."
+
+**Condition-based maintenance constraints:**
+
+8. `condition_thresholds_all_zero`
+   → WARN: "All condition-based maintenance thresholds are set to zero (disabled). The vault will not surface maintenance tasks. Consider enabling at least orphan detection and dangling link checks."
+
+9. `task_stack == false` (Primitive 13)
+   → BLOCK: "The task stack is INVARIANT. Without task tracking, the agent has no lifecycle visibility and cannot answer 'what should I work on?'"
+
+---
+
+## Compensating Mechanisms
+
+Some dimension mismatches can be compensated rather than blocked:
+
+| Mismatch | Compensating Mechanism | Effectiveness |
+|----------|----------------------|---------------|
+| Atomic + medium processing | Semantic search compensates for missing explicit links | Moderate — finds connections but doesn't create them |
+| Dense schema + convention | Good templates reduce manual validation burden | Moderate — helps at capture, not at maintenance |
+| High volume + shallow nav | Strong semantic search enables discovery without deep hierarchy | Moderate — works for retrieval, not for orientation |
+| Manual + moderate processing | Batch processing sessions compensate for missing automation | Low — depends on user discipline |
+
+The key question: are interaction constraints hard (violating them produces failure) or soft (violating them produces friction that compensating mechanisms can overcome)? The answer appears to be: granularity cascades are hard, automation cascades are soft. You cannot atomic-granularity your way out of missing navigation depth. But you can manually maintain a dense schema if you're disciplined enough — it's just friction, not failure.
+
+---
+
+## Derivation Application
+
+When generating a system:
+
+1. **Start from use-case preset** (or tradition preset) — these are pre-validated coherence points
+2. **Allow user customization** — but check each change against interaction constraints
+3. **Cascade recommendations** — if user changes granularity from moderate to atomic, recommend increasing processing and navigation depth
+4. **Document justification** — include interaction constraint reasoning in the derivation rationale section of the generated context file
+5. **Flag unresolved tensions** — if user overrides a warning, note it in the generated system for future reseeding
+
+The derivation rationale should include which constraints were active and how they were resolved. This enables principled reseeding when friction patterns emerge later.