@vue-skuilder/db 0.1.16 → 0.1.18

package/docs/brainstorm-navigation-paradigm.md

@@ -0,0 +1,369 @@
+# Brainstorm: Navigation Paradigm Exploration
+
+> **Status:** Informal brainstorming. Not a design doc or TODO.
+
+This document explores the navigation strategy paradigm: what other strategy types
+might exist, where the current design shines or struggles, and what alternative
+approaches could achieve similar aims.
+
+---
+
+## Potential Strategy Classes
+
+### Generators (Candidate Sources)
+
+These produce candidate cards, typically with initial scores.
+
+| Strategy | Description | Trigger/Source |
+|----------|-------------|----------------|
+| **ELO** (exists) | Skill-proximity matching | Always active, scores by distance |
+| **SRS** (planned) | Spaced repetition scheduling | Time-based, scores by overdueness |
+| **HardcodedOrder** (exists) | Fixed sequence | Index position |
+| **TriggerResponse** | Activated by specific events | See below |
+| **RecentFailures** | Re-surfaces cards with recent errors | Performance signal |
+| **Adaptive Drill** | Intensive repetition on weak spots | Threshold-triggered |
+| **Random** | Uniform random selection | Always active (baseline) |
+| **CurriculumSequence** | Author-defined learning path | Progress gates |
+| **SocialSignal** | Cards others struggled with | Cohort data |
+
+### Filters (Score Transformers)
+
+These modify scores from upstream candidates.
+
+| Strategy | Description | Effect |
+|----------|-------------|--------|
+| **HierarchyDefinition** (exists) | Prerequisite gating | Hard filter (score=0) |
+| **InterferenceMitigator** (exists) | Confusable concept separation | Soft penalty |
+| **RelativePriority** (exists) | Utility-based boosting | Soft boost |
+| **TimeSinceTag** | Recency of tag exposure | Soft penalty/boost |
+| **SessionCoherence** | Thematic consistency within session | Soft boost for related |
+| **FatigueAdjuster** | Reduce difficulty as session progresses | Soft penalty for hard cards late |
+| **UserPreference** | User-stated preferences/interests | Soft boost |
+| **Novelty** | Freshness vs. review balance | Score adjustment |
+
+### Trigger-Response Generators (Your Interest)
+
+These are **event-driven generators** that activate under specific conditions:
+
+#### Intensive Response Intervention
+
+```
+Trigger: User fails card C with tag T at mastery level M
+         AND failure count on T exceeds threshold in window
+
+Response: Generator activates
+          - Surfaces remedial content for tag T
+          - May include prerequisite tags
+          - Runs for N cards or until success threshold
+
+Exit: Deactivates when recovery criteria met
+```
+
+#### Other Trigger Scenarios
+
+| Trigger | Response |
+|---------|----------|
+| **Frustration signal** (N failures in M minutes) | Shift to easier content, motivational material |
+| **Plateau detection** (no ELO movement for K sessions) | Surface challenging content, introduce new tags |
+| **Mastery celebration** (tag mastery achieved) | Capstone card, related advanced content |
+| **Inactivity return** (first session after gap) | Review-heavy, confidence rebuilding |
+| **Learning velocity drop** | Adjust presentation cadence, check interference |
+| **Specific error pattern** (e.g., always confuses X and Y) | Targeted contrast exercises |
+
+#### Implementation Considerations
+
+Trigger-response generators need:
+1. **Event bus / signal mechanism** — How do strategies observe events?
+2. **Activation state** — How to track "I am currently intervening on tag T"?
+3. **Priority over base generator** — When active, should it preempt or blend?
+4. **Exit criteria** — When does intervention end?
+
+This argues for strategies having **lifecycle state**, not just stateless scoring.
+See `todo-strategy-state-storage.md` for related concerns.
+
+---
+
+## Hybrid Generators
+
+Some strategies might combine generation and filtering:
+
+| Strategy | Behavior |
+|----------|----------|
+| **Contextual ELO** | ELO generator that adjusts target based on session state |
+| **Pacing Controller** | Generates from pool but enforces cadence rules |
+| **Multi-Objective** | Balances multiple generators (ELO + SRS + Priority) |
+
+---
+
+## Strengths of Current Paradigm
+
+### 1. Composability via Delegation
+
+The delegate pattern is genuinely elegant:
+```
+Priority(Interference(Hierarchy(ELO)))
+```
+
+Each layer does one thing. Easy to reason about, test, and swap components.
+
+### 2. Unified Score Semantics
+
+Everything speaks `WeightedCard`. Scores compose multiplicatively. A hard filter
+(score=0) works at any layer. Soft preferences blend naturally.
+
+### 3. Separation of Concerns
+
+- **Generators** know about card selection
+- **Filters** know about constraints/preferences
+- **SessionController** knows about time/queue management
+- **No cross-cutting entanglement**
+
+### 4. Backward Compatibility
+
+Legacy `getNewCards()` / `getPendingReviews()` still work. Migration is incremental.
+
+### 5. Extensibility
+
+New strategies plug in via `ContentNavigator.create()` dynamic loading. No central
+registry modification required.
+
+---
+
+## Weaknesses of Current Paradigm
+
+### 1. Stateless by Default
+
+Strategies are instantiated per-request. No persistence of:
+- "I am currently running an intervention"
+- "Last time I surfaced tag X was..."
+- "This user responds well to strategy Y"
+
+**Mitigation:** `todo-strategy-state-storage.md` (not implemented)
+
+### 2. Pull-Only Model
+
+Strategies are invoked via `getWeightedCards()`. They can't:
+- React to events (card failure, session end)
+- Proactively signal "I have urgent candidates"
+- Coordinate across sessions
+
+**Mitigation:** Would need event subscription / push mechanism
+
+### 3. Limited Context Passing
+
+Strategies receive `user`, `course`, `strategyData`. They don't receive:
+- Current session state (cards seen, failures)
+- Temporal context (time of day, day of week)
+- User's explicit goals for this session
+
+**Mitigation:** Expand context object passed to strategies
+
+### 4. No Inter-Strategy Communication
+
+Strategies can't:
+- "I'm handling this, others back off"
+- Share computed data (e.g., interference detection useful to multiple)
+- Negotiate priority
+
+**Mitigation:** Shared context / blackboard pattern
+
+### 5. Pipeline Assembly Gap
+
+(Addressed in `todo-naive-orchestration.md`)
+
+The delegate pattern exists but isn't wired up. Each filter creates its own
+delegate internally based on `serializedData`. No central orchestration.
+
+### 6. Linear Pipeline Limitations
+
+Current model is strictly linear: `A(B(C(D)))`. Some scenarios want:
+- Parallel generators merged: `merge(ELO, SRS)`
+- Conditional branches: `if frustrated then X else Y`
+- Dynamic reconfiguration mid-session
+
+---
+
+## Alternative Mechanisms
+
+### 1. Blackboard Architecture
+
+Instead of a linear pipeline, strategies write to a shared "blackboard":
+
+```
+Blackboard {
+  candidates: WeightedCard[]
+  signals: { frustration: 0.3, velocity: 0.8, ... }
+  interventions: { activeTag: 'vowel-sounds', since: ... }
+  vetoes: Set<cardId>
+}
+
+Strategies read/write blackboard in phases:
+1. Generators add candidates
+2. Observers add signals
+3. Filters modify scores
+4. Interventions override/inject
+5. Final selection
+```
+
+**Pros:** Richer coordination, event-driven capable
+**Cons:** More complex, harder to reason about, ordering sensitive
+
+### 2. Reactive / Event-Driven
+
+Replace pull model with event streams:
+
+```typescript
+interface NavigationEventSource {
+  onCardResult(result: CardResult): void;
+  onSessionStart(session: SessionContext): void;
+  onSessionEnd(stats: SessionStats): void;
+}
+
+interface ReactiveCandidateSource {
+  candidates$: Observable<WeightedCard[]>;
+  priority$: Observable<number>;
+}
+```
+
+Strategies subscribe to events and emit candidates when they have something urgent.
+
+**Pros:** Natural for trigger-response patterns
+**Cons:** Complexity, backpressure, ordering
+
+### 3. Rule Engine
+
+Declarative rules instead of imperative strategies:
+
+```yaml
+rules:
+  - name: interference-cooldown
+    when:
+      - card.tags intersects session.recentTags
+      - tag.maturity < threshold
+    then:
+      - score *= 0.5
+      
+  - name: frustration-intervention
+    when:
+      - session.failureRate > 0.4
+      - session.consecutiveFailures >= 3
+    then:
+      - activate: remedial-generator
+      - target: session.lastFailedTag
+```
+
+**Pros:** Declarative, inspectable, author-configurable
+**Cons:** Limited expressiveness, new DSL to maintain
+
+### 4. Bandit Selection (Planned)
+
+Instead of a fixed pipeline, select among N candidate pipelines:
+
+```
+PipelineA: ELO
+PipelineB: Hierarchy(ELO)
+PipelineC: Interference(Hierarchy(ELO))
+
+Orchestrator selects pipeline per-session based on:
+- User cohort
+- Historical effectiveness
+- Exploration/exploitation balance
+```
+
+**Pros:** Learns what works, self-improving
+**Cons:** Requires outcome measurement, cold-start problem
+
+See `todo-evolutionary-orchestration.md`.
+
+### 5. LLM-Guided Selection
+
+Use a language model to interpret learner state and select content:
+
+```
+Given:
+- User profile: { elo: 950, strugglingWith: ['vowel-sounds'], ... }
+- Session context: { duration: 12min, cardsSeen: 8, recentFailures: 2 }
+- Available cards: [...]
+
+Select the next 5 cards and explain reasoning.
+```
+
+**Pros:** Flexible, can incorporate nuance
+**Cons:** Latency, cost, unpredictability, hard to debug
+
+### 6. Constraint Satisfaction
+
+Frame card selection as constraint satisfaction:
+
+```
+Constraints:
+- At most 3 new tags per session
+- No more than 2 cards from same tag consecutively  
+- Prerequisite tags must be mastered
+- Session should have 60% review, 40% new
+
+Objective: Maximize expected learning gain
+```
+
+Solver finds valid card sequence.
+
+**Pros:** Principled, globally optimal
+**Cons:** Computational cost, hard to specify constraints
+
+---
+
+## Recommendation: Incremental Extension
+
+Rather than wholesale paradigm shift, extend current model:
+
+### Near-term (compatible with naive orchestration)
+
+1. **Richer context** — Pass session state to strategies
+2. **Trigger-response generator** — New generator type with activation state
+3. **Session-scoped state** — Allow strategies to persist within-session
+
+### Medium-term (with strategy state storage)
+
+4. **Cross-session state** — Strategies remember past sessions
+5. **Event hooks** — Strategies can subscribe to card results
+6. **Intervention protocol** — Active intervention preempts normal generation
+
+### Long-term (with evolutionary orchestration)
+
+7. **Bandit selection** among pipelines
+8. **Outcome measurement** for strategy effectiveness
+9. **Hybrid architectures** where blackboard/reactive patterns augment pipeline
+
+---
+
+## Questions to Explore
+
+1. **What's the API for trigger-response generators?**
+   - `shouldActivate(sessionState): boolean`?
+   - `getInterventionCandidates(): WeightedCard[]`?
+   - How to represent "I am currently intervening"?
+
+2. **How do strategies share expensive computations?**
+   - E.g., tag lookups (addressed in `todo-pipeline-optimization.md`)
+   - E.g., user mastery state
+   - Shared context object? Memoization layer?
+
+3. **What events should be observable?**
+   - Card shown, card answered (correct/incorrect), session start/end
+   - ELO change, tag mastery change
+   - User-initiated actions (skip, flag, etc.)
+
+4. **How much state is too much?**
+   - Stateless is simple but limited
+   - Full state enables sophisticated behavior but complexity cost
+   - Right level of state for different strategy types?
+
+---
+
+## Related Documents
+
+- `navigators-architecture.md` — Current architecture
+- `todo-naive-orchestration.md` — Pipeline assembly (prerequisite)
+- `todo-strategy-state-storage.md` — State persistence
+- `todo-evolutionary-orchestration.md` — Bandit selection vision
+- `todo-provenance.md` — Audit trail (transparency)

package/docs/navigators-architecture.md

@@ -0,0 +1,265 @@
+# Navigation Strategy Architecture
+
+## Overview
+
+The navigation strategy system selects and scores cards for study sessions. It uses a
+**Pipeline architecture** where generators produce candidates and filters transform scores.
+
+## Core Concepts
+
+### WeightedCard
+
+A card with a suitability score and audit trail:
+
+```typescript
+interface WeightedCard {
+  cardId: string;
+  courseId: string;
+  score: number;              // 0-1 suitability score
+  provenance: StrategyContribution[];  // Audit trail
+}
+
+interface StrategyContribution {
+  strategy: string;           // Type: 'elo', 'srs', 'hierarchyDefinition'
+  strategyName: string;       // Human-readable: "ELO (default)"
+  strategyId: string;         // Document ID: 'NAVIGATION_STRATEGY-ELO-default'
+  action: 'generated' | 'passed' | 'boosted' | 'penalized';
+  score: number;              // Score after this strategy
+  reason: string;             // Human-readable explanation
+}
+```
+
+### CardGenerator
+
+Produces candidate cards with initial scores:
+
+```typescript
+interface CardGenerator {
+  name: string;
+  getWeightedCards(limit: number, context: GeneratorContext): Promise<WeightedCard[]>;
+}
+```
+
+**Implementations:**
+- `ELONavigator` — New cards scored by ELO proximity to user skill
+- `SRSNavigator` — Review cards scored by overdueness and interval recency
+- `HardcodedOrderNavigator` — Fixed sequence defined by course author
+- `CompositeGenerator` — Merges multiple generators with frequency boost
+
+### CardFilter
+
+Transforms card scores (pure function, no side effects):
+
+```typescript
+interface CardFilter {
+  name: string;
+  transform(cards: WeightedCard[], context: FilterContext): Promise<WeightedCard[]>;
+}
+```
+
+**Implementations:**
+- `HierarchyDefinitionNavigator` — Gates cards by prerequisite mastery (score=0 if locked)
+- `InterferenceMitigatorNavigator` — Reduces scores for confusable content
+- `RelativePriorityNavigator` — Boosts scores for high-utility content
+- `createEloDistanceFilter()` — Penalizes cards far from user's current ELO
+
+### Pipeline
+
+Orchestrates generator and filters:
+
+```typescript
+class Pipeline {
+  constructor(
+    generator: CardGenerator,
+    filters: CardFilter[],
+    user: UserDBInterface,
+    course: CourseDBInterface
+  )
+
+  async getWeightedCards(limit: number): Promise<WeightedCard[]> {
+    const context = await this.buildContext();
+    let cards = await this.generator.getWeightedCards(fetchLimit, context);
+    
+    for (const filter of this.filters) {
+      cards = await filter.transform(cards, context);
+    }
+    
+    return cards.filter(c => c.score > 0)
+                .sort((a, b) => b.score - a.score)
+                .slice(0, limit);
+  }
+}
+```
+
+## Pipeline Assembly
+
+`PipelineAssembler` builds pipelines from strategy documents:
+
+```typescript
+const assembler = new PipelineAssembler();
+const { pipeline, warnings } = await assembler.assemble({
+  strategies: allStrategies,
+  user,
+  course,
+});
+```
+
+Assembly logic:
+1. Separate strategies into generators and filters by `NavigatorRole`
+2. Instantiate generators — wrap multiple in `CompositeGenerator`
+3. Instantiate filters — sorted alphabetically for determinism
+4. Return `Pipeline(generator, filters)`
+
+If no strategies are configured, `courseDB.createNavigator()` returns a default pipeline:
+```typescript
+Pipeline(
+  CompositeGenerator([ELONavigator, SRSNavigator]),
+  [eloDistanceFilter]
+)
+```
+
+## Score Semantics
+
+| Score | Meaning |
+|-------|---------|
+| 1.0 | Fully suitable |
+| 0.5 | Neutral |
+| 0.0 | Exclude (hard filter) |
+| 0.x | Proportional suitability |
+
+**All filters are multipliers.** This means:
+- Filter order doesn't affect final scores (multiplication is commutative)
+- Score 0 from any filter excludes the card
+- Filters are applied alphabetically for determinism
+
+## Provenance Tracking
+
+Each card's provenance shows how it was scored:
+
+```typescript
+provenance: [
+  {
+    strategy: 'elo',
+    strategyName: 'ELO (default)',
+    strategyId: 'NAVIGATION_STRATEGY-ELO-default',
+    action: 'generated',
+    score: 0.85,
+    reason: 'ELO distance 75 (card: 1025, user: 1100), new card'
+  },
+  {
+    strategy: 'hierarchyDefinition',
+    strategyName: 'Hierarchy: Phonics Basics',
+    strategyId: 'NAVIGATION_STRATEGY-hierarchy-phonics',
+    action: 'passed',
+    score: 0.85,
+    reason: 'Prerequisites met, tags: letter-sounds'
+  },
+  {
+    strategy: 'eloDistance',
+    strategyName: 'ELO Distance Filter',
+    strategyId: 'ELO_DISTANCE_FILTER',
+    action: 'penalized',
+    score: 0.72,
+    reason: 'ELO distance 150 (card: 1150, user: 1000) → 0.85x'
+  }
+]
+```
+
+Use `getCardOrigin(card)` to extract 'new', 'review', or 'failed' from provenance.
+
+## Creating New Strategies
+
+### Generator
+
+```typescript
+class MyGenerator extends ContentNavigator implements CardGenerator {
+  name = 'My Generator';
+
+  async getWeightedCards(limit: number, context?: GeneratorContext): Promise<WeightedCard[]> {
+    const candidates = await this.findCandidates(limit);
+    
+    return candidates.map(c => ({
+      cardId: c.id,
+      courseId: this.course.getCourseID(),
+      score: this.computeScore(c),
+      provenance: [{
+        strategy: 'myGenerator',
+        strategyName: this.name,
+        strategyId: this.strategyId || 'MY_GENERATOR',
+        action: 'generated',
+        score: this.computeScore(c),
+        reason: 'Explanation here, new card'
+      }]
+    }));
+  }
+
+  // Legacy methods - stub or implement for backward compat
+  async getNewCards() { return []; }
+  async getPendingReviews() { return []; }
+}
+```
+
+Register in `NavigatorRoles` as `NavigatorRole.GENERATOR`.
+
+### Filter
+
+```typescript
+class MyFilter extends ContentNavigator implements CardFilter {
+  name = 'My Filter';
+
+  async transform(cards: WeightedCard[], context: FilterContext): Promise<WeightedCard[]> {
+    return cards.map(card => {
+      const multiplier = this.computeMultiplier(card, context);
+      const newScore = card.score * multiplier;
+      const action = multiplier < 1 ? 'penalized' : multiplier > 1 ? 'boosted' : 'passed';
+      
+      return {
+        ...card,
+        score: newScore,
+        provenance: [...card.provenance, {
+          strategy: 'myFilter',
+          strategyName: this.name,
+          strategyId: this.strategyId || 'MY_FILTER',
+          action,
+          score: newScore,
+          reason: 'Explanation here'
+        }]
+      };
+    });
+  }
+
+  // Legacy methods - filters don't generate cards
+  async getWeightedCards() { throw new Error('Use transform() via Pipeline'); }
+  async getNewCards() { return []; }
+  async getPendingReviews() { return []; }
+}
+```
+
+Register in `NavigatorRoles` as `NavigatorRole.FILTER`.
+
+## File Reference
+
+| File | Purpose |
+|------|---------|
+| `core/navigators/index.ts` | `ContentNavigator`, `WeightedCard`, `NavigatorRole` |
+| `core/navigators/generators/types.ts` | `CardGenerator`, `GeneratorContext` |
+| `core/navigators/filters/types.ts` | `CardFilter`, `FilterContext` |
+| `core/navigators/Pipeline.ts` | Pipeline orchestration |
+| `core/navigators/PipelineAssembler.ts` | Builds Pipeline from strategy docs |
+| `core/navigators/CompositeGenerator.ts` | Merges multiple generators |
+| `core/navigators/elo.ts` | ELO generator |
+| `core/navigators/srs.ts` | SRS generator |
+| `core/navigators/hardcodedOrder.ts` | Fixed-order generator |
+| `core/navigators/hierarchyDefinition.ts` | Prerequisite filter |
+| `core/navigators/interferenceMitigator.ts` | Interference filter |
+| `core/navigators/relativePriority.ts` | Priority filter |
+| `core/navigators/filters/eloDistance.ts` | ELO distance filter |
+| `impl/couch/courseDB.ts` | `createNavigator()` entry point |
+
+## Related TODOs
+
+- `todo-pipeline-optimization.md` - Batch tag hydration for filter efficiency
+- `todo-strategy-authoring.md` - ux and dx for authoring strategies
+- todo-pipeline-optimization.md - 
+- todo-strategy-state-storage
+- todo-evolutionary-orchestration