mindlore 0.5.0 → 0.5.2

package/skills/mindlore-diary/SKILL.md

@@ -0,0 +1,76 @@
+---
+name: mindlore-diary
+description: LLM-powered session analysis — decisions, discoveries, frictions, learnings. Promotes episodes to semantic knowledge.
+---
+
+# /mindlore-diary
+
+## Scope
+
+Determine target using `getActiveMindloreDir()` logic:
+- If CWD has `.mindlore/` -> project scope
+- Otherwise -> global `~/.mindlore/`
+
+## Trigger
+
+`/mindlore-diary` or `/mindlore-log diary`
+
+## On Start — Read skill_memory
+
+```bash
+node dist/scripts/lib/skill-memory.js get mindlore-diary last_diary_date
+```
+If last_diary_date is today, warn: "Diary already ran today. Continue anyway?"
+
+## Flow
+
+1. Read active episodes: `WHERE status = 'active' AND source IN ('hook', 'diary')`
+2. Filter by time: default last 24h, or `--days N` flag
+3. Present summary: "Found N episodes spanning DATE1 to DATE2"
+4. LLM analyzes episodes for semantic patterns:
+   - **Decisions:** choices made, alternatives considered
+   - **Discoveries:** new insights, broken assumptions
+   - **Frictions:** recurring blockers, errors, slowdowns
+   - **Learnings:** techniques that worked, patterns validated
+   - **Preferences:** user workflow preferences detected
+   - **Events:** notable milestones, completions
+
+5. For each detected item, create enriched episode:
+   ```sql
+   INSERT INTO episodes (summary, body, kind, source, status, project, created_at)
+   VALUES (?, ?, ?, 'diary', 'active', ?, ?)
+   ```
+   Where kind = 'decision' | 'discovery' | 'friction' | 'learning' | 'preference' | 'event'
+
+6. Output structured report:
+   ```
+   -- Diary Raporu ({date}, {N} episode analiz edildi) --
+
+   Decisions ({count}):
+     - {summary}
+
+   Discoveries ({count}):
+     - {summary}
+
+   Frictions ({count}):
+     - {summary}
+
+   Learnings ({count}):
+     - {summary}
+
+   {M} yeni episode olusturuldu.
+   ```
+
+## On End — Write skill_memory
+
+```bash
+node dist/scripts/lib/skill-memory.js set mindlore-diary last_diary_date "$(date -I)"
+node dist/scripts/lib/skill-memory.js set mindlore-diary last_episode_count "{N}"
+```
+
+## Rules
+
+- NEVER create episodes without user seeing the analysis first
+- Each episode must have proper `kind` field — don't default everything to 'event'
+- Group related items — don't create 5 episodes for the same friction
+- Append to `log.md`: `| {date} | diary | {N} episodes created |`

package/skills/mindlore-ingest/SKILL.md

@@ -3,6 +3,8 @@ name: mindlore-ingest
 description: Add new knowledge sources to .mindlore/ (URL, text, file, PDF, GitHub repo)
 effort: medium
 allowed-tools: [Read, Write, Edit, Bash, Grep, Glob, Agent, WebFetch]
+context: fork
+agent: coder
 ---
 
 # /mindlore-ingest
@@ -23,60 +25,50 @@ User shares a URL, text, file, or says "kaynak ekle", "source ingest", "bu linki
 
 ## Modes
 
-### URL Mode
+### URL Mode (v0.5.2 — Zero-Token Pipeline)
 
-**Agent Delegation:** URL fetch + raw/sources yazımını subagent'a delege et (maliyet optimizasyonu).
+**Trigger:** Input starts with `http://` or `https://`
 
-1. Spawn an Agent with the following pattern:
+**Pre-check (before fetch):**
+```bash
+node dist/scripts/lib/skill-memory.js get mindlore-ingest last_ingest_urls
+```
+If URL already in the list, warn user: "This URL was ingested recently. Re-ingest?"
+
+**Flow:**
+
+1. **Fetch raw content (zero token):**
+   ```bash
+   node dist/scripts/fetch-raw.js "$URL" --out-dir "$MINDLORE_DIR/raw"
+   ```
+   Script output: `{ "saved": "/path/to/raw/2026-04-18-abc123.md", "chars": 14823, "method": "curl" }`
+
+2. **Read first 3000 chars (heading-aware truncation):**
+   - Read the saved raw file
+   - Find the last `##` heading before char 3000
+   - Truncate at that heading boundary (not mid-paragraph)
+   - If no heading found before 3000, truncate at last paragraph break
+
+3. **Write sources/ summary from truncated content:**
+   - Extract: title, description (first paragraph), key topics
+   - Generate frontmatter: slug, type: source, source_url, date_captured, tags, quality
+   - Write to `$MINDLORE_DIR/sources/{slug}.md`
+
+4. **Update INDEX.md** with new source entry
+
+5. **Update skill_memory:**
+   ```bash
+   node dist/scripts/lib/skill-memory.js set mindlore-ingest last_ingest_urls "$URL"
    ```
-   Agent({
-     description: "mindlore ingest: <slug>",
-     subagent_type: "researcher",
-     prompt: "[mindlore:ingest] <aşağıdaki talimatları buraya koy>"
-   })
+
+6. **Return to caller (this is all the ana session sees):**
+   ```json
+   { "source_id": "abc123", "title": "Extracted Title" }
    ```
 
-   Agent talimatları:
-   - Extract content from URL:
-     - If `markitdown` is available: `markitdown <url>` (best quality, zero tokens)
-     - Else: use `WebFetch` or `ctx_fetch_and_index`
-     - **YouTube URL** detected (`youtube.com` or `youtu.be`):
-       1. markitdown installed → `markitdown <url>` (includes transcript)
-       2. Else youtube-transcript npm → `hasYoutubeTranscript()` check
-       3. Else → return error, ask user to paste transcript
-   - Save raw capture to `.mindlore/raw/` with frontmatter:
-     ```yaml
-     ---
-     slug: source-name-kebab
-     type: raw
-     source_url: https://...
-     date_captured: YYYY-MM-DD
-     tags: [tag1, tag2]
-     ---
-     ```
-   - Summarize into `.mindlore/sources/` with full frontmatter:
-     ```yaml
-     ---
-     slug: source-name-kebab
-     type: source
-     title: Human Readable Title
-     source_url: https://...
-     source_type: github-repo|blog|docs|video|x-thread
-     date_captured: YYYY-MM-DD
-     tags: [tag1, tag2]
-     quality: high|medium|low
-     ingested: true
-     ---
-     ```
-   - Agent must report: created file paths, slug, quality assigned
-
-2. After agent returns — verify raw/ and sources/ files exist and have valid frontmatter
-3. Update relevant domain page(s) in `.mindlore/domains/` (max 2)
-4. Update `.mindlore/INDEX.md` stats line
-5. Append entry to `.mindlore/log.md`
-6. Run FTS5 re-index: `npm run index`
-
-**IMPORTANT:** The `[mindlore:ingest]` marker in the Agent prompt is required — it triggers the model-router hook to use the cost-optimized model (haiku by default).
+**Token budget:** ~2-3k tokens in fork context (vs ~40-50k before). Ana session: ~50 tokens.
+
+**Fallback:** If fetch-raw.js fails (network error, unsupported format), fall back to existing WebFetch-based flow with warning: "Zero-token fetch failed, using legacy flow (higher token cost)."
 
 ### Text Mode
 1. User pastes text directly
@@ -129,8 +121,14 @@ N source, N analysis, N total
 
 ## Post-Ingest Quality Gate
 
-After every ingest, verify all 6 checkpoints before reporting success:
+After every ingest, verify all 7 checkpoints before reporting success:
 
+0. **Duplicate check** — Ingest öncesi mevcut DB'de benzer içerik ara:
+   ```bash
+   node dist/scripts/lib/similarity.js "<title or first 100 chars>"
+   ```
+   Eğer score > 0.7 olan sonuç varsa KULLANICIYA SOR: "Bu içerik '${slug}' ile benzer görünüyor. Yine de eklensin mi?"
+   Kullanıcı onaylarsa devam et, yoksa atla.
 1. **raw/ file exists** — immutable capture written with frontmatter (slug, type, source_url)
 2. **sources/ summary exists** — processed summary with full frontmatter (slug, type, title, tags, quality, description)
 3. **INDEX.md updated** — stats line incremented, Recent section has new entry

package/skills/mindlore-log/SKILL.md

@@ -12,7 +12,9 @@ Determine target using `getActiveMindloreDir()` logic:
 
 ## Trigger
 
-`/mindlore-log <mode>` where mode is `log`, `diary`, `reflect`, `status`, or `save`.
+`/mindlore-log <mode>` where mode is `log`, `status`, or `save`.
+
+For diary analysis, use `/mindlore-diary`. For pattern extraction, use `/mindlore-reflect`.
 
 ## Modes
 
@@ -36,148 +38,6 @@ date: 2026-04-11
 4. Body: user's note as-is
 5. Append to `log.md`: `| {date} | log | {slug}.md |`
 
-### diary
-
-LLM-driven session analysis → enriched episodes in the episodes table.
-
-**Trigger:** User runs `/mindlore-log diary` or Stop hook asks "Diary analizi yapayım mı?"
-
-**Model:** `[mindlore:diary]` marker → sonnet (analysis needed)
-
-**Flow:**
-1. Open `~/.mindlore/mindlore.db`, ensure episodes table exists
-2. Find the latest bare session episode for current project: `WHERE kind = 'session' AND project = ? AND source = 'hook' ORDER BY created_at DESC LIMIT 1`
-3. Gather context:
-   - The bare episode's body (commits, files, read stats)
-   - Git log last 10 commits
-   - Decision-detector captures (if any in session)
-4. LLM analyzes and extracts structured episodes:
-   - **Decisions** → `kind: 'decision'` — architectural/tool/format choices
-   - **Discoveries** → `kind: 'discovery'` — assumption vs reality findings
-   - **Frictions** → `kind: 'friction'` — tool errors, blockers, recurring issues
-   - **Learnings** → `kind: 'learning'` — reusable knowledge
-   - **Preferences** → `kind: 'preference'` — user behavioral preferences
-   - **Events** → `kind: 'event'` — releases, incidents, milestones
-5. **Deduplication rule:** Each finding belongs to exactly ONE kind. Priority: `decision > discovery > friction > learning > preference > event`. Never write the same finding to multiple kinds.
-6. Present to user, get approval
-7. Write approved episodes to DB:
-   - `source: 'diary'`
-   - `parent_id: {bare_session_episode_id}` — links enriched episodes to source session
-   - `scope: 'project'` (default) or `'global'` if cross-project
-8. Optionally mirror to FTS5 for text search
-9. Append to `log.md`: `| {date} | diary | {N} episodes extracted from session |`
-
-**Rules:**
-- NEVER write episodes without user approval
-- parent_id always points to the source session episode
-- Each episode gets its own summary (max 100 chars) and body (markdown, unbounded)
-- entities field: JSON array of relevant file paths (max 10)
-
-### reflect
-
-LLM-driven pattern extraction from episodes → persistent learnings.
-
-**Flow (v0.4 — episodes-powered):**
-1. Read active episodes: `WHERE status = 'active' AND source IN ('hook', 'diary')`
-2. Optionally filter by time: `--days 7` (default 7), `--days 30`
-3. Present summary: "Found N episodes spanning DATE1 to DATE2"
-4. LLM analyzes episodes (not deltas) for patterns:
-   - Repeated decisions (same choice 2+ times)
-   - Recurring frictions (same blocker/error)
-   - Discovery patterns (assumptions that keep breaking)
-   - Workflow patterns that worked well
-5. **3-Tier Confidence Assessment:**
-   For each detected pattern, count occurrences across episodes:
-
-   | Tekrar | Tier | Aksiyon |
-   |--------|------|---------|
-   | 1x | Note | Sessiz — episode olarak kalır, raporda göster |
-   | 2x | Learning | `kind: learning` episode oluştur, learnings/ dosyasına yaz |
-   | 3x+ | Nomination | `kind: nomination, status: staged, source: reflect` episode oluştur |
-
-6. **Structured report output:**
-
-```
-── Reflect Raporu (son {days} gün, {N} episode) ──
-
-Friction ({count}):
-  - {summary} — {repeat_count}x tekrar
-
-Discoveries ({count}):
-  - {summary}
-
-Decisions ({count}):
-  - {summary}
-
-Patterns:
-  - "CO-EVOLUTION sync hatası" → 3x tekrar → NOMINATION (staged)
-  - "ESM import sorunu" → 2x tekrar → LEARNING
-  - "Test mock karmaşıklığı" → 1x → NOTE
-
-Önerilen:
-  [ ] {rule} ({repeat_count}x, {confidence} confidence)
-```
-
-7. **Nomination oluşturma (3x+ tekrar):**
-   - `kind: nomination`, `status: staged`, `source: reflect`
-   - Body formatı:
-     ```markdown
-     ## Target: learnings
-     ## Rule
-     YAPMA: Schema değişikliğinde tek dosyayı güncelleme — CO-EVOLUTION sync zorunlu
-     ## Evidence
-     - ep-xxx: episodes.ts güncellendi ama common.cjs unutuldu (2026-04-10)
-     - ep-yyy: Aynı hata tekrar (2026-04-12)
-     - ep-zzz: Test'te yakalandı (2026-04-13)
-     ## Confidence
-     3x tekrar, 3 gün içinde
-     ```
-   - Target options: `learnings` | `claude.md` | `domain:{slug}`
-
-8. **Pending nominations check:**
-   Reflect başlarken staged nomination'ları kontrol et:
-   ```sql
-   SELECT id, summary, body, created_at FROM episodes
-   WHERE kind = 'nomination' AND status = 'staged' AND project = ?
-   ORDER BY created_at ASC
-   ```
-   Varsa kullanıcıya sun:
-   ```
-   ── Bekleyen Nomination'lar ({N} adet) ──
-   1. "CO-EVOLUTION sync zorunlu" (staged 2 gün önce)
-      Target: learnings | Confidence: 3x
-   2. "Test before commit" (staged 5 gün önce)
-      Target: claude.md | Confidence: 4x
-
-   Onaylamak istediğin numara(lar)ı seç, veya 'skip' de:
-   ```
-
-9. **Nomination approval flow:**
-   Kullanıcı onaylarsa:
-   - `status: staged → approved`
-   - Target'a göre yaz:
-     - `learnings` → ilgili `learnings/{topic}.md` dosyasına YAPMA/BEST PRACTICE ekle
-     - `claude.md` → ilgili projenin CLAUDE.md'sine kural ekle (kullanıcıya göster, onay al)
-     - `domain:{slug}` → ilgili domain sayfasına ekle
-   Kullanıcı reddederse:
-   - `status: staged → rejected`
-   - Body'ye `## Rejection Reason\n{kullanıcı açıklaması}` ekle
-
-10. User approves new learnings → write to `learnings/{topic}.md`
-11. Format: `YAPMA:` / `BEST PRACTICE:` / `KRITIK:` prefixed rules
-12. Update relevant domain page if pattern relates to an existing domain
-13. Mark processed episodes: future reflect skips already-processed timeranges
-14. Append to `log.md`: `| {date} | reflect | {N} episodes processed, {M} learnings written |`
-
-**Fallback:** Also reads non-archived delta files if episodes table is empty (backward compat with v0.3 deltas).
-
-**Rules:**
-- NEVER write learnings or nominations without user approval
-- Group related patterns into existing topic files (don't create one file per pattern)
-- Reflect scans both project + global diary/ in `--all` mode
-- Deduplication: same pattern found in both episodes and deltas → episodes win
-- Nominations with `status: staged` are hidden from default queries — only reflect sees them
-
 ### status
 
 Show recent session summary.

package/skills/mindlore-reflect/SKILL.md

@@ -0,0 +1,104 @@
+---
+name: mindlore-reflect
+description: Pattern extraction from episodes — 3-tier confidence, nomination pipeline, CLAUDE.md update proposals.
+---
+
+# /mindlore-reflect
+
+## Scope
+
+Scans both project + global `~/.mindlore/` diary/ for patterns.
+
+## Trigger
+
+`/mindlore-reflect` or `/mindlore-log reflect`
+
+## On Start — Check pending nominations + skill_memory
+
+```bash
+node dist/scripts/lib/skill-memory.js get mindlore-reflect last_reflect_date
+node dist/scripts/lib/skill-memory.js get mindlore-reflect nomination_count
+```
+
+Check pending nominations:
+```sql
+SELECT id, summary, body, created_at FROM episodes
+WHERE kind = 'nomination' AND status = 'staged' AND project = ?
+ORDER BY created_at ASC
+```
+
+If pending nominations exist, present them first:
+```
+-- Bekleyen Nomination'lar ({N} adet) --
+1. "{summary}" (staged {days} gun once)
+   Target: learnings | Confidence: 3x
+
+Onaylamak istediklerini sec, veya 'skip':
+```
+
+## Flow
+
+1. Read active episodes: `WHERE status = 'active' AND source IN ('hook', 'diary')`
+2. Filter by time: `--days 7` (default), `--days 30`
+3. Present summary: "Found N episodes spanning DATE1 to DATE2"
+4. LLM analyzes episodes for recurring patterns:
+   - Repeated decisions (same choice 2+ times)
+   - Recurring frictions (same blocker/error)
+   - Discovery patterns (assumptions that keep breaking)
+   - Workflow patterns that worked well
+
+5. **3-Tier Confidence Assessment:**
+
+   | Tekrar | Tier | Aksiyon |
+   |--------|------|---------|
+   | 1x | Note | Sessiz — episode olarak kalir, raporda goster |
+   | 2x | Learning | `kind: learning` episode olustur, learnings/ dosyasina yaz |
+   | 3x+ | Nomination | `kind: nomination, status: staged, source: reflect` episode olustur |
+
+6. **Structured report:**
+   ```
+   -- Reflect Raporu (son {days} gun, {N} episode) --
+
+   Friction ({count}):
+     - {summary} — {repeat_count}x tekrar
+
+   Discoveries ({count}):
+     - {summary}
+
+   Patterns:
+     - "{pattern}" -> 3x tekrar -> NOMINATION (staged)
+     - "{pattern}" -> 2x tekrar -> LEARNING
+     - "{pattern}" -> 1x -> NOTE
+
+   Onerilen:
+     [ ] {rule} ({repeat_count}x, {confidence} confidence)
+   ```
+
+7. **Nomination creation (3x+ tekrar):**
+   ```sql
+   INSERT INTO episodes (summary, body, kind, status, source, project, created_at)
+   VALUES (?, ?, 'nomination', 'staged', 'reflect', ?, ?)
+   ```
+
+8. **Nomination approval flow:**
+   User approves -> `status: staged -> approved` -> write to target:
+   - `learnings` -> `learnings/{topic}.md`
+   - `claude.md` -> project CLAUDE.md'ye kural ekle
+   - `domain:{slug}` -> ilgili domain sayfasina ekle
+
+   User rejects -> `status: staged -> rejected` + rejection reason
+
+## On End — Write skill_memory
+
+```bash
+node dist/scripts/lib/skill-memory.js set mindlore-reflect last_reflect_date "$(date -I)"
+node dist/scripts/lib/skill-memory.js set mindlore-reflect nomination_count "{staged_count}"
+```
+
+## Rules
+
+- NEVER write learnings or nominations without user approval
+- Group related patterns into existing topic files
+- Reflect READS episodes, diary WRITES episodes — clear separation
+- Mark processed episodes so future reflect skips them
+- Append to `log.md`: `| {date} | reflect | {N} episodes processed, {M} learnings written |`

package/templates/config.json

@@ -1,5 +1,5 @@
 {
-  "version": "0.5.0",
+  "version": "0.5.2",
   "models": {
     "ingest": "haiku",
     "evolve": "sonnet",
@@ -26,5 +26,10 @@
     "ftsWeight": 0.4,
     "vecWeight": 0.6,
     "k": 60
+  },
+  "tokenBudget": {
+    "sessionInject": 2000,
+    "searchResults": 1500,
+    "perResult": 500
   }
 }