stellavault 0.4.0 → 0.4.1

package/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## [0.4.1] - 2026-04-07
+
+### Features
+- **Binary File Ingest** — PDF (unpdf), DOCX (mammoth), PPTX (officeparser), XLSX (SheetJS) text extraction
+- **Auto Pipeline** — `ingest` now auto-runs `compile` → wiki generated automatically
+- **Web File Upload** — Drag & drop files in browser (POST /api/ingest/file, 50MB limit)
+- **Mobile/PWA** — Responsive IngestPanel, service worker registration, installable as app
+- **Express: stellavault draft** — Generate blog/report/outline from vault knowledge
+- **MCP generate-draft** — 20th MCP tool: Claude writes drafts using vault context (free in Claude Code)
+- **CLI --ai flag** — `stellavault draft --ai` uses Claude API for full AI-generated drafts
+- **Configurable folders** — Override raw/_wiki/_literature/ in .stellavault.json
+- **Multilingual embeddings** — paraphrase-multilingual-MiniLM-L12-v2 (50+ languages)
+
+### Changes
+- Binary files default to fleeting stage (Zettelkasten principle: all inputs start as fleeting)
+- Repositioned as "Self-compiling knowledge MCP server"
+- .npmignore excludes PDF (was 16MB in npm package)
+- MCP tools: 19 → 20
+
+### Fixes
+- CLI YouTube ingest now properly calls extractYouTubeContent
+- officeparser type cast fix (OfficeParserAST → string)
+- unpdf text array handling (pages returned as array, not string)
+
 ## [0.4.0] - 2026-04-06
 
 ### Features

package/README.md

@@ -1,124 +1,89 @@
 # Stellavault
 
-> **Notes die in folders. Stellavault keeps your knowledge alive.**
+> **Self-compiling knowledge MCP server** — ingest anything, auto-organize into Zettelkasten wiki, and let Claude access your entire knowledge base.
 
-Your Obsidian vault is more than files — it's a living network. Stellavault turns it into a self-compiling knowledge system that **discovers hidden connections**, **tracks fading memories**, **finds blind spots**, and **gives AI agents direct access to everything you know**.
+Drop a PDF, paste a YouTube link, type a thought — Stellavault compiles it into structured knowledge, connects the dots, and gives your AI agent full access through 20 MCP tools.
 
 <p align="center">
   <img src="images/screenshots/graph-dark-full.png" alt="3D Knowledge Graph" width="800" />
   <br><em>Your vault as a neural network. Clusters form constellations.</em>
 </p>
 
-## Why Stellavault?
+## Two Core Ideas
 
-| Problem | Stellavault Solution |
-|---------|---------------------|
-| Notes pile up but never become knowledge | **Self-compiling wiki** — raw notes auto-organized into linked concepts |
-| You forget what you wrote | **FSRS decay tracking** — see what's fading, get review suggestions |
-| Search finds words, not meaning | **Hybrid AI search** — BM25 + vector + RRF fusion, locally |
-| AI agents can't access your knowledge | **17 MCP tools** — Claude remembers everything you know |
-| No way to see the big picture | **3D neural graph** — constellations, heatmaps, timeline |
-
-## 5-Minute Setup
-
-```bash
-# Install
-npm install -g stellavault
-
-# Interactive setup (indexes your vault + verifies search)
-stellavault init
-
-# Launch 3D graph + API server
-stellavault graph
+**1. "Drop it and forget it"** (Karpathy's Self-Compiling Knowledge)
 ```
+Any input → auto-classify → raw/ → compile → wiki → connected knowledge
+```
+PDF, DOCX, YouTube, URL, text — everything goes through the same pipeline. You never manually organize.
 
-> **Prerequisites**: Node.js 20+ (`node --version`)
-
-## Give Your AI Agent Memory
-
+**2. "Claude knows what you know"** (MCP Integration)
 ```bash
 claude mcp add stellavault -- stellavault serve
 ```
+20 MCP tools give Claude direct access to search, ask, draft, and navigate your entire knowledge base.
 
-Now Claude Code can search your notes, track decisions, detect knowledge gaps, and generate daily briefings — using 17 MCP tools.
+## 5-Minute Setup
 
-## Screenshots
+```bash
+npm install -g stellavault
+stellavault init          # Interactive setup + vault indexing
+stellavault graph         # Launch 3D graph + API server
+```
 
-<p align="center">
-  <img src="images/screenshots/graph-search-highlight.png" alt="Semantic Search" width="800" />
-  <br><em>Search by meaning. Matching nodes pulse and glow.</em>
-</p>
+> **Prerequisites**: Node.js 20+
 
-<details>
-<summary>More screenshots</summary>
+## The Pipeline
 
-| Light Mode | Timeline Slider | Heatmap |
-|:---:|:---:|:---:|
-| <img src="images/screenshots/graph-light-mode.png" width="260" /> | <img src="images/screenshots/graph-timeline.png" width="260" /> | <img src="images/screenshots/graph-heatmap.png" width="260" /> |
+```
+Capture ──→ Organize ──→ Distill ──→ Express
 
-</details>
+stellavault ingest <anything>     # PDF, DOCX, URL, YouTube, text
+  → auto-extract text             # unpdf, mammoth, yt-dlp
+  → raw/ (fleeting)               # Zettelkasten inbox
+  → compile → _wiki/              # Auto: concepts + backlinks
+  → stellavault draft "topic"     # Blog, report, or outline
+```
 
-## Core Workflow
+### Ingest Anything
 
-```
-Capture → Compile → Connect → Review
-
-stellavault fleeting "idea"     # Instant capture
-stellavault ingest <url>        # Clip any URL/file
-stellavault compile             # Raw → structured wiki
-stellavault autopilot           # Full flywheel: inbox → compile → lint
-stellavault ask "question"      # Q&A with auto-filing (--save)
-stellavault lint                # Knowledge health check (score 0-100)
-```
+| Input | How |
+|-------|-----|
+| PDF, DOCX, PPTX, XLSX | `stellavault ingest report.pdf` — auto text extraction |
+| YouTube | `stellavault ingest https://youtu.be/...` — transcript + timestamps |
+| URL | `stellavault ingest https://...` — HTML → clean text |
+| Text | `stellavault ingest "quick thought"` |
+| Web UI | Drag & drop files in browser (mobile too) |
 
-## Daily Commands
+### Express: Get Knowledge Out
 
 ```bash
-stellavault brief               # Morning knowledge briefing
-stellavault decay               # What's fading from memory?
-stellavault learn               # AI-generated learning path
-stellavault digest --visual     # Weekly report with Mermaid charts
-stellavault review              # FSRS spaced repetition session
-stellavault gaps                # Knowledge gap detection
+stellavault draft "AI"                    # Rule-based scaffold (free)
+stellavault draft "AI" --ai               # Claude API writes full draft ($0.03)
+stellavault draft "AI" --format report    # Formal report format
+stellavault draft --format outline        # All-knowledge outline
 ```
 
-## Intelligence Features
-
-| Feature | Command | What it does |
-|---------|---------|-------------|
-| **FSRS Decay** | `sv decay` | Track memory strength with spaced repetition |
-| **Gap Detection** | `sv gaps` | Find missing connections between topics |
-| **Contradiction Detection** | `sv contradictions` | Spot conflicting statements |
-| **Duplicate Detection** | `sv duplicates` | Find redundant notes |
-| **Knowledge Lint** | `sv lint` | Health score + issues + suggestions |
-| **Learning Path** | `sv learn` | AI-recommended review order |
-| **Semantic Evolution** | MCP `get-evolution` | Track how ideas change over time |
-| **Code Linker** | MCP `link-code` | Connect code files to knowledge notes |
-| **Adaptive Search** | MCP `search` | Context-aware result reranking |
-
-## Zettelkasten Automation
+Or in Claude Code: *"Write a blog post about machine learning from my notes"* — Claude uses MCP `generate-draft` tool (free, no API key).
 
-Inspired by Luhmann + Karpathy's "Self-Compiling Zettelkasten":
+## Daily Commands
 
 ```bash
-stellavault fleeting "raw idea"          # Capture → raw/
-stellavault ingest https://article.com   # Any URL → auto-classified
-stellavault compile                      # Raw → wiki with concepts + backlinks
-stellavault promote file.md --to permanent  # Upgrade note stage
-stellavault autopilot                    # Full cycle: inbox → compile → lint → archive
+stellavault ask "What did I learn about X?"   # Q&A from vault
+stellavault brief                              # Morning knowledge briefing
+stellavault decay                              # What's fading from memory?
+stellavault lint                               # Health score (0-100)
+stellavault learn                              # AI learning path
+stellavault digest --visual                    # Weekly Mermaid chart report
 ```
 
-- **Frontmatter-first scanning** — 10x token reduction
-- **Luhmann index codes** — auto-assigned hierarchical numbering (1A → 1A1)
-- **Inbox Zero** — processed notes auto-archived
-- **Atomicity check** — detects notes with too many topics
-- **3-stage classification** — fleeting → literature → permanent
-
-## MCP Tools (17)
+## MCP Tools (20)
 
 | Tool | What it does |
 |------|-------------|
 | `search` | Hybrid search (BM25 + vector + RRF) |
+| `ask` | Q&A with optional vault filing |
+| `generate-draft` | Gather vault context for AI draft writing |
 | `get-document` | Full document with metadata |
 | `get-related` | Semantically similar documents |
 | `list-topics` | Topic cloud |
@@ -128,7 +93,6 @@ stellavault autopilot                    # Full cycle: inbox → compile → lin
 | `detect-gaps` | Knowledge gap analysis |
 | `get-evolution` | Semantic drift tracking |
 | `link-code` | Code-knowledge connections |
-| `ask` | Q&A with optional vault filing |
 | `create-knowledge-node` | AI creates wiki-quality notes |
 | `create-knowledge-link` | AI connects existing notes |
 | `log-decision` / `find-decisions` | Decision journal |
@@ -136,38 +100,53 @@ stellavault autopilot                    # Full cycle: inbox → compile → lin
 | `generate-claude-md` | Auto-generate CLAUDE.md |
 | `export` | JSON/CSV export |
 
-## 3D Visualization
+## Zettelkasten (Luhmann + Karpathy)
 
-- **Neural graph** — force-directed layout with cluster coloring
-- **Constellation view** — MST-based star patterns with LOD zoom
-- **Heatmap overlay** — activity score (cold → fire gradient)
-- **Timeline slider** — filter by creation/modification date
-- **Multiverse view** — see connected Federation peers
-- **Decay overlay** — visualize fading knowledge
-- **Dark/Light theme** — monochrome light mode with color-on-interaction
-- **Export** — PNG screenshots + WebM recording
+```bash
+stellavault fleeting "raw idea"                # → raw/
+stellavault ingest report.pdf                  # → auto text extract → raw/
+stellavault compile                            # → raw/ → _wiki/ (concepts + backlinks)
+stellavault promote note.md --to permanent     # Upgrade stage
+stellavault autopilot                          # Full cycle: inbox → compile → lint → archive
+```
 
-## Obsidian Plugin
+- **3-stage flow**: fleeting → literature → permanent
+- **Luhmann index codes**: auto-assigned (1A → 1A1)
+- **Frontmatter-first scanning**: 10x token reduction
+- **Configurable folders**: override raw/_wiki/_literature/ in `.stellavault.json`
+
+```json
+{
+  "vaultPath": "/path/to/vault",
+  "folders": {
+    "fleeting": "01-Inbox",
+    "literature": "02-Reading",
+    "permanent": "03-Notes",
+    "wiki": "04-Wiki"
+  }
+}
+```
 
-Manual install (community plugin review pending):
+## Intelligence
 
-1. Download `main.js`, `manifest.json`, `styles.css` from [GitHub Releases](https://github.com/Evanciel/stellavault-obsidian/releases/tag/0.1.0)
-2. Copy to `.obsidian/plugins/stellavault/`
-3. Enable in Settings > Community plugins
-4. **Run `stellavault graph` first** — the plugin connects to the API server
+| Feature | Command |
+|---------|---------|
+| FSRS Decay | `sv decay` — spaced repetition memory tracking |
+| Gap Detection | `sv gaps` — missing connections between topics |
+| Contradictions | `sv contradictions` — conflicting statements |
+| Duplicates | `sv duplicates` — redundant notes |
+| Learning Path | `sv learn` — AI review recommendations |
+| Code Linker | MCP `link-code` — connect code to knowledge |
 
-Features: semantic search modal, memory decay sidebar, learning path suggestions.
+## 3D Visualization
 
-## Architecture
-
-```
-stellavault/
-├── packages/
-│   ├── core/       Search, MCP, API, Intelligence, Federation, Cloud, Team, i18n
-│   ├── cli/        36+ commands
-│   ├── graph/      3D visualization (React Three Fiber)
-│   └── sync/       Notion ↔ Obsidian sync
-```
+- Neural graph with cluster coloring
+- Constellation view (MST star patterns)
+- Heatmap overlay (activity score)
+- Timeline slider (creation/modification filter)
+- Decay overlay (fading knowledge)
+- Dark/Light theme
+- Mobile responsive + PWA installable
 
 ## Tech Stack
 
@@ -175,46 +154,12 @@ stellavault/
 |-------|------|
 | Runtime | Node.js 20+ (ESM, TypeScript) |
 | Vector Store | SQLite-vec (local, no server) |
-| Embedding | all-MiniLM-L6-v2 (local, no API key) |
+| Embedding | paraphrase-multilingual-MiniLM-L12-v2 (local, 50+ languages) |
 | Search | BM25 + Cosine + RRF Fusion |
+| File Parsing | unpdf, mammoth, officeparser, SheetJS |
 | Memory | FSRS (Free Spaced Repetition Scheduler) |
-| 3D | React Three Fiber + Three.js + Zustand |
-| P2P | Hyperswarm (NAT traversal + DHT) |
-| AI | MCP (Model Context Protocol) |
-| Security | AES-256-GCM, HMAC-SHA256, Differential Privacy |
-
-## Troubleshooting
-
-<details>
-<summary>"Stellavault API server not found"</summary>
-
-The Obsidian plugin and 3D graph require the API server. Open a terminal in your vault:
-
-```bash
-stellavault graph
-```
-
-Default port: 3333. Change in `.stellavault.json`: `{ "mcp": { "port": 3334 } }`
-</details>
-
-<details>
-<summary>"No documents indexed"</summary>
-
-```bash
-stellavault index /path/to/your/vault
-```
-
-First indexing takes 2-5 minutes for large vaults (1000+ notes).
-</details>
-
-<details>
-<summary>Port conflict</summary>
-
-```bash
-# Use a different port
-stellavault graph --port 3334
-```
-</details>
+| 3D | React Three Fiber + Three.js |
+| AI | MCP (Model Context Protocol) + Anthropic SDK |
 
 ## License
 
@@ -223,5 +168,5 @@ MIT
 ## Links
 
 - [Obsidian Plugin](https://github.com/Evanciel/stellavault-obsidian)
-- [Wiki: Vault Structure Guide](docs/wiki/vault-structure.md)
-- [Wiki: Federation Guide](docs/wiki/federation-guide.md)
+- [npm](https://www.npmjs.com/package/stellavault)
+- [GitHub Releases](https://github.com/Evanciel/stellavault/releases)

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "stellavault",
-  "version": "0.4.0",
-  "description": "Turn your Obsidian vault into a 3D neural knowledge graph with AI-powered search, memory decay tracking, and MCP integration.",
+  "version": "0.4.1",
+  "description": "Self-compiling knowledge MCP server — ingest anything, auto-organize into Zettelkasten wiki, search with AI, and let Claude access your entire knowledge base.",
   "repository": {
     "type": "git",
     "url": "https://github.com/Evanciel/stellavault"

package/packages/cli/dist/commands/draft-cmd.d.ts

@@ -0,0 +1,5 @@
+export declare function draftCommand(topic: string | undefined, options: {
+    format?: string;
+    ai?: boolean;
+}): Promise<void>;
+//# sourceMappingURL=draft-cmd.d.ts.map

package/packages/cli/dist/commands/draft-cmd.js

@@ -0,0 +1,99 @@
+// stellavault draft — Express: 지식에서 초안 생성
+// 카파시 자가 컴파일 → 외부 표현의 출구
+// 두 가지 모드: rule-based (무료) + --ai (Claude API로 실제 글 생성)
+import chalk from 'chalk';
+import { loadConfig } from '@stellavault/core';
+import { generateDraft } from '@stellavault/core/intelligence/draft-generator';
+export async function draftCommand(topic, options) {
+    const config = loadConfig();
+    if (!config.vaultPath) {
+        console.error(chalk.red('No vault configured. Run `stellavault init` first.'));
+        process.exit(1);
+    }
+    const format = (options.format ?? 'blog');
+    try {
+        // Step 1: rule-based 초안 생성 (스캐폴딩 + 소스 수집)
+        const result = generateDraft(config.vaultPath, { topic, format }, config.folders);
+        if (options.ai) {
+            // Step 2: --ai 모드 → Claude API로 실제 글 생성
+            console.log(chalk.dim('  AI mode: generating with Claude...'));
+            await enhanceWithAI(config.vaultPath, result.filePath, topic ?? 'knowledge', format);
+        }
+        console.log(chalk.green(`Draft generated: ${result.title}`));
+        console.log(chalk.dim(`  Format: ${format}`));
+        console.log(chalk.dim(`  Mode: ${options.ai ? 'AI-enhanced (Claude)' : 'rule-based'}`));
+        console.log(chalk.dim(`  Saved: ${result.filePath}`));
+        console.log(chalk.dim(`  Words: ${result.wordCount}`));
+        console.log(chalk.dim(`  Sources: ${result.sourceCount} documents`));
+        if (result.concepts.length > 0) {
+            console.log(chalk.dim(`  Concepts: ${result.concepts.join(', ')}`));
+        }
+        console.log('');
+        if (!options.ai) {
+            console.log(chalk.dim('Tip: Use --ai to generate with Claude API, or use MCP generate-draft tool in Claude Code.'));
+        }
+    }
+    catch (err) {
+        console.error(chalk.red(err instanceof Error ? err.message : 'Draft generation failed'));
+        process.exit(1);
+    }
+}
+async function enhanceWithAI(vaultPath, draftPath, topic, format) {
+    const { readFileSync, writeFileSync } = await import('node:fs');
+    const { resolve } = await import('node:path');
+    const fullPath = resolve(vaultPath, draftPath);
+    const scaffold = readFileSync(fullPath, 'utf-8');
+    // Extract source excerpts from scaffold
+    const excerpts = scaffold
+        .split('\n')
+        .filter(l => l.startsWith('> '))
+        .map(l => l.slice(2))
+        .join('\n');
+    const apiKey = process.env.ANTHROPIC_API_KEY;
+    if (!apiKey) {
+        console.error(chalk.yellow('  ANTHROPIC_API_KEY not set. Falling back to rule-based draft.'));
+        console.error(chalk.yellow('  Set it with: export ANTHROPIC_API_KEY=sk-ant-...'));
+        return;
+    }
+    try {
+        const Anthropic = (await import('@anthropic-ai/sdk')).default;
+        const client = new Anthropic({ apiKey });
+        const message = await client.messages.create({
+            model: 'claude-sonnet-4-20250514',
+            max_tokens: 4096,
+            messages: [{
+                    role: 'user',
+                    content: `You are writing a ${format} about "${topic}" based ONLY on the following knowledge notes.
+Do not add information beyond what's in the notes. Write in the same language as the source material.
+Use [[title]] wikilink format for citations.
+
+SOURCE NOTES:
+${excerpts.slice(0, 8000)}
+
+Write a complete, well-structured ${format}. No TODO placeholders.`,
+                }],
+        });
+        const aiContent = message.content[0];
+        if (aiContent.type === 'text') {
+            const enhanced = `---
+title: "Draft: ${topic}"
+type: draft
+format: ${format}
+mode: ai-enhanced
+sources: ${excerpts.split('\n').length} excerpts
+created: ${new Date().toISOString()}
+---
+
+${aiContent.text}
+
+---
+*Generated by \`stellavault draft --ai\` using Claude API at ${new Date().toISOString()}*
+`;
+            writeFileSync(fullPath, enhanced, 'utf-8');
+        }
+    }
+    catch (err) {
+        console.error(chalk.yellow(`  AI enhancement failed: ${err instanceof Error ? err.message : 'unknown'}. Keeping rule-based draft.`));
+    }
+}
+//# sourceMappingURL=draft-cmd.js.map

package/packages/cli/dist/commands/ingest-cmd.js

@@ -2,7 +2,7 @@
 import chalk from 'chalk';
 import { loadConfig, ingest, promoteNote } from '@stellavault/core';
 import { readFileSync, existsSync } from 'node:fs';
-import { extname } from 'node:path';
+import { extname, resolve } from 'node:path';
 export async function ingestCommand(input, options) {
     if (!input) {
         console.error(chalk.yellow('Usage: stellavault ingest <url|file|text> [--tags t1,t2] [--stage fleeting|literature|permanent]'));
@@ -85,15 +85,46 @@ export async function ingestCommand(input, options) {
     else if (existsSync(input)) {
         // 파일
         const ext = extname(input).toLowerCase();
-        const fileContent = readFileSync(input, 'utf-8');
-        ingestInput = {
-            type: ext === '.pdf' ? 'pdf-text' : 'file',
-            content: fileContent,
-            tags,
-            stage,
-            title: options.title,
-            source: input,
-        };
+        const binaryExts = new Set(['.pdf', '.docx', '.pptx', '.xlsx', '.xls']);
+        if (binaryExts.has(ext)) {
+            // 바이너리 파일: 전용 파서로 텍스트 추출
+            try {
+                const { extractFileContent } = await import('@stellavault/core/intelligence/file-extractors');
+                const extracted = await extractFileContent(resolve(input));
+                console.log(chalk.dim(`  Extracted ${extracted.metadata.wordCount} words from ${ext} file`));
+                ingestInput = {
+                    type: extracted.sourceFormat,
+                    content: extracted.text,
+                    tags: [...tags, extracted.sourceFormat],
+                    stage, // 제텔카스텐: 모든 인풋은 fleeting에서 시작
+                    title: options.title ?? extracted.metadata.title,
+                    source: input,
+                };
+            }
+            catch (err) {
+                console.error(chalk.yellow(`Binary file extraction failed, saving as-is. (${err instanceof Error ? err.message : 'error'})`));
+                ingestInput = {
+                    type: 'file',
+                    content: readFileSync(input, 'utf-8'),
+                    tags,
+                    stage,
+                    title: options.title,
+                    source: input,
+                };
+            }
+        }
+        else {
+            // 텍스트 파일: 기존 동작
+            const fileContent = readFileSync(input, 'utf-8');
+            ingestInput = {
+                type: 'file',
+                content: fileContent,
+                tags,
+                stage,
+                title: options.title,
+                source: input,
+            };
+        }
     }
     else {
         // 플레인 텍스트
@@ -114,9 +145,8 @@ export async function ingestCommand(input, options) {
         console.log(chalk.dim(`  Index: ${result.indexCode}`));
     if (result.tags.length > 0)
         console.log(chalk.dim(`  Tags: ${result.tags.join(', ')}`));
+    console.log(chalk.dim('  Wiki: auto-compiled'));
     console.log('');
-    console.log(chalk.dim('Run `stellavault compile` to process into wiki.'));
-    console.log(chalk.dim('Run `stellavault autopilot` for full pipeline.'));
 }
 export async function promoteCommand(filePath, options) {
     const config = loadConfig();

package/packages/cli/dist/index.js

@@ -23,6 +23,7 @@ import { vaultAddCommand, vaultListCommand, vaultRemoveCommand, vaultSearchAllCo
 import { captureCommand } from './commands/capture-cmd.js';
 import { askCommand } from './commands/ask-cmd.js';
 import { compileCommand } from './commands/compile-cmd.js';
+import { draftCommand } from './commands/draft-cmd.js';
 import { lintCommand } from './commands/lint-cmd.js';
 import { fleetingCommand } from './commands/fleeting-cmd.js';
 import { ingestCommand, promoteCommand } from './commands/ingest-cmd.js';
@@ -139,6 +140,12 @@ program
     .option('-w, --wiki <dir>', 'Wiki output directory (default: _wiki/)')
     .option('-f, --force', 'Overwrite existing wiki files')
     .action((opts) => compileCommand(opts));
+program
+    .command('draft [topic]')
+    .description('Express: Generate a blog post, report, or outline draft from your knowledge')
+    .option('--format <type>', 'Output format: blog, report, outline (default: blog)')
+    .option('--ai', 'Use Claude API for AI-enhanced draft (requires ANTHROPIC_API_KEY)')
+    .action((topic, opts) => draftCommand(topic, opts));
 program
     .command('lint')
     .description('Knowledge health check — find gaps, duplicates, contradictions, stale notes')