@gmickel/gno 0.3.0 → 0.3.5

package/README.md

@@ -1,256 +1,262 @@
-# GNO: Your Local Second Brain
+# GNO
 
-**Index, Search, and Synthesize Your Entire Digital Life.**
+**Your Local Second Brain** — Index, search, and synthesize your entire digital life.
 
-GNO is a **Local Knowledge Engine** designed for privacy-conscious individuals and AI agents. It indexes your notes, code, documents (Markdown, PDF, Office, and more), and meeting transcripts, providing lightning-fast semantic search and AI-powered answers—all on your machine.
+[![npm](./assets/badges/npm.svg)](https://www.npmjs.com/package/@gmickel/gno)
+[![MIT License](./assets/badges/license.svg)](./LICENSE)
+[![Website](./assets/badges/website.svg)](https://gno.sh)
+[![Twitter](./assets/badges/twitter.svg)](https://twitter.com/gmickel)
+
+GNO is a local knowledge engine for privacy-conscious developers and AI agents. Index your notes, code, PDFs, and Office docs. Get hybrid search (BM25 + vector + reranking) and AI-powered answers—all running 100% on your machine.
 
 ---
 
-## ✨ Key Features
+## Contents
 
-*   **Universal Indexing**: Effortlessly ingest and search across Markdown, PDF, DOCX, XLSX, PPTX, and plain text files.
-*   **Hybrid Search Pipeline**: Combines **BM25 keyword search** with **vector semantic search** and **AI re-ranking** for unparalleled retrieval accuracy.
-*   **Local LLM Integration**: Get grounded AI answers with citations using **node-llama-cpp** and auto-downloaded GGUF models. No external services, maximum privacy.
-*   **Agent-First Design (MCP)**: Seamlessly integrate GNO with AI agents via the Model Context Protocol (MCP) server.
-*   **Deterministic Output**: Stable, schema-driven JSON, file-line, and markdown outputs for reliable scripting.
-*   **Multilingual Support**: Robust handling of multiple languages in indexing and retrieval.
-*   **Privacy-Preserving**: All processing happens locally. Your data never leaves your device.
-*   **World-Class Engineering**: Spec-driven development, rigorous testing, and eval gates ensure reliability and quality.
+- [Quick Start](#quick-start)
+- [Installation](#installation)
+- [Search Modes](#search-modes)
+- [Agent Integration](#agent-integration)
+- [How It Works](#how-it-works)
+- [Features](#features)
+- [Local Models](#local-models)
+- [Architecture](#architecture)
+- [Development](#development)
 
 ---
 
-## 🚀 Quick Start
+## Quick Start
 
-Get searching in minutes with the 3-command workflow:
+```bash
+gno init ~/notes --name notes    # Point at your docs
+gno index                        # Build search index
+gno query "auth best practices"  # Hybrid search
+gno ask "summarize the API" --answer  # AI answer with citations
+```
 
-1.  **Initialize your knowledge base**:
-    ```sh
-    # Create a collection for your notes (adjust path and name as needed)
-    gno init ~/my-notes --name notes --pattern "**/*.md"
+---
 
-    # Full index: sync files + generate embeddings
-    gno index
-    ```
+## Installation
 
-2.  **Ask a question**:
-    ```sh
-    # Get a direct, cited answer from your documents
-    gno ask "What are the best practices for API authentication?" --collection notes
+### Install GNO
 
-    # Search with keywords or natural language
-    gno query "Q4 planning meeting summary" --collection notes
-    ```
+Requires [Bun](https://bun.sh/) >= 1.0.0.
 
-3.  **Explore your data**:
-    ```sh
-    # Retrieve specific document content
-    gno get "notes/2024-01-15.md"
+```bash
+bun install -g @gmickel/gno
+```
 
-    # Get results in a machine-readable format for agents
-    gno search "project deadlines" --json -n 10
-    ```
+**macOS**: Vector search requires Homebrew SQLite:
 
----
+```bash
+brew install sqlite3
+```
+
+Verify everything works:
 
-## 🧠 For Humans & AI Agents
+```bash
+gno doctor
+```
 
-GNO is built for both worlds:
+### Connect to AI Agents
 
-*   **For Humans**: A powerful, yet intuitive CLI to quickly find information, get answers, and explore your local knowledge base.
-*   **For AI Agents**: Exposes a stable MCP server and structured output formats (`--json`, `--files`) for seamless integration with LLMs and agentic workflows.
+#### MCP Server (Claude Desktop, Cursor, Zed, etc.)
 
----
+One command to add GNO to your AI assistant:
 
-## 🔎 Search Modes
+```bash
+gno mcp install                      # Claude Desktop (default)
+gno mcp install --target cursor      # Cursor
+gno mcp install --target claude-code # Claude Code CLI
+gno mcp install --target zed         # Zed
+gno mcp install --target windsurf    # Windsurf
+gno mcp install --target codex       # OpenAI Codex CLI
+gno mcp install --target opencode    # OpenCode
+gno mcp install --target amp         # Amp
+gno mcp install --target lmstudio    # LM Studio
+gno mcp install --target librechat   # LibreChat
+```
 
-GNO offers multiple search strategies to suit your needs:
+Check status: `gno mcp status`
 
-| Command     | Mode           | Description                                                     | Best For                                         |
-| :---------- | :------------- | :-------------------------------------------------------------- | :----------------------------------------------- |
-| `gno search`| **BM25**       | Fast, keyword-based full-text search.                           | Exact phrase matching, known terms.              |
-| `gno vsearch`| **Vector**     | Semantic search based on meaning, not just keywords.            | Natural language queries, conceptual understanding. |
-| `gno query` | **Hybrid**     | Combines BM25 and Vector search with LLM reranking and fusion.  | Highest accuracy, nuanced understanding.         |
-| `gno ask`   | **RAG-focused**| Hybrid search providing a synthesized, cited answer from results. | Getting direct answers to complex questions.     |
+#### Skills (Claude Code, Codex, OpenCode)
 
----
+Skills integrate via CLI—no MCP overhead:
 
-## 🤖 Agent Integration
+```bash
+gno skill install --scope user       # User-wide
+gno skill install --target codex     # Codex
+gno skill install --target all       # Both Claude + Codex
+```
 
-GNO is designed to be the knowledge backbone for your AI agents.
+> **Full setup guide**: [MCP Integration](https://gno.sh/docs/MCP/) · [CLI Reference](https://gno.sh/docs/CLI/)
 
-### CLI Output Formats
+---
 
-Use `--json` or `--files` for machine-readable output:
+## Search Modes
 
-```sh
-# Get JSON results for LLM processing
-gno query "meeting notes on user feedback" --json -n 5
+| Command | Mode | Best For |
+|:--------|:-----|:---------|
+| `gno search` | BM25 | Exact phrases, code identifiers |
+| `gno vsearch` | Vector | Natural language, concepts |
+| `gno query` | Hybrid | Best accuracy (BM25 + vector + reranking) |
+| `gno ask --answer` | RAG | Direct answers with citations |
 
-# Get file paths and scores for agent tool use
-gno search "API design" --files --min-score 0.3
+```bash
+gno search "handleAuth"              # Find exact matches
+gno vsearch "error handling patterns" # Semantic similarity
+gno query "database optimization"    # Full pipeline
+gno ask "what did we decide" --answer # AI synthesis
 ```
 
-### Skill Installation (Recommended for Claude Code/Codex/OpenCode)
+Output formats: `--json`, `--files`, `--csv`, `--md`, `--xml`
 
-Skills integrate via CLI - the agent runs GNO commands directly. No MCP overhead, no context pollution.
+---
 
-```bash
-gno skill install --scope user    # User-wide for Claude Code
-gno skill install --target codex  # For Codex
-gno skill install --target all    # Both
-```
+## Agent Integration
 
-After install, restart your agent. It will detect GNO and can search your indexed documents.
+### MCP Server
 
-### MCP Server (For Claude Desktop/Cursor)
+GNO exposes 6 tools via [Model Context Protocol](https://modelcontextprotocol.io):
 
-Exposes an MCP server for GUI-based AI applications.
+| Tool | Description |
+|:-----|:------------|
+| `gno_search` | BM25 keyword search |
+| `gno_vsearch` | Vector semantic search |
+| `gno_query` | Hybrid search (recommended) |
+| `gno_get` | Retrieve document by ID |
+| `gno_multi_get` | Batch document retrieval |
+| `gno_status` | Index health check |
 
-**Tools Exposed:**
-*   `gno_search` (BM25)
-*   `gno_vsearch` (Vector)
-*   `gno_query` (Hybrid)
-*   `gno_get` (Document retrieval)
-*   `gno_multi_get` (Batch retrieval)
-*   `gno_status` (Index health)
+**Design**: MCP tools are retrieval-only. Your AI assistant (Claude, GPT-4) synthesizes answers from retrieved context—best retrieval (GNO) + best reasoning (your LLM).
 
-**Example Claude Desktop Configuration** (`~/Library/Application Support/Claude/claude_desktop_config.json`):
+### Skills
 
-```json
-{
-  "mcpServers": {
-    "gno": {
-      "command": "gno",
-      "args": ["mcp"]
-    }
-  }
-}
+Skills add GNO search to Claude Code/Codex without MCP protocol overhead:
+
+```bash
+gno skill install --scope user
 ```
 
-*(Adjust path and `mcpServers` key based on your agent's configuration.)*
+Then ask your agent: *"Search my notes for the auth discussion"*
 
----
+> **Detailed docs**: [MCP Integration](https://gno.sh/docs/MCP/) · [Use Cases](https://gno.sh/docs/USE-CASES/)
 
-## ⚙️ How It Works: The GNO Pipeline
+---
 
-GNO employs a sophisticated, multi-stage retrieval process for optimal results:
+## How It Works
 
 ```mermaid
 graph TD
-    A[User Query] --> B(Query Expansion);
-    B --> C{Lexical Variants};
-    B --> D{Semantic Variants};
-    B --> E{Optional HyDE};
-    A --> F[Original Query];
-
-    C --> G(BM25 Retrieval);
-    D --> H(Vector Search);
-    E --> H;
-    F --> G;
-    F --> H;
-
-    G --> I(Ranked List 1);
-    H --> J(Ranked List 2);
-    I --> K{RRF Fusion + Bonus};
-    J --> K;
-
-    K --> L(Top Candidates);
-    L --> M(LLM Re-ranking);
-    M --> N(Position-Aware Blending);
-    N --> O(Final Results);
-
-    subgraph "Search Stages"
-        B; C; D; E; F; G; H; I; J; K; L; M; N; O;
-    end
+    A[User Query] --> B(Query Expansion)
+    B --> C{Lexical Variants}
+    B --> D{Semantic Variants}
+    B --> E{HyDE Passage}
+
+    C --> G(BM25 Search)
+    D --> H(Vector Search)
+    E --> H
+    A --> G
+    A --> H
+
+    G --> I(Ranked Results)
+    H --> J(Ranked Results)
+    I --> K{RRF Fusion}
+    J --> K
+
+    K --> L(Top 20 Candidates)
+    L --> M(Cross-Encoder Rerank)
+    M --> N[Final Results]
 ```
 
-### Search Pipeline Details:
+1. **Query Expansion** — LLM generates lexical variants, semantic rephrases, and a [HyDE](https://arxiv.org/abs/2212.10496) passage
+2. **Parallel Retrieval** — BM25 + vector search run concurrently on all variants
+3. **Fusion** — Reciprocal Rank Fusion merges results with position-based scoring
+4. **Reranking** — Cross-encoder rescores top 20, blended with fusion scores
 
-1.  **Query Expansion**: Generates alternative queries (lexical and semantic) and an optional synthetic "HyDE" document using a local LLM for richer retrieval.
-2.  **Parallel Retrieval**: Executes BM25 (keyword) and Vector (semantic) searches concurrently.
-3.  **Fusion**: Combines results using Reciprocal Rank Fusion (RRF) with a weighted boost for original query matches and a top-rank bonus.
-4.  **Re-ranking**: An LLM-based cross-encoder re-scores the top candidates for final relevance.
-5.  **Blending**: Dynamically adjusts the mix of retrieval vs. reranked scores based on rank position to preserve accuracy.
-
-**Score Normalization**: Raw scores from FTS, vector distance, and reranker are normalized to a 0-1 scale for consistent fusion.
+> **Deep dive**: [How Search Works](https://gno.sh/docs/HOW-SEARCH-WORKS/)
 
 ---
 
-## 📦 Installation
+## Features
 
-Requires **Bun** >= 1.0.0.
+| Feature | Description |
+|:--------|:------------|
+| **Hybrid Search** | BM25 + vector + RRF fusion + cross-encoder reranking |
+| **Multi-Format** | Markdown, PDF, DOCX, XLSX, PPTX, plain text |
+| **Local LLM** | AI answers via llama.cpp—no API keys |
+| **Privacy First** | 100% offline, zero telemetry, your data stays yours |
+| **MCP Server** | Works with Claude Desktop, Cursor, Zed, + 8 more |
+| **Collections** | Organize sources with patterns, excludes, contexts |
+| **Multilingual** | 30+ languages, auto-detection, cross-lingual search |
+| **Incremental** | SHA-256 tracking—only changed files re-indexed |
 
-```sh
-# Install globally
-bun install -g @gmickel/gno
-```
+---
 
-**macOS users**: For optimal vector search performance, install Homebrew SQLite:
-```sh
-brew install sqlite3
-```
+## Local Models
 
-Verify your installation:
-```sh
-gno doctor
-```
+Models auto-download on first use to `~/.cache/gno/models/`.
 
----
+| Model | Purpose | Size |
+|:------|:--------|:-----|
+| bge-m3 | Embeddings (1024-dim, multilingual) | ~500MB |
+| bge-reranker-v2-m3 | Cross-encoder reranking | ~700MB |
+| Qwen/SmolLM | Query expansion + AI answers | ~600MB-1.2GB |
 
-## 🏠 Local LLM Models
+### Model Presets
 
-GNO runs embeddings, reranking, and query expansion locally using GGUF models via `node-llama-cpp`. Models are automatically downloaded and cached on first use in `~/.cache/gno/models/`.
+| Preset | Disk | Best For |
+|:-------|:-----|:---------|
+| `slim` | ~1GB | Fast, lower quality |
+| `balanced` | ~2GB | Good balance (default) |
+| `quality` | ~2.5GB | Best answers |
 
-| Model                 | Purpose           | Size (approx.) |
-| :-------------------- | :---------------- | :------------- |
-| `bge-m3`              | Multilingual Embeddings | ~500MB         |
-| `bge-reranker-v2-m3`  | Cross-Encoder Re-ranking | ~700MB         |
-| `Qwen-Instruct`       | Query Expansion / HyDE | ~600MB         |
+```bash
+gno models use balanced
+gno models pull --all
+```
 
-*(Specific GGUF versions are pinned for stability.)*
+> **Configuration**: [Model Setup](https://gno.sh/docs/CONFIGURATION/)
 
 ---
 
-## 📜 Architecture Overview
-
-GNO follows a layered, Ports and Adapters architecture for maintainability and testability:
+## Architecture
 
 ```
-┌───────────────────────────────────────────────────────────┐
-│                       GNO CLI / MCP                       │
-├───────────────────────────────────────────────────────────┤
-│ Ports: Converter, Store, Embedding, Generation, Rerank... │
-├───────────────────────────────────────────────────────────┤
-│ Adapters: SQLite, FTS5, sqlite-vec, node-llama-cpp, CLI   │
-├───────────────────────────────────────────────────────────┤
-│ Core Domain: Identity, Mirrors, Chunking, Retrieval       │
-└───────────────────────────────────────────────────────────┘
+┌─────────────────────────────────────────────────┐
+│                 GNO CLI / MCP                   │
+├─────────────────────────────────────────────────┤
+│  Ports: Converter, Store, Embedding, Rerank    │
+├─────────────────────────────────────────────────┤
+│  Adapters: SQLite, FTS5, sqlite-vec, llama-cpp │
+├─────────────────────────────────────────────────┤
+│  Core: Identity, Mirrors, Chunking, Retrieval  │
+└─────────────────────────────────────────────────┘
 ```
 
+> **Details**: [Architecture](https://gno.sh/docs/ARCHITECTURE/)
+
 ---
 
-## 💻 Development
+## Development
 
 ```bash
-# Clone the repository
-git clone https://github.com/gmickel/gno.git
-cd gno
-
-# Install dependencies
+git clone https://github.com/gmickel/gno.git && cd gno
 bun install
-
-# Run tests
 bun test
-
-# Lint and format code
-bun run lint
-
-# Type check
-bun run typecheck
+bun run lint && bun run typecheck
 ```
 
+> **Contributing**: [CONTRIBUTING.md](.github/CONTRIBUTING.md)
+
 ---
 
-## 📄 License
+## License
+
+[MIT](./LICENSE)
+
+---
 
-[MIT License](./LICENSE)
+<p align="center">
+  made with ❤️ by <a href="https://twitter.com/gmickel">@gmickel</a>
+</p>

package/assets/badges/license.svg

@@ -0,0 +1,12 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="85" height="28" viewBox="0 0 85 28">
+  <defs>
+    <linearGradient id="bg-lic" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#1a1a2e"/>
+      <stop offset="100%" style="stop-color:#0f0f1a"/>
+    </linearGradient>
+  </defs>
+  <rect width="85" height="28" rx="6" fill="url(#bg-lic)"/>
+  <rect x="1" y="1" width="83" height="26" rx="5" fill="none" stroke="#2dd4bf" stroke-opacity="0.3"/>
+  <text x="10" y="18" font-family="ui-monospace,SFMono-Regular,Menlo,Monaco,monospace" font-size="11" fill="#94a3b8">MIT</text>
+  <text x="34" y="18" font-family="ui-monospace,SFMono-Regular,Menlo,Monaco,monospace" font-size="11" fill="#2dd4bf">License</text>
+</svg>

package/assets/badges/npm.svg

@@ -0,0 +1,13 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="66" height="28" viewBox="0 0 66 28">
+  <defs>
+    <linearGradient id="bg-npm" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#1a1a2e"/>
+      <stop offset="100%" style="stop-color:#0f0f1a"/>
+    </linearGradient>
+  </defs>
+  <rect width="66" height="28" rx="6" fill="url(#bg-npm)"/>
+  <rect x="1" y="1" width="64" height="26" rx="5" fill="none" stroke="#cb3837" stroke-opacity="0.4"/>
+  <rect x="8" y="7" width="32" height="14" rx="3" fill="#cb3837"/>
+  <text x="13" y="18" font-family="ui-monospace,SFMono-Regular,Menlo,Monaco,monospace" font-size="10" font-weight="700" fill="#fff">npm</text>
+  <text x="46" y="18" font-family="ui-monospace,SFMono-Regular,Menlo,Monaco,monospace" font-size="12" fill="#e2e8f0">↗</text>
+</svg>

package/assets/badges/twitter.svg

@@ -0,0 +1,22 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="130" height="28" viewBox="0 0 130 28">
+  <defs>
+    <linearGradient id="bg2" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#1a1a2e"/>
+      <stop offset="100%" style="stop-color:#0f0f1a"/>
+    </linearGradient>
+    <filter id="glow2">
+      <feGaussianBlur stdDeviation="0.5" result="coloredBlur"/>
+      <feMerge>
+        <feMergeNode in="coloredBlur"/>
+        <feMergeNode in="SourceGraphic"/>
+      </feMerge>
+    </filter>
+  </defs>
+  <rect width="130" height="28" rx="6" fill="url(#bg2)"/>
+  <rect x="1" y="1" width="128" height="26" rx="5" fill="none" stroke="#2dd4bf" stroke-opacity="0.3"/>
+  <!-- X logo -->
+  <g transform="translate(10, 7)" fill="#e2e8f0">
+    <path d="M10.5 1.5L6.75 6.5L10.75 12.5H8.5L5.75 8.5L2.5 12.5H0.25L4.25 7L0.5 1.5H2.75L5.25 5.25L8.25 1.5H10.5Z" transform="scale(1.1)"/>
+  </g>
+  <text x="32" y="18" font-family="ui-monospace,SFMono-Regular,Menlo,Monaco,monospace" font-size="12" font-weight="500" fill="#e2e8f0" filter="url(#glow2)">@gmickel</text>
+</svg>

package/assets/badges/website.svg

@@ -0,0 +1,22 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="120" height="28" viewBox="0 0 120 28">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#1a1a2e"/>
+      <stop offset="100%" style="stop-color:#0f0f1a"/>
+    </linearGradient>
+    <filter id="glow">
+      <feGaussianBlur stdDeviation="1" result="coloredBlur"/>
+      <feMerge>
+        <feMergeNode in="coloredBlur"/>
+        <feMergeNode in="SourceGraphic"/>
+      </feMerge>
+    </filter>
+  </defs>
+  <rect width="120" height="28" rx="6" fill="url(#bg)"/>
+  <rect x="1" y="1" width="118" height="26" rx="5" fill="none" stroke="#2dd4bf" stroke-opacity="0.3"/>
+  <text x="10" y="18" font-family="ui-monospace,SFMono-Regular,Menlo,Monaco,monospace" font-size="11" fill="#2dd4bf" filter="url(#glow)">❯</text>
+  <text x="24" y="18" font-family="ui-monospace,SFMono-Regular,Menlo,Monaco,monospace" font-size="12" font-weight="600" fill="#e2e8f0">gno.sh</text>
+  <rect x="74" y="9" width="2" height="12" fill="#2dd4bf" opacity="0.8">
+    <animate attributeName="opacity" values="0.8;0.2;0.8" dur="1.2s" repeatCount="indefinite"/>
+  </rect>
+</svg>

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@gmickel/gno",
-  "version": "0.3.0",
+  "version": "0.3.5",
   "description": "Local semantic search for your documents. Index Markdown, PDF, and Office files with hybrid BM25 + vector search.",
   "keywords": [
     "search",
@@ -47,6 +47,7 @@
     "test:watch": "bun test --watch",
     "test:coverage": "bun test --coverage",
     "test:coverage:html": "bun test --coverage --html",
+    "test:fixtures": "bun scripts/generate-test-fixtures.ts",
     "typecheck": "tsgo --noEmit",
     "lint:typeaware": "bun x oxlint --type-aware",
     "reset": "bun run src/index.ts reset --confirm",
@@ -79,9 +80,13 @@
     "@typescript/native-preview": "^7.0.0-dev.20251215.1",
     "ajv": "^8.17.1",
     "ajv-formats": "^3.0.1",
+    "docx": "^9.5.1",
     "evalite": "^1.0.0-beta.15",
+    "exceljs": "^4.4.0",
     "lefthook": "^2.0.12",
     "oxlint-tsgolint": "^0.10.0",
+    "pdf-lib": "^1.17.1",
+    "pptxgenjs": "^4.0.1",
     "ultracite": "^6.5.0"
   },
   "peerDependencies": {