@soulcraft/brainy 0.36.0 → 0.38.0

package/README.md

@@ -7,1689 +7,927 @@
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.4.5-blue.svg)](https://www.typescriptlang.org/)
 [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md)
 
-[//]: # ([![Cartographer](https://img.shields.io/badge/Cartographer-Official%20Standard-brightgreen)](https://github.com/sodal-project/cartographer))
-
 **A powerful graph & vector data platform for AI applications across any environment**
 
 </div>
 
-## ✨ Overview
-
-Brainy combines the power of vector search with graph relationships in a lightweight, cross-platform database. Whether
-you're building AI applications, recommendation systems, or knowledge graphs, Brainy provides the tools you need to
-store, connect, and retrieve your data intelligently.
-
-What makes Brainy special? It intelligently adapts to your environment! Brainy automatically detects your platform,
-adjusts its storage strategy, and optimizes performance based on your usage patterns. The more you use it, the smarter
-it gets - learning from your data to provide increasingly relevant results and connections.
-
-### 🚀 Key Features
-
-- **Run Everywhere** - Works in browsers, Node.js, serverless functions, and containers
-- **Vector Search** - Find semantically similar content using embeddings
-- **Advanced JSON Document Search** - Search within specific fields of JSON documents with field prioritization and
-  service-based field standardization
-- **Graph Relationships** - Connect data with meaningful relationships
-- **Streaming Pipeline** - Process data in real-time as it flows through the system
-- **Extensible Augmentations** - Customize and extend functionality with pluggable components
-- **Built-in Conduits** - Sync and scale across instances with WebSocket and WebRTC
-- **TensorFlow Integration** - Use TensorFlow.js for high-quality embeddings
-- **Adaptive Intelligence** - Automatically optimizes for your environment and usage patterns
-- **Persistent Storage** - Data persists across sessions and scales to any size
-- **TypeScript Support** - Fully typed API with generics
-- **CLI Tools & Web Service** - Command-line interface and REST API web service for data management
-- **Model Control Protocol (MCP)** - Allow external AI models to access Brainy data and use augmentation pipeline as
-  tools
-
-## 🚀 Live Demo
-
-**[Try the live demo](https://soulcraft-research.github.io/brainy/demo/index.html)** - Check out the interactive demo on
-GitHub Pages that showcases Brainy's main features.
-
-## 📊 What Can You Build?
-
-- **Semantic Search Engines** - Find content based on meaning, not just keywords
-- **Recommendation Systems** - Suggest similar items based on vector similarity
-- **Knowledge Graphs** - Build connected data structures with relationships
-- **AI Applications** - Store and retrieve embeddings for machine learning models
-- **AI-Enhanced Applications** - Build applications that leverage vector embeddings for intelligent data processing
-- **Data Organization Tools** - Automatically categorize and connect related information
-- **Adaptive Experiences** - Create applications that learn and evolve with your users
-- **Model-Integrated Systems** - Connect external AI models to Brainy data and tools using MCP
+## ✨ What is Brainy?
 
-## 🔧 Installation
+Imagine a database that thinks like you do - connecting ideas, finding patterns, and getting smarter over time. Brainy is the **AI-native database** that brings vector search and knowledge graphs together in one powerful, ridiculously easy-to-use package.
 
-```bash
-npm install @soulcraft/brainy
-```
-
-TensorFlow.js packages are included as bundled dependencies and will be automatically installed without any additional
-configuration.
+### 🆕 NEW: Distributed Mode (v0.38+)
+**Scale horizontally with zero configuration!** Brainy now supports distributed deployments with automatic coordination:
+- **🌐 Multi-Instance Coordination** - Multiple readers and writers working in harmony
+- **🏷️ Smart Domain Detection** - Automatically categorizes data (medical, legal, product, etc.)
+- **📊 Real-Time Health Monitoring** - Track performance across all instances
+- **🔄 Automatic Role Optimization** - Readers optimize for cache, writers for throughput
+- **🗂️ Intelligent Partitioning** - Hash-based partitioning for perfect load distribution
 
-### Additional Packages
+### 🚀 Why Developers Love Brainy
 
-Brainy offers specialized packages for different use cases:
+- **🧠 It Just Works™** - No config files, no tuning parameters, no DevOps headaches. Brainy auto-detects your environment and optimizes itself
+- **🌍 True Write-Once, Run-Anywhere** - Same code runs in React, Angular, Vue, Node.js, Deno, Bun, serverless, edge workers, and even vanilla HTML
+- **⚡ Scary Fast** - Handles millions of vectors with sub-millisecond search. Built-in GPU acceleration when available
+- **🎯 Self-Learning** - Like having a database that goes to the gym. Gets faster and smarter the more you use it
+- **🔮 AI-First Design** - Built for the age of embeddings, RAG, and semantic search. Your LLMs will thank you
+- **🎮 Actually Fun to Use** - Clean API, great DX, and it does the heavy lifting so you can build cool stuff
 
-#### CLI Package
+## 🚀 Quick Start (30 seconds!)
 
+### Node.js TLDR
 ```bash
-npm install -g @soulcraft/brainy-cli
-```
-
-Command-line interface for data management, bulk operations, and database administration.
-
-#### Web Service Package
+# Install
+npm install brainy
 
-```bash
-npm install @soulcraft/brainy-web-service
+# Use it
 ```
+```javascript
+import { createAutoBrainy, NounType, VerbType } from 'brainy'
 
-REST API web service wrapper that provides HTTP endpoints for search operations and database queries.
-
-## 🏁 Quick Start
-
-Brainy uses a unified build that automatically adapts to your environment (Node.js, browser, or serverless):
-
-```typescript
-import { BrainyData, NounType, VerbType } from '@soulcraft/brainy'
-
-// Create and initialize the database
-const db = new BrainyData()
-await db.init()
+const brainy = createAutoBrainy()
 
-// Add data (automatically converted to vectors)
-const catId = await db.add("Cats are independent pets", {
+// Add data with Nouns (entities)
+const catId = await brainy.add("Siamese cats are elegant and vocal", { 
   noun: NounType.Thing,
-  category: 'animal'
+  breed: "Siamese",
+  category: "animal" 
 })
 
-const dogId = await db.add("Dogs are loyal companions", {
-  noun: NounType.Thing,
-  category: 'animal'
+const ownerId = await brainy.add("John loves his pets", { 
+  noun: NounType.Person,
+  name: "John Smith" 
 })
 
-// Search for similar items
-const results = await db.searchText("feline pets", 2)
-console.log(results)
-
-// Add a relationship between items
-await db.addVerb(catId, dogId, {
-  verb: VerbType.RelatedTo,
-  description: 'Both are common household pets'
+// Connect with Verbs (relationships)
+await brainy.addVerb(ownerId, catId, { 
+  verb: VerbType.Owns,
+  since: "2020-01-01" 
 })
-```
-
-### Import Options
-
-```typescript
-// Standard import - automatically adapts to any environment
-import { BrainyData } from '@soulcraft/brainy'
-
-// Minified version for production
-import { BrainyData } from '@soulcraft/brainy/min'
-```
 
-> **Note**: The CLI functionality is available as a separate package `@soulcraft/brainy-cli` to reduce the bundle size
-> of the main package. Install it globally with `npm install -g @soulcraft/brainy-cli` to use the command-line
-> interface.
+// Search by meaning
+const results = await brainy.searchText("feline companions", 5)
 
-### Browser Usage
-
-```html
-
-<script type="module">
-  // Use local files instead of CDN
-  import { BrainyData } from './dist/unified.js'
-
-  // Or minified version
-  // import { BrainyData } from './dist/unified.min.js'
+// Search JSON documents by specific fields
+const docs = await brainy.searchDocuments("Siamese", {
+  fields: ['breed', 'category'],  // Search these fields
+  weights: { breed: 2.0 },         // Prioritize breed matches
+  limit: 10
+})
 
-  const db = new BrainyData()
-  await db.init()
-  // ...
-</script>
+// Find relationships
+const johnsPets = await brainy.getVerbsBySource(ownerId, VerbType.Owns)
 ```
 
-Modern bundlers like Webpack, Rollup, and Vite will automatically use the unified build which adapts to any environment.
+That's it! No config, no setup, it just works™
 
-## 🧩 How It Works
-
-Brainy combines four key technologies to create its adaptive intelligence:
+### 🌐 Distributed Mode Example (NEW!)
+```javascript
+// Writer Instance - Ingests data from multiple sources
+const writer = createAutoBrainy({
+  storage: { type: 's3', bucket: 'my-bucket' },
+  distributed: { role: 'writer' }  // Explicit role for safety
+})
 
-1. **Vector Embeddings** - Converts data (text, images, etc.) into numerical vectors that capture semantic meaning
-2. **HNSW Algorithm** - Enables fast similarity search through a hierarchical graph structure
-3. **Adaptive Environment Detection** - Automatically senses your platform and optimizes accordingly:
-    - Detects browser, Node.js, and serverless environments
-    - Adjusts performance parameters based on available resources
-    - Learns from query patterns to optimize future searches
-    - Tunes itself for your specific use cases
-4. **Intelligent Storage Selection** - Uses the best available storage option for your environment:
-    - Browser: Origin Private File System (OPFS)
-    - Node.js: File system
-    - Server: S3-compatible storage (optional)
-    - Serverless: In-memory storage with optional cloud persistence
-    - Fallback: In-memory storage
-    - Automatically migrates between storage types as needed
-    - Uses a simplified, consolidated storage structure for all noun types
+// Reader Instance - Optimized for search queries
+const reader = createAutoBrainy({
+  storage: { type: 's3', bucket: 'my-bucket' },
+  distributed: { role: 'reader' }  // 80% memory for cache
+})
 
-## 🚀 The Brainy Pipeline
+// Data automatically gets domain tags
+await writer.add("Patient shows symptoms of...", { 
+  diagnosis: "flu"  // Auto-tagged as 'medical' domain
+})
 
-Brainy's data processing pipeline transforms raw data into searchable, connected knowledge that gets smarter over time:
+// Domain-aware search across all partitions
+const results = await reader.search("medical symptoms", 10, {
+  filter: { domain: 'medical' }  // Only search medical data
+})
 
+// Monitor health across all instances
+const health = reader.getHealthStatus()
+console.log(`Instance ${health.instanceId}: ${health.status}`)
 ```
-Raw Data → Embedding → Vector Storage → Graph Connections → Adaptive Learning → Query & Retrieval
-```
-
-Each time data flows through this pipeline, Brainy learns more about your usage patterns and environment, making future
-operations faster and more relevant.
-
-### Pipeline Stages
-
-1. **Data Ingestion**
-    - Raw text or pre-computed vectors enter the pipeline
-    - Data is validated and prepared for processing
-
-2. **Embedding Generation**
-    - Text is transformed into numerical vectors using embedding models
-    - Uses TensorFlow Universal Sentence Encoder for high-quality text embeddings
-    - Custom embedding functions can be plugged in for specialized domains
-
-3. **Vector Indexing**
-    - Vectors are indexed using the HNSW algorithm
-    - Hierarchical structure enables fast similarity search
-    - Configurable parameters for precision vs. performance tradeoffs
-
-4. **Graph Construction**
-    - Nouns (entities) become nodes in the knowledge graph
-    - Verbs (relationships) connect related entities
-    - Typed relationships add semantic meaning to connections
-
-5. **Adaptive Learning**
-    - Analyzes usage patterns to optimize future operations
-    - Tunes performance parameters based on your environment
-    - Adjusts search strategies based on query history
-    - Becomes more efficient and relevant the more you use it
-
-6. **Intelligent Storage**
-    - Data is saved using the optimal storage for your environment
-    - Automatic selection between OPFS, filesystem, S3, or memory
-    - Migrates between storage types as your application's needs evolve
-    - Scales from tiny datasets to massive data collections
-    - Configurable storage adapters for custom persistence needs
-
-### Augmentation Types
-
-Brainy uses a powerful augmentation system to extend functionality. Augmentations are processed in the following order:
-
-1. **SENSE**
-    - Ingests and processes raw, unstructured data into nouns and verbs
-    - Handles text, images, audio streams, and other input formats
-    - Example: Converting raw text into structured entities
-
-2. **MEMORY**
-    - Provides storage capabilities for data in different formats
-    - Manages persistence across sessions
-    - Example: Storing vectors in OPFS or filesystem
-
-3. **COGNITION**
-    - Enables advanced reasoning, inference, and logical operations
-    - Analyzes relationships between entities
-    - Examples:
-        - Inferring new connections between existing data
-        - Deriving insights from graph relationships
-
-4. **CONDUIT**
-    - Establishes channels for structured data exchange
-    - Connects with external systems and syncs between Brainy instances
-    - Two built-in iConduit augmentations for scaling out and syncing:
-        - **WebSocket iConduit** - Syncs data between browsers and servers
-        - **WebRTC iConduit** - Direct peer-to-peer syncing between browsers
-    - Examples:
-        - Integrating with third-party APIs
-        - Syncing Brainy instances between browsers using WebSockets
-        - Peer-to-peer syncing between browsers using WebRTC
-
-5. **ACTIVATION**
-    - Initiates actions, responses, or data manipulations
-    - Triggers events based on data changes
-    - Example: Sending notifications when new data is processed
-
-6. **PERCEPTION**
-    - Interprets, contextualizes, and visualizes identified nouns and verbs
-    - Creates meaningful representations of data
-    - Example: Generating visualizations of graph relationships
-
-7. **DIALOG**
-    - Facilitates natural language understanding and generation
-    - Enables conversational interactions
-    - Example: Processing user queries and generating responses
-
-8. **WEBSOCKET**
-    - Enables real-time communication via WebSockets
-    - Can be combined with other augmentation types
-    - Example: Streaming data processing in real-time
-
-### Streaming Data Support
-
-Brainy's pipeline is designed to handle streaming data efficiently:
-
-1. **WebSocket Integration**
-    - Built-in support for WebSocket connections
-    - Process data as it arrives without blocking
-    - Example: `setupWebSocketPipeline(url, dataType, options)`
-
-2. **Asynchronous Processing**
-    - Non-blocking architecture for real-time data handling
-    - Parallel processing of incoming streams
-    - Example: `createWebSocketHandler(connection, dataType, options)`
-
-3. **Event-Based Architecture**
-    - Augmentations can listen to data feeds and streams
-    - Real-time updates propagate through the pipeline
-    - Example: `listenToFeed(feedUrl, callback)`
-
-4. **Threaded Execution**
-    - Comprehensive multi-threading for high-performance operations
-    - Parallel processing for batch operations, vector calculations, and embedding generation
-    - Configurable execution modes (SEQUENTIAL, PARALLEL, THREADED)
-    - Automatic thread management based on environment capabilities
-    - Example: `executeTypedPipeline(augmentations, method, args, { mode: ExecutionMode.THREADED })`
-
-### Running the Pipeline
-
-The pipeline runs automatically when you:
-
-```typescript
-// Add data (runs embedding → indexing → storage)
-const id = await db.add("Your text data here", { metadata })
 
-// Search (runs embedding → similarity search)
-const results = await db.searchText("Your query here", 5)
+## 🎭 Key Features
 
-// Connect entities (runs graph construction → storage)
-await db.addVerb(sourceId, targetId, { verb: VerbType.RelatedTo })
-```
+### Core Capabilities
+- **Vector Search** - Find semantically similar content using embeddings
+- **Graph Relationships** - Connect data with meaningful relationships
+- **JSON Document Search** - Search within specific fields with prioritization
+- **Distributed Mode** - Scale horizontally with automatic coordination between instances
+- **Real-Time Syncing** - WebSocket and WebRTC for distributed instances
+- **Streaming Pipeline** - Process data in real-time as it flows through
+- **Model Control Protocol** - Let AI models access your data
+
+### Smart Optimizations
+- **Auto-Configuration** - Detects environment and optimizes automatically
+- **Adaptive Learning** - Gets smarter with usage, optimizes itself over time
+- **Intelligent Partitioning** - Hash-based partitioning for perfect load distribution
+- **Role-Based Optimization** - Readers maximize cache, writers optimize throughput
+- **Domain-Aware Indexing** - Automatic categorization improves search relevance
+- **Multi-Level Caching** - Hot/warm/cold caching with predictive prefetching
+- **Memory Optimization** - 75% reduction with compression for large datasets
+
+### Developer Experience
+- **TypeScript Support** - Fully typed API with generics
+- **Extensible Augmentations** - Customize and extend functionality
+- **REST API** - Web service wrapper for HTTP endpoints
+- **Auto-Complete** - IntelliSense for all APIs and types
 
-Using the CLI:
+## 📦 Installation
 
+### Main Package
 ```bash
-# Add data through the CLI pipeline
-brainy add "Your text data here" '{"noun":"Thing"}'
-
-# Search through the CLI pipeline
-brainy search "Your query here" --limit 5
-
-# Connect entities through the CLI
-brainy addVerb <sourceId> <targetId> RelatedTo
-```
-
-### Extending the Pipeline
-
-Brainy's pipeline is designed for extensibility at every stage:
-
-1. **Custom Embedding**
-   ```typescript
-   // Create your own embedding function
-   const myEmbedder = async (text) => {
-     // Your custom embedding logic here
-     return [0.1, 0.2, 0.3, ...] // Return a vector
-   }
-
-   // Use it in Brainy
-   const db = new BrainyData({
-     embeddingFunction: myEmbedder
-   })
-   ```
-
-2. **Custom Distance Functions**
-   ```typescript
-   // Define your own distance function
-   const myDistance = (a, b) => {
-     // Your custom distance calculation
-     return Math.sqrt(a.reduce((sum, val, i) => sum + Math.pow(val - b[i], 2), 0))
-   }
-
-   // Use it in Brainy
-   const db = new BrainyData({
-     distanceFunction: myDistance
-   })
-   ```
-
-3. **Custom Storage Adapters**
-   ```typescript
-   // Implement the StorageAdapter interface
-   class MyStorage implements StorageAdapter {
-     // Your storage implementation
-   }
-
-   // Use it in Brainy
-   const db = new BrainyData({
-     storageAdapter: new MyStorage()
-   })
-   ```
-
-4. **Augmentations System**
-   ```typescript
-   // Create custom augmentations to extend functionality
-   const myAugmentation = {
-     type: 'memory',
-     name: 'my-custom-storage',
-     // Implementation details
-   }
-
-   // Register with Brainy
-   db.registerAugmentation(myAugmentation)
-   ```
-
-## Data Model
-
-Brainy uses a graph-based data model with two primary concepts:
-
-### Nouns (Entities)
-
-The main entities in your data (nodes in the graph):
-
-- Each noun has a unique ID, vector representation, and metadata
-- Nouns can be categorized by type (Person, Place, Thing, Event, Concept, etc.)
-- Nouns are automatically vectorized for similarity search
-
-### Verbs (Relationships)
-
-Connections between nouns (edges in the graph):
-
-- Each verb connects a source noun to a target noun
-- Verbs have types that define the relationship (RelatedTo, Controls, Contains, etc.)
-- Verbs can have their own metadata to describe the relationship
-
-### Type Utilities
-
-Brainy provides utility functions to access lists of noun and verb types:
-
-```typescript
-import {
-  NounType,
-  VerbType,
-  getNounTypes,
-  getVerbTypes,
-  getNounTypeMap,
-  getVerbTypeMap
-} from '@soulcraft/brainy'
-
-// At development time:
-// Access specific types directly from the NounType and VerbType objects
-console.log(NounType.Person)        // 'person'
-console.log(VerbType.Contains)      // 'contains'
-
-// At runtime:
-// Get a list of all noun types
-const nounTypes = getNounTypes()    // ['person', 'organization', 'location', ...]
-
-// Get a list of all verb types
-const verbTypes = getVerbTypes()    // ['relatedTo', 'contains', 'partOf', ...]
-
-// Get a map of noun type keys to values
-const nounTypeMap = getNounTypeMap() // { Person: 'person', Organization: 'organization', ... }
-
-// Get a map of verb type keys to values
-const verbTypeMap = getVerbTypeMap() // { RelatedTo: 'relatedTo', Contains: 'contains', ... }
+npm install brainy
 ```
 
-These utility functions make it easy to:
-
-- Get a complete list of available noun and verb types
-- Validate user input against valid types
-- Create dynamic UI components that display or select from available types
-- Map between type keys and their string values
-
-## Command Line Interface
-
-Brainy includes a powerful CLI for managing your data. The CLI is available as a separate package
-`@soulcraft/brainy-cli` to reduce the bundle size of the main package.
-
-### Installing and Using the CLI
-
+### Optional: Offline Models Package
 ```bash
-# Install the CLI globally
-npm install -g @soulcraft/brainy-cli
-
-# Initialize a database
-brainy init
-
-# Add some data
-brainy add "Cats are independent pets" '{"noun":"Thing","category":"animal"}'
-brainy add "Dogs are loyal companions" '{"noun":"Thing","category":"animal"}'
-
-# Search for similar items
-brainy search "feline pets" 5
-
-# Add relationships between items
-brainy addVerb <sourceId> <targetId> RelatedTo '{"description":"Both are pets"}'
-
-# Visualize the graph structure
-brainy visualize
-brainy visualize --root <id> --depth 3
+npm install @soulcraft/brainy-models
 ```
 
-### Using the CLI in Your Code
+The `@soulcraft/brainy-models` package provides **offline access** to the Universal Sentence Encoder model, eliminating network dependencies and ensuring consistent performance. Perfect for:
+- **Air-gapped environments** - No internet? No problem
+- **Consistent performance** - No network latency or throttling
+- **Privacy-focused apps** - Keep everything local
+- **High-reliability systems** - No external dependencies
 
-The CLI functionality is available as a separate package `@soulcraft/brainy-cli`. If you need CLI functionality in your
-application, install the CLI package:
+```javascript
+import { createAutoBrainy } from 'brainy'
+import { BundledUniversalSentenceEncoder } from '@soulcraft/brainy-models'
 
-```bash
-npm install @soulcraft/brainy-cli
+// Use the bundled model for offline operation
+const brainy = createAutoBrainy({
+  embeddingModel: BundledUniversalSentenceEncoder
+})
 ```
 
-Then you can use the CLI commands programmatically or through the command line interface.
+## 🎨 Build Amazing Things
 
-### Available Commands
+**🤖 AI Chat Applications** - Build ChatGPT-like apps with long-term memory and context awareness  
+**🔍 Semantic Search Engines** - Search by meaning, not keywords. Find "that thing that's like a cat but bigger" → returns "tiger"  
+**🎯 Recommendation Engines** - "Users who liked this also liked..." but actually good  
+**🧬 Knowledge Graphs** - Connect everything to everything. Wikipedia meets Neo4j meets magic  
+**👁️ Computer Vision Apps** - Store and search image embeddings. "Find all photos with dogs wearing hats"  
+**🎵 Music Discovery** - Find songs that "feel" similar. Spotify's Discover Weekly in your app  
+**📚 Smart Documentation** - Docs that answer questions. "How do I deploy to production?" → relevant guides  
+**🛡️ Fraud Detection** - Find patterns humans can't see. Anomaly detection on steroids  
+**🌐 Real-Time Collaboration** - Sync vector data across devices. Figma for AI data  
+**🏥 Medical Diagnosis Tools** - Match symptoms to conditions using embedding similarity
 
-#### Basic Database Operations:
+## 🧬 The Power of Nouns & Verbs
 
-- `init` - Initialize a new database
-- `add <text> [metadata]` - Add a new noun with text and optional metadata
-- `search <query> [limit]` - Search for nouns similar to the query
-- `get <id>` - Get a noun by ID
-- `delete <id>` - Delete a noun by ID
-- `addVerb <sourceId> <targetId> <verbType> [metadata]` - Add a relationship
-- `getVerbs <id>` - Get all relationships for a noun
-- `status` - Show database status
-- `clear` - Clear all data from the database
-- `generate-random-graph` - Generate test data
-- `visualize` - Visualize the graph structure
-- `completion-setup` - Setup shell autocomplete
+Brainy uses a **graph-based data model** that mirrors how humans think - with **Nouns** (entities) connected by **Verbs** (relationships). This isn't just vectors in a void; it's structured, meaningful data.
 
-#### Pipeline and Augmentation Commands:
+### 📝 Nouns (What Things Are)
 
-- `list-augmentations` - List all available augmentation types and registered augmentations
-- `augmentation-info <type>` - Get detailed information about a specific augmentation type
-- `test-pipeline [text]` - Test the sequential pipeline with sample data
-    - `-t, --data-type <type>` - Type of data to process (default: 'text')
-    - `-m, --mode <mode>` - Execution mode: sequential, parallel, threaded (default: 'sequential')
-    - `-s, --stop-on-error` - Stop execution if an error occurs
-    - `-v, --verbose` - Show detailed output
-- `stream-test` - Test streaming data through the pipeline (simulated)
-    - `-c, --count <number>` - Number of data items to stream (default: 5)
-    - `-i, --interval <ms>` - Interval between data items in milliseconds (default: 1000)
-    - `-t, --data-type <type>` - Type of data to process (default: 'text')
-    - `-v, --verbose` - Show detailed output
+Nouns are your entities - the "things" in your data. Each noun has:
+- A unique ID
+- A vector representation (for similarity search)
+- A type (Person, Document, Concept, etc.)
+- Custom metadata
 
-## API Reference
+**Available Noun Types:**
 
-### Database Management
+| Category | Types | Use For |
+|----------|-------|---------|
+| **Core Entities** | `Person`, `Organization`, `Location`, `Thing`, `Concept`, `Event` | People, companies, places, objects, ideas, happenings |
+| **Digital Content** | `Document`, `Media`, `File`, `Message`, `Content` | PDFs, images, videos, emails, posts, generic content |
+| **Collections** | `Collection`, `Dataset` | Groups of items, structured data sets |
+| **Business** | `Product`, `Service`, `User`, `Task`, `Project` | E-commerce, SaaS, project management |
+| **Descriptive** | `Process`, `State`, `Role` | Workflows, conditions, responsibilities |
 
-```typescript
-// Initialize the database
-await db.init()
+### 🔗 Verbs (How Things Connect)
 
-// Clear all data
-await db.clear()
+Verbs are your relationships - they give meaning to connections. Not just "these vectors are similar" but "this OWNS that" or "this CAUSES that".
 
-// Get database status
-const status = await db.status()
+**Available Verb Types:**
 
-// Backup all data from the database
-const backupData = await db.backup()
+| Category | Types | Examples |
+|----------|-------|----------|
+| **Core** | `RelatedTo`, `Contains`, `PartOf`, `LocatedAt`, `References` | Generic relations, containment, location |
+| **Temporal** | `Precedes`, `Succeeds`, `Causes`, `DependsOn`, `Requires` | Time sequences, causality, dependencies |
+| **Creation** | `Creates`, `Transforms`, `Becomes`, `Modifies`, `Consumes` | Creation, change, consumption |
+| **Ownership** | `Owns`, `AttributedTo`, `CreatedBy`, `BelongsTo` | Ownership, authorship, belonging |
+| **Social** | `MemberOf`, `WorksWith`, `FriendOf`, `Follows`, `Likes`, `ReportsTo` | Social networks, organizations |
+| **Functional** | `Describes`, `Implements`, `Validates`, `Triggers`, `Serves` | Functions, implementations, services |
 
-// Restore data into the database
-const restoreResult = await db.restore(backupData, { clearExisting: true })
-```
+### 💡 Why This Matters
 
-### Database Statistics
+```javascript
+// Traditional vector DB: Just similarity
+const similar = await vectorDB.search(embedding, 10)
+// Result: [vector1, vector2, ...] - What do these mean? 🤷
 
-Brainy provides a way to get statistics about the current state of the database. For detailed information about the
-statistics system, including implementation details, scalability improvements, and usage examples, see
-our [Statistics Guide](STATISTICS.md).
+// Brainy: Similarity + Meaning + Relationships
+const catId = await brainy.add("Siamese cat", { 
+  noun: NounType.Thing,
+  breed: "Siamese" 
+})
+const ownerId = await brainy.add("John Smith", { 
+  noun: NounType.Person 
+})
+await brainy.addVerb(ownerId, catId, { 
+  verb: VerbType.Owns,
+  since: "2020-01-01" 
+})
 
-```typescript
-import { BrainyData, getStatistics } from '@soulcraft/brainy'
+// Now you can search with context!
+const johnsPets = await brainy.getVerbsBySource(ownerId, VerbType.Owns)
+const catOwners = await brainy.getVerbsByTarget(catId, VerbType.Owns)
+```
 
-// Create and initialize the database
-const db = new BrainyData()
-await db.init()
+## 🌍 Distributed Mode (New!)
 
-// Get statistics using the instance method
-const stats = await db.getStatistics()
-console.log(stats)
-// Output: { nounCount: 0, verbCount: 0, metadataCount: 0, hnswIndexSize: 0, serviceBreakdown: {...} }
-```
+Brainy now supports **distributed deployments** with multiple specialized instances sharing the same data. Perfect for scaling your AI applications across multiple servers.
 
-### Working with Nouns (Entities)
+### Distributed Setup
 
-```typescript
-// Add a noun (automatically vectorized)
-const id = await db.add(textOrVector, {
-  noun: NounType.Thing,
-  // other metadata...
+```javascript
+// Single instance (no change needed!)
+const brainy = createAutoBrainy({
+  storage: { type: 's3', bucket: 'my-bucket' }
 })
 
-// Add multiple nouns in parallel (with multithreading and batch embedding)
-const ids = await db.addBatch([
-  {
-    vectorOrData: "First item to add",
-    metadata: { noun: NounType.Thing, category: 'example' }
-  },
-  {
-    vectorOrData: "Second item to add",
-    metadata: { noun: NounType.Thing, category: 'example' }
-  },
-  // More items...
-], {
-  forceEmbed: false,
-  concurrency: 4, // Control the level of parallelism (default: 4)
-  batchSize: 50   // Control the number of items to process in a single batch (default: 50)
+// Distributed mode requires explicit role configuration
+// Option 1: Via environment variable
+process.env.BRAINY_ROLE = 'writer'  // or 'reader' or 'hybrid'
+const brainy = createAutoBrainy({
+  storage: { type: 's3', bucket: 'my-bucket' },
+  distributed: true
 })
 
-// Retrieve a noun
-const noun = await db.get(id)
-
-// Update noun metadata
-await db.updateMetadata(id, {
-  noun: NounType.Thing,
-  // updated metadata...
+// Option 2: Via configuration
+const writer = createAutoBrainy({
+  storage: { type: 's3', bucket: 'my-bucket' },
+  distributed: { role: 'writer' }  // Handles data ingestion
 })
 
-// Delete a noun
-await db.delete(id)
-
-// Search for similar nouns
-const results = await db.search(vectorOrText, numResults)
-const textResults = await db.searchText("query text", numResults)
-
-// Search by noun type
-const thingNouns = await db.searchByNounTypes([NounType.Thing], numResults)
+const reader = createAutoBrainy({
+  storage: { type: 's3', bucket: 'my-bucket' },
+  distributed: { role: 'reader' }  // Optimized for queries
+})
 
-// Search within specific fields of JSON documents
-const fieldResults = await db.search("Acme Corporation", 10, {
-  searchField: "company"
+// Option 3: Via read/write mode (role auto-inferred)
+const writer = createAutoBrainy({
+  storage: { type: 's3', bucket: 'my-bucket' },
+  writeOnly: true,  // Automatically becomes 'writer' role
+  distributed: true
 })
 
-// Search using standard field names across different services
-const titleResults = await db.searchByStandardField("title", "climate change", 10)
-const authorResults = await db.searchByStandardField("author", "johndoe", 10, {
-  services: ["github", "reddit"]
+const reader = createAutoBrainy({
+  storage: { type: 's3', bucket: 'my-bucket' },
+  readOnly: true,   // Automatically becomes 'reader' role
+  distributed: true
 })
 ```
 
-### Field Standardization and Service Tracking
+### Key Distributed Features
 
-Brainy automatically tracks field names from JSON documents and associates them with the service that inserted the data.
-This enables powerful cross-service search capabilities:
+**🎯 Explicit Role Configuration**
+- Roles must be explicitly set (no dangerous auto-assignment)
+- Can use environment variables, config, or read/write modes
+- Clear separation between writers and readers
 
-```typescript
-// Get all available field names organized by service
-const fieldNames = await db.getAvailableFieldNames()
-// Example output: { "github": ["repository.name", "issue.title"], "reddit": ["title", "selftext"] }
+**#️⃣ Hash-Based Partitioning**
+- Handles multiple writers with different data types
+- Even distribution across partitions
+- No semantic conflicts with mixed data
+
+**🏷️ Domain Tagging**
+- Automatic domain detection (medical, legal, product, etc.)
+- Filter searches by domain
+- Logical separation without complexity
 
-// Get standard field mappings
-const standardMappings = await db.getStandardFieldMappings()
-// Example output: { "title": { "github": ["repository.name"], "reddit": ["title"] } }
+```javascript
+// Data is automatically tagged with domains
+await brainy.add({ 
+  symptoms: "fever", 
+  diagnosis: "flu" 
+}, metadata)  // Auto-tagged as 'medical'
+
+// Search within specific domains
+const medicalResults = await brainy.search(query, 10, {
+  filter: { domain: 'medical' }
+})
 ```
 
-When adding data, specify the service name to ensure proper field tracking:
+**📊 Health Monitoring**
+- Real-time health metrics
+- Automatic dead instance cleanup
+- Performance tracking
 
-```typescript
-// Add data with service name
-await db.add(jsonData, metadata, { service: "github" })
+```javascript
+// Get health status
+const health = brainy.getHealthStatus()
+// {
+//   status: 'healthy',
+//   role: 'reader',
+//   vectorCount: 1000000,
+//   cacheHitRate: 0.95,
+//   requestsPerSecond: 150
+// }
+```
+
+**⚡ Role-Optimized Performance**
+- **Readers**: 80% memory for cache, aggressive prefetching
+- **Writers**: Optimized write batching, minimal cache
+- **Hybrid**: Adaptive based on workload
+
+### Deployment Examples
+
+**Docker Compose**
+```yaml
+services:
+  writer:
+    image: myapp
+    environment:
+      BRAINY_ROLE: writer  # Optional - auto-detects
+      
+  reader:
+    image: myapp
+    environment:
+      BRAINY_ROLE: reader  # Optional - auto-detects
+    scale: 5
+```
+
+**Kubernetes**
+```yaml
+# Automatically detects role from deployment type
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: brainy-readers
+spec:
+  replicas: 10  # Multiple readers
+  template:
+    spec:
+      containers:
+      - name: app
+        image: myapp
+        # Role auto-detected as 'reader' (multiple replicas)
+```
+
+**Benefits**
+- ✅ **50-70% faster searches** with parallel readers
+- ✅ **No coordination complexity** - Shared JSON config in S3
+- ✅ **Zero downtime scaling** - Add/remove instances anytime
+- ✅ **Automatic failover** - Dead instances cleaned up automatically
+
+## 🤔 Why Choose Brainy?
+
+### vs. Traditional Databases
+❌ **PostgreSQL with pgvector** - Requires complex setup, tuning, and DevOps expertise  
+✅ **Brainy** - Zero config, auto-optimizes, works everywhere from browser to cloud
+
+### vs. Vector Databases
+❌ **Pinecone/Weaviate/Qdrant** - Cloud-only, expensive, vendor lock-in  
+✅ **Brainy** - Run locally, in browser, or cloud. Your choice, your data
+
+### vs. Graph Databases
+❌ **Neo4j** - Great for graphs, no vector support  
+✅ **Brainy** - Vectors + graphs in one. Best of both worlds
+
+### vs. DIY Solutions
+❌ **Building your own** - Months of work, optimization nightmares  
+✅ **Brainy** - Production-ready in 30 seconds
+
+## 🚀 Getting Started in 30 Seconds
+
+### React
+
+```jsx
+import { createAutoBrainy } from 'brainy'
+import { useEffect, useState } from 'react'
+
+function SemanticSearch() {
+  const [brainy] = useState(() => createAutoBrainy())
+  const [results, setResults] = useState([])
+
+  const search = async (query) => {
+    const items = await brainy.searchText(query, 10)
+    setResults(items)
+  }
+
+  return (
+    <input onChange={(e) => search(e.target.value)} 
+           placeholder="Search by meaning..." />
+  )
+}
 ```
 
-### Working with Verbs (Relationships)
+### Angular
 
 ```typescript
-// Add a relationship between nouns
-await db.addVerb(sourceId, targetId, {
-  verb: VerbType.RelatedTo,
-  // other metadata...
+import { Component, OnInit } from '@angular/core'
+import { createAutoBrainy } from 'brainy'
+
+@Component({
+  selector: 'app-search',
+  template: `
+    <input (input)="search($event.target.value)" 
+           placeholder="Semantic search...">
+    <div *ngFor="let result of results">
+      {{ result.text }}
+    </div>
+  `
 })
+export class SearchComponent implements OnInit {
+  brainy = createAutoBrainy()
+  results = []
 
-// Add a relationship with auto-creation of missing nouns
-// This is useful when the target noun might not exist yet
-await db.addVerb(sourceId, targetId, {
-  verb: VerbType.RelatedTo,
-  // Enable auto-creation of missing nouns
-  autoCreateMissingNouns: true,
-  // Optional metadata for auto-created nouns
-  missingNounMetadata: {
-    noun: NounType.Concept,
-    description: 'Auto-created noun'
+  async search(query: string) {
+    this.results = await this.brainy.searchText(query, 10)
   }
-})
+}
+```
 
-// Get all relationships
-const verbs = await db.getAllVerbs()
+### Vue 3
 
-// Get relationships by source noun
-const outgoingVerbs = await db.getVerbsBySource(sourceId)
+```vue
+<script setup>
+import { createAutoBrainy } from 'brainy'
+import { ref } from 'vue'
 
-// Get relationships by target noun
-const incomingVerbs = await db.getVerbsByTarget(targetId)
+const brainy = createAutoBrainy()
+const results = ref([])
 
-// Get relationships by type
-const containsVerbs = await db.getVerbsByType(VerbType.Contains)
+const search = async (query) => {
+  results.value = await brainy.searchText(query, 10)
+}
+</script>
 
-// Get a specific relationship
-const verb = await db.getVerb(verbId)
+<template>
+  <input @input="search($event.target.value)" 
+         placeholder="Find similar content...">
+  <div v-for="result in results" :key="result.id">
+    {{ result.text }}
+  </div>
+</template>
+```
+
+### Svelte
+
+```svelte
+<script>
+  import { createAutoBrainy } from 'brainy'
+  
+  const brainy = createAutoBrainy()
+  let results = []
+  
+  async function search(e) {
+    results = await brainy.searchText(e.target.value, 10)
+  }
+</script>
 
-// Delete a relationship
-await db.deleteVerb(verbId)
+<input on:input={search} placeholder="AI-powered search...">
+{#each results as result}
+  <div>{result.text}</div>
+{/each}
 ```
 
-## Advanced Configuration
-
-### Database Modes
+### Next.js (App Router)
 
-Brainy supports special operational modes that restrict certain operations:
+```jsx
+// app/search/page.js
+import { createAutoBrainy } from 'brainy'
 
-```typescript
-import { BrainyData } from '@soulcraft/brainy'
+export default function SearchPage() {
+  async function search(formData) {
+    'use server'
+    const brainy = createAutoBrainy({ bucketName: 'vectors' })
+    const query = formData.get('query')
+    return await brainy.searchText(query, 10)
+  }
 
-// Create and initialize the database
-const db = new BrainyData()
-await db.init()
+  return (
+    <form action={search}>
+      <input name="query" placeholder="Search..." />
+      <button type="submit">Search</button>
+    </form>
+  )
+}
+```
 
-// Set the database to read-only mode (prevents write operations)
-db.setReadOnly(true)
+### Node.js / Bun / Deno
 
-// Check if the database is in read-only mode
-const isReadOnly = db.isReadOnly() // Returns true
+```javascript
+import { createAutoBrainy } from 'brainy'
 
-// Set the database to write-only mode (prevents search operations)
-db.setWriteOnly(true)
+const brainy = createAutoBrainy()
 
-// Check if the database is in write-only mode
-const isWriteOnly = db.isWriteOnly() // Returns true
+// Add some data
+await brainy.add("TypeScript is a typed superset of JavaScript", {
+  category: 'programming'
+})
 
-// Reset to normal mode (allows both read and write operations)
-db.setReadOnly(false)
-db.setWriteOnly(false)
+// Search for similar content
+const results = await brainy.searchText("JavaScript with types", 5)
+console.log(results)
 ```
 
-- **Read-Only Mode**: When enabled, prevents all write operations (add, update, delete). Useful for deployment scenarios
-  where you want to prevent modifications to the database.
-- **Write-Only Mode**: When enabled, prevents all search operations. Useful for initial data loading or when you want to
-  optimize for write performance.
+### Vanilla JavaScript
 
-### Embedding
+```html
+<!DOCTYPE html>
+<html>
+<head>
+  <script type="module">
+    import { createAutoBrainy } from 'https://unpkg.com/brainy/dist/unified.min.js'
+    
+    window.brainy = createAutoBrainy()
+    
+    window.search = async function(query) {
+      const results = await brainy.searchText(query, 10)
+      document.getElementById('results').innerHTML = 
+        results.map(r => `<div>${r.text}</div>`).join('')
+    }
+  </script>
+</head>
+<body>
+  <input onkeyup="search(this.value)" placeholder="Search...">
+  <div id="results"></div>
+</body>
+</html>
+```
 
-```typescript
-import {
-  BrainyData,
-  createTensorFlowEmbeddingFunction,
-  createThreadedEmbeddingFunction
-} from '@soulcraft/brainy'
-
-// Use the standard TensorFlow Universal Sentence Encoder embedding function
-const db = new BrainyData({
-  embeddingFunction: createTensorFlowEmbeddingFunction()
-})
-await db.init()
+### Cloudflare Workers
 
-// Or use the threaded embedding function for better performance
-const threadedDb = new BrainyData({
-  embeddingFunction: createThreadedEmbeddingFunction()
-})
-await threadedDb.init()
-
-// Directly embed text to vectors
-const vector = await db.embed("Some text to convert to a vector")
-
-// Calculate similarity between two texts or vectors
-const similarity = await db.calculateSimilarity(
-  "Cats are furry pets",
-  "Felines make good companions"
-)
-console.log(`Similarity score: ${similarity}`) // Higher value means more similar
-
-// Calculate similarity with custom options
-const vectorA = await db.embed("First text")
-const vectorB = await db.embed("Second text")
-const customSimilarity = await db.calculateSimilarity(
-  vectorA, // Can use pre-computed vectors
-  vectorB,
-  {
-    forceEmbed: false, // Skip embedding if inputs are already vectors
-    distanceFunction: cosineDistance // Optional custom distance function
+```javascript
+import { createAutoBrainy } from 'brainy'
+
+export default {
+  async fetch(request, env) {
+    const brainy = createAutoBrainy({ 
+      bucketName: env.R2_BUCKET 
+    })
+    
+    const url = new URL(request.url)
+    const query = url.searchParams.get('q')
+    
+    const results = await brainy.searchText(query, 10)
+    return Response.json(results)
   }
-)
+}
 ```
 
-The threaded embedding function runs in a separate thread (Web Worker in browsers, Worker Thread in Node.js) to improve
-performance, especially for embedding operations. It uses GPU acceleration when available (via WebGL in browsers) and
-falls back to CPU processing for compatibility. Universal Sentence Encoder is always used for embeddings. The
-implementation includes worker reuse and model caching for optimal performance.
-
-### Performance Tuning
-
-Brainy includes comprehensive performance optimizations that work across all environments (browser, CLI, Node.js,
-container, server):
+### AWS Lambda
 
-#### GPU and CPU Optimization
-
-Brainy uses GPU and CPU optimization for compute-intensive operations:
-
-1. **GPU-Accelerated Embeddings**: Generate text embeddings using TensorFlow.js with WebGL backend when available
-2. **Automatic Fallback**: Falls back to CPU backend when GPU is not available
-3. **Optimized Distance Calculations**: Perform vector similarity calculations with optimized algorithms
-4. **Cross-Environment Support**: Works consistently across browsers and Node.js environments
-5. **Memory Management**: Properly disposes of tensors to prevent memory leaks
-
-#### Multithreading Support
-
-Brainy includes comprehensive multithreading support to improve performance across all environments:
-
-1. **Parallel Batch Processing**: Add multiple items concurrently with controlled parallelism
-2. **Multithreaded Vector Search**: Perform distance calculations in parallel for faster search operations
-3. **Threaded Embedding Generation**: Generate embeddings in separate threads to avoid blocking the main thread
-4. **Worker Reuse**: Maintains a pool of workers to avoid the overhead of creating and terminating workers
-5. **Model Caching**: Initializes the embedding model once per worker and reuses it for multiple operations
-6. **Batch Embedding**: Processes multiple items in a single embedding operation for better performance
-7. **Automatic Environment Detection**: Adapts to browser (Web Workers) and Node.js (Worker Threads) environments
+```javascript
+import { createAutoBrainy } from 'brainy'
 
-```typescript
-import { BrainyData, euclideanDistance } from '@soulcraft/brainy'
-
-// Configure with custom options
-const db = new BrainyData({
-  // Use Euclidean distance instead of default cosine distance
-  distanceFunction: euclideanDistance,
-
-  // HNSW index configuration for search performance
-  hnsw: {
-    M: 16,              // Max connections per noun
-    efConstruction: 200, // Construction candidate list size
-    efSearch: 50,       // Search candidate list size
-  },
-
-  // Performance optimization options
-  performance: {
-    useParallelization: true, // Enable multithreaded search operations
-  },
-
-  // Noun and Verb type validation
-  typeValidation: {
-    enforceNounTypes: true,  // Validate noun types against NounType enum
-    enforceVerbTypes: true,  // Validate verb types against VerbType enum
-  },
-
-  // Storage configuration
-  storage: {
-    requestPersistentStorage: true,
-    // Example configuration for cloud storage (replace with your own values):
-    // s3Storage: {
-    //   bucketName: 'your-s3-bucket-name',
-    //   region: 'your-aws-region'
-    //   // Credentials should be provided via environment variables
-    //   // AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
-    // }
+export const handler = async (event) => {
+  const brainy = createAutoBrainy({ 
+    bucketName: process.env.S3_BUCKET 
+  })
+  
+  const results = await brainy.searchText(event.query, 10)
+  
+  return {
+    statusCode: 200,
+    body: JSON.stringify(results)
   }
-})
+}
 ```
 
-### Optimized HNSW for Large Datasets
-
-Brainy includes an optimized HNSW index implementation for large datasets that may not fit entirely in memory, using a
-hybrid approach:
+### Azure Functions
 
-1. **Product Quantization** - Reduces vector dimensionality while preserving similarity relationships
-2. **Disk-Based Storage** - Offloads vectors to disk when memory usage exceeds a threshold
-3. **Memory-Efficient Indexing** - Optimizes memory usage for large-scale vector collections
+```javascript
+import { createAutoBrainy } from 'brainy'
 
-```typescript
-import { BrainyData } from '@soulcraft/brainy'
-
-// Configure with optimized HNSW index for large datasets
-const db = new BrainyData({
-  hnswOptimized: {
-    // Standard HNSW parameters
-    M: 16,              // Max connections per noun
-    efConstruction: 200, // Construction candidate list size
-    efSearch: 50,       // Search candidate list size
-
-    // Memory threshold in bytes - when exceeded, will use disk-based approach
-    memoryThreshold: 1024 * 1024 * 1024, // 1GB default threshold
-
-    // Product quantization settings for dimensionality reduction
-    productQuantization: {
-      enabled: true,              // Enable product quantization
-      numSubvectors: 16,          // Number of subvectors to split the vector into
-      numCentroids: 256           // Number of centroids per subvector
-    },
-
-    // Whether to use disk-based storage for the index
-    useDiskBasedIndex: true         // Enable disk-based storage
-  },
-
-  // Storage configuration (required for disk-based index)
-  storage: {
-    requestPersistentStorage: true
+module.exports = async function (context, req) {
+  const brainy = createAutoBrainy({
+    bucketName: process.env.AZURE_STORAGE_CONTAINER
+  })
+  
+  const results = await brainy.searchText(req.query.q, 10)
+  
+  context.res = {
+    body: results
   }
-})
-
-// The optimized index automatically adapts based on dataset size:
-// 1. For small datasets: Uses standard in-memory approach
-// 2. For medium datasets: Applies product quantization to reduce memory usage
-// 3. For large datasets: Combines product quantization with disk-based storage
-
-// Check status to see memory usage and optimization details
-const status = await db.status()
-console.log(status.details.index)
+}
 ```
 
-## Distance Functions
+### Google Cloud Functions
 
-Brainy provides several distance functions for vector similarity calculations:
-
-- `cosineDistance` (default): Measures the cosine of the angle between vectors (1 - cosine similarity)
-- `euclideanDistance`: Measures the straight-line distance between vectors
-- `manhattanDistance`: Measures the sum of absolute differences between vector components
-- `dotProductDistance`: Measures the negative dot product between vectors
-
-All distance functions are optimized for performance and automatically use the most efficient implementation based on
-the dataset size and available resources. For large datasets and high-dimensional vectors, Brainy uses batch processing
-and multithreading when available to improve performance.
-
-## Backup and Restore
-
-Brainy provides backup and restore capabilities that allow you to:
-
-- Back up your data
-- Transfer data between Brainy instances
-- Restore existing data into Brainy for vectorization and indexing
-- Backup data for analysis or visualization in other tools
-
-### Backing Up Data
-
-```typescript
-// Backup all data from the database
-const backupData = await db.backup()
-
-// The backup data includes:
-// - All nouns (entities) with their vectors and metadata
-// - All verbs (relationships) between nouns
-// - Noun types and verb types
-// - HNSW index data for fast similarity search
-// - Version information
-
-// Save the backup data to a file (Node.js environment)
-import fs from 'fs'
+```javascript
+import { createAutoBrainy } from 'brainy'
 
-fs.writeFileSync('brainy-backup.json', JSON.stringify(backupData, null, 2))
+export const searchHandler = async (req, res) => {
+  const brainy = createAutoBrainy({
+    bucketName: process.env.GCS_BUCKET
+  })
+  
+  const results = await brainy.searchText(req.query.q, 10)
+  res.json(results)
+}
 ```
 
-### Restoring Data
+### Google Cloud Run
 
-Brainy's restore functionality can handle:
+```dockerfile
+# Dockerfile
+FROM node:20-alpine
+USER node
+WORKDIR /app
+COPY package*.json ./
+RUN npm install brainy
+COPY . .
+CMD ["node", "server.js"]
+```
 
-1. Complete backups with vectors and index data
-2. Sparse data without vectors (vectors will be created during restore)
-3. Data without HNSW index (index will be reconstructed if needed)
+```javascript
+// server.js
+import { createAutoBrainy } from 'brainy'
+import express from 'express'
 
-```typescript
-// Restore data with all options
-const restoreResult = await db.restore(backupData, {
-  clearExisting: true // Whether to clear existing data before restore
+const app = express()
+const brainy = createAutoBrainy({
+  bucketName: process.env.GCS_BUCKET
 })
 
-// Import sparse data (without vectors)
-// Vectors will be automatically created using the embedding function
-const sparseData = {
-  nouns: [
-    {
-      id: '123',
-      // No vector field - will be created during import
-      metadata: {
-        noun: 'Thing',
-        text: 'This text will be used to generate a vector'
-      }
-    }
-  ],
-  verbs: [],
-  version: '1.0.0'
-}
+app.get('/search', async (req, res) => {
+  const results = await brainy.searchText(req.query.q, 10)
+  res.json(results)
+})
 
-const sparseImportResult = await db.importSparseData(sparseData)
+const port = process.env.PORT || 8080
+app.listen(port, () => console.log(`Brainy on Cloud Run: ${port}`))
 ```
 
-### CLI Backup/Restore
-
 ```bash
-# Backup data to a file
-brainy backup --output brainy-backup.json
-
-# Restore data from a file
-brainy restore --input brainy-backup.json --clear-existing
-
-# Import sparse data (without vectors)
-brainy import-sparse --input sparse-data.json
+# Deploy to Cloud Run
+gcloud run deploy brainy-api \
+  --source . \
+  --platform managed \
+  --region us-central1 \
+  --allow-unauthenticated
 ```
 
-## Embedding
-
-Brainy uses the following embedding approach:
+### Vercel Edge Functions
 
-- TensorFlow Universal Sentence Encoder (high-quality text embeddings)
-- GPU acceleration when available (via WebGL in browsers)
-- Batch embedding for processing multiple items efficiently
-- Worker reuse and model caching for optimal performance
-- Custom embedding functions can be plugged in for specialized domains
-
-## Extensions
-
-Brainy includes an augmentation system for extending functionality:
-
-- **Memory Augmentations**: Different storage backends
-- **Sense Augmentations**: Process raw data
-- **Cognition Augmentations**: Reasoning and inference
-- **Dialog Augmentations**: Text processing and interaction
-- **Perception Augmentations**: Data interpretation and visualization
-- **Activation Augmentations**: Trigger actions
-
-### Simplified Augmentation System
-
-Brainy provides a simplified factory system for creating, importing, and executing augmentations with minimal
-boilerplate:
-
-```typescript
-import {
-  createMemoryAugmentation,
-  createConduitAugmentation,
-  createSenseAugmentation,
-  addWebSocketSupport,
-  executeStreamlined,
-  processStaticData,
-  processStreamingData,
-  createPipeline
-} from '@soulcraft/brainy'
-
-// Create a memory augmentation with minimal code
-const memoryAug = createMemoryAugmentation({
-  name: 'simple-memory',
-  description: 'A simple in-memory storage augmentation',
-  autoRegister: true,
-  autoInitialize: true,
-
-  // Implement only the methods you need
-  storeData: async (key, data) => {
-    // Your implementation here
-    return {
-      success: true,
-      data: true
-    }
-  },
-
-  retrieveData: async (key) => {
-    // Your implementation here
-    return {
-      success: true,
-      data: { example: 'data', key }
-    }
-  }
-})
+```javascript
+import { createAutoBrainy } from 'brainy'
 
-// Add WebSocket support to any augmentation
-const wsAugmentation = addWebSocketSupport(memoryAug, {
-  connectWebSocket: async (url) => {
-    // Your implementation here
-    return {
-      connectionId: 'ws-1',
-      url,
-      status: 'connected'
-    }
-  }
-})
+export const config = {
+  runtime: 'edge'
+}
 
-// Process static data through a pipeline
-const result = await processStaticData(
-  'Input data',
-  [
-    {
-      augmentation: senseAug,
-      method: 'processRawData',
-      transformArgs: (data) => [data, 'text']
-    },
-    {
-      augmentation: memoryAug,
-      method: 'storeData',
-      transformArgs: (data) => ['processed-data', data]
-    }
-  ]
-)
-
-// Create a reusable pipeline
-const pipeline = createPipeline([
-  {
-    augmentation: senseAug,
-    method: 'processRawData',
-    transformArgs: (data) => [data, 'text']
-  },
-  {
-    augmentation: memoryAug,
-    method: 'storeData',
-    transformArgs: (data) => ['processed-data', data]
-  }
-])
+export default async function handler(request) {
+  const brainy = createAutoBrainy()
+  const { searchParams } = new URL(request.url)
+  const query = searchParams.get('q')
+  
+  const results = await brainy.searchText(query, 10)
+  return Response.json(results)
+}
+```
 
-// Use the pipeline
-const result = await pipeline('New input data')
+### Netlify Functions
 
-// Dynamically load augmentations at runtime
-const loadedAugmentations = await loadAugmentationModule(
-  import('./my-augmentations.js'),
-  {
-    autoRegister: true,
-    autoInitialize: true
+```javascript
+import { createAutoBrainy } from 'brainy'
+
+export async function handler(event, context) {
+  const brainy = createAutoBrainy()
+  const query = event.queryStringParameters.q
+  
+  const results = await brainy.searchText(query, 10)
+  
+  return {
+    statusCode: 200,
+    body: JSON.stringify(results)
   }
-)
+}
 ```
 
-The simplified augmentation system provides:
-
-1. **Factory Functions** - Create augmentations with minimal boilerplate
-2. **WebSocket Support** - Add WebSocket capabilities to any augmentation
-3. **Streamlined Pipeline** - Process data through augmentations more efficiently
-4. **Dynamic Loading** - Load augmentations at runtime when needed
-5. **Static & Streaming Data** - Handle both static and streaming data with the same API
-
-#### WebSocket Augmentation Types
-
-Brainy exports several WebSocket augmentation types that can be used by augmentation creators to add WebSocket
-capabilities to their augmentations:
+### Supabase Edge Functions
 
 ```typescript
-import {
-  // Base WebSocket support interface
-  IWebSocketSupport,
-
-  // Combined WebSocket augmentation types
-  IWebSocketSenseAugmentation,
-  IWebSocketConduitAugmentation,
-  IWebSocketCognitionAugmentation,
-  IWebSocketMemoryAugmentation,
-  IWebSocketPerceptionAugmentation,
-  IWebSocketDialogAugmentation,
-  IWebSocketActivationAugmentation,
-
-  // Function to add WebSocket support to any augmentation
-  addWebSocketSupport
-} from '@soulcraft/brainy'
-
-// Example: Creating a typed WebSocket-enabled sense augmentation
-const mySenseAug = createSenseAugmentation({
-  name: 'my-sense',
-  processRawData: async (data, dataType) => {
-    // Implementation
-    return {
-      success: true,
-      data: { nouns: [], verbs: [] }
-    }
-  }
-}) as IWebSocketSenseAugmentation
-
-// Add WebSocket support
-addWebSocketSupport(mySenseAug, {
-  connectWebSocket: async (url) => {
-    // WebSocket implementation
-    return {
-      connectionId: 'ws-1',
-      url,
-      status: 'connected'
-    }
-  },
-  sendWebSocketMessage: async (connectionId, data) => {
-    // Send message implementation
-  },
-  onWebSocketMessage: async (connectionId, callback) => {
-    // Register callback implementation
-  },
-  offWebSocketMessage: async (connectionId, callback) => {
-    // Remove callback implementation
-  },
-  closeWebSocket: async (connectionId, code, reason) => {
-    // Close connection implementation
-  }
+import { createAutoBrainy } from 'brainy'
+import { serve } from 'https://deno.land/std@0.168.0/http/server.ts'
+
+serve(async (req) => {
+  const brainy = createAutoBrainy()
+  const url = new URL(req.url)
+  const query = url.searchParams.get('q')
+  
+  const results = await brainy.searchText(query, 10)
+  
+  return new Response(JSON.stringify(results), {
+    headers: { 'Content-Type': 'application/json' }
+  })
 })
-
-// Now mySenseAug has both sense augmentation methods and WebSocket methods
-await mySenseAug.processRawData('data', 'text')
-await mySenseAug.connectWebSocket('wss://example.com')
 ```
 
-These WebSocket augmentation types combine the base augmentation interfaces with the `IWebSocketSupport` interface,
-providing type safety and autocompletion for augmentations with WebSocket capabilities.
-
-### Model Control Protocol (MCP)
-
-Brainy includes a Model Control Protocol (MCP) implementation that allows external models to access Brainy data and use
-the augmentation pipeline as tools:
-
-- **BrainyMCPAdapter**: Provides access to Brainy data through MCP
-- **MCPAugmentationToolset**: Exposes the augmentation pipeline as tools
-- **BrainyMCPService**: Integrates the adapter and toolset, providing WebSocket and REST server implementations
-
-Environment compatibility:
+### Docker Container
 
-- **BrainyMCPAdapter** and **MCPAugmentationToolset** can run in any environment (browser, Node.js, server)
-- **BrainyMCPService** core functionality works in any environment
+```dockerfile
+FROM node:20-alpine
+USER node
+WORKDIR /app
+COPY package*.json ./
+RUN npm install brainy
+COPY . .
 
-For detailed documentation and usage examples, see the [MCP documentation](src/mcp/README.md).
-
-## Cross-Environment Compatibility
-
-Brainy is designed to run seamlessly in any environment, from browsers to Node.js to serverless functions and
-containers. All Brainy data, functions, and augmentations are environment-agnostic, allowing you to use the same code
-everywhere.
-
-### Environment Detection
-
-Brainy automatically detects the environment it's running in:
-
-```typescript
-import { environment } from '@soulcraft/brainy'
-
-// Check which environment we're running in
-console.log(`Running in ${
-  environment.isBrowser ? 'browser' :
-    environment.isNode ? 'Node.js' :
-      'serverless/unknown'
-} environment`)
+CMD ["node", "server.js"]
 ```
 
-### Adaptive Storage
-
-Storage adapters are automatically selected based on the environment:
-
-- **Browser**: Uses Origin Private File System (OPFS) when available, falls back to in-memory storage
-- **Node.js**: Uses file system storage by default, with options for S3-compatible cloud storage
-- **Serverless**: Uses in-memory storage with options for cloud persistence
-- **Container**: Automatically detects and uses the appropriate storage based on available capabilities
-
-### Dynamic Imports
-
-Brainy uses dynamic imports to load environment-specific dependencies only when needed, keeping the bundle size small
-and ensuring compatibility across environments.
-
-### Browser Support
-
-Works in all modern browsers:
-
-- Chrome 86+
-- Edge 86+
-- Opera 72+
-- Chrome for Android 86+
+```javascript
+// server.js
+import { createAutoBrainy } from 'brainy'
+import express from 'express'
 
-For browsers without OPFS support, falls back to in-memory storage.
+const app = express()
+const brainy = createAutoBrainy()
 
-## Related Projects
+app.get('/search', async (req, res) => {
+  const results = await brainy.searchText(req.query.q, 10)
+  res.json(results)
+})
 
-- **[Cartographer](https://github.com/sodal-project/cartographer)** - A companion project that provides standardized
-  interfaces for interacting with Brainy
+app.listen(3000, () => console.log('Brainy running on port 3000'))
+```
 
-## Demo
+### Kubernetes
 
-The repository includes a comprehensive demo that showcases Brainy's main features:
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: brainy-api
+spec:
+  replicas: 3
+  template:
+    spec:
+      containers:
+      - name: brainy
+        image: your-registry/brainy-api:latest
+        env:
+        - name: S3_BUCKET
+          value: "your-vector-bucket"
+```
 
-- `demo/index.html` - A single demo page with animations demonstrating Brainy's features.
-    - **[Try the live demo](https://soulcraft-research.github.io/brainy/demo/index.html)** - Check out the
-      interactive demo on
-      GitHub Pages
-    - Or run it locally with `npm run demo` (see [demo instructions](demo.md) for details)
-    - To deploy your own version to GitHub Pages, use the GitHub Actions workflow in
-      `.github/workflows/deploy-demo.yml`,
-      which automatically deploys when pushing to the main branch or can be manually triggered
-    - To use a custom domain (like www.soulcraft.com):
-        1. A CNAME file is already included in the demo directory
-            2. In your GitHub repository settings, go to Pages > Custom domain and enter your domain
-            3. Configure your domain's DNS settings to point to GitHub Pages:
+### Railway.app
 
-            - Add a CNAME record for www pointing to `<username>.github.io` (e.g., `soulcraft-research.github.io`)
-            - Or for an apex domain (soulcraft.com), add A records pointing to GitHub Pages IP addresses
+```javascript
+// server.js
+import { createAutoBrainy } from 'brainy'
 
-The demo showcases:
+const brainy = createAutoBrainy({
+  bucketName: process.env.RAILWAY_VOLUME_NAME
+})
 
-- How Brainy runs in different environments (browser, Node.js, server, cloud)
-- How the noun-verb data model works
-- How HNSW search works
+// Railway automatically handles the rest!
+```
 
-## Syncing Brainy Instances
+### Render.com
 
-You can use the conduit augmentations to sync Brainy instances:
+```yaml
+# render.yaml
+services:
+  - type: web
+    name: brainy-api
+    env: node
+    buildCommand: npm install brainy
+    startCommand: node server.js
+    envVars:
+      - key: BRAINY_STORAGE
+        value: persistent-disk
+```
 
-- **WebSocket iConduit**: For syncing between browsers and servers, or between servers. WebSockets cannot be used for
-  direct browser-to-browser communication without a server in the middle.
-- **WebRTC iConduit**: For direct peer-to-peer syncing between browsers. This is the recommended approach for
-  browser-to-browser communication.
+## 🚀 Quick Examples
 
-#### WebSocket Sync Example
+### Basic Usage
 
-```typescript
-import {
-  BrainyData,
-  pipeline,
-  createConduitAugmentation
-} from '@soulcraft/brainy'
+```javascript
+import { BrainyData, NounType, VerbType } from 'brainy'
 
-// Create and initialize the database
+// Initialize
 const db = new BrainyData()
 await db.init()
 
-// Create a WebSocket conduit augmentation
-const wsConduit = await createConduitAugmentation('websocket', 'my-websocket-sync')
-
-// Register the augmentation with the pipeline
-pipeline.register(wsConduit)
-
-// Connect to another Brainy instance (server or browser)
-// Replace the example URL below with your actual WebSocket server URL
-const connectionResult = await pipeline.executeConduitPipeline(
-  'establishConnection',
-  ['wss://example-websocket-server.com/brainy-sync', { protocols: 'brainy-sync' }]
-)
-
-if (connectionResult[0] && (await connectionResult[0]).success) {
-  const connection = (await connectionResult[0]).data
-
-  // Read data from the remote instance
-  const readResult = await pipeline.executeConduitPipeline(
-    'readData',
-    [{ connectionId: connection.connectionId, query: { type: 'getAllNouns' } }]
-  )
+// Add data (automatically vectorized)
+const catId = await db.add("Cats are independent pets", {
+  noun: NounType.Thing,
+  category: 'animal'
+})
 
-  // Process and add the received data to the local instance
-  if (readResult[0] && (await readResult[0]).success) {
-    const remoteNouns = (await readResult[0]).data
-    for (const noun of remoteNouns) {
-      await db.add(noun.vector, noun.metadata)
-    }
-  }
+// Search for similar items
+const results = await db.searchText("feline pets", 5)
 
-  // Set up real-time sync by monitoring the stream
-  await wsConduit.monitorStream(connection.connectionId, async (data) => {
-    // Handle incoming data (e.g., new nouns, verbs, updates)
-    if (data.type === 'newNoun') {
-      await db.add(data.vector, data.metadata)
-    } else if (data.type === 'newVerb') {
-      await db.addVerb(data.sourceId, data.targetId, data.vector, data.options)
-    }
-  })
-}
+// Add relationships
+await db.addVerb(catId, dogId, {
+  verb: VerbType.RelatedTo,
+  description: 'Both are pets'
+})
 ```
 
-#### WebRTC Peer-to-Peer Sync Example
+### AutoBrainy (Recommended)
 
-```typescript
-import {
-  BrainyData,
-  pipeline,
-  createConduitAugmentation
-} from '@soulcraft/brainy'
-
-// Create and initialize the database
-const db = new BrainyData()
-await db.init()
+```javascript
+import { createAutoBrainy } from 'brainy'
 
-// Create a WebRTC conduit augmentation
-const webrtcConduit = await createConduitAugmentation('webrtc', 'my-webrtc-sync')
-
-// Register the augmentation with the pipeline
-pipeline.register(webrtcConduit)
-
-// Connect to a peer using a signaling server
-// Replace the example values below with your actual configuration
-const connectionResult = await pipeline.executeConduitPipeline(
-  'establishConnection',
-  [
-    'peer-id-to-connect-to', // Replace with actual peer ID
-    {
-      signalServerUrl: 'wss://example-signal-server.com', // Replace with your signal server
-      localPeerId: 'my-local-peer-id', // Replace with your local peer ID
-      iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] // Public STUN server
-    }
-  ]
-)
-
-if (connectionResult[0] && (await connectionResult[0]).success) {
-  const connection = (await connectionResult[0]).data
-
-  // Set up real-time sync by monitoring the stream
-  await webrtcConduit.monitorStream(connection.connectionId, async (data) => {
-    // Handle incoming data (e.g., new nouns, verbs, updates)
-    if (data.type === 'newNoun') {
-      await db.add(data.vector, data.metadata)
-    } else if (data.type === 'newVerb') {
-      await db.addVerb(data.sourceId, data.targetId, data.vector, data.options)
-    }
-  })
+// Everything auto-configured!
+const brainy = createAutoBrainy()
 
-  // When adding new data locally, also send to the peer
-  const nounId = await db.add("New data to sync", { noun: "Thing" })
-
-  // Send the new noun to the peer
-  await pipeline.executeConduitPipeline(
-    'writeData',
-    [
-      {
-        connectionId: connection.connectionId,
-        data: {
-          type: 'newNoun',
-          id: nounId,
-          vector: (await db.get(nounId)).vector,
-          metadata: (await db.get(nounId)).metadata
-        }
-      }
-    ]
-  )
-}
+// Just start using it
+await brainy.addVector({ id: '1', vector: [0.1, 0.2, 0.3], text: 'Hello' })
+const results = await brainy.search([0.1, 0.2, 0.3], 10)
 ```
 
-#### Browser-Server Search Example
-
-Brainy supports searching a server-hosted instance from a browser, storing results locally, and performing further
-searches against the local instance:
-
-```typescript
-import { BrainyData } from '@soulcraft/brainy'
-
-// Create and initialize the database with remote server configuration
-// Replace the example URL below with your actual Brainy server URL
-const db = new BrainyData({
-  remoteServer: {
-    url: 'wss://example-brainy-server.com/ws', // Replace with your server URL
-    protocols: 'brainy-sync',
-    autoConnect: true // Connect automatically during initialization
-  }
-})
-await db.init()
-
-// Or connect manually after initialization
-if (!db.isConnectedToRemoteServer()) {
-  // Replace the example URL below with your actual Brainy server URL
-  await db.connectToRemoteServer('wss://example-brainy-server.com/ws', 'brainy-sync')
-}
-
-// Search the remote server (results are stored locally)
-const remoteResults = await db.searchText('machine learning', 5, { searchMode: 'remote' })
-
-// Search the local database (includes previously stored results)
-const localResults = await db.searchText('machine learning', 5, { searchMode: 'local' })
+### Scenario-Based Setup
 
-// Perform a combined search (local first, then remote if needed)
-const combinedResults = await db.searchText('neural networks', 5, { searchMode: 'combined' })
+```javascript
+import { createQuickBrainy } from 'brainy'
 
-// Add data to both local and remote instances
-const id = await db.addToBoth('Deep learning is a subset of machine learning', {
-  noun: 'Concept',
-  category: 'AI',
-  tags: ['deep learning', 'neural networks']
+// Choose your scale: 'small', 'medium', 'large', 'enterprise'
+const brainy = await createQuickBrainy('large', {
+  bucketName: 'my-vector-db'
 })
-
-// Clean up when done (this also cleans up worker pools)
-await db.shutDown()
 ```
 
----
-
-## 📈 Scaling Strategy
-
-Brainy is designed to handle datasets of various sizes, from small collections to large-scale deployments. For
-terabyte-scale data that can't fit entirely in memory, we provide several approaches:
-
-- **Disk-Based HNSW**: Modified implementations using intelligent caching and partial loading
-- **Distributed HNSW**: Sharding and partitioning across multiple machines
-- **Hybrid Solutions**: Combining quantization techniques with multi-tier architectures
-
-For detailed information on how to scale Brainy for large datasets, vector dimension standardization, threading
-implementation, storage testing, and other technical topics, see our
-comprehensive [Technical Guides](TECHNICAL_GUIDES.md).
-
-## Recent Changes and Performance Improvements
-
-### Enhanced Memory Management and Scalability
-
-Brainy has been significantly improved to handle larger datasets more efficiently:
-
-- **Pagination Support**: All data retrieval methods now support pagination to avoid loading entire datasets into memory
-  at once. The deprecated `getAllNouns()` and `getAllVerbs()` methods have been replaced with `getNouns()` and
-  `getVerbs()` methods that support pagination, filtering, and cursor-based navigation.
-
-- **Multi-level Caching**: A sophisticated three-level caching strategy has been implemented:
-    - **Level 1**: Hot cache (most accessed nodes) - RAM (automatically detecting and adjusting in each environment)
-    - **Level 2**: Warm cache (recent nodes) - OPFS, Filesystem or S3 depending on environment
-    - **Level 3**: Cold storage (all nodes) - OPFS, Filesystem or S3 depending on environment
-
-- **Adaptive Memory Usage**: The system automatically detects available memory and adjusts cache sizes accordingly:
-    - In Node.js: Uses 10% of free memory (minimum 1000 entries)
-    - In browsers: Scales based on device memory (500 entries per GB, minimum 1000)
-
-- **Intelligent Cache Eviction**: Implements a Least Recently Used (LRU) policy that evicts the oldest 20% of items when
-  the cache reaches the configured threshold.
-
-- **Prefetching Strategy**: Implements batch prefetching to improve performance while avoiding overwhelming system
-  resources.
-
-### S3-Compatible Storage Improvements
-
-- **Enhanced Cloud Storage**: Improved support for S3-compatible storage services including AWS S3, Cloudflare R2, and
-  others.
-
-- **Optimized Data Access**: Batch operations and error handling for efficient cloud storage access.
-
-- **Change Log Management**: Efficient synchronization through change logs to track updates.
-
-### Data Compatibility
-
-Yes, you can use existing data indexed from an old version. Brainy includes robust data migration capabilities:
-
-- **Vector Regeneration**: If vectors are missing in imported data, they will be automatically created using the
-  embedding function.
-
-- **HNSW Index Reconstruction**: The system can reconstruct the HNSW index from backup data, ensuring compatibility with
-  previous versions.
-
-- **Sparse Data Import**: Support for importing sparse data (without vectors) through the `importSparseData()` method.
-
-### System Requirements
-
-#### Default Mode
-
-- **Memory**:
-    - Minimum: 512MB RAM
-    - Recommended: 2GB+ RAM for medium datasets, 8GB+ for large datasets
-
-- **CPU**:
-    - Minimum: 2 cores
-    - Recommended: 4+ cores for better performance with parallel operations
-
-- **Storage**:
-    - Minimum: 1GB available storage
-    - Recommended: Storage space at least 3x the size of your dataset
-
-#### Read-Only Mode
-
-Read-only mode prevents all write operations (add, update, delete) and is optimized for search operations.
-
-- **Memory**:
-    - Minimum: 256MB RAM
-    - Recommended: 1GB+ RAM
-
-- **CPU**:
-    - Minimum: 1 core
-    - Recommended: 2+ cores
-
-- **Storage**:
-    - Minimum: Storage space equal to the size of your dataset
-    - Recommended: 2x the size of your dataset for caching
-
-- **New Feature**: Lazy loading support in read-only mode for improved performance with large datasets.
-
-#### Write-Only Mode
-
-Write-only mode prevents all search operations and is optimized for initial data loading or when you want to optimize
-for write performance.
-
-- **Memory**:
-    - Minimum: 512MB RAM
-    - Recommended: 2GB+ RAM
-
-- **CPU**:
-    - Minimum: 2 cores
-    - Recommended: 4+ cores for faster data ingestion
-
-- **Storage**:
-    - Minimum: Storage space at least 2x the size of your dataset
-    - Recommended: 4x the size of your dataset for optimal performance
-
-### Performance Tuning Parameters
-
-Brainy offers comprehensive configuration options for performance tuning, with enhanced support for large datasets in S3
-or other remote storage. **All configuration is optional** - the system automatically detects the optimal settings based
-on your environment, dataset size, and usage patterns.
-
-#### Intelligent Defaults
-
-Brainy uses intelligent defaults that automatically adapt to your environment:
-
-- **Environment Detection**: Automatically detects whether you're running in Node.js, browser, or worker environment
-- **Memory-Aware Caching**: Adjusts cache sizes based on available system memory
-- **Dataset Size Adaptation**: Tunes parameters based on the size of your dataset
-- **Usage Pattern Optimization**: Adjusts to read-heavy vs. write-heavy workloads
-- **Storage Type Awareness**: Optimizes for local vs. remote storage (S3, R2, etc.)
-- **Operating Mode Specialization**: Special optimizations for read-only and write-only modes
-
-#### Cache Configuration (Optional)
-
-You can override any of these automatically tuned parameters if needed:
-
-- **Hot Cache Size**: Control the maximum number of items to keep in memory.
-    - For large datasets (>100K items), consider values between 5,000-50,000 depending on available memory.
-    - In read-only mode, larger values (10,000-100,000) can be used for better performance.
-
-- **Eviction Threshold**: Set the threshold at which cache eviction begins (default: 0.8 or 80% of max size).
-    - For write-heavy workloads, lower values (0.6-0.7) may improve performance.
-    - For read-heavy workloads, higher values (0.8-0.9) are recommended.
-
-- **Warm Cache TTL**: Set the time-to-live for items in the warm cache (default: 3600000 ms or 1 hour).
-    - For frequently changing data, shorter TTLs are recommended.
-    - For relatively static data, longer TTLs improve performance.
-
-- **Batch Size**: Control the number of items to process in a single batch for operations like prefetching.
-    - For S3 or remote storage with large datasets, larger values (50-200) significantly improve throughput.
-    - In read-only mode with remote storage, even larger values (100-300) can be used.
-
-#### Auto-Tuning (Enabled by Default)
-
-- **Auto-Tune**: Enable or disable automatic tuning of cache parameters based on usage patterns (default: true).
-- **Auto-Tune Interval**: Set how frequently the system adjusts cache parameters (default: 60000 ms or 1 minute).
-
-#### Read-Only Mode Optimizations (Automatic)
-
-Read-only mode includes special optimizations for search performance that are automatically applied:
-
-- **Larger Cache Sizes**: Automatically uses more memory for caching (up to 40% of free memory for large datasets).
-- **Aggressive Prefetching**: Loads more data in each batch to reduce the number of storage requests.
-- **Prefetch Strategy**: Defaults to 'aggressive' prefetching strategy in read-only mode.
-
-#### Example Configuration for Large S3 Datasets
+### With Offline Models
 
 ```javascript
-const brainy = new BrainyData({
-  readOnly: true,
-  lazyLoadInReadOnlyMode: true,
-  storage: {
-    type: 's3',
-    s3Storage: {
-      bucketName: 'your-bucket',
-      accessKeyId: 'your-access-key',
-      secretAccessKey: 'your-secret-key',
-      region: 'your-region'
-    }
-  },
-  cache: {
-    hotCacheMaxSize: 20000,
-    hotCacheEvictionThreshold: 0.85,
-    batchSize: 100,
-    readOnlyMode: {
-      hotCacheMaxSize: 50000,
-      batchSize: 200,
-      prefetchStrategy: 'aggressive'
-    }
-  }
-});
-```
-
-These configuration options make Brainy more efficient, scalable, and adaptable to different environments and usage
-patterns, especially for large datasets in cloud storage.
-
-## Testing
-
-Brainy uses Vitest for testing. For detailed information about testing in Brainy, including test configuration, scripts,
-reporting tools, and best practices, see our [Testing Guide](docs/technical/TESTING.md).
+import { createAutoBrainy } from 'brainy'
+import { BundledUniversalSentenceEncoder } from '@soulcraft/brainy-models'
 
-Here are some common test commands:
-
-```bash
-# Run all tests
-npm test
-
-# Run tests with comprehensive reporting
-npm run test:report
+// Use bundled model for offline operation
+const brainy = createAutoBrainy({
+  embeddingModel: BundledUniversalSentenceEncoder,
+  // Model loads from local files, no network needed!
+})
 
-# Run tests with coverage
-npm run test:coverage
+// Works exactly the same, but 100% offline
+await brainy.add("This works without internet!", { 
+  noun: NounType.Content 
+})
 ```
 
-## Contributing
+## 🌐 Live Demo
 
-For detailed contribution guidelines, please see [CONTRIBUTING.md](CONTRIBUTING.md).
+**[Try the interactive demo](https://soulcraft-research.github.io/brainy/demo/index.html)** - See Brainy in action with animations and examples.
 
-For developer documentation, including building, testing, and publishing instructions, please
-see [DEVELOPERS.md](DEVELOPERS.md).
+## 🔧 Environment Support
 
-We have a [Code of Conduct](CODE_OF_CONDUCT.md) that all contributors are expected to follow.
+| Environment | Storage | Threading | Auto-Configured |
+|-------------|---------|-----------|-----------------|
+| Browser | OPFS | Web Workers | ✅ |
+| Node.js | FileSystem/S3 | Worker Threads | ✅ |
+| Serverless | Memory/S3 | Limited | ✅ |
+| Edge Functions | Memory/KV | Limited | ✅ |
 
-### Commit Message Format
+## 📚 Documentation
 
-For best results with automatic changelog generation, follow
-the [Conventional Commits](https://www.conventionalcommits.org/) specification for your commit messages:
+### Getting Started
+- [**Quick Start Guide**](docs/getting-started/) - Get up and running in minutes
+- [**Installation**](docs/getting-started/installation.md) - Detailed setup instructions
+- [**Environment Setup**](docs/getting-started/environment-setup.md) - Platform-specific configuration
 
-```
-AI Template for automated commit messages:
+### User Guides
+- [**Search and Metadata**](docs/user-guides/) - Advanced search techniques
+- [**JSON Document Search**](docs/guides/json-document-search.md) - Field-based searching
+- [**Production Migration**](docs/guides/production-migration-guide.md) - Deployment best practices
 
-Use Conventional Commit format
-Specify the changes in a structured format
-Add information about the purpose of the commit
-```
+### API Reference
+- [**Core API**](docs/api-reference/) - Complete method reference
+- [**Configuration Options**](docs/api-reference/configuration.md) - All configuration parameters
+- [**Auto-Configuration API**](docs/api-reference/auto-configuration-api.md) - Intelligent setup
 
-```
-<type>(<scope>): <description>
+### Optimization & Scaling
+- [**Large-Scale Optimizations**](docs/optimization-guides/) - Handle millions of vectors
+- [**Memory Management**](docs/optimization-guides/memory-optimization.md) - Efficient resource usage
+- [**S3 Migration Guide**](docs/optimization-guides/s3-migration-guide.md) - Cloud storage setup
 
-[optional body]
+### Examples & Patterns
+- [**Code Examples**](docs/examples/) - Real-world usage patterns
+- [**Integrations**](docs/examples/integrations.md) - Third-party services
+- [**Performance Patterns**](docs/examples/performance.md) - Optimization techniques
 
-[optional footer(s)]
-```
+### Technical Documentation
+- [**Architecture Overview**](docs/technical/) - System design and internals
+- [**Testing Guide**](docs/technical/TESTING.md) - Testing strategies
+- [**Statistics & Monitoring**](docs/technical/STATISTICS.md) - Performance tracking
 
-Where `<type>` is one of:
+## 🤝 Contributing
 
-- `feat`: A new feature (maps to **Added** section)
-- `fix`: A bug fix (maps to **Fixed** section)
-- `chore`: Regular maintenance tasks (maps to **Changed** section)
-- `docs`: Documentation changes (maps to **Documentation** section)
-- `refactor`: Code changes that neither fix bugs nor add features (maps to **Changed** section)
-- `perf`: Performance improvements (maps to **Changed** section)
+We welcome contributions! Please see:
+- [Contributing Guidelines](CONTRIBUTING.md)
+- [Developer Documentation](docs/development/DEVELOPERS.md)
+- [Code of Conduct](CODE_OF_CONDUCT.md)
 
-### Manual Release Process
+## 📄 License
 
-If you need more control over the release process, you can use the individual commands:
+[MIT](LICENSE)
 
-```bash
-# Update version and generate changelog
-npm run _release:patch  # or _release:minor, _release:major
+## 🔗 Related Projects
 
-# Create GitHub release
-npm run _github-release
+- [**Cartographer**](https://github.com/sodal-project/cartographer) - Standardized interfaces for Brainy
 
-# Publish to NPM
-npm publish
-```
-
-## License
+---
 
-[MIT](LICENSE)
+<div align="center">
+<strong>Ready to build something amazing? Get started with Brainy today!</strong>
+</div>