@rbalchii/anchor-engine 4.7.0

package/docs/archive/query-builder-documentation.md

@@ -0,0 +1,218 @@
+# Anchor Query Builder Documentation
+
+## Overview
+
+The Anchor Query Builder provides a simplified, fluent API for constructing database queries with JavaScript transformations. It's designed to be LLM-friendly and human-readable while maintaining performance by leveraging the underlying PGlite database.
+
+## Basic Usage
+
+### Import the Query Builder
+```javascript
+import { createQueryBuilder } from './services/query-builder/QueryBuilder.js';
+import { db } from '../core/db.js';
+```
+
+### Simple Query Example
+```javascript
+// Basic query to get recent atoms
+const queryBuilder = createQueryBuilder(db, 'atoms');
+const results = await queryBuilder
+  .select(['timestamp', 'content', 'buckets'])
+  .where('content', 'LIKE', 'test')
+  .orderBy('timestamp', 'DESC')
+  .limit(10)
+  .execute();
+```
+
+## Available Methods
+
+### `select(fields: string[])`
+Specify which fields to retrieve from the table.
+
+```javascript
+// Select specific fields
+createQueryBuilder(db, 'atoms').select(['content', 'timestamp']);
+
+// Select all fields - use ['*'] or omit select() entirely
+createQueryBuilder(db, 'atoms').select(['*']);
+// or
+createQueryBuilder(db, 'atoms'); // defaults to SELECT *
+```
+
+**Note:** The wildcard `'*'` is specially handled and will generate `SELECT *` without quotes. All other field names are validated and quoted for SQL safety.
+
+### `where(field: string, operator: string, value: any)`
+Add a WHERE condition to filter results.
+
+```javascript
+// Various operators
+createQueryBuilder(db, 'atoms').where('content', 'LIKE', 'search term');
+createQueryBuilder(db, 'atoms').where('content', 'CONTAINS', 'specific text');
+createQueryBuilder(db, 'atoms').where('timestamp', '>', 1640995200);
+createQueryBuilder(db, 'atoms').where('timestamp', '>=', 1640995200);
+createQueryBuilder(db, 'atoms').where('timestamp', '<', 1672531200);
+createQueryBuilder(db, 'atoms').where('timestamp', '<=', 1672531200);
+createQueryBuilder(db, 'atoms').where('buckets', '=', 'inbox');
+```
+
+### `orderBy(field: string, direction: 'ASC' | 'DESC')`
+Sort the results by a specific field.
+
+```javascript
+// Ascending order (default)
+createQueryBuilder(db, 'atoms').orderBy('timestamp', 'ASC');
+
+// Descending order
+createQueryBuilder(db, 'atoms').orderBy('timestamp', 'DESC');
+```
+
+### `limit(count: number)`
+Limit the number of results returned.
+
+```javascript
+// Get only the first 5 results
+createQueryBuilder(db, 'atoms').limit(5);
+```
+
+### `transform(transformations: Record<string, TransformFunction>)`
+Apply JavaScript transformations to create computed fields.
+
+```javascript
+// Transform data during query execution
+const queryBuilder = createQueryBuilder(db, 'atoms');
+const results = await queryBuilder
+  .select(['timestamp', 'content'])
+  .where('content', 'LIKE', 'Order Delivered')
+  .transform({
+    date: (row) => new Date(row.timestamp / 1000).toISOString(),
+    vendor: (row) => row.content.split(',')[0],
+    items: (row) => row.content.split(',')[1],
+    revenue: (row) => {
+      const match = row.content.match(/,([\d\.]+),"Order Delivered/);
+      return match ? parseFloat(match[1]) / 100.00 : 0;
+    }
+  })
+  .execute();
+```
+
+### `execute(): Promise<any[]>`
+Execute the query and return results.
+
+```javascript
+const queryBuilder = createQueryBuilder(db, 'atoms');
+const results = await queryBuilder
+  .select(['*'])
+  .where('content', 'LIKE', 'test')
+  .execute();
+```
+
+### `toDataFrame(): Promise<DataFrame>`
+Execute the query and return a DataFrame for in-memory operations.
+
+```javascript
+const queryBuilder = createQueryBuilder(db, 'atoms');
+const df = await queryBuilder
+  .select(['timestamp', 'content'])
+  .where('content', 'LIKE', 'test')
+  .toDataFrame();
+```
+
+### `export(filename: string, format: 'csv' | 'json' | 'yaml' | 'table')`
+Execute the query and export results to a file in the specified format.
+
+```javascript
+// Export to CSV
+const queryBuilder = createQueryBuilder(db, 'atoms');
+await queryBuilder
+  .select(['timestamp', 'content'])
+  .where('content', 'LIKE', 'test')
+  .export('results.csv', 'csv');
+
+// Export to JSON
+await queryBuilder
+  .select(['timestamp', 'content'])
+  .where('content', 'LIKE', 'test')
+  .export('results.json', 'json');
+```
+
+## Complete Examples
+
+### DoorDash Data Analysis
+```javascript
+// Analyze DoorDash delivery data
+import { createQueryBuilder } from './services/query-builder/QueryBuilder.js';
+import { db } from '../core/db.js';
+
+const queryBuilder = createQueryBuilder(db, 'atoms');
+const dashResults = await queryBuilder
+  .select(['timestamp', 'content'])
+  .where('content', 'LIKE', 'Order Delivered')
+  .orderBy('timestamp', 'ASC')
+  .limit(100)
+  .transform({
+    date: (row) => new Date(row.timestamp / 1000).toISOString(),
+    vendor: (row) => row.content.split(',')[0],
+    items: (row) => row.content.split(',')[1],
+    revenue: (row) => {
+      const match = row.content.match(/,([\d\.]+),"Order Delivered/);
+      return match ? parseFloat(match[1]) / 100.00 : 0;
+    }
+  })
+  .execute();
+
+console.log(dashResults);
+```
+
+### Export Data to Different Formats
+```javascript
+import { createQueryBuilder } from './services/query-builder/QueryBuilder.js';
+import { db } from '../core/db.js';
+
+const queryBuilder = createQueryBuilder(db, 'atoms');
+
+// Export to CSV
+await queryBuilder
+  .select(['timestamp', 'content', 'buckets'])
+  .where('buckets', '=', 'inbox')
+  .orderBy('timestamp', 'DESC')
+  .limit(50)
+  .export('inbox_data.csv', 'csv');
+
+// Export to JSON
+await queryBuilder
+  .select(['timestamp', 'content', 'buckets'])
+  .where('buckets', '=', 'inbox')
+  .orderBy('timestamp', 'DESC')
+  .limit(50)
+  .export('inbox_data.json', 'json');
+```
+
+## Using with the Anchor CLI
+
+The query builder is available programmatically in your scripts. Import it using:
+
+```javascript
+import { createQueryBuilder } from './services/query-builder/QueryBuilder.js';
+import { db } from '../core/db.js';
+```
+
+## Performance Considerations
+
+- The query builder translates operations to efficient SQL queries
+- Filtering and sorting happen at the database level for optimal performance
+- JavaScript transformations are applied after data retrieval
+- Use `limit()` to restrict result set sizes for better performance
+
+## Error Handling
+
+```javascript
+try {
+  const queryBuilder = createQueryBuilder(db, 'atoms');
+  const results = await queryBuilder
+    .select(['*'])
+    .where('content', 'LIKE', 'test')
+    .execute();
+} catch (error) {
+  console.error('Query failed:', error);
+}
+```

package/docs/archive/quick-reference.md

@@ -0,0 +1,40 @@
+# ECE_Core Quick Reference
+
+## Core Concepts
+- **Tag-Walker Protocol**: Graph-based search replacing vector search
+- **Atomic Architecture**: Compounds → Molecules → Atoms (V4 taxonomy)
+- **Sovereign Context**: Local-first, privacy-preserving memory engine
+- **Relationship Discovery**: Entity co-occurrence for relationship mapping
+
+## Key Endpoints
+```
+POST /v1/ingest          # Ingest content
+POST /v1/memory/search   # Semantic search
+GET  /health             # System health
+GET  /monitoring/metrics # Performance metrics
+GET  /monitoring/dashboard # Real-time monitoring
+```
+
+## Performance Features
+- Native C++ acceleration for critical operations
+- SimHash for deduplication
+- Graph-based retrieval (no vector embeddings)
+- CozoDB (RocksDB) for unified storage
+
+## Configuration
+- Settings in `user_settings.json`
+- Model paths and parameters
+- Performance tuning options
+- Ingestion settings
+
+## Monitoring
+- Real-time dashboard access
+- Performance metrics tracking
+- Health check endpoints
+- Request tracing capabilities
+
+## Best Practices
+- Use semantic buckets for organization
+- Apply meaningful tags for retrieval
+- Monitor ingestion performance
+- Leverage relationship discovery for insights

package/docs/archive/quickstart.md

@@ -0,0 +1,63 @@
+# Quickstart Guide - Essential User Workflow
+
+**Status:** Active | **Authority:** Human-Locked | **Domain:** User Onboarding
+
+## Quick Launch
+
+### Prerequisites
+- Node.js v18+
+- PNPM package manager
+
+### Setup & Launch
+```bash
+# Clone and setup
+git clone <repository-url>
+cd Anchor
+pnpm install
+pnpm build
+
+# Launch the system
+./start.sh        # Linux/macOS
+.\start.bat      # Windows
+```
+
+Access UI at the configured server URL (default: http://localhost:3160, configurable in user_settings.json)
+
+## Core Operations
+
+### Data Ingestion
+- **File Drop**: Place files in `context/` directory
+- **API**: `POST /v1/ingest` endpoint
+- **Real-time Sync**: Watchdog monitors `context/` directory
+
+### Search Capabilities
+- **Natural Language**: Ask questions in plain English
+- **Tag-Based**: Use hashtags for filtering (`#tagname`)
+- **Tag-Walker**: Retrieve related concepts using graph-based associative search with 70/30 keyword/associative split
+
+### API Examples
+```bash
+# Ingest content
+curl -X POST http://localhost:3160/v1/ingest \
+  -H "Content-Type: application/json" \
+  -d '{"content": "Your content", "buckets": ["inbox"]}'
+
+# Search memory
+curl -X POST http://localhost:3160/v1/memory/search \
+  -H "Content-Type: application/json" \
+  -d '{"query": "search terms", "buckets": ["inbox"]}'
+
+# Get server configuration
+curl -X GET http://localhost:3160/v1/config
+```
+
+## Troubleshooting
+
+### Common Issues
+- **Slow startup**: First run includes database initialization
+- **UI delays**: Electron wrapper may take up to 15 seconds
+- **Direct access**: Available at the configured server URL (default: http://localhost:3160, configurable in user_settings.json) if needed
+
+### Health Checks
+- System status: `GET /health`
+- Performance metrics: `GET /monitoring/metrics`

package/docs/archive/relationship-narrative-discovery.md

@@ -0,0 +1,141 @@
+# Relationship Narrative Discovery Guide
+
+## Overview
+The ECE Core now implements the Semantic Shift Architecture (Standard 084) that enables relationship narrative discovery across any domain. This transforms the system from a keyword indexer to a relationship historian.
+
+## How It Works
+
+### 1. Entity Co-occurrence Detection
+When the system processes text, it looks for multiple entities appearing in the same semantic molecule:
+
+```
+Input: "Alice and Bob worked on the project together"
+Processing:
+  - Extract entities: ["Alice", "Bob"]
+  - Detect relationship indicator: "and" (connecting two people)
+  - Detect temporal reference: "together"
+  - Apply semantic tags: #Relationship, #Narrative
+```
+
+### 2. Semantic Category Application
+Instead of granular tags, the system applies high-level semantic categories:
+- `#Relationship`: When 2+ person entities appear together
+- `#Narrative`: When person + time reference appear together
+- `#Technical`: When technical terms appear
+- `#Location`: When geographic references appear
+
+### 3. Cross-Domain Application
+The same architecture works for any domain:
+
+**Personal Domain:**
+- Query: "Tell me about Alice and Bob"
+- System finds: Molecules containing both "Alice" and "Bob"
+- Returns: Relationship narratives between these entities
+
+**Industrial Domain:**
+- Query: "Tell me about CO2 and Sequestration"
+- System finds: Molecules containing both "CO2" and "Sequestration"
+- Returns: Industrial relationship narratives between these concepts
+
+## Search Examples
+
+### Relationship Queries
+```
+GET /v1/memory/search
+{
+  "query": "Alice and Bob relationship",
+  "buckets": [],
+  "max_chars": 20000,
+  "provenance": "all"
+}
+```
+
+### Narrative Queries
+```
+GET /v1/memory/search
+{
+  "query": "story about Alice and Bob",
+  "buckets": [],
+  "max_chars": 20000,
+  "provenance": "all"
+}
+```
+
+### Technical Queries
+```
+GET /v1/memory/search
+{
+  "query": "GLM-4.7-Flash performance",
+  "buckets": [],
+  "max_chars": 20000,
+  "provenance": "all"
+}
+```
+
+## Implementation Benefits
+
+### 1. Improved Relevance
+- Relationship queries return only relationship content
+- Technical queries return only technical content
+- Narrative queries return only story content
+
+### 2. Reduced Noise
+- No more "Jade" appearing in random code comments
+- No more technical logs mixed with personal conversations
+- Semantic filtering keeps domains separate
+
+### 3. Universal Application
+- Same architecture works for personal relationships
+- Same architecture works for industrial data
+- Same architecture works for technical documentation
+
+## Architecture Components
+
+### Semantic Molecule Processor
+- Processes text chunks into semantic molecules
+- Applies semantic categories based on entity interactions
+- Extracts atomic entities from within semantic molecules
+
+### Relationship Discovery Engine
+- Detects when entities appear together
+- Creates relationship tags for entity pairs
+- Builds narrative arcs from temporal sequences
+
+### Stateless Context Provider
+- Each query gets fresh context from ECE search
+- No session memory affecting responses
+- Responses grounded in relevant knowledge graph data only
+
+## Use Cases
+
+### Personal Domain
+- Finding relationship narratives between people
+- Tracking personal story arcs over time
+- Separating personal content from technical content
+
+### Industrial Domain  
+- Finding relationship patterns in industrial data
+- Tracking technical concept connections
+- Separating business data from personal notes
+
+### Research Domain
+- Finding connections between research topics
+- Tracking academic narrative arcs
+- Separating research content from personal content
+
+## Best Practices
+
+### For Relationship Queries
+- Use phrases like "X and Y" to trigger relationship detection
+- Include temporal references for narrative extraction
+- Use semantic categories like `#Relationship` to filter results
+
+### For Technical Queries
+- Use technical terms to trigger `#Technical` categorization
+- Include code context for better technical result filtering
+- Use `#Technical` semantic category to isolate technical content
+
+### For Cross-Domain Queries
+- The system automatically detects domain context
+- Entity co-occurrence works across all domains
+- Same search interface for all content types