amalfa 1.0.2 → 1.0.4

package/docs/COMMIT_GUIDELINES.md

@@ -1,367 +0,0 @@
-# Commit Guidelines
-
-**Purpose:** Define what should and should NOT be committed to the polyvis repository.
-
----
-
-## ✅ What TO Commit
-
-### Source Code
-- `.ts`, `.js` - TypeScript/JavaScript source files
-- `.html`, `.css` - HTML and stylesheets
-- `.md` - Documentation (markdown)
-- `.json` - Configuration files (package.json, tsconfig.json)
-- `.yaml`, `.yml` - Configuration files
-
-### Project Configuration
-- `.gitignore`, `.gitattributes`
-- `biome.json`, `tsconfig.json`
-- `package.json` (but NOT `bun.lockb` - see exceptions)
-- `.beads/` directory (Beads issue tracking)
-
-### Documentation & Assets
-- Small images (<500 KB) for documentation
-- Architecture diagrams (optimized)
-- Example data files (small samples, <100 KB)
-- README files, playbooks, guides
-
-### Scripts & Tools
-- Build scripts (`scripts/*.ts`, `scripts/*.sh`)
-- Development utilities
-- Migration scripts (with documentation)
-
----
-
-## ❌ What NOT to Commit
-
-### Generated Artifacts
-
-**Database Files:**
-```
-❌ *.db
-❌ *.db-wal
-❌ *.db-shm
-❌ *.sqlite
-❌ *.sqlite-wal
-❌ *.sqlite-shm
-```
-
-**Why:** Database files are generated from source JSON. The JSON is the source of truth.
-
-**What to do instead:**
-- Commit source JSON files in `public/data/`
-- Document database generation in README
-- Use `scripts/build-db.ts` to regenerate
-
----
-
-**Node Modules & Dependencies:**
-```
-❌ node_modules/
-❌ bun.lockb (too large, changes frequently)
-```
-
-**Why:** Dependencies should be installed via `bun install`, not committed.
-
-**What to do instead:**
-- List dependencies in `package.json`
-- Document installation steps in README
-- Use `.nvmrc` or similar for version pinning
-
----
-
-**Built/Bundled Files:**
-```
-❌ dist/*.bundle.js
-❌ dist/*.min.js
-❌ build/
-❌ out/
-```
-
-**Why:** Built files are generated from source via build scripts.
-
-**What to do instead:**
-- Commit source files in `src/`
-- Document build process in README
-- Use CI/CD to build on deployment
-
----
-
-**Cache & Temp Files:**
-```
-❌ .resonance/cache/
-❌ local_cache/
-❌ *.log
-❌ .DS_Store
-❌ *.tmp
-```
-
-**Why:** These are ephemeral, machine-specific files.
-
----
-
-### Large Binary Files
-
-**Research Papers:**
-```
-❌ *.pdf (research papers)
-❌ Large images (>1 MB)
-```
-
-**Why:** Git is optimized for text, not large binaries. They bloat the repo.
-
-**What to do instead:**
-- Link to papers via URL (arXiv, DOI)
-- Create `docs/REFERENCES.md` with links
-- For critical PDFs, use external storage (GitHub releases, S3)
-
----
-
-**Machine Learning Models:**
-```
-❌ *.gguf
-❌ *.pt
-❌ *.safetensors
-❌ *.onnx (>50 MB)
-```
-
-**Why:** Models are huge and change frequently.
-
-**What to do instead:**
-- Document model download URLs
-- Use model registry (Hugging Face, etc.)
-- Store model metadata/config only
-
----
-
-### Backups
-
-**Database Backups:**
-```
-❌ backups/db/*.db
-❌ *.backup
-❌ *.bak
-```
-
-**Why:** Backups belong in backup systems, not version control.
-
-**What to do instead:**
-- Use external backup solutions
-- Document backup/restore procedures
-- Keep backups out of git entirely
-
----
-
-### Test Artifacts
-
-**Test Databases:**
-```
-❌ test-*.db
-❌ canary-*.db
-❌ *-test.sqlite
-```
-
-**Why:** Test artifacts should be created on-demand during testing.
-
-**What to do instead:**
-- Use `beforeEach()` to create fresh test DBs
-- Clean up test artifacts in `afterEach()`
-- Gitignore test patterns
-
----
-
-## 🤔 Edge Cases
-
-### Small Sample Databases
-
-**✅ Acceptable IF:**
-- < 1 MB in size
-- Essential for examples/documentation
-- Clearly named (e.g., `examples/sample-small.db`)
-- Documented in README
-
-**❌ Never acceptable:**
-- Production database dumps
-- Full-sized test databases
-- Database backups
-
----
-
-### Configuration Files with Secrets
-
-**✅ Commit:**
-```
-config.example.yaml  # Template with placeholders
-.env.example         # Example environment variables
-```
-
-**❌ Never commit:**
-```
-.env                 # Actual secrets
-config.yaml          # With real API keys
-```
-
-**Pattern:** Commit examples/templates, never actual secrets.
-
----
-
-### Generated Documentation
-
-**It depends:**
-
-- **Commit:** Hand-written docs, playbooks, guides
-- **Don't commit:** Auto-generated API docs (can be regenerated)
-
-**Rule of thumb:** If a human wrote it, commit it. If a script generated it, don't.
-
----
-
-## 🔍 How to Check Before Committing
-
-### 1. Review Your Staged Files
-
-```bash
-git status
-git diff --cached
-```
-
-**Red flags:**
-- Files in `dist/`, `build/`, or `out/`
-- Files ending in `.db`, `.sqlite`, `.log`
-- Files > 1 MB
-- Files with "backup" or "test" in the name
-
----
-
-### 2. Use Git Hooks (Pre-commit)
-
-Create `.git/hooks/pre-commit`:
-
-```bash
-#!/bin/bash
-# Check for large files
-git diff --cached --name-only | while read file; do
-    if [ -f "$file" ]; then
-        size=$(wc -c < "$file" | awk '{print $1}')
-        if [ $size -gt 1048576 ]; then  # 1 MB
-            echo "⚠️  Warning: $file is larger than 1 MB ($size bytes)"
-            echo "   Consider adding to .gitignore or using external storage"
-        fi
-    fi
-done
-
-# Check for database files
-if git diff --cached --name-only | grep -E '\.(db|sqlite)$'; then
-    echo "❌ Error: Database files detected!"
-    echo "   Database files should not be committed."
-    exit 1
-fi
-```
-
----
-
-### 3. Audit Repository Size
-
-```bash
-# Check current repo size
-du -sh .git
-
-# List largest files in history
-git rev-list --objects --all \
-  | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \
-  | awk '/^blob/ {print $3, $4}' \
-  | sort -rn \
-  | head -20
-```
-
-If you see large files, they may need to be removed from history (see cleanup script).
-
----
-
-## 🧹 Cleaning Up Mistakes
-
-### If You Accidentally Committed Artifacts
-
-**Before pushing:**
-
-```bash
-# Remove from staging
-git reset HEAD path/to/artifact.db
-
-# Remove from last commit
-git rm --cached path/to/artifact.db
-git commit --amend --no-edit
-```
-
-**After pushing:**
-
-Use the cleanup script:
-
-```bash
-./scripts/cleanup-repo-artifacts.sh
-```
-
-**⚠️ Warning:** This rewrites history. Coordinate with your team first.
-
----
-
-## 📋 Quick Checklist
-
-Before committing, ask yourself:
-
-- [ ] Is this a source file I wrote/edited?
-- [ ] Can this file be regenerated from source?
-- [ ] Is this file larger than 1 MB?
-- [ ] Does this file contain secrets/credentials?
-- [ ] Is this a database, cache, or log file?
-- [ ] Would someone else need this to build the project?
-
-**If you answered:**
-- "Yes" to 1 or 6 → Commit it
-- "Yes" to 2-5 → DON'T commit it
-
----
-
-## 🎯 Summary Table
-
-| File Type | Commit? | Rationale |
-|-----------|---------|-----------|
-| `.ts`, `.js` (source) | ✅ Yes | Source code |
-| `.db`, `.sqlite` | ❌ No | Generated artifacts |
-| `package.json` | ✅ Yes | Dependency manifest |
-| `bun.lockb` | ❌ No | Too large, auto-generated |
-| `dist/*.js` | ❌ No | Built artifacts |
-| `.md` (docs) | ✅ Yes | Documentation |
-| `.pdf` (papers) | ❌ No | Too large, use links |
-| `*.log` | ❌ No | Ephemeral logs |
-| `.env` | ❌ No | Contains secrets |
-| `.env.example` | ✅ Yes | Template without secrets |
-| Small images (<500 KB) | ✅ Yes | Documentation assets |
-| Large images (>1 MB) | ❌ No | Use external hosting |
-| Test fixtures (<100 KB) | ✅ Yes | Required for tests |
-| Test databases | ❌ No | Generated during tests |
-| Scripts (`.sh`, `.ts`) | ✅ Yes | Automation tools |
-| Backups (`.bak`) | ❌ No | Use backup systems |
-
----
-
-## 🔗 Related Documentation
-
-- **Cleanup Script:** `scripts/cleanup-repo-artifacts.sh`
-- **Gitignore:** `.gitignore` (see patterns)
-- **Beads Playbooks:** `playbooks/beads-human-playbook.md` (for issues)
-- **Development Workflow:** `playbooks/development-workflow-playbook.md`
-
----
-
-## 💡 Philosophy
-
-**Guiding principle:** *The repository should contain the minimum necessary to build and understand the project.*
-
-**Corollary:** If it can be generated, downloaded, or derived from committed files, it should NOT be committed.
-
-**Goal:** Keep the repository lean, fast to clone, and easy to navigate.
-
----
-
-**Last updated:** 2026-01-05

package/docs/CONFIG_E2E_VALIDATION.md

@@ -1,147 +0,0 @@
-# Configuration System End-to-End Validation
-
-**Date**: 2026-01-07  
-**Test Status**: ✅ PASSED  
-**Database**: `.amalfa/multi-source-test.db`  
-**Test Script**: `scripts/test-config-search.ts`
-
-## Test Overview
-
-Validated the unified configuration system end-to-end after migrating from `polyvis.settings.json` to `amalfa.config.json`.
-
-## Test Sequence
-
-### 1. Document Creation
-Created test document: `../polyvis/docs/config-unification-test.md`
-- Purpose: Validate indexing and search
-- Content: Configuration migration strategy details
-- Tags: #configuration #migration #typescript #amalfa #testing
-
-### 2. Ingestion
-```bash
-bun run src/cli.ts init
-```
-
-**Results**:
-- Files processed: 96 (including new test document)
-- Nodes created: 95
-- Embeddings: 95 (384-dim BAAI/bge-small-en-v1.5)
-- Database: `.amalfa/multi-source-test.db` (0.79 MB)
-
-### 3. Vector Search Tests
-
-**Query 1**: "configuration migration strategy"
-```
-1. [84.1%] config-unification-test.md ✅
-2. [79.8%] embeddings-and-fafcas-protocol-playbook.md
-3. [79.3%] report-graph-first-strategy.md
-```
-
-**Query 2**: "TypeScript compilation errors"
-```
-1. [76.0%] tooling-showcase.md
-2. [75.6%] problem-solving-playbook.md
-3. [74.8%] compare-src-and-resonance-folders.md
-```
-
-**Query 3**: "database path validation"
-```
-1. [77.3%] database-capabilities.md
-2. [76.6%] schema-playbook.md
-3. [76.4%] data-architecture.md
-```
-
-### 4. Test Document Verification
-
-**Query**: "config unification test document clean-slate migration"
-
-**Result**: ✅ **SUCCESS**
-- Best match: `config-unification-test.md`
-- Score: 83.5%
-- Status: TEST DOCUMENT INDEXED SUCCESSFULLY!
-
-## System Validation
-
-### Configuration Loading ✅
-- Config file: `amalfa.config.json`
-- Sources: `../polyvis/docs`, `../polyvis/playbooks`
-- Database: `.amalfa/multi-source-test.db`
-- Model: `BAAI/bge-small-en-v1.5`
-
-### Database Access ✅
-- Path loaded from config (not hardcoded)
-- WAL mode active
-- 95 nodes with embeddings
-- Full-text search disabled (hollow nodes)
-
-### Vector Search ✅
-- Model initialized correctly
-- Embeddings generated
-- Dot product similarity working
-- Results ranked by relevance
-
-### Semantic Accuracy ✅
-- Query 1: 84.1% match on exact test document
-- Query 2-3: 76-77% matches on related technical content
-- Clear differentiation between topics
-
-## Performance Metrics
-
-- **Database size**: 0.79 MB (95 nodes)
-- **Ingestion time**: ~0.26 seconds (96 files)
-- **Search latency**: <100ms per query
-- **Best match accuracy**: 83.5% for exact topic match
-
-## Configuration Benefits Observed
-
-1. **Single Source of Truth**: No ambiguity about config location
-2. **Lazy Loading**: Config loaded only when needed by each command
-3. **Predictable Paths**: All paths resolved from config
-4. **Clean Migration**: Zero legacy imports remain
-5. **Validation Tooling**: `validate-config` provides safety net
-
-## Issues Encountered
-
-None. System working as expected.
-
-## Related Tests
-
-- **CLI Test**: `amalfa stats` ✅
-- **Daemon Test**: File watcher started successfully ✅
-- **MCP Server**: Ready for Warp Preview testing ⏭️
-
-## Conclusion
-
-The unified configuration system migration is **complete and validated**. All components:
-- Load config from `amalfa.config.json`
-- Access database from config path
-- Perform vector search correctly
-- Index new documents successfully
-
-**Recommendation**: Ship to production. System stable and performant.
-
----
-
-## Test Reproduction
-
-To reproduce this test:
-
-```bash
-# 1. Create test document
-vim ../polyvis/docs/test-doc.md
-
-# 2. Ingest
-bun run src/cli.ts init
-
-# 3. Run validation
-bun run scripts/test-config-search.ts
-
-# Expected: All tests pass, document indexed, searches work
-```
-
-## Next Steps
-
-1. ✅ Config system validated
-2. ⏭️ Test MCP server in Warp Preview
-3. ⏭️ Test daemon file watching (auto-update on save)
-4. ⏭️ Production deployment readiness check

package/docs/CONFIG_UNIFICATION.md

@@ -1,187 +0,0 @@
-# Configuration Unification Plan
-
-**Date:** 2026-01-07  
-**Status:** Proposed
-
-## Current State
-
-### Two Active Configs
-
-1. **`amalfa.config.json`** (Primary)
-   - User-facing AMALFA configuration
-   - Clean, minimal, focused
-   - Controls: sources, database, embeddings, file watching
-   - Loaded by: `src/config/defaults.ts`
-
-2. **`polyvis.settings.json`** (Legacy)
-   - Resonance pipeline configuration
-   - Complex, with graph tuning and persona paths
-   - Controls: legacy database paths, graph algorithms, persona fixtures
-   - Loaded by: `src/resonance/config.ts`
-
-### Issue
-
-- Confusing for new users (two configs)
-- Potential for conflicting settings (both define database paths)
-- Unclear which config to edit
-
-## Decision: Keep Separate (For Now)
-
-**Rationale:**
-
-1. **Different audiences:**
-   - `amalfa.config.json` - End users configuring AMALFA
-   - `polyvis.settings.json` - Internal Resonance library settings
-
-2. **Different lifecycles:**
-   - AMALFA config is stable and user-friendly
-   - Resonance config will be deprecated as features migrate
-
-3. **Clean migration path:**
-   - Mark `polyvis.settings.json` as deprecated
-   - Gradually move needed settings to `amalfa.config.json`
-   - Remove when fully migrated
-
-## Recommendation: Document Ownership
-
-Create clear boundaries:
-
-### `amalfa.config.json` - AMALFA Core (User-Facing)
-
-```json
-{
-  "sources": ["./docs", "./playbooks"],
-  "database": ".amalfa/resonance.db",
-  "embeddings": {
-    "model": "BAAI/bge-small-en-v1.5",
-    "dimensions": 384
-  },
-  "watch": {
-    "enabled": true,
-    "debounce": 1000
-  },
-  "excludePatterns": ["node_modules", ".git", ".amalfa"],
-  
-  // Future additions (not yet implemented):
-  // "daemons": {
-  //   "vector": {
-  //     "enabled": true,
-  //     "port": 3010  // Only if really needed
-  //   },
-  //   "watcher": {
-  //     "enabled": true
-  //   }
-  // }
-}
-```
-
-**Controls:**
-- MCP server behavior
-- File watcher daemon
-- Vector daemon
-- Database location
-- Source directories
-- Embedding model selection
-
-### `polyvis.settings.json` - Resonance Library (Internal/Deprecated)
-
-**Current usage:**
-- Graph tuning parameters (Louvain algorithm)
-- Persona fixture paths (lexicon, CDA)
-- Legacy "experience" source structure
-
-**Migration path:**
-1. Mark file as deprecated in v1.0
-2. Add warning when loaded: "polyvis.settings.json is deprecated, migrate to amalfa.config.json"
-3. Remove in v2.0
-
-**What to migrate:**
-- Graph tuning → `amalfa.config.json` (if users need it)
-- Persona paths → Hard-coded defaults or removed (likely unused)
-- Experience sources → Already handled by `sources` in amalfa.config.json
-
-## Port Numbers Decision
-
-**NO** - Do not add port configuration to `amalfa.config.json`
-
-**Reasons:**
-1. Ports rarely change in single-user, local-first systems
-2. Adds complexity for minimal benefit
-3. Already support env var override: `VECTOR_PORT=3011 bun run start`
-
-**Current approach (keep):**
-```typescript
-// Code constant with env override
-const PORT = Number(process.env.VECTOR_PORT || 3010);
-```
-
-**When to add ports to config:**
-- Multi-user deployments
-- Cloud-native setups
-- Conflict resolution needs
-
-For AMALFA's use case: **Not needed.**
-
-## Action Plan
-
-### Phase 1: Document (Now)
-- [x] Create this document
-- [ ] Update `_current-config-status.md` with clear ownership
-- [ ] Add comments to `polyvis.settings.json` marking as deprecated
-- [ ] Update README with config documentation
-
-### Phase 2: Deprecation Warnings (v0.9)
-- [ ] Add warning log when `polyvis.settings.json` is loaded
-- [ ] Migrate graph tuning to `amalfa.config.json` (optional section)
-- [ ] Remove unused persona paths from codebase
-
-### Phase 3: Removal (v2.0)
-- [ ] Delete `polyvis.settings.json`
-- [ ] Remove `src/resonance/config.ts`
-- [ ] Single config: `amalfa.config.json`
-
-## Example: Unified Config (Future v2.0)
-
-```json
-{
-  "sources": ["./docs", "./playbooks"],
-  "database": ".amalfa/resonance.db",
-  
-  "embeddings": {
-    "model": "BAAI/bge-small-en-v1.5",
-    "dimensions": 384
-  },
-  
-  "watch": {
-    "enabled": true,
-    "debounce": 1000
-  },
-  
-  "excludePatterns": ["node_modules", ".git", ".amalfa"],
-  
-  "graph": {
-    "tuning": {
-      "louvain": {
-        "resolution": 0.3
-      }
-    }
-  }
-}
-```
-
-## Configuration Schema (Future)
-
-Consider adding schema validation:
-- `amalfa.config.schema.json` for IDE autocomplete
-- Validate on load with `zod` or similar
-- Clear error messages for invalid config
-
-## Summary
-
-**Decision:** Keep configs separate for now, document clearly, deprecate `polyvis.settings.json` gradually.
-
-**Port numbers:** NO - use env vars, keep config simple.
-
-**Timeline:**
-- v1.0: Document + deprecation warnings
-- v2.0: Single unified config