ai-database 2.0.2 → 2.1.1

package/TEST_SUMMARY.md

@@ -1,250 +0,0 @@
-# ai-database Test Suite Summary
-
-## Overview
-
-Comprehensive test suite for the ai-database package with **174 tests** across **5 test files**, all passing.
-
-## Test Files
-
-### 1. `src/schema.test.ts` (22 tests)
-**Purpose**: Schema parsing and bi-directional relationship validation
-
-**Coverage**:
-- Primitive field types (string, number, boolean, date, datetime, json, markdown, url)
-- Optional fields with `?` modifier
-- Array fields with `[]` and `[type]` syntax
-- Simple relations and bi-directional relationships
-- Automatic backref generation (e.g., `Post.author: 'Author.posts'` auto-creates `Author.posts`)
-- Complex multi-entity schemas
-- Self-referential relations
-- Edge cases (empty schemas, missing entities, etc.)
-- TypeScript type inference validation
-
-**Key Tests**:
-- ✅ Parses all primitive types correctly
-- ✅ Handles optional and array modifiers
-- ✅ Creates automatic backrefs for one-to-many relations
-- ✅ Creates automatic backrefs for many-to-many relations
-- ✅ Validates DB factory creates typed database
-
----
-
-### 2. `src/memory-provider.test.ts` (40 tests)
-**Purpose**: In-memory database provider implementation
-
-**Coverage**:
-- **CRUD Operations**: Create, get, update, delete
-- **Querying**: List with filtering, sorting, pagination
-- **Search**: Full-text search with scoring and field filtering
-- **Relationships**: Create, query, and remove relations between entities
-- **Utility Methods**: `clear()` and `stats()`
-- **Concurrency**: Multiple simultaneous operations
-- **Special Cases**: UUID generation, timestamp tracking, error handling
-
-**Key Tests**:
-- ✅ Creates entities with auto-generated or explicit IDs
-- ✅ Lists with where clauses, ordering, limits, and offsets
-- ✅ Searches with relevance scoring and min score filtering
-- ✅ Creates and queries many-to-many relationships
-- ✅ Cleans up relations when entities are deleted
-- ✅ Tracks entity and relation counts
-
----
-
-### 3. `src/index.test.ts` (31 tests)
-**Purpose**: Integration tests for the full DB API
-
-**Coverage**:
-- **DB Factory**: Creating typed database instances from schemas
-- **Entity Operations**: All CRUD methods on typed entities
-- **List & Query**: Filtering, sorting, pagination
-- **Search**: Entity-specific and global search
-- **Relationships**: Managing relations through both DB API and provider
-- **Global Methods**: `db.get()` by URL, `db.search()` across all types
-- **Type Safety**: Verifying TypeScript inference works correctly
-- **Complex Scenarios**: Multi-entity operations, self-referential relations
-
-**Key Tests**:
-- ✅ Creates, reads, updates, deletes entities via typed API
-- ✅ Upserts work for both create and update scenarios
-- ✅ Lists with various query options and pagination
-- ✅ Searches within entity type and globally
-- ✅ Iterates over entities with `forEach()`
-- ✅ Gets entities by URL (multiple formats supported)
-- ✅ Handles complex multi-entity schemas
-
----
-
-### 4. `test/provider-resolution.test.ts` (38 tests)
-**Purpose**: DATABASE_URL parsing and provider resolution
-
-**Coverage**:
-- **URL Format Detection**: All supported DATABASE_URL formats
-  - `:memory:` → In-memory provider
-  - `./content` → Filesystem provider
-  - `sqlite://./content` → Local SQLite provider
-  - `libsql://db.turso.io` → Remote Turso provider
-  - `chdb://./content` → Local ClickHouse (chDB)
-  - `clickhouse://host:8123` → Remote ClickHouse
-- **Provider Selection**: Environment variable handling
-- **URL Parsing**: Extracting provider type, paths, and remote URLs
-- **Provider Initialization**: Memory provider setup and isolation
-- **Interface Compliance**: Ensuring all providers implement required methods
-- **Documentation Examples**: Verifying all README examples work
-
-**Key Tests**:
-- ✅ Detects all provider types from URL format
-- ✅ Generates correct .db folder paths for local providers
-- ✅ Handles query parameters and database names in URLs
-- ✅ Memory provider implements all required methods
-- ✅ Provider instances are properly isolated
-- ✅ All README URL examples are valid
-
----
-
-### 5. `test/edge-cases.test.ts` (43 tests)
-**Purpose**: Edge cases, boundary conditions, and error scenarios
-
-**Coverage**:
-- **Empty/Minimal Schemas**: Handling edge case configurations
-- **Special Characters**: IDs and data with unicode, symbols, paths
-- **Large Data**: 100KB strings, 1000+ entities, bulk operations
-- **Concurrent Operations**: Parallel creates, updates, queries
-- **Optional Fields**: Missing, undefined, and null values
-- **Array Fields**: Empty arrays, array mutations
-- **URL Parsing**: Various URL formats, query params, hashes
-- **Search Edge Cases**: Empty queries, special regex chars, extreme scores
-- **Pagination**: Out-of-bounds offsets, zero/negative limits
-- **Type Coercion**: Handling different value types
-
-**Key Tests**:
-- ✅ Handles empty schemas and entities with no fields
-- ✅ Supports IDs with hyphens, underscores, dots, slashes, UUIDs
-- ✅ Stores unicode characters and emojis correctly
-- ✅ Handles 100KB strings and 1000+ entities
-- ✅ Concurrent operations don't interfere with each other
-- ✅ Optional fields can be set, unset, and omitted
-- ✅ Parses full HTTPS URLs, HTTP URLs, and type/id paths
-- ✅ Search handles empty strings and special regex characters
-- ✅ Pagination handles out-of-bounds offsets gracefully
-
----
-
-## Test Execution
-
-```bash
-# Run all tests
-pnpm test
-
-# Run tests once (non-watch mode)
-pnpm test run
-
-# Run specific test file
-pnpm test src/schema.test.ts
-
-# Build package (verifies no TypeScript errors)
-pnpm build
-```
-
-## Test Results
-
-```
-✓ src/index.test.ts (31 tests) 7ms
-✓ test/edge-cases.test.ts (43 tests) 6ms
-✓ src/memory-provider.test.ts (40 tests) 14ms
-✓ src/schema.test.ts (22 tests) 2ms
-✓ test/provider-resolution.test.ts (38 tests) 2ms
-
-Test Files  5 passed (5)
-Tests       174 passed (174)
-Duration    215ms
-```
-
-## Coverage Areas
-
-| Area | Test File | Tests | Status |
-|------|-----------|-------|--------|
-| Schema Parsing | `src/schema.test.ts` | 22 | ✅ 100% |
-| Memory Provider | `src/memory-provider.test.ts` | 40 | ✅ 100% |
-| Integration | `src/index.test.ts` | 31 | ✅ 100% |
-| Provider Resolution | `test/provider-resolution.test.ts` | 38 | ✅ 100% |
-| Edge Cases | `test/edge-cases.test.ts` | 43 | ✅ 100% |
-| **TOTAL** | | **174** | **✅ 100%** |
-
-## Key Features Tested
-
-### ✅ Schema-First Design
-- Declarative schema definitions
-- Automatic bi-directional relationships
-- TypeScript type inference from schema
-- Support for all primitive types
-
-### ✅ CRUD Operations
-- Create with auto-generated or explicit IDs
-- Get by ID with null for not-found
-- Update with partial data merging
-- Upsert for create-or-update semantics
-- Delete with cascade cleanup of relations
-
-### ✅ Querying & Search
-- List with where clauses
-- Sorting (ascending/descending)
-- Pagination (limit/offset)
-- Full-text search with scoring
-- Field-specific search
-- Global search across all entity types
-
-### ✅ Relationships
-- One-to-many with automatic backrefs
-- Many-to-many with array syntax
-- Self-referential relations
-- Lazy-loaded relation traversal
-- Relation cleanup on entity deletion
-
-### ✅ Provider System
-- In-memory provider (fully implemented)
-- Pluggable provider interface
-- DATABASE_URL parsing for all provider types
-- Graceful fallback to memory provider
-- Provider isolation for testing
-
-### ✅ Type Safety
-- Full TypeScript inference
-- Typed entity operations
-- Type-safe query results
-- Compile-time schema validation
-
-## Implementation Notes
-
-1. **In-Memory Provider**: Fully functional with all operations implemented
-2. **Dynamic Imports**: Other providers (`@mdxdb/fs`, `@mdxdb/sqlite`, `@mdxdb/clickhouse`) are dynamically imported with graceful fallback
-3. **Test Isolation**: Each test suite uses fresh provider instances via `beforeEach`
-4. **TypeScript**: All code compiles without errors, strict mode enabled
-5. **Vitest**: Using vitest as test runner with fork pool for isolation
-
-## Future Test Additions
-
-Potential areas for expansion once additional providers are implemented:
-
-1. **Filesystem Provider Tests**: When `@mdxdb/fs` is implemented
-2. **SQLite Provider Tests**: When `@mdxdb/sqlite` is implemented
-3. **ClickHouse Provider Tests**: When `@mdxdb/clickhouse` is implemented
-4. **Cross-Provider Tests**: Verifying behavior consistency
-5. **Performance Benchmarks**: Large dataset tests with timing
-6. **Migration Tests**: Schema evolution and data migration
-7. **Transaction Tests**: If/when transaction support is added
-8. **Validation Tests**: Runtime schema validation
-9. **Index Tests**: Query optimization with indexes
-
-## Conclusion
-
-The ai-database package has **comprehensive test coverage** with **174 passing tests** covering:
-- ✅ All core functionality
-- ✅ Schema parsing and validation
-- ✅ Complete CRUD operations
-- ✅ Advanced querying and search
-- ✅ Relationship management
-- ✅ Provider resolution and initialization
-- ✅ Edge cases and error scenarios
-
-The test suite provides a **solid foundation** for the package and ensures reliability as new providers and features are added.

package/TODO.md

@@ -1,128 +0,0 @@
-# ai-database TODO
-
-## Core Features
-
-### Natural Language Queries
-- [x] Define `NLQueryResult`, `NLQueryFn`, `NLQueryContext`, `NLQueryPlan` types
-- [x] Implement `setNLQueryGenerator()` for custom AI integration
-- [x] Implement `buildNLQueryContext()` to extract schema metadata
-- [x] Implement `executeNLQuery()` with fallback to search
-- [x] Add tagged template support for `db\`query\`` syntax
-- [ ] Wire up `db.ask` and type-specific NL queries in DB factory
-- [ ] Add streaming support for NL query results
-- [ ] Add query caching/memoization
-- [ ] Add query history/suggestions
-
-### Self-Describing Schema
-- [x] Define `ThingSchema`, `NounSchema`, `VerbSchema`, `EdgeSchema`
-- [x] Define `SystemSchema` combining all system types
-- [ ] Auto-populate Noun records when DB() is called
-- [ ] Auto-populate Edge records when relationships are defined
-- [ ] Auto-populate Verb records for standard actions
-- [ ] Add `db.Noun`, `db.Verb`, `db.Edge` accessors
-- [ ] Implement `Thing.type` -> `Noun` relationship
-- [ ] Implement `Noun.things` -> all instances backref
-
-### Noun & Verb Types
-- [x] Define `Noun`, `NounProperty`, `NounRelationship` interfaces
-- [x] Define `Verb` interface with all conjugation forms
-- [x] Implement `defineNoun()` and `defineVerb()` helpers
-- [x] Implement `nounToSchema()` converter
-- [x] Define standard `Verbs` constant (create, update, delete, publish, archive)
-
-### AI-Powered Linguistic Inference
-- [x] Basic `conjugate()`, `pluralize()`, `singularize()` with common rules
-- [x] Basic `inferNoun()` from type name
-- [ ] **Use AI for unknown nouns/verbs** - if not in Things table, ask AI once and cache
-- [ ] Remove brittle rule-based code in favor of AI inference
-- [ ] Store AI-generated forms in Noun/Verb records for future lookups
-- [ ] Add confidence scores to AI-generated linguistic forms
-
-### TypeMeta
-- [x] Define `TypeMeta` interface
-- [x] Implement `Type()` accessor function
-- [x] Include slug/slugPlural for URL generation
-- [x] Include event type names (created, updated, deleted)
-- [x] Include verb-derived fields (createdAt, createdBy, etc.)
-
-## Provider Implementation
-
-### MemoryProvider
-- [x] Implement `Semaphore` for concurrency control
-- [x] Implement Event storage and emission
-- [x] Implement Action lifecycle management
-- [x] Implement Artifact storage with invalidation
-- [x] Add pattern matching for event subscriptions
-- [ ] Add persistence option (save/load to JSON)
-- [ ] Add TTL support for artifacts
-
-> **Note**: Production providers (SQLite, ClickHouse, Postgres, etc.) are implemented in `@mdxdb/*` packages.
-
-## Query Capabilities
-
-### Filtering
-- [x] Document SQL-style operators ($gt, $lt, $in, etc.)
-- [x] Document Document-style nested queries
-- [x] Document Graph-style relationship traversal
-- [ ] Implement all documented operators in MemoryProvider
-- [ ] Add query validation
-- [ ] Add query optimization hints
-
-### Search
-- [x] Define SearchOptions interface
-- [ ] Implement hybrid search (vector + BM25)
-- [ ] Add embedding generation
-- [ ] Add chunking for long content
-- [ ] Add re-ranking
-
-## Testing
-
-### Unit Tests
-- [x] Schema parsing tests
-- [x] Noun/Verb type tests
-- [x] Basic conjugate/pluralize/singularize tests
-- [x] TypeMeta tests
-- [x] Semaphore tests
-- [x] Event emission tests
-- [x] Action lifecycle tests
-- [x] Artifact storage tests
-- [x] System schema tests
-- [x] Edge creation tests
-- [ ] NL query execution tests
-- [ ] AI linguistic inference tests
-
-### Integration Tests
-- [ ] Full CRUD workflow
-- [ ] Relationship traversal
-- [ ] Event-driven workflows
-- [ ] Search accuracy benchmarks
-
-## Documentation
-
-- [x] README: Core primitives
-- [x] README: Actions for durable execution
-- [x] README: Events for reactivity
-- [x] README: Artifacts for caching
-- [x] README: Query styles (SQL, Document, Graph)
-- [x] README: Natural language queries
-- [x] README: Self-describing schema
-- [x] README: Noun & Verb types
-- [x] README: AI auto-generation
-- [ ] API reference with examples
-- [ ] Migration guide
-- [ ] Performance tuning guide
-
-## Future Ideas
-
-- [ ] Schema migrations
-- [ ] Real-time subscriptions (WebSocket)
-- [ ] Offline-first with sync
-- [ ] Multi-tenancy support
-- [ ] Row-level security
-- [ ] Audit logging
-- [ ] Data lineage tracking
-- [ ] Schema versioning
-- [ ] Import/export (JSON, CSV, Parquet)
-- [ ] GraphQL API generation
-- [ ] REST API generation
-- [ ] Admin UI