@joystick.js/db-canary 0.0.0-canary.2209

package/prompts/FULL_TEXT_SEARCH.md

@@ -0,0 +1,293 @@
+# Full Text Search Implementation Prompt
+
+## Overview
+
+Implement full text search capabilities for the JoystickDB LMDB-based database server. This feature should provide fast, scalable text search across document fields while maintaining the existing clustered architecture and performance characteristics.
+
+## Requirements
+
+### Core Functionality
+- Support full text search across specified document fields
+- Implement inverted index strategy for optimal search performance
+- Provide boolean query operators (AND, OR, NOT)
+- Support phrase matching with quotes
+- Include fuzzy matching capabilities for typos/variations
+- Case-insensitive search by default with case-sensitive option
+
+### Text Processing
+- Tokenization: Split text into searchable terms
+- Normalization: Convert to lowercase, handle Unicode
+- Stop word filtering: Remove common words (the, and, or, etc.)
+- Stemming: Reduce words to root forms (running → run)
+- Support for multiple languages (start with English)
+
+### Index Management
+- Automatic index creation for designated searchable fields
+- Atomic index updates during document modifications
+- Index rebuilding capabilities for schema changes
+- Configurable index settings per field
+
+### Query Interface
+- Add new operation type: `text_search`
+- Query syntax: `{ "query": "search terms", "fields": ["title", "content"], "options": {...} }`
+- Support for field-specific searches: `title:"exact phrase" content:keyword`
+- Result ranking based on relevance scores
+- Pagination support for large result sets
+
+## Technical Implementation
+
+### LMDB Storage Strategy
+```
+Document Storage:
+- Key: `doc:{collection}:{id}`
+- Value: Original document data
+
+Inverted Index Storage:
+- Key: `idx:{collection}:{field}:{term}`
+- Value: Array of document IDs with positions
+
+Term Frequency Storage:
+- Key: `tf:{collection}:{field}:{doc_id}`
+- Value: Term frequency map for relevance scoring
+
+Metadata Storage:
+- Key: `meta:{collection}:search_config`
+- Value: Search configuration and field mappings
+```
+
+### Architecture Integration
+
+#### Master Process Responsibilities
+- Handle index updates during write operations
+- Coordinate index rebuilding across workers
+- Manage search configuration changes
+- Ensure index consistency during bulk operations
+
+#### Worker Process Responsibilities
+- Execute search queries against local LMDB instances
+- Perform text processing and tokenization
+- Calculate relevance scores
+- Handle search result pagination
+
+### New Operation Types
+Add to `op_types.js`:
+```javascript
+"text_search",
+"create_text_index",
+"drop_text_index",
+"rebuild_text_index"
+```
+
+### Search Query Structure
+```javascript
+{
+  "op": "text_search",
+  "data": {
+    "collection": "documents",
+    "query": "machine learning algorithms",
+    "fields": ["title", "content", "tags"],
+    "options": {
+      "fuzzy": true,
+      "case_sensitive": false,
+      "max_results": 50,
+      "offset": 0,
+      "highlight": true,
+      "min_score": 0.1
+    }
+  }
+}
+```
+
+### Response Format
+```javascript
+{
+  "success": true,
+  "data": {
+    "results": [
+      {
+        "document": { /* original document */ },
+        "score": 0.85,
+        "highlights": {
+          "title": ["<mark>machine learning</mark> basics"],
+          "content": ["advanced <mark>algorithms</mark> for..."]
+        }
+      }
+    ],
+    "total_count": 127,
+    "search_time_ms": 23,
+    "query_info": {
+      "parsed_terms": ["machine", "learning", "algorithms"],
+      "fields_searched": ["title", "content", "tags"]
+    }
+  }
+}
+```
+
+## Implementation Steps
+
+### Phase 1: Core Infrastructure
+1. Create text processing utilities (tokenization, normalization, stemming)
+2. Implement inverted index data structures
+3. Add LMDB storage patterns for search indexes
+4. Create search configuration management
+
+### Phase 2: Basic Search
+1. Implement simple keyword search
+2. Add boolean operators (AND, OR, NOT)
+3. Create relevance scoring algorithm
+4. Add result ranking and pagination
+
+### Phase 3: Advanced Features
+1. Implement phrase matching
+2. Add fuzzy search capabilities
+3. Create result highlighting
+4. Add field-specific search syntax
+
+### Phase 4: Integration
+1. Integrate with existing cluster architecture
+2. Add search operations to worker/master communication
+3. Implement atomic index updates
+4. Add search-specific error handling
+
+### Phase 5: Optimization
+1. Performance tuning for large datasets
+2. Memory usage optimization
+3. Index compression techniques
+4. Query optimization strategies
+
+## File Structure
+
+```
+src/server/search/
+├── index.js                 # Main search module exports
+├── text_processor.js        # Tokenization, stemming, normalization
+├── inverted_index.js        # Index creation and management
+├── query_parser.js          # Parse search query syntax
+├── relevance_scorer.js      # Calculate document relevance scores
+├── search_executor.js       # Execute search queries
+├── index_manager.js         # Manage index lifecycle
+└── search_config.js         # Search configuration utilities
+
+src/server/lib/
+├── search_operations.js     # Search-specific database operations
+└── search_utils.js          # Shared search utilities
+
+test/server/search/
+├── text_processor.test.js
+├── inverted_index.test.js
+├── query_parser.test.js
+├── relevance_scorer.test.js
+└── search_integration.test.js
+```
+
+## Configuration Options
+
+### Database Settings
+```javascript
+{
+  "search": {
+    "default_language": "en",
+    "stop_words_enabled": true,
+    "stemming_enabled": true,
+    "fuzzy_threshold": 0.8,
+    "max_index_memory_mb": 512,
+    "index_update_batch_size": 1000,
+    "search_timeout_ms": 5000
+  }
+}
+```
+
+### Field-Level Configuration
+```javascript
+{
+  "collections": {
+    "documents": {
+      "search_fields": {
+        "title": {
+          "weight": 2.0,
+          "stemming": true,
+          "fuzzy": true
+        },
+        "content": {
+          "weight": 1.0,
+          "stemming": true,
+          "fuzzy": false
+        },
+        "tags": {
+          "weight": 1.5,
+          "stemming": false,
+          "fuzzy": true
+        }
+      }
+    }
+  }
+}
+```
+
+## Performance Considerations
+
+### Memory Management
+- Use LMDB's memory-mapped files for large indexes
+- Implement index compression for frequently accessed terms
+- Cache frequently used search results
+- Limit concurrent search operations per worker
+
+### Scalability
+- Distribute index updates across multiple workers
+- Implement index sharding for very large collections
+- Use background processes for index maintenance
+- Monitor index size and performance metrics
+
+### Query Optimization
+- Implement query plan optimization
+- Use term frequency for query reordering
+- Cache parsed queries for repeated searches
+- Implement search result caching
+
+## Testing Strategy
+
+### Unit Tests
+- Text processing functions
+- Index creation and updates
+- Query parsing logic
+- Relevance scoring algorithms
+
+### Integration Tests
+- End-to-end search operations
+- Index consistency during concurrent writes
+- Search performance under load
+- Error handling and recovery
+
+### Performance Tests
+- Search latency benchmarks
+- Index update performance
+- Memory usage patterns
+- Concurrent search handling
+
+## Success Criteria
+
+1. **Functionality**: All search operations work correctly with accurate results
+2. **Performance**: Search queries complete within 100ms for typical datasets
+3. **Scalability**: System handles 1000+ concurrent searches without degradation
+4. **Reliability**: Index consistency maintained during all write operations
+5. **Usability**: Intuitive query syntax with comprehensive error messages
+
+## Dependencies
+
+### New Package Dependencies
+```javascript
+{
+  "natural": "^6.0.0",        // Text processing and stemming
+  "stopword": "^2.0.0",       // Stop word removal
+  "fuse.js": "^7.0.0"         // Fuzzy search capabilities
+}
+```
+
+### Development Dependencies
+```javascript
+{
+  "benchmark": "^2.1.4"       // Performance testing
+}
+```
+
+This implementation will provide a robust, scalable full text search solution that integrates seamlessly with the existing JoystickDB architecture while maintaining high performance and reliability.

package/prompts/PROMPTS.md

@@ -0,0 +1,158 @@
+# Prompt 1: Build JoystickDB TCP Server (Production-Grade, Secure, Reliable)
+
+You are building a Node.js app that acts as the **JoystickDB TCP server**.
+
+Requirements:
+- Use ESM syntax (`import`).
+- Follow coding conventions: snake_case for variables/functions/keys, arrow functions preferred.
+- Install dependencies: `lmdb`, `msgpack-lite`, `bcrypt`, `@aws-sdk/client-s3`, `crypto`, plus standard Node libs.
+- The server should:
+
+### Core
+1. **Auto-calculate map_size at bootstrap**:  
+   - Detect total disk size (e.g. via `df -k .`).  
+   - Set `map_size = disk_size * 0.8`.  
+   - Example: 3 TB disk → map_size = ~2.4 TB.  
+   - This ensures JoystickDB never consumes 100% of disk, leaving margin for OS, swap, logs, and backups.  
+   - Map_size must be **grown automatically** if data approaches 80% of current map_size (double it, capped at 80% of total disk).  
+2. Open an LMDB environment at `./data` with that map_size.  
+3. Start a TCP server on port `6000` (Node `net` module).  
+4. Each message = 4-byte length prefix + MessagePack payload.  
+5. Decode MessagePack payload into JSON-like object:  
+   ```json
+   {
+     "op": "auth" | "find_one" | "find" | "insert_one" | "update_one" | "delete_one" | "bulk_write" | "admin" | "ping",
+     "collection": "string",
+     "filter": { ... },
+     "update": { ... },
+     "options": { ... },
+     "operations": [ ... ],
+     "admin_action": "stats" | "list_collections" | "create_index" | "drop_index" | "get_indexes" | "list_documents" | "get_document" | "query_documents" | "insert_document" | "update_document" | "delete_document" | "backup_now" | "restore_backup" | "list_backups",
+     "password": "string"
+   }
+   ```
+
+### Query Language
+6. Implement a **MongoDB-style query language** (1:1 with MongoDB except no aggregation pipeline).  
+7. **Secondary indexing**:  
+   - Maintain sub-databases for indexed fields.  
+   - Keys: `index:<collection>:<field>:<value> → [document_ids...]`.  
+   - Update indexes on insert/update.  
+   - Use indexes for queries when filters match indexed fields.  
+
+### Writes
+8. **Bulk writes**:  
+   - Accept array of insert/update/delete ops.  
+   - Run in a single LMDB write transaction (atomic).  
+   - Roll back entire batch if any fail.  
+9. **Write serialization**:  
+   - Only one LMDB write transaction at a time.  
+   - Queue concurrent writes.  
+
+### Authentication
+10. **Auth flow**:  
+    - Bootstrap: provision 32-char password via SSH.  
+    - Store `bcrypt.hash(password)` in `./auth.json`.  
+    - First op must be `{ op: "auth", password: "..." }`.  
+    - Rate-limit failures (5/min per IP).  
+
+### Admin Operations
+11. **Admin actions**:  
+    - Metadata: `stats`, `list_collections`, `create_index`, `drop_index`, `get_indexes`.  
+    - Data: `list_documents`, `get_document`, `query_documents`, `insert_document`, `update_document`, `delete_document`.  
+    - Backups: `backup_now`, `list_backups`, `restore_backup`.  
+
+### Backups & Restore
+12. **Automatic backups**:  
+    - If S3 configured in `settings.json`, run `backup_now` hourly.  
+    - Store compressed sparse backup (`tar --sparse`) in `./backups` and S3.  
+    - Retention: keep 24 hourly + 30 daily, rotate older.  
+    - Backup includes SHA256 checksum file.  
+13. **restore_from one-shot**:  
+    - If `settings.json` has `"restore_from": "<backup_file>"`:  
+      - Download from S3 (or local).  
+      - Verify SHA256 checksum.  
+      - Replace `./data/data.mdb`.  
+      - If restore fails: log + boot fresh LMDB.  
+      - If restore succeeds: rewrite `settings.json` without `restore_from`.  
+
+### Safety
+14. **Health**:  
+    - `ping` always returns `{ ok: 1 }`.  
+    - `stats` includes: memory, disk, ops/sec, doc count, map_size usage %.  
+15. **Timeouts**:  
+    - Max 5s per request.  
+16. **Connections**:  
+    - Max ~1000 connections.  
+    - Drop idle after 10m.  
+17. **Logging**:  
+    - Structured logs: `{ ts, client_id, op, collection, duration_ms, status }`.  
+18. **Protocol versioning**:  
+    - After auth: `{ ok: 1, version: "1.0.0" }`.  
+    - API never changes; future changes must remain backward-compatible.  
+
+---
+
+# Prompt 2: Build JoystickDB TCP Client
+
+- ESM + snake_case.  
+- Install `msgpack-lite`.  
+- Export `create_client({ host, port, password })`.  
+- On connect: send `{ op: "auth", password: "..." }`.  
+- Methods:  
+  - CRUD: `insert_one`, `find_one`, `find`, `update_one`, `delete_one`.  
+  - Bulk: `bulk_write`.  
+  - Admin Metadata: `get_stats`, `list_collections`, `create_index`, `drop_index`, `get_indexes`.  
+  - Admin Data: `list_documents`, `get_document`, `query_documents`, `insert_document`, `update_document`, `delete_document`.  
+  - Backups: `backup_now`, `list_backups`, `restore_backup`.  
+  - Health: `ping`.  
+- Per-request timeout: 5s.  
+- Auto reconnect + queueing.  
+
+---
+
+# Prompt 3: NGINX TLS Termination
+
+```nginx
+stream {
+  upstream joystickdb_backend {
+    server 127.0.0.1:6000;
+  }
+
+  server {
+    listen 443 ssl;
+    proxy_pass joystickdb_backend;
+
+    ssl_certificate     /etc/letsencrypt/live/db.cheatcode.co/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/db.cheatcode.co/privkey.pem;
+
+    ssl_protocols TLSv1.3;
+    ssl_ciphers HIGH:!aNULL:!MD5;
+    ssl_prefer_server_ciphers on;
+    ssl_session_cache shared:SSL:10m;
+    ssl_session_timeout 10m;
+  }
+}
+```
+
+---
+
+# Prompt 4: settings.json Schema
+
+```json
+{
+  "s3": {
+    "bucket": "joystickdb-backups",
+    "region": "us-east-1",
+    "access_key": "AKIA...",
+    "secret_key": "abcd...",
+    "endpoint": "https://s3.amazonaws.com"
+  },
+  "backup_schedule": "hourly",
+  "restore_from": "joystickdb-backup-2025-08-28T12:00:00.mdb"
+}
+```
+
+- `s3`: optional.  
+- `backup_schedule`: default hourly if omitted.  
+- `restore_from`: one-shot restore, auto-cleared after success.  

package/prompts/README.md

@@ -0,0 +1,221 @@
+# JoystickDB Implementation Tasks
+
+This directory contains detailed prompts for implementing the complete production-grade JoystickDB TCP server. Each task builds upon the previous ones and should be completed in order.
+
+## Current Status
+✅ **COMPLETED**: Basic query engine with CRUD operations, authentication, and admin/reload operations (119 tests passing)
+✅ **COMPLETED**: Task 1 - Core Infrastructure Upgrades (153 tests passing)
+✅ **COMPLETED**: Task 2 - Secondary Indexing System (196 tests passing)
+✅ **COMPLETED**: Task 3 - Write Serialization and Queuing (214 tests passing)
+✅ **COMPLETED**: Task 4 - Enhanced Authentication System (251 tests passing)
+✅ **COMPLETED**: Task 5 - Comprehensive Admin Operations (266 tests passing)
+✅ **COMPLETED**: Task 6 - Backup and Restore System (296 tests passing)
+✅ **COMPLETED**: Task 7 - Production Safety Features (316 tests passing)
+✅ **COMPLETED**: Task 8 - TCP Client Library (All original tasks complete)
+
+## Next Phase: API and Infrastructure Enhancements
+✅ **COMPLETED**: Task 9 - API Method Chaining Enhancement (347 tests passing)
+✅ **COMPLETED**: Task 10 - Automatic Index Creation (381 tests passing)
+✅ **COMPLETED**: Task 11 - Operation Naming Consistency (383 tests passing)
+✅ **COMPLETED**: Task 12 - TCP Replication System (423 tests passing)
+
+## Phase 3: Core Improvements and User Experience
+✅ **COMPLETED**: Task 13 - Master Node Read and Write Operations (420 tests passing)
+✅ **COMPLETED**: Task 14 - Index Upsert Operations (428 tests passing)
+✅ **COMPLETED**: Task 15 - Client API Return Types Enhancement (428 tests passing)
+✅ **COMPLETED**: Task 16 - Server Setup UI (451 tests passing - Test Suite Stabilized)
+✅ **COMPLETED**: Task 17 - Emergency Password Change System (474 tests passing)
+
+## All Core Tasks Complete
+🎉 **ALL TASKS COMPLETED**: JoystickDB production-grade implementation is complete with comprehensive emergency password recovery system.
+
+## Task Overview
+
+### 1. Core Infrastructure Upgrades
+**File**: `01-core-infrastructure.md`
+**Priority**: High
+**Dependencies**: None
+**Description**: Implement auto-calculating LMDB map_size, proper TCP protocol with length prefixing, connection limits, timeouts, and enhanced logging.
+
+### 2. Secondary Indexing System
+**File**: `02-secondary-indexing.md`
+**Priority**: High
+**Dependencies**: Task 1
+**Description**: Build comprehensive indexing system for query optimization with index management operations and automatic index maintenance.
+
+### 3. Write Serialization and Queuing
+**File**: `03-write-serialization.md`
+**Priority**: High
+**Dependencies**: Task 1
+**Description**: Implement write queue system to ensure only one LMDB write transaction at a time with proper concurrent read support.
+
+### 4. Enhanced Authentication System
+**File**: `04-enhanced-authentication.md`
+**Priority**: Medium
+**Dependencies**: Task 1
+**Description**: Add setup command for password generation, rate limiting, protocol versioning, and secure authentication.json file management.
+
+### 5. Comprehensive Admin Operations
+**File**: `05-comprehensive-admin-operations.md`
+**Priority**: Medium
+**Dependencies**: Tasks 1, 2
+**Description**: Expand admin operations with metadata management, data operations, and system monitoring capabilities.
+
+### 6. Backup and Restore System
+**File**: `06-backup-and-restore-system.md`
+**Priority**: Medium
+**Dependencies**: Tasks 1, 5
+**Description**: Implement S3-integrated backup system with automatic scheduling, retention management, and one-shot restore capabilities.
+
+### 7. Production Safety Features
+**File**: `07-production-safety-features.md`
+**Priority**: High
+**Dependencies**: Tasks 1, 3
+**Description**: Add comprehensive health monitoring, request timeouts, connection management, and enhanced structured logging.
+
+### 8. TCP Client Library
+**File**: `08-tcp-client-library.md`
+**Priority**: Medium
+**Dependencies**: Tasks 1, 4, 5
+**Description**: Create Node.js client library with auto-reconnection, full operation support, and comprehensive error handling.
+
+### 9. API Method Chaining Enhancement
+**File**: `09-api-method-chaining.md`
+**Priority**: High
+**Dependencies**: Task 8
+**Description**: Update the client API to chain methods off of `collection()` instead of passing collection names as the first argument, providing a more intuitive and fluent API interface.
+
+### 10. Automatic Index Creation
+**File**: `10-automatic-index-creation.md`
+**Priority**: Medium
+**Dependencies**: Task 2
+**Description**: Implement automatic index creation based on query patterns to optimize database performance without requiring manual index management.
+
+### 11. Operation Naming Consistency
+**File**: `11-operation-naming-consistency.md`
+**Priority**: Medium
+**Dependencies**: Task 9
+**Description**: Standardize the operation object structure to use snake_case naming consistently across the entire codebase and API.
+
+### 12. TCP Replication System
+**File**: `12-tcp-replication-system.md`
+**Priority**: High
+**Dependencies**: Tasks 1, 8
+**Description**: Implement TCP-based replication to read-only secondary nodes with write forwarding, providing high availability and read scaling capabilities.
+
+### 13. Master Node Read and Write Operations
+**File**: `13-master-read-write-operations.md`
+**Priority**: Medium
+**Dependencies**: Task 12
+**Description**: Ensure that master nodes in the cluster can handle both read and write operations, not just writes, improving load distribution and eliminating artificial restrictions.
+
+### 14. Index Upsert Operations
+**File**: `14-index-upsert-operations.md`
+**Priority**: Medium
+**Dependencies**: Task 2
+**Description**: Implement index upsert functionality where indexes are created if they don't exist or updated if they do exist, providing seamless "create or update" operations for index management.
+
+### 15. Client API Return Types Enhancement
+**File**: `15-client-api-return-types.md`
+**Priority**: High
+**Dependencies**: Task 9
+**Description**: Update the client API so that `find()` returns an array of matching documents and `find_one()` returns a single document (or null), providing more intuitive return types for developers.
+
+### 16. Server Setup UI
+**File**: `16-server-setup-ui.md`
+**Priority**: Medium
+**Dependencies**: Task 4
+**Description**: Implement a minimalist web UI hosted by the server that provides a secure setup interface using randomly generated tokens, eliminating the need to connect a separate client for initial setup.
+
+### 17. Emergency Password Change System
+**File**: `17-emergency-password-change.md`
+**Priority**: High
+**Dependencies**: Tasks 4, 16
+**Description**: Implement a secure emergency password change system using CLI-generated recovery tokens, allowing administrators to quickly change passwords in case of security breaches with minimal downtime.
+
+## Implementation Guidelines
+
+### Code Standards
+- Use ESM syntax (`import`/`export`)
+- Follow snake_case for variables, functions, and keys
+- Prefer arrow functions
+- Two-space indentation
+- Minimal comments (only when necessary for clarity)
+
+### Testing Requirements
+- All existing tests must continue to pass (currently 119 tests)
+- Add comprehensive test suites for each new feature
+- Use AVA test runner with `--serial` flag
+- Include both unit and integration tests
+
+### Dependencies
+- Keep existing: `lmdb`, `msgpackr`, `bcrypt`
+- Add as needed: `@aws-sdk/client-s3`, `crypto`
+- Maintain compatibility with existing codebase
+
+## Getting Started
+
+1. Choose a task based on priority and dependencies
+2. Read the detailed prompt in the corresponding `.md` file
+3. Create a new branch for the task (see Git Workflow below)
+4. Implement the features following the specifications
+5. Ensure all existing tests continue to pass
+6. Add comprehensive tests for new functionality
+7. Debug and fix any failing tests (no shortcuts or hacks allowed)
+8. Merge branch to main only when all tests pass
+9. Move to the next task
+
+## Git Workflow
+
+Each task must be implemented in a separate branch to maintain code quality and enable proper review:
+
+### Branch Management
+- **Branch Naming**: Use format `task-N-description` (e.g., `task-1-core-infrastructure`)
+- **Create Branch**: `git checkout -b task-N-description` from main
+- **Work Isolation**: All development for a task happens in its dedicated branch
+- **No Partial Merges**: Only merge when task is 100% complete with all tests passing
+
+### Development Process
+1. **Start Task**: Create new branch from main
+2. **Implement Features**: Follow task specifications exactly
+3. **Write Tests**: Comprehensive test coverage for all new functionality
+4. **Debug Thoroughly**: Fix all failing tests without shortcuts or hacks
+5. **Verify Quality**: Run full test suite (`npm test`) - all 119+ tests must pass
+6. **Merge to Main**: Only when everything works perfectly
+
+### Quality Gates
+- ✅ All existing tests continue to pass (119 baseline tests)
+- ✅ New comprehensive tests for implemented features
+- ✅ No failing tests (debug and fix, don't skip or hack)
+- ✅ Code follows established patterns and standards
+- ✅ No temporary workarounds or incomplete implementations
+
+### Merge Requirements
+```bash
+# Before merging, verify:
+npm test                    # All tests pass
+git status                  # Clean working directory
+git checkout main           # Switch to main
+git merge task-N-description # Merge completed task
+git branch -d task-N-description # Clean up branch
+```
+
+## Cline Chat Management
+
+This project is being developed using Cline, an AI coding assistant. To manage context window limits:
+
+- **Context Threshold**: When context usage exceeds 50K tokens during development
+- **Task Completion Rule**: Always complete the current task before starting a new chat
+- **New Chat Process**: Once a task is complete and context > 50K tokens:
+  1. Update this README.md with task completion status
+  2. Generate a detailed prompt for the next task
+  3. Use Cline's "new task" feature to start fresh with the next task prompt
+  4. Include full context about completed work and current project state
+
+## Notes
+
+- Tasks 1, 3, and 7 are critical for production readiness
+- Task 2 (indexing) significantly improves query performance
+- Tasks 4, 5, 6, and 8 add important functionality but are less critical
+- Each task is designed to be completed independently while building on previous work
+- Context management ensures efficient development without losing important project context