sqlew 3.2.5 → 3.6.0

package/docs/AUTO_FILE_TRACKING.md

@@ -2,6 +2,52 @@
 
 **Zero-Token Task Management** - Automatic task status transitions based on file changes and acceptance criteria validation.
 
+## Quick Start
+
+### Basic Usage (v3.4.1+)
+
+Create a task with automatic file watching in one step:
+
+```typescript
+// Create task with files to watch
+task action=create
+  title: "Implement user authentication"
+  acceptance_criteria: [
+    {type: "tests_pass", command: "npm test auth", expected_pattern: "passing"},
+    {type: "code_contains", file: "src/auth.ts", pattern: "export class AuthService"}
+  ]
+  watch_files: ["src/auth.ts", "src/auth.test.ts"]
+  assigned_agent: "backend-dev"
+  status: "todo"
+```
+
+That's it! The file watcher automatically:
+1. Monitors `src/auth.ts` and `src/auth.test.ts`
+2. Moves task to `in_progress` when files change
+3. Runs acceptance checks when files stabilize
+4. Moves task to `done` if all checks pass
+
+**Token Savings**: 97% reduction (4,650 tokens saved per 6-task session)
+
+### Alternative: Add Files to Existing Task
+
+```typescript
+// Add files to watch using the watch_files action
+task action=watch_files
+  task_id: 123
+  action: watch
+  file_paths: ["src/auth.ts", "src/auth.test.ts"]
+```
+
+### Check What's Being Watched
+
+```typescript
+// List all watched files for a task
+task action=watch_files
+  task_id: 123
+  action: list
+```
+
 ## Overview
 
 The Auto File Tracking system monitors files linked to tasks and automatically manages task lifecycle:
@@ -10,7 +56,266 @@ The Auto File Tracking system monitors files linked to tasks and automatically m
 2. **Auto-Completion**: When all acceptance criteria pass for an `in_progress` task, it automatically moves to `done`
 3. **Zero Token Overhead**: All status updates happen automatically without manual MCP tool calls
 
-**Token Savings**: 97% reduction (4,650 tokens saved per 6-task session)
+## How It Works
+
+### Step 1: Create Task with Files
+
+**Method 1: Using watch_files parameter (v3.4.1+, Recommended)**
+
+```typescript
+task action=create
+  title: "Implement user authentication"
+  acceptance_criteria: [
+    {type: "tests_pass", command: "npm test auth", expected_pattern: "passing"},
+    {type: "code_contains", file: "src/auth.ts", pattern: "export class AuthService"}
+  ]
+  watch_files: ["src/auth.ts"]
+  assigned_agent: "backend-dev"
+  status: "todo"
+```
+
+**Method 2: Using watch_files action (v3.4.1+)**
+
+```typescript
+// Create task first
+task action=create
+  title: "Implement user authentication"
+  acceptance_criteria: [...]
+  status: "todo"
+
+// Then add files to watch
+task action=watch_files
+  task_id: 123
+  action: watch
+  file_paths: ["src/auth.ts"]
+```
+
+**Method 3: Using task.link (Deprecated)**
+
+```typescript
+// ⚠️  DEPRECATED: This still works but shows deprecation warning
+task action=link
+  task_id: 123
+  link_type: "file"
+  target_id: "src/auth.ts"
+```
+
+### Step 2: Automatic Monitoring
+
+When the task is created and files are linked:
+
+1. FileWatcher registers `src/auth.ts` for monitoring
+2. Chokidar starts watching the file for changes
+3. Task remains in `todo` status
+
+### Step 3: Auto-Transition (todo → in_progress)
+
+When an AI agent edits `src/auth.ts`:
+
+1. Chokidar detects file change (after 2s debounce)
+2. FileWatcher identifies task #123 is linked to this file
+3. Task status automatically updates: `todo` → `in_progress`
+4. Activity log records the auto-transition
+
+**Console Output:**
+```
+📝 File changed: auth.ts
+  ✓ Task #123 "Implement user authentication": todo → in_progress
+```
+
+### Step 4: Auto-Completion (in_progress → done)
+
+After the file change, FileWatcher checks acceptance criteria:
+
+1. Executes `npm test auth` command
+2. Checks if output contains "passing"
+3. Validates `src/auth.ts` contains `export class AuthService`
+4. If all checks pass → task moves to `done`
+
+**Console Output:**
+```
+  🔍 Checking acceptance criteria for task #123...
+    ✓ Check 1: Command succeeded and output matches pattern "passing"
+    ✓ Check 2: Pattern "export class AuthService" found in "src/auth.ts"
+  🎉 Task #123 "Implement user authentication": in_progress → done (all checks passed!)
+```
+
+## Smart Review Detection (v3.4.1)
+
+### Overview
+
+The Smart Review Detection feature automatically transitions tasks from `in_progress` to `waiting_review` when quality gates are met. This is purely algorithmic - no AI instructions needed!
+
+### Quality Gates
+
+Four quality gates determine when a task is ready for review:
+
+1. **All Watched Files Modified** - Every file linked to the task must be edited at least once
+2. **TypeScript Compiles** - If .ts/.tsx files present, `tsc --noEmit` must succeed
+3. **Tests Pass** - If test files present (*.test.ts, *.spec.ts), tests must pass
+4. **Idle Time** - No file modifications for 15 minutes (configurable)
+
+### How It Works
+
+**Automatic Tracking:**
+- Every file modification updates `lastModifiedTime` for the task
+- FileWatcher tracks which files have been modified
+- After each file change, a timer starts for the idle period
+
+**Quality Check Sequence:**
+```
+File modified → Track modification → Wait idle period (15min)
+                                           ↓
+                              Check quality gates:
+                              ✓ All files modified?
+                              ✓ TypeScript compiles?
+                              ✓ Tests pass?
+                                           ↓
+                              All passed? → waiting_review
+                              Some failed? → stay in_progress
+```
+
+**Console Output:**
+```
+📝 File changed: auth.ts
+  ✓ Task #123 "Implement user authentication": todo → in_progress
+
+[15 minutes of idle time...]
+
+  ✓ Quality checks passed for task #123
+    • all_files_modified: All 2 watched files have been modified
+    • typescript_compiles: TypeScript compilation successful (2 .ts/.tsx files)
+    • tests_pass: Tests passed (1 test files)
+  → Task #123 auto-transitioned to waiting_review
+```
+
+### Configuration
+
+Configure quality gates in `.sqlew/config.toml`:
+
+```toml
+[tasks]
+# Idle time before checking for review readiness (minutes)
+review_idle_minutes = 15
+
+# Require all watched files to be modified (boolean)
+review_require_all_files_modified = true
+
+# Require tests to pass (boolean)
+review_require_tests_pass = true
+
+# Require TypeScript to compile (boolean)
+review_require_compile = true
+```
+
+**Using MCP Tools:**
+```typescript
+config action=update
+  key: "review_idle_minutes"
+  value: 20  // Wait 20 minutes before checking
+```
+
+### Hybrid Mode: Skip waiting_review
+
+Tasks with `acceptance_criteria` can bypass `waiting_review` and go directly to `done`:
+
+```typescript
+task action=create
+  title: "Implement feature"
+  acceptance_criteria: [
+    {type: "tests_pass", command: "npm test", expected_pattern: "passing"}
+  ]
+  watch_files: ["src/feature.ts"]
+```
+
+**Behavior:**
+- File modified → `todo` → `in_progress`
+- Acceptance criteria met → `in_progress` → `done` (skips waiting_review)
+- Quality gates still checked but not used for auto-transition
+
+### When Quality Checks Fail
+
+If any gate fails, task stays in `in_progress` with diagnostic output:
+
+```
+ℹ Task #123 not ready for review (2 checks failed)
+  • all_files_modified: 1 of 3 watched files not yet modified
+    Unmodified: src/auth.test.ts
+  • typescript_compiles: TypeScript compilation failed
+    src/auth.ts:15:3 - error TS2322: Type 'string' is not assignable to type 'number'
+```
+
+### Status Transition Flow (Updated)
+
+```
+todo → in_progress → waiting_review → done
+  ↓         ↓              ↓
+  File    Quality      Manual
+  change   gates        review
+           met
+
+Alternative (with acceptance_criteria):
+todo → in_progress → done
+  ↓         ↓
+  File   Acceptance
+  change  criteria
+          met
+```
+
+## API Changes (v3.4.1)
+
+### New: watch_files Parameter
+
+Add files to watch when creating or updating tasks:
+
+```typescript
+// In create
+task action=create
+  title: "..."
+  watch_files: ["src/file1.ts", "src/file2.ts"]
+
+// In update
+task action=update
+  task_id: 123
+  watch_files: ["src/file3.ts"]  // Adds to existing watch list
+```
+
+### New: watch_files Action
+
+Dedicated action for managing file watches:
+
+```typescript
+// Watch files
+task action=watch_files
+  task_id: 123
+  action: watch
+  file_paths: ["src/auth.ts", "src/auth.test.ts"]
+
+// Unwatch files
+task action=watch_files
+  task_id: 123
+  action: unwatch
+  file_paths: ["src/auth.test.ts"]
+
+// List watched files
+task action=watch_files
+  task_id: 123
+  action: list
+```
+
+### Deprecated: task.link(link_type="file")
+
+Still works but shows deprecation warning. Use `watch_files` instead:
+
+```typescript
+// OLD (deprecated)
+task action=link task_id=123 link_type=file target_id="src/auth.ts"
+
+// NEW (recommended)
+task action=create watch_files=["src/auth.ts"] ...
+// OR
+task action=watch_files task_id=123 action=watch file_paths=["src/auth.ts"]
+```
 
 ## Architecture
 
@@ -61,66 +366,99 @@ CREATE TABLE t_task_file_links (
 );
 ```
 
-## How It Works
+## Smart File Filtering (v3.4.1)
 
-### Step 1: Create Task with File Links
+### GitIgnore Support
 
-```typescript
-// Create task
-task action=create
-  title: "Implement user authentication"
-  acceptance_criteria: [
-    {type: "tests_pass", command: "npm test auth", expected_pattern: "passing"},
-    {type: "code_contains", file: "src/auth.ts", pattern: "export class AuthService"}
-  ]
-  assigned_agent: "backend-dev"
-  status: "todo"
+The FileWatcher automatically respects `.gitignore` patterns from your project root:
 
-// Link files to task
-task action=link
-  task_id: 123
-  link_type: "file"
-  target_id: "src/auth.ts"
+```bash
+# .gitignore example
+node_modules/
+dist/
+*.log
+.env
 ```
 
-### Step 2: Automatic Monitoring
+**Files matching these patterns will NOT trigger task transitions**, even if linked to a task.
 
-When the task is created and files are linked:
+### Built-In Ignore Patterns
 
-1. FileWatcher registers `src/auth.ts` for monitoring
-2. Chokidar starts watching the file for changes
-3. Task remains in `todo` status
+In addition to `.gitignore`, the watcher has **70+ built-in patterns** that are always ignored:
 
-### Step 3: Auto-Transition (todo → in_progress)
+**Version Control:**
+- `.git`, `.gitignore`, `.gitattributes`
 
-When an AI agent edits `src/auth.ts`:
+**Dependencies:**
+- `node_modules`, `bower_components`, `jspm_packages`
 
-1. Chokidar detects file change (after 2s debounce)
-2. FileWatcher identifies task #123 is linked to this file
-3. Task status automatically updates: `todo` → `in_progress`
-4. Activity log records the auto-transition
+**Build Outputs:**
+- `dist`, `build`, `out`, `.next`, `.nuxt`, `.cache`, `.parcel-cache`, `.vite`
 
-**Console Output:**
-```
-📝 File changed: auth.ts
-  ✓ Task #123 "Implement user authentication": todo → in_progress
+**Logs:**
+- `*.log`, `logs`, `npm-debug.log*`, `yarn-debug.log*`, `pnpm-debug.log*`
+
+**OS Files:**
+- `.DS_Store`, `Thumbs.db`, `desktop.ini`
+
+**IDE/Editor Files:**
+- `.vscode`, `.idea`, `.sublime-project`, `*.swp`, `*.swo`, `*~`
+
+**Temporary Files:**
+- `*.tmp`, `*.temp`, `.tmp`, `.temp`
+
+**Environment Files:**
+- `.env`, `.env.local`, `.env.*.local`
+
+**Database Files:**
+- `*.db`, `*.sqlite`, `*.sqlite3`, `.mcp-context`
+
+**Test Coverage:**
+- `coverage`, `.nyc_output`
+
+**Package Manager Locks:**
+- `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`
+
+### How It Works
+
+```typescript
+// 1. When FileWatcher starts:
+const gitignoreParser = createGitIgnoreParser(projectRoot);
+
+// 2. Chokidar uses the parser to filter files:
+chokidar.watch([], {
+  ignored: (path: string) => {
+    return gitignoreParser.shouldIgnore(path);
+  }
+});
+
+// 3. Files are automatically filtered:
+// ✓ src/auth.ts → watched
+// ✗ node_modules/package/file.js → ignored (built-in pattern)
+// ✗ dist/bundle.js → ignored (.gitignore + built-in pattern)
+// ✗ .env → ignored (built-in pattern for security)
 ```
 
-### Step 4: Auto-Completion (in_progress → done)
+### Benefits
 
-After the file change, FileWatcher checks acceptance criteria:
+1. **Security**: Prevents watching sensitive files (`.env`, credentials)
+2. **Performance**: Doesn't watch thousands of `node_modules` files
+3. **Accuracy**: Only tracks files you actually care about
+4. **Zero Configuration**: Works out of the box with sensible defaults
 
-1. Executes `npm test auth` command
-2. Checks if output contains "passing"
-3. Validates `src/auth.ts` contains `export class AuthService`
-4. If all checks pass → task moves to `done`
+### Project Root Detection
 
-**Console Output:**
+The watcher automatically detects your project root as `process.cwd()` and:
+- Loads `.gitignore` from the root (if it exists)
+- Applies built-in patterns relative to the root
+- Normalizes all file paths relative to the root
+
+**Console Output on Startup:**
 ```
-  🔍 Checking acceptance criteria for task #123...
-    ✓ Check 1: Command succeeded and output matches pattern "passing"
-    ✓ Check 2: Pattern "export class AuthService" found in "src/auth.ts"
-  🎉 Task #123 "Implement user authentication": in_progress → done (all checks passed!)
+✓ File watcher started successfully
+  Project root: /home/user/my-project
+  Watching 3 files for 2 tasks
+  .gitignore patterns loaded: Yes
 ```
 
 ## Acceptance Criteria Types
@@ -158,16 +496,18 @@ File watcher settings are built into the system:
 **Symptoms**: File changes detected but task stays in `todo`
 
 **Solutions**:
-1. Verify file is linked to task:
+1. Verify files are linked to task:
    ```typescript
+   task action=watch_files task_id=123 action=list
+   // OR
    task action=get task_id=123
    // Check for file links in response
    ```
 
 2. Check file watcher status:
    ```typescript
-   stats action=db_stats
-   // Look for file_watcher_running: true
+   task action=watcher subaction=status
+   // Should show: running: true, files_watched: N
    ```
 
 3. Verify task is in correct status:
@@ -207,8 +547,13 @@ File watcher settings are built into the system:
      Watching 3 files for 2 tasks
    ```
 
-2. Restart MCP server if watcher failed to start
-3. Check file paths are absolute (not relative)
+2. Check watcher status via MCP tool:
+   ```typescript
+   task action=watcher subaction=status
+   ```
+
+3. Restart MCP server if watcher failed to start
+4. Check file paths are absolute (not relative)
 
 ### Issue: Tests Timing Out
 
@@ -264,18 +609,33 @@ For 6 tasks: **4,800 tokens saved** (97% reduction in status management overhead
 
 ## Best Practices
 
-### 1. Link Only Relevant Files
+### 1. Use watch_files Parameter for New Tasks (v3.4.1+)
 
 ```typescript
-// GOOD: Link specific implementation files
+// GOOD: Create task with files in one step
+task action=create
+  title: "Implement auth"
+  watch_files: ["src/auth.ts", "src/auth.test.ts"]
+  acceptance_criteria: [...]
+
+// BAD: Using deprecated link API
+task action=create title="Implement auth"
 task action=link task_id=123 link_type=file target_id="src/auth.ts"
-task action=link task_id=123 link_type=file target_id="src/auth.test.ts"
+```
+
+### 2. Link Only Relevant Files
+
+```typescript
+// GOOD: Link specific implementation files
+task action=create
+  watch_files: ["src/auth.ts", "src/auth.test.ts"]
 
 // BAD: Don't link entire directories or unrelated files
-task action=link task_id=123 link_type=file target_id="src/"  // ❌ Not a file
+task action=create
+  watch_files: ["src/"]  // ❌ Not a file
 ```
 
-### 2. Use Specific Acceptance Criteria
+### 3. Use Specific Acceptance Criteria
 
 ```typescript
 // GOOD: Specific, testable criteria
@@ -290,7 +650,7 @@ acceptance_criteria: [
 ]
 ```
 
-### 3. Set Realistic Timeouts
+### 4. Set Realistic Timeouts
 
 ```typescript
 // GOOD: Appropriate timeouts based on test complexity
@@ -300,7 +660,7 @@ acceptance_criteria: [
 ]
 ```
 
-### 4. Combine Multiple Check Types
+### 5. Combine Multiple Check Types
 
 ```typescript
 acceptance_criteria: [
@@ -318,7 +678,7 @@ acceptance_criteria: [
 ]
 ```
 
-### 5. Archive Completed Tasks
+### 6. Archive Completed Tasks
 
 ```typescript
 // Move done tasks to archived to stop file watching
@@ -348,9 +708,16 @@ acceptance_criteria: [
 Link multiple files to a single task:
 
 ```typescript
-task action=link task_id=123 link_type=file target_id="src/auth.ts"
-task action=link task_id=123 link_type=file target_id="src/auth.test.ts"
-task action=link task_id=123 link_type=file target_id="docs/auth.md"
+// Using watch_files parameter (v3.4.1+)
+task action=create
+  title: "Implement authentication"
+  watch_files: ["src/auth.ts", "src/auth.test.ts", "docs/auth.md"]
+
+// OR using watch_files action
+task action=watch_files
+  task_id: 123
+  action: watch
+  file_paths: ["src/auth.ts", "src/auth.test.ts", "docs/auth.md"]
 ```
 
 Any file change triggers the workflow.
@@ -367,70 +734,107 @@ acceptance_criteria: [
 ]
 ```
 
-## Migration from Manual Task Management
-
-See [TASK_MIGRATION.md](./TASK_MIGRATION.md) for detailed migration guide.
-
-## API Reference
-
-See [TOOL_REFERENCE.md](./TOOL_REFERENCE.md) for complete API documentation.
-
 ## Examples
 
-### Example 1: Simple Bug Fix
+### Example 1: Simple Bug Fix (v3.4.1)
 
 ```typescript
-// Create task
+// Create task with file watching
 task action=create
   title: "Fix login button styling"
+  watch_files: ["src/LoginButton.css"]
   acceptance_criteria: [
     {type: "code_contains", file: "src/LoginButton.css", pattern: "background-color: #007bff"},
     {type: "tests_pass", command: "npm test LoginButton", expected_pattern: "passing"}
   ]
 
-// Link file
-task action=link task_id=45 link_type=file target_id="src/LoginButton.css"
-
 // Edit file → auto-transitions to in_progress
 // Tests pass → auto-completes to done
 ```
 
-### Example 2: Feature Implementation
+### Example 2: Feature Implementation (v3.4.1)
 
 ```typescript
-// Create task
+// Create task with multiple files
 task action=create
   title: "Add user profile endpoint"
+  watch_files: ["src/routes/profile.ts", "src/routes/profile.test.ts"]
   acceptance_criteria: [
     {type: "file_exists", file: "src/routes/profile.ts"},
     {type: "code_contains", file: "src/routes/profile.ts", pattern: "router.get\\('/profile'"},
     {type: "tests_pass", command: "npm test routes/profile", expected_pattern: "5 passing"}
   ]
-
-// Link implementation and test files
-task action=link task_id=46 link_type=file target_id="src/routes/profile.ts"
-task action=link task_id=46 link_type=file target_id="src/routes/profile.test.ts"
 ```
 
-### Example 3: Refactoring Task
+### Example 3: Refactoring Task (v3.4.1)
 
 ```typescript
 // Create task
 task action=create
   title: "Remove deprecated API endpoints"
+  watch_files: ["src/api/legacy.ts"]
   acceptance_criteria: [
     {type: "code_removed", file: "src/api/legacy.ts", pattern: "router.get\\('/old-endpoint'"},
     {type: "code_removed", file: "src/api/legacy.ts", pattern: "// DEPRECATED"},
     {type: "tests_pass", command: "npm test", expected_pattern: "0 failing"}
   ]
+```
+
+### Example 4: Adding Files to Existing Task
 
-// Link file to watch
-task action=link task_id=47 link_type=file target_id="src/api/legacy.ts"
+```typescript
+// Task already exists, add more files to watch
+task action=watch_files
+  task_id: 47
+  action: watch
+  file_paths: ["src/api/v2.ts", "src/api/v2.test.ts"]
+
+// Check what's being watched
+task action=watch_files
+  task_id: 47
+  action: list
+// Returns: { files: ["src/api/legacy.ts", "src/api/v2.ts", "src/api/v2.test.ts"] }
+
+// Remove file from watch list
+task action=watch_files
+  task_id: 47
+  action: unwatch
+  file_paths: ["src/api/legacy.ts"]
+```
+
+## Migration from v3.2.x to v3.4.1
+
+### Before (v3.2.x)
+
+```typescript
+// Create task
+task action=create title="Implement auth" ...
+
+// Link files separately
+task action=link task_id=123 link_type=file target_id="src/auth.ts"
+task action=link task_id=123 link_type=file target_id="src/auth.test.ts"
+```
+
+### After (v3.4.1)
+
+```typescript
+// Create task with files in one call
+task action=create
+  title: "Implement auth"
+  watch_files: ["src/auth.ts", "src/auth.test.ts"]
+  ...
 ```
 
+**Benefits**:
+- Fewer MCP calls (1 instead of 3)
+- Clearer intent
+- Better error handling
+- Batch file registration with watcher
+
 ## See Also
 
 - [ACCEPTANCE_CRITERIA.md](./ACCEPTANCE_CRITERIA.md) - All acceptance check types
 - [WORKFLOWS.md](./WORKFLOWS.md) - Multi-agent coordination patterns
 - [BEST_PRACTICES.md](./BEST_PRACTICES.md) - Common errors and solutions
 - [TASK_MIGRATION.md](./TASK_MIGRATION.md) - Migrate from decision-based tracking
+- [MIGRATION_v3.3.md](./MIGRATION_v3.3.md) - Migration guide from v3.2.x to v3.4.1