speccrew 0.6.14 → 0.6.16

package/.speccrew/agents/speccrew-product-manager.md

@@ -408,6 +408,12 @@ No knowledge base exists. A lightweight feature inventory scan is triggered to d
 > 🛑 **MANDATORY -- Path C -> Path B Sequence**:
 > After init-features completes (features-*.json generated), you MUST immediately execute Path B:
 >
+> **Path B Recovery Check**:
+> Before starting Step 1, check if `{workspace_path}/knowledges/bizs/DISPATCH-PROGRESS.json` exists:
+> - If exists with pending tasks → resume from Step 3 (skip Step 1 and Step 1.5)
+> - If exists with all completed → skip to Step 5 (Update Features Status)
+> - If not exists → start from Step 1
+>
 > **Path B Step 1: Module Matching**
 > Dispatch Worker with `speccrew-pm-module-matcher` skill to match requirement features against the generated features inventory.
 >
@@ -422,6 +428,26 @@ No knowledge base exists. A lightweight feature inventory scan is triggered to d
 >     requirement_summary: <brief summary of user's requirement>
 > ```
 >
+> **Path B Step 1.5: Initialize Knowledge Dispatch Progress**
+>
+> Save the matcher Worker's output to a temporary file:
+> - File path: `{workspace_path}/knowledges/bizs/.matcher-result.json`
+> - Content: The matcher's full JSON output (matched modules with platform_id, module_name, confidence)
+>
+> Run the dispatch progress initialization script:
+> ```bash
+> node "{update_progress_script}" init-knowledge-tasks `
+>   --file "{workspace_path}/knowledges/bizs/DISPATCH-PROGRESS.json" `
+>   --matcher-result "{workspace_path}/knowledges/bizs/.matcher-result.json" `
+>   --features-dir "{workspace_path}/knowledges/bizs"
+> ```
+>
+> Verify the DISPATCH-PROGRESS.json was created successfully and log the total task count.
+>
+> 🛑 **HARD GATE — DISPATCH-PROGRESS.json MUST exist before proceeding**
+> DO NOT proceed to Step 2 until DISPATCH-PROGRESS.json is created and contains tasks.
+> If the script fails, diagnose the error and retry. DO NOT skip this step.
+>
 > 🛑 **CRITICAL GATE — DO NOT SKIP Steps 2-5**:
 > After matcher completes, you MUST execute Steps 2-5 to deep-initialize the matched modules' knowledge base.
 > DO NOT jump to Phase 2 (Complexity Assessment) or Phase 3 (Requirement Clarification) until ALL Steps complete.
@@ -455,33 +481,70 @@ No knowledge base exists. A lightweight feature inventory scan is triggered to d
 >
 > Wait for ALL module-initializer Workers to complete. Collect all task plan JSON outputs.
 >
-> **Path B Step 3: Execute Feature Analysis**
-> Based on the task plans from Step 2, dispatch Worker for EACH pending feature.
+> **Path B Step 3: Execute Feature Analysis (Dispatch-Tracked)**
 >
-> For backend features: dispatch Worker with `speccrew-knowledge-bizs-api-analyze` skill
-> For web/mobile/desktop features: dispatch Worker with `speccrew-knowledge-bizs-ui-analyze` skill
+> ⚠️ **MANDATORY RULES FOR THIS STEP:**
+> 1. MUST dispatch Workers for ALL tasks with status "pending" in DISPATCH-PROGRESS.json
+> 2. DO NOT skip any pending task — the matcher already determined relevance
+> 3. DO NOT stop mid-way to ask user for options (A/B/C) — execute ALL tasks
+> 4. After each Worker completes, update task status via script
 >
-> **Agent Tool Invocation for each feature**:
-> ```
-> Use the Agent tool to invoke speccrew-task-worker:
-> - agent: speccrew-task-worker
-> - task: Execute {analyzer_skill} for feature "{fileName}"
-> - context:
->     skill: {analyzer_skill}
->     fileName: {task.fileName}
->     sourcePath: {source_path}/{task.sourcePath}
->     documentPath: {workspace_path}/{task.documentPath}
->     module: {task.module}
->     analyzed: false
->     platform_type: {task.platform_type}
->     platform_subtype: {task.platform_subtype}
->     tech_stack: {task.tech_stack}
->     language: {language}
->     completed_dir: {sync_state_bizs_dir}/completed
->     sourceFile: features-{platform_id}.json
-> ```
+> **Recovery Check**: Read `{workspace_path}/knowledges/bizs/DISPATCH-PROGRESS.json`:
+> - If tasks with `status: "completed"` exist, skip them (resuming from interruption)
+> - Only dispatch tasks with `status: "pending"`
+> - Log: "Resuming: {completed_count} completed, {pending_count} remaining"
 >
-> Wait for ALL analyze Workers to complete.
+> **Dispatch Loop**:
+> For each task in DISPATCH-PROGRESS.json where status == "pending":
+>
+> 1. Dispatch `speccrew-task-worker` with:
+>    - **skill**: `{task.analyzer_skill}` (e.g., `speccrew-knowledge-bizs-api-analyze` or `speccrew-knowledge-bizs-ui-analyze`)
+>    - **context**: All task fields (module, platform_id, fileName, sourcePath, etc.)
+>
+>    **Agent Tool Invocation**:
+>    ```
+>    Use the Agent tool to invoke speccrew-task-worker:
+>    - agent: speccrew-task-worker
+>    - task: Execute {task.analyzer_skill} for feature "{task.fileName}"
+>    - context:
+>        skill: {task.analyzer_skill}
+>        fileName: {task.fileName}
+>        sourcePath: {source_path}/{task.sourcePath}
+>        documentPath: {workspace_path}/{task.documentPath}
+>        module: {task.module}
+>        platform_id: {task.platform_id}
+>        platform_type: {task.platform_type}
+>        platform_subtype: {task.platform_subtype}
+>        tech_stack: {task.tech_stack}
+>        language: {language}
+>        completed_dir: {sync_state_bizs_dir}/completed
+>        sourceFile: features-{task.platform_id}.json
+>    ```
+>
+> 2. After Worker completes successfully, update progress:
+>    ```bash
+>    node "{update_progress_script}" update-task `
+>      --file "{workspace_path}/knowledges/bizs/DISPATCH-PROGRESS.json" `
+>      --task-id "{task.id}" `
+>      --status completed
+>    ```
+>
+> 3. If Worker fails, update status to "failed" and continue with next task:
+>    ```bash
+>    node "{update_progress_script}" update-task `
+>      --file "{workspace_path}/knowledges/bizs/DISPATCH-PROGRESS.json" `
+>      --task-id "{task.id}" `
+>      --status failed
+>    ```
+>
+> **Completion Check**: After all dispatches complete, read DISPATCH-PROGRESS.json:
+> - If all tasks are "completed" → proceed to Step 4
+> - If some tasks "failed" → log failures and proceed (do not block on failures)
+>
+> 🛑 **CONTINUOUS EXECUTION — DO NOT INTERRUPT**
+> Process ALL pending tasks without stopping for user confirmation.
+> The scope was already determined by the matcher in Step 1.
+> Asking "do you want to continue?" mid-way is FORBIDDEN.
 >
 > **Path B Step 4: Generate Module Summaries**
 > For each matched module, dispatch Worker with `speccrew-knowledge-module-summarize` skill:
@@ -503,6 +566,7 @@ No knowledge base exists. A lightweight feature inventory scan is triggered to d
 >
 > 🛑 **Path B Completion Check**:
 > Before proceeding to Phase 2, verify:
+> - [ ] Step 1.5 completed: DISPATCH-PROGRESS.json created with all tasks
 > - [ ] Step 2 completed: task plan JSON generated for each matched module
 > - [ ] Step 3 completed: analyze Workers dispatched and completed for ALL pending features
 > - [ ] Step 4 completed: module-summarize Workers completed for ALL matched modules
@@ -530,12 +594,39 @@ This context will be passed to Phase 3 (Requirement Clarification) and Phase 4 (
 
 ## Phase 2: Complexity Assessment & Skill Routing
 
-> **PRE-CONDITION**: Path B Steps 2-5 must be completed before entering Phase 2.
-> If matched_modules exist but knowledges/bizs/ directories are empty, Path B was not fully executed.
-> Go back and execute the remaining Path B steps.
-
 Before starting requirement analysis, assess the requirement complexity to determine the appropriate skill path.
 
+### Phase 2.0: Knowledge Initialization Verification (MANDATORY)
+
+> ⚠️ **THIS STEP IS MANDATORY AND CANNOT BE SKIPPED**
+> You MUST execute the verification commands below before ANY complexity assessment.
+
+**Step 2.0.1**: Check if DISPATCH-PROGRESS.json exists:
+```bash
+# Read the file — if it does not exist, Path B Step 1.5 was not executed
+cat "{workspace_path}/knowledges/bizs/DISPATCH-PROGRESS.json"
+```
+
+**Step 2.0.2**: Evaluate the file content:
+
+| Condition | Action |
+|-----------|--------|
+| File does NOT exist | Path B was not executed. Go back to Phase 1.2 Path B Step 1. |
+| File exists, `counts.pending > 0` | Knowledge initialization is INCOMPLETE. Go back to Path B Step 3 and dispatch remaining tasks. |
+| File exists, `counts.pending == 0` AND `counts.completed > 0` | Knowledge initialization is COMPLETE. Proceed to Phase 2.1. |
+| File exists, `counts.total == 0` | No features to analyze (all already analyzed). Proceed to Phase 2.1. |
+
+**Step 2.0.3**: If going back to Path B:
+1. Read DISPATCH-PROGRESS.json to get pending task list
+2. For each module with pending tasks, dispatch `speccrew-task-worker` with `speccrew-pm-module-initializer` skill (if Step 2 not done)
+3. Then dispatch analyze Workers for all pending features (Step 3)
+4. Then dispatch summarize Workers (Step 4)
+5. Update features status (Step 5)
+6. Return here to Phase 2.0 and re-verify
+
+> 🔴 **ABSOLUTE RULE**: DO NOT proceed to Phase 2.1 (Complexity Assessment) while `counts.pending > 0`.
+> The matcher identified these features as relevant. Skipping their analysis means the PRD will lack existing system feature descriptions.
+
 ### 2.1 Complexity Indicators
 
 Evaluate the user's requirement against these indicators:

package/.speccrew/skills/speccrew-pm-module-initializer/SKILL.md

@@ -52,6 +52,7 @@ Generate analyze task plan for a single business module. Reads features-*.json,
   "analyzer_skill": "speccrew-knowledge-bizs-ui-analyze",
   "tasks": [
     {
+      "id": "ki-web-vue-system-index",
       "fileName": "index",
       "sourcePath": "src/views/system/user/index.vue",
       "documentPath": "knowledges/bizs/web-vue/system/features",
@@ -59,7 +60,8 @@ Generate analyze task plan for a single business module. Reads features-*.json,
       "platform_type": "web",
       "platform_subtype": "vue",
       "tech_stack": ["vue", "typescript"],
-      "language": "zh"
+      "language": "zh",
+      "status": "pending"
     }
   ],
   "total_pending": 90,
@@ -83,6 +85,18 @@ Generate analyze task plan for a single business module. Reads features-*.json,
 | `total_pending` | Count of pending features |
 | `summarize_params` | Parameters for module-summarize skill (to be executed after all analyze tasks complete) |
 
+**Task ID Naming Convention**:
+
+The `id` field follows the format: `ki-{platform_id}-{module}-{fileName}`
+- `ki` = knowledge initialization
+- `platform_id` = Platform identifier (e.g., "web-vue", "admin-api")
+- `module` = Business module name
+- `fileName` = Feature file name
+
+Example: `"ki-web-vue-system-index"`
+
+> **Note**: This ID is consistent with task IDs in DISPATCH-PROGRESS.json and is used for progress tracking.
+
 ## Workflow
 
 ```mermaid
@@ -132,6 +146,7 @@ For each pending feature from Step 1, build a task object with analyzer paramete
 
 ```json
 {
+  "id": "ki-{platform_id}-{feature.module}-{feature.fileName}",
   "fileName": "{feature.fileName}",
   "sourcePath": "{feature.sourcePath}",
   "documentPath": "{output_path}/bizs/{platform_id}/{feature.module}/features",
@@ -139,7 +154,8 @@ For each pending feature from Step 1, build a task object with analyzer paramete
   "platform_type": "{platform_type}",
   "platform_subtype": "{platform_subtype}",
   "tech_stack": "{tech_stack}",
-  "language": "{language}"
+  "language": "{language}",
+  "status": "pending"
 }
 ```
 
@@ -151,6 +167,8 @@ For backend features (api-analyze), also include:
 }
 ```
 
+**Note**: The `id` field format is `ki-{platform_id}-{module}-{fileName}` where `ki` stands for "knowledge initialization". This ID is consistent with task IDs in DISPATCH-PROGRESS.json for progress tracking.
+
 **Output**: "Step 3 Status: ✅ COMPLETED - Built {count} task entries"
 
 ### Step 4: Build Summarize Parameters

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "speccrew",
-  "version": "0.6.14",
+  "version": "0.6.16",
   "description": "Spec-Driven Development toolkit for AI-powered IDEs",
   "author": "charlesmu99",
   "repository": {

package/workspace-template/scripts/update-progress.js

@@ -56,6 +56,14 @@
  *      --status <status>       Status: pending/in_progress/completed/confirmed (required)
  *      --output <text>         Output information (optional)
  *
+ * 7. init-knowledge-tasks - Generate knowledge initialization tasks from matcher results
+ *    node update-progress.js init-knowledge-tasks --file <path> --matcher-result <path> --features-dir <dir> [--force]
+ *    Options:
+ *      --file <path>           Progress file path (required)
+ *      --matcher-result <path> Matcher result JSON file path (required)
+ *      --features-dir <dir>    Directory containing features-*.json files (required)
+ *      --force                 Overwrite existing file
+ *
  * Output Format:
  *   Success: {"success": true, "message": "...", "data": {...}}
  *   Failure: {"success": false, "error": "..."} (output to stderr, exit code 1)
@@ -345,6 +353,10 @@ function parseArgs() {
             case '-Force':
                 result.force = true;
                 break;
+            case '--matcher-result':
+            case '-Matcher-Result':
+                result.matcherResult = args[++i];
+                break;
         }
     }
 
@@ -816,6 +828,201 @@ function cmdUpdateWorkflow(args) {
     }
 }
 
+/**
+ * Determine analyzer skill based on platform_id
+ * @param {string} platformId - Platform identifier
+ * @returns {string} Analyzer skill name
+ */
+function getAnalyzerSkill(platformId) {
+    const pid = platformId.toLowerCase();
+    if (pid.includes('web') || pid.includes('mobile')) {
+        return 'speccrew-knowledge-bizs-ui-analyze';
+    }
+    if (pid.includes('backend') || pid.includes('api')) {
+        return 'speccrew-knowledge-bizs-api-analyze';
+    }
+    // Default to UI analyzer for unknown platforms
+    return 'speccrew-knowledge-bizs-ui-analyze';
+}
+
+/**
+ * Command: init-knowledge-tasks - Generate knowledge initialization tasks from matcher results
+ * 
+ * Reads matcher result and features-*.json files to generate a task list for knowledge initialization.
+ * Each task corresponds to a feature that needs to be analyzed.
+ */
+function cmdInitKnowledgeTasks(args) {
+    // Validate required arguments
+    if (!args.file || !args.matcherResult || !args.featuresDir) {
+        outputError('Usage: init-knowledge-tasks --file <path> --matcher-result <path> --features-dir <dir> [--force]');
+    }
+
+    const filePath = path.resolve(args.file);
+    const matcherResultPath = path.resolve(args.matcherResult);
+    const featuresDir = path.resolve(args.featuresDir);
+
+    // Check if matcher result file exists
+    if (!fs.existsSync(matcherResultPath)) {
+        outputError(`Matcher result file not found: ${matcherResultPath}`);
+    }
+
+    // Check if features directory exists
+    if (!fs.existsSync(featuresDir)) {
+        outputError(`Features directory not found: ${featuresDir}`);
+    }
+
+    // Check if target file already exists (prevent overwrite without --force)
+    if (fs.existsSync(filePath) && !args.force) {
+        outputError(`Progress file already exists: ${filePath}. Use --force to overwrite.`);
+    }
+
+    // Read matcher result
+    let matcherResult;
+    try {
+        matcherResult = readJsonFile(matcherResultPath);
+    } catch (e) {
+        outputError(`Failed to read matcher result: ${e.message}`);
+    }
+
+    // Extract matched modules (high + medium confidence)
+    const matchedModules = matcherResult.matched_modules || [];
+    if (matchedModules.length === 0) {
+        outputError('No matched modules found in matcher result');
+    }
+
+    // Filter to high and medium confidence matches
+    const validConfidences = ['high', 'medium'];
+    const filteredModules = matchedModules.filter(m => 
+        validConfidences.includes((m.confidence || '').toLowerCase())
+    );
+
+    if (filteredModules.length === 0) {
+        outputError('No modules with high or medium confidence found');
+    }
+
+    // Scan features-*.json files
+    const featuresFiles = fs.readdirSync(featuresDir).filter(f => 
+        f.startsWith('features-') && f.endsWith('.json')
+    );
+
+    if (featuresFiles.length === 0) {
+        outputError(`No features-*.json files found in: ${featuresDir}`);
+    }
+
+    // Load all features data indexed by platform_id
+    const featuresDataByPlatform = {};
+    for (const file of featuresFiles) {
+        const filePath = path.join(featuresDir, file);
+        try {
+            const data = readJsonFile(filePath);
+            if (data.platform_id) {
+                featuresDataByPlatform[data.platform_id] = data;
+            }
+        } catch (e) {
+            outputError(`Failed to parse features file ${file}: ${e.message}`);
+        }
+    }
+
+    // Generate tasks from matched modules and features
+    const tasks = [];
+    const now = getTimestamp();
+
+    for (const matchedModule of filteredModules) {
+        const { module_name, platform_id, features: matchedFeatures } = matchedModule;
+
+        // Determine analyzer skill based on platform
+        const analyzerSkill = getAnalyzerSkill(platform_id);
+
+        // Get features for this module
+        let featuresToProcess = [];
+
+        if (matchedFeatures && Array.isArray(matchedFeatures) && matchedFeatures.length > 0) {
+            // Use features directly from matcher result
+            featuresToProcess = matchedFeatures;
+        } else {
+            // Look up features from features-*.json files
+            const platformData = featuresDataByPlatform[platform_id];
+            if (platformData && platformData.modules && platformData.modules[module_name]) {
+                const moduleData = platformData.modules[module_name];
+                if (moduleData.features && Array.isArray(moduleData.features)) {
+                    featuresToProcess = moduleData.features;
+                }
+            }
+        }
+
+        // Filter to only unanalyzed features (analyzed !== true)
+        const unanalyzedFeatures = featuresToProcess.filter(f => f.analyzed !== true);
+
+        // Create task for each unanalyzed feature
+        for (const feature of unanalyzedFeatures) {
+            const fileName = feature.fileName || feature.file_name || 'unknown';
+            const sourcePath = feature.sourcePath || feature.source_path || '';
+
+            // Generate task ID: ki-{platform_id}-{module}-{fileName}
+            // Sanitize fileName for ID (remove extension, replace special chars)
+            const fileNameForId = fileName.replace(/\.[^.]+$/, '').replace(/[^a-zA-Z0-9_-]/g, '_');
+            const taskId = `ki-${platform_id}-${module_name}-${fileNameForId}`;
+
+            tasks.push({
+                id: taskId,
+                name: `Analyze ${module_name}.${fileName} (${platform_id})`,
+                status: 'pending',
+                module: module_name,
+                platform_id: platform_id,
+                fileName: fileName,
+                sourcePath: sourcePath,
+                analyzer_skill: analyzerSkill,
+                created_at: now
+            });
+        }
+    }
+
+    // Sort tasks: by platform_id, then module, then fileName
+    tasks.sort((a, b) => {
+        if (a.platform_id !== b.platform_id) {
+            return a.platform_id.localeCompare(b.platform_id);
+        }
+        if (a.module !== b.module) {
+            return a.module.localeCompare(b.module);
+        }
+        return a.fileName.localeCompare(b.fileName);
+    });
+
+    // Create progress file structure
+    const progressData = {
+        stage: 'knowledge_initialization',
+        created_at: now,
+        updated_at: now,
+        counts: calculateCounts(tasks),
+        tasks: tasks,
+        checkpoints: {}
+    };
+
+    // Ensure directory exists
+    const dir = path.dirname(filePath);
+    if (!fs.existsSync(dir)) {
+        fs.mkdirSync(dir, { recursive: true });
+    }
+
+    // Acquire lock and write
+    let lockPath = null;
+    try {
+        lockPath = acquireLock(filePath);
+        atomicWriteJson(filePath, progressData);
+        outputSuccess(
+            `Generated ${tasks.length} knowledge initialization tasks`,
+            {
+                file: filePath,
+                stage: 'knowledge_initialization',
+                matched_modules: filteredModules.length,
+                counts: progressData.counts
+            }
+        );
+    } finally {
+        if (lockPath) releaseLock(lockPath);
+    }
+}
+
 /**
  * Command: init-tasks - Scan feature-design directory to generate task list
  */
@@ -983,6 +1190,7 @@ function main() {
         console.error('  write-checkpoint Write or update a checkpoint');
         console.error('  update-workflow  Update a workflow stage status');
         console.error('  init-tasks       Generate tasks from feature-spec files');
+        console.error('  init-knowledge-tasks  Generate knowledge initialization tasks from matcher results');
         console.error('');
         console.error('Run "node update-progress.js <command> --help" for more information.');
         process.exit(1);
@@ -1012,6 +1220,9 @@ function main() {
             case 'init-tasks':
                 cmdInitTasks(args);
                 break;
+            case 'init-knowledge-tasks':
+                cmdInitKnowledgeTasks(args);
+                break;
             default:
                 outputError(`Unknown command: ${args.command}`);
         }