specweave 0.28.57 → 0.28.61

package/plugins/specweave/commands/specweave-jobs.md

@@ -1,7 +1,7 @@
 ---
 name: specweave:jobs
-description: Monitor background jobs (repo cloning, issue import). Shows progress, allows pause/resume.
-usage: /specweave:jobs [--active|--all|--id <job-id>] [--resume <job-id>]
+description: Monitor background jobs (repo cloning, issue import). Shows progress, allows pause/resume/kill.
+usage: /specweave:jobs [--active|--all|--id <job-id>] [--resume <job-id>] [--kill <job-id>] [--follow <job-id>] [--logs <job-id>]
 ---
 
 # Background Jobs Monitor
@@ -17,6 +17,25 @@ Monitor and manage long-running background operations:
 - **Issue import** (10K+ items from GitHub/JIRA/ADO)
 - **External sync** operations
 
+**ASYNC ARCHITECTURE (2025-12-01)**:
+- Jobs run as **detached processes** that survive terminal close
+- Progress tracked via filesystem (`.specweave/state/jobs/`)
+- Can check status anytime with `/specweave:jobs`
+
+---
+
+## Command Options
+
+| Option | Description |
+|--------|-------------|
+| (none) | Show active jobs |
+| `--all` | Show all jobs (including completed) |
+| `--id <jobId>` | Show details for specific job |
+| `--follow <jobId>` | Follow job progress in real-time |
+| `--logs <jobId>` | Show worker log output |
+| `--resume <jobId>` | Resume paused job |
+| `--kill <jobId>` | Kill running background job |
+
 ---
 
 ## Check Job Status
@@ -26,6 +45,12 @@ Read the background jobs state file and display status:
 ```bash
 # Find and read the state file
 STATE_FILE=".specweave/state/background-jobs.json"
+
+# Also check job-specific files:
+# .specweave/state/jobs/<jobId>/config.json  - Job configuration
+# .specweave/state/jobs/<jobId>/worker.pid   - Process ID (if running)
+# .specweave/state/jobs/<jobId>/worker.log   - Worker output log
+# .specweave/state/jobs/<jobId>/result.json  - Results (when complete)
 ```
 
 ### Display Format
@@ -34,25 +59,28 @@ STATE_FILE=".specweave/state/background-jobs.json"
 📋 Background Jobs
 
 🔄 Running (1):
-  [abc12345] clone-repos
-     Progress: 2/4 (50%) → sw-meeting-cost-be
-     Rate: 0.5/s | ETA: ~4s
-     Started: 2 mins ago
+  [abc12345] import-issues (ADO)
+     Progress: 2,500/10,000 (25%)
+     Rate: 15.2/s | ETA: ~8m 14s
+     PID: 45678 | Started: 2 mins ago
 
 ⏸️  Paused (1):
   [def67890] import-issues (GitHub)
      Progress: 1,234/10,000 (12%)
-     Reason: Rate limited
+     Reason: Rate limited (resumes in 45s)
      Resume: /specweave:jobs --resume def67890
 
 ✅ Completed (2):
-  [ghi11111] clone-repos - 4/4 repos - 5 mins ago
-  [jkl22222] import-issues - 500 items - 1 hour ago
+  [ghi11111] import-issues - 4,500 items - 5 mins ago
+  [jkl22222] clone-repos - 4/4 repos - 1 hour ago
 
 💡 Commands:
-   /specweave:jobs --id abc12345  → Details for specific job
-   /specweave:jobs --resume def67890  → Resume paused job
-   /specweave:jobs --all  → Show all jobs (including old)
+   /specweave:jobs --id abc12345     → Details for specific job
+   /specweave:jobs --follow abc12345 → Follow progress live
+   /specweave:jobs --logs abc12345   → View worker logs
+   /specweave:jobs --resume def67890 → Resume paused job
+   /specweave:jobs --kill abc12345   → Kill running job
+   /specweave:jobs --all             → Show all jobs (including old)
 ```
 
 ---
@@ -66,32 +94,92 @@ STATE_FILE=".specweave/state/background-jobs.json"
 
 📦 Job Details: abc12345
 
-Type: clone-repos
+Type: import-issues
 Status: running
+Provider: ADO
+PID: 45678
+
 Started: 2024-01-15 10:30:00
 Updated: 2024-01-15 10:32:15
 
-Progress: 2/4 (50%)
-Current: sw-meeting-cost-be
-Rate: 0.5 repos/sec
-ETA: ~4 seconds
+Progress: 2,500/10,000 (25%)
+Current: OlySense\Core-Operations
+Rate: 15.2 items/sec
+ETA: ~8 minutes
+
+Files:
+  Config: .specweave/state/jobs/abc12345/config.json
+  Logs: .specweave/state/jobs/abc12345/worker.log
+  PID: .specweave/state/jobs/abc12345/worker.pid
+```
+
+### Follow Job Progress Live
+
+Watch job progress in real-time (like `tail -f`):
 
-Completed:
-  ✅ sw-meeting-cost-fe
-  ✅ sw-meeting-cost-shared
+```
+/specweave:jobs --follow abc12345
 
-Remaining:
-  ⏳ sw-meeting-cost-be
-  ⏳ sw-meeting-cost
+📦 Following job abc12345 (Ctrl+C to stop)
 
-Config:
-  Project: /path/to/project
-  Repos: 4 total
+[10:30:15] Progress: 2,500/10,000 (25%) - OlySense\Core-Operations
+[10:30:16] Progress: 2,520/10,000 (25%) - OlySense\Core-Operations
+[10:30:17] Progress: 2,545/10,000 (25%) - OlySense\AI-Platform
+[10:30:18] Progress: 2,570/10,000 (26%) - OlySense\AI-Platform
+...
+```
+
+**Implementation**: Read `.specweave/state/jobs/<jobId>/worker.log` with tail-like behavior, or poll the job state file every second.
+
+### View Worker Logs
+
+Show detailed worker output:
+
+```
+/specweave:jobs --logs abc12345
+
+📋 Worker Logs for abc12345 (last 50 lines):
+
+[2024-01-15T10:30:00.123Z] Worker started for job abc12345
+[2024-01-15T10:30:00.456Z] Project path: /Users/dev/my-project
+[2024-01-15T10:30:00.789Z] PID: 45678
+[2024-01-15T10:30:01.234Z] Dependencies loaded, starting import...
+[2024-01-15T10:30:02.567Z] Progress: 100/10000 - ado OlySense\Core-Operations
+[2024-01-15T10:30:03.890Z] Progress: 200/10000 - ado OlySense\Core-Operations
+...
+```
+
+**Implementation**: Read `.specweave/state/jobs/<jobId>/worker.log`
+
+### Kill Running Job
+
+Stop a background job:
+
+```
+/specweave:jobs --kill abc12345
+
+⚠️  Killing job abc12345...
+   Type: import-issues
+   PID: 45678
+   Progress: 2,500/10,000 (25%)
+
+✅ Job killed. Status changed to 'paused'.
+   Resume later: /specweave:jobs --resume abc12345
+```
+
+**Implementation**:
+```typescript
+import { killJob } from '../../../src/core/background/job-launcher.js';
+
+const success = killJob(projectPath, jobId);
+if (success) {
+  console.log('Job killed successfully');
+}
 ```
 
 ### Resume Paused Job
 
-When a job is paused (rate limited, user requested), resume it:
+When a job is paused (rate limited, killed, or user requested), resume it:
 
 ```
 /specweave:jobs --resume def67890
@@ -101,8 +189,24 @@ When a job is paused (rate limited, user requested), resume it:
    Provider: GitHub
    Resuming from: item 1,234 of 10,000
 
-⏳ Import in progress...
-   [1,234/10,000] 12% → PROJ-1234
+⏳ Spawning background worker...
+   New PID: 45679
+
+✅ Job resumed in background.
+   Check progress: /specweave:jobs --follow def67890
+```
+
+**Implementation**:
+```typescript
+import { launchImportJob } from '../../../src/core/background/job-launcher.js';
+
+// Re-launch the worker with existing config
+const result = await launchImportJob({
+  type: 'import-issues',
+  projectPath,
+  coordinatorConfig: existingConfig,
+  estimatedTotal: job.progress.total
+});
 ```
 
 ---

package/plugins/specweave-ado/commands/specweave-ado-clone-repos.md

@@ -0,0 +1,379 @@
+---
+name: specweave-ado:clone-repos
+description: Clone Azure DevOps repositories to local workspace. Use after init if cloning was skipped, or to add repos later.
+---
+
+# Clone Azure DevOps Repositories Command
+
+You are an Azure DevOps repository cloning expert. Help users clone repositories from ADO projects to their local workspace.
+
+## Purpose
+
+This command clones Azure DevOps repositories **after** initial SpecWeave setup (`specweave init`). Use when:
+- User skipped cloning during init
+- Adding repositories from additional projects
+- Re-cloning after cleanup
+- Selective cloning with pattern filtering
+
+## Command Syntax
+
+```bash
+# Interactive mode (prompts for everything)
+/specweave-ado:clone-repos
+
+# With pattern filter
+/specweave-ado:clone-repos --pattern "sw-*"
+
+# Regex pattern
+/specweave-ado:clone-repos --pattern "regex:^api-.*$"
+
+# Specific project only
+/specweave-ado:clone-repos --project "MyProject"
+
+# Dry-run (preview only)
+/specweave-ado:clone-repos --dry-run
+```
+
+## Your Task
+
+When the user runs this command:
+
+### Step 1: Check Prerequisites
+
+```typescript
+import { readEnvFile, parseEnvFile } from '../../../src/utils/env-file.js';
+import chalk from 'chalk';
+
+const projectPath = process.cwd();
+const envContent = readEnvFile(projectPath);
+
+if (!envContent) {
+  console.log(chalk.red('❌ No .env file found. Run `specweave init` first.'));
+  return;
+}
+
+const parsed = parseEnvFile(envContent);
+
+if (!parsed.AZURE_DEVOPS_PAT || !parsed.AZURE_DEVOPS_ORG) {
+  console.log(chalk.red('❌ Missing Azure DevOps credentials.'));
+  console.log(chalk.gray('   Run `specweave init` with Azure DevOps provider.'));
+  return;
+}
+
+const org = parsed.AZURE_DEVOPS_ORG;
+const pat = parsed.AZURE_DEVOPS_PAT;
+
+console.log(chalk.blue('\n📦 ADO Repository Cloning\n'));
+console.log(chalk.gray(`   Organization: ${org}`));
+```
+
+### Step 2: Get Project Selection
+
+```typescript
+import { AzureDevOpsProvider } from '../../../src/core/repo-structure/providers/azure-devops-provider.js';
+
+const provider = new AzureDevOpsProvider();
+
+// If project specified via CLI, use it
+let selectedProjects: string[] = [];
+
+if (args.project) {
+  selectedProjects = [args.project];
+  console.log(chalk.gray(`   Project: ${args.project} (from CLI)`));
+} else {
+  // Fetch available projects
+  console.log(chalk.gray('\n   Fetching projects...'));
+
+  const response = await fetch(
+    `https://dev.azure.com/${org}/_apis/projects?api-version=7.0`,
+    {
+      headers: {
+        'Authorization': `Basic ${Buffer.from(':' + pat).toString('base64')}`,
+        'Accept': 'application/json'
+      }
+    }
+  );
+
+  if (!response.ok) {
+    console.log(chalk.red(`❌ Failed to fetch projects: ${response.status}`));
+    return;
+  }
+
+  const data = await response.json();
+  const projects = data.value || [];
+
+  if (projects.length === 0) {
+    console.log(chalk.yellow('⚠️  No projects found in organization.'));
+    return;
+  }
+
+  // Prompt for project selection
+  const { checkbox } = await import('@inquirer/prompts');
+
+  selectedProjects = await checkbox({
+    message: 'Select project(s) to clone repositories from:',
+    choices: projects.map(p => ({ name: p.name, value: p.name })),
+    required: true
+  });
+
+  console.log(chalk.green(`   ✓ ${selectedProjects.length} project(s) selected`));
+}
+```
+
+### Step 3: Fetch Repositories
+
+```typescript
+const allRepos = [];
+
+for (const project of selectedProjects) {
+  console.log(chalk.gray(`\n   Fetching repos from ${project}...`));
+
+  try {
+    const repos = await provider.listRepositories(org, project, pat);
+    const reposWithProject = repos.map(r => ({ ...r, project }));
+    allRepos.push(...reposWithProject);
+    console.log(chalk.green(`   ✓ Found ${repos.length} repositories`));
+  } catch (error) {
+    console.log(chalk.yellow(`   ⚠️ Failed: ${error.message}`));
+  }
+}
+
+if (allRepos.length === 0) {
+  console.log(chalk.yellow('\n⚠️  No repositories found.'));
+  return;
+}
+
+console.log(chalk.blue(`\n📋 Total: ${allRepos.length} repositories available\n`));
+```
+
+### Step 4: Apply Pattern Filter
+
+```typescript
+import { filterRepositoriesByPattern } from '../../../src/cli/helpers/selection-strategy.js';
+
+let filteredRepos = allRepos;
+let patternDescription = 'all';
+
+if (args.pattern) {
+  // Determine pattern type
+  const isRegex = args.pattern.startsWith('regex:');
+  const pattern = isRegex ? args.pattern.slice(6) : args.pattern;
+
+  const clonePattern = {
+    strategy: isRegex ? 'pattern-regex' : 'pattern-glob',
+    pattern: pattern,
+    isRegex
+  };
+
+  filteredRepos = filterRepositoriesByPattern(allRepos, clonePattern);
+  patternDescription = `matching "${pattern}"`;
+
+  console.log(chalk.gray(`   Pattern: ${args.pattern}`));
+  console.log(chalk.gray(`   Matched: ${filteredRepos.length} of ${allRepos.length} repos\n`));
+} else {
+  // Prompt for pattern selection
+  const { select, input } = await import('@inquirer/prompts');
+
+  const strategy = await select({
+    message: 'How do you want to select repositories?',
+    choices: [
+      { name: 'All - Clone all repositories', value: 'all' },
+      { name: 'Pattern (glob) - e.g., "sw-*", "*-backend"', value: 'pattern-glob' },
+      { name: 'Pattern (regex) - e.g., "^api-.*$"', value: 'pattern-regex' }
+    ]
+  });
+
+  if (strategy !== 'all') {
+    const pattern = await input({
+      message: 'Enter pattern:',
+      validate: v => v.trim() ? true : 'Pattern required'
+    });
+
+    const clonePattern = {
+      strategy,
+      pattern: pattern.trim(),
+      isRegex: strategy === 'pattern-regex'
+    };
+
+    filteredRepos = filterRepositoriesByPattern(allRepos, clonePattern);
+    patternDescription = `matching "${pattern.trim()}"`;
+  }
+}
+
+if (filteredRepos.length === 0) {
+  console.log(chalk.yellow(`⚠️  No repositories ${patternDescription}.`));
+  return;
+}
+```
+
+### Step 5: Preview and Confirm
+
+```typescript
+console.log(chalk.blue(`\n📦 Repositories to clone (${filteredRepos.length}):\n`));
+
+// Show preview (max 20)
+filteredRepos.slice(0, 20).forEach(repo => {
+  console.log(chalk.gray(`   • ${repo.name} (${repo.project})`));
+});
+
+if (filteredRepos.length > 20) {
+  console.log(chalk.gray(`   ... and ${filteredRepos.length - 20} more\n`));
+}
+
+if (args.dryRun) {
+  console.log(chalk.cyan('\n🔎 DRY RUN: No repositories will be cloned.\n'));
+  return;
+}
+
+const { confirm } = await import('@inquirer/prompts');
+
+const confirmed = await confirm({
+  message: `Clone ${filteredRepos.length} repositories to ./repos/?`,
+  default: true
+});
+
+if (!confirmed) {
+  console.log(chalk.gray('\n⏭️  Cloning cancelled.\n'));
+  return;
+}
+```
+
+### Step 6: Start Background Cloning
+
+```typescript
+import { triggerAdoRepoCloning } from '../../../src/cli/helpers/init/ado-repo-cloning.js';
+
+// Build adoProjectSelection
+const adoProjectSelection = {
+  org,
+  pat,
+  projects: selectedProjects
+};
+
+// Build clonePattern
+const clonePatternResult = args.pattern
+  ? {
+      strategy: args.pattern.startsWith('regex:') ? 'pattern-regex' : 'pattern-glob',
+      pattern: args.pattern.startsWith('regex:') ? args.pattern.slice(6) : args.pattern
+    }
+  : { strategy: 'all' };
+
+// Trigger background cloning
+await triggerAdoRepoCloning(projectPath, adoProjectSelection, clonePatternResult);
+```
+
+## Examples
+
+### Example 1: Interactive Clone
+**User**: `/specweave-ado:clone-repos`
+
+**Output**:
+```
+📦 ADO Repository Cloning
+
+   Organization: mycompany
+
+   Fetching projects...
+   ✓ 3 projects found
+
+Select project(s) to clone repositories from:
+> [x] Platform
+  [x] Shared
+  [ ] Legacy
+
+   ✓ 2 project(s) selected
+
+   Fetching repos from Platform...
+   ✓ Found 12 repositories
+   Fetching repos from Shared...
+   ✓ Found 4 repositories
+
+📋 Total: 16 repositories available
+
+How do you want to select repositories?
+> All - Clone all repositories
+
+📦 Repositories to clone (16):
+
+   • sw-frontend (Platform)
+   • sw-backend (Platform)
+   • sw-shared-lib (Shared)
+   ... and 13 more
+
+Clone 16 repositories to ./repos/? (Y/n)
+
+📦 Fetching ADO Repositories
+
+   Fetching from mycompany/Platform...
+   ✓ Found 12 repositories in Platform
+   Fetching from mycompany/Shared...
+   ✓ Found 4 repositories in Shared
+
+🔄 Cloning 16 repositories in background...
+
+   Repositories will be cloned to: ./repos/
+   Job ID: abc12345
+
+   Check progress: /specweave:jobs
+   Resume if interrupted: /specweave:jobs --resume abc12345
+```
+
+### Example 2: Pattern Filter
+**User**: `/specweave-ado:clone-repos --pattern "sw-*"`
+
+**Output**:
+```
+📦 ADO Repository Cloning
+
+   Organization: mycompany
+   Pattern: sw-*
+   Matched: 8 of 16 repos
+
+📦 Repositories to clone (8):
+
+   • sw-frontend (Platform)
+   • sw-backend (Platform)
+   • sw-shared-lib (Shared)
+   ...
+
+Clone 8 repositories to ./repos/? (Y/n)
+```
+
+### Example 3: Dry Run
+**User**: `/specweave-ado:clone-repos --dry-run`
+
+**Output**:
+```
+📦 Repositories to clone (16):
+
+   • sw-frontend (Platform)
+   • sw-backend (Platform)
+   ...
+
+🔎 DRY RUN: No repositories will be cloned.
+```
+
+## Important Notes
+
+- **Background Cloning**: Repositories clone in background (non-blocking)
+- **Progress Tracking**: Check progress with `/specweave:jobs`
+- **Resumable**: Interrupted clones can resume with `/specweave:jobs --resume <id>`
+- **Auth via PAT**: Uses HTTPS clone URLs with PAT authentication
+- **Target Directory**: Repos are cloned to `./repos/<repo-name>/`
+
+## Related Commands
+
+- `/specweave:init` - Initial SpecWeave setup (includes repo cloning option)
+- `/specweave:jobs` - Monitor background jobs
+- `/specweave-ado:import-projects` - Import ADO projects with area path mapping
+
+## Error Handling
+
+- **Missing Credentials**: Prompt to run `specweave init` first
+- **Auth Failures**: Check PAT permissions (vso.code_read scope required)
+- **Clone Failures**: Individual repo failures logged, others continue
+- **Network Errors**: Job pauses, resume with `/specweave:jobs --resume`
+
+---
+
+**Multi-Repo Support**: Perfect for microservices architectures with many ADO repositories.