@aiready/pattern-detect 0.1.2 → 0.2.0

package/README.md

@@ -24,6 +24,19 @@ AI coding assistants (GitHub Copilot, ChatGPT, Claude) generate functionally sim
 | Refactoring Suggestions | ❌ Generic | ✅ Specific to pattern type |
 | Output Formats | Text/JSON | Console/JSON/HTML with rich formatting |
 
+#### How We Differ (and When to Use Each)
+
+- **Semantic intent vs exact clones**: jscpd flags copy-paste or near-duplicates; we detect functionally similar code even when structure differs (e.g., two API handlers with different frameworks).
+- **Pattern typing**: We classify duplicates into `api-handler`, `validator`, `utility`, `component`, etc., so teams can prioritize coherent refactors.
+- **AI context cost**: We estimate tokens wasted to quantify impact on AI tools (larger context, higher cost, more confusion).
+- **Refactoring guidance**: We propose targeted fixes per pattern type (e.g., extract middleware or create base handler).
+- **Performance profile**: We use Jaccard similarity with candidate filtering; ~2–3s for ~500 blocks on medium repos.
+
+Recommended workflow:
+- Run **jscpd** in CI to enforce low clone percentage (blocking).
+- Run **@aiready/pattern-detect** to surface semantic duplicates and token waste (advisory), feeding a refactoring backlog.
+- Use both for comprehensive hygiene: jscpd for exact clones; AIReady for intent-level duplication that AI tends to reintroduce.
+
 ## 🚀 Installation
 
 ```bash
@@ -47,6 +60,9 @@ aiready-patterns ./src --similarity 0.9
 # Only look at larger patterns
 aiready-patterns ./src --min-lines 10
 
+# Memory optimization for large codebases
+aiready-patterns ./src --max-blocks 1000 --batch-size 200
+
 # Export to JSON
 aiready-patterns ./src --output json --output-file report.json
 
@@ -54,6 +70,33 @@ aiready-patterns ./src --output json --output-file report.json
 aiready-patterns ./src --output html
 ```
 
+#### Presets (quick copy/paste)
+
+```bash
+# Speed-first (large repos)
+aiready-patterns ./src \
+  --min-shared-tokens 12 \
+  --max-candidates 60 \
+  --max-blocks 300
+
+# Coverage-first (more findings)
+aiready-patterns ./src \
+  --min-shared-tokens 6 \
+  --max-candidates 150
+
+# Short-block focus (helpers/utilities)
+aiready-patterns ./src \
+  --min-lines 5 \
+  --min-shared-tokens 6 \
+  --max-candidates 120 \
+  --exclude "**/test/**"
+
+# Deep dive with streaming (comprehensive detection)
+aiready-patterns ./src \
+  --no-approx \
+  --stream-results
+```
+
 ### Programmatic API
 
 ```typescript
@@ -165,12 +208,38 @@ router.get('/posts/:id', createResourceHandler('Post', database.posts.findOne));
 
 ## ⚙️ Configuration
 
+### Common Options
+
 | Option | Description | Default |
 |--------|-------------|---------|
-| `minSimilarity` | Similarity threshold (0-1) | `0.85` |
+| `minSimilarity` | Similarity threshold (0-1). Default `0.40` (Jaccard). Raise for only obvious duplicates; lower to catch more | `0.40` |
+| `minSimilarity` | Similarity threshold (0-1). Default `0.40` (Jaccard). Raise for only obvious duplicates; lower to catch more | `0.40` |
 | `minLines` | Minimum lines to consider a pattern | `5` |
-| `include` | File patterns to include | `['**/*.ts', '**/*.js']` |
-| `exclude` | File patterns to exclude | `['**/node_modules/**', '**/*.test.*']` |
+| `maxBlocks` | Maximum code blocks to analyze (prevents OOM) | `500` |
+| `include` | File patterns to include | `['**/*.{ts,tsx,js,jsx,py,java}']` |
+| `exclude` | File patterns to exclude | See below |
+
+### Exclude Patterns (Default)
+
+By default, these patterns are excluded:
+```bash
+**/node_modules/**
+**/dist/**
+**/build/**
+**/.git/**
+**/coverage/**
+**/*.min.js
+**/*.bundle.js
+```
+
+Override with `--exclude` flag:
+```bash
+# Exclude test files and generated code
+aiready-patterns ./src --exclude "**/test/**,**/generated/**,**/__snapshots__/**"
+
+# Add to defaults (comma-separated)
+aiready-patterns ./src --exclude "**/node_modules/**,**/dist/**,**/build/**,**/*.spec.ts"
+```
 
 ## 📈 Understanding the Output
 
@@ -205,6 +274,148 @@ Estimated tokens wasted when AI tools process duplicate code:
 4. **Use pattern types**: Prioritize refactoring by category (API handlers → validators → utilities)
 5. **Export reports**: Generate HTML reports for team reviews
 
+## ⚠️ Performance & Memory
+
+### Algorithm Complexity
+
+**Jaccard Similarity**: **O(B × C × T)** where:
+- B = number of blocks
+- C = average candidates per block (~100)  
+- T = average tokens per block (~50)
+- **O(T) per comparison** instead of O(N²)
+- **Default threshold: 0.40** (comprehensive detection including tests and helpers)
+
+### Performance Benchmarks
+
+| Repo Size | Blocks | Analysis Time |
+|-----------|--------|--------------|
+| Small (<100 files) | ~50 | <1s |
+| Medium (100-500 files) | ~500 | ~2s |
+| Large (500+ files) | ~500 (capped) | ~2s |
+
+**Example:** 828 code blocks → limited to 500 → **2.4s** analysis time
+
+### Tuning Options
+
+```bash
+# Default (40% threshold - comprehensive detection)
+aiready-patterns ./src
+
+# Higher threshold for only obvious duplicates
+aiready-patterns ./src --similarity 0.65
+
+# Lower threshold for more potential duplicates
+aiready-patterns ./src --similarity 0.55
+
+# Approximate mode is default (fast, with candidate filtering)
+aiready-patterns ./src
+
+# Exact mode with progress tracking (shows % and ETA)
+aiready-patterns ./src --no-approx --stream-results
+
+# Maximum speed (aggressive filtering)
+aiready-patterns ./src --min-shared-tokens 12 --min-lines 10
+```
+
+## 🎛️ Tuning Playbook
+
+Use these presets to quickly balance precision, recall, and runtime:
+
+- Speed-first (large repos):
+  - `aiready-patterns ./src --min-shared-tokens 12 --max-candidates 60 --max-blocks 300`
+  - Cuts weak candidates early; best for fast, iterative scans.
+
+- Coverage-first (more findings):
+  - `aiready-patterns ./src --min-shared-tokens 6 --max-candidates 150`
+  - Expands candidate pool; expect more results and longer runtime.
+
+- Short-block focus (helpers/utilities):
+  - `aiready-patterns ./src --min-lines 5 --min-shared-tokens 6 --max-candidates 120`
+  - Better recall for small functions; consider `--exclude "**/test/**"` to reduce noise.
+
+### Minimum Lines vs Min Shared Tokens
+
+- `minLines` filters which blocks are extracted; lower values include smaller functions that have fewer tokens overall.
+- Smaller blocks naturally share fewer tokens; to avoid missing true matches when `minLines` is low (≤5–6), consider lowering `minSharedTokens` by 1–2.
+- Recommended pairs:
+  - `minLines 5–6` → `minSharedTokens 6–8` (recall-friendly; watch noise)
+  - `minLines 8–10` → `minSharedTokens 8–10` (precision-first)
+- Default balance: `minLines=5`, `minSharedTokens=8` works well for most repos. Reduce `minSharedTokens` only when you specifically want to catch more short helpers.
+
+**CLI Options:**
+- `--stream-results` - Output duplicates as found (enabled by default)
+- `--no-approx` - Disable approximate mode (slower, O(B²) complexity, use with caution)
+- `--min-lines N` - Filter blocks smaller than N lines (default 5)
+
+### Controlling Analysis Scope
+
+The tool analyzes **all extracted code blocks** by default. Control scope using:
+
+**1. `--min-lines` (primary filter):**
+- Filters blocks during extraction (most efficient)
+- Higher values = focus on substantial functions
+- Lower values = catch smaller utility duplicates
+
+**2. `--no-approx` mode (use with caution):**
+- Disables approximate mode (candidate pre-filtering)
+- O(B²) complexity - compares every block to every other block
+- **Automatic safety limit:** 500K comparisons (~1000 blocks max)
+- Shows warning when used with >500 blocks
+- Approximate mode (default) is recommended for all use cases
+
+**Examples:**
+```bash
+# Focus on substantial functions only
+aiready-patterns ./src --min-lines 15
+
+# Comprehensive scan of all functions (recommended)
+aiready-patterns ./src --min-lines 5
+
+# Quick scan of major duplicates
+aiready-patterns ./src --min-lines 20
+```
+
+**Recommendations by codebase size:**
+
+| Repo Size | Files | Strategy | Expected Time |
+|-----------|-------|----------|---------------|
+| **Small** | <100 | Use defaults | <1s ✅ |
+| **Medium** | 100-500 | Use defaults | 1-5s ✅ |
+| **Large** | 500-1,000 | Use defaults or `--min-lines 10` | 3-10s ✅ |
+| **Very Large** | 1,000-5,000 | `--min-lines 15` or analyze by module | 5-20s ⚠️ |
+| **Super Large** | 5,000+ | **Analyze by module** (see below) | 10-60s per module ⚠️ |
+
+### Analyzing Very Large Repositories
+
+For repos with 1,000+ files, use modular analysis:
+
+```bash
+# Analyze by top-level directory
+for dir in src/*/; do
+  echo "Analyzing $dir"
+  aiready-patterns "$dir" --min-lines 10
+done
+
+# Or focus on specific high-value areas
+aiready-patterns ./src/api --min-lines 10
+aiready-patterns ./src/core --min-lines 10
+aiready-patterns ./src/services --min-lines 10
+
+# For super large repos (5K+ files), increase thresholds
+aiready-patterns ./src/backend --min-lines 20 --similarity 0.50
+```
+
+**Why modular analysis?**
+- Ensures comprehensive coverage (100% of each module)
+- Avoids hitting comparison budget limits
+- Provides focused, actionable results per module
+- Better for CI/CD integration (parallel jobs)
+
+**Progress Indicators:**
+- **Approx mode**: Shows blocks processed + duplicates found
+- **Exact mode**: Shows % complete, ETA, and comparisons processed
+- **Stream mode**: Prints each duplicate immediately when found (enabled by default)
+
 ## 🔧 CI/CD Integration
 
 ### GitHub Actions

package/dist/chunk-AXHGYYYZ.mjs

@@ -0,0 +1,404 @@
+// src/index.ts
+import { scanFiles, readFileContent } from "@aiready/core";
+
+// src/detector.ts
+import { estimateTokens } from "@aiready/core";
+function categorizePattern(code) {
+  const lower = code.toLowerCase();
+  if (lower.includes("request") && lower.includes("response") || lower.includes("router.") || lower.includes("app.get") || lower.includes("app.post") || lower.includes("express") || lower.includes("ctx.body")) {
+    return "api-handler";
+  }
+  if (lower.includes("validate") || lower.includes("schema") || lower.includes("zod") || lower.includes("yup") || lower.includes("if") && lower.includes("throw")) {
+    return "validator";
+  }
+  if (lower.includes("return (") || lower.includes("jsx") || lower.includes("component") || lower.includes("props")) {
+    return "component";
+  }
+  if (lower.includes("class ") || lower.includes("this.")) {
+    return "class-method";
+  }
+  if (lower.includes("return ") && !lower.includes("this") && !lower.includes("new ")) {
+    return "utility";
+  }
+  if (lower.includes("function") || lower.includes("=>")) {
+    return "function";
+  }
+  return "unknown";
+}
+function extractCodeBlocks(content, minLines) {
+  const lines = content.split("\n");
+  const blocks = [];
+  let currentBlock = [];
+  let blockStart = 0;
+  let braceDepth = 0;
+  let inFunction = false;
+  for (let i = 0; i < lines.length; i++) {
+    const line = lines[i];
+    const trimmed = line.trim();
+    if (!inFunction && (trimmed.includes("function ") || trimmed.includes("=>") || trimmed.includes("async ") || /^(export\s+)?(async\s+)?function\s+/.test(trimmed) || /^(export\s+)?const\s+\w+\s*=\s*(async\s*)?\(/.test(trimmed))) {
+      inFunction = true;
+      blockStart = i;
+    }
+    for (const char of line) {
+      if (char === "{") braceDepth++;
+      if (char === "}") braceDepth--;
+    }
+    if (inFunction) {
+      currentBlock.push(line);
+    }
+    if (inFunction && braceDepth === 0 && currentBlock.length >= minLines) {
+      const blockContent = currentBlock.join("\n");
+      const linesOfCode = currentBlock.filter(
+        (l) => l.trim() && !l.trim().startsWith("//")
+      ).length;
+      blocks.push({
+        content: blockContent,
+        startLine: blockStart + 1,
+        endLine: i + 1,
+        patternType: categorizePattern(blockContent),
+        linesOfCode
+      });
+      currentBlock = [];
+      inFunction = false;
+    } else if (inFunction && braceDepth === 0) {
+      currentBlock = [];
+      inFunction = false;
+    }
+  }
+  return blocks;
+}
+function normalizeCode(code) {
+  return code.replace(/\/\/.*$/gm, "").replace(/\/\*[\s\S]*?\*\//g, "").replace(/"[^"]*"/g, '"STR"').replace(/'[^']*'/g, "'STR'").replace(/`[^`]*`/g, "`STR`").replace(/\b\d+\b/g, "NUM").replace(/\s+/g, " ").trim();
+}
+function jaccardSimilarity(tokens1, tokens2) {
+  const set1 = new Set(tokens1);
+  const set2 = new Set(tokens2);
+  let intersection = 0;
+  for (const token of set1) {
+    if (set2.has(token)) intersection++;
+  }
+  const union = set1.size + set2.size - intersection;
+  return union === 0 ? 0 : intersection / union;
+}
+async function detectDuplicatePatterns(files, options) {
+  const {
+    minSimilarity,
+    minLines,
+    maxBlocks = 500,
+    batchSize = 100,
+    approx = true,
+    minSharedTokens = 8,
+    maxCandidatesPerBlock = 100,
+    maxComparisons = 5e4,
+    // Cap at 50K comparisons by default
+    streamResults = false
+  } = options;
+  const duplicates = [];
+  let allBlocks = files.flatMap(
+    (file) => extractCodeBlocks(file.content, minLines).map((block) => ({
+      content: block.content,
+      startLine: block.startLine,
+      endLine: block.endLine,
+      file: file.file,
+      normalized: normalizeCode(block.content),
+      patternType: block.patternType,
+      tokenCost: estimateTokens(block.content),
+      linesOfCode: block.linesOfCode
+    }))
+  );
+  console.log(`Extracted ${allBlocks.length} code blocks for analysis`);
+  if (allBlocks.length > maxBlocks) {
+    console.log(`\u26A0\uFE0F  Limiting to ${maxBlocks} blocks (sorted by size) to prevent memory issues`);
+    console.log(`   Use --max-blocks to increase limit or --min-lines to filter smaller blocks`);
+    allBlocks = allBlocks.sort((a, b) => b.linesOfCode - a.linesOfCode).slice(0, maxBlocks);
+  }
+  const stopwords = /* @__PURE__ */ new Set([
+    "return",
+    "const",
+    "let",
+    "var",
+    "function",
+    "class",
+    "new",
+    "if",
+    "else",
+    "for",
+    "while",
+    "async",
+    "await",
+    "try",
+    "catch",
+    "switch",
+    "case",
+    "default",
+    "import",
+    "export",
+    "from",
+    "true",
+    "false",
+    "null",
+    "undefined",
+    "this"
+  ]);
+  const tokenize = (norm) => norm.split(/[\s(){}\[\];,\.]+/).filter((t) => t && t.length >= 3 && !stopwords.has(t.toLowerCase()));
+  const blockTokens = allBlocks.map((b) => tokenize(b.normalized));
+  const invertedIndex = /* @__PURE__ */ new Map();
+  if (approx) {
+    for (let i = 0; i < blockTokens.length; i++) {
+      for (const tok of blockTokens[i]) {
+        let arr = invertedIndex.get(tok);
+        if (!arr) {
+          arr = [];
+          invertedIndex.set(tok, arr);
+        }
+        arr.push(i);
+      }
+    }
+  }
+  const totalComparisons = approx ? void 0 : allBlocks.length * (allBlocks.length - 1) / 2;
+  if (totalComparisons !== void 0) {
+    console.log(`Processing ${totalComparisons.toLocaleString()} comparisons in batches...`);
+  } else {
+    console.log(`Using approximate candidate selection to reduce comparisons...`);
+  }
+  let comparisonsProcessed = 0;
+  let comparisonsBudgetExhausted = false;
+  const startTime = Date.now();
+  for (let i = 0; i < allBlocks.length; i++) {
+    if (maxComparisons && comparisonsProcessed >= maxComparisons) {
+      comparisonsBudgetExhausted = true;
+      break;
+    }
+    if (i % batchSize === 0 && i > 0) {
+      const elapsed = ((Date.now() - startTime) / 1e3).toFixed(1);
+      const duplicatesFound = duplicates.length;
+      if (totalComparisons !== void 0) {
+        const progress = (comparisonsProcessed / totalComparisons * 100).toFixed(1);
+        const remaining = totalComparisons - comparisonsProcessed;
+        const rate = comparisonsProcessed / parseFloat(elapsed);
+        const eta = remaining > 0 ? (remaining / rate).toFixed(0) : 0;
+        console.log(`   ${progress}% (${comparisonsProcessed.toLocaleString()}/${totalComparisons.toLocaleString()} comparisons, ${elapsed}s elapsed, ~${eta}s remaining, ${duplicatesFound} duplicates)`);
+      } else {
+        console.log(`   Processed ${i.toLocaleString()}/${allBlocks.length} blocks (${elapsed}s elapsed, ${duplicatesFound} duplicates)`);
+      }
+      await new Promise((resolve) => setImmediate(resolve));
+    }
+    const block1 = allBlocks[i];
+    let candidates = null;
+    if (approx) {
+      const counts = /* @__PURE__ */ new Map();
+      for (const tok of blockTokens[i]) {
+        const ids = invertedIndex.get(tok);
+        if (!ids) continue;
+        for (const j of ids) {
+          if (j <= i) continue;
+          if (allBlocks[j].file === block1.file) continue;
+          counts.set(j, (counts.get(j) || 0) + 1);
+        }
+      }
+      candidates = Array.from(counts.entries()).filter(([, shared]) => shared >= minSharedTokens).sort((a, b) => b[1] - a[1]).slice(0, maxCandidatesPerBlock).map(([j, shared]) => ({ j, shared }));
+    }
+    if (approx && candidates) {
+      for (const { j } of candidates) {
+        if (maxComparisons && comparisonsProcessed >= maxComparisons) break;
+        comparisonsProcessed++;
+        const block2 = allBlocks[j];
+        const similarity = jaccardSimilarity(blockTokens[i], blockTokens[j]);
+        if (similarity >= minSimilarity) {
+          const duplicate = {
+            file1: block1.file,
+            file2: block2.file,
+            line1: block1.startLine,
+            line2: block2.startLine,
+            endLine1: block1.endLine,
+            endLine2: block2.endLine,
+            similarity,
+            snippet: block1.content.split("\n").slice(0, 5).join("\n") + "\n...",
+            patternType: block1.patternType,
+            tokenCost: block1.tokenCost + block2.tokenCost,
+            linesOfCode: block1.linesOfCode
+          };
+          duplicates.push(duplicate);
+          if (streamResults) {
+            console.log(`
+   \u2705 Found: ${duplicate.patternType} ${Math.round(similarity * 100)}% similar`);
+            console.log(`      ${duplicate.file1}:${duplicate.line1}-${duplicate.endLine1} \u21D4 ${duplicate.file2}:${duplicate.line2}-${duplicate.endLine2}`);
+            console.log(`      Token cost: ${duplicate.tokenCost.toLocaleString()}`);
+          }
+        }
+      }
+    } else {
+      for (let j = i + 1; j < allBlocks.length; j++) {
+        if (maxComparisons && comparisonsProcessed >= maxComparisons) break;
+        comparisonsProcessed++;
+        const block2 = allBlocks[j];
+        if (block1.file === block2.file) continue;
+        const similarity = jaccardSimilarity(blockTokens[i], blockTokens[j]);
+        if (similarity >= minSimilarity) {
+          const duplicate = {
+            file1: block1.file,
+            file2: block2.file,
+            line1: block1.startLine,
+            line2: block2.startLine,
+            endLine1: block1.endLine,
+            endLine2: block2.endLine,
+            similarity,
+            snippet: block1.content.split("\n").slice(0, 5).join("\n") + "\n...",
+            patternType: block1.patternType,
+            tokenCost: block1.tokenCost + block2.tokenCost,
+            linesOfCode: block1.linesOfCode
+          };
+          duplicates.push(duplicate);
+          if (streamResults) {
+            console.log(`
+   \u2705 Found: ${duplicate.patternType} ${Math.round(similarity * 100)}% similar`);
+            console.log(`      ${duplicate.file1}:${duplicate.line1}-${duplicate.endLine1} \u21D4 ${duplicate.file2}:${duplicate.line2}-${duplicate.endLine2}`);
+            console.log(`      Token cost: ${duplicate.tokenCost.toLocaleString()}`);
+          }
+        }
+      }
+    }
+  }
+  if (comparisonsBudgetExhausted) {
+    console.log(`\u26A0\uFE0F  Comparison budget exhausted (${maxComparisons.toLocaleString()} comparisons). Use --max-comparisons to increase.`);
+  }
+  return duplicates.sort(
+    (a, b) => b.similarity - a.similarity || b.tokenCost - a.tokenCost
+  );
+}
+
+// src/index.ts
+function getRefactoringSuggestion(patternType, similarity) {
+  const baseMessages = {
+    "api-handler": "Extract common middleware or create a base handler class",
+    validator: "Consolidate validation logic into shared schema validators (Zod/Yup)",
+    utility: "Move to a shared utilities file and reuse across modules",
+    "class-method": "Consider inheritance or composition to share behavior",
+    component: "Extract shared logic into a custom hook or HOC",
+    function: "Extract into a shared helper function",
+    unknown: "Extract common logic into a reusable module"
+  };
+  const urgency = similarity > 0.95 ? " (CRITICAL: Nearly identical code)" : similarity > 0.9 ? " (HIGH: Very similar, refactor soon)" : "";
+  return baseMessages[patternType] + urgency;
+}
+async function analyzePatterns(options) {
+  const {
+    minSimilarity = 0.4,
+    // Jaccard similarity default (40% threshold)
+    minLines = 5,
+    maxBlocks = 500,
+    batchSize = 100,
+    approx = true,
+    minSharedTokens = 8,
+    maxCandidatesPerBlock = 100,
+    maxComparisons = 5e4,
+    streamResults = false,
+    ...scanOptions
+  } = options;
+  const files = await scanFiles(scanOptions);
+  const results = [];
+  const fileContents = await Promise.all(
+    files.map(async (file) => ({
+      file,
+      content: await readFileContent(file)
+    }))
+  );
+  const duplicates = await detectDuplicatePatterns(fileContents, {
+    minSimilarity,
+    minLines,
+    maxBlocks,
+    batchSize,
+    approx,
+    minSharedTokens,
+    maxCandidatesPerBlock,
+    maxComparisons,
+    streamResults
+  });
+  for (const file of files) {
+    const fileDuplicates = duplicates.filter(
+      (dup) => dup.file1 === file || dup.file2 === file
+    );
+    const issues = fileDuplicates.map((dup) => {
+      const otherFile = dup.file1 === file ? dup.file2 : dup.file1;
+      const severity = dup.similarity > 0.95 ? "critical" : dup.similarity > 0.9 ? "major" : "minor";
+      return {
+        type: "duplicate-pattern",
+        severity,
+        message: `${dup.patternType} pattern ${Math.round(dup.similarity * 100)}% similar to ${otherFile} (${dup.tokenCost} tokens wasted)`,
+        location: {
+          file,
+          line: dup.file1 === file ? dup.line1 : dup.line2
+        },
+        suggestion: getRefactoringSuggestion(dup.patternType, dup.similarity)
+      };
+    });
+    const totalTokenCost = fileDuplicates.reduce(
+      (sum, dup) => sum + dup.tokenCost,
+      0
+    );
+    results.push({
+      fileName: file,
+      issues,
+      metrics: {
+        tokenCost: totalTokenCost,
+        consistencyScore: Math.max(0, 1 - fileDuplicates.length * 0.1)
+      }
+    });
+  }
+  return results;
+}
+function generateSummary(results) {
+  const allIssues = results.flatMap((r) => r.issues);
+  const totalTokenCost = results.reduce(
+    (sum, r) => sum + (r.metrics.tokenCost || 0),
+    0
+  );
+  const patternsByType = {
+    "api-handler": 0,
+    validator: 0,
+    utility: 0,
+    "class-method": 0,
+    component: 0,
+    function: 0,
+    unknown: 0
+  };
+  allIssues.forEach((issue) => {
+    const match = issue.message.match(/^(\S+(?:-\S+)*) pattern/);
+    if (match) {
+      const type = match[1];
+      patternsByType[type] = (patternsByType[type] || 0) + 1;
+    }
+  });
+  const topDuplicates = allIssues.slice(0, 10).map((issue) => {
+    const similarityMatch = issue.message.match(/(\d+)% similar/);
+    const tokenMatch = issue.message.match(/\((\d+) tokens/);
+    const typeMatch = issue.message.match(/^(\S+(?:-\S+)*) pattern/);
+    const fileMatch = issue.message.match(/similar to (.+?) \(/);
+    return {
+      file1: issue.location.file,
+      file2: fileMatch?.[1] || "unknown",
+      line1: issue.location.line,
+      line2: 0,
+      // Not available from Issue
+      endLine1: 0,
+      // Not available from Issue
+      endLine2: 0,
+      // Not available from Issue
+      similarity: similarityMatch ? parseInt(similarityMatch[1]) / 100 : 0,
+      patternType: typeMatch?.[1] || "unknown",
+      tokenCost: tokenMatch ? parseInt(tokenMatch[1]) : 0
+    };
+  });
+  return {
+    totalPatterns: allIssues.length,
+    totalTokenCost,
+    patternsByType,
+    topDuplicates
+  };
+}
+
+export {
+  detectDuplicatePatterns,
+  analyzePatterns,
+  generateSummary
+};