dslop 1.5.1 → 1.6.0

package/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## v1.6.0
+
+[compare changes](https://github.com/turf-sports/dslop/compare/v1.5.2...v1.6.0)
+
+### 🚀 Enhancements
+
+- Smarter defaults - auto-switch to full scan when no changes ([fb918a8](https://github.com/turf-sports/dslop/commit/fb918a8))
+
+### ❤️ Contributors
+
+- Siddharth Sharma <sharmasiddharthcs@gmail.com>
+
+## v1.5.2
+
+[compare changes](https://github.com/turf-sports/dslop/compare/v1.5.1...v1.5.2)
+
+### 📖 Documentation
+
+- Update README with AST-based detection details ([2bdb4f4](https://github.com/turf-sports/dslop/commit/2bdb4f4))
+
+### ❤️ Contributors
+
+- Siddharth Sharma <sharmasiddharthcs@gmail.com>
+
 ## v1.5.1
 
 [compare changes](https://github.com/turf-sports/dslop/compare/v1.5.0...v1.5.1)

package/README.md

@@ -6,7 +6,7 @@ Find duplicate code in your codebase.
 npx dslop
 ```
 
-By default, checks your branch changes (committed + uncommitted) against the codebase.
+By default, checks your branch changes against the codebase. If no changes found, automatically does a full scan.
 
 ## Install
 
@@ -17,17 +17,18 @@ npm i -g dslop
 ## Usage
 
 ```bash
-dslop                        # check your PR/branch for dupes
-dslop --all                  # scan entire codebase
-dslop ./src -m 6 -s 80       # 6 line min, 80% similarity  
-dslop --all --cross-package  # cross-package dupes (monorepos)
+dslop                        # check PR changes (or full scan if none)
+dslop ./apps/web             # scan apps/web (full if no changes there)
+dslop -c                     # changes only, exit if none found
+dslop --cross-package        # cross-package dupes (monorepos)
 ```
 
 ## Options
 
 | Flag | Description |
 |------|-------------|
-| `-a, --all` | scan entire codebase (default: uncommitted only) |
+| `-a, --all` | force full codebase scan |
+| `-c, --changes` | force changes-only mode (exit if no changes) |
 | `-m, --min-lines` | min lines per block (default: 4) |
 | `-s, --similarity` | similarity threshold 0-100 (default: 70) |
 | `-e, --extensions` | file extensions (default: ts,tsx,js,jsx) |
@@ -36,27 +37,40 @@ dslop --all --cross-package  # cross-package dupes (monorepos)
 
 ## How it works
 
-**Block extraction:** Sliding window over source files. Extracts overlapping blocks at sizes 4, 6, 9, 13... lines. For blocks <10 lines, step=1 (every line). Larger blocks use step=blockSize/2.
+dslop uses two detection methods in parallel:
 
-**Normalization:** Before hashing, code is normalized:
+### 1. AST-based detection (functions/classes)
+
+Parses TypeScript/JavaScript with Babel to extract functions and classes. Normalizes the AST by replacing all identifiers with generic placeholders (`$0`, `$1`, etc.), preserving only the code structure.
+
+**This catches:**
+- Functions with identical logic but different variable names
+- Renamed copies of existing functions
+- Structurally identical classes
+
+Example: `calculateSum(numbers)` and `computeTotal(items)` with the same loop structure will match.
+
+### 2. Text-based detection (code blocks)
+
+Sliding window over source files extracts overlapping blocks at sizes 4, 6, 9, 13... lines. Before hashing, code is normalized:
 - String literals → `"<STRING>"`
 - Numbers → `<NUMBER>`
 - Whitespace collapsed
 - Comments preserved (intentional - comments often indicate copy-paste)
 
-**Matching:** Normalized blocks are hashed. Exact hash matches = exact duplicates. For similar (non-exact) matches, uses character-level similarity on a sample of blocks per hash bucket.
+Exact hash matches = exact duplicates. For similar (non-exact) matches, uses character-level similarity.
 
-**Declaration detection** (`--all` mode): Regex-based extraction of types, interfaces, functions, classes. Compares by name similarity (Levenshtein + word overlap) and content similarity.
+### Smart defaults
 
-**Changed-line filtering** (default mode): Parses `git diff` output to get exact line ranges. Only reports duplicates where your changed lines match code elsewhere.
+1. If you have branch changes → checks those against the codebase
+2. If no changes found → automatically scans the entire target path
+3. Use `-c` to force changes-only mode (useful in CI)
 
 ## Limitations
 
-- **Text-based, not AST:** Doesn't understand code structure. A reformatted function won't match the original. Two semantically identical functions with different variable names won't match.
-- **TypeScript/JavaScript focused:** Default extensions are ts/tsx/js/jsx. Works on any text but tuned for JS-like syntax.
+- **TypeScript/JavaScript only for AST:** AST parsing uses Babel with TS/JSX plugins. Other languages fall back to text-based only.
 - **No cross-language:** Won't detect a Python function duplicated in TypeScript.
-- **Comments affect matching:** Intentional tradeoff. Copy-pasted code often includes comments.
-- **Declaration detection is regex:** Can miss edge cases like multi-line generics or decorators.
+- **Comments affect text matching:** Intentional tradeoff. Copy-pasted code often includes comments.
 - **Minimum 4 lines:** Shorter duplicates ignored to reduce noise. Use `-m 2` for stricter.
 - **Memory:** Loads all blocks in memory. Very large codebases (>1M lines) may be slow.
 

package/dist/index.cjs

@@ -51724,7 +51724,7 @@ async function scanDirectory(targetPath, options, enableAST = true) {
 }
 
 // index.ts
-var VERSION = process.env.npm_package_version || "1.5.1";
+var VERSION = process.env.npm_package_version || "1.6.0";
 function parseDiffOutput(diff, cwd) {
   const changes = /* @__PURE__ */ new Map();
   let currentFile = null;
@@ -51812,13 +51812,15 @@ dslop - Detect Similar/Duplicate Lines Of Programming
 Usage:
   dslop [path] [options]
 
-By default, checks your branch changes (committed + uncommitted) against the codebase.
+By default, checks your branch changes against the codebase.
+If no changes are found, automatically scans the entire path.
 
 Arguments:
   path                  Directory to scan (default: current directory)
 
 Options:
-  -a, --all             Scan entire codebase (not just uncommitted changes)
+  -a, --all             Force full codebase scan
+  -c, --changes         Force changes-only mode (exit if no changes found)
   -m, --min-lines <n>   Minimum block size in lines (default: ${DEFAULT_MIN_LINES})
   -s, --similarity <n>  Minimum similarity threshold 0-100 (default: ${Math.round(DEFAULT_SIMILARITY * 100)})
   -e, --extensions <s>  File extensions to scan (default: ${DEFAULT_EXTENSIONS.join(",")})
@@ -51830,10 +51832,10 @@ Options:
   -v, --version         Show version
 
 Examples:
-  dslop                            Check your PR/branch changes for duplicates
-  dslop --all                      Scan entire codebase
-  dslop ./src -m 6 -s 80           Scan src with 6 line min, 80% similarity
-  dslop --all --cross-package      Cross-package duplicates in entire codebase
+  dslop                            Check your PR changes (or full scan if none)
+  dslop ./apps/web                 Scan apps/web (full scan if no changes there)
+  dslop -c                         Check PR changes only, exit if none
+  dslop --cross-package            Cross-package duplicates
 `);
 }
 function showVersion() {
@@ -51850,6 +51852,7 @@ async function main() {
       normalize: { type: "boolean", default: true },
       "no-normalize": { type: "boolean", default: false },
       all: { type: "boolean", short: "a", default: false },
+      changes: { type: "boolean", short: "c", default: false },
       "cross-package": { type: "boolean", default: false },
       json: { type: "boolean", default: false },
       help: { type: "boolean", short: "h", default: false },
@@ -51871,9 +51874,11 @@ async function main() {
   const extensions = values.extensions.split(",").map((e) => e.trim());
   const ignorePatterns = values.ignore.split(",").map((p) => p.trim());
   const normalize2 = !values["no-normalize"];
-  const scanAll = values.all;
+  const forceAll = values.all;
+  const forceChanges = values.changes;
   const crossPackage = values["cross-package"];
   const jsonOutput = values.json;
+  const hasExplicitPath = positionals.length > 0;
   if (minLines < 2) {
     console.error("Error: --min-lines must be at least 2");
     process.exit(1);
@@ -51888,32 +51893,47 @@ async function main() {
     minLines,
     normalize: normalize2
   };
-  const changedLines = !scanAll ? getChangedLines(targetPath) : null;
-  if (!scanAll && changedLines?.size === 0) {
-    console.log("\nNo changes found. Use --all to scan entire codebase.");
-    process.exit(0);
+  let scanAll = forceAll;
+  let changedLines = null;
+  if (!forceAll) {
+    changedLines = getChangedLines(targetPath);
+    if (changedLines.size === 0) {
+      if (forceChanges) {
+        console.log("\nNo changes found on current branch.");
+        process.exit(0);
+      }
+      scanAll = true;
+      if (!jsonOutput) {
+        console.log(`
+No changes detected${hasExplicitPath ? ` in ${targetPath}` : ""}, defaulting to full scan...`);
+      }
+    }
   }
-  console.log(`
+  if (!jsonOutput) {
+    console.log(`
 Scanning ${targetPath}...`);
-  if (!scanAll && changedLines) {
-    console.log(`  Mode: checking changed lines in ${changedLines.size} files`);
-  } else {
-    console.log(`  Mode: full codebase scan`);
-  }
-  console.log(`  Extensions: ${extensions.join(", ")}`);
-  console.log(`  Min block size: ${minLines} lines`);
-  console.log(`  Similarity threshold: ${Math.round(similarity * 100)}%`);
-  if (crossPackage) {
-    console.log(`  Cross-package: enabled`);
+    if (!scanAll && changedLines && changedLines.size > 0) {
+      console.log(`  Mode: checking changed lines in ${changedLines.size} files`);
+    } else {
+      console.log(`  Mode: full codebase scan`);
+    }
+    console.log(`  Extensions: ${extensions.join(", ")}`);
+    console.log(`  Min block size: ${minLines} lines`);
+    console.log(`  Similarity threshold: ${Math.round(similarity * 100)}%`);
+    if (crossPackage) {
+      console.log(`  Cross-package: enabled`);
+    }
+    console.log();
   }
-  console.log();
   try {
     const startTime = performance.now();
     const { blocks, declarations, astBlocks, fileCount, totalLines } = await scanDirectory(targetPath, scanOptions, true);
     const scanTime = performance.now() - startTime;
-    console.log(`Scanned ${fileCount} files (${totalLines.toLocaleString()} lines) in ${Math.round(scanTime)}ms`);
-    console.log(`Extracted ${blocks.length.toLocaleString()} blocks, ${astBlocks.length.toLocaleString()} functions/classes
+    if (!jsonOutput) {
+      console.log(`Scanned ${fileCount} files (${totalLines.toLocaleString()} lines) in ${Math.round(scanTime)}ms`);
+      console.log(`Extracted ${blocks.length.toLocaleString()} blocks, ${astBlocks.length.toLocaleString()} functions/classes
 `);
+    }
     if (blocks.length === 0 && astBlocks.length === 0) {
       console.log("No code found to analyze.");
       process.exit(0);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dslop",
-  "version": "1.5.1",
+  "version": "1.6.0",
   "description": "Detect Similar/Duplicate Lines Of Programming - Find code duplication in your codebase",
   "type": "module",
   "main": "./dist/index.cjs",