unslop 0.1.5 → 0.1.7

package/README.md

@@ -2,8 +2,6 @@
 
 Standalone CLI tool for detecting duplicated code, dead code, inlined utilities, and semantic anti-patterns in AI-generated codebases. No AI/LLM in the detection pipeline — deterministic analysis only.
 
-Like `react-doctor` but for de-slopping entire codebases.
-
 ## Install
 
 ```bash
@@ -55,36 +53,160 @@ Use `--changed-only` to focus on uncommitted files and prioritize reuse against
 
 ## What It Detects
 
-| Category | Engine | Reliability |
-|----------|--------|:-----------:|
-| Identical constants across files | Clone | 99% |
-| Copy-pasted functions (same names) | Clone | 99% |
-| Copy-pasted functions (renamed params) | Clone | 95% |
-| Reformatted JSX components | Clone | 90% |
-| Similar functions (small edits) | Clone | 80% |
-| Equivalent regex patterns | Clone | 95% |
-| Cross-package export matches | Clone | 99% |
-| `a>b?a:b` → `Math.max` | Oxlint | 99% |
-| Dead branches | Oxlint | 90% |
-| Unreachable code | Oxlint | 99% |
-| Inlined utilities | Oxlint | 95% |
-| Dead exports | Clone | 95% |
-| Complexity budget breaches | Practices | 90% |
-| Boundary violations (`packages -> apps`, cross-app deps, cycles) | Practices | 90-98% |
-| Ignored Go errors | Practices | 93% |
-| Go context leaks | Practices | 87% |
-| React broad barrel re-exports | Practices | 78-86% |
+### Quick Reference
+
+| Category                               | Engine    | Reliability |
+| -------------------------------------- | --------- | :---------: |
+| Identical constants across files       | Clone     |     99%     |
+| Copy-pasted functions (same names)     | Clone     |     99%     |
+| Copy-pasted functions (renamed params) | Clone     |     95%     |
+| Reformatted JSX components             | Clone     |     90%     |
+| Similar functions (small edits)        | Clone     |     80%     |
+| Equivalent regex patterns              | Clone     |     95%     |
+| Cross-package export matches           | Clone     |     99%     |
+| `a>b?a:b` → `Math.max`                 | Oxlint    |     99%     |
+| Dead branches                          | Oxlint    |     90%     |
+| Unreachable code                       | Oxlint    |     99%     |
+| Inlined utilities                      | Oxlint    |     95%     |
+| Dead exports                           | Clone     |     95%     |
+| Complexity budget breaches             | Practices |     90%     |
+
+## Rules Reference
+
+10 rules across 3 engines. Use `unslop --list-rules` to see defaults for your config.
+
+---
+
+### Engine 1: Clone Detection (3 rules)
+
+Tree-sitter parses source into a CST, then a language plugin normalizes it to a language-agnostic tree with alpha-renamed identifiers (`a`, `b`, `c` instead of real names). All three rules operate on these normalized trees.
+
+#### `exact-clone` — Tier A, Error
+
+SHA-256 fingerprint of the entire normalized tree. Two fragments with identical hashes are exact clones.
+
+- **Algorithm**: deterministic S-expression serialization `(Kind:Label child1 child2 ...)` → SHA-256. O(n) grouping by hash.
+- **Minimum size**: 50 tokens, 8 nodes.
+- **Filters**: same-file duplicates, import-linked pairs, rule boilerplate scaffolding (<30 line span in rules scaffold paths), repeated framework idioms (3+ occurrences with diverse names across 2+ files).
+
+#### `near-clone` — Tier A, Warning
+
+Finds near-miss duplicates via suffix tree / LCS analysis on linearized token sequences.
+
+- **Linearization**: pre-order traversal of normalized tree → flat token sequence with `^` sentinel tokens marking end-of-children.
+- **Bucketing**: logarithmic buckets by token count (20–39 → bucket 0, 40–79 → bucket 1, etc.) to avoid O(n²). Each sequence placed in adjacent bucket +1 for cross-boundary matches.
+- **Algorithm**: rolling DP longest common substring (two rows, O(n) space), capped at **220,000 DP cells** per pair. Falls back to bounded longest common subsequence for edited clones with insertions/deletions.
+- **Threshold**: **80% similarity** (`matchLen / max(len(A), len(B))`).
+- **Limits**: max 10,000 comparisons, min 50 tokens.
+- **Post-processing**: deterministic one-to-one matching per file pair — sorts by similarity descending, prefers same-name pairs, greedy assignment (each fragment matched at most once per file pair). Excludes declarative clones (different names + no control flow or ≤3 statements).
+
+#### `structural-similarity` — Tier A, Warning
+
+PQ-gram tree profiling for structural similarity between same-kind fragments.
+
+- **Algorithm**: PQ-gram with **p=2** ancestors, **q=3** siblings. Each gram = stem of p ancestor labels + base of q sibling labels. Similarity via **Sørensen–Dice coefficient**: `2 * |intersection| / (|A| + |B|)`.
+- **Eligibility**: same fragment Kind only (function-to-function, class-to-class), different files, min 8 nodes, size ratio ≤ 1.8×.
+- **Thresholds** (variable by file type):
+  - Both production files: **70%**
+  - One test + one production: **86%**
+  - Both test files: **90%**
+- **Limits**: max 50,000 comparisons. Profiles cached per fragment.
+
+---
+
+### Engine 2: Linter Integration (5 rules)
+
+Wraps external linters (e.g. oxlint) run as subprocesses with configurable timeouts. Parses JSON output into findings. Non-fatal — a missing linter produces a warning, not an error.
+
+#### `inlined-utility` — Tier A, Warning
+
+Hand-rolled code that could use a standard library or utility call (e.g. `a > b ? a : b` → `Math.max`).
+
+#### `dead-code` — Tier A, Error, Gateable
+
+Unreachable code blocks that can never execute.
+
+#### `dead-branch` — Tier A, Warning
+
+Conditional branches (if/else, ternary) that can never be taken.
+
+#### `duplication` — Tier A, Warning
+
+Duplicate imports, exports, or repeated patterns detected by the linter.
+
+#### `dead-export` — Tier A, Warning
+
+Exported symbols that nothing imports.
+
+---
+
+### Engine 3: Best Practices (1 rule)
+
+In-process deterministic checks — pattern matching and metric computation. No external dependencies.
+
+#### `complexity-budget` — Tier A, Warning
+
+Flags functions that exceed thresholds on **2+ of 3 metrics** simultaneously.
+
+| Metric                                 | Strict | Balanced (default) | Lenient |
+| -------------------------------------- | -----: | -----------------: | ------: |
+| Decision count (if/loop/switch/select) |    ≥10 |                ≥12 |     ≥14 |
+| Max nesting depth                      |     ≥3 |                 ≥4 |      ≥5 |
+| Source lines                           |    ≥75 |                ≥90 |    ≥110 |
+
+- Functions under 20 lines are always excluded.
+- **Headroom** for special paths: tooling/infrastructure paths get +20 decisions, +3 nesting, +80 lines. Script functions (`/scripts/`, `.mjs`/`.cjs`, `main`/`walk`) get +14/+2/+40. Orchestration functions (`/cmd/`, `/routes/`, `/handlers/`, names starting with `handle`/`route`/`serve`) get +2/+1/+25.
+- Also fires on **single-metric extreme outliers**: decisions ≥ threshold+8, nesting ≥ threshold+2, or lines ≥ threshold+40 (when some complexity is also present).
+
+---
+
+### Slop Score
+
+A 0–100 weighted composite of findings and structural smell metrics.
+
+**Category weights** (base, before profile multiplier):
+
+| Category              | Weight | Cap |
+| --------------------- | -----: | --: |
+| exact-clone           |      9 |  18 |
+| near-clone            |      6 |  18 |
+| structural-similarity |      5 |  18 |
+| dead-code             |      5 |  18 |
+| complexity-budget     |      5 |  14 |
+| inlined-utility       |      4 |  18 |
+| duplication           |      4 |  18 |
+| dead-export           |      4 |  18 |
+| dead-branch           |      3 |  18 |
+
+**Profile multipliers**: strict = 1.2×, balanced = 1.0×, lenient = 0.8×.
+
+**Per-finding formula**: `points = weight × profileMultiplier × (0.6 + 0.4 × confidence) × (1.0 + min(0.5, 0.1 × (locations - 1)))`. Category sums are capped, then total finding points capped at 70.
+
+**Smell metrics** (up to 30 points):
+
+- **wrapper-function-density** (45% weight) — single-statement wrapper functions as a ratio of total functions.
+- **trivial-declaration-density** (35% weight) — declarations with ≤6 nodes and no control flow.
+- **reused-fragment-names** (20% weight) — function/class/variable names appearing in 3+ distinct files.
+
+**Score bands**:
+
+|  Range | Band     |
+| -----: | -------- |
+|   0–14 | Minimal  |
+|  15–34 | Low      |
+|  35–59 | Moderate |
+|  60–79 | High     |
+| 80–100 | Severe   |
 
 ## Architecture
 
-Four artifact engines run in parallel and are evaluated through a rule catalog:
+Three artifact engines run in parallel and are evaluated through a rule catalog:
 
 1. **Clone Detection** (in-process) — tree-sitter parsing → pre-normalization (cached) → alpha-renaming → hash/suffix-tree/PQ-gram comparison
 2. **Semantic Analysis** (subprocess) — external linter diagnostics
-3. **Best Practices** (in-process) — deterministic clean-code and architecture checks
-4. **Repo Policy** (in-process) — repository policy signals
+3. **Best Practices** (in-process) — deterministic clean-code checks
 
-Each rule is now a first-class catalog entry with defaults and per-rule controls.
+Each rule is a first-class catalog entry with defaults and per-rule controls.
 
 Built on Semgrep's model: **tree-sitter CST → language-specific normalizer → generic normalized tree → language-agnostic analysis**.
 
@@ -149,8 +271,6 @@ rules:
   exact-clone:
     severity: error
     gateable: true
-  barrel-import:
-    enabled: false
   complexity-budget:
     paths:
       include: ["apps/**"]
@@ -158,7 +278,7 @@ rules:
 
 gates:
   max_score: 35
-  fail_on_rules: ["exact-clone", "boundary-violation-cycle"]
+  fail_on_rules: ["exact-clone"]
   fail_on_tiers: ["A"]
 
 slop_score:
@@ -166,42 +286,6 @@ slop_score:
   top_contributors: 5
 ```
 
-See `/Users/danieledeling/projects/go/unslop/docs/config-v2.md` and `/Users/danieledeling/projects/go/unslop/docs/rules.md` for full details.
-
-## v2 Breaking Change
-
-Configuration is now strict v2:
-
-- `version: 2` is required when using a config file
-- unknown top-level keys fail parsing
-- unknown non-legacy rule IDs fail parsing
-- removed legacy rule IDs are silently ignored in config/CLI rule selectors
-- unknown fields under `rules.<id>` fail parsing
-- there is no v1 compatibility bridge
-
-Example terminal section (`--verbose`):
-
-```text
-── high-priority (2) ──
-
-  [exact-clone] Exact duplicate: parseConfig
-    ./src/a.ts:10
-    ./src/b.ts:12
-
-── low-priority (1) ──
-
-  [exact-clone] Exact duplicate: makeTempDir
-    ./src/foo.test.ts:5
-    ./src/bar.test.ts:9
-
-── slop-score ──
-
-  Slop Score: 42/100 (Moderate)
-  Top contributors:
-    exact-clone: +18.4 (3 findings)
-    wrapper-function-density: +8.7 (31% of functions)
-```
-
 ## Development
 
 ```bash

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "unslop",
-  "version": "0.1.5",
+  "version": "0.1.7",
   "description": "Detect duplicated code, dead code, and anti-patterns in AI-generated codebases",
   "bin": {
     "unslop": "bin.js"
@@ -9,11 +9,11 @@
     "bin.js"
   ],
   "optionalDependencies": {
-    "@unslop/darwin-arm64": "0.1.5",
-    "@unslop/darwin-x64": "0.1.5",
-    "@unslop/linux-arm64": "0.1.5",
-    "@unslop/linux-x64": "0.1.5",
-    "@unslop/win32-x64": "0.1.5"
+    "@unslop/darwin-arm64": "0.1.7",
+    "@unslop/darwin-x64": "0.1.7",
+    "@unslop/linux-arm64": "0.1.7",
+    "@unslop/linux-x64": "0.1.7",
+    "@unslop/win32-x64": "0.1.7"
   },
   "license": "Apache-2.0",
   "repository": {