@aiready/pattern-detect 0.11.29 β 0.11.30
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +32 -274
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,8 +1,13 @@
|
|
|
1
1
|
# @aiready/pattern-detect
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> AIReady Spoke: Semantic duplicate pattern detection for AI-generated code.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
[](https://npmjs.com/package/@aiready/pattern-detect)
|
|
6
|
+
[](https://opensource.org/licenses/MIT)
|
|
7
|
+
|
|
8
|
+
## Overview
|
|
9
|
+
|
|
10
|
+
AI models often suffer from "semantic confusion" when multiple variants of the same logic exist in the codebase. The **Pattern Detect** analyzer helps you identify these duplicates and consolidate them into reusable components.
|
|
6
11
|
|
|
7
12
|
## ποΈ Architecture
|
|
8
13
|
|
|
@@ -10,289 +15,42 @@ Finds semantically similar but syntactically different code patterns that waste
|
|
|
10
15
|
π― USER
|
|
11
16
|
β
|
|
12
17
|
βΌ
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
ββββββββββ βββββ¬βββββ
|
|
23
|
-
β β
|
|
24
|
-
β βββββββββββββββββββββββΌββββββββββββββββββββββ
|
|
25
|
-
β βΌ βΌ βΌ
|
|
26
|
-
β ββββββββββ ββββββββββ ββββββββββ
|
|
27
|
-
β βπ PAT- β βπ¦ CON- β βπ§ CON- β
|
|
28
|
-
β βTERN β βTEXT β βSISTENCYβ
|
|
29
|
-
β βDETECT β βANALYZERβ β β
|
|
30
|
-
β β β β β β β
|
|
31
|
-
β ββ
Readyβ ββ
Readyβ ββ
Readyβ
|
|
32
|
-
β ββββββββββ ββββββββββ ββββββββββ
|
|
33
|
-
β β β β
|
|
34
|
-
β β β YOU ARE HERE βββ β
|
|
35
|
-
β β β
|
|
36
|
-
βββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββββββ
|
|
37
|
-
β
|
|
38
|
-
βΌ
|
|
39
|
-
π’ HUB (@aiready/core)
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
## π Language Support
|
|
43
|
-
|
|
44
|
-
**Currently Supported (64% market coverage):**
|
|
45
|
-
- β
**TypeScript** (`.ts`, `.tsx`) - AST-based pattern extraction
|
|
46
|
-
- β
**JavaScript** (`.js`, `.jsx`) - AST-based pattern extraction
|
|
47
|
-
- β
**Python** (`.py`) - Function/class pattern extraction, similarity scoring
|
|
48
|
-
|
|
49
|
-
**Roadmap:**
|
|
50
|
-
- π **Java** (Q3 2026) - Method/class patterns, Spring annotations
|
|
51
|
-
- π **Go** (Q4 2026) - Function patterns, interface implementations
|
|
52
|
-
- π **Rust** (Q4 2026) - Function/trait patterns, macro detection
|
|
53
|
-
- π **C#** (Q1 2027) - Method/class patterns, LINQ queries
|
|
54
|
-
|
|
55
|
-
## π Quick Start
|
|
56
|
-
|
|
57
|
-
**Zero config, works out of the box:**
|
|
58
|
-
|
|
59
|
-
```bash
|
|
60
|
-
# Run without installation (recommended)
|
|
61
|
-
npx @aiready/pattern-detect ./src
|
|
62
|
-
|
|
63
|
-
# Or use the unified CLI (includes all AIReady tools)
|
|
64
|
-
npx @aiready/cli scan ./src
|
|
65
|
-
|
|
66
|
-
# Or install globally for simpler command and faster runs
|
|
67
|
-
npm install -g @aiready/pattern-detect
|
|
68
|
-
aiready-patterns ./src
|
|
69
|
-
```
|
|
70
|
-
|
|
71
|
-
### π― Input & Output
|
|
72
|
-
|
|
73
|
-
**Input:** Path to your source code directory
|
|
74
|
-
```bash
|
|
75
|
-
aiready-patterns ./src
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
**Output:** Terminal report + optional JSON file (saved to `.aiready/` directory)
|
|
79
|
-
```
|
|
80
|
-
π Duplicate Pattern Analysis
|
|
81
|
-
βββββββββββββββββββββββββββββ
|
|
82
|
-
π Files analyzed: 47
|
|
83
|
-
β οΈ Duplicate patterns: 12 files with 23 issues
|
|
84
|
-
π° Wasted tokens: 8,450
|
|
85
|
-
|
|
86
|
-
CRITICAL (6 files)
|
|
87
|
-
src/handlers/users.ts - 4 duplicates (1,200 tokens)
|
|
88
|
-
src/handlers/posts.ts - 3 duplicates (950 tokens)
|
|
89
|
-
```
|
|
90
|
-
|
|
91
|
-
### β¨ Smart Defaults (Zero Config)
|
|
92
|
-
|
|
93
|
-
- β
**Auto-excludes** test files (`**/*.test.*`, `**/*.spec.*`, `**/__tests__/**`)
|
|
94
|
-
- β
**Auto-excludes** build outputs (`dist/`, `build/`, `.next/`)
|
|
95
|
-
- β
**Auto-excludes** dependencies (`node_modules/`)
|
|
96
|
-
- β
**Adaptive threshold**: Adjusts similarity detection based on codebase size
|
|
97
|
-
- β
**Pattern classification**: Automatically categorizes duplicates (API handlers, validators, etc.)
|
|
98
|
-
|
|
99
|
-
> Override defaults with `--include-tests` or `--exclude <patterns>` as needed
|
|
100
|
-
|
|
101
|
-
## π― What It Does
|
|
102
|
-
|
|
103
|
-
AI tools generate similar code in different ways because they lack awareness of your codebase patterns. This tool:
|
|
104
|
-
|
|
105
|
-
- **Semantic detection**: Finds functionally similar code (not just copy-paste) using Jaccard similarity on AST tokens
|
|
106
|
-
- **Pattern classification**: Groups duplicates by type (API handlers, validators, utilities, etc.)
|
|
107
|
-
- **Token cost analysis**: Shows wasted AI context budget
|
|
108
|
-
- **Refactoring guidance**: Suggests specific fixes per pattern type
|
|
109
|
-
|
|
110
|
-
### How It Works
|
|
111
|
-
|
|
112
|
-
The tool uses **Jaccard similarity** to compare code semantically:
|
|
113
|
-
1. Parses TypeScript/JavaScript files into Abstract Syntax Trees (AST)
|
|
114
|
-
2. Extracts semantic tokens (identifiers, operators, keywords) from each function
|
|
115
|
-
3. Calculates Jaccard similarity between token sets: `|A β© B| / |A βͺ B|`
|
|
116
|
-
4. Groups similar functions above the similarity threshold
|
|
117
|
-
|
|
118
|
-
This approach catches duplicates even when variable names or minor logic differs.
|
|
119
|
-
|
|
120
|
-
### Example Output
|
|
18
|
+
ποΈ @aiready/cli (orchestrator)
|
|
19
|
+
β β β β β β β β β
|
|
20
|
+
βΌ βΌ βΌ βΌ βΌ βΌ βΌ βΌ βΌ
|
|
21
|
+
[PAT] [CTX] [CON] [AMP] [DEP] [DOC] [SIG] [AGT] [TST]
|
|
22
|
+
β β β β β β β β β
|
|
23
|
+
βββββββ΄ββββββ΄ββββββ΄ββββββ΄ββββββ΄ββββββ΄ββββββ΄ββββββ
|
|
24
|
+
β
|
|
25
|
+
βΌ
|
|
26
|
+
π’ @aiready/core
|
|
121
27
|
|
|
28
|
+
Legend:
|
|
29
|
+
PAT = pattern-detect β
CTX = context-analyzer
|
|
30
|
+
CON = consistency AMP = change-amplification
|
|
31
|
+
DEP = deps-health DOC = doc-drift
|
|
32
|
+
SIG = ai-signal-clarity AGT = agent-grounding
|
|
33
|
+
TST = testability β
= YOU ARE HERE
|
|
122
34
|
```
|
|
123
|
-
π Files analyzed: 47
|
|
124
|
-
β Duplicate patterns found: 23
|
|
125
|
-
π° Token cost (wasted): 8,450
|
|
126
35
|
|
|
127
|
-
|
|
128
|
-
β validator 8 patterns
|
|
129
|
-
π§ utility 3 patterns
|
|
36
|
+
## Features
|
|
130
37
|
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
β Create generic handler function
|
|
135
|
-
```
|
|
38
|
+
- **Semantic Duplication**: Detects code blocks that perform the same task with different syntax.
|
|
39
|
+
- **Context Waste**: Identifies how much duplicate code is bloat for AI context windows.
|
|
40
|
+
- **Refactoring Suggestions**: Recommends extraction points for shared utilities.
|
|
136
41
|
|
|
137
|
-
##
|
|
42
|
+
## Installation
|
|
138
43
|
|
|
139
44
|
```bash
|
|
140
|
-
|
|
141
|
-
aiready patterns ./src
|
|
142
|
-
|
|
143
|
-
# Focus on obvious duplicates
|
|
144
|
-
aiready patterns ./src --similarity 0.9
|
|
145
|
-
|
|
146
|
-
# Include smaller patterns
|
|
147
|
-
aiready patterns ./src --min-lines 3
|
|
148
|
-
|
|
149
|
-
# Export results (saved to .aiready/ by default)
|
|
150
|
-
aiready patterns ./src --output json
|
|
151
|
-
|
|
152
|
-
# Or specify custom path
|
|
153
|
-
aiready patterns ./src --output json --output-file custom-report.json
|
|
45
|
+
pnpm add @aiready/pattern-detect
|
|
154
46
|
```
|
|
155
47
|
|
|
156
|
-
|
|
48
|
+
## Usage
|
|
157
49
|
|
|
158
|
-
## ποΈ Tuning Guide
|
|
159
|
-
|
|
160
|
-
### Main Parameters
|
|
161
|
-
|
|
162
|
-
| Parameter | Default | Effect | Use When |
|
|
163
|
-
|-----------|---------|--------|----------|
|
|
164
|
-
| `--similarity` | `0.4` | Similarity threshold (0-1) | Want more/less sensitive detection |
|
|
165
|
-
| `--min-lines` | `5` | Minimum lines per pattern | Include/exclude small functions |
|
|
166
|
-
| `--min-shared-tokens` | `8` | Tokens that must match | Control comparison strictness |
|
|
167
|
-
|
|
168
|
-
### Quick Tuning Scenarios
|
|
169
|
-
|
|
170
|
-
**Want more results?** (catch subtle duplicates)
|
|
171
50
|
```bash
|
|
172
|
-
|
|
173
|
-
aiready patterns ./src --similarity 0.3
|
|
174
|
-
|
|
175
|
-
# Include smaller functions
|
|
176
|
-
aiready patterns ./src --min-lines 3
|
|
177
|
-
|
|
178
|
-
# Both together
|
|
179
|
-
aiready patterns ./src --similarity 0.3 --min-lines 3
|
|
180
|
-
```
|
|
181
|
-
|
|
182
|
-
**Want fewer but higher quality results?** (focus on obvious duplicates)
|
|
183
|
-
```bash
|
|
184
|
-
# Higher similarity threshold
|
|
185
|
-
aiready patterns ./src --similarity 0.8
|
|
186
|
-
|
|
187
|
-
# Larger patterns only
|
|
188
|
-
aiready patterns ./src --min-lines 10
|
|
189
|
-
```
|
|
190
|
-
|
|
191
|
-
**Analysis too slow?** (optimize for speed)
|
|
192
|
-
```bash
|
|
193
|
-
# Focus on substantial functions
|
|
194
|
-
aiready patterns ./src --min-lines 10
|
|
195
|
-
|
|
196
|
-
# Reduce comparison candidates
|
|
197
|
-
aiready patterns ./src --min-shared-tokens 12
|
|
198
|
-
```
|
|
199
|
-
|
|
200
|
-
### Parameter Tradeoffs
|
|
201
|
-
|
|
202
|
-
| Adjustment | More Results | Faster | Higher Quality | Tradeoff |
|
|
203
|
-
|------------|--------------|--------|----------------|----------|
|
|
204
|
-
| Lower `--similarity` | β
| β | β | More false positives |
|
|
205
|
-
| Lower `--min-lines` | β
| β | β | Includes trivial duplicates |
|
|
206
|
-
| Higher `--similarity` | β | β
| β
| Misses subtle duplicates |
|
|
207
|
-
| Higher `--min-lines` | β | β
| β
| Misses small but important patterns |
|
|
208
|
-
|
|
209
|
-
### Common Workflows
|
|
210
|
-
|
|
211
|
-
**First run** (broad discovery):
|
|
212
|
-
```bash
|
|
213
|
-
aiready patterns ./src # Default settings
|
|
214
|
-
```
|
|
215
|
-
|
|
216
|
-
**Focus on critical issues** (production ready):
|
|
217
|
-
```bash
|
|
218
|
-
aiready patterns ./src --similarity 0.8 --min-lines 8
|
|
219
|
-
```
|
|
220
|
-
|
|
221
|
-
**Catch everything** (comprehensive audit):
|
|
222
|
-
```bash
|
|
223
|
-
aiready patterns ./src --similarity 0.3 --min-lines 3
|
|
51
|
+
aiready scan . --tools pattern-detect
|
|
224
52
|
```
|
|
225
53
|
|
|
226
|
-
|
|
227
|
-
```bash
|
|
228
|
-
aiready patterns ./src --min-lines 10 --min-shared-tokens 10
|
|
229
|
-
```
|
|
230
|
-
|
|
231
|
-
## π Configuration File
|
|
232
|
-
|
|
233
|
-
Create an `aiready.json` or `aiready.config.json` file in your project root:
|
|
234
|
-
|
|
235
|
-
```json
|
|
236
|
-
{
|
|
237
|
-
"scan": {
|
|
238
|
-
"include": ["src/**/*.{ts,tsx,js,jsx}"],
|
|
239
|
-
"exclude": ["**/*.test.*", "**/dist/**"]
|
|
240
|
-
},
|
|
241
|
-
"tools": {
|
|
242
|
-
"pattern-detect": {
|
|
243
|
-
"minSimilarity": 0.6,
|
|
244
|
-
"minLines": 8,
|
|
245
|
-
"maxResults": 20,
|
|
246
|
-
"minSharedTokens": 10,
|
|
247
|
-
"maxCandidatesPerBlock": 100
|
|
248
|
-
}
|
|
249
|
-
},
|
|
250
|
-
"output": {
|
|
251
|
-
"format": "console",
|
|
252
|
-
"file": ".aiready/pattern-report.json"
|
|
253
|
-
}
|
|
254
|
-
}
|
|
255
|
-
```
|
|
256
|
-
|
|
257
|
-
**Configuration Options:**
|
|
258
|
-
|
|
259
|
-
| Option | Type | Default | Description |
|
|
260
|
-
|--------|------|---------|-------------|
|
|
261
|
-
| `minSimilarity` | number | `0.4` | Similarity threshold (0-1) |
|
|
262
|
-
| `minLines` | number | `5` | Minimum lines to consider |
|
|
263
|
-
| `maxResults` | number | `10` | Max results to display in console |
|
|
264
|
-
| `minSharedTokens` | number | `8` | Min tokens that must match |
|
|
265
|
-
| `maxCandidatesPerBlock` | number | `100` | Performance tuning limit |
|
|
266
|
-
| `approx` | boolean | `true` | Use approximate candidate selection |
|
|
267
|
-
| `severity` | string | `'all'` | Filter: `'critical'`, `'high'`, `'medium'`, `'all'` |
|
|
268
|
-
|
|
269
|
-
**Use the unified CLI** for all AIReady tools:
|
|
270
|
-
|
|
271
|
-
```bash
|
|
272
|
-
npm install -g @aiready/cli
|
|
273
|
-
|
|
274
|
-
# Pattern detection
|
|
275
|
-
aiready patterns ./src
|
|
276
|
-
|
|
277
|
-
# Context analysis (token costs, fragmentation)
|
|
278
|
-
aiready context ./src
|
|
279
|
-
|
|
280
|
-
# Consistency checking (naming, patterns)
|
|
281
|
-
aiready consistency ./src
|
|
282
|
-
|
|
283
|
-
# Full codebase analysis
|
|
284
|
-
aiready scan ./src
|
|
285
|
-
```
|
|
286
|
-
|
|
287
|
-
**Related packages:**
|
|
288
|
-
- [**@aiready/cli**](https://www.npmjs.com/package/@aiready/cli) - Unified CLI with all tools
|
|
289
|
-
- [**@aiready/context-analyzer**](https://www.npmjs.com/package/@aiready/context-analyzer) - Context window cost analysis
|
|
290
|
-
- [**@aiready/consistency**](https://www.npmjs.com/package/@aiready/consistency) - Consistency checking
|
|
291
|
-
|
|
292
|
-
## π Visit Our Website
|
|
293
|
-
|
|
294
|
-
**Try AIReady tools online and optimize your codebase:** [getaiready.dev](https://getaiready.dev)
|
|
295
|
-
|
|
296
|
-
---
|
|
54
|
+
## License
|
|
297
55
|
|
|
298
|
-
|
|
56
|
+
MIT
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@aiready/pattern-detect",
|
|
3
|
-
"version": "0.11.
|
|
3
|
+
"version": "0.11.30",
|
|
4
4
|
"description": "Semantic duplicate pattern detection for AI-generated code - finds similar implementations that waste AI context tokens",
|
|
5
5
|
"main": "./dist/index.js",
|
|
6
6
|
"module": "./dist/index.mjs",
|
|
@@ -45,7 +45,7 @@
|
|
|
45
45
|
"dependencies": {
|
|
46
46
|
"commander": "^14.0.0",
|
|
47
47
|
"chalk": "^5.3.0",
|
|
48
|
-
"@aiready/core": "0.9.
|
|
48
|
+
"@aiready/core": "0.9.31"
|
|
49
49
|
},
|
|
50
50
|
"devDependencies": {
|
|
51
51
|
"tsup": "^8.3.5",
|