@ninjaxtools/slopdex 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/slopdex/SKILL.md +181 -258
- package/README.md +177 -207
- package/dist/cli.js +8 -0
- package/dist/cli.js.map +1 -1
- package/docs/implementation.md +161 -0
- package/package.json +2 -1
|
@@ -1,340 +1,263 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: slopdex
|
|
3
|
-
description: Use when
|
|
3
|
+
description: Use when using the slopdex CLI to search code by meaning or purpose, find duplicate-function candidates, analyze physical code cohesion, or maintain indexes of Python, JavaScript/JSX, TypeScript/TSX, Rust, Go, Java, and C code.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# Slopdex
|
|
6
|
+
# Slopdex operator guide for agents
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
Slopdex searches named functions by meaning, finds similar-code candidates, and identifies related functions stored far apart. Languages are detected automatically and can coexist in one index.
|
|
9
9
|
|
|
10
|
-
##
|
|
10
|
+
## Choose the command that answers the task
|
|
11
11
|
|
|
12
|
-
Run the
|
|
12
|
+
Run the requested operation directly. Do not precede it with status, help, version, executable lookup, or credential probes unless those are the user's task or needed to diagnose a reported failure. Missing indexes initialize automatically.
|
|
13
13
|
|
|
14
|
-
|
|
15
|
-
- For duplicate-code clusters, run `slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.9 --limit 5`.
|
|
16
|
-
- For related functions that are physically separated, run `slopdex cohesion --format summary --threshold 0.8 --neighbors 20 --limit 50`.
|
|
17
|
-
- For an explicit request to refresh the current index, run `slopdex update-git`.
|
|
18
|
-
- Use `slopdex status` only when the user asks for index metadata or checkpoint information.
|
|
19
|
-
|
|
20
|
-
Every command refreshes its index from committed `HEAD`, then overlays working-tree changes. A missing index is created automatically; let the requested command perform the refresh rather than initializing separately. Without Git or a Git repository, every command warns on stderr and fully re-indexes the working tree. Use `--no-reindex` only when explicitly asked to skip Git working-tree overlays or reuse an existing non-empty index without Git.
|
|
21
|
-
|
|
22
|
-
## Prerequisites
|
|
23
|
-
|
|
24
|
-
- Run commands from the repository root or pass `--root <path>`.
|
|
25
|
-
- Set `OPENAI_API_KEY` or `JINA_API_KEY` for the configured embedding provider.
|
|
26
|
-
- Use Node.js 24 or newer.
|
|
27
|
-
- Store optional configuration in `.slopdex/config.json`:
|
|
28
|
-
|
|
29
|
-
```json
|
|
30
|
-
{
|
|
31
|
-
"provider": "jina",
|
|
32
|
-
"model": "jina-embeddings-v4",
|
|
33
|
-
"dimensions": 1024,
|
|
34
|
-
"exclude": ["**/fixtures/**"]
|
|
35
|
-
}
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
Do not expose API keys in commands, output, configuration files, or commits.
|
|
39
|
-
|
|
40
|
-
## Indexing
|
|
41
|
-
|
|
42
|
-
Root and nested `.gitignore` files restrict indexing, including tracked files. Working-tree updates use current ignore files; historical or committed-only Git updates use ignore files from the selected commit. A normal refresh removes newly ignored files from the index. Explicit updates reject ignored paths.
|
|
43
|
-
|
|
44
|
-
Indexing records parse, callable-extraction, file-read, and file-size failures in SQLite while retaining healthy callables. Commands warn on stderr when unresolved diagnostics remain. Inspect them with `slopdex index-errors --format summary` (or JSON by default); this reads saved errors without refreshing the index or calling providers. Records include paths, source locations, recoverable function names, messages, and available source text. `status` reports `indexingErrorCount` and `failedFileCount`. Failed files are retried during updates; fixes, deletions, and exclusions clear their diagnostics. `--ignore-errors` silences diagnostic warnings without discarding the records.
|
|
45
|
-
|
|
46
|
-
If a CLI command cannot find its source index, Slopdex prints a notice to stderr and automatically creates and populates it from committed `HEAD`, then overlays working-tree changes. Missing cross-search target indexes are initialized from the target repository's `HEAD` and working tree as well.
|
|
47
|
-
|
|
48
|
-
Index the current committed snapshot and working-tree overlay:
|
|
14
|
+
### Find code by meaning
|
|
49
15
|
|
|
50
16
|
```bash
|
|
51
|
-
slopdex
|
|
17
|
+
slopdex search "validate an authenticated session" --format summary --limit 10
|
|
18
|
+
slopdex search "persist user data" -e 'save|persist' --format summary --limit 5
|
|
52
19
|
```
|
|
53
20
|
|
|
54
|
-
|
|
21
|
+
Describe behavior rather than guessing a symbol name. `-e` restricts result qualified names before limiting. Read the matched source to establish behavior and callers.
|
|
55
22
|
|
|
56
|
-
|
|
57
|
-
slopdex update-files src/service.ts src/model.ts
|
|
58
|
-
```
|
|
59
|
-
|
|
60
|
-
Remove deleted files from the index when using explicit updates:
|
|
23
|
+
### Search function purpose
|
|
61
24
|
|
|
62
25
|
```bash
|
|
63
|
-
slopdex
|
|
26
|
+
slopdex use-summaries
|
|
27
|
+
slopdex search-summary "keep the repository index synchronized" --format summary --limit 10
|
|
64
28
|
```
|
|
65
29
|
|
|
66
|
-
|
|
30
|
+
Use this workflow when purpose-summary generation/search is requested. `use-summaries` enables persistent automatic updates and adds API generation work. It needs `OPENAI_API_KEY` even with Jina embeddings. Repeating it with unchanged inputs reuses summaries. `search-summary` requires summaries to be enabled and includes summary text in results.
|
|
67
31
|
|
|
68
|
-
|
|
69
|
-
slopdex update-git --target HEAD
|
|
70
|
-
slopdex update-git --target HEAD --rebuild-on-divergence
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
Rebuild automatically when an existing index is incompatible with the current provider, model, dimensions, strategy, schema, or repository:
|
|
32
|
+
To select another model:
|
|
74
33
|
|
|
75
34
|
```bash
|
|
76
|
-
slopdex
|
|
35
|
+
slopdex use-summaries --summary-model <model-id>
|
|
77
36
|
```
|
|
78
37
|
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
Skip the otherwise mandatory full working-tree refresh when Git is unavailable and a non-empty index already exists:
|
|
82
|
-
|
|
83
|
-
```bash
|
|
84
|
-
slopdex search "query" --no-reindex
|
|
85
|
-
```
|
|
38
|
+
The model persists for future updates. Do not enable summaries as a routine prerequisite for ordinary code search or duplicate discovery.
|
|
86
39
|
|
|
87
|
-
|
|
40
|
+
### Find duplicate candidates
|
|
88
41
|
|
|
89
42
|
```bash
|
|
90
|
-
slopdex
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
Git updates reconcile committed blobs first, then overlay working-tree contents. Explicit file updates do not advance the Git checkpoint.
|
|
94
|
-
|
|
95
|
-
## Failure Handling
|
|
96
|
-
|
|
97
|
-
- Preserve and report the exact failure from the requested command. Do not retry equivalent initialization commands.
|
|
98
|
-
- Provider authentication and configuration failures are actionable as printed. Never display key values, and do not probe whether keys are set unless the error specifically indicates missing credentials and the user asks for diagnosis.
|
|
99
|
-
- A bare system error such as `Invalid argument` is a Slopdex/runtime failure, not evidence that a different indexing command is needed. Stop retrying, report the command and error, and recommend diagnosing or updating Slopdex.
|
|
100
|
-
- `slopdex --help` is the supported capability reference. There is no `slopdex --version` option; never invoke it.
|
|
101
|
-
|
|
102
|
-
## Analysis Examples
|
|
103
|
-
|
|
104
|
-
### Duplicate Analysis
|
|
105
|
-
|
|
106
|
-
```bash
|
|
107
|
-
slopdex cross-search \
|
|
108
|
-
--cross-file-only \
|
|
109
|
-
--min-lines 4 \
|
|
110
|
-
--threshold 0.9 \
|
|
111
|
-
--limit 5
|
|
112
|
-
```
|
|
113
|
-
|
|
114
|
-
```text
|
|
115
|
-
Cluster 1 (3 functions, similarity 0.9124-0.9568)
|
|
116
|
-
src/auth/session.ts:18:1 :: validateSession
|
|
117
|
-
src/http/middleware.ts:42:1 :: authenticate
|
|
118
|
-
src/users/user-service.ts:27:3 :: UserService.authenticate
|
|
43
|
+
slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.9 --limit 5
|
|
119
44
|
```
|
|
120
45
|
|
|
121
|
-
|
|
46
|
+
The default output is connected clusters. This excludes same-file matches and short functions. `--limit` is neighbors **per source**, not a limit on total findings or clusters. For source-by-source matches, add `--format summary`.
|
|
122
47
|
|
|
123
|
-
|
|
48
|
+
Broaden discovery through adjacent score bands when needed:
|
|
124
49
|
|
|
125
50
|
```bash
|
|
126
|
-
slopdex
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
```text
|
|
130
|
-
Cohesion: 184 functions analyzed, 37 semantic edges
|
|
131
|
-
same file 35.1% same folder 29.7% remote 35.2% mean distance 1.84
|
|
132
|
-
|
|
133
|
-
1. gap 0.6053 similarity 0.9400 distance 4 reciprocal
|
|
134
|
-
src/auth/session.ts:18:1 :: validateSession
|
|
135
|
-
packages/http/middleware.ts:42:1 :: authenticate
|
|
51
|
+
slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.85-0.9 --limit 5
|
|
52
|
+
slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.8-0.85 --limit 5
|
|
136
53
|
```
|
|
137
54
|
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
## Reading Analysis Output
|
|
141
|
-
|
|
142
|
-
Interpret values only within the same embedding profile and similar command settings. Changing the model, threshold, neighbor count, source scope, or minimum line count changes the candidate graph and makes direct comparisons unreliable.
|
|
143
|
-
|
|
144
|
-
### Duplicate Clusters
|
|
145
|
-
|
|
146
|
-
For `Cluster 1 (3 functions, similarity 0.9124-0.9568)`:
|
|
147
|
-
|
|
148
|
-
- `Cluster 1` is the display identifier. Clusters are ordered by function count and then name, not severity, so a lower number is not inherently worse.
|
|
149
|
-
- `3 functions` counts unique callables connected by observed edges. Higher can indicate a larger duplicate family, but may also result from generic helpers or transitive links.
|
|
150
|
-
- `similarity 0.9124-0.9568` is the weakest-to-strongest raw cosine similarity among observed edges. Higher means greater semantic resemblance according to the configured model. A high minimum means every observed link is strong; a wide range can identify a weaker bridge.
|
|
151
|
-
- Callable lines use `path:line:column :: qualifiedFunctionName`. Location is contextual rather than scored; inspect architectural roles before consolidating code.
|
|
152
|
-
|
|
153
|
-
### Cohesion Summary
|
|
154
|
-
|
|
155
|
-
For `Cohesion: 184 functions analyzed, 37 semantic edges`:
|
|
55
|
+
Ranges include the lower bound and exclude the upper bound. Use `--min-lines 1` when one-line wrappers are relevant. Treat matches as review candidates; inspect source before suggesting consolidation.
|
|
156
56
|
|
|
157
|
-
|
|
158
|
-
- `semantic edges` counts unique top-neighbor pairs meeting the threshold before output limiting. Higher can reflect more overlapping responsibilities, but also increases with more neighbors or a lower threshold.
|
|
159
|
-
|
|
160
|
-
For `same file 35.1% same folder 29.7% remote 35.2% mean distance 1.84`:
|
|
161
|
-
|
|
162
|
-
- Higher `same file` generally means stronger co-location, though very high values can indicate oversized files.
|
|
163
|
-
- Higher `same folder` means related code is split into nearby modules but remains locally grouped.
|
|
164
|
-
- Higher `remote` means more semantic affinity crosses folder boundaries and indicates weaker physical cohesion.
|
|
165
|
-
- Lower `mean distance` generally means stronger physical cohesion. Zero is entirely within files, one reaches only other files in the same folder, and larger values indicate greater dispersion.
|
|
166
|
-
|
|
167
|
-
### Cohesion Findings
|
|
168
|
-
|
|
169
|
-
For `1. gap 0.6053 similarity 0.9400 distance 4 reciprocal`:
|
|
170
|
-
|
|
171
|
-
- A lower rank number is a higher review priority.
|
|
172
|
-
- Higher `gap` means stronger semantic affinity combined with greater physical separation; same-file pairs have zero gap.
|
|
173
|
-
- Higher `similarity` means stronger model-assessed resemblance, but does not prove duplication and is not comparable across embedding profiles.
|
|
174
|
-
- Higher `distance` means more file and directory-tree separation: zero is the same file and one is different files in the same folder.
|
|
175
|
-
- `reciprocal` strengthens confidence because both functions rank each other as neighbors. Its absence means one-directional or unevaluated under a source filter.
|
|
176
|
-
|
|
177
|
-
In JSON, higher `semanticWeight` means similarity lies farther above the threshold, and higher `separationWeight` means greater path distance. `sourceTestPair: true` flags an often-intentional source/test relationship. Higher file `externalAffinityRatio` means more observed related-function affinity lies outside that file's folder; lower means relationships are primarily internal or local.
|
|
178
|
-
|
|
179
|
-
## Semantic Search
|
|
180
|
-
|
|
181
|
-
Search indexed functions by intent:
|
|
57
|
+
### Review changes or a module
|
|
182
58
|
|
|
183
59
|
```bash
|
|
184
|
-
slopdex search
|
|
60
|
+
slopdex cross-search --uncommitted --cross-file-only --min-lines 4 --threshold 0.9
|
|
61
|
+
slopdex cross-search --changed-since origin/main --format summary --threshold 0.9
|
|
62
|
+
slopdex cross-search --source-path src/services -e '^UserService\.' --format summary --threshold 0.9
|
|
185
63
|
```
|
|
186
64
|
|
|
187
|
-
|
|
65
|
+
These restrict sources while searching the full eligible index. The same filters work with `cohesion`. All supplied restrictions intersect:
|
|
188
66
|
|
|
189
67
|
```bash
|
|
190
|
-
slopdex search
|
|
191
|
-
--
|
|
192
|
-
--threshold 0.
|
|
193
|
-
--limit 10
|
|
68
|
+
slopdex cross-search --source-path src -e 'validate' \
|
|
69
|
+
--changed-since origin/main --uncommitted \
|
|
70
|
+
--cross-file-only --min-lines 4 --threshold 0.9
|
|
194
71
|
```
|
|
195
72
|
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
## Duplicate Discovery
|
|
199
|
-
|
|
200
|
-
Cross-search defaults to connected clusters. Treat each cluster as a source-review candidate rather than proof of duplication. Lower `--threshold` to broaden discovery, or lower `--min-lines` when short wrappers are relevant.
|
|
201
|
-
|
|
202
|
-
Summary output can instead group matches beneath each source:
|
|
203
|
-
|
|
204
|
-
```text
|
|
205
|
-
src/users.ts :: Users.authenticate
|
|
206
|
-
0.9321 src/session.ts :: validateSession
|
|
207
|
-
0.8475 src/auth.ts :: authenticate
|
|
208
|
-
```
|
|
209
|
-
|
|
210
|
-
Interpret high similarity as a candidate requiring source review, not proof of duplication. Public facade methods, API wrappers, interface implementations, and test doubles often score highly while serving distinct roles.
|
|
211
|
-
|
|
212
|
-
Cross-search excludes one-line callables by default. Raise `--min-lines` for more substantial duplicate candidates, or use `--min-lines 1` when short wrappers are relevant:
|
|
73
|
+
Here a source must have changed since the commit and belong to an uncommitted file, within the selected path/name scope. `--regex` is different from `-e`: it restricts **both** sources and candidates.
|
|
213
74
|
|
|
214
|
-
|
|
215
|
-
slopdex cross-search --min-lines 4 --threshold 0.9
|
|
216
|
-
```
|
|
217
|
-
|
|
218
|
-
Filter source symbols by qualified callable name while searching the whole eligible index:
|
|
75
|
+
### Review physical cohesion
|
|
219
76
|
|
|
220
77
|
```bash
|
|
221
|
-
slopdex
|
|
78
|
+
slopdex cohesion --threshold 0.8 --neighbors 20 --limit 50 --format summary
|
|
79
|
+
slopdex cohesion --source-path src/services --threshold 0.8 --format summary
|
|
222
80
|
```
|
|
223
81
|
|
|
224
|
-
|
|
82
|
+
Ranks related functions by semantic affinity and file/folder separation. For structured processing use `--format json`; add `--include-source` only when full callable bodies are needed.
|
|
225
83
|
|
|
226
|
-
|
|
84
|
+
### Compare repositories
|
|
227
85
|
|
|
228
86
|
```bash
|
|
229
|
-
slopdex cross-search
|
|
87
|
+
slopdex cross-search \
|
|
88
|
+
--target-root /path/to/other/repo \
|
|
89
|
+
--target-index /path/to/other/repo/.slopdex/index.sqlite \
|
|
90
|
+
--threshold 0.9 --format summary
|
|
230
91
|
```
|
|
231
92
|
|
|
232
|
-
|
|
93
|
+
Both target options are required. Both indexes refresh and must have identical embedding profiles. The target refresh uses the source command's embedding provider and the target's file-selection/summary configuration. Use `--target-config <path>` for a custom target config.
|
|
233
94
|
|
|
234
|
-
|
|
95
|
+
### Inspect or maintain the index
|
|
235
96
|
|
|
236
97
|
```bash
|
|
237
|
-
slopdex
|
|
238
|
-
slopdex
|
|
239
|
-
slopdex
|
|
98
|
+
slopdex status
|
|
99
|
+
slopdex index-errors --format summary
|
|
100
|
+
slopdex update-git
|
|
101
|
+
slopdex update-files src/service.ts src/model.ts
|
|
102
|
+
slopdex delete-files src/removed.ts
|
|
103
|
+
slopdex --version
|
|
240
104
|
```
|
|
241
105
|
|
|
242
|
-
|
|
106
|
+
- `status` refreshes, then reports coverage, checkpoint, profiles, and error counts; use when metadata is requested.
|
|
107
|
+
- `index-errors` reads saved failures without refreshing or needing API credentials.
|
|
108
|
+
- `update-git` explicitly refreshes HEAD and working-tree changes.
|
|
109
|
+
- `update-files` reparses specified working-tree files after automatic refresh, even when their contents are unchanged.
|
|
110
|
+
- `delete-files` removes index entries after automatic refresh, not source files. Eligible files can return on later refresh.
|
|
111
|
+
- `--version` prints the built package version. `--help` describes available commands/options.
|
|
112
|
+
|
|
113
|
+
## Command-line reference
|
|
114
|
+
|
|
115
|
+
Usage: `slopdex <command> [arguments] [options]`. Quote queries and regexes. Boolean flags default to off. Use options only with their applicable commands.
|
|
116
|
+
|
|
117
|
+
### General settings
|
|
118
|
+
|
|
119
|
+
| Argument | Meaning / default |
|
|
120
|
+
| --- | --- |
|
|
121
|
+
| `--root <path>` | Repository root; current directory by default. |
|
|
122
|
+
| `--config <path>` | Config; `<root>/.slopdex/config.json`. |
|
|
123
|
+
| `--index <path>` | Index; `<root>/.slopdex/index.sqlite`. Overrides config `indexPath`. |
|
|
124
|
+
| `--provider <openai\|jina>` | Embedding provider; `openai`. |
|
|
125
|
+
| `--model <name>` | Embedding model; OpenAI `text-embedding-3-large`, Jina `jina-embeddings-v4`. |
|
|
126
|
+
| `--dimensions <number>` | Positive dimensions supported by the model; OpenAI `3072`, Jina `1024`. |
|
|
127
|
+
| `--summary-model <name>` | OpenAI summary model; initially `gpt-5.6-sol`, then the persisted selection. |
|
|
128
|
+
| `--ignore-errors` | Silence saved-diagnostic warnings without deleting records. |
|
|
129
|
+
| `-h`, `--help` | Usage; no refresh. |
|
|
130
|
+
| `--version` | Package version; exits without refresh or saved-diagnostic warnings. |
|
|
131
|
+
|
|
132
|
+
Explicit relative config/index paths resolve from the current directory. Source paths and explicit file arguments resolve within `--root`. Prefer absolute paths when operating across repositories. CLI settings override config.
|
|
133
|
+
|
|
134
|
+
### Query and analysis options
|
|
135
|
+
|
|
136
|
+
| Argument | Applies to / behavior |
|
|
137
|
+
| --- | --- |
|
|
138
|
+
| `--limit <number>` | Positive integer. `search`/`search-summary`: matches, default `10`. Cross-search: neighbors per source, default `5`. Cohesion: reported pairs and file rows, default `50`. |
|
|
139
|
+
| `--threshold <number\|min-max>` | Both query searches and analyses. Inclusive minimum or half-open range. Default `-1` for query/cross-search; `0.8` for cohesion. Cohesion minimum must be in `[-1, 1)`. |
|
|
140
|
+
| `--format <json\|summary\|clusters>` | Both query searches, cross-search, cohesion, index-errors. `clusters` only supports cross-search; output defaults below. |
|
|
141
|
+
| `-e <regex>`, `--regexp <regex>` | Case-sensitive JavaScript regex on qualified names. Query searches filter results before limiting; cross-search/cohesion filter sources only. |
|
|
142
|
+
| `--regex <regex>` | Cross-search/cohesion: filter both source and candidate qualified names. |
|
|
143
|
+
| `--min-lines <number>` | Cross-search/cohesion: positive source/candidate length minimum, default `2`. |
|
|
144
|
+
| `--source-path <path>` | Cross-search/cohesion: source file or recursive directory within the root. |
|
|
145
|
+
| `--changed-since <commit>` | Cross-search/cohesion: added, modified, or moved functions since an ancestor of the indexed Git checkpoint, including working-tree changes. Requires Git. |
|
|
146
|
+
| `--uncommitted` | Cross-search/cohesion: functions indexed from working-tree files; in Git these are staged, unstaged, or untracked changes. Without Git this selects all working-tree functions. |
|
|
147
|
+
| `--cross-file-only` | Cross-search: exclude same-physical-file matches. |
|
|
148
|
+
| `--include-symmetric-duplicates` | Cross-search: allow both directions of same-index matches; otherwise each unordered pair appears once. |
|
|
149
|
+
| `--neighbors <number>` | Cohesion: positive neighbor count per source, default `20`; changes analysis scope. |
|
|
150
|
+
| `--include-source` | Cohesion JSON: include callable bodies; omitted by default. |
|
|
151
|
+
| `--target-root <path>` | Cross-search: second repository; requires `--target-index`. |
|
|
152
|
+
| `--target-index <path>` | Cross-search: second index file; requires `--target-root`. |
|
|
153
|
+
| `--target-config <path>` | Cross-search: target config, default `<target-root>/.slopdex/config.json`; requires both target options. |
|
|
154
|
+
|
|
155
|
+
### Refresh and recovery options
|
|
156
|
+
|
|
157
|
+
| Argument | Behavior |
|
|
158
|
+
| --- | --- |
|
|
159
|
+
| `--target <ref>` | `update-git` snapshot, default `HEAD`. Non-HEAD targets are committed-only; later commands normally return to HEAD. |
|
|
160
|
+
| `--rebuild-on-divergence` | Permit reconciliation after non-descendant history changes, such as a rebase/branch switch. |
|
|
161
|
+
| `--force-reindex` | Recreate an incompatible index. Compatible indexes still use normal refresh; this is not an unconditional reparse flag. |
|
|
162
|
+
| `--no-reindex` | With Git, reconcile the committed snapshot but omit working-tree overlays. Without Git, reuse a non-empty index; missing/empty indexes still populate. Not an offline mode. |
|
|
163
|
+
|
|
164
|
+
Use `--no-reindex` when the task calls for committed-only results or reuse of an existing non-Git index, rather than silently weakening freshness.
|
|
165
|
+
|
|
166
|
+
## Interpret and report results
|
|
167
|
+
|
|
168
|
+
### Output formats
|
|
169
|
+
|
|
170
|
+
| Command | Default | Alternatives |
|
|
171
|
+
| --- | --- | --- |
|
|
172
|
+
| `search`, `search-summary` | JSON array | `summary`; purpose search includes generated summary text |
|
|
173
|
+
| `cross-search` | `clusters` | `summary`, or `json` for JSONL with one row per matched source |
|
|
174
|
+
| `cohesion` | One JSON report | `summary` for ranked pairs |
|
|
175
|
+
| `index-errors` | JSON array | `summary` |
|
|
176
|
+
| `status`, update commands, `use-summaries` | JSON object | — |
|
|
177
|
+
|
|
178
|
+
Prefer summary output for compact source review, clusters for duplicate families, and JSON/JSONL for structured processing. Stdout carries results; stderr carries notices and warnings. Cross-search omits sources without emitted matches. Empty output means no findings under the chosen coverage/filters, not proof that no similar code exists.
|
|
179
|
+
|
|
180
|
+
### Similarity and clusters
|
|
243
181
|
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
182
|
+
```text
|
|
183
|
+
Cluster 1 (3 functions, similarity 0.9124-0.9568)
|
|
184
|
+
src/auth/session.ts:18:1 :: validateSession
|
|
185
|
+
src/http/middleware.ts:42:1 :: authenticate
|
|
186
|
+
src/users/user-service.ts:27:3 :: UserService.authenticate
|
|
248
187
|
```
|
|
249
188
|
|
|
250
|
-
|
|
189
|
+
- Similarity is a model-dependent resemblance score, not a duplication probability.
|
|
190
|
+
- The range describes observed links. Members can be connected transitively; not all pairs necessarily match.
|
|
191
|
+
- Cluster numbers reflect ordering by member count and name, not severity.
|
|
192
|
+
- Inspect listed locations and callers. Tests, facades, adapters, and intentional layers can resemble each other without being redundant.
|
|
251
193
|
|
|
252
|
-
|
|
253
|
-
slopdex cross-search --cross-file-only --format summary --threshold 0.9
|
|
254
|
-
```
|
|
194
|
+
When reporting candidates, identify paths/symbols, summarize the shared behavior you verified, and explain whether consolidation is appropriate. Do not infer equivalence from the score alone.
|
|
255
195
|
|
|
256
|
-
|
|
196
|
+
### Purpose-aware scoring
|
|
257
197
|
|
|
258
|
-
|
|
259
|
-
slopdex cross-search --format clusters --threshold 0.9
|
|
260
|
-
```
|
|
198
|
+
`search` uses code only; `search-summary` uses purpose summaries only. Cross-search and cohesion automatically use **50% code + 50% summary similarity** when summaries are enabled and complete. Cross-repository analysis needs completeness on both sides; otherwise all scores are code-only. Summary-generator models may differ even though embedding profiles must match.
|
|
261
199
|
|
|
262
|
-
|
|
200
|
+
Thresholds and limits apply to the selected score. Text labels combined scoring; JSON includes component scores and mode/weights (`scoring` in cross-search, `parameters` in cohesion). Compare runs only with matching scoring mode, weights, embedding and summary-generator profiles, threshold, neighbor count, and source/candidate filters.
|
|
263
201
|
|
|
264
|
-
|
|
265
|
-
slopdex cross-search --source-path src/services --format summary --threshold 0.9
|
|
266
|
-
```
|
|
202
|
+
### Cohesion
|
|
267
203
|
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
204
|
+
```text
|
|
205
|
+
Cohesion: 184 functions analyzed, 37 semantic edges
|
|
206
|
+
same file 35.1% same folder 29.7% remote 35.2% mean distance 1.84
|
|
271
207
|
|
|
272
|
-
|
|
273
|
-
|
|
208
|
+
1. gap 0.6053 similarity 0.9400 distance 4 reciprocal
|
|
209
|
+
src/auth/session.ts:18:1 :: validateSession
|
|
210
|
+
packages/http/middleware.ts:42:1 :: authenticate
|
|
274
211
|
```
|
|
275
212
|
|
|
276
|
-
|
|
213
|
+
- **Functions analyzed / edges:** selected-source coverage and unique qualifying neighbor pairs before report limiting, not quality grades.
|
|
214
|
+
- **Same file / same folder / remote:** shares of weighted semantic affinity. More remote affinity means more related code crosses folder boundaries.
|
|
215
|
+
- **Mean distance:** weighted physical separation; `0` is same file, `1` is different files in one folder, larger means farther apart.
|
|
216
|
+
- **Gap / rank:** a `0–1` review score combining similarity above the threshold with separation. Higher gap ranks earlier; same-file pairs have zero gap.
|
|
217
|
+
- **Reciprocal:** both functions selected each other as neighbors. JSON `null` means an endpoint was not evaluated because of source filtering.
|
|
218
|
+
- **JSON details:** `semanticWeight` reflects similarity above threshold; `separationWeight` reflects distance; `sourceTestPair` flags a path-inferred source/test relationship; file `externalAffinityRatio` measures affinity outside that file's folder.
|
|
277
219
|
|
|
278
|
-
|
|
279
|
-
slopdex cross-search --uncommitted --format summary --threshold 0.9
|
|
280
|
-
```
|
|
281
|
-
|
|
282
|
-
Search against another compatible index:
|
|
220
|
+
The example merits reviewing separated authentication responsibilities, while accounting for intentional layering. There is no universal pass/fail threshold. Use comparable runs to evaluate changes. Filtered reports describe selected sources, not the full repository. Summary metrics use all qualifying edges; pairs/files are limited, and groups use reported pairs.
|
|
283
221
|
|
|
284
|
-
|
|
285
|
-
slopdex cross-search \
|
|
286
|
-
--target-root /path/to/other/repository \
|
|
287
|
-
--target-index /path/to/other/repository/.slopdex/index.sqlite \
|
|
288
|
-
--format summary \
|
|
289
|
-
--threshold 0.8
|
|
290
|
-
```
|
|
222
|
+
## Operational properties
|
|
291
223
|
|
|
292
|
-
|
|
293
|
-
|
|
224
|
+
- Requires Node.js 24+. Install with `npm install -g @ninjaxtools/slopdex` if installation is the task.
|
|
225
|
+
- Run from the repository root or pass `--root`. The default local index is `.slopdex/index.sqlite`; add `.slopdex/` to `.gitignore`.
|
|
226
|
+
- Most commands, including `status`, refresh before operating. With Git, the default is HEAD plus current working-tree changes; the checkpoint records the committed base. Without Git, refresh scans the working tree and warns. `index-errors`, help, and version do not refresh.
|
|
227
|
+
- Source filters narrow analysis, not the preceding refresh. Indexing and summary generation can make many API calls; unchanged inputs reuse cached results.
|
|
228
|
+
- Embedding keys are `OPENAI_API_KEY` or `JINA_API_KEY`; other than diagnostics/help/version, CLI commands require the configured embedding key even for cached analyses. Summaries also require OpenAI credentials when generation is needed.
|
|
229
|
+
- Function source and queries go to the embedding provider. Enabled summary generation sends repository name, path, callable source, and file context to OpenAI, and summary text to the embedding provider. Results and diagnostics remain in the local index. Never expose key values in tool calls, output, or commits.
|
|
230
|
+
- Coverage: Python `.py/.pyw`; JavaScript `.js/.mjs/.cjs/.jsx`; TypeScript `.ts/.mts/.cts/.tsx`; Rust `.rs`; Go `.go`; Java `.java`; C `.c/.h`. Named callables with bodies, including supported bound closures and nested functions, are indexed. Anonymous callbacks, bodyless declarations, macro expansion, and runtime relationships are outside coverage.
|
|
231
|
+
- Root/nested `.gitignore` rules apply even to tracked files and without Git. Current overlays use current rules; committed-only snapshots use committed rules. Refresh removes newly ignored files; explicit updates reject ignored paths.
|
|
232
|
+
- Dependency/build directories are excluded: `.git`, `.slopdex`, `node_modules`, `dist`, `build`, `coverage`, `vendor`, `generated`, `.venv`, `venv`, `__pycache__`, `.tox`, `.mypy_cache`, `.pytest_cache`, `target`. Config/ignore exceptions cannot override built-in exclusions or an ignored parent directory.
|
|
294
233
|
|
|
295
|
-
|
|
234
|
+
### Optional configuration
|
|
296
235
|
|
|
297
|
-
|
|
236
|
+
`<root>/.slopdex/config.json`:
|
|
298
237
|
|
|
299
|
-
```
|
|
300
|
-
|
|
238
|
+
```json
|
|
239
|
+
{
|
|
240
|
+
"provider": "jina",
|
|
241
|
+
"model": "jina-embeddings-v4",
|
|
242
|
+
"dimensions": 1024,
|
|
243
|
+
"exclude": ["**/fixtures/**"]
|
|
244
|
+
}
|
|
301
245
|
```
|
|
302
246
|
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
Treat findings as review candidates. Tests, facades, adapters, and intentionally layered implementations can be semantically related while correctly living in separate locations.
|
|
247
|
+
Supported properties: `provider`, `model`, `dimensions`, `summaryModel`, `indexPath`, `include`, `exclude`, `maxFileSize`, `embeddingBatchSize`. Includes/excludes are repository-relative globs; an empty include list permits all eligible supported files. `maxFileSize` defaults to `1048576` bytes; `embeddingBatchSize` to `32`; both are positive integers. Keep credentials in the environment. Embedding-profile changes require `--force-reindex`.
|
|
306
248
|
|
|
307
|
-
##
|
|
249
|
+
## Failures and incomplete coverage
|
|
308
250
|
|
|
309
|
-
-
|
|
310
|
-
- `--format clusters` groups overlapping pairs and lists each function once with its source line.
|
|
311
|
-
- The default `search` output is formatted JSON.
|
|
312
|
-
- The default `cross-search` output is connected clusters. Use `--format json` for JSONL with one object per source function.
|
|
313
|
-
- The default `cohesion` output is one compact JSON report. Use `--format summary` for ranked pairs.
|
|
314
|
-
- Functions with no matches after threshold filtering are omitted from cross-search output.
|
|
251
|
+
Parse, extraction, read, and file-size failures are saved while healthy functions remain searchable. Inspect `slopdex index-errors --format summary`; JSON adds locations, recovered names, source, and snapshot provenance. `status` reports `indexingErrorCount` and `failedFileCount`, and `functionCount` counts searchable callables.
|
|
315
252
|
|
|
316
|
-
|
|
253
|
+
Saved errors warn on stderr, including on cached runs and target indexes. `--ignore-errors` only silences the warning. Updates retry failed files; successful indexing, deletion, or exclusion clears records. Mention relevant incomplete coverage when interpreting results.
|
|
317
254
|
|
|
318
|
-
|
|
255
|
+
Preserve the exact command and error when an operation fails. Fix the reported cause instead of retrying equivalent initialization commands:
|
|
319
256
|
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
--model <name> Embedding model
|
|
326
|
-
--dimensions <number> Embedding dimensions
|
|
327
|
-
--force-rebuild Rebuild an incompatible existing index
|
|
328
|
-
--limit <number> Result limit
|
|
329
|
-
--neighbors <number> Semantic neighbors per function for cohesion
|
|
330
|
-
--threshold <number|range> Similarity threshold or half-open range
|
|
331
|
-
--format <json|summary|clusters> Output format
|
|
332
|
-
--include-source Include callable source in cohesion JSON
|
|
333
|
-
--cross-file-only Exclude matches from the source file
|
|
334
|
-
--min-lines <number> Minimum cross-search callable length
|
|
335
|
-
-e, --regexp <regex> Match qualified symbols (analysis: sources only)
|
|
336
|
-
--regex <regex> Match both analysis source and candidate names
|
|
337
|
-
--target-config <path> Target repository configuration file
|
|
338
|
-
```
|
|
257
|
+
- Missing credentials or provider/configuration errors: report the actionable message without displaying keys.
|
|
258
|
+
- Divergent checkpoint: use `--rebuild-on-divergence` when proceeding with the requested snapshot.
|
|
259
|
+
- Incompatible index: `--force-reindex` recreates it with the requested profile.
|
|
260
|
+
- Source changed during indexing: rerun after edits settle.
|
|
261
|
+
- Bare runtime errors such as `Invalid argument`: report the failure and diagnose the runtime/tool rather than trying unrelated refresh commands.
|
|
339
262
|
|
|
340
|
-
|
|
263
|
+
Exit codes: `0` success, `2` argument/domain errors, `1` other failures or missing command. Use help to resolve capability questions and version to report the installed package version when needed.
|