@ninjaxtools/slopdex 0.6.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (9) hide show
  1. package/.agents/skills/slopdex/SKILL.md +181 -245
  2. package/README.md +197 -97
  3. package/dist/cli.js +1088 -234
  4. package/dist/cli.js.map +1 -1
  5. package/dist/index.d.ts +109 -11
  6. package/dist/index.js +916 -196
  7. package/dist/index.js.map +1 -1
  8. package/docs/implementation.md +161 -0
  9. package/package.json +11 -3
package/.agents/skills/slopdex/SKILL.md CHANGED
@@ -1,112 +1,184 @@
1
1
  ---
2
2
  name: slopdex
3
- description: Use when indexing TypeScript or JavaScript code, running semantic function search, finding duplicate function candidates, or analyzing physical code cohesion with the slopdex command-line tool.
3
+ description: Use when using the slopdex CLI to search code by meaning or purpose, find duplicate-function candidates, analyze physical code cohesion, or maintain indexes of Python, JavaScript/JSX, TypeScript/TSX, Rust, Go, Java, and C code.
4
4
  ---
5
5
 
6
- # Slopdex CLI
6
+ # Slopdex operator guide for agents
7
7
 
8
- Use `slopdex` to index named JavaScript and TypeScript callables, search them by meaning, identify similar or duplicated functions, and find related functions scattered across a repository.
8
+ Slopdex searches named functions by meaning, finds similar-code candidates, and identifies related functions stored far apart. Languages are detected automatically and can coexist in one index.
9
9
 
10
- ## Default Workflow
10
+ ## Choose the command that answers the task
11
11
 
12
- Run the command that satisfies the user's request immediately. Do not begin with `slopdex status`, `slopdex --help`, executable lookup, API-key probes, or version probes. Slopdex performs its own validation and reports missing credentials or incompatible state.
12
+ Run the requested operation directly. Do not precede it with status, help, version, executable lookup, or credential probes unless those are the user's task or needed to diagnose a reported failure. Missing indexes initialize automatically.
13
13
 
14
- - For semantic search, run `slopdex search "<query>" --format summary --limit 10`.
15
- - For duplicate-code clusters, run `slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.9 --limit 5`.
16
- - For related functions that are physically separated, run `slopdex cohesion --format summary --threshold 0.8 --neighbors 20 --limit 50`.
17
- - For an explicit request to refresh the current index, run `slopdex update-git`.
18
- - Use `slopdex status` only when the user asks for index metadata or checkpoint information.
14
+ ### Find code by meaning
19
15
 
20
- Every command refreshes its index from committed `HEAD`, then overlays working-tree changes. A missing index is created automatically; let the requested command perform the refresh rather than initializing separately. Without Git or a Git repository, every command warns on stderr and fully re-indexes the working tree. Use `--no-reindex` only when explicitly asked to skip Git working-tree overlays or reuse an existing non-empty index without Git.
16
+ ```bash
17
+ slopdex search "validate an authenticated session" --format summary --limit 10
18
+ slopdex search "persist user data" -e 'save|persist' --format summary --limit 5
19
+ ```
21
20
 
22
- ## Prerequisites
21
+ Describe behavior rather than guessing a symbol name. `-e` restricts result qualified names before limiting. Read the matched source to establish behavior and callers.
23
22
 
24
- - Run commands from the repository root or pass `--root <path>`.
25
- - Set `OPENAI_API_KEY` or `JINA_API_KEY` for the configured embedding provider.
26
- - Use Node.js 24 or newer.
27
- - Store optional configuration in `.slopdex/config.json`:
23
+ ### Search function purpose
28
24
 
29
- ```json
30
- {
31
- "provider": "jina",
32
- "model": "jina-embeddings-v4",
33
- "dimensions": 1024,
34
- "exclude": ["**/fixtures/**"]
35
- }
25
+ ```bash
26
+ slopdex use-summaries
27
+ slopdex search-summary "keep the repository index synchronized" --format summary --limit 10
36
28
  ```
37
29
 
38
- Do not expose API keys in commands, output, configuration files, or commits.
39
-
40
- ## Indexing
41
-
42
- If a CLI command cannot find its source index, Slopdex prints a notice to stderr and automatically creates and populates it from committed `HEAD`, then overlays working-tree changes. Missing cross-search target indexes are initialized from the target repository's `HEAD` and working tree as well.
30
+ Use this workflow when purpose-summary generation/search is requested. `use-summaries` enables persistent automatic updates and adds API generation work. It needs `OPENAI_API_KEY` even with Jina embeddings. Repeating it with unchanged inputs reuses summaries. `search-summary` requires summaries to be enabled and includes summary text in results.
43
31
 
44
- Index the current committed snapshot and working-tree overlay:
32
+ To select another model:
45
33
 
46
34
  ```bash
47
- slopdex update-git
35
+ slopdex use-summaries --summary-model <model-id>
48
36
  ```
49
37
 
50
- `update-git` reconciles the committed snapshot and, when the target is the checked-out `HEAD`, indexes staged, unstaged, and untracked changes. Historical or other-branch targets remain exact committed snapshots. The Git checkpoint remains the committed base hash. After the mandatory automatic refresh, explicitly re-index selected working-tree files with:
38
+ The model persists for future updates. Do not enable summaries as a routine prerequisite for ordinary code search or duplicate discovery.
39
+
40
+ ### Find duplicate candidates
51
41
 
52
42
  ```bash
53
- slopdex update-files src/service.ts src/model.ts
43
+ slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.9 --limit 5
54
44
  ```
55
45
 
56
- Remove deleted files from the index when using explicit updates:
46
+ The default output is connected clusters. This excludes same-file matches and short functions. `--limit` is neighbors **per source**, not a limit on total findings or clusters. For source-by-source matches, add `--format summary`.
47
+
48
+ Broaden discovery through adjacent score bands when needed:
57
49
 
58
50
  ```bash
59
- slopdex delete-files src/removed.ts
51
+ slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.85-0.9 --limit 5
52
+ slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.8-0.85 --limit 5
60
53
  ```
61
54
 
62
- Index another commit or recover after changing to a divergent branch:
55
+ Ranges include the lower bound and exclude the upper bound. Use `--min-lines 1` when one-line wrappers are relevant. Treat matches as review candidates; inspect source before suggesting consolidation.
56
+
57
+ ### Review changes or a module
63
58
 
64
59
  ```bash
65
- slopdex update-git --target HEAD
66
- slopdex update-git --target HEAD --rebuild-on-divergence
60
+ slopdex cross-search --uncommitted --cross-file-only --min-lines 4 --threshold 0.9
61
+ slopdex cross-search --changed-since origin/main --format summary --threshold 0.9
62
+ slopdex cross-search --source-path src/services -e '^UserService\.' --format summary --threshold 0.9
67
63
  ```
68
64
 
69
- Rebuild automatically when an existing index is incompatible with the current provider, model, dimensions, strategy, schema, or repository:
65
+ These restrict sources while searching the full eligible index. The same filters work with `cohesion`. All supplied restrictions intersect:
70
66
 
71
67
  ```bash
72
- slopdex update-git --force-rebuild
68
+ slopdex cross-search --source-path src -e 'validate' \
69
+ --changed-since origin/main --uncommitted \
70
+ --cross-file-only --min-lines 4 --threshold 0.9
73
71
  ```
74
72
 
75
- This removes the incompatible index and prints a warning before rebuilding it.
73
+ Here a source must have changed since the commit and belong to an uncommitted file, within the selected path/name scope. `--regex` is different from `-e`: it restricts **both** sources and candidates.
76
74
 
77
- Skip the otherwise mandatory full working-tree refresh when Git is unavailable and a non-empty index already exists:
75
+ ### Review physical cohesion
78
76
 
79
77
  ```bash
80
- slopdex search "query" --no-reindex
78
+ slopdex cohesion --threshold 0.8 --neighbors 20 --limit 50 --format summary
79
+ slopdex cohesion --source-path src/services --threshold 0.8 --format summary
81
80
  ```
82
81
 
83
- Check index metadata and its Git checkpoint:
82
+ Ranks related functions by semantic affinity and file/folder separation. For structured processing use `--format json`; add `--include-source` only when full callable bodies are needed.
83
+
84
+ ### Compare repositories
84
85
 
85
86
  ```bash
86
- slopdex status
87
+ slopdex cross-search \
88
+ --target-root /path/to/other/repo \
89
+ --target-index /path/to/other/repo/.slopdex/index.sqlite \
90
+ --threshold 0.9 --format summary
87
91
  ```
88
92
 
89
- Git updates reconcile committed blobs first, then overlay working-tree contents. Explicit file updates do not advance the Git checkpoint.
90
-
91
- ## Failure Handling
92
-
93
- - Preserve and report the exact failure from the requested command. Do not retry equivalent initialization commands.
94
- - Provider authentication and configuration failures are actionable as printed. Never display key values, and do not probe whether keys are set unless the error specifically indicates missing credentials and the user asks for diagnosis.
95
- - A bare system error such as `Invalid argument` is a Slopdex/runtime failure, not evidence that a different indexing command is needed. Stop retrying, report the command and error, and recommend diagnosing or updating Slopdex.
96
- - `slopdex --help` is the supported capability reference. There is no `slopdex --version` option; never invoke it.
97
-
98
- ## Analysis Examples
93
+ Both target options are required. Both indexes refresh and must have identical embedding profiles. The target refresh uses the source command's embedding provider and the target's file-selection/summary configuration. Use `--target-config <path>` for a custom target config.
99
94
 
100
- ### Duplicate Analysis
95
+ ### Inspect or maintain the index
101
96
 
102
97
  ```bash
103
- slopdex cross-search \
104
- --cross-file-only \
105
- --min-lines 4 \
106
- --threshold 0.9 \
107
- --limit 5
98
+ slopdex status
99
+ slopdex index-errors --format summary
100
+ slopdex update-git
101
+ slopdex update-files src/service.ts src/model.ts
102
+ slopdex delete-files src/removed.ts
103
+ slopdex --version
108
104
  ```
109
105
 
106
+ - `status` refreshes, then reports coverage, checkpoint, profiles, and error counts; use when metadata is requested.
107
+ - `index-errors` reads saved failures without refreshing or needing API credentials.
108
+ - `update-git` explicitly refreshes HEAD and working-tree changes.
109
+ - `update-files` reparses specified working-tree files after automatic refresh, even when their contents are unchanged.
110
+ - `delete-files` removes index entries after automatic refresh, not source files. Eligible files can return on later refresh.
111
+ - `--version` prints the built package version. `--help` describes available commands/options.
112
+
113
+ ## Command-line reference
114
+
115
+ Usage: `slopdex <command> [arguments] [options]`. Quote queries and regexes. Boolean flags default to off. Use options only with their applicable commands.
116
+
117
+ ### General settings
118
+
119
+ | Argument | Meaning / default |
120
+ | --- | --- |
121
+ | `--root <path>` | Repository root; current directory by default. |
122
+ | `--config <path>` | Config; `<root>/.slopdex/config.json`. |
123
+ | `--index <path>` | Index; `<root>/.slopdex/index.sqlite`. Overrides config `indexPath`. |
124
+ | `--provider <openai\|jina>` | Embedding provider; `openai`. |
125
+ | `--model <name>` | Embedding model; OpenAI `text-embedding-3-large`, Jina `jina-embeddings-v4`. |
126
+ | `--dimensions <number>` | Positive dimensions supported by the model; OpenAI `3072`, Jina `1024`. |
127
+ | `--summary-model <name>` | OpenAI summary model; initially `gpt-5.6-sol`, then the persisted selection. |
128
+ | `--ignore-errors` | Silence saved-diagnostic warnings without deleting records. |
129
+ | `-h`, `--help` | Usage; no refresh. |
130
+ | `--version` | Package version; exits without refresh or saved-diagnostic warnings. |
131
+
132
+ Explicit relative config/index paths resolve from the current directory. Source paths and explicit file arguments resolve within `--root`. Prefer absolute paths when operating across repositories. CLI settings override config.
133
+
134
+ ### Query and analysis options
135
+
136
+ | Argument | Applies to / behavior |
137
+ | --- | --- |
138
+ | `--limit <number>` | Positive integer. `search`/`search-summary`: matches, default `10`. Cross-search: neighbors per source, default `5`. Cohesion: reported pairs and file rows, default `50`. |
139
+ | `--threshold <number\|min-max>` | Both query searches and analyses. Inclusive minimum or half-open range. Default `-1` for query/cross-search; `0.8` for cohesion. Cohesion minimum must be in `[-1, 1)`. |
140
+ | `--format <json\|summary\|clusters>` | Both query searches, cross-search, cohesion, index-errors. `clusters` only supports cross-search; output defaults below. |
141
+ | `-e <regex>`, `--regexp <regex>` | Case-sensitive JavaScript regex on qualified names. Query searches filter results before limiting; cross-search/cohesion filter sources only. |
142
+ | `--regex <regex>` | Cross-search/cohesion: filter both source and candidate qualified names. |
143
+ | `--min-lines <number>` | Cross-search/cohesion: positive source/candidate length minimum, default `2`. |
144
+ | `--source-path <path>` | Cross-search/cohesion: source file or recursive directory within the root. |
145
+ | `--changed-since <commit>` | Cross-search/cohesion: added, modified, or moved functions since an ancestor of the indexed Git checkpoint, including working-tree changes. Requires Git. |
146
+ | `--uncommitted` | Cross-search/cohesion: functions indexed from working-tree files; in Git these are staged, unstaged, or untracked changes. Without Git this selects all working-tree functions. |
147
+ | `--cross-file-only` | Cross-search: exclude same-physical-file matches. |
148
+ | `--include-symmetric-duplicates` | Cross-search: allow both directions of same-index matches; otherwise each unordered pair appears once. |
149
+ | `--neighbors <number>` | Cohesion: positive neighbor count per source, default `20`; changes analysis scope. |
150
+ | `--include-source` | Cohesion JSON: include callable bodies; omitted by default. |
151
+ | `--target-root <path>` | Cross-search: second repository; requires `--target-index`. |
152
+ | `--target-index <path>` | Cross-search: second index file; requires `--target-root`. |
153
+ | `--target-config <path>` | Cross-search: target config, default `<target-root>/.slopdex/config.json`; requires both target options. |
154
+
155
+ ### Refresh and recovery options
156
+
157
+ | Argument | Behavior |
158
+ | --- | --- |
159
+ | `--target <ref>` | `update-git` snapshot, default `HEAD`. Non-HEAD targets are committed-only; later commands normally return to HEAD. |
160
+ | `--rebuild-on-divergence` | Permit reconciliation after non-descendant history changes, such as a rebase/branch switch. |
161
+ | `--force-reindex` | Recreate an incompatible index. Compatible indexes still use normal refresh; this is not an unconditional reparse flag. |
162
+ | `--no-reindex` | With Git, reconcile the committed snapshot but omit working-tree overlays. Without Git, reuse a non-empty index; missing/empty indexes still populate. Not an offline mode. |
163
+
164
+ Use `--no-reindex` when the task calls for committed-only results or reuse of an existing non-Git index, rather than silently weakening freshness.
165
+
166
+ ## Interpret and report results
167
+
168
+ ### Output formats
169
+
170
+ | Command | Default | Alternatives |
171
+ | --- | --- | --- |
172
+ | `search`, `search-summary` | JSON array | `summary`; purpose search includes generated summary text |
173
+ | `cross-search` | `clusters` | `summary`, or `json` for JSONL with one row per matched source |
174
+ | `cohesion` | One JSON report | `summary` for ranked pairs |
175
+ | `index-errors` | JSON array | `summary` |
176
+ | `status`, update commands, `use-summaries` | JSON object | — |
177
+
178
+ Prefer summary output for compact source review, clusters for duplicate families, and JSON/JSONL for structured processing. Stdout carries results; stderr carries notices and warnings. Cross-search omits sources without emitted matches. Empty output means no findings under the chosen coverage/filters, not proof that no similar code exists.
179
+
180
+ ### Similarity and clusters
181
+
110
182
  ```text
111
183
  Cluster 1 (3 functions, similarity 0.9124-0.9568)
112
184
  src/auth/session.ts:18:1 :: validateSession
@@ -114,13 +186,20 @@ Cluster 1 (3 functions, similarity 0.9124-0.9568)
114
186
  src/users/user-service.ts:27:3 :: UserService.authenticate
115
187
  ```
116
188
 
117
- This result found three substantial authentication functions in three files with very high similarity. Review them for repeated validation or session-handling logic that could move into one shared implementation. The middleware and service locations may represent intentional architectural layers, so treat the result as evidence to inspect rather than proof that the functions should be merged. The range describes observed links in a connected component; transitive clustering means every function is not necessarily directly similar to every other function.
189
+ - Similarity is a model-dependent resemblance score, not a duplication probability.
190
+ - The range describes observed links. Members can be connected transitively; not all pairs necessarily match.
191
+ - Cluster numbers reflect ordering by member count and name, not severity.
192
+ - Inspect listed locations and callers. Tests, facades, adapters, and intentional layers can resemble each other without being redundant.
118
193
 
119
- ### Cohesion Analysis
194
+ When reporting candidates, identify paths/symbols, summarize the shared behavior you verified, and explain whether consolidation is appropriate. Do not infer equivalence from the score alone.
120
195
 
121
- ```bash
122
- slopdex cohesion --format summary --threshold 0.8 --neighbors 20 --limit 50
123
- ```
196
+ ### Purpose-aware scoring
197
+
198
+ `search` uses code only; `search-summary` uses purpose summaries only. Cross-search and cohesion automatically use **50% code + 50% summary similarity** when summaries are enabled and complete. Cross-repository analysis needs completeness on both sides; otherwise all scores are code-only. Summary-generator models may differ even though embedding profiles must match.
199
+
200
+ Thresholds and limits apply to the selected score. Text labels combined scoring; JSON includes component scores and mode/weights (`scoring` in cross-search, `parameters` in cohesion). Compare runs only with matching scoring mode, weights, embedding and summary-generator profiles, threshold, neighbor count, and source/candidate filters.
201
+
202
+ ### Cohesion
124
203
 
125
204
  ```text
126
205
  Cohesion: 184 functions analyzed, 37 semantic edges
@@ -131,197 +210,54 @@ Cohesion: 184 functions analyzed, 37 semantic edges
131
210
  packages/http/middleware.ts:42:1 :: authenticate
132
211
  ```
133
212
 
134
- There is no universal pass/fail cutoff for cohesion, but this example has several warning signs. More than a third of weighted semantic affinity crosses folder boundaries, and the mean distance of 1.84 is above the same-folder distance of one. The top pair is strongly related at 0.94 similarity yet four distance units apart, producing a relatively high gap of 0.6053; the reciprocal match strengthens that signal. A more cohesive result under the same settings would concentrate affinity in the same-file and same-folder percentages, have a lower mean distance, and contain few high-gap remote pairs. Inspect whether shared authentication behavior belongs in one module, while accounting for the possibility that session and middleware responsibilities are intentionally separated. Compare modules or repository history rather than treating one percentage as a fixed quality threshold.
135
-
136
- ## Reading Analysis Output
137
-
138
- Interpret values only within the same embedding profile and similar command settings. Changing the model, threshold, neighbor count, source scope, or minimum line count changes the candidate graph and makes direct comparisons unreliable.
139
-
140
- ### Duplicate Clusters
141
-
142
- For `Cluster 1 (3 functions, similarity 0.9124-0.9568)`:
143
-
144
- - `Cluster 1` is the display identifier. Clusters are ordered by function count and then name, not severity, so a lower number is not inherently worse.
145
- - `3 functions` counts unique callables connected by observed edges. Higher can indicate a larger duplicate family, but may also result from generic helpers or transitive links.
146
- - `similarity 0.9124-0.9568` is the weakest-to-strongest raw cosine similarity among observed edges. Higher means greater semantic resemblance according to the configured model. A high minimum means every observed link is strong; a wide range can identify a weaker bridge.
147
- - Callable lines use `path:line:column :: qualifiedFunctionName`. Location is contextual rather than scored; inspect architectural roles before consolidating code.
148
-
149
- ### Cohesion Summary
150
-
151
- For `Cohesion: 184 functions analyzed, 37 semantic edges`:
152
-
153
- - `functions analyzed` is coverage after filters, not a quality value.
154
- - `semantic edges` counts unique top-neighbor pairs meeting the threshold before output limiting. Higher can reflect more overlapping responsibilities, but also increases with more neighbors or a lower threshold.
155
-
156
- For `same file 35.1% same folder 29.7% remote 35.2% mean distance 1.84`:
157
-
158
- - Higher `same file` generally means stronger co-location, though very high values can indicate oversized files.
159
- - Higher `same folder` means related code is split into nearby modules but remains locally grouped.
160
- - Higher `remote` means more semantic affinity crosses folder boundaries and indicates weaker physical cohesion.
161
- - Lower `mean distance` generally means stronger physical cohesion. Zero is entirely within files, one reaches only other files in the same folder, and larger values indicate greater dispersion.
162
-
163
- ### Cohesion Findings
164
-
165
- For `1. gap 0.6053 similarity 0.9400 distance 4 reciprocal`:
166
-
167
- - A lower rank number is a higher review priority.
168
- - Higher `gap` means stronger semantic affinity combined with greater physical separation; same-file pairs have zero gap.
169
- - Higher `similarity` means stronger model-assessed resemblance, but does not prove duplication and is not comparable across embedding profiles.
170
- - Higher `distance` means more file and directory-tree separation: zero is the same file and one is different files in the same folder.
171
- - `reciprocal` strengthens confidence because both functions rank each other as neighbors. Its absence means one-directional or unevaluated under a source filter.
172
-
173
- In JSON, higher `semanticWeight` means similarity lies farther above the threshold, and higher `separationWeight` means greater path distance. `sourceTestPair: true` flags an often-intentional source/test relationship. Higher file `externalAffinityRatio` means more observed related-function affinity lies outside that file's folder; lower means relationships are primarily internal or local.
174
-
175
- ## Semantic Search
176
-
177
- Search indexed functions by intent:
178
-
179
- ```bash
180
- slopdex search "validate an authenticated session" --limit 10
181
- ```
182
-
183
- Use compact human-readable output and filter weak results:
184
-
185
- ```bash
186
- slopdex search "validate an authenticated session" \
187
- --format summary \
188
- --threshold 0.8 \
189
- --limit 10
190
- ```
213
+ - **Functions analyzed / edges:** selected-source coverage and unique qualifying neighbor pairs before report limiting, not quality grades.
214
+ - **Same file / same folder / remote:** shares of weighted semantic affinity. More remote affinity means more related code crosses folder boundaries.
215
+ - **Mean distance:** weighted physical separation; `0` is same file, `1` is different files in one folder, larger means farther apart.
216
+ - **Gap / rank:** a `0–1` review score combining similarity above the threshold with separation. Higher gap ranks earlier; same-file pairs have zero gap.
217
+ - **Reciprocal:** both functions selected each other as neighbors. JSON `null` means an endpoint was not evaluated because of source filtering.
218
+ - **JSON details:** `semanticWeight` reflects similarity above threshold; `separationWeight` reflects distance; `sourceTestPair` flags a path-inferred source/test relationship; file `externalAffinityRatio` measures affinity outside that file's folder.
191
219
 
192
- `--threshold` is the minimum raw cosine similarity. Use a half-open range such as `--threshold 0.85-0.95` to include the left bound and exclude the right bound.
220
+ The example merits reviewing separated authentication responsibilities, while accounting for intentional layering. There is no universal pass/fail threshold. Use comparable runs to evaluate changes. Filtered reports describe selected sources, not the full repository. Summary metrics use all qualifying edges; pairs/files are limited, and groups use reported pairs.
193
221
 
194
- ## Duplicate Discovery
222
+ ## Operational properties
195
223
 
196
- Cross-search defaults to connected clusters. Treat each cluster as a source-review candidate rather than proof of duplication. Lower `--threshold` to broaden discovery, or lower `--min-lines` when short wrappers are relevant.
224
+ - Requires Node.js 24+. Install with `npm install -g @ninjaxtools/slopdex` if installation is the task.
225
+ - Run from the repository root or pass `--root`. The default local index is `.slopdex/index.sqlite`; add `.slopdex/` to `.gitignore`.
226
+ - Most commands, including `status`, refresh before operating. With Git, the default is HEAD plus current working-tree changes; the checkpoint records the committed base. Without Git, refresh scans the working tree and warns. `index-errors`, help, and version do not refresh.
227
+ - Source filters narrow analysis, not the preceding refresh. Indexing and summary generation can make many API calls; unchanged inputs reuse cached results.
228
+ - Embedding keys are `OPENAI_API_KEY` or `JINA_API_KEY`; other than diagnostics/help/version, CLI commands require the configured embedding key even for cached analyses. Summaries also require OpenAI credentials when generation is needed.
229
+ - Function source and queries go to the embedding provider. Enabled summary generation sends repository name, path, callable source, and file context to OpenAI, and summary text to the embedding provider. Results and diagnostics remain in the local index. Never expose key values in tool calls, output, or commits.
230
+ - Coverage: Python `.py/.pyw`; JavaScript `.js/.mjs/.cjs/.jsx`; TypeScript `.ts/.mts/.cts/.tsx`; Rust `.rs`; Go `.go`; Java `.java`; C `.c/.h`. Named callables with bodies, including supported bound closures and nested functions, are indexed. Anonymous callbacks, bodyless declarations, macro expansion, and runtime relationships are outside coverage.
231
+ - Root/nested `.gitignore` rules apply even to tracked files and without Git. Current overlays use current rules; committed-only snapshots use committed rules. Refresh removes newly ignored files; explicit updates reject ignored paths.
232
+ - Dependency/build directories are excluded: `.git`, `.slopdex`, `node_modules`, `dist`, `build`, `coverage`, `vendor`, `generated`, `.venv`, `venv`, `__pycache__`, `.tox`, `.mypy_cache`, `.pytest_cache`, `target`. Config/ignore exceptions cannot override built-in exclusions or an ignored parent directory.
197
233
 
198
- Summary output can instead group matches beneath each source:
234
+ ### Optional configuration
199
235
 
200
- ```text
201
- src/users.ts :: Users.authenticate
202
- 0.9321 src/session.ts :: validateSession
203
- 0.8475 src/auth.ts :: authenticate
204
- ```
236
+ `<root>/.slopdex/config.json`:
205
237
 
206
- Interpret high similarity as a candidate requiring source review, not proof of duplication. Public facade methods, API wrappers, interface implementations, and test doubles often score highly while serving distinct roles.
207
-
208
- Cross-search excludes one-line callables by default. Raise `--min-lines` for more substantial duplicate candidates, or use `--min-lines 1` when short wrappers are relevant:
209
-
210
- ```bash
211
- slopdex cross-search --min-lines 4 --threshold 0.9
212
- ```
213
-
214
- Filter both source and matching candidates by qualified callable name with a JavaScript regular expression:
215
-
216
- ```bash
217
- slopdex cross-search --regex '^(User|Session)\.' --threshold 0.9
218
- ```
219
-
220
- Use `--threshold <minimum>-<maximum>` for a half-open similarity range, such as `--threshold 0.85-0.95`. It includes the minimum, excludes the maximum, and is applied before `--limit`.
221
-
222
- Review duplicate candidates iteratively from high confidence to lower-confidence bands instead of requesting one broad result set:
223
-
224
- ```bash
225
- slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.9 --limit 5
226
- slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.85-0.9 --limit 5
227
- slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.8-0.85 --limit 5
228
- ```
229
-
230
- Because range upper bounds are exclusive, adjacent passes do not repeat boundary candidates.
231
-
232
- Same-index search reports each unordered pair once by default. Include both `A -> B` and `B -> A` only when explicitly needed:
233
-
234
- ```bash
235
- slopdex cross-search --include-symmetric-duplicates
236
- ```
237
-
238
- Exclude candidates from the source function's file when looking for duplication across files:
239
-
240
- ```bash
241
- slopdex cross-search --cross-file-only --format summary --threshold 0.9
242
- ```
243
-
244
- Group overlapping pairs into connected components and list each function once:
245
-
246
- ```bash
247
- slopdex cross-search --format clusters --threshold 0.9
248
- ```
249
-
250
- Restrict source functions to a file or recursive directory while still matching them against the whole codebase:
251
-
252
- ```bash
253
- slopdex cross-search --source-path src/services --format summary --threshold 0.9
254
- ```
255
-
256
- `--source-path` restricts only source functions. It can be combined with `--changed-since` or `--uncommitted`.
257
-
258
- Restrict source functions to additions, modifications, and moves relative to a commit, including current working-tree changes:
259
-
260
- ```bash
261
- slopdex cross-search --changed-since origin/main --format summary --threshold 0.9
262
- ```
263
-
264
- Restrict source functions to uncommitted files:
265
-
266
- ```bash
267
- slopdex cross-search --uncommitted --format summary --threshold 0.9
268
- ```
269
-
270
- Search against another compatible index:
271
-
272
- ```bash
273
- slopdex cross-search \
274
- --target-root /path/to/other/repository \
275
- --target-index /path/to/other/repository/.slopdex/index.sqlite \
276
- --format summary \
277
- --threshold 0.8
278
- ```
279
-
280
- Cross-index searches require identical provider, model, dimensions, and embedding strategy profiles.
281
- Use `--target-config <path>` when the target repository does not use `.slopdex/config.json`.
282
-
283
- ## Cohesion Analysis
284
-
285
- Use JSON when passing the report to another tool or an LLM:
286
-
287
- ```bash
288
- slopdex cohesion --format json --threshold 0.8 --neighbors 20 --limit 50
238
+ ```json
239
+ {
240
+ "provider": "jina",
241
+ "model": "jina-embeddings-v4",
242
+ "dimensions": 1024,
243
+ "exclude": ["**/fixtures/**"]
244
+ }
289
245
  ```
290
246
 
291
- The report includes raw similarity, physical path distance, a combined cohesion-gap score, reciprocal-neighbor status, repository metrics, file-level external affinity, and connected groups for navigation. Reciprocity is `null` when the other endpoint was not searched because a source filter is active. Source code is omitted from JSON by default; add `--include-source` only when the consumer needs complete callable bodies.
247
+ Supported properties: `provider`, `model`, `dimensions`, `summaryModel`, `indexPath`, `include`, `exclude`, `maxFileSize`, `embeddingBatchSize`. Includes/excludes are repository-relative globs; an empty include list permits all eligible supported files. `maxFileSize` defaults to `1048576` bytes; `embeddingBatchSize` to `32`; both are positive integers. Keep credentials in the environment. Embedding-profile changes require `--force-reindex`.
292
248
 
293
- Treat findings as review candidates. Tests, facades, adapters, and intentionally layered implementations can be semantically related while correctly living in separate locations.
249
+ ## Failures and incomplete coverage
294
250
 
295
- ## Output Formats
251
+ Parse, extraction, read, and file-size failures are saved while healthy functions remain searchable. Inspect `slopdex index-errors --format summary`; JSON adds locations, recovered names, source, and snapshot provenance. `status` reports `indexingErrorCount` and `failedFileCount`, and `functionCount` counts searchable callables.
296
252
 
297
- - `--format summary` is intended for human review and includes file and qualified function names.
298
- - `--format clusters` groups overlapping pairs and lists each function once with its source line.
299
- - The default `search` output is formatted JSON.
300
- - The default `cross-search` output is connected clusters. Use `--format json` for JSONL with one object per source function.
301
- - The default `cohesion` output is one compact JSON report. Use `--format summary` for ranked pairs.
302
- - Functions with no matches after threshold filtering are omitted from cross-search output.
253
+ Saved errors warn on stderr, including on cached runs and target indexes. `--ignore-errors` only silences the warning. Updates retry failed files; successful indexing, deletion, or exclusion clears records. Mention relevant incomplete coverage when interpreting results.
303
254
 
304
- Prefer JSON or JSONL when another command will consume the results. Prefer summary output when presenting candidates to a user.
255
+ Preserve the exact command and error when an operation fails. Fix the reported cause instead of retrying equivalent initialization commands:
305
256
 
306
- ## Common Options
307
-
308
- ```text
309
- --root <path> Repository root
310
- --config <path> Configuration file
311
- --index <path> SQLite index path
312
- --provider <openai|jina> Embedding provider
313
- --model <name> Embedding model
314
- --dimensions <number> Embedding dimensions
315
- --force-rebuild Rebuild an incompatible existing index
316
- --limit <number> Result limit
317
- --neighbors <number> Semantic neighbors per function for cohesion
318
- --threshold <number|range> Similarity threshold or half-open range
319
- --format <json|summary|clusters> Output format
320
- --include-source Include callable source in cohesion JSON
321
- --cross-file-only Exclude matches from the source file
322
- --min-lines <number> Minimum cross-search callable length
323
- --regex <regex> Match qualified callable names
324
- --target-config <path> Target repository configuration file
325
- ```
257
+ - Missing credentials or provider/configuration errors: report the actionable message without displaying keys.
258
+ - Divergent checkpoint: use `--rebuild-on-divergence` when proceeding with the requested snapshot.
259
+ - Incompatible index: `--force-reindex` recreates it with the requested profile.
260
+ - Source changed during indexing: rerun after edits settle.
261
+ - Bare runtime errors such as `Invalid argument`: report the failure and diagnose the runtime/tool rather than trying unrelated refresh commands.
326
262
 
327
- Run `slopdex --help` for the complete current option list.
263
+ Exit codes: `0` success, `2` argument/domain errors, `1` other failures or missing command. Use help to resolve capability questions and version to report the installed package version when needed.