@ninjaxtools/slopdex 0.11.0 → 0.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: slopdex
3
- description: Semantic code search, find duplicate-function candidates, analyze physical code cohesion
3
+ description: Semantic code search, duplicate-function candidates, and physical-distance re-ranking
4
4
  ---
5
5
 
6
6
  # Slopdex operator guide for agents
@@ -16,6 +16,16 @@ slopdex search "persist user data" -e 'save|persist' --format summary --limit 5
16
16
 
17
17
  Describe behavior rather than guessing a symbol name. `-e` is a regex that restricts which symbols (functions) are searched.
18
18
 
19
+ Hosted reranking is optional and persists in repository config:
20
+
21
+ ```bash
22
+ slopdex config reranker cohere
23
+ # Or: slopdex config reranker jina
24
+ # Or use an LLM: slopdex config reranker openai
25
+ ```
26
+
27
+ Set `COHERE_API_KEY`, `JINA_API_KEY`, or `OPENAI_API_KEY` respectively. The OpenAI LLM reranker defaults to `gpt-5.6-luna`, high reasoning, and the top 10 embedding candidates; configure the pool with `slopdex config reranker openai --reranker-candidates 20`. Disable reranking with `slopdex config reranker disable`. Reranking applies to `search` and `search-description`, not cross-search. It preserves embedding `similarity`, adds `rerankScore`, and orders a wider candidate set by that score.
28
+
19
29
  ### Search function purpose
20
30
 
21
31
  Code purpose-description generation needs to be enabled once:
@@ -30,7 +40,8 @@ Then purpose descriptions can be searched:
30
40
  slopdex search-description "keep the repository index synchronized" --format summary --limit 10
31
41
  ```
32
42
 
33
- Enabling needs `OPENAI_API_KEY` in the environment.
43
+ Enabling needs `OPENAI_API_KEY` by default. Select OpenCode Zen or Go with
44
+ `--description-provider opencode` or `--description-provider opencode-go` and set `OPENCODE_API_KEY`.
34
45
 
35
46
  To select another model:
36
47
 
@@ -38,6 +49,17 @@ To select another model:
38
49
  slopdex descriptions enable --description-model <model-id>
39
50
  ```
40
51
 
52
+ List and persist a published OpenCode model without creating an index:
53
+
54
+ ```bash
55
+ slopdex models opencode-go
56
+ slopdex config model opencode-go/gpt-5.6-luna
57
+ slopdex config descriptions enable
58
+ ```
59
+
60
+ The next index-using command applies the configured description state. A bare model ID passed to
61
+ `config model` resolves automatically only when it belongs to one of Zen or Go; qualify shared IDs.
62
+
41
63
  ### Find duplicate candidates
42
64
 
43
65
  ```bash
@@ -63,7 +85,7 @@ slopdex cross-search --changed-since origin/main --format summary --threshold 0.
63
85
  slopdex cross-search --source-path src/services -e '^UserService\.' --format summary --threshold 0.9
64
86
  ```
65
87
 
66
- These restrict sources while searching the full eligible index. The same filters work with `cohesion`. All supplied restrictions intersect:
88
+ These restrict sources while searching the full eligible index. All supplied restrictions intersect:
67
89
 
68
90
  ```bash
69
91
  slopdex cross-search --source-path src -e 'validate' \
@@ -76,11 +98,11 @@ Here a source must have changed since the commit and belong to an uncommitted fi
76
98
  ### Review physical cohesion
77
99
 
78
100
  ```bash
79
- slopdex cohesion --threshold 0.8 --neighbors 20 --limit 50 --format summary
80
- slopdex cohesion --source-path src/services --threshold 0.8 --format summary
101
+ slopdex cross-search --cohesion --threshold 0.8 --limit 20 --format summary
102
+ slopdex cross-search --cohesion --source-path src/services --threshold 0.8 --format summary
81
103
  ```
82
104
 
83
- Ranks related functions by semantic affinity and file/folder separation. add `--include-source` only when full callable bodies are needed.
105
+ `--cohesion` keeps cross-search's semantic matches and orders each source's matches from greatest to least physical path distance. Similarity breaks distance ties. Summary output includes the distance; JSONL matches include `physicalDistance`.
84
106
 
85
107
  ### Compare repositories
86
108
 
@@ -100,6 +122,8 @@ slopdex status
100
122
  slopdex index-errors --format summary
101
123
  slopdex update-git
102
124
  slopdex update-files src/service.ts src/model.ts
125
+ slopdex reindex-files
126
+ slopdex reindex-files --callables
103
127
  slopdex delete-files src/removed.ts
104
128
  slopdex --version
105
129
  ```
@@ -108,6 +132,7 @@ slopdex --version
108
132
  - `index-errors` reads saved failures without refreshing or needing API credentials.
109
133
  - `update-git` explicitly refreshes HEAD and working-tree changes.
110
134
  - `update-files` reparses specified working-tree files after automatic refresh, even when their contents are unchanged.
135
+ - `reindex-files` regenerates stale file descriptions and embeddings. Add `--callables` to also replace callable descriptions in those files.
111
136
  - `delete-files` removes index entries after automatic refresh, not source files. Eligible files can return on later refresh.
112
137
  - `--version` prints the built package version. `--help` describes available commands/options.
113
138
 
@@ -125,7 +150,9 @@ Usage: `slopdex <command> [arguments] [options]`. Quote queries and regexes. Boo
125
150
  | `--provider <openai\|jina>` | Embedding provider; `openai`. |
126
151
  | `--model <name>` | Embedding model; OpenAI `text-embedding-3-large`, Jina `jina-embeddings-v4`. |
127
152
  | `--dimensions <number>` | Positive dimensions supported by the model; OpenAI `3072`, Jina `1024`. |
128
- | `--description-model <name>` | OpenAI description model; initially `gpt-5.6-sol`, then the persisted selection. |
153
+ | `--description-provider <openai\|opencode\|opencode-go>` | Description provider; OpenAI by default. OpenCode values require `OPENCODE_API_KEY`. |
154
+ | `--description-model <name>` | Description model; `gpt-5.6-sol` for OpenAI/Zen and `gpt-5.6-luna` for Go. |
155
+ | `--reranker-candidates <number>` | With `config reranker openai`, embedding-ranked functions sent to the LLM; range `1`-`100`, default `10`. |
129
156
  | `--ignore-errors` | Silence saved-diagnostic warnings without deleting records. |
130
157
  | `-h`, `--help` | Usage; no refresh. |
131
158
  | `--version` | Package version; exits without refresh or saved-diagnostic warnings. |
@@ -136,18 +163,17 @@ Explicit relative config/index paths resolve from the current directory. Source
136
163
 
137
164
  | Argument | Applies to / behavior |
138
165
  | --- | --- |
139
- | `--limit <number>` | Positive integer. `search`/`search-description`: matches, default `10`. Cross-search: neighbors per source, default `5`. Cohesion: reported pairs and file rows, default `50`. |
140
- | `--threshold <number\|min-max>` | Both query searches and analyses. Inclusive minimum or half-open range. Default `-1` for query/cross-search; `0.8` for cohesion. Cohesion minimum must be in `[-1, 1)`. |
141
- | `--format <json\|summary\|clusters>` | Both query searches, cross-search, cohesion, index-errors. `clusters` only supports cross-search; output defaults below. |
142
- | `-e <regex>`, `--regexp <regex>`, `--regex <regex>` | Equivalent case-sensitive JavaScript regex options on qualified names. Query searches filter results before limiting; cross-search/cohesion filter sources only. |
143
- | `--min-lines <number>` | Cross-search/cohesion: positive source/candidate length minimum, default `2`. |
144
- | `--source-path <path>` | Cross-search/cohesion: source file or recursive directory within the root. |
145
- | `--changed-since <commit>` | Cross-search/cohesion: added, modified, or moved functions since an ancestor of the indexed Git checkpoint, including working-tree changes. Requires Git. |
146
- | `--uncommitted` | Cross-search/cohesion: functions indexed from working-tree files; in Git these are staged, unstaged, or untracked changes. Without Git this selects all working-tree functions. |
166
+ | `--limit <number>` | Positive integer. `search`/`search-description`: matches, default `10`. Cross-search: neighbors per source, default `5`. |
167
+ | `--threshold <number\|min-max>` | Both query searches and cross-search. Inclusive minimum or half-open range; default `-1`. |
168
+ | `--format <json\|summary\|clusters>` | Both query searches, cross-search, and index-errors. Cohesion-ranked cross-search supports summary or JSONL, not clusters. |
169
+ | `-e <regex>`, `--regexp <regex>`, `--regex <regex>` | Equivalent case-sensitive JavaScript regex options on qualified names. Query searches filter results before limiting; cross-search filters sources only. |
170
+ | `--min-lines <number>` | Cross-search: positive source/candidate length minimum, default `2`. |
171
+ | `--source-path <path>` | Cross-search: source file or recursive directory within the root. |
172
+ | `--changed-since <commit>` | Cross-search: added, modified, or moved functions since an ancestor of the indexed Git checkpoint, including working-tree changes. Requires Git. |
173
+ | `--uncommitted` | Cross-search: functions indexed from working-tree files; in Git these are staged, unstaged, or untracked changes. Without Git this selects all working-tree functions. |
147
174
  | `--cross-file-only` | Cross-search: exclude same-physical-file matches. |
148
175
  | `--include-symmetric-duplicates` | Cross-search: allow both directions of same-index matches; otherwise each unordered pair appears once. |
149
- | `--neighbors <number>` | Cohesion: positive neighbor count per source, default `20`; changes analysis scope. |
150
- | `--include-source` | Cohesion JSON: include callable bodies; omitted by default. |
176
+ | `--cohesion` | Cross-search: add physical distance and order each source's matches from farthest to nearest. Defaults to summary output. |
151
177
  | `--target-root <path>` | Cross-search: second repository; requires `--target-index`. |
152
178
  | `--target-index <path>` | Cross-search: second index file; requires `--target-root`. |
153
179
  | `--target-config <path>` | Cross-search: target config, default `<target-root>/.slopdex/config.json`; requires both target options. |
@@ -160,6 +186,7 @@ Explicit relative config/index paths resolve from the current directory. Source
160
186
  | `--rebuild-on-divergence` | Permit reconciliation after non-descendant history changes, such as a rebase/branch switch. |
161
187
  | `--force-reindex` | Recreate an incompatible index. Compatible indexes still use normal refresh; this is not an unconditional reparse flag. |
162
188
  | `--no-reindex` | With Git, reconcile the committed snapshot but omit working-tree overlays. Without Git, reuse a non-empty index; missing/empty indexes still populate. Not an offline mode. |
189
+ | `--callables` | With `reindex-files`, continue after the file description and regenerate every callable description in each stale file. |
163
190
 
164
191
  Use `--no-reindex` when the task calls for committed-only results or reuse of an existing non-Git index, rather than silently weakening freshness.
165
192
 
@@ -171,12 +198,13 @@ Use `--no-reindex` when the task calls for committed-only results or reuse of an
171
198
  | --- | --- | --- |
172
199
  | `search`, `search-description` | `summary` | JSON array; purpose search includes generated description text |
173
200
  | `cross-search` | `clusters` | `summary`, or `json` for JSONL with one row per matched source |
174
- | `cohesion` | `summary` | One JSON report |
175
201
  | `index-errors` | `summary` | JSON array |
176
202
  | `status`, update commands, `descriptions` | JSON object | — |
177
203
 
178
204
  Prefer summary output for compact source review, clusters for duplicate families, and JSON/JSONL for structured processing. Stdout carries results; stderr carries notices and warnings. Cross-search omits sources without emitted matches. Empty output means no findings under the chosen coverage/filters, not proof that no similar code exists.
179
205
 
206
+ With reranking enabled, query summaries display both reranker relevance and embedding similarity. Similarity thresholds filter candidates before reranking; limits apply to the reranked output. LLM candidate documents include descriptions when available and function metadata/source code.
207
+
180
208
  ### Similarity and clusters
181
209
 
182
210
  ```text
@@ -195,26 +223,16 @@ When reporting candidates, identify paths/symbols, summarize the shared behavior
195
223
 
196
224
  ### Purpose-aware scoring
197
225
 
198
- `search` uses code only; `search-description` uses purpose descriptions only. Cross-search and cohesion automatically use **50% code + 50% description similarity** when descriptions are enabled and complete. Cross-repository analysis needs completeness on both sides; otherwise all scores are code-only. Description-generator models may differ even though embedding profiles must match.
226
+ When descriptions are complete, `search` and cross-search average code, callable-description, and file-description similarity with equal one-third weights. `search-description` averages callable and file descriptions. Cross-repository analysis needs completeness on both sides; otherwise all scores are code-only. Stale file descriptions remain in scoring until `reindex-files` refreshes them. Description-generator models may differ even though embedding profiles must match.
199
227
 
200
- Thresholds and limits apply to the selected score. Text labels combined scoring; JSON includes component scores and mode/weights (`scoring` in cross-search, `parameters` in cohesion). Compare runs only with matching scoring mode, weights, embedding and description-generator profiles, threshold, neighbor count, and source/candidate filters.
228
+ Thresholds and limits apply to the selected score. Text labels combined scoring; JSON includes component scores and mode/weights in `scoring`. Compare runs only with matching scoring mode, weights, embedding and description-generator profiles, threshold, and source/candidate filters.
201
229
 
202
230
  ### Cohesion
203
231
 
204
232
  ```text
205
- Cohesion: 184 functions analyzed, 37 semantic edges
206
- same file 35.1% same folder 29.7% remote 35.2% mean distance 1.84
207
-
208
- 1. gap 0.6053 similarity 0.9400 distance 4 reciprocal
209
- src/auth/session.ts:18:1 :: validateSession
210
- packages/http/middleware.ts:42:1 :: authenticate
233
+ src/auth/session.ts :: validateSession
234
+ 0.9400 packages/http/middleware.ts :: authenticate [distance 4]
235
+ 0.9300 src/auth/token.ts :: validateToken [distance 1]
211
236
  ```
212
237
 
213
- - **Functions analyzed / edges:** selected-source coverage and unique qualifying neighbor pairs before report limiting, not quality grades.
214
- - **Same file / same folder / remote:** shares of weighted semantic affinity. More remote affinity means more related code crosses folder boundaries.
215
- - **Mean distance:** weighted physical separation; `0` is same file, `1` is different files in one folder, larger means farther apart.
216
- - **Gap / rank:** a `0–1` review score combining similarity above the threshold with separation. Higher gap ranks earlier; same-file pairs have zero gap.
217
- - **Reciprocal:** both functions selected each other as neighbors. JSON `null` means an endpoint was not evaluated because of source filtering.
218
- - **JSON details:** `semanticWeight` reflects similarity above threshold; `separationWeight` reflects distance; `sourceTestPair` flags a path-inferred source/test relationship; file `externalAffinityRatio` measures affinity outside that file's folder.
219
-
220
- The example merits reviewing separated authentication responsibilities, while accounting for intentional layering. There is no universal pass/fail threshold. Use comparable runs to evaluate changes. Filtered reports describe selected sources, not the full repository. Cohesion metrics use all qualifying edges; pairs/files are limited, and groups use reported pairs.
238
+ Physical distance is `0` within one file, `1` between files in one folder, and `1` plus directory-tree hops across folders. The option only reorders the selected semantic matches; it does not change similarity or prove that distant code should be moved. Review architectural layers, tests, adapters, and other intentional separation before recommending consolidation.
package/README.md CHANGED
@@ -24,6 +24,17 @@ Add `.slopdex/` to your repository's `.gitignore`.
24
24
  slopdex search "keep the repository index synchronized" --format summary --limit 10
25
25
  ```
26
26
 
27
+ Optionally enable a hosted second-stage reranker for `search` and `search-description`:
28
+
29
+ ```bash
30
+ export COHERE_API_KEY="your-api-key"
31
+ slopdex config reranker cohere
32
+ # Or: export JINA_API_KEY="your-api-key" && slopdex config reranker jina
33
+ # Or use an LLM: export OPENAI_API_KEY="your-api-key" && slopdex config reranker openai
34
+ ```
35
+
36
+ The config command is the only CLI switch for reranking. Use `slopdex config reranker disable` to return to embedding-only ordering. An optional final argument selects a model, for example `slopdex config reranker cohere rerank-v4.0-fast`. OpenAI LLM reranking defaults to `gpt-5.6-luna` with high reasoning and receives the top 10 embedding results; change the pool with `--reranker-candidates`, for example `slopdex config reranker openai gpt-5.6-luna --reranker-candidates 20`.
37
+
27
38
  To search generated descriptions of each function's role instead:
28
39
 
29
40
  ```bash
@@ -31,10 +42,21 @@ slopdex descriptions enable
31
42
  slopdex search-description "keep the repository index synchronized" --format summary --limit 10
32
43
  ```
33
44
 
34
- This requires `OPENAI_API_KEY` even when Jina supplies embeddings, and adds generation costs. See [descriptions and scoring](#descriptions-and-scoring).
45
+ The default description provider requires `OPENAI_API_KEY` even when Jina supplies embeddings. OpenCode Zen and Go use `OPENCODE_API_KEY`. Description generation can add provider costs; see [descriptions and scoring](#descriptions-and-scoring).
35
46
 
36
47
  `descriptions disable` turns off description generation while retaining cached data.
37
48
 
49
+ To choose from OpenCode's current published models and persist description settings without creating an index:
50
+
51
+ ```bash
52
+ slopdex models opencode-go
53
+ slopdex config model opencode-go/gpt-5.6-luna
54
+ slopdex config descriptions enable
55
+ ```
56
+
57
+ The next index-using command creates or refreshes the index and applies the configured description state.
58
+ You can also pass the selection separately as `slopdex config model --description-provider opencode-go --description-model gpt-5.6-luna`.
59
+
38
60
  ### Find duplicate-code candidates
39
61
 
40
62
  ```bash
@@ -51,15 +73,15 @@ slopdex cross-search --changed-since origin/main --format summary --threshold 0.
51
73
  slopdex cross-search --source-path src/services -e '^UserService\.' --format summary --threshold 0.9
52
74
  ```
53
75
 
54
- These select source functions while keeping the full eligible index available for matches. The same source filters work with `cohesion`.
76
+ These select source functions while keeping the full eligible index available for matches.
55
77
 
56
78
  ### Find related code stored far apart
57
79
 
58
80
  ```bash
59
- slopdex cohesion --threshold 0.8 --neighbors 20 --limit 50 --format summary
81
+ slopdex cross-search --cohesion --threshold 0.8 --limit 20 --format summary
60
82
  ```
61
83
 
62
- Ranks semantically related pairs by their physical separation.
84
+ `--cohesion` keeps the semantic matches selected by cross-search, annotates them with physical distance, and orders each source's matches from farthest to nearest. Similarity breaks distance ties.
63
85
 
64
86
  ### Compare repositories
65
87
 
@@ -88,15 +110,19 @@ Usage: `slopdex <command> [arguments] [options]`.
88
110
 
89
111
  | Command | Purpose | Output |
90
112
  | --- | --- | --- |
113
+ | `models [opencode\|opencode-go]` | Fetch valid models from the current published Zen and/or Go catalogs. | Qualified `provider/model` lines; optional JSON array |
114
+ | `config model <model\|provider/model>` | Validate a published OpenCode model and persist its provider/model selection without opening an index. Bare IDs auto-resolve only when unambiguous. | Updated setting summary; optional JSON |
115
+ | `config descriptions <enable\|disable>` | Persist whether the next index-using command should enable or disable descriptions. Does not open an index. | Updated setting summary; optional JSON |
116
+ | `config reranker <cohere\|jina\|openai\|disable> [model]` | Enable a hosted or OpenAI LLM query reranker, optionally selecting a model, or disable it. OpenAI accepts `--reranker-candidates <number>` from 1 to 100 and defaults to 10. Does not open an index. | Updated setting summary; optional JSON |
91
117
  | `search <query>` | Search function code by meaning. Quote multiword queries. | Summary; optional JSON array |
92
118
  | `descriptions <enable\|disable>` | Enable or disable automatic purpose descriptions. Re-enabling with unchanged inputs reuses cached descriptions. | JSON statistics |
93
119
  | `search-description <query>` | Search purpose descriptions after enabling them. | Summary including description text; optional JSON array |
94
120
  | `cross-search` | Find neighbors for each selected function in this or another index. | `clusters` by default; optional `summary` or JSONL |
95
- | `cohesion` | Analyze semantic relationships versus file/folder separation. | Summary; optional JSON report |
96
121
  | `status` | Refresh and show index metadata, counts, and profiles. | JSON object |
97
122
  | `index-errors` | Read saved file/function indexing failures. | Summary; optional JSON array |
98
123
  | `update-git` | Explicitly refresh a Git snapshot, with current working-tree changes when targeting HEAD. | JSON update statistics |
99
124
  | `update-files <path...>` | After automatic refresh, explicitly reparse selected working-tree files. Paths are repository-relative or absolute within the root. | JSON update statistics |
125
+ | `reindex-files [--callables]` | Regenerate descriptions for files changed since their stored file description. By default stops after each file description; `--callables` also regenerates its callable descriptions. | JSON description statistics |
100
126
  | `delete-files <path...>` | After automatic refresh, remove paths from the index; source files are not deleted. A later refresh can restore eligible files. | JSON update statistics |
101
127
 
102
128
  Manual maintenance examples:
@@ -104,6 +130,8 @@ Manual maintenance examples:
104
130
  ```bash
105
131
  slopdex update-git
106
132
  slopdex update-files src/service.ts src/model.ts
133
+ slopdex reindex-files
134
+ slopdex reindex-files --callables
107
135
  slopdex delete-files src/removed.ts
108
136
  slopdex update-git --target HEAD --rebuild-on-divergence
109
137
  slopdex update-git --force-reindex
@@ -119,7 +147,8 @@ slopdex update-git --force-reindex
119
147
  | `--provider <openai\|jina>` | Embedding provider; `openai` by default. |
120
148
  | `--model <name>` | Embedding model; `text-embedding-3-large` for OpenAI, `jina-embeddings-v4` for Jina. |
121
149
  | `--dimensions <number>` | Positive embedding dimension count; OpenAI `3072`, Jina `1024`. Must be supported by the model. |
122
- | `--description-model <name>` | OpenAI description model; `gpt-5.6-sol` initially, then the persisted model unless overridden. |
150
+ | `--description-provider <openai\|opencode\|opencode-go>` | Description provider; OpenAI by default. OpenCode values use Zen or Go with `OPENCODE_API_KEY`. |
151
+ | `--description-model <name>` | Description model; `gpt-5.6-sol` for OpenAI/Zen and `gpt-5.6-luna` for Go, then the persisted model unless overridden. Published OpenCode models use their documented protocol. |
123
152
  | `--ignore-errors` | Silence warnings about saved indexing errors; records remain available. |
124
153
  | `-h`, `--help` | Show CLI usage without refreshing. |
125
154
  | `--version` | Print the package version and exit. |
@@ -130,18 +159,17 @@ Explicit relative config and index paths resolve from the current directory, not
130
159
 
131
160
  | Argument | Applies to | Meaning / default |
132
161
  | --- | --- | --- |
133
- | `--limit <number>` | Both query searches, cross-search, cohesion | Positive integer. Query matches: `10`; cross-search neighbors per source: `5`; cohesion reported pairs and file rows: `50`. |
134
- | `--threshold <number\|min-max>` | Both query searches, cross-search, cohesion | Minimum similarity, or range with inclusive minimum and exclusive maximum. Default `-1` for query/cross-search, `0.8` for cohesion. Cohesion minimum must be at least `-1` and below `1`. |
135
- | `--format <json\|summary\|clusters>` | Both query searches, cross-search, cohesion, index-errors | Output format; see the commands table. `clusters` is only for cross-search. |
136
- | `-e <regex>`, `--regexp <regex>`, `--regex <regex>` | Both query searches, cross-search, cohesion | Equivalent case-sensitive JavaScript regex options over qualified names. Query searches: filter results before limiting. Analysis: filter sources only. |
137
- | `--min-lines <number>` | Cross-search, cohesion | Minimum source and candidate callable length; positive integer, default `2`. Use `1` to include one-line wrappers. |
138
- | `--source-path <path>` | Cross-search, cohesion | Select sources in a file or recursive directory, relative to the repository root (or absolute within it). |
139
- | `--changed-since <commit>` | Cross-search, cohesion | Select added, modified, or moved functions relative to an ancestor of the indexed Git checkpoint, including working-tree changes. Requires Git. |
140
- | `--uncommitted` | Cross-search, cohesion | Select functions indexed from working-tree files: staged, unstaged, or untracked changes in Git; all working-tree functions without Git. |
162
+ | `--limit <number>` | Both query searches, cross-search | Positive integer. Query matches: `10`; cross-search neighbors per source: `5`. |
163
+ | `--threshold <number\|min-max>` | Both query searches, cross-search | Minimum similarity, or range with inclusive minimum and exclusive maximum. Default `-1`. |
164
+ | `--format <json\|summary\|clusters>` | Both query searches, cross-search, index-errors | Output format; see the commands table. `clusters` is only for ordinary cross-search. |
165
+ | `-e <regex>`, `--regexp <regex>`, `--regex <regex>` | Both query searches, cross-search | Equivalent case-sensitive JavaScript regex options over qualified names. Query searches filter results before limiting; cross-search filters sources only. |
166
+ | `--min-lines <number>` | Cross-search | Minimum source and candidate callable length; positive integer, default `2`. Use `1` to include one-line wrappers. |
167
+ | `--source-path <path>` | Cross-search | Select sources in a file or recursive directory, relative to the repository root (or absolute within it). |
168
+ | `--changed-since <commit>` | Cross-search | Select added, modified, or moved functions relative to an ancestor of the indexed Git checkpoint, including working-tree changes. Requires Git. |
169
+ | `--uncommitted` | Cross-search | Select functions indexed from working-tree files: staged, unstaged, or untracked changes in Git; all working-tree functions without Git. |
141
170
  | `--cross-file-only` | Cross-search | Exclude matches from the same physical file. |
142
171
  | `--include-symmetric-duplicates` | Cross-search | Allow both directions of same-index matches; otherwise each unordered pair is emitted once. |
143
- | `--neighbors <number>` | Cohesion | Neighbors considered per source; positive integer, default `20`. Changes the analysis graph. |
144
- | `--include-source` | Cohesion JSON | Include callable bodies; omitted by default. |
172
+ | `--cohesion` | Cross-search | Add `physicalDistance` and re-rank each source's matches by descending distance, with similarity as the tie-breaker. Defaults to summary output; incompatible with clusters. |
145
173
  | `--target-root <path>` | Cross-search | Second repository root; requires `--target-index`. |
146
174
  | `--target-index <path>` | Cross-search | Second index file; requires `--target-root`. |
147
175
  | `--target-config <path>` | Cross-search | Target config; defaults to `<target-root>/.slopdex/config.json`. Requires both target options. |
@@ -168,6 +196,8 @@ slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.85-0.9 --limi
168
196
 
169
197
  Similarity is a model-dependent score, not a probability of duplication. Higher scores mean greater semantic resemblance. Query summaries show `score path :: qualifiedName`; cross-search summaries group those lines beneath each source. Functions without matches are omitted from cross-search output.
170
198
 
199
+ With reranking enabled, query summaries show `rerankScore rerank (similarity similarity)`. JSON retains `similarity` and adds `rerankScore`. The similarity threshold first filters embedding candidates. Cohere and Jina receive up to five times the requested limit. The OpenAI LLM receives the configured number of top embedding results, 10 by default, or the requested result limit when it is larger. Candidates include function descriptions when available and function metadata/source code. Cross-search and cohesion analysis are not sent to rerankers.
200
+
171
201
  ```text
172
202
  Cluster 1 (3 functions, similarity 0.9124-0.9568)
173
203
  src/auth/session.ts:18:1 :: validateSession
@@ -179,42 +209,33 @@ Cluster 1 (3 functions, similarity 0.9124-0.9568)
179
209
  - Clusters sort by member count, then name. Cluster number is not severity.
180
210
  - Locations identify where to inspect behavior, callers, and architectural roles. Wrappers, adapters, tests, and separate interface implementations can legitimately resemble one another.
181
211
 
182
- ### Cohesion
212
+ ### Physical cohesion
183
213
 
184
214
  ```text
185
- Cohesion: 184 functions analyzed, 37 semantic edges
186
- same file 35.1% same folder 29.7% remote 35.2% mean distance 1.84
187
-
188
- 1. gap 0.6053 similarity 0.9400 distance 4 reciprocal
189
- src/auth/session.ts:18:1 :: validateSession
190
- packages/http/middleware.ts:42:1 :: authenticate
215
+ src/auth/session.ts :: validateSession
216
+ 0.9400 packages/http/middleware.ts :: authenticate [distance 4]
217
+ 0.9300 src/auth/token.ts :: validateToken [distance 1]
191
218
  ```
192
219
 
193
- | Field | Interpretation |
194
- | --- | --- |
195
- | Functions analyzed / semantic edges | Selected-source coverage / unique qualifying neighbor pairs, before report limiting. Not quality scores. |
196
- | Same file / same folder / remote | Shares of weighted semantic affinity. Higher remote affinity means more related code crosses folder boundaries. |
197
- | Mean distance | Weighted physical separation: `0` for the same file, `1` for different files in one folder, larger across folders. |
198
- | Gap / rank | A `0–1` review score combining similarity above the threshold and separation; higher gap ranks first. Same-file pairs have zero gap. |
199
- | Reciprocal | Both functions selected each other as neighbors. JSON `null` means the other endpoint was not evaluated under source filtering. |
200
- | `sourceTestPair` | A source/test relationship inferred from paths; separation may be intentional. |
201
- | `externalAffinityRatio` | In JSON file reports, the share of observed affinity outside that file's folder. |
202
-
203
- The example suggests reviewing separated authentication responsibilities. It does not establish that they belong in one module. There is no universal cohesion pass threshold. Filtered reports describe selected sources, not the entire repository. Cohesion metrics cover all qualifying edges; reported pairs/files are limited, and groups are built from reported pairs.
220
+ Run cross-search with `--cohesion` to put physically distant matches first. Distance is `0` within one file, `1` between files in one folder, and `1` plus directory-tree hops across folders. The option only changes the order of each source's selected semantic matches; it does not change similarity scores or establish that distant code belongs together.
204
221
 
205
- For automation, pass `--format json`: query searches and diagnostics return JSON arrays; cross-search returns **JSONL**, one row per matched source; cohesion returns one JSON object containing `repository`, `parameters`, `metrics`, `pairs`, `files`, and `groups`. Results go to stdout; notices and warnings go to stderr.
222
+ For automation, pass `--format json`: query searches and diagnostics return JSON arrays; cross-search returns **JSONL**, one row per matched source. With `--cohesion`, each match includes `physicalDistance`. Results go to stdout; notices and warnings go to stderr.
206
223
 
207
224
  ## System behavior
208
225
 
209
226
  ### Descriptions and scoring
210
227
 
211
- Descriptions are optional and disabled initially. `descriptions enable` persists the selected description model and keeps descriptions current on later updates, including file-context and path changes. Use `slopdex descriptions enable --description-model <model-id>` to change it, or `slopdex descriptions disable` to stop automatic updates and description-based searching/scoring while retaining cached descriptions. `status` exposes `descriptionsEnabled`, `descriptionCount`, and `descriptionProfile`.
228
+ Descriptions are optional and disabled initially. `descriptions enable` persists the selected provider and model and keeps callable descriptions current on later updates. Use `slopdex descriptions enable --description-provider opencode-go` for OpenCode Go, or combine `--description-provider` and `--description-model` to change both. `slopdex descriptions disable` stops automatic updates and description-based searching/scoring while retaining cached descriptions. `status` exposes callable/file description counts, stale file-description count, enabled state, and profile.
229
+
230
+ Descriptions are generated in source order through one conversation per file. Instructions and complete file source form a stable prefix; Slopdex asks for the overall file description first, then each callable request and answer extends that conversation. This allows supported providers to reuse their prompt cache instead of receiving a separate duplicated file context for every callable.
231
+
232
+ Ordinary updates preserve an existing file description even when its source changes, while still refreshing callable descriptions. `slopdex reindex-files` explicitly regenerates stale file descriptions and their embeddings; add `--callables` to continue through and replace every callable description in those files.
212
233
 
213
234
  Tree-sitter extraction, generated descriptions, and document/query vectors are content-addressed in the same SQLite database. Each validated result is committed immediately, independently of the final logical index update. If indexing is interrupted or a later provider call fails, rerunning reuses every completed result whose profile, operation, input, and source context hash still match.
214
235
 
215
- `search` always searches code; `search-description` always searches purpose descriptions. When all callables have enabled descriptions, cross-search and cohesion automatically use **50% code similarity + 50% description similarity**. Cross-repository analysis needs complete descriptions on both sides; otherwise the entire analysis uses code-only scores. Thresholds and neighbor limits apply to the selected score.
236
+ With complete descriptions, `search` and cross-search average **one-third code similarity + one-third callable-description similarity + one-third file-description similarity**. `search-description` averages callable and file descriptions without code. Cross-repository analysis needs complete descriptions on both sides; otherwise the entire analysis uses code-only scores. Stale file descriptions remain searchable until explicitly reindexed. Thresholds and limits apply to the selected score.
216
237
 
217
- Text output labels combined scores. JSON exposes `codeSimilarity`, `descriptionSimilarity`, and scoring mode/weights (`scoring` for cross-search, `parameters` for cohesion). Compare runs only with matching scoring mode, weights, embedding and description-generator profiles, threshold, neighbor count, and source/candidate filters.
238
+ Text output labels combined scores. JSON exposes `codeSimilarity`, `descriptionSimilarity`, `fileDescriptionSimilarity`, and cross-search scoring mode/weights. Compare runs only with matching scoring mode, weights, embedding and description-generator profiles, threshold, and source/candidate filters.
218
239
 
219
240
  ### Exclusions
220
241
 
@@ -228,17 +249,22 @@ Parse, extraction, read, and file-size failures are saved while healthy callable
228
249
 
229
250
  Saved failures trigger stderr warnings, including on cached runs, help, and cross-search targets. `--ignore-errors` silences warnings without clearing records. Updates retry failed files; successful indexing, deletion, or exclusion clears their diagnostics. Version output bypasses diagnostics.
230
251
 
231
- Use the recovery flag named in the error: `--rebuild-on-divergence` for Git history changes, `--force-reindex` for incompatible indexes. Schema versions before 5 require `--force-reindex`; schema-5 rebuilds preserve reusable artifact caches. For provider/authentication failures, fix the reported configuration and rerun. For source-change-during-indexing errors, rerun after edits settle. Exit status is `0` on success, `2` for argument/domain errors, and `1` for other failures (or invocation without a command).
252
+ Use the recovery flag named in the error: `--rebuild-on-divergence` for Git history changes, `--force-reindex` for incompatible indexes. Schema versions 6 and 7 migrate in place; earlier schemas require `--force-reindex`, with compatible rebuilds preserving reusable artifact caches. For provider/authentication failures, fix the reported configuration and rerun. For source-change-during-indexing errors, rerun after edits settle. Exit status is `0` on success, `2` for argument/domain errors, and `1` for other failures (or invocation without a command).
232
253
 
233
254
  ## Configuration
234
255
 
235
- Optional file: `<root>/.slopdex/config.json`. Example using Jina (requires `JINA_API_KEY`):
256
+ Optional file: `<root>/.slopdex/config.json`. Example using Jina embeddings, OpenAI LLM reranking, and OpenCode Go descriptions (requires `JINA_API_KEY`, `OPENAI_API_KEY` for query searches, plus `OPENCODE_API_KEY` when descriptions are enabled):
236
257
 
237
258
  ```json
238
259
  {
239
260
  "provider": "jina",
240
261
  "model": "jina-embeddings-v4",
241
262
  "dimensions": 1024,
263
+ "rerankingEnabled": true,
264
+ "rerankerProvider": "openai",
265
+ "rerankerModel": "gpt-5.6-luna",
266
+ "rerankerCandidates": 10,
267
+ "descriptionProvider": "opencode-go",
242
268
  "exclude": ["**/fixtures/**"]
243
269
  }
244
270
  ```
@@ -246,14 +272,20 @@ Optional file: `<root>/.slopdex/config.json`. Example using Jina (requires `JINA
246
272
  | Property | Purpose / default |
247
273
  | --- | --- |
248
274
  | `provider`, `model`, `dimensions` | Embedding settings; defaults are listed in the CLI table. |
249
- | `descriptionModel` | Description model; initially `gpt-5.6-sol`. |
275
+ | `descriptionProvider` | Description provider: `openai`, `opencode` (Zen), or `opencode-go`; defaults to `openai`. |
276
+ | `descriptionModel` | Description model; provider default unless explicitly set. |
277
+ | `descriptionsEnabled` | When true or false, the next index-using command applies that enabled state during its normal refresh. Unset leaves persisted index state unchanged. |
278
+ | `rerankingEnabled` | Enables second-stage ranking for `search` and `search-description`; disabled/unset by default. Prefer changing it through `config reranker`. |
279
+ | `rerankerProvider` | Reranker: `cohere`, `jina`, or `openai`. OpenAI uses an LLM rather than a dedicated reranking endpoint. |
280
+ | `rerankerModel` | Provider model; defaults to Cohere `rerank-v4.0-pro`, Jina `jina-reranker-v3.5`, or OpenAI `gpt-5.6-luna`. |
281
+ | `rerankerCandidates` | Embedding-ranked candidates sent to the OpenAI LLM; integer from `1` to `100`, default `10`. The requested result limit takes precedence when larger, up to 100. |
250
282
  | `indexPath` | Index location; `<root>/.slopdex/index.sqlite`. |
251
283
  | `include` | Repository-relative glob array; empty/unset includes all supported eligible files. |
252
284
  | `exclude` | Additional repository-relative exclusion globs. |
253
285
  | `maxFileSize` | Maximum source-file size in bytes; positive integer, default `1048576`. |
254
286
  | `embeddingBatchSize` | Embedding inputs per batch; positive integer, default `32`. |
255
287
 
256
- Keep keys in the environment (`OPENAI_API_KEY`, `JINA_API_KEY`). Changing the embedding profile requires rebuilding with `--force-reindex`.
288
+ Keep keys in the environment (`OPENAI_API_KEY`, `JINA_API_KEY`, `COHERE_API_KEY`, `OPENCODE_API_KEY`). Reranker settings do not change the stored index and do not require a rebuild. Changing the embedding profile requires rebuilding with `--force-reindex`.
257
289
 
258
290
  ## Development
259
291