@ninjaxtools/slopdex 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/slopdex/SKILL.md +33 -32
- package/README.md +45 -38
- package/dist/cli.js +583 -462
- package/dist/cli.js.map +1 -1
- package/dist/index.d.ts +91 -2
- package/dist/index.js +393 -27
- package/dist/index.js.map +1 -1
- package/docs/implementation.md +14 -8
- package/package.json +1 -1
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: slopdex
|
|
3
|
-
description: Semantic code search,
|
|
3
|
+
description: Semantic code search, duplicate-function candidates, and physical-distance re-ranking
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Slopdex operator guide for agents
|
|
@@ -16,6 +16,16 @@ slopdex search "persist user data" -e 'save|persist' --format summary --limit 5
|
|
|
16
16
|
|
|
17
17
|
Describe behavior rather than guessing a symbol name. `-e` is a regex that restricts which symbols (functions) are searched.
|
|
18
18
|
|
|
19
|
+
Hosted reranking is optional and persists in repository config:
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
slopdex config reranker cohere
|
|
23
|
+
# Or: slopdex config reranker jina
|
|
24
|
+
# Or use an LLM: slopdex config reranker openai
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Set `COHERE_API_KEY`, `JINA_API_KEY`, or `OPENAI_API_KEY` respectively. The OpenAI LLM reranker defaults to `gpt-5.6-luna`, high reasoning, and the top 10 embedding candidates; configure the pool with `slopdex config reranker openai --reranker-candidates 20`. Disable reranking with `slopdex config reranker disable`. Reranking applies to `search` and `search-description`, not cross-search. It preserves embedding `similarity`, adds `rerankScore`, and orders a wider candidate set by that score.
|
|
28
|
+
|
|
19
29
|
### Search function purpose
|
|
20
30
|
|
|
21
31
|
Code purpose-description generation needs to be enabled once:
|
|
@@ -75,7 +85,7 @@ slopdex cross-search --changed-since origin/main --format summary --threshold 0.
|
|
|
75
85
|
slopdex cross-search --source-path src/services -e '^UserService\.' --format summary --threshold 0.9
|
|
76
86
|
```
|
|
77
87
|
|
|
78
|
-
These restrict sources while searching the full eligible index.
|
|
88
|
+
These restrict sources while searching the full eligible index. All supplied restrictions intersect:
|
|
79
89
|
|
|
80
90
|
```bash
|
|
81
91
|
slopdex cross-search --source-path src -e 'validate' \
|
|
@@ -88,11 +98,11 @@ Here a source must have changed since the commit and belong to an uncommitted fi
|
|
|
88
98
|
### Review physical cohesion
|
|
89
99
|
|
|
90
100
|
```bash
|
|
91
|
-
slopdex cohesion --threshold 0.8 --
|
|
92
|
-
slopdex cohesion --source-path src/services --threshold 0.8 --format summary
|
|
101
|
+
slopdex cross-search --cohesion --threshold 0.8 --limit 20 --format summary
|
|
102
|
+
slopdex cross-search --cohesion --source-path src/services --threshold 0.8 --format summary
|
|
93
103
|
```
|
|
94
104
|
|
|
95
|
-
|
|
105
|
+
`--cohesion` keeps cross-search's semantic matches and orders each source's matches from greatest to least physical path distance. Similarity breaks distance ties. Summary output includes the distance; JSONL matches include `physicalDistance`.
|
|
96
106
|
|
|
97
107
|
### Compare repositories
|
|
98
108
|
|
|
@@ -142,6 +152,7 @@ Usage: `slopdex <command> [arguments] [options]`. Quote queries and regexes. Boo
|
|
|
142
152
|
| `--dimensions <number>` | Positive dimensions supported by the model; OpenAI `3072`, Jina `1024`. |
|
|
143
153
|
| `--description-provider <openai\|opencode\|opencode-go>` | Description provider; OpenAI by default. OpenCode values require `OPENCODE_API_KEY`. |
|
|
144
154
|
| `--description-model <name>` | Description model; `gpt-5.6-sol` for OpenAI/Zen and `gpt-5.6-luna` for Go. |
|
|
155
|
+
| `--reranker-candidates <number>` | With `config reranker openai`, embedding-ranked functions sent to the LLM; range `1`-`100`, default `10`. |
|
|
145
156
|
| `--ignore-errors` | Silence saved-diagnostic warnings without deleting records. |
|
|
146
157
|
| `-h`, `--help` | Usage; no refresh. |
|
|
147
158
|
| `--version` | Package version; exits without refresh or saved-diagnostic warnings. |
|
|
@@ -152,18 +163,17 @@ Explicit relative config/index paths resolve from the current directory. Source
|
|
|
152
163
|
|
|
153
164
|
| Argument | Applies to / behavior |
|
|
154
165
|
| --- | --- |
|
|
155
|
-
| `--limit <number>` | Positive integer. `search`/`search-description`: matches, default `10`. Cross-search: neighbors per source, default `5`.
|
|
156
|
-
| `--threshold <number\|min-max>` | Both query searches and
|
|
157
|
-
| `--format <json\|summary\|clusters>` | Both query searches, cross-search,
|
|
158
|
-
| `-e <regex>`, `--regexp <regex>`, `--regex <regex>` | Equivalent case-sensitive JavaScript regex options on qualified names. Query searches filter results before limiting; cross-search
|
|
159
|
-
| `--min-lines <number>` | Cross-search
|
|
160
|
-
| `--source-path <path>` | Cross-search
|
|
161
|
-
| `--changed-since <commit>` | Cross-search
|
|
162
|
-
| `--uncommitted` | Cross-search
|
|
166
|
+
| `--limit <number>` | Positive integer. `search`/`search-description`: matches, default `10`. Cross-search: neighbors per source, default `5`. |
|
|
167
|
+
| `--threshold <number\|min-max>` | Both query searches and cross-search. Inclusive minimum or half-open range; default `-1`. |
|
|
168
|
+
| `--format <json\|summary\|clusters>` | Both query searches, cross-search, and index-errors. Cohesion-ranked cross-search supports summary or JSONL, not clusters. |
|
|
169
|
+
| `-e <regex>`, `--regexp <regex>`, `--regex <regex>` | Equivalent case-sensitive JavaScript regex options on qualified names. Query searches filter results before limiting; cross-search filters sources only. |
|
|
170
|
+
| `--min-lines <number>` | Cross-search: positive source/candidate length minimum, default `2`. |
|
|
171
|
+
| `--source-path <path>` | Cross-search: source file or recursive directory within the root. |
|
|
172
|
+
| `--changed-since <commit>` | Cross-search: added, modified, or moved functions since an ancestor of the indexed Git checkpoint, including working-tree changes. Requires Git. |
|
|
173
|
+
| `--uncommitted` | Cross-search: functions indexed from working-tree files; in Git these are staged, unstaged, or untracked changes. Without Git this selects all working-tree functions. |
|
|
163
174
|
| `--cross-file-only` | Cross-search: exclude same-physical-file matches. |
|
|
164
175
|
| `--include-symmetric-duplicates` | Cross-search: allow both directions of same-index matches; otherwise each unordered pair appears once. |
|
|
165
|
-
| `--
|
|
166
|
-
| `--include-source` | Cohesion JSON: include callable bodies; omitted by default. |
|
|
176
|
+
| `--cohesion` | Cross-search: add physical distance and order each source's matches from farthest to nearest. Defaults to summary output. |
|
|
167
177
|
| `--target-root <path>` | Cross-search: second repository; requires `--target-index`. |
|
|
168
178
|
| `--target-index <path>` | Cross-search: second index file; requires `--target-root`. |
|
|
169
179
|
| `--target-config <path>` | Cross-search: target config, default `<target-root>/.slopdex/config.json`; requires both target options. |
|
|
@@ -188,12 +198,13 @@ Use `--no-reindex` when the task calls for committed-only results or reuse of an
|
|
|
188
198
|
| --- | --- | --- |
|
|
189
199
|
| `search`, `search-description` | `summary` | JSON array; purpose search includes generated description text |
|
|
190
200
|
| `cross-search` | `clusters` | `summary`, or `json` for JSONL with one row per matched source |
|
|
191
|
-
| `cohesion` | `summary` | One JSON report |
|
|
192
201
|
| `index-errors` | `summary` | JSON array |
|
|
193
202
|
| `status`, update commands, `descriptions` | JSON object | — |
|
|
194
203
|
|
|
195
204
|
Prefer summary output for compact source review, clusters for duplicate families, and JSON/JSONL for structured processing. Stdout carries results; stderr carries notices and warnings. Cross-search omits sources without emitted matches. Empty output means no findings under the chosen coverage/filters, not proof that no similar code exists.
|
|
196
205
|
|
|
206
|
+
With reranking enabled, query summaries display both reranker relevance and embedding similarity. Similarity thresholds filter candidates before reranking; limits apply to the reranked output. LLM candidate documents include descriptions when available and function metadata/source code.
|
|
207
|
+
|
|
197
208
|
### Similarity and clusters
|
|
198
209
|
|
|
199
210
|
```text
|
|
@@ -212,26 +223,16 @@ When reporting candidates, identify paths/symbols, summarize the shared behavior
|
|
|
212
223
|
|
|
213
224
|
### Purpose-aware scoring
|
|
214
225
|
|
|
215
|
-
When descriptions are complete, `search
|
|
226
|
+
When descriptions are complete, `search` and cross-search average code, callable-description, and file-description similarity with equal one-third weights. `search-description` averages callable and file descriptions. Cross-repository analysis needs completeness on both sides; otherwise all scores are code-only. Stale file descriptions remain in scoring until `reindex-files` refreshes them. Description-generator models may differ even though embedding profiles must match.
|
|
216
227
|
|
|
217
|
-
Thresholds and limits apply to the selected score. Text labels combined scoring; JSON includes component scores and mode/weights
|
|
228
|
+
Thresholds and limits apply to the selected score. Text labels combined scoring; JSON includes component scores and mode/weights in `scoring`. Compare runs only with matching scoring mode, weights, embedding and description-generator profiles, threshold, and source/candidate filters.
|
|
218
229
|
|
|
219
230
|
### Cohesion
|
|
220
231
|
|
|
221
232
|
```text
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
1. gap 0.6053 similarity 0.9400 distance 4 reciprocal
|
|
226
|
-
src/auth/session.ts:18:1 :: validateSession
|
|
227
|
-
packages/http/middleware.ts:42:1 :: authenticate
|
|
233
|
+
src/auth/session.ts :: validateSession
|
|
234
|
+
0.9400 packages/http/middleware.ts :: authenticate [distance 4]
|
|
235
|
+
0.9300 src/auth/token.ts :: validateToken [distance 1]
|
|
228
236
|
```
|
|
229
237
|
|
|
230
|
-
-
|
|
231
|
-
- **Same file / same folder / remote:** shares of weighted semantic affinity. More remote affinity means more related code crosses folder boundaries.
|
|
232
|
-
- **Mean distance:** weighted physical separation; `0` is same file, `1` is different files in one folder, larger means farther apart.
|
|
233
|
-
- **Gap / rank:** a `0–1` review score combining similarity above the threshold with separation. Higher gap ranks earlier; same-file pairs have zero gap.
|
|
234
|
-
- **Reciprocal:** both functions selected each other as neighbors. JSON `null` means an endpoint was not evaluated because of source filtering.
|
|
235
|
-
- **JSON details:** `semanticWeight` reflects similarity above threshold; `separationWeight` reflects distance; `sourceTestPair` flags a path-inferred source/test relationship; file `externalAffinityRatio` measures affinity outside that file's folder.
|
|
236
|
-
|
|
237
|
-
The example merits reviewing separated authentication responsibilities, while accounting for intentional layering. There is no universal pass/fail threshold. Use comparable runs to evaluate changes. Filtered reports describe selected sources, not the full repository. Cohesion metrics use all qualifying edges; pairs/files are limited, and groups use reported pairs.
|
|
238
|
+
Physical distance is `0` within one file, `1` between files in one folder, and `1` plus directory-tree hops across folders. The option only reorders the selected semantic matches; it does not change similarity or prove that distant code should be moved. Review architectural layers, tests, adapters, and other intentional separation before recommending consolidation.
|
package/README.md
CHANGED
|
@@ -24,6 +24,17 @@ Add `.slopdex/` to your repository's `.gitignore`.
|
|
|
24
24
|
slopdex search "keep the repository index synchronized" --format summary --limit 10
|
|
25
25
|
```
|
|
26
26
|
|
|
27
|
+
Optionally enable a hosted second-stage reranker for `search` and `search-description`:
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
export COHERE_API_KEY="your-api-key"
|
|
31
|
+
slopdex config reranker cohere
|
|
32
|
+
# Or: export JINA_API_KEY="your-api-key" && slopdex config reranker jina
|
|
33
|
+
# Or use an LLM: export OPENAI_API_KEY="your-api-key" && slopdex config reranker openai
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
The config command is the only CLI switch for reranking. Use `slopdex config reranker disable` to return to embedding-only ordering. An optional final argument selects a model, for example `slopdex config reranker cohere rerank-v4.0-fast`. OpenAI LLM reranking defaults to `gpt-5.6-luna` with high reasoning and receives the top 10 embedding results; change the pool with `--reranker-candidates`, for example `slopdex config reranker openai gpt-5.6-luna --reranker-candidates 20`.
|
|
37
|
+
|
|
27
38
|
To search generated descriptions of each function's role instead:
|
|
28
39
|
|
|
29
40
|
```bash
|
|
@@ -62,15 +73,15 @@ slopdex cross-search --changed-since origin/main --format summary --threshold 0.
|
|
|
62
73
|
slopdex cross-search --source-path src/services -e '^UserService\.' --format summary --threshold 0.9
|
|
63
74
|
```
|
|
64
75
|
|
|
65
|
-
These select source functions while keeping the full eligible index available for matches.
|
|
76
|
+
These select source functions while keeping the full eligible index available for matches.
|
|
66
77
|
|
|
67
78
|
### Find related code stored far apart
|
|
68
79
|
|
|
69
80
|
```bash
|
|
70
|
-
slopdex cohesion --threshold 0.8 --
|
|
81
|
+
slopdex cross-search --cohesion --threshold 0.8 --limit 20 --format summary
|
|
71
82
|
```
|
|
72
83
|
|
|
73
|
-
|
|
84
|
+
`--cohesion` keeps the semantic matches selected by cross-search, annotates them with physical distance, and orders each source's matches from farthest to nearest. Similarity breaks distance ties.
|
|
74
85
|
|
|
75
86
|
### Compare repositories
|
|
76
87
|
|
|
@@ -102,11 +113,11 @@ Usage: `slopdex <command> [arguments] [options]`.
|
|
|
102
113
|
| `models [opencode\|opencode-go]` | Fetch valid models from the current published Zen and/or Go catalogs. | Qualified `provider/model` lines; optional JSON array |
|
|
103
114
|
| `config model <model\|provider/model>` | Validate a published OpenCode model and persist its provider/model selection without opening an index. Bare IDs auto-resolve only when unambiguous. | Updated setting summary; optional JSON |
|
|
104
115
|
| `config descriptions <enable\|disable>` | Persist whether the next index-using command should enable or disable descriptions. Does not open an index. | Updated setting summary; optional JSON |
|
|
116
|
+
| `config reranker <cohere\|jina\|openai\|disable> [model]` | Enable a hosted or OpenAI LLM query reranker, optionally selecting a model, or disable it. OpenAI accepts `--reranker-candidates <number>` from 1 to 100 and defaults to 10. Does not open an index. | Updated setting summary; optional JSON |
|
|
105
117
|
| `search <query>` | Search function code by meaning. Quote multiword queries. | Summary; optional JSON array |
|
|
106
118
|
| `descriptions <enable\|disable>` | Enable or disable automatic purpose descriptions. Re-enabling with unchanged inputs reuses cached descriptions. | JSON statistics |
|
|
107
119
|
| `search-description <query>` | Search purpose descriptions after enabling them. | Summary including description text; optional JSON array |
|
|
108
120
|
| `cross-search` | Find neighbors for each selected function in this or another index. | `clusters` by default; optional `summary` or JSONL |
|
|
109
|
-
| `cohesion` | Analyze semantic relationships versus file/folder separation. | Summary; optional JSON report |
|
|
110
121
|
| `status` | Refresh and show index metadata, counts, and profiles. | JSON object |
|
|
111
122
|
| `index-errors` | Read saved file/function indexing failures. | Summary; optional JSON array |
|
|
112
123
|
| `update-git` | Explicitly refresh a Git snapshot, with current working-tree changes when targeting HEAD. | JSON update statistics |
|
|
@@ -148,18 +159,17 @@ Explicit relative config and index paths resolve from the current directory, not
|
|
|
148
159
|
|
|
149
160
|
| Argument | Applies to | Meaning / default |
|
|
150
161
|
| --- | --- | --- |
|
|
151
|
-
| `--limit <number>` | Both query searches, cross-search
|
|
152
|
-
| `--threshold <number\|min-max>` | Both query searches, cross-search
|
|
153
|
-
| `--format <json\|summary\|clusters>` | Both query searches, cross-search,
|
|
154
|
-
| `-e <regex>`, `--regexp <regex>`, `--regex <regex>` | Both query searches, cross-search
|
|
155
|
-
| `--min-lines <number>` | Cross-search
|
|
156
|
-
| `--source-path <path>` | Cross-search
|
|
157
|
-
| `--changed-since <commit>` | Cross-search
|
|
158
|
-
| `--uncommitted` | Cross-search
|
|
162
|
+
| `--limit <number>` | Both query searches, cross-search | Positive integer. Query matches: `10`; cross-search neighbors per source: `5`. |
|
|
163
|
+
| `--threshold <number\|min-max>` | Both query searches, cross-search | Minimum similarity, or range with inclusive minimum and exclusive maximum. Default `-1`. |
|
|
164
|
+
| `--format <json\|summary\|clusters>` | Both query searches, cross-search, index-errors | Output format; see the commands table. `clusters` is only for ordinary cross-search. |
|
|
165
|
+
| `-e <regex>`, `--regexp <regex>`, `--regex <regex>` | Both query searches, cross-search | Equivalent case-sensitive JavaScript regex options over qualified names. Query searches filter results before limiting; cross-search filters sources only. |
|
|
166
|
+
| `--min-lines <number>` | Cross-search | Minimum source and candidate callable length; positive integer, default `2`. Use `1` to include one-line wrappers. |
|
|
167
|
+
| `--source-path <path>` | Cross-search | Select sources in a file or recursive directory, relative to the repository root (or absolute within it). |
|
|
168
|
+
| `--changed-since <commit>` | Cross-search | Select added, modified, or moved functions relative to an ancestor of the indexed Git checkpoint, including working-tree changes. Requires Git. |
|
|
169
|
+
| `--uncommitted` | Cross-search | Select functions indexed from working-tree files: staged, unstaged, or untracked changes in Git; all working-tree functions without Git. |
|
|
159
170
|
| `--cross-file-only` | Cross-search | Exclude matches from the same physical file. |
|
|
160
171
|
| `--include-symmetric-duplicates` | Cross-search | Allow both directions of same-index matches; otherwise each unordered pair is emitted once. |
|
|
161
|
-
| `--
|
|
162
|
-
| `--include-source` | Cohesion JSON | Include callable bodies; omitted by default. |
|
|
172
|
+
| `--cohesion` | Cross-search | Add `physicalDistance` and re-rank each source's matches by descending distance, with similarity as the tie-breaker. Defaults to summary output; incompatible with clusters. |
|
|
163
173
|
| `--target-root <path>` | Cross-search | Second repository root; requires `--target-index`. |
|
|
164
174
|
| `--target-index <path>` | Cross-search | Second index file; requires `--target-root`. |
|
|
165
175
|
| `--target-config <path>` | Cross-search | Target config; defaults to `<target-root>/.slopdex/config.json`. Requires both target options. |
|
|
@@ -186,6 +196,8 @@ slopdex cross-search --cross-file-only --min-lines 4 --threshold 0.85-0.9 --limi
|
|
|
186
196
|
|
|
187
197
|
Similarity is a model-dependent score, not a probability of duplication. Higher scores mean greater semantic resemblance. Query summaries show `score path :: qualifiedName`; cross-search summaries group those lines beneath each source. Functions without matches are omitted from cross-search output.
|
|
188
198
|
|
|
199
|
+
With reranking enabled, query summaries show `rerankScore rerank (similarity similarity)`. JSON retains `similarity` and adds `rerankScore`. The similarity threshold first filters embedding candidates. Cohere and Jina receive up to five times the requested limit. The OpenAI LLM receives the configured number of top embedding results, 10 by default, or the requested result limit when it is larger. Candidates include function descriptions when available and function metadata/source code. Cross-search and cohesion analysis are not sent to rerankers.
|
|
200
|
+
|
|
189
201
|
```text
|
|
190
202
|
Cluster 1 (3 functions, similarity 0.9124-0.9568)
|
|
191
203
|
src/auth/session.ts:18:1 :: validateSession
|
|
@@ -197,30 +209,17 @@ Cluster 1 (3 functions, similarity 0.9124-0.9568)
|
|
|
197
209
|
- Clusters sort by member count, then name. Cluster number is not severity.
|
|
198
210
|
- Locations identify where to inspect behavior, callers, and architectural roles. Wrappers, adapters, tests, and separate interface implementations can legitimately resemble one another.
|
|
199
211
|
|
|
200
|
-
###
|
|
212
|
+
### Physical cohesion
|
|
201
213
|
|
|
202
214
|
```text
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
1. gap 0.6053 similarity 0.9400 distance 4 reciprocal
|
|
207
|
-
src/auth/session.ts:18:1 :: validateSession
|
|
208
|
-
packages/http/middleware.ts:42:1 :: authenticate
|
|
215
|
+
src/auth/session.ts :: validateSession
|
|
216
|
+
0.9400 packages/http/middleware.ts :: authenticate [distance 4]
|
|
217
|
+
0.9300 src/auth/token.ts :: validateToken [distance 1]
|
|
209
218
|
```
|
|
210
219
|
|
|
211
|
-
|
|
212
|
-
| --- | --- |
|
|
213
|
-
| Functions analyzed / semantic edges | Selected-source coverage / unique qualifying neighbor pairs, before report limiting. Not quality scores. |
|
|
214
|
-
| Same file / same folder / remote | Shares of weighted semantic affinity. Higher remote affinity means more related code crosses folder boundaries. |
|
|
215
|
-
| Mean distance | Weighted physical separation: `0` for the same file, `1` for different files in one folder, larger across folders. |
|
|
216
|
-
| Gap / rank | A `0–1` review score combining similarity above the threshold and separation; higher gap ranks first. Same-file pairs have zero gap. |
|
|
217
|
-
| Reciprocal | Both functions selected each other as neighbors. JSON `null` means the other endpoint was not evaluated under source filtering. |
|
|
218
|
-
| `sourceTestPair` | A source/test relationship inferred from paths; separation may be intentional. |
|
|
219
|
-
| `externalAffinityRatio` | In JSON file reports, the share of observed affinity outside that file's folder. |
|
|
220
|
-
|
|
221
|
-
The example suggests reviewing separated authentication responsibilities. It does not establish that they belong in one module. There is no universal cohesion pass threshold. Filtered reports describe selected sources, not the entire repository. Cohesion metrics cover all qualifying edges; reported pairs/files are limited, and groups are built from reported pairs.
|
|
220
|
+
Run cross-search with `--cohesion` to put physically distant matches first. Distance is `0` within one file, `1` between files in one folder, and `1` plus directory-tree hops across folders. The option only changes the order of each source's selected semantic matches; it does not change similarity scores or establish that distant code belongs together.
|
|
222
221
|
|
|
223
|
-
For automation, pass `--format json`: query searches and diagnostics return JSON arrays; cross-search returns **JSONL**, one row per matched source
|
|
222
|
+
For automation, pass `--format json`: query searches and diagnostics return JSON arrays; cross-search returns **JSONL**, one row per matched source. With `--cohesion`, each match includes `physicalDistance`. Results go to stdout; notices and warnings go to stderr.
|
|
224
223
|
|
|
225
224
|
## System behavior
|
|
226
225
|
|
|
@@ -234,9 +233,9 @@ Ordinary updates preserve an existing file description even when its source chan
|
|
|
234
233
|
|
|
235
234
|
Tree-sitter extraction, generated descriptions, and document/query vectors are content-addressed in the same SQLite database. Each validated result is committed immediately, independently of the final logical index update. If indexing is interrupted or a later provider call fails, rerunning reuses every completed result whose profile, operation, input, and source context hash still match.
|
|
236
235
|
|
|
237
|
-
With complete descriptions, `search
|
|
236
|
+
With complete descriptions, `search` and cross-search average **one-third code similarity + one-third callable-description similarity + one-third file-description similarity**. `search-description` averages callable and file descriptions without code. Cross-repository analysis needs complete descriptions on both sides; otherwise the entire analysis uses code-only scores. Stale file descriptions remain searchable until explicitly reindexed. Thresholds and limits apply to the selected score.
|
|
238
237
|
|
|
239
|
-
Text output labels combined scores. JSON exposes `codeSimilarity`, `descriptionSimilarity`, `fileDescriptionSimilarity`, and scoring mode/weights
|
|
238
|
+
Text output labels combined scores. JSON exposes `codeSimilarity`, `descriptionSimilarity`, `fileDescriptionSimilarity`, and cross-search scoring mode/weights. Compare runs only with matching scoring mode, weights, embedding and description-generator profiles, threshold, and source/candidate filters.
|
|
240
239
|
|
|
241
240
|
### Exclusions
|
|
242
241
|
|
|
@@ -250,17 +249,21 @@ Parse, extraction, read, and file-size failures are saved while healthy callable
|
|
|
250
249
|
|
|
251
250
|
Saved failures trigger stderr warnings, including on cached runs, help, and cross-search targets. `--ignore-errors` silences warnings without clearing records. Updates retry failed files; successful indexing, deletion, or exclusion clears their diagnostics. Version output bypasses diagnostics.
|
|
252
251
|
|
|
253
|
-
Use the recovery flag named in the error: `--rebuild-on-divergence` for Git history changes, `--force-reindex` for incompatible indexes. Schema versions
|
|
252
|
+
Use the recovery flag named in the error: `--rebuild-on-divergence` for Git history changes, `--force-reindex` for incompatible indexes. Schema versions 6 and 7 migrate in place; earlier schemas require `--force-reindex`, with compatible rebuilds preserving reusable artifact caches. For provider/authentication failures, fix the reported configuration and rerun. For source-change-during-indexing errors, rerun after edits settle. Exit status is `0` on success, `2` for argument/domain errors, and `1` for other failures (or invocation without a command).
|
|
254
253
|
|
|
255
254
|
## Configuration
|
|
256
255
|
|
|
257
|
-
Optional file: `<root>/.slopdex/config.json`. Example using Jina embeddings and OpenCode Go descriptions (requires `JINA_API_KEY`, plus `OPENCODE_API_KEY` when descriptions are enabled):
|
|
256
|
+
Optional file: `<root>/.slopdex/config.json`. Example using Jina embeddings, OpenAI LLM reranking, and OpenCode Go descriptions (requires `JINA_API_KEY`, `OPENAI_API_KEY` for query searches, plus `OPENCODE_API_KEY` when descriptions are enabled):
|
|
258
257
|
|
|
259
258
|
```json
|
|
260
259
|
{
|
|
261
260
|
"provider": "jina",
|
|
262
261
|
"model": "jina-embeddings-v4",
|
|
263
262
|
"dimensions": 1024,
|
|
263
|
+
"rerankingEnabled": true,
|
|
264
|
+
"rerankerProvider": "openai",
|
|
265
|
+
"rerankerModel": "gpt-5.6-luna",
|
|
266
|
+
"rerankerCandidates": 10,
|
|
264
267
|
"descriptionProvider": "opencode-go",
|
|
265
268
|
"exclude": ["**/fixtures/**"]
|
|
266
269
|
}
|
|
@@ -272,13 +275,17 @@ Optional file: `<root>/.slopdex/config.json`. Example using Jina embeddings and
|
|
|
272
275
|
| `descriptionProvider` | Description provider: `openai`, `opencode` (Zen), or `opencode-go`; defaults to `openai`. |
|
|
273
276
|
| `descriptionModel` | Description model; provider default unless explicitly set. |
|
|
274
277
|
| `descriptionsEnabled` | When true or false, the next index-using command applies that enabled state during its normal refresh. Unset leaves persisted index state unchanged. |
|
|
278
|
+
| `rerankingEnabled` | Enables second-stage ranking for `search` and `search-description`; disabled/unset by default. Prefer changing it through `config reranker`. |
|
|
279
|
+
| `rerankerProvider` | Reranker: `cohere`, `jina`, or `openai`. OpenAI uses an LLM rather than a dedicated reranking endpoint. |
|
|
280
|
+
| `rerankerModel` | Provider model; defaults to Cohere `rerank-v4.0-pro`, Jina `jina-reranker-v3.5`, or OpenAI `gpt-5.6-luna`. |
|
|
281
|
+
| `rerankerCandidates` | Embedding-ranked candidates sent to the OpenAI LLM; integer from `1` to `100`, default `10`. The requested result limit takes precedence when larger, up to 100. |
|
|
275
282
|
| `indexPath` | Index location; `<root>/.slopdex/index.sqlite`. |
|
|
276
283
|
| `include` | Repository-relative glob array; empty/unset includes all supported eligible files. |
|
|
277
284
|
| `exclude` | Additional repository-relative exclusion globs. |
|
|
278
285
|
| `maxFileSize` | Maximum source-file size in bytes; positive integer, default `1048576`. |
|
|
279
286
|
| `embeddingBatchSize` | Embedding inputs per batch; positive integer, default `32`. |
|
|
280
287
|
|
|
281
|
-
Keep keys in the environment (`OPENAI_API_KEY`, `JINA_API_KEY`, `OPENCODE_API_KEY`). Changing the embedding profile requires rebuilding with `--force-reindex`.
|
|
288
|
+
Keep keys in the environment (`OPENAI_API_KEY`, `JINA_API_KEY`, `COHERE_API_KEY`, `OPENCODE_API_KEY`). Reranker settings do not change the stored index and do not require a rebuild. Changing the embedding profile requires rebuilding with `--force-reindex`.
|
|
282
289
|
|
|
283
290
|
## Development
|
|
284
291
|
|