@ninjaxtools/slopdex 0.14.0 → 0.15.0

package/.agents/skills/slopdex/SKILL.md

@@ -154,6 +154,7 @@ Usage: `slopdex <command> [arguments] [options]`. Quote queries and regexes. Boo
 | `--description-model <name>` | Description model; `gpt-5.6-sol` for OpenAI/Zen and `gpt-5.6-luna` for Go. |
 | `--reranker-candidates <number>` | With `config reranker openai`, embedding-ranked functions sent to the LLM; range `1`-`100`, default `10`. |
 | `--ignore-errors` | Silence saved-diagnostic warnings without deleting records. |
+| `--verbose` | Report every external model request on stderr instead of once per call kind/provider/model. Config `"verbose": true` has the same effect. |
 | `-h`, `--help` | Usage; no refresh. |
 | `--version` | Package version; exits without refresh or saved-diagnostic warnings. |
 
@@ -201,7 +202,7 @@ Use `--no-reindex` when the task calls for committed-only results or reuse of an
 | `index-errors` | `summary` | JSON array |
 | `status`, update commands, `descriptions` | JSON object | — |
 
-Prefer summary output for compact source review, clusters for duplicate families, and JSON/JSONL for structured processing. Stdout carries results; stderr carries notices and warnings. Cross-search omits sources without emitted matches. Empty output means no findings under the chosen coverage/filters, not proof that no similar code exists.
+Prefer summary output for compact source review, clusters for duplicate families, and JSON/JSONL for structured processing. Stdout carries results; stderr carries notices and warnings. External vector, description, and reranking requests identify their provider and model once per combination, or for every request with `--verbose`. Cross-search omits sources without emitted matches. Empty output means no findings under the chosen coverage/filters, not proof that no similar code exists.
 
 With reranking enabled, query summaries display both reranker relevance and embedding similarity. Similarity thresholds filter candidates before reranking; limits apply to the reranked output. LLM candidate documents include descriptions when available and function metadata/source code.
 

package/README.md

@@ -150,6 +150,7 @@ slopdex update-git --force-reindex
 | `--description-provider <openai\|opencode\|opencode-go>` | Description provider; OpenAI by default. OpenCode values use Zen or Go with `OPENCODE_API_KEY`. |
 | `--description-model <name>` | Description model; `gpt-5.6-sol` for OpenAI/Zen and `gpt-5.6-luna` for Go, then the persisted model unless overridden. Published OpenCode models use their documented protocol. |
 | `--ignore-errors` | Silence warnings about saved indexing errors; records remain available. |
+| `--verbose` | Write one stderr notice for every external model call instead of one per call kind/provider/model. |
 | `-h`, `--help` | Show CLI usage without refreshing. |
 | `--version` | Print the package version and exit. |
 
@@ -233,6 +234,8 @@ Ordinary updates preserve an existing file description even when its source chan
 
 Tree-sitter extraction, generated descriptions, and document/query vectors are content-addressed in the same SQLite database. Each validated result is committed immediately, independently of the final logical index update. If indexing is interrupted or a later provider call fails, rerunning reuses every completed result whose profile, operation, input, and source context hash still match.
 
+When Slopdex makes external vector, description, or reranking model calls, stderr identifies the call kind, provider, and model. By default each combination is reported once per process regardless of request count. Pass `--verbose`, or set `"verbose": true` in config, to report every request. Cache hits do not produce notices because they do not call a model.
+
 With complete descriptions, `search` and cross-search average **one-third code similarity + one-third callable-description similarity + one-third file-description similarity**. `search-description` averages callable and file descriptions without code. Cross-repository analysis needs complete descriptions on both sides; otherwise the entire analysis uses code-only scores. Stale file descriptions remain searchable until explicitly reindexed. Thresholds and limits apply to the selected score.
 
 Text output labels combined scores. JSON exposes `codeSimilarity`, `descriptionSimilarity`, `fileDescriptionSimilarity`, and cross-search scoring mode/weights. Compare runs only with matching scoring mode, weights, embedding and description-generator profiles, threshold, and source/candidate filters.
@@ -265,6 +268,7 @@ Optional file: `<root>/.slopdex/config.json`. Example using Jina embeddings, Ope
   "rerankerModel": "gpt-5.6-luna",
   "rerankerCandidates": 10,
   "descriptionProvider": "opencode-go",
+  "verbose": true,
   "exclude": ["**/fixtures/**"]
 }
 ```
@@ -284,6 +288,7 @@ Optional file: `<root>/.slopdex/config.json`. Example using Jina embeddings, Ope
 | `exclude` | Additional repository-relative exclusion globs. |
 | `maxFileSize` | Maximum source-file size in bytes; positive integer, default `1048576`. |
 | `embeddingBatchSize` | Embedding inputs per batch; positive integer, default `32`. |
+| `verbose` | When true, report every external model request on stderr; false/unset reports each call kind/provider/model once per process. |
 
 Keep keys in the environment (`OPENAI_API_KEY`, `JINA_API_KEY`, `COHERE_API_KEY`, `OPENCODE_API_KEY`). Reranker settings do not change the stored index and do not require a rebuild. Changing the embedding profile requires rebuilding with `--force-reindex`.
 

package/dist/cli.js

@@ -1745,6 +1745,20 @@ function reconcileFunctions(current, previous) {
 import { createOpenAI } from "@ai-sdk/openai";
 import { APICallError, generateText } from "ai";
 import { randomUUID } from "node:crypto";
+
+// src/model-call-notice.ts
+var reportedCalls = /* @__PURE__ */ new Set();
+function reportModelCall(kind, profile, verbose = false) {
+  const key = JSON.stringify([kind, profile.provider, profile.model]);
+  if (!verbose && reportedCalls.has(key)) return;
+  reportedCalls.add(key);
+  process.stderr.write(
+    `slopdex: notice: external model call: kind=${kind} provider=${JSON.stringify(profile.provider)} model=${JSON.stringify(profile.model)}
+`
+  );
+}
+
+// src/descriptions/openai.ts
 var DESCRIPTION_PROVIDER_NAMES = ["openai", "opencode", "opencode-go"];
 function isDescriptionProviderName(value) {
   return DESCRIPTION_PROVIDER_NAMES.includes(value);
@@ -1769,6 +1783,7 @@ var OpenAIDescriptionProvider = class {
   #apiKeyName;
   #baseUrl;
   #openCode;
+  #verbose;
   constructor(options = {}) {
     const provider = options.provider ?? "openai";
     const defaults = PROVIDERS[provider];
@@ -1776,6 +1791,7 @@ var OpenAIDescriptionProvider = class {
     this.#apiKey = options.apiKey ?? process.env[this.#apiKeyName] ?? "";
     this.#baseUrl = (options.baseUrl ?? defaults.baseUrl).replace(/\/$/, "");
     this.#openCode = provider !== "openai";
+    this.#verbose = options.verbose ?? false;
     this.profile = {
       provider,
       model: options.model ?? defaults.model,
@@ -1825,6 +1841,7 @@ var OpenAIDescriptionProvider = class {
     throwIfAborted(options?.signal);
     if (!this.#apiKey) throw new CodeIndexError(`${this.#apiKeyName} is required to generate descriptions.`);
     const { model, responses } = await this.#languageModel(headers);
+    reportModelCall("descriptions", this.profile, this.#verbose);
     let text;
     try {
       ({ text } = await generateText({
@@ -1951,7 +1968,8 @@ var CodeIndex = class {
     this.#database = new IndexDatabase(this.indexPath, this.rootDir, profile, options.readOnly ?? false);
     const storedDescriptionProfile = this.#database.descriptionProfile();
     this.descriptionProvider = options.descriptionProvider ?? new OpenAIDescriptionProvider({
-      ...storedDescriptionProfile && isDescriptionProviderName(storedDescriptionProfile.provider) ? { provider: storedDescriptionProfile.provider, model: storedDescriptionProfile.model } : {}
+      ...storedDescriptionProfile && isDescriptionProviderName(storedDescriptionProfile.provider) ? { provider: storedDescriptionProfile.provider, model: storedDescriptionProfile.model } : {},
+      ...options.verbose ? { verbose: true } : {}
     });
     this.#policy = new SourcePolicy(options.include, options.exclude);
     this.#maxFileSize = options.maxFileSize ?? DEFAULT_MAX_FILE_SIZE;
@@ -2946,6 +2964,7 @@ var JinaEmbeddingProvider = class {
   profile;
   #apiKey;
   #baseUrl;
+  #verbose;
   constructor(options = {}) {
     this.#apiKey = options.apiKey ?? process.env.JINA_API_KEY ?? "";
     if (!this.#apiKey) throw new Error("JINA_API_KEY is required.");
@@ -2958,11 +2977,13 @@ var JinaEmbeddingProvider = class {
       dimensions,
       strategyVersion: "callable-v2:code-query-passage"
     };
+    this.#verbose = options.verbose ?? false;
     const url = (options.baseUrl ?? "https://api.jina.ai/v1/embeddings").replace(/\/$/, "");
     this.#baseUrl = url.endsWith("/embeddings") ? url.slice(0, -"/embeddings".length) : url;
   }
   async embedDocuments(inputs, options) {
     if (inputs.length === 0) return [];
+    reportModelCall("vectors", this.profile, this.#verbose);
     return requestEmbeddings(
       embedMany({
         model: this.#model("code.passage"),
@@ -2975,6 +2996,7 @@ var JinaEmbeddingProvider = class {
     );
   }
   async embedQuery(input, options) {
+    reportModelCall("vectors", this.profile, this.#verbose);
     return (await requestEmbeddings(
       embed({
         model: this.#model("code.query"),
@@ -3019,6 +3041,7 @@ function truncateInput(input) {
 var OpenAIEmbeddingProvider = class {
   profile;
   #model;
+  #verbose;
   constructor(options = {}) {
     const apiKey = options.apiKey ?? process.env.OPENAI_API_KEY ?? "";
     if (!apiKey) throw new Error("OPENAI_API_KEY is required.");
@@ -3026,6 +3049,7 @@ var OpenAIEmbeddingProvider = class {
     const dimensions = options.dimensions ?? 3072;
     if (!Number.isInteger(dimensions) || dimensions < 1) throw new Error("dimensions must be a positive integer.");
     this.profile = { provider: "openai", model, dimensions, strategyVersion: "callable-v2" };
+    this.#verbose = options.verbose ?? false;
     this.#model = createOpenAI3({
       apiKey,
       baseURL: (options.baseUrl ?? "https://api.openai.com/v1").replace(/\/$/, "")
@@ -3033,6 +3057,7 @@ var OpenAIEmbeddingProvider = class {
   }
   async embedDocuments(inputs, options) {
     if (inputs.length === 0) return [];
+    reportModelCall("vectors", this.profile, this.#verbose);
     return requestEmbeddings(
       embedMany2({
         model: this.#model,
@@ -3045,6 +3070,7 @@ var OpenAIEmbeddingProvider = class {
     );
   }
   async embedQuery(input, options) {
+    reportModelCall("vectors", this.profile, this.#verbose);
     return (await requestEmbeddings(
       embed2({
         model: this.#model,
@@ -3144,12 +3170,14 @@ var HostedReranker = class {
   #apiKey;
   #url;
   #returnDocuments;
+  #verbose;
   constructor(options, settings) {
     this.#apiKey = options.apiKey ?? process.env[settings.apiKeyName] ?? "";
     if (!this.#apiKey) throw new Error(`${settings.apiKeyName} is required.`);
     const model = options.model ?? settings.defaultModel;
     if (!model.trim()) throw new Error("reranker model must not be empty.");
     this.profile = { provider: settings.provider, model };
+    this.#verbose = options.verbose ?? false;
     const url = (options.baseUrl ?? settings.defaultUrl).replace(/\/$/, "");
     this.#url = url.endsWith("/rerank") ? url : `${url}/rerank`;
     this.#returnDocuments = settings.returnDocuments;
@@ -3159,6 +3187,7 @@ var HostedReranker = class {
     const limit = options.limit ?? documents.length;
     assertPositiveInteger(limit, "rerank limit");
     throwIfAborted(options.signal);
+    reportModelCall("reranking", this.profile, this.#verbose);
     let response;
     try {
       response = await fetch(this.#url, {
@@ -3248,9 +3277,11 @@ var OpenAILLMReranker = class {
   maximumCandidateCount = MAX_CANDIDATES;
   #apiKey;
   #baseUrl;
+  #verbose;
   constructor(options = {}) {
     this.#apiKey = options.apiKey ?? process.env.OPENAI_API_KEY ?? "";
     this.#baseUrl = (options.baseUrl ?? "https://api.openai.com/v1").replace(/\/$/, "");
+    this.#verbose = options.verbose ?? false;
     this.candidateCount = options.candidateCount ?? 10;
     assertPositiveInteger(this.candidateCount, "reranker candidate count");
     if (this.candidateCount > MAX_CANDIDATES) {
@@ -3302,6 +3333,7 @@ var OpenAILLMReranker = class {
           document: tokens.length <= candidateTokenLimit ? document : tokenizer2.decode(tokens.slice(0, candidateTokenLimit))
         };
       });
+      reportModelCall("reranking", this.profile, this.#verbose);
       ({ output } = await generateText2({
         model: createOpenAI4({ apiKey: this.#apiKey, baseURL: this.#baseUrl }).responses(this.profile.model),
         prompt: JSON.stringify({
@@ -3532,6 +3564,7 @@ var parsed = (() => {
         "no-reindex": { type: "boolean", default: false },
         callables: { type: "boolean", default: false },
         "ignore-errors": { type: "boolean", default: false },
+        verbose: { type: "boolean", default: false },
         version: { type: "boolean", default: false },
         help: { type: "boolean", short: "h", default: false }
       }
@@ -3545,7 +3578,7 @@ var parsed = (() => {
 var [command, ...positionals] = parsed.positionals;
 var descriptionsAction = command === "descriptions" ? positionals[0] : void 0;
 if (parsed.values.version) {
-  const version = true ? "0.14.0" : JSON.parse(readFileSync2(new URL("../package.json", import.meta.url), "utf8")).version;
+  const version = true ? "0.15.0" : JSON.parse(readFileSync2(new URL("../package.json", import.meta.url), "utf8")).version;
   process.stdout.write(`${version}
 `);
   process.exit(0);
@@ -3620,7 +3653,8 @@ async function main() {
     ...config.include ? { include: config.include } : {},
     ...config.exclude ? { exclude: config.exclude } : {},
     ...config.maxFileSize ? { maxFileSize: config.maxFileSize } : {},
-    ...config.embeddingBatchSize ? { embeddingBatchSize: config.embeddingBatchSize } : {}
+    ...config.embeddingBatchSize ? { embeddingBatchSize: config.embeddingBatchSize } : {},
+    ...config.verbose ? { verbose: true } : {}
   };
   const updateTarget = command === "update-git" ? parsed.values.target ?? "HEAD" : "HEAD";
   const updateStats = await ensureIndexUpdated(
@@ -3708,6 +3742,7 @@ async function runCrossSearch(source, sourceOptions, provider) {
     ...targetConfig?.exclude ? { exclude: targetConfig.exclude } : {},
     ...targetConfig?.maxFileSize ? { maxFileSize: targetConfig.maxFileSize } : {},
     ...targetConfig?.embeddingBatchSize ? { embeddingBatchSize: targetConfig.embeddingBatchSize } : {},
+    ...targetConfig?.verbose ? { verbose: true } : {},
     ...targetDescriptionProvider ? { descriptionProvider: targetDescriptionProvider } : {}
   } : void 0;
   if (targetOptions) {
@@ -3807,7 +3842,8 @@ async function initializeIndex(options, indexPath, label, target, noReindex, des
     indexPath,
     ...descriptionProfile && !options.descriptionProvider ? { descriptionProvider: new OpenAIDescriptionProvider({
       provider: descriptionProfile.provider,
-      model: descriptionProfile.model
+      model: descriptionProfile.model,
+      ...options.verbose ? { verbose: true } : {}
     }) } : {}
   });
   try {
@@ -3888,6 +3924,9 @@ function commandLineConfig(config) {
   if (config.rerankingEnabled !== void 0 && typeof config.rerankingEnabled !== "boolean") {
     throw new CodeIndexError("rerankingEnabled must be a boolean.");
   }
+  if (config.verbose !== void 0 && typeof config.verbose !== "boolean") {
+    throw new CodeIndexError("verbose must be a boolean.");
+  }
   if (config.rerankingEnabled === true) {
     if (config.rerankerProvider !== "cohere" && config.rerankerProvider !== "jina" && config.rerankerProvider !== "openai") {
       throw new CodeIndexError(`Unsupported reranker provider: ${String(config.rerankerProvider)}`);
@@ -3908,14 +3947,16 @@ function commandLineConfig(config) {
     ...parsed.values.model ? { model: parsed.values.model } : {},
     ...descriptionProviderValue ? { descriptionProvider: descriptionProviderValue } : {},
     ...parsed.values["description-model"] ? { descriptionModel: parsed.values["description-model"] } : {},
-    ...dimensions ? { dimensions } : {}
+    ...dimensions ? { dimensions } : {},
+    ...parsed.values.verbose ? { verbose: true } : {}
   };
 }
 function createDescriptionProvider(config) {
   if (!config.descriptionProvider && !config.descriptionModel) return void 0;
   return new OpenAIDescriptionProvider({
     ...config.descriptionProvider ? { provider: config.descriptionProvider } : {},
-    ...config.descriptionModel ? { model: config.descriptionModel } : {}
+    ...config.descriptionModel ? { model: config.descriptionModel } : {},
+    ...config.verbose ? { verbose: true } : {}
   });
 }
 function descriptionRefresh(config) {
@@ -4096,26 +4137,35 @@ function createProvider(config) {
   if ((config.provider ?? "openai") === "jina") {
     return new JinaEmbeddingProvider({
       ...config.model ? { model: config.model } : {},
-      ...config.dimensions ? { dimensions: config.dimensions } : {}
+      ...config.dimensions ? { dimensions: config.dimensions } : {},
+      ...config.verbose ? { verbose: true } : {}
     });
   }
   return new OpenAIEmbeddingProvider({
     ...config.model ? { model: config.model } : {},
-    ...config.dimensions ? { dimensions: config.dimensions } : {}
+    ...config.dimensions ? { dimensions: config.dimensions } : {},
+    ...config.verbose ? { verbose: true } : {}
   });
 }
 function createReranker(config) {
   if (config.rerankingEnabled !== true) return void 0;
   if (config.rerankerProvider === "cohere") {
-    return new CohereReranker(config.rerankerModel ? { model: config.rerankerModel } : {});
+    return new CohereReranker({
+      ...config.rerankerModel ? { model: config.rerankerModel } : {},
+      ...config.verbose ? { verbose: true } : {}
+    });
   }
   if (config.rerankerProvider === "jina") {
-    return new JinaReranker(config.rerankerModel ? { model: config.rerankerModel } : {});
+    return new JinaReranker({
+      ...config.rerankerModel ? { model: config.rerankerModel } : {},
+      ...config.verbose ? { verbose: true } : {}
+    });
   }
   if (config.rerankerProvider === "openai") {
     return new OpenAILLMReranker({
       ...config.rerankerModel ? { model: config.rerankerModel } : {},
-      ...config.rerankerCandidates ? { candidateCount: config.rerankerCandidates } : {}
+      ...config.rerankerCandidates ? { candidateCount: config.rerankerCandidates } : {},
+      ...config.verbose ? { verbose: true } : {}
     });
   }
   throw new CodeIndexError("rerankerProvider is required when rerankingEnabled is true.");
@@ -4376,6 +4426,7 @@ Options:
   --no-reindex                        Skip worktree overlays or reuse a non-Git index
   --callables                         With reindex-files, also regenerate callable descriptions
   --ignore-errors                     Silence warnings about persisted indexing errors
+  --verbose                           Log every external model call instead of one per kind/model
   --limit <number>                    Search result limit
   --threshold <number|range>          Show similarities at/above a value or within a range
   --format <json|summary|clusters>    Output format (default: summary; cross-search: clusters)