pi-knowledge 0.5.5 → 0.5.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,13 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.5.6] - 2026-07-23
4
+
5
+ ### Added
6
+ - Added runtime search tuning profiles (`auto`, `low_token`, `precision`, `recall`, `long_context`, `code`, `docs`, `balanced`) with diagnostics that report the applied result limit, snippet length, hybrid threshold, candidate pool, adaptive context, and deep rerank breadth.
7
+
8
+ ### Changed
9
+ - Added bounded environment overrides for search defaults so slow local model users can reduce result count, lengthen snippets, tighten hybrid precision, or tune candidate breadth without editing source.
10
+
3
11
  ## [0.5.5] - 2026-07-23
4
12
 
5
13
  ### Added
package/README.md CHANGED
@@ -125,6 +125,8 @@ omp install ./pi-knowledge
125
125
  - `auto`: selects a primary mode from the query shape and retries alternate modes when results are empty or weak.
126
126
  - `code`, `config`, `errors`, `docs`, `decision`: intent aliases for agents. `code`, `config`, and `errors` bias toward exact lexical/symbol evidence; `docs` and `decision` keep the intended source type explicit.
127
127
 
128
+ Search profiles tune result count, snippet length, hybrid strictness, candidate breadth, adaptive context, and deep rerank breadth. `profile: "auto"` selects a runtime profile from query shape, mode, and KB source type; explicit tool parameters still win over profile defaults. Use `profile: "low_token"` for slow local models that need fewer, stricter, longer results in one call; use `precision` for identifiers/errors, `recall` for broad discovery, `long_context` for prose/specs, and `balanced` for compatibility.
129
+
128
130
  Mode selection contract:
129
131
 
130
132
  - Start with `hybrid` for most project questions that contain useful lexical anchors.
@@ -134,7 +136,7 @@ Mode selection contract:
134
136
  - Use `deep` for high-stakes answers, ambiguous top results, or final verification when slower reranking is acceptable.
135
137
  - If results are empty or weak but the KB should contain the answer, retry once with a different mode before concluding no answer exists.
136
138
 
137
- Search results use balanced diversity reranking by default so near-duplicate chunks from the same file do not dominate the top results. Diversity scoring considers lexical overlap, same-file line proximity, overlapping adaptive windows, available embedding-vector similarity, file-level interleaving, and a small KB trust multiplier that favors ready file/directory sources over stale, URL, or imported text sources. Use `diversity: "off"` only when raw ranking order is needed for diagnostics. Agents can request search diagnostics to inspect mode fallback, ranking coverage, path/source/test boosts, adjusted scores, and provenance such as chunk id, chunk hash, match reason, stale flag, and source freshness where available.
139
+ Search results use balanced diversity reranking by default so near-duplicate chunks from the same file do not dominate the top results. Diversity scoring considers lexical overlap, same-file line proximity, overlapping adaptive windows, available embedding-vector similarity, file-level interleaving, and a small KB trust multiplier that favors ready file/directory sources over stale, URL, or imported text sources. Use `diversity: "off"` only when raw ranking order is needed for diagnostics. Agents can request search diagnostics to inspect mode fallback, selected profile, applied search tuning, ranking coverage, path/source/test boosts, adjusted scores, and provenance such as chunk id, chunk hash, match reason, stale flag, and source freshness where available.
138
140
 
139
141
  For best search quality, rebuild or update existing knowledge bases after upgrading. New indexes use contextual retrieval units: embeddings and FTS include file path, file type, Markdown heading breadcrumbs, and code symbol names while returned results keep the original chunk text readable. This improves queries that mention project structure, filenames, sections, or functions, and reduces duplicate-looking chunk hits.
140
142
  Symbol/config/heading metadata is also rebuilt during `knowledge_update`; older KBs created before this feature may show a doctor action recommending update.
@@ -176,7 +178,7 @@ Full configuration details are in [docs/configuration.md](docs/configuration.md)
176
178
  | Model worker and cache | `PI_KNOWLEDGE_MODEL_CACHE_DIR`, `PI_KNOWLEDGE_NODE_PATH` |
177
179
  | Embedding provider | `PI_KNOWLEDGE_EMBEDDING`, `OPENAI_API_KEY`, `PI_KNOWLEDGE_EMBEDDING_BASE_URL`, `OPENAI_BASE_URL`, `PI_KNOWLEDGE_EMBEDDING_MAX_CHARS`, `PI_KNOWLEDGE_EMBEDDING_API_FALLBACK` |
178
180
  | Native lifecycle | `PI_KNOWLEDGE_ENABLE_NATIVE_IDLE_DISPOSE`, `PI_KNOWLEDGE_EMBEDDING_IDLE_MS` |
179
- | Runtime features | `PI_KNOWLEDGE_WATCH`, `PI_KNOWLEDGE_AUTO_INJECT`, `PI_KNOWLEDGE_STALE_INDEXING_MS`, `PI_KNOWLEDGE_OFFLINE` |
181
+ | Runtime features | `PI_KNOWLEDGE_WATCH`, `PI_KNOWLEDGE_AUTO_INJECT`, `PI_KNOWLEDGE_STALE_INDEXING_MS`, `PI_KNOWLEDGE_OFFLINE`, `PI_KNOWLEDGE_SEARCH_PROFILE`, `PI_KNOWLEDGE_SEARCH_DEFAULT_LIMIT`, `PI_KNOWLEDGE_SNIPPET_MAX_LENGTH`, `PI_KNOWLEDGE_MIN_HYBRID_SCORE`, `PI_KNOWLEDGE_SEARCH_CANDIDATE_MIN`, `PI_KNOWLEDGE_SEARCH_CANDIDATE_MULTIPLIER`, `PI_KNOWLEDGE_ADAPTIVE_CONTEXT_LINES`, `PI_KNOWLEDGE_ADAPTIVE_MAX_CHARS`, `PI_KNOWLEDGE_ADAPTIVE_NEIGHBOR_TARGET`, `PI_KNOWLEDGE_DEEP_RERANK_CANDIDATES`, `PI_KNOWLEDGE_DEEP_RERANK_TOPK_MULTIPLIER` |
180
182
  | Release fixtures | `PI_KNOWLEDGE_E2E_PDF`, `PI_KNOWLEDGE_E2E_DOCX` |
181
183
 
182
184
  ## Pi and OMP Support
package/dist/index.js CHANGED
@@ -363,6 +363,7 @@ export default function (pi) {
363
363
  "Use mode 'semantic' for broad conceptual questions when exact terms may differ from the indexed wording or hybrid returns no lexical matches",
364
364
  "Use mode 'adaptive' when the user needs surrounding implementation context, related nearby sections, or enough context to make a code change",
365
365
  "Use mode 'deep' for high-stakes answers, ambiguous top results, or final verification where slower cross-encoder reranking is acceptable",
366
+ "Use profile 'low_token' for slow local models that need fewer, stricter, longer snippets; use profile 'precision' for identifiers/errors and 'recall' or 'long_context' for broad prose/document research",
366
367
  "If a search returns no results or obviously weak results, retry once with a different mode before concluding the KB lacks the answer",
367
368
  "If top results are repetitive, retry with diversity 'strong' or mode 'adaptive' instead of increasing the limit first",
368
369
  ],
@@ -381,7 +382,17 @@ export default function (pi) {
381
382
  Type.Literal("errors"),
382
383
  Type.Literal("decision"),
383
384
  ])),
384
- limit: Type.Optional(Type.Number({ description: "Max results (default 10)" })),
385
+ profile: Type.Optional(Type.Union([
386
+ Type.Literal("auto"),
387
+ Type.Literal("balanced"),
388
+ Type.Literal("low_token"),
389
+ Type.Literal("precision"),
390
+ Type.Literal("recall"),
391
+ Type.Literal("long_context"),
392
+ Type.Literal("code"),
393
+ Type.Literal("docs"),
394
+ ])),
395
+ limit: Type.Optional(Type.Number({ description: "Max results (default comes from search profile/env)" })),
385
396
  kb_id: Type.Optional(Type.String({ description: "Limit search to a specific KB by ID or exact name" })),
386
397
  offset: Type.Optional(Type.Number({ description: "Pagination offset" })),
387
398
  file_type: Type.Optional(Type.String({ description: "Filter by file type (e.g. typescript, markdown, python)" })),
@@ -391,15 +402,18 @@ export default function (pi) {
391
402
  }),
392
403
  async execute(_id, params, _signal) {
393
404
  const { engine } = await ensureInitialized();
394
- const { query, mode, limit, kb_id, offset, file_type, path_pattern, diversity, diagnostics } = params;
405
+ const { query, mode, profile, limit, kb_id, offset, file_type, path_pattern, diversity, diagnostics } = params;
395
406
  const filters = file_type || path_pattern ? { file_type, path_pattern } : undefined;
396
- const response = await engine.search(query, { mode, limit, kb_id, offset, filters, diversity }, _signal);
407
+ const response = await engine.search(query, { mode, profile, limit, kb_id, offset, filters, diversity }, _signal);
397
408
  if (response.results.length === 0) {
398
409
  const details = [
399
410
  "No results found.",
400
411
  response.mode_used ? `Mode used: ${response.mode_used}` : "",
401
412
  response.retry_modes?.length ? `Retried: ${response.retry_modes.join(", ")}` : "",
402
413
  response.warnings?.length ? `Warnings: ${response.warnings.join(" | ")}` : "",
414
+ diagnostics && response.tuning
415
+ ? `Tuning: profile=${response.tuning.requested_profile}->${response.tuning.selected_profile}, limit=${response.tuning.limit}, snippet=${response.tuning.snippet_max_length}, min_hybrid=${response.tuning.min_hybrid_score}, candidates=${response.tuning.candidate_limit}`
416
+ : "",
403
417
  response.suggestions?.length ? `Suggestions: ${response.suggestions.join(" | ")}` : "",
404
418
  ]
405
419
  .filter(Boolean)
@@ -411,6 +425,9 @@ export default function (pi) {
411
425
  output += ` — mode: ${response.mode_used}`;
412
426
  if (response.retry_modes?.length)
413
427
  output += ` — retried: ${response.retry_modes.join(", ")}`;
428
+ if (diagnostics && response.tuning) {
429
+ output += ` — profile: ${response.tuning.requested_profile}->${response.tuning.selected_profile}`;
430
+ }
414
431
  output += ":\n\n";
415
432
  if (response.warnings?.length) {
416
433
  output = `Warnings:\n- ${response.warnings.join("\n- ")}\n\n${output}`;
@@ -423,7 +440,10 @@ export default function (pi) {
423
440
  const provenance = diagnostics && r.provenance
424
441
  ? `\nProvenance: chunk=${r.provenance.chunk_id}, hash=${r.provenance.chunk_hash.slice(0, 12)}, reason=${r.provenance.match_reason}, indexed_at=${r.provenance.indexed_at}, source_mtime=${r.provenance.source_mtime ?? "unknown"}, stale=${r.provenance.stale}`
425
442
  : "";
426
- return `[${i + 1}] ${r.file_path} (${r.kb_name}, score: ${r.score.toFixed(3)})\n${r.snippet}${diag}${provenance}`;
443
+ const tuning = diagnostics && response.tuning && i === 0
444
+ ? `\nTuning: limit=${response.tuning.limit}, snippet=${response.tuning.snippet_max_length}, min_hybrid=${response.tuning.min_hybrid_score}, candidates=${response.tuning.candidate_limit}`
445
+ : "";
446
+ return `[${i + 1}] ${r.file_path} (${r.kb_name}, score: ${r.score.toFixed(3)})\n${r.snippet}${diag}${provenance}${tuning}`;
427
447
  })
428
448
  .join("\n\n");
429
449
  return { content: [{ type: "text", text: output }], details: diagnostics ? response : undefined };
@@ -1,10 +1,12 @@
1
1
  import { type DiagnosticResult } from "./diagnostics/health.ts";
2
2
  import { createSkippedScanStats } from "./indexer/chunker.ts";
3
3
  import { type RankingDiagnostics } from "./search/ranking.ts";
4
+ import { type SearchProfile, type SearchTuningSummary } from "./search/tuning.ts";
4
5
  import { type KnowledgeBase, type KnowledgeSymbol } from "./storage/sqlite.ts";
5
6
  export type SearchMode = "auto" | "fast" | "semantic" | "hybrid" | "deep" | "adaptive" | "code" | "config" | "docs" | "errors" | "decision";
6
7
  export interface SearchOptions {
7
8
  mode?: SearchMode;
9
+ profile?: SearchProfile;
8
10
  limit?: number;
9
11
  offset?: number;
10
12
  kb_id?: string;
@@ -43,6 +45,7 @@ export interface SearchResponse {
43
45
  mode_used?: SearchMode;
44
46
  retry_modes?: SearchMode[];
45
47
  suggestions?: string[];
48
+ tuning?: SearchTuningSummary;
46
49
  }
47
50
  export type ProgressCallback = (msg: string) => void;
48
51
  export interface AddOptions {
@@ -9,14 +9,12 @@ import { shutdownModelWorker } from "./model-worker-client.js";
9
9
  import { searchBM25 } from "./search/bm25.js";
10
10
  import { weightedScoreFusion } from "./search/fusion.js";
11
11
  import { normalizedQueryText, tokenizeForSearch } from "./search/query.js";
12
- import { hasAnyLexicalEvidence, hasEnoughLexicalEvidence, MIN_HYBRID_SCORE, normalizeFileTypeFilter, queryCoverage, scoreChunkForQuery, } from "./search/ranking.js";
12
+ import { hasAnyLexicalEvidence, hasEnoughLexicalEvidence, normalizeFileTypeFilter, queryCoverage, scoreChunkForQuery, } from "./search/ranking.js";
13
13
  import { disposeReranker, prepareRerankerForShutdown, rerank } from "./search/reranker.js";
14
+ import { resolveSearchTuning, summarizeSearchTuning, } from "./search/tuning.js";
14
15
  import { searchVectorFile } from "./search/vector.js";
15
16
  import { countSymbols, createKB, deleteChunksByIds, deleteKB, deleteSymbolsByKB, finishIndexingJob, getChunkById, getChunkCount, getChunksByFile, getFileCount, getKB, getKBByName, getSymbolCount, insertChunks, insertSymbols, iterateChunkIdsByKB, iterateChunksByKB, listKBs, openDatabase, searchSymbols, startIndexingJob, updateIndexingJob, updateKBCounts, updateKBStatus, } from "./storage/sqlite.js";
16
17
  export const CURRENT_EMBEDDING_MODEL = "multilingual-e5-small";
17
- const ADAPTIVE_CONTEXT_LINES = 80;
18
- const ADAPTIVE_MAX_CONTEXT_CHARS = 6_000;
19
- const ADAPTIVE_NEIGHBOR_TARGET = 5;
20
18
  const INDEX_EMBED_BATCH_SIZE = 64;
21
19
  const VECTOR_REDUNDANCY_WEIGHT = 0.35;
22
20
  function isCancellationError(error) {
@@ -256,7 +254,7 @@ function buildQuerySnippet(content, query, maxLength = 240) {
256
254
  const suffix = end < content.length ? "..." : "";
257
255
  return `${prefix}${content.slice(start, end)}${suffix}`;
258
256
  }
259
- function buildAdaptiveContext(seed, chunks, queryTokens) {
257
+ function buildAdaptiveContext(seed, chunks, queryTokens, options) {
260
258
  const scored = chunks
261
259
  .map((chunk) => {
262
260
  const distance = chunk.id === seed.id
@@ -268,7 +266,7 @@ function buildAdaptiveContext(seed, chunks, queryTokens) {
268
266
  return { chunk, score: seedBoost + proximity + coverage };
269
267
  })
270
268
  .sort((a, b) => b.score - a.score)
271
- .slice(0, ADAPTIVE_NEIGHBOR_TARGET);
269
+ .slice(0, options.neighborTarget);
272
270
  if (!scored.some((item) => item.chunk.id === seed.id))
273
271
  scored.push({ chunk: seed, score: Number.MAX_SAFE_INTEGER });
274
272
  const selectedIds = new Set(scored.map((item) => item.chunk.id));
@@ -279,9 +277,9 @@ function buildAdaptiveContext(seed, chunks, queryTokens) {
279
277
  const sourceChunkIds = [];
280
278
  let totalLength = 0;
281
279
  for (const chunk of ordered) {
282
- if (totalLength >= ADAPTIVE_MAX_CONTEXT_CHARS)
280
+ if (totalLength >= options.maxContextChars)
283
281
  break;
284
- const remaining = ADAPTIVE_MAX_CONTEXT_CHARS - totalLength;
282
+ const remaining = options.maxContextChars - totalLength;
285
283
  const text = chunk.content.slice(0, remaining);
286
284
  parts.push(text);
287
285
  sourceChunkIds.push(chunk.id);
@@ -1109,12 +1107,14 @@ export class KnowledgeEngine {
1109
1107
  const attempts = [primaryMode, ...fallbackModesFor(primaryMode)];
1110
1108
  const tried = [];
1111
1109
  const warnings = [];
1110
+ let lastTuning;
1112
1111
  for (const mode of attempts) {
1113
1112
  if (tried.includes(mode))
1114
1113
  continue;
1115
1114
  tried.push(mode);
1116
1115
  throwIfAborted(signal);
1117
1116
  const response = await this.search(query, { ...options, mode }, signal);
1117
+ lastTuning = response.tuning;
1118
1118
  if (response.warnings)
1119
1119
  warnings.push(...response.warnings);
1120
1120
  if (!isWeakAutoResponse(query, response, primaryMode, mode)) {
@@ -1136,16 +1136,16 @@ export class KnowledgeEngine {
1136
1136
  warnings: warnings.length > 0 ? [...new Set(warnings)] : undefined,
1137
1137
  mode_used: tried.at(-1),
1138
1138
  retry_modes: tried.slice(0, -1),
1139
+ tuning: lastTuning,
1139
1140
  suggestions: [
1140
1141
  "No results after auto mode fallback. Check knowledge_status, try a more exact term, or rebuild the KB if indexing rules changed.",
1141
1142
  ],
1142
1143
  };
1143
1144
  }
1144
1145
  const db = this.db;
1145
- const { mode = "hybrid", limit = 10, offset = 0, kb_id, filters, diversity = "balanced" } = options;
1146
+ const { mode = "hybrid", offset = 0, kb_id, filters, diversity = "balanced" } = options;
1146
1147
  const resolvedMode = retrievalModeFor(mode);
1147
1148
  const retrievalMode = resolvedMode === "adaptive" ? "hybrid" : resolvedMode;
1148
- const candidateLimit = Math.max(50, offset + limit * 12);
1149
1149
  const normalizedQuery = normalizedQueryText(query);
1150
1150
  const queryTokens = tokenizeForSimilarity(query);
1151
1151
  const normalizedFileType = normalizeFileTypeFilter(filters?.file_type);
@@ -1159,6 +1159,17 @@ export class KnowledgeEngine {
1159
1159
  warnings.push(`"${kb.name}" is ${kb.status}; search skipped until indexing is ready`);
1160
1160
  }
1161
1161
  }
1162
+ const tuning = resolveSearchTuning({
1163
+ query,
1164
+ mode,
1165
+ profile: options.profile,
1166
+ kbSourceTypes: kbs.map((kb) => kb.source_type),
1167
+ });
1168
+ const limit = typeof options.limit === "number" && Number.isFinite(options.limit) && options.limit > 0
1169
+ ? Math.trunc(options.limit)
1170
+ : tuning.defaultLimit;
1171
+ const candidateLimit = Math.max(tuning.candidateMin, offset + limit * tuning.candidateMultiplier);
1172
+ const tuningSummary = summarizeSearchTuning(tuning, limit, candidateLimit);
1162
1173
  if (kbs.length === 0) {
1163
1174
  return {
1164
1175
  results: [],
@@ -1166,6 +1177,7 @@ export class KnowledgeEngine {
1166
1177
  has_more: false,
1167
1178
  warnings: warnings.length > 0 ? warnings : undefined,
1168
1179
  mode_used: mode,
1180
+ tuning: tuningSummary,
1169
1181
  };
1170
1182
  }
1171
1183
  const allResults = [];
@@ -1240,7 +1252,7 @@ export class KnowledgeEngine {
1240
1252
  const chunk = getChunkById(db, result.chunkId);
1241
1253
  if (!chunk)
1242
1254
  return false;
1243
- return result.score >= MIN_HYBRID_SCORE && hasEnoughLexicalEvidence(chunk, queryTokens);
1255
+ return result.score >= tuning.minHybridScore && hasEnoughLexicalEvidence(chunk, queryTokens);
1244
1256
  });
1245
1257
  }
1246
1258
  scored.sort((a, b) => b.score - a.score);
@@ -1260,14 +1272,13 @@ export class KnowledgeEngine {
1260
1272
  }
1261
1273
  if (mode === "deep" && filtered.length > 0) {
1262
1274
  const candidates = filtered
1263
- .slice(0, 30)
1275
+ .slice(0, tuning.deepRerankCandidates)
1264
1276
  .map((r) => {
1265
1277
  const chunk = getChunkById(db, r.chunkId);
1266
1278
  return chunk ? { chunkId: r.chunkId, content: chunk.content } : null;
1267
1279
  })
1268
1280
  .filter(Boolean);
1269
- const reranked = await rerank(query, candidates, Math.max(limit * 3, limit), signal);
1270
- throwIfAborted(signal);
1281
+ const reranked = await rerank(query, candidates, Math.max(limit * tuning.deepRerankTopKMultiplier, limit), signal);
1271
1282
  const ranked = [];
1272
1283
  for (const r of reranked) {
1273
1284
  throwIfAborted(signal);
@@ -1281,7 +1292,7 @@ export class KnowledgeEngine {
1281
1292
  ranking: scoreChunkForQuery(r.score, chunk, queryTokens),
1282
1293
  score: r.score,
1283
1294
  content: chunk.content,
1284
- snippet: buildQuerySnippet(chunk.content, query),
1295
+ snippet: buildQuerySnippet(chunk.content, query, tuning.snippetMaxLength),
1285
1296
  startLine: chunk.start_line,
1286
1297
  endLine: chunk.end_line,
1287
1298
  sourceChunkIds: [chunk.id],
@@ -1319,6 +1330,7 @@ export class KnowledgeEngine {
1319
1330
  has_more: offset + limit < diversified.length,
1320
1331
  warnings: warnings.length > 0 ? warnings : undefined,
1321
1332
  mode_used: mode,
1333
+ tuning: tuningSummary,
1322
1334
  };
1323
1335
  }
1324
1336
  const ranked = [];
@@ -1329,15 +1341,18 @@ export class KnowledgeEngine {
1329
1341
  continue;
1330
1342
  const kb = getKB(db, chunk.kb_id);
1331
1343
  if (mode === "adaptive") {
1332
- const contextChunks = getChunksByFile(db, chunk.kb_id, chunk.file_path, Math.max(1, chunk.start_line - ADAPTIVE_CONTEXT_LINES), chunk.end_line + ADAPTIVE_CONTEXT_LINES);
1333
- const context = buildAdaptiveContext(chunk, contextChunks.length > 0 ? contextChunks : [chunk], queryTokens);
1344
+ const contextChunks = getChunksByFile(db, chunk.kb_id, chunk.file_path, Math.max(1, chunk.start_line - tuning.adaptiveContextLines), chunk.end_line + tuning.adaptiveContextLines);
1345
+ const context = buildAdaptiveContext(chunk, contextChunks.length > 0 ? contextChunks : [chunk], queryTokens, {
1346
+ maxContextChars: tuning.adaptiveMaxContextChars,
1347
+ neighborTarget: tuning.adaptiveNeighborTarget,
1348
+ });
1334
1349
  pushAdaptiveCandidate(ranked, {
1335
1350
  chunk,
1336
1351
  kbName: kb?.name ?? "unknown",
1337
1352
  ranking: rankingByChunkId.get(r.chunkId),
1338
1353
  score: r.score,
1339
1354
  content: context.content,
1340
- snippet: buildQuerySnippet(context.content, query),
1355
+ snippet: buildQuerySnippet(context.content, query, tuning.snippetMaxLength),
1341
1356
  startLine: context.startLine,
1342
1357
  endLine: context.endLine,
1343
1358
  sourceChunkIds: context.sourceChunkIds,
@@ -1350,7 +1365,7 @@ export class KnowledgeEngine {
1350
1365
  ranking: rankingByChunkId.get(r.chunkId),
1351
1366
  score: r.score,
1352
1367
  content: chunk.content,
1353
- snippet: buildQuerySnippet(chunk.content, query),
1368
+ snippet: buildQuerySnippet(chunk.content, query, tuning.snippetMaxLength),
1354
1369
  startLine: chunk.start_line,
1355
1370
  endLine: chunk.end_line,
1356
1371
  sourceChunkIds: [chunk.id],
@@ -1390,6 +1405,7 @@ export class KnowledgeEngine {
1390
1405
  has_more: offset + limit < total,
1391
1406
  warnings: warnings.length > 0 ? warnings : undefined,
1392
1407
  mode_used: mode,
1408
+ tuning: tuningSummary,
1393
1409
  suggestions: results.length === 0
1394
1410
  ? [
1395
1411
  "Try mode 'fast' for exact symbols or mode 'semantic' for conceptual wording.",
@@ -0,0 +1,41 @@
1
+ export type SearchProfile = "auto" | "balanced" | "low_token" | "precision" | "recall" | "long_context" | "code" | "docs";
2
+ export type RuntimeProfile = Exclude<SearchProfile, "auto">;
3
+ type SearchModeForTuning = "auto" | "fast" | "semantic" | "hybrid" | "deep" | "adaptive" | "code" | "config" | "docs" | "errors" | "decision";
4
+ export interface SearchTuning {
5
+ requestedProfile: SearchProfile;
6
+ selectedProfile: RuntimeProfile;
7
+ defaultLimit: number;
8
+ snippetMaxLength: number;
9
+ minHybridScore: number;
10
+ candidateMin: number;
11
+ candidateMultiplier: number;
12
+ adaptiveContextLines: number;
13
+ adaptiveMaxContextChars: number;
14
+ adaptiveNeighborTarget: number;
15
+ deepRerankCandidates: number;
16
+ deepRerankTopKMultiplier: number;
17
+ }
18
+ export interface SearchTuningSummary {
19
+ requested_profile: SearchProfile;
20
+ selected_profile: RuntimeProfile;
21
+ limit: number;
22
+ snippet_max_length: number;
23
+ min_hybrid_score: number;
24
+ candidate_limit: number;
25
+ candidate_min: number;
26
+ candidate_multiplier: number;
27
+ adaptive_context_lines: number;
28
+ adaptive_max_context_chars: number;
29
+ adaptive_neighbor_target: number;
30
+ deep_rerank_candidates: number;
31
+ deep_rerank_top_k_multiplier: number;
32
+ }
33
+ interface SearchTuningInput {
34
+ query: string;
35
+ mode: SearchModeForTuning;
36
+ profile?: SearchProfile;
37
+ kbSourceTypes: string[];
38
+ }
39
+ export declare function resolveSearchTuning(input: SearchTuningInput): SearchTuning;
40
+ export declare function summarizeSearchTuning(tuning: SearchTuning, limit: number, candidateLimit: number): SearchTuningSummary;
41
+ export {};
@@ -0,0 +1,190 @@
1
+ import { MIN_HYBRID_SCORE } from "./ranking.js";
2
+ const PROFILE_DEFAULTS = {
3
+ balanced: {
4
+ defaultLimit: 10,
5
+ snippetMaxLength: 240,
6
+ minHybridScore: MIN_HYBRID_SCORE,
7
+ candidateMin: 50,
8
+ candidateMultiplier: 12,
9
+ adaptiveContextLines: 80,
10
+ adaptiveMaxContextChars: 6_000,
11
+ adaptiveNeighborTarget: 5,
12
+ deepRerankCandidates: 30,
13
+ deepRerankTopKMultiplier: 3,
14
+ },
15
+ low_token: {
16
+ defaultLimit: 5,
17
+ snippetMaxLength: 768,
18
+ minHybridScore: 0.4,
19
+ candidateMin: 35,
20
+ candidateMultiplier: 8,
21
+ adaptiveContextLines: 80,
22
+ adaptiveMaxContextChars: 6_000,
23
+ adaptiveNeighborTarget: 5,
24
+ deepRerankCandidates: 20,
25
+ deepRerankTopKMultiplier: 2,
26
+ },
27
+ precision: {
28
+ defaultLimit: 5,
29
+ snippetMaxLength: 400,
30
+ minHybridScore: 0.4,
31
+ candidateMin: 30,
32
+ candidateMultiplier: 6,
33
+ adaptiveContextLines: 60,
34
+ adaptiveMaxContextChars: 4_000,
35
+ adaptiveNeighborTarget: 4,
36
+ deepRerankCandidates: 20,
37
+ deepRerankTopKMultiplier: 2,
38
+ },
39
+ recall: {
40
+ defaultLimit: 15,
41
+ snippetMaxLength: 320,
42
+ minHybridScore: 0.12,
43
+ candidateMin: 75,
44
+ candidateMultiplier: 16,
45
+ adaptiveContextLines: 100,
46
+ adaptiveMaxContextChars: 8_000,
47
+ adaptiveNeighborTarget: 6,
48
+ deepRerankCandidates: 40,
49
+ deepRerankTopKMultiplier: 3,
50
+ },
51
+ long_context: {
52
+ defaultLimit: 5,
53
+ snippetMaxLength: 1_000,
54
+ minHybridScore: 0.25,
55
+ candidateMin: 50,
56
+ candidateMultiplier: 10,
57
+ adaptiveContextLines: 120,
58
+ adaptiveMaxContextChars: 10_000,
59
+ adaptiveNeighborTarget: 7,
60
+ deepRerankCandidates: 30,
61
+ deepRerankTopKMultiplier: 2,
62
+ },
63
+ code: {
64
+ defaultLimit: 10,
65
+ snippetMaxLength: 320,
66
+ minHybridScore: 0.25,
67
+ candidateMin: 50,
68
+ candidateMultiplier: 12,
69
+ adaptiveContextLines: 80,
70
+ adaptiveMaxContextChars: 6_000,
71
+ adaptiveNeighborTarget: 5,
72
+ deepRerankCandidates: 30,
73
+ deepRerankTopKMultiplier: 3,
74
+ },
75
+ docs: {
76
+ defaultLimit: 8,
77
+ snippetMaxLength: 768,
78
+ minHybridScore: 0.22,
79
+ candidateMin: 60,
80
+ candidateMultiplier: 12,
81
+ adaptiveContextLines: 120,
82
+ adaptiveMaxContextChars: 9_000,
83
+ adaptiveNeighborTarget: 7,
84
+ deepRerankCandidates: 30,
85
+ deepRerankTopKMultiplier: 3,
86
+ },
87
+ };
88
+ const SEARCH_PROFILES = new Set([
89
+ "auto",
90
+ "balanced",
91
+ "low_token",
92
+ "precision",
93
+ "recall",
94
+ "long_context",
95
+ "code",
96
+ "docs",
97
+ ]);
98
+ function parseProfile(value) {
99
+ const normalized = value?.trim().toLowerCase();
100
+ if (!normalized)
101
+ return undefined;
102
+ return SEARCH_PROFILES.has(normalized) ? normalized : undefined;
103
+ }
104
+ function envNumber(name) {
105
+ const raw = process.env[name]?.trim();
106
+ if (!raw)
107
+ return undefined;
108
+ const value = Number(raw);
109
+ return Number.isFinite(value) ? value : undefined;
110
+ }
111
+ function clampInt(value, fallback, min, max) {
112
+ if (value === undefined)
113
+ return fallback;
114
+ return Math.min(max, Math.max(min, Math.trunc(value)));
115
+ }
116
+ function clampNumber(value, fallback, min, max) {
117
+ if (value === undefined)
118
+ return fallback;
119
+ return Math.min(max, Math.max(min, value));
120
+ }
121
+ function looksExactOrCodeQuery(query) {
122
+ const trimmed = query.trim();
123
+ if (trimmed.length === 0)
124
+ return false;
125
+ const lower = trimmed.toLowerCase();
126
+ if (/^[A-Z][A-Z0-9_]{2,}$/.test(trimmed))
127
+ return true;
128
+ if (/^[\w./-]+\.(ts|tsx|js|jsx|json|md|py|go|rs|java|yml|yaml|toml)$/.test(lower))
129
+ return true;
130
+ return /[A-Za-z_$][\w$]*\([^)]*\)|[A-Za-z_$][\w$]*\.[A-Za-z_$][\w$]*|\b[A-Z][A-Za-z0-9_$]*[A-Z][A-Za-z0-9_$]*\b|\b[a-z]+[A-Z][A-Za-z0-9_$]*\b/.test(trimmed);
131
+ }
132
+ function looksErrorQuery(query) {
133
+ return /\b(error|exception|failed|failure|stack|trace|enoent|eacces|timeout|abort|cancelled|錯誤|失敗)\b/i.test(query);
134
+ }
135
+ function looksLongContextQuery(query) {
136
+ const wordCount = query.trim().split(/\s+/).filter(Boolean).length;
137
+ return (wordCount >= 10 ||
138
+ /\b(explain|why|how|architecture|design|overview|docs?|spec|manual|guide)\b|流程|架構|概念|為什麼|如何|文件|規格/i.test(query));
139
+ }
140
+ function chooseRuntimeProfile(input) {
141
+ if (input.mode === "code" || input.mode === "config" || input.mode === "errors" || input.mode === "fast") {
142
+ return "precision";
143
+ }
144
+ if (input.mode === "docs" || input.mode === "decision" || input.mode === "adaptive")
145
+ return "long_context";
146
+ if (input.mode === "semantic")
147
+ return "recall";
148
+ if (looksErrorQuery(input.query) || looksExactOrCodeQuery(input.query))
149
+ return "precision";
150
+ const hasDocumentSource = input.kbSourceTypes.some((sourceType) => sourceType === "url" || sourceType === "text");
151
+ if (looksLongContextQuery(input.query) || hasDocumentSource)
152
+ return "long_context";
153
+ return "balanced";
154
+ }
155
+ export function resolveSearchTuning(input) {
156
+ const requestedProfile = input.profile ?? parseProfile(process.env.PI_KNOWLEDGE_SEARCH_PROFILE) ?? "auto";
157
+ const selectedProfile = requestedProfile === "auto" ? chooseRuntimeProfile(input) : requestedProfile;
158
+ const base = PROFILE_DEFAULTS[selectedProfile];
159
+ return {
160
+ requestedProfile,
161
+ selectedProfile,
162
+ defaultLimit: clampInt(envNumber("PI_KNOWLEDGE_SEARCH_DEFAULT_LIMIT"), base.defaultLimit, 1, 50),
163
+ snippetMaxLength: clampInt(envNumber("PI_KNOWLEDGE_SNIPPET_MAX_LENGTH"), base.snippetMaxLength, 80, 4_000),
164
+ minHybridScore: clampNumber(envNumber("PI_KNOWLEDGE_MIN_HYBRID_SCORE"), base.minHybridScore, 0, 1),
165
+ candidateMin: clampInt(envNumber("PI_KNOWLEDGE_SEARCH_CANDIDATE_MIN"), base.candidateMin, 10, 500),
166
+ candidateMultiplier: clampInt(envNumber("PI_KNOWLEDGE_SEARCH_CANDIDATE_MULTIPLIER"), base.candidateMultiplier, 2, 30),
167
+ adaptiveContextLines: clampInt(envNumber("PI_KNOWLEDGE_ADAPTIVE_CONTEXT_LINES"), base.adaptiveContextLines, 10, 500),
168
+ adaptiveMaxContextChars: clampInt(envNumber("PI_KNOWLEDGE_ADAPTIVE_MAX_CHARS"), base.adaptiveMaxContextChars, 1_000, 50_000),
169
+ adaptiveNeighborTarget: clampInt(envNumber("PI_KNOWLEDGE_ADAPTIVE_NEIGHBOR_TARGET"), base.adaptiveNeighborTarget, 1, 20),
170
+ deepRerankCandidates: clampInt(envNumber("PI_KNOWLEDGE_DEEP_RERANK_CANDIDATES"), base.deepRerankCandidates, 5, 100),
171
+ deepRerankTopKMultiplier: clampInt(envNumber("PI_KNOWLEDGE_DEEP_RERANK_TOPK_MULTIPLIER"), base.deepRerankTopKMultiplier, 1, 10),
172
+ };
173
+ }
174
+ export function summarizeSearchTuning(tuning, limit, candidateLimit) {
175
+ return {
176
+ requested_profile: tuning.requestedProfile,
177
+ selected_profile: tuning.selectedProfile,
178
+ limit,
179
+ snippet_max_length: tuning.snippetMaxLength,
180
+ min_hybrid_score: tuning.minHybridScore,
181
+ candidate_limit: candidateLimit,
182
+ candidate_min: tuning.candidateMin,
183
+ candidate_multiplier: tuning.candidateMultiplier,
184
+ adaptive_context_lines: tuning.adaptiveContextLines,
185
+ adaptive_max_context_chars: tuning.adaptiveMaxContextChars,
186
+ adaptive_neighbor_target: tuning.adaptiveNeighborTarget,
187
+ deep_rerank_candidates: tuning.deepRerankCandidates,
188
+ deep_rerank_top_k_multiplier: tuning.deepRerankTopKMultiplier,
189
+ };
190
+ }
@@ -41,10 +41,21 @@ API embedding failures intentionally surface by default. Silent fallback can hid
41
41
  | `PI_KNOWLEDGE_WATCH` | unset | Set to `true` to start file watchers for directory KBs. The polling fallback remains available when native `fs.watch` fails. |
42
42
  | `PI_KNOWLEDGE_AUTO_INJECT` | unset | Set to `true` to auto-search KB context before model calls. This is opt-in. |
43
43
  | `PI_KNOWLEDGE_STALE_INDEXING_MS` | built-in stale threshold | Override the stale indexing threshold used by diagnostics and `knowledge_doctor`. |
44
-
45
- Search diagnostics include result provenance in both text output and structured `details`: chunk id, chunk hash, match reason, indexed timestamp, stale flag, and source mtime when the original source file is available. This provenance is stored in SQLite and derived from indexed source metadata; it does not create or modify files in the indexed project.
46
-
47
- `knowledge_search` supports `file_type` and `path_pattern` filters. `hybrid` mode is lexical-anchored to prevent low-evidence semantic false positives; use `semantic` when the query is conceptual and exact terms may differ from indexed wording.
44
+ | `PI_KNOWLEDGE_SEARCH_PROFILE` | `auto` | Select the default search tuning profile. Values: `auto`, `balanced`, `low_token`, `precision`, `recall`, `long_context`, `code`, `docs`. Tool `profile` parameters override this. |
45
+ | `PI_KNOWLEDGE_SEARCH_DEFAULT_LIMIT` | profile default | Override the result limit used only when `knowledge_search` does not pass `limit`. Clamped to 1-50. |
46
+ | `PI_KNOWLEDGE_SNIPPET_MAX_LENGTH` | profile default | Override returned snippet length. Clamped to 80-4000 characters. |
47
+ | `PI_KNOWLEDGE_MIN_HYBRID_SCORE` | profile default | Override the hybrid confidence gate. Higher values reduce noisy results; lower values increase recall. Clamped to 0-1. |
48
+ | `PI_KNOWLEDGE_SEARCH_CANDIDATE_MIN` | profile default | Override the minimum internal retrieval candidate pool. Clamped to 10-500. |
49
+ | `PI_KNOWLEDGE_SEARCH_CANDIDATE_MULTIPLIER` | profile default | Override the internal candidate multiplier applied to the effective result limit. Clamped to 2-30. |
50
+ | `PI_KNOWLEDGE_ADAPTIVE_CONTEXT_LINES` | profile default | Override surrounding lines fetched for `adaptive` context expansion. Clamped to 10-500. |
51
+ | `PI_KNOWLEDGE_ADAPTIVE_MAX_CHARS` | profile default | Override maximum adaptive context characters per result. Clamped to 1000-50000. |
52
+ | `PI_KNOWLEDGE_ADAPTIVE_NEIGHBOR_TARGET` | profile default | Override the number of nearby/query-relevant chunks considered for adaptive windows. Clamped to 1-20. |
53
+ | `PI_KNOWLEDGE_DEEP_RERANK_CANDIDATES` | profile default | Override how many filtered candidates `deep` mode sends to the reranker. Clamped to 5-100. |
54
+ | `PI_KNOWLEDGE_DEEP_RERANK_TOPK_MULTIPLIER` | profile default | Override deep rerank top-k multiplier relative to the effective result limit. Clamped to 1-10. |
55
+
56
+ Search diagnostics include result provenance and applied search tuning in both text output and structured `details`: selected profile, effective limit, snippet length, hybrid threshold, candidate pool size, chunk id, chunk hash, match reason, indexed timestamp, stale flag, and source mtime when the original source file is available. This provenance is stored in SQLite and derived from indexed source metadata; it does not create or modify files in the indexed project.
57
+
58
+ `knowledge_search` supports `file_type`, `path_pattern`, `profile`, and explicit `limit` filters. Search profiles tune result count, snippets, hybrid strictness, candidate breadth, adaptive context, and deep rerank breadth at runtime. `auto` chooses from query shape, mode, and KB source type; explicit tool parameters win over env overrides, env overrides win over profile defaults, and profile defaults win over built-in compatibility values. `hybrid` mode remains lexical-anchored to prevent low-evidence semantic false positives; use `semantic` when the query is conceptual and exact terms may differ from indexed wording.
48
59
 
49
60
  `knowledge_doctor` emits both human-readable issues and machine-readable actions in structured `details`. Action codes include `run_update`, `rebuild_kb`, `wait_for_indexing`, `review_skipped_scope`, `rebuild_vectors`, `check_source`, and `none`. Agents should prefer these action codes over parsing the English action text.
50
61
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-knowledge",
3
- "version": "0.5.5",
3
+ "version": "0.5.6",
4
4
  "description": "Local-first RAG knowledge base for Pi and OMP coding agents with hybrid code/doc search, reranking, diagnostics, and large-project indexing",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",