pi-knowledge 0.10.2 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,11 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.11.0] - 2026-09-23
4
+
5
+ ### Added
6
+ - Added `PI_KNOWLEDGE_EMBEDDING_PREFIXES=on|off` so OpenAI-compatible and local embeddings can preserve or omit `query: ` / `passage: ` input prefixes while keeping default E5 behavior compatible with existing KB signatures (#18).
7
+ - Added `semantic_hybrid` search mode for vector-backed weighted fusion without the default hybrid lexical-evidence gate, giving semantic and multilingual queries an explicit recall-oriented path while preserving `hybrid` precision defaults (#17).
8
+
3
9
  ## [0.10.2] - 2026-09-19
4
10
 
5
11
  ### Fixed
package/README.md CHANGED
@@ -67,7 +67,7 @@ TUI rendering currently uses Pi's stable default tool rendering plus targeted re
67
67
  - **Contextual Retrieval without remote chunk rewriting**: applies Anthropic's Contextual Retrieval insight locally by embedding file path, file type, Markdown breadcrumbs, and AST-derived code structure such as symbols, scope, parent symbol, and signatures with each chunk. This improves standalone chunk meaning without sending private source chunks to an LLM for context generation.
68
68
  - **Hybrid retrieval with diagnosable scores**: combines lexical BM25 anchors and vectors with normalized weighted score fusion. RRF (Cormack et al. 2009) remains the baseline reference, but weighted fusion is used by default because project dogfood showed RRF compressed scores too much for ranking diagnostics.
69
69
  - **MMR-style diversity**: uses Maximal Marginal Relevance ideas (Goldstein and Carbonell 1998), file interleaving, vector redundancy checks, and bounded adaptive-window overlap collapse so repeated README or same-file chunks do not dominate top results. Code KBs with AST metadata can also use same-parent/sibling structure during adaptive expansion.
70
- - **Intent-aware and self-correcting agent UX**: mode selection (`auto`, `fast`, `semantic`, `hybrid`, `adaptive`, `deep`), ranking diagnostics, and `knowledge_doctor` turn retrieval failures into concrete next actions instead of silent bad answers.
70
+ - **Intent-aware and self-correcting agent UX**: mode selection (`auto`, `fast`, `semantic`, `semantic_hybrid`, `hybrid`, `adaptive`, `deep`), ranking diagnostics, and `knowledge_doctor` turn retrieval failures into concrete next actions instead of silent bad answers.
71
71
  - **Confidence gating**: low-evidence hybrid matches can return zero results instead of unrelated chunks, reducing false confidence when the KB does not contain the answer.
72
72
 
73
73
  This is intentionally not a heavy ColBERT-style late-interaction index yet (Khattab and Zaharia 2020). The current product chooses lightweight local embeddings, BM25, query-aware ranking, optional cross-encoder reranking, streamed vector scans, and health diagnostics for commercial usefulness with low setup cost.
@@ -125,6 +125,7 @@ omp install ./pi-knowledge
125
125
  - `fast`: BM25 keyword search for exact symbols, commands, and identifiers.
126
126
  - `semantic`: vector search for conceptual matches.
127
127
  - `hybrid`: lexical-anchored BM25 + vector search with normalized weighted score fusion. It requires keyword evidence to avoid low-confidence semantic false positives.
128
+ - `semantic_hybrid`: vector search plus optional BM25 weighted fusion without the hybrid lexical-evidence gate; use explicitly for semantic/multilingual queries where lexical overlap is weak but fused scoring is still useful.
128
129
  - `deep`: hybrid retrieval followed by cross-encoder reranking.
129
130
  - `adaptive`: hybrid retrieval followed by query-time contextual window expansion around seed chunks. It keeps the matched seed, prefers nearby/query-relevant neighboring chunks, and collapses overlapping windows from the same file.
130
131
  - `auto`: selects a primary mode from the query shape and retries alternate modes when results are empty or weak.
@@ -136,7 +137,7 @@ Mode selection contract:
136
137
 
137
138
  - Start with `hybrid` for most project questions that contain useful lexical anchors.
138
139
  - Use `fast` for exact symbols, filenames, commands, error codes, API names, config keys, or quoted strings.
139
- - Use `semantic` when the query is conceptual, exact terms may differ from indexed wording, or hybrid returns no lexical matches.
140
+ - Use `semantic` when the query is conceptual and exact terms may differ from indexed wording; use `semantic_hybrid` when weak lexical overlap should not block vector-backed fused results.
140
141
  - Use `adaptive` when the answer needs nearby code, neighboring documentation sections, AST-related sibling methods where indexed, or enough context to make a safe edit.
141
142
  - Use `deep` for high-stakes answers, ambiguous top results, or final verification when slower reranking is acceptable.
142
143
  - If results are empty or weak but the KB should contain the answer, retry once with a different mode before concluding no answer exists.
@@ -185,7 +186,7 @@ Full configuration details are in [docs/configuration.md](docs/configuration.md)
185
186
  |------|-----------------------|
186
187
  | Storage path | `PI_KNOWLEDGE_DIR`, `OMP_KNOWLEDGE_DIR`, `PI_CODING_AGENT_DIR`, `OMP_CODING_AGENT_DIR`, `OMP_PROFILE` |
187
188
  | Model worker and cache | `PI_KNOWLEDGE_MODEL_CACHE_DIR`, `PI_KNOWLEDGE_NODE_PATH` |
188
- | Embedding provider | `PI_KNOWLEDGE_EMBEDDING`, `PI_KNOWLEDGE_EMBEDDING_API_KEY`, `OPENAI_API_KEY`, `PI_KNOWLEDGE_EMBEDDING_BASE_URL`, `OPENAI_BASE_URL`, `PI_KNOWLEDGE_EMBEDDING_MAX_CHARS`, `PI_KNOWLEDGE_EMBEDDING_BATCH_SIZE`, `PI_KNOWLEDGE_EMBEDDING_API_FALLBACK` |
189
+ | Embedding provider | `PI_KNOWLEDGE_EMBEDDING`, `PI_KNOWLEDGE_EMBEDDING_API_KEY`, `OPENAI_API_KEY`, `PI_KNOWLEDGE_EMBEDDING_BASE_URL`, `OPENAI_BASE_URL`, `PI_KNOWLEDGE_EMBEDDING_PREFIXES`, `PI_KNOWLEDGE_EMBEDDING_MAX_CHARS`, `PI_KNOWLEDGE_EMBEDDING_BATCH_SIZE`, `PI_KNOWLEDGE_EMBEDDING_API_FALLBACK` |
189
190
  | Reranker provider | `PI_KNOWLEDGE_RERANKER`, `PI_KNOWLEDGE_RERANKER_REVISION`, `PI_KNOWLEDGE_RERANKER_DTYPE`, `PI_KNOWLEDGE_RERANKER_REMOTE_HOST`, `PI_KNOWLEDGE_RERANKER_REMOTE_PATH_TEMPLATE`, `PI_KNOWLEDGE_RERANKER_RAW_LOGITS`, `PI_KNOWLEDGE_RERANKER_API_ENDPOINT`, `PI_KNOWLEDGE_RERANKER_API_BASE_URL`, `PI_KNOWLEDGE_RERANKER_API_KEY`, `PI_KNOWLEDGE_RERANKER_API_FORMAT`, `PI_KNOWLEDGE_RERANKER_API_TIMEOUT_MS`, `PI_KNOWLEDGE_RERANKER_MAX_DOC_CHARS`, `PI_KNOWLEDGE_RERANKER_API_RESULTS_PATH`, `PI_KNOWLEDGE_RERANKER_API_INDEX_FIELD`, `PI_KNOWLEDGE_RERANKER_API_SCORE_FIELD`, `PI_KNOWLEDGE_RERANKER_API_SCORE_DIRECTION` |
190
191
  | Native lifecycle | `PI_KNOWLEDGE_ENABLE_NATIVE_IDLE_DISPOSE`, `PI_KNOWLEDGE_EMBEDDING_IDLE_MS` |
191
192
  | Runtime features | `PI_KNOWLEDGE_WATCH`, `PI_KNOWLEDGE_AUTO_INJECT`, `PI_KNOWLEDGE_STALE_INDEXING_MS`, `PI_KNOWLEDGE_OFFLINE`, `PI_KNOWLEDGE_SEARCH_PROFILE`, `PI_KNOWLEDGE_SEARCH_DEFAULT_LIMIT`, `PI_KNOWLEDGE_SNIPPET_MAX_LENGTH`, `PI_KNOWLEDGE_MIN_HYBRID_SCORE`, `PI_KNOWLEDGE_SEARCH_CANDIDATE_MIN`, `PI_KNOWLEDGE_SEARCH_CANDIDATE_MULTIPLIER`, `PI_KNOWLEDGE_ADAPTIVE_CONTEXT_LINES`, `PI_KNOWLEDGE_ADAPTIVE_MAX_CHARS`, `PI_KNOWLEDGE_ADAPTIVE_NEIGHBOR_TARGET`, `PI_KNOWLEDGE_DEEP_RERANK_CANDIDATES`, `PI_KNOWLEDGE_DEEP_RERANK_TOPK_MULTIPLIER` |
package/dist/index.js CHANGED
@@ -364,7 +364,7 @@ export default function (pi) {
364
364
  "Use knowledge_search to find relevant context before answering domain questions",
365
365
  "Default to mode 'hybrid' for most project questions with useful lexical anchors; it fuses BM25 and semantic vectors but still requires keyword evidence",
366
366
  "Use mode 'fast' for exact symbols, filenames, commands, error codes, API names, config keys, or quoted strings",
367
- "Use mode 'semantic' for broad conceptual questions when exact terms may differ from the indexed wording or hybrid returns no lexical matches",
367
+ "Use mode 'semantic' for pure vector search when exact terms may differ from indexed wording; use mode 'semantic_hybrid' when you want vector recall plus BM25/fusion scoring without the hybrid lexical evidence gate",
368
368
  "Use mode 'adaptive' when the user needs surrounding implementation context, related nearby sections, or enough context to make a code change",
369
369
  "Use mode 'deep' for high-stakes answers, ambiguous top results, or final verification where slower cross-encoder reranking is acceptable",
370
370
  "Use profile 'low_token' for slow local models that need fewer, stricter, longer snippets; use profile 'precision' for identifiers/errors and 'recall' or 'long_context' for broad prose/document research",
@@ -378,6 +378,7 @@ export default function (pi) {
378
378
  Type.Literal("fast"),
379
379
  Type.Literal("semantic"),
380
380
  Type.Literal("hybrid"),
381
+ Type.Literal("semantic_hybrid"),
381
382
  Type.Literal("deep"),
382
383
  Type.Literal("adaptive"),
383
384
  Type.Literal("code"),
@@ -1,5 +1,6 @@
1
1
  export type EmbeddingProvider = "local" | "openai";
2
2
  export type EmbeddingPrefix = "query" | "passage";
3
+ export type EmbeddingPrefixStrategy = "on" | "off";
3
4
  export interface EmbeddingConfig {
4
5
  provider: EmbeddingProvider;
5
6
  model: string;
@@ -7,6 +8,7 @@ export interface EmbeddingConfig {
7
8
  maxChars: number;
8
9
  queryPrefix: "query";
9
10
  documentPrefix: "passage";
11
+ prefixes: EmbeddingPrefixStrategy;
10
12
  pooling: "mean";
11
13
  normalize: true;
12
14
  }
@@ -13,13 +13,14 @@ export const DEFAULT_API_MAX_EMBED_CHARS = 20_000;
13
13
  const IDLE_TIMEOUT_MS = Number(process.env.PI_KNOWLEDGE_EMBEDDING_IDLE_MS ?? 30_000);
14
14
  const ENABLE_NATIVE_IDLE_DISPOSE = process.env.PI_KNOWLEDGE_ENABLE_NATIVE_IDLE_DISPOSE === "true";
15
15
  const API_FALLBACK_TO_LOCAL = process.env.PI_KNOWLEDGE_EMBEDDING_API_FALLBACK === "local";
16
- function localEmbeddingConfig(maxChars) {
16
+ function localEmbeddingConfig(maxChars, prefixes) {
17
17
  return {
18
18
  provider: "local",
19
19
  model: DEFAULT_LOCAL_EMBEDDING_MODEL,
20
20
  maxChars,
21
21
  queryPrefix: "query",
22
22
  documentPrefix: "passage",
23
+ prefixes,
23
24
  pooling: "mean",
24
25
  normalize: true,
25
26
  };
@@ -65,6 +66,14 @@ function waitForNoActiveRuns() {
65
66
  function providerInstanceHash(value) {
66
67
  return createHash("sha256").update(value).digest("hex").slice(0, 12);
67
68
  }
69
+ function resolveEmbeddingPrefixStrategy(env) {
70
+ const raw = cleanEnv(env.PI_KNOWLEDGE_EMBEDDING_PREFIXES)?.toLowerCase();
71
+ if (!raw)
72
+ return "on";
73
+ if (raw === "on" || raw === "off")
74
+ return raw;
75
+ throw new Error(`Unsupported embedding prefix strategy: ${raw}. Use PI_KNOWLEDGE_EMBEDDING_PREFIXES=on or off.`);
76
+ }
68
77
  export function resolveEmbeddingConfig(env = process.env) {
69
78
  const raw = cleanEnv(env.PI_KNOWLEDGE_EMBEDDING) ?? `local:${DEFAULT_LOCAL_EMBEDDING_MODEL}`;
70
79
  const separator = raw.indexOf(":");
@@ -72,12 +81,13 @@ export function resolveEmbeddingConfig(env = process.env) {
72
81
  const model = separator >= 0 ? raw.slice(separator + 1).trim() : "";
73
82
  const maxCharsValue = Number(cleanEnv(env.PI_KNOWLEDGE_EMBEDDING_MAX_CHARS) ?? DEFAULT_API_MAX_EMBED_CHARS);
74
83
  const maxChars = Number.isFinite(maxCharsValue) && maxCharsValue > 0 ? Math.trunc(maxCharsValue) : DEFAULT_API_MAX_EMBED_CHARS;
84
+ const prefixes = resolveEmbeddingPrefixStrategy(env);
75
85
  if (provider === "local") {
76
86
  const localModel = model || DEFAULT_LOCAL_EMBEDDING_MODEL;
77
87
  if (localModel !== DEFAULT_LOCAL_EMBEDDING_MODEL) {
78
88
  throw new Error(`Unsupported local embedding model: ${localModel}`);
79
89
  }
80
- return localEmbeddingConfig(maxChars);
90
+ return localEmbeddingConfig(maxChars, prefixes);
81
91
  }
82
92
  if (provider === "openai") {
83
93
  return {
@@ -87,6 +97,7 @@ export function resolveEmbeddingConfig(env = process.env) {
87
97
  maxChars,
88
98
  queryPrefix: "query",
89
99
  documentPrefix: "passage",
100
+ prefixes,
90
101
  pooling: "mean",
91
102
  normalize: true,
92
103
  };
@@ -98,7 +109,7 @@ export function embeddingConfigLabel(config) {
98
109
  }
99
110
  export function embeddingSignature(config, dimension) {
100
111
  const base = config.baseUrl ? `:base-sha256=${providerInstanceHash(config.baseUrl)}` : "";
101
- return [
112
+ const parts = [
102
113
  `${config.provider}:${config.model}${base}`,
103
114
  `dim=${dimension}`,
104
115
  `apiMax=${config.provider === "openai" ? config.maxChars : "none"}`,
@@ -106,7 +117,10 @@ export function embeddingSignature(config, dimension) {
106
117
  `normalize=${config.normalize}`,
107
118
  `q=${config.queryPrefix}`,
108
119
  `d=${config.documentPrefix}`,
109
- ].join(":");
120
+ ];
121
+ if (config.prefixes === "off")
122
+ parts.push("prefixes=off");
123
+ return parts.join(":");
110
124
  }
111
125
  export async function dispose() {
112
126
  clearIdleTimer();
@@ -124,9 +138,14 @@ export async function prepareForShutdown() {
124
138
  clearIdleTimer();
125
139
  await waitForNoActiveRuns();
126
140
  }
141
+ function applyEmbeddingPrefix(text, prefix, config) {
142
+ return config.prefixes === "on" ? `${prefix}: ${text}` : text;
143
+ }
127
144
  async function embedViaAPI(texts, prefix, config, signal) {
128
- const prefixedTexts = texts.map((t) => `${prefix}: ${t}`);
129
- const safeTexts = prefixedTexts.map((text) => text.length > config.maxChars ? text.slice(0, config.maxChars) : text);
145
+ const safeTexts = texts.map((text) => {
146
+ const embeddingText = applyEmbeddingPrefix(text, prefix, config);
147
+ return embeddingText.length > config.maxChars ? embeddingText.slice(0, config.maxChars) : embeddingText;
148
+ });
130
149
  const apiKey = cleanEnv(process.env.PI_KNOWLEDGE_EMBEDDING_API_KEY) ?? cleanEnv(process.env.OPENAI_API_KEY);
131
150
  if (!config.baseUrl)
132
151
  throw new Error("OpenAI embedding base URL is not configured");
@@ -162,12 +181,12 @@ export async function embedTextsWithConfig(texts, prefix, signal, options = {})
162
181
  if (!API_FALLBACK_TO_LOCAL || !options.allowApiFallback)
163
182
  throw error;
164
183
  console.warn(`pi-knowledge: embedding API failed; falling back to local model because PI_KNOWLEDGE_EMBEDDING_API_FALLBACK=local (${error instanceof Error ? error.message : String(error)})`);
165
- actualConfig = localEmbeddingConfig(config.maxChars);
184
+ actualConfig = localEmbeddingConfig(config.maxChars, config.prefixes);
166
185
  }
167
186
  }
168
187
  beginRun();
169
188
  try {
170
- return { vectors: await embedInModelWorker(texts, prefix, signal), config: actualConfig };
189
+ return { vectors: await embedInModelWorker(texts, prefix, actualConfig.prefixes, signal), config: actualConfig };
171
190
  }
172
191
  finally {
173
192
  endRun();
@@ -3,7 +3,7 @@ import { createSkippedScanStats } from "./indexer/chunker.ts";
3
3
  import { type RankingDiagnostics } from "./search/ranking.ts";
4
4
  import { type SearchProfile, type SearchTuningSummary } from "./search/tuning.ts";
5
5
  import { type KnowledgeBase, type KnowledgeSymbol } from "./storage/sqlite.ts";
6
- export type SearchMode = "auto" | "fast" | "semantic" | "hybrid" | "deep" | "adaptive" | "code" | "config" | "docs" | "errors" | "decision";
6
+ export type SearchMode = "auto" | "fast" | "semantic" | "hybrid" | "semantic_hybrid" | "deep" | "adaptive" | "code" | "config" | "docs" | "errors" | "decision";
7
7
  export interface SearchOptions {
8
8
  mode?: SearchMode;
9
9
  profile?: SearchProfile;
@@ -1371,8 +1371,9 @@ export class KnowledgeEngine {
1371
1371
  vectorsByChunkId.set(chunkId, vector);
1372
1372
  }
1373
1373
  else {
1374
+ const semanticHybrid = retrievalMode === "semantic_hybrid";
1374
1375
  const bm25Results = searchBM25(db, normalizedQuery || query, candidateLimit, kb.id);
1375
- if (bm25Results.length === 0)
1376
+ if (bm25Results.length === 0 && !semanticHybrid)
1376
1377
  continue;
1377
1378
  let vecResults = [];
1378
1379
  const { vector: queryVec, config: queryEmbeddingConfig } = await embedQueryWithConfig(query, signal);
@@ -1422,6 +1423,8 @@ export class KnowledgeEngine {
1422
1423
  const chunk = getChunkById(db, result.chunkId);
1423
1424
  if (!chunk)
1424
1425
  return false;
1426
+ if (retrievalMode === "semantic_hybrid")
1427
+ return result.score >= tuning.minHybridScore;
1425
1428
  return result.score >= tuning.minHybridScore && hasEnoughLexicalEvidence(chunk, queryTokens);
1426
1429
  });
1427
1430
  }
@@ -1578,7 +1581,7 @@ export class KnowledgeEngine {
1578
1581
  tuning: tuningSummary,
1579
1582
  suggestions: results.length === 0
1580
1583
  ? [
1581
- "Try mode 'fast' for exact symbols or mode 'semantic' for conceptual wording.",
1584
+ "Try mode 'fast' for exact symbols, mode 'semantic' for pure vector search, or mode 'semantic_hybrid' when weak lexical overlap should not block fused vector results.",
1582
1585
  "Run knowledge_status if the KB should contain this answer.",
1583
1586
  ]
1584
1587
  : undefined,
@@ -2,7 +2,7 @@ import type { HfRerankerConfig } from "./search/reranker-config.ts";
2
2
  export declare function validateModelWorkerNodePath(nodePath: string): string;
3
3
  export declare function resolveModelWorkerNodePath(): string;
4
4
  export declare function configureModelWorkerNodePath(nodePath?: string): string;
5
- export declare function embedInModelWorker(texts: string[], prefix: "query" | "passage", signal?: AbortSignal): Promise<Float32Array[]>;
5
+ export declare function embedInModelWorker(texts: string[], prefix: "query" | "passage", prefixesOrSignal?: "on" | "off" | AbortSignal, signal?: AbortSignal): Promise<Float32Array[]>;
6
6
  export interface RerankWorkerCandidate {
7
7
  chunkId: string;
8
8
  content: string;
@@ -385,8 +385,10 @@ function getWorker() {
385
385
  IpcModelWorkerTransport.create(workerPath, nodeExecPath) ?? new StdioModelWorkerTransport(workerPath, nodeExecPath);
386
386
  return worker;
387
387
  }
388
- export async function embedInModelWorker(texts, prefix, signal) {
389
- const result = await getWorker().request({ type: "embed", texts, prefix }, signal);
388
+ export async function embedInModelWorker(texts, prefix, prefixesOrSignal, signal) {
389
+ const prefixes = prefixesOrSignal === "off" ? "off" : "on";
390
+ const requestSignal = typeof prefixesOrSignal === "object" ? prefixesOrSignal : signal;
391
+ const result = await getWorker().request({ type: "embed", texts, prefix, prefixes }, requestSignal);
390
392
  if (!Array.isArray(result))
391
393
  throw new Error("Invalid embedding worker response");
392
394
  return result.map((vector) => {
@@ -66,11 +66,17 @@ async function loadRerankerPipeline(config) {
66
66
  rerankerPipeline = { key, pipe: loaded };
67
67
  return loaded;
68
68
  }
69
+ function applyEmbeddingPrefix(text, prefix, prefixes) {
70
+ return prefixes === "on" ? `${prefix}: ${text}` : text;
71
+ }
69
72
  async function handleEmbed(request) {
70
73
  const pipe = await loadEmbeddingPipeline();
71
74
  const vectors = [];
72
75
  for (const text of request.texts) {
73
- const output = await pipe(`${request.prefix}: ${text}`, { pooling: "mean", normalize: true });
76
+ const output = await pipe(applyEmbeddingPrefix(text, request.prefix, request.prefixes), {
77
+ pooling: "mean",
78
+ normalize: true,
79
+ });
74
80
  vectors.push(Array.from(output.data));
75
81
  }
76
82
  return vectors;
@@ -110,7 +116,11 @@ function parseWorkerRequest(message) {
110
116
  if (!("prefix" in message) || (message.prefix !== "query" && message.prefix !== "passage")) {
111
117
  throw new Error("Invalid embed prefix");
112
118
  }
113
- return { id: message.id, type: "embed", texts: message.texts, prefix: message.prefix };
119
+ const prefixes = "prefixes" in message && (message.prefixes === "on" || message.prefixes === "off") ? message.prefixes : "on";
120
+ if ("prefixes" in message && message.prefixes !== "on" && message.prefixes !== "off") {
121
+ throw new Error("Invalid embed prefix strategy");
122
+ }
123
+ return { id: message.id, type: "embed", texts: message.texts, prefix: message.prefix, prefixes };
114
124
  }
115
125
  if (message.type === "rerank") {
116
126
  if (!("query" in message) || typeof message.query !== "string")
@@ -1,6 +1,6 @@
1
1
  export type SearchProfile = "auto" | "balanced" | "low_token" | "precision" | "recall" | "long_context" | "code" | "docs";
2
2
  export type RuntimeProfile = Exclude<SearchProfile, "auto">;
3
- type SearchModeForTuning = "auto" | "fast" | "semantic" | "hybrid" | "deep" | "adaptive" | "code" | "config" | "docs" | "errors" | "decision";
3
+ type SearchModeForTuning = "auto" | "fast" | "semantic" | "hybrid" | "semantic_hybrid" | "deep" | "adaptive" | "code" | "config" | "docs" | "errors" | "decision";
4
4
  export interface SearchTuning {
5
5
  requestedProfile: SearchProfile;
6
6
  selectedProfile: RuntimeProfile;
@@ -143,7 +143,7 @@ function chooseRuntimeProfile(input) {
143
143
  }
144
144
  if (input.mode === "docs" || input.mode === "decision" || input.mode === "adaptive")
145
145
  return "long_context";
146
- if (input.mode === "semantic")
146
+ if (input.mode === "semantic" || input.mode === "semantic_hybrid")
147
147
  return "recall";
148
148
  if (looksErrorQuery(input.query) || looksExactOrCodeQuery(input.query))
149
149
  return "precision";
@@ -25,6 +25,7 @@ Default data storage is `~/.pi/knowledge` under Pi and `~/.omp/knowledge` under
25
25
  | `OPENAI_API_KEY` | unset | Fallback API key used when `PI_KNOWLEDGE_EMBEDDING=openai:<model>`. Required by OpenAI's hosted API; many local OpenAI-compatible servers do not need it. |
26
26
  | `PI_KNOWLEDGE_EMBEDDING_BASE_URL` | unset | OpenAI-compatible embedding API root, such as `http://127.0.0.1:8080/v1`. Takes precedence over `OPENAI_BASE_URL`. |
27
27
  | `OPENAI_BASE_URL` | `https://api.openai.com/v1` | Common OpenAI-compatible API root fallback. |
28
+ | `PI_KNOWLEDGE_EMBEDDING_PREFIXES` | `on` | Controls whether `query: ` and `passage: ` prefixes are prepended before embedding. Values: `on`, `off`. Keep `on` for the default E5 local model; use `off` only for models or self-hosted APIs that are evaluated without E5-style prefixes. |
28
29
  | `PI_KNOWLEDGE_EMBEDDING_MAX_CHARS` | `20000` | Final per-input API embedding safety cap for OpenAI-compatible servers with smaller context windows. This does not replace chunker bounds. |
29
30
  | `PI_KNOWLEDGE_EMBEDDING_BATCH_SIZE` | `64` | Maximum number of document chunks sent in one embedding request during add, update, and import. Lower this for self-hosted embedding servers with smaller request batch limits. |
30
31
  | `PI_KNOWLEDGE_EMBEDDING_API_FALLBACK` | unset | Set to `local` to explicitly fall back to local embeddings for query embeddings after API failures. Indexing/import/update document batches do not fall back because mixed-provider vector files are unsafe. |
@@ -32,7 +33,7 @@ Default data storage is `~/.pi/knowledge` under Pi and `~/.omp/knowledge` under
32
33
  | `PI_KNOWLEDGE_EMBEDDING_IDLE_MS` | `30000` | Idle timer used only when native idle disposal coordination is enabled. Mainly for lifecycle stress tests. |
33
34
  | `PI_KNOWLEDGE_OFFLINE` | unset | Use with a pre-populated model cache for offline local model operation. See `docs/offline-mode.md`. |
34
35
 
35
- Each KB stores the embedding model label, vector dimension, and a non-secret embedding signature that includes provider/model semantics, query/document prefixes, pooling, normalization, API input cap, and a hash of the API base URL when applicable. `knowledge_search` skips vector retrieval for KBs whose stored signature or dimension is missing or incompatible with the current query embedding and uses BM25-only fallback for hybrid searches. Run `knowledge_update` to rebuild vectors after changing embedding provider/model/max-character settings or after upgrading older KBs with no signature metadata.
36
+ Each KB stores the embedding model label, vector dimension, and a non-secret embedding signature that includes provider/model semantics, query/document prefixes, non-default prefix strategy, pooling, normalization, API input cap, and a hash of the API base URL when applicable. `knowledge_search` skips vector retrieval for KBs whose stored signature or dimension is missing or incompatible with the current query embedding and uses BM25-only fallback for hybrid searches. Run `knowledge_update` to rebuild vectors after changing embedding provider/model/max-character/prefix settings or after upgrading older KBs with no signature metadata.
36
37
 
37
38
  ## Reranker
38
39
 
@@ -84,7 +85,7 @@ API embedding failures intentionally surface by default. Silent fallback can hid
84
85
 
85
86
  Search diagnostics include result provenance and applied search tuning in both text output and structured `details`: selected profile, effective limit, snippet length, hybrid threshold, candidate pool size, chunk id, chunk hash, match reason, indexed timestamp, stale flag, and source mtime when the original source file is available. This provenance is stored in SQLite and derived from indexed source metadata; it does not create or modify files in the indexed project.
86
87
 
87
- `knowledge_search` supports `file_type`, `path_pattern`, `profile`, and explicit `limit` filters. Search profiles tune result count, snippets, hybrid strictness, candidate breadth, adaptive context, and deep rerank breadth at runtime. `auto` chooses from query shape, mode, and KB source type; explicit tool parameters win over env overrides, env overrides win over profile defaults, and profile defaults win over built-in compatibility values. `hybrid` mode remains lexical-anchored to prevent low-evidence semantic false positives; use `semantic` when the query is conceptual and exact terms may differ from indexed wording. `adaptive` uses bounded same-file context expansion; when AST metadata is present it can prefer same-parent/sibling chunks, but it remains metadata-driven retrieval rather than LSP, caller/callee, or code graph analysis.
88
+ `knowledge_search` supports `file_type`, `path_pattern`, `profile`, and explicit `limit` filters. Search profiles tune result count, snippets, hybrid strictness, candidate breadth, adaptive context, and deep rerank breadth at runtime. `auto` chooses from query shape, mode, and KB source type; explicit tool parameters win over env overrides, env overrides win over profile defaults, and profile defaults win over built-in compatibility values. `hybrid` mode remains lexical-anchored to prevent low-evidence semantic false positives; use `semantic` for pure vector search, or `semantic_hybrid` when weak lexical overlap should not block vector-backed weighted fusion. `adaptive` uses bounded same-file context expansion; when AST metadata is present it can prefer same-parent/sibling chunks, but it remains metadata-driven context expansion rather than caller/callee analysis.
88
89
 
89
90
  `knowledge_doctor` emits both human-readable issues and machine-readable actions in structured `details`. Action codes include `run_update`, `rebuild_kb`, `wait_for_indexing`, `review_skipped_scope`, `rebuild_vectors`, `check_source`, and `none`. Agents should prefer these action codes over parsing the English action text.
90
91
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-knowledge",
3
- "version": "0.10.2",
3
+ "version": "0.11.0",
4
4
  "description": "Local-first RAG knowledge base for Pi and OMP coding agents with hybrid code/doc search, reranking, diagnostics, and large-project indexing",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",