@dudousxd/nestjs-agent-rag 0.8.0 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -96,6 +96,61 @@ Either way, retrieved passages carry a `source` and surface as citations through
96
96
  tool-call machinery (rendered in the chat UI, shown in the `-telescope` Agent tab). Each retrieval
97
97
  also emits an `aviary:agent:retrieved` diagnostics event.
98
98
 
99
+ ## Retrieval telemetry
100
+
101
+ A tool-call row says a retrieval *happened*. It cannot say how long it took, how much came back, how
102
+ good the scores were, which store answered, or whether it found nothing at all — a retrieval that
103
+ returns zero passages and one that returns five perfect ones are the same row with the same `ok`
104
+ status. So retrieval also publishes its own event, on `aviary:rag:retrieval`:
105
+
106
+ ```ts
107
+ // envelope: { durationMs, ts, traceId } · payload:
108
+ {
109
+ retriever: 'embedding' | 'keyword' | 'lexical' | 'hybrid' | 'filtered' | 'reranking' | 'unknown',
110
+ store?: 'memory' | 'pg' | 'redis', // absent for an in-process retriever
111
+ collection?: string, // pg table / RediSearch index, or your own label
112
+ topK: number,
113
+ chunks: number,
114
+ zeroHit: boolean,
115
+ topScore?: number, // 4dp; absent on a zero-hit
116
+ meanScore?: number,
117
+ failed: boolean,
118
+ error?: string,
119
+ }
120
+ ```
121
+
122
+ Counts, durations and kinds only — never the query text, never passage text.
123
+
124
+ `createRetrievalTool` instruments the retriever it wraps **by default**, so agentic retrieval needs
125
+ no wiring. Wrap it yourself for inject mode, or for any code path that calls `retrieve` directly:
126
+
127
+ ```ts
128
+ import { instrumentRetriever } from '@dudousxd/nestjs-agent-rag';
129
+
130
+ const retriever = instrumentRetriever(
131
+ new HybridRetriever([new EmbeddingRetriever(embedder, store), new LexicalRetriever(store)]),
132
+ // optional: name the collection per call, when one index holds many
133
+ { collection: (filter) => (typeof filter?.collectionId === 'string' ? filter.collectionId : undefined) },
134
+ );
135
+ ```
136
+
137
+ Three things worth knowing:
138
+
139
+ - **A composed retrieval is ONE event.** The outermost instrumented retriever reports; anything
140
+ instrumented inside it stays silent. `RerankingRetriever` over a `HybridRetriever` over two legs is
141
+ one retrieval, reported as `retriever: 'reranking'` with the duration and scores the caller got.
142
+ Wrapping a leg as well as the composite is safe — it will not double-count.
143
+ - **It costs nothing when nobody listens.** `emit` is gated on `channel.hasSubscribers`, and the
144
+ wrapper checks the same gate before it reads a clock or touches a score, so an install with no
145
+ Telescope (or no subscriber at all) delegates straight through.
146
+ - **It never changes behaviour.** Passages come back verbatim; a throwing retriever is reported as
147
+ `failed` and rethrown unchanged.
148
+
149
+ `@dudousxd/nestjs-agent-telescope` subscribes to this channel and renders the **Retrieval** and
150
+ **Retrieval sources** sections of the Agent tab. Anything else can subscribe too —
151
+ `RAG_RETRIEVAL_CHANNEL` is the exported wire contract, and a host that needs retrieval history to
152
+ outlive a restart persists it from there.
153
+
99
154
  ## pgvector store
100
155
 
101
156
  `PgVectorStore` takes an injected `PgClient` — adapt your own `pg` / `postgres.js`:
@@ -323,11 +378,15 @@ library upgrade. The interface is open, so it can be added later without changin
323
378
  - `HybridRetriever(retrievers, { k?, fetchTopK?, weights? })` — RRF fusion of the two.
324
379
  - `MemoryVectorStore` / `PgVectorStore` / `RedisVectorStore` — `VectorStore` adapters:
325
380
  `upsert(records)`, `search(embedding, opts)`, `remove(documentId)`,
326
- `updateMetadata(documentId, patch)`, `listDocuments(filter?)`.
381
+ `updateMetadata(documentId, patch)`, `listDocuments(filter?)`,
382
+ `listChunks(documentId, { limit?, offset? })`.
327
383
  - `applyMetadataPatch(metadata, patch)` — the `MetadataPatch` merge semantics, exported so your own
328
384
  `VectorStore` can implement `updateMetadata` with exactly the same ones.
329
385
  - `isLexicalVectorStore(store)` — type guard for the optional `LexicalVectorStore` capability.
330
- - `createRetrievalTool(retriever, { name?, description?, topK? })` — the agentic-retrieval tool.
386
+ - `createRetrievalTool(retriever, { name?, description?, topK?, telemetry? })` — the
387
+ agentic-retrieval tool. `telemetry` defaults to on; pass `false` to opt out.
388
+ - `instrumentRetriever(retriever, { describe?, collection? })` — wrap any `Retriever` so each
389
+ retrieval publishes on `aviary:rag:retrieval` (`RAG_RETRIEVAL_CHANNEL`).
331
390
 
332
391
  ## Collection maintenance
333
392
 
@@ -363,6 +422,33 @@ would mean stamping the document id as an indexed field at write time, and chunk
363
422
  change carry no such field — the search would find none of them and `remove` would silently stop
364
423
  deleting. The methods above need no new field, so they have no migration.
365
424
 
425
+ ## Reading chunks back
426
+
427
+ Everything above enumerates *documents*. The text itself is otherwise write-only: `search` needs an
428
+ embedding, `searchText` needs a query, and both return what **matched**, in relevance order, capped
429
+ at `topK`. Neither answers "show me what was indexed for this document, in order" — the question you
430
+ ask when retrieval returned the wrong passage and you need to know whether the chunker cut badly or
431
+ the retriever chose badly.
432
+
433
+ ```ts
434
+ const chunks = await store.listChunks('handbook.pdf', { limit: 20 });
435
+ // [{ id: 'handbook.pdf#0', index: 0, text: 'Chapter 1 …', metadata: { collection: 'handbook' } }, …]
436
+ ```
437
+
438
+ Ordered by `index` (the `n` of `${documentId}#<n>`, numerically — `#10` after `#2`), so paging with
439
+ `limit`/`offset` walks the document front to back. A document stored under a bare id is chunk `0`. An
440
+ unknown `documentId` yields `[]`.
441
+
442
+ Document-scoped rather than filter-scoped, matching `remove` and `updateMetadata`: ordering only
443
+ means something once a document is picked, so enumerate with `listDocumentIds(filter)` first when you
444
+ need to sweep. Two things it deliberately is not:
445
+
446
+ - **not an access-control seam** — it takes no filter and applies none, so a consumer that gates
447
+ retrieval on a metadata field must resolve that gate itself before calling, exactly as before
448
+ `remove`. It is shaped for an operator/debug surface, not for serving end users;
449
+ - **not a hot path** — on Redis it costs one keyspace `SCAN`, the same as `remove`. Fine for one
450
+ document on demand, wrong inside a loop over a corpus.
451
+
366
452
  ### Why these are required and `searchText` is optional
367
453
 
368
454
  A method is **required on `VectorStore`** when every backend can implement it from what it already
package/dist/index.cjs CHANGED
@@ -28,17 +28,27 @@ __export(index_exports, {
28
28
  LexicalRetriever: () => LexicalRetriever,
29
29
  MemoryVectorStore: () => MemoryVectorStore,
30
30
  PgVectorStore: () => PgVectorStore,
31
+ RAG_DIAGNOSTIC_LIB: () => RAG_DIAGNOSTIC_LIB,
32
+ RAG_RETRIEVAL_CHANNEL: () => RAG_RETRIEVAL_CHANNEL,
33
+ RAG_RETRIEVAL_EVENT: () => RAG_RETRIEVAL_EVENT,
31
34
  RedisVectorSchemaMismatchError: () => RedisVectorSchemaMismatchError,
32
35
  RedisVectorStore: () => RedisVectorStore,
33
36
  RerankingRetriever: () => RerankingRetriever,
34
37
  UnsafeRemovalError: () => UnsafeRemovalError,
35
38
  applyMetadataPatch: () => applyMetadataPatch,
36
39
  chunkDocuments: () => chunkDocuments,
40
+ chunkIndexOf: () => chunkIndexOf,
37
41
  chunkText: () => chunkText,
38
42
  createRetrievalTool: () => createRetrievalTool,
43
+ describeRetrieval: () => describeRetrieval,
44
+ describeSharedSource: () => describeSharedSource,
45
+ describeSource: () => describeSource,
46
+ describesRetrieval: () => describesRetrieval,
39
47
  documentIdOf: () => documentIdOf,
48
+ emitRagRetrieval: () => emitRagRetrieval,
40
49
  ingestChunks: () => ingestChunks,
41
50
  ingestDocuments: () => ingestDocuments,
51
+ instrumentRetriever: () => instrumentRetriever,
42
52
  isLexicalVectorStore: () => isLexicalVectorStore
43
53
  });
44
54
  module.exports = __toCommonJS(index_exports);
@@ -133,6 +143,11 @@ var UnsafeRemovalError = class extends Error {
133
143
  this.name = "UnsafeRemovalError";
134
144
  }
135
145
  };
146
+ function chunkIndexOf(chunkId) {
147
+ const match = /#(\d+)$/.exec(chunkId);
148
+ return match?.[1] === void 0 ? 0 : Number(match[1]);
149
+ }
150
+ __name(chunkIndexOf, "chunkIndexOf");
136
151
  function documentIdOf(chunkId) {
137
152
  return chunkId.replace(/#\d+$/, "");
138
153
  }
@@ -202,6 +217,16 @@ var MemoryVectorStore = class {
202
217
  __name(this, "MemoryVectorStore");
203
218
  }
204
219
  records = /* @__PURE__ */ new Map();
220
+ /**
221
+ * Telemetry self-description. No `collection`: there is no namespace to name — the corpus IS this
222
+ * instance, and two `MemoryVectorStore`s in one process are two corpora with nothing to tell them
223
+ * apart on a dashboard.
224
+ */
225
+ describeRetrieval() {
226
+ return {
227
+ store: "memory"
228
+ };
229
+ }
205
230
  async upsert(records) {
206
231
  for (const record of records) {
207
232
  this.records.set(record.id, record);
@@ -257,6 +282,34 @@ var MemoryVectorStore = class {
257
282
  ...documents.values()
258
283
  ];
259
284
  }
285
+ /**
286
+ * The chunks of one document, in document order. See {@link VectorStore.listChunks}.
287
+ *
288
+ * Insertion order is not document order — `upsert` stores whatever the caller passed, and a
289
+ * re-ingest that rewrites chunk 3 alone moves it to the end of the Map. So this sorts by the index
290
+ * parsed from the id, the same thing the SQL and Redis adapters order by, rather than trusting the
291
+ * Map. A reference adapter that returned "usually ordered" would let a consumer ship a bug that
292
+ * only shows up on the store tests are NOT written against.
293
+ */
294
+ async listChunks(documentId, options) {
295
+ const chunks = [];
296
+ for (const record of this.records.values()) {
297
+ if (documentIdOf(record.id) !== documentId) {
298
+ continue;
299
+ }
300
+ chunks.push({
301
+ id: record.id,
302
+ index: chunkIndexOf(record.id),
303
+ text: record.text,
304
+ ...record.metadata !== void 0 ? {
305
+ metadata: record.metadata
306
+ } : {}
307
+ });
308
+ }
309
+ chunks.sort((a, b) => a.index - b.index);
310
+ const offset = options?.offset ?? 0;
311
+ return options?.limit === void 0 ? chunks.slice(offset) : chunks.slice(offset, offset + options.limit);
312
+ }
260
313
  async listDocumentIds(filter) {
261
314
  if (filterMatchesNothing(filter)) {
262
315
  return [];
@@ -368,6 +421,13 @@ var PgVectorStore = class {
368
421
  this.table = options.table ?? "agent_rag_chunks";
369
422
  this.dimensions = options.dimensions ?? 1536;
370
423
  }
424
+ /** Telemetry self-description — the table is the namespace a retrieval was served from. */
425
+ describeRetrieval() {
426
+ return {
427
+ store: "pg",
428
+ collection: this.table
429
+ };
430
+ }
371
431
  /** Idempotent DDL: the `vector` extension, the chunks table, and the cosine HNSW index. */
372
432
  async ensureSchema() {
373
433
  await this.client.query("CREATE EXTENSION IF NOT EXISTS vector");
@@ -443,6 +503,44 @@ var PgVectorStore = class {
443
503
  } : {}
444
504
  }));
445
505
  }
506
+ /**
507
+ * The chunks of one document, in document order. See {@link VectorStore.listChunks}.
508
+ *
509
+ * The order is computed from the id rather than stored: `#12` must sort after `#2`, which a plain
510
+ * `ORDER BY id` gets wrong on every document past ten chunks because it compares text. So the
511
+ * trailing number is extracted and cast to `int`, and a bare id (a single-chunk document, which
512
+ * carries no `#n` suffix) collapses to `0` — the same rule `chunkIndexOf` applies in the other two
513
+ * adapters.
514
+ *
515
+ * `embedding` is left out of the projection: it is the widest column in the table and useless to a
516
+ * caller reading text back.
517
+ */
518
+ async listChunks(documentId, options) {
519
+ const params = [
520
+ documentId
521
+ ];
522
+ let sql = `SELECT id, ${CHUNK_INDEX_FROM_ID} AS chunk_index, text, metadata
523
+ FROM ${this.table}
524
+ WHERE ${DOCUMENT_ID_FROM_CHUNK} = $1
525
+ ORDER BY chunk_index`;
526
+ if (options?.limit !== void 0) {
527
+ params.push(options.limit);
528
+ sql += ` LIMIT $${params.length}`;
529
+ }
530
+ if (options?.offset !== void 0) {
531
+ params.push(options.offset);
532
+ sql += ` OFFSET $${params.length}`;
533
+ }
534
+ const rows = await this.client.query(sql, params);
535
+ return rows.map((row) => ({
536
+ id: row.id,
537
+ index: Number(row.chunk_index),
538
+ text: row.text,
539
+ ...row.metadata !== null ? {
540
+ metadata: row.metadata
541
+ } : {}
542
+ }));
543
+ }
446
544
  /** {@link listDocuments} without the metadata: one `DISTINCT` over the derived document id. */
447
545
  async listDocumentIds(filter) {
448
546
  if (filterMatchesNothing(filter)) {
@@ -553,6 +651,7 @@ function buildWhere(filter, startIndex) {
553
651
  }
554
652
  __name(buildWhere, "buildWhere");
555
653
  var DOCUMENT_ID_FROM_CHUNK = "regexp_replace(id, '#[0-9]+$', '')";
654
+ var CHUNK_INDEX_FROM_ID = "COALESCE((substring(id from '#([0-9]+)$'))::int, 0)";
556
655
  function toVectorLiteral(embedding) {
557
656
  return `[${embedding.join(",")}]`;
558
657
  }
@@ -591,6 +690,17 @@ var RedisVectorStore = class {
591
690
  this.lexicalScorer = options.lexicalScorer ?? "BM25";
592
691
  }
593
692
  /**
693
+ * Telemetry self-description — the RediSearch INDEX, not the key prefix: the index is what both
694
+ * legs query, and two stores sharing one index (the dense and lexical halves of a hybrid) must
695
+ * report the same namespace or `describeSharedSource` would see them disagree and drop the store.
696
+ */
697
+ describeRetrieval() {
698
+ return {
699
+ store: "redis",
700
+ collection: this.index
701
+ };
702
+ }
703
+ /**
594
704
  * Idempotent, and drift-aware: create the HNSW/cosine index if `FT.INFO` says it doesn't exist —
595
705
  * and if it DOES exist, compare what's there against this store's configuration instead of
596
706
  * assuming they agree. An index that outlives a config change is the normal case (schemas live in
@@ -845,6 +955,58 @@ var RedisVectorStore = class {
845
955
  ...keys
846
956
  ];
847
957
  }
958
+ /**
959
+ * The chunks of one document, in document order. See {@link VectorStore.listChunks}.
960
+ *
961
+ * Reads the hashes directly (`HMGET text metadata_json`) rather than going through `FT.SEARCH`.
962
+ * Two reasons, and the second is the one that matters: the query language has no way to ask for
963
+ * "every chunk of this document" — `id` is the key name, not an indexed field — and `FT.SEARCH`
964
+ * caps at `LIMIT`, so paging a document through it would mean ordering by a field the index does
965
+ * not hold. `chunkKeys` is the same enumeration {@link RedisVectorStore.remove} and
966
+ * {@link RedisVectorStore.updateMetadata} already use, so "the chunks this returns" and "the chunks
967
+ * a delete would take" cannot drift apart.
968
+ *
969
+ * It inherits that method's cost too: one keyspace `SCAN` per call. Acceptable here because this
970
+ * is an inspection path (one document, on demand) and not something retrieval runs — but it is why
971
+ * this must not be put on a hot path or called in a loop over a corpus.
972
+ *
973
+ * `embedding` is deliberately not fetched: it is by far the largest field on the hash and nothing
974
+ * a caller reading text back can do with it.
975
+ */
976
+ async listChunks(documentId, options) {
977
+ const keys = await this.chunkKeys(documentId);
978
+ const ordered = keys.map((key) => ({
979
+ key,
980
+ id: key.slice(this.prefix.length)
981
+ })).map((entry) => ({
982
+ ...entry,
983
+ index: chunkIndexOf(entry.id)
984
+ })).sort((a, b) => a.index - b.index);
985
+ const offset = options?.offset ?? 0;
986
+ const page = options?.limit === void 0 ? ordered.slice(offset) : ordered.slice(offset, offset + options.limit);
987
+ const chunks = [];
988
+ for (const entry of page) {
989
+ const reply = await this.client.sendCommand([
990
+ "HMGET",
991
+ entry.key,
992
+ "text",
993
+ "metadata_json"
994
+ ]);
995
+ const fields = Array.isArray(reply) ? reply : [];
996
+ const text = fields[0] === null || fields[0] === void 0 ? void 0 : toStr(fields[0]);
997
+ if (text === void 0) {
998
+ continue;
999
+ }
1000
+ const raw = fields[1] === null || fields[1] === void 0 ? void 0 : toStr(fields[1]);
1001
+ chunks.push({
1002
+ id: entry.id,
1003
+ index: entry.index,
1004
+ text,
1005
+ ...parseMetadata(raw)
1006
+ });
1007
+ }
1008
+ return chunks;
1009
+ }
848
1010
  async listDocuments(filter) {
849
1011
  if (filterMatchesNothing(filter)) {
850
1012
  return [];
@@ -1367,6 +1529,31 @@ function parseMetadata(raw) {
1367
1529
  }
1368
1530
  __name(parseMetadata, "parseMetadata");
1369
1531
 
1532
+ // src/retrieval-descriptor.ts
1533
+ function describesRetrieval(value) {
1534
+ return typeof value === "object" && value !== null && typeof Reflect.get(value, "describeRetrieval") === "function";
1535
+ }
1536
+ __name(describesRetrieval, "describesRetrieval");
1537
+ function describeRetrieval(candidate) {
1538
+ return describesRetrieval(candidate) ? candidate.describeRetrieval() : {};
1539
+ }
1540
+ __name(describeRetrieval, "describeRetrieval");
1541
+ function describeSource(base) {
1542
+ const { retriever: _ignored, ...source } = describeRetrieval(base);
1543
+ return source;
1544
+ }
1545
+ __name(describeSource, "describeSource");
1546
+ function describeSharedSource(legs) {
1547
+ const sources = legs.map(describeSource);
1548
+ const [first] = sources;
1549
+ if (first === void 0) {
1550
+ return {};
1551
+ }
1552
+ const agrees = sources.every((source) => source.store === first.store && source.collection === first.collection);
1553
+ return agrees ? first : {};
1554
+ }
1555
+ __name(describeSharedSource, "describeSharedSource");
1556
+
1370
1557
  // src/embedding-retriever.ts
1371
1558
  var EmbeddingRetriever = class {
1372
1559
  static {
@@ -1380,6 +1567,13 @@ var EmbeddingRetriever = class {
1380
1567
  this.store = store;
1381
1568
  this.options = options;
1382
1569
  }
1570
+ /** Telemetry self-description — the dense strategy, over whatever store it was handed. */
1571
+ describeRetrieval() {
1572
+ return {
1573
+ retriever: "embedding",
1574
+ ...describeSource(this.store)
1575
+ };
1576
+ }
1383
1577
  async retrieve(query, options = {}) {
1384
1578
  const [embedding] = await this.embedder.embed([
1385
1579
  query
@@ -1412,6 +1606,16 @@ var KeywordRetriever = class {
1412
1606
  this.k1 = options.k1 ?? 1.5;
1413
1607
  this.b = options.b ?? 0.75;
1414
1608
  }
1609
+ /**
1610
+ * Telemetry self-description. No `store`: this retriever holds its own copy of the corpus in
1611
+ * process, so attributing its retrievals to whichever vector store happens to hold the same chunks
1612
+ * would credit a backend that was never queried.
1613
+ */
1614
+ describeRetrieval() {
1615
+ return {
1616
+ retriever: "keyword"
1617
+ };
1618
+ }
1415
1619
  /** How many chunks are currently indexed. */
1416
1620
  get size() {
1417
1621
  return this.docs.size;
@@ -1529,6 +1733,13 @@ var LexicalRetriever = class {
1529
1733
  constructor(store) {
1530
1734
  this.store = store;
1531
1735
  }
1736
+ /** Telemetry self-description — the store's own lexical index, so the store is the same one. */
1737
+ describeRetrieval() {
1738
+ return {
1739
+ retriever: "lexical",
1740
+ ...describeSource(this.store)
1741
+ };
1742
+ }
1532
1743
  async retrieve(query, options = {}) {
1533
1744
  return this.store.searchText(query, {
1534
1745
  topK: options.topK ?? 5,
@@ -1550,6 +1761,13 @@ var HybridRetriever = class {
1550
1761
  this.retrievers = retrievers;
1551
1762
  this.options = options;
1552
1763
  }
1764
+ /** Telemetry self-description — the store only when every leg agrees; see `describeSharedSource`. */
1765
+ describeRetrieval() {
1766
+ return {
1767
+ retriever: "hybrid",
1768
+ ...describeSharedSource(this.retrievers)
1769
+ };
1770
+ }
1553
1771
  async retrieve(query, options = {}) {
1554
1772
  const k = this.options.k ?? 60;
1555
1773
  const fetchTopK = this.options.fetchTopK ?? 20;
@@ -1598,6 +1816,13 @@ var RerankingRetriever = class {
1598
1816
  this.reranker = reranker;
1599
1817
  this.options = options;
1600
1818
  }
1819
+ /** Telemetry self-description — the two-stage strategy, over the base's store. */
1820
+ describeRetrieval() {
1821
+ return {
1822
+ retriever: "reranking",
1823
+ ...describeSource(this.base)
1824
+ };
1825
+ }
1601
1826
  async retrieve(query, options = {}) {
1602
1827
  const candidates = await this.base.retrieve(query, {
1603
1828
  topK: this.options.fetchTopK ?? 20,
@@ -1622,6 +1847,13 @@ var FilteredRetriever = class {
1622
1847
  this.base = base;
1623
1848
  this.filter = filter;
1624
1849
  }
1850
+ /** Telemetry self-description — this scope, over the base's store. */
1851
+ describeRetrieval() {
1852
+ return {
1853
+ retriever: "filtered",
1854
+ ...describeSource(this.base)
1855
+ };
1856
+ }
1625
1857
  async retrieve(query, options = {}) {
1626
1858
  return this.base.retrieve(query, {
1627
1859
  ...options,
@@ -1671,6 +1903,108 @@ async function ingestDocuments(documents, options) {
1671
1903
  }
1672
1904
  __name(ingestDocuments, "ingestDocuments");
1673
1905
 
1906
+ // src/retrieval-telemetry.ts
1907
+ var import_node_async_hooks = require("async_hooks");
1908
+ var import_nestjs_diagnostics = require("@dudousxd/nestjs-diagnostics");
1909
+ var RAG_DIAGNOSTIC_LIB = "rag";
1910
+ var RAG_RETRIEVAL_EVENT = "retrieval";
1911
+ var RAG_RETRIEVAL_CHANNEL = (0, import_nestjs_diagnostics.channelName)(RAG_DIAGNOSTIC_LIB, RAG_RETRIEVAL_EVENT);
1912
+ function emitRagRetrieval(event, durationMs) {
1913
+ (0, import_nestjs_diagnostics.emit)(RAG_DIAGNOSTIC_LIB, RAG_RETRIEVAL_EVENT, event, {
1914
+ durationMs
1915
+ });
1916
+ }
1917
+ __name(emitRagRetrieval, "emitRagRetrieval");
1918
+ var DEFAULT_TOP_K = 5;
1919
+ var inRetrieval = new import_node_async_hooks.AsyncLocalStorage();
1920
+ function round4(value) {
1921
+ return Math.round(value * 1e4) / 1e4;
1922
+ }
1923
+ __name(round4, "round4");
1924
+ function scoreStats(passages) {
1925
+ const scores = passages.map((passage) => passage.score).filter((score) => Number.isFinite(score));
1926
+ if (scores.length === 0) {
1927
+ return {};
1928
+ }
1929
+ const sum = scores.reduce((total, score) => total + score, 0);
1930
+ return {
1931
+ topScore: round4(Math.max(...scores)),
1932
+ meanScore: round4(sum / scores.length)
1933
+ };
1934
+ }
1935
+ __name(scoreStats, "scoreStats");
1936
+ var InstrumentedRetriever = class InstrumentedRetriever2 {
1937
+ static {
1938
+ __name(this, "InstrumentedRetriever");
1939
+ }
1940
+ base;
1941
+ options;
1942
+ constructor(base, options = {}) {
1943
+ this.base = base;
1944
+ this.options = options;
1945
+ }
1946
+ /** Transparent to detection: wrapping a retriever must not turn it into an `unknown` one. */
1947
+ describeRetrieval() {
1948
+ return {
1949
+ ...describeRetrieval(this.base),
1950
+ ...this.options.describe
1951
+ };
1952
+ }
1953
+ async retrieve(query, options = {}) {
1954
+ if (!(0, import_nestjs_diagnostics.getChannel)(RAG_DIAGNOSTIC_LIB, RAG_RETRIEVAL_EVENT).hasSubscribers) {
1955
+ return this.base.retrieve(query, options);
1956
+ }
1957
+ if (inRetrieval.getStore() === true) {
1958
+ return this.base.retrieve(query, options);
1959
+ }
1960
+ const descriptor = this.describeRetrieval();
1961
+ const topK = options.topK ?? DEFAULT_TOP_K;
1962
+ const startedAt = Date.now();
1963
+ try {
1964
+ const passages = await inRetrieval.run(true, () => this.base.retrieve(query, options));
1965
+ emitRagRetrieval({
1966
+ retriever: descriptor.retriever ?? "unknown",
1967
+ ...descriptor.store !== void 0 ? {
1968
+ store: descriptor.store
1969
+ } : {},
1970
+ ...this.collectionOf(options.filter, descriptor),
1971
+ topK,
1972
+ chunks: passages.length,
1973
+ zeroHit: passages.length === 0,
1974
+ ...scoreStats(passages),
1975
+ failed: false
1976
+ }, Date.now() - startedAt);
1977
+ return passages;
1978
+ } catch (error) {
1979
+ emitRagRetrieval({
1980
+ retriever: descriptor.retriever ?? "unknown",
1981
+ ...descriptor.store !== void 0 ? {
1982
+ store: descriptor.store
1983
+ } : {},
1984
+ ...this.collectionOf(options.filter, descriptor),
1985
+ topK,
1986
+ chunks: 0,
1987
+ zeroHit: true,
1988
+ failed: true,
1989
+ error: error instanceof Error ? error.message : String(error)
1990
+ }, Date.now() - startedAt);
1991
+ throw error;
1992
+ }
1993
+ }
1994
+ /** The host's collection for this call, else the store namespace, else nothing. */
1995
+ collectionOf(filter, descriptor) {
1996
+ const { collection } = this.options;
1997
+ const resolved = typeof collection === "function" ? collection(filter) : collection ?? descriptor.collection;
1998
+ return resolved !== void 0 ? {
1999
+ collection: resolved
2000
+ } : {};
2001
+ }
2002
+ };
2003
+ function instrumentRetriever(retriever, options = {}) {
2004
+ return new InstrumentedRetriever(retriever, options);
2005
+ }
2006
+ __name(instrumentRetriever, "instrumentRetriever");
2007
+
1674
2008
  // src/retrieval-tool.ts
1675
2009
  function isQueryInput(value) {
1676
2010
  return typeof value === "object" && value !== null && "query" in value && typeof value.query === "string";
@@ -1697,6 +2031,8 @@ var querySchema = {
1697
2031
  };
1698
2032
  function createRetrievalTool(retriever, options = {}) {
1699
2033
  const topK = options.topK;
2034
+ const { telemetry = true } = options;
2035
+ const instrumented = telemetry === false ? retriever : instrumentRetriever(retriever, telemetry === true ? {} : telemetry);
1700
2036
  return {
1701
2037
  spec: {
1702
2038
  name: options.name ?? "search_knowledge",
@@ -1709,7 +2045,7 @@ function createRetrievalTool(retriever, options = {}) {
1709
2045
  if (!isQueryInput(input)) {
1710
2046
  throw new Error("retrieval tool expects { query: string }");
1711
2047
  }
1712
- const passages = await retriever.retrieve(input.query, topK !== void 0 ? {
2048
+ const passages = await instrumented.retrieve(input.query, topK !== void 0 ? {
1713
2049
  topK
1714
2050
  } : {});
1715
2051
  return {
@@ -1729,17 +2065,27 @@ __name(createRetrievalTool, "createRetrievalTool");
1729
2065
  LexicalRetriever,
1730
2066
  MemoryVectorStore,
1731
2067
  PgVectorStore,
2068
+ RAG_DIAGNOSTIC_LIB,
2069
+ RAG_RETRIEVAL_CHANNEL,
2070
+ RAG_RETRIEVAL_EVENT,
1732
2071
  RedisVectorSchemaMismatchError,
1733
2072
  RedisVectorStore,
1734
2073
  RerankingRetriever,
1735
2074
  UnsafeRemovalError,
1736
2075
  applyMetadataPatch,
1737
2076
  chunkDocuments,
2077
+ chunkIndexOf,
1738
2078
  chunkText,
1739
2079
  createRetrievalTool,
2080
+ describeRetrieval,
2081
+ describeSharedSource,
2082
+ describeSource,
2083
+ describesRetrieval,
1740
2084
  documentIdOf,
2085
+ emitRagRetrieval,
1741
2086
  ingestChunks,
1742
2087
  ingestDocuments,
2088
+ instrumentRetriever,
1743
2089
  isLexicalVectorStore
1744
2090
  });
1745
2091
  //# sourceMappingURL=index.cjs.map