@drewswiredin/backstage-plugin-assistants-backend-module-semantic-search 0.1.2 → 0.1.3

package/README.md

@@ -90,9 +90,17 @@ sections and delete removed ones. Changing `embeddingModel` rebuilds the index.
 Add `search-knowledge` to the assistant's tools in the assistant editor. It
 pairs with `read-techdocs` (from `builtinActions: true`), which opens the full
 page behind a TechDocs result. Until the first index run finishes, the tool
-answers with an error that points the model at `search-techdocs` and
+answers with an error that points the model at keyword search: Backstage's
+`query` action when the assistant has it, otherwise `search-techdocs` and
 `search-catalog`.
 
+The tool's description carries the search strategy, so agents that see only
+tool descriptions (some MCP clients drop server instructions) still follow it:
+`search-knowledge` first, keyword search when the results don't clearly answer
+or the question is a bare name or acronym, and nothing reported missing until
+both come back empty. The built-in `search-catalog` and `search-techdocs`
+descriptions say the same.
+
 The action registers under the `assistants` plugin, so `assistants` must be in
 `backend.actions.pluginSources`, as it already is for the built-in tools:
 
@@ -114,18 +122,24 @@ the embedding, a generated full-text vector, and a trigram index over its text.
 The table is a rebuildable cache, not migrated data.
 
 **Retrieval.** `search-knowledge` takes the question and up to three alternative
-phrasings and builds one semantic list per phrasing, a keyword list, and a
-fuzzy list:
-
-| List     | Matches                                       | Postgres                                                    |
-| -------- | --------------------------------------------- | ----------------------------------------------------------- |
-| Semantic | Meaning, per phrasing                         | pgvector cosine distance (HNSW index up to 2000 dimensions) |
-| Keyword  | Any query word, ranked by match density       | `tsvector` / `websearch_to_tsquery`                         |
-| Fuzzy    | Near spellings of words absent from the index | `pg_trgm` word similarity                                   |
-
-Reciprocal rank fusion merges the lists. The top candidates' entities are
-fetched from the catalog as the user; results for entities the user cannot
-read are dropped.
+phrasings and builds one semantic list per phrasing, a keyword list, a rare-word
+list, and a fuzzy list:
+
+| List      | Matches                                       | Postgres                                                    |
+| --------- | --------------------------------------------- | ----------------------------------------------------------- |
+| Semantic  | Meaning, per phrasing                         | pgvector cosine distance (HNSW index up to 2000 dimensions) |
+| Keyword   | Any query word, ranked by match density       | `tsvector` / `websearch_to_tsquery`                         |
+| Rare word | Query words found in three chunks or fewer    | `tsvector` / `websearch_to_tsquery`                         |
+| Fuzzy     | Near spellings of words absent from the index | `pg_trgm` word similarity                                   |
+
+Query words include adjacent pairs written as one word, so "tally up" also
+matches "TallyUp". Reciprocal rank fusion merges the lists with one vote per
+method: the semantic lists share one vote, so extra phrasings broaden recall
+without outvoting an exact match. The rare-word list keeps a name or acronym
+mentioned once in a paragraph about something else from being outranked by
+sections that are only near the question in meaning. The top candidates'
+entities are fetched from the catalog as the user; results for entities the
+user cannot read are dropped, and repeated text is shown once.
 
 ## Reference
 

package/dist/action.cjs.js

@@ -6,6 +6,7 @@ var rank = require('./rank.cjs.js');
 const CANDIDATES = 30;
 const FUSED = 40;
 const EXCERPT_CHARS = 1500;
+const RARE_MAX_CHUNKS = 3;
 function registerSearchKnowledgeAction({
   actionsRegistry,
   store,
@@ -19,7 +20,12 @@ function registerSearchKnowledgeAction({
     attributes: { destructive: false, readOnly: true, idempotent: true },
     description: `Search the organization's TechDocs and software catalog by meaning, not only by exact words. Best for questions in the user's own words: it finds a page even when the question is worded differently from it ("fix errored messages" finds "Retry dead-letter messages") and tolerates misspelled names ("Pilon" finds "Pylon"). It returns the most relevant sections, not every match, and cannot filter by entity kind.
 
-Pass the user's question as "query". Pass up to three "alternatives": other phrasings, synonyms, or the internal terms the docs might use.
+How to search the organization's docs and catalog:
+1. Start here, with the user's question.
+2. If the results don't clearly answer, also run a keyword search: the query tool (catalog and TechDocs at once) when it is available, otherwise search-techdocs and search-catalog. Keyword search finds rare names, acronyms and jargon that ranking by meaning can miss. For a bare name, term or acronym, run both searches from the start.
+3. Report something as missing only after both searches come back empty.
+
+Pass the user's question as the "query" parameter. Pass up to three "alternatives": other phrasings, synonyms, or the internal terms the docs might use.
 
 Each result has a title (its heading path), an excerpt, and a location. For TechDocs results, pass the location to read-techdocs to read the full page. For catalog results, the location is the entity ref.`,
     schema: {
@@ -43,21 +49,28 @@ Each result has a title (its heading path), an excerpt, and a location. For Tech
     action: async ({ input, credentials }) => {
       if (!await store.isReady()) {
         throw new Error(
-          "The knowledge index is still being built; use search-techdocs or search-catalog meanwhile."
+          "The knowledge index is still being built; meanwhile use keyword search: the query tool when it is available, otherwise search-techdocs and search-catalog."
         );
       }
       const phrasings = [input.query, ...input.alternatives ?? []];
       const words = rank.queryWords(phrasings);
-      const [{ embeddings }, unknown] = await Promise.all([
+      const joined = rank.joinedWords(phrasings).filter((w) => !words.includes(w));
+      const [{ embeddings }, frequencies] = await Promise.all([
         ai.embedMany({ model: embeddingModel, values: phrasings }),
-        store.unknownWords(words)
+        store.documentFrequencies([...words, ...joined], RARE_MAX_CHUNKS + 1)
       ]);
+      const known = [...words, ...joined].filter(
+        (w) => (frequencies.get(w) ?? 0) > 0
+      );
+      const rare = known.filter((w) => frequencies.get(w) <= RARE_MAX_CHUNKS);
+      const unknown = words.filter((w) => frequencies.get(w) === 0);
       const lists = await Promise.all([
         ...embeddings.map(async (embedding) => ({
           ids: await store.nearest(embedding, CANDIDATES),
-          weight: 1
+          weight: 1 / embeddings.length
         })),
-        store.keyword(words, CANDIDATES).then((ids) => ({ ids, weight: 1 })),
+        store.keyword(known, CANDIDATES).then((ids) => ({ ids, weight: 1 })),
+        store.keyword(rare, CANDIDATES).then((ids) => ({ ids, weight: 1 })),
         store.fuzzy(unknown, CANDIDATES).then((ids) => ({ ids, weight: 1 }))
       ]);
       const ranked = rank.fuseRankings(lists).slice(0, FUSED);
@@ -72,10 +85,11 @@ Each result has a title (its heading path), an excerpt, and a location. For Tech
       const results = [];
       for (const chunk of candidates) {
         const key = `${chunk.location}|${chunk.title}`;
-        if (!readable.has(chunk.entityRef) || seen.has(key)) {
+        if (!readable.has(chunk.entityRef) || seen.has(key) || seen.has(chunk.content)) {
           continue;
         }
         seen.add(key);
+        seen.add(chunk.content);
         results.push({
           source: chunk.source,
           title: chunk.title,

package/dist/action.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"action.cjs.js","sources":["../src/action.ts"],"sourcesContent":["import type {\n  AuthService,\n  BackstageCredentials,\n} from '@backstage/backend-plugin-api';\nimport type { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport type { CatalogApi } from '@backstage/catalog-client';\nimport { embedMany, type EmbeddingModel } from 'ai';\nimport { fuseRankings, queryWords, type RankedList } from './rank';\nimport type { KnowledgeStore, StoredChunk } from './store';\n\n/** Candidates per retrieval list, before fusion and permission filtering. */\nconst CANDIDATES = 30;\n/** Fused candidates checked for permission; over-fetched so filtering rarely empties the page. */\nconst FUSED = 40;\nconst EXCERPT_CHARS = 1500;\n\nexport interface SearchKnowledgeDeps {\n  actionsRegistry: ActionsRegistryService;\n  store: KnowledgeStore;\n  catalog: CatalogApi;\n  auth: AuthService;\n  embeddingModel: EmbeddingModel;\n}\n\n/**\n * Registers `search-knowledge`: hybrid retrieval over the knowledge index. Each\n * phrasing is embedded and matched by meaning; the query words are matched\n * exactly (full-text, any word) and, for words absent from the index, by\n * near spelling (trigram). The lists are fused by reciprocal rank, then\n * filtered to entities the caller can read in the catalog.\n */\nexport function registerSearchKnowledgeAction({\n  actionsRegistry,\n  store,\n  catalog,\n  auth,\n  embeddingModel,\n}: SearchKnowledgeDeps): void {\n  actionsRegistry.register({\n    name: 'search-knowledge',\n    title: 'Search Knowledge',\n    attributes: { destructive: false, readOnly: true, idempotent: true },\n    description: `Search the organization's TechDocs and software catalog by meaning, not only by exact words. Best for questions in the user's own words: it finds a page even when the question is worded differently from it (\"fix errored messages\" finds \"Retry dead-letter messages\") and tolerates misspelled names (\"Pilon\" finds \"Pylon\"). It returns the most relevant sections, not every match, and cannot filter by entity kind.\n\nPass the user's question as \"query\". Pass up to three \"alternatives\": other phrasings, synonyms, or the internal terms the docs might use.\n\nEach result has a title (its heading path), an excerpt, and a location. For TechDocs results, pass the location to read-techdocs to read the full page. For catalog results, the location is the entity ref.`,\n    schema: {\n      input: z =>\n        z.object({\n          query: z.string().min(1).describe('The question or topic to find'),\n          alternatives: z\n            .array(z.string())\n            .max(3)\n            .optional()\n            .describe('Other phrasings or terms for the same thing'),\n          limit: z\n            .number()\n            .int()\n            .min(1)\n            .max(20)\n            .optional()\n            .default(8)\n            .describe('Number of results to return'),\n        }),\n      output: z =>\n        z.object({\n          results: z.array(\n            z.object({\n              source: z.enum(['techdocs', 'catalog']),\n              title: z.string(),\n              location: z.string(),\n              entityRef: z.string(),\n              excerpt: z.string(),\n            }),\n          ),\n        }),\n    },\n    action: async ({ input, credentials }) => {\n      if (!(await store.isReady())) {\n        throw new Error(\n          'The knowledge index is still being built; use search-techdocs or search-catalog meanwhile.',\n        );\n      }\n\n      const phrasings = [input.query, ...(input.alternatives ?? [])];\n      const words = queryWords(phrasings);\n      const [{ embeddings }, unknown] = await Promise.all([\n        embedMany({ model: embeddingModel, values: phrasings }),\n        store.unknownWords(words),\n      ]);\n\n      const lists: RankedList[] = await Promise.all([\n        ...embeddings.map(async embedding => ({\n          ids: await store.nearest(embedding, CANDIDATES),\n          weight: 1,\n        })),\n        store.keyword(words, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n        store.fuzzy(unknown, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n      ]);\n\n      const ranked = fuseRankings(lists).slice(0, FUSED);\n      const byId = new Map(\n        (await store.chunks(ranked)).map(chunk => [chunk.id, chunk]),\n      );\n      const candidates = ranked\n        .map(id => byId.get(id))\n        .filter((chunk): chunk is StoredChunk => Boolean(chunk));\n\n      const readable = await readableEntityRefs(catalog, auth, credentials, [\n        ...new Set(candidates.map(chunk => chunk.entityRef)),\n      ]);\n\n      const seen = new Set<string>();\n      const results = [];\n      for (const chunk of candidates) {\n        const key = `${chunk.location}|${chunk.title}`;\n        if (!readable.has(chunk.entityRef) || seen.has(key)) {\n          continue;\n        }\n        seen.add(key);\n        results.push({\n          source: chunk.source,\n          title: chunk.title,\n          location: chunk.location,\n          entityRef: chunk.entityRef,\n          excerpt: chunk.content.slice(0, EXCERPT_CHARS),\n        });\n        if (results.length === input.limit) {\n          break;\n        }\n      }\n      return { output: { results } };\n    },\n  });\n}\n\n/**\n * The subset of entity refs the caller can read, asked of the catalog as the\n * caller — so the catalog's own permission policy decides, and a deleted\n * entity's leftover chunks never surface.\n */\nasync function readableEntityRefs(\n  catalog: CatalogApi,\n  auth: AuthService,\n  credentials: BackstageCredentials,\n  entityRefs: string[],\n): Promise<Set<string>> {\n  if (entityRefs.length === 0) {\n    return new Set();\n  }\n  const { token } = await auth.getPluginRequestToken({\n    onBehalfOf: credentials,\n    targetPluginId: 'catalog',\n  });\n  const { items } = await catalog.getEntitiesByRefs(\n    { entityRefs, fields: ['kind', 'metadata.namespace', 'metadata.name'] },\n    { token },\n  );\n  return new Set(entityRefs.filter((_, i) => items[i]));\n}\n"],"names":["queryWords","embedMany","fuseRankings"],"mappings":";;;;;AAWA,MAAM,UAAA,GAAa,EAAA;AAEnB,MAAM,KAAA,GAAQ,EAAA;AACd,MAAM,aAAA,GAAgB,IAAA;AAiBf,SAAS,6BAAA,CAA8B;AAAA,EAC5C,eAAA;AAAA,EACA,KAAA;AAAA,EACA,OAAA;AAAA,EACA,IAAA;AAAA,EACA;AACF,CAAA,EAA8B;AAC5B,EAAA,eAAA,CAAgB,QAAA,CAAS;AAAA,IACvB,IAAA,EAAM,kBAAA;AAAA,IACN,KAAA,EAAO,kBAAA;AAAA,IACP,YAAY,EAAE,WAAA,EAAa,OAAO,QAAA,EAAU,IAAA,EAAM,YAAY,IAAA,EAAK;AAAA,IACnE,WAAA,EAAa,CAAA;;AAAA;;AAAA,4MAAA,CAAA;AAAA,IAKb,MAAA,EAAQ;AAAA,MACN,KAAA,EAAO,CAAA,CAAA,KACL,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,KAAA,EAAO,EAAE,MAAA,EAAO,CAAE,IAAI,CAAC,CAAA,CAAE,SAAS,+BAA+B,CAAA;AAAA,QACjE,YAAA,EAAc,CAAA,CACX,KAAA,CAAM,CAAA,CAAE,MAAA,EAAQ,CAAA,CAChB,GAAA,CAAI,CAAC,CAAA,CACL,QAAA,EAAS,CACT,SAAS,6CAA6C,CAAA;AAAA,QACzD,OAAO,CAAA,CACJ,MAAA,GACA,GAAA,EAAI,CACJ,IAAI,CAAC,CAAA,CACL,GAAA,CAAI,EAAE,EACN,QAAA,EAAS,CACT,QAAQ,CAAC,CAAA,CACT,SAAS,6BAA6B;AAAA,OAC1C,CAAA;AAAA,MACH,MAAA,EAAQ,CAAA,CAAA,KACN,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,SAAS,CAAA,CAAE,KAAA;AAAA,UACT,EAAE,MAAA,CAAO;AAAA,YACP,QAAQ,CAAA,CAAE,IAAA,CAAK,CAAC,UAAA,EAAY,SAAS,CAAC,CAAA;AAAA,YACtC,KAAA,EAAO,EAAE,MAAA,EAAO;AAAA,YAChB,QAAA,EAAU,EAAE,MAAA,EAAO;AAAA,YACnB,SAAA,EAAW,EAAE,MAAA,EAAO;AAAA,YACpB,OAAA,EAAS,EAAE,MAAA;AAAO,WACnB;AAAA;AACH,OACD;AAAA,KACL;AAAA,IACA,MAAA,EAAQ,OAAO,EAAE,KAAA,EAAO,aAAY,KAAM;AACxC,MAAA,IAAI,CAAE,MAAM,KAAA,CAAM,OAAA,EAAQ,EAAI;AAC5B,QAAA,MAAM,IAAI,KAAA;AAAA,UACR;AAAA,SACF;AAAA,MACF;AAEA,MAAA,MAAM,SAAA,GAAY,CAAC,KAAA,CAAM,KAAA,EAAO,GAAI,KAAA,CAAM,YAAA,IAAgB,EAAG,CAAA;AAC7D,MAAA,MAAM,KAAA,GAAQA,gBAAW,SAAS,CAAA;AAClC,MAAA,MAAM,CAAC,EAAE,UAAA,EAAW,EAAG,OAAO,CAAA,GAAI,MAAM,QAAQ,GAAA,CAAI;AAAA,QAClDC,aAAU,EAAE,KAAA,EAAO,cAAA,EAAgB,MAAA,EAAQ,WAAW,CAAA;AAAA,QACtD,KAAA,CAAM,aAAa,KAAK;AAAA,OACzB,CAAA;AAED,MAAA,MAAM,KAAA,GAAsB,MAAM,OAAA,CAAQ,GAAA,CAAI;AAAA,QAC5C,GAAG,UAAA,CAAW,GAAA,CAAI,OAAM,SAAA,MAAc;AAAA,UACpC,GAAA,EAAK,MAAM,KAAA,CAAM,OAAA,CAAQ,WAAW,UAAU,CAAA;AAAA,UAC9C,MAAA,EAAQ;AAAA,SACV,CAAE,CAAA;AAAA,QACF,KAAA,CAAM,OAAA,CAAQ,KAAA,EAAO,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE,CAAA;AAAA,QACjE,KAAA,CAAM,KAAA,CAAM,OAAA,EAAS,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE;AAAA,OAClE,CAAA;AAED,MAAA,MAAM,SAASC,iBAAA,CAAa,KAAK,CAAA,CAAE,KAAA,CAAM,GAAG,KAAK,CAAA;AACjD,MAAA,MAAM,OAAO,IAAI,GAAA;AAAA,QAAA,CACd,MAAM,KAAA,CAAM,MAAA,CAAO,MAAM,CAAA,EAAG,GAAA,CAAI,CAAA,KAAA,KAAS,CAAC,KAAA,CAAM,EAAA,EAAI,KAAK,CAAC;AAAA,OAC7D;AACA,MAAA,MAAM,UAAA,GAAa,MAAA,CAChB,GAAA,CAAI,CAAA,EAAA,KAAM,KAAK,GAAA,CAAI,EAAE,CAAC,CAAA,CACtB,MAAA,CAAO,CAAC,KAAA,KAAgC,OAAA,CAAQ,KAAK,CAAC,CAAA;AAEzD,MAAA,MAAM,QAAA,GAAW,MAAM,kBAAA,CAAmB,OAAA,EAAS,MAAM,WAAA,EAAa;AAAA,QACpE,GAAG,IAAI,GAAA,CAAI,UAAA,CAAW,IAAI,CAAA,KAAA,KAAS,KAAA,CAAM,SAAS,CAAC;AAAA,OACpD,CAAA;AAED,MAAA,MAAM,IAAA,uBAAW,GAAA,EAAY;AAC7B,MAAA,MAAM,UAAU,EAAC;AACjB,MAAA,KAAA,MAAW,SAAS,UAAA,EAAY;AAC9B,QAAA,MAAM,MAAM,CAAA,EAAG,KAAA,CAAM,QAAQ,CAAA,CAAA,EAAI,MAAM,KAAK,CAAA,CAAA;AAC5C,QAAA,IAAI,CAAC,SAAS,GAAA,CAAI,KAAA,CAAM,SAAS,CAAA,IAAK,IAAA,CAAK,GAAA,CAAI,GAAG,CAAA,EAAG;AACnD,UAAA;AAAA,QACF;AACA,QAAA,IAAA,CAAK,IAAI,GAAG,CAAA;AACZ,QAAA,OAAA,CAAQ,IAAA,CAAK;AAAA,UACX,QAAQ,KAAA,CAAM,MAAA;AAAA,UACd,OAAO,KAAA,CAAM,KAAA;AAAA,UACb,UAAU,KAAA,CAAM,QAAA;AAAA,UAChB,WAAW,KAAA,CAAM,SAAA;AAAA,UACjB,OAAA,EAAS,KAAA,CAAM,OAAA,CAAQ,KAAA,CAAM,GAAG,aAAa;AAAA,SAC9C,CAAA;AACD,QAAA,IAAI,OAAA,CAAQ,MAAA,KAAW,KAAA,CAAM,KAAA,EAAO;AAClC,UAAA;AAAA,QACF;AAAA,MACF;AACA,MAAA,OAAO,EAAE,MAAA,EAAQ,EAAE,OAAA,EAAQ,EAAE;AAAA,IAC/B;AAAA,GACD,CAAA;AACH;AAOA,eAAe,kBAAA,CACb,OAAA,EACA,IAAA,EACA,WAAA,EACA,UAAA,EACsB;AACtB,EAAA,IAAI,UAAA,CAAW,WAAW,CAAA,EAAG;AAC3B,IAAA,2BAAW,GAAA,EAAI;AAAA,EACjB;AACA,EAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,KAAK,qBAAA,CAAsB;AAAA,IACjD,UAAA,EAAY,WAAA;AAAA,IACZ,cAAA,EAAgB;AAAA,GACjB,CAAA;AACD,EAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,OAAA,CAAQ,iBAAA;AAAA,IAC9B,EAAE,UAAA,EAAY,MAAA,EAAQ,CAAC,MAAA,EAAQ,oBAAA,EAAsB,eAAe,CAAA,EAAE;AAAA,IACtE,EAAE,KAAA;AAAM,GACV;AACA,EAAA,OAAO,IAAI,GAAA,CAAI,UAAA,CAAW,MAAA,CAAO,CAAC,GAAG,CAAA,KAAM,KAAA,CAAM,CAAC,CAAC,CAAC,CAAA;AACtD;;;;"}
+{"version":3,"file":"action.cjs.js","sources":["../src/action.ts"],"sourcesContent":["import type {\n  AuthService,\n  BackstageCredentials,\n} from '@backstage/backend-plugin-api';\nimport type { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport type { CatalogApi } from '@backstage/catalog-client';\nimport { embedMany, type EmbeddingModel } from 'ai';\nimport { fuseRankings, joinedWords, queryWords, type RankedList } from './rank';\nimport type { KnowledgeStore, StoredChunk } from './store';\n\n/** Candidates per retrieval list, before fusion and permission filtering. */\nconst CANDIDATES = 30;\n/** Fused candidates checked for permission; over-fetched so filtering rarely empties the page. */\nconst FUSED = 40;\nconst EXCERPT_CHARS = 1500;\n/** A word in this many chunks or fewer is rare: strong evidence of what the user means. */\nconst RARE_MAX_CHUNKS = 3;\n\nexport interface SearchKnowledgeDeps {\n  actionsRegistry: ActionsRegistryService;\n  store: KnowledgeStore;\n  catalog: CatalogApi;\n  auth: AuthService;\n  embeddingModel: EmbeddingModel;\n}\n\n/**\n * Registers `search-knowledge`: hybrid retrieval over the knowledge index. Each\n * phrasing is embedded and matched by meaning; the query words are matched\n * exactly (full-text, any word) and, for words absent from the index, by\n * near spelling (trigram). Words found in only a few chunks get a list of\n * their own, so a rare name or acronym buried in a paragraph about something\n * else is not outvoted by sections that are merely near the question.\n *\n * The lists are fused by reciprocal rank with one vote per method: the\n * phrasings share the meaning vote, so extra alternatives broaden recall\n * without drowning out exact matches. Results are filtered to entities the\n * caller can read in the catalog, and repeated text is shown once.\n */\nexport function registerSearchKnowledgeAction({\n  actionsRegistry,\n  store,\n  catalog,\n  auth,\n  embeddingModel,\n}: SearchKnowledgeDeps): void {\n  actionsRegistry.register({\n    name: 'search-knowledge',\n    title: 'Search Knowledge',\n    attributes: { destructive: false, readOnly: true, idempotent: true },\n    description: `Search the organization's TechDocs and software catalog by meaning, not only by exact words. Best for questions in the user's own words: it finds a page even when the question is worded differently from it (\"fix errored messages\" finds \"Retry dead-letter messages\") and tolerates misspelled names (\"Pilon\" finds \"Pylon\"). It returns the most relevant sections, not every match, and cannot filter by entity kind.\n\nHow to search the organization's docs and catalog:\n1. Start here, with the user's question.\n2. If the results don't clearly answer, also run a keyword search: the query tool (catalog and TechDocs at once) when it is available, otherwise search-techdocs and search-catalog. Keyword search finds rare names, acronyms and jargon that ranking by meaning can miss. For a bare name, term or acronym, run both searches from the start.\n3. Report something as missing only after both searches come back empty.\n\nPass the user's question as the \"query\" parameter. Pass up to three \"alternatives\": other phrasings, synonyms, or the internal terms the docs might use.\n\nEach result has a title (its heading path), an excerpt, and a location. For TechDocs results, pass the location to read-techdocs to read the full page. For catalog results, the location is the entity ref.`,\n    schema: {\n      input: z =>\n        z.object({\n          query: z.string().min(1).describe('The question or topic to find'),\n          alternatives: z\n            .array(z.string())\n            .max(3)\n            .optional()\n            .describe('Other phrasings or terms for the same thing'),\n          limit: z\n            .number()\n            .int()\n            .min(1)\n            .max(20)\n            .optional()\n            .default(8)\n            .describe('Number of results to return'),\n        }),\n      output: z =>\n        z.object({\n          results: z.array(\n            z.object({\n              source: z.enum(['techdocs', 'catalog']),\n              title: z.string(),\n              location: z.string(),\n              entityRef: z.string(),\n              excerpt: z.string(),\n            }),\n          ),\n        }),\n    },\n    action: async ({ input, credentials }) => {\n      if (!(await store.isReady())) {\n        throw new Error(\n          'The knowledge index is still being built; meanwhile use keyword search: the query tool when it is available, otherwise search-techdocs and search-catalog.',\n        );\n      }\n\n      const phrasings = [input.query, ...(input.alternatives ?? [])];\n      const words = queryWords(phrasings);\n      const joined = joinedWords(phrasings).filter(w => !words.includes(w));\n      const [{ embeddings }, frequencies] = await Promise.all([\n        embedMany({ model: embeddingModel, values: phrasings }),\n        store.documentFrequencies([...words, ...joined], RARE_MAX_CHUNKS + 1),\n      ]);\n      const known = [...words, ...joined].filter(\n        w => (frequencies.get(w) ?? 0) > 0,\n      );\n      const rare = known.filter(w => frequencies.get(w)! <= RARE_MAX_CHUNKS);\n      const unknown = words.filter(w => frequencies.get(w) === 0);\n\n      const lists: RankedList[] = await Promise.all([\n        ...embeddings.map(async embedding => ({\n          ids: await store.nearest(embedding, CANDIDATES),\n          weight: 1 / embeddings.length,\n        })),\n        store.keyword(known, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n        store.keyword(rare, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n        store.fuzzy(unknown, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n      ]);\n\n      const ranked = fuseRankings(lists).slice(0, FUSED);\n      const byId = new Map(\n        (await store.chunks(ranked)).map(chunk => [chunk.id, chunk]),\n      );\n      const candidates = ranked\n        .map(id => byId.get(id))\n        .filter((chunk): chunk is StoredChunk => Boolean(chunk));\n\n      const readable = await readableEntityRefs(catalog, auth, credentials, [\n        ...new Set(candidates.map(chunk => chunk.entityRef)),\n      ]);\n\n      const seen = new Set<string>();\n      const results = [];\n      for (const chunk of candidates) {\n        const key = `${chunk.location}|${chunk.title}`;\n        if (\n          !readable.has(chunk.entityRef) ||\n          seen.has(key) ||\n          seen.has(chunk.content)\n        ) {\n          continue;\n        }\n        seen.add(key);\n        seen.add(chunk.content);\n        results.push({\n          source: chunk.source,\n          title: chunk.title,\n          location: chunk.location,\n          entityRef: chunk.entityRef,\n          excerpt: chunk.content.slice(0, EXCERPT_CHARS),\n        });\n        if (results.length === input.limit) {\n          break;\n        }\n      }\n      return { output: { results } };\n    },\n  });\n}\n\n/**\n * The subset of entity refs the caller can read, asked of the catalog as the\n * caller — so the catalog's own permission policy decides, and a deleted\n * entity's leftover chunks never surface.\n */\nasync function readableEntityRefs(\n  catalog: CatalogApi,\n  auth: AuthService,\n  credentials: BackstageCredentials,\n  entityRefs: string[],\n): Promise<Set<string>> {\n  if (entityRefs.length === 0) {\n    return new Set();\n  }\n  const { token } = await auth.getPluginRequestToken({\n    onBehalfOf: credentials,\n    targetPluginId: 'catalog',\n  });\n  const { items } = await catalog.getEntitiesByRefs(\n    { entityRefs, fields: ['kind', 'metadata.namespace', 'metadata.name'] },\n    { token },\n  );\n  return new Set(entityRefs.filter((_, i) => items[i]));\n}\n"],"names":["queryWords","joinedWords","embedMany","fuseRankings"],"mappings":";;;;;AAWA,MAAM,UAAA,GAAa,EAAA;AAEnB,MAAM,KAAA,GAAQ,EAAA;AACd,MAAM,aAAA,GAAgB,IAAA;AAEtB,MAAM,eAAA,GAAkB,CAAA;AAuBjB,SAAS,6BAAA,CAA8B;AAAA,EAC5C,eAAA;AAAA,EACA,KAAA;AAAA,EACA,OAAA;AAAA,EACA,IAAA;AAAA,EACA;AACF,CAAA,EAA8B;AAC5B,EAAA,eAAA,CAAgB,QAAA,CAAS;AAAA,IACvB,IAAA,EAAM,kBAAA;AAAA,IACN,KAAA,EAAO,kBAAA;AAAA,IACP,YAAY,EAAE,WAAA,EAAa,OAAO,QAAA,EAAU,IAAA,EAAM,YAAY,IAAA,EAAK;AAAA,IACnE,WAAA,EAAa,CAAA;;AAAA;AAAA;AAAA;AAAA;;AAAA;;AAAA,4MAAA,CAAA;AAAA,IAUb,MAAA,EAAQ;AAAA,MACN,KAAA,EAAO,CAAA,CAAA,KACL,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,KAAA,EAAO,EAAE,MAAA,EAAO,CAAE,IAAI,CAAC,CAAA,CAAE,SAAS,+BAA+B,CAAA;AAAA,QACjE,YAAA,EAAc,CAAA,CACX,KAAA,CAAM,CAAA,CAAE,MAAA,EAAQ,CAAA,CAChB,GAAA,CAAI,CAAC,CAAA,CACL,QAAA,EAAS,CACT,SAAS,6CAA6C,CAAA;AAAA,QACzD,OAAO,CAAA,CACJ,MAAA,GACA,GAAA,EAAI,CACJ,IAAI,CAAC,CAAA,CACL,GAAA,CAAI,EAAE,EACN,QAAA,EAAS,CACT,QAAQ,CAAC,CAAA,CACT,SAAS,6BAA6B;AAAA,OAC1C,CAAA;AAAA,MACH,MAAA,EAAQ,CAAA,CAAA,KACN,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,SAAS,CAAA,CAAE,KAAA;AAAA,UACT,EAAE,MAAA,CAAO;AAAA,YACP,QAAQ,CAAA,CAAE,IAAA,CAAK,CAAC,UAAA,EAAY,SAAS,CAAC,CAAA;AAAA,YACtC,KAAA,EAAO,EAAE,MAAA,EAAO;AAAA,YAChB,QAAA,EAAU,EAAE,MAAA,EAAO;AAAA,YACnB,SAAA,EAAW,EAAE,MAAA,EAAO;AAAA,YACpB,OAAA,EAAS,EAAE,MAAA;AAAO,WACnB;AAAA;AACH,OACD;AAAA,KACL;AAAA,IACA,MAAA,EAAQ,OAAO,EAAE,KAAA,EAAO,aAAY,KAAM;AACxC,MAAA,IAAI,CAAE,MAAM,KAAA,CAAM,OAAA,EAAQ,EAAI;AAC5B,QAAA,MAAM,IAAI,KAAA;AAAA,UACR;AAAA,SACF;AAAA,MACF;AAEA,MAAA,MAAM,SAAA,GAAY,CAAC,KAAA,CAAM,KAAA,EAAO,GAAI,KAAA,CAAM,YAAA,IAAgB,EAAG,CAAA;AAC7D,MAAA,MAAM,KAAA,GAAQA,gBAAW,SAAS,CAAA;AAClC,MAAA,MAAM,MAAA,GAASC,gBAAA,CAAY,SAAS,CAAA,CAAE,MAAA,CAAO,OAAK,CAAC,KAAA,CAAM,QAAA,CAAS,CAAC,CAAC,CAAA;AACpE,MAAA,MAAM,CAAC,EAAE,UAAA,EAAW,EAAG,WAAW,CAAA,GAAI,MAAM,QAAQ,GAAA,CAAI;AAAA,QACtDC,aAAU,EAAE,KAAA,EAAO,cAAA,EAAgB,MAAA,EAAQ,WAAW,CAAA;AAAA,QACtD,KAAA,CAAM,oBAAoB,CAAC,GAAG,OAAO,GAAG,MAAM,CAAA,EAAG,eAAA,GAAkB,CAAC;AAAA,OACrE,CAAA;AACD,MAAA,MAAM,QAAQ,CAAC,GAAG,KAAA,EAAO,GAAG,MAAM,CAAA,CAAE,MAAA;AAAA,QAClC,CAAA,CAAA,KAAA,CAAM,WAAA,CAAY,GAAA,CAAI,CAAC,KAAK,CAAA,IAAK;AAAA,OACnC;AACA,MAAA,MAAM,IAAA,GAAO,MAAM,MAAA,CAAO,CAAA,CAAA,KAAK,YAAY,GAAA,CAAI,CAAC,KAAM,eAAe,CAAA;AACrE,MAAA,MAAM,OAAA,GAAU,MAAM,MAAA,CAAO,CAAA,CAAA,KAAK,YAAY,GAAA,CAAI,CAAC,MAAM,CAAC,CAAA;AAE1D,MAAA,MAAM,KAAA,GAAsB,MAAM,OAAA,CAAQ,GAAA,CAAI;AAAA,QAC5C,GAAG,UAAA,CAAW,GAAA,CAAI,OAAM,SAAA,MAAc;AAAA,UACpC,GAAA,EAAK,MAAM,KAAA,CAAM,OAAA,CAAQ,WAAW,UAAU,CAAA;AAAA,UAC9C,MAAA,EAAQ,IAAI,UAAA,CAAW;AAAA,SACzB,CAAE,CAAA;AAAA,QACF,KAAA,CAAM,OAAA,CAAQ,KAAA,EAAO,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE,CAAA;AAAA,QACjE,KAAA,CAAM,OAAA,CAAQ,IAAA,EAAM,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE,CAAA;AAAA,QAChE,KAAA,CAAM,KAAA,CAAM,OAAA,EAAS,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE;AAAA,OAClE,CAAA;AAED,MAAA,MAAM,SAASC,iBAAA,CAAa,KAAK,CAAA,CAAE,KAAA,CAAM,GAAG,KAAK,CAAA;AACjD,MAAA,MAAM,OAAO,IAAI,GAAA;AAAA,QAAA,CACd,MAAM,KAAA,CAAM,MAAA,CAAO,MAAM,CAAA,EAAG,GAAA,CAAI,CAAA,KAAA,KAAS,CAAC,KAAA,CAAM,EAAA,EAAI,KAAK,CAAC;AAAA,OAC7D;AACA,MAAA,MAAM,UAAA,GAAa,MAAA,CAChB,GAAA,CAAI,CAAA,EAAA,KAAM,KAAK,GAAA,CAAI,EAAE,CAAC,CAAA,CACtB,MAAA,CAAO,CAAC,KAAA,KAAgC,OAAA,CAAQ,KAAK,CAAC,CAAA;AAEzD,MAAA,MAAM,QAAA,GAAW,MAAM,kBAAA,CAAmB,OAAA,EAAS,MAAM,WAAA,EAAa;AAAA,QACpE,GAAG,IAAI,GAAA,CAAI,UAAA,CAAW,IAAI,CAAA,KAAA,KAAS,KAAA,CAAM,SAAS,CAAC;AAAA,OACpD,CAAA;AAED,MAAA,MAAM,IAAA,uBAAW,GAAA,EAAY;AAC7B,MAAA,MAAM,UAAU,EAAC;AACjB,MAAA,KAAA,MAAW,SAAS,UAAA,EAAY;AAC9B,QAAA,MAAM,MAAM,CAAA,EAAG,KAAA,CAAM,QAAQ,CAAA,CAAA,EAAI,MAAM,KAAK,CAAA,CAAA;AAC5C,QAAA,IACE,CAAC,QAAA,CAAS,GAAA,CAAI,KAAA,CAAM,SAAS,CAAA,IAC7B,IAAA,CAAK,GAAA,CAAI,GAAG,CAAA,IACZ,IAAA,CAAK,GAAA,CAAI,KAAA,CAAM,OAAO,CAAA,EACtB;AACA,UAAA;AAAA,QACF;AACA,QAAA,IAAA,CAAK,IAAI,GAAG,CAAA;AACZ,QAAA,IAAA,CAAK,GAAA,CAAI,MAAM,OAAO,CAAA;AACtB,QAAA,OAAA,CAAQ,IAAA,CAAK;AAAA,UACX,QAAQ,KAAA,CAAM,MAAA;AAAA,UACd,OAAO,KAAA,CAAM,KAAA;AAAA,UACb,UAAU,KAAA,CAAM,QAAA;AAAA,UAChB,WAAW,KAAA,CAAM,SAAA;AAAA,UACjB,OAAA,EAAS,KAAA,CAAM,OAAA,CAAQ,KAAA,CAAM,GAAG,aAAa;AAAA,SAC9C,CAAA;AACD,QAAA,IAAI,OAAA,CAAQ,MAAA,KAAW,KAAA,CAAM,KAAA,EAAO;AAClC,UAAA;AAAA,QACF;AAAA,MACF;AACA,MAAA,OAAO,EAAE,MAAA,EAAQ,EAAE,OAAA,EAAQ,EAAE;AAAA,IAC/B;AAAA,GACD,CAAA;AACH;AAOA,eAAe,kBAAA,CACb,OAAA,EACA,IAAA,EACA,WAAA,EACA,UAAA,EACsB;AACtB,EAAA,IAAI,UAAA,CAAW,WAAW,CAAA,EAAG;AAC3B,IAAA,2BAAW,GAAA,EAAI;AAAA,EACjB;AACA,EAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,KAAK,qBAAA,CAAsB;AAAA,IACjD,UAAA,EAAY,WAAA;AAAA,IACZ,cAAA,EAAgB;AAAA,GACjB,CAAA;AACD,EAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,OAAA,CAAQ,iBAAA;AAAA,IAC9B,EAAE,UAAA,EAAY,MAAA,EAAQ,CAAC,MAAA,EAAQ,oBAAA,EAAsB,eAAe,CAAA,EAAE;AAAA,IACtE,EAAE,KAAA;AAAM,GACV;AACA,EAAA,OAAO,IAAI,GAAA,CAAI,UAAA,CAAW,MAAA,CAAO,CAAC,GAAG,CAAA,KAAM,KAAA,CAAM,CAAC,CAAC,CAAC,CAAA;AACtD;;;;"}

package/dist/rank.cjs.js

@@ -4,11 +4,13 @@ const STOPWORDS = new Set(
   "a about an and are as at be but by can do does for from how i if in is it its me my of on or our should so that the their them then there these this to us was we what when where which who why will with you your".split(" ")
 );
 const MAX_WORDS = 8;
+function tokens(phrasing) {
+  return phrasing.toLowerCase().split(/[^\p{L}\p{N}_-]+/u).map((raw) => raw.replace(/^[-_]+|[-_]+$/g, "")).filter(Boolean);
+}
 function queryWords(phrasings) {
   const words = /* @__PURE__ */ new Set();
   for (const phrasing of phrasings) {
-    for (const raw of phrasing.toLowerCase().split(/[^\p{L}\p{N}_-]+/u)) {
-      const word = raw.replace(/^[-_]+|[-_]+$/g, "");
+    for (const word of tokens(phrasing)) {
       if (word.length >= 3 && !STOPWORDS.has(word)) {
         words.add(word);
       }
@@ -16,6 +18,16 @@ function queryWords(phrasings) {
   }
   return [...words].slice(0, MAX_WORDS);
 }
+function joinedWords(phrasings) {
+  const words = /* @__PURE__ */ new Set();
+  for (const phrasing of phrasings) {
+    const parts = tokens(phrasing);
+    for (let i = 1; i < parts.length; i++) {
+      words.add(parts[i - 1] + parts[i]);
+    }
+  }
+  return [...words].slice(0, MAX_WORDS);
+}
 const RRF_K = 60;
 function fuseRankings(lists) {
   const scores = /* @__PURE__ */ new Map();
@@ -28,5 +40,6 @@ function fuseRankings(lists) {
 }
 
 exports.fuseRankings = fuseRankings;
+exports.joinedWords = joinedWords;
 exports.queryWords = queryWords;
 //# sourceMappingURL=rank.cjs.js.map

package/dist/rank.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"rank.cjs.js","sources":["../src/rank.ts"],"sourcesContent":["const STOPWORDS = new Set(\n  (\n    'a about an and are as at be but by can do does for from how i if in is it ' +\n    'its me my of on or our should so that the their them then there these ' +\n    'this to us was we what when where which who why will with you your'\n  ).split(' '),\n);\n\nconst MAX_WORDS = 8;\n\n/**\n * The distinct content words of the given phrasings: lowercased, at least\n * three characters, stopwords dropped, capped at eight. These drive the\n * keyword and fuzzy lists; the full phrasings drive the vector list.\n */\nexport function queryWords(phrasings: string[]): string[] {\n  const words = new Set<string>();\n  for (const phrasing of phrasings) {\n    for (const raw of phrasing.toLowerCase().split(/[^\\p{L}\\p{N}_-]+/u)) {\n      const word = raw.replace(/^[-_]+|[-_]+$/g, '');\n      if (word.length >= 3 && !STOPWORDS.has(word)) {\n        words.add(word);\n      }\n    }\n  }\n  return [...words].slice(0, MAX_WORDS);\n}\n\n/** A ranked list of chunk ids from one retrieval method, with its fusion weight. */\nexport interface RankedList {\n  ids: string[];\n  weight: number;\n}\n\nconst RRF_K = 60;\n\n/**\n * Reciprocal rank fusion: each list contributes `weight / (60 + rank)` per id.\n * Rewards ids several methods agree on without comparing their raw scores,\n * which live on different scales.\n */\nexport function fuseRankings(lists: RankedList[]): string[] {\n  const scores = new Map<string, number>();\n  for (const { ids, weight } of lists) {\n    ids.forEach((id, index) => {\n      scores.set(id, (scores.get(id) ?? 0) + weight / (RRF_K + index + 1));\n    });\n  }\n  return [...scores.entries()].sort((a, b) => b[1] - a[1]).map(([id]) => id);\n}\n"],"names":[],"mappings":";;AAAA,MAAM,YAAY,IAAI,GAAA;AAAA,EAElB,oNAAA,CAGA,MAAM,GAAG;AACb,CAAA;AAEA,MAAM,SAAA,GAAY,CAAA;AAOX,SAAS,WAAW,SAAA,EAA+B;AACxD,EAAA,MAAM,KAAA,uBAAY,GAAA,EAAY;AAC9B,EAAA,KAAA,MAAW,YAAY,SAAA,EAAW;AAChC,IAAA,KAAA,MAAW,OAAO,QAAA,CAAS,WAAA,EAAY,CAAE,KAAA,CAAM,mBAAmB,CAAA,EAAG;AACnE,MAAA,MAAM,IAAA,GAAO,GAAA,CAAI,OAAA,CAAQ,gBAAA,EAAkB,EAAE,CAAA;AAC7C,MAAA,IAAI,KAAK,MAAA,IAAU,CAAA,IAAK,CAAC,SAAA,CAAU,GAAA,CAAI,IAAI,CAAA,EAAG;AAC5C,QAAA,KAAA,CAAM,IAAI,IAAI,CAAA;AAAA,MAChB;AAAA,IACF;AAAA,EACF;AACA,EAAA,OAAO,CAAC,GAAG,KAAK,CAAA,CAAE,KAAA,CAAM,GAAG,SAAS,CAAA;AACtC;AAQA,MAAM,KAAA,GAAQ,EAAA;AAOP,SAAS,aAAa,KAAA,EAA+B;AAC1D,EAAA,MAAM,MAAA,uBAAa,GAAA,EAAoB;AACvC,EAAA,KAAA,MAAW,EAAE,GAAA,EAAK,MAAA,EAAO,IAAK,KAAA,EAAO;AACnC,IAAA,GAAA,CAAI,OAAA,CAAQ,CAAC,EAAA,EAAI,KAAA,KAAU;AACzB,MAAA,MAAA,CAAO,GAAA,CAAI,EAAA,EAAA,CAAK,MAAA,CAAO,GAAA,CAAI,EAAE,KAAK,CAAA,IAAK,MAAA,IAAU,KAAA,GAAQ,KAAA,GAAQ,CAAA,CAAE,CAAA;AAAA,IACrE,CAAC,CAAA;AAAA,EACH;AACA,EAAA,OAAO,CAAC,GAAG,MAAA,CAAO,OAAA,EAAS,CAAA,CAAE,IAAA,CAAK,CAAC,CAAA,EAAG,CAAA,KAAM,CAAA,CAAE,CAAC,CAAA,GAAI,CAAA,CAAE,CAAC,CAAC,CAAA,CAAE,IAAI,CAAC,CAAC,EAAE,CAAA,KAAM,EAAE,CAAA;AAC3E;;;;;"}
+{"version":3,"file":"rank.cjs.js","sources":["../src/rank.ts"],"sourcesContent":["const STOPWORDS = new Set(\n  (\n    'a about an and are as at be but by can do does for from how i if in is it ' +\n    'its me my of on or our should so that the their them then there these ' +\n    'this to us was we what when where which who why will with you your'\n  ).split(' '),\n);\n\nconst MAX_WORDS = 8;\n\nfunction tokens(phrasing: string): string[] {\n  return phrasing\n    .toLowerCase()\n    .split(/[^\\p{L}\\p{N}_-]+/u)\n    .map(raw => raw.replace(/^[-_]+|[-_]+$/g, ''))\n    .filter(Boolean);\n}\n\n/**\n * The distinct content words of the given phrasings: lowercased, at least\n * three characters, stopwords dropped, capped at eight. These drive the\n * keyword and fuzzy lists; the full phrasings drive the vector lists.\n */\nexport function queryWords(phrasings: string[]): string[] {\n  const words = new Set<string>();\n  for (const phrasing of phrasings) {\n    for (const word of tokens(phrasing)) {\n      if (word.length >= 3 && !STOPWORDS.has(word)) {\n        words.add(word);\n      }\n    }\n  }\n  return [...words].slice(0, MAX_WORDS);\n}\n\n/**\n * Adjacent word pairs written as one word (\"tally up\" → \"tallyup\"), capped at\n * eight, for terms the user split in two. Only pairs the index contains are\n * searched, so the nonsense pairs cost nothing.\n */\nexport function joinedWords(phrasings: string[]): string[] {\n  const words = new Set<string>();\n  for (const phrasing of phrasings) {\n    const parts = tokens(phrasing);\n    for (let i = 1; i < parts.length; i++) {\n      words.add(parts[i - 1] + parts[i]);\n    }\n  }\n  return [...words].slice(0, MAX_WORDS);\n}\n\n/** A ranked list of chunk ids from one retrieval method, with its fusion weight. */\nexport interface RankedList {\n  ids: string[];\n  weight: number;\n}\n\nconst RRF_K = 60;\n\n/**\n * Reciprocal rank fusion: each list contributes `weight / (60 + rank)` per id.\n * Rewards ids several methods agree on without comparing their raw scores,\n * which live on different scales.\n */\nexport function fuseRankings(lists: RankedList[]): string[] {\n  const scores = new Map<string, number>();\n  for (const { ids, weight } of lists) {\n    ids.forEach((id, index) => {\n      scores.set(id, (scores.get(id) ?? 0) + weight / (RRF_K + index + 1));\n    });\n  }\n  return [...scores.entries()].sort((a, b) => b[1] - a[1]).map(([id]) => id);\n}\n"],"names":[],"mappings":";;AAAA,MAAM,YAAY,IAAI,GAAA;AAAA,EAElB,oNAAA,CAGA,MAAM,GAAG;AACb,CAAA;AAEA,MAAM,SAAA,GAAY,CAAA;AAElB,SAAS,OAAO,QAAA,EAA4B;AAC1C,EAAA,OAAO,QAAA,CACJ,WAAA,EAAY,CACZ,KAAA,CAAM,mBAAmB,CAAA,CACzB,GAAA,CAAI,CAAA,GAAA,KAAO,GAAA,CAAI,QAAQ,gBAAA,EAAkB,EAAE,CAAC,CAAA,CAC5C,OAAO,OAAO,CAAA;AACnB;AAOO,SAAS,WAAW,SAAA,EAA+B;AACxD,EAAA,MAAM,KAAA,uBAAY,GAAA,EAAY;AAC9B,EAAA,KAAA,MAAW,YAAY,SAAA,EAAW;AAChC,IAAA,KAAA,MAAW,IAAA,IAAQ,MAAA,CAAO,QAAQ,CAAA,EAAG;AACnC,MAAA,IAAI,KAAK,MAAA,IAAU,CAAA,IAAK,CAAC,SAAA,CAAU,GAAA,CAAI,IAAI,CAAA,EAAG;AAC5C,QAAA,KAAA,CAAM,IAAI,IAAI,CAAA;AAAA,MAChB;AAAA,IACF;AAAA,EACF;AACA,EAAA,OAAO,CAAC,GAAG,KAAK,CAAA,CAAE,KAAA,CAAM,GAAG,SAAS,CAAA;AACtC;AAOO,SAAS,YAAY,SAAA,EAA+B;AACzD,EAAA,MAAM,KAAA,uBAAY,GAAA,EAAY;AAC9B,EAAA,KAAA,MAAW,YAAY,SAAA,EAAW;AAChC,IAAA,MAAM,KAAA,GAAQ,OAAO,QAAQ,CAAA;AAC7B,IAAA,KAAA,IAAS,CAAA,GAAI,CAAA,EAAG,CAAA,GAAI,KAAA,CAAM,QAAQ,CAAA,EAAA,EAAK;AACrC,MAAA,KAAA,CAAM,IAAI,KAAA,CAAM,CAAA,GAAI,CAAC,CAAA,GAAI,KAAA,CAAM,CAAC,CAAC,CAAA;AAAA,IACnC;AAAA,EACF;AACA,EAAA,OAAO,CAAC,GAAG,KAAK,CAAA,CAAE,KAAA,CAAM,GAAG,SAAS,CAAA;AACtC;AAQA,MAAM,KAAA,GAAQ,EAAA;AAOP,SAAS,aAAa,KAAA,EAA+B;AAC1D,EAAA,MAAM,MAAA,uBAAa,GAAA,EAAoB;AACvC,EAAA,KAAA,MAAW,EAAE,GAAA,EAAK,MAAA,EAAO,IAAK,KAAA,EAAO;AACnC,IAAA,GAAA,CAAI,OAAA,CAAQ,CAAC,EAAA,EAAI,KAAA,KAAU;AACzB,MAAA,MAAA,CAAO,GAAA,CAAI,EAAA,EAAA,CAAK,MAAA,CAAO,GAAA,CAAI,EAAE,KAAK,CAAA,IAAK,MAAA,IAAU,KAAA,GAAQ,KAAA,GAAQ,CAAA,CAAE,CAAA;AAAA,IACrE,CAAC,CAAA;AAAA,EACH;AACA,EAAA,OAAO,CAAC,GAAG,MAAA,CAAO,OAAA,EAAS,CAAA,CAAE,IAAA,CAAK,CAAC,CAAA,EAAG,CAAA,KAAM,CAAA,CAAE,CAAC,CAAA,GAAI,CAAA,CAAE,CAAC,CAAC,CAAA,CAAE,IAAI,CAAC,CAAC,EAAE,CAAA,KAAM,EAAE,CAAA;AAC3E;;;;;;"}

package/dist/store.cjs.js

@@ -132,23 +132,32 @@ class KnowledgeStore {
     );
     return rows.map((row) => row.id);
   }
-  /** The words that match nothing in the index exactly — likely misspellings. */
-  async unknownWords(words) {
+  /**
+   * How many chunks contain each word, counted up to `cap`: 0 for a word the
+   * index lacks (likely a misspelling), absent for a word Postgres ignores as
+   * a stopword.
+   */
+  async documentFrequencies(words, cap) {
     if (words.length === 0) {
-      return [];
+      return /* @__PURE__ */ new Map();
     }
     const { rows } = await this.db.raw(
-      `SELECT w FROM unnest(?::text[]) AS w
-        WHERE NOT EXISTS (
-          SELECT 1 FROM ${CHUNKS} WHERE tsv @@ plainto_tsquery('english', w)
-        )`,
-      [words]
+      `SELECT w, (
+          SELECT count(*) FROM (
+            SELECT 1 FROM ${CHUNKS} WHERE tsv @@ plainto_tsquery('english', w) LIMIT ?
+          ) hits
+        ) AS n
+        FROM unnest(?::text[]) AS w
+        WHERE numnode(plainto_tsquery('english', w)) > 0`,
+      [cap, words]
+    );
+    return new Map(
+      rows.map((row) => [row.w, Number(row.n)])
     );
-    return rows.map((row) => row.w);
   }
   /**
    * Chunks whose text contains a near spelling of the words, ranked by summed
-   * trigram word similarity. Run only for {@link unknownWords}: a word that
+   * trigram word similarity. Run only for words the index lacks: a word that
    * matches exactly is already covered by {@link keyword}.
    */
   async fuzzy(words, limit) {

package/dist/store.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"store.cjs.js","sources":["../src/store.ts"],"sourcesContent":["import type { DatabaseService } from '@backstage/backend-plugin-api';\nimport type { Chunk } from './chunk';\n\ntype Knex = Awaited<ReturnType<DatabaseService['getClient']>>;\n\nconst CHUNKS = 'assistants_knowledge_chunks';\nconst META = 'assistants_knowledge_meta';\nconst SIGNATURE_KEY = 'embedding';\n\n/** The Postgres extensions the index needs. */\nexport const REQUIRED_EXTENSIONS = ['vector', 'pg_trgm'] as const;\n\n/** pgvector's HNSW index covers up to 2000 dimensions; larger vectors scan exactly. */\nconst HNSW_MAX_DIMENSIONS = 2000;\n\n/** Below the `pg_trgm` default of 0.6 so one typo in a five-letter word (\"pilon\" → \"pylon\") still matches. */\nconst FUZZY_THRESHOLD = 0.45;\n\nexport interface StoredChunk {\n  id: string;\n  source: Chunk['source'];\n  entityRef: string;\n  location: string;\n  title: string;\n  content: string;\n}\n\n/**\n * The knowledge index in the plugin's Postgres database: one row per chunk\n * carrying its embedding (pgvector), a generated full-text vector, and a\n * trigram index over its text.\n *\n * The table is a rebuildable cache of TechDocs and the catalog, so it is\n * managed here rather than by migrations: changing the embedding model or its\n * dimensions drops and recreates it, and the next run refills it.\n */\nexport class KnowledgeStore {\n  constructor(\n    private readonly db: Knex,\n    private readonly embeddingModelId: string,\n  ) {}\n\n  static isPostgres(db: Knex): boolean {\n    return db.client.config.client === 'pg';\n  }\n\n  /**\n   * Enables the required extensions where the connecting role may, and\n   * returns the ones still missing. `pg_trgm` is trusted, so a database owner\n   * can enable it; `vector` usually needs an admin.\n   */\n  async ensureExtensions(): Promise<string[]> {\n    const missing: string[] = [];\n    for (const extension of REQUIRED_EXTENSIONS) {\n      try {\n        await this.db.raw(`CREATE EXTENSION IF NOT EXISTS ${extension}`);\n      } catch {\n        // Not permitted for this role; checked below.\n      }\n      const { rows } = await this.db.raw(\n        'SELECT 1 FROM pg_extension WHERE extname = ?',\n        [extension],\n      );\n      if (rows.length === 0) {\n        missing.push(extension);\n      }\n    }\n    return missing;\n  }\n\n  /** Creates the table for this model's dimensions, rebuilding it if the model changed. */\n  async ensureSchema(dimensions: number): Promise<void> {\n    if (!Number.isInteger(dimensions) || dimensions < 1) {\n      throw new Error(`Invalid embedding dimensions: ${dimensions}`);\n    }\n    const signature = `${this.embeddingModelId}#${dimensions}`;\n\n    await this.db.raw(\n      `CREATE TABLE IF NOT EXISTS ${META} (key text PRIMARY KEY, value text NOT NULL)`,\n    );\n    const current = await this.db(META).where({ key: SIGNATURE_KEY }).first();\n    if (current?.value === signature) {\n      return;\n    }\n\n    await this.db.transaction(async tx => {\n      await tx.raw(`DROP TABLE IF EXISTS ${CHUNKS}`);\n      await tx.raw(`\n        CREATE TABLE ${CHUNKS} (\n          id text PRIMARY KEY,\n          source text NOT NULL,\n          entity_ref text NOT NULL,\n          location text NOT NULL,\n          title text NOT NULL,\n          content text NOT NULL,\n          content_hash text NOT NULL,\n          embedding vector(${dimensions}) NOT NULL,\n          tsv tsvector GENERATED ALWAYS AS (to_tsvector('english', title || ' ' || content)) STORED,\n          updated_at timestamptz NOT NULL DEFAULT now()\n        )`);\n      if (dimensions <= HNSW_MAX_DIMENSIONS) {\n        await tx.raw(\n          `CREATE INDEX ${CHUNKS}_embedding ON ${CHUNKS} USING hnsw (embedding vector_cosine_ops)`,\n        );\n      }\n      await tx.raw(`CREATE INDEX ${CHUNKS}_tsv ON ${CHUNKS} USING gin (tsv)`);\n      await tx.raw(\n        `CREATE INDEX ${CHUNKS}_trgm ON ${CHUNKS} USING gin ((title || ' ' || content) gin_trgm_ops)`,\n      );\n      await tx(META)\n        .insert({ key: SIGNATURE_KEY, value: signature })\n        .onConflict('key')\n        .merge();\n    });\n  }\n\n  /** True once a run has built the table for the configured model. */\n  async isReady(): Promise<boolean> {\n    const { rows } = await this.db.raw(`SELECT to_regclass(?) AS t`, [META]);\n    if (!rows[0]?.t) {\n      return false;\n    }\n    const current = await this.db(META).where({ key: SIGNATURE_KEY }).first();\n    return Boolean(current?.value.startsWith(`${this.embeddingModelId}#`));\n  }\n\n  async hashes(): Promise<Map<string, string>> {\n    const rows = await this.db(CHUNKS).select('id', 'content_hash');\n    return new Map(rows.map(row => [row.id, row.content_hash]));\n  }\n\n  async upsert(chunks: Chunk[], embeddings: number[][]): Promise<void> {\n    if (chunks.length === 0) {\n      return;\n    }\n    await this.db(CHUNKS)\n      .insert(\n        chunks.map((chunk, i) => ({\n          id: chunk.id,\n          source: chunk.source,\n          entity_ref: chunk.entityRef,\n          location: chunk.location,\n          title: chunk.title,\n          content: chunk.content,\n          content_hash: chunk.hash,\n          embedding: this.db.raw('?::vector', [vectorLiteral(embeddings[i])]),\n          updated_at: this.db.fn.now(),\n        })),\n      )\n      .onConflict('id')\n      .merge();\n  }\n\n  /** Deletes chunks not seen this run, except those of entities whose docs failed to load. */\n  async deleteExcept(\n    keepIds: string[],\n    keepEntityRefs: string[],\n  ): Promise<number> {\n    return this.db(CHUNKS)\n      .whereRaw('NOT (id = ANY(?::text[]))', [keepIds])\n      .whereRaw('NOT (entity_ref = ANY(?::text[]))', [keepEntityRefs])\n      .delete();\n  }\n\n  /** Nearest chunks to an embedding by cosine distance. */\n  async nearest(embedding: number[], limit: number): Promise<string[]> {\n    const rows = await this.db(CHUNKS)\n      .select('id')\n      .orderByRaw('embedding <=> ?::vector', [vectorLiteral(embedding)])\n      .limit(limit);\n    return rows.map(row => row.id);\n  }\n\n  /** Chunks matching ANY of the words, ranked by how many and how densely. */\n  async keyword(words: string[], limit: number): Promise<string[]> {\n    if (words.length === 0) {\n      return [];\n    }\n    const { rows } = await this.db.raw(\n      `SELECT id FROM ${CHUNKS}, websearch_to_tsquery('english', ?) q\n        WHERE tsv @@ q\n        ORDER BY ts_rank_cd(tsv, q) DESC\n        LIMIT ?`,\n      [words.join(' or '), limit],\n    );\n    return rows.map((row: { id: string }) => row.id);\n  }\n\n  /** The words that match nothing in the index exactly — likely misspellings. */\n  async unknownWords(words: string[]): Promise<string[]> {\n    if (words.length === 0) {\n      return [];\n    }\n    const { rows } = await this.db.raw(\n      `SELECT w FROM unnest(?::text[]) AS w\n        WHERE NOT EXISTS (\n          SELECT 1 FROM ${CHUNKS} WHERE tsv @@ plainto_tsquery('english', w)\n        )`,\n      [words],\n    );\n    return rows.map((row: { w: string }) => row.w);\n  }\n\n  /**\n   * Chunks whose text contains a near spelling of the words, ranked by summed\n   * trigram word similarity. Run only for {@link unknownWords}: a word that\n   * matches exactly is already covered by {@link keyword}.\n   */\n  async fuzzy(words: string[], limit: number): Promise<string[]> {\n    if (words.length === 0) {\n      return [];\n    }\n    const text = `(title || ' ' || content)`;\n    return this.db.transaction(async tx => {\n      await tx.raw(\n        `SELECT set_config('pg_trgm.word_similarity_threshold', ?, true)`,\n        [String(FUZZY_THRESHOLD)],\n      );\n      const { rows } = await tx.raw(\n        `SELECT id FROM ${CHUNKS}\n          WHERE ${words.map(() => `? <% ${text}`).join(' OR ')}\n          ORDER BY ${words\n            .map(() => `word_similarity(?, ${text})`)\n            .join(' + ')} DESC\n          LIMIT ?`,\n        [...words, ...words, limit],\n      );\n      return rows.map((row: { id: string }) => row.id);\n    });\n  }\n\n  async chunks(ids: string[]): Promise<StoredChunk[]> {\n    if (ids.length === 0) {\n      return [];\n    }\n    const rows = await this.db(CHUNKS)\n      .select('id', 'source', 'entity_ref', 'location', 'title', 'content')\n      .whereRaw('id = ANY(?::text[])', [ids]);\n    return rows.map(row => ({\n      id: row.id,\n      source: row.source,\n      entityRef: row.entity_ref,\n      location: row.location,\n      title: row.title,\n      content: row.content,\n    }));\n  }\n}\n\nfunction vectorLiteral(embedding: number[]): string {\n  return `[${embedding.join(',')}]`;\n}\n"],"names":[],"mappings":";;AAKA,MAAM,MAAA,GAAS,6BAAA;AACf,MAAM,IAAA,GAAO,2BAAA;AACb,MAAM,aAAA,GAAgB,WAAA;AAGf,MAAM,mBAAA,GAAsB,CAAC,QAAA,EAAU,SAAS;AAGvD,MAAM,mBAAA,GAAsB,GAAA;AAG5B,MAAM,eAAA,GAAkB,IAAA;AAoBjB,MAAM,cAAA,CAAe;AAAA,EAC1B,WAAA,CACmB,IACA,gBAAA,EACjB;AAFiB,IAAA,IAAA,CAAA,EAAA,GAAA,EAAA;AACA,IAAA,IAAA,CAAA,gBAAA,GAAA,gBAAA;AAAA,EAChB;AAAA,EAFgB,EAAA;AAAA,EACA,gBAAA;AAAA,EAGnB,OAAO,WAAW,EAAA,EAAmB;AACnC,IAAA,OAAO,EAAA,CAAG,MAAA,CAAO,MAAA,CAAO,MAAA,KAAW,IAAA;AAAA,EACrC;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,MAAM,gBAAA,GAAsC;AAC1C,IAAA,MAAM,UAAoB,EAAC;AAC3B,IAAA,KAAA,MAAW,aAAa,mBAAA,EAAqB;AAC3C,MAAA,IAAI;AACF,QAAA,MAAM,IAAA,CAAK,EAAA,CAAG,GAAA,CAAI,CAAA,+BAAA,EAAkC,SAAS,CAAA,CAAE,CAAA;AAAA,MACjE,CAAA,CAAA,MAAQ;AAAA,MAER;AACA,MAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,KAAK,EAAA,CAAG,GAAA;AAAA,QAC7B,8CAAA;AAAA,QACA,CAAC,SAAS;AAAA,OACZ;AACA,MAAA,IAAI,IAAA,CAAK,WAAW,CAAA,EAAG;AACrB,QAAA,OAAA,CAAQ,KAAK,SAAS,CAAA;AAAA,MACxB;AAAA,IACF;AACA,IAAA,OAAO,OAAA;AAAA,EACT;AAAA;AAAA,EAGA,MAAM,aAAa,UAAA,EAAmC;AACpD,IAAA,IAAI,CAAC,MAAA,CAAO,SAAA,CAAU,UAAU,CAAA,IAAK,aAAa,CAAA,EAAG;AACnD,MAAA,MAAM,IAAI,KAAA,CAAM,CAAA,8BAAA,EAAiC,UAAU,CAAA,CAAE,CAAA;AAAA,IAC/D;AACA,IAAA,MAAM,SAAA,GAAY,CAAA,EAAG,IAAA,CAAK,gBAAgB,IAAI,UAAU,CAAA,CAAA;AAExD,IAAA,MAAM,KAAK,EAAA,CAAG,GAAA;AAAA,MACZ,8BAA8B,IAAI,CAAA,4CAAA;AAAA,KACpC;AACA,IAAA,MAAM,OAAA,GAAU,MAAM,IAAA,CAAK,EAAA,CAAG,IAAI,CAAA,CAAE,KAAA,CAAM,EAAE,GAAA,EAAK,aAAA,EAAe,CAAA,CAAE,KAAA,EAAM;AACxE,IAAA,IAAI,OAAA,EAAS,UAAU,SAAA,EAAW;AAChC,MAAA;AAAA,IACF;AAEA,IAAA,MAAM,IAAA,CAAK,EAAA,CAAG,WAAA,CAAY,OAAM,EAAA,KAAM;AACpC,MAAA,MAAM,EAAA,CAAG,GAAA,CAAI,CAAA,qBAAA,EAAwB,MAAM,CAAA,CAAE,CAAA;AAC7C,MAAA,MAAM,GAAG,GAAA,CAAI;AAAA,qBAAA,EACI,MAAM,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,2BAAA,EAQA,UAAU,CAAA;AAAA;AAAA;AAAA,SAAA,CAG7B,CAAA;AACJ,MAAA,IAAI,cAAc,mBAAA,EAAqB;AACrC,QAAA,MAAM,EAAA,CAAG,GAAA;AAAA,UACP,CAAA,aAAA,EAAgB,MAAM,CAAA,cAAA,EAAiB,MAAM,CAAA,yCAAA;AAAA,SAC/C;AAAA,MACF;AACA,MAAA,MAAM,GAAG,GAAA,CAAI,CAAA,aAAA,EAAgB,MAAM,CAAA,QAAA,EAAW,MAAM,CAAA,gBAAA,CAAkB,CAAA;AACtE,MAAA,MAAM,EAAA,CAAG,GAAA;AAAA,QACP,CAAA,aAAA,EAAgB,MAAM,CAAA,SAAA,EAAY,MAAM,CAAA,mDAAA;AAAA,OAC1C;AACA,MAAA,MAAM,EAAA,CAAG,IAAI,CAAA,CACV,MAAA,CAAO,EAAE,GAAA,EAAK,aAAA,EAAe,KAAA,EAAO,SAAA,EAAW,CAAA,CAC/C,UAAA,CAAW,KAAK,EAChB,KAAA,EAAM;AAAA,IACX,CAAC,CAAA;AAAA,EACH;AAAA;AAAA,EAGA,MAAM,OAAA,GAA4B;AAChC,IAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,IAAA,CAAK,GAAG,GAAA,CAAI,CAAA,0BAAA,CAAA,EAA8B,CAAC,IAAI,CAAC,CAAA;AACvE,IAAA,IAAI,CAAC,IAAA,CAAK,CAAC,CAAA,EAAG,CAAA,EAAG;AACf,MAAA,OAAO,KAAA;AAAA,IACT;AACA,IAAA,MAAM,OAAA,GAAU,MAAM,IAAA,CAAK,EAAA,CAAG,IAAI,CAAA,CAAE,KAAA,CAAM,EAAE,GAAA,EAAK,aAAA,EAAe,CAAA,CAAE,KAAA,EAAM;AACxE,IAAA,OAAO,OAAA,CAAQ,SAAS,KAAA,CAAM,UAAA,CAAW,GAAG,IAAA,CAAK,gBAAgB,GAAG,CAAC,CAAA;AAAA,EACvE;AAAA,EAEA,MAAM,MAAA,GAAuC;AAC3C,IAAA,MAAM,IAAA,GAAO,MAAM,IAAA,CAAK,EAAA,CAAG,MAAM,CAAA,CAAE,MAAA,CAAO,MAAM,cAAc,CAAA;AAC9D,IAAA,OAAO,IAAI,GAAA,CAAI,IAAA,CAAK,GAAA,CAAI,CAAA,GAAA,KAAO,CAAC,GAAA,CAAI,EAAA,EAAI,GAAA,CAAI,YAAY,CAAC,CAAC,CAAA;AAAA,EAC5D;AAAA,EAEA,MAAM,MAAA,CAAO,MAAA,EAAiB,UAAA,EAAuC;AACnE,IAAA,IAAI,MAAA,CAAO,WAAW,CAAA,EAAG;AACvB,MAAA;AAAA,IACF;AACA,IAAA,MAAM,IAAA,CAAK,EAAA,CAAG,MAAM,CAAA,CACjB,MAAA;AAAA,MACC,MAAA,CAAO,GAAA,CAAI,CAAC,KAAA,EAAO,CAAA,MAAO;AAAA,QACxB,IAAI,KAAA,CAAM,EAAA;AAAA,QACV,QAAQ,KAAA,CAAM,MAAA;AAAA,QACd,YAAY,KAAA,CAAM,SAAA;AAAA,QAClB,UAAU,KAAA,CAAM,QAAA;AAAA,QAChB,OAAO,KAAA,CAAM,KAAA;AAAA,QACb,SAAS,KAAA,CAAM,OAAA;AAAA,QACf,cAAc,KAAA,CAAM,IAAA;AAAA,QACpB,SAAA,EAAW,IAAA,CAAK,EAAA,CAAG,GAAA,CAAI,WAAA,EAAa,CAAC,aAAA,CAAc,UAAA,CAAW,CAAC,CAAC,CAAC,CAAC,CAAA;AAAA,QAClE,UAAA,EAAY,IAAA,CAAK,EAAA,CAAG,EAAA,CAAG,GAAA;AAAI,OAC7B,CAAE;AAAA,KACJ,CACC,UAAA,CAAW,IAAI,CAAA,CACf,KAAA,EAAM;AAAA,EACX;AAAA;AAAA,EAGA,MAAM,YAAA,CACJ,OAAA,EACA,cAAA,EACiB;AACjB,IAAA,OAAO,KAAK,EAAA,CAAG,MAAM,CAAA,CAClB,QAAA,CAAS,6BAA6B,CAAC,OAAO,CAAC,CAAA,CAC/C,SAAS,mCAAA,EAAqC,CAAC,cAAc,CAAC,EAC9D,MAAA,EAAO;AAAA,EACZ;AAAA;AAAA,EAGA,MAAM,OAAA,CAAQ,SAAA,EAAqB,KAAA,EAAkC;AACnE,IAAA,MAAM,OAAO,MAAM,IAAA,CAAK,GAAG,MAAM,CAAA,CAC9B,OAAO,IAAI,CAAA,CACX,UAAA,CAAW,yBAAA,EAA2B,CAAC,aAAA,CAAc,SAAS,CAAC,CAAC,CAAA,CAChE,MAAM,KAAK,CAAA;AACd,IAAA,OAAO,IAAA,CAAK,GAAA,CAAI,CAAA,GAAA,KAAO,GAAA,CAAI,EAAE,CAAA;AAAA,EAC/B;AAAA;AAAA,EAGA,MAAM,OAAA,CAAQ,KAAA,EAAiB,KAAA,EAAkC;AAC/D,IAAA,IAAI,KAAA,CAAM,WAAW,CAAA,EAAG;AACtB,MAAA,OAAO,EAAC;AAAA,IACV;AACA,IAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,KAAK,EAAA,CAAG,GAAA;AAAA,MAC7B,kBAAkB,MAAM,CAAA;AAAA;AAAA;AAAA,eAAA,CAAA;AAAA,MAIxB,CAAC,KAAA,CAAM,IAAA,CAAK,MAAM,GAAG,KAAK;AAAA,KAC5B;AACA,IAAA,OAAO,IAAA,CAAK,GAAA,CAAI,CAAC,GAAA,KAAwB,IAAI,EAAE,CAAA;AAAA,EACjD;AAAA;AAAA,EAGA,MAAM,aAAa,KAAA,EAAoC;AACrD,IAAA,IAAI,KAAA,CAAM,WAAW,CAAA,EAAG;AACtB,MAAA,OAAO,EAAC;AAAA,IACV;AACA,IAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,KAAK,EAAA,CAAG,GAAA;AAAA,MAC7B,CAAA;AAAA;AAAA,wBAAA,EAEoB,MAAM,CAAA;AAAA,SAAA,CAAA;AAAA,MAE1B,CAAC,KAAK;AAAA,KACR;AACA,IAAA,OAAO,IAAA,CAAK,GAAA,CAAI,CAAC,GAAA,KAAuB,IAAI,CAAC,CAAA;AAAA,EAC/C;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,MAAM,KAAA,CAAM,KAAA,EAAiB,KAAA,EAAkC;AAC7D,IAAA,IAAI,KAAA,CAAM,WAAW,CAAA,EAAG;AACtB,MAAA,OAAO,EAAC;AAAA,IACV;AACA,IAAA,MAAM,IAAA,GAAO,CAAA,yBAAA,CAAA;AACb,IAAA,OAAO,IAAA,CAAK,EAAA,CAAG,WAAA,CAAY,OAAM,EAAA,KAAM;AACrC,MAAA,MAAM,EAAA,CAAG,GAAA;AAAA,QACP,CAAA,+DAAA,CAAA;AAAA,QACA,CAAC,MAAA,CAAO,eAAe,CAAC;AAAA,OAC1B;AACA,MAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,EAAA,CAAG,GAAA;AAAA,QACxB,kBAAkB,MAAM;AAAA,gBAAA,EACd,KAAA,CAAM,IAAI,MAAM,CAAA,KAAA,EAAQ,IAAI,CAAA,CAAE,CAAA,CAAE,IAAA,CAAK,MAAM,CAAC;AAAA,mBAAA,EACzC,KAAA,CACR,IAAI,MAAM,CAAA,mBAAA,EAAsB,IAAI,CAAA,CAAA,CAAG,CAAA,CACvC,IAAA,CAAK,KAAK,CAAC,CAAA;AAAA,iBAAA,CAAA;AAAA,QAEhB,CAAC,GAAG,KAAA,EAAO,GAAG,OAAO,KAAK;AAAA,OAC5B;AACA,MAAA,OAAO,IAAA,CAAK,GAAA,CAAI,CAAC,GAAA,KAAwB,IAAI,EAAE,CAAA;AAAA,IACjD,CAAC,CAAA;AAAA,EACH;AAAA,EAEA,MAAM,OAAO,GAAA,EAAuC;AAClD,IAAA,IAAI,GAAA,CAAI,WAAW,CAAA,EAAG;AACpB,MAAA,OAAO,EAAC;AAAA,IACV;AACA,IAAA,MAAM,OAAO,MAAM,IAAA,CAAK,GAAG,MAAM,CAAA,CAC9B,OAAO,IAAA,EAAM,QAAA,EAAU,YAAA,EAAc,UAAA,EAAY,SAAS,SAAS,CAAA,CACnE,SAAS,qBAAA,EAAuB,CAAC,GAAG,CAAC,CAAA;AACxC,IAAA,OAAO,IAAA,CAAK,IAAI,CAAA,GAAA,MAAQ;AAAA,MACtB,IAAI,GAAA,CAAI,EAAA;AAAA,MACR,QAAQ,GAAA,CAAI,MAAA;AAAA,MACZ,WAAW,GAAA,CAAI,UAAA;AAAA,MACf,UAAU,GAAA,CAAI,QAAA;AAAA,MACd,OAAO,GAAA,CAAI,KAAA;AAAA,MACX,SAAS,GAAA,CAAI;AAAA,KACf,CAAE,CAAA;AAAA,EACJ;AACF;AAEA,SAAS,cAAc,SAAA,EAA6B;AAClD,EAAA,OAAO,CAAA,CAAA,EAAI,SAAA,CAAU,IAAA,CAAK,GAAG,CAAC,CAAA,CAAA,CAAA;AAChC;;;;;"}
+{"version":3,"file":"store.cjs.js","sources":["../src/store.ts"],"sourcesContent":["import type { DatabaseService } from '@backstage/backend-plugin-api';\nimport type { Chunk } from './chunk';\n\ntype Knex = Awaited<ReturnType<DatabaseService['getClient']>>;\n\nconst CHUNKS = 'assistants_knowledge_chunks';\nconst META = 'assistants_knowledge_meta';\nconst SIGNATURE_KEY = 'embedding';\n\n/** The Postgres extensions the index needs. */\nexport const REQUIRED_EXTENSIONS = ['vector', 'pg_trgm'] as const;\n\n/** pgvector's HNSW index covers up to 2000 dimensions; larger vectors scan exactly. */\nconst HNSW_MAX_DIMENSIONS = 2000;\n\n/** Below the `pg_trgm` default of 0.6 so one typo in a five-letter word (\"pilon\" → \"pylon\") still matches. */\nconst FUZZY_THRESHOLD = 0.45;\n\nexport interface StoredChunk {\n  id: string;\n  source: Chunk['source'];\n  entityRef: string;\n  location: string;\n  title: string;\n  content: string;\n}\n\n/**\n * The knowledge index in the plugin's Postgres database: one row per chunk\n * carrying its embedding (pgvector), a generated full-text vector, and a\n * trigram index over its text.\n *\n * The table is a rebuildable cache of TechDocs and the catalog, so it is\n * managed here rather than by migrations: changing the embedding model or its\n * dimensions drops and recreates it, and the next run refills it.\n */\nexport class KnowledgeStore {\n  constructor(\n    private readonly db: Knex,\n    private readonly embeddingModelId: string,\n  ) {}\n\n  static isPostgres(db: Knex): boolean {\n    return db.client.config.client === 'pg';\n  }\n\n  /**\n   * Enables the required extensions where the connecting role may, and\n   * returns the ones still missing. `pg_trgm` is trusted, so a database owner\n   * can enable it; `vector` usually needs an admin.\n   */\n  async ensureExtensions(): Promise<string[]> {\n    const missing: string[] = [];\n    for (const extension of REQUIRED_EXTENSIONS) {\n      try {\n        await this.db.raw(`CREATE EXTENSION IF NOT EXISTS ${extension}`);\n      } catch {\n        // Not permitted for this role; checked below.\n      }\n      const { rows } = await this.db.raw(\n        'SELECT 1 FROM pg_extension WHERE extname = ?',\n        [extension],\n      );\n      if (rows.length === 0) {\n        missing.push(extension);\n      }\n    }\n    return missing;\n  }\n\n  /** Creates the table for this model's dimensions, rebuilding it if the model changed. */\n  async ensureSchema(dimensions: number): Promise<void> {\n    if (!Number.isInteger(dimensions) || dimensions < 1) {\n      throw new Error(`Invalid embedding dimensions: ${dimensions}`);\n    }\n    const signature = `${this.embeddingModelId}#${dimensions}`;\n\n    await this.db.raw(\n      `CREATE TABLE IF NOT EXISTS ${META} (key text PRIMARY KEY, value text NOT NULL)`,\n    );\n    const current = await this.db(META).where({ key: SIGNATURE_KEY }).first();\n    if (current?.value === signature) {\n      return;\n    }\n\n    await this.db.transaction(async tx => {\n      await tx.raw(`DROP TABLE IF EXISTS ${CHUNKS}`);\n      await tx.raw(`\n        CREATE TABLE ${CHUNKS} (\n          id text PRIMARY KEY,\n          source text NOT NULL,\n          entity_ref text NOT NULL,\n          location text NOT NULL,\n          title text NOT NULL,\n          content text NOT NULL,\n          content_hash text NOT NULL,\n          embedding vector(${dimensions}) NOT NULL,\n          tsv tsvector GENERATED ALWAYS AS (to_tsvector('english', title || ' ' || content)) STORED,\n          updated_at timestamptz NOT NULL DEFAULT now()\n        )`);\n      if (dimensions <= HNSW_MAX_DIMENSIONS) {\n        await tx.raw(\n          `CREATE INDEX ${CHUNKS}_embedding ON ${CHUNKS} USING hnsw (embedding vector_cosine_ops)`,\n        );\n      }\n      await tx.raw(`CREATE INDEX ${CHUNKS}_tsv ON ${CHUNKS} USING gin (tsv)`);\n      await tx.raw(\n        `CREATE INDEX ${CHUNKS}_trgm ON ${CHUNKS} USING gin ((title || ' ' || content) gin_trgm_ops)`,\n      );\n      await tx(META)\n        .insert({ key: SIGNATURE_KEY, value: signature })\n        .onConflict('key')\n        .merge();\n    });\n  }\n\n  /** True once a run has built the table for the configured model. */\n  async isReady(): Promise<boolean> {\n    const { rows } = await this.db.raw(`SELECT to_regclass(?) AS t`, [META]);\n    if (!rows[0]?.t) {\n      return false;\n    }\n    const current = await this.db(META).where({ key: SIGNATURE_KEY }).first();\n    return Boolean(current?.value.startsWith(`${this.embeddingModelId}#`));\n  }\n\n  async hashes(): Promise<Map<string, string>> {\n    const rows = await this.db(CHUNKS).select('id', 'content_hash');\n    return new Map(rows.map(row => [row.id, row.content_hash]));\n  }\n\n  async upsert(chunks: Chunk[], embeddings: number[][]): Promise<void> {\n    if (chunks.length === 0) {\n      return;\n    }\n    await this.db(CHUNKS)\n      .insert(\n        chunks.map((chunk, i) => ({\n          id: chunk.id,\n          source: chunk.source,\n          entity_ref: chunk.entityRef,\n          location: chunk.location,\n          title: chunk.title,\n          content: chunk.content,\n          content_hash: chunk.hash,\n          embedding: this.db.raw('?::vector', [vectorLiteral(embeddings[i])]),\n          updated_at: this.db.fn.now(),\n        })),\n      )\n      .onConflict('id')\n      .merge();\n  }\n\n  /** Deletes chunks not seen this run, except those of entities whose docs failed to load. */\n  async deleteExcept(\n    keepIds: string[],\n    keepEntityRefs: string[],\n  ): Promise<number> {\n    return this.db(CHUNKS)\n      .whereRaw('NOT (id = ANY(?::text[]))', [keepIds])\n      .whereRaw('NOT (entity_ref = ANY(?::text[]))', [keepEntityRefs])\n      .delete();\n  }\n\n  /** Nearest chunks to an embedding by cosine distance. */\n  async nearest(embedding: number[], limit: number): Promise<string[]> {\n    const rows = await this.db(CHUNKS)\n      .select('id')\n      .orderByRaw('embedding <=> ?::vector', [vectorLiteral(embedding)])\n      .limit(limit);\n    return rows.map(row => row.id);\n  }\n\n  /** Chunks matching ANY of the words, ranked by how many and how densely. */\n  async keyword(words: string[], limit: number): Promise<string[]> {\n    if (words.length === 0) {\n      return [];\n    }\n    const { rows } = await this.db.raw(\n      `SELECT id FROM ${CHUNKS}, websearch_to_tsquery('english', ?) q\n        WHERE tsv @@ q\n        ORDER BY ts_rank_cd(tsv, q) DESC\n        LIMIT ?`,\n      [words.join(' or '), limit],\n    );\n    return rows.map((row: { id: string }) => row.id);\n  }\n\n  /**\n   * How many chunks contain each word, counted up to `cap`: 0 for a word the\n   * index lacks (likely a misspelling), absent for a word Postgres ignores as\n   * a stopword.\n   */\n  async documentFrequencies(\n    words: string[],\n    cap: number,\n  ): Promise<Map<string, number>> {\n    if (words.length === 0) {\n      return new Map();\n    }\n    const { rows } = await this.db.raw(\n      `SELECT w, (\n          SELECT count(*) FROM (\n            SELECT 1 FROM ${CHUNKS} WHERE tsv @@ plainto_tsquery('english', w) LIMIT ?\n          ) hits\n        ) AS n\n        FROM unnest(?::text[]) AS w\n        WHERE numnode(plainto_tsquery('english', w)) > 0`,\n      [cap, words],\n    );\n    return new Map(\n      rows.map((row: { w: string; n: string }) => [row.w, Number(row.n)]),\n    );\n  }\n\n  /**\n   * Chunks whose text contains a near spelling of the words, ranked by summed\n   * trigram word similarity. Run only for words the index lacks: a word that\n   * matches exactly is already covered by {@link keyword}.\n   */\n  async fuzzy(words: string[], limit: number): Promise<string[]> {\n    if (words.length === 0) {\n      return [];\n    }\n    const text = `(title || ' ' || content)`;\n    return this.db.transaction(async tx => {\n      await tx.raw(\n        `SELECT set_config('pg_trgm.word_similarity_threshold', ?, true)`,\n        [String(FUZZY_THRESHOLD)],\n      );\n      const { rows } = await tx.raw(\n        `SELECT id FROM ${CHUNKS}\n          WHERE ${words.map(() => `? <% ${text}`).join(' OR ')}\n          ORDER BY ${words\n            .map(() => `word_similarity(?, ${text})`)\n            .join(' + ')} DESC\n          LIMIT ?`,\n        [...words, ...words, limit],\n      );\n      return rows.map((row: { id: string }) => row.id);\n    });\n  }\n\n  async chunks(ids: string[]): Promise<StoredChunk[]> {\n    if (ids.length === 0) {\n      return [];\n    }\n    const rows = await this.db(CHUNKS)\n      .select('id', 'source', 'entity_ref', 'location', 'title', 'content')\n      .whereRaw('id = ANY(?::text[])', [ids]);\n    return rows.map(row => ({\n      id: row.id,\n      source: row.source,\n      entityRef: row.entity_ref,\n      location: row.location,\n      title: row.title,\n      content: row.content,\n    }));\n  }\n}\n\nfunction vectorLiteral(embedding: number[]): string {\n  return `[${embedding.join(',')}]`;\n}\n"],"names":[],"mappings":";;AAKA,MAAM,MAAA,GAAS,6BAAA;AACf,MAAM,IAAA,GAAO,2BAAA;AACb,MAAM,aAAA,GAAgB,WAAA;AAGf,MAAM,mBAAA,GAAsB,CAAC,QAAA,EAAU,SAAS;AAGvD,MAAM,mBAAA,GAAsB,GAAA;AAG5B,MAAM,eAAA,GAAkB,IAAA;AAoBjB,MAAM,cAAA,CAAe;AAAA,EAC1B,WAAA,CACmB,IACA,gBAAA,EACjB;AAFiB,IAAA,IAAA,CAAA,EAAA,GAAA,EAAA;AACA,IAAA,IAAA,CAAA,gBAAA,GAAA,gBAAA;AAAA,EAChB;AAAA,EAFgB,EAAA;AAAA,EACA,gBAAA;AAAA,EAGnB,OAAO,WAAW,EAAA,EAAmB;AACnC,IAAA,OAAO,EAAA,CAAG,MAAA,CAAO,MAAA,CAAO,MAAA,KAAW,IAAA;AAAA,EACrC;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,MAAM,gBAAA,GAAsC;AAC1C,IAAA,MAAM,UAAoB,EAAC;AAC3B,IAAA,KAAA,MAAW,aAAa,mBAAA,EAAqB;AAC3C,MAAA,IAAI;AACF,QAAA,MAAM,IAAA,CAAK,EAAA,CAAG,GAAA,CAAI,CAAA,+BAAA,EAAkC,SAAS,CAAA,CAAE,CAAA;AAAA,MACjE,CAAA,CAAA,MAAQ;AAAA,MAER;AACA,MAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,KAAK,EAAA,CAAG,GAAA;AAAA,QAC7B,8CAAA;AAAA,QACA,CAAC,SAAS;AAAA,OACZ;AACA,MAAA,IAAI,IAAA,CAAK,WAAW,CAAA,EAAG;AACrB,QAAA,OAAA,CAAQ,KAAK,SAAS,CAAA;AAAA,MACxB;AAAA,IACF;AACA,IAAA,OAAO,OAAA;AAAA,EACT;AAAA;AAAA,EAGA,MAAM,aAAa,UAAA,EAAmC;AACpD,IAAA,IAAI,CAAC,MAAA,CAAO,SAAA,CAAU,UAAU,CAAA,IAAK,aAAa,CAAA,EAAG;AACnD,MAAA,MAAM,IAAI,KAAA,CAAM,CAAA,8BAAA,EAAiC,UAAU,CAAA,CAAE,CAAA;AAAA,IAC/D;AACA,IAAA,MAAM,SAAA,GAAY,CAAA,EAAG,IAAA,CAAK,gBAAgB,IAAI,UAAU,CAAA,CAAA;AAExD,IAAA,MAAM,KAAK,EAAA,CAAG,GAAA;AAAA,MACZ,8BAA8B,IAAI,CAAA,4CAAA;AAAA,KACpC;AACA,IAAA,MAAM,OAAA,GAAU,MAAM,IAAA,CAAK,EAAA,CAAG,IAAI,CAAA,CAAE,KAAA,CAAM,EAAE,GAAA,EAAK,aAAA,EAAe,CAAA,CAAE,KAAA,EAAM;AACxE,IAAA,IAAI,OAAA,EAAS,UAAU,SAAA,EAAW;AAChC,MAAA;AAAA,IACF;AAEA,IAAA,MAAM,IAAA,CAAK,EAAA,CAAG,WAAA,CAAY,OAAM,EAAA,KAAM;AACpC,MAAA,MAAM,EAAA,CAAG,GAAA,CAAI,CAAA,qBAAA,EAAwB,MAAM,CAAA,CAAE,CAAA;AAC7C,MAAA,MAAM,GAAG,GAAA,CAAI;AAAA,qBAAA,EACI,MAAM,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,2BAAA,EAQA,UAAU,CAAA;AAAA;AAAA;AAAA,SAAA,CAG7B,CAAA;AACJ,MAAA,IAAI,cAAc,mBAAA,EAAqB;AACrC,QAAA,MAAM,EAAA,CAAG,GAAA;AAAA,UACP,CAAA,aAAA,EAAgB,MAAM,CAAA,cAAA,EAAiB,MAAM,CAAA,yCAAA;AAAA,SAC/C;AAAA,MACF;AACA,MAAA,MAAM,GAAG,GAAA,CAAI,CAAA,aAAA,EAAgB,MAAM,CAAA,QAAA,EAAW,MAAM,CAAA,gBAAA,CAAkB,CAAA;AACtE,MAAA,MAAM,EAAA,CAAG,GAAA;AAAA,QACP,CAAA,aAAA,EAAgB,MAAM,CAAA,SAAA,EAAY,MAAM,CAAA,mDAAA;AAAA,OAC1C;AACA,MAAA,MAAM,EAAA,CAAG,IAAI,CAAA,CACV,MAAA,CAAO,EAAE,GAAA,EAAK,aAAA,EAAe,KAAA,EAAO,SAAA,EAAW,CAAA,CAC/C,UAAA,CAAW,KAAK,EAChB,KAAA,EAAM;AAAA,IACX,CAAC,CAAA;AAAA,EACH;AAAA;AAAA,EAGA,MAAM,OAAA,GAA4B;AAChC,IAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,IAAA,CAAK,GAAG,GAAA,CAAI,CAAA,0BAAA,CAAA,EAA8B,CAAC,IAAI,CAAC,CAAA;AACvE,IAAA,IAAI,CAAC,IAAA,CAAK,CAAC,CAAA,EAAG,CAAA,EAAG;AACf,MAAA,OAAO,KAAA;AAAA,IACT;AACA,IAAA,MAAM,OAAA,GAAU,MAAM,IAAA,CAAK,EAAA,CAAG,IAAI,CAAA,CAAE,KAAA,CAAM,EAAE,GAAA,EAAK,aAAA,EAAe,CAAA,CAAE,KAAA,EAAM;AACxE,IAAA,OAAO,OAAA,CAAQ,SAAS,KAAA,CAAM,UAAA,CAAW,GAAG,IAAA,CAAK,gBAAgB,GAAG,CAAC,CAAA;AAAA,EACvE;AAAA,EAEA,MAAM,MAAA,GAAuC;AAC3C,IAAA,MAAM,IAAA,GAAO,MAAM,IAAA,CAAK,EAAA,CAAG,MAAM,CAAA,CAAE,MAAA,CAAO,MAAM,cAAc,CAAA;AAC9D,IAAA,OAAO,IAAI,GAAA,CAAI,IAAA,CAAK,GAAA,CAAI,CAAA,GAAA,KAAO,CAAC,GAAA,CAAI,EAAA,EAAI,GAAA,CAAI,YAAY,CAAC,CAAC,CAAA;AAAA,EAC5D;AAAA,EAEA,MAAM,MAAA,CAAO,MAAA,EAAiB,UAAA,EAAuC;AACnE,IAAA,IAAI,MAAA,CAAO,WAAW,CAAA,EAAG;AACvB,MAAA;AAAA,IACF;AACA,IAAA,MAAM,IAAA,CAAK,EAAA,CAAG,MAAM,CAAA,CACjB,MAAA;AAAA,MACC,MAAA,CAAO,GAAA,CAAI,CAAC,KAAA,EAAO,CAAA,MAAO;AAAA,QACxB,IAAI,KAAA,CAAM,EAAA;AAAA,QACV,QAAQ,KAAA,CAAM,MAAA;AAAA,QACd,YAAY,KAAA,CAAM,SAAA;AAAA,QAClB,UAAU,KAAA,CAAM,QAAA;AAAA,QAChB,OAAO,KAAA,CAAM,KAAA;AAAA,QACb,SAAS,KAAA,CAAM,OAAA;AAAA,QACf,cAAc,KAAA,CAAM,IAAA;AAAA,QACpB,SAAA,EAAW,IAAA,CAAK,EAAA,CAAG,GAAA,CAAI,WAAA,EAAa,CAAC,aAAA,CAAc,UAAA,CAAW,CAAC,CAAC,CAAC,CAAC,CAAA;AAAA,QAClE,UAAA,EAAY,IAAA,CAAK,EAAA,CAAG,EAAA,CAAG,GAAA;AAAI,OAC7B,CAAE;AAAA,KACJ,CACC,UAAA,CAAW,IAAI,CAAA,CACf,KAAA,EAAM;AAAA,EACX;AAAA;AAAA,EAGA,MAAM,YAAA,CACJ,OAAA,EACA,cAAA,EACiB;AACjB,IAAA,OAAO,KAAK,EAAA,CAAG,MAAM,CAAA,CAClB,QAAA,CAAS,6BAA6B,CAAC,OAAO,CAAC,CAAA,CAC/C,SAAS,mCAAA,EAAqC,CAAC,cAAc,CAAC,EAC9D,MAAA,EAAO;AAAA,EACZ;AAAA;AAAA,EAGA,MAAM,OAAA,CAAQ,SAAA,EAAqB,KAAA,EAAkC;AACnE,IAAA,MAAM,OAAO,MAAM,IAAA,CAAK,GAAG,MAAM,CAAA,CAC9B,OAAO,IAAI,CAAA,CACX,UAAA,CAAW,yBAAA,EAA2B,CAAC,aAAA,CAAc,SAAS,CAAC,CAAC,CAAA,CAChE,MAAM,KAAK,CAAA;AACd,IAAA,OAAO,IAAA,CAAK,GAAA,CAAI,CAAA,GAAA,KAAO,GAAA,CAAI,EAAE,CAAA;AAAA,EAC/B;AAAA;AAAA,EAGA,MAAM,OAAA,CAAQ,KAAA,EAAiB,KAAA,EAAkC;AAC/D,IAAA,IAAI,KAAA,CAAM,WAAW,CAAA,EAAG;AACtB,MAAA,OAAO,EAAC;AAAA,IACV;AACA,IAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,KAAK,EAAA,CAAG,GAAA;AAAA,MAC7B,kBAAkB,MAAM,CAAA;AAAA;AAAA;AAAA,eAAA,CAAA;AAAA,MAIxB,CAAC,KAAA,CAAM,IAAA,CAAK,MAAM,GAAG,KAAK;AAAA,KAC5B;AACA,IAAA,OAAO,IAAA,CAAK,GAAA,CAAI,CAAC,GAAA,KAAwB,IAAI,EAAE,CAAA;AAAA,EACjD;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,MAAM,mBAAA,CACJ,KAAA,EACA,GAAA,EAC8B;AAC9B,IAAA,IAAI,KAAA,CAAM,WAAW,CAAA,EAAG;AACtB,MAAA,2BAAW,GAAA,EAAI;AAAA,IACjB;AACA,IAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,KAAK,EAAA,CAAG,GAAA;AAAA,MAC7B,CAAA;AAAA;AAAA,0BAAA,EAEsB,MAAM,CAAA;AAAA;AAAA;AAAA;AAAA,wDAAA,CAAA;AAAA,MAK5B,CAAC,KAAK,KAAK;AAAA,KACb;AACA,IAAA,OAAO,IAAI,GAAA;AAAA,MACT,IAAA,CAAK,GAAA,CAAI,CAAC,GAAA,KAAkC,CAAC,GAAA,CAAI,CAAA,EAAG,MAAA,CAAO,GAAA,CAAI,CAAC,CAAC,CAAC;AAAA,KACpE;AAAA,EACF;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,MAAM,KAAA,CAAM,KAAA,EAAiB,KAAA,EAAkC;AAC7D,IAAA,IAAI,KAAA,CAAM,WAAW,CAAA,EAAG;AACtB,MAAA,OAAO,EAAC;AAAA,IACV;AACA,IAAA,MAAM,IAAA,GAAO,CAAA,yBAAA,CAAA;AACb,IAAA,OAAO,IAAA,CAAK,EAAA,CAAG,WAAA,CAAY,OAAM,EAAA,KAAM;AACrC,MAAA,MAAM,EAAA,CAAG,GAAA;AAAA,QACP,CAAA,+DAAA,CAAA;AAAA,QACA,CAAC,MAAA,CAAO,eAAe,CAAC;AAAA,OAC1B;AACA,MAAA,MAAM,EAAE,IAAA,EAAK,GAAI,MAAM,EAAA,CAAG,GAAA;AAAA,QACxB,kBAAkB,MAAM;AAAA,gBAAA,EACd,KAAA,CAAM,IAAI,MAAM,CAAA,KAAA,EAAQ,IAAI,CAAA,CAAE,CAAA,CAAE,IAAA,CAAK,MAAM,CAAC;AAAA,mBAAA,EACzC,KAAA,CACR,IAAI,MAAM,CAAA,mBAAA,EAAsB,IAAI,CAAA,CAAA,CAAG,CAAA,CACvC,IAAA,CAAK,KAAK,CAAC,CAAA;AAAA,iBAAA,CAAA;AAAA,QAEhB,CAAC,GAAG,KAAA,EAAO,GAAG,OAAO,KAAK;AAAA,OAC5B;AACA,MAAA,OAAO,IAAA,CAAK,GAAA,CAAI,CAAC,GAAA,KAAwB,IAAI,EAAE,CAAA;AAAA,IACjD,CAAC,CAAA;AAAA,EACH;AAAA,EAEA,MAAM,OAAO,GAAA,EAAuC;AAClD,IAAA,IAAI,GAAA,CAAI,WAAW,CAAA,EAAG;AACpB,MAAA,OAAO,EAAC;AAAA,IACV;AACA,IAAA,MAAM,OAAO,MAAM,IAAA,CAAK,GAAG,MAAM,CAAA,CAC9B,OAAO,IAAA,EAAM,QAAA,EAAU,YAAA,EAAc,UAAA,EAAY,SAAS,SAAS,CAAA,CACnE,SAAS,qBAAA,EAAuB,CAAC,GAAG,CAAC,CAAA;AACxC,IAAA,OAAO,IAAA,CAAK,IAAI,CAAA,GAAA,MAAQ;AAAA,MACtB,IAAI,GAAA,CAAI,EAAA;AAAA,MACR,QAAQ,GAAA,CAAI,MAAA;AAAA,MACZ,WAAW,GAAA,CAAI,UAAA;AAAA,MACf,UAAU,GAAA,CAAI,QAAA;AAAA,MACd,OAAO,GAAA,CAAI,KAAA;AAAA,MACX,SAAS,GAAA,CAAI;AAAA,KACf,CAAE,CAAA;AAAA,EACJ;AACF;AAEA,SAAS,cAAc,SAAA,EAA6B;AAClD,EAAA,OAAO,CAAA,CAAA,EAAI,SAAA,CAAU,IAAA,CAAK,GAAG,CAAC,CAAA,CAAA,CAAA;AAChC;;;;;"}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@drewswiredin/backstage-plugin-assistants-backend-module-semantic-search",
-  "version": "0.1.2",
+  "version": "0.1.3",
   "description": "Knowledge index for AI Assistants for Backstage: semantic, fuzzy, and keyword search over TechDocs and the catalog in Postgres.",
   "license": "Apache-2.0",
   "publishConfig": {