@drewswiredin/backstage-plugin-assistants-backend-module-semantic-search 0.1.4 → 0.2.0

package/README.md

@@ -1,16 +1,24 @@
 # @drewswiredin/backstage-plugin-assistants-backend-module-semantic-search
 
-Knowledge index for AI Assistants for Backstage. It embeds every published
-TechDocs section and every catalog entity except `Location` into the assistants plugin's Postgres database and
-gives assistants a `search-knowledge` tool that finds them by meaning, by near
-spelling, and by exact words:
+Knowledge index for AI Assistants for Backstage. It copies everything your
+Backstage search indexes (TechDocs, the catalog, and any other source a
+collator brings in, such as Confluence or a wiki), embeds it into the
+assistants plugin's Postgres database, and gives assistants a
+`search-knowledge` tool that finds it by meaning, by near spelling, and by
+exact words:
 
 - "how do I fix errored-out messages" finds _Dead-letter handling › Replaying
   dead-lettered messages_.
 - "Pilon deployment" finds the _Pylon Gateway › Deployment runbook_.
 - "dlq-replay" finds the page that names the job.
 
-Results are filtered to entities the user can read in the catalog.
+It reads no source itself. Each time a search collator finishes a run, the
+module copies that run's documents, so the index follows your search schedule
+and a rate-limited source such as Confluence is read once, by search.
+
+Catalog and TechDocs results are filtered to entities the user can read in the
+catalog. Results from other sources are visible to every assistant user, as
+they are in Backstage search.
 
 Adds to the backend plugin
 [`@drewswiredin/backstage-plugin-assistants-backend`](https://www.npmjs.com/package/@drewswiredin/backstage-plugin-assistants-backend).
@@ -18,6 +26,8 @@ Adds to the backend plugin
 ## Requirements
 
 - The assistants backend plugin, with its database on Postgres 13 or later.
+- Backstage search (`@backstage/plugin-search-backend`) in the same backend,
+  with the collators for the sources you want: any search engine works.
 - The `vector` ([pgvector](https://github.com/pgvector/pgvector)) and
   `pg_trgm` extensions in the assistants plugin's database.
 - An embedding model from an `openai`, `azure`, or `openai-compatible`
@@ -34,6 +44,7 @@ yarn --cwd packages/backend add @drewswiredin/backstage-plugin-assistants-backen
 ```ts
 // packages/backend/src/index.ts
 backend.add(import('@drewswiredin/backstage-plugin-assistants-backend'));
+backend.add(import('@backstage/plugin-search-backend'));
 backend.add(
   import(
     '@drewswiredin/backstage-plugin-assistants-backend-module-semantic-search'
@@ -41,6 +52,13 @@ backend.add(
 );
 ```
 
+The one `backend.add` installs two modules: the index and `search-knowledge`
+under the `assistants` plugin, and a document decorator under the `search`
+plugin that copies what each search run collected. To add them separately,
+import `assistantsModuleSemanticSearch` and `searchModuleAssistantsKnowledge`
+by name. Without the search plugin the module logs a warning at startup and
+the index stays empty.
+
 ## Enable the extensions
 
 The module enables both extensions itself when its database role may. `pg_trgm`
@@ -82,14 +100,18 @@ assistants:
 On Azure OpenAI, deploy an embedding model and name the deployment:
 `embeddingModel: azure:<embedding-deployment>`.
 
-The first run embeds everything; later runs embed only new and changed
-sections and delete removed ones. Changing `embeddingModel` rebuilds the index.
+The index fills as search runs: each document type arrives when its collator
+next runs, on the schedule you gave that collator. A source's first run embeds
+all of it; later runs embed only new and changed sections and delete removed
+ones. Changing `embeddingModel` empties the index, and each type refills on its
+next search run.
 
 ## Give an assistant the tool
 
 Add `search-knowledge` to the assistant's tools in the assistant editor. It
 pairs with `read-techdocs` (from `builtinActions: true`), which opens the full
-page behind a TechDocs result. Until the first index run finishes, the tool
+page behind a TechDocs result; other results carry a link. Until the first
+search run is copied, the tool
 answers with an error that points the model at keyword search: Backstage's
 `query` action when the assistant has it, otherwise `search-techdocs` and
 `search-catalog`.
@@ -113,13 +135,27 @@ backend:
 
 ## How it works
 
-**Indexing.** A scheduled task (one replica at a time) reads each published
-TechDocs site's MkDocs search index (the same source Backstage's TechDocs
-search uses) and every catalog entity except `Location`. Each TechDocs section
-becomes one or more chunks of up to ~1600 characters, titled with its heading
-path (`Site › Page › Section`) and embedded with that title. Each row stores
-the embedding, a generated full-text vector, and a trigram index over its text.
-The table is a rebuildable cache, not migrated data.
+**Indexing.** A search decorator sits on every document type's pipeline. It
+passes each document on to the search engine unchanged and keeps it; when the
+collator finishes, it hands the run's documents to the index. A failed run
+hands over nothing, and a run with no documents changes nothing. Runs are
+copied one at a time, in the backend that ran the collator.
+
+Each document becomes one or more chunks of up to ~1600 characters, titled
+with its heading path (`Site › Page › Section` for TechDocs) and embedded with
+that title. The facts collators attach (entity ref, type, owner, lifecycle)
+join the text, so a catalog entity with no description is still found. A
+document gated by a catalog entity (its `authorization.resourceRef`) keeps that
+entity for the permission check; a document gated by any other kind of
+resource is skipped, since the index can only check entity read permission.
+
+A run does not say which document type it is, so the index recognizes a source
+by its locations: the run joins the stored collection that already holds most
+of its locations, and that collection's chunks the run no longer has are
+deleted. A collection no search run has refreshed in 30 days (a source removed
+from search, or one whose locations all changed at once) is dropped. Each row stores the embedding, a generated full-text vector, and a
+trigram index over its text. The table is a rebuildable copy, not migrated
+data.
 
 **Retrieval.** `search-knowledge` takes the question and up to three alternative
 phrasings and builds one semantic list per phrasing, a keyword list, a rare-word
@@ -139,14 +175,17 @@ without outvoting an exact match. The rare-word list keeps a name or acronym
 mentioned once in a paragraph about something else from being outranked by
 sections that are only near the question in meaning. The top candidates'
 entities are fetched from the catalog as the user; results for entities the
-user cannot read are dropped, and repeated text is shown once.
+user cannot read are dropped, results with no entity are kept, and repeated
+text is shown once.
 
 ## Reference
 
-| Key                                        | Default                                              | Meaning                                                                                       |
-| ------------------------------------------ | ---------------------------------------------------- | --------------------------------------------------------------------------------------------- |
-| `assistants.semanticSearch.embeddingModel` | (required)                                           | `<providerId>:<model>` from `assistants.providers`.                                           |
-| `assistants.semanticSearch.schedule`       | every 30 minutes, 1 hour timeout, 15 s initial delay | Backstage task schedule for refreshing the index; a configured schedule replaces the default. |
+| Key                                        | Default    | Meaning                                             |
+| ------------------------------------------ | ---------- | --------------------------------------------------- |
+| `assistants.semanticSearch.embeddingModel` | (required) | `<providerId>:<model>` from `assistants.providers`. |
+
+The index refreshes when search does; set each collator's schedule in its own
+config.
 
 ## License
 

package/config.schema.json

@@ -11,168 +11,12 @@
             "embeddingModel": {
               "type": "string",
               "description": "Embedding model in `<providerId>:<model>` format, resolved against `assistants.providers`, e.g. `openai:text-embedding-3-small`. The provider must be `openai`, `azure`, or `openai-compatible`. Changing it rebuilds the index."
-            },
-            "schedule": {
-              "type": "object",
-              "properties": {
-                "frequency": {
-                  "anyOf": [
-                    {
-                      "type": "object",
-                      "properties": {
-                        "cron": {
-                          "type": "string",
-                          "description": "A crontab style string."
-                        }
-                      },
-                      "required": [
-                        "cron"
-                      ]
-                    },
-                    {
-                      "type": "string"
-                    },
-                    {
-                      "type": "object",
-                      "properties": {
-                        "years": {
-                          "type": "number"
-                        },
-                        "months": {
-                          "type": "number"
-                        },
-                        "weeks": {
-                          "type": "number"
-                        },
-                        "days": {
-                          "type": "number"
-                        },
-                        "hours": {
-                          "type": "number"
-                        },
-                        "minutes": {
-                          "type": "number"
-                        },
-                        "seconds": {
-                          "type": "number"
-                        },
-                        "milliseconds": {
-                          "type": "number"
-                        }
-                      },
-                      "description": "Human friendly durations object."
-                    },
-                    {
-                      "type": "object",
-                      "properties": {
-                        "trigger": {
-                          "type": "string",
-                          "const": "manual"
-                        }
-                      },
-                      "required": [
-                        "trigger"
-                      ]
-                    }
-                  ],
-                  "description": "How often you want the task to run. The system does its best to avoid overlapping invocations."
-                },
-                "timeout": {
-                  "anyOf": [
-                    {
-                      "type": "string"
-                    },
-                    {
-                      "type": "object",
-                      "properties": {
-                        "years": {
-                          "type": "number"
-                        },
-                        "months": {
-                          "type": "number"
-                        },
-                        "weeks": {
-                          "type": "number"
-                        },
-                        "days": {
-                          "type": "number"
-                        },
-                        "hours": {
-                          "type": "number"
-                        },
-                        "minutes": {
-                          "type": "number"
-                        },
-                        "seconds": {
-                          "type": "number"
-                        },
-                        "milliseconds": {
-                          "type": "number"
-                        }
-                      },
-                      "description": "Human friendly durations object."
-                    }
-                  ],
-                  "description": "The maximum amount of time that a single task invocation can take, before it's considered timed out and gets \"released\" such that a new invocation is permitted to take place (possibly, then, on a different worker)."
-                },
-                "initialDelay": {
-                  "anyOf": [
-                    {
-                      "type": "string"
-                    },
-                    {
-                      "type": "object",
-                      "properties": {
-                        "years": {
-                          "type": "number"
-                        },
-                        "months": {
-                          "type": "number"
-                        },
-                        "weeks": {
-                          "type": "number"
-                        },
-                        "days": {
-                          "type": "number"
-                        },
-                        "hours": {
-                          "type": "number"
-                        },
-                        "minutes": {
-                          "type": "number"
-                        },
-                        "seconds": {
-                          "type": "number"
-                        },
-                        "milliseconds": {
-                          "type": "number"
-                        }
-                      },
-                      "description": "Human friendly durations object."
-                    }
-                  ],
-                  "description": "The amount of time that should pass before the first invocation happens."
-                },
-                "scope": {
-                  "type": "string",
-                  "enum": [
-                    "global",
-                    "local"
-                  ],
-                  "description": "Sets the scope of concurrency control / locking to apply for invocations of this task."
-                }
-              },
-              "required": [
-                "frequency",
-                "timeout"
-              ],
-              "description": "How often the index refreshes. Each run embeds only new and changed content. Default: every 30 minutes, 1 hour timeout, 15 s initial delay; a configured schedule replaces it."
             }
           },
           "required": [
             "embeddingModel"
           ],
-          "description": "The knowledge index: published TechDocs sections and catalog entities (except `Location`) embedded into the assistants plugin's Postgres database and served to assistants as the `search-knowledge` action. Requires the `vector` and `pg_trgm` extensions."
+          "description": "The knowledge index: a copy of everything Backstage search indexes (TechDocs, the catalog, and any other collator's documents), embedded into the assistants plugin's Postgres database as each search run finishes and served to assistants as the `search-knowledge` action. Requires the search plugin in the same backend, and the `vector` and `pg_trgm` extensions."
         }
       }
     }

package/dist/action.cjs.js

@@ -18,7 +18,7 @@ function registerSearchKnowledgeAction({
     name: "search-knowledge",
     title: "Search Knowledge",
     attributes: { destructive: false, readOnly: true, idempotent: true },
-    description: `Search the organization's TechDocs and software catalog by meaning, not only by exact words. Best for questions in the user's own words: it finds a page even when the question is worded differently from it ("fix errored messages" finds "Retry dead-letter messages") and tolerates misspelled names ("Pilon" finds "Pylon"). It returns the most relevant sections, not every match, and cannot filter by entity kind.
+    description: `Search everything the organization's Backstage search indexes (TechDocs, the software catalog, and any other source it collects, such as Confluence or wiki pages) by meaning, not only by exact words. Best for questions in the user's own words: it finds a page even when the question is worded differently from it ("fix errored messages" finds "Retry dead-letter messages") and tolerates misspelled names ("Pilon" finds "Pylon"). It returns the most relevant sections, not every match, and cannot filter by entity kind or source.
 
 How to search the organization's docs and catalog:
 1. Start here, with the user's question.
@@ -27,7 +27,7 @@ How to search the organization's docs and catalog:
 
 Pass the user's question as the "query" parameter. Pass up to three "alternatives": other phrasings, synonyms, or the internal terms the docs might use.
 
-Each result has a title (its heading path), an excerpt, and a location. For TechDocs results, pass the location to read-techdocs to read the full page. For catalog results, the location is the entity ref.`,
+Each result has a title (its heading path), an excerpt, a location, and, for catalog and TechDocs results, the entity ref. A location starting with /docs/ is a TechDocs page: pass it to read-techdocs to read the full page. Any other location is a link to the page or entity; cite it.`,
     schema: {
       input: (z) => z.object({
         query: z.string().min(1).describe("The question or topic to find"),
@@ -37,10 +37,9 @@ Each result has a title (its heading path), an excerpt, and a location. For Tech
       output: (z) => z.object({
         results: z.array(
           z.object({
-            source: z.enum(["techdocs", "catalog"]),
             title: z.string(),
             location: z.string(),
-            entityRef: z.string(),
+            entityRef: z.string().optional(),
             excerpt: z.string()
           })
         )
@@ -79,22 +78,25 @@ Each result has a title (its heading path), an excerpt, and a location. For Tech
       );
       const candidates = ranked.map((id) => byId.get(id)).filter((chunk) => Boolean(chunk));
       const readable = await readableEntityRefs(catalog, auth, credentials, [
-        ...new Set(candidates.map((chunk) => chunk.entityRef))
+        ...new Set(
+          candidates.flatMap(
+            (chunk) => chunk.entityRef ? [chunk.entityRef] : []
+          )
+        )
       ]);
       const seen = /* @__PURE__ */ new Set();
       const results = [];
       for (const chunk of candidates) {
         const key = `${chunk.location}|${chunk.title}`;
-        if (!readable.has(chunk.entityRef) || seen.has(key) || seen.has(chunk.content)) {
+        if (chunk.entityRef && !readable.has(chunk.entityRef) || seen.has(key) || seen.has(chunk.content)) {
           continue;
         }
         seen.add(key);
         seen.add(chunk.content);
         results.push({
-          source: chunk.source,
           title: chunk.title,
           location: chunk.location,
-          entityRef: chunk.entityRef,
+          ...chunk.entityRef ? { entityRef: chunk.entityRef } : {},
           excerpt: chunk.content.slice(0, EXCERPT_CHARS)
         });
         if (results.length === input.limit) {

package/dist/action.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"action.cjs.js","sources":["../src/action.ts"],"sourcesContent":["import type {\n  AuthService,\n  BackstageCredentials,\n} from '@backstage/backend-plugin-api';\nimport type { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport type { CatalogApi } from '@backstage/catalog-client';\nimport { embedMany, type EmbeddingModel } from 'ai';\nimport { fuseRankings, joinedWords, queryWords, type RankedList } from './rank';\nimport type { KnowledgeStore, StoredChunk } from './store';\n\n/** Candidates per retrieval list, before fusion and permission filtering. */\nconst CANDIDATES = 30;\n/** Fused candidates checked for permission; over-fetched so filtering rarely empties the page. */\nconst FUSED = 40;\nconst EXCERPT_CHARS = 1500;\n/** A word in this many chunks or fewer is rare: strong evidence of what the user means. */\nconst RARE_MAX_CHUNKS = 3;\n\nexport interface SearchKnowledgeDeps {\n  actionsRegistry: ActionsRegistryService;\n  store: KnowledgeStore;\n  catalog: CatalogApi;\n  auth: AuthService;\n  embeddingModel: EmbeddingModel;\n}\n\n/**\n * Registers `search-knowledge`: hybrid retrieval over the knowledge index. Each\n * phrasing is embedded and matched by meaning; the query words are matched\n * exactly (full-text, any word) and, for words absent from the index, by\n * near spelling (trigram). Words found in only a few chunks get a list of\n * their own, so a rare name or acronym buried in a paragraph about something\n * else is not outvoted by sections that are merely near the question.\n *\n * The lists are fused by reciprocal rank with one vote per method: the\n * phrasings share the meaning vote, so extra alternatives broaden recall\n * without drowning out exact matches. Results are filtered to entities the\n * caller can read in the catalog, and repeated text is shown once.\n */\nexport function registerSearchKnowledgeAction({\n  actionsRegistry,\n  store,\n  catalog,\n  auth,\n  embeddingModel,\n}: SearchKnowledgeDeps): void {\n  actionsRegistry.register({\n    name: 'search-knowledge',\n    title: 'Search Knowledge',\n    attributes: { destructive: false, readOnly: true, idempotent: true },\n    description: `Search the organization's TechDocs and software catalog by meaning, not only by exact words. Best for questions in the user's own words: it finds a page even when the question is worded differently from it (\"fix errored messages\" finds \"Retry dead-letter messages\") and tolerates misspelled names (\"Pilon\" finds \"Pylon\"). It returns the most relevant sections, not every match, and cannot filter by entity kind.\n\nHow to search the organization's docs and catalog:\n1. Start here, with the user's question.\n2. If the results don't clearly answer, also run a keyword search: the query tool (catalog and TechDocs at once) when it is available, otherwise search-techdocs and search-catalog. Keyword search finds rare names, acronyms and jargon that ranking by meaning can miss. For a bare name, term or acronym, run both searches from the start.\n3. Report something as missing only after both searches come back empty.\n\nPass the user's question as the \"query\" parameter. Pass up to three \"alternatives\": other phrasings, synonyms, or the internal terms the docs might use.\n\nEach result has a title (its heading path), an excerpt, and a location. For TechDocs results, pass the location to read-techdocs to read the full page. For catalog results, the location is the entity ref.`,\n    schema: {\n      input: z =>\n        z.object({\n          query: z.string().min(1).describe('The question or topic to find'),\n          alternatives: z\n            .array(z.string())\n            .max(3)\n            .optional()\n            .describe('Other phrasings or terms for the same thing'),\n          limit: z\n            .number()\n            .int()\n            .min(1)\n            .max(20)\n            .optional()\n            .default(8)\n            .describe('Number of results to return'),\n        }),\n      output: z =>\n        z.object({\n          results: z.array(\n            z.object({\n              source: z.enum(['techdocs', 'catalog']),\n              title: z.string(),\n              location: z.string(),\n              entityRef: z.string(),\n              excerpt: z.string(),\n            }),\n          ),\n        }),\n    },\n    action: async ({ input, credentials }) => {\n      if (!(await store.isReady())) {\n        throw new Error(\n          'The knowledge index is still being built; meanwhile use keyword search: the query tool when it is available, otherwise search-techdocs and search-catalog.',\n        );\n      }\n\n      const phrasings = [input.query, ...(input.alternatives ?? [])];\n      const words = queryWords(phrasings);\n      const joined = joinedWords(phrasings).filter(w => !words.includes(w));\n      const [{ embeddings }, frequencies] = await Promise.all([\n        embedMany({ model: embeddingModel, values: phrasings }),\n        store.documentFrequencies([...words, ...joined], RARE_MAX_CHUNKS + 1),\n      ]);\n      const known = [...words, ...joined].filter(\n        w => (frequencies.get(w) ?? 0) > 0,\n      );\n      const rare = known.filter(w => frequencies.get(w)! <= RARE_MAX_CHUNKS);\n      const unknown = words.filter(w => frequencies.get(w) === 0);\n\n      const lists: RankedList[] = await Promise.all([\n        ...embeddings.map(async embedding => ({\n          ids: await store.nearest(embedding, CANDIDATES),\n          weight: 1 / embeddings.length,\n        })),\n        store.keyword(known, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n        store.keyword(rare, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n        store.fuzzy(unknown, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n      ]);\n\n      const ranked = fuseRankings(lists).slice(0, FUSED);\n      const byId = new Map(\n        (await store.chunks(ranked)).map(chunk => [chunk.id, chunk]),\n      );\n      const candidates = ranked\n        .map(id => byId.get(id))\n        .filter((chunk): chunk is StoredChunk => Boolean(chunk));\n\n      const readable = await readableEntityRefs(catalog, auth, credentials, [\n        ...new Set(candidates.map(chunk => chunk.entityRef)),\n      ]);\n\n      const seen = new Set<string>();\n      const results = [];\n      for (const chunk of candidates) {\n        const key = `${chunk.location}|${chunk.title}`;\n        if (\n          !readable.has(chunk.entityRef) ||\n          seen.has(key) ||\n          seen.has(chunk.content)\n        ) {\n          continue;\n        }\n        seen.add(key);\n        seen.add(chunk.content);\n        results.push({\n          source: chunk.source,\n          title: chunk.title,\n          location: chunk.location,\n          entityRef: chunk.entityRef,\n          excerpt: chunk.content.slice(0, EXCERPT_CHARS),\n        });\n        if (results.length === input.limit) {\n          break;\n        }\n      }\n      return { output: { results } };\n    },\n  });\n}\n\n/**\n * The subset of entity refs the caller can read, asked of the catalog as the\n * caller — so the catalog's own permission policy decides, and a deleted\n * entity's leftover chunks never surface.\n */\nasync function readableEntityRefs(\n  catalog: CatalogApi,\n  auth: AuthService,\n  credentials: BackstageCredentials,\n  entityRefs: string[],\n): Promise<Set<string>> {\n  if (entityRefs.length === 0) {\n    return new Set();\n  }\n  const { token } = await auth.getPluginRequestToken({\n    onBehalfOf: credentials,\n    targetPluginId: 'catalog',\n  });\n  const { items } = await catalog.getEntitiesByRefs(\n    { entityRefs, fields: ['kind', 'metadata.namespace', 'metadata.name'] },\n    { token },\n  );\n  return new Set(entityRefs.filter((_, i) => items[i]));\n}\n"],"names":["queryWords","joinedWords","embedMany","fuseRankings"],"mappings":";;;;;AAWA,MAAM,UAAA,GAAa,EAAA;AAEnB,MAAM,KAAA,GAAQ,EAAA;AACd,MAAM,aAAA,GAAgB,IAAA;AAEtB,MAAM,eAAA,GAAkB,CAAA;AAuBjB,SAAS,6BAAA,CAA8B;AAAA,EAC5C,eAAA;AAAA,EACA,KAAA;AAAA,EACA,OAAA;AAAA,EACA,IAAA;AAAA,EACA;AACF,CAAA,EAA8B;AAC5B,EAAA,eAAA,CAAgB,QAAA,CAAS;AAAA,IACvB,IAAA,EAAM,kBAAA;AAAA,IACN,KAAA,EAAO,kBAAA;AAAA,IACP,YAAY,EAAE,WAAA,EAAa,OAAO,QAAA,EAAU,IAAA,EAAM,YAAY,IAAA,EAAK;AAAA,IACnE,WAAA,EAAa,CAAA;;AAAA;AAAA;AAAA;AAAA;;AAAA;;AAAA,4MAAA,CAAA;AAAA,IAUb,MAAA,EAAQ;AAAA,MACN,KAAA,EAAO,CAAA,CAAA,KACL,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,KAAA,EAAO,EAAE,MAAA,EAAO,CAAE,IAAI,CAAC,CAAA,CAAE,SAAS,+BAA+B,CAAA;AAAA,QACjE,YAAA,EAAc,CAAA,CACX,KAAA,CAAM,CAAA,CAAE,MAAA,EAAQ,CAAA,CAChB,GAAA,CAAI,CAAC,CAAA,CACL,QAAA,EAAS,CACT,SAAS,6CAA6C,CAAA;AAAA,QACzD,OAAO,CAAA,CACJ,MAAA,GACA,GAAA,EAAI,CACJ,IAAI,CAAC,CAAA,CACL,GAAA,CAAI,EAAE,EACN,QAAA,EAAS,CACT,QAAQ,CAAC,CAAA,CACT,SAAS,6BAA6B;AAAA,OAC1C,CAAA;AAAA,MACH,MAAA,EAAQ,CAAA,CAAA,KACN,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,SAAS,CAAA,CAAE,KAAA;AAAA,UACT,EAAE,MAAA,CAAO;AAAA,YACP,QAAQ,CAAA,CAAE,IAAA,CAAK,CAAC,UAAA,EAAY,SAAS,CAAC,CAAA;AAAA,YACtC,KAAA,EAAO,EAAE,MAAA,EAAO;AAAA,YAChB,QAAA,EAAU,EAAE,MAAA,EAAO;AAAA,YACnB,SAAA,EAAW,EAAE,MAAA,EAAO;AAAA,YACpB,OAAA,EAAS,EAAE,MAAA;AAAO,WACnB;AAAA;AACH,OACD;AAAA,KACL;AAAA,IACA,MAAA,EAAQ,OAAO,EAAE,KAAA,EAAO,aAAY,KAAM;AACxC,MAAA,IAAI,CAAE,MAAM,KAAA,CAAM,OAAA,EAAQ,EAAI;AAC5B,QAAA,MAAM,IAAI,KAAA;AAAA,UACR;AAAA,SACF;AAAA,MACF;AAEA,MAAA,MAAM,SAAA,GAAY,CAAC,KAAA,CAAM,KAAA,EAAO,GAAI,KAAA,CAAM,YAAA,IAAgB,EAAG,CAAA;AAC7D,MAAA,MAAM,KAAA,GAAQA,gBAAW,SAAS,CAAA;AAClC,MAAA,MAAM,MAAA,GAASC,gBAAA,CAAY,SAAS,CAAA,CAAE,MAAA,CAAO,OAAK,CAAC,KAAA,CAAM,QAAA,CAAS,CAAC,CAAC,CAAA;AACpE,MAAA,MAAM,CAAC,EAAE,UAAA,EAAW,EAAG,WAAW,CAAA,GAAI,MAAM,QAAQ,GAAA,CAAI;AAAA,QACtDC,aAAU,EAAE,KAAA,EAAO,cAAA,EAAgB,MAAA,EAAQ,WAAW,CAAA;AAAA,QACtD,KAAA,CAAM,oBAAoB,CAAC,GAAG,OAAO,GAAG,MAAM,CAAA,EAAG,eAAA,GAAkB,CAAC;AAAA,OACrE,CAAA;AACD,MAAA,MAAM,QAAQ,CAAC,GAAG,KAAA,EAAO,GAAG,MAAM,CAAA,CAAE,MAAA;AAAA,QAClC,CAAA,CAAA,KAAA,CAAM,WAAA,CAAY,GAAA,CAAI,CAAC,KAAK,CAAA,IAAK;AAAA,OACnC;AACA,MAAA,MAAM,IAAA,GAAO,MAAM,MAAA,CAAO,CAAA,CAAA,KAAK,YAAY,GAAA,CAAI,CAAC,KAAM,eAAe,CAAA;AACrE,MAAA,MAAM,OAAA,GAAU,MAAM,MAAA,CAAO,CAAA,CAAA,KAAK,YAAY,GAAA,CAAI,CAAC,MAAM,CAAC,CAAA;AAE1D,MAAA,MAAM,KAAA,GAAsB,MAAM,OAAA,CAAQ,GAAA,CAAI;AAAA,QAC5C,GAAG,UAAA,CAAW,GAAA,CAAI,OAAM,SAAA,MAAc;AAAA,UACpC,GAAA,EAAK,MAAM,KAAA,CAAM,OAAA,CAAQ,WAAW,UAAU,CAAA;AAAA,UAC9C,MAAA,EAAQ,IAAI,UAAA,CAAW;AAAA,SACzB,CAAE,CAAA;AAAA,QACF,KAAA,CAAM,OAAA,CAAQ,KAAA,EAAO,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE,CAAA;AAAA,QACjE,KAAA,CAAM,OAAA,CAAQ,IAAA,EAAM,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE,CAAA;AAAA,QAChE,KAAA,CAAM,KAAA,CAAM,OAAA,EAAS,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE;AAAA,OAClE,CAAA;AAED,MAAA,MAAM,SAASC,iBAAA,CAAa,KAAK,CAAA,CAAE,KAAA,CAAM,GAAG,KAAK,CAAA;AACjD,MAAA,MAAM,OAAO,IAAI,GAAA;AAAA,QAAA,CACd,MAAM,KAAA,CAAM,MAAA,CAAO,MAAM,CAAA,EAAG,GAAA,CAAI,CAAA,KAAA,KAAS,CAAC,KAAA,CAAM,EAAA,EAAI,KAAK,CAAC;AAAA,OAC7D;AACA,MAAA,MAAM,UAAA,GAAa,MAAA,CAChB,GAAA,CAAI,CAAA,EAAA,KAAM,KAAK,GAAA,CAAI,EAAE,CAAC,CAAA,CACtB,MAAA,CAAO,CAAC,KAAA,KAAgC,OAAA,CAAQ,KAAK,CAAC,CAAA;AAEzD,MAAA,MAAM,QAAA,GAAW,MAAM,kBAAA,CAAmB,OAAA,EAAS,MAAM,WAAA,EAAa;AAAA,QACpE,GAAG,IAAI,GAAA,CAAI,UAAA,CAAW,IAAI,CAAA,KAAA,KAAS,KAAA,CAAM,SAAS,CAAC;AAAA,OACpD,CAAA;AAED,MAAA,MAAM,IAAA,uBAAW,GAAA,EAAY;AAC7B,MAAA,MAAM,UAAU,EAAC;AACjB,MAAA,KAAA,MAAW,SAAS,UAAA,EAAY;AAC9B,QAAA,MAAM,MAAM,CAAA,EAAG,KAAA,CAAM,QAAQ,CAAA,CAAA,EAAI,MAAM,KAAK,CAAA,CAAA;AAC5C,QAAA,IACE,CAAC,QAAA,CAAS,GAAA,CAAI,KAAA,CAAM,SAAS,CAAA,IAC7B,IAAA,CAAK,GAAA,CAAI,GAAG,CAAA,IACZ,IAAA,CAAK,GAAA,CAAI,KAAA,CAAM,OAAO,CAAA,EACtB;AACA,UAAA;AAAA,QACF;AACA,QAAA,IAAA,CAAK,IAAI,GAAG,CAAA;AACZ,QAAA,IAAA,CAAK,GAAA,CAAI,MAAM,OAAO,CAAA;AACtB,QAAA,OAAA,CAAQ,IAAA,CAAK;AAAA,UACX,QAAQ,KAAA,CAAM,MAAA;AAAA,UACd,OAAO,KAAA,CAAM,KAAA;AAAA,UACb,UAAU,KAAA,CAAM,QAAA;AAAA,UAChB,WAAW,KAAA,CAAM,SAAA;AAAA,UACjB,OAAA,EAAS,KAAA,CAAM,OAAA,CAAQ,KAAA,CAAM,GAAG,aAAa;AAAA,SAC9C,CAAA;AACD,QAAA,IAAI,OAAA,CAAQ,MAAA,KAAW,KAAA,CAAM,KAAA,EAAO;AAClC,UAAA;AAAA,QACF;AAAA,MACF;AACA,MAAA,OAAO,EAAE,MAAA,EAAQ,EAAE,OAAA,EAAQ,EAAE;AAAA,IAC/B;AAAA,GACD,CAAA;AACH;AAOA,eAAe,kBAAA,CACb,OAAA,EACA,IAAA,EACA,WAAA,EACA,UAAA,EACsB;AACtB,EAAA,IAAI,UAAA,CAAW,WAAW,CAAA,EAAG;AAC3B,IAAA,2BAAW,GAAA,EAAI;AAAA,EACjB;AACA,EAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,KAAK,qBAAA,CAAsB;AAAA,IACjD,UAAA,EAAY,WAAA;AAAA,IACZ,cAAA,EAAgB;AAAA,GACjB,CAAA;AACD,EAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,OAAA,CAAQ,iBAAA;AAAA,IAC9B,EAAE,UAAA,EAAY,MAAA,EAAQ,CAAC,MAAA,EAAQ,oBAAA,EAAsB,eAAe,CAAA,EAAE;AAAA,IACtE,EAAE,KAAA;AAAM,GACV;AACA,EAAA,OAAO,IAAI,GAAA,CAAI,UAAA,CAAW,MAAA,CAAO,CAAC,GAAG,CAAA,KAAM,KAAA,CAAM,CAAC,CAAC,CAAC,CAAA;AACtD;;;;"}
+{"version":3,"file":"action.cjs.js","sources":["../src/action.ts"],"sourcesContent":["import type {\n  AuthService,\n  BackstageCredentials,\n} from '@backstage/backend-plugin-api';\nimport type { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport type { CatalogApi } from '@backstage/catalog-client';\nimport { embedMany, type EmbeddingModel } from 'ai';\nimport { fuseRankings, joinedWords, queryWords, type RankedList } from './rank';\nimport type { KnowledgeStore, StoredChunk } from './store';\n\n/** Candidates per retrieval list, before fusion and permission filtering. */\nconst CANDIDATES = 30;\n/** Fused candidates checked for permission; over-fetched so filtering rarely empties the page. */\nconst FUSED = 40;\nconst EXCERPT_CHARS = 1500;\n/** A word in this many chunks or fewer is rare: strong evidence of what the user means. */\nconst RARE_MAX_CHUNKS = 3;\n\nexport interface SearchKnowledgeDeps {\n  actionsRegistry: ActionsRegistryService;\n  store: KnowledgeStore;\n  catalog: CatalogApi;\n  auth: AuthService;\n  embeddingModel: EmbeddingModel;\n}\n\n/**\n * Registers `search-knowledge`: hybrid retrieval over the knowledge index. Each\n * phrasing is embedded and matched by meaning; the query words are matched\n * exactly (full-text, any word) and, for words absent from the index, by\n * near spelling (trigram). Words found in only a few chunks get a list of\n * their own, so a rare name or acronym buried in a paragraph about something\n * else is not outvoted by sections that are merely near the question.\n *\n * The lists are fused by reciprocal rank with one vote per method: the\n * phrasings share the meaning vote, so extra alternatives broaden recall\n * without drowning out exact matches. Results gated by a catalog entity are\n * filtered to entities the caller can read; the rest are visible to every\n * assistant user, as they are in Backstage search. Repeated text is shown\n * once.\n */\nexport function registerSearchKnowledgeAction({\n  actionsRegistry,\n  store,\n  catalog,\n  auth,\n  embeddingModel,\n}: SearchKnowledgeDeps): void {\n  actionsRegistry.register({\n    name: 'search-knowledge',\n    title: 'Search Knowledge',\n    attributes: { destructive: false, readOnly: true, idempotent: true },\n    description: `Search everything the organization's Backstage search indexes (TechDocs, the software catalog, and any other source it collects, such as Confluence or wiki pages) by meaning, not only by exact words. Best for questions in the user's own words: it finds a page even when the question is worded differently from it (\"fix errored messages\" finds \"Retry dead-letter messages\") and tolerates misspelled names (\"Pilon\" finds \"Pylon\"). It returns the most relevant sections, not every match, and cannot filter by entity kind or source.\n\nHow to search the organization's docs and catalog:\n1. Start here, with the user's question.\n2. If the results don't clearly answer, also run a keyword search: the query tool (catalog and TechDocs at once) when it is available, otherwise search-techdocs and search-catalog. Keyword search finds rare names, acronyms and jargon that ranking by meaning can miss. For a bare name, term or acronym, run both searches from the start.\n3. Report something as missing only after both searches come back empty.\n\nPass the user's question as the \"query\" parameter. Pass up to three \"alternatives\": other phrasings, synonyms, or the internal terms the docs might use.\n\nEach result has a title (its heading path), an excerpt, a location, and, for catalog and TechDocs results, the entity ref. A location starting with /docs/ is a TechDocs page: pass it to read-techdocs to read the full page. Any other location is a link to the page or entity; cite it.`,\n    schema: {\n      input: z =>\n        z.object({\n          query: z.string().min(1).describe('The question or topic to find'),\n          alternatives: z\n            .array(z.string())\n            .max(3)\n            .optional()\n            .describe('Other phrasings or terms for the same thing'),\n          limit: z\n            .number()\n            .int()\n            .min(1)\n            .max(20)\n            .optional()\n            .default(8)\n            .describe('Number of results to return'),\n        }),\n      output: z =>\n        z.object({\n          results: z.array(\n            z.object({\n              title: z.string(),\n              location: z.string(),\n              entityRef: z.string().optional(),\n              excerpt: z.string(),\n            }),\n          ),\n        }),\n    },\n    action: async ({ input, credentials }) => {\n      if (!(await store.isReady())) {\n        throw new Error(\n          'The knowledge index is still being built; meanwhile use keyword search: the query tool when it is available, otherwise search-techdocs and search-catalog.',\n        );\n      }\n\n      const phrasings = [input.query, ...(input.alternatives ?? [])];\n      const words = queryWords(phrasings);\n      const joined = joinedWords(phrasings).filter(w => !words.includes(w));\n      const [{ embeddings }, frequencies] = await Promise.all([\n        embedMany({ model: embeddingModel, values: phrasings }),\n        store.documentFrequencies([...words, ...joined], RARE_MAX_CHUNKS + 1),\n      ]);\n      const known = [...words, ...joined].filter(\n        w => (frequencies.get(w) ?? 0) > 0,\n      );\n      const rare = known.filter(w => frequencies.get(w)! <= RARE_MAX_CHUNKS);\n      const unknown = words.filter(w => frequencies.get(w) === 0);\n\n      const lists: RankedList[] = await Promise.all([\n        ...embeddings.map(async embedding => ({\n          ids: await store.nearest(embedding, CANDIDATES),\n          weight: 1 / embeddings.length,\n        })),\n        store.keyword(known, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n        store.keyword(rare, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n        store.fuzzy(unknown, CANDIDATES).then(ids => ({ ids, weight: 1 })),\n      ]);\n\n      const ranked = fuseRankings(lists).slice(0, FUSED);\n      const byId = new Map(\n        (await store.chunks(ranked)).map(chunk => [chunk.id, chunk]),\n      );\n      const candidates = ranked\n        .map(id => byId.get(id))\n        .filter((chunk): chunk is StoredChunk => Boolean(chunk));\n\n      const readable = await readableEntityRefs(catalog, auth, credentials, [\n        ...new Set(\n          candidates.flatMap(chunk =>\n            chunk.entityRef ? [chunk.entityRef] : [],\n          ),\n        ),\n      ]);\n\n      const seen = new Set<string>();\n      const results = [];\n      for (const chunk of candidates) {\n        const key = `${chunk.location}|${chunk.title}`;\n        if (\n          (chunk.entityRef && !readable.has(chunk.entityRef)) ||\n          seen.has(key) ||\n          seen.has(chunk.content)\n        ) {\n          continue;\n        }\n        seen.add(key);\n        seen.add(chunk.content);\n        results.push({\n          title: chunk.title,\n          location: chunk.location,\n          ...(chunk.entityRef ? { entityRef: chunk.entityRef } : {}),\n          excerpt: chunk.content.slice(0, EXCERPT_CHARS),\n        });\n        if (results.length === input.limit) {\n          break;\n        }\n      }\n      return { output: { results } };\n    },\n  });\n}\n\n/**\n * The subset of entity refs the caller can read, asked of the catalog as the\n * caller — so the catalog's own permission policy decides, and a deleted\n * entity's leftover chunks never surface.\n */\nasync function readableEntityRefs(\n  catalog: CatalogApi,\n  auth: AuthService,\n  credentials: BackstageCredentials,\n  entityRefs: string[],\n): Promise<Set<string>> {\n  if (entityRefs.length === 0) {\n    return new Set();\n  }\n  const { token } = await auth.getPluginRequestToken({\n    onBehalfOf: credentials,\n    targetPluginId: 'catalog',\n  });\n  const { items } = await catalog.getEntitiesByRefs(\n    { entityRefs, fields: ['kind', 'metadata.namespace', 'metadata.name'] },\n    { token },\n  );\n  return new Set(entityRefs.filter((_, i) => items[i]));\n}\n"],"names":["queryWords","joinedWords","embedMany","fuseRankings"],"mappings":";;;;;AAWA,MAAM,UAAA,GAAa,EAAA;AAEnB,MAAM,KAAA,GAAQ,EAAA;AACd,MAAM,aAAA,GAAgB,IAAA;AAEtB,MAAM,eAAA,GAAkB,CAAA;AAyBjB,SAAS,6BAAA,CAA8B;AAAA,EAC5C,eAAA;AAAA,EACA,KAAA;AAAA,EACA,OAAA;AAAA,EACA,IAAA;AAAA,EACA;AACF,CAAA,EAA8B;AAC5B,EAAA,eAAA,CAAgB,QAAA,CAAS;AAAA,IACvB,IAAA,EAAM,kBAAA;AAAA,IACN,KAAA,EAAO,kBAAA;AAAA,IACP,YAAY,EAAE,WAAA,EAAa,OAAO,QAAA,EAAU,IAAA,EAAM,YAAY,IAAA,EAAK;AAAA,IACnE,WAAA,EAAa,CAAA;;AAAA;AAAA;AAAA;AAAA;;AAAA;;AAAA,2RAAA,CAAA;AAAA,IAUb,MAAA,EAAQ;AAAA,MACN,KAAA,EAAO,CAAA,CAAA,KACL,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,KAAA,EAAO,EAAE,MAAA,EAAO,CAAE,IAAI,CAAC,CAAA,CAAE,SAAS,+BAA+B,CAAA;AAAA,QACjE,YAAA,EAAc,CAAA,CACX,KAAA,CAAM,CAAA,CAAE,MAAA,EAAQ,CAAA,CAChB,GAAA,CAAI,CAAC,CAAA,CACL,QAAA,EAAS,CACT,SAAS,6CAA6C,CAAA;AAAA,QACzD,OAAO,CAAA,CACJ,MAAA,GACA,GAAA,EAAI,CACJ,IAAI,CAAC,CAAA,CACL,GAAA,CAAI,EAAE,EACN,QAAA,EAAS,CACT,QAAQ,CAAC,CAAA,CACT,SAAS,6BAA6B;AAAA,OAC1C,CAAA;AAAA,MACH,MAAA,EAAQ,CAAA,CAAA,KACN,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,SAAS,CAAA,CAAE,KAAA;AAAA,UACT,EAAE,MAAA,CAAO;AAAA,YACP,KAAA,EAAO,EAAE,MAAA,EAAO;AAAA,YAChB,QAAA,EAAU,EAAE,MAAA,EAAO;AAAA,YACnB,SAAA,EAAW,CAAA,CAAE,MAAA,EAAO,CAAE,QAAA,EAAS;AAAA,YAC/B,OAAA,EAAS,EAAE,MAAA;AAAO,WACnB;AAAA;AACH,OACD;AAAA,KACL;AAAA,IACA,MAAA,EAAQ,OAAO,EAAE,KAAA,EAAO,aAAY,KAAM;AACxC,MAAA,IAAI,CAAE,MAAM,KAAA,CAAM,OAAA,EAAQ,EAAI;AAC5B,QAAA,MAAM,IAAI,KAAA;AAAA,UACR;AAAA,SACF;AAAA,MACF;AAEA,MAAA,MAAM,SAAA,GAAY,CAAC,KAAA,CAAM,KAAA,EAAO,GAAI,KAAA,CAAM,YAAA,IAAgB,EAAG,CAAA;AAC7D,MAAA,MAAM,KAAA,GAAQA,gBAAW,SAAS,CAAA;AAClC,MAAA,MAAM,MAAA,GAASC,gBAAA,CAAY,SAAS,CAAA,CAAE,MAAA,CAAO,OAAK,CAAC,KAAA,CAAM,QAAA,CAAS,CAAC,CAAC,CAAA;AACpE,MAAA,MAAM,CAAC,EAAE,UAAA,EAAW,EAAG,WAAW,CAAA,GAAI,MAAM,QAAQ,GAAA,CAAI;AAAA,QACtDC,aAAU,EAAE,KAAA,EAAO,cAAA,EAAgB,MAAA,EAAQ,WAAW,CAAA;AAAA,QACtD,KAAA,CAAM,oBAAoB,CAAC,GAAG,OAAO,GAAG,MAAM,CAAA,EAAG,eAAA,GAAkB,CAAC;AAAA,OACrE,CAAA;AACD,MAAA,MAAM,QAAQ,CAAC,GAAG,KAAA,EAAO,GAAG,MAAM,CAAA,CAAE,MAAA;AAAA,QAClC,CAAA,CAAA,KAAA,CAAM,WAAA,CAAY,GAAA,CAAI,CAAC,KAAK,CAAA,IAAK;AAAA,OACnC;AACA,MAAA,MAAM,IAAA,GAAO,MAAM,MAAA,CAAO,CAAA,CAAA,KAAK,YAAY,GAAA,CAAI,CAAC,KAAM,eAAe,CAAA;AACrE,MAAA,MAAM,OAAA,GAAU,MAAM,MAAA,CAAO,CAAA,CAAA,KAAK,YAAY,GAAA,CAAI,CAAC,MAAM,CAAC,CAAA;AAE1D,MAAA,MAAM,KAAA,GAAsB,MAAM,OAAA,CAAQ,GAAA,CAAI;AAAA,QAC5C,GAAG,UAAA,CAAW,GAAA,CAAI,OAAM,SAAA,MAAc;AAAA,UACpC,GAAA,EAAK,MAAM,KAAA,CAAM,OAAA,CAAQ,WAAW,UAAU,CAAA;AAAA,UAC9C,MAAA,EAAQ,IAAI,UAAA,CAAW;AAAA,SACzB,CAAE,CAAA;AAAA,QACF,KAAA,CAAM,OAAA,CAAQ,KAAA,EAAO,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE,CAAA;AAAA,QACjE,KAAA,CAAM,OAAA,CAAQ,IAAA,EAAM,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE,CAAA;AAAA,QAChE,KAAA,CAAM,KAAA,CAAM,OAAA,EAAS,UAAU,CAAA,CAAE,IAAA,CAAK,CAAA,GAAA,MAAQ,EAAE,GAAA,EAAK,MAAA,EAAQ,CAAA,EAAE,CAAE;AAAA,OAClE,CAAA;AAED,MAAA,MAAM,SAASC,iBAAA,CAAa,KAAK,CAAA,CAAE,KAAA,CAAM,GAAG,KAAK,CAAA;AACjD,MAAA,MAAM,OAAO,IAAI,GAAA;AAAA,QAAA,CACd,MAAM,KAAA,CAAM,MAAA,CAAO,MAAM,CAAA,EAAG,GAAA,CAAI,CAAA,KAAA,KAAS,CAAC,KAAA,CAAM,EAAA,EAAI,KAAK,CAAC;AAAA,OAC7D;AACA,MAAA,MAAM,UAAA,GAAa,MAAA,CAChB,GAAA,CAAI,CAAA,EAAA,KAAM,KAAK,GAAA,CAAI,EAAE,CAAC,CAAA,CACtB,MAAA,CAAO,CAAC,KAAA,KAAgC,OAAA,CAAQ,KAAK,CAAC,CAAA;AAEzD,MAAA,MAAM,QAAA,GAAW,MAAM,kBAAA,CAAmB,OAAA,EAAS,MAAM,WAAA,EAAa;AAAA,QACpE,GAAG,IAAI,GAAA;AAAA,UACL,UAAA,CAAW,OAAA;AAAA,YAAQ,WACjB,KAAA,CAAM,SAAA,GAAY,CAAC,KAAA,CAAM,SAAS,IAAI;AAAC;AACzC;AACF,OACD,CAAA;AAED,MAAA,MAAM,IAAA,uBAAW,GAAA,EAAY;AAC7B,MAAA,MAAM,UAAU,EAAC;AACjB,MAAA,KAAA,MAAW,SAAS,UAAA,EAAY;AAC9B,QAAA,MAAM,MAAM,CAAA,EAAG,KAAA,CAAM,QAAQ,CAAA,CAAA,EAAI,MAAM,KAAK,CAAA,CAAA;AAC5C,QAAA,IACG,MAAM,SAAA,IAAa,CAAC,QAAA,CAAS,GAAA,CAAI,MAAM,SAAS,CAAA,IACjD,IAAA,CAAK,GAAA,CAAI,GAAG,CAAA,IACZ,IAAA,CAAK,GAAA,CAAI,KAAA,CAAM,OAAO,CAAA,EACtB;AACA,UAAA;AAAA,QACF;AACA,QAAA,IAAA,CAAK,IAAI,GAAG,CAAA;AACZ,QAAA,IAAA,CAAK,GAAA,CAAI,MAAM,OAAO,CAAA;AACtB,QAAA,OAAA,CAAQ,IAAA,CAAK;AAAA,UACX,OAAO,KAAA,CAAM,KAAA;AAAA,UACb,UAAU,KAAA,CAAM,QAAA;AAAA,UAChB,GAAI,MAAM,SAAA,GAAY,EAAE,WAAW,KAAA,CAAM,SAAA,KAAc,EAAC;AAAA,UACxD,OAAA,EAAS,KAAA,CAAM,OAAA,CAAQ,KAAA,CAAM,GAAG,aAAa;AAAA,SAC9C,CAAA;AACD,QAAA,IAAI,OAAA,CAAQ,MAAA,KAAW,KAAA,CAAM,KAAA,EAAO;AAClC,UAAA;AAAA,QACF;AAAA,MACF;AACA,MAAA,OAAO,EAAE,MAAA,EAAQ,EAAE,OAAA,EAAQ,EAAE;AAAA,IAC/B;AAAA,GACD,CAAA;AACH;AAOA,eAAe,kBAAA,CACb,OAAA,EACA,IAAA,EACA,WAAA,EACA,UAAA,EACsB;AACtB,EAAA,IAAI,UAAA,CAAW,WAAW,CAAA,EAAG;AAC3B,IAAA,2BAAW,GAAA,EAAI;AAAA,EACjB;AACA,EAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,KAAK,qBAAA,CAAsB;AAAA,IACjD,UAAA,EAAY,WAAA;AAAA,IACZ,cAAA,EAAgB;AAAA,GACjB,CAAA;AACD,EAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,OAAA,CAAQ,iBAAA;AAAA,IAC9B,EAAE,UAAA,EAAY,MAAA,EAAQ,CAAC,MAAA,EAAQ,oBAAA,EAAsB,eAAe,CAAA,EAAE;AAAA,IACtE,EAAE,KAAA;AAAM,GACV;AACA,EAAA,OAAO,IAAI,GAAA,CAAI,UAAA,CAAW,MAAA,CAAO,CAAC,GAAG,CAAA,KAAM,KAAA,CAAM,CAAC,CAAC,CAAC,CAAA;AACtD;;;;"}

package/dist/chunk.cjs.js

@@ -6,9 +6,8 @@ const MAX_CHARS = 1600;
 const OVERLAP_CHARS = 200;
 function chunkDocument(doc) {
   return splitText(doc.text).map((content, index) => ({
-    id: sha256(`${doc.source}|${doc.key}|${index}`),
-    source: doc.source,
-    entityRef: doc.entityRef,
+    id: sha256(`${doc.location}|${index}`),
+    ...doc.entityRef ? { entityRef: doc.entityRef } : {},
     location: doc.location,
     title: doc.title,
     content,

package/dist/chunk.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"chunk.cjs.js","sources":["../src/chunk.ts"],"sourcesContent":["import { createHash } from 'node:crypto';\nimport type { SourceDocument } from './sources';\n\n/** A stored, embedded unit of a {@link SourceDocument}. */\nexport interface Chunk {\n  id: string;\n  source: SourceDocument['source'];\n  entityRef: string;\n  location: string;\n  title: string;\n  content: string;\n  /** Hash of what gets embedded; unchanged hash = no re-embedding. */\n  hash: string;\n}\n\nconst MAX_CHARS = 1600;\nconst OVERLAP_CHARS = 200;\n\n/**\n * Splits a document into chunks of at most ~1600 characters on word\n * boundaries, overlapping by ~200 so a sentence cut at a boundary survives in\n * one piece. MkDocs sections are usually one chunk already.\n */\nexport function chunkDocument(doc: SourceDocument): Chunk[] {\n  return splitText(doc.text).map((content, index) => ({\n    id: sha256(`${doc.source}|${doc.key}|${index}`),\n    source: doc.source,\n    entityRef: doc.entityRef,\n    location: doc.location,\n    title: doc.title,\n    content,\n    hash: sha256(embeddingInput(doc.title, content)),\n  }));\n}\n\n/** The text a chunk is embedded as: heading path first, so a terse section keeps its context. */\nexport function embeddingInput(title: string, content: string): string {\n  return `${title}\\n\\n${content}`;\n}\n\nexport function splitText(text: string): string[] {\n  if (text.length <= MAX_CHARS) {\n    return [text];\n  }\n  const pieces: string[] = [];\n  let start = 0;\n  while (start < text.length) {\n    let end = Math.min(start + MAX_CHARS, text.length);\n    if (end < text.length) {\n      const space = text.lastIndexOf(' ', end);\n      if (space > start + MAX_CHARS / 2) {\n        end = space;\n      }\n    }\n    pieces.push(text.slice(start, end).trim());\n    if (end >= text.length) {\n      break;\n    }\n    const restart = text.indexOf(' ', end - OVERLAP_CHARS);\n    start = restart > start && restart < end ? restart + 1 : end;\n  }\n  return pieces.filter(Boolean);\n}\n\nfunction sha256(value: string): string {\n  return createHash('sha256').update(value).digest('hex');\n}\n"],"names":["createHash"],"mappings":";;;;AAeA,MAAM,SAAA,GAAY,IAAA;AAClB,MAAM,aAAA,GAAgB,GAAA;AAOf,SAAS,cAAc,GAAA,EAA8B;AAC1D,EAAA,OAAO,UAAU,GAAA,CAAI,IAAI,EAAE,GAAA,CAAI,CAAC,SAAS,KAAA,MAAW;AAAA,IAClD,EAAA,EAAI,MAAA,CAAO,CAAA,EAAG,GAAA,CAAI,MAAM,IAAI,GAAA,CAAI,GAAG,CAAA,CAAA,EAAI,KAAK,CAAA,CAAE,CAAA;AAAA,IAC9C,QAAQ,GAAA,CAAI,MAAA;AAAA,IACZ,WAAW,GAAA,CAAI,SAAA;AAAA,IACf,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,OAAO,GAAA,CAAI,KAAA;AAAA,IACX,OAAA;AAAA,IACA,MAAM,MAAA,CAAO,cAAA,CAAe,GAAA,CAAI,KAAA,EAAO,OAAO,CAAC;AAAA,GACjD,CAAE,CAAA;AACJ;AAGO,SAAS,cAAA,CAAe,OAAe,OAAA,EAAyB;AACrE,EAAA,OAAO,GAAG,KAAK;;AAAA,EAAO,OAAO,CAAA,CAAA;AAC/B;AAEO,SAAS,UAAU,IAAA,EAAwB;AAChD,EAAA,IAAI,IAAA,CAAK,UAAU,SAAA,EAAW;AAC5B,IAAA,OAAO,CAAC,IAAI,CAAA;AAAA,EACd;AACA,EAAA,MAAM,SAAmB,EAAC;AAC1B,EAAA,IAAI,KAAA,GAAQ,CAAA;AACZ,EAAA,OAAO,KAAA,GAAQ,KAAK,MAAA,EAAQ;AAC1B,IAAA,IAAI,MAAM,IAAA,CAAK,GAAA,CAAI,KAAA,GAAQ,SAAA,EAAW,KAAK,MAAM,CAAA;AACjD,IAAA,IAAI,GAAA,GAAM,KAAK,MAAA,EAAQ;AACrB,MAAA,MAAM,KAAA,GAAQ,IAAA,CAAK,WAAA,CAAY,GAAA,EAAK,GAAG,CAAA;AACvC,MAAA,IAAI,KAAA,GAAQ,KAAA,GAAQ,SAAA,GAAY,CAAA,EAAG;AACjC,QAAA,GAAA,GAAM,KAAA;AAAA,MACR;AAAA,IACF;AACA,IAAA,MAAA,CAAO,KAAK,IAAA,CAAK,KAAA,CAAM,OAAO,GAAG,CAAA,CAAE,MAAM,CAAA;AACzC,IAAA,IAAI,GAAA,IAAO,KAAK,MAAA,EAAQ;AACtB,MAAA;AAAA,IACF;AACA,IAAA,MAAM,OAAA,GAAU,IAAA,CAAK,OAAA,CAAQ,GAAA,EAAK,MAAM,aAAa,CAAA;AACrD,IAAA,KAAA,GAAQ,OAAA,GAAU,KAAA,IAAS,OAAA,GAAU,GAAA,GAAM,UAAU,CAAA,GAAI,GAAA;AAAA,EAC3D;AACA,EAAA,OAAO,MAAA,CAAO,OAAO,OAAO,CAAA;AAC9B;AAEA,SAAS,OAAO,KAAA,EAAuB;AACrC,EAAA,OAAOA,uBAAW,QAAQ,CAAA,CAAE,OAAO,KAAK,CAAA,CAAE,OAAO,KAAK,CAAA;AACxD;;;;;;"}
+{"version":3,"file":"chunk.cjs.js","sources":["../src/chunk.ts"],"sourcesContent":["import { createHash } from 'node:crypto';\nimport type { SourceDocument } from './documents';\n\n/** A stored, embedded unit of a {@link SourceDocument}. */\nexport interface Chunk {\n  id: string;\n  entityRef?: string;\n  location: string;\n  title: string;\n  content: string;\n  /** Hash of what gets embedded; unchanged hash = no re-embedding. */\n  hash: string;\n}\n\nconst MAX_CHARS = 1600;\nconst OVERLAP_CHARS = 200;\n\n/**\n * Splits a document into chunks of at most ~1600 characters on word\n * boundaries, overlapping by ~200 so a sentence cut at a boundary survives in\n * one piece. TechDocs sections are usually one chunk already.\n */\nexport function chunkDocument(doc: SourceDocument): Chunk[] {\n  return splitText(doc.text).map((content, index) => ({\n    id: sha256(`${doc.location}|${index}`),\n    ...(doc.entityRef ? { entityRef: doc.entityRef } : {}),\n    location: doc.location,\n    title: doc.title,\n    content,\n    hash: sha256(embeddingInput(doc.title, content)),\n  }));\n}\n\n/** The text a chunk is embedded as: heading path first, so a terse section keeps its context. */\nexport function embeddingInput(title: string, content: string): string {\n  return `${title}\\n\\n${content}`;\n}\n\nexport function splitText(text: string): string[] {\n  if (text.length <= MAX_CHARS) {\n    return [text];\n  }\n  const pieces: string[] = [];\n  let start = 0;\n  while (start < text.length) {\n    let end = Math.min(start + MAX_CHARS, text.length);\n    if (end < text.length) {\n      const space = text.lastIndexOf(' ', end);\n      if (space > start + MAX_CHARS / 2) {\n        end = space;\n      }\n    }\n    pieces.push(text.slice(start, end).trim());\n    if (end >= text.length) {\n      break;\n    }\n    const restart = text.indexOf(' ', end - OVERLAP_CHARS);\n    start = restart > start && restart < end ? restart + 1 : end;\n  }\n  return pieces.filter(Boolean);\n}\n\nfunction sha256(value: string): string {\n  return createHash('sha256').update(value).digest('hex');\n}\n"],"names":["createHash"],"mappings":";;;;AAcA,MAAM,SAAA,GAAY,IAAA;AAClB,MAAM,aAAA,GAAgB,GAAA;AAOf,SAAS,cAAc,GAAA,EAA8B;AAC1D,EAAA,OAAO,UAAU,GAAA,CAAI,IAAI,EAAE,GAAA,CAAI,CAAC,SAAS,KAAA,MAAW;AAAA,IAClD,IAAI,MAAA,CAAO,CAAA,EAAG,IAAI,QAAQ,CAAA,CAAA,EAAI,KAAK,CAAA,CAAE,CAAA;AAAA,IACrC,GAAI,IAAI,SAAA,GAAY,EAAE,WAAW,GAAA,CAAI,SAAA,KAAc,EAAC;AAAA,IACpD,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,OAAO,GAAA,CAAI,KAAA;AAAA,IACX,OAAA;AAAA,IACA,MAAM,MAAA,CAAO,cAAA,CAAe,GAAA,CAAI,KAAA,EAAO,OAAO,CAAC;AAAA,GACjD,CAAE,CAAA;AACJ;AAGO,SAAS,cAAA,CAAe,OAAe,OAAA,EAAyB;AACrE,EAAA,OAAO,GAAG,KAAK;;AAAA,EAAO,OAAO,CAAA,CAAA;AAC/B;AAEO,SAAS,UAAU,IAAA,EAAwB;AAChD,EAAA,IAAI,IAAA,CAAK,UAAU,SAAA,EAAW;AAC5B,IAAA,OAAO,CAAC,IAAI,CAAA;AAAA,EACd;AACA,EAAA,MAAM,SAAmB,EAAC;AAC1B,EAAA,IAAI,KAAA,GAAQ,CAAA;AACZ,EAAA,OAAO,KAAA,GAAQ,KAAK,MAAA,EAAQ;AAC1B,IAAA,IAAI,MAAM,IAAA,CAAK,GAAA,CAAI,KAAA,GAAQ,SAAA,EAAW,KAAK,MAAM,CAAA;AACjD,IAAA,IAAI,GAAA,GAAM,KAAK,MAAA,EAAQ;AACrB,MAAA,MAAM,KAAA,GAAQ,IAAA,CAAK,WAAA,CAAY,GAAA,EAAK,GAAG,CAAA;AACvC,MAAA,IAAI,KAAA,GAAQ,KAAA,GAAQ,SAAA,GAAY,CAAA,EAAG;AACjC,QAAA,GAAA,GAAM,KAAA;AAAA,MACR;AAAA,IACF;AACA,IAAA,MAAA,CAAO,KAAK,IAAA,CAAK,KAAA,CAAM,OAAO,GAAG,CAAA,CAAE,MAAM,CAAA;AACzC,IAAA,IAAI,GAAA,IAAO,KAAK,MAAA,EAAQ;AACtB,MAAA;AAAA,IACF;AACA,IAAA,MAAM,OAAA,GAAU,IAAA,CAAK,OAAA,CAAQ,GAAA,EAAK,MAAM,aAAa,CAAA;AACrD,IAAA,KAAA,GAAQ,OAAA,GAAU,KAAA,IAAS,OAAA,GAAU,GAAA,GAAM,UAAU,CAAA,GAAI,GAAA;AAAA,EAC3D;AACA,EAAA,OAAO,MAAA,CAAO,OAAO,OAAO,CAAA;AAC9B;AAEA,SAAS,OAAO,KAAA,EAAuB;AACrC,EAAA,OAAOA,uBAAW,QAAQ,CAAA,CAAE,OAAO,KAAK,CAAA,CAAE,OAAO,KAAK,CAAA;AACxD;;;;;;"}

package/dist/config.cjs.js

@@ -1,15 +1,9 @@
 'use strict';
 
-var backendPluginApi = require('@backstage/backend-plugin-api');
 var errors = require('@backstage/errors');
 var azure = require('@ai-sdk/azure');
 var openai = require('@ai-sdk/openai');
 
-const DEFAULT_SCHEDULE = {
-  frequency: { minutes: 30 },
-  timeout: { hours: 1 },
-  initialDelay: { seconds: 15 }
-};
 function readSemanticSearchConfig(root) {
   const settings = root.getOptionalConfig("assistants.semanticSearch");
   if (!settings) {
@@ -32,10 +26,7 @@ function readSemanticSearchConfig(root) {
   }
   return {
     embeddingModelId,
-    embeddingModel: buildEmbeddingModel(providerId, provider, modelName),
-    schedule: settings.has("schedule") ? backendPluginApi.readSchedulerServiceTaskScheduleDefinitionFromConfig(
-      settings.getConfig("schedule")
-    ) : DEFAULT_SCHEDULE
+    embeddingModel: buildEmbeddingModel(providerId, provider, modelName)
   };
 }
 function buildEmbeddingModel(providerId, provider, modelName) {

package/dist/config.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"config.cjs.js","sources":["../src/config.ts"],"sourcesContent":["import {\n  readSchedulerServiceTaskScheduleDefinitionFromConfig,\n  type SchedulerServiceTaskScheduleDefinition,\n} from '@backstage/backend-plugin-api';\nimport { Config } from '@backstage/config';\nimport { InputError } from '@backstage/errors';\nimport { createAzure } from '@ai-sdk/azure';\nimport { createOpenAI } from '@ai-sdk/openai';\nimport type { EmbeddingModel } from 'ai';\n\n/** The `assistants.semanticSearch` block, resolved. */\nexport interface SemanticSearchConfig {\n  /** `<providerId>:<model>` as configured; part of the index signature. */\n  embeddingModelId: string;\n  embeddingModel: EmbeddingModel;\n  schedule: SchedulerServiceTaskScheduleDefinition;\n}\n\nconst DEFAULT_SCHEDULE: SchedulerServiceTaskScheduleDefinition = {\n  frequency: { minutes: 30 },\n  timeout: { hours: 1 },\n  initialDelay: { seconds: 15 },\n};\n\n/**\n * Reads `assistants.semanticSearch`, or `undefined` when it is absent (the module is\n * installed but off). The embedding model resolves against the same\n * `assistants.providers` block the chat models use, so keys live in one place.\n */\nexport function readSemanticSearchConfig(\n  root: Config,\n): SemanticSearchConfig | undefined {\n  const settings = root.getOptionalConfig('assistants.semanticSearch');\n  if (!settings) {\n    return undefined;\n  }\n\n  const embeddingModelId = settings.getString('embeddingModel');\n  const separator = embeddingModelId.indexOf(':');\n  if (separator <= 0 || separator === embeddingModelId.length - 1) {\n    throw new InputError(\n      `assistants.semanticSearch.embeddingModel must be '<providerId>:<model>'; got '${embeddingModelId}'`,\n    );\n  }\n  const providerId = embeddingModelId.slice(0, separator);\n  const modelName = embeddingModelId.slice(separator + 1);\n\n  const provider = root\n    .getOptionalConfig('assistants.providers')\n    ?.getOptionalConfig(providerId);\n  if (!provider) {\n    throw new InputError(\n      `assistants.semanticSearch.embeddingModel references provider '${providerId}', which is not in assistants.providers`,\n    );\n  }\n\n  return {\n    embeddingModelId,\n    embeddingModel: buildEmbeddingModel(providerId, provider, modelName),\n    schedule: settings.has('schedule')\n      ? readSchedulerServiceTaskScheduleDefinitionFromConfig(\n          settings.getConfig('schedule'),\n        )\n      : DEFAULT_SCHEDULE,\n  };\n}\n\n/**\n * The AI SDK embedding model for a configured provider. Anthropic has no\n * embeddings API, so only the OpenAI-shaped types qualify.\n */\nfunction buildEmbeddingModel(\n  providerId: string,\n  provider: Config,\n  modelName: string,\n): EmbeddingModel {\n  const type = provider.getString('type');\n  const apiKey = provider.getString('apiKey');\n  const baseURL = provider.getOptionalString('baseUrl');\n  const options =\n    (provider.getOptional('options') as Record<string, unknown>) ?? {};\n\n  switch (type) {\n    case 'openai':\n    case 'openai-compatible':\n      return createOpenAI({ apiKey, baseURL, ...options }).embedding(modelName);\n    case 'azure':\n      return createAzure({ apiKey, baseURL, ...options }).embedding(modelName);\n    default:\n      throw new InputError(\n        `assistants.semanticSearch.embeddingModel: provider '${providerId}' is type '${type}', which has no embeddings API; use an openai, azure, or openai-compatible provider`,\n      );\n  }\n}\n"],"names":["InputError","readSchedulerServiceTaskScheduleDefinitionFromConfig","createOpenAI","createAzure"],"mappings":";;;;;;;AAkBA,MAAM,gBAAA,GAA2D;AAAA,EAC/D,SAAA,EAAW,EAAE,OAAA,EAAS,EAAA,EAAG;AAAA,EACzB,OAAA,EAAS,EAAE,KAAA,EAAO,CAAA,EAAE;AAAA,EACpB,YAAA,EAAc,EAAE,OAAA,EAAS,EAAA;AAC3B,CAAA;AAOO,SAAS,yBACd,IAAA,EACkC;AAClC,EAAA,MAAM,QAAA,GAAW,IAAA,CAAK,iBAAA,CAAkB,2BAA2B,CAAA;AACnE,EAAA,IAAI,CAAC,QAAA,EAAU;AACb,IAAA,OAAO,MAAA;AAAA,EACT;AAEA,EAAA,MAAM,gBAAA,GAAmB,QAAA,CAAS,SAAA,CAAU,gBAAgB,CAAA;AAC5D,EAAA,MAAM,SAAA,GAAY,gBAAA,CAAiB,OAAA,CAAQ,GAAG,CAAA;AAC9C,EAAA,IAAI,SAAA,IAAa,CAAA,IAAK,SAAA,KAAc,gBAAA,CAAiB,SAAS,CAAA,EAAG;AAC/D,IAAA,MAAM,IAAIA,iBAAA;AAAA,MACR,iFAAiF,gBAAgB,CAAA,CAAA;AAAA,KACnG;AAAA,EACF;AACA,EAAA,MAAM,UAAA,GAAa,gBAAA,CAAiB,KAAA,CAAM,CAAA,EAAG,SAAS,CAAA;AACtD,EAAA,MAAM,SAAA,GAAY,gBAAA,CAAiB,KAAA,CAAM,SAAA,GAAY,CAAC,CAAA;AAEtD,EAAA,MAAM,WAAW,IAAA,CACd,iBAAA,CAAkB,sBAAsB,CAAA,EACvC,kBAAkB,UAAU,CAAA;AAChC,EAAA,IAAI,CAAC,QAAA,EAAU;AACb,IAAA,MAAM,IAAIA,iBAAA;AAAA,MACR,iEAAiE,UAAU,CAAA,uCAAA;AAAA,KAC7E;AAAA,EACF;AAEA,EAAA,OAAO;AAAA,IACL,gBAAA;AAAA,IACA,cAAA,EAAgB,mBAAA,CAAoB,UAAA,EAAY,QAAA,EAAU,SAAS,CAAA;AAAA,IACnE,QAAA,EAAU,QAAA,CAAS,GAAA,CAAI,UAAU,CAAA,GAC7BC,qEAAA;AAAA,MACE,QAAA,CAAS,UAAU,UAAU;AAAA,KAC/B,GACA;AAAA,GACN;AACF;AAMA,SAAS,mBAAA,CACP,UAAA,EACA,QAAA,EACA,SAAA,EACgB;AAChB,EAAA,MAAM,IAAA,GAAO,QAAA,CAAS,SAAA,CAAU,MAAM,CAAA;AACtC,EAAA,MAAM,MAAA,GAAS,QAAA,CAAS,SAAA,CAAU,QAAQ,CAAA;AAC1C,EAAA,MAAM,OAAA,GAAU,QAAA,CAAS,iBAAA,CAAkB,SAAS,CAAA;AACpD,EAAA,MAAM,OAAA,GACH,QAAA,CAAS,WAAA,CAAY,SAAS,KAAiC,EAAC;AAEnE,EAAA,QAAQ,IAAA;AAAM,IACZ,KAAK,QAAA;AAAA,IACL,KAAK,mBAAA;AACH,MAAA,OAAOC,mBAAA,CAAa,EAAE,MAAA,EAAQ,OAAA,EAAS,GAAG,OAAA,EAAS,CAAA,CAAE,SAAA,CAAU,SAAS,CAAA;AAAA,IAC1E,KAAK,OAAA;AACH,MAAA,OAAOC,iBAAA,CAAY,EAAE,MAAA,EAAQ,OAAA,EAAS,GAAG,OAAA,EAAS,CAAA,CAAE,SAAA,CAAU,SAAS,CAAA;AAAA,IACzE;AACE,MAAA,MAAM,IAAIH,iBAAA;AAAA,QACR,CAAA,oDAAA,EAAuD,UAAU,CAAA,WAAA,EAAc,IAAI,CAAA,mFAAA;AAAA,OACrF;AAAA;AAEN;;;;"}
+{"version":3,"file":"config.cjs.js","sources":["../src/config.ts"],"sourcesContent":["import { Config } from '@backstage/config';\nimport { InputError } from '@backstage/errors';\nimport { createAzure } from '@ai-sdk/azure';\nimport { createOpenAI } from '@ai-sdk/openai';\nimport type { EmbeddingModel } from 'ai';\n\n/** The `assistants.semanticSearch` block, resolved. */\nexport interface SemanticSearchConfig {\n  /** `<providerId>:<model>` as configured; part of the index signature. */\n  embeddingModelId: string;\n  embeddingModel: EmbeddingModel;\n}\n\n/**\n * Reads `assistants.semanticSearch`, or `undefined` when it is absent (the module is\n * installed but off). The embedding model resolves against the same\n * `assistants.providers` block the chat models use, so keys live in one place.\n */\nexport function readSemanticSearchConfig(\n  root: Config,\n): SemanticSearchConfig | undefined {\n  const settings = root.getOptionalConfig('assistants.semanticSearch');\n  if (!settings) {\n    return undefined;\n  }\n\n  const embeddingModelId = settings.getString('embeddingModel');\n  const separator = embeddingModelId.indexOf(':');\n  if (separator <= 0 || separator === embeddingModelId.length - 1) {\n    throw new InputError(\n      `assistants.semanticSearch.embeddingModel must be '<providerId>:<model>'; got '${embeddingModelId}'`,\n    );\n  }\n  const providerId = embeddingModelId.slice(0, separator);\n  const modelName = embeddingModelId.slice(separator + 1);\n\n  const provider = root\n    .getOptionalConfig('assistants.providers')\n    ?.getOptionalConfig(providerId);\n  if (!provider) {\n    throw new InputError(\n      `assistants.semanticSearch.embeddingModel references provider '${providerId}', which is not in assistants.providers`,\n    );\n  }\n\n  return {\n    embeddingModelId,\n    embeddingModel: buildEmbeddingModel(providerId, provider, modelName),\n  };\n}\n\n/**\n * The AI SDK embedding model for a configured provider. Anthropic has no\n * embeddings API, so only the OpenAI-shaped types qualify.\n */\nfunction buildEmbeddingModel(\n  providerId: string,\n  provider: Config,\n  modelName: string,\n): EmbeddingModel {\n  const type = provider.getString('type');\n  const apiKey = provider.getString('apiKey');\n  const baseURL = provider.getOptionalString('baseUrl');\n  const options =\n    (provider.getOptional('options') as Record<string, unknown>) ?? {};\n\n  switch (type) {\n    case 'openai':\n    case 'openai-compatible':\n      return createOpenAI({ apiKey, baseURL, ...options }).embedding(modelName);\n    case 'azure':\n      return createAzure({ apiKey, baseURL, ...options }).embedding(modelName);\n    default:\n      throw new InputError(\n        `assistants.semanticSearch.embeddingModel: provider '${providerId}' is type '${type}', which has no embeddings API; use an openai, azure, or openai-compatible provider`,\n      );\n  }\n}\n"],"names":["InputError","createOpenAI","createAzure"],"mappings":";;;;;;AAkBO,SAAS,yBACd,IAAA,EACkC;AAClC,EAAA,MAAM,QAAA,GAAW,IAAA,CAAK,iBAAA,CAAkB,2BAA2B,CAAA;AACnE,EAAA,IAAI,CAAC,QAAA,EAAU;AACb,IAAA,OAAO,MAAA;AAAA,EACT;AAEA,EAAA,MAAM,gBAAA,GAAmB,QAAA,CAAS,SAAA,CAAU,gBAAgB,CAAA;AAC5D,EAAA,MAAM,SAAA,GAAY,gBAAA,CAAiB,OAAA,CAAQ,GAAG,CAAA;AAC9C,EAAA,IAAI,SAAA,IAAa,CAAA,IAAK,SAAA,KAAc,gBAAA,CAAiB,SAAS,CAAA,EAAG;AAC/D,IAAA,MAAM,IAAIA,iBAAA;AAAA,MACR,iFAAiF,gBAAgB,CAAA,CAAA;AAAA,KACnG;AAAA,EACF;AACA,EAAA,MAAM,UAAA,GAAa,gBAAA,CAAiB,KAAA,CAAM,CAAA,EAAG,SAAS,CAAA;AACtD,EAAA,MAAM,SAAA,GAAY,gBAAA,CAAiB,KAAA,CAAM,SAAA,GAAY,CAAC,CAAA;AAEtD,EAAA,MAAM,WAAW,IAAA,CACd,iBAAA,CAAkB,sBAAsB,CAAA,EACvC,kBAAkB,UAAU,CAAA;AAChC,EAAA,IAAI,CAAC,QAAA,EAAU;AACb,IAAA,MAAM,IAAIA,iBAAA;AAAA,MACR,iEAAiE,UAAU,CAAA,uCAAA;AAAA,KAC7E;AAAA,EACF;AAEA,EAAA,OAAO;AAAA,IACL,gBAAA;AAAA,IACA,cAAA,EAAgB,mBAAA,CAAoB,UAAA,EAAY,QAAA,EAAU,SAAS;AAAA,GACrE;AACF;AAMA,SAAS,mBAAA,CACP,UAAA,EACA,QAAA,EACA,SAAA,EACgB;AAChB,EAAA,MAAM,IAAA,GAAO,QAAA,CAAS,SAAA,CAAU,MAAM,CAAA;AACtC,EAAA,MAAM,MAAA,GAAS,QAAA,CAAS,SAAA,CAAU,QAAQ,CAAA;AAC1C,EAAA,MAAM,OAAA,GAAU,QAAA,CAAS,iBAAA,CAAkB,SAAS,CAAA;AACpD,EAAA,MAAM,OAAA,GACH,QAAA,CAAS,WAAA,CAAY,SAAS,KAAiC,EAAC;AAEnE,EAAA,QAAQ,IAAA;AAAM,IACZ,KAAK,QAAA;AAAA,IACL,KAAK,mBAAA;AACH,MAAA,OAAOC,mBAAA,CAAa,EAAE,MAAA,EAAQ,OAAA,EAAS,GAAG,OAAA,EAAS,CAAA,CAAE,SAAA,CAAU,SAAS,CAAA;AAAA,IAC1E,KAAK,OAAA;AACH,MAAA,OAAOC,iBAAA,CAAY,EAAE,MAAA,EAAQ,OAAA,EAAS,GAAG,OAAA,EAAS,CAAA,CAAE,SAAA,CAAU,SAAS,CAAA;AAAA,IACzE;AACE,MAAA,MAAM,IAAIF,iBAAA;AAAA,QACR,CAAA,oDAAA,EAAuD,UAAU,CAAA,WAAA,EAAc,IAAI,CAAA,mFAAA;AAAA,OACrF;AAAA;AAEN;;;;"}

package/dist/documents.cjs.js

@@ -0,0 +1,84 @@
+'use strict';
+
+var catalogModel = require('@backstage/catalog-model');
+var nodeHtmlParser = require('node-html-parser');
+
+function toSourceDocuments(docs) {
+  const pageTitles = /* @__PURE__ */ new Map();
+  for (const doc of docs) {
+    if (!doc.location.includes("#") && !pageTitles.has(doc.location)) {
+      pageTitles.set(doc.location, htmlToText(doc.title));
+    }
+  }
+  const documents = [];
+  const seen = /* @__PURE__ */ new Set();
+  let skipped = 0;
+  for (const doc of docs) {
+    const resourceRef = doc.authorization?.resourceRef;
+    const entityRef = resourceRef ? asEntityRef(resourceRef) : void 0;
+    if (resourceRef && !entityRef) {
+      skipped++;
+      continue;
+    }
+    if (seen.has(doc.location)) {
+      continue;
+    }
+    const text = [htmlToText(doc.text), facts(doc, entityRef)].filter(Boolean).join("\n");
+    if (!text) {
+      continue;
+    }
+    seen.add(doc.location);
+    documents.push({
+      location: doc.location,
+      ...entityRef ? { entityRef } : {},
+      title: headingPath([
+        field(doc, "entityTitle"),
+        pageTitles.get(doc.location.split("#")[0]),
+        htmlToText(doc.title)
+      ]),
+      text
+    });
+  }
+  return { documents, skipped };
+}
+function asEntityRef(ref) {
+  try {
+    const { kind, namespace, name } = catalogModel.parseEntityRef(ref);
+    return `${kind}:${namespace}/${name}`.toLocaleLowerCase("en-US");
+  } catch {
+    return void 0;
+  }
+}
+function field(doc, key) {
+  const value = doc[key];
+  return typeof value === "string" && value ? value : void 0;
+}
+function facts(doc, entityRef) {
+  const type = field(doc, "type") ?? field(doc, "componentType");
+  const owner = field(doc, "owner");
+  const lifecycle = field(doc, "lifecycle");
+  return [
+    entityRef && (type ? `${entityRef} (${type})` : entityRef),
+    owner && `Owner: ${owner}`,
+    lifecycle && `Lifecycle: ${lifecycle}`
+  ].filter(Boolean).join(" \xB7 ");
+}
+function htmlToText(html) {
+  if (!html) {
+    return "";
+  }
+  return nodeHtmlParser.parse(html).text.replace(/\s+/g, " ").trim();
+}
+function headingPath(parts) {
+  const path = [];
+  for (const part of parts) {
+    if (part && part !== path[path.length - 1]) {
+      path.push(part);
+    }
+  }
+  return path.join(" \u203A ");
+}
+
+exports.htmlToText = htmlToText;
+exports.toSourceDocuments = toSourceDocuments;
+//# sourceMappingURL=documents.cjs.js.map

package/dist/documents.cjs.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"documents.cjs.js","sources":["../src/documents.ts"],"sourcesContent":["import { parseEntityRef } from '@backstage/catalog-model';\nimport type { IndexableDocument } from '@backstage/plugin-search-common';\nimport { parse } from 'node-html-parser';\n\n/**\n * One searchable unit before chunking: a document from a Backstage search\n * run (a TechDocs section, a catalog entity, a Confluence page, …) as text.\n */\nexport interface SourceDocument {\n  /** Its search result location: a link, and its identity across runs. */\n  location: string;\n  /**\n   * The catalog entity whose read permission gates it. Without one, every\n   * assistant user may see it, as everyone may in Backstage search.\n   */\n  entityRef?: string;\n  /** Heading path, e.g. `Pylon › Messaging › Dead-letter handling`. */\n  title: string;\n  text: string;\n}\n\nexport interface MappedDocuments {\n  documents: SourceDocument[];\n  /** Documents whose `authorization.resourceRef` is not an entity ref, so no permission check applies. */\n  skipped: number;\n}\n\n/**\n * Maps the documents of one search run. TechDocs sections take their page's\n * title from the page entry in the same run; HTML (MkDocs section text) is\n * flattened; the facts collators attach (entity ref, type, owner, lifecycle)\n * join the text, so a catalog entity with no description is still findable.\n * A document gated by anything other than a catalog entity is skipped: the\n * index can only check entity read permission. Pure; unit-tested.\n */\nexport function toSourceDocuments(docs: IndexableDocument[]): MappedDocuments {\n  const pageTitles = new Map<string, string>();\n  for (const doc of docs) {\n    if (!doc.location.includes('#') && !pageTitles.has(doc.location)) {\n      pageTitles.set(doc.location, htmlToText(doc.title));\n    }\n  }\n\n  const documents: SourceDocument[] = [];\n  const seen = new Set<string>();\n  let skipped = 0;\n  for (const doc of docs) {\n    const resourceRef = doc.authorization?.resourceRef;\n    const entityRef = resourceRef ? asEntityRef(resourceRef) : undefined;\n    if (resourceRef && !entityRef) {\n      skipped++;\n      continue;\n    }\n    if (seen.has(doc.location)) {\n      continue;\n    }\n    const text = [htmlToText(doc.text), facts(doc, entityRef)]\n      .filter(Boolean)\n      .join('\\n');\n    if (!text) {\n      continue;\n    }\n    seen.add(doc.location);\n    documents.push({\n      location: doc.location,\n      ...(entityRef ? { entityRef } : {}),\n      title: headingPath([\n        field(doc, 'entityTitle'),\n        pageTitles.get(doc.location.split('#')[0]),\n        htmlToText(doc.title),\n      ]),\n      text,\n    });\n  }\n  return { documents, skipped };\n}\n\n/** The ref in full form when it names a catalog entity. */\nfunction asEntityRef(ref: string): string | undefined {\n  try {\n    const { kind, namespace, name } = parseEntityRef(ref);\n    return `${kind}:${namespace}/${name}`.toLocaleLowerCase('en-US');\n  } catch {\n    return undefined;\n  }\n}\n\nfunction field(doc: IndexableDocument, key: string): string | undefined {\n  const value = (doc as unknown as Record<string, unknown>)[key];\n  return typeof value === 'string' && value ? value : undefined;\n}\n\nfunction facts(doc: IndexableDocument, entityRef: string | undefined): string {\n  const type = field(doc, 'type') ?? field(doc, 'componentType');\n  const owner = field(doc, 'owner');\n  const lifecycle = field(doc, 'lifecycle');\n  return [\n    entityRef && (type ? `${entityRef} (${type})` : entityRef),\n    owner && `Owner: ${owner}`,\n    lifecycle && `Lifecycle: ${lifecycle}`,\n  ]\n    .filter(Boolean)\n    .join(' · ');\n}\n\nexport function htmlToText(html: string | undefined): string {\n  if (!html) {\n    return '';\n  }\n  return parse(html).text.replace(/\\s+/g, ' ').trim();\n}\n\nfunction headingPath(parts: Array<string | undefined>): string {\n  const path: string[] = [];\n  for (const part of parts) {\n    if (part && part !== path[path.length - 1]) {\n      path.push(part);\n    }\n  }\n  return path.join(' › ');\n}\n"],"names":["parseEntityRef","parse"],"mappings":";;;;;AAmCO,SAAS,kBAAkB,IAAA,EAA4C;AAC5E,EAAA,MAAM,UAAA,uBAAiB,GAAA,EAAoB;AAC3C,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,IAAI,CAAC,GAAA,CAAI,QAAA,CAAS,QAAA,CAAS,GAAG,CAAA,IAAK,CAAC,UAAA,CAAW,GAAA,CAAI,GAAA,CAAI,QAAQ,CAAA,EAAG;AAChE,MAAA,UAAA,CAAW,IAAI,GAAA,CAAI,QAAA,EAAU,UAAA,CAAW,GAAA,CAAI,KAAK,CAAC,CAAA;AAAA,IACpD;AAAA,EACF;AAEA,EAAA,MAAM,YAA8B,EAAC;AACrC,EAAA,MAAM,IAAA,uBAAW,GAAA,EAAY;AAC7B,EAAA,IAAI,OAAA,GAAU,CAAA;AACd,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,MAAM,WAAA,GAAc,IAAI,aAAA,EAAe,WAAA;AACvC,IAAA,MAAM,SAAA,GAAY,WAAA,GAAc,WAAA,CAAY,WAAW,CAAA,GAAI,MAAA;AAC3D,IAAA,IAAI,WAAA,IAAe,CAAC,SAAA,EAAW;AAC7B,MAAA,OAAA,EAAA;AACA,MAAA;AAAA,IACF;AACA,IAAA,IAAI,IAAA,CAAK,GAAA,CAAI,GAAA,CAAI,QAAQ,CAAA,EAAG;AAC1B,MAAA;AAAA,IACF;AACA,IAAA,MAAM,IAAA,GAAO,CAAC,UAAA,CAAW,GAAA,CAAI,IAAI,CAAA,EAAG,KAAA,CAAM,GAAA,EAAK,SAAS,CAAC,CAAA,CACtD,MAAA,CAAO,OAAO,CAAA,CACd,KAAK,IAAI,CAAA;AACZ,IAAA,IAAI,CAAC,IAAA,EAAM;AACT,MAAA;AAAA,IACF;AACA,IAAA,IAAA,CAAK,GAAA,CAAI,IAAI,QAAQ,CAAA;AACrB,IAAA,SAAA,CAAU,IAAA,CAAK;AAAA,MACb,UAAU,GAAA,CAAI,QAAA;AAAA,MACd,GAAI,SAAA,GAAY,EAAE,SAAA,KAAc,EAAC;AAAA,MACjC,OAAO,WAAA,CAAY;AAAA,QACjB,KAAA,CAAM,KAAK,aAAa,CAAA;AAAA,QACxB,UAAA,CAAW,IAAI,GAAA,CAAI,QAAA,CAAS,MAAM,GAAG,CAAA,CAAE,CAAC,CAAC,CAAA;AAAA,QACzC,UAAA,CAAW,IAAI,KAAK;AAAA,OACrB,CAAA;AAAA,MACD;AAAA,KACD,CAAA;AAAA,EACH;AACA,EAAA,OAAO,EAAE,WAAW,OAAA,EAAQ;AAC9B;AAGA,SAAS,YAAY,GAAA,EAAiC;AACpD,EAAA,IAAI;AACF,IAAA,MAAM,EAAE,IAAA,EAAM,SAAA,EAAW,IAAA,EAAK,GAAIA,4BAAe,GAAG,CAAA;AACpD,IAAA,OAAO,CAAA,EAAG,IAAI,CAAA,CAAA,EAAI,SAAS,IAAI,IAAI,CAAA,CAAA,CAAG,kBAAkB,OAAO,CAAA;AAAA,EACjE,CAAA,CAAA,MAAQ;AACN,IAAA,OAAO,MAAA;AAAA,EACT;AACF;AAEA,SAAS,KAAA,CAAM,KAAwB,GAAA,EAAiC;AACtE,EAAA,MAAM,KAAA,GAAS,IAA2C,GAAG,CAAA;AAC7D,EAAA,OAAO,OAAO,KAAA,KAAU,QAAA,IAAY,KAAA,GAAQ,KAAA,GAAQ,MAAA;AACtD;AAEA,SAAS,KAAA,CAAM,KAAwB,SAAA,EAAuC;AAC5E,EAAA,MAAM,OAAO,KAAA,CAAM,GAAA,EAAK,MAAM,CAAA,IAAK,KAAA,CAAM,KAAK,eAAe,CAAA;AAC7D,EAAA,MAAM,KAAA,GAAQ,KAAA,CAAM,GAAA,EAAK,OAAO,CAAA;AAChC,EAAA,MAAM,SAAA,GAAY,KAAA,CAAM,GAAA,EAAK,WAAW,CAAA;AACxC,EAAA,OAAO;AAAA,IACL,cAAc,IAAA,GAAO,CAAA,EAAG,SAAS,CAAA,EAAA,EAAK,IAAI,CAAA,CAAA,CAAA,GAAM,SAAA,CAAA;AAAA,IAChD,KAAA,IAAS,UAAU,KAAK,CAAA,CAAA;AAAA,IACxB,SAAA,IAAa,cAAc,SAAS,CAAA;AAAA,GACtC,CACG,MAAA,CAAO,OAAO,CAAA,CACd,KAAK,QAAK,CAAA;AACf;AAEO,SAAS,WAAW,IAAA,EAAkC;AAC3D,EAAA,IAAI,CAAC,IAAA,EAAM;AACT,IAAA,OAAO,EAAA;AAAA,EACT;AACA,EAAA,OAAOC,oBAAA,CAAM,IAAI,CAAA,CAAE,IAAA,CAAK,QAAQ,MAAA,EAAQ,GAAG,EAAE,IAAA,EAAK;AACpD;AAEA,SAAS,YAAY,KAAA,EAA0C;AAC7D,EAAA,MAAM,OAAiB,EAAC;AACxB,EAAA,KAAA,MAAW,QAAQ,KAAA,EAAO;AACxB,IAAA,IAAI,QAAQ,IAAA,KAAS,IAAA,CAAK,IAAA,CAAK,MAAA,GAAS,CAAC,CAAA,EAAG;AAC1C,MAAA,IAAA,CAAK,KAAK,IAAI,CAAA;AAAA,IAChB;AAAA,EACF;AACA,EAAA,OAAO,IAAA,CAAK,KAAK,UAAK,CAAA;AACxB;;;;;"}