@backstage/plugin-catalog-backend 3.7.0-next.2 → 3.7.0

package/CHANGELOG.md

@@ -1,5 +1,85 @@
 # @backstage/plugin-catalog-backend
 
+## 3.7.0
+
+### Minor Changes
+
+- c2de113: **BREAKING**: When paginating entities with an order field via `/entities/by-query`, entities that lack the order field are now excluded from both the result set and the `totalItems` count. Previously these entities appeared at the end of the sorted result via `NULLS LAST`, but cursor-based pagination could not actually reach them past the first page — the count over-reported the number of navigable entities. The new behavior aligns the count with what is actually returned.
+
+  This also removes the `DISTINCT` deduplication from the sort-field CTE, which is a prerequisite for the planner to use the `(key, value, entity_id)` index in sort order and short-circuit on `LIMIT`. Installations with duplicate search rows should land the search-table deduplication migration before adopting this change.
+
+### Patch Changes
+
+- 3f5e7ec: Added `catalog.actions.experimentalCatalogLayersDescriptions.enabled` config option. When enabled, the `query-catalog-entities` action description references `get-catalog-model-description` for field information instead of embedding a static model description.
+- ccbad9d: Improved the performance of the `catalog_entities_count` metric.
+
+  The legacy Prometheus and OpenTelemetry observable gauges previously each ran their own copy of the per-kind count query against the `search` table on every metrics scrape. On large catalogs this could pile up faster than the queries completed, contending for buffers and stalling the database.
+
+  The two callbacks now share a single query result with a short in-process TTL cache, and the underlying query reads from `final_entities` instead of `search`, avoiding the bitmap heap scans that dominated the previous form. The emitted labels and values are unchanged.
+
+- 17a9550: Deprecated immediate mode stitching (`catalog.stitchingStrategy.mode: 'immediate'`). A warning is now logged on startup when immediate mode is configured. Immediate mode will be removed in the next Backstage release. Migrate to deferred mode (the default) by removing the `stitchingStrategy` configuration or setting `mode: 'deferred'`.
+- add5d1a: Restructured the entity listing endpoint so that, when a sort field is specified, the search-by-key index drives the query rather than being side-joined onto `final_entities`. This lets PostgreSQL walk the `(key, value, entity_id)` index in already-sorted order and short-circuit on `LIMIT`, reducing typical broad-filter paginated list times from seconds to milliseconds. Entities that lack the sort field still appear at the end of sorted results (NULLS LAST semantics preserved), ordered by `entity_id`.
+- 387ea7d: Simplified the entity facets aggregation from `COUNT(DISTINCT entity_id)` to `COUNT(*)`. The unique constraint on `(entity_id, key, value)` guarantees each entity appears at most once per search row group, making the `DISTINCT` unnecessary. This allows the database to use a simpler aggregation plan.
+- 3f55b73: Improved the performance of the entity facets endpoint when filters are applied. The filtered entity set is now combined with the search table through an inner join rather than a `WHERE entity_id IN (subquery)`. Results are unchanged; on large catalogs the query planner is able to choose dramatically cheaper plans, with measured improvements ranging from roughly 1.2× on already-fast cases to 7× or more on high-cardinality facets.
+- b33f845: Fixed several database migration `down` functions that were not properly reversible, causing the SQL report to show warnings:
+
+  - `20241003170511_alter_target_in_locations.js`: both `up` and `down` now include `.notNullable()` when altering the `locations.target` column, preventing the `NOT NULL` constraint from being accidentally dropped when widening the column type from `varchar(255)` to `text`.
+  - `20220116144621_remove_legacy.js`: the `down` function now properly recreates the three dropped legacy tables (`entities`, `entities_search`, `entities_relations`) with correct columns and indices.
+  - `20210302150147_refresh_state.js`: the `down` function now drops dependent tables in the correct order (avoiding a FK constraint violation) and fixes a typo where the table was referred to as `references` instead of `refresh_state_references`.
+  - `20201005122705_add_entity_full_name.js`: the `down` function now drops the `full_name` column from `entities` (not `entities_search`), and restores the `entities_unique_name` index with the correct column order `(kind, name, namespace)`.
+  - `20200702153613_entities.js`: the `down` function now uses `table.integer('generation')` instead of `table.string('generation')`, restoring the correct column type.
+
+- cde3643: Added missing description to the `type` parameter on the `unregister-entity` MCP action.
+- cf195de: Fixed a performance regression in the `/entity-facets` endpoint when filters or permission conditions are applied, by routing the EXISTS-based filter through `final_entities` instead of correlating against the much larger `search` table.
+- 07ec25d: Moved `generateStableHash` out of shared utility file to avoid pulling `node:crypto` into browser bundles
+- bc32c13: Added a missing index on `relations.target_entity_ref`. Several query paths (orphan deletion, entity ancestry, eager pruning) join or filter on this column, but no index existed — causing full sequential scans of the relations table on every invocation. On a production catalog with ~3.5M relation rows, individual lookups were taking ~122ms (full table scan) instead of <1ms (index scan).
+- 744fa1f: Removed duplicated entries that appeared in both `dependencies` and `devDependencies`.
+- e9b78e9: Removed the `uuid` dependency and replaced usage with the built-in `crypto.randomUUID()`.
+- 7445f0f: Added a migration that removes duplicate rows from the `search` table, creates covering indices for improved query performance, and adds a `UNIQUE` constraint on `(entity_id, key, value)`.
+
+  This is a long-running migration on large catalogs. On PostgreSQL with millions of search rows, the index creation may take 5-15 minutes per index. During this time, other pods running the previous version will continue to serve traffic normally — the index creation does not block reads or writes. However, if a Kubernetes liveness probe kills the pod before the index build completes, the build is lost and the next startup will start over. On large tables this can repeat indefinitely.
+
+  **For large installations**, it is recommended to run the following SQL commands against your PostgreSQL database **before deploying** this version. Each index build takes a few minutes but does not block reads or writes. If these have already completed, the migration will detect the existing indices and skip all work — startup will be instant.
+
+  ```sql
+  -- Step 1: Remove duplicate search rows
+  WITH cte AS (
+    SELECT ctid, row_number() OVER (PARTITION BY entity_id, key, value) AS rn
+    FROM search
+  )
+  DELETE FROM search USING cte WHERE search.ctid = cte.ctid AND cte.rn > 1;
+
+  -- Step 2: Create new indices (run each separately)
+  CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS
+    search_entity_key_value_idx ON search (entity_id, key, value);
+  CREATE INDEX CONCURRENTLY IF NOT EXISTS
+    search_key_value_entity_idx ON search (key, value, entity_id);
+  CREATE INDEX CONCURRENTLY IF NOT EXISTS
+    search_facets_covering_idx ON search (key, original_value, entity_id)
+    WHERE original_value IS NOT NULL;
+
+  -- Step 3: Drop old indices that are no longer needed
+  DROP INDEX CONCURRENTLY IF EXISTS search_key_value_idx;
+  DROP INDEX CONCURRENTLY IF EXISTS search_key_original_value_idx;
+  ```
+
+  Also fixed `buildEntitySearch` to remove duplicate output for entities with duplicate array values, and added `ON CONFLICT DO UPDATE` to `syncSearchRows` so that concurrent stitching races are handled gracefully.
+
+- Updated dependencies
+  - @backstage/catalog-model@1.9.0
+  - @backstage/errors@1.3.1
+  - @backstage/backend-plugin-api@1.9.1
+  - @backstage/plugin-catalog-node@2.2.1
+  - @backstage/filter-predicates@0.1.3
+  - @backstage/integration@2.0.2
+  - @backstage/plugin-permission-node@0.11.0
+  - @backstage/plugin-permission-common@0.9.9
+  - @backstage/backend-openapi-utils@0.6.9
+  - @backstage/catalog-client@1.15.1
+  - @backstage/config@1.3.8
+  - @backstage/plugin-catalog-common@1.1.10
+  - @backstage/plugin-events-node@0.4.22
+
 ## 3.7.0-next.2
 
 ### Minor Changes

package/config.d.ts

@@ -179,7 +179,11 @@ export interface Config {
      */
     stitchingStrategy?:
       | {
-          /** Perform stitching in-band immediately when needed */
+          /**
+           * Perform stitching in-band immediately when needed.
+           *
+           * @deprecated Immediate mode stitching has been deprecated and will be removed in a future release. Migrate to deferred mode (the default).
+           */
           mode: 'immediate';
         }
       | {

package/dist/actions/createQueryCatalogEntitiesAction.cjs.js

@@ -2,19 +2,44 @@
 
 var filterPredicates = require('@backstage/filter-predicates');
 
-const createQueryCatalogEntitiesAction = ({
-  catalog,
-  actionsRegistry
-}) => {
-  actionsRegistry.register({
-    name: "query-catalog-entities",
-    title: "Query Catalog Entities",
-    attributes: {
-      destructive: false,
-      readOnly: true,
-      idempotent: true
-    },
-    description: `
+const QUERY_SYNTAX = `
+## Query Syntax
+
+The query uses predicate expressions with dot-notation field paths.
+
+Simple matching:
+  { query: { kind: "Component" } }
+  { query: { kind: "Component", "spec.type": "service" } }
+
+Value operators:
+  { query: { kind: { "$in": ["API", "Component"] } } }
+  { query: { "metadata.annotations.backstage.io/techdocs-ref": { "$exists": true } } }
+  { query: { "metadata.tags": { "$contains": "java" } } }
+  { query: { "metadata.name": { "$hasPrefix": "team-" } } }
+
+Logical operators:
+  { query: { "$all": [{ kind: "Component" }, { "spec.lifecycle": "production" }] } }
+  { query: { "$any": [{ "spec.type": "service" }, { "spec.type": "website" }] } }
+  { query: { "$not": { kind: "Group" } } }
+
+Querying relations - find all entities owned by a specific group:
+  { query: { "relations.ownedby": "group:default/team-alpha" } }
+
+Combined example - find production services or websites with TechDocs:
+  { query: { "$all": [
+    { kind: "Component", "spec.lifecycle": "production" },
+    { "$any": [{ "spec.type": "service" }, { "spec.type": "website" }] },
+    { "metadata.annotations.backstage.io/techdocs-ref": { "$exists": true } }
+  ] } }
+
+## Other Options
+
+Limit returned fields: { fields: ["kind", "metadata.name", "metadata.namespace"] }
+Sort results: { orderFields: { field: "metadata.name", order: "asc" } }
+Full text search: { fullTextFilter: { term: "auth", fields: ["metadata.name", "metadata.title"] } }
+Pagination: Use limit (e.g. 20) and the returned nextPageCursor for subsequent requests via cursor.
+`;
+const INLINE_MODEL_DESCRIPTION = `
 Query entities from the Backstage Software Catalog using predicate filters.
 
 ## Catalog Model
@@ -58,43 +83,26 @@ Entities have bidirectional relations stored in the "relations" array. Common re
 Relations can be queried via "relations.<type>" e.g. "relations.ownedby: user:default/jane-doe". The value there must always be a valid entity reference.
 
 When querying for entity relationships, prefer using relations over spec fields. For example, use "relations.ownedby" instead of "spec.owner" to find entities owned by a particular group or user.
+${QUERY_SYNTAX}`;
+const MODEL_REFERENCE_DESCRIPTION = `
+Query entities from the Backstage Software Catalog using predicate filters.
 
-## Query Syntax
-
-The query uses predicate expressions with dot-notation field paths.
-
-Simple matching:
-  { query: { kind: "Component" } }
-  { query: { kind: "Component", "spec.type": "service" } }
-
-Value operators:
-  { query: { kind: { "$in": ["API", "Component"] } } }
-  { query: { "metadata.annotations.backstage.io/techdocs-ref": { "$exists": true } } }
-  { query: { "metadata.tags": { "$contains": "java" } } }
-  { query: { "metadata.name": { "$hasPrefix": "team-" } } }
-
-Logical operators:
-  { query: { "$all": [{ kind: "Component" }, { "spec.lifecycle": "production" }] } }
-  { query: { "$any": [{ "spec.type": "service" }, { "spec.type": "website" }] } }
-  { query: { "$not": { kind: "Group" } } }
-
-Querying relations - find all entities owned by a specific group:
-  { query: { "relations.ownedby": "group:default/team-alpha" } }
-
-Combined example - find production services or websites with TechDocs:
-  { query: { "$all": [
-    { kind: "Component", "spec.lifecycle": "production" },
-    { "$any": [{ "spec.type": "service" }, { "spec.type": "website" }] },
-    { "metadata.annotations.backstage.io/techdocs-ref": { "$exists": true } }
-  ] } }
-
-## Other Options
-
-Limit returned fields: { fields: ["kind", "metadata.name", "metadata.namespace"] }
-Sort results: { orderFields: { field: "metadata.name", order: "asc" } }
-Full text search: { fullTextFilter: { term: "auth", fields: ["metadata.name", "metadata.title"] } }
-Pagination: Use limit (e.g. 20) and the returned nextPageCursor for subsequent requests via cursor.
-    `,
+For a complete list of entity kinds, fields, relations, and other queryable attributes available in the catalog, use \`get-catalog-model-description\`.
+${QUERY_SYNTAX}`;
+const createQueryCatalogEntitiesAction = ({
+  catalog,
+  actionsRegistry,
+  useExperimentalCatalogLayersDescriptions
+}) => {
+  actionsRegistry.register({
+    name: "query-catalog-entities",
+    title: "Query Catalog Entities",
+    attributes: {
+      destructive: false,
+      readOnly: true,
+      idempotent: true
+    },
+    description: useExperimentalCatalogLayersDescriptions ? MODEL_REFERENCE_DESCRIPTION : INLINE_MODEL_DESCRIPTION,
     schema: {
       input: (z) => z.object({
         query: filterPredicates.createZodV3FilterPredicateSchema(z).optional().describe(

package/dist/actions/createQueryCatalogEntitiesAction.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"createQueryCatalogEntitiesAction.cjs.js","sources":["../../src/actions/createQueryCatalogEntitiesAction.ts"],"sourcesContent":["/*\n * Copyright 2025 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\nimport { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport { CatalogService } from '@backstage/plugin-catalog-node';\nimport { createZodV3FilterPredicateSchema } from '@backstage/filter-predicates';\n\nexport const createQueryCatalogEntitiesAction = ({\n  catalog,\n  actionsRegistry,\n}: {\n  catalog: CatalogService;\n  actionsRegistry: ActionsRegistryService;\n}) => {\n  actionsRegistry.register({\n    name: 'query-catalog-entities',\n    title: 'Query Catalog Entities',\n    attributes: {\n      destructive: false,\n      readOnly: true,\n      idempotent: true,\n    },\n    description: `\nQuery entities from the Backstage Software Catalog using predicate filters.\n\n## Catalog Model\n\nThe catalog contains entities of different kinds. Every entity has \"kind\", \"apiVersion\", \"metadata\", and optionally \"spec\" and \"relations\". Fields use dot notation for querying.\n\nCommon metadata fields on all entities: name, namespace (default: \"default\"), title, description, labels, annotations, tags (string array), links.\n\nEntity references use the format \"kind:namespace/name\", e.g. \"component:default/my-service\" or \"user:default/jane.doe\".\n\n### Entity Kinds\n\n**Component** - A piece of software such as a service, website, or library.\n  spec fields: type (e.g. \"service\", \"website\", \"library\"), lifecycle (e.g. \"production\", \"experimental\", \"deprecated\"), owner (entity ref), system, subcomponentOf, providesApis, consumesApis, dependsOn, dependencyOf.\n\n**API** - An interface that components expose, such as REST APIs or event streams.\n  spec fields: type (e.g. \"openapi\", \"asyncapi\", \"graphql\", \"grpc\"), lifecycle, owner (entity ref), definition (the API spec content), system.\n\n**System** - A collection of components, APIs, and resources that together expose some functionality.\n  spec fields: owner (entity ref), domain, type.\n\n**Domain** - A grouping of systems that share terminology, domain models, and business purpose.\n  spec fields: owner (entity ref), subdomainOf, type.\n\n**Resource** - Infrastructure required to operate a component, such as databases or storage buckets.\n  spec fields: type, owner (entity ref), system, dependsOn, dependencyOf.\n\n**Group** - An organizational entity such as a team or business unit.\n  spec fields: type (e.g. \"team\", \"business-unit\"), children (entity refs), parent (entity ref), members (entity refs), profile (displayName, email, picture).\n\n**User** - A person, such as an employee or contractor.\n  spec fields: memberOf (entity refs), profile (displayName, email, picture).\n\n**Location** - A marker that references other catalog descriptor files to be ingested.\n  spec fields: type, target, targets, presence.\n\n### Relations\n\nEntities have bidirectional relations stored in the \"relations\" array. Common relation types: ownedBy/ownerOf, dependsOn/dependencyOf, providesApi/apiProvidedBy, consumesApi/apiConsumedBy, parentOf/childOf, memberOf/hasMember, partOf/hasPart.\n\nRelations can be queried via \"relations.<type>\" e.g. \"relations.ownedby: user:default/jane-doe\". The value there must always be a valid entity reference.\n\nWhen querying for entity relationships, prefer using relations over spec fields. For example, use \"relations.ownedby\" instead of \"spec.owner\" to find entities owned by a particular group or user.\n\n## Query Syntax\n\nThe query uses predicate expressions with dot-notation field paths.\n\nSimple matching:\n  { query: { kind: \"Component\" } }\n  { query: { kind: \"Component\", \"spec.type\": \"service\" } }\n\nValue operators:\n  { query: { kind: { \"$in\": [\"API\", \"Component\"] } } }\n  { query: { \"metadata.annotations.backstage.io/techdocs-ref\": { \"$exists\": true } } }\n  { query: { \"metadata.tags\": { \"$contains\": \"java\" } } }\n  { query: { \"metadata.name\": { \"$hasPrefix\": \"team-\" } } }\n\nLogical operators:\n  { query: { \"$all\": [{ kind: \"Component\" }, { \"spec.lifecycle\": \"production\" }] } }\n  { query: { \"$any\": [{ \"spec.type\": \"service\" }, { \"spec.type\": \"website\" }] } }\n  { query: { \"$not\": { kind: \"Group\" } } }\n\nQuerying relations - find all entities owned by a specific group:\n  { query: { \"relations.ownedby\": \"group:default/team-alpha\" } }\n\nCombined example - find production services or websites with TechDocs:\n  { query: { \"$all\": [\n    { kind: \"Component\", \"spec.lifecycle\": \"production\" },\n    { \"$any\": [{ \"spec.type\": \"service\" }, { \"spec.type\": \"website\" }] },\n    { \"metadata.annotations.backstage.io/techdocs-ref\": { \"$exists\": true } }\n  ] } }\n\n## Other Options\n\nLimit returned fields: { fields: [\"kind\", \"metadata.name\", \"metadata.namespace\"] }\nSort results: { orderFields: { field: \"metadata.name\", order: \"asc\" } }\nFull text search: { fullTextFilter: { term: \"auth\", fields: [\"metadata.name\", \"metadata.title\"] } }\nPagination: Use limit (e.g. 20) and the returned nextPageCursor for subsequent requests via cursor.\n    `,\n    schema: {\n      input: z =>\n        z.object({\n          query: createZodV3FilterPredicateSchema(z)\n            .optional()\n            .describe(\n              'Entity predicate query. Supports field matching, $all, $any, $not, $exists, $in, $contains, and $hasPrefix operators.',\n            ),\n          fields: z\n            .array(z.string())\n            .optional()\n            .describe(\n              'Specific fields to include in the response. If not provided, all fields are returned. Each entry is a dot separated path into an entity, e.g. `spec.type`.',\n            ),\n          limit: z\n            .number()\n            .int()\n            .positive()\n            .optional()\n            .describe('Maximum number of entities to return at a time.'),\n          offset: z\n            .number()\n            .int()\n            .min(0)\n            .optional()\n            .describe('Number of entities to skip before returning results.'),\n          orderFields: z\n            .union([\n              z.object({\n                field: z\n                  .string()\n                  .describe(\n                    'Field to order by. The format is a dot separated path into an entity, e.g. `spec.type`.',\n                  ),\n                order: z.enum(['asc', 'desc']).describe('Sort order'),\n              }),\n              z.array(\n                z.object({\n                  field: z\n                    .string()\n                    .describe(\n                      'Field to order by. The format is a dot separated path into an entity, e.g. `spec.type`.',\n                    ),\n                  order: z.enum(['asc', 'desc']).describe('Sort order'),\n                }),\n              ),\n            ])\n            .optional()\n            .describe(\n              'Ordering criteria for the results. Can be a single order directive or an array for multi-field sorting.',\n            ),\n          fullTextFilter: z\n            .object({\n              term: z.string().describe('Full text search term'),\n              fields: z\n                .array(z.string())\n                .optional()\n                .describe(\n                  'Fields to search within. Each entry is a dot separated path into an entity, e.g. `spec.type`.',\n                ),\n            })\n            .optional()\n            .describe('Full text search criteria'),\n          cursor: z\n            .string()\n            .optional()\n            .describe(\n              'Cursor for pagination. This can be used only after the first request with a response containing a cursor. If a cursor is given it takes precedence over `offset`.',\n            ),\n        }),\n      output: z =>\n        z.object({\n          items: z\n            .array(z.object({}).passthrough())\n            .describe('List of entities'),\n          totalItems: z.number().describe('Total number of entities'),\n          hasMoreEntities: z\n            .boolean()\n            .describe('Whether more entities are available'),\n          nextPageCursor: z\n            .string()\n            .optional()\n            .describe('Next page cursor used to fetch next page of entities'),\n        }),\n    },\n    action: async ({ input, credentials }) => {\n      const response = await catalog.queryEntities(\n        {\n          ...input,\n          query: input.query,\n        },\n        { credentials },\n      );\n\n      return {\n        output: {\n          items: response.items,\n          totalItems: response.totalItems,\n          hasMoreEntities: !!response.pageInfo.nextCursor,\n          nextPageCursor: response.pageInfo.nextCursor,\n        },\n      };\n    },\n  });\n};\n"],"names":["createZodV3FilterPredicateSchema"],"mappings":";;;;AAmBO,MAAM,mCAAmC,CAAC;AAAA,EAC/C,OAAA;AAAA,EACA;AACF,CAAA,KAGM;AACJ,EAAA,eAAA,CAAgB,QAAA,CAAS;AAAA,IACvB,IAAA,EAAM,wBAAA;AAAA,IACN,KAAA,EAAO,wBAAA;AAAA,IACP,UAAA,EAAY;AAAA,MACV,WAAA,EAAa,KAAA;AAAA,MACb,QAAA,EAAU,IAAA;AAAA,MACV,UAAA,EAAY;AAAA,KACd;AAAA,IACA,WAAA,EAAa;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;AAAA;AAAA;;AAAA;AAAA;AAAA;AAAA;AAAA;;AAAA;AAAA;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;;AAAA;;AAAA;AAAA;AAAA;AAAA;AAAA,IAAA,CAAA;AAAA,IAiFb,MAAA,EAAQ;AAAA,MACN,KAAA,EAAO,CAAA,CAAA,KACL,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,KAAA,EAAOA,iDAAA,CAAiC,CAAC,CAAA,CACtC,UAAS,CACT,QAAA;AAAA,UACC;AAAA,SACF;AAAA,QACF,MAAA,EAAQ,EACL,KAAA,CAAM,CAAA,CAAE,QAAQ,CAAA,CAChB,UAAS,CACT,QAAA;AAAA,UACC;AAAA,SACF;AAAA,QACF,KAAA,EAAO,CAAA,CACJ,MAAA,EAAO,CACP,GAAA,EAAI,CACJ,QAAA,EAAS,CACT,QAAA,EAAS,CACT,QAAA,CAAS,iDAAiD,CAAA;AAAA,QAC7D,MAAA,EAAQ,CAAA,CACL,MAAA,EAAO,CACP,GAAA,EAAI,CACJ,GAAA,CAAI,CAAC,CAAA,CACL,QAAA,EAAS,CACT,QAAA,CAAS,sDAAsD,CAAA;AAAA,QAClE,WAAA,EAAa,EACV,KAAA,CAAM;AAAA,UACL,EAAE,MAAA,CAAO;AAAA,YACP,KAAA,EAAO,CAAA,CACJ,MAAA,EAAO,CACP,QAAA;AAAA,cACC;AAAA,aACF;AAAA,YACF,KAAA,EAAO,EAAE,IAAA,CAAK,CAAC,OAAO,MAAM,CAAC,CAAA,CAAE,QAAA,CAAS,YAAY;AAAA,WACrD,CAAA;AAAA,UACD,CAAA,CAAE,KAAA;AAAA,YACA,EAAE,MAAA,CAAO;AAAA,cACP,KAAA,EAAO,CAAA,CACJ,MAAA,EAAO,CACP,QAAA;AAAA,gBACC;AAAA,eACF;AAAA,cACF,KAAA,EAAO,EAAE,IAAA,CAAK,CAAC,OAAO,MAAM,CAAC,CAAA,CAAE,QAAA,CAAS,YAAY;AAAA,aACrD;AAAA;AACH,SACD,CAAA,CACA,QAAA,EAAS,CACT,QAAA;AAAA,UACC;AAAA,SACF;AAAA,QACF,cAAA,EAAgB,EACb,MAAA,CAAO;AAAA,UACN,IAAA,EAAM,CAAA,CAAE,MAAA,EAAO,CAAE,SAAS,uBAAuB,CAAA;AAAA,UACjD,MAAA,EAAQ,EACL,KAAA,CAAM,CAAA,CAAE,QAAQ,CAAA,CAChB,UAAS,CACT,QAAA;AAAA,YACC;AAAA;AACF,SACH,CAAA,CACA,QAAA,EAAS,CACT,SAAS,2BAA2B,CAAA;AAAA,QACvC,MAAA,EAAQ,CAAA,CACL,MAAA,EAAO,CACP,UAAS,CACT,QAAA;AAAA,UACC;AAAA;AACF,OACH,CAAA;AAAA,MACH,MAAA,EAAQ,CAAA,CAAA,KACN,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,KAAA,EAAO,CAAA,CACJ,KAAA,CAAM,CAAA,CAAE,MAAA,CAAO,EAAE,CAAA,CAAE,WAAA,EAAa,CAAA,CAChC,QAAA,CAAS,kBAAkB,CAAA;AAAA,QAC9B,UAAA,EAAY,CAAA,CAAE,MAAA,EAAO,CAAE,SAAS,0BAA0B,CAAA;AAAA,QAC1D,eAAA,EAAiB,CAAA,CACd,OAAA,EAAQ,CACR,SAAS,qCAAqC,CAAA;AAAA,QACjD,gBAAgB,CAAA,CACb,MAAA,GACA,QAAA,EAAS,CACT,SAAS,sDAAsD;AAAA,OACnE;AAAA,KACL;AAAA,IACA,MAAA,EAAQ,OAAO,EAAE,KAAA,EAAO,aAAY,KAAM;AACxC,MAAA,MAAM,QAAA,GAAW,MAAM,OAAA,CAAQ,aAAA;AAAA,QAC7B;AAAA,UACE,GAAG,KAAA;AAAA,UACH,OAAO,KAAA,CAAM;AAAA,SACf;AAAA,QACA,EAAE,WAAA;AAAY,OAChB;AAEA,MAAA,OAAO;AAAA,QACL,MAAA,EAAQ;AAAA,UACN,OAAO,QAAA,CAAS,KAAA;AAAA,UAChB,YAAY,QAAA,CAAS,UAAA;AAAA,UACrB,eAAA,EAAiB,CAAC,CAAC,QAAA,CAAS,QAAA,CAAS,UAAA;AAAA,UACrC,cAAA,EAAgB,SAAS,QAAA,CAAS;AAAA;AACpC,OACF;AAAA,IACF;AAAA,GACD,CAAA;AACH;;;;"}
+{"version":3,"file":"createQueryCatalogEntitiesAction.cjs.js","sources":["../../src/actions/createQueryCatalogEntitiesAction.ts"],"sourcesContent":["/*\n * Copyright 2025 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\nimport { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport { CatalogService } from '@backstage/plugin-catalog-node';\nimport { createZodV3FilterPredicateSchema } from '@backstage/filter-predicates';\n\nconst QUERY_SYNTAX = `\n## Query Syntax\n\nThe query uses predicate expressions with dot-notation field paths.\n\nSimple matching:\n  { query: { kind: \"Component\" } }\n  { query: { kind: \"Component\", \"spec.type\": \"service\" } }\n\nValue operators:\n  { query: { kind: { \"$in\": [\"API\", \"Component\"] } } }\n  { query: { \"metadata.annotations.backstage.io/techdocs-ref\": { \"$exists\": true } } }\n  { query: { \"metadata.tags\": { \"$contains\": \"java\" } } }\n  { query: { \"metadata.name\": { \"$hasPrefix\": \"team-\" } } }\n\nLogical operators:\n  { query: { \"$all\": [{ kind: \"Component\" }, { \"spec.lifecycle\": \"production\" }] } }\n  { query: { \"$any\": [{ \"spec.type\": \"service\" }, { \"spec.type\": \"website\" }] } }\n  { query: { \"$not\": { kind: \"Group\" } } }\n\nQuerying relations - find all entities owned by a specific group:\n  { query: { \"relations.ownedby\": \"group:default/team-alpha\" } }\n\nCombined example - find production services or websites with TechDocs:\n  { query: { \"$all\": [\n    { kind: \"Component\", \"spec.lifecycle\": \"production\" },\n    { \"$any\": [{ \"spec.type\": \"service\" }, { \"spec.type\": \"website\" }] },\n    { \"metadata.annotations.backstage.io/techdocs-ref\": { \"$exists\": true } }\n  ] } }\n\n## Other Options\n\nLimit returned fields: { fields: [\"kind\", \"metadata.name\", \"metadata.namespace\"] }\nSort results: { orderFields: { field: \"metadata.name\", order: \"asc\" } }\nFull text search: { fullTextFilter: { term: \"auth\", fields: [\"metadata.name\", \"metadata.title\"] } }\nPagination: Use limit (e.g. 20) and the returned nextPageCursor for subsequent requests via cursor.\n`;\n\nconst INLINE_MODEL_DESCRIPTION = `\nQuery entities from the Backstage Software Catalog using predicate filters.\n\n## Catalog Model\n\nThe catalog contains entities of different kinds. Every entity has \"kind\", \"apiVersion\", \"metadata\", and optionally \"spec\" and \"relations\". Fields use dot notation for querying.\n\nCommon metadata fields on all entities: name, namespace (default: \"default\"), title, description, labels, annotations, tags (string array), links.\n\nEntity references use the format \"kind:namespace/name\", e.g. \"component:default/my-service\" or \"user:default/jane.doe\".\n\n### Entity Kinds\n\n**Component** - A piece of software such as a service, website, or library.\n  spec fields: type (e.g. \"service\", \"website\", \"library\"), lifecycle (e.g. \"production\", \"experimental\", \"deprecated\"), owner (entity ref), system, subcomponentOf, providesApis, consumesApis, dependsOn, dependencyOf.\n\n**API** - An interface that components expose, such as REST APIs or event streams.\n  spec fields: type (e.g. \"openapi\", \"asyncapi\", \"graphql\", \"grpc\"), lifecycle, owner (entity ref), definition (the API spec content), system.\n\n**System** - A collection of components, APIs, and resources that together expose some functionality.\n  spec fields: owner (entity ref), domain, type.\n\n**Domain** - A grouping of systems that share terminology, domain models, and business purpose.\n  spec fields: owner (entity ref), subdomainOf, type.\n\n**Resource** - Infrastructure required to operate a component, such as databases or storage buckets.\n  spec fields: type, owner (entity ref), system, dependsOn, dependencyOf.\n\n**Group** - An organizational entity such as a team or business unit.\n  spec fields: type (e.g. \"team\", \"business-unit\"), children (entity refs), parent (entity ref), members (entity refs), profile (displayName, email, picture).\n\n**User** - A person, such as an employee or contractor.\n  spec fields: memberOf (entity refs), profile (displayName, email, picture).\n\n**Location** - A marker that references other catalog descriptor files to be ingested.\n  spec fields: type, target, targets, presence.\n\n### Relations\n\nEntities have bidirectional relations stored in the \"relations\" array. Common relation types: ownedBy/ownerOf, dependsOn/dependencyOf, providesApi/apiProvidedBy, consumesApi/apiConsumedBy, parentOf/childOf, memberOf/hasMember, partOf/hasPart.\n\nRelations can be queried via \"relations.<type>\" e.g. \"relations.ownedby: user:default/jane-doe\". The value there must always be a valid entity reference.\n\nWhen querying for entity relationships, prefer using relations over spec fields. For example, use \"relations.ownedby\" instead of \"spec.owner\" to find entities owned by a particular group or user.\n${QUERY_SYNTAX}`;\n\nconst MODEL_REFERENCE_DESCRIPTION = `\nQuery entities from the Backstage Software Catalog using predicate filters.\n\nFor a complete list of entity kinds, fields, relations, and other queryable attributes available in the catalog, use \\`get-catalog-model-description\\`.\n${QUERY_SYNTAX}`;\n\nexport const createQueryCatalogEntitiesAction = ({\n  catalog,\n  actionsRegistry,\n  useExperimentalCatalogLayersDescriptions,\n}: {\n  catalog: CatalogService;\n  actionsRegistry: ActionsRegistryService;\n  useExperimentalCatalogLayersDescriptions?: boolean;\n}) => {\n  actionsRegistry.register({\n    name: 'query-catalog-entities',\n    title: 'Query Catalog Entities',\n    attributes: {\n      destructive: false,\n      readOnly: true,\n      idempotent: true,\n    },\n    description: useExperimentalCatalogLayersDescriptions\n      ? MODEL_REFERENCE_DESCRIPTION\n      : INLINE_MODEL_DESCRIPTION,\n    schema: {\n      input: z =>\n        z.object({\n          query: createZodV3FilterPredicateSchema(z)\n            .optional()\n            .describe(\n              'Entity predicate query. Supports field matching, $all, $any, $not, $exists, $in, $contains, and $hasPrefix operators.',\n            ),\n          fields: z\n            .array(z.string())\n            .optional()\n            .describe(\n              'Specific fields to include in the response. If not provided, all fields are returned. Each entry is a dot separated path into an entity, e.g. `spec.type`.',\n            ),\n          limit: z\n            .number()\n            .int()\n            .positive()\n            .optional()\n            .describe('Maximum number of entities to return at a time.'),\n          offset: z\n            .number()\n            .int()\n            .min(0)\n            .optional()\n            .describe('Number of entities to skip before returning results.'),\n          orderFields: z\n            .union([\n              z.object({\n                field: z\n                  .string()\n                  .describe(\n                    'Field to order by. The format is a dot separated path into an entity, e.g. `spec.type`.',\n                  ),\n                order: z.enum(['asc', 'desc']).describe('Sort order'),\n              }),\n              z.array(\n                z.object({\n                  field: z\n                    .string()\n                    .describe(\n                      'Field to order by. The format is a dot separated path into an entity, e.g. `spec.type`.',\n                    ),\n                  order: z.enum(['asc', 'desc']).describe('Sort order'),\n                }),\n              ),\n            ])\n            .optional()\n            .describe(\n              'Ordering criteria for the results. Can be a single order directive or an array for multi-field sorting.',\n            ),\n          fullTextFilter: z\n            .object({\n              term: z.string().describe('Full text search term'),\n              fields: z\n                .array(z.string())\n                .optional()\n                .describe(\n                  'Fields to search within. Each entry is a dot separated path into an entity, e.g. `spec.type`.',\n                ),\n            })\n            .optional()\n            .describe('Full text search criteria'),\n          cursor: z\n            .string()\n            .optional()\n            .describe(\n              'Cursor for pagination. This can be used only after the first request with a response containing a cursor. If a cursor is given it takes precedence over `offset`.',\n            ),\n        }),\n      output: z =>\n        z.object({\n          items: z\n            .array(z.object({}).passthrough())\n            .describe('List of entities'),\n          totalItems: z.number().describe('Total number of entities'),\n          hasMoreEntities: z\n            .boolean()\n            .describe('Whether more entities are available'),\n          nextPageCursor: z\n            .string()\n            .optional()\n            .describe('Next page cursor used to fetch next page of entities'),\n        }),\n    },\n    action: async ({ input, credentials }) => {\n      const response = await catalog.queryEntities(\n        {\n          ...input,\n          query: input.query,\n        },\n        { credentials },\n      );\n\n      return {\n        output: {\n          items: response.items,\n          totalItems: response.totalItems,\n          hasMoreEntities: !!response.pageInfo.nextCursor,\n          nextPageCursor: response.pageInfo.nextCursor,\n        },\n      };\n    },\n  });\n};\n"],"names":["createZodV3FilterPredicateSchema"],"mappings":";;;;AAmBA,MAAM,YAAA,GAAe;AAAA;;AAAA;;AAAA;AAAA;AAAA;;AAAA;AAAA;AAAA;AAAA;AAAA;;AAAA;AAAA;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;;AAAA;;AAAA;AAAA;AAAA;AAAA;AAAA,CAAA;AAsCrB,MAAM,wBAAA,GAA2B;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;AAAA;;AAAA;;AAAA;;AAAA;;AAAA;AAAA,EA4C/B,YAAY,CAAA,CAAA;AAEd,MAAM,2BAAA,GAA8B;AAAA;;AAAA;AAAA,EAIlC,YAAY,CAAA,CAAA;AAEP,MAAM,mCAAmC,CAAC;AAAA,EAC/C,OAAA;AAAA,EACA,eAAA;AAAA,EACA;AACF,CAAA,KAIM;AACJ,EAAA,eAAA,CAAgB,QAAA,CAAS;AAAA,IACvB,IAAA,EAAM,wBAAA;AAAA,IACN,KAAA,EAAO,wBAAA;AAAA,IACP,UAAA,EAAY;AAAA,MACV,WAAA,EAAa,KAAA;AAAA,MACb,QAAA,EAAU,IAAA;AAAA,MACV,UAAA,EAAY;AAAA,KACd;AAAA,IACA,WAAA,EAAa,2CACT,2BAAA,GACA,wBAAA;AAAA,IACJ,MAAA,EAAQ;AAAA,MACN,KAAA,EAAO,CAAA,CAAA,KACL,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,KAAA,EAAOA,iDAAA,CAAiC,CAAC,CAAA,CACtC,UAAS,CACT,QAAA;AAAA,UACC;AAAA,SACF;AAAA,QACF,MAAA,EAAQ,EACL,KAAA,CAAM,CAAA,CAAE,QAAQ,CAAA,CAChB,UAAS,CACT,QAAA;AAAA,UACC;AAAA,SACF;AAAA,QACF,KAAA,EAAO,CAAA,CACJ,MAAA,EAAO,CACP,GAAA,EAAI,CACJ,QAAA,EAAS,CACT,QAAA,EAAS,CACT,QAAA,CAAS,iDAAiD,CAAA;AAAA,QAC7D,MAAA,EAAQ,CAAA,CACL,MAAA,EAAO,CACP,GAAA,EAAI,CACJ,GAAA,CAAI,CAAC,CAAA,CACL,QAAA,EAAS,CACT,QAAA,CAAS,sDAAsD,CAAA;AAAA,QAClE,WAAA,EAAa,EACV,KAAA,CAAM;AAAA,UACL,EAAE,MAAA,CAAO;AAAA,YACP,KAAA,EAAO,CAAA,CACJ,MAAA,EAAO,CACP,QAAA;AAAA,cACC;AAAA,aACF;AAAA,YACF,KAAA,EAAO,EAAE,IAAA,CAAK,CAAC,OAAO,MAAM,CAAC,CAAA,CAAE,QAAA,CAAS,YAAY;AAAA,WACrD,CAAA;AAAA,UACD,CAAA,CAAE,KAAA;AAAA,YACA,EAAE,MAAA,CAAO;AAAA,cACP,KAAA,EAAO,CAAA,CACJ,MAAA,EAAO,CACP,QAAA;AAAA,gBACC;AAAA,eACF;AAAA,cACF,KAAA,EAAO,EAAE,IAAA,CAAK,CAAC,OAAO,MAAM,CAAC,CAAA,CAAE,QAAA,CAAS,YAAY;AAAA,aACrD;AAAA;AACH,SACD,CAAA,CACA,QAAA,EAAS,CACT,QAAA;AAAA,UACC;AAAA,SACF;AAAA,QACF,cAAA,EAAgB,EACb,MAAA,CAAO;AAAA,UACN,IAAA,EAAM,CAAA,CAAE,MAAA,EAAO,CAAE,SAAS,uBAAuB,CAAA;AAAA,UACjD,MAAA,EAAQ,EACL,KAAA,CAAM,CAAA,CAAE,QAAQ,CAAA,CAChB,UAAS,CACT,QAAA;AAAA,YACC;AAAA;AACF,SACH,CAAA,CACA,QAAA,EAAS,CACT,SAAS,2BAA2B,CAAA;AAAA,QACvC,MAAA,EAAQ,CAAA,CACL,MAAA,EAAO,CACP,UAAS,CACT,QAAA;AAAA,UACC;AAAA;AACF,OACH,CAAA;AAAA,MACH,MAAA,EAAQ,CAAA,CAAA,KACN,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,KAAA,EAAO,CAAA,CACJ,KAAA,CAAM,CAAA,CAAE,MAAA,CAAO,EAAE,CAAA,CAAE,WAAA,EAAa,CAAA,CAChC,QAAA,CAAS,kBAAkB,CAAA;AAAA,QAC9B,UAAA,EAAY,CAAA,CAAE,MAAA,EAAO,CAAE,SAAS,0BAA0B,CAAA;AAAA,QAC1D,eAAA,EAAiB,CAAA,CACd,OAAA,EAAQ,CACR,SAAS,qCAAqC,CAAA;AAAA,QACjD,gBAAgB,CAAA,CACb,MAAA,GACA,QAAA,EAAS,CACT,SAAS,sDAAsD;AAAA,OACnE;AAAA,KACL;AAAA,IACA,MAAA,EAAQ,OAAO,EAAE,KAAA,EAAO,aAAY,KAAM;AACxC,MAAA,MAAM,QAAA,GAAW,MAAM,OAAA,CAAQ,aAAA;AAAA,QAC7B;AAAA,UACE,GAAG,KAAA;AAAA,UACH,OAAO,KAAA,CAAM;AAAA,SACf;AAAA,QACA,EAAE,WAAA;AAAY,OAChB;AAEA,MAAA,OAAO;AAAA,QACL,MAAA,EAAQ;AAAA,UACN,OAAO,QAAA,CAAS,KAAA;AAAA,UAChB,YAAY,QAAA,CAAS,UAAA;AAAA,UACrB,eAAA,EAAiB,CAAC,CAAC,QAAA,CAAS,QAAA,CAAS,UAAA;AAAA,UACrC,cAAA,EAAgB,SAAS,QAAA,CAAS;AAAA;AACpC,OACF;AAAA,IACF;AAAA,GACD,CAAA;AACH;;;;"}

package/dist/actions/index.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.cjs.js","sources":["../../src/actions/index.ts"],"sourcesContent":["/*\n * Copyright 2025 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport { CatalogService } from '@backstage/plugin-catalog-node';\nimport { ModelHolder } from '../model/ModelHolder';\nimport { createGetCatalogModelDescriptionAction } from './createGetCatalogModelDescriptionAction.ts';\nimport { createGetCatalogEntityAction } from './createGetCatalogEntityAction.ts';\nimport { createValidateEntityAction } from './createValidateEntityAction.ts';\nimport { createRegisterCatalogEntitiesAction } from './createRegisterCatalogEntitiesAction.ts';\nimport { createUnregisterCatalogEntitiesAction } from './createUnregisterCatalogEntitiesAction.ts';\nimport { createQueryCatalogEntitiesAction } from './createQueryCatalogEntitiesAction.ts';\n\nexport const createCatalogActions = (options: {\n  actionsRegistry: ActionsRegistryService;\n  catalog: CatalogService;\n  modelHolder: ModelHolder | undefined;\n}) => {\n  createGetCatalogModelDescriptionAction(options);\n  createGetCatalogEntityAction(options);\n  createValidateEntityAction(options);\n  createRegisterCatalogEntitiesAction(options);\n  createUnregisterCatalogEntitiesAction(options);\n  createQueryCatalogEntitiesAction(options);\n};\n"],"names":["createGetCatalogModelDescriptionAction","createGetCatalogEntityAction","createValidateEntityAction","createRegisterCatalogEntitiesAction","createUnregisterCatalogEntitiesAction","createQueryCatalogEntitiesAction"],"mappings":";;;;;;;;;AA0BO,MAAM,oBAAA,GAAuB,CAAC,OAAA,KAI/B;AACJ,EAAAA,6EAAA,CAAuC,OAAO,CAAA;AAC9C,EAAAC,yDAAA,CAA6B,OAAO,CAAA;AACpC,EAAAC,qDAAA,CAA2B,OAAO,CAAA;AAClC,EAAAC,uEAAA,CAAoC,OAAO,CAAA;AAC3C,EAAAC,2EAAA,CAAsC,OAAO,CAAA;AAC7C,EAAAC,iEAAA,CAAiC,OAAO,CAAA;AAC1C;;;;"}
+{"version":3,"file":"index.cjs.js","sources":["../../src/actions/index.ts"],"sourcesContent":["/*\n * Copyright 2025 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport { CatalogService } from '@backstage/plugin-catalog-node';\nimport { ModelHolder } from '../model/ModelHolder';\nimport { createGetCatalogModelDescriptionAction } from './createGetCatalogModelDescriptionAction.ts';\nimport { createGetCatalogEntityAction } from './createGetCatalogEntityAction.ts';\nimport { createValidateEntityAction } from './createValidateEntityAction.ts';\nimport { createRegisterCatalogEntitiesAction } from './createRegisterCatalogEntitiesAction.ts';\nimport { createUnregisterCatalogEntitiesAction } from './createUnregisterCatalogEntitiesAction.ts';\nimport { createQueryCatalogEntitiesAction } from './createQueryCatalogEntitiesAction.ts';\n\nexport const createCatalogActions = (options: {\n  actionsRegistry: ActionsRegistryService;\n  catalog: CatalogService;\n  modelHolder: ModelHolder | undefined;\n  useExperimentalCatalogLayersDescriptions?: boolean;\n}) => {\n  createGetCatalogModelDescriptionAction(options);\n  createGetCatalogEntityAction(options);\n  createValidateEntityAction(options);\n  createRegisterCatalogEntitiesAction(options);\n  createUnregisterCatalogEntitiesAction(options);\n  createQueryCatalogEntitiesAction(options);\n};\n"],"names":["createGetCatalogModelDescriptionAction","createGetCatalogEntityAction","createValidateEntityAction","createRegisterCatalogEntitiesAction","createUnregisterCatalogEntitiesAction","createQueryCatalogEntitiesAction"],"mappings":";;;;;;;;;AA0BO,MAAM,oBAAA,GAAuB,CAAC,OAAA,KAK/B;AACJ,EAAAA,6EAAA,CAAuC,OAAO,CAAA;AAC9C,EAAAC,yDAAA,CAA6B,OAAO,CAAA;AACpC,EAAAC,qDAAA,CAA2B,OAAO,CAAA;AAClC,EAAAC,uEAAA,CAAoC,OAAO,CAAA;AAC3C,EAAAC,2EAAA,CAAsC,OAAO,CAAA;AAC7C,EAAAC,iEAAA,CAAiC,OAAO,CAAA;AAC1C;;;;"}

package/dist/database/operations/stitcher/performStitching.cjs.js

@@ -2,12 +2,20 @@
 
 var catalogClient = require('@backstage/catalog-client');
 var catalogModel = require('@backstage/catalog-model');
+var node_crypto = require('node:crypto');
+var stableStringify = require('fast-json-stable-stringify');
 var buildEntitySearch = require('./buildEntitySearch.cjs.js');
 var markDeferredStitchCompleted = require('./markDeferredStitchCompleted.cjs.js');
 var syncSearchRows = require('./syncSearchRows.cjs.js');
-var util = require('./util.cjs.js');
 var backendPluginApi = require('@backstage/backend-plugin-api');
 
+function _interopDefaultCompat (e) { return e && typeof e === 'object' && 'default' in e ? e : { default: e }; }
+
+var stableStringify__default = /*#__PURE__*/_interopDefaultCompat(stableStringify);
+
+function generateStableHash(entity) {
+  return node_crypto.createHash("sha1").update(stableStringify__default.default({ ...entity })).digest("hex");
+}
 const scriptProtocolPattern = (
   // eslint-disable-next-line no-control-regex
   /^[\u0000-\u001F ]*j[\r\n\t]*a[\r\n\t]*v[\r\n\t]*a[\r\n\t]*s[\r\n\t]*c[\r\n\t]*r[\r\n\t]*i[\r\n\t]*p[\r\n\t]*t[\r\n\t]*\:/i
@@ -110,7 +118,7 @@ async function performStitching(options) {
         items: [...entity.status?.items ?? [], ...statusItems]
       };
     }
-    const hash = util.generateStableHash(entity);
+    const hash = generateStableHash(entity);
     if (hash === previousHash) {
       logger.debug(`Skipped stitching of ${entityRef}, no changes`);
       return "unchanged";

package/dist/database/operations/stitcher/performStitching.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"performStitching.cjs.js","sources":["../../../../src/database/operations/stitcher/performStitching.ts"],"sourcesContent":["/*\n * Copyright 2023 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { ENTITY_STATUS_CATALOG_PROCESSING_TYPE } from '@backstage/catalog-client';\nimport {\n  ANNOTATION_EDIT_URL,\n  ANNOTATION_VIEW_URL,\n  EntityRelation,\n} from '@backstage/catalog-model';\nimport { AlphaEntity, EntityStatusItem } from '@backstage/catalog-model/alpha';\nimport { SerializedError } from '@backstage/errors';\nimport { Knex } from 'knex';\nimport { StitchingStrategy } from '../../../stitching/types';\nimport {\n  DbFinalEntitiesRow,\n  DbRefreshStateRow,\n  DbStitchQueueRow,\n} from '../../tables';\nimport { buildEntitySearch } from './buildEntitySearch';\nimport { markDeferredStitchCompleted } from './markDeferredStitchCompleted';\nimport { syncSearchRows } from './syncSearchRows';\nimport { generateStableHash } from './util';\nimport {\n  LoggerService,\n  isDatabaseConflictError,\n} from '@backstage/backend-plugin-api';\n\n// See https://github.com/facebook/react/blob/f0cf832e1d0c8544c36aa8b310960885a11a847c/packages/react-dom-bindings/src/shared/sanitizeURL.js\nconst scriptProtocolPattern =\n  // eslint-disable-next-line no-control-regex\n  /^[\\u0000-\\u001F ]*j[\\r\\n\\t]*a[\\r\\n\\t]*v[\\r\\n\\t]*a[\\r\\n\\t]*s[\\r\\n\\t]*c[\\r\\n\\t]*r[\\r\\n\\t]*i[\\r\\n\\t]*p[\\r\\n\\t]*t[\\r\\n\\t]*\\:/i;\n\n/**\n * Performs the act of stitching - to take all of the various outputs from the\n * ingestion process, and stitching them together into the final entity JSON\n * shape.\n */\nexport async function performStitching(options: {\n  knex: Knex | Knex.Transaction;\n  logger: LoggerService;\n  strategy: StitchingStrategy;\n  entityRef: string;\n  stitchTicket?: string;\n}): Promise<'changed' | 'unchanged' | 'abandoned'> {\n  const { knex, logger, entityRef } = options;\n  const stitchTicket = options.stitchTicket;\n\n  // In deferred mode, the entity is removed from the stitch queue on ANY\n  // completion, except when an exception is thrown. In the latter case, the\n  // entity will be retried at a later time.\n  let removeFromStitchQueueOnCompletion = options.strategy.mode === 'deferred';\n\n  try {\n    const entityResult = await knex<DbRefreshStateRow>('refresh_state')\n      .where({ entity_ref: entityRef })\n      .limit(1)\n      .select('entity_id');\n    if (!entityResult.length) {\n      // Entity does no exist in refresh state table, no stitching required.\n      return 'abandoned';\n    }\n\n    // Ensure that a final_entities row exists for this entity.\n    try {\n      await knex<DbFinalEntitiesRow>('final_entities')\n        .insert({\n          entity_id: entityResult[0].entity_id,\n          hash: '',\n          entity_ref: entityRef,\n        })\n        .onConflict('entity_id')\n        .ignore();\n    } catch (error) {\n      // It's possible to hit a race where a refresh_state table delete + insert\n      // is done just after we read the entity_id from it. This conflict is safe\n      // to ignore because the current stitching operation will be triggered by\n      // the old entry, and the new entry will trigger it's own stitching that\n      // will update the entity.\n      if (isDatabaseConflictError(error)) {\n        logger.debug(`Skipping stitching of ${entityRef}, conflict`, error);\n        return 'abandoned';\n      }\n\n      throw error;\n    }\n\n    // Selecting from refresh_state and final_entities should yield exactly\n    // one row (except in abnormal cases where the stitch was invoked for\n    // something that didn't exist at all, in which case it's zero rows).\n    // The join with the temporary incoming_references still gives one row.\n    const [processedResult, relationsResult] = await Promise.all([\n      knex\n        .with('incoming_references', function incomingReferences(builder) {\n          return builder\n            .from('refresh_state_references')\n            .where({ target_entity_ref: entityRef })\n            .count({ count: '*' });\n        })\n        .select({\n          entityId: 'refresh_state.entity_id',\n          processedEntity: 'refresh_state.processed_entity',\n          errors: 'refresh_state.errors',\n          incomingReferenceCount: 'incoming_references.count',\n          previousHash: 'final_entities.hash',\n        })\n        .from('refresh_state')\n        .where({ 'refresh_state.entity_ref': entityRef })\n        .crossJoin(knex.raw('incoming_references'))\n        .leftOuterJoin('final_entities', {\n          'final_entities.entity_id': 'refresh_state.entity_id',\n        }),\n      knex\n        .distinct({\n          relationType: 'type',\n          relationTarget: 'target_entity_ref',\n        })\n        .from('relations')\n        .where({ source_entity_ref: entityRef })\n        .orderBy('relationType', 'asc')\n        .orderBy('relationTarget', 'asc'),\n    ]);\n\n    // If there were no rows returned, it would mean that there was no\n    // matching row even in the refresh_state. This can happen for example\n    // if we emit a relation to something that hasn't been ingested yet.\n    // It's safe to ignore this stitch attempt in that case.\n    if (!processedResult.length) {\n      logger.debug(\n        `Unable to stitch ${entityRef}, item does not exist in refresh state table`,\n      );\n      return 'abandoned';\n    }\n\n    const {\n      entityId,\n      processedEntity,\n      errors,\n      incomingReferenceCount,\n      previousHash,\n    } = processedResult[0];\n\n    // If there was no processed entity in place, the target hasn't been\n    // through the processing steps yet. It's safe to ignore this stitch\n    // attempt in that case, since another stitch will be triggered when\n    // that processing has finished.\n    if (!processedEntity) {\n      logger.debug(\n        `Unable to stitch ${entityRef}, the entity has not yet been processed`,\n      );\n      return 'abandoned';\n    }\n\n    // Grab the processed entity and stitch all of the relevant data into\n    // it\n    const entity = JSON.parse(processedEntity) as AlphaEntity;\n    const isOrphan = Number(incomingReferenceCount) === 0;\n    let statusItems: EntityStatusItem[] = [];\n\n    if (isOrphan) {\n      logger.debug(`${entityRef} is an orphan`);\n      entity.metadata.annotations = {\n        ...entity.metadata.annotations,\n        ['backstage.io/orphan']: 'true',\n      };\n    }\n    if (errors) {\n      const parsedErrors = JSON.parse(errors) as SerializedError[];\n      if (Array.isArray(parsedErrors) && parsedErrors.length) {\n        statusItems = parsedErrors.map(e => ({\n          type: ENTITY_STATUS_CATALOG_PROCESSING_TYPE,\n          level: 'error',\n          message: `${e.name}: ${e.message}`,\n          error: e,\n        }));\n      }\n    }\n    // We opt to do this check here as we otherwise can't guarantee that it will be run after all processors\n    for (const annotation of [ANNOTATION_VIEW_URL, ANNOTATION_EDIT_URL]) {\n      const value = entity.metadata.annotations?.[annotation];\n      if (typeof value === 'string' && scriptProtocolPattern.test(value)) {\n        entity.metadata.annotations![annotation] =\n          'https://backstage.io/annotation-rejected-for-security-reasons';\n      }\n    }\n\n    // TODO: entityRef is lower case and should be uppercase in the final\n    // result\n    entity.relations = relationsResult\n      .filter(row => row.relationType /* exclude null row, if relevant */)\n      .map<EntityRelation>(row => ({\n        type: row.relationType!,\n        targetRef: row.relationTarget!,\n      }));\n    if (statusItems.length) {\n      entity.status = {\n        ...entity.status,\n        items: [...(entity.status?.items ?? []), ...statusItems],\n      };\n    }\n\n    // If the output entity was actually not changed, just abort\n    const hash = generateStableHash(entity);\n    if (hash === previousHash) {\n      logger.debug(`Skipped stitching of ${entityRef}, no changes`);\n      return 'unchanged';\n    }\n\n    entity.metadata.uid = entityId;\n    if (!entity.metadata.etag) {\n      // If the original data source did not have its own etag handling,\n      // use the hash as a good-quality etag\n      entity.metadata.etag = hash;\n    }\n\n    // This may throw if the entity is invalid, so we call it before\n    // the final_entities write, even though we may end up not needing\n    // to write the search index.\n    const searchEntries = buildEntitySearch(entityId, entity);\n\n    let updateQuery = knex<DbFinalEntitiesRow>('final_entities')\n      .update({\n        final_entity: JSON.stringify(entity),\n        hash,\n        last_updated_at: knex.fn.now(),\n      })\n      .where('entity_id', entityId);\n\n    // In deferred mode, guard against concurrent stitchers by checking that\n    // the stitch_ticket in stitch_queue still matches what we were given.\n    if (options.strategy.mode === 'deferred' && stitchTicket) {\n      updateQuery = updateQuery.whereExists(\n        knex<DbStitchQueueRow>('stitch_queue')\n          .where('stitch_queue.entity_ref', entityRef)\n          .where('stitch_queue.stitch_ticket', stitchTicket)\n          .select(knex.raw('1')),\n      );\n    }\n\n    const amountOfRowsChanged = await updateQuery;\n\n    if (amountOfRowsChanged === 0) {\n      logger.debug(`Entity ${entityRef} is already stitched, skipping write.`);\n      return 'abandoned';\n    }\n\n    await syncSearchRows(knex, entityId, searchEntries);\n\n    return 'changed';\n  } catch (error) {\n    removeFromStitchQueueOnCompletion = false;\n    throw error;\n  } finally {\n    if (removeFromStitchQueueOnCompletion && stitchTicket) {\n      await markDeferredStitchCompleted({\n        knex: knex,\n        entityRef,\n        stitchTicket,\n      });\n    }\n  }\n}\n"],"names":["isDatabaseConflictError","ENTITY_STATUS_CATALOG_PROCESSING_TYPE","ANNOTATION_VIEW_URL","ANNOTATION_EDIT_URL","generateStableHash","buildEntitySearch","syncSearchRows","markDeferredStitchCompleted"],"mappings":";;;;;;;;;;AAyCA,MAAM,qBAAA;AAAA;AAAA,EAEJ;AAAA,CAAA;AAOF,eAAsB,iBAAiB,OAAA,EAMY;AACjD,EAAA,MAAM,EAAE,IAAA,EAAM,MAAA,EAAQ,SAAA,EAAU,GAAI,OAAA;AACpC,EAAA,MAAM,eAAe,OAAA,CAAQ,YAAA;AAK7B,EAAA,IAAI,iCAAA,GAAoC,OAAA,CAAQ,QAAA,CAAS,IAAA,KAAS,UAAA;AAElE,EAAA,IAAI;AACF,IAAA,MAAM,YAAA,GAAe,MAAM,IAAA,CAAwB,eAAe,EAC/D,KAAA,CAAM,EAAE,UAAA,EAAY,SAAA,EAAW,CAAA,CAC/B,KAAA,CAAM,CAAC,CAAA,CACP,OAAO,WAAW,CAAA;AACrB,IAAA,IAAI,CAAC,aAAa,MAAA,EAAQ;AAExB,MAAA,OAAO,WAAA;AAAA,IACT;AAGA,IAAA,IAAI;AACF,MAAA,MAAM,IAAA,CAAyB,gBAAgB,CAAA,CAC5C,MAAA,CAAO;AAAA,QACN,SAAA,EAAW,YAAA,CAAa,CAAC,CAAA,CAAE,SAAA;AAAA,QAC3B,IAAA,EAAM,EAAA;AAAA,QACN,UAAA,EAAY;AAAA,OACb,CAAA,CACA,UAAA,CAAW,WAAW,EACtB,MAAA,EAAO;AAAA,IACZ,SAAS,KAAA,EAAO;AAMd,MAAA,IAAIA,wCAAA,CAAwB,KAAK,CAAA,EAAG;AAClC,QAAA,MAAA,CAAO,KAAA,CAAM,CAAA,sBAAA,EAAyB,SAAS,CAAA,UAAA,CAAA,EAAc,KAAK,CAAA;AAClE,QAAA,OAAO,WAAA;AAAA,MACT;AAEA,MAAA,MAAM,KAAA;AAAA,IACR;AAMA,IAAA,MAAM,CAAC,eAAA,EAAiB,eAAe,CAAA,GAAI,MAAM,QAAQ,GAAA,CAAI;AAAA,MAC3D,IAAA,CACG,IAAA,CAAK,qBAAA,EAAuB,SAAS,mBAAmB,OAAA,EAAS;AAChE,QAAA,OAAO,OAAA,CACJ,IAAA,CAAK,0BAA0B,CAAA,CAC/B,MAAM,EAAE,iBAAA,EAAmB,SAAA,EAAW,CAAA,CACtC,KAAA,CAAM,EAAE,KAAA,EAAO,KAAK,CAAA;AAAA,MACzB,CAAC,EACA,MAAA,CAAO;AAAA,QACN,QAAA,EAAU,yBAAA;AAAA,QACV,eAAA,EAAiB,gCAAA;AAAA,QACjB,MAAA,EAAQ,sBAAA;AAAA,QACR,sBAAA,EAAwB,2BAAA;AAAA,QACxB,YAAA,EAAc;AAAA,OACf,CAAA,CACA,IAAA,CAAK,eAAe,CAAA,CACpB,KAAA,CAAM,EAAE,0BAAA,EAA4B,SAAA,EAAW,CAAA,CAC/C,UAAU,IAAA,CAAK,GAAA,CAAI,qBAAqB,CAAC,CAAA,CACzC,cAAc,gBAAA,EAAkB;AAAA,QAC/B,0BAAA,EAA4B;AAAA,OAC7B,CAAA;AAAA,MACH,KACG,QAAA,CAAS;AAAA,QACR,YAAA,EAAc,MAAA;AAAA,QACd,cAAA,EAAgB;AAAA,OACjB,CAAA,CACA,IAAA,CAAK,WAAW,CAAA,CAChB,MAAM,EAAE,iBAAA,EAAmB,SAAA,EAAW,EACtC,OAAA,CAAQ,cAAA,EAAgB,KAAK,CAAA,CAC7B,OAAA,CAAQ,kBAAkB,KAAK;AAAA,KACnC,CAAA;AAMD,IAAA,IAAI,CAAC,gBAAgB,MAAA,EAAQ;AAC3B,MAAA,MAAA,CAAO,KAAA;AAAA,QACL,oBAAoB,SAAS,CAAA,4CAAA;AAAA,OAC/B;AACA,MAAA,OAAO,WAAA;AAAA,IACT;AAEA,IAAA,MAAM;AAAA,MACJ,QAAA;AAAA,MACA,eAAA;AAAA,MACA,MAAA;AAAA,MACA,sBAAA;AAAA,MACA;AAAA,KACF,GAAI,gBAAgB,CAAC,CAAA;AAMrB,IAAA,IAAI,CAAC,eAAA,EAAiB;AACpB,MAAA,MAAA,CAAO,KAAA;AAAA,QACL,oBAAoB,SAAS,CAAA,uCAAA;AAAA,OAC/B;AACA,MAAA,OAAO,WAAA;AAAA,IACT;AAIA,IAAA,MAAM,MAAA,GAAS,IAAA,CAAK,KAAA,CAAM,eAAe,CAAA;AACzC,IAAA,MAAM,QAAA,GAAW,MAAA,CAAO,sBAAsB,CAAA,KAAM,CAAA;AACpD,IAAA,IAAI,cAAkC,EAAC;AAEvC,IAAA,IAAI,QAAA,EAAU;AACZ,MAAA,MAAA,CAAO,KAAA,CAAM,CAAA,EAAG,SAAS,CAAA,aAAA,CAAe,CAAA;AACxC,MAAA,MAAA,CAAO,SAAS,WAAA,GAAc;AAAA,QAC5B,GAAG,OAAO,QAAA,CAAS,WAAA;AAAA,QACnB,CAAC,qBAAqB,GAAG;AAAA,OAC3B;AAAA,IACF;AACA,IAAA,IAAI,MAAA,EAAQ;AACV,MAAA,MAAM,YAAA,GAAe,IAAA,CAAK,KAAA,CAAM,MAAM,CAAA;AACtC,MAAA,IAAI,KAAA,CAAM,OAAA,CAAQ,YAAY,CAAA,IAAK,aAAa,MAAA,EAAQ;AACtD,QAAA,WAAA,GAAc,YAAA,CAAa,IAAI,CAAA,CAAA,MAAM;AAAA,UACnC,IAAA,EAAMC,mDAAA;AAAA,UACN,KAAA,EAAO,OAAA;AAAA,UACP,SAAS,CAAA,EAAG,CAAA,CAAE,IAAI,CAAA,EAAA,EAAK,EAAE,OAAO,CAAA,CAAA;AAAA,UAChC,KAAA,EAAO;AAAA,SACT,CAAE,CAAA;AAAA,MACJ;AAAA,IACF;AAEA,IAAA,KAAA,MAAW,UAAA,IAAc,CAACC,gCAAA,EAAqBC,gCAAmB,CAAA,EAAG;AACnE,MAAA,MAAM,KAAA,GAAQ,MAAA,CAAO,QAAA,CAAS,WAAA,GAAc,UAAU,CAAA;AACtD,MAAA,IAAI,OAAO,KAAA,KAAU,QAAA,IAAY,qBAAA,CAAsB,IAAA,CAAK,KAAK,CAAA,EAAG;AAClE,QAAA,MAAA,CAAO,QAAA,CAAS,WAAA,CAAa,UAAU,CAAA,GACrC,+DAAA;AAAA,MACJ;AAAA,IACF;AAIA,IAAA,MAAA,CAAO,YAAY,eAAA,CAChB,MAAA;AAAA,MAAO,SAAO,GAAA,CAAI;AAAA;AAAA,KAAgD,CAClE,IAAoB,CAAA,GAAA,MAAQ;AAAA,MAC3B,MAAM,GAAA,CAAI,YAAA;AAAA,MACV,WAAW,GAAA,CAAI;AAAA,KACjB,CAAE,CAAA;AACJ,IAAA,IAAI,YAAY,MAAA,EAAQ;AACtB,MAAA,MAAA,CAAO,MAAA,GAAS;AAAA,QACd,GAAG,MAAA,CAAO,MAAA;AAAA,QACV,KAAA,EAAO,CAAC,GAAI,MAAA,CAAO,QAAQ,KAAA,IAAS,EAAC,EAAI,GAAG,WAAW;AAAA,OACzD;AAAA,IACF;AAGA,IAAA,MAAM,IAAA,GAAOC,wBAAmB,MAAM,CAAA;AACtC,IAAA,IAAI,SAAS,YAAA,EAAc;AACzB,MAAA,MAAA,CAAO,KAAA,CAAM,CAAA,qBAAA,EAAwB,SAAS,CAAA,YAAA,CAAc,CAAA;AAC5D,MAAA,OAAO,WAAA;AAAA,IACT;AAEA,IAAA,MAAA,CAAO,SAAS,GAAA,GAAM,QAAA;AACtB,IAAA,IAAI,CAAC,MAAA,CAAO,QAAA,CAAS,IAAA,EAAM;AAGzB,MAAA,MAAA,CAAO,SAAS,IAAA,GAAO,IAAA;AAAA,IACzB;AAKA,IAAA,MAAM,aAAA,GAAgBC,mCAAA,CAAkB,QAAA,EAAU,MAAM,CAAA;AAExD,IAAA,IAAI,WAAA,GAAc,IAAA,CAAyB,gBAAgB,CAAA,CACxD,MAAA,CAAO;AAAA,MACN,YAAA,EAAc,IAAA,CAAK,SAAA,CAAU,MAAM,CAAA;AAAA,MACnC,IAAA;AAAA,MACA,eAAA,EAAiB,IAAA,CAAK,EAAA,CAAG,GAAA;AAAI,KAC9B,CAAA,CACA,KAAA,CAAM,WAAA,EAAa,QAAQ,CAAA;AAI9B,IAAA,IAAI,OAAA,CAAQ,QAAA,CAAS,IAAA,KAAS,UAAA,IAAc,YAAA,EAAc;AACxD,MAAA,WAAA,GAAc,WAAA,CAAY,WAAA;AAAA,QACxB,IAAA,CAAuB,cAAc,CAAA,CAClC,KAAA,CAAM,2BAA2B,SAAS,CAAA,CAC1C,KAAA,CAAM,4BAAA,EAA8B,YAAY,CAAA,CAChD,MAAA,CAAO,IAAA,CAAK,GAAA,CAAI,GAAG,CAAC;AAAA,OACzB;AAAA,IACF;AAEA,IAAA,MAAM,sBAAsB,MAAM,WAAA;AAElC,IAAA,IAAI,wBAAwB,CAAA,EAAG;AAC7B,MAAA,MAAA,CAAO,KAAA,CAAM,CAAA,OAAA,EAAU,SAAS,CAAA,qCAAA,CAAuC,CAAA;AACvE,MAAA,OAAO,WAAA;AAAA,IACT;AAEA,IAAA,MAAMC,6BAAA,CAAe,IAAA,EAAM,QAAA,EAAU,aAAa,CAAA;AAElD,IAAA,OAAO,SAAA;AAAA,EACT,SAAS,KAAA,EAAO;AACd,IAAA,iCAAA,GAAoC,KAAA;AACpC,IAAA,MAAM,KAAA;AAAA,EACR,CAAA,SAAE;AACA,IAAA,IAAI,qCAAqC,YAAA,EAAc;AACrD,MAAA,MAAMC,uDAAA,CAA4B;AAAA,QAChC,IAAA;AAAA,QACA,SAAA;AAAA,QACA;AAAA,OACD,CAAA;AAAA,IACH;AAAA,EACF;AACF;;;;"}
+{"version":3,"file":"performStitching.cjs.js","sources":["../../../../src/database/operations/stitcher/performStitching.ts"],"sourcesContent":["/*\n * Copyright 2023 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { ENTITY_STATUS_CATALOG_PROCESSING_TYPE } from '@backstage/catalog-client';\nimport {\n  ANNOTATION_EDIT_URL,\n  ANNOTATION_VIEW_URL,\n  Entity,\n  EntityRelation,\n} from '@backstage/catalog-model';\nimport { AlphaEntity, EntityStatusItem } from '@backstage/catalog-model/alpha';\nimport { SerializedError } from '@backstage/errors';\nimport { Knex } from 'knex';\nimport { createHash } from 'node:crypto';\nimport stableStringify from 'fast-json-stable-stringify';\nimport { StitchingStrategy } from '../../../stitching/types';\nimport {\n  DbFinalEntitiesRow,\n  DbRefreshStateRow,\n  DbStitchQueueRow,\n} from '../../tables';\nimport { buildEntitySearch } from './buildEntitySearch';\nimport { markDeferredStitchCompleted } from './markDeferredStitchCompleted';\nimport { syncSearchRows } from './syncSearchRows';\nimport {\n  LoggerService,\n  isDatabaseConflictError,\n} from '@backstage/backend-plugin-api';\n\nfunction generateStableHash(entity: Entity) {\n  return createHash('sha1')\n    .update(stableStringify({ ...entity }))\n    .digest('hex');\n}\n\n// See https://github.com/facebook/react/blob/f0cf832e1d0c8544c36aa8b310960885a11a847c/packages/react-dom-bindings/src/shared/sanitizeURL.js\nconst scriptProtocolPattern =\n  // eslint-disable-next-line no-control-regex\n  /^[\\u0000-\\u001F ]*j[\\r\\n\\t]*a[\\r\\n\\t]*v[\\r\\n\\t]*a[\\r\\n\\t]*s[\\r\\n\\t]*c[\\r\\n\\t]*r[\\r\\n\\t]*i[\\r\\n\\t]*p[\\r\\n\\t]*t[\\r\\n\\t]*\\:/i;\n\n/**\n * Performs the act of stitching - to take all of the various outputs from the\n * ingestion process, and stitching them together into the final entity JSON\n * shape.\n */\nexport async function performStitching(options: {\n  knex: Knex | Knex.Transaction;\n  logger: LoggerService;\n  strategy: StitchingStrategy;\n  entityRef: string;\n  stitchTicket?: string;\n}): Promise<'changed' | 'unchanged' | 'abandoned'> {\n  const { knex, logger, entityRef } = options;\n  const stitchTicket = options.stitchTicket;\n\n  // In deferred mode, the entity is removed from the stitch queue on ANY\n  // completion, except when an exception is thrown. In the latter case, the\n  // entity will be retried at a later time.\n  let removeFromStitchQueueOnCompletion = options.strategy.mode === 'deferred';\n\n  try {\n    const entityResult = await knex<DbRefreshStateRow>('refresh_state')\n      .where({ entity_ref: entityRef })\n      .limit(1)\n      .select('entity_id');\n    if (!entityResult.length) {\n      // Entity does no exist in refresh state table, no stitching required.\n      return 'abandoned';\n    }\n\n    // Ensure that a final_entities row exists for this entity.\n    try {\n      await knex<DbFinalEntitiesRow>('final_entities')\n        .insert({\n          entity_id: entityResult[0].entity_id,\n          hash: '',\n          entity_ref: entityRef,\n        })\n        .onConflict('entity_id')\n        .ignore();\n    } catch (error) {\n      // It's possible to hit a race where a refresh_state table delete + insert\n      // is done just after we read the entity_id from it. This conflict is safe\n      // to ignore because the current stitching operation will be triggered by\n      // the old entry, and the new entry will trigger it's own stitching that\n      // will update the entity.\n      if (isDatabaseConflictError(error)) {\n        logger.debug(`Skipping stitching of ${entityRef}, conflict`, error);\n        return 'abandoned';\n      }\n\n      throw error;\n    }\n\n    // Selecting from refresh_state and final_entities should yield exactly\n    // one row (except in abnormal cases where the stitch was invoked for\n    // something that didn't exist at all, in which case it's zero rows).\n    // The join with the temporary incoming_references still gives one row.\n    const [processedResult, relationsResult] = await Promise.all([\n      knex\n        .with('incoming_references', function incomingReferences(builder) {\n          return builder\n            .from('refresh_state_references')\n            .where({ target_entity_ref: entityRef })\n            .count({ count: '*' });\n        })\n        .select({\n          entityId: 'refresh_state.entity_id',\n          processedEntity: 'refresh_state.processed_entity',\n          errors: 'refresh_state.errors',\n          incomingReferenceCount: 'incoming_references.count',\n          previousHash: 'final_entities.hash',\n        })\n        .from('refresh_state')\n        .where({ 'refresh_state.entity_ref': entityRef })\n        .crossJoin(knex.raw('incoming_references'))\n        .leftOuterJoin('final_entities', {\n          'final_entities.entity_id': 'refresh_state.entity_id',\n        }),\n      knex\n        .distinct({\n          relationType: 'type',\n          relationTarget: 'target_entity_ref',\n        })\n        .from('relations')\n        .where({ source_entity_ref: entityRef })\n        .orderBy('relationType', 'asc')\n        .orderBy('relationTarget', 'asc'),\n    ]);\n\n    // If there were no rows returned, it would mean that there was no\n    // matching row even in the refresh_state. This can happen for example\n    // if we emit a relation to something that hasn't been ingested yet.\n    // It's safe to ignore this stitch attempt in that case.\n    if (!processedResult.length) {\n      logger.debug(\n        `Unable to stitch ${entityRef}, item does not exist in refresh state table`,\n      );\n      return 'abandoned';\n    }\n\n    const {\n      entityId,\n      processedEntity,\n      errors,\n      incomingReferenceCount,\n      previousHash,\n    } = processedResult[0];\n\n    // If there was no processed entity in place, the target hasn't been\n    // through the processing steps yet. It's safe to ignore this stitch\n    // attempt in that case, since another stitch will be triggered when\n    // that processing has finished.\n    if (!processedEntity) {\n      logger.debug(\n        `Unable to stitch ${entityRef}, the entity has not yet been processed`,\n      );\n      return 'abandoned';\n    }\n\n    // Grab the processed entity and stitch all of the relevant data into\n    // it\n    const entity = JSON.parse(processedEntity) as AlphaEntity;\n    const isOrphan = Number(incomingReferenceCount) === 0;\n    let statusItems: EntityStatusItem[] = [];\n\n    if (isOrphan) {\n      logger.debug(`${entityRef} is an orphan`);\n      entity.metadata.annotations = {\n        ...entity.metadata.annotations,\n        ['backstage.io/orphan']: 'true',\n      };\n    }\n    if (errors) {\n      const parsedErrors = JSON.parse(errors) as SerializedError[];\n      if (Array.isArray(parsedErrors) && parsedErrors.length) {\n        statusItems = parsedErrors.map(e => ({\n          type: ENTITY_STATUS_CATALOG_PROCESSING_TYPE,\n          level: 'error',\n          message: `${e.name}: ${e.message}`,\n          error: e,\n        }));\n      }\n    }\n    // We opt to do this check here as we otherwise can't guarantee that it will be run after all processors\n    for (const annotation of [ANNOTATION_VIEW_URL, ANNOTATION_EDIT_URL]) {\n      const value = entity.metadata.annotations?.[annotation];\n      if (typeof value === 'string' && scriptProtocolPattern.test(value)) {\n        entity.metadata.annotations![annotation] =\n          'https://backstage.io/annotation-rejected-for-security-reasons';\n      }\n    }\n\n    // TODO: entityRef is lower case and should be uppercase in the final\n    // result\n    entity.relations = relationsResult\n      .filter(row => row.relationType /* exclude null row, if relevant */)\n      .map<EntityRelation>(row => ({\n        type: row.relationType!,\n        targetRef: row.relationTarget!,\n      }));\n    if (statusItems.length) {\n      entity.status = {\n        ...entity.status,\n        items: [...(entity.status?.items ?? []), ...statusItems],\n      };\n    }\n\n    // If the output entity was actually not changed, just abort\n    const hash = generateStableHash(entity);\n    if (hash === previousHash) {\n      logger.debug(`Skipped stitching of ${entityRef}, no changes`);\n      return 'unchanged';\n    }\n\n    entity.metadata.uid = entityId;\n    if (!entity.metadata.etag) {\n      // If the original data source did not have its own etag handling,\n      // use the hash as a good-quality etag\n      entity.metadata.etag = hash;\n    }\n\n    // This may throw if the entity is invalid, so we call it before\n    // the final_entities write, even though we may end up not needing\n    // to write the search index.\n    const searchEntries = buildEntitySearch(entityId, entity);\n\n    let updateQuery = knex<DbFinalEntitiesRow>('final_entities')\n      .update({\n        final_entity: JSON.stringify(entity),\n        hash,\n        last_updated_at: knex.fn.now(),\n      })\n      .where('entity_id', entityId);\n\n    // In deferred mode, guard against concurrent stitchers by checking that\n    // the stitch_ticket in stitch_queue still matches what we were given.\n    if (options.strategy.mode === 'deferred' && stitchTicket) {\n      updateQuery = updateQuery.whereExists(\n        knex<DbStitchQueueRow>('stitch_queue')\n          .where('stitch_queue.entity_ref', entityRef)\n          .where('stitch_queue.stitch_ticket', stitchTicket)\n          .select(knex.raw('1')),\n      );\n    }\n\n    const amountOfRowsChanged = await updateQuery;\n\n    if (amountOfRowsChanged === 0) {\n      logger.debug(`Entity ${entityRef} is already stitched, skipping write.`);\n      return 'abandoned';\n    }\n\n    await syncSearchRows(knex, entityId, searchEntries);\n\n    return 'changed';\n  } catch (error) {\n    removeFromStitchQueueOnCompletion = false;\n    throw error;\n  } finally {\n    if (removeFromStitchQueueOnCompletion && stitchTicket) {\n      await markDeferredStitchCompleted({\n        knex: knex,\n        entityRef,\n        stitchTicket,\n      });\n    }\n  }\n}\n"],"names":["createHash","stableStringify","isDatabaseConflictError","ENTITY_STATUS_CATALOG_PROCESSING_TYPE","ANNOTATION_VIEW_URL","ANNOTATION_EDIT_URL","buildEntitySearch","syncSearchRows","markDeferredStitchCompleted"],"mappings":";;;;;;;;;;;;;;;AA0CA,SAAS,mBAAmB,MAAA,EAAgB;AAC1C,EAAA,OAAOA,sBAAA,CAAW,MAAM,CAAA,CACrB,MAAA,CAAOC,gCAAA,CAAgB,EAAE,GAAG,MAAA,EAAQ,CAAC,CAAA,CACrC,MAAA,CAAO,KAAK,CAAA;AACjB;AAGA,MAAM,qBAAA;AAAA;AAAA,EAEJ;AAAA,CAAA;AAOF,eAAsB,iBAAiB,OAAA,EAMY;AACjD,EAAA,MAAM,EAAE,IAAA,EAAM,MAAA,EAAQ,SAAA,EAAU,GAAI,OAAA;AACpC,EAAA,MAAM,eAAe,OAAA,CAAQ,YAAA;AAK7B,EAAA,IAAI,iCAAA,GAAoC,OAAA,CAAQ,QAAA,CAAS,IAAA,KAAS,UAAA;AAElE,EAAA,IAAI;AACF,IAAA,MAAM,YAAA,GAAe,MAAM,IAAA,CAAwB,eAAe,EAC/D,KAAA,CAAM,EAAE,UAAA,EAAY,SAAA,EAAW,CAAA,CAC/B,KAAA,CAAM,CAAC,CAAA,CACP,OAAO,WAAW,CAAA;AACrB,IAAA,IAAI,CAAC,aAAa,MAAA,EAAQ;AAExB,MAAA,OAAO,WAAA;AAAA,IACT;AAGA,IAAA,IAAI;AACF,MAAA,MAAM,IAAA,CAAyB,gBAAgB,CAAA,CAC5C,MAAA,CAAO;AAAA,QACN,SAAA,EAAW,YAAA,CAAa,CAAC,CAAA,CAAE,SAAA;AAAA,QAC3B,IAAA,EAAM,EAAA;AAAA,QACN,UAAA,EAAY;AAAA,OACb,CAAA,CACA,UAAA,CAAW,WAAW,EACtB,MAAA,EAAO;AAAA,IACZ,SAAS,KAAA,EAAO;AAMd,MAAA,IAAIC,wCAAA,CAAwB,KAAK,CAAA,EAAG;AAClC,QAAA,MAAA,CAAO,KAAA,CAAM,CAAA,sBAAA,EAAyB,SAAS,CAAA,UAAA,CAAA,EAAc,KAAK,CAAA;AAClE,QAAA,OAAO,WAAA;AAAA,MACT;AAEA,MAAA,MAAM,KAAA;AAAA,IACR;AAMA,IAAA,MAAM,CAAC,eAAA,EAAiB,eAAe,CAAA,GAAI,MAAM,QAAQ,GAAA,CAAI;AAAA,MAC3D,IAAA,CACG,IAAA,CAAK,qBAAA,EAAuB,SAAS,mBAAmB,OAAA,EAAS;AAChE,QAAA,OAAO,OAAA,CACJ,IAAA,CAAK,0BAA0B,CAAA,CAC/B,MAAM,EAAE,iBAAA,EAAmB,SAAA,EAAW,CAAA,CACtC,KAAA,CAAM,EAAE,KAAA,EAAO,KAAK,CAAA;AAAA,MACzB,CAAC,EACA,MAAA,CAAO;AAAA,QACN,QAAA,EAAU,yBAAA;AAAA,QACV,eAAA,EAAiB,gCAAA;AAAA,QACjB,MAAA,EAAQ,sBAAA;AAAA,QACR,sBAAA,EAAwB,2BAAA;AAAA,QACxB,YAAA,EAAc;AAAA,OACf,CAAA,CACA,IAAA,CAAK,eAAe,CAAA,CACpB,KAAA,CAAM,EAAE,0BAAA,EAA4B,SAAA,EAAW,CAAA,CAC/C,UAAU,IAAA,CAAK,GAAA,CAAI,qBAAqB,CAAC,CAAA,CACzC,cAAc,gBAAA,EAAkB;AAAA,QAC/B,0BAAA,EAA4B;AAAA,OAC7B,CAAA;AAAA,MACH,KACG,QAAA,CAAS;AAAA,QACR,YAAA,EAAc,MAAA;AAAA,QACd,cAAA,EAAgB;AAAA,OACjB,CAAA,CACA,IAAA,CAAK,WAAW,CAAA,CAChB,MAAM,EAAE,iBAAA,EAAmB,SAAA,EAAW,EACtC,OAAA,CAAQ,cAAA,EAAgB,KAAK,CAAA,CAC7B,OAAA,CAAQ,kBAAkB,KAAK;AAAA,KACnC,CAAA;AAMD,IAAA,IAAI,CAAC,gBAAgB,MAAA,EAAQ;AAC3B,MAAA,MAAA,CAAO,KAAA;AAAA,QACL,oBAAoB,SAAS,CAAA,4CAAA;AAAA,OAC/B;AACA,MAAA,OAAO,WAAA;AAAA,IACT;AAEA,IAAA,MAAM;AAAA,MACJ,QAAA;AAAA,MACA,eAAA;AAAA,MACA,MAAA;AAAA,MACA,sBAAA;AAAA,MACA;AAAA,KACF,GAAI,gBAAgB,CAAC,CAAA;AAMrB,IAAA,IAAI,CAAC,eAAA,EAAiB;AACpB,MAAA,MAAA,CAAO,KAAA;AAAA,QACL,oBAAoB,SAAS,CAAA,uCAAA;AAAA,OAC/B;AACA,MAAA,OAAO,WAAA;AAAA,IACT;AAIA,IAAA,MAAM,MAAA,GAAS,IAAA,CAAK,KAAA,CAAM,eAAe,CAAA;AACzC,IAAA,MAAM,QAAA,GAAW,MAAA,CAAO,sBAAsB,CAAA,KAAM,CAAA;AACpD,IAAA,IAAI,cAAkC,EAAC;AAEvC,IAAA,IAAI,QAAA,EAAU;AACZ,MAAA,MAAA,CAAO,KAAA,CAAM,CAAA,EAAG,SAAS,CAAA,aAAA,CAAe,CAAA;AACxC,MAAA,MAAA,CAAO,SAAS,WAAA,GAAc;AAAA,QAC5B,GAAG,OAAO,QAAA,CAAS,WAAA;AAAA,QACnB,CAAC,qBAAqB,GAAG;AAAA,OAC3B;AAAA,IACF;AACA,IAAA,IAAI,MAAA,EAAQ;AACV,MAAA,MAAM,YAAA,GAAe,IAAA,CAAK,KAAA,CAAM,MAAM,CAAA;AACtC,MAAA,IAAI,KAAA,CAAM,OAAA,CAAQ,YAAY,CAAA,IAAK,aAAa,MAAA,EAAQ;AACtD,QAAA,WAAA,GAAc,YAAA,CAAa,IAAI,CAAA,CAAA,MAAM;AAAA,UACnC,IAAA,EAAMC,mDAAA;AAAA,UACN,KAAA,EAAO,OAAA;AAAA,UACP,SAAS,CAAA,EAAG,CAAA,CAAE,IAAI,CAAA,EAAA,EAAK,EAAE,OAAO,CAAA,CAAA;AAAA,UAChC,KAAA,EAAO;AAAA,SACT,CAAE,CAAA;AAAA,MACJ;AAAA,IACF;AAEA,IAAA,KAAA,MAAW,UAAA,IAAc,CAACC,gCAAA,EAAqBC,gCAAmB,CAAA,EAAG;AACnE,MAAA,MAAM,KAAA,GAAQ,MAAA,CAAO,QAAA,CAAS,WAAA,GAAc,UAAU,CAAA;AACtD,MAAA,IAAI,OAAO,KAAA,KAAU,QAAA,IAAY,qBAAA,CAAsB,IAAA,CAAK,KAAK,CAAA,EAAG;AAClE,QAAA,MAAA,CAAO,QAAA,CAAS,WAAA,CAAa,UAAU,CAAA,GACrC,+DAAA;AAAA,MACJ;AAAA,IACF;AAIA,IAAA,MAAA,CAAO,YAAY,eAAA,CAChB,MAAA;AAAA,MAAO,SAAO,GAAA,CAAI;AAAA;AAAA,KAAgD,CAClE,IAAoB,CAAA,GAAA,MAAQ;AAAA,MAC3B,MAAM,GAAA,CAAI,YAAA;AAAA,MACV,WAAW,GAAA,CAAI;AAAA,KACjB,CAAE,CAAA;AACJ,IAAA,IAAI,YAAY,MAAA,EAAQ;AACtB,MAAA,MAAA,CAAO,MAAA,GAAS;AAAA,QACd,GAAG,MAAA,CAAO,MAAA;AAAA,QACV,KAAA,EAAO,CAAC,GAAI,MAAA,CAAO,QAAQ,KAAA,IAAS,EAAC,EAAI,GAAG,WAAW;AAAA,OACzD;AAAA,IACF;AAGA,IAAA,MAAM,IAAA,GAAO,mBAAmB,MAAM,CAAA;AACtC,IAAA,IAAI,SAAS,YAAA,EAAc;AACzB,MAAA,MAAA,CAAO,KAAA,CAAM,CAAA,qBAAA,EAAwB,SAAS,CAAA,YAAA,CAAc,CAAA;AAC5D,MAAA,OAAO,WAAA;AAAA,IACT;AAEA,IAAA,MAAA,CAAO,SAAS,GAAA,GAAM,QAAA;AACtB,IAAA,IAAI,CAAC,MAAA,CAAO,QAAA,CAAS,IAAA,EAAM;AAGzB,MAAA,MAAA,CAAO,SAAS,IAAA,GAAO,IAAA;AAAA,IACzB;AAKA,IAAA,MAAM,aAAA,GAAgBC,mCAAA,CAAkB,QAAA,EAAU,MAAM,CAAA;AAExD,IAAA,IAAI,WAAA,GAAc,IAAA,CAAyB,gBAAgB,CAAA,CACxD,MAAA,CAAO;AAAA,MACN,YAAA,EAAc,IAAA,CAAK,SAAA,CAAU,MAAM,CAAA;AAAA,MACnC,IAAA;AAAA,MACA,eAAA,EAAiB,IAAA,CAAK,EAAA,CAAG,GAAA;AAAI,KAC9B,CAAA,CACA,KAAA,CAAM,WAAA,EAAa,QAAQ,CAAA;AAI9B,IAAA,IAAI,OAAA,CAAQ,QAAA,CAAS,IAAA,KAAS,UAAA,IAAc,YAAA,EAAc;AACxD,MAAA,WAAA,GAAc,WAAA,CAAY,WAAA;AAAA,QACxB,IAAA,CAAuB,cAAc,CAAA,CAClC,KAAA,CAAM,2BAA2B,SAAS,CAAA,CAC1C,KAAA,CAAM,4BAAA,EAA8B,YAAY,CAAA,CAChD,MAAA,CAAO,IAAA,CAAK,GAAA,CAAI,GAAG,CAAC;AAAA,OACzB;AAAA,IACF;AAEA,IAAA,MAAM,sBAAsB,MAAM,WAAA;AAElC,IAAA,IAAI,wBAAwB,CAAA,EAAG;AAC7B,MAAA,MAAA,CAAO,KAAA,CAAM,CAAA,OAAA,EAAU,SAAS,CAAA,qCAAA,CAAuC,CAAA;AACvE,MAAA,OAAO,WAAA;AAAA,IACT;AAEA,IAAA,MAAMC,6BAAA,CAAe,IAAA,EAAM,QAAA,EAAU,aAAa,CAAA;AAElD,IAAA,OAAO,SAAA;AAAA,EACT,SAAS,KAAA,EAAO;AACd,IAAA,iCAAA,GAAoC,KAAA;AACpC,IAAA,MAAM,KAAA;AAAA,EACR,CAAA,SAAE;AACA,IAAA,IAAI,qCAAqC,YAAA,EAAc;AACrD,MAAA,MAAMC,uDAAA,CAA4B;AAAA,QAChC,IAAA;AAAA,QACA,SAAA;AAAA,QACA;AAAA,OACD,CAAA;AAAA,IACH;AAAA,EACF;AACF;;;;"}

package/dist/database/operations/stitcher/util.cjs.js

@@ -1,19 +1,8 @@
 'use strict';
 
-var node_crypto = require('node:crypto');
-var stableStringify = require('fast-json-stable-stringify');
-
-function _interopDefaultCompat (e) { return e && typeof e === 'object' && 'default' in e ? e : { default: e }; }
-
-var stableStringify__default = /*#__PURE__*/_interopDefaultCompat(stableStringify);
-
 const BATCH_SIZE = 50;
 const NULL_SENTINEL = "
";
-function generateStableHash(entity) {
-  return node_crypto.createHash("sha1").update(stableStringify__default.default({ ...entity })).digest("hex");
-}
 
 exports.BATCH_SIZE = BATCH_SIZE;
 exports.NULL_SENTINEL = NULL_SENTINEL;
-exports.generateStableHash = generateStableHash;
 //# sourceMappingURL=util.cjs.js.map

package/dist/database/operations/stitcher/util.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"util.cjs.js","sources":["../../../../src/database/operations/stitcher/util.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Entity } from '@backstage/catalog-model';\nimport { createHash } from 'node:crypto';\nimport stableStringify from 'fast-json-stable-stringify';\n\n// The number of items that are sent per batch to the database layer, when\n// doing .batchInsert calls to knex. This needs to be low enough to not cause\n// errors in the underlying engine due to exceeding query limits, but large\n// enough to get the speed benefits.\nexport const BATCH_SIZE = 50;\n\n// The SOH (Start of Heading) control character, used as a stand-in for NULL\n// in contexts where NULL cannot participate in equality comparisons (SQL\n// COALESCE, JS dedup keys). It cannot appear in real entity metadata values\n// since they are human-readable strings.\nexport const NULL_SENTINEL = '\\x01';\n\nexport function generateStableHash(entity: Entity) {\n  return createHash('sha1')\n    .update(stableStringify({ ...entity }))\n    .digest('hex');\n}\n"],"names":["createHash","stableStringify"],"mappings":";;;;;;;;;AAwBO,MAAM,UAAA,GAAa;AAMnB,MAAM,aAAA,GAAgB;AAEtB,SAAS,mBAAmB,MAAA,EAAgB;AACjD,EAAA,OAAOA,sBAAA,CAAW,MAAM,CAAA,CACrB,MAAA,CAAOC,gCAAA,CAAgB,EAAE,GAAG,MAAA,EAAQ,CAAC,CAAA,CACrC,MAAA,CAAO,KAAK,CAAA;AACjB;;;;;;"}
+{"version":3,"file":"util.cjs.js","sources":["../../../../src/database/operations/stitcher/util.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\n// The number of items that are sent per batch to the database layer, when\n// doing .batchInsert calls to knex. This needs to be low enough to not cause\n// errors in the underlying engine due to exceeding query limits, but large\n// enough to get the speed benefits.\nexport const BATCH_SIZE = 50;\n\n// The SOH (Start of Heading) control character, used as a stand-in for NULL\n// in contexts where NULL cannot participate in equality comparisons (SQL\n// COALESCE, JS dedup keys). It cannot appear in real entity metadata values\n// since they are human-readable strings.\nexport const NULL_SENTINEL = '\\x01';\n"],"names":[],"mappings":";;AAoBO,MAAM,UAAA,GAAa;AAMnB,MAAM,aAAA,GAAgB;;;;;"}

package/dist/processing/DefaultCatalogProcessingEngine.cjs.js

@@ -249,7 +249,9 @@ class DefaultCatalogProcessingEngine {
       return () => {
       };
     }
-    const stitchingStrategy = types.stitchingStrategyFromConfig(this.config);
+    const stitchingStrategy = types.stitchingStrategyFromConfig(this.config, {
+      logger: this.logger
+    });
     const runOnce = async () => {
       try {
         const n = await deleteOrphanedEntities.deleteOrphanedEntities({

package/dist/processing/DefaultCatalogProcessingEngine.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"DefaultCatalogProcessingEngine.cjs.js","sources":["../../src/processing/DefaultCatalogProcessingEngine.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport {\n  ANNOTATION_LOCATION,\n  Entity,\n  stringifyEntityRef,\n} from '@backstage/catalog-model';\nimport { serializeError, stringifyError, toError } from '@backstage/errors';\nimport { Hash } from 'node:crypto';\nimport stableStringify from 'fast-json-stable-stringify';\nimport { Knex } from 'knex';\nimport { trace } from '@opentelemetry/api';\nimport { ProcessingDatabase, RefreshStateItem } from '../database/types';\nimport { createCounterMetric, createSummaryMetric } from '../util/metrics';\nimport { CatalogProcessingOrchestrator, EntityProcessingResult } from './types';\nimport { Stitcher, stitchingStrategyFromConfig } from '../stitching/types';\nimport { startTaskPipeline } from './TaskPipeline';\nimport { Config } from '@backstage/config';\nimport {\n  addEntityAttributes,\n  TRACER_ID,\n  withActiveSpan,\n} from '../util/opentelemetry';\nimport { deleteOrphanedEntities } from '../database/operations/util/deleteOrphanedEntities';\nimport { EventsService } from '@backstage/plugin-events-node';\nimport { CATALOG_ERRORS_TOPIC } from '../constants';\nimport { LoggerService, SchedulerService } from '@backstage/backend-plugin-api';\nimport { MetricsService } from '@backstage/backend-plugin-api/alpha';\n\nconst CACHE_TTL = 5;\n\nconst tracer = trace.getTracer(TRACER_ID);\n\nexport type ProgressTracker = ReturnType<typeof progressTracker>;\n\nconst stableStringifyArray = (arr: any[]) => {\n  const sorted = arr.map(stableStringify).sort();\n  return `[${sorted.join(',')}]`;\n};\n\n// NOTE(freben): Perhaps surprisingly, this class does not implement the\n// CatalogProcessingEngine type. That type is externally visible and its name is\n// the way it is for historic reasons. This class has no particular reason to\n// implement that precise interface; nowadays there are several different\n// engines \"hiding\" behind the CatalogProcessingEngine interface, of which this\n// is just one.\nexport class DefaultCatalogProcessingEngine {\n  private readonly config: Config;\n  private readonly scheduler: SchedulerService;\n  private readonly logger: LoggerService;\n  private readonly knex: Knex;\n  private readonly processingDatabase: ProcessingDatabase;\n  private readonly orchestrator: CatalogProcessingOrchestrator;\n  private readonly stitcher: Stitcher;\n  private readonly createHash: () => Hash;\n  private readonly pollingIntervalMs: number;\n  private readonly orphanCleanupIntervalMs: number;\n  private readonly onProcessingError?: (event: {\n    unprocessedEntity: Entity;\n    errors: Error[];\n  }) => Promise<void> | void;\n  private readonly tracker: ProgressTracker;\n  private readonly events: EventsService;\n\n  private stopFunc?: () => void;\n\n  constructor(options: {\n    config: Config;\n    scheduler: SchedulerService;\n    logger: LoggerService;\n    knex: Knex;\n    processingDatabase: ProcessingDatabase;\n    orchestrator: CatalogProcessingOrchestrator;\n    stitcher: Stitcher;\n    createHash: () => Hash;\n    pollingIntervalMs?: number;\n    orphanCleanupIntervalMs?: number;\n    onProcessingError?: (event: {\n      unprocessedEntity: Entity;\n      errors: Error[];\n    }) => Promise<void> | void;\n    tracker?: ProgressTracker;\n    events: EventsService;\n    metrics: MetricsService;\n  }) {\n    this.config = options.config;\n    this.scheduler = options.scheduler;\n    this.logger = options.logger;\n    this.knex = options.knex;\n    this.processingDatabase = options.processingDatabase;\n    this.orchestrator = options.orchestrator;\n    this.stitcher = options.stitcher;\n    this.createHash = options.createHash;\n    this.pollingIntervalMs = options.pollingIntervalMs ?? 1_000;\n    this.orphanCleanupIntervalMs = options.orphanCleanupIntervalMs ?? 30_000;\n    this.onProcessingError = options.onProcessingError;\n    this.tracker = options.tracker ?? progressTracker(options.metrics);\n    this.events = options.events;\n\n    this.stopFunc = undefined;\n  }\n\n  async start() {\n    if (this.stopFunc) {\n      throw new Error('Processing engine is already started');\n    }\n\n    const stopPipeline = this.startPipeline();\n    const stopCleanup = this.startOrphanCleanup();\n\n    this.stopFunc = () => {\n      stopPipeline();\n      stopCleanup();\n    };\n  }\n\n  async stop() {\n    if (this.stopFunc) {\n      this.stopFunc();\n      this.stopFunc = undefined;\n    }\n  }\n\n  private startPipeline(): () => void {\n    return startTaskPipeline<RefreshStateItem>({\n      lowWatermark: 5,\n      highWatermark: 10,\n      pollingIntervalMs: this.pollingIntervalMs,\n      loadTasks: async count => {\n        try {\n          const { items } = await this.processingDatabase.transaction(\n            async tx => {\n              return this.processingDatabase.getProcessableEntities(tx, {\n                processBatchSize: count,\n              });\n            },\n          );\n          return items;\n        } catch (error) {\n          this.logger.warn('Failed to load processing items', error);\n          return [];\n        }\n      },\n      processTask: async item => {\n        await withActiveSpan(tracer, 'ProcessingRun', async span => {\n          const track = this.tracker.processStart(item, this.logger);\n          addEntityAttributes(span, item.unprocessedEntity);\n\n          try {\n            const {\n              id,\n              state,\n              unprocessedEntity,\n              entityRef,\n              locationKey,\n              resultHash: previousResultHash,\n            } = item;\n            const result = await this.orchestrator.process({\n              entity: unprocessedEntity,\n              state,\n            });\n\n            track.markProcessorsCompleted(result);\n\n            if (result.ok) {\n              const { ttl: _, ...stateWithoutTtl } = state ?? {};\n              if (\n                stableStringify(stateWithoutTtl) !==\n                stableStringify(result.state)\n              ) {\n                await this.processingDatabase.transaction(async tx => {\n                  await this.processingDatabase.updateEntityCache(tx, {\n                    id,\n                    state: {\n                      ttl: CACHE_TTL,\n                      ...result.state,\n                    },\n                  });\n                });\n              }\n            } else {\n              const maybeTtl = state?.ttl;\n              const ttl = Number.isInteger(maybeTtl) ? (maybeTtl as number) : 0;\n              await this.processingDatabase.transaction(async tx => {\n                await this.processingDatabase.updateEntityCache(tx, {\n                  id,\n                  state: ttl > 0 ? { ...state, ttl: ttl - 1 } : {},\n                });\n              });\n            }\n\n            const location =\n              unprocessedEntity?.metadata?.annotations?.[ANNOTATION_LOCATION];\n            if (result.errors.length) {\n              this.events.publish({\n                topic: CATALOG_ERRORS_TOPIC,\n                eventPayload: {\n                  entity: entityRef,\n                  location,\n                  errors: result.errors,\n                },\n              });\n            }\n            const errorsString = JSON.stringify(\n              result.errors.map(e => serializeError(e)),\n            );\n\n            let hashBuilder = this.createHash().update(errorsString);\n\n            if (result.ok) {\n              const { entityRefs: parents } =\n                await this.processingDatabase.transaction(tx =>\n                  this.processingDatabase.listParents(tx, {\n                    entityRefs: [\n                      entityRef,\n                      ...result.deferredEntities.map(e =>\n                        stringifyEntityRef(e.entity),\n                      ),\n                    ],\n                  }),\n                );\n\n              hashBuilder = hashBuilder\n                .update(stableStringify({ ...result.completedEntity }))\n                .update(stableStringifyArray([...result.deferredEntities]))\n                .update(stableStringifyArray([...result.relations]))\n                .update(stableStringifyArray([...result.refreshKeys]))\n                .update(stableStringifyArray([...parents]));\n            }\n\n            const resultHash = hashBuilder.digest('hex');\n            if (resultHash === previousResultHash) {\n              // If nothing changed in our produced outputs, we cannot have any\n              // significant effect on our surroundings; therefore, we just abort\n              // without any updates / stitching.\n              track.markSuccessfulWithNoChanges();\n              return;\n            }\n\n            // If the result was marked as not OK, it signals that some part of the\n            // processing pipeline threw an exception. This can happen both as part of\n            // non-catastrophic things such as due to validation errors, as well as if\n            // something fatal happens inside the processing for other reasons. In any\n            // case, this means we can't trust that anything in the output is okay. So\n            // just store the errors and trigger a stitch so that they become visible to\n            // the outside.\n            if (!result.ok) {\n              // notify the error listener if the entity can not be processed.\n              Promise.resolve(undefined)\n                .then(() =>\n                  this.onProcessingError?.({\n                    unprocessedEntity,\n                    errors: result.errors,\n                  }),\n                )\n                .catch(error => {\n                  this.logger.debug(\n                    `Processing error listener threw an exception, ${stringifyError(\n                      error,\n                    )}`,\n                  );\n                });\n\n              await this.processingDatabase.transaction(async tx => {\n                await this.processingDatabase.updateProcessedEntityErrors(tx, {\n                  id,\n                  errors: errorsString,\n                  resultHash,\n                });\n              });\n\n              await this.stitcher.stitch({\n                entityRefs: [stringifyEntityRef(unprocessedEntity)],\n              });\n\n              track.markSuccessfulWithErrors();\n              return;\n            }\n\n            result.completedEntity.metadata.uid = id;\n            let oldRelationSources: Map<string, string>;\n            await this.processingDatabase.transaction(async tx => {\n              const { previous } =\n                await this.processingDatabase.updateProcessedEntity(tx, {\n                  id,\n                  processedEntity: result.completedEntity,\n                  resultHash,\n                  errors: errorsString,\n                  relations: result.relations,\n                  deferredEntities: result.deferredEntities,\n                  locationKey,\n                  refreshKeys: result.refreshKeys,\n                });\n              oldRelationSources = new Map(\n                previous.relations.map(r => [\n                  `${r.source_entity_ref}:${r.type}->${r.target_entity_ref}`,\n                  r.source_entity_ref,\n                ]),\n              );\n            });\n\n            const newRelationSources = new Map<string, string>(\n              result.relations.map(relation => {\n                const sourceEntityRef = stringifyEntityRef(relation.source);\n                const targetEntityRef = stringifyEntityRef(relation.target);\n                return [\n                  `${sourceEntityRef}:${relation.type}->${targetEntityRef}`,\n                  sourceEntityRef,\n                ];\n              }),\n            );\n\n            const setOfThingsToStitch = new Set<string>([\n              stringifyEntityRef(result.completedEntity),\n            ]);\n            newRelationSources.forEach((sourceEntityRef, uniqueKey) => {\n              if (!oldRelationSources.has(uniqueKey)) {\n                setOfThingsToStitch.add(sourceEntityRef);\n              }\n            });\n            oldRelationSources!.forEach((sourceEntityRef, uniqueKey) => {\n              if (!newRelationSources.has(uniqueKey)) {\n                setOfThingsToStitch.add(sourceEntityRef);\n              }\n            });\n\n            await this.stitcher.stitch({\n              entityRefs: setOfThingsToStitch,\n            });\n\n            track.markSuccessfulWithChanges();\n          } catch (error) {\n            track.markFailed(toError(error));\n          }\n        });\n      },\n    });\n  }\n\n  private startOrphanCleanup(): () => void {\n    const orphanStrategy =\n      this.config.getOptionalString('catalog.orphanStrategy') ?? 'delete';\n    if (orphanStrategy !== 'delete') {\n      return () => {};\n    }\n\n    const stitchingStrategy = stitchingStrategyFromConfig(this.config);\n\n    const runOnce = async () => {\n      try {\n        const n = await deleteOrphanedEntities({\n          knex: this.knex,\n          strategy: stitchingStrategy,\n        });\n        if (n > 0) {\n          this.logger.info(`Deleted ${n} orphaned entities`);\n        }\n      } catch (error) {\n        this.logger.warn(`Failed to delete orphaned entities`, error);\n      }\n    };\n\n    const abortController = new AbortController();\n    this.scheduler.scheduleTask({\n      id: 'catalog_orphan_cleanup',\n      frequency: { milliseconds: this.orphanCleanupIntervalMs },\n      timeout: { milliseconds: this.orphanCleanupIntervalMs * 0.8 },\n      fn: runOnce,\n      signal: abortController.signal,\n    });\n\n    return () => {\n      abortController.abort();\n    };\n  }\n}\n\n// Helps wrap the timing and logging behaviors\nfunction progressTracker(metrics: MetricsService) {\n  // prom-client metrics are deprecated in favour of OpenTelemetry metrics.\n  const promProcessedEntities = createCounterMetric({\n    name: 'catalog_processed_entities_count',\n    help: 'Amount of entities processed, DEPRECATED, use OpenTelemetry metrics instead',\n    labelNames: ['result'],\n  });\n  const promProcessingDuration = createSummaryMetric({\n    name: 'catalog_processing_duration_seconds',\n    help: 'Time spent executing the full processing flow, DEPRECATED, use OpenTelemetry metrics instead',\n    labelNames: ['result'],\n  });\n  const promProcessorsDuration = createSummaryMetric({\n    name: 'catalog_processors_duration_seconds',\n    help: 'Time spent executing catalog processors, DEPRECATED, use OpenTelemetry metrics instead',\n    labelNames: ['result'],\n  });\n  const promProcessingQueueDelay = createSummaryMetric({\n    name: 'catalog_processing_queue_delay_seconds',\n    help: 'The amount of delay between being scheduled for processing, and the start of actually being processed, DEPRECATED, use OpenTelemetry metrics instead',\n  });\n\n  const processedEntities = metrics.createCounter(\n    'catalog.processed.entities.count',\n    { description: 'Amount of entities processed' },\n  );\n\n  const processingDuration = metrics.createHistogram(\n    'catalog.processing.duration',\n    {\n      description: 'Time spent executing the full processing flow',\n      unit: 'seconds',\n    },\n  );\n\n  const processorsDuration = metrics.createHistogram(\n    'catalog.processors.duration',\n    {\n      description: 'Time spent executing catalog processors',\n      unit: 'seconds',\n    },\n  );\n\n  const processingQueueDelay = metrics.createHistogram(\n    'catalog.processing.queue.delay',\n    {\n      description:\n        'The amount of delay between being scheduled for processing, and the start of actually being processed',\n      unit: 'seconds',\n    },\n  );\n\n  function processStart(item: RefreshStateItem, logger: LoggerService) {\n    const startTime = process.hrtime();\n    const endOverallTimer = promProcessingDuration.startTimer();\n    const endProcessorsTimer = promProcessorsDuration.startTimer();\n\n    logger.debug(`Processing ${item.entityRef}`);\n\n    if (item.nextUpdateAt) {\n      const seconds = -item.nextUpdateAt.diffNow().as('seconds');\n      promProcessingQueueDelay.observe(seconds);\n      processingQueueDelay.record(seconds);\n    }\n\n    function endTime() {\n      const delta = process.hrtime(startTime);\n      return delta[0] + delta[1] / 1e9;\n    }\n\n    function markProcessorsCompleted(result: EntityProcessingResult) {\n      endProcessorsTimer({ result: result.ok ? 'ok' : 'failed' });\n      processorsDuration.record(endTime(), {\n        result: result.ok ? 'ok' : 'failed',\n      });\n    }\n\n    function markSuccessfulWithNoChanges() {\n      endOverallTimer({ result: 'unchanged' });\n      promProcessedEntities.inc({ result: 'unchanged' }, 1);\n\n      processingDuration.record(endTime(), { result: 'unchanged' });\n      processedEntities.add(1, { result: 'unchanged' });\n    }\n\n    function markSuccessfulWithErrors() {\n      endOverallTimer({ result: 'errors' });\n      promProcessedEntities.inc({ result: 'errors' }, 1);\n\n      processingDuration.record(endTime(), { result: 'errors' });\n      processedEntities.add(1, { result: 'errors' });\n    }\n\n    function markSuccessfulWithChanges() {\n      endOverallTimer({ result: 'changed' });\n      promProcessedEntities.inc({ result: 'changed' }, 1);\n\n      processingDuration.record(endTime(), { result: 'changed' });\n      processedEntities.add(1, { result: 'changed' });\n    }\n\n    function markFailed(error: Error) {\n      promProcessedEntities.inc({ result: 'failed' }, 1);\n      processedEntities.add(1, { result: 'failed' });\n      logger.warn(`Processing of ${item.entityRef} failed`, error);\n    }\n\n    return {\n      markProcessorsCompleted,\n      markSuccessfulWithNoChanges,\n      markSuccessfulWithErrors,\n      markSuccessfulWithChanges,\n      markFailed,\n    };\n  }\n\n  return { processStart };\n}\n"],"names":["trace","TRACER_ID","stableStringify","startTaskPipeline","withActiveSpan","addEntityAttributes","ANNOTATION_LOCATION","CATALOG_ERRORS_TOPIC","serializeError","stringifyEntityRef","stringifyError","toError","stitchingStrategyFromConfig","deleteOrphanedEntities","metrics","createCounterMetric","createSummaryMetric"],"mappings":";;;;;;;;;;;;;;;;;AA2CA,MAAM,SAAA,GAAY,CAAA;AAElB,MAAM,MAAA,GAASA,SAAA,CAAM,SAAA,CAAUC,uBAAS,CAAA;AAIxC,MAAM,oBAAA,GAAuB,CAAC,GAAA,KAAe;AAC3C,EAAA,MAAM,MAAA,GAAS,GAAA,CAAI,GAAA,CAAIC,gCAAe,EAAE,IAAA,EAAK;AAC7C,EAAA,OAAO,CAAA,CAAA,EAAI,MAAA,CAAO,IAAA,CAAK,GAAG,CAAC,CAAA,CAAA,CAAA;AAC7B,CAAA;AAQO,MAAM,8BAAA,CAA+B;AAAA,EACzB,MAAA;AAAA,EACA,SAAA;AAAA,EACA,MAAA;AAAA,EACA,IAAA;AAAA,EACA,kBAAA;AAAA,EACA,YAAA;AAAA,EACA,QAAA;AAAA,EACA,UAAA;AAAA,EACA,iBAAA;AAAA,EACA,uBAAA;AAAA,EACA,iBAAA;AAAA,EAIA,OAAA;AAAA,EACA,MAAA;AAAA,EAET,QAAA;AAAA,EAER,YAAY,OAAA,EAkBT;AACD,IAAA,IAAA,CAAK,SAAS,OAAA,CAAQ,MAAA;AACtB,IAAA,IAAA,CAAK,YAAY,OAAA,CAAQ,SAAA;AACzB,IAAA,IAAA,CAAK,SAAS,OAAA,CAAQ,MAAA;AACtB,IAAA,IAAA,CAAK,OAAO,OAAA,CAAQ,IAAA;AACpB,IAAA,IAAA,CAAK,qBAAqB,OAAA,CAAQ,kBAAA;AAClC,IAAA,IAAA,CAAK,eAAe,OAAA,CAAQ,YAAA;AAC5B,IAAA,IAAA,CAAK,WAAW,OAAA,CAAQ,QAAA;AACxB,IAAA,IAAA,CAAK,aAAa,OAAA,CAAQ,UAAA;AAC1B,IAAA,IAAA,CAAK,iBAAA,GAAoB,QAAQ,iBAAA,IAAqB,GAAA;AACtD,IAAA,IAAA,CAAK,uBAAA,GAA0B,QAAQ,uBAAA,IAA2B,GAAA;AAClE,IAAA,IAAA,CAAK,oBAAoB,OAAA,CAAQ,iBAAA;AACjC,IAAA,IAAA,CAAK,OAAA,GAAU,OAAA,CAAQ,OAAA,IAAW,eAAA,CAAgB,QAAQ,OAAO,CAAA;AACjE,IAAA,IAAA,CAAK,SAAS,OAAA,CAAQ,MAAA;AAEtB,IAAA,IAAA,CAAK,QAAA,GAAW,MAAA;AAAA,EAClB;AAAA,EAEA,MAAM,KAAA,GAAQ;AACZ,IAAA,IAAI,KAAK,QAAA,EAAU;AACjB,MAAA,MAAM,IAAI,MAAM,sCAAsC,CAAA;AAAA,IACxD;AAEA,IAAA,MAAM,YAAA,GAAe,KAAK,aAAA,EAAc;AACxC,IAAA,MAAM,WAAA,GAAc,KAAK,kBAAA,EAAmB;AAE5C,IAAA,IAAA,CAAK,WAAW,MAAM;AACpB,MAAA,YAAA,EAAa;AACb,MAAA,WAAA,EAAY;AAAA,IACd,CAAA;AAAA,EACF;AAAA,EAEA,MAAM,IAAA,GAAO;AACX,IAAA,IAAI,KAAK,QAAA,EAAU;AACjB,MAAA,IAAA,CAAK,QAAA,EAAS;AACd,MAAA,IAAA,CAAK,QAAA,GAAW,MAAA;AAAA,IAClB;AAAA,EACF;AAAA,EAEQ,aAAA,GAA4B;AAClC,IAAA,OAAOC,8BAAA,CAAoC;AAAA,MACzC,YAAA,EAAc,CAAA;AAAA,MACd,aAAA,EAAe,EAAA;AAAA,MACf,mBAAmB,IAAA,CAAK,iBAAA;AAAA,MACxB,SAAA,EAAW,OAAM,KAAA,KAAS;AACxB,QAAA,IAAI;AACF,UAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,KAAK,kBAAA,CAAmB,WAAA;AAAA,YAC9C,OAAM,EAAA,KAAM;AACV,cAAA,OAAO,IAAA,CAAK,kBAAA,CAAmB,sBAAA,CAAuB,EAAA,EAAI;AAAA,gBACxD,gBAAA,EAAkB;AAAA,eACnB,CAAA;AAAA,YACH;AAAA,WACF;AACA,UAAA,OAAO,KAAA;AAAA,QACT,SAAS,KAAA,EAAO;AACd,UAAA,IAAA,CAAK,MAAA,CAAO,IAAA,CAAK,iCAAA,EAAmC,KAAK,CAAA;AACzD,UAAA,OAAO,EAAC;AAAA,QACV;AAAA,MACF,CAAA;AAAA,MACA,WAAA,EAAa,OAAM,IAAA,KAAQ;AACzB,QAAA,MAAMC,4BAAA,CAAe,MAAA,EAAQ,eAAA,EAAiB,OAAM,IAAA,KAAQ;AAC1D,UAAA,MAAM,QAAQ,IAAA,CAAK,OAAA,CAAQ,YAAA,CAAa,IAAA,EAAM,KAAK,MAAM,CAAA;AACzD,UAAAC,iCAAA,CAAoB,IAAA,EAAM,KAAK,iBAAiB,CAAA;AAEhD,UAAA,IAAI;AACF,YAAA,MAAM;AAAA,cACJ,EAAA;AAAA,cACA,KAAA;AAAA,cACA,iBAAA;AAAA,cACA,SAAA;AAAA,cACA,WAAA;AAAA,cACA,UAAA,EAAY;AAAA,aACd,GAAI,IAAA;AACJ,YAAA,MAAM,MAAA,GAAS,MAAM,IAAA,CAAK,YAAA,CAAa,OAAA,CAAQ;AAAA,cAC7C,MAAA,EAAQ,iBAAA;AAAA,cACR;AAAA,aACD,CAAA;AAED,YAAA,KAAA,CAAM,wBAAwB,MAAM,CAAA;AAEpC,YAAA,IAAI,OAAO,EAAA,EAAI;AACb,cAAA,MAAM,EAAE,GAAA,EAAK,CAAA,EAAG,GAAG,eAAA,EAAgB,GAAI,SAAS,EAAC;AACjD,cAAA,IACEH,iCAAgB,eAAe,CAAA,KAC/BA,gCAAA,CAAgB,MAAA,CAAO,KAAK,CAAA,EAC5B;AACA,gBAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,OAAM,EAAA,KAAM;AACpD,kBAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,iBAAA,CAAkB,EAAA,EAAI;AAAA,oBAClD,EAAA;AAAA,oBACA,KAAA,EAAO;AAAA,sBACL,GAAA,EAAK,SAAA;AAAA,sBACL,GAAG,MAAA,CAAO;AAAA;AACZ,mBACD,CAAA;AAAA,gBACH,CAAC,CAAA;AAAA,cACH;AAAA,YACF,CAAA,MAAO;AACL,cAAA,MAAM,WAAW,KAAA,EAAO,GAAA;AACxB,cAAA,MAAM,GAAA,GAAM,MAAA,CAAO,SAAA,CAAU,QAAQ,IAAK,QAAA,GAAsB,CAAA;AAChE,cAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,OAAM,EAAA,KAAM;AACpD,gBAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,iBAAA,CAAkB,EAAA,EAAI;AAAA,kBAClD,EAAA;AAAA,kBACA,KAAA,EAAO,GAAA,GAAM,CAAA,GAAI,EAAE,GAAG,OAAO,GAAA,EAAK,GAAA,GAAM,CAAA,EAAE,GAAI;AAAC,iBAChD,CAAA;AAAA,cACH,CAAC,CAAA;AAAA,YACH;AAEA,YAAA,MAAM,QAAA,GACJ,iBAAA,EAAmB,QAAA,EAAU,WAAA,GAAcI,gCAAmB,CAAA;AAChE,YAAA,IAAI,MAAA,CAAO,OAAO,MAAA,EAAQ;AACxB,cAAA,IAAA,CAAK,OAAO,OAAA,CAAQ;AAAA,gBAClB,KAAA,EAAOC,8BAAA;AAAA,gBACP,YAAA,EAAc;AAAA,kBACZ,MAAA,EAAQ,SAAA;AAAA,kBACR,QAAA;AAAA,kBACA,QAAQ,MAAA,CAAO;AAAA;AACjB,eACD,CAAA;AAAA,YACH;AACA,YAAA,MAAM,eAAe,IAAA,CAAK,SAAA;AAAA,cACxB,OAAO,MAAA,CAAO,GAAA,CAAI,CAAA,CAAA,KAAKC,qBAAA,CAAe,CAAC,CAAC;AAAA,aAC1C;AAEA,YAAA,IAAI,WAAA,GAAc,IAAA,CAAK,UAAA,EAAW,CAAE,OAAO,YAAY,CAAA;AAEvD,YAAA,IAAI,OAAO,EAAA,EAAI;AACb,cAAA,MAAM,EAAE,UAAA,EAAY,OAAA,EAAQ,GAC1B,MAAM,KAAK,kBAAA,CAAmB,WAAA;AAAA,gBAAY,CAAA,EAAA,KACxC,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,EAAA,EAAI;AAAA,kBACtC,UAAA,EAAY;AAAA,oBACV,SAAA;AAAA,oBACA,GAAG,OAAO,gBAAA,CAAiB,GAAA;AAAA,sBAAI,CAAA,CAAA,KAC7BC,+BAAA,CAAmB,CAAA,CAAE,MAAM;AAAA;AAC7B;AACF,iBACD;AAAA,eACH;AAEF,cAAA,WAAA,GAAc,YACX,MAAA,CAAOP,gCAAA,CAAgB,EAAE,GAAG,MAAA,CAAO,iBAAiB,CAAC,EACrD,MAAA,CAAO,oBAAA,CAAqB,CAAC,GAAG,MAAA,CAAO,gBAAgB,CAAC,CAAC,EACzD,MAAA,CAAO,oBAAA,CAAqB,CAAC,GAAG,MAAA,CAAO,SAAS,CAAC,CAAC,EAClD,MAAA,CAAO,oBAAA,CAAqB,CAAC,GAAG,MAAA,CAAO,WAAW,CAAC,CAAC,EACpD,MAAA,CAAO,oBAAA,CAAqB,CAAC,GAAG,OAAO,CAAC,CAAC,CAAA;AAAA,YAC9C;AAEA,YAAA,MAAM,UAAA,GAAa,WAAA,CAAY,MAAA,CAAO,KAAK,CAAA;AAC3C,YAAA,IAAI,eAAe,kBAAA,EAAoB;AAIrC,cAAA,KAAA,CAAM,2BAAA,EAA4B;AAClC,cAAA;AAAA,YACF;AASA,YAAA,IAAI,CAAC,OAAO,EAAA,EAAI;AAEd,cAAA,OAAA,CAAQ,OAAA,CAAQ,MAAS,CAAA,CACtB,IAAA;AAAA,gBAAK,MACJ,KAAK,iBAAA,GAAoB;AAAA,kBACvB,iBAAA;AAAA,kBACA,QAAQ,MAAA,CAAO;AAAA,iBAChB;AAAA,eACH,CACC,MAAM,CAAA,KAAA,KAAS;AACd,gBAAA,IAAA,CAAK,MAAA,CAAO,KAAA;AAAA,kBACV,CAAA,8CAAA,EAAiDQ,qBAAA;AAAA,oBAC/C;AAAA,mBACD,CAAA;AAAA,iBACH;AAAA,cACF,CAAC,CAAA;AAEH,cAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,OAAM,EAAA,KAAM;AACpD,gBAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,2BAAA,CAA4B,EAAA,EAAI;AAAA,kBAC5D,EAAA;AAAA,kBACA,MAAA,EAAQ,YAAA;AAAA,kBACR;AAAA,iBACD,CAAA;AAAA,cACH,CAAC,CAAA;AAED,cAAA,MAAM,IAAA,CAAK,SAAS,MAAA,CAAO;AAAA,gBACzB,UAAA,EAAY,CAACD,+BAAA,CAAmB,iBAAiB,CAAC;AAAA,eACnD,CAAA;AAED,cAAA,KAAA,CAAM,wBAAA,EAAyB;AAC/B,cAAA;AAAA,YACF;AAEA,YAAA,MAAA,CAAO,eAAA,CAAgB,SAAS,GAAA,GAAM,EAAA;AACtC,YAAA,IAAI,kBAAA;AACJ,YAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,OAAM,EAAA,KAAM;AACpD,cAAA,MAAM,EAAE,QAAA,EAAS,GACf,MAAM,IAAA,CAAK,kBAAA,CAAmB,sBAAsB,EAAA,EAAI;AAAA,gBACtD,EAAA;AAAA,gBACA,iBAAiB,MAAA,CAAO,eAAA;AAAA,gBACxB,UAAA;AAAA,gBACA,MAAA,EAAQ,YAAA;AAAA,gBACR,WAAW,MAAA,CAAO,SAAA;AAAA,gBAClB,kBAAkB,MAAA,CAAO,gBAAA;AAAA,gBACzB,WAAA;AAAA,gBACA,aAAa,MAAA,CAAO;AAAA,eACrB,CAAA;AACH,cAAA,kBAAA,GAAqB,IAAI,GAAA;AAAA,gBACvB,QAAA,CAAS,SAAA,CAAU,GAAA,CAAI,CAAA,CAAA,KAAK;AAAA,kBAC1B,CAAA,EAAG,EAAE,iBAAiB,CAAA,CAAA,EAAI,EAAE,IAAI,CAAA,EAAA,EAAK,EAAE,iBAAiB,CAAA,CAAA;AAAA,kBACxD,CAAA,CAAE;AAAA,iBACH;AAAA,eACH;AAAA,YACF,CAAC,CAAA;AAED,YAAA,MAAM,qBAAqB,IAAI,GAAA;AAAA,cAC7B,MAAA,CAAO,SAAA,CAAU,GAAA,CAAI,CAAA,QAAA,KAAY;AAC/B,gBAAA,MAAM,eAAA,GAAkBA,+BAAA,CAAmB,QAAA,CAAS,MAAM,CAAA;AAC1D,gBAAA,MAAM,eAAA,GAAkBA,+BAAA,CAAmB,QAAA,CAAS,MAAM,CAAA;AAC1D,gBAAA,OAAO;AAAA,kBACL,GAAG,eAAe,CAAA,CAAA,EAAI,QAAA,CAAS,IAAI,KAAK,eAAe,CAAA,CAAA;AAAA,kBACvD;AAAA,iBACF;AAAA,cACF,CAAC;AAAA,aACH;AAEA,YAAA,MAAM,mBAAA,uBAA0B,GAAA,CAAY;AAAA,cAC1CA,+BAAA,CAAmB,OAAO,eAAe;AAAA,aAC1C,CAAA;AACD,YAAA,kBAAA,CAAmB,OAAA,CAAQ,CAAC,eAAA,EAAiB,SAAA,KAAc;AACzD,cAAA,IAAI,CAAC,kBAAA,CAAmB,GAAA,CAAI,SAAS,CAAA,EAAG;AACtC,gBAAA,mBAAA,CAAoB,IAAI,eAAe,CAAA;AAAA,cACzC;AAAA,YACF,CAAC,CAAA;AACD,YAAA,kBAAA,CAAoB,OAAA,CAAQ,CAAC,eAAA,EAAiB,SAAA,KAAc;AAC1D,cAAA,IAAI,CAAC,kBAAA,CAAmB,GAAA,CAAI,SAAS,CAAA,EAAG;AACtC,gBAAA,mBAAA,CAAoB,IAAI,eAAe,CAAA;AAAA,cACzC;AAAA,YACF,CAAC,CAAA;AAED,YAAA,MAAM,IAAA,CAAK,SAAS,MAAA,CAAO;AAAA,cACzB,UAAA,EAAY;AAAA,aACb,CAAA;AAED,YAAA,KAAA,CAAM,yBAAA,EAA0B;AAAA,UAClC,SAAS,KAAA,EAAO;AACd,YAAA,KAAA,CAAM,UAAA,CAAWE,cAAA,CAAQ,KAAK,CAAC,CAAA;AAAA,UACjC;AAAA,QACF,CAAC,CAAA;AAAA,MACH;AAAA,KACD,CAAA;AAAA,EACH;AAAA,EAEQ,kBAAA,GAAiC;AACvC,IAAA,MAAM,cAAA,GACJ,IAAA,CAAK,MAAA,CAAO,iBAAA,CAAkB,wBAAwB,CAAA,IAAK,QAAA;AAC7D,IAAA,IAAI,mBAAmB,QAAA,EAAU;AAC/B,MAAA,OAAO,MAAM;AAAA,MAAC,CAAA;AAAA,IAChB;AAEA,IAAA,MAAM,iBAAA,GAAoBC,iCAAA,CAA4B,IAAA,CAAK,MAAM,CAAA;AAEjE,IAAA,MAAM,UAAU,YAAY;AAC1B,MAAA,IAAI;AACF,QAAA,MAAM,CAAA,GAAI,MAAMC,6CAAA,CAAuB;AAAA,UACrC,MAAM,IAAA,CAAK,IAAA;AAAA,UACX,QAAA,EAAU;AAAA,SACX,CAAA;AACD,QAAA,IAAI,IAAI,CAAA,EAAG;AACT,UAAA,IAAA,CAAK,MAAA,CAAO,IAAA,CAAK,CAAA,QAAA,EAAW,CAAC,CAAA,kBAAA,CAAoB,CAAA;AAAA,QACnD;AAAA,MACF,SAAS,KAAA,EAAO;AACd,QAAA,IAAA,CAAK,MAAA,CAAO,IAAA,CAAK,CAAA,kCAAA,CAAA,EAAsC,KAAK,CAAA;AAAA,MAC9D;AAAA,IACF,CAAA;AAEA,IAAA,MAAM,eAAA,GAAkB,IAAI,eAAA,EAAgB;AAC5C,IAAA,IAAA,CAAK,UAAU,YAAA,CAAa;AAAA,MAC1B,EAAA,EAAI,wBAAA;AAAA,MACJ,SAAA,EAAW,EAAE,YAAA,EAAc,IAAA,CAAK,uBAAA,EAAwB;AAAA,MACxD,OAAA,EAAS,EAAE,YAAA,EAAc,IAAA,CAAK,0BAA0B,GAAA,EAAI;AAAA,MAC5D,EAAA,EAAI,OAAA;AAAA,MACJ,QAAQ,eAAA,CAAgB;AAAA,KACzB,CAAA;AAED,IAAA,OAAO,MAAM;AACX,MAAA,eAAA,CAAgB,KAAA,EAAM;AAAA,IACxB,CAAA;AAAA,EACF;AACF;AAGA,SAAS,gBAAgBC,SAAA,EAAyB;AAEhD,EAAA,MAAM,wBAAwBC,2BAAA,CAAoB;AAAA,IAChD,IAAA,EAAM,kCAAA;AAAA,IACN,IAAA,EAAM,6EAAA;AAAA,IACN,UAAA,EAAY,CAAC,QAAQ;AAAA,GACtB,CAAA;AACD,EAAA,MAAM,yBAAyBC,2BAAA,CAAoB;AAAA,IACjD,IAAA,EAAM,qCAAA;AAAA,IACN,IAAA,EAAM,8FAAA;AAAA,IACN,UAAA,EAAY,CAAC,QAAQ;AAAA,GACtB,CAAA;AACD,EAAA,MAAM,yBAAyBA,2BAAA,CAAoB;AAAA,IACjD,IAAA,EAAM,qCAAA;AAAA,IACN,IAAA,EAAM,wFAAA;AAAA,IACN,UAAA,EAAY,CAAC,QAAQ;AAAA,GACtB,CAAA;AACD,EAAA,MAAM,2BAA2BA,2BAAA,CAAoB;AAAA,IACnD,IAAA,EAAM,wCAAA;AAAA,IACN,IAAA,EAAM;AAAA,GACP,CAAA;AAED,EAAA,MAAM,oBAAoBF,SAAA,CAAQ,aAAA;AAAA,IAChC,kCAAA;AAAA,IACA,EAAE,aAAa,8BAAA;AAA+B,GAChD;AAEA,EAAA,MAAM,qBAAqBA,SAAA,CAAQ,eAAA;AAAA,IACjC,6BAAA;AAAA,IACA;AAAA,MACE,WAAA,EAAa,+CAAA;AAAA,MACb,IAAA,EAAM;AAAA;AACR,GACF;AAEA,EAAA,MAAM,qBAAqBA,SAAA,CAAQ,eAAA;AAAA,IACjC,6BAAA;AAAA,IACA;AAAA,MACE,WAAA,EAAa,yCAAA;AAAA,MACb,IAAA,EAAM;AAAA;AACR,GACF;AAEA,EAAA,MAAM,uBAAuBA,SAAA,CAAQ,eAAA;AAAA,IACnC,gCAAA;AAAA,IACA;AAAA,MACE,WAAA,EACE,uGAAA;AAAA,MACF,IAAA,EAAM;AAAA;AACR,GACF;AAEA,EAAA,SAAS,YAAA,CAAa,MAAwB,MAAA,EAAuB;AACnE,IAAA,MAAM,SAAA,GAAY,QAAQ,MAAA,EAAO;AACjC,IAAA,MAAM,eAAA,GAAkB,uBAAuB,UAAA,EAAW;AAC1D,IAAA,MAAM,kBAAA,GAAqB,uBAAuB,UAAA,EAAW;AAE7D,IAAA,MAAA,CAAO,KAAA,CAAM,CAAA,WAAA,EAAc,IAAA,CAAK,SAAS,CAAA,CAAE,CAAA;AAE3C,IAAA,IAAI,KAAK,YAAA,EAAc;AACrB,MAAA,MAAM,UAAU,CAAC,IAAA,CAAK,aAAa,OAAA,EAAQ,CAAE,GAAG,SAAS,CAAA;AACzD,MAAA,wBAAA,CAAyB,QAAQ,OAAO,CAAA;AACxC,MAAA,oBAAA,CAAqB,OAAO,OAAO,CAAA;AAAA,IACrC;AAEA,IAAA,SAAS,OAAA,GAAU;AACjB,MAAA,MAAM,KAAA,GAAQ,OAAA,CAAQ,MAAA,CAAO,SAAS,CAAA;AACtC,MAAA,OAAO,KAAA,CAAM,CAAC,CAAA,GAAI,KAAA,CAAM,CAAC,CAAA,GAAI,GAAA;AAAA,IAC/B;AAEA,IAAA,SAAS,wBAAwB,MAAA,EAAgC;AAC/D,MAAA,kBAAA,CAAmB,EAAE,MAAA,EAAQ,MAAA,CAAO,EAAA,GAAK,IAAA,GAAO,UAAU,CAAA;AAC1D,MAAA,kBAAA,CAAmB,MAAA,CAAO,SAAQ,EAAG;AAAA,QACnC,MAAA,EAAQ,MAAA,CAAO,EAAA,GAAK,IAAA,GAAO;AAAA,OAC5B,CAAA;AAAA,IACH;AAEA,IAAA,SAAS,2BAAA,GAA8B;AACrC,MAAA,eAAA,CAAgB,EAAE,MAAA,EAAQ,WAAA,EAAa,CAAA;AACvC,MAAA,qBAAA,CAAsB,GAAA,CAAI,EAAE,MAAA,EAAQ,WAAA,IAAe,CAAC,CAAA;AAEpD,MAAA,kBAAA,CAAmB,OAAO,OAAA,EAAQ,EAAG,EAAE,MAAA,EAAQ,aAAa,CAAA;AAC5D,MAAA,iBAAA,CAAkB,GAAA,CAAI,CAAA,EAAG,EAAE,MAAA,EAAQ,aAAa,CAAA;AAAA,IAClD;AAEA,IAAA,SAAS,wBAAA,GAA2B;AAClC,MAAA,eAAA,CAAgB,EAAE,MAAA,EAAQ,QAAA,EAAU,CAAA;AACpC,MAAA,qBAAA,CAAsB,GAAA,CAAI,EAAE,MAAA,EAAQ,QAAA,IAAY,CAAC,CAAA;AAEjD,MAAA,kBAAA,CAAmB,OAAO,OAAA,EAAQ,EAAG,EAAE,MAAA,EAAQ,UAAU,CAAA;AACzD,MAAA,iBAAA,CAAkB,GAAA,CAAI,CAAA,EAAG,EAAE,MAAA,EAAQ,UAAU,CAAA;AAAA,IAC/C;AAEA,IAAA,SAAS,yBAAA,GAA4B;AACnC,MAAA,eAAA,CAAgB,EAAE,MAAA,EAAQ,SAAA,EAAW,CAAA;AACrC,MAAA,qBAAA,CAAsB,GAAA,CAAI,EAAE,MAAA,EAAQ,SAAA,IAAa,CAAC,CAAA;AAElD,MAAA,kBAAA,CAAmB,OAAO,OAAA,EAAQ,EAAG,EAAE,MAAA,EAAQ,WAAW,CAAA;AAC1D,MAAA,iBAAA,CAAkB,GAAA,CAAI,CAAA,EAAG,EAAE,MAAA,EAAQ,WAAW,CAAA;AAAA,IAChD;AAEA,IAAA,SAAS,WAAW,KAAA,EAAc;AAChC,MAAA,qBAAA,CAAsB,GAAA,CAAI,EAAE,MAAA,EAAQ,QAAA,IAAY,CAAC,CAAA;AACjD,MAAA,iBAAA,CAAkB,GAAA,CAAI,CAAA,EAAG,EAAE,MAAA,EAAQ,UAAU,CAAA;AAC7C,MAAA,MAAA,CAAO,IAAA,CAAK,CAAA,cAAA,EAAiB,IAAA,CAAK,SAAS,WAAW,KAAK,CAAA;AAAA,IAC7D;AAEA,IAAA,OAAO;AAAA,MACL,uBAAA;AAAA,MACA,2BAAA;AAAA,MACA,wBAAA;AAAA,MACA,yBAAA;AAAA,MACA;AAAA,KACF;AAAA,EACF;AAEA,EAAA,OAAO,EAAE,YAAA,EAAa;AACxB;;;;"}
+{"version":3,"file":"DefaultCatalogProcessingEngine.cjs.js","sources":["../../src/processing/DefaultCatalogProcessingEngine.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport {\n  ANNOTATION_LOCATION,\n  Entity,\n  stringifyEntityRef,\n} from '@backstage/catalog-model';\nimport { serializeError, stringifyError, toError } from '@backstage/errors';\nimport { Hash } from 'node:crypto';\nimport stableStringify from 'fast-json-stable-stringify';\nimport { Knex } from 'knex';\nimport { trace } from '@opentelemetry/api';\nimport { ProcessingDatabase, RefreshStateItem } from '../database/types';\nimport { createCounterMetric, createSummaryMetric } from '../util/metrics';\nimport { CatalogProcessingOrchestrator, EntityProcessingResult } from './types';\nimport { Stitcher, stitchingStrategyFromConfig } from '../stitching/types';\nimport { startTaskPipeline } from './TaskPipeline';\nimport { Config } from '@backstage/config';\nimport {\n  addEntityAttributes,\n  TRACER_ID,\n  withActiveSpan,\n} from '../util/opentelemetry';\nimport { deleteOrphanedEntities } from '../database/operations/util/deleteOrphanedEntities';\nimport { EventsService } from '@backstage/plugin-events-node';\nimport { CATALOG_ERRORS_TOPIC } from '../constants';\nimport { LoggerService, SchedulerService } from '@backstage/backend-plugin-api';\nimport { MetricsService } from '@backstage/backend-plugin-api/alpha';\n\nconst CACHE_TTL = 5;\n\nconst tracer = trace.getTracer(TRACER_ID);\n\nexport type ProgressTracker = ReturnType<typeof progressTracker>;\n\nconst stableStringifyArray = (arr: any[]) => {\n  const sorted = arr.map(stableStringify).sort();\n  return `[${sorted.join(',')}]`;\n};\n\n// NOTE(freben): Perhaps surprisingly, this class does not implement the\n// CatalogProcessingEngine type. That type is externally visible and its name is\n// the way it is for historic reasons. This class has no particular reason to\n// implement that precise interface; nowadays there are several different\n// engines \"hiding\" behind the CatalogProcessingEngine interface, of which this\n// is just one.\nexport class DefaultCatalogProcessingEngine {\n  private readonly config: Config;\n  private readonly scheduler: SchedulerService;\n  private readonly logger: LoggerService;\n  private readonly knex: Knex;\n  private readonly processingDatabase: ProcessingDatabase;\n  private readonly orchestrator: CatalogProcessingOrchestrator;\n  private readonly stitcher: Stitcher;\n  private readonly createHash: () => Hash;\n  private readonly pollingIntervalMs: number;\n  private readonly orphanCleanupIntervalMs: number;\n  private readonly onProcessingError?: (event: {\n    unprocessedEntity: Entity;\n    errors: Error[];\n  }) => Promise<void> | void;\n  private readonly tracker: ProgressTracker;\n  private readonly events: EventsService;\n\n  private stopFunc?: () => void;\n\n  constructor(options: {\n    config: Config;\n    scheduler: SchedulerService;\n    logger: LoggerService;\n    knex: Knex;\n    processingDatabase: ProcessingDatabase;\n    orchestrator: CatalogProcessingOrchestrator;\n    stitcher: Stitcher;\n    createHash: () => Hash;\n    pollingIntervalMs?: number;\n    orphanCleanupIntervalMs?: number;\n    onProcessingError?: (event: {\n      unprocessedEntity: Entity;\n      errors: Error[];\n    }) => Promise<void> | void;\n    tracker?: ProgressTracker;\n    events: EventsService;\n    metrics: MetricsService;\n  }) {\n    this.config = options.config;\n    this.scheduler = options.scheduler;\n    this.logger = options.logger;\n    this.knex = options.knex;\n    this.processingDatabase = options.processingDatabase;\n    this.orchestrator = options.orchestrator;\n    this.stitcher = options.stitcher;\n    this.createHash = options.createHash;\n    this.pollingIntervalMs = options.pollingIntervalMs ?? 1_000;\n    this.orphanCleanupIntervalMs = options.orphanCleanupIntervalMs ?? 30_000;\n    this.onProcessingError = options.onProcessingError;\n    this.tracker = options.tracker ?? progressTracker(options.metrics);\n    this.events = options.events;\n\n    this.stopFunc = undefined;\n  }\n\n  async start() {\n    if (this.stopFunc) {\n      throw new Error('Processing engine is already started');\n    }\n\n    const stopPipeline = this.startPipeline();\n    const stopCleanup = this.startOrphanCleanup();\n\n    this.stopFunc = () => {\n      stopPipeline();\n      stopCleanup();\n    };\n  }\n\n  async stop() {\n    if (this.stopFunc) {\n      this.stopFunc();\n      this.stopFunc = undefined;\n    }\n  }\n\n  private startPipeline(): () => void {\n    return startTaskPipeline<RefreshStateItem>({\n      lowWatermark: 5,\n      highWatermark: 10,\n      pollingIntervalMs: this.pollingIntervalMs,\n      loadTasks: async count => {\n        try {\n          const { items } = await this.processingDatabase.transaction(\n            async tx => {\n              return this.processingDatabase.getProcessableEntities(tx, {\n                processBatchSize: count,\n              });\n            },\n          );\n          return items;\n        } catch (error) {\n          this.logger.warn('Failed to load processing items', error);\n          return [];\n        }\n      },\n      processTask: async item => {\n        await withActiveSpan(tracer, 'ProcessingRun', async span => {\n          const track = this.tracker.processStart(item, this.logger);\n          addEntityAttributes(span, item.unprocessedEntity);\n\n          try {\n            const {\n              id,\n              state,\n              unprocessedEntity,\n              entityRef,\n              locationKey,\n              resultHash: previousResultHash,\n            } = item;\n            const result = await this.orchestrator.process({\n              entity: unprocessedEntity,\n              state,\n            });\n\n            track.markProcessorsCompleted(result);\n\n            if (result.ok) {\n              const { ttl: _, ...stateWithoutTtl } = state ?? {};\n              if (\n                stableStringify(stateWithoutTtl) !==\n                stableStringify(result.state)\n              ) {\n                await this.processingDatabase.transaction(async tx => {\n                  await this.processingDatabase.updateEntityCache(tx, {\n                    id,\n                    state: {\n                      ttl: CACHE_TTL,\n                      ...result.state,\n                    },\n                  });\n                });\n              }\n            } else {\n              const maybeTtl = state?.ttl;\n              const ttl = Number.isInteger(maybeTtl) ? (maybeTtl as number) : 0;\n              await this.processingDatabase.transaction(async tx => {\n                await this.processingDatabase.updateEntityCache(tx, {\n                  id,\n                  state: ttl > 0 ? { ...state, ttl: ttl - 1 } : {},\n                });\n              });\n            }\n\n            const location =\n              unprocessedEntity?.metadata?.annotations?.[ANNOTATION_LOCATION];\n            if (result.errors.length) {\n              this.events.publish({\n                topic: CATALOG_ERRORS_TOPIC,\n                eventPayload: {\n                  entity: entityRef,\n                  location,\n                  errors: result.errors,\n                },\n              });\n            }\n            const errorsString = JSON.stringify(\n              result.errors.map(e => serializeError(e)),\n            );\n\n            let hashBuilder = this.createHash().update(errorsString);\n\n            if (result.ok) {\n              const { entityRefs: parents } =\n                await this.processingDatabase.transaction(tx =>\n                  this.processingDatabase.listParents(tx, {\n                    entityRefs: [\n                      entityRef,\n                      ...result.deferredEntities.map(e =>\n                        stringifyEntityRef(e.entity),\n                      ),\n                    ],\n                  }),\n                );\n\n              hashBuilder = hashBuilder\n                .update(stableStringify({ ...result.completedEntity }))\n                .update(stableStringifyArray([...result.deferredEntities]))\n                .update(stableStringifyArray([...result.relations]))\n                .update(stableStringifyArray([...result.refreshKeys]))\n                .update(stableStringifyArray([...parents]));\n            }\n\n            const resultHash = hashBuilder.digest('hex');\n            if (resultHash === previousResultHash) {\n              // If nothing changed in our produced outputs, we cannot have any\n              // significant effect on our surroundings; therefore, we just abort\n              // without any updates / stitching.\n              track.markSuccessfulWithNoChanges();\n              return;\n            }\n\n            // If the result was marked as not OK, it signals that some part of the\n            // processing pipeline threw an exception. This can happen both as part of\n            // non-catastrophic things such as due to validation errors, as well as if\n            // something fatal happens inside the processing for other reasons. In any\n            // case, this means we can't trust that anything in the output is okay. So\n            // just store the errors and trigger a stitch so that they become visible to\n            // the outside.\n            if (!result.ok) {\n              // notify the error listener if the entity can not be processed.\n              Promise.resolve(undefined)\n                .then(() =>\n                  this.onProcessingError?.({\n                    unprocessedEntity,\n                    errors: result.errors,\n                  }),\n                )\n                .catch(error => {\n                  this.logger.debug(\n                    `Processing error listener threw an exception, ${stringifyError(\n                      error,\n                    )}`,\n                  );\n                });\n\n              await this.processingDatabase.transaction(async tx => {\n                await this.processingDatabase.updateProcessedEntityErrors(tx, {\n                  id,\n                  errors: errorsString,\n                  resultHash,\n                });\n              });\n\n              await this.stitcher.stitch({\n                entityRefs: [stringifyEntityRef(unprocessedEntity)],\n              });\n\n              track.markSuccessfulWithErrors();\n              return;\n            }\n\n            result.completedEntity.metadata.uid = id;\n            let oldRelationSources: Map<string, string>;\n            await this.processingDatabase.transaction(async tx => {\n              const { previous } =\n                await this.processingDatabase.updateProcessedEntity(tx, {\n                  id,\n                  processedEntity: result.completedEntity,\n                  resultHash,\n                  errors: errorsString,\n                  relations: result.relations,\n                  deferredEntities: result.deferredEntities,\n                  locationKey,\n                  refreshKeys: result.refreshKeys,\n                });\n              oldRelationSources = new Map(\n                previous.relations.map(r => [\n                  `${r.source_entity_ref}:${r.type}->${r.target_entity_ref}`,\n                  r.source_entity_ref,\n                ]),\n              );\n            });\n\n            const newRelationSources = new Map<string, string>(\n              result.relations.map(relation => {\n                const sourceEntityRef = stringifyEntityRef(relation.source);\n                const targetEntityRef = stringifyEntityRef(relation.target);\n                return [\n                  `${sourceEntityRef}:${relation.type}->${targetEntityRef}`,\n                  sourceEntityRef,\n                ];\n              }),\n            );\n\n            const setOfThingsToStitch = new Set<string>([\n              stringifyEntityRef(result.completedEntity),\n            ]);\n            newRelationSources.forEach((sourceEntityRef, uniqueKey) => {\n              if (!oldRelationSources.has(uniqueKey)) {\n                setOfThingsToStitch.add(sourceEntityRef);\n              }\n            });\n            oldRelationSources!.forEach((sourceEntityRef, uniqueKey) => {\n              if (!newRelationSources.has(uniqueKey)) {\n                setOfThingsToStitch.add(sourceEntityRef);\n              }\n            });\n\n            await this.stitcher.stitch({\n              entityRefs: setOfThingsToStitch,\n            });\n\n            track.markSuccessfulWithChanges();\n          } catch (error) {\n            track.markFailed(toError(error));\n          }\n        });\n      },\n    });\n  }\n\n  private startOrphanCleanup(): () => void {\n    const orphanStrategy =\n      this.config.getOptionalString('catalog.orphanStrategy') ?? 'delete';\n    if (orphanStrategy !== 'delete') {\n      return () => {};\n    }\n\n    const stitchingStrategy = stitchingStrategyFromConfig(this.config, {\n      logger: this.logger,\n    });\n\n    const runOnce = async () => {\n      try {\n        const n = await deleteOrphanedEntities({\n          knex: this.knex,\n          strategy: stitchingStrategy,\n        });\n        if (n > 0) {\n          this.logger.info(`Deleted ${n} orphaned entities`);\n        }\n      } catch (error) {\n        this.logger.warn(`Failed to delete orphaned entities`, error);\n      }\n    };\n\n    const abortController = new AbortController();\n    this.scheduler.scheduleTask({\n      id: 'catalog_orphan_cleanup',\n      frequency: { milliseconds: this.orphanCleanupIntervalMs },\n      timeout: { milliseconds: this.orphanCleanupIntervalMs * 0.8 },\n      fn: runOnce,\n      signal: abortController.signal,\n    });\n\n    return () => {\n      abortController.abort();\n    };\n  }\n}\n\n// Helps wrap the timing and logging behaviors\nfunction progressTracker(metrics: MetricsService) {\n  // prom-client metrics are deprecated in favour of OpenTelemetry metrics.\n  const promProcessedEntities = createCounterMetric({\n    name: 'catalog_processed_entities_count',\n    help: 'Amount of entities processed, DEPRECATED, use OpenTelemetry metrics instead',\n    labelNames: ['result'],\n  });\n  const promProcessingDuration = createSummaryMetric({\n    name: 'catalog_processing_duration_seconds',\n    help: 'Time spent executing the full processing flow, DEPRECATED, use OpenTelemetry metrics instead',\n    labelNames: ['result'],\n  });\n  const promProcessorsDuration = createSummaryMetric({\n    name: 'catalog_processors_duration_seconds',\n    help: 'Time spent executing catalog processors, DEPRECATED, use OpenTelemetry metrics instead',\n    labelNames: ['result'],\n  });\n  const promProcessingQueueDelay = createSummaryMetric({\n    name: 'catalog_processing_queue_delay_seconds',\n    help: 'The amount of delay between being scheduled for processing, and the start of actually being processed, DEPRECATED, use OpenTelemetry metrics instead',\n  });\n\n  const processedEntities = metrics.createCounter(\n    'catalog.processed.entities.count',\n    { description: 'Amount of entities processed' },\n  );\n\n  const processingDuration = metrics.createHistogram(\n    'catalog.processing.duration',\n    {\n      description: 'Time spent executing the full processing flow',\n      unit: 'seconds',\n    },\n  );\n\n  const processorsDuration = metrics.createHistogram(\n    'catalog.processors.duration',\n    {\n      description: 'Time spent executing catalog processors',\n      unit: 'seconds',\n    },\n  );\n\n  const processingQueueDelay = metrics.createHistogram(\n    'catalog.processing.queue.delay',\n    {\n      description:\n        'The amount of delay between being scheduled for processing, and the start of actually being processed',\n      unit: 'seconds',\n    },\n  );\n\n  function processStart(item: RefreshStateItem, logger: LoggerService) {\n    const startTime = process.hrtime();\n    const endOverallTimer = promProcessingDuration.startTimer();\n    const endProcessorsTimer = promProcessorsDuration.startTimer();\n\n    logger.debug(`Processing ${item.entityRef}`);\n\n    if (item.nextUpdateAt) {\n      const seconds = -item.nextUpdateAt.diffNow().as('seconds');\n      promProcessingQueueDelay.observe(seconds);\n      processingQueueDelay.record(seconds);\n    }\n\n    function endTime() {\n      const delta = process.hrtime(startTime);\n      return delta[0] + delta[1] / 1e9;\n    }\n\n    function markProcessorsCompleted(result: EntityProcessingResult) {\n      endProcessorsTimer({ result: result.ok ? 'ok' : 'failed' });\n      processorsDuration.record(endTime(), {\n        result: result.ok ? 'ok' : 'failed',\n      });\n    }\n\n    function markSuccessfulWithNoChanges() {\n      endOverallTimer({ result: 'unchanged' });\n      promProcessedEntities.inc({ result: 'unchanged' }, 1);\n\n      processingDuration.record(endTime(), { result: 'unchanged' });\n      processedEntities.add(1, { result: 'unchanged' });\n    }\n\n    function markSuccessfulWithErrors() {\n      endOverallTimer({ result: 'errors' });\n      promProcessedEntities.inc({ result: 'errors' }, 1);\n\n      processingDuration.record(endTime(), { result: 'errors' });\n      processedEntities.add(1, { result: 'errors' });\n    }\n\n    function markSuccessfulWithChanges() {\n      endOverallTimer({ result: 'changed' });\n      promProcessedEntities.inc({ result: 'changed' }, 1);\n\n      processingDuration.record(endTime(), { result: 'changed' });\n      processedEntities.add(1, { result: 'changed' });\n    }\n\n    function markFailed(error: Error) {\n      promProcessedEntities.inc({ result: 'failed' }, 1);\n      processedEntities.add(1, { result: 'failed' });\n      logger.warn(`Processing of ${item.entityRef} failed`, error);\n    }\n\n    return {\n      markProcessorsCompleted,\n      markSuccessfulWithNoChanges,\n      markSuccessfulWithErrors,\n      markSuccessfulWithChanges,\n      markFailed,\n    };\n  }\n\n  return { processStart };\n}\n"],"names":["trace","TRACER_ID","stableStringify","startTaskPipeline","withActiveSpan","addEntityAttributes","ANNOTATION_LOCATION","CATALOG_ERRORS_TOPIC","serializeError","stringifyEntityRef","stringifyError","toError","stitchingStrategyFromConfig","deleteOrphanedEntities","metrics","createCounterMetric","createSummaryMetric"],"mappings":";;;;;;;;;;;;;;;;;AA2CA,MAAM,SAAA,GAAY,CAAA;AAElB,MAAM,MAAA,GAASA,SAAA,CAAM,SAAA,CAAUC,uBAAS,CAAA;AAIxC,MAAM,oBAAA,GAAuB,CAAC,GAAA,KAAe;AAC3C,EAAA,MAAM,MAAA,GAAS,GAAA,CAAI,GAAA,CAAIC,gCAAe,EAAE,IAAA,EAAK;AAC7C,EAAA,OAAO,CAAA,CAAA,EAAI,MAAA,CAAO,IAAA,CAAK,GAAG,CAAC,CAAA,CAAA,CAAA;AAC7B,CAAA;AAQO,MAAM,8BAAA,CAA+B;AAAA,EACzB,MAAA;AAAA,EACA,SAAA;AAAA,EACA,MAAA;AAAA,EACA,IAAA;AAAA,EACA,kBAAA;AAAA,EACA,YAAA;AAAA,EACA,QAAA;AAAA,EACA,UAAA;AAAA,EACA,iBAAA;AAAA,EACA,uBAAA;AAAA,EACA,iBAAA;AAAA,EAIA,OAAA;AAAA,EACA,MAAA;AAAA,EAET,QAAA;AAAA,EAER,YAAY,OAAA,EAkBT;AACD,IAAA,IAAA,CAAK,SAAS,OAAA,CAAQ,MAAA;AACtB,IAAA,IAAA,CAAK,YAAY,OAAA,CAAQ,SAAA;AACzB,IAAA,IAAA,CAAK,SAAS,OAAA,CAAQ,MAAA;AACtB,IAAA,IAAA,CAAK,OAAO,OAAA,CAAQ,IAAA;AACpB,IAAA,IAAA,CAAK,qBAAqB,OAAA,CAAQ,kBAAA;AAClC,IAAA,IAAA,CAAK,eAAe,OAAA,CAAQ,YAAA;AAC5B,IAAA,IAAA,CAAK,WAAW,OAAA,CAAQ,QAAA;AACxB,IAAA,IAAA,CAAK,aAAa,OAAA,CAAQ,UAAA;AAC1B,IAAA,IAAA,CAAK,iBAAA,GAAoB,QAAQ,iBAAA,IAAqB,GAAA;AACtD,IAAA,IAAA,CAAK,uBAAA,GAA0B,QAAQ,uBAAA,IAA2B,GAAA;AAClE,IAAA,IAAA,CAAK,oBAAoB,OAAA,CAAQ,iBAAA;AACjC,IAAA,IAAA,CAAK,OAAA,GAAU,OAAA,CAAQ,OAAA,IAAW,eAAA,CAAgB,QAAQ,OAAO,CAAA;AACjE,IAAA,IAAA,CAAK,SAAS,OAAA,CAAQ,MAAA;AAEtB,IAAA,IAAA,CAAK,QAAA,GAAW,MAAA;AAAA,EAClB;AAAA,EAEA,MAAM,KAAA,GAAQ;AACZ,IAAA,IAAI,KAAK,QAAA,EAAU;AACjB,MAAA,MAAM,IAAI,MAAM,sCAAsC,CAAA;AAAA,IACxD;AAEA,IAAA,MAAM,YAAA,GAAe,KAAK,aAAA,EAAc;AACxC,IAAA,MAAM,WAAA,GAAc,KAAK,kBAAA,EAAmB;AAE5C,IAAA,IAAA,CAAK,WAAW,MAAM;AACpB,MAAA,YAAA,EAAa;AACb,MAAA,WAAA,EAAY;AAAA,IACd,CAAA;AAAA,EACF;AAAA,EAEA,MAAM,IAAA,GAAO;AACX,IAAA,IAAI,KAAK,QAAA,EAAU;AACjB,MAAA,IAAA,CAAK,QAAA,EAAS;AACd,MAAA,IAAA,CAAK,QAAA,GAAW,MAAA;AAAA,IAClB;AAAA,EACF;AAAA,EAEQ,aAAA,GAA4B;AAClC,IAAA,OAAOC,8BAAA,CAAoC;AAAA,MACzC,YAAA,EAAc,CAAA;AAAA,MACd,aAAA,EAAe,EAAA;AAAA,MACf,mBAAmB,IAAA,CAAK,iBAAA;AAAA,MACxB,SAAA,EAAW,OAAM,KAAA,KAAS;AACxB,QAAA,IAAI;AACF,UAAA,MAAM,EAAE,KAAA,EAAM,GAAI,MAAM,KAAK,kBAAA,CAAmB,WAAA;AAAA,YAC9C,OAAM,EAAA,KAAM;AACV,cAAA,OAAO,IAAA,CAAK,kBAAA,CAAmB,sBAAA,CAAuB,EAAA,EAAI;AAAA,gBACxD,gBAAA,EAAkB;AAAA,eACnB,CAAA;AAAA,YACH;AAAA,WACF;AACA,UAAA,OAAO,KAAA;AAAA,QACT,SAAS,KAAA,EAAO;AACd,UAAA,IAAA,CAAK,MAAA,CAAO,IAAA,CAAK,iCAAA,EAAmC,KAAK,CAAA;AACzD,UAAA,OAAO,EAAC;AAAA,QACV;AAAA,MACF,CAAA;AAAA,MACA,WAAA,EAAa,OAAM,IAAA,KAAQ;AACzB,QAAA,MAAMC,4BAAA,CAAe,MAAA,EAAQ,eAAA,EAAiB,OAAM,IAAA,KAAQ;AAC1D,UAAA,MAAM,QAAQ,IAAA,CAAK,OAAA,CAAQ,YAAA,CAAa,IAAA,EAAM,KAAK,MAAM,CAAA;AACzD,UAAAC,iCAAA,CAAoB,IAAA,EAAM,KAAK,iBAAiB,CAAA;AAEhD,UAAA,IAAI;AACF,YAAA,MAAM;AAAA,cACJ,EAAA;AAAA,cACA,KAAA;AAAA,cACA,iBAAA;AAAA,cACA,SAAA;AAAA,cACA,WAAA;AAAA,cACA,UAAA,EAAY;AAAA,aACd,GAAI,IAAA;AACJ,YAAA,MAAM,MAAA,GAAS,MAAM,IAAA,CAAK,YAAA,CAAa,OAAA,CAAQ;AAAA,cAC7C,MAAA,EAAQ,iBAAA;AAAA,cACR;AAAA,aACD,CAAA;AAED,YAAA,KAAA,CAAM,wBAAwB,MAAM,CAAA;AAEpC,YAAA,IAAI,OAAO,EAAA,EAAI;AACb,cAAA,MAAM,EAAE,GAAA,EAAK,CAAA,EAAG,GAAG,eAAA,EAAgB,GAAI,SAAS,EAAC;AACjD,cAAA,IACEH,iCAAgB,eAAe,CAAA,KAC/BA,gCAAA,CAAgB,MAAA,CAAO,KAAK,CAAA,EAC5B;AACA,gBAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,OAAM,EAAA,KAAM;AACpD,kBAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,iBAAA,CAAkB,EAAA,EAAI;AAAA,oBAClD,EAAA;AAAA,oBACA,KAAA,EAAO;AAAA,sBACL,GAAA,EAAK,SAAA;AAAA,sBACL,GAAG,MAAA,CAAO;AAAA;AACZ,mBACD,CAAA;AAAA,gBACH,CAAC,CAAA;AAAA,cACH;AAAA,YACF,CAAA,MAAO;AACL,cAAA,MAAM,WAAW,KAAA,EAAO,GAAA;AACxB,cAAA,MAAM,GAAA,GAAM,MAAA,CAAO,SAAA,CAAU,QAAQ,IAAK,QAAA,GAAsB,CAAA;AAChE,cAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,OAAM,EAAA,KAAM;AACpD,gBAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,iBAAA,CAAkB,EAAA,EAAI;AAAA,kBAClD,EAAA;AAAA,kBACA,KAAA,EAAO,GAAA,GAAM,CAAA,GAAI,EAAE,GAAG,OAAO,GAAA,EAAK,GAAA,GAAM,CAAA,EAAE,GAAI;AAAC,iBAChD,CAAA;AAAA,cACH,CAAC,CAAA;AAAA,YACH;AAEA,YAAA,MAAM,QAAA,GACJ,iBAAA,EAAmB,QAAA,EAAU,WAAA,GAAcI,gCAAmB,CAAA;AAChE,YAAA,IAAI,MAAA,CAAO,OAAO,MAAA,EAAQ;AACxB,cAAA,IAAA,CAAK,OAAO,OAAA,CAAQ;AAAA,gBAClB,KAAA,EAAOC,8BAAA;AAAA,gBACP,YAAA,EAAc;AAAA,kBACZ,MAAA,EAAQ,SAAA;AAAA,kBACR,QAAA;AAAA,kBACA,QAAQ,MAAA,CAAO;AAAA;AACjB,eACD,CAAA;AAAA,YACH;AACA,YAAA,MAAM,eAAe,IAAA,CAAK,SAAA;AAAA,cACxB,OAAO,MAAA,CAAO,GAAA,CAAI,CAAA,CAAA,KAAKC,qBAAA,CAAe,CAAC,CAAC;AAAA,aAC1C;AAEA,YAAA,IAAI,WAAA,GAAc,IAAA,CAAK,UAAA,EAAW,CAAE,OAAO,YAAY,CAAA;AAEvD,YAAA,IAAI,OAAO,EAAA,EAAI;AACb,cAAA,MAAM,EAAE,UAAA,EAAY,OAAA,EAAQ,GAC1B,MAAM,KAAK,kBAAA,CAAmB,WAAA;AAAA,gBAAY,CAAA,EAAA,KACxC,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,EAAA,EAAI;AAAA,kBACtC,UAAA,EAAY;AAAA,oBACV,SAAA;AAAA,oBACA,GAAG,OAAO,gBAAA,CAAiB,GAAA;AAAA,sBAAI,CAAA,CAAA,KAC7BC,+BAAA,CAAmB,CAAA,CAAE,MAAM;AAAA;AAC7B;AACF,iBACD;AAAA,eACH;AAEF,cAAA,WAAA,GAAc,YACX,MAAA,CAAOP,gCAAA,CAAgB,EAAE,GAAG,MAAA,CAAO,iBAAiB,CAAC,EACrD,MAAA,CAAO,oBAAA,CAAqB,CAAC,GAAG,MAAA,CAAO,gBAAgB,CAAC,CAAC,EACzD,MAAA,CAAO,oBAAA,CAAqB,CAAC,GAAG,MAAA,CAAO,SAAS,CAAC,CAAC,EAClD,MAAA,CAAO,oBAAA,CAAqB,CAAC,GAAG,MAAA,CAAO,WAAW,CAAC,CAAC,EACpD,MAAA,CAAO,oBAAA,CAAqB,CAAC,GAAG,OAAO,CAAC,CAAC,CAAA;AAAA,YAC9C;AAEA,YAAA,MAAM,UAAA,GAAa,WAAA,CAAY,MAAA,CAAO,KAAK,CAAA;AAC3C,YAAA,IAAI,eAAe,kBAAA,EAAoB;AAIrC,cAAA,KAAA,CAAM,2BAAA,EAA4B;AAClC,cAAA;AAAA,YACF;AASA,YAAA,IAAI,CAAC,OAAO,EAAA,EAAI;AAEd,cAAA,OAAA,CAAQ,OAAA,CAAQ,MAAS,CAAA,CACtB,IAAA;AAAA,gBAAK,MACJ,KAAK,iBAAA,GAAoB;AAAA,kBACvB,iBAAA;AAAA,kBACA,QAAQ,MAAA,CAAO;AAAA,iBAChB;AAAA,eACH,CACC,MAAM,CAAA,KAAA,KAAS;AACd,gBAAA,IAAA,CAAK,MAAA,CAAO,KAAA;AAAA,kBACV,CAAA,8CAAA,EAAiDQ,qBAAA;AAAA,oBAC/C;AAAA,mBACD,CAAA;AAAA,iBACH;AAAA,cACF,CAAC,CAAA;AAEH,cAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,OAAM,EAAA,KAAM;AACpD,gBAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,2BAAA,CAA4B,EAAA,EAAI;AAAA,kBAC5D,EAAA;AAAA,kBACA,MAAA,EAAQ,YAAA;AAAA,kBACR;AAAA,iBACD,CAAA;AAAA,cACH,CAAC,CAAA;AAED,cAAA,MAAM,IAAA,CAAK,SAAS,MAAA,CAAO;AAAA,gBACzB,UAAA,EAAY,CAACD,+BAAA,CAAmB,iBAAiB,CAAC;AAAA,eACnD,CAAA;AAED,cAAA,KAAA,CAAM,wBAAA,EAAyB;AAC/B,cAAA;AAAA,YACF;AAEA,YAAA,MAAA,CAAO,eAAA,CAAgB,SAAS,GAAA,GAAM,EAAA;AACtC,YAAA,IAAI,kBAAA;AACJ,YAAA,MAAM,IAAA,CAAK,kBAAA,CAAmB,WAAA,CAAY,OAAM,EAAA,KAAM;AACpD,cAAA,MAAM,EAAE,QAAA,EAAS,GACf,MAAM,IAAA,CAAK,kBAAA,CAAmB,sBAAsB,EAAA,EAAI;AAAA,gBACtD,EAAA;AAAA,gBACA,iBAAiB,MAAA,CAAO,eAAA;AAAA,gBACxB,UAAA;AAAA,gBACA,MAAA,EAAQ,YAAA;AAAA,gBACR,WAAW,MAAA,CAAO,SAAA;AAAA,gBAClB,kBAAkB,MAAA,CAAO,gBAAA;AAAA,gBACzB,WAAA;AAAA,gBACA,aAAa,MAAA,CAAO;AAAA,eACrB,CAAA;AACH,cAAA,kBAAA,GAAqB,IAAI,GAAA;AAAA,gBACvB,QAAA,CAAS,SAAA,CAAU,GAAA,CAAI,CAAA,CAAA,KAAK;AAAA,kBAC1B,CAAA,EAAG,EAAE,iBAAiB,CAAA,CAAA,EAAI,EAAE,IAAI,CAAA,EAAA,EAAK,EAAE,iBAAiB,CAAA,CAAA;AAAA,kBACxD,CAAA,CAAE;AAAA,iBACH;AAAA,eACH;AAAA,YACF,CAAC,CAAA;AAED,YAAA,MAAM,qBAAqB,IAAI,GAAA;AAAA,cAC7B,MAAA,CAAO,SAAA,CAAU,GAAA,CAAI,CAAA,QAAA,KAAY;AAC/B,gBAAA,MAAM,eAAA,GAAkBA,+BAAA,CAAmB,QAAA,CAAS,MAAM,CAAA;AAC1D,gBAAA,MAAM,eAAA,GAAkBA,+BAAA,CAAmB,QAAA,CAAS,MAAM,CAAA;AAC1D,gBAAA,OAAO;AAAA,kBACL,GAAG,eAAe,CAAA,CAAA,EAAI,QAAA,CAAS,IAAI,KAAK,eAAe,CAAA,CAAA;AAAA,kBACvD;AAAA,iBACF;AAAA,cACF,CAAC;AAAA,aACH;AAEA,YAAA,MAAM,mBAAA,uBAA0B,GAAA,CAAY;AAAA,cAC1CA,+BAAA,CAAmB,OAAO,eAAe;AAAA,aAC1C,CAAA;AACD,YAAA,kBAAA,CAAmB,OAAA,CAAQ,CAAC,eAAA,EAAiB,SAAA,KAAc;AACzD,cAAA,IAAI,CAAC,kBAAA,CAAmB,GAAA,CAAI,SAAS,CAAA,EAAG;AACtC,gBAAA,mBAAA,CAAoB,IAAI,eAAe,CAAA;AAAA,cACzC;AAAA,YACF,CAAC,CAAA;AACD,YAAA,kBAAA,CAAoB,OAAA,CAAQ,CAAC,eAAA,EAAiB,SAAA,KAAc;AAC1D,cAAA,IAAI,CAAC,kBAAA,CAAmB,GAAA,CAAI,SAAS,CAAA,EAAG;AACtC,gBAAA,mBAAA,CAAoB,IAAI,eAAe,CAAA;AAAA,cACzC;AAAA,YACF,CAAC,CAAA;AAED,YAAA,MAAM,IAAA,CAAK,SAAS,MAAA,CAAO;AAAA,cACzB,UAAA,EAAY;AAAA,aACb,CAAA;AAED,YAAA,KAAA,CAAM,yBAAA,EAA0B;AAAA,UAClC,SAAS,KAAA,EAAO;AACd,YAAA,KAAA,CAAM,UAAA,CAAWE,cAAA,CAAQ,KAAK,CAAC,CAAA;AAAA,UACjC;AAAA,QACF,CAAC,CAAA;AAAA,MACH;AAAA,KACD,CAAA;AAAA,EACH;AAAA,EAEQ,kBAAA,GAAiC;AACvC,IAAA,MAAM,cAAA,GACJ,IAAA,CAAK,MAAA,CAAO,iBAAA,CAAkB,wBAAwB,CAAA,IAAK,QAAA;AAC7D,IAAA,IAAI,mBAAmB,QAAA,EAAU;AAC/B,MAAA,OAAO,MAAM;AAAA,MAAC,CAAA;AAAA,IAChB;AAEA,IAAA,MAAM,iBAAA,GAAoBC,iCAAA,CAA4B,IAAA,CAAK,MAAA,EAAQ;AAAA,MACjE,QAAQ,IAAA,CAAK;AAAA,KACd,CAAA;AAED,IAAA,MAAM,UAAU,YAAY;AAC1B,MAAA,IAAI;AACF,QAAA,MAAM,CAAA,GAAI,MAAMC,6CAAA,CAAuB;AAAA,UACrC,MAAM,IAAA,CAAK,IAAA;AAAA,UACX,QAAA,EAAU;AAAA,SACX,CAAA;AACD,QAAA,IAAI,IAAI,CAAA,EAAG;AACT,UAAA,IAAA,CAAK,MAAA,CAAO,IAAA,CAAK,CAAA,QAAA,EAAW,CAAC,CAAA,kBAAA,CAAoB,CAAA;AAAA,QACnD;AAAA,MACF,SAAS,KAAA,EAAO;AACd,QAAA,IAAA,CAAK,MAAA,CAAO,IAAA,CAAK,CAAA,kCAAA,CAAA,EAAsC,KAAK,CAAA;AAAA,MAC9D;AAAA,IACF,CAAA;AAEA,IAAA,MAAM,eAAA,GAAkB,IAAI,eAAA,EAAgB;AAC5C,IAAA,IAAA,CAAK,UAAU,YAAA,CAAa;AAAA,MAC1B,EAAA,EAAI,wBAAA;AAAA,MACJ,SAAA,EAAW,EAAE,YAAA,EAAc,IAAA,CAAK,uBAAA,EAAwB;AAAA,MACxD,OAAA,EAAS,EAAE,YAAA,EAAc,IAAA,CAAK,0BAA0B,GAAA,EAAI;AAAA,MAC5D,EAAA,EAAI,OAAA;AAAA,MACJ,QAAQ,eAAA,CAAgB;AAAA,KACzB,CAAA;AAED,IAAA,OAAO,MAAM;AACX,MAAA,eAAA,CAAgB,KAAA,EAAM;AAAA,IACxB,CAAA;AAAA,EACF;AACF;AAGA,SAAS,gBAAgBC,SAAA,EAAyB;AAEhD,EAAA,MAAM,wBAAwBC,2BAAA,CAAoB;AAAA,IAChD,IAAA,EAAM,kCAAA;AAAA,IACN,IAAA,EAAM,6EAAA;AAAA,IACN,UAAA,EAAY,CAAC,QAAQ;AAAA,GACtB,CAAA;AACD,EAAA,MAAM,yBAAyBC,2BAAA,CAAoB;AAAA,IACjD,IAAA,EAAM,qCAAA;AAAA,IACN,IAAA,EAAM,8FAAA;AAAA,IACN,UAAA,EAAY,CAAC,QAAQ;AAAA,GACtB,CAAA;AACD,EAAA,MAAM,yBAAyBA,2BAAA,CAAoB;AAAA,IACjD,IAAA,EAAM,qCAAA;AAAA,IACN,IAAA,EAAM,wFAAA;AAAA,IACN,UAAA,EAAY,CAAC,QAAQ;AAAA,GACtB,CAAA;AACD,EAAA,MAAM,2BAA2BA,2BAAA,CAAoB;AAAA,IACnD,IAAA,EAAM,wCAAA;AAAA,IACN,IAAA,EAAM;AAAA,GACP,CAAA;AAED,EAAA,MAAM,oBAAoBF,SAAA,CAAQ,aAAA;AAAA,IAChC,kCAAA;AAAA,IACA,EAAE,aAAa,8BAAA;AAA+B,GAChD;AAEA,EAAA,MAAM,qBAAqBA,SAAA,CAAQ,eAAA;AAAA,IACjC,6BAAA;AAAA,IACA;AAAA,MACE,WAAA,EAAa,+CAAA;AAAA,MACb,IAAA,EAAM;AAAA;AACR,GACF;AAEA,EAAA,MAAM,qBAAqBA,SAAA,CAAQ,eAAA;AAAA,IACjC,6BAAA;AAAA,IACA;AAAA,MACE,WAAA,EAAa,yCAAA;AAAA,MACb,IAAA,EAAM;AAAA;AACR,GACF;AAEA,EAAA,MAAM,uBAAuBA,SAAA,CAAQ,eAAA;AAAA,IACnC,gCAAA;AAAA,IACA;AAAA,MACE,WAAA,EACE,uGAAA;AAAA,MACF,IAAA,EAAM;AAAA;AACR,GACF;AAEA,EAAA,SAAS,YAAA,CAAa,MAAwB,MAAA,EAAuB;AACnE,IAAA,MAAM,SAAA,GAAY,QAAQ,MAAA,EAAO;AACjC,IAAA,MAAM,eAAA,GAAkB,uBAAuB,UAAA,EAAW;AAC1D,IAAA,MAAM,kBAAA,GAAqB,uBAAuB,UAAA,EAAW;AAE7D,IAAA,MAAA,CAAO,KAAA,CAAM,CAAA,WAAA,EAAc,IAAA,CAAK,SAAS,CAAA,CAAE,CAAA;AAE3C,IAAA,IAAI,KAAK,YAAA,EAAc;AACrB,MAAA,MAAM,UAAU,CAAC,IAAA,CAAK,aAAa,OAAA,EAAQ,CAAE,GAAG,SAAS,CAAA;AACzD,MAAA,wBAAA,CAAyB,QAAQ,OAAO,CAAA;AACxC,MAAA,oBAAA,CAAqB,OAAO,OAAO,CAAA;AAAA,IACrC;AAEA,IAAA,SAAS,OAAA,GAAU;AACjB,MAAA,MAAM,KAAA,GAAQ,OAAA,CAAQ,MAAA,CAAO,SAAS,CAAA;AACtC,MAAA,OAAO,KAAA,CAAM,CAAC,CAAA,GAAI,KAAA,CAAM,CAAC,CAAA,GAAI,GAAA;AAAA,IAC/B;AAEA,IAAA,SAAS,wBAAwB,MAAA,EAAgC;AAC/D,MAAA,kBAAA,CAAmB,EAAE,MAAA,EAAQ,MAAA,CAAO,EAAA,GAAK,IAAA,GAAO,UAAU,CAAA;AAC1D,MAAA,kBAAA,CAAmB,MAAA,CAAO,SAAQ,EAAG;AAAA,QACnC,MAAA,EAAQ,MAAA,CAAO,EAAA,GAAK,IAAA,GAAO;AAAA,OAC5B,CAAA;AAAA,IACH;AAEA,IAAA,SAAS,2BAAA,GAA8B;AACrC,MAAA,eAAA,CAAgB,EAAE,MAAA,EAAQ,WAAA,EAAa,CAAA;AACvC,MAAA,qBAAA,CAAsB,GAAA,CAAI,EAAE,MAAA,EAAQ,WAAA,IAAe,CAAC,CAAA;AAEpD,MAAA,kBAAA,CAAmB,OAAO,OAAA,EAAQ,EAAG,EAAE,MAAA,EAAQ,aAAa,CAAA;AAC5D,MAAA,iBAAA,CAAkB,GAAA,CAAI,CAAA,EAAG,EAAE,MAAA,EAAQ,aAAa,CAAA;AAAA,IAClD;AAEA,IAAA,SAAS,wBAAA,GAA2B;AAClC,MAAA,eAAA,CAAgB,EAAE,MAAA,EAAQ,QAAA,EAAU,CAAA;AACpC,MAAA,qBAAA,CAAsB,GAAA,CAAI,EAAE,MAAA,EAAQ,QAAA,IAAY,CAAC,CAAA;AAEjD,MAAA,kBAAA,CAAmB,OAAO,OAAA,EAAQ,EAAG,EAAE,MAAA,EAAQ,UAAU,CAAA;AACzD,MAAA,iBAAA,CAAkB,GAAA,CAAI,CAAA,EAAG,EAAE,MAAA,EAAQ,UAAU,CAAA;AAAA,IAC/C;AAEA,IAAA,SAAS,yBAAA,GAA4B;AACnC,MAAA,eAAA,CAAgB,EAAE,MAAA,EAAQ,SAAA,EAAW,CAAA;AACrC,MAAA,qBAAA,CAAsB,GAAA,CAAI,EAAE,MAAA,EAAQ,SAAA,IAAa,CAAC,CAAA;AAElD,MAAA,kBAAA,CAAmB,OAAO,OAAA,EAAQ,EAAG,EAAE,MAAA,EAAQ,WAAW,CAAA;AAC1D,MAAA,iBAAA,CAAkB,GAAA,CAAI,CAAA,EAAG,EAAE,MAAA,EAAQ,WAAW,CAAA;AAAA,IAChD;AAEA,IAAA,SAAS,WAAW,KAAA,EAAc;AAChC,MAAA,qBAAA,CAAsB,GAAA,CAAI,EAAE,MAAA,EAAQ,QAAA,IAAY,CAAC,CAAA;AACjD,MAAA,iBAAA,CAAkB,GAAA,CAAI,CAAA,EAAG,EAAE,MAAA,EAAQ,UAAU,CAAA;AAC7C,MAAA,MAAA,CAAO,IAAA,CAAK,CAAA,cAAA,EAAiB,IAAA,CAAK,SAAS,WAAW,KAAK,CAAA;AAAA,IAC7D;AAEA,IAAA,OAAO;AAAA,MACL,uBAAA;AAAA,MACA,2BAAA;AAAA,MACA,wBAAA;AAAA,MACA,yBAAA;AAAA,MACA;AAAA,KACF;AAAA,EACF;AAEA,EAAA,OAAO,EAAE,YAAA,EAAa;AACxB;;;;"}

package/dist/service/CatalogPlugin.cjs.js

@@ -206,7 +206,10 @@ const catalogPlugin = backendPluginApi.createBackendPlugin({
         index.createCatalogActions({
           catalog,
           actionsRegistry,
-          modelHolder
+          modelHolder,
+          useExperimentalCatalogLayersDescriptions: config.getOptionalBoolean(
+            "catalog.actions.experimentalCatalogLayersDescriptions.enabled"
+          ) ?? false
         });
         const scmEventsMessagesCounter = metrics.createCounter("catalog.events.scm.messages", {
           description: "Number of SCM event messages received by the catalog backend",

package/dist/service/CatalogPlugin.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"CatalogPlugin.cjs.js","sources":["../../src/service/CatalogPlugin.ts"],"sourcesContent":["/*\n * Copyright 2022 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport {\n  coreServices,\n  createBackendPlugin,\n} from '@backstage/backend-plugin-api';\nimport {\n  actionsRegistryServiceRef,\n  metricsServiceRef,\n} from '@backstage/backend-plugin-api/alpha';\nimport { Entity, Validators } from '@backstage/catalog-model';\nimport { CatalogModelSource } from '@backstage/catalog-model/alpha';\nimport { ForwardedError } from '@backstage/errors';\nimport {\n  catalogAnalysisExtensionPoint,\n  CatalogLocationsExtensionPoint,\n  catalogLocationsExtensionPoint,\n  catalogProcessingExtensionPoint,\n  CatalogProcessor,\n  CatalogProcessorParser,\n  catalogServiceRef,\n  LocationAnalyzer,\n  PlaceholderResolver,\n  ScmLocationAnalyzer,\n} from '@backstage/plugin-catalog-node';\nimport {\n  CatalogModelExtensionPoint,\n  catalogModelExtensionPoint,\n  catalogScmEventsServiceRef,\n} from '@backstage/plugin-catalog-node/alpha';\nimport { eventsServiceRef } from '@backstage/plugin-events-node';\nimport { merge } from 'lodash';\nimport { createCatalogActions } from '../actions';\nimport { ModelHolder } from '../model/ModelHolder';\nimport type { EntityProviderEntry } from '../processing/connectEntityProviders';\nimport { CatalogBuilder } from './CatalogBuilder';\n\nclass CatalogLocationsExtensionPointImpl\n  implements CatalogLocationsExtensionPoint\n{\n  #locationTypes: string[] | undefined;\n\n  setAllowedLocationTypes(locationTypes: Array<string>) {\n    this.#locationTypes = locationTypes;\n  }\n\n  get allowedLocationTypes() {\n    return this.#locationTypes;\n  }\n}\n\nclass CatalogModelExtensionPointImpl implements CatalogModelExtensionPoint {\n  #fieldValidators: Partial<Validators> = {};\n\n  setFieldValidators(validators: Partial<Validators>): void {\n    merge(this.#fieldValidators, validators);\n  }\n\n  get fieldValidators() {\n    return this.#fieldValidators;\n  }\n\n  #entityDataParser?: CatalogProcessorParser;\n\n  setEntityDataParser(parser: CatalogProcessorParser): void {\n    if (this.#entityDataParser) {\n      throw new Error(\n        'Attempted to install second EntityDataParser. Only one can be set.',\n      );\n    }\n    this.#entityDataParser = parser;\n  }\n\n  get entityDataParser() {\n    return this.#entityDataParser;\n  }\n\n  #modelSources: CatalogModelSource[] = [];\n\n  addModelSource(source: CatalogModelSource): void {\n    this.#modelSources.push(source);\n  }\n\n  get modelSources() {\n    return this.#modelSources;\n  }\n}\n\n/**\n * Catalog plugin\n * @public\n */\nexport const catalogPlugin = createBackendPlugin({\n  pluginId: 'catalog',\n  register(env) {\n    const processors = new Array<CatalogProcessor>();\n    const entityProviders = new Array<EntityProviderEntry>();\n    const placeholderResolvers: Record<string, PlaceholderResolver> = {};\n    let onProcessingError:\n      | ((event: {\n          unprocessedEntity: Entity;\n          errors: Error[];\n        }) => Promise<void> | void)\n      | undefined = undefined;\n\n    env.registerExtensionPoint({\n      extensionPoint: catalogProcessingExtensionPoint,\n      factory: context => ({\n        addProcessor: (...newProcessors) => {\n          processors.push(...newProcessors.flat());\n        },\n        addEntityProvider: (...providers) => {\n          entityProviders.push(\n            ...providers.flat().map(provider => ({\n              provider,\n              context,\n            })),\n          );\n        },\n        addPlaceholderResolver: (key, resolver) => {\n          if (key in placeholderResolvers) {\n            throw new Error(\n              `A placeholder resolver for '${key}' has already been set up, please check your config.`,\n            );\n          }\n          placeholderResolvers[key] = resolver;\n        },\n        setOnProcessingErrorHandler: handler => {\n          onProcessingError = handler;\n        },\n      }),\n    });\n\n    let locationAnalyzerFactory:\n      | ((options: {\n          scmLocationAnalyzers: ScmLocationAnalyzer[];\n        }) => Promise<{ locationAnalyzer: LocationAnalyzer }>)\n      | undefined = undefined;\n    const scmLocationAnalyzers = new Array<ScmLocationAnalyzer>();\n    env.registerExtensionPoint(catalogAnalysisExtensionPoint, {\n      setLocationAnalyzer(analyzerOrFactory) {\n        if (locationAnalyzerFactory) {\n          throw new Error('LocationAnalyzer has already been set');\n        }\n        if (typeof analyzerOrFactory === 'function') {\n          locationAnalyzerFactory = analyzerOrFactory;\n        } else {\n          locationAnalyzerFactory = async () => ({\n            locationAnalyzer: analyzerOrFactory,\n          });\n        }\n      },\n      addScmLocationAnalyzer(analyzer: ScmLocationAnalyzer) {\n        scmLocationAnalyzers.push(analyzer);\n      },\n    });\n\n    const modelExtensions = new CatalogModelExtensionPointImpl();\n    env.registerExtensionPoint(catalogModelExtensionPoint, modelExtensions);\n\n    const locationTypeExtensions = new CatalogLocationsExtensionPointImpl();\n    env.registerExtensionPoint(\n      catalogLocationsExtensionPoint,\n      locationTypeExtensions,\n    );\n\n    env.registerInit({\n      deps: {\n        logger: coreServices.logger,\n        config: coreServices.rootConfig,\n        reader: coreServices.urlReader,\n        permissions: coreServices.permissions,\n        permissionsRegistry: coreServices.permissionsRegistry,\n        database: coreServices.database,\n        httpRouter: coreServices.httpRouter,\n        lifecycle: coreServices.rootLifecycle,\n        scheduler: coreServices.scheduler,\n        auth: coreServices.auth,\n        httpAuth: coreServices.httpAuth,\n        auditor: coreServices.auditor,\n        events: eventsServiceRef,\n        catalog: catalogServiceRef,\n        actionsRegistry: actionsRegistryServiceRef,\n        catalogScmEvents: catalogScmEventsServiceRef,\n        metrics: metricsServiceRef,\n      },\n      async init({\n        logger,\n        config,\n        reader,\n        database,\n        permissions,\n        permissionsRegistry,\n        httpRouter,\n        lifecycle,\n        scheduler,\n        auth,\n        httpAuth,\n        catalog,\n        actionsRegistry,\n        auditor,\n        events,\n        catalogScmEvents,\n        metrics,\n      }) {\n        const modelHolder = modelExtensions.modelSources.length\n          ? await ModelHolder.create({\n              sources: modelExtensions.modelSources,\n              logger,\n              lifecycle,\n            })\n          : undefined;\n\n        const builder = await CatalogBuilder.create({\n          config,\n          modelHolder,\n          reader,\n          permissions,\n          permissionsRegistry,\n          database,\n          scheduler,\n          logger,\n          auth,\n          httpAuth,\n          auditor,\n          events,\n          catalogScmEvents,\n          metrics,\n        });\n\n        if (onProcessingError) {\n          builder.subscribe({ onProcessingError });\n        }\n        builder.addProcessor(...processors);\n        builder.addEntityProvider(...entityProviders);\n\n        if (modelExtensions.entityDataParser) {\n          builder.setEntityDataParser(modelExtensions.entityDataParser);\n        }\n\n        Object.entries(placeholderResolvers).forEach(([key, resolver]) =>\n          builder.setPlaceholderResolver(key, resolver),\n        );\n        if (locationAnalyzerFactory) {\n          const { locationAnalyzer } = await locationAnalyzerFactory({\n            scmLocationAnalyzers,\n          }).catch(e => {\n            throw new ForwardedError('Failed to create LocationAnalyzer', e);\n          });\n          builder.setLocationAnalyzer(locationAnalyzer);\n        } else {\n          builder.addLocationAnalyzers(...scmLocationAnalyzers);\n        }\n        builder.setFieldFormatValidators(modelExtensions.fieldValidators);\n\n        if (locationTypeExtensions.allowedLocationTypes) {\n          builder.setAllowedLocationTypes(\n            locationTypeExtensions.allowedLocationTypes,\n          );\n        }\n\n        const { processingEngine, router } = await builder.build();\n\n        if (config.getOptional('catalog.processingInterval') ?? true) {\n          lifecycle.addStartupHook(async () => {\n            await processingEngine.start();\n          });\n          lifecycle.addShutdownHook(() => processingEngine.stop());\n        }\n\n        httpRouter.use(router);\n\n        createCatalogActions({\n          catalog,\n          actionsRegistry,\n          modelHolder,\n        });\n\n        const scmEventsMessagesCounter = metrics.createCounter<{\n          eventType: string;\n        }>('catalog.events.scm.messages', {\n          description:\n            'Number of SCM event messages received by the catalog backend',\n          unit: 'short',\n        });\n        catalogScmEvents.subscribe({\n          onEvents: async e => {\n            for (const event of e) {\n              const eventType = event.type.split('.')[0];\n              scmEventsMessagesCounter.add(1, { eventType });\n            }\n          },\n        });\n      },\n    });\n  },\n});\n"],"names":["merge","createBackendPlugin","catalogProcessingExtensionPoint","catalogAnalysisExtensionPoint","catalogModelExtensionPoint","catalogLocationsExtensionPoint","coreServices","eventsServiceRef","catalogServiceRef","actionsRegistryServiceRef","catalogScmEventsServiceRef","metricsServiceRef","ModelHolder","CatalogBuilder","ForwardedError","createCatalogActions"],"mappings":";;;;;;;;;;;;;AAmDA,MAAM,kCAAA,CAEN;AAAA,EACE,cAAA;AAAA,EAEA,wBAAwB,aAAA,EAA8B;AACpD,IAAA,IAAA,CAAK,cAAA,GAAiB,aAAA;AAAA,EACxB;AAAA,EAEA,IAAI,oBAAA,GAAuB;AACzB,IAAA,OAAO,IAAA,CAAK,cAAA;AAAA,EACd;AACF;AAEA,MAAM,8BAAA,CAAqE;AAAA,EACzE,mBAAwC,EAAC;AAAA,EAEzC,mBAAmB,UAAA,EAAuC;AACxD,IAAAA,YAAA,CAAM,IAAA,CAAK,kBAAkB,UAAU,CAAA;AAAA,EACzC;AAAA,EAEA,IAAI,eAAA,GAAkB;AACpB,IAAA,OAAO,IAAA,CAAK,gBAAA;AAAA,EACd;AAAA,EAEA,iBAAA;AAAA,EAEA,oBAAoB,MAAA,EAAsC;AACxD,IAAA,IAAI,KAAK,iBAAA,EAAmB;AAC1B,MAAA,MAAM,IAAI,KAAA;AAAA,QACR;AAAA,OACF;AAAA,IACF;AACA,IAAA,IAAA,CAAK,iBAAA,GAAoB,MAAA;AAAA,EAC3B;AAAA,EAEA,IAAI,gBAAA,GAAmB;AACrB,IAAA,OAAO,IAAA,CAAK,iBAAA;AAAA,EACd;AAAA,EAEA,gBAAsC,EAAC;AAAA,EAEvC,eAAe,MAAA,EAAkC;AAC/C,IAAA,IAAA,CAAK,aAAA,CAAc,KAAK,MAAM,CAAA;AAAA,EAChC;AAAA,EAEA,IAAI,YAAA,GAAe;AACjB,IAAA,OAAO,IAAA,CAAK,aAAA;AAAA,EACd;AACF;AAMO,MAAM,gBAAgBC,oCAAA,CAAoB;AAAA,EAC/C,QAAA,EAAU,SAAA;AAAA,EACV,SAAS,GAAA,EAAK;AACZ,IAAA,MAAM,UAAA,GAAa,IAAI,KAAA,EAAwB;AAC/C,IAAA,MAAM,eAAA,GAAkB,IAAI,KAAA,EAA2B;AACvD,IAAA,MAAM,uBAA4D,EAAC;AACnE,IAAA,IAAI,iBAAA,GAKY,MAAA;AAEhB,IAAA,GAAA,CAAI,sBAAA,CAAuB;AAAA,MACzB,cAAA,EAAgBC,iDAAA;AAAA,MAChB,SAAS,CAAA,OAAA,MAAY;AAAA,QACnB,YAAA,EAAc,IAAI,aAAA,KAAkB;AAClC,UAAA,UAAA,CAAW,IAAA,CAAK,GAAG,aAAA,CAAc,IAAA,EAAM,CAAA;AAAA,QACzC,CAAA;AAAA,QACA,iBAAA,EAAmB,IAAI,SAAA,KAAc;AACnC,UAAA,eAAA,CAAgB,IAAA;AAAA,YACd,GAAG,SAAA,CAAU,IAAA,EAAK,CAAE,IAAI,CAAA,QAAA,MAAa;AAAA,cACnC,QAAA;AAAA,cACA;AAAA,aACF,CAAE;AAAA,WACJ;AAAA,QACF,CAAA;AAAA,QACA,sBAAA,EAAwB,CAAC,GAAA,EAAK,QAAA,KAAa;AACzC,UAAA,IAAI,OAAO,oBAAA,EAAsB;AAC/B,YAAA,MAAM,IAAI,KAAA;AAAA,cACR,+BAA+B,GAAG,CAAA,oDAAA;AAAA,aACpC;AAAA,UACF;AACA,UAAA,oBAAA,CAAqB,GAAG,CAAA,GAAI,QAAA;AAAA,QAC9B,CAAA;AAAA,QACA,6BAA6B,CAAA,OAAA,KAAW;AACtC,UAAA,iBAAA,GAAoB,OAAA;AAAA,QACtB;AAAA,OACF;AAAA,KACD,CAAA;AAED,IAAA,IAAI,uBAAA,GAIY,MAAA;AAChB,IAAA,MAAM,oBAAA,GAAuB,IAAI,KAAA,EAA2B;AAC5D,IAAA,GAAA,CAAI,uBAAuBC,+CAAA,EAA+B;AAAA,MACxD,oBAAoB,iBAAA,EAAmB;AACrC,QAAA,IAAI,uBAAA,EAAyB;AAC3B,UAAA,MAAM,IAAI,MAAM,uCAAuC,CAAA;AAAA,QACzD;AACA,QAAA,IAAI,OAAO,sBAAsB,UAAA,EAAY;AAC3C,UAAA,uBAAA,GAA0B,iBAAA;AAAA,QAC5B,CAAA,MAAO;AACL,UAAA,uBAAA,GAA0B,aAAa;AAAA,YACrC,gBAAA,EAAkB;AAAA,WACpB,CAAA;AAAA,QACF;AAAA,MACF,CAAA;AAAA,MACA,uBAAuB,QAAA,EAA+B;AACpD,QAAA,oBAAA,CAAqB,KAAK,QAAQ,CAAA;AAAA,MACpC;AAAA,KACD,CAAA;AAED,IAAA,MAAM,eAAA,GAAkB,IAAI,8BAAA,EAA+B;AAC3D,IAAA,GAAA,CAAI,sBAAA,CAAuBC,kCAA4B,eAAe,CAAA;AAEtE,IAAA,MAAM,sBAAA,GAAyB,IAAI,kCAAA,EAAmC;AACtE,IAAA,GAAA,CAAI,sBAAA;AAAA,MACFC,gDAAA;AAAA,MACA;AAAA,KACF;AAEA,IAAA,GAAA,CAAI,YAAA,CAAa;AAAA,MACf,IAAA,EAAM;AAAA,QACJ,QAAQC,6BAAA,CAAa,MAAA;AAAA,QACrB,QAAQA,6BAAA,CAAa,UAAA;AAAA,QACrB,QAAQA,6BAAA,CAAa,SAAA;AAAA,QACrB,aAAaA,6BAAA,CAAa,WAAA;AAAA,QAC1B,qBAAqBA,6BAAA,CAAa,mBAAA;AAAA,QAClC,UAAUA,6BAAA,CAAa,QAAA;AAAA,QACvB,YAAYA,6BAAA,CAAa,UAAA;AAAA,QACzB,WAAWA,6BAAA,CAAa,aAAA;AAAA,QACxB,WAAWA,6BAAA,CAAa,SAAA;AAAA,QACxB,MAAMA,6BAAA,CAAa,IAAA;AAAA,QACnB,UAAUA,6BAAA,CAAa,QAAA;AAAA,QACvB,SAASA,6BAAA,CAAa,OAAA;AAAA,QACtB,MAAA,EAAQC,iCAAA;AAAA,QACR,OAAA,EAASC,mCAAA;AAAA,QACT,eAAA,EAAiBC,iCAAA;AAAA,QACjB,gBAAA,EAAkBC,gCAAA;AAAA,QAClB,OAAA,EAASC;AAAA,OACX;AAAA,MACA,MAAM,IAAA,CAAK;AAAA,QACT,MAAA;AAAA,QACA,MAAA;AAAA,QACA,MAAA;AAAA,QACA,QAAA;AAAA,QACA,WAAA;AAAA,QACA,mBAAA;AAAA,QACA,UAAA;AAAA,QACA,SAAA;AAAA,QACA,SAAA;AAAA,QACA,IAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA,eAAA;AAAA,QACA,OAAA;AAAA,QACA,MAAA;AAAA,QACA,gBAAA;AAAA,QACA;AAAA,OACF,EAAG;AACD,QAAA,MAAM,cAAc,eAAA,CAAgB,YAAA,CAAa,MAAA,GAC7C,MAAMC,wBAAY,MAAA,CAAO;AAAA,UACvB,SAAS,eAAA,CAAgB,YAAA;AAAA,UACzB,MAAA;AAAA,UACA;AAAA,SACD,CAAA,GACD,MAAA;AAEJ,QAAA,MAAM,OAAA,GAAU,MAAMC,6BAAA,CAAe,MAAA,CAAO;AAAA,UAC1C,MAAA;AAAA,UACA,WAAA;AAAA,UACA,MAAA;AAAA,UACA,WAAA;AAAA,UACA,mBAAA;AAAA,UACA,QAAA;AAAA,UACA,SAAA;AAAA,UACA,MAAA;AAAA,UACA,IAAA;AAAA,UACA,QAAA;AAAA,UACA,OAAA;AAAA,UACA,MAAA;AAAA,UACA,gBAAA;AAAA,UACA;AAAA,SACD,CAAA;AAED,QAAA,IAAI,iBAAA,EAAmB;AACrB,UAAA,OAAA,CAAQ,SAAA,CAAU,EAAE,iBAAA,EAAmB,CAAA;AAAA,QACzC;AACA,QAAA,OAAA,CAAQ,YAAA,CAAa,GAAG,UAAU,CAAA;AAClC,QAAA,OAAA,CAAQ,iBAAA,CAAkB,GAAG,eAAe,CAAA;AAE5C,QAAA,IAAI,gBAAgB,gBAAA,EAAkB;AACpC,UAAA,OAAA,CAAQ,mBAAA,CAAoB,gBAAgB,gBAAgB,CAAA;AAAA,QAC9D;AAEA,QAAA,MAAA,CAAO,OAAA,CAAQ,oBAAoB,CAAA,CAAE,OAAA;AAAA,UAAQ,CAAC,CAAC,GAAA,EAAK,QAAQ,MAC1D,OAAA,CAAQ,sBAAA,CAAuB,KAAK,QAAQ;AAAA,SAC9C;AACA,QAAA,IAAI,uBAAA,EAAyB;AAC3B,UAAA,MAAM,EAAE,gBAAA,EAAiB,GAAI,MAAM,uBAAA,CAAwB;AAAA,YACzD;AAAA,WACD,CAAA,CAAE,KAAA,CAAM,CAAA,CAAA,KAAK;AACZ,YAAA,MAAM,IAAIC,qBAAA,CAAe,mCAAA,EAAqC,CAAC,CAAA;AAAA,UACjE,CAAC,CAAA;AACD,UAAA,OAAA,CAAQ,oBAAoB,gBAAgB,CAAA;AAAA,QAC9C,CAAA,MAAO;AACL,UAAA,OAAA,CAAQ,oBAAA,CAAqB,GAAG,oBAAoB,CAAA;AAAA,QACtD;AACA,QAAA,OAAA,CAAQ,wBAAA,CAAyB,gBAAgB,eAAe,CAAA;AAEhE,QAAA,IAAI,uBAAuB,oBAAA,EAAsB;AAC/C,UAAA,OAAA,CAAQ,uBAAA;AAAA,YACN,sBAAA,CAAuB;AAAA,WACzB;AAAA,QACF;AAEA,QAAA,MAAM,EAAE,gBAAA,EAAkB,MAAA,EAAO,GAAI,MAAM,QAAQ,KAAA,EAAM;AAEzD,QAAA,IAAI,MAAA,CAAO,WAAA,CAAY,4BAA4B,CAAA,IAAK,IAAA,EAAM;AAC5D,UAAA,SAAA,CAAU,eAAe,YAAY;AACnC,YAAA,MAAM,iBAAiB,KAAA,EAAM;AAAA,UAC/B,CAAC,CAAA;AACD,UAAA,SAAA,CAAU,eAAA,CAAgB,MAAM,gBAAA,CAAiB,IAAA,EAAM,CAAA;AAAA,QACzD;AAEA,QAAA,UAAA,CAAW,IAAI,MAAM,CAAA;AAErB,QAAAC,0BAAA,CAAqB;AAAA,UACnB,OAAA;AAAA,UACA,eAAA;AAAA,UACA;AAAA,SACD,CAAA;AAED,QAAA,MAAM,wBAAA,GAA2B,OAAA,CAAQ,aAAA,CAEtC,6BAAA,EAA+B;AAAA,UAChC,WAAA,EACE,8DAAA;AAAA,UACF,IAAA,EAAM;AAAA,SACP,CAAA;AACD,QAAA,gBAAA,CAAiB,SAAA,CAAU;AAAA,UACzB,QAAA,EAAU,OAAM,CAAA,KAAK;AACnB,YAAA,KAAA,MAAW,SAAS,CAAA,EAAG;AACrB,cAAA,MAAM,YAAY,KAAA,CAAM,IAAA,CAAK,KAAA,CAAM,GAAG,EAAE,CAAC,CAAA;AACzC,cAAA,wBAAA,CAAyB,GAAA,CAAI,CAAA,EAAG,EAAE,SAAA,EAAW,CAAA;AAAA,YAC/C;AAAA,UACF;AAAA,SACD,CAAA;AAAA,MACH;AAAA,KACD,CAAA;AAAA,EACH;AACF,CAAC;;;;"}
+{"version":3,"file":"CatalogPlugin.cjs.js","sources":["../../src/service/CatalogPlugin.ts"],"sourcesContent":["/*\n * Copyright 2022 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport {\n  coreServices,\n  createBackendPlugin,\n} from '@backstage/backend-plugin-api';\nimport {\n  actionsRegistryServiceRef,\n  metricsServiceRef,\n} from '@backstage/backend-plugin-api/alpha';\nimport { Entity, Validators } from '@backstage/catalog-model';\nimport { CatalogModelSource } from '@backstage/catalog-model/alpha';\nimport { ForwardedError } from '@backstage/errors';\nimport {\n  catalogAnalysisExtensionPoint,\n  CatalogLocationsExtensionPoint,\n  catalogLocationsExtensionPoint,\n  catalogProcessingExtensionPoint,\n  CatalogProcessor,\n  CatalogProcessorParser,\n  catalogServiceRef,\n  LocationAnalyzer,\n  PlaceholderResolver,\n  ScmLocationAnalyzer,\n} from '@backstage/plugin-catalog-node';\nimport {\n  CatalogModelExtensionPoint,\n  catalogModelExtensionPoint,\n  catalogScmEventsServiceRef,\n} from '@backstage/plugin-catalog-node/alpha';\nimport { eventsServiceRef } from '@backstage/plugin-events-node';\nimport { merge } from 'lodash';\nimport { createCatalogActions } from '../actions';\nimport { ModelHolder } from '../model/ModelHolder';\nimport type { EntityProviderEntry } from '../processing/connectEntityProviders';\nimport { CatalogBuilder } from './CatalogBuilder';\n\nclass CatalogLocationsExtensionPointImpl\n  implements CatalogLocationsExtensionPoint\n{\n  #locationTypes: string[] | undefined;\n\n  setAllowedLocationTypes(locationTypes: Array<string>) {\n    this.#locationTypes = locationTypes;\n  }\n\n  get allowedLocationTypes() {\n    return this.#locationTypes;\n  }\n}\n\nclass CatalogModelExtensionPointImpl implements CatalogModelExtensionPoint {\n  #fieldValidators: Partial<Validators> = {};\n\n  setFieldValidators(validators: Partial<Validators>): void {\n    merge(this.#fieldValidators, validators);\n  }\n\n  get fieldValidators() {\n    return this.#fieldValidators;\n  }\n\n  #entityDataParser?: CatalogProcessorParser;\n\n  setEntityDataParser(parser: CatalogProcessorParser): void {\n    if (this.#entityDataParser) {\n      throw new Error(\n        'Attempted to install second EntityDataParser. Only one can be set.',\n      );\n    }\n    this.#entityDataParser = parser;\n  }\n\n  get entityDataParser() {\n    return this.#entityDataParser;\n  }\n\n  #modelSources: CatalogModelSource[] = [];\n\n  addModelSource(source: CatalogModelSource): void {\n    this.#modelSources.push(source);\n  }\n\n  get modelSources() {\n    return this.#modelSources;\n  }\n}\n\n/**\n * Catalog plugin\n * @public\n */\nexport const catalogPlugin = createBackendPlugin({\n  pluginId: 'catalog',\n  register(env) {\n    const processors = new Array<CatalogProcessor>();\n    const entityProviders = new Array<EntityProviderEntry>();\n    const placeholderResolvers: Record<string, PlaceholderResolver> = {};\n    let onProcessingError:\n      | ((event: {\n          unprocessedEntity: Entity;\n          errors: Error[];\n        }) => Promise<void> | void)\n      | undefined = undefined;\n\n    env.registerExtensionPoint({\n      extensionPoint: catalogProcessingExtensionPoint,\n      factory: context => ({\n        addProcessor: (...newProcessors) => {\n          processors.push(...newProcessors.flat());\n        },\n        addEntityProvider: (...providers) => {\n          entityProviders.push(\n            ...providers.flat().map(provider => ({\n              provider,\n              context,\n            })),\n          );\n        },\n        addPlaceholderResolver: (key, resolver) => {\n          if (key in placeholderResolvers) {\n            throw new Error(\n              `A placeholder resolver for '${key}' has already been set up, please check your config.`,\n            );\n          }\n          placeholderResolvers[key] = resolver;\n        },\n        setOnProcessingErrorHandler: handler => {\n          onProcessingError = handler;\n        },\n      }),\n    });\n\n    let locationAnalyzerFactory:\n      | ((options: {\n          scmLocationAnalyzers: ScmLocationAnalyzer[];\n        }) => Promise<{ locationAnalyzer: LocationAnalyzer }>)\n      | undefined = undefined;\n    const scmLocationAnalyzers = new Array<ScmLocationAnalyzer>();\n    env.registerExtensionPoint(catalogAnalysisExtensionPoint, {\n      setLocationAnalyzer(analyzerOrFactory) {\n        if (locationAnalyzerFactory) {\n          throw new Error('LocationAnalyzer has already been set');\n        }\n        if (typeof analyzerOrFactory === 'function') {\n          locationAnalyzerFactory = analyzerOrFactory;\n        } else {\n          locationAnalyzerFactory = async () => ({\n            locationAnalyzer: analyzerOrFactory,\n          });\n        }\n      },\n      addScmLocationAnalyzer(analyzer: ScmLocationAnalyzer) {\n        scmLocationAnalyzers.push(analyzer);\n      },\n    });\n\n    const modelExtensions = new CatalogModelExtensionPointImpl();\n    env.registerExtensionPoint(catalogModelExtensionPoint, modelExtensions);\n\n    const locationTypeExtensions = new CatalogLocationsExtensionPointImpl();\n    env.registerExtensionPoint(\n      catalogLocationsExtensionPoint,\n      locationTypeExtensions,\n    );\n\n    env.registerInit({\n      deps: {\n        logger: coreServices.logger,\n        config: coreServices.rootConfig,\n        reader: coreServices.urlReader,\n        permissions: coreServices.permissions,\n        permissionsRegistry: coreServices.permissionsRegistry,\n        database: coreServices.database,\n        httpRouter: coreServices.httpRouter,\n        lifecycle: coreServices.rootLifecycle,\n        scheduler: coreServices.scheduler,\n        auth: coreServices.auth,\n        httpAuth: coreServices.httpAuth,\n        auditor: coreServices.auditor,\n        events: eventsServiceRef,\n        catalog: catalogServiceRef,\n        actionsRegistry: actionsRegistryServiceRef,\n        catalogScmEvents: catalogScmEventsServiceRef,\n        metrics: metricsServiceRef,\n      },\n      async init({\n        logger,\n        config,\n        reader,\n        database,\n        permissions,\n        permissionsRegistry,\n        httpRouter,\n        lifecycle,\n        scheduler,\n        auth,\n        httpAuth,\n        catalog,\n        actionsRegistry,\n        auditor,\n        events,\n        catalogScmEvents,\n        metrics,\n      }) {\n        const modelHolder = modelExtensions.modelSources.length\n          ? await ModelHolder.create({\n              sources: modelExtensions.modelSources,\n              logger,\n              lifecycle,\n            })\n          : undefined;\n\n        const builder = await CatalogBuilder.create({\n          config,\n          modelHolder,\n          reader,\n          permissions,\n          permissionsRegistry,\n          database,\n          scheduler,\n          logger,\n          auth,\n          httpAuth,\n          auditor,\n          events,\n          catalogScmEvents,\n          metrics,\n        });\n\n        if (onProcessingError) {\n          builder.subscribe({ onProcessingError });\n        }\n        builder.addProcessor(...processors);\n        builder.addEntityProvider(...entityProviders);\n\n        if (modelExtensions.entityDataParser) {\n          builder.setEntityDataParser(modelExtensions.entityDataParser);\n        }\n\n        Object.entries(placeholderResolvers).forEach(([key, resolver]) =>\n          builder.setPlaceholderResolver(key, resolver),\n        );\n        if (locationAnalyzerFactory) {\n          const { locationAnalyzer } = await locationAnalyzerFactory({\n            scmLocationAnalyzers,\n          }).catch(e => {\n            throw new ForwardedError('Failed to create LocationAnalyzer', e);\n          });\n          builder.setLocationAnalyzer(locationAnalyzer);\n        } else {\n          builder.addLocationAnalyzers(...scmLocationAnalyzers);\n        }\n        builder.setFieldFormatValidators(modelExtensions.fieldValidators);\n\n        if (locationTypeExtensions.allowedLocationTypes) {\n          builder.setAllowedLocationTypes(\n            locationTypeExtensions.allowedLocationTypes,\n          );\n        }\n\n        const { processingEngine, router } = await builder.build();\n\n        if (config.getOptional('catalog.processingInterval') ?? true) {\n          lifecycle.addStartupHook(async () => {\n            await processingEngine.start();\n          });\n          lifecycle.addShutdownHook(() => processingEngine.stop());\n        }\n\n        httpRouter.use(router);\n\n        createCatalogActions({\n          catalog,\n          actionsRegistry,\n          modelHolder,\n          useExperimentalCatalogLayersDescriptions:\n            config.getOptionalBoolean(\n              'catalog.actions.experimentalCatalogLayersDescriptions.enabled',\n            ) ?? false,\n        });\n\n        const scmEventsMessagesCounter = metrics.createCounter<{\n          eventType: string;\n        }>('catalog.events.scm.messages', {\n          description:\n            'Number of SCM event messages received by the catalog backend',\n          unit: 'short',\n        });\n        catalogScmEvents.subscribe({\n          onEvents: async e => {\n            for (const event of e) {\n              const eventType = event.type.split('.')[0];\n              scmEventsMessagesCounter.add(1, { eventType });\n            }\n          },\n        });\n      },\n    });\n  },\n});\n"],"names":["merge","createBackendPlugin","catalogProcessingExtensionPoint","catalogAnalysisExtensionPoint","catalogModelExtensionPoint","catalogLocationsExtensionPoint","coreServices","eventsServiceRef","catalogServiceRef","actionsRegistryServiceRef","catalogScmEventsServiceRef","metricsServiceRef","ModelHolder","CatalogBuilder","ForwardedError","createCatalogActions"],"mappings":";;;;;;;;;;;;;AAmDA,MAAM,kCAAA,CAEN;AAAA,EACE,cAAA;AAAA,EAEA,wBAAwB,aAAA,EAA8B;AACpD,IAAA,IAAA,CAAK,cAAA,GAAiB,aAAA;AAAA,EACxB;AAAA,EAEA,IAAI,oBAAA,GAAuB;AACzB,IAAA,OAAO,IAAA,CAAK,cAAA;AAAA,EACd;AACF;AAEA,MAAM,8BAAA,CAAqE;AAAA,EACzE,mBAAwC,EAAC;AAAA,EAEzC,mBAAmB,UAAA,EAAuC;AACxD,IAAAA,YAAA,CAAM,IAAA,CAAK,kBAAkB,UAAU,CAAA;AAAA,EACzC;AAAA,EAEA,IAAI,eAAA,GAAkB;AACpB,IAAA,OAAO,IAAA,CAAK,gBAAA;AAAA,EACd;AAAA,EAEA,iBAAA;AAAA,EAEA,oBAAoB,MAAA,EAAsC;AACxD,IAAA,IAAI,KAAK,iBAAA,EAAmB;AAC1B,MAAA,MAAM,IAAI,KAAA;AAAA,QACR;AAAA,OACF;AAAA,IACF;AACA,IAAA,IAAA,CAAK,iBAAA,GAAoB,MAAA;AAAA,EAC3B;AAAA,EAEA,IAAI,gBAAA,GAAmB;AACrB,IAAA,OAAO,IAAA,CAAK,iBAAA;AAAA,EACd;AAAA,EAEA,gBAAsC,EAAC;AAAA,EAEvC,eAAe,MAAA,EAAkC;AAC/C,IAAA,IAAA,CAAK,aAAA,CAAc,KAAK,MAAM,CAAA;AAAA,EAChC;AAAA,EAEA,IAAI,YAAA,GAAe;AACjB,IAAA,OAAO,IAAA,CAAK,aAAA;AAAA,EACd;AACF;AAMO,MAAM,gBAAgBC,oCAAA,CAAoB;AAAA,EAC/C,QAAA,EAAU,SAAA;AAAA,EACV,SAAS,GAAA,EAAK;AACZ,IAAA,MAAM,UAAA,GAAa,IAAI,KAAA,EAAwB;AAC/C,IAAA,MAAM,eAAA,GAAkB,IAAI,KAAA,EAA2B;AACvD,IAAA,MAAM,uBAA4D,EAAC;AACnE,IAAA,IAAI,iBAAA,GAKY,MAAA;AAEhB,IAAA,GAAA,CAAI,sBAAA,CAAuB;AAAA,MACzB,cAAA,EAAgBC,iDAAA;AAAA,MAChB,SAAS,CAAA,OAAA,MAAY;AAAA,QACnB,YAAA,EAAc,IAAI,aAAA,KAAkB;AAClC,UAAA,UAAA,CAAW,IAAA,CAAK,GAAG,aAAA,CAAc,IAAA,EAAM,CAAA;AAAA,QACzC,CAAA;AAAA,QACA,iBAAA,EAAmB,IAAI,SAAA,KAAc;AACnC,UAAA,eAAA,CAAgB,IAAA;AAAA,YACd,GAAG,SAAA,CAAU,IAAA,EAAK,CAAE,IAAI,CAAA,QAAA,MAAa;AAAA,cACnC,QAAA;AAAA,cACA;AAAA,aACF,CAAE;AAAA,WACJ;AAAA,QACF,CAAA;AAAA,QACA,sBAAA,EAAwB,CAAC,GAAA,EAAK,QAAA,KAAa;AACzC,UAAA,IAAI,OAAO,oBAAA,EAAsB;AAC/B,YAAA,MAAM,IAAI,KAAA;AAAA,cACR,+BAA+B,GAAG,CAAA,oDAAA;AAAA,aACpC;AAAA,UACF;AACA,UAAA,oBAAA,CAAqB,GAAG,CAAA,GAAI,QAAA;AAAA,QAC9B,CAAA;AAAA,QACA,6BAA6B,CAAA,OAAA,KAAW;AACtC,UAAA,iBAAA,GAAoB,OAAA;AAAA,QACtB;AAAA,OACF;AAAA,KACD,CAAA;AAED,IAAA,IAAI,uBAAA,GAIY,MAAA;AAChB,IAAA,MAAM,oBAAA,GAAuB,IAAI,KAAA,EAA2B;AAC5D,IAAA,GAAA,CAAI,uBAAuBC,+CAAA,EAA+B;AAAA,MACxD,oBAAoB,iBAAA,EAAmB;AACrC,QAAA,IAAI,uBAAA,EAAyB;AAC3B,UAAA,MAAM,IAAI,MAAM,uCAAuC,CAAA;AAAA,QACzD;AACA,QAAA,IAAI,OAAO,sBAAsB,UAAA,EAAY;AAC3C,UAAA,uBAAA,GAA0B,iBAAA;AAAA,QAC5B,CAAA,MAAO;AACL,UAAA,uBAAA,GAA0B,aAAa;AAAA,YACrC,gBAAA,EAAkB;AAAA,WACpB,CAAA;AAAA,QACF;AAAA,MACF,CAAA;AAAA,MACA,uBAAuB,QAAA,EAA+B;AACpD,QAAA,oBAAA,CAAqB,KAAK,QAAQ,CAAA;AAAA,MACpC;AAAA,KACD,CAAA;AAED,IAAA,MAAM,eAAA,GAAkB,IAAI,8BAAA,EAA+B;AAC3D,IAAA,GAAA,CAAI,sBAAA,CAAuBC,kCAA4B,eAAe,CAAA;AAEtE,IAAA,MAAM,sBAAA,GAAyB,IAAI,kCAAA,EAAmC;AACtE,IAAA,GAAA,CAAI,sBAAA;AAAA,MACFC,gDAAA;AAAA,MACA;AAAA,KACF;AAEA,IAAA,GAAA,CAAI,YAAA,CAAa;AAAA,MACf,IAAA,EAAM;AAAA,QACJ,QAAQC,6BAAA,CAAa,MAAA;AAAA,QACrB,QAAQA,6BAAA,CAAa,UAAA;AAAA,QACrB,QAAQA,6BAAA,CAAa,SAAA;AAAA,QACrB,aAAaA,6BAAA,CAAa,WAAA;AAAA,QAC1B,qBAAqBA,6BAAA,CAAa,mBAAA;AAAA,QAClC,UAAUA,6BAAA,CAAa,QAAA;AAAA,QACvB,YAAYA,6BAAA,CAAa,UAAA;AAAA,QACzB,WAAWA,6BAAA,CAAa,aAAA;AAAA,QACxB,WAAWA,6BAAA,CAAa,SAAA;AAAA,QACxB,MAAMA,6BAAA,CAAa,IAAA;AAAA,QACnB,UAAUA,6BAAA,CAAa,QAAA;AAAA,QACvB,SAASA,6BAAA,CAAa,OAAA;AAAA,QACtB,MAAA,EAAQC,iCAAA;AAAA,QACR,OAAA,EAASC,mCAAA;AAAA,QACT,eAAA,EAAiBC,iCAAA;AAAA,QACjB,gBAAA,EAAkBC,gCAAA;AAAA,QAClB,OAAA,EAASC;AAAA,OACX;AAAA,MACA,MAAM,IAAA,CAAK;AAAA,QACT,MAAA;AAAA,QACA,MAAA;AAAA,QACA,MAAA;AAAA,QACA,QAAA;AAAA,QACA,WAAA;AAAA,QACA,mBAAA;AAAA,QACA,UAAA;AAAA,QACA,SAAA;AAAA,QACA,SAAA;AAAA,QACA,IAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA,eAAA;AAAA,QACA,OAAA;AAAA,QACA,MAAA;AAAA,QACA,gBAAA;AAAA,QACA;AAAA,OACF,EAAG;AACD,QAAA,MAAM,cAAc,eAAA,CAAgB,YAAA,CAAa,MAAA,GAC7C,MAAMC,wBAAY,MAAA,CAAO;AAAA,UACvB,SAAS,eAAA,CAAgB,YAAA;AAAA,UACzB,MAAA;AAAA,UACA;AAAA,SACD,CAAA,GACD,MAAA;AAEJ,QAAA,MAAM,OAAA,GAAU,MAAMC,6BAAA,CAAe,MAAA,CAAO;AAAA,UAC1C,MAAA;AAAA,UACA,WAAA;AAAA,UACA,MAAA;AAAA,UACA,WAAA;AAAA,UACA,mBAAA;AAAA,UACA,QAAA;AAAA,UACA,SAAA;AAAA,UACA,MAAA;AAAA,UACA,IAAA;AAAA,UACA,QAAA;AAAA,UACA,OAAA;AAAA,UACA,MAAA;AAAA,UACA,gBAAA;AAAA,UACA;AAAA,SACD,CAAA;AAED,QAAA,IAAI,iBAAA,EAAmB;AACrB,UAAA,OAAA,CAAQ,SAAA,CAAU,EAAE,iBAAA,EAAmB,CAAA;AAAA,QACzC;AACA,QAAA,OAAA,CAAQ,YAAA,CAAa,GAAG,UAAU,CAAA;AAClC,QAAA,OAAA,CAAQ,iBAAA,CAAkB,GAAG,eAAe,CAAA;AAE5C,QAAA,IAAI,gBAAgB,gBAAA,EAAkB;AACpC,UAAA,OAAA,CAAQ,mBAAA,CAAoB,gBAAgB,gBAAgB,CAAA;AAAA,QAC9D;AAEA,QAAA,MAAA,CAAO,OAAA,CAAQ,oBAAoB,CAAA,CAAE,OAAA;AAAA,UAAQ,CAAC,CAAC,GAAA,EAAK,QAAQ,MAC1D,OAAA,CAAQ,sBAAA,CAAuB,KAAK,QAAQ;AAAA,SAC9C;AACA,QAAA,IAAI,uBAAA,EAAyB;AAC3B,UAAA,MAAM,EAAE,gBAAA,EAAiB,GAAI,MAAM,uBAAA,CAAwB;AAAA,YACzD;AAAA,WACD,CAAA,CAAE,KAAA,CAAM,CAAA,CAAA,KAAK;AACZ,YAAA,MAAM,IAAIC,qBAAA,CAAe,mCAAA,EAAqC,CAAC,CAAA;AAAA,UACjE,CAAC,CAAA;AACD,UAAA,OAAA,CAAQ,oBAAoB,gBAAgB,CAAA;AAAA,QAC9C,CAAA,MAAO;AACL,UAAA,OAAA,CAAQ,oBAAA,CAAqB,GAAG,oBAAoB,CAAA;AAAA,QACtD;AACA,QAAA,OAAA,CAAQ,wBAAA,CAAyB,gBAAgB,eAAe,CAAA;AAEhE,QAAA,IAAI,uBAAuB,oBAAA,EAAsB;AAC/C,UAAA,OAAA,CAAQ,uBAAA;AAAA,YACN,sBAAA,CAAuB;AAAA,WACzB;AAAA,QACF;AAEA,QAAA,MAAM,EAAE,gBAAA,EAAkB,MAAA,EAAO,GAAI,MAAM,QAAQ,KAAA,EAAM;AAEzD,QAAA,IAAI,MAAA,CAAO,WAAA,CAAY,4BAA4B,CAAA,IAAK,IAAA,EAAM;AAC5D,UAAA,SAAA,CAAU,eAAe,YAAY;AACnC,YAAA,MAAM,iBAAiB,KAAA,EAAM;AAAA,UAC/B,CAAC,CAAA;AACD,UAAA,SAAA,CAAU,eAAA,CAAgB,MAAM,gBAAA,CAAiB,IAAA,EAAM,CAAA;AAAA,QACzD;AAEA,QAAA,UAAA,CAAW,IAAI,MAAM,CAAA;AAErB,QAAAC,0BAAA,CAAqB;AAAA,UACnB,OAAA;AAAA,UACA,eAAA;AAAA,UACA,WAAA;AAAA,UACA,0CACE,MAAA,CAAO,kBAAA;AAAA,YACL;AAAA,WACF,IAAK;AAAA,SACR,CAAA;AAED,QAAA,MAAM,wBAAA,GAA2B,OAAA,CAAQ,aAAA,CAEtC,6BAAA,EAA+B;AAAA,UAChC,WAAA,EACE,8DAAA;AAAA,UACF,IAAA,EAAM;AAAA,SACP,CAAA;AACD,QAAA,gBAAA,CAAiB,SAAA,CAAU;AAAA,UACzB,QAAA,EAAU,OAAM,CAAA,KAAK;AACnB,YAAA,KAAA,MAAW,SAAS,CAAA,EAAG;AACrB,cAAA,MAAM,YAAY,KAAA,CAAM,IAAA,CAAK,KAAA,CAAM,GAAG,EAAE,CAAC,CAAA;AACzC,cAAA,wBAAA,CAAyB,GAAA,CAAI,CAAA,EAAG,EAAE,SAAA,EAAW,CAAA;AAAA,YAC/C;AAAA,UACF;AAAA,SACD,CAAA;AAAA,MACH;AAAA,KACD,CAAA;AAAA,EACH;AACF,CAAC;;;;"}

package/dist/stitching/DefaultStitcher.cjs.js

@@ -24,7 +24,9 @@ class DefaultStitcher {
       knex: options.knex,
       logger: options.logger,
       metrics: options.metrics,
-      strategy: types.stitchingStrategyFromConfig(config)
+      strategy: types.stitchingStrategyFromConfig(config, {
+        logger: options.logger
+      })
     });
   }
   constructor(options) {

package/dist/stitching/DefaultStitcher.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"DefaultStitcher.cjs.js","sources":["../../src/stitching/DefaultStitcher.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Config } from '@backstage/config';\nimport { durationToMilliseconds, HumanDuration } from '@backstage/types';\nimport { Knex } from 'knex';\nimport splitToChunks from 'lodash/chunk';\nimport { DateTime } from 'luxon';\nimport { getDeferredStitchableEntities } from '../database/operations/stitcher/getDeferredStitchableEntities';\nimport { markForStitching } from '../database/operations/stitcher/markForStitching';\nimport { performStitching } from '../database/operations/stitcher/performStitching';\nimport { DbRefreshStateRow } from '../database/tables';\nimport { startTaskPipeline } from '../processing/TaskPipeline';\nimport { progressTracker } from './progressTracker';\nimport {\n  Stitcher,\n  StitchingStrategy,\n  stitchingStrategyFromConfig,\n} from './types';\nimport { LoggerService } from '@backstage/backend-plugin-api';\nimport { MetricsService } from '@backstage/backend-plugin-api/alpha';\n\ntype DeferredStitchItem = Awaited<\n  ReturnType<typeof getDeferredStitchableEntities>\n>[0];\n\ntype StitchProgressTracker = ReturnType<typeof progressTracker>;\n\n/**\n * Performs the act of stitching - to take all of the various outputs from the\n * ingestion process, and stitching them together into the final entity JSON\n * shape.\n */\nexport class DefaultStitcher implements Stitcher {\n  private readonly knex: Knex;\n  private readonly logger: LoggerService;\n  private readonly strategy: StitchingStrategy;\n  private readonly tracker: StitchProgressTracker;\n  private stopFunc?: () => void;\n\n  static fromConfig(\n    config: Config,\n    options: {\n      knex: Knex;\n      logger: LoggerService;\n      metrics: MetricsService;\n    },\n  ): DefaultStitcher {\n    return new DefaultStitcher({\n      knex: options.knex,\n      logger: options.logger,\n      metrics: options.metrics,\n      strategy: stitchingStrategyFromConfig(config),\n    });\n  }\n\n  constructor(options: {\n    knex: Knex;\n    logger: LoggerService;\n    metrics: MetricsService;\n    strategy: StitchingStrategy;\n  }) {\n    this.knex = options.knex;\n    this.logger = options.logger;\n    this.strategy = options.strategy;\n    this.tracker = progressTracker(\n      options.knex,\n      options.logger,\n      options.metrics,\n    );\n  }\n\n  async stitch(options: {\n    entityRefs?: Iterable<string>;\n    entityIds?: Iterable<string>;\n  }) {\n    const { entityRefs, entityIds } = options;\n\n    if (this.strategy.mode === 'deferred') {\n      await markForStitching({\n        knex: this.knex,\n        strategy: this.strategy,\n        entityRefs,\n        entityIds,\n      });\n      return;\n    }\n\n    if (entityRefs) {\n      for (const entityRef of entityRefs) {\n        await this.#stitchOne({ entityRef });\n      }\n    }\n\n    if (entityIds) {\n      const chunks = splitToChunks(\n        Array.isArray(entityIds) ? entityIds : [...entityIds],\n        100,\n      );\n      for (const chunk of chunks) {\n        const rows = await this.knex<DbRefreshStateRow>('refresh_state')\n          .select('entity_ref')\n          .whereIn('entity_id', chunk);\n        for (const row of rows) {\n          await this.#stitchOne({ entityRef: row.entity_ref });\n        }\n      }\n    }\n  }\n\n  async start() {\n    if (this.strategy.mode === 'deferred') {\n      if (this.stopFunc) {\n        throw new Error('Processing engine is already started');\n      }\n\n      const { pollingInterval, stitchTimeout } = this.strategy;\n\n      const stopPipeline = startTaskPipeline<DeferredStitchItem>({\n        lowWatermark: 2,\n        highWatermark: 5,\n        pollingIntervalMs: durationToMilliseconds(pollingInterval),\n        loadTasks: async count => {\n          return await this.#getStitchableEntities(count, stitchTimeout);\n        },\n        processTask: async item => {\n          return await this.#stitchOne({\n            entityRef: item.entityRef,\n            stitchTicket: item.stitchTicket,\n            stitchRequestedAt: item.stitchRequestedAt,\n          });\n        },\n      });\n\n      this.stopFunc = () => {\n        stopPipeline();\n      };\n    }\n  }\n\n  async stop() {\n    if (this.strategy.mode === 'deferred') {\n      if (this.stopFunc) {\n        this.stopFunc();\n        this.stopFunc = undefined;\n      }\n    }\n  }\n\n  async #getStitchableEntities(count: number, stitchTimeout: HumanDuration) {\n    try {\n      return await getDeferredStitchableEntities({\n        knex: this.knex,\n        batchSize: count,\n        stitchTimeout: stitchTimeout,\n      });\n    } catch (error) {\n      this.logger.warn('Failed to load stitchable entities', error);\n      return [];\n    }\n  }\n\n  async #stitchOne(options: {\n    entityRef: string;\n    stitchTicket?: string;\n    stitchRequestedAt?: DateTime;\n  }) {\n    const track = this.tracker.stitchStart({\n      entityRef: options.entityRef,\n      stitchRequestedAt: options.stitchRequestedAt,\n    });\n\n    try {\n      const result = await performStitching({\n        knex: this.knex,\n        logger: this.logger,\n        strategy: this.strategy,\n        entityRef: options.entityRef,\n        stitchTicket: options.stitchTicket,\n      });\n      track.markComplete(result);\n    } catch (error) {\n      track.markFailed(error);\n    }\n  }\n}\n"],"names":["stitchingStrategyFromConfig","progressTracker","markForStitching","splitToChunks","startTaskPipeline","durationToMilliseconds","getDeferredStitchableEntities","performStitching"],"mappings":";;;;;;;;;;;;;;;AA8CO,MAAM,eAAA,CAAoC;AAAA,EAC9B,IAAA;AAAA,EACA,MAAA;AAAA,EACA,QAAA;AAAA,EACA,OAAA;AAAA,EACT,QAAA;AAAA,EAER,OAAO,UAAA,CACL,MAAA,EACA,OAAA,EAKiB;AACjB,IAAA,OAAO,IAAI,eAAA,CAAgB;AAAA,MACzB,MAAM,OAAA,CAAQ,IAAA;AAAA,MACd,QAAQ,OAAA,CAAQ,MAAA;AAAA,MAChB,SAAS,OAAA,CAAQ,OAAA;AAAA,MACjB,QAAA,EAAUA,kCAA4B,MAAM;AAAA,KAC7C,CAAA;AAAA,EACH;AAAA,EAEA,YAAY,OAAA,EAKT;AACD,IAAA,IAAA,CAAK,OAAO,OAAA,CAAQ,IAAA;AACpB,IAAA,IAAA,CAAK,SAAS,OAAA,CAAQ,MAAA;AACtB,IAAA,IAAA,CAAK,WAAW,OAAA,CAAQ,QAAA;AACxB,IAAA,IAAA,CAAK,OAAA,GAAUC,+BAAA;AAAA,MACb,OAAA,CAAQ,IAAA;AAAA,MACR,OAAA,CAAQ,MAAA;AAAA,MACR,OAAA,CAAQ;AAAA,KACV;AAAA,EACF;AAAA,EAEA,MAAM,OAAO,OAAA,EAGV;AACD,IAAA,MAAM,EAAE,UAAA,EAAY,SAAA,EAAU,GAAI,OAAA;AAElC,IAAA,IAAI,IAAA,CAAK,QAAA,CAAS,IAAA,KAAS,UAAA,EAAY;AACrC,MAAA,MAAMC,iCAAA,CAAiB;AAAA,QACrB,MAAM,IAAA,CAAK,IAAA;AAAA,QACX,UAAU,IAAA,CAAK,QAAA;AAAA,QACf,UAAA;AAAA,QACA;AAAA,OACD,CAAA;AACD,MAAA;AAAA,IACF;AAEA,IAAA,IAAI,UAAA,EAAY;AACd,MAAA,KAAA,MAAW,aAAa,UAAA,EAAY;AAClC,QAAA,MAAM,IAAA,CAAK,UAAA,CAAW,EAAE,SAAA,EAAW,CAAA;AAAA,MACrC;AAAA,IACF;AAEA,IAAA,IAAI,SAAA,EAAW;AACb,MAAA,MAAM,MAAA,GAASC,8BAAA;AAAA,QACb,MAAM,OAAA,CAAQ,SAAS,IAAI,SAAA,GAAY,CAAC,GAAG,SAAS,CAAA;AAAA,QACpD;AAAA,OACF;AACA,MAAA,KAAA,MAAW,SAAS,MAAA,EAAQ;AAC1B,QAAA,MAAM,IAAA,GAAO,MAAM,IAAA,CAAK,IAAA,CAAwB,eAAe,CAAA,CAC5D,MAAA,CAAO,YAAY,CAAA,CACnB,OAAA,CAAQ,WAAA,EAAa,KAAK,CAAA;AAC7B,QAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,UAAA,MAAM,KAAK,UAAA,CAAW,EAAE,SAAA,EAAW,GAAA,CAAI,YAAY,CAAA;AAAA,QACrD;AAAA,MACF;AAAA,IACF;AAAA,EACF;AAAA,EAEA,MAAM,KAAA,GAAQ;AACZ,IAAA,IAAI,IAAA,CAAK,QAAA,CAAS,IAAA,KAAS,UAAA,EAAY;AACrC,MAAA,IAAI,KAAK,QAAA,EAAU;AACjB,QAAA,MAAM,IAAI,MAAM,sCAAsC,CAAA;AAAA,MACxD;AAEA,MAAA,MAAM,EAAE,eAAA,EAAiB,aAAA,EAAc,GAAI,IAAA,CAAK,QAAA;AAEhD,MAAA,MAAM,eAAeC,8BAAA,CAAsC;AAAA,QACzD,YAAA,EAAc,CAAA;AAAA,QACd,aAAA,EAAe,CAAA;AAAA,QACf,iBAAA,EAAmBC,+BAAuB,eAAe,CAAA;AAAA,QACzD,SAAA,EAAW,OAAM,KAAA,KAAS;AACxB,UAAA,OAAO,MAAM,IAAA,CAAK,sBAAA,CAAuB,KAAA,EAAO,aAAa,CAAA;AAAA,QAC/D,CAAA;AAAA,QACA,WAAA,EAAa,OAAM,IAAA,KAAQ;AACzB,UAAA,OAAO,MAAM,KAAK,UAAA,CAAW;AAAA,YAC3B,WAAW,IAAA,CAAK,SAAA;AAAA,YAChB,cAAc,IAAA,CAAK,YAAA;AAAA,YACnB,mBAAmB,IAAA,CAAK;AAAA,WACzB,CAAA;AAAA,QACH;AAAA,OACD,CAAA;AAED,MAAA,IAAA,CAAK,WAAW,MAAM;AACpB,QAAA,YAAA,EAAa;AAAA,MACf,CAAA;AAAA,IACF;AAAA,EACF;AAAA,EAEA,MAAM,IAAA,GAAO;AACX,IAAA,IAAI,IAAA,CAAK,QAAA,CAAS,IAAA,KAAS,UAAA,EAAY;AACrC,MAAA,IAAI,KAAK,QAAA,EAAU;AACjB,QAAA,IAAA,CAAK,QAAA,EAAS;AACd,QAAA,IAAA,CAAK,QAAA,GAAW,MAAA;AAAA,MAClB;AAAA,IACF;AAAA,EACF;AAAA,EAEA,MAAM,sBAAA,CAAuB,KAAA,EAAe,aAAA,EAA8B;AACxE,IAAA,IAAI;AACF,MAAA,OAAO,MAAMC,2DAAA,CAA8B;AAAA,QACzC,MAAM,IAAA,CAAK,IAAA;AAAA,QACX,SAAA,EAAW,KAAA;AAAA,QACX;AAAA,OACD,CAAA;AAAA,IACH,SAAS,KAAA,EAAO;AACd,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA,CAAK,oCAAA,EAAsC,KAAK,CAAA;AAC5D,MAAA,OAAO,EAAC;AAAA,IACV;AAAA,EACF;AAAA,EAEA,MAAM,WAAW,OAAA,EAId;AACD,IAAA,MAAM,KAAA,GAAQ,IAAA,CAAK,OAAA,CAAQ,WAAA,CAAY;AAAA,MACrC,WAAW,OAAA,CAAQ,SAAA;AAAA,MACnB,mBAAmB,OAAA,CAAQ;AAAA,KAC5B,CAAA;AAED,IAAA,IAAI;AACF,MAAA,MAAM,MAAA,GAAS,MAAMC,iCAAA,CAAiB;AAAA,QACpC,MAAM,IAAA,CAAK,IAAA;AAAA,QACX,QAAQ,IAAA,CAAK,MAAA;AAAA,QACb,UAAU,IAAA,CAAK,QAAA;AAAA,QACf,WAAW,OAAA,CAAQ,SAAA;AAAA,QACnB,cAAc,OAAA,CAAQ;AAAA,OACvB,CAAA;AACD,MAAA,KAAA,CAAM,aAAa,MAAM,CAAA;AAAA,IAC3B,SAAS,KAAA,EAAO;AACd,MAAA,KAAA,CAAM,WAAW,KAAK,CAAA;AAAA,IACxB;AAAA,EACF;AACF;;;;"}
+{"version":3,"file":"DefaultStitcher.cjs.js","sources":["../../src/stitching/DefaultStitcher.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Config } from '@backstage/config';\nimport { durationToMilliseconds, HumanDuration } from '@backstage/types';\nimport { Knex } from 'knex';\nimport splitToChunks from 'lodash/chunk';\nimport { DateTime } from 'luxon';\nimport { getDeferredStitchableEntities } from '../database/operations/stitcher/getDeferredStitchableEntities';\nimport { markForStitching } from '../database/operations/stitcher/markForStitching';\nimport { performStitching } from '../database/operations/stitcher/performStitching';\nimport { DbRefreshStateRow } from '../database/tables';\nimport { startTaskPipeline } from '../processing/TaskPipeline';\nimport { progressTracker } from './progressTracker';\nimport {\n  Stitcher,\n  StitchingStrategy,\n  stitchingStrategyFromConfig,\n} from './types';\nimport { LoggerService } from '@backstage/backend-plugin-api';\nimport { MetricsService } from '@backstage/backend-plugin-api/alpha';\n\ntype DeferredStitchItem = Awaited<\n  ReturnType<typeof getDeferredStitchableEntities>\n>[0];\n\ntype StitchProgressTracker = ReturnType<typeof progressTracker>;\n\n/**\n * Performs the act of stitching - to take all of the various outputs from the\n * ingestion process, and stitching them together into the final entity JSON\n * shape.\n */\nexport class DefaultStitcher implements Stitcher {\n  private readonly knex: Knex;\n  private readonly logger: LoggerService;\n  private readonly strategy: StitchingStrategy;\n  private readonly tracker: StitchProgressTracker;\n  private stopFunc?: () => void;\n\n  static fromConfig(\n    config: Config,\n    options: {\n      knex: Knex;\n      logger: LoggerService;\n      metrics: MetricsService;\n    },\n  ): DefaultStitcher {\n    return new DefaultStitcher({\n      knex: options.knex,\n      logger: options.logger,\n      metrics: options.metrics,\n      strategy: stitchingStrategyFromConfig(config, {\n        logger: options.logger,\n      }),\n    });\n  }\n\n  constructor(options: {\n    knex: Knex;\n    logger: LoggerService;\n    metrics: MetricsService;\n    strategy: StitchingStrategy;\n  }) {\n    this.knex = options.knex;\n    this.logger = options.logger;\n    this.strategy = options.strategy;\n    this.tracker = progressTracker(\n      options.knex,\n      options.logger,\n      options.metrics,\n    );\n  }\n\n  async stitch(options: {\n    entityRefs?: Iterable<string>;\n    entityIds?: Iterable<string>;\n  }) {\n    const { entityRefs, entityIds } = options;\n\n    if (this.strategy.mode === 'deferred') {\n      await markForStitching({\n        knex: this.knex,\n        strategy: this.strategy,\n        entityRefs,\n        entityIds,\n      });\n      return;\n    }\n\n    if (entityRefs) {\n      for (const entityRef of entityRefs) {\n        await this.#stitchOne({ entityRef });\n      }\n    }\n\n    if (entityIds) {\n      const chunks = splitToChunks(\n        Array.isArray(entityIds) ? entityIds : [...entityIds],\n        100,\n      );\n      for (const chunk of chunks) {\n        const rows = await this.knex<DbRefreshStateRow>('refresh_state')\n          .select('entity_ref')\n          .whereIn('entity_id', chunk);\n        for (const row of rows) {\n          await this.#stitchOne({ entityRef: row.entity_ref });\n        }\n      }\n    }\n  }\n\n  async start() {\n    if (this.strategy.mode === 'deferred') {\n      if (this.stopFunc) {\n        throw new Error('Processing engine is already started');\n      }\n\n      const { pollingInterval, stitchTimeout } = this.strategy;\n\n      const stopPipeline = startTaskPipeline<DeferredStitchItem>({\n        lowWatermark: 2,\n        highWatermark: 5,\n        pollingIntervalMs: durationToMilliseconds(pollingInterval),\n        loadTasks: async count => {\n          return await this.#getStitchableEntities(count, stitchTimeout);\n        },\n        processTask: async item => {\n          return await this.#stitchOne({\n            entityRef: item.entityRef,\n            stitchTicket: item.stitchTicket,\n            stitchRequestedAt: item.stitchRequestedAt,\n          });\n        },\n      });\n\n      this.stopFunc = () => {\n        stopPipeline();\n      };\n    }\n  }\n\n  async stop() {\n    if (this.strategy.mode === 'deferred') {\n      if (this.stopFunc) {\n        this.stopFunc();\n        this.stopFunc = undefined;\n      }\n    }\n  }\n\n  async #getStitchableEntities(count: number, stitchTimeout: HumanDuration) {\n    try {\n      return await getDeferredStitchableEntities({\n        knex: this.knex,\n        batchSize: count,\n        stitchTimeout: stitchTimeout,\n      });\n    } catch (error) {\n      this.logger.warn('Failed to load stitchable entities', error);\n      return [];\n    }\n  }\n\n  async #stitchOne(options: {\n    entityRef: string;\n    stitchTicket?: string;\n    stitchRequestedAt?: DateTime;\n  }) {\n    const track = this.tracker.stitchStart({\n      entityRef: options.entityRef,\n      stitchRequestedAt: options.stitchRequestedAt,\n    });\n\n    try {\n      const result = await performStitching({\n        knex: this.knex,\n        logger: this.logger,\n        strategy: this.strategy,\n        entityRef: options.entityRef,\n        stitchTicket: options.stitchTicket,\n      });\n      track.markComplete(result);\n    } catch (error) {\n      track.markFailed(error);\n    }\n  }\n}\n"],"names":["stitchingStrategyFromConfig","progressTracker","markForStitching","splitToChunks","startTaskPipeline","durationToMilliseconds","getDeferredStitchableEntities","performStitching"],"mappings":";;;;;;;;;;;;;;;AA8CO,MAAM,eAAA,CAAoC;AAAA,EAC9B,IAAA;AAAA,EACA,MAAA;AAAA,EACA,QAAA;AAAA,EACA,OAAA;AAAA,EACT,QAAA;AAAA,EAER,OAAO,UAAA,CACL,MAAA,EACA,OAAA,EAKiB;AACjB,IAAA,OAAO,IAAI,eAAA,CAAgB;AAAA,MACzB,MAAM,OAAA,CAAQ,IAAA;AAAA,MACd,QAAQ,OAAA,CAAQ,MAAA;AAAA,MAChB,SAAS,OAAA,CAAQ,OAAA;AAAA,MACjB,QAAA,EAAUA,kCAA4B,MAAA,EAAQ;AAAA,QAC5C,QAAQ,OAAA,CAAQ;AAAA,OACjB;AAAA,KACF,CAAA;AAAA,EACH;AAAA,EAEA,YAAY,OAAA,EAKT;AACD,IAAA,IAAA,CAAK,OAAO,OAAA,CAAQ,IAAA;AACpB,IAAA,IAAA,CAAK,SAAS,OAAA,CAAQ,MAAA;AACtB,IAAA,IAAA,CAAK,WAAW,OAAA,CAAQ,QAAA;AACxB,IAAA,IAAA,CAAK,OAAA,GAAUC,+BAAA;AAAA,MACb,OAAA,CAAQ,IAAA;AAAA,MACR,OAAA,CAAQ,MAAA;AAAA,MACR,OAAA,CAAQ;AAAA,KACV;AAAA,EACF;AAAA,EAEA,MAAM,OAAO,OAAA,EAGV;AACD,IAAA,MAAM,EAAE,UAAA,EAAY,SAAA,EAAU,GAAI,OAAA;AAElC,IAAA,IAAI,IAAA,CAAK,QAAA,CAAS,IAAA,KAAS,UAAA,EAAY;AACrC,MAAA,MAAMC,iCAAA,CAAiB;AAAA,QACrB,MAAM,IAAA,CAAK,IAAA;AAAA,QACX,UAAU,IAAA,CAAK,QAAA;AAAA,QACf,UAAA;AAAA,QACA;AAAA,OACD,CAAA;AACD,MAAA;AAAA,IACF;AAEA,IAAA,IAAI,UAAA,EAAY;AACd,MAAA,KAAA,MAAW,aAAa,UAAA,EAAY;AAClC,QAAA,MAAM,IAAA,CAAK,UAAA,CAAW,EAAE,SAAA,EAAW,CAAA;AAAA,MACrC;AAAA,IACF;AAEA,IAAA,IAAI,SAAA,EAAW;AACb,MAAA,MAAM,MAAA,GAASC,8BAAA;AAAA,QACb,MAAM,OAAA,CAAQ,SAAS,IAAI,SAAA,GAAY,CAAC,GAAG,SAAS,CAAA;AAAA,QACpD;AAAA,OACF;AACA,MAAA,KAAA,MAAW,SAAS,MAAA,EAAQ;AAC1B,QAAA,MAAM,IAAA,GAAO,MAAM,IAAA,CAAK,IAAA,CAAwB,eAAe,CAAA,CAC5D,MAAA,CAAO,YAAY,CAAA,CACnB,OAAA,CAAQ,WAAA,EAAa,KAAK,CAAA;AAC7B,QAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,UAAA,MAAM,KAAK,UAAA,CAAW,EAAE,SAAA,EAAW,GAAA,CAAI,YAAY,CAAA;AAAA,QACrD;AAAA,MACF;AAAA,IACF;AAAA,EACF;AAAA,EAEA,MAAM,KAAA,GAAQ;AACZ,IAAA,IAAI,IAAA,CAAK,QAAA,CAAS,IAAA,KAAS,UAAA,EAAY;AACrC,MAAA,IAAI,KAAK,QAAA,EAAU;AACjB,QAAA,MAAM,IAAI,MAAM,sCAAsC,CAAA;AAAA,MACxD;AAEA,MAAA,MAAM,EAAE,eAAA,EAAiB,aAAA,EAAc,GAAI,IAAA,CAAK,QAAA;AAEhD,MAAA,MAAM,eAAeC,8BAAA,CAAsC;AAAA,QACzD,YAAA,EAAc,CAAA;AAAA,QACd,aAAA,EAAe,CAAA;AAAA,QACf,iBAAA,EAAmBC,+BAAuB,eAAe,CAAA;AAAA,QACzD,SAAA,EAAW,OAAM,KAAA,KAAS;AACxB,UAAA,OAAO,MAAM,IAAA,CAAK,sBAAA,CAAuB,KAAA,EAAO,aAAa,CAAA;AAAA,QAC/D,CAAA;AAAA,QACA,WAAA,EAAa,OAAM,IAAA,KAAQ;AACzB,UAAA,OAAO,MAAM,KAAK,UAAA,CAAW;AAAA,YAC3B,WAAW,IAAA,CAAK,SAAA;AAAA,YAChB,cAAc,IAAA,CAAK,YAAA;AAAA,YACnB,mBAAmB,IAAA,CAAK;AAAA,WACzB,CAAA;AAAA,QACH;AAAA,OACD,CAAA;AAED,MAAA,IAAA,CAAK,WAAW,MAAM;AACpB,QAAA,YAAA,EAAa;AAAA,MACf,CAAA;AAAA,IACF;AAAA,EACF;AAAA,EAEA,MAAM,IAAA,GAAO;AACX,IAAA,IAAI,IAAA,CAAK,QAAA,CAAS,IAAA,KAAS,UAAA,EAAY;AACrC,MAAA,IAAI,KAAK,QAAA,EAAU;AACjB,QAAA,IAAA,CAAK,QAAA,EAAS;AACd,QAAA,IAAA,CAAK,QAAA,GAAW,MAAA;AAAA,MAClB;AAAA,IACF;AAAA,EACF;AAAA,EAEA,MAAM,sBAAA,CAAuB,KAAA,EAAe,aAAA,EAA8B;AACxE,IAAA,IAAI;AACF,MAAA,OAAO,MAAMC,2DAAA,CAA8B;AAAA,QACzC,MAAM,IAAA,CAAK,IAAA;AAAA,QACX,SAAA,EAAW,KAAA;AAAA,QACX;AAAA,OACD,CAAA;AAAA,IACH,SAAS,KAAA,EAAO;AACd,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA,CAAK,oCAAA,EAAsC,KAAK,CAAA;AAC5D,MAAA,OAAO,EAAC;AAAA,IACV;AAAA,EACF;AAAA,EAEA,MAAM,WAAW,OAAA,EAId;AACD,IAAA,MAAM,KAAA,GAAQ,IAAA,CAAK,OAAA,CAAQ,WAAA,CAAY;AAAA,MACrC,WAAW,OAAA,CAAQ,SAAA;AAAA,MACnB,mBAAmB,OAAA,CAAQ;AAAA,KAC5B,CAAA;AAED,IAAA,IAAI;AACF,MAAA,MAAM,MAAA,GAAS,MAAMC,iCAAA,CAAiB;AAAA,QACpC,MAAM,IAAA,CAAK,IAAA;AAAA,QACX,QAAQ,IAAA,CAAK,MAAA;AAAA,QACb,UAAU,IAAA,CAAK,QAAA;AAAA,QACf,WAAW,OAAA,CAAQ,SAAA;AAAA,QACnB,cAAc,OAAA,CAAQ;AAAA,OACvB,CAAA;AACD,MAAA,KAAA,CAAM,aAAa,MAAM,CAAA;AAAA,IAC3B,SAAS,KAAA,EAAO;AACd,MAAA,KAAA,CAAM,WAAW,KAAK,CAAA;AAAA,IACxB;AAAA,EACF;AACF;;;;"}

package/dist/stitching/types.cjs.js

@@ -2,11 +2,18 @@
 
 var config = require('@backstage/config');
 
-function stitchingStrategyFromConfig(config$1) {
+let immediateDeprecationLogged = false;
+function stitchingStrategyFromConfig(config$1, options) {
   const strategyMode = config$1.getOptionalString(
     "catalog.stitchingStrategy.mode"
   );
   if (strategyMode === "immediate") {
+    if (!immediateDeprecationLogged) {
+      immediateDeprecationLogged = true;
+      options.logger.warn(
+        "DEPRECATED: Immediate mode stitching has been deprecated, and will be removed in the next Backstage release."
+      );
+    }
     return {
       mode: "immediate"
     };

package/dist/stitching/types.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"types.cjs.js","sources":["../../src/stitching/types.ts"],"sourcesContent":["/*\n * Copyright 2023 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Config, readDurationFromConfig } from '@backstage/config';\nimport { HumanDuration } from '@backstage/types';\n\n/**\n * Performs the act of stitching - to take all of the various outputs from the\n * ingestion process, and stitching them together into the final entity JSON\n * shape.\n */\nexport interface Stitcher {\n  stitch(options: {\n    entityRefs?: Iterable<string>;\n    entityIds?: Iterable<string>;\n  }): Promise<void>;\n}\n\n/**\n * The strategies supported by the stitching process, in terms of when to\n * perform stitching.\n *\n * @remarks\n *\n * In immediate mode, stitching happens \"in-band\" (blocking) immediately when\n * each processing task finishes. When set to `'deferred'`, stitching is instead\n * deferred to happen on a separate asynchronous worker queue just like\n * processing.\n *\n * Deferred stitching should make performance smoother when ingesting large\n * amounts of entities, and reduce p99 processing times and repeated\n * over-stitching of hot spot entities when fan-out/fan-in in terms of relations\n * is very large. It does however also come with some performance cost due to\n * the queuing with how much wall-clock time some types of task take.\n */\nexport type StitchingStrategy =\n  | {\n      mode: 'immediate';\n    }\n  | {\n      mode: 'deferred';\n      pollingInterval: HumanDuration;\n      stitchTimeout: HumanDuration;\n    };\n\nexport function stitchingStrategyFromConfig(config: Config): StitchingStrategy {\n  const strategyMode = config.getOptionalString(\n    'catalog.stitchingStrategy.mode',\n  );\n\n  if (strategyMode === 'immediate') {\n    return {\n      mode: 'immediate',\n    };\n  } else if (strategyMode === undefined || strategyMode === 'deferred') {\n    const pollingIntervalKey = 'catalog.stitchingStrategy.pollingInterval';\n    const stitchTimeoutKey = 'catalog.stitchingStrategy.stitchTimeout';\n\n    const pollingInterval = config.has(pollingIntervalKey)\n      ? readDurationFromConfig(config, { key: pollingIntervalKey })\n      : { seconds: 1 };\n    const stitchTimeout = config.has(stitchTimeoutKey)\n      ? readDurationFromConfig(config, { key: stitchTimeoutKey })\n      : { seconds: 60 };\n\n    return {\n      mode: 'deferred',\n      pollingInterval: pollingInterval,\n      stitchTimeout: stitchTimeout,\n    };\n  }\n\n  throw new Error(\n    `Invalid stitching strategy mode '${strategyMode}', expected one of 'immediate' or 'deferred'`,\n  );\n}\n"],"names":["config","readDurationFromConfig"],"mappings":";;;;AA0DO,SAAS,4BAA4BA,QAAA,EAAmC;AAC7E,EAAA,MAAM,eAAeA,QAAA,CAAO,iBAAA;AAAA,IAC1B;AAAA,GACF;AAEA,EAAA,IAAI,iBAAiB,WAAA,EAAa;AAChC,IAAA,OAAO;AAAA,MACL,IAAA,EAAM;AAAA,KACR;AAAA,EACF,CAAA,MAAA,IAAW,YAAA,KAAiB,MAAA,IAAa,YAAA,KAAiB,UAAA,EAAY;AACpE,IAAA,MAAM,kBAAA,GAAqB,2CAAA;AAC3B,IAAA,MAAM,gBAAA,GAAmB,yCAAA;AAEzB,IAAA,MAAM,eAAA,GAAkBA,QAAA,CAAO,GAAA,CAAI,kBAAkB,IACjDC,6BAAA,CAAuBD,QAAA,EAAQ,EAAE,GAAA,EAAK,kBAAA,EAAoB,CAAA,GAC1D,EAAE,SAAS,CAAA,EAAE;AACjB,IAAA,MAAM,aAAA,GAAgBA,QAAA,CAAO,GAAA,CAAI,gBAAgB,IAC7CC,6BAAA,CAAuBD,QAAA,EAAQ,EAAE,GAAA,EAAK,gBAAA,EAAkB,CAAA,GACxD,EAAE,SAAS,EAAA,EAAG;AAElB,IAAA,OAAO;AAAA,MACL,IAAA,EAAM,UAAA;AAAA,MACN,eAAA;AAAA,MACA;AAAA,KACF;AAAA,EACF;AAEA,EAAA,MAAM,IAAI,KAAA;AAAA,IACR,oCAAoC,YAAY,CAAA,4CAAA;AAAA,GAClD;AACF;;;;"}
+{"version":3,"file":"types.cjs.js","sources":["../../src/stitching/types.ts"],"sourcesContent":["/*\n * Copyright 2023 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { LoggerService } from '@backstage/backend-plugin-api';\nimport { Config, readDurationFromConfig } from '@backstage/config';\nimport { HumanDuration } from '@backstage/types';\n\n/**\n * Performs the act of stitching - to take all of the various outputs from the\n * ingestion process, and stitching them together into the final entity JSON\n * shape.\n */\nexport interface Stitcher {\n  stitch(options: {\n    entityRefs?: Iterable<string>;\n    entityIds?: Iterable<string>;\n  }): Promise<void>;\n}\n\n/**\n * The strategies supported by the stitching process, in terms of when to\n * perform stitching.\n *\n * @remarks\n *\n * In immediate mode, stitching happens \"in-band\" (blocking) immediately when\n * each processing task finishes. When set to `'deferred'`, stitching is instead\n * deferred to happen on a separate asynchronous worker queue just like\n * processing.\n *\n * Deferred stitching should make performance smoother when ingesting large\n * amounts of entities, and reduce p99 processing times and repeated\n * over-stitching of hot spot entities when fan-out/fan-in in terms of relations\n * is very large. It does however also come with some performance cost due to\n * the queuing with how much wall-clock time some types of task take.\n *\n * Note: Immediate mode is deprecated and will be removed in a future release.\n */\nexport type StitchingStrategy =\n  | {\n      mode: 'immediate';\n    }\n  | {\n      mode: 'deferred';\n      pollingInterval: HumanDuration;\n      stitchTimeout: HumanDuration;\n    };\n\nlet immediateDeprecationLogged = false;\n\nexport function stitchingStrategyFromConfig(\n  config: Config,\n  options: { logger: LoggerService },\n): StitchingStrategy {\n  const strategyMode = config.getOptionalString(\n    'catalog.stitchingStrategy.mode',\n  );\n\n  if (strategyMode === 'immediate') {\n    if (!immediateDeprecationLogged) {\n      immediateDeprecationLogged = true;\n      options.logger.warn(\n        'DEPRECATED: Immediate mode stitching has been deprecated, and will be removed in the next Backstage release.',\n      );\n    }\n    return {\n      mode: 'immediate',\n    };\n  } else if (strategyMode === undefined || strategyMode === 'deferred') {\n    const pollingIntervalKey = 'catalog.stitchingStrategy.pollingInterval';\n    const stitchTimeoutKey = 'catalog.stitchingStrategy.stitchTimeout';\n\n    const pollingInterval = config.has(pollingIntervalKey)\n      ? readDurationFromConfig(config, { key: pollingIntervalKey })\n      : { seconds: 1 };\n    const stitchTimeout = config.has(stitchTimeoutKey)\n      ? readDurationFromConfig(config, { key: stitchTimeoutKey })\n      : { seconds: 60 };\n\n    return {\n      mode: 'deferred',\n      pollingInterval: pollingInterval,\n      stitchTimeout: stitchTimeout,\n    };\n  }\n\n  throw new Error(\n    `Invalid stitching strategy mode '${strategyMode}', expected one of 'immediate' or 'deferred'`,\n  );\n}\n"],"names":["config","readDurationFromConfig"],"mappings":";;;;AA6DA,IAAI,0BAAA,GAA6B,KAAA;AAE1B,SAAS,2BAAA,CACdA,UACA,OAAA,EACmB;AACnB,EAAA,MAAM,eAAeA,QAAA,CAAO,iBAAA;AAAA,IAC1B;AAAA,GACF;AAEA,EAAA,IAAI,iBAAiB,WAAA,EAAa;AAChC,IAAA,IAAI,CAAC,0BAAA,EAA4B;AAC/B,MAAA,0BAAA,GAA6B,IAAA;AAC7B,MAAA,OAAA,CAAQ,MAAA,CAAO,IAAA;AAAA,QACb;AAAA,OACF;AAAA,IACF;AACA,IAAA,OAAO;AAAA,MACL,IAAA,EAAM;AAAA,KACR;AAAA,EACF,CAAA,MAAA,IAAW,YAAA,KAAiB,MAAA,IAAa,YAAA,KAAiB,UAAA,EAAY;AACpE,IAAA,MAAM,kBAAA,GAAqB,2CAAA;AAC3B,IAAA,MAAM,gBAAA,GAAmB,yCAAA;AAEzB,IAAA,MAAM,eAAA,GAAkBA,QAAA,CAAO,GAAA,CAAI,kBAAkB,IACjDC,6BAAA,CAAuBD,QAAA,EAAQ,EAAE,GAAA,EAAK,kBAAA,EAAoB,CAAA,GAC1D,EAAE,SAAS,CAAA,EAAE;AACjB,IAAA,MAAM,aAAA,GAAgBA,QAAA,CAAO,GAAA,CAAI,gBAAgB,IAC7CC,6BAAA,CAAuBD,QAAA,EAAQ,EAAE,GAAA,EAAK,gBAAA,EAAkB,CAAA,GACxD,EAAE,SAAS,EAAA,EAAG;AAElB,IAAA,OAAO;AAAA,MACL,IAAA,EAAM,UAAA;AAAA,MACN,eAAA;AAAA,MACA;AAAA,KACF;AAAA,EACF;AAEA,EAAA,MAAM,IAAI,KAAA;AAAA,IACR,oCAAoC,YAAY,CAAA,4CAAA;AAAA,GAClD;AACF;;;;"}

package/migrations/20260516000000_relations_target_index.js

@@ -0,0 +1,92 @@
+/*
+ * Copyright 2026 The Backstage Authors
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// @ts-check
+
+/**
+ * Adds an index on `relations.target_entity_ref`.
+ *
+ * The `relations` table had indexes on `originating_entity_id` and
+ * `source_entity_ref` but none on `target_entity_ref`. Several query
+ * paths (orphan deletion, entity ancestry, eager pruning) join or
+ * filter on `target_entity_ref`, causing full sequential scans of the
+ * table (~3.5M rows, ~714 MB heap).
+ *
+ * On PostgreSQL this uses CREATE INDEX CONCURRENTLY to avoid blocking
+ * reads and writes. The index is ~141 MB based on the column's average
+ * width of ~35 bytes across ~3.5M rows.
+ */
+
+/**
+ * @param {import('knex').Knex} knex
+ */
+exports.up = async function up(knex) {
+  const client = knex.client.config.client;
+
+  if (client.includes('pg')) {
+    // Check if index already exists in the current schema (idempotent).
+    // The pg_class lookup is scoped via pg_namespace to handle
+    // schema-division mode where each plugin has its own schema.
+    const result = await knex.raw(
+      `SELECT c.oid, i.indisvalid
+       FROM pg_class c
+       JOIN pg_namespace n ON n.oid = c.relnamespace
+       LEFT JOIN pg_index i ON i.indexrelid = c.oid
+       WHERE c.relname = ?
+         AND c.relkind = 'i'
+         AND n.nspname = current_schema()`,
+      ['relations_target_entity_ref_idx'],
+    );
+
+    if (result.rows.length > 0) {
+      if (result.rows[0].indisvalid) {
+        return; // Already exists and valid
+      }
+      // Invalid — drop and recreate
+      await knex.raw(
+        'DROP INDEX CONCURRENTLY IF EXISTS relations_target_entity_ref_idx',
+      );
+    }
+
+    await knex.raw(
+      'CREATE INDEX CONCURRENTLY relations_target_entity_ref_idx ON relations (target_entity_ref)',
+    );
+  } else {
+    // SQLite / MySQL — simple CREATE INDEX
+    await knex.schema.alterTable('relations', table => {
+      table.index(['target_entity_ref'], 'relations_target_entity_ref_idx');
+    });
+  }
+};
+
+/**
+ * @param {import('knex').Knex} knex
+ */
+exports.down = async function down(knex) {
+  const client = knex.client.config.client;
+
+  if (client.includes('pg')) {
+    await knex.raw(
+      'DROP INDEX CONCURRENTLY IF EXISTS relations_target_entity_ref_idx',
+    );
+  } else {
+    await knex.schema.alterTable('relations', table => {
+      table.dropIndex([], 'relations_target_entity_ref_idx');
+    });
+  }
+};
+
+exports.config = { transaction: false };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@backstage/plugin-catalog-backend",
-  "version": "3.7.0-next.2",
+  "version": "3.7.0",
   "description": "The Backstage backend plugin that provides the Backstage catalog",
   "backstage": {
     "role": "backend-plugin",
@@ -76,20 +76,20 @@
     "test": "backstage-cli package test"
   },
   "dependencies": {
-    "@backstage/backend-openapi-utils": "0.6.9-next.0",
-    "@backstage/backend-plugin-api": "1.9.1-next.1",
-    "@backstage/catalog-client": "1.15.1-next.0",
-    "@backstage/catalog-model": "1.8.1-next.1",
-    "@backstage/config": "1.3.8-next.0",
-    "@backstage/errors": "1.3.1-next.0",
-    "@backstage/filter-predicates": "0.1.3-next.0",
-    "@backstage/integration": "2.0.2-next.1",
-    "@backstage/plugin-catalog-common": "1.1.10-next.0",
-    "@backstage/plugin-catalog-node": "2.2.1-next.1",
-    "@backstage/plugin-events-node": "0.4.22-next.0",
-    "@backstage/plugin-permission-common": "0.9.9-next.1",
-    "@backstage/plugin-permission-node": "0.10.13-next.0",
-    "@backstage/types": "1.2.2",
+    "@backstage/backend-openapi-utils": "^0.6.9",
+    "@backstage/backend-plugin-api": "^1.9.1",
+    "@backstage/catalog-client": "^1.15.1",
+    "@backstage/catalog-model": "^1.9.0",
+    "@backstage/config": "^1.3.8",
+    "@backstage/errors": "^1.3.1",
+    "@backstage/filter-predicates": "^0.1.3",
+    "@backstage/integration": "^2.0.2",
+    "@backstage/plugin-catalog-common": "^1.1.10",
+    "@backstage/plugin-catalog-node": "^2.2.1",
+    "@backstage/plugin-events-node": "^0.4.22",
+    "@backstage/plugin-permission-common": "^0.9.9",
+    "@backstage/plugin-permission-node": "^0.11.0",
+    "@backstage/types": "^1.2.2",
     "@opentelemetry/api": "^1.9.0",
     "ajv": "^8.10.0",
     "ajv-errors": "^3.0.0",
@@ -112,12 +112,12 @@
     "zod-validation-error": "^4.0.2"
   },
   "devDependencies": {
-    "@backstage/backend-defaults": "0.17.1-next.2",
-    "@backstage/backend-test-utils": "1.11.3-next.2",
-    "@backstage/cli": "0.36.2-next.1",
-    "@backstage/plugin-catalog-backend-module-logs": "0.1.22-next.1",
-    "@backstage/plugin-scaffolder-common": "2.2.0-next.1",
-    "@backstage/repo-tools": "0.17.2-next.0",
+    "@backstage/backend-defaults": "^0.17.1",
+    "@backstage/backend-test-utils": "^1.11.3",
+    "@backstage/cli": "^0.36.2",
+    "@backstage/plugin-catalog-backend-module-logs": "^0.1.22",
+    "@backstage/plugin-scaffolder-common": "^2.2.0",
+    "@backstage/repo-tools": "^0.17.2",
     "@types/core-js": "^2.5.4",
     "@types/express": "^4.17.6",
     "@types/git-url-parse": "^9.0.0",