@backstage/plugin-catalog-backend 3.6.2-next.1 → 3.7.0-next.2

package/CHANGELOG.md

@@ -1,5 +1,58 @@
 # @backstage/plugin-catalog-backend
 
+## 3.7.0-next.2
+
+### Minor Changes
+
+- c2de113: **BREAKING**: When paginating entities with an order field via `/entities/by-query`, entities that lack the order field are now excluded from both the result set and the `totalItems` count. Previously these entities appeared at the end of the sorted result via `NULLS LAST`, but cursor-based pagination could not actually reach them past the first page — the count over-reported the number of navigable entities. The new behavior aligns the count with what is actually returned.
+
+  This also removes the `DISTINCT` deduplication from the sort-field CTE, which is a prerequisite for the planner to use the `(key, value, entity_id)` index in sort order and short-circuit on `LIMIT`. Installations with duplicate search rows should land the search-table deduplication migration before adopting this change.
+
+### Patch Changes
+
+- ccbad9d: Improved the performance of the `catalog_entities_count` metric.
+
+  The legacy Prometheus and OpenTelemetry observable gauges previously each ran their own copy of the per-kind count query against the `search` table on every metrics scrape. On large catalogs this could pile up faster than the queries completed, contending for buffers and stalling the database.
+
+  The two callbacks now share a single query result with a short in-process TTL cache, and the underlying query reads from `final_entities` instead of `search`, avoiding the bitmap heap scans that dominated the previous form. The emitted labels and values are unchanged.
+
+- add5d1a: Restructured the entity listing endpoint so that, when a sort field is specified, the search-by-key index drives the query rather than being side-joined onto `final_entities`. This lets PostgreSQL walk the `(key, value, entity_id)` index in already-sorted order and short-circuit on `LIMIT`, reducing typical broad-filter paginated list times from seconds to milliseconds. Entities that lack the sort field still appear at the end of sorted results (NULLS LAST semantics preserved), ordered by `entity_id`.
+- 387ea7d: Simplified the entity facets aggregation from `COUNT(DISTINCT entity_id)` to `COUNT(*)`. The unique constraint on `(entity_id, key, value)` guarantees each entity appears at most once per search row group, making the `DISTINCT` unnecessary. This allows the database to use a simpler aggregation plan.
+- 3f55b73: Improved the performance of the entity facets endpoint when filters are applied. The filtered entity set is now combined with the search table through an inner join rather than a `WHERE entity_id IN (subquery)`. Results are unchanged; on large catalogs the query planner is able to choose dramatically cheaper plans, with measured improvements ranging from roughly 1.2× on already-fast cases to 7× or more on high-cardinality facets.
+- cde3643: Added missing description to the `type` parameter on the `unregister-entity` MCP action.
+- 7445f0f: Added a migration that removes duplicate rows from the `search` table, creates covering indices for improved query performance, and adds a `UNIQUE` constraint on `(entity_id, key, value)`.
+
+  This is a long-running migration on large catalogs. On PostgreSQL with millions of search rows, the index creation may take 5-15 minutes per index. During this time, other pods running the previous version will continue to serve traffic normally — the index creation does not block reads or writes. However, if a Kubernetes liveness probe kills the pod before the index build completes, the build is lost and the next startup will start over. On large tables this can repeat indefinitely.
+
+  **For large installations**, it is recommended to run the following SQL commands against your PostgreSQL database **before deploying** this version. Each index build takes a few minutes but does not block reads or writes. If these have already completed, the migration will detect the existing indices and skip all work — startup will be instant.
+
+  ```sql
+  -- Step 1: Remove duplicate search rows
+  WITH cte AS (
+    SELECT ctid, row_number() OVER (PARTITION BY entity_id, key, value) AS rn
+    FROM search
+  )
+  DELETE FROM search USING cte WHERE search.ctid = cte.ctid AND cte.rn > 1;
+
+  -- Step 2: Create new indices (run each separately)
+  CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS
+    search_entity_key_value_idx ON search (entity_id, key, value);
+  CREATE INDEX CONCURRENTLY IF NOT EXISTS
+    search_key_value_entity_idx ON search (key, value, entity_id);
+  CREATE INDEX CONCURRENTLY IF NOT EXISTS
+    search_facets_covering_idx ON search (key, original_value, entity_id)
+    WHERE original_value IS NOT NULL;
+
+  -- Step 3: Drop old indices that are no longer needed
+  DROP INDEX CONCURRENTLY IF EXISTS search_key_value_idx;
+  DROP INDEX CONCURRENTLY IF EXISTS search_key_original_value_idx;
+  ```
+
+  Also fixed `buildEntitySearch` to remove duplicate output for entities with duplicate array values, and added `ON CONFLICT DO UPDATE` to `syncSearchRows` so that concurrent stitching races are handled gracefully.
+
+- Updated dependencies
+  - @backstage/backend-plugin-api@1.9.1-next.1
+
 ## 3.6.2-next.1
 
 ### Patch Changes

package/dist/actions/createUnregisterCatalogEntitiesAction.cjs.js

@@ -31,10 +31,10 @@ Once completed, all entities associated with the Location will be deleted from t
               `URL of the catalog-info.yaml file to unregister for example: https://github.com/backstage/demo/blob/master/catalog-info.yaml`
             )
           })
-        ])
-      }).describe(
-        "The type to the unregister-entity action. Either locationId or locationUrl must be provided."
-      ),
+        ]).describe(
+          "Identifies the entity to unregister. Provide either locationId or locationUrl."
+        )
+      }),
       output: (z) => z.object({})
     },
     action: async ({ input: { type }, credentials }) => {

package/dist/actions/createUnregisterCatalogEntitiesAction.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"createUnregisterCatalogEntitiesAction.cjs.js","sources":["../../src/actions/createUnregisterCatalogEntitiesAction.ts"],"sourcesContent":["/*\n * Copyright 2025 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\nimport { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport { NotFoundError } from '@backstage/errors';\nimport { CatalogService } from '@backstage/plugin-catalog-node';\n\nexport const createUnregisterCatalogEntitiesAction = ({\n  catalog,\n  actionsRegistry,\n}: {\n  catalog: CatalogService;\n  actionsRegistry: ActionsRegistryService;\n}) => {\n  actionsRegistry.register({\n    name: 'unregister-entity',\n    title: 'Unregister entity from the Catalog',\n    attributes: {\n      destructive: true,\n      readOnly: false,\n      idempotent: true,\n    },\n    description: `Unregisters a Location entity and all entities it owns from the Backstage catalog.\n\nThis action is similar to the \"Unregister location\" function in the Backstage UI, where you provide the unique identifier (locationId) of a Location entity. Alternatively, you can provide the URL used to register the location. The action will remove the specified Location from the catalog as well as all entities that were created when the Location was imported.\n\nOnce completed, all entities associated with the Location will be deleted from the catalog.\n`,\n    schema: {\n      input: z =>\n        z\n          .object({\n            type: z.union([\n              z.object({\n                locationId: z\n                  .string()\n                  .describe(`Location ID of the Entity to unregister`),\n              }),\n              z.object({\n                locationUrl: z\n                  .string()\n                  .describe(\n                    `URL of the catalog-info.yaml file to unregister for example: https://github.com/backstage/demo/blob/master/catalog-info.yaml`,\n                  ),\n              }),\n            ]),\n          })\n          .describe(\n            'The type to the unregister-entity action. Either locationId or locationUrl must be provided.',\n          ),\n      output: z => z.object({}),\n    },\n    action: async ({ input: { type }, credentials }) => {\n      if ('locationId' in type) {\n        await catalog.removeLocationById(type.locationId, {\n          credentials,\n        });\n      } else {\n        const locations = await catalog\n          .getLocations(\n            {},\n            {\n              credentials,\n            },\n          )\n          .then(response =>\n            response.items.filter(\n              location =>\n                location.target.toLowerCase() ===\n                type.locationUrl.toLowerCase(),\n            ),\n          );\n\n        if (locations.length === 0) {\n          throw new NotFoundError(\n            `Location with URL ${type.locationUrl} not found`,\n          );\n        }\n\n        for (const location of locations) {\n          await catalog.removeLocationById(location.id, {\n            credentials,\n          });\n        }\n      }\n\n      return { output: {} };\n    },\n  });\n};\n"],"names":["NotFoundError"],"mappings":";;;;AAmBO,MAAM,wCAAwC,CAAC;AAAA,EACpD,OAAA;AAAA,EACA;AACF,CAAA,KAGM;AACJ,EAAA,eAAA,CAAgB,QAAA,CAAS;AAAA,IACvB,IAAA,EAAM,mBAAA;AAAA,IACN,KAAA,EAAO,oCAAA;AAAA,IACP,UAAA,EAAY;AAAA,MACV,WAAA,EAAa,IAAA;AAAA,MACb,QAAA,EAAU,KAAA;AAAA,MACV,UAAA,EAAY;AAAA,KACd;AAAA,IACA,WAAA,EAAa,CAAA;;AAAA;;AAAA;AAAA,CAAA;AAAA,IAMb,MAAA,EAAQ;AAAA,MACN,KAAA,EAAO,CAAA,CAAA,KACL,CAAA,CACG,MAAA,CAAO;AAAA,QACN,IAAA,EAAM,EAAE,KAAA,CAAM;AAAA,UACZ,EAAE,MAAA,CAAO;AAAA,YACP,UAAA,EAAY,CAAA,CACT,MAAA,EAAO,CACP,SAAS,CAAA,uCAAA,CAAyC;AAAA,WACtD,CAAA;AAAA,UACD,EAAE,MAAA,CAAO;AAAA,YACP,WAAA,EAAa,CAAA,CACV,MAAA,EAAO,CACP,QAAA;AAAA,cACC,CAAA,4HAAA;AAAA;AACF,WACH;AAAA,SACF;AAAA,OACF,CAAA,CACA,QAAA;AAAA,QACC;AAAA,OACF;AAAA,MACJ,MAAA,EAAQ,CAAA,CAAA,KAAK,CAAA,CAAE,MAAA,CAAO,EAAE;AAAA,KAC1B;AAAA,IACA,MAAA,EAAQ,OAAO,EAAE,KAAA,EAAO,EAAE,IAAA,EAAK,EAAG,aAAY,KAAM;AAClD,MAAA,IAAI,gBAAgB,IAAA,EAAM;AACxB,QAAA,MAAM,OAAA,CAAQ,kBAAA,CAAmB,IAAA,CAAK,UAAA,EAAY;AAAA,UAChD;AAAA,SACD,CAAA;AAAA,MACH,CAAA,MAAO;AACL,QAAA,MAAM,SAAA,GAAY,MAAM,OAAA,CACrB,YAAA;AAAA,UACC,EAAC;AAAA,UACD;AAAA,YACE;AAAA;AACF,SACF,CACC,IAAA;AAAA,UAAK,CAAA,QAAA,KACJ,SAAS,KAAA,CAAM,MAAA;AAAA,YACb,cACE,QAAA,CAAS,MAAA,CAAO,aAAY,KAC5B,IAAA,CAAK,YAAY,WAAA;AAAY;AACjC,SACF;AAEF,QAAA,IAAI,SAAA,CAAU,WAAW,CAAA,EAAG;AAC1B,UAAA,MAAM,IAAIA,oBAAA;AAAA,YACR,CAAA,kBAAA,EAAqB,KAAK,WAAW,CAAA,UAAA;AAAA,WACvC;AAAA,QACF;AAEA,QAAA,KAAA,MAAW,YAAY,SAAA,EAAW;AAChC,UAAA,MAAM,OAAA,CAAQ,kBAAA,CAAmB,QAAA,CAAS,EAAA,EAAI;AAAA,YAC5C;AAAA,WACD,CAAA;AAAA,QACH;AAAA,MACF;AAEA,MAAA,OAAO,EAAE,MAAA,EAAQ,EAAC,EAAE;AAAA,IACtB;AAAA,GACD,CAAA;AACH;;;;"}
+{"version":3,"file":"createUnregisterCatalogEntitiesAction.cjs.js","sources":["../../src/actions/createUnregisterCatalogEntitiesAction.ts"],"sourcesContent":["/*\n * Copyright 2025 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\nimport { ActionsRegistryService } from '@backstage/backend-plugin-api/alpha';\nimport { NotFoundError } from '@backstage/errors';\nimport { CatalogService } from '@backstage/plugin-catalog-node';\n\nexport const createUnregisterCatalogEntitiesAction = ({\n  catalog,\n  actionsRegistry,\n}: {\n  catalog: CatalogService;\n  actionsRegistry: ActionsRegistryService;\n}) => {\n  actionsRegistry.register({\n    name: 'unregister-entity',\n    title: 'Unregister entity from the Catalog',\n    attributes: {\n      destructive: true,\n      readOnly: false,\n      idempotent: true,\n    },\n    description: `Unregisters a Location entity and all entities it owns from the Backstage catalog.\n\nThis action is similar to the \"Unregister location\" function in the Backstage UI, where you provide the unique identifier (locationId) of a Location entity. Alternatively, you can provide the URL used to register the location. The action will remove the specified Location from the catalog as well as all entities that were created when the Location was imported.\n\nOnce completed, all entities associated with the Location will be deleted from the catalog.\n`,\n    schema: {\n      input: z =>\n        z.object({\n          type: z\n            .union([\n              z.object({\n                locationId: z\n                  .string()\n                  .describe(`Location ID of the Entity to unregister`),\n              }),\n              z.object({\n                locationUrl: z\n                  .string()\n                  .describe(\n                    `URL of the catalog-info.yaml file to unregister for example: https://github.com/backstage/demo/blob/master/catalog-info.yaml`,\n                  ),\n              }),\n            ])\n            .describe(\n              'Identifies the entity to unregister. Provide either locationId or locationUrl.',\n            ),\n        }),\n      output: z => z.object({}),\n    },\n    action: async ({ input: { type }, credentials }) => {\n      if ('locationId' in type) {\n        await catalog.removeLocationById(type.locationId, {\n          credentials,\n        });\n      } else {\n        const locations = await catalog\n          .getLocations(\n            {},\n            {\n              credentials,\n            },\n          )\n          .then(response =>\n            response.items.filter(\n              location =>\n                location.target.toLowerCase() ===\n                type.locationUrl.toLowerCase(),\n            ),\n          );\n\n        if (locations.length === 0) {\n          throw new NotFoundError(\n            `Location with URL ${type.locationUrl} not found`,\n          );\n        }\n\n        for (const location of locations) {\n          await catalog.removeLocationById(location.id, {\n            credentials,\n          });\n        }\n      }\n\n      return { output: {} };\n    },\n  });\n};\n"],"names":["NotFoundError"],"mappings":";;;;AAmBO,MAAM,wCAAwC,CAAC;AAAA,EACpD,OAAA;AAAA,EACA;AACF,CAAA,KAGM;AACJ,EAAA,eAAA,CAAgB,QAAA,CAAS;AAAA,IACvB,IAAA,EAAM,mBAAA;AAAA,IACN,KAAA,EAAO,oCAAA;AAAA,IACP,UAAA,EAAY;AAAA,MACV,WAAA,EAAa,IAAA;AAAA,MACb,QAAA,EAAU,KAAA;AAAA,MACV,UAAA,EAAY;AAAA,KACd;AAAA,IACA,WAAA,EAAa,CAAA;;AAAA;;AAAA;AAAA,CAAA;AAAA,IAMb,MAAA,EAAQ;AAAA,MACN,KAAA,EAAO,CAAA,CAAA,KACL,CAAA,CAAE,MAAA,CAAO;AAAA,QACP,IAAA,EAAM,EACH,KAAA,CAAM;AAAA,UACL,EAAE,MAAA,CAAO;AAAA,YACP,UAAA,EAAY,CAAA,CACT,MAAA,EAAO,CACP,SAAS,CAAA,uCAAA,CAAyC;AAAA,WACtD,CAAA;AAAA,UACD,EAAE,MAAA,CAAO;AAAA,YACP,WAAA,EAAa,CAAA,CACV,MAAA,EAAO,CACP,QAAA;AAAA,cACC,CAAA,4HAAA;AAAA;AACF,WACH;AAAA,SACF,CAAA,CACA,QAAA;AAAA,UACC;AAAA;AACF,OACH,CAAA;AAAA,MACH,MAAA,EAAQ,CAAA,CAAA,KAAK,CAAA,CAAE,MAAA,CAAO,EAAE;AAAA,KAC1B;AAAA,IACA,MAAA,EAAQ,OAAO,EAAE,KAAA,EAAO,EAAE,IAAA,EAAK,EAAG,aAAY,KAAM;AAClD,MAAA,IAAI,gBAAgB,IAAA,EAAM;AACxB,QAAA,MAAM,OAAA,CAAQ,kBAAA,CAAmB,IAAA,CAAK,UAAA,EAAY;AAAA,UAChD;AAAA,SACD,CAAA;AAAA,MACH,CAAA,MAAO;AACL,QAAA,MAAM,SAAA,GAAY,MAAM,OAAA,CACrB,YAAA;AAAA,UACC,EAAC;AAAA,UACD;AAAA,YACE;AAAA;AACF,SACF,CACC,IAAA;AAAA,UAAK,CAAA,QAAA,KACJ,SAAS,KAAA,CAAM,MAAA;AAAA,YACb,cACE,QAAA,CAAS,MAAA,CAAO,aAAY,KAC5B,IAAA,CAAK,YAAY,WAAA;AAAY;AACjC,SACF;AAEF,QAAA,IAAI,SAAA,CAAU,WAAW,CAAA,EAAG;AAC1B,UAAA,MAAM,IAAIA,oBAAA;AAAA,YACR,CAAA,kBAAA,EAAqB,KAAK,WAAW,CAAA,UAAA;AAAA,WACvC;AAAA,QACF;AAEA,QAAA,KAAA,MAAW,YAAY,SAAA,EAAW;AAChC,UAAA,MAAM,OAAA,CAAQ,kBAAA,CAAmB,QAAA,CAAS,EAAA,EAAI;AAAA,YAC5C;AAAA,WACD,CAAA;AAAA,QACH;AAAA,MACF;AAEA,MAAA,OAAO,EAAE,MAAA,EAAQ,EAAC,EAAE;AAAA,IACtB;AAAA,GACD,CAAA;AACH;;;;"}

package/dist/database/metrics.cjs.js

@@ -2,26 +2,68 @@
 
 var metrics = require('../util/metrics.cjs.js');
 
+const ENTITIES_COUNT_TTL_MS = 3e4;
+function createEntitiesCountByKind(knex, options) {
+  const ttlMs = ENTITIES_COUNT_TTL_MS;
+  let inflight;
+  let cached;
+  return () => {
+    if (inflight) {
+      return inflight;
+    }
+    if (cached && Date.now() - cached.at < ttlMs) {
+      return Promise.resolve(cached.data);
+    }
+    inflight = (async () => {
+      try {
+        const data = await queryEntitiesCountByKind(knex);
+        cached = { at: Date.now(), data };
+        return data;
+      } finally {
+        inflight = void 0;
+      }
+    })();
+    return inflight;
+  };
+}
+async function queryEntitiesCountByKind(knex) {
+  const kindExpr = entityRefKindExpression(knex);
+  const rows = await knex(
+    "final_entities"
+  ).whereNotNull("final_entity").select({ kind: kindExpr, count: knex.raw("count(*)") }).groupBy(kindExpr);
+  return new Map(rows.map((row) => [String(row.kind), Number(row.count)]));
+}
+function entityRefKindExpression(knex) {
+  const client = knex.client.config.client;
+  if (client.includes("pg")) {
+    return knex.raw(`split_part(entity_ref, ':', 1)`);
+  }
+  if (client.includes("mysql")) {
+    return knex.raw(`substring_index(entity_ref, ':', 1)`);
+  }
+  return knex.raw(`substr(entity_ref, 1, instr(entity_ref, ':') - 1)`);
+}
 function initDatabaseMetrics(knex, metrics$1) {
   const seenProm = /* @__PURE__ */ new Set();
   const seen = /* @__PURE__ */ new Set();
+  const getEntitiesCountByKind = createEntitiesCountByKind(knex);
   return {
     entities_count_prom: metrics.createGaugeMetric({
       name: "catalog_entities_count",
       help: "Total amount of entities in the catalog. DEPRECATED: Please use opentelemetry metrics instead.",
       labelNames: ["kind"],
       async collect() {
-        const results = await knex("search").where("key", "=", "kind").whereNotNull("value").select({ kind: "value", count: knex.raw("count(*)") }).groupBy("value");
-        results.forEach(({ kind, count }) => {
+        const results = await getEntitiesCountByKind();
+        for (const [kind, count] of results) {
           seenProm.add(kind);
-          this.set({ kind }, Number(count));
-        });
-        seenProm.forEach((kind) => {
-          if (!results.some((r) => r.kind === kind)) {
+          this.set({ kind }, count);
+        }
+        for (const kind of seenProm) {
+          if (!results.has(kind)) {
             this.set({ kind }, 0);
             seenProm.delete(kind);
           }
-        });
+        }
       }
     }),
     registered_locations_prom: metrics.createGaugeMetric({
@@ -47,17 +89,17 @@ function initDatabaseMetrics(knex, metrics$1) {
     entities_count: metrics$1.createObservableGauge("catalog_entities_count", {
       description: "Total amount of entities in the catalog"
     }).addCallback(async (gauge) => {
-      const results = await knex("search").where("key", "=", "kind").whereNotNull("value").select({ kind: "value", count: knex.raw("count(*)") }).groupBy("value");
-      results.forEach(({ kind, count }) => {
+      const results = await getEntitiesCountByKind();
+      for (const [kind, count] of results) {
         seen.add(kind);
-        gauge.observe(Number(count), { kind });
-      });
-      seen.forEach((kind) => {
-        if (!results.some((r) => r.kind === kind)) {
+        gauge.observe(count, { kind });
+      }
+      for (const kind of seen) {
+        if (!results.has(kind)) {
           gauge.observe(0, { kind });
           seen.delete(kind);
         }
-      });
+      }
     }),
     registered_locations: metrics$1.createObservableGauge("catalog_registered_locations_count", {
       description: "Total amount of registered locations in the catalog"
@@ -96,5 +138,7 @@ function initDatabaseMetrics(knex, metrics$1) {
   };
 }
 
+exports.createEntitiesCountByKind = createEntitiesCountByKind;
 exports.initDatabaseMetrics = initDatabaseMetrics;
+exports.queryEntitiesCountByKind = queryEntitiesCountByKind;
 //# sourceMappingURL=metrics.cjs.js.map

package/dist/database/metrics.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"metrics.cjs.js","sources":["../../src/database/metrics.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Knex } from 'knex';\nimport { createGaugeMetric } from '../util/metrics';\nimport { DbRelationsRow, DbLocationsRow, DbSearchRow } from './tables';\nimport { MetricsService } from '@backstage/backend-plugin-api/alpha';\n\nexport function initDatabaseMetrics(knex: Knex, metrics: MetricsService) {\n  const seenProm = new Set<string>();\n  const seen = new Set<string>();\n\n  return {\n    entities_count_prom: createGaugeMetric({\n      name: 'catalog_entities_count',\n      help: 'Total amount of entities in the catalog. DEPRECATED: Please use opentelemetry metrics instead.',\n      labelNames: ['kind'],\n      async collect() {\n        const results = await knex<DbSearchRow>('search')\n          .where('key', '=', 'kind')\n          .whereNotNull('value')\n          .select({ kind: 'value', count: knex.raw('count(*)') })\n          .groupBy('value');\n\n        results.forEach(({ kind, count }) => {\n          seenProm.add(kind);\n          this.set({ kind }, Number(count));\n        });\n\n        // Set all the entities that were not seenProm to 0 and delete them from the seenProm set.\n        seenProm.forEach(kind => {\n          if (!results.some(r => r.kind === kind)) {\n            this.set({ kind }, 0);\n            seenProm.delete(kind);\n          }\n        });\n      },\n    }),\n    registered_locations_prom: createGaugeMetric({\n      name: 'catalog_registered_locations_count',\n      help: 'Total amount of registered locations in the catalog. DEPRECATED: Please use opentelemetry metrics instead.',\n      async collect() {\n        const total = await knex<DbLocationsRow>('locations').count({\n          count: '*',\n        });\n        this.set(Number(total[0].count));\n      },\n    }),\n    relations_prom: createGaugeMetric({\n      name: 'catalog_relations_count',\n      help: 'Total amount of relations between entities. DEPRECATED: Please use opentelemetry metrics instead.',\n      async collect() {\n        const total = await knex<DbRelationsRow>('relations').count({\n          count: '*',\n        });\n        this.set(Number(total[0].count));\n      },\n    }),\n    entities_count: metrics\n      .createObservableGauge('catalog_entities_count', {\n        description: 'Total amount of entities in the catalog',\n      })\n      .addCallback(async gauge => {\n        const results = await knex<DbSearchRow>('search')\n          .where('key', '=', 'kind')\n          .whereNotNull('value')\n          .select({ kind: 'value', count: knex.raw('count(*)') })\n          .groupBy('value');\n\n        results.forEach(({ kind, count }) => {\n          seen.add(kind);\n          gauge.observe(Number(count), { kind });\n        });\n\n        // Set all the entities that were not seen to 0 and delete them from the seen set.\n        seen.forEach(kind => {\n          if (!results.some(r => r.kind === kind)) {\n            gauge.observe(0, { kind });\n            seen.delete(kind);\n          }\n        });\n      }),\n    registered_locations: metrics\n      .createObservableGauge('catalog_registered_locations_count', {\n        description: 'Total amount of registered locations in the catalog',\n      })\n      .addCallback(async gauge => {\n        if (knex.client.config.client === 'pg') {\n          // https://stackoverflow.com/questions/7943233/fast-way-to-discover-the-row-count-of-a-table-in-postgresql\n          const total = await knex.raw(`\n            SELECT reltuples::bigint AS estimate\n            FROM   pg_class\n            WHERE  oid = 'locations'::regclass;\n          `);\n          gauge.observe(Number(total.rows[0].estimate));\n        } else {\n          const total = await knex<DbLocationsRow>('locations').count({\n            count: '*',\n          });\n          gauge.observe(Number(total[0].count));\n        }\n      }),\n    relations: metrics\n      .createObservableGauge('catalog_relations_count', {\n        description: 'Total amount of relations between entities',\n      })\n      .addCallback(async gauge => {\n        if (knex.client.config.client === 'pg') {\n          // https://stackoverflow.com/questions/7943233/fast-way-to-discover-the-row-count-of-a-table-in-postgresql\n          const total = await knex.raw(`\n            SELECT reltuples::bigint AS estimate\n            FROM   pg_class\n            WHERE  oid = 'relations'::regclass;\n          `);\n          gauge.observe(Number(total.rows[0].estimate));\n        } else {\n          const total = await knex<DbRelationsRow>('relations').count({\n            count: '*',\n          });\n          gauge.observe(Number(total[0].count));\n        }\n      }),\n  };\n}\n"],"names":["metrics","createGaugeMetric"],"mappings":";;;;AAqBO,SAAS,mBAAA,CAAoB,MAAYA,SAAA,EAAyB;AACvE,EAAA,MAAM,QAAA,uBAAe,GAAA,EAAY;AACjC,EAAA,MAAM,IAAA,uBAAW,GAAA,EAAY;AAE7B,EAAA,OAAO;AAAA,IACL,qBAAqBC,yBAAA,CAAkB;AAAA,MACrC,IAAA,EAAM,wBAAA;AAAA,MACN,IAAA,EAAM,gGAAA;AAAA,MACN,UAAA,EAAY,CAAC,MAAM,CAAA;AAAA,MACnB,MAAM,OAAA,GAAU;AACd,QAAA,MAAM,OAAA,GAAU,MAAM,IAAA,CAAkB,QAAQ,CAAA,CAC7C,MAAM,KAAA,EAAO,GAAA,EAAK,MAAM,CAAA,CACxB,YAAA,CAAa,OAAO,EACpB,MAAA,CAAO,EAAE,IAAA,EAAM,OAAA,EAAS,KAAA,EAAO,IAAA,CAAK,GAAA,CAAI,UAAU,CAAA,EAAG,CAAA,CACrD,OAAA,CAAQ,OAAO,CAAA;AAElB,QAAA,OAAA,CAAQ,OAAA,CAAQ,CAAC,EAAE,IAAA,EAAM,OAAM,KAAM;AACnC,UAAA,QAAA,CAAS,IAAI,IAAI,CAAA;AACjB,UAAA,IAAA,CAAK,IAAI,EAAE,IAAA,EAAK,EAAG,MAAA,CAAO,KAAK,CAAC,CAAA;AAAA,QAClC,CAAC,CAAA;AAGD,QAAA,QAAA,CAAS,QAAQ,CAAA,IAAA,KAAQ;AACvB,UAAA,IAAI,CAAC,OAAA,CAAQ,IAAA,CAAK,OAAK,CAAA,CAAE,IAAA,KAAS,IAAI,CAAA,EAAG;AACvC,YAAA,IAAA,CAAK,GAAA,CAAI,EAAE,IAAA,EAAK,EAAG,CAAC,CAAA;AACpB,YAAA,QAAA,CAAS,OAAO,IAAI,CAAA;AAAA,UACtB;AAAA,QACF,CAAC,CAAA;AAAA,MACH;AAAA,KACD,CAAA;AAAA,IACD,2BAA2BA,yBAAA,CAAkB;AAAA,MAC3C,IAAA,EAAM,oCAAA;AAAA,MACN,IAAA,EAAM,4GAAA;AAAA,MACN,MAAM,OAAA,GAAU;AACd,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAqB,WAAW,EAAE,KAAA,CAAM;AAAA,UAC1D,KAAA,EAAO;AAAA,SACR,CAAA;AACD,QAAA,IAAA,CAAK,IAAI,MAAA,CAAO,KAAA,CAAM,CAAC,CAAA,CAAE,KAAK,CAAC,CAAA;AAAA,MACjC;AAAA,KACD,CAAA;AAAA,IACD,gBAAgBA,yBAAA,CAAkB;AAAA,MAChC,IAAA,EAAM,yBAAA;AAAA,MACN,IAAA,EAAM,mGAAA;AAAA,MACN,MAAM,OAAA,GAAU;AACd,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAqB,WAAW,EAAE,KAAA,CAAM;AAAA,UAC1D,KAAA,EAAO;AAAA,SACR,CAAA;AACD,QAAA,IAAA,CAAK,IAAI,MAAA,CAAO,KAAA,CAAM,CAAC,CAAA,CAAE,KAAK,CAAC,CAAA;AAAA,MACjC;AAAA,KACD,CAAA;AAAA,IACD,cAAA,EAAgBD,SAAA,CACb,qBAAA,CAAsB,wBAAA,EAA0B;AAAA,MAC/C,WAAA,EAAa;AAAA,KACd,CAAA,CACA,WAAA,CAAY,OAAM,KAAA,KAAS;AAC1B,MAAA,MAAM,OAAA,GAAU,MAAM,IAAA,CAAkB,QAAQ,CAAA,CAC7C,MAAM,KAAA,EAAO,GAAA,EAAK,MAAM,CAAA,CACxB,YAAA,CAAa,OAAO,EACpB,MAAA,CAAO,EAAE,IAAA,EAAM,OAAA,EAAS,KAAA,EAAO,IAAA,CAAK,GAAA,CAAI,UAAU,CAAA,EAAG,CAAA,CACrD,OAAA,CAAQ,OAAO,CAAA;AAElB,MAAA,OAAA,CAAQ,OAAA,CAAQ,CAAC,EAAE,IAAA,EAAM,OAAM,KAAM;AACnC,QAAA,IAAA,CAAK,IAAI,IAAI,CAAA;AACb,QAAA,KAAA,CAAM,QAAQ,MAAA,CAAO,KAAK,CAAA,EAAG,EAAE,MAAM,CAAA;AAAA,MACvC,CAAC,CAAA;AAGD,MAAA,IAAA,CAAK,QAAQ,CAAA,IAAA,KAAQ;AACnB,QAAA,IAAI,CAAC,OAAA,CAAQ,IAAA,CAAK,OAAK,CAAA,CAAE,IAAA,KAAS,IAAI,CAAA,EAAG;AACvC,UAAA,KAAA,CAAM,OAAA,CAAQ,CAAA,EAAG,EAAE,IAAA,EAAM,CAAA;AACzB,UAAA,IAAA,CAAK,OAAO,IAAI,CAAA;AAAA,QAClB;AAAA,MACF,CAAC,CAAA;AAAA,IACH,CAAC,CAAA;AAAA,IACH,oBAAA,EAAsBA,SAAA,CACnB,qBAAA,CAAsB,oCAAA,EAAsC;AAAA,MAC3D,WAAA,EAAa;AAAA,KACd,CAAA,CACA,WAAA,CAAY,OAAM,KAAA,KAAS;AAC1B,MAAA,IAAI,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,MAAA,KAAW,IAAA,EAAM;AAEtC,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAK,GAAA,CAAI;AAAA;AAAA;AAAA;AAAA,UAAA,CAI5B,CAAA;AACD,QAAA,KAAA,CAAM,QAAQ,MAAA,CAAO,KAAA,CAAM,KAAK,CAAC,CAAA,CAAE,QAAQ,CAAC,CAAA;AAAA,MAC9C,CAAA,MAAO;AACL,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAqB,WAAW,EAAE,KAAA,CAAM;AAAA,UAC1D,KAAA,EAAO;AAAA,SACR,CAAA;AACD,QAAA,KAAA,CAAM,QAAQ,MAAA,CAAO,KAAA,CAAM,CAAC,CAAA,CAAE,KAAK,CAAC,CAAA;AAAA,MACtC;AAAA,IACF,CAAC,CAAA;AAAA,IACH,SAAA,EAAWA,SAAA,CACR,qBAAA,CAAsB,yBAAA,EAA2B;AAAA,MAChD,WAAA,EAAa;AAAA,KACd,CAAA,CACA,WAAA,CAAY,OAAM,KAAA,KAAS;AAC1B,MAAA,IAAI,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,MAAA,KAAW,IAAA,EAAM;AAEtC,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAK,GAAA,CAAI;AAAA;AAAA;AAAA;AAAA,UAAA,CAI5B,CAAA;AACD,QAAA,KAAA,CAAM,QAAQ,MAAA,CAAO,KAAA,CAAM,KAAK,CAAC,CAAA,CAAE,QAAQ,CAAC,CAAA;AAAA,MAC9C,CAAA,MAAO;AACL,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAqB,WAAW,EAAE,KAAA,CAAM;AAAA,UAC1D,KAAA,EAAO;AAAA,SACR,CAAA;AACD,QAAA,KAAA,CAAM,QAAQ,MAAA,CAAO,KAAA,CAAM,CAAC,CAAA,CAAE,KAAK,CAAC,CAAA;AAAA,MACtC;AAAA,IACF,CAAC;AAAA,GACL;AACF;;;;"}
+{"version":3,"file":"metrics.cjs.js","sources":["../../src/database/metrics.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Knex } from 'knex';\nimport { createGaugeMetric } from '../util/metrics';\nimport { DbRelationsRow, DbLocationsRow } from './tables';\nimport { MetricsService } from '@backstage/backend-plugin-api/alpha';\n\nconst ENTITIES_COUNT_TTL_MS = 30_000;\n\n/**\n * Returns a function that produces a Map of entity kind -> count, with\n * a single-flight cache that coalesces overlapping calls.\n *\n * The OpenTelemetry observable gauge and the legacy Prometheus gauge are\n * both registered to emit the same `catalog_entities_count` series, and\n * both fire on every metrics scrape. Without coalescing, that means two\n * identical heavy queries per scrape per pod, which can pile up against\n * the database faster than they complete.\n *\n * Concurrent callers share the same in-flight promise, so a query is\n * never overlapped by another instance of itself. The TTL is the\n * minimum age at which the next caller is allowed to start a fresh\n * query; if a query is still running when the TTL would have elapsed,\n * waiting callers continue to share that one rather than starting a\n * duplicate.\n *\n * @internal exported for testing\n */\nexport function createEntitiesCountByKind(\n  knex: Knex,\n  options?: { ttlMs?: number },\n): () => Promise<Map<string, number>> {\n  const ttlMs = options?.ttlMs ?? ENTITIES_COUNT_TTL_MS;\n  let inflight: Promise<Map<string, number>> | undefined;\n  let cached: { at: number; data: Map<string, number> } | undefined;\n  return () => {\n    if (inflight) {\n      return inflight;\n    }\n    if (cached && Date.now() - cached.at < ttlMs) {\n      return Promise.resolve(cached.data);\n    }\n    inflight = (async () => {\n      try {\n        const data = await queryEntitiesCountByKind(knex);\n        cached = { at: Date.now(), data };\n        return data;\n      } finally {\n        inflight = undefined;\n      }\n    })();\n    return inflight;\n  };\n}\n\n/**\n * Reads kind counts straight from `final_entities` (one row per entity)\n * rather than from `search` (one row per entity per indexed key, often\n * 20-30x larger). The kind is parsed out of `entity_ref`, which is the\n * canonical lowercased `kind:namespace/name` form -- producing the same\n * lowercased labels the previous search-based query did.\n *\n * Rows where `final_entity` is null are excluded. They represent entities\n * that haven't been stitched yet, or tombstones for in-progress deletions.\n * Counting them would over-report by including entities the rest of the\n * catalog API treats as not present.\n *\n * @internal exported for testing\n */\nexport async function queryEntitiesCountByKind(\n  knex: Knex,\n): Promise<Map<string, number>> {\n  const kindExpr = entityRefKindExpression(knex);\n\n  const rows: { kind: string; count: string | number }[] = await knex(\n    'final_entities',\n  )\n    .whereNotNull('final_entity')\n    .select({ kind: kindExpr, count: knex.raw('count(*)') })\n    .groupBy(kindExpr);\n\n  return new Map(rows.map(row => [String(row.kind), Number(row.count)]));\n}\n\nfunction entityRefKindExpression(knex: Knex): Knex.Raw {\n  const client = knex.client.config.client as string;\n  if (client.includes('pg')) {\n    return knex.raw(`split_part(entity_ref, ':', 1)`);\n  }\n  if (client.includes('mysql')) {\n    return knex.raw(`substring_index(entity_ref, ':', 1)`);\n  }\n  // sqlite (better-sqlite3, sqlite3)\n  return knex.raw(`substr(entity_ref, 1, instr(entity_ref, ':') - 1)`);\n}\n\nexport function initDatabaseMetrics(knex: Knex, metrics: MetricsService) {\n  const seenProm = new Set<string>();\n  const seen = new Set<string>();\n  const getEntitiesCountByKind = createEntitiesCountByKind(knex);\n\n  return {\n    entities_count_prom: createGaugeMetric({\n      name: 'catalog_entities_count',\n      help: 'Total amount of entities in the catalog. DEPRECATED: Please use opentelemetry metrics instead.',\n      labelNames: ['kind'],\n      async collect() {\n        const results = await getEntitiesCountByKind();\n\n        for (const [kind, count] of results) {\n          seenProm.add(kind);\n          this.set({ kind }, count);\n        }\n\n        // Set all the entities that were not seenProm to 0 and delete them from the seenProm set.\n        for (const kind of seenProm) {\n          if (!results.has(kind)) {\n            this.set({ kind }, 0);\n            seenProm.delete(kind);\n          }\n        }\n      },\n    }),\n    registered_locations_prom: createGaugeMetric({\n      name: 'catalog_registered_locations_count',\n      help: 'Total amount of registered locations in the catalog. DEPRECATED: Please use opentelemetry metrics instead.',\n      async collect() {\n        const total = await knex<DbLocationsRow>('locations').count({\n          count: '*',\n        });\n        this.set(Number(total[0].count));\n      },\n    }),\n    relations_prom: createGaugeMetric({\n      name: 'catalog_relations_count',\n      help: 'Total amount of relations between entities. DEPRECATED: Please use opentelemetry metrics instead.',\n      async collect() {\n        const total = await knex<DbRelationsRow>('relations').count({\n          count: '*',\n        });\n        this.set(Number(total[0].count));\n      },\n    }),\n    entities_count: metrics\n      .createObservableGauge('catalog_entities_count', {\n        description: 'Total amount of entities in the catalog',\n      })\n      .addCallback(async gauge => {\n        const results = await getEntitiesCountByKind();\n\n        for (const [kind, count] of results) {\n          seen.add(kind);\n          gauge.observe(count, { kind });\n        }\n\n        // Set all the entities that were not seen to 0 and delete them from the seen set.\n        for (const kind of seen) {\n          if (!results.has(kind)) {\n            gauge.observe(0, { kind });\n            seen.delete(kind);\n          }\n        }\n      }),\n    registered_locations: metrics\n      .createObservableGauge('catalog_registered_locations_count', {\n        description: 'Total amount of registered locations in the catalog',\n      })\n      .addCallback(async gauge => {\n        if (knex.client.config.client === 'pg') {\n          // https://stackoverflow.com/questions/7943233/fast-way-to-discover-the-row-count-of-a-table-in-postgresql\n          const total = await knex.raw(`\n            SELECT reltuples::bigint AS estimate\n            FROM   pg_class\n            WHERE  oid = 'locations'::regclass;\n          `);\n          gauge.observe(Number(total.rows[0].estimate));\n        } else {\n          const total = await knex<DbLocationsRow>('locations').count({\n            count: '*',\n          });\n          gauge.observe(Number(total[0].count));\n        }\n      }),\n    relations: metrics\n      .createObservableGauge('catalog_relations_count', {\n        description: 'Total amount of relations between entities',\n      })\n      .addCallback(async gauge => {\n        if (knex.client.config.client === 'pg') {\n          // https://stackoverflow.com/questions/7943233/fast-way-to-discover-the-row-count-of-a-table-in-postgresql\n          const total = await knex.raw(`\n            SELECT reltuples::bigint AS estimate\n            FROM   pg_class\n            WHERE  oid = 'relations'::regclass;\n          `);\n          gauge.observe(Number(total.rows[0].estimate));\n        } else {\n          const total = await knex<DbRelationsRow>('relations').count({\n            count: '*',\n          });\n          gauge.observe(Number(total[0].count));\n        }\n      }),\n  };\n}\n"],"names":["metrics","createGaugeMetric"],"mappings":";;;;AAqBA,MAAM,qBAAA,GAAwB,GAAA;AAqBvB,SAAS,yBAAA,CACd,MACA,OAAA,EACoC;AACpC,EAAA,MAAM,KAAA,GAA0B,qBAAA;AAChC,EAAA,IAAI,QAAA;AACJ,EAAA,IAAI,MAAA;AACJ,EAAA,OAAO,MAAM;AACX,IAAA,IAAI,QAAA,EAAU;AACZ,MAAA,OAAO,QAAA;AAAA,IACT;AACA,IAAA,IAAI,UAAU,IAAA,CAAK,GAAA,EAAI,GAAI,MAAA,CAAO,KAAK,KAAA,EAAO;AAC5C,MAAA,OAAO,OAAA,CAAQ,OAAA,CAAQ,MAAA,CAAO,IAAI,CAAA;AAAA,IACpC;AACA,IAAA,QAAA,GAAA,CAAY,YAAY;AACtB,MAAA,IAAI;AACF,QAAA,MAAM,IAAA,GAAO,MAAM,wBAAA,CAAyB,IAAI,CAAA;AAChD,QAAA,MAAA,GAAS,EAAE,EAAA,EAAI,IAAA,CAAK,GAAA,IAAO,IAAA,EAAK;AAChC,QAAA,OAAO,IAAA;AAAA,MACT,CAAA,SAAE;AACA,QAAA,QAAA,GAAW,MAAA;AAAA,MACb;AAAA,IACF,CAAA,GAAG;AACH,IAAA,OAAO,QAAA;AAAA,EACT,CAAA;AACF;AAgBA,eAAsB,yBACpB,IAAA,EAC8B;AAC9B,EAAA,MAAM,QAAA,GAAW,wBAAwB,IAAI,CAAA;AAE7C,EAAA,MAAM,OAAmD,MAAM,IAAA;AAAA,IAC7D;AAAA,IAEC,YAAA,CAAa,cAAc,CAAA,CAC3B,MAAA,CAAO,EAAE,IAAA,EAAM,QAAA,EAAU,KAAA,EAAO,IAAA,CAAK,IAAI,UAAU,CAAA,EAAG,CAAA,CACtD,QAAQ,QAAQ,CAAA;AAEnB,EAAA,OAAO,IAAI,GAAA,CAAI,IAAA,CAAK,GAAA,CAAI,SAAO,CAAC,MAAA,CAAO,GAAA,CAAI,IAAI,GAAG,MAAA,CAAO,GAAA,CAAI,KAAK,CAAC,CAAC,CAAC,CAAA;AACvE;AAEA,SAAS,wBAAwB,IAAA,EAAsB;AACrD,EAAA,MAAM,MAAA,GAAS,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,MAAA;AAClC,EAAA,IAAI,MAAA,CAAO,QAAA,CAAS,IAAI,CAAA,EAAG;AACzB,IAAA,OAAO,IAAA,CAAK,IAAI,CAAA,8BAAA,CAAgC,CAAA;AAAA,EAClD;AACA,EAAA,IAAI,MAAA,CAAO,QAAA,CAAS,OAAO,CAAA,EAAG;AAC5B,IAAA,OAAO,IAAA,CAAK,IAAI,CAAA,mCAAA,CAAqC,CAAA;AAAA,EACvD;AAEA,EAAA,OAAO,IAAA,CAAK,IAAI,CAAA,iDAAA,CAAmD,CAAA;AACrE;AAEO,SAAS,mBAAA,CAAoB,MAAYA,SAAA,EAAyB;AACvE,EAAA,MAAM,QAAA,uBAAe,GAAA,EAAY;AACjC,EAAA,MAAM,IAAA,uBAAW,GAAA,EAAY;AAC7B,EAAA,MAAM,sBAAA,GAAyB,0BAA0B,IAAI,CAAA;AAE7D,EAAA,OAAO;AAAA,IACL,qBAAqBC,yBAAA,CAAkB;AAAA,MACrC,IAAA,EAAM,wBAAA;AAAA,MACN,IAAA,EAAM,gGAAA;AAAA,MACN,UAAA,EAAY,CAAC,MAAM,CAAA;AAAA,MACnB,MAAM,OAAA,GAAU;AACd,QAAA,MAAM,OAAA,GAAU,MAAM,sBAAA,EAAuB;AAE7C,QAAA,KAAA,MAAW,CAAC,IAAA,EAAM,KAAK,CAAA,IAAK,OAAA,EAAS;AACnC,UAAA,QAAA,CAAS,IAAI,IAAI,CAAA;AACjB,UAAA,IAAA,CAAK,GAAA,CAAI,EAAE,IAAA,EAAK,EAAG,KAAK,CAAA;AAAA,QAC1B;AAGA,QAAA,KAAA,MAAW,QAAQ,QAAA,EAAU;AAC3B,UAAA,IAAI,CAAC,OAAA,CAAQ,GAAA,CAAI,IAAI,CAAA,EAAG;AACtB,YAAA,IAAA,CAAK,GAAA,CAAI,EAAE,IAAA,EAAK,EAAG,CAAC,CAAA;AACpB,YAAA,QAAA,CAAS,OAAO,IAAI,CAAA;AAAA,UACtB;AAAA,QACF;AAAA,MACF;AAAA,KACD,CAAA;AAAA,IACD,2BAA2BA,yBAAA,CAAkB;AAAA,MAC3C,IAAA,EAAM,oCAAA;AAAA,MACN,IAAA,EAAM,4GAAA;AAAA,MACN,MAAM,OAAA,GAAU;AACd,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAqB,WAAW,EAAE,KAAA,CAAM;AAAA,UAC1D,KAAA,EAAO;AAAA,SACR,CAAA;AACD,QAAA,IAAA,CAAK,IAAI,MAAA,CAAO,KAAA,CAAM,CAAC,CAAA,CAAE,KAAK,CAAC,CAAA;AAAA,MACjC;AAAA,KACD,CAAA;AAAA,IACD,gBAAgBA,yBAAA,CAAkB;AAAA,MAChC,IAAA,EAAM,yBAAA;AAAA,MACN,IAAA,EAAM,mGAAA;AAAA,MACN,MAAM,OAAA,GAAU;AACd,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAqB,WAAW,EAAE,KAAA,CAAM;AAAA,UAC1D,KAAA,EAAO;AAAA,SACR,CAAA;AACD,QAAA,IAAA,CAAK,IAAI,MAAA,CAAO,KAAA,CAAM,CAAC,CAAA,CAAE,KAAK,CAAC,CAAA;AAAA,MACjC;AAAA,KACD,CAAA;AAAA,IACD,cAAA,EAAgBD,SAAA,CACb,qBAAA,CAAsB,wBAAA,EAA0B;AAAA,MAC/C,WAAA,EAAa;AAAA,KACd,CAAA,CACA,WAAA,CAAY,OAAM,KAAA,KAAS;AAC1B,MAAA,MAAM,OAAA,GAAU,MAAM,sBAAA,EAAuB;AAE7C,MAAA,KAAA,MAAW,CAAC,IAAA,EAAM,KAAK,CAAA,IAAK,OAAA,EAAS;AACnC,QAAA,IAAA,CAAK,IAAI,IAAI,CAAA;AACb,QAAA,KAAA,CAAM,OAAA,CAAQ,KAAA,EAAO,EAAE,IAAA,EAAM,CAAA;AAAA,MAC/B;AAGA,MAAA,KAAA,MAAW,QAAQ,IAAA,EAAM;AACvB,QAAA,IAAI,CAAC,OAAA,CAAQ,GAAA,CAAI,IAAI,CAAA,EAAG;AACtB,UAAA,KAAA,CAAM,OAAA,CAAQ,CAAA,EAAG,EAAE,IAAA,EAAM,CAAA;AACzB,UAAA,IAAA,CAAK,OAAO,IAAI,CAAA;AAAA,QAClB;AAAA,MACF;AAAA,IACF,CAAC,CAAA;AAAA,IACH,oBAAA,EAAsBA,SAAA,CACnB,qBAAA,CAAsB,oCAAA,EAAsC;AAAA,MAC3D,WAAA,EAAa;AAAA,KACd,CAAA,CACA,WAAA,CAAY,OAAM,KAAA,KAAS;AAC1B,MAAA,IAAI,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,MAAA,KAAW,IAAA,EAAM;AAEtC,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAK,GAAA,CAAI;AAAA;AAAA;AAAA;AAAA,UAAA,CAI5B,CAAA;AACD,QAAA,KAAA,CAAM,QAAQ,MAAA,CAAO,KAAA,CAAM,KAAK,CAAC,CAAA,CAAE,QAAQ,CAAC,CAAA;AAAA,MAC9C,CAAA,MAAO;AACL,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAqB,WAAW,EAAE,KAAA,CAAM;AAAA,UAC1D,KAAA,EAAO;AAAA,SACR,CAAA;AACD,QAAA,KAAA,CAAM,QAAQ,MAAA,CAAO,KAAA,CAAM,CAAC,CAAA,CAAE,KAAK,CAAC,CAAA;AAAA,MACtC;AAAA,IACF,CAAC,CAAA;AAAA,IACH,SAAA,EAAWA,SAAA,CACR,qBAAA,CAAsB,yBAAA,EAA2B;AAAA,MAChD,WAAA,EAAa;AAAA,KACd,CAAA,CACA,WAAA,CAAY,OAAM,KAAA,KAAS;AAC1B,MAAA,IAAI,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,MAAA,KAAW,IAAA,EAAM;AAEtC,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAK,GAAA,CAAI;AAAA;AAAA;AAAA;AAAA,UAAA,CAI5B,CAAA;AACD,QAAA,KAAA,CAAM,QAAQ,MAAA,CAAO,KAAA,CAAM,KAAK,CAAC,CAAA,CAAE,QAAQ,CAAC,CAAA;AAAA,MAC9C,CAAA,MAAO;AACL,QAAA,MAAM,KAAA,GAAQ,MAAM,IAAA,CAAqB,WAAW,EAAE,KAAA,CAAM;AAAA,UAC1D,KAAA,EAAO;AAAA,SACR,CAAA;AACD,QAAA,KAAA,CAAM,QAAQ,MAAA,CAAO,KAAA,CAAM,CAAC,CAAA,CAAE,KAAK,CAAC,CAAA;AAAA,MACtC;AAAA,IACF,CAAC;AAAA,GACL;AACF;;;;;;"}

package/dist/database/operations/stitcher/buildEntitySearch.cjs.js

@@ -2,6 +2,7 @@
 
 var catalogModel = require('@backstage/catalog-model');
 var errors = require('@backstage/errors');
+var util = require('./util.cjs.js');
 
 const SPECIAL_KEYS = [
   "attachments",
@@ -113,7 +114,14 @@ function buildEntitySearch(entityId, entity) {
       `Entity has duplicate keys that vary only in casing, ${badKeys}`
     );
   }
-  return mapToRows(raw, entityId);
+  const rows = mapToRows(raw, entityId);
+  const seen = /* @__PURE__ */ new Set();
+  return rows.filter((row) => {
+    const k = `${row.key}\0${row.value === null ? util.NULL_SENTINEL : row.value}`;
+    if (seen.has(k)) return false;
+    seen.add(k);
+    return true;
+  });
 }
 
 exports.buildEntitySearch = buildEntitySearch;

package/dist/database/operations/stitcher/buildEntitySearch.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"buildEntitySearch.cjs.js","sources":["../../../../src/database/operations/stitcher/buildEntitySearch.ts"],"sourcesContent":["/*\n * Copyright 2020 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { DEFAULT_NAMESPACE, Entity } from '@backstage/catalog-model';\nimport { InputError } from '@backstage/errors';\nimport { DbSearchRow } from '../../tables';\n\n// These are excluded in the generic loop, either because they do not make sense\n// to index, or because they are special-case always inserted whether they are\n// null or not\nconst SPECIAL_KEYS = [\n  'attachments',\n  'relations',\n  'status',\n  'metadata.name',\n  'metadata.namespace',\n  'metadata.uid',\n  'metadata.etag',\n];\n\n// The maximum length allowed for search values. These columns are indexed, and\n// database engines do not like to index on massive values. For example,\n// postgres will balk after 8191 byte line sizes.\nconst MAX_KEY_LENGTH = 200;\nconst MAX_VALUE_LENGTH = 200;\n\ntype Kv = {\n  key: string;\n  value: unknown;\n};\n\n// Helper for traversing through a nested structure and outputting a list of\n// path->value entries of the leaves.\n//\n// For example, this yaml structure\n//\n// a: 1\n// b:\n//   c: null\n//   e: [f, g]\n// h:\n//  - i: 1\n//    j: k\n//  - i: 2\n//    j: l\n//\n// will result in\n//\n// \"a\", 1\n// \"b.c\", null\n// \"b.e\": \"f\"\n// \"b.e.f\": true\n// \"b.e\": \"g\"\n// \"b.e.g\": true\n// \"h.i\": 1\n// \"h.j\": \"k\"\n// \"h.i\": 2\n// \"h.j\": \"l\"\nexport function traverse(root: unknown): Kv[] {\n  const output: Kv[] = [];\n  // Use a Set for O(1) case-insensitive duplicate detection of synthetic\n  // boolean path keys (e.g. \"metadata.tags.java\"), instead of the previous\n  // O(n) Array.some() linear scan which caused O(n²) overall complexity\n  // and severe event loop blocking for entities with large arrays.\n  const seenPathKeys = new Set<string>();\n\n  function visit(path: string, current: unknown) {\n    if (SPECIAL_KEYS.includes(path)) {\n      return;\n    }\n\n    // empty or scalar\n    if (\n      current === undefined ||\n      current === null ||\n      ['string', 'number', 'boolean'].includes(typeof current)\n    ) {\n      output.push({ key: path, value: current });\n      return;\n    }\n\n    // unknown\n    if (typeof current !== 'object') {\n      return;\n    }\n\n    // array\n    if (Array.isArray(current)) {\n      for (const item of current) {\n        // NOTE(freben): The reason that these are output in two different ways,\n        // is to support use cases where you want to express that MORE than one\n        // tag is present in a list. Since the EntityFilters structure is a\n        // record, you can't have several entries of the same key. Therefore\n        // you will have to match on\n        //\n        // { \"a.b\": [\"true\"], \"a.c\": [\"true\"] }\n        //\n        // rather than\n        //\n        // { \"a\": [\"b\", \"c\"] }\n        //\n        // because the latter means EITHER b or c has to be present.\n        visit(path, item);\n        if (typeof item === 'string') {\n          const pathKey = `${path}.${item}`;\n          const lowerKey = pathKey.toLocaleLowerCase('en-US');\n          if (!seenPathKeys.has(lowerKey)) {\n            seenPathKeys.add(lowerKey);\n            output.push({ key: pathKey, value: true });\n          }\n        }\n      }\n      return;\n    }\n\n    // object\n    for (const [key, value] of Object.entries(current!)) {\n      visit(path ? `${path}.${key}` : key, value);\n    }\n  }\n\n  visit('', root);\n\n  return output;\n}\n\n// Translates a number of raw data rows to search table rows\nexport function mapToRows(input: Kv[], entityId: string): DbSearchRow[] {\n  const result: DbSearchRow[] = [];\n\n  for (const { key: rawKey, value: rawValue } of input) {\n    const key = rawKey.toLocaleLowerCase('en-US');\n    if (key.length > MAX_KEY_LENGTH) {\n      continue;\n    }\n    if (rawValue === undefined || rawValue === null) {\n      result.push({\n        entity_id: entityId,\n        key,\n        original_value: null,\n        value: null,\n      });\n    } else {\n      const value = String(rawValue).toLocaleLowerCase('en-US');\n      if (value.length <= MAX_VALUE_LENGTH) {\n        result.push({\n          entity_id: entityId,\n          key,\n          original_value: String(rawValue),\n          value: value,\n        });\n      } else {\n        result.push({\n          entity_id: entityId,\n          key,\n          original_value: null,\n          value: null,\n        });\n      }\n    }\n  }\n\n  return result;\n}\n\n/**\n * Generates all of the search rows that are relevant for this entity.\n *\n * @param entityId - The uid of the entity\n * @param entity - The entity\n * @returns A list of entity search rows\n */\nexport function buildEntitySearch(\n  entityId: string,\n  entity: Entity,\n): DbSearchRow[] {\n  // Visit the base structure recursively\n  const raw = traverse(entity);\n\n  // Start with some special keys that are always present because you want to\n  // be able to easily search for null specifically\n  raw.push({ key: 'metadata.name', value: entity.metadata.name });\n  raw.push({ key: 'metadata.namespace', value: entity.metadata.namespace });\n  raw.push({ key: 'metadata.uid', value: entity.metadata.uid });\n\n  // Namespace not specified has the default value \"default\", so we want to\n  // match on that as well\n  if (!entity.metadata.namespace) {\n    raw.push({ key: 'metadata.namespace', value: DEFAULT_NAMESPACE });\n  }\n\n  // Visit relations\n  for (const relation of entity.relations ?? []) {\n    raw.push({\n      key: `relations.${relation.type}`,\n      value: relation.targetRef,\n    });\n  }\n\n  // This validates that there are no keys that vary only in casing, such\n  // as `spec.foo` and `spec.Foo`.\n  const keys = new Set(raw.map(r => r.key));\n  const lowerKeys = new Set(raw.map(r => r.key.toLocaleLowerCase('en-US')));\n  if (keys.size !== lowerKeys.size) {\n    const difference = [];\n    for (const key of keys) {\n      const lower = key.toLocaleLowerCase('en-US');\n      if (!lowerKeys.delete(lower)) {\n        difference.push(lower);\n      }\n    }\n    const badKeys = `'${difference.join(\"', '\")}'`;\n    throw new InputError(\n      `Entity has duplicate keys that vary only in casing, ${badKeys}`,\n    );\n  }\n\n  return mapToRows(raw, entityId);\n}\n"],"names":["DEFAULT_NAMESPACE","InputError"],"mappings":";;;;;AAuBA,MAAM,YAAA,GAAe;AAAA,EACnB,aAAA;AAAA,EACA,WAAA;AAAA,EACA,QAAA;AAAA,EACA,eAAA;AAAA,EACA,oBAAA;AAAA,EACA,cAAA;AAAA,EACA;AACF,CAAA;AAKA,MAAM,cAAA,GAAiB,GAAA;AACvB,MAAM,gBAAA,GAAmB,GAAA;AAkClB,SAAS,SAAS,IAAA,EAAqB;AAC5C,EAAA,MAAM,SAAe,EAAC;AAKtB,EAAA,MAAM,YAAA,uBAAmB,GAAA,EAAY;AAErC,EAAA,SAAS,KAAA,CAAM,MAAc,OAAA,EAAkB;AAC7C,IAAA,IAAI,YAAA,CAAa,QAAA,CAAS,IAAI,CAAA,EAAG;AAC/B,MAAA;AAAA,IACF;AAGA,IAAA,IACE,OAAA,KAAY,MAAA,IACZ,OAAA,KAAY,IAAA,IACZ,CAAC,QAAA,EAAU,QAAA,EAAU,SAAS,CAAA,CAAE,QAAA,CAAS,OAAO,OAAO,CAAA,EACvD;AACA,MAAA,MAAA,CAAO,KAAK,EAAE,GAAA,EAAK,IAAA,EAAM,KAAA,EAAO,SAAS,CAAA;AACzC,MAAA;AAAA,IACF;AAGA,IAAA,IAAI,OAAO,YAAY,QAAA,EAAU;AAC/B,MAAA;AAAA,IACF;AAGA,IAAA,IAAI,KAAA,CAAM,OAAA,CAAQ,OAAO,CAAA,EAAG;AAC1B,MAAA,KAAA,MAAW,QAAQ,OAAA,EAAS;AAc1B,QAAA,KAAA,CAAM,MAAM,IAAI,CAAA;AAChB,QAAA,IAAI,OAAO,SAAS,QAAA,EAAU;AAC5B,UAAA,MAAM,OAAA,GAAU,CAAA,EAAG,IAAI,CAAA,CAAA,EAAI,IAAI,CAAA,CAAA;AAC/B,UAAA,MAAM,QAAA,GAAW,OAAA,CAAQ,iBAAA,CAAkB,OAAO,CAAA;AAClD,UAAA,IAAI,CAAC,YAAA,CAAa,GAAA,CAAI,QAAQ,CAAA,EAAG;AAC/B,YAAA,YAAA,CAAa,IAAI,QAAQ,CAAA;AACzB,YAAA,MAAA,CAAO,KAAK,EAAE,GAAA,EAAK,OAAA,EAAS,KAAA,EAAO,MAAM,CAAA;AAAA,UAC3C;AAAA,QACF;AAAA,MACF;AACA,MAAA;AAAA,IACF;AAGA,IAAA,KAAA,MAAW,CAAC,GAAA,EAAK,KAAK,KAAK,MAAA,CAAO,OAAA,CAAQ,OAAQ,CAAA,EAAG;AACnD,MAAA,KAAA,CAAM,OAAO,CAAA,EAAG,IAAI,IAAI,GAAG,CAAA,CAAA,GAAK,KAAK,KAAK,CAAA;AAAA,IAC5C;AAAA,EACF;AAEA,EAAA,KAAA,CAAM,IAAI,IAAI,CAAA;AAEd,EAAA,OAAO,MAAA;AACT;AAGO,SAAS,SAAA,CAAU,OAAa,QAAA,EAAiC;AACtE,EAAA,MAAM,SAAwB,EAAC;AAE/B,EAAA,KAAA,MAAW,EAAE,GAAA,EAAK,MAAA,EAAQ,KAAA,EAAO,QAAA,MAAc,KAAA,EAAO;AACpD,IAAA,MAAM,GAAA,GAAM,MAAA,CAAO,iBAAA,CAAkB,OAAO,CAAA;AAC5C,IAAA,IAAI,GAAA,CAAI,SAAS,cAAA,EAAgB;AAC/B,MAAA;AAAA,IACF;AACA,IAAA,IAAI,QAAA,KAAa,MAAA,IAAa,QAAA,KAAa,IAAA,EAAM;AAC/C,MAAA,MAAA,CAAO,IAAA,CAAK;AAAA,QACV,SAAA,EAAW,QAAA;AAAA,QACX,GAAA;AAAA,QACA,cAAA,EAAgB,IAAA;AAAA,QAChB,KAAA,EAAO;AAAA,OACR,CAAA;AAAA,IACH,CAAA,MAAO;AACL,MAAA,MAAM,KAAA,GAAQ,MAAA,CAAO,QAAQ,CAAA,CAAE,kBAAkB,OAAO,CAAA;AACxD,MAAA,IAAI,KAAA,CAAM,UAAU,gBAAA,EAAkB;AACpC,QAAA,MAAA,CAAO,IAAA,CAAK;AAAA,UACV,SAAA,EAAW,QAAA;AAAA,UACX,GAAA;AAAA,UACA,cAAA,EAAgB,OAAO,QAAQ,CAAA;AAAA,UAC/B;AAAA,SACD,CAAA;AAAA,MACH,CAAA,MAAO;AACL,QAAA,MAAA,CAAO,IAAA,CAAK;AAAA,UACV,SAAA,EAAW,QAAA;AAAA,UACX,GAAA;AAAA,UACA,cAAA,EAAgB,IAAA;AAAA,UAChB,KAAA,EAAO;AAAA,SACR,CAAA;AAAA,MACH;AAAA,IACF;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AASO,SAAS,iBAAA,CACd,UACA,MAAA,EACe;AAEf,EAAA,MAAM,GAAA,GAAM,SAAS,MAAM,CAAA;AAI3B,EAAA,GAAA,CAAI,IAAA,CAAK,EAAE,GAAA,EAAK,eAAA,EAAiB,OAAO,MAAA,CAAO,QAAA,CAAS,MAAM,CAAA;AAC9D,EAAA,GAAA,CAAI,IAAA,CAAK,EAAE,GAAA,EAAK,oBAAA,EAAsB,OAAO,MAAA,CAAO,QAAA,CAAS,WAAW,CAAA;AACxE,EAAA,GAAA,CAAI,IAAA,CAAK,EAAE,GAAA,EAAK,cAAA,EAAgB,OAAO,MAAA,CAAO,QAAA,CAAS,KAAK,CAAA;AAI5D,EAAA,IAAI,CAAC,MAAA,CAAO,QAAA,CAAS,SAAA,EAAW;AAC9B,IAAA,GAAA,CAAI,KAAK,EAAE,GAAA,EAAK,oBAAA,EAAsB,KAAA,EAAOA,gCAAmB,CAAA;AAAA,EAClE;AAGA,EAAA,KAAA,MAAW,QAAA,IAAY,MAAA,CAAO,SAAA,IAAa,EAAC,EAAG;AAC7C,IAAA,GAAA,CAAI,IAAA,CAAK;AAAA,MACP,GAAA,EAAK,CAAA,UAAA,EAAa,QAAA,CAAS,IAAI,CAAA,CAAA;AAAA,MAC/B,OAAO,QAAA,CAAS;AAAA,KACjB,CAAA;AAAA,EACH;AAIA,EAAA,MAAM,IAAA,GAAO,IAAI,GAAA,CAAI,GAAA,CAAI,IAAI,CAAA,CAAA,KAAK,CAAA,CAAE,GAAG,CAAC,CAAA;AACxC,EAAA,MAAM,SAAA,GAAY,IAAI,GAAA,CAAI,GAAA,CAAI,GAAA,CAAI,CAAA,CAAA,KAAK,CAAA,CAAE,GAAA,CAAI,iBAAA,CAAkB,OAAO,CAAC,CAAC,CAAA;AACxE,EAAA,IAAI,IAAA,CAAK,IAAA,KAAS,SAAA,CAAU,IAAA,EAAM;AAChC,IAAA,MAAM,aAAa,EAAC;AACpB,IAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,MAAA,MAAM,KAAA,GAAQ,GAAA,CAAI,iBAAA,CAAkB,OAAO,CAAA;AAC3C,MAAA,IAAI,CAAC,SAAA,CAAU,MAAA,CAAO,KAAK,CAAA,EAAG;AAC5B,QAAA,UAAA,CAAW,KAAK,KAAK,CAAA;AAAA,MACvB;AAAA,IACF;AACA,IAAA,MAAM,OAAA,GAAU,CAAA,CAAA,EAAI,UAAA,CAAW,IAAA,CAAK,MAAM,CAAC,CAAA,CAAA,CAAA;AAC3C,IAAA,MAAM,IAAIC,iBAAA;AAAA,MACR,uDAAuD,OAAO,CAAA;AAAA,KAChE;AAAA,EACF;AAEA,EAAA,OAAO,SAAA,CAAU,KAAK,QAAQ,CAAA;AAChC;;;;;;"}
+{"version":3,"file":"buildEntitySearch.cjs.js","sources":["../../../../src/database/operations/stitcher/buildEntitySearch.ts"],"sourcesContent":["/*\n * Copyright 2020 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { DEFAULT_NAMESPACE, Entity } from '@backstage/catalog-model';\nimport { InputError } from '@backstage/errors';\nimport { DbSearchRow } from '../../tables';\nimport { NULL_SENTINEL } from './util';\n\n// These are excluded in the generic loop, either because they do not make sense\n// to index, or because they are special-case always inserted whether they are\n// null or not\nconst SPECIAL_KEYS = [\n  'attachments',\n  'relations',\n  'status',\n  'metadata.name',\n  'metadata.namespace',\n  'metadata.uid',\n  'metadata.etag',\n];\n\n// The maximum length allowed for search values. These columns are indexed, and\n// database engines do not like to index on massive values. For example,\n// postgres will balk after 8191 byte line sizes.\nconst MAX_KEY_LENGTH = 200;\nconst MAX_VALUE_LENGTH = 200;\n\ntype Kv = {\n  key: string;\n  value: unknown;\n};\n\n// Helper for traversing through a nested structure and outputting a list of\n// path->value entries of the leaves.\n//\n// For example, this yaml structure\n//\n// a: 1\n// b:\n//   c: null\n//   e: [f, g]\n// h:\n//  - i: 1\n//    j: k\n//  - i: 2\n//    j: l\n//\n// will result in\n//\n// \"a\", 1\n// \"b.c\", null\n// \"b.e\": \"f\"\n// \"b.e.f\": true\n// \"b.e\": \"g\"\n// \"b.e.g\": true\n// \"h.i\": 1\n// \"h.j\": \"k\"\n// \"h.i\": 2\n// \"h.j\": \"l\"\nexport function traverse(root: unknown): Kv[] {\n  const output: Kv[] = [];\n  // Use a Set for O(1) case-insensitive duplicate detection of synthetic\n  // boolean path keys (e.g. \"metadata.tags.java\"), instead of the previous\n  // O(n) Array.some() linear scan which caused O(n²) overall complexity\n  // and severe event loop blocking for entities with large arrays.\n  const seenPathKeys = new Set<string>();\n\n  function visit(path: string, current: unknown) {\n    if (SPECIAL_KEYS.includes(path)) {\n      return;\n    }\n\n    // empty or scalar\n    if (\n      current === undefined ||\n      current === null ||\n      ['string', 'number', 'boolean'].includes(typeof current)\n    ) {\n      output.push({ key: path, value: current });\n      return;\n    }\n\n    // unknown\n    if (typeof current !== 'object') {\n      return;\n    }\n\n    // array\n    if (Array.isArray(current)) {\n      for (const item of current) {\n        // NOTE(freben): The reason that these are output in two different ways,\n        // is to support use cases where you want to express that MORE than one\n        // tag is present in a list. Since the EntityFilters structure is a\n        // record, you can't have several entries of the same key. Therefore\n        // you will have to match on\n        //\n        // { \"a.b\": [\"true\"], \"a.c\": [\"true\"] }\n        //\n        // rather than\n        //\n        // { \"a\": [\"b\", \"c\"] }\n        //\n        // because the latter means EITHER b or c has to be present.\n        visit(path, item);\n        if (typeof item === 'string') {\n          const pathKey = `${path}.${item}`;\n          const lowerKey = pathKey.toLocaleLowerCase('en-US');\n          if (!seenPathKeys.has(lowerKey)) {\n            seenPathKeys.add(lowerKey);\n            output.push({ key: pathKey, value: true });\n          }\n        }\n      }\n      return;\n    }\n\n    // object\n    for (const [key, value] of Object.entries(current!)) {\n      visit(path ? `${path}.${key}` : key, value);\n    }\n  }\n\n  visit('', root);\n\n  return output;\n}\n\n// Translates a number of raw data rows to search table rows\nexport function mapToRows(input: Kv[], entityId: string): DbSearchRow[] {\n  const result: DbSearchRow[] = [];\n\n  for (const { key: rawKey, value: rawValue } of input) {\n    const key = rawKey.toLocaleLowerCase('en-US');\n    if (key.length > MAX_KEY_LENGTH) {\n      continue;\n    }\n    if (rawValue === undefined || rawValue === null) {\n      result.push({\n        entity_id: entityId,\n        key,\n        original_value: null,\n        value: null,\n      });\n    } else {\n      const value = String(rawValue).toLocaleLowerCase('en-US');\n      if (value.length <= MAX_VALUE_LENGTH) {\n        result.push({\n          entity_id: entityId,\n          key,\n          original_value: String(rawValue),\n          value: value,\n        });\n      } else {\n        result.push({\n          entity_id: entityId,\n          key,\n          original_value: null,\n          value: null,\n        });\n      }\n    }\n  }\n\n  return result;\n}\n\n/**\n * Generates all of the search rows that are relevant for this entity.\n *\n * @param entityId - The uid of the entity\n * @param entity - The entity\n * @returns A list of entity search rows\n */\nexport function buildEntitySearch(\n  entityId: string,\n  entity: Entity,\n): DbSearchRow[] {\n  // Visit the base structure recursively\n  const raw = traverse(entity);\n\n  // Start with some special keys that are always present because you want to\n  // be able to easily search for null specifically\n  raw.push({ key: 'metadata.name', value: entity.metadata.name });\n  raw.push({ key: 'metadata.namespace', value: entity.metadata.namespace });\n  raw.push({ key: 'metadata.uid', value: entity.metadata.uid });\n\n  // Namespace not specified has the default value \"default\", so we want to\n  // match on that as well\n  if (!entity.metadata.namespace) {\n    raw.push({ key: 'metadata.namespace', value: DEFAULT_NAMESPACE });\n  }\n\n  // Visit relations\n  for (const relation of entity.relations ?? []) {\n    raw.push({\n      key: `relations.${relation.type}`,\n      value: relation.targetRef,\n    });\n  }\n\n  // This validates that there are no keys that vary only in casing, such\n  // as `spec.foo` and `spec.Foo`.\n  const keys = new Set(raw.map(r => r.key));\n  const lowerKeys = new Set(raw.map(r => r.key.toLocaleLowerCase('en-US')));\n  if (keys.size !== lowerKeys.size) {\n    const difference = [];\n    for (const key of keys) {\n      const lower = key.toLocaleLowerCase('en-US');\n      if (!lowerKeys.delete(lower)) {\n        difference.push(lower);\n      }\n    }\n    const badKeys = `'${difference.join(\"', '\")}'`;\n    throw new InputError(\n      `Entity has duplicate keys that vary only in casing, ${badKeys}`,\n    );\n  }\n\n  const rows = mapToRows(raw, entityId);\n\n  // Deduplicate by (key, value). Duplicate array values in the entity data\n  // (e.g. tags: ['java', 'java']) produce identical search rows which would\n  // violate the unique constraint on (entity_id, key, value).\n  const seen = new Set<string>();\n  return rows.filter(row => {\n    const k = `${row.key}\\0${row.value === null ? NULL_SENTINEL : row.value}`;\n    if (seen.has(k)) return false;\n    seen.add(k);\n    return true;\n  });\n}\n"],"names":["DEFAULT_NAMESPACE","InputError","NULL_SENTINEL"],"mappings":";;;;;;AAwBA,MAAM,YAAA,GAAe;AAAA,EACnB,aAAA;AAAA,EACA,WAAA;AAAA,EACA,QAAA;AAAA,EACA,eAAA;AAAA,EACA,oBAAA;AAAA,EACA,cAAA;AAAA,EACA;AACF,CAAA;AAKA,MAAM,cAAA,GAAiB,GAAA;AACvB,MAAM,gBAAA,GAAmB,GAAA;AAkClB,SAAS,SAAS,IAAA,EAAqB;AAC5C,EAAA,MAAM,SAAe,EAAC;AAKtB,EAAA,MAAM,YAAA,uBAAmB,GAAA,EAAY;AAErC,EAAA,SAAS,KAAA,CAAM,MAAc,OAAA,EAAkB;AAC7C,IAAA,IAAI,YAAA,CAAa,QAAA,CAAS,IAAI,CAAA,EAAG;AAC/B,MAAA;AAAA,IACF;AAGA,IAAA,IACE,OAAA,KAAY,MAAA,IACZ,OAAA,KAAY,IAAA,IACZ,CAAC,QAAA,EAAU,QAAA,EAAU,SAAS,CAAA,CAAE,QAAA,CAAS,OAAO,OAAO,CAAA,EACvD;AACA,MAAA,MAAA,CAAO,KAAK,EAAE,GAAA,EAAK,IAAA,EAAM,KAAA,EAAO,SAAS,CAAA;AACzC,MAAA;AAAA,IACF;AAGA,IAAA,IAAI,OAAO,YAAY,QAAA,EAAU;AAC/B,MAAA;AAAA,IACF;AAGA,IAAA,IAAI,KAAA,CAAM,OAAA,CAAQ,OAAO,CAAA,EAAG;AAC1B,MAAA,KAAA,MAAW,QAAQ,OAAA,EAAS;AAc1B,QAAA,KAAA,CAAM,MAAM,IAAI,CAAA;AAChB,QAAA,IAAI,OAAO,SAAS,QAAA,EAAU;AAC5B,UAAA,MAAM,OAAA,GAAU,CAAA,EAAG,IAAI,CAAA,CAAA,EAAI,IAAI,CAAA,CAAA;AAC/B,UAAA,MAAM,QAAA,GAAW,OAAA,CAAQ,iBAAA,CAAkB,OAAO,CAAA;AAClD,UAAA,IAAI,CAAC,YAAA,CAAa,GAAA,CAAI,QAAQ,CAAA,EAAG;AAC/B,YAAA,YAAA,CAAa,IAAI,QAAQ,CAAA;AACzB,YAAA,MAAA,CAAO,KAAK,EAAE,GAAA,EAAK,OAAA,EAAS,KAAA,EAAO,MAAM,CAAA;AAAA,UAC3C;AAAA,QACF;AAAA,MACF;AACA,MAAA;AAAA,IACF;AAGA,IAAA,KAAA,MAAW,CAAC,GAAA,EAAK,KAAK,KAAK,MAAA,CAAO,OAAA,CAAQ,OAAQ,CAAA,EAAG;AACnD,MAAA,KAAA,CAAM,OAAO,CAAA,EAAG,IAAI,IAAI,GAAG,CAAA,CAAA,GAAK,KAAK,KAAK,CAAA;AAAA,IAC5C;AAAA,EACF;AAEA,EAAA,KAAA,CAAM,IAAI,IAAI,CAAA;AAEd,EAAA,OAAO,MAAA;AACT;AAGO,SAAS,SAAA,CAAU,OAAa,QAAA,EAAiC;AACtE,EAAA,MAAM,SAAwB,EAAC;AAE/B,EAAA,KAAA,MAAW,EAAE,GAAA,EAAK,MAAA,EAAQ,KAAA,EAAO,QAAA,MAAc,KAAA,EAAO;AACpD,IAAA,MAAM,GAAA,GAAM,MAAA,CAAO,iBAAA,CAAkB,OAAO,CAAA;AAC5C,IAAA,IAAI,GAAA,CAAI,SAAS,cAAA,EAAgB;AAC/B,MAAA;AAAA,IACF;AACA,IAAA,IAAI,QAAA,KAAa,MAAA,IAAa,QAAA,KAAa,IAAA,EAAM;AAC/C,MAAA,MAAA,CAAO,IAAA,CAAK;AAAA,QACV,SAAA,EAAW,QAAA;AAAA,QACX,GAAA;AAAA,QACA,cAAA,EAAgB,IAAA;AAAA,QAChB,KAAA,EAAO;AAAA,OACR,CAAA;AAAA,IACH,CAAA,MAAO;AACL,MAAA,MAAM,KAAA,GAAQ,MAAA,CAAO,QAAQ,CAAA,CAAE,kBAAkB,OAAO,CAAA;AACxD,MAAA,IAAI,KAAA,CAAM,UAAU,gBAAA,EAAkB;AACpC,QAAA,MAAA,CAAO,IAAA,CAAK;AAAA,UACV,SAAA,EAAW,QAAA;AAAA,UACX,GAAA;AAAA,UACA,cAAA,EAAgB,OAAO,QAAQ,CAAA;AAAA,UAC/B;AAAA,SACD,CAAA;AAAA,MACH,CAAA,MAAO;AACL,QAAA,MAAA,CAAO,IAAA,CAAK;AAAA,UACV,SAAA,EAAW,QAAA;AAAA,UACX,GAAA;AAAA,UACA,cAAA,EAAgB,IAAA;AAAA,UAChB,KAAA,EAAO;AAAA,SACR,CAAA;AAAA,MACH;AAAA,IACF;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AASO,SAAS,iBAAA,CACd,UACA,MAAA,EACe;AAEf,EAAA,MAAM,GAAA,GAAM,SAAS,MAAM,CAAA;AAI3B,EAAA,GAAA,CAAI,IAAA,CAAK,EAAE,GAAA,EAAK,eAAA,EAAiB,OAAO,MAAA,CAAO,QAAA,CAAS,MAAM,CAAA;AAC9D,EAAA,GAAA,CAAI,IAAA,CAAK,EAAE,GAAA,EAAK,oBAAA,EAAsB,OAAO,MAAA,CAAO,QAAA,CAAS,WAAW,CAAA;AACxE,EAAA,GAAA,CAAI,IAAA,CAAK,EAAE,GAAA,EAAK,cAAA,EAAgB,OAAO,MAAA,CAAO,QAAA,CAAS,KAAK,CAAA;AAI5D,EAAA,IAAI,CAAC,MAAA,CAAO,QAAA,CAAS,SAAA,EAAW;AAC9B,IAAA,GAAA,CAAI,KAAK,EAAE,GAAA,EAAK,oBAAA,EAAsB,KAAA,EAAOA,gCAAmB,CAAA;AAAA,EAClE;AAGA,EAAA,KAAA,MAAW,QAAA,IAAY,MAAA,CAAO,SAAA,IAAa,EAAC,EAAG;AAC7C,IAAA,GAAA,CAAI,IAAA,CAAK;AAAA,MACP,GAAA,EAAK,CAAA,UAAA,EAAa,QAAA,CAAS,IAAI,CAAA,CAAA;AAAA,MAC/B,OAAO,QAAA,CAAS;AAAA,KACjB,CAAA;AAAA,EACH;AAIA,EAAA,MAAM,IAAA,GAAO,IAAI,GAAA,CAAI,GAAA,CAAI,IAAI,CAAA,CAAA,KAAK,CAAA,CAAE,GAAG,CAAC,CAAA;AACxC,EAAA,MAAM,SAAA,GAAY,IAAI,GAAA,CAAI,GAAA,CAAI,GAAA,CAAI,CAAA,CAAA,KAAK,CAAA,CAAE,GAAA,CAAI,iBAAA,CAAkB,OAAO,CAAC,CAAC,CAAA;AACxE,EAAA,IAAI,IAAA,CAAK,IAAA,KAAS,SAAA,CAAU,IAAA,EAAM;AAChC,IAAA,MAAM,aAAa,EAAC;AACpB,IAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,MAAA,MAAM,KAAA,GAAQ,GAAA,CAAI,iBAAA,CAAkB,OAAO,CAAA;AAC3C,MAAA,IAAI,CAAC,SAAA,CAAU,MAAA,CAAO,KAAK,CAAA,EAAG;AAC5B,QAAA,UAAA,CAAW,KAAK,KAAK,CAAA;AAAA,MACvB;AAAA,IACF;AACA,IAAA,MAAM,OAAA,GAAU,CAAA,CAAA,EAAI,UAAA,CAAW,IAAA,CAAK,MAAM,CAAC,CAAA,CAAA,CAAA;AAC3C,IAAA,MAAM,IAAIC,iBAAA;AAAA,MACR,uDAAuD,OAAO,CAAA;AAAA,KAChE;AAAA,EACF;AAEA,EAAA,MAAM,IAAA,GAAO,SAAA,CAAU,GAAA,EAAK,QAAQ,CAAA;AAKpC,EAAA,MAAM,IAAA,uBAAW,GAAA,EAAY;AAC7B,EAAA,OAAO,IAAA,CAAK,OAAO,CAAA,GAAA,KAAO;AACxB,IAAA,MAAM,CAAA,GAAI,CAAA,EAAG,GAAA,CAAI,GAAG,CAAA,EAAA,EAAK,IAAI,KAAA,KAAU,IAAA,GAAOC,kBAAA,GAAgB,GAAA,CAAI,KAAK,CAAA,CAAA;AACvE,IAAA,IAAI,IAAA,CAAK,GAAA,CAAI,CAAC,CAAA,EAAG,OAAO,KAAA;AACxB,IAAA,IAAA,CAAK,IAAI,CAAC,CAAA;AACV,IAAA,OAAO,IAAA;AAAA,EACT,CAAC,CAAA;AACH;;;;;;"}

package/dist/database/operations/stitcher/syncSearchRows.cjs.js

@@ -2,20 +2,27 @@
 
 var util = require('./util.cjs.js');
 
-const NULL_SENTINEL = "
";
 function filterSentinelValues(entries) {
   return entries.filter(
-    (r) => r.value !== NULL_SENTINEL && r.original_value !== NULL_SENTINEL
+    (r) => r.value !== util.NULL_SENTINEL && r.original_value !== util.NULL_SENTINEL
   );
 }
 async function syncSearchRows(knex, entityId, searchEntries) {
+  const dedupMap = /* @__PURE__ */ new Map();
+  for (const entry of searchEntries) {
+    const k = `${entry.key}\0${entry.value === null ? util.NULL_SENTINEL : entry.value}`;
+    if (!dedupMap.has(k)) {
+      dedupMap.set(k, entry);
+    }
+  }
+  const deduped = [...dedupMap.values()];
   const client = knex.client.config.client;
   if (client === "pg") {
-    await syncPostgres(knex, entityId, searchEntries);
+    await syncPostgres(knex, entityId, deduped);
   } else if (client.includes("mysql")) {
-    await syncMysql(knex, entityId, searchEntries);
+    await syncMysql(knex, entityId, deduped);
   } else {
-    await syncBulkReplace(knex, entityId, searchEntries);
+    await syncBulkReplace(knex, entityId, deduped);
   }
 }
 async function syncPostgres(knex, entityId, searchEntries) {
@@ -50,6 +57,8 @@ async function syncPostgres(knex, entityId, searchEntries) {
         AND COALESCE(s.value, chr(1)) = COALESCE(d.value, chr(1))
         AND COALESCE(s.original_value, chr(1)) = COALESCE(d.original_value, chr(1))
     )
+    ON CONFLICT (entity_id, key, value)
+    DO UPDATE SET original_value = EXCLUDED.original_value
     `,
     [keys, values, originalValues, entityId, entityId, entityId]
   );

package/dist/database/operations/stitcher/syncSearchRows.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"syncSearchRows.cjs.js","sources":["../../../../src/database/operations/stitcher/syncSearchRows.ts"],"sourcesContent":["/*\n * Copyright 2026 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Knex } from 'knex';\nimport { DbSearchRow } from '../../tables';\nimport { BATCH_SIZE } from './util';\n\n// The Postgres sync uses COALESCE(x, NULL_SENTINEL) to allow Postgres to\n// include nullable columns in the Hash Cond of anti-joins (IS NOT DISTINCT\n// FROM prevents this). As a consequence, values that are exactly this\n// sentinel character are not searchable — they would be treated as NULL.\n// This is the SOH (Start of Heading) control character which does not\n// appear in real entity metadata.\nconst NULL_SENTINEL = '\\x01';\n\nfunction filterSentinelValues(entries: DbSearchRow[]): DbSearchRow[] {\n  return entries.filter(\n    r => r.value !== NULL_SENTINEL && r.original_value !== NULL_SENTINEL,\n  );\n}\n\n/**\n * Synchronizes the search table rows for a given entity, applying only the\n * minimal set of changes needed. Rows that already exist with the correct\n * values are left untouched, new rows are inserted, and stale rows are\n * deleted — minimizing write churn, dead tuples, and WAL traffic.\n *\n * Uses database-specific strategies:\n * - Postgres: Single writable CTE with unnest (one round-trip, no DDL)\n * - MySQL: Temporary table merge (two queries in a transaction)\n * - SQLite: Simple bulk replace (sufficient for dev/test)\n */\nexport async function syncSearchRows(\n  knex: Knex | Knex.Transaction,\n  entityId: string,\n  searchEntries: DbSearchRow[],\n): Promise<void> {\n  const client = knex.client.config.client;\n\n  if (client === 'pg') {\n    await syncPostgres(knex, entityId, searchEntries);\n  } else if (client.includes('mysql')) {\n    await syncMysql(knex, entityId, searchEntries);\n  } else {\n    await syncBulkReplace(knex, entityId, searchEntries);\n  }\n}\n\n// ---------------------------------------------------------------------------\n// Postgres: writable CTE + unnest\n//\n// All CTE branches see the same pre-modification snapshot, so the DELETE\n// and INSERT do not interfere with each other. This is a single atomic\n// statement — no explicit transaction wrapper needed.\n//\n// Nullable columns use COALESCE(x, chr(1)) instead of IS NOT DISTINCT FROM\n// so that Postgres can include all three columns in the Hash Cond of the\n// anti-join, rather than pushing nullable comparisons into a Join Filter\n// that degrades to O(n*m) when many rows share the same key. chr(1) (SOH\n// control character) is used as the NULL sentinel — it cannot appear in\n// real entity values since they are human-readable strings.\n// ---------------------------------------------------------------------------\nasync function syncPostgres(\n  knex: Knex | Knex.Transaction,\n  entityId: string,\n  searchEntries: DbSearchRow[],\n): Promise<void> {\n  const filtered = filterSentinelValues(searchEntries);\n  const keys = filtered.map(r => r.key);\n  const values = filtered.map(r => r.value);\n  const originalValues = filtered.map(r => r.original_value);\n\n  await knex.raw(\n    `\n    WITH desired AS (\n      SELECT *\n      FROM unnest(?::text[], ?::text[], ?::text[])\n        AS d(key, value, original_value)\n    ),\n    deleted AS (\n      DELETE FROM \"search\" s\n      WHERE s.entity_id = ?\n        AND NOT EXISTS (\n          SELECT 1 FROM desired d\n          WHERE d.key = s.key\n            AND COALESCE(d.value, chr(1)) = COALESCE(s.value, chr(1))\n            AND COALESCE(d.original_value, chr(1)) = COALESCE(s.original_value, chr(1))\n        )\n    )\n    INSERT INTO \"search\" (entity_id, key, value, original_value)\n    SELECT ?, d.key, d.value, d.original_value\n    FROM desired d\n    WHERE NOT EXISTS (\n      SELECT 1 FROM \"search\" s\n      WHERE s.entity_id = ?\n        AND s.key = d.key\n        AND COALESCE(s.value, chr(1)) = COALESCE(d.value, chr(1))\n        AND COALESCE(s.original_value, chr(1)) = COALESCE(d.original_value, chr(1))\n    )\n    `,\n    [keys, values, originalValues, entityId, entityId, entityId],\n  );\n}\n\n// ---------------------------------------------------------------------------\n// MySQL: temporary table merge with deadlock retry\n//\n// MySQL does not support data-modifying CTEs, so we materialize the desired\n// state into a session-scoped temporary table and then merge it into the\n// real table with two queries. The temp table is created inside the\n// transaction to guarantee it exists on the same pooled connection.\n// CREATE/DROP TEMPORARY TABLE does not cause an implicit commit in MySQL\n// (unlike regular DDL), so this is transaction-safe.\n//\n// InnoDB's next-key (gap) locking can cause deadlocks between concurrent\n// transactions operating on different entity_ids when their gap locks\n// overlap on shared index pages. We retry on deadlock (error 1213) since\n// the operation is idempotent.\n// ---------------------------------------------------------------------------\nconst MYSQL_DEADLOCK_MAX_RETRIES = 3;\n\nasync function syncMysql(\n  knex: Knex | Knex.Transaction,\n  entityId: string,\n  searchEntries: DbSearchRow[],\n): Promise<void> {\n  for (let attempt = 1; ; attempt++) {\n    try {\n      await knex.transaction(async trx => {\n        // Create the temp table inside the transaction so it's guaranteed\n        // to be on the same pooled connection as the merge queries.\n        // CREATE TEMPORARY TABLE does not cause an implicit commit in\n        // MySQL (unlike regular CREATE TABLE), so this is safe.\n        await trx.raw(\n          'CREATE TEMPORARY TABLE IF NOT EXISTS `_desired_search` (' +\n            '`key` VARCHAR(255) NOT NULL, ' +\n            '`value` VARCHAR(255) NULL, ' +\n            '`original_value` VARCHAR(255) NULL' +\n            ')',\n        );\n        // Clear stale data from any previous call on this connection.\n        // Uses DELETE (DML) instead of TRUNCATE (DDL) to avoid an\n        // implicit commit that would break transaction atomicity.\n        await trx.raw('DELETE FROM `_desired_search`');\n\n        if (searchEntries.length > 0) {\n          await trx.batchInsert(\n            '_desired_search',\n            searchEntries.map(r => ({\n              key: r.key,\n              value: r.value,\n              original_value: r.original_value,\n            })),\n            BATCH_SIZE,\n          );\n        }\n\n        // Delete rows that are no longer in the desired set\n        await trx.raw(\n          'DELETE s FROM `search` s ' +\n            'WHERE s.entity_id = ? ' +\n            'AND NOT EXISTS (' +\n            '  SELECT 1 FROM `_desired_search` d' +\n            '  WHERE d.`key` = s.`key`' +\n            '    AND d.`value` <=> s.`value`' +\n            '    AND BINARY d.`original_value` <=> BINARY s.`original_value`' +\n            ')',\n          [entityId],\n        );\n\n        // Insert rows that are new in the desired set. The original_value\n        // column preserves the original casing and must be compared with\n        // BINARY to avoid MySQL's default case-insensitive collation\n        // treating e.g. \"Team-A\" and \"team-a\" as equal.\n        await trx.raw(\n          'INSERT INTO `search` (entity_id, `key`, `value`, `original_value`) ' +\n            'SELECT ?, d.`key`, d.`value`, d.`original_value` ' +\n            'FROM `_desired_search` d ' +\n            'WHERE NOT EXISTS (' +\n            '  SELECT 1 FROM `search` s' +\n            '  WHERE s.entity_id = ?' +\n            '    AND s.`key` = d.`key`' +\n            '    AND s.`value` <=> d.`value`' +\n            '    AND BINARY s.`original_value` <=> BINARY d.`original_value`' +\n            ')',\n          [entityId, entityId],\n        );\n      });\n      return;\n    } catch (error) {\n      // MySQL error 1213: ER_LOCK_DEADLOCK\n      if (\n        (error as any)?.errno === 1213 &&\n        attempt < MYSQL_DEADLOCK_MAX_RETRIES\n      ) {\n        continue;\n      }\n      throw error;\n    }\n  }\n}\n\n// ---------------------------------------------------------------------------\n// SQLite (and fallback): bulk replace\n// ---------------------------------------------------------------------------\nasync function syncBulkReplace(\n  knex: Knex | Knex.Transaction,\n  entityId: string,\n  searchEntries: DbSearchRow[],\n): Promise<void> {\n  await knex.transaction(async trx => {\n    await trx<DbSearchRow>('search').where({ entity_id: entityId }).delete();\n    await trx.batchInsert('search', searchEntries, BATCH_SIZE);\n  });\n}\n"],"names":["BATCH_SIZE"],"mappings":";;;;AA0BA,MAAM,aAAA,GAAgB,GAAA;AAEtB,SAAS,qBAAqB,OAAA,EAAuC;AACnE,EAAA,OAAO,OAAA,CAAQ,MAAA;AAAA,IACb,CAAA,CAAA,KAAK,CAAA,CAAE,KAAA,KAAU,aAAA,IAAiB,EAAE,cAAA,KAAmB;AAAA,GACzD;AACF;AAaA,eAAsB,cAAA,CACpB,IAAA,EACA,QAAA,EACA,aAAA,EACe;AACf,EAAA,MAAM,MAAA,GAAS,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,MAAA;AAElC,EAAA,IAAI,WAAW,IAAA,EAAM;AACnB,IAAA,MAAM,YAAA,CAAa,IAAA,EAAM,QAAA,EAAU,aAAa,CAAA;AAAA,EAClD,CAAA,MAAA,IAAW,MAAA,CAAO,QAAA,CAAS,OAAO,CAAA,EAAG;AACnC,IAAA,MAAM,SAAA,CAAU,IAAA,EAAM,QAAA,EAAU,aAAa,CAAA;AAAA,EAC/C,CAAA,MAAO;AACL,IAAA,MAAM,eAAA,CAAgB,IAAA,EAAM,QAAA,EAAU,aAAa,CAAA;AAAA,EACrD;AACF;AAgBA,eAAe,YAAA,CACb,IAAA,EACA,QAAA,EACA,aAAA,EACe;AACf,EAAA,MAAM,QAAA,GAAW,qBAAqB,aAAa,CAAA;AACnD,EAAA,MAAM,IAAA,GAAO,QAAA,CAAS,GAAA,CAAI,CAAA,CAAA,KAAK,EAAE,GAAG,CAAA;AACpC,EAAA,MAAM,MAAA,GAAS,QAAA,CAAS,GAAA,CAAI,CAAA,CAAA,KAAK,EAAE,KAAK,CAAA;AACxC,EAAA,MAAM,cAAA,GAAiB,QAAA,CAAS,GAAA,CAAI,CAAA,CAAA,KAAK,EAAE,cAAc,CAAA;AAEzD,EAAA,MAAM,IAAA,CAAK,GAAA;AAAA,IACT;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,IAAA,CAAA;AAAA,IA2BA,CAAC,IAAA,EAAM,MAAA,EAAQ,cAAA,EAAgB,QAAA,EAAU,UAAU,QAAQ;AAAA,GAC7D;AACF;AAiBA,MAAM,0BAAA,GAA6B,CAAA;AAEnC,eAAe,SAAA,CACb,IAAA,EACA,QAAA,EACA,aAAA,EACe;AACf,EAAA,KAAA,IAAS,OAAA,GAAU,KAAK,OAAA,EAAA,EAAW;AACjC,IAAA,IAAI;AACF,MAAA,MAAM,IAAA,CAAK,WAAA,CAAY,OAAM,GAAA,KAAO;AAKlC,QAAA,MAAM,GAAA,CAAI,GAAA;AAAA,UACR;AAAA,SAKF;AAIA,QAAA,MAAM,GAAA,CAAI,IAAI,+BAA+B,CAAA;AAE7C,QAAA,IAAI,aAAA,CAAc,SAAS,CAAA,EAAG;AAC5B,UAAA,MAAM,GAAA,CAAI,WAAA;AAAA,YACR,iBAAA;AAAA,YACA,aAAA,CAAc,IAAI,CAAA,CAAA,MAAM;AAAA,cACtB,KAAK,CAAA,CAAE,GAAA;AAAA,cACP,OAAO,CAAA,CAAE,KAAA;AAAA,cACT,gBAAgB,CAAA,CAAE;AAAA,aACpB,CAAE,CAAA;AAAA,YACFA;AAAA,WACF;AAAA,QACF;AAGA,QAAA,MAAM,GAAA,CAAI,GAAA;AAAA,UACR,4NAAA;AAAA,UAQA,CAAC,QAAQ;AAAA,SACX;AAMA,QAAA,MAAM,GAAA,CAAI,GAAA;AAAA,UACR,0UAAA;AAAA,UAUA,CAAC,UAAU,QAAQ;AAAA,SACrB;AAAA,MACF,CAAC,CAAA;AACD,MAAA;AAAA,IACF,SAAS,KAAA,EAAO;AAEd,MAAA,IACG,KAAA,EAAe,KAAA,KAAU,IAAA,IAC1B,OAAA,GAAU,0BAAA,EACV;AACA,QAAA;AAAA,MACF;AACA,MAAA,MAAM,KAAA;AAAA,IACR;AAAA,EACF;AACF;AAKA,eAAe,eAAA,CACb,IAAA,EACA,QAAA,EACA,aAAA,EACe;AACf,EAAA,MAAM,IAAA,CAAK,WAAA,CAAY,OAAM,GAAA,KAAO;AAClC,IAAA,MAAM,GAAA,CAAiB,QAAQ,CAAA,CAAE,KAAA,CAAM,EAAE,SAAA,EAAW,QAAA,EAAU,CAAA,CAAE,MAAA,EAAO;AACvE,IAAA,MAAM,GAAA,CAAI,WAAA,CAAY,QAAA,EAAU,aAAA,EAAeA,eAAU,CAAA;AAAA,EAC3D,CAAC,CAAA;AACH;;;;"}
+{"version":3,"file":"syncSearchRows.cjs.js","sources":["../../../../src/database/operations/stitcher/syncSearchRows.ts"],"sourcesContent":["/*\n * Copyright 2026 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Knex } from 'knex';\nimport { DbSearchRow } from '../../tables';\nimport { BATCH_SIZE, NULL_SENTINEL } from './util';\n\nfunction filterSentinelValues(entries: DbSearchRow[]): DbSearchRow[] {\n  return entries.filter(\n    r => r.value !== NULL_SENTINEL && r.original_value !== NULL_SENTINEL,\n  );\n}\n\n/**\n * Synchronizes the search table rows for a given entity, applying only the\n * minimal set of changes needed. Rows that already exist with the correct\n * values are left untouched, new rows are inserted, and stale rows are\n * deleted — minimizing write churn, dead tuples, and WAL traffic.\n *\n * Uses database-specific strategies:\n * - Postgres: Single writable CTE with unnest (one round-trip, no DDL)\n * - MySQL: Temporary table merge (two queries in a transaction)\n * - SQLite: Simple bulk replace (sufficient for dev/test)\n */\nexport async function syncSearchRows(\n  knex: Knex | Knex.Transaction,\n  entityId: string,\n  searchEntries: DbSearchRow[],\n): Promise<void> {\n  // Dedup by (key, value) — the UNIQUE constraint on (entity_id, key, value)\n  // rejects duplicates, and the same lowercased value with different original\n  // casing is semantically a single entry. Keep the first occurrence, which\n  // matches the first-wins semantics of buildEntitySearch so that both layers\n  // consistently pick the same original_value for a given input order.\n  const dedupMap = new Map<string, DbSearchRow>();\n  for (const entry of searchEntries) {\n    const k = `${entry.key}\\0${\n      entry.value === null ? NULL_SENTINEL : entry.value\n    }`;\n    if (!dedupMap.has(k)) {\n      dedupMap.set(k, entry);\n    }\n  }\n  const deduped = [...dedupMap.values()];\n\n  const client = knex.client.config.client;\n\n  if (client === 'pg') {\n    await syncPostgres(knex, entityId, deduped);\n  } else if (client.includes('mysql')) {\n    await syncMysql(knex, entityId, deduped);\n  } else {\n    await syncBulkReplace(knex, entityId, deduped);\n  }\n}\n\n// ---------------------------------------------------------------------------\n// Postgres: writable CTE + unnest\n//\n// All CTE branches see the same pre-modification snapshot, so the DELETE\n// and INSERT do not interfere with each other. This is a single atomic\n// statement — no explicit transaction wrapper needed.\n//\n// Nullable columns use COALESCE(x, chr(1)) instead of IS NOT DISTINCT FROM\n// so that Postgres can include all three columns in the Hash Cond of the\n// anti-join, rather than pushing nullable comparisons into a Join Filter\n// that degrades to O(n*m) when many rows share the same key. chr(1) (SOH\n// control character) is used as the NULL sentinel — it cannot appear in\n// real entity values since they are human-readable strings.\n// ---------------------------------------------------------------------------\nasync function syncPostgres(\n  knex: Knex | Knex.Transaction,\n  entityId: string,\n  searchEntries: DbSearchRow[],\n): Promise<void> {\n  const filtered = filterSentinelValues(searchEntries);\n  const keys = filtered.map(r => r.key);\n  const values = filtered.map(r => r.value);\n  const originalValues = filtered.map(r => r.original_value);\n\n  await knex.raw(\n    `\n    WITH desired AS (\n      SELECT *\n      FROM unnest(?::text[], ?::text[], ?::text[])\n        AS d(key, value, original_value)\n    ),\n    deleted AS (\n      DELETE FROM \"search\" s\n      WHERE s.entity_id = ?\n        AND NOT EXISTS (\n          SELECT 1 FROM desired d\n          WHERE d.key = s.key\n            AND COALESCE(d.value, chr(1)) = COALESCE(s.value, chr(1))\n            AND COALESCE(d.original_value, chr(1)) = COALESCE(s.original_value, chr(1))\n        )\n    )\n    INSERT INTO \"search\" (entity_id, key, value, original_value)\n    SELECT ?, d.key, d.value, d.original_value\n    FROM desired d\n    WHERE NOT EXISTS (\n      SELECT 1 FROM \"search\" s\n      WHERE s.entity_id = ?\n        AND s.key = d.key\n        AND COALESCE(s.value, chr(1)) = COALESCE(d.value, chr(1))\n        AND COALESCE(s.original_value, chr(1)) = COALESCE(d.original_value, chr(1))\n    )\n    ON CONFLICT (entity_id, key, value)\n    DO UPDATE SET original_value = EXCLUDED.original_value\n    `,\n    [keys, values, originalValues, entityId, entityId, entityId],\n  );\n}\n\n// ---------------------------------------------------------------------------\n// MySQL: temporary table merge with deadlock retry\n//\n// MySQL does not support data-modifying CTEs, so we materialize the desired\n// state into a session-scoped temporary table and then merge it into the\n// real table with two queries. The temp table is created inside the\n// transaction to guarantee it exists on the same pooled connection.\n// CREATE/DROP TEMPORARY TABLE does not cause an implicit commit in MySQL\n// (unlike regular DDL), so this is transaction-safe.\n//\n// InnoDB's next-key (gap) locking can cause deadlocks between concurrent\n// transactions operating on different entity_ids when their gap locks\n// overlap on shared index pages. We retry on deadlock (error 1213) since\n// the operation is idempotent.\n// ---------------------------------------------------------------------------\nconst MYSQL_DEADLOCK_MAX_RETRIES = 3;\n\nasync function syncMysql(\n  knex: Knex | Knex.Transaction,\n  entityId: string,\n  searchEntries: DbSearchRow[],\n): Promise<void> {\n  for (let attempt = 1; ; attempt++) {\n    try {\n      await knex.transaction(async trx => {\n        // Create the temp table inside the transaction so it's guaranteed\n        // to be on the same pooled connection as the merge queries.\n        // CREATE TEMPORARY TABLE does not cause an implicit commit in\n        // MySQL (unlike regular CREATE TABLE), so this is safe.\n        await trx.raw(\n          'CREATE TEMPORARY TABLE IF NOT EXISTS `_desired_search` (' +\n            '`key` VARCHAR(255) NOT NULL, ' +\n            '`value` VARCHAR(255) NULL, ' +\n            '`original_value` VARCHAR(255) NULL' +\n            ')',\n        );\n        // Clear stale data from any previous call on this connection.\n        // Uses DELETE (DML) instead of TRUNCATE (DDL) to avoid an\n        // implicit commit that would break transaction atomicity.\n        await trx.raw('DELETE FROM `_desired_search`');\n\n        if (searchEntries.length > 0) {\n          await trx.batchInsert(\n            '_desired_search',\n            searchEntries.map(r => ({\n              key: r.key,\n              value: r.value,\n              original_value: r.original_value,\n            })),\n            BATCH_SIZE,\n          );\n        }\n\n        // Delete rows that are no longer in the desired set\n        await trx.raw(\n          'DELETE s FROM `search` s ' +\n            'WHERE s.entity_id = ? ' +\n            'AND NOT EXISTS (' +\n            '  SELECT 1 FROM `_desired_search` d' +\n            '  WHERE d.`key` = s.`key`' +\n            '    AND d.`value` <=> s.`value`' +\n            '    AND BINARY d.`original_value` <=> BINARY s.`original_value`' +\n            ')',\n          [entityId],\n        );\n\n        // Insert rows that are new in the desired set. The original_value\n        // column preserves the original casing and must be compared with\n        // BINARY to avoid MySQL's default case-insensitive collation\n        // treating e.g. \"Team-A\" and \"team-a\" as equal.\n        await trx.raw(\n          'INSERT INTO `search` (entity_id, `key`, `value`, `original_value`) ' +\n            'SELECT ?, d.`key`, d.`value`, d.`original_value` ' +\n            'FROM `_desired_search` d ' +\n            'WHERE NOT EXISTS (' +\n            '  SELECT 1 FROM `search` s' +\n            '  WHERE s.entity_id = ?' +\n            '    AND s.`key` = d.`key`' +\n            '    AND s.`value` <=> d.`value`' +\n            '    AND BINARY s.`original_value` <=> BINARY d.`original_value`' +\n            ')',\n          [entityId, entityId],\n        );\n      });\n      return;\n    } catch (error) {\n      // MySQL error 1213: ER_LOCK_DEADLOCK\n      if (\n        (error as any)?.errno === 1213 &&\n        attempt < MYSQL_DEADLOCK_MAX_RETRIES\n      ) {\n        continue;\n      }\n      throw error;\n    }\n  }\n}\n\n// ---------------------------------------------------------------------------\n// SQLite (and fallback): bulk replace\n// ---------------------------------------------------------------------------\nasync function syncBulkReplace(\n  knex: Knex | Knex.Transaction,\n  entityId: string,\n  searchEntries: DbSearchRow[],\n): Promise<void> {\n  await knex.transaction(async trx => {\n    await trx<DbSearchRow>('search').where({ entity_id: entityId }).delete();\n    await trx.batchInsert('search', searchEntries, BATCH_SIZE);\n  });\n}\n"],"names":["NULL_SENTINEL","BATCH_SIZE"],"mappings":";;;;AAoBA,SAAS,qBAAqB,OAAA,EAAuC;AACnE,EAAA,OAAO,OAAA,CAAQ,MAAA;AAAA,IACb,CAAA,CAAA,KAAK,CAAA,CAAE,KAAA,KAAUA,kBAAA,IAAiB,EAAE,cAAA,KAAmBA;AAAA,GACzD;AACF;AAaA,eAAsB,cAAA,CACpB,IAAA,EACA,QAAA,EACA,aAAA,EACe;AAMf,EAAA,MAAM,QAAA,uBAAe,GAAA,EAAyB;AAC9C,EAAA,KAAA,MAAW,SAAS,aAAA,EAAe;AACjC,IAAA,MAAM,CAAA,GAAI,CAAA,EAAG,KAAA,CAAM,GAAG,CAAA,EAAA,EACpB,MAAM,KAAA,KAAU,IAAA,GAAOA,kBAAA,GAAgB,KAAA,CAAM,KAC/C,CAAA,CAAA;AACA,IAAA,IAAI,CAAC,QAAA,CAAS,GAAA,CAAI,CAAC,CAAA,EAAG;AACpB,MAAA,QAAA,CAAS,GAAA,CAAI,GAAG,KAAK,CAAA;AAAA,IACvB;AAAA,EACF;AACA,EAAA,MAAM,OAAA,GAAU,CAAC,GAAG,QAAA,CAAS,QAAQ,CAAA;AAErC,EAAA,MAAM,MAAA,GAAS,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,MAAA;AAElC,EAAA,IAAI,WAAW,IAAA,EAAM;AACnB,IAAA,MAAM,YAAA,CAAa,IAAA,EAAM,QAAA,EAAU,OAAO,CAAA;AAAA,EAC5C,CAAA,MAAA,IAAW,MAAA,CAAO,QAAA,CAAS,OAAO,CAAA,EAAG;AACnC,IAAA,MAAM,SAAA,CAAU,IAAA,EAAM,QAAA,EAAU,OAAO,CAAA;AAAA,EACzC,CAAA,MAAO;AACL,IAAA,MAAM,eAAA,CAAgB,IAAA,EAAM,QAAA,EAAU,OAAO,CAAA;AAAA,EAC/C;AACF;AAgBA,eAAe,YAAA,CACb,IAAA,EACA,QAAA,EACA,aAAA,EACe;AACf,EAAA,MAAM,QAAA,GAAW,qBAAqB,aAAa,CAAA;AACnD,EAAA,MAAM,IAAA,GAAO,QAAA,CAAS,GAAA,CAAI,CAAA,CAAA,KAAK,EAAE,GAAG,CAAA;AACpC,EAAA,MAAM,MAAA,GAAS,QAAA,CAAS,GAAA,CAAI,CAAA,CAAA,KAAK,EAAE,KAAK,CAAA;AACxC,EAAA,MAAM,cAAA,GAAiB,QAAA,CAAS,GAAA,CAAI,CAAA,CAAA,KAAK,EAAE,cAAc,CAAA;AAEzD,EAAA,MAAM,IAAA,CAAK,GAAA;AAAA,IACT;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,IAAA,CAAA;AAAA,IA6BA,CAAC,IAAA,EAAM,MAAA,EAAQ,cAAA,EAAgB,QAAA,EAAU,UAAU,QAAQ;AAAA,GAC7D;AACF;AAiBA,MAAM,0BAAA,GAA6B,CAAA;AAEnC,eAAe,SAAA,CACb,IAAA,EACA,QAAA,EACA,aAAA,EACe;AACf,EAAA,KAAA,IAAS,OAAA,GAAU,KAAK,OAAA,EAAA,EAAW;AACjC,IAAA,IAAI;AACF,MAAA,MAAM,IAAA,CAAK,WAAA,CAAY,OAAM,GAAA,KAAO;AAKlC,QAAA,MAAM,GAAA,CAAI,GAAA;AAAA,UACR;AAAA,SAKF;AAIA,QAAA,MAAM,GAAA,CAAI,IAAI,+BAA+B,CAAA;AAE7C,QAAA,IAAI,aAAA,CAAc,SAAS,CAAA,EAAG;AAC5B,UAAA,MAAM,GAAA,CAAI,WAAA;AAAA,YACR,iBAAA;AAAA,YACA,aAAA,CAAc,IAAI,CAAA,CAAA,MAAM;AAAA,cACtB,KAAK,CAAA,CAAE,GAAA;AAAA,cACP,OAAO,CAAA,CAAE,KAAA;AAAA,cACT,gBAAgB,CAAA,CAAE;AAAA,aACpB,CAAE,CAAA;AAAA,YACFC;AAAA,WACF;AAAA,QACF;AAGA,QAAA,MAAM,GAAA,CAAI,GAAA;AAAA,UACR,4NAAA;AAAA,UAQA,CAAC,QAAQ;AAAA,SACX;AAMA,QAAA,MAAM,GAAA,CAAI,GAAA;AAAA,UACR,0UAAA;AAAA,UAUA,CAAC,UAAU,QAAQ;AAAA,SACrB;AAAA,MACF,CAAC,CAAA;AACD,MAAA;AAAA,IACF,SAAS,KAAA,EAAO;AAEd,MAAA,IACG,KAAA,EAAe,KAAA,KAAU,IAAA,IAC1B,OAAA,GAAU,0BAAA,EACV;AACA,QAAA;AAAA,MACF;AACA,MAAA,MAAM,KAAA;AAAA,IACR;AAAA,EACF;AACF;AAKA,eAAe,eAAA,CACb,IAAA,EACA,QAAA,EACA,aAAA,EACe;AACf,EAAA,MAAM,IAAA,CAAK,WAAA,CAAY,OAAM,GAAA,KAAO;AAClC,IAAA,MAAM,GAAA,CAAiB,QAAQ,CAAA,CAAE,KAAA,CAAM,EAAE,SAAA,EAAW,QAAA,EAAU,CAAA,CAAE,MAAA,EAAO;AACvE,IAAA,MAAM,GAAA,CAAI,WAAA,CAAY,QAAA,EAAU,aAAA,EAAeA,eAAU,CAAA;AAAA,EAC3D,CAAC,CAAA;AACH;;;;"}

package/dist/database/operations/stitcher/util.cjs.js

@@ -8,10 +8,12 @@ function _interopDefaultCompat (e) { return e && typeof e === 'object' && 'defau
 var stableStringify__default = /*#__PURE__*/_interopDefaultCompat(stableStringify);
 
 const BATCH_SIZE = 50;
+const NULL_SENTINEL = "
";
 function generateStableHash(entity) {
   return node_crypto.createHash("sha1").update(stableStringify__default.default({ ...entity })).digest("hex");
 }
 
 exports.BATCH_SIZE = BATCH_SIZE;
+exports.NULL_SENTINEL = NULL_SENTINEL;
 exports.generateStableHash = generateStableHash;
 //# sourceMappingURL=util.cjs.js.map

package/dist/database/operations/stitcher/util.cjs.js.map

@@ -1 +1 @@
-{"version":3,"file":"util.cjs.js","sources":["../../../../src/database/operations/stitcher/util.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Entity } from '@backstage/catalog-model';\nimport { createHash } from 'node:crypto';\nimport stableStringify from 'fast-json-stable-stringify';\n\n// The number of items that are sent per batch to the database layer, when\n// doing .batchInsert calls to knex. This needs to be low enough to not cause\n// errors in the underlying engine due to exceeding query limits, but large\n// enough to get the speed benefits.\nexport const BATCH_SIZE = 50;\n\nexport function generateStableHash(entity: Entity) {\n  return createHash('sha1')\n    .update(stableStringify({ ...entity }))\n    .digest('hex');\n}\n"],"names":["createHash","stableStringify"],"mappings":";;;;;;;;;AAwBO,MAAM,UAAA,GAAa;AAEnB,SAAS,mBAAmB,MAAA,EAAgB;AACjD,EAAA,OAAOA,sBAAA,CAAW,MAAM,CAAA,CACrB,MAAA,CAAOC,gCAAA,CAAgB,EAAE,GAAG,MAAA,EAAQ,CAAC,CAAA,CACrC,MAAA,CAAO,KAAK,CAAA;AACjB;;;;;"}
+{"version":3,"file":"util.cjs.js","sources":["../../../../src/database/operations/stitcher/util.ts"],"sourcesContent":["/*\n * Copyright 2021 The Backstage Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n *     http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\nimport { Entity } from '@backstage/catalog-model';\nimport { createHash } from 'node:crypto';\nimport stableStringify from 'fast-json-stable-stringify';\n\n// The number of items that are sent per batch to the database layer, when\n// doing .batchInsert calls to knex. This needs to be low enough to not cause\n// errors in the underlying engine due to exceeding query limits, but large\n// enough to get the speed benefits.\nexport const BATCH_SIZE = 50;\n\n// The SOH (Start of Heading) control character, used as a stand-in for NULL\n// in contexts where NULL cannot participate in equality comparisons (SQL\n// COALESCE, JS dedup keys). It cannot appear in real entity metadata values\n// since they are human-readable strings.\nexport const NULL_SENTINEL = '\\x01';\n\nexport function generateStableHash(entity: Entity) {\n  return createHash('sha1')\n    .update(stableStringify({ ...entity }))\n    .digest('hex');\n}\n"],"names":["createHash","stableStringify"],"mappings":";;;;;;;;;AAwBO,MAAM,UAAA,GAAa;AAMnB,MAAM,aAAA,GAAgB;AAEtB,SAAS,mBAAmB,MAAA,EAAgB;AACjD,EAAA,OAAOA,sBAAA,CAAW,MAAM,CAAA,CACrB,MAAA,CAAOC,gCAAA,CAAgB,EAAE,GAAG,MAAA,EAAQ,CAAC,CAAA,CACrC,MAAA,CAAO,KAAK,CAAA;AACjB;;;;;;"}