@cosmicdrift/kumiko-framework 0.406.0 → 0.409.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agents/api/framework-db.md +1 -0
- package/agents/api/framework-engine.md +15 -2
- package/agents/api/framework-ui-types.md +15 -0
- package/agents/api/renderer-web.md +156 -2
- package/agents/api/types.md +113 -71
- package/agents/api.md +4 -4
- package/agents/boot-rules.md +743 -0
- package/agents/examples/contact.md +167 -0
- package/agents/examples/delivery-notifications.md +34 -1
- package/agents/examples/record-detail-layout.md +13 -2
- package/agents/examples/search.md +65 -3
- package/agents/examples/secrets-demo.md +96 -2
- package/agents/examples/tenant-inactivity.md +138 -0
- package/agents/examples.md +2 -0
- package/agents/features-index.md +4 -0
- package/agents/features.md +24 -1
- package/agents/widgets.md +12 -1
- package/dist/api/export/export-rows.js +1 -0
- package/dist/compliance/index.d.ts +1 -1
- package/dist/db/event-store-executor-read-where.js +15 -5
- package/dist/db/index.d.ts +1 -1
- package/dist/db/index.js +1 -1
- package/dist/db/migrate-generator.js +3 -1
- package/dist/db/queries/event-consumer.js +11 -7
- package/dist/db/queries/event-store.d.ts +2 -1
- package/dist/db/queries/event-store.js +25 -6
- package/dist/db/queries/rls-boot-catalog.js +3 -1
- package/dist/db/queries/session-advisory-lock.d.ts +7 -0
- package/dist/db/queries/session-advisory-lock.js +23 -0
- package/dist/db/query.js +7 -10
- package/dist/db/row-level-security-connection.js +4 -2
- package/dist/engine/active-membership.js +1 -0
- package/dist/engine/boot-validator/access-declarations.js +57 -33
- package/dist/engine/boot-validator/action-wiring.js +8 -6
- package/dist/engine/boot-validator/agent-effect-rules.d.ts +12 -0
- package/dist/engine/boot-validator/agent-effect-rules.js +12 -0
- package/dist/engine/boot-validator/agent-effect.d.ts +2 -0
- package/dist/engine/boot-validator/agent-effect.js +20 -0
- package/dist/engine/boot-validator/boot-context.js +5 -1
- package/dist/engine/boot-validator/boot-environment-rules.d.ts +12 -0
- package/dist/engine/boot-validator/boot-environment-rules.js +12 -0
- package/dist/engine/boot-validator/boot-phases-app.js +5 -1
- package/dist/engine/boot-validator/boot-phases.js +3 -1
- package/dist/engine/boot-validator/boot-rules.d.ts +2619 -0
- package/dist/engine/boot-validator/boot-rules.js +58 -0
- package/dist/engine/boot-validator/bundled-lists-table.js +23 -9
- package/dist/engine/boot-validator/config-deps.js +71 -23
- package/dist/engine/boot-validator/config-edit-action-rules.d.ts +17 -0
- package/dist/engine/boot-validator/config-edit-action-rules.js +17 -0
- package/dist/engine/boot-validator/config-key-rules.d.ts +117 -0
- package/dist/engine/boot-validator/config-key-rules.js +117 -0
- package/dist/engine/boot-validator/custom-screen-write-qns.js +6 -2
- package/dist/engine/boot-validator/derived-sql-fields-rules.d.ts +77 -0
- package/dist/engine/boot-validator/derived-sql-fields-rules.js +77 -0
- package/dist/engine/boot-validator/derived-sql-fields.js +53 -17
- package/dist/engine/boot-validator/detail-screens.js +10 -4
- package/dist/engine/boot-validator/entity-declaration-rules.d.ts +92 -0
- package/dist/engine/boot-validator/entity-declaration-rules.js +92 -0
- package/dist/engine/boot-validator/entity-handler-access-rules.d.ts +102 -0
- package/dist/engine/boot-validator/entity-handler-access-rules.js +102 -0
- package/dist/engine/boot-validator/entity-handler.d.ts +0 -7
- package/dist/engine/boot-validator/entity-handler.js +259 -355
- package/dist/engine/boot-validator/entity-image-variants.d.ts +2 -0
- package/dist/engine/boot-validator/entity-image-variants.js +91 -0
- package/dist/engine/boot-validator/entity-list-aggregate-options.js +8 -2
- package/dist/engine/boot-validator/entity-reference-embedded-rules.d.ts +117 -0
- package/dist/engine/boot-validator/entity-reference-embedded-rules.js +117 -0
- package/dist/engine/boot-validator/entity-select-image-rules.d.ts +122 -0
- package/dist/engine/boot-validator/entity-select-image-rules.js +122 -0
- package/dist/engine/boot-validator/entity-select.d.ts +8 -0
- package/dist/engine/boot-validator/entity-select.js +197 -0
- package/dist/engine/boot-validator/entity-structure-rules.d.ts +107 -0
- package/dist/engine/boot-validator/entity-structure-rules.js +107 -0
- package/dist/engine/boot-validator/field-conditions-rules.d.ts +92 -0
- package/dist/engine/boot-validator/field-conditions-rules.js +92 -0
- package/dist/engine/boot-validator/field-conditions.js +67 -22
- package/dist/engine/boot-validator/form-derivations-rules.d.ts +77 -0
- package/dist/engine/boot-validator/form-derivations-rules.js +77 -0
- package/dist/engine/boot-validator/form-derivations.js +52 -18
- package/dist/engine/boot-validator/global-tenancy.js +13 -7
- package/dist/engine/boot-validator/handler-declaration-rules.d.ts +72 -0
- package/dist/engine/boot-validator/handler-declaration-rules.js +72 -0
- package/dist/engine/boot-validator/handler-field-references-rules.d.ts +22 -0
- package/dist/engine/boot-validator/handler-field-references-rules.js +22 -0
- package/dist/engine/boot-validator/handler-field-references.js +14 -6
- package/dist/engine/boot-validator/list-bulk-actions-rules.d.ts +42 -0
- package/dist/engine/boot-validator/list-bulk-actions-rules.js +42 -0
- package/dist/engine/boot-validator/list-bulk-actions.js +29 -11
- package/dist/engine/boot-validator/list-column-layout.js +22 -10
- package/dist/engine/boot-validator/list-query-rules.d.ts +107 -0
- package/dist/engine/boot-validator/list-query-rules.js +107 -0
- package/dist/engine/boot-validator/list-table-aggregate-options.js +43 -15
- package/dist/engine/boot-validator/list-table-aggregate-query.js +7 -3
- package/dist/engine/boot-validator/list-table-aggregate-rules.d.ts +82 -0
- package/dist/engine/boot-validator/list-table-aggregate-rules.js +82 -0
- package/dist/engine/boot-validator/list-table-export-options.js +19 -7
- package/dist/engine/boot-validator/list-table-options.js +33 -13
- package/dist/engine/boot-validator/list-table-reject-unknown-keys.js +6 -2
- package/dist/engine/boot-validator/list-table-rules.d.ts +132 -0
- package/dist/engine/boot-validator/list-table-rules.js +132 -0
- package/dist/engine/boot-validator/nav-rules.d.ts +42 -0
- package/dist/engine/boot-validator/nav-rules.js +42 -0
- package/dist/engine/boot-validator/nav.js +29 -15
- package/dist/engine/boot-validator/ownership-rules.d.ts +27 -0
- package/dist/engine/boot-validator/ownership-rules.js +27 -0
- package/dist/engine/boot-validator/ownership.js +28 -12
- package/dist/engine/boot-validator/parent-ref.js +32 -20
- package/dist/engine/boot-validator/projection-list-screens.js +76 -35
- package/dist/engine/boot-validator/query-output-columns.js +9 -3
- package/dist/engine/boot-validator/query-refs.js +20 -10
- package/dist/engine/boot-validator/query-rules.d.ts +37 -0
- package/dist/engine/boot-validator/query-rules.js +37 -0
- package/dist/engine/boot-validator/read-gate.js +14 -10
- package/dist/engine/boot-validator/record-owned.js +5 -1
- package/dist/engine/boot-validator/related-list-table-rules.js +17 -7
- package/dist/engine/boot-validator/screens-actions-rules.d.ts +185 -0
- package/dist/engine/boot-validator/screens-actions-rules.js +185 -0
- package/dist/engine/boot-validator/screens-actions.js +169 -93
- package/dist/engine/boot-validator/screens-dashboard-chart-rules.d.ts +87 -0
- package/dist/engine/boot-validator/screens-dashboard-chart-rules.js +87 -0
- package/dist/engine/boot-validator/screens-dashboard-rules.d.ts +227 -0
- package/dist/engine/boot-validator/screens-dashboard-rules.js +227 -0
- package/dist/engine/boot-validator/screens-dashboard.js +222 -85
- package/dist/engine/boot-validator/screens-forms-layout-rules.d.ts +122 -0
- package/dist/engine/boot-validator/screens-forms-layout-rules.js +122 -0
- package/dist/engine/boot-validator/screens-forms-rules.d.ts +152 -0
- package/dist/engine/boot-validator/screens-forms-rules.js +152 -0
- package/dist/engine/boot-validator/screens-forms.js +240 -112
- package/dist/engine/boot-validator/screens-kind-config-edit.js +72 -16
- package/dist/engine/boot-validator/screens-kind-entity-edit.js +15 -7
- package/dist/engine/boot-validator/screens-kind-entity-list.js +57 -32
- package/dist/engine/boot-validator/screens-kind-projection-detail-rules.d.ts +92 -0
- package/dist/engine/boot-validator/screens-kind-projection-detail-rules.js +92 -0
- package/dist/engine/boot-validator/screens-kind-projection-detail.js +85 -47
- package/dist/engine/boot-validator/screens-kind-rules.d.ts +137 -0
- package/dist/engine/boot-validator/screens-kind-rules.js +137 -0
- package/dist/engine/boot-validator/screens-kind-secret-mint.js +18 -8
- package/dist/engine/boot-validator/screens-kind-shared.js +40 -22
- package/dist/engine/boot-validator/screens-lists-rules.d.ts +82 -0
- package/dist/engine/boot-validator/screens-lists-rules.js +82 -0
- package/dist/engine/boot-validator/screens-lists.js +90 -45
- package/dist/engine/boot-validator/screens-rules.d.ts +57 -0
- package/dist/engine/boot-validator/screens-rules.js +57 -0
- package/dist/engine/boot-validator/screens.js +17 -12
- package/dist/engine/boot-validator/transaction-phases.js +12 -4
- package/dist/engine/boot-validator/transfer-graph.js +17 -11
- package/dist/engine/boot-validator/url-prefill-fields.js +9 -5
- package/dist/engine/boot-validator/workspaces.js +11 -5
- package/dist/engine/boot-validator/zod-shape.js +6 -2
- package/dist/engine/entity-handlers.d.ts +1 -0
- package/dist/engine/entity-handlers.js +3 -0
- package/dist/engine/entity-list-runtime-options.js +23 -29
- package/dist/engine/extensions/tenant-data.d.ts +3 -0
- package/dist/engine/factories.d.ts +1 -1
- package/dist/engine/factories.js +4 -3
- package/dist/engine/index.d.ts +4 -2
- package/dist/engine/index.js +2 -0
- package/dist/engine/project-app-schema-for-roles.js +29 -14
- package/dist/engine/registry-facade.js +7 -8
- package/dist/engine/registry-validate.js +5 -1
- package/dist/engine/soft-delete-cleanup.d.ts +1 -0
- package/dist/engine/soft-delete-cleanup.js +5 -0
- package/dist/engine/steps/describe-workflow-step-error.js +1 -0
- package/dist/es-ops/operations-schema.d.ts +1 -1
- package/dist/event-store/ensure-dead-letter-table.d.ts +2 -0
- package/dist/event-store/ensure-dead-letter-table.js +10 -0
- package/dist/event-store/rebuild-dead-letter.js +2 -8
- package/dist/event-store/upcaster-dead-letter.js +2 -8
- package/dist/files/copy-pending-file-refs-to-tenant.d.ts +28 -0
- package/dist/files/copy-pending-file-refs-to-tenant.js +87 -0
- package/dist/files/file-routes.d.ts +1 -1
- package/dist/files/file-routes.js +21 -6
- package/dist/files/index.d.ts +2 -0
- package/dist/files/index.js +1 -0
- package/dist/i18n/required-surface-keys.js +1 -0
- package/dist/jobs/job-runner-helpers.js +1 -0
- package/dist/jobs/job-runner.js +7 -12
- package/dist/logging/utils.js +30 -60
- package/dist/observability/metrics-handle.js +10 -21
- package/dist/pipeline/cascade-handler.js +7 -6
- package/dist/pipeline/dispatch-query.js +18 -5
- package/dist/pipeline/dispatch-shared.d.ts +1 -0
- package/dist/pipeline/dispatch-shared.js +3 -1
- package/dist/pipeline/dispatch-write.js +10 -12
- package/dist/pipeline/dispatcher.js +12 -14
- package/dist/pipeline/event-dispatcher-admin.js +9 -8
- package/dist/pipeline/event-dispatcher-delivery.d.ts +19 -1
- package/dist/pipeline/event-dispatcher-delivery.js +121 -29
- package/dist/pipeline/event-dispatcher.d.ts +5 -1
- package/dist/pipeline/event-dispatcher.js +28 -21
- package/dist/pipeline/handler-context-phase-denials.js +2 -0
- package/dist/pipeline/multi-stream-apply-context.js +12 -20
- package/dist/pipeline/system-hooks.d.ts +4 -1
- package/dist/pipeline/system-hooks.js +79 -6
- package/dist/pipeline/system-identity-switch.js +10 -7
- package/dist/search/build-search-snippet.d.ts +9 -0
- package/dist/search/build-search-snippet.js +40 -0
- package/dist/search/index.d.ts +1 -0
- package/dist/search/index.js +1 -0
- package/dist/search/meilisearch-adapter.d.ts +32 -0
- package/dist/search/meilisearch-adapter.js +153 -56
- package/dist/search/meilisearch-chunking.d.ts +21 -0
- package/dist/search/meilisearch-chunking.js +96 -0
- package/dist/search/reindex-entity.d.ts +1 -0
- package/dist/search/reindex-entity.js +97 -52
- package/dist/ui-types/index.d.ts +1 -1
- package/dist/upgrade-cli.js +3 -14
- package/package.json +3 -3
- package/src/changes.json +73 -0
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
import { meilisearchDocId } from "./meilisearch-adapter-internals.js";
|
|
2
|
+
// Meilisearch indexes at most 65_535 word positions per attribute, and a hard
|
|
3
|
+
// separator advances the position by up to 8. 8_000 characters yield at most
|
|
4
|
+
// 8_000 words (single-character words separated by one space), so a piece of this
|
|
5
|
+
// size stays far below the limit even in the worst case. Longer values are split
|
|
6
|
+
// into pieces instead of being cut: nothing is silently dropped from the index.
|
|
7
|
+
export const CHUNK_MAX_CHARS = 8_000;
|
|
8
|
+
export const CHUNK_OVERLAP_CHARS = 200;
|
|
9
|
+
const WHITESPACE = /\s/;
|
|
10
|
+
function lastWhitespaceBefore(value, end, floor) {
|
|
11
|
+
for (let i = end - 1; i > floor; i--) {
|
|
12
|
+
if (WHITESPACE.test(value.charAt(i)))
|
|
13
|
+
return i;
|
|
14
|
+
}
|
|
15
|
+
return -1;
|
|
16
|
+
}
|
|
17
|
+
// Start of the next piece: about CHUNK_OVERLAP_CHARS before the cut, moved forward to a word
|
|
18
|
+
// start, so a word or short phrase straddling the cut is whole in at least one piece.
|
|
19
|
+
function nextPieceStart(value, pieceStart, cut) {
|
|
20
|
+
const rawStart = Math.max(cut - CHUNK_OVERLAP_CHARS, pieceStart + 1);
|
|
21
|
+
for (let i = rawStart; i < cut; i++) {
|
|
22
|
+
if (WHITESPACE.test(value.charAt(i - 1)))
|
|
23
|
+
return i;
|
|
24
|
+
}
|
|
25
|
+
return cut;
|
|
26
|
+
}
|
|
27
|
+
/** Splits a long value into pieces of at most CHUNK_MAX_CHARS, cut at whitespace. */
|
|
28
|
+
export function splitLongText(value) {
|
|
29
|
+
if (value.length <= CHUNK_MAX_CHARS)
|
|
30
|
+
return [value];
|
|
31
|
+
const pieces = [];
|
|
32
|
+
let start = 0;
|
|
33
|
+
while (value.length - start > CHUNK_MAX_CHARS) {
|
|
34
|
+
const limit = start + CHUNK_MAX_CHARS;
|
|
35
|
+
const space = lastWhitespaceBefore(value, limit, start + CHUNK_OVERLAP_CHARS);
|
|
36
|
+
const cut = space === -1 ? limit : space;
|
|
37
|
+
pieces.push(value.slice(start, cut));
|
|
38
|
+
start = nextPieceStart(value, start, cut);
|
|
39
|
+
}
|
|
40
|
+
pieces.push(value.slice(start));
|
|
41
|
+
return pieces;
|
|
42
|
+
}
|
|
43
|
+
export function pieceDocId(baseId, chunk) {
|
|
44
|
+
return chunk === 0 ? baseId : `${baseId}_c${chunk}`;
|
|
45
|
+
}
|
|
46
|
+
/**
|
|
47
|
+
* One Meilisearch document per piece. Piece 0 keeps the id the entity always had (existing
|
|
48
|
+
* documents stay valid, no duplicates) and carries all short fields; piece n holds the n-th
|
|
49
|
+
* parts of the long fields.
|
|
50
|
+
*/
|
|
51
|
+
export function buildPieceDocuments(doc) {
|
|
52
|
+
const shortFields = {};
|
|
53
|
+
const longPieces = [];
|
|
54
|
+
for (const [key, value] of Object.entries(doc.fields)) {
|
|
55
|
+
if (typeof value === "string" && value.length > CHUNK_MAX_CHARS) {
|
|
56
|
+
longPieces.push([key, splitLongText(value)]);
|
|
57
|
+
}
|
|
58
|
+
else {
|
|
59
|
+
shortFields[key] = value;
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
const baseId = meilisearchDocId(doc.entityType, doc.entityId);
|
|
63
|
+
const pieceCount = Math.max(1, ...longPieces.map(([, pieces]) => pieces.length));
|
|
64
|
+
const documents = [];
|
|
65
|
+
for (let chunk = 0; chunk < pieceCount; chunk++) {
|
|
66
|
+
const piece = chunk === 0 ? { ...shortFields } : {};
|
|
67
|
+
for (const [key, pieces] of longPieces) {
|
|
68
|
+
const part = pieces[chunk];
|
|
69
|
+
if (part !== undefined)
|
|
70
|
+
piece[key] = part;
|
|
71
|
+
}
|
|
72
|
+
documents.push({
|
|
73
|
+
...piece,
|
|
74
|
+
_id: pieceDocId(baseId, chunk),
|
|
75
|
+
_type: doc.entityType,
|
|
76
|
+
_weight: doc.weight,
|
|
77
|
+
_entityId: doc.entityId,
|
|
78
|
+
_chunk: chunk,
|
|
79
|
+
});
|
|
80
|
+
}
|
|
81
|
+
return documents;
|
|
82
|
+
}
|
|
83
|
+
function quoted(value) {
|
|
84
|
+
return `"${value.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"`;
|
|
85
|
+
}
|
|
86
|
+
/** Filter matching every piece (and a legacy unchunked document) of one entity. */
|
|
87
|
+
export function entityPiecesFilter(entityType, entityId) {
|
|
88
|
+
return `(_type = ${quoted(entityType)} AND _entityId = ${quoted(String(entityId))})`;
|
|
89
|
+
}
|
|
90
|
+
/** Filter matching the pieces of one entity from `fromChunk` on: the leftovers of a longer earlier version. */
|
|
91
|
+
export function stalePiecesFilter(entityType, entityId, fromChunk) {
|
|
92
|
+
return `(_type = ${quoted(entityType)} AND _entityId = ${quoted(String(entityId))} AND _chunk >= ${fromChunk})`;
|
|
93
|
+
}
|
|
94
|
+
export function filterTypeClause(entityType) {
|
|
95
|
+
return `_type = ${quoted(entityType)}`;
|
|
96
|
+
}
|
|
@@ -14,5 +14,6 @@ export type ReindexEntityResult = {
|
|
|
14
14
|
export type ReindexEntityOptions = {
|
|
15
15
|
readonly batchSize?: number;
|
|
16
16
|
readonly dryRun?: boolean;
|
|
17
|
+
readonly afterBatchRead?: () => Promise<void>;
|
|
17
18
|
};
|
|
18
19
|
export declare function reindexEntity(db: DbRunner, registry: Registry, searchAdapter: SearchAdapter, entityName: string, tenantId: TenantId, options?: ReindexEntityOptions): Promise<ReindexEntityResult>;
|
|
@@ -40,6 +40,87 @@ async function readBatchWithoutStatementTimeout(db, sqlText, params) {
|
|
|
40
40
|
return executeRawQueryRead(tx, sqlText, params);
|
|
41
41
|
});
|
|
42
42
|
}
|
|
43
|
+
async function buildBatchDocs(batch, rows, failures) {
|
|
44
|
+
const docs = [];
|
|
45
|
+
for (const row of rows) {
|
|
46
|
+
const entityId = String(row["id"]);
|
|
47
|
+
try {
|
|
48
|
+
const rawState = rowToState(row, batch.fieldNames);
|
|
49
|
+
// `pii + searchable` fields hold ciphertext on the read-table row —
|
|
50
|
+
// decrypt the same way the live write-path consumer does. An
|
|
51
|
+
// erased subject key doesn't throw, it swaps the value for
|
|
52
|
+
// PII_ERASED_SENTINEL (see decryptSearchableSubjectFields) — check
|
|
53
|
+
// for that AFTER decrypting, mirroring the softDelete filter in reindexEntity:
|
|
54
|
+
// resurrecting a crypto-shredded row here would undo
|
|
55
|
+
// purgeSearchDocumentsForSubject and make it findable again.
|
|
56
|
+
const state = await decryptSearchableSubjectFields(batch.entityName, rawState, batch.registry);
|
|
57
|
+
if (hasErasedSearchableSubjectField(batch.entityName, state, batch.registry))
|
|
58
|
+
continue;
|
|
59
|
+
const doc = await buildSearchDocument(batch.entityName, entityId, state, batch.registry);
|
|
60
|
+
if (doc)
|
|
61
|
+
docs.push({ entityId, doc });
|
|
62
|
+
}
|
|
63
|
+
catch (e) {
|
|
64
|
+
failures.push({ entityId, reason: e instanceof Error ? e.message : String(e) });
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
return docs;
|
|
68
|
+
}
|
|
69
|
+
async function writeBatchDocs(searchAdapter, tenantId, docs, failures) {
|
|
70
|
+
if (searchAdapter.indexBatch) {
|
|
71
|
+
try {
|
|
72
|
+
await searchAdapter.indexBatch(tenantId, docs.map((d) => d.doc));
|
|
73
|
+
return docs.length;
|
|
74
|
+
}
|
|
75
|
+
catch (e) {
|
|
76
|
+
const reason = e instanceof Error ? e.message : String(e);
|
|
77
|
+
for (const { entityId } of docs)
|
|
78
|
+
failures.push({ entityId, reason });
|
|
79
|
+
return 0;
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
let written = 0;
|
|
83
|
+
for (const { entityId, doc } of docs) {
|
|
84
|
+
try {
|
|
85
|
+
await searchAdapter.index(tenantId, doc);
|
|
86
|
+
written++;
|
|
87
|
+
}
|
|
88
|
+
catch (e) {
|
|
89
|
+
failures.push({ entityId, reason: e instanceof Error ? e.message : String(e) });
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
return written;
|
|
93
|
+
}
|
|
94
|
+
// The live consumer keeps running while a reindex scans. A row deleted, purged or edited after
|
|
95
|
+
// its batch was read would otherwise leave stale text in the index, the one thing a delete or a
|
|
96
|
+
// subject purge must not leave behind. So the written ids are read again: rows that vanished,
|
|
97
|
+
// were soft-deleted or became erased leave the index, rows whose version moved are rewritten fresh.
|
|
98
|
+
async function reconcileWrittenBatch(batch, searchAdapter, tenantId, readRows, written, failures) {
|
|
99
|
+
const writtenIds = new Set(written.map((w) => w.entityId));
|
|
100
|
+
const versionAtRead = new Map(readRows.map((row) => [String(row["id"]), row["version"]]));
|
|
101
|
+
const ids = [...writtenIds];
|
|
102
|
+
const placeholders = ids.map((_, i) => `$${i + 2}`).join(", ");
|
|
103
|
+
const fresh = await readBatchWithoutStatementTimeout(batch.db, `SELECT * FROM ${quoteIdent(batch.tableName)}
|
|
104
|
+
WHERE ${quoteIdent("tenant_id")} = $1 AND ${quoteIdent("id")} IN (${placeholders})`, [tenantId, ...ids]);
|
|
105
|
+
const live = fresh.filter((row) => !(batch.deletedFilter && row["is_deleted"] === true));
|
|
106
|
+
const liveIds = new Set(live.map((row) => String(row["id"])));
|
|
107
|
+
const changed = live.filter((row) => row["version"] !== versionAtRead.get(String(row["id"])));
|
|
108
|
+
const rewritten = await buildBatchDocs(batch, changed, failures);
|
|
109
|
+
const rewrittenIds = new Set(rewritten.map((w) => w.entityId));
|
|
110
|
+
const changedIds = new Set(changed.map((row) => String(row["id"])));
|
|
111
|
+
const toRemove = ids
|
|
112
|
+
.filter((id) => !liveIds.has(id) || (changedIds.has(id) && !rewrittenIds.has(id)))
|
|
113
|
+
.map((entityId) => ({ entityType: batch.entityName, entityId }));
|
|
114
|
+
if (toRemove.length > 0) {
|
|
115
|
+
if (searchAdapter.removeBatch)
|
|
116
|
+
await searchAdapter.removeBatch(tenantId, toRemove);
|
|
117
|
+
else
|
|
118
|
+
for (const item of toRemove)
|
|
119
|
+
await searchAdapter.remove(tenantId, item.entityType, item.entityId);
|
|
120
|
+
}
|
|
121
|
+
if (rewritten.length > 0)
|
|
122
|
+
await writeBatchDocs(searchAdapter, tenantId, rewritten, failures);
|
|
123
|
+
}
|
|
43
124
|
export async function reindexEntity(db, registry, searchAdapter, entityName, tenantId, options = {}) {
|
|
44
125
|
return runInSystemRowLevelSecurityScope("search reindex reads entity rows of all tenants", async () => {
|
|
45
126
|
const entity = registry.getEntity(entityName);
|
|
@@ -61,6 +142,14 @@ export async function reindexEntity(db, registry, searchAdapter, entityName, ten
|
|
|
61
142
|
const deletedFilter = entity.softDelete === true ? `AND ${quoteIdent("is_deleted")} IS NOT TRUE` : "";
|
|
62
143
|
const failures = [];
|
|
63
144
|
const result = { scannedRows: 0, indexedRows: 0, wouldIndexRows: 0, failures };
|
|
145
|
+
const batch = {
|
|
146
|
+
db,
|
|
147
|
+
registry,
|
|
148
|
+
entityName,
|
|
149
|
+
tableName,
|
|
150
|
+
fieldNames,
|
|
151
|
+
deletedFilter: entity.softDelete === true,
|
|
152
|
+
};
|
|
64
153
|
let lastId;
|
|
65
154
|
// Whether a searchable field lacks a matching read-table column depends
|
|
66
155
|
// only on the table's column set (rowToState/Object.hasOwn), not on any
|
|
@@ -89,60 +178,16 @@ export async function reindexEntity(db, registry, searchAdapter, entityName, ten
|
|
|
89
178
|
`column ${quoteIdent(tableName)} — likely a dropped/never-migrated column.`);
|
|
90
179
|
}
|
|
91
180
|
}
|
|
92
|
-
const docs =
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
try {
|
|
97
|
-
const rawState = rowToState(row, fieldNames);
|
|
98
|
-
// `pii + searchable` fields hold ciphertext on the read-table row —
|
|
99
|
-
// decrypt the same way the live write-path consumer does. An
|
|
100
|
-
// erased subject key doesn't throw, it swaps the value for
|
|
101
|
-
// PII_ERASED_SENTINEL (see decryptSearchableSubjectFields) — check
|
|
102
|
-
// for that AFTER decrypting, mirroring the softDelete filter above:
|
|
103
|
-
// resurrecting a crypto-shredded row here would undo
|
|
104
|
-
// purgeSearchDocumentsForSubject and make it findable again.
|
|
105
|
-
const state = await decryptSearchableSubjectFields(entityName, rawState, registry);
|
|
106
|
-
if (hasErasedSearchableSubjectField(entityName, state, registry))
|
|
107
|
-
continue;
|
|
108
|
-
const doc = await buildSearchDocument(entityName, entityId, state, registry);
|
|
109
|
-
if (doc)
|
|
110
|
-
docs.push({ entityId, doc });
|
|
111
|
-
}
|
|
112
|
-
catch (e) {
|
|
113
|
-
result.failures.push({ entityId, reason: e instanceof Error ? e.message : String(e) });
|
|
114
|
-
}
|
|
115
|
-
}
|
|
116
|
-
if (!options.dryRun && docs.length > 0) {
|
|
117
|
-
if (searchAdapter.indexBatch) {
|
|
118
|
-
try {
|
|
119
|
-
await searchAdapter.indexBatch(tenantId, docs.map((d) => d.doc));
|
|
120
|
-
result.indexedRows += docs.length;
|
|
121
|
-
}
|
|
122
|
-
catch (e) {
|
|
123
|
-
const reason = e instanceof Error ? e.message : String(e);
|
|
124
|
-
for (const { entityId } of docs)
|
|
125
|
-
result.failures.push({ entityId, reason });
|
|
126
|
-
}
|
|
127
|
-
}
|
|
128
|
-
else {
|
|
129
|
-
for (const { entityId, doc } of docs) {
|
|
130
|
-
try {
|
|
131
|
-
await searchAdapter.index(tenantId, doc);
|
|
132
|
-
result.indexedRows++;
|
|
133
|
-
}
|
|
134
|
-
catch (e) {
|
|
135
|
-
result.failures.push({
|
|
136
|
-
entityId,
|
|
137
|
-
reason: e instanceof Error ? e.message : String(e),
|
|
138
|
-
});
|
|
139
|
-
}
|
|
140
|
-
}
|
|
141
|
-
}
|
|
142
|
-
}
|
|
143
|
-
else if (options.dryRun) {
|
|
181
|
+
const docs = await buildBatchDocs(batch, rows, result.failures);
|
|
182
|
+
result.scannedRows += rows.length;
|
|
183
|
+
await options.afterBatchRead?.();
|
|
184
|
+
if (options.dryRun) {
|
|
144
185
|
result.wouldIndexRows += docs.length;
|
|
145
186
|
}
|
|
187
|
+
else if (docs.length > 0) {
|
|
188
|
+
result.indexedRows += await writeBatchDocs(searchAdapter, tenantId, docs, result.failures);
|
|
189
|
+
await reconcileWrittenBatch(batch, searchAdapter, tenantId, rows, docs, result.failures);
|
|
190
|
+
}
|
|
146
191
|
lastId = rows[rows.length - 1]?.["id"];
|
|
147
192
|
if (rows.length < batchSize)
|
|
148
193
|
break;
|
package/dist/ui-types/index.d.ts
CHANGED
|
@@ -11,7 +11,7 @@ export type { EmbeddedDerivedCellDef, EntityDefinition, FieldDefinition, Options
|
|
|
11
11
|
export type { AccessRule } from "../engine/types/handlers.ts";
|
|
12
12
|
export { isOpenToAllGranted, isUiAccessGranted } from "../engine/types/handlers.ts";
|
|
13
13
|
export type { IconKey, NavDefinition, NavIconKey } from "../engine/types/nav.ts";
|
|
14
|
-
export type { ActionFormFooterAction, ActionFormLoad, ActionFormRedirect, ActionFormScreenDefinition, ActionFormSuccessFact, ActionFormSuccessLink, ConfigEditScreenDefinition, CustomScreenDefinition, DashboardChartMarkerKind, DashboardChartPanel, DashboardCurrencyFormat, DashboardCustomPanel, DashboardDateParam, DashboardFeedPanel, DashboardFilterDefinition, DashboardI18nText, DashboardListPanel, DashboardMoneyParam, DashboardPanelDefinition, DashboardPanelEmptyState, DashboardPanelQueryOptions, DashboardPanelSpan, DashboardPanelVisibility, DashboardProgressListPanel, DashboardProgressTone, DashboardRefresh, DashboardRelativeTimeParam, DashboardScopeDefinition, DashboardScreenDefinition, DashboardScreenPanel, DashboardStatGroupPanel, DashboardStatPanel, DashboardText, DashboardTextBlock, DashboardTextPanel, DashboardTextParam, DashboardTimeRangeDefinition, EditExtensionSection, EditFieldSpec, EditFieldsSection, EditLayout, EditRelatedListSection, EditSectionSpec, EditWriteFormSection, EmbeddedListEditSpec, EntityEditAction, EntityEditScreenDefinition, EntityListFacetConfig, EntityListScreenDefinition, FieldAffix, FieldCondition, FieldFillAction, FieldIconKey, FieldRenderer, FormWidth, ListBulkAction, ListColumnSpec, ListEmptyStateSpec, ListFacetSpec, ListSortSpec, ListTableOptions, MetricNavigate, PlatformComponent, ProjectionDetailScreenDefinition, ProjectionListScreenDefinition, RecordHeaderSubtitlePart, RelatedListEmptyStateSpec, RelatedListEmptyStateVariant, RelatedListRowTone, RelatedListToolbarAction, RowAction, RowActionClientAction, RowActionDisplay, RowActionDrawer, RowActionMode, RowActionNavigate, RowActionWriteHandler, RowFieldExtractor, ScreenDefinition, ScreenFallback, ScreenFallbackByReason, ScreenFallbackReason, ScreenSlots, ScreenVisibility, SecretMintConfirmStep, SecretMintScreenDefinition, SecretsEditScreenDefinition, ToolbarAction, } from "../engine/types/screen.ts";
|
|
14
|
+
export type { ActionFormFooterAction, ActionFormLoad, ActionFormRedirect, ActionFormScreenDefinition, ActionFormSuccessFact, ActionFormSuccessItems, ActionFormSuccessLink, ConfigEditAction, ConfigEditScreenDefinition, CustomScreenDefinition, DashboardChartMarkerKind, DashboardChartPanel, DashboardCurrencyFormat, DashboardCustomPanel, DashboardDateParam, DashboardFeedPanel, DashboardFilterDefinition, DashboardI18nText, DashboardListPanel, DashboardMoneyParam, DashboardPanelDefinition, DashboardPanelEmptyState, DashboardPanelQueryOptions, DashboardPanelSpan, DashboardPanelVisibility, DashboardProgressListPanel, DashboardProgressTone, DashboardRefresh, DashboardRelativeTimeParam, DashboardScopeDefinition, DashboardScreenDefinition, DashboardScreenPanel, DashboardStatGroupPanel, DashboardStatPanel, DashboardText, DashboardTextBlock, DashboardTextPanel, DashboardTextParam, DashboardTimeRangeDefinition, EditExtensionSection, EditFieldSpec, EditFieldsSection, EditLayout, EditRelatedListSection, EditSectionSpec, EditWriteFormSection, EmbeddedListEditSpec, EntityEditAction, EntityEditScreenDefinition, EntityListFacetConfig, EntityListScreenDefinition, FieldAffix, FieldCondition, FieldFillAction, FieldIconKey, FieldRenderer, FormWidth, ListBulkAction, ListColumnSpec, ListEmptyStateSpec, ListFacetSpec, ListSortSpec, ListTableOptions, MetricNavigate, PlatformComponent, ProjectionDetailScreenDefinition, ProjectionListScreenDefinition, RecordHeaderSubtitlePart, RelatedListEmptyStateSpec, RelatedListEmptyStateVariant, RelatedListRowTone, RelatedListToolbarAction, RowAction, RowActionClientAction, RowActionDisplay, RowActionDrawer, RowActionMode, RowActionNavigate, RowActionWriteHandler, RowFieldExtractor, ScreenDefinition, ScreenFallback, ScreenFallbackByReason, ScreenFallbackReason, ScreenSlots, ScreenVisibility, SecretMintConfirmStep, SecretMintScreenDefinition, SecretsEditScreenDefinition, ToolbarAction, } from "../engine/types/screen.ts";
|
|
15
15
|
export { MAX_BULK_SELECTION } from "../engine/types/screen.ts";
|
|
16
16
|
export type { TargetRef } from "../engine/types/target-ref.ts";
|
|
17
17
|
export type { TreeAction } from "../engine/types/tree-node.ts";
|
package/dist/upgrade-cli.js
CHANGED
|
@@ -276,18 +276,6 @@ function carryOverManual(previous, newManual) {
|
|
|
276
276
|
...(previous.resolvedManual.length > 0 && { resolvedManual: previous.resolvedManual }),
|
|
277
277
|
};
|
|
278
278
|
}
|
|
279
|
-
// The marker records every codemod ever applied, not just the last bump's, so the
|
|
280
|
-
// CI guard and a resumed run see the whole history.
|
|
281
|
-
function accumulateCodemods(previous, ran) {
|
|
282
|
-
const seen = new Set();
|
|
283
|
-
return [...previous, ...ran].filter((entry) => {
|
|
284
|
-
const key = `${entry.version}\0${entry.codemod}`;
|
|
285
|
-
if (seen.has(key))
|
|
286
|
-
return false;
|
|
287
|
-
seen.add(key);
|
|
288
|
-
return true;
|
|
289
|
-
});
|
|
290
|
-
}
|
|
291
279
|
function writeUpgradeMarker(targetDir, marker) {
|
|
292
280
|
const dir = join(targetDir, ".kumiko");
|
|
293
281
|
mkdirSync(dir, { recursive: true });
|
|
@@ -380,7 +368,8 @@ function writePartialMarker(out, targetDir, pending, manualEntries, failedVersio
|
|
|
380
368
|
writeUpgradeMarker(targetDir, {
|
|
381
369
|
version,
|
|
382
370
|
appliedAt: Temporal.Now.instant().toString(),
|
|
383
|
-
|
|
371
|
+
// The marker keeps every codemod ever applied, so the CI guard and a resumed run see the whole history.
|
|
372
|
+
codemods: uniqueCodemodRunsPerVersion([...previous.codemods, ...ran]),
|
|
384
373
|
...carryOverManual(previous, manualEntries.map(toMarkerManual)),
|
|
385
374
|
});
|
|
386
375
|
out.err(` ⚠ ${ran.length} codemod(s) already applied; wrote marker at ${version} so a re-run resumes at the failed one.`);
|
|
@@ -483,7 +472,7 @@ markerVersion, previous, fromVersion, manualDetails) {
|
|
|
483
472
|
writeUpgradeMarker(targetDir, {
|
|
484
473
|
version: markerVersion,
|
|
485
474
|
appliedAt: Temporal.Now.instant().toString(),
|
|
486
|
-
codemods:
|
|
475
|
+
codemods: uniqueCodemodRunsPerVersion([...previous.codemods, ...ran]),
|
|
487
476
|
...carried,
|
|
488
477
|
});
|
|
489
478
|
writeUpgradeChangelog(targetDir, {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cosmicdrift/kumiko-framework",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.409.0",
|
|
4
4
|
"description": "Framework core — engine, pipeline, API, DB, and every other bit that makes Kumiko go.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Marc Frost <marc@cosmicdriftgamestudio.com>",
|
|
@@ -203,7 +203,7 @@
|
|
|
203
203
|
"./package.json": "./package.json"
|
|
204
204
|
},
|
|
205
205
|
"dependencies": {
|
|
206
|
-
"@cosmicdrift/kumiko-types": "0.
|
|
206
|
+
"@cosmicdrift/kumiko-types": "0.409.0",
|
|
207
207
|
"bullmq": "^5.81.5",
|
|
208
208
|
"hono": "^4.13.14",
|
|
209
209
|
"i18next": "^26.5.12",
|
|
@@ -225,7 +225,7 @@
|
|
|
225
225
|
}
|
|
226
226
|
},
|
|
227
227
|
"devDependencies": {
|
|
228
|
-
"@cosmicdrift/kumiko-testing": "0.
|
|
228
|
+
"@cosmicdrift/kumiko-testing": "0.409.0",
|
|
229
229
|
"bun-types": "^1.4.4",
|
|
230
230
|
"fflate": "^0.8.3",
|
|
231
231
|
"meilisearch": "^0.62.0",
|
package/src/changes.json
CHANGED
|
@@ -1,4 +1,77 @@
|
|
|
1
1
|
[
|
|
2
|
+
{
|
|
3
|
+
"version": "0.409.0",
|
|
4
|
+
"type": "improvement",
|
|
5
|
+
"title": "All boot validation errors have a rule id",
|
|
6
|
+
"detail": "The boot validators that still threw plain errors now throw `bootRuleError` with a rule from a typed table, for example `form/affix-unknown-field`, `entity/derived-field-collision`, `list-table/unknown-keys` or `config-key/bounds-min-above-max`. A violation in `BootValidationError.violations` and in `kumiko-schema validate` (text and `--json`) carries that id instead of `unclassified/<check>`, plus a `Fix:` line and `Rule: agents/boot-rules.md#<id>`. `agents/boot-rules.md` lists every rule with summary and fix. Message texts are unchanged, so tests that match on the message keep passing; a test that asserted an `unclassified/...` id now sees the real id. Collection boundaries are unchanged: a validator that stopped at its first violation still does."
|
|
7
|
+
},
|
|
8
|
+
{
|
|
9
|
+
"version": "0.409.0",
|
|
10
|
+
"type": "fix",
|
|
11
|
+
"title": "Fully qualified screen targets of another feature boot in navigate actions",
|
|
12
|
+
"detail": "The boot validator no longer prefixes the owning feature to a navigate target that is already a `<feature>:screen:<id>` name (entityList toolbarActions, row actions, metric and detail links). Before, such a target failed boot with a doubled name like `contact:screen:publicstatus:screen:x`. Unknown qualified targets are still rejected, and a `tab` is checked against the target screen in the other feature. The renderer resolves such qualified targets like short ids: the link or action is offered when the target screen exists and is accessible, and the URL carries the short screen id."
|
|
13
|
+
},
|
|
14
|
+
{
|
|
15
|
+
"version": "0.409.0",
|
|
16
|
+
"type": "improvement",
|
|
17
|
+
"title": "SystemAdmin may read files of the system tenant from any tenant session",
|
|
18
|
+
"detail": "`GET /files/:id`, `/files/:id/meta` and `/files/:id/download-url` also serve a file owned by SYSTEM_TENANT_ID when the caller has the SystemAdmin role, whatever tenant the session is in; the storage provider is resolved for the file's tenant. The file access guard still decides. Delete, variants and upload stay tenant-equal and files of other tenants stay unreadable."
|
|
19
|
+
},
|
|
20
|
+
{
|
|
21
|
+
"version": "0.409.0",
|
|
22
|
+
"type": "improvement",
|
|
23
|
+
"title": "List payload flag onlyDeleted for trash lists",
|
|
24
|
+
"detail": "The entity list query accepts onlyDeleted: true and returns only is_deleted rows. Tenant, ownership and read scopes, filter, search, totalCount and aggregate apply unchanged, with the same access rule as includeDeleted. onlyDeleted together with includeDeleted is a validation error, as is onlyDeleted on an entity without softDelete. Custom query handlers read it as ctx.onlyDeleted; export payloads drop it like includeDeleted."
|
|
25
|
+
},
|
|
26
|
+
{
|
|
27
|
+
"version": "0.409.0",
|
|
28
|
+
"type": "improvement",
|
|
29
|
+
"title": "longText find \"fuzzy\", chunked Meilisearch documents, buildSearchSnippet",
|
|
30
|
+
"detail": "createLongTextField accepts find: \"fuzzy\" (no blind-index lookup, \"exact\" stays text-only); the decrypted value goes into the search document like any other searchable field. The Meilisearch adapter splits string fields over 8,000 characters into pieces cut at whitespace with about 200 characters of overlap, because Meilisearch indexes at most 65,535 word positions per attribute; nothing is cut off. Piece 0 keeps the document id every entity already had, further pieces are `<id>_c<n>` with _entityId and _chunk, the index gets distinctAttribute _entityId (an entity matches once) and the filterable attributes _entityId and _chunk. index/indexBatch delete leftover pieces of an earlier longer version, remove/removeBatch delete all pieces of an entity by filter. Default-configured indexes from before are upgraded on first use. New buildSearchSnippet(text, term, { radius }) from @cosmicdrift/kumiko-framework/search returns before/hit/after around the first hit. Also new: tryWithSessionAdvisoryLock (db) and the es-ops operation type search-reindex.",
|
|
31
|
+
"migration": "None. On the first start after the update the app reindexes the affected entities in the background, no manual step. Set KUMIKO_SKIP_SEARCH_REINDEX=1 to skip it."
|
|
32
|
+
},
|
|
33
|
+
{
|
|
34
|
+
"version": "0.409.0",
|
|
35
|
+
"type": "fix",
|
|
36
|
+
"title": "reindexEntity reconciles each written batch against the live table",
|
|
37
|
+
"detail": "After a batch is written, reindexEntity reads the same ids again. Rows that vanished or were soft-deleted leave the index, rows whose version changed since the read are rewritten from the fresh row (or leave the index when they are erased by then). Before, a delete, purge or edit processed by the live search consumer between the read and the write of a batch could put the old text back. ReindexEntityOptions gets afterBatchRead, a hook that runs between reading and writing a batch."
|
|
38
|
+
},
|
|
39
|
+
{
|
|
40
|
+
"version": "0.409.0",
|
|
41
|
+
"type": "fix",
|
|
42
|
+
"title": "Meilisearch keeps _entityId and _chunk filterable on every index",
|
|
43
|
+
"detail": "The Meilisearch adapter merges _entityId and _chunk into the stored filterableAttributes of an index it keeps (also one configured with its own searchable attributes), instead of adding them only when it applies its default configuration. Before, removing an entity from such an index failed because the delete filter used attributes that were not filterable."
|
|
44
|
+
},
|
|
45
|
+
{
|
|
46
|
+
"version": "0.409.0",
|
|
47
|
+
"type": "fix",
|
|
48
|
+
"title": "dropAllIndexes tolerates concurrent index deletions on a shared Meilisearch",
|
|
49
|
+
"detail": "The Meilisearch adapter enqueues all index deletions of a drop together and treats an index another client already removed (index_not_found or the vanished index files) as dropped. When the index listing fails because another client is deleting an index, it waits for the deletions in flight and lists once more. The adapter's configured-tenant cache is cleared even when the listing fails."
|
|
50
|
+
},
|
|
51
|
+
{
|
|
52
|
+
"version": "0.409.0",
|
|
53
|
+
"type": "improvement",
|
|
54
|
+
"title": "Soft-delete retention period is exported",
|
|
55
|
+
"detail": "The engine barrel exports SOFT_DELETE_GRACE_DAYS_KEY, SOFT_DELETE_DEFAULT_GRACE_DAYS (30) and readSoftDeleteGraceDays(ctx.config), which returns the tenant's configured period or the default. A trash screen can show how long deleted rows stay restorable without keeping its own constant."
|
|
56
|
+
},
|
|
57
|
+
{
|
|
58
|
+
"version": "0.408.0",
|
|
59
|
+
"type": "improvement",
|
|
60
|
+
"title": "findEntity is exported from the engine entry point",
|
|
61
|
+
"detail": "findEntity(features, entityName) from @cosmicdrift/kumiko-framework/engine returns the entity definition a mounted feature declares under that name, or undefined. It reads the raw feature definitions, not the merged registry entity map. Internal duplicates in logging, jobs, metrics, event consumers, transactions, dispatch and the role projection now share helpers; messages and behavior are unchanged."
|
|
62
|
+
},
|
|
63
|
+
{
|
|
64
|
+
"version": "0.407.0",
|
|
65
|
+
"type": "fix",
|
|
66
|
+
"title": "A failing search index holds only its own tenant",
|
|
67
|
+
"detail": "The search consumer runs on one ordered cursor for all tenants, so a single tenant whose Meilisearch index rejected writes (a foreign primary key, settings changed by another writer, a poison document) halted indexing for every tenant until the consumer was dead-lettered. The consumer now holds only the failing operations, flushes everything else, and backs off per tenant from 1 to 60 seconds; while a tenant backs off, all of its operations wait so per-entity order holds. Held events stay in the consumer's pending_gaps, survive restarts and are retried first. The consumer stays idle with lastError naming the tenant, last_failed_event_id points at the failing event so skipPoisonEvent skips exactly it, and after ten failed retries in a row lastError and an error log say which event is stuck and how to skip it. Holding never counts towards maxAttempts, because dead-lettering the shared consumer is the stall for all tenants. For consumers with a batch handler, pending ids now fill at most half of a dispatcher batch, so held events cannot starve newer ones; when that cap leaves pending events out, newer events of their tenants wait as pending gaps too. Consumers without a batch handler keep the full batch for pending ids. The new dispatcher option `maxPendingGapRanges` (default 2000) bounds the cursor row: a batch consumer at the bound takes no new events and every pass that delivers nothing counts as a failed attempt, so it ends dead with lastError naming the limit, the held tenant and its first held event, and the existing rearm and rearm-exhausted alert take over. A batch handler holds events through the new third argument `turn.holdForRetry(eventIds, reason)`; existing batch handlers keep their behavior. The Meilisearch adapter forgets a tenant's configured state when a write fails, so the next write re-checks the index settings, and its task errors carry the Meilisearch error code and task uid but no longer the message, which can embed the rejected document."
|
|
68
|
+
},
|
|
69
|
+
{
|
|
70
|
+
"version": "0.407.0",
|
|
71
|
+
"type": "improvement",
|
|
72
|
+
"title": "heldFileRefIds on TenantDataHookCtx",
|
|
73
|
+
"detail": "TenantDataHookCtx has an optional heldFileRefIds set with the fileRef ids that rows held by retention still point at. Hooks that delete files must skip them. TenantDataExtensionHooks has an optional phase (\"app-data\" by default, or \"tenant-row\") that runs a hook right before the tenant row is tombstoned, after the hold check. Existing hooks keep compiling and keep their behavior."
|
|
74
|
+
},
|
|
2
75
|
{
|
|
3
76
|
"version": "0.406.0",
|
|
4
77
|
"type": "improvement",
|