hikoutei 0.3.2 → 0.3.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +12 -2
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmMappedObservation.d.ts +2 -0
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmMappedObservation.d.ts.map +1 -1
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmMappedObservation.js +5 -0
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmMappedObservation.js.map +1 -1
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPolling.d.ts +52 -0
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPolling.d.ts.map +1 -1
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPolling.js +195 -13
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPolling.js.map +1 -1
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingFastPath.d.ts +24 -0
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingFastPath.d.ts.map +1 -0
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingFastPath.js +124 -0
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingFastPath.js.map +1 -0
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingInspection.js +1 -1
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingInspection.js.map +1 -1
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingPersistence.d.ts +15 -2
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingPersistence.d.ts.map +1 -1
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingPersistence.js +27 -3
- package/dist/adapter/persistence/providers/mikro-orm/observation/MikroOrmUserInputPollingPersistence.js.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/errors.d.ts +1 -0
- package/dist/adapter/sheets/providers/apps-script-gateway/errors.d.ts.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/errors.js +1 -0
- package/dist/adapter/sheets/providers/apps-script-gateway/errors.js.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/index.d.ts +2 -0
- package/dist/adapter/sheets/providers/apps-script-gateway/index.d.ts.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/index.js +1 -0
- package/dist/adapter/sheets/providers/apps-script-gateway/index.js.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/effect/effectOperation.d.ts +3 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/effect/effectOperation.d.ts.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/effect/effectOperation.js +9 -0
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/effect/effectOperation.js.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/effect/effectOperationScript.d.ts.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/effect/effectOperationScript.js +137 -20
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/effect/effectOperationScript.js.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/observation/observationOperation.js +82 -34
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/observation/observationOperation.js.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/write/batchAppendOperation.d.ts +31 -0
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/write/batchAppendOperation.d.ts.map +1 -0
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/write/batchAppendOperation.js +567 -0
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/write/batchAppendOperation.js.map +1 -0
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/write/fastAppendOperation.d.ts +12 -17
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/write/fastAppendOperation.d.ts.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/write/fastAppendOperation.js +27 -213
- package/dist/adapter/sheets/providers/apps-script-gateway/operations/write/fastAppendOperation.js.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/transport/operationClient.d.ts.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/transport/operationClient.js +81 -8
- package/dist/adapter/sheets/providers/apps-script-gateway/transport/operationClient.js.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/transport/operationSyncGateway.d.ts +1 -6
- package/dist/adapter/sheets/providers/apps-script-gateway/transport/operationSyncGateway.d.ts.map +1 -1
- package/dist/adapter/sheets/providers/apps-script-gateway/transport/operationSyncGateway.js +15 -9
- package/dist/adapter/sheets/providers/apps-script-gateway/transport/operationSyncGateway.js.map +1 -1
- package/dist/application/orm/mapping/projection.js +1 -1
- package/dist/application/orm/mapping/projection.js.map +1 -1
- package/dist/application/orm/persistence/flush/flushCoordinator.d.ts.map +1 -1
- package/dist/application/orm/persistence/flush/flushCoordinator.js +1 -0
- package/dist/application/orm/persistence/flush/flushCoordinator.js.map +1 -1
- package/dist/application/orm/persistence/lifecycle/entityLifecycle.js +1 -1
- package/dist/application/orm/persistence/lifecycle/entityLifecycle.js.map +1 -1
- package/dist/application/sync/gateway/SyncGatewayBootstrap.d.ts +3 -1
- package/dist/application/sync/gateway/SyncGatewayBootstrap.d.ts.map +1 -1
- package/dist/application/sync/gateway/SyncGatewayBootstrap.js +3 -1
- package/dist/application/sync/gateway/SyncGatewayBootstrap.js.map +1 -1
- package/dist/application/sync/gateway/conflictProjection.d.ts +13 -0
- package/dist/application/sync/gateway/conflictProjection.d.ts.map +1 -0
- package/dist/application/sync/gateway/conflictProjection.js +58 -0
- package/dist/application/sync/gateway/conflictProjection.js.map +1 -0
- package/dist/application/sync/gateway/conflictProjectionRegistration.d.ts +11 -0
- package/dist/application/sync/gateway/conflictProjectionRegistration.d.ts.map +1 -0
- package/dist/application/sync/gateway/conflictProjectionRegistration.js +70 -0
- package/dist/application/sync/gateway/conflictProjectionRegistration.js.map +1 -0
- package/dist/application/sync/gateway/coordinator/CoordinatedSyncGateway.d.ts +102 -0
- package/dist/application/sync/gateway/coordinator/CoordinatedSyncGateway.d.ts.map +1 -0
- package/dist/application/sync/gateway/coordinator/CoordinatedSyncGateway.js +221 -0
- package/dist/application/sync/gateway/coordinator/CoordinatedSyncGateway.js.map +1 -0
- package/dist/application/sync/gateway/coordinator/asyncMutex.d.ts +51 -0
- package/dist/application/sync/gateway/coordinator/asyncMutex.d.ts.map +1 -0
- package/dist/application/sync/gateway/coordinator/asyncMutex.js +65 -0
- package/dist/application/sync/gateway/coordinator/asyncMutex.js.map +1 -0
- package/dist/application/sync/gateway/coordinator/coordinatorTelemetry.d.ts +31 -0
- package/dist/application/sync/gateway/coordinator/coordinatorTelemetry.d.ts.map +1 -0
- package/dist/application/sync/gateway/coordinator/coordinatorTelemetry.js +13 -0
- package/dist/application/sync/gateway/coordinator/coordinatorTelemetry.js.map +1 -0
- package/dist/application/sync/gateway/syncGateway.d.ts +26 -6
- package/dist/application/sync/gateway/syncGateway.d.ts.map +1 -1
- package/dist/application/sync/gateway/syncGateway.js +10 -2
- package/dist/application/sync/gateway/syncGateway.js.map +1 -1
- package/dist/application/sync/gateway/transportClassification.d.ts +62 -0
- package/dist/application/sync/gateway/transportClassification.d.ts.map +1 -0
- package/dist/application/sync/gateway/transportClassification.js +133 -0
- package/dist/application/sync/gateway/transportClassification.js.map +1 -0
- package/dist/application/sync/inbound/autoSystemConflictResolution.d.ts +26 -0
- package/dist/application/sync/inbound/autoSystemConflictResolution.d.ts.map +1 -0
- package/dist/application/sync/inbound/autoSystemConflictResolution.js +335 -0
- package/dist/application/sync/inbound/autoSystemConflictResolution.js.map +1 -0
- package/dist/application/sync/outbound/effects/AdaptiveEffectBatchController.d.ts +50 -0
- package/dist/application/sync/outbound/effects/AdaptiveEffectBatchController.d.ts.map +1 -0
- package/dist/application/sync/outbound/effects/AdaptiveEffectBatchController.js +97 -0
- package/dist/application/sync/outbound/effects/AdaptiveEffectBatchController.js.map +1 -0
- package/dist/application/sync/outbound/effects/SyncEffectSupervisor.d.ts +4 -1
- package/dist/application/sync/outbound/effects/SyncEffectSupervisor.d.ts.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectSupervisor.js +35 -1
- package/dist/application/sync/outbound/effects/SyncEffectSupervisor.js.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorker.d.ts +14 -2
- package/dist/application/sync/outbound/effects/SyncEffectWorker.d.ts.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorker.js +190 -31
- package/dist/application/sync/outbound/effects/SyncEffectWorker.js.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorkerConstants.d.ts +25 -2
- package/dist/application/sync/outbound/effects/SyncEffectWorkerConstants.d.ts.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorkerConstants.js +25 -2
- package/dist/application/sync/outbound/effects/SyncEffectWorkerConstants.js.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorkerDispatch.d.ts +14 -7
- package/dist/application/sync/outbound/effects/SyncEffectWorkerDispatch.d.ts.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorkerDispatch.js +118 -14
- package/dist/application/sync/outbound/effects/SyncEffectWorkerDispatch.js.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorkerRouting.d.ts +30 -9
- package/dist/application/sync/outbound/effects/SyncEffectWorkerRouting.d.ts.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorkerRouting.js +102 -18
- package/dist/application/sync/outbound/effects/SyncEffectWorkerRouting.js.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorkerTransitions.d.ts.map +1 -1
- package/dist/application/sync/outbound/effects/SyncEffectWorkerTransitions.js +49 -14
- package/dist/application/sync/outbound/effects/SyncEffectWorkerTransitions.js.map +1 -1
- package/dist/application/sync/outbound/reconciliation/ReconciliationScanner.d.ts.map +1 -1
- package/dist/application/sync/outbound/reconciliation/ReconciliationScanner.js +97 -17
- package/dist/application/sync/outbound/reconciliation/ReconciliationScanner.js.map +1 -1
- package/dist/application/sync/service/SyncServiceBootstrap.d.ts +17 -0
- package/dist/application/sync/service/SyncServiceBootstrap.d.ts.map +1 -1
- package/dist/application/sync/service/SyncServiceBootstrap.js +125 -10
- package/dist/application/sync/service/SyncServiceBootstrap.js.map +1 -1
- package/dist/application/sync/service/contracts.d.ts +2 -0
- package/dist/application/sync/service/contracts.d.ts.map +1 -1
- package/dist/application/sync/telemetry/syncTiming.d.ts +6 -0
- package/dist/application/sync/telemetry/syncTiming.d.ts.map +1 -1
- package/dist/application/sync/telemetry/syncTiming.js +2 -0
- package/dist/application/sync/telemetry/syncTiming.js.map +1 -1
- package/dist/domain/conflict/index.d.ts +1 -1
- package/dist/domain/conflict/index.d.ts.map +1 -1
- package/dist/domain/conflict/index.js +1 -1
- package/dist/domain/conflict/index.js.map +1 -1
- package/dist/domain/conflict/transitions.d.ts +3 -0
- package/dist/domain/conflict/transitions.d.ts.map +1 -1
- package/dist/domain/conflict/transitions.js +3 -2
- package/dist/domain/conflict/transitions.js.map +1 -1
- package/dist/domain/model/constants.d.ts +1 -0
- package/dist/domain/model/constants.d.ts.map +1 -1
- package/dist/domain/model/constants.js +1 -0
- package/dist/domain/model/constants.js.map +1 -1
- package/dist/infrastructure/storage/index.d.ts +4 -2
- package/dist/infrastructure/storage/index.d.ts.map +1 -1
- package/dist/infrastructure/storage/index.js +2 -1
- package/dist/infrastructure/storage/index.js.map +1 -1
- package/dist/infrastructure/storage/sqlite/migrateSchema.d.ts.map +1 -1
- package/dist/infrastructure/storage/sqlite/migrateSchema.js +59 -1
- package/dist/infrastructure/storage/sqlite/migrateSchema.js.map +1 -1
- package/dist/infrastructure/storage/sqlite/schema.d.ts +12 -1
- package/dist/infrastructure/storage/sqlite/schema.d.ts.map +1 -1
- package/dist/infrastructure/storage/sqlite/schema.js +47 -3
- package/dist/infrastructure/storage/sqlite/schema.js.map +1 -1
- package/dist/infrastructure/storage/sqlite/schemaTypes.d.ts +2 -2
- package/dist/infrastructure/storage/sqlite/schemaTypes.d.ts.map +1 -1
- package/dist/infrastructure/storage/state/canonical/canonicalCommit.js +1 -1
- package/dist/infrastructure/storage/state/canonical/canonicalCommit.js.map +1 -1
- package/dist/infrastructure/storage/state/mapped/mappedPersistenceSql.d.ts +1 -0
- package/dist/infrastructure/storage/state/mapped/mappedPersistenceSql.d.ts.map +1 -1
- package/dist/infrastructure/storage/state/mapped/mappedPersistenceSql.js +1 -1
- package/dist/infrastructure/storage/state/mapped/mappedPersistenceSql.js.map +1 -1
- package/dist/infrastructure/storage/state/observation/observationAudit.js +5 -1
- package/dist/infrastructure/storage/state/observation/observationAudit.js.map +1 -1
- package/dist/infrastructure/storage/state/resolution/resolutionWriter.d.ts +10 -0
- package/dist/infrastructure/storage/state/resolution/resolutionWriter.d.ts.map +1 -1
- package/dist/infrastructure/storage/state/resolution/resolutionWriter.js +106 -20
- package/dist/infrastructure/storage/state/resolution/resolutionWriter.js.map +1 -1
- package/dist/infrastructure/storage/state/resolution/resolutionWriterContracts.d.ts +9 -1
- package/dist/infrastructure/storage/state/resolution/resolutionWriterContracts.d.ts.map +1 -1
- package/dist/infrastructure/storage/state/resolution/resolutionWriterContracts.js +3 -0
- package/dist/infrastructure/storage/state/resolution/resolutionWriterContracts.js.map +1 -1
- package/dist/infrastructure/storage/state/resolution/resolutionWriterSql.d.ts +11 -0
- package/dist/infrastructure/storage/state/resolution/resolutionWriterSql.d.ts.map +1 -1
- package/dist/infrastructure/storage/state/resolution/resolutionWriterSql.js +35 -0
- package/dist/infrastructure/storage/state/resolution/resolutionWriterSql.js.map +1 -1
- package/dist/infrastructure/storage/sync/outbound/effectOutbox.d.ts +12 -4
- package/dist/infrastructure/storage/sync/outbound/effectOutbox.d.ts.map +1 -1
- package/dist/infrastructure/storage/sync/outbound/effectOutbox.js +46 -15
- package/dist/infrastructure/storage/sync/outbound/effectOutbox.js.map +1 -1
- package/dist/infrastructure/storage/sync/outbound/effectOutboxContracts.d.ts +24 -0
- package/dist/infrastructure/storage/sync/outbound/effectOutboxContracts.d.ts.map +1 -1
- package/dist/infrastructure/storage/sync/outbound/effectOutboxContracts.js +1 -0
- package/dist/infrastructure/storage/sync/outbound/effectOutboxContracts.js.map +1 -1
- package/dist/infrastructure/storage/sync/outbound/effectOutboxSql.d.ts +9 -7
- package/dist/infrastructure/storage/sync/outbound/effectOutboxSql.d.ts.map +1 -1
- package/dist/infrastructure/storage/sync/outbound/effectOutboxSql.js +44 -8
- package/dist/infrastructure/storage/sync/outbound/effectOutboxSql.js.map +1 -1
- package/dist/infrastructure/storage/sync/outbound/effectOutboxSupport.d.ts +3 -1
- package/dist/infrastructure/storage/sync/outbound/effectOutboxSupport.d.ts.map +1 -1
- package/dist/infrastructure/storage/sync/outbound/effectOutboxSupport.js +35 -2
- package/dist/infrastructure/storage/sync/outbound/effectOutboxSupport.js.map +1 -1
- package/dist/infrastructure/storage/sync/outbound/reconciliationSql.js +1 -0
- package/dist/infrastructure/storage/sync/outbound/reconciliationSql.js.map +1 -1
- package/dist/infrastructure/storage/sync/shared/spreadsheetAuthority.d.ts +31 -0
- package/dist/infrastructure/storage/sync/shared/spreadsheetAuthority.d.ts.map +1 -0
- package/dist/infrastructure/storage/sync/shared/spreadsheetAuthority.js +92 -0
- package/dist/infrastructure/storage/sync/shared/spreadsheetAuthority.js.map +1 -0
- package/dist/infrastructure/storage/sync/shared/writerLease.js +3 -1
- package/dist/infrastructure/storage/sync/shared/writerLease.js.map +1 -1
- package/docs/advanced-sheets-gateway-concurrency-problem.md +434 -0
- package/docs/architecture.md +53 -11
- package/docs/ci.md +31 -14
- package/docs/google-sheets-sync-scaling-strategy.md +459 -0
- package/docs/quick-start.md +15 -1
- package/docs/sync-bulk-write-benchmark.md +520 -0
- package/docs/sync-observability.md +39 -3
- package/docs/write-and-synchronization-flow.md +76 -17
- package/package.json +1 -1
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
/** Durable per-spreadsheet authority records used to fence remote mutations. */
|
|
2
|
+
import { STORAGE_ERROR_CODES, StorageError } from "../../errors.js";
|
|
3
|
+
import { isFencingValidWithSql } from "./writerLease.js";
|
|
4
|
+
const READ_PHYSICAL_AUTHORITY_SQL = `
|
|
5
|
+
SELECT physical.spreadsheet_id, authority.owner_id, authority.authority_epoch,
|
|
6
|
+
authority.authority_token, authority.updated_at
|
|
7
|
+
FROM physical_sheet_registry AS physical
|
|
8
|
+
LEFT JOIN spreadsheet_authority AS authority
|
|
9
|
+
ON authority.spreadsheet_id = physical.spreadsheet_id
|
|
10
|
+
WHERE physical.physical_sheet_id = ?
|
|
11
|
+
`;
|
|
12
|
+
const INSERT_AUTHORITY_SQL = `
|
|
13
|
+
INSERT INTO spreadsheet_authority (
|
|
14
|
+
spreadsheet_id, owner_id, authority_epoch, authority_token, updated_at
|
|
15
|
+
) VALUES (?, ?, ?, ?, ?)
|
|
16
|
+
`;
|
|
17
|
+
const UPDATE_AUTHORITY_SQL = `
|
|
18
|
+
UPDATE spreadsheet_authority
|
|
19
|
+
SET owner_id = ?, authority_epoch = ?, authority_token = ?, updated_at = ?
|
|
20
|
+
WHERE spreadsheet_id = ?
|
|
21
|
+
AND authority_epoch <= ?
|
|
22
|
+
`;
|
|
23
|
+
/** Ensures the current writer fence owns one registered spreadsheet authority. */
|
|
24
|
+
export async function ensureSpreadsheetAuthorityWithSql(sql, options) {
|
|
25
|
+
if (options.ownerId.length === 0 || options.physicalSheetId.length === 0) {
|
|
26
|
+
throw new StorageError(STORAGE_ERROR_CODES.INVALID_WRITER_LEASE_OPTIONS, "spreadsheet authority owner and physical sheet ID are required");
|
|
27
|
+
}
|
|
28
|
+
if (!(await isFencingValidWithSql(sql, options)))
|
|
29
|
+
return { kind: "fenced_out" };
|
|
30
|
+
const row = await sql.get(READ_PHYSICAL_AUTHORITY_SQL, [options.physicalSheetId]);
|
|
31
|
+
if (row === undefined || row.spreadsheet_id.length === 0) {
|
|
32
|
+
throw new StorageError(STORAGE_ERROR_CODES.SYNC_REGISTRY_TARGET_UNAVAILABLE, "physical sheet is not registered for spreadsheet authority");
|
|
33
|
+
}
|
|
34
|
+
if (row.authority_epoch !== undefined && row.authority_epoch !== null && (row.authority_epoch > options.writerEpoch ||
|
|
35
|
+
row.authority_epoch === options.writerEpoch && (row.authority_token !== options.fencingToken || row.owner_id !== options.ownerId))) {
|
|
36
|
+
return { kind: "fenced_out" };
|
|
37
|
+
}
|
|
38
|
+
const authority = {
|
|
39
|
+
spreadsheetId: row.spreadsheet_id,
|
|
40
|
+
ownerId: options.ownerId,
|
|
41
|
+
authorityEpoch: options.writerEpoch,
|
|
42
|
+
authorityToken: options.fencingToken,
|
|
43
|
+
updatedAt: options.now,
|
|
44
|
+
};
|
|
45
|
+
if (row.authority_epoch === undefined || row.authority_epoch === null) {
|
|
46
|
+
const inserted = await sql.run(INSERT_AUTHORITY_SQL, [
|
|
47
|
+
authority.spreadsheetId,
|
|
48
|
+
authority.ownerId,
|
|
49
|
+
authority.authorityEpoch,
|
|
50
|
+
authority.authorityToken,
|
|
51
|
+
authority.updatedAt,
|
|
52
|
+
]);
|
|
53
|
+
if (inserted.changes !== 1)
|
|
54
|
+
return { kind: "fenced_out" };
|
|
55
|
+
return { kind: "claimed", authority };
|
|
56
|
+
}
|
|
57
|
+
const updated = await sql.run(UPDATE_AUTHORITY_SQL, [
|
|
58
|
+
authority.ownerId,
|
|
59
|
+
authority.authorityEpoch,
|
|
60
|
+
authority.authorityToken,
|
|
61
|
+
authority.updatedAt,
|
|
62
|
+
authority.spreadsheetId,
|
|
63
|
+
authority.authorityEpoch,
|
|
64
|
+
]);
|
|
65
|
+
return updated.changes === 1
|
|
66
|
+
? { kind: "claimed", authority }
|
|
67
|
+
: { kind: "fenced_out" };
|
|
68
|
+
}
|
|
69
|
+
/** Ensures a spreadsheet authority through an adapter-owned transaction. */
|
|
70
|
+
export async function ensureSpreadsheetAuthorityWithAdapter(storage, options) {
|
|
71
|
+
return storage.transaction(({ sql }) => ensureSpreadsheetAuthorityWithSql(sql, options));
|
|
72
|
+
}
|
|
73
|
+
/** Reads the durable authority attached to one registered physical sheet. */
|
|
74
|
+
export async function readSpreadsheetAuthorityWithSql(sql, physicalSheetId) {
|
|
75
|
+
const row = await sql.get(READ_PHYSICAL_AUTHORITY_SQL, [physicalSheetId]);
|
|
76
|
+
if (row === undefined || row.authority_epoch === undefined || row.authority_epoch === null ||
|
|
77
|
+
row.authority_token === undefined || row.authority_token === null) {
|
|
78
|
+
return undefined;
|
|
79
|
+
}
|
|
80
|
+
return {
|
|
81
|
+
spreadsheetId: row.spreadsheet_id,
|
|
82
|
+
ownerId: row.owner_id ?? "",
|
|
83
|
+
authorityEpoch: row.authority_epoch,
|
|
84
|
+
authorityToken: row.authority_token,
|
|
85
|
+
updatedAt: row.updated_at ?? 0,
|
|
86
|
+
};
|
|
87
|
+
}
|
|
88
|
+
/** Reads one durable spreadsheet authority through a fresh adapter context. */
|
|
89
|
+
export async function readSpreadsheetAuthorityWithAdapter(storage, physicalSheetId) {
|
|
90
|
+
return storage.read(({ sql }) => readSpreadsheetAuthorityWithSql(sql, physicalSheetId));
|
|
91
|
+
}
|
|
92
|
+
//# sourceMappingURL=spreadsheetAuthority.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"spreadsheetAuthority.js","sourceRoot":"","sources":["../../../../../src/infrastructure/storage/sync/shared/spreadsheetAuthority.ts"],"names":[],"mappings":"AAAA,gFAAgF;AAEhF,OAAO,EAAE,mBAAmB,EAAE,YAAY,EAAE,MAAM,iBAAiB,CAAC;AAEpE,OAAO,EAAE,qBAAqB,EAAE,MAAM,kBAAkB,CAAC;AAGzD,MAAM,2BAA2B,GAAG;;;;;;;CAOnC,CAAC;AAEF,MAAM,oBAAoB,GAAG;;;;CAI5B,CAAC;AAEF,MAAM,oBAAoB,GAAG;;;;;CAK5B,CAAC;AAcF,kFAAkF;AAClF,MAAM,CAAC,KAAK,UAAU,iCAAiC,CACrD,GAAgB,EAChB,OAAwF;IAExF,IAAI,OAAO,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,IAAI,OAAO,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACzE,MAAM,IAAI,YAAY,CACpB,mBAAmB,CAAC,4BAA4B,EAChD,gEAAgE,CACjE,CAAC;IACJ,CAAC;IACD,IAAI,CAAC,CAAC,MAAM,qBAAqB,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;QAAE,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,CAAC;IAChF,MAAM,GAAG,GAAG,MAAM,GAAG,CAAC,GAAG,CAAe,2BAA2B,EAAE,CAAC,OAAO,CAAC,eAAe,CAAC,CAAC,CAAC;IAChG,IAAI,GAAG,KAAK,SAAS,IAAI,GAAG,CAAC,cAAc,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACzD,MAAM,IAAI,YAAY,CACpB,mBAAmB,CAAC,gCAAgC,EACpD,4DAA4D,CAC7D,CAAC;IACJ,CAAC;IACD,IAAI,GAAG,CAAC,eAAe,KAAK,SAAS,IAAI,GAAG,CAAC,eAAe,KAAK,IAAI,IAAI,CACvE,GAAG,CAAC,eAAe,GAAG,OAAO,CAAC,WAAW;QACzC,GAAG,CAAC,eAAe,KAAK,OAAO,CAAC,WAAW,IAAI,CAC7C,GAAG,CAAC,eAAe,KAAK,OAAO,CAAC,YAAY,IAAI,GAAG,CAAC,QAAQ,KAAK,OAAO,CAAC,OAAO,CACjF,CACF,EAAE,CAAC;QACF,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,CAAC;IAChC,CAAC;IAED,MAAM,SAAS,GAAyB;QACtC,aAAa,EAAE,GAAG,CAAC,cAAc;QACjC,OAAO,EAAE,OAAO,CAAC,OAAO;QACxB,cAAc,EAAE,OAAO,CAAC,WAAW;QACnC,cAAc,EAAE,OAAO,CAAC,YAAY;QACpC,SAAS,EAAE,OAAO,CAAC,GAAG;KACvB,CAAC;IACF,IAAI,GAAG,CAAC,eAAe,KAAK,SAAS,IAAI,GAAG,CAAC,eAAe,KAAK,IAAI,EAAE,CAAC;QACtE,MAAM,QAAQ,GAAG,MAAM,GAAG,CAAC,GAAG,CAAC,oBAAoB,EAAE;YACnD,SAAS,CAAC,aAAa;YACvB,SAAS,CAAC,OAAO;YACjB,SAAS,CAAC,cAAc;YACxB,SAAS,CAAC,cAAc;YACxB,SAAS,CAAC,SAAS;SACpB,CAAC,CAAC;QACH,IAAI,QAAQ,CAAC,OAAO,KAAK,CAAC;YAAE,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,CAAC;QAC1D,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,SAAS,EAAE,CAAC;IACxC,CAAC;IAED,MAAM,OAAO,GAAG,MAAM,GAAG,CAAC,GAAG,CAAC,oBAAoB,EAAE;QAClD,SAAS,CAAC,OAAO;QACjB,SAAS,CAAC,cAAc;QACxB,SAAS,CAAC,cAAc;QACxB,SAAS,CAAC,SAAS;QACnB,SAAS,CAAC,aAAa;QACvB,SAAS,CAAC,cAAc;KACzB,CAAC,CAAC;IACH,OAAO,OAAO,CAAC,OAAO,KAAK,CAAC;QAC1B,CAAC,CAAC,EAAE,IAAI,EAAE,SAAS,EAAE,SAAS,EAAE;QAChC,CAAC,CAAC,EAAE,IAAI,EAAE,YAAY,EAAE,CAAC;AAC7B,CAAC;AAED,4EAA4E;AAC5E,MAAM,CAAC,KAAK,UAAU,qCAAqC,CACzD,OAA0B,EAC1B,OAAwF;IAExF,OAAO,OAAO,CAAC,WAAW,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,EAAE,CAAC,iCAAiC,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC,CAAC;AAC3F,CAAC;AAED,6EAA6E;AAC7E,MAAM,CAAC,KAAK,UAAU,+BAA+B,CACnD,GAAgB,EAChB,eAAuB;IAEvB,MAAM,GAAG,GAAG,MAAM,GAAG,CAAC,GAAG,CAAe,2BAA2B,EAAE,CAAC,eAAe,CAAC,CAAC,CAAC;IACxF,IAAI,GAAG,KAAK,SAAS,IAAI,GAAG,CAAC,eAAe,KAAK,SAAS,IAAI,GAAG,CAAC,eAAe,KAAK,IAAI;QACtF,GAAG,CAAC,eAAe,KAAK,SAAS,IAAI,GAAG,CAAC,eAAe,KAAK,IAAI,EAAE,CAAC;QACtE,OAAO,SAAS,CAAC;IACnB,CAAC;IACD,OAAO;QACL,aAAa,EAAE,GAAG,CAAC,cAAc;QACjC,OAAO,EAAE,GAAG,CAAC,QAAQ,IAAI,EAAE;QAC3B,cAAc,EAAE,GAAG,CAAC,eAAe;QACnC,cAAc,EAAE,GAAG,CAAC,eAAe;QACnC,SAAS,EAAE,GAAG,CAAC,UAAU,IAAI,CAAC;KAC/B,CAAC;AACJ,CAAC;AAED,+EAA+E;AAC/E,MAAM,CAAC,KAAK,UAAU,mCAAmC,CACvD,OAA0B,EAC1B,eAAuB;IAEvB,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,EAAE,CAAC,+BAA+B,CAAC,GAAG,EAAE,eAAe,CAAC,CAAC,CAAC;AAC1F,CAAC"}
|
|
@@ -169,7 +169,9 @@ function makeLease(role, writerId, writerEpoch, leaseUntil) {
|
|
|
169
169
|
role,
|
|
170
170
|
writerId,
|
|
171
171
|
writerEpoch,
|
|
172
|
-
|
|
172
|
+
// Include the owner identity as well as the monotonically increasing epoch;
|
|
173
|
+
// equal epochs from different roles must never share a remote authority token.
|
|
174
|
+
fencingToken: `fence-${writerEpoch}:${writerId}`,
|
|
173
175
|
leaseUntil,
|
|
174
176
|
};
|
|
175
177
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"writerLease.js","sourceRoot":"","sources":["../../../../../src/infrastructure/storage/sync/shared/writerLease.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,EAAE,mBAAmB,EAAE,YAAY,EAAE,MAAM,iBAAiB,CAAC;AACpE,OAAO,EAAE,mBAAmB,EAAE,MAAM,uCAAuC,CAAC;AAG5E,OAAO,EAAE,gBAAgB,EAAE,MAAM,gCAAgC,CAAC;AAElE,MAAM,qBAAqB,GACzB,mGAAmG,CAAC;AAEtG,MAAM,uBAAuB,GAAG;;;CAG/B,CAAC;AAEF,MAAM,sBAAsB,GAAG;;;;;CAK9B,CAAC;AAEF,MAAM,yBAAyB,GAAG;;;;CAIjC,CAAC;AAUF,yEAAyE;AACzE,MAAM,CAAC,MAAM,+BAA+B,GAAG;IAC7C,OAAO,EAAE,SAAS;IAClB,WAAW,EAAE,aAAa;CAClB,CAAC;AAMX,0EAA0E;AAC1E,MAAM,CAAC,MAAM,kCAAkC,GAAG;IAChD,aAAa,EAAE,eAAe;IAC9B,yBAAyB,EAAE,2BAA2B;IACtD,iBAAiB,EAAE,mBAAmB;IACtC,kBAAkB,EAAE,oBAAoB;CAChC,CAAC;AA+BX,6EAA6E;AAC7E,MAAM,CAAC,MAAM,gBAAgB,GAAG;;;CAG/B,CAAC;AAEF,wEAAwE;AACxE,MAAM,UAAU,eAAe,CAC7B,KAAqB;IAErB,OAAO,CAAC,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,WAAW,EAAE,KAAK,CAAC,YAAY,EAAE,KAAK,CAAC,GAAG,CAAC,CAAC;AACxE,CAAC;AAED;;;;;GAKG;AACH,MAAM,CAAC,KAAK,UAAU,uBAAuB,CAC3C,GAAgB,EAChB,OAA0B;IAE1B,oBAAoB,CAAC,OAAO,CAAC,CAAC;IAE9B,OAAO,gBAAgB,CAAC,GAAG,EAAE,oBAAoB,EAAE,KAAK,IAAI,EAAE;QAC5D,MAAM,QAAQ,GAAG,MAAM,mBAAmB,CAAC,GAAG,EAAE,OAAO,CAAC,IAAI,CAAC,CAAC;QAC9D,MAAM,aAAa,GAAG,OAAO,CAAC,GAAG,GAAG,OAAO,CAAC,eAAe,CAAC;QAE5D,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;YAC3B,MAAM,KAAK,GAAG,SAAS,CAAC,OAAO,CAAC,IAAI,EAAE,OAAO,CAAC,QAAQ,EAAE,CAAC,EAAE,aAAa,CAAC,CAAC;YAC1E,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,GAAG,CAAC,uBAAuB,EAAE;gBACpD,KAAK,CAAC,IAAI;gBACV,KAAK,CAAC,QAAQ;gBACd,KAAK,CAAC,WAAW;gBACjB,KAAK,CAAC,YAAY;gBAClB,KAAK,CAAC,UAAU;aACjB,CAAC,CAAC;YACH,OAAO,MAAM,CAAC,OAAO,KAAK,CAAC;gBACzB,CAAC,CAAC,EAAE,IAAI,EAAE,+BAA+B,CAAC,OAAO,EAAE,KAAK,EAAE;gBAC1D,CAAC,CAAC;oBACE,IAAI,EAAE,+BAA+B,CAAC,WAAW;oBACjD,MAAM,EAAE,kCAAkC,CAAC,yBAAyB;iBACrE,CAAC;QACR,CAAC;QAED,IAAI,QAAQ,CAAC,SAAS,KAAK,OAAO,CAAC,QAAQ,IAAI,QAAQ,CAAC,WAAW,GAAG,OAAO,CAAC,GAAG,EAAE,CAAC;YAClF,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,GAAG,CAAC,sBAAsB,EAAE;gBACnD,aAAa;gBACb,OAAO,CAAC,IAAI;gBACZ,OAAO,CAAC,QAAQ;gBAChB,QAAQ,CAAC,YAAY;gBACrB,QAAQ,CAAC,aAAa;gBACtB,OAAO,CAAC,GAAG;aACZ,CAAC,CAAC;YACH,OAAO,MAAM,CAAC,OAAO,KAAK,CAAC;gBACzB,CAAC,CAAC;oBACE,IAAI,EAAE,+BAA+B,CAAC,OAAO;oBAC7C,KAAK,EAAE;wBACL,IAAI,EAAE,QAAQ,CAAC,IAAI;wBACnB,QAAQ,EAAE,QAAQ,CAAC,SAAS;wBAC5B,WAAW,EAAE,QAAQ,CAAC,YAAY;wBAClC,YAAY,EAAE,QAAQ,CAAC,aAAa;wBACpC,UAAU,EAAE,aAAa;qBAC1B;iBACF;gBACH,CAAC,CAAC;oBACE,IAAI,EAAE,+BAA+B,CAAC,WAAW;oBACjD,MAAM,EAAE,kCAAkC,CAAC,iBAAiB;iBAC7D,CAAC;QACR,CAAC;QAED,IAAI,QAAQ,CAAC,WAAW,GAAG,OAAO,CAAC,GAAG,EAAE,CAAC;YACvC,OAAO;gBACL,IAAI,EAAE,+BAA+B,CAAC,WAAW;gBACjD,MAAM,EAAE,kCAAkC,CAAC,aAAa;aACzD,CAAC;QACJ,CAAC;QAED,MAAM,QAAQ,GAAG,SAAS,CACxB,OAAO,CAAC,IAAI,EACZ,OAAO,CAAC,QAAQ,EAChB,QAAQ,CAAC,YAAY,GAAG,CAAC,EACzB,aAAa,CACd,CAAC;QACF,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,GAAG,CAAC,yBAAyB,EAAE;YACtD,QAAQ,CAAC,QAAQ;YACjB,QAAQ,CAAC,WAAW;YACpB,QAAQ,CAAC,YAAY;YACrB,QAAQ,CAAC,UAAU;YACnB,OAAO,CAAC,IAAI;YACZ,QAAQ,CAAC,YAAY;YACrB,QAAQ,CAAC,aAAa;YACtB,OAAO,CAAC,GAAG;SACZ,CAAC,CAAC;QACH,OAAO,MAAM,CAAC,OAAO,KAAK,CAAC;YACzB,CAAC,CAAC,EAAE,IAAI,EAAE,+BAA+B,CAAC,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE;YACpE,CAAC,CAAC;gBACE,IAAI,EAAE,+BAA+B,CAAC,WAAW;gBACjD,MAAM,EAAE,kCAAkC,CAAC,kBAAkB;aAC9D,CAAC;IACR,CAAC,CAAC,CAAC;AACL,CAAC;AAED,wEAAwE;AACxE,MAAM,CAAC,KAAK,UAAU,2BAA2B,CAC/C,OAA0B,EAC1B,OAA0B;IAE1B,OAAO,OAAO,CAAC,WAAW,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,EAAE,CAAC,uBAAuB,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC,CAAC;AACjF,CAAC;AAED,wEAAwE;AACxE,MAAM,CAAC,KAAK,UAAU,sBAAsB,CAC1C,GAAgB,EAChB,IAAY;IAEZ,MAAM,GAAG,GAAG,MAAM,mBAAmB,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;IACjD,OAAO,GAAG,KAAK,SAAS;QACtB,CAAC,CAAC,EAAE,IAAI,EAAE,mBAAmB,CAAC,SAAS,EAAE;QACzC,CAAC,CAAC;YACE,IAAI,EAAE,mBAAmB,CAAC,KAAK;YAC/B,KAAK,EAAE;gBACL,IAAI,EAAE,GAAG,CAAC,IAAI;gBACd,QAAQ,EAAE,GAAG,CAAC,SAAS;gBACvB,WAAW,EAAE,GAAG,CAAC,YAAY;gBAC7B,YAAY,EAAE,GAAG,CAAC,aAAa;gBAC/B,UAAU,EAAE,GAAG,CAAC,WAAW;aAC5B;SACF,CAAC;AACR,CAAC;AAED,iEAAiE;AACjE,MAAM,CAAC,KAAK,UAAU,0BAA0B,CAC9C,OAA0B,EAC1B,IAAY;IAEZ,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,EAAE,CAAC,sBAAsB,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC,CAAC;AACtE,CAAC;AAED,yEAAyE;AACzE,MAAM,CAAC,KAAK,UAAU,qBAAqB,CACzC,GAAgB,EAChB,KAAqB;IAErB,MAAM,KAAK,GAAG,MAAM,sBAAsB,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,CAAC,CAAC;IAC5D,IAAI,KAAK,CAAC,IAAI,KAAK,mBAAmB,CAAC,SAAS;QAAE,OAAO,KAAK,CAAC;IAC/D,OAAO,CACL,KAAK,CAAC,KAAK,CAAC,WAAW,KAAK,KAAK,CAAC,WAAW;QAC7C,KAAK,CAAC,KAAK,CAAC,YAAY,KAAK,KAAK,CAAC,YAAY;QAC/C,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,KAAK,CAAC,GAAG,CACnC,CAAC;AACJ,CAAC;AAED,mEAAmE;AACnE,MAAM,CAAC,KAAK,UAAU,yBAAyB,CAC7C,OAA0B,EAC1B,KAAqB;IAErB,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,EAAE,CAAC,qBAAqB,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC,CAAC;AACtE,CAAC;AAUD,SAAS,mBAAmB,CAAC,GAAgB,EAAE,IAAY;IACzD,OAAO,GAAG,CAAC,GAAG,CAAW,qBAAqB,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC;AAC1D,CAAC;AAED,SAAS,SAAS,CAChB,IAAY,EACZ,QAAgB,EAChB,WAAmB,EACnB,UAAkB;IAElB,OAAO;QACL,IAAI;QACJ,QAAQ;QACR,WAAW;QACX,YAAY,EAAE,SAAS,WAAW,EAAE;
|
|
1
|
+
{"version":3,"file":"writerLease.js","sourceRoot":"","sources":["../../../../../src/infrastructure/storage/sync/shared/writerLease.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,EAAE,mBAAmB,EAAE,YAAY,EAAE,MAAM,iBAAiB,CAAC;AACpE,OAAO,EAAE,mBAAmB,EAAE,MAAM,uCAAuC,CAAC;AAG5E,OAAO,EAAE,gBAAgB,EAAE,MAAM,gCAAgC,CAAC;AAElE,MAAM,qBAAqB,GACzB,mGAAmG,CAAC;AAEtG,MAAM,uBAAuB,GAAG;;;CAG/B,CAAC;AAEF,MAAM,sBAAsB,GAAG;;;;;CAK9B,CAAC;AAEF,MAAM,yBAAyB,GAAG;;;;CAIjC,CAAC;AAUF,yEAAyE;AACzE,MAAM,CAAC,MAAM,+BAA+B,GAAG;IAC7C,OAAO,EAAE,SAAS;IAClB,WAAW,EAAE,aAAa;CAClB,CAAC;AAMX,0EAA0E;AAC1E,MAAM,CAAC,MAAM,kCAAkC,GAAG;IAChD,aAAa,EAAE,eAAe;IAC9B,yBAAyB,EAAE,2BAA2B;IACtD,iBAAiB,EAAE,mBAAmB;IACtC,kBAAkB,EAAE,oBAAoB;CAChC,CAAC;AA+BX,6EAA6E;AAC7E,MAAM,CAAC,MAAM,gBAAgB,GAAG;;;CAG/B,CAAC;AAEF,wEAAwE;AACxE,MAAM,UAAU,eAAe,CAC7B,KAAqB;IAErB,OAAO,CAAC,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,WAAW,EAAE,KAAK,CAAC,YAAY,EAAE,KAAK,CAAC,GAAG,CAAC,CAAC;AACxE,CAAC;AAED;;;;;GAKG;AACH,MAAM,CAAC,KAAK,UAAU,uBAAuB,CAC3C,GAAgB,EAChB,OAA0B;IAE1B,oBAAoB,CAAC,OAAO,CAAC,CAAC;IAE9B,OAAO,gBAAgB,CAAC,GAAG,EAAE,oBAAoB,EAAE,KAAK,IAAI,EAAE;QAC5D,MAAM,QAAQ,GAAG,MAAM,mBAAmB,CAAC,GAAG,EAAE,OAAO,CAAC,IAAI,CAAC,CAAC;QAC9D,MAAM,aAAa,GAAG,OAAO,CAAC,GAAG,GAAG,OAAO,CAAC,eAAe,CAAC;QAE5D,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;YAC3B,MAAM,KAAK,GAAG,SAAS,CAAC,OAAO,CAAC,IAAI,EAAE,OAAO,CAAC,QAAQ,EAAE,CAAC,EAAE,aAAa,CAAC,CAAC;YAC1E,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,GAAG,CAAC,uBAAuB,EAAE;gBACpD,KAAK,CAAC,IAAI;gBACV,KAAK,CAAC,QAAQ;gBACd,KAAK,CAAC,WAAW;gBACjB,KAAK,CAAC,YAAY;gBAClB,KAAK,CAAC,UAAU;aACjB,CAAC,CAAC;YACH,OAAO,MAAM,CAAC,OAAO,KAAK,CAAC;gBACzB,CAAC,CAAC,EAAE,IAAI,EAAE,+BAA+B,CAAC,OAAO,EAAE,KAAK,EAAE;gBAC1D,CAAC,CAAC;oBACE,IAAI,EAAE,+BAA+B,CAAC,WAAW;oBACjD,MAAM,EAAE,kCAAkC,CAAC,yBAAyB;iBACrE,CAAC;QACR,CAAC;QAED,IAAI,QAAQ,CAAC,SAAS,KAAK,OAAO,CAAC,QAAQ,IAAI,QAAQ,CAAC,WAAW,GAAG,OAAO,CAAC,GAAG,EAAE,CAAC;YAClF,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,GAAG,CAAC,sBAAsB,EAAE;gBACnD,aAAa;gBACb,OAAO,CAAC,IAAI;gBACZ,OAAO,CAAC,QAAQ;gBAChB,QAAQ,CAAC,YAAY;gBACrB,QAAQ,CAAC,aAAa;gBACtB,OAAO,CAAC,GAAG;aACZ,CAAC,CAAC;YACH,OAAO,MAAM,CAAC,OAAO,KAAK,CAAC;gBACzB,CAAC,CAAC;oBACE,IAAI,EAAE,+BAA+B,CAAC,OAAO;oBAC7C,KAAK,EAAE;wBACL,IAAI,EAAE,QAAQ,CAAC,IAAI;wBACnB,QAAQ,EAAE,QAAQ,CAAC,SAAS;wBAC5B,WAAW,EAAE,QAAQ,CAAC,YAAY;wBAClC,YAAY,EAAE,QAAQ,CAAC,aAAa;wBACpC,UAAU,EAAE,aAAa;qBAC1B;iBACF;gBACH,CAAC,CAAC;oBACE,IAAI,EAAE,+BAA+B,CAAC,WAAW;oBACjD,MAAM,EAAE,kCAAkC,CAAC,iBAAiB;iBAC7D,CAAC;QACR,CAAC;QAED,IAAI,QAAQ,CAAC,WAAW,GAAG,OAAO,CAAC,GAAG,EAAE,CAAC;YACvC,OAAO;gBACL,IAAI,EAAE,+BAA+B,CAAC,WAAW;gBACjD,MAAM,EAAE,kCAAkC,CAAC,aAAa;aACzD,CAAC;QACJ,CAAC;QAED,MAAM,QAAQ,GAAG,SAAS,CACxB,OAAO,CAAC,IAAI,EACZ,OAAO,CAAC,QAAQ,EAChB,QAAQ,CAAC,YAAY,GAAG,CAAC,EACzB,aAAa,CACd,CAAC;QACF,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,GAAG,CAAC,yBAAyB,EAAE;YACtD,QAAQ,CAAC,QAAQ;YACjB,QAAQ,CAAC,WAAW;YACpB,QAAQ,CAAC,YAAY;YACrB,QAAQ,CAAC,UAAU;YACnB,OAAO,CAAC,IAAI;YACZ,QAAQ,CAAC,YAAY;YACrB,QAAQ,CAAC,aAAa;YACtB,OAAO,CAAC,GAAG;SACZ,CAAC,CAAC;QACH,OAAO,MAAM,CAAC,OAAO,KAAK,CAAC;YACzB,CAAC,CAAC,EAAE,IAAI,EAAE,+BAA+B,CAAC,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE;YACpE,CAAC,CAAC;gBACE,IAAI,EAAE,+BAA+B,CAAC,WAAW;gBACjD,MAAM,EAAE,kCAAkC,CAAC,kBAAkB;aAC9D,CAAC;IACR,CAAC,CAAC,CAAC;AACL,CAAC;AAED,wEAAwE;AACxE,MAAM,CAAC,KAAK,UAAU,2BAA2B,CAC/C,OAA0B,EAC1B,OAA0B;IAE1B,OAAO,OAAO,CAAC,WAAW,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,EAAE,CAAC,uBAAuB,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC,CAAC;AACjF,CAAC;AAED,wEAAwE;AACxE,MAAM,CAAC,KAAK,UAAU,sBAAsB,CAC1C,GAAgB,EAChB,IAAY;IAEZ,MAAM,GAAG,GAAG,MAAM,mBAAmB,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;IACjD,OAAO,GAAG,KAAK,SAAS;QACtB,CAAC,CAAC,EAAE,IAAI,EAAE,mBAAmB,CAAC,SAAS,EAAE;QACzC,CAAC,CAAC;YACE,IAAI,EAAE,mBAAmB,CAAC,KAAK;YAC/B,KAAK,EAAE;gBACL,IAAI,EAAE,GAAG,CAAC,IAAI;gBACd,QAAQ,EAAE,GAAG,CAAC,SAAS;gBACvB,WAAW,EAAE,GAAG,CAAC,YAAY;gBAC7B,YAAY,EAAE,GAAG,CAAC,aAAa;gBAC/B,UAAU,EAAE,GAAG,CAAC,WAAW;aAC5B;SACF,CAAC;AACR,CAAC;AAED,iEAAiE;AACjE,MAAM,CAAC,KAAK,UAAU,0BAA0B,CAC9C,OAA0B,EAC1B,IAAY;IAEZ,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,EAAE,CAAC,sBAAsB,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC,CAAC;AACtE,CAAC;AAED,yEAAyE;AACzE,MAAM,CAAC,KAAK,UAAU,qBAAqB,CACzC,GAAgB,EAChB,KAAqB;IAErB,MAAM,KAAK,GAAG,MAAM,sBAAsB,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,CAAC,CAAC;IAC5D,IAAI,KAAK,CAAC,IAAI,KAAK,mBAAmB,CAAC,SAAS;QAAE,OAAO,KAAK,CAAC;IAC/D,OAAO,CACL,KAAK,CAAC,KAAK,CAAC,WAAW,KAAK,KAAK,CAAC,WAAW;QAC7C,KAAK,CAAC,KAAK,CAAC,YAAY,KAAK,KAAK,CAAC,YAAY;QAC/C,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,KAAK,CAAC,GAAG,CACnC,CAAC;AACJ,CAAC;AAED,mEAAmE;AACnE,MAAM,CAAC,KAAK,UAAU,yBAAyB,CAC7C,OAA0B,EAC1B,KAAqB;IAErB,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,EAAE,CAAC,qBAAqB,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC,CAAC;AACtE,CAAC;AAUD,SAAS,mBAAmB,CAAC,GAAgB,EAAE,IAAY;IACzD,OAAO,GAAG,CAAC,GAAG,CAAW,qBAAqB,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC;AAC1D,CAAC;AAED,SAAS,SAAS,CAChB,IAAY,EACZ,QAAgB,EAChB,WAAmB,EACnB,UAAkB;IAElB,OAAO;QACL,IAAI;QACJ,QAAQ;QACR,WAAW;QACX,4EAA4E;QAC5E,+EAA+E;QAC/E,YAAY,EAAE,SAAS,WAAW,IAAI,QAAQ,EAAE;QAChD,UAAU;KACX,CAAC;AACJ,CAAC;AAED,SAAS,oBAAoB,CAAC,OAA0B;IACtD,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,OAAO,CAAC,GAAG,GAAG,CAAC,EAAE,CAAC;QAC1D,MAAM,IAAI,YAAY,CACpB,mBAAmB,CAAC,4BAA4B,EAChD,sDAAsD,CACvD,CAAC;IACJ,CAAC;IACD,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,OAAO,CAAC,eAAe,CAAC,IAAI,OAAO,CAAC,eAAe,IAAI,CAAC,EAAE,CAAC;QACnF,MAAM,IAAI,YAAY,CACpB,mBAAmB,CAAC,4BAA4B,EAChD,uDAAuD,CACxD,CAAC;IACJ,CAAC;IACD,IAAI,OAAO,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC,IAAI,OAAO,CAAC,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC/D,MAAM,IAAI,YAAY,CACpB,mBAAmB,CAAC,4BAA4B,EAChD,8CAA8C,CAC/C,CAAC;IACJ,CAAC;AACH,CAAC"}
|
|
@@ -0,0 +1,434 @@
|
|
|
1
|
+
# Hikoutei Advanced Sheets Gateway 동시성/정합성 문제 해결 요청서
|
|
2
|
+
|
|
3
|
+
> 이 문서만 읽고도 현재 문제의 원인을 분석하고, 락 제거 여부와 대체 정합성 모델을 설계할 수 있도록 작성한 독립 문서다.
|
|
4
|
+
|
|
5
|
+
## 1. 해결해야 할 문제
|
|
6
|
+
|
|
7
|
+
Hikoutei는 SQLite를 애플리케이션의 authority로 사용하고, Google Sheets를 비동기 projection 및 human input surface로 사용한다. SQLite의 flush는 entity table, canonical sync state, durable Sheet effect outbox를 하나의 SQLite transaction으로 커밋한다. Google Sheets 원격 반영은 flush 이후 worker가 비동기로 수행한다.
|
|
8
|
+
|
|
9
|
+
현재 Apps Script Gateway write path에 Advanced Sheets `Sheets.Spreadsheets.batchUpdate`를 도입했다. 그러나 Locust 부하에서 Gateway latency와 오류가 급증한다.
|
|
10
|
+
|
|
11
|
+
핵심 질문은 다음이다.
|
|
12
|
+
|
|
13
|
+
> `batchUpdate`를 사용해도 Apps Script 전역 Script Lock과 원격 read/validate/write 구조 때문에 병목이 남는다. 전역 락을 제거하고 SQLite authority, effect receipt, CAS/fencing, durable outbox를 이용한 다른 정합성 모델로 바꾸는 것이 안전한가? 안전하다면 정확한 설계와 단계별 변경안을 제시하라.
|
|
14
|
+
|
|
15
|
+
단순히 락을 제거하는 패치를 제안하지 말고, 동시 append, response loss, retry, stale CAS, duplicate identity, multi-worker fencing을 모두 고려해야 한다.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## 2. 반드시 보존해야 하는 불변조건
|
|
20
|
+
|
|
21
|
+
1. **SQLite authority**
|
|
22
|
+
- 애플리케이션은 정상 entity data를 Sheets에서 읽지 않는다.
|
|
23
|
+
- Sheets는 비동기 projection이며 User_Input을 제외하면 사람이 직접 수정하는 source of truth가 아니다.
|
|
24
|
+
|
|
25
|
+
2. **Durable outbox**
|
|
26
|
+
- 효과는 flush 시 SQLite outbox에 저장된다.
|
|
27
|
+
- 메모리에 효과를 30초 이상 모아두지 않는다.
|
|
28
|
+
- process crash 이후에도 pending/retry/recovery가 가능해야 한다.
|
|
29
|
+
|
|
30
|
+
3. **Effect idempotency**
|
|
31
|
+
- effect ID와 payload hash가 receipt에 기록된다.
|
|
32
|
+
- 같은 effect ID를 같은 payload로 재시도하면 `already_applied`/`applied`로 안전하게 회복해야 한다.
|
|
33
|
+
- 같은 effect ID를 다른 payload로 재사용하면 fail-closed 해야 한다.
|
|
34
|
+
- effect ID를 확인했다고 해서 row postcondition이 검증된 것으로 간주하면 안 된다.
|
|
35
|
+
|
|
36
|
+
4. **CAS와 fencing**
|
|
37
|
+
- guarded update/delete는 expected visible revision/hash와 target evidence를 사용한다.
|
|
38
|
+
- 오래된 worker나 lease를 잃은 worker가 원격 row를 덮어쓰면 안 된다.
|
|
39
|
+
- SQLite writer lease와 effect lease의 fencing semantics를 보존해야 한다.
|
|
40
|
+
|
|
41
|
+
5. **Duplicate identity fail-closed**
|
|
42
|
+
- business identity, `Conflict_ID`, physical anchor가 중복되면 자동 삭제하지 않는다.
|
|
43
|
+
- 중복 row는 진단/quarantine 대상으로 남겨야 한다.
|
|
44
|
+
|
|
45
|
+
6. **Public boundary 유지**
|
|
46
|
+
- `src/index.ts`와 public EntityManager API를 변경하지 않는다.
|
|
47
|
+
- Apps Script `Code.gs` dispatcher/business logic은 현재 단계에서 변경하지 않는다.
|
|
48
|
+
- SQLite authority, outbox, receipt, recovery semantics를 약화하지 않는다.
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## 3. 현재 시스템 구조
|
|
53
|
+
|
|
54
|
+
### 3.1 Node/Application side
|
|
55
|
+
|
|
56
|
+
- Entity flush가 SQLite entity table과 canonical state를 저장한다.
|
|
57
|
+
- 같은 transaction에서 Sheet effect outbox를 만든다.
|
|
58
|
+
- `SyncEffectWorker`가 SQLite에서 effect를 claim하고 route별로 전송한다.
|
|
59
|
+
- worker는 adaptive batch controller를 사용한다.
|
|
60
|
+
- coalescing window: 기본 500ms
|
|
61
|
+
- 내부 effect batch 범위: 5~20
|
|
62
|
+
- initial batch: 10
|
|
63
|
+
- gateway의 공식 제한이 아니라 내부 방어 상한이다.
|
|
64
|
+
- effect lease 기본값은 120초, writer lease 기본값은 180초다.
|
|
65
|
+
- worker와 polling supervisor가 별도 경로로 Gateway에 요청한다.
|
|
66
|
+
- load harness의 `POST /__test/user-input`은 테스트용으로 `User_Input` row를 직접 수정하는 별도 control operation을 보낸다. 이것은 production effect가 아니다.
|
|
67
|
+
|
|
68
|
+
### 3.2 Apps Script Gateway
|
|
69
|
+
|
|
70
|
+
`apps-script/gateway/Code.gs`는 다음만 수행한다.
|
|
71
|
+
|
|
72
|
+
1. signed POST 검증
|
|
73
|
+
2. HMAC/body hash/time/sheet allowlist 검증
|
|
74
|
+
3. `applyOperations`의 serialized function source를 실행
|
|
75
|
+
4. 응답 JSON envelope 생성
|
|
76
|
+
5. 마지막에 `SpreadsheetApp.flush()` 호출
|
|
77
|
+
|
|
78
|
+
Data-plane의 `Code.gs` 자체는 일반 effect마다 Script Lock을 잡지 않는다. 다만 eval로 실행되는 operation source들이 `LockService.getScriptLock()`을 사용한다.
|
|
79
|
+
|
|
80
|
+
### 3.3 Operation source별 락
|
|
81
|
+
|
|
82
|
+
#### `batchAppendOperation.ts`
|
|
83
|
+
|
|
84
|
+
`LockService.getScriptLock()`을 잡은 뒤 다음 전체를 수행한다.
|
|
85
|
+
|
|
86
|
+
- target sheet와 header 확인
|
|
87
|
+
- receipt sheet 확인/생성
|
|
88
|
+
- 모든 receipt 읽기
|
|
89
|
+
- effect ID/payload hash 확인
|
|
90
|
+
- target sheet의 identity 중복 확인
|
|
91
|
+
- `lastRow` 계산
|
|
92
|
+
- Advanced Sheets `batchUpdate` 실행
|
|
93
|
+
- row reservation
|
|
94
|
+
- row cell write
|
|
95
|
+
- developer metadata anchor 생성
|
|
96
|
+
- receipt row 삽입/기록
|
|
97
|
+
- receipt/visible evidence 검증
|
|
98
|
+
- lock release
|
|
99
|
+
|
|
100
|
+
즉, lock critical section 안에 원격 read, 전체 receipt scan, identity scan, write가 모두 들어 있다.
|
|
101
|
+
|
|
102
|
+
#### `effectOperationScript.ts`
|
|
103
|
+
|
|
104
|
+
일반 update/delete/effect materialization도 `LockService.getScriptLock()`을 약 20초 timeout으로 잡고, effect별 precondition/read/write/receipt 처리를 수행한다.
|
|
105
|
+
|
|
106
|
+
#### `observationOperation.ts`
|
|
107
|
+
|
|
108
|
+
anchor assignment가 포함된 full observation은 Script Lock을 사용한다. 이 경로는 다음을 할 수 있다.
|
|
109
|
+
|
|
110
|
+
- registered range read
|
|
111
|
+
- Developer Metadata anchor 검색
|
|
112
|
+
- 누락 anchor 생성
|
|
113
|
+
- values/formulas/display values/merged range 읽기
|
|
114
|
+
- snapshot hash 생성
|
|
115
|
+
- duplicate anchor 진단
|
|
116
|
+
|
|
117
|
+
반면 `READ_SNAPSHOT_OPERATION_SOURCE`는 anchor mutation이 없는 read-only snapshot path로 만들기 위해 lock prologue/epilogue를 제거한 별도 source다.
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## 4. 왜 `batchUpdate`만으로 문제가 해결되지 않는가
|
|
122
|
+
|
|
123
|
+
Advanced Sheets `batchUpdate`는 **하나의 API 요청 안의 request 배열을 원자적으로 처리**할 수 있다. 그러나 다음까지 보장하지 않는다.
|
|
124
|
+
|
|
125
|
+
- 서로 다른 두 HTTP 요청 사이의 serializable isolation
|
|
126
|
+
- read 후 write 사이의 compare-and-set
|
|
127
|
+
- effect ID/receipt의 unique constraint
|
|
128
|
+
- developer metadata anchor의 create-if-absent
|
|
129
|
+
- `lastRow` 계산의 동시성 안전성
|
|
130
|
+
- User_Input row에 대한 conditional update
|
|
131
|
+
|
|
132
|
+
예를 들어 두 요청 A/B가 동시에 다음을 수행하면 된다.
|
|
133
|
+
|
|
134
|
+
```text
|
|
135
|
+
A: receipt에 e1 없음
|
|
136
|
+
B: receipt에 e1 없음
|
|
137
|
+
A: identity u1 없음
|
|
138
|
+
B: identity u1 없음
|
|
139
|
+
A: lastRow = 100
|
|
140
|
+
B: lastRow = 100
|
|
141
|
+
A: row 101에 u1 append
|
|
142
|
+
B: row 101 또는 shifted row에 u1 append
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
각 요청의 `batchUpdate`는 내부적으로 원자적일 수 있어도, A와 B 사이의 unique identity 보장은 없다. 락을 완전히 제거하면 다음 문제가 가능하다.
|
|
146
|
+
|
|
147
|
+
- 같은 effect의 duplicate row
|
|
148
|
+
- 같은 business identity의 duplicate row
|
|
149
|
+
- 같은 anchor의 duplicate metadata
|
|
150
|
+
- receipt와 data row의 서로 다른 상태
|
|
151
|
+
- stale `User_Input` candidate overwrite
|
|
152
|
+
- response loss 이후 재시도 시 duplicate materialization
|
|
153
|
+
|
|
154
|
+
따라서 “`batchUpdate`이므로 전역 락을 그냥 삭제해도 된다”는 가정은 성립하지 않는다.
|
|
155
|
+
|
|
156
|
+
---
|
|
157
|
+
|
|
158
|
+
## 5. 관측된 실제 증상
|
|
159
|
+
|
|
160
|
+
### 5.1 이전 누적 상태 run
|
|
161
|
+
|
|
162
|
+
이전 run은 stale SQLite/Sheet state와 duplicate/failure effect가 섞여 있었다.
|
|
163
|
+
|
|
164
|
+
- 2,648 requests
|
|
165
|
+
- HTTP failure 12.61%
|
|
166
|
+
- Gateway p50 4.34초
|
|
167
|
+
- Gateway p95 34.25초
|
|
168
|
+
- 최대 60.01초
|
|
169
|
+
|
|
170
|
+
이 run은 신규 구현의 공정한 성능 비교로 사용하면 안 된다.
|
|
171
|
+
|
|
172
|
+
### 5.2 Fresh run
|
|
173
|
+
|
|
174
|
+
- 날짜: 2026-08-03
|
|
175
|
+
- branch: `perf/adaptive-sync-performance`
|
|
176
|
+
- fresh SQLite 및 fresh `System_State`, `User_Input`, `Sync_Conflicts` tabs
|
|
177
|
+
- Locust: 10 users, spawn rate 2/s, 60초
|
|
178
|
+
- command:
|
|
179
|
+
|
|
180
|
+
```sh
|
|
181
|
+
locust -f .local/locustfile.py \
|
|
182
|
+
--host http://127.0.0.1:8787 \
|
|
183
|
+
--headless \
|
|
184
|
+
--users 10 \
|
|
185
|
+
--spawn-rate 2 \
|
|
186
|
+
--run-time 60s \
|
|
187
|
+
--csv .local/locust-20260803-231300-clean2 \
|
|
188
|
+
--html .local/locust-20260803-231300-clean2.html \
|
|
189
|
+
--only-summary
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
Locust 결과:
|
|
193
|
+
|
|
194
|
+
| Endpoint | Requests | Failures | p50 | p95 | Max |
|
|
195
|
+
| --- | ---: | ---: | ---: | ---: | ---: |
|
|
196
|
+
| `GET /users/:id` | 134 | 0 | 2ms | 4ms | 6ms |
|
|
197
|
+
| `POST /users` | 81 | 0 | 5ms | 6ms | 19ms |
|
|
198
|
+
| `PATCH /users/:id` | 76 | 0 | 5ms | 8ms | 22ms |
|
|
199
|
+
| `POST /__test/user-input` | 38 | 8 | 2.2s | 31.0s | 31.9s |
|
|
200
|
+
| **Total** | **339** | **8 (2.36%)** | **4ms** | **2.3s** | **31.9s** |
|
|
201
|
+
|
|
202
|
+
서버-side Gateway snapshot은 workload 종료 후 background worker/polling을 포함해 다음과 같았다.
|
|
203
|
+
|
|
204
|
+
- Gateway requests: 100
|
|
205
|
+
- Gateway failures: 34
|
|
206
|
+
- Gateway p50: 3.71초
|
|
207
|
+
- Gateway p95: 33.47초
|
|
208
|
+
- Gateway max: 60.00초
|
|
209
|
+
- outbox: `pending 306`, `processing 8`, `applied 2`, `superseded 2`
|
|
210
|
+
|
|
211
|
+
주요 오류:
|
|
212
|
+
|
|
213
|
+
- `Code.gs response was not valid JSON`
|
|
214
|
+
- `Code.gs operation request timed out`
|
|
215
|
+
- 이전 run에는 `Could not acquire the sync observation gateway lock`도 관측됨
|
|
216
|
+
|
|
217
|
+
Fresh server startup 중에도 Gateway가 간헐적으로 HTTP 404/non-JSON을 반환했다. idle 상태에서 단일 signed no-op probe를 순차 실행하면 성공하기도 했으므로, 단순한 함수 문법 오류보다 원격 실행 queue/lock/transport/deployment 상태를 분리해 조사해야 한다.
|
|
218
|
+
|
|
219
|
+
### 5.3 해석
|
|
220
|
+
|
|
221
|
+
이 run은 다음을 보여준다.
|
|
222
|
+
|
|
223
|
+
- local SQLite/entity flush 경로는 낮은 latency로 동작한다.
|
|
224
|
+
- entity create/read/update HTTP API는 실패하지 않았다.
|
|
225
|
+
- Gateway boundary는 10-user 정도의 mixed workload에서도 tail latency와 non-JSON/timeout을 보였다.
|
|
226
|
+
- `User_Input` test control path가 stage-1 bulk append benchmark와 섞여 결과를 오염시켰다.
|
|
227
|
+
- outbox가 workload보다 빠르게 drain되지 않았으므로 성공적인 성능 benchmark가 아니다.
|
|
228
|
+
|
|
229
|
+
따라서 현재 결과만으로 `batchUpdate` 자체의 throughput이나 correctness를 판정할 수 없다. 하지만 Gateway의 concurrency/transport 병목은 실제로 존재한다.
|
|
230
|
+
|
|
231
|
+
---
|
|
232
|
+
|
|
233
|
+
## 6. 해결책을 설계할 때 반드시 분리할 문제
|
|
234
|
+
|
|
235
|
+
### A. Transport/deployment 문제
|
|
236
|
+
|
|
237
|
+
다음은 Script Lock 제거와 별개일 수 있다.
|
|
238
|
+
|
|
239
|
+
- HTTP 404
|
|
240
|
+
- redirect 뒤 non-JSON response
|
|
241
|
+
- Apps Script execution timeout
|
|
242
|
+
- deployed Code.gs 버전/manifest/Advanced Sheets service 불일치
|
|
243
|
+
- Gateway quota 또는 transient execution failure
|
|
244
|
+
|
|
245
|
+
먼저 no-op signed operation, provisioning operation, single append operation을 각각 독립적으로 측정해야 한다.
|
|
246
|
+
|
|
247
|
+
### B. Script Lock 경합
|
|
248
|
+
|
|
249
|
+
다음은 현재 전역 Script Lock으로 직접 악화될 수 있다.
|
|
250
|
+
|
|
251
|
+
- effect operation이 receipt/identity 전체 scan 중인 동안 observation이 대기
|
|
252
|
+
- full metadata observation이 anchor/hash를 계산하는 동안 append가 대기
|
|
253
|
+
- 20초 `tryLock` timeout과 60초 HTTP timeout이 겹침
|
|
254
|
+
- polling, outbound worker, test control operation이 동일 spreadsheet/script에 동시에 접근
|
|
255
|
+
|
|
256
|
+
### C. 락 제거 시 생기는 정합성 문제
|
|
257
|
+
|
|
258
|
+
락을 삭제하면 해결되는 latency와 새로 생기는 correctness failure를 구분해야 한다.
|
|
259
|
+
|
|
260
|
+
- lock-free append에서 duplicate identity를 어떻게 막을 것인가?
|
|
261
|
+
- receipt의 create-if-absent를 어떻게 보장할 것인가?
|
|
262
|
+
- `lastRow`/row reservation race를 어떻게 처리할 것인가?
|
|
263
|
+
- stale CAS를 어떻게 원격에서 거절할 것인가?
|
|
264
|
+
- response loss 후 같은 effect의 duplicate row를 어떻게 방지할 것인가?
|
|
265
|
+
- 여러 Node process가 동시에 같은 spreadsheet를 dispatch할 때 어떻게 fencing할 것인가?
|
|
266
|
+
|
|
267
|
+
---
|
|
268
|
+
|
|
269
|
+
## 7. 우선 검토할 대체 설계
|
|
270
|
+
|
|
271
|
+
아래 설계를 비교하고, 더 나은 설계가 있다면 이유를 제시하라.
|
|
272
|
+
|
|
273
|
+
### 설계안 1: SQLite durable single remote writer
|
|
274
|
+
|
|
275
|
+
- spreadsheet/route별 remote dispatch ownership을 SQLite writer lease로 결정한다.
|
|
276
|
+
- lease holder만 해당 spreadsheet의 mutation Gateway call을 보낸다.
|
|
277
|
+
- 효과를 메모리에 오래 모으지 않고, SQLite outbox에서 bounded batch만 읽어 보낸다.
|
|
278
|
+
- process가 죽어도 outbox가 남으므로 재시작 후 recovery한다.
|
|
279
|
+
- 같은 process의 worker/polling/test control이 Gateway로 직접 병렬 mutation을 보내지 않게 한다.
|
|
280
|
+
- remote lock을 제거하거나 최소화하고, serialization을 SQLite authority 쪽으로 옮긴다.
|
|
281
|
+
|
|
282
|
+
검토할 것:
|
|
283
|
+
|
|
284
|
+
- polling read는 mutation writer와 병렬화해도 되는가?
|
|
285
|
+
- full observation의 anchor mutation은 어느 lane에 넣어야 하는가?
|
|
286
|
+
- 여러 Node process의 writer lease fencing이 충분한가?
|
|
287
|
+
- lease 만료 중 remote call이 끝나는 경우 response-loss/postcondition 처리는 어떻게 하는가?
|
|
288
|
+
|
|
289
|
+
### 설계안 2: operation class별 락
|
|
290
|
+
|
|
291
|
+
- 순수 values/read snapshot: lock-free
|
|
292
|
+
- append-only `System_State`/`Sync_Conflicts`: 별도 mutation lane 또는 짧은 route별 serialization
|
|
293
|
+
- CAS update/delete/User_Input: 당분간 serialization 유지
|
|
294
|
+
- full metadata/anchor assignment: 낮은 빈도의 별도 safety lane
|
|
295
|
+
|
|
296
|
+
Apps Script `LockService`에는 일반적인 arbitrary route key lock이 없으므로, “route별 lock”을 제안할 경우 실제 구현 방식(SQLite lease, PropertiesService, sheet lease row 등)의 원자성과 crash recovery를 설명해야 한다.
|
|
297
|
+
|
|
298
|
+
### 설계안 3: append-only command/effect ledger
|
|
299
|
+
|
|
300
|
+
Sheet에 직접 unique row를 materialize하기보다 effect command/receipt를 append-only로 기록하고, 단일 materializer가 projection row를 만든다.
|
|
301
|
+
|
|
302
|
+
장점:
|
|
303
|
+
|
|
304
|
+
- append-only write가 row update보다 단순하다.
|
|
305
|
+
- replay와 audit가 쉽다.
|
|
306
|
+
|
|
307
|
+
단점:
|
|
308
|
+
|
|
309
|
+
- ledger 자체의 duplicate effect ID 방지 문제가 남는다.
|
|
310
|
+
- materializer가 결국 단일 writer가 되어야 한다.
|
|
311
|
+
- 기존 Sheet 사용자-facing layout과의 migration 비용이 크다.
|
|
312
|
+
|
|
313
|
+
### 설계안 4: optimistic concurrency/version column
|
|
314
|
+
|
|
315
|
+
각 row에 revision/version을 두고 expected version을 같이 전송한다.
|
|
316
|
+
|
|
317
|
+
단, Google Sheets Advanced API의 `batchUpdate`가 서버-side conditional compare-and-set을 제공하는지 확인해야 한다. 단순히 version을 읽은 뒤 update하는 것은 CAS가 아니다. 실제로 조건부 update를 보장하지 못한다면 User_Input guarded path의 정합성 대체안으로 인정하지 않는다.
|
|
318
|
+
|
|
319
|
+
---
|
|
320
|
+
|
|
321
|
+
## 8. 권장 단계
|
|
322
|
+
|
|
323
|
+
1. **Gateway 단독 안정성 확인**
|
|
324
|
+
- signed no-op 10회
|
|
325
|
+
- provisioning/read 10회
|
|
326
|
+
- 단일 append 1회
|
|
327
|
+
- status, redirect, body classification, execution duration 기록
|
|
328
|
+
|
|
329
|
+
2. **단일 effect correctness 확인**
|
|
330
|
+
- `System_State` append
|
|
331
|
+
- receipt 확인
|
|
332
|
+
- 같은 effect replay
|
|
333
|
+
- payload hash mismatch
|
|
334
|
+
- response loss simulation
|
|
335
|
+
- duplicate identity simulation
|
|
336
|
+
|
|
337
|
+
3. **stage-1 benchmark 분리**
|
|
338
|
+
- Locust에서 `User_Input` control traffic 제외
|
|
339
|
+
- `System_State`와 `Sync_Conflicts` outbound path만 측정
|
|
340
|
+
- batch 1/5/10/20 sweep
|
|
341
|
+
- workload window와 background drain window를 분리
|
|
342
|
+
|
|
343
|
+
4. **concurrency correctness 확인**
|
|
344
|
+
- 같은 effect ID 동시 2회
|
|
345
|
+
- 서로 다른 effect의 동시 append
|
|
346
|
+
- 같은 business identity 동시 append
|
|
347
|
+
- full observation과 append 동시 실행
|
|
348
|
+
- worker lease expiry 중 원격 response 도착
|
|
349
|
+
|
|
350
|
+
5. **User_Input 별도 검증**
|
|
351
|
+
- guarded update/delete의 stale candidate 거부
|
|
352
|
+
- remote candidate 재관찰/quarantine
|
|
353
|
+
- CAS path를 lock-free로 바꿀 수 있는지 별도 판단
|
|
354
|
+
|
|
355
|
+
6. **최종 Locust**
|
|
356
|
+
- fresh SQLite/tabs
|
|
357
|
+
- 한 writer/한 Gateway deployment
|
|
358
|
+
- Gateway p50/p95/max
|
|
359
|
+
- non-JSON/timeout 비율
|
|
360
|
+
- lock wait
|
|
361
|
+
- pending/processing/applied/failed/superseded/blocked_candidate
|
|
362
|
+
- outbox drain rate와 effect creation rate 비교
|
|
363
|
+
|
|
364
|
+
---
|
|
365
|
+
|
|
366
|
+
## 9. 금지해야 할 단순 해결책
|
|
367
|
+
|
|
368
|
+
다음 제안은 충분한 정합성 증명 없이는 채택하지 않는다.
|
|
369
|
+
|
|
370
|
+
- `LockService`를 모두 삭제하고 retry만 추가
|
|
371
|
+
- `batchUpdate`이므로 cross-request CAS도 된다고 가정
|
|
372
|
+
- duplicate identity를 자동 삭제
|
|
373
|
+
- receipt에 effect ID만 있으면 성공으로 판정
|
|
374
|
+
- timeout을 늘려 lock contention을 숨김
|
|
375
|
+
- pending effect를 메모리에 모아 Gateway latency를 숨김
|
|
376
|
+
- User_Input 실패를 전체 benchmark에서 제외하고 성공으로 보고
|
|
377
|
+
- 404/non-JSON을 단순 retry로만 처리
|
|
378
|
+
- public API나 `Code.gs` dispatcher를 변경해 내부 병목을 숨김
|
|
379
|
+
|
|
380
|
+
---
|
|
381
|
+
|
|
382
|
+
## 10. 답변을 작성할 GPT에게 요구하는 결과
|
|
383
|
+
|
|
384
|
+
다음 형식으로 답하라.
|
|
385
|
+
|
|
386
|
+
1. **Root cause 분리**
|
|
387
|
+
- Script Lock contention
|
|
388
|
+
- Apps Script/HTTP transport/deployment instability
|
|
389
|
+
- read/validate/write race
|
|
390
|
+
- worker scheduling contention
|
|
391
|
+
을 각각 어느 증거가 지지하는지 설명하라.
|
|
392
|
+
|
|
393
|
+
2. **추천 consistency model**
|
|
394
|
+
- 어떤 경로에서 lock을 제거하는가?
|
|
395
|
+
- 무엇이 SQLite에서 serialize되는가?
|
|
396
|
+
- 어떤 경로는 왜 lock/CAS가 여전히 필요한가?
|
|
397
|
+
- response loss와 duplicate identity를 어떻게 처리하는가?
|
|
398
|
+
|
|
399
|
+
3. **상태 전이와 fencing**
|
|
400
|
+
- outbox status
|
|
401
|
+
- writer lease
|
|
402
|
+
- effect lease
|
|
403
|
+
- remote receipt
|
|
404
|
+
- postcondition
|
|
405
|
+
사이의 정확한 전이를 제시하라.
|
|
406
|
+
|
|
407
|
+
4. **구체적인 코드 변경 위치**
|
|
408
|
+
- 변경할 파일
|
|
409
|
+
- 변경하지 않을 파일
|
|
410
|
+
- operation source의 critical section
|
|
411
|
+
- worker dispatch/scheduler
|
|
412
|
+
- 필요한 schema/telemetry
|
|
413
|
+
를 명시하라.
|
|
414
|
+
|
|
415
|
+
5. **실패 주입 테스트**
|
|
416
|
+
- timeout
|
|
417
|
+
- HTTP 404/non-JSON
|
|
418
|
+
- response loss
|
|
419
|
+
- lock contention
|
|
420
|
+
- concurrent append
|
|
421
|
+
- stale CAS
|
|
422
|
+
- duplicate identity
|
|
423
|
+
를 재현하고 기대 결과를 정의하라.
|
|
424
|
+
|
|
425
|
+
6. **성능 측정 계획**
|
|
426
|
+
- stage-1과 stage-2를 분리하라.
|
|
427
|
+
- setup/no-setup/steady-state/drain을 분리하라.
|
|
428
|
+
- 성공률뿐 아니라 outbox convergence와 correctness를 포함하라.
|
|
429
|
+
|
|
430
|
+
7. **채택 기준과 rollback 기준**
|
|
431
|
+
- 어떤 수치와 invariant를 만족해야 lock-free 또는 reduced-lock 설계를 채택하는가?
|
|
432
|
+
- 어떤 오류가 발생하면 기존 안전 경로로 rollback하는가?
|
|
433
|
+
|
|
434
|
+
핵심은 **락을 무조건 제거하는 것**이 아니라, Apps Script의 긴 전역 critical section을 줄이면서도 SQLite authority, durable outbox, effect receipt, CAS/fencing, duplicate fail-closed를 유지하는 것이다.
|
package/docs/architecture.md
CHANGED
|
@@ -95,9 +95,11 @@ projection registry/state
|
|
|
95
95
|
Sheet effect outbox
|
|
96
96
|
```
|
|
97
97
|
|
|
98
|
-
The service-side configuration supplies
|
|
99
|
-
and
|
|
100
|
-
options.
|
|
98
|
+
The service-side configuration supplies the required `System_State`,
|
|
99
|
+
`User_Input`, and `Sync_Conflicts` routes, spreadsheet identity, and user-owned
|
|
100
|
+
fields. Those values are not entity metadata or public ORM options. Every
|
|
101
|
+
internal sync runtime fails closed if any of the three physical routes or its
|
|
102
|
+
fixed headers are missing or drifted.
|
|
101
103
|
|
|
102
104
|
## Google Sheets projection
|
|
103
105
|
|
|
@@ -109,11 +111,27 @@ The application does not call Sheet operations. A successful public `flush()`
|
|
|
109
111
|
means that the local SQLite transaction committed. It does not mean that a
|
|
110
112
|
remote Sheet write has completed.
|
|
111
113
|
|
|
114
|
+
The MVP gateway is a single file, `apps-script/gateway/Code.gs`, copied into a
|
|
115
|
+
spreadsheet-bound Apps Script project and deployed as a Web App. The deployer
|
|
116
|
+
pastes the `/exec` URL into `TYPED_SHEETS_GATEWAY_URL` and runs
|
|
117
|
+
`setupSyncGateway()` from the editor, which stores the bound spreadsheet ID and
|
|
118
|
+
a generated shared secret in Script Properties. This default path requires no
|
|
119
|
+
`appsscript.json` manifest, no Apps Script Advanced Sheets Service, no Google
|
|
120
|
+
Cloud Sheets API activation, and no service account. Sheet consistency does
|
|
121
|
+
not rely on cross-request Sheet transactions; it comes from the Apps Script
|
|
122
|
+
script lock, the hidden effect-receipt tab, effect-id/payload-hash dedupe, the
|
|
123
|
+
SQLite durable outbox, fencing, and postcondition recovery.
|
|
124
|
+
|
|
112
125
|
`System_State` is materialized from canonical SQLite state. `User_Input` is
|
|
113
126
|
observed by the internal polling loop, evaluated with ownership and field-level
|
|
114
|
-
compare-and-set rules, and then committed back to SQLite.
|
|
115
|
-
|
|
116
|
-
|
|
127
|
+
compare-and-set rules, and then committed back to SQLite. `Sync_Conflicts` is a
|
|
128
|
+
system-owned audit projection of field-level conflict evidence and resolution
|
|
129
|
+
outcomes; it is created even when empty and resolved rows remain visible for
|
|
130
|
+
audit. A detected conflict is resolved with the fenced `acknowledge_system`
|
|
131
|
+
policy: revision, candidate-hash, and epoch CAS clear the active candidate and
|
|
132
|
+
queue the canonical User_Input rewrite. A newer remote edit that fails that CAS
|
|
133
|
+
starts a new conflict instead of being overwritten. Stale, conflicting, and
|
|
134
|
+
malformed input is therefore never silently written over the entity table.
|
|
117
135
|
|
|
118
136
|
## Transaction and lifecycle boundary
|
|
119
137
|
|
|
@@ -124,26 +142,50 @@ em.persist(entity) / em.remove(entity)
|
|
|
124
142
|
SQLite transaction (internal sync service mode)
|
|
125
143
|
├─ entity table
|
|
126
144
|
├─ canonical state and conflict evidence
|
|
145
|
+
├─ automatic system-wins resolution receipt/state
|
|
127
146
|
└─ durable Sheet effect outbox
|
|
128
147
|
│
|
|
129
148
|
▼
|
|
130
149
|
internal outbound effect supervisor
|
|
131
|
-
├─ claim leases and send signed
|
|
132
|
-
├─
|
|
150
|
+
├─ claim leases and send bounded signed operation batches
|
|
151
|
+
├─ persist delivery uncertainty and due postcondition probes in SQLite
|
|
152
|
+
├─ fence remote routes with the current spreadsheet authority epoch/token
|
|
133
153
|
└─ reconcile remote drift
|
|
134
154
|
│
|
|
135
155
|
▼
|
|
136
156
|
Apps Script gateway ──▶ Google Sheets
|
|
137
157
|
|
|
138
158
|
internal User_Input polling
|
|
139
|
-
├─
|
|
159
|
+
├─ adaptive values-only preflight over registered projections
|
|
160
|
+
├─ escalate changed/ambiguous/schema tables to full metadata
|
|
161
|
+
├─ periodic safety full scan for formula/merged/error fidelity
|
|
140
162
|
├─ evaluate ownership/revisions/CAS
|
|
163
|
+
├─ persist conflict evidence and automatic system-wins resolution in SQLite
|
|
141
164
|
└─ persist accepted observations and entity mutations in SQLite
|
|
142
165
|
```
|
|
143
166
|
|
|
167
|
+
Inbound polling defaults to an adaptive values-only preflight that reads the
|
|
168
|
+
cheap value surface and only escalates a table to the metadata-preserving
|
|
169
|
+
snapshot read when it changed, is ambiguous (unknown or duplicate business key,
|
|
170
|
+
a missing expected entity, or a type-invalid cell), or when the periodic safety
|
|
171
|
+
full scan falls due. The safety scan (default one minute, configurable) keeps
|
|
172
|
+
formula, merged, and error cell fidelity because the values-only read drops that
|
|
173
|
+
metadata; the scan's overdue lag is recorded for diagnostics. Outbound effects
|
|
174
|
+
are split into bounded sub-batches per physical route so one gateway call returns
|
|
175
|
+
a complete result set, and a pass that only requeued uncertain work backs off with
|
|
176
|
+
a bounded delay instead of retrying a struggling remote in a tight loop.
|
|
177
|
+
|
|
144
178
|
The service bootstrap starts provisioning, outbound delivery, and inbound polling
|
|
145
|
-
as one internal runtime.
|
|
146
|
-
|
|
179
|
+
as one internal runtime. Outbound dispatch keeps the SQLite outbox as its only
|
|
180
|
+
durable buffer, coalesces only a short in-process burst, and adapts each route's
|
|
181
|
+
Gateway batch between five and twenty effects from latency and response-loss
|
|
182
|
+
signals. The effect lease is longer than the configured Gateway timeout so a
|
|
183
|
+
slow but valid request is recovered only after its remote result is checked.
|
|
184
|
+
Ambiguous delivery is stored as `delivery_uncertain` with `uncertain_since`,
|
|
185
|
+
`next_probe_at`, and `dispatch_id`; only a due probe may return it to processing.
|
|
186
|
+
A terminal structured remote failure does not enter the ambiguous probe path.
|
|
187
|
+
Shutdown stops polling first, waits for remote calls, stops the outbound
|
|
188
|
+
supervisor, and only then closes SQLite.
|
|
147
189
|
|
|
148
190
|
## Design limits
|
|
149
191
|
|