flatsql 3.11.0 → 3.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (181) hide show
  1. package/README.md +61 -795
  2. package/bin/flatsql.mjs +76 -0
  3. package/cpp/README.md +18 -0
  4. package/cpp/include/flatsql/flatsql.h +162 -0
  5. package/cpp/include/flatsql/flatsql_io.h +159 -0
  6. package/docs/API-3.12.md +114 -0
  7. package/package.json +28 -123
  8. package/wasm/flatsql.wasm +0 -0
  9. package/wasm/index.d.ts +61 -548
  10. package/wasm/index.js +280 -1281
  11. package/wasm/integrity.json +6 -26
  12. package/wasm/runtime/LICENSE.sdk.txt +190 -0
  13. package/wasm/runtime/NOTICE.txt +2 -0
  14. package/wasm/runtime/browser-guest.mjs +1 -0
  15. package/wasm/runtime/browser-host.mjs +1 -0
  16. package/wasm/runtime/node-guest.mjs +2 -0
  17. package/wasm/runtime/node-host.mjs +5 -0
  18. package/wasm/runtime/opfsIoWorker.mjs +1 -0
  19. package/wasm/runtime/wasiThreadBrowserWorker.mjs +1 -0
  20. package/dist/artifacts/index.d.ts +0 -3
  21. package/dist/artifacts/index.d.ts.map +0 -1
  22. package/dist/artifacts/index.js +0 -3
  23. package/dist/artifacts/index.js.map +0 -1
  24. package/dist/artifacts/transport.d.ts +0 -5
  25. package/dist/artifacts/transport.d.ts.map +0 -1
  26. package/dist/artifacts/transport.js +0 -45
  27. package/dist/artifacts/transport.js.map +0 -1
  28. package/dist/artifacts/types.d.ts +0 -34
  29. package/dist/artifacts/types.d.ts.map +0 -1
  30. package/dist/artifacts/types.js +0 -2
  31. package/dist/artifacts/types.js.map +0 -1
  32. package/dist/btree/btree.d.ts +0 -34
  33. package/dist/btree/btree.d.ts.map +0 -1
  34. package/dist/btree/btree.js +0 -234
  35. package/dist/btree/btree.js.map +0 -1
  36. package/dist/btree/index.d.ts +0 -3
  37. package/dist/btree/index.d.ts.map +0 -1
  38. package/dist/btree/index.js +0 -3
  39. package/dist/btree/index.js.map +0 -1
  40. package/dist/btree/types.d.ts +0 -36
  41. package/dist/btree/types.d.ts.map +0 -1
  42. package/dist/btree/types.js +0 -55
  43. package/dist/btree/types.js.map +0 -1
  44. package/dist/cluster/index.d.ts +0 -11
  45. package/dist/cluster/index.d.ts.map +0 -1
  46. package/dist/cluster/index.js +0 -59
  47. package/dist/cluster/index.js.map +0 -1
  48. package/dist/response/artifact.d.ts +0 -4
  49. package/dist/response/artifact.d.ts.map +0 -1
  50. package/dist/response/artifact.js +0 -102
  51. package/dist/response/artifact.js.map +0 -1
  52. package/dist/response/cache-key.d.ts +0 -6
  53. package/dist/response/cache-key.d.ts.map +0 -1
  54. package/dist/response/cache-key.js +0 -114
  55. package/dist/response/cache-key.js.map +0 -1
  56. package/dist/response/index.d.ts +0 -5
  57. package/dist/response/index.d.ts.map +0 -1
  58. package/dist/response/index.js +0 -5
  59. package/dist/response/index.js.map +0 -1
  60. package/dist/response/memory-cache.d.ts +0 -17
  61. package/dist/response/memory-cache.d.ts.map +0 -1
  62. package/dist/response/memory-cache.js +0 -76
  63. package/dist/response/memory-cache.js.map +0 -1
  64. package/dist/response/types.d.ts +0 -57
  65. package/dist/response/types.d.ts.map +0 -1
  66. package/dist/response/types.js +0 -2
  67. package/dist/response/types.js.map +0 -1
  68. package/dist/schema/index.d.ts +0 -3
  69. package/dist/schema/index.d.ts.map +0 -1
  70. package/dist/schema/index.js +0 -3
  71. package/dist/schema/index.js.map +0 -1
  72. package/dist/schema/parser.d.ts +0 -25
  73. package/dist/schema/parser.d.ts.map +0 -1
  74. package/dist/schema/parser.js +0 -474
  75. package/dist/schema/parser.js.map +0 -1
  76. package/dist/schema/types.d.ts +0 -33
  77. package/dist/schema/types.d.ts.map +0 -1
  78. package/dist/schema/types.js +0 -29
  79. package/dist/schema/types.js.map +0 -1
  80. package/dist/standalone/index.d.ts +0 -3
  81. package/dist/standalone/index.d.ts.map +0 -1
  82. package/dist/standalone/index.js +0 -3
  83. package/dist/standalone/index.js.map +0 -1
  84. package/dist/standalone/process-client.d.ts +0 -85
  85. package/dist/standalone/process-client.d.ts.map +0 -1
  86. package/dist/standalone/process-client.js +0 -537
  87. package/dist/standalone/process-client.js.map +0 -1
  88. package/dist/standalone/wasmedge-runner.d.ts +0 -19
  89. package/dist/standalone/wasmedge-runner.d.ts.map +0 -1
  90. package/dist/standalone/wasmedge-runner.js +0 -108
  91. package/dist/standalone/wasmedge-runner.js.map +0 -1
  92. package/dist/storage/index.d.ts +0 -3
  93. package/dist/storage/index.d.ts.map +0 -1
  94. package/dist/storage/index.js +0 -3
  95. package/dist/storage/index.js.map +0 -1
  96. package/dist/storage/stacked-flatbuffers.d.ts +0 -38
  97. package/dist/storage/stacked-flatbuffers.d.ts.map +0 -1
  98. package/dist/storage/stacked-flatbuffers.js +0 -253
  99. package/dist/storage/stacked-flatbuffers.js.map +0 -1
  100. package/dist/storage/types.d.ts +0 -23
  101. package/dist/storage/types.d.ts.map +0 -1
  102. package/dist/storage/types.js +0 -27
  103. package/dist/storage/types.js.map +0 -1
  104. package/dist/stress/aggregate.d.ts +0 -3
  105. package/dist/stress/aggregate.d.ts.map +0 -1
  106. package/dist/stress/aggregate.js +0 -61
  107. package/dist/stress/aggregate.js.map +0 -1
  108. package/dist/stress/docker-compose.d.ts +0 -17
  109. package/dist/stress/docker-compose.d.ts.map +0 -1
  110. package/dist/stress/docker-compose.js +0 -156
  111. package/dist/stress/docker-compose.js.map +0 -1
  112. package/dist/stress/index.d.ts +0 -10
  113. package/dist/stress/index.d.ts.map +0 -1
  114. package/dist/stress/index.js +0 -10
  115. package/dist/stress/index.js.map +0 -1
  116. package/dist/stress/metrics.d.ts +0 -3
  117. package/dist/stress/metrics.d.ts.map +0 -1
  118. package/dist/stress/metrics.js +0 -83
  119. package/dist/stress/metrics.js.map +0 -1
  120. package/dist/stress/native-publish-event-workload.d.ts +0 -3
  121. package/dist/stress/native-publish-event-workload.d.ts.map +0 -1
  122. package/dist/stress/native-publish-event-workload.js +0 -465
  123. package/dist/stress/native-publish-event-workload.js.map +0 -1
  124. package/dist/stress/production-gate.d.ts +0 -4
  125. package/dist/stress/production-gate.d.ts.map +0 -1
  126. package/dist/stress/production-gate.js +0 -12
  127. package/dist/stress/production-gate.js.map +0 -1
  128. package/dist/stress/report.d.ts +0 -3
  129. package/dist/stress/report.d.ts.map +0 -1
  130. package/dist/stress/report.js +0 -174
  131. package/dist/stress/report.js.map +0 -1
  132. package/dist/stress/sds-discovery.d.ts +0 -4
  133. package/dist/stress/sds-discovery.d.ts.map +0 -1
  134. package/dist/stress/sds-discovery.js +0 -40
  135. package/dist/stress/sds-discovery.js.map +0 -1
  136. package/dist/stress/smoke-runner.d.ts +0 -3
  137. package/dist/stress/smoke-runner.d.ts.map +0 -1
  138. package/dist/stress/smoke-runner.js +0 -64
  139. package/dist/stress/smoke-runner.js.map +0 -1
  140. package/dist/stress/types.d.ts +0 -219
  141. package/dist/stress/types.d.ts.map +0 -1
  142. package/dist/stress/types.js +0 -2
  143. package/dist/stress/types.js.map +0 -1
  144. package/dist/stress/workload.d.ts +0 -6
  145. package/dist/stress/workload.d.ts.map +0 -1
  146. package/dist/stress/workload.js +0 -144
  147. package/dist/stress/workload.js.map +0 -1
  148. package/docs/PARTITION-STORE-WASM.md +0 -338
  149. package/docs/PARTITION-STORE.md +0 -2380
  150. package/docs/agents/response-artifacts.md +0 -65
  151. package/docs/response-artifacts.md +0 -86
  152. package/scripts/stress-docker.sh +0 -279
  153. package/sdm/flatsql-spatial.wasm +0 -0
  154. package/sdm/index.d.ts +0 -88
  155. package/sdm/index.js +0 -253
  156. package/sdm/manifest.json +0 -402
  157. package/stress/docker/Dockerfile +0 -11
  158. package/stress/docker/docker-compose.yml +0 -51
  159. package/stress/sds/aggregate.mjs +0 -97
  160. package/stress/sds/run.mjs +0 -105
  161. package/stress/sds/write-compose.mjs +0 -96
  162. package/wasm/flatsql-client.js +0 -280
  163. package/wasm/flatsql-io.d.ts +0 -97
  164. package/wasm/flatsql-io.js +0 -621
  165. package/wasm/flatsql-p4-threads.wasm +0 -0
  166. package/wasm/flatsql-ps-threads.wasm +0 -0
  167. package/wasm/flatsql-wasi-noeh.wasm +0 -0
  168. package/wasm/flatsql-wasi.wasm +0 -0
  169. package/wasm/flatsql.js +0 -2
  170. package/wasm/flatsql.worker.js +0 -209
  171. package/wasm/native/flatsql_wasmedge_runner.cpp +0 -858
  172. package/wasm/ps-node-fault.d.mts +0 -23
  173. package/wasm/ps-node-fault.mjs +0 -381
  174. package/wasm/ps-node-guest.mjs +0 -173
  175. package/wasm/ps-node-host.d.mts +0 -95
  176. package/wasm/ps-node-host.mjs +0 -476
  177. package/wasm/ps-node-imports.mjs +0 -847
  178. package/wasm/ps.d.ts +0 -21
  179. package/wasm/ps.js +0 -75
  180. package/wasm/wasi.d.ts +0 -11
  181. package/wasm/wasi.js +0 -51
@@ -1,144 +0,0 @@
1
- export const DEFAULT_USE_CASES = [
2
- {
3
- id: 'schema-sweep',
4
- label: 'SDS schema sweep',
5
- operation: 'schema-parse',
6
- description: 'Discover every SDS FlatBuffer schema and record compatibility metadata.',
7
- },
8
- {
9
- id: 'bulk-streaming-ingest',
10
- label: 'Bulk streaming ingest',
11
- operation: 'stream-ingest',
12
- description: 'Stream generated FlatBuffers in size-prefixed batches.',
13
- },
14
- {
15
- id: 'hot-file-id-cache',
16
- label: 'Hot FILE_ID query cache',
17
- operation: 'query-template',
18
- description: 'Repeat the same PNM/DPM-style FILE_ID lookup to measure cache behavior.',
19
- },
20
- {
21
- id: 'cold-file-id-fanout',
22
- label: 'Cold FILE_ID fanout',
23
- operation: 'query-template',
24
- description: 'Fan out mostly unique FILE_ID lookups and measure cache churn.',
25
- },
26
- {
27
- id: 'mixed-hot-cold-query',
28
- label: 'Mixed hot/cold query distribution',
29
- operation: 'query-template',
30
- description: 'Run a skewed query mix with hot and long-tail FILE_ID values.',
31
- },
32
- {
33
- id: 'raw-flatbuffer-retrieval',
34
- label: 'Raw FlatBuffer retrieval',
35
- operation: 'raw-flatbuffer',
36
- description: 'Return original FlatBuffer bytes through indexed lookup paths.',
37
- },
38
- {
39
- id: 'sql-projection-query',
40
- label: 'SQL projection query',
41
- operation: 'parameterized-sql',
42
- description: 'Return a narrow SQL projection to compare against binary payload transfer.',
43
- },
44
- {
45
- id: 'raw-sql-query',
46
- label: 'Raw SQL query',
47
- operation: 'raw-sql',
48
- description: 'Execute untemplated SQL and compare request overhead with query templates.',
49
- },
50
- {
51
- id: 'large-result-query',
52
- label: 'Large result query',
53
- operation: 'large-result',
54
- description: 'Return more rows than the result-cache row cap and measure transport cost.',
55
- },
56
- {
57
- id: 'node-fanout-query',
58
- label: 'Node fanout query',
59
- operation: 'fanout-query',
60
- description: 'Send one logical query across many nodes and measure tail latency.',
61
- },
62
- {
63
- id: 'backfill-sync',
64
- label: 'Backfill sync',
65
- operation: 'backfill-stream',
66
- description: 'Bootstrap a node by streaming records from another source.',
67
- },
68
- {
69
- id: 'restart-rebuild',
70
- label: 'Restart and rebuild',
71
- operation: 'restart-rebuild',
72
- description: 'Restart a node and measure rebuild or reload behavior.',
73
- },
74
- {
75
- id: 'cache-invalidation-churn',
76
- label: 'Cache invalidation churn',
77
- operation: 'ingest-query-churn',
78
- description: 'Alternate ingest and hot queries to expose broad invalidation cost.',
79
- },
80
- {
81
- id: 'bandwidth-constrained-streaming',
82
- label: 'Bandwidth-constrained streaming',
83
- operation: 'throttled-stream',
84
- description: 'Measure chunked streaming behavior under an artificial transport limit.',
85
- },
86
- {
87
- id: 'sds-domain-query-pack',
88
- label: 'SDS domain query pack',
89
- operation: 'domain-query',
90
- description: 'Exercise named space-domain lookups such as catalog, ephemeris, sensor, and weather queries.',
91
- },
92
- {
93
- id: 'udl-style-api-shape',
94
- label: 'UDL-style API comparison shape',
95
- operation: 'api-shape',
96
- description: 'Run centralized API-shaped requests without calling an external UDL service.',
97
- },
98
- ];
99
- export function normalizeStressRunConfig(config = {}) {
100
- const mode = config.mode ?? 'smoke';
101
- return {
102
- mode,
103
- runId: config.runId ?? `sds-${mode}-${Date.now()}`,
104
- nodeCount: config.nodeCount ?? (mode === 'full' ? 100 : 2),
105
- nodeIdOffset: config.nodeIdOffset ?? 0,
106
- nodeStorageGb: config.nodeStorageGb ?? (mode === 'full' ? 1 : 0.01),
107
- recordsPerNode: config.recordsPerNode ?? (mode === 'full' ? 20_000 : 100),
108
- batchBytes: config.batchBytes ?? 1024 * 1024,
109
- queryConcurrency: config.queryConcurrency ?? (mode === 'full' ? 64 : 4),
110
- hotQueryRatio: config.hotQueryRatio ?? 0.9,
111
- runtime: config.runtime ?? 'standalone',
112
- transport: config.transport ?? 'in-process',
113
- nodeIsolation: config.nodeIsolation ?? 'logical',
114
- seed: config.seed ?? 0x5d5,
115
- };
116
- }
117
- export function buildNodeAssignments(schemas, config) {
118
- const schemaNames = schemas.map((schema) => schema.name);
119
- return Array.from({ length: config.nodeCount }, (_, localNodeIndex) => {
120
- const nodeId = config.nodeIdOffset + localNodeIndex;
121
- const assigned = schemaNames.filter((_, index) => index % config.nodeCount === localNodeIndex);
122
- if (assigned.length === 0 && schemaNames.length > 0) {
123
- assigned.push(schemaNames[nodeId % schemaNames.length]);
124
- }
125
- return {
126
- nodeId,
127
- storageGb: config.nodeStorageGb,
128
- schemas: assigned,
129
- };
130
- });
131
- }
132
- export function buildWorkloadManifest(config, schemas) {
133
- const normalized = normalizeStressRunConfig(config);
134
- const sortedSchemas = [...schemas].sort((left, right) => left.name.localeCompare(right.name));
135
- return {
136
- ...normalized,
137
- createdAt: new Date().toISOString(),
138
- schemaCount: sortedSchemas.length,
139
- schemas: sortedSchemas,
140
- useCases: DEFAULT_USE_CASES,
141
- assignments: buildNodeAssignments(sortedSchemas, normalized),
142
- };
143
- }
144
- //# sourceMappingURL=workload.js.map
@@ -1 +0,0 @@
1
- {"version":3,"file":"workload.js","sourceRoot":"","sources":["../../src/stress/workload.ts"],"names":[],"mappings":"AASA,MAAM,CAAC,MAAM,iBAAiB,GAAoB;IAChD;QACE,EAAE,EAAE,cAAc;QAClB,KAAK,EAAE,kBAAkB;QACzB,SAAS,EAAE,cAAc;QACzB,WAAW,EAAE,yEAAyE;KACvF;IACD;QACE,EAAE,EAAE,uBAAuB;QAC3B,KAAK,EAAE,uBAAuB;QAC9B,SAAS,EAAE,eAAe;QAC1B,WAAW,EAAE,wDAAwD;KACtE;IACD;QACE,EAAE,EAAE,mBAAmB;QACvB,KAAK,EAAE,yBAAyB;QAChC,SAAS,EAAE,gBAAgB;QAC3B,WAAW,EAAE,yEAAyE;KACvF;IACD;QACE,EAAE,EAAE,qBAAqB;QACzB,KAAK,EAAE,qBAAqB;QAC5B,SAAS,EAAE,gBAAgB;QAC3B,WAAW,EAAE,gEAAgE;KAC9E;IACD;QACE,EAAE,EAAE,sBAAsB;QAC1B,KAAK,EAAE,mCAAmC;QAC1C,SAAS,EAAE,gBAAgB;QAC3B,WAAW,EAAE,+DAA+D;KAC7E;IACD;QACE,EAAE,EAAE,0BAA0B;QAC9B,KAAK,EAAE,0BAA0B;QACjC,SAAS,EAAE,gBAAgB;QAC3B,WAAW,EAAE,gEAAgE;KAC9E;IACD;QACE,EAAE,EAAE,sBAAsB;QAC1B,KAAK,EAAE,sBAAsB;QAC7B,SAAS,EAAE,mBAAmB;QAC9B,WAAW,EAAE,4EAA4E;KAC1F;IACD;QACE,EAAE,EAAE,eAAe;QACnB,KAAK,EAAE,eAAe;QACtB,SAAS,EAAE,SAAS;QACpB,WAAW,EAAE,4EAA4E;KAC1F;IACD;QACE,EAAE,EAAE,oBAAoB;QACxB,KAAK,EAAE,oBAAoB;QAC3B,SAAS,EAAE,cAAc;QACzB,WAAW,EAAE,4EAA4E;KAC1F;IACD;QACE,EAAE,EAAE,mBAAmB;QACvB,KAAK,EAAE,mBAAmB;QAC1B,SAAS,EAAE,cAAc;QACzB,WAAW,EAAE,oEAAoE;KAClF;IACD;QACE,EAAE,EAAE,eAAe;QACnB,KAAK,EAAE,eAAe;QACtB,SAAS,EAAE,iBAAiB;QAC5B,WAAW,EAAE,4DAA4D;KAC1E;IACD;QACE,EAAE,EAAE,iBAAiB;QACrB,KAAK,EAAE,qBAAqB;QAC5B,SAAS,EAAE,iBAAiB;QAC5B,WAAW,EAAE,wDAAwD;KACtE;IACD;QACE,EAAE,EAAE,0BAA0B;QAC9B,KAAK,EAAE,0BAA0B;QACjC,SAAS,EAAE,oBAAoB;QAC/B,WAAW,EAAE,qEAAqE;KACnF;IACD;QACE,EAAE,EAAE,iCAAiC;QACrC,KAAK,EAAE,iCAAiC;QACxC,SAAS,EAAE,kBAAkB;QAC7B,WAAW,EAAE,yEAAyE;KACvF;IACD;QACE,EAAE,EAAE,uBAAuB;QAC3B,KAAK,EAAE,uBAAuB;QAC9B,SAAS,EAAE,cAAc;QACzB,WAAW,EAAE,8FAA8F;KAC5G;IACD;QACE,EAAE,EAAE,qBAAqB;QACzB,KAAK,EAAE,gCAAgC;QACvC,SAAS,EAAE,WAAW;QACtB,WAAW,EAAE,8EAA8E;KAC5F;CACF,CAAC;AAEF,MAAM,UAAU,wBAAwB,CAAC,SAA0B,EAAE;IACnE,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,IAAI,OAAO,CAAC;IACpC,OAAO;QACL,IAAI;QACJ,KAAK,EAAE,MAAM,CAAC,KAAK,IAAI,OAAO,IAAI,IAAI,IAAI,CAAC,GAAG,EAAE,EAAE;QAClD,SAAS,EAAE,MAAM,CAAC,SAAS,IAAI,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC;QAC1D,YAAY,EAAE,MAAM,CAAC,YAAY,IAAI,CAAC;QACtC,aAAa,EAAE,MAAM,CAAC,aAAa,IAAI,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QACnE,cAAc,EAAE,MAAM,CAAC,cAAc,IAAI,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,GAAG,CAAC;QACzE,UAAU,EAAE,MAAM,CAAC,UAAU,IAAI,IAAI,GAAG,IAAI;QAC5C,gBAAgB,EAAE,MAAM,CAAC,gBAAgB,IAAI,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;QACvE,aAAa,EAAE,MAAM,CAAC,aAAa,IAAI,GAAG;QAC1C,OAAO,EAAE,MAAM,CAAC,OAAO,IAAI,YAAY;QACvC,SAAS,EAAE,MAAM,CAAC,SAAS,IAAI,YAAY;QAC3C,aAAa,EAAE,MAAM,CAAC,aAAa,IAAI,SAAS;QAChD,IAAI,EAAE,MAAM,CAAC,IAAI,IAAI,KAAK;KAC3B,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,oBAAoB,CAClC,OAAiC,EACjC,MAAuF;IAEvF,MAAM,WAAW,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;IACzD,OAAO,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,SAAS,EAAE,EAAE,CAAC,CAAC,EAAE,cAAc,EAAE,EAAE;QACpE,MAAM,MAAM,GAAG,MAAM,CAAC,YAAY,GAAG,cAAc,CAAC;QACpD,MAAM,QAAQ,GAAG,WAAW,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,GAAG,MAAM,CAAC,SAAS,KAAK,cAAc,CAAC,CAAC;QAC/F,IAAI,QAAQ,CAAC,MAAM,KAAK,CAAC,IAAI,WAAW,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YACpD,QAAQ,CAAC,IAAI,CAAC,WAAW,CAAC,MAAM,GAAG,WAAW,CAAC,MAAM,CAAC,CAAC,CAAC;QAC1D,CAAC;QACD,OAAO;YACL,MAAM;YACN,SAAS,EAAE,MAAM,CAAC,aAAa;YAC/B,OAAO,EAAE,QAAQ;SAClB,CAAC;IACJ,CAAC,CAAC,CAAC;AACL,CAAC;AAED,MAAM,UAAU,qBAAqB,CACnC,MAAuB,EACvB,OAAiC;IAEjC,MAAM,UAAU,GAAG,wBAAwB,CAAC,MAAM,CAAC,CAAC;IACpD,MAAM,aAAa,GAAG,CAAC,GAAG,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC;IAC9F,OAAO;QACL,GAAG,UAAU;QACb,SAAS,EAAE,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE;QACnC,WAAW,EAAE,aAAa,CAAC,MAAM;QACjC,OAAO,EAAE,aAAa;QACtB,QAAQ,EAAE,iBAAiB;QAC3B,WAAW,EAAE,oBAAoB,CAAC,aAAa,EAAE,UAAU,CAAC;KAC7D,CAAC;AACJ,CAAC"}
@@ -1,338 +0,0 @@
1
- # FlatSQL partition store: the wasm artifact and the Node host (T4)
2
-
3
- The partition store engine (docs/PARTITION-STORE.md, T1 and T2) built as one
4
- `wasm32-wasip1-threads` artifact, `wasm/flatsql-ps-threads.wasm`, and the
5
- Node host that runs it and the engine's test suites with real threads. The
6
- design is the stack's `docs/architecture/flatsql-partition-store.md`: §5.5,
7
- §5.6, §18 T4 and the §22 amendments naming T4 (A1, A34, 22.3a-6/7). This file
8
- records what was built, how to run it, the measured acceptance, and every
9
- departure from the design.
10
-
11
- ## 1. The artifact
12
-
13
- | | |
14
- |---|---|
15
- | Build | `bash scripts/build-wasm.sh --ps` (artifact) or `--ps-tests` (plus the test commands). CMake: `cpp/cmake/wasi-threads-toolchain.cmake` and `cpp/cmake/flatsql_ps_wasm.cmake`, included from `cpp/CMakeLists.txt` only under the WASI toolchain. Engine and test sources are globbed (`src/ps/*.cpp`, `test/ps/*_test.cpp`), so files other tasks add build here unchanged. |
16
- | Toolchain | wasi-sdk 30.0 (clang 21.1.4), target `wasm32-wasip1-threads`, `-pthread -matomics -mbulk-memory -fno-exceptions`, Release (`-O3 -DNDEBUG`), `-ffile-prefix-map` for the source, flatbuffers and SDK paths, `--strip-debug`. The build script refuses any other wasi-sdk version. |
17
- | Bytes | The released artifact is the Linux wasi-sdk 30 build: Linux arm64 (Docker) and the x86_64 CI runner produce the same sha256 from different checkout paths. The macOS wasi-sdk 30 tarball ships a different sysroot (libc, libc++), compiler-rt and clang build and links different (equivalent) bytes; on macOS `bash scripts/build-wasm.sh --ps --linux` builds the released bytes in Docker (the Linux tarball, sha256-pinned). `npm-publish.yml` rebuilds and refuses to publish unless the build equals the committed file; CI warns when it differs. `wasm/integrity.json` holds the sha256, sha384 SRI and size of every shipped wasm (`scripts/write-integrity.mjs`, `npm run check:integrity`). |
18
- | Shape | A reactor (`-mexec-model=reactor`). A host instantiates one instance per writer or reader instance; every guest thread is a new instance of the same module over the same shared memory. |
19
- | Imports | `wasi_snapshot_preview1`: `clock_time_get fd_close fd_prestat_dir_name fd_prestat_get fd_seek fd_write proc_exit random_get sched_yield` (all provided by the SDK's browser and Node pool hosts); `wasi.thread-spawn`; `env.memory` (shared, 512 initial, 32768 maximum pages); the seven `env.flatsql_io_*`. Nothing else: `scripts/check-wasm-imports.mjs` and `test/standalone-wasm-imports.test.ts` fail on any change. |
20
- | Exports | `flatsql_ps_init start stop pump wake layout reader_layout stats register_type register_partition ring` (docs/PARTITION-STORE.md §6), `flatsql_ps_alloc`/`flatsql_ps_free` (host-owned buffers for config TLVs and paths), `wasi_thread_start`, `_initialize`, `memory`. |
21
- | SQLite | The lanes' SQL layer (ruling 1): `SQLITE_THREADSAFE=2`, `DEFAULT_MEMSTATUS=0`, `OMIT_WAL`, `TEMP_STORE=3`, `SQLITE_OS_OTHER`. No unix VFS (it would import WASI file calls): `src/ps/capi_wasm.cpp` registers a default VFS that opens no file, and installs pthread mutexes, because `SQLITE_OS_OTHER` compiles only no-op ones. The lanes open `:memory:` on `flatsql_ps_null` as natively. |
22
- | Threads | `std::thread` defaults to 1 MiB stacks (wasi-libc's default is 128 KiB and a shadow stack has no guard page); lanes set their own (`ReaderConfig::stackBytes`). `sleepNs` is a timed `memory.atomic.wait32`, so the artifact never imports `poll_oneoff`. |
23
- | Guard | `assertPthreadArtifact` (space-data-module-sdk 0.8.24) passes: shared memory, 1,636 atomic instructions, the wasi-threads contract, no emscripten thread hooks. |
24
-
25
- ### Host obligations
26
-
27
- - Create `env.memory` shared, with a maximum no larger than the import's
28
- (browser writer 256 MiB, reader 128 MiB, design §5.5).
29
- - **V8 hosts (Node, browsers): grow the heap to the memory's maximum before
30
- `flatsql_ps_start`, so it never grows while guest threads run.** Allocate
31
- through `flatsql_ps_alloc` in steps from 1 GiB down to 1 MiB until each
32
- fails, then free them all (the allocator keeps the blocks; wasm memory never
33
- shrinks; the guest allocator grows at most 2 GiB per call). V8 bounds-checks
34
- `memory.fill`/`memory.copy` and atomics against each thread's cached view of
35
- a shared memory's size, and another thread's `memory.grow` updates that view
36
- only at this thread's next stack check. A thread that fills a block carved
37
- from memory another thread has just grown then traps "memory access out of
38
- bounds" (§4.3). A partial pre-grow only moves the failure: the 1,536 MiB the
39
- test main used to grow was passed at 1,651 MiB in CI. Stop 16 MiB short of
40
- 4 GiB with a 4 GiB maximum: a heap segment ending at 2^32 wraps the
41
- allocator's pointer arithmetic. `createPsNodeInstance` does this
42
- (`growHeap: false` opts out, `heapBytes` caps it), and the Node host reports
43
- `memoryGrownWhileThreadsRan` for every guest and warns when it is not 0.
44
- - **WasmEdge hosts do not pre-grow** (and SDN recycles reader instances on
45
- their page count). WasmEdge keeps one page count shared by every thread's
46
- instance and grows in place in a fixed reservation, so a grow is visible to
47
- every thread at once. WasmEdge 0.16.4's AOT compiler must carry SDN's
48
- `04-atomic-memarg-offset` patch (§4.3).
49
- - Give every guest thread a distinct `tid` below 2^29 (wasi-libc keeps it in
50
- mutex words).
51
-
52
- ## 2. The Node host (design §5.6)
53
-
54
- `wasm/ps-node-host.mjs` (types `ps-node-host.d.mts`), with
55
- `ps-node-guest.mjs`, `ps-node-imports.mjs` and `ps-node-fault.mjs`. Node 24+,
56
- optional peer dependency `space-data-module-sdk` ≥ 0.8.24.
57
-
58
- - **Supervisor.** The main thread never runs guest code. The guest's own
59
- thread (`_start` for a command, the exec thread for a reactor instance) is a
60
- worker; so is every guest thread.
61
- - **wasi.thread-spawn** claims a pre-started worker from a pool the supervisor
62
- keeps (16 warm, grown on demand to a cap of 512). The claim is a CAS on the
63
- pool's SharedArrayBuffer and needs no event loop, so any guest thread can
64
- spawn, and workers are reused: a crash harness that starts and stops engines
65
- 12,000 times runs on 17 workers. Tids come from one shared counter.
66
- - **flatsql_io** is the SDK's Node sync-fs provider (`createNodeSyncFsIo`: a
67
- virtual-handle table in a SharedArrayBuffer, lazily opened fds per worker,
68
- `fdatasync`/`F_FULLFSYNC`, `CREATE_PARENTS`, `UNLINK_IF_UNUSED`, A23
69
- revocation) over the host root.
70
- - **WASI preview1** is the host's own: a descriptor table in a
71
- SharedArrayBuffer (a descriptor opened on one thread works on all, as in a
72
- process), preopens, process-wide clocks (`process.hrtime`; a worker's
73
- `performance.now()` has its own origin, which would break the engine's
74
- cross-thread time comparisons), seeded `random_get` in deterministic mode,
75
- `poll_oneoff` clock waits, `proc_exit` from any thread ending the guest.
76
- - **File system.** `root` is the guest's "/" for WASI and for flatsql_io;
77
- `mounts` preopen more host directories (the CLI mounts the checkout at its
78
- own path so the test commands find their vectors). Default: a private
79
- temporary root, removed afterwards.
80
- - **Faults.** A guest thread that traps writes the trap to stderr from its own
81
- worker and sets a shared fault word; the supervisor ends the run (the thread
82
- that would join the dead one may be blocked forever).
83
- - **API.** `runPsCommand(wasm, options)` (commands: exit code, kill, timeout,
84
- thread report); `createPsNodeInstance(wasm, options)` (one artifact instance:
85
- `call`, `write`, `read`, `threads`, `close`); CLI `node wasm/ps-node-host.mjs
86
- <command.wasm> [args]`.
87
-
88
- ### Fault overlay (§19, 22.3a-7)
89
-
90
- `wasm/ps-node-fault.mjs`: a SharedArrayBuffer page table every worker
91
- consults, interposed under the SDK provider's syscalls. It records the
92
- pre-image of every 4 KiB page written since the file's last sync (pages wholly
93
- past the synced size need none), the synced size, and files created since
94
- their parent directory's last sync. The supervisor freezes it (later mutating
95
- calls fail with EIO), stops the guest, and applies a crash: `dropAll` (every
96
- unsynced page and size change undone, unsynced directory entries gone),
97
- `pageSubset` (each unsynced page survives or not, in any order; the size is
98
- the synced or the current one), `keepAll` (kill -9). Overflowing its page
99
- budget fails the crash instead of making it partial.
100
-
101
- ## 3. Test commands
102
-
103
- | Command | Contents |
104
- |---|---|
105
- | `cpp/build-ps-wasm/flatsql-ps-test.wasm` | `flatsql_ps_test` (every test in `cpp/test/ps`) as a wasi-threads command; host I/O through the seven imports. `--list` prints the tests. |
106
- | `cpp/build-ps-wasm/flatsql-ps-test-memio.wasm` | The same command with the seven functions defined in the guest (`test/ps/wasm_io_stub.cpp`, all ACCESS): imports only WASI, thread-spawn and memory, so a WASI-only host (the SDK WasmEdge C runner) runs every in-memory test. |
107
-
108
- Added tests (`PS_SLOW_TEST`, run only when named): `parity_canonical_dump`,
109
- `parity_deterministic_bytes` (`cpp/test/ps/parity_test.cpp`) and
110
- `host_crash_ingest` / `host_crash_verify` (`host_crash_test.cpp`). The wasm
111
- test main grows the heap to the memory's maximum before any thread starts
112
- (§1; `PS_WASM_HEAP_MB` caps it).
113
-
114
- ```
115
- npm run build:wasm:ps-tests # artifact + test commands
116
- node wasm/ps-node-host.mjs cpp/build-ps-wasm/flatsql-ps-test.wasm --test=lane_
117
- npm run test:ps-wasm # scripts/ps-wasm-suite.mjs: every default test + 1,000 crashes
118
- FLATSQL_PS_WASM_SUITE=1 npm test -- test/ps-node-threads.test.ts
119
- npm test -- test/ps-host-fault.test.ts test/ps-parity-vectors.test.ts
120
- FLATSQL_PS_WASMEDGE=1 npm test -- test/ps-parity-vectors.test.ts # + the C runner lane
121
- ```
122
-
123
- ## 4. Acceptance (§18 T4 as amended)
124
-
125
- Machine: the owner's Mac Studio (Darwin 25.3.0, arm64, 28 hardware threads,
126
- APFS), Node 25.4.0, shared with other lanes: 1-minute load average 18–25
127
- during these runs (37–41 for the release rerun, §4.2). Native builds
128
- RelWithDebInfo; wasm builds as §1.
129
-
130
- | # | Acceptance | Result |
131
- |---|---|---|
132
- | 1 | Imports exactly WASI p1 + `wasi.thread-spawn` + shared `env.memory` (≤ 32768 pages) + the 7 `env.flatsql_io_*`; exports `wasi_thread_start`; 0 emscripten imports; passes `assertPthreadArtifact` | **Pass.** §1 table; `scripts/check-wasm-imports.mjs`, `test/standalone-wasm-imports.test.ts` (the SDK guard runs in a child process). |
133
- | 2 | The T1 and T2 suites, including 1,000 fault crashes, pass under the Node host with distinct OS thread ids ≥ writers + lanes | **Pass.** `scripts/ps-wasm-suite.mjs`: all 53 default-suite tests, each in its own guest process, plus `crash_faults_T1_1 --crash-trials=1000` (1,000 trials over the five modes, 298,923 acked records verified, 41.8 s). `readers_under_saturating_writers_T2_1` (8 writers, 16 lanes) ran 54 guest threads at once on 54 distinct worker (OS) threads. `test/ps-node-threads.test.ts`: the shipped artifact as a writer instance (4 writers) and a reader instance (4 lanes), each running thread on its own worker. `writer_backpressure_flood_T1_5` runs with `--bp-samples=40` (deviation 3). `crash_faults_T1_1_full` (10,000 trials, 64 types, 256 partitions): §4.1. |
134
- | 3 | Parity: canonical dumps (by (pid, pseq)) and query results identical native vs Node; deterministic mode byte-identical across native, Node and the SDK WasmEdge C runner (patched 0.16.4) | **Pass.** Canonical dump, 4 writers, 11,313 lines (rows, counters, lane counters and six queries, partition and type level): sha256 `f3b419a2…` natively and under Node, three runs each. Deterministic mode, 122 files: in memory `e7ab8592…` natively, under Node and under the C runner (18.7 s, interpreted); on the host's files `c7e34c55…` natively and under Node. `test/ps-parity-vectors.test.ts`. |
135
- | 4 | A published npm version with provenance; the wasm sha256 in `integrity.json`; `npm ls sql.js` empty; the package root exports only the wasm bindings | **Pass.** `flatsql@3.0.0`, §4.2 and §6. |
136
-
137
- Host-level crashes (the overlay, §2): `test/ps-host-fault.test.ts` kills the
138
- guest mid-ingest at a random 0.4–2.0 s, crashes the store in the three modes
139
- in turn and verifies every acked record (bytes by CID), gap-free pseqs,
140
- counters and 0 `d-*` bytes read at open; 9 cycles, half of them with the A8
141
- journal. A negative control (half a data segment cut off) fails the verifier.
142
-
143
- ### 4.1 Slow runs
144
-
145
- - `crash_faults_T1_1_full` under the Node host (10,000 trials, 64 types, 256
146
- partitions, the five crash modes, an A4 second crash every other trial, half
147
- the stores journaled): pass in 377 s; 2,122,844 acked records verified,
148
- 410,159 merges, 1,167,880 journal records replayed, 125,016 guest threads
149
- spawned on 10 pooled workers. Mac, load 24 rising to 35 during the run
150
- (natively 243 s).
151
- - The whole default suite under the host, one guest process per test:
152
- 155 s of test time; the most concurrent guest threads was 54
153
- (`readers_under_saturating_writers_T2_1`), each on its own worker thread;
154
- the largest guest memory 1,902 MiB (the 1,536 MiB pre-grown heap included).
155
- - Deterministic vector under the C runner: 18.7 s interpreted (0.12 s natively).
156
-
157
- ### 4.2 Release
158
-
159
- - `flatsql@3.0.0`, published by `npm-publish.yml` from tag `v3.0.0` =
160
- `79fd694` (run 36486543061) with a signed provenance statement
161
- (`npm audit signatures`: verified attestation); `gitHead` 79fd694, dist-tag
162
- `latest`, `dependencies` `{}`.
163
- - `wasm/flatsql-ps-threads.wasm`: 2,074,707 bytes, sha256
164
- `572a4bc38345d07ff91c345fc8d81e28873d72efa0fb8864a2edc5242145f9f5`, the same
165
- in `wasm/integrity.json`, from the Linux arm64 Docker build, the x86_64
166
- publish runner and the installed tarball.
167
- - A fresh `npm install flatsql@3.0.0`: `npm ls sql.js` is empty; the root's
168
- exports equal `flatsql/wasm`'s; `loadFlatSQLPsThreads()` checks the bytes
169
- against `integrity.json`.
170
- - At the release sources (`dac8d89` engine), under the Node host: the 55
171
- default-suite tests and `crash_faults_T1_1 --crash-trials=1000` pass (1,000
172
- trials, 403,798 acked records verified, 47.3 s; load 37–41). Main CI green
173
- on `79fd694` (run 36482613908).
174
- - Two engine defects the Node host found before the release, fixed in the
175
- sources the artifact builds: the open path dropped the lane checkpoint
176
- pointer of a head with more than 32 live lanes (`5c7e769`; the 1,000-trial
177
- crash run, trial 317), and the type owner never merged its L0 directory
178
- when `mergeL0Blocks` exceeded the directory cap, so labeling, partition
179
- merges and acks stopped (`dac8d89`).
180
-
181
- ### 4.3 The shared-memory grow race (V8) and AOT wait/notify (WasmEdge)
182
-
183
- **V8.** CI ps-wasm run 36603876540 (attempt 1) trapped "memory access out of
184
- bounds" in `writeTypeMergeOutputs` on the type-merge helper thread during
185
- `readers_under_saturating_writers_T2_1`, with memory at 1,651 MiB, past the
186
- test main's 1,536 MiB pre-grow. A Docker build with the Linux wasi-sdk 30 at
187
- the CI checkout path reproduces the CI test command exactly (its function
188
- indices match the trace, and its `flatsql-ps-threads.wasm` is the committed
189
- artifact's sha256). The trap offset `0x18d31a` is a `memory.fill`: the
190
- zero-fill of `blocks[i].resize(de.l0Len)` right after its `operator new`.
191
- Each thread's instance caches the memory size, and V8 bounds-checks
192
- `memory.fill`/`memory.copy` and atomics against that cache. Another thread's
193
- `memory.grow` updates the cache only at this thread's next stack check, so a
194
- block the allocator carves from memory another thread has just grown fails
195
- that check. Plain loads and stores are checked by guard pages and never trap.
196
- Measured on the Mac, Node 25.4.0, load 35–65:
197
-
198
- | Heap before threads start | T2_1 runs | Traps |
199
- |---|---|---|
200
- | none (`PS_WASM_HEAP_MB=0`) | 31 | 8, every one a `memory.fill`/`memory.copy` right after an allocation (vector push_back, string copy, the resize above) |
201
- | 2,000 MiB (past the test's peak) | 30 | 0 |
202
- | the memory's maximum (§1, now the default) | 500 | 0, and 0 runs grew memory after a thread started (1 run ended in a Node 25.4.0 SIGILL inside V8's JIT code, a host crash) |
203
-
204
- Natively the same test under ASan+UBSan (Docker linux/arm64, 4 CPUs, a CPU
205
- hog) found no memory error in 488 runs: the engine never overran.
206
-
207
- **WasmEdge, the SDN host.** Every thread's instance imports one shared
208
- `MemoryInstance`. On 64-bit hosts it is reserved in full up front (12 GiB,
209
- `lib/system/allocator.cpp`) and `memory.grow` maps pages in place, so the
210
- base never moves. The page count is one field that `memory.fill`/`copy`/
211
- `init` and atomic wait/notify check live, and plain loads and stores are
212
- checked by guard pages. The SDN C host I/O resolves every guest pointer per
213
- call (`WasmEdge_MemoryInstanceGetPointer`), and the Go accessor re-reads the
214
- page count on a miss. A grow is visible to every thread when the growing
215
- thread's `memory.grow` returns, and the guest allocator's lock orders it
216
- before any other thread can allocate from it. Measured: the same T2_1 with
217
- no pre-grow on the patched WasmEdge 0.16.4, AOT, had 0 traps in 26 completed
218
- runs.
219
-
220
- **WasmEdge AOT wait/notify.** Those WasmEdge runs also hung at teardown: 4 of
221
- 30 with no pre-grow, 5 of 30 with a 1,536 MiB pre-grow. WasmEdge 0.16.4's AOT
222
- compiler passed `memory.atomic.notify` and `memory.atomic.wait32/64` the bare
223
- address operand and dropped the instruction's memarg offset; the interpreter
224
- adds it. wasi-libc addresses its thread-list lock as `i32.const 0;
225
- memory.atomic.notify offset=<lock>`, so those wakeups went to address 0 and
226
- thread exit and join hung. A plain spawn/join guest with no flatsql code, 50
227
- rounds of 16 threads, hung 11 of 20 runs AOT, 0 of 30 interpreted and 0 of 20
228
- under V8. `sleepNs` (a timed wait on a stack local whose frame offset the
229
- compiler folds into the memarg) compared another stack word and returned at
230
- once: 100 × 10 ms took 0.000 s AOT and 1.159 s interpreted. SDN's static
231
- WasmEdge carries patch `04-atomic-memarg-offset` from SDN `74db37e14`, and
232
- its substrate self-test checks it (tag `sdn3`, so threaded AOT artifacts
233
- recompile). With the patch, the plain guest ran 40 of 40, and T2_1 AOT ran
234
- 140 times with no hang. Two of those runs failed. One ran out of the 4 GiB
235
- address space: the memio command keeps every file in guest memory, and
236
- passing runs end near 2.7 GiB. The other run's output was not kept.
237
- `memory_safety_sleep_and_notify_at_folded_offsets` passes. Without it, that test fails (20 of 20 folded-offset wakeups lost).
238
- `sleepNs` now passes its address through a volatile, so its wait has a zero
239
- memarg offset and sleeps even on an unpatched runtime. The artifact still has
240
- 4 folded wait/notify sites, all in wasi-libc's thread code.
241
-
242
- ## 5. Design deviations
243
-
244
- 1. **The Node host's thread spawn is its own pool, not the SDK's Node
245
- branch** (§5.6: "The SDK's Node branch (lazy worker_threads) runs the guest
246
- threads"). `createWasiThreadSpawn` (0.8.24) removes a finished worker only on
247
- its `exit` event, which runs on the spawning thread's event loop; that
248
- thread is inside the guest, blocked, for the whole command. After `poolSize`
249
- spawns every later spawn is declined, and without a pool the finished
250
- threads are never joined. The crash harness spawns more than 12,000 guest
251
- threads per run. The host keeps a pool whose idle/assign handshake is a
252
- SharedArrayBuffer CAS, so no event loop is involved and any thread can
253
- spawn. Reported to the module-sdk lane (SDK 0.8.25); the SDK's Node sync-fs
254
- flatsql_io provider is used as designed.
255
- 2. **WASI is the host's own**, not the SDK shim: the shim keeps descriptors
256
- and clocks per worker (a worker's `performance.now()` has its own origin),
257
- has no preopens and no `poll_oneoff`. The artifact itself imports only names
258
- the SDK shim provides (§1), so browser hosts are unaffected.
259
- 3. **The flood test runs 40 samples under wasm32** (`--bp-samples=40`, default
260
- 150): its in-memory host keeps every flooded byte twice for as long as the
261
- timed measurement lasts (12 GB resident natively with 150 samples), past
262
- wasm32's 4 GiB. Its ring, credit, pool and zero-loss checks are unchanged;
263
- the ack-p99 ratio is reported (1.066) and, as on any box with fewer than 8
264
- hardware threads, not enforced: WASI reports one CPU. The same holds for the
265
- T2 latency bounds (`latencyBoxQuiet` is false under wasm): they are native
266
- Linux-8 acceptance numbers. `lane_needs_bulk_under_1ms_zero_rows_T2_4`
267
- enforced its 1 ms p99 on every machine; it now follows the same rule
268
- (enforced on a quiet box, reported elsewhere) after a Debug build on the
269
- loaded 3-vCPU macOS CI runner and the Node host on the Linux runner
270
- measured past it. Its status, rows and examined counts are enforced
271
- everywhere. `type_arrivals_segments_and_fence_A15` caps type commits at 40
272
- rows: a slow type owner labeled more than 40 rows in one commit and filled
273
- an empty arrivals segment past its 40-entry seal (a batch never splits
274
- across segments), which failed on the macOS runner (main 8aea463) and
275
- under the Node host.
276
- 4. **The C runner lane runs the memio command.** The SDK's WasmEdge
277
- wasi-threads C runner provides WASI and thread-spawn only, so the byte
278
- vectors there use the in-memory host, which is also compared natively and
279
- under Node; the host-file vector is compared natively and under Node.
280
- Adding flatsql_io to the runner (as the SDN C host module does) is an SDK
281
- change.
282
- 5. **Parity inputs are built deterministically.** A fixture expression such
283
- as `a + b * c` is contracted to a fused multiply-add on arm64 and not in
284
- wasm, which changes the input bytes (1 ULP), not the engine's output; the
285
- parity workload builds its frames from exact doubles. `buildRecordAttr`
286
- (`src/ps/writer_pool.cpp`) creates its strings inside one call's argument
287
- list, whose evaluation order C++ leaves unspecified: GCC on x86_64 builds
288
- them in the opposite order from clang and from GCC on arm64, so the same
289
- attributes serialize to different bytes (measured on the CI runner and under
290
- x86_64 emulation). The parity workload builds its attributes in a fixed
291
- order; the helper itself belongs to the writer files T3 holds (reported). The canonical dump masks
292
- `kRowAttrInM` (where attributes are stored until a merge moves them, like
293
- offsets) and injects the engine's wall clock (TOMB arrival times).
294
- 6. **Deterministic mode injects both clocks in the guest** (`setTestClock`,
295
- `src/ps/platform.cpp`): the store UUID hashes the monotonic clock, and a
296
- frozen clock keeps every time-based trigger out of the byte stream. A host
297
- running the artifact deterministically injects `clock_time_get` and
298
- `random_get` instead (the Node host's `deterministic` option seeds
299
- `random_get`).
300
- 7. **A wasm32 defect fixed in the lane arena**: the 256 GiB request bound was
301
- `size_t(1) << 38`, undefined where `size_t` is 32 bits; every lane
302
- allocation failed and every lane died at `sqlite3_open`.
303
-
304
- ## 6. Package 3.0.0 (A1, A34)
305
-
306
- Semver-major (A34). The package root is the wasm bindings: `wasm/index.js`
307
- (the SQLite engine, as `flatsql/wasm`) plus the partition store locator
308
- (`getFlatSQLPsThreadsURL`, `loadFlatSQLPsThreads`,
309
- `FLATSQL_PS_THREADS_SHA256`). New subpaths: `flatsql/ps-threads.wasm`,
310
- `flatsql/ps/node-host`, `flatsql/ps/fault`, `flatsql/integrity.json`.
311
-
312
- Removed (A1 ledger row T4):
313
-
314
- | Removed | Replaced by |
315
- |---|---|
316
- | `src/core/` (the TS engine) and the root exports `FlatSQLDatabase`, `DirectAccessor`, `FlatcAccessor`, `TableStore` | the wasm engines |
317
- | the `node:sqlite` artifact builder (`FlatSQLArtifactBuilder`, its worker) and the `flatsql/artifacts/standalone` builder | — |
318
- | `wasm/standalone.js` and `flatsql/standalone` | `flatsql/ps/node-host` for the partition store. The legacy WASI engine's own tests keep a test-only copy (`test/support/legacy-standalone.js`, not shipped) until T8 retires that engine. |
319
- | `sql.js`, and the npm `flatbuffers` package (its only user was the deleted TS workload) | — |
320
-
321
- The BTree, stacked store, schema parser and cluster detection stay, as
322
- `flatsql/btree`, `flatsql/storage`, `flatsql/schema` and `flatsql/cluster`.
323
-
324
- **A1 parity check (the receipt).** A1 allows the deletion when each consumer's
325
- suite passes on the new bindings or the consumer pins the last pre-T4 major.
326
- Every consumer pins an earlier major, so none resolves 3.0.0:
327
-
328
- | Consumer (stack pin) | flatsql |
329
- |---|---|
330
- | space-data-module-sdk 0.8.24 (450aa78; `src/runtime-host/flatsqlRuntimeStore.js`) | `^2.0.0` |
331
- | OrbPro (216083d024; `packages/wasm-engine`, root) | `^1.4.4` |
332
- | OrbPro `packages/space-data-module-sdk` 0.8.23 | `^2.0.0` |
333
- | spacedatastandards.org 1.228.0 (cd86467de5; `test-flatsql*.js`) | `^1.3.0` |
334
- | spacedatastandards `packages/standards-explorer` | `^0.3.9` |
335
- | sdn-js 3.1.0 (space-data-network f20dbc35d) | `2.0.3` |
336
-
337
- Moving each consumer to 3.x (or recording that it stays on 2.x) is a ripple
338
- task per repository (A1, A34).