@amalgm/live 0.2.57 → 0.2.58
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/PURPOSE.md +10 -4
- package/dist/transfer/download.d.ts +7 -0
- package/dist/transfer/download.d.ts.map +1 -1
- package/dist/transfer/download.js +49 -11
- package/dist/transfer/download.js.map +1 -1
- package/dist-cjs/transfer/download.js +49 -11
- package/dist-cjs/transfer/download.js.map +1 -1
- package/package.json +1 -1
package/PURPOSE.md
CHANGED
|
@@ -111,8 +111,11 @@ its cutover.
|
|
|
111
111
|
binding to Watch. The intent clears only after coverage is healthy. Entity
|
|
112
112
|
rows therefore never mean pending download, and visible Add ground is never
|
|
113
113
|
left without its durable identity. Hidden staging that crosses a restart is
|
|
114
|
-
|
|
115
|
-
|
|
114
|
+
untrusted resume substrate: ordinary retained chunks are reread and
|
|
115
|
+
re-hashed against the pinned manifest, missing or corrupt chunks are
|
|
116
|
+
fetched again, and the ordered whole artifact is proven before rows or
|
|
117
|
+
reveal are trusted. Revealed ground that crosses that blind interval
|
|
118
|
+
remains completely suspect until Detect settles it. The
|
|
116
119
|
local SQLite table contains workspace rows only for boundaries materialized
|
|
117
120
|
on that machine, while cloud reconciliation preserves every unmaterialized
|
|
118
121
|
boundary. A fresh verified install may be Watch's initial baseline so the
|
|
@@ -191,8 +194,11 @@ its cutover.
|
|
|
191
194
|
ground itself — and verifies every byte it sends against the captured
|
|
192
195
|
content head. A typed source mismatch returns control to Register so it
|
|
193
196
|
can recapture current ground; stale bytes can never be sealed or
|
|
194
|
-
published.
|
|
195
|
-
|
|
197
|
+
published. During Add, ordinary text and binary bytes land directly in
|
|
198
|
+
their hidden staged destinations in manifest order while set-wide fetches
|
|
199
|
+
remain bounded and concurrent, so their whole-artifact proof advances from
|
|
200
|
+
arriving bytes without rereading staging; links and repository transports
|
|
201
|
+
keep their dedicated adapter/cache rails. No file type or size permits a
|
|
196
202
|
complete-file allocation in the bounded transfer path.
|
|
197
203
|
31. **A content block is not a round trip.** Text and binary retain independent
|
|
198
204
|
content-addressed block identities. Cold upload may move complete small
|
|
@@ -10,6 +10,13 @@
|
|
|
10
10
|
* still its own storage request — the set changes scheduling, not the
|
|
11
11
|
* request count.
|
|
12
12
|
*
|
|
13
|
+
* Fetches complete in whatever order the network answers, but each artifact
|
|
14
|
+
* stores its chunks in chunk order: a verified chunk waits until every
|
|
15
|
+
* earlier gap of the same artifact has been stored. Gaps are admitted round
|
|
16
|
+
* robin across artifacts, so one large artifact cannot occupy the whole
|
|
17
|
+
* window while unrelated work waits. Held bytes stay inside the in-flight
|
|
18
|
+
* byte bound.
|
|
19
|
+
*
|
|
13
20
|
* Every chunk keeps one content-addressed identity, whether its sealed
|
|
14
21
|
* envelope lives alone or as a byte range inside an immutable pack. The
|
|
15
22
|
* host's ports resolve and fetch those identities; parallelism, storage
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"download.d.ts","sourceRoot":"","sources":["../../src/transfer/download.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"download.d.ts","sourceRoot":"","sources":["../../src/transfer/download.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AAEH,OAAO,EAAqC,KAAK,eAAe,EAAE,MAAM,yBAAyB,CAAC;AAClG,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,sBAAsB,CAAC;AACtD,OAAO,EAA0B,KAAK,gBAAgB,EAAE,MAAM,gBAAgB,CAAC;AAO/E,MAAM,WAAW,aAAa,CAAC,aAAa;IAC1C,QAAQ,CAAC,SAAS,EAAE,SAAS,CAAC;IAC9B,WAAW,CAAC,QAAQ,EAAE,gBAAgB,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;IAC1D,6EAA6E;IAC7E,QAAQ,CACN,QAAQ,EAAE,gBAAgB,EAC1B,QAAQ,EAAE,eAAe,EACzB,KAAK,EAAE,MAAM,GACZ,OAAO,CAAC,OAAO,CAAC,CAAC;IACpB,QAAQ,CACN,QAAQ,EAAE,gBAAgB,EAC1B,QAAQ,EAAE,eAAe,EACzB,KAAK,EAAE,MAAM,GACZ,OAAO,CAAC,UAAU,CAAC,CAAC;IACvB,QAAQ,CACN,QAAQ,EAAE,gBAAgB,EAC1B,QAAQ,EAAE,eAAe,EACzB,KAAK,EAAE,MAAM,EACb,KAAK,EAAE,UAAU,GAChB,OAAO,CAAC,IAAI,CAAC,CAAC;IACjB,8EAA8E;IAC9E,YAAY,CACV,QAAQ,EAAE,gBAAgB,EAC1B,QAAQ,EAAE,eAAe,GACxB,OAAO,CAAC,cAAc,CAAC,aAAa,CAAC,CAAC,CAAC;CAC3C;AAED,MAAM,WAAW,cAAc,CAAC,aAAa;IAC3C,QAAQ,CAAC,KAAK,EAAE,aAAa,CAAC;IAC9B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;CAC9B;AAED,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,WAAW,CAAC,EAAE,MAAM,CAAC;IAC9B;sEACkE;IAClE,QAAQ,CAAC,gBAAgB,CAAC,EAAE,MAAM,CAAC;CACpC;AAED,MAAM,WAAW,eAAe,CAAC,aAAa;IAC5C,QAAQ,CAAC,QAAQ,EAAE,gBAAgB,CAAC;IACpC,QAAQ,CAAC,QAAQ,EAAE,eAAe,CAAC;IACnC,QAAQ,CAAC,KAAK,EAAE,aAAa,CAAC;IAC9B,QAAQ,CAAC,gBAAgB,EAAE,MAAM,CAAC;IAClC,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;CAC/B;AAmBD;;uCAEuC;AACvC,wBAAsB,WAAW,CAAC,aAAa,EAC7C,SAAS,EAAE,SAAS,gBAAgB,EAAE,EACtC,KAAK,EAAE,aAAa,CAAC,aAAa,CAAC,EACnC,OAAO,GAAE,eAAoB,GAC5B,OAAO,CAAC,SAAS,eAAe,CAAC,aAAa,CAAC,EAAE,CAAC,CA0FpD;AAED,sDAAsD;AACtD,wBAAsB,QAAQ,CAAC,aAAa,EAC1C,QAAQ,EAAE,gBAAgB,EAC1B,KAAK,EAAE,aAAa,CAAC,aAAa,CAAC,EACnC,OAAO,GAAE,eAAoB,GAC5B,OAAO,CAAC,eAAe,CAAC,aAAa,CAAC,CAAC,CAIzC"}
|
|
@@ -10,6 +10,13 @@
|
|
|
10
10
|
* still its own storage request — the set changes scheduling, not the
|
|
11
11
|
* request count.
|
|
12
12
|
*
|
|
13
|
+
* Fetches complete in whatever order the network answers, but each artifact
|
|
14
|
+
* stores its chunks in chunk order: a verified chunk waits until every
|
|
15
|
+
* earlier gap of the same artifact has been stored. Gaps are admitted round
|
|
16
|
+
* robin across artifacts, so one large artifact cannot occupy the whole
|
|
17
|
+
* window while unrelated work waits. Held bytes stay inside the in-flight
|
|
18
|
+
* byte bound.
|
|
19
|
+
*
|
|
13
20
|
* Every chunk keeps one content-addressed identity, whether its sealed
|
|
14
21
|
* envelope lives alone or as a byte range inside an immutable pack. The
|
|
15
22
|
* host's ports resolve and fetch those identities; parallelism, storage
|
|
@@ -44,18 +51,49 @@ export async function downloadAll(artifacts, ports, options = {}) {
|
|
|
44
51
|
}
|
|
45
52
|
plans[slot] = { artifact, manifest, missing };
|
|
46
53
|
});
|
|
47
|
-
const gaps =
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
54
|
+
const gaps = [];
|
|
55
|
+
const priorByPlan = new Map();
|
|
56
|
+
const maxGaps = plans.reduce((maximum, plan) => Math.max(maximum, plan?.missing.length ?? 0), 0);
|
|
57
|
+
for (let depth = 0; depth < maxGaps; depth += 1) {
|
|
58
|
+
for (const plan of plans) {
|
|
59
|
+
if (plan === undefined)
|
|
60
|
+
continue;
|
|
61
|
+
const index = plan.missing[depth];
|
|
62
|
+
if (index === undefined)
|
|
63
|
+
continue;
|
|
64
|
+
let stored;
|
|
65
|
+
let failed;
|
|
66
|
+
const delivered = new Promise((resolve, reject) => {
|
|
67
|
+
stored = resolve;
|
|
68
|
+
failed = reject;
|
|
69
|
+
});
|
|
70
|
+
// A failed effect may leave no later chunk admitted to observe the
|
|
71
|
+
// rejection; the set still fails through the effect's own throw.
|
|
72
|
+
delivered.catch(() => { });
|
|
73
|
+
gaps.push({ plan, index, priorStored: priorByPlan.get(plan), stored, failed });
|
|
74
|
+
priorByPlan.set(plan, delivered);
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
await weightedParallelForEach(gaps, concurrency, maxInFlightBytes, ({ plan, index }) => plan.manifest.chunks[index]?.bytes ?? 0, async ({ plan, index, priorStored, stored, failed }) => {
|
|
78
|
+
try {
|
|
79
|
+
const expected = plan.manifest.chunks[index];
|
|
80
|
+
if (expected === undefined)
|
|
81
|
+
throw new Error('download manifest named an unknown chunk index');
|
|
82
|
+
const bytes = await ports.getChunk(plan.artifact, plan.manifest, index);
|
|
83
|
+
if (!(bytes instanceof Uint8Array)
|
|
84
|
+
|| bytes.byteLength !== expected.bytes
|
|
85
|
+
|| ports.sha256Hex(bytes) !== expected.sha256) {
|
|
86
|
+
throw new Error(`downloaded content chunk ${index} failed verification`);
|
|
87
|
+
}
|
|
88
|
+
if (priorStored !== undefined)
|
|
89
|
+
await priorStored;
|
|
90
|
+
await ports.putChunk(plan.artifact, plan.manifest, index, bytes);
|
|
91
|
+
stored();
|
|
92
|
+
}
|
|
93
|
+
catch (error) {
|
|
94
|
+
failed(error);
|
|
95
|
+
throw error;
|
|
57
96
|
}
|
|
58
|
-
await ports.putChunk(plan.artifact, plan.manifest, index, bytes);
|
|
59
97
|
});
|
|
60
98
|
const receipts = new Array(artifacts.length);
|
|
61
99
|
await parallelForEach([...plans.keys()], concurrency, async (slot) => {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"download.js","sourceRoot":"","sources":["../../src/transfer/download.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"download.js","sourceRoot":"","sources":["../../src/transfer/download.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AAEH,OAAO,EAAE,WAAW,EAAE,oBAAoB,EAAwB,MAAM,yBAAyB,CAAC;AAElG,OAAO,EAAE,sBAAsB,EAAyB,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EACL,4BAA4B,EAC5B,eAAe,EACf,uBAAuB,GACxB,MAAM,eAAe,CAAC;AAmEvB;;uCAEuC;AACvC,MAAM,CAAC,KAAK,UAAU,WAAW,CAC/B,SAAsC,EACtC,KAAmC,EACnC,UAA2B,EAAE;IAE7B,MAAM,WAAW,GAAG,OAAO,CAAC,WAAW,IAAI,4BAA4B,CAAC;IACxE,MAAM,gBAAgB,GAAG,OAAO,CAAC,gBAAgB;WAC5C,4BAA4B,GAAG,WAAW,CAAC;IAChD,KAAK,MAAM,QAAQ,IAAI,SAAS;QAAE,sBAAsB,CAAC,QAAQ,CAAC,CAAC;IAEnE,MAAM,KAAK,GAAiC,IAAI,KAAK,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;IACxE,MAAM,eAAe,CAAC,CAAC,GAAG,SAAS,CAAC,IAAI,EAAE,CAAC,EAAE,WAAW,EAAE,KAAK,EAAE,IAAI,EAAE,EAAE;QACvE,MAAM,QAAQ,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;QACjC,IAAI,QAAQ,KAAK,SAAS;YAAE,OAAO;QACnC,MAAM,OAAO,GAAG,oBAAoB,CAClC,MAAM,KAAK,CAAC,WAAW,CAAC,QAAQ,CAAC,EACjC,QAAQ,CAAC,WAAW,CACrB,CAAC;QACF,IAAI,CAAC,OAAO,CAAC,EAAE;YAAE,MAAM,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;QAChD,MAAM,QAAQ,GAAG,OAAO,CAAC,KAAK,CAAC;QAC/B,MAAM,OAAO,GAAa,EAAE,CAAC;QAC7B,KAAK,MAAM,KAAK,IAAI,QAAQ,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;YAC3C,IAAI,CAAC,MAAM,KAAK,CAAC,QAAQ,CAAC,QAAQ,EAAE,QAAQ,EAAE,KAAK,CAAC;gBAAE,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;QAC5E,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,GAAG,EAAE,QAAQ,EAAE,QAAQ,EAAE,OAAO,EAAE,CAAC;IAChD,CAAC,CAAC,CAAC;IAEH,MAAM,IAAI,GAAkB,EAAE,CAAC;IAC/B,MAAM,WAAW,GAAG,IAAI,GAAG,EAA+B,CAAC;IAC3D,MAAM,OAAO,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,CAC7C,IAAI,CAAC,GAAG,CAAC,OAAO,EAAE,IAAI,EAAE,OAAO,CAAC,MAAM,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IACnD,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,OAAO,EAAE,KAAK,IAAI,CAAC,EAAE,CAAC;QAChD,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;YACzB,IAAI,IAAI,KAAK,SAAS;gBAAE,SAAS;YACjC,MAAM,KAAK,GAAG,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;YAClC,IAAI,KAAK,KAAK,SAAS;gBAAE,SAAS;YAClC,IAAI,MAAmB,CAAC;YACxB,IAAI,MAAiC,CAAC;YACtC,MAAM,SAAS,GAAG,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;gBACtD,MAAM,GAAG,OAAO,CAAC;gBACjB,MAAM,GAAG,MAAM,CAAC;YAClB,CAAC,CAAC,CAAC;YACH,mEAAmE;YACnE,iEAAiE;YACjE,SAAS,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;YAC1B,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,WAAW,EAAE,WAAW,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC;YAC/E,WAAW,CAAC,GAAG,CAAC,IAAI,EAAE,SAAS,CAAC,CAAC;QACnC,CAAC;IACH,CAAC;IACD,MAAM,uBAAuB,CAC3B,IAAI,EACJ,WAAW,EACX,gBAAgB,EAChB,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,EAAE,EAAE,CAAC,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,KAAK,IAAI,CAAC,EAC5D,KAAK,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,WAAW,EAAE,MAAM,EAAE,MAAM,EAAE,EAAE,EAAE;QACrD,IAAI,CAAC;YACH,MAAM,QAAQ,GAAG,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;YAC7C,IAAI,QAAQ,KAAK,SAAS;gBAAE,MAAM,IAAI,KAAK,CAAC,gDAAgD,CAAC,CAAC;YAC9F,MAAM,KAAK,GAAG,MAAM,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC;YACxE,IAAI,CAAC,CAAC,KAAK,YAAY,UAAU,CAAC;mBAC7B,KAAK,CAAC,UAAU,KAAK,QAAQ,CAAC,KAAK;mBACnC,KAAK,CAAC,SAAS,CAAC,KAAK,CAAC,KAAK,QAAQ,CAAC,MAAM,EAAE,CAAC;gBAChD,MAAM,IAAI,KAAK,CAAC,4BAA4B,KAAK,sBAAsB,CAAC,CAAC;YAC3E,CAAC;YACD,IAAI,WAAW,KAAK,SAAS;gBAAE,MAAM,WAAW,CAAC;YACjD,MAAM,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,QAAQ,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC;YACjE,MAAM,EAAE,CAAC;QACX,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,CAAC,KAAK,CAAC,CAAC;YACd,MAAM,KAAK,CAAC;QACd,CAAC;IACH,CAAC,CACF,CAAC;IAEF,MAAM,QAAQ,GAAmD,IAAI,KAAK,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;IAC7F,MAAM,eAAe,CAAC,CAAC,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC,EAAE,WAAW,EAAE,KAAK,EAAE,IAAI,EAAE,EAAE;QACnE,MAAM,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC,CAAC;QACzB,IAAI,IAAI,KAAK,SAAS;YAAE,OAAO;QAC/B,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,YAAY,CAAC,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC;QACtE,IAAI,MAAM,CAAC,KAAK,KAAK,IAAI,CAAC,QAAQ,CAAC,KAAK,IAAI,MAAM,CAAC,WAAW,KAAK,IAAI,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC;YAC7F,MAAM,IAAI,KAAK,CAAC,uBAAuB,IAAI,CAAC,QAAQ,CAAC,WAAW,sBAAsB,CAAC,CAAC;QAC1F,CAAC;QACD,QAAQ,CAAC,IAAI,CAAC,GAAG;YACf,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,KAAK,EAAE,MAAM,CAAC,KAAK;YACnB,gBAAgB,EAAE,IAAI,CAAC,OAAO,CAAC,MAAM;YACrC,YAAY,EAAE,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM;SAChE,CAAC;IACJ,CAAC,CAAC,CAAC;IACH,OAAO,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE;QAC9B,IAAI,OAAO,KAAK,SAAS;YAAE,MAAM,IAAI,KAAK,CAAC,uCAAuC,CAAC,CAAC;QACpF,OAAO,OAAO,CAAC;IACjB,CAAC,CAAC,CAAC;AACL,CAAC;AAED,sDAAsD;AACtD,MAAM,CAAC,KAAK,UAAU,QAAQ,CAC5B,QAA0B,EAC1B,KAAmC,EACnC,UAA2B,EAAE;IAE7B,MAAM,CAAC,OAAO,CAAC,GAAG,MAAM,WAAW,CAAC,CAAC,QAAQ,CAAC,EAAE,KAAK,EAAE,OAAO,CAAC,CAAC;IAChE,IAAI,OAAO,KAAK,SAAS;QAAE,MAAM,IAAI,KAAK,CAAC,uCAAuC,CAAC,CAAC;IACpF,OAAO,OAAO,CAAC;AACjB,CAAC"}
|
|
@@ -11,6 +11,13 @@
|
|
|
11
11
|
* still its own storage request — the set changes scheduling, not the
|
|
12
12
|
* request count.
|
|
13
13
|
*
|
|
14
|
+
* Fetches complete in whatever order the network answers, but each artifact
|
|
15
|
+
* stores its chunks in chunk order: a verified chunk waits until every
|
|
16
|
+
* earlier gap of the same artifact has been stored. Gaps are admitted round
|
|
17
|
+
* robin across artifacts, so one large artifact cannot occupy the whole
|
|
18
|
+
* window while unrelated work waits. Held bytes stay inside the in-flight
|
|
19
|
+
* byte bound.
|
|
20
|
+
*
|
|
14
21
|
* Every chunk keeps one content-addressed identity, whether its sealed
|
|
15
22
|
* envelope lives alone or as a byte range inside an immutable pack. The
|
|
16
23
|
* host's ports resolve and fetch those identities; parallelism, storage
|
|
@@ -48,18 +55,49 @@ async function downloadAll(artifacts, ports, options = {}) {
|
|
|
48
55
|
}
|
|
49
56
|
plans[slot] = { artifact, manifest, missing };
|
|
50
57
|
});
|
|
51
|
-
const gaps =
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
58
|
+
const gaps = [];
|
|
59
|
+
const priorByPlan = new Map();
|
|
60
|
+
const maxGaps = plans.reduce((maximum, plan) => Math.max(maximum, plan?.missing.length ?? 0), 0);
|
|
61
|
+
for (let depth = 0; depth < maxGaps; depth += 1) {
|
|
62
|
+
for (const plan of plans) {
|
|
63
|
+
if (plan === undefined)
|
|
64
|
+
continue;
|
|
65
|
+
const index = plan.missing[depth];
|
|
66
|
+
if (index === undefined)
|
|
67
|
+
continue;
|
|
68
|
+
let stored;
|
|
69
|
+
let failed;
|
|
70
|
+
const delivered = new Promise((resolve, reject) => {
|
|
71
|
+
stored = resolve;
|
|
72
|
+
failed = reject;
|
|
73
|
+
});
|
|
74
|
+
// A failed effect may leave no later chunk admitted to observe the
|
|
75
|
+
// rejection; the set still fails through the effect's own throw.
|
|
76
|
+
delivered.catch(() => { });
|
|
77
|
+
gaps.push({ plan, index, priorStored: priorByPlan.get(plan), stored, failed });
|
|
78
|
+
priorByPlan.set(plan, delivered);
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
await (0, parallel_js_1.weightedParallelForEach)(gaps, concurrency, maxInFlightBytes, ({ plan, index }) => plan.manifest.chunks[index]?.bytes ?? 0, async ({ plan, index, priorStored, stored, failed }) => {
|
|
82
|
+
try {
|
|
83
|
+
const expected = plan.manifest.chunks[index];
|
|
84
|
+
if (expected === undefined)
|
|
85
|
+
throw new Error('download manifest named an unknown chunk index');
|
|
86
|
+
const bytes = await ports.getChunk(plan.artifact, plan.manifest, index);
|
|
87
|
+
if (!(bytes instanceof Uint8Array)
|
|
88
|
+
|| bytes.byteLength !== expected.bytes
|
|
89
|
+
|| ports.sha256Hex(bytes) !== expected.sha256) {
|
|
90
|
+
throw new Error(`downloaded content chunk ${index} failed verification`);
|
|
91
|
+
}
|
|
92
|
+
if (priorStored !== undefined)
|
|
93
|
+
await priorStored;
|
|
94
|
+
await ports.putChunk(plan.artifact, plan.manifest, index, bytes);
|
|
95
|
+
stored();
|
|
96
|
+
}
|
|
97
|
+
catch (error) {
|
|
98
|
+
failed(error);
|
|
99
|
+
throw error;
|
|
61
100
|
}
|
|
62
|
-
await ports.putChunk(plan.artifact, plan.manifest, index, bytes);
|
|
63
101
|
});
|
|
64
102
|
const receipts = new Array(artifacts.length);
|
|
65
103
|
await (0, parallel_js_1.parallelForEach)([...plans.keys()], concurrency, async (slot) => {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"download.js","sourceRoot":"","sources":["../../src/transfer/download.ts"],"names":[],"mappings":";AAAA
|
|
1
|
+
{"version":3,"file":"download.js","sourceRoot":"","sources":["../../src/transfer/download.ts"],"names":[],"mappings":";AAAA;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;;AA+EH,kCA8FC;AAGD,4BAQC;AAtLD,wDAAkG;AAElG,iDAA+E;AAC/E,+CAIuB;AAmEvB;;uCAEuC;AAChC,KAAK,UAAU,WAAW,CAC/B,SAAsC,EACtC,KAAmC,EACnC,UAA2B,EAAE;IAE7B,MAAM,WAAW,GAAG,OAAO,CAAC,WAAW,IAAI,0CAA4B,CAAC;IACxE,MAAM,gBAAgB,GAAG,OAAO,CAAC,gBAAgB;WAC5C,0CAA4B,GAAG,wBAAW,CAAC;IAChD,KAAK,MAAM,QAAQ,IAAI,SAAS;QAAE,IAAA,qCAAsB,EAAC,QAAQ,CAAC,CAAC;IAEnE,MAAM,KAAK,GAAiC,IAAI,KAAK,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;IACxE,MAAM,IAAA,6BAAe,EAAC,CAAC,GAAG,SAAS,CAAC,IAAI,EAAE,CAAC,EAAE,WAAW,EAAE,KAAK,EAAE,IAAI,EAAE,EAAE;QACvE,MAAM,QAAQ,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;QACjC,IAAI,QAAQ,KAAK,SAAS;YAAE,OAAO;QACnC,MAAM,OAAO,GAAG,IAAA,iCAAoB,EAClC,MAAM,KAAK,CAAC,WAAW,CAAC,QAAQ,CAAC,EACjC,QAAQ,CAAC,WAAW,CACrB,CAAC;QACF,IAAI,CAAC,OAAO,CAAC,EAAE;YAAE,MAAM,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;QAChD,MAAM,QAAQ,GAAG,OAAO,CAAC,KAAK,CAAC;QAC/B,MAAM,OAAO,GAAa,EAAE,CAAC;QAC7B,KAAK,MAAM,KAAK,IAAI,QAAQ,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;YAC3C,IAAI,CAAC,MAAM,KAAK,CAAC,QAAQ,CAAC,QAAQ,EAAE,QAAQ,EAAE,KAAK,CAAC;gBAAE,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;QAC5E,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,GAAG,EAAE,QAAQ,EAAE,QAAQ,EAAE,OAAO,EAAE,CAAC;IAChD,CAAC,CAAC,CAAC;IAEH,MAAM,IAAI,GAAkB,EAAE,CAAC;IAC/B,MAAM,WAAW,GAAG,IAAI,GAAG,EAA+B,CAAC;IAC3D,MAAM,OAAO,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,CAC7C,IAAI,CAAC,GAAG,CAAC,OAAO,EAAE,IAAI,EAAE,OAAO,CAAC,MAAM,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IACnD,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,OAAO,EAAE,KAAK,IAAI,CAAC,EAAE,CAAC;QAChD,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;YACzB,IAAI,IAAI,KAAK,SAAS;gBAAE,SAAS;YACjC,MAAM,KAAK,GAAG,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;YAClC,IAAI,KAAK,KAAK,SAAS;gBAAE,SAAS;YAClC,IAAI,MAAmB,CAAC;YACxB,IAAI,MAAiC,CAAC;YACtC,MAAM,SAAS,GAAG,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;gBACtD,MAAM,GAAG,OAAO,CAAC;gBACjB,MAAM,GAAG,MAAM,CAAC;YAClB,CAAC,CAAC,CAAC;YACH,mEAAmE;YACnE,iEAAiE;YACjE,SAAS,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;YAC1B,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,WAAW,EAAE,WAAW,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC;YAC/E,WAAW,CAAC,GAAG,CAAC,IAAI,EAAE,SAAS,CAAC,CAAC;QACnC,CAAC;IACH,CAAC;IACD,MAAM,IAAA,qCAAuB,EAC3B,IAAI,EACJ,WAAW,EACX,gBAAgB,EAChB,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,EAAE,EAAE,CAAC,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,KAAK,IAAI,CAAC,EAC5D,KAAK,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,WAAW,EAAE,MAAM,EAAE,MAAM,EAAE,EAAE,EAAE;QACrD,IAAI,CAAC;YACH,MAAM,QAAQ,GAAG,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;YAC7C,IAAI,QAAQ,KAAK,SAAS;gBAAE,MAAM,IAAI,KAAK,CAAC,gDAAgD,CAAC,CAAC;YAC9F,MAAM,KAAK,GAAG,MAAM,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC;YACxE,IAAI,CAAC,CAAC,KAAK,YAAY,UAAU,CAAC;mBAC7B,KAAK,CAAC,UAAU,KAAK,QAAQ,CAAC,KAAK;mBACnC,KAAK,CAAC,SAAS,CAAC,KAAK,CAAC,KAAK,QAAQ,CAAC,MAAM,EAAE,CAAC;gBAChD,MAAM,IAAI,KAAK,CAAC,4BAA4B,KAAK,sBAAsB,CAAC,CAAC;YAC3E,CAAC;YACD,IAAI,WAAW,KAAK,SAAS;gBAAE,MAAM,WAAW,CAAC;YACjD,MAAM,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,QAAQ,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC;YACjE,MAAM,EAAE,CAAC;QACX,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,CAAC,KAAK,CAAC,CAAC;YACd,MAAM,KAAK,CAAC;QACd,CAAC;IACH,CAAC,CACF,CAAC;IAEF,MAAM,QAAQ,GAAmD,IAAI,KAAK,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;IAC7F,MAAM,IAAA,6BAAe,EAAC,CAAC,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC,EAAE,WAAW,EAAE,KAAK,EAAE,IAAI,EAAE,EAAE;QACnE,MAAM,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC,CAAC;QACzB,IAAI,IAAI,KAAK,SAAS;YAAE,OAAO;QAC/B,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,YAAY,CAAC,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC;QACtE,IAAI,MAAM,CAAC,KAAK,KAAK,IAAI,CAAC,QAAQ,CAAC,KAAK,IAAI,MAAM,CAAC,WAAW,KAAK,IAAI,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC;YAC7F,MAAM,IAAI,KAAK,CAAC,uBAAuB,IAAI,CAAC,QAAQ,CAAC,WAAW,sBAAsB,CAAC,CAAC;QAC1F,CAAC;QACD,QAAQ,CAAC,IAAI,CAAC,GAAG;YACf,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,KAAK,EAAE,MAAM,CAAC,KAAK;YACnB,gBAAgB,EAAE,IAAI,CAAC,OAAO,CAAC,MAAM;YACrC,YAAY,EAAE,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM;SAChE,CAAC;IACJ,CAAC,CAAC,CAAC;IACH,OAAO,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE;QAC9B,IAAI,OAAO,KAAK,SAAS;YAAE,MAAM,IAAI,KAAK,CAAC,uCAAuC,CAAC,CAAC;QACpF,OAAO,OAAO,CAAC;IACjB,CAAC,CAAC,CAAC;AACL,CAAC;AAED,sDAAsD;AAC/C,KAAK,UAAU,QAAQ,CAC5B,QAA0B,EAC1B,KAAmC,EACnC,UAA2B,EAAE;IAE7B,MAAM,CAAC,OAAO,CAAC,GAAG,MAAM,WAAW,CAAC,CAAC,QAAQ,CAAC,EAAE,KAAK,EAAE,OAAO,CAAC,CAAC;IAChE,IAAI,OAAO,KAAK,SAAS;QAAE,MAAM,IAAI,KAAK,CAAC,uCAAuC,CAAC,CAAC;IACpF,OAAO,OAAO,CAAC;AACjB,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@amalgm/live",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.58",
|
|
4
4
|
"description": "Portable realtime SDK for Amalgm — wire contracts, mutation discipline, and delivery state machines behind host-injected adapters.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist-cjs/index.js",
|