@xyo-network/xl1-rest-block-publisher 2.1.2 → 2.1.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -5,26 +5,31 @@
5
5
  [![npm-badge][]][npm-link]
6
6
  [![license-badge][]][license-link]
7
7
 
8
- > Publishes a finalized XL1 chain's immutable files as the static REST file layout that `RestBlockViewer` reads — pre-compressed, immutable, CDN-friendly.
8
+ > Publishes an XL1 chain as the static REST file layout the Rest viewers read — pre-compressed, CDN-friendly, one runner per concern.
9
9
 
10
10
  ## About
11
11
 
12
- Finalized blocks and completed steps never change, so the chain can be materialized once as static files on S3-compatible object storage (e.g. Cloudflare R2) and served forever over plain HTTP. `RestBlockPublisher` writes the finalized (immutable) part of that layout from any source `BlockViewer`:
12
+ Finalized blocks and completed steps never change, so the chain can be materialized once as static files on S3-compatible object storage (e.g. Cloudflare R2) and served forever over plain HTTP. Three locator-configurable runners (implementing the `BlockPublishRunner`, `IndexPublishRunner`, and `ChainStatePublishRunner` interfaces from `@xyo-network/xl1-protocol-lib`) write that layout from any source `BlockViewer`:
13
+
14
+ - **`S3BlockPublishRunner`** — finalized blocks and their payloads
15
+ - **`S3IndexPublishRunner`** — completed steps' block summaries (the chain index)
16
+ - **`S3ChainStatePublishRunner`** — the mutable chain state (the head pointer), targeting the chain-state bucket
13
17
 
14
18
  ```text
15
- /block/number/<n>.json ← immutable
16
- /block/hash/<hash>.json ← immutable, same content
17
- /blocks/step/<level>/<index>.json ← immutable, completed steps only
18
- /payload/<hash>.json ← immutable
19
+ /block/number/<n>.json ← immutable (block runner)
20
+ /block/hash/<hash>.json ← immutable, same content (block runner)
21
+ /blocks/step/<level>/<index>.json ← immutable, completed steps only (index runner)
22
+ /payload/<hash>.json ← immutable (block runner)
23
+ /chain/head.json ← mutable, chain-state bucket (chain state runner)
19
24
  ```
20
25
 
21
- The mutable chain state (the head pointer, `/chain/head.json`) is **not** written by this publisher it lives in a separate chain-state bucket and is written by the chain's finalizer (the `FinalizerActor` in xyo-chain) after the finalized files for that head exist, so readers never see a head that references missing files.
22
-
23
- Bodies are pre-compressed at write time (brotli by default, gzip or none configurable) and stored with the matching `Content-Encoding` plus `application/json`, so HTTP clients decompress transparently — keys keep their `.json` extension. Everything written gets `Cache-Control: public, max-age=31536000, immutable`.
26
+ Bodies are pre-compressed at write time (brotli by default, gzip or none configurable) and stored with the matching `Content-Encoding` plus `application/json`, so HTTP clients decompress transparently keys keep their `.json` extension. Finalized files get `Cache-Control: public, max-age=31536000, immutable`; the head pointer gets `must-revalidate`.
24
27
 
25
28
  ```ts
26
29
  import { S3Client } from '@aws-sdk/client-s3'
27
- import { RestBlockPublisher } from '@xyo-network/xl1-rest-block-publisher'
30
+ import {
31
+ S3BlockPublishRunner, S3ChainStatePublishRunner, S3IndexPublishRunner,
32
+ } from '@xyo-network/xl1-rest-block-publisher'
28
33
 
29
34
  const client = new S3Client({
30
35
  region: 'auto',
@@ -32,11 +37,18 @@ const client = new S3Client({
32
37
  credentials: { accessKeyId, secretAccessKey },
33
38
  })
34
39
 
35
- const publisher = await RestBlockPublisher.create({ bucket: 'chain', client, source: blockViewer })
36
- const range = await publisher.sync(from) // publishes [from, sourceHead]; null when up to date
40
+ const blocks = await S3BlockPublishRunner.create({ bucket: 'finalized', client, context, source })
41
+ const index = await S3IndexPublishRunner.create({ bucket: 'finalized', client, context, source })
42
+ const chainState = await S3ChainStatePublishRunner.create({ bucket: 'chain-state', client, context, source })
43
+
44
+ const range = await blocks.sync(from) // publishes [from, sourceHead]; null when up to date
45
+ if (range) {
46
+ for (let n = range[0]; n <= range[1]; n++) await index.publishStepsCompletedBy(n)
47
+ await chainState.publishHead() // last, so readers never see a head referencing missing files
48
+ }
37
49
  ```
38
50
 
39
- `sync(from)` resumes from a caller-supplied cursor — typically the block after the last published head pointer, which the finalizer owns. Omitting `from` republishes from genesis, which is safe (idempotent — deterministic keys, immutable content) but slow, so re-running after a crash is always safe. The path builders come from [`@xyo-network/xl1-rest-block-viewer`](https://www.npmjs.com/package/@xyo-network/xl1-rest-block-viewer), so reader and writer share one layout contract.
51
+ `sync(from)` resumes from a caller-supplied cursor — typically the block after the last published head pointer. Omitting `from` republishes from genesis, which is safe (idempotent — deterministic keys, immutable content) but slow, so re-running after a crash is always safe. The path builders come from [`@xyo-network/xl1-rest-block-viewer`](https://www.npmjs.com/package/@xyo-network/xl1-rest-block-viewer), so readers and writers share one layout contract.
40
52
 
41
53
  ## Install
42
54
 
@@ -68,7 +80,10 @@ bun add @xyo-network/xl1-rest-block-publisher
68
80
 
69
81
  ## What's Inside
70
82
 
71
- - **`RestBlockPublisher`** — `sync(from)` (cursor-resumed publish), `publishBlock`, `publishStep`/`publishStepsCompletedBy` (completed steps only).
83
+ - **`S3BlockPublishRunner`** — `sync(from)` (cursor-resumed publish) and `publishBlock`; implements `BlockPublishRunner`.
84
+ - **`S3IndexPublishRunner`** — `publishStep`/`publishStepsCompletedBy` (completed steps only); implements `IndexPublishRunner`.
85
+ - **`S3ChainStatePublishRunner`** — `publishHead` with must-revalidate caching; implements `ChainStatePublishRunner`.
86
+ - **`AbstractS3PublishRunner`** — shared S3 target params (bucket, client, prefix, encoding) for custom runners.
72
87
  - **`encodeBody`/`decodeBody`** — brotli/gzip/none body codecs used for storage and read-back.
73
88
 
74
89
  ## Building Locally
@@ -0,0 +1,34 @@
1
+ import type { S3Client } from '@aws-sdk/client-s3';
2
+ import type { CreatableProviderEventData, CreatableProviderParams } from '@xyo-network/xl1-protocol-sdk';
3
+ import { AbstractCreatableProvider } from '@xyo-network/xl1-protocol-sdk';
4
+ import type { RestContentEncoding } from './encoding.ts';
5
+ /** Shared parameters for S3-backed providers: where and how files are stored. */
6
+ export interface S3ProviderParams extends CreatableProviderParams {
7
+ bucket: string;
8
+ /** S3-compatible client pointed at the target endpoint (e.g. Cloudflare R2). */
9
+ client: S3Client;
10
+ /** Storage/wire encoding for published files. Defaults to 'br' (brotli). */
11
+ contentEncoding?: RestContentEncoding;
12
+ /** Optional key prefix, allowing the layout to live in a shared bucket. */
13
+ prefix?: string;
14
+ }
15
+ /** Finalized files never change once written. */
16
+ export declare const IMMUTABLE_CACHE_CONTROL = "public, max-age=31536000, immutable";
17
+ /** Chain state is rewritten as the chain advances. */
18
+ export declare const MUTABLE_CACHE_CONTROL = "public, max-age=0, must-revalidate";
19
+ /**
20
+ * Base for providers targeting S3-compatible storage. Bodies are pre-compressed at write
21
+ * time and stored with the matching Content-Encoding plus application/json, so HTTP clients
22
+ * decompress transparently — keys keep their `.json` extension. Reads decode the stored
23
+ * Content-Encoding back to JSON.
24
+ */
25
+ export declare abstract class AbstractS3Provider<TParams extends S3ProviderParams = S3ProviderParams, TEventData extends CreatableProviderEventData = CreatableProviderEventData> extends AbstractCreatableProvider<TParams, TEventData> {
26
+ get bucket(): string;
27
+ get client(): S3Client;
28
+ get contentEncoding(): RestContentEncoding;
29
+ get prefix(): string;
30
+ /** Reads and decodes a stored JSON file, or undefined when the key does not exist. */
31
+ protected getJson(path: string): Promise<unknown>;
32
+ protected putJson(path: string, json: string, cacheControl: string): Promise<void>;
33
+ }
34
+ //# sourceMappingURL=AbstractS3Provider.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"AbstractS3Provider.d.ts","sourceRoot":"","sources":["../../src/AbstractS3Provider.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,oBAAoB,CAAA;AAGlD,OAAO,KAAK,EAAE,0BAA0B,EAAE,uBAAuB,EAAE,MAAM,+BAA+B,CAAA;AACxG,OAAO,EAAE,yBAAyB,EAAE,MAAM,+BAA+B,CAAA;AAEzE,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,eAAe,CAAA;AAGxD,iFAAiF;AACjF,MAAM,WAAW,gBAAiB,SAAQ,uBAAuB;IAC/D,MAAM,EAAE,MAAM,CAAA;IACd,gFAAgF;IAChF,MAAM,EAAE,QAAQ,CAAA;IAChB,4EAA4E;IAC5E,eAAe,CAAC,EAAE,mBAAmB,CAAA;IACrC,2EAA2E;IAC3E,MAAM,CAAC,EAAE,MAAM,CAAA;CAChB;AAED,iDAAiD;AACjD,eAAO,MAAM,uBAAuB,wCAAwC,CAAA;AAC5E,sDAAsD;AACtD,eAAO,MAAM,qBAAqB,uCAAuC,CAAA;AAEzE;;;;;GAKG;AACH,8BAAsB,kBAAkB,CACtC,OAAO,SAAS,gBAAgB,GAAG,gBAAgB,EACnD,UAAU,SAAS,0BAA0B,GAAG,0BAA0B,CAC1E,SAAQ,yBAAyB,CAAC,OAAO,EAAE,UAAU,CAAC;IACtD,IAAI,MAAM,IAAI,MAAM,CAEnB;IAED,IAAI,MAAM,IAAI,QAAQ,CAErB;IAED,IAAI,eAAe,IAAI,mBAAmB,CAEzC;IAED,IAAI,MAAM,IAAI,MAAM,CAEnB;IAED,sFAAsF;cACtE,OAAO,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;cAYvC,OAAO,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,YAAY,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;CAWzF"}
@@ -0,0 +1,55 @@
1
+ import type { BlockPublishRunner, BlockViewer, SignedHydratedBlockWithHashMeta, XL1BlockNumber, XL1BlockRange } from '@xyo-network/xl1-protocol-lib';
2
+ import type { S3ProviderParams } from './AbstractS3Provider.ts';
3
+ import { AbstractS3Provider } from './AbstractS3Provider.ts';
4
+ /** A progress event emitted while publishing blocks. */
5
+ export interface BlockPublishProgress {
6
+ blockNumber: XL1BlockNumber;
7
+ published: number;
8
+ total: number;
9
+ }
10
+ /** Parameters for S3BlockPublishRunner. */
11
+ export interface S3BlockPublishRunnerParams extends S3ProviderParams {
12
+ /** Maximum concurrent block publishes during sync. */
13
+ concurrency?: number;
14
+ /**
15
+ * Called every `progressInterval` blocks (and at pass completion). When omitted, the same
16
+ * events are logged via the params logger so long backfills are observable out of the box.
17
+ */
18
+ onProgress?: (event: BlockPublishProgress) => void;
19
+ /** How many block publishes between progress reports during sync. */
20
+ progressInterval?: number;
21
+ /**
22
+ * The viewer to publish from (only finalized chains should be published). When omitted,
23
+ * resolved from the locator via BlockViewerMoniker.
24
+ */
25
+ source?: BlockViewer;
26
+ }
27
+ /**
28
+ * Publishes finalized blocks (and their payloads) as the static REST file layout that
29
+ * `RestBlockViewer` reads (see `paths.ts` in @xyo-network/xl1-rest-block-viewer).
30
+ *
31
+ * Publishing is idempotent: keys are deterministic and contents immutable, so re-publishing
32
+ * any range is safe. `sync(from)` resumes from a caller-supplied cursor (typically the block
33
+ * after the last published head); omitting it republishes from genesis, which is safe.
34
+ */
35
+ export declare class S3BlockPublishRunner extends AbstractS3Provider<S3BlockPublishRunnerParams> implements BlockPublishRunner {
36
+ static readonly defaultMoniker: "BlockPublishRunner";
37
+ static readonly dependencies: "BlockViewer"[];
38
+ static readonly monikers: "BlockPublishRunner"[];
39
+ moniker: "BlockPublishRunner";
40
+ protected _source?: BlockViewer;
41
+ get concurrency(): number;
42
+ get progressInterval(): number;
43
+ get source(): BlockViewer;
44
+ createHandler(): Promise<void>;
45
+ /** Publishes one block at its by-number and by-hash paths, plus its payload files. */
46
+ publishBlock(blockNumber: XL1BlockNumber): Promise<SignedHydratedBlockWithHashMeta | null>;
47
+ /**
48
+ * Publishes every block from `from` (inclusive, default genesis) through the source head.
49
+ * Returns the published range, or null when already up to date.
50
+ */
51
+ sync(from?: XL1BlockNumber): Promise<XL1BlockRange | null>;
52
+ /** Emits a progress event to the onProgress callback, or logs it when no callback is set. */
53
+ private report;
54
+ }
55
+ //# sourceMappingURL=S3BlockPublishRunner.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"S3BlockPublishRunner.d.ts","sourceRoot":"","sources":["../../src/S3BlockPublishRunner.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EACV,kBAAkB,EAAE,WAAW,EAAE,+BAA+B,EAAE,cAAc,EAAE,aAAa,EAChG,MAAM,+BAA+B,CAAA;AAUtC,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAA;AAC/D,OAAO,EAAE,kBAAkB,EAA2B,MAAM,yBAAyB,CAAA;AAErF,wDAAwD;AACxD,MAAM,WAAW,oBAAoB;IACnC,WAAW,EAAE,cAAc,CAAA;IAC3B,SAAS,EAAE,MAAM,CAAA;IACjB,KAAK,EAAE,MAAM,CAAA;CACd;AAED,2CAA2C;AAC3C,MAAM,WAAW,0BAA2B,SAAQ,gBAAgB;IAClE,sDAAsD;IACtD,WAAW,CAAC,EAAE,MAAM,CAAA;IACpB;;;OAGG;IACH,UAAU,CAAC,EAAE,CAAC,KAAK,EAAE,oBAAoB,KAAK,IAAI,CAAA;IAClD,qEAAqE;IACrE,gBAAgB,CAAC,EAAE,MAAM,CAAA;IACzB;;;OAGG;IACH,MAAM,CAAC,EAAE,WAAW,CAAA;CACrB;AAED;;;;;;;GAOG;AACH,qBACa,oBAAqB,SAAQ,kBAAkB,CAAC,0BAA0B,CAAE,YAAW,kBAAkB;IACpH,MAAM,CAAC,QAAQ,CAAC,cAAc,uBAA4B;IAC1D,MAAM,CAAC,QAAQ,CAAC,YAAY,kBAAuB;IACnD,MAAM,CAAC,QAAQ,CAAC,QAAQ,yBAA8B;IACtD,OAAO,uBAAsC;IAE7C,SAAS,CAAC,OAAO,CAAC,EAAE,WAAW,CAAA;IAE/B,IAAI,WAAW,IAAI,MAAM,CAExB;IAED,IAAI,gBAAgB,IAAI,MAAM,CAE7B;IAED,IAAI,MAAM,IAAI,WAAW,CAExB;IAEc,aAAa;IAK5B,sFAAsF;IAChF,YAAY,CAAC,WAAW,EAAE,cAAc,GAAG,OAAO,CAAC,+BAA+B,GAAG,IAAI,CAAC;IAYhG;;;OAGG;IACG,IAAI,CAAC,IAAI,GAAE,cAA0C,GAAG,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC;IAqB3F,6FAA6F;IAC7F,OAAO,CAAC,MAAM;CAQf"}
@@ -0,0 +1,31 @@
1
+ import type { ChainStatePublishRunner, ChainStateViewerMethods, SignedHydratedBlockWithHashMeta } from '@xyo-network/xl1-protocol-lib';
2
+ import type { S3ProviderParams } from './AbstractS3Provider.ts';
3
+ import { AbstractS3Provider } from './AbstractS3Provider.ts';
4
+ /** Parameters for S3ChainStatePublishRunner. */
5
+ export interface S3ChainStatePublishRunnerParams extends S3ProviderParams {
6
+ /**
7
+ * Where the current head is read from (any BlockViewer satisfies this). When omitted,
8
+ * resolved from the locator via BlockViewerMoniker.
9
+ */
10
+ source?: ChainStateViewerMethods;
11
+ }
12
+ /**
13
+ * Publishes the mutable chain state — the head pointer (`chain/head.json`) — that
14
+ * `RestChainStateViewer` reads. This is the one mutable file in the REST layout, so it is
15
+ * stored with must-revalidate caching and should target the chain-state bucket.
16
+ *
17
+ * Callers must publish the finalized files for a head before publishing the head itself,
18
+ * so readers never see a head that references missing files.
19
+ */
20
+ export declare class S3ChainStatePublishRunner extends AbstractS3Provider<S3ChainStatePublishRunnerParams> implements ChainStatePublishRunner {
21
+ static readonly defaultMoniker: "ChainStatePublishRunner";
22
+ static readonly dependencies: "BlockViewer"[];
23
+ static readonly monikers: "ChainStatePublishRunner"[];
24
+ moniker: "ChainStatePublishRunner";
25
+ protected _source?: ChainStateViewerMethods;
26
+ get source(): ChainStateViewerMethods;
27
+ createHandler(): Promise<void>;
28
+ /** Publishes the source's current head to the mutable chain state pointer. */
29
+ publishHead(): Promise<SignedHydratedBlockWithHashMeta>;
30
+ }
31
+ //# sourceMappingURL=S3ChainStatePublishRunner.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"S3ChainStatePublishRunner.d.ts","sourceRoot":"","sources":["../../src/S3ChainStatePublishRunner.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAEV,uBAAuB,EAAE,uBAAuB,EAAE,+BAA+B,EAClF,MAAM,+BAA+B,CAAA;AAKtC,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAA;AAC/D,OAAO,EAAE,kBAAkB,EAAyB,MAAM,yBAAyB,CAAA;AAEnF,gDAAgD;AAChD,MAAM,WAAW,+BAAgC,SAAQ,gBAAgB;IACvE;;;OAGG;IACH,MAAM,CAAC,EAAE,uBAAuB,CAAA;CACjC;AAED;;;;;;;GAOG;AACH,qBACa,yBAA0B,SAAQ,kBAAkB,CAAC,+BAA+B,CAAE,YAAW,uBAAuB;IACnI,MAAM,CAAC,QAAQ,CAAC,cAAc,4BAAiC;IAC/D,MAAM,CAAC,QAAQ,CAAC,YAAY,kBAAuB;IACnD,MAAM,CAAC,QAAQ,CAAC,QAAQ,8BAAmC;IAC3D,OAAO,4BAA2C;IAElD,SAAS,CAAC,OAAO,CAAC,EAAE,uBAAuB,CAAA;IAE3C,IAAI,MAAM,IAAI,uBAAuB,CAEpC;IAEc,aAAa;IAK5B,8EAA8E;IACxE,WAAW,IAAI,OAAO,CAAC,+BAA+B,CAAC;CAM9D"}
@@ -0,0 +1,26 @@
1
+ import type { Hash } from '@xylabs/sdk-js';
2
+ import type { ChainStateViewer, SignedHydratedBlockWithHashMeta, XL1BlockNumber } from '@xyo-network/xl1-protocol-lib';
3
+ import type { S3ProviderParams } from './AbstractS3Provider.ts';
4
+ import { AbstractS3Provider } from './AbstractS3Provider.ts';
5
+ /** Parameters for S3ChainStateViewer. */
6
+ export interface S3ChainStateViewerParams extends S3ProviderParams {
7
+ }
8
+ /**
9
+ * A ChainStateViewer over the chain-state bucket, reading the published head pointer back
10
+ * through the S3 API (authenticated, not the public HTTP layout — see RestChainStateViewer
11
+ * for anonymous reads). This is how publishers learn their resume cursor: the head pointer
12
+ * is only written after the finalized files it references exist.
13
+ */
14
+ export declare class S3ChainStateViewer extends AbstractS3Provider<S3ChainStateViewerParams, ChainStateViewer['eventData']> implements ChainStateViewer {
15
+ static readonly defaultMoniker: "ChainStateViewer";
16
+ static readonly dependencies: never[];
17
+ static readonly monikers: "ChainStateViewer"[];
18
+ moniker: "ChainStateViewer";
19
+ /** The published head; throws when no head pointer has been published yet. */
20
+ currentBlock(): Promise<SignedHydratedBlockWithHashMeta>;
21
+ currentBlockHash(): Promise<Hash>;
22
+ currentBlockNumber(): Promise<XL1BlockNumber>;
23
+ /** The published head, or undefined when no head pointer has been published yet. */
24
+ tryCurrentBlock(): Promise<SignedHydratedBlockWithHashMeta | undefined>;
25
+ }
26
+ //# sourceMappingURL=S3ChainStateViewer.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"S3ChainStateViewer.d.ts","sourceRoot":"","sources":["../../src/S3ChainStateViewer.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,gBAAgB,CAAA;AAE1C,OAAO,KAAK,EACV,gBAAgB,EAAE,+BAA+B,EAAE,cAAc,EAClE,MAAM,+BAA+B,CAAA;AAKtC,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAA;AAC/D,OAAO,EAAE,kBAAkB,EAAE,MAAM,yBAAyB,CAAA;AAE5D,yCAAyC;AACzC,MAAM,WAAW,wBAAyB,SAAQ,gBAAgB;CAAG;AAErE;;;;;GAKG;AACH,qBACa,kBAAmB,SAAQ,kBAAkB,CAAC,wBAAwB,EAAE,gBAAgB,CAAC,WAAW,CAAC,CAChH,YAAW,gBAAgB;IAC3B,MAAM,CAAC,QAAQ,CAAC,cAAc,qBAA0B;IACxD,MAAM,CAAC,QAAQ,CAAC,YAAY,UAAK;IACjC,MAAM,CAAC,QAAQ,CAAC,QAAQ,uBAA4B;IACpD,OAAO,qBAAoC;IAE3C,8EAA8E;IACxE,YAAY,IAAI,OAAO,CAAC,+BAA+B,CAAC;IAOxD,gBAAgB,IAAI,OAAO,CAAC,IAAI,CAAC;IAIjC,kBAAkB,IAAI,OAAO,CAAC,cAAc,CAAC;IAInD,oFAAoF;IAC9E,eAAe,IAAI,OAAO,CAAC,+BAA+B,GAAG,SAAS,CAAC;CAI9E"}
@@ -0,0 +1,33 @@
1
+ import type { BlockViewer, IndexPublishRunner, XL1BlockNumber } from '@xyo-network/xl1-protocol-lib';
2
+ import type { S3ProviderParams } from './AbstractS3Provider.ts';
3
+ import { AbstractS3Provider } from './AbstractS3Provider.ts';
4
+ /** Parameters for S3IndexPublishRunner. */
5
+ export interface S3IndexPublishRunnerParams extends S3ProviderParams {
6
+ /**
7
+ * The viewer to publish from (only finalized chains should be published). When omitted,
8
+ * resolved from the locator via BlockViewerMoniker.
9
+ */
10
+ source?: BlockViewer;
11
+ }
12
+ /**
13
+ * Publishes completed steps' block summaries (the chain index) as the static REST file
14
+ * layout that `RestIndexViewer` reads. Step files are immutable: only complete steps are
15
+ * ever published, and a completed step's blocks never change.
16
+ */
17
+ export declare class S3IndexPublishRunner extends AbstractS3Provider<S3IndexPublishRunnerParams> implements IndexPublishRunner {
18
+ static readonly defaultMoniker: "IndexPublishRunner";
19
+ static readonly dependencies: "BlockViewer"[];
20
+ static readonly monikers: "IndexPublishRunner"[];
21
+ moniker: "IndexPublishRunner";
22
+ protected _source?: BlockViewer;
23
+ get source(): BlockViewer;
24
+ createHandler(): Promise<void>;
25
+ /**
26
+ * Publishes one completed step's blocks as a BlocksStepSummary file (blocks oldest-first).
27
+ * Asserts the step is complete — partial tail steps are never published.
28
+ */
29
+ publishStep(stepLevel: number, stepIndex: number): Promise<void>;
30
+ /** Publishes every step (at every level) completed by the given block. */
31
+ publishStepsCompletedBy(blockNumber: XL1BlockNumber): Promise<void>;
32
+ }
33
+ //# sourceMappingURL=S3IndexPublishRunner.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"S3IndexPublishRunner.d.ts","sourceRoot":"","sources":["../../src/S3IndexPublishRunner.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EACS,WAAW,EAAE,kBAAkB,EAAE,cAAc,EACnE,MAAM,+BAA+B,CAAA;AAOtC,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAA;AAC/D,OAAO,EAAE,kBAAkB,EAA2B,MAAM,yBAAyB,CAAA;AAErF,2CAA2C;AAC3C,MAAM,WAAW,0BAA2B,SAAQ,gBAAgB;IAClE;;;OAGG;IACH,MAAM,CAAC,EAAE,WAAW,CAAA;CACrB;AAED;;;;GAIG;AACH,qBACa,oBAAqB,SAAQ,kBAAkB,CAAC,0BAA0B,CAAE,YAAW,kBAAkB;IACpH,MAAM,CAAC,QAAQ,CAAC,cAAc,uBAA4B;IAC1D,MAAM,CAAC,QAAQ,CAAC,YAAY,kBAAuB;IACnD,MAAM,CAAC,QAAQ,CAAC,QAAQ,yBAA8B;IACtD,OAAO,uBAAsC;IAE7C,SAAS,CAAC,OAAO,CAAC,EAAE,WAAW,CAAA;IAE/B,IAAI,MAAM,IAAI,WAAW,CAExB;IAEc,aAAa;IAK5B;;;OAGG;IACG,WAAW,CAAC,SAAS,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAkBtE,0EAA0E;IACpE,uBAAuB,CAAC,WAAW,EAAE,cAAc,GAAG,OAAO,CAAC,IAAI,CAAC;CAQ1E"}
@@ -1,3 +1,7 @@
1
+ export * from './AbstractS3Provider.ts';
1
2
  export * from './encoding.ts';
2
- export * from './RestBlockPublisher.ts';
3
+ export * from './S3BlockPublishRunner.ts';
4
+ export * from './S3ChainStatePublishRunner.ts';
5
+ export * from './S3ChainStateViewer.ts';
6
+ export * from './S3IndexPublishRunner.ts';
3
7
  //# sourceMappingURL=index.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,eAAe,CAAA;AAC7B,cAAc,yBAAyB,CAAA"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,yBAAyB,CAAA;AACvC,cAAc,eAAe,CAAA;AAC7B,cAAc,2BAA2B,CAAA;AACzC,cAAc,gCAAgC,CAAA;AAC9C,cAAc,yBAAyB,CAAA;AACvC,cAAc,2BAA2B,CAAA"}
@@ -1,5 +1,6 @@
1
1
  var __defProp = Object.defineProperty;
2
2
  var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
3
+ var __defNormalProp = (obj, key, value) => key in obj ? __defProp(obj, key, { enumerable: true, configurable: true, writable: true, value }) : obj[key] = value;
3
4
  var __decorateClass = (decorators, target, key, kind) => {
4
5
  var result = kind > 1 ? void 0 : kind ? __getOwnPropDesc(target, key) : target;
5
6
  for (var i = decorators.length - 1, decorator; i >= 0; i--)
@@ -8,6 +9,12 @@ var __decorateClass = (decorators, target, key, kind) => {
8
9
  if (kind && result) __defProp(target, key, result);
9
10
  return result;
10
11
  };
12
+ var __publicField = (obj, key, value) => __defNormalProp(obj, typeof key !== "symbol" ? key + "" : key, value);
13
+
14
+ // src/AbstractS3Provider.ts
15
+ import { GetObjectCommand, PutObjectCommand } from "@aws-sdk/client-s3";
16
+ import { assertEx } from "@xylabs/sdk-js";
17
+ import { AbstractCreatableProvider } from "@xyo-network/xl1-protocol-sdk";
11
18
 
12
19
  // src/encoding.ts
13
20
  import ZLIB from "node:zlib";
@@ -38,100 +45,97 @@ function decodeBody(body, contentEncoding) {
38
45
  }
39
46
  }
40
47
 
41
- // src/RestBlockPublisher.ts
42
- import { PutObjectCommand } from "@aws-sdk/client-s3";
43
- import {
44
- AbstractCreatable,
45
- assertEx,
46
- creatable,
47
- isNull
48
- } from "@xylabs/sdk-js";
49
- import { asXL1BlockNumber, StepSizes } from "@xyo-network/xl1-protocol-lib";
50
- import { blocksMaxStep, BlocksStepSummarySchema } from "@xyo-network/xl1-protocol-sdk";
51
- import {
52
- blockHashPath,
53
- blockNumberPath,
54
- blocksStepPath,
55
- payloadPath
56
- } from "@xyo-network/xl1-rest-block-viewer";
57
- import { Semaphore } from "async-mutex";
48
+ // src/AbstractS3Provider.ts
58
49
  var IMMUTABLE_CACHE_CONTROL = "public, max-age=31536000, immutable";
59
- var RestBlockPublisher = class extends AbstractCreatable {
50
+ var MUTABLE_CACHE_CONTROL = "public, max-age=0, must-revalidate";
51
+ var AbstractS3Provider = class extends AbstractCreatableProvider {
60
52
  get bucket() {
61
53
  return assertEx(this.params.bucket, () => "No bucket specified");
62
54
  }
63
55
  get client() {
64
56
  return assertEx(this.params.client, () => "No client specified");
65
57
  }
66
- get concurrency() {
67
- return this.params.concurrency ?? 8;
68
- }
69
58
  get contentEncoding() {
70
59
  return this.params.contentEncoding ?? "br";
71
60
  }
72
61
  get prefix() {
73
62
  return this.params.prefix ?? "";
74
63
  }
64
+ /** Reads and decodes a stored JSON file, or undefined when the key does not exist. */
65
+ async getJson(path) {
66
+ try {
67
+ const response = await this.client.send(new GetObjectCommand({ Bucket: this.bucket, Key: `${this.prefix}${path}` }));
68
+ const body = await response.Body?.transformToByteArray();
69
+ if (body === void 0) return void 0;
70
+ return JSON.parse(decodeBody(body, response.ContentEncoding));
71
+ } catch (error) {
72
+ if (isNotFoundError(error)) return void 0;
73
+ throw error;
74
+ }
75
+ }
76
+ async putJson(path, json, cacheControl) {
77
+ const { body, contentEncoding } = encodeBody(json, this.contentEncoding);
78
+ await this.client.send(new PutObjectCommand({
79
+ Bucket: this.bucket,
80
+ Key: `${this.prefix}${path}`,
81
+ Body: body,
82
+ CacheControl: cacheControl,
83
+ ContentEncoding: contentEncoding,
84
+ ContentType: "application/json"
85
+ }));
86
+ }
87
+ };
88
+ var isNotFoundError = (error) => {
89
+ if (typeof error !== "object" || error === null) return false;
90
+ const { name, $metadata } = error;
91
+ return name === "NoSuchKey" || name === "NotFound" || $metadata?.httpStatusCode === 404;
92
+ };
93
+
94
+ // src/S3BlockPublishRunner.ts
95
+ import { assertEx as assertEx2, isNull } from "@xylabs/sdk-js";
96
+ import {
97
+ asXL1BlockNumber,
98
+ BlockPublishRunnerMoniker,
99
+ BlockViewerMoniker
100
+ } from "@xyo-network/xl1-protocol-lib";
101
+ import { creatableProvider } from "@xyo-network/xl1-protocol-sdk";
102
+ import {
103
+ blockHashPath,
104
+ blockNumberPath,
105
+ payloadPath
106
+ } from "@xyo-network/xl1-rest-block-viewer";
107
+ import { Semaphore } from "async-mutex";
108
+ var S3BlockPublishRunner = class extends AbstractS3Provider {
109
+ moniker = S3BlockPublishRunner.defaultMoniker;
110
+ _source;
111
+ get concurrency() {
112
+ return this.params.concurrency ?? 8;
113
+ }
75
114
  get progressInterval() {
76
115
  return this.params.progressInterval ?? 100;
77
116
  }
78
117
  get source() {
79
- return assertEx(this.params.source, () => "No source specified");
118
+ return assertEx2(this._source, () => "No source specified");
119
+ }
120
+ async createHandler() {
121
+ await super.createHandler();
122
+ this._source = this.params.source ?? await this.locator.getInstance(BlockViewerMoniker);
80
123
  }
81
124
  /** Publishes one block at its by-number and by-hash paths, plus its payload files. */
82
125
  async publishBlock(blockNumber) {
83
126
  const block = await this.source.blockByNumber(blockNumber);
84
127
  if (isNull(block)) return null;
85
128
  const json = JSON.stringify(block);
86
- await this.putJson(blockNumberPath(block[0].block), json);
87
- await this.putJson(blockHashPath(block[0]._hash), json);
129
+ await this.putJson(blockNumberPath(block[0].block), json, IMMUTABLE_CACHE_CONTROL);
130
+ await this.putJson(blockHashPath(block[0]._hash), json, IMMUTABLE_CACHE_CONTROL);
88
131
  for (const payload of block[1]) {
89
- await this.putJson(payloadPath(payload._hash), JSON.stringify(payload));
132
+ await this.putJson(payloadPath(payload._hash), JSON.stringify(payload), IMMUTABLE_CACHE_CONTROL);
90
133
  }
91
134
  return block;
92
135
  }
93
136
  /**
94
- * Publishes one completed step's blocks as a BlocksStepSummary file (blocks oldest-first).
95
- * Asserts the step is complete partial tail steps are never published.
96
- */
97
- async publishStep(stepLevel, stepIndex) {
98
- const size = StepSizes[stepLevel];
99
- assertEx(stepLevel <= blocksMaxStep, () => `publishStep does not support step levels above ${blocksMaxStep} (requested ${stepLevel})`);
100
- const lastBlockNumber = (stepIndex + 1) * size - 1;
101
- const headNumber = await this.source.currentBlockNumber();
102
- assertEx(lastBlockNumber <= headNumber, () => `Step ${stepLevel}/${stepIndex} is not complete (head ${headNumber})`);
103
- const newestFirst = await this.source.blocksByStep(stepLevel, stepIndex);
104
- const blocks = newestFirst.toReversed();
105
- const summary = {
106
- schema: BlocksStepSummarySchema,
107
- hash: assertEx(blocks.at(-1), () => `No blocks for step ${stepLevel}/${stepIndex}`)[0]._hash,
108
- stepSize: size,
109
- blocks
110
- };
111
- await this.putJson(blocksStepPath(stepLevel, stepIndex), JSON.stringify(summary));
112
- this.report({
113
- stepIndex,
114
- stepLevel,
115
- type: "step"
116
- });
117
- }
118
- /** Publishes every step (at every level) completed by the given block. */
119
- async publishStepsCompletedBy(blockNumber) {
120
- for (let stepLevel = 0; stepLevel <= blocksMaxStep; stepLevel++) {
121
- const size = StepSizes[stepLevel];
122
- if ((blockNumber + 1) % size === 0) {
123
- await this.publishStep(stepLevel, (blockNumber + 1) / size - 1);
124
- }
125
- }
126
- }
127
- /**
128
- * Publishes everything from `from` (inclusive, default genesis) to the source head
129
- * (inclusive), then the completed steps in that range. Returns the published range, or
130
- * null when already up to date.
131
- *
132
- * The caller supplies the resume cursor — typically the block after the last published
133
- * head pointer, which is written by the finalizer rather than this publisher. Omitting
134
- * `from` republishes from genesis, which is safe (idempotent) but slow.
137
+ * Publishes every block from `from` (inclusive, default genesis) through the source head.
138
+ * Returns the published range, or null when already up to date.
135
139
  */
136
140
  async sync(from = asXL1BlockNumber(0, true)) {
137
141
  const sourceHead = await this.source.currentBlock();
@@ -142,33 +146,18 @@ var RestBlockPublisher = class extends AbstractCreatable {
142
146
  const total = numbers.length;
143
147
  let publishedCount = 0;
144
148
  await Promise.all(numbers.map((blockNumber) => semaphore.runExclusive(async () => {
145
- assertEx(await this.publishBlock(blockNumber), () => `Block not found in source [${blockNumber}]`);
149
+ assertEx2(await this.publishBlock(blockNumber), () => `Block not found in source [${blockNumber}]`);
146
150
  publishedCount += 1;
147
151
  if (publishedCount % this.progressInterval === 0 || publishedCount === total) {
148
152
  this.report({
149
153
  blockNumber,
150
154
  published: publishedCount,
151
- total,
152
- type: "block"
155
+ total
153
156
  });
154
157
  }
155
158
  })));
156
- for (const blockNumber of numbers) {
157
- await this.publishStepsCompletedBy(blockNumber);
158
- }
159
159
  return [from, end];
160
160
  }
161
- async putJson(path, json) {
162
- const { body, contentEncoding } = encodeBody(json, this.contentEncoding);
163
- await this.client.send(new PutObjectCommand({
164
- Bucket: this.bucket,
165
- Key: `${this.prefix}${path}`,
166
- Body: body,
167
- CacheControl: IMMUTABLE_CACHE_CONTROL,
168
- ContentEncoding: contentEncoding,
169
- ContentType: "application/json"
170
- }));
171
- }
172
161
  /** Emits a progress event to the onProgress callback, or logs it when no callback is set. */
173
162
  report(event) {
174
163
  const onProgress = this.params.onProgress;
@@ -176,23 +165,144 @@ var RestBlockPublisher = class extends AbstractCreatable {
176
165
  onProgress(event);
177
166
  return;
178
167
  }
179
- switch (event.type) {
180
- case "block": {
181
- this.logger?.info(`RestBlockPublisher: published block ${event.blockNumber} (${event.published}/${event.total})`);
182
- break;
183
- }
184
- case "step": {
185
- this.logger?.info(`RestBlockPublisher: published step ${event.stepLevel}/${event.stepIndex}`);
186
- break;
168
+ this.logger?.info(`S3BlockPublishRunner: published block ${event.blockNumber} (${event.published}/${event.total})`);
169
+ }
170
+ };
171
+ __publicField(S3BlockPublishRunner, "defaultMoniker", BlockPublishRunnerMoniker);
172
+ __publicField(S3BlockPublishRunner, "dependencies", [BlockViewerMoniker]);
173
+ __publicField(S3BlockPublishRunner, "monikers", [BlockPublishRunnerMoniker]);
174
+ S3BlockPublishRunner = __decorateClass([
175
+ creatableProvider()
176
+ ], S3BlockPublishRunner);
177
+
178
+ // src/S3ChainStatePublishRunner.ts
179
+ import { assertEx as assertEx3 } from "@xylabs/sdk-js";
180
+ import { BlockViewerMoniker as BlockViewerMoniker2, ChainStatePublishRunnerMoniker } from "@xyo-network/xl1-protocol-lib";
181
+ import { creatableProvider as creatableProvider2 } from "@xyo-network/xl1-protocol-sdk";
182
+ import { headPath } from "@xyo-network/xl1-rest-block-viewer";
183
+ var S3ChainStatePublishRunner = class extends AbstractS3Provider {
184
+ moniker = S3ChainStatePublishRunner.defaultMoniker;
185
+ _source;
186
+ get source() {
187
+ return assertEx3(this._source, () => "No source specified");
188
+ }
189
+ async createHandler() {
190
+ await super.createHandler();
191
+ this._source = this.params.source ?? await this.locator.getInstance(BlockViewerMoniker2);
192
+ }
193
+ /** Publishes the source's current head to the mutable chain state pointer. */
194
+ async publishHead() {
195
+ const head = await this.source.currentBlock();
196
+ await this.putJson(headPath(), JSON.stringify(head), MUTABLE_CACHE_CONTROL);
197
+ this.logger?.info(`S3ChainStatePublishRunner: published head ${head[0].block}`);
198
+ return head;
199
+ }
200
+ };
201
+ __publicField(S3ChainStatePublishRunner, "defaultMoniker", ChainStatePublishRunnerMoniker);
202
+ __publicField(S3ChainStatePublishRunner, "dependencies", [BlockViewerMoniker2]);
203
+ __publicField(S3ChainStatePublishRunner, "monikers", [ChainStatePublishRunnerMoniker]);
204
+ S3ChainStatePublishRunner = __decorateClass([
205
+ creatableProvider2()
206
+ ], S3ChainStatePublishRunner);
207
+
208
+ // src/S3ChainStateViewer.ts
209
+ import { assertEx as assertEx4 } from "@xylabs/sdk-js";
210
+ import { asSignedHydratedBlockWithHashMeta, ChainStateViewerMoniker } from "@xyo-network/xl1-protocol-lib";
211
+ import { creatableProvider as creatableProvider3 } from "@xyo-network/xl1-protocol-sdk";
212
+ import { headPath as headPath2 } from "@xyo-network/xl1-rest-block-viewer";
213
+ var S3ChainStateViewer = class extends AbstractS3Provider {
214
+ moniker = S3ChainStateViewer.defaultMoniker;
215
+ /** The published head; throws when no head pointer has been published yet. */
216
+ async currentBlock() {
217
+ return await this.spanAsync("currentBlock", async () => {
218
+ const parsed = assertEx4(await this.getJson(headPath2()), () => "Head not found");
219
+ return asSignedHydratedBlockWithHashMeta(parsed, true);
220
+ }, this.context);
221
+ }
222
+ async currentBlockHash() {
223
+ return (await this.currentBlock())[0]._hash;
224
+ }
225
+ async currentBlockNumber() {
226
+ return (await this.currentBlock())[0].block;
227
+ }
228
+ /** The published head, or undefined when no head pointer has been published yet. */
229
+ async tryCurrentBlock() {
230
+ const parsed = await this.getJson(headPath2());
231
+ return parsed === void 0 ? void 0 : asSignedHydratedBlockWithHashMeta(parsed, true);
232
+ }
233
+ };
234
+ __publicField(S3ChainStateViewer, "defaultMoniker", ChainStateViewerMoniker);
235
+ __publicField(S3ChainStateViewer, "dependencies", []);
236
+ __publicField(S3ChainStateViewer, "monikers", [ChainStateViewerMoniker]);
237
+ S3ChainStateViewer = __decorateClass([
238
+ creatableProvider3()
239
+ ], S3ChainStateViewer);
240
+
241
+ // src/S3IndexPublishRunner.ts
242
+ import { assertEx as assertEx5 } from "@xylabs/sdk-js";
243
+ import {
244
+ BlocksStepSummarySchema,
245
+ BlockViewerMoniker as BlockViewerMoniker3,
246
+ IndexPublishRunnerMoniker,
247
+ StepSizes
248
+ } from "@xyo-network/xl1-protocol-lib";
249
+ import { blocksMaxStep, creatableProvider as creatableProvider4 } from "@xyo-network/xl1-protocol-sdk";
250
+ import { blocksStepPath } from "@xyo-network/xl1-rest-block-viewer";
251
+ var S3IndexPublishRunner = class extends AbstractS3Provider {
252
+ moniker = S3IndexPublishRunner.defaultMoniker;
253
+ _source;
254
+ get source() {
255
+ return assertEx5(this._source, () => "No source specified");
256
+ }
257
+ async createHandler() {
258
+ await super.createHandler();
259
+ this._source = this.params.source ?? await this.locator.getInstance(BlockViewerMoniker3);
260
+ }
261
+ /**
262
+ * Publishes one completed step's blocks as a BlocksStepSummary file (blocks oldest-first).
263
+ * Asserts the step is complete — partial tail steps are never published.
264
+ */
265
+ async publishStep(stepLevel, stepIndex) {
266
+ const size = StepSizes[stepLevel];
267
+ assertEx5(stepLevel <= blocksMaxStep, () => `publishStep does not support step levels above ${blocksMaxStep} (requested ${stepLevel})`);
268
+ const lastBlockNumber = (stepIndex + 1) * size - 1;
269
+ const headNumber = await this.source.currentBlockNumber();
270
+ assertEx5(lastBlockNumber <= headNumber, () => `Step ${stepLevel}/${stepIndex} is not complete (head ${headNumber})`);
271
+ const newestFirst = await this.source.blocksByStep(stepLevel, stepIndex);
272
+ const blocks = newestFirst.toReversed();
273
+ const summary = {
274
+ schema: BlocksStepSummarySchema,
275
+ hash: assertEx5(blocks.at(-1), () => `No blocks for step ${stepLevel}/${stepIndex}`)[0]._hash,
276
+ stepSize: size,
277
+ blocks
278
+ };
279
+ await this.putJson(blocksStepPath(stepLevel, stepIndex), JSON.stringify(summary), IMMUTABLE_CACHE_CONTROL);
280
+ this.logger?.info(`S3IndexPublishRunner: published step ${stepLevel}/${stepIndex}`);
281
+ }
282
+ /** Publishes every step (at every level) completed by the given block. */
283
+ async publishStepsCompletedBy(blockNumber) {
284
+ for (let stepLevel = 0; stepLevel <= blocksMaxStep; stepLevel++) {
285
+ const size = StepSizes[stepLevel];
286
+ if ((blockNumber + 1) % size === 0) {
287
+ await this.publishStep(stepLevel, (blockNumber + 1) / size - 1);
187
288
  }
188
289
  }
189
290
  }
190
291
  };
191
- RestBlockPublisher = __decorateClass([
192
- creatable()
193
- ], RestBlockPublisher);
292
+ __publicField(S3IndexPublishRunner, "defaultMoniker", IndexPublishRunnerMoniker);
293
+ __publicField(S3IndexPublishRunner, "dependencies", [BlockViewerMoniker3]);
294
+ __publicField(S3IndexPublishRunner, "monikers", [IndexPublishRunnerMoniker]);
295
+ S3IndexPublishRunner = __decorateClass([
296
+ creatableProvider4()
297
+ ], S3IndexPublishRunner);
194
298
  export {
195
- RestBlockPublisher,
299
+ AbstractS3Provider,
300
+ IMMUTABLE_CACHE_CONTROL,
301
+ MUTABLE_CACHE_CONTROL,
302
+ S3BlockPublishRunner,
303
+ S3ChainStatePublishRunner,
304
+ S3ChainStateViewer,
305
+ S3IndexPublishRunner,
196
306
  decodeBody,
197
307
  encodeBody
198
308
  };
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "version": 3,
3
- "sources": ["../../src/encoding.ts", "../../src/RestBlockPublisher.ts"],
4
- "sourcesContent": ["import ZLIB from 'node:zlib'\n\n/** Wire/storage encoding for published files. */\nexport type RestContentEncoding = 'br' | 'gzip' | 'none'\n\n/** An encoded file body plus the HTTP Content-Encoding it should be served with. */\nexport interface EncodedBody {\n body: Uint8Array\n contentEncoding?: 'br' | 'gzip'\n}\n\n/**\n * Encodes a JSON string for storage. Bodies are pre-compressed at write time and stored\n * with the matching Content-Encoding so HTTP clients decompress transparently \u2014 keys keep\n * their .json extension and Content-Type stays application/json.\n */\nexport function encodeBody(json: string, encoding: RestContentEncoding): EncodedBody {\n switch (encoding) {\n case 'br': {\n return { body: ZLIB.brotliCompressSync(json), contentEncoding: 'br' }\n }\n case 'gzip': {\n return { body: ZLIB.gzipSync(json), contentEncoding: 'gzip' }\n }\n case 'none': {\n return { body: new TextEncoder().encode(json) }\n }\n }\n}\n\n/** Decodes a stored body back to its JSON string using the object's Content-Encoding. */\nexport function decodeBody(body: Uint8Array, contentEncoding?: string): string {\n switch (contentEncoding) {\n case 'br': {\n return ZLIB.brotliDecompressSync(body).toString('utf8')\n }\n case 'gzip': {\n return ZLIB.gunzipSync(body).toString('utf8')\n }\n default: {\n return new TextDecoder().decode(body)\n }\n }\n}\n", "import type { S3Client } from '@aws-sdk/client-s3'\nimport { PutObjectCommand } from '@aws-sdk/client-s3'\nimport type { CreatableParams } from '@xylabs/sdk-js'\nimport {\n AbstractCreatable, assertEx, creatable, isNull,\n} from '@xylabs/sdk-js'\nimport type {\n BlockViewer, SignedHydratedBlockWithHashMeta, XL1BlockNumber, XL1BlockRange,\n} from '@xyo-network/xl1-protocol-lib'\nimport { asXL1BlockNumber, StepSizes } from '@xyo-network/xl1-protocol-lib'\nimport type { BlocksStepSummary } from '@xyo-network/xl1-protocol-sdk'\nimport { blocksMaxStep, BlocksStepSummarySchema } from '@xyo-network/xl1-protocol-sdk'\nimport {\n blockHashPath, blockNumberPath, blocksStepPath, payloadPath,\n} from '@xyo-network/xl1-rest-block-viewer'\nimport { Semaphore } from 'async-mutex'\n\nimport type { RestContentEncoding } from './encoding.ts'\nimport { encodeBody } from './encoding.ts'\n\n/** A progress event emitted while publishing. */\nexport type RestPublishEvent\n = | { blockNumber: XL1BlockNumber; published: number; total: number; type: 'block' }\n | { stepIndex: number; stepLevel: number; type: 'step' }\n\n/** Parameters for RestBlockPublisher. */\nexport interface RestBlockPublisherParams extends CreatableParams {\n bucket: string\n /** S3-compatible client pointed at the target endpoint (e.g. Cloudflare R2). */\n client: S3Client\n /** Maximum concurrent block publishes during sync. */\n concurrency?: number\n /** Storage/wire encoding for published files. Defaults to 'br' (brotli). */\n contentEncoding?: RestContentEncoding\n /**\n * Called as publishing progresses: every `progressInterval` blocks (and at pass completion)\n * and per step file. When omitted, the same events are logged via the params logger so\n * long backfills are observable out of the box.\n */\n onProgress?: (event: RestPublishEvent) => void\n /** Optional key prefix, allowing the layout to live in a shared bucket. */\n prefix?: string\n /** How many block publishes between progress reports during sync. */\n progressInterval?: number\n /** The viewer to publish from (only finalized chains should be published). */\n source: BlockViewer\n}\n\n/** Everything this publisher writes is immutable once written. */\nconst IMMUTABLE_CACHE_CONTROL = 'public, max-age=31536000, immutable'\n\n/**\n * Publishes a chain's immutable files as the static REST file layout that `RestBlockViewer`\n * reads (see `paths.ts` in @xyo-network/xl1-rest-block-viewer).\n *\n * The mutable chain state (the head pointer, `chain/head.json`) is NOT written here \u2014 it\n * lives in a separate chain-state bucket and is written by the chain's finalizer after the\n * finalized files for that head exist, so readers never see a head that references missing\n * files.\n *\n * Publishing is idempotent: keys are deterministic and contents immutable, so re-publishing\n * any range is safe. `sync(from)` resumes from a caller-supplied cursor (typically the block\n * after the last published head); omitting it republishes from genesis, which is safe.\n */\n@creatable()\nexport class RestBlockPublisher extends AbstractCreatable<RestBlockPublisherParams> {\n get bucket(): string {\n return assertEx(this.params.bucket, () => 'No bucket specified')\n }\n\n get client(): S3Client {\n return assertEx(this.params.client, () => 'No client specified')\n }\n\n get concurrency(): number {\n return this.params.concurrency ?? 8\n }\n\n get contentEncoding(): RestContentEncoding {\n return this.params.contentEncoding ?? 'br'\n }\n\n get prefix(): string {\n return this.params.prefix ?? ''\n }\n\n get progressInterval(): number {\n return this.params.progressInterval ?? 100\n }\n\n get source(): BlockViewer {\n return assertEx(this.params.source, () => 'No source specified')\n }\n\n /** Publishes one block at its by-number and by-hash paths, plus its payload files. */\n async publishBlock(blockNumber: XL1BlockNumber): Promise<SignedHydratedBlockWithHashMeta | null> {\n const block = await this.source.blockByNumber(blockNumber)\n if (isNull(block)) return null\n const json = JSON.stringify(block)\n await this.putJson(blockNumberPath(block[0].block), json)\n await this.putJson(blockHashPath(block[0]._hash), json)\n for (const payload of block[1]) {\n await this.putJson(payloadPath(payload._hash), JSON.stringify(payload))\n }\n return block\n }\n\n /**\n * Publishes one completed step's blocks as a BlocksStepSummary file (blocks oldest-first).\n * Asserts the step is complete \u2014 partial tail steps are never published.\n */\n async publishStep(stepLevel: number, stepIndex: number): Promise<void> {\n const size = StepSizes[stepLevel]\n assertEx(stepLevel <= blocksMaxStep, () => `publishStep does not support step levels above ${blocksMaxStep} (requested ${stepLevel})`)\n const lastBlockNumber = (stepIndex + 1) * size - 1\n const headNumber = await this.source.currentBlockNumber()\n assertEx(lastBlockNumber <= headNumber, () => `Step ${stepLevel}/${stepIndex} is not complete (head ${headNumber})`)\n const newestFirst = await this.source.blocksByStep(stepLevel, stepIndex)\n const blocks = newestFirst.toReversed()\n const summary: BlocksStepSummary = {\n schema: BlocksStepSummarySchema,\n hash: assertEx(blocks.at(-1), () => `No blocks for step ${stepLevel}/${stepIndex}`)[0]._hash,\n stepSize: size,\n blocks,\n }\n await this.putJson(blocksStepPath(stepLevel, stepIndex), JSON.stringify(summary))\n this.report({\n stepIndex, stepLevel, type: 'step',\n })\n }\n\n /** Publishes every step (at every level) completed by the given block. */\n async publishStepsCompletedBy(blockNumber: XL1BlockNumber): Promise<void> {\n for (let stepLevel = 0; stepLevel <= blocksMaxStep; stepLevel++) {\n const size = StepSizes[stepLevel]\n if ((blockNumber + 1) % size === 0) {\n await this.publishStep(stepLevel, (blockNumber + 1) / size - 1)\n }\n }\n }\n\n /**\n * Publishes everything from `from` (inclusive, default genesis) to the source head\n * (inclusive), then the completed steps in that range. Returns the published range, or\n * null when already up to date.\n *\n * The caller supplies the resume cursor \u2014 typically the block after the last published\n * head pointer, which is written by the finalizer rather than this publisher. Omitting\n * `from` republishes from genesis, which is safe (idempotent) but slow.\n */\n async sync(from: XL1BlockNumber = asXL1BlockNumber(0, true)): Promise<XL1BlockRange | null> {\n const sourceHead = await this.source.currentBlock()\n const end = sourceHead[0].block\n if (from > end) return null\n\n const semaphore = new Semaphore(this.concurrency)\n const numbers = Array.from({ length: end - from + 1 }, (_, i) => asXL1BlockNumber(from + i, true))\n const total = numbers.length\n let publishedCount = 0\n await Promise.all(numbers.map(blockNumber => semaphore.runExclusive(async () => {\n assertEx(await this.publishBlock(blockNumber), () => `Block not found in source [${blockNumber}]`)\n publishedCount += 1\n if (publishedCount % this.progressInterval === 0 || publishedCount === total) {\n this.report({\n blockNumber, published: publishedCount, total, type: 'block',\n })\n }\n })))\n for (const blockNumber of numbers) {\n await this.publishStepsCompletedBy(blockNumber)\n }\n return [from, end]\n }\n\n private async putJson(path: string, json: string): Promise<void> {\n const { body, contentEncoding } = encodeBody(json, this.contentEncoding)\n await this.client.send(new PutObjectCommand({\n Bucket: this.bucket,\n Key: `${this.prefix}${path}`,\n Body: body,\n CacheControl: IMMUTABLE_CACHE_CONTROL,\n ContentEncoding: contentEncoding,\n ContentType: 'application/json',\n }))\n }\n\n /** Emits a progress event to the onProgress callback, or logs it when no callback is set. */\n private report(event: RestPublishEvent): void {\n const onProgress = this.params.onProgress\n if (onProgress) {\n onProgress(event)\n return\n }\n switch (event.type) {\n case 'block': {\n this.logger?.info(`RestBlockPublisher: published block ${event.blockNumber} (${event.published}/${event.total})`)\n break\n }\n case 'step': {\n this.logger?.info(`RestBlockPublisher: published step ${event.stepLevel}/${event.stepIndex}`)\n break\n }\n }\n }\n}\n"],
5
- "mappings": ";;;;;;;;;;;;AAAA,OAAO,UAAU;AAgBV,SAAS,WAAW,MAAc,UAA4C;AACnF,UAAQ,UAAU;AAAA,IAChB,KAAK,MAAM;AACT,aAAO,EAAE,MAAM,KAAK,mBAAmB,IAAI,GAAG,iBAAiB,KAAK;AAAA,IACtE;AAAA,IACA,KAAK,QAAQ;AACX,aAAO,EAAE,MAAM,KAAK,SAAS,IAAI,GAAG,iBAAiB,OAAO;AAAA,IAC9D;AAAA,IACA,KAAK,QAAQ;AACX,aAAO,EAAE,MAAM,IAAI,YAAY,EAAE,OAAO,IAAI,EAAE;AAAA,IAChD;AAAA,EACF;AACF;AAGO,SAAS,WAAW,MAAkB,iBAAkC;AAC7E,UAAQ,iBAAiB;AAAA,IACvB,KAAK,MAAM;AACT,aAAO,KAAK,qBAAqB,IAAI,EAAE,SAAS,MAAM;AAAA,IACxD;AAAA,IACA,KAAK,QAAQ;AACX,aAAO,KAAK,WAAW,IAAI,EAAE,SAAS,MAAM;AAAA,IAC9C;AAAA,IACA,SAAS;AACP,aAAO,IAAI,YAAY,EAAE,OAAO,IAAI;AAAA,IACtC;AAAA,EACF;AACF;;;AC1CA,SAAS,wBAAwB;AAEjC;AAAA,EACE;AAAA,EAAmB;AAAA,EAAU;AAAA,EAAW;AAAA,OACnC;AAIP,SAAS,kBAAkB,iBAAiB;AAE5C,SAAS,eAAe,+BAA+B;AACvD;AAAA,EACE;AAAA,EAAe;AAAA,EAAiB;AAAA,EAAgB;AAAA,OAC3C;AACP,SAAS,iBAAiB;AAkC1B,IAAM,0BAA0B;AAgBzB,IAAM,qBAAN,cAAiC,kBAA4C;AAAA,EAClF,IAAI,SAAiB;AACnB,WAAO,SAAS,KAAK,OAAO,QAAQ,MAAM,qBAAqB;AAAA,EACjE;AAAA,EAEA,IAAI,SAAmB;AACrB,WAAO,SAAS,KAAK,OAAO,QAAQ,MAAM,qBAAqB;AAAA,EACjE;AAAA,EAEA,IAAI,cAAsB;AACxB,WAAO,KAAK,OAAO,eAAe;AAAA,EACpC;AAAA,EAEA,IAAI,kBAAuC;AACzC,WAAO,KAAK,OAAO,mBAAmB;AAAA,EACxC;AAAA,EAEA,IAAI,SAAiB;AACnB,WAAO,KAAK,OAAO,UAAU;AAAA,EAC/B;AAAA,EAEA,IAAI,mBAA2B;AAC7B,WAAO,KAAK,OAAO,oBAAoB;AAAA,EACzC;AAAA,EAEA,IAAI,SAAsB;AACxB,WAAO,SAAS,KAAK,OAAO,QAAQ,MAAM,qBAAqB;AAAA,EACjE;AAAA;AAAA,EAGA,MAAM,aAAa,aAA8E;AAC/F,UAAM,QAAQ,MAAM,KAAK,OAAO,cAAc,WAAW;AACzD,QAAI,OAAO,KAAK,EAAG,QAAO;AAC1B,UAAM,OAAO,KAAK,UAAU,KAAK;AACjC,UAAM,KAAK,QAAQ,gBAAgB,MAAM,CAAC,EAAE,KAAK,GAAG,IAAI;AACxD,UAAM,KAAK,QAAQ,cAAc,MAAM,CAAC,EAAE,KAAK,GAAG,IAAI;AACtD,eAAW,WAAW,MAAM,CAAC,GAAG;AAC9B,YAAM,KAAK,QAAQ,YAAY,QAAQ,KAAK,GAAG,KAAK,UAAU,OAAO,CAAC;AAAA,IACxE;AACA,WAAO;AAAA,EACT;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,MAAM,YAAY,WAAmB,WAAkC;AACrE,UAAM,OAAO,UAAU,SAAS;AAChC,aAAS,aAAa,eAAe,MAAM,kDAAkD,aAAa,eAAe,SAAS,GAAG;AACrI,UAAM,mBAAmB,YAAY,KAAK,OAAO;AACjD,UAAM,aAAa,MAAM,KAAK,OAAO,mBAAmB;AACxD,aAAS,mBAAmB,YAAY,MAAM,QAAQ,SAAS,IAAI,SAAS,0BAA0B,UAAU,GAAG;AACnH,UAAM,cAAc,MAAM,KAAK,OAAO,aAAa,WAAW,SAAS;AACvE,UAAM,SAAS,YAAY,WAAW;AACtC,UAAM,UAA6B;AAAA,MACjC,QAAQ;AAAA,MACR,MAAM,SAAS,OAAO,GAAG,EAAE,GAAG,MAAM,sBAAsB,SAAS,IAAI,SAAS,EAAE,EAAE,CAAC,EAAE;AAAA,MACvF,UAAU;AAAA,MACV;AAAA,IACF;AACA,UAAM,KAAK,QAAQ,eAAe,WAAW,SAAS,GAAG,KAAK,UAAU,OAAO,CAAC;AAChF,SAAK,OAAO;AAAA,MACV;AAAA,MAAW;AAAA,MAAW,MAAM;AAAA,IAC9B,CAAC;AAAA,EACH;AAAA;AAAA,EAGA,MAAM,wBAAwB,aAA4C;AACxE,aAAS,YAAY,GAAG,aAAa,eAAe,aAAa;AAC/D,YAAM,OAAO,UAAU,SAAS;AAChC,WAAK,cAAc,KAAK,SAAS,GAAG;AAClC,cAAM,KAAK,YAAY,YAAY,cAAc,KAAK,OAAO,CAAC;AAAA,MAChE;AAAA,IACF;AAAA,EACF;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAWA,MAAM,KAAK,OAAuB,iBAAiB,GAAG,IAAI,GAAkC;AAC1F,UAAM,aAAa,MAAM,KAAK,OAAO,aAAa;AAClD,UAAM,MAAM,WAAW,CAAC,EAAE;AAC1B,QAAI,OAAO,IAAK,QAAO;AAEvB,UAAM,YAAY,IAAI,UAAU,KAAK,WAAW;AAChD,UAAM,UAAU,MAAM,KAAK,EAAE,QAAQ,MAAM,OAAO,EAAE,GAAG,CAAC,GAAG,MAAM,iBAAiB,OAAO,GAAG,IAAI,CAAC;AACjG,UAAM,QAAQ,QAAQ;AACtB,QAAI,iBAAiB;AACrB,UAAM,QAAQ,IAAI,QAAQ,IAAI,iBAAe,UAAU,aAAa,YAAY;AAC9E,eAAS,MAAM,KAAK,aAAa,WAAW,GAAG,MAAM,8BAA8B,WAAW,GAAG;AACjG,wBAAkB;AAClB,UAAI,iBAAiB,KAAK,qBAAqB,KAAK,mBAAmB,OAAO;AAC5E,aAAK,OAAO;AAAA,UACV;AAAA,UAAa,WAAW;AAAA,UAAgB;AAAA,UAAO,MAAM;AAAA,QACvD,CAAC;AAAA,MACH;AAAA,IACF,CAAC,CAAC,CAAC;AACH,eAAW,eAAe,SAAS;AACjC,YAAM,KAAK,wBAAwB,WAAW;AAAA,IAChD;AACA,WAAO,CAAC,MAAM,GAAG;AAAA,EACnB;AAAA,EAEA,MAAc,QAAQ,MAAc,MAA6B;AAC/D,UAAM,EAAE,MAAM,gBAAgB,IAAI,WAAW,MAAM,KAAK,eAAe;AACvE,UAAM,KAAK,OAAO,KAAK,IAAI,iBAAiB;AAAA,MAC1C,QAAQ,KAAK;AAAA,MACb,KAAK,GAAG,KAAK,MAAM,GAAG,IAAI;AAAA,MAC1B,MAAM;AAAA,MACN,cAAc;AAAA,MACd,iBAAiB;AAAA,MACjB,aAAa;AAAA,IACf,CAAC,CAAC;AAAA,EACJ;AAAA;AAAA,EAGQ,OAAO,OAA+B;AAC5C,UAAM,aAAa,KAAK,OAAO;AAC/B,QAAI,YAAY;AACd,iBAAW,KAAK;AAChB;AAAA,IACF;AACA,YAAQ,MAAM,MAAM;AAAA,MAClB,KAAK,SAAS;AACZ,aAAK,QAAQ,KAAK,uCAAuC,MAAM,WAAW,KAAK,MAAM,SAAS,IAAI,MAAM,KAAK,GAAG;AAChH;AAAA,MACF;AAAA,MACA,KAAK,QAAQ;AACX,aAAK,QAAQ,KAAK,sCAAsC,MAAM,SAAS,IAAI,MAAM,SAAS,EAAE;AAC5F;AAAA,MACF;AAAA,IACF;AAAA,EACF;AACF;AA3Ia,qBAAN;AAAA,EADN,UAAU;AAAA,GACE;",
6
- "names": []
3
+ "sources": ["../../src/AbstractS3Provider.ts", "../../src/encoding.ts", "../../src/S3BlockPublishRunner.ts", "../../src/S3ChainStatePublishRunner.ts", "../../src/S3ChainStateViewer.ts", "../../src/S3IndexPublishRunner.ts"],
4
+ "sourcesContent": ["import type { S3Client } from '@aws-sdk/client-s3'\nimport { GetObjectCommand, PutObjectCommand } from '@aws-sdk/client-s3'\nimport { assertEx } from '@xylabs/sdk-js'\nimport type { CreatableProviderEventData, CreatableProviderParams } from '@xyo-network/xl1-protocol-sdk'\nimport { AbstractCreatableProvider } from '@xyo-network/xl1-protocol-sdk'\n\nimport type { RestContentEncoding } from './encoding.ts'\nimport { decodeBody, encodeBody } from './encoding.ts'\n\n/** Shared parameters for S3-backed providers: where and how files are stored. */\nexport interface S3ProviderParams extends CreatableProviderParams {\n bucket: string\n /** S3-compatible client pointed at the target endpoint (e.g. Cloudflare R2). */\n client: S3Client\n /** Storage/wire encoding for published files. Defaults to 'br' (brotli). */\n contentEncoding?: RestContentEncoding\n /** Optional key prefix, allowing the layout to live in a shared bucket. */\n prefix?: string\n}\n\n/** Finalized files never change once written. */\nexport const IMMUTABLE_CACHE_CONTROL = 'public, max-age=31536000, immutable'\n/** Chain state is rewritten as the chain advances. */\nexport const MUTABLE_CACHE_CONTROL = 'public, max-age=0, must-revalidate'\n\n/**\n * Base for providers targeting S3-compatible storage. Bodies are pre-compressed at write\n * time and stored with the matching Content-Encoding plus application/json, so HTTP clients\n * decompress transparently \u2014 keys keep their `.json` extension. Reads decode the stored\n * Content-Encoding back to JSON.\n */\nexport abstract class AbstractS3Provider<\n TParams extends S3ProviderParams = S3ProviderParams,\n TEventData extends CreatableProviderEventData = CreatableProviderEventData,\n> extends AbstractCreatableProvider<TParams, TEventData> {\n get bucket(): string {\n return assertEx(this.params.bucket, () => 'No bucket specified')\n }\n\n get client(): S3Client {\n return assertEx(this.params.client, () => 'No client specified')\n }\n\n get contentEncoding(): RestContentEncoding {\n return this.params.contentEncoding ?? 'br'\n }\n\n get prefix(): string {\n return this.params.prefix ?? ''\n }\n\n /** Reads and decodes a stored JSON file, or undefined when the key does not exist. */\n protected async getJson(path: string): Promise<unknown> {\n try {\n const response = await this.client.send(new GetObjectCommand({ Bucket: this.bucket, Key: `${this.prefix}${path}` }))\n const body = await response.Body?.transformToByteArray()\n if (body === undefined) return undefined\n return JSON.parse(decodeBody(body, response.ContentEncoding)) as unknown\n } catch (error) {\n if (isNotFoundError(error)) return undefined\n throw error\n }\n }\n\n protected async putJson(path: string, json: string, cacheControl: string): Promise<void> {\n const { body, contentEncoding } = encodeBody(json, this.contentEncoding)\n await this.client.send(new PutObjectCommand({\n Bucket: this.bucket,\n Key: `${this.prefix}${path}`,\n Body: body,\n CacheControl: cacheControl,\n ContentEncoding: contentEncoding,\n ContentType: 'application/json',\n }))\n }\n}\n\nconst isNotFoundError = (error: unknown): boolean => {\n if (typeof error !== 'object' || error === null) return false\n const { name, $metadata } = error as { $metadata?: { httpStatusCode?: number }; name?: string }\n return name === 'NoSuchKey' || name === 'NotFound' || $metadata?.httpStatusCode === 404\n}\n", "import ZLIB from 'node:zlib'\n\n/** Wire/storage encoding for published files. */\nexport type RestContentEncoding = 'br' | 'gzip' | 'none'\n\n/** An encoded file body plus the HTTP Content-Encoding it should be served with. */\nexport interface EncodedBody {\n body: Uint8Array\n contentEncoding?: 'br' | 'gzip'\n}\n\n/**\n * Encodes a JSON string for storage. Bodies are pre-compressed at write time and stored\n * with the matching Content-Encoding so HTTP clients decompress transparently \u2014 keys keep\n * their .json extension and Content-Type stays application/json.\n */\nexport function encodeBody(json: string, encoding: RestContentEncoding): EncodedBody {\n switch (encoding) {\n case 'br': {\n return { body: ZLIB.brotliCompressSync(json), contentEncoding: 'br' }\n }\n case 'gzip': {\n return { body: ZLIB.gzipSync(json), contentEncoding: 'gzip' }\n }\n case 'none': {\n return { body: new TextEncoder().encode(json) }\n }\n }\n}\n\n/** Decodes a stored body back to its JSON string using the object's Content-Encoding. */\nexport function decodeBody(body: Uint8Array, contentEncoding?: string): string {\n switch (contentEncoding) {\n case 'br': {\n return ZLIB.brotliDecompressSync(body).toString('utf8')\n }\n case 'gzip': {\n return ZLIB.gunzipSync(body).toString('utf8')\n }\n default: {\n return new TextDecoder().decode(body)\n }\n }\n}\n", "import { assertEx, isNull } from '@xylabs/sdk-js'\nimport type {\n BlockPublishRunner, BlockViewer, SignedHydratedBlockWithHashMeta, XL1BlockNumber, XL1BlockRange,\n} from '@xyo-network/xl1-protocol-lib'\nimport {\n asXL1BlockNumber, BlockPublishRunnerMoniker, BlockViewerMoniker,\n} from '@xyo-network/xl1-protocol-lib'\nimport { creatableProvider } from '@xyo-network/xl1-protocol-sdk'\nimport {\n blockHashPath, blockNumberPath, payloadPath,\n} from '@xyo-network/xl1-rest-block-viewer'\nimport { Semaphore } from 'async-mutex'\n\nimport type { S3ProviderParams } from './AbstractS3Provider.ts'\nimport { AbstractS3Provider, IMMUTABLE_CACHE_CONTROL } from './AbstractS3Provider.ts'\n\n/** A progress event emitted while publishing blocks. */\nexport interface BlockPublishProgress {\n blockNumber: XL1BlockNumber\n published: number\n total: number\n}\n\n/** Parameters for S3BlockPublishRunner. */\nexport interface S3BlockPublishRunnerParams extends S3ProviderParams {\n /** Maximum concurrent block publishes during sync. */\n concurrency?: number\n /**\n * Called every `progressInterval` blocks (and at pass completion). When omitted, the same\n * events are logged via the params logger so long backfills are observable out of the box.\n */\n onProgress?: (event: BlockPublishProgress) => void\n /** How many block publishes between progress reports during sync. */\n progressInterval?: number\n /**\n * The viewer to publish from (only finalized chains should be published). When omitted,\n * resolved from the locator via BlockViewerMoniker.\n */\n source?: BlockViewer\n}\n\n/**\n * Publishes finalized blocks (and their payloads) as the static REST file layout that\n * `RestBlockViewer` reads (see `paths.ts` in @xyo-network/xl1-rest-block-viewer).\n *\n * Publishing is idempotent: keys are deterministic and contents immutable, so re-publishing\n * any range is safe. `sync(from)` resumes from a caller-supplied cursor (typically the block\n * after the last published head); omitting it republishes from genesis, which is safe.\n */\n@creatableProvider()\nexport class S3BlockPublishRunner extends AbstractS3Provider<S3BlockPublishRunnerParams> implements BlockPublishRunner {\n static readonly defaultMoniker = BlockPublishRunnerMoniker\n static readonly dependencies = [BlockViewerMoniker]\n static readonly monikers = [BlockPublishRunnerMoniker]\n moniker = S3BlockPublishRunner.defaultMoniker\n\n protected _source?: BlockViewer\n\n get concurrency(): number {\n return this.params.concurrency ?? 8\n }\n\n get progressInterval(): number {\n return this.params.progressInterval ?? 100\n }\n\n get source(): BlockViewer {\n return assertEx(this._source, () => 'No source specified')\n }\n\n override async createHandler() {\n await super.createHandler()\n this._source = this.params.source ?? await this.locator.getInstance(BlockViewerMoniker)\n }\n\n /** Publishes one block at its by-number and by-hash paths, plus its payload files. */\n async publishBlock(blockNumber: XL1BlockNumber): Promise<SignedHydratedBlockWithHashMeta | null> {\n const block = await this.source.blockByNumber(blockNumber)\n if (isNull(block)) return null\n const json = JSON.stringify(block)\n await this.putJson(blockNumberPath(block[0].block), json, IMMUTABLE_CACHE_CONTROL)\n await this.putJson(blockHashPath(block[0]._hash), json, IMMUTABLE_CACHE_CONTROL)\n for (const payload of block[1]) {\n await this.putJson(payloadPath(payload._hash), JSON.stringify(payload), IMMUTABLE_CACHE_CONTROL)\n }\n return block\n }\n\n /**\n * Publishes every block from `from` (inclusive, default genesis) through the source head.\n * Returns the published range, or null when already up to date.\n */\n async sync(from: XL1BlockNumber = asXL1BlockNumber(0, true)): Promise<XL1BlockRange | null> {\n const sourceHead = await this.source.currentBlock()\n const end = sourceHead[0].block\n if (from > end) return null\n\n const semaphore = new Semaphore(this.concurrency)\n const numbers = Array.from({ length: end - from + 1 }, (_, i) => asXL1BlockNumber(from + i, true))\n const total = numbers.length\n let publishedCount = 0\n await Promise.all(numbers.map(blockNumber => semaphore.runExclusive(async () => {\n assertEx(await this.publishBlock(blockNumber), () => `Block not found in source [${blockNumber}]`)\n publishedCount += 1\n if (publishedCount % this.progressInterval === 0 || publishedCount === total) {\n this.report({\n blockNumber, published: publishedCount, total,\n })\n }\n })))\n return [from, end]\n }\n\n /** Emits a progress event to the onProgress callback, or logs it when no callback is set. */\n private report(event: BlockPublishProgress): void {\n const onProgress = this.params.onProgress\n if (onProgress) {\n onProgress(event)\n return\n }\n this.logger?.info(`S3BlockPublishRunner: published block ${event.blockNumber} (${event.published}/${event.total})`)\n }\n}\n", "import { assertEx } from '@xylabs/sdk-js'\nimport type {\n BlockViewer,\n ChainStatePublishRunner, ChainStateViewerMethods, SignedHydratedBlockWithHashMeta,\n} from '@xyo-network/xl1-protocol-lib'\nimport { BlockViewerMoniker, ChainStatePublishRunnerMoniker } from '@xyo-network/xl1-protocol-lib'\nimport { creatableProvider } from '@xyo-network/xl1-protocol-sdk'\nimport { headPath } from '@xyo-network/xl1-rest-block-viewer'\n\nimport type { S3ProviderParams } from './AbstractS3Provider.ts'\nimport { AbstractS3Provider, MUTABLE_CACHE_CONTROL } from './AbstractS3Provider.ts'\n\n/** Parameters for S3ChainStatePublishRunner. */\nexport interface S3ChainStatePublishRunnerParams extends S3ProviderParams {\n /**\n * Where the current head is read from (any BlockViewer satisfies this). When omitted,\n * resolved from the locator via BlockViewerMoniker.\n */\n source?: ChainStateViewerMethods\n}\n\n/**\n * Publishes the mutable chain state \u2014 the head pointer (`chain/head.json`) \u2014 that\n * `RestChainStateViewer` reads. This is the one mutable file in the REST layout, so it is\n * stored with must-revalidate caching and should target the chain-state bucket.\n *\n * Callers must publish the finalized files for a head before publishing the head itself,\n * so readers never see a head that references missing files.\n */\n@creatableProvider()\nexport class S3ChainStatePublishRunner extends AbstractS3Provider<S3ChainStatePublishRunnerParams> implements ChainStatePublishRunner {\n static readonly defaultMoniker = ChainStatePublishRunnerMoniker\n static readonly dependencies = [BlockViewerMoniker]\n static readonly monikers = [ChainStatePublishRunnerMoniker]\n moniker = S3ChainStatePublishRunner.defaultMoniker\n\n protected _source?: ChainStateViewerMethods\n\n get source(): ChainStateViewerMethods {\n return assertEx(this._source, () => 'No source specified')\n }\n\n override async createHandler() {\n await super.createHandler()\n this._source = this.params.source ?? await this.locator.getInstance<BlockViewer>(BlockViewerMoniker)\n }\n\n /** Publishes the source's current head to the mutable chain state pointer. */\n async publishHead(): Promise<SignedHydratedBlockWithHashMeta> {\n const head = await this.source.currentBlock()\n await this.putJson(headPath(), JSON.stringify(head), MUTABLE_CACHE_CONTROL)\n this.logger?.info(`S3ChainStatePublishRunner: published head ${head[0].block}`)\n return head\n }\n}\n", "import type { Hash } from '@xylabs/sdk-js'\nimport { assertEx } from '@xylabs/sdk-js'\nimport type {\n ChainStateViewer, SignedHydratedBlockWithHashMeta, XL1BlockNumber,\n} from '@xyo-network/xl1-protocol-lib'\nimport { asSignedHydratedBlockWithHashMeta, ChainStateViewerMoniker } from '@xyo-network/xl1-protocol-lib'\nimport { creatableProvider } from '@xyo-network/xl1-protocol-sdk'\nimport { headPath } from '@xyo-network/xl1-rest-block-viewer'\n\nimport type { S3ProviderParams } from './AbstractS3Provider.ts'\nimport { AbstractS3Provider } from './AbstractS3Provider.ts'\n\n/** Parameters for S3ChainStateViewer. */\nexport interface S3ChainStateViewerParams extends S3ProviderParams {}\n\n/**\n * A ChainStateViewer over the chain-state bucket, reading the published head pointer back\n * through the S3 API (authenticated, not the public HTTP layout \u2014 see RestChainStateViewer\n * for anonymous reads). This is how publishers learn their resume cursor: the head pointer\n * is only written after the finalized files it references exist.\n */\n@creatableProvider()\nexport class S3ChainStateViewer extends AbstractS3Provider<S3ChainStateViewerParams, ChainStateViewer['eventData']>\n implements ChainStateViewer {\n static readonly defaultMoniker = ChainStateViewerMoniker\n static readonly dependencies = []\n static readonly monikers = [ChainStateViewerMoniker]\n moniker = S3ChainStateViewer.defaultMoniker\n\n /** The published head; throws when no head pointer has been published yet. */\n async currentBlock(): Promise<SignedHydratedBlockWithHashMeta> {\n return await this.spanAsync('currentBlock', async () => {\n const parsed = assertEx(await this.getJson(headPath()), () => 'Head not found')\n return asSignedHydratedBlockWithHashMeta(parsed, true)\n }, this.context)\n }\n\n async currentBlockHash(): Promise<Hash> {\n return (await this.currentBlock())[0]._hash\n }\n\n async currentBlockNumber(): Promise<XL1BlockNumber> {\n return (await this.currentBlock())[0].block\n }\n\n /** The published head, or undefined when no head pointer has been published yet. */\n async tryCurrentBlock(): Promise<SignedHydratedBlockWithHashMeta | undefined> {\n const parsed = await this.getJson(headPath())\n return parsed === undefined ? undefined : asSignedHydratedBlockWithHashMeta(parsed, true)\n }\n}\n", "import { assertEx } from '@xylabs/sdk-js'\nimport type {\n BlocksStepSummary, BlockViewer, IndexPublishRunner, XL1BlockNumber,\n} from '@xyo-network/xl1-protocol-lib'\nimport {\n BlocksStepSummarySchema, BlockViewerMoniker, IndexPublishRunnerMoniker, StepSizes,\n} from '@xyo-network/xl1-protocol-lib'\nimport { blocksMaxStep, creatableProvider } from '@xyo-network/xl1-protocol-sdk'\nimport { blocksStepPath } from '@xyo-network/xl1-rest-block-viewer'\n\nimport type { S3ProviderParams } from './AbstractS3Provider.ts'\nimport { AbstractS3Provider, IMMUTABLE_CACHE_CONTROL } from './AbstractS3Provider.ts'\n\n/** Parameters for S3IndexPublishRunner. */\nexport interface S3IndexPublishRunnerParams extends S3ProviderParams {\n /**\n * The viewer to publish from (only finalized chains should be published). When omitted,\n * resolved from the locator via BlockViewerMoniker.\n */\n source?: BlockViewer\n}\n\n/**\n * Publishes completed steps' block summaries (the chain index) as the static REST file\n * layout that `RestIndexViewer` reads. Step files are immutable: only complete steps are\n * ever published, and a completed step's blocks never change.\n */\n@creatableProvider()\nexport class S3IndexPublishRunner extends AbstractS3Provider<S3IndexPublishRunnerParams> implements IndexPublishRunner {\n static readonly defaultMoniker = IndexPublishRunnerMoniker\n static readonly dependencies = [BlockViewerMoniker]\n static readonly monikers = [IndexPublishRunnerMoniker]\n moniker = S3IndexPublishRunner.defaultMoniker\n\n protected _source?: BlockViewer\n\n get source(): BlockViewer {\n return assertEx(this._source, () => 'No source specified')\n }\n\n override async createHandler() {\n await super.createHandler()\n this._source = this.params.source ?? await this.locator.getInstance(BlockViewerMoniker)\n }\n\n /**\n * Publishes one completed step's blocks as a BlocksStepSummary file (blocks oldest-first).\n * Asserts the step is complete \u2014 partial tail steps are never published.\n */\n async publishStep(stepLevel: number, stepIndex: number): Promise<void> {\n const size = StepSizes[stepLevel]\n assertEx(stepLevel <= blocksMaxStep, () => `publishStep does not support step levels above ${blocksMaxStep} (requested ${stepLevel})`)\n const lastBlockNumber = (stepIndex + 1) * size - 1\n const headNumber = await this.source.currentBlockNumber()\n assertEx(lastBlockNumber <= headNumber, () => `Step ${stepLevel}/${stepIndex} is not complete (head ${headNumber})`)\n const newestFirst = await this.source.blocksByStep(stepLevel, stepIndex)\n const blocks = newestFirst.toReversed()\n const summary: BlocksStepSummary = {\n schema: BlocksStepSummarySchema,\n hash: assertEx(blocks.at(-1), () => `No blocks for step ${stepLevel}/${stepIndex}`)[0]._hash,\n stepSize: size,\n blocks,\n }\n await this.putJson(blocksStepPath(stepLevel, stepIndex), JSON.stringify(summary), IMMUTABLE_CACHE_CONTROL)\n this.logger?.info(`S3IndexPublishRunner: published step ${stepLevel}/${stepIndex}`)\n }\n\n /** Publishes every step (at every level) completed by the given block. */\n async publishStepsCompletedBy(blockNumber: XL1BlockNumber): Promise<void> {\n for (let stepLevel = 0; stepLevel <= blocksMaxStep; stepLevel++) {\n const size = StepSizes[stepLevel]\n if ((blockNumber + 1) % size === 0) {\n await this.publishStep(stepLevel, (blockNumber + 1) / size - 1)\n }\n }\n }\n}\n"],
5
+ "mappings": ";;;;;;;;;;;;;;AACA,SAAS,kBAAkB,wBAAwB;AACnD,SAAS,gBAAgB;AAEzB,SAAS,iCAAiC;;;ACJ1C,OAAO,UAAU;AAgBV,SAAS,WAAW,MAAc,UAA4C;AACnF,UAAQ,UAAU;AAAA,IAChB,KAAK,MAAM;AACT,aAAO,EAAE,MAAM,KAAK,mBAAmB,IAAI,GAAG,iBAAiB,KAAK;AAAA,IACtE;AAAA,IACA,KAAK,QAAQ;AACX,aAAO,EAAE,MAAM,KAAK,SAAS,IAAI,GAAG,iBAAiB,OAAO;AAAA,IAC9D;AAAA,IACA,KAAK,QAAQ;AACX,aAAO,EAAE,MAAM,IAAI,YAAY,EAAE,OAAO,IAAI,EAAE;AAAA,IAChD;AAAA,EACF;AACF;AAGO,SAAS,WAAW,MAAkB,iBAAkC;AAC7E,UAAQ,iBAAiB;AAAA,IACvB,KAAK,MAAM;AACT,aAAO,KAAK,qBAAqB,IAAI,EAAE,SAAS,MAAM;AAAA,IACxD;AAAA,IACA,KAAK,QAAQ;AACX,aAAO,KAAK,WAAW,IAAI,EAAE,SAAS,MAAM;AAAA,IAC9C;AAAA,IACA,SAAS;AACP,aAAO,IAAI,YAAY,EAAE,OAAO,IAAI;AAAA,IACtC;AAAA,EACF;AACF;;;ADtBO,IAAM,0BAA0B;AAEhC,IAAM,wBAAwB;AAQ9B,IAAe,qBAAf,cAGG,0BAA+C;AAAA,EACvD,IAAI,SAAiB;AACnB,WAAO,SAAS,KAAK,OAAO,QAAQ,MAAM,qBAAqB;AAAA,EACjE;AAAA,EAEA,IAAI,SAAmB;AACrB,WAAO,SAAS,KAAK,OAAO,QAAQ,MAAM,qBAAqB;AAAA,EACjE;AAAA,EAEA,IAAI,kBAAuC;AACzC,WAAO,KAAK,OAAO,mBAAmB;AAAA,EACxC;AAAA,EAEA,IAAI,SAAiB;AACnB,WAAO,KAAK,OAAO,UAAU;AAAA,EAC/B;AAAA;AAAA,EAGA,MAAgB,QAAQ,MAAgC;AACtD,QAAI;AACF,YAAM,WAAW,MAAM,KAAK,OAAO,KAAK,IAAI,iBAAiB,EAAE,QAAQ,KAAK,QAAQ,KAAK,GAAG,KAAK,MAAM,GAAG,IAAI,GAAG,CAAC,CAAC;AACnH,YAAM,OAAO,MAAM,SAAS,MAAM,qBAAqB;AACvD,UAAI,SAAS,OAAW,QAAO;AAC/B,aAAO,KAAK,MAAM,WAAW,MAAM,SAAS,eAAe,CAAC;AAAA,IAC9D,SAAS,OAAO;AACd,UAAI,gBAAgB,KAAK,EAAG,QAAO;AACnC,YAAM;AAAA,IACR;AAAA,EACF;AAAA,EAEA,MAAgB,QAAQ,MAAc,MAAc,cAAqC;AACvF,UAAM,EAAE,MAAM,gBAAgB,IAAI,WAAW,MAAM,KAAK,eAAe;AACvE,UAAM,KAAK,OAAO,KAAK,IAAI,iBAAiB;AAAA,MAC1C,QAAQ,KAAK;AAAA,MACb,KAAK,GAAG,KAAK,MAAM,GAAG,IAAI;AAAA,MAC1B,MAAM;AAAA,MACN,cAAc;AAAA,MACd,iBAAiB;AAAA,MACjB,aAAa;AAAA,IACf,CAAC,CAAC;AAAA,EACJ;AACF;AAEA,IAAM,kBAAkB,CAAC,UAA4B;AACnD,MAAI,OAAO,UAAU,YAAY,UAAU,KAAM,QAAO;AACxD,QAAM,EAAE,MAAM,UAAU,IAAI;AAC5B,SAAO,SAAS,eAAe,SAAS,cAAc,WAAW,mBAAmB;AACtF;;;AEjFA,SAAS,YAAAA,WAAU,cAAc;AAIjC;AAAA,EACE;AAAA,EAAkB;AAAA,EAA2B;AAAA,OACxC;AACP,SAAS,yBAAyB;AAClC;AAAA,EACE;AAAA,EAAe;AAAA,EAAiB;AAAA,OAC3B;AACP,SAAS,iBAAiB;AAuCnB,IAAM,uBAAN,cAAmC,mBAA6E;AAAA,EAIrH,UAAU,qBAAqB;AAAA,EAErB;AAAA,EAEV,IAAI,cAAsB;AACxB,WAAO,KAAK,OAAO,eAAe;AAAA,EACpC;AAAA,EAEA,IAAI,mBAA2B;AAC7B,WAAO,KAAK,OAAO,oBAAoB;AAAA,EACzC;AAAA,EAEA,IAAI,SAAsB;AACxB,WAAOC,UAAS,KAAK,SAAS,MAAM,qBAAqB;AAAA,EAC3D;AAAA,EAEA,MAAe,gBAAgB;AAC7B,UAAM,MAAM,cAAc;AAC1B,SAAK,UAAU,KAAK,OAAO,UAAU,MAAM,KAAK,QAAQ,YAAY,kBAAkB;AAAA,EACxF;AAAA;AAAA,EAGA,MAAM,aAAa,aAA8E;AAC/F,UAAM,QAAQ,MAAM,KAAK,OAAO,cAAc,WAAW;AACzD,QAAI,OAAO,KAAK,EAAG,QAAO;AAC1B,UAAM,OAAO,KAAK,UAAU,KAAK;AACjC,UAAM,KAAK,QAAQ,gBAAgB,MAAM,CAAC,EAAE,KAAK,GAAG,MAAM,uBAAuB;AACjF,UAAM,KAAK,QAAQ,cAAc,MAAM,CAAC,EAAE,KAAK,GAAG,MAAM,uBAAuB;AAC/E,eAAW,WAAW,MAAM,CAAC,GAAG;AAC9B,YAAM,KAAK,QAAQ,YAAY,QAAQ,KAAK,GAAG,KAAK,UAAU,OAAO,GAAG,uBAAuB;AAAA,IACjG;AACA,WAAO;AAAA,EACT;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,MAAM,KAAK,OAAuB,iBAAiB,GAAG,IAAI,GAAkC;AAC1F,UAAM,aAAa,MAAM,KAAK,OAAO,aAAa;AAClD,UAAM,MAAM,WAAW,CAAC,EAAE;AAC1B,QAAI,OAAO,IAAK,QAAO;AAEvB,UAAM,YAAY,IAAI,UAAU,KAAK,WAAW;AAChD,UAAM,UAAU,MAAM,KAAK,EAAE,QAAQ,MAAM,OAAO,EAAE,GAAG,CAAC,GAAG,MAAM,iBAAiB,OAAO,GAAG,IAAI,CAAC;AACjG,UAAM,QAAQ,QAAQ;AACtB,QAAI,iBAAiB;AACrB,UAAM,QAAQ,IAAI,QAAQ,IAAI,iBAAe,UAAU,aAAa,YAAY;AAC9E,MAAAA,UAAS,MAAM,KAAK,aAAa,WAAW,GAAG,MAAM,8BAA8B,WAAW,GAAG;AACjG,wBAAkB;AAClB,UAAI,iBAAiB,KAAK,qBAAqB,KAAK,mBAAmB,OAAO;AAC5E,aAAK,OAAO;AAAA,UACV;AAAA,UAAa,WAAW;AAAA,UAAgB;AAAA,QAC1C,CAAC;AAAA,MACH;AAAA,IACF,CAAC,CAAC,CAAC;AACH,WAAO,CAAC,MAAM,GAAG;AAAA,EACnB;AAAA;AAAA,EAGQ,OAAO,OAAmC;AAChD,UAAM,aAAa,KAAK,OAAO;AAC/B,QAAI,YAAY;AACd,iBAAW,KAAK;AAChB;AAAA,IACF;AACA,SAAK,QAAQ,KAAK,yCAAyC,MAAM,WAAW,KAAK,MAAM,SAAS,IAAI,MAAM,KAAK,GAAG;AAAA,EACpH;AACF;AAvEE,cADW,sBACK,kBAAiB;AACjC,cAFW,sBAEK,gBAAe,CAAC,kBAAkB;AAClD,cAHW,sBAGK,YAAW,CAAC,yBAAyB;AAH1C,uBAAN;AAAA,EADN,kBAAkB;AAAA,GACN;;;AClDb,SAAS,YAAAC,iBAAgB;AAKzB,SAAS,sBAAAC,qBAAoB,sCAAsC;AACnE,SAAS,qBAAAC,0BAAyB;AAClC,SAAS,gBAAgB;AAuBlB,IAAM,4BAAN,cAAwC,mBAAuF;AAAA,EAIpI,UAAU,0BAA0B;AAAA,EAE1B;AAAA,EAEV,IAAI,SAAkC;AACpC,WAAOC,UAAS,KAAK,SAAS,MAAM,qBAAqB;AAAA,EAC3D;AAAA,EAEA,MAAe,gBAAgB;AAC7B,UAAM,MAAM,cAAc;AAC1B,SAAK,UAAU,KAAK,OAAO,UAAU,MAAM,KAAK,QAAQ,YAAyBC,mBAAkB;AAAA,EACrG;AAAA;AAAA,EAGA,MAAM,cAAwD;AAC5D,UAAM,OAAO,MAAM,KAAK,OAAO,aAAa;AAC5C,UAAM,KAAK,QAAQ,SAAS,GAAG,KAAK,UAAU,IAAI,GAAG,qBAAqB;AAC1E,SAAK,QAAQ,KAAK,6CAA6C,KAAK,CAAC,EAAE,KAAK,EAAE;AAC9E,WAAO;AAAA,EACT;AACF;AAvBE,cADW,2BACK,kBAAiB;AACjC,cAFW,2BAEK,gBAAe,CAACA,mBAAkB;AAClD,cAHW,2BAGK,YAAW,CAAC,8BAA8B;AAH/C,4BAAN;AAAA,EADNC,mBAAkB;AAAA,GACN;;;AC7Bb,SAAS,YAAAC,iBAAgB;AAIzB,SAAS,mCAAmC,+BAA+B;AAC3E,SAAS,qBAAAC,0BAAyB;AAClC,SAAS,YAAAC,iBAAgB;AAelB,IAAM,qBAAN,cAAiC,mBACV;AAAA,EAI5B,UAAU,mBAAmB;AAAA;AAAA,EAG7B,MAAM,eAAyD;AAC7D,WAAO,MAAM,KAAK,UAAU,gBAAgB,YAAY;AACtD,YAAM,SAASC,UAAS,MAAM,KAAK,QAAQC,UAAS,CAAC,GAAG,MAAM,gBAAgB;AAC9E,aAAO,kCAAkC,QAAQ,IAAI;AAAA,IACvD,GAAG,KAAK,OAAO;AAAA,EACjB;AAAA,EAEA,MAAM,mBAAkC;AACtC,YAAQ,MAAM,KAAK,aAAa,GAAG,CAAC,EAAE;AAAA,EACxC;AAAA,EAEA,MAAM,qBAA8C;AAClD,YAAQ,MAAM,KAAK,aAAa,GAAG,CAAC,EAAE;AAAA,EACxC;AAAA;AAAA,EAGA,MAAM,kBAAwE;AAC5E,UAAM,SAAS,MAAM,KAAK,QAAQA,UAAS,CAAC;AAC5C,WAAO,WAAW,SAAY,SAAY,kCAAkC,QAAQ,IAAI;AAAA,EAC1F;AACF;AA1BE,cAFW,oBAEK,kBAAiB;AACjC,cAHW,oBAGK,gBAAe,CAAC;AAChC,cAJW,oBAIK,YAAW,CAAC,uBAAuB;AAJxC,qBAAN;AAAA,EADNC,mBAAkB;AAAA,GACN;;;ACtBb,SAAS,YAAAC,iBAAgB;AAIzB;AAAA,EACE;AAAA,EAAyB,sBAAAC;AAAA,EAAoB;AAAA,EAA2B;AAAA,OACnE;AACP,SAAS,eAAe,qBAAAC,0BAAyB;AACjD,SAAS,sBAAsB;AAoBxB,IAAM,uBAAN,cAAmC,mBAA6E;AAAA,EAIrH,UAAU,qBAAqB;AAAA,EAErB;AAAA,EAEV,IAAI,SAAsB;AACxB,WAAOC,UAAS,KAAK,SAAS,MAAM,qBAAqB;AAAA,EAC3D;AAAA,EAEA,MAAe,gBAAgB;AAC7B,UAAM,MAAM,cAAc;AAC1B,SAAK,UAAU,KAAK,OAAO,UAAU,MAAM,KAAK,QAAQ,YAAYC,mBAAkB;AAAA,EACxF;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,MAAM,YAAY,WAAmB,WAAkC;AACrE,UAAM,OAAO,UAAU,SAAS;AAChC,IAAAD,UAAS,aAAa,eAAe,MAAM,kDAAkD,aAAa,eAAe,SAAS,GAAG;AACrI,UAAM,mBAAmB,YAAY,KAAK,OAAO;AACjD,UAAM,aAAa,MAAM,KAAK,OAAO,mBAAmB;AACxD,IAAAA,UAAS,mBAAmB,YAAY,MAAM,QAAQ,SAAS,IAAI,SAAS,0BAA0B,UAAU,GAAG;AACnH,UAAM,cAAc,MAAM,KAAK,OAAO,aAAa,WAAW,SAAS;AACvE,UAAM,SAAS,YAAY,WAAW;AACtC,UAAM,UAA6B;AAAA,MACjC,QAAQ;AAAA,MACR,MAAMA,UAAS,OAAO,GAAG,EAAE,GAAG,MAAM,sBAAsB,SAAS,IAAI,SAAS,EAAE,EAAE,CAAC,EAAE;AAAA,MACvF,UAAU;AAAA,MACV;AAAA,IACF;AACA,UAAM,KAAK,QAAQ,eAAe,WAAW,SAAS,GAAG,KAAK,UAAU,OAAO,GAAG,uBAAuB;AACzG,SAAK,QAAQ,KAAK,wCAAwC,SAAS,IAAI,SAAS,EAAE;AAAA,EACpF;AAAA;AAAA,EAGA,MAAM,wBAAwB,aAA4C;AACxE,aAAS,YAAY,GAAG,aAAa,eAAe,aAAa;AAC/D,YAAM,OAAO,UAAU,SAAS;AAChC,WAAK,cAAc,KAAK,SAAS,GAAG;AAClC,cAAM,KAAK,YAAY,YAAY,cAAc,KAAK,OAAO,CAAC;AAAA,MAChE;AAAA,IACF;AAAA,EACF;AACF;AA/CE,cADW,sBACK,kBAAiB;AACjC,cAFW,sBAEK,gBAAe,CAACC,mBAAkB;AAClD,cAHW,sBAGK,YAAW,CAAC,yBAAyB;AAH1C,uBAAN;AAAA,EADNC,mBAAkB;AAAA,GACN;",
6
+ "names": ["assertEx", "assertEx", "assertEx", "BlockViewerMoniker", "creatableProvider", "assertEx", "BlockViewerMoniker", "creatableProvider", "assertEx", "creatableProvider", "headPath", "assertEx", "headPath", "creatableProvider", "assertEx", "BlockViewerMoniker", "creatableProvider", "assertEx", "BlockViewerMoniker", "creatableProvider"]
7
7
  }
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "$schema": "http://json.schemastore.org/package.json",
3
3
  "name": "@xyo-network/xl1-rest-block-publisher",
4
- "version": "2.1.2",
4
+ "version": "2.1.4",
5
5
  "description": "XYO Layer One static REST chain layout publisher",
6
6
  "homepage": "https://xylabs.com",
7
7
  "bugs": {
@@ -35,12 +35,12 @@
35
35
  "README.md"
36
36
  ],
37
37
  "dependencies": {
38
- "@xyo-network/xl1-protocol-lib": "~2.1.2",
39
- "@xyo-network/xl1-protocol-sdk": "~2.1.2",
40
- "@xyo-network/xl1-rest-block-viewer": "~2.1.2"
38
+ "@xyo-network/xl1-protocol-lib": "~2.1.4",
39
+ "@xyo-network/xl1-protocol-sdk": "~2.1.4",
40
+ "@xyo-network/xl1-rest-block-viewer": "~2.1.4"
41
41
  },
42
42
  "devDependencies": {
43
- "@aws-sdk/client-s3": "^3.1065.0",
43
+ "@aws-sdk/client-s3": "^3.1066.0",
44
44
  "@bitauth/libauth": "~3.0.0",
45
45
  "@metamask/providers": "^22.1.1",
46
46
  "@noble/post-quantum": "~0.6.1",
@@ -48,18 +48,16 @@
48
48
  "@opentelemetry/sdk-trace-base": "^2.7.1",
49
49
  "@scure/base": "~2.2.0",
50
50
  "@scure/bip39": "~2.2.0",
51
- "@xylabs/geo": "^6.1.0",
52
- "@xylabs/sdk-js": "^6.1.0",
53
- "@xylabs/threads": "~6.1",
51
+ "@xylabs/geo": "^6.1.1",
52
+ "@xylabs/sdk-js": "^6.1.1",
53
+ "@xylabs/threads": "^6.1.1",
54
54
  "@xylabs/toolchain": "~8.1.20",
55
55
  "@xylabs/tsconfig": "~8.1.20",
56
- "@xyo-network/address": "^6.0.9",
57
- "@xyo-network/sdk-js": "^6.0.4",
58
- "@xyo-network/sdk-protocol-js": "~6.0.9",
56
+ "@xyo-network/address": "^6.1.0",
57
+ "@xyo-network/sdk-js": "^6.1.0",
58
+ "@xyo-network/sdk-protocol-js": "~6.1",
59
59
  "ajv": "^8.20.0",
60
60
  "async-mutex": "^0.5.0",
61
- "bn.js": "^5.2.3",
62
- "buffer": "^6.0.3",
63
61
  "cosmiconfig": "^9.0.2",
64
62
  "debug": "~4.4.3",
65
63
  "eslint": "^10.4.1",
@@ -88,11 +86,9 @@
88
86
  "@xylabs/threads": "^6.1",
89
87
  "@xyo-network/address": "^6.0",
90
88
  "@xyo-network/sdk-js": "^6.0",
91
- "@xyo-network/sdk-protocol-js": "~6.0",
89
+ "@xyo-network/sdk-protocol-js": "~6.1",
92
90
  "ajv": "^8.20",
93
91
  "async-mutex": "^0.5",
94
- "bn.js": "^5.2",
95
- "buffer": "^6.0",
96
92
  "cosmiconfig": "^9.0",
97
93
  "debug": "~4.4",
98
94
  "ethers": "^6.16",
@@ -1,83 +0,0 @@
1
- import type { S3Client } from '@aws-sdk/client-s3';
2
- import type { CreatableParams } from '@xylabs/sdk-js';
3
- import { AbstractCreatable } from '@xylabs/sdk-js';
4
- import type { BlockViewer, SignedHydratedBlockWithHashMeta, XL1BlockNumber, XL1BlockRange } from '@xyo-network/xl1-protocol-lib';
5
- import type { RestContentEncoding } from './encoding.ts';
6
- /** A progress event emitted while publishing. */
7
- export type RestPublishEvent = {
8
- blockNumber: XL1BlockNumber;
9
- published: number;
10
- total: number;
11
- type: 'block';
12
- } | {
13
- stepIndex: number;
14
- stepLevel: number;
15
- type: 'step';
16
- };
17
- /** Parameters for RestBlockPublisher. */
18
- export interface RestBlockPublisherParams extends CreatableParams {
19
- bucket: string;
20
- /** S3-compatible client pointed at the target endpoint (e.g. Cloudflare R2). */
21
- client: S3Client;
22
- /** Maximum concurrent block publishes during sync. */
23
- concurrency?: number;
24
- /** Storage/wire encoding for published files. Defaults to 'br' (brotli). */
25
- contentEncoding?: RestContentEncoding;
26
- /**
27
- * Called as publishing progresses: every `progressInterval` blocks (and at pass completion)
28
- * and per step file. When omitted, the same events are logged via the params logger so
29
- * long backfills are observable out of the box.
30
- */
31
- onProgress?: (event: RestPublishEvent) => void;
32
- /** Optional key prefix, allowing the layout to live in a shared bucket. */
33
- prefix?: string;
34
- /** How many block publishes between progress reports during sync. */
35
- progressInterval?: number;
36
- /** The viewer to publish from (only finalized chains should be published). */
37
- source: BlockViewer;
38
- }
39
- /**
40
- * Publishes a chain's immutable files as the static REST file layout that `RestBlockViewer`
41
- * reads (see `paths.ts` in @xyo-network/xl1-rest-block-viewer).
42
- *
43
- * The mutable chain state (the head pointer, `chain/head.json`) is NOT written here — it
44
- * lives in a separate chain-state bucket and is written by the chain's finalizer after the
45
- * finalized files for that head exist, so readers never see a head that references missing
46
- * files.
47
- *
48
- * Publishing is idempotent: keys are deterministic and contents immutable, so re-publishing
49
- * any range is safe. `sync(from)` resumes from a caller-supplied cursor (typically the block
50
- * after the last published head); omitting it republishes from genesis, which is safe.
51
- */
52
- export declare class RestBlockPublisher extends AbstractCreatable<RestBlockPublisherParams> {
53
- get bucket(): string;
54
- get client(): S3Client;
55
- get concurrency(): number;
56
- get contentEncoding(): RestContentEncoding;
57
- get prefix(): string;
58
- get progressInterval(): number;
59
- get source(): BlockViewer;
60
- /** Publishes one block at its by-number and by-hash paths, plus its payload files. */
61
- publishBlock(blockNumber: XL1BlockNumber): Promise<SignedHydratedBlockWithHashMeta | null>;
62
- /**
63
- * Publishes one completed step's blocks as a BlocksStepSummary file (blocks oldest-first).
64
- * Asserts the step is complete — partial tail steps are never published.
65
- */
66
- publishStep(stepLevel: number, stepIndex: number): Promise<void>;
67
- /** Publishes every step (at every level) completed by the given block. */
68
- publishStepsCompletedBy(blockNumber: XL1BlockNumber): Promise<void>;
69
- /**
70
- * Publishes everything from `from` (inclusive, default genesis) to the source head
71
- * (inclusive), then the completed steps in that range. Returns the published range, or
72
- * null when already up to date.
73
- *
74
- * The caller supplies the resume cursor — typically the block after the last published
75
- * head pointer, which is written by the finalizer rather than this publisher. Omitting
76
- * `from` republishes from genesis, which is safe (idempotent) but slow.
77
- */
78
- sync(from?: XL1BlockNumber): Promise<XL1BlockRange | null>;
79
- private putJson;
80
- /** Emits a progress event to the onProgress callback, or logs it when no callback is set. */
81
- private report;
82
- }
83
- //# sourceMappingURL=RestBlockPublisher.d.ts.map
@@ -1 +0,0 @@
1
- {"version":3,"file":"RestBlockPublisher.d.ts","sourceRoot":"","sources":["../../src/RestBlockPublisher.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,oBAAoB,CAAA;AAElD,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAA;AACrD,OAAO,EACL,iBAAiB,EAClB,MAAM,gBAAgB,CAAA;AACvB,OAAO,KAAK,EACV,WAAW,EAAE,+BAA+B,EAAE,cAAc,EAAE,aAAa,EAC5E,MAAM,+BAA+B,CAAA;AAStC,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,eAAe,CAAA;AAGxD,iDAAiD;AACjD,MAAM,MAAM,gBAAgB,GACtB;IAAE,WAAW,EAAE,cAAc,CAAC;IAAC,SAAS,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,OAAO,CAAA;CAAE,GAChF;IAAE,SAAS,EAAE,MAAM,CAAC;IAAC,SAAS,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,CAAA;AAE5D,yCAAyC;AACzC,MAAM,WAAW,wBAAyB,SAAQ,eAAe;IAC/D,MAAM,EAAE,MAAM,CAAA;IACd,gFAAgF;IAChF,MAAM,EAAE,QAAQ,CAAA;IAChB,sDAAsD;IACtD,WAAW,CAAC,EAAE,MAAM,CAAA;IACpB,4EAA4E;IAC5E,eAAe,CAAC,EAAE,mBAAmB,CAAA;IACrC;;;;OAIG;IACH,UAAU,CAAC,EAAE,CAAC,KAAK,EAAE,gBAAgB,KAAK,IAAI,CAAA;IAC9C,2EAA2E;IAC3E,MAAM,CAAC,EAAE,MAAM,CAAA;IACf,qEAAqE;IACrE,gBAAgB,CAAC,EAAE,MAAM,CAAA;IACzB,8EAA8E;IAC9E,MAAM,EAAE,WAAW,CAAA;CACpB;AAKD;;;;;;;;;;;;GAYG;AACH,qBACa,kBAAmB,SAAQ,iBAAiB,CAAC,wBAAwB,CAAC;IACjF,IAAI,MAAM,IAAI,MAAM,CAEnB;IAED,IAAI,MAAM,IAAI,QAAQ,CAErB;IAED,IAAI,WAAW,IAAI,MAAM,CAExB;IAED,IAAI,eAAe,IAAI,mBAAmB,CAEzC;IAED,IAAI,MAAM,IAAI,MAAM,CAEnB;IAED,IAAI,gBAAgB,IAAI,MAAM,CAE7B;IAED,IAAI,MAAM,IAAI,WAAW,CAExB;IAED,sFAAsF;IAChF,YAAY,CAAC,WAAW,EAAE,cAAc,GAAG,OAAO,CAAC,+BAA+B,GAAG,IAAI,CAAC;IAYhG;;;OAGG;IACG,WAAW,CAAC,SAAS,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAoBtE,0EAA0E;IACpE,uBAAuB,CAAC,WAAW,EAAE,cAAc,GAAG,OAAO,CAAC,IAAI,CAAC;IASzE;;;;;;;;OAQG;IACG,IAAI,CAAC,IAAI,GAAE,cAA0C,GAAG,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC;YAwB7E,OAAO;IAYrB,6FAA6F;IAC7F,OAAO,CAAC,MAAM;CAiBf"}