@bitsocial/pubsub-voting 0.0.6 → 0.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -7
- package/dist/client/voter.js +16 -1
- package/dist/transport/announce/node.d.ts +32 -5
- package/dist/transport/announce/node.js +65 -15
- package/package.json +2 -2
- package/dist/client/root-puller.d.ts +0 -49
- package/dist/client/root-puller.js +0 -140
- package/dist/store/indexeddb.d.ts +0 -9
- package/dist/store/indexeddb.js +0 -72
- package/dist/store/memory.d.ts +0 -15
- package/dist/store/memory.js +0 -22
- package/dist/store/select.d.ts +0 -15
- package/dist/store/select.js +0 -64
- package/dist/store/sqlite.d.ts +0 -11
- package/dist/store/sqlite.js +0 -68
- package/dist/store/types.d.ts +0 -57
- package/dist/store/types.js +0 -1
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Trustless, leaderless voting over libp2p pubsub, designed to run on top of a host node's shared libp2p/Helia instance.
|
|
4
4
|
|
|
5
|
-
> **Status: engine, reactive facade, and live-delta transport implemented and unit-tested.** The zod schemas, canonical dag-cbor encoding, topic derivation, the verify pipeline (signature + constraints + on-chain gate + community-name resolution), the LWW winner-set CRDT with its binary bundle codec, the tally, the transport's **validate-before-forward gossip gate** over **inline bundle deltas**, and the **root-record checkpoint sync** (on-demand encode, suppressed 10-minute topic heartbeat, libp2p-fetch pull, divergent roots chased via directed bitswap) are all implemented — so the reactive `PubsubVoter` / `Contest` (`createContest`) / `ContestVote` (`createContestVote`) facade is live. The gate runs the full validity pipeline on the message bytes in an async gossipsub topic validator *before* re-forwarding, so an invalid bundle (bad signature, wallet the gate rejects, squatted name) is never propagated and `reject` scores the sender. Cold-join checkpoint bundles instead admit on the synchronous offline checks and settle their **deferred chain checks in the background, batched via multicall3** — the tally renders immediately with per-row `chainVerified`/`nameResolved` flags and refines as they land, and a node's own checkpoint only ever serves fully verified bundles (see [DESIGN.md, Background chain verification](./DESIGN.md#background-chain-verification)). **Keeping a live vote from decaying is the consuming client's job** — this library publishes each vote once and exposes `republishIntervalBuckets` so the client can schedule its own refreshes (see [DESIGN.md, Republishing is the client's job](./DESIGN.md#republishing-is-the-clients-job-not-this-librarys)).
|
|
5
|
+
> **Status: engine, reactive facade, and live-delta transport implemented and unit-tested.** The zod schemas, canonical dag-cbor encoding, topic derivation, the verify pipeline (signature + constraints + on-chain gate + community-name resolution), the LWW winner-set CRDT with its binary bundle codec, the tally, the transport's **validate-before-forward gossip gate** over **inline bundle deltas**, and the **root-record checkpoint sync** (on-demand encode, suppressed 10-minute topic heartbeat, libp2p-fetch pull, divergent roots chased via directed bitswap) are all implemented — so the reactive `PubsubVoter` / `Contest` (`createContest`) / `ContestVote` (`createContestVote`) facade is live. The gate runs the full validity pipeline on the message bytes in an async gossipsub topic validator *before* re-forwarding, so an invalid bundle (bad signature, wallet the gate rejects, squatted name) is never propagated and `reject` scores the sender. Cold-join checkpoint bundles instead admit on the synchronous offline checks and settle their **deferred chain checks in the background, batched via multicall3** — the tally renders immediately with per-row `chainVerified`/`nameResolved` flags and refines as they land, and a node's own checkpoint only ever serves fully verified bundles (see [DESIGN.md, Background chain verification](./DESIGN.md#background-chain-verification)). **Keeping a live vote from decaying is the consuming client's job** — this library publishes each vote once and exposes `republishIntervalBuckets` so the client can schedule its own refreshes (see [DESIGN.md, Republishing is the client's job](./DESIGN.md#republishing-is-the-clients-job-not-this-librarys)). The host side has since caught up: pkc-js registers gossipsub (`16.0.2`) and `@libp2p/fetch` on the shared node as of `0.0.63` ([pkc-js#183](https://github.com/pkcprotocol/pkc-js/issues/183) is closed), so a stock pkc-js host passes both construction guards — the remaining pkc-js work is the version-stable Helia accessor and gossipsub score tuning. See [ROADMAP.md](./ROADMAP.md), [DESIGN.md](./DESIGN.md), the [Transport gate](./DESIGN.md#transport-gossipsub-topic--validation), and [open questions](./DESIGN.md#open-questions).
|
|
6
6
|
|
|
7
7
|
## What it is for
|
|
8
8
|
|
|
@@ -18,7 +18,7 @@ The same engine generalizes to the original use case in [pkc-js issue #25](https
|
|
|
18
18
|
- The engine is reusable across clients and contests.
|
|
19
19
|
- The core (`schema/`, `verify/`, `crdt/`, `tally/`) is transport-agnostic and unit-testable without a network. libp2p only appears in `transport/`.
|
|
20
20
|
|
|
21
|
-
This library does not start its own node. It consumes the host's running Helia node directly — no adapter — and drives that node's gossipsub service and blockstore itself. The node must carry a pubsub service at `libp2p.services.pubsub` (a plain Helia node does not — register e.g. `@
|
|
21
|
+
This library does not start its own node. It consumes the host's running Helia node directly — no adapter — and drives that node's gossipsub service and blockstore itself. The node must carry a pubsub service at `libp2p.services.pubsub` (a plain Helia node does not — register e.g. `@libp2p/gossipsub`), a usable `blockstore`, and a libp2p fetch service at `libp2p.services.fetch` (register `@libp2p/fetch` — the checkpoint root-record pull rides it); construction throws `MissingPubsubError` / `MissingBlockstoreError` / `MissingFetchError` otherwise. With pkc-js today that node is reached at `pkc.clients.libp2pJsClients[key]._helia`; a version-stable accessor on pkc-js is a planned follow-up (see [DESIGN.md, Deferred pkc-js work](./DESIGN.md#deferred-pkc-js-work)).
|
|
22
22
|
|
|
23
23
|
## Design at a glance
|
|
24
24
|
|
|
@@ -34,12 +34,12 @@ The library never starts a node and never takes a host SDK (there is no `pkc` ar
|
|
|
34
34
|
|
|
35
35
|
| Seam | Type | Required | Purpose |
|
|
36
36
|
|---|---|---|---|
|
|
37
|
-
| `helia` | `HeliaInstance` | yes | the host's running Helia node; must carry a gossipsub service at `libp2p.services.pubsub` (else `MissingPubsubError`)
|
|
37
|
+
| `helia` | `HeliaInstance` | yes | the host's running Helia node; must carry a gossipsub service at `libp2p.services.pubsub` (else `MissingPubsubError`), a `blockstore` (else `MissingBlockstoreError`), and a libp2p fetch service at `libp2p.services.fetch` (else `MissingFetchError`) |
|
|
38
38
|
| `chains` | `ChainClientFactory` | yes | builds a viem `PublicClient` per chain; rules read through it for the gate and weight |
|
|
39
39
|
| `signer` | `VoteSigner` | no | the voting wallet's address + EIP-712 ballot signing; omit for a read-only voter |
|
|
40
40
|
| `nameResolvers` | `NameResolver[]` | no | community-name resolvers (same interface and instances as pkc-js's `nameResolvers`, e.g. `@bitsocial/bso-resolver` for `name.bso`); each vote's `community.name` claim is verified through them — inline at the forward-gate for live votes, in the background verifier for cold-join admits — and a bundle whose name resolves to a different `publicKey` than claimed is dropped/evicted |
|
|
41
41
|
| `dataPath` | `string \| false` | no | directory for the voter's persistent caches (gate results + name resolutions), the pkc-js `dataPath` equivalent. Node default: `{cwd}/.bitsocial-pubsub-voting` (better-sqlite3 under `{dataPath}/lru-storage/`); in the browser the path is ignored and the caches live in IndexedDB. Pass `false` for in-memory-only (the pkc-js `noData` equivalent). A restart re-serves settled gate reads and fresh name resolutions from the store instead of the RPC |
|
|
42
|
-
| `httpRouterUrls` | `string[]` | no | Delegated Routing V1 router base URLs to **announce provider records to** (one unsigned `PUT /routing/v1/providers` per router; `Keys` batches every joined contest's criteria CID + current checkpoint root + chunk CIDs — hourly, debounced on root changes, and on address changes). **Seeders only**: absent/empty means never announce (the default — plain clients are not dialable), and the browser build never announces regardless. The node must be publicly
|
|
42
|
+
| `httpRouterUrls` | `string[]` | no | Delegated Routing V1 router base URLs to **announce provider records to** (one unsigned `PUT /routing/v1/providers` per router; `Keys` batches every joined contest's criteria CID + current checkpoint root + chunk CIDs — hourly, debounced on root changes, and on address changes). **Seeders only**: absent/empty means never announce (the default — plain clients are not dialable), and the browser build never announces regardless. The node must be publicly **reachable** (its listening port open/forwarded/published), but it does not need to know its own public IP: private, loopback, and link-local addrs are filtered client-side, and when nothing survives — the normal zero-config case behind NAT or a Docker bridge, and even on public-IP hosts, since libp2p withholds unconfirmed public addrs pending AutoNAT — the announcer sends the wildcard sentinels (`/ip4/0.0.0.0/...`, `/ip6/::/...`) that the router rewrites to the PUT's observed source IP, exactly as kubo announces work. Configured `addresses.announce` values (concrete public addrs, DNS/AutoTLS, or a kubo-style wildcard) are used as-is. Only a loopback-only node announces nothing. *Querying* needs no URLs here — cold-join discovery uses the injected node's `libp2p.contentRouting`, which the host wires its routers into |
|
|
43
43
|
|
|
44
44
|
A contest is addressed by its **full criteria document**, passed to `createContest` / `createContestVote`. The document is strictly validated there (`CriteriaSchema` + the rule registry), and its canonical bytes derive the topic — so the exact document every participant shares is the only contest configuration that exists.
|
|
45
45
|
|
|
@@ -54,7 +54,7 @@ const voter = new PubsubVoter({
|
|
|
54
54
|
signer: mySigner, // optional; omit → read-only voter
|
|
55
55
|
nameResolvers: [bsoResolver], // optional; verifies community-name claims (e.g. @bitsocial/bso-resolver)
|
|
56
56
|
dataPath: "/path/to/data", // optional; persistent-cache directory (default {cwd}/.bitsocial-pubsub-voting; false → in-memory)
|
|
57
|
-
httpRouterUrls: [ // optional, SEEDERS ONLY (publicly
|
|
57
|
+
httpRouterUrls: [ // optional, SEEDERS ONLY (publicly reachable node): announce provider
|
|
58
58
|
"https://routing.example" // records (criteria CID + checkpoint root + chunks) so cold joiners
|
|
59
59
|
] // can discover this node via the routers; clients omit this
|
|
60
60
|
});
|
|
@@ -139,7 +139,7 @@ import { deriveDirectoryCriteria } from "@bitsocial/pubsub-voting";
|
|
|
139
139
|
import stripJsonComments from "strip-json-comments";
|
|
140
140
|
|
|
141
141
|
const manifest = JSON.parse(stripJsonComments(manifestJsonc)) as unknown;
|
|
142
|
-
const allCriteria = deriveDirectoryCriteria(manifest); // → Criteria[], throws on invalid entries or duplicate contestIds
|
|
142
|
+
const allCriteria = deriveDirectoryCriteria(manifest); // → Criteria[], throws on invalid entries or duplicate contestIds (DuplicateContestIdError)
|
|
143
143
|
|
|
144
144
|
const contests = await Promise.all(allCriteria.map((criteria) => voter.createContest({ criteria }))); // → Contest[]
|
|
145
145
|
for (const contest of contests) await contest.update(); // a full host joins + serves the whole directory
|
|
@@ -170,7 +170,7 @@ Full, type-checked call patterns for a pkc-js host, a plebbit/seedit host, and a
|
|
|
170
170
|
|
|
171
171
|
### Custom rules
|
|
172
172
|
|
|
173
|
-
The gate and weight are a single flat registry of rules, one `type` per file, mirroring the pkc-js challenge registry. Each rule owns its option schema and is evaluated at the bundle's bucket block. Chain-reading rules get `ctx.chain` — the viem `PublicClient` for their `options.chain` — and write their own reads (`readContract`, `getBalance`, ...), pinning each call to the sampled block with `blockNumber: BigInt(ctx.blockNumber)`. There is **one kind**: `evaluate → { score: bigint }`, a non-negative score where `0n` means "does not qualify" (a result object, not a bare `bigint`, so slot-specific fields can be added later). The criteria has two *slots* drawing from the one registry — the **rule** slot treats the score as a gate (`> 0n` admits), the **weight** slot as the vote's magnitude. A wallet's vote counts as `rule.score > 0n ? weight.score : 0n`. A rule that needs a threshold returns `0n` below it (so `erc721-min-balance`'s optional `min` gates), which lets the same rule serve either slot.
|
|
173
|
+
The gate and weight are a single flat registry of rules, one `type` per file, mirroring the pkc-js challenge registry. Each rule owns its option schema and is evaluated at the bundle's bucket block. Chain-reading rules get `ctx.chain` — the viem `PublicClient` for their `options.chain` — and write their own reads (`readContract`, `getBalance`, ...), pinning each call to the sampled block with `blockNumber: BigInt(ctx.blockNumber)`. There is **one kind**: `evaluate → { score: bigint }`, a non-negative score where `0n` means "does not qualify" (a result object, not a bare `bigint`, so slot-specific fields can be added later). The criteria has two *slots* drawing from the one registry — the **rule** slot treats the score as a gate (`> 0n` admits), the **weight** slot as the vote's magnitude. A wallet's vote counts as `rule.score > 0n ? weight.score : 0n`. A rule that needs a threshold returns `0n` below it (so `erc721-min-balance`'s optional `min` gates), which lets the same rule serve either slot. A chain-reading rule may also implement the optional `evaluateMany(walletAddresses, ctx)` batch hook — its semantics MUST equal mapping `evaluate` — which the background verifier uses to batch a cold join's gate reads (`erc721-min-balance` implements it over multicall3; see [DESIGN.md, Background chain verification](./DESIGN.md#background-chain-verification)).
|
|
174
174
|
|
|
175
175
|
Built-ins: `erc721-min-balance` (v1) and `constant` (v1). A host adds or shadows rules by `type` via the `rules` option — this is how clients like 5chan or seedit register custom rules without forking the library:
|
|
176
176
|
|
|
@@ -215,6 +215,7 @@ src/
|
|
|
215
215
|
checkpoint/ deterministic checkpoint codec (root manifest + size-capped chunks) [implemented]
|
|
216
216
|
transport/ async validate-before-forward gossip gate + message codec (inline bundle / root record) + root chase + transport [implemented]
|
|
217
217
|
tally/ deterministic aggregation over pre-validated bundles [implemented]
|
|
218
|
+
storage/ persistent-cache backends: better-sqlite3 (Node) / IndexedDB (browser-field remap) / in-memory [implemented]
|
|
218
219
|
index.ts public entry: re-exports + facade + design types
|
|
219
220
|
```
|
|
220
221
|
|
package/dist/client/voter.js
CHANGED
|
@@ -392,6 +392,10 @@ class ContestEngine {
|
|
|
392
392
|
for (const cb of [...this.#errorListeners])
|
|
393
393
|
cb(error);
|
|
394
394
|
}
|
|
395
|
+
/** Surface a voter-level failure (e.g. a provider announce) through this contest's error event. */
|
|
396
|
+
emitError(error) {
|
|
397
|
+
this.#emitError(error);
|
|
398
|
+
}
|
|
395
399
|
#chainFor(ticker) {
|
|
396
400
|
const client = this.#chainClients[ticker];
|
|
397
401
|
if (!client)
|
|
@@ -1180,7 +1184,18 @@ export class PubsubVoter {
|
|
|
1180
1184
|
this.#announcer = makeAnnouncer({
|
|
1181
1185
|
routerUrls: [...options.httpRouterUrls],
|
|
1182
1186
|
libp2p: options.helia.libp2p,
|
|
1183
|
-
keys: this.#announceKeys
|
|
1187
|
+
keys: this.#announceKeys,
|
|
1188
|
+
// An announce failure is a discoverability degradation for every joined contest,
|
|
1189
|
+
// so it surfaces through each one's error event (observational, like the announce
|
|
1190
|
+
// itself: never retried, never thrown) — a silent announce failure otherwise looks
|
|
1191
|
+
// exactly like a healthy seeder that nobody can find.
|
|
1192
|
+
onError: (url, error) => {
|
|
1193
|
+
const announceError = new Error(`provider announce to router ${url} failed: ${error instanceof Error ? error.message : String(error)}`, { cause: error });
|
|
1194
|
+
for (const engine of this.#engines.values()) {
|
|
1195
|
+
if (engine.joined)
|
|
1196
|
+
engine.emitError(announceError);
|
|
1197
|
+
}
|
|
1198
|
+
}
|
|
1184
1199
|
});
|
|
1185
1200
|
}
|
|
1186
1201
|
}
|
|
@@ -9,6 +9,17 @@ import type { Announcer, AnnouncerOptions } from "./types.js";
|
|
|
9
9
|
* exists), and its anti-spoofing keeps `/ip4`/`/ip6` addrs only when the IP matches the PUT's
|
|
10
10
|
* source IP — which a seeder announcing its own addresses passes naturally.
|
|
11
11
|
*
|
|
12
|
+
* Addresses: the announceable set is `getMultiaddrs()` filtered to public/DNS addrs plus
|
|
13
|
+
* exactly-unspecified addrs (`0.0.0.0`/`::`), which the production router rewrites to the PUT's
|
|
14
|
+
* observed source IP (`cleanAddrs` — how kubo's announces work). When the filter comes up EMPTY,
|
|
15
|
+
* the announcer synthesizes those wildcard sentinels itself from the node's listen ports
|
|
16
|
+
* ({@link sentinelAddrs}), because libp2p never reports one: a wildcard listen is expanded to
|
|
17
|
+
* concrete interface addrs, and a PUBLIC interface addr is withheld from `getMultiaddrs()` until
|
|
18
|
+
* AutoNAT confirms it — which a seeder with no inbound peers yet can never pass (the announce is
|
|
19
|
+
* what brings the first peer). So the zero-config seeder — NAT'd, Docker-bridged, or a bare
|
|
20
|
+
* public-IP host without AutoNAT — announces the sentinel and the router fills in the IP it can
|
|
21
|
+
* actually see.
|
|
22
|
+
*
|
|
12
23
|
* Ticks: the debounced change trigger ({@link Announcer.notifyChange} — contest joins, checkpoint
|
|
13
24
|
* root changes, `self:peer:update` address changes) plus an hourly re-announce (pkc-js's
|
|
14
25
|
* `providePubsubTopicRoutingCidsIfNeeded` cadence; the production router's record TTL is 24h, so
|
|
@@ -27,12 +38,28 @@ export declare const ANNOUNCE_DEBOUNCE_MS = 10000;
|
|
|
27
38
|
export declare const ANNOUNCE_ROUTER_TIMEOUT_MS = 10000;
|
|
28
39
|
/**
|
|
29
40
|
* Filter a node's multiaddrs down to what belongs in a public provider record: public `/ip4` /
|
|
30
|
-
* `/ip6` addrs
|
|
31
|
-
* WSS addrs travel this way, and the production router passes DNS through unvalidated)
|
|
32
|
-
*
|
|
33
|
-
* the
|
|
34
|
-
*
|
|
41
|
+
* `/ip6` addrs, DNS addrs (`/dns4`/`/dns6`/`/dnsaddr` — the AutoTLS `<peerId>.libp2p.direct`
|
|
42
|
+
* WSS addrs travel this way, and the production router passes DNS through unvalidated), and
|
|
43
|
+
* EXACTLY-unspecified addrs (`/ip4/0.0.0.0/...`, `/ip6/::/...`) — the router rewrites those to
|
|
44
|
+
* the PUT's observed source IP (`cleanAddrs`, the same mechanism kubo's announces rely on), so
|
|
45
|
+
* they are how a node that cannot see its own public IP still announces a dialable record.
|
|
46
|
+
* Private, loopback, link-local, and CGNAT IPs are dropped CLIENT-side rather than trusting the
|
|
47
|
+
* router to drop them; a `p2p-circuit` addr is judged by its relay's leading component like any
|
|
48
|
+
* other. Exported for the announcer's unit tests.
|
|
35
49
|
*/
|
|
36
50
|
export declare function announceableAddrs(addrs: readonly string[]): string[];
|
|
51
|
+
/**
|
|
52
|
+
* Synthesize the router's "rewrite me" wildcard sentinels for a node with NO announceable addr:
|
|
53
|
+
* every non-loopback `/ip4`/`/ip6` interface addr, with its IP swapped for the unspecified addr
|
|
54
|
+
* of its family and the rest of the multiaddr (port, transport, `/p2p/` suffix) kept, deduped.
|
|
55
|
+
* libp2p never reports a wildcard itself — a `0.0.0.0` listen is expanded to concrete interface
|
|
56
|
+
* addrs, and a public interface addr is withheld until AutoNAT confirms it — so behind NAT, on a
|
|
57
|
+
* Docker bridge, or on a public-IP host without AutoNAT the whole set filters away and the listen
|
|
58
|
+
* ports here are the only truthful thing left to announce; the router substitutes the source IP
|
|
59
|
+
* it observed (dropping the family it did not see the PUT from). Loopback addrs are excluded as
|
|
60
|
+
* synthesis sources: a loopback-only node deliberately isn't listening on any interface a rewrite
|
|
61
|
+
* could make dialable, and it must keep announcing nothing. Exported for the announcer's tests.
|
|
62
|
+
*/
|
|
63
|
+
export declare function sentinelAddrs(addrs: readonly string[]): string[];
|
|
37
64
|
/** Build the Node announcer. The browser build never sees this file (package.json `browser` remap). */
|
|
38
65
|
export declare function makeAnnouncer(options: AnnouncerOptions): Announcer;
|
|
@@ -8,6 +8,17 @@
|
|
|
8
8
|
* exists), and its anti-spoofing keeps `/ip4`/`/ip6` addrs only when the IP matches the PUT's
|
|
9
9
|
* source IP — which a seeder announcing its own addresses passes naturally.
|
|
10
10
|
*
|
|
11
|
+
* Addresses: the announceable set is `getMultiaddrs()` filtered to public/DNS addrs plus
|
|
12
|
+
* exactly-unspecified addrs (`0.0.0.0`/`::`), which the production router rewrites to the PUT's
|
|
13
|
+
* observed source IP (`cleanAddrs` — how kubo's announces work). When the filter comes up EMPTY,
|
|
14
|
+
* the announcer synthesizes those wildcard sentinels itself from the node's listen ports
|
|
15
|
+
* ({@link sentinelAddrs}), because libp2p never reports one: a wildcard listen is expanded to
|
|
16
|
+
* concrete interface addrs, and a PUBLIC interface addr is withheld from `getMultiaddrs()` until
|
|
17
|
+
* AutoNAT confirms it — which a seeder with no inbound peers yet can never pass (the announce is
|
|
18
|
+
* what brings the first peer). So the zero-config seeder — NAT'd, Docker-bridged, or a bare
|
|
19
|
+
* public-IP host without AutoNAT — announces the sentinel and the router fills in the IP it can
|
|
20
|
+
* actually see.
|
|
21
|
+
*
|
|
11
22
|
* Ticks: the debounced change trigger ({@link Announcer.notifyChange} — contest joins, checkpoint
|
|
12
23
|
* root changes, `self:peer:update` address changes) plus an hourly re-announce (pkc-js's
|
|
13
24
|
* `providePubsubTopicRoutingCidsIfNeeded` cadence; the production router's record TTL is 24h, so
|
|
@@ -24,13 +35,16 @@ export const ANNOUNCE_INTERVAL_MS = 3_600_000;
|
|
|
24
35
|
export const ANNOUNCE_DEBOUNCE_MS = 10_000;
|
|
25
36
|
/** Per-router PUT deadline — same order as the cold-join router lookup deadline. */
|
|
26
37
|
export const ANNOUNCE_ROUTER_TIMEOUT_MS = 10_000;
|
|
27
|
-
/**
|
|
38
|
+
/** The exactly-unspecified IPs — the production router's "rewrite me to the PUT's source IP" sentinels. */
|
|
39
|
+
const UNSPECIFIED_IP4 = "0.0.0.0";
|
|
40
|
+
const UNSPECIFIED_IP6 = "::";
|
|
41
|
+
/** RFC1918/loopback/link-local/CGNAT/"this network" IPv4 — never dialable as announced. */
|
|
28
42
|
function isPrivateIp4(ip) {
|
|
29
43
|
const parts = ip.split(".").map(Number);
|
|
30
44
|
if (parts.length !== 4 || parts.some((p) => !Number.isInteger(p) || p < 0 || p > 255))
|
|
31
45
|
return true;
|
|
32
46
|
const [a, b] = parts;
|
|
33
|
-
return (a === 0 || //
|
|
47
|
+
return (a === 0 || // "this network" 0.0.0.0/8 (the exact unspecified addr is special-cased by callers)
|
|
34
48
|
a === 10 ||
|
|
35
49
|
a === 127 || // loopback
|
|
36
50
|
(a === 100 && b >= 64 && b < 128) || // CGNAT 100.64/10
|
|
@@ -38,21 +52,24 @@ function isPrivateIp4(ip) {
|
|
|
38
52
|
(a === 172 && b >= 16 && b < 32) ||
|
|
39
53
|
(a === 192 && b === 168));
|
|
40
54
|
}
|
|
41
|
-
/** Loopback/link-local/ULA
|
|
55
|
+
/** Loopback/link-local/ULA IPv6 — never dialable as announced. */
|
|
42
56
|
function isPrivateIp6(ip) {
|
|
43
57
|
const lower = ip.toLowerCase();
|
|
44
|
-
if (lower ===
|
|
58
|
+
if (lower === UNSPECIFIED_IP6 || lower === "::1")
|
|
45
59
|
return true;
|
|
46
60
|
// fe80::/10 link-local (fe8x..febx), fc00::/7 unique-local (fcxx/fdxx).
|
|
47
61
|
return /^fe[89ab]/.test(lower) || /^f[cd]/.test(lower);
|
|
48
62
|
}
|
|
49
63
|
/**
|
|
50
64
|
* Filter a node's multiaddrs down to what belongs in a public provider record: public `/ip4` /
|
|
51
|
-
* `/ip6` addrs
|
|
52
|
-
* WSS addrs travel this way, and the production router passes DNS through unvalidated)
|
|
53
|
-
*
|
|
54
|
-
* the
|
|
55
|
-
*
|
|
65
|
+
* `/ip6` addrs, DNS addrs (`/dns4`/`/dns6`/`/dnsaddr` — the AutoTLS `<peerId>.libp2p.direct`
|
|
66
|
+
* WSS addrs travel this way, and the production router passes DNS through unvalidated), and
|
|
67
|
+
* EXACTLY-unspecified addrs (`/ip4/0.0.0.0/...`, `/ip6/::/...`) — the router rewrites those to
|
|
68
|
+
* the PUT's observed source IP (`cleanAddrs`, the same mechanism kubo's announces rely on), so
|
|
69
|
+
* they are how a node that cannot see its own public IP still announces a dialable record.
|
|
70
|
+
* Private, loopback, link-local, and CGNAT IPs are dropped CLIENT-side rather than trusting the
|
|
71
|
+
* router to drop them; a `p2p-circuit` addr is judged by its relay's leading component like any
|
|
72
|
+
* other. Exported for the announcer's unit tests.
|
|
56
73
|
*/
|
|
57
74
|
export function announceableAddrs(addrs) {
|
|
58
75
|
return addrs.filter((addr) => {
|
|
@@ -62,12 +79,40 @@ export function announceableAddrs(addrs) {
|
|
|
62
79
|
if (proto === "dns4" || proto === "dns6" || proto === "dnsaddr" || proto === "dns")
|
|
63
80
|
return true;
|
|
64
81
|
if (proto === "ip4")
|
|
65
|
-
return !isPrivateIp4(value);
|
|
82
|
+
return value === UNSPECIFIED_IP4 || !isPrivateIp4(value);
|
|
66
83
|
if (proto === "ip6")
|
|
67
|
-
return !isPrivateIp6(value);
|
|
84
|
+
return value.toLowerCase() === UNSPECIFIED_IP6 || !isPrivateIp6(value);
|
|
68
85
|
return false;
|
|
69
86
|
});
|
|
70
87
|
}
|
|
88
|
+
/**
|
|
89
|
+
* Synthesize the router's "rewrite me" wildcard sentinels for a node with NO announceable addr:
|
|
90
|
+
* every non-loopback `/ip4`/`/ip6` interface addr, with its IP swapped for the unspecified addr
|
|
91
|
+
* of its family and the rest of the multiaddr (port, transport, `/p2p/` suffix) kept, deduped.
|
|
92
|
+
* libp2p never reports a wildcard itself — a `0.0.0.0` listen is expanded to concrete interface
|
|
93
|
+
* addrs, and a public interface addr is withheld until AutoNAT confirms it — so behind NAT, on a
|
|
94
|
+
* Docker bridge, or on a public-IP host without AutoNAT the whole set filters away and the listen
|
|
95
|
+
* ports here are the only truthful thing left to announce; the router substitutes the source IP
|
|
96
|
+
* it observed (dropping the family it did not see the PUT from). Loopback addrs are excluded as
|
|
97
|
+
* synthesis sources: a loopback-only node deliberately isn't listening on any interface a rewrite
|
|
98
|
+
* could make dialable, and it must keep announcing nothing. Exported for the announcer's tests.
|
|
99
|
+
*/
|
|
100
|
+
export function sentinelAddrs(addrs) {
|
|
101
|
+
const sentinels = new Set();
|
|
102
|
+
for (const addr of addrs) {
|
|
103
|
+
const parts = addr.split("/");
|
|
104
|
+
const [, proto, value] = parts;
|
|
105
|
+
if (proto === undefined || value === undefined)
|
|
106
|
+
continue;
|
|
107
|
+
if (proto === "ip4" && value !== UNSPECIFIED_IP4 && value.split(".")[0] !== "127") {
|
|
108
|
+
sentinels.add(["", proto, UNSPECIFIED_IP4, ...parts.slice(3)].join("/"));
|
|
109
|
+
}
|
|
110
|
+
else if (proto === "ip6" && !["::", "::1"].includes(value.toLowerCase())) {
|
|
111
|
+
sentinels.add(["", proto, UNSPECIFIED_IP6, ...parts.slice(3)].join("/"));
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
return [...sentinels];
|
|
115
|
+
}
|
|
71
116
|
/** One unsigned kubo-shape provider PUT; throws on timeout or a non-2xx answer. */
|
|
72
117
|
async function putProviders(baseUrl, body, timeoutMs) {
|
|
73
118
|
const endpoint = `${baseUrl.replace(/\/+$/, "")}/routing/v1/providers`;
|
|
@@ -103,10 +148,15 @@ export function makeAnnouncer(options) {
|
|
|
103
148
|
do {
|
|
104
149
|
rerun = false;
|
|
105
150
|
const keys = await options.keys();
|
|
106
|
-
const
|
|
107
|
-
|
|
108
|
-
//
|
|
109
|
-
//
|
|
151
|
+
const all = options.libp2p.getMultiaddrs().map((a) => a.toString());
|
|
152
|
+
let addrs = announceableAddrs(all);
|
|
153
|
+
// No announceable addr — behind NAT/Docker-bridge, or a public interface addr
|
|
154
|
+
// libp2p is still withholding pending AutoNAT — announce the wildcard sentinels
|
|
155
|
+
// and let the router substitute the source IP it observes (see sentinelAddrs).
|
|
156
|
+
if (addrs.length === 0)
|
|
157
|
+
addrs = sentinelAddrs(all);
|
|
158
|
+
// Nothing joined, or loopback-only (not listening on any rewritable interface):
|
|
159
|
+
// announce nothing — the production router drops addr-less providers anyway.
|
|
110
160
|
if (keys.length === 0 || addrs.length === 0)
|
|
111
161
|
continue;
|
|
112
162
|
const body = JSON.stringify({
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bitsocial/pubsub-voting",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.8",
|
|
4
4
|
"description": "Trustless pubsub voting over a shared libp2p/Helia node.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "GPL-3.0-or-later",
|
|
@@ -47,7 +47,7 @@
|
|
|
47
47
|
"better-sqlite3": "12.9.0",
|
|
48
48
|
"helia": "6.1.4",
|
|
49
49
|
"localforage": "1.10.0",
|
|
50
|
-
"multiformats": "
|
|
50
|
+
"multiformats": "14.0.0",
|
|
51
51
|
"p-limit": "6.2.0",
|
|
52
52
|
"viem": "2.54.0",
|
|
53
53
|
"zod": "4.3.6"
|
|
@@ -1,49 +0,0 @@
|
|
|
1
|
-
import type { PeerId } from "@libp2p/interface";
|
|
2
|
-
import type { FetchServiceLike } from "../transport/types.js";
|
|
3
|
-
import { type FetchRootRecord } from "../transport/messages.js";
|
|
4
|
-
/**
|
|
5
|
-
* The cold-start root puller: the voter-wide seam every engine's cold-join pull goes through
|
|
6
|
-
* (see DESIGN.md "Checkpoints"). One instance per voter, because everything it guards is
|
|
7
|
-
* per-PEER, not per-topic:
|
|
8
|
-
*
|
|
9
|
-
* - **Batching.** Pulls to the same peer that arrive within {@link BATCH_WINDOW_MS} coalesce
|
|
10
|
-
* into ONE fetch stream carrying a batch key ({@link batchRootsFetchKey}), so a directory-
|
|
11
|
-
* scale join pays the ~2-RTT multistream-select negotiation once per peer instead of once
|
|
12
|
-
* per contest. A single pending topic skips the batch key entirely (the common single-board
|
|
13
|
-
* case has zero new wire surface). A responder that predates the batch key answers
|
|
14
|
-
* NOT_FOUND (or garbage), and the puller falls back to today's per-topic keys.
|
|
15
|
-
* - **Budget.** At most {@link COLD_START_PEER_FETCH_LIMIT} concurrent fetch streams per peer
|
|
16
|
-
* across ALL contests, under libp2p's default per-protocol caps (32 inbound on the
|
|
17
|
-
* responder, 64 outbound on us — both enforced PER CONNECTION per direction, so one
|
|
18
|
-
* connection's budget is exactly the scope of the remote cap; other users of a shared
|
|
19
|
-
* seeder arrive on their own connections and do not eat these slots). 24 rather than the
|
|
20
|
-
* full 32 because running at the cliff still resets: our slot frees when the response
|
|
21
|
-
* lands, but the responder only decrements its count when it sees the stream *close*, so
|
|
22
|
-
* back-to-back reuse races that bookkeeping — and the same connection can carry fetch
|
|
23
|
-
* streams the budget cannot see (the host's own IPNS-over-pubsub record fetches ride the
|
|
24
|
-
* same protocol; so would a second voter on the shared node).
|
|
25
|
-
* - **Retry.** A THROWN fetch retries with full-jittered exponential backoff until
|
|
26
|
-
* {@link COLD_START_FETCH_DEADLINE_MS} — the safety net for a responder saturated by
|
|
27
|
-
* streams the budget cannot see. While the cap is saturated every freed slot is instantly
|
|
28
|
-
* retaken, so a fixed attempt count can lose the race and strand a board (measured: no
|
|
29
|
-
* retry → 32/63 boards converge; 5 fixed retries → 53/63; retry-to-deadline → 63/63). Only
|
|
30
|
-
* a throw retries — a definitive `undefined`/`null` ("no record") returns as-is — and a
|
|
31
|
-
* pull whose contest was torn down (`isLive()` false) abandons quietly. Each attempt (not
|
|
32
|
-
* the whole loop, so a backoff sleep never holds a slot) passes through the budget; queue
|
|
33
|
-
* wait counts against the same deadline.
|
|
34
|
-
*/
|
|
35
|
-
export interface RootPuller {
|
|
36
|
-
/**
|
|
37
|
-
* Pull one topic's root record from one peer: the decoded record, or `null`/`undefined`
|
|
38
|
-
* ("no record", definitive), or a rejection (unreachable peer past the deadline, or a
|
|
39
|
-
* garbage answer). `isLive` is polled between retries and before resolving, so a contest
|
|
40
|
-
* left mid-pull abandons instead of holding work alive.
|
|
41
|
-
*/
|
|
42
|
-
pull(peer: PeerId, topic: string, isLive: () => boolean): Promise<FetchRootRecord | null | undefined>;
|
|
43
|
-
}
|
|
44
|
-
/** See the budget note on {@link RootPuller}. */
|
|
45
|
-
export declare const COLD_START_PEER_FETCH_LIMIT = 24;
|
|
46
|
-
/** See the retry note on {@link RootPuller}. */
|
|
47
|
-
export declare const COLD_START_FETCH_DEADLINE_MS = 30000;
|
|
48
|
-
/** Build the voter-wide puller over the host's fetch service. */
|
|
49
|
-
export declare function makeRootPuller(fetch: FetchServiceLike): RootPuller;
|
|
@@ -1,140 +0,0 @@
|
|
|
1
|
-
import pLimit from "p-limit";
|
|
2
|
-
import { batchRootsFetchKey, decodeBatchRootsResponse, decodeRootRecord, rootFetchKey, MAX_BATCH_ROOT_KEYS } from "../transport/messages.js";
|
|
3
|
-
/** See the budget note on {@link RootPuller}. */
|
|
4
|
-
export const COLD_START_PEER_FETCH_LIMIT = 24;
|
|
5
|
-
/** See the retry note on {@link RootPuller}. */
|
|
6
|
-
export const COLD_START_FETCH_DEADLINE_MS = 30_000;
|
|
7
|
-
const COLD_START_FETCH_BACKOFF_MS = 400;
|
|
8
|
-
const COLD_START_FETCH_BACKOFF_CAP_MS = 4_000;
|
|
9
|
-
/**
|
|
10
|
-
* How long a peer's first pending pull waits for same-peer company before its batch flushes.
|
|
11
|
-
* A directory join fires all its cold starts in one synchronous burst, so one tick would
|
|
12
|
-
* usually do; a few ms of slack covers joins interleaved with per-contest async work (topic
|
|
13
|
-
* hashing, chain-client setup) without adding perceptible latency to a lone join.
|
|
14
|
-
*/
|
|
15
|
-
const BATCH_WINDOW_MS = 20;
|
|
16
|
-
/**
|
|
17
|
-
* One `pLimit(limitPerPeer)` per peer id, created on first use and dropped once its queue
|
|
18
|
-
* drains, so a long-lived voter does not accumulate limiters for every peer it ever
|
|
19
|
-
* cold-started against.
|
|
20
|
-
*/
|
|
21
|
-
function makePerPeerBudget(limitPerPeer) {
|
|
22
|
-
const limiters = new Map();
|
|
23
|
-
return async (peerId, task) => {
|
|
24
|
-
let limiter = limiters.get(peerId);
|
|
25
|
-
if (limiter === undefined) {
|
|
26
|
-
limiter = pLimit(limitPerPeer);
|
|
27
|
-
limiters.set(peerId, limiter);
|
|
28
|
-
}
|
|
29
|
-
try {
|
|
30
|
-
return await limiter(task);
|
|
31
|
-
}
|
|
32
|
-
finally {
|
|
33
|
-
if (limiter.activeCount === 0 && limiter.pendingCount === 0)
|
|
34
|
-
limiters.delete(peerId);
|
|
35
|
-
}
|
|
36
|
-
};
|
|
37
|
-
}
|
|
38
|
-
/** Build the voter-wide puller over the host's fetch service. */
|
|
39
|
-
export function makeRootPuller(fetch) {
|
|
40
|
-
const budget = makePerPeerBudget(COLD_START_PEER_FETCH_LIMIT);
|
|
41
|
-
const pending = new Map();
|
|
42
|
-
/** One budgeted+retried fetch of one key (see the retry note on {@link RootPuller}). */
|
|
43
|
-
const fetchWithRetry = async (peer, key, isLive) => {
|
|
44
|
-
const deadline = Date.now() + COLD_START_FETCH_DEADLINE_MS;
|
|
45
|
-
let lastError;
|
|
46
|
-
for (let attempt = 0;; attempt++) {
|
|
47
|
-
if (attempt > 0) {
|
|
48
|
-
if (!isLive() || Date.now() >= deadline)
|
|
49
|
-
break; // left or out of time
|
|
50
|
-
const ceiling = Math.min(COLD_START_FETCH_BACKOFF_CAP_MS, COLD_START_FETCH_BACKOFF_MS * 2 ** (attempt - 1));
|
|
51
|
-
await new Promise((resolve) => setTimeout(resolve, Math.random() * ceiling));
|
|
52
|
-
if (!isLive())
|
|
53
|
-
return undefined; // left mid-backoff — abandon quietly
|
|
54
|
-
}
|
|
55
|
-
try {
|
|
56
|
-
return await budget(peer.toString(), () => fetch.fetch(peer, key));
|
|
57
|
-
}
|
|
58
|
-
catch (error) {
|
|
59
|
-
lastError = error; // transient (e.g. responder over its inbound-stream cap) — back off and retry
|
|
60
|
-
}
|
|
61
|
-
}
|
|
62
|
-
throw lastError;
|
|
63
|
-
};
|
|
64
|
-
/** The per-topic path: today's `<topic>/root` key; a garbage answer rejects the waiters. */
|
|
65
|
-
const pullSingle = async (peer, topic, waiters) => {
|
|
66
|
-
try {
|
|
67
|
-
const value = await fetchWithRetry(peer, rootFetchKey(topic), () => waiters.some((w) => w.isLive()));
|
|
68
|
-
const record = value === undefined || value === null ? value : decodeRootRecord(value);
|
|
69
|
-
waiters.forEach((w) => w.resolve(record));
|
|
70
|
-
}
|
|
71
|
-
catch (error) {
|
|
72
|
-
waiters.forEach((w) => w.reject(error));
|
|
73
|
-
}
|
|
74
|
-
};
|
|
75
|
-
/**
|
|
76
|
-
* The batch path: one stream, one key carrying every pending topic, answers distributed by
|
|
77
|
-
* request order. NOT_FOUND (a responder without the batch key), a malformed response, or a
|
|
78
|
-
* length mismatch all degrade to the per-topic path — never to silence.
|
|
79
|
-
*/
|
|
80
|
-
const pullBatch = async (peer, topics) => {
|
|
81
|
-
const order = [...topics.keys()];
|
|
82
|
-
const isLive = () => [...topics.values()].some((waiters) => waiters.some((w) => w.isLive()));
|
|
83
|
-
try {
|
|
84
|
-
const value = await fetchWithRetry(peer, batchRootsFetchKey(order), isLive);
|
|
85
|
-
if (value !== undefined && value !== null) {
|
|
86
|
-
let records;
|
|
87
|
-
try {
|
|
88
|
-
records = decodeBatchRootsResponse(value);
|
|
89
|
-
}
|
|
90
|
-
catch {
|
|
91
|
-
records = undefined; // hostile/buggy answer — fall through to per-topic
|
|
92
|
-
}
|
|
93
|
-
if (records !== undefined && records.length === order.length) {
|
|
94
|
-
order.forEach((topic, i) => topics.get(topic).forEach((w) => w.resolve(records[i] ?? null)));
|
|
95
|
-
return;
|
|
96
|
-
}
|
|
97
|
-
}
|
|
98
|
-
// Old responder (NOT_FOUND) or malformed batch answer: degrade to per-topic keys.
|
|
99
|
-
await Promise.all([...topics.entries()].map(([topic, waiters]) => pullSingle(peer, topic, waiters)));
|
|
100
|
-
}
|
|
101
|
-
catch (error) {
|
|
102
|
-
for (const waiters of topics.values())
|
|
103
|
-
waiters.forEach((w) => w.reject(error));
|
|
104
|
-
}
|
|
105
|
-
};
|
|
106
|
-
const flush = (peerId) => {
|
|
107
|
-
const batch = pending.get(peerId);
|
|
108
|
-
if (batch === undefined)
|
|
109
|
-
return;
|
|
110
|
-
pending.delete(peerId);
|
|
111
|
-
clearTimeout(batch.timer);
|
|
112
|
-
// A lone topic keeps today's per-topic key — no batch envelope for the common
|
|
113
|
-
// single-board join; two or more ride one batch stream.
|
|
114
|
-
if (batch.topics.size === 1) {
|
|
115
|
-
for (const [topic, waiters] of batch.topics)
|
|
116
|
-
void pullSingle(batch.peer, topic, waiters);
|
|
117
|
-
}
|
|
118
|
-
else {
|
|
119
|
-
void pullBatch(batch.peer, batch.topics);
|
|
120
|
-
}
|
|
121
|
-
};
|
|
122
|
-
return {
|
|
123
|
-
pull: (peer, topic, isLive) => new Promise((resolve, reject) => {
|
|
124
|
-
const peerId = peer.toString();
|
|
125
|
-
let batch = pending.get(peerId);
|
|
126
|
-
if (batch === undefined) {
|
|
127
|
-
const timer = setTimeout(() => flush(peerId), BATCH_WINDOW_MS);
|
|
128
|
-
timer.unref?.();
|
|
129
|
-
batch = { peer, topics: new Map(), timer };
|
|
130
|
-
pending.set(peerId, batch);
|
|
131
|
-
}
|
|
132
|
-
const waiters = batch.topics.get(topic) ?? [];
|
|
133
|
-
waiters.push({ isLive, resolve, reject });
|
|
134
|
-
batch.topics.set(topic, waiters);
|
|
135
|
-
// A full batch flushes immediately; the next pull opens a fresh window.
|
|
136
|
-
if (batch.topics.size >= MAX_BATCH_ROOT_KEYS)
|
|
137
|
-
flush(peerId);
|
|
138
|
-
})
|
|
139
|
-
};
|
|
140
|
-
}
|
|
@@ -1,9 +0,0 @@
|
|
|
1
|
-
import type { VoteIntent, VoteStore } from "./types.js";
|
|
2
|
-
export declare class IndexedDbVoteStore implements VoteStore {
|
|
3
|
-
#private;
|
|
4
|
-
list(): Promise<VoteIntent[]>;
|
|
5
|
-
get(topic: string): Promise<VoteIntent | undefined>;
|
|
6
|
-
put(intent: VoteIntent): Promise<void>;
|
|
7
|
-
delete(topic: string): Promise<void>;
|
|
8
|
-
destroy(): Promise<void>;
|
|
9
|
-
}
|
package/dist/store/indexeddb.js
DELETED
|
@@ -1,72 +0,0 @@
|
|
|
1
|
-
import { z } from "zod";
|
|
2
|
-
import { VoteSchema } from "../schema/votes.js";
|
|
3
|
-
/**
|
|
4
|
-
* The browser {@link VoteStore} backend: this wallet's re-signable vote intents in an
|
|
5
|
-
* IndexedDB object store, so republishing survives a page reload (see DESIGN.md "Persistence").
|
|
6
|
-
* It holds only *this* voter's choices — never the CRDT of everyone's bundles.
|
|
7
|
-
*
|
|
8
|
-
* Selected by `selectVoteStore` when a global `indexedDB` is present (the browser); Node uses
|
|
9
|
-
* the SQLite backend instead. No third-party dependency — plain IndexedDB behind small
|
|
10
|
-
* promise wrappers. One record per contest, keyed by `topic`; each record is re-validated
|
|
11
|
-
* through {@link VoteSchema} on read so a corrupt entry cannot smuggle a malformed vote back
|
|
12
|
-
* into the signer.
|
|
13
|
-
*/
|
|
14
|
-
const DB_NAME = "bitsocial-pubsub-votes";
|
|
15
|
-
const STORE_NAME = "vote_intents";
|
|
16
|
-
/** The persisted intent shape, re-validated on read (IndexedDB returns `any`). */
|
|
17
|
-
const StoredIntentSchema = z.object({
|
|
18
|
-
topic: z.string(),
|
|
19
|
-
address: z.string(),
|
|
20
|
-
votes: z.array(VoteSchema),
|
|
21
|
-
lastBucket: z.number()
|
|
22
|
-
});
|
|
23
|
-
/** Wrap an IndexedDB request as a promise. */
|
|
24
|
-
function promisify(request) {
|
|
25
|
-
return new Promise((resolve, reject) => {
|
|
26
|
-
request.onsuccess = () => resolve(request.result);
|
|
27
|
-
request.onerror = () => reject(request.error);
|
|
28
|
-
});
|
|
29
|
-
}
|
|
30
|
-
export class IndexedDbVoteStore {
|
|
31
|
-
#db;
|
|
32
|
-
/** Open (creating the object store on first use) the vote-intents database. */
|
|
33
|
-
async #open() {
|
|
34
|
-
if (this.#db !== undefined)
|
|
35
|
-
return this.#db;
|
|
36
|
-
const db = await new Promise((resolve, reject) => {
|
|
37
|
-
const request = indexedDB.open(DB_NAME, 1);
|
|
38
|
-
request.onupgradeneeded = () => {
|
|
39
|
-
if (!request.result.objectStoreNames.contains(STORE_NAME)) {
|
|
40
|
-
request.result.createObjectStore(STORE_NAME, { keyPath: "topic" });
|
|
41
|
-
}
|
|
42
|
-
};
|
|
43
|
-
request.onsuccess = () => resolve(request.result);
|
|
44
|
-
request.onerror = () => reject(request.error);
|
|
45
|
-
});
|
|
46
|
-
this.#db = db;
|
|
47
|
-
return db;
|
|
48
|
-
}
|
|
49
|
-
async #tx(mode, run) {
|
|
50
|
-
const db = await this.#open();
|
|
51
|
-
const store = db.transaction(STORE_NAME, mode).objectStore(STORE_NAME);
|
|
52
|
-
return run(store);
|
|
53
|
-
}
|
|
54
|
-
async list() {
|
|
55
|
-
const rows = await this.#tx("readonly", (store) => promisify(store.getAll()));
|
|
56
|
-
return rows.map((row) => StoredIntentSchema.parse(row));
|
|
57
|
-
}
|
|
58
|
-
async get(topic) {
|
|
59
|
-
const row = await this.#tx("readonly", (store) => promisify(store.get(topic)));
|
|
60
|
-
return row === undefined ? undefined : StoredIntentSchema.parse(row);
|
|
61
|
-
}
|
|
62
|
-
async put(intent) {
|
|
63
|
-
await this.#tx("readwrite", (store) => promisify(store.put(intent)));
|
|
64
|
-
}
|
|
65
|
-
async delete(topic) {
|
|
66
|
-
await this.#tx("readwrite", (store) => promisify(store.delete(topic)));
|
|
67
|
-
}
|
|
68
|
-
async destroy() {
|
|
69
|
-
this.#db?.close();
|
|
70
|
-
this.#db = undefined;
|
|
71
|
-
}
|
|
72
|
-
}
|
package/dist/store/memory.d.ts
DELETED
|
@@ -1,15 +0,0 @@
|
|
|
1
|
-
import type { VoteIntent, VoteStore } from "./types.js";
|
|
2
|
-
/**
|
|
3
|
-
* In-memory {@link VoteStore}: the fallback used on Node when no `dataPath` is given (the
|
|
4
|
-
* durable backends are the browser's IndexedDB and Node's SQLite-under-`dataPath` — see
|
|
5
|
-
* `selectVoteStore` and DESIGN.md "Persistence"). Intents live only for the lifetime of the
|
|
6
|
-
* process, so republishing does NOT survive a restart with this backend. It exists so the
|
|
7
|
-
* voter's lifecycle works with no configured persistence and so unit tests run with no I/O.
|
|
8
|
-
*/
|
|
9
|
-
export declare class MemoryVoteStore implements VoteStore {
|
|
10
|
-
#private;
|
|
11
|
-
list(): Promise<VoteIntent[]>;
|
|
12
|
-
get(topic: string): Promise<VoteIntent | undefined>;
|
|
13
|
-
put(intent: VoteIntent): Promise<void>;
|
|
14
|
-
delete(topic: string): Promise<void>;
|
|
15
|
-
}
|
package/dist/store/memory.js
DELETED
|
@@ -1,22 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* In-memory {@link VoteStore}: the fallback used on Node when no `dataPath` is given (the
|
|
3
|
-
* durable backends are the browser's IndexedDB and Node's SQLite-under-`dataPath` — see
|
|
4
|
-
* `selectVoteStore` and DESIGN.md "Persistence"). Intents live only for the lifetime of the
|
|
5
|
-
* process, so republishing does NOT survive a restart with this backend. It exists so the
|
|
6
|
-
* voter's lifecycle works with no configured persistence and so unit tests run with no I/O.
|
|
7
|
-
*/
|
|
8
|
-
export class MemoryVoteStore {
|
|
9
|
-
#byTopic = new Map();
|
|
10
|
-
async list() {
|
|
11
|
-
return [...this.#byTopic.values()];
|
|
12
|
-
}
|
|
13
|
-
async get(topic) {
|
|
14
|
-
return this.#byTopic.get(topic);
|
|
15
|
-
}
|
|
16
|
-
async put(intent) {
|
|
17
|
-
this.#byTopic.set(intent.topic, intent);
|
|
18
|
-
}
|
|
19
|
-
async delete(topic) {
|
|
20
|
-
this.#byTopic.delete(topic);
|
|
21
|
-
}
|
|
22
|
-
}
|
package/dist/store/select.d.ts
DELETED
|
@@ -1,15 +0,0 @@
|
|
|
1
|
-
import type { VoteStore } from "./types.js";
|
|
2
|
-
/**
|
|
3
|
-
* Pick the vote store for a voter by environment:
|
|
4
|
-
* - a global `indexedDB` (the browser) → the IndexedDB backend;
|
|
5
|
-
* - otherwise Node with a `dataPath` → a **SQLite file under `dataPath`** (WAL mode, the same
|
|
6
|
-
* `dataPath` convention pkc-js and `@bitsocial/bso-resolver` use);
|
|
7
|
-
* - otherwise (Node, no `dataPath`) → in-memory (intents lost on restart).
|
|
8
|
-
*
|
|
9
|
-
* The concrete backends are imported **lazily** (dynamic `import()` inside {@link LazyVoteStore}):
|
|
10
|
-
* the Node backend pulls in the native `better-sqlite3`, which must never enter a browser bundle,
|
|
11
|
-
* and the browser backend touches `indexedDB`, absent on Node. Deferring the import to first use
|
|
12
|
-
* keeps each out of the other environment's module graph while leaving this function synchronous
|
|
13
|
-
* (the `PubsubVoter` constructor stays sync). See DESIGN.md "Persistence".
|
|
14
|
-
*/
|
|
15
|
-
export declare function selectVoteStore(dataPath: string | undefined): VoteStore;
|
package/dist/store/select.js
DELETED
|
@@ -1,64 +0,0 @@
|
|
|
1
|
-
import { MemoryVoteStore } from "./memory.js";
|
|
2
|
-
/**
|
|
3
|
-
* Pick the vote store for a voter by environment:
|
|
4
|
-
* - a global `indexedDB` (the browser) → the IndexedDB backend;
|
|
5
|
-
* - otherwise Node with a `dataPath` → a **SQLite file under `dataPath`** (WAL mode, the same
|
|
6
|
-
* `dataPath` convention pkc-js and `@bitsocial/bso-resolver` use);
|
|
7
|
-
* - otherwise (Node, no `dataPath`) → in-memory (intents lost on restart).
|
|
8
|
-
*
|
|
9
|
-
* The concrete backends are imported **lazily** (dynamic `import()` inside {@link LazyVoteStore}):
|
|
10
|
-
* the Node backend pulls in the native `better-sqlite3`, which must never enter a browser bundle,
|
|
11
|
-
* and the browser backend touches `indexedDB`, absent on Node. Deferring the import to first use
|
|
12
|
-
* keeps each out of the other environment's module graph while leaving this function synchronous
|
|
13
|
-
* (the `PubsubVoter` constructor stays sync). See DESIGN.md "Persistence".
|
|
14
|
-
*/
|
|
15
|
-
export function selectVoteStore(dataPath) {
|
|
16
|
-
if (typeof indexedDB !== "undefined") {
|
|
17
|
-
return new LazyVoteStore(async () => {
|
|
18
|
-
const { IndexedDbVoteStore } = await import("./indexeddb.js");
|
|
19
|
-
return new IndexedDbVoteStore();
|
|
20
|
-
});
|
|
21
|
-
}
|
|
22
|
-
if (dataPath !== undefined) {
|
|
23
|
-
return new LazyVoteStore(async () => {
|
|
24
|
-
const { SqliteVoteStore } = await import("./sqlite.js");
|
|
25
|
-
return new SqliteVoteStore(dataPath);
|
|
26
|
-
});
|
|
27
|
-
}
|
|
28
|
-
return new MemoryVoteStore();
|
|
29
|
-
}
|
|
30
|
-
/**
|
|
31
|
-
* A {@link VoteStore} that defers constructing its backend until the first method call, so the
|
|
32
|
-
* backend module (and its environment-specific dependency) is only `import()`ed when actually
|
|
33
|
-
* used. The factory runs at most once; every method delegates to the resolved backend.
|
|
34
|
-
*/
|
|
35
|
-
class LazyVoteStore {
|
|
36
|
-
factory;
|
|
37
|
-
#backend;
|
|
38
|
-
constructor(factory) {
|
|
39
|
-
this.factory = factory;
|
|
40
|
-
}
|
|
41
|
-
#resolve() {
|
|
42
|
-
if (this.#backend === undefined)
|
|
43
|
-
this.#backend = this.factory();
|
|
44
|
-
return this.#backend;
|
|
45
|
-
}
|
|
46
|
-
async list() {
|
|
47
|
-
return (await this.#resolve()).list();
|
|
48
|
-
}
|
|
49
|
-
async get(topic) {
|
|
50
|
-
return (await this.#resolve()).get(topic);
|
|
51
|
-
}
|
|
52
|
-
async put(intent) {
|
|
53
|
-
return (await this.#resolve()).put(intent);
|
|
54
|
-
}
|
|
55
|
-
async delete(topic) {
|
|
56
|
-
return (await this.#resolve()).delete(topic);
|
|
57
|
-
}
|
|
58
|
-
async destroy() {
|
|
59
|
-
// Nothing to release if the backend was never opened.
|
|
60
|
-
if (this.#backend === undefined)
|
|
61
|
-
return;
|
|
62
|
-
await (await this.#backend).destroy?.();
|
|
63
|
-
}
|
|
64
|
-
}
|
package/dist/store/sqlite.d.ts
DELETED
|
@@ -1,11 +0,0 @@
|
|
|
1
|
-
import type { VoteIntent, VoteStore } from "./types.js";
|
|
2
|
-
export declare class SqliteVoteStore implements VoteStore {
|
|
3
|
-
#private;
|
|
4
|
-
/** Open (creating if absent) the vote-intents DB inside the `dataPath` directory. */
|
|
5
|
-
constructor(dataPath: string);
|
|
6
|
-
list(): Promise<VoteIntent[]>;
|
|
7
|
-
get(topic: string): Promise<VoteIntent | undefined>;
|
|
8
|
-
put(intent: VoteIntent): Promise<void>;
|
|
9
|
-
delete(topic: string): Promise<void>;
|
|
10
|
-
destroy(): Promise<void>;
|
|
11
|
-
}
|
package/dist/store/sqlite.js
DELETED
|
@@ -1,68 +0,0 @@
|
|
|
1
|
-
import { mkdirSync } from "node:fs";
|
|
2
|
-
import { join } from "node:path";
|
|
3
|
-
import Database from "better-sqlite3";
|
|
4
|
-
import { z } from "zod";
|
|
5
|
-
import { VoteSchema } from "../schema/votes.js";
|
|
6
|
-
/**
|
|
7
|
-
* The Node {@link VoteStore} backend: this wallet's re-signable vote intents in a SQLite file
|
|
8
|
-
* under the constructor's `dataPath` directory (WAL mode), so republishing survives a process
|
|
9
|
-
* restart (see DESIGN.md "Persistence"). It holds only *this* voter's choices — never the CRDT
|
|
10
|
-
* of everyone's bundles, which lives in the host's Helia blockstore.
|
|
11
|
-
*
|
|
12
|
-
* `better-sqlite3` is a native Node module, so this file is imported **only dynamically** (via
|
|
13
|
-
* `selectVoteStore`), keeping it out of any browser bundle — the browser uses IndexedDB. One
|
|
14
|
-
* row per contest, keyed by `topic`; `votes` is stored as JSON and re-validated through
|
|
15
|
-
* {@link VoteSchema} on read so a hand-edited or corrupt row cannot smuggle a malformed vote
|
|
16
|
-
* back into the signer.
|
|
17
|
-
*/
|
|
18
|
-
/** The persisted `votes` column shape, re-validated on read (no `any` from `JSON.parse`). */
|
|
19
|
-
const StoredVotesSchema = z.array(VoteSchema);
|
|
20
|
-
/** The SQLite file name kept inside the `dataPath` directory. */
|
|
21
|
-
const DB_FILENAME = "pubsub-votes.sqlite";
|
|
22
|
-
export class SqliteVoteStore {
|
|
23
|
-
#db;
|
|
24
|
-
#listStmt;
|
|
25
|
-
#getStmt;
|
|
26
|
-
#putStmt;
|
|
27
|
-
#deleteStmt;
|
|
28
|
-
/** Open (creating if absent) the vote-intents DB inside the `dataPath` directory. */
|
|
29
|
-
constructor(dataPath) {
|
|
30
|
-
mkdirSync(dataPath, { recursive: true });
|
|
31
|
-
this.#db = new Database(join(dataPath, DB_FILENAME));
|
|
32
|
-
this.#db.pragma("journal_mode = WAL");
|
|
33
|
-
this.#db.exec(`CREATE TABLE IF NOT EXISTS vote_intents (
|
|
34
|
-
topic TEXT PRIMARY KEY,
|
|
35
|
-
address TEXT NOT NULL,
|
|
36
|
-
votes TEXT NOT NULL,
|
|
37
|
-
last_bucket INTEGER NOT NULL
|
|
38
|
-
)`);
|
|
39
|
-
this.#listStmt = this.#db.prepare("SELECT topic, address, votes, last_bucket FROM vote_intents");
|
|
40
|
-
this.#getStmt = this.#db.prepare("SELECT topic, address, votes, last_bucket FROM vote_intents WHERE topic = ?");
|
|
41
|
-
this.#putStmt = this.#db.prepare("INSERT OR REPLACE INTO vote_intents (topic, address, votes, last_bucket) VALUES (?, ?, ?, ?)");
|
|
42
|
-
this.#deleteStmt = this.#db.prepare("DELETE FROM vote_intents WHERE topic = ?");
|
|
43
|
-
}
|
|
44
|
-
#rowToIntent(row) {
|
|
45
|
-
return {
|
|
46
|
-
topic: row.topic,
|
|
47
|
-
address: row.address,
|
|
48
|
-
votes: StoredVotesSchema.parse(JSON.parse(row.votes)),
|
|
49
|
-
lastBucket: row.last_bucket
|
|
50
|
-
};
|
|
51
|
-
}
|
|
52
|
-
async list() {
|
|
53
|
-
return this.#listStmt.all().map((row) => this.#rowToIntent(row));
|
|
54
|
-
}
|
|
55
|
-
async get(topic) {
|
|
56
|
-
const row = this.#getStmt.get(topic);
|
|
57
|
-
return row === undefined ? undefined : this.#rowToIntent(row);
|
|
58
|
-
}
|
|
59
|
-
async put(intent) {
|
|
60
|
-
this.#putStmt.run(intent.topic, intent.address, JSON.stringify(intent.votes), intent.lastBucket);
|
|
61
|
-
}
|
|
62
|
-
async delete(topic) {
|
|
63
|
-
this.#deleteStmt.run(topic);
|
|
64
|
-
}
|
|
65
|
-
async destroy() {
|
|
66
|
-
this.#db.close();
|
|
67
|
-
}
|
|
68
|
-
}
|
package/dist/store/types.d.ts
DELETED
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
import type { Vote } from "../schema/votes.js";
|
|
2
|
-
/**
|
|
3
|
-
* Vote-intent persistence.
|
|
4
|
-
*
|
|
5
|
-
* A live vote decays on its own: a bundle is valid for only `voteExpiryBuckets` after its
|
|
6
|
-
* `blockNumber` (see DESIGN.md "Passive expiry"). Keeping a vote alive means periodically
|
|
7
|
-
* re-signing the *same choice* with a fresh `blockNumber` and re-broadcasting it. To do
|
|
8
|
-
* that across a process restart the voter must remember what it chose — but not the signed
|
|
9
|
-
* bundles (those are immutable, content-addressed, and live in the host's Helia blockstore;
|
|
10
|
-
* a stale `blockNumber` makes an old bundle useless). What it persists is the re-signable
|
|
11
|
-
* *intent*: which communities this wallet picked in which contest. On `start()` the voter loads
|
|
12
|
-
* every stored intent and republishes it; the republish scheduler re-signs each on the
|
|
13
|
-
* liveness cadence (`ceil(voteExpiryBuckets / 2)` buckets — see DESIGN.md "Lifecycle").
|
|
14
|
-
*
|
|
15
|
-
* This is a plain key-value contract, keyed by `topic` (one intent per contest per wallet).
|
|
16
|
-
* It is internal: the library picks the backend by environment — IndexedDB in the browser,
|
|
17
|
-
* a SQLite file under the constructor's `dataPath` on Node — with no host-facing store seam.
|
|
18
|
-
* Declared as its own interface (like `BundleStore` in crdt/types.ts) so the voter can be
|
|
19
|
-
* tested against an in-memory store with no I/O.
|
|
20
|
-
*/
|
|
21
|
-
/** One contest's re-signable (or, when empty, re-announceable) choice for this wallet. */
|
|
22
|
-
export interface VoteIntent {
|
|
23
|
-
/** The gossipsub topic this intent votes in = "bitsocial-votes/" + CID(dag-cbor(criteria)). */
|
|
24
|
-
topic: string;
|
|
25
|
-
/** The voting wallet address (recovered from the bundle signature). One intent per topic per address. */
|
|
26
|
-
address: string;
|
|
27
|
-
/**
|
|
28
|
-
* The communities this wallet chose. A non-empty intent is an active vote the scheduler re-signs
|
|
29
|
-
* (fresh `blockNumber`) each cadence to keep alive. An **empty array is a withdrawal
|
|
30
|
-
* tombstone**: the empty bundle supersedes the prior vote under LWW, and the scheduler
|
|
31
|
-
* re-announces its existing CID (never re-signing it) each cadence until it expires, then
|
|
32
|
-
* deletes the intent — the bundle decays on its own via expiry + prune. See DESIGN.md
|
|
33
|
-
* "Cancelling a vote".
|
|
34
|
-
*/
|
|
35
|
-
votes: Vote[];
|
|
36
|
-
/** The bucket of the last successful republish, so the scheduler knows when the next is due. */
|
|
37
|
-
lastBucket: number;
|
|
38
|
-
}
|
|
39
|
-
/**
|
|
40
|
-
* Where the voter keeps its own vote intents. Only ever holds *this* voter's choices, not
|
|
41
|
-
* the CRDT of everyone's bundles. `list` feeds the republish loop on `start()`; `put` is
|
|
42
|
-
* written on every cast and every successful republish; `delete` drops an intent that has
|
|
43
|
-
* expired or been explicitly cancelled. `destroy` releases backend handles (close the DB
|
|
44
|
-
* or file) and is called by `PubsubVoter.destroy()`.
|
|
45
|
-
*/
|
|
46
|
-
export interface VoteStore {
|
|
47
|
-
/** Every intent to keep alive. Called once on `start()` to seed the republish scheduler. */
|
|
48
|
-
list(): Promise<VoteIntent[]>;
|
|
49
|
-
/** The intent for one contest, or `undefined` if this wallet has not voted there. */
|
|
50
|
-
get(topic: string): Promise<VoteIntent | undefined>;
|
|
51
|
-
/** Insert or replace one contest's intent (last write wins, mirroring the CRDT). */
|
|
52
|
-
put(intent: VoteIntent): Promise<void>;
|
|
53
|
-
/** Drop one contest's intent (expiry or explicit cancel). Idempotent. */
|
|
54
|
-
delete(topic: string): Promise<void>;
|
|
55
|
-
/** Release backend handles. Optional: the in-memory store has nothing to release. */
|
|
56
|
-
destroy?(): Promise<void>;
|
|
57
|
-
}
|
package/dist/store/types.js
DELETED
|
@@ -1 +0,0 @@
|
|
|
1
|
-
export {};
|