@steve02081504/fount-p2p 0.0.44 → 0.0.48
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +17 -4
- package/dag/storage.mjs +157 -79
- package/discovery/nostr/census.mjs +5 -5
- package/discovery/nostr/census_monitor.mjs +2 -2
- package/discovery/nostr/constants.mjs +1 -0
- package/discovery/nostr/index.mjs +24 -12
- package/discovery/nostr/session.mjs +96 -12
- package/docs/nostr_relay_discovery.md +2 -0
- package/docs/signaling.md +15 -3
- package/docs/transports.md +15 -1
- package/federation/part_query/runtime.mjs +4 -4
- package/index.mjs +5 -0
- package/link/providers/ble_gatt.mjs +2 -2
- package/link/providers/index.mjs +2 -1
- package/link/providers/nostr/index.mjs +19 -6
- package/link/providers/webrtc.mjs +235 -56
- package/link/rtc/polyfill.mjs +9 -6
- package/mailbox/store.mjs +3 -22
- package/node/signaling_config.mjs +30 -8
- package/node/verification.mjs +163 -0
- package/package.json +4 -2
- package/sim/AGENTS.md +1 -1
- package/sim/{cold_start.md → docs/cold_start.md} +1 -1
- package/timeline/prune.mjs +27 -23
- package/timeline/retention.mjs +38 -34
- package/transport/link_registry.mjs +75 -6
- package/transport/peer_health.mjs +43 -25
- package/transport/runtime_bootstrap.mjs +44 -9
- package/utils/async_mutex.mjs +11 -4
- package/utils/shuffle.mjs +1 -1
- package/wire/adapter.mjs +1 -1
- package/pages/index.html +0 -36
- package/pages/index.mjs +0 -62
package/AGENTS.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
## Package layers (`@steve02081504/fount-p2p`)
|
|
4
4
|
|
|
5
5
|
| Layer | Directory | Role |
|
|
6
|
-
|
|
6
|
+
| --- | --- | --- |
|
|
7
7
|
| L0 | `core/` | IDs, logical entity, canonical JSON, bytes |
|
|
8
8
|
| L1 | `crypto/`, `wire/`, `schemas/` | Crypto, wire ingress, canonical validation |
|
|
9
9
|
| L2 | `node/` | `initNode`, identity, entity store, denylist, reputation, storage plugins |
|
|
@@ -19,18 +19,19 @@ Detail docs: [transports](docs/transports.md) · [mesh](docs/mesh.md) · [signal
|
|
|
19
19
|
### Runtime: isomorphic vs Node
|
|
20
20
|
|
|
21
21
|
| Surface | Runtime | Notes |
|
|
22
|
-
|
|
22
|
+
| --- | --- | --- |
|
|
23
23
|
| `core/*` | Node + browser | No `node:*` builtins |
|
|
24
24
|
| `crypto/crypto.mjs` | Node + browser | `@noble/hashes` + `@noble/curves` only — never `node:crypto` |
|
|
25
25
|
| `crypto/key.mjs` / `crypto/channel.mjs`, disk I/O, LAN/BT, `ws`, CLI / `startNode` | Node (+ Deno bridge) | Do not load the whole package via esm.sh in the browser |
|
|
26
26
|
|
|
27
|
-
**Browser pages reusing package code:** pages import browser-safe package modules (`core/*`, `crypto/crypto.mjs`, `discovery/nostr/constants|_math|_verify|_monitor`) from esm.sh; their bare `@noble/*` deps resolve via an injected importmap. The frontend static test server (`test/frontend/census_page.test.mjs`) serves the
|
|
27
|
+
**Browser pages reusing package code:** pages import browser-safe package modules (`core/*`, `crypto/crypto.mjs`, `discovery/nostr/constants|_math|_verify|_monitor`) from esm.sh; their bare `@noble/*` deps resolve via an injected importmap. The frontend static test server (`test/frontend/census_page.test.mjs`) serves the monorepo root (offline) — the page lives in `pages/`, the package in `js/` — injects an importmap into served HTML (`https://esm.sh/@steve02081504/fount-p2p/` → local `js/` package root, `@noble/` → `https://esm.sh/@noble/`), and runs the page live against a fake relay (`test/helpers/fake_relay.mjs` with `store`), injecting its `ws://` URL via `?relay=` — no demo mode. Page code stays a thin shell — pass display fn + optional relays; logic lives in the package (`discovery/nostr/census_monitor.mjs`).
|
|
28
28
|
|
|
29
29
|
Deno / native / BT: [runtime.md](docs/runtime.md).
|
|
30
30
|
|
|
31
31
|
## Conventions
|
|
32
32
|
|
|
33
33
|
- Prefer shared helpers under `utils/`, `core/`, `wire/subscribe`, `wire/adapter` — do not reimplement LRU/TTL/inflight/atomic-fs/shuffle.
|
|
34
|
+
- **JSONL read-modify-write:** do it inside `withAsyncMutex(jsonlMutexKey(path))`. `rewriteJsonlKeeping` / `appendJsonlSynced` / `writeJsonlSynced` take that per-file mutex themselves and are **not reentrant** — never call them while already holding the same key (use lock-free `writeJsonl` / `writeJsonlLines` instead). Rewrites preserve the original raw line (a `sanitize` passed to `rewriteJsonlKeeping` / `readJsonlEntries` only affects the value handed to the predicate) and skip the rename when nothing is dropped.
|
|
34
35
|
- **No pure-forward aliases:** do not add `fooText`/`fooAlias` that only `return foo(sameArgs)` when the callee already accepts those types (e.g. never wrap `sha256Hex` as `sha256TextHex`). Domain names must add logic or type narrowing, not just rename.
|
|
35
36
|
- **Heterogeneous backends:** normalize at the load boundary (e.g. `link/rtc/ice_local_hostname.mjs` wraps W3C RTC backends); call sites speak one contract.
|
|
36
37
|
- **File naming:** parent directory is scope — short child names. Tunables default `<dir>/tunables.json` (exception: `schemas/part_query.tunables.json`). Subpath `package.json` exports mirror filenames.
|
|
@@ -46,12 +47,24 @@ Deno / native / BT: [runtime.md](docs/runtime.md).
|
|
|
46
47
|
- `npm run test:live` — live link / LAN smoke
|
|
47
48
|
- `npm run test:fount` — cross-repo Deno bridge (`test/fount/`); see [runtime.md](docs/runtime.md)
|
|
48
49
|
- `npm run test:sim` — tunables co-evolution (dev-only; [sim/AGENTS.md](sim/AGENTS.md))
|
|
50
|
+
- `npm run test:checks` — the three static-health suites below only
|
|
51
|
+
- `npm run check:text_lf:fix` — rewrite line-ending violations (the suites themselves never write)
|
|
49
52
|
- `node scripts/check-imports.mjs` — relative import check
|
|
50
53
|
- `node scripts/find-unused-exports.mjs` — dead-export scan (`--fount <path>` optional)
|
|
51
54
|
- Assertions: `test/helpers/assert.mjs`
|
|
52
55
|
- Fixed-seed identity: `test/helpers/identity.mjs`
|
|
53
56
|
- Mock discovery: `test/helpers/mock_discovery.mjs`
|
|
54
57
|
|
|
58
|
+
## Static checks (`scripts/checks/`, ported from fount's check standard)
|
|
59
|
+
|
|
60
|
+
| Check | Enforces |
|
|
61
|
+
| --- | --- |
|
|
62
|
+
| `text_lf` | Every UTF-8 text file (fatal decode, no NUL / C0 control byte; empty files exempt) uses LF, ends with exactly one LF (single-line `.svg` instead ends with none), and does not start with LF (leading UTF-8 BOM skipped). Binary files such as `gradle-wrapper.jar` are skipped. Scanner: `scripts/checks/text_lf.mjs`; `npm run check:text_lf:fix` writes the fix |
|
|
63
|
+
| `jsdoc_no_english` | JSDoc summaries are Chinese (CJK required; a pure-English summary or a missing summary fails); multi-line blocks put a line break right after `/**` and before `*/`. Tag-only blocks (`@param` / `@typedef` …) are fine, empty `/** */` stubs are not. Scanner: `scripts/checks/jsdoc_no_english.mjs` |
|
|
64
|
+
| `agents_md_english` | Every `AGENTS.md` and each `.md` reachable through its local links is English (no CJK) and its links resolve; a non-`AGENTS.md` doc in that closure must live under a `docs/` directory. Scanner: `scripts/checks/agents_md_english.mjs` |
|
|
65
|
+
|
|
66
|
+
The scanners run from `js/test/pure/*.test.mjs` (repo-wide scope: they walk up to the monorepo root, so `kotlin/AGENTS.md` and the root docs are covered too). Docs language is unified to English — write agent docs and `.md` in the AGENTS.md closure in English.
|
|
67
|
+
|
|
55
68
|
## Hard rules
|
|
56
69
|
|
|
57
70
|
### Trust / ingress
|
|
@@ -76,7 +89,7 @@ Deno / native / BT: [runtime.md](docs/runtime.md).
|
|
|
76
89
|
## Tunables JSON
|
|
77
90
|
|
|
78
91
|
| File | Directory |
|
|
79
|
-
|
|
92
|
+
| --- | --- |
|
|
80
93
|
| `tunables.json` | `transport/`, `infra/`, `reputation/`, `trust_graph/`, `mailbox/`, `governance/`, `dag/` |
|
|
81
94
|
| `part_query.tunables.json` | `schemas/` |
|
|
82
95
|
|
package/dag/storage.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { Buffer } from 'node:buffer'
|
|
2
2
|
import { createReadStream, createWriteStream } from 'node:fs'
|
|
3
|
-
import { appendFile, mkdir, open,
|
|
4
|
-
import { dirname } from 'node:path'
|
|
3
|
+
import { appendFile, mkdir, open, unlink, writeFile } from 'node:fs/promises'
|
|
4
|
+
import { dirname, resolve } from 'node:path'
|
|
5
5
|
import { createInterface } from 'node:readline'
|
|
6
6
|
import { Readable } from 'node:stream'
|
|
7
7
|
import { pipeline } from 'node:stream/promises'
|
|
@@ -29,26 +29,52 @@ function parseJsonlLine(line, sanitize) {
|
|
|
29
29
|
}
|
|
30
30
|
|
|
31
31
|
/**
|
|
32
|
-
*
|
|
32
|
+
* 逐行读取 JSONL 原始行(不解析、不净化)。文件缺失(ENOENT)视为空流,
|
|
33
|
+
* 兼容 cleanup 竞态:群目录被删后台仍在尾巴上读它;其他读错误抛出。
|
|
34
|
+
* @param {string} filePath 文件路径
|
|
35
|
+
* @returns {AsyncGenerator<string>} 原始行(readline 已去掉行尾换行)
|
|
36
|
+
*/
|
|
37
|
+
async function* readJsonlRawLines(filePath) {
|
|
38
|
+
const input = createReadStream(filePath, { encoding: 'utf8' })
|
|
39
|
+
// stream 内部异步 open 失败会触发 'error' 事件;提前订阅避免 unhandled error,
|
|
40
|
+
// 真实错误仍由下方 for-await 抛出,由本函数统一收口。
|
|
41
|
+
input.on('error', () => { })
|
|
42
|
+
const lines = createInterface({ input, crlfDelay: Infinity })
|
|
43
|
+
try {
|
|
44
|
+
for await (const line of lines) yield line
|
|
45
|
+
}
|
|
46
|
+
catch (error) {
|
|
47
|
+
if (error?.code !== 'ENOENT') throw error
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
/**
|
|
52
|
+
* 读取 JSONL 为 `{ row, raw }` 列表,保留原始行以便无损写回。
|
|
53
|
+
* 空行/解析失败/净化抛错的行跳过;文件缺失(ENOENT)返回空数组,其他读错误抛出。
|
|
54
|
+
* @param {string} filePath 文件路径
|
|
55
|
+
* @param {{ sanitize?: (row: object) => object }} [options] 行净化
|
|
56
|
+
* @returns {Promise<Array<{ row: object, raw: string }>>} 行条目列表
|
|
57
|
+
*/
|
|
58
|
+
export async function readJsonlEntries(filePath, options = {}) {
|
|
59
|
+
const sanitize = options.sanitize ?? (row => row)
|
|
60
|
+
/** @type {Array<{ row: object, raw: string }>} */
|
|
61
|
+
const entries = []
|
|
62
|
+
for await (const raw of readJsonlRawLines(filePath)) {
|
|
63
|
+
const row = parseJsonlLine(raw, sanitize)
|
|
64
|
+
if (row != null) entries.push({ row, raw })
|
|
65
|
+
}
|
|
66
|
+
return entries
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
/**
|
|
70
|
+
* 读取 JSONL 文件并解析为对象数组;缺失(ENOENT)返回空数组,其他读错误抛出。
|
|
33
71
|
* @param {string} filePath 文件系统路径
|
|
34
72
|
* @param {{ sanitize?: (row: object) => object }} [options] 可选净化函数
|
|
35
73
|
* @returns {Promise<object[]>} 各行解析后的对象列表
|
|
36
74
|
*/
|
|
37
75
|
export async function readJsonl(filePath, options = {}) {
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
const sanitize = options.sanitize ?? (row => row)
|
|
41
|
-
/** @type {object[]} */
|
|
42
|
-
const rows = []
|
|
43
|
-
for (const line of text.split('\n')) {
|
|
44
|
-
const row = parseJsonlLine(line, sanitize)
|
|
45
|
-
if (row) rows.push(row)
|
|
46
|
-
}
|
|
47
|
-
return rows
|
|
48
|
-
}
|
|
49
|
-
catch {
|
|
50
|
-
return []
|
|
51
|
-
}
|
|
76
|
+
const entries = await readJsonlEntries(filePath, options)
|
|
77
|
+
return entries.map(entry => entry.row).filter(Boolean)
|
|
52
78
|
}
|
|
53
79
|
|
|
54
80
|
/**
|
|
@@ -60,69 +86,93 @@ export async function readJsonl(filePath, options = {}) {
|
|
|
60
86
|
*/
|
|
61
87
|
export async function* readJsonlStream(filePath, options = {}) {
|
|
62
88
|
const sanitize = options.sanitize ?? (row => row)
|
|
63
|
-
const
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
input.on('error', () => { })
|
|
67
|
-
const lines = createInterface({ input, crlfDelay: Infinity })
|
|
68
|
-
try {
|
|
69
|
-
for await (const line of lines) {
|
|
70
|
-
const row = parseJsonlLine(line, sanitize)
|
|
71
|
-
if (row) yield row
|
|
72
|
-
}
|
|
73
|
-
}
|
|
74
|
-
catch (error) {
|
|
75
|
-
if (error?.code !== 'ENOENT') throw error
|
|
89
|
+
for await (const line of readJsonlRawLines(filePath)) {
|
|
90
|
+
const row = parseJsonlLine(line, sanitize)
|
|
91
|
+
if (row) yield row
|
|
76
92
|
}
|
|
77
93
|
}
|
|
78
94
|
|
|
79
95
|
/**
|
|
80
|
-
* 流式过滤重写 JSONL:保留 `keep(row)===true`
|
|
96
|
+
* 流式过滤重写 JSONL:保留 `keep(row)===true` 的行,写回的是原始行字节,
|
|
97
|
+
* `sanitize` 只影响传给 `keep` 的值。
|
|
98
|
+
*
|
|
99
|
+
* 在 `jsonlMutexKey(filePath)` 互斥锁内执行;调用方不得已持有同一文件的锁(不可重入)。
|
|
81
100
|
* @param {string} filePath 目标路径
|
|
82
|
-
* @param {(row: object) => boolean} keep
|
|
101
|
+
* @param {(row: object) => boolean | Promise<boolean>} keep 保留谓词(可异步)
|
|
83
102
|
* @param {{ sanitize?: (row: object) => object }} [options] 读行净化
|
|
84
103
|
* @returns {Promise<{ kept: number, dropped: number }>} 统计
|
|
85
104
|
*/
|
|
86
105
|
export async function rewriteJsonlKeeping(filePath, keep, options = {}) {
|
|
87
|
-
|
|
88
|
-
|
|
106
|
+
return withAsyncMutex(jsonlMutexKey(filePath), () => rewriteJsonlKeepingUnlocked(filePath, keep, options))
|
|
107
|
+
}
|
|
108
|
+
|
|
109
|
+
/**
|
|
110
|
+
* `rewriteJsonlKeeping` 的无锁实现,调用方须已持有该文件互斥锁。
|
|
111
|
+
* @param {string} filePath 目标路径
|
|
112
|
+
* @param {(row: object) => boolean | Promise<boolean>} keep 保留谓词(可异步)
|
|
113
|
+
* @param {{ sanitize?: (row: object) => object }} [options] 读行净化
|
|
114
|
+
* @returns {Promise<{ kept: number, dropped: number }>} 统计
|
|
115
|
+
*/
|
|
116
|
+
async function rewriteJsonlKeepingUnlocked(filePath, keep, options = {}) {
|
|
117
|
+
const sanitize = options.sanitize ?? (row => row)
|
|
89
118
|
const temporaryPath = atomicTemporaryPath(filePath)
|
|
90
|
-
|
|
119
|
+
let tempCreated = false
|
|
120
|
+
/** @type {string[]} */
|
|
91
121
|
const buffer = []
|
|
92
122
|
let kept = 0
|
|
93
123
|
let dropped = 0
|
|
94
|
-
/** @returns {Promise<void>} */
|
|
124
|
+
/** @returns {Promise<void>} 将缓冲区原始行写入临时文件 */
|
|
95
125
|
const flush = async () => {
|
|
96
126
|
if (!buffer.length) return
|
|
97
|
-
|
|
98
|
-
for (const row of buffer)
|
|
99
|
-
block += `${JSON.stringify(row)}\n`
|
|
127
|
+
const block = buffer.map(line => `${line}\n`).join('')
|
|
100
128
|
await appendFile(temporaryPath, block, 'utf8')
|
|
129
|
+
tempCreated = true
|
|
101
130
|
buffer.length = 0
|
|
102
131
|
}
|
|
103
132
|
try {
|
|
104
|
-
for await (const
|
|
105
|
-
if (
|
|
106
|
-
|
|
133
|
+
for await (const raw of readJsonlRawLines(filePath)) {
|
|
134
|
+
if (!raw.trim()) continue
|
|
135
|
+
const row = parseJsonlLine(raw, sanitize)
|
|
136
|
+
if (row == null) {
|
|
137
|
+
dropped++
|
|
138
|
+
continue
|
|
139
|
+
}
|
|
140
|
+
if (await keep(row)) {
|
|
141
|
+
buffer.push(raw)
|
|
107
142
|
kept++
|
|
108
|
-
if (buffer.length >= WRITE_JSONL_CHUNK_LINES)
|
|
109
|
-
await flush()
|
|
143
|
+
if (buffer.length >= WRITE_JSONL_CHUNK_LINES) await flush()
|
|
110
144
|
}
|
|
111
145
|
else dropped++
|
|
146
|
+
}
|
|
112
147
|
await flush()
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
148
|
+
if (dropped === 0) {
|
|
149
|
+
if (tempCreated) await unlink(temporaryPath).catch(() => { })
|
|
150
|
+
return { kept, dropped }
|
|
151
|
+
}
|
|
152
|
+
await mkdir(dirname(filePath), { recursive: true })
|
|
153
|
+
if (!tempCreated) {
|
|
154
|
+
await writeFile(temporaryPath, '', 'utf8')
|
|
155
|
+
tempCreated = true
|
|
156
|
+
}
|
|
157
|
+
const fileHandle = await open(temporaryPath, 'r+')
|
|
158
|
+
try {
|
|
159
|
+
await fileHandle.sync()
|
|
160
|
+
}
|
|
161
|
+
finally {
|
|
162
|
+
await fileHandle.close()
|
|
163
|
+
}
|
|
116
164
|
await finalizeAtomicRename(temporaryPath, filePath)
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
165
|
+
return { kept, dropped }
|
|
166
|
+
}
|
|
167
|
+
catch (error) {
|
|
168
|
+
if (tempCreated) await unlink(temporaryPath).catch(() => { })
|
|
169
|
+
throw error
|
|
170
|
+
}
|
|
122
171
|
}
|
|
123
172
|
|
|
124
173
|
/**
|
|
125
|
-
* 读取 JSONL
|
|
174
|
+
* 读取 JSONL 末条可解析事件的 `id`(DAG tip);从文件尾部最多读 1 MiB,
|
|
175
|
+
* 跳过撕裂/坏行后取最后一个非空且 `id` 非 null 的行;空文件或缺失为 null。
|
|
126
176
|
* @param {string} filePath 文件路径
|
|
127
177
|
* @returns {Promise<string | null>} tip event id
|
|
128
178
|
*/
|
|
@@ -132,14 +182,20 @@ export async function readJsonlTipId(filePath) {
|
|
|
132
182
|
try {
|
|
133
183
|
const { size } = await fh.stat()
|
|
134
184
|
if (!size) return null
|
|
135
|
-
const chunk = Math.min(size,
|
|
185
|
+
const chunk = Math.min(size, 1_048_576)
|
|
136
186
|
const buffer = Buffer.alloc(chunk)
|
|
137
187
|
await fh.read(buffer, 0, chunk, size - chunk)
|
|
138
|
-
const lines = buffer.toString('utf8').split('\n')
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
188
|
+
const lines = buffer.toString('utf8').split('\n')
|
|
189
|
+
for (let index = lines.length - 1; index >= 0; index--) {
|
|
190
|
+
const line = lines[index].trim()
|
|
191
|
+
if (!line) continue
|
|
192
|
+
try {
|
|
193
|
+
const row = JSON.parse(line)
|
|
194
|
+
if (row && typeof row === 'object' && row.id != null) return String(row.id)
|
|
195
|
+
}
|
|
196
|
+
catch { /* 撕裂/坏行:继续向前找 */ }
|
|
197
|
+
}
|
|
198
|
+
return null
|
|
143
199
|
}
|
|
144
200
|
finally {
|
|
145
201
|
await fh.close()
|
|
@@ -151,30 +207,50 @@ export async function readJsonlTipId(filePath) {
|
|
|
151
207
|
}
|
|
152
208
|
|
|
153
209
|
/**
|
|
154
|
-
*
|
|
210
|
+
* 流式写入原始 JSONL 行(临时文件 + fsync + rename),每行自动补 `\n`。
|
|
211
|
+
* 不取锁——调用方需自行持有 `jsonlMutexKey(filePath)`。
|
|
155
212
|
* @param {string} filePath 目标路径
|
|
156
|
-
* @param {
|
|
213
|
+
* @param {Iterable<string>} lines 原始行(不含换行)
|
|
157
214
|
* @returns {Promise<void>}
|
|
158
215
|
*/
|
|
159
|
-
export async function
|
|
216
|
+
export async function writeJsonlLines(filePath, lines) {
|
|
160
217
|
const dir = dirname(filePath)
|
|
161
218
|
await mkdir(dir, { recursive: true })
|
|
162
219
|
const temporaryPath = atomicTemporaryPath(filePath)
|
|
163
|
-
/** @returns {Generator<string>}
|
|
164
|
-
function*
|
|
165
|
-
for (const
|
|
166
|
-
yield `${
|
|
220
|
+
/** @returns {Generator<string>} 带换行的行 */
|
|
221
|
+
function* withEol() {
|
|
222
|
+
for (const line of lines)
|
|
223
|
+
yield `${line}\n`
|
|
224
|
+
}
|
|
225
|
+
await pipeline(Readable.from(withEol()), createWriteStream(temporaryPath, { encoding: 'utf8' }))
|
|
226
|
+
const fileHandle = await open(temporaryPath, 'r+')
|
|
227
|
+
try {
|
|
228
|
+
await fileHandle.sync()
|
|
229
|
+
}
|
|
230
|
+
finally {
|
|
231
|
+
await fileHandle.close()
|
|
167
232
|
}
|
|
168
|
-
await pipeline(Readable.from(lines()), createWriteStream(temporaryPath, { encoding: 'utf8' }))
|
|
169
233
|
await finalizeAtomicRename(temporaryPath, filePath)
|
|
170
234
|
}
|
|
171
235
|
|
|
236
|
+
/**
|
|
237
|
+
* 流式重写 JSONL(临时文件 + fsync + rename),避免大数组 join 的内存峰值。
|
|
238
|
+
* 不取锁——`mailbox/store.mjs` 等调用方在自己的 `jsonlMutexKey` 锁内调用;
|
|
239
|
+
* 需要锁时用 `writeJsonlSynced`。
|
|
240
|
+
* @param {string} filePath 目标路径
|
|
241
|
+
* @param {object[]} records 行对象列表
|
|
242
|
+
* @returns {Promise<void>}
|
|
243
|
+
*/
|
|
244
|
+
export async function writeJsonl(filePath, records) {
|
|
245
|
+
return writeJsonlLines(filePath, records.map(record => JSON.stringify(record)))
|
|
246
|
+
}
|
|
247
|
+
|
|
172
248
|
/**
|
|
173
249
|
* @param {string} filePath JSONL 路径
|
|
174
|
-
* @returns {string}
|
|
250
|
+
* @returns {string} 进程内互斥键(路径已规范化为绝对路径)
|
|
175
251
|
*/
|
|
176
252
|
export function jsonlMutexKey(filePath) {
|
|
177
|
-
return `jsonl:${filePath}`
|
|
253
|
+
return `jsonl:${resolve(filePath)}`
|
|
178
254
|
}
|
|
179
255
|
|
|
180
256
|
/**
|
|
@@ -188,21 +264,23 @@ export async function writeJsonlSynced(filePath, records) {
|
|
|
188
264
|
}
|
|
189
265
|
|
|
190
266
|
/**
|
|
191
|
-
* 追加一行 JSONL 并 `fsync
|
|
267
|
+
* 追加一行 JSONL 并 `fsync`;与 rewrite 共用同一 per-file 互斥锁。
|
|
192
268
|
* @param {string} filePath 目标路径
|
|
193
269
|
* @param {object} record 记录对象
|
|
194
270
|
* @returns {Promise<void>}
|
|
195
271
|
*/
|
|
196
272
|
export async function appendJsonlSynced(filePath, record) {
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
273
|
+
return withAsyncMutex(jsonlMutexKey(filePath), async () => {
|
|
274
|
+
await mkdir(dirname(filePath), { recursive: true })
|
|
275
|
+
const fh = await open(filePath, 'a')
|
|
276
|
+
try {
|
|
277
|
+
await fh.appendFile(`${JSON.stringify(record)}\n`, 'utf8')
|
|
278
|
+
await fh.sync()
|
|
279
|
+
}
|
|
280
|
+
finally {
|
|
281
|
+
await fh.close()
|
|
282
|
+
}
|
|
283
|
+
})
|
|
206
284
|
}
|
|
207
285
|
|
|
208
286
|
/**
|
|
@@ -21,17 +21,17 @@ import { ensureNodeSeed, getNodeHash } from '../../node/identity.mjs'
|
|
|
21
21
|
import { getP2PFeatures, isNodeInitialized } from '../../node/instance.mjs'
|
|
22
22
|
import { nodeDebug } from '../../node/log.mjs'
|
|
23
23
|
|
|
24
|
-
import {
|
|
25
|
-
CENSUS_TAG_FOUNT,
|
|
26
|
-
CENSUS_TAG_X,
|
|
27
|
-
NOSTR_CENSUS_KIND,
|
|
28
|
-
} from './constants.mjs'
|
|
29
24
|
import {
|
|
30
25
|
CENSUS_TARGET_EVENTS,
|
|
31
26
|
estimatePopulation,
|
|
32
27
|
nextInclusionProbability,
|
|
33
28
|
} from './census_math.mjs'
|
|
34
29
|
import { CENSUS_TTL_MS, buildCensusMessage, verifyCensusBytes } from './census_verify.mjs'
|
|
30
|
+
import {
|
|
31
|
+
CENSUS_TAG_FOUNT,
|
|
32
|
+
CENSUS_TAG_X,
|
|
33
|
+
NOSTR_CENSUS_KIND,
|
|
34
|
+
} from './constants.mjs'
|
|
35
35
|
import { resolveRelayConnectTarget } from './relays.mjs'
|
|
36
36
|
|
|
37
37
|
/** census 订阅标签数组(发布用)。 */
|
|
@@ -10,6 +10,8 @@
|
|
|
10
10
|
*/
|
|
11
11
|
import { base64ToBytes } from '../../core/bytes_codec.mjs'
|
|
12
12
|
|
|
13
|
+
import { estimatePopulation } from './census_math.mjs'
|
|
14
|
+
import { CENSUS_TTL_MS, verifyCensusBytes } from './census_verify.mjs'
|
|
13
15
|
import {
|
|
14
16
|
CENSUS_TAG_FOUNT,
|
|
15
17
|
CENSUS_TAG_X,
|
|
@@ -17,8 +19,6 @@ import {
|
|
|
17
19
|
NIP66_BOOTSTRAP_RELAYS,
|
|
18
20
|
NOSTR_CENSUS_KIND,
|
|
19
21
|
} from './constants.mjs'
|
|
20
|
-
import { estimatePopulation } from './census_math.mjs'
|
|
21
|
-
import { CENSUS_TTL_MS, verifyCensusBytes } from './census_verify.mjs'
|
|
22
22
|
|
|
23
23
|
/** NIP-66 relay 注册 kind。 */
|
|
24
24
|
const NIP66_KIND = 30166
|
|
@@ -57,4 +57,5 @@ export const NOSTR_CENSUS_KIND = 30789
|
|
|
57
57
|
|
|
58
58
|
/** census 订阅/发布标签:`t=fount` + `x=census`(subscribeNostrKind 以 rendezvousKey/tagX 匹配)。 */
|
|
59
59
|
export const CENSUS_TAG_FOUNT = 'fount'
|
|
60
|
+
/** census 标签 `x` 的取值(`t=fount` + `x=census` 共同标识 census 事件)。 */
|
|
60
61
|
export const CENSUS_TAG_X = 'census'
|
|
@@ -18,8 +18,8 @@ import {
|
|
|
18
18
|
} from '../internal/signal_crypto.mjs'
|
|
19
19
|
import { noteDiscoveryPeerClue } from '../peer_clue.mjs'
|
|
20
20
|
|
|
21
|
-
import { DEFAULT_RELAY_URLS } from './constants.mjs'
|
|
22
21
|
import { createNostrCensus } from './census.mjs'
|
|
22
|
+
import { DEFAULT_RELAY_URLS } from './constants.mjs'
|
|
23
23
|
import {
|
|
24
24
|
getListenRelays,
|
|
25
25
|
getPeerRoute,
|
|
@@ -203,7 +203,8 @@ async function signNostrEvent(kind, tags, content, secretKey) {
|
|
|
203
203
|
}
|
|
204
204
|
|
|
205
205
|
/**
|
|
206
|
-
* 全量发布到给定 relay
|
|
206
|
+
* 全量发布到给定 relay:任一成功即返回,其余 relay 的发布留在后台自行结算。
|
|
207
|
+
* 每个 relay 尝试都有界(连接超时 / OK 超时 / 入队等待上限),故后台不会永久悬挂。
|
|
207
208
|
* @param {string[]} relayUrls 中继 URL 列表
|
|
208
209
|
* @param {object} event 待发布事件
|
|
209
210
|
* @param {AbortSignal} [signal] 取消信号
|
|
@@ -212,19 +213,30 @@ async function signNostrEvent(kind, tags, content, secretKey) {
|
|
|
212
213
|
async function publishEvent(relayUrls, event, signal) {
|
|
213
214
|
const urls = dedupeRelayUrls(relayUrls)
|
|
214
215
|
if (!urls.length) throw new Error('nostr: no relay')
|
|
215
|
-
|
|
216
|
-
let lastError = null
|
|
217
|
-
await Promise.allSettled(urls.map(async relayUrl => {
|
|
216
|
+
const targets = (await Promise.all(urls.map(async relayUrl => {
|
|
218
217
|
try {
|
|
219
|
-
|
|
220
|
-
if (!connectTarget) return
|
|
221
|
-
if (await publishViaSharedRelay(relayUrl, event, signal, connectTarget)) published = true
|
|
218
|
+
return { relayUrl, connectTarget: await resolveRelayConnectTarget(relayUrl) }
|
|
222
219
|
}
|
|
223
|
-
catch
|
|
224
|
-
|
|
220
|
+
catch {
|
|
221
|
+
return { relayUrl, connectTarget: null }
|
|
225
222
|
}
|
|
226
|
-
}))
|
|
227
|
-
|
|
223
|
+
}))).filter(target => target.connectTarget)
|
|
224
|
+
let lastError = null
|
|
225
|
+
if (!targets.length) throw new Error('nostr: no relay')
|
|
226
|
+
const attempts = targets.map(target => publishViaSharedRelay(target.relayUrl, event, signal, target.connectTarget)
|
|
227
|
+
.then(ok => {
|
|
228
|
+
if (ok) return true
|
|
229
|
+
lastError = new Error(`nostr: relay rejected publish (${target.relayUrl})`)
|
|
230
|
+
return false
|
|
231
|
+
}, error => {
|
|
232
|
+
lastError = error
|
|
233
|
+
return false
|
|
234
|
+
}))
|
|
235
|
+
// 首个成功的 relay 即结算调用方;其余尝试继续在后台跑(失败已就地吞掉,不会变成 unhandled rejection)。
|
|
236
|
+
const accepted = await Promise.race(attempts.map((attempt, index) => attempt.then(ok => ok ? index : -1)))
|
|
237
|
+
if (accepted >= 0) return
|
|
238
|
+
await Promise.all(attempts)
|
|
239
|
+
throw lastError || new Error('nostr: no relay accepted publish')
|
|
228
240
|
}
|
|
229
241
|
|
|
230
242
|
/**
|