@steve02081504/fount-p2p 0.0.39 → 0.0.41

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENTS.md CHANGED
@@ -39,7 +39,7 @@ Deno / native / BT: [runtime.md](docs/runtime.md).
39
39
 
40
40
  ## Tests / tools
41
41
 
42
- - `npm test` — pure + integration (Node; `--test-force-exit`)
42
+ - `npm test` — pure + integration + frontend (Node; `--test-force-exit`)
43
43
  - `npm run test:live` — live link / LAN smoke
44
44
  - `npm run test:fount` — cross-repo Deno bridge (`test/fount/`); see [runtime.md](docs/runtime.md)
45
45
  - `npm run test:sim` — tunables co-evolution (dev-only; [sim/AGENTS.md](sim/AGENTS.md))
@@ -73,7 +73,7 @@ Deno / native / BT: [runtime.md](docs/runtime.md).
73
73
 
74
74
  | File | Directory |
75
75
  |---|---|
76
- | `tunables.json` | `reputation/`, `trust_graph/`, `mailbox/`, `governance/`, `dag/` |
76
+ | `tunables.json` | `transport/`, `infra/`, `reputation/`, `trust_graph/`, `mailbox/`, `governance/`, `dag/` |
77
77
  | `part_query.tunables.json` | `schemas/` |
78
78
 
79
79
  Sim harness: `sim/tunables_bundle.mjs` (dev-only). See [sim/AGENTS.md](sim/AGENTS.md).
package/docs/evfs.md CHANGED
@@ -9,6 +9,17 @@ Day-to-day package rules: [AGENTS.md](../AGENTS.md). Implementation: `files/` (`
9
9
  | Ciphertext chunks (CAS) | `{nodeDir}/chunks/` |
10
10
  | Manifests | `{EntityStoreRoot}/{entityHash}/files/{path}.manifest.json` |
11
11
 
12
+ ## Chunk garbage collection
13
+
14
+ Chunk store is content-addressed and **write-only**: overwriting or deleting a file only rewrites/removes the manifest, old blocks stay behind forever. Reclamation is explicit, via `files/gc.mjs`:
15
+
16
+ - `mapChunkGarbage()` — read-only scan. Root set = the `parts[].hash` of **every** on-disk manifest (local writes, public cache, targeted non-public cache). Returns `{ manifests, brokenManifests, referenced, candidates: [{hash,size}], freedBytes }` plus zeroed delete counters. No side effects.
17
+ - `cleanChunkGarbage({ targets? })` — with `targets` (hashes or `{hash}`) it removes only that set without scanning and without touching manifests; without `targets` it scans, deletes every orphan chunk, and auto-removes manifests that fail `normalizeFileManifest` (reported as `brokenDeleted`).
18
+ - `deleteFileManifest(ownerEntityHash, logicalPath)` — explicit manifest delete (orphans its chunks for the next GC); requires the entity store to implement `deleteManifest`.
19
+ - `deleteChunk(hash)` — standalone primitive (deleting a chunk that is still referenced by a manifest breaks the file).
20
+
21
+ Concurrency: `putChunk` / `putChunkFromStream` / `deleteChunk` / the GC delete phase share one in-process mutex (`withChunkStoreLock`), so a GC never unlinks a block mid-write. Unlink is best-effort on ENOENT and skips EBUSY/EPERM (Windows open-handle = block is being streamed; retried next GC). Lock is process-local — do not run GC in a second process against the same `nodeDir`.
22
+
12
23
  ## Public publish / verify
13
24
 
14
25
  - `publishPublicFile` signs with the recovery key.
@@ -22,12 +33,19 @@ Day-to-day package rules: [AGENTS.md](../AGENTS.md). Implementation: `files/` (`
22
33
  | --- | --- |
23
34
  | Default options | Node-scope fanout; **only** signed public manifests accepted; no cache write |
24
35
  | Local public hit with `publicSig` | Return immediately; fanout revalidates in the background. With `cache: true`, write only when remote `publishedAt` is newer |
36
+ | `revalidate: true` on local public hit | Block until the fanout settles; prefer the newer `publishedAt` (write back when `cache: true`), else fall back to the local copy (offline-safe) |
25
37
  | Local non-public hit | Return immediately (**targeted mode only**); no revalidation (no publish order to compare); public mode refuses non-public local |
26
38
  | `fanoutTargets` given | Fanout **only** to that node set; accepts signed public **or** normalized non-public manifests; non-public hit is cached locally by default |
27
39
  | Same key in flight | Deduped via `utils/inflight_table` (key includes `public`/`targeted` mode) |
28
40
 
29
41
  Outer caller timeouts must **not** abort the in-flight work — background fill continues after the caller gives up.
30
42
 
43
+ ## `fetchChunk` / public-file chunk reads
44
+
45
+ - `fetchChunk` accepts the same `fanoutTargets` as `fetchManifest`. With targets it fanouts `fed_chunk_get` **only** to that node set; without targets it falls back to node-scope fanout. Same `username`+chunk hash+mode (`targeted`/`public`) is deduped in-flight.
46
+ - `readPublicFile` / `readManifestPlaintext` forward `options.fanoutTargets` to both the manifest fetch and the chunk fetch, so a cross-node public read can pull profile/avatar content straight from the owner node instead of depending on the node-scope fanout.
47
+ - The public (non-targeted) fanout does **not** block the request window on dialing the whole peer pool: already-linked / group-room-reachable peers are sent immediately, and unreachable peers are dialed in the background and re-sent once linked. (`ensureLinkToNode` no longer gates the send.)
48
+
31
49
  ## Non-public (ACL-gated) manifests
32
50
 
33
51
  - `file-master-key-wrap` / `vault-wrap` / `identity-wrap` manifests carry no author signature, so the remote-acquisition trust boundary is **not** a signature — it is the **fanout target set** plus the **serving node's authorization**.
@@ -26,10 +26,14 @@ const chunkInflight = createInflightTable({
26
26
  * ciphertextHash: string,
27
27
  * ownerEntityHash?: string,
28
28
  * groupId?: string,
29
+ * fanoutTargets?: string[],
29
30
  * }} FetchChunkContext
30
31
  */
31
32
 
32
33
  /**
34
+ * 拉取密文块。默认走 node-scope public 语义;
35
+ * 传入 `fanoutTargets` 时只向目标集 fanout(public 文件定向拉取,如 profile/avatar 直接向 owner 节点取块)。
36
+ * 同 username+hash+模式(含规范化目标集)in-flight 去重;本地已缓存块直接返回。
33
37
  * @param {FetchChunkContext} context 上下文
34
38
  * @returns {Promise<Uint8Array | null>} 密文块
35
39
  */
@@ -50,7 +54,7 @@ export async function fetchChunk(context) {
50
54
 
51
55
  const shared = beginFedFanoutFetch({
52
56
  inflight: chunkInflight,
53
- inflightKey: `${username}\0${hash}`,
57
+ inflightKeyBase: `${username}\0${hash}`,
54
58
  username,
55
59
  action: 'fed_chunk_get',
56
60
  /**
@@ -69,6 +73,7 @@ export async function fetchChunk(context) {
69
73
  chunkHash: hash,
70
74
  ownerEntityHash: context.ownerEntityHash,
71
75
  }),
76
+ fanoutTargets: context.fanoutTargets,
72
77
  })
73
78
  if (!shared) return null
74
79
 
@@ -7,6 +7,21 @@ import { pipeline } from 'node:stream/promises'
7
7
  import { isHex64 } from '../../core/hexIds.mjs'
8
8
  import { trackFileStream } from '../../node/handles.mjs'
9
9
  import { getNodeDir } from '../../node/instance.mjs'
10
+ import { withAsyncMutex } from '../../utils/async_mutex.mjs'
11
+
12
+ /** chunk store 互斥键:GC 删除与写入互斥。 */
13
+ const CHUNK_STORE_LOCK_KEY = 'chunk-store'
14
+
15
+ /**
16
+ * chunk store 全局互斥锁(进程内)。GC 删除阶段与 `putChunk`/`putChunkFromStream` 共用,
17
+ * 避免「GC unlink 与写入竞争」。单进程假设,不跨进程。
18
+ * @template T
19
+ * @param {() => Promise<T>} criticalSection 临界区
20
+ * @returns {Promise<T>} 临界区返回值
21
+ */
22
+ export function withChunkStoreLock(criticalSection) {
23
+ return withAsyncMutex(CHUNK_STORE_LOCK_KEY, criticalSection)
24
+ }
10
25
 
11
26
  /**
12
27
  * @returns {string} `{nodeDir}/chunks`
@@ -60,9 +75,7 @@ export function createChunkReadStream(hash) {
60
75
  * @returns {Promise<void>}
61
76
  */
62
77
  export async function putChunk(hash, data) {
63
- const filePath = chunkStorePath(hash)
64
- await fsp.mkdir(dirname(filePath), { recursive: true })
65
- await fsp.writeFile(filePath, Buffer.from(data))
78
+ return withChunkStoreLock(() => writeChunkFile(hash, Buffer.from(data)))
66
79
  }
67
80
 
68
81
  /**
@@ -71,7 +84,49 @@ export async function putChunk(hash, data) {
71
84
  * @returns {Promise<void>}
72
85
  */
73
86
  export async function putChunkFromStream(hash, readable) {
87
+ return withChunkStoreLock(async () => {
88
+ const filePath = chunkStorePath(hash)
89
+ await fsp.mkdir(dirname(filePath), { recursive: true })
90
+ await pipeline(readable, trackFileStream(fs.createWriteStream(filePath)))
91
+ })
92
+ }
93
+
94
+ /**
95
+ * @param {string} hash 64 位十六进制
96
+ * @param {string | Buffer | Uint8Array} data 块数据
97
+ * @returns {Promise<void>} 直接写文件(无锁;调用方须持 `withChunkStoreLock`)
98
+ */
99
+ async function writeChunkFile(hash, data) {
74
100
  const filePath = chunkStorePath(hash)
75
101
  await fsp.mkdir(dirname(filePath), { recursive: true })
76
- await pipeline(readable, trackFileStream(fs.createWriteStream(filePath)))
102
+ await fsp.writeFile(filePath, data)
103
+ }
104
+
105
+ /**
106
+ * @param {string} hash 64 位十六进制
107
+ * @returns {Promise<{ deleted: boolean, size: number }>} 删除结果;`deleted:false` 表示仍在被读(Windows open-handle)
108
+ */
109
+ export async function unlinkChunkFile(hash) {
110
+ const filePath = chunkStorePath(hash)
111
+ try {
112
+ const { size } = await fsp.stat(filePath)
113
+ await fsp.unlink(filePath)
114
+ return { deleted: true, size }
115
+ }
116
+ catch (error) {
117
+ const code = /** @type {NodeJS.ErrnoException} */ error.code
118
+ if (code === 'ENOENT') return { deleted: true, size: 0 }
119
+ // Windows 上文件被读流打开时 unlink 会 EBUSY/EPERM——视为仍在使用,留给下次 GC。
120
+ if (code === 'EBUSY' || code === 'EPERM') return { deleted: false, size: 0 }
121
+ throw error
122
+ }
123
+ }
124
+
125
+ /**
126
+ * 删除单个 chunk(带互斥锁)。重复删除视为成功。
127
+ * @param {string} hash 64 位十六进制
128
+ * @returns {Promise<{ deleted: boolean, size: number }>} 删除结果
129
+ */
130
+ export async function deleteChunk(hash) {
131
+ return withChunkStoreLock(() => unlinkChunkFile(hash))
77
132
  }
package/files/evfs.mjs CHANGED
@@ -32,7 +32,7 @@ function transferKeyDependenciesForReplica(replicaUsername, manifest) {
32
32
  /**
33
33
  * @param {string} username 拉取身份
34
34
  * @param {import('./manifest/normalize.mjs').FileManifest} manifest 清单
35
- * @param {{ fetchChunk?: Function }} [options] miss 拉取
35
+ * @param {{ fetchChunk?: Function, fanoutTargets?: string[] }} [options] miss 拉取
36
36
  * @returns {Promise<boolean>} 全部 part 是否已就位
37
37
  */
38
38
  async function ensureManifestPartsLocal(username, manifest, options = {}) {
@@ -43,6 +43,7 @@ async function ensureManifestPartsLocal(username, manifest, options = {}) {
43
43
  ciphertextHash: part.hash,
44
44
  ownerEntityHash: manifest.ownerEntityHash,
45
45
  groupId: manifest.transferKeyDescriptor.groupId,
46
+ fanoutTargets: options.fanoutTargets,
46
47
  })
47
48
  if (!fetchedChunk) return false
48
49
  await putChunk(part.hash, fetchedChunk)
@@ -67,6 +68,20 @@ export async function saveFileManifest(manifest) {
67
68
  await getEntityStore().writeManifest(manifest.ownerEntityHash, manifest.logicalPath, manifest)
68
69
  }
69
70
 
71
+ /**
72
+ * 删除本机已写盘 manifest;其引用的 chunk 孤儿化后由 `cleanChunkGarbage` 回收。
73
+ * @param {string} ownerEntityHash 所有者
74
+ * @param {string} logicalPath 路径
75
+ * @returns {Promise<void>} 删除成功或不存在
76
+ */
77
+ export async function deleteFileManifest(ownerEntityHash, logicalPath) {
78
+ const entityStore = getEntityStore()
79
+ if (entityStore.deleteManifest)
80
+ await entityStore.deleteManifest(ownerEntityHash, logicalPath)
81
+ else
82
+ throw new Error('entityStore lacks deleteManifest')
83
+ }
84
+
70
85
  /**
71
86
  * @param {import('./manifest/normalize.mjs').FileManifest} manifest 清单
72
87
  * @param {Array<Buffer | Uint8Array>} partBytes 密文块
@@ -80,7 +95,7 @@ export async function storeManifestParts(manifest, partBytes) {
80
95
  /**
81
96
  * @param {string} replicaUsername 副本用户名
82
97
  * @param {import('./manifest/normalize.mjs').FileManifest} manifest 清单
83
- * @param {{ username?: string, fetchChunk?: Function }} [options] miss 拉取
98
+ * @param {{ username?: string, fetchChunk?: Function, fanoutTargets?: string[] }} [options] miss 拉取
84
99
  * @returns {Promise<Buffer | null>} 明文内容
85
100
  */
86
101
  export async function readManifestPlaintext(replicaUsername, manifest, options = {}) {
@@ -104,7 +119,7 @@ export async function readManifestPlaintext(replicaUsername, manifest, options =
104
119
  /**
105
120
  * @param {string} replicaUsername 副本用户名
106
121
  * @param {import('./manifest/normalize.mjs').FileManifest} manifest 清单
107
- * @param {{ username?: string, fetchChunk?: Function }} [options] miss 拉取
122
+ * @param {{ username?: string, fetchChunk?: Function, fanoutTargets?: string[] }} [options] miss 拉取
108
123
  * @returns {Promise<import('node:stream').Readable | null>} 明文流
109
124
  */
110
125
  export async function readManifestPlaintextStream(replicaUsername, manifest, options = {}) {
@@ -215,10 +230,11 @@ export async function putFileManifestFromStream(parameters) {
215
230
 
216
231
  /**
217
232
  * 读取实体公开文件:本地 miss 时经网络取回签名 manifest,chunk miss 走既有 fetchChunk。
233
+ * 传入 `fanoutTargets` 时 manifest 与 chunk 拉取均定向到目标集(public 文件跨节点直取 owner 节点,避免 node-scope 大扇出)。
218
234
  * @param {string} replicaUsername 副本用户名
219
235
  * @param {string} entityHash owner entityHash
220
236
  * @param {string} logicalPath EVFS 逻辑路径
221
- * @param {{ username?: string, fetchChunk?: Function }} [options] miss 拉取
237
+ * @param {{ username?: string, fetchChunk?: Function, revalidate?: boolean, fanoutTargets?: string[] }} [options] miss 拉取;`revalidate` 强制阻塞等待 fanout 择新
222
238
  * @returns {Promise<Buffer | null>} 明文或 null
223
239
  */
224
240
  export async function readPublicFile(replicaUsername, entityHash, logicalPath, options = {}) {
@@ -228,6 +244,8 @@ export async function readPublicFile(replicaUsername, entityHash, logicalPath, o
228
244
  ownerEntityHash: entityHash,
229
245
  logicalPath,
230
246
  cache: true,
247
+ revalidate: options.revalidate === true,
248
+ fanoutTargets: options.fanoutTargets,
231
249
  })
232
250
  if (!manifest) return null
233
251
  return readManifestPlaintext(replicaUsername, manifest, options)
@@ -1,14 +1,15 @@
1
1
  import { randomUUID } from 'node:crypto'
2
2
 
3
3
  import { resolveNodeHash } from '../chunk/provider_registry.mjs'
4
- import { fanoutFedFetch } from '../fetch_fanout.mjs'
4
+ import { canonicalizeFanoutTargets, fanoutFedFetch } from '../fetch_fanout.mjs'
5
5
 
6
6
  /**
7
7
  * inflight + pending wait + fanout 共用骨架。
8
+ * 规范化目标集并拼入 inflight key:`fanoutTargets` 为数组(含空)即定向模式,undefined 为 node-scope public 模式。
8
9
  * @template T
9
10
  * @param {{
10
11
  * inflight: { acquire: (key: string, start: () => { done: Promise<T | null>, cancel: () => void }) => Promise<T | null> | null },
11
- * inflightKey: string,
12
+ * inflightKeyBase: string,
12
13
  * username: string,
13
14
  * action: string,
14
15
  * registerWait: (requestId: string) => { done: Promise<T | null>, cancel: () => void },
@@ -19,20 +20,23 @@ import { fanoutFedFetch } from '../fetch_fanout.mjs'
19
20
  */
20
21
  export function beginFedFanoutFetch({
21
22
  inflight,
22
- inflightKey,
23
+ inflightKeyBase,
23
24
  username,
24
25
  action,
25
26
  registerWait,
26
27
  buildPayload,
27
28
  fanoutTargets,
28
29
  }) {
30
+ const targeted = Array.isArray(fanoutTargets)
31
+ const canonicalTargets = targeted ? canonicalizeFanoutTargets(fanoutTargets) : undefined
32
+ const inflightKey = `${inflightKeyBase}\0${targeted ? 'targeted' : 'public'}` + (canonicalTargets?.length ? `\0${canonicalTargets.join('\0')}` : '')
29
33
  return inflight.acquire(inflightKey, () => {
30
34
  const requestId = randomUUID()
31
35
  const wait = registerWait(requestId)
32
36
  void (async () => {
33
37
  try {
34
38
  const { nodeHash } = await resolveNodeHash(username)
35
- await fanoutFedFetch(username, action, buildPayload(requestId, nodeHash), fanoutTargets)
39
+ await fanoutFedFetch(username, action, buildPayload(requestId, nodeHash), canonicalTargets)
36
40
  }
37
41
  catch { /* pending wait 超时/cancel 负责 settle */ }
38
42
  })()
@@ -32,6 +32,7 @@ export function canonicalizeFanoutTargets(targets) {
32
32
  /**
33
33
  * 全局 miss 请求扇出:先向已知 peer 定向发送,再 trust-graph top-K fanout。
34
34
  * 已直连 / follow hint peer 可能不在 trust-graph top-K(非成员 emoji CAS / Social 预览路径),故先定向发送。
35
+ * 拨号不阻塞请求窗口:已直连 peer 立即经现有链路投递,未直连 peer 先经群房间/overlay 尝试并后台拨号补发。
35
36
  * @param {string} username 用户
36
37
  * @param {string} action wire action 名
37
38
  * @param {object} payload 请求载荷
@@ -52,8 +53,20 @@ export async function fanoutFedFetch(username, action, payload, fanoutTargets) {
52
53
 
53
54
  const graph = await trustGraph.buildMergedGraph(username)
54
55
  const peerTargets = fetchPeerTargets()
55
- await Promise.all(peerTargets.map(nodeHash => ensureLinkToNode(nodeHash).catch(() => null)))
56
- for (const nodeHash of peerTargets)
57
- void trustGraph.sendToNode(username, nodeHash, action, payload, graph)
56
+ // 已直连 peer 立即经现有链路投递;其余 peer 后台并行:先经群房间/overlay 尝试投递,同时拨号,拨通且首投失败再补发。
57
+ // 全程不阻塞请求窗口(大 peer 池下 await 全量拨号会超窗口)。
58
+ const linked = new Set(listLinks().map(({ nodeHash }) => nodeHash))
59
+ for (const nodeHash of peerTargets) {
60
+ if (linked.has(nodeHash)) {
61
+ void trustGraph.sendToNode(username, nodeHash, action, payload, graph)
62
+ continue
63
+ }
64
+ const dialed = ensureLinkToNode(nodeHash).catch(() => null)
65
+ void trustGraph.sendToNode(username, nodeHash, action, payload, graph).then(async sent => {
66
+ if (sent) return
67
+ const link = await dialed
68
+ if (link) await trustGraph.sendToNode(username, nodeHash, action, payload, graph)
69
+ })
70
+ }
58
71
  await trustGraph.fanoutToTopNodes(username, action, payload, FEDERATION_CHUNK_FETCH_FANOUT_K)
59
72
  }
package/files/gc.mjs ADDED
@@ -0,0 +1,192 @@
1
+ import fsp from 'node:fs/promises'
2
+ import { join } from 'node:path'
3
+
4
+ import { isHex64 } from '../core/hexIds.mjs'
5
+ import { getEntityStore, getNodeLogger } from '../node/instance.mjs'
6
+
7
+ import {
8
+ chunkStoreRoot,
9
+ unlinkChunkFile,
10
+ withChunkStoreLock,
11
+ } from './chunk/store.mjs'
12
+ import { normalizeFileManifest } from './manifest/normalize.mjs'
13
+
14
+ /**
15
+ * @typedef {{ hash: string, size: number }} GcChunkTarget
16
+ */
17
+
18
+ /**
19
+ * @typedef {{
20
+ * manifests: number,
21
+ * brokenManifests: Array<{ ownerEntityHash: string, logicalPath: string }>,
22
+ * referenced: number,
23
+ * candidates: Array<GcChunkTarget>,
24
+ * deleted: number,
25
+ * retained: number,
26
+ * freedBytes: number,
27
+ * brokenDeleted: number,
28
+ * }} GarbageCollectionReport
29
+ */
30
+
31
+ /**
32
+ * 纯读扫描:以全部写盘 manifest 的 `parts[].hash` 为根集,枚举 `chunks/` 孤儿块与坏 manifest。
33
+ * 无副作用(不删文件)。
34
+ * @returns {Promise<GarbageCollectionReport>} 扫描报告
35
+ */
36
+ async function scanChunkGarbage() {
37
+ const entityStore = getEntityStore()
38
+ const referenced = new Set()
39
+ const brokenManifests = []
40
+ let manifests = 0
41
+
42
+ const entityHashes = await entityStore.listEntityHashes()
43
+ for (const entityHash of entityHashes) {
44
+ const logicalPaths = await entityStore.listEntityFiles(entityHash)
45
+ for (const logicalPath of logicalPaths) {
46
+ manifests++
47
+ const raw = await entityStore.readManifest(entityHash, logicalPath)
48
+ const normalized = normalizeFileManifest(raw)
49
+ if (!normalized) {
50
+ brokenManifests.push({ ownerEntityHash: entityHash, logicalPath })
51
+ continue
52
+ }
53
+ for (const part of normalized.parts)
54
+ referenced.add(part.hash)
55
+ }
56
+ }
57
+
58
+ /** @type {GcChunkTarget[]} */
59
+ const candidates = []
60
+ let freedBytes = 0
61
+ let prefixEntries
62
+ try {
63
+ prefixEntries = await fsp.readdir(chunkStoreRoot(), { withFileTypes: true })
64
+ }
65
+ catch {
66
+ prefixEntries = []
67
+ }
68
+ for (const prefixEntry of prefixEntries) {
69
+ if (!prefixEntry.isDirectory() || prefixEntry.name.length !== 2) continue
70
+ let files
71
+ try {
72
+ files = await fsp.readdir(join(chunkStoreRoot(), prefixEntry.name))
73
+ }
74
+ catch { continue }
75
+ for (const name of files) {
76
+ if (!name.endsWith('.bin')) continue
77
+ const hash = name.slice(0, -4)
78
+ if (!isHex64(hash) || referenced.has(hash)) continue
79
+ const size = await chunkFileSize(join(chunkStoreRoot(), prefixEntry.name, name))
80
+ candidates.push({ hash, size })
81
+ freedBytes += size
82
+ }
83
+ }
84
+
85
+ return {
86
+ manifests,
87
+ brokenManifests,
88
+ referenced: referenced.size,
89
+ candidates,
90
+ deleted: 0,
91
+ retained: 0,
92
+ freedBytes,
93
+ brokenDeleted: 0,
94
+ }
95
+ }
96
+
97
+ /**
98
+ * @param {string} filePath chunk 文件绝对路径
99
+ * @returns {Promise<number>} 文件字节数;stat 失败为 0
100
+ */
101
+ async function chunkFileSize(filePath) {
102
+ try {
103
+ const { size } = await fsp.stat(filePath)
104
+ return size
105
+ }
106
+ catch { return 0 }
107
+ }
108
+
109
+ /**
110
+ * 持锁删除一批 chunk,统计删除结果并顺带清理已空前缀目录。
111
+ * @param {string[]} hashes 待删 64 位 hex 集合
112
+ * @returns {Promise<{ deleted: number, retained: number, freedBytes: number }>} 删除结果统计
113
+ */
114
+ async function removeChunkHashes(hashes) {
115
+ let deleted = 0
116
+ let retained = 0
117
+ let freedBytes = 0
118
+ const emptiedPrefixes = new Set()
119
+ await withChunkStoreLock(async () => {
120
+ for (const hash of hashes) {
121
+ const result = await unlinkChunkFile(hash)
122
+ if (result.deleted) {
123
+ deleted++
124
+ freedBytes += result.size
125
+ emptiedPrefixes.add(hash.slice(0, 2))
126
+ }
127
+ else retained++
128
+ }
129
+ for (const prefix of emptiedPrefixes) try {
130
+ await fsp.rmdir(join(chunkStoreRoot(), prefix))
131
+ } catch { /* 目录非空或已被删,忽略 */ }
132
+ })
133
+ return { deleted, retained, freedBytes }
134
+ }
135
+
136
+ /**
137
+ * 扫描并报告 chunk 垃圾(只读,不删除)。
138
+ * 根集 = 全部写盘 manifest 的 `parts[].hash`(含公共/非 public 缓存);`chunks/` 中不在根集的可回收。
139
+ * @returns {Promise<GarbageCollectionReport>} 统计报告
140
+ */
141
+ export async function mapChunkGarbage() {
142
+ return scanChunkGarbage()
143
+ }
144
+
145
+ /**
146
+ * 清理 chunk 垃圾。缺省 `targets` 时先扫描再删除(并自动清除扫描到的坏 manifest);
147
+ * 提供 `targets` 时只按给定集合删除,不做扫描、不碰 manifest。
148
+ * @param {{ targets?: Array<string | GcChunkTarget> }} [options] 可选显式目标集
149
+ * @returns {Promise<GarbageCollectionReport>} 执行报告
150
+ */
151
+ export async function cleanChunkGarbage(options = {}) {
152
+ const targets = Array.isArray(options.targets)
153
+
154
+ /** @type {string[]} */
155
+ let hashes
156
+ if (targets) {
157
+ hashes = options.targets.map(target => {
158
+ const hash = typeof target === 'string' ? target : target?.hash
159
+ if (!isHex64(hash)) throw new Error(`cleanChunkGarbage: invalid chunk hash ${hash}`)
160
+ return hash
161
+ })
162
+ const { deleted, retained, freedBytes } = await removeChunkHashes(hashes)
163
+ return {
164
+ manifests: 0,
165
+ brokenManifests: [],
166
+ referenced: 0,
167
+ candidates: hashes.map(hash => ({ hash, size: 0 })),
168
+ deleted,
169
+ retained,
170
+ freedBytes,
171
+ brokenDeleted: 0,
172
+ }
173
+ }
174
+
175
+ const snapshot = await scanChunkGarbage()
176
+ const entityStore = getEntityStore()
177
+ const logger = getNodeLogger()
178
+ let brokenDeleted = 0
179
+ if (snapshot.brokenManifests.length) {
180
+ for (const broken of snapshot.brokenManifests) {
181
+ if (!entityStore.deleteManifest) break
182
+ await entityStore.deleteManifest(broken.ownerEntityHash, broken.logicalPath)
183
+ brokenDeleted++
184
+ }
185
+ logger?.info?.(`chunk GC: removed ${brokenDeleted} broken manifest(s)`)
186
+ }
187
+
188
+ const { deleted, retained, freedBytes } = await removeChunkHashes(
189
+ snapshot.candidates.map(candidate => candidate.hash),
190
+ )
191
+ return { ...snapshot, deleted, retained, freedBytes, brokenDeleted }
192
+ }
@@ -30,9 +30,10 @@ const manifestInflight = createInflightTable({
30
30
  * 拉取 manifest。默认走 node-scope public 语义(仅接受验签公开 manifest);
31
31
  * 传入 `fanoutTargets` 时只向目标集 fanout,并接受无签名的非 public manifest(信任边界 = 目标集 + 服务端 servicer)。
32
32
  * 非 public 命中默认写盘;public 按 `cache: true` 择新写盘。
33
- * 本地已验签公开缓存立即返回(`cache: true` 时后台 fanout 刷新);非 public / 未验签 public 本地命中仅 targeted 模式返回,
34
- * public 模式拒绝。同 key(含规范化目标集)in-flight 去重;调用方外层超时不 abort,后台继续填缓存。
35
- * @param {{ username: string, ownerEntityHash: string, logicalPath: string, fanoutTargets?: string[], cache?: boolean, timeoutMs?: number }} context - 拉取上下文
33
+ * 本地已验签公开缓存默认立即返回(`cache: true` 时后台 fanout 刷新);`revalidate: true` 时改为阻塞等待本次 fanout 择新,
34
+ * 无新则回退本地。非 public / 未验签 public 本地命中仅 targeted 模式返回,public 模式拒绝。
35
+ * key(含规范化目标集)in-flight 去重;调用方外层超时不 abort,后台继续填缓存。
36
+ * @param {{ username: string, ownerEntityHash: string, logicalPath: string, fanoutTargets?: string[], cache?: boolean, revalidate?: boolean, timeoutMs?: number }} context - 拉取上下文
36
37
  * @returns {Promise<import('./normalize.mjs').FileManifest | null>} 校验后的 manifest,失败为 null
37
38
  */
38
39
  export async function fetchManifest(context) {
@@ -40,6 +41,7 @@ export async function fetchManifest(context) {
40
41
  const logicalPath = context.logicalPath.replace(/^\/+/, '')
41
42
  if (!ownerEntityHash || !logicalPath || !username) return null
42
43
 
44
+ const revalidate = context.revalidate === true
43
45
  const timeoutMs = Number(context.timeoutMs) > 0
44
46
  ? Number(context.timeoutMs)
45
47
  : DEFAULT_MANIFEST_FETCH_TIMEOUT_MS
@@ -52,7 +54,7 @@ export async function fetchManifest(context) {
52
54
  const localPromise = loadFileManifest(ownerEntityHash, logicalPath)
53
55
  const shared = beginFedFanoutFetch({
54
56
  inflight: manifestInflight,
55
- inflightKey: `${username}\0${expectedKey}\0${targeted ? 'targeted' : 'public'}` + (fanoutTargets?.length ? `\0${fanoutTargets.join('\0')}` : ''),
57
+ inflightKeyBase: `${username}\0${expectedKey}`,
56
58
  username,
57
59
  action: 'fed_manifest_get',
58
60
  /**
@@ -78,10 +80,18 @@ export async function fetchManifest(context) {
78
80
  })
79
81
  const local = await localPromise
80
82
 
81
- // 本地已验签公开命中:立即返回;cache: true 时后台 fanout 择新刷新。
83
+ // 本地已验签公开命中:默认立即返回,cache: true 时后台 fanout 择新刷新;
84
+ // revalidate: true 时阻塞等待本次 fanout,取新择旧并写回,fanout 无新(含超时)则回退本地。
82
85
  const isLocalPublic = local?.transferKeyDescriptor?.type === 'public' && !!local?.meta?.publicSig
83
86
  if (isLocalPublic) {
84
- if (wantCache && shared) void shared.then(async result => {
87
+ if (shared && revalidate) {
88
+ const result = await shared
89
+ if (result && shouldPreferIncomingPublicManifest(local, result)) {
90
+ if (wantCache) await cachePublicManifest(ownerEntityHash, logicalPath, result)
91
+ return result
92
+ }
93
+ }
94
+ else if (wantCache && shared) void shared.then(async result => {
85
95
  if (result && shouldPreferIncomingPublicManifest(local, result))
86
96
  await cachePublicManifest(ownerEntityHash, logicalPath, result)
87
97
  })
@@ -17,6 +17,7 @@ import { readJsonFile, writeJsonFile } from '../utils/json_io.mjs'
17
17
  * @property {(entityHash: string, logicalPath: string) => Promise<object | null>} readManifest
18
18
  * @property {(entityHash: string, logicalPath: string, data: object) => Promise<void>} writeManifest
19
19
  * @property {(entityHash: string, logicalPath: string) => Promise<boolean>} statManifest
20
+ * @property {(entityHash: string, logicalPath: string) => Promise<void>} [deleteManifest]
20
21
  * @property {() => Promise<string[]>} listEntityHashes
21
22
  */
22
23
 
@@ -210,5 +211,19 @@ export function createFsEntityStore(baseDir) {
210
211
  return false
211
212
  }
212
213
  },
214
+
215
+ /**
216
+ * @param {string} entityHash 128 位十六进制
217
+ * @param {string} logicalPath EVFS 逻辑路径
218
+ * @returns {Promise<void>} 删除 manifest(不存在视为成功)
219
+ */
220
+ async deleteManifest(entityHash, logicalPath) {
221
+ try {
222
+ await fsp.unlink(manifestPath(entityHash, logicalPath))
223
+ }
224
+ catch (error) {
225
+ if (/** @type {NodeJS.ErrnoException} */ error.code !== 'ENOENT') throw error
226
+ }
227
+ },
213
228
  }
214
229
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@steve02081504/fount-p2p",
3
- "version": "0.0.39",
3
+ "version": "0.0.41",
4
4
  "description": "fount federation P2P layer — link, trust graph, mailbox, DAG, EVFS.",
5
5
  "keywords": [
6
6
  "network",