@steve02081504/fount-p2p 0.0.38 → 0.0.40
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +1 -1
- package/docs/evfs.md +18 -3
- package/files/chunk/store.mjs +59 -4
- package/files/evfs.mjs +16 -1
- package/files/gc.mjs +192 -0
- package/files/manifest/acl.mjs +44 -15
- package/files/manifest/fetch.mjs +19 -9
- package/files/manifest/routing.mjs +38 -0
- package/files/manifest/servicer_registry.mjs +18 -26
- package/files/transfer_key_registry.mjs +10 -46
- package/node/entity_store.mjs +15 -0
- package/package.json +1 -1
- package/files/manifest/acl_registry.mjs +0 -84
package/AGENTS.md
CHANGED
|
@@ -58,7 +58,7 @@ Deno / native / BT: [runtime.md](docs/runtime.md).
|
|
|
58
58
|
- **Fanout vs targeted / timed collect:** [wire.md](docs/wire.md).
|
|
59
59
|
- **Channel encryption:** per-channel `K_ch`, scheme `channel-key` (`CHANNEL_KEY_SCHEME`); decrypted payloads are untrusted outside DAG Ed25519 context.
|
|
60
60
|
- **Denylist vs personal lists:** node `denylist.json` vs per-entity `personal_block.json` / `personal_hide.json`.
|
|
61
|
-
- **Manifest ACL / transfer owner / servicer:**
|
|
61
|
+
- **Manifest ACL / transfer owner / servicer:** ownership is routed by `registerManifestOwner` matchers only; ACL & servicer handlers are keyed by `ownerId` (a `type` may be reused across families but never acts as a routing key). Core does not hard-code chat/social types. Non-public manifests are served cross-node only via a registered servicer; unclaimed non-public is denied (no `type` fallback).
|
|
62
62
|
|
|
63
63
|
### Node / network
|
|
64
64
|
|
package/docs/evfs.md
CHANGED
|
@@ -9,6 +9,17 @@ Day-to-day package rules: [AGENTS.md](../AGENTS.md). Implementation: `files/` (`
|
|
|
9
9
|
| Ciphertext chunks (CAS) | `{nodeDir}/chunks/` |
|
|
10
10
|
| Manifests | `{EntityStoreRoot}/{entityHash}/files/{path}.manifest.json` |
|
|
11
11
|
|
|
12
|
+
## Chunk garbage collection
|
|
13
|
+
|
|
14
|
+
Chunk store is content-addressed and **write-only**: overwriting or deleting a file only rewrites/removes the manifest, old blocks stay behind forever. Reclamation is explicit, via `files/gc.mjs`:
|
|
15
|
+
|
|
16
|
+
- `mapChunkGarbage()` — read-only scan. Root set = the `parts[].hash` of **every** on-disk manifest (local writes, public cache, targeted non-public cache). Returns `{ manifests, brokenManifests, referenced, candidates: [{hash,size}], freedBytes }` plus zeroed delete counters. No side effects.
|
|
17
|
+
- `cleanChunkGarbage({ targets? })` — with `targets` (hashes or `{hash}`) it removes only that set without scanning and without touching manifests; without `targets` it scans, deletes every orphan chunk, and auto-removes manifests that fail `normalizeFileManifest` (reported as `brokenDeleted`).
|
|
18
|
+
- `deleteFileManifest(ownerEntityHash, logicalPath)` — explicit manifest delete (orphans its chunks for the next GC); requires the entity store to implement `deleteManifest`.
|
|
19
|
+
- `deleteChunk(hash)` — standalone primitive (deleting a chunk that is still referenced by a manifest breaks the file).
|
|
20
|
+
|
|
21
|
+
Concurrency: `putChunk` / `putChunkFromStream` / `deleteChunk` / the GC delete phase share one in-process mutex (`withChunkStoreLock`), so a GC never unlinks a block mid-write. Unlink is best-effort on ENOENT and skips EBUSY/EPERM (Windows open-handle = block is being streamed; retried next GC). Lock is process-local — do not run GC in a second process against the same `nodeDir`.
|
|
22
|
+
|
|
12
23
|
## Public publish / verify
|
|
13
24
|
|
|
14
25
|
- `publishPublicFile` signs with the recovery key.
|
|
@@ -22,6 +33,7 @@ Day-to-day package rules: [AGENTS.md](../AGENTS.md). Implementation: `files/` (`
|
|
|
22
33
|
| --- | --- |
|
|
23
34
|
| Default options | Node-scope fanout; **only** signed public manifests accepted; no cache write |
|
|
24
35
|
| Local public hit with `publicSig` | Return immediately; fanout revalidates in the background. With `cache: true`, write only when remote `publishedAt` is newer |
|
|
36
|
+
| `revalidate: true` on local public hit | Block until the fanout settles; prefer the newer `publishedAt` (write back when `cache: true`), else fall back to the local copy (offline-safe) |
|
|
25
37
|
| Local non-public hit | Return immediately (**targeted mode only**); no revalidation (no publish order to compare); public mode refuses non-public local |
|
|
26
38
|
| `fanoutTargets` given | Fanout **only** to that node set; accepts signed public **or** normalized non-public manifests; non-public hit is cached locally by default |
|
|
27
39
|
| Same key in flight | Deduped via `utils/inflight_table` (key includes `public`/`targeted` mode) |
|
|
@@ -32,10 +44,13 @@ Outer caller timeouts must **not** abort the in-flight work — background fill
|
|
|
32
44
|
|
|
33
45
|
- `file-master-key-wrap` / `vault-wrap` / `identity-wrap` manifests carry no author signature, so the remote-acquisition trust boundary is **not** a signature — it is the **fanout target set** plus the **serving node's authorization**.
|
|
34
46
|
- Client: the caller passes the authorized node set (group roster / cabinet members) as `fanoutTargets`. `fed_manifest_get` is sent only to those nodes; a response is accepted when it normalizes, its `ownerEntityHash`+`logicalPath` match the request, and its authenticated `senderNodeHash` belongs to the target set.
|
|
35
|
-
- Server: `handleIncomingManifestGet` refuses non-public manifests unless a **servicer** is registered for the
|
|
47
|
+
- Server: `handleIncomingManifestGet` refuses non-public manifests unless a **servicer** is registered for the matched owner family. Shells call `registerManifestOwner(ownerId, match)` to claim ownership and `registerManifestServicer(ownerId, handler)` to serve; the handler receives `{ manifest, ownerEntityHash, logicalPath, requesterNodeHash, peerId, payload }` and returns a boolean. `requesterNodeHash` is **always** the transport-authenticated sender nodeHash (`peerId`); the client no longer self-asserts a `nodeHash` in `fed_manifest_get`, so the fanout target node can never be mistaken for the requester — authorization must still be enforced from the trust graph / group membership.
|
|
36
48
|
- Non-public responses carry the **full manifest including `meta`** (public responses strip meta to `publicSig`): the reader needs `meta.dagParts` / `meta.groupId` to read DAG plaintext.
|
|
37
49
|
- Confidentiality does not rely on manifest secrecy: ciphertext blocks are content-addressed, AES-GCM authenticated, and decryption requires a wrap key held by the shell. A forged manifest can at worst fail the read (DoS), never decrypt content.
|
|
38
50
|
|
|
39
|
-
## ACL
|
|
51
|
+
## ACL & owner routing
|
|
40
52
|
|
|
41
|
-
|
|
53
|
+
- Ownership is decided **only** by matchers: `registerManifestOwner(ownerId, match)` with `match(manifest, ownerEntityHash)` returning a boolean. `transferKeyDescriptor.type` is **not** a routing key — the same type (e.g. `file-master-key-wrap`) may be reused by several families (chat group, cabinet shared, vault, …).
|
|
54
|
+
- Matchers must be mutually exclusive: a manifest belongs to exactly one family; resolution is first-match in registration order. A broad matcher that claims another family's entities will shadow them, so matchers should key on their own entity namespace (e.g. the `fount:chat:group:` prefix).
|
|
55
|
+
- Local read/write ACL (`registerManifestAcl(ownerId, handler)`) and cross-node serve (`registerManifestServicer(ownerId, handler)`) are both keyed by `ownerId` (same `ownerId` as `registerTransferKeyDependencies`). A claimed family with no handler denies (fail-closed); an unclaimed non-public manifest is denied cross-node — there is no `type` fallback.
|
|
56
|
+
- Core does not hard-code chat/social entity types.
|
package/files/chunk/store.mjs
CHANGED
|
@@ -7,6 +7,21 @@ import { pipeline } from 'node:stream/promises'
|
|
|
7
7
|
import { isHex64 } from '../../core/hexIds.mjs'
|
|
8
8
|
import { trackFileStream } from '../../node/handles.mjs'
|
|
9
9
|
import { getNodeDir } from '../../node/instance.mjs'
|
|
10
|
+
import { withAsyncMutex } from '../../utils/async_mutex.mjs'
|
|
11
|
+
|
|
12
|
+
/** chunk store 互斥键:GC 删除与写入互斥。 */
|
|
13
|
+
const CHUNK_STORE_LOCK_KEY = 'chunk-store'
|
|
14
|
+
|
|
15
|
+
/**
|
|
16
|
+
* chunk store 全局互斥锁(进程内)。GC 删除阶段与 `putChunk`/`putChunkFromStream` 共用,
|
|
17
|
+
* 避免「GC unlink 与写入竞争」。单进程假设,不跨进程。
|
|
18
|
+
* @template T
|
|
19
|
+
* @param {() => Promise<T>} criticalSection 临界区
|
|
20
|
+
* @returns {Promise<T>} 临界区返回值
|
|
21
|
+
*/
|
|
22
|
+
export function withChunkStoreLock(criticalSection) {
|
|
23
|
+
return withAsyncMutex(CHUNK_STORE_LOCK_KEY, criticalSection)
|
|
24
|
+
}
|
|
10
25
|
|
|
11
26
|
/**
|
|
12
27
|
* @returns {string} `{nodeDir}/chunks`
|
|
@@ -60,9 +75,7 @@ export function createChunkReadStream(hash) {
|
|
|
60
75
|
* @returns {Promise<void>}
|
|
61
76
|
*/
|
|
62
77
|
export async function putChunk(hash, data) {
|
|
63
|
-
|
|
64
|
-
await fsp.mkdir(dirname(filePath), { recursive: true })
|
|
65
|
-
await fsp.writeFile(filePath, Buffer.from(data))
|
|
78
|
+
return withChunkStoreLock(() => writeChunkFile(hash, Buffer.from(data)))
|
|
66
79
|
}
|
|
67
80
|
|
|
68
81
|
/**
|
|
@@ -71,7 +84,49 @@ export async function putChunk(hash, data) {
|
|
|
71
84
|
* @returns {Promise<void>}
|
|
72
85
|
*/
|
|
73
86
|
export async function putChunkFromStream(hash, readable) {
|
|
87
|
+
return withChunkStoreLock(async () => {
|
|
88
|
+
const filePath = chunkStorePath(hash)
|
|
89
|
+
await fsp.mkdir(dirname(filePath), { recursive: true })
|
|
90
|
+
await pipeline(readable, trackFileStream(fs.createWriteStream(filePath)))
|
|
91
|
+
})
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
/**
|
|
95
|
+
* @param {string} hash 64 位十六进制
|
|
96
|
+
* @param {string | Buffer | Uint8Array} data 块数据
|
|
97
|
+
* @returns {Promise<void>} 直接写文件(无锁;调用方须持 `withChunkStoreLock`)
|
|
98
|
+
*/
|
|
99
|
+
async function writeChunkFile(hash, data) {
|
|
74
100
|
const filePath = chunkStorePath(hash)
|
|
75
101
|
await fsp.mkdir(dirname(filePath), { recursive: true })
|
|
76
|
-
await
|
|
102
|
+
await fsp.writeFile(filePath, data)
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
/**
|
|
106
|
+
* @param {string} hash 64 位十六进制
|
|
107
|
+
* @returns {Promise<{ deleted: boolean, size: number }>} 删除结果;`deleted:false` 表示仍在被读(Windows open-handle)
|
|
108
|
+
*/
|
|
109
|
+
export async function unlinkChunkFile(hash) {
|
|
110
|
+
const filePath = chunkStorePath(hash)
|
|
111
|
+
try {
|
|
112
|
+
const { size } = await fsp.stat(filePath)
|
|
113
|
+
await fsp.unlink(filePath)
|
|
114
|
+
return { deleted: true, size }
|
|
115
|
+
}
|
|
116
|
+
catch (error) {
|
|
117
|
+
const code = /** @type {NodeJS.ErrnoException} */ error.code
|
|
118
|
+
if (code === 'ENOENT') return { deleted: true, size: 0 }
|
|
119
|
+
// Windows 上文件被读流打开时 unlink 会 EBUSY/EPERM——视为仍在使用,留给下次 GC。
|
|
120
|
+
if (code === 'EBUSY' || code === 'EPERM') return { deleted: false, size: 0 }
|
|
121
|
+
throw error
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
/**
|
|
126
|
+
* 删除单个 chunk(带互斥锁)。重复删除视为成功。
|
|
127
|
+
* @param {string} hash 64 位十六进制
|
|
128
|
+
* @returns {Promise<{ deleted: boolean, size: number }>} 删除结果
|
|
129
|
+
*/
|
|
130
|
+
export async function deleteChunk(hash) {
|
|
131
|
+
return withChunkStoreLock(() => unlinkChunkFile(hash))
|
|
77
132
|
}
|
package/files/evfs.mjs
CHANGED
|
@@ -67,6 +67,20 @@ export async function saveFileManifest(manifest) {
|
|
|
67
67
|
await getEntityStore().writeManifest(manifest.ownerEntityHash, manifest.logicalPath, manifest)
|
|
68
68
|
}
|
|
69
69
|
|
|
70
|
+
/**
|
|
71
|
+
* 删除本机已写盘 manifest;其引用的 chunk 孤儿化后由 `cleanChunkGarbage` 回收。
|
|
72
|
+
* @param {string} ownerEntityHash 所有者
|
|
73
|
+
* @param {string} logicalPath 路径
|
|
74
|
+
* @returns {Promise<void>} 删除成功或不存在
|
|
75
|
+
*/
|
|
76
|
+
export async function deleteFileManifest(ownerEntityHash, logicalPath) {
|
|
77
|
+
const entityStore = getEntityStore()
|
|
78
|
+
if (entityStore.deleteManifest)
|
|
79
|
+
await entityStore.deleteManifest(ownerEntityHash, logicalPath)
|
|
80
|
+
else
|
|
81
|
+
throw new Error('entityStore lacks deleteManifest')
|
|
82
|
+
}
|
|
83
|
+
|
|
70
84
|
/**
|
|
71
85
|
* @param {import('./manifest/normalize.mjs').FileManifest} manifest 清单
|
|
72
86
|
* @param {Array<Buffer | Uint8Array>} partBytes 密文块
|
|
@@ -218,7 +232,7 @@ export async function putFileManifestFromStream(parameters) {
|
|
|
218
232
|
* @param {string} replicaUsername 副本用户名
|
|
219
233
|
* @param {string} entityHash owner entityHash
|
|
220
234
|
* @param {string} logicalPath EVFS 逻辑路径
|
|
221
|
-
* @param {{ username?: string, fetchChunk?: Function }} [options] miss
|
|
235
|
+
* @param {{ username?: string, fetchChunk?: Function, revalidate?: boolean }} [options] miss 拉取;`revalidate` 强制阻塞等待 fanout 择新
|
|
222
236
|
* @returns {Promise<Buffer | null>} 明文或 null
|
|
223
237
|
*/
|
|
224
238
|
export async function readPublicFile(replicaUsername, entityHash, logicalPath, options = {}) {
|
|
@@ -228,6 +242,7 @@ export async function readPublicFile(replicaUsername, entityHash, logicalPath, o
|
|
|
228
242
|
ownerEntityHash: entityHash,
|
|
229
243
|
logicalPath,
|
|
230
244
|
cache: true,
|
|
245
|
+
revalidate: options.revalidate === true,
|
|
231
246
|
})
|
|
232
247
|
if (!manifest) return null
|
|
233
248
|
return readManifestPlaintext(replicaUsername, manifest, options)
|
package/files/gc.mjs
ADDED
|
@@ -0,0 +1,192 @@
|
|
|
1
|
+
import fsp from 'node:fs/promises'
|
|
2
|
+
import { join } from 'node:path'
|
|
3
|
+
|
|
4
|
+
import { isHex64 } from '../core/hexIds.mjs'
|
|
5
|
+
import { getEntityStore, getNodeLogger } from '../node/instance.mjs'
|
|
6
|
+
|
|
7
|
+
import {
|
|
8
|
+
chunkStoreRoot,
|
|
9
|
+
unlinkChunkFile,
|
|
10
|
+
withChunkStoreLock,
|
|
11
|
+
} from './chunk/store.mjs'
|
|
12
|
+
import { normalizeFileManifest } from './manifest/normalize.mjs'
|
|
13
|
+
|
|
14
|
+
/**
|
|
15
|
+
* @typedef {{ hash: string, size: number }} GcChunkTarget
|
|
16
|
+
*/
|
|
17
|
+
|
|
18
|
+
/**
|
|
19
|
+
* @typedef {{
|
|
20
|
+
* manifests: number,
|
|
21
|
+
* brokenManifests: Array<{ ownerEntityHash: string, logicalPath: string }>,
|
|
22
|
+
* referenced: number,
|
|
23
|
+
* candidates: Array<GcChunkTarget>,
|
|
24
|
+
* deleted: number,
|
|
25
|
+
* retained: number,
|
|
26
|
+
* freedBytes: number,
|
|
27
|
+
* brokenDeleted: number,
|
|
28
|
+
* }} GarbageCollectionReport
|
|
29
|
+
*/
|
|
30
|
+
|
|
31
|
+
/**
|
|
32
|
+
* 纯读扫描:以全部写盘 manifest 的 `parts[].hash` 为根集,枚举 `chunks/` 孤儿块与坏 manifest。
|
|
33
|
+
* 无副作用(不删文件)。
|
|
34
|
+
* @returns {Promise<GarbageCollectionReport>} 扫描报告
|
|
35
|
+
*/
|
|
36
|
+
async function scanChunkGarbage() {
|
|
37
|
+
const entityStore = getEntityStore()
|
|
38
|
+
const referenced = new Set()
|
|
39
|
+
const brokenManifests = []
|
|
40
|
+
let manifests = 0
|
|
41
|
+
|
|
42
|
+
const entityHashes = await entityStore.listEntityHashes()
|
|
43
|
+
for (const entityHash of entityHashes) {
|
|
44
|
+
const logicalPaths = await entityStore.listEntityFiles(entityHash)
|
|
45
|
+
for (const logicalPath of logicalPaths) {
|
|
46
|
+
manifests++
|
|
47
|
+
const raw = await entityStore.readManifest(entityHash, logicalPath)
|
|
48
|
+
const normalized = normalizeFileManifest(raw)
|
|
49
|
+
if (!normalized) {
|
|
50
|
+
brokenManifests.push({ ownerEntityHash: entityHash, logicalPath })
|
|
51
|
+
continue
|
|
52
|
+
}
|
|
53
|
+
for (const part of normalized.parts)
|
|
54
|
+
referenced.add(part.hash)
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
/** @type {GcChunkTarget[]} */
|
|
59
|
+
const candidates = []
|
|
60
|
+
let freedBytes = 0
|
|
61
|
+
let prefixEntries
|
|
62
|
+
try {
|
|
63
|
+
prefixEntries = await fsp.readdir(chunkStoreRoot(), { withFileTypes: true })
|
|
64
|
+
}
|
|
65
|
+
catch {
|
|
66
|
+
prefixEntries = []
|
|
67
|
+
}
|
|
68
|
+
for (const prefixEntry of prefixEntries) {
|
|
69
|
+
if (!prefixEntry.isDirectory() || prefixEntry.name.length !== 2) continue
|
|
70
|
+
let files
|
|
71
|
+
try {
|
|
72
|
+
files = await fsp.readdir(join(chunkStoreRoot(), prefixEntry.name))
|
|
73
|
+
}
|
|
74
|
+
catch { continue }
|
|
75
|
+
for (const name of files) {
|
|
76
|
+
if (!name.endsWith('.bin')) continue
|
|
77
|
+
const hash = name.slice(0, -4)
|
|
78
|
+
if (!isHex64(hash) || referenced.has(hash)) continue
|
|
79
|
+
const size = await chunkFileSize(join(chunkStoreRoot(), prefixEntry.name, name))
|
|
80
|
+
candidates.push({ hash, size })
|
|
81
|
+
freedBytes += size
|
|
82
|
+
}
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
return {
|
|
86
|
+
manifests,
|
|
87
|
+
brokenManifests,
|
|
88
|
+
referenced: referenced.size,
|
|
89
|
+
candidates,
|
|
90
|
+
deleted: 0,
|
|
91
|
+
retained: 0,
|
|
92
|
+
freedBytes,
|
|
93
|
+
brokenDeleted: 0,
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
/**
|
|
98
|
+
* @param {string} filePath chunk 文件绝对路径
|
|
99
|
+
* @returns {Promise<number>} 文件字节数;stat 失败为 0
|
|
100
|
+
*/
|
|
101
|
+
async function chunkFileSize(filePath) {
|
|
102
|
+
try {
|
|
103
|
+
const { size } = await fsp.stat(filePath)
|
|
104
|
+
return size
|
|
105
|
+
}
|
|
106
|
+
catch { return 0 }
|
|
107
|
+
}
|
|
108
|
+
|
|
109
|
+
/**
|
|
110
|
+
* 持锁删除一批 chunk,统计删除结果并顺带清理已空前缀目录。
|
|
111
|
+
* @param {string[]} hashes 待删 64 位 hex 集合
|
|
112
|
+
* @returns {Promise<{ deleted: number, retained: number, freedBytes: number }>} 删除结果统计
|
|
113
|
+
*/
|
|
114
|
+
async function removeChunkHashes(hashes) {
|
|
115
|
+
let deleted = 0
|
|
116
|
+
let retained = 0
|
|
117
|
+
let freedBytes = 0
|
|
118
|
+
const emptiedPrefixes = new Set()
|
|
119
|
+
await withChunkStoreLock(async () => {
|
|
120
|
+
for (const hash of hashes) {
|
|
121
|
+
const result = await unlinkChunkFile(hash)
|
|
122
|
+
if (result.deleted) {
|
|
123
|
+
deleted++
|
|
124
|
+
freedBytes += result.size
|
|
125
|
+
emptiedPrefixes.add(hash.slice(0, 2))
|
|
126
|
+
}
|
|
127
|
+
else retained++
|
|
128
|
+
}
|
|
129
|
+
for (const prefix of emptiedPrefixes) try {
|
|
130
|
+
await fsp.rmdir(join(chunkStoreRoot(), prefix))
|
|
131
|
+
} catch { /* 目录非空或已被删,忽略 */ }
|
|
132
|
+
})
|
|
133
|
+
return { deleted, retained, freedBytes }
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
/**
|
|
137
|
+
* 扫描并报告 chunk 垃圾(只读,不删除)。
|
|
138
|
+
* 根集 = 全部写盘 manifest 的 `parts[].hash`(含公共/非 public 缓存);`chunks/` 中不在根集的可回收。
|
|
139
|
+
* @returns {Promise<GarbageCollectionReport>} 统计报告
|
|
140
|
+
*/
|
|
141
|
+
export async function mapChunkGarbage() {
|
|
142
|
+
return scanChunkGarbage()
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
/**
|
|
146
|
+
* 清理 chunk 垃圾。缺省 `targets` 时先扫描再删除(并自动清除扫描到的坏 manifest);
|
|
147
|
+
* 提供 `targets` 时只按给定集合删除,不做扫描、不碰 manifest。
|
|
148
|
+
* @param {{ targets?: Array<string | GcChunkTarget> }} [options] 可选显式目标集
|
|
149
|
+
* @returns {Promise<GarbageCollectionReport>} 执行报告
|
|
150
|
+
*/
|
|
151
|
+
export async function cleanChunkGarbage(options = {}) {
|
|
152
|
+
const targets = Array.isArray(options.targets)
|
|
153
|
+
|
|
154
|
+
/** @type {string[]} */
|
|
155
|
+
let hashes
|
|
156
|
+
if (targets) {
|
|
157
|
+
hashes = options.targets.map(target => {
|
|
158
|
+
const hash = typeof target === 'string' ? target : target?.hash
|
|
159
|
+
if (!isHex64(hash)) throw new Error(`cleanChunkGarbage: invalid chunk hash ${hash}`)
|
|
160
|
+
return hash
|
|
161
|
+
})
|
|
162
|
+
const { deleted, retained, freedBytes } = await removeChunkHashes(hashes)
|
|
163
|
+
return {
|
|
164
|
+
manifests: 0,
|
|
165
|
+
brokenManifests: [],
|
|
166
|
+
referenced: 0,
|
|
167
|
+
candidates: hashes.map(hash => ({ hash, size: 0 })),
|
|
168
|
+
deleted,
|
|
169
|
+
retained,
|
|
170
|
+
freedBytes,
|
|
171
|
+
brokenDeleted: 0,
|
|
172
|
+
}
|
|
173
|
+
}
|
|
174
|
+
|
|
175
|
+
const snapshot = await scanChunkGarbage()
|
|
176
|
+
const entityStore = getEntityStore()
|
|
177
|
+
const logger = getNodeLogger()
|
|
178
|
+
let brokenDeleted = 0
|
|
179
|
+
if (snapshot.brokenManifests.length) {
|
|
180
|
+
for (const broken of snapshot.brokenManifests) {
|
|
181
|
+
if (!entityStore.deleteManifest) break
|
|
182
|
+
await entityStore.deleteManifest(broken.ownerEntityHash, broken.logicalPath)
|
|
183
|
+
brokenDeleted++
|
|
184
|
+
}
|
|
185
|
+
logger?.info?.(`chunk GC: removed ${brokenDeleted} broken manifest(s)`)
|
|
186
|
+
}
|
|
187
|
+
|
|
188
|
+
const { deleted, retained, freedBytes } = await removeChunkHashes(
|
|
189
|
+
snapshot.candidates.map(candidate => candidate.hash),
|
|
190
|
+
)
|
|
191
|
+
return { ...snapshot, deleted, retained, freedBytes, brokenDeleted }
|
|
192
|
+
}
|
package/files/manifest/acl.mjs
CHANGED
|
@@ -1,38 +1,67 @@
|
|
|
1
1
|
import { isLogicalEntityHash } from '../../core/logical_entity.mjs'
|
|
2
2
|
import { isWritableLocalEntity } from '../../node/identity.mjs'
|
|
3
3
|
|
|
4
|
-
import {
|
|
4
|
+
import { resolveManifestOwner } from './routing.mjs'
|
|
5
5
|
|
|
6
6
|
/**
|
|
7
|
+
* @typedef {{
|
|
8
|
+
* replicaUsername: string,
|
|
9
|
+
* ownerEntityHash: string,
|
|
10
|
+
* manifest: import('./normalize.mjs').FileManifest | object,
|
|
11
|
+
* }} ManifestAclContext
|
|
12
|
+
*/
|
|
13
|
+
|
|
14
|
+
/** @type {Map<string, (context: ManifestAclContext, logicalPath?: string) => Promise<boolean>>} */
|
|
15
|
+
const aclHandlers = new Map()
|
|
16
|
+
|
|
17
|
+
/**
|
|
18
|
+
* 注册本族 manifest 的本地读写授权判定(ownerId 须与 registerManifestOwner 一致)。
|
|
19
|
+
* @param {string} ownerId 注册方
|
|
20
|
+
* @param {(context: ManifestAclContext, logicalPath?: string) => Promise<boolean>} handler 授权判定
|
|
21
|
+
* @returns {void}
|
|
22
|
+
*/
|
|
23
|
+
export function registerManifestAcl(ownerId, handler) {
|
|
24
|
+
aclHandlers.set(ownerId, handler)
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
/**
|
|
28
|
+
* @param {string} ownerId 注册方
|
|
29
|
+
* @returns {void}
|
|
30
|
+
*/
|
|
31
|
+
export function unregisterManifestAcl(ownerId) {
|
|
32
|
+
aclHandlers.delete(ownerId)
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
/**
|
|
36
|
+
* 本地读授权:仅认 matcher 结果——命中族须注册 ACL handler,否则 deny。
|
|
7
37
|
* @param {string} replicaUsername 请求 replica
|
|
8
38
|
* @param {string} ownerEntityHash 文件 owner
|
|
9
39
|
* @param {import('./normalize.mjs').FileManifest} manifest 清单
|
|
10
40
|
* @returns {Promise<boolean>} 是否允许读
|
|
11
41
|
*/
|
|
12
42
|
export async function canReadManifest(replicaUsername, ownerEntityHash, manifest) {
|
|
13
|
-
const
|
|
14
|
-
if (
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
if (isLogicalEntityHash(ownerEntityHash))
|
|
43
|
+
const ownerId = resolveManifestOwner(manifest, ownerEntityHash)
|
|
44
|
+
if (ownerId) {
|
|
45
|
+
const handler = aclHandlers.get(ownerId)
|
|
46
|
+
if (handler) return handler({ replicaUsername, ownerEntityHash, manifest })
|
|
18
47
|
return false
|
|
19
|
-
|
|
20
|
-
return
|
|
48
|
+
}
|
|
49
|
+
return !isLogicalEntityHash(ownerEntityHash)
|
|
21
50
|
}
|
|
22
51
|
|
|
23
52
|
/**
|
|
53
|
+
* 本地写授权:仅认 matcher 结果(写路径无 manifest,只按 ownerEntityHash 判定)。
|
|
24
54
|
* @param {string} replicaUsername 请求 replica
|
|
25
55
|
* @param {string} ownerEntityHash 所有者
|
|
26
56
|
* @param {string} logicalPath 路径
|
|
27
57
|
* @returns {Promise<boolean>} 是否允许写
|
|
28
58
|
*/
|
|
29
59
|
export async function canWriteManifestPath(replicaUsername, ownerEntityHash, logicalPath) {
|
|
30
|
-
const
|
|
31
|
-
if (
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
if (isLogicalEntityHash(ownerEntityHash))
|
|
60
|
+
const ownerId = resolveManifestOwner(null, ownerEntityHash)
|
|
61
|
+
if (ownerId) {
|
|
62
|
+
const handler = aclHandlers.get(ownerId)
|
|
63
|
+
if (handler) return handler({ replicaUsername, ownerEntityHash, manifest: {} }, logicalPath)
|
|
35
64
|
return false
|
|
36
|
-
|
|
37
|
-
return isWritableLocalEntity(ownerEntityHash)
|
|
65
|
+
}
|
|
66
|
+
return !isLogicalEntityHash(ownerEntityHash) && isWritableLocalEntity(ownerEntityHash)
|
|
38
67
|
}
|
package/files/manifest/fetch.mjs
CHANGED
|
@@ -8,7 +8,6 @@ import { loadFileManifest, saveFileManifest } from '../evfs.mjs'
|
|
|
8
8
|
import { beginFedFanoutFetch } from '../fed/fetch_shared.mjs'
|
|
9
9
|
import { canonicalizeFanoutTargets } from '../fetch_fanout.mjs'
|
|
10
10
|
|
|
11
|
-
import { resolveManifestAclType } from './acl_registry.mjs'
|
|
12
11
|
import { normalizeFileManifest } from './normalize.mjs'
|
|
13
12
|
import {
|
|
14
13
|
manifestFetchExpectedKey,
|
|
@@ -16,6 +15,7 @@ import {
|
|
|
16
15
|
registerManifestFetchWait,
|
|
17
16
|
} from './pending.mjs'
|
|
18
17
|
import { shouldPreferIncomingPublicManifest } from './public.mjs'
|
|
18
|
+
import { resolveManifestOwner } from './routing.mjs'
|
|
19
19
|
import { getManifestServicer } from './servicer_registry.mjs'
|
|
20
20
|
|
|
21
21
|
const DEFAULT_MANIFEST_FETCH_TIMEOUT_MS = ms('8s')
|
|
@@ -30,9 +30,10 @@ const manifestInflight = createInflightTable({
|
|
|
30
30
|
* 拉取 manifest。默认走 node-scope public 语义(仅接受验签公开 manifest);
|
|
31
31
|
* 传入 `fanoutTargets` 时只向目标集 fanout,并接受无签名的非 public manifest(信任边界 = 目标集 + 服务端 servicer)。
|
|
32
32
|
* 非 public 命中默认写盘;public 按 `cache: true` 择新写盘。
|
|
33
|
-
*
|
|
34
|
-
* public
|
|
35
|
-
*
|
|
33
|
+
* 本地已验签公开缓存默认立即返回(`cache: true` 时后台 fanout 刷新);`revalidate: true` 时改为阻塞等待本次 fanout 择新,
|
|
34
|
+
* 无新则回退本地。非 public / 未验签 public 本地命中仅 targeted 模式返回,public 模式拒绝。
|
|
35
|
+
* 同 key(含规范化目标集)in-flight 去重;调用方外层超时不 abort,后台继续填缓存。
|
|
36
|
+
* @param {{ username: string, ownerEntityHash: string, logicalPath: string, fanoutTargets?: string[], cache?: boolean, revalidate?: boolean, timeoutMs?: number }} context - 拉取上下文
|
|
36
37
|
* @returns {Promise<import('./normalize.mjs').FileManifest | null>} 校验后的 manifest,失败为 null
|
|
37
38
|
*/
|
|
38
39
|
export async function fetchManifest(context) {
|
|
@@ -40,6 +41,7 @@ export async function fetchManifest(context) {
|
|
|
40
41
|
const logicalPath = context.logicalPath.replace(/^\/+/, '')
|
|
41
42
|
if (!ownerEntityHash || !logicalPath || !username) return null
|
|
42
43
|
|
|
44
|
+
const revalidate = context.revalidate === true
|
|
43
45
|
const timeoutMs = Number(context.timeoutMs) > 0
|
|
44
46
|
? Number(context.timeoutMs)
|
|
45
47
|
: DEFAULT_MANIFEST_FETCH_TIMEOUT_MS
|
|
@@ -78,10 +80,18 @@ export async function fetchManifest(context) {
|
|
|
78
80
|
})
|
|
79
81
|
const local = await localPromise
|
|
80
82
|
|
|
81
|
-
//
|
|
83
|
+
// 本地已验签公开命中:默认立即返回,cache: true 时后台 fanout 择新刷新;
|
|
84
|
+
// revalidate: true 时阻塞等待本次 fanout,取新择旧并写回,fanout 无新(含超时)则回退本地。
|
|
82
85
|
const isLocalPublic = local?.transferKeyDescriptor?.type === 'public' && !!local?.meta?.publicSig
|
|
83
86
|
if (isLocalPublic) {
|
|
84
|
-
if (
|
|
87
|
+
if (shared && revalidate) {
|
|
88
|
+
const result = await shared
|
|
89
|
+
if (result && shouldPreferIncomingPublicManifest(local, result)) {
|
|
90
|
+
if (wantCache) await cachePublicManifest(ownerEntityHash, logicalPath, result)
|
|
91
|
+
return result
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
else if (wantCache && shared) void shared.then(async result => {
|
|
85
95
|
if (result && shouldPreferIncomingPublicManifest(local, result))
|
|
86
96
|
await cachePublicManifest(ownerEntityHash, logicalPath, result)
|
|
87
97
|
})
|
|
@@ -153,9 +163,9 @@ export async function handleIncomingManifestGet(payload, sendResponse, peerId) {
|
|
|
153
163
|
return
|
|
154
164
|
}
|
|
155
165
|
|
|
156
|
-
// 非 public
|
|
157
|
-
const
|
|
158
|
-
const servicer = getManifestServicer(
|
|
166
|
+
// 非 public:仅认 matcher 命中的族 servicer(无 matcher 即 deny,不按 transferKeyDescriptor.type 兜底路由)。
|
|
167
|
+
const ownerId = resolveManifestOwner(manifest, ownerEntityHash)
|
|
168
|
+
const servicer = getManifestServicer(ownerId)
|
|
159
169
|
if (!servicer) return
|
|
160
170
|
const allowed = await servicer({
|
|
161
171
|
manifest,
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Manifest 族路由:matcher 声明「该 manifest / 实体归哪个族(ownerId)所有」。
|
|
3
|
+
* matcher 是唯一路由依据(注册序先命中者);`transferKeyDescriptor.type` 可被多族复用、不参与路由。
|
|
4
|
+
* 族间 matcher 必须互斥,否则按注册顺序先到先得。
|
|
5
|
+
*/
|
|
6
|
+
|
|
7
|
+
/** @type {Array<{ ownerId: string, match: (manifest: import('./normalize.mjs').FileManifest | null, ownerEntityHash: string) => boolean }>} */
|
|
8
|
+
const owners = []
|
|
9
|
+
|
|
10
|
+
/**
|
|
11
|
+
* @param {string} ownerId 注册方
|
|
12
|
+
* @param {(manifest: import('./normalize.mjs').FileManifest | null, ownerEntityHash: string) => boolean} match 归属判定
|
|
13
|
+
* @returns {void}
|
|
14
|
+
*/
|
|
15
|
+
export function registerManifestOwner(ownerId, match) {
|
|
16
|
+
owners.push({ ownerId, match })
|
|
17
|
+
}
|
|
18
|
+
|
|
19
|
+
/**
|
|
20
|
+
* @param {string} ownerId 注册方
|
|
21
|
+
* @returns {void}
|
|
22
|
+
*/
|
|
23
|
+
export function unregisterManifestOwner(ownerId) {
|
|
24
|
+
for (let index = owners.length - 1; index >= 0; index--)
|
|
25
|
+
if (owners[index].ownerId === ownerId) owners.splice(index, 1)
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
/**
|
|
29
|
+
* 解析 manifest 归属族;无命中为 null(非 public 跨节点即 deny)。
|
|
30
|
+
* @param {import('./normalize.mjs').FileManifest | null} manifest manifest(写路径可空)
|
|
31
|
+
* @param {string} ownerEntityHash 所有者
|
|
32
|
+
* @returns {string | null} 族 id
|
|
33
|
+
*/
|
|
34
|
+
export function resolveManifestOwner(manifest, ownerEntityHash) {
|
|
35
|
+
for (const { ownerId, match } of owners)
|
|
36
|
+
if (match(manifest, ownerEntityHash)) return ownerId
|
|
37
|
+
return null
|
|
38
|
+
}
|
|
@@ -1,51 +1,43 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* Manifest servicer
|
|
3
|
-
*
|
|
2
|
+
* Manifest servicer 注册表:按族(ownerId)路由,决定跨节点是否对外提供非 public manifest。
|
|
3
|
+
* type 可被多族复用;路由只经 registerManifestOwner 的 matcher 唯一命中。服务端缺省 deny。
|
|
4
4
|
*/
|
|
5
5
|
|
|
6
|
-
/** @type {Map<string,
|
|
7
|
-
const
|
|
6
|
+
/** @type {Map<string, (context: ManifestServicerContext) => Promise<boolean> | boolean>} */
|
|
7
|
+
const servicersByOwner = new Map()
|
|
8
8
|
|
|
9
9
|
/**
|
|
10
10
|
* @typedef {{
|
|
11
|
-
*
|
|
12
|
-
*
|
|
13
|
-
*
|
|
14
|
-
*
|
|
15
|
-
*
|
|
16
|
-
*
|
|
11
|
+
* manifest: import('./normalize.mjs').FileManifest,
|
|
12
|
+
* ownerEntityHash: string,
|
|
13
|
+
* logicalPath: string,
|
|
14
|
+
* requesterNodeHash: string,
|
|
15
|
+
* peerId: string,
|
|
16
|
+
* payload: object,
|
|
17
17
|
* }} ManifestServicerContext
|
|
18
18
|
*/
|
|
19
19
|
|
|
20
20
|
/**
|
|
21
|
-
* @param {string}
|
|
22
|
-
* @param {string} ownerId 注册方
|
|
21
|
+
* @param {string} ownerId 注册方(族,须与 registerManifestOwner 一致)
|
|
23
22
|
* @param {(context: ManifestServicerContext) => Promise<boolean> | boolean} handler 授权判定
|
|
24
23
|
* @returns {void}
|
|
25
24
|
*/
|
|
26
|
-
export function registerManifestServicer(
|
|
27
|
-
|
|
28
|
-
if (existing && existing.ownerId !== ownerId)
|
|
29
|
-
throw new Error(`manifest servicer for '${type}' already registered by '${existing.ownerId}'`)
|
|
30
|
-
servicersByType.set(type, { ownerId, handler })
|
|
25
|
+
export function registerManifestServicer(ownerId, handler) {
|
|
26
|
+
servicersByOwner.set(ownerId, handler)
|
|
31
27
|
}
|
|
32
28
|
|
|
33
29
|
/**
|
|
34
|
-
* @param {string} type ACL 类型
|
|
35
30
|
* @param {string} ownerId 注册方
|
|
36
31
|
* @returns {void}
|
|
37
32
|
*/
|
|
38
|
-
export function unregisterManifestServicer(
|
|
39
|
-
|
|
40
|
-
if (!existing) return
|
|
41
|
-
if (existing.ownerId === ownerId)
|
|
42
|
-
servicersByType.delete(type)
|
|
33
|
+
export function unregisterManifestServicer(ownerId) {
|
|
34
|
+
servicersByOwner.delete(ownerId)
|
|
43
35
|
}
|
|
44
36
|
|
|
45
37
|
/**
|
|
46
|
-
* @param {string}
|
|
38
|
+
* @param {string} ownerId 族 id
|
|
47
39
|
* @returns {(context: ManifestServicerContext) => Promise<boolean> | boolean | undefined} 已注册 handler
|
|
48
40
|
*/
|
|
49
|
-
export function getManifestServicer(
|
|
50
|
-
return
|
|
41
|
+
export function getManifestServicer(ownerId) {
|
|
42
|
+
return servicersByOwner.get(ownerId)
|
|
51
43
|
}
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
import { resolveManifestOwner } from './manifest/routing.mjs'
|
|
2
|
+
|
|
1
3
|
/** @type {Map<string, { getGroupFileMasterKey?: (replicaUsername: string, groupId: string, keyGeneration?: number) => Promise<Buffer | string | null>, getVaultMasterKey?: (replicaUsername: string, entityHash: string) => Promise<Buffer | string | null> }>} */
|
|
2
4
|
const transferDependenciesByOwner = new Map()
|
|
3
5
|
|
|
@@ -5,14 +7,13 @@ const transferDependenciesByOwner = new Map()
|
|
|
5
7
|
const dagPlaintextReadersByOwner = new Map()
|
|
6
8
|
|
|
7
9
|
/**
|
|
8
|
-
* @param {string} ownerId
|
|
10
|
+
* @param {string} ownerId 注册方(族,须与 registerManifestOwner 一致)
|
|
9
11
|
* @param {{ getGroupFileMasterKey?: (replicaUsername: string, groupId: string, keyGeneration?: number) => Promise<Buffer | string | null>, getVaultMasterKey?: (replicaUsername: string, entityHash: string) => Promise<Buffer | string | null> }} dependencies 密钥源
|
|
10
12
|
* @returns {void}
|
|
11
13
|
*/
|
|
12
14
|
export function registerTransferKeyDependencies(ownerId, dependencies) {
|
|
13
|
-
const
|
|
14
|
-
|
|
15
|
-
transferDependenciesByOwner.set(key, { ...prev, ...dependencies })
|
|
15
|
+
const prev = transferDependenciesByOwner.get(ownerId) || {}
|
|
16
|
+
transferDependenciesByOwner.set(ownerId, { ...prev, ...dependencies })
|
|
16
17
|
}
|
|
17
18
|
|
|
18
19
|
/**
|
|
@@ -24,48 +25,13 @@ export function registerDagManifestPlaintextReader(ownerId, reader) {
|
|
|
24
25
|
dagPlaintextReadersByOwner.set(ownerId, reader)
|
|
25
26
|
}
|
|
26
27
|
|
|
27
|
-
/** @type {Array<{ ownerId: string, match: (manifest: import('./manifest/normalize.mjs').FileManifest) => boolean }>} */
|
|
28
|
-
const manifestOwnerMatchers = []
|
|
29
|
-
|
|
30
|
-
/**
|
|
31
|
-
* @param {string} ownerId 注册方
|
|
32
|
-
* @param {(manifest: import('./manifest/normalize.mjs').FileManifest) => boolean} match manifest 匹配谓词
|
|
33
|
-
* @returns {void}
|
|
34
|
-
*/
|
|
35
|
-
export function registerManifestOwnerMatcher(ownerId, match) {
|
|
36
|
-
manifestOwnerMatchers.push({ ownerId, match })
|
|
37
|
-
}
|
|
38
|
-
|
|
39
|
-
/**
|
|
40
|
-
* @param {string} ownerId 注册方
|
|
41
|
-
* @returns {void}
|
|
42
|
-
*/
|
|
43
|
-
export function unregisterManifestOwnerMatchers(ownerId) {
|
|
44
|
-
const id = ownerId
|
|
45
|
-
for (let i = manifestOwnerMatchers.length - 1; i >= 0; i--)
|
|
46
|
-
if (manifestOwnerMatchers[i].ownerId === id) manifestOwnerMatchers.splice(i, 1)
|
|
47
|
-
}
|
|
48
|
-
|
|
49
28
|
/**
|
|
50
29
|
* @param {string} ownerId 注册方
|
|
51
30
|
* @returns {void}
|
|
52
31
|
*/
|
|
53
32
|
export function unregisterTransferKeyDependencies(ownerId) {
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
dagPlaintextReadersByOwner.delete(key)
|
|
57
|
-
unregisterManifestOwnerMatchers(key)
|
|
58
|
-
}
|
|
59
|
-
|
|
60
|
-
/**
|
|
61
|
-
* 从 manifest 推断 transfer key 注册方 id(按 registerManifestOwnerMatcher 注册顺序匹配)。
|
|
62
|
-
* @param {import('./manifest/normalize.mjs').FileManifest} manifest 清单
|
|
63
|
-
* @returns {string | null} 所有者 id
|
|
64
|
-
*/
|
|
65
|
-
export function resolveManifestTransferOwnerId(manifest) {
|
|
66
|
-
for (const { ownerId, match } of manifestOwnerMatchers)
|
|
67
|
-
if (match(manifest)) return ownerId
|
|
68
|
-
return null
|
|
33
|
+
transferDependenciesByOwner.delete(ownerId)
|
|
34
|
+
dagPlaintextReadersByOwner.delete(ownerId)
|
|
69
35
|
}
|
|
70
36
|
|
|
71
37
|
/**
|
|
@@ -74,7 +40,7 @@ export function resolveManifestTransferOwnerId(manifest) {
|
|
|
74
40
|
* @returns {{ getGroupFileMasterKey?: (replicaUsername: string, groupId: string, keyGeneration?: number) => Promise<Buffer | string | null>, getVaultMasterKey?: (replicaUsername: string, entityHash: string) => Promise<Buffer | string | null> }} 依赖
|
|
75
41
|
*/
|
|
76
42
|
export function resolveTransferKeyDependencies(ownerId, manifest) {
|
|
77
|
-
const resolved = ownerId || (manifest ?
|
|
43
|
+
const resolved = ownerId || (manifest ? resolveManifestOwner(manifest, manifest.ownerEntityHash) : null)
|
|
78
44
|
if (resolved) return transferDependenciesByOwner.get(resolved) || {}
|
|
79
45
|
return {}
|
|
80
46
|
}
|
|
@@ -85,7 +51,7 @@ export function resolveTransferKeyDependencies(ownerId, manifest) {
|
|
|
85
51
|
* @returns {Promise<Buffer | null>} 明文;无 reader 或未命中为 null
|
|
86
52
|
*/
|
|
87
53
|
export async function readDagManifestPlaintext(replicaUsername, manifest) {
|
|
88
|
-
const ownerId =
|
|
54
|
+
const ownerId = resolveManifestOwner(manifest, manifest.ownerEntityHash)
|
|
89
55
|
if (ownerId) {
|
|
90
56
|
const reader = dagPlaintextReadersByOwner.get(ownerId)
|
|
91
57
|
if (reader)
|
|
@@ -93,9 +59,7 @@ export async function readDagManifestPlaintext(replicaUsername, manifest) {
|
|
|
93
59
|
const buffer = await reader(replicaUsername, manifest)
|
|
94
60
|
if (buffer?.length) return buffer
|
|
95
61
|
}
|
|
96
|
-
catch { /*
|
|
97
|
-
return null
|
|
62
|
+
catch { /* 读取失败则回落到分块路径 */ }
|
|
98
63
|
}
|
|
99
|
-
|
|
100
64
|
return null
|
|
101
65
|
}
|
package/node/entity_store.mjs
CHANGED
|
@@ -17,6 +17,7 @@ import { readJsonFile, writeJsonFile } from '../utils/json_io.mjs'
|
|
|
17
17
|
* @property {(entityHash: string, logicalPath: string) => Promise<object | null>} readManifest
|
|
18
18
|
* @property {(entityHash: string, logicalPath: string, data: object) => Promise<void>} writeManifest
|
|
19
19
|
* @property {(entityHash: string, logicalPath: string) => Promise<boolean>} statManifest
|
|
20
|
+
* @property {(entityHash: string, logicalPath: string) => Promise<void>} [deleteManifest]
|
|
20
21
|
* @property {() => Promise<string[]>} listEntityHashes
|
|
21
22
|
*/
|
|
22
23
|
|
|
@@ -210,5 +211,19 @@ export function createFsEntityStore(baseDir) {
|
|
|
210
211
|
return false
|
|
211
212
|
}
|
|
212
213
|
},
|
|
214
|
+
|
|
215
|
+
/**
|
|
216
|
+
* @param {string} entityHash 128 位十六进制
|
|
217
|
+
* @param {string} logicalPath EVFS 逻辑路径
|
|
218
|
+
* @returns {Promise<void>} 删除 manifest(不存在视为成功)
|
|
219
|
+
*/
|
|
220
|
+
async deleteManifest(entityHash, logicalPath) {
|
|
221
|
+
try {
|
|
222
|
+
await fsp.unlink(manifestPath(entityHash, logicalPath))
|
|
223
|
+
}
|
|
224
|
+
catch (error) {
|
|
225
|
+
if (/** @type {NodeJS.ErrnoException} */ error.code !== 'ENOENT') throw error
|
|
226
|
+
}
|
|
227
|
+
},
|
|
213
228
|
}
|
|
214
229
|
}
|
package/package.json
CHANGED
|
@@ -1,84 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Manifest ACL 注册表:按 transferKeyDescriptor.type 路由,不硬编码业务概念。
|
|
3
|
-
*/
|
|
4
|
-
|
|
5
|
-
/** @type {Map<string, { ownerId: string, handler: (context: ManifestAclContext, logicalPath?: string) => Promise<boolean> }>} */
|
|
6
|
-
const handlersByType = new Map()
|
|
7
|
-
|
|
8
|
-
/** @type {Map<string, { ownerId: string, match: (manifest: import('./normalize.mjs').FileManifest | null | undefined, ownerEntityHash: string) => string | null }>} */
|
|
9
|
-
const matchersByOwner = new Map()
|
|
10
|
-
|
|
11
|
-
/**
|
|
12
|
-
* @typedef {{
|
|
13
|
-
* replicaUsername: string,
|
|
14
|
-
* ownerEntityHash: string,
|
|
15
|
-
* manifest: import('./normalize.mjs').FileManifest,
|
|
16
|
-
* }} ManifestAclContext
|
|
17
|
-
*/
|
|
18
|
-
|
|
19
|
-
/**
|
|
20
|
-
* @param {string} type transferKeyDescriptor.type(如 vault-wrap、file-master-key-wrap)
|
|
21
|
-
* @param {string} ownerId 注册方
|
|
22
|
-
* @param {(context: ManifestAclContext, logicalPath?: string) => Promise<boolean>} handler ACL 检查
|
|
23
|
-
* @returns {void}
|
|
24
|
-
*/
|
|
25
|
-
export function registerManifestAcl(type, ownerId, handler) {
|
|
26
|
-
const existing = handlersByType.get(type)
|
|
27
|
-
if (existing && existing.ownerId !== ownerId)
|
|
28
|
-
throw new Error(`manifest acl handler for '${type}' already registered by '${existing.ownerId}'`)
|
|
29
|
-
handlersByType.set(type, { ownerId, handler })
|
|
30
|
-
}
|
|
31
|
-
|
|
32
|
-
/**
|
|
33
|
-
* @param {string} type transferKeyDescriptor.type
|
|
34
|
-
* @param {string} ownerId 注册方
|
|
35
|
-
* @returns {void}
|
|
36
|
-
*/
|
|
37
|
-
export function unregisterManifestAcl(type, ownerId) {
|
|
38
|
-
const existing = handlersByType.get(type)
|
|
39
|
-
if (!existing) return
|
|
40
|
-
if (existing.ownerId === ownerId)
|
|
41
|
-
handlersByType.delete(type)
|
|
42
|
-
}
|
|
43
|
-
|
|
44
|
-
/**
|
|
45
|
-
* @param {string} ownerId 注册方
|
|
46
|
-
* @param {(manifest: import('./normalize.mjs').FileManifest | null | undefined, ownerEntityHash: string) => string | null} match ACL 类型匹配
|
|
47
|
-
* @returns {void}
|
|
48
|
-
*/
|
|
49
|
-
export function registerManifestAclMatcher(ownerId, match) {
|
|
50
|
-
matchersByOwner.set(ownerId, { ownerId, match })
|
|
51
|
-
}
|
|
52
|
-
|
|
53
|
-
/**
|
|
54
|
-
* @param {string} ownerId 注册方
|
|
55
|
-
* @returns {void}
|
|
56
|
-
*/
|
|
57
|
-
export function unregisterManifestAclMatcher(ownerId) {
|
|
58
|
-
matchersByOwner.delete(ownerId)
|
|
59
|
-
}
|
|
60
|
-
|
|
61
|
-
/**
|
|
62
|
-
* @param {import('./normalize.mjs').FileManifest | null | undefined} manifest manifest
|
|
63
|
-
* @param {string} ownerEntityHash 所有者
|
|
64
|
-
* @returns {string | null} ACL 类型
|
|
65
|
-
*/
|
|
66
|
-
export function resolveManifestAclType(manifest, ownerEntityHash) {
|
|
67
|
-
for (const { match } of matchersByOwner.values()) {
|
|
68
|
-
const type = match(manifest, ownerEntityHash)
|
|
69
|
-
if (type) return type
|
|
70
|
-
}
|
|
71
|
-
return null
|
|
72
|
-
}
|
|
73
|
-
|
|
74
|
-
/**
|
|
75
|
-
* @param {string} type transferKeyDescriptor.type
|
|
76
|
-
* @param {ManifestAclContext} context 上下文
|
|
77
|
-
* @param {string} [logicalPath] 写路径(仅写 ACL 需要)
|
|
78
|
-
* @returns {Promise<boolean>} 是否允许(未注册为 false)
|
|
79
|
-
*/
|
|
80
|
-
export async function checkManifestAcl(type, context, logicalPath) {
|
|
81
|
-
const row = handlersByType.get(type)
|
|
82
|
-
if (!row?.handler) return false
|
|
83
|
-
return row.handler(context, logicalPath)
|
|
84
|
-
}
|