@hozon/blob 0.0.0-stage → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Paul Le Cam
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,4 +1,82 @@
1
- # Temporary Holding Version
1
+ # @hozon/blob
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
4
- If no other versions are published within 30 days, this package and version will be deleted.
3
+ Content-addressed blob service: verified writes over a `BlobBackend` with metadata in `@hozon/store-blob`.
4
+ Portable: no Node.js APIs.
5
+
6
+ ```sh
7
+ pnpm add @hozon/blob @hozon/blob-backend @hozon/store-blob @hozon/db
8
+ ```
9
+
10
+ ```ts
11
+ import { MemoryBlobBackend } from '@hozon/blob-backend'
12
+ import { createBlobService } from '@hozon/blob'
13
+ import { blobStoreDefinition } from '@hozon/store-blob'
14
+
15
+ // Registering the store definition is the caller's job.
16
+ db.register(blobStoreDefinition)
17
+ const blobs = createBlobService({ db, backend: new MemoryBlobBackend() })
18
+
19
+ const { entry, created } = await blobs.write({ stream, contentType: 'image/png' })
20
+ ```
21
+
22
+ Options: `codec` (default `blake3Codec`), `chunkSize` (default 1 MiB), `lock` (default `createMemoryBlobLock()`), and `limits` (`maxBlobSize` 1 GiB, `maxChunkSize` 16 MiB, `minChunkSize` 1 KiB).
23
+ `createBlobService` throws `Invalid chunkSize` when `chunkSize` is outside the chunk limits, and `Invalid limits` for invalid limits.
24
+
25
+ ## Write
26
+
27
+ `write(stream, { contentType?, encrypted?, keyID?, expectedID?, maxSize?, signal? })` returns `{ entry, created }`.
28
+
29
+ 1. `expectedID` is canonicalized and decoded before the body is read. A content type embedded in the ID fills an omitted `contentType`; a different one throws `ContentTypeMismatchError`. A decoded length above the limit throws `BlobTooLargeError`.
30
+ 2. The body streams into a fresh staging area while hashing. Crossing `min(maxSize, limits.maxBlobSize)` throws `BlobTooLargeError`. A stream error aborts staging and rethrows it.
31
+ 3. A computed ID different from `expectedID` throws `BlobIDMismatchError`.
32
+ 4. Under the blob lock: an already-local blob keeps its metadata (a null `contentType` is filled from this write) and returns `created: false`. Otherwise bytes are committed, then the entry and manifest are recorded, replacing any `remote-only` or `partial` state and discarding that transfer's staging area.
33
+
34
+ Bytes are committed before the row, so a failure leaves only orphan bytes, never a row without bytes.
35
+ A `contentType` that is not a string of at most 255 characters throws `Error('Invalid contentType')` before the body is read.
36
+ Aborting `signal` stops reading, discards staging, and throws `BlobWriteAbortedError`; the signal is checked again before commit.
37
+
38
+ `writeWith({ stream, ...options, fn })` records the entry and calls `fn(tx, entry)` in one store transaction, returning `{ entry, created, result }`.
39
+ It takes the blob lock before the transaction; the caller must not already be inside a store transaction (single-connection SQLite would deadlock).
40
+ If `fn` throws, the transaction rolls back and the committed bytes remain as orphans.
41
+ While `fn` runs the service holds the blob lock and, on SQLite, the single connection. `fn` must use only `tx`; it must not call blob service methods or use the outer `db`, which would deadlock.
42
+
43
+ ## Read and manage
44
+
45
+ `get(id)`, `list({ limit, cursor })`, `has(id)` (true only for `local` entries), `getChunkDigests(id)` (returns `[]` for non-`local` entries: unverified peer digests are never re-served), `createReadStream(id, range?)` (local blobs only, inclusive `BlobRange`), `setPinned(id, pinned)`, and `delete(id)`.
46
+ Every method canonicalizes `id` first and throws `InvalidBlobIDError` when it is invalid.
47
+ `delete` removes the row (manifest and transfer rows cascade), aborts any transfer staging area, then deletes the bytes.
48
+
49
+ ## Peer transfer
50
+
51
+ - `beginFetch(id, { contentLength, chunkSize, chunks, contentType?, encrypted?, keyID? })` validates the untrusted manifest (`InvalidManifestError`) against the ID and limits, then records the entry as `partial` with a fresh staging ID. A local blob is a no-op; an identical manifest with live staging is idempotent; a different one resets the transfer first. An existing stub whose stored `contentLength` differs from the manifest throws `InvalidManifestError`. A zero-length blob records the service's `chunkSize`, not the peer's.
52
+ - `getPresentChunks(id)` lists staged indexes for resume. If the staging area vanished, the transfer is reset and the list is empty.
53
+ - `stageChunk({ id, index, bytes })` checks length (`ChunkLengthError`) and digest (`ChunkDigestMismatchError`) before writing.
54
+ - `completeFetch(id)` throws `TransferIncompleteError` while chunks are missing. It verifies length and whole-blob digest from the staged bytes before committing; on mismatch it aborts staging, resets the transfer, and throws `BlobIDMismatchError`.
55
+
56
+ ## Staging maintenance
57
+
58
+ `pruneStaging(olderThan)` returns `{ removed, failed }`. It skips areas modified after `olderThan`.
59
+ Only areas with the service's own `w-` (write) and `t-` (transfer) ID prefixes are considered; other names are left untouched. A failure on one area (for example a backend abort error) is counted in `failed` and does not stop the others.
60
+ Transfer areas are aborted and reset only if, under the blob lock, the session still points at that staging ID and is itself older than `olderThan`.
61
+ Write areas are aborted unless they belong to a write in progress in this process. Writes in other processes are protected only by the age threshold: `olderThan` must exceed the longest upload duration.
62
+ A backend without `listStaging` returns `{ removed: 0, failed: 0 }`.
63
+ The backend's staging namespace must not be shared with non-service writers that use `w-`/`t-` IDs: such areas would be pruned.
64
+ Abandoned `remote-only` stub rows (for example after a reset) are never removed automatically; there is no garbage collection for them.
65
+
66
+ ## Locking
67
+
68
+ `BlobLock` is `withLock(id, fn)`, keyed by canonical blob ID.
69
+ Lock order is always blob lock, then store transaction. Locks are non-reentrant: public methods take the lock once and call unlocked helpers.
70
+ `write` (commit and record), `delete`, `setPinned`, `beginFetch`, `stageChunk`, `completeFetch`, transfer resets, and pruning of a transfer area run under the lock.
71
+
72
+ - `createMemoryBlobLock()` (default): in-process keyed mutex, correct for a single process.
73
+ - `createFileBlobLock({ directory, acquireTimeoutMs? })`: from `@hozon/blob-node-fs`: cross-process lock for processes on one host sharing storage, one lock file per ID. `directory` must be on a local filesystem. Exclusion is as strong as `@sozai/lock` provides; its stale-lock reaping has a small documented exclusion gap.
74
+ - Multi-host deployments sharing a backend and database need a distributed `BlobLock`. None ships; supply your own. A failed acquire throws `BlobLockTimeoutError`.
75
+
76
+ ## Store registration
77
+
78
+ The caller must register `blobStoreDefinition` on the `HozonDB` before creating the service. The service does not register it.
79
+
80
+ ## Errors
81
+
82
+ `BlobNotFoundError`, `BlobTooLargeError`, `BlobIDMismatchError`, `ContentTypeMismatchError`, `ChunkDigestMismatchError`, `ChunkLengthError`, `InvalidManifestError`, `InvalidRangeError`, `BlobWriteAbortedError`, `TransferIncompleteError`, and the re-exported `InvalidBlobIDError` and `BlobLockTimeoutError`.
@@ -0,0 +1,52 @@
1
+ export declare class BlobNotFoundError extends Error {
2
+ constructor(id: string, options?: ErrorOptions);
3
+ }
4
+ export declare class BlobTooLargeError extends Error {
5
+ constructor(limit: number, options?: ErrorOptions);
6
+ }
7
+ export type BlobIDMismatchErrorParams = {
8
+ expected: string;
9
+ actual: string;
10
+ };
11
+ export declare class BlobIDMismatchError extends Error {
12
+ constructor(params: BlobIDMismatchErrorParams, options?: ErrorOptions);
13
+ }
14
+ export type ContentTypeMismatchErrorParams = {
15
+ expected: string;
16
+ actual: string;
17
+ };
18
+ export declare class ContentTypeMismatchError extends Error {
19
+ constructor(params: ContentTypeMismatchErrorParams, options?: ErrorOptions);
20
+ }
21
+ export type ChunkDigestMismatchErrorParams = {
22
+ id: string;
23
+ index: number;
24
+ };
25
+ export declare class ChunkDigestMismatchError extends Error {
26
+ constructor(params: ChunkDigestMismatchErrorParams, options?: ErrorOptions);
27
+ }
28
+ export type ChunkLengthErrorParams = {
29
+ id: string;
30
+ index: number;
31
+ expected: number;
32
+ actual: number;
33
+ };
34
+ export declare class ChunkLengthError extends Error {
35
+ constructor(params: ChunkLengthErrorParams, options?: ErrorOptions);
36
+ }
37
+ export declare class InvalidManifestError extends Error {
38
+ constructor(message: string, options?: ErrorOptions);
39
+ }
40
+ export declare class InvalidRangeError extends Error {
41
+ constructor(message: string, options?: ErrorOptions);
42
+ }
43
+ export declare class BlobWriteAbortedError extends Error {
44
+ constructor(message?: string, options?: ErrorOptions);
45
+ }
46
+ export type TransferIncompleteErrorParams = {
47
+ id: string;
48
+ missing: number;
49
+ };
50
+ export declare class TransferIncompleteError extends Error {
51
+ constructor(params: TransferIncompleteErrorParams, options?: ErrorOptions);
52
+ }
package/lib/errors.js ADDED
@@ -0,0 +1,60 @@
1
+ export class BlobNotFoundError extends Error {
2
+ constructor(id, options){
3
+ super(`Blob not found: ${id}`, options);
4
+ this.name = 'BlobNotFoundError';
5
+ }
6
+ }
7
+ export class BlobTooLargeError extends Error {
8
+ constructor(limit, options){
9
+ super(`Blob exceeds the size limit of ${limit} bytes`, options);
10
+ this.name = 'BlobTooLargeError';
11
+ }
12
+ }
13
+ export class BlobIDMismatchError extends Error {
14
+ constructor(params, options){
15
+ super(`Blob ID mismatch: expected ${params.expected}, got ${params.actual}`, options);
16
+ this.name = 'BlobIDMismatchError';
17
+ }
18
+ }
19
+ export class ContentTypeMismatchError extends Error {
20
+ constructor(params, options){
21
+ super(`Content type mismatch: expected ${JSON.stringify(params.expected)}, got ${JSON.stringify(params.actual)}`, options);
22
+ this.name = 'ContentTypeMismatchError';
23
+ }
24
+ }
25
+ export class ChunkDigestMismatchError extends Error {
26
+ constructor(params, options){
27
+ super(`Chunk ${params.index} of blob ${params.id} does not match its manifest digest`, options);
28
+ this.name = 'ChunkDigestMismatchError';
29
+ }
30
+ }
31
+ export class ChunkLengthError extends Error {
32
+ constructor(params, options){
33
+ super(`Chunk ${params.index} of blob ${params.id} must be ${params.expected} bytes, got ${params.actual}`, options);
34
+ this.name = 'ChunkLengthError';
35
+ }
36
+ }
37
+ export class InvalidManifestError extends Error {
38
+ constructor(message, options){
39
+ super(`Invalid manifest: ${message}`, options);
40
+ this.name = 'InvalidManifestError';
41
+ }
42
+ }
43
+ export class InvalidRangeError extends Error {
44
+ constructor(message, options){
45
+ super(`Invalid range: ${message}`, options);
46
+ this.name = 'InvalidRangeError';
47
+ }
48
+ }
49
+ export class BlobWriteAbortedError extends Error {
50
+ constructor(message = 'Blob write aborted', options){
51
+ super(message, options);
52
+ this.name = 'BlobWriteAbortedError';
53
+ }
54
+ }
55
+ export class TransferIncompleteError extends Error {
56
+ constructor(params, options){
57
+ super(`Cannot complete transfer of ${params.id}: ${params.missing} chunk(s) missing`, options);
58
+ this.name = 'TransferIncompleteError';
59
+ }
60
+ }
package/lib/fresh.d.ts ADDED
@@ -0,0 +1,2 @@
1
+ import type { BlobEntry, BlobStoreAPI } from '@hozon/store-blob';
2
+ export declare function getFreshEntry(store: BlobStoreAPI, blobID: string): Promise<BlobEntry | null>;
package/lib/fresh.js ADDED
@@ -0,0 +1,6 @@
1
+ // A scope remembers getEntry results; reads made under the blob lock must see the current row.
2
+ export async function getFreshEntry(store, blobID) {
3
+ return (await store.getEntries([
4
+ blobID
5
+ ])).get(blobID) ?? null;
6
+ }
package/lib/index.d.ts ADDED
@@ -0,0 +1,8 @@
1
+ export { BlobLockTimeoutError } from '@hozon/blob-backend';
2
+ export { InvalidBlobIDError } from '@hozon/blob-id';
3
+ export { InvalidCursorError } from '@hozon/store-blob';
4
+ export { BlobIDMismatchError, type BlobIDMismatchErrorParams, BlobNotFoundError, BlobTooLargeError, BlobWriteAbortedError, ChunkDigestMismatchError, type ChunkDigestMismatchErrorParams, ChunkLengthError, type ChunkLengthErrorParams, ContentTypeMismatchError, type ContentTypeMismatchErrorParams, InvalidManifestError, InvalidRangeError, TransferIncompleteError, type TransferIncompleteErrorParams, } from './errors.js';
5
+ export { type BlobLimits, DEFAULT_CHUNK_SIZE, DEFAULT_LIMITS } from './limits.js';
6
+ export { type BlobService, type BlobServiceParams, createBlobService } from './service.js';
7
+ export type { StageChunkParams, TransferManifest } from './transfer.js';
8
+ export type { WriteOptions, WriteParams, WriteResult } from './write.js';
package/lib/index.js ADDED
@@ -0,0 +1,6 @@
1
+ export { BlobLockTimeoutError } from '@hozon/blob-backend';
2
+ export { InvalidBlobIDError } from '@hozon/blob-id';
3
+ export { InvalidCursorError } from '@hozon/store-blob';
4
+ export { BlobIDMismatchError, BlobNotFoundError, BlobTooLargeError, BlobWriteAbortedError, ChunkDigestMismatchError, ChunkLengthError, ContentTypeMismatchError, InvalidManifestError, InvalidRangeError, TransferIncompleteError } from './errors.js';
5
+ export { DEFAULT_CHUNK_SIZE, DEFAULT_LIMITS } from './limits.js';
6
+ export { createBlobService } from './service.js';
@@ -0,0 +1,9 @@
1
+ export type BlobLimits = {
2
+ maxBlobSize: number;
3
+ maxChunkSize: number;
4
+ minChunkSize: number;
5
+ };
6
+ export declare const DEFAULT_CHUNK_SIZE: number;
7
+ export declare const DEFAULT_LIMITS: BlobLimits;
8
+ export declare function resolveLimits(limits?: Partial<BlobLimits>): BlobLimits;
9
+ export declare function assertChunkSize(chunkSize: number, limits: BlobLimits): void;
package/lib/limits.js ADDED
@@ -0,0 +1,25 @@
1
+ export const DEFAULT_CHUNK_SIZE = 1024 * 1024;
2
+ export const DEFAULT_LIMITS = {
3
+ maxBlobSize: 1024 * 1024 * 1024,
4
+ maxChunkSize: 16 * 1024 * 1024,
5
+ minChunkSize: 1024
6
+ };
7
+ function isPositiveInteger(value) {
8
+ return Number.isSafeInteger(value) && value > 0;
9
+ }
10
+ export function resolveLimits(limits = {}) {
11
+ const resolved = {
12
+ maxBlobSize: limits.maxBlobSize ?? DEFAULT_LIMITS.maxBlobSize,
13
+ maxChunkSize: limits.maxChunkSize ?? DEFAULT_LIMITS.maxChunkSize,
14
+ minChunkSize: limits.minChunkSize ?? DEFAULT_LIMITS.minChunkSize
15
+ };
16
+ if (!Number.isSafeInteger(resolved.maxBlobSize) || resolved.maxBlobSize < 0 || !isPositiveInteger(resolved.minChunkSize) || !isPositiveInteger(resolved.maxChunkSize) || resolved.minChunkSize > resolved.maxChunkSize) {
17
+ throw new Error('Invalid limits');
18
+ }
19
+ return resolved;
20
+ }
21
+ export function assertChunkSize(chunkSize, limits) {
22
+ if (!Number.isSafeInteger(chunkSize) || chunkSize < limits.minChunkSize || chunkSize > limits.maxChunkSize) {
23
+ throw new Error('Invalid chunkSize');
24
+ }
25
+ }
@@ -0,0 +1,11 @@
1
+ import type { BlobBackend, BlobLock } from '@hozon/blob-backend';
2
+ import type { BlobIDCodec } from '@hozon/blob-id';
3
+ import type { StoreProvider } from '@hozon/db';
4
+ export type ManageContext = {
5
+ db: StoreProvider;
6
+ backend: BlobBackend;
7
+ codec: BlobIDCodec;
8
+ lock: BlobLock;
9
+ };
10
+ export declare function setBlobPinned(ctx: ManageContext, id: string, pinned: boolean): Promise<void>;
11
+ export declare function deleteBlob(ctx: ManageContext, id: string): Promise<boolean>;
package/lib/manage.js ADDED
@@ -0,0 +1,31 @@
1
+ import { getBlobStore } from '@hozon/store-blob';
2
+ import { BlobNotFoundError } from './errors.js';
3
+ import { getFreshEntry } from './fresh.js';
4
+ export async function setBlobPinned(ctx, id, pinned) {
5
+ const blobID = ctx.codec.canonicalize(id);
6
+ await ctx.lock.withLock(blobID, async ()=>{
7
+ const store = await getBlobStore(ctx.db);
8
+ if (await getFreshEntry(store, blobID) === null) throw new BlobNotFoundError(blobID);
9
+ await store.setPinned(blobID, pinned);
10
+ });
11
+ }
12
+ export async function deleteBlob(ctx, id) {
13
+ const blobID = ctx.codec.canonicalize(id);
14
+ return await ctx.lock.withLock(blobID, async ()=>{
15
+ const store = await getBlobStore(ctx.db);
16
+ const entry = await getFreshEntry(store, blobID);
17
+ if (entry === null) return false;
18
+ const transfer = entry.state === 'partial' ? await store.getTransfer(blobID) : null;
19
+ // The row goes first: a crash then leaves orphaned bytes, never a row without bytes.
20
+ await store.deleteEntry(blobID);
21
+ if (transfer !== null) {
22
+ try {
23
+ await ctx.backend.abortStaging(transfer.stagingID);
24
+ } catch {
25
+ // Reclaimed by staging pruning.
26
+ }
27
+ }
28
+ if (entry.state === 'local') await ctx.backend.delete(blobID);
29
+ return true;
30
+ });
31
+ }
package/lib/prune.d.ts ADDED
@@ -0,0 +1,5 @@
1
+ import type { WriteContext } from './write.js';
2
+ export declare function pruneStaging(ctx: WriteContext, olderThan: Date): Promise<{
3
+ removed: number;
4
+ failed: number;
5
+ }>;
package/lib/prune.js ADDED
@@ -0,0 +1,56 @@
1
+ import { getBlobStore } from '@hozon/store-blob';
2
+ // Removes abandoned staging areas last modified before `olderThan`. Only areas
3
+ // with this service's `w-` and `t-` ID prefixes are considered. A failure on one
4
+ // area is counted in `failed` and does not stop the others.
5
+ export async function pruneStaging(ctx, olderThan) {
6
+ if (ctx.backend.listStaging === undefined) return {
7
+ removed: 0,
8
+ failed: 0
9
+ };
10
+ const cutoff = olderThan.getTime();
11
+ const store = await getBlobStore(ctx.db);
12
+ let removed = 0;
13
+ let failed = 0;
14
+ // Collect first: aborting while iterating could disturb the backend's listing.
15
+ const old = [];
16
+ for await (const area of ctx.backend.listStaging()){
17
+ if (!isOwnedStagingID(area.stagingID)) continue;
18
+ if (area.modifiedAt.getTime() < cutoff) old.push(area.stagingID);
19
+ }
20
+ for (const stagingID of old){
21
+ try {
22
+ const session = await store.getTransferByStagingID(stagingID);
23
+ if (session === null) {
24
+ // Write staging IDs are random and never reused, so a flag check is enough.
25
+ // Other processes' writes are protected by the age threshold.
26
+ if (ctx.activeWrites.has(stagingID)) continue;
27
+ await ctx.backend.abortStaging(stagingID);
28
+ removed++;
29
+ continue;
30
+ }
31
+ if (session.updatedAt >= cutoff) continue;
32
+ // One lock acquisition per transfer area; the session is re-read under it
33
+ // because a chunk may have been staged since the listing.
34
+ const pruned = await ctx.lock.withLock(session.blobID, async ()=>{
35
+ const locked = await getBlobStore(ctx.db);
36
+ const current = await locked.getTransfer(session.blobID);
37
+ if (current === null || current.stagingID !== stagingID || current.updatedAt >= cutoff) {
38
+ return false;
39
+ }
40
+ await ctx.backend.abortStaging(stagingID);
41
+ await locked.resetTransfer(session.blobID);
42
+ return true;
43
+ });
44
+ if (pruned) removed++;
45
+ } catch {
46
+ failed++;
47
+ }
48
+ }
49
+ return {
50
+ removed,
51
+ failed
52
+ };
53
+ }
54
+ function isOwnedStagingID(stagingID) {
55
+ return stagingID.startsWith('w-') || stagingID.startsWith('t-');
56
+ }
package/lib/read.d.ts ADDED
@@ -0,0 +1,20 @@
1
+ import type { BlobBackend, BlobRange } from '@hozon/blob-backend';
2
+ import type { BlobIDCodec } from '@hozon/blob-id';
3
+ import type { StoreProvider } from '@hozon/db';
4
+ import type { BlobEntry } from '@hozon/store-blob';
5
+ export type ReadContext = {
6
+ db: StoreProvider;
7
+ backend: BlobBackend;
8
+ codec: BlobIDCodec;
9
+ };
10
+ export declare function getEntry(ctx: ReadContext, id: string): Promise<BlobEntry | null>;
11
+ export declare function listEntries(ctx: ReadContext, params: {
12
+ limit: number;
13
+ cursor?: string;
14
+ }): Promise<{
15
+ entries: Array<BlobEntry>;
16
+ nextCursor: string | null;
17
+ }>;
18
+ export declare function hasBlob(ctx: ReadContext, id: string): Promise<boolean>;
19
+ export declare function getChunkDigests(ctx: ReadContext, id: string): Promise<Array<Uint8Array>>;
20
+ export declare function createBlobReadStream(ctx: ReadContext, id: string, range?: BlobRange): Promise<ReadableStream<Uint8Array>>;
package/lib/read.js ADDED
@@ -0,0 +1,37 @@
1
+ import { getBlobStore } from '@hozon/store-blob';
2
+ import { BlobNotFoundError, InvalidRangeError } from './errors.js';
3
+ export async function getEntry(ctx, id) {
4
+ const store = await getBlobStore(ctx.db);
5
+ return await store.getEntry(ctx.codec.canonicalize(id));
6
+ }
7
+ export async function listEntries(ctx, params) {
8
+ const store = await getBlobStore(ctx.db);
9
+ return await store.listEntries(params);
10
+ }
11
+ export async function hasBlob(ctx, id) {
12
+ return (await getEntry(ctx, id))?.state === 'local';
13
+ }
14
+ export async function getChunkDigests(ctx, id) {
15
+ const blobID = ctx.codec.canonicalize(id);
16
+ const store = await getBlobStore(ctx.db);
17
+ // Digests of non-local entries come from an unverified peer manifest.
18
+ const entry = await store.getEntry(blobID);
19
+ if (entry?.state !== 'local') return [];
20
+ return await store.getChunkDigests(blobID);
21
+ }
22
+ export async function createBlobReadStream(ctx, id, range) {
23
+ const blobID = ctx.codec.canonicalize(id);
24
+ const store = await getBlobStore(ctx.db);
25
+ const entry = await store.getEntry(blobID);
26
+ if (entry === null || entry.state !== 'local') throw new BlobNotFoundError(blobID);
27
+ if (range !== undefined) {
28
+ const { start, end } = range;
29
+ if (!Number.isSafeInteger(start) || !Number.isSafeInteger(end) || start < 0 || start > end) {
30
+ throw new InvalidRangeError(`bytes ${start}-${end}`);
31
+ }
32
+ if (end >= entry.contentLength) {
33
+ throw new InvalidRangeError(`end ${end} is beyond content length ${entry.contentLength}`);
34
+ }
35
+ }
36
+ return await ctx.backend.createReadStream(blobID, range);
37
+ }
@@ -0,0 +1,49 @@
1
+ import type { BlobBackend, BlobLock, BlobRange } from '@hozon/blob-backend';
2
+ import type { BlobIDCodec } from '@hozon/blob-id';
3
+ import type { StoreProvider } from '@hozon/db';
4
+ import type { BlobEntry } from '@hozon/store-blob';
5
+ import type { Runtime } from '@sozai/runtime';
6
+ import { type BlobLimits } from './limits.js';
7
+ import { type StageChunkParams, type TransferManifest } from './transfer.js';
8
+ import { type WriteParams, type WriteResult } from './write.js';
9
+ export type BlobServiceParams = {
10
+ db: StoreProvider;
11
+ backend: BlobBackend;
12
+ codec?: BlobIDCodec;
13
+ chunkSize?: number;
14
+ lock?: BlobLock;
15
+ limits?: Partial<BlobLimits>;
16
+ runtime?: Runtime;
17
+ };
18
+ export type BlobService = {
19
+ write(params: WriteParams): Promise<WriteResult>;
20
+ writeWith<T>(params: WriteParams & {
21
+ fn: (tx: StoreProvider, entry: BlobEntry) => Promise<T>;
22
+ }): Promise<WriteResult & {
23
+ result: T;
24
+ }>;
25
+ get(id: string): Promise<BlobEntry | null>;
26
+ list(params: {
27
+ limit: number;
28
+ cursor?: string;
29
+ }): Promise<{
30
+ entries: Array<BlobEntry>;
31
+ nextCursor: string | null;
32
+ }>;
33
+ has(id: string): Promise<boolean>;
34
+ getChunkDigests(id: string): Promise<Array<Uint8Array>>;
35
+ createReadStream(id: string, range?: BlobRange): Promise<ReadableStream<Uint8Array>>;
36
+ setPinned(id: string, pinned: boolean): Promise<void>;
37
+ delete(id: string): Promise<boolean>;
38
+ beginFetch(id: string, manifest: TransferManifest): Promise<void>;
39
+ getPresentChunks(id: string): Promise<Array<number>>;
40
+ stageChunk(params: StageChunkParams): Promise<void>;
41
+ completeFetch(id: string): Promise<BlobEntry>;
42
+ pruneStaging(olderThan: Date): Promise<{
43
+ removed: number;
44
+ failed: number;
45
+ }>;
46
+ readonly codec: BlobIDCodec;
47
+ readonly limits: BlobLimits;
48
+ };
49
+ export declare function createBlobService(params: BlobServiceParams): BlobService;
package/lib/service.js ADDED
@@ -0,0 +1,55 @@
1
+ import { createMemoryBlobLock } from '@hozon/blob-backend';
2
+ import { blake3Codec } from '@hozon/blob-id';
3
+ import { createRuntime } from '@sozai/runtime';
4
+ import { assertChunkSize, DEFAULT_CHUNK_SIZE, resolveLimits } from './limits.js';
5
+ import { deleteBlob, setBlobPinned } from './manage.js';
6
+ import { pruneStaging } from './prune.js';
7
+ import { createBlobReadStream, getChunkDigests, getEntry, hasBlob, listEntries } from './read.js';
8
+ import { beginFetch, completeFetch, getPresentChunks, stageChunk } from './transfer.js';
9
+ import { writeBlob } from './write.js';
10
+ export function createBlobService(params) {
11
+ const limits = resolveLimits(params.limits);
12
+ const chunkSize = params.chunkSize ?? DEFAULT_CHUNK_SIZE;
13
+ assertChunkSize(chunkSize, limits);
14
+ const ctx = {
15
+ db: params.db,
16
+ backend: params.backend,
17
+ codec: params.codec ?? blake3Codec,
18
+ chunkSize,
19
+ lock: params.lock ?? createMemoryBlobLock(),
20
+ limits: Object.freeze(limits),
21
+ runtime: params.runtime ?? createRuntime(),
22
+ activeWrites: new Set()
23
+ };
24
+ return {
25
+ codec: ctx.codec,
26
+ limits: ctx.limits,
27
+ get: (id)=>getEntry(ctx, id),
28
+ list: (listParams)=>listEntries(ctx, listParams),
29
+ has: (id)=>hasBlob(ctx, id),
30
+ getChunkDigests: (id)=>getChunkDigests(ctx, id),
31
+ createReadStream: (id, range)=>createBlobReadStream(ctx, id, range),
32
+ setPinned: (id, pinned)=>setBlobPinned(ctx, id, pinned),
33
+ delete: (id)=>deleteBlob(ctx, id),
34
+ beginFetch: (id, manifest)=>beginFetch(ctx, id, manifest),
35
+ getPresentChunks: (id)=>getPresentChunks(ctx, id),
36
+ stageChunk: (params)=>stageChunk(ctx, params),
37
+ completeFetch: (id)=>completeFetch(ctx, id),
38
+ pruneStaging: (olderThan)=>pruneStaging(ctx, olderThan),
39
+ async write (params) {
40
+ const { entry, created } = await writeBlob(ctx, params);
41
+ return {
42
+ entry,
43
+ created
44
+ };
45
+ },
46
+ async writeWith (params) {
47
+ const { entry, created, result } = await writeBlob(ctx, params);
48
+ return {
49
+ entry,
50
+ created,
51
+ result: result
52
+ };
53
+ }
54
+ };
55
+ }
@@ -0,0 +1,32 @@
1
+ import type { BlobBackend, BlobLock } from '@hozon/blob-backend';
2
+ import type { BlobIDCodec } from '@hozon/blob-id';
3
+ import type { StoreProvider } from '@hozon/db';
4
+ import type { BlobEntry } from '@hozon/store-blob';
5
+ import type { Runtime } from '@sozai/runtime';
6
+ import type { BlobLimits } from './limits.js';
7
+ export type TransferManifest = {
8
+ contentLength: number;
9
+ chunkSize: number;
10
+ chunks: Array<Uint8Array>;
11
+ contentType?: string;
12
+ encrypted?: boolean;
13
+ keyID?: string;
14
+ };
15
+ export type StageChunkParams = {
16
+ id: string;
17
+ index: number;
18
+ bytes: Uint8Array;
19
+ };
20
+ export type TransferContext = {
21
+ db: StoreProvider;
22
+ backend: BlobBackend;
23
+ codec: BlobIDCodec;
24
+ chunkSize: number;
25
+ lock: BlobLock;
26
+ limits: BlobLimits;
27
+ runtime: Runtime;
28
+ };
29
+ export declare function beginFetch(ctx: TransferContext, id: string, manifest: TransferManifest): Promise<void>;
30
+ export declare function getPresentChunks(ctx: TransferContext, id: string): Promise<Array<number>>;
31
+ export declare function stageChunk(ctx: TransferContext, params: StageChunkParams): Promise<void>;
32
+ export declare function completeFetch(ctx: TransferContext, id: string): Promise<BlobEntry>;
@@ -0,0 +1,297 @@
1
+ import { getBlobStore } from '@hozon/store-blob';
2
+ import { BlobIDMismatchError, BlobNotFoundError, ChunkDigestMismatchError, ChunkLengthError, InvalidManifestError, TransferIncompleteError } from './errors.js';
3
+ import { getFreshEntry } from './fresh.js';
4
+ import { createStagingID } from './write.js';
5
+ function bytesEqual(a, b) {
6
+ if (a.length !== b.length) return false;
7
+ for(let i = 0; i < a.length; i++){
8
+ if (a[i] !== b[i]) return false;
9
+ }
10
+ return true;
11
+ }
12
+ function digestsEqual(a, b) {
13
+ return a.length === b.length && a.every((digest, i)=>bytesEqual(digest, b[i]));
14
+ }
15
+ function isSafeNonNegativeInteger(value) {
16
+ return typeof value === 'number' && Number.isSafeInteger(value) && value >= 0;
17
+ }
18
+ function chunkCount(contentLength, chunkSize) {
19
+ return contentLength === 0 ? 0 : Math.ceil(contentLength / chunkSize);
20
+ }
21
+ // Expected length of chunk `index`: `chunkSize`, or the remainder for the last chunk.
22
+ function chunkLength(entry, count, index) {
23
+ return index < count - 1 ? entry.chunkSize : entry.contentLength - (count - 1) * entry.chunkSize;
24
+ }
25
+ // Returns the content type to record: the manifest's, or the one embedded in the ID.
26
+ function validateManifest(ctx, blobID, manifest) {
27
+ const { contentLength, chunkSize, chunks } = manifest;
28
+ if (!isSafeNonNegativeInteger(contentLength)) {
29
+ throw new InvalidManifestError(`contentLength ${contentLength} is not a non-negative integer`);
30
+ }
31
+ if (!isSafeNonNegativeInteger(chunkSize)) {
32
+ throw new InvalidManifestError(`chunkSize ${chunkSize} is not a non-negative integer`);
33
+ }
34
+ // A zero-length blob has no chunks, so its chunk size is not constrained.
35
+ if (contentLength > 0 && (chunkSize < ctx.limits.minChunkSize || chunkSize > ctx.limits.maxChunkSize)) {
36
+ throw new InvalidManifestError(`chunkSize ${chunkSize} is outside [${ctx.limits.minChunkSize}, ${ctx.limits.maxChunkSize}]`);
37
+ }
38
+ if (contentLength > ctx.limits.maxBlobSize) {
39
+ throw new InvalidManifestError(`contentLength ${contentLength} exceeds the size limit of ${ctx.limits.maxBlobSize} bytes`);
40
+ }
41
+ const expectedCount = chunkCount(contentLength, chunkSize);
42
+ if (!Array.isArray(chunks) || chunks.length !== expectedCount) {
43
+ throw new InvalidManifestError(`expected ${expectedCount} chunk digest(s)`);
44
+ }
45
+ for (const [index, digest] of chunks.entries()){
46
+ if (!(digest instanceof Uint8Array) || digest.length !== ctx.codec.digestLength) {
47
+ throw new InvalidManifestError(`chunk ${index} digest must be ${ctx.codec.digestLength} bytes`);
48
+ }
49
+ }
50
+ const { contentType, encrypted, keyID } = manifest;
51
+ if (contentType !== undefined && (typeof contentType !== 'string' || contentType.length > 255)) {
52
+ throw new InvalidManifestError('contentType must be a string of at most 255 characters');
53
+ }
54
+ if (encrypted !== undefined && typeof encrypted !== 'boolean') {
55
+ throw new InvalidManifestError('encrypted must be a boolean');
56
+ }
57
+ if (keyID !== undefined && (typeof keyID !== 'string' || keyID.length > 255)) {
58
+ throw new InvalidManifestError('keyID must be a string of at most 255 characters');
59
+ }
60
+ const decoded = ctx.codec.decode(blobID);
61
+ if (decoded.contentLength !== contentLength) {
62
+ throw new InvalidManifestError(`contentLength ${contentLength} does not match the ID (${decoded.contentLength})`);
63
+ }
64
+ if (decoded.contentType === undefined) return manifest.contentType;
65
+ if (manifest.contentType !== undefined && manifest.contentType !== decoded.contentType) {
66
+ throw new InvalidManifestError(`contentType ${JSON.stringify(manifest.contentType)} does not match the ID (${JSON.stringify(decoded.contentType)})`);
67
+ }
68
+ return decoded.contentType;
69
+ }
70
+ async function abortQuietly(backend, stagingID) {
71
+ try {
72
+ await backend.abortStaging(stagingID);
73
+ } catch {
74
+ // Reclaimed by staging pruning.
75
+ }
76
+ }
77
+ // Whether the staging area of a session with staged chunks still exists.
78
+ // Backends create staging lazily, so a session without chunks has no area yet.
79
+ async function stagingExists(backend, transfer, chunkSize) {
80
+ const first = transfer.presentChunks[0];
81
+ if (first === undefined) return true;
82
+ const start = first * chunkSize;
83
+ let probe;
84
+ try {
85
+ probe = await backend.createStagingReadStream(transfer.stagingID, {
86
+ start,
87
+ end: start
88
+ });
89
+ } catch {
90
+ return false;
91
+ }
92
+ try {
93
+ await probe.cancel();
94
+ } catch {
95
+ // The probe only needed to open.
96
+ }
97
+ return true;
98
+ }
99
+ // Lock held: the transfer of a `partial` entry, or BlobNotFoundError.
100
+ async function requireTransfer(ctx, blobID) {
101
+ const store = await getBlobStore(ctx.db);
102
+ const entry = await getFreshEntry(store, blobID);
103
+ const transfer = entry?.state === 'partial' ? await store.getTransfer(blobID) : null;
104
+ if (entry === null || transfer === null) {
105
+ throw new BlobNotFoundError(blobID, {
106
+ cause: new Error('No active transfer')
107
+ });
108
+ }
109
+ return {
110
+ store,
111
+ entry,
112
+ transfer,
113
+ digests: await store.getChunkDigests(blobID)
114
+ };
115
+ }
116
+ export async function beginFetch(ctx, id, manifest) {
117
+ const blobID = ctx.codec.canonicalize(id);
118
+ const contentType = validateManifest(ctx, blobID, manifest);
119
+ // A zero-length blob has no chunks, so the peer's chunk size is meaningless.
120
+ const chunkSize = manifest.contentLength === 0 ? ctx.chunkSize : manifest.chunkSize;
121
+ await ctx.lock.withLock(blobID, async ()=>{
122
+ const store = await getBlobStore(ctx.db);
123
+ const entry = await getFreshEntry(store, blobID);
124
+ if (entry?.state === 'local') return;
125
+ // A stub with another length could never complete against this manifest.
126
+ if (entry !== null && entry.contentLength !== manifest.contentLength) {
127
+ throw new InvalidManifestError(`contentLength ${manifest.contentLength} does not match the stored entry (${entry.contentLength})`);
128
+ }
129
+ const transfer = entry === null ? null : await store.getTransfer(blobID);
130
+ if (entry !== null && transfer !== null) {
131
+ const identical = entry.state === 'partial' && entry.chunkSize === chunkSize && digestsEqual(await store.getChunkDigests(blobID), manifest.chunks);
132
+ if (identical && await stagingExists(ctx.backend, transfer, entry.chunkSize)) {
133
+ await store.touchTransfer(blobID);
134
+ return;
135
+ }
136
+ await abortQuietly(ctx.backend, transfer.stagingID);
137
+ }
138
+ if (entry === null) {
139
+ await store.insertEntry({
140
+ blobID,
141
+ contentLength: manifest.contentLength,
142
+ encrypted: manifest.encrypted ?? false,
143
+ keyID: manifest.keyID ?? null,
144
+ chunkSize,
145
+ state: 'remote-only',
146
+ createdAt: Date.now(),
147
+ contentType: contentType ?? null
148
+ }, []);
149
+ } else {
150
+ // Drops the previous manifest and progress so the new manifest replaces them.
151
+ await store.resetTransfer(blobID);
152
+ if (contentType !== undefined) await store.fillContentType(blobID, contentType);
153
+ }
154
+ const chunks = manifest.chunks.map((digest, index)=>({
155
+ index,
156
+ digest
157
+ }));
158
+ await store.beginTransfer({
159
+ blobID,
160
+ chunkSize,
161
+ chunks,
162
+ stagingID: createStagingID(ctx.runtime, 't')
163
+ });
164
+ });
165
+ }
166
+ export async function getPresentChunks(ctx, id) {
167
+ const blobID = ctx.codec.canonicalize(id);
168
+ return await ctx.lock.withLock(blobID, async ()=>{
169
+ const store = await getBlobStore(ctx.db);
170
+ const entry = await getFreshEntry(store, blobID);
171
+ if (entry?.state !== 'partial') return [];
172
+ const transfer = await store.getTransfer(blobID);
173
+ if (transfer === null) return [];
174
+ if (!await stagingExists(ctx.backend, transfer, entry.chunkSize)) {
175
+ await store.resetTransfer(blobID);
176
+ return [];
177
+ }
178
+ return transfer.presentChunks;
179
+ });
180
+ }
181
+ export async function stageChunk(ctx, params) {
182
+ const { id, index, bytes } = params;
183
+ const blobID = ctx.codec.canonicalize(id);
184
+ await ctx.lock.withLock(blobID, async ()=>{
185
+ const { store, entry, transfer, digests } = await requireTransfer(ctx, blobID);
186
+ if (!Number.isSafeInteger(index) || index < 0 || index >= digests.length) {
187
+ throw new InvalidManifestError(`chunk index ${index} is outside [0, ${digests.length})`);
188
+ }
189
+ const expectedLength = chunkLength(entry, digests.length, index);
190
+ if (bytes.length !== expectedLength) {
191
+ throw new ChunkLengthError({
192
+ id: blobID,
193
+ index,
194
+ expected: expectedLength,
195
+ actual: bytes.length
196
+ });
197
+ }
198
+ const hasher = ctx.codec.createHasher();
199
+ hasher.update(bytes);
200
+ if (!bytesEqual(hasher.digest(), digests[index])) {
201
+ throw new ChunkDigestMismatchError({
202
+ id: blobID,
203
+ index
204
+ });
205
+ }
206
+ await ctx.backend.writeChunk(transfer.stagingID, index * entry.chunkSize, bytes);
207
+ await store.recordTransferChunk(blobID, index);
208
+ });
209
+ }
210
+ // Streams staged bytes through the hasher; never buffers the whole blob.
211
+ async function hashStaging(ctx, stream, limit) {
212
+ const reader = stream.getReader();
213
+ const hasher = ctx.codec.createHasher();
214
+ let length = 0;
215
+ try {
216
+ while(true){
217
+ const { done, value } = await reader.read();
218
+ if (done) break;
219
+ length += value.length;
220
+ // Already a mismatch; stop reading.
221
+ if (length > limit) {
222
+ await reader.cancel();
223
+ break;
224
+ }
225
+ hasher.update(value);
226
+ }
227
+ } finally{
228
+ reader.releaseLock();
229
+ }
230
+ return {
231
+ digest: hasher.digest(),
232
+ length
233
+ };
234
+ }
235
+ export async function completeFetch(ctx, id) {
236
+ const blobID = ctx.codec.canonicalize(id);
237
+ return await ctx.lock.withLock(blobID, async ()=>{
238
+ const { store, entry, transfer, digests } = await requireTransfer(ctx, blobID);
239
+ const present = new Set(transfer.presentChunks);
240
+ const missing = digests.length - digests.filter((_, index)=>present.has(index)).length;
241
+ if (missing > 0) {
242
+ throw new TransferIncompleteError({
243
+ id: blobID,
244
+ missing
245
+ });
246
+ }
247
+ // A zero-length blob stages no chunks; materialize its empty area for commit.
248
+ if (digests.length === 0) {
249
+ await ctx.backend.writeChunk(transfer.stagingID, 0, new Uint8Array(0));
250
+ }
251
+ const expected = ctx.codec.decode(blobID);
252
+ let stream;
253
+ try {
254
+ stream = await ctx.backend.createStagingReadStream(transfer.stagingID);
255
+ } catch (cause) {
256
+ // The staging area vanished: the caller must begin again.
257
+ await store.resetTransfer(blobID);
258
+ throw new BlobNotFoundError(blobID, {
259
+ cause
260
+ });
261
+ }
262
+ const hashed = await hashStaging(ctx, stream, expected.contentLength);
263
+ if (hashed.length !== expected.contentLength || !bytesEqual(hashed.digest, expected.digest)) {
264
+ await abortQuietly(ctx.backend, transfer.stagingID);
265
+ await store.resetTransfer(blobID);
266
+ const actual = ctx.codec.encode({
267
+ digest: hashed.digest,
268
+ contentLength: hashed.length,
269
+ ...expected.contentType === undefined ? {} : {
270
+ contentType: expected.contentType
271
+ }
272
+ });
273
+ throw new BlobIDMismatchError({
274
+ expected: blobID,
275
+ actual
276
+ });
277
+ }
278
+ // Verified: bytes are committed before the row.
279
+ await ctx.backend.commit(transfer.stagingID, blobID);
280
+ await store.promoteEntry({
281
+ blobID,
282
+ contentLength: entry.contentLength,
283
+ encrypted: entry.encrypted,
284
+ keyID: entry.keyID,
285
+ chunkSize: entry.chunkSize,
286
+ pinned: entry.pinned,
287
+ createdAt: entry.createdAt,
288
+ contentType: entry.contentType
289
+ }, digests.map((digest, index)=>({
290
+ index,
291
+ digest
292
+ })));
293
+ const promoted = await getFreshEntry(store, blobID);
294
+ if (promoted === null) throw new Error(`Blob entry ${blobID} missing after promotion`);
295
+ return promoted;
296
+ });
297
+ }
package/lib/write.d.ts ADDED
@@ -0,0 +1,38 @@
1
+ import type { BlobBackend, BlobLock } from '@hozon/blob-backend';
2
+ import type { BlobIDCodec } from '@hozon/blob-id';
3
+ import type { StoreProvider } from '@hozon/db';
4
+ import type { BlobEntry } from '@hozon/store-blob';
5
+ import type { Runtime } from '@sozai/runtime';
6
+ import type { BlobLimits } from './limits.js';
7
+ export type WriteOptions = {
8
+ contentType?: string;
9
+ encrypted?: boolean;
10
+ keyID?: string;
11
+ expectedID?: string;
12
+ maxSize?: number;
13
+ signal?: AbortSignal;
14
+ };
15
+ export type WriteParams = WriteOptions & {
16
+ stream: ReadableStream<Uint8Array>;
17
+ };
18
+ export type WriteResult = {
19
+ entry: BlobEntry;
20
+ created: boolean;
21
+ };
22
+ export type WriteContext = {
23
+ db: StoreProvider;
24
+ backend: BlobBackend;
25
+ codec: BlobIDCodec;
26
+ chunkSize: number;
27
+ lock: BlobLock;
28
+ limits: BlobLimits;
29
+ runtime: Runtime;
30
+ activeWrites: Set<string>;
31
+ };
32
+ export type RecordFn<T> = (tx: StoreProvider, entry: BlobEntry) => Promise<T>;
33
+ export declare function createStagingID(runtime: Runtime, prefix: 'w' | 't'): string;
34
+ export declare function writeBlob<T>(ctx: WriteContext, params: WriteParams & {
35
+ fn?: RecordFn<T>;
36
+ }): Promise<WriteResult & {
37
+ result: T | undefined;
38
+ }>;
package/lib/write.js ADDED
@@ -0,0 +1,195 @@
1
+ import { hashStream } from '@hozon/blob-id';
2
+ import { getBlobStore } from '@hozon/store-blob';
3
+ import { BlobIDMismatchError, BlobTooLargeError, BlobWriteAbortedError, ContentTypeMismatchError } from './errors.js';
4
+ import { getFreshEntry } from './fresh.js';
5
+ // Random staging ID: `prefix` (`w` for writes, `t` for transfers), a dash, 32 hex chars.
6
+ export function createStagingID(runtime, prefix) {
7
+ const bytes = runtime.getRandomValues(new Uint8Array(16));
8
+ return `${prefix}-${Array.from(bytes, (byte)=>byte.toString(16).padStart(2, '0')).join('')}`;
9
+ }
10
+ function sizeLimiter(limit) {
11
+ let total = 0;
12
+ return new TransformStream({
13
+ transform (piece, controller) {
14
+ total += piece.length;
15
+ if (total > limit) throw new BlobTooLargeError(limit);
16
+ controller.enqueue(piece);
17
+ }
18
+ });
19
+ }
20
+ function throwIfAborted(signal) {
21
+ if (signal?.aborted) throw new BlobWriteAbortedError(undefined, {
22
+ cause: signal.reason
23
+ });
24
+ }
25
+ async function cancelQuietly(stream, reason) {
26
+ try {
27
+ await stream.cancel(reason);
28
+ } catch {
29
+ // The caller's error matters more than a failed cancel.
30
+ }
31
+ }
32
+ // Records the blob under the lock, inside a transaction when `fn` is given.
33
+ async function record(ctx, fn, step) {
34
+ if (fn === undefined) {
35
+ return {
36
+ entry: await step(await getBlobStore(ctx.db)),
37
+ result: undefined
38
+ };
39
+ }
40
+ return await ctx.db.withTransaction(async (tx)=>{
41
+ const store = await getBlobStore(tx);
42
+ const entry = await step(store);
43
+ return {
44
+ entry,
45
+ result: await fn(tx, entry)
46
+ };
47
+ });
48
+ }
49
+ export async function writeBlob(ctx, params) {
50
+ const { stream, fn, ...options } = params;
51
+ const { signal } = options;
52
+ if (options.contentType !== undefined && (typeof options.contentType !== 'string' || options.contentType.length > 255)) {
53
+ await cancelQuietly(stream, new Error('Invalid contentType'));
54
+ throw new Error('Invalid contentType');
55
+ }
56
+ if (options.maxSize !== undefined && (!Number.isSafeInteger(options.maxSize) || options.maxSize < 0)) {
57
+ await cancelQuietly(stream, new Error('Invalid maxSize'));
58
+ throw new Error('Invalid maxSize');
59
+ }
60
+ const limit = Math.min(options.maxSize ?? Number.POSITIVE_INFINITY, ctx.limits.maxBlobSize);
61
+ let contentType = options.contentType;
62
+ let expectedID;
63
+ // Step 1: validate the expected ID before reading the body.
64
+ try {
65
+ throwIfAborted(signal);
66
+ if (options.expectedID !== undefined) {
67
+ expectedID = ctx.codec.canonicalize(options.expectedID);
68
+ const expected = ctx.codec.decode(expectedID);
69
+ if (expected.contentType !== undefined) {
70
+ if (contentType === undefined) {
71
+ contentType = expected.contentType;
72
+ } else if (contentType !== expected.contentType) {
73
+ throw new ContentTypeMismatchError({
74
+ expected: expected.contentType,
75
+ actual: contentType
76
+ });
77
+ }
78
+ }
79
+ if (expected.contentLength > limit) throw new BlobTooLargeError(limit);
80
+ }
81
+ } catch (error) {
82
+ await cancelQuietly(stream, error);
83
+ throw error;
84
+ }
85
+ // Step 2: stream into a fresh staging area while hashing.
86
+ const stagingID = createStagingID(ctx.runtime, 'w');
87
+ ctx.activeWrites.add(stagingID);
88
+ let staged = true;
89
+ const discardStaging = async ()=>{
90
+ if (!staged) return;
91
+ staged = false;
92
+ try {
93
+ await ctx.backend.abortStaging(stagingID);
94
+ } catch {
95
+ // An orphaned staging area is reclaimed by staging pruning.
96
+ }
97
+ };
98
+ try {
99
+ let hashed;
100
+ try {
101
+ const sink = await ctx.backend.createStaging(stagingID);
102
+ const hasher = hashStream({
103
+ codec: ctx.codec,
104
+ chunkSize: ctx.chunkSize
105
+ });
106
+ await stream.pipeThrough(sizeLimiter(limit)).pipeThrough(hasher.transform).pipeTo(sink, signal === undefined ? {} : {
107
+ signal
108
+ });
109
+ hashed = await hasher.result;
110
+ } catch (error) {
111
+ // Releases the source when staging failed before the pipe started.
112
+ if (!stream.locked) await cancelQuietly(stream, error);
113
+ if (signal?.aborted) throw new BlobWriteAbortedError(undefined, {
114
+ cause: error
115
+ });
116
+ throw error;
117
+ }
118
+ // Step 3: derive the ID and check it against the expected one.
119
+ const blobID = ctx.codec.encode({
120
+ digest: hashed.digest,
121
+ contentLength: hashed.contentLength,
122
+ ...contentType === undefined ? {} : {
123
+ contentType
124
+ }
125
+ });
126
+ if (expectedID !== undefined && blobID !== expectedID) {
127
+ throw new BlobIDMismatchError({
128
+ expected: expectedID,
129
+ actual: blobID
130
+ });
131
+ }
132
+ // Step 4: commit bytes, then record the entry, under the blob lock.
133
+ return await ctx.lock.withLock(blobID, async ()=>{
134
+ throwIfAborted(signal);
135
+ const store = await getBlobStore(ctx.db);
136
+ const existing = await getFreshEntry(store, blobID);
137
+ if (existing?.state === 'local') {
138
+ await discardStaging();
139
+ // Existing metadata wins; only a null content type is filled in.
140
+ const fill = async (target)=>{
141
+ if (existing.contentType !== null || contentType === undefined) return existing;
142
+ await target.fillContentType(blobID, contentType);
143
+ return await getFreshEntry(target, blobID) ?? existing;
144
+ };
145
+ const { entry, result } = await record(ctx, fn, fill);
146
+ return {
147
+ entry,
148
+ created: false,
149
+ result
150
+ };
151
+ }
152
+ const transfer = existing === null ? null : await store.getTransfer(blobID);
153
+ // Step 5: bytes are committed before the row. Re-check: the awaits above may outlast an abort.
154
+ throwIfAborted(signal);
155
+ await ctx.backend.commit(stagingID, blobID);
156
+ staged = false;
157
+ const input = {
158
+ blobID,
159
+ contentLength: hashed.contentLength,
160
+ encrypted: options.encrypted ?? false,
161
+ keyID: options.keyID ?? null,
162
+ chunkSize: ctx.chunkSize,
163
+ createdAt: Date.now(),
164
+ contentType: contentType ?? null
165
+ };
166
+ const chunks = hashed.chunks.map((digest, index)=>({
167
+ index,
168
+ digest
169
+ }));
170
+ const promote = async (target)=>{
171
+ await target.promoteEntry(input, chunks);
172
+ const entry = await getFreshEntry(target, blobID);
173
+ if (entry === null) throw new Error(`Blob entry ${blobID} missing after promotion`);
174
+ return entry;
175
+ };
176
+ const { entry, result } = await record(ctx, fn, promote);
177
+ // The promoted entry no longer references the transfer's staging area.
178
+ if (transfer !== null) {
179
+ try {
180
+ await ctx.backend.abortStaging(transfer.stagingID);
181
+ } catch {
182
+ // Reclaimed by staging pruning.
183
+ }
184
+ }
185
+ return {
186
+ entry,
187
+ created: true,
188
+ result
189
+ };
190
+ });
191
+ } finally{
192
+ await discardStaging();
193
+ ctx.activeWrites.delete(stagingID);
194
+ }
195
+ }
package/package.json CHANGED
@@ -1,6 +1,47 @@
1
1
  {
2
2
  "name": "@hozon/blob",
3
- "version": "0.0.0-stage",
4
- "stub": true,
5
- "description": "Temporary package placeholder for staged publishing"
3
+ "version": "0.1.1",
4
+ "description": "Content-addressed blob service with verified writes for Hozon",
5
+ "repository": {
6
+ "type": "git",
7
+ "url": "git+https://github.com/TairuFramework/hozon.git",
8
+ "directory": "packages/blob"
9
+ },
10
+ "license": "MIT",
11
+ "sideEffects": false,
12
+ "type": "module",
13
+ "exports": {
14
+ ".": "./lib/index.js"
15
+ },
16
+ "files": [
17
+ "lib/*"
18
+ ],
19
+ "dependencies": {
20
+ "@hozon/blob-backend": "^0.2.0",
21
+ "@hozon/blob-id": "^0.1.1",
22
+ "@hozon/db": "^0.3.0",
23
+ "@hozon/store-blob": "^0.2.2",
24
+ "@sozai/runtime": "^0.1.0"
25
+ },
26
+ "devDependencies": {
27
+ "@hozon/blob-node-fs": "^0.2.0",
28
+ "@hozon/node-sqlite": "^0.1.0",
29
+ "@sozai/codec": "^0.4.3",
30
+ "@types/node": "^26.6.5",
31
+ "del-cli": "^8.0.1",
32
+ "vitest": "5.0.5"
33
+ },
34
+ "publishConfig": {
35
+ "access": "public"
36
+ },
37
+ "scripts": {
38
+ "build": "pnpm run build:clean && pnpm run build:js && pnpm run build:types",
39
+ "build:clean": "del-cli lib",
40
+ "build:js": "del-cli \"lib/**/*.js\" && swc src -d ./lib --config-file ../../node_modules/@kigu/dev/swc.json --strip-leading-paths",
41
+ "build:types": "del-cli \"lib/**/*.d.ts\" \"lib/**/*.d.ts.map\" && tsc --emitDeclarationOnly --skipLibCheck",
42
+ "build:types:ci": "tsc --emitDeclarationOnly --skipLibCheck --declarationMap false",
43
+ "test": "pnpm run test:types && pnpm run test:unit",
44
+ "test:types": "tsc --noEmit --skipLibCheck -p tsconfig.test.json",
45
+ "test:unit": "vitest run"
46
+ }
6
47
  }